Skip to content

Commit

Permalink
Merge branch 'dev' into default
Browse files Browse the repository at this point in the history
  • Loading branch information
hallvictoria authored Dec 20, 2024
2 parents 2921117 + ed26bd0 commit d6d3dff
Show file tree
Hide file tree
Showing 176 changed files with 1,650 additions and 1,111 deletions.
2 changes: 1 addition & 1 deletion CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@
# For all file changes, github would automatically
# include the following people in the PRs.

* @vrdmr @gavin-aguiar @YunchuWang @pdthummar @hallvictoria
* @vrdmr @gavin-aguiar @hallvictoria
33 changes: 16 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,35 +1,34 @@
# <img src="https://raw.githubusercontent.com/Azure/azure-functions-python-worker/dev/docs/Azure.Functions.svg" width = "30" alt="Functions Header Image - Lightning Logo"> Azure Functions Python Worker

| Branch | Status | CodeCov | Unittests | E2E tests |
|--------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------|
| main | [![Build Status](https://azfunc.visualstudio.com/Azure%20Functions/_apis/build/status/Azure.azure-functions-python-worker?branchName=main)](https://azfunc.visualstudio.com/Azure%20Functions/_build/latest?definitionId=57&branchName=main) | [![codecov](https://codecov.io/gh/Azure/azure-functions-python-worker/branch/main/graph/badge.svg)](https://codecov.io/gh/Azure/azure-functions-python-worker) | ![CI Unit tests](https://github.com/Azure/azure-functions-python-worker/workflows/CI%20Unit%20tests/badge.svg?branch=main) | ![CI E2E tests](https://github.com/Azure/azure-functions-python-worker/workflows/CI%20E2E%20tests/badge.svg?branch=main) |
| dev | [![Build Status](https://azfunc.visualstudio.com/Azure%20Functions/_apis/build/status/Azure.azure-functions-python-worker?branchName=dev)](https://azfunc.visualstudio.com/Azure%20Functions/_build/latest?definitionId=57&branchName=dev) | [![codecov](https://codecov.io/gh/Azure/azure-functions-python-worker/branch/dev/graph/badge.svg)](https://codecov.io/gh/Azure/azure-functions-python-worker) | ![CI Unit tests](https://github.com/Azure/azure-functions-python-worker/workflows/CI%20Unit%20tests/badge.svg?branch=dev) | ![CI E2E tests](https://github.com/Azure/azure-functions-python-worker/workflows/CI%20E2E%20tests/badge.svg?branch=dev) |
| Branch | Build Status | CodeCov | Test Status |
|--------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| dev | [![Build Status](https://img.shields.io/azure-devops/build/azfunc/public/658/dev)](https://azfunc.visualstudio.com/public/_build/latest?definitionId=658&branchName=dev) | [![codecov](https://codecov.io/gh/Azure/azure-functions-python-worker/branch/dev/graph/badge.svg)](https://codecov.io/gh/Azure/azure-functions-python-worker) | [![Test Status](https://img.shields.io/azure-devops/build/azfunc/public/658/dev)](https://azfunc.visualstudio.com/public/_build/latest?definitionId=658&branchName=dev) |

Python support for Azure Functions is based on Python 3.6, 3.7, 3.8, 3.9, and 3.10 serverless hosting on Linux and the Functions 2.0, 3.0 and 4.0 runtime.
Python support for Azure Functions is based on Python 3.8, 3.9, 3.10, 3.11, and 3.12 serverless hosting on Linux and the Functions 4.0 runtime.

Here is the current status of Python in Azure Functions:

What are the supported Python versions?

| Azure Functions Runtime | Python 3.6 | Python 3.7 | Python 3.8 | Python 3.9 | Python 3.10 | Python 3.11 |
|----------------------------------|------------|------------|------------|------------|-------------|-------------|
| Azure Functions 2.0 (deprecated) ||| - | - | - | - |
| Azure Functions 3.0 (deprecated) ||||| - | - |
| Azure Functions 4.0 | - | - |||||
| Azure Functions Runtime | Python 3.8 | Python 3.9 | Python 3.10 | Python 3.11 | Python 3.12 |
|----------------------------------|------------|------------|-------------|-------------|-------------|
| Azure Functions 3.0 (deprecated) ||| - | - | - |
| Azure Functions 4.0 ||||||

For information about Azure Functions Runtime, please refer to [Azure Functions runtime versions overview](https://docs.microsoft.com/en-us/azure/azure-functions/functions-versions) page.

### What's available?

- Build, test, debug and publish using Azure Functions Core Tools (CLI) or Visual Studio Code
- Deploy Python Function project onto consumption, dedicated, or elastic premium plan.
- Deploy Python Function project in a custom docker image onto dedicated, or elastic premium plan.
- Triggers / Bindings : HTTP, Blob, Queue, Timer, Cosmos DB, Event Grid, Event Hubs and Service Bus
- Build, test, debug, and publish using Azure Functions Core Tools (CLI) or Visual Studio Code
- Deploy Python Function project onto consumption, dedicated, elastic premium, or flex consumption plan.
- Deploy Python Function project in a custom docker image onto dedicated or elastic premium plan.
- Triggers / Bindings : Blob, Cosmos DB, Event Grid, Event Hub, HTTP, Kafka, MySQL, Queue, ServiceBus, SQL, Timer, and Warmup
- Triggers / Bindings : Custom binding support

What's coming?
### What's new?

- [Durable Functions For Python](https://github.com/Azure/azure-functions-durable-python)
- [SDK Type Bindings for Blob](https://techcommunity.microsoft.com/t5/azure-compute-blog/azure-functions-sdk-type-bindings-for-azure-blob-storage-with/ba-p/4146744)
- [HTTP Streaming](https://techcommunity.microsoft.com/t5/azure-compute-blog/azure-functions-support-for-http-streams-in-python-is-now-in/ba-p/4146697)

### Get Started

Expand Down Expand Up @@ -72,4 +71,4 @@ provided by the bot. You will only need to do this once across all repos using o

This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or
contact [[email protected]](mailto:[email protected]) with any additional questions or comments.
contact [[email protected]](mailto:[email protected]) with any additional questions or comments.
2 changes: 1 addition & 1 deletion azure_functions_worker/version.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License.

VERSION = '4.33.0'
VERSION = '4.34.0'
46 changes: 46 additions & 0 deletions eng/ci/emulator-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
trigger: none # ensure this is not ran as a CI build

pr:
branches:
include:
- dev
- release/*

schedules:
- cron: "0 8 * * 1,2,3,4,5"
displayName: Monday to Friday 3 AM CST build
branches:
include:
- dev
always: true

resources:
repositories:
- repository: 1es
type: git
name: 1ESPipelineTemplates/1ESPipelineTemplates
ref: refs/tags/release
- repository: eng
type: git
name: engineering
ref: refs/tags/release

variables:
- template: /ci/variables/build.yml@eng
- template: /ci/variables/cfs.yml@eng
- template: /eng/templates/utils/variables.yml@self

extends:
template: v1/1ES.Unofficial.PipelineTemplate.yml@1es
parameters:
pool:
name: 1es-pool-azfunc
image: 1es-windows-2022
os: windows

stages:
- stage: RunEmulatorTests
jobs:
- template: /eng/templates/jobs/ci-emulator-tests.yml@self
parameters:
PoolName: 1es-pool-azfunc
8 changes: 8 additions & 0 deletions eng/ci/official-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ resources:
variables:
- template: /eng/templates/utils/variables.yml@self
- template: /eng/templates/utils/official-variables.yml@self
- name: codeql.excludePathPatterns
value: deps/,build/

extends:
template: v1/1ES.Official.PipelineTemplate.yml@1es
Expand All @@ -52,6 +54,12 @@ extends:
dependsOn: Build
jobs:
- template: /eng/templates/official/jobs/ci-e2e-tests.yml@self
- stage: RunEmulatorTests
dependsOn: Build
jobs:
- template: /eng/templates/jobs/ci-emulator-tests.yml@self
parameters:
PoolName: 1es-pool-azfunc
- stage: RunUnitTests
dependsOn: Build
jobs:
Expand Down
21 changes: 15 additions & 6 deletions eng/ci/public-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,14 @@ resources:

variables:
- template: /eng/templates/utils/variables.yml@self
- name: codeql.excludePathPatterns
value: deps/,build/
- name: codeql.compiled.enabled
value: true
- name: codeql.runSourceLanguagesInSourceAnalysis
value: true
- name: codeql.sourceLanguages
value: python, powershell

extends:
template: v1/1ES.Unofficial.PipelineTemplate.yml@1es
Expand All @@ -36,11 +44,6 @@ extends:
name: 1es-pool-azfunc-public
image: 1es-windows-2022
os: windows
sdl:
codeql:
compiled:
enabled: true # still only runs for default branch
runSourceLanguagesInSourceAnalysis: true
settings:
skipBuildTagsForGitHubPullRequests: ${{ variables['System.PullRequest.IsFork'] }}
stages:
Expand All @@ -50,4 +53,10 @@ extends:
- stage: RunUnitTests
dependsOn: Build
jobs:
- template: /eng/templates/jobs/ci-unit-tests.yml@self
- template: /eng/templates/jobs/ci-unit-tests.yml@self
- stage: RunEmulatorTests
dependsOn: Build
jobs:
- template: /eng/templates/jobs/ci-emulator-tests.yml@self
parameters:
PoolName: 1es-pool-azfunc-public
6 changes: 5 additions & 1 deletion eng/templates/jobs/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,8 @@ jobs:
python -m pip install .
displayName: 'Build python worker'
# Skip the build stage for SDK and Extensions release branches. This stage will fail because pyproject.toml contains the updated (and unreleased) library version
condition: and(eq(variables.isSdkRelease, false), eq(variables.isExtensionsRelease, false), eq(variables['USETESTPYTHONSDK'], false), eq(variables['USETESTPYTHONEXTENSIONS'], false))
condition: and(eq(variables.isSdkRelease, false), eq(variables.isExtensionsRelease, false), eq(variables['USETESTPYTHONSDK'], false), eq(variables['USETESTPYTHONEXTENSIONS'], false))
- bash: |
pip install pip-audit
pip-audit -r requirements.txt
displayName: 'Run vulnerability scan'
100 changes: 100 additions & 0 deletions eng/templates/jobs/ci-emulator-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
jobs:
- job: "TestPython"
displayName: "Run Python Emulator Tests"

pool:
name: ${{ parameters.PoolName }}
image: 1es-ubuntu-22.04
os: linux

strategy:
matrix:
Python37:
PYTHON_VERSION: '3.7'
Python38:
PYTHON_VERSION: '3.8'
Python39:
PYTHON_VERSION: '3.9'
Python310:
PYTHON_VERSION: '3.10'
Python311:
PYTHON_VERSION: '3.11'
Python312:
PYTHON_VERSION: '3.12'
steps:
- task: UsePythonVersion@0
inputs:
versionSpec: $(PYTHON_VERSION)
- task: UseDotNet@2
displayName: 'Install .NET 8'
inputs:
version: 8.0.x
- bash: |
chmod +x eng/scripts/install-dependencies.sh
chmod +x eng/scripts/test-setup.sh
eng/scripts/install-dependencies.sh $(PYTHON_VERSION)
eng/scripts/test-setup.sh
displayName: 'Install dependencies and the worker'
condition: and(eq(variables.isSdkRelease, false), eq(variables.isExtensionsRelease, false), eq(variables['USETESTPYTHONSDK'], false), eq(variables['USETESTPYTHONEXTENSIONS'], false))
- task: DownloadPipelineArtifact@2
displayName: 'Download Python SDK Artifact'
inputs:
buildType: specific
artifactName: 'azure-functions'
project: 'internal'
definition: 679
buildVersionToDownload: latest
targetPath: '$(Pipeline.Workspace)/PythonSdkArtifact'
condition: or(eq(variables.isSdkRelease, true), eq(variables['USETESTPYTHONSDK'], true))
- bash: |
chmod +x eng/scripts/test-sdk.sh
chmod +x eng/scripts/test-setup.sh
eng/scripts/test-sdk.sh $(Pipeline.Workspace) $(PYTHON_VERSION)
eng/scripts/test-setup.sh
displayName: 'Install test python sdk, dependencies and the worker'
condition: or(eq(variables.isSdkRelease, true), eq(variables['USETESTPYTHONSDK'], true))
- task: DownloadPipelineArtifact@2
displayName: 'Download Python Extension Artifact'
inputs:
buildType: specific
artifactName: $(PYTHONEXTENSIONNAME)
project: 'internal'
definition: 798
buildVersionToDownload: latest
targetPath: '$(Pipeline.Workspace)/PythonExtensionArtifact'
condition: or(eq(variables.isExtensionsRelease, true), eq(variables['USETESTPYTHONEXTENSIONS'], true))
- bash: |
chmod +x eng/scripts/test-setup.sh
chmod +x eng/scripts/test-extensions.sh
eng/scripts/test-extensions.sh $(Pipeline.Workspace) $(PYTHON_VERSION)
eng/scripts/test-setup.sh
displayName: 'Install test python extension, dependencies and the worker'
condition: or(eq(variables.isExtensionsRelease, true), eq(variables['USETESTPYTHONEXTENSIONS'], true))
- bash: |
docker compose -f tests/emulator_tests/utils/eventhub/docker-compose.yml pull
docker compose -f tests/emulator_tests/utils/eventhub/docker-compose.yml up -d
displayName: 'Install Azurite and Start EventHub Emulator'
- bash: |
python -m pytest -q -n auto --dist loadfile --reruns 4 --ignore=tests/emulator_tests/test_servicebus_functions.py tests/emulator_tests
env:
AzureWebJobsStorage: "UseDevelopmentStorage=true"
AzureWebJobsEventHubConnectionString: "Endpoint=sb://localhost;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=SAS_KEY_VALUE;UseDevelopmentEmulator=true;"
displayName: "Running $(PYTHON_VERSION) Python Linux Emulator Tests"
- bash: |
# Stop and remove EventHub Emulator container to free up the port
docker stop eventhubs-emulator
docker container rm --force eventhubs-emulator
docker compose -f tests/emulator_tests/utils/servicebus/docker-compose.yml pull
docker compose -f tests/emulator_tests/utils/servicebus/docker-compose.yml up -d
env:
AzureWebJobsSQLPassword: $(AzureWebJobsSQLPassword)
displayName: 'Install Azurite and Start ServiceBus Emulator'
- bash: |
python -m pytest -q -n auto --dist loadfile --reruns 4 tests/emulator_tests/test_servicebus_functions.py
env:
AzureWebJobsStorage: "UseDevelopmentStorage=true"
AzureWebJobsServiceBusConnectionString: "Endpoint=sb://localhost;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=SAS_KEY_VALUE;UseDevelopmentEmulator=true;"
displayName: "Running $(PYTHON_VERSION) Python ServiceBus Linux Emulator Tests"
21 changes: 20 additions & 1 deletion eng/templates/official/jobs/ci-e2e-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,25 @@ jobs:
eng/scripts/test-setup.sh
displayName: 'Install test python extension, dependencies and the worker'
condition: or(eq(variables.isExtensionsRelease, true), eq(variables['USETESTPYTHONEXTENSIONS'], true))
- powershell: |
$pipelineVarSet = "$(USETESTPYTHONSDK)"
Write-Host "pipelineVarSet: $pipelineVarSet"
$branch = "$(Build.SourceBranch)"
Write-Host "Branch: $branch"
if($branch.StartsWith("refs/heads/sdk/") -or $pipelineVarSet -eq "true")
{
Write-Host "##vso[task.setvariable variable=skipTest;]true"
}
else
{
Write-Host "##vso[task.setvariable variable=skipTest;]false"
}
displayName: 'Set skipTest variable'
condition: or(eq(variables.isSdkRelease, true), eq(variables['USETESTPYTHONSDK'], true))
- powershell: |
Write-Host "skipTest: $(skipTest)"
displayName: 'Display skipTest variable'
condition: or(eq(variables.isSdkRelease, true), eq(variables['USETESTPYTHONSDK'], true))
- bash: |
python -m pytest -q -n auto --dist loadfile --reruns 4 --cov=./azure_functions_worker --cov-report xml --cov-branch --cov-append tests/endtoend tests/extension_tests/deferred_bindings_tests tests/extension_tests/http_v2_tests
env:
Expand All @@ -125,5 +144,5 @@ jobs:
AzureWebJobsSqlConnectionString: $(SQL_CONNECTION)
AzureWebJobsEventGridTopicUri: $(EVENTGRID_URI)
AzureWebJobsEventGridConnectionKey: $(EVENTGRID_CONNECTION)
USETESTPYTHONSDK: or(eq(variables.isSdkRelease, true), eq(variables['USETESTPYTHONSDK'], true))
skipTest: $(skipTest)
displayName: "Running $(PYTHON_VERSION) Python E2E Tests"
Loading

0 comments on commit d6d3dff

Please sign in to comment.