Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Flex Function #64

Draft
wants to merge 33 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
69b952d
Move app insights to monitoing RG
marvinbuss Nov 1, 2024
c650029
Add app service plan for function
marvinbuss Nov 1, 2024
9ff6d50
Add UAI for function
marvinbuss Nov 1, 2024
d545c7a
Update rg name
marvinbuss Nov 1, 2024
377b8e4
test
marvinbuss Nov 1, 2024
434c615
Add subnet for flex function
marvinbuss Nov 1, 2024
8cfbad1
Add required rp registration
marvinbuss Nov 1, 2024
5ec1924
Add missing role assignment for uai
marvinbuss Nov 1, 2024
2637c80
Add prereq dns config
marvinbuss Nov 1, 2024
9f5a5ce
Add flex function
marvinbuss Nov 1, 2024
6940f45
Add missing name to function
marvinbuss Nov 4, 2024
cb1f6e5
Update address range for subnet
marvinbuss Nov 4, 2024
a901121
Update maximum instance count to valid value
marvinbuss Nov 4, 2024
20cd5a8
Update key vault reference id for function
marvinbuss Nov 4, 2024
a6f1e7e
Fix umi reference
marvinbuss Nov 4, 2024
f07cc0e
Update function config
marvinbuss Nov 4, 2024
50f658c
Add function baseline
marvinbuss Nov 4, 2024
7527eac
Add function GH workflow and update webapp workflow
marvinbuss Nov 4, 2024
07d5793
lint
marvinbuss Nov 4, 2024
d789c2c
Test function deploy
marvinbuss Nov 4, 2024
18eca38
Fix auth bug in function workflow
marvinbuss Nov 4, 2024
cb75b74
Fix config
marvinbuss Nov 4, 2024
02e317a
Test deployment with funct tools
marvinbuss Nov 4, 2024
d300941
Add language stack
marvinbuss Nov 4, 2024
399215e
Update function app config
marvinbuss Nov 4, 2024
721eef5
Fix resource group name
marvinbuss Nov 4, 2024
a08ebe6
Reference RG name
marvinbuss Nov 4, 2024
bc3cf58
Fix asp
marvinbuss Nov 4, 2024
d97974c
Update function flex
marvinbuss Nov 4, 2024
fade40d
Simplify function config
marvinbuss Nov 4, 2024
cc66665
lint
marvinbuss Nov 4, 2024
cc66994
Update function name
marvinbuss Nov 4, 2024
f8ca9dd
Update function dployment
marvinbuss Nov 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
97 changes: 97 additions & 0 deletions .github/workflows/_functionDeployTemplate.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
name: Function App Deploy Template

on:
workflow_call:
inputs:
environment:
required: true
type: string
default: "dev"
description: "Specifies the environment of the deployment."
python_version:
required: true
type: string
default: "3.11"
description: "Specifies the python version."
function_directory:
required: true
type: string
description: "Specifies the directory of the Azure Web App."
function_name:
required: true
type: string
description: "Specifies the name of the Azure Web App."
tenant_id:
required: true
type: string
description: "Specifies the tenant id of the deployment."
subscription_id:
required: true
type: string
description: "Specifies the subscription id of the deployment."
secrets:
CLIENT_ID:
required: true
description: "Specifies the client id."

permissions:
id-token: write
contents: read

jobs:
deployment:
name: Function App Deploy
runs-on: [ubuntu-latest]
continue-on-error: false
environment: ${{ inputs.environment }}
concurrency:
group: function-${{ inputs.function_name }}-${{ inputs.environment }}
cancel-in-progress: false

steps:
# Check Out Repository
- name: Check Out Repository
id: checkout_repository
uses: actions/checkout@v4

# Setup Python
- name: Setup Python
id: python_setup
uses: actions/setup-python@v5
with:
python-version: ${{ inputs.python_version }}
cache: "pip"
cache-dependency-path: |
${{ inputs.function_directory }}/requirements.txt

# Install Web App Dependencies
- name: Resolve Web App Dependencies
id: function_dependencies
shell: bash
run: |
pushd "${WEBAPP_DIRECTORY}"
python -m pip install --upgrade pip
pip install -r requirements.txt --target=".python_packages/lib/site-packages"
popd
env:
WEBAPP_DIRECTORY: ${{ inputs.function_directory }}

# Azure login
- name: Azure login
id: azure_login
uses: azure/login@v2
with:
client-id: ${{ secrets.CLIENT_ID }}
tenant-id: ${{ inputs.tenant_id }}
subscription-id: ${{ inputs.subscription_id }}

# Deploy Function
- name: Deploy Function
id: function_deploy
uses: Azure/functions-action@v1
with:
app-name: ${{ inputs.function_name }}
package: ${{ inputs.function_directory }}
sku: flexconsumption
remote-build: true
respect-funcignore: true
39 changes: 39 additions & 0 deletions .github/workflows/function.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: Function Deployment
on:
push:
branches:
- main
paths:
- "code/function/**"
- "config/**"
- "requirements.txt"

pull_request:
branches:
- main
paths:
- "code/function/**"
- "config/**"
- "requirements.txt"

jobs:
function_test:
uses: ./.github/workflows/_webappTestTemplate.yml
name: "Function Test"
with:
python_version: "3.11"
webapp_directory: "./code/function"

function_dev:
uses: ./.github/workflows/_functionDeployTemplate.yml
name: "Function - Dev"
needs: [function_test]
with:
environment: "dev"
python_version: "3.11"
function_directory: "./code/function"
function_name: "bfr-dev-fctn001"
tenant_id: "37963dd4-f4e6-40f8-a7d6-24b97919e452"
subscription_id: "1fdab118-1638-419a-8b12-06c9543714a0"
secrets:
CLIENT_ID: ${{ secrets.CLIENT_ID }}
4 changes: 0 additions & 4 deletions .github/workflows/webapp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,16 @@ on:
branches:
- main
paths:
- "**.py"
- "code/backend/**"
- "config/**"
- "tests/**"
- "requirements.txt"

pull_request:
branches:
- main
paths:
- "**.py"
- "code/backend/**"
- "config/**"
- "tests/**"
- "requirements.txt"

jobs:
Expand Down
10 changes: 10 additions & 0 deletions code/function/.funcignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
.git*
.vscode
__azurite_db*__.json
__blobstorage__
__queuestorage__
__pycache__
local.settings.json
tests
.venv
.vscode
Empty file.
29 changes: 29 additions & 0 deletions code/function/blobfiletrigger/function.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
import logging

import azure.functions as func
import azurefunctions.extensions.bindings.blob as blob
from shared.config import settings
from shared.utils import get_blob_properties

bp = func.Blueprint()


@bp.function_name("VideoUpload")
@bp.blob_trigger(
arg_name="client",
path="upload-newsvideos/{name}",
connection="BlobTrigger",
# source="LogsAndContainerScan",
)
async def upload_video(client: blob.BlobClient):
logging.info("File upload detected.")

# Initialize
logging.info("Initialize")
_ = await get_blob_properties(
storage_domain_name=f"{client.account_name}.blob.core.windows.net",
storage_container_name=client.container_name,
storage_blob_name=client.blob_name,
managed_identity_client_id=settings.MANAGED_IDENTITY_CLIENT_ID,
)
logging.info(f"Completed Function run successfully.")
10 changes: 10 additions & 0 deletions code/function/function_app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
import azure.functions as func

# from blobfiletrigger.function import bp as bp_blobfiletrigger
from health.function import bp as bp_health

app = func.FunctionApp(http_auth_level=func.AuthLevel.FUNCTION)

# Register blueprints
app.register_functions(bp_health)
# app.register_functions(bp_blobfiletrigger)
Empty file.
17 changes: 17 additions & 0 deletions code/function/health/function.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import logging

import azure.functions as func
from health.models import HeartbeatResponse

bp = func.Blueprint()


@bp.function_name("Health")
@bp.route(route="v1/heartbeat")
@bp.http_type(http_type=func.HttpMethod.GET)
async def heartbeat(req: func.HttpRequest) -> func.HttpResponse:
logging.debug("Received Health Request.")
response = HeartbeatResponse(is_alive=True).model_dump_json()
return func.HttpResponse(
body=response, status_code=200, mimetype="application/json"
)
5 changes: 5 additions & 0 deletions code/function/health/models.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
from pydantic import BaseModel


class HeartbeatResponse(BaseModel):
is_alive: bool = True
42 changes: 42 additions & 0 deletions code/function/host.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
{
"version": "2.0",
"functionTimeout": "00:30:00",
"logging": {
"fileLoggingMode": "debugOnly",
"logLevel": {
"default": "Information",
"Worker": "Information",
"Host": "Information",
"Host.Aggregator": "Information",
"Host.Results": "Information",
"Function": "Information"
},
"applicationInsights": {
"samplingSettings": {
"isEnabled": true,
"excludedTypes": "Request;Exception"
},
"enableLiveMetrics": true,
"enableDependencyTracking": true,
"enablePerformanceCountersCollection": true,
"httpAutoCollectionOptions": {
"enableHttpTriggerExtendedInfoCollection": true,
"enableW3CDistributedTracing": true,
"enableResponseHeaderInjection": true
}
}
},
"extensionBundle": {
"id": "Microsoft.Azure.Functions.ExtensionBundle",
"version": "[4.*, 5.0.0)"
},
"extensions": {
"http": {
"routePrefix": "api"
},
"blobs": {
"maxDegreeOfParallelism": 4,
"poisonBlobThreshold": 1
}
}
}
9 changes: 9 additions & 0 deletions code/function/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
azure-functions~=1.21.3
azurefunctions-extensions-bindings-blob==1.0.0b2
azure-identity~=1.19.0
aiohttp~=3.10.5
httpx~=0.27.2
pydantic-settings~=2.6.1
pydantic~=2.9.2
langchain~=0.3.7
langchain-openai~=0.2.5
Empty file.
27 changes: 27 additions & 0 deletions code/function/shared/config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
import logging

from pydantic import Field
from pydantic_settings import BaseSettings


class Settings(BaseSettings):
# General config
PROJECT_NAME: str = "VideoAnalyzer"
SERVER_NAME: str = "VideoAnalyzer"
APP_VERSION: str = "v0.0.1"
WEBSITE_NAME: str = Field(default="test", alias="WEBSITE_SITE_NAME")
WEBSITE_INSTANCE_ID: str = Field(default="0", alias="WEBSITE_INSTANCE_ID")
HOME_DIRECTORY: str = Field(default="", alias="HOME")

# Identity settings
MANAGED_IDENTITY_CLIENT_ID: str

# Logging settings
LOGGING_LEVEL: int = logging.INFO
DEBUG: bool = False
APPLICATIONINSIGHTS_CONNECTION_STRING: str = Field(
default="", alias="APPLICATIONINSIGHTS_CONNECTION_STRING"
)


settings = Settings()
9 changes: 9 additions & 0 deletions code/function/shared/models.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
from typing import Any

from pydantic import BaseModel


class ErrorModel(BaseModel):
error_code: int
error_message: str
error_details: Any
58 changes: 58 additions & 0 deletions code/function/shared/utils.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
import logging

from azure.identity.aio import DefaultAzureCredential
from azure.storage.blob import BlobProperties
from azure.storage.blob.aio import BlobServiceClient


def get_azure_credential(
managed_identity_client_id: str = None,
) -> DefaultAzureCredential:
"""Creates a default azure crendetial used for authentication.

managed_identity_client_id (str): Specifies the client id of a managed identity.
RETURNS (str): Returns a default azure credential.
"""
if managed_identity_client_id is None:
return DefaultAzureCredential()
else:
return DefaultAzureCredential(
managed_identity_client_id=managed_identity_client_id,
)


async def get_blob_properties(
storage_domain_name: str,
storage_container_name: str,
storage_blob_name: str,
managed_identity_client_id: str = None,
) -> BlobProperties:
"""Copy file from source blob storage container async to sink blob storage container.

storage_domain_name (str): The domain name of the storage account to which the file will be copied.
storage_container_name (str): The container name of the storage account.
storage_blob_name (str): The blob name of the storage account.
managed_identity_client_id (str): Specifies the managed identity client id used for auth.
RETURNS (BlobProperties): Returns the properties of the blob.
"""
logging.info(
f"Get properties for blob: 'https://{storage_domain_name}/{storage_container_name}/{storage_blob_name}'."
)

# Create credentials
credential = get_azure_credential(
managed_identity_client_id=managed_identity_client_id
)

# Copy blob file
async with BlobServiceClient(
account_url=f"https://{storage_domain_name}",
credential=credential,
) as blob_service_client:
blob_client = blob_service_client.get_blob_client(
container=storage_container_name,
blob=storage_blob_name,
)
blob_properties = await blob_client.get_blob_properties()

return blob_properties
Empty file added code/function/tests/__init__.py
Empty file.
13 changes: 13 additions & 0 deletions code/function/tests/test_sample.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
import pytest


def test_get_guid():
# init
a = 1
b = 2

# act
a += 1

# validate
assert a == b
Loading
Loading