Skip to content

Python Packages Loading Order and Sys Path

Jump to bottom
gavin-aguiar edited this page Aug 23, 2023 · 4 revisions

Introduction

This article is demonstrating which package will be loaded when it is installed on both worker dependencies and customer dependencies. (e.g. azure.functions, google.protobuf, grpc).

To fully isolate worker dependencies and ensure customer's code will use packages defined in their requirements.txt, please consider adding PYTHON_ISOLATE_WORKER_DEPENDENCIES = true to your app settings.

The following sections are the scenarios when PYTHON_ISOLATE_WORKER_DEPENDENCIES is not set or it is set to False.

Dependency Paths

Python worker mainly works in three different SKU: Linux Consumption, Linux Dedicated, Custom Container (Docker), and Core Tools (Local Dev). A Python package can either be loaded from worker's dependencies or customer's dependencies:

  1. Worker Dependencies Location
    • In production: /azure-functions-host/workers/python/3.7/LINUX/X64
    • In core tools: %appdata%/azure-functions-core-tools/bin/workers/python/3.6/WINDOWS/X64
  2. Customer Dependencies Location
    • In production: /home/site/wwwroot/.python_packages/lib/site-packages
    • In core tools: $VIRTUAL_ENV/Lib/site-packages

Linux Consumption

When there's a package installed on both locations, customer's code will use worker's dependencies, which contradicts the sys.path. This is due to the code used for reloading azure. and google. namespace is improperly implemented in __handle_environment_reload_request(). It is a known issue.

sys.path: [
    "/tmp/functions\\standby\\wwwroot", # Placeholder folder
    "/home/site/wwwroot/.python_packages/lib/site-packages", # Customer's dependency path
    "/azure-functions-host/workers/python/3.7/LINUX/X64", # Worker's dependency path
    "/home/site/wwwroot" # Customer's Working Directory
]

Linux Dedicated/Premium

The Linux Dedidcated/Premium plan does follow the sys.path implementation. Thus, our worker's code will use customer's dependencies.

sys.path: [
    "/home/site/wwwroot", # Customers's Working Directory
    "/home/site/wwwroot/.python_packages/lib/site-packages", # Customer's dependency path
    "/azure-functions-host/workers/python/3.7/LINUX/X64", # Worker's dependency path
]

Custom Containers

In custom containers, the customer's code will use worker's dependencies.

sys.path: [
    "/azure-functions-host/workers/python/3.7/LINUX/X64",  # Worker's dependency path
    "/usr/local/lib/python3.7/site-packages",  # Customer's dependency path
    "/home/site/wwwroot"  # Customer's Working Directory
]

Core Tools

In core tools, the customer's code will use worker's dependencies.

sys.path: [
    "%appdata%\\azure-functions-core-tools\\bin\\workers\\python\\3.7\\WINDOWS\\X64", # Worker's dependency path
     "C:\\Users\\user\\Project\\.venv37\\lib\\site-packages", # Customer's dependency path
     "C:\\Users\\user\\Project", # Customer's Working Directory
]

Summary

When two different versions of the same package is installed in both worker's dependency path and customer's dependency path, here is the table to demonstrate which package is loaded.

SKUs Isolation Enabled Worker Code Customer Code
Linux Consumption True Use Worker Deps Use Customer Deps
Linux Consumption False Use Worker Deps Use Worker Deps
Linux Dedicated/Premium True Use Worker Deps Use Customer Deps
Linux Dedicated/Premium False Use Customer Deps Use Customer Deps
Custom Container True Use Worker Deps Use Customer Deps
Custom Container False Use Worker Deps Use Worker Deps
Core Tools True Use Worker Deps Use Customer Deps
Core Tools False Use Worker Deps Use Worker Deps
Clone this wiki locally