From 9f17a9a6dbf3d45a07dd02a8d811ac1628570a52 Mon Sep 17 00:00:00 2001 From: Yun Kim <35776586+Yun-Kim@users.noreply.github.com> Date: Fri, 24 Jan 2025 15:22:50 -0500 Subject: [PATCH] fix(openai): support Python 3.12, 3.13 (#12014) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Resolves #11994. We were previously not testing our OpenAI integration with Python versions >= 3.12 (at the time, OpenAI did not support those newer Python versions). We recently saw that our bytecode wrapping for OpenAI methods broke in Python 3.13, so this PR attempts to address that: - Switch out our bytecode wrapping with `wrapt.wrap_function_wrapper()` to make our patching compatible with newer Python versions. - Add Python 3.12 and 3.13 to tested versions for the OpenAI integration. - Implements unpatching and adds patching tests We previously were using direct bytecode wrapping in the OpenAI integration, which was not supported for Python 3.13, and instead switched to using `wrapt.wrap_function_wrapper()` to wrap OpenAI methods. This meant that we needed to change wrapping formats, including: - use function/attribute names (string) rather than references as inputs to the wrapping functions - replace nested functions with `@with_traced_module()` to pass the traced OpenAI module reference between traced methods - implement unpatching - add patching tests (note this is messy because we support both v0 and v1 openai versions which have separate method names) **Note**: the issue in #11994 was only reproducible via the `AsyncOpenAI.Moderations.create(), AsyncOpenAI.Embeddings.create()` endpoints (chat and completions were not affected). However to avoid any risk introduced by the now unsupported bytecode wrapping, we are changing the entire OpenAI integration to use `wrapt.wrap_function_wrapper()` instead. Since ddtrace 3.0 will be removing support for Python 3.7, we are also dropping support accordingly in this PR. This also coincides with removing support for OpenAI v0 since v0 was last released more than a 16 months ago, and requires a large maintenance burden and pollutes the codebase. Note that this PR will just drop testing support, but a future PR will remove tracing support for v0 in the form of a refactor/cleanup of the OpenAI integration. Azure OpenAI requests are changed in latest versions of OpenAI, which fail our cassette tests (recorded in previous versions of OpenAI). However on manual testing, our openai integration is unchanged in support for newer versions. With consideration for time/effort to produce new cassette test files which is time consuming, we are going to skip cassette tests for Azure endpoints with latest openai versions until we improve our testing framework entirely (move off cassette files). Here are the manual traces submitted with Azure OpenAI for reference: Screenshot 2025-01-23 at 2 20 00 PM Screenshot 2025-01-23 at 2 20 14 PM - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) --- .riot/requirements/107d8f2.txt | 54 ++++ .riot/requirements/130158f.txt | 48 +++ .riot/requirements/13804af.txt | 57 ---- .riot/requirements/13fec34.txt | 49 ---- .riot/requirements/1825740.txt | 12 +- .riot/requirements/18de44f.txt | 52 ++++ .riot/requirements/1ad89c5.txt | 50 ++++ .../requirements/{1db5311.txt => 1e6bd37.txt} | 47 +-- .riot/requirements/1ec15f5.txt | 57 ---- .riot/requirements/1ee49b9.txt | 53 ---- .riot/requirements/2634bf7.txt | 48 +++ .riot/requirements/35ce786.txt | 55 ---- .riot/requirements/4a85f6d.txt | 50 ++++ .riot/requirements/4d27459.txt | 48 +++ .riot/requirements/530c983.txt | 52 ++++ .riot/requirements/5da4fd8.txt | 49 ---- .riot/requirements/84ec59a.txt | 53 ---- .riot/requirements/87a1fff.txt | 53 ---- .riot/requirements/b5d5a35.txt | 52 ++++ .riot/requirements/c74f6e0.txt | 51 ---- .riot/requirements/cd2e4ea.txt | 53 ---- .../requirements/{181216c.txt => df60af6.txt} | 16 +- .riot/requirements/f1c37b1.txt | 51 ---- .riot/requirements/f7c30a0.txt | 51 ---- .../internal/openai/_endpoint_hooks.py | 110 +++---- ddtrace/contrib/internal/openai/patch.py | 267 +++++++++-------- .../fix-openai-313-29ec43cbf2f35947.yaml | 7 + riotfile.py | 16 +- .../v1/completion_stream_wrong_api_key.yaml | 77 +++++ tests/contrib/openai/test_openai_llmobs.py | 16 + tests/contrib/openai/test_openai_patch.py | 276 +++++++++++++++++- tests/contrib/openai/test_openai_v0.py | 38 --- tests/contrib/openai/test_openai_v1.py | 91 ++---- 33 files changed, 1094 insertions(+), 965 deletions(-) create mode 100644 .riot/requirements/107d8f2.txt create mode 100644 .riot/requirements/130158f.txt delete mode 100644 .riot/requirements/13804af.txt delete mode 100644 .riot/requirements/13fec34.txt create mode 100644 .riot/requirements/18de44f.txt create mode 100644 .riot/requirements/1ad89c5.txt rename .riot/requirements/{1db5311.txt => 1e6bd37.txt} (55%) delete mode 100644 .riot/requirements/1ec15f5.txt delete mode 100644 .riot/requirements/1ee49b9.txt create mode 100644 .riot/requirements/2634bf7.txt delete mode 100644 .riot/requirements/35ce786.txt create mode 100644 .riot/requirements/4a85f6d.txt create mode 100644 .riot/requirements/4d27459.txt create mode 100644 .riot/requirements/530c983.txt delete mode 100644 .riot/requirements/5da4fd8.txt delete mode 100644 .riot/requirements/84ec59a.txt delete mode 100644 .riot/requirements/87a1fff.txt create mode 100644 .riot/requirements/b5d5a35.txt delete mode 100644 .riot/requirements/c74f6e0.txt delete mode 100644 .riot/requirements/cd2e4ea.txt rename .riot/requirements/{181216c.txt => df60af6.txt} (82%) delete mode 100644 .riot/requirements/f1c37b1.txt delete mode 100644 .riot/requirements/f7c30a0.txt create mode 100644 releasenotes/notes/fix-openai-313-29ec43cbf2f35947.yaml create mode 100644 tests/contrib/openai/cassettes/v1/completion_stream_wrong_api_key.yaml diff --git a/.riot/requirements/107d8f2.txt b/.riot/requirements/107d8f2.txt new file mode 100644 index 00000000000..7bed129ddaf --- /dev/null +++ b/.riot/requirements/107d8f2.txt @@ -0,0 +1,54 @@ +# +# This file is autogenerated by pip-compile with Python 3.9 +# by the following command: +# +# pip-compile --no-annotate --resolver=backtracking .riot/requirements/107d8f2.in +# +annotated-types==0.7.0 +anyio==4.8.0 +attrs==24.3.0 +certifi==2024.12.14 +coverage[toml]==7.6.10 +distro==1.9.0 +exceptiongroup==1.2.2 +h11==0.14.0 +httpcore==1.0.7 +httpx==0.27.2 +hypothesis==6.45.0 +idna==3.10 +importlib-metadata==8.6.1 +iniconfig==2.0.0 +mock==5.1.0 +multidict==6.1.0 +numpy==2.0.2 +openai[datalib,embeddings]==1.30.1 +opentracing==2.4.0 +packaging==24.2 +pandas==2.2.3 +pandas-stubs==2.2.2.240807 +pillow==9.5.0 +pluggy==1.5.0 +propcache==0.2.1 +pydantic==2.10.5 +pydantic-core==2.27.2 +pytest==8.3.4 +pytest-asyncio==0.21.1 +pytest-cov==6.0.0 +pytest-mock==3.14.0 +pytest-randomly==3.16.0 +python-dateutil==2.9.0.post0 +pytz==2024.2 +pyyaml==6.0.2 +six==1.17.0 +sniffio==1.3.1 +sortedcontainers==2.4.0 +tomli==2.2.1 +tqdm==4.67.1 +types-pytz==2024.2.0.20241221 +typing-extensions==4.12.2 +tzdata==2025.1 +urllib3==1.26.20 +vcrpy==4.2.1 +wrapt==1.17.2 +yarl==1.18.3 +zipp==3.21.0 diff --git a/.riot/requirements/130158f.txt b/.riot/requirements/130158f.txt new file mode 100644 index 00000000000..037c7010f33 --- /dev/null +++ b/.riot/requirements/130158f.txt @@ -0,0 +1,48 @@ +# +# This file is autogenerated by pip-compile with Python 3.12 +# by the following command: +# +# pip-compile --no-annotate .riot/requirements/130158f.in +# +annotated-types==0.7.0 +anyio==4.8.0 +attrs==24.3.0 +certifi==2024.12.14 +charset-normalizer==3.4.1 +coverage[toml]==7.6.10 +distro==1.9.0 +h11==0.14.0 +httpcore==1.0.7 +httpx==0.28.1 +hypothesis==6.45.0 +idna==3.10 +iniconfig==2.0.0 +jiter==0.8.2 +mock==5.1.0 +multidict==6.1.0 +openai==1.60.0 +opentracing==2.4.0 +packaging==24.2 +pillow==11.1.0 +pluggy==1.5.0 +propcache==0.2.1 +pydantic==2.10.5 +pydantic-core==2.27.2 +pytest==8.3.4 +pytest-asyncio==0.21.1 +pytest-cov==6.0.0 +pytest-mock==3.14.0 +pytest-randomly==3.16.0 +pyyaml==6.0.2 +regex==2024.11.6 +requests==2.32.3 +six==1.17.0 +sniffio==1.3.1 +sortedcontainers==2.4.0 +tiktoken==0.8.0 +tqdm==4.67.1 +typing-extensions==4.12.2 +urllib3==1.26.20 +vcrpy==4.2.1 +wrapt==1.17.2 +yarl==1.18.3 diff --git a/.riot/requirements/13804af.txt b/.riot/requirements/13804af.txt deleted file mode 100644 index 7035a764386..00000000000 --- a/.riot/requirements/13804af.txt +++ /dev/null @@ -1,57 +0,0 @@ -# -# This file is autogenerated by pip-compile with Python 3.8 -# by the following command: -# -# pip-compile --allow-unsafe --no-annotate .riot/requirements/13804af.in -# -annotated-types==0.7.0 -anyio==4.4.0 -attrs==24.2.0 -certifi==2024.7.4 -charset-normalizer==3.3.2 -coverage[toml]==7.6.1 -distro==1.9.0 -exceptiongroup==1.2.2 -h11==0.14.0 -httpcore==1.0.5 -httpx==0.27.0 -hypothesis==6.45.0 -idna==3.8 -importlib-metadata==8.4.0 -iniconfig==2.0.0 -mock==5.1.0 -multidict==6.0.5 -numpy==1.24.4 -openai[datalib]==1.30.1 -opentracing==2.4.0 -packaging==24.1 -pandas==2.0.3 -pandas-stubs==2.0.3.230814 -pillow==10.1.0 -pluggy==1.5.0 -pydantic==2.8.2 -pydantic-core==2.20.1 -pytest==8.3.2 -pytest-asyncio==0.21.1 -pytest-cov==5.0.0 -pytest-mock==3.14.0 -pytest-randomly==3.15.0 -python-dateutil==2.9.0.post0 -pytz==2024.1 -pyyaml==6.0.2 -regex==2024.7.24 -requests==2.32.3 -six==1.16.0 -sniffio==1.3.1 -sortedcontainers==2.4.0 -tiktoken==0.7.0 -tomli==2.0.1 -tqdm==4.66.5 -types-pytz==2024.1.0.20240417 -typing-extensions==4.12.2 -tzdata==2024.1 -urllib3==1.26.19 -vcrpy==4.2.1 -wrapt==1.16.0 -yarl==1.9.4 -zipp==3.20.1 diff --git a/.riot/requirements/13fec34.txt b/.riot/requirements/13fec34.txt deleted file mode 100644 index 8858506f793..00000000000 --- a/.riot/requirements/13fec34.txt +++ /dev/null @@ -1,49 +0,0 @@ -# -# This file is autogenerated by pip-compile with Python 3.11 -# by the following command: -# -# pip-compile --allow-unsafe --no-annotate .riot/requirements/13fec34.in -# -annotated-types==0.7.0 -anyio==3.7.1 -attrs==24.2.0 -certifi==2024.7.4 -coverage[toml]==7.6.1 -distro==1.9.0 -h11==0.14.0 -httpcore==1.0.5 -httpx==0.27.0 -hypothesis==6.45.0 -idna==3.8 -iniconfig==2.0.0 -mock==5.1.0 -multidict==6.0.5 -numpy==2.1.0 -openai[datalib,embeddings]==1.1.1 -opentracing==2.4.0 -packaging==24.1 -pandas==2.2.2 -pandas-stubs==2.2.2.240807 -pillow==9.5.0 -pluggy==1.5.0 -pydantic==2.8.2 -pydantic-core==2.20.1 -pytest==8.3.2 -pytest-asyncio==0.21.1 -pytest-cov==5.0.0 -pytest-mock==3.14.0 -pytest-randomly==3.15.0 -python-dateutil==2.9.0.post0 -pytz==2024.1 -pyyaml==6.0.2 -six==1.16.0 -sniffio==1.3.1 -sortedcontainers==2.4.0 -tqdm==4.66.5 -types-pytz==2024.1.0.20240417 -typing-extensions==4.12.2 -tzdata==2024.1 -urllib3==1.26.19 -vcrpy==4.2.1 -wrapt==1.16.0 -yarl==1.9.4 diff --git a/.riot/requirements/1825740.txt b/.riot/requirements/1825740.txt index b4660fad985..d1ef7a92bc0 100644 --- a/.riot/requirements/1825740.txt +++ b/.riot/requirements/1825740.txt @@ -9,13 +9,13 @@ aiosignal==1.3.1 async-timeout==4.0.3 asynctest==0.13.0 attrs==24.2.0 -certifi==2024.7.4 -charset-normalizer==3.3.2 +certifi==2024.12.14 +charset-normalizer==3.4.1 coverage[toml]==7.2.7 exceptiongroup==1.2.2 frozenlist==1.3.3 hypothesis==6.45.0 -idna==3.8 +idna==3.10 importlib-metadata==6.7.0 iniconfig==2.0.0 joblib==1.3.2 @@ -36,13 +36,13 @@ pyyaml==6.0.1 requests==2.31.0 scikit-learn==1.0.2 scipy==1.7.3 -six==1.16.0 +six==1.17.0 sortedcontainers==2.4.0 threadpoolctl==3.1.0 tomli==2.0.1 -tqdm==4.66.5 +tqdm==4.67.1 typing-extensions==4.7.1 -urllib3==1.26.19 +urllib3==1.26.20 vcrpy==4.2.1 wrapt==1.16.0 yarl==1.9.4 diff --git a/.riot/requirements/18de44f.txt b/.riot/requirements/18de44f.txt new file mode 100644 index 00000000000..702b980c641 --- /dev/null +++ b/.riot/requirements/18de44f.txt @@ -0,0 +1,52 @@ +# +# This file is autogenerated by pip-compile with Python 3.8 +# by the following command: +# +# pip-compile --no-annotate --resolver=backtracking .riot/requirements/18de44f.in +# +annotated-types==0.7.0 +anyio==4.5.2 +attrs==24.3.0 +certifi==2024.12.14 +charset-normalizer==3.4.1 +coverage[toml]==7.6.1 +distro==1.9.0 +exceptiongroup==1.2.2 +h11==0.14.0 +httpcore==1.0.7 +httpx==0.28.1 +hypothesis==6.45.0 +idna==3.10 +importlib-metadata==8.5.0 +iniconfig==2.0.0 +jiter==0.8.2 +mock==5.1.0 +multidict==6.1.0 +openai==1.60.0 +opentracing==2.4.0 +packaging==24.2 +pillow==10.4.0 +pluggy==1.5.0 +propcache==0.2.0 +pydantic==2.10.5 +pydantic-core==2.27.2 +pytest==8.3.4 +pytest-asyncio==0.21.1 +pytest-cov==5.0.0 +pytest-mock==3.14.0 +pytest-randomly==3.15.0 +pyyaml==6.0.2 +regex==2024.11.6 +requests==2.32.3 +six==1.17.0 +sniffio==1.3.1 +sortedcontainers==2.4.0 +tiktoken==0.7.0 +tomli==2.2.1 +tqdm==4.67.1 +typing-extensions==4.12.2 +urllib3==1.26.20 +vcrpy==4.2.1 +wrapt==1.17.2 +yarl==1.15.2 +zipp==3.20.2 diff --git a/.riot/requirements/1ad89c5.txt b/.riot/requirements/1ad89c5.txt new file mode 100644 index 00000000000..b10206e12d9 --- /dev/null +++ b/.riot/requirements/1ad89c5.txt @@ -0,0 +1,50 @@ +# +# This file is autogenerated by pip-compile with Python 3.10 +# by the following command: +# +# pip-compile --no-annotate .riot/requirements/1ad89c5.in +# +annotated-types==0.7.0 +anyio==4.8.0 +attrs==24.3.0 +certifi==2024.12.14 +charset-normalizer==3.4.1 +coverage[toml]==7.6.10 +distro==1.9.0 +exceptiongroup==1.2.2 +h11==0.14.0 +httpcore==1.0.7 +httpx==0.28.1 +hypothesis==6.45.0 +idna==3.10 +iniconfig==2.0.0 +jiter==0.8.2 +mock==5.1.0 +multidict==6.1.0 +openai==1.60.0 +opentracing==2.4.0 +packaging==24.2 +pillow==11.1.0 +pluggy==1.5.0 +propcache==0.2.1 +pydantic==2.10.5 +pydantic-core==2.27.2 +pytest==8.3.4 +pytest-asyncio==0.21.1 +pytest-cov==6.0.0 +pytest-mock==3.14.0 +pytest-randomly==3.16.0 +pyyaml==6.0.2 +regex==2024.11.6 +requests==2.32.3 +six==1.17.0 +sniffio==1.3.1 +sortedcontainers==2.4.0 +tiktoken==0.8.0 +tomli==2.2.1 +tqdm==4.67.1 +typing-extensions==4.12.2 +urllib3==1.26.20 +vcrpy==4.2.1 +wrapt==1.17.2 +yarl==1.18.3 diff --git a/.riot/requirements/1db5311.txt b/.riot/requirements/1e6bd37.txt similarity index 55% rename from .riot/requirements/1db5311.txt rename to .riot/requirements/1e6bd37.txt index c29bc9bdb8f..11bb5871c14 100644 --- a/.riot/requirements/1db5311.txt +++ b/.riot/requirements/1e6bd37.txt @@ -2,52 +2,53 @@ # This file is autogenerated by pip-compile with Python 3.8 # by the following command: # -# pip-compile --allow-unsafe --no-annotate .riot/requirements/1db5311.in +# pip-compile --no-annotate --resolver=backtracking .riot/requirements/1e6bd37.in # annotated-types==0.7.0 -anyio==4.4.0 -attrs==24.2.0 -certifi==2024.7.4 +anyio==4.5.2 +attrs==24.3.0 +certifi==2024.12.14 coverage[toml]==7.6.1 distro==1.9.0 exceptiongroup==1.2.2 h11==0.14.0 -httpcore==1.0.5 -httpx==0.27.0 +httpcore==1.0.7 +httpx==0.27.2 hypothesis==6.45.0 -idna==3.8 -importlib-metadata==8.4.0 +idna==3.10 +importlib-metadata==8.5.0 iniconfig==2.0.0 mock==5.1.0 -multidict==6.0.5 +multidict==6.1.0 numpy==1.24.4 openai[datalib,embeddings]==1.30.1 opentracing==2.4.0 -packaging==24.1 +packaging==24.2 pandas==2.0.3 pandas-stubs==2.0.3.230814 pillow==9.5.0 pluggy==1.5.0 -pydantic==2.8.2 -pydantic-core==2.20.1 -pytest==8.3.2 +propcache==0.2.0 +pydantic==2.10.5 +pydantic-core==2.27.2 +pytest==8.3.4 pytest-asyncio==0.21.1 pytest-cov==5.0.0 pytest-mock==3.14.0 pytest-randomly==3.15.0 python-dateutil==2.9.0.post0 -pytz==2024.1 +pytz==2024.2 pyyaml==6.0.2 -six==1.16.0 +six==1.17.0 sniffio==1.3.1 sortedcontainers==2.4.0 -tomli==2.0.1 -tqdm==4.66.5 -types-pytz==2024.1.0.20240417 +tomli==2.2.1 +tqdm==4.67.1 +types-pytz==2024.2.0.20241221 typing-extensions==4.12.2 -tzdata==2024.1 -urllib3==1.26.19 +tzdata==2025.1 +urllib3==1.26.20 vcrpy==4.2.1 -wrapt==1.16.0 -yarl==1.9.4 -zipp==3.20.1 +wrapt==1.17.2 +yarl==1.15.2 +zipp==3.20.2 diff --git a/.riot/requirements/1ec15f5.txt b/.riot/requirements/1ec15f5.txt deleted file mode 100644 index b4479a2fb39..00000000000 --- a/.riot/requirements/1ec15f5.txt +++ /dev/null @@ -1,57 +0,0 @@ -# -# This file is autogenerated by pip-compile with Python 3.9 -# by the following command: -# -# pip-compile --allow-unsafe --no-annotate .riot/requirements/1ec15f5.in -# -annotated-types==0.7.0 -anyio==4.4.0 -attrs==24.2.0 -certifi==2024.7.4 -charset-normalizer==3.3.2 -coverage[toml]==7.6.1 -distro==1.9.0 -exceptiongroup==1.2.2 -h11==0.14.0 -httpcore==1.0.5 -httpx==0.27.0 -hypothesis==6.45.0 -idna==3.8 -importlib-metadata==8.4.0 -iniconfig==2.0.0 -mock==5.1.0 -multidict==6.0.5 -numpy==2.0.1 -openai[datalib]==1.30.1 -opentracing==2.4.0 -packaging==24.1 -pandas==2.2.2 -pandas-stubs==2.2.2.240807 -pillow==10.1.0 -pluggy==1.5.0 -pydantic==2.8.2 -pydantic-core==2.20.1 -pytest==8.3.2 -pytest-asyncio==0.21.1 -pytest-cov==5.0.0 -pytest-mock==3.14.0 -pytest-randomly==3.15.0 -python-dateutil==2.9.0.post0 -pytz==2024.1 -pyyaml==6.0.2 -regex==2024.7.24 -requests==2.32.3 -six==1.16.0 -sniffio==1.3.1 -sortedcontainers==2.4.0 -tiktoken==0.7.0 -tomli==2.0.1 -tqdm==4.66.5 -types-pytz==2024.1.0.20240417 -typing-extensions==4.12.2 -tzdata==2024.1 -urllib3==1.26.19 -vcrpy==4.2.1 -wrapt==1.16.0 -yarl==1.9.4 -zipp==3.20.1 diff --git a/.riot/requirements/1ee49b9.txt b/.riot/requirements/1ee49b9.txt deleted file mode 100644 index f170e2885c4..00000000000 --- a/.riot/requirements/1ee49b9.txt +++ /dev/null @@ -1,53 +0,0 @@ -# -# This file is autogenerated by pip-compile with Python 3.11 -# by the following command: -# -# pip-compile --allow-unsafe --no-annotate .riot/requirements/1ee49b9.in -# -annotated-types==0.7.0 -anyio==4.4.0 -attrs==24.2.0 -certifi==2024.7.4 -charset-normalizer==3.3.2 -coverage[toml]==7.6.1 -distro==1.9.0 -h11==0.14.0 -httpcore==1.0.5 -httpx==0.27.0 -hypothesis==6.45.0 -idna==3.8 -iniconfig==2.0.0 -mock==5.1.0 -multidict==6.0.5 -numpy==2.1.0 -openai[datalib]==1.30.1 -opentracing==2.4.0 -packaging==24.1 -pandas==2.2.2 -pandas-stubs==2.2.2.240807 -pillow==10.1.0 -pluggy==1.5.0 -pydantic==2.8.2 -pydantic-core==2.20.1 -pytest==8.3.2 -pytest-asyncio==0.21.1 -pytest-cov==5.0.0 -pytest-mock==3.14.0 -pytest-randomly==3.15.0 -python-dateutil==2.9.0.post0 -pytz==2024.1 -pyyaml==6.0.2 -regex==2024.7.24 -requests==2.32.3 -six==1.16.0 -sniffio==1.3.1 -sortedcontainers==2.4.0 -tiktoken==0.7.0 -tqdm==4.66.5 -types-pytz==2024.1.0.20240417 -typing-extensions==4.12.2 -tzdata==2024.1 -urllib3==1.26.19 -vcrpy==4.2.1 -wrapt==1.16.0 -yarl==1.9.4 diff --git a/.riot/requirements/2634bf7.txt b/.riot/requirements/2634bf7.txt new file mode 100644 index 00000000000..0000f6e28ff --- /dev/null +++ b/.riot/requirements/2634bf7.txt @@ -0,0 +1,48 @@ +# +# This file is autogenerated by pip-compile with Python 3.11 +# by the following command: +# +# pip-compile --no-annotate --resolver=backtracking .riot/requirements/2634bf7.in +# +annotated-types==0.7.0 +anyio==4.8.0 +attrs==24.3.0 +certifi==2024.12.14 +charset-normalizer==3.4.1 +coverage[toml]==7.6.10 +distro==1.9.0 +h11==0.14.0 +httpcore==1.0.7 +httpx==0.28.1 +hypothesis==6.45.0 +idna==3.10 +iniconfig==2.0.0 +jiter==0.8.2 +mock==5.1.0 +multidict==6.1.0 +openai==1.60.0 +opentracing==2.4.0 +packaging==24.2 +pillow==11.1.0 +pluggy==1.5.0 +propcache==0.2.1 +pydantic==2.10.5 +pydantic-core==2.27.2 +pytest==8.3.4 +pytest-asyncio==0.21.1 +pytest-cov==6.0.0 +pytest-mock==3.14.0 +pytest-randomly==3.16.0 +pyyaml==6.0.2 +regex==2024.11.6 +requests==2.32.3 +six==1.17.0 +sniffio==1.3.1 +sortedcontainers==2.4.0 +tiktoken==0.8.0 +tqdm==4.67.1 +typing-extensions==4.12.2 +urllib3==1.26.20 +vcrpy==4.2.1 +wrapt==1.17.2 +yarl==1.18.3 diff --git a/.riot/requirements/35ce786.txt b/.riot/requirements/35ce786.txt deleted file mode 100644 index 3489155be91..00000000000 --- a/.riot/requirements/35ce786.txt +++ /dev/null @@ -1,55 +0,0 @@ -# -# This file is autogenerated by pip-compile with Python 3.10 -# by the following command: -# -# pip-compile --allow-unsafe --no-annotate .riot/requirements/35ce786.in -# -annotated-types==0.7.0 -anyio==4.4.0 -attrs==24.2.0 -certifi==2024.7.4 -charset-normalizer==3.3.2 -coverage[toml]==7.6.1 -distro==1.9.0 -exceptiongroup==1.2.2 -h11==0.14.0 -httpcore==1.0.5 -httpx==0.27.0 -hypothesis==6.45.0 -idna==3.8 -iniconfig==2.0.0 -mock==5.1.0 -multidict==6.0.5 -numpy==2.1.0 -openai[datalib]==1.30.1 -opentracing==2.4.0 -packaging==24.1 -pandas==2.2.2 -pandas-stubs==2.2.2.240807 -pillow==10.1.0 -pluggy==1.5.0 -pydantic==2.8.2 -pydantic-core==2.20.1 -pytest==8.3.2 -pytest-asyncio==0.21.1 -pytest-cov==5.0.0 -pytest-mock==3.14.0 -pytest-randomly==3.15.0 -python-dateutil==2.9.0.post0 -pytz==2024.1 -pyyaml==6.0.2 -regex==2024.7.24 -requests==2.32.3 -six==1.16.0 -sniffio==1.3.1 -sortedcontainers==2.4.0 -tiktoken==0.7.0 -tomli==2.0.1 -tqdm==4.66.5 -types-pytz==2024.1.0.20240417 -typing-extensions==4.12.2 -tzdata==2024.1 -urllib3==1.26.19 -vcrpy==4.2.1 -wrapt==1.16.0 -yarl==1.9.4 diff --git a/.riot/requirements/4a85f6d.txt b/.riot/requirements/4a85f6d.txt new file mode 100644 index 00000000000..41953c69178 --- /dev/null +++ b/.riot/requirements/4a85f6d.txt @@ -0,0 +1,50 @@ +# +# This file is autogenerated by pip-compile with Python 3.11 +# by the following command: +# +# pip-compile --no-annotate --resolver=backtracking .riot/requirements/4a85f6d.in +# +annotated-types==0.7.0 +anyio==4.8.0 +attrs==24.3.0 +certifi==2024.12.14 +coverage[toml]==7.6.10 +distro==1.9.0 +h11==0.14.0 +httpcore==1.0.7 +httpx==0.27.2 +hypothesis==6.45.0 +idna==3.10 +iniconfig==2.0.0 +mock==5.1.0 +multidict==6.1.0 +numpy==2.2.2 +openai[datalib,embeddings]==1.30.1 +opentracing==2.4.0 +packaging==24.2 +pandas==2.2.3 +pandas-stubs==2.2.3.241126 +pillow==9.5.0 +pluggy==1.5.0 +propcache==0.2.1 +pydantic==2.10.5 +pydantic-core==2.27.2 +pytest==8.3.4 +pytest-asyncio==0.21.1 +pytest-cov==6.0.0 +pytest-mock==3.14.0 +pytest-randomly==3.16.0 +python-dateutil==2.9.0.post0 +pytz==2024.2 +pyyaml==6.0.2 +six==1.17.0 +sniffio==1.3.1 +sortedcontainers==2.4.0 +tqdm==4.67.1 +types-pytz==2024.2.0.20241221 +typing-extensions==4.12.2 +tzdata==2025.1 +urllib3==1.26.20 +vcrpy==4.2.1 +wrapt==1.17.2 +yarl==1.18.3 diff --git a/.riot/requirements/4d27459.txt b/.riot/requirements/4d27459.txt new file mode 100644 index 00000000000..630c81558f3 --- /dev/null +++ b/.riot/requirements/4d27459.txt @@ -0,0 +1,48 @@ +# +# This file is autogenerated by pip-compile with Python 3.13 +# by the following command: +# +# pip-compile --no-annotate .riot/requirements/4d27459.in +# +annotated-types==0.7.0 +anyio==4.8.0 +attrs==24.3.0 +certifi==2024.12.14 +charset-normalizer==3.4.1 +coverage[toml]==7.6.10 +distro==1.9.0 +h11==0.14.0 +httpcore==1.0.7 +httpx==0.28.1 +hypothesis==6.45.0 +idna==3.10 +iniconfig==2.0.0 +jiter==0.8.2 +mock==5.1.0 +multidict==6.1.0 +openai==1.60.0 +opentracing==2.4.0 +packaging==24.2 +pillow==11.1.0 +pluggy==1.5.0 +propcache==0.2.1 +pydantic==2.10.5 +pydantic-core==2.27.2 +pytest==8.3.4 +pytest-asyncio==0.21.1 +pytest-cov==6.0.0 +pytest-mock==3.14.0 +pytest-randomly==3.16.0 +pyyaml==6.0.2 +regex==2024.11.6 +requests==2.32.3 +six==1.17.0 +sniffio==1.3.1 +sortedcontainers==2.4.0 +tiktoken==0.8.0 +tqdm==4.67.1 +typing-extensions==4.12.2 +urllib3==1.26.20 +vcrpy==4.2.1 +wrapt==1.17.2 +yarl==1.18.3 diff --git a/.riot/requirements/530c983.txt b/.riot/requirements/530c983.txt new file mode 100644 index 00000000000..c07f9a6b918 --- /dev/null +++ b/.riot/requirements/530c983.txt @@ -0,0 +1,52 @@ +# +# This file is autogenerated by pip-compile with Python 3.9 +# by the following command: +# +# pip-compile --no-annotate --resolver=backtracking .riot/requirements/530c983.in +# +annotated-types==0.7.0 +anyio==4.8.0 +attrs==24.3.0 +certifi==2024.12.14 +charset-normalizer==3.4.1 +coverage[toml]==7.6.10 +distro==1.9.0 +exceptiongroup==1.2.2 +h11==0.14.0 +httpcore==1.0.7 +httpx==0.28.1 +hypothesis==6.45.0 +idna==3.10 +importlib-metadata==8.6.1 +iniconfig==2.0.0 +jiter==0.8.2 +mock==5.1.0 +multidict==6.1.0 +openai==1.60.0 +opentracing==2.4.0 +packaging==24.2 +pillow==11.1.0 +pluggy==1.5.0 +propcache==0.2.1 +pydantic==2.10.5 +pydantic-core==2.27.2 +pytest==8.3.4 +pytest-asyncio==0.21.1 +pytest-cov==6.0.0 +pytest-mock==3.14.0 +pytest-randomly==3.16.0 +pyyaml==6.0.2 +regex==2024.11.6 +requests==2.32.3 +six==1.17.0 +sniffio==1.3.1 +sortedcontainers==2.4.0 +tiktoken==0.8.0 +tomli==2.2.1 +tqdm==4.67.1 +typing-extensions==4.12.2 +urllib3==1.26.20 +vcrpy==4.2.1 +wrapt==1.17.2 +yarl==1.18.3 +zipp==3.21.0 diff --git a/.riot/requirements/5da4fd8.txt b/.riot/requirements/5da4fd8.txt deleted file mode 100644 index a700b91bf81..00000000000 --- a/.riot/requirements/5da4fd8.txt +++ /dev/null @@ -1,49 +0,0 @@ -# -# This file is autogenerated by pip-compile with Python 3.11 -# by the following command: -# -# pip-compile --allow-unsafe --no-annotate .riot/requirements/5da4fd8.in -# -annotated-types==0.7.0 -anyio==4.4.0 -attrs==24.2.0 -certifi==2024.7.4 -coverage[toml]==7.6.1 -distro==1.9.0 -h11==0.14.0 -httpcore==1.0.5 -httpx==0.27.0 -hypothesis==6.45.0 -idna==3.8 -iniconfig==2.0.0 -mock==5.1.0 -multidict==6.0.5 -numpy==2.1.0 -openai[datalib,embeddings]==1.30.1 -opentracing==2.4.0 -packaging==24.1 -pandas==2.2.2 -pandas-stubs==2.2.2.240807 -pillow==9.5.0 -pluggy==1.5.0 -pydantic==2.8.2 -pydantic-core==2.20.1 -pytest==8.3.2 -pytest-asyncio==0.21.1 -pytest-cov==5.0.0 -pytest-mock==3.14.0 -pytest-randomly==3.15.0 -python-dateutil==2.9.0.post0 -pytz==2024.1 -pyyaml==6.0.2 -six==1.16.0 -sniffio==1.3.1 -sortedcontainers==2.4.0 -tqdm==4.66.5 -types-pytz==2024.1.0.20240417 -typing-extensions==4.12.2 -tzdata==2024.1 -urllib3==1.26.19 -vcrpy==4.2.1 -wrapt==1.16.0 -yarl==1.9.4 diff --git a/.riot/requirements/84ec59a.txt b/.riot/requirements/84ec59a.txt deleted file mode 100644 index 9b079cf3a38..00000000000 --- a/.riot/requirements/84ec59a.txt +++ /dev/null @@ -1,53 +0,0 @@ -# -# This file is autogenerated by pip-compile with Python 3.8 -# by the following command: -# -# pip-compile --allow-unsafe --no-annotate .riot/requirements/84ec59a.in -# -annotated-types==0.7.0 -anyio==3.7.1 -attrs==24.2.0 -certifi==2024.7.4 -coverage[toml]==7.6.1 -distro==1.9.0 -exceptiongroup==1.2.2 -h11==0.14.0 -httpcore==1.0.5 -httpx==0.27.0 -hypothesis==6.45.0 -idna==3.8 -importlib-metadata==8.4.0 -iniconfig==2.0.0 -mock==5.1.0 -multidict==6.0.5 -numpy==1.24.4 -openai[datalib,embeddings]==1.1.1 -opentracing==2.4.0 -packaging==24.1 -pandas==2.0.3 -pandas-stubs==2.0.3.230814 -pillow==9.5.0 -pluggy==1.5.0 -pydantic==2.8.2 -pydantic-core==2.20.1 -pytest==8.3.2 -pytest-asyncio==0.21.1 -pytest-cov==5.0.0 -pytest-mock==3.14.0 -pytest-randomly==3.15.0 -python-dateutil==2.9.0.post0 -pytz==2024.1 -pyyaml==6.0.2 -six==1.16.0 -sniffio==1.3.1 -sortedcontainers==2.4.0 -tomli==2.0.1 -tqdm==4.66.5 -types-pytz==2024.1.0.20240417 -typing-extensions==4.12.2 -tzdata==2024.1 -urllib3==1.26.19 -vcrpy==4.2.1 -wrapt==1.16.0 -yarl==1.9.4 -zipp==3.20.1 diff --git a/.riot/requirements/87a1fff.txt b/.riot/requirements/87a1fff.txt deleted file mode 100644 index b85e76cdd56..00000000000 --- a/.riot/requirements/87a1fff.txt +++ /dev/null @@ -1,53 +0,0 @@ -# -# This file is autogenerated by pip-compile with Python 3.9 -# by the following command: -# -# pip-compile --allow-unsafe --no-annotate .riot/requirements/87a1fff.in -# -annotated-types==0.7.0 -anyio==4.4.0 -attrs==24.2.0 -certifi==2024.7.4 -coverage[toml]==7.6.1 -distro==1.9.0 -exceptiongroup==1.2.2 -h11==0.14.0 -httpcore==1.0.5 -httpx==0.27.0 -hypothesis==6.45.0 -idna==3.8 -importlib-metadata==8.4.0 -iniconfig==2.0.0 -mock==5.1.0 -multidict==6.0.5 -numpy==2.0.1 -openai[datalib,embeddings]==1.30.1 -opentracing==2.4.0 -packaging==24.1 -pandas==2.2.2 -pandas-stubs==2.2.2.240807 -pillow==9.5.0 -pluggy==1.5.0 -pydantic==2.8.2 -pydantic-core==2.20.1 -pytest==8.3.2 -pytest-asyncio==0.21.1 -pytest-cov==5.0.0 -pytest-mock==3.14.0 -pytest-randomly==3.15.0 -python-dateutil==2.9.0.post0 -pytz==2024.1 -pyyaml==6.0.2 -six==1.16.0 -sniffio==1.3.1 -sortedcontainers==2.4.0 -tomli==2.0.1 -tqdm==4.66.5 -types-pytz==2024.1.0.20240417 -typing-extensions==4.12.2 -tzdata==2024.1 -urllib3==1.26.19 -vcrpy==4.2.1 -wrapt==1.16.0 -yarl==1.9.4 -zipp==3.20.1 diff --git a/.riot/requirements/b5d5a35.txt b/.riot/requirements/b5d5a35.txt new file mode 100644 index 00000000000..7838b7abd2c --- /dev/null +++ b/.riot/requirements/b5d5a35.txt @@ -0,0 +1,52 @@ +# +# This file is autogenerated by pip-compile with Python 3.10 +# by the following command: +# +# pip-compile --no-annotate .riot/requirements/b5d5a35.in +# +annotated-types==0.7.0 +anyio==4.8.0 +attrs==24.3.0 +certifi==2024.12.14 +coverage[toml]==7.6.10 +distro==1.9.0 +exceptiongroup==1.2.2 +h11==0.14.0 +httpcore==1.0.7 +httpx==0.27.2 +hypothesis==6.45.0 +idna==3.10 +iniconfig==2.0.0 +mock==5.1.0 +multidict==6.1.0 +numpy==2.2.2 +openai[datalib,embeddings]==1.30.1 +opentracing==2.4.0 +packaging==24.2 +pandas==2.2.3 +pandas-stubs==2.2.3.241126 +pillow==9.5.0 +pluggy==1.5.0 +propcache==0.2.1 +pydantic==2.10.5 +pydantic-core==2.27.2 +pytest==8.3.4 +pytest-asyncio==0.21.1 +pytest-cov==6.0.0 +pytest-mock==3.14.0 +pytest-randomly==3.16.0 +python-dateutil==2.9.0.post0 +pytz==2024.2 +pyyaml==6.0.2 +six==1.17.0 +sniffio==1.3.1 +sortedcontainers==2.4.0 +tomli==2.2.1 +tqdm==4.67.1 +types-pytz==2024.2.0.20241221 +typing-extensions==4.12.2 +tzdata==2025.1 +urllib3==1.26.20 +vcrpy==4.2.1 +wrapt==1.17.2 +yarl==1.18.3 diff --git a/.riot/requirements/c74f6e0.txt b/.riot/requirements/c74f6e0.txt deleted file mode 100644 index 63345853661..00000000000 --- a/.riot/requirements/c74f6e0.txt +++ /dev/null @@ -1,51 +0,0 @@ -# -# This file is autogenerated by pip-compile with Python 3.10 -# by the following command: -# -# pip-compile --allow-unsafe --no-annotate .riot/requirements/c74f6e0.in -# -annotated-types==0.7.0 -anyio==4.4.0 -attrs==24.2.0 -certifi==2024.7.4 -coverage[toml]==7.6.1 -distro==1.9.0 -exceptiongroup==1.2.2 -h11==0.14.0 -httpcore==1.0.5 -httpx==0.27.0 -hypothesis==6.45.0 -idna==3.8 -iniconfig==2.0.0 -mock==5.1.0 -multidict==6.0.5 -numpy==2.1.0 -openai[datalib,embeddings]==1.30.1 -opentracing==2.4.0 -packaging==24.1 -pandas==2.2.2 -pandas-stubs==2.2.2.240807 -pillow==9.5.0 -pluggy==1.5.0 -pydantic==2.8.2 -pydantic-core==2.20.1 -pytest==8.3.2 -pytest-asyncio==0.21.1 -pytest-cov==5.0.0 -pytest-mock==3.14.0 -pytest-randomly==3.15.0 -python-dateutil==2.9.0.post0 -pytz==2024.1 -pyyaml==6.0.2 -six==1.16.0 -sniffio==1.3.1 -sortedcontainers==2.4.0 -tomli==2.0.1 -tqdm==4.66.5 -types-pytz==2024.1.0.20240417 -typing-extensions==4.12.2 -tzdata==2024.1 -urllib3==1.26.19 -vcrpy==4.2.1 -wrapt==1.16.0 -yarl==1.9.4 diff --git a/.riot/requirements/cd2e4ea.txt b/.riot/requirements/cd2e4ea.txt deleted file mode 100644 index 24353dafa0c..00000000000 --- a/.riot/requirements/cd2e4ea.txt +++ /dev/null @@ -1,53 +0,0 @@ -# -# This file is autogenerated by pip-compile with Python 3.9 -# by the following command: -# -# pip-compile --allow-unsafe --no-annotate .riot/requirements/cd2e4ea.in -# -annotated-types==0.7.0 -anyio==3.7.1 -attrs==24.2.0 -certifi==2024.7.4 -coverage[toml]==7.6.1 -distro==1.9.0 -exceptiongroup==1.2.2 -h11==0.14.0 -httpcore==1.0.5 -httpx==0.27.0 -hypothesis==6.45.0 -idna==3.8 -importlib-metadata==8.4.0 -iniconfig==2.0.0 -mock==5.1.0 -multidict==6.0.5 -numpy==2.0.1 -openai[datalib,embeddings]==1.1.1 -opentracing==2.4.0 -packaging==24.1 -pandas==2.2.2 -pandas-stubs==2.2.2.240807 -pillow==9.5.0 -pluggy==1.5.0 -pydantic==2.8.2 -pydantic-core==2.20.1 -pytest==8.3.2 -pytest-asyncio==0.21.1 -pytest-cov==5.0.0 -pytest-mock==3.14.0 -pytest-randomly==3.15.0 -python-dateutil==2.9.0.post0 -pytz==2024.1 -pyyaml==6.0.2 -six==1.16.0 -sniffio==1.3.1 -sortedcontainers==2.4.0 -tomli==2.0.1 -tqdm==4.66.5 -types-pytz==2024.1.0.20240417 -typing-extensions==4.12.2 -tzdata==2024.1 -urllib3==1.26.19 -vcrpy==4.2.1 -wrapt==1.16.0 -yarl==1.9.4 -zipp==3.20.1 diff --git a/.riot/requirements/181216c.txt b/.riot/requirements/df60af6.txt similarity index 82% rename from .riot/requirements/181216c.txt rename to .riot/requirements/df60af6.txt index ac739930363..5143f0e0a74 100644 --- a/.riot/requirements/181216c.txt +++ b/.riot/requirements/df60af6.txt @@ -2,13 +2,13 @@ # This file is autogenerated by pip-compile with Python 3.7 # by the following command: # -# pip-compile --allow-unsafe --config=pyproject.toml --no-annotate --resolver=backtracking .riot/requirements/181216c.in +# pip-compile --allow-unsafe --config=pyproject.toml --no-annotate --resolver=backtracking .riot/requirements/df60af6.in # annotated-types==0.5.0 anyio==3.7.1 attrs==24.2.0 cached-property==1.5.2 -certifi==2024.7.4 +certifi==2024.12.14 coverage[toml]==7.2.7 distro==1.9.0 exceptiongroup==1.2.2 @@ -16,13 +16,13 @@ h11==0.14.0 httpcore==0.17.3 httpx==0.24.1 hypothesis==6.45.0 -idna==3.8 +idna==3.10 importlib-metadata==6.7.0 iniconfig==2.0.0 mock==5.1.0 multidict==6.0.5 numpy==1.21.6 -openai[datalib,embeddings]==1.30.1 +openai[datalib]==1.30.1 opentracing==2.4.0 packaging==24.0 pandas==1.3.5 @@ -37,15 +37,15 @@ pytest-cov==4.1.0 pytest-mock==3.11.1 pytest-randomly==3.12.0 python-dateutil==2.9.0.post0 -pytz==2024.1 +pytz==2024.2 pyyaml==6.0.1 -six==1.16.0 +six==1.17.0 sniffio==1.3.1 sortedcontainers==2.4.0 tomli==2.0.1 -tqdm==4.66.5 +tqdm==4.67.1 typing-extensions==4.7.1 -urllib3==1.26.19 +urllib3==1.26.20 vcrpy==4.2.1 wrapt==1.16.0 yarl==1.9.4 diff --git a/.riot/requirements/f1c37b1.txt b/.riot/requirements/f1c37b1.txt deleted file mode 100644 index 4da5078a988..00000000000 --- a/.riot/requirements/f1c37b1.txt +++ /dev/null @@ -1,51 +0,0 @@ -# -# This file is autogenerated by pip-compile with Python 3.10 -# by the following command: -# -# pip-compile --allow-unsafe --no-annotate .riot/requirements/f1c37b1.in -# -annotated-types==0.7.0 -anyio==3.7.1 -attrs==24.2.0 -certifi==2024.7.4 -coverage[toml]==7.6.1 -distro==1.9.0 -exceptiongroup==1.2.2 -h11==0.14.0 -httpcore==1.0.5 -httpx==0.27.0 -hypothesis==6.45.0 -idna==3.8 -iniconfig==2.0.0 -mock==5.1.0 -multidict==6.0.5 -numpy==2.1.0 -openai[datalib,embeddings]==1.1.1 -opentracing==2.4.0 -packaging==24.1 -pandas==2.2.2 -pandas-stubs==2.2.2.240807 -pillow==9.5.0 -pluggy==1.5.0 -pydantic==2.8.2 -pydantic-core==2.20.1 -pytest==8.3.2 -pytest-asyncio==0.21.1 -pytest-cov==5.0.0 -pytest-mock==3.14.0 -pytest-randomly==3.15.0 -python-dateutil==2.9.0.post0 -pytz==2024.1 -pyyaml==6.0.2 -six==1.16.0 -sniffio==1.3.1 -sortedcontainers==2.4.0 -tomli==2.0.1 -tqdm==4.66.5 -types-pytz==2024.1.0.20240417 -typing-extensions==4.12.2 -tzdata==2024.1 -urllib3==1.26.19 -vcrpy==4.2.1 -wrapt==1.16.0 -yarl==1.9.4 diff --git a/.riot/requirements/f7c30a0.txt b/.riot/requirements/f7c30a0.txt deleted file mode 100644 index 3e4716aede1..00000000000 --- a/.riot/requirements/f7c30a0.txt +++ /dev/null @@ -1,51 +0,0 @@ -# -# This file is autogenerated by pip-compile with Python 3.7 -# by the following command: -# -# pip-compile --allow-unsafe --config=pyproject.toml --no-annotate --resolver=backtracking .riot/requirements/f7c30a0.in -# -annotated-types==0.5.0 -anyio==3.7.1 -attrs==24.2.0 -certifi==2024.7.4 -coverage[toml]==7.2.7 -distro==1.9.0 -exceptiongroup==1.2.2 -h11==0.14.0 -httpcore==0.17.3 -httpx==0.24.1 -hypothesis==6.45.0 -idna==3.8 -importlib-metadata==6.7.0 -iniconfig==2.0.0 -mock==5.1.0 -multidict==6.0.5 -numpy==1.21.6 -openai[datalib,embeddings]==1.1.1 -opentracing==2.4.0 -packaging==24.0 -pandas==1.3.5 -pandas-stubs==1.2.0.62 -pillow==9.5.0 -pluggy==1.2.0 -pydantic==2.5.3 -pydantic-core==2.14.6 -pytest==7.4.4 -pytest-asyncio==0.21.1 -pytest-cov==4.1.0 -pytest-mock==3.11.1 -pytest-randomly==3.12.0 -python-dateutil==2.9.0.post0 -pytz==2024.1 -pyyaml==6.0.1 -six==1.16.0 -sniffio==1.3.1 -sortedcontainers==2.4.0 -tomli==2.0.1 -tqdm==4.66.5 -typing-extensions==4.7.1 -urllib3==1.26.19 -vcrpy==4.2.1 -wrapt==1.16.0 -yarl==1.9.4 -zipp==3.15.0 diff --git a/ddtrace/contrib/internal/openai/_endpoint_hooks.py b/ddtrace/contrib/internal/openai/_endpoint_hooks.py index 73a2b2511c9..fc96a296761 100644 --- a/ddtrace/contrib/internal/openai/_endpoint_hooks.py +++ b/ddtrace/contrib/internal/openai/_endpoint_hooks.py @@ -37,7 +37,7 @@ class _EndpointHook: OPERATION_ID = "" # Each endpoint hook must provide an operationID as specified in the OpenAI API specs: # https://raw.githubusercontent.com/openai/openai-openapi/master/openapi.yaml - def _record_request(self, pin, integration, span, args, kwargs): + def _record_request(self, pin, integration, instance, span, args, kwargs): """ Set base-level openai tags, as well as request params from args and kwargs. All inherited EndpointHook classes should include a super call to this method before performing @@ -45,12 +45,12 @@ def _record_request(self, pin, integration, span, args, kwargs): """ endpoint = self.ENDPOINT_NAME if endpoint is None: - endpoint = "%s" % args[0].OBJECT_NAME + endpoint = "%s" % getattr(instance, "OBJECT_NAME", "") span.set_tag_str("openai.request.endpoint", "/%s/%s" % (API_VERSION, endpoint)) span.set_tag_str("openai.request.method", self.HTTP_METHOD_TYPE) if self._request_arg_params and len(self._request_arg_params) > 1: - for idx, arg in enumerate(self._request_arg_params, 1): + for idx, arg in enumerate(self._request_arg_params): if idx >= len(args): break if arg is None or args[idx] is None: @@ -74,8 +74,8 @@ def _record_request(self, pin, integration, span, args, kwargs): else: span.set_tag_str("openai.request.%s" % kw_attr, str(kwargs[kw_attr])) - def handle_request(self, pin, integration, span, args, kwargs): - self._record_request(pin, integration, span, args, kwargs) + def handle_request(self, pin, integration, instance, span, args, kwargs): + self._record_request(pin, integration, instance, span, args, kwargs) resp, error = yield if hasattr(resp, "parse"): # Users can request the raw response, in which case we need to process on the parsed response @@ -186,8 +186,8 @@ class _CompletionHook(_BaseCompletionHook): HTTP_METHOD_TYPE = "POST" OPERATION_ID = "createCompletion" - def _record_request(self, pin, integration, span, args, kwargs): - super()._record_request(pin, integration, span, args, kwargs) + def _record_request(self, pin, integration, instance, span, args, kwargs): + super()._record_request(pin, integration, instance, span, args, kwargs) if integration.is_pc_sampled_span(span): prompt = kwargs.get("prompt", "") if isinstance(prompt, str): @@ -241,8 +241,8 @@ class _ChatCompletionHook(_BaseCompletionHook): HTTP_METHOD_TYPE = "POST" OPERATION_ID = "createChatCompletion" - def _record_request(self, pin, integration, span, args, kwargs): - super()._record_request(pin, integration, span, args, kwargs) + def _record_request(self, pin, integration, instance, span, args, kwargs): + super()._record_request(pin, integration, instance, span, args, kwargs) for idx, m in enumerate(kwargs.get("messages", [])): role = getattr(m, "role", "") name = getattr(m, "name", "") @@ -297,12 +297,12 @@ class _EmbeddingHook(_EndpointHook): HTTP_METHOD_TYPE = "POST" OPERATION_ID = "createEmbedding" - def _record_request(self, pin, integration, span, args, kwargs): + def _record_request(self, pin, integration, instance, span, args, kwargs): """ Embedding endpoint allows multiple inputs, each of which we specify a request tag for, so have to manually set them in _pre_response(). """ - super()._record_request(pin, integration, span, args, kwargs) + super()._record_request(pin, integration, instance, span, args, kwargs) embedding_input = kwargs.get("input", "") if integration.is_pc_sampled_span(span): if isinstance(embedding_input, str) or isinstance(embedding_input[0], int): @@ -332,8 +332,8 @@ class _ListHook(_EndpointHook): HTTP_METHOD_TYPE = "GET" OPERATION_ID = "list" - def _record_request(self, pin, integration, span, args, kwargs): - super()._record_request(pin, integration, span, args, kwargs) + def _record_request(self, pin, integration, instance, span, args, kwargs): + super()._record_request(pin, integration, instance, span, args, kwargs) endpoint = span.get_tag("openai.request.endpoint") if endpoint.endswith("/models"): span.resource = "listModels" @@ -391,15 +391,21 @@ class _RetrieveHook(_EndpointHook): HTTP_METHOD_TYPE = "GET" OPERATION_ID = "retrieve" - def _record_request(self, pin, integration, span, args, kwargs): - super()._record_request(pin, integration, span, args, kwargs) + def _record_request(self, pin, integration, instance, span, args, kwargs): + super()._record_request(pin, integration, instance, span, args, kwargs) endpoint = span.get_tag("openai.request.endpoint") if endpoint.endswith("/models"): span.resource = "retrieveModel" - span.set_tag_str("openai.request.model", args[1] if len(args) >= 2 else kwargs.get("model", "")) + if len(args) >= 1: + span.set_tag_str("openai.request.model", args[0]) + else: + span.set_tag_str("openai.request.model", kwargs.get("model", kwargs.get("id", ""))) elif endpoint.endswith("/files"): span.resource = "retrieveFile" - span.set_tag_str("openai.request.file_id", args[1] if len(args) >= 2 else kwargs.get("file_id", "")) + if len(args) >= 1: + span.set_tag_str("openai.request.file_id", args[0]) + else: + span.set_tag_str("openai.request.file_id", kwargs.get("file_id", kwargs.get("id", ""))) span.set_tag_str("openai.request.endpoint", "%s/*" % endpoint) def _record_response(self, pin, integration, span, args, kwargs, resp, error): @@ -426,10 +432,6 @@ class _ModelRetrieveHook(_RetrieveHook): ENDPOINT_NAME = "models" OPERATION_ID = "retrieveModel" - def _record_request(self, pin, integration, span, args, kwargs): - super()._record_request(pin, integration, span, args, kwargs) - span.set_tag_str("openai.request.model", args[1] if len(args) >= 2 else kwargs.get("model", "")) - class _FileRetrieveHook(_RetrieveHook): """ @@ -439,10 +441,6 @@ class _FileRetrieveHook(_RetrieveHook): ENDPOINT_NAME = "files" OPERATION_ID = "retrieveFile" - def _record_request(self, pin, integration, span, args, kwargs): - super()._record_request(pin, integration, span, args, kwargs) - span.set_tag_str("openai.request.file_id", args[1] if len(args) >= 2 else kwargs.get("file_id", "")) - class _DeleteHook(_EndpointHook): """Hook for openai.DeletableAPIResource, which is used by File.delete, and Model.delete.""" @@ -453,15 +451,21 @@ class _DeleteHook(_EndpointHook): HTTP_METHOD_TYPE = "DELETE" OPERATION_ID = "delete" - def _record_request(self, pin, integration, span, args, kwargs): - super()._record_request(pin, integration, span, args, kwargs) + def _record_request(self, pin, integration, instance, span, args, kwargs): + super()._record_request(pin, integration, instance, span, args, kwargs) endpoint = span.get_tag("openai.request.endpoint") if endpoint.endswith("/models"): span.resource = "deleteModel" - span.set_tag_str("openai.request.model", args[1] if len(args) >= 2 else kwargs.get("model", "")) + if len(args) >= 1: + span.set_tag_str("openai.request.model", args[0]) + else: + span.set_tag_str("openai.request.model", kwargs.get("model", kwargs.get("sid", ""))) elif endpoint.endswith("/files"): span.resource = "deleteFile" - span.set_tag_str("openai.request.file_id", args[1] if len(args) >= 2 else kwargs.get("file_id", "")) + if len(args) >= 1: + span.set_tag_str("openai.request.file_id", args[0]) + else: + span.set_tag_str("openai.request.file_id", kwargs.get("file_id", kwargs.get("sid", ""))) span.set_tag_str("openai.request.endpoint", "%s/*" % endpoint) def _record_response(self, pin, integration, span, args, kwargs, resp, error): @@ -500,8 +504,8 @@ class _ImageHook(_EndpointHook): ENDPOINT_NAME = "images" HTTP_METHOD_TYPE = "POST" - def _record_request(self, pin, integration, span, args, kwargs): - super()._record_request(pin, integration, span, args, kwargs) + def _record_request(self, pin, integration, instance, span, args, kwargs): + super()._record_request(pin, integration, instance, span, args, kwargs) span.set_tag_str("openai.request.model", "dall-e") def _record_response(self, pin, integration, span, args, kwargs, resp, error): @@ -518,10 +522,10 @@ def _record_response(self, pin, integration, span, args, kwargs, resp, error): if "prompt" in self._request_kwarg_params: attrs_dict.update({"prompt": kwargs.get("prompt", "")}) if "image" in self._request_kwarg_params: - image = args[1] if len(args) >= 2 else kwargs.get("image", "") + image = args[0] if len(args) >= 1 else kwargs.get("image", "") attrs_dict.update({"image": image.name.split("/")[-1]}) if "mask" in self._request_kwarg_params: - mask = args[2] if len(args) >= 3 else kwargs.get("mask", "") + mask = args[1] if len(args) >= 2 else kwargs.get("mask", "") attrs_dict.update({"mask": mask.name.split("/")[-1]}) integration.log( span, "info" if error is None else "error", "sampled %s" % self.OPERATION_ID, attrs=attrs_dict @@ -552,12 +556,12 @@ class _ImageEditHook(_ImageHook): ENDPOINT_NAME = "images/edits" OPERATION_ID = "createImageEdit" - def _record_request(self, pin, integration, span, args, kwargs): - super()._record_request(pin, integration, span, args, kwargs) + def _record_request(self, pin, integration, instance, span, args, kwargs): + super()._record_request(pin, integration, instance, span, args, kwargs) if not integration.is_pc_sampled_span: return - image = args[1] if len(args) >= 2 else kwargs.get("image", "") - mask = args[2] if len(args) >= 3 else kwargs.get("mask", "") + image = args[0] if len(args) >= 1 else kwargs.get("image", "") + mask = args[1] if len(args) >= 2 else kwargs.get("mask", "") if image: if hasattr(image, "name"): span.set_tag_str("openai.request.image", integration.trunc(image.name.split("/")[-1])) @@ -576,11 +580,11 @@ class _ImageVariationHook(_ImageHook): ENDPOINT_NAME = "images/variations" OPERATION_ID = "createImageVariation" - def _record_request(self, pin, integration, span, args, kwargs): - super()._record_request(pin, integration, span, args, kwargs) + def _record_request(self, pin, integration, instance, span, args, kwargs): + super()._record_request(pin, integration, instance, span, args, kwargs) if not integration.is_pc_sampled_span: return - image = args[1] if len(args) >= 2 else kwargs.get("image", "") + image = args[0] if len(args) >= 1 else kwargs.get("image", "") if image: if hasattr(image, "name"): span.set_tag_str("openai.request.image", integration.trunc(image.name.split("/")[-1])) @@ -594,11 +598,11 @@ class _BaseAudioHook(_EndpointHook): ENDPOINT_NAME = "audio" HTTP_METHOD_TYPE = "POST" - def _record_request(self, pin, integration, span, args, kwargs): - super()._record_request(pin, integration, span, args, kwargs) + def _record_request(self, pin, integration, instance, span, args, kwargs): + super()._record_request(pin, integration, instance, span, args, kwargs) if not integration.is_pc_sampled_span: return - audio_file = args[2] if len(args) >= 3 else kwargs.get("file", "") + audio_file = args[1] if len(args) >= 2 else kwargs.get("file", "") if audio_file and hasattr(audio_file, "name"): span.set_tag_str("openai.request.filename", integration.trunc(audio_file.name.split("/")[-1])) else: @@ -618,7 +622,7 @@ def _record_response(self, pin, integration, span, args, kwargs, resp, error): if integration.is_pc_sampled_span(span): span.set_tag_str("openai.response.text", integration.trunc(text)) if integration.is_pc_sampled_log(span): - file_input = args[2] if len(args) >= 3 else kwargs.get("file", "") + file_input = args[1] if len(args) >= 2 else kwargs.get("file", "") integration.log( span, "info" if error is None else "error", @@ -677,8 +681,8 @@ class _ModerationHook(_EndpointHook): HTTP_METHOD_TYPE = "POST" OPERATION_ID = "createModeration" - def _record_request(self, pin, integration, span, args, kwargs): - super()._record_request(pin, integration, span, args, kwargs) + def _record_request(self, pin, integration, instance, span, args, kwargs): + super()._record_request(pin, integration, instance, span, args, kwargs) def _record_response(self, pin, integration, span, args, kwargs, resp, error): resp = super()._record_response(pin, integration, span, args, kwargs, resp, error) @@ -710,14 +714,14 @@ class _FileCreateHook(_BaseFileHook): "organization", "user_provided_filename", ) - _request_kwarg_params = ("purpose",) + _request_kwarg_params = ("purpose", "user_provided_filename") _response_attrs = ("id", "bytes", "created_at", "filename", "purpose", "status", "status_details") HTTP_METHOD_TYPE = "POST" OPERATION_ID = "createFile" - def _record_request(self, pin, integration, span, args, kwargs): - super()._record_request(pin, integration, span, args, kwargs) - fp = args[1] if len(args) >= 2 else kwargs.get("file", "") + def _record_request(self, pin, integration, instance, span, args, kwargs): + super()._record_request(pin, integration, instance, span, args, kwargs) + fp = args[0] if len(args) >= 1 else kwargs.get("file", "") if fp and hasattr(fp, "name"): span.set_tag_str("openai.request.filename", fp.name.split("/")[-1]) else: @@ -734,9 +738,9 @@ class _FileDownloadHook(_BaseFileHook): OPERATION_ID = "downloadFile" ENDPOINT_NAME = "files/*/content" - def _record_request(self, pin, integration, span, args, kwargs): - super()._record_request(pin, integration, span, args, kwargs) - span.set_tag_str("openai.request.file_id", args[1] if len(args) >= 2 else kwargs.get("file_id", "")) + def _record_request(self, pin, integration, instance, span, args, kwargs): + super()._record_request(pin, integration, instance, span, args, kwargs) + span.set_tag_str("openai.request.file_id", args[0] if len(args) >= 1 else kwargs.get("file_id", "")) def _record_response(self, pin, integration, span, args, kwargs, resp, error): resp = super()._record_response(pin, integration, span, args, kwargs, resp, error) diff --git a/ddtrace/contrib/internal/openai/patch.py b/ddtrace/contrib/internal/openai/patch.py index 4ad76a17084..830e72757a1 100644 --- a/ddtrace/contrib/internal/openai/patch.py +++ b/ddtrace/contrib/internal/openai/patch.py @@ -6,12 +6,14 @@ from ddtrace import config from ddtrace.contrib.internal.openai import _endpoint_hooks from ddtrace.contrib.internal.openai.utils import _format_openai_api_key +from ddtrace.contrib.trace_utils import unwrap +from ddtrace.contrib.trace_utils import with_traced_module +from ddtrace.contrib.trace_utils import wrap from ddtrace.internal.logger import get_logger from ddtrace.internal.schema import schematize_service_name from ddtrace.internal.utils.formats import asbool from ddtrace.internal.utils.formats import deep_getattr from ddtrace.internal.utils.version import parse_version -from ddtrace.internal.wrapping import wrap from ddtrace.llmobs._integrations import OpenAIIntegration from ddtrace.pin import Pin @@ -80,8 +82,9 @@ def get_version(): else: _RESOURCES = { "model.Model": { - "list": _endpoint_hooks._ListHook, - "retrieve": _endpoint_hooks._RetrieveHook, + "list": _endpoint_hooks._ModelListHook, + "retrieve": _endpoint_hooks._ModelRetrieveHook, + "delete": _endpoint_hooks._ModelDeleteHook, }, "completion.Completion": { "create": _endpoint_hooks._CompletionHook, @@ -105,19 +108,15 @@ def get_version(): "create": _endpoint_hooks._ModerationHook, }, "file.File": { - # File.list() and File.retrieve() share the same underlying method as Model.list() and Model.retrieve() - # which means they are already wrapped + "list": _endpoint_hooks._FileListHook, + "retrieve": _endpoint_hooks._FileRetrieveHook, "create": _endpoint_hooks._FileCreateHook, - "delete": _endpoint_hooks._DeleteHook, + "delete": _endpoint_hooks._FileDeleteHook, "download": _endpoint_hooks._FileDownloadHook, }, } -def _wrap_classmethod(obj, wrapper): - wrap(obj.__func__, wrapper) - - def patch(): # Avoid importing openai at the module level, eventually will be an import hook import openai @@ -127,72 +126,106 @@ def patch(): Pin().onto(openai) integration = OpenAIIntegration(integration_config=config.openai, openai=openai) + openai._datadog_integration = integration if OPENAI_VERSION >= (1, 0, 0): if OPENAI_VERSION >= (1, 8, 0): - wrap(openai._base_client.SyncAPIClient._process_response, _patched_convert(openai, integration)) - wrap(openai._base_client.AsyncAPIClient._process_response, _patched_convert(openai, integration)) + wrap(openai, "_base_client.SyncAPIClient._process_response", patched_convert(openai)) + wrap(openai, "_base_client.AsyncAPIClient._process_response", patched_convert(openai)) else: - wrap(openai._base_client.BaseClient._process_response, _patched_convert(openai, integration)) - wrap(openai.OpenAI.__init__, _patched_client_init(openai, integration)) - wrap(openai.AsyncOpenAI.__init__, _patched_client_init(openai, integration)) - wrap(openai.AzureOpenAI.__init__, _patched_client_init(openai, integration)) - wrap(openai.AsyncAzureOpenAI.__init__, _patched_client_init(openai, integration)) + wrap(openai, "_base_client.BaseClient._process_response", patched_convert(openai)) + wrap(openai, "OpenAI.__init__", patched_client_init(openai)) + wrap(openai, "AsyncOpenAI.__init__", patched_client_init(openai)) + wrap(openai, "AzureOpenAI.__init__", patched_client_init(openai)) + wrap(openai, "AsyncAzureOpenAI.__init__", patched_client_init(openai)) for resource, method_hook_dict in _RESOURCES.items(): if deep_getattr(openai.resources, resource) is None: continue for method_name, endpoint_hook in method_hook_dict.items(): - sync_method = deep_getattr(openai.resources, "%s.%s" % (resource, method_name)) - async_method = deep_getattr( - openai.resources, "%s.%s" % (".Async".join(resource.split(".")), method_name) - ) - wrap(sync_method, _patched_endpoint(openai, integration, endpoint_hook)) - wrap(async_method, _patched_endpoint_async(openai, integration, endpoint_hook)) + sync_method = "resources.{}.{}".format(resource, method_name) + async_method = "resources.{}.{}".format(".Async".join(resource.split(".")), method_name) + wrap(openai, sync_method, _patched_endpoint(openai, endpoint_hook)) + wrap(openai, async_method, _patched_endpoint_async(openai, endpoint_hook)) else: import openai.api_requestor - wrap(openai.api_requestor._make_session, _patched_make_session) - wrap(openai.util.convert_to_openai_object, _patched_convert(openai, integration)) + wrap(openai, "api_requestor._make_session", _patched_make_session) + wrap(openai, "util.convert_to_openai_object", patched_convert(openai)) for resource, method_hook_dict in _RESOURCES.items(): if deep_getattr(openai.api_resources, resource) is None: continue for method_name, endpoint_hook in method_hook_dict.items(): - sync_method = deep_getattr(openai.api_resources, "%s.%s" % (resource, method_name)) - async_method = deep_getattr(openai.api_resources, "%s.a%s" % (resource, method_name)) - _wrap_classmethod(sync_method, _patched_endpoint(openai, integration, endpoint_hook)) - _wrap_classmethod(async_method, _patched_endpoint_async(openai, integration, endpoint_hook)) + sync_method = "api_resources.{}.{}".format(resource, method_name) + async_method = "api_resources.{}.a{}".format(resource, method_name) + wrap(openai, sync_method, _patched_endpoint(openai, endpoint_hook)) + wrap(openai, async_method, _patched_endpoint_async(openai, endpoint_hook)) openai.__datadog_patch = True def unpatch(): - # FIXME: add unpatching. The current wrapping.unwrap method requires - # the wrapper function to be provided which we don't keep a reference to. - pass + import openai + if not getattr(openai, "__datadog_patch", False): + return -def _patched_client_init(openai, integration): - """ - Patch for `openai.OpenAI/AsyncOpenAI` client init methods to add the client object to the OpenAIIntegration object. - """ + openai.__datadog_patch = False - def patched_client_init(func, args, kwargs): - func(*args, **kwargs) - client = args[0] - integration._client = client - api_key = kwargs.get("api_key") - if api_key is None: - api_key = client.api_key - if api_key is not None: - integration.user_api_key = api_key - return + if OPENAI_VERSION >= (1, 0, 0): + if OPENAI_VERSION >= (1, 8, 0): + unwrap(openai._base_client.SyncAPIClient, "_process_response") + unwrap(openai._base_client.AsyncAPIClient, "_process_response") + else: + unwrap(openai._base_client.BaseClient, "_process_response") + unwrap(openai.OpenAI, "__init__") + unwrap(openai.AsyncOpenAI, "__init__") + unwrap(openai.AzureOpenAI, "__init__") + unwrap(openai.AsyncAzureOpenAI, "__init__") + + for resource, method_hook_dict in _RESOURCES.items(): + if deep_getattr(openai.resources, resource) is None: + continue + for method_name, _ in method_hook_dict.items(): + sync_resource = deep_getattr(openai.resources, resource) + async_resource = deep_getattr(openai.resources, ".Async".join(resource.split("."))) + unwrap(sync_resource, method_name) + unwrap(async_resource, method_name) + else: + import openai.api_requestor + + unwrap(openai.api_requestor, "_make_session") + unwrap(openai.util, "convert_to_openai_object") - return patched_client_init + for resource, method_hook_dict in _RESOURCES.items(): + if deep_getattr(openai.api_resources, resource) is None: + continue + for method_name, _ in method_hook_dict.items(): + resource_obj = deep_getattr(openai.api_resources, resource) + unwrap(resource_obj, method_name) + unwrap(resource_obj, "a{}".format(method_name)) + delattr(openai, "_datadog_integration") -def _patched_make_session(func, args, kwargs): + +@with_traced_module +def patched_client_init(openai, pin, func, instance, args, kwargs): + """ + Patch for `openai.OpenAI/AsyncOpenAI` client init methods to add the client object to the OpenAIIntegration object. + """ + func(*args, **kwargs) + integration = openai._datadog_integration + integration._client = instance + api_key = kwargs.get("api_key") + if api_key is None: + api_key = instance.api_key + if api_key is not None: + integration.user_api_key = api_key + return + + +def _patched_make_session(func, instance, args, kwargs): """Patch for `openai.api_requestor._make_session` which sets the service name on the requests session so that spans from the requests integration will use the service name openai. This is done so that the service break down will include OpenAI time spent querying the OpenAI backend. @@ -205,7 +238,7 @@ def _patched_make_session(func, args, kwargs): return session -def _traced_endpoint(endpoint_hook, integration, pin, args, kwargs): +def _traced_endpoint(endpoint_hook, integration, instance, pin, args, kwargs): span = integration.trace(pin, endpoint_hook.OPERATION_ID) openai_api_key = _format_openai_api_key(kwargs.get("api_key")) err = None @@ -214,7 +247,7 @@ def _traced_endpoint(endpoint_hook, integration, pin, args, kwargs): span.set_tag_str("openai.user.api_key", openai_api_key) try: # Start the hook - hook = endpoint_hook().handle_request(pin, integration, span, args, kwargs) + hook = endpoint_hook().handle_request(pin, integration, instance, span, args, kwargs) hook.send(None) resp, err = yield @@ -238,19 +271,11 @@ def _traced_endpoint(endpoint_hook, integration, pin, args, kwargs): integration.metric(span, "dist", "request.duration", span.duration_ns) -def _patched_endpoint(openai, integration, patch_hook): - def patched_endpoint(func, args, kwargs): - # FIXME: this is a temporary workaround for the fact that our bytecode wrapping seems to modify - # a function keyword argument into a cell when it shouldn't. This is only an issue on - # Python 3.11+. - if sys.version_info >= (3, 11) and kwargs.get("encoding_format", None): - kwargs["encoding_format"] = kwargs["encoding_format"].cell_contents - - pin = Pin._find(openai, args[0]) - if not pin or not pin.enabled(): - return func(*args, **kwargs) - - g = _traced_endpoint(patch_hook, integration, pin, args, kwargs) +def _patched_endpoint(openai, patch_hook): + @with_traced_module + def patched_endpoint(openai, pin, func, instance, args, kwargs): + integration = openai._datadog_integration + g = _traced_endpoint(patch_hook, integration, instance, pin, args, kwargs) g.send(None) resp, err = None, None try: @@ -267,22 +292,15 @@ def patched_endpoint(func, args, kwargs): # This return takes priority over `return resp` return e.value # noqa: B012 - return patched_endpoint + return patched_endpoint(openai) -def _patched_endpoint_async(openai, integration, patch_hook): +def _patched_endpoint_async(openai, patch_hook): # Same as _patched_endpoint but async - async def patched_endpoint(func, args, kwargs): - # FIXME: this is a temporary workaround for the fact that our bytecode wrapping seems to modify - # a function keyword argument into a cell when it shouldn't. This is only an issue on - # Python 3.11+. - if sys.version_info >= (3, 11) and kwargs.get("encoding_format", None): - kwargs["encoding_format"] = kwargs["encoding_format"].cell_contents - - pin = Pin._find(openai, args[0]) - if not pin or not pin.enabled(): - return await func(*args, **kwargs) - g = _traced_endpoint(patch_hook, integration, pin, args, kwargs) + @with_traced_module + async def patched_endpoint(openai, pin, func, instance, args, kwargs): + integration = openai._datadog_integration + g = _traced_endpoint(patch_hook, integration, instance, pin, args, kwargs) g.send(None) resp, err = None, None try: @@ -304,59 +322,54 @@ async def patched_endpoint(func, args, kwargs): # This return takes priority over `return resp` return e.value # noqa: B012 - return patched_endpoint + return patched_endpoint(openai) -def _patched_convert(openai, integration): - def patched_convert(func, args, kwargs): - """Patch convert captures header information in the openai response""" - pin = Pin.get_from(openai) - if not pin or not pin.enabled(): - return func(*args, **kwargs) - - span = pin.tracer.current_span() - if not span: - return func(*args, **kwargs) +@with_traced_module +def patched_convert(openai, pin, func, instance, args, kwargs): + """Patch convert captures header information in the openai response""" + integration = openai._datadog_integration + span = pin.tracer.current_span() + if not span: + return func(*args, **kwargs) - if OPENAI_VERSION < (1, 0, 0): - resp = args[0] - if not isinstance(resp, openai.openai_response.OpenAIResponse): - return func(*args, **kwargs) - headers = resp._headers - else: - resp = kwargs.get("response", {}) - headers = resp.headers - # This function is called for each chunk in the stream. - # To prevent needlessly setting the same tags for each chunk, short-circuit here. - if span.get_tag("openai.organization.name") is not None: + if OPENAI_VERSION < (1, 0, 0): + resp = args[0] + if not isinstance(resp, openai.openai_response.OpenAIResponse): return func(*args, **kwargs) - if headers.get("openai-organization"): - org_name = headers.get("openai-organization") - span.set_tag_str("openai.organization.name", org_name) - - # Gauge total rate limit - if headers.get("x-ratelimit-limit-requests"): - v = headers.get("x-ratelimit-limit-requests") - if v is not None: - integration.metric(span, "gauge", "ratelimit.requests", int(v)) - span.set_metric("openai.organization.ratelimit.requests.limit", int(v)) - if headers.get("x-ratelimit-limit-tokens"): - v = headers.get("x-ratelimit-limit-tokens") - if v is not None: - integration.metric(span, "gauge", "ratelimit.tokens", int(v)) - span.set_metric("openai.organization.ratelimit.tokens.limit", int(v)) - # Gauge and set span info for remaining requests and tokens - if headers.get("x-ratelimit-remaining-requests"): - v = headers.get("x-ratelimit-remaining-requests") - if v is not None: - integration.metric(span, "gauge", "ratelimit.remaining.requests", int(v)) - span.set_metric("openai.organization.ratelimit.requests.remaining", int(v)) - if headers.get("x-ratelimit-remaining-tokens"): - v = headers.get("x-ratelimit-remaining-tokens") - if v is not None: - integration.metric(span, "gauge", "ratelimit.remaining.tokens", int(v)) - span.set_metric("openai.organization.ratelimit.tokens.remaining", int(v)) - + headers = resp._headers + else: + resp = kwargs.get("response", {}) + headers = resp.headers + # This function is called for each chunk in the stream. + # To prevent needlessly setting the same tags for each chunk, short-circuit here. + if span.get_tag("openai.organization.name") is not None: return func(*args, **kwargs) - - return patched_convert + if headers.get("openai-organization"): + org_name = headers.get("openai-organization") + span.set_tag_str("openai.organization.name", org_name) + + # Gauge total rate limit + if headers.get("x-ratelimit-limit-requests"): + v = headers.get("x-ratelimit-limit-requests") + if v is not None: + integration.metric(span, "gauge", "ratelimit.requests", int(v)) + span.set_metric("openai.organization.ratelimit.requests.limit", int(v)) + if headers.get("x-ratelimit-limit-tokens"): + v = headers.get("x-ratelimit-limit-tokens") + if v is not None: + integration.metric(span, "gauge", "ratelimit.tokens", int(v)) + span.set_metric("openai.organization.ratelimit.tokens.limit", int(v)) + # Gauge and set span info for remaining requests and tokens + if headers.get("x-ratelimit-remaining-requests"): + v = headers.get("x-ratelimit-remaining-requests") + if v is not None: + integration.metric(span, "gauge", "ratelimit.remaining.requests", int(v)) + span.set_metric("openai.organization.ratelimit.requests.remaining", int(v)) + if headers.get("x-ratelimit-remaining-tokens"): + v = headers.get("x-ratelimit-remaining-tokens") + if v is not None: + integration.metric(span, "gauge", "ratelimit.remaining.tokens", int(v)) + span.set_metric("openai.organization.ratelimit.tokens.remaining", int(v)) + + return func(*args, **kwargs) diff --git a/releasenotes/notes/fix-openai-313-29ec43cbf2f35947.yaml b/releasenotes/notes/fix-openai-313-29ec43cbf2f35947.yaml new file mode 100644 index 00000000000..59d818de9a9 --- /dev/null +++ b/releasenotes/notes/fix-openai-313-29ec43cbf2f35947.yaml @@ -0,0 +1,7 @@ +--- +features: + - | + openai: Introduces tracing support to the OpenAI integration for Python versions 3.12 and 3.13. +fixes: + - | + openai: Fixes a patching issue where asynchronous moderation endpoint calls resulted in coroutine scheduling errors. diff --git a/riotfile.py b/riotfile.py index 5d44c657fb2..f5c6e2e4075 100644 --- a/riotfile.py +++ b/riotfile.py @@ -2475,18 +2475,26 @@ def select_pys(min_version: str = MIN_PYTHON_VERSION, max_version: str = MAX_PYT }, ), Venv( - pys=select_pys(min_version="3.7", max_version="3.11"), + pys="3.7", pkgs={ - "openai[embeddings,datalib]": ["==1.1.1", "==1.30.1"], + "openai[datalib]": "==1.30.1", "pillow": "==9.5.0", }, ), Venv( pys=select_pys(min_version="3.8", max_version="3.11"), pkgs={ - "openai[datalib]": ["==1.30.1"], + "openai[embeddings,datalib]": "==1.30.1", + "pillow": "==9.5.0", + "httpx": "==0.27.2", + }, + ), + Venv( + pys=select_pys(min_version="3.8"), + pkgs={ + "openai": latest, "tiktoken": latest, - "pillow": "==10.1.0", + "pillow": latest, }, env={"TIKTOKEN_AVAILABLE": "True"}, ), diff --git a/tests/contrib/openai/cassettes/v1/completion_stream_wrong_api_key.yaml b/tests/contrib/openai/cassettes/v1/completion_stream_wrong_api_key.yaml new file mode 100644 index 00000000000..512263ce56c --- /dev/null +++ b/tests/contrib/openai/cassettes/v1/completion_stream_wrong_api_key.yaml @@ -0,0 +1,77 @@ +interactions: +- request: + body: '{"model":"text-curie-001","prompt":"how does openai tokenize prompts?","max_tokens":150,"n":1,"stream":true,"temperature":0.8}' + headers: + accept: + - application/json + accept-encoding: + - gzip, deflate + connection: + - keep-alive + content-length: + - '126' + content-type: + - application/json + host: + - api.openai.com + user-agent: + - OpenAI/Python 1.59.7 + x-stainless-arch: + - arm64 + x-stainless-async: + - 'false' + x-stainless-lang: + - python + x-stainless-os: + - MacOS + x-stainless-package-version: + - 1.59.7 + x-stainless-retry-count: + - '0' + x-stainless-runtime: + - CPython + x-stainless-runtime-version: + - 3.13.1 + method: POST + uri: https://api.openai.com/v1/completions + response: + body: + string: "{\n \"error\": {\n \"message\": \"Incorrect API key provided: + sk-wrong****-key. You can find your API key at https://platform.openai.com/account/api-keys.\",\n + \ \"type\": \"invalid_request_error\",\n \"param\": null,\n \"code\": + \"invalid_api_key\"\n }\n}\n" + headers: + CF-Cache-Status: + - DYNAMIC + CF-RAY: + - 9058b3cc3bcdd63c-IAD + Connection: + - keep-alive + Content-Length: + - '266' + Content-Type: + - application/json; charset=utf-8 + Date: + - Tue, 21 Jan 2025 16:32:48 GMT + Server: + - cloudflare + Set-Cookie: + - __cf_bm=WUZdhCkUNTJUEkju8qgk4MKCHL7CFOaIUNvU0L9XmvA-1737477168-1.0.1.1-RJ7MOiDyJEfHrXSN0WQVgZFtkxlkwBL3p.5t3._uu77WPJSM8tYzI3wMHSu.yMwD9QkrbgR5yavkTN.RTWl_1A; + path=/; expires=Tue, 21-Jan-25 17:02:48 GMT; domain=.api.openai.com; HttpOnly; + Secure; SameSite=None + - _cfuvid=7KOfpy1ICNI532AjhDxBh2qtnyNpsjauHeWi6dEJgT4-1737477168271-0.0.1.1-604800000; + path=/; domain=.api.openai.com; HttpOnly; Secure; SameSite=None + X-Content-Type-Options: + - nosniff + alt-svc: + - h3=":443"; ma=86400 + strict-transport-security: + - max-age=31536000; includeSubDomains; preload + vary: + - Origin + x-request-id: + - req_c45bfc7515dca54ef87c667f8210af23 + status: + code: 401 + message: Unauthorized +version: 1 diff --git a/tests/contrib/openai/test_openai_llmobs.py b/tests/contrib/openai/test_openai_llmobs.py index a1a2b93a5ca..71c100abba8 100644 --- a/tests/contrib/openai/test_openai_llmobs.py +++ b/tests/contrib/openai/test_openai_llmobs.py @@ -339,6 +339,10 @@ def test_completion(self, openai, ddtrace_global_config, mock_llmobs_writer, moc ) ) + @pytest.mark.skipif( + parse_version(openai_module.version.VERSION) >= (1, 60), + reason="latest openai versions use modified azure requests", + ) def test_completion_azure( self, openai, azure_openai_config, ddtrace_global_config, mock_llmobs_writer, mock_tracer ): @@ -369,6 +373,10 @@ def test_completion_azure( ) ) + @pytest.mark.skipif( + parse_version(openai_module.version.VERSION) >= (1, 60), + reason="latest openai versions use modified azure requests", + ) async def test_completion_azure_async( self, openai, azure_openai_config, ddtrace_global_config, mock_llmobs_writer, mock_tracer ): @@ -458,6 +466,10 @@ def test_chat_completion(self, openai, ddtrace_global_config, mock_llmobs_writer ) ) + @pytest.mark.skipif( + parse_version(openai_module.version.VERSION) >= (1, 60), + reason="latest openai versions use modified azure requests", + ) def test_chat_completion_azure( self, openai, azure_openai_config, ddtrace_global_config, mock_llmobs_writer, mock_tracer ): @@ -488,6 +500,10 @@ def test_chat_completion_azure( ) ) + @pytest.mark.skipif( + parse_version(openai_module.version.VERSION) >= (1, 60), + reason="latest openai versions use modified azure requests", + ) async def test_chat_completion_azure_async( self, openai, azure_openai_config, ddtrace_global_config, mock_llmobs_writer, mock_tracer ): diff --git a/tests/contrib/openai/test_openai_patch.py b/tests/contrib/openai/test_openai_patch.py index e2fa8cb88c3..6a995213180 100644 --- a/tests/contrib/openai/test_openai_patch.py +++ b/tests/contrib/openai/test_openai_patch.py @@ -3,14 +3,10 @@ # removed the ``_generated`` suffix from the file name, to prevent the content # from being overwritten by future re-generations. -from ddtrace.contrib.openai import get_version -from ddtrace.contrib.openai.patch import patch - - -try: - from ddtrace.contrib.openai.patch import unpatch -except ImportError: - unpatch = None +from ddtrace.contrib.internal.openai.patch import OPENAI_VERSION +from ddtrace.contrib.internal.openai.patch import get_version +from ddtrace.contrib.internal.openai.patch import patch +from ddtrace.contrib.internal.openai.patch import unpatch from tests.contrib.patch import PatchTestCase @@ -22,10 +18,268 @@ class TestOpenaiPatch(PatchTestCase.Base): __get_version__ = get_version def assert_module_patched(self, openai): - pass + if OPENAI_VERSION >= (1, 0, 0): + if OPENAI_VERSION >= (1, 8, 0): + self.assert_wrapped(openai._base_client.SyncAPIClient._process_response) + self.assert_wrapped(openai._base_client.AsyncAPIClient._process_response) + else: + self.assert_wrapped(openai._base_client.BaseClient._process_response) + self.assert_wrapped(openai.OpenAI.__init__) + self.assert_wrapped(openai.AsyncOpenAI.__init__) + self.assert_wrapped(openai.AzureOpenAI.__init__) + self.assert_wrapped(openai.AsyncAzureOpenAI.__init__) + self.assert_wrapped(openai.resources.models.Models.list) + self.assert_wrapped(openai.resources.models.Models.retrieve) + self.assert_wrapped(openai.resources.models.Models.delete) + self.assert_wrapped(openai.resources.models.AsyncModels.list) + self.assert_wrapped(openai.resources.models.AsyncModels.retrieve) + self.assert_wrapped(openai.resources.models.AsyncModels.delete) + self.assert_wrapped(openai.resources.completions.Completions.create) + self.assert_wrapped(openai.resources.chat.Completions.create) + self.assert_wrapped(openai.resources.completions.AsyncCompletions.create) + self.assert_wrapped(openai.resources.chat.AsyncCompletions.create) + self.assert_wrapped(openai.resources.images.Images.generate) + self.assert_wrapped(openai.resources.images.Images.edit) + self.assert_wrapped(openai.resources.images.Images.create_variation) + self.assert_wrapped(openai.resources.images.AsyncImages.generate) + self.assert_wrapped(openai.resources.images.AsyncImages.edit) + self.assert_wrapped(openai.resources.images.AsyncImages.create_variation) + self.assert_wrapped(openai.resources.audio.Transcriptions.create) + self.assert_wrapped(openai.resources.audio.Translations.create) + self.assert_wrapped(openai.resources.audio.AsyncTranscriptions.create) + self.assert_wrapped(openai.resources.audio.AsyncTranslations.create) + self.assert_wrapped(openai.resources.embeddings.Embeddings.create) + self.assert_wrapped(openai.resources.moderations.Moderations.create) + self.assert_wrapped(openai.resources.embeddings.AsyncEmbeddings.create) + self.assert_wrapped(openai.resources.moderations.AsyncModerations.create) + self.assert_wrapped(openai.resources.files.Files.create) + self.assert_wrapped(openai.resources.files.Files.retrieve) + self.assert_wrapped(openai.resources.files.Files.list) + self.assert_wrapped(openai.resources.files.Files.delete) + self.assert_wrapped(openai.resources.files.Files.retrieve_content) + self.assert_wrapped(openai.resources.files.AsyncFiles.create) + self.assert_wrapped(openai.resources.files.AsyncFiles.retrieve) + self.assert_wrapped(openai.resources.files.AsyncFiles.list) + self.assert_wrapped(openai.resources.files.AsyncFiles.delete) + self.assert_wrapped(openai.resources.files.AsyncFiles.retrieve_content) + else: + self.assert_wrapped(openai.api_resources.completion.Completion.create) + self.assert_wrapped(openai.api_resources.completion.Completion.acreate) + self.assert_wrapped(openai.api_requestor._make_session) + self.assert_wrapped(openai.util.convert_to_openai_object) + self.assert_wrapped(openai.api_resources.embedding.Embedding.create) + self.assert_wrapped(openai.api_resources.embedding.Embedding.acreate) + if hasattr(openai, "Model"): + self.assert_wrapped(openai.api_resources.model.Model.list) + self.assert_wrapped(openai.api_resources.model.Model.retrieve) + self.assert_wrapped(openai.api_resources.model.Model.delete) + self.assert_wrapped(openai.api_resources.model.Model.alist) + self.assert_wrapped(openai.api_resources.model.Model.aretrieve) + self.assert_wrapped(openai.api_resources.model.Model.adelete) + if hasattr(openai, "ChatCompletion"): + self.assert_wrapped(openai.api_resources.chat_completion.ChatCompletion.create) + self.assert_wrapped(openai.api_resources.chat_completion.ChatCompletion.acreate) + if hasattr(openai, "Image"): + self.assert_wrapped(openai.api_resources.image.Image.create) + self.assert_wrapped(openai.api_resources.image.Image.acreate) + self.assert_wrapped(openai.api_resources.image.Image.create_edit) + self.assert_wrapped(openai.api_resources.image.Image.acreate_edit) + self.assert_wrapped(openai.api_resources.image.Image.create_variation) + self.assert_wrapped(openai.api_resources.image.Image.acreate_variation) + if hasattr(openai, "Audio"): + self.assert_wrapped(openai.api_resources.audio.Audio.transcribe) + self.assert_wrapped(openai.api_resources.audio.Audio.atranscribe) + self.assert_wrapped(openai.api_resources.audio.Audio.translate) + self.assert_wrapped(openai.api_resources.audio.Audio.atranslate) + if hasattr(openai, "Moderation"): + self.assert_wrapped(openai.api_resources.moderation.Moderation.create) + self.assert_wrapped(openai.api_resources.moderation.Moderation.acreate) + if hasattr(openai, "File"): + self.assert_wrapped(openai.api_resources.file.File.create) + self.assert_wrapped(openai.api_resources.file.File.retrieve) + self.assert_wrapped(openai.api_resources.file.File.list) + self.assert_wrapped(openai.api_resources.file.File.delete) + self.assert_wrapped(openai.api_resources.file.File.download) + self.assert_wrapped(openai.api_resources.file.File.acreate) + self.assert_wrapped(openai.api_resources.file.File.aretrieve) + self.assert_wrapped(openai.api_resources.file.File.alist) + self.assert_wrapped(openai.api_resources.file.File.adelete) + self.assert_wrapped(openai.api_resources.file.File.adownload) def assert_not_module_patched(self, openai): - pass + if OPENAI_VERSION >= (1, 0, 0): + if OPENAI_VERSION >= (1, 8, 0): + self.assert_not_wrapped(openai._base_client.SyncAPIClient._process_response) + self.assert_not_wrapped(openai._base_client.AsyncAPIClient._process_response) + else: + self.assert_not_wrapped(openai._base_client.BaseClient._process_response) + self.assert_not_wrapped(openai.OpenAI.__init__) + self.assert_not_wrapped(openai.AsyncOpenAI.__init__) + self.assert_not_wrapped(openai.AzureOpenAI.__init__) + self.assert_not_wrapped(openai.AsyncAzureOpenAI.__init__) + self.assert_not_wrapped(openai.resources.models.Models.list) + self.assert_not_wrapped(openai.resources.models.Models.retrieve) + self.assert_not_wrapped(openai.resources.models.Models.delete) + self.assert_not_wrapped(openai.resources.models.AsyncModels.list) + self.assert_not_wrapped(openai.resources.models.AsyncModels.retrieve) + self.assert_not_wrapped(openai.resources.models.AsyncModels.delete) + self.assert_not_wrapped(openai.resources.completions.Completions.create) + self.assert_not_wrapped(openai.resources.chat.Completions.create) + self.assert_not_wrapped(openai.resources.completions.AsyncCompletions.create) + self.assert_not_wrapped(openai.resources.chat.AsyncCompletions.create) + self.assert_not_wrapped(openai.resources.images.Images.generate) + self.assert_not_wrapped(openai.resources.images.Images.edit) + self.assert_not_wrapped(openai.resources.images.Images.create_variation) + self.assert_not_wrapped(openai.resources.images.AsyncImages.generate) + self.assert_not_wrapped(openai.resources.images.AsyncImages.edit) + self.assert_not_wrapped(openai.resources.images.AsyncImages.create_variation) + self.assert_not_wrapped(openai.resources.audio.Transcriptions.create) + self.assert_not_wrapped(openai.resources.audio.Translations.create) + self.assert_not_wrapped(openai.resources.audio.AsyncTranscriptions.create) + self.assert_not_wrapped(openai.resources.audio.AsyncTranslations.create) + self.assert_not_wrapped(openai.resources.embeddings.Embeddings.create) + self.assert_not_wrapped(openai.resources.moderations.Moderations.create) + self.assert_not_wrapped(openai.resources.embeddings.AsyncEmbeddings.create) + self.assert_not_wrapped(openai.resources.moderations.AsyncModerations.create) + self.assert_not_wrapped(openai.resources.files.Files.create) + self.assert_not_wrapped(openai.resources.files.Files.retrieve) + self.assert_not_wrapped(openai.resources.files.Files.list) + self.assert_not_wrapped(openai.resources.files.Files.delete) + self.assert_not_wrapped(openai.resources.files.AsyncFiles.retrieve_content) + self.assert_not_wrapped(openai.resources.files.AsyncFiles.create) + self.assert_not_wrapped(openai.resources.files.AsyncFiles.retrieve) + self.assert_not_wrapped(openai.resources.files.AsyncFiles.list) + self.assert_not_wrapped(openai.resources.files.AsyncFiles.delete) + self.assert_not_wrapped(openai.resources.files.AsyncFiles.retrieve_content) + else: + self.assert_not_wrapped(openai.api_resources.completion.Completion.create) + self.assert_not_wrapped(openai.api_resources.completion.Completion.acreate) + self.assert_not_wrapped(openai.api_requestor._make_session) + self.assert_not_wrapped(openai.util.convert_to_openai_object) + self.assert_not_wrapped(openai.api_resources.embedding.Embedding.create) + self.assert_not_wrapped(openai.api_resources.embedding.Embedding.acreate) + if hasattr(openai, "Model"): + self.assert_not_wrapped(openai.api_resources.model.Model.list) + self.assert_not_wrapped(openai.api_resources.model.Model.retrieve) + self.assert_not_wrapped(openai.api_resources.model.Model.delete) + self.assert_not_wrapped(openai.api_resources.model.Model.alist) + self.assert_not_wrapped(openai.api_resources.model.Model.aretrieve) + self.assert_not_wrapped(openai.api_resources.model.Model.adelete) + if hasattr(openai, "ChatCompletion"): + self.assert_not_wrapped(openai.api_resources.chat_completion.ChatCompletion.create) + self.assert_not_wrapped(openai.api_resources.chat_completion.ChatCompletion.acreate) + if hasattr(openai, "Image"): + self.assert_not_wrapped(openai.api_resources.image.Image.create) + self.assert_not_wrapped(openai.api_resources.image.Image.acreate) + self.assert_not_wrapped(openai.api_resources.image.Image.create_edit) + self.assert_not_wrapped(openai.api_resources.image.Image.acreate_edit) + self.assert_not_wrapped(openai.api_resources.image.Image.create_variation) + self.assert_not_wrapped(openai.api_resources.image.Image.acreate_variation) + if hasattr(openai, "Audio"): + self.assert_not_wrapped(openai.api_resources.audio.Audio.transcribe) + self.assert_not_wrapped(openai.api_resources.audio.Audio.atranscribe) + self.assert_not_wrapped(openai.api_resources.audio.Audio.translate) + self.assert_not_wrapped(openai.api_resources.audio.Audio.atranslate) + if hasattr(openai, "Moderation"): + self.assert_not_wrapped(openai.api_resources.moderation.Moderation.create) + self.assert_not_wrapped(openai.api_resources.moderation.Moderation.acreate) + if hasattr(openai, "File"): + self.assert_not_wrapped(openai.api_resources.file.File.create) + self.assert_not_wrapped(openai.api_resources.file.File.retrieve) + self.assert_not_wrapped(openai.api_resources.file.File.list) + self.assert_not_wrapped(openai.api_resources.file.File.delete) + self.assert_not_wrapped(openai.api_resources.file.File.download) + self.assert_not_wrapped(openai.api_resources.file.File.acreate) + self.assert_not_wrapped(openai.api_resources.file.File.aretrieve) + self.assert_not_wrapped(openai.api_resources.file.File.alist) + self.assert_not_wrapped(openai.api_resources.file.File.adelete) + self.assert_not_wrapped(openai.api_resources.file.File.adownload) def assert_not_module_double_patched(self, openai): - pass + if OPENAI_VERSION >= (1, 0, 0): + if OPENAI_VERSION >= (1, 8, 0): + self.assert_not_double_wrapped(openai._base_client.SyncAPIClient._process_response) + self.assert_not_double_wrapped(openai._base_client.AsyncAPIClient._process_response) + else: + self.assert_not_double_wrapped(openai._base_client.BaseClient._process_response) + self.assert_not_double_wrapped(openai.OpenAI.__init__) + self.assert_not_double_wrapped(openai.AsyncOpenAI.__init__) + self.assert_not_double_wrapped(openai.AzureOpenAI.__init__) + self.assert_not_double_wrapped(openai.AsyncAzureOpenAI.__init__) + self.assert_not_double_wrapped(openai.resources.models.Models.list) + self.assert_not_double_wrapped(openai.resources.models.Models.retrieve) + self.assert_not_double_wrapped(openai.resources.models.Models.delete) + self.assert_not_double_wrapped(openai.resources.models.AsyncModels.list) + self.assert_not_double_wrapped(openai.resources.models.AsyncModels.retrieve) + self.assert_not_double_wrapped(openai.resources.models.AsyncModels.delete) + self.assert_not_double_wrapped(openai.resources.completions.Completions.create) + self.assert_not_double_wrapped(openai.resources.chat.Completions.create) + self.assert_not_double_wrapped(openai.resources.completions.AsyncCompletions.create) + self.assert_not_double_wrapped(openai.resources.chat.AsyncCompletions.create) + self.assert_not_double_wrapped(openai.resources.images.Images.generate) + self.assert_not_double_wrapped(openai.resources.images.Images.edit) + self.assert_not_double_wrapped(openai.resources.images.Images.create_variation) + self.assert_not_double_wrapped(openai.resources.images.AsyncImages.generate) + self.assert_not_double_wrapped(openai.resources.images.AsyncImages.edit) + self.assert_not_double_wrapped(openai.resources.images.AsyncImages.create_variation) + self.assert_not_double_wrapped(openai.resources.audio.Transcriptions.create) + self.assert_not_double_wrapped(openai.resources.audio.Translations.create) + self.assert_not_double_wrapped(openai.resources.audio.AsyncTranscriptions.create) + self.assert_not_double_wrapped(openai.resources.audio.AsyncTranslations.create) + self.assert_not_double_wrapped(openai.resources.embeddings.Embeddings.create) + self.assert_not_double_wrapped(openai.resources.moderations.Moderations.create) + self.assert_not_double_wrapped(openai.resources.embeddings.AsyncEmbeddings.create) + self.assert_not_double_wrapped(openai.resources.moderations.AsyncModerations.create) + self.assert_not_double_wrapped(openai.resources.files.Files.create) + self.assert_not_double_wrapped(openai.resources.files.Files.retrieve) + self.assert_not_double_wrapped(openai.resources.files.Files.list) + self.assert_not_double_wrapped(openai.resources.files.Files.delete) + self.assert_not_double_wrapped(openai.resources.files.Files.retrieve_content) + self.assert_not_double_wrapped(openai.resources.files.AsyncFiles.create) + self.assert_not_double_wrapped(openai.resources.files.AsyncFiles.retrieve) + self.assert_not_double_wrapped(openai.resources.files.AsyncFiles.list) + self.assert_not_double_wrapped(openai.resources.files.AsyncFiles.delete) + self.assert_not_double_wrapped(openai.resources.files.AsyncFiles.retrieve_content) + else: + self.assert_not_double_wrapped(openai.api_resources.completion.Completion.create) + self.assert_not_double_wrapped(openai.api_resources.completion.Completion.acreate) + self.assert_not_double_wrapped(openai.api_requestor._make_session) + self.assert_not_double_wrapped(openai.util.convert_to_openai_object) + self.assert_not_double_wrapped(openai.api_resources.embedding.Embedding.create) + self.assert_not_double_wrapped(openai.api_resources.embedding.Embedding.acreate) + if hasattr(openai, "Model"): + self.assert_not_double_wrapped(openai.api_resources.model.Model.list) + self.assert_not_double_wrapped(openai.api_resources.model.Model.retrieve) + self.assert_not_double_wrapped(openai.api_resources.model.Model.delete) + self.assert_not_double_wrapped(openai.api_resources.model.Model.alist) + self.assert_not_double_wrapped(openai.api_resources.model.Model.aretrieve) + self.assert_not_double_wrapped(openai.api_resources.model.Model.adelete) + if hasattr(openai, "ChatCompletion"): + self.assert_not_double_wrapped(openai.api_resources.chat_completion.ChatCompletion.create) + self.assert_not_double_wrapped(openai.api_resources.chat_completion.ChatCompletion.acreate) + if hasattr(openai, "Image"): + self.assert_not_double_wrapped(openai.api_resources.image.Image.create) + self.assert_not_double_wrapped(openai.api_resources.image.Image.acreate) + self.assert_not_double_wrapped(openai.api_resources.image.Image.create_edit) + self.assert_not_double_wrapped(openai.api_resources.image.Image.acreate_edit) + self.assert_not_double_wrapped(openai.api_resources.image.Image.create_variation) + self.assert_not_double_wrapped(openai.api_resources.image.Image.acreate_variation) + if hasattr(openai, "Audio"): + self.assert_not_double_wrapped(openai.api_resources.audio.Audio.transcribe) + self.assert_not_double_wrapped(openai.api_resources.audio.Audio.atranscribe) + self.assert_not_double_wrapped(openai.api_resources.audio.Audio.translate) + self.assert_not_double_wrapped(openai.api_resources.audio.Audio.atranslate) + if hasattr(openai, "Moderation"): + self.assert_not_double_wrapped(openai.api_resources.moderation.Moderation.create) + self.assert_not_double_wrapped(openai.api_resources.moderation.Moderation.acreate) + if hasattr(openai, "File"): + self.assert_not_double_wrapped(openai.api_resources.file.File.create) + self.assert_not_double_wrapped(openai.api_resources.file.File.retrieve) + self.assert_not_double_wrapped(openai.api_resources.file.File.list) + self.assert_not_double_wrapped(openai.api_resources.file.File.delete) + self.assert_not_double_wrapped(openai.api_resources.file.File.download) + self.assert_not_double_wrapped(openai.api_resources.file.File.acreate) + self.assert_not_double_wrapped(openai.api_resources.file.File.aretrieve) + self.assert_not_double_wrapped(openai.api_resources.file.File.alist) + self.assert_not_double_wrapped(openai.api_resources.file.File.adelete) + self.assert_not_double_wrapped(openai.api_resources.file.File.adownload) diff --git a/tests/contrib/openai/test_openai_v0.py b/tests/contrib/openai/test_openai_v0.py index 04654f4a4cf..39bf486f3cb 100644 --- a/tests/contrib/openai/test_openai_v0.py +++ b/tests/contrib/openai/test_openai_v0.py @@ -9,9 +9,7 @@ import pytest import ddtrace -from ddtrace import patch from ddtrace.contrib.internal.openai.utils import _est_tokens -from ddtrace.contrib.trace_utils import iswrapped from ddtrace.internal.utils.version import parse_version from tests.contrib.openai.utils import chat_completion_custom_functions from tests.contrib.openai.utils import chat_completion_input_description @@ -41,42 +39,6 @@ def test_config(ddtrace_config_openai, mock_tracer, openai): assert ddtrace.config.openai.metrics_enabled is ddtrace_config_openai["metrics_enabled"] -def test_patching(openai): - """Ensure that the correct objects are patched and not double patched.""" - - # for some reason these can't be specified as the real python objects... - # no clue why (eg. openai.Completion.create doesn't work) - methods = [ - (openai.Completion, "create"), - (openai.api_resources.completion.Completion, "create"), - (openai.Completion, "acreate"), - (openai.api_resources.completion.Completion, "acreate"), - (openai.api_requestor, "_make_session"), - (openai.util, "convert_to_openai_object"), - (openai.Embedding, "create"), - (openai.Embedding, "acreate"), - ] - if hasattr(openai, "ChatCompletion"): - methods += [ - (openai.ChatCompletion, "create"), - (openai.api_resources.chat_completion.ChatCompletion, "create"), - (openai.ChatCompletion, "acreate"), - (openai.api_resources.chat_completion.ChatCompletion, "acreate"), - ] - - for m in methods: - assert not iswrapped(getattr(m[0], m[1])) - - patch(openai=True) - for m in methods: - assert iswrapped(getattr(m[0], m[1])) - - # Ensure double patching does not occur - patch(openai=True) - for m in methods: - assert not iswrapped(getattr(m[0], m[1]).__dd_wrapped__) - - @pytest.mark.parametrize("api_key_in_env", [True, False]) def test_model_list(api_key_in_env, request_api_key, openai, openai_vcr, mock_metrics, snapshot_tracer): with snapshot_context( diff --git a/tests/contrib/openai/test_openai_v1.py b/tests/contrib/openai/test_openai_v1.py index f13de144fc5..3cacb57651b 100644 --- a/tests/contrib/openai/test_openai_v1.py +++ b/tests/contrib/openai/test_openai_v1.py @@ -5,9 +5,7 @@ import pytest import ddtrace -from ddtrace import patch from ddtrace.contrib.internal.openai.utils import _est_tokens -from ddtrace.contrib.trace_utils import iswrapped from ddtrace.internal.utils.version import parse_version from tests.contrib.openai.utils import chat_completion_custom_functions from tests.contrib.openai.utils import chat_completion_input_description @@ -37,56 +35,6 @@ def test_config(ddtrace_config_openai, mock_tracer, openai): assert ddtrace.config.openai.metrics_enabled is ddtrace_config_openai["metrics_enabled"] -def test_patching(openai): - """Ensure that the correct objects are patched and not double patched.""" - methods = [ - (openai.resources.completions.Completions, "create"), - (openai.resources.completions.AsyncCompletions, "create"), - (openai.resources.chat.Completions, "create"), - (openai.resources.chat.AsyncCompletions, "create"), - (openai.resources.embeddings.Embeddings, "create"), - (openai.resources.embeddings.AsyncEmbeddings, "create"), - (openai.resources.models.Models, "list"), - (openai.resources.models.Models, "retrieve"), - (openai.resources.models.AsyncModels, "list"), - (openai.resources.models.AsyncModels, "retrieve"), - (openai.resources.images.Images, "generate"), - (openai.resources.images.Images, "edit"), - (openai.resources.images.Images, "create_variation"), - (openai.resources.images.AsyncImages, "generate"), - (openai.resources.images.AsyncImages, "edit"), - (openai.resources.images.AsyncImages, "create_variation"), - (openai.resources.audio.Transcriptions, "create"), - (openai.resources.audio.AsyncTranscriptions, "create"), - (openai.resources.audio.Translations, "create"), - (openai.resources.audio.AsyncTranslations, "create"), - (openai.resources.moderations.Moderations, "create"), - (openai.resources.moderations.AsyncModerations, "create"), - (openai.resources.files.Files, "create"), - (openai.resources.files.Files, "retrieve"), - (openai.resources.files.Files, "list"), - (openai.resources.files.Files, "delete"), - (openai.resources.files.Files, "retrieve_content"), - (openai.resources.files.AsyncFiles, "create"), - (openai.resources.files.AsyncFiles, "retrieve"), - (openai.resources.files.AsyncFiles, "list"), - (openai.resources.files.AsyncFiles, "delete"), - (openai.resources.files.AsyncFiles, "retrieve_content"), - ] - - for m in methods: - assert not iswrapped(getattr(m[0], m[1])) - - patch(openai=True) - for m in methods: - assert iswrapped(getattr(m[0], m[1])) - - # Ensure double patching does not occur - patch(openai=True) - for m in methods: - assert not iswrapped(getattr(m[0], m[1]).__dd_wrapped__) - - @pytest.mark.parametrize("api_key_in_env", [True, False]) def test_model_list(api_key_in_env, request_api_key, openai, openai_vcr, mock_metrics, snapshot_tracer): with snapshot_context( @@ -908,17 +856,16 @@ def test_misuse(openai, snapshot_tracer): ) def test_span_finish_on_stream_error(openai, openai_vcr, snapshot_tracer): with openai_vcr.use_cassette("completion_stream_wrong_api_key.yaml"): - with pytest.raises(openai.APIConnectionError): - with pytest.raises(openai.AuthenticationError): - client = openai.OpenAI(api_key="sk-wrong-api-key") - client.completions.create( - model="text-curie-001", - prompt="how does openai tokenize prompts?", - temperature=0.8, - n=1, - max_tokens=150, - stream=True, - ) + with pytest.raises((openai.APIConnectionError, openai.AuthenticationError)): + client = openai.OpenAI(api_key="sk-wrong-api-key") + client.completions.create( + model="text-curie-001", + prompt="how does openai tokenize prompts?", + temperature=0.8, + n=1, + max_tokens=150, + stream=True, + ) def test_completion_stream(openai, openai_vcr, mock_metrics, mock_tracer): @@ -1546,6 +1493,9 @@ def test_est_tokens(): ) # oracle: 92 +@pytest.mark.skipif( + parse_version(openai_module.version.VERSION) >= (1, 60), reason="latest openai versions use modified azure requests" +) @pytest.mark.snapshot( token="tests.contrib.openai.test_openai.test_azure_openai_completion", ignores=["meta.http.useragent", "meta.openai.api_base", "meta.openai.api_type", "meta.openai.api_version"], @@ -1568,6 +1518,9 @@ def test_azure_openai_completion(openai, azure_openai_config, openai_vcr, snapsh ) +@pytest.mark.skipif( + parse_version(openai_module.version.VERSION) >= (1, 60), reason="latest openai versions use modified azure requests" +) @pytest.mark.snapshot( token="tests.contrib.openai.test_openai.test_azure_openai_completion", ignores=[ @@ -1597,6 +1550,9 @@ async def test_azure_openai_acompletion(openai, azure_openai_config, openai_vcr, ) +@pytest.mark.skipif( + parse_version(openai_module.version.VERSION) >= (1, 60), reason="latest openai versions use modified azure requests" +) @pytest.mark.snapshot( token="tests.contrib.openai.test_openai.test_azure_openai_chat_completion", ignores=["meta.http.useragent", "meta.openai.api_base", "meta.openai.api_type", "meta.openai.api_version"], @@ -1619,6 +1575,9 @@ def test_azure_openai_chat_completion(openai, azure_openai_config, openai_vcr, s ) +@pytest.mark.skipif( + parse_version(openai_module.version.VERSION) >= (1, 60), reason="latest openai versions use modified azure requests" +) @pytest.mark.snapshot( token="tests.contrib.openai.test_openai.test_azure_openai_chat_completion", ignores=["meta.http.useragent", "meta.openai.api_base", "meta.openai.api_type", "meta.openai.api_version"], @@ -1641,6 +1600,9 @@ async def test_azure_openai_chat_acompletion(openai, azure_openai_config, openai ) +@pytest.mark.skipif( + parse_version(openai_module.version.VERSION) >= (1, 60), reason="latest openai versions use modified azure requests" +) @pytest.mark.snapshot( token="tests.contrib.openai.test_openai.test_azure_openai_embedding", ignores=["meta.http.useragent", "meta.openai.api_base", "meta.openai.api_type", "meta.openai.api_version"], @@ -1660,6 +1622,9 @@ def test_azure_openai_embedding(openai, azure_openai_config, openai_vcr, snapsho ) +@pytest.mark.skipif( + parse_version(openai_module.version.VERSION) >= (1, 60), reason="latest openai versions use modified azure requests" +) @pytest.mark.snapshot( token="tests.contrib.openai.test_openai.test_azure_openai_embedding", ignores=["meta.http.useragent", "meta.openai.api_base", "meta.openai.api_type", "meta.openai.api_version"],