fix(openai): support Python 3.12, 3.13 [backport #12014 to 2.19] #12083
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Resolves #11994. We were previously not testing our OpenAI integration with Python versions >= 3.12 (at the time, OpenAI did not support those newer Python versions). We recently saw that our bytecode wrapping for OpenAI methods broke in Python 3.13, so this PR attempts to address that: - Switch out our bytecode wrapping with `wrapt.wrap_function_wrapper()` to make our patching compatible with newer Python versions. - Add Python 3.12 and 3.13 to tested versions for the OpenAI integration. - Implements unpatching and adds patching tests We previously were using direct bytecode wrapping in the OpenAI integration, which was not supported for Python 3.13, and instead switched to using `wrapt.wrap_function_wrapper()` to wrap OpenAI methods. This meant that we needed to change wrapping formats, including: - use function/attribute names (string) rather than references as inputs to the wrapping functions - replace nested functions with `@with_traced_module()` to pass the traced OpenAI module reference between traced methods - implement unpatching - add patching tests (note this is messy because we support both v0 and v1 openai versions which have separate method names) **Note**: the issue in #11994 was only reproducible via the `AsyncOpenAI.Moderations.create(), AsyncOpenAI.Embeddings.create()` endpoints (chat and completions were not affected). However to avoid any risk introduced by the now unsupported bytecode wrapping, we are changing the entire OpenAI integration to use `wrapt.wrap_function_wrapper()` instead. Since ddtrace 3.0 will be removing support for Python 3.7, we are also dropping support accordingly in this PR. This also coincides with removing support for OpenAI v0 since v0 was last released more than a 16 months ago, and requires a large maintenance burden and pollutes the codebase. Note that this PR will just drop testing support, but a future PR will remove tracing support for v0 in the form of a refactor/cleanup of the OpenAI integration. Azure OpenAI requests are changed in latest versions of OpenAI, which fail our cassette tests (recorded in previous versions of OpenAI). However on manual testing, our openai integration is unchanged in support for newer versions. With consideration for time/effort to produce new cassette test files which is time consuming, we are going to skip cassette tests for Azure endpoints with latest openai versions until we improve our testing framework entirely (move off cassette files). Here are the manual traces submitted with Azure OpenAI for reference: <img width="578" alt="Screenshot 2025-01-23 at 2 20 00 PM" src="https://github.com/user-attachments/assets/19f34edc-4c20-4cfd-804f-3e7bce26f4df" /> <img width="545" alt="Screenshot 2025-01-23 at 2 20 14 PM" src="https://github.com/user-attachments/assets/a58bc888-6bee-4d67-9fda-0661aed14423" /> - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)
|
|
lievan
approved these changes
Jan 24, 2025
Kyle-Verhoog
approved these changes
Jan 24, 2025
BenchmarksBenchmark execution time: 2025-01-24 21:07:37 Comparing candidate commit 9f17a9a in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 394 metrics, 2 unstable metrics. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backports #12014 to 2.19.
Resolves #11994.
We were previously not testing our OpenAI integration with Python versions >= 3.12 (at the time, OpenAI did not support those newer Python versions). We recently saw that our bytecode wrapping for OpenAI methods broke in Python 3.13, so this PR attempts to address that:
wrapt.wrap_function_wrapper()to make our patching compatible with newer Python versions.We previously were using direct bytecode wrapping in the OpenAI integration, which was not supported for Python 3.13, and instead switched to using
wrapt.wrap_function_wrapper()to wrap OpenAI methods. This meant that we needed to change wrapping formats, including:@with_traced_module()to pass the traced OpenAI module reference between traced methodsNote: the issue in #11994 was only reproducible via the
AsyncOpenAI.Moderations.create(), AsyncOpenAI.Embeddings.create()endpoints (chat and completions were not affected). However to avoid any risk introduced by the now unsupported bytecode wrapping, we are changing the entire OpenAI integration to usewrapt.wrap_function_wrapper()instead.Since ddtrace 3.0 will be removing support for Python 3.7, we are also dropping support accordingly in this PR. This also coincides with removing support for OpenAI v0 since v0 was last released more than a 16 months ago, and requires a large maintenance burden and pollutes the codebase. Note that this PR will just drop testing support, but a future PR will remove tracing support for v0 in the form of a refactor/cleanup of the OpenAI integration.
Azure OpenAI requests are changed in latest versions of OpenAI, which fail our cassette tests (recorded in previous versions of OpenAI). However on manual testing, our openai integration is unchanged in support for newer versions. With consideration for time/effort to produce new cassette test files which is time consuming, we are going to skip cassette tests for Azure endpoints with latest openai versions until we improve our testing framework entirely (move off cassette files). Here are the manual traces submitted with Azure OpenAI for reference:
Checklist
Reviewer Checklist