-
Notifications
You must be signed in to change notification settings - Fork 215
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: litellm conflict with openai on lib override #480
base: main
Are you sure you want to change the base?
Conversation
Removed original_oai_create and original_oai_create_async variables since we no longer override OpenAI's methods Modified override() and undo_override() to only handle LiteLLM's methods Updated _override_completion() and _override_async_completion() to only store and patch LiteLLM's methods This way, when both providers are used: OpenAIProvider will handle overriding OpenAI's completion methods LiteLLMProvider will only handle overriding LiteLLM's completion methods No more conflicts between the two providers
Signed-off-by: Teo <[email protected]>
I ran this in
|
@the-praxs Can you give it a look? |
Testing it - same issue as yesterday - only the first I am trying to troubleshoot the root cause. |
closing in favor of #476 |
If we want to avoid client conflicts, perhaps we can instead use their official observability documentation https://docs.litellm.ai/docs/observability/arize_integration |
Since the other PR was closed by the author will consider this as "Resolves #471 " |
Fixes #471
original_oai_create
andoriginal_oai_create_async
variables since we no longer override OpenAI's methodsoverride()
andundo_override()
to only handle LiteLLM's methods_override_completion()
and_override_async_completion()
to only store and patch LiteLLM's methodsThis way, when both providers are used:
A bit more of explanation
LiteLLM's completion method is completely separate from OpenAI's
Completions.create
Even though LiteLLM uses OpenAI's format internally, it has its own implementation
When we call
litellm.completion()
, it doesn't actually call OpenAI'sCompletions.create
Tests
Breakdown
test_provider_override_independence()
:test_provider_override_order_independence()
:What's being verified