[Bug]: AsyncIO RuntimeError in stream success log when running in gevent loop (from celery)

[TheJKM]

What happened?

I'm making completion calls through LiteLLM SDK in a celery environment running on the gevent runner, so from within a gevent loop. Therefore, it is not possible to run asyncio event loops.
After the streaming has been completed, I get an exception from run_success_logging_and_cache_storage line 1600, which tries to run an asyncio event loop.
I haven't seen any references that asyncio is required to use LiteLLM. So I assume this is a bug. Is there a possibility to disable asnycio in LiteLLM completely?

Relevant log output

[2024-11-26 19:55:31,085: WARNING/MainProcess] Exception in thread 
[2024-11-26 19:55:31,085: WARNING/MainProcess] Thread-123 (run_success_logging_and_cache_storage)
[2024-11-26 19:55:31,085: WARNING/MainProcess] :
[2024-11-26 19:55:31,085: WARNING/MainProcess] Traceback (most recent call last):
[2024-11-26 19:55:31,085: WARNING/MainProcess]   File "/usr/local/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
[2024-11-26 19:55:31,085: WARNING/MainProcess]     
[2024-11-26 19:55:31,085: WARNING/MainProcess] self.run()
[2024-11-26 19:55:31,086: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/threading.py", line 101, in run
[2024-11-26 19:55:31,086: WARNING/MainProcess]     
[2024-11-26 19:55:31,086: WARNING/MainProcess] return _run_old_run_func()
[2024-11-26 19:55:31,086: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/threading.py", line 96, in _run_old_run_func
[2024-11-26 19:55:31,086: WARNING/MainProcess]     
[2024-11-26 19:55:31,086: WARNING/MainProcess] reraise(*_capture_exception())
[2024-11-26 19:55:31,086: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/sentry_sdk/utils.py", line 1723, in reraise
[2024-11-26 19:55:31,086: WARNING/MainProcess]     
[2024-11-26 19:55:31,086: WARNING/MainProcess] raise value
[2024-11-26 19:55:31,086: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/threading.py", line 94, in _run_old_run_func
[2024-11-26 19:55:31,086: WARNING/MainProcess]     
[2024-11-26 19:55:31,086: WARNING/MainProcess] return old_run_func(self, *a, **kw)
[2024-11-26 19:55:31,086: WARNING/MainProcess]   File "/usr/local/lib/python3.10/threading.py", line 953, in run
[2024-11-26 19:55:31,087: WARNING/MainProcess]     
[2024-11-26 19:55:31,087: WARNING/MainProcess] self._target(*self._args, **self._kwargs)
[2024-11-26 19:55:31,087: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 1600, in run_success_logging_and_cache_storage
[2024-11-26 19:55:31,087: WARNING/MainProcess]     
[2024-11-26 19:55:31,087: WARNING/MainProcess] asyncio.run(
[2024-11-26 19:55:31,087: WARNING/MainProcess]   File "/usr/local/lib/python3.10/asyncio/runners.py", line 33, in run
[2024-11-26 19:55:31,087: WARNING/MainProcess]     
[2024-11-26 19:55:31,087: WARNING/MainProcess] raise RuntimeError(
[2024-11-26 19:55:31,087: WARNING/MainProcess] RuntimeError
[2024-11-26 19:55:31,087: WARNING/MainProcess] : 
[2024-11-26 19:55:31,087: WARNING/MainProcess] asyncio.run() cannot be called from a running event loop

Twitter / LinkedIn details

No response