Open issues

[edwinjosegeorge]

Opening a discussion for bugs caused due to returning async types

connected issues are:

• [Bug]: clarifai provider in litellm[proxy] not work  #3862
• [Bug]: TypeError: vars() argument must have __dict__ attribute #3820
• [Bug]: clarifai provider in litellm[proxy] not work #3796
• [Bug]: Nextcloud -> LiteLLM -> Ollama -> async_generator object is not iterable #3357
• [Bug]: REDIS_SSL does not work #2332
• [Bug]: Using v1.15.8 and ollama_chat only working if I add manually "stream": True to the JSON data packet??? #1243

[edwinjosegeorge]

It seems the reason for failure is due to passing coroutines into fastapi, which in turn tries to convert to dict. However, making use of asyncio.run(coroutine) would result in blocking code in high traffic. Does it mean that async code may be handled like steam=True, as execution can be delayed?

[krrishdholakia]

these are different errors - with unrelated code paths (redis_ssl for example has nothing to do with ollama async)

What exactly is the point of this?

[edwinjosegeorge]

It's true that they are at different code path. One thing common is that they are caused in proxy and not in SDK. Sometimes we pass coroutines into fastapi, leading to error. At times coroutines are returned from sync function. But as mentioned in #4102 use of asyncio.run would be blocking call.

The purpose of this discussion is to pool in idea in how to handle async from sync without blocking, as this error is now seen in different issues.

[ishaan-jaff]

closing this discussion - all the issues linked are unrelated and it's unclear what the point of this discussion is