Looking for Feedback - Adding RAG/Vector DBs to LiteLLM ?

[ishaan-jaff]

A User requested this, starting a discussion to collect feedback if we should look into adding RAG/Vector DB support to LiteLLM

It would be cool if we can do something like this:

response = completion(
    model="ollama/llama2",
    messages=messages,
    api_base="http://localhost:11434/",
    data_source="vector doc id or doc link",
 provider="pinecone/qdrant/chroma"
)

Pros

• Convenience for RAG users

Cons

• LiteLLM would lose focus on being really good for LLM I/O
• You can use langchain + litellm proxy to do something similar today