IBM watsonx.ai integration

[bernong]

Describe the feature or potential improvement

Access to integration to IBM watsonx.ai to access Granite models which comes with indemnification (also Apache 2.0 license). Helps with building agents for regulatory industries especially financial services. watsonx also comes with other open-weight/source models - Llama and Mistral

Additional information

No response

[marcklingen]

Thanks for raising this. I hoped that IBM would just implement the openai api schema but it seems like they use a bespoke one.

Have you considered using ibm watsonx via a proxy such as LiteLLM?