Google Gemini support?

[jaygz316]

Is there any chance there might be support for Gemini in the future? I have come to really like it and I already have my stuff in one dashboard, just wondering if there was a chance.

[kamtschatka]

probably makes sense to switch to something like https://github.com/themaximalist/llm.js or https://github.com/samestrin/llm-interface, since everyone is thinking that their LLM is better than the other anyway and we'll not add support for every single one of them.

[hobybrenner]

Also, supporting this might be an additional solution. The AI Gateway streamlines requests to 250+ language, vision, audio and image models with a unified API.
https://github.com/Portkey-AI/gateway/

[Inrego]

Another reason to support Gemini is that it has a pretty generous free tier (free as in you pay with data instead of money)

[MohamedBassem]

did anyone try this: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/call-vertex-using-openai-library

Seems like gemini has openai compatible API. You just need to set the correct base url and it'll work.

[lucius346346]

I have problem with setting this up? Should it be:
OPENAI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash-latest:generateContent
OPENAI_API_KEY={api key}

[sebp]

I got it working using

OPENAI_BASE_URL="https://generativelanguage.googleapis.com/v1beta/"
OPENAI_API_KEY="XXXXXXXX"
INFERENCE_TEXT_MODEL="gemini-1.5-flash-8b"
INFERENCE_IMAGE_MODEL="gemini-1.5-flash-8b"

In addition, I had to add the following tagging rule under User Settings → AI Settings

You must respond in JSON using this JSON schema:
     Tags = {"tags": list[str]}
     Return: Tags

[lucius346346]

I got it working using

OPENAI_BASE_URL="https://generativelanguage.googleapis.com/v1beta/"
OPENAI_API_KEY="XXXXXXXX"
INFERENCE_TEXT_MODEL="gemini-1.5-flash-8b"
INFERENCE_IMAGE_MODEL="gemini-1.5-flash-8b"

In addition, I had to add the following tagging rule under User Settings → AI Settings

You must respond in JSON using this JSON schema:
     Tags = {"tags": list[str]}
     Return: Tags

Awesome! Thanks a lot!

[jbohnslav]

If you self host a litellm proxy, you should be able to use whatever LLM you want.