Using custom embedding endpoint

[cmkeane-agi]

I am trying to get the system to recognize a custom embedding endpoint to use a special embedding model. The system serving it is openai api compliant with the /v1/embeddings path. My full embedding endpoint is: http://embedder.example.com:8000/v1/embeddings.

I have tried in the llamaindex embedding settings to set it to openai and change the OPENAI_API_BASE to http://embedder.example.com:8000. Also tried with adding /v1 and /v1/embeddings. It always timeouts with can't connect. When I look at the embedder log, it shows no attempts.

I have no problem otherwise utilizing this endpoint directly in python, for instance.

Should I be taking another approach?

[szczyglis-dev]

Hmm... a custom endpoint should work with Llama index as long as it is compatible with the OpenAI API.

Have you tried passing the endpoint address as an argument in Embeddings -> Provider **kwargs?

Try providing http://embedder.example.com:8000/v1 here and set a small timeout, for example, 5 seconds (default is 60)

[mayphilc]

would that work to use my local llama 3.2 3b?

[proitservices]

I crafted a simple embeddings replacement using elmo. It's a 1:1 drop in for openAI ada
https://github.com/proitservices/elmo_embedding_api.git

runs in docker, provides 1024 vectors

[szczyglis-dev]

would that work to use my local llama 3.2 3b?

It should work, Llama 3.1 works on my machine, just select the ollama provider in the Indexes -> Embeddings settings and define the model_name keyword argument with the model name.