[please test] BYOK with ollama

[olegklimov]

With the ollama project it's easy to host our own AI models.

You can set up bring-your-own-key (BYOK) to connect to ollama server, and see if you can use StarCoder2 for code completion, llama models for chat.

Does it work at all? What we need to fix to make it better?

[pardeep-singh]

@olegklimov I would like to take this up. Can you please some docs/example of how this can done? Do we need to test the integration here or make changes as well to make it work?

[olegklimov]

Oh, here https://docs.refact.ai/byok/ you can test if we have documentation that is any good :D

[avie66]

Hi @pardeep-singh
Did you had a look over Oleg's approach?