Support for another models (ollama models)

[testing0mon21]

@b4rtaz Hey, thank you for your wonderful work. Could you please offer some details about how to add supported model? For example, how to to convert some ollama models like command+r or starcoder or llama3 70b to ddlama

https://ollama.com/library/command-r-plus
https://ollama.com/library/llama3:70b
https://ollama.com/library/starcoder2

[b4rtaz]

Hello @testing0mon21,

• starcoder2 uses starcoder2 architecture,
• c4ai-command-r-plus uses cohere architecture.

So from your list only llama3 is supported now.

To convert Llama 3 you have 2 options, you can do it by using Meta files and convert them by using convert-llama.py script, here is the tutorial. The second option is download .safetensor weights from Huggingface and convert it by using convert-hf.py.

[testing0mon21]

Did I understand correctly, for other architectures it will be difficult to implement the same thing that you implemented with llama? @b4rtaz

[b4rtaz]

I think this depends on a specyfic architecture. Some architectures will be easy, some not. Adding new architecture is always non-zero effort. Currently DL supports: llama, mixtral and grok1.