Performance

[agileandy]

Hi, I'm a bit of a LLM noob, just starting to explore what is possible. At the moment I'm running a number of models through Ollama, but was keen to explore Apple native models and came across your repo.

Would it be reasonable of me to assume that an MLX model would run faster/more efficiently on Apple Silicon, than the 'equivalent' Ollama model running on Apple Silicon?

I'm using Msty as a desktop app with models served through Ollama, as it has some cools features, especially for RAG. Do you know if it possible to serve MLX model through Ollama so they can be used in Msty?

[kspviswa]

Ollama runs models based on llamacpp project which is based on PyTorch . Mlx models follows a different architecture. If you wanna use PyOllaMx with Mlx models try using another sister project called PyOMlx.