Mistral on CPU

[pruthvi1990]

Hi,

I was reading through the quickstart documentation, I see the requirement is to have a GPU with @least 24G of VRAM.

• I want to know is there a way to run Mistral on CPU's?. if so, could you please provide the link to QuickStart documentation for the same ?
• If its currently not supported, is there any future plans to support Mistral on CPU's?

[h3ndrik]

llama.cpp

[Frank-Buss]

This worked for me: https://ollama.ai/library/mistral You need to install ollama first, as described on the ollama.ai homepage (one bash script line for Mac and Linux), then just download and run the model with "ollama run mistral". Needs 8 GB RAM. It generates about 4 tokens/s on my PC (Intel i7-6700K CPU @ 4.00GHz).