python3: No module named main

[happybeing]

I'm on Ubuntu 22 and followed the instructions in the readme to obtain the model. I'm specifying python3 because I don't have python aliased but it gives an error trying to run the demo:

$ python3 -m main demo ./mistral-7B-v0.1/
/usr/bin/python3: No module named main

SOLVED:

It took me a while to realise that I need to be in the mistral-src directory when running the above command.

I suggest you mention that in the README for those of us who aren't familiar with the python CLI.

[happybeing]

While the command now runs I get the following error. I've no idea what GPU I have if I have one at all. I'm using a top of the range Dell laptop from about five years ago so will have to check.

RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx

Maybe mention the hardware and driver requirements in the README too.

[ssergorp]

If you want to run any model more than 7B locally, you are going to want at least a mid-range gaming GPU with around 12GB of VRAM. You don't have one, you would know. Nvidia is pretty much the only game in town right now on that front.

There are ways to run models locally with a nice CPU and enough RAM, you'll be either sacrificing speed or complexity in the corpus, but search for Ollama, and try one of the smaller models. Otherwise you can plug into an API. Good luck.