Running the example code from Optimum documentation fails

[salembegic]

Hello,

I was just trying to get the example code from optimum documentation up and running before I try to get an LLM to run on the NPU, but I keep getting this error: ImportError: cannot import name 'AutoQuantizationConfig' from 'optimum.amd.ryzenai'.

I checked the files and I can't find AutoQuantizationConfig class anywhere. I also checked the versions, and I am on the latest one (optimum - 1.17.1, optimum-amd - 0.1.0)

[fxmarty]

Hi @salembegic thank you for the report. Could you try from main?

pip uninstall optimum-amd
pip install git+https://github.com/huggingface/optimum-amd.git

[salembegic]

Yep, that worked. Thanks for the help.

Just a quick question while I have you here. Is process the same if I wanted to run an LLM on the NPU (so export the model to onnx, quantize for the NPU, and then run), or are there any special steps?

[fxmarty]

@salembegic https://github.com/amd/RyzenAI-SW/tree/main/example/transformers dropped last week with an API exposed to PyTorch so it may be easier than going through ONNX + ORT.

We're also working to enable quantization through Brevitas & usage in ONNX but this is WIP.

[salembegic]

Awsome, I'll check it out later today. Once again, thanks for the help😁😁.