Error in loading 8bit

[zhoustan]

When trying to run the following command:
python mllm/demo/webdemo.py --model_path /path/to/shikra/ckpt --load_in_8bit

Got the value error:
ValueError: A device map needs to be passed to run convert models into mixed-int8 format. Please run.from_pretrained with device_map='auto'

And I can't set "device_map='auto" in the ShikraLlamaForCausalLM.from_pretrained().