NoneType object not subscriptable error in quantize.py

[HyungjoonYang]

I'm following the steps in the MLLama example and the TensorRT-LLM multimodal documentation.

When I run the following command:

python tensorrt_llm/examples/quantization/quantize.py \
--model_dir /data/Llama-3.2-90B-Vision-Instruct \                                           
--dtype bfloat16 \                                            
--qformat fp8 \                                            
--output_dir /tmp/trt_models/Llama-3.2-90B-Vision-Instruct/fp8/1-gpu \                                            
--kv_cache_dtype fp8 \                                            
--calib_size 512 \                                            
--calib_dataset scienceqa \

The process runs for a while but eventually fails with the following error:

TypeError: 'NoneType' object is not subscriptable

trying to find model.architecture[0]

Model: LLaMA 3.2-90B-Vision
Precision: bfloat16 → fp8
OS: Ubuntu
GPU: GH200 x 2
CUDA version: 12.6
cuDNN version: 9.7.1.26-1

Please let me know if additional logs or debugging steps are needed.

[nv-guomingz]

Hi @HyungjoonYang , may I know your modelopt version?

[HyungjoonYang]

Hi @HyungjoonYang , may I know your modelopt version?

I am using modelopt version 0.23.0 in docker container.