You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Please add Gemma 2 9b LoRA support. I have tried to build the Tensorrt-llm engine, and when I set the --lora_plugin auto or any other type, it gives me this error
Traceback (most recent call last):
File "/usr/local/bin/trtllm-build", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/commands/build.py", line 602, in main
parallel_build(model_config, ckpt_dir, build_config, args.output_dir,
File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/commands/build.py", line 425, in parallel_build
passed = build_and_save(rank, rank % workers, ckpt_dir,
File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/commands/build.py", line 390, in build_and_save
engine = build_model(build_config,
File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/commands/build.py", line 383, in build_model
return build(model, build_config)
File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/builder.py", line 1096, in build
model = optimize_model_with_config(model, build_config)
File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/builder.py", line 828, in optimize_model_with_config
model.use_lora(build_config.lora_config)
File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/top_model_mixin.py", line 54, in use_lora
raise NotImplementedError("Subclass shall override this")
NotImplementedError: Subclass shall override this
The text was updated successfully, but these errors were encountered:
Please add Gemma 2 9b LoRA support. I have tried to build the Tensorrt-llm engine, and when I set the
--lora_plugin auto
or any other type, it gives me this errorThe text was updated successfully, but these errors were encountered: