Skip to content

Issues: NVIDIA/TensorRT-LLM

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

324 Open 1,791 Closed
324 Open 1,791 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Failed to build engine with lookahead_decoding bug Something isn't working
#2641 opened Dec 31, 2024 by aikitoria
2 of 4 tasks
greedy search results vary with each inference
#2640 opened Dec 31, 2024 by fclearner
2
Attention clarification
#2639 opened Dec 30, 2024 by Saeedmatt3r
Multi-Modal on TRT-LLM on aarch64 (Holoscan IGX Devkit) fails to covert VILA checkpoints bug Something isn't working
#2638 opened Dec 30, 2024 by MMelQin
2 of 4 tasks
How to make it not display info information?executorExampleBasic.cpp
#2637 opened Dec 28, 2024 by aaIce
1
RuntimeError: Encountered an error when fetching new request: Inplace update to inference tensor outside InferenceMode is not allowed.You can make a clone to get a normal tensor before doing inplace update.
#2636 opened Dec 28, 2024 by anapple-hub
Cpp runner outputs wrong results when using lora + tensor parallelism bug Something isn't working
#2634 opened Dec 28, 2024 by ShuaiShao93
4 tasks
Troubleshoot mistral model bug Something isn't working
#2632 opened Dec 26, 2024 by krishnanpooja
1 of 4 tasks
1
[Performance] KV cache reuse is slower when batch size > 1
#2631 opened Dec 26, 2024 by ReginaZh
Qwen2.5-72B-Instruct YaRN BUG bug Something isn't working
#2630 opened Dec 26, 2024 by PaulX1029
1 of 4 tasks
Error with LoRA Weights Data Type in Quantized TensorRT-LLM Model Execution
#2628 opened Dec 25, 2024 by Alireza3242
[Performance] why do compiled trtllm models have bad performance compared to torch.compile models?
#2627 opened Dec 25, 2024 by FPTMMC
4
[bug] forwardAsync assertion failed: Unable to get batch slot for reqId
#2626 opened Dec 25, 2024 by akhoroshev
Does trtllm-serve support toolparser and guided-decoding? Any plan?
#2624 opened Dec 25, 2024 by dwq370
1
Error during TensorRT-LLM build - Invalid shape and type mismatch in elementwise addition
#2622 opened Dec 24, 2024 by cocovoc
decoder_input_ids must be set to 2 when using fairseq model
#2621 opened Dec 24, 2024 by cocovoc
support for T4 triaged Issue has been triaged by maintainers
#2620 opened Dec 24, 2024 by krishnanpooja
4 tasks
2
[Performance] What is the purpose of compiling a model？ triaged Issue has been triaged by maintainers
#2617 opened Dec 24, 2024 by Flynn-Zh
4 tasks
1
Phi4 support? triaged Issue has been triaged by maintainers
#2616 opened Dec 24, 2024 by oscarbg
1
gather_generation_logits doesn't seem to work correctly for SequenceClassification models Generic Runtime Investigating triaged Issue has been triaged by maintainers
#2615 opened Dec 24, 2024 by TriLoo
Performance of streaming requests is worse than non-streaming bug Something isn't working Investigating Performance Issue about performance number triaged Issue has been triaged by maintainers
#2613 opened Dec 24, 2024 by activezhao
2 of 4 tasks
How to suppress the WARNING logging？
#2610 opened Dec 24, 2024 by lxp3
Adding custom sampling config triaged Issue has been triaged by maintainers
#2609 opened Dec 23, 2024 by buddhapuneeth
1 of 4 tasks
2
Does prefix caching support VLM models?
#2608 opened Dec 23, 2024 by sleepwalker2017
Gemma 2 LoRA support Investigating Lora/P-tuning triaged Issue has been triaged by maintainers
#2606 opened Dec 21, 2024 by Aquasar11
@byshiue
ProTip! Type g i on any issue or pull request to go back to the issue listing page.