v0.20.0rc1

[kaiyux]

Highlights

• Features
  • PyTorch workflow
    • Added LoRA support. (Add running E2E LoRA flow #3648) (add passing E2E LoRA flow #3788)
    • Added return logits support. (feat: Return logits in PyTorch flow #3221)
  • Part 1 of large-scale EP: Added MNNVL MoE A2A support. (feat: large-scale EP(part 1: Add MNNVL MoE A2A support) #3504)
  • Added smart router for the MoE module. (feat: Add smart router for moe module #3641)
  • Added head size 72 support for QKV preprocessing kernel. (feat: Add head size 72 support for QKV Preprocessing kernel #3743)

What's Changed

• move pytorch tests of LLM API into separate test files by @QiJune in move pytorch tests of LLM API into separate test files #3745
• Fix double link to fp8_blockscale_gemm_src by @WilliamTambellini in Fix double link to fp8_blockscale_gemm_src #3707
• feat: add QMMA-based MLA kernels by @PerkzZheng in feat: add QMMA-based MLA kernels #3752
• chore: add pull request template by @byshiue in chore: add pull request template #3760
• Add running E2E LoRA flow by @shaharmor98 in Add running E2E LoRA flow #3648
• [infra] Waive L0 tests by @yiqingy0 in [infra] Waive L0 tests #3784
• feat: Add smart router for moe module by @zongfeijing in feat: Add smart router for moe module #3641
• test: add rcca tests 4753548 by @xinhe-nv in test: add rcca tests 4753548 #3716
• fix: nvbugs/5234029 fix Qwen2.5-VL image test by @yechank-nvidia in fix: nvbugs/5234029 fix Qwen2.5-VL image test #3726
• test: [CI] Add failed cases into waives.txt by @xinhe-nv in test: [CI] Add failed cases into waives.txt #3696
• fix: Intercept the error of multi-ranks bound to a single device by @Shixiaowei02 in fix: Intercept the error of multi-ranks bound to a single device #3525
• fix: remove the unnecessary metadata changes in mtp by @lfr-0531 in fix: remove the unnecessary metadata changes in mtp #3787
• test: Add DeepSeek-V3-Lite GSM8K tests by @syuoni in test: Add DeepSeek-V3-Lite GSM8K tests #3771
• infra: [TRTLLM-4417]Support auto trigger special test stage for special file change by @ZhanruiSunCh in infra: [TRTLLM-4417]Support auto trigger special test stage for special file change #3478
• [TRTLLM-4763][test] Accuracy test improvement (Part 3.6): Deprecate mmlu_llmapi.py by @syuoni in [TRTLLM-4763][test] Accuracy test improvement (Part 3.6): Deprecate mmlu_llmapi.py #3802
• add passing E2E LoRA flow by @shaharmor98 in add passing E2E LoRA flow #3788
• fix: Limit llama4 context length to 8k by @mikeiovine in fix: Limit llama4 context length to 8k #3778
• fix: Fix C++ decoder synchronization in PyTorch by @dcampora in fix: Fix C++ decoder synchronization in PyTorch #3106
• fix: 5197419 and removed unused runtime kernels by @hypdeb in fix: 5197419 and removed unused runtime kernels #3631
• chore: reorganize some unit tests of PyTorch by @QiJune in chore: reorganize some unit tests of PyTorch #3780
• doc: fix path after examples migration by @kaiyux in doc: fix path after examples migration #3814
• chore: fix some invalid paths of contrib models by @QiJune in chore: fix some invalid paths of contrib models #3818
• chore: Fix KV cache block reuse flag name in quickstart_advanced by @mikeiovine in chore: Fix KV cache block reuse flag name in quickstart_advanced #3781
• Fix create_weights in attention by @hlu1 in Fix create_weights in attention #3692
• test: [CI] Add failed cases into waives.txt by @xinhe-nv in test: [CI] Add failed cases into waives.txt #3777
• [https://nvbugspro.nvidia.com/bug/5238602][fix] Package lm_eval configuration files by @syuoni in [https://nvbugspro.nvidia.com/bug/5238602][fix] Package lm_eval configuration files #3809
• [https://nvbugspro.nvidia.com/bug/5238599][fix] Normalize example path in accuracy tests by @syuoni in [https://nvbugspro.nvidia.com/bug/5238599][fix] Normalize example path in accuracy tests #3805
• fix: Set default prompts and media for multimodal quickstart example by @qixiang-99 in fix: Set default prompts and media for multimodal quickstart example #3792
• Fix: Revert commit 25f9669 by @Shixiaowei02 in Fix: Revert commit 25f9669 #3832
• chore: bump version to 0.20.0rc1 by @ZhanruiSunCh in chore: bump version to 0.20.0rc1 #3834
• feat(part 2): Enhance the integrated robustness of scaffolding with init_.py Enhance the integrated robustness of scaffolding with __init__.py #3305 by @WeiHaocheng in feat(part 2): Enhance the integrated robustness of scaffolding with _init__.py #3305 #3731
• fix: fix lora case failure by @HuiGao-NV in fix: fix lora case failure #3838
• Added NemotronH to PyTorch supported models by @vegaluisjose in Added NemotronH to PyTorch supported models #3663
• Adding local paths to the datasets to make them loadable in offline mode by @rakib-hasan in Adding local paths to the datasets to make them loadable in offline mode #3750
• fix: [Deepseek] Pass hidden_states_fp4 to shared_experts by @hlu1 in fix: [Deepseek] Pass hidden_states_fp4 to shared_experts #3819
• chore: increase A30 for cpp test by @QiJune in chore: increase A30 for cpp test #3811
• feat: Return logits in PyTorch flow by @tongyuantongyu in feat: Return logits in PyTorch flow #3221
• feat: large-scale EP(part 1: Add MNNVL MoE A2A support) by @dongxuy04 in feat: large-scale EP(part 1: Add MNNVL MoE A2A support) #3504
• [infra] Waive L0 tests by @yiqingy0 in [infra] Waive L0 tests #3853
• [chore] Add Llama 4 Maverick to quickstart README by @mikeiovine in [chore] Add Llama 4 Maverick to quickstart README #3848
• fix:[AutoDeploy] Patch for torch load_state_dict() by @sugunav14 in fix:[AutoDeploy] Patch for torch load_state_dict() #3847
• feat: Add head size 72 support for QKV Preprocessing kernel by @qixiang-99 in feat: Add head size 72 support for QKV Preprocessing kernel #3743
• chore: update pytorch only change file list by @QiJune in chore: update pytorch only change file list #3873
• Test: Split C++ unit tests for CI granularity by @DomBrown in Test: Split C++ unit tests for CI granularity #3868
• TRTLLM-4875 feat: Add version switcher to doc by @kaiyux in TRTLLM-4875 feat: Add version switcher to doc #3846

Full Changelog: v0.20.0rc0...v0.20.0rc1

---

This discussion was created from the release v0.20.0rc1.