Update on the development branch

[kaiyux]

Hi,

The TensorRT-LLM team is pleased to announce that we have pushed an update to the development branch (and the Triton backend) this Dec 17, 2024.

This update includes:

• Features
  • Added FP8 support for the Llama-3.2 VLM model.
  • Added PDL support for userbuffer based AllReduce-Norm fusion kernel.
  • Added runtime support for seamless lookahead decoding.
  • Added token-aligned arbitrary output tensors support for the C++ executor API.
• Bug fixes
  • Fixed Deepseek-V2 model accuracy.
• Known Issues
  • Windows build is broken and the team is working on it.

Thanks,
The TensorRT-LLM Engineering Team