Update on the development branch

[kaiyux]

Hi,

The TensorRT-LLM team is pleased to announce that we are pushing an update to the development branch (and the Triton backend) this February 6th, 2024.

This update includes:

• Model Support
  • Skywork model support
• Features
  • More head sizes support for LLaMA-like models
    • Ampere (sm80, sm86), Ada (sm89), Hopper(sm90) all support head sizes [32, 40, 64, 80, 96, 104, 128, 160, 256] now.
• Bug fixes
  • Minor bug fixes
• Documentation
  • Add blog: Speed up inference with SOTA quantization techniques in TRT-LLM
  • A bunch of documentation fixes for the new building workflow

Thanks,
The TensorRT-LLM Engineering Team

[cody-moveworks]

When is the next official release coming out? Very eager to use it :)

[juney-nvidia]

Hi @cody-moveworks ,

The team is working hard with the next official release, the current release plan targets the end of February, but you know for software release it is not surprising to have the surprise:)

BTW, you can always pick up our latest main branch to try with the new features.

Thanks
June