Update on the development branch

[kaiyux]

Hi,

The TensorRT-LLM team is pleased to announce that we are pushing an update to the development branch (and the Triton backend) this April 9th, 2024.

This update includes:

• Model Support
  • DBRX model support, thanks to the contribution from @megha95 in Support for DBRX #1363
• Bug fixes
  • Fix some unexpected behaviors in beam search and early stopping, so that the outputs are more accurate
• Benchmark
  • Enable streaming and support “Time To the First Token (TTFT)” latency and “Inter-Token Latency (ITL)” metrics

Thanks,
The TensorRT-LLM Engineering Team