Update of the Development branch

[jdemouth-nvidia]

Hi,

The TensorRT-LLM team is pleased to announce that we are pushing an update to the development branch this October 27th, 2023.

This update includes:

• Fix bugs related to the improper management of the end_id for various models [C++ and Python],
• Fix memory leaks [C++ code and Python models – it should help with issue High CPU memory usage (Llama build Killed) #102,
• Fix the std::alloc error when running the gptManagerBenchmark -- issue gptManagerBenchmark std::bad_alloc error #66,
• Fix a bug in pipeline parallelism when beam-width > 1,
• Fix a bug with Llama GPTQ due to improper support of GQA,
• Fix for issue set max_input_len, max_output_len=4096, But The actual input cannot reach this level #88,
• Fix for issue Huggingface Transformers version should be bumped #16,
• Code improvements + bug fixes for ChatGLM2 [including issues does the chatglm2-6B example support codegeex2-6b's building and running? #93 and benchmark for chatglm2-6b failed #138],
• Add sequence_length tensor to support proper lengths in beam-search (when beam-width > 1),
• Improve support for bad/stop words lists (option to not add special tokens),
• Reporting more runtime statistics related to active requests and KV cache utilization from the batch manager,
• Improve the performance of pipeline parallelism,
• Documentation updates and typo fixes.

The TensorRT-LLM team has also integrated the following Pull Requests and want to thank their contributors.

• Fix link jump in windows readme.md #30 - by https://github.com/yuanlehome
• Fix typo in batchScheduler.h #56 - by https://github.com/eltociear
• Perceived #58 - by https://github.com/RichardScottOZ

Thanks,
The TensorRT-LLM Engineering Team