Releases: AniZpZ/AutoSmoothQuant
Releases · AniZpZ/AutoSmoothQuant
Release v0.0.2
Add models: Mixtral and Baichuan 7B.
Optimization: Simplified quant config, all models share the same config format now.
Bug Fix: 1. Fix bugs in quantizing Llama2(when pretrain_tp>1). 2. Fix bugs in Opt Inference.