From ad9a9b00514b0fa3e6bf9969be30ba1e6b728b26 Mon Sep 17 00:00:00 2001 From: dongmao zhang Date: Mon, 22 Jul 2024 10:52:54 -0700 Subject: [PATCH] Update docs/source/quantization/bnb.rst Co-authored-by: Michael Goin --- docs/source/quantization/bnb.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/source/quantization/bnb.rst b/docs/source/quantization/bnb.rst index a5900977fbae4..aefb54a8acb65 100644 --- a/docs/source/quantization/bnb.rst +++ b/docs/source/quantization/bnb.rst @@ -15,11 +15,10 @@ Below are the steps to utilize BitsAndBytes with vLLM. vLLM reads the model's config file and supports both in-flight quantization and pre-quantized checkpoint. -Read quantized checkpoint. - You can find bitsandbytes quantized models on https://huggingface.co/models?other=bitsandbytes. And usually, these repositories have a config.json file that includes a quantization_config section. +Read quantized checkpoint. -------------------------- .. code-block:: python