Skip to content

Commit

Permalink
Update docs/source/quantization/bnb.rst
Browse files Browse the repository at this point in the history
Co-authored-by: Michael Goin <[email protected]>
  • Loading branch information
thesues and mgoin committed Jul 23, 2024
1 parent 0c720bf commit ad9a9b0
Showing 1 changed file with 1 addition and 2 deletions.
3 changes: 1 addition & 2 deletions docs/source/quantization/bnb.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,10 @@ Below are the steps to utilize BitsAndBytes with vLLM.
vLLM reads the model's config file and supports both in-flight quantization and pre-quantized checkpoint.

Read quantized checkpoint.

You can find bitsandbytes quantized models on https://huggingface.co/models?other=bitsandbytes.
And usually, these repositories have a config.json file that includes a quantization_config section.

Read quantized checkpoint.
--------------------------

.. code-block:: python
Expand Down

0 comments on commit ad9a9b0

Please sign in to comment.