vllm docs update

dongxiaolong · Sep 22, 2023 · 5baeb38 · 5baeb38
1 parent c147858
commit 5baeb38
Showing 1 changed file with 5 additions and 0 deletions.
diff --git a/docs/vllm_integration.md b/docs/vllm_integration.md
@@ -18,3 +18,8 @@ See the supported models [here](https://vllm.readthedocs.io/en/latest/models/sup
    ```
    python3 -m fastchat.serve.vllm_worker --model-path lmsys/vicuna-7b-v1.3 --tokenizer hf-internal-testing/llama-tokenizer
    ```
+
+   if you use a awq model, try
+   '''
+   python3 -m fastchat.serve.vllm_worker --model-path TheBloke/vicuna-7B-v1.5-AWQ --quantization awq
+   '''