From 5baeb38bb75357a43222a323e2ee33abf2bcf62b Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=E8=91=A3=E6=99=93=E9=BE=99?= <dongxiaolong@shiyanjia.com>
Date: Fri, 22 Sep 2023 11:15:54 +0800
Subject: [PATCH] vllm docs update

---
 docs/vllm_integration.md | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/docs/vllm_integration.md b/docs/vllm_integration.md
index 1886b1009..021fc3853 100644
--- a/docs/vllm_integration.md
+++ b/docs/vllm_integration.md
@@ -18,3 +18,8 @@ See the supported models [here](https://vllm.readthedocs.io/en/latest/models/sup
    ```
    python3 -m fastchat.serve.vllm_worker --model-path lmsys/vicuna-7b-v1.3 --tokenizer hf-internal-testing/llama-tokenizer
    ```
+
+   if you use a awq model, try
+   '''
+   python3 -m fastchat.serve.vllm_worker --model-path TheBloke/vicuna-7B-v1.5-AWQ --quantization awq
+   '''