diff --git a/docs/source/models/lora.rst b/docs/source/models/lora.rst index 934887a607a6a..5cc3076073fbd 100644 --- a/docs/source/models/lora.rst +++ b/docs/source/models/lora.rst @@ -64,7 +64,10 @@ LoRA adapted models can also be served with the Open-AI compatible vLLM server. python -m vllm.entrypoints.openai.api_server \ --model meta-llama/Llama-2-7b-hf \ --enable-lora \ - --lora-modules sql-lora=~/.cache/huggingface/hub/models--yard1--llama-2-7b-sql-lora-test/ + --lora-modules sql-lora=$HOME/.cache/huggingface/hub/models--yard1--llama-2-7b-sql-lora-test/snapshots/0dfa347e8877a4d4ed19ee56c140fa518470028c/ + +.. note:: + The commit ID `0dfa347e8877a4d4ed19ee56c140fa518470028c` may change over time. Please check the latest commit ID in your environment to ensure you are using the correct one. The server entrypoint accepts all other LoRA configuration parameters (``max_loras``, ``max_lora_rank``, ``max_cpu_loras``, etc.), which will apply to all forthcoming requests. Upon querying the ``/models`` endpoint, we should see our LoRA along