-
-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Doc] Improve quickstart documentation #9256
Conversation
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
👋 Hi! Thank you for contributing to the vLLM project. Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can do one of these:
🚀 |
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @rafvasq for improving the quickstart guide. I have a few nits and some questions. I'm new rst formatting as and some of my suggestion are likely just personal preference. Feel free to ignore those :-)
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
Thanks @ckadner, I believe I addressed everything! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good. Thanks @rafvasq
Signed-off-by: Rafael Vasquez <[email protected]>
This looks great! Can you also update the chat/server section to ues https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct so that the users don't need a chat template to start with (advanced feature)? |
Hey @simon-mo, would removing the extra note about chat templates suffice? It doesn't seem to me like a user needs to know anything about chat templates to use the guide with facebook/opt-125m as written since vLLM just uses a default quietly. |
ah that's a good point. however my main problem with the opt model is that just generates gibberish for chat. |
Signed-off-by: Rafael Vasquez <[email protected]>
Makes sense, I've updated the model for those sections |
Signed-off-by: Rafael Vasquez <[email protected]> Signed-off-by: Erkin Sagiroglu <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]> Signed-off-by: Erkin Sagiroglu <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]> Signed-off-by: Shanshan Wang <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]> Signed-off-by: Shanshan Wang <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]> Signed-off-by: qishuai <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]> Signed-off-by: Randall Smith <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]> Signed-off-by: NickLucche <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]> Signed-off-by: NickLucche <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]> Signed-off-by: Linkun Chen <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]> Signed-off-by: Sumit Dubey <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]> Signed-off-by: Maxime Fournioux <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]> Signed-off-by: Tyler Michael Smith <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
This PR tries to improve the existing quickstart guide for newcomers to the project who want to generate something with vLLM quickly.