Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Doc] Improve quickstart documentation #9256

Merged
merged 13 commits into from
Oct 25, 2024

Conversation

rafvasq
Copy link
Contributor

@rafvasq rafvasq commented Oct 10, 2024

This PR tries to improve the existing quickstart guide for newcomers to the project who want to generate something with vLLM quickly.

  • Include pre-req system requirements
  • Include installation step
  • Reorganize sections
  • Add links to help with LLM terminology
  • Improve step-by-step guide readability
  • Use Qwen2.5-1.5B-Instruct for chat/server sections

Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
Copy link

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

  • Add ready label to the PR
  • Enable auto-merge.

🚀

@rafvasq rafvasq marked this pull request as ready for review October 10, 2024 17:25
@rafvasq rafvasq marked this pull request as draft October 10, 2024 17:26
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
@rafvasq rafvasq marked this pull request as ready for review October 10, 2024 18:11
Copy link

@ckadner ckadner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @rafvasq for improving the quickstart guide. I have a few nits and some questions. I'm new rst formatting as and some of my suggestion are likely just personal preference. Feel free to ignore those :-)

docs/source/getting_started/quickstart.rst Outdated Show resolved Hide resolved
docs/source/getting_started/quickstart.rst Show resolved Hide resolved
docs/source/getting_started/quickstart.rst Outdated Show resolved Hide resolved
docs/source/getting_started/quickstart.rst Outdated Show resolved Hide resolved
docs/source/getting_started/quickstart.rst Outdated Show resolved Hide resolved
docs/source/getting_started/quickstart.rst Outdated Show resolved Hide resolved
docs/source/getting_started/quickstart.rst Outdated Show resolved Hide resolved
docs/source/getting_started/quickstart.rst Outdated Show resolved Hide resolved
docs/source/getting_started/quickstart.rst Outdated Show resolved Hide resolved
docs/source/getting_started/quickstart.rst Outdated Show resolved Hide resolved
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
@rafvasq rafvasq requested a review from ckadner October 16, 2024 20:43
@rafvasq
Copy link
Contributor Author

rafvasq commented Oct 16, 2024

Thanks @ckadner, I believe I addressed everything!

Copy link

@ckadner ckadner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Thanks @rafvasq

Signed-off-by: Rafael Vasquez <[email protected]>
@simon-mo
Copy link
Collaborator

This looks great! Can you also update the chat/server section to ues https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct so that the users don't need a chat template to start with (advanced feature)?

@rafvasq
Copy link
Contributor Author

rafvasq commented Oct 23, 2024

! Can you also update the chat/server section to ues https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct so that the users don't need a chat template to start with (advanced feature)?

Hey @simon-mo, would removing the extra note about chat templates suffice? It doesn't seem to me like a user needs to know anything about chat templates to use the guide with facebook/opt-125m as written since vLLM just uses a default quietly.

@simon-mo
Copy link
Collaborator

ah that's a good point. however my main problem with the opt model is that just generates gibberish for chat.

Signed-off-by: Rafael Vasquez <[email protected]>
@rafvasq
Copy link
Contributor Author

rafvasq commented Oct 25, 2024

Makes sense, I've updated the model for those sections

@simon-mo simon-mo merged commit 228cfbd into vllm-project:main Oct 25, 2024
27 of 30 checks passed
@rafvasq rafvasq deleted the update-quickstart-doc branch October 25, 2024 21:38
MErkinSag pushed a commit to MErkinSag/vllm that referenced this pull request Oct 26, 2024
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Erkin Sagiroglu <[email protected]>
MErkinSag pushed a commit to MErkinSag/vllm that referenced this pull request Oct 26, 2024
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Erkin Sagiroglu <[email protected]>
cooleel pushed a commit to cooleel/vllm that referenced this pull request Oct 28, 2024
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Shanshan Wang <[email protected]>
cooleel pushed a commit to cooleel/vllm that referenced this pull request Oct 28, 2024
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Shanshan Wang <[email protected]>
FerdinandZhong pushed a commit to FerdinandZhong/vllm that referenced this pull request Oct 29, 2024
rasmith pushed a commit to rasmith/vllm that referenced this pull request Oct 30, 2024
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Randall Smith <[email protected]>
NickLucche pushed a commit to NickLucche/vllm that referenced this pull request Oct 31, 2024
NickLucche pushed a commit to NickLucche/vllm that referenced this pull request Oct 31, 2024
lk-chen pushed a commit to lk-chen/vllm that referenced this pull request Nov 4, 2024
lk-chen pushed a commit to lk-chen/vllm that referenced this pull request Nov 4, 2024
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Linkun Chen <[email protected]>
sumitd2 pushed a commit to sumitd2/vllm that referenced this pull request Nov 14, 2024
KuntaiDu pushed a commit to KuntaiDu/vllm that referenced this pull request Nov 20, 2024
mfournioux pushed a commit to mfournioux/vllm that referenced this pull request Nov 20, 2024
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Maxime Fournioux <[email protected]>
tlrmchlsmth pushed a commit to neuralmagic/vllm that referenced this pull request Nov 23, 2024
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Tyler Michael Smith <[email protected]>
sleepwalker2017 pushed a commit to sleepwalker2017/vllm that referenced this pull request Dec 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants