Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] fix wrong output when using lora and num_scheduler_steps=8 #9689

Closed
wants to merge 654 commits into from
Closed

[BugFix] fix wrong output when using lora and num_scheduler_steps=8 #9689

wants to merge 654 commits into from

Conversation

sleepwalker2017
Copy link

@sleepwalker2017 sleepwalker2017 commented Oct 25, 2024 •
edited by jeejeelee
Loading

FIX issue #9688 #11086

@github-actions GitHub Actions
Copy link

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

  • Add ready label to the PR
  • Enable auto-merge.

🚀

@sleepwalker2017 sleepwalker2017 changed the title bug fix for issue 9688 [BugFix] bug fix for issue 9688 Oct 25, 2024
@sleepwalker2017 sleepwalker2017 changed the title [BugFix] bug fix for issue 9688 [BugFix] fix wrong output when using lora and num_scheduler_steps=8 Oct 29, 2024
@sleepwalker2017 sleepwalker2017 changed the title [BugFix] fix wrong output when using lora and num_scheduler_steps=8 [BugFix] fix wrong output when using lora and num_scheduler_steps=8 Oct 29, 2024
@jeejeelee jeejeelee mentioned this pull request Dec 11, 2024
Open
1 task
@jeejeelee
Copy link
Collaborator

ping @sleepwalker2017

@sleepwalker2017
Copy link
Author

sleepwalker2017 commented Dec 13, 2024 •
edited by jeejeelee
Loading

ping @sleepwalker2017

seems they are too busy to review the PR

@jeejeelee
Copy link
Collaborator

@sleepwalker2017 Thanks for your feedback, I'll look into this PR over the weekend, could you plz sync main firstly?

@jeejeelee jeejeelee self-requested a review December 13, 2024 02:35
WoosukKwon and others added 15 commits December 13, 2024 14:16
@WoosukKwon
[Minor] Fix line-too-long (vllm-project#10563)
b66ff68 
Signed-off-by: Woosuk Kwon <[email protected]>
@youkaichao @njhill
[platforms] absorb worker cls difference into platforms folder (vllm-…
0dbd379 
…project#10555)

Signed-off-by: youkaichao <[email protected]>
Co-authored-by: Nick Hill <[email protected]>
@Isotr0py
[Bugfix] Fix Phi-3 BNB quantization with tensor parallel (vllm-projec…
c86fe94 
…t#9948)

Signed-off-by: Isotr0py <[email protected]>
@noamgat
Remove token-adding chat embedding params (vllm-project#10551)
cd4074a 
Signed-off-by: Noam Gat <[email protected]>
@youkaichao
[bugfix] fix full graph tests (vllm-project#10581)
5fce5dc 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
[torch.compile] support all attention backends (vllm-project#10558)
3a0776c 
Signed-off-by: youkaichao <[email protected]>
@rickyyx @comaniac
[v1] Refactor KVCacheManager for more hash input than token ids (vllm…
85203ae 
…-project#10507)

Signed-off-by: rickyx <[email protected]>
Signed-off-by: Cody Yu <[email protected]>
Co-authored-by: Cody Yu <[email protected]>
@zixuanzhang226
support bitsandbytes quantization with qwen model (vllm-project#10549)
37b4361 
Signed-off-by: Ubuntu <[email protected]>
@russellb
[Core] remove temporary local variables in LLMEngine.__init__ (vllm-p…
5975d75 
…roject#10577)

Signed-off-by: Russell Bryant <[email protected]>
@Abatom
[V1] EngineCore supports profiling (vllm-project#10564)
0a1bc43 
Signed-off-by: Abatom <[email protected]>
@youkaichao
[bugfix] fix cpu tests (vllm-project#10585)
5a6402a 
Signed-off-by: youkaichao <[email protected]>
@tjohnson31415
[Bugfix][Frontend] Update Llama Chat Templates to also support Non-To…
a5bb1f7 
…ol use (vllm-project#10164)

Signed-off-by: Travis Johnson <[email protected]>
@russellb
[Core] Fix broken log configuration (vllm-project#10458)
3ef3d0f 
Signed-off-by: Russell Bryant <[email protected]>
@tlrmchlsmth
[Misc] Add pynccl wrappers for all_gather and reduce_scatter (vllm-pr…
1cb5fb4 
…oject#9432)
@youkaichao
[core] gemma2 full context length support (vllm-project#10584)
d32ce32 
Signed-off-by: youkaichao <[email protected]>
terrytangyuan and others added 18 commits December 13, 2024 14:16
@terrytangyuan
Update link to LlamaStack remote vLLM guide in serving_with_llamastac…
ff19114 
…k.rst (vllm-project#11112)

Signed-off-by: Yuan Tang <[email protected]>
@russellb
[Core] cleanup zmq ipc sockets on exit (vllm-project#11115)
e1de8c5 
Signed-off-by: Russell Bryant <[email protected]>
@pooyadavoodi
[Model] Add support for embedding model GritLM (vllm-project#10816)
7b6b448 
Signed-off-by: Pooya Davoodi <[email protected]>
@WoosukKwon
[V1] Use more persistent buffers to optimize input preparation overhe…
2e703c8 
…ads (vllm-project#11111)

Signed-off-by: Woosuk Kwon <[email protected]>
@SanjuCSudhakaran
[Hardware][Intel-Gaudi] Enable LoRA support for Intel Gaudi (HPU) (vl…
861d6c7 
…lm-project#10565)

Signed-off-by: Sanju C Sudhakaran <[email protected]>
@youkaichao
[core][distributed] initialization from StatelessProcessGroup (vllm-p…
9390b74 
…roject#10986)

Signed-off-by: youkaichao <[email protected]>
@Jeffwan @jeejeelee
[Misc][LoRA] Ensure Lora Adapter requests return adapter name (vllm-p…
0504d5b 
…roject#11094)

Signed-off-by: Jiaxin Shan <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]>
Co-authored-by: Jee Jee Li <[email protected]>
@ywang96
[V1] Fix torch profiling for offline inference (vllm-project#11125)
95f6240 
Signed-off-by: Roger Wang <[email protected]>
@ramonziai
fix(docs): typo in helm install instructions (vllm-project#11141)
f279193 
Signed-off-by: Ramon Ziai <[email protected]>
@sjuxax
[Bugfix] Quick fix to make Pixtral-HF load correctly again after 39e227c
bdd078f 
. (vllm-project#11024)
@comaniac
[Misc] Validate grammar and fail early (vllm-project#11119)
003f4f2
@JArnoldAMD
Fix logging of the vLLM Config (vllm-project#11143)
c93f4bf
@ShangmingCai
[Bugfix] Fix value unpack error of simple connector for KVCache trans…
4f0bdeb 
…fer. (vllm-project#11058)

Signed-off-by: ShangmingCai <[email protected]>
@comaniac
[Misc][V1] Fix type in v1 prefix caching (vllm-project#11151)
1d07ac8
@ProExpertProg
[torch.compile] Dynamic fp8 + rms_norm fusion (vllm-project#10906)
ff20e57 
Signed-off-by: luka <[email protected]>
Co-authored-by: Varun Sundar Rabindranath <[email protected]>
@pooyadavoodi
[Bugfix] Use runner_type instead of task in GritLM (vllm-project#11144)
2833dcf 
Signed-off-by: Pooya Davoodi <[email protected]>
@dsikka
[Bugfix] Update starcoder2 to remap k/v scale names for kv_cache quan…
38f77f0 
…tization (vllm-project#11148)
@gshtras
[ROCm][AMD] Disable auto enabling chunked prefill on ROCm (vllm-proje…
58f71e5 
…ct#11146)

Signed-off-by: Gregory Shtrasberg <[email protected]>
@mergify mergify bot added documentation Improvements or additions to documentation ci/build frontend labels Dec 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@WoosukKwon WoosukKwon Awaiting requested review from WoosukKwon WoosukKwon is a code owner

@zhuohan123 zhuohan123 Awaiting requested review from zhuohan123 zhuohan123 is a code owner

@youkaichao youkaichao Awaiting requested review from youkaichao youkaichao is a code owner

@alexm-neuralmagic alexm-neuralmagic Awaiting requested review from alexm-neuralmagic alexm-neuralmagic is a code owner

@comaniac comaniac Awaiting requested review from comaniac comaniac is a code owner

@njhill njhill Awaiting requested review from njhill njhill is a code owner

@jeejeelee jeejeelee Awaiting requested review from jeejeelee

@mgoin mgoin Awaiting requested review from mgoin mgoin is a code owner

@simon-mo simon-mo Awaiting requested review from simon-mo simon-mo is a code owner

@robertgshaw2-neuralmagic robertgshaw2-neuralmagic Awaiting requested review from robertgshaw2-neuralmagic robertgshaw2-neuralmagic is a code owner

@tlrmchlsmth tlrmchlsmth Awaiting requested review from tlrmchlsmth tlrmchlsmth is a code owner

@LiuXiaoxuanPKU LiuXiaoxuanPKU Awaiting requested review from LiuXiaoxuanPKU LiuXiaoxuanPKU is a code owner

@KuntaiDu KuntaiDu Awaiting requested review from KuntaiDu KuntaiDu is a code owner

@DarkLight1337 DarkLight1337 Awaiting requested review from DarkLight1337 DarkLight1337 is a code owner

@ywang96 ywang96 Awaiting requested review from ywang96 ywang96 is a code owner

Assignees
No one assigned
Labels
ci/build documentation Improvements or additions to documentation frontend
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.