Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upstream merge 24 10 28 #248

Merged
merged 105 commits into from
Oct 29, 2024
Merged

Upstream merge 24 10 28 #248

merged 105 commits into from
Oct 29, 2024

Conversation

gshtras
Copy link
Collaborator

@gshtras gshtras commented Oct 29, 2024

Including the fix for mllama MLP quantization and partially quantized models

njhill and others added 30 commits October 21, 2024 13:28
@njhill
[Frontend] Reduce frequency of client cancellation checking (vllm-pro…
9d9186b 
…ject#7959)
@youkaichao
[doc] fix format (vllm-project#9562)
d621c43
@njhill @qibaoyuan
[BugFix] Update draft model TP size check to allow matching target TP…
15713e3 
… size (vllm-project#9394)

Co-authored-by: Baoyuan Qi <[email protected]>
@wallashss
[Frontend] Don't log duplicate error stacktrace for every request in …
711f3a7 
…the batch (vllm-project#9023)

Signed-off-by: Wallas Santos <[email protected]>
@KuntaiDu
[CI] Make format checker error message more user-friendly by using em…
575dceb 
…oji (vllm-project#9564)

This PR makes format checker error message more user-friendly by adding emojis.
@joerunde
🐛 Fixup more test failures from memory profiling (vllm-project#9563)
ef7faad 
Signed-off-by: Joe Runde <[email protected]>
@youkaichao
[core] move parallel sampling out from vllm core (vllm-project#9302)
76a5e13
@tjohnson31415 @DarkLight1337
[Bugfix]: serialize config by value for --trust-remote-code (vllm-pro…
b729901 
…ject#6751)

Signed-off-by: Travis Johnson <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
@DarkLight1337
[CI/Build] Remove unnecessary fork_new_process (vllm-project#9484)
f085995
@ngrozae
[Bugfix][OpenVINO] fix_dockerfile_openvino (vllm-project#9552)
29acd2c
@Falko1 @Isotr0py
[Bugfix]: phi.py get rope_theta from config file (vllm-project#9503)
7469242 
Co-authored-by: Isotr0py <[email protected]>
@wallashss
[CI/Build] Replaced some models on tests for smaller ones (vllm-proje…
c029221 
…ct#9570)

Signed-off-by: Wallas Santos <[email protected]>
@KuntaiDu
[Core] Remove evictor_v1 (vllm-project#9572)
ca30c3c
@rafvasq
[Doc] Use shell code-blocks and fix section headers (vllm-project#9508)
f7db5f0 
Signed-off-by: Rafael Vasquez <[email protected]>
@chenqianfzh
support TP in qwen2 bnb (vllm-project#9574)
0d02747
@wangshuai09
[Hardware][CPU] using current_platform.is_cpu (vllm-project#9536)
3ddbe25
@WoosukKwon
[V1] Implement vLLM V1 [1/N] (vllm-project#9289)
6c5af09
@jeejeelee
[CI/Build][LoRA] Temporarily fix long context failure issue (vllm-pro…
a48e3ec 
…ject#9579)
@xendo
[Neuron] [Bugfix] Fix neuron startup (vllm-project#9374)
9dbcce8 
Co-authored-by: Jerzy Zagorski <[email protected]>
@Isotr0py
[Model][VLM] Initialize support for Mono-InternVL model (vllm-project…
bb392ea 
…#9528)
@gopalsarda
[Bugfix] Eagle: change config name for fc bias (vllm-project#9580)
08075c3
@zhouyuan @rafvasq @adgubrud
[Hardware][Intel CPU][DOC] Update docs for CPU backend (vllm-project#…
32a1ee7 
…6212)

Signed-off-by: Yuan Zhou <[email protected]>
Co-authored-by: Rafael Vasquez <[email protected]>
Co-authored-by: Gubrud, Aaron D <[email protected]>
Co-authored-by: adgubrud <[email protected]>
@guoyuhong
[Frontend] Support custom request_id from request (vllm-project#9550)
434984e 
Co-authored-by: Yuhong Guo <[email protected]>
@ronensc
[BugFix] Prevent exporting duplicate OpenTelemetry spans (vllm-projec…
cd5601a 
…t#9017)
@youkaichao
[torch.compile] auto infer dynamic_arg_dims from type annotation (vll…
17c79f3 
…m-project#9589)
@aurickq
[Bugfix] fix detokenizer shallow copy (vllm-project#5919)
23b899a
@JArnoldAMD
[Misc] Make benchmarks use EngineArgs (vllm-project#9529)
cb6fdaa
@LucasWilkinson
[Bugfix] Fix spurious "No compiled cutlass_scaled_mm ..." for W8A8 on…
d1e8240 
… Turing (vllm-project#9487)
@yuleil
[BugFix] Fix metrics error for --num-scheduler-steps > 1 (vllm-projec…
b17046e 
…t#8234)
@sethkimmel3 @KaunilD
[Doc]: Update tensorizer docs to include vllm[tensorizer] (vllm-proje…
208cb34 
…ct#7889)

Co-authored-by: Kaunil Dhruv <[email protected]>
bnellnm and others added 27 commits October 27, 2024 09:44
@bnellnm @youkaichao
[Misc] Upgrade to pytorch 2.5 (vllm-project#9588)
3cb07a3 
Signed-off-by: Bill Nell <[email protected]>
Signed-off-by: youkaichao <[email protected]>
Co-authored-by: youkaichao <[email protected]>
@hmellor
Fix cache management in "Close inactive issues and PRs" actions workf…
e130c40 
…low (vllm-project#9734)
@madt2709
[Bugfix] Fix load config when using bools (vllm-project#9533)
34a9941
@wangshuai09
[Hardware][ROCM] using current_platform.is_rocm (vllm-project#9642)
4e2d95e 
Signed-off-by: wangshuai09 <[email protected]>
@youkaichao
[torch.compile] support moe models (vllm-project#9632)
32176fe 
Signed-off-by: youkaichao <[email protected]>
@robertgshaw2-neuralmagic
Fix beam search eos (vllm-project#9627)
feb92fb
@yma11
[Bugfix] Fix ray instance detect issue (vllm-project#9439)
2adb440
@russellb
[CI/Build] Adopt Mergify for auto-labeling PRs (vllm-project#9259)
8b0e4f2 
Signed-off-by: Russell Bryant <[email protected]>
@litianjian @DarkLight1337
[Model][VLM] Add multi-video support for LLaVA-Onevision (vllm-projec…
5f8d807 
…t#8905)

Co-authored-by: litianjian <[email protected]>
Co-authored-by: DarkLight1337 <[email protected]>
@CRZbulabula
Adding "torch compile" annotations to moe models (vllm-project#9758)
aa0addb
@youkaichao
[misc] avoid circular import (vllm-project#9765)
97b61bf 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
[torch.compile] add deepseek v2 compile (vllm-project#9775)
76ed534 
Signed-off-by: youkaichao <[email protected]>
@gshtras
Merge remote-tracking branch 'upstream/main'
b20aa29
@gshtras
Fix for dynamic quantization of the vision part of llama 3.2; Fix for…
c0eb092 
… _scaled_mm of 3 dimensional input tensor
@russellb
[Doc] fix third-party model example (vllm-project#9771)
c5d7fb9 
Signed-off-by: Russell Bryant <[email protected]>
@jeejeelee
[Model][LoRA]LoRA support added for Qwen (vllm-project#9622)
7a4df5f 
Signed-off-by: Jee Jee Li <[email protected]>
@DarkLight1337
[Doc] Specify async engine args in docs (vllm-project#9726)
e74f2d4
@DarkLight1337
[Bugfix] Use temporary directory in registry (vllm-project#9721)
eae3d48
@FerdinandZhong
[Frontend] re-enable multi-modality input in the new beam search impl…
ef7865b 
…ementation (vllm-project#9427)

Signed-off-by: Qishuai [email protected]
@Isotr0py
[Model] Add BNB quantization support for Mllama (vllm-project#9720)
09500f7
@gshtras
Fix support for non quantized visual layers in otherwise quantized ml…
2454f4a 
…lama model, including missing scaling factors

Signed-off-by: Gregory Shtrasberg <[email protected]>
@wangshuai09
[Hardware] using current_platform.seed_everything (vllm-project#9785)
622b7ab 
Signed-off-by: wangshuai09 <[email protected]>
@gshtras
Reorganize imports; Restrict additional supported tensors in _scaled_…
a23a23c 
…mm to 3D; Use constant for default fp8 scale

Signed-off-by: Gregory Shtrasberg <[email protected]>
@gshtras
Merge remote-tracking branch 'upstream/main' into upstream_merge_24_1…
b0a8c5d 
…0_28
@gshtras
Merge remote-tracking branch 'origin/partially_quantized_mllama_fix' …
ab3f100 
…into upstream_merge_24_10_28
@gshtras
Merge remote-tracking branch 'origin/main' into upstream_merge_24_10_28
64f51a5
@gshtras
fix is_hip
cfd7388
@gshtras gshtras merged commit 7aa6982 into main Oct 29, 2024
17 of 18 checks passed
@gshtras gshtras deleted the upstream_merge_24_10_28 branch October 29, 2024 21:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shajrawi shajrawi shajrawi approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.