Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upstream merge 24/09/16 #187

Merged
merged 83 commits into from
Sep 16, 2024
Merged

Upstream merge 24/09/16 #187

merged 83 commits into from
Sep 16, 2024

Conversation

gshtras
Copy link
Collaborator

@gshtras gshtras commented Sep 16, 2024

Catch up with 0.6.1_post2

kylesayrs and others added 30 commits September 9, 2024 16:27
@kylesayrs
[Misc] GPTQ Activation Ordering (vllm-project#8135)
c7cb5c3
@dsikka
[Misc] Fused MoE Marlin support for GPTQ (vllm-project#8217)
6cd5e5b
@simon-mo
Add NVIDIA Meetup slides, announce AMD meetup, and add contact info (v…
a1d8742 
…llm-project#8319)
@DarkLight1337
[Bugfix] Fix missing post_layernorm in CLIP (vllm-project#8155)
da1a844
@dtrifiro
[CI/Build] enable ccache/scccache for HIP builds (vllm-project#8327)
6234385
@DarkLight1337
[Frontend] Clean up type annotations for mistral tokenizer (vllm-proj…
8c054b7 
…ect#8314)
@alexeykondrat
[CI/Build] Enabling kernels tests for AMD, ignoring some of then that…
f421f3c 
… fail (vllm-project#8130)
@sumitd2
Fix ppc64le buildkite job (vllm-project#8309)
02751a7
@kevin314
[Spec Decode] Move ops.advance_step to flash attn advance_step (vllm-…
5faedf1 
…project#8224)
@prashantgupta24
[Misc] remove peft as dependency for prompt models (vllm-project#8162)
04e7c4e
@comaniac
[MISC] Keep chunked prefill enabled by default with long context when…
b1f3e18 
… prefix caching is enabled (vllm-project#8342)
@alexm-neuralmagic
[Bugfix] lookahead block table with cuda graph max capture (vllm-proj…
22f3a4b 
…ect#8340)

[Bugfix] Ensure multistep lookahead allocation is compatible with cuda graph max capture (vllm-project#8340)
@SolitaryThinker
[Core/Bugfix] pass VLLM_ATTENTION_BACKEND to ray workers (vllm-projec…
1d5e397 
…t#8172)
@tlrmchlsmth
[CI/Build][Kernel] Update CUTLASS to 3.5.1 tag (vllm-project#8043)
94144e7
@jeejeelee
[Misc] Skip loading extra bias for Qwen2-MOE GPTQ models (vllm-projec…
e497b8a 
…t#8329)
@Isotr0py
[Bugfix] Fix InternVL2 vision embeddings process with pipeline parall…
1230263 
…el (vllm-project#8299)
@pavanimajety
[Hardware][NV] Add support for ModelOpt static scaling checkpoints. (v…
efcf946 
…llm-project#6112)
@TKONIY @ywang96 @DarkLight1337
[model] Support for Llava-Next-Video model (vllm-project#7559)
6a512a0 
Co-authored-by: Roger Wang <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
@pooyadavoodi
[Frontend] Create ErrorResponse instead of raising exceptions in run_…
cea95df 
…batch (vllm-project#8347)
@fyabc @ywang96 @DarkLight1337
[Model][VLM] Add Qwen2-VL model support (vllm-project#7905)
3b7fea7 
Co-authored-by: Roger Wang <[email protected]>
Co-authored-by: DarkLight1337 <[email protected]>
@bigPYJ1151
[Hardware][Intel] Support compressed-tensor W8A8 for CPU backend (vll…
0b952af 
…m-project#7257)
@alexeykondrat
[CI/Build] Excluding test_moe.py from AMD Kernels tests for investiga…
aea02f3 
…tion (vllm-project#8373)
@DarkLight1337
[Bugfix] Add missing attributes in mistral tokenizer (vllm-project#8364)
7015417
@bnellnm @SageMoore
[Kernel][Misc] register ops to prevent graph breaks (vllm-project#6917)
73202db 
Co-authored-by: Sage Moore <[email protected]>
@akx
[Misc] Move device options to a single place (vllm-project#8322)
8baa454
@LiuXiaoxuanPKU @youkaichao
[Speculative Decoding] Test refactor (vllm-project#8317)
775f00f 
Co-authored-by: youkaichao <[email protected]>
@patrickvonplaten @ywang96
Pixtral (vllm-project#8377)
d394787 
Co-authored-by: Roger Wang <[email protected]>
@simon-mo
Bump version to v0.6.1 (vllm-project#8379)
3fd2b0d
@comaniac
[MISC] Dump model runner inputs when crashing (vllm-project#8305)
a65cb16
@youkaichao
[misc] remove engine_use_ray (vllm-project#8126)
f842a7a
jeejeelee and others added 21 commits September 13, 2024 07:58
@jeejeelee
[Misc] Skip loading extra bias for Qwen2-VL GPTQ-Int8 (vllm-project#8442
06311e2 
)
@youkaichao
[misc][ci] fix quant test (vllm-project#8449)
a246912
@DarkLight1337
[Installation] Gate FastAPI version for Python 3.8 (vllm-project#8456)
ecd7a1d
@youkaichao
[plugin][torch.compile] allow to add custom compile backend (vllm-pro…
0a4806f 
…ject#8445)
@DarkLight1337
[CI/Build] Reorganize models tests (vllm-project#7820)
a84e598
@Isotr0py
[Doc] Add oneDNN installation to CPU backend documentation (vllm-proj…
f57092c 
…ect#8467)
@njhill
[HotFix] Fix final output truncation with stop string + streaming (vl…
18e9e1f 
…lm-project#8468)
@simon-mo
bump version to v0.6.1.post2 (vllm-project#8473)
9ba0817
@SolitaryThinker
[bugfix] add multi-step advance_step to ROCmFlashAttentionMetadata
daddc14
@SolitaryThinker
add rocm to MULTI_STEP_ATTENTION_BACKENDS
306f21f
@jikunshang @yma11
[Hardware][intel GPU] bump up ipex version to 2.3 (vllm-project#8365)
8517252 
Co-authored-by: Yan Ma <[email protected]>
@charlifu
[Kernel][Hardware][Amd]Custom paged attention kernel for rocm (vllm-p…
1ef0d2e 
…roject#8310)
@SUDA-HLT-ywfang @DarkLight1337
[Model] support minicpm3 (vllm-project#8297)
8a0cf1d 
Co-authored-by: DarkLight1337 <[email protected]>
@youkaichao
[torch.compile] fix functionalization (vllm-project#8480)
a36e070
@youkaichao
[torch.compile] add a flag to disable custom op (vllm-project#8488)
47790f3
@WoosukKwon
[TPU] Implement multi-step scheduling (vllm-project#8489)
50e9ec4
@chrisociepa
[Bugfix][Model] Fix Python 3.8 compatibility in Pixtral model by upda…
3724d5f 
…ting type annotations (vllm-project#8490)
@Isotr0py
[Bugfix][Kernel] Add IQ1_M quantization implementation to GGUF kern…
fc990f9 
…el (vllm-project#8357)
@gshtras
Merge remote-tracking branch 'upstream/main'
0f397c3
@gshtras
New llm_engine output format
b0a39a4
@gshtras
Merge remote-tracking branch 'st/ms-rocm-advance-step' into upstream_…
30a9875 
…merge_24_09_16
@github-actions GitHub Actions
Copy link

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

  • Add ready label to the PR
  • Enable auto-merge.

🚀

@gshtras gshtras force-pushed the upstream_merge_24_09_16 branch 3 times, most recently from 533f64b to 6ed41b8 Compare September 16, 2024 21:23
@gshtras gshtras force-pushed the upstream_merge_24_09_16 branch from 6ed41b8 to c27753d Compare September 16, 2024 22:25
shajrawi
shajrawi approved these changes Sep 16, 2024
View reviewed changes
Copy link
Collaborator

@shajrawi shajrawi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work!

@gshtras gshtras merged commit ad9026c into main Sep 16, 2024
16 checks passed
@gshtras gshtras deleted the upstream_merge_24_09_16 branch September 16, 2024 22:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shajrawi shajrawi shajrawi approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.