Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upstream merge 24 12 16 #330

Merged
merged 102 commits into from
Dec 16, 2024
Merged

Upstream merge 24 12 16 #330

merged 102 commits into from
Dec 16, 2024

Conversation

gshtras
Copy link
Collaborator

@gshtras gshtras commented Dec 16, 2024

Also deprecating sync openai entrypoint

richardsliu and others added 30 commits December 9, 2024 23:07
@richardsliu @khluu
Build tpu image in release pipeline (vllm-project#10936)
5ed5d5f 
Signed-off-by: Richard Liu <[email protected]>
Co-authored-by: Kevin H. Luu <[email protected]>
@WoosukKwon
[V1] Do not store None in self.generators (vllm-project#11038)
6faec54 
Signed-off-by: Woosuk Kwon <[email protected]>
@mgoin @tlrmchlsmth
[Docs] Add dedicated tool calling page to docs (vllm-project#10554)
6d52528 
Signed-off-by: mgoin <[email protected]>
Co-authored-by: Tyler Michael Smith <[email protected]>
@Isotr0py
[Model] Add has_weight to RMSNorm and re-enable weights loading track…
d1f6d1c 
…er for Mamba (vllm-project#10739)

Signed-off-by: Isotr0py <[email protected]>
@DarkLight1337
[Bugfix] Fix usage of deprecated decorator (vllm-project#11025)
391d7b2 
Signed-off-by: DarkLight1337 <[email protected]>
@joerunde
[Frontend] Use request id from header (vllm-project#10968)
980ad39 
Signed-off-by: Joe Runde <[email protected]>
@patrickvonplaten
[Pixtral] Improve loading (vllm-project#11040)
bc192a2
@tlrmchlsmth
[V1] Multiprocessing Tensor Parallel Support for v1 (vllm-project#9856)
28b3a1c 
Signed-off-by: Tyler Michael Smith <[email protected]>
@youkaichao
monitor metrics of tokens per step using cudagraph batchsizes (vllm-p…
ebf7780 
…roject#11031)

Signed-off-by: youkaichao <[email protected]>
@sjuxax
[Bugfix] Fix xgrammar failing to read a vocab_size from LlavaConfig o…
e35879c 
…n PixtralHF. (vllm-project#11043)
@dmoliveira
Update README.md (vllm-project#11034)
bfd6104
@GeneDer
[Bugfix] cuda error running llama 3.2 (vllm-project#11047)
82c73fd
@mfournioux
Add example of helm chart for vllm deployment on k8s (vllm-project#9199)
fe2e10c 
Signed-off-by: Maxime Fournioux <[email protected]>
@tjohnson31415
[Bugfix] Handle <|tool_call|> token in granite tool parser (vllm-proj…
beb16b2 
…ect#11039)

Signed-off-by: Travis Johnson <[email protected]>
@jeejeelee
[Misc][LoRA] Add PEFTHelper for LoRA (vllm-project#11003)
d05f886 
Signed-off-by: Jee Jee Li <[email protected]>
@joerunde
[Bugfix] Backport request id validation to v0 (vllm-project#11036)
9b9cef3 
Signed-off-by: Joe Runde <[email protected]>
@flaviabeo
[BUG] Remove token param vllm-project#10921 (vllm-project#11022)
250ee65 
Signed-off-by: Flavia Beo <[email protected]>
@russellb
[Core] Update to outlines >= 0.1.8 (vllm-project#10576)
e739194 
Signed-off-by: Russell Bryant <[email protected]>
@youkaichao
[torch.compile] add a flag to track batchsize statistics (vllm-projec…
75f89dc 
…t#11059)

Signed-off-by: youkaichao <[email protected]>
@WoosukKwon
[V1][Bugfix] Always set enable_chunked_prefill = True for V1 (vllm-pr…
134810b 
…oject#11061)

Signed-off-by: Woosuk Kwon <[email protected]>
@tlrmchlsmth
[Bugfix] Fix Mamba multistep (vllm-project#11071)
9a93973 
Signed-off-by: Tyler Michael Smith <[email protected]>
@aurickq
[Misc] LoRA + Chunked Prefill (vllm-project#9057)
d5c5154
@mzusman
[Model] PP support for Mamba-like models (vllm-project#10992)
ffa48c9 
Signed-off-by: mzusman <[email protected]>
@maxdebayser
Fix streaming for granite tool call when <|tool_call|> is present (vl…
e39400a 
…lm-project#11069)

Signed-off-by: Max de Bayser <[email protected]>
@DarkLight1337
[CI/Build] Check transformers v4.47 (vllm-project#10991)
2e33fe4 
Signed-off-by: DarkLight1337 <[email protected]>
@khluu
[ci/build] Fix AMD CI dependencies (vllm-project#11087)
3fb4b4f
@khluu
[ci/build] Fix entrypoints test and pin outlines version (vllm-projec…
9974fca 
…t#11088)
@russellb
[Core] v1: Use atexit to handle engine core client shutdown (vllm-pro…
61b1d2f 
…ject#11076)

Signed-off-by: Russell Bryant <[email protected]>
@B-201
[Bugfix] Fix Idefics3 fails during multi-image inference (vllm-projec…
2e32f5d 
…t#11080)

Signed-off-by: B-201 <[email protected]>
@rafvasq
[Bugfix]: Clamp -inf logprob values in prompt_logprobs (vllm-projec…
40766ca 
…t#11073)

Signed-off-by: Rafael Vasquez <[email protected]>
tlrmchlsmth and others added 27 commits December 13, 2024 17:02
@tlrmchlsmth
[V1][Bugfix] Fix EngineCoreProc profile (vllm-project#11185)
4b5b8a6 
Signed-off-by: Tyler Michael Smith <[email protected]>
@comaniac
[Bugfix][V1] Re-compute an entire block when fully cache hit (vllm-pr…
9855aea 
…oject#11186)

Signed-off-by: Cody Yu <[email protected]>
@dhuangnm
update compressed-tensors to latest version (vllm-project#11183)
24a3d12 
Co-authored-by: dhuangnm <[email protected]>
@russellb
[Core] Update outlines and increase its threadpool size (vllm-project…
4825926 
…#11140)

Signed-off-by: Russell Bryant <[email protected]>
@tlrmchlsmth
[V1][Bugfix] Fix V1 TP trust-remote-code (vllm-project#11182)
ea7bd68 
Signed-off-by: Tyler Michael Smith <[email protected]>
@jeejeelee
[Misc] Minor improvements to the readability of PunicaWrapperBase (vl…
3cb5769 
…lm-project#11200)

Signed-off-by: Jee Jee Li <[email protected]>
@bradhilton
[Frontend] Add logits_processors as an extra completion argument (v…
9c3dadd 
…llm-project#11150)

Signed-off-by: Brad Hilton <[email protected]>
@DarkLight1337
[VLM] Fully dynamic prompt replacement in merged input processor (vll…
93abf23 
…m-project#11199)

Signed-off-by: DarkLight1337 <[email protected]>
@markmc
Enable mypy checking on V1 code (vllm-project#11105)
6d917d0 
Signed-off-by: Mark McLoughlin <[email protected]>
@llsj14
[Performance][Core] Optimize the performance of evictor v1 and v2 by …
8869368 
…applying a priority queue and lazy deletion (vllm-project#7209)
@jeejeelee
[[Misc]Upgrade bitsandbytes to the latest version 0.45.0 (vllm-projec…
15859f2 
…t#11201)
@youkaichao
[torch.compile] allow tracking forward time (vllm-project#11081)
a1c0205 
Signed-off-by: youkaichao <[email protected]>
@DarkLight1337
[Misc] Clean up multi-modal processor (vllm-project#11207)
b10609e 
Signed-off-by: DarkLight1337 <[email protected]>
@DarkLight1337
[Bugfix] Fix error handling of unsupported sliding window (vllm-proje…
96d673e 
…ct#11213)

Signed-off-by: DarkLight1337 <[email protected]>
@KuntaiDu
[Doc] add documentation for disaggregated prefilling (vllm-project#11197
38e599d 
)

Signed-off-by: Kuntai Du <[email protected]>
@ShangmingCai
[Core] Support disaggregated prefill with Mooncake Transfer Engine (v…
d263bd9 
…llm-project#10884)

Signed-off-by: Shangming Cai <[email protected]>
@WoosukKwon
[V1][Minor] Cache np arange to reduce input preparation overhead (vll…
25ebed2 
…m-project#11214)

Signed-off-by: Woosuk Kwon <[email protected]>
@AlexHe99
Update deploying_with_k8s.rst (vllm-project#10922)
da6f409
@chenqianfzh
[Bugfix] Fix block size validation (vllm-project#10938)
69ba344
@yansh97
[Bugfix] Fix the default value for temperature in ChatCompletionReque…
17138af 
…st (vllm-project#11219)
@cennn @drikster80
WIP: [CI/Build] simplify Dockerfile build for ARM64 / GH200 (vllm-pro…
b3b1526 
…ject#11212)

Signed-off-by: drikster80 <[email protected]>
Co-authored-by: drikster80 <[email protected]>
@janimo
[Model] Support Cohere2ForCausalLM (Cohere R7B) (vllm-project#11203)
bddbbcb
@Isotr0py @DarkLight1337
[Model] Refactor Ultravox to use merged input processor (vllm-project…
d927dbc 
…#11198)

Signed-off-by: Isotr0py <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
@Isotr0py
[Doc] Reorder vision language examples in alphabet order (vllm-projec…
2ca830d 
…t#11228)

Signed-off-by: Isotr0py <[email protected]>
@gshtras
Merge remote-tracking branch 'upstream/main'
1a8e549
@gshtras
Deprecating sync_openai
78440dc
@gshtras
Remove new irrelevant action
ddec133
@gshtras gshtras merged commit 1f0f4c6 into main Dec 16, 2024
10 of 11 checks passed
@gshtras gshtras deleted the upstream_merge_24_12_16 branch December 16, 2024 21:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shajrawi shajrawi shajrawi approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.