Skip to content

prefill metadata splitting #112

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 325 commits into
base: sage/dbo-full-cudagraphs
Choose a base branch
from
Draft

prefill metadata splitting #112

wants to merge 325 commits into from

Conversation

@LucasWilkinson
Copy link

@LucasWilkinson LucasWilkinson commented Sep 5, 2025 •
edited by github-actions bot
Loading

Purpose

Test Plan

Test Result

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

DarkLight1337 and others added 30 commits September 17, 2025 12:19
@DarkLight1337
[Misc] Avoid use of deprecated AutoModelForVision2Seq (vllm-project…
e120533 
…#25065)

Signed-off-by: DarkLight1337 <[email protected]>
@danielafrimi
Add RADIO Vision Encoder Support to vLLM (vllm-project#24595)
252ada5 
Signed-off-by: Daniel Afrimi <[email protected]>
Co-authored-by: root <[email protected]>
@bigPYJ1151
[Bugfix] Fix Stream usage in CPU model runner and OneDNN kernel check (…
9fccd04 
…vllm-project#25046)

Signed-off-by: jiang1.li <[email protected]>
@Aidyn-A
Apply fixes for CUDA 13 (vllm-project#24599)
bfe9380 
Signed-off-by: Aidyn-A <[email protected]>
@dolpm @jeejeelee
[fix] lora benchmarks pass no_lora_flag_cpu (vllm-project#23774)
1b962e2 
Signed-off-by: Dylan Maloy <[email protected]>
Co-authored-by: Jee Jee Li <[email protected]>
@sighingnow
[Bugfix][Qwen3-Next] fixes the varlen issue in qwen3-next's MTP imple…
dd6a910 
…mentation. (vllm-project#24957)

Signed-off-by: Tao He <[email protected]>
@samzong
[Docs] improve code formatting and comments for eliminate griffe buil…
47f670b 
…d warning. (vllm-project#25010)

Signed-off-by: samzong <[email protected]>
@MatthewBonanni
Remove old cutlass mla (vllm-project#23961)
8f3616f 
Signed-off-by: Matthew Bonanni <[email protected]>
Signed-off-by: Matthew Bonanni <[email protected]>
@samzong
[Docs] vllm/benchmarks/datasets.py fix docstring param format. (vllm-…
4a2d33e 
…project#24970)

Signed-off-by: samzong <[email protected]>
@mgoin
[CI Bugfix] Fix failing test_invalid_env (vllm-project#25078)
087c6ff 
Signed-off-by: mgoin <[email protected]>
@WoosukKwon
[V0 Deprecation] Remove V0 Core tests (vllm-project#25082)
4b946d6 
Signed-off-by: Woosuk Kwon <[email protected]>
@simon-mo @jeejeelee
cleanup: remove adapter commons (vllm-project#25045)
4aa8c7b 
Signed-off-by: Jee Jee Li <[email protected]>
Co-authored-by: Jee Jee Li <[email protected]>
@simon-mo
Remove unused find_cuda_init helper script (vllm-project#25044)
d6a518f
@WoosukKwon
[V0 Deprecation] Remove unused output processor util (vllm-project#25023
99cc41a 
)

Signed-off-by: Woosuk Kwon <[email protected]>
@mgoin
Change log level from info to debug for IOProcessor (vllm-project#24999)
8b32464 
Signed-off-by: Michael Goin <[email protected]>
@WoosukKwon
[CI] Revert back prepare_prompts and check_answers (vllm-project#25087)
eb68c2d 
Signed-off-by: Woosuk Kwon <[email protected]>
@WoosukKwon
[V0 Deprecation] Remove V0 tests in test_sequence.py (vllm-project#25088
9d442b7 
)

Signed-off-by: Woosuk Kwon <[email protected]>
@mgoin
[CI Bugfix] Fix failing test_model_load_with_params tests due to toke…
e3db5eb 
…nizer refactor (vllm-project#25086)

Signed-off-by: mgoin <[email protected]>
@afeldman-nm @JosephMarinier
[V1] Logits processor docs (vllm-project#22919)
7ae9887 
Signed-off-by: Andrew Feldman <[email protected]>
Signed-off-by: afeldman-nm <[email protected]>
Co-authored-by: Joseph Marinier <[email protected]>
@ApostaC
[Misc] Update owners for KV connector and V1 offloading (vllm-project…
ee5fd49 
…#25041)

Signed-off-by: ApostaC <[email protected]>
@LucasWilkinson
cleanup
bd04165 
Signed-off-by: Lucas Wilkinson <[email protected]>
@mmangkad
[Bugfix] Update import path for bc_linter_include (vllm-project#24766)
8831315 
Signed-off-by: Mohammad Miadh Angkad <[email protected]>
@ahao-anyscale
[BUG] Exclude .pth files when pulling remote files (vllm-project#25092)
f20c3b0 
Signed-off-by: ahao-anyscale <[email protected]>
@czhu-cohere
[Kernel] Faster pre-processing time for W4A8 (vllm-project#23972)
3c068c6 
Signed-off-by: czhu-cohere <[email protected]>
@qandrew
[gpt-oss][2] fix types for streaming (vllm-project#24556)
bff2e5f 
Signed-off-by: Andrew Xia <[email protected]>
@alexm-redhat @mgoin
[Bugfix][B200] Fix cutlass_mla hang (vllm-project#24966)
fedb75f 
Signed-off-by: Alexander Matveev <[email protected]>
Co-authored-by: Michael Goin <[email protected]>
@dllehr-amd
Aiter mha fp8 fix (vllm-project#24991)
1a456c7 
Signed-off-by: Doug Lehr <[email protected]>
Co-authored-by: Doug Lehr <[email protected]>
@mgoin
Disable failing GPT-OSS Eval (Blackwell) for now (vllm-project#25107)
9f882d8 
Signed-off-by: mgoin <[email protected]>
@elvischenv @mgoin
[Bugfix] Refactor Flashinfer TRTLLM attention kernel selection logic (v…
e67a79d 
…llm-project#24600)

Signed-off-by: elvischenv <[email protected]>
Co-authored-by: Michael Goin <[email protected]>
@karan @gemini-code-assist
Add a batched auto tune script (vllm-project#25076)
2a4d641 
Signed-off-by: Karan Goel <[email protected]>
Signed-off-by: Karan Goel <[email protected]>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
alecsolder and others added 30 commits September 22, 2025 23:38
@alecsolder @yeqcharlotte
[Frontend] Responses API MCP tools for built in tools and to pass thr…
45d7d85 
…ough headers (vllm-project#24628)

Signed-off-by: Alec Solder <[email protected]>
Signed-off-by: Alec S <[email protected]>
Co-authored-by: Alec Solder <[email protected]>
Co-authored-by: Ye (Charlotte) Qi <[email protected]>
@ProExpertProg
[Bugfix] fix custom op test (vllm-project#25429)
d588cd2 
Signed-off-by: Luka Govedič <[email protected]>
@russellb
[Core] Drop overly aggressive whisper assertion (vllm-project#25408)
f31ff87 
Signed-off-by: Russell Bryant <[email protected]>
@NickLucche
[Bugfix] Fix missing clear_connector_metadata (vllm-project#25397)
0901970 
Signed-off-by: NickLucche <[email protected]>
@MatthewBonanni @robertgshaw2-redhat @Bam4d
[BugFix] [DP/EP] Fix slow execution when BS <= DP (vllm-project#25407)
ac0048c 
Signed-off-by: Matthew Bonanni <[email protected]>
Co-authored-by: Robert Shaw <[email protected]>
Co-authored-by: Chris Bamford <[email protected]>
@alexm-redhat
[Performance] Remove input pads in cutlass_mla and optimize v_proj ou…
0b7bed9 
…tput handling (vllm-project#25184)

Signed-off-by: Alexander Matveev <[email protected]>
@yewentao256
[Perf] Apply torch.compile for per_block_cast_to_fp8 (vllm-project#…
9949aa2 
…24611)

Signed-off-by: yewentao256 <[email protected]>
@Isotr0py
[V0 deprecation] Remove platform v1 controling interface (vllm-projec…
6fa78d8 
…t#25410)

Signed-off-by: Isotr0py <[email protected]>
@Isotr0py
[V0 deprecation] Remove _set_default_args_v0 function (vllm-project…
c625f90 
…#25409)

Signed-off-by: Isotr0py <[email protected]>
@yewentao256
[Bug] Fix Long Context OOM Issue (vllm-project#25290)
4741239 
Signed-off-by: yewentao256 <[email protected]>
@tlrmchlsmth
Merge branch 'main' into lwilkinson/dbo-prefill
4e9e2ce 
Signed-off-by: Tyler Michael Smith <[email protected]>
@LucasWilkinson
fix pre-commit
356ddcb 
Signed-off-by: Lucas Wilkinson <[email protected]>
@JJJYmmm
[feat] Support MRoPE + YaRN (vllm-project#25384)
fc97733 
Signed-off-by: liuye.hj <[email protected]>
Co-authored-by: liuye.hj <[email protected]>
@jikunshang
[XPU] Fix compile_size is None case. (vllm-project#25433)
f225ea7 
Signed-off-by: Kunshang Ji <[email protected]>
@luccafong
[benchmarks]allow skip ready check for bench serve (vllm-project#25420)
eea1783 
Signed-off-by: Lu Fang <[email protected]>
Signed-off-by: Lucia Fang <[email protected]>
Co-authored-by: Lucia (Lu) Fang <[email protected]>
@mgoin
[Bugfix] Remove contiguous output req for context parallel MLA (vllm-…
78237e4 
…project#25414)

Signed-off-by: Michael Goin <[email protected]>
@windsonsea
[Docs] Fix griffe warnings in vllm/lora/ops (vllm-project#25369)
fafbe11 
Signed-off-by: windsonsea <[email protected]>
@varun-sundar-rabindranath
[DP/EP][GPTOSS] Use triton matmul-ogs kernels for GPTOSS DP/EP (vllm-…
e8db44f 
…project#24588)

Signed-off-by: Varun Sundar Rabindranath <[email protected]>
Co-authored-by: Varun Sundar Rabindranath <[email protected]>
@xuechendi
[NIXL][OOT platform] support nixl_connector with oot platform and oth…
5774b0a 
…er nixl_backend (vllm-project#25121)

Signed-off-by: Chendi Xue <[email protected]>
@DarkLight1337
[Model] Enable DP for ViT in Qwen2-VL (vllm-project#25445)
c98be0a 
Signed-off-by: DarkLight1337 <[email protected]>
@minosfuture
Handle triton kernel import exception (vllm-project#25319)
ba8d216 
Signed-off-by: Ming Yang <[email protected]>
@Zhikaiiii
[Frontend] Add a new xml-based tool parser for qwen3-coder (vllm-proj…
9383cd6 
…ect#25028)

Signed-off-by: Zhikaiiii <[email protected]>
@DarkLight1337
[Misc] Move DP for ViT code inside model executor dir (vllm-project#2…
babad6e 
…5459)

Signed-off-by: DarkLight1337 <[email protected]>
@ahartel
[Test]: Hermes tool parser stream output error in Qwen3 case (vllm-pr…
4322c55 
…oject#25203)

Signed-off-by: Andreas Hartel <[email protected]>
@Isotr0py
[Bugfix] Fix idefics3 tie_word_embeddings (vllm-project#25454)
231c2c6 
Signed-off-by: Isotr0py <[email protected]>
@jeejeelee
[Core] Optimize LoRA weight loading (vllm-project#25403)
273690a 
Signed-off-by: Jee Jee Li <[email protected]>
@vllmellm
[docs] Benchmark Serving Incorrect Arg (vllm-project#25474)
0d9fe26 
Signed-off-by: vllmellm <[email protected]>
@Isotr0py
[CI/Build] Fix disabled v1 attention backend selection test (vllm-pro…
b6a136b 
…ject#25471)

Signed-off-by: Isotr0py <[email protected]>
@tlrmchlsmth
Merge branch 'main' into lwilkinson/dbo-prefill
4c631db 
Signed-off-by: Tyler Michael Smith <[email protected]>
@LucasWilkinson
precommit fix
f0bc569 
Signed-off-by: Lucas Wilkinson <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@NickLucche NickLucche Awaiting requested review from NickLucche NickLucche will be requested when the pull request is marked ready for review NickLucche is a code owner

@mgoin mgoin Awaiting requested review from mgoin mgoin will be requested when the pull request is marked ready for review mgoin is a code owner

@robertgshaw2-redhat robertgshaw2-redhat Awaiting requested review from robertgshaw2-redhat robertgshaw2-redhat will be requested when the pull request is marked ready for review robertgshaw2-redhat is a code owner

@yewentao256 yewentao256 Awaiting requested review from yewentao256 yewentao256 will be requested when the pull request is marked ready for review yewentao256 is a code owner

@tlrmchlsmth tlrmchlsmth Awaiting requested review from tlrmchlsmth tlrmchlsmth will be requested when the pull request is marked ready for review tlrmchlsmth is a code owner

@youkaichao youkaichao Awaiting requested review from youkaichao youkaichao will be requested when the pull request is marked ready for review youkaichao is a code owner

@njhill njhill Awaiting requested review from njhill njhill will be requested when the pull request is marked ready for review njhill is a code owner

@alexm-redhat alexm-redhat Awaiting requested review from alexm-redhat alexm-redhat will be requested when the pull request is marked ready for review alexm-redhat is a code owner

@ProExpertProg ProExpertProg Awaiting requested review from ProExpertProg ProExpertProg will be requested when the pull request is marked ready for review ProExpertProg is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.