[2/N] executor pass the complete config to worker/modelrunner #9938

youkaichao · 2024-11-01T21:41:31Z

continue of #9933

Signed-off-by: youkaichao <[email protected]>

github-actions · 2024-11-01T21:41:42Z

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

Add ready label to the PR
Enable auto-merge.

🚀

Signed-off-by: youkaichao <[email protected]>

vllm/worker/neuron_model_runner.py

robertgshaw2-neuralmagic · 2024-11-01T22:11:12Z

I personally find it confusing that we call the members/arguments vllm_config when it is of type EngineConfig. I would personally choose the name engine_config, but I dont have religion around it.

Other than the ModelConfig --> EngineConfig comment, approving to unblock

njhill

Nice cleanup!

vllm/worker/multi_step_worker.py

vllm/worker/neuron_model_runner.py

vllm/worker/neuron_worker.py

njhill · 2024-11-01T22:17:58Z

I personally find it confusing that we call the members/arguments vllm_config when it is of type EngineConfig. I would personally choose the name engine_config, but I dont have religion around it.

Agree .. alternatively EngineConfig could be renamed to VLLMConfig :)

Signed-off-by: youkaichao <[email protected]>

Co-authored-by: Nick Hill <[email protected]>

Signed-off-by: youkaichao <[email protected]>

…tor-worker

Signed-off-by: youkaichao <[email protected]>

mergify · 2024-11-02T06:23:41Z

This pull request has merge conflicts that must be resolved before it can be
merged. @youkaichao please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

…roject#9938) Signed-off-by: youkaichao <[email protected]> Co-authored-by: Nick Hill <[email protected]>

…roject#9938) Signed-off-by: youkaichao <[email protected]> Co-authored-by: Nick Hill <[email protected]> Signed-off-by: Linkun Chen <[email protected]>

…roject#9938) Signed-off-by: youkaichao <[email protected]> Co-authored-by: Nick Hill <[email protected]> Signed-off-by: Richard Liu <[email protected]>

…roject#9938) Signed-off-by: youkaichao <[email protected]> Co-authored-by: Nick Hill <[email protected]>

Signed-off-by: youkaichao <[email protected]> Co-authored-by: Nick Hill <[email protected]>

…roject#9938) Signed-off-by: youkaichao <[email protected]> Co-authored-by: Nick Hill <[email protected]>

…roject#9938) Signed-off-by: youkaichao <[email protected]> Co-authored-by: Nick Hill <[email protected]> Signed-off-by: Loc Huynh <[email protected]>

…roject#9938) Signed-off-by: youkaichao <[email protected]> Co-authored-by: Nick Hill <[email protected]> Signed-off-by: Sumit Dubey <[email protected]>

…roject#9938) Signed-off-by: youkaichao <[email protected]> Co-authored-by: Nick Hill <[email protected]>

…roject#9938) Signed-off-by: youkaichao <[email protected]> Co-authored-by: Nick Hill <[email protected]> Signed-off-by: Maxime Fournioux <[email protected]>

…roject#9938) Signed-off-by: youkaichao <[email protected]> Co-authored-by: Nick Hill <[email protected]> Signed-off-by: Tyler Michael Smith <[email protected]>

…roject#9938) Signed-off-by: youkaichao <[email protected]> Co-authored-by: Nick Hill <[email protected]>

youkaichao added 7 commits November 1, 2024 14:01

one worker

6f087a6

Signed-off-by: youkaichao <[email protected]>

stage

bc53874

Signed-off-by: youkaichao <[email protected]>

xpu

b39e45b

Signed-off-by: youkaichao <[email protected]>

tpu

2a8678f

Signed-off-by: youkaichao <[email protected]>

openvino

d076d08

Signed-off-by: youkaichao <[email protected]>

neuron

14b8af4

Signed-off-by: youkaichao <[email protected]>

cpu

b4670b5

Signed-off-by: youkaichao <[email protected]>

youkaichao requested review from WoosukKwon, zhuohan123, alexm-neuralmagic, comaniac and njhill as code owners November 1, 2024 21:41

youkaichao marked this pull request as draft November 1, 2024 21:41

youkaichao added 2 commits November 1, 2024 14:44

fix

a02f2b3

Signed-off-by: youkaichao <[email protected]>

executor -> worker

00e3a77

Signed-off-by: youkaichao <[email protected]>

youkaichao marked this pull request as ready for review November 1, 2024 21:53

robertgshaw2-neuralmagic reviewed Nov 1, 2024

View reviewed changes

vllm/worker/neuron_model_runner.py Outdated Show resolved Hide resolved

robertgshaw2-neuralmagic self-requested a review November 1, 2024 22:11

robertgshaw2-neuralmagic approved these changes Nov 1, 2024

View reviewed changes

njhill reviewed Nov 1, 2024

View reviewed changes

vllm/worker/multi_step_worker.py Outdated Show resolved Hide resolved

vllm/worker/neuron_model_runner.py Outdated Show resolved Hide resolved

vllm/worker/neuron_worker.py Outdated Show resolved Hide resolved

youkaichao and others added 7 commits November 1, 2024 17:05

fix neuron

02242c7

Signed-off-by: youkaichao <[email protected]>

Update vllm/worker/multi_step_worker.py

5dbdf1a

Co-authored-by: Nick Hill <[email protected]>

Update vllm/worker/neuron_worker.py

1addf53

Co-authored-by: Nick Hill <[email protected]>

fix

ae484c6

Signed-off-by: youkaichao <[email protected]>

Merge branch 'excutor-worker' of github.com:youkaichao/vllm into excu…

ba362e7

…tor-worker

rename to vllm config

030c76c

Signed-off-by: youkaichao <[email protected]>

remove dead code

fb70c02

Signed-off-by: youkaichao <[email protected]>

youkaichao added 2 commits November 1, 2024 21:53

ngram

6b4871d

Signed-off-by: youkaichao <[email protected]>

spec decode

8daf26a

Signed-off-by: youkaichao <[email protected]>

youkaichao added the ready ONLY add when PR is ready to merge/full CI is needed label Nov 2, 2024

fix

5636c56

Signed-off-by: youkaichao <[email protected]>

mergify bot added the needs-rebase label Nov 2, 2024

Merge branch 'main' into excutor-worker

e3e917e

mergify bot removed the needs-rebase label Nov 2, 2024

youkaichao enabled auto-merge (squash) November 2, 2024 07:48

youkaichao disabled auto-merge November 2, 2024 14:34

youkaichao merged commit e893795 into vllm-project:main Nov 2, 2024
63 of 66 checks passed

youkaichao deleted the excutor-worker branch November 2, 2024 14:35

This was referenced Nov 2, 2024

[3/N] model runner pass the whole config to model #9958

Merged

[Core] Implementing disaggregated prefilling, and caching KV cache in CPU/disk/database. #8498

Closed

robertgshaw2-neuralmagic mentioned this pull request Nov 3, 2024

[V1] Fix EngineArgs refactor on V1 #9954

Merged

lk-chen pushed a commit to lk-chen/vllm that referenced this pull request Nov 4, 2024

[2/N] executor pass the complete config to worker/modelrunner (vllm-p…

cca0550

…roject#9938) Signed-off-by: youkaichao <[email protected]> Co-authored-by: Nick Hill <[email protected]>

bigPYJ1151 pushed a commit to bigPYJ1151/vllm that referenced this pull request Nov 5, 2024

[2/N] executor pass the complete config to worker/modelrunner (vllm-p…

8468796

…roject#9938) Signed-off-by: youkaichao <[email protected]> Co-authored-by: Nick Hill <[email protected]>

DarkLight1337 pushed a commit that referenced this pull request Nov 5, 2024

[2/N] executor pass the complete config to worker/modelrunner (#9938)

e41a52e

Signed-off-by: youkaichao <[email protected]> Co-authored-by: Nick Hill <[email protected]>

DarkLight1337 pushed a commit that referenced this pull request Nov 5, 2024

[2/N] executor pass the complete config to worker/modelrunner (#9938)

5350bac

Signed-off-by: youkaichao <[email protected]> Co-authored-by: Nick Hill <[email protected]>

kzawora-intel mentioned this pull request Nov 5, 2024

[Hardware][Intel-Gaudi] Add Intel Gaudi (HPU) inference backend #6143

Merged

jberkhahn mentioned this pull request Nov 5, 2024

Begin refactoring executor_base ABC #9392

Closed

hissu-hyvarinen pushed a commit to ROCm/vllm that referenced this pull request Nov 6, 2024

[2/N] executor pass the complete config to worker/modelrunner (vllm-p…

1b58216

…roject#9938) Signed-off-by: youkaichao <[email protected]> Co-authored-by: Nick Hill <[email protected]>

KuntaiDu pushed a commit to KuntaiDu/vllm that referenced this pull request Nov 20, 2024

[2/N] executor pass the complete config to worker/modelrunner (vllm-p…

5b14c25

…roject#9938) Signed-off-by: youkaichao <[email protected]> Co-authored-by: Nick Hill <[email protected]>

sleepwalker2017 pushed a commit to sleepwalker2017/vllm that referenced this pull request Dec 13, 2024

[2/N] executor pass the complete config to worker/modelrunner (vllm-p…

1e8a83f

…roject#9938) Signed-off-by: youkaichao <[email protected]> Co-authored-by: Nick Hill <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[2/N] executor pass the complete config to worker/modelrunner #9938

[2/N] executor pass the complete config to worker/modelrunner #9938

youkaichao commented Nov 1, 2024

github-actions bot commented Nov 1, 2024

robertgshaw2-neuralmagic commented Nov 1, 2024

njhill left a comment

njhill commented Nov 1, 2024

mergify bot commented Nov 2, 2024

[2/N] executor pass the complete config to worker/modelrunner #9938

[2/N] executor pass the complete config to worker/modelrunner #9938

Conversation

youkaichao commented Nov 1, 2024

github-actions bot commented Nov 1, 2024

robertgshaw2-neuralmagic commented Nov 1, 2024

njhill left a comment

Choose a reason for hiding this comment

njhill commented Nov 1, 2024

mergify bot commented Nov 2, 2024