Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Core] Add Gloo and NCCL migration backends with elastic support #7

Merged
merged 2 commits into from
Aug 23, 2024
Merged

[Core] Add Gloo and NCCL migration backends with elastic support #7

merged 2 commits into from
Aug 23, 2024

Conversation

KuilongCui
Copy link
Contributor

@KuilongCui KuilongCui commented Jul 25, 2024 •
edited
Loading

  1. add gloo and nccl migration backend support for kv cache transfer
  2. optimize kv-cache transfer with layer-wise pipeline
  3. add elastic support for gloo and nccl migration backend

@CLAassistant
Copy link

CLAassistant commented Jul 25, 2024 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

@zhypku zhypku requested review from ZeldaHuang and s5u13b July 25, 2024 08:09
@KuilongCui KuilongCui force-pushed the col_dev branch 3 times, most recently from 118833c to 08b9924 Compare July 31, 2024 11:56
@KuilongCui KuilongCui closed this Jul 31, 2024
@KuilongCui KuilongCui reopened this Jul 31, 2024
@KuilongCui KuilongCui force-pushed the col_dev branch 3 times, most recently from febad11 to f07eb64 Compare August 2, 2024 05:46
@KuilongCui KuilongCui force-pushed the col_dev branch 2 times, most recently from 952dc24 to 8496ada Compare August 14, 2024 06:00
s5u13b
s5u13b reviewed Aug 19, 2024
View reviewed changes
llumnix/arg_utils.py Outdated Show resolved Hide resolved
s5u13b
s5u13b reviewed Aug 22, 2024
View reviewed changes
llumnix/arg_utils.py Outdated Show resolved Hide resolved
zhypku
zhypku reviewed Aug 22, 2024
View reviewed changes
Makefile Show resolved Hide resolved
docs/Quickstart.md Outdated Show resolved Hide resolved
llumnix/arg_utils.py Outdated Show resolved Hide resolved
@KuilongCui KuilongCui changed the title [Core] Support for more migration backend types with added flexibility [Core] Add Gloo and NCCL as migration backends with elastic support Aug 22, 2024
@KuilongCui KuilongCui changed the title [Core] Add Gloo and NCCL as migration backends with elastic support [Core] Add Gloo and NCCL migration backends with elastic support Aug 23, 2024
zhypku
zhypku approved these changes Aug 23, 2024
View reviewed changes
Copy link
Collaborator

@zhypku zhypku left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, TGIF :D

@KuilongCui KuilongCui merged commit 6b7b099 into main Aug 23, 2024
4 checks passed
@KuilongCui KuilongCui deleted the col_dev branch August 28, 2024 11:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@s5u13b s5u13b s5u13b left review comments

@ZeldaHuang ZeldaHuang ZeldaHuang approved these changes

@zhypku zhypku zhypku approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@KuilongCui @CLAassistant @zhypku @s5u13b @ZeldaHuang