Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ConsisID for xdit #405

Merged
merged 11 commits into from
Dec 24, 2024
Merged

ConsisID for xdit #405

merged 11 commits into from
Dec 24, 2024

Conversation

pkuhxy
Copy link
Contributor

@pkuhxy pkuhxy commented Dec 22, 2024

I'm sorry that my misoperation caused the previous repository to become invalid, so I resubmitted a pull request.

examples/run_cogvideo.sh Outdated Show resolved Hide resolved
# PARALLLEL_VAE="--use_parallel_vae"
ENABLE_TILING="--enable_tiling"
# COMPILE_FLAG="--use_torch_compile"
INPUT_IMAGE='--image_path /storage/hxy/cvpr2025/ConsisID/asserts/example_images/4.png'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please use a relative path here and give instruction to users how to download this image?

xfuser/core/distributed/runtime_state.py Outdated Show resolved Hide resolved
@@ -112,6 +116,9 @@ def __init__(self, pipeline: DiffusionPipeline, config: EngineConfig):
backbone_inner_dim=pipeline.transformer.config.num_attention_heads
* pipeline.transformer.config.attention_head_dim,
)

print(f"self.cogvideox:{self.cogvideox}")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we remove the debug code?

xfuser/model_executor/layers/attention_processor.py Outdated Show resolved Hide resolved
@feifeibear
Copy link
Collaborator

You can also update the readme page with consistid on news and supported model session.

@pkuhxy pkuhxy changed the title consisID for xdit ConsisID for xdit Dec 23, 2024
@pkuhxy pkuhxy marked this pull request as draft December 23, 2024 07:57
@feifeibear
Copy link
Collaborator

@pkuhxy Xianyi, the PR looks perfect!
I assume you have nailed the PipeFusion + USP for consistid.
If you can paste a screenshot demonstrating the consistent run with hybrid parallel successfully, I will proceed with merging the PR.

@SHYuanBest
Copy link
Contributor

SHYuanBest commented Dec 23, 2024

I have fix some bugs and reorganize the code. to do:

  • Add consisid_usp_example.py
  • Update main README.md
  • Add Performance README.md

screen_example:
image

screen_usp_example:
image

input: https://github.com/PKU-YuanGroup/ConsisID/blob/main/asserts/example_images/2.png?raw=true

consisid_dp1_cfg1_ulysses2_ring3_tp1_pp1_patchNone_720x480.mp4

input: https://github.com/PKU-YuanGroup/ConsisID/blob/main/asserts/example_images/1.png?raw=true

c6e86f71ca0c192c8e0f538900311553.mp4

@pkuhxy
Copy link
Contributor Author

pkuhxy commented Dec 23, 2024

@pkuhxy Xianyi, the PR looks perfect! I assume you have nailed the PipeFusion + USP for consistid. If you can paste a screenshot demonstrating the consistent run with hybrid parallel successfully, I will proceed with merging the PR.

We have completed all parts of this PR. Please review the code and merge it. Thank you!

@pkuhxy pkuhxy marked this pull request as ready for review December 23, 2024 15:21
@@ -0,0 +1,41 @@
#!/bin/bash
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dose this script work with pipefusion degree>1?

examples/consisid_example.py Show resolved Hide resolved
@feifeibear feifeibear merged commit 970d5ce into xdit-project:main Dec 24, 2024
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants