Welcome to xDiT Discussions!

[feifeibear]

Please let us know which DiT model you are most interested in.

[lsy789]

Now I am learning ai system and cuda.Please how could i join in this project?

[feifeibear]

Here are some suggested tasks to help you get started:

1. Implement some unit tests for the project and try to understand the attention mechanism implementations of different models.
   Link: https://github.com/xdit-project/xDiT/blob/main/xfuser/model_executor/layers/attention_processor.py

2. Support some new models and learn about their usage.

3. Optimize the code using methods like torch.compile, onediff, etc.

[lsy789]

Thanks!

[lsy789]

(base) ➜ xDiT git:(main) python ./examples/pixartalpha_example.py -h
/environment/miniconda3/lib/python3.10/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: '/environment/miniconda3/lib/python3.10/site-packages/torchvision/image.so: undefined symbol: _ZN3c1017RegisterOperatorsD1Ev'If you don't plan on using image functionality from torchvision.io, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have libjpeg or libpng installed before building torchvision from source?
warn(
2024-08-21 11:31:07.491529: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-08-21 11:31:08.107901: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Traceback (most recent call last):
File "/home/featurize/work/xDiT/./examples/pixartalpha_example.py", line 7, in
from xfuser.core.distributed import (
ModuleNotFoundError: No module named 'xfuser.core'
(base) ➜ xDiT git:(main) python ./examples/pixartalpha_example.py -h
(base) ➜ xDiT git:(main) pip show xfuser
Name: xfuser
Version: 0.2
Summary: xDiT: A Scalable Inference Engine for Diffusion Transformers (DiTs) on multi-GPU Clusters
Home-page: https://github.com/xdit-project/xDiT.
Author: Jiannan Wang, Jiarui Fang, Jinzhe Pan, Aoyu Li, Pengcheng Yang
Author-email: [email protected]
License:
Location: /environment/miniconda3/lib/python3.10/site-packages
Requires: accelerate, beautifulsoup4, diffusers, distvae, ftfy, sentencepiece, torch, transformers
Required-by:
(base) ➜ xDiT git:(main) pip show tensorrt
Name: tensorrt
Version: 10.3.0
Summary: TensorRT Metapackage
Home-page: https://developer.nvidia.com/tensorrt
Author: NVIDIA Corporation
Author-email:
License: Proprietary
Location: /environment/miniconda3/lib/python3.10/site-packages
Requires: tensorrt-cu12
Required-by:

[lsy789]

Please i have installed tensort and xfuser, but excueting " python ./examples/pixartalpha_example.py -h" has not find tensort and xfuser.core. What do i need to do?

[feifeibear]

You should pip install from the latest version.
I guess you install it from pip.
I will update the pip version ASAP.

[lsy789]

thanks! i try again

[feifeibear]

I have upgrade the xDiT version to 0.3 and upload new wheel to pypi. You can install
pip install xfuser==0.3

[dukeqiao]

really need support for diffusers flux with controlnet

[feifeibear]

Thanks for your suggestion. Most of people use ControlNet+Flux.1 in ComfyUI.

We just wonder how people use controllers with diffusers' flux. We are planing to implement controlnet in ComfyUI.

[dukeqiao]

the reason I trying to use diffusers fluxpipeline is the inference speed, ComfyUI pipeline is too slow for online deployment

[feifeibear]

Would you like to help us add controlnet in fluxpipeline. We can draft a issue to give you some detail instructions?

[dukeqiao]

sure, in fact I already tried, but there is some problems with controlnet input shape with running _sync_pipeline function

[Lay2000]

sure, in fact I already tried, but there is some problems with controlnet input shape with running _sync_pipeline function

Hi, Duke! Could you please provide more details on the problem you met? Like the entry scripts with hyperparameters you used and the Error information.