support FLUX？

[algorithmconquer]

Is there any plan to support FLUX in the future?https://github.com/NVIDIA/TensorRT，this currently does not support multi-GPU parallelism.

[hello-11]

@algorithmconquer thanks for your interest in TrtLLM. I cannot open the link. Could you send the right link?

[algorithmconquer]

@hello-11 The right link is https://github.com/NVIDIA/TensorRT;

[algorithmconquer]

@hello-11 Can you give a convert_checkpoint.py to transform the dit(transformer) of flux?

[hello-11]

@algorithmconquer Can this example of dit help you?

[algorithmconquer]

@hello-11 I have two questions; Question1: In https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/dit, why only support pp_size = 1? Question2: dit model layer name is different from the dit(transformer) of flux, how to achieve convert_checkpoint.py according to https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/dit?

[ChunhuanLin]

@algorithmconquer We did have plan to support Flux in the near future.

[ChunhuanLin]

@hello-11 I have two questions; Question1: In https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/dit, why only support pp_size = 1? Question2: dit model layer name is different from the dit(transformer) of flux, how to achieve convert_checkpoint.py according to https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/dit?

1. Please go to this why Dit does not support pp_size > 1 #2427 for the discussion.
2. We need modify convert_checkpoint.py to follow the Flux checkpoint. Besides this script, we also need to support Flux model here.
   We still need some works to support Flux indeed. And we are going to support it in the near future.

[algorithmconquer]

@ChunhuanLin Thank you for your response!

[MagicRUBICK]

Is there a planned release date for flux.1-dev?

[pashanitw]

i would love to see the flux support and stable diffusion 3.5