New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

How to use quantization methods to load models in a multi-machine environment #415

Open

etersin opened this issue Dec 27, 2024 · 1 comment

etersin commented Dec 27, 2024

I have four 3070 GPUs. How can I use quantization methods to load the model and run Flux.1-dev?

Collaborator

feifeibear commented Dec 31, 2024

You should find a int8 flux checkpoint for diffusers.
You can try this one.
https://huggingface.co/Disty0/FLUX.1-dev-qint8

However, xDiT has not been tested with int8 flux-dev version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment