Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Brandon/flux model loading #6739

Merged
merged 113 commits into from
Aug 27, 2024
Merged

Brandon/flux model loading #6739

merged 113 commits into from
Aug 27, 2024

Conversation

brandonrising
Copy link
Collaborator

@brandonrising brandonrising commented Aug 12, 2024 •
edited
Loading

Summary

Add FLUX support to the model manager.

QA Instructions

Install the FLUX model via the "Starter Models" tab:

  • Test fresh install and inference of all FLUX variants
    • FLUX schnell
    • FLUX schnell quantized
    • FLUX dev
    • FLUX dev quantized
  • Test quantized on 16GB GPU
  • Maybe: Test quantized on 8GB GPU
  • Test that the app still runs on CPU and MPS systems (main concern is pip install, or that we try to import bitsandbytes)

Merge Plan

Can be merged with proper approvals

@github-actions github-actions bot added python PRs that change python files invocations PRs that change invocations backend PRs that change backend files frontend PRs that change frontend files labels Aug 12, 2024
@sammcj sammcj mentioned this pull request Aug 14, 2024
Closed
1 task
@github-actions github-actions bot added Root python-deps PRs that change python dependencies services PRs that change app services labels Aug 15, 2024
@RyanJDick RyanJDick changed the base branch from ryan/flux to main August 20, 2024 18:10
@github-actions github-actions bot added the python-tests PRs that change python tests label Aug 20, 2024
RyanJDick
RyanJDick reviewed Aug 20, 2024
View reviewed changes
invokeai/app/invocations/fields.py Outdated Show resolved Hide resolved
invokeai/app/invocations/fields.py Outdated Show resolved Hide resolved
invokeai/app/invocations/flux_text_encoder.py Show resolved Hide resolved
invokeai/app/invocations/flux_text_encoder.py Outdated Show resolved Hide resolved
invokeai/app/invocations/flux_text_encoder.py Outdated Show resolved Hide resolved
invokeai/backend/quantization/load_flux_model_bnb_llm_int8.py Outdated Show resolved Hide resolved
invokeai/backend/requantize.py Outdated Show resolved Hide resolved
invokeai/backend/requantize.py Outdated Show resolved Hide resolved
invokeai/configs/flux/flux1-schnell.yaml Outdated Show resolved Hide resolved
pyproject.toml Outdated Show resolved Hide resolved
psychedelicious
psychedelicious reviewed Aug 20, 2024
View reviewed changes
pyproject.toml Outdated Show resolved Hide resolved
@brandonrising brandonrising force-pushed the brandon/flux-model-loading branch 2 times, most recently from 5606eea to 5edec7f Compare August 21, 2024 13:11
RyanJDick
RyanJDick reviewed Aug 22, 2024
View reviewed changes
pyproject.toml Outdated Show resolved Hide resolved
RyanJDick and others added 25 commits August 26, 2024 19:21
@RyanJDick @brandonrising
Update the T5 8-bit quantized starter model to use the BnB LLM.int8()…
54c48c3 
… variant.
@RyanJDick @brandonrising
Remove all references to optimum-quanto and downgrade diffusers.
5af214b
@RyanJDick @brandonrising
Update docs for T5 quantization script.
f4612a9
@RyanJDick @brandonrising
Move quantization scripts to a scripts/ subdir.
18c0ec3
@RyanJDick @brandonrising
Downgrade revert torch version after removing optimum-qanto, and othe…
098db5c 
…r minor version-related fixes.
@brandonrising
Update t5 encoder formats to accurately reflect the quantization stra…
dbdd851 
…tegy and data type
@RyanJDick @brandonrising
Switch the CLIP-L start model to use our hosted version - which is mu…
1d6c83b 
…ch smaller.
@RyanJDick @brandonrising
Replace swish() with torch.nn.functional.silu(h). They are functional…
1413ff9 
…ly equivalent, but in my test VAE deconding was ~8% faster after the change.
@brandonrising
Setup scaffolding for in progress images and add ability to cancel th…
bd1b37d 
…e flux node
@brandonrising
Remove dependency on flux config files
d159fe6
@RyanJDick @brandonrising
ruff
877b88e
@RyanJDick @brandonrising
Remove flux repo dependency
ae94e48
@RyanJDick @brandonrising
Downgrade accelerate and huggingface-hub deps to original versions.
f046a38
@RyanJDick @brandonrising
ruff format
9f6f404
@RyanJDick @brandonrising
Remove outdated TODO.
9a530c7
@brandonrising
Only install starter models if not already installed
bb80697
@brandonrising
Remove in progress images until we're able to make the valuable
642a953
@brandonrising
Remove no longer used code in the flux denoise function
a90d098
@brandonrising
Fix type error in tsc
40a3fa5
@brandonrising
Run ruff
b9238b6
@brandonrising
Rename params for flux and flux vae, add comments explaining use of t…
5a5ca10 
…he config_path in model config
@brandonrising
update default workflow for flux
bf59ab3
@brandonrising
remove prompt
bd2692b
@brandonrising
Run ruff
5d42e67
@brandonrising
default workflow: add steps to exposed fields, add more notes
c510234
@brandonrising brandonrising merged commit 50085b4 into main Aug 27, 2024
14 checks passed
@brandonrising brandonrising deleted the brandon/flux-model-loading branch August 27, 2024 00:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@RyanJDick RyanJDick RyanJDick left review comments

@StAlKeR7779 StAlKeR7779 StAlKeR7779 left review comments

@psychedelicious psychedelicious psychedelicious left review comments

@hipsterusername hipsterusername hipsterusername approved these changes

@maryhipp maryhipp maryhipp approved these changes

@blessedcoolant blessedcoolant Awaiting requested review from blessedcoolant blessedcoolant is a code owner

@lstein lstein Awaiting requested review from lstein lstein is a code owner

@ebr ebr Awaiting requested review from ebr ebr is a code owner

Assignees
No one assigned
Labels
backend PRs that change backend files CI-CD Continuous integration / Continuous delivery frontend PRs that change frontend files invocations PRs that change invocations python PRs that change python files python-deps PRs that change python dependencies python-tests PRs that change python tests Root services PRs that change app services
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@brandonrising @psychedelicious @maryhipp @StAlKeR7779 @RyanJDick @hipsterusername