I don't think there is anything Forge specific here.
Go to the Extensions tab, then Install from URL, use the URL for this repository.
Easiest way to ensure necessary versions are installed is to edit requirements.txt or requirements_versions.txt in the webUI folder.
diffusers>=0.28.1
tokenizers>=0.19
transformers==4.40
peft
These upgrades didn't break anything for me.
current UI
At your own risk. This is barely moderately somewhat tested, and even then but only on my computer.
Models will be downloaded automatically, on demand (so if you never generate with the 256 model, it'll never be downloaded). The T5 text encoder is around 18Gb and the image models are about 2.3Gb each.
The models include the float32 T5 model, which I convert to fp16 on first use and save the result for all future runs. This conversion and saving is automatic. Once done, the full size float32 model could be deleted to reclaim some space.
Note that Alpha and Sigma both use the same T5 text encoder, my caching system means it should only be downloaded once regardless for how many models are used.
I can generate using all models, though the 2K Sigma model is very slow on my limited hardware. It's undertrained anyway, at present.
- couple of bugfixes, thanks to @BlipOnNobodysRadar for reports.
- improved yesterday's effort. More compatibility, multi-line, etc.
- some code cleanups
- added prompt parsing to automatically fill in details like seed, steps, etc.
- tweaked Florence-2: model now runs on GPU so is faster.
- added huggingface emoji button to toggle sending of huggingface access token. Some repos may be gated, requiring acceptance of terms before use and an access token. Downloading/updating these models is not possible without these steps, but already downloaded models will continue to work
(current example: ptx0/pixart-900m-1024-ft). The official models are not gated.- Sign up / log in, go to your profile, create an access token. Copy it. Make a textfile called
huggingface_access_token.txtin the main webui folder, i.e.{forge install directory}\webui, and paste the token in there. (same instructions as for SD3, only needs done once).
- Sign up / log in, go to your profile, create an access token. Copy it. Make a textfile called
- added option to caption using Florence-2, in image to image section. 'P' button toggles overwriting prompt, results always written to console.
- minor code improvements
- tweaked logic for model identification. Added new finetune, seems likely to continue to be updated, ptx0/pixart-900m-1024-ft to models list. Info
- settings to colourize the initial noise. This is essentially free extra control. Leave strength at 0.0 to bypass it. Doesn't seem as effective with PixArt as with other models, but still something to experiment with.
- added modelsListPA.py in extension directory - edit in any text editor to hide models you don't care about or add new models (must be huggingface diffusers style checkpoints)
- added button to toggle resolution binning, enabled by default and generally best left enabled IMO. I was testing disabling it for potential hires fix type uses, with some limited success around x1.5 upscale. x2 was messy. Maybe repeated smaller upscales could work, but probably better to send to img2img and use an sdXL checkpoint to upscale.
- added flash diffusion. This is a distilled Lora (211MB) on top of Alpha1024: 4 steps, low CFG (1-2), seems much better than LCM. Forced LCM scheduler. Needs peft library.
- small update to work with diffusers >= 0.28.1 : Transformer2DModel is now PixArtTransformer2DModel
- override sampler for LCM and DMD in right place, a bit earlier than before
- added saving of DMD step in infotext when using DMD model; also don't save CFG and step count for DMD (would be redundant)
avoid error message when Set Width/Height from Image without an image loaded
- various tweaks. The K button enables karras sigmas for schedulers/samplers, which could be worth experimenting with.
- img2img
- UI reshuffle. again
- made using OpenAI consistency decoder VAE optional (previously DMD always used it, and other Alpha models never did, Sigma models can't) - it's VRAM greedy (for me only usable on 512 models) and not always better.
- added SA-solver, DPM SDE
- forced default sampler for LCM, DMD models
- forced empty negative, 1 step, 1 CFG for DMD
- cache VAE to share between models. Sharing is default behaviour for PixArt-Sigma, but the Alpha models each had their own copy. Now it'll only be downloaded the first time.
- added LCM, DMD
note: DMD doesn't install correctly - repository is missing tokenizer directory. Seems like the symlinks (follow the Snapshot folder) can simply be copied from another Alpha model.This shouldn't be an issue anymore, as I always fetch tokenizer from the same place rather than from each model.
- simple optimisation: if prompts not changed, text_encoder stage is skipped
- styles,
really should be in own file for easy editing
- support for PixArt-Alpha models - they use the same T5 text encoder.
- fixed
(but still commented out)saving of fp16 text encoder. Previously might not have saved in the right place.
- samplers
- captions in gallery (where linebreaks?)
- correct seeds for batches
prompt: portrait photograph, woman with red hair, wearing green blazer over yellow tshirt and blue trousers, on sunny beach with dark clouds on horizon
Thanks to:
- frutiemax92 for inference_pipeline.py
- benjamin-bertram for ui details