RAUNet and MSW-MSA for Forge - new feature in the dev - upstream

[Torcho12]

Hi dear reForgers,
Could someone please explain how this new RAUNet and MSW-MSA for Forge feature works?

[Panchovix]

Some users were requesting to add HiDiffusion into Forge, so I adapted it from comfy https://github.com/blepping/comfyui_jankhidiffusion into reforge! There you can read about how it works.

Also, thanks @blepping for your good work on the implementation.

[blepping]

@Panchovix

Some users were requesting to add HiDiffusion into Forge, so I adapted it from comfy https://github.com/blepping/comfyui_jankhidiffusion into reforge! There you can read about how it works.

Also, thanks @blepping for your good work on the implementation.

you're definitely welcome to use it, but since this is basically an exact copy of my code (at the moment at least) and not just referenced from it or something like that, please include some kind of attribution. since my implementation was based on the original HiDiffusion project's version they should be credited also. something like a README.md file in the extension directory saying:

---

Credits

Based on implementation from https://github.com/blepping/comfyui_jankhidiffusion

Which was based on https://github.com/megvii-research/HiDiffusion

---

would be fine or a comment in the source code at least.

[Panchovix]

Sure! Just added the credits on a new readme in the extensions itself.

[blepping]

Sure! Just added the credits on a new readme in the extensions itself.

thanks! i appreciate it.

if you have any questions about my implementation or whatever that would help you adapt it to forge, please feel free to ask. i'll help if i can. (also if you happen to find out why it doesn't seem to work very well with SDXL i'd love to get a ping, still haven't managed to resolve that issue.)

[Panchovix]

Thanks! Well in the ruanet script I adapted to the gradio UI, and there adapt to disable RAUNet/MSW-MSA, but not sure if it's working correctly. Maybe it does but it involves a lot of UnetPatcher.

About SDXL, I think RAUNet works fine for semi-high resolutions, but very high ones seems to add some artifacts I think.

MSW-MSA seems to work in general good on SDXL, well it isn't much difference in output but performance is some % faster.

[blepping]

Thanks! Well in the ruanet script I adapted to the gradio UI, and there adapt to disable RAUNet/MSW-MSA, but not sure if it's working correctly. Maybe it does but it involves a lot of UnetPatcher.

MSW-MSA attention is just a normal patch so if your model patching backend works like Comfy then there shouldn't be any problem.

RAU-net is the weird one because it has to replace the up/down blocks so the important thing is that it gets called at least once with enabled set to False when disabled. it looks like you're already doing that so i think that part shouldn't be an issue either.

About SDXL, I think RAUNet works fine for semi-high resolutions, but very high ones seems to add some artifacts I think.

i personally haven't had much success even with pretty mild settings like 1792x1792.

MSW-MSA seems to work in general good on SDXL, well it isn't much difference in output but performance is some % faster.

interesting. it's a big speed increase on SD1.5 (for me at least) but i didn't notice much difference with SDXL. it does seem to help with artifacts though, i'd actually just use it as a quality enhancer even if it was slower.