Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ui): upscaling tab #6633

Merged
merged 43 commits into from
Jul 23, 2024
Merged

feat(ui): upscaling tab #6633

merged 43 commits into from
Jul 23, 2024

Conversation

maryhipp
Copy link
Collaborator

@maryhipp maryhipp commented Jul 19, 2024

Summary

A new tab for multidiffusion upscaling ✨

  • Requires installed spandrel image-to-image model and tile controlnet for the base model used in multidiffusion step

Related Issues / Discussions

QA Instructions

Merge Plan

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • Documentation added / updated (if applicable)

@github-actions github-actions bot added the frontend PRs that change frontend files label Jul 19, 2024
@psychedelicious
Copy link
Collaborator

psychedelicious commented Jul 19, 2024

I fixed a couple things and broke others.

I "fixed" a problem where scale would be null if the upscale model used didn't have a specific pattern of x then a number. I "fixed" it by removing that parameter. This breaks the size preview.

Also removed the resize node, because it wanted scale. This breaks when the image size doesn't end up as a multiple of 8.

I'll propose an alternative way to handle the initial upscaling logic - create a new node:

  • Accepts image, model, tile_size, scale: float, overshoot: float, fit_to_multiple_of_8: bool
  • It runs the model until upscaled_image.width >= scale * overshoot, then uses a simple resize to the nearest multiple of 8 of the scale.
  • overshoot = 1 means you do minimal upscaling
  • Higher values let the upscaler generate more detail, which results in finer details after scaling back down, which I found yields better results.

This approach also reduces the number of intermediate images created, which improves perf - a 8k image is about 100MB and will take many seconds to save.

It also means you can precisely tune how large you want your output to be, and eliminates the need to figure out upscale model's scale factor.

@maryhipp
Copy link
Collaborator Author

@psychedelicious Is it not safe to assume that these models will have x# in the name? Or just default to 2 if it doesn't?

@maryhipp maryhipp marked this pull request as ready for review July 22, 2024 20:13
@github-actions github-actions bot added python PRs that change python files invocations PRs that change invocations labels Jul 23, 2024
@psychedelicious psychedelicious enabled auto-merge (rebase) July 23, 2024 00:52
@psychedelicious psychedelicious merged commit 8756a6b into main Jul 23, 2024
14 checks passed
@psychedelicious psychedelicious deleted the maryhipp/upscaling-tab branch July 23, 2024 00:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
frontend PRs that change frontend files invocations PRs that change invocations python PRs that change python files
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants