Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the VAE tile size configurable for tiled VAE #6555

Merged
merged 3 commits into from
Jul 2, 2024

Conversation

RyanJDick
Copy link
Collaborator

@RyanJDick RyanJDick commented Jun 28, 2024

Summary

  • This PR exposes a tile_size field on ImageToLatentsInvocation and LatentsToImageInvocation.
    • Setting tile_size = 0 preserves the default behaviour.
  • This feature is primarily intended to support upscaling workflows that require VAE encoding/decoding high resolution images. In the future, we may want to expose the tile size as a global application config, but that's a separate conversation.
  • As a general rule, larger tile sizes produce better results at the cost of higher memory usage.

Example:

Original (5472x5472)
orig

VAE roundtrip with 512x512 tiles (note the discoloration)
vae_roundtrip_512x512

VAE roundtrip with 1024x1024 tiles (some discoloration still present, but less severe than at 512x512)
vae_roundtrip_1024x1024

Related Issues / Discussions

Related: #6144

QA Instructions

  • Test image generation via the Linear tab
  • Test VAE roundtrip with tiling disabled
  • Test VAE roundtrip with tiling and tile_size = 0
  • Test VAE roundtrip with tiling and tile_size > 0

Merge Plan

No special instructions.

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • Documentation added / updated (if applicable)

@github-actions github-actions bot added python PRs that change python files invocations PRs that change invocations backend PRs that change backend files python-tests PRs that change python tests labels Jun 28, 2024
@RyanJDick RyanJDick force-pushed the ryan/tiled-vae-investigation branch from e7a5f78 to 3752509 Compare July 2, 2024 13:07
@RyanJDick RyanJDick enabled auto-merge July 2, 2024 13:07
@RyanJDick RyanJDick merged commit e9936c2 into main Jul 2, 2024
14 checks passed
@RyanJDick RyanJDick deleted the ryan/tiled-vae-investigation branch July 2, 2024 13:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend PRs that change backend files invocations PRs that change invocations python PRs that change python files python-tests PRs that change python tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants