Skip to content

Conversation

@DN6
Copy link
Collaborator

@DN6 DN6 commented Nov 10, 2025

What does this PR do?

Removed the type cast in the AutoencoderKL Decoder in this PR in order to address graph breaks. The PR breaks some Mellon Nodes and default SDXL inference. The issue is that the casting is used to address an issue in the base SDXL VAE which overflows when running in FP16.

The overflow issue has been fixed for a while and almost all finetunes use the fixed VAE, so this logic that conditionally upcasts the upsample layers of the VAE is no longer needed. The memory savings from this selective layer casting in also quite minimal, so we can safely upcast the all layers in the VAE if casting is needed.

This PR

  1. Deprecates the upcast_vae method in the SDXL based pipelines that selectively upcasts only the upsample layers of the decoder in favour of upcasting the entire VAE is upcasting is required.

Fixes # (issue)

Before submitting

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@DN6 DN6 requested a review from yiyixuxu November 10, 2025 07:55
@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants