Implement CustomDiffusionAttnProcessor2_0.

[eliphatfs]

What does this PR do?

Fixes #4588

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Did you read our philosophy doc (important for complex PRs)?
• Was this discussed/approved via a Github issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

@sayakpaul @nupurkmr9

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[sayakpaul]

@eliphatfs let me know if this PR is ready for review.

[eliphatfs]

Hi, how do I do the ko-documentation? I don't know any Korean.
Edit: guessing from the file structure it seems there is nothing to do with the attention processors. I will just skip then.

[eliphatfs]

I don't find existing tests for CustomDiffusionAttnProcessor, so I will skip adding that for 2_0.

[sayakpaul]

Looking good, thank you!

• Could we add this conditional initialization of this class in the training script? Here's an example: https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/train_dreambooth_lora.py#L876C1-L880C1.

@yiyixuxu could you also give this a look?

[eliphatfs]

The CI complains about my code style. How do I spot the problem?

[sayakpaul]

You can run make style && make quality from you local fork of diffusers.

Guidance: https://github.com/huggingface/diffusers/blob/main/CONTRIBUTING.md

[eliphatfs]

Sorry I didn't read the guide carefully; I have updated the code formatting, but there are extra errors when I wrong make quality that seems nothing to do with me:

src/diffusers/pipelines/audio_diffusion/pipeline_audio_diffusion.py:181:12: E721 Do not compare types, use `isinstance()`
src/diffusers/pipelines/stable_diffusion_xl/pipeline_stable_diffusion_xl.py:793:42: E721 Do not compare types, use `isinstance()`
src/diffusers/pipelines/stable_diffusion_xl/pipeline_stable_diffusion_xl_img2img.py:878:20: E721 Do not compare types, use `isinstance()`
src/diffusers/pipelines/stable_diffusion_xl/pipeline_stable_diffusion_xl_inpaint.py:1083:20: E721 Do not compare types, use `isinstance()`
tests/pipelines/consistency_models/test_consistency_models.py:190:12: E721 Do not compare types, use `isinstance()`
tests/pipelines/unidiffuser/test_unidiffuser.py:112:12: E721 Do not compare types, use `isinstance()`
tests/pipelines/unidiffuser/test_unidiffuser.py:548:12: E721 Do not compare types, use `isinstance()`
tests/pipelines/unidiffuser/test_unidiffuser.py:651:12: E721 Do not compare types, use `isinstance()`

[sayakpaul]

Cc @DN6 could you help here?

[DN6]

@eliphatfs What version of ruff do you have installed? You can check by running ruff -V.
Could you share the output of that command please?

[eliphatfs]

Oh it was 0.0.284; after installing 0.0.280 as specified the errors vanished. 0.0.280 was not installed on my machine due to some version clash in flax which refers to orbax package that is no longer installable: google/orbax#436
I have fixed all style problems. Thanks for your help!!

[eliphatfs]

• Could we add this conditional initialization of this class in the training script? Here's an example: https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/train_dreambooth_lora.py#L876C1-L880C1.

Done!

[eliphatfs]

Forgot to update loaders. Now test_examples pass on my local clone.

[sayakpaul]

@DN6 could you give this a look?

[eliphatfs]

Bump?

[sayakpaul]

We are doing two major refactors just as an FYI which might influence this PR:

#4473
#4765

Let's maybe revisit this after that? WDYT?

[eliphatfs]

I see the PRs you mentioned have been merged. Shall we continue working on this thing?

[sayakpaul]

Nice this works for me. Thank you so much for implementing this!

[sayakpaul]

Ah, we have some failures on the CI and merge conflicts. Let's make sure to resolve them.

[eliphatfs]

I have resolved the conflict; looking into the CI failures.

[eliphatfs]

Sorry but I request a re-run of the tests. It seemed to me that the errors are not relevant to this PR. CI is complaining about a field missing in a huggingface API result. I guess it may be due to API changes and since I have merged the changes in diffusers main branch maybe we can retry the tests.

[patrickvonplaten]

Ok for me. @sayakpaul @DN6 @yiyixuxu wdyt?

[sayakpaul]

Okay for me to merge once the CI is green (except the documentation workflow).