Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add alternate morphing functions to prompt-morph. #4

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

kdreibel
Copy link

@kdreibel kdreibel commented Nov 13, 2022

Overview

prompt_morph.py currently always morphs between each prompt in a linear way. This PR adds some alternate morphing functions in order to spend more/less time at keyframes, depending on the user's preference.

Comparison

Here is a series of gifs illustrating the difference. The gifs are stuck together so that the keyframes should always synchronize. The circle and square are the keyframes, the hands (?) generally appear in the middle.

A (red circle|blue square), viewed from the side. Clip art. Steps: 28, Sampler: Euler a, CFG scale: 28.5, Seed: 713180077

morphin-time

From left-to-right: Linear, Sine, S-Parabola, Parabolic, Parabolic Bounce

x/y graphs

Linear

This is the current behavior.

linear

Sine (but secretly Cosine)

Start slow, speed through the middle, then slow again. The difference from Linear is barely noticeable, though.

sine

S-parabola

Start and end fast, linger in the middle for a bit. NOTE: I am pretty sure there is a proper mathematical name for this, but I don't know it.

S-Parabola

Parabolic

Start slow and accelerate. Drop back to zero velocity on every keyframe.

parabolic

Parabola bounce

Alternate between accelerating and decelerating. If linear would cause a ball to move up-down-up at steady velocity, Parabola bounce would cause it to look like it was bouncing off the ground like a normal ball affected by gravity. In the gifs above, you can see that (4) and (5) are identical from circle to square, but then differ on the subsequent morph.

Parabolic bounce

Discussion / Rationale

  • My theory was that if there was a morph where e.g. a basketball went from someone's hand to the ground, the parabolic bounce would allow it to better simulate gravity. But then it turned out I couldn't find any sort of prompt pair to generate that kind of behavior.
  • Another use case was for animating body movement. person with arms up -> person with arms down. The morphs didn't produce expected movement though.
  • I guess my question is, the original rationale behind this PR didn't pan out as expected, but is the functionality still useful?

Future work

  • I liked the idea of the Sine function spending more time near the keyframes, and "glossing over" the chaotic parts between keyframes. But it's hardly noticeable. Perhaps I'll revisit this to find a better function to meet that criteria.
  • On the other hand, maybe we should scrap the dropdown entirely, and support the complex inline math that Deforum uses?

Also also wik

  • A line starting with # will be treated as a comment, and ignored.

@kdreibel kdreibel marked this pull request as ready for review November 13, 2022 08:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant