Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hide the tongue better #375

Open
simplyexperimental opened this issue Sep 7, 2024 · 0 comments
Open

Hide the tongue better #375

simplyexperimental opened this issue Sep 7, 2024 · 0 comments

Comments

@simplyexperimental
Copy link

simplyexperimental commented Sep 7, 2024

I have encountered this issue in some other programs that are similar - they really struggle with the tongue when the tongue is sticking out a lot. The result is some half blend where the original tongue is still partially overlaying the skin.

I understand that it's very difficult maybe to simulate a tongue but perhaps at least live portrait can recognize that a tongue is sticking very far out and make the decision to completely put the tongue back into the mouth so it will not impact the final video swap.

Even the keep lip silent struggles with this - if the tongue sticks out far enough it will still be present under the closed lips to some degree. The keep lips silent also has a bug where it makes the head shake uncontrollably (reported issue before).

This is example of how it usually looks which you can see is a problem that i think we can solve by at least putting tongue back in the mouth.
As you can see in the second image if at any time in the DRIVING VIDEO the person puts their tongue back into their mouth then the TARGET VIDEO will put their tongue back in as well and it looks fine but of course we want to make sure that as long as live portrait sees a tongue sticking all the way out and doesn't know what to do with it - that it will delete the tongue or move it as going back inside the mouth so it can be filtered so tongue out doesn't affect end result.
AI's Challenge

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant