Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lip sync error #1

Open
omcar17 opened this issue Feb 21, 2019 · 0 comments
Open

Lip sync error #1

omcar17 opened this issue Feb 21, 2019 · 0 comments

Comments

@omcar17
Copy link

omcar17 commented Feb 21, 2019

Hello,
Thank you for the excellent work and publicly available code.
I am using syncnet to find if there is lip-sync error in the video. I am getting very random values of AV offset and confidence, after using the train weights available on official website.

I am confused about this paragraph from the paper -

Determining the lip-sync error -
To find the time offset between the audio and the video, we take a sliding-window
approach. For each sample, the distance is computed between one 5-frame video
feature and all audio features in the ± 1 second range. The correct offset is when
this distance is at a minimum. However as Table 2 suggests, not all samples in
a clip are discriminative (for example, there may be samples in which nothing
is being said at that particular time), therefore multiple samples are taken for
each clip, and then averaged.

I am missing something in this paragraph. How do I collect multiple samples for each clip?
I would like to know how to get a proper value of metric (AV offset, Confidence) that show the out of sync of video and audio on sample.

Thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant