Mouth Percent Open Gesture

[nskaiser]

Hi Vlad,

Do you have any recommendations on how to get better results out of the "Mouth % Open" gesture? I am trying to determine if someone is speaking and use the microphone volume paired with the percentage of mouth open (can't rely just on microphone volume as it could be system audio, etc). It works reasonably well, but there are quite a lot of times when there's no % returned (or the opposite, it thinks the mouth is open when it's really not). Just wondering if you had any thoughts on ideas to explore for this? Other than modifying your 10% threshold, anything else you can think of? Thanks!

[vladmandic]

couple of ideas...

first, make sure to squeeze every bit of performance - run detection in a webworker and make sure that any unnecessary modules are disabled (iris, description, emotion, etc.)

second, dont treat each detection separately
mouth opens and closes pretty fast during speech - set "mouth open" variable when its first detected and set a timer on it, lets say 200ms - each time another mouth open is detected, reset a timer. if person stops speaking, timer will run out in 200ms and that can be considered as end event when variable flips back to closed

third, i can take a look at the actual implementation for accuracy - been a while since i've implemented this.

[nskaiser]

Thank you Vlad. Good tips.