switch default runtime to tfjs

[ziyuan-linn]

This PR changes the default runtime of handPose, bodySegmentation, faceMesh, and bodyPose to tfjs as discussed in #134.

[shiffman]

Everything looks good, however, I may have found a small issue with the BodyPose model? The mediapipe runtime performs smoothly:

But when I switch to this branch, there is an additional jitter / gltichiness to the results:

This is on Mac OS with chrome. Anyone else seeing this?

(gifs recorded at SFO ✈️ 😂)

[shiffman]

Oh, the above is with blazepose, I'm not seeing the same issue with MoveNet actually (I don't think?). I'll have to test this more thoroughly later!

[ziyuan-linn]

I remember looking into this a while ago. The enableSmoothing option is true by default for bodyPose. However, the smoothing algorithm is not implemented on the tfjs runtime.

Perhaps we can implement it within ml5? I think mediapipe runtime uses a temporal smoothing algorithm, which should be fairly simple to implement.

[shiffman]

I remember looking into this a while ago. The enableSmoothing option is true by default for bodyPose. However, the smoothing algorithm is not implemented on the tfjs runtime.

Perhaps we can implement it within ml5? I think mediapipe runtime uses a temporal smoothing algorithm, which should be fairly simple to implement.

Ah, ok, that makes sense! I think we could leave this to later especially since if someone needs it they could manually switch the runtime to mediapipe, is that correct? We should find a place to document this somewhere and perhaps open a new issue for future work / investigation? I don't think this needs to be completed before the upcoming soft launch.

Also, the MoveNet model performs quite stably even without the smoothing.

Hi webteam @QuinnHe @MOQN @myrahsa @alanvww regarding how to document the runtime choices and info about smoothing? Or perhaps this is something we put on a temporary wiki page and link from the documentation as it is not core to the beginner experience?

[QuinnHe]

I remember looking into this a while ago. The enableSmoothing option is true by default for bodyPose. However, the smoothing algorithm is not implemented on the tfjs runtime.
Perhaps we can implement it within ml5? I think mediapipe runtime uses a temporal smoothing algorithm, which should be fairly simple to implement.

Ah, ok, that makes sense! I think we could leave this to later especially since if someone needs it they could manually switch the runtime to mediapipe, is that correct? We should find a place to document this somewhere and perhaps open a new issue for future work / investigation? I don't think this needs to be completed before the upcoming soft launch.

Also, the MoveNet model performs quite stably even without the smoothing.

Hi webteam @QuinnHe @MOQN @myrahsa @alanvww regarding how to document the runtime choices and info about smoothing? Or perhaps this is something we put on a temporary wiki page and link from the documentation as it is not core to the beginner experience?

Thanks Dan! Yes, we will take care of it after the soft launch. Just throw an idea for our future reference - maybe add runtime in ml5 glossary, and add a lightbulb tip within Methods of model pages to link to the ml5 glossary page.

[shiffman]

Thanks Dan! Yes, we will take care of it after the soft launch. Just throw an idea for our future reference - maybe add runtime in ml5 glossary, and add a lightbulb tip within Methods of model pages to link to the ml5 glossary page.

Yes, I love this idea! I'm going to merge this so @QuinnHe let's open an issue in the docs repo for tracking later and @ziyuan-linn perhaps you can also open an issue in this repo for us to track the smoothing discussion as related to this and #69 and #129.