About the ROI

[liangzhenghao2000]

The landmarks detected in this project are sometimes inaccurate and jittery.After reading the code, I think it is because the ROI here is inconsistent with the ROI croped by the original processing method of mediapipe.In addition to find the face area, I think it is necessary to do affine transformation to the area before feed it to landmark model.Also, mediapipe uses the landmark-refine model, is it possible to implement it in this project?

[pntt3011]

Hi @liangzhenghao2000,

I appreciate your opinion about the ROI cropped. According to the face detection graph, I think that the affine transform at the end is to project the detections from [0, 1] to the origin size. We should dig into their implementation to find out what they really do.

Another thing I just found is that they resize the input images but keeping the aspect ratio and pad the shorter size with zero while I resize them directly.

About the landmark attention model, it is also mentioned in #12 that we must customize the tensorflow lite lib and recompile it.

However, I did this project last year for my hobby and didn't expect it to get this much attention. I'm sorry that I cannot help you further.

[liangzhenghao2000]

Thank you for your reply! If I have time I will go through their papers to confirm these.