How to match rendered face area in image (segmentation mask matching?)

[timlod]

I'm using pytorch3d inverse rendering to fit a model that deforms a mesh.
I would like to ensure that certain rasterized faces match with a pre-computed segmentation mask, however the way I think about the operation is not differentiable (which would be indexing pix_to_face with the faces I know belong ie. to the eyes of a face, and then comparing the resulting boolean mask to the eye segmentation as output by another model.

Is there some differentiable way in pytorch3d to match the 'true' segmentation mask of a render (as taken from knowledge about the mesh, and then using pix_to_face) to one computed on the target image?

Thanks!

[timlod]

Quick note - I wasn't aware that discussions were being used (few projects use them so unfortunately I ignored this section), since when opening an issue it gives you this option:

❓ Questions
How do I do X with PyTorch3D? How does PyTorch3D do X?

@patricklabatut It may be good to remove this from the options for creating issues to prevent questions springing up there which are just moved to discussions.