Would it be possible to output the "gaps" as a separate mask or fill them with a solid color instead of the current tiling modes?

[ProtoBelisarius]

I have little knowledge on how shaders work, but I would like to use LaMa to inpaint the gaps instead of the current solutions with repeat/mirror.

So I was wondering if its feasible to output the frame with whatever tiling mode selected and a black/white mask depicting the gap.

Or directly fill the gaps in the frame with bright green or something instead of a tiling mode.

[Tremeschin]

Excellent question! 🙂

I've tried this in the past and couldn't get it just right, but also didn't try enough haha

I know a way to quantify the gaps, which brings me back to good old calculus 3 days. We need to compute the gradient of each surface point (a normal vector with extra info), the more this vector is tilted to the sides, the more we're on a cliff in the scene

Calculating the tilt is easy, it's the angle between the gradient and the scene's plane, but defining what are gaps based on the cliffs isn't inherently mathematical and subjective. We'll get a value between 0 and 1 (after normalization) telling how steep a given pixel takes place on the scene's surface, which has zero knowledge of the surroundings and could give false positives

I'll give this a second chance in time, as I probably messed up the gradient calculation before

[Tremeschin]

This seems quite accurate (?), would you say it works for the use case of inpainting?

video.mp4

I don't think I can do much better than this math-wise with my current shader experience, the artifacts are mostly due the relatively low resolution depth maps, and edges are.. complicated 😅