Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Copy UV coordinates from a reference mesh. #6625

Open
3 tasks done
ssheorey opened this issue Jan 26, 2024 · 5 comments · May be fixed by #6740
Open
3 tasks done

Copy UV coordinates from a reference mesh. #6625

ssheorey opened this issue Jan 26, 2024 · 5 comments · May be fixed by #6740

Comments

@ssheorey
Copy link
Member

ssheorey commented Jan 26, 2024

Checklist

Proposed new feature or change

In many mesh processing workflows, UV coordinates are not preserved (e.g. mesh simplification). UV coordinates are required for many downstream tasks. One way to propagate UV coordinates through mesh processing workflows is by copying them from the original mesh. Here is an outline:

  1. Mesh A is processed to mesh B. Mesh A contains UV coordinates (and textures), but mesh B does not have them.
  2. Call meshB.InterpolateTextureCoordinatesFrom(meshA). This does:
  • Create a ray casting scene with mesh A.
  • For each vertex in mesh B, find nearest point in mesh A.
  • Assign UV coordinates of nearest point in mesh A to the vertex in mesh B.
  • Optionally, copy over any textures from mesh B to mesh A.

References

No response

Additional information

It may be necessary to convert from per-triangle UVs to per-vertex UVs (or vice versa). The following code shows an example of how to make that conversion: https://github.com/isl-org/Open3D/blob/main/cpp/open3d/visualization/rendering/filament/TriangleMeshBuffers.cpp#L614

@ssheorey
Copy link
Member Author

Related: #6615

@Amandeep4282
Copy link

Greeting,
I am new to contributing to Open Source.
I would like to give it a try and try to resolve this issue if I may.

Could you please provide me some brief info about the issue and location of files/folder in which it need to be fixed.

@cdbharath
Copy link
Contributor

cdbharath commented Mar 16, 2024

I implemented this feature and was testing it. I noticed the following anomaly that I am not sure how to resolve.

To test the code, I performed the following

  1. Load a sphere mesh
  2. Apply FilterSmoothTaubin filter
  3. Use InterpolateTextureCoordinatesFrom to interpolate the UV coordinates to the filtered mesh
  4. Visualize the original and the filtered mesh

The following images visualizes the textured mesh before and after filtering + UV interpolation
Before filtering + UV interpolation:
Screenshot

After filtering + UV interpolation:
Screenshot
After filtering and UV interpolation, a single portion of the sphere doesn't have the same texture as the original.

This happens because per-triangle UVs and per-vertex UVs don't map one to one while converting the former to latter. An example is the following texture image from Open3D documentation. The UVs for same mesh vertex could be at the opposite end of the texture image. This results in the entire portion of the image being compressed and assigned to the single stretch of triangles.

Screenshot

How would you suggest I handle this case?

@shanagr
Copy link
Contributor

shanagr commented Mar 24, 2024

Disclaimer: I'm new to Open3D as well, so take this as an idea to explore. Someone more experienced can probably help you better.

Assume we have a way to detect if the vertices of a triangle have "crossed the edge and wrapped around". Let, for example, y1 and y2 be two y coordinates in a triangle, and we know y2 has fallen off the edge. Instead of using y1 we can use 1.0 + y1 in our computations (and take the result "modulo" 1.0 in the end once all computations are done).
How do we know if something has fallen off the edge? The simplest heuristic I can think of is check if |y1 - y2| > 0.5. If yes, we use 1.0 + min(y1, y2) instead of min(y1, y2).

@cdbharath cdbharath linked a pull request Apr 4, 2024 that will close this issue
9 tasks
@ssheorey ssheorey moved this to In progress in Open3D 2024 Apr 29, 2024
@ssheorey ssheorey moved this from In progress to In review in Open3D 2024 Apr 29, 2024
@ssheorey ssheorey added this to the v0.19 milestone Apr 29, 2024
@ssheorey ssheorey added the mesh label Apr 29, 2024
@dbs4261
Copy link
Contributor

dbs4261 commented Jul 15, 2024

I'm wondering if I am using this correctly. I tried applying this to a mesh that I then applied quadric simplification to and the render didn't come out making sense. The dataset is about 300 MB and is derived from the ODM Test Dataset Sheffield Cross, I did modify it to bake the texutres into a single atlas. I then used this code snippet to try coping the UV coordinates.

int main(int argc, char** argv) {
    const std::string source_path = "combined_mesh.obj";

    open3d::geometry::TriangleMesh mesh;
    if (!open3d::io::ReadTriangleMesh(source_path, mesh, {})) {
        open3d::utility::LogError("Failed to read {}", source_path);
    }

    open3d::geometry::TriangleMesh processed = mesh;
    processed.materials_.clear();
    processed.textures_.clear();
    processed.triangle_material_ids_.clear();
    processed = *processed.SimplifyQuadricDecimation(
            static_cast<std::size_t>(processed.triangles_.size() * 0.5),
            std::numeric_limits<double>::max(), 100.0);

    processed.InterpolateTextureCoordinatesFrom(mesh);
    const std::string out_path = "UVTransfered.obj";
    if (!open3d::io::WriteTriangleMesh(out_path, processed)) {
        open3d::utility::LogError("Failed to write to {}", out_path);
    }
    return EXIT_SUCCESS;
}

Here is a screenshot of the input mesh:
Image
And here was the output (after I updated the mtl file):
Image

I was expecting to see some issues around texture seams, but this strange result makes me think I applied it wrong. Thoughts?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: In review
Development

Successfully merging a pull request may close this issue.

5 participants