Is it possible to get more shots from various angles ?

[altava-sgp]

At first thanks for your great job! 👍

I have a question.
Is it possible to get more shots from various angles ?
A only single shot is possible ?

Please excuse me if I am missing anything.

[haodong2000]

Hi @altava-sgp thanks so much for your interest! Sorry for my late response.

Do you mean to get multi views of depth and normal given a single RGB image?

For depth estimation: you can lift the depth maps into 3D point clouds using camera intrinsics and then render new depth maps or RGB from different views. As shown in this issue.

For normal, if you want mulri views of normal maps, you probably should do: normal -> depth -> 3D point clouds -> depth (new view) -> normal (new view). You can refer this repo for normal2depth and this repo for depth2normal.

Best,

[altava-sgp]

@haodong2000 Thanks for your answer. I will try as you said. 👍

[altava-sgp]

@haodong2000 I also hope Lotus can generate various normals like I said.
It will be much helpful for AI generation.
Please consider this seriously. 🙏

[altava-sgp]

Like these.

[haodong2000]

Hi @altava-sgp , thanks for response.

I get your point. Currently, Lotus only supports generating relative normal maps only for the current view. That is the way we feed the GTs during training.

If you require normal maps that remain consistent across different views, I recommend leveraging camera extrinsics to transform the normal maps into world coordinates. It's training-free.

I am not sure how precise and consistent you need the normal maps to be or what kind of application you are aiming to build. This simple solution may struggle to achieve perfectly consistent multi-view normal maps. If you require absolute consistency, an alternative approach could be to first reconstruct the 3D model and then generate the normal maps through rendering.

[altava-sgp]

@haodong2000 Thank you for your kind answer.
Actually, I am trying to generate a good 3D model from a single image.
I will try other methods, including the suggestions you gave. 😄

[haodong2000]

Hi, thanks for response!

Actually I have tried to reconstruct 3D mesh using purely relative normal and depth maps, please see Fig. 14 of the paper for more details.

It seems that the performance is good but not in very high-quality.

Thus, I believe that by aligning all views into same (world) coordinates, the performance will boost significantly.

Best,