Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Registration] "Add new rigid & non-rigid registration algorithms" #6061

Open
yaoyx689 opened this issue Jun 6, 2024 · 8 comments
Open

[Registration] "Add new rigid & non-rigid registration algorithms" #6061

yaoyx689 opened this issue Jun 6, 2024 · 8 comments

Comments

@yaoyx689
Copy link

yaoyx689 commented Jun 6, 2024

Is your feature request related to a problem? Please describe.

Hello, I am a researcher working on point cloud registration algorithms.

I have noticed that PCL includes some rigid point cloud registration algorithms, such as ICP and GICP. However, these algorithms are not very recent, and the speed and accuracy of them can be further improved. Additionally, it appears that there is no implementation of non-rigid registration algorithms available in the library, which handle point clouds with dynamic deformations.

Context

I am wondering if you have considered adding recent rigid & non-rigid registration algorithms to PCL?

Recently, we have published two works:
[1] Juyong Zhang, Yuxin Yao and Bailin Deng. "Fast and robust iterative closest point." IEEE Transactions on Pattern Analysis and Machine Intelligence 44(7), 2021: 3450-3466.
[2] Yuxin Yao, Bailin Deng, Weiwei Xu and Juyong Zhang. "Fast and robust non-rigid registration using accelerated majorization-minimization." IEEE Transactions on Pattern Analysis and Machine Intelligence 45(8), 2023: 9681-9698.

**Expected behavior **

FRICP[1] can speed up ICP and improve its robustness. In most of the experiments conducted in [1], our accelerated version can reduce the computation time of the ICP algorithm by half, and our robust version can improve the accuracy of the ICP algorithm by approximately an order of magnitude. The algorithm has gained popularity among users of our open-source code.

AMM-NRR[2] is our recent publication on non-rigid registration, and its performance surpasses state-of-the-art methods in terms of both speed and accuracy when we publish our paper. (source code)

Describe the solution you'd like

I would like to know if you are interested in these two algorithms, and whether it would be possible for us to contribute them to the PCL library.

Looking forward to your reply.

@yaoyx689 yaoyx689 added kind: request Type of issue status: triage Labels incomplete labels Jun 6, 2024
@mvieth
Copy link
Member

mvieth commented Jun 7, 2024

Hi, first of all thank you for the offer. I am not so familiar with non-rigid registration, so I will look at FRICP first. I will check out the paper and the code, and will let you know my thoughts.

@mvieth mvieth added module: registration and removed status: triage Labels incomplete labels Jun 7, 2024
@yaoyx689
Copy link
Author

Okay, thank you for your consideration and reply.

@QiuYilin
Copy link
Contributor

Great!

@mvieth
Copy link
Member

mvieth commented Jun 15, 2024

I looked at the paper [1] and if I understood correctly, there are basically two parts: the Anderson acceleration and the robustness enhancement via Welsch's function. In PCL, ICP uses CorrespondenceEstimation to compute the correspondences and TransformationEstimationSVD to compute the transformation. TransformationEstimationSVD is a subclass of the abstract TransformationEstimation, which is the parent class of all transformation estimation methods in PCL (e.g. also point-to-plane methods). Did I understand correctly, that for the Anderson acceleration it does not matter how x^(k+1) is computed? If that is the case, I would propose to create a new subclass of TransformationEstimation, e.g. called TransformationEstimationAccelerated, which is a wrapper around another transformation estimation class given by the user. This new class would forward the correspondences to the wrapped transformation estimation, receive the next transformation x^(k+1), and compute the accelerated estimate x^(k+1)_AA.
The robust transformation estimation via Welsch's function could be another subclass of TransformationEstimation, for example called TransformationEstimationRobust.
What do you think? I believe this strategy would give users the most flexibility and would nicely integrate with PCL's other classes. I think it would be best to do this in two separate pull requests (one for the acceleration, one for the robust estimation), otherwise the pull request would be too big and reviewing would take too long. If you have any question, feel free to ask.

@QiuYilin
Copy link
Contributor

QiuYilin commented Jun 17, 2024

Other TransformationEstimations are named after mathematical methods. Would TransformationEstimationMM and TransformationEstimationWelschMM be better? Then class IterativeClosestPointFast Inherited from class IterativeClosestPoint uses TransformationEstimationMM and IterativeClosestPointRubust uses TransformationEstimationWelschMM. Of course new classes inherit from ICP are not necessary, I just want to express that the adjectives Fast and Rubust can be reflected on ICP types. Or we can add a new class named FastAndRubustIterativeClosestPoint that uses these two transformation methods to switch between Fast and Rubust options, which can preserve the proper names in the paper .

@yaoyx689
Copy link
Author

@mvieth Thank you for your reply, I think your suggestion is feasible.

Thanks @QiuYilin for your suggestion, but such naming is not easy to understand for people who are not very familiar with it, such as WelschMM, etc. These names may need further consideration. As for the specific implementation method, I am not sure whether it is more appropriate to add a new class or set an option.
@QiuYilin , I think you are more familiar with PCL's registration algorithm than I am. Are you interested in contributing this method to PCL with me?

@QiuYilin
Copy link
Contributor

QiuYilin commented Jun 19, 2024

Thanks @QiuYilin for your suggestion, but such naming is not easy to understand for people who are not very familiar with it, such as WelschMM, etc. These names may need further consideration. As for the specific implementation method, I am not sure whether it is more appropriate to add a new class or set an option.

What I mean is that general users can use the convenience class FastAndRobustIterativeClosestPoint, which is also the name in the paper. @mvieth What do you think of this?

image

@QiuYilin , I think you are more familiar with PCL's registration algorithm than I am. Are you interested in contributing this method to PCL with me?

I have been working on point cloud registration at the application level in recent days, and I am not sure whether MM and WelschMM accurately express the mathematical principles involved. I would be happy to participate in this work.@yaoyx689

@mvieth
Copy link
Member

mvieth commented Jul 3, 2024

TransformationEstimationMM

What would MM mean here? In any case, we don't have to decide the naming right now, as it can be changed later in the pull request.

Would FastAndRobustIterativeClosestPoint be a class like IterativeClosestPointNonLinear? I am okay with adding such a convenience class later, but first the transformation estimation methods need to be implemented.

I have been working on point cloud registration at the application level in recent days, and I am not sure whether MM and WelschMM accurately express the mathematical principles involved. I would be happy to participate in this work

One person would have to start a pull request, then the other person could add code in a review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants