Unstable calibration results for the long-focus camera's intrinsic

[treememory]

System information (version)

• Operating System / Platform => ubuntu 20.04
• OpenCV => 4.5.5
• Ceres => 1.14.0
• Boost => 1.71
• C++ => c++17
• Compiler => clang10

Vision system

• Number of cameras => 1
• Types of cameras => perspective (perspective, fisheye, hybrid)
• Multicamera configurations => No (overlapping, non-overlapping, converging)
• Configuration file => in next link (i.e. *.yml)

Describe the issue / bug

Hello, thank you for your work. I have been using your approach to calibrate the intrinsic parameters of a single long-focus camera (FOV≈30deg). I have observed instability in the calibration results, with variations of over 100 pixels in the cx and cy parameters across multiple calibrations (image size: 3840*2160). However, each calibration's reprojection error remains within 0.5px, indicating high accuracy.

During data collection, I have taken several precautions, such as avoiding image blur, ensuring complete coverage of the calibration pattern in the image, and capturing data from different angles and depths of field. The provided link includes five sets of data, out of which two were independently collected, while the three "mix" sets were created by randomly selecting samples from the first two sets. However, there are significant differences in the calibration results among the different data sets.

I would like to know whether the issue lies with my data collection method or if our algorithm does not support long-focus camera calibration.

The following are two images: one showing the calibration results for different data sets and the other displaying the spatial distribution of the calibration patterns for the two data sets.

Do you have any suggestions on how to obtain stable calibration results for the intrinsic parameters? Thank you, and best regards.

[rameau-fr]

Thank you very much for your interest in our project.
Given the long focal length you are estimating, I am not surprised by a discrepancy of 100px (about 1~2% of variation), which might satisfy robotics applications. I assume that most "conventional" calibration toolboxes will provide similar or worse results for such a scenario.
If you are looking for a very accurate calibration of such a narrow FoV camera, other options might be considered. To be clear, the calibration of long focal length cameras is a well-known problem in the community because of their shallow depth of field, which forces you to acquire all the images at almost the same distance. I had a look at your data, and it might indeed lack a bit diversity in terms of distance between the camera and the checkboard. Also when the focal is very long, some ambiguity between the distortion parameters and the focal length might occur.
I recommend you look at "Accurate Camera Calibration Robust to Defocus using a Smartphone" which is explicitly designed to deal with such challenges, but I am not entirely sure it will be practical in your context.
I will try to see if I can find some other alternative options.

[treememory]

Thank you for your reply~

I have a few more questions to inquire about.

For wide-angle lenses, would there still be 1-2% uncertainty? Any suggestions regarding data collection techniques? I have already noticed: covering the calibration board evenly across the entire field of view, considering different angles and depth of field positions of the calibration board, etc. Are there any other tips?

You mentioned that there can be a blur effect on distortion parameters and angles when the focal length is very long. How should I understand this specifically? In practical operations, I have also noticed that the distortion of telephoto lenses tends to increase, but the difference between the undistorted and distorted images seems to be minimal. I'm not sure if this is what you meant.

I will definitely read the recommended paper, thank you. Are you suggesting that accurate calibration of telephoto lenses is still a challenging task in the academic field? In the autonomous driving domain, many applications use telephoto lenses, so are there specific methods available for achieving good calibration?

[rameau-fr]

Regarding the accuracy of a fisheye camera, it is also a tricky question as it will depend on many factors, and very large FoVs also come with their own challenges. For instance, I saw many times in the past that the ambiguity between distortion and focal is even more pronounced for very large fields of view. This can be explained by the fact that radial distortion and focal affect the coordinates displacement in the same exact fashion (displacement along the radial lines of the image).

You are totally right, in your case, the amount of radial distortion should be very limited, but the focal and radial distortion might interfere during optimization, especially when the distribution of the points is limited. Also, I am pretty sure that despite this discrepancy in focal length, the resulting undistorted images would all look very similar (you can look at the PSNR difference with different calibration results).

Regarding a recommendation for your specific use case, I have some old memory of a toolbox specifically designed for a very highly accurate setup ... But I am maybe hallucinating this ;-) I will try to have another look online

[rameau-fr]

I found this paper that might interest you: https://elib.dlr.de/62660/1/Strobl-Iros09_calib.pdf