Camera trajectory distortion with Orbbec Femto Bolt

[NIkolaStaykov]

Dear Spectacular AI Team,

I using the sai SDK with the Orbbec wrapper (version 1.34) to generate a point cloud and camera trajectory with an Orbbec Femto Bolt camera. The trajectory received has sudden jumps, which do not correspond to real movements. Has this happened to you and do you have any suggestions? Where does sai look for the camera parameters of the specific model and the camera to imu transformation? Should I look for faults in the hardware and the imu data itself? An image of the trajectory generated below.

Thanks in advance!

[kaatrasa]

Hey,

The jumps in the trajectory could be caused by various different reasons, and without the original recording I can only guess. For example, it is possible that there are problems in the recording (too many dropped frames/IMU samples). Or, it could be that the recording is too challenging (excessive motion blur, few features, ...) for best performance with Femto Bolt that isn't really a tracking camera.

Also, how was the pose trail generated? Are those real-time poses (which can contain SLAM jumps) or final SLAM map poses that should generally be smoother and consistent with all poses in the map. Also, the parameters used for reconstruction greatly affect the result. For best reconstructions, I would recommend using the sai-cli process tool (installed using pip install spectacularAI[full]).

You can try checking any potential issues with the recording using this script:
https://github.com/SpectacularAI/calibration-scripts/blob/master/visualize.py

[NIkolaStaykov]

Thanks for the reply!
Those are real-time poses; your suggestion with the SLAM jumps is also highly likely. I used your smoothing script utilizing the final slam map and got much better results!
PS The Femto Bolt has IMU sampling time inconsistencies, could that be part of the problem?

[kaatrasa]

Good to hear that the smoothing script is working better! If you need consistent/smooth poses for all frames, then it is the best option to use.

PS The Femto Bolt has IMU sampling time inconsistencies, could that be part of the problem?

That depends on how many IMU samples are dropped. By default, our Orbbec wrapper uses 1kHz IMU frequenzy which is quite high, and it is OK to occasionally/randomly drop IMU samples. However, if there are noticeable gaps in the IMU signal (i.e. many/all consecutive samples are dropped) then this is very bad for VIO. If you post the visualize.py output image, then I can take a very quick look.