Issue with 3D Keypoint Generation Returning Zero Values

[Lilien86]

Operating system: windows 11
Software version: v0.2

After calibrating my 3 cameras, I created intri.yml and extri.yml files. To verify the calibration accuracy, I used the following command:
python apps/calibration/check_calib.py ${root}/ground1f --out ${root}/ground1f --mode cube --write --show, The calibration was confirmed to be correct.
I attempted several methods to generate 2D keypoints, such as:
python apps/preprocess/extract_keypoints.py ${data} --mode yolo-hrnet
python .\scripts\preprocess\extract_video.py .\0_input\data\ --openpose ${openpose}
Upon checking the 2D keypoints with:
python apps/annotation/annot_keypoints.py ${data}, it's good.

the results seemed correct, indicating successful 2D keypoint generation. Here's a screenshot of my dataset visualization:

With accurate calibration and 2D keypoints, I proceeded to generate 3D keypoints using the command:
emc --data config/datasets/mvimage.yml --exp config/mv1p/detect_triangulate.yml --root ${data} --subs_vis 1 2 3
However, this returned 3D keypoints equal to zero. Attempting an alternative approach yielded the same result: python .\apps\demo\mv1p.py .\0_input\data\ --out 1_output/data --vis_det --vis_repro --undis --sub_vis 1 2 3 --vis_smpl

Error:
Despite the dataset appearing correct, the generation of 3D keypoints fails, with all keypoints returning values of zero. There were no error messages to indicate what might be wrong.

Request for Assistance:
I would greatly appreciate any guidance or suggestions on how to resolve this issue with 3D keypoint generation. Thank you in advance for your time and assistance.

[jeffreyhoang]

Hey, I was wondering if you had solved this issue. I am having the exact same problem. Is it a problem related to our camera calibration?

[brian208579]

Hi @Lilien86, I have the same problem, please let me know if you solved this problem, thank you so much.

[jeffreyhoang]

@brian208579 Hey I was wondering if you had solved this issue yet? I had tried recalibrating my camera but I am still getting incorrect results.

[brian208579]

@jeffreyhoang hi Jeffrey, I haven't solved this problem yet.

[jeffreyhoang]

@brian208579 I think I know the problem. How did you calibrate your extrinsic parameters?

[brian208579]

@jeffreyhoang hi Jeffrey, I just followed this link to calibrate camera extrinsic & intrinsic parameters, could you share your thoughts with me ? thank you very much !

[raflitp]

Try using this steps to calibrate your cameras :

Intrinsic

python scripts/preprocess/extract_video.py calibration/intrinsic --no2d

python apps/calibration/detect_chessboard.py calibration/intrinsic --out calibration --pattern 9,6 --grid 0.15

python apps/calibration/calib_intri.py calibration/intrinsic --num 25

Grid 15cm in meters = 0.15
--num = Number of desired frames used to calibrate the more the better, over 100 frames will took longer processing time.
--sample = Every 50 frames only take 1 frame to be used to calibrate.

Extrinsic

python apps/preprocess/extract_image.py calibration/extrinsic

python apps/calibration/detect_chessboard.py calibration/extrinsic --out calibration --pattern 9,6 --grid 0.15

python apps/calibration/calib_extri.py calibration/extrinsic --intri calibration/intrinsic/output/intri.yml

Then check your calibration cube result using this code, make sure the box isn't distorted from every frame.

python apps/calibration/check_calib.py calibration/extrinsic --out calibration/extrinsic --show

If there's anything you needed to know you can feel free to ask me

[jeffreyhoang]

@raflitp How did you take your video for calibrating your intrinsic parameters? (did you use different angles and have the checkerboard in different positions on the camera?)

[raflitp]

@raflitp How did you take your video for calibrating your intrinsic parameters? (did you use different angles and have the checkerboard in different positions on the camera?)

For intrinsic you need to have the checkerboard on every side of the frame, for example : top left, mid left, bottom left, middle bottom etc. To get the coverage of all the frames. Intrinsic is better when having 100 or more frames. So you might want to cut your video to use only a few frame for the side of the frame. For example : top left 3 frames, mid left 3 frames and so on. No need to be in a whole video sequence just crop it to what you need. You might want to take the coverage of tilting upwards and downward depending on the position. Remember lighting also affects your calibration results, make sure you have a good lighting without any occlusions or shadow covering the checkerboard to get a better amount of results.

Here I give you an example of how I do my calibration :

1.mp4

Let me know your results !!