A package to use cameras in ros-independent manner and with deoxys_control (to move the robot to collect apriltag images)
1. Setup [deoxys_control](https://github.com/UT-Austin-RPL/deoxys_control/tree/main/deoxys) (This usually should be setup)
2. Activate the conda environment (Usually should be deoxys_base)
3. Navigate into deoxys_vision folder
4. pip install -e .
This is to calibrate the fixed-base camera. This assumes that a marker mount is attached to the robot’s end-effector, and the marker is default to be AprilTag in our lab
Run the script:
python camera_calibration/record_robot_joints.py
The robot will run with OSC controller, with low-impedance value. In that case, it’s easy to move the arm around and record the desired joint values. For recording, press the grasping button of SpaceMouse (one of the buttons on the side). When you want to finish the process, press the other button of SpaceMouse which will terminate the process and let you decide 1) whether or not save the recorded joints, and 2) specify the file name to record the joints.
In order to see which joints are good for detecting tags, we suggest opening up the visualization of camera (and leave it running):
python scripts/deoxys_camera_node.py --camera-ref rs_0 --use-rgb --use-depth --use-rec --visualization --eval
Step 2: Replay robot joints and compute the extrinsics (the transformation of camera from the robot base frame)
python camera_calibration/hand_eye_calibration.py --camera-ref rs_0 --calibration-type eye-to-hand --config-filename calibration_joints.json
There are two useful options for running this script. One is --use-saved-images, which will use the previously recorded images without actually running the robot. Another one is --debug, which will show some detailed information of the calibration process.
The calibrated value will be saved into the file camera_{ID}_extrinsics.json in the default folder ~/.deoxys_vision/calibration.
To test if the calibration was successful use the following script. Ensure that the calibrated value (the extrinsic matrix) from camera_{ID}_extrinsics.json is being used
python scripts/test_calibration.py --camera-ref rs_0 --calibration-method horaud
TODO