Skip to content

adolfos94/ROS-Stereo-Vision

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

66 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Stereo Vision

Computer stereo vision is the extraction of 3D information from digital images that can be used for object identification, tracking and manipulation tasks.

Modules

ROS Stereo Vision

This is a ROS Package for Jetson CSI Stereo Camera for Computer Vision Tasks.

This package contains the nodes:

  • src/stereo_camera_pub: Obtains left and right rectified images from IMX219-83 Stereo Camera.
  • src/stereo_camera_sub: Displays the images using OpenCV.
  • src/depth_perception_sub: Compute depth based on stereo images.

Hardware Stereo Module

IMX219-83 Stereo Camera

Waveshare Documentation: IMX219-83 Stereo Camera.

Test with Jetson Nano

Hardware Connection

  • Connect the camera to the CSI interfaces of Jetson Nano.
  • Connect an HDMI LCD to Jetson Nano.
  • Connect the I2C interface (only the SDA and SCL pins are required) of the Camera to I2C interface of the Jetson Nano Developer Kit (the Pin3, and Pin5).

Sofware Testing

  • Open a Terminal.

  • Check the video devices with command:

    ls /dev/video*
  • Check if both video0 and video1 are detected

    • video0
    DISPLAY=:0.0 gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=3280, height=2464, format=(string)NV12, framerate=(fraction)20/1' ! nvoverlaysink -e
    • video1
    DISPLAY=:0.0 gst-launch-1.0 nvarguscamerasrc sensor-id=1 ! 'video/x-raw(memory:NVMM), width=3280, height=2464, format=(string)NV12, framerate=(fraction)20/1' ! nvoverlaysink -e
  • Test the ICM20948

    wget https://www.waveshare.com/w/upload/a/a4/D219-9dof.tar.gz
    tar zxvf D219-9dof.tar.gz]
    cd D219-9dof/03-double-camera-display
    mkdir build
    cd build
    cmake ..
    make
    ./double-camera-display
  • If you find that the image captured is red. You can try to download .isp file and install it:

    wget https://www.waveshare.com/w/upload/e/eb/Camera_overrides.tar.gz
    tar zxvf Camera_overrides.tar.gz
    sudo cp camera_overrides.isp /var/nvidia/nvcam/settings/
    sudo chmod 664 /var/nvidia/nvcam/settings/camera_overrides.isp
    sudo chown root:root /var/nvidia/nvcam/settings/camera_overrides.isp

Installation

Requirements

  • JetPack 4.6.1
  • OpenCV (Installed with JetPack)
    • Optional
      1. Change the file
      sudo nano /opt/ros/melodic/share/cv_bridge/cmake/cv_bridgeConfig.cmake
      1. Change these lines:
      set(_include_dirs "include;/usr/include;/usr/include/opencv")
      to:
      set(_include_dirs "include;/usr/include;/usr/include/opencv4")
  • ZSH (If you are using bash adapt your command in Step 4)

Install ROS

Integrate this repo into your ROS Workspace.

1. Clone this repo as a ROS Workspace.

cd ~/
git clone https://github.com/adolfos94/ROS-Stereo-Camera.git

2. Navigate to the new ROS Workspace or you can integrate the source files to another ROS Workspace.

cd ROS-Stereo-Camera

3. Build the package in the catking workspace.

$ adolfo in ROS-Stereo-Camera at jetson-nano
catkin_make

4. Add the workspace to your ROS environment.

$ adolfo in ROS-Stereo-Camera at jetson-nano
source devel/setup.zsh

5. Start the Publisher.

$ adolfo in ROS-Stereo-Camera at jetson-nano
rosrun stereo_camera_pub stereo_camera_pub_node

6. Start the Suscriber.

$ adolfo in ROS-Stereo-Camera at jetson-nano
rosrun stereo_camera_sub stereo_camera_sub_node