Udacity Self-Driving Car Engineer Nanodegree

Final Project: System Integration

This is Team Vulture project repo for the final project of the Udacity Self-Driving Car Engineer Nanodegree: Programming a Real Self-Driving Car. The project will require the use of Ubuntu Linux (the operating system of Carla) and a new simulator with integration with the Robotic Operation System or ROS. This project will restrict the driving speed of Carla to ~ 10 MPH for field testing.

NOTE: You will find our development, testing logs and SAFETY NOTES in the REPORT.md file.

WARNING: To maintain integrity of the SDCND Final System Integration Project, the throttle and brake actuation settings in this prepository are NOT what were used in the successful navigation of Carla around the test track. The last Udacity Capstone Project repository merge pulled into this github repository was on September 9, 2017: https://github.com/udacity/CarND-Capstone/commit/140691dada1e659422693007eee48c6d5faca14f. Further merges with upstream repository may render the project in this repository inoperable. The final project results are now static, and no further update requests will be expected nor entertained.

The Team

The following are the members of Team Vulture. Artwork for printing our team T-Shirt can be found here.

	Name	Location	LinkedIn	Image
Team Lead	John Chen	Ashburn, VA	linkedin.com/in/diyjac
	Rainer Bareiß	Stuttgart, Germany	linkedin.com/in/rainer-bareiss
	Sebastian Trick	Karlsruhe, Germany	linkedin.com/in/sebastian-trick-500194ab
	Yuesong Xie	Southfield, MI	linkedin.com/in/yuesongxie
	Kunfeng Chen	Mountain View, CA	linkedin.com/in/kunfengchen

Team Vulture Accomplishments

1. First team to move Carla

2. First team to successfully navigate Carla around the test track and stop appropriately for the lights

3. First team to complete Udacity Self-Driving Car Engineer Nanodegree Final Project

4. Rainer Bareiß Designed and Made Team Vulture T-Shirts!

5. Team Meeting at IAA 2017 in Frankfurt, Germany.

6. Team Meeting at Udacity Europe, SDC 2017 Student Dinner with David Silver

7. Kunfeng Chen and Sebastian Thrun at Udacity Hiring Partner Event 2017 in Mountain View, California

GO VULTURE!

Installation

• Be sure that your workstation is running Ubuntu 16.04 Xenial Xerus or Ubuntu 14.04 Trusty Tahir. Ubuntu downloads can be found here.

• If using a Virtual Machine to install Ubuntu, use the following configuration as minimum:

  • 2 CPU
  • 2 GB system memory
  • 25 GB of free hard drive space

  The Udacity provided virtual machine has ROS and Dataspeed DBW already installed, so you can skip the next two steps if you are using this.

  NOTE: We experienced poor performance using VM, so did not use it for integration and testing.

• Follow these instructions to install ROS

  • ROS Kinetic if you have Ubuntu 16.04.
  • ROS Indigo if you have Ubuntu 14.04.
  • Dataspeed DBW
  • Use this option to install the SDK on a workstation that already has ROS installed: One Line SDK Install (binary)

• Download the Udacity Simulator v1.2.

  NOTE: If you are installing in native Ubuntu 16.04, the Dataspeed DBW One Line SDK binary install will auto install 4.4.0-92-generic Linux kernel, which will break CUDA and the NVIDIA 375 drivers if you have NVIDIA GPU in your native Ubuntu 16.04 build. This will cause starting the simulator to fail because it can no longer open the OpenGL drivers provided by NVIDIA:

  

  To fix this issue, you will have to:

  • remove the 4.4.0-92-generic kernel (or any kernel newer than 4.4.0-91-generic):

    sudo apt-get remove linux-image-4.4.0-92-generic

  • reinstall the NVIDIA 375 drivers (follow the instructions):

    https://askubuntu.com/questions/760934/graphics-issues-after-while-installing-ubuntu-16-04-16-10-with-nvidia-graphics

Usage

1. Clone the project repository

git clone https://github.com/diyjac/SDC-System-Integration.git
cd SDC-System-Integration

2. OPTIONAL: Verify, List, Switch or Create your own branch in the repository
   • verify current branch

     git status

   • list existing branches

     git branch

   • switch to a different branch

     git checkout <different branch>

   • create new branch from current branch and push to remote repository

     git checkout -b <your branch>
     git push -u origin <your branch>

3. Install python dependencies

• With Pygame and MoviePy for Diagnostics and converting rosbags
  • For non-GPU Linux systems

    sudo -H pip install -r requirements-pygame.txt

  • For GPU Linux systems NOTE: Make sure to install CUDA and its dependencies! http://www.nvidia.com/object/gpu-accelerated-applications-tensorflow-installation.html

    sudo -H pip install -r requirements-gpu-pygame.txt

• Without Pygame nor MoviePy
  • For non-GPU Linux systems

    pip install -r requirements.txt

  • For GPU Linux systems NOTE: Make sure to install CUDA and its dependencies! http://www.nvidia.com/object/gpu-accelerated-applications-tensorflow-installation.html

    pip install -r requirements-gpu.txt

4. Make and run styx

cd ros
catkin_make
source devel/setup.bash
roslaunch launch/styx.launch

5. Run the simulator

unzip linux_sys_int.zip
cd linux_sys_int
chmod +x system_integration.x86_64
./system_integration.x86_64

6. To test grab a raw camera image

rosrun tools grabFrontCameraImage.py ../imgs/sampleout.jpg

7. To dump the waypoints from the /base_waypoints topic

rosrun tools dumpWaypoints.py ../data/simulator_waypoints.csv

8. To dump the final waypoints from the /final_waypoints topic

rosrun tools dumpFinalWaypoints.py ../data/final_waypoints.csv

9. To view the diagnostics screen in real-time when the integrated system is running

• NOTE: Requires pygame!

rosrun tools diagScreen.py --screensize 2 --maxhistory 800 --textspacing 75 --fontsize 1.5

10. To view sample Udacity provided rosbags, convert them to MP4, GIFS or JPG use the following:

• NOTE: Requires pygame and moviepy!

cd ../tools
python view_rosbag_video_with_objdet.py --dataset <rosbags>
python rosbag_video_2_mp4.py --dataset <rosbags> <path to mp4 file>
python rosbag_video_2_gif.py --dataset <rosbags> <path to gif file>
python rosbag_video_2_jpg.py --dataset <rosbags> '<path to rosbag_%04d.jpg>'

Full length MP4 videos of the Udacity provided sample rosbags are available for download:

• ./imgs/just_traffic_light.mp4
• ./imgs/loop_with_traffic_light.mp4

Samples of jpeg images extracted:

CSV files with pose and manually updated labels:

• ./test_images/loop_with_traffic_light.csv
• ./test_images/just_traffic_light.csv

11. Sample training, validation and testing images have been collected and are in data/collections/samples directory. There is a session1.csv file that will provide the features and labels. To collect additional training, validation and testing images for the traffic light classifier, use the autoTLDataCollector.py tool once you have started roslaunch launch/styx.launch and the simulator:

cd SDC-System-Integration
mkdir data/collections/mysamples
cd ros
rosrun tools autoTLDataCollector.py mysamples/session1

Real world testing

1. Download training bag that was recorded on the Udacity self-driving car
2. Unzip the file

unzip traffic_light_bag_files.zip

3. Play the bag file

rosbag play -l traffic_light_bag_files/loop_with_traffic_light.bag

4. Run the Rosbag Diagnostics

rosrun tools diagScreenRosbag.py

5. Launch your project in site mode

cd SDC-System-Integration/ros
source devel/setup.bash
roslaunch launch/site-rosbag.launch