Skip to content

Replicating Udacity's Self Driving Car Simulator. Predicting steering angle for the Autonomous Vehicle in Unity3D simulated environment.

License

Notifications You must be signed in to change notification settings

MohammadWasil/Self-Driving-Car-Python

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

36 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Self-Driving-Car

Hits

This project is base on Udacity's Self-Driving Car Simulator.

sdccar

How it is going to look(Video) !

Dependencies

You can install depedencies by running the following command in Anaconda prompt:

With Theano Backend

# OpenCV
conda install -c conda-forge opencv

# Theano
conda install mingw libpython
conda install mkl=2017.0.3

# Keras
pip install keras

After installing all the dependencies, Change the backend of the Keras to "theano".
For that, Go to C:\Users\YourSystemName.keras and open Kersas.json file.
Change the backend to "thenao".
It would look something like this:

{
    "backend": "theano",
    "image_data_format": "channels_last",
    "floatx": "float32",
    "epsilon": 1e-07
}

With Tensorflow Backend

To install Tensorflow, follow this Youtube video:

Install Tensorflow-GPU in 5 mins - EASY!!

Steps

  1. Create an environment
    conda create -n tensorflow_gpu python=3.7

  2. Activate the environment
    conda activate tensorflow_gpu

  3. Install jupyter beforehand (since it downgrade the tensorflow version, if it is executed after tensorflow has been installed)
    conda install -c anaconda jupyter

  4. Install the ipykernel
    pip install ipykernel

  5. Register your environment
    python -m ipykernel install --user --name tensorflow_gpu --display-name "tensorflow_gpu"

  6. Install tensorflow gpu
    conda install -c anaconda tensorflow-gpu

  7. And, finally, install Keras
    pip install keras

Also, you need to have Unity3D game engine, which can be downloaded from here and install it.

Working with Unity3D Game Engine.

  1. Launch Unity3D Game Engine.
  2. Select "Open Project", select "Self Driving Car", from this project.
  3. Your project will open in Unity3D.

Workflow

This project is divided into three parts:

  1. Data Generation.
  2. Training the data.
  3. Testing.

Data Generation

In Unity3D simulator environment, we drive the car manually, using keyboard or joystick, and simulataneously, captures the frames or image, using python PIL library through Anaconda’s Spyder, and accessing their associated Steering Angle, Throttle, Velocity, and sending these values and storing it in a CSV file, along with the images.

How To:

  1. In "DataGeneration.py" file, change the directory, where the csv file is going to be saved.
  2. In "DataGeneration.py" file, change the directory, where all your images will be going to be saved.
  3. In Unity3D simulator, in Hierarchy, select car, and then in Inspector, under "Car user Control" script, check Generate Data, and uncheck Drive Car.
    car generatecar inspector
  4. In Unity3D simulator, in Hierarchy, select Network, in Inspector, active(or check) "Network Data Gen" script, and inactive(uncheck) "Network Con" script.
    network datagen inspector
  5. Run the simulator in unity3D, then run the program "DataGeneration.py" from spyder. Drive the car manually using keyboard or JoyStick.

Train the data

First, the data saved in CSV file is loaded, and image processing is done. In Image processing, the size of the mage is decreased. The actual size was (420, 750) , which was actually very big. It’s size was reduced to (66, 200). Then, the images, and Steering Angle are splitted into training and validation datasets, so that we can use some data for training, and some data for testing. Then, we apply series of Convolutional Neural Networks on image training datasets, followed by training the model, using Mean Squared Error loss function. This will create several “.h5” files, which we will be used for testing.

How To:

  1. Just run the script, "TrainCNN.py".

Testing

First, image is captured and current steering angle, throttle, and velocity from Unity3D simulator is send over to the socket TCP connection to the Spyder. The image is processed, reducing its size to (66, 200). Then we predict the Steering Angle on the basis of the image, and corresponding throttle value is calculated from the equation. This Steering Angle and Throttle is send back to the Unity3D simulator, and applies to the car, which starts driving by itself. Then again, image is captured, steering angle, throttle, and velocity is send back to the Spyder, and it goes on. It captures the frames(or an image) at 5 frames per seconds.

How To:

  1. In "Drive.py" file, change the directory, to load the model, which was created by training the data in TrainCNN.py.
  2. Next, change the directory of "filename", where all current data in a CSV file will be going to be saved.
  3. And then, change the directory of "path", where all the current images will be going to be saved.
  4. In Unity3D simulator, in Hierarchy, select car, and then in Inspector, under "Car user Control" script, check Drive Car, and uncheck Generate Data.
    car drivecar inspector
  5. In Unity3D simulator, in Hierarchy, select Network, in Inspector, active(or check) "Network Con" script, and inactive(uncheck) "Network Data Gen" script.
    network drivecar inspector
  6. Run the simulator in unity3D, then run the program "Drive.py" from spyder. Car will start driving by itself.

Caution!

Before running the codes, try to change the path of each files and images.

More Information:

To know more about Convolutional Neural Network, check this github page.
Check this and this for Self Driving Car.

About

Replicating Udacity's Self Driving Car Simulator. Predicting steering angle for the Autonomous Vehicle in Unity3D simulated environment.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages