A from-scratch implementation of Q-learning in C++ used to train a 2D car how to drive around a simple racetrack. The system works by simulating epochs many times. For each epoch, the car is rewarded for travelling a distance. However, if it runs off the track it crashes and is penalized.
These rewards and penalties are used to develop an in-memory Q-table. This is a hashmap which relates state to action. In this simulation, the state is the car's sensors which measure the distance from the car to the edge of the road on pre-derermined angles. This state is related to Q-values which are fed through a series of functions to determine the best action.
When the car is training, it updates it Q-table at each step. When it is evaluated, it simply references the Q-table to make choices. I also implemented my own random number generator to generate floats and integers, just for fun!
This project depends on SFML to display the simulation window. You can visit
the SFML link below in Resources for install instructions. Apart from this
one dependency, this follows a typical CMake workflow:
- Create
build/directory in this repo - Navigate to build and run
cmake .. - Run
make - Run the binary
On Linux that is (assuming you are in the base directory of this repo):
mkdir build
cd build
cmake ..
make
./autodriveImportant
autodrive can only be run from the build/ directory. This is because
it loads files from other directories in the project in a relative fashion.
Simply running the binary will launch a train and evaluation cycle. There
are a few hyperparameters described in main.cpp which can be tweaked,
but their default settings work well.
