Udacity Banana collector

Introduction

This project has been built for the Udacity Deep Reinforcement Learning Nano Degree Program. With this you can train an agent to navigate and collect bananas in a large, square world.

Project Details

The goal is to collect as many yellow bananas available while avoiding blue bananas.

The agent is trained using an algorithm called Deep Q-Learning. Deep Q-Learning combines the best of reinforcement learning with recent advances in deep learning. For details see the jupyter notebook provided with this repository.

The environment for training the agent provided is an adapted version of the Banana Collector provided by the Unity ml-agents library

The state space has 37 dimensions and contains the agent's velocity, along with ray-based perception of objects around agent's forward direction. Given this information, the agent has to learn how to best select actions. Four discrete actions are available, corresponding to:

• 0 - move forward.
• 1 - move backward.
• 2 - turn left.
• 3 - turn right.

The task is episodic, and in order to solve the environment, the agent must get an average score of +13 over 100 consecutive episodes.

Getting Started

1. Clone this repository, and navigate to the udacity-p1/ folder. Then, install several dependencies.

git clone https://github.com/primeMover2011/udacity-p1.git
cd udacity-p1

2. Download the environment from one of the links below. You need only select the environment that matches your operating system:

   • Linux: click here
   • Mac OSX: click here
   • Windows (32-bit): click here
   • Windows (64-bit): click here

   (For Windows users) Check out this link if you need help with determining if your computer is running a 32-bit version or 64-bit version of the Windows operating system.

   (For AWS) If you'd like to train the agent on AWS (and have not enabled a virtual screen), then please use this link to obtain the environment.

3. Extract the contents of the file to a folder of you choice, preferably as a subfolder of this repository.

4. Install conda

5. cd in to the directory where you cloned this repository, create a virtual environment and install the required python packages using these commands

cd udacity-p1
conda env create -f environment.yml

activate the environment using

conda activate udacity-banana

6. Create an IPython kernel for the udacity-p1 environment.

python -m ipykernel install --user --name udacity-p1 --display-name "udacity-p1"

5. Before running code in a notebook, change the kernel to match the udacity-p1 environment by using the drop-down Kernel menu.

6. Open the notebook Report.ipynb and execute each cell to train an agent using Deep Q-Learning with a Dueling network.

Enjoy!