Skip to content

Latest commit

 

History

History
64 lines (46 loc) · 2.49 KB

README.md

File metadata and controls

64 lines (46 loc) · 2.49 KB
Raw

rl-snake

Some reinforcement learning algorithms to play snake game taken from Sutton's book

Usage

$ python snake.py --help

Algorithms

Tabular

  1. Monte Carlo
  2. one-step Q-learning
  3. n-step SARSA

To stop training press Ctrl-C and Q.pkl file will be saved in current directory. Then it can be used to continue training or to follow learned policy with visualization.

Non-growing snake

python snake.py --train --x=5 --y=5 --algo=mc

It is enough to train non-growing snake on a 5x5 grid to be able to use it on arbitrary large grid.
The more you train the better it becomes. Test it:

python snake.py --x=10 --y=10

Growing snake

python snake.py --train --x=5 --y=5 --grow --algo=sarsa --step=4

Additional 9 boolean indicators are added to a snake's state for each cell around a head, indicating if cell belongs to a snake. So snake is myopic in terms of what it can see. Adding all grid cells is not tractable due to enourmouse ammount of possible states.

python snake.py --x=5 --y=5 --grow --delay=0.3

Compare

Left to right: Monte Carlo, 1-step SARSA, 4-step SARSA, 1-step Q-learning

Parameters

Monte Carlo 1-SARSA 4-SARSA Q-learning
SINGLE
epsilon 0.5 0.3 0.3 0.5
alpha --- 0.05 0.05 0.005
episodes 1080k 2044k 253k 1030k
------------- ------------- --------- --------- ------------
GROWING
epsilon 0.05 0.05 0.05 0.1
alpha --- 0.05 0.05 0.0005
episodes 4152k 1559k 1559k 2023k