A Guide to DeepMind's StarCraft AI Environment.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A Guide to DeepMind's StarCraft AI Environment\n",
    "\n",
    "## Demo -- We're going to setup and install the necessary tools to run a pretrained Deep Q Network model on the CollectMineralShards mini-game of DeepMind's StarCraft II Environment.\n",
    "\n",
    "![alt text](https://cdn.technologyreview.com/i/images/20161104patrick-strackblizzcon16070a0766.jpg?sw=2360&cx=0&cy=0&cw=950&ch=633 \"Logo Title Text 1\")\n",
    "\n",
    "## History\n",
    "\n",
    "Deepmind already beat Atari Games with the Deep Q Learner\n",
    "\n",
    "![alt text](https://rubenfiszel.github.io/posts/rl4j/conv.png \"Logo Title Text 1\")\n",
    "![alt text](https://s3-ap-south-1.amazonaws.com/av-blog-media/wp-content/uploads/2017/01/12042140/11038f3.jpg \"Logo Title Text 1\")\n",
    "![alt text](https://www.intelnervana.com/wp-content/uploads/sites/53/2017/06/Screen-Shot-2015-12-21-at-11.23.43-AM-1.png \"Logo Title Text 1\")\n",
    "\n",
    "Then they beat the \"unbeatable\" game of \"Go\" with AlphaGo\n",
    "\n",
    "![alt text](https://image.slidesharecdn.com/masteringthegameofgowithdeepneuralnetworksandtreesearch-160321115146/95/mastering-the-game-of-go-with-deep-neural-networks-and-tree-search-10-638.jpg?cb=1458635243 \"Logo Title Text 1\")\n",
    "![alt text](https://image.slidesharecdn.com/howdeepmindmasteredthegameofgo-160903224536/95/how-deepmind-mastered-the-game-of-go-20-638.jpg?cb=1472943238 \"Logo Title Text 1\")\n",
    "\n",
    "And now they've set their sights on Starcraft. For an AI to play StarCraft well, it'll need\n",
    "\n",
    "- An effective use of memory\n",
    "- an ability to plan over a long time\n",
    "- The capacity to adapt plans based on new information. \n",
    "- To execute something as simple as “expand your base to some location”, one must coordinate mouse clicks, camera, and available resources.  This makes actions and planning hierarchical, which is a challenging aspect of Reinforcement Learning.\n",
    "\n",
    "Blizzard's StarCraft II API is an interface that provides full external control of StarCraft II.\n",
    "\n",
    "This API exposes functionality for developing software for:\n",
    "\n",
    "- Scripted bots.\n",
    "- Machine-learning based bots.\n",
    "- Replay analysis.\n",
    "- Tool assisted human play.\n",
    "\n",
    "DeepMind's PySC2 - StarCraft II Learning Environment exposes it as a Python RL Environment. \n",
    "\n",
    "- A Machine Learning API developed by Blizzard that gives researchers and developers hooks into the game. This includes the release of tools for Linux for the first time.\n",
    "- A dataset of anonymised game replays, which will increase from 65k to more than half a million in the coming weeks.  \n",
    "- An open source version of DeepMind’s toolset, PySC2, to allow researchers to easily use Blizzard’s feature-layer API with their agents.\n",
    "- A series of simple RL mini-games to allow researchers to test the performance of agents on specific tasks.\n",
    "- A joint paper that outlines the environment, and reports initial baseline results on the mini-games, supervised learning from replays, and the full 1v1 ladder game against the built-in AI.\n",
    "\n",
    "Starcraft II is a real-time strategy game developed by Blizzard entertainment, otherwise known as the makers of World of Warcraft. It's the sequel to Starcraft, a game from 1998 that many regard as one of the greatest PC games ever released. Even now, over a decade on, it's still played regularly by people all over the world; in Korea, it's so popular that there are professional leagues dedicated solely to playing the game.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Installation Steps\n",
    "\n",
    "\n",
    "Steps\n",
    "\n",
    "1) Install pysc2\n",
    "\n",
    "2) Clone pysc2-examples repository\n",
    "\n",
    "3) Download mini-games StarCraft II Maps\n",
    "\n",
    "4) Install Tensorflow, baselines libraries\n",
    "\n",
    "5) Open the project with IntelliJ \n",
    "\n",
    "6) Run the training script\n",
    "\n",
    "7) Run the pre-trained model\n",
    "\n",
    "\n",
    "\n",
    "## Step 1 - Install pysc2\n",
    "\n",
    "`pip3 install pysc2`\n",
    "\n",
    "\n",
    "## Step 2 - Git Clone psyc2 examples\n",
    "\n",
    "`git clone https://github.com/llSourcell/A-Guide-to-DeepMinds-StarCraft-AI-Environment`\n",
    "\n",
    "## Step 3 - Download mini-games StarCraft II Maps\n",
    "\n",
    "https://github.com/deepmind/pysc2/releases/download/v1.0/mini_games.zip\n",
    "\n",
    "save these maps to StarCraft II/Maps \n",
    "\n",
    "## Step 4 - Install Tensorflow + OpenAI Baselines\n",
    "\n",
    "`pip3 install tensorflow`\n",
    "`pip3 install baselines`\n",
    "\n",
    "## Step 5 - Open the Project with Intellij\n",
    "\n",
    "### Start training\n",
    "\n",
    "`python3 train_mineral_shards.py`\n",
    "\n",
    "### Open project , Python 3 SDK \n",
    "\n",
    "## Step 6  Run training script\n",
    "\n",
    "Right click the train_mineral_shards.py and select [Run 'train_mineral_shards'] menu.\n",
    "\n",
    "This is the brief explanation of console logs.\n",
    "\n",
    "- steps : The number of commands that we sent to marines.\n",
    "- episodes : The number of games that we played.\n",
    "- mean 100 episode reward : mean rewards of last 100 episodes.\n",
    "- mean 100 episode min… : mean minerals of last 100 episodes.\n",
    "- % time spent exploring : The percentage of Exploring (Exploration & Exploit)\n",
    "\n",
    "\n",
    "## Step 7 Run pre-trained model\n",
    "\n",
    "- Right click the enjoy_mineral_shards.py and select [Run 'enjoy_mineral_shards'] menu.\n",
    "\n",
    "Then we can see the pre-trained agent of CollectMineralShards map.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}