courses/machine_learning/deepdive/08_image/flowers_fromscratch.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# Flowers Image Classification with TensorFlow on Cloud ML Engine\n",
    "\n",
    "This notebook demonstrates how to do image classification from scratch on a flowers dataset using Estimator/Experiment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "          <script src=\"/static/components/requirejs/require.js\"></script>\n",
       "          <script>\n",
       "            requirejs.config({\n",
       "              paths: {\n",
       "                base: '/static/base',\n",
       "              },\n",
       "            });\n",
       "          </script>\n",
       "          "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import os\n",
    "PROJECT = 'cloud-training-demos' # REPLACE WITH YOUR PROJECT ID\n",
    "BUCKET = 'cloud-training-demos-ml' # REPLACE WITH YOUR BUCKET NAME\n",
    "REGION = 'us-central1' # REPLACE WITH YOUR BUCKET REGION e.g. us-central1\n",
    "MODEL_TYPE = 'cnn'\n",
    "\n",
    "# do not change these\n",
    "os.environ['PROJECT'] = PROJECT\n",
    "os.environ['BUCKET'] = BUCKET\n",
    "os.environ['REGION'] = REGION\n",
    "os.environ['MODEL_TYPE'] = MODEL_TYPE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "          <script src=\"/static/components/requirejs/require.js\"></script>\n",
       "          <script>\n",
       "            requirejs.config({\n",
       "              paths: {\n",
       "                base: '/static/base',\n",
       "              },\n",
       "            });\n",
       "          </script>\n",
       "          "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Updated property [core/project].\n",
      "Updated property [compute/region].\n"
     ]
    }
   ],
   "source": [
    "%bash\n",
    "gcloud config set project $PROJECT\n",
    "gcloud config set compute/region $REGION"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Input functions to read JPEG images\n",
    "\n",
    "The key difference between this notebook and [the MNIST one](./mnist_models.ipynb) is in the input function.\n",
    "In the input function here, we are doing the following:\n",
    "* Reading JPEG images, rather than 2D integer arrays.\n",
    "* Reading in batches of batch_size images (see the parameter to reader.read) rather than slicing our in-memory structure to be batch_size images.\n",
    "* Resizing the images to the expected HEIGHT, WIDTH. Because this is a real-world dataset, the images are of different sizes. We need to preprocess the data to, at the very least, resize them to constant size."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "          <script src=\"/static/components/requirejs/require.js\"></script>\n",
       "          <script>\n",
       "            requirejs.config({\n",
       "              paths: {\n",
       "                base: '/static/base',\n",
       "              },\n",
       "            });\n",
       "          </script>\n",
       "          "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting /tmp/test.csv\n"
     ]
    }
   ],
   "source": [
    "%writefile /tmp/test.csv\n",
    "gs://cloud-ml-data/img/flower_photos/daisy/754296579_30a9ae018c_n.jpg,daisy\n",
    "gs://cloud-ml-data/img/flower_photos/dandelion/18089878729_907ed2c7cd_m.jpg,dandelion\n",
    "gs://cloud-ml-data/img/flower_photos/dandelion/284497199_93a01f48f6.jpg,dandelion"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "          <script src=\"/static/components/requirejs/require.js\"></script>\n",
       "          <script>\n",
       "            requirejs.config({\n",
       "              paths: {\n",
       "                base: '/static/base',\n",
       "              },\n",
       "            });\n",
       "          </script>\n",
       "          "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['gs://cloud-ml-data/img/flower_photos/daisy/754296579_30a9ae018c_n.jpg,daisy', 'gs://cloud-ml-data/img/flower_photos/dandelion/18089878729_907ed2c7cd_m.jpg,dandelion']\n",
      "gs://cloud-ml-data/img/flower_photos/dandelion/18089878729_907ed2c7cd_m.jpg\n",
      "dandelion\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "def read_lines(filename):\n",
    "    from tensorflow.python.lib.io.file_io import FileIO\n",
    "    with FileIO(filename, 'rb') as f:\n",
    "      lines = f.readlines()\n",
    "      return [line.rstrip() for line in lines]   \n",
    "\n",
    "with tf.Session() as sess:\n",
    "  #csvfile = 'gs://cloud-ml-data/img/flower_photos/train_set.csv'\n",
    "  csvfile = '/tmp/test.csv'\n",
    "  lines = read_lines(csvfile)\n",
    "  print lines[:2]\n",
    "  filename, label = tf.decode_csv(lines, record_defaults=[['a'],['b']])\n",
    "  print filename[1].eval()\n",
    "  print label[1].eval()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Run as a Python module\n",
    "\n",
    "Let's run it as Python module.  Note the --model parameter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "          <script src=\"/static/components/requirejs/require.js\"></script>\n",
       "          <script>\n",
       "            requirejs.config({\n",
       "              paths: {\n",
       "                base: '/static/base',\n",
       "              },\n",
       "            });\n",
       "          </script>\n",
       "          "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "usage: task.py [-h] [--train_batch_size TRAIN_BATCH_SIZE]\n",
      "               [--learning_rate LEARNING_RATE] [--train_steps TRAIN_STEPS]\n",
      "               --output_dir OUTPUT_DIR --model MODEL [--job-dir JOB_DIR]\n",
      "               [--ksize1 KSIZE1] [--ksize2 KSIZE2] [--nfil1 NFIL1]\n",
      "               [--nfil2 NFIL2] [--dprob DPROB] [--batch_norm]\n",
      "\n",
      "optional arguments:\n",
      "  -h, --help            show this help message and exit\n",
      "  --train_batch_size TRAIN_BATCH_SIZE\n",
      "                        Batch size for training steps\n",
      "  --learning_rate LEARNING_RATE\n",
      "                        Initial learning rate for training\n",
      "  --train_steps TRAIN_STEPS\n",
      "                        Steps to run the training job for. A step is one\n",
      "                        batch-size,\n",
      "  --output_dir OUTPUT_DIR\n",
      "                        GCS location to write checkpoints and export models\n",
      "  --model MODEL         Type of model. Supported types are ['cnn',\n",
      "                        'dnn_dropout', 'dnn', 'linear']\n",
      "  --job-dir JOB_DIR     this model ignores this field, but it is required by\n",
      "                        gcloud\n",
      "  --ksize1 KSIZE1       kernel size of first layer for CNN\n",
      "  --ksize2 KSIZE2       kernel size of second layer for CNN\n",
      "  --nfil1 NFIL1         number of filters in first layer for CNN\n",
      "  --nfil2 NFIL2         number of filters in second layer for CNN\n",
      "  --dprob DPROB         dropout probability for CNN\n",
      "  --batch_norm          if specified, do batch_norm for CNN\n"
     ]
    }
   ],
   "source": [
    "%bash\n",
    "rm -rf flowersmodel.tar.gz flowers_trained\n",
    "export PYTHONPATH=${PYTHONPATH}:${PWD}/flowersmodel\n",
    "python -m trainer.task --help"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Now, let's do it on ML Engine. Note the --model parameter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%bash\n",
    "OUTDIR=gs://${BUCKET}/flowers/trained_${MODEL_TYPE}\n",
    "JOBNAME=flowers_${MODEL_TYPE}_$(date -u +%y%m%d_%H%M%S)\n",
    "echo $OUTDIR $REGION $JOBNAME\n",
    "gsutil -m rm -rf $OUTDIR\n",
    "gcloud ml-engine jobs submit training $JOBNAME \\\n",
    "   --region=$REGION \\\n",
    "   --module-name=trainer.task \\\n",
    "   --package-path=${PWD}/flowersmodel/trainer \\\n",
    "   --job-dir=$OUTDIR \\\n",
    "   --staging-bucket=gs://$BUCKET \\\n",
    "   --scale-tier=BASIC_GPU \\\n",
    "   --runtime-version=1.2 \\\n",
    "   -- \\\n",
    "   --output_dir=$OUTDIR \\\n",
    "   --train_steps=5000 --learning_rate=0.01 --train_batch_size=40 \\\n",
    "   --model=$MODEL_TYPE --batch_norm --augment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Here are my results:\n",
    "\n",
    "Model | Accuracy | Time taken | Run time parameters\n",
    "--- | :---: | ---\n",
    "cnn with batch-norm | 0.582 | 47 min | 1000 steps, LR=0.01, Batch=40\n",
    "as above, plus augment | 0.615 | 3 hr | 5000 steps, LR=0.01, Batch=40"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Deploying and predicting with model\n",
    "\n",
    "Deploy the model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%bash\n",
    "MODEL_NAME=\"flowers\"\n",
    "MODEL_VERSION=${MODEL_TYPE}\n",
    "MODEL_LOCATION=$(gsutil ls gs://${BUCKET}/flowers/trained_${MODEL_TYPE}/export/Servo | tail -1)\n",
    "echo \"Deleting and deploying $MODEL_NAME $MODEL_VERSION from $MODEL_LOCATION ... this will take a few minutes\"\n",
    "gcloud ml-engine versions delete --quiet ${MODEL_VERSION} --model ${MODEL_NAME}\n",
    "#gcloud ml-engine models delete ${MODEL_NAME}\n",
    "#gcloud ml-engine models create ${MODEL_NAME} --regions $REGION\n",
    "gcloud ml-engine versions create ${MODEL_VERSION} --model ${MODEL_NAME} --origin ${MODEL_LOCATION}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "To predict with the model, let's take one of the example images that is available on Google Cloud Storage <img src=\"http://storage.googleapis.com/cloud-ml-data/img/flower_photos/sunflowers/1022552002_2b93faf9e7_n.jpg\" />"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "          <script src=\"/static/components/requirejs/require.js\"></script>\n",
       "          <script>\n",
       "            requirejs.config({\n",
       "              paths: {\n",
       "                base: '/static/base',\n",
       "              },\n",
       "            });\n",
       "          </script>\n",
       "          "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting test.json\n"
     ]
    }
   ],
   "source": [
    "%writefile test.json\n",
    "{\"imageurl\": \"gs://cloud-ml-data/img/flower_photos/sunflowers/1022552002_2b93faf9e7_n.jpg\"}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Send it to the prediction service"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "          <script src=\"/static/components/requirejs/require.js\"></script>\n",
       "          <script>\n",
       "            requirejs.config({\n",
       "              paths: {\n",
       "                base: '/static/base',\n",
       "              },\n",
       "            });\n",
       "          </script>\n",
       "          "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CLASS       CLASSID  PROBABILITIES\n",
      "sunflowers  3        [0.005260971374809742, 0.0005250766407698393, 0.008500268682837486, 0.9802882671356201, 0.0054253824055194855]\n"
     ]
    }
   ],
   "source": [
    "%bash\n",
    "gcloud ml-engine predict --model=flowers --version=${MODEL_TYPE} --json-instances=./test.json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<pre>\n",
    "# Copyright 2017 Google Inc. All Rights Reserved.\n",
    "#\n",
    "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "#      http://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License.\n",
    "</pre>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}