Chapter08/ch8_nb1_action_recognition.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-28T16:34:03.288666Z",
     "start_time": "2019-03-28T16:34:03.284827Z"
    }
   },
   "source": [
    "<p style=\"border: 1px solid #e7692c; border-left: 15px solid #e7692c; padding: 10px; text-align:justify;\">\n",
    "    <strong style=\"color: #e7692c\">Tip.</strong> <a style=\"color: #000000;\" href=\"https://nbviewer.jupyter.org/github/PacktPublishing/Hands-On-Computer-Vision-with-Tensorflow/blob/master/Chapter08/ch8_nb1_action_recognition.ipynb\" title=\"View with Jupyter Online\">Click here to view this notebook on <code>nbviewer.jupyter.org</code></a>. \n",
    "    <br/>These notebooks are better read there, as Github default viewer ignores some of the formatting and interactive content.\n",
    "    </p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-28T16:34:19.809997Z",
     "start_time": "2019-03-28T16:34:19.803458Z"
    }
   },
   "source": [
    "<table style=\"font-size: 1em; padding: 0; margin: 0;\">\n",
    "    <tr style=\"vertical-align: top; padding: 0; margin: 0;background-color: #ffffff\">\n",
    "        <td style=\"vertical-align: top; padding: 0; margin: 0; padding-right: 15px;\">\n",
    "    <p style=\"background: #363636; color:#ffffff; text-align:justify; padding: 10px 25px;\">\n",
    "        <strong style=\"font-size: 1.0em;\"><span style=\"font-size: 1.2em;\"><span style=\"color: #e7692c;\">Hands-on</span> Computer Vision with TensorFlow 2</span><br/>by <em>Eliot Andres</em> & <em>Benjamin Planche</em> (Packt Pub.)</strong><br/><br/>\n",
    "        <strong>> Chapter 8: Video and Recurrent Neural Networks</strong><br/>\n",
    "    </p>\n",
    "\n",
    "<h1 style=\"width: 100%; text-align: left; padding: 0px 25px;\"><small style=\"color: #e7692c;\">Notebook 1:</small><br/>Action recognition in video using LSTMs </h1>\n",
    "<br/>\n",
    "<p style=\"border-left: 15px solid #363636; text-align:justify; padding: 0 10px;\">\n",
    "    In this chapter, we covered the inner workings of the basic RNN as well as LSTMs.\n",
    "<br/><br/>\n",
    "    As a practical application for this new type of neural networks, we will build a model to recognize actions in videos.\n",
    "</p>\n",
    "<br/>\n",
    "\n",
    "<p style=\"border-left: 15px solid #363636; text-align:justify; padding: 0 10px;\">\n",
    "    <strong> Requirements </strong>\n",
    "<br/><br/>\n",
    "    To run this notebook, you need to download the <a href=\"https://www.crcv.ucf.edu/data/UCF101.php\">UCF101 dataset</a> and extract it. When done, change the `BASE_PATH` variable to point to the dataset folder.\n",
    "</p>\n",
    "<br/>\n",
    "<p style=\"border-left: 15px solid #e7692c; padding: 0 10px; text-align:justify;\">\n",
    "    <strong style=\"color: #e7692c;\">Tip.</strong> The notebooks shared on this git repository illustrate some notions from the book \"<em><strong>Hands-on Computer Vision with TensorFlow 2</strong></em>\" written by Eliot Andres and Benjamin Planche, published by Packt. If you enjoyed the insights shared here, <a href=\"https://www.amazon.com/Hands-Computer-Vision-TensorFlow-processing/dp/1788830644\" title=\"Learn more about the book!\"><strong>please consider acquiring the book!</strong></a>\n",
    "<br/><br/>\n",
    "The book provides further guidance for those eager to learn about computer vision and to harness the power of TensorFlow 2 and Keras to build efficient recognition systems for object detection, segmentation, video processing, smartphone applications, and more.</p>\n",
    "        </td>\n",
    "        <td style=\"vertical-align: top; padding: 0; margin: 0; width: 280px;\">\n",
    "    <a href=\"https://www.amazon.com/Hands-Computer-Vision-TensorFlow-processing/dp/1788830644\" title=\"Learn more about the book!\" target=\"_blank\">\n",
    "        <img src=\"../banner_images/book_cover.png\" width=280>\n",
    "    </a>\n",
    "    <p style=\"background: #e7692c; color:#ffffff; padding: 10px; text-align:justify;\"><strong>Leverage deep learning to create powerful image processing apps with TensorFlow 2 and Keras. <br/></strong>Get the book for more insights!</p>\n",
    "    <ul style=\"height: 32px; white-space: nowrap; text-align: center; margin: 0px; padding: 0px; padding-top: 10px;\">\n",
    "    <li style=\"display: block;height: 100%;float: left;vertical-align: middle;margin: 0 25px 10px;padding: 0px;\">\n",
    "        <a href=\"https://www.amazon.com/Hands-Computer-Vision-TensorFlow-processing/dp/1788830644\" title=\"Get the book on Amazon (paperback or Kindle version)!\" target=\"_blank\">\n",
    "        <img style=\"vertical-align: middle; max-width: 72px; max-height: 32px;\" src=\"../banner_images/logo_amazon.png\" width=\"75px\">\n",
    "        </a>\n",
    "    </li>\n",
    "    <li style=\"display: inline-block;height: 100%;vertical-align: middle;float: right;margin: -5px 25px 10px;padding: 0px;\">\n",
    "        <a href=\"https://www.packtpub.com/application-development/hands-computer-vision-tensorflow-2\" title=\"Get your Packt book (paperback, PDF, ePUB, or MOBI version)!\" target=\"_blank\">\n",
    "        <img style=\"vertical-align: middle; max-width: 72px; max-height: 32px;\" src=\"../banner_images/logo_packt.png\" width=\"75px\">\n",
    "        </a>\n",
    "    </li>\n",
    "    </ul>\n",
    "        </td>\n",
    "        </tr>\n",
    "        </table>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-05-26T16:00:52.595561Z",
     "start_time": "2019-05-26T16:00:50.075919Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: opencv-python in /root/anaconda3/envs/tensorflow2/lib/python3.6/site-packages (4.0.0.21)\n",
      "Requirement already satisfied: numpy>=1.11.3 in /root/anaconda3/envs/tensorflow2/lib/python3.6/site-packages (from opencv-python) (1.15.4)\n",
      "Requirement already satisfied: matplotlib in /root/anaconda3/envs/tensorflow2/lib/python3.6/site-packages (3.0.3)\n",
      "Requirement already satisfied: numpy>=1.10.0 in /root/anaconda3/envs/tensorflow2/lib/python3.6/site-packages (from matplotlib) (1.15.4)\n",
      "Requirement already satisfied: kiwisolver>=1.0.1 in /root/anaconda3/envs/tensorflow2/lib/python3.6/site-packages (from matplotlib) (1.0.1)\n",
      "Requirement already satisfied: python-dateutil>=2.1 in /root/anaconda3/envs/tensorflow2/lib/python3.6/site-packages (from matplotlib) (2.7.5)\n",
      "Requirement already satisfied: cycler>=0.10 in /root/anaconda3/envs/tensorflow2/lib/python3.6/site-packages (from matplotlib) (0.10.0)\n",
      "Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /root/anaconda3/envs/tensorflow2/lib/python3.6/site-packages (from matplotlib) (2.3.1)\n",
      "Requirement already satisfied: setuptools in /root/anaconda3/envs/tensorflow2/lib/python3.6/site-packages (from kiwisolver>=1.0.1->matplotlib) (40.6.3)\n",
      "Requirement already satisfied: six>=1.5 in /root/anaconda3/envs/tensorflow2/lib/python3.6/site-packages (from python-dateutil>=2.1->matplotlib) (1.12.0)\n",
      "Requirement already satisfied: tqdm in /root/anaconda3/envs/tensorflow2/lib/python3.6/site-packages (4.31.1)\n",
      "Requirement already satisfied: scikit-learn in /root/anaconda3/envs/tensorflow2/lib/python3.6/site-packages (0.20.3)\n",
      "Requirement already satisfied: numpy>=1.8.2 in /root/anaconda3/envs/tensorflow2/lib/python3.6/site-packages (from scikit-learn) (1.15.4)\n",
      "Requirement already satisfied: scipy>=0.13.3 in /root/anaconda3/envs/tensorflow2/lib/python3.6/site-packages (from scikit-learn) (1.2.1)\n"
     ]
    }
   ],
   "source": [
    "# Install packages in the current environment\n",
    "import sys\n",
    "!{sys.executable} -m pip install opencv-python \n",
    "!{sys.executable} -m pip install matplotlib\n",
    "!{sys.executable} -m pip install tqdm\n",
    "!{sys.executable} -m pip install scikit-learn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-05-26T16:00:53.810865Z",
     "start_time": "2019-05-26T16:00:52.598200Z"
    },
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import os\n",
    "import cv2\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import tqdm\n",
    "from sklearn.preprocessing import LabelBinarizer\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-05-26T16:00:53.813936Z",
     "start_time": "2019-05-26T16:00:53.811881Z"
    },
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "BASE_PATH = '../data/UCF-101'\n",
    "VIDEOS_PATH = os.path.join(BASE_PATH, '**','*.avi')\n",
    "SEQUENCE_LENGTH = 40\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1 - Extract features from videos and cache them in files"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-08T00:57:45.101242Z",
     "start_time": "2019-04-08T00:57:45.096277Z"
    }
   },
   "source": [
    "### Sample 'SEQUENCE_LENGTH' frames from each video"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-05-26T16:00:54.250300Z",
     "start_time": "2019-05-26T16:00:53.815151Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: Logging before flag parsing goes to stderr.\n",
      "W0526 18:00:54.234601 140024173102848 deprecation.py:323] From /root/anaconda3/envs/tensorflow2/lib/python3.6/site-packages/tensorflow/python/data/ops/dataset_ops.py:499: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "tf.py_func is deprecated in TF V2. Instead, there are two\n",
      "    options available in V2.\n",
      "    - tf.py_function takes a python function which manipulates tf eager\n",
      "    tensors instead of numpy arrays. It's easy to convert a tf eager tensor to\n",
      "    an ndarray (just call tensor.numpy()) but having access to eager tensors\n",
      "    means `tf.py_function`s can use accelerators such as GPUs as well as\n",
      "    being differentiable using a gradient tape.\n",
      "    - tf.numpy_function maintains the semantics of the deprecated tf.py_func\n",
      "    (it is not differentiable, and manipulates numpy arrays). It drops the\n",
      "    stateful argument making all functions stateful.\n",
      "    \n"
     ]
    }
   ],
   "source": [
    "def frame_generator():\n",
    "    video_paths = tf.io.gfile.glob(VIDEOS_PATH)\n",
    "    np.random.shuffle(video_paths)\n",
    "    for video_path in video_paths:\n",
    "        frames = []\n",
    "        cap = cv2.VideoCapture(video_path)\n",
    "        num_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))\n",
    "        sample_every_frame = max(1, num_frames // SEQUENCE_LENGTH)\n",
    "        current_frame = 0\n",
    "\n",
    "        label = os.path.basename(os.path.dirname(video_path))\n",
    "\n",
    "        max_images = SEQUENCE_LENGTH\n",
    "        while True:\n",
    "            success, frame = cap.read()\n",
    "            if not success:\n",
    "                break\n",
    "\n",
    "            if current_frame % sample_every_frame == 0:\n",
    "                # OPENCV reads in BGR, tensorflow expects RGB so we invert the order\n",
    "                frame = frame[:, :, ::-1]\n",
    "                img = tf.image.resize(frame, (299, 299))\n",
    "                img = tf.keras.applications.inception_v3.preprocess_input(\n",
    "                    img)\n",
    "                max_images -= 1\n",
    "                yield img, video_path\n",
    "\n",
    "            if max_images == 0:\n",
    "                break\n",
    "            current_frame += 1\n",
    "\n",
    "# `from_generator` might throw a warning, expected to disappear in upcoming versions:\n",
    "# https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/data/Dataset#for_example_2\n",
    "dataset = tf.data.Dataset.from_generator(frame_generator,\n",
    "             output_types=(tf.float32, tf.string),\n",
    "             output_shapes=((299, 299, 3), ()))\n",
    "\n",
    "dataset = dataset.batch(16).prefetch(tf.data.experimental.AUTOTUNE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feature extraction model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-05-26T16:00:58.352463Z",
     "start_time": "2019-05-26T16:00:54.251438Z"
    },
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "inception_v3 = tf.keras.applications.InceptionV3(include_top=False, weights='imagenet')\n",
    "\n",
    "x = inception_v3.output\n",
    "\n",
    "# We add Average Pooling to transform the feature map from\n",
    "# 8 * 8 * 2048 to 1 x 2048, as we don't need spatial information\n",
    "pooling_output = tf.keras.layers.GlobalAveragePooling2D()(x)\n",
    "\n",
    "feature_extraction_model = tf.keras.Model(inception_v3.input, pooling_output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extract features and store them in .npy files\n",
    "\n",
    "Extraction takes about ~1h20 minutes on an NVIDIA 1080 GPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-05-26T16:02:20.594553Z",
     "start_time": "2019-05-26T16:01:03.420149Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "13320it [01:05:16,  3.35it/s]"
     ]
    }
   ],
   "source": [
    "current_path = None\n",
    "all_features = []\n",
    "\n",
    "for img, batch_paths in tqdm.tqdm(dataset):\n",
    "    batch_features = feature_extraction_model(img)\n",
    "    batch_features = tf.reshape(batch_features, \n",
    "                              (batch_features.shape[0], -1))\n",
    "    \n",
    "    for features, path in zip(batch_features.numpy(), batch_paths.numpy()):\n",
    "        if path != current_path and current_path is not None:\n",
    "            output_path = current_path.decode().replace('.avi', '.npy')\n",
    "            np.save(output_path, all_features)\n",
    "            all_features = []\n",
    "            \n",
    "        current_path = path\n",
    "        all_features.append(features)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-08T01:00:03.256769Z",
     "start_time": "2019-04-08T01:00:03.251932Z"
    }
   },
   "source": [
    "## Step 2: Train the LSTM on video features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Labels preprocessing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-05-26T16:02:24.834899Z",
     "start_time": "2019-05-26T16:02:24.826062Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LabelBinarizer(neg_label=0, pos_label=1, sparse_output=False)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "LABELS = ['UnevenBars','ApplyLipstick','TableTennisShot','Fencing','Mixing','SumoWrestling','HulaHoop','PommelHorse','HorseRiding','SkyDiving','BenchPress','GolfSwing','HeadMassage','FrontCrawl','Haircut','HandstandWalking','Skiing','PlayingDaf','PlayingSitar','FrisbeeCatch','CliffDiving','BoxingSpeedBag','Kayaking','Rafting','WritingOnBoard','VolleyballSpiking','Archery','MoppingFloor','JumpRope','Lunges','BasketballDunk','Surfing','SkateBoarding','FloorGymnastics','Billiards','CuttingInKitchen','BlowingCandles','PlayingCello','JugglingBalls','Drumming','ThrowDiscus','BaseballPitch','SoccerPenalty','Hammering','BodyWeightSquats','SoccerJuggling','CricketShot','BandMarching','PlayingPiano','BreastStroke','ApplyEyeMakeup','HighJump','IceDancing','HandstandPushups','RockClimbingIndoor','HammerThrow','WallPushups','RopeClimbing','Basketball','Shotput','Nunchucks','WalkingWithDog','PlayingFlute','PlayingDhol','PullUps','CricketBowling','BabyCrawling','Diving','TaiChi','YoYo','BlowDryHair','PushUps','ShavingBeard','Knitting','HorseRace','TrampolineJumping','Typing','Bowling','CleanAndJerk','MilitaryParade','FieldHockeyPenalty','PlayingViolin','Skijet','PizzaTossing','LongJump','PlayingTabla','PlayingGuitar','BrushingTeeth','PoleVault','Punch','ParallelBars','Biking','BalanceBeam','Swing','JavelinThrow','Rowing','StillRings','SalsaSpin','TennisSwing','JumpingJack','BoxingPunchingBag'] \n",
    "encoder = LabelBinarizer()\n",
    "encoder.fit(LABELS)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Defining the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-05-26T17:54:21.102700Z",
     "start_time": "2019-05-26T17:54:21.094574Z"
    }
   },
   "outputs": [],
   "source": [
    "model = tf.keras.Sequential([\n",
    "    tf.keras.layers.Masking(mask_value=0.),\n",
    "    tf.keras.layers.LSTM(512, dropout=0.5, recurrent_dropout=0.5),\n",
    "    tf.keras.layers.Dense(256, activation='relu'),\n",
    "    tf.keras.layers.Dropout(0.5),\n",
    "    tf.keras.layers.Dense(len(LABELS), activation='softmax')\n",
    "])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-05-26T17:54:21.368352Z",
     "start_time": "2019-05-26T17:54:21.362664Z"
    },
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model.compile(loss='categorical_crossentropy',\n",
    "              optimizer='rmsprop',\n",
    "              metrics=['accuracy', 'top_k_categorical_accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-05-26T17:54:21.615229Z",
     "start_time": "2019-05-26T17:54:21.594034Z"
    },
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "test_file = os.path.join('data', 'testlist01.txt')\n",
    "train_file = os.path.join('data', 'trainlist01.txt')\n",
    "\n",
    "with open('data/testlist01.txt') as f:\n",
    "    test_list = [row.strip() for row in list(f)]\n",
    "\n",
    "with open('data/trainlist01.txt') as f:\n",
    "    train_list = [row.strip() for row in list(f)]\n",
    "    train_list = [row.split(' ')[0] for row in train_list]\n",
    "\n",
    "\n",
    "def make_generator(file_list):\n",
    "    def generator():\n",
    "        np.random.shuffle(file_list)\n",
    "        for path in file_list:\n",
    "            full_path = os.path.join(BASE_PATH, path).replace('.avi', '.npy')\n",
    "\n",
    "            label = os.path.basename(os.path.dirname(path))\n",
    "            features = np.load(full_path)\n",
    "\n",
    "            padded_sequence = np.zeros((SEQUENCE_LENGTH, 2048))\n",
    "            padded_sequence[0:len(features)] = np.array(features)\n",
    "\n",
    "            transformed_label = encoder.transform([label])\n",
    "            yield padded_sequence, transformed_label[0]\n",
    "    return generator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-05-26T17:54:21.895631Z",
     "start_time": "2019-05-26T17:54:21.868124Z"
    },
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train_dataset = tf.data.Dataset.from_generator(make_generator(train_list),\n",
    "                 output_types=(tf.float32, tf.int16),\n",
    "                 output_shapes=((SEQUENCE_LENGTH, 2048), (len(LABELS))))\n",
    "train_dataset = train_dataset.batch(16).prefetch(tf.data.experimental.AUTOTUNE)\n",
    "\n",
    "\n",
    "valid_dataset = tf.data.Dataset.from_generator(make_generator(test_list),\n",
    "                 output_types=(tf.float32, tf.int16),\n",
    "                 output_shapes=((SEQUENCE_LENGTH, 2048), (len(LABELS))))\n",
    "valid_dataset = valid_dataset.batch(16).prefetch(tf.data.experimental.AUTOTUNE)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-05-26T18:03:35.186985Z",
     "start_time": "2019-05-26T17:54:22.387196Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/17\n",
      "      1/Unknown - 2s 2s/step - loss: 4.5858 - accuracy: 0.0625 - top_k_categorical_accuracy: 0.1250"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "W0526 19:54:25.142841 140024173102848 callbacks.py:257] Method (on_train_batch_end) is slow compared to the batch update (0.223131). Check your callbacks.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "597/597 [==============================] - 36s 60ms/step - loss: 3.5954 - accuracy: 0.0949 - top_k_categorical_accuracy: 0.2145 - val_loss: 0.0000e+00 - val_accuracy: 0.0000e+00 - val_top_k_categorical_accuracy: 0.0000e+00\n",
      "Epoch 2/17\n",
      "597/597 [==============================] - 32s 54ms/step - loss: 2.1150 - accuracy: 0.3860 - top_k_categorical_accuracy: 0.6971 - val_loss: 1.7180 - val_accuracy: 0.5379 - val_top_k_categorical_accuracy: 0.8245\n",
      "Epoch 3/17\n",
      "597/597 [==============================] - 32s 54ms/step - loss: 1.5764 - accuracy: 0.5209 - top_k_categorical_accuracy: 0.8247 - val_loss: 1.4598 - val_accuracy: 0.5908 - val_top_k_categorical_accuracy: 0.8570\n",
      "Epoch 4/17\n",
      "597/597 [==============================] - 32s 54ms/step - loss: 1.3100 - accuracy: 0.6132 - top_k_categorical_accuracy: 0.8826 - val_loss: 1.3417 - val_accuracy: 0.6410 - val_top_k_categorical_accuracy: 0.8594\n",
      "Epoch 5/17\n",
      "597/597 [==============================] - 32s 54ms/step - loss: 1.1325 - accuracy: 0.6662 - top_k_categorical_accuracy: 0.9100 - val_loss: 1.3027 - val_accuracy: 0.6521 - val_top_k_categorical_accuracy: 0.8824\n",
      "Epoch 6/17\n",
      "597/597 [==============================] - 32s 54ms/step - loss: 1.0173 - accuracy: 0.7064 - top_k_categorical_accuracy: 0.9248 - val_loss: 1.4239 - val_accuracy: 0.6561 - val_top_k_categorical_accuracy: 0.8750\n",
      "Epoch 7/17\n",
      "597/597 [==============================] - 32s 54ms/step - loss: 0.9143 - accuracy: 0.7302 - top_k_categorical_accuracy: 0.9358 - val_loss: 1.2006 - val_accuracy: 0.7021 - val_top_k_categorical_accuracy: 0.9062\n",
      "Epoch 8/17\n",
      "597/597 [==============================] - 32s 54ms/step - loss: 0.8686 - accuracy: 0.7478 - top_k_categorical_accuracy: 0.9427 - val_loss: 1.3447 - val_accuracy: 0.6638 - val_top_k_categorical_accuracy: 0.8887\n",
      "Epoch 9/17\n",
      "597/597 [==============================] - 32s 54ms/step - loss: 0.7727 - accuracy: 0.7748 - top_k_categorical_accuracy: 0.9523 - val_loss: 1.2708 - val_accuracy: 0.6693 - val_top_k_categorical_accuracy: 0.9001\n",
      "Epoch 10/17\n",
      "597/597 [==============================] - 32s 54ms/step - loss: 0.7559 - accuracy: 0.7685 - top_k_categorical_accuracy: 0.9528 - val_loss: 1.1961 - val_accuracy: 0.7026 - val_top_k_categorical_accuracy: 0.9022\n",
      "Epoch 11/17\n",
      "597/597 [==============================] - 32s 54ms/step - loss: 0.7049 - accuracy: 0.7966 - top_k_categorical_accuracy: 0.9580 - val_loss: 1.4150 - val_accuracy: 0.6727 - val_top_k_categorical_accuracy: 0.8972\n",
      "Epoch 12/17\n",
      "597/597 [==============================] - 32s 54ms/step - loss: 0.6784 - accuracy: 0.8165 - top_k_categorical_accuracy: 0.9636 - val_loss: 1.3719 - val_accuracy: 0.6923 - val_top_k_categorical_accuracy: 0.9011\n",
      "Epoch 13/17\n",
      "597/597 [==============================] - 32s 54ms/step - loss: 0.6610 - accuracy: 0.8179 - top_k_categorical_accuracy: 0.9679 - val_loss: 1.2835 - val_accuracy: 0.6833 - val_top_k_categorical_accuracy: 0.9025\n",
      "Epoch 14/17\n",
      "597/597 [==============================] - 32s 54ms/step - loss: 0.6007 - accuracy: 0.8305 - top_k_categorical_accuracy: 0.9726 - val_loss: 1.4945 - val_accuracy: 0.6907 - val_top_k_categorical_accuracy: 0.9025\n",
      "Epoch 15/17\n",
      "597/597 [==============================] - 32s 54ms/step - loss: 0.6059 - accuracy: 0.8358 - top_k_categorical_accuracy: 0.9742 - val_loss: 1.3228 - val_accuracy: 0.7050 - val_top_k_categorical_accuracy: 0.9001\n",
      "Epoch 16/17\n",
      "597/597 [==============================] - 32s 54ms/step - loss: 0.6244 - accuracy: 0.8341 - top_k_categorical_accuracy: 0.9700 - val_loss: 1.3025 - val_accuracy: 0.7164 - val_top_k_categorical_accuracy: 0.9194\n",
      "Epoch 17/17\n",
      "597/597 [==============================] - 32s 54ms/step - loss: 0.5961 - accuracy: 0.8413 - top_k_categorical_accuracy: 0.9764 - val_loss: 1.3090 - val_accuracy: 0.7026 - val_top_k_categorical_accuracy: 0.9149\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f5630f3f518>"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir='/tmp', update_freq=1000)\n",
    "model.fit(train_dataset, epochs=17, callbacks=[tensorboard_callback], validation_data=valid_dataset)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "tf2",
   "language": "python",
   "name": "tf2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  },
  "notify_time": "5"
 },
 "nbformat": 4,
 "nbformat_minor": 2
}