Image_Recognition_Tensorflow_CoreML.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Image_Recognition_Tensorflow_CoreML.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/TeyimPila/machine-learning/blob/master/Image_Recognition_Tensorflow_CoreML.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D_tG7dsh0QsF",
        "colab_type": "text"
      },
      "source": [
        "# Intro"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e-rxun5qXCvH",
        "colab_type": "text"
      },
      "source": [
        "In this post, you  are going to build a basic image classification model for processing images sent by members of a conversation in an iOS app integrated with Nexmo In-App Messaging. After a user uploads an image, a caption describing the image will appear along with the image!\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o0EvKMrbNh2P",
        "colab_type": "text"
      },
      "source": [
        "We're going to use Python to build our image classifcation model. Don't worry if you have not worked with Python or you have no prior knowlege into machine learning.  "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JDSbuXrJOlKR",
        "colab_type": "text"
      },
      "source": [
        "# What is image classification?\n",
        "\n",
        "---\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QJr5hImZOoA3",
        "colab_type": "text"
      },
      "source": [
        "Image classification in machine learning is when you have a photo, and the machine learning model will be able to tell what subject is in the photo. For example, if you take a picture of a dog, the machine learning model will be able to say \"This is a dog\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aUI6E5DlogYu",
        "colab_type": "text"
      },
      "source": [
        "First, in order to build a machine learning model, we need to have data in order to train our model.\n",
        "\n",
        "A machine learning model uses training data for the model to learn. To start, we'll need to choose the training data. For this post, we'll use the  [CIFAIR-10](https://www.cs.toronto.edu/~kriz/cifar.html) data set.\n",
        "\n",
        "This data set contains images in 10 classes, with 6000 images per class. Its a well used data set for machine learning, and it will be a good start for our project. Since the data set is fairly small, we can train the model locally. \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X5QaMahPWAnG",
        "colab_type": "text"
      },
      "source": [
        "# Running this notebook"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kSU76Me2WIPU",
        "colab_type": "text"
      },
      "source": [
        "This notebook is currently being hosted on [Google Colab](https://colab.research.google.com). Colaboratory is a free Jupyter notebook environment that requires no setup and runs entirely in the cloud. \n",
        "\n",
        "You can view this notebook  [here](https://colab.research.google.com/drive/1JHTWFhkp1_KjJV5D9ww4iz078k8iUXBA). Note, you will need to have a Google account to run the notebook. \n",
        "\n",
        "Running the notebook is super easy,  in every cell that contains code, there is a run button to the left of the cell. Tap the run button to run the code. You can also use the keyboard command `Shift` then `Enter`\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LnKvGq42Ybs2",
        "colab_type": "text"
      },
      "source": [
        "# Building out the Model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xuZFgrFjXTOp",
        "colab_type": "text"
      },
      "source": [
        "The first thing we need to do is import our pacakges. \n",
        "These packages are preinstalled on Google Collab so we don't need to install them."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "31G4vI2XXYnC",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import tensorflow as tf\n",
        "from tensorflow import keras\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dWzsH7Gkdi8N",
        "colab_type": "text"
      },
      "source": [
        "Notice, were using Tensorflow and Keras as a frontend to Tensorflow.\n",
        "Keras is a great framework that allows you to build models easier, without having to use the more verbose methods in Tensorflow. More information about Keras is [here](https://www.tensorflow.org/guide/keras)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DXYThnXaaYkW",
        "colab_type": "text"
      },
      "source": [
        "Next, we'll load the CIFAR data set. Using Keras, we're able to download the dataset very easily.\n",
        "\n",
        "We split the dataset into 2 groups, one for training `(x_train, y_train)`, the other for testing `(x_test, y_test)`. \n",
        "\n",
        "Splitting the data set allows the model to learn from the training set, then, when we test the model, we want to see how well it learned, by using the test set. This will give us our accuracy, or, how we'll the model did."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "H6hioSlLX-KU",
        "colab_type": "code",
        "outputId": "47df65fa-547b-4ba8-f28c-609137a50ac7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 170
        }
      },
      "source": [
        "from keras.datasets import cifar10\n",
        "\n",
        "(x_train, y_train), (x_test, y_test) = cifar10.load_data()\n",
        "print('x_train shape:', x_train.shape)\n",
        "print(x_train.shape[0], 'train samples')\n",
        "\n",
        "print('y_train shape', y_train.shape)\n",
        "print(x_test.shape[0], 'test samples')\n",
        "\n",
        "print('x_test shape', x_test.shape)\n",
        "print(y_test.shape[0], 'test samples')\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Using TensorFlow backend.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Downloading data from https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz\n",
            "170500096/170498071 [==============================] - 75s 0us/step\n",
            "x_train shape: (50000, 32, 32, 3)\n",
            "50000 train samples\n",
            "y_train shape (50000, 1)\n",
            "10000 test samples\n",
            "x_test shape (10000, 32, 32, 3)\n",
            "10000 test samples\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VsFtm_ODPzex",
        "colab_type": "text"
      },
      "source": [
        "Next, we'll declare some constants.\n",
        "\n",
        "*  `batch_size` is the number of samples that going to be propagated through the network. \n",
        "*  `epochs` are how many times we train on the full dataset.\n",
        "* `class_names` is a list of all the possible lables in the CIFAR-10 dataset.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aGptaSCu8Qj2",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Declare variables\n",
        "batch_size = 32 \n",
        "# 32 examples in a mini-batch, smaller batch size means more updates in one epoch\n",
        "epochs = 100\n",
        "class_names = [\"airplane\",\"automobile\",\"bird\",\"cat\",\"deer\",\"dog\",\"frog\",\"horse\",\"ship\",\"truck\"]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2O8AvJ6xlXGS",
        "colab_type": "text"
      },
      "source": [
        "We'll use this constants later, when coverting our model into CoreML."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PIQ3hprb7_fV",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def plot_images(x, y, number_of_images=2):\n",
        "  fig, axes1 = plt.subplots(number_of_images,number_of_images,figsize=(10,10))\n",
        "  for j in range(number_of_images):\n",
        "      for k in range(number_of_images):\n",
        "          i = np.random.choice(range(len(x)))\n",
        "          title = class_names[y[i:i+1][0][0]]\n",
        "          axes1[j][k].title.set_text(title)\n",
        "          axes1[j][k].set_axis_off()\n",
        "          axes1[j][k].imshow(x[i:i+1][0])        "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mL4y0R5a1lQt",
        "colab_type": "text"
      },
      "source": [
        "First, lets have a look at the images which are 32x32 in size.\n",
        "We have a function that plots a series of images and their corresponding label"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OSqihPKPDmaA",
        "colab_type": "code",
        "outputId": "d77c733f-a30b-4a2a-ec07-97e880185cb7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 606
        }
      },
      "source": [
        "plot_images(x_train, y_train)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkgAAAJNCAYAAADK04ocAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3WmwZWW95/n/Wns+Y+bJPDkDmZCZ\nIArK0HqtEqmbtgoXSohobrdWg4a2UVGhdgQhYTiEEU6hL3whtoa8MrRowTBE7IpABskSq9Dr9V6Q\nxCvJZDLncHI487SHNfQLb1HywPP7p7nNg5Lfzyvln8/aa6/1rGf9cxPPj6Qsy9IAAADwovTVPgEA\nAIC/NDRIAAAAARokAACAAA0SAABAgAYJAAAgQIMEAAAQoEHCCdu/f7+de+65r1i75ZZb7Otf//oK\nnxEAnNj6o9YznJqqr/YJ4LXp2muvfbVPAcApivUHfw40SDguWZbZ5z73OXvwwQetKAo7++yz7WMf\n+5iZmf3oRz+ym2++2ebm5uwTn/iEXXnllfbNb37TJiYm7Mtf/rLt2rXL3ve+99ndd99tBw8etPe+\n9712/fXXv8rfCMBrwW233Wbf+c53LM9zGx8ft69+9av24x//+MX157rrrrMLL7zQ7r33Xvvyl79s\nN954o735zW+2X/ziF7Z//37btWuXfeELX3jJMYuisC996Uv2q1/9ynq9nl100UX2la98xWq1mn3q\nU5+yTZs22Z49e+zZZ5+1rVu32k033WStVsv27dtnn//85+3o0aNWr9ftK1/5ip133nmv0pVBv/hX\nbDguv/zlL23//v12zz332L333mvbt2+3hx9+2IqisF6vZ3fccYd9+tOfjv6s/fDDD9ttt91md955\np33/+9+3xx9/fIW/AYDXmsnJSfviF79o3/3ud+3ee++1008/3W666aaX/blHHnnE7rzzTrvwwgvN\nzOz++++3m2++2X72s5/ZAw88YD//+c9f8ud3795tDz74oP3kJz+xu+++2/bu3Wt33XXXi/V77rnH\nbrzxRtu9e7dNTU3Z7t27rSgK++hHP2pXXXWV/fSnP7XPf/7z9pGPfMSyLDu5FwEnDQ0SjsvY2Jg9\n9dRTtnv3blteXrbrr7/e3va2t1lZlnb11Vebmdm5555rExMTrzj+6quvtkqlYmvWrLGLLrrIHnro\noZU8fQCvQWvWrLHf/OY3tmHDBjMzu/jii+2FF1542Z+79NJLLU3/5+vuiiuusFarZa1Wyy655BLb\ns2fPS/78u9/9brv99tutVqtZo9Gw88477yXHvfTSS23VqlVWrVZt586ddujQIXv66adtcnLSrrnm\nGjMzu+iii2xsbOxlx8ZfD/4VG47L+eefb5/97Gfte9/7nn3yk5+0Xbt22Yc//GGrVCrWarXMzCxN\nUyuK4hXHj46OvuR/z83Nrch5A3jtyvPcvvGNb9h9991neZ7b4uKibdu27WV/7o/Xn/D/j46O2pEj\nR15Sn5qasi996Uv26KOPWpIkduzYMfvABz7wYn14ePjF/12pVCzPc5ubm7N2u22XX375i7WFhQWb\nmZnp+3vi1UGDhON22WWX2WWXXWYzMzP2mc98xr797W8f99jp6ekX//fMzMzLFiwA+FPddddddt99\n99ktt9xiY2Nj9sMf/tDuuOMOd9wfr0ezs7MvW49uvPFGq1ardscdd1i9XrcbbrjBPea6detscHDQ\n7rnnnj/9i+AvEv+KDcfl9ttvt29961tmZrZq1So788wzLUmS4x5/1113WVEUduzYMXvooYfs4osv\nPlmnCuAUMTk5aZs3b7axsTGbnp62u+++2xYXF91xu3fvtm63a0tLS3b//fe/bD2anJy0nTt3Wr1e\nt8cff9z27NljS0tL8pibN2+2DRs2vNggTU1N2cc//nF3HP5y0SDhuLzjHe+wvXv32rve9S67/PLL\nbd++ffbBD37wuMfv2LHDrrnmGrviiivsuuuusx07dpzEswVwKrjyyittZmbG3vnOd9oNN9xg119/\nvU1MTLhN0gUXXGDvf//7bdeuXfaWt7zF3v72t7+k/qEPfch+8IMf2OWXX2633nqrffKTn7TbbrvN\n7r777ugxkySxr33ta3brrbfaZZddZtdee6299a1vtYGBgT/Ld8XKS8qyLF/tk8Br265du+yrX/0q\nvxoBeNVdd911ds0119hVV131ap8K/sLxCxIAAECABgkAACDAv2IDAAAI8AsSAABAYEVykH58u86l\niIULmpm/lTzVde8HMvXZf8k/rv1xKuwr+VO24P+pvEMnyYn33d5591vv5572M5e88f3OtdL5bOvj\n8N73eu/7/rcTP/hfiZ//9jlZ7yzHa3mmr9/waEvW06q+eVMzs9Fa1/nsrNDHbvdyWe+JV0izob9X\nqS6amY0PN2X9jI1j0Vq16MmxvZ6uL/b0dZtZil+X6VkdgtuoVWR9eFDvetuwdiRa2ziiX+llqb9X\nt6uvi9qRV63q71Wr1WQ9z/Vc63Q60Vqvp/9zLkmiP3vtmqFX/Of8ggQAABCgQQIAAAjQIAEAAARo\nkAAAAAI0SAAAAAEaJAAAgMCKbPOvN+qyXoitpt7259LZv+yNV9vC+xn7auvn3Precn4S4xH63eav\neNvZPa/qfPBiH/q4JV6kxKlg2+ZxWT90YCpaS1O9/o2vWyXrSUXPy8F6/P5kzrM4OR2PCDAzm1nQ\n49MyvrV7uDUox1pFv35adb1tvCXeK41EH7vb1cfu5jqCYLQZf9ZbSUOOHRrQ8QVDLV0fHYof39tK\nv7ysv1ea6vENEd2QOItMv9E06r2SO0v3wlJX1tdG/jkrHwAAQIAGCQAAIECDBAAAEKBBAgAACNAg\nAQAABGiQAAAAAiuyzd/bPJ2L/8Kwu83fq/exdbvfz/5r3R7tbbf0tsN7/1VmxbtmJzNCwNP3XBT1\nfiMGPImdeJwFzMbE1mozs95IfGt2lun/Qnrd2rLequvPXr1pTbSWVvXzdHhARxDse/6YrM8uxedt\ne2Fejh2o6c8ecLa7Z9341u1a1VubZdlqFf0HiiK+xjUH4lvhzczGVo3IeiXV51704t970fmv2meZ\nrlcqOv4gz+LnVq3p94b3XvDWwE6nE60tOfEFi+0TW1//Ot/gAAAAJxENEgAAQIAGCQAAIECDBAAA\nEKBBAgAACNAgAQAABGiQAAAAAiuSg7Tc1jkfKofFy+RJnbqu6uwFLx+m37wgdXbOoV39nns/x+5n\nfL9ZQ/3kCXnXpJ9rZtbfdev3mhd9XPOTndH012CwrvNh1owORGtZHs+tMTNrNvSx63X999iaOLfC\nmbOb1q2W9U5HZ9fkB6ejtXZPz5vCuS5ZT+ckLSzGzy1t1eTYWs2pN3SWUc/i19x7Urulk7HU0del\nJzJ/6hV9v6tV/cpPnICobjee6VWp6mtacbOl9HxJ0/h3q9f1XKln5CABAAD8WdAgAQAABGiQAAAA\nAjRIAAAAARokAACAAA0SAABAgAYJAAAgsCI5SFmhszSKPJ5RkIjsAzOzipN05OUk9ZPJkzqZET51\n/P4yd/rN7FG8793Pdek3i8ir95O51S91Xfo5bzP9DP3rEZx63Mm+Ln8NnHgZGxpsxouJlw+j713i\np7nFxxZOhpJz5DUjOg9IxcscnV6UYzttnffTbsfzfszMet34e6XMMzl2cEhfl4WOHr/ciecBNZ1M\nnoUlnQuYivtppt8Ma9auOuGxZma9nv7euciu6nT0eTcaerZ560y93ojWanXx/JlZL9dzKYZfkAAA\nAAI0SAAAAAEaJAAAgAANEgAAQIAGCQAAIECDBAAAEKBBAgAACKxIDpKXRWSq7sS3FKWT/5JWdF0k\nQ3jJMU40jZtfoi9Lv3lAsmypzJdysqWcbKp+cnP8qaJ7ei8vyKv3M9bPMjqx2vHUE+fvOkUhnhPv\nkngRS7BqVWRcpe5KIqvevVWjS3Xfzaws9GePDOpMn2ojnk2TO5N2dkafW72uX0/tXvz4CyKnyMws\nqetsvvklndGUlPHjN5r6vOtirpiZDbZ09lSjNhStDQ8PyLFyHTCzpSWdF5Rl8ZykPNfXPM/1965W\n9XWrVtW7XK+9q4b0NY3hFyQAAIAADRIAAECABgkAACBAgwQAABCgQQIAAAjQIAEAAARWZJu/5Xpr\nYSK2g7pbxiv6K6TONn+97dHZMi6rxyPen6bOdnZ3O7yztVhd1zT1Pru/iAFVLr0tz4m3jd/bk97P\nXev3jssN2XKke02dNItSXPXCeT7dKI1TQOFsh+/n3vq8OR+veeedO/WKE+lRr8THj43ordVFV2+l\n73b1VvxaJT7pvfWz53zvXh7fzm5mNiIiCEYG49EHZmYDLf3OatR0vSbeed766K0jXrSCejV4ESze\nNn7vveNt5VdazdoJjeMXJAAAgAANEgAAQIAGCQAAIECDBAAAEKBBAgAACNAgAQAABGiQAAAAAiuS\ng5RlOlNC8bITKk52Qj8JJG4ekHNwPy8oXi9VuIn5mRPeZ6ci38Q7tndV3dwc8d28++UlYRTOdZOf\n7Y3tkzp+/5/dz1zTR85lVtipIXdycdQjkzoXuDTn+jrPclHG697am6Y6HyZx5lVVnPuqAX3sfHhA\n1qemF2W9Xolf9ETUzMwK75pbT1YrlXq01mg6OUgDOrTMW33lO895lotCZ0tVnVxBWXffOf29s/pZ\nPxP3zfLK58YvSAAAAAEaJAAAgAANEgAAQIAGCQAAIECDBAAAEKBBAgAACNAgAQAABFYkBynPdfaC\nykfw8g2KwgsjcspO9oL8bDcPSNdTcXKJkxnhZhE51yUpRYZIou+Xd09KLwdJnbuXHeUd2Qv1kcc/\n8RwOf7SZil7xY5C8a65Hp+J7+/kj+tingk7HyRMS0TbVqvMsu4Fq3ryLf/jTzzwjx3ad77Vly2my\nPjQ0GK1533toqCXrpVijzMy6WXydanc7cqyn4q5DJ76OpGqymFnqPOtLS0vR2qGDB+TY0dFRWV+z\nZq2sVyrxc09Ett6//gmnrqn11+sxvDdSJTJX+QUJAAAgQIMEAAAQoEECAAAI0CABAAAEaJAAAAAC\nNEgAAAABGiQAAIDAiuQgeYoinlLQ6/Xk2CR3MkKcPCGVAePlvxRO3o+Xk6Q+W+VNmJmlif5eqZtJ\nceLhNqWbyeOkTqgL65y29729e5KI8f2ldPjjU5FN5d+N/s4uy+PXxc05cvJoTgXtjpflFq81mvr6\nVSrOGuVe/vjcaDSacuTv/uU3sv7AA3tkfdOmjdHamTvPlGPHxtfLeqtVk/WiHa91F0TRzKqNhqw3\n63VZr9fir85eT2dLPf/8EVl/6sknZP3JJ588ofMyM3vPe94j6/57I66fTEEzfx1SWUden+Ctn43G\nK881Vj4AAIAADRIAAECABgkAACBAgwQAABCgQQIAAAjQIAEAAARWZJu/t/2vVotv5+x2u3JsIbb+\nmfkdoDozb/ezt6U89/YtiuuSi+gDM7PUiy9wvnnh7u0Wx3a3c574Nv/SOa+qmCvHI8/VfNLfy/ve\nqXNJK31cc58+NzWdvGvuzfNTQbenr0G1Gr/+aebFfehIj4oz70px77ds3qI/25k3j/wuvqXczOx3\nv9sbrf36wV/LsRs2b5b1nWe/QdY3b9kareW53mpvHX1Peh393nl6/4Fo7YXnn9Nj9z0m6/Mzk7J+\n2mnx6/b2S94mx46NrZZ1L15GTUV3eXNeGyrux8wsz+L3NHP6AOfQUfyCBAAAEKBBAgAACNAgAQAA\nBGiQAAAAAjRIAAAAARokAACAAA0SAABAYEVykMzJUalV46fRc3KQvLQiL+NFVfNcn3fptJdJxcsq\nigdD+JE5XjaK5n23fo6dJCee9+NlYaxZOyrr7eUlWZ+ba0drXraUx/vWKq/GHeyFiHjPQaGyp/SR\nVf7IqaLn5KxkYt5mhb5+7fiUNDOzgVZL1pvNRrRWqegl/owzzpD1deMbZH3n2WdGaw889KAc+7DI\nUDIze+LJZ2V905bTo7V142vl2OWlZVl/7ln92bPTc9Ha0qI+9tqxVbL+9rf/ray/5S0XRGubNm6U\nY+v1+Fw5HmqtKJ01yMvc8vPY4nXv2Gmq852i405oFAAAwGsYDRIAAECABgkAACBAgwQAABCgQQIA\nAAjQIAEAAARokAAAAAIrkoNUS3QuRCkiRmpFT45V2QhmZkla0/V6M1qrNIbk2Hamz225ozN5UhGk\nVDWd25CUOvchqerrUhHZVKUT8JQnetoUTg5S1eK5MGWi71fu5ByVywuyrq5q6s0lL2vIyeLIKvFP\nTxLnfnqxVSLn6A91cc2doVlPZwCdCipV/UzU6/F5m+U6y21uVs9ZtT7+4bPrarQ+tnPzGyJjycxs\n587t0dr4xnVy7I5zzpP1f35gj6w/9vsnorXpqUk5dnFuXtYPvLBf1jdu2hytXXb55XLsuTvPlvUN\n63VO0rp1I9Fa1cne8/LUskxPNjVfktTLOZJly52sMZWRlzgZdkl6Yq0OvyABAAAEaJAAAAACNEgA\nAAABGiQAAIAADRIAAECABgkAACBAgwQAABBYmRykNJ7BYmZWTeJ5QktLs3Ls3JHDsj62RmdKbD7z\nrGhteKwlxy4XOiPkhcOLst5eakdrSenkNzk5SbWGPve6yIXodnS+U0XkUfzhDzj3u4h/b0vjuVRm\nZtnsjKwvLznZU8Pro7XcySKqmJMR4uQ/WSpyPLxcK31kK03fk0xklJTOZ3e9Dz8FlKWe041mfN4O\nVPU6Ua+qHCMzy/WzLp/XRM9ZZ8pbtaZfEaX4O/bIyLAcu/2seIaSmVmtpnPozhIZTA8/9JAcuzyv\ns6d2/bt/J+sXXHhxtDa6alyOXbNqVNZHhvU1T8VNSxL9m0fh5KVlmZ7n3W4808vLJEydrCJvMpbi\nvdPLdNaYWnvNzGzwld+3/IIEAAAQoEECAAAI0CABAAAEaJAAAAACNEgAAAABGiQAAIDAimzzn5ua\nlPXx9Rujtamjh+TYmeefk/X5A7Js7emD0dqWrZvl2EpLb6Ufb43I+tTScrSWOduKaw29BXZwSG+X\nr6TxrcNtZyt90dNbKhdyfe4jQwPRWr0xKMfmcyIiwMyqud4qumydaK0U0QdmZqWIozAzS01vc03E\ndu3U2yIrq2al89m52P7rfLQlib6fp4Ijhydk/eGH9kdrY2vG5NgNYv0zMxse1OtIoSJBnHtbljoG\nwEv0qNfFZ3sRAk59bDS+TpiZDQ2cFq0dek6/F2Ynp2X9jRdcKOtrN8TvWaej18daXX/xmnNhsm58\nHXISIdxt/O22Pvcsi392UtGTrch0fXlZr+1PPfVMtDY7OyfHvuG8N8r6urFXfp/yCxIAAECABgkA\nACBAgwQAABCgQQIAAAjQIAEAAARokAAAAAI0SAAAAIEVyUHqObkQR4/EM0aqqQ7iWDO+XtYPH4zn\nk5iZHTlyNFprDunL0+7GM3XMzE47Y7usN0V/2nUCTDas1XlBZRrPWDIzm52bj9YGa042Sk9nThQd\nnRc0Nrw6WktyPVfKYkrW08qCrB9ZXozWWqvG5diKk1+SlLpeWj1a6ywvybGWOxlMTv6JSsXKMv2M\n1VL92aeC7Wduk/Uyjz8z//RPv5Zjjx37r7I+NqZzlF73utdFa+ecc44cW6/H56SZWenksXVFJk+S\n6OchTfXfz1eP6BykLIvP6mZN5DOZWber15mpaZ2TNLp2XbRWq+v3RqPpvHadYLIsi881bw3Kcv2s\n95ycJJVVNLcwK8c++tijsr53r64PDAxHa//+yvfIsatXxd85Cr8gAQAABGiQAAAAAjRIAAAAARok\nAACAAA0SAABAgAYJAAAgQIMEAAAQWJEcpLzQ2QwzxyajtSTXuQyNZkvW1249S9aHhuM5IHlNn3e3\nqzMlfv/Us7K+dnxDvLYuXjMzs47OnMg68WtqZpYvxfNLFno6I6S6rD97rKeziLKjM9Ha1JzOWNq6\nViX6mNVrDVkvBs+I1ibm9f0+PKuzUTZt3CLrGzbF7+nMlM7UajV0Xs3yor5ux44ei9aGhkbk2IqR\ng1RzcnV27twRrW3btlWOnZ3Vz9Njjz0u6/v27YvWWi29Pq5bF8/zMfNzktR1abb0s5qYft68az44\nGK83nOelKPR7ZXpGP+s7K/HgseHhITnWibizvNDvlVJct46TObi4GM+BMzNbWNBr9yOPPBKt/erX\nOu9rq5MldtllV8r6+vWborV6Td9vldel8AsSAABAgAYJAAAgQIMEAAAQoEECAAAI0CABAAAEaJAA\nAAACNEgAAACBFclBGl49LuvrTtscrc0eOyrH5l2dZ9FzcpRml+L5MaO1QTl2vp3L+v4D8ewZM7O8\nMhCtDY6MyrGTE/q61J2cpKUynlEyP78kx562Rud8HJnRWRxNi2eIzOg4INv7zEFZf+v5O2W96Ma/\n2+FZfT+fPnBY1hfmdNbG888ciNaajfg1MTPbsE7PhzknP2pqNn5PDh2O51KZmeWZE9xyCvDyZZIk\nnk1Tr+tsrnXr1sv6hg06Ey3P4/P2nnvukWMffPBBWX/rW98q68PDw9FalunnYWBAr6+Viv77u/re\nIyN6jRK3y8zMek4WXK0aP7deRy9iy4nOOWrn+rrNz8fX9llnHTh8WK9hv/3tb2X9nHPOjtb+43/8\nT3Ls0MgqWS9yvc70evH7vbysr1m9fmJrGL8gAQAABGiQAAAAAjRIAAAAARokAACAAA0SAABAgAYJ\nAAAgsCLb/MteW9Y788vRWi3RPdx8Oz7WzGx6Tm9hnpuNb5lM0o1ybDvX57Zx2zZZH2nFx+cTT8ix\na0r9vZdnpmQ9LeK3frRS18c+piMEji3pPbRVEZ+QpXpKzvX0sdtdZ/zkkWitUjtTjt24epOsTx16\nXtYzi29FXTW2Wo49cni/rC87z0FtOB6lcfiIjqM4dGhC1k8FvZ7eRpxl8TiRNNXrRLWq56xXHxqK\nx4VUqzU59rHHHpP1Sy65RNbb7fjaPjU1KccODcUjAszM1q5dI+sDA61obXBIRwgMOfVeT2/VL8v4\nlvNuW4/NOnrL+dzMtKwfOhCPC2kNxONbzMzqdb22HzyoY1R27frbaG3zaafJsXMLeo1qL+t4GZHq\nYHVnntfrTq5DBL8gAQAABGiQAAAAAjRIAAAAARokAACAAA0SAABAgAYJAAAgQIMEAAAQWJEcpKXp\no7Ke5fEsjfkZnY3w3/55j6wXTj7C+Nq10Vq7F8/MMTNLWjpzoiEyQszMWvPx77Z5WOeujLZ1ds32\nih5f1uJZHO2uCJwws5lZnfMxWFsl64tz8c9OOl05tujo73Xsd0/J+uD4GdHam16vc6tmFvV1OdjQ\n+SZPHnwhWstSnU9SdfJLamlD1h/f93S0Nj07J8cePBjPXTlVFEUh6yonqVbTa5CXc1SWTm7O3EK0\ndpqTTfPGN75R1iuViqxvE1lvzz//rBx7xx0/kfW/+7vLZX3Lli3R2pEjh+XYWl1f8+lpnSM3cTCe\nS7Zp4wY5dtZ53h548Dey/sbzz4/WLrwgXjMz27fv97I+NjYm60NDQ9Fat6fX7nZXvw8t1VlFFZGJ\nWDoxR97zG8MvSAAAAAEaJAAAgAANEgAAQIAGCQAAIECDBAAAEKBBAgAACNAgAQAABFYkBymt6D6s\nXm1Fa81BPfboUZ1XMbussxf2iXyYupNf0hzVeT/Dq4Zl/UAji9aWNuqxm6r6e21txrNRzMxa9Xgu\nRCryJszMms34/TIzGy1mZL2WiWyrXOcc9bq6vmq1zm0pRuOBGVNtnXt12rn/i6z3RLaUmdmeZ56N\n1rpLOltqZETPh6Sm79lyJ3783MkIWW7rczsVeHNeZRmlqb43Xs6RN15ROUVmZhs3bpT1fjKaajWd\n3dV25lWe63l54EA8V+zJJ5+QYxsNfW6LSzp/b+8jv43WmnUne2/delk/fETnBpYWX8O8+7V+vf7s\n//Af3ifr69ati9a6mV6bLdHz3BJ9v0uLj08r+nsnTk5S9LgnNgwAAOC1iwYJAAAgQIMEAAAQoEEC\nAAAI0CABAAAEaJAAAAACNEgAAACBFclBGhoZkfVMxB8cnjygD17RuTf1ekPWJyYmojUvOiGZ1VlD\ntaNN/dlp/Is/s09/+qCO2rBGU9/aVaPxHJDxsVE59vSN47JetXi+k5lZZyl+T5bmF+XYsqeP3VrW\n45f2x3Ovnu1O62OveVDWZyd1JtdyO37um87cKceu2bRa1nttndvyxgsvitb2v7Bfjl1wMmFOBZXU\nWQ3EOuTlHHn1Xs/Jl9FHl9VmU6+P5pxbtxPPY1szNibH/u9/f42sDzvvjSeeeCxaGxockGNP33am\nrE8cPizrwwPx4/e6OqOu5VzzXbt2yfrGTfHsqvkF/U5qNPRnb9y4QdaVaqHnSr2q39VZqZ+xJImP\nrzhZYYnlsh7DL0gAAAABGiQAAIAADRIAAECABgkAACBAgwQAABCgQQIAAAisyDb/WkNvd588NhOt\n/fd//LUcOzMfH2tmtmXLGbJeq8e3Di7M6y2TvUJvHexkHVmfK+Pb/OeX9LHF0D/U00FZr0/Etw7X\nU/29Ww29Hb6oOFsqk/h20KyrtzQnzm5rceg/HL+MxwAkNR0RsH6trqeFPvcjk/HrtmrTVjm2rMRj\nGczMiqq+5ioVYnBYb6feduZ2WT8VtNvLsq626qfOFmRLdD3LnedJzXnneSkLvZDUqvoAFXHqlYp+\nvWzetEnWe06kx86d50RrZ555lhybpHrL+dbTT5f1oeH4Nv9aTT+rFeeabj9rm6yr3fSLy3qeVp25\nWHHmixpeOr+3VAt98FpFZ9eo65pleu3NnHd1DL8gAQAABGiQAAAAAjRIAAAAARokAACAAA0SAABA\ngAYJAAAgQIMEAAAQWJEcpOVOV9Yf3PO7aO2pF/bLsTvP1hkta8fGZH3bti3RWqejc4y6Pf29ernO\nZuhl8WyGpXZbjl1a0vV6Vd/abm8pfmwnS6OzrL93RXwvM7NGPZ530Wg15FgvE6aX6PynUoSINCr6\nfrUaOjulVdN5X91uPNdlcFCfd6bCT8ysdAJvMpF3U6vra75qtX6GTgWdnp53ubi+Xg5S4ty70smm\nKcRne8d2c8WceVerxp+JrrM+tw8lAAAgAElEQVQOuF/MyYcaHByK1rxcHC9jafXoKllvDsSf9UpV\n5/nkmc6eKnJ9zdUtKbzMLe+aO3JxT8vSyxpyngMnm6os4/e0EDUzs9wLDozgFyQAAIAADRIAAECA\nBgkAACBAgwQAABCgQQIAAAjQIAEAAARokAAAAAIrkoP03HPPy/pv/+WRaG39xk1y7PpNG2T9Nw/8\nk6wPDw9Ha42GzoepVnSmRKupx6t8lNaQHnvGGfH8JjOzWk/nQswtzEZr84s6/8k5tFVLnTHSEllH\njWZLjm339LmpTBgzs6IXz3gaaOr8ktpwPHfFzKxRH5D1teProrVBZ65kTiZX4WSQqGwVLwcpd7Jw\nTgWZk8mjrn7p5Nokia5XUicnqY+qm8HknHtWxJ91J7LMKql+3ioVJzdHfECWO7k3TgBUrV6X9Uws\nce4a5J6azgNSSud+u3lqTkxSKoaX3jKReNfFOfdu/MWjcsjM/LyvGH5BAgAACNAgAQAABGiQAAAA\nAjRIAAAAARokAACAAA0SAABAgAYJAAAgsCI5SP/wjzqLqN3rRmvnnnW2HPvC4UOyntd0rs6RucVo\nLa3Ea2ZmqZPrkDjZKUW1Ga85wQ1H55dkvaqjiKyTxTMlSieno5Y7oRKV+P00Myva8SyiIp2XY7uJ\n/mKVig5fqfbi595qD8qxw85FzeanZL0zH7/m549vlGNbXv6Tk1ej8nAKJ7DGiek5JUwcOSzr6hJN\nHNJrlBcgc8ZpOvNs9erV4tBO7o1zb0sng0lNu9IJ1XFzbwo9L1NzFjnBm9LeuZlYI4sTzNz5H7z3\nhlqgvRwk74YX3uIvvnhSOuftXJjSeZ/qz9ZzJSv0OymGX5AAAAACNEgAAAABGiQAAIAADRIAAECA\nBgkAACBAgwQAABBYkW3+j+3bJ+tbd54brTUHh+TY6acWZL3SGJD1ahrvESsVb7+msz3aGZ+Iy1/L\n9RbWcnlO1ntOTEClXovWqs4200GnXk30tMrEZWv39DVtVvU21Uqpr1teqUdrSUvPlVKMNTMrnXPP\nxXXLnS3NpbMV39udW4o/kDvHLpxzOxXc/9/ulXW1NfvYsWNyrBezcGz7Dlk/7bTTorWxsTE5dmBA\nR1s0mvEoEjOztBJ/1hOryLGlszXb22pfpPF64qx/bryBLltF/gEnWkFlBJj/rHfzeFzI4rJ+H9ar\n+p4kzjstFRcutfg7xczMnBiAmjO8KOLfe3LqiBzb7ej4mO3bznzFf84vSAAAAAEaJAAAgAANEgAA\nQIAGCQAAIECDBAAAEKBBAgAACNAgAQAABFYkB6lwAg42nXF6tFZt6uyZ8XXrZH1+blbWS4tn37Ra\nDTm2UtP9Zbu7LOs1EZfRdMYOWUfXR3S+SXNIZP44mTpOXIVVnPySUky7dqcrx6ZJPAvDzGzIySBp\nt0ajtQNVnYM01NLXtFjS92RhOZ7R1Gy25Fgvq6gsde6LyjLyju3VTwWdBZ1lVIp8mLozZweG9bya\nmZyQ9fbCTLS2emy1HDsyMiLrg0Px58XMbGAgnlNXr+sMpaynF5paQ49PxHtF3Q8zs9TJSfJylBJ1\n6s5n97K2rM8v6Iy7o0fj82F65rAc22ro92m9qt9pKkdpwMkcbDb0+7TZ1BlNvTz+TpxycpBMZEcp\n/IIEAAAQoEECAAAI0CABAAAEaJAAAAACNEgAAAABGiQAAIAADRIAAEBgRXKQ1m7cJOtDI/GsjcIJ\n5Rkf1zlIw0PxnA4zs6KIZ9M0mjrxp1LXl6+UYRlmKv1kzMk5arV1vtPqAZ0hkqbxnI9OV2cRFaX+\nXrmTo5QnIu9iQOd01Ap9XVanuuc/1ohf9WWRkWRmNjKg67bg3LNuPE+oUtEZIFkWn6dmx5ODFL8p\nKiPJjBwkM7Ozt22WdXV/1LU3M6vW9LPqJY8lYs7rWWGWdZZkfXJJZ/ZMJvF8qKHBYTm229Vz2rsu\niZMdpng5SXUnu68Qz0TW1evA4qLOOZqaPirr7eX4+DTRn50t6RnRcN5pqciZ6zo5RnUnQC8r9Fzr\ndOejtSLXc6m75LyUIvgFCQAAIECDBAAAEKBBAgAACNAgAQAABGiQAAAAAjRIAAAAARokAACAwIrk\nIK1bp3OQyjyerVA4+S+DrQFZbzlZGWUZz7Moip4cmzn5MXWRuWNmVhXj08TJPnGyiApxTc3Makn8\n1ldTnWfRcbKpslR/diEyRrLushzbsoasVypOfkkaz1kadnKOqon+7DzRc1VljHi5LJmTRVQUenzp\nzBd97BMf+1rxD/c/KOtZFr/+aUXfm0RkkpmZVet6HVE5VV6+Vr2u53RXL4FWFc/yli1b5NiKeBbN\nzJwlzJpD8evSaul1v+tkvXnZXwsL8fyoxUW9hi0vL8r6kpNxZyILrlrqG1ZxcuLMWSdK8c5yppol\nznsj8d4bZXx9zXr6e5f5if0WxC9IAAAAARokAACAAA0SAABAgAYJAAAgQIMEAAAQoEECAAAIrMg2\n/+HBEVnvtsW2RWdbYmF6b6G7QVnusHW231b0NlXL9eXtiXNfSPT3Wrb4NTMz6zmtbyuNb88tq/p7\nL5neAtvN9Xb3RGwN9rb2tpzPnnO2+beT+NbgoqOveWbOVnyx1dvMLE/EbHTiLKz0/i7jPCdiq74X\nMeBteT4VTExMy7q6fdWavr7OlLWsmJJ1df/qdb1GeREDifOKGByKx6w899yCHJvn+rN7PX3dBsRn\nDw7qaISesy3cqy+Jd1bPe5S9rfRJW9fzeIxA1Yn78Lb5p069JtbuTs/5Xs7a3unod5q6bt4aleip\nFsUvSAAAAAEaJAAAgAANEgAAQIAGCQAAIECDBAAAEKBBAgAACNAgAQAABFYkBylxshV0hoEOTyi6\nOjvBvJyPJH78xAlPqDjHNieTwqrx3B2dhGFWVuIZIGZmvYq+5kvie3u5OB0ne8oSPa0SEXZUSRpy\n7FKi8y7aTl7QYhKv97r62LmTqpU5103FnxSF/uxU3K9/PYKuihwkVTMzy7yMplNAUToZLWKtqDlZ\nRI2W86x2dZ7Q8nJ8tUhL/Tw1q7peceZdYovR2uKinje9jp53y8t6/NRkfJ3x8p/6Volfl8x5nrxz\nGxzS62etFV8rhhotObYsnIw655XWbMbnau7k3+XOZxdOllGeixwk99gntobxCxIAAECABgkAACBA\ngwQAABCgQQIAAAjQIAEAAARokAAAAAI0SAAAAIGk9EJvAAAATjH8ggQAABCgQQIAAAjQIAEAAARo\nkAAAAAI0SAAAAAEaJAAAgAANEgAAQIAGCQAAIECDBAAAEKBBAgAACNAgAQAABGiQAAAAAjRIAAAA\nARokAACAAA0SAABAgAYJAAAgQIMEAAAQoEECAAAI0CABAAAEaJAAAAACNEgAAAABGiQAAIAADRIA\nAECABgkAACBAgwQAABCgQQIAAAjQIAEAAARokAAAAAI0SAAAAAEaJAAAgAANEgAAQIAGCQAAIECD\nBAAAEKBBAgAACNAgAQAABGiQcML2799v55577ivWbrnlFvv617++wmcEACe2/qj1DKem6qt9Anht\nuvbaa1/tUwBwimL9wZ8DDRKOS5Zl9rnPfc4efPBBK4rCzj77bPvYxz5mZmY/+tGP7Oabb7a5uTn7\nxCc+YVdeeaV985vftImJCfvyl79su3btsve97312991328GDB+29732vXX/99a/yNwLwWnDbbbfZ\nd77zHcvz3MbHx+2rX/2q/fjHP35x/bnuuuvswgsvtHvvvde+/OUv24033mhvfvOb7Re/+IXt37/f\ndu3aZV/4whdecsyiKOxLX/qS/epXv7Jer2cXXXSRfeUrX7FarWaf+tSnbNOmTbZnzx579tlnbevW\nrXbTTTdZq9Wyffv22ec//3k7evSo1et1+8pXvmLnnXfeq3Rl0C/+FRuOyy9/+Uvbv3+/3XPPPXbv\nvffa9u3b7eGHH7aiKKzX69kdd9xhn/70p6M/az/88MN222232Z133mnf//737fHHH1/hbwDgtWZy\nctK++MUv2ne/+12799577fTTT7ebbrrpZX/ukUcesTvvvNMuvPBCMzO7//777eabb7af/exn9sAD\nD9jPf/7zl/z53bt324MPPmg/+clP7O6777a9e/faXXfd9WL9nnvusRtvvNF2795tU1NTtnv3biuK\nwj760Y/aVVddZT/96U/t85//vH3kIx+xLMtO7kXASUODhOMyNjZmTz31lO3evduWl5ft+uuvt7e9\n7W1WlqVdffXVZmZ27rnn2sTExCuOv/rqq61SqdiaNWvsoosusoceemglTx/Aa9CaNWvsN7/5jW3Y\nsMHMzC6++GJ74YUXXvbnLr30UkvT//m6u+KKK6zValmr1bJLLrnE9uzZ85I//+53v9tuv/12q9Vq\n1mg07LzzznvJcS+99FJbtWqVVatV27lzpx06dMiefvppm5yctGuuucbMzC666CIbGxt72bHx14N/\nxYbjcv7559tnP/tZ+973vmef/OQnbdeuXfbhD3/YKpWKtVotMzNL09SKonjF8aOjoy/533Nzcyty\n3gBeu/I8t2984xt23333WZ7ntri4aNu2bXvZn/vj9Sf8/6Ojo3bkyJGX1KempuxLX/qSPfroo5Yk\niR07dsw+8IEPvFgfHh5+8X9XKhXL89zm5uas3W7b5Zdf/mJtYWHBZmZm+v6eeHXQIOG4XXbZZXbZ\nZZfZzMyMfeYzn7Fvf/vbxz12enr6xf89MzPzsgULAP5Ud911l9133312yy232NjYmP3whz+0O+64\nwx33x+vR7Ozsy9ajG2+80arVqt1xxx1Wr9fthhtucI+5bt06GxwctHvuuedP/yL4i8S/YsNxuf32\n2+1b3/qWmZmtWrXKzjzzTEuS5LjH33XXXVYUhR07dsweeughu/jii0/WqQI4RUxOTtrmzZttbGzM\npqen7e6777bFxUV33O7du63b7drS0pLdf//9L1uPJicnbefOnVav1+3xxx+3PXv22NLSkjzm5s2b\nbcOGDS82SFNTU/bxj3/cHYe/XDRIOC7veMc7bO/evfaud73LLr/8ctu3b5998IMfPO7xO3bssGuu\nucauuOIKu+6662zHjh0n8WwBnAquvPJKm5mZsXe+8512ww032PXXX28TExNuk3TBBRfY+9//ftu1\na5e95S1vsbe//e0vqX/oQx+yH/zgB3b55Zfbrbfeap/85Cfttttus7vvvjt6zCRJ7Gtf+5rdeuut\ndtlll9m1115rb33rW21gYODP8l2x8pKyLMtX+yTw2rZr1y776le/yq9GAF511113nV1zzTV21VVX\nvdqngr9w/IIEAAAQoEECAAAI8K/YAAAAAvyCBAAAEFiRHKRvfuf/lfVYuKCZvST99ETq3lb0/n5A\n87a5H/82+JeNdM7bqxeWn7TPdscXzviT+Zuld7/Fhyd93C8zPY+9zy4LfVHU2ONR9nFPU2fs//3B\n//OEj/3X4ts/+P9kfXZhOlorul05drTRlPWq6XlVrcbvT7VWk2NLb8660y7+B4pCr0F9LjOSt657\n36ssvfdG/Lp5x/aui3fuam33zrtw1rjMWYfSSrxlyEzPtXanJ+tja8f1+OX4c9Ro6p2CO898eXjo\nH/vbv3nDK/5zfkECAAAI0CABAAAEaJAAAAACNEgAAAABGiQAAIAADRIAAEBgRbb5VyoVWe9nW3m/\n2+H729J+Mo/tfLJz7LR0et9EbHdP+uub+9gZ7HJjG5yeX25q7iMi4A/DnflQiu257lTpcy7JrcPO\nXEpP4n7svxJpVW9hrtfj24wX2nqbf1fvtLdqVa+f1Ur8/gw6/6HUTqcj672e3pp9Mte4k6u/81aP\nU7+XxF9H9HxQUmd9rDjPuno3eDEnuRNf4MWsqNdSnulnbMH5jxfH8AsSAABAgAYJAAAgQIMEAAAQ\noEECAAAI0CABAAAEaJAAAAACK7LNvx8nd5t+v5+tx6v/4vPJlvbR+iYiAuB4FM74E99w7sszvS25\nWqtHa0Wi75d7N1P9J1K1zdX5r2j3S83VtM95fipIU721ulKJxwCklficMzPrOX9PLZ2HOS+yaC3L\n4jWzk7xN35tXznD3v2ovtpV7Y73/6r13dur4zkcfx7k548W55c7Ywjl47qxDjVYjWku8FTKJx5z8\n6wGkiphPqfPOWVpimz8AAMCfBQ0SAABAgAYJAAAgQIMEAAAQoEECAAAI0CABAAAEaJAAAAACfxE5\nSCqLI3UyQLxMiX4+u38nN9tG6edbJe5595dv0s/JefNhpBHPozEzW5ifjx+7rh+Hwv3rhHddxAHc\nTK0Tz4QxM6uIDyidDJHcOfapoNJPRkuq51UqsrnMzHqlzvbKe/F6mXbkWDfrzcl/kmOdupsH5EYV\niTnd7xqW6Idd5e54z2LqHNvLKirEhfHXCV3v5Do3qybOPanquVImeh57r2K19FedB7S9TA4SAADA\nnwUNEgAAQIAGCQAAIECDBAAAEKBBAgAACNAgAQAABGiQAAAAAiuUg+RkUqjshj5zjvrJSfIzkpxj\nO/kyJ5P30eqbJU5GiJcxcjKzpapO1sb46hFZr5fdaG2x29Yf7syl3JwcEPG4edfUncelzl5RZTc7\npc9n8LUgKXNZr4uQFj9JyMlwyfRnp3n8/lRK/XfgalW/Arw8IJWZVjGdSeblrblTXvyBxJ2zbsjS\nSfts73nzc+gEJ+eo29G5WEempmS9FJldjYFBOdbN3HJuSWLxRczP3NIZTDH8ggQAABCgQQIAAAjQ\nIAEAAARokAAAAAI0SAAAAAEaJAAAgAANEgAAQGBFcpCqIr/AzKwQuQ+Jl+/SXzyMzGZIvYN7uQ1u\nDlI/eUFOdo3z2TLryI3h6DPnKBF5Ft416+lMmG6mp/TwQPzvBO2DR/RHO/OhMjYu65nIpPHzunQ9\nde5JKspezlF6EnOt/lo0qvoa1JP4vFus6r+HFs7lPTo9K+vd5cVobefOMTm2MTQk66nIdzLTK4Hz\ntd28NZcY7h9bn1zp5An1k69XFPqllDsvrUysn1mux3Zm52T9qRdekPWk2YrWNg8Py7HmzCUvCCkV\ni5jKSDIzW1rUz1D0M09oFAAAwGsYDRIAAECABgkAACBAgwQAABCgQQIAAAjQIAEAAARWZJt/pchk\nXW6nd7fx662BZdnHVtJcbylPvG2Lam+1mR3HfvoTViT63NUeWW9rrznXNHG2wJap2pLp5DI4c6m9\npIf3FmaiteqyHpy2neiEESd6IY3XvW3FLmdrsKrm3jxnm78NNGqyXmbx+9d09rv3KvqzFxbj2/jN\nzI4cicdTvOFNF8qx9QG9NTtx1jD1zbx4lzRx4g/62ErvHbt0YgByZ7t8P9v8LddrWFE6z6OJurPu\nV8U2fTOzdZs3y/rg6lXRWumsExXnvZJWdL3T60Vrhw4fkmMPHHhO1s0++Mrn5IwCAAA45dAgAQAA\nBGiQAAAAAjRIAAAAARokAACAAA0SAABAgAYJAAAgsCI5SEXhZLQ4dcnJ5PHSKhKRh+Hlv3hZGH62\nzYl/thprZpaWTu8r8jKSIp438QdORkjhZJCIe1aqjA8zS1MdGrOmOSLrL8zORmuN8fVy7FCuH5dF\ncwJtCvHd+slVsZP7jPV5aq8JadnWdTEvmyL/yswszfTztuMMnU2z9fRN0VrNyWAqnPyspNRzWh29\n7uT9ZDW9hmVOlpEV8WyqRl6XQ/OK/t55Vd+TRK2BThZRafrYucj7MTNLevHvVjjvnFpzQNZ3nn+e\nrLez+HNQONlRWUc/Q8tz8Yw6M7OJQweitSd/v0+OnZublvUYfkECAAAI0CABAAAEaJAAAAACNEgA\nAAABGiQAAIAADRIAAECABgkAACCwIjlIea5zIfrJaPHzgrRS5Aml3rGdDKbCCZBR5+5lz6SJzidJ\nnN63LONZG/WG89kVfXLtZSfHQ2SrlE4ui8pQMjMbrumTr9aa0dpCqj+76jwuuRt7JbKnnMQuby6V\nzjOkj9/fM3RK6Ok1rNZsRWuVqpPJU+q8oFZNZ9eMDQxFa80kfl5mZlUn26uS63UkSeP1TjWeU2Rm\nljnrSO7M+ZpYA6vOnG46dSfCydSrM0/0926neo3Ky46sz87G84KarWE5tuZktdWcd3W3E1/bF7rz\ncuzcsUOyPlZ3cuga8XNfvVp/725nQdZj+AUJAAAgQIMEAAAQoEECAAAI0CABAAAEaJAAAAACNEgA\nAAABGiQAAIDAiuQgeTlHKifJyznyooq8hBeVtOFl7rhSPV5lMLmfnOiMkGZNZ3HUG/EsjqFhJ0vI\nmTX79x/Tf0BcFycGycrc+QOJrtca8ZNfXNIZIJkzH0onRykVX87NvXIuTOLMB6X0Mpa8m3IKSDKd\nH9NZFmuYyN4y83NvOk62lzXiWUdJRZ93xbn3VSejyYr4M9FN9EJRc7K/Bgudp9YoFqO1eqHPu13V\n9cL56aCWx+9pPdPZU61S38+5dlfWnzz4fLS2ev0mOXZoeJWse7+YDIuMpwEn12p2Tr8XBtfovK/l\n3lK01l2clmPLbnyswi9IAAAAARokAACAAA0SAABAgAYJAAAgQIMEAAAQoEECAAAIrMg2/9zb5i/q\naer0cM4WZGenvSXi+F6EQOFsU7VEn3sixmfONfP2hVebzrkV8fpQa0QOHRkdlfXFRb1NdXY+vuUy\nE+dlZlZN9ZQt63q8Gt7I9A335nGW6pgAK+N1LwrDq/ezVb9wrnmfYRevCXmh57TaTV8zvaW85cRD\ndIq2rA+W9Whtlelt/oO5/l6Nnt5q3xT1kSN6a3XS1vEGaXtZj+/Ft/l3FvS279w5dpnrtbvdjT8V\ny4WOWOnV9Hb2SlV/9ukiNWJ6KX5NzMyKbVtl3Zz37XJ7LlprPP97Obb56KOyPnfssKyfd+X/Gq2d\n9fodcuyPbvuxrMfwCxIAAECABgkAACBAgwQAABCgQQIAAAjQIAEAAARokAAAAAI0SAAAAIEVyUGy\njpOHkcdzQJKK7uEKkS1jZpYlXp5QvO5ELLlZRakKR3EMjgzL+uvOPVvWe4s6B+Tggf3R2vj4G+TY\n08/YKutbtpwu63sfeyxae+qpZ+TYWs2Zsk09X3oiO6Xm3O92R+fZtLv6AJUiPr4UNTOz3MkqKkrn\nORFz1ctYSiskIZWVeVkfagxFa81S39sRp96Y1zlIa2bjz3oj0zlHy4cnZD2bmZL1zvxstHZ4YVKO\n7XWcfDwnlyzL4hlMy8vxvB4zs8rkgqxXnWe5LZ7XtrNELTjvpE6pv3elvjpaywZbcuzi1tNkvXDe\nt4eeia/Po+KdYmY2Oq+zpxZG9bl3Lz43Wnvde66QY8fGV8l6DL8gAQAABGiQAAAAAjRIAAAAARok\nAACAAA0SAABAgAYJAAAgQIMEAAAQWJEcpKUjL8h6WcYzXipOllC30DlIRaIzRooinqVhudc/6suX\npnq8SrZp1vQnHzjwnKz32vG8HzOzerMZrw005NiFJZ0xUk30996+dUu0tnbVgBxbcXI6yqq+cKuH\n48dPKh05dn5CX9OZSZ33VZN5N2Iempkl+jnIy7qs6xgl/QwlXpbYKWDjE8/Kej4dz9XpHtFZQkvT\n8SwhM7MFkTVkZnZkfiZa6y3pOZsv6zmrZ5WZiuyZrutnMR+M5/mYmfXqOgsuU9lfmc7Uaa+Kr39m\nZnlPZ08NLsSva3n4kBzb6Oh7Mjqgz31uMX78bqaf1eKRf5H1usgkNDPbbvEbvsb0GjXk5Fo9l+ms\nsQf+68+itfKcs+TYd73zXbIewy9IAAAAARokAACAAA0SAABAgAYJAAAgQIMEAAAQoEECAAAI0CAB\nAAAEViQHaTHTuQ9KUujshF4mA16s3dE5HxWR8ZIUOtehkuqUEBHvZGZm9Wo8b2h2UmdCPPr7p2W9\n1dIfrvKEJqcm5dgs05k9ZVfX/+2/eWu09sYLzpNjjx6ZkPV/+KcHZT1vx/NNRgeH5Niyoh+XmbbO\n3FJ3pCx1/kjiTCZvrjmfrj+bHCT779/9nqyn4t6nuc6ZynvOvMl1Jk9Sid+f3Mkky5y6lyuWtOLP\nTHfNGXJsukXXV28/R9Zr3fh1Kyd19lS5qLOl9j/5uKzPzk5Ha61l/bysS3UGU897r4j4qOaUft+N\nOrlXa1WwlZmNiFqlqs970Mlqm3FykpaOxfO+fn3/P8qxH/q//pOsx/ALEgAAQIAGCQAAIECDBAAA\nEKBBAgAACNAgAQAABGiQAAAAAiuyzX8u0x9Tij3KRaG3TFYreit+z3Q978S30K4ZWSPHls65ja4a\nlPWqxbc9pk29JfLCM86X9V53TtaXl5fjRWfL+dLCgqwfO3xY1mdm49t3VfyAmVmn25H1hbn49lsz\ns5GB+D0ZG18vxx6Y0REDk7P6mme1+PberPC2eut7Uin1dVE7+UvTx3Z2/p4SFhZ1VIlaC5y0EMta\nut7Qu/xtpBqf0wu5jnBYcLZmZwM6+kJt828OiP3oZlZZpesTiY4LWZp8IVqbe2iPHFt7Nj7WzKzu\nrHHNMh7dsMpZwwYb+poXva6sj9UH4rVhfb8GMr3Nf8hZh9I0fk/aAzoSolbqeIORVjz2xsxseGA4\nWvv5z34px/7dO98j62ed9aZX/Of8ggQAABCgQQIAAAjQIAEAAARokAAAAAI0SAAAAAEaJAAAgAAN\nEgAAQGBFcpCmFnQQSClyd8pCh7C0BvSxx9fqbJskj+cBbR7fKMeuX7dK1s/asUnWi078e+eJ7l3n\n2l4ekM6c2LljZ7RWd3I6FuZ1Rsjs9JSsr1s/Hj/2gs4SajlZGW+/5N/KeipCfZqNeM6GmdnUrM4n\nGano3Ja5Tnx8V8fVWOn8Xabo6SwjVS5TJyuMHCRrOPenLYKmOpke3HOCkrJc3/tzxDpVLoq8MzN7\ntqvznZbbek7nnXju2LAeamtbOgDqyLNPyvpSO57pUz14VI7dNqWz2tY7c36txf/A6kTnAdWXncwz\nJ9uv1otf2IFBnYNUDCSA36kAABV9SURBVOr3QjXRc7VWj8/FodWjeqxzXcaca75tXfxdnq975Ryj\n/2Htqvg7R+EXJAAAgAANEgAAQIAGCQAAIECDBAAAEKBBAgAACNAgAQAABGiQAAAAAiuSg9TVES1W\nimyaxMkDmlnSeUDPHXxG1scG45dgeV5niCy3N8h6VujcnKFGPNNn5+viOUVmZoOjg7LeHhmR9bHV\n8QynPM/l2EZN51msGdN5GIOD8fyTxYV5OXZ+UeckbT1jmx4/Ez/+xP5Dcuz6Yf29z9+oM0aWRKbM\nbEXfz4lJnT1VONkpx5ZE5pbIdDEzy53n91RwoBfP3DEz64jL3011rli31GtcV+TEmZl1Dh+MF5t6\nTqZOVltbZO6YmVV68bVi3QZ97HxBz+m5xjpZn996erT2xkWdUXfegs5JGq/qV+NQJ35dWs766WUN\nFc5bOWvG58ti2pZj8wE91+qlXuMqIhStu+C8Lyv6GcqcvLDVFn/I/v7/+Hs5dv2W02Q9hl+QAAAA\nAjRIAAAAARokAACAAA0SAABAgAYJAAAgQIMEAAAQoEECAAAIrFAO0qKsl2U8/6Be1xki1Yqut3s6\nQ+S3ex+L1qbWjsuxe/f+XtZXj66W9V2X/JtobXB4Sh97nc4a2rhhvaw3GvHrNj83K8dWnRykWj2e\n72Rm1mzFs1nm5qfl2KHBAVmfOCQyYcxsdnImWtuwRt/vvKVzPObG9PeulvHHLd24Q4796S8e1J/d\n1plbtSKeIVI6OUhlrvNJTgk7dS5ZbSA+p8edDJYDk/p5O/Dwb2X9qRfi+V0t59a98Rx9btt36Hry\n5AvR2pon9smxB9fGc4zMzNZe/g5ZP2vzOdHaeFWvUZufvV/WG12dr9cUj0y9qn93SCv6eSsqevxS\nLZ6z1Mt0btWCjmiybEDn55XNeIZdVuprNn9YZ08lPZ3lNnc4/k48bfuZcuyyc01j+AUJAAAgQIME\nAAAQoEECAAAI0CABAAAEaJAAAAACNEgAAACBFdnmnzhbTYsi/gd6md6m38j1vsUdp50l680k3iP2\nMr1tsZLqbYm/P6S3NXb+Ib599/HDerv75s1rZH3D2JCsrx6Nb+ccbOrt6kOD8S3NZmar1+hz6/Xi\n96zo6cmyekzHGzzxxOOy/piIdXhuaJUc26zo63JsakHWO1PHorX6nBxqo3X9qFb07bZ2Nb61+NBC\nJsfmpuf5qWDzpXrLeVf8VXNkwwY59thTz8v6loPxaAozs95EfN6tdf4OPLr3GVkfmdVzevDZA9Ha\n+pl5Oba65U2yPn7ORlnfNBCP3fjpRh1zspTGt6ubmdVKfe7dNP48dUVsjZlZz6kn+rVjuYggSBK9\nTnTE+87MrDOgIwgO9+IxAr05/c7KxVgzszLTnz29EP/e//yQjsIoh9fK+oa3rXvFf84vSAAAAAEa\nJAAAgAANEgAAQIAGCQAAIECDBAAAEKBBAgAACNAgAQAABFYkB6lacTJcLJ6LU3FClM5YPybrO87c\nKuuLr4/nJD3y+CNy7Pzysqxnqc7NeeT5iXjtsM4+abT0NV0zqD97bHRY1OIZSWZmG8b1NV81Oijr\nwwPxcxsdGpBjV6+Kn7eZWZHrrI20GR9/bDaeq2Jm1qrqvKB0dFzWa434dxtcrcc2yylZnzwyqcc3\natFauugEr+goslNCc+MZsj5SiWdFpZX4tTcz2zyuj22bjsjy4t590dqQkz1TO6rnVWvisKwPivW5\n09B//x7K9XUZmtX1qshMy4b0Gjbf6cp6uaivW6UaX38z03k+vaq+LkWpH7jBtrjmiT7vJb2E2dSi\nzu6bqcUz8JoiG8rMbOCcc2T9aFuWzc6Iv6vv+dk/y6FrxnUW2bve9rZX/Of8ggQAABCgQQIAAAjQ\nIAEAAARokAAAAAI0SAAAAAEaJAAAgAANEgAAQGBFcpC6XZ05MTYUz1a4+E1vkGMveL3OVhjzcnOS\neObEjm1r5diDTvbM0/t1ltF9//i7aO2Jg/rY2YK+dVPz8WwpM7PqkXiGU1kckmObdZ1P0qrrvnug\nWRc1nd/UFGPNzIYG9HVZNRQ//kA1nmVjZjba0Oe2ekTnPxVlfJ4/9C+PybG/2/uErC+29TO2fkM8\nB6Rr+n6Wpq/5qWDj+EZZH2u2orXZIzprKE11ds1zhc6pWkjj975T1fOilep1opXrHLrFMp59U+/o\n52XqkJ7TB47qXLLG9vOitYXn9sixne6irCdZ/Fk1M+tlKotIDrUs1+vjXKIP0BX3bGC1zn/qDMbn\nqZnZ4x1nrq2L57VdkOjPHvibC2X9zf/2Ull/bjke4jQ3p0OUjj71tKzH8AsSAABAgAYJAAAgQIME\nAAAQoEECAAAI0CABAAAEaJAAAAACNEgAAACBFclBWl7WeRbpcDxzYtvpW+TY1auHZL3qfMNEZE6c\nefpmOfai83VGU5noLI3xsVXR2jf/821y7OS8zvFYjMcc/av4985znY2SVnReUKPl3BORJ1Sr6jwL\nL1NruadzPCppPL+kVsZzNszMGonOhBkd0t97tBWfD935aTl2YU4/Q4PDOoPkDa9/XbS2eo3O+5pe\n0PfkVNCemJD1fZPx3LInHtd5P08/f0B/+NIxWU5q8Sy3vNTP8pqWzirqFPp5S5L4Alvkev2bP7hX\n1p//L/+PrL+w5eJo7XXPPC/H/lNFZ38VQ7qeV+Pr53RPr83zuV6jlkq9zqzZfma0dvUlb5NjX3/B\n+bKe7XlI1n93+HC0NntQf+/DR+Jjzcy64pqamWVjY9Hats2jcuzpa+P5TQq/IAEAAARokAAAAAI0\nSAAAAAEaJAAAgAANEgAAQIAGCQAAILAi2/yTRPdhU7Pz0dqd994nxw60nO2amd5SOTYa3x69/axt\ncuzm9Xrr4OmbN8n6v3/H30RrzYre6vn4Pr2N9ffPPCXrk1PxbeUzc/H7YWa21NHbvttdvV2+vRzf\nzrkkYhfMzHKnp89MRxCUabxeMX3Nlyv63BZn9XVZWopvmR7oOVEYvZ6sF209zzeIubp2TG+RrSZT\nsn4quOWbesv5ocP7o7W5XG+Vr65dI+ub1+l1Zl5EnVQLHT1xxs7Xy7oOCdDRFxu27ZBja+v0+nju\n3IKsL7Tq0droOW+XY5d36bV9ePVqWd/77JPR2r5HHpRjD83o7e4956LvOP2saG12bIMcO7F/Rtbf\ntHa7rP/+V49HayNibTUzO3rkiKz/4q6fyvrqrTujtan1Oqpk2/h6WY/hFyQAAIAADRIAAECABgkA\nACBAgwQAABCgQQIAAAjQIAEAAARokAAAAAJ/ETlInTyeL/PsRDyvx8ysVtXZNFlnWdYtPxAtPfOC\nzm0YHR526i1Z37FjS7R28Xk6Q+RN5+i8imcPxrMyzMyOTc9Ga3ML+po980I888XM7NEnnpH1o8fi\nWRztTIeALDoZS/XCyVHK47ktmemxRTWeu2Jmltd1JtfQ6vh8OWNQz6VNo6tkvZvozz6wP56bNXtU\nP58vHDgo66eCo3N6HTrj/Hie0FzFyQVr6nuX1xqyPj8Yz58ZrOix2XadB9R15nSrHn8mNp5zkRzb\nPO1Nsr48OyfrG4bi83bEyTE6+kw8x8jMbGBQr91jZRGtDR/T7405J+ttudR5bGMbT4/Waht0ttST\ne5+Q9TLTeWszSXx9bqzVWUR/t+ttsv5f/uUxWS/b8ay4/Yf1GrW0tCjrMfyCBAAAEKBBAgAACNAg\nAQAABGiQAAAAAjRIAAAAARokAACAAA0SAABAYEVykEon1yFN4zkepZNNs5zpjJHC4sc2MytEnsVz\nR50Mpql5Wa+k8WObmT35/7dzLz1yXGUYx9+q6qrqnum25+K7HTvkSgIkEBAEViAEUT4ALBAfhTXi\nW7DMgi0EFCSwYAOWkLIgEpLj4HGM7fG4Pffprjv7I97nSEaaRfz/bd85XbdTp99p6Txbd93axc1/\nyrHXLl3W9RuXZP3NN15za1mmp8WbX9YZTetndGbPrX987NaWdSXHrkynsr4ROfb2Iz+j5OH8qRxr\nIvPFzGzj/Kas37jon9s3X9bP8/vf+oas/+fBtqz/7ZM7bq1c0/kls9lZWX8e/ODnP5P17WM/22v/\n8SM5Nun1GtaO9P+xmXhnFpFcsTt7OrNnNlmR9V7MnXuNPvZro1rWm1yvn73Iz2sbneeTrOrr6kp9\nzweRvzcudfbUOB/L+qjT82F91R8/vXhejk07fU8/+t1vZX1v5H+X39zfkWPffqTXqEcL/X367gt+\nxtMbX31Hjv3jH34v6x5+QQIAAAjQIAEAAARokAAAAAI0SAAAAAEaJAAAgAANEgAAQIAGCQAAIHAq\nOUhJJMuoqfwcjyaSc9QMOmujNZ3B1Pd+LkSS6PMe57o+SvWxl3snbu3ewydy7K2PP5H18+s6L+jt\nr33FrV2OZGlcu+LnUZiZ/fiHP5L173z7e24tSXROx9r6GVmP5bbc27rv1u7c92tmZk2i/5/Y2NAZ\nTMXg576k9bEcWzU6M2Z1RWevvP7Ky27t7PqGHHv16guy/jz4193bsj5fHLm1LvJvaNfrNaxZ6vrJ\nsX9si+Te7O7p7K++1XlCKjPt8PhAjt3f18fuer2+DoN/Y/f2dJ7afFdn9hwe6Qy8nSd+flQfeZ55\nridEGsmhU/ft7ud35dhqrp/J7Lxe+3/xq1+6tb/fuiXH/vqDD2Q9mem1PRFzebPQ2VLvvvV1Wffw\nCxIAAECABgkAACBAgwQAABCgQQIAAAjQIAEAAARokAAAAAKnss2/qvUW5bbxt5LWomZm1kUiBKpW\nb7nUO/n1Z9cjHUFQ1wtZL0Z+f7pxVm95HOe63vf6vt25c8+tHR3oLef7u2JbsZmdPbMu61euXHVr\nFy7obaZFqafsENmW/PpL/pb1L714TY5tdGqDtZ2e50PtxzosT/RcSbJM1tdn+r5NNv25WhSFHNtH\ntoo/D3YXh7LemH+PThZLOTbp9cTqItEXKhpjiIw9EfEEZmZFrufdwVHu1ra3dWxGEdmaXeS6nqb+\nuSWpvu75fFvWD4/2ZP3o2J8PTRt53pH4l0x/7Zgl/nfaUeWvMWbx6IVz13WkRz+ZuLX3fvoTOfY3\nN2/K+qf/vivrn33qR21896135NiN2aqse/gFCQAAIECDBAAAEKBBAgAACNAgAQAABGiQAAAAAjRI\nAAAAARokAACAwKnkIPWRXIem93Md0pHO4bBEX0IayQNSGS99r7M0ikIfezrTWUVl4WeIFGOdTbO6\n6udRmJlNEp21UYr7uljqfKd7nz+Q9Wkxl/UHW1tu7cLFTTn2pZdelPXYfSlLP1tlUeu5sqh03UQW\njplZVfmZM4dHOkMpyVdkfTqNvCetn4+Sjyp97JT/o04G/Xya4dmziPJI1lAeuf9F6We8ZIkeW+Z6\nnSlLf40yMzORN9Sbfl+yTGfU9ZF7fnLsz9ui0NfddLGsIv3M+t4/dhrJOSoK/bx7MZfMzLLc/0Lt\nM33sYaK/s3YiOUkf/fUvbu3VV16VY6tKrzO96APMzHZ2/eyqR7uP5Nihi4TYOVj5AAAAAjRIAAAA\nARokAACAAA0SAABAgAYJAAAgQIMEAAAQoEECAAAInEoOkmWRLI4VP7umrnUWxijVlzCO9IBt62d1\n5IXOCLFI1lAWycMwkXdRVzqnY77wM3XMzKYiY8nMbCzqZSSXZZzre34cyY86WfiZPLsHu3Ls/Qf3\nZX2y4mfCmJn14pEcHfnnZWZ2dKyfySjXc23Z+Dkg84OFHJsUOlNrsjqV9Vnp10aR93NtfU3W33tf\nlr8QioleCxLxLneNDoLL9OtiaSRHLhHPr8j1OlBG1rg0cvBErHFVfSjHziPZNUWhs78G8TIvI5lm\nVX0s63WtM3sG8zN7RpHoqLzX79sw6Hted/657S/25dhEZCiZmT0+eCzrG0/9Z7b14W059nBnR9Yv\nXzwn6+XUX8Q+27knxxbP2OrwCxIAAECABgkAACBAgwQAABCgQQIAAAjQIAEAAARokAAAAAI0SAAA\nAIFTyUFaVjpTohFZR7EcpDT9/3q8Snz+LJ3JseOxzhCZFPr2qrihJpKDlGc6z2Jl5mdLmZll5meI\nxG7pyiyWT6I/oBLzIRvpe7a9r/Of5lsPZP1YZBkNnb6nTdXK+qDjo+xw6ecsPdzR+SVVp+9LMdH5\nT5Oxf22zqc5Qun7juqw/D/JINlhd+etIEnmfkkjOUZ7pYxciyyiNZFzFjj1Est56kQe0rPS7ur//\nRNYnEz0v+8E/t8gtMzMdPtUP+l03cd1FoQ/eDvqZdJ0+t673M54OFgdybB7JDexT/X37pz9/6Nba\npzrLbXND5xxNzujvrLLwJ+vu0VM59sxYzyUPvyABAAAEaJAAAAACNEgAAAABGiQAAIAADRIAAECA\nBgkAACBwKtv828bflmhmluf+NtUkskc2TfU+1VGuL3HS6a2FymLhb9s2M1se662iY3Fu4yLXB49s\nh9871Fts1W1TEQBmZk/39Jb0k8rfAmtmluf+tW2un5FjY/clH+vnmbb+Ftw6ct5WlrI8RLYO56tj\nt7ZZrMmxJ0t9bolF9jVn/viTSCzD7a2H+rOfA0mv7//Q+fUh8j5lYv0zMxuP/XljZjYu1fqp18cy\nMqfbXs/pTGSC5Lk+dtvp+Jeqjq39qh7JL4hIIvEGudjKP+r0eZem17Aucs/73j+3qtXxMH1kPkSm\nml264K/P1SQyOHJdQ6/nQ9b5fcRiV2/zL6ex2Ib/jV+QAAAAAjRIAAAAARokAACAAA0SAABAgAYJ\nAAAgQIMEAAAQoEECAAAInEoO0nKp8w3GpZ/NkGa6h2tbna3QNLWs9yKbIZbftGz0dfWR7JS1yapb\nq/RHWzvovItIfJR1g39uWSTPp4xkT3WRZ7Yqxi+W+roG/Tgtj+R8jMS5LVKdfdKJrBszs8htsyHx\nX7cu1TkdmY7KsUnkD/KJn73SdfrE6+rZMkS+SMqRzplqx/79zyKZZasrK7I+EblhZjrxJ7Z+jiLn\n1kbex0R8fjaKZdTpemy8ysCLrX9p5A8ikT2WDf6xi8jzSjO9zoiPNjOzpvFPbjC9RnWDPralevz0\nrJ8zN57oTK35XOfntbXO7juXnndrpciGMjNL6sgXh4NfkAAAAAI0SAAAAAEaJAAAgAANEgAAQIAG\nCQAAIECDBAAAEKBBAgAACCTDEAtGAAAAeL7wCxIAAECABgkAACBAgwQAABCgQQIAAAjQIAEAAARo\nkAAAAAI0SAAAAAEaJAAAgAANEgAAQIAGCQAAIECDBAAAEKBBAgAACNAgAQAABGiQAAAAAjRIAAAA\nARokAACAAA0SAABAgAYJAAAgQIMEAAAQoEECAAAI0CABAAAEaJAAAAACNEgAAACB/wKrSHgyU9ns\ncgAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f47e7194240>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s9ei-KX1p_B9",
        "colab_type": "text"
      },
      "source": [
        "# Building the model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ldpPL_D3alSR",
        "colab_type": "text"
      },
      "source": [
        "Now, we'll setup a simple model. You don't need to understand everything going on here. We are creating a deep neural network using  [convolutions](http://timdettmers.com/2015/03/26/convolution-deep-learning/), [dropout](https://medium.com/@amarbudhiraja/https-medium-com-amarbudhiraja-learning-less-to-learn-better-dropout-in-deep-machine-learning-74334da4bfc5), and [max pooling](https://www.quora.com/What-is-max-pooling-in-convolutional-neural-networks)\n",
        "\n",
        "At the end, we'll flatten the network and use [Relu](https://en.wikipedia.org/wiki/Rectifier_(neural_networks), followed by a [Softmax](https://en.wikipedia.org/wiki/Softmax_function). \n",
        "\n",
        "\n",
        "This will give us a vector (1-dimenstion matrix), filled with mostly 0's.\n",
        "\n",
        "It will look like this.\n",
        "\n",
        "```\n",
        "[0,0,0,0,0,0,1,0,0,0]\n",
        "```\n",
        "\n",
        "This vector corrsponds to the given label from the image\n",
        "So in this example, the `1` in the seventh place would be a frog, since 'frog' is at the 7th place in `class_names` list.\n",
        "\n",
        "---\n",
        "\n",
        "The following shows the entire network."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mGv6JeG7YmZL",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model = tf.keras.Sequential()\n",
        "\n",
        "model.add(tf.keras.layers.Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=(32, 32, 3)))\n",
        "model.add(tf.keras.layers.Conv2D(64, kernel_size=(3, 3), activation='relu'))\n",
        "model.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2)))\n",
        "model.add(tf.keras.layers.Dropout(0.25))\n",
        "\n",
        "model.add(tf.keras.layers.Conv2D(128, kernel_size=(3, 3), activation='relu'))\n",
        "model.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2)))\n",
        "model.add(tf.keras.layers.Conv2D(128, kernel_size=(3, 3), activation='relu'))\n",
        "model.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2)))\n",
        "model.add(tf.keras.layers.Dropout(0.25))\n",
        "\n",
        "model.add(tf.keras.layers.Flatten())\n",
        "model.add(tf.keras.layers.Dense(1024, activation='relu'))\n",
        "model.add(tf.keras.layers.Dropout(0.5))\n",
        "model.add(tf.keras.layers.Dense(10, activation='softmax'))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Fi-HoS6X3Baa",
        "colab_type": "text"
      },
      "source": [
        "That's it! "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WBR89DlwZURX",
        "colab_type": "text"
      },
      "source": [
        "# Training The Model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Usmc-ST7nX0D",
        "colab_type": "text"
      },
      "source": [
        "First we compile the model to get its loss.\n",
        "The loss is a measure of how well the model did during testing. A high loss, means that the model did poorly. \n",
        "\n",
        "Here you use [Adam Optimizer](https://keras.io/optimizers/), an algorithm extending to a stochastic gradient descent widely used for machine learning, to calculate the loss.\n",
        "\n",
        "\n",
        "Then we'll call `.fit` which will train the model for 100 epochs. This means that the full training dataset will trained 100 times,\n",
        "The `batch_size` of 32 is the number samples that going to be propagated through the network.\n",
        "\n",
        "We then see how well it did after every epoch using `model.evaluate`\n",
        "It gives us a score for the model (higher numbers are better) and the loss (lower numbers are better).\n",
        "\n",
        "\n",
        "Note, this took about 15 minutes  running on Colab. If you just wants to see the results quicker, just set the `epochs` parameter to 1 or 2. Its accuracy wont be as good, however.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8V6jjkCyaxQJ",
        "colab_type": "code",
        "outputId": "fef46107-872e-4278-de8f-1f9c3bb93aed",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 3505
        }
      },
      "source": [
        "# Compile the model\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=tf.keras.optimizers.Adam(lr=0.0001, decay=1e-6),\n",
        "              metrics=['accuracy'])\n",
        "\n",
        "# Train the model\n",
        "model.fit(x_train / 255.0, tf.keras.utils.to_categorical(y_train),\n",
        "          batch_size=batch_size,\n",
        "          shuffle=True,\n",
        "          epochs=epochs,\n",
        "          validation_data=(x_test / 255.0, tf.keras.utils.to_categorical(y_test))\n",
        "          )\n",
        "\n",
        "# Evaluate the model\n",
        "scores = model.evaluate(x_test / 255.0, tf.keras.utils.to_categorical(y_test))\n",
        "\n",
        "print('Loss: %.3f' % scores[0])\n",
        "print('Accuracy: %.3f' % scores[1])"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train on 50000 samples, validate on 10000 samples\n",
            "Epoch 1/100\n",
            "50000/50000 [==============================] - 32s 641us/step - loss: 1.6316 - acc: 0.3976 - val_loss: 1.4677 - val_acc: 0.4692\n",
            "Epoch 2/100\n",
            "50000/50000 [==============================] - 32s 632us/step - loss: 1.4277 - acc: 0.4804 - val_loss: 1.2824 - val_acc: 0.5430\n",
            "Epoch 3/100\n",
            "50000/50000 [==============================] - 31s 619us/step - loss: 1.2916 - acc: 0.5360 - val_loss: 1.1822 - val_acc: 0.5742\n",
            "Epoch 4/100\n",
            "50000/50000 [==============================] - 31s 620us/step - loss: 1.1982 - acc: 0.5716 - val_loss: 1.0952 - val_acc: 0.6105\n",
            "Epoch 5/100\n",
            "50000/50000 [==============================] - 31s 619us/step - loss: 1.1240 - acc: 0.6007 - val_loss: 1.0197 - val_acc: 0.6409\n",
            "Epoch 6/100\n",
            "50000/50000 [==============================] - 31s 621us/step - loss: 1.0665 - acc: 0.6242 - val_loss: 0.9897 - val_acc: 0.6535\n",
            "Epoch 7/100\n",
            "50000/50000 [==============================] - 31s 620us/step - loss: 1.0082 - acc: 0.6477 - val_loss: 0.9433 - val_acc: 0.6662\n",
            "Epoch 8/100\n",
            "50000/50000 [==============================] - 31s 617us/step - loss: 0.9656 - acc: 0.6605 - val_loss: 0.8940 - val_acc: 0.6801\n",
            "Epoch 9/100\n",
            "50000/50000 [==============================] - 31s 617us/step - loss: 0.9181 - acc: 0.6779 - val_loss: 0.8740 - val_acc: 0.6936\n",
            "Epoch 10/100\n",
            "50000/50000 [==============================] - 31s 618us/step - loss: 0.8834 - acc: 0.6910 - val_loss: 0.8412 - val_acc: 0.7033\n",
            "Epoch 11/100\n",
            "50000/50000 [==============================] - 31s 618us/step - loss: 0.8501 - acc: 0.7038 - val_loss: 0.8080 - val_acc: 0.7217\n",
            "Epoch 12/100\n",
            "50000/50000 [==============================] - 31s 618us/step - loss: 0.8191 - acc: 0.7129 - val_loss: 0.8048 - val_acc: 0.7182\n",
            "Epoch 13/100\n",
            "50000/50000 [==============================] - 31s 620us/step - loss: 0.7934 - acc: 0.7236 - val_loss: 0.7577 - val_acc: 0.7328\n",
            "Epoch 14/100\n",
            "50000/50000 [==============================] - 31s 620us/step - loss: 0.7647 - acc: 0.7327 - val_loss: 0.7651 - val_acc: 0.7302\n",
            "Epoch 15/100\n",
            "50000/50000 [==============================] - 31s 616us/step - loss: 0.7389 - acc: 0.7412 - val_loss: 0.7262 - val_acc: 0.7489\n",
            "Epoch 16/100\n",
            "50000/50000 [==============================] - 31s 618us/step - loss: 0.7144 - acc: 0.7485 - val_loss: 0.7169 - val_acc: 0.7538\n",
            "Epoch 17/100\n",
            "50000/50000 [==============================] - 31s 618us/step - loss: 0.6947 - acc: 0.7556 - val_loss: 0.7121 - val_acc: 0.7544\n",
            "Epoch 18/100\n",
            "50000/50000 [==============================] - 31s 619us/step - loss: 0.6688 - acc: 0.7661 - val_loss: 0.6829 - val_acc: 0.7591\n",
            "Epoch 19/100\n",
            "50000/50000 [==============================] - 31s 619us/step - loss: 0.6485 - acc: 0.7732 - val_loss: 0.6828 - val_acc: 0.7628\n",
            "Epoch 20/100\n",
            "50000/50000 [==============================] - 31s 618us/step - loss: 0.6322 - acc: 0.7803 - val_loss: 0.6561 - val_acc: 0.7716\n",
            "Epoch 21/100\n",
            "50000/50000 [==============================] - 31s 621us/step - loss: 0.6112 - acc: 0.7868 - val_loss: 0.6497 - val_acc: 0.7760\n",
            "Epoch 22/100\n",
            "50000/50000 [==============================] - 31s 624us/step - loss: 0.5918 - acc: 0.7931 - val_loss: 0.6435 - val_acc: 0.7774\n",
            "Epoch 23/100\n",
            "50000/50000 [==============================] - 31s 630us/step - loss: 0.5793 - acc: 0.7972 - val_loss: 0.6343 - val_acc: 0.7797\n",
            "Epoch 24/100\n",
            "50000/50000 [==============================] - 31s 623us/step - loss: 0.5669 - acc: 0.8033 - val_loss: 0.6323 - val_acc: 0.7813\n",
            "Epoch 25/100\n",
            "50000/50000 [==============================] - 32s 633us/step - loss: 0.5476 - acc: 0.8088 - val_loss: 0.6124 - val_acc: 0.7866\n",
            "Epoch 26/100\n",
            "50000/50000 [==============================] - 32s 637us/step - loss: 0.5373 - acc: 0.8123 - val_loss: 0.6451 - val_acc: 0.7791\n",
            "Epoch 27/100\n",
            "50000/50000 [==============================] - 32s 636us/step - loss: 0.5233 - acc: 0.8159 - val_loss: 0.6052 - val_acc: 0.7929\n",
            "Epoch 28/100\n",
            "50000/50000 [==============================] - 32s 635us/step - loss: 0.5082 - acc: 0.8225 - val_loss: 0.6162 - val_acc: 0.7913\n",
            "Epoch 29/100\n",
            "50000/50000 [==============================] - 32s 637us/step - loss: 0.4948 - acc: 0.8244 - val_loss: 0.5903 - val_acc: 0.7959\n",
            "Epoch 30/100\n",
            "50000/50000 [==============================] - 32s 636us/step - loss: 0.4823 - acc: 0.8305 - val_loss: 0.6053 - val_acc: 0.7916\n",
            "Epoch 31/100\n",
            "50000/50000 [==============================] - 32s 635us/step - loss: 0.4713 - acc: 0.8336 - val_loss: 0.6254 - val_acc: 0.7893\n",
            "Epoch 32/100\n",
            "50000/50000 [==============================] - 32s 636us/step - loss: 0.4595 - acc: 0.8399 - val_loss: 0.5850 - val_acc: 0.7977\n",
            "Epoch 33/100\n",
            "50000/50000 [==============================] - 32s 637us/step - loss: 0.4463 - acc: 0.8441 - val_loss: 0.5986 - val_acc: 0.7964\n",
            "Epoch 34/100\n",
            "50000/50000 [==============================] - 32s 646us/step - loss: 0.4356 - acc: 0.8459 - val_loss: 0.6012 - val_acc: 0.7979\n",
            "Epoch 35/100\n",
            "50000/50000 [==============================] - 31s 621us/step - loss: 0.4279 - acc: 0.8495 - val_loss: 0.5935 - val_acc: 0.7971\n",
            "Epoch 36/100\n",
            "50000/50000 [==============================] - 32s 634us/step - loss: 0.4172 - acc: 0.8524 - val_loss: 0.5789 - val_acc: 0.8014\n",
            "Epoch 37/100\n",
            "50000/50000 [==============================] - 31s 627us/step - loss: 0.4026 - acc: 0.8565 - val_loss: 0.5828 - val_acc: 0.8062\n",
            "Epoch 38/100\n",
            "50000/50000 [==============================] - 31s 623us/step - loss: 0.3987 - acc: 0.8599 - val_loss: 0.5774 - val_acc: 0.8079\n",
            "Epoch 39/100\n",
            "50000/50000 [==============================] - 31s 625us/step - loss: 0.3893 - acc: 0.8620 - val_loss: 0.5821 - val_acc: 0.8027\n",
            "Epoch 40/100\n",
            "50000/50000 [==============================] - 31s 617us/step - loss: 0.3786 - acc: 0.8656 - val_loss: 0.5906 - val_acc: 0.8018\n",
            "Epoch 41/100\n",
            "50000/50000 [==============================] - 31s 622us/step - loss: 0.3675 - acc: 0.8697 - val_loss: 0.5928 - val_acc: 0.8045\n",
            "Epoch 42/100\n",
            "50000/50000 [==============================] - 31s 625us/step - loss: 0.3610 - acc: 0.8729 - val_loss: 0.5804 - val_acc: 0.8049\n",
            "Epoch 43/100\n",
            "50000/50000 [==============================] - 31s 623us/step - loss: 0.3519 - acc: 0.8746 - val_loss: 0.5728 - val_acc: 0.8084\n",
            "Epoch 44/100\n",
            "50000/50000 [==============================] - 31s 618us/step - loss: 0.3460 - acc: 0.8755 - val_loss: 0.5778 - val_acc: 0.8095\n",
            "Epoch 45/100\n",
            "50000/50000 [==============================] - 31s 623us/step - loss: 0.3337 - acc: 0.8825 - val_loss: 0.5768 - val_acc: 0.8061\n",
            "Epoch 46/100\n",
            "50000/50000 [==============================] - 31s 621us/step - loss: 0.3298 - acc: 0.8828 - val_loss: 0.5888 - val_acc: 0.8070\n",
            "Epoch 47/100\n",
            "50000/50000 [==============================] - 31s 611us/step - loss: 0.3220 - acc: 0.8845 - val_loss: 0.5856 - val_acc: 0.8094\n",
            "Epoch 48/100\n",
            "50000/50000 [==============================] - 31s 611us/step - loss: 0.3156 - acc: 0.8874 - val_loss: 0.6057 - val_acc: 0.8045\n",
            "Epoch 49/100\n",
            "50000/50000 [==============================] - 30s 609us/step - loss: 0.3064 - acc: 0.8905 - val_loss: 0.5850 - val_acc: 0.8098\n",
            "Epoch 50/100\n",
            "50000/50000 [==============================] - 30s 608us/step - loss: 0.2987 - acc: 0.8938 - val_loss: 0.5837 - val_acc: 0.8121\n",
            "Epoch 51/100\n",
            "50000/50000 [==============================] - 31s 611us/step - loss: 0.2960 - acc: 0.8943 - val_loss: 0.5770 - val_acc: 0.8124\n",
            "Epoch 52/100\n",
            "50000/50000 [==============================] - 31s 614us/step - loss: 0.2883 - acc: 0.8972 - val_loss: 0.5878 - val_acc: 0.8114\n",
            "Epoch 53/100\n",
            "50000/50000 [==============================] - 30s 609us/step - loss: 0.2852 - acc: 0.8979 - val_loss: 0.5914 - val_acc: 0.8119\n",
            "Epoch 54/100\n",
            "50000/50000 [==============================] - 30s 606us/step - loss: 0.2769 - acc: 0.9020 - val_loss: 0.5896 - val_acc: 0.8125\n",
            "Epoch 55/100\n",
            "50000/50000 [==============================] - 30s 608us/step - loss: 0.2718 - acc: 0.9024 - val_loss: 0.5985 - val_acc: 0.8092\n",
            "Epoch 56/100\n",
            "50000/50000 [==============================] - 30s 605us/step - loss: 0.2650 - acc: 0.9059 - val_loss: 0.5931 - val_acc: 0.8142\n",
            "Epoch 57/100\n",
            "50000/50000 [==============================] - 30s 602us/step - loss: 0.2627 - acc: 0.9063 - val_loss: 0.6015 - val_acc: 0.8119\n",
            "Epoch 58/100\n",
            "50000/50000 [==============================] - 30s 604us/step - loss: 0.2561 - acc: 0.9081 - val_loss: 0.5958 - val_acc: 0.8108\n",
            "Epoch 59/100\n",
            "50000/50000 [==============================] - 30s 604us/step - loss: 0.2523 - acc: 0.9098 - val_loss: 0.5948 - val_acc: 0.8131\n",
            "Epoch 60/100\n",
            "50000/50000 [==============================] - 31s 611us/step - loss: 0.2494 - acc: 0.9105 - val_loss: 0.5940 - val_acc: 0.8126\n",
            "Epoch 61/100\n",
            "50000/50000 [==============================] - 30s 603us/step - loss: 0.2401 - acc: 0.9140 - val_loss: 0.6047 - val_acc: 0.8109\n",
            "Epoch 62/100\n",
            "50000/50000 [==============================] - 30s 606us/step - loss: 0.2356 - acc: 0.9144 - val_loss: 0.6181 - val_acc: 0.8100\n",
            "Epoch 63/100\n",
            "50000/50000 [==============================] - 31s 611us/step - loss: 0.2282 - acc: 0.9186 - val_loss: 0.6155 - val_acc: 0.8145\n",
            "Epoch 64/100\n",
            "50000/50000 [==============================] - 30s 609us/step - loss: 0.2278 - acc: 0.9176 - val_loss: 0.6290 - val_acc: 0.8106\n",
            "Epoch 65/100\n",
            "50000/50000 [==============================] - 30s 610us/step - loss: 0.2225 - acc: 0.9203 - val_loss: 0.6242 - val_acc: 0.8111\n",
            "Epoch 66/100\n",
            "50000/50000 [==============================] - 31s 610us/step - loss: 0.2189 - acc: 0.9211 - val_loss: 0.6167 - val_acc: 0.8165\n",
            "Epoch 67/100\n",
            "50000/50000 [==============================] - 31s 616us/step - loss: 0.2162 - acc: 0.9236 - val_loss: 0.6090 - val_acc: 0.8155\n",
            "Epoch 68/100\n",
            "50000/50000 [==============================] - 31s 617us/step - loss: 0.2088 - acc: 0.9251 - val_loss: 0.6200 - val_acc: 0.8126\n",
            "Epoch 69/100\n",
            "50000/50000 [==============================] - 31s 617us/step - loss: 0.2112 - acc: 0.9240 - val_loss: 0.6220 - val_acc: 0.8142\n",
            "Epoch 70/100\n",
            "50000/50000 [==============================] - 30s 606us/step - loss: 0.2078 - acc: 0.9248 - val_loss: 0.6490 - val_acc: 0.8149\n",
            "Epoch 71/100\n",
            "50000/50000 [==============================] - 30s 608us/step - loss: 0.2005 - acc: 0.9288 - val_loss: 0.6174 - val_acc: 0.8149\n",
            "Epoch 72/100\n",
            "50000/50000 [==============================] - 30s 605us/step - loss: 0.1976 - acc: 0.9292 - val_loss: 0.6086 - val_acc: 0.8179\n",
            "Epoch 73/100\n",
            "50000/50000 [==============================] - 30s 609us/step - loss: 0.1983 - acc: 0.9287 - val_loss: 0.6373 - val_acc: 0.8134\n",
            "Epoch 74/100\n",
            "50000/50000 [==============================] - 30s 606us/step - loss: 0.1916 - acc: 0.9311 - val_loss: 0.6311 - val_acc: 0.8125\n",
            "Epoch 75/100\n",
            "50000/50000 [==============================] - 30s 604us/step - loss: 0.1860 - acc: 0.9351 - val_loss: 0.6386 - val_acc: 0.8186\n",
            "Epoch 76/100\n",
            "50000/50000 [==============================] - 30s 606us/step - loss: 0.1847 - acc: 0.9347 - val_loss: 0.6356 - val_acc: 0.8149\n",
            "Epoch 77/100\n",
            "50000/50000 [==============================] - 30s 607us/step - loss: 0.1812 - acc: 0.9353 - val_loss: 0.6478 - val_acc: 0.8135\n",
            "Epoch 78/100\n",
            "50000/50000 [==============================] - 30s 607us/step - loss: 0.1815 - acc: 0.9356 - val_loss: 0.6439 - val_acc: 0.8123\n",
            "Epoch 79/100\n",
            "50000/50000 [==============================] - 30s 610us/step - loss: 0.1814 - acc: 0.9353 - val_loss: 0.6480 - val_acc: 0.8143\n",
            "Epoch 80/100\n",
            "50000/50000 [==============================] - 30s 606us/step - loss: 0.1739 - acc: 0.9385 - val_loss: 0.6543 - val_acc: 0.8188\n",
            "Epoch 81/100\n",
            "50000/50000 [==============================] - 30s 605us/step - loss: 0.1711 - acc: 0.9402 - val_loss: 0.6456 - val_acc: 0.8149\n",
            "Epoch 82/100\n",
            "50000/50000 [==============================] - 30s 609us/step - loss: 0.1709 - acc: 0.9400 - val_loss: 0.6496 - val_acc: 0.8139\n",
            "Epoch 83/100\n",
            "50000/50000 [==============================] - 30s 607us/step - loss: 0.1655 - acc: 0.9401 - val_loss: 0.6647 - val_acc: 0.8162\n",
            "Epoch 84/100\n",
            "50000/50000 [==============================] - 30s 608us/step - loss: 0.1643 - acc: 0.9407 - val_loss: 0.6673 - val_acc: 0.8138\n",
            "Epoch 85/100\n",
            "50000/50000 [==============================] - 30s 607us/step - loss: 0.1616 - acc: 0.9412 - val_loss: 0.6435 - val_acc: 0.8162\n",
            "Epoch 86/100\n",
            "50000/50000 [==============================] - 30s 605us/step - loss: 0.1604 - acc: 0.9441 - val_loss: 0.6633 - val_acc: 0.8187\n",
            "Epoch 87/100\n",
            "50000/50000 [==============================] - 30s 603us/step - loss: 0.1583 - acc: 0.9426 - val_loss: 0.6902 - val_acc: 0.8138\n",
            "Epoch 88/100\n",
            "50000/50000 [==============================] - 30s 596us/step - loss: 0.1564 - acc: 0.9427 - val_loss: 0.6433 - val_acc: 0.8180\n",
            "Epoch 89/100\n",
            "50000/50000 [==============================] - 30s 600us/step - loss: 0.1540 - acc: 0.9455 - val_loss: 0.6748 - val_acc: 0.8146\n",
            "Epoch 90/100\n",
            "50000/50000 [==============================] - 30s 600us/step - loss: 0.1538 - acc: 0.9445 - val_loss: 0.6791 - val_acc: 0.8136\n",
            "Epoch 91/100\n",
            "50000/50000 [==============================] - 30s 601us/step - loss: 0.1494 - acc: 0.9469 - val_loss: 0.6756 - val_acc: 0.8129\n",
            "Epoch 92/100\n",
            "50000/50000 [==============================] - 30s 599us/step - loss: 0.1512 - acc: 0.9471 - val_loss: 0.6760 - val_acc: 0.8170\n",
            "Epoch 93/100\n",
            "50000/50000 [==============================] - 30s 598us/step - loss: 0.1452 - acc: 0.9482 - val_loss: 0.6747 - val_acc: 0.8174\n",
            "Epoch 94/100\n",
            "50000/50000 [==============================] - 30s 600us/step - loss: 0.1495 - acc: 0.9470 - val_loss: 0.6690 - val_acc: 0.8171\n",
            "Epoch 95/100\n",
            "50000/50000 [==============================] - 30s 599us/step - loss: 0.1417 - acc: 0.9497 - val_loss: 0.6917 - val_acc: 0.8200\n",
            "Epoch 96/100\n",
            "50000/50000 [==============================] - 30s 599us/step - loss: 0.1427 - acc: 0.9491 - val_loss: 0.6778 - val_acc: 0.8156\n",
            "Epoch 97/100\n",
            "50000/50000 [==============================] - 30s 602us/step - loss: 0.1355 - acc: 0.9514 - val_loss: 0.6887 - val_acc: 0.8168\n",
            "Epoch 98/100\n",
            "50000/50000 [==============================] - 30s 605us/step - loss: 0.1364 - acc: 0.9515 - val_loss: 0.6810 - val_acc: 0.8174\n",
            "Epoch 99/100\n",
            "50000/50000 [==============================] - 30s 606us/step - loss: 0.1368 - acc: 0.9520 - val_loss: 0.6792 - val_acc: 0.8188\n",
            "Epoch 100/100\n",
            "50000/50000 [==============================] - 30s 603us/step - loss: 0.1378 - acc: 0.9518 - val_loss: 0.7136 - val_acc: 0.8116\n",
            "10000/10000 [==============================] - 2s 151us/step\n",
            "Loss: 0.714\n",
            "Accuracy: 0.812\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "30lMTf95btqY",
        "colab_type": "text"
      },
      "source": [
        "Our final accuracy was 81%, and our loss was 0.7, which is pretty good.\n",
        "\n",
        "To reiterate, accuracy is how well the model was able to classify each image, whille loss indicate how bad the model's predictions were.\n",
        "\n",
        "For more information, check out this defination of loss and accuracy on [Google\n",
        "s Machine Learning crash course](https://developers.google.com/machine-learning/crash-course/descending-into-ml/training-and-loss)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ulDRlAQQ4MKN",
        "colab_type": "text"
      },
      "source": [
        "# Coverting the model to CoreML"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WX6LUzj2KpWK",
        "colab_type": "text"
      },
      "source": [
        "After we have trained the model, we can save it, then convert into the coreML format.\n",
        "\n",
        "Released at WWDC 2018 Core ML enables iOS developers to integrate a broad variety of machine learning model types into an iOS app. Here you use this technology with Nexmo In-App Messaging to facilitate your own deep learning for processing images."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v7b64a7l2W1R",
        "colab_type": "text"
      },
      "source": [
        "First, we need to save the trained model.\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fxX-HW8OKu3q",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model.save('cifar-model.h5')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-H2PJ_FDK1SG",
        "colab_type": "text"
      },
      "source": [
        "We'll use coremltools, which will convert the model into a format that our Stitch app can use. \n",
        "\n",
        "---\n",
        "\n",
        "Note, the CoreML package isnt preinstalled on Collab, so we need to install it using `pip`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "x8aN28emb73Q",
        "colab_type": "code",
        "outputId": "f8be9f64-a5c0-4aa7-ba65-10c5a15db2a5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 284
        }
      },
      "source": [
        "!pip install coremltools"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting coremltools\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/8f/ab/b4dea5ab2503f3e601052958985153cd41bd4f9a336fb74f6789151d976e/coremltools-0.8-py3.5-none-manylinux1_x86_64.whl (2.5MB)\n",
            "\u001b[K    100% |████████████████████████████████| 2.5MB 9.9MB/s \n",
            "\u001b[?25hRequirement already satisfied: protobuf>=3.1.0 in /usr/local/lib/python3.6/dist-packages (from coremltools) (3.6.0)\n",
            "Collecting six==1.10.0 (from coremltools)\n",
            "  Downloading https://files.pythonhosted.org/packages/c8/0a/b6723e1bc4c516cb687841499455a8505b44607ab535be01091c0f24f079/six-1.10.0-py2.py3-none-any.whl\n",
            "Requirement already satisfied: numpy>=1.10.0 in /usr/local/lib/python3.6/dist-packages (from coremltools) (1.14.5)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.6/dist-packages (from protobuf>=3.1.0->coremltools) (39.1.0)\n",
            "Installing collected packages: six, coremltools\n",
            "  Found existing installation: six 1.11.0\n",
            "    Uninstalling six-1.11.0:\n",
            "      Successfully uninstalled six-1.11.0\n",
            "Successfully installed coremltools-0.8 six-1.10.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "twPDKR6gcilU",
        "colab_type": "text"
      },
      "source": [
        "From above, you can see that the package was installed in our notebook.\n",
        "\n",
        "Next, we'll convert the saved model into CoreML. \n",
        "\n",
        "Since we have used Keras to train our model, Its really easy to convert to CoreML. However, this varies based on how you built your model. CoreML tools has other functions to use for other machine learning packages including Tensorflow and Scikit Learn.  See the [coremltools repo](https://github.com/apple/coremltools) for more info."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2W71YwoxOts5",
        "colab_type": "code",
        "outputId": "e7e9bd88-cf92-4e21-a43a-f428ceaf5562",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 306
        }
      },
      "source": [
        "from keras.models import load_model  \n",
        "import coremltools\n",
        "\n",
        "model = load_model('cifar-model.h5')\n",
        "coreml_model = coremltools.converters.keras.convert(model,\n",
        "\tinput_names=\"image\",\n",
        "\timage_input_names=\"image\",\n",
        "\timage_scale=1/255.0,\n",
        "\tclass_labels=class_names)\n",
        "\n",
        "coreml_model.save('CIFAR.mlmodel')"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0 : conv2d_input, <keras.engine.topology.InputLayer object at 0x7fa7c829fac8>\n",
            "1 : conv2d, <keras.layers.convolutional.Conv2D object at 0x7fa7c829f358>\n",
            "2 : conv2d__activation__, <keras.layers.core.Activation object at 0x7fa7c75bf198>\n",
            "3 : conv2d_1, <keras.layers.convolutional.Conv2D object at 0x7fa7c80e40b8>\n",
            "4 : conv2d_1__activation__, <keras.layers.core.Activation object at 0x7fa7c75bf438>\n",
            "5 : max_pooling2d, <keras.layers.pooling.MaxPooling2D object at 0x7fa7c80e4550>\n",
            "6 : conv2d_2, <keras.layers.convolutional.Conv2D object at 0x7fa7c77434a8>\n",
            "7 : conv2d_2__activation__, <keras.layers.core.Activation object at 0x7fa7c73f9240>\n",
            "8 : max_pooling2d_1, <keras.layers.pooling.MaxPooling2D object at 0x7fa7c7743f28>\n",
            "9 : conv2d_3, <keras.layers.convolutional.Conv2D object at 0x7fa7c87ad1d0>\n",
            "10 : conv2d_3__activation__, <keras.layers.core.Activation object at 0x7fa7c7262dd8>\n",
            "11 : max_pooling2d_2, <keras.layers.pooling.MaxPooling2D object at 0x7fa7c7743f60>\n",
            "12 : flatten, <keras.layers.core.Flatten object at 0x7fa7c76fac50>\n",
            "13 : dense, <keras.layers.core.Dense object at 0x7fa7c76b42b0>\n",
            "14 : dense__activation__, <keras.layers.core.Activation object at 0x7fa7c71ff390>\n",
            "15 : dense_1, <keras.layers.core.Dense object at 0x7fa7c7670358>\n",
            "16 : dense_1__activation__, <keras.layers.core.Activation object at 0x7fa7c71ff898>\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7xOpPnaNcQpX",
        "colab_type": "text"
      },
      "source": [
        "The output above shows all the layers inside the model. These directly correlate to how we created the model in this [cell](https://colab.research.google.com/drive/1KJ2PPzzbUjAo8SASpDcuiikBRNFl1VGV#scrollTo=mGv6JeG7YmZL&line=3&uniqifier=1)\n",
        "\n",
        "Take a look at the parameters for the `convert` function.\n",
        "Here, we'll set the input to be an `image` for both the `input_names`  and `image_input_names` parameters. This will help the Core ML model know what type of input it is expecting, which is an image.\n",
        "\n",
        "Then scale the images down in `image_scale` parameter to a number between 0 and 1.\n",
        "\n",
        "Next, we set the `class_labels` parameter  to `class_names` constant that we created previously.\n",
        "\n",
        "When we use this model in Xcode, the result will be a `String`, corresponding the the predicted label of the image.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HCq0x50-cwvk",
        "colab_type": "text"
      },
      "source": [
        "Now, we can have a look at the Core ML model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NeeM4z1FO2aX",
        "colab_type": "code",
        "outputId": "7c46a945-a1c8-49c8-b1e9-3144f81c7b3b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 493
        }
      },
      "source": [
        "coreml_model"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "input {\n",
              "  name: \"image\"\n",
              "  type {\n",
              "    imageType {\n",
              "      width: 32\n",
              "      height: 32\n",
              "      colorSpace: RGB\n",
              "    }\n",
              "  }\n",
              "}\n",
              "output {\n",
              "  name: \"output1\"\n",
              "  type {\n",
              "    dictionaryType {\n",
              "      stringKeyType {\n",
              "      }\n",
              "    }\n",
              "  }\n",
              "}\n",
              "output {\n",
              "  name: \"classLabel\"\n",
              "  type {\n",
              "    stringType {\n",
              "    }\n",
              "  }\n",
              "}\n",
              "predictedFeatureName: \"classLabel\"\n",
              "predictedProbabilitiesName: \"output1\""
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 84
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "24GcMs_H5nSv",
        "colab_type": "text"
      },
      "source": [
        "You can see that our `input` is a 32x32 pixel image, and our output is a String, called `classLabel`\n",
        "\n",
        "Next, we save the mlmodel locally using a Google Collab package to download the file to our machine. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "suCKlGU9c2SD",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from google.colab import files\n",
        "files.download('CIFAR.mlmodel')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TrvJwoNxz2zf",
        "colab_type": "text"
      },
      "source": [
        "# Incorporating the model into our Stitch app"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OvjRUM-Sz9QT",
        "colab_type": "text"
      },
      "source": [
        "Once our model is saved, we can now import it into our app. To do this, just drag the model that was just saved into xcode.\n",
        "Make sure the model is included in the target by verifing that  Target Membership is selected.\n",
        "![alt text](https://)\n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "Next, we'll write the code that will use the model.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xmzCaPKZDVFb",
        "colab_type": "text"
      },
      "source": [
        "In our Stitch Demo Application, users' are able to upload a photo into a existing Conversation.\n",
        "\n",
        "Nexmo's In-App Messaging enables users, who have becoming members through joining a conversation, to trigger not just `TextEvents` but `ImageEvents` by uploading a photo into an existing conversation. For this sample, we use try to predict the photo that the user uploaded.\n",
        "You integrate the functionality for observing ImageEvents for Core ML directly into your instance of UITableView, which is located [here](https://github.com/Nexmo/Stitch-Sample-App/blob/ea6e5af6dcca15a3917657079e9d67decdb904c4/Stitch-Demo/ChatTableViewController.swift#L255)\n",
        "\n",
        "\n",
        "```\n",
        "tableView(_ tableView: UITableView, cellForRowAt indexPath: IndexPath)\n",
        "\n",
        "```\n",
        "---\n",
        "\n",
        "\n",
        "Note, we instantiate the model using \n",
        " \n",
        "```\n",
        "let model = CIFAR()\n",
        "```\n",
        "\n",
        "This will go into the `ViewDidLoad` method in our class"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Zryev3Zme47A",
        "colab_type": "text"
      },
      "source": [
        "Now, lets add the code in Swift."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dVjhndIQEUri",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "let imageEvent = (event as! ImageEvent)\n",
        "guard let imagePath = imageEvent.path(of: IPS.ImageType.thumbnail), let image = UIImage(contentsOfFile: imagePath) else {\n",
        "    break\n",
        "}\n",
        "\n",
        "cell.imageView?.image = image\n",
        "\n",
        "#convert the image to a pixelBuffer\n",
        "#using https://github.com/hollance/CoreMLHelpers.git\n",
        "if let pixelBuffer = image.pixelBuffer(width: 32, height: 32) {\n",
        "    #initalize the model with the pixelBuffer\n",
        "    let input = CIFARInput(image: pixelBuffer)\n",
        "    \n",
        "    #perform the prediction\n",
        "    if let output = try? model.prediction(input: input)  {\n",
        "        #outut returns `classLabel` which is the label of the photo\n",
        "        cell.textLabel?.text = (imageEvent.from?.name)! + \" uploaded a photo of a \\(output.classLabel)\"\n",
        "    }\n",
        "    else {\n",
        "        #we could not get a prediction, just use generic text\n",
        "        cell.textLabel?.text = (imageEvent.from?.name)! + \" uploaded a photo\"\n",
        "    }\n",
        "}\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KWL94P0iEZAw",
        "colab_type": "text"
      },
      "source": [
        "Here, we check if the Conversation Event is an `ImageEvent` \n",
        "If so, we load the thumbnail and set cell's `imageView` to that image\n",
        "\n",
        "Then, we take the image, convert it to a `PixelBuffer`, at a size of 32x32 then, feed it into the model.\n",
        "The reason why we have to resample the image is because the model is trained on images of 32x32 pixels, so if we don't resize the images, the model won't be able to give a prediction (We'll see an error in xcode saying that the image size is incorrect)\n",
        "\n",
        "The model will then return a `classLabel`. This will be the name of the image that the model predicted, which could be one of the following lables.\n",
        "\n",
        "\"airplane\", \"automobile\", \"bird\", \"cat\",\" deer\", \"dog\", \"frog\", \"horse\", \"ship\" or \"truck\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J-XCbGg8Kqhe",
        "colab_type": "text"
      },
      "source": [
        "# Where do we go from here"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WWC6-742KuTv",
        "colab_type": "text"
      },
      "source": [
        "After looking at our predictions, we can tell that the model will only be able to recognize only 10 labels. The full notebook is available [here](https://github.com/nexmo-community/image-recognition-coreml-stitch/blob/master/Image_Recognition_Tensorflow_CoreML.ipynb)\n",
        "\n",
        "This is good for a demo, but not for a production example. In a future post, we'll look at building an image reconition model with more data.  We'll look into the popular [ImageNet database](http://www.image-net.org),  which contains 14,197,122 labeled images.\n",
        "Its 150gb download, so, we'll look at how to download it, train, and integrate into our Stitch demo app."
      ]
    }
  ]
}