tensorflow-mnist-cnn-tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tutorial is an introduction to Convolutional Neural Networks using TensorFlow 2.x Keras API. The dataset that we will work it is the MNIST dataset, a dataset of handwritten digits 0-9, and we will use a Sequential CNN to predict which digit was drawn.\n",
    "\n",
    "This model reaches 99.3% accuracy.\n",
    "\n",
    "To prepare our notebook, run the next cell to import the necessary packages and change the accelerator from ```None``` to ```GPU```."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.1.0\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "import seaborn as sns\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.image as mpimg\n",
    "\n",
    "print(tf.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Data Preprocessing\n",
    "\n",
    "Before building any ML model, it is important to preprocess the data. In fact, data preprocessing will generally take up the most time in any ML pipeline. The following module goes over the steps to preprocess the MNIST dataset for our purposes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.1 Load Data\n",
    "\n",
    "Our first step is to load the data and divide it into a training and testing dataset. The MNIST dataset can be downloaded directly from TensorFlow and has already been divided. Run the next cell to import the data.\n",
    "\n",
    "``` x_train ``` is the dataset of 28x28 images of handwritten digits that the model will be trained on.\n",
    "\n",
    "```y_train``` is the dataset of labels that correspond to ```x_train```. \n",
    "\n",
    "``` x_test ``` is the dataset of 28x28 images of handwritten digits that the model will be tested on.\n",
    "\n",
    "```y_test``` is the dataset of labels that correspond to ```x_test```. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz\n",
      "11493376/11490434 [==============================] - 0s 0us/step\n"
     ]
    }
   ],
   "source": [
    "mnist = tf.keras.datasets.mnist\n",
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the following code to see the counts of each digit present in our training dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7f3b53e84950>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAD6CAYAAABQ6WtbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAV30lEQVR4nO3df7DddX3n8efLRBGoGaEENubSDd1mXQO7/uBOSssMdU0pcWsN60onzigZl504TOrgbmdbaGe2tp3MsLutU7HCTEaUpP5gImqhjljZWHXbIvEGcUOILFEQYmJy1bqiu4Mmfe8f55PxmFzyvYF7vveEPB8zZ77f8z7fz/m8bybwyvf7+Z5zU1VIknQ8z5vvBiRJ48+wkCR1MiwkSZ0MC0lSJ8NCktTJsJAkdRpZWCR5aZIHhh7fT/KOJGcnuSfJI2171tCYG5LsSfJwkiuG6hcn2dleuylJRtW3JOlY6eNzFkkWAN8EfhHYAHy3qm5Mcj1wVlX9bpIVwEeAlcBLgP8B/POqOpxkO3Ad8EXgU8BNVXX38eY855xzatmyZSP7mSTpuWjHjh3frqrFR9cX9jT/KuBrVfWNJGuAV7f6ZuBzwO8Ca4Dbq+op4NEke4CVSR4DFlXVvQBJtgBXAscNi2XLljE1NTWCH0WSnruSfGOmel9rFmsZnDUAnFdV+wHa9txWXwo8MTRmb6stbftH1yVJPRl5WCR5AfB64KNdh85Qq+PUZ5prfZKpJFPT09Mn1qgk6Wn1cWbxWuD+qjrQnh9IsgSgbQ+2+l7g/KFxE8C+Vp+YoX6MqtpUVZNVNbl48TGX3CRJz1AfYfEmfnIJCuAuYF3bXwfcOVRfm+S0JBcAy4Ht7VLVk0kuaXdBXT00RpLUg5EucCc5A7gceNtQ+UZga5JrgMeBqwCqaleSrcBDwCFgQ1UdbmOuBW4DTmewsH3cxW1J0tzq5dbZ+TA5OVneDSVJJybJjqqaPLruJ7glSZ0MC0lSJ8NCktSpr09wC3j8j/5lL/P83H/Z2cs8kk4dnllIkjoZFpKkToaFJKmTYSFJ6mRYSJI6GRaSpE6GhSSpk2EhSepkWEiSOhkWkqROhoUkqZNhIUnqZFhIkjr5rbOSxsI73/nO5+RczxWeWUiSOhkWkqROhoUkqZNrFurd5y/7ld7m+pUvfL63uaTnspGeWSR5cZI7knw1ye4kv5Tk7CT3JHmkbc8aOv6GJHuSPJzkiqH6xUl2ttduSpJR9i1J+mmjvgz1buDTVfUvgJcDu4HrgW1VtRzY1p6TZAWwFrgQWA3cnGRBe59bgPXA8vZYPeK+JUlDRhYWSRYBlwG3AlTVj6rqe8AaYHM7bDNwZdtfA9xeVU9V1aPAHmBlkiXAoqq6t6oK2DI0RpLUg1GeWfw8MA18IMmXk7wvyZnAeVW1H6Btz23HLwWeGBq/t9WWtv2j65KknowyLBYCrwJuqapXAj+kXXJ6GjOtQ9Rx6se+QbI+yVSSqenp6RPtV5L0NEZ5N9ReYG9V3dee38EgLA4kWVJV+9slpoNDx58/NH4C2NfqEzPUj1FVm4BNAJOTkzMGyqnu0vdc2ttcf/f2v+ttLum55OV3/HVvc33ljVd0H8QIw6KqvpXkiSQvraqHgVXAQ+2xDrixbe9sQ+4CPpzkXcBLGCxkb6+qw0meTHIJcB9wNfCeE+nl4v+8ZU5+ptnY8d+v7m0uaa7s3vjZ3uZ62e+/pre5NHdG/TmLtwMfSvIC4OvAWxlc+tqa5BrgceAqgKralWQrgzA5BGyoqsPtfa4FbgNOB+5uD0lST0YaFlX1ADA5w0urnub4jcDGGepTwEVz251OdX/+23/V21y/9ae/0dtcena2fnRlb3P95lXbe5vr2fLrPiRJnQwLSVInw0KS1MmwkCR1MiwkSZ0MC0lSJ8NCktTJsJAkdTIsJEmdDAtJUifDQpLUybCQJHUa9bfOSjqOjW9+Y29z/f4H7+htLj33eGYhSepkWEiSOhkWkqROhoUkqZNhIUnqZFhIkjoZFpKkToaFJKmTYSFJ6jTSsEjyWJKdSR5IMtVqZye5J8kjbXvW0PE3JNmT5OEkVwzVL27vsyfJTUkyyr4lST+tjzOLf11Vr6iqyfb8emBbVS0HtrXnJFkBrAUuBFYDNydZ0MbcAqwHlrfH6h76liQ183EZag2wue1vBq4cqt9eVU9V1aPAHmBlkiXAoqq6t6oK2DI0RpLUg1GHRQGfSbIjyfpWO6+q9gO07bmtvhR4Ymjs3lZb2vaPrh8jyfokU0mmpqen5/DHkKRT26i/dfbSqtqX5FzgniRfPc6xM61D1HHqxxarNgGbACYnJ2c8RpJ04kZ6ZlFV+9r2IPAJYCVwoF1aom0PtsP3AucPDZ8A9rX6xAx1SVJPRhYWSc5M8qIj+8CvAQ8CdwHr2mHrgDvb/l3A2iSnJbmAwUL29nap6skkl7S7oK4eGiNJ6sEoL0OdB3yi3eW6EPhwVX06yZeArUmuAR4HrgKoql1JtgIPAYeADVV1uL3XtcBtwOnA3e0hSerJyMKiqr4OvHyG+neAVU8zZiOwcYb6FHDRXPcoSZodP8EtSepkWEiSOhkWkqROhoUkqZNhIUnqZFhIkjoZFpKkToaFJKmTYSFJ6mRYSJI6GRaSpE6GhSSpk2EhSepkWEiSOhkWkqROhoUkqZNhIUnqZFhIkjoZFpKkToaFJKmTYSFJ6jTysEiyIMmXk3yyPT87yT1JHmnbs4aOvSHJniQPJ7liqH5xkp3ttZuSZNR9S5J+oo8zi+uA3UPPrwe2VdVyYFt7TpIVwFrgQmA1cHOSBW3MLcB6YHl7rO6hb0lSM9KwSDIB/DrwvqHyGmBz298MXDlUv72qnqqqR4E9wMokS4BFVXVvVRWwZWiMJKkHoz6z+DPgd4B/HKqdV1X7Adr23FZfCjwxdNzeVlva9o+uS5J6MrKwSPI64GBV7ZjtkBlqdZz6THOuTzKVZGp6enqW00qSuozyzOJS4PVJHgNuB16T5IPAgXZpibY92I7fC5w/NH4C2NfqEzPUj1FVm6pqsqomFy9ePJc/iySd0kYWFlV1Q1VNVNUyBgvXn62qNwN3AevaYeuAO9v+XcDaJKcluYDBQvb2dqnqySSXtLugrh4aI0nqwazCIsm22dRm6Ubg8iSPAJe351TVLmAr8BDwaWBDVR1uY65lsEi+B/gacPcznFuS9AwsPN6LSV4InAGc0z4PcWT9YBHwktlOUlWfAz7X9r8DrHqa4zYCG2eoTwEXzXY+SdLcOm5YAG8D3sEgGHbwk7D4PvDeEfYlSRojxw2Lqno38O4kb6+q9/TUkyRpzHSdWQBQVe9J8svAsuExVbVlRH1JksbIrMIiyV8A/wx4ADiy6Hzk09SSpOe4WYUFMAmsaF+3IUk6xcz2cxYPAv9klI1IksbXbM8szgEeSrIdeOpIsapeP5KuJEljZbZh8c5RNiFJGm+zvRvq86NuRJI0vmZ7N9ST/OSbXl8APB/4YVUtGlVjkqTxMdszixcNP09yJbByJB1JksbOM/rW2ar6S+A1c9yLJGlMzfYy1BuGnj6Pwecu/MyFJJ0iZns31G8M7R8CHmPwO7MlSaeA2a5ZvHXUjUiSxtdsf/nRRJJPJDmY5ECSjyWZ6B4pSXoumO0C9wcY/NrTlwBLgb9qNUnSKWC2YbG4qj5QVYfa4zZg8Qj7kiSNkdmGxbeTvDnJgvZ4M/CdUTYmSRofsw2Lfw/8JvAtYD/wRsBFb0k6Rcz21tk/BtZV1T8AJDkb+BMGISJJeo6b7ZnFvzoSFABV9V3glaNpSZI0bmYbFs9LctaRJ+3M4rhnJUlemGR7kq8k2ZXkD4+MTXJPkkfadvh9b0iyJ8nDSa4Yql+cZGd77aYkObEfU5L0bMw2LP4U+Pskf5zkj4C/B/5bx5ingNdU1cuBVwCrk1wCXA9sq6rlwLb2nCQrgLXAhcBq4OYkC9p73QKsB5a3x+pZ9i1JmgOzCouq2gL8O+AAMA28oar+omNMVdUP2tPnt0cx+JqQza2+Gbiy7a8Bbq+qp6rqUWAPsDLJEmBRVd3bfgf4lqExkqQezHaBm6p6CHjoRN68nRnsAH4BeG9V3ZfkvKra395zf5Jz2+FLgS8ODd/baj9u+0fXJUk9eUZfUT5bVXW4ql4BTDA4S7joOIfPtA5Rx6kf+wbJ+iRTSaamp6dPvGFJ0oxGGhZHVNX3gM8xWGs40C4t0bYH22F7gfOHhk0A+1p9Yob6TPNsqqrJqppcvNgPmEvSXBlZWCRZnOTFbf904FeBrzL4jql17bB1wJ1t/y5gbZLTklzAYCF7e7tk9WSSS9pdUFcPjZEk9WDWaxbPwBJgc1u3eB6wtao+meReYGuSa4DHgasAqmpXkq0M1kUOARuq6nB7r2uB24DTgbvbQ5LUk5GFRVX9L2b44F5VfQdY9TRjNgIbZ6hPAcdb75AkjVAvaxaSpJObYSFJ6mRYSJI6GRaSpE6GhSSpk2EhSepkWEiSOhkWkqROhoUkqZNhIUnqZFhIkjoZFpKkToaFJKmTYSFJ6mRYSJI6GRaSpE6GhSSpk2EhSepkWEiSOhkWkqROhoUkqZNhIUnqNLKwSHJ+kr9JsjvJriTXtfrZSe5J8kjbnjU05oYke5I8nOSKofrFSXa2125KklH1LUk61ijPLA4Bv11VLwMuATYkWQFcD2yrquXAtvac9tpa4EJgNXBzkgXtvW4B1gPL22P1CPuWJB1lZGFRVfur6v62/ySwG1gKrAE2t8M2A1e2/TXA7VX1VFU9CuwBViZZAiyqqnurqoAtQ2MkST3oZc0iyTLglcB9wHlVtR8GgQKc2w5bCjwxNGxvqy1t+0fXZ5pnfZKpJFPT09Nz+SNI0ilt5GGR5GeAjwHvqKrvH+/QGWp1nPqxxapNVTVZVZOLFy8+8WYlSTMaaVgkeT6DoPhQVX28lQ+0S0u07cFW3wucPzR8AtjX6hMz1CVJPRnl3VABbgV2V9W7hl66C1jX9tcBdw7V1yY5LckFDBayt7dLVU8muaS959VDYyRJPVg4wve+FHgLsDPJA632e8CNwNYk1wCPA1cBVNWuJFuBhxjcSbWhqg63cdcCtwGnA3e3hySpJyMLi6r6W2ZebwBY9TRjNgIbZ6hPARfNXXeSpBPhJ7glSZ0MC0lSJ8NCktTJsJAkdTIsJEmdDAtJUifDQpLUybCQJHUyLCRJnQwLSVInw0KS1MmwkCR1MiwkSZ0MC0lSJ8NCktTJsJAkdTIsJEmdDAtJUifDQpLUybCQJHUyLCRJnUYWFknen+RgkgeHamcnuSfJI2171tBrNyTZk+ThJFcM1S9OsrO9dlOSjKpnSdLMRnlmcRuw+qja9cC2qloObGvPSbICWAtc2MbcnGRBG3MLsB5Y3h5Hv6ckacRGFhZV9QXgu0eV1wCb2/5m4Mqh+u1V9VRVPQrsAVYmWQIsqqp7q6qALUNjJEk96XvN4ryq2g/Qtue2+lLgiaHj9rba0rZ/dF2S1KNxWeCeaR2ijlOf+U2S9UmmkkxNT0/PWXOSdKrrOywOtEtLtO3BVt8LnD903ASwr9UnZqjPqKo2VdVkVU0uXrx4ThuXpFNZ32FxF7Cu7a8D7hyqr01yWpILGCxkb2+Xqp5Mckm7C+rqoTGSpJ4sHNUbJ/kI8GrgnCR7gT8AbgS2JrkGeBy4CqCqdiXZCjwEHAI2VNXh9lbXMriz6nTg7vaQJPVoZGFRVW96mpdWPc3xG4GNM9SngIvmsDVJ0gkalwVuSdIYMywkSZ0MC0lSJ8NCktTJsJAkdTIsJEmdDAtJUifDQpLUybCQJHUyLCRJnQwLSVInw0KS1MmwkCR1MiwkSZ0MC0lSJ8NCktTJsJAkdTIsJEmdDAtJUifDQpLUybCQJHUyLCRJnU6asEiyOsnDSfYkuX6++5GkU8lJERZJFgDvBV4LrADelGTF/HYlSaeOkyIsgJXAnqr6elX9CLgdWDPPPUnSKeNkCYulwBNDz/e2miSpB6mq+e6hU5KrgCuq6j+0528BVlbV2486bj2wvj19KfDws5j2HODbz2L8XBmHPsahBxiPPsahBxiPPsahBxiPPsahB5ibPv5pVS0+urjwWb5pX/YC5w89nwD2HX1QVW0CNs3FhEmmqmpyLt7rZO9jHHoYlz7GoYdx6WMcehiXPsahh1H3cbJchvoSsDzJBUleAKwF7prnniTplHFSnFlU1aEkvwX8NbAAeH9V7ZrntiTplHFShAVAVX0K+FSPU87J5aw5MA59jEMPMB59jEMPMB59jEMPMB59jEMPMMI+TooFbknS/DpZ1iwkSfPIsJjBOHy1SJL3JzmY5MH5mL/1cH6Sv0myO8muJNfNQw8vTLI9yVdaD3/Ydw9H9bMgyZeTfHKe5n8syc4kDySZmo8eWh8vTnJHkq+2vx+/1PP8L21/Bkce30/yjj57GOrlP7a/mw8m+UiSF85DD9e1+XeN6s/By1BHaV8t8r+Byxncsvsl4E1V9VDPfVwG/ADYUlUX9Tn3UA9LgCVVdX+SFwE7gCv7/LNIEuDMqvpBkucDfwtcV1Vf7KuHo/r5T8AksKiqXjcP8z8GTFbVvN7Tn2Qz8D+r6n3tDsUzqup789TLAuCbwC9W1Td6nnspg7+TK6rq/yXZCnyqqm7rsYeLGHyrxUrgR8CngWur6pG5nMczi2ONxVeLVNUXgO/2Pe9RPeyvqvvb/pPAbnr+5HwN/KA9fX57zMu/cJJMAL8OvG8+5h8XSRYBlwG3AlTVj+YrKJpVwNf6DoohC4HTkywEzmCGz4CN2MuAL1bV/62qQ8DngX8715MYFsfyq0VmkGQZ8ErgvnmYe0GSB4CDwD1V1XsPzZ8BvwP84zzND4Og/EySHe0bC+bDzwPTwAfaJbn3JTlznnqBweeuPjIfE1fVN4E/AR4H9gP/p6o+03MbDwKXJfnZJGcA/4af/hDznDAsjpUZaqf0tbokPwN8DHhHVX2/7/mr6nBVvYLBJ/dXttPuXiV5HXCwqnb0PfdRLq2qVzH4BuYN7XJl3xYCrwJuqapXAj8E5mtt7wXA64GPztP8ZzG48nAB8BLgzCRv7rOHqtoN/FfgHgaXoL4CHJrreQyLY83qq0VOFW2d4GPAh6rq4/PZS7vU8Tlg9TxMfynw+rZmcDvwmiQf7LuJqtrXtgeBTzC4bNq3vcDeoTO8OxiEx3x4LXB/VR2Yp/l/FXi0qqar6sfAx4Ff7ruJqrq1ql5VVZcxuHw9p+sVYFjMxK8Wadri8q3A7qp61zz1sDjJi9v+6Qz+4/xq331U1Q1VNVFVyxj8nfhsVfX6L8gkZ7YbDWiXfX6NwSWIXlXVt4Ankry0lVYBvd4AMuRNzNMlqOZx4JIkZ7T/XlYxWNvrVZJz2/bngDcwgj+Tk+YT3H0Zl68WSfIR4NXAOUn2An9QVbf23MalwFuAnW3NAOD32qfp+7IE2NzueHkesLWq5uW21TFwHvCJwf+TWAh8uKo+PU+9vB34UPsH1deBt/bdQLs+fznwtr7nPqKq7ktyB3A/g0s/X2Z+Ps39sSQ/C/wY2FBV/zDXE3jrrCSpk5ehJEmdDAtJUifDQpLUybCQJHUyLCRJnQwLSVInw0KS1MmwkCR1+v/zAoM1Q5j3/gAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.countplot(y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are similar counts for each digit. This is good as the model will have enough images for each class to train the features for each class. There is no need to downsample or upweigh."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2 Check for NaN Values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.isnan(x_train).any()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.isnan(x_test).any()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are no NaN values in our dataset. There is no need to preprocess the data to deal with Nan's."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.3 Normalization and Reshaping\n",
    "\n",
    "Since the values in our ```x_train``` dataset are 28x28 images, our input shape must be specified so that our model will know what is being inputed.\n",
    "\n",
    "The first convolution layer expects a single 60000x28x28x1 tensor instead of 60000 28x28x1 tensors.\n",
    "\n",
    "Models generally run better on normalized values. The best way to normalize the data depends on each individual dataset. For the MNIST dataset, we want each value to be between 0.0 and 1.0. As all values originally fall under the 0.0-255.0 range, divide by 255.0.\n",
    "\n",
    "Run the following cell to define the ```input_shape``` and to normalize and reshape the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "input_shape = (28, 28, 1)\n",
    "\n",
    "x_train=x_train.reshape(x_train.shape[0], x_train.shape[1], x_train.shape[2], 1)\n",
    "x_train=x_train / 255.0\n",
    "x_test = x_test.reshape(x_test.shape[0], x_test.shape[1], x_test.shape[2], 1)\n",
    "x_test=x_test/255.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.4 Label Encoding\n",
    "\n",
    "The labels for the training and the testing dataset are currently categorical and is not continuous. To include categorical dataset in our model, our labels should be converted to one-hot encodings.\n",
    "\n",
    "For example, ```2``` becomes ```[0,0,1,0,0,0,0,0,0,0]``` and ```7``` becomes ```[0,0,0,0,0,0,0,1,0,0]```.\n",
    "\n",
    "Run the following cell to transform the labels into one-hot encodings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_train = tf.one_hot(y_train.astype(np.int32), depth=10)\n",
    "y_test = tf.one_hot(y_test.astype(np.int32), depth=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.5 Visualize Data\n",
    "\n",
    "Run the following cell to visualize an image in our dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tf.Tensor([0. 0. 0. 0. 0. 1. 0. 0. 0. 0.], shape=(10,), dtype=float32)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAANNUlEQVR4nO3df4wc9XnH8c8HYx/EgOqLwXFtCzB1lFohIcnFVHEUEdEix1Fl0ipp3F9uRXOpGiSipm0obRVUVa2bFqL0h1AvxY3zC0qVAK5q0jinRISGOJyRY+zYCcY1YGzZULc1RMW+s5/+cePoMDdz553ZH+fn/ZJWszvPzs7jkT83szuz+3VECMC577xuNwCgMwg7kARhB5Ig7EAShB1I4vxOrmyO++ICze3kKoFUXtaPdCKOe7JarbDbXiXp05JmSfrHiFhf9fwLNFfX+vo6qwRQYWsMl9ZaPoy3PUvS30t6j6TlktbaXt7q6wForzrv2VdI2hsR+yLihKR7Ja1ppi0ATasT9kWSnp3w+EAx7xVsD9oesT0yquM1Vgegjjphn+xDgFddexsRQxExEBEDs9VXY3UA6qgT9gOSlkx4vFjSwXrtAGiXOmF/TNIy21faniPpg5I2NdMWgKa1fOotIsZs3yzp3zV+6m1DROxqrDMAjap1nj0iNkva3FAvANqIy2WBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSdQastn2fkkvSjopaSwiBppoCkDzaoW98O6IeKGB1wHQRhzGA0nUDXtI+prtbbYHJ3uC7UHbI7ZHRnW85uoAtKruYfzKiDho+zJJW2zviYiHJz4hIoYkDUnSJe6PmusD0KJae/aIOFhMj0i6X9KKJpoC0LyWw257ru2LT9+XdIOknU01BqBZdQ7jF0i63/bp1/lSRHy1ka7QOefNqiyfv+DSyvqJq15XWd/7K3POuqXTvvXeOyvri8+/qLL+1OhLpbU1d/1B5bKL1n+7sj4TtRz2iNgn6c0N9gKgjTj1BiRB2IEkCDuQBGEHkiDsQBJNfBEGXTbr0vLTY8/98rLKZePd/11Z3/b2L7TUUxN+OFp9WvDrxy6rrO99+erS2pKHqv/dpyqrMxN7diAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgvPs54A9f7K0tPaDX/zbDnbyartHR0trG//rHZXLbvvjt1XW+x56rKWexu2usezMxJ4dSIKwA0kQdiAJwg4kQdiBJAg7kARhB5LgPPsM8J/3vqmy/p2VVT+5fEHlsv976uXK+rv+4fcr66/9/snK+oWHy4f88n9sr1y2T3XOo+NM7NmBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnOs88Av778u5X1eedVn0uvsvPExZX1JX927g1dnNWUe3bbG2wfsb1zwrx+21tsP1lM57W3TQB1Tecw/rOSVp0x71ZJwxGxTNJw8RhAD5sy7BHxsKSjZ8xeI2ljcX+jpBsb7gtAw1r9gG5BRBySpGJaOuiW7UHbI7ZHRlV+nTSA9mr7p/ERMRQRAxExMFt97V4dgBKthv2w7YWSVEyPNNcSgHZoNeybJK0r7q+T9GAz7QBolynPs9u+R9J1kubbPiDpE5LWS7rP9k2SnpH0/nY2md0X9ry9sv7xlbtafu3fun+wsn6VvtPya6O3TBn2iFhbUrq+4V4AtBGXywJJEHYgCcIOJEHYgSQIO5AEX3GdAS78ZvXXULWyvHQ8yodMlqTFw9U/BY1zB3t2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiC8+znuJej+jx630MMi5wFe3YgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSUwZdtsbbB+xvXPCvNttP2d7e3Fb3d42AdQ1nT37ZyWtmmT+pyLimuK2udm2ADRtyrBHxMOSjnagFwBtVOc9+822dxSH+fPKnmR70PaI7ZFRHa+xOgB1tBr2uyRdJekaSYck3VH2xIgYioiBiBiYrb4WVwegrpbCHhGHI+JkRJyS9BlJK5ptC0DTWgq77YUTHr5P0s6y5wLoDVP+brzteyRdJ2m+7QOSPiHpOtvXSApJ+yV9uI09pveT//pMZf3R35tVWnvznOq/5+e96Q2V9VM79lTWMXNMGfaIWDvJ7Lvb0AuANuIKOiAJwg4kQdiBJAg7kARhB5JgyOYZYOzZA5X1/zn5mtLaa1w9ZPMfPnBvZf17/3d5ZX0qf/Nv5V+IXHbHU5XLnjx8pNa68Urs2YEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCUdEx1Z2ifvjWl/fsfVl8dJXl5bWvnn1v3Swk7Pzm09X/1945pOvr6xf+MB3m2znnLA1hnUsjnqyGnt2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiC77OfAy5a/XRp7Y1/enPlsv27qq+zeP6tk56y/bEPrfp6Zf13+8t/ivqfLh+uXPb1711WXX+gsowzsGcHkiDsQBKEHUiCsANJEHYgCcIOJEHYgST4PjtqOX/pFZX1X9r8SGlt7cWHK5f98xeurqw/+rby38uXpBgbq6yfi2p9n932EtvfsL3b9i7btxTz+21vsf1kMZ3XdOMAmjOdw/gxSR+LiJ+W9DOSPmJ7uaRbJQ1HxDJJw8VjAD1qyrBHxKGIeLy4/6Kk3ZIWSVojaWPxtI2SbmxXkwDqO6sP6GxfIektkrZKWhARh6TxPwiSLitZZtD2iO2RUR2v1y2Alk077LYvkvRlSR+NiGPTXS4ihiJiICIGZquvlR4BNGBaYbc9W+NB/2JEfKWYfdj2wqK+UBJDbgI9bMqvuNq2pLsl7Y6IOyeUNklaJ2l9MX2wLR2ip43t219Z/8uNHyitrfqdv6pc9rb5T1TWf37WOyrrSnjqrcp0vs++UtKvSXrC9vZi3m0aD/l9tm+S9Iyk97enRQBNmDLsEfGIpLJfMOAKGWCG4HJZIAnCDiRB2IEkCDuQBGEHkuCnpNFWi//i26W1f/7V5ZXL/vZP7Gu6ndTYswNJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEpxnR1vN+qkrS2tL+8qHc0bz2LMDSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKcZ0db7bll0lHBJEk3XPijymXvPPqG6hc/ebKVltJizw4kQdiBJAg7kARhB5Ig7EAShB1IgrADSUxnfPYlkj4n6XWSTkkaiohP275d0ockPV889baI2NyuRjEzzR+p2J/8QvWy9/3dz1a/9tijLXSU13QuqhmT9LGIeNz2xZK22d5S1D4VEX/dvvYANGU647MfknSouP+i7d2SFrW7MQDNOqv37LavkPQWSVuLWTfb3mF7g+15JcsM2h6xPTKq47WaBdC6aYfd9kWSvizpoxFxTNJdkq6SdI3G9/x3TLZcRAxFxEBEDMxWXwMtA2jFtMJue7bGg/7FiPiKJEXE4Yg4GRGnJH1G0or2tQmgrinDbtuS7pa0OyLunDB/4YSnvU/SzubbA9AUR0T1E+x3SvqWpCc0fupNkm6TtFbjh/Ahab+kDxcf5pW6xP1xra+v2TKAMltjWMfiqCerTefT+EckTbYw59SBGYQr6IAkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0lM+X32RldmPy/p6Qmz5kt6oWMNnJ1e7a1X+5LorVVN9nZ5RFw6WaGjYX/Vyu2RiBjoWgMVerW3Xu1LordWdao3DuOBJAg7kES3wz7U5fVX6dXeerUvid5a1ZHeuvqeHUDndHvPDqBDCDuQRFfCbnuV7R/Y3mv71m70UMb2fttP2N5ue6TLvWywfcT2zgnz+m1vsf1kMZ10jL0u9Xa77eeKbbfd9uou9bbE9jds77a9y/YtxfyubruKvjqy3Tr+nt32LEk/lPRzkg5IekzS2oj4fkcbKWF7v6SBiOj6BRi23yXpJUmfi4g3FvM+KeloRKwv/lDOi4iP90hvt0t6qdvDeBejFS2cOMy4pBsl/Ya6uO0q+vqAOrDdurFnXyFpb0Tsi4gTku6VtKYLffS8iHhY0tEzZq+RtLG4v1Hj/1k6rqS3nhARhyLi8eL+i5JODzPe1W1X0VdHdCPsiyQ9O+HxAfXWeO8h6Wu2t9ke7HYzk1hwepitYnpZl/s505TDeHfSGcOM98y2a2X487q6EfbJhpLqpfN/KyPirZLeI+kjxeEqpmdaw3h3yiTDjPeEVoc/r6sbYT8gacmEx4slHexCH5OKiIPF9Iik+9V7Q1EfPj2CbjE90uV+fqyXhvGebJhx9cC26+bw590I+2OSltm+0vYcSR+UtKkLfbyK7bnFByeyPVfSDeq9oag3SVpX3F8n6cEu9vIKvTKMd9kw4+rytuv68OcR0fGbpNUa/0T+KUl/1I0eSvpaKul7xW1Xt3uTdI/GD+tGNX5EdJOk10oalvRkMe3vod4+r/GhvXdoPFgLu9TbOzX+1nCHpO3FbXW3t11FXx3ZblwuCyTBFXRAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kMT/AzRO9ZWIKmVqAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(x_train[100][:,:,0])\n",
    "print(y_train[100])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The image is an image of a handwritten ```5```. The one-hot encoding holds the value of ```5```."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. CNN\n",
    "\n",
    "In this module, we will build our CNN model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.1 Define the Model\n",
    "\n",
    "Run the following cell to define ```batch_size```, ```num_classes```, and ```epochs```. Try changing the values and test how different values affect the accuracy of the CNN model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "batch_size = 64\n",
    "num_classes = 10\n",
    "epochs = 5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the following cell to build the model. The model contains various layers stacked on top of each other. The output of one layer feeds into the input of the next layer.\n",
    "\n",
    "Conv2D layers are convolutions. Each filter (32 in the first two convolution layers and 64 in the next two convolution layers) transforms a part of the image (5x5 for the first two Conv2D layers and 3x3 for the next two Conv2D layers). The transformation is applied on the whole image.\n",
    "\n",
    "MaxPool2D is a downsampling filter. It reduces a 2x2 matrix of the image to a single pixel with the maximum value of the 2x2 matrix. The filter aims to conserve the main features of the image while reducing the size.\n",
    "\n",
    "Dropout is a regularization layer. In our model, 25% of the nodes in the layer are randomly ignores, allowing the network to learn different features. This prevents overfitting.\n",
    "\n",
    "```relu``` is the rectifier, and it is used to find nonlinearity in the data. It works by returning the input value if the input value >= 0. If the input is negative, it returns 0.\n",
    "\n",
    "Flatten converts the tensors into a 1D vector.\n",
    "\n",
    "The Dense layers are an artificial neural network (ANN). The last layer returns the probability that an image is in each class (one for each digit).\n",
    "\n",
    "As this model aims to categorize the images, we will use a ```categorical_crossentropy``` loss function. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = tf.keras.models.Sequential([\n",
    "    tf.keras.layers.Conv2D(32, (5,5), padding='same', activation='relu', input_shape=input_shape),\n",
    "    tf.keras.layers.Conv2D(32, (5,5), padding='same', activation='relu'),\n",
    "    tf.keras.layers.MaxPool2D(),\n",
    "    tf.keras.layers.Dropout(0.25),\n",
    "    tf.keras.layers.Conv2D(64, (3,3), padding='same', activation='relu'),\n",
    "    tf.keras.layers.Conv2D(64, (3,3), padding='same', activation='relu'),\n",
    "    tf.keras.layers.MaxPool2D(strides=(2,2)),\n",
    "    tf.keras.layers.Dropout(0.25),\n",
    "    tf.keras.layers.Flatten(),\n",
    "    tf.keras.layers.Dense(128, activation='relu'),\n",
    "    tf.keras.layers.Dropout(0.5),\n",
    "    tf.keras.layers.Dense(num_classes, activation='softmax')\n",
    "])\n",
    "\n",
    "model.compile(optimizer=tf.keras.optimizers.RMSprop(epsilon=1e-08), loss='categorical_crossentropy', metrics=['acc'])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.2 Fit the Training Data\n",
    "\n",
    "The next step is to fit our training data. If we achieve a certain level of accuracy, it may not be necessary to continue training the model, especially if time and resources are limited.\n",
    "\n",
    "The following cell defines a CallBack so that if 99.5% accuracy is achieved, the model stops training. The model is not likely to stop prematurely if only 5 epochs are specified. Try it out with more epochs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "class myCallback(tf.keras.callbacks.Callback):\n",
    "  def on_epoch_end(self, epoch, logs={}):\n",
    "    if(logs.get('acc')>0.995):\n",
    "      print(\"\\nReached 99.5% accuracy so cancelling training!\")\n",
    "      self.model.stop_training = True\n",
    "\n",
    "callbacks = myCallback()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Testing the model on a validation dataset prevents overfitting of the data. We specified a 10% validation and 90% training split."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 54000 samples, validate on 6000 samples\n",
      "Epoch 1/5\n",
      "54000/54000 [==============================] - 10s 181us/sample - loss: 0.2200 - acc: 0.9317 - val_loss: 0.0437 - val_acc: 0.9872\n",
      "Epoch 2/5\n",
      "54000/54000 [==============================] - 5s 98us/sample - loss: 0.0748 - acc: 0.9782 - val_loss: 0.0313 - val_acc: 0.9915\n",
      "Epoch 3/5\n",
      "54000/54000 [==============================] - 6s 103us/sample - loss: 0.0600 - acc: 0.9828 - val_loss: 0.0294 - val_acc: 0.9917\n",
      "Epoch 4/5\n",
      "54000/54000 [==============================] - 5s 99us/sample - loss: 0.0522 - acc: 0.9854 - val_loss: 0.0321 - val_acc: 0.9922\n",
      "Epoch 5/5\n",
      "54000/54000 [==============================] - 6s 105us/sample - loss: 0.0507 - acc: 0.9862 - val_loss: 0.0454 - val_acc: 0.9903\n"
     ]
    }
   ],
   "source": [
    "history = model.fit(x_train, y_train,\n",
    "                    batch_size=batch_size,\n",
    "                    epochs=epochs,\n",
    "                    validation_split=0.1,\n",
    "                    callbacks=[callbacks])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. Evaluate the Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.1 Loss and Accuracy Curves\n",
    "\n",
    "Run the following cell to evaluate the loss and accuracy of our model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXgUVbr48e+bBAIhyBZ20AAiSICEEAMCQnCZQWEEWQZQ1MgMKrMooOPCuKBert4rd1TmQf2hgwgyIC4gMggKsqiMSgJECYKyZBSRVQnBBMlyfn+cTmhCd9JJOqnuzvt5nnrS3VVd9faheev0qXNOiTEGpZRSoSvM6QCUUkpVL030SikV4jTRK6VUiNNEr5RSIU4TvVJKhbgIpwPwJCYmxsTGxjodhlJKBY309PRjxpjmntYFZKKPjY0lLS3N6TCUUipoiMh/vK3TphullApxIZXof/7Z6QiUUirwhEyiP3ECkpPhr38FHeyrlFJnBWQbfWVER8OAAfDf/w0HD8LcuVCnjtNRKRXYzpw5w969e8nNzXU6FOWjqKgoOnXqRN26dX1+T8gk+ogIePFFaN0aHnsMjhyBpUuhQQOnI1MqcO3du5fGjRvTpUsXwsJC5gd+yCoqKuLw4cN88803XHrppT7/m4XUv6wIzJhhE/7q1XDllXDsmNNRKRW4cnNzadmypSb5IBEWFkbLli3Jy8tjyZIl/OzjhcmQ/Ne94w546y344gvo3x/273c6IqUClyb54BIWFoaIcPz4cdavX+/be6o5JseMGAEffGCbcPr1g+3bnY5IKaX8Jzo6mqNHj/q0bcgmerAXZz/+2LbfDxwIH37odERKKXfHjx8nISGBhIQEWrVqRdu2bUuenzlzpsz3pqWlcdddd5V7jH79+vkl1g0bNjBs2DC/7MsfRARf7ydSpYuxIjIEeA4IB142xjxVav1NwP2up6eAycaYjKocs6Li4mDzZhgyBK69FhYuhN/+tiYjUEp506xZM7a7fm7PmDGD6Oho7r333pL1BQUFRER4TlNJSUkkJSWVe4zNmzf7J9ggVukavYiEA3OAa4FuwHgR6VZqs/3AIGNMT+AJYG5lj1cV7dvbmn1yMowbB7NnOxGFUsoXqampTJs2jcGDB3P//ffz+eef069fP3r16kW/fv3YvXs3cG4Ne8aMGUycOJGUlBQ6duzIbLf/5NHR0SXbp6SkMHr0aLp27cpNN91UUiNetWoVXbt2ZcCAAdx1110VqrkvXryYHj160L17d+6/39ZrCwsLSU1NpXv37vTo0YNnnnkGgNmzZ9OtWzd69uzJuHHjql5YPqpKjT4Z2GOM2QcgIkuA4cDO4g2MMe6n0k+BdlU4XpU0aQLvvw833gh332372j/5pO2po5SCKVP8fy0rIQGefbbi7/v6669Zu3Yt4eHhnDx5kk2bNhEREcHatWuZPn06b7311nnv2bVrF+vXrycnJ4cuXbowefJk6pQaTLNt2zYyMzNp06YN/fv355NPPiEpKYk77riDTZs20aFDB8aPH+9znAcPHuT+++8nPT2dJk2a8Ktf/Yrly5fTvn17vv/+e3bs2AHAiRMnAHjqqafYv38/kZGRJa/VhKq00bcFvnN7fsD1mje/A97ztlJEbheRNBFJ8/UCQ0XVrw9vvml75fzP/0BqKuTnV8uhlFJVMGbMGMLDwwHIzs5mzJgxdO/enalTp5KZmenxPUOHDiUyMpKYmBhatGjB4cOHz9smOTmZdu3aERYWRkJCAllZWezatYuOHTvSoUMHgAol+i1btpCSkkLz5s2JiIjgpptuYtOmTXTs2JF9+/bx5z//mdWrV3PBBRcA0LNnT2666SZee+01r01S1aEqR/JUF/Z4ZUBEBmMT/QBvOzPGzMXVtJOUlFRtkxiEh8MLL0DbtvDII7ZXzhtv2JG1StVmlal5V5cGbiMdH374YQYPHsyyZcvIysoiJSXF43siIyNLHoeHh1NQUODTNr5e0PTE23ubNGlCRkYGa9asYc6cOSxdupR58+bxr3/9i02bNrFixQqeeOIJMjMzayThV6VGfwBo7/a8HXCw9EYi0hN4GRhujDleheP5jQg8/LCdJuH99+3Aqmr6EaGUqqLs7GzatrWNBfPnz/f7/rt27cq+ffvIysoC4PXXX/f5vX369GHjxo0cO3aMwsJCFi9ezKBBgzh27BhFRUWMGjWKJ554gq1bt1JUVMR3333H4MGD+d///V9OnDjBqVOn/P55PKnKqWQL0FlEOgDfA+OAG903EJELgbeBm40xX1fhWNVi0iRo2RLGjrUDq1avho4dnY5KKeXuvvvu49Zbb+Vvf/sbV155pd/3X79+fZ5//nmGDBlCTEwMycnJXrddt24d7dqdvdT4xhtv8OSTTzJ48GCMMVx33XUMHz6cjIwMbrvtNoqKigB48sknKSwsZMKECWRnZ2OMYerUqTRu3Njvn8cTqcrPFhG5DngW271ynjFmpojcCWCMeVFEXgZGAcUT4hcYY8rtD5WUlGRq8sYjmzfDsGFQty689x706lVjh1bKUenp6fTu3dvpMBx36tQpoqOjMcbwxz/+kc6dOzN16lSnw/IqPT2dL7/8kqKiIiZOnAiAiKR7y69VGjBljFlljLnEGNPJGDPT9dqLxpgXXY9/b4xpYoxJcC3ld3p1QL9+8MknNtEPGgTr1jkdkVKqJr300kskJCQQFxdHdnY2d9xxh9Mh+VVIj4ytiEsvtTX7iy6yA6uWLHE6IqVUTZk6dSrbt29n586dLFq0iKioKKdD8itN9G7atYNNm6BvXxg/PrB6ISilVGVpoi+leGDVyJEwdSrcdx+4rqcopVRQ0kTvQb169qYlkyfD00/rwCqlVHALmTtM+Vt4OMyZYwdWPfSQHVj15ps6sEopFXy0Rl8GEXuz8ZdftnPbDx5sE75Syj9SUlJYs2bNOa89++yz/OEPfyjzPcXdr6+77jqPc8bMmDGDWbNmlXns5cuXs3NnydRcPPLII6xdu7Yi4XsUaNMZgyZ6n/zud7B8OWRm2oFV+/Y5HZFSoWH8+PEsKdXFbcmSJT7PN7Nq1apKDzoqnegff/xxrr766krtK9BpovfRb35j+9f/+CNcfjls3ep0REoFv9GjR7Ny5Up++eUXALKysjh48CADBgxg8uTJJCUlERcXx6OPPurx/bGxsRxz3Rh65syZdOnShauvvrpkKmOwfeQvu+wy4uPjGTVqFLm5uWzevJkVK1bwl7/8hYSEBPbu3UtqaipvvvkmYEfA9urVix49ejBx4sSS+GJjY3n00UdJTEykR48e7Nq1y+fP6uR0xtpGXwGXX27ntR8yxA6sWrYMQrQCoGojB+YpbtasGcnJyaxevZrhw4ezZMkSxo4di4gwc+ZMmjZtSmFhIVdddRVffPEFPXv29Lif9PR0lixZwrZt2ygoKCAxMbFkxO/IkSOZNGkSAA899BD/+Mc/+POf/8z111/PsGHDGD169Dn7On36NKmpqaxbt45LLrmEW265hRdeeIEpU6YAEBMTw9atW3n++eeZNWsWL7/8crnF4PR0xlqjr6DigVUdOsB118E//+l0REoFN/fmG/dmm6VLl5KYmEivXr3IzMw8p5mltI8++ogbbriBqKgoLrjgAq6//vqSdTt27OCKK66gR48eLFq0yOs0x8V2795Nhw4duOSSSwC49dZb2bRpU8n6kSNHAtC7d++SidDK4/R0xlqjr4S2be3AqhEj4Kab4NAhmDbN6aiUqiKHRgiOGDGCadOmsXXrVvLy8khMTGT//v3MmjWLLVu20KRJE1JTUzl9+nSZ+xEvdxFKTU1l+fLlxMfHM3/+fDZs2FDmfsqb/6t4qmNvUyFXZJ81NZ2x1ugrqXFjO9vl6NFwzz1w7706sEqpyoiOjiYlJYWJEyeW1OZPnjxJgwYNaNSoEYcPH+a997zeswiAgQMHsmzZMvLy8sjJyeHdd98tWZeTk0Pr1q3Jz89n0aJFJa83bNiQnJyc8/bVtWtXsrKy2LNnDwALFy5k0KBBVfqMTk9nrDX6KqhXz86Jc/fd8H//Z2v28+bZydGUUr4bP348I0eOLGnCiY+Pp1evXsTFxdGxY0f69+9f5vsTExMZO3YsCQkJXHTRRVxxxRUl65544gn69OnDRRddRI8ePUqS+7hx45g0aRKzZ88uuQgLUK9ePV555RXGjBlDQUEBl112GXfeeWeFPk+gTWdcpWmKq0tNT1NcVcbY+8/+9a9wzTXw1lvQsKHTUSlVPp2mODjV6DTFyhKB6dNtbf7DDyElBTzcrlIppRyhid6PbrsN3nkHvvrKznHvauJTSilHaaL3s6FDba0+O9uOok1PdzoipcpWpL0Igkpl/r000VeDvn3tHavq17cDq95/3+mIlPIsKiqKQ4cOabIPEkVFRRw6dIj8Ck6nq71uqkmXLnZg1bXX2lr+/Pm2z71SgaRTp05s376dgwcPeu2HrgJLfn4+3377LUVFRT73rddEX43atDk7sGrCBNv98p57nI5KqbPq1q1L/fr1+eCDD2jUqBFhYfojPxgYYzhx4gTJyck+ba+Jvpo1amQHVt18sx1U9f33MGsW6P8nFSi6d+9Ofn4+X331VYWbBJQzIiIi6NOnD3379vVt+2qORwGRkXZgVatW8MwztmY/f74OrFKBQURITEwkMTHR6VBUNdFEX0PCwuC552xzzoMPwtGjdmCVaw4jpZSqNtqAUINE4IEH4JVXYP16O7Dq0CGno1JKhTpN9A5ITYV334Xdu+3Aqm++cToipVQo00TvkGuvtQOrTp60A6u2bHE6IqVUqNJE76A+fWxf+wYN7I3HV692OiKlVCjSRO+wSy6xyf7ii+19aRcudDoipVSo0UQfAFq3ho0bYeBAuOUWePppO/WxUkr5gyb6ANGoEaxaBWPHwn332VsT6vQjSil/0H70ASQy0t5svFUre/vOH36AV1+1ryulVGVpog8wYWF29GybNnD//XZg1bJlOrBKKVV52nQTgERs882rr9pJ0QYNsrV7pZSqDE30AeyWW+zAqm++sQOrvv7a6YiUUsFIE32AGzLETpdw6pQdWPX5505HpJQKNprog8Bll9m+9g0b2oFV773ndERKqWCiiT5IdO5sk32XLnZg1auvOh2RUipYaKIPIq1awYYNdtbL1FT4n//RgVVKqfJVKdGLyBAR2S0ie0TkAQ/ru4rIv0XkFxG5tyrHUtYFF9iBVePH2ymPp0zRgVVKqbJVuh+9iIQDc4BrgAPAFhFZYYzZ6bbZj8BdwIgqRanOUbcuvPbauXesWrBAB1YppTyrSo0+GdhjjNlnjDkDLAGGu29gjDlijNkC6I0o/SwsDP72NzsvztKltndOdrbTUSmlAlFVEn1b4Du35wdcr1WKiNwuImkiknb06NEqhFW73HuvnfHy4491YJVSyrOqJHrx8FqlLw0aY+YaY5KMMUnNmzevQli1z4QJ8K9/wZ49cPnl9s5VSilVrCqJ/gDQ3u15O+Bg1cJRlfWrX9keObm5dmDVZ585HZFSKlBUJdFvATqLSAcRqQuMA1b4JyxVGUlJtq99o0Zw5ZW2lq+UUpVO9MaYAuBPwBrgK2CpMSZTRO4UkTsBRKSViBwApgEPicgBEdF5GKvRxRfbZN+1KwwfDq+84nRESimnVWmaYmPMKmBVqddedHt8CNuko2pQy5a2GWfkSJg40V6gffBBOyumUqr20ZGxIaphQ9t0c+ON8Ne/wl13QWGh01EppZygNx4JYXXr2q6XrVvD//2fHVi1cCHUq+d0ZEqpmqSJPsSFhcGsWTbZ33uvvWPVO+/YC7ZKqdpBm25qiXvusdMmbN4MAwfCQe0Iq1StoYm+FrnpJttuv2+fHVi1a5fTESmlaoI23dQy11wDGzfCtdfagVUPPQTt2tmeOq1a2b8XXKA9dJQKJZroa6HERNuEM2wYTJt2/vp69c5N/O6PS/+NjtaTglKBThN9LdWpE2RmwrFjcPiw7ZHj6W9WFnz6qb2I6+kmJ1FRnk8Anl5r0KDGP6ZSCk30tVpYGLRoYZcePcretqCg/JPCN9/YWTSPHfO8j+josn8duD+uX9//n1ep2koTvfJJRIRNwq1aQXx82dvm59tfAGWdFHbtsqN3f/zR8z4uuKDsk4L7yUFvuKJU2TTRK7+rUwfatLFLec6cgSNHyj4p7NgBa9fCiROe99G4sW8nhRYt7CAypWobTfTKUXXr2l4/7XyYEen06fJPCtu328cnT3reR9Omvp8UIvR/hwoR+lVWQaNePbjwQruUJy/PJv6yTgppafbxqVPnv18EmjU7/yTQtKmdRyg62vvf6Gh7kVp7I6lAoYlehaT69SE21i7l+fnn8k8Kn35qH+fm+nZ8kbJPBhX9Gx1tm8SUqgxN9KrWa9AAOna0S3nOnLG/AHJyKvf3++/Pfe7p14Q3kZH+PXnor47aQxO9UhVQt65tvmna1D/7KyqyvxJKnxB8PXnk5Nh5i9xfP3PGt2MX/+rw58lDf3UEJk30SjkoLOxssvUXX391eFv3ww/w9dfnvu6r4l8dxcm/Xj0ID7efMyzM/4+rc99OHD8iwl4P8jdN9EqFmOr61VGRJqrix6dP2/cXFdkb3xQ/zs8//3X39f567Gk0dyBr2dJeC/I3TfRKqTK5/+po1crpaCrGmKqfMKrjBOTtcXUN/tNEr5QKWSK2WSQ8PECuHxhjB3kcPep5CQsDnvb7YUMr0aek2H/NZs0gJsYu3h5HRTkdrVIq2BUV2SHb3hK3p8Xb1fL69eHii+FpTfTeGWPvj3f0KPznP3ZmrZ9+8r59vXrlnwxKP9b+aEqFtsJCOH7c96R97Jh9jycNG0Lz5nZp1w569Tr73NNSjdO7hk6iF7E3Q3VXUGCT/bFj9h/v2DHvj7dutY+9zbIFZ08Ovp4Yin856MlBKWfk59v/3+Ul7CNH7N8ff/R+Bbdx47NJuVMne5s2b0k7JsbmiwAROonek4iIswXvq+KTQ3knhmPHYNu2s78cvH056tXzfAIo6wTRoIGeHJTy5PTpijWTeJsJr3iOi+L8EBdXdm07JiZAGvkrJ7QTfWVU5uRQWOj7L4ft28/+cvB2coiMrPgvBz05qGD088/l17LdF2+d+sPD7f+D5s3tjHSJiWUn7qZN7XtqCU30/lD8JYuJ8f09xScHX345ZGTYv+WdHCpyYtCTgyqtuC9ifr79ZVv815fH3tbn5JRd487L8xxL3brnJuaLLy47cTdu7OqxojzRRO8U95NDly6+vaew0P4ULe/EcPw4fPFF+SeHunXtNYTi/me6lL8YU/FkV93b+nNf1SUq6mxSbtGi/KaShg21EuJHmuiDSXi4rZE3a1a5k0PxycD9pJCXZ7epynLmTNX34b4E23DG6lLc+Tsiwi7Fjz29VvpxvXqVe5+/ty1O8HrDYEdpog917ieHYGGMf08c/lyKJySp7mQZHq41WuU3muhV4BE5m/SUUlWmVy+UUirEaaJXSqkQJyYAL3yJyFHgP5V8ewxwzI/h+IvGVTEaV8VoXBUTinFdZIzxOAAoIBN9VYhImjEmyek4StO4KkbjqhiNq2JqW1zadKOUUiFOE71SSoW4UEz0c50OwAuNq2I0rorRuCqmVsUVcm30SimlzhWKNXqllFJuNNErpVSIC8pELyJDRGS3iOwRkQc8rBcRme1a/4WIJAZIXCkiki0i213LIzUU1zwROSIiO7ysd6q8yovLqfJqLyLrReQrEckUkbs9bFPjZeZjXDVeZiJST0Q+F5EMV1yPedjGifLyJS5HvmOuY4eLyDYRWelhnX/LyxgTVAsQDuwFOgJ1gQygW6ltrgPeAwToC3wWIHGlACsdKLOBQCKww8v6Gi8vH+NyqrxaA4muxw2BrwPkO+ZLXDVeZq4yiHY9rgN8BvQNgPLyJS5HvmOuY08D/unp+P4ur2Cs0ScDe4wx+4wxZ4AlwPBS2wwHFhjrU6CxiLQOgLgcYYzZBJRxM1xHysuXuBxhjPnBGLPV9TgH+ApoW2qzGi8zH+Oqca4yKL71Ux3XUrqXhxPl5UtcjhCRdsBQ4GUvm/i1vIIx0bcFvnN7foDzv+y+bONEXACXu35KvicicdUck6+cKC9fOVpeIhIL9MLWBt05WmZlxAUOlJmrGWI7cAT4wBgTEOXlQ1zgzHfsWeA+oMjLer+WVzAmek+TdJc+S/uyjb/5csyt2Pko4oG/A8urOSZfOVFevnC0vEQkGngLmGKMOVl6tYe31EiZlROXI2VmjCk0xiQA7YBkEeleahNHysuHuGq8vERkGHDEGJNe1mYeXqt0eQVjoj8AtHd73g44WIltajwuY8zJ4p+SxphVQB0RqcCNZquNE+VVLifLS0TqYJPpImPM2x42caTMyovL6e+YMeYEsAEYUmqVo98xb3E5VF79getFJAvbxHuliLxWahu/llcwJvotQGcR6SAidYFxwIpS26wAbnFdue4LZBtjfnA6LhFpJWJvGyQiydjyP17NcfnCifIql1Pl5TrmP4CvjDF/87JZjZeZL3E5UWYi0lxEGrse1weuBnaV2syJ8io3LifKyxjzoDGmnTEmFpsnPjTGTCi1mV/LK+hu4WOMKRCRPwFrsD1d5hljMkXkTtf6F4FV2KvWe4Bc4LYAiWs0MFlECoA8YJxxXWKvTiKyGNu7IEZEDgCPYi9MOVZePsblSHlha1w3A1+62ncBpgMXusXmRJn5EpcTZdYaeFVEwrGJcqkxZqXT/yd9jMup79h5qrO8dAoEpZQKccHYdKOUUqoCNNErpVSI00SvlFIhLiAvxsbExJjY2Finw1BKqaCRnp5+zHi5Z2xAJvrY2FjS0tKcDkMppYKGiPzH2zptulFKqRAXkDV6pZSfFRXZpbCwZv+GhUH9+meXqKhzn0dGgnga7a/8SRN9bfPLL3D0qF2OHIHTp8GY8xeo+uv+2EegHdOY6kuK1ZlwA5UI1Kt3/gnA24nBl3VlvV63bq08sWiiD3YFBXDsmE3axcm7+K+nx9nZTkccGETOLqWfl/V6WJhdwsP98zcszNZq/bnPQPpbWAh5eWeX3Nxzn5f3+uHDnl8/c6by/+7VfTJxfx4gJxZN9IGmqAh+/LH8hF38+Ecv07mHhUHz5tCihf2blHT2cYsWZx9HRdntfU103l73xz5q6pgq+JU+gVT2RFL69ZMnPb+en1+5OItPLL6eTGJiYOZM/5YVmuirnzG2Fu1r4j52zPNPbRFo2vRsku7e/dyEXTqJN2lik71SoSg8HKKj7VITCgpsM2dFTyTlnWSys899rXFjTfQBwRj4+WffE/fRo95rA40anU3MF18M/fp5rnW3aAHNmkGE/nMp5YiIiJo9sfiZZg6wZ1L3C5TlJe68PM/7iY4+m5jbt4fExPMTdvHjmBjbNquUUtUsNBN9fn7ZFyZLv5aT43k/kZHQsuXZJN2tm/fE7d7erZRSASR0Er0xEBcHP/wAJ0543iYi4twk3amT52aS4sfR0XrxTikV9EIn0YvAgAG2Fu4tcTdurIlbKVUhxUMnCgvtNdnixf25P9YVFtremDfe6P/PEDqJHmDuXKcjUKpWKiqCU6dsK6j7cvq0f5NheYmyOvZfWFhz5diypSZ6pZSfGGN795VOzOUtnpJ5To7tiFYdIiLOLuHhnh+Xt65+/cq9z9s6f+zD27o6daqpHKtnt0opfzLGzl7hj6RcvM7XmRGioqBhw3OX1q3hkkvOf71hQ3tpq/hx/fo2eVUmGeowEP/RRK9UNcnPr3pCdl8KCnw7bmTk+cm3WTOIjfWcmEsn59Kvh4dXazGpGqCJXikviqdacV9++sn3GvUvv/h2nIgIz0m2TZuKJ+aGDavv578KXproVa2Sm2sT9qFD5ydx9+XQIe/DK0Q8J9iYmMolZp2pV1U3nxK9iAwBngPCgZeNMU+VWt8EmAd0Ak4DE40xO1zrpgK/BwzwJXCbMea03z6BqvVOnfKcqD0l8FOnPO+jaVPb46FlSzugufhxy5bQqtXZx02b2jZrTcwqmJSb6EUkHJgDXAMcALaIyApjzE63zaYD240xN4hIV9f2V4lIW+AuoJsxJk9ElgLjgPl+/hwqhBhja9Pl1biLH+fmet5Ps2ZnE3Vy8rnJ231p0cL2X1YqVPlSo08G9hhj9gGIyBJgOOCe6LsBTwIYY3aJSKyItHQ7Rn0RyQeigIP+Cl4FD2PsDLDl1biLF0/TCYnY5pHiBH355efXuIuX5s21rVqpYr4k+rbAd27PDwB9Sm2TAYwEPhaRZOAioJ0xJl1EZgHfAnnA+8aY96setgoExtjZJsqrcRcvni5OhoXZ5F2crDt39lzrbtXKbqcTeCpVcb78t/HUGmlKPX8KeE5EtmPb4bcBBa62++FAB+AE8IaITDDGvHbeQURuB24HuPDCC33/BMqvjLE9S3y5WHnkiOcb/YSH2xp1cZLu2tV7zTsmRrvvKVXdfEn0B4D2bs/bUar5xRhzErgNQEQE2O9afg3sN8Ycda17G+gHnJfojTFzgbkASUlJpU8kys+MgfnzYdOmcxP4kSOep88PD7dt2cXJOi7Oc627ZUvbNq6DXZQKHL4k+i1AZxHpAHyPvZh6zmwMItIYyDXGnMH2sNlkjDkpIt8CfUUkCtt0cxWQ5s8PoCquqAimTIG//90m5zZtbILu2dN7zbtpU03eSgWrchO9MaZARP4ErMF2r5xnjMkUkTtd618ELgUWiEgh9iLt71zrPhORN4GtQAG2SUdnHnPQL7/ArbfC66/D1Kkwa5YmcKVCnRgTeK0kSUlJJi1NK/7+lpMDI0fC2rXw1FNw333aH1ypUCEi6caYJE/rtA9DLXHkCAwdCtu2wbx5cNttTkeklKopmuhrgf374de/hgMHYPlyGDbM6YiUUjVJE32Iy8iAIUNs2/zatdCvn9MRKaVqml6GC2EbN8LAgbZr5EcfaZJXqrbSRB+ili2zzTVt2sDmzbbfu1KqdtJEH4JeeglGj4aEBPj4Y9CBxkrVbproQ4gx8F//Bbffbmvz69bZUapKqdpNL8aGiMJCuPtumDMHJkywXSh19kalFGiNPiT88gvceIKSzJoAABZySURBVKNN8vfcA6++qkleKXWW1uiD3MmTcMMN8OGH8PTTcO+9TkeklAo0muiD2OHDcN11tq/8q6/CLbc4HZFSKhBpog9S+/bZC67ffw8rVtiEr5RSnmiiD0Lbt8O119q2+XXr7C31lFLKG70YG2Q2bIBBg+wt9T7+WJO8Uqp8muiDyFtv2eaatm3taNdu3ZyOSCkVDDTRB4kXX4QxY6B3b1uTb9++/PcopRRoog94xsBjj8HkyfaC69q19rZ+SinlK70YG8AKC+Guu+D55+3t/156SQdCKaUqTmv0AeqXX2DcOJvk77sPXnlFk7xSqnK0Rh+ATp6EESNg/Xp78+577nE6IqVUMNNEH2AOHbJ95HfsgIUL7QRlSilVFT413YjIEBHZLSJ7ROQBD+ubiMgyEflCRD4Xke5u6xqLyJsisktEvhIR7fntxd690L8/fP21He2qSV4p5Q/lJnoRCQfmANcC3YDxIlK6B/d0YLsxpidwC/Cc27rngNXGmK5APPCVPwIPNdu22Vv9ZWfbCcquvdbpiJRSocKXGn0ysMcYs88YcwZYAgwvtU03YB2AMWYXECsiLUXkAmAg8A/XujPGmBN+iz5ErF9vR7tGRto+8n36OB2RUiqU+JLo2wLfuT0/4HrNXQYwEkBEkoGLgHZAR+Ao8IqIbBORl0WkgaeDiMjtIpImImlHjx6t4McIXm++CUOG2Nv9bd4MXbs6HZFSKtT4kujFw2um1POngCYish34M7ANKMBe7E0EXjDG9AJ+Bs5r4wcwxsw1xiQZY5KaN2/ua/xB7YUX4Le/haQk2LQJ2rVzOiKlVCjypdfNAcB9wH074KD7BsaYk8BtACIiwH7XEgUcMMZ85tr0Tbwk+trEGJgxAx5/HIYNg9dfh6gop6NSSoUqX2r0W4DOItJBROoC44AV7hu4etbUdT39PbDJGHPSGHMI+E5EurjWXQXs9FPsQamw0E5n8PjjcNttsGyZJnmlVPUqt0ZvjCkQkT8Ba4BwYJ4xJlNE7nStfxG4FFggIoXYRP47t138GVjkOhHsw1Xzr41On4abboK334YHHoD//m8QTw1jSinlR2JM6eZ25yUlJZm0tDSnw/Cr7Gw72nXDBnjmGZgyxemIlFKhRETSjTFJntbpyNga8MMPtl98ZiYsWgQ33uh0REqp2kQTfTXbswd+9Ss4cgRWrrQ3DlFKqZqkib4apafbmnxRkR3tmpzsdERKqdpIE301WbfOtsk3bQrvvw9dupT/HqVKO3PmDHv37iU3N9fpUFSAiIqKolOnTtStW7f8jV000VeDpUvh5pvhkktg9Wp7j1elKmPv3r00btyYLl26EBamt4+o7YqKijh06BAZGRlER0fTtWtXxIeue/rN8bM5c+wNQ5KT7WhXTfKqKnJzc2nZsqUmeQVAWFgYrVq1QkR47733yMjI8O191RxXrWEMPPww/OlP8Jvf2OaaJk2cjkqFAk3yyl1YWBgiQqNGjfjyyy99eo823fhBQQH84Q/2nq4TJ8L/+38QoSWrlKpG4eHh5Ofn+7StVhWq6PRpGDPGJvnp0+HllzXJq9Bx/PhxEhISSEhIoFWrVrRt27bk+ZkzZ8p8b1paGnfddVe5x+jXr5+/wgXg7rvvpm3bthQVFfl1v8FMU1IVnDgBw4fbtvjnngMfvtNKBZVmzZqxfft2AGbMmEF0dDT33ntvyfqCggIivNRskpKSSEryOFDzHJs3b/ZPsNiLlcuWLaN9+/Zs2rSJlJQUv+3bXWFhIeHh4dWy7+qgib6SfvjBziP/1VeweLG9AKtUdZoyBVw5128SEuDZZyv2ntTUVJo2bcq2bdtITExk7NixTJkyhby8POrXr88rr7xCly5d2LBhA7NmzWLlypXMmDGDb7/9ln379vHtt98yZcqUktp+dHQ0p06dYsOGDcyYMYOYmBh27NhB7969ee211xARVq1axbRp04iJiSExMZF9+/axcuXK82Jbv3493bt3Z+zYsSxevLgk0R8+fJg777yTffv2AfDCCy/Qr18/FixYwKxZsxARevbsycKFC0lNTWXYsGGMHj36vPgee+wxWrduzfbt29m5cycjRozgu+++4/Tp09x9993cfvvtAKxevZrp06dTWFhITEwMH3zwAV26dGHz5s00b96coqIiLrnkEj799FNiYmIq+a/nO030lfDNN3a069Gj8K9/wTXXOB2RUjXr66+/Zu3atYSHh3Py5Ek2bdpEREQEa9euZfr06bz11lvnvWfXrl2sX7+enJwcunTpwuTJk6lTp84522zbto3MzEzatGlD//79+eSTT0hKSuKOO+5g06ZNdOjQgfHjx3uNa/HixYwfP57hw4czffp08vPzqVOnDnfddReDBg1i2bJlFBYWcurUKTIzM5k5cyaffPIJMTEx/Pjjj+V+7s8//5wdO3bQoUMHAObNm0fTpk3Jy8vjsssuY9SoURQVFTFp0qSSeH/88UfCwsKYMGECixYtYsqUKaxdu5b4+PgaSfKgib7C0tLguutsL5sNG+xNQ5SqCRWteVenMWPGlDRdZGdnc+utt/LNN98gIl4vEA4dOpTIyEgiIyNp0aIFhw8fpl2pu+0kJyeXvJaQkEBWVhbR0dF07NixJLmOHz+euXPnnrf/M2fOsGrVKp555hkaNmxInz59eP/99xk6dCgffvghCxYsAOxFzEaNGrFgwQJGjx5dkmybNm1a7udOTk4uiQNg9uzZLFu2DIDvvvuOb775hqNHjzJw4MCS7Yr3O3HiRIYPH86UKVOYN28et91WcxP5aqKvgA8+gJEjISYG1qyxA6KUqo0aNDh7R9CHH36YwYMHs2zZMrKysry2i0dGRpY8Dg8Pp6CgwKdtfJ1hd/Xq1WRnZ9OjRw/AjkGIiopi6NChHrc3xngcbBQREVFyIdcYc85FZ/fPvWHDBtauXcu///1voqKiSElJ4fTp01732759e1q2bMmHH37IZ599xqJFi3z6XP6gvW58tGQJDB0KHTrAJ59okleqWHZ2Nm1dIwPnz5/v9/137dqVffv2kZWVBcDrr7/ucbvFixfz8ssvk5WVRVZWFvv37+f9998nNzeXq666ihdeeAGwF1JPnjzJVVddxdKlSzl+/DhASdNNbGws6enpALzzzjtef6FkZ2fTpEkToqKi2LVrF59++ikAl19+ORs3bmT//v3n7Bfg97//PRMmTOC3v/1tjV7M1UTvg9mzYfx46NvX9rBp08bpiJQKHPfddx8PPvgg/fv3p7Cw0O/7r1+/Ps8//zxDhgxhwIABtGzZkkaNGp2zTW5uLmvWrDmn9t6gQQMGDBjAu+++y3PPPcf69evp0aMHvXv3JjMzk7i4OP76178yaNAg4uPjmTZtGgCTJk1i48aNJCcn89lnn51Ti3c3ZMgQCgoK6NmzJw8//DB9+/YFoHnz5sydO5eRI0cSHx/P2LFjS95z/fXXc+rUqRpttgG98UiZike7zpxpJyj75z+hfn2no1K1SXp6Or1793Y6DMedOnWK6OhojDH88Y9/pHPnzkydOtXpsCosLS2NqVOn8tFHH1VpP+np6Xz55ZcUFRUxceJEoOwbj2iN3ouCApg0ySb5SZPgjTc0ySvllJdeeomEhATi4uLIzs7mjjvucDqkCnvqqacYNWoUTz75ZI0fW2v0HuTl2aaad96xNfrHHtN7uypnaI1eeVLRGr32uinlp5/g+uvtBde//91OUqaUUsFME72bgwftrf5277a9bH77W6cjUkqpqvOpjV5EhojIbhHZIyIPeFjfRESWicgXIvK5iHQvtT5cRLaJyPljlgPE119Dv36QlQWrVmmSV0qFjnITvYiEA3OAa4FuwHgR6VZqs+nAdmNMT+AW4LlS6+8Gvqp6uNVjyxbo3x9yc+1o16uvdjoipZTyH19q9MnAHmPMPmPMGWAJMLzUNt2AdQDGmF1ArIi0BBCRdsBQ4GW/Re1H778PgwdDw4a2XV6veyl1VkpKCmvWrDnntWeffZY//OEPZb6nuDPFddddx4kTJ87bZsaMGcyaNavMYy9fvpydO3eWPH/kkUdYu3ZtRcIvU22aztiXRN8W+M7t+QHXa+4ygJEAIpIMXAQUT2LxLHAfUGZpisjtIpImImlHjx71Iayq++c/7WjXiy+2Sb5z5xo5rFJBY/z48SxZsuSc15YsWVLmxGLuVq1aRePGjSt17NKJ/vHHH+dqP/3cLj2dcXWpjgFkleFLovfUsbB0n8yngCYish34M7ANKBCRYcARY0x6eQcxxsw1xiQZY5KaN2/uQ1hV89xzcNNNtslm40Zo3braD6lU1UyZAikp/l2mTCnzkKNHj2blypX88ssvAGRlZXHw4EEGDBjA5MmTSUpKIi4ujkcffdTj+2NjYzl27BgAM2fOpEuXLlx99dXs3r27ZJuXXnqJyy67jPj4eEaNGkVubi6bN29mxYoV/OUvfyEhIYG9e/eSmprKm2++CcC6devo1asXPXr0YOLEiSXxxcbG8uijj5KYmEiPHj3YtWuXx7iKpzOePHkyixcvLnn98OHD3HDDDcTHxxMfH18yV/6CBQvo2bMn8fHx3HzzzQDnxAN2OmOwc+AMHjyYG2+8sWTenREjRtC7d2/i4uLOmZBt9erVJCYmEh8fz1VXXUVRURGdO3emuLJbVFTExRdfXFKGleVLoj8AtHd73g446L6BMeakMeY2Y0wCto2+ObAf6A9cLyJZ2CafK0XktSpFXEXGwIMP2u/3yJGwejWUGk2tlHJp1qwZycnJrF69GrC1+bFjxyIizJw5k7S0NL744gs2btzIF1984XU/6enpLFmyhG3btvH222+zZcuWknUjR45ky5YtZGRkcOmll/KPf/yDfv36cf311/P000+zfft2OnXqVLL96dOnSU1N5fXXX+fLL7+koKCgZB4bgJiYGLZu3crkyZO9Ng8VT2d8ww03sHLlypL5bIqnM87IyGDr1q3ExcWVTGf84YcfkpGRwXPPlb4Eeb7PP/+cmTNnlvwimTdvHunp6aSlpTF79myOHz/O0aNHmTRpEm+99RYZGRm88cYb50xnDPhtOmNfulduATqLSAfge2AccKP7BiLSGMh1teH/HthkjDkJPOhaEJEU4F5jzIQqRVwFBQVwxx0wb579O2cOBNFNYlRt59A8xcXNN8OHD2fJkiXMmzcPgKVLlzJ37lwKCgr44Ycf2LlzJz179vS4j48++ogbbriBqKgowM75UmzHjh089NBDnDhxglOnTvHrX/+6zHh2795Nhw4duMQ1s+Ctt97KnDlzmOL6dTJy5EgAevfuzdtvv33e+2vjdMblJnpjTIGI/AlYA4QD84wxmSJyp2v9i8ClwAIRKQR2Ar+rcmR+lptr7wL17rvwyCMwY4aOdlXKFyNGjGDatGls3bqVvLw8EhMT2b9/P7NmzWLLli00adKE1NRUTp8+XeZ+PE3dC7YJZPny5cTHxzN//nw2bNhQ5n7KG81fPNWxt6mQa+N0xj71ozfGrDLGXGKM6WSMmel67UVXkscY829jTGdjTFdjzEhjzE8e9rHBGDOsyhFXwk8/2TtCrVxpa/E6pYFSvouOjiYlJYWJEyeWXIQ9efIkDRo0oFGjRhw+fJj33nuvzH0MHDiQZcuWkZeXR05ODu+++27JupycHFq3bk1+fv45Sa1hw4bk5OSct6+uXbuSlZXFnj17AFi4cCGDBg3y+fPUxumMQ35Ss++/hyuusH3lly6FMnqFKaW8GD9+PBkZGYxz3Rw5Pj6eXr16ERcXx8SJE+nfv3+Z7y++t2xCQgKjRo3iiiuuKFn3xBNP0KdPH6655hq6du1a8vq4ceN4+umn6dWrF3v37i15vV69erzyyiuMGTOGHj16EBYWxp133unT56it0xmH9KRmu3bZKQ1++gmWL4crr/RDcErVIJ3UrHYqbzpjndTM5bPPbB/5iAjbfbJXL6cjUkqp8j311FO88MILfr3VYEg23axebWvvjRrZgVCa5JVSweKBBx7gP//5DwMGDPDbPkMu0S9aBL/5jb2n6yefgFv3W6WCUm0Yoq98V5nvQ0gl+meegQkT7MXXjRuhVSunI1KqaqKiojh06JAmewXYJH/o0CGvPXy8CZk2+uPH4amnYPRoWLgQ6tVzOiKlqq5Tp05kZmZy8OBBr/3QVe2Sn5/Pt99+y88//0xrH+duCZlE36wZfPopXHihjnZVoaNu3bp069aNFStWaLJX56hXr57P4wdCJtEDuI04VipkREZGMmLECI4fP+5xpKeqfcLCwmjSpAn169f3afuQSvRKhao6derQSi86qUoKqYuxSimlzheQI2NF5Cjwn0q+PQao2uTN1UPjqhiNq2I0rooJxbguMsZ4vJlHQCb6qhCRNG/DgJ2kcVWMxlUxGlfF1La4tOlGKaVCnCZ6pZQKcaGY6OeWv4kjNK6K0bgqRuOqmFoVV8i10SullDpXKNbolVJKudFEr5RSIS4oE72IDBGR3SKyR0Qe8LBeRGS2a/0XIpIYIHGliEi2iGx3LY/UUFzzROSIiOzwst6p8iovLqfKq72IrBeRr0QkU0Tu9rBNjZeZj3HVeJmJSD0R+VxEMlxxPeZhGyfKy5e4HPmOuY4dLiLbRGSlh3X+LS9jTFAtQDiwF+gI1AUygG6ltrkOeA8QoC/wWYDElQKsdKDMBgKJwA4v62u8vHyMy6nyag0kuh43BL4OkO+YL3HVeJm5yiDa9bgO8BnQNwDKy5e4HPmOuY49Dfinp+P7u7yCsUafDOwxxuwzxpwBlgDDS20zHFhgrE+BxiLi23ye1RuXI4wxm4Afy9jEifLyJS5HGGN+MMZsdT3OAb4C2pbarMbLzMe4apyrDE65ntZxLaV7eThRXr7E5QgRaQcMBV72solfyysYE31b4Du35wc4/8vuyzZOxAVwueun5HsiElfNMfnKifLylaPlJSKxQC9sbdCdo2VWRlzgQJm5miG2A0eAD4wxAVFePsQFznzHngXuA7zdUcav5RWMid7ThNylz9K+bONvvhxzK3Y+injg78Dyao7JV06Uly8cLS8RiQbeAqYYY06WXu3hLTVSZuXE5UiZGWMKjTEJQDsgWUS6l9rEkfLyIa4aLy8RGQYcMcakl7WZh9cqXV7BmOgPAO3dnrcDDlZimxqPyxhzsvinpDFmFVBHRGKqOS5fOFFe5XKyvESkDjaZLjLGvO1hE0fKrLy4nP6OGWNOABuAIaVWOfod8xaXQ+XVH7heRLKwTbxXishrpbbxa3kFY6LfAnQWkQ4iUhcYB6wotc0K4BbXleu+QLYx5gen4xKRViL2FkEikowt/+PVHJcvnCivcjlVXq5j/gP4yhjzNy+b1XiZ+RKXE2UmIs1FpLHrcX3gamBXqc2cKK9y43KivIwxDxpj2hljYrF54kNjzIRSm/m1vILuxiPGmAIR+ROwBtvTZZ4xJlNE7nStfxFYhb1qvQfIBW4LkLhGA5NFpADIA8YZ1yX26iQii7G9C2JE5ADwKPbClGPl5WNcjpQXtsZ1M/Clq30XYDpwoVtsTpSZL3E5UWatgVdFJBybKJcaY1Y6/X/Sx7ic+o6dpzrLS6dAUEqpEBeMTTdKKaUqQBO9UkqFOE30SikV4jTRK6VUiNNEr5RSIU4TvVJKhThN9EopFeL+P1Z3J8NSLRcIAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(2,1)\n",
    "ax[0].plot(history.history['loss'], color='b', label=\"Training Loss\")\n",
    "ax[0].plot(history.history['val_loss'], color='r', label=\"Validation Loss\",axes =ax[0])\n",
    "legend = ax[0].legend(loc='best', shadow=True)\n",
    "\n",
    "ax[1].plot(history.history['acc'], color='b', label=\"Training Accuracy\")\n",
    "ax[1].plot(history.history['val_acc'], color='r',label=\"Validation Accuracy\")\n",
    "legend = ax[1].legend(loc='best', shadow=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The accuracy increases over time and the loss decreases over time. However, the accuracy of our validation set seems to slightly decrease towards the end even thought our training accuracy increased. Running the model for more epochs might cause our model to be susceptible to overfitting."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.2 Predict Results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10000/10000 [==============================] - 1s 119us/sample - loss: 0.0282 - acc: 0.9928\n"
     ]
    }
   ],
   "source": [
    "test_loss, test_acc = model.evaluate(x_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our model runs pretty well, with an accuracy of 99.3% on our testing data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.3 Confusion Matrix\n",
    "\n",
    "Run the following cell to compute our confusion matrix using TensorFlow."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Predict the values from the testing dataset\n",
    "Y_pred = model.predict(x_test)\n",
    "# Convert predictions classes to one hot vectors \n",
    "Y_pred_classes = np.argmax(Y_pred,axis = 1) \n",
    "# Convert testing observations to one hot vectors\n",
    "Y_true = np.argmax(y_test,axis = 1)\n",
    "# compute the confusion matrix\n",
    "confusion_mtx = tf.math.confusion_matrix(Y_true, Y_pred_classes) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the following cell to plot the confusion matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7f3b1c040090>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAi8AAAHSCAYAAAAkMCseAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzde3wU1f3/8ddnk3C/FwSSUEBBBWsVCXhXLAp4AdQq2oq1ra3WUi9fFS9ftPzot/SrVqz4rdqCCHgDom1FVBAFFWkRiBAQwv1SDATQco8ISfb8/sgaQUg2iZvMHPf97GMf7s7OMO89nWxOPufMjDnnEBEREfFFJOgAIiIiIlWhzouIiIh4RZ0XERER8Yo6LyIiIuIVdV5ERETEK+q8iIiIiFdSa3oH+6c/4dW52I0HPhx0BBER8Vzxwc1Wm/sr+mx9wn/XprU8tlY/Q1Wo8iIiIiJeqfHKi4iIiNSwaEnQCWqVKi8iIiLiFVVeREREfOeiQSeoVaq8iIiIiFdUeREREfFdNLkqL+q8iIiIeM5p2EhEREQkvFR5ERER8V2SDRup8iIiIiJeUeVFRETEd0k250WdFxEREd/pCrsiIiIi4aXKi4iIiO+SbNhIlRcRERHxiiovIiIivtOp0uHx4vtL+OFDk7jyoZd44b0lANwz4S0GPTKZQY9M5uIRzzHokcll66/e8hk/+dMrXPnQS1z18CQOFBUHFf0Iffv0YvmyOazMm8s9Q4cEHSeusWNGsSV/CbmLZwUdpdJ8a2PwL7NvecG/zL797GVmpvPOzJf5eOl7LMmdza2/uTHoSJXi23ERj3PRhD/CzJxzNbqD/dOfqNYO1hb8h3snzuSFO68iLSWFIX+dxn9ffT7tWzUrW2fUq3NpVK8uN/frQXFJlB89ms3vB1/ICRkt2VX4BY3r1yElUrX+WeOBD1cnboUikQgrln9Av0t+RH5+AR/Oe5PB1/+aFSvWJHxfiXLuOaezb18h48eP5tRuvYOOE5ePbexbZt/ygp+ZffvZa9PmGNq2OYbFucto1KghC+bP4IdX/TzUbVwbx0Xxwc2WsH+sEg6s+zDhv8zrHndGrX6Gqoj7m93MTjSze83sCTMbHXvepaaDrd+2k+93aE39OmmkpkToflw6s5euL3vfOcfM3HX0694ZgHmrNtE5/TuckNESgGYN61W541JTevboxrp1G9mwYRNFRUVkZ09lQP++Qceq0Adz57Nj566gY1Saj23sW2bf8oKfmX372du6dTuLc5cBsG9fIStXriEjvU3AqSrm43ERVzSa+EeIVfjb3czuBSYDBiwAFsaeTzKz+2oyWKc2Lfho3RZ2FX7B/oNFzM37N9t27St7f9H6Ar7TuH5ZJebf23djBrc8/RrXPjqF8bMW1WS8KknPaMMn+VvKXudvLiA95D/cvvGxjX3L7Fte8DOzz9q3z+TUU77H/AWLg45SIR0X/os3YfdG4CTnXNGhC83sMWA58FBNBTu2TQt+1vs0fvX0VBrUSeP4jJaHVVJmfLSafqd1LntdEo2yeH0BL955NfXqpHLzk1Pp2q4Vpx/frqYiVprZkZW3mh6uSzY+trFvmX3LC35m9lXDhg3InjKWO+8ezt69++JvEKBv5XER8jkqiRZvXCUKpB9ledvYe0dlZjeZWY6Z5Yyb/q9qh7vijK5Mvvsanr3tSpo0qMt3WzUFoLgkyqyl6+nb7avOS+tmjeh+XAbNG9Wnfp00zunanhX5n1Z734m0Ob+AdplfNWNmRlsKCrYFmOjbx8c29i2zb3nBz8w+Sk1N5eUpY5k06R+8+ur0oOPEpePCf/E6L3cAs8xsupmNiT1mALOA28vbyDk3xjmX5ZzLuvHis6odbsfezwEo2LmX2UvXc3Gs0jJ/9Sd0bN2c1s0ala171ontWFPwGfsPFlFcEuWjdVs4tnWLau87kRbm5NKpU0c6dGhHWloagwYNZNrrM4OO9a3iYxv7ltm3vOBnZh+NHTOKFSvX8vjoMUFHqZRv5XERLUn8I8QqHDZyzs0ws+OBnkAGpfNd8oGFzrka/2R3jZ/B7sIvSE2JcP9V59GkQT0AZixae9iQEUCTBvW4vtepXPfYyxjGOV3bc95JHWo6YqWUlJRw+x0P8OYbL5ESiTBh4hTy8lYHHatCLzz/JOefdyYtW7Zg4/ocRvzuUcZPmBx/w4D42Ma+ZfYtL/iZ2befvbPP6sH1g69i6cd55Cws7QA8+OBDTJ8xO+Bk5fPxuIgryYaNQnuqdFBq4lRpERFJLrV+qvSKdxN/qnSXC0J7qrSusCsiIuK7kJ/anGjhuBCKiIiISCWp8iIiIuK7JJvzos6LiIiI7zRsJCIiIhJeqryIiIh4rhauXhIqqryIiIiIV1R5ERER8Z0m7IqIiIhXNGFXREREJLxUeREREfFdkg0bqfIiIiIiXlHlRURExHfR5DpVWp0XERER32nYSERERCS8VHkRERHxnU6VFhEREQmvGq+8NB74cE3vIqH2b/kg6AhVVj/93KAjiIhIkDTnRURERCS8NOdFRETEd0k250WdFxEREd8lWedFw0YiIiJSZWb2rJltN7NlhyxrYWZvm9ma2H+bH/Le/Wa21sxWmVnfQ5Z3N7OPY+89YWYWb9/qvIiIiHjOuZKEPyphAtDva8vuA2Y55zoDs2KvMbOuwLXASbFtnjKzlNg2TwM3AZ1jj6//m0dQ50VERESqzDk3B9jxtcUDgYmx5xOByw9ZPtk5d8A5twFYC/Q0s7ZAE+fcPOecA547ZJtyac6LiIiI72pgzouZ3URpReRLY5xzY+Js1to5VwDgnCsws2NiyzOADw9ZLz+2rCj2/OvLK6TOi4iIiO9q4DovsY5KvM5KZR1tHourYHmFNGwkIiIiibItNhRE7L/bY8vzgXaHrJcJbIktzzzK8gqp8yIiIuK7aDTxj+p5Dbgh9vwGYOohy681s7pm1pHSibkLYkNMe83sjNhZRj85ZJtyadhIREREqszMJgG9gJZmlg8MBx4Css3sRmATcDWAc265mWUDeUAxMMR9dUrTLZSeuVQfmB57VEidFxEREd8FcG8j59yPynmrdznrjwRGHmV5DvC9quxbnRcRERHf6Qq7IiIiIuGlyouIiIjvAhg2CpIqLyIiIuIVLzsvffv0YvmyOazMm8s9Q4cEmuWBPzzGeZdey+WDf1W27K3ZHzDwups5+ZxLWLZiddnyj/NW8cMbhvDDG4Zw5Q2/5p33/1n23ui/TqD3FdfT48IrajV/ecaOGcWW/CXkLp4VdJRK8S3vl8J0LFeGb3l9PC58a2NQO4dCeE6VrhXedV4ikQhPjB7JZf0Hc/IpF3DNNZfTpUvnwPJcfslF/OWx3x+2rNOx7Xn8Dw/S/dTvHbF8yrgn+NvEJ/nrqN/zu0f+j+Li0jPFep19OpPHjq613PE891w2l152XdAxKs23vBC+Yzke3/KCf8eFj20Mamepfd51Xnr26Ma6dRvZsGETRUVFZGdPZUD/vvE3rCFZp55M0yaND1t2XIfv0rF95hHr1q9Xj9TU0ptoHjh4EA656/cp3+tCq5YtajZsFXwwdz47du4KOkal+ZYXwncsx+NbXvDvuPCxjUHtHAqqvIRbekYbPsn/6srB+ZsLSE9vE2Ciqlm6fCUDr7uZK35yC78d+puyzowkH9+OZd/y+khtXDu+le3sool/hFi1Oy9m9rNEBqnCfo9YVnoXbT98/6QTmfriX5n8zGieeT6bAwcOBh1JAuLbsexbXh+pjWuH2tl/36TyMqK8N8zsJjPLMbOcaLTwG+ziSJvzC2iXmV72OjOjLQUF2xK6j9pwXIfvUr9ePdas3xh0FAmIb8eyb3l9pDauHd/Kdtaw0VfMbGk5j4+B1uVt55wb45zLcs5lRSINExp4YU4unTp1pEOHdqSlpTFo0ECmvT4zofuoKflbtpZN0N2ydRsbN+WT0bbcZpRvOd+OZd/y+khtXDvUzv6Ld5G61kBfYOfXlhvwrxpJFEdJSQm33/EAb77xEimRCBMmTiEvb3X8DWvI0OEPsXDxUnbt2kPvywfz6xuvp2mTRvzvn55mx67d/HrocE7sfCxj/jSSRUuXM+75bFJTU4lEjAfuHkLzZk0BGPXkON58+12++OIAvS8fzJX9+zHkxsGBfa4Xnn+S8887k5YtW7BxfQ4jfvco4ydMDixPPL7lhfAdy/H4lhf8Oy58bGNQO4dCyOeoJJpVNM5nZuOA8c65uUd57yXn3I/j7SC1ToZXA4n7t3wQdIQqq59+btARRETkEMUHNx85saYG7f/HQwn/XVv/ivtq9TNURYWVF+fcjRW8F7fjIiIiIpJoureRiIiI75Js2Mi767yIiIhIclPlRURExHchP7U50dR5ERER8V2SdV40bCQiIiJeUeVFRETEd0l2ewNVXkRERMQrqryIiIj4TnNeRERERMJLlRcRERHfJVnlRZ0XERER3+kKuyIiIiLhpcqLiIiI75Js2EiVFxEREfGKKi8iIiK+S7KL1KnzIiIi4rskGzZS5+Vr6qefG3SEKts357GgI1RZo/PuDDqCiIh4Sp0XERER3yVZ5UUTdkVERMQrqryIiIj4LskuUqfOi4iIiOdcNLnONtKwkYiIiHhFlRcRERHfacKuiIiISHip8iIiIuK7JJuwq8qLiIiIeEWVFxEREd8l2dlG6ryIiIj4ThN2RURERMJLlRcRERHfqfIiIiIiEl6qvIiIiPjOacKuiIiI+ETDRiIiIiLh5V3nJTMznXdmvszHS99jSe5sbv3NjUFHimvsmFFsyV9C7uJZQUfht8/8g16/eZgr//vPZct27/ucmx+ZQP97HufmRyawp3A/APOWreXa3z7ND4f9mWt/+zTz89aXbXPLo89x9QNPcsX9/8f/THiNkhD0+vv26cXyZXNYmTeXe4YOCTpOpfiW2be84GfmSCTCwgVvMfUfE4OOEpeP38ng53FRoahL/CPEvOu8FBcXM/SeEZz8/V6cfU5/brnlp3Tp0jnoWBV67rlsLr3suqBjADDwnG48fff1hy179o0P6Nn1WKY9cgc9ux7LuNc/AKBZ44Y88V/X8beRv+F/brqSYX/9W9k2fxwyiJd/P4S//+E37Nz7OTMXLK/Vz/F1kUiEJ0aP5LL+gzn5lAu45prLQ39c+JbZt7zgZ2aA2279BStXrgk6RqX4+J3s63EhX4nbeTGzE82st5k1+tryfjUXq3xbt25nce4yAPbtK2TlyjVkpLcJIkqlfTB3Pjt27go6BgDdT+xAk4b1D1v27qKVDDinGwADzunGu4tWANClfVuOad4EgE4Zx3CwqJiDRcUANKpfD4DikihFxcWY1dYnOLqePbqxbt1GNmzYRFFREdnZUxnQv2+woeLwLbNvecHPzBkZbbnk4t48++ykoKNUio/fyT4eF3G5aOIfIVZh58XMbgOmArcCy8xs4CFv/6Emg1VG+/aZnHrK95i/YHHQUby2Y08hrZo1BqBVs8bs2FN4xDrv5ORxYvu21En7ao73r/44kQtufZiG9epyUY+Tai3v0aRntOGT/C1lr/M3F5Ae8i9Q3zL7lhf8zPzYqBHcd//viYZgKLaqfPlO9vG4iEvDRof5JdDdOXc50At40Mxuj70X6N/aDRs2IHvKWO68ezh79+4LMsq33tr87Tw+ZSYP/nTAYcv/MvQGZo0eysHiEhYcMh8mCHaU0o8L+amDvmX2LS/4l/nSSy5k+/bPWLT446CjVJlP38m+HRdypHinSqc45/YBOOc2mlkv4BUza08FnRczuwm4CcBSmhKJNExQ3FKpqam8PGUskyb9g1dfnZ7QfzsZtWjSkE937aVVs8Z8umsvLZp89f/Xth27+a8nJvH7m66kXesWR2xbt04avbqdwLuLVnLm9zrVZuzDbM4voF1metnrzIy2FBRsCyxPZfiW2be84F/ms87Kov9lfbi43w+oV68uTZo0ZuKEJ7jhp7cFHa1Cvn0n+3ZcVIbzsFL3TcSrvGw1s1O/fBHryFwGtAROLm8j59wY51yWcy4r0R0XKD17Z8XKtTw+ekzC/+1k1Kvbibw2t7TM+9rcxVxw2okA7Cncz28ee4Hbr76Qbse3L1v/8y8O8OmuvQAUl5TwwZI1dGzbqvaDH2JhTi6dOnWkQ4d2pKWlMWjQQKa9PjPQTPH4ltm3vOBf5mEPPESHY7PodPwZXDf417z77j9D33EB/76TfTsu5EjxKi8/AYoPXeCcKwZ+YmZ/rbFUFTj7rB5cP/gqln6cR87C0oPtwQcfYvqM2UHEqZQXnn+S8887k5YtW7BxfQ4jfvco4ydMDiTLvU+9TM7KDeza9zkX3fEot1xxAT+/7FyGPjmFV+csos13mvLokGsAmPzOfDZt28GY195nzGvvA/D00J+Ag9sff5GDRSWURKP07HosV/8gK5DP86WSkhJuv+MB3nzjJVIiESZMnEJe3upAM8XjW2bf8oKfmX3j43fyt/K4CPkclUSzmh7nS62TkVwtGoB9cx4LOkKVNTrvzqAjiIjUmOKDm2t1XmjhyJ8k/Hdtw2HPBXweafl0ewARERHfhfzU5kRT50VERMR3STZs5N0VdkVERCS5qfIiIiLiO50qLSIiIhJeqryIiIj4LsnmvKjzIiIi4rskO9tIw0YiIiLiFVVeREREfJdkw0aqvIiIiIhX1HkRERHxnItGE/6oDDP7LzNbbmbLzGySmdUzsxZm9raZrYn9t/kh699vZmvNbJWZ9a3u51XnRURExHdRl/hHHGaWAdwGZDnnvgekANcC9wGznHOdgVmx15hZ19j7JwH9gKfMLKU6H1edFxEREamuVKC+maUCDYAtwEBgYuz9icDlsecDgcnOuQPOuQ3AWqBndXaqzouIiIjvAqi8OOc2A48Cm4ACYLdzbibQ2jlXEFunADgmtkkG8Mkh/0R+bFmVqfMiIiIiRzCzm8ws55DHTV97vzml1ZSOQDrQ0MwGV/RPHmVZtU6T0qnSIiIivquBi9Q558YAYypY5UJgg3PuUwAz+ztwFrDNzNo65wrMrC2wPbZ+PtDukO0zKR1mqjJVXkRERKQ6NgFnmFkDMzOgN7ACeA24IbbODcDU2PPXgGvNrK6ZdQQ6Awuqs2NVXr4FGp13Z9ARqmzvOyODjlAljS8cFnQEEZHyBXCROufcfDN7BVgEFAOLKa3UNAKyzexGSjs4V8fWX25m2UBebP0hzrmS6uxbnRcRERHPuYCusOucGw4M/9riA5RWYY62/kjgG//1qmEjERER8YoqLyIiIr7TvY1EREREwkuVFxEREd9V8l5E3xbqvIiIiPhOw0YiIiIi4aXKi4iIiO9UeREREREJL1VeREREPOdcclVe1HkRERHxnYaNRERERMJLlRcRERHfqfIiIiIiEl6qvIiIiHguqLtKB0WVFxEREfGKl52Xvn16sXzZHFbmzeWeoUOCjlMpvmUOU97hE97ggjtH88PhY8uW7S7cz82PTaL/sL9w82OT2FO4v+y9cW/+i/7//TQDH/gr/1q2HoDCLw4waMS4skev/3qcRya/Xeuf5evC1M6V4Vte8C+zb3nr1q3LvH++zkc5b7MkdzbDf3tX0JEqxbd2jivqEv8IMe86L5FIhCdGj+Sy/oM5+ZQLuOaay+nSpXPQsSrkW+aw5R1w1sk8dfs1hy17dvo8Tu/SgWkjf8XpXTrw7PQPAVi35TPeWriCv434JU/dfg1/eOktSqJRGtarS/bwG8sebVs0pfdpJwTxccqErZ3j8S0v+JfZt7wABw4c4MI+g+iedRHds/rQt08vTu95WtCxKuRjO8cVrYFHiHnXeenZoxvr1m1kw4ZNFBUVkZ09lQH9+wYdq0K+ZQ5b3u7Hf5cmDesdtuy93DX0P/NkAPqfeTLv5q6OLV9N3x5dqJOWSkarZrRr1ZxlG7Yctu2/t+1gx95CTuvcrnY+QDnC1s7x+JYX/MvsW94vFRZ+DkBaWiqpaWmhv2Car+0sX4nbeTGznmbWI/a8q5ndaWaX1Hy0o0vPaMMn+V/9MsrfXEB6epug4lSKb5l9yPufPYW0atYIgFbNGrFjb+mX5/Zde2nToknZeq2bN2b7rn2HbTtjQR59e3TBzGov8FH40M6H8i0v+JfZt7xfikQi5CycScHmpcyaNYcFCxcHHalCvrZzRVzUJfwRZhV2XsxsOPAE8LSZ/S/wZ6ARcJ+ZDauFfEfLdMSysPfyfcvsW95DHS3m1z/NWwvz6Neza63kqYhv7exbXvAvs295vxSNRsnq0Yf2HbPokdWNk04Kdkg2Hl/bWb4S71Tpq4BTgbrAViDTObfHzP4IzAdGHm0jM7sJuAnAUpoSiTRMWODN+QW0y0wve52Z0ZaCgm0J+/drgm+Zfcj7nSYN+XTXPlo1a8Snu/bRonEDoLTSsnXHnrL1tu3cW1ahAVj1yTaKS6J0bd+21jN/nQ/tfCjf8oJ/mX3L+3W7d+/h/Tn/Kp0Mu3xV0HHK5Xs7H1XIKyWJFm/YqNg5V+Kc+xxY55zbA+Cc208F03mcc2Occ1nOuaxEdlwAFubk0qlTRzp0aEdaWhqDBg1k2uszE7qPRPMtsw95zz+lM9PmfQzAtHkf0+vUzmXL31q4goNFxWz+dBebtu/kex2/+pKasSAcVRfwo50P5Vte8C+zb3kBWrZsQdOmpUO19erVo/cPzmXVqnUBp6qYj+0cV5JN2I1XeTloZg1inZfuXy40s6YE9NFKSkq4/Y4HePONl0iJRJgwcQp5eauDiFJpvmUOW977xrxKzupN7Nq3nz5D/8wtA87l5xefwT1/fZV/zF1C2xZN+OOvrgCgU0YrLso6kSuHjyUlEuH+H/chJfJVH31mzkr+fNugoD7KYcLWzvH4lhf8y+xbXoC2bVvz7LjHSUmJEIlEeOWVabzx5jtBx6qQj+0sh7OKxvnMrK5z7sBRlrcE2jrnPo63g9Q6GclVy5JK2fvOUUccQ6vxhYFM8RIRTxUf3FyrZwTsvLpXwn/XNn/5vWDPaqhAhZWXo3VcYss/Az6rkUQiIiIiFdC9jURERHwX8jkqiabOi4iIiOfCfl2WRPPuCrsiIiKS3FR5ERER8V2SDRup8iIiIiJeUeVFRETEcy7JKi/qvIiIiPguyTovGjYSERERr6jyIiIi4rlkGzZS5UVERES8osqLiIiI71R5EREREQkvVV5EREQ8l2xzXtR5ERER8VyydV40bCQiIiJeUeVFRETEc8lWeVHnRQLR+MJhQUeokr1T7w06QpU1Hvhw0BFERGqEOi8iIiK+cxZ0glqlzouIiIjnkm3YSBN2RURExCuqvIiIiHjORZNr2EiVFxEREfGKKi8iIiKeS7Y5L+q8iIiIeM4l2dlGGjYSERERr6jyIiIi4rlkGzZS5UVERES8osqLiIiI53SqtIiIiEiIqfIiIiLiOeeCTlC71HkRERHxnIaNREREREJMlRcRERHPqfIiIiIiEmJedl769unF8mVzWJk3l3uGDgk6TqX4ltm3vBDezC++v4QfPjSJKx96iRfeWwLAPRPeYtAjkxn0yGQuHvEcgx6ZXLb+6i2f8ZM/vcKVD73EVQ9P4kBRcVDRjxDWNq6Ib5l9y/ulSCTCwgVvMfUfE4OOEldmZjrvzHyZj5e+x5Lc2dz6mxuDjvSNOZf4R5h5N2wUiUR4YvRI+l3yI/LzC/hw3ptMe30mK1asCTpauXzL7FteCG/mtQX/4e/z8njhzqtIS0lhyF+nce5J7Xnkp33L1hn16lwa1asLQHFJlGHPv8PvB1/ICRkt2VX4Bakp4fgbI6xtXBHfMvuW91C33foLVq5cQ5PGjYOOEldxcTFD7xnB4txlNGrUkAXzZ/DOrDletHN5NGwUh5k9VxNBKqtnj26sW7eRDRs2UVRURHb2VAb07xt/wwD5ltm3vBDezOu37eT7HVpTv04aqSkRuh+Xzuyl68ved84xM3cd/bp3BmDeqk10Tv8OJ2S0BKBZw3qkRMLReQlrG1fEt8y+5f1SRkZbLrm4N88+OynoKJWydet2FucuA2DfvkJWrlxDRnqbgFNJVVT4rWhmr33tMQ248svXtZTxMOkZbfgkf0vZ6/zNBaSH/KDzLbNveSG8mTu1acFH67awq/AL9h8sYm7ev9m2a1/Z+4vWF/CdxvVp36oZAP/evhszuOXp17j20SmMn7UoqOhHCGsbV8S3zL7l/dJjo0Zw3/2/Jxr17wY77dtncuop32P+gsVBR/lGnLOEP8Is3rBRJpAHPAM4wIAsYFQN5yqX2ZEN6kI+OOdbZt/yQngzH9umBT/rfRq/enoqDeqkcXxGy8MqKTM+Wk2/0zqXvS6JRlm8voAX77yaenVSufnJqXRt14rTj28XRPzDhLWNK+JbZt/yAlx6yYVs3/4ZixZ/zPnnnRl0nCpp2LAB2VPGcufdw9m7d1/8DSQ04tWjs4CPgGHAbufce8B+59z7zrn3y9vIzG4ysxwzy4lGCxOXFticX0C7zPSy15kZbSko2JbQfSSab5l9ywvhznzFGV2ZfPc1PHvblTRpUJfvtmoKlM5vmbV0PX27fdV5ad2sEd2Py6B5o/rUr5PGOV3bsyL/06CiHybMbVwe3zL7lhfgrLOy6H9ZH9au/pAXX3iKCy44m4kTngg6Vlypqam8PGUskyb9g1dfnR50nG/MRRP/CLMKOy/Ouahz7k/Az4BhZvZnKjHJ1zk3xjmX5ZzLikQaJihqqYU5uXTq1JEOHdqRlpbGoEEDmfb6zITuI9F8y+xbXgh35h17PwegYOdeZi9dz8WxSsv81Z/QsXVzWjdrVLbuWSe2Y03BZ+w/WERxSZSP1m3h2NYtAsn9dWFu4/L4ltm3vADDHniIDsdm0en4M7hu8K95991/csNPbws6Vlxjx4xixcq1PD56TNBREiLqLOGPMKvU2UbOuXzgajO7FNhTs5EqVlJSwu13PMCbb7xESiTChIlTyMtbHWSkuHzL7FteCHfmu8bPYHfsrKH7rzqPJg3qATBj0drDhowAmjSox/W9TuW6x17GMM7p2p7zTuoQQOojhbmNy+NbZt/y+urss3pw/eCrWPpxHjkLSzuHDz74ENNnzA44mXYvsAIAACAASURBVFSW1fR4amqdjHAP2IpUwt6p9wYdocoaD3w46AgiSav44OZaLV2sOvHihP+uPWHl9NCWX8JxDqaIiIh4x8yamdkrZrbSzFaY2Zlm1sLM3jazNbH/Nj9k/fvNbK2ZrTKzal8HQJ0XERERz7moJfxRSaOBGc65E4FTgBXAfcAs51xnYFbsNWbWFbgWOAnoBzxlZinV+bzqvIiIiEiVmVkT4DxgHIBz7qBzbhcwEPjyPhETgctjzwcCk51zB5xzG4C1QM/q7FudFxEREc8FdG+jY4FPgfFmttjMnjGzhkBr51xBaS5XABwTWz8D+OSQ7fNjy6pMnRcRERHP1cSw0aHXbIs9bvrablOB04CnnXPdgEJiQ0TlONpYVLUmGnt3Y0YRERGpec65MUBFF8LJB/Kdc/Njr1+htPOyzczaOucKzKwtsP2Q9Q+9XHgmsIVqUOVFRETEc0FcpM45txX4xMxOiC3qTekthV4DbogtuwGYGnv+GnCtmdU1s45AZ2BBdT6vKi8iIiJSXbcCL5pZHWA9pVfkjwDZZnYjsAm4GsA5t9zMsint4BQDQ5xzJdXZqTovIiIingvqLtDOuVxK74P4db3LWX8kMPKb7ledFxEREc+F/ObjCac5LyIiIuIVVV5EREQ8F/a7QCeaKi8iIiLiFVVeREREPBfUhN2gqPMiIiLiOU3YFREREQkxVV5EREQ8l2wTdtV5kUD49mPWeODDQUeosj1/7B90hCprMnRa0BFExAPqvIiIiHgu2Sbsas6LiIiIeEWVFxEREc9pzouIiIh4JcnOlNawkYiIiPhFlRcRERHPJduwkSovIiIi4hVVXkRERDyXbKdKq/MiIiLiuWjQAWqZho1ERETEK6q8iIiIeM55d9OVb0aVFxEREfGKKi8iIiKeiybZVerUeREREfFcVMNGIiIiIuGlyouIiIjnNGE35MaOGcWW/CXkLp4VdJQq6dunF8uXzWFl3lzuGTok6Dhx+ZYX4Pbbfklu7mwWL57F888/Sd26dYOOFFdY2zm1W2/qDf4t9a57kDr9boSUVOpc/Avq/XhY6eNnI6n342GHbWONm1P/lsdJPe2igFIfqW7dusz75+t8lPM2S3JnM/y3dwUdKS7fvuN8bOPMzHTemfkyHy99jyW5s7n1NzcGHUmqyLvOy3PPZXPpZdcFHaNKIpEIT4weyWX9B3PyKRdwzTWX06VL56Bjlcu3vADp6W0YMuTnnHHGJXTr1puUlBSuGTQw6FgVCms7W8NmpJ5yAV9M+l++ePF/wCKkHN+Dg9Of4YuXRvLFSyMpWbuI4rWLD9su7byrKfn38oBSH92BAwe4sM8gumddRPesPvTt04vTe54WdKwK+fYd52MbFxcXM/SeEZz8/V6cfU5/brnlp6H42fsmojXwCDPvOi8fzJ3Pjp27go5RJT17dGPduo1s2LCJoqIisrOnMqB/36Bjlcu3vF9KTU2lfv16pKSk0KB+fbYUbA06UoVC3c6RCKSmgUWwtDRc4eE/cymdu1OyOuer18eegtv9Ge4/BbWdNK7Cws8BSEtLJTUtDefCfVqGj99xvrXx1q3bWZy7DIB9+wpZuXINGeltAk4lVVGlzouZnWNmd5pZn5oK9G2UntGGT/K3lL3O31xAeoh/UHzLC7Bly1b+9Ke/sH7dAj7ZtJg9e/bwzjtzgo5VobC2syvcRfGid6j/8z9Q/xcP4w58QXTTirL3I+mdcJ/vxe3aXrogtQ6pWX0pmv9GQIkrFolEyFk4k4LNS5k1aw4LFi6Ov5FUic9t3L59Jqee8j3mL/An89E4LOGPMKuw82JmCw55/kvgz0BjYLiZ3VfD2b41zI48CML8l4lveQGaNWtK//596Xz8GXy3/Wk0aNiAH//4yqBjVSi07Vy3ASnHfp/9Ex5g/7h7Ia0OKSf0LHs75YQelKxaWPY67Yz+FC+eBUUHgkgbVzQaJatHH9p3zKJHVjdOOumEoCN96/jaxg0bNiB7yljuvHs4e/fuCzrON6Jho8OlHfL8JuAi59wIoA9Q7qCsmd1kZjlmlhONFiYgpt825xfQLjO97HVmRlsKCrYFmKhivuUF6N37XDZu3MRnn+2guLiYV1+dzplnZAUdq0JhbeeUdifi9vwH9u+DaJSStYuJpB9X+qZFSO3UjZI1Xw0ZRdp0IO2cK6n3s5GkdvsBaT36kfr9XsGEr8Du3Xt4f86/6NunV9BRvrV8auPU1FRenjKWSZP+wauvTg86jlRRvM5LxMyam9l3AHPOfQrgnCsEisvbyDk3xjmX5ZzLikQaJjCunxbm5NKpU0c6dGhHWloagwYNZNrrM4OOVS7f8gJ8smkzPU8/jfr16wHwgwvOYeXKNQGnqlhY29nt3UGkTcfSOS/EOjM7SueyRL57ItEdW3H7vpqTceCVUXwxfhhfjB9G8eLZFC2cQfHS94KIfoSWLVvQtGkTAOrVq0fvH5zLqlXrAk717eJrG48dM4oVK9fy+OgxQUdJiGSrvMS7zktT4CPAAGdmbZxzW82sUWxZrXvh+Sc5/7wzadmyBRvX5zDid48yfsLkIKJUWklJCbff8QBvvvESKZEIEyZOIS9vddCxyuVbXoAFCxfz97+/wYIFb1FcXMyS3OWMfebFoGNVKKztHN22kZK1i6j3o2EQLSH66ScUL5sLQOrxPShZvTDOvxAebdu25tlxj5OSEiESifDKK9N44813go5VId++43xs47PP6sH1g69i6cd55Cws/YPhwQcfYvqM2QEnk8qy6oyxm1kDoLVzbkO8dVPrZIRgEF/CJtxTwY7k40G854/9g45QZU2GTgs6gkhCFB/cXKtfc2+0/lHCv6Yu3TYptF/V1brCrnPucyBux0VERERqXjS03Yya4d11XkRERCS56d5GIiIintNdpUVERERCTJUXERERz/l4UsE3oc6LiIiI58J+XZZE07CRiIiIeEWVFxEREc9Fj3KvtG8zVV5ERETEK6q8iIiIeC7ZJuyq8iIiIiJeUeVFRETEc8l2tpE6LyIiIp7TvY1EREREQkyVFxEREc/p3kYiIiIiIabKi4iIiOeS7VTpGu+8+FbISrYDIChq55rXZOi0oCNU2d7JQ4KOUCWNr30y6AgigCbsioiIiISaho1EREQ8l2zXeVHlRURERLyiyouIiIjnkm0eoTovIiIintOEXREREZEQU+VFRETEc5qwKyIiIhJiqryIiIh4TpUXERERkRBT5UVERMRzLsnONlLnRURExHMaNhIREREJMVVeREREPKfKi4iIiEiIqfIiIiLiuWS7t5EqLyIiIp6LWuIflWVmKWa22Mxej71uYWZvm9ma2H+bH7Lu/Wa21sxWmVnf6n5eLzsva1Z/yOJF75CzcCYfznsz6Dhx1a1bl3n/fJ2Pct5mSe5shv/2rqAjxdW3Ty+WL5vDyry53DN0SNBxKsW3zGPHjGJL/hJyF88KOkqlhbWNX/xnHj98fCpX/ulVXpibd9h7E+cs49T7J7Kz8IuyZasLdvCTp97kyj+9ylWPT+VAUUltRy5XWNu4PD5+v4F/7RxytwMrDnl9HzDLOdcZmBV7jZl1Ba4FTgL6AU+ZWUp1duhl5wXgwouuJqtHH84485Kgo8R14MABLuwziO5ZF9E9qw99+/Ti9J6nBR2rXJFIhCdGj+Sy/oM5+ZQLuOaay+nSpXPQsSrkY+bnnsvm0suuCzpGpYW1jddu3cnfF67hhV9fSvZtA/hgZT7//mwPAFt3FfLh2gLaNmtYtn5xSZRh2XMZdsUZ/P2/LueZX/YlNSUcF8kIaxtXxLfvN/CzneOJ1sCjMswsE7gUeOaQxQOBibHnE4HLD1k+2Tl3wDm3AVgL9KziRwXidF7M7HQzaxJ7Xt/MRpjZNDN72MyaVmeHyaqw8HMA0tJSSU1Lw7nwjlD27NGNdes2smHDJoqKisjOnsqA/tWu7tUKHzN/MHc+O3buCjpGpYW1jdd/upvvt2tF/TqppKZE6N6xNbOXbwLg0TcWcsfF3Q9bf96aLXRu05wT2rYAoFnDeqREwvF3XFjbOB6fvt/A33YOqceBezi8v9PaOVcAEPvvMbHlGcAnh6yXH1tWZfF+Yp8FPo89Hw00BR6OLRtfnR0mgnOO6W9OYv6H0/nFjX785RqJRMhZOJOCzUuZNWsOCxYuDjpSudIz2vBJ/pay1/mbC0hPbxNgovh8zOybsLZxp9bN+GjDNnYVfsH+g8XMXbWZbbsLeS9vE62aNCjrpHzp35/twYBbnn2ba/9vGuPfXxZM8KMIaxvH49P3G/jbzhWpicqLmd1kZjmHPG46dJ9mdhmw3Tn3USVjHq3EWa2ebryzjSLOueLY8yzn3Je1wLlmlludHSbC+b0up6BgG61afYcZ0yezctVa5s6dH1ScSolGo2T16EPTpk3428vjOOmkE1i+fFXQsY7K7MjjK+x/SfmY2TdhbeNjj2nGz87/Hr969m0a1Enl+LbNSYkYz7z7MU/feNER65dEoyz+93ZeHHIp9dJSufmZmXTN+A6nd2obQPrDhbWN4/Hp+w38beeK1ER659wYYEwFq5wNDDCzS4B6QBMzewHYZmZtnXMFZtYW2B5bPx9od8j2mcAWqiFe5WWZmf0s9nyJmWUBmNnxQFF5Gx3aW4tGC6uTq0IFBdsA+PTT//Dq1On06HFqwvdRU3bv3sP7c/5F3z69go5Srs35BbTLTC97nZnRtqzNw8rHzL4Jcxtf0aMzk2/tz7M3X0yTBnVJb96IzTv3MWj0a1z88Cts3/M5P/q/1/ls735aN21I946tad6wHvXrpHLOCRms2PKfoD8CEO42rgwfvt/A/3YOC+fc/c65TOdcB0on4s52zg0GXgNuiK12AzA19vw14Fozq2tmHYHOwILq7Dte5+UXwPlmtg7oCswzs/XA2Nh75X2gMc65LOdcViTSsLzVqqVBg/o0atSw7PlFF54f6h4+QMuWLWjatAkA9erVo/cPzmXVqnUBpyrfwpxcOnXqSIcO7UhLS2PQoIFMe31m0LEq5GNm34S5jXfs2w9Awa59zF7+b/qfdhzvPnAN0++9iun3XsUxTRow6dbLaNm4Pmcdn86arTvZf7CY4pIoH23YxrHHNAv4E5QKcxuXx7fvN/CzneMJ8lTpo3gIuMjM1gAXxV7jnFsOZAN5wAxgiHOuWqf6VThs5JzbDfzUzBoDx8bWz3fOBdZFbd26Fa+8PA6AlNQUJk9+lZkz3wsqTqW0bduaZ8c9TkpKhEgkwiuvTOONN98JOla5SkpKuP2OB3jzjZdIiUSYMHEKeXmrg45VIR8zv/D8k5x/3pm0bNmCjetzGPG7Rxk/YXLQscoV5ja+68X32P35AVIjEe4fcAZN6tctd90m9ety/Tldue7J1zEzzjkhg/NOzKzFtOULcxuXx7fvN/CzncPOOfce8F7s+X+A3uWsNxIY+U33ZzU9zpdWJ8OrgUSvwop8y+yd7Nf1Nhpf+2TQESSkig9urtXz7x9qPzjhv77u+/cL4biGwFGE4/xAERERkUrSvY1EREQ8l2yjBuq8iIiIeC6aZN0XDRuJiIiIV1R5ERER8Vxl70X0baHKi4iIiHhFlRcRERHPJdeMF3VeREREvKdhIxEREZEQU+VFRETEc9/wXkTeUeVFREREvKLKi4iIiOeS7SJ16ryIiIh4Lrm6Lho2EhEREc+o8iIiIuI5nSotIiIiEmKqvIiIiHhOE3YTLLmaU0S+icbXPhl0hCrZ9/6jQUeoskbn3x10BKkByfa7VsNGIiIi4hUNG4mIiHhOE3ZFREREQkyVFxEREc8l24RdVV5ERETEK6q8iIiIeC656i7qvIiIiHhPE3ZFREREQkyVFxEREc+5JBs4UuVFREREvKLKi4iIiOeSbc6LOi8iIiKe03VeREREREJMlRcRERHPJVfdRZUXERER8YwqLyIiIp7TnBcP9O3Ti+XL5rAyby73DB0SdJy4MjPTeWfmy3y89D2W5M7m1t/cGHSkSolEIixc8BZT/zEx6ChxjR0zii35S8hdPCvoKFXi27HsW14IT+bfjnuVXrc+wpXDnixbtnvf59z8x+fof+8T3PzH59hTuB+AXfs+58aHJnDGzSP5w/NvHPbvTP/wY374wFNc9cBT3PLo8+zcW1irn+NowtLGlVW3bl3m/fN1Psp5myW5sxn+27uCjvSNRWvgEWbedV4ikQhPjB7JZf0Hc/IpF3DNNZfTpUvnoGNVqLi4mKH3jODk7/fi7HP6c8stPw19ZoDbbv0FK1euCTpGpTz3XDaXXnZd0DGqxLdj2be8EK7MA885lafvGnzYsmffmEvPLh2Z9vBt9OzSkXFvzAWgTloqQ668gDuv6XPY+sUlJTz84nSeufcGXvn9rzm+XWsmv7Og1j7D0YSpjSvrwIEDXNhnEN2zLqJ7Vh/69unF6T1PCzqWVIF3nZeePbqxbt1GNmzYRFFREdnZUxnQv2/QsSq0det2FucuA2DfvkJWrlxDRnqbgFNVLCOjLZdc3Jtnn50UdJRK+WDufHbs3BV0jCrx7Vj2LS+EK3P3EzrQpGH9w5a9u3gVA845FYAB55zKu4tWAtCgbh1OO749ddMOH9l3sZGB/QeKcM6xb/8BWjVvXPPhKxCmNq6KwsLPAUhLSyU1LQ3n/B52cTXwvzCrsPNiZreZWbvaClMZ6Rlt+CR/S9nr/M0FpIe8I3Co9u0zOfWU7zF/weKgo1TosVEjuO/+3xONhr146C/fjmXf8kL4M+/YvY9WzUo7H62aNWbHnoqHgNJSUxj2k0u56oGnuPCOUazf8ilXnBdsxSDsbVyeSCRCzsKZFGxeyqxZc1iwMNzfyXK4eJWX/wHmm9kHZvZrM2tVG6EqYmZHLPOlx9ywYQOyp4zlzruHs3fvvqDjlOvSSy5k+/bPWLT446CjfKv5diz7lhf8zFyRouISsmfnMOV3v+Kdx++ic7vWjHv9g0Az+drG0WiUrB59aN8xix5Z3TjppBOCjvSNaM7L4dYDmZR2YroDeWY2w8xuMLNya5VmdpOZ5ZhZTjSa2Mlkm/MLaJeZXvY6M6MtBQXbErqPmpCamsrLU8YyadI/ePXV6UHHqdBZZ2XR/7I+rF39IS++8BQXXHA2Eyc8EXSsbx3fjmXf8kL4M7do2ohPd+0F4NNde2nRpGGF66/atBWAdse0wMzo2/Mklqz9pMZzViTsbRzP7t17eH/Ov+jbp1fQUaQK4nVenHMu6pyb6Zy7EUgHngL6UdqxKW+jMc65LOdcViRS8Q9jVS3MyaVTp4506NCOtLQ0Bg0ayLTXZyZ0HzVh7JhRrFi5lsdHjwk6SlzDHniIDsdm0en4M7hu8K95991/csNPbws61reOb8eyb3kh/Jl7nXoCr83NBeC1ublc0K3iv/6Pad6Y9Vs+LRtemrdsPR3bBlsQD3sbH03Lli1o2rQJAPXq1aP3D85l1ap1Aaf6ZpJtzku867wcVg90zhUBrwGvmVn9o29Ss0pKSrj9jgd4842XSIlEmDBxCnl5q4OIUmlnn9WD6wdfxdKP88hZWPpD/eCDDzF9xuyAk317vPD8k5x/3pm0bNmCjetzGPG7Rxk/YXLQsSrk27HsW14IV+Z7n36FnJUb2bXvcy76r1HccvkF/Pyycxj65Mu8+sFi2rRoyqNDri5b/+K7/sS+Lw5QVFzCu4tW8pe7r+e4jGO4eeD5/Px/x5OaEqHtd5rxP7+8PJDP86UwtXFltW3bmmfHPU5KSoRIJMIrr0zjjTffCTrWNxL2YZ5Es4rGJs3seOfcNzoKU+tkhLv7JiJSTfvefzToCFXW6Py7g46QFIoPbj5yMlANuqHDDxP+u3bixr/V6meoigorL9+04yIiIiI1L+rBJOlE8u46LyIiIpLcdG8jERERzyVX3UWdFxEREe/pxowiIiIiIabKi4iIiOfCfl2WRFPlRURERLyiyouIiIjnku0ideq8iIiIeE4TdkVERERCTJUXERERz2nCroiIiEiIqfIiIiLiuWSbsKvKi4iIiHhFlRcRERHPuSS7q7Q6LyIiIp7TqdIiIiIiIabKi4hINTU6/+6gI1TZ3qn3Bh2hShoPfDjoCF7QhF0RERGREFPlRURExHPJdpE6dV5EREQ8pwm7IiIiIiGmyouIiIjnku06L6q8iIiIiFfUeREREfFctAYe8ZhZOzN718xWmNlyM7s9tryFmb1tZmti/21+yDb3m9laM1tlZn2r+3nVeREREfGcq4H/VUIxcJdzrgtwBjDEzLoC9wGznHOdgVmx18TeuxY4CegHPGVmKdX5vOq8iIiISJU55wqcc4tiz/cCK4AMYCAwMbbaRODy2POBwGTn3AHn3AZgLdCzOvvWhF0RERHPBX2qtJl1ALoB84HWzrkCKO3gmNkxsdUygA8P2Sw/tqzKVHkRERGRI5jZTWaWc8jjpnLWawT8DbjDObenon/yKMuq1etS5UVERMRzNXGqtHNuDDCmonXMLI3SjsuLzrm/xxZvM7O2sapLW2B7bHk+0O6QzTOBLdXJpsqLiIiIVJmZGTAOWOGce+yQt14Dbog9vwGYesjya82srpl1BDoDC6qzb1VeREREPBfQnJezgeuBj80sN7bsv4GHgGwzuxHYBFwN4JxbbmbZQB6lZyoNcc6VVGfH6ryIiIh4LogbMzrn5nL0eSwAvcvZZiQw8pvuW8NGIiIi4hVVXkRERDwX1b2Nwq9vn14sXzaHlXlzuWfokKDjVIpvmX3LO3bMKLbkLyF38aygo1SJb+3sW17wL3OY8774/hJ++NAkrnzoJV54bwkA90x4i0GPTGbQI5O5eMRzDHpkMgDzVn3Cjx7N5qqHJ/GjR7NZsDo/yOiH8fX7Qr5iNX0nytQ6GQndQSQSYcXyD+h3yY/Izy/gw3lvMvj6X7NixZpE7iahfMvsW16Ac885nX37Chk/fjSndjvqUGvo+NbOvuUF/zLXRt69U++t1nZrC/7DvRNn8sKdV5GWksKQv07jv68+n/atmpWtM+rVuTSqV5eb+/VgZf6ntGjcgGOaNmRtwX+45S/TeHvET6u838YDH65W3orUxvdF8cHN5c0FqRHnZvRO+C/zDzbPqtXPUBUVVl7MrI6Z/cTMLoy9/rGZ/dnMhsTO7a51PXt0Y926jWzYsImioiKys6cyoH+17+1UK3zL7FtegA/mzmfHzl1Bx6gS39rZt7zgX+Yw512/bSff79Ca+nXSSE2J0P24dGYvXV/2vnOOmbnr6Ne9MwAnZrbimKYNATiuTQsOFhVzsLhaJ5YknI/fF/FEcQl/hFm8YaPxwKXA7Wb2PKWnO80HegDP1HC2o0rPaMMn+V9d0yZ/cwHp6W2CiFJpvmX2La+vfGtn3/KCf5nDnLdTmxZ8tG4Luwq/YP/BIubm/Zttu/aVvb9ofQHfaVz/sErMl95Zso4TM1tRJ7Va9+ATOUK8CbsnO+e+b2apwGYg3TlXYmYvAEtqPt6RSq+Jc7iaHvr6pnzL7FteX/nWzr7lBf8yhznvsW1a8LPep/Grp6fSoE4ax2e0JCXy1d+/Mz5aTb/TOh+x3dqC/zB62jyevmVAbcZNOmGvlCRavM5LxMzqAA2BBkBTYAdQFyh32Ch2/4ObACylKZFIw8SkBTbnF9AuM73sdWZGWwoKtiXs368JvmX2La+vfGtn3/KCf5nDnveKM7pyxRldAXji9Xm0btYIgOKSKLOWrmfS3YMOW3/brn3c+ex0/ue6C2nXsmmt55Vvr3jDRuOAlUAuMAx42czGAguByeVt5Jwb45zLcs5lJbLjArAwJ5dOnTrSoUM70tLSGDRoINNen5nQfSSab5l9y+sr39rZt7zgX+aw592x93MACnbuZfbS9Vwcq7TMX/0JHVs3L+vMAOz5/AC3jnmd2y47k27Htg0kbzJxziX8EWYVVl6cc38ysymx51vM7DngQmCsc65a9yP4pkpKSrj9jgd4842XSIlEmDBxCnl5q4OIUmm+ZfYtL8ALzz/J+eedScuWLdi4PocRv3uU8RPK7V+Hgm/t7Fte8C9z2PPeNX4Guwu/IDUlwv1XnUeTBvUAmLFo7RFDRlPmfsymz3Yz5q2FjHlrIQB/uWUALRo3qPXcX+fj90U8yTZs5N2p0iIiUn3VPVU6KDVxqnRtqO1TpXumn5/w37ULtrwf2lOldYVdERERzwVxb6MgeXmFXREREUleqryIiIh4LuwTbBNNlRcRERHxiiovIiIinku2s43UeREREfGcho1EREREQkyVFxEREc8l27CRKi8iIiLiFVVeREREPJdsF6lT50VERMRzUU3YFREREQkvVV5EREQ8l2zDRqq8iIiIiFdUeREREfFcss15UedFRETEcxo2EhEREQkxVV5ERJJI44EPBx2hSva+MzLoCF5ItmEjVV5ERETEK6q8iIiIeE5zXkRERERCTJUXERERzyXbnBd1XkRERDynYSMRERGREFPlRURExHPORYOOUKtUeRERERGvqPIiIiLiuWiSzXlR50VERMRzLsnONtKwkYiIiHhFlRcRERHPJduwkSovIiIi4hVVXkRERDyXbHNe1HkRERHxXLLdHkDDRiIiIuIVLzsvffv0YvmyOazMm8s9Q4cEHadSfMvsW17wL3PdunWZ98/X+SjnbZbkzmb4b+8KOlJcvrUx+Jd57JhRbMlfQu7iWUFHqbQwtfHwCW9wwZ2j+eHwsWXLdhfu5+bHJtF/2F+4+bFJ7CncX/beuDf/Rf//fpqBD/yVfy1bX7b8xj++yMAH/sqgEeMYNGIcO/YU1urnqCpXA/8LM+86L5FIhCdGj+Sy/oM5+ZQLuOaay+nSpXPQsSrkW2bf8oKfmQ8cOMCFfQbRPesiumf1oW+fXpze87SgY5XLxzb2MfNzz2Vz6WXXBR2j0sLWxgPOOpmnbr/msGXPTp/H6V06MG3kkTmkYQAACexJREFUrzi9Sweenf4hAOu2fMZbC1fwtxG/5Knbr+EPL71FSfSry+z/4RcDyB5+I9nDb6RFk4a1+jmkYt51Xnr26Ma6dRvZsGETRUVFZGdPZUD/vkHHqpBvmX3LC35mBigs/ByAtLRUUtPSQj3pzsc29jHzB3Pns2PnrqBjVFrY2rj78d+lScN6hy17L3cN/c88GYD+Z57Mu7mrY8tX07dHF+qkpZLRqhntWjVn2YYttZ45EZxzCX+EWdzOi5kdZ2Z3m9loMxtlZr8ys6a1Ee5o0jPa8En+VwdX/uYC0tPbBBWnUnzL7Fte8DMzlP7VmrNwJgWblzJr1hwWLFwcdKRy+djGPmb2jQ9t/J89hbRq1giAVs0asWNv6R8N23ftpU2LJmXrtW7emO279pW9Hj7hDQaNGMeY1+eG/pd5sqmw82JmtwF/AeoBPYD6QDtgnpn1qvF0R890xLKwH1S+ZfYtL/iZGSAajZLVow/tO2bRI6sbJ510QtCRyuVjG/uY2Tc+t/HRYn75af7wiwG88v9+wfh7B7NozSe8Pm9ZrWarqigu4Y8wi1d5+SXQzzn3e+BCoKtzbhjQD/hTeRuZ2U1mlmNmOdFoYic5bc4voF1metnrzIy2FBRsS+g+Es23zL7lBT8zH2r37j28P+df9O3TK+go5fKxjX3M7Bsf2vg7TRryaayi8umufbRo3AAorbRs3bGnbL1tO/f+//buLkbK8gzj+P+aZZWPgopQA0gDDcSqJEVCUGtcjRQircHKgdGmTdoe0FrbQJvUaNPE2KQHpqbhoKYpBYVWBS2IESWKpl+aiOVDKN+IlCJoCw1QoSWFXe4ezKvZ8rEzu92d53061y+Z7M5kd99r3+zO3Pvcz3vvRys0l10yGIBB/S9kxpSr2bK33O0kt43O9uEsmAuBwQARsQ9oPd8nRMT8iJgcEZMrld7d5LR23UbGjRvLmDGjaW1t5c47b2flC6t79Ri9LbfMueWFPDMPGzaUiy6qLln379+fqbfcyM6d7yROdX45nuMcM+cmh3N806fHs/KNzQCsfGMzN08c/9HjL6/dzslT7Rw4dJR9B48wYexI2jtOc6RoLZ1q7+C1P+1m3MjhyfLb2WoNqVsArJW0BmgDHgaQNBw43MfZzqmjo4M5c3/AqhefoqVSYdHip9m2bVeKKHXLLXNueSHPzCNGXMZjC+fR0lKhUqmwbNlKXlz1aupY55XjOc4x8xO/epSb2q5n2LCh7N2zjod++AiPL1qaOtZ5le0c3z//Odbt2sfR4yeY/r2fcs/MG/najOu47+fPseL1TYwYOoQff+MOAMaNGs60yZ9i1oO/oKVS4YEvTqelUuHEv0/yzXlP097RQcfp4NqrxjCrbWKy76kezTakTrWWhiRdDVwJbImIHd09QL8LRjXXGTUzs15z7NUfpY7QIwPavnL2ZqA+NHTw+F5/rT187O2Gfg/dUfPfA0TEVmBrA7KYmZlZD5R9j0pv8/82MjMzy1zZrw7qbdkNqTMzM7Pm5pUXMzOzzDVb28grL2ZmZpYVr7yYmZllrtkulXbxYmZmlrnwhl0zMzOz8vLKi5mZWeaarW3klRczMzPLildezMzMMudLpc3MzMxKzCsvZmZmmWu2q41cvJiZmWXObSMzMzOzOki6VdJOSbsl3d+o43rlxczMLHMpVl4ktQCPAtOA/cBaSc9HxLa+PrZXXszMzKwnpgC7I2JPRJwElgK3N+LALl7MzMwyF31wq8Mo4N1O9/cXj/W5Pm8btZ88oL762pJmR8T8vvr6vS23vJBf5tzygjM3Qm55wZkbIbe8XemL11pJs4HZnR6af8b5OtcxG9K/yn3lZXbtDymV3PJCfplzywvO3Ai55QVnboTc8jZURMyPiMmdbmcWevuB0Z3uXw6814hsuRcvZmZmlsZaYLyksZIuAO4Cnm/EgX21kZmZmXVbRLRL+hbwMtACPBYRWxtx7NyLl9x6lbnlhfwy55YXnLkRcssLztwIueUtnYhYBaxq9HHVbFP5zMzMLG/e82JmZmZZybJ4STWOuKckPSbpoKQtqbPUQ9JoSb+VtF3SVklzUmeqRVJ/SX+UtKnI/FDqTPWQ1CLpLUkvpM5SD0l7JW2WtFHSutR56iHpYknLJO0ofqavT52pK5KuKM7vh7cPJM1Nnasrkr5T/N5tkbREUv/UmWqRNKfIu7Xs59fOll3bqBhHvItO44iBuxsxjrinJLUBx4FfRsSE1HlqkTQCGBERGyQNBtYDXyj5ORYwKCKOS2oFXgfmRMSaxNG6JOm7wGRgSETcljpPLZL2ApMj4u+ps9RL0mLgtYhYUFwRMTAijqbOVY/i+e4AcG1E/CV1nnORNIrq79tVEXFC0jPAqohYlDbZ+UmaQHUa7BTgJPAScE9EvJ00mNUtx5WXZOOIeyoi/gAcTp2jXhHxfkRsKN4/BmynQVMTeyqqjhd3W4tbqStzSZcDnwcWpM7y/0rSEKANWAgQESdzKVwKU4F3ylq4dNIPGCCpHzCQBs36+B9cCayJiH9FRDvwe+COxJmsG3IsXpKNI25GksYA1wBvpk1SW9GC2QgcBF6JiLJnngfcB5xOHaQbAlgtaX0xfbPsPgkcAh4v2nMLJA1KHaob7gKWpA7RlYg4ADwC7APeB/4REavTpqppC9Am6VJJA4HP8d/D1qzkcixeko0jbjaSPgYsB+ZGxAep89QSER0RMZHqlMcpxdJwKUm6DTgYEetTZ+mmGyJiEjADuLdoiZZZP2AS8LOIuAb4J1D6fXIARYtrJvDr1Fm6IukSqqvfY4GRwCBJX0qbqmsRsR14GHiFastoE9CeNJR1S47FS7JxxM2k2DeyHHgyIp5Nnac7irbA74BbE0fpyg3AzGIPyVLgFklPpI1UW0S8V7w9CKyg2sYts/3A/k6rcMuoFjM5mAFsiIi/pQ5Sw2eBP0fEoYg4BTwLfCZxppoiYmFETIqINqptfe93yUiOxUuyccTNotj8uhDYHhE/SZ2nHpKGS7q4eH8A1SfUHWlTnV9EPBARl0fEGKo/w7+JiFL/tSppULGBm6L1Mp3q8ntpRcRfgXclXVE8NBUo7cbzM9xNyVtGhX3AdZIGFs8dU6nukys1SR8v3n4CmEUe59oK2U3YTTmOuKckLQFuBoZJ2g88GBEL06bq0g3Al4HNxR4SgO8XkxTLagSwuLg6owI8ExFZXH6ckcuAFdXXJ/oBT0XES2kj1eXbwJPFHzt7gK8mzlNTsQ9jGvD11FlqiYg3JS0DNlBtvbxFHpNrl0u6FDgF3BsRR1IHsvpld6m0mZmZNbcc20ZmZmbWxFy8mJmZWVZcvJiZmVlWXLyYmZlZVly8mJmZWVZcvJiZmVlWXLyYmZlZVly8mJmZWVb+A8Jb5kkZH081AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 720x576 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(10, 8))\n",
    "sns.heatmap(confusion_mtx, annot=True, fmt='g')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There seems to be a slightly higher confusion between (0,6) and (4,9). This is reasonable as 0's and 6's look similar with their loops and 4's and 9's can be mistaken when the 4's are more rounded and 9's are more angular."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}