23-ConvolutionalNeuralNetwork-Hoda-Keras.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"http://akhavanpour.ir/notebook/images/srttu.gif\" alt=\"SRTTU\" style=\"width: 150px;\"/>\n",
    "\n",
    "[![Azure Notebooks](https://notebooks.azure.com/launch.png)](https://notebooks.azure.com/import/gh/Alireza-Akhavan/class.vision)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <div style=\"direction:rtl;text-align:right;font-family:B Lotus, B Nazanin, Tahoma\">مقدمه ای بر شبکه‌های عصبی کانولوشنالی(Convolutionl Neural Networks - CNN)</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"direction:rtl;text-align:right;font-family:Tahoma\">\n",
    "در ابتدا معماری شبکه را مشخص میکنیم.\n",
    "<br>\n",
    "به لایه های conv و pool دقت کنید.\n",
    "<br>\n",
    "قبل از اولین لایه Dense یا Fully Connected همیشه متد Flatten فراخوانی میشود تا نورون ها به صورت یک وکتور در بیایند.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import keras\n",
    "from keras import layers\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Dropout, Activation\n",
    "import numpy as np\n",
    "from dataset import load_hoda\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.Conv2D(32, (3, 3), activation='relu',\n",
    "                        input_shape=(28, 28, 1)))\n",
    "model.add(layers.MaxPooling2D((2, 2)))\n",
    "model.add(layers.Conv2D(64, (3, 3), activation='relu'))\n",
    "model.add(layers.MaxPooling2D((2, 2)))\n",
    "model.add(layers.Conv2D(64, (3, 3), activation='relu'))\n",
    "model.add(layers.Flatten())\n",
    "model.add(layers.Dense(64, activation='relu'))\n",
    "model.add(Dropout(0.5))\n",
    "model.add(layers.Dense(10, activation='softmax'))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"direction:rtl;text-align:right;font-family:Tahoma\">\n",
    "نگاهی به تنسور وردی و خروجی هر لایه بیندازیم.\n",
    "<br>\n",
    "تصویر ورودی 28x28x3 بوده است\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv2d_4 (Conv2D)            (None, 26, 26, 32)        320       \n",
      "_________________________________________________________________\n",
      "max_pooling2d_3 (MaxPooling2 (None, 13, 13, 32)        0         \n",
      "_________________________________________________________________\n",
      "conv2d_5 (Conv2D)            (None, 11, 11, 64)        18496     \n",
      "_________________________________________________________________\n",
      "max_pooling2d_4 (MaxPooling2 (None, 5, 5, 64)          0         \n",
      "_________________________________________________________________\n",
      "conv2d_6 (Conv2D)            (None, 3, 3, 64)          36928     \n",
      "_________________________________________________________________\n",
      "flatten_2 (Flatten)          (None, 576)               0         \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 64)                36928     \n",
      "_________________________________________________________________\n",
      "dropout_2 (Dropout)          (None, 64)                0         \n",
      "_________________________________________________________________\n",
      "dense_4 (Dense)              (None, 10)                650       \n",
      "=================================================================\n",
      "Total params: 93,322\n",
      "Trainable params: 93,322\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <div style=\"direction:rtl;text-align:right;font-family:B Lotus, B Nazanin, Tahoma\">کد یک شبکه کانولوشنالی و آموزش آن از ابتدا تا انتها بر روی مجموعه داده هدی</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"direction:rtl;text-align:right;font-family:Tahoma\">\n",
    "تصاویر مجموعه داده هدی در تابعی که قبلا نوشته ایم، load_hoda به صورت flat شده و یک وکتور در آمده اند.\n",
    "<br>\n",
    "در این فراخوانی طول و عرض تصاویر 28 قرار داده شده است، پس خروجی این تابع وکتورهای 784تایی است.\n",
    "<br>\n",
    "** دقت کنید که قبل از ورودی شبکه کانولوشنالی تصویر را به شکل اصلی خود یعنی 28x28 برگردانده ایم.**\n",
    "<br>\n",
    "همچنین چون تصاویر سیاه و سفید است تعداد کانال تصویر را 1 قرار داده ایم.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 3500 samples, validate on 200 samples\n",
      "Epoch 1/200\n",
      "3500/3500 [==============================] - 4s 1ms/step - loss: 2.0580 - acc: 0.3263 - val_loss: 1.4164 - val_acc: 0.6800\n",
      "Epoch 2/200\n",
      "3500/3500 [==============================] - 3s 858us/step - loss: 1.1864 - acc: 0.5931 - val_loss: 0.6343 - val_acc: 0.7900\n",
      "Epoch 3/200\n",
      "3500/3500 [==============================] - 3s 760us/step - loss: 0.8085 - acc: 0.7169 - val_loss: 0.4461 - val_acc: 0.8200\n",
      "Epoch 4/200\n",
      "3500/3500 [==============================] - 3s 760us/step - loss: 0.6250 - acc: 0.7780 - val_loss: 0.2562 - val_acc: 0.9150\n",
      "Epoch 5/200\n",
      "3500/3500 [==============================] - 3s 754us/step - loss: 0.4894 - acc: 0.8329 - val_loss: 0.2039 - val_acc: 0.9300\n",
      "Epoch 6/200\n",
      "3500/3500 [==============================] - 3s 751us/step - loss: 0.4241 - acc: 0.8540 - val_loss: 0.1629 - val_acc: 0.9500\n",
      "Epoch 7/200\n",
      "3500/3500 [==============================] - 3s 758us/step - loss: 0.3679 - acc: 0.8771 - val_loss: 0.1269 - val_acc: 0.9650\n",
      "Epoch 8/200\n",
      "3500/3500 [==============================] - 3s 748us/step - loss: 0.3074 - acc: 0.8983 - val_loss: 0.0986 - val_acc: 0.9750\n",
      "Epoch 9/200\n",
      "3500/3500 [==============================] - 3s 746us/step - loss: 0.2953 - acc: 0.9060 - val_loss: 0.0904 - val_acc: 0.9800\n",
      "Epoch 10/200\n",
      "3500/3500 [==============================] - 3s 761us/step - loss: 0.2560 - acc: 0.9186 - val_loss: 0.0698 - val_acc: 0.9850\n",
      "Epoch 11/200\n",
      "3500/3500 [==============================] - 3s 735us/step - loss: 0.2361 - acc: 0.9220 - val_loss: 0.0765 - val_acc: 0.9700\n",
      "Epoch 12/200\n",
      "3500/3500 [==============================] - 3s 754us/step - loss: 0.2076 - acc: 0.9294 - val_loss: 0.0693 - val_acc: 0.9800\n",
      "Epoch 13/200\n",
      "3500/3500 [==============================] - 3s 745us/step - loss: 0.1971 - acc: 0.9337 - val_loss: 0.0577 - val_acc: 0.9800\n",
      "Epoch 14/200\n",
      "3500/3500 [==============================] - 3s 748us/step - loss: 0.2180 - acc: 0.9277 - val_loss: 0.0477 - val_acc: 0.9900\n",
      "Epoch 15/200\n",
      "3500/3500 [==============================] - 3s 742us/step - loss: 0.1928 - acc: 0.9363 - val_loss: 0.0425 - val_acc: 0.9900\n",
      "Epoch 16/200\n",
      "3500/3500 [==============================] - 3s 755us/step - loss: 0.1759 - acc: 0.9434 - val_loss: 0.0581 - val_acc: 0.9800\n",
      "Epoch 17/200\n",
      "3500/3500 [==============================] - 3s 741us/step - loss: 0.1528 - acc: 0.9494 - val_loss: 0.0411 - val_acc: 0.9900\n",
      "Epoch 18/200\n",
      "3500/3500 [==============================] - 3s 753us/step - loss: 0.1553 - acc: 0.9500 - val_loss: 0.0348 - val_acc: 0.9900\n",
      "Epoch 19/200\n",
      "3500/3500 [==============================] - 3s 755us/step - loss: 0.1427 - acc: 0.9557 - val_loss: 0.0313 - val_acc: 0.9900\n",
      "Epoch 20/200\n",
      "3500/3500 [==============================] - 3s 730us/step - loss: 0.1311 - acc: 0.9569 - val_loss: 0.0474 - val_acc: 0.9850\n",
      "Epoch 21/200\n",
      "3500/3500 [==============================] - 3s 749us/step - loss: 0.1273 - acc: 0.9611 - val_loss: 0.0507 - val_acc: 0.9750\n",
      "Epoch 22/200\n",
      "3500/3500 [==============================] - 3s 751us/step - loss: 0.1076 - acc: 0.9686 - val_loss: 0.0353 - val_acc: 0.9850\n",
      "Epoch 23/200\n",
      "3500/3500 [==============================] - 3s 734us/step - loss: 0.1062 - acc: 0.9643 - val_loss: 0.0379 - val_acc: 0.9900\n",
      "Epoch 24/200\n",
      "3500/3500 [==============================] - 3s 733us/step - loss: 0.1101 - acc: 0.9594 - val_loss: 0.0301 - val_acc: 0.9900\n",
      "Epoch 25/200\n",
      "3500/3500 [==============================] - 3s 752us/step - loss: 0.0897 - acc: 0.9706 - val_loss: 0.0268 - val_acc: 0.9900\n",
      "Epoch 26/200\n",
      "3500/3500 [==============================] - 3s 734us/step - loss: 0.0956 - acc: 0.9680 - val_loss: 0.0292 - val_acc: 0.9850\n",
      "Epoch 27/200\n",
      "3500/3500 [==============================] - 3s 748us/step - loss: 0.0900 - acc: 0.9697 - val_loss: 0.0346 - val_acc: 0.9900\n",
      "Epoch 28/200\n",
      "3500/3500 [==============================] - 3s 737us/step - loss: 0.0858 - acc: 0.9726 - val_loss: 0.0364 - val_acc: 0.9800\n",
      "Epoch 29/200\n",
      "3500/3500 [==============================] - 3s 747us/step - loss: 0.0869 - acc: 0.9691 - val_loss: 0.0298 - val_acc: 0.9850\n",
      "Epoch 30/200\n",
      "3500/3500 [==============================] - 3s 745us/step - loss: 0.0802 - acc: 0.9723 - val_loss: 0.0398 - val_acc: 0.9850\n",
      "Epoch 31/200\n",
      "3500/3500 [==============================] - 3s 744us/step - loss: 0.0722 - acc: 0.9757 - val_loss: 0.0413 - val_acc: 0.9800\n",
      "Epoch 32/200\n",
      "3500/3500 [==============================] - 3s 735us/step - loss: 0.0780 - acc: 0.9746 - val_loss: 0.0460 - val_acc: 0.9850\n",
      "Epoch 33/200\n",
      "3500/3500 [==============================] - 3s 742us/step - loss: 0.0728 - acc: 0.9749 - val_loss: 0.0494 - val_acc: 0.9850\n",
      "Epoch 34/200\n",
      "3500/3500 [==============================] - 3s 744us/step - loss: 0.0757 - acc: 0.9729 - val_loss: 0.0321 - val_acc: 0.9900\n",
      "Epoch 35/200\n",
      "3500/3500 [==============================] - 3s 741us/step - loss: 0.0681 - acc: 0.9783 - val_loss: 0.0477 - val_acc: 0.9800\n",
      "Epoch 36/200\n",
      "3500/3500 [==============================] - 3s 746us/step - loss: 0.0641 - acc: 0.9791 - val_loss: 0.0253 - val_acc: 0.9850\n",
      "Epoch 37/200\n",
      "3500/3500 [==============================] - 3s 749us/step - loss: 0.0563 - acc: 0.9809 - val_loss: 0.0133 - val_acc: 1.0000\n",
      "Epoch 38/200\n",
      "3500/3500 [==============================] - 3s 758us/step - loss: 0.0579 - acc: 0.9800 - val_loss: 0.0262 - val_acc: 0.9800\n",
      "Epoch 39/200\n",
      "3500/3500 [==============================] - 3s 751us/step - loss: 0.0661 - acc: 0.9783 - val_loss: 0.0358 - val_acc: 0.9850\n",
      "Epoch 40/200\n",
      "3500/3500 [==============================] - 3s 756us/step - loss: 0.0689 - acc: 0.9766 - val_loss: 0.0339 - val_acc: 0.9850\n",
      "Epoch 41/200\n",
      "3500/3500 [==============================] - 3s 752us/step - loss: 0.0605 - acc: 0.9806 - val_loss: 0.0252 - val_acc: 0.9900\n",
      "Epoch 42/200\n",
      "3500/3500 [==============================] - 3s 752us/step - loss: 0.0583 - acc: 0.9800 - val_loss: 0.0469 - val_acc: 0.9800\n",
      "Epoch 43/200\n",
      "3500/3500 [==============================] - 3s 746us/step - loss: 0.0580 - acc: 0.9803 - val_loss: 0.0570 - val_acc: 0.9650\n",
      "Epoch 44/200\n",
      "3500/3500 [==============================] - 3s 753us/step - loss: 0.0458 - acc: 0.9843 - val_loss: 0.0326 - val_acc: 0.9900\n",
      "Epoch 45/200\n",
      "3500/3500 [==============================] - 3s 742us/step - loss: 0.0448 - acc: 0.9831 - val_loss: 0.0202 - val_acc: 0.9900\n",
      "Epoch 46/200\n",
      "3500/3500 [==============================] - 3s 748us/step - loss: 0.0420 - acc: 0.9866 - val_loss: 0.0214 - val_acc: 0.9900\n",
      "Epoch 47/200\n",
      "3500/3500 [==============================] - 3s 731us/step - loss: 0.0401 - acc: 0.9871 - val_loss: 0.0223 - val_acc: 0.9900\n",
      "Epoch 48/200\n",
      "3500/3500 [==============================] - 3s 758us/step - loss: 0.0393 - acc: 0.9834 - val_loss: 0.0398 - val_acc: 0.9800\n",
      "Epoch 49/200\n",
      "3500/3500 [==============================] - 3s 747us/step - loss: 0.0448 - acc: 0.9846 - val_loss: 0.0116 - val_acc: 0.9950\n",
      "Epoch 50/200\n",
      "3500/3500 [==============================] - 3s 755us/step - loss: 0.0448 - acc: 0.9843 - val_loss: 0.0315 - val_acc: 0.9900\n",
      "Epoch 51/200\n",
      "3500/3500 [==============================] - 3s 740us/step - loss: 0.0338 - acc: 0.9883 - val_loss: 0.0206 - val_acc: 0.9900\n",
      "Epoch 52/200\n",
      "3500/3500 [==============================] - 3s 752us/step - loss: 0.0381 - acc: 0.9846 - val_loss: 0.0205 - val_acc: 0.9900\n",
      "Epoch 53/200\n",
      "3500/3500 [==============================] - 3s 735us/step - loss: 0.0390 - acc: 0.9869 - val_loss: 0.0347 - val_acc: 0.9750\n",
      "Epoch 54/200\n",
      "3500/3500 [==============================] - 3s 738us/step - loss: 0.0392 - acc: 0.9840 - val_loss: 0.0298 - val_acc: 0.9900\n",
      "Epoch 55/200\n",
      "3500/3500 [==============================] - 3s 740us/step - loss: 0.0326 - acc: 0.9871 - val_loss: 0.0276 - val_acc: 0.9850\n",
      "Epoch 56/200\n",
      "3500/3500 [==============================] - 3s 752us/step - loss: 0.0350 - acc: 0.9883 - val_loss: 0.0207 - val_acc: 0.9950\n",
      "Epoch 57/200\n",
      "3500/3500 [==============================] - 3s 743us/step - loss: 0.0401 - acc: 0.9840 - val_loss: 0.0197 - val_acc: 0.9900\n",
      "Epoch 58/200\n",
      "3500/3500 [==============================] - 3s 749us/step - loss: 0.0350 - acc: 0.9851 - val_loss: 0.0292 - val_acc: 0.9850\n",
      "Epoch 59/200\n",
      "3500/3500 [==============================] - 3s 742us/step - loss: 0.0280 - acc: 0.9900 - val_loss: 0.0243 - val_acc: 0.9900\n",
      "Epoch 60/200\n",
      "3500/3500 [==============================] - 3s 745us/step - loss: 0.0273 - acc: 0.9906 - val_loss: 0.0347 - val_acc: 0.9800\n",
      "Epoch 61/200\n",
      "3500/3500 [==============================] - 3s 750us/step - loss: 0.0257 - acc: 0.9900 - val_loss: 0.0239 - val_acc: 0.9850\n",
      "Epoch 62/200\n",
      "3500/3500 [==============================] - 3s 746us/step - loss: 0.0255 - acc: 0.9917 - val_loss: 0.0546 - val_acc: 0.9800\n",
      "Epoch 63/200\n",
      "3500/3500 [==============================] - 3s 750us/step - loss: 0.0278 - acc: 0.9920 - val_loss: 0.0203 - val_acc: 0.9900\n",
      "Epoch 64/200\n",
      "3500/3500 [==============================] - 3s 756us/step - loss: 0.0205 - acc: 0.9940 - val_loss: 0.0355 - val_acc: 0.9850\n",
      "Epoch 65/200\n",
      "3500/3500 [==============================] - 3s 737us/step - loss: 0.0209 - acc: 0.9926 - val_loss: 0.0325 - val_acc: 0.9800\n",
      "Epoch 66/200\n",
      "3500/3500 [==============================] - 3s 748us/step - loss: 0.0245 - acc: 0.9906 - val_loss: 0.0268 - val_acc: 0.9850\n",
      "Epoch 67/200\n",
      "3500/3500 [==============================] - 3s 752us/step - loss: 0.0241 - acc: 0.9909 - val_loss: 0.0255 - val_acc: 0.9850\n",
      "Epoch 68/200\n",
      "3500/3500 [==============================] - 3s 736us/step - loss: 0.0195 - acc: 0.9949 - val_loss: 0.0599 - val_acc: 0.9800\n",
      "Epoch 69/200\n",
      "3500/3500 [==============================] - 3s 727us/step - loss: 0.0214 - acc: 0.9917 - val_loss: 0.0128 - val_acc: 0.9950\n",
      "Epoch 70/200\n",
      "3500/3500 [==============================] - 3s 744us/step - loss: 0.0266 - acc: 0.9914 - val_loss: 0.0432 - val_acc: 0.9800\n",
      "Epoch 71/200\n",
      "3500/3500 [==============================] - 3s 750us/step - loss: 0.0259 - acc: 0.9903 - val_loss: 0.0237 - val_acc: 0.9900\n",
      "Epoch 72/200\n",
      "3500/3500 [==============================] - 3s 748us/step - loss: 0.0215 - acc: 0.9920 - val_loss: 0.0177 - val_acc: 0.9900\n",
      "Epoch 73/200\n",
      "3500/3500 [==============================] - 3s 759us/step - loss: 0.0213 - acc: 0.9931 - val_loss: 0.0257 - val_acc: 0.9900\n",
      "Epoch 74/200\n",
      "3500/3500 [==============================] - 3s 735us/step - loss: 0.0234 - acc: 0.9909 - val_loss: 0.0297 - val_acc: 0.9900\n",
      "Epoch 75/200\n",
      "3500/3500 [==============================] - 3s 751us/step - loss: 0.0248 - acc: 0.9897 - val_loss: 0.0292 - val_acc: 0.9800\n",
      "Epoch 76/200\n",
      "3500/3500 [==============================] - 3s 742us/step - loss: 0.0244 - acc: 0.9920 - val_loss: 0.0269 - val_acc: 0.9850\n",
      "Epoch 77/200\n",
      "3500/3500 [==============================] - 3s 739us/step - loss: 0.0252 - acc: 0.9911 - val_loss: 0.0240 - val_acc: 0.9950\n",
      "Epoch 78/200\n",
      "3500/3500 [==============================] - 3s 740us/step - loss: 0.0226 - acc: 0.9926 - val_loss: 0.0249 - val_acc: 0.9850\n",
      "Epoch 79/200\n",
      "3500/3500 [==============================] - 3s 740us/step - loss: 0.0207 - acc: 0.9931 - val_loss: 0.0166 - val_acc: 0.9900\n",
      "Epoch 80/200\n",
      "3500/3500 [==============================] - 3s 741us/step - loss: 0.0177 - acc: 0.9943 - val_loss: 0.0382 - val_acc: 0.9800\n",
      "Epoch 81/200\n",
      "3500/3500 [==============================] - 3s 728us/step - loss: 0.0209 - acc: 0.9929 - val_loss: 0.0212 - val_acc: 0.9850\n",
      "Epoch 82/200\n",
      "3500/3500 [==============================] - 3s 749us/step - loss: 0.0138 - acc: 0.9960 - val_loss: 0.0240 - val_acc: 0.9850\n",
      "Epoch 83/200\n",
      "3500/3500 [==============================] - 3s 739us/step - loss: 0.0200 - acc: 0.9917 - val_loss: 0.0230 - val_acc: 0.9900\n",
      "Epoch 84/200\n",
      "3500/3500 [==============================] - 3s 749us/step - loss: 0.0201 - acc: 0.9954 - val_loss: 0.0233 - val_acc: 0.9900\n",
      "Epoch 85/200\n",
      "3500/3500 [==============================] - 3s 746us/step - loss: 0.0136 - acc: 0.9969 - val_loss: 0.0285 - val_acc: 0.9850\n",
      "Epoch 86/200\n",
      "3500/3500 [==============================] - 3s 757us/step - loss: 0.0130 - acc: 0.9960 - val_loss: 0.0172 - val_acc: 0.9950\n",
      "Epoch 87/200\n",
      "3500/3500 [==============================] - 3s 741us/step - loss: 0.0112 - acc: 0.9954 - val_loss: 0.0240 - val_acc: 0.9850\n",
      "Epoch 88/200\n",
      "3500/3500 [==============================] - 3s 749us/step - loss: 0.0164 - acc: 0.9937 - val_loss: 0.0252 - val_acc: 0.9900\n",
      "Epoch 89/200\n",
      "3500/3500 [==============================] - 3s 757us/step - loss: 0.0169 - acc: 0.9937 - val_loss: 0.0408 - val_acc: 0.9850\n",
      "Epoch 90/200\n",
      "3500/3500 [==============================] - 3s 763us/step - loss: 0.0160 - acc: 0.9940 - val_loss: 0.0555 - val_acc: 0.9800\n",
      "Epoch 91/200\n",
      "3500/3500 [==============================] - 3s 746us/step - loss: 0.0193 - acc: 0.9923 - val_loss: 0.0484 - val_acc: 0.9800\n",
      "Epoch 92/200\n",
      "3500/3500 [==============================] - 3s 748us/step - loss: 0.0179 - acc: 0.9937 - val_loss: 0.0755 - val_acc: 0.9800\n",
      "Epoch 93/200\n",
      "3500/3500 [==============================] - 3s 745us/step - loss: 0.0245 - acc: 0.9917 - val_loss: 0.0707 - val_acc: 0.9850\n",
      "Epoch 94/200\n",
      "3500/3500 [==============================] - 3s 751us/step - loss: 0.0168 - acc: 0.9951 - val_loss: 0.0421 - val_acc: 0.9850\n",
      "Epoch 95/200\n",
      "3500/3500 [==============================] - 3s 736us/step - loss: 0.0207 - acc: 0.9926 - val_loss: 0.0686 - val_acc: 0.9800\n",
      "Epoch 96/200\n",
      "3500/3500 [==============================] - 3s 736us/step - loss: 0.0269 - acc: 0.9911 - val_loss: 0.0331 - val_acc: 0.9900\n",
      "Epoch 97/200\n",
      "3500/3500 [==============================] - 3s 743us/step - loss: 0.0226 - acc: 0.9929 - val_loss: 0.0561 - val_acc: 0.9800\n",
      "Epoch 98/200\n",
      "3500/3500 [==============================] - 3s 751us/step - loss: 0.0233 - acc: 0.9897 - val_loss: 0.0548 - val_acc: 0.9850\n",
      "Epoch 99/200\n",
      "3500/3500 [==============================] - 3s 745us/step - loss: 0.0226 - acc: 0.9917 - val_loss: 0.0493 - val_acc: 0.9900\n",
      "Epoch 100/200\n",
      "3500/3500 [==============================] - 3s 743us/step - loss: 0.0145 - acc: 0.9951 - val_loss: 0.0431 - val_acc: 0.9850\n",
      "Epoch 101/200\n",
      "3500/3500 [==============================] - 3s 735us/step - loss: 0.0172 - acc: 0.9949 - val_loss: 0.0547 - val_acc: 0.9900\n",
      "Epoch 102/200\n",
      "3500/3500 [==============================] - 3s 742us/step - loss: 0.0142 - acc: 0.9949 - val_loss: 0.0554 - val_acc: 0.9850\n",
      "Epoch 103/200\n",
      "3500/3500 [==============================] - 3s 748us/step - loss: 0.0156 - acc: 0.9943 - val_loss: 0.0386 - val_acc: 0.9900\n",
      "Epoch 104/200\n",
      "3500/3500 [==============================] - 3s 748us/step - loss: 0.0117 - acc: 0.9966 - val_loss: 0.0582 - val_acc: 0.9800\n",
      "Epoch 105/200\n",
      "3500/3500 [==============================] - 3s 755us/step - loss: 0.0129 - acc: 0.9943 - val_loss: 0.0118 - val_acc: 0.9950\n",
      "Epoch 106/200\n",
      "3500/3500 [==============================] - 3s 742us/step - loss: 0.0093 - acc: 0.9969 - val_loss: 0.0312 - val_acc: 0.9800\n",
      "Epoch 107/200\n",
      "3500/3500 [==============================] - 3s 760us/step - loss: 0.0108 - acc: 0.9963 - val_loss: 0.0679 - val_acc: 0.9850\n",
      "Epoch 108/200\n",
      "3500/3500 [==============================] - 3s 737us/step - loss: 0.0109 - acc: 0.9969 - val_loss: 0.0137 - val_acc: 0.9900\n",
      "Epoch 109/200\n",
      "3500/3500 [==============================] - 3s 744us/step - loss: 0.0102 - acc: 0.9966 - val_loss: 0.0311 - val_acc: 0.9900\n",
      "Epoch 110/200\n",
      "3500/3500 [==============================] - 3s 727us/step - loss: 0.0123 - acc: 0.9960 - val_loss: 0.0421 - val_acc: 0.9850\n",
      "Epoch 111/200\n",
      "3500/3500 [==============================] - 3s 740us/step - loss: 0.0118 - acc: 0.9957 - val_loss: 0.0453 - val_acc: 0.9850\n",
      "Epoch 112/200\n",
      "3500/3500 [==============================] - 3s 747us/step - loss: 0.0103 - acc: 0.9963 - val_loss: 0.0243 - val_acc: 0.9900\n",
      "Epoch 113/200\n",
      "3500/3500 [==============================] - 3s 741us/step - loss: 0.0106 - acc: 0.9957 - val_loss: 0.0699 - val_acc: 0.9800\n",
      "Epoch 114/200\n",
      "3500/3500 [==============================] - 3s 746us/step - loss: 0.0069 - acc: 0.9989 - val_loss: 0.0584 - val_acc: 0.9850\n",
      "Epoch 115/200\n",
      "3500/3500 [==============================] - 3s 753us/step - loss: 0.0144 - acc: 0.9957 - val_loss: 0.0104 - val_acc: 0.9950\n",
      "Epoch 116/200\n",
      "3500/3500 [==============================] - 3s 745us/step - loss: 0.0127 - acc: 0.9946 - val_loss: 0.0593 - val_acc: 0.9850\n",
      "Epoch 117/200\n",
      "3500/3500 [==============================] - 3s 755us/step - loss: 0.0164 - acc: 0.9934 - val_loss: 0.0677 - val_acc: 0.9850\n",
      "Epoch 118/200\n",
      "3500/3500 [==============================] - 3s 754us/step - loss: 0.0115 - acc: 0.9960 - val_loss: 0.0772 - val_acc: 0.9800\n",
      "Epoch 119/200\n",
      "3500/3500 [==============================] - 3s 750us/step - loss: 0.0114 - acc: 0.9963 - val_loss: 0.0801 - val_acc: 0.9850\n",
      "Epoch 120/200\n",
      "3500/3500 [==============================] - 3s 758us/step - loss: 0.0126 - acc: 0.9957 - val_loss: 0.0603 - val_acc: 0.9750\n",
      "Epoch 121/200\n",
      "3500/3500 [==============================] - 3s 748us/step - loss: 0.0132 - acc: 0.9966 - val_loss: 0.0203 - val_acc: 0.9900\n",
      "Epoch 122/200\n",
      "3500/3500 [==============================] - 3s 740us/step - loss: 0.0122 - acc: 0.9960 - val_loss: 0.0774 - val_acc: 0.9900\n",
      "Epoch 123/200\n",
      "3500/3500 [==============================] - 3s 743us/step - loss: 0.0102 - acc: 0.9966 - val_loss: 0.0600 - val_acc: 0.9800\n",
      "Epoch 124/200\n",
      "3500/3500 [==============================] - 3s 740us/step - loss: 0.0075 - acc: 0.9986 - val_loss: 0.0520 - val_acc: 0.9900\n",
      "Epoch 125/200\n",
      "3500/3500 [==============================] - 3s 736us/step - loss: 0.0155 - acc: 0.9951 - val_loss: 0.0742 - val_acc: 0.9750\n",
      "Epoch 126/200\n",
      "3500/3500 [==============================] - 3s 746us/step - loss: 0.0120 - acc: 0.9960 - val_loss: 0.0639 - val_acc: 0.9850\n",
      "Epoch 127/200\n",
      "3500/3500 [==============================] - 3s 745us/step - loss: 0.0061 - acc: 0.9986 - val_loss: 0.0373 - val_acc: 0.9850\n",
      "Epoch 128/200\n",
      "3500/3500 [==============================] - 3s 747us/step - loss: 0.0118 - acc: 0.9960 - val_loss: 0.0388 - val_acc: 0.9850\n",
      "Epoch 129/200\n",
      "3500/3500 [==============================] - 3s 743us/step - loss: 0.0084 - acc: 0.9963 - val_loss: 0.0637 - val_acc: 0.9750\n",
      "Epoch 130/200\n",
      "3500/3500 [==============================] - 3s 745us/step - loss: 0.0071 - acc: 0.9983 - val_loss: 0.0394 - val_acc: 0.9900\n",
      "Epoch 131/200\n",
      "3500/3500 [==============================] - 3s 734us/step - loss: 0.0151 - acc: 0.9949 - val_loss: 0.1052 - val_acc: 0.9800\n",
      "Epoch 132/200\n",
      "3500/3500 [==============================] - 3s 741us/step - loss: 0.0112 - acc: 0.9954 - val_loss: 0.0600 - val_acc: 0.9850\n",
      "Epoch 133/200\n",
      "3500/3500 [==============================] - 3s 727us/step - loss: 0.0137 - acc: 0.9951 - val_loss: 0.0301 - val_acc: 0.9850\n",
      "Epoch 134/200\n",
      "3500/3500 [==============================] - 3s 754us/step - loss: 0.0076 - acc: 0.9977 - val_loss: 0.0535 - val_acc: 0.9800\n",
      "Epoch 135/200\n",
      "3500/3500 [==============================] - 3s 735us/step - loss: 0.0121 - acc: 0.9954 - val_loss: 0.0513 - val_acc: 0.9750\n",
      "Epoch 136/200\n",
      "3500/3500 [==============================] - 3s 751us/step - loss: 0.0219 - acc: 0.9943 - val_loss: 0.0256 - val_acc: 0.9850\n",
      "Epoch 137/200\n",
      "3500/3500 [==============================] - 3s 744us/step - loss: 0.0199 - acc: 0.9929 - val_loss: 0.0581 - val_acc: 0.9850\n",
      "Epoch 138/200\n",
      "3500/3500 [==============================] - 3s 756us/step - loss: 0.0197 - acc: 0.9929 - val_loss: 0.0430 - val_acc: 0.9850\n",
      "Epoch 139/200\n",
      "3500/3500 [==============================] - 3s 760us/step - loss: 0.0160 - acc: 0.9937 - val_loss: 0.0681 - val_acc: 0.9850\n",
      "Epoch 140/200\n",
      "3500/3500 [==============================] - 3s 743us/step - loss: 0.0095 - acc: 0.9969 - val_loss: 0.0808 - val_acc: 0.9900\n",
      "Epoch 141/200\n",
      "3500/3500 [==============================] - 3s 725us/step - loss: 0.0101 - acc: 0.9960 - val_loss: 0.0695 - val_acc: 0.9900\n",
      "Epoch 142/200\n",
      "3500/3500 [==============================] - 3s 750us/step - loss: 0.0107 - acc: 0.9969 - val_loss: 0.0718 - val_acc: 0.9850\n",
      "Epoch 143/200\n",
      "3500/3500 [==============================] - 3s 741us/step - loss: 0.0131 - acc: 0.9963 - val_loss: 0.0371 - val_acc: 0.9900\n",
      "Epoch 144/200\n",
      "3500/3500 [==============================] - 3s 747us/step - loss: 0.0087 - acc: 0.9971 - val_loss: 0.0355 - val_acc: 0.9900\n",
      "Epoch 145/200\n",
      "3500/3500 [==============================] - 3s 750us/step - loss: 0.0080 - acc: 0.9983 - val_loss: 0.0319 - val_acc: 0.9850\n",
      "Epoch 146/200\n",
      "3500/3500 [==============================] - 3s 736us/step - loss: 0.0161 - acc: 0.9954 - val_loss: 0.0531 - val_acc: 0.9850\n",
      "Epoch 147/200\n",
      "3500/3500 [==============================] - 3s 745us/step - loss: 0.0129 - acc: 0.9954 - val_loss: 0.0540 - val_acc: 0.9850\n",
      "Epoch 148/200\n",
      "3500/3500 [==============================] - 3s 738us/step - loss: 0.0076 - acc: 0.9966 - val_loss: 0.0573 - val_acc: 0.9850\n",
      "Epoch 149/200\n",
      "3500/3500 [==============================] - 3s 760us/step - loss: 0.0145 - acc: 0.9951 - val_loss: 0.0687 - val_acc: 0.9850\n",
      "Epoch 150/200\n",
      "3500/3500 [==============================] - 3s 735us/step - loss: 0.0096 - acc: 0.9963 - val_loss: 0.0479 - val_acc: 0.9850\n",
      "Epoch 151/200\n",
      "3500/3500 [==============================] - 3s 750us/step - loss: 0.0096 - acc: 0.9966 - val_loss: 0.0672 - val_acc: 0.9850\n",
      "Epoch 152/200\n",
      "3500/3500 [==============================] - 3s 746us/step - loss: 0.0120 - acc: 0.9954 - val_loss: 0.0708 - val_acc: 0.9850\n",
      "Epoch 153/200\n",
      "3500/3500 [==============================] - 3s 757us/step - loss: 0.0101 - acc: 0.9969 - val_loss: 0.0819 - val_acc: 0.9800\n",
      "Epoch 154/200\n",
      "3500/3500 [==============================] - 3s 743us/step - loss: 0.0105 - acc: 0.9963 - val_loss: 0.0559 - val_acc: 0.9850\n",
      "Epoch 155/200\n",
      "3500/3500 [==============================] - 3s 738us/step - loss: 0.0084 - acc: 0.9969 - val_loss: 0.0766 - val_acc: 0.9800\n",
      "Epoch 156/200\n",
      "3500/3500 [==============================] - 3s 746us/step - loss: 0.0054 - acc: 0.9989 - val_loss: 0.0653 - val_acc: 0.9850\n",
      "Epoch 157/200\n",
      "3500/3500 [==============================] - 3s 752us/step - loss: 0.0081 - acc: 0.9971 - val_loss: 0.0790 - val_acc: 0.9800\n",
      "Epoch 158/200\n",
      "3500/3500 [==============================] - 3s 737us/step - loss: 0.0060 - acc: 0.9977 - val_loss: 0.0918 - val_acc: 0.9850\n",
      "Epoch 159/200\n",
      "3500/3500 [==============================] - 3s 741us/step - loss: 0.0054 - acc: 0.9986 - val_loss: 0.0713 - val_acc: 0.9800\n",
      "Epoch 160/200\n",
      "3500/3500 [==============================] - 3s 749us/step - loss: 0.0111 - acc: 0.9957 - val_loss: 0.0761 - val_acc: 0.9800\n",
      "Epoch 161/200\n",
      "3500/3500 [==============================] - 3s 755us/step - loss: 0.0091 - acc: 0.9969 - val_loss: 0.0797 - val_acc: 0.9850\n",
      "Epoch 162/200\n",
      "3500/3500 [==============================] - 3s 740us/step - loss: 0.0109 - acc: 0.9960 - val_loss: 0.0460 - val_acc: 0.9850\n",
      "Epoch 163/200\n",
      "3500/3500 [==============================] - 3s 740us/step - loss: 0.0060 - acc: 0.9980 - val_loss: 0.0487 - val_acc: 0.9950\n",
      "Epoch 164/200\n",
      "3500/3500 [==============================] - 3s 751us/step - loss: 0.0051 - acc: 0.9983 - val_loss: 0.0657 - val_acc: 0.9850\n",
      "Epoch 165/200\n",
      "3500/3500 [==============================] - 3s 738us/step - loss: 0.0051 - acc: 0.9983 - val_loss: 0.0866 - val_acc: 0.9850\n",
      "Epoch 166/200\n",
      "3500/3500 [==============================] - 3s 760us/step - loss: 0.0080 - acc: 0.9966 - val_loss: 0.0990 - val_acc: 0.9750\n",
      "Epoch 167/200\n",
      "3500/3500 [==============================] - 3s 743us/step - loss: 0.0064 - acc: 0.9971 - val_loss: 0.0958 - val_acc: 0.9850\n",
      "Epoch 168/200\n",
      "3500/3500 [==============================] - 3s 749us/step - loss: 0.0038 - acc: 0.9989 - val_loss: 0.1057 - val_acc: 0.9800\n",
      "Epoch 169/200\n",
      "3500/3500 [==============================] - 3s 742us/step - loss: 0.0079 - acc: 0.9957 - val_loss: 0.0694 - val_acc: 0.9900\n",
      "Epoch 170/200\n",
      "3500/3500 [==============================] - 3s 730us/step - loss: 0.0115 - acc: 0.9969 - val_loss: 0.0846 - val_acc: 0.9800\n",
      "Epoch 171/200\n",
      "3500/3500 [==============================] - 3s 738us/step - loss: 0.0094 - acc: 0.9957 - val_loss: 0.0694 - val_acc: 0.9800\n",
      "Epoch 172/200\n",
      "3500/3500 [==============================] - 3s 753us/step - loss: 0.0148 - acc: 0.9937 - val_loss: 0.0305 - val_acc: 0.9900\n",
      "Epoch 173/200\n",
      "3500/3500 [==============================] - 3s 760us/step - loss: 0.0120 - acc: 0.9963 - val_loss: 0.0417 - val_acc: 0.9950\n",
      "Epoch 174/200\n",
      "3500/3500 [==============================] - 3s 748us/step - loss: 0.0122 - acc: 0.9966 - val_loss: 0.0812 - val_acc: 0.9800\n",
      "Epoch 175/200\n",
      "3500/3500 [==============================] - 3s 752us/step - loss: 0.0116 - acc: 0.9963 - val_loss: 0.0330 - val_acc: 0.9950\n",
      "Epoch 176/200\n",
      "3500/3500 [==============================] - 3s 744us/step - loss: 0.0146 - acc: 0.9934 - val_loss: 0.0538 - val_acc: 0.9900\n",
      "Epoch 177/200\n",
      "3500/3500 [==============================] - 3s 747us/step - loss: 0.0078 - acc: 0.9977 - val_loss: 0.0466 - val_acc: 0.9850\n",
      "Epoch 178/200\n",
      "3500/3500 [==============================] - 3s 743us/step - loss: 0.0048 - acc: 0.9991 - val_loss: 0.0563 - val_acc: 0.9850\n",
      "Epoch 179/200\n",
      "3500/3500 [==============================] - 3s 745us/step - loss: 0.0087 - acc: 0.9969 - val_loss: 0.0751 - val_acc: 0.9850\n",
      "Epoch 180/200\n",
      "3500/3500 [==============================] - 3s 751us/step - loss: 0.0061 - acc: 0.9980 - val_loss: 0.0393 - val_acc: 0.9850\n",
      "Epoch 181/200\n",
      "3500/3500 [==============================] - 3s 746us/step - loss: 0.0101 - acc: 0.9966 - val_loss: 0.0403 - val_acc: 0.9850\n",
      "Epoch 182/200\n",
      "3500/3500 [==============================] - 3s 740us/step - loss: 0.0061 - acc: 0.9980 - val_loss: 0.0490 - val_acc: 0.9850\n",
      "Epoch 183/200\n",
      "3500/3500 [==============================] - 3s 741us/step - loss: 0.0076 - acc: 0.9974 - val_loss: 0.0706 - val_acc: 0.9850\n",
      "Epoch 184/200\n",
      "3500/3500 [==============================] - 3s 751us/step - loss: 0.0102 - acc: 0.9971 - val_loss: 0.0427 - val_acc: 0.9900\n",
      "Epoch 185/200\n",
      "3500/3500 [==============================] - 3s 746us/step - loss: 0.0082 - acc: 0.9971 - val_loss: 0.0414 - val_acc: 0.9850\n",
      "Epoch 186/200\n",
      "3500/3500 [==============================] - 3s 741us/step - loss: 0.0054 - acc: 0.9989 - val_loss: 0.0881 - val_acc: 0.9800\n",
      "Epoch 187/200\n",
      "3500/3500 [==============================] - 3s 749us/step - loss: 0.0038 - acc: 0.9991 - val_loss: 0.0625 - val_acc: 0.9850\n",
      "Epoch 188/200\n",
      "3500/3500 [==============================] - 3s 733us/step - loss: 0.0042 - acc: 0.9991 - val_loss: 0.0474 - val_acc: 0.9850\n",
      "Epoch 189/200\n",
      "3500/3500 [==============================] - 3s 752us/step - loss: 0.0048 - acc: 0.9983 - val_loss: 0.0284 - val_acc: 0.9900\n",
      "Epoch 190/200\n",
      "3500/3500 [==============================] - 3s 750us/step - loss: 0.0058 - acc: 0.9974 - val_loss: 0.0343 - val_acc: 0.9900\n",
      "Epoch 191/200\n",
      "3500/3500 [==============================] - 3s 750us/step - loss: 0.0057 - acc: 0.9983 - val_loss: 0.0641 - val_acc: 0.9800\n",
      "Epoch 192/200\n",
      "3500/3500 [==============================] - 3s 755us/step - loss: 0.0050 - acc: 0.9983 - val_loss: 0.0674 - val_acc: 0.9850\n",
      "Epoch 193/200\n",
      "3500/3500 [==============================] - 3s 757us/step - loss: 0.0067 - acc: 0.9977 - val_loss: 0.0554 - val_acc: 0.9850\n",
      "Epoch 194/200\n",
      "3500/3500 [==============================] - 3s 759us/step - loss: 0.0058 - acc: 0.9986 - val_loss: 0.0410 - val_acc: 0.9850\n",
      "Epoch 195/200\n",
      "3500/3500 [==============================] - 3s 752us/step - loss: 0.0048 - acc: 0.9980 - val_loss: 0.0284 - val_acc: 0.9850\n",
      "Epoch 196/200\n",
      "3500/3500 [==============================] - 3s 746us/step - loss: 0.0074 - acc: 0.9971 - val_loss: 0.0416 - val_acc: 0.9900\n",
      "Epoch 197/200\n",
      "3500/3500 [==============================] - 3s 763us/step - loss: 0.0077 - acc: 0.9969 - val_loss: 0.0521 - val_acc: 0.9850\n",
      "Epoch 198/200\n",
      "3500/3500 [==============================] - 3s 756us/step - loss: 0.0071 - acc: 0.9966 - val_loss: 0.0498 - val_acc: 0.9900\n",
      "Epoch 199/200\n",
      "3500/3500 [==============================] - 3s 760us/step - loss: 0.0078 - acc: 0.9977 - val_loss: 0.0359 - val_acc: 0.9850\n",
      "Epoch 200/200\n",
      "3500/3500 [==============================] - 3s 757us/step - loss: 0.0088 - acc: 0.9971 - val_loss: 0.0757 - val_acc: 0.9850\n"
     ]
    }
   ],
   "source": [
    "# 1. Import libraries and modules\n",
    "import keras\n",
    "from keras import layers\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Dropout, Activation\n",
    "import numpy as np\n",
    "from dataset import load_hoda\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "np.random.seed(123)  # for reproducibility\n",
    "\n",
    "# Load pre-shuffled HODA data into train and test sets\n",
    "x_train_original, y_train_original, x_test_original, y_test_original = load_hoda(\n",
    "                                                                        training_sample_size=3500,\n",
    "                                                                        test_sample_size=400,size=28)\n",
    "\n",
    "# Preprocess input data\n",
    "''' 3.1: input data in numpy array format'''\n",
    "x_train = np.array(x_train_original)\n",
    "x_test = np.array(x_test_original)\n",
    "'''3.2 normalize our data values to the range [0, 1]'''\n",
    "x_train = x_train.astype('float32')\n",
    "x_test = x_test.astype('float32')\n",
    "x_train /= 255\n",
    "x_test /= 255\n",
    "\n",
    "# Reshape to original image shape (n x 784)  ==> (n x 28 x 28 x 1)\n",
    "x_train = x_train.reshape(-1,28,28,1)\n",
    "x_test = x_test.reshape(-1,28,28,1)\n",
    "\n",
    "\n",
    "# 4. Preprocess class labels\n",
    "y_train = keras.utils.to_categorical(y_train_original, num_classes=10)\n",
    "y_test = keras.utils.to_categorical(y_test_original, num_classes=10)\n",
    "\n",
    "\n",
    "# test and validation set\n",
    "x_val = x_test[200:]\n",
    "x_test = x_test[:200]\n",
    "y_val = y_test[200:]\n",
    "y_test = y_test[:200]\n",
    "\n",
    "# 5. Define model architecture\n",
    "model = Sequential()\n",
    "model.add(layers.Conv2D(32, (3, 3), activation='relu',\n",
    "                        input_shape=(28, 28, 1)))\n",
    "model.add(layers.MaxPooling2D((2, 2)))\n",
    "model.add(layers.Conv2D(64, (3, 3), activation='relu'))\n",
    "model.add(layers.MaxPooling2D((2, 2)))\n",
    "model.add(layers.Conv2D(64, (3, 3), activation='relu'))\n",
    "model.add(layers.Flatten())\n",
    "model.add(layers.Dense(64, activation='relu'))\n",
    "model.add(Dropout(0.5))\n",
    "model.add(layers.Dense(10, activation='softmax'))\n",
    "\n",
    "\n",
    "# 6. Compile model\n",
    "model.compile(loss='categorical_crossentropy',\n",
    "              optimizer='adam',\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "\n",
    "# 7. Fit model on training data\n",
    "history = model.fit(x_train, y_train,\n",
    "          epochs=200, batch_size=256, validation_data = (x_val, y_val))\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XmcFNW5//HPwwAiu8zgEpAlBoO4oDBB/YkLURM0ClckRsTEJYoScUtM4hWjXqPZY9RcohKjMTpKuBoNJi5RJBLjxmDYkUVFHEAdFlEcFAee3x+nmulpuqd7hplpuub7fr3q1V1Vp6qeru555vSp06fM3RERkXhple8ARESk8Sm5i4jEkJK7iEgMKbmLiMSQkruISAwpuYuIxJCSe4yZWZGZbTKzXo1ZNp/M7Atm1uj9d83sBDNbkTS/xMyOzqVsA451t5ld09DtRXLROt8BSA0z25Q02x74FNgazV/k7mX12Z+7bwU6NnbZlsDdv9gY+zGzC4Cz3f24pH1f0Bj7FqmLkvsuxN23J9eoZniBuz+bqbyZtXb36uaITSQbfR53LWqWKSBmdpOZ/dnMHjKzj4CzzexIM3vZzD4wszVmdruZtYnKtzYzN7M+0fwD0fonzewjM3vJzPrWt2y0/iQzW2pmG83st2b2bzM7N0PcucR4kZktN7MNZnZ70rZFZvYbM1tnZm8Cw+s4PxPNbErKsklmdkv0/AIzWxy9njeiWnWmfVWY2XHR8/Zmdn8U20JgcErZa83szWi/C81sRLT8YOB/gaOjJq+1Sef2hqTtL45e+zoze8zM9snl3NTnPCfiMbNnzWy9mb1rZj9IOs6PonPyoZmVm9nn0jWBmdkLifc5Op8zo+OsB641s35mNiM6xtrovHVJ2r539Boro/W3mVm7KOYDksrtY2ZVZlac6fVKFu6uaRecgBXACSnLbgK2AKcS/jHvDnwJOJzwLezzwFJgQlS+NeBAn2j+AWAtUAq0Af4MPNCAsnsCHwEjo3XfBT4Dzs3wWnKJ8a9AF6APsD7x2oEJwEKgJ1AMzAwf27TH+TywCeiQtO/3gdJo/tSojAFfBjYDh0TrTgBWJO2rAjguev4r4J/AHkBvYFFK2TOAfaL35Kwohr2idRcA/0yJ8wHghuj5V6IYDwXaAb8Dnsvl3NTzPHcB3gMuB3YDOgNDonX/DcwF+kWv4VCgG/CF1HMNvJB4n6PXVg2MB4oIn8f9geOBttHn5N/Ar5Jez4LofHaIyh8VrZsM3Jx0nO8Bj+b777CQp7wHoCnDG5M5uT+XZburgP+LnqdL2HcmlR0BLGhA2fOBfyWtM2ANGZJ7jjEekbT+L8BV0fOZhOapxLqTUxNOyr5fBs6Knp8ELKmj7N+AS6LndSX3lcnvBfCd5LJp9rsA+Fr0PFtyvw/4SdK6zoTrLD2znZt6nudvArMylHsjEW/K8lyS+5tZYhidOC5wNPAuUJSm3FHAW4BF83OAUY39d9WSJjXLFJ53kmfMrL+Z/T36mv0hcCNQUsf27yY9r6Lui6iZyn4uOQ4Pf40VmXaSY4w5HQt4u454AR4ExkTPz4rmE3GcYmavRE0GHxBqzXWdq4R96orBzM41s7lR08IHQP8c9wvh9W3fn7t/CGwAeiSVyek9y3Ke9yUk8XTqWpdN6udxbzObamarohj+mBLDCg8X72tx938TvgUMNbODgF7A3xsYk6A290KU2g3wLkJN8Qvu3hm4jlCTbkprCDVLAMzMqJ2MUu1MjGsISSEhW1fNqcAJZtaD0Gz0YBTj7sDDwE8JTSZdgX/kGMe7mWIws88DdxCaJoqj/b6etN9s3TZXE5p6EvvrRGj+WZVDXKnqOs/vAPtl2C7Tuo+jmNonLds7pUzq6/s5oZfXwVEM56bE0NvMijLE8SfgbMK3jKnu/mmGcpIDJffC1wnYCHwcXZC6qBmO+TdgkJmdamatCe243ZsoxqnAFWbWI7q49sO6Crv7u4Smgz8SmmSWRat2I7QDVwJbzewUQttwrjFcY2ZdLfwOYELSuo6EBFdJ+D93IaHmnvAe0DP5wmaKh4Bvm9khZrYb4Z/Pv9w94zehOtR1nqcBvcxsgpntZmadzWxItO5u4CYz28+CQ82sG+Gf2ruEC/dFZjaOpH9EdcTwMbDRzPYlNA0lvASsA35i4SL17mZ2VNL6+wnNOGcREr3sBCX3wvc94BzCBc67CBc+m5S7vwd8A7iF8Me6H/AfQo2tsWO8A5gOzAdmEWrf2TxIaEPf3iTj7h8AVwKPEi5Kjib8k8rF9YRvECuAJ0lKPO4+D/gt8GpU5ovAK0nbPgMsA94zs+TmlcT2TxGaTx6Ntu8FjM0xrlQZz7O7bwROBE4n/MNZChwbrf4l8BjhPH9IuLjZLmpuuxC4hnBx/Qspry2d64EhhH8y04BHkmKoBk4BDiDU4lcS3ofE+hWE9/lTd3+xnq9dUiQuXog0WPQ1ezUw2t3/le94pHCZ2Z8IF2lvyHcshU4/YpIGMbPhhJ4pmwld6T4j1F5FGiS6fjESODjfscSBmmWkoYYCbxLamr8KnKYLYNJQZvZTQl/7n7j7ynzHEwdqlhERiSHV3EVEYihvbe4lJSXep0+ffB1eRKQgzZ49e62719X1GMhjcu/Tpw/l5eX5OryISEEys2y/0gbULCMiEktK7iIiMaTkLiISQ0ruIiIxpOQuIhJDWZO7md1jZu+b2YIM6y26zdZyM5tnZoMaP0wRkdyUlUGfPtCqVXgsq9dt5eMjl5r7H6njvpWEu930i6ZxhFH8RGKjrAxKSsAsTCUl9U8YyQmnpAQ6dqx7f5mOWZ/Elals6vLvfKd2bInjtm4dHhPbpr6GkpK69536OpOnoqKa15XYT/KxW7VKv12mqWPHmm3PPhvefhvcw+M3v1n79dR335nizvba0p2XdOe1yeRyuybCvRsXZFh3FzAmaX4JsE+2fQ4ePNil6dx/v/vq1Y23vwcecO/d2x3ci4rCY+/eYXlqGbMd1zXkOGbhsa6pVavwWFzs3qFD+jLFxe7jx4fHdOs7dKhZl8sxs8WyM/vQ1LKm9u3r/3cClLvnkLdzKlR3cv8bMDRpfjrRDYnTlB0HlAPlvXr1qv9ffhPZti3/+6mudq+qcv/ss50/9sqV4Z39r/+q/36SE3SvXmF+/Pi6E1bHjumXZ0tySoaaNIW/t/rINbk36wVVd5/s7qXuXtq9e9Zfzza5LVvgrLNgv/1g6dKd29eDD8Kee8LMmbB4MXTrBs89l9u2n3wCBxwA7dtDcTGsXAl/+xvssQe8/nrd227eDKedBkceCdXVYdmr0cC7jz0Gs2bVLp/89TDd19Pkr7QrV4b5O+4I85ls2pR+eV3bAGzblls5kThb2URjYDbG8AOrqH1/yZ407P6PzeqTT+DrXw9JtHNnOOYYePbZkGT/93/hzTdDObPQZjd4cJivqIDf/jZsn7BpE9x7b0hS3/8+7LsvbNgAkyfDscfCT38KlZVh3xddFPaZ7K67YNky+O53w7Gvvx5eeQU++AB+9COYNAluvRU+/jiUf/11eOkl+Oij2vvp2hWqqqBTp5C4AYYMCfNHHhnKv/RSTXklVZH865XtrsANlUv1nrqbZb5GuPWYAUcAr+ayz3y2uW/a5H7CCeEr0R13uC9e7P65z4V21xEjwvIuXdy7dnVv08b9oINCs8kbb4SvUK1bh3XJ0+jR7rffXvNVq2tX9913d//d78J8p07hcfx491WrQnv46tXub73l3rlz2Gfq17XddguPiTbuVq3Sl9OkSVNhTnltcyfcwHcN4U47FcC3gYuBi6P1BkwC3iDc/zBte3vqlM/kfvbZIVH+8Y81y5Yvr7mQd/PNNcunTAnLbrgh/APo1s29vDz9frdsce/Xz32PPdwffzxs17p12G+vXvn/IGnSpGnXmRra6aDRkntTTflK7nPmhFd99dU7rluzxn369NrLtm51HzgwbLPnnu7z5tW9/6VLwz8HJfN4TR06uJ93nvt++zXO/hLf5Pbfv+YbWurUrVv447/uuh2XjxuX+UJ2LlP79jXfCBMXtDt1CvtO7u30wAO1exm1bRvK1LXvdu1CBSe119S2be7HHFNz/L33DmX22cf9jDNqKleJ1/fRRzV/V++8437XXe533+1eWRmWPfOM+513hvdl993Dtm3auJeUhG0vuqgm1t693X/601D+zjvdb7stxNizp/uSJe7f/GaIu6IixPn3v9eU/epXw7orrwz72muv7D3CUs9b4hz36hUql+vWZU1VGSm5u/vHH7svWlR7Ovnk0OSyfn3u+3n+efdjj3V//fUd1yX3LqmrO15cpmzdCpM/yNmmdL1lEss6dAjdOTP5/e/djzvO/Te/qUkK7du7n3pq+IfsHuKEmqasdH+M9e2++eGH7qNGhcpB4jgLF7ofcURotsvk9dfdjz7avaysZtltt4V/Fn36hOmoo0IzXTqXXeZ++eWhebA+6np9zz8fYmpIolm/3v3EE90ffNB9xQr3448PCTGb2bNDgn/ttfofsy6rV9ck+B/+MLdt5sxx7949TGbu3/9++nJvvFHzGRo+vPFibigld3f/ylfSJ5WbbmrY/lL/UMaPDwkl3wm3uRL4hRfWfT4SiaMx+rs3hr/8JcR93335Ob40rx/8ILzfc+bkvs2iReGbQ6dO7mvXZi530UVh35maZJtTrsk9b/dQLS0t9aa8WcfHH4euhCNHwujRNcvbtIFTToG2beu3v7IyGDcu9EaJg+JiuO02GDs2t/IvvgiHHx5+fVcotm2Df/8bhg7dsYeSxM+WLVBeDv/v/9Vvu/ffDz3T9t8/c5mqKpg/P/wN5JuZzXb30qzl4prc//EP+OpX4amnwuPOKimBdet2fj+NqbgYzjgDnngi9E03C3XsbNvUJ6mLyK4l1+Qe21Ehp08PtfShQxu+j+TxPZo6sZvB+PEhOT/wQEjC6coA9O4dyqxdC7/7HaxYEba7//6wDmpq2ImyicaVtWuV2EVagtjW3EtLwy8+Z86s/7ZlZXD55c1XU+/dG26+ecekW1YGEyeGX7D16pW+jIi0LC225r5yJfz1r/Daa3D88fXf/jvfCb9IbYzEnhg1LlF7fuCB8Dx5mXuoeadL2mPHhnXbtmUuIyKSTmMMP7BLOflkWLgwPB9e10DFaZSVwZ13Zm+3zqS4GNavr7uWrQQtIs0hVjX3VatCYv/e9+p/ZbusDM45p2GJPdFevnatatkismuIVXKfPj08fvObcNBBuW+XaIrZujW38h06hFp6onnl/vvDhU0RkV1FrJplnnsu9G45+ODct6lPU4y6EYpIoYhNzd091NyHDasZ7rYuiXHNzz677sSe3EVR3QhFpFDEpua+bFkYa/3LX667XH26ORYVwX33KaGLSOGJTXJP9GcfNixzme98J/cmGDMldhEpXLFplpk1K9yJKNP4EPVpWzeDiy9WYheRwhWbmvurr4ZfpaYbIKo+3RzVFCMicRCLmvvmzaFf+5AhO65LjOaYazfHbduU2EWk8MUiuc+ZE5L3l76047qJE+s3TG+T3axWRKQZxSK5z5oVHtMl95Urc99P+/Zh2AARkUIXm+S+zz7Qo8eO67p1S79NUVHov548kNfkyWqSEZF4yOmCqpkNB24DioC73f1nKet7A/cA3YH1wNnuXtHIsWb02mvhYmqqsjL48MMdl7dtC/fco0QuIvGVteZuZkXAJOAkYAAwxswGpBT7FfAndz8EuBH4aWMHWpfKyh1r7YkeMp99tmP5Tp2U2EUk3nJplhkCLHf3N919CzAFGJlSZgDwXPR8Rpr1TerDD6Fz55r5bD1k1q9vnrhERPIll+TeA3gnab4iWpZsLjAqen4a0MnM0tworvF9+mmYkpN7th4y6hEjInHXWBdUrwKONbP/AMcCq4Ad6s1mNs7Mys2svLKyslEO/NFH4bFLl5pldfWQUY8YEWkJcknuq4B9k+Z7Rsu2c/fV7j7K3Q8DJkbLPkjdkbtPdvdSdy/t3r37ToRdY+PG8Jhcc89UMy8qUo8YEWkZcknus4B+ZtbXzNoCZwLTkguYWYmZJfb134SeM80i0RsmObnffHOooSdr317DCohIy5E1ubt7NTABeBpYDEx194VmdqOZjYiKHQcsMbOlwF5AszV8pCb3srKaNveiorBMfdhFpKXJqZ+7uz8BPJGy7Lqk5w8DDzduaLlJTu6JXjKJi6lbt9a0sSuxi0hLUvC/UE1O7ul6yVRVheUiIi1JrJJ7pl4y9RlfRkQkDmKV3DP1klG/dhFpaWKR3IuKYPfdM/eSUb92EWlpYpHcO3cOIzuOHRt6xWikRxFp6WKT3KGmG+TKlaEpRr1kRKSlKvh7qCaSe2o3yLffDvOgBC8iLU9sau7qBikiUiM2yV3dIEVEasQmuasbpIhIjdgkd3WDFBGpEZvkrm6QIiI1Crq3THV1uGia6Ao5dqySuYgIFHjNPXEXpmXLoE8faNUqPJaV5TMqEZH8K+iae2JcmalTYcuW8Fz920VECrzmnkjuicSeoP7tItLSxSK5p6P+7SLSksU2uat/u4i0ZLFI7u3a1V6u/u0i0tIVdHJfvz48/uIX6t8uIpKsoHvLvP46dOgAl1wCl16a72hERHYdOdXczWy4mS0xs+VmdnWa9b3MbIaZ/cfM5pnZyY0f6o4WLIADDwz920VEpEbWtGhmRcAk4CRgADDGzAakFLsWmOruhwFnAr9r7EDTmT8fDjqoOY4kIlJYcqnzDgGWu/ub7r4FmAKMTCnjQDQIAF2A1Y0XYnrvvw+VlXDwwU19JBGRwpNLcu8BvJM0XxEtS3YDcLaZVQBPAGlbwM1snJmVm1l5ZWVlA8KtMX9+eFTNXURkR43VWj0G+KO79wROBu43sx327e6T3b3U3Uu7d+++UwdcsCA8KrmLiOwol+S+Ctg3ab5ntCzZt4GpAO7+EtAOKGmMADOZPx9KSmCvvZryKCIihSmX5D4L6Gdmfc2sLeGC6bSUMiuB4wHM7ABCct+5dpcsFiwItXazpjyKiEhhyprc3b0amAA8DSwm9IpZaGY3mtmIqNj3gAvNbC7wEHCuu3tTBQ2wdCn07x+G99VwvyIiteX0IyZ3f4JwoTR52XVJzxcBRzVuaHXFAxs3wpo1YXjfqqqwXMP9iogEBfnzn6oq2LYNnn++JrEnr9NwvyLS0hVkck8MGPbBB+nXa7hfEWnpCjq5l2Toj6PhfkWkpSvo5H7OOWF432Qa7ldEpMCT+4gRYXhfDfcrIlJbQQ75m0juXbrAMccomYuIpCromnvnznWXExFpqZTcRURiqKCTe6dO+Y1DRGRXVbDJvV07aNs235GIiOyaCja5q0lGRCQzJXcRkRhSchcRiSEldxGRGFJyFxGJISV3EZEYKtjk/u67ugOTiEgmBTm2zAcfwD//CdXVYV53YBIRqa3gau6ffhqSeiKxJ+gOTCIiNQouuSeGHkhHd2ASEQlySu5mNtzMlpjZcjO7Os3635jZnGhaamYZboC38+pK7roDk4hIkLXN3cyKgEnAiUAFMMvMprn7okQZd78yqfylwGFNECsAGzeGx7ZtYcuWmuW6A5OISI1cau5DgOXu/qa7bwGmACPrKD8GeKgxgksnUXO/6irdgUlEJJNcesv0AN5Jmq8ADk9X0Mx6A32B5zKsHweMA+jVwDaURHIfNUo1dRGRTBr7guqZwMPuvjXdSnef7O6l7l7avXv3Bh1AN+oQEckul+S+Ctg3ab5ntCydM2nCJhlQchcRyUUuyX0W0M/M+ppZW0ICn5ZayMz6A3sALzVuiLUpuYuIZJc1ubt7NTABeBpYDEx194VmdqOZjUgqeiYwxd29aUINLrss/CK1XbumPIqISGGzJs7FGZWWlnp5eXleji0iUqjMbLa7l2YrV3C/UBURkeyU3EVEYkjJXUQkhpTcRURiSMldRCSGlNxFRGJIyV1EJIaU3EVEYkjJXUQkhpTcRURiSMldRCSGlNxFRGJIyV1EJIaU3EVEYkjJXUQkhpTcRURiSMldRCSGlNxFRGJIyV1EJIaU3EVEYkjJXUQkhnJK7mY23MyWmNlyM7s6Q5kzzGyRmS00swcbN0wREamP1tkKmFkRMAk4EagAZpnZNHdflFSmH/DfwFHuvsHM9myqgEVEJLtcau5DgOXu/qa7bwGmACNTylwITHL3DQDu/n7jhikiIvWRS3LvAbyTNF8RLUu2P7C/mf3bzF42s+HpdmRm48ys3MzKKysrGxYxUFYGffpAq1bhsayswbsSEYmlrM0y9dhPP+A4oCcw08wOdvcPkgu5+2RgMkBpaak35EBlZTBuHFRVhfm33w7zAGPHNix4EZG4yaXmvgrYN2m+Z7QsWQUwzd0/c/e3gKWEZN/oJk6sSewJVVVhuYiIBLkk91lAPzPra2ZtgTOBaSllHiPU2jGzEkIzzZuNGOd2K1fWb7mISEuUNbm7ezUwAXgaWAxMdfeFZnajmY2Iij0NrDOzRcAM4Pvuvq4pAu7Vq37LRURaInNvUNP3TistLfXy8vJ6b5fa5g7Qvj1Mnqw2dxGJPzOb7e6l2coV3C9Ux44Nibx3bzALj0rsIiK1NVZvmWY1dqySuYhIXQqu5i4iItkpuYuIxJCSu4hIDCm5i4jEkJK7iEgMKbmLiMSQkruISAwpuYuIxJCSu4hIDCm5i4jEkJK7iEgMKbmLiMSQkruISAwpuYuIxJCSu4hIDCm5i4jEkJK7iEgMKbmLiMRQTsndzIab2RIzW25mV6dZf66ZVZrZnGi6oPFDFRGRXGW9h6qZFQGTgBOBCmCWmU1z90UpRf/s7hOaIEYREamnXGruQ4Dl7v6mu28BpgAjmzYsERHZGbkk9x7AO0nzFdGyVKeb2Twze9jM9k23IzMbZ2blZlZeWVnZgHBFRCQXjXVB9XGgj7sfAjwD3JeukLtPdvdSdy/t3r17Ix1aRERS5ZLcVwHJNfGe0bLt3H2du38azd4NDG6c8EREpCFySe6zgH5m1tfM2gJnAtOSC5jZPkmzI4DFjReiiIjUV9beMu5ebWYTgKeBIuAed19oZjcC5e4+DbjMzEYA1cB64NwmjFlERLIwd8/LgUtLS728vDwvxxYRKVRmNtvdS7OV0y9URURiSMldRCSGlNxFRGJIyV1EJIaU3EVEYkjJXUQkhpTcRURiSMldRCSGlNxFRGJIyV1EJIaU3EVEYkjJXUQkhpTcRURiSMldRCSGlNxFRGJIyV1EJIaU3EVEYijrbfZEJF4+++wzKioq+OSTT/IditShXbt29OzZkzZt2jRoeyV3kRamoqKCTp060adPH8ws3+FIGu7OunXrqKiooG/fvg3ah5plRFqYTz75hOLiYiX2XZiZUVxcvFPfrnJK7mY23MyWmNlyM7u6jnKnm5mbWdabt4pI/iix7/p29j3KmtzNrAiYBJwEDADGmNmANOU6AZcDr+xURCIistNyqbkPAZa7+5vuvgWYAoxMU+7HwM8BXaURiZGyMujTB1q1Co9lZTu3v3Xr1nHooYdy6KGHsvfee9OjR4/t81u2bMlpH+eddx5Lliyps8ykSZMo29lgC1guF1R7AO8kzVcAhycXMLNBwL7u/ncz+34jxicieVRWBuPGQVVVmH/77TAPMHZsw/ZZXFzMnDlzALjhhhvo2LEjV111Va0y7o6706pV+vrnvffem/U4l1xyScMCjImdvqBqZq2AW4Dv5VB2nJmVm1l5ZWXlzh5aRJrYxIk1iT2hqiosb2zLly9nwIABjB07lgMPPJA1a9Ywbtw4SktLOfDAA7nxxhu3lx06dChz5syhurqarl27cvXVVzNw4ECOPPJI3n//fQCuvfZabr311u3lr776aoYMGcIXv/hFXnzxRQA+/vhjTj/9dAYMGMDo0aMpLS3d/o8n2fXXX8+XvvQlDjroIC6++GLcHYClS5fy5S9/mYEDBzJo0CBWrFgBwE9+8hMOPvhgBg4cyMSmOFk5yCW5rwL2TZrvGS1L6AQcBPzTzFYARwDT0l1UdffJ7l7q7qXdu3dveNQi0ixWrqzf8p31+uuvc+WVV7Jo0SJ69OjBz372M8rLy5k7dy7PPPMMixYt2mGbjRs3cuyxxzJ37lyOPPJI7rnnnrT7dndeffVVfvnLX27/R/Hb3/6Wvffem0WLFvGjH/2I//znP2m3vfzyy5k1axbz589n48aNPPXUUwCMGTOGK6+8krlz5/Liiy+y55578vjjj/Pkk0/y6quvMnfuXL73vaz13iaRS3KfBfQzs75m1hY4E5iWWOnuG929xN37uHsf4GVghLuXN0nEItJsevWq3/Kdtd9++1FaWlMvfOihhxg0aBCDBg1i8eLFaZP77rvvzkknnQTA4MGDt9eeU40aNWqHMi+88AJnnnkmAAMHDuTAAw9Mu+306dMZMmQIAwcO5Pnnn2fhwoVs2LCBtWvXcuqppwLhR0ft27fn2Wef5fzzz2f33XcHoFu3bvU/EY0ga3J392pgAvA0sBiY6u4LzexGMxvR1AGKSP7cfDO0b197Wfv2YXlT6NChw/bny5Yt47bbbuO5555j3rx5DB8+PG2/77Zt225/XlRURHV1ddp977bbblnLpFNVVcWECRN49NFHmTdvHueff35B/Lo3pzZ3d3/C3fd39/3c/eZo2XXuPi1N2eNUaxeJh7FjYfJk6N0bzMLj5MkNv5haHx9++CGdOnWic+fOrFmzhqeffrrRj3HUUUcxdepUAObPn5/2m8HmzZtp1aoVJSUlfPTRRzzyyCMA7LHHHnTv3p3HH38cCD8Oq6qq4sQTT+See+5h8+bNAKxfv77R486Fhh8QkTqNHds8yTzVoEGDGDBgAP3796d3794cddRRjX6MSy+9lG9961sMGDBg+9SlS5daZYqLiznnnHMYMGAA++yzD4cfXtNZsKysjIsuuoiJEyfStm1bHnnkEU455RTmzp1LaWkpbdq04dRTT+XHP/5xo8eejSWu+ja30tJSLy9XBV+kuS1evJgDDjgg32HsEqqrq6murqZdu3YsW7aMr3zlKyxbtozWrXeNem+698rMZrt71lEAdo1XICKSB5s2beL444+nurrlScoaAAALJUlEQVQad+euu+7aZRL7zorHqxARaYCuXbsye/bsfIfRJDQqpIhIDCm5i4jEkJK7iEgMKbmLiMSQkruINKthw4bt8IOkW2+9lfHjx9e5XceOHQFYvXo1o0ePTlvmuOOOI1sX61tvvZWqpNHQTj75ZD744INcQi8oSu4i0qzGjBnDlClTai2bMmUKY8aMyWn7z33uczz88MMNPn5qcn/iiSfo2rVrg/e3q1JXSJEW7IorIM0Itzvl0EMhGmk3rdGjR3PttdeyZcsW2rZty4oVK1i9ejVHH300mzZtYuTIkWzYsIHPPvuMm266iZEja98baMWKFZxyyiksWLCAzZs3c9555zF37lz69++//Sf/AOPHj2fWrFls3ryZ0aNH8z//8z/cfvvtrF69mmHDhlFSUsKMGTPo06cP5eXllJSUcMstt2wfVfKCCy7giiuuYMWKFZx00kkMHTqUF198kR49evDXv/51+8BgCY8//jg33XQTW7Zsobi4mLKyMvbaay82bdrEpZdeSnl5OWbG9ddfz+mnn85TTz3FNddcw9atWykpKWH69OmN9yag5C4izaxbt24MGTKEJ598kpEjRzJlyhTOOOMMzIx27drx6KOP0rlzZ9auXcsRRxzBiBEjMt5P9I477qB9+/YsXryYefPmMWjQoO3rbr75Zrp168bWrVs5/vjjmTdvHpdddhm33HILM2bMoKSkpNa+Zs+ezb333ssrr7yCu3P44Ydz7LHHsscee7Bs2TIeeughfv/733PGGWfwyCOPcPbZZ9fafujQobz88suYGXfffTe/+MUv+PWvf82Pf/xjunTpwvz58wHYsGEDlZWVXHjhhcycOZO+ffs2yfgzSu4iLVhdNeymlGiaSST3P/zhD0AYc/2aa65h5syZtGrVilWrVvHee++x9957p93PzJkzueyyywA45JBDOOSQQ7avmzp1KpMnT6a6upo1a9awaNGiWutTvfDCC5x22mnbR6YcNWoU//rXvxgxYgR9+/bl0EMPBTIPK1xRUcE3vvEN1qxZw5YtW+jbty8Azz77bK1mqD322IPHH3+cY445ZnuZphgWuKDa3Bv7Xo4ikh8jR45k+vTpvPbaa1RVVTF48GAgDMRVWVnJ7NmzmTNnDnvttVeDhtd96623+NWvfsX06dOZN28eX/va13ZqmN7EcMGQecjgSy+9lAkTJjB//nzuuuuuvA8LXDDJPXEvx7ffBveaezkqwYsUno4dOzJs2DDOP//8WhdSN27cyJ577kmbNm2YMWMGb7/9dp37OeaYY3jwwQcBWLBgAfPmzQPCcMEdOnSgS5cuvPfeezz55JPbt+nUqRMfffTRDvs6+uijeeyxx6iqquLjjz/m0Ucf5eijj875NW3cuJEePXoAcN99921ffuKJJzJp0qTt8xs2bOCII45g5syZvPXWW0DTDAtcMMm9Oe/lKCJNb8yYMcydO7dWch87dizl5eUcfPDB/OlPf6J///517mP8+PFs2rSJAw44gOuuu277N4CBAwdy2GGH0b9/f84666xawwWPGzeO4cOHM2zYsFr7GjRoEOeeey5Dhgzh8MMP54ILLuCwww7L+fXccMMNfP3rX2fw4MG12vOvvfZaNmzYwEEHHcTAgQOZMWMG3bt3Z/LkyYwaNYqBAwfyjW98I+fj5Kpghvxt1SrU2FOZwbZtjRiYSMxpyN/CsTND/hZMzb257+UoIlLICia5N/e9HEVEClnBJPd83stRJG7y1RwrudvZ96ig+rnn616OInHSrl071q1bR3FxccYfB0l+uTvr1q2jXbt2Dd5HTsndzIYDtwFFwN3u/rOU9RcDlwBbgU3AOHff8TbiIpJ3PXv2pKKigsrKynyHInVo164dPXv2bPD2WZO7mRUBk4ATgQpglplNS0neD7r7nVH5EcAtwPAGRyUiTaZNmzbbfxkp8ZVLm/sQYLm7v+nuW4ApQK2RfNz9w6TZDoAa9ERE8iiXZpkewDtJ8xXA4amFzOwS4LtAW+DL6XZkZuOAcQC91IdRRKTJNFpvGXef5O77AT8Ers1QZrK7l7p7affu3Rvr0CIikiKXmvsqYN+k+Z7RskymAHdk2+ns2bPXmlndA0dkVgKsbeC2TW1XjU1x1Y/iqr9dNba4xdU7l0K5JPdZQD8z60tI6mcCZyUXMLN+7r4smv0asIws3L3BVXczK8/l57f5sKvGprjqR3HV364aW0uNK2tyd/dqM5sAPE3oCnmPuy80sxuBcnefBkwwsxOAz4ANwDlNFbCIiGSXUz93d38CeCJl2XVJzy9v5LhERGQnFMzwAykm5zuAOuyqsSmu+lFc9berxtYi48rbkL8iItJ0CrXmLiIidVByFxGJoYJL7mY23MyWmNlyM7s6j3Hsa2YzzGyRmS00s8uj5TeY2SozmxNNJ+chthVmNj86fnm0rJuZPWNmy6LHPZo5pi8mnZM5ZvahmV2Rr/NlZveY2ftmtiBpWdpzZMHt0WdunpkNaua4fmlmr0fHftTMukbL+5jZ5qRzd2czx5XxvTOz/47O1xIz+2pTxVVHbH9OimuFmc2JljfLOasjPzTfZ8zdC2YidMV8A/g8YZiDucCAPMWyDzAoet4JWAoMAG4ArsrzeVoBlKQs+wVwdfT8auDneX4f3yX8GCMv5ws4BhgELMh2joCTgScBA44AXmnmuL4CtI6e/zwprj7J5fJwvtK+d9HfwVxgN6Bv9Ddb1Jyxpaz/NXBdc56zOvJDs33GCq3mnnUQs+bi7mvc/bXo+UfAYsI4PLuqkUDiluz3Af+Vx1iOB95w94b+QnmnuftMIPWW85nO0UjgTx68DHQ1s32aKy53/4e7V0ezLxN+Jd6sMpyvTEYCU9z9U3d/C1hO+Ntt9tjMzIAzgIea6vgZYsqUH5rtM1ZoyT3dIGZ5T6hm1gc4DHglWjQh+mp1T3M3f0Qc+IeZzbYwWBvAXu6+Jnr+LrBXHuJKOJPaf2z5Pl8Jmc7RrvS5O59Qw0voa2b/MbPnzezoPMST7r3blc7X0cB7XvMLemjmc5aSH5rtM1ZoyX2XY2YdgUeAKzwMfXwHsB9wKLCG8JWwuQ1190HAScAlZnZM8koP3wPz0gfWzNoCI4D/ixbtCudrB/k8R5mY2USgGiiLFq0Bern7YYQRWR80s87NGNIu+d6lGEPtikSznrM0+WG7pv6MFVpyr+8gZk3KzNoQ3rgyd/8LgLu/5+5b3X0b8Hua8OtoJu6+Knp8H3g0iuG9xNe86PH95o4rchLwmru/F8WY9/OVJNM5yvvnzszOBU4BxkZJgajZY130fDahbXv/5oqpjvcu7+cLwMxaA6OAPyeWNec5S5cfaMbPWKEl9+2DmEU1wDOBafkIJGrL+wOw2N1vSVqe3E52GrAgddsmjquDmXVKPCdcjFtAOE+JMX/OAf7anHElqVWTyvf5SpHpHE0DvhX1aDgC2Jj01brJWbjN5Q+AEe5elbS8u4U7pWFmnwf6AW82Y1yZ3rtpwJlmtpuFAQf7Aa82V1xJTgBed/eKxILmOmeZ8gPN+Rlr6qvGjT0RriovJfzHnZjHOIYSvlLNA+ZE08nA/cD8aPk0YJ9mjuvzhJ4Kc4GFiXMEFAPTCSN2Pgt0y8M56wCsA7okLcvL+SL8g1lDGOyuAvh2pnNE6MEwKfrMzQdKmzmu5YT22MTn7M6o7OnRezwHeA04tZnjyvjeAROj87UEOKm538to+R+Bi1PKNss5qyM/NNtnTMMPiIjEUKE1y4iISA6U3EVEYkjJXUQkhpTcRURiSMldRCSGlNxFRGJIyV1EJIb+P5khchOBZkcnAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XmcFNW5//HPwzDsmwwYF4TB6FVAEXBEvYiIeg2uhGgMOLgb1JgYNbkRwUQvCS/XqMEYjcl1BSFc/Rk3lCRKgt7cqEAQVERQQVlUGAVEMDDw/P441dM9Q3dPzzDTPd3zfb9e9equqtNVT1d1P3X6VPUpc3dERKSwtMh1ACIi0vCU3EVECpCSu4hIAVJyFxEpQEruIiIFSMldRKQAKblLUmZWZGabzaxnQ5bNJTM7wMwa/NpfMzvRzFYkjC81s6GZlK3Hun5vZhPq+/o0y/2FmT3U0MuV3GmZ6wCkYZjZ5oTRdsC/gB3R+KXuPq0uy3P3HUCHhi7bHLj7QQ2xHDO7BBjr7sclLPuShli2FD4l9wLh7lXJNaoZXuLuf0lV3sxauntlNmITkexTs0wzEf3s/oOZTTezL4CxZna0mf3DzDaY2Vozm2JmxVH5lmbmZlYajU+N5j9vZl+Y2f+ZWe+6lo3mn2xm75rZRjO728z+18wuSBF3JjFeambLzexzM5uS8NoiM7vTzCrM7H1gRJrtM9HMZtSYdo+Z3RE9v8TMlkTv572oVp1qWavM7LjoeTszezSK7S3g8Bplrzez96PlvmVmZ0TTDwV+DQyNmrzWJ2zbGxNef1n03ivM7I9mtncm26Y2ZjYqimeDmb1kZgclzJtgZmvMbJOZvZPwXo8yswXR9E/M7LZM1yeNwN01FNgArABOrDHtF8A24HTCQb0tcARwJOEX3P7Au8D3o/ItAQdKo/GpwHqgDCgG/gBMrUfZPYEvgJHRvGuA7cAFKd5LJjE+BXQGSoHPYu8d+D7wFtADKAHmho980vXsD2wG2ics+1OgLBo/PSpjwPHAVqB/NO9EYEXCslYBx0XPbwf+CuwB9ALerlH2bGDvaJ+cE8XwtWjeJcBfa8Q5Fbgxen5SFOMAoA3wG+ClTLZNkvf/C+Ch6HmfKI7jo300AVgaPe8HrAT2isr2BvaPnr8OjImedwSOzPV3oTkPqrk3L6+4+zPuvtPdt7r76+7+qrtXuvv7wP3AsDSvf9zd57n7dmAaIanUtexpwEJ3fyqadyfhQJBUhjHe5O4b3X0FIZHG1nU2cKe7r3L3CuDmNOt5H3iTcNAB+A/gc3efF81/xt3f9+Al4EUg6UnTGs4GfuHun7v7SkJtPHG9M919bbRPHiMcmMsyWC5AOfB7d1/o7l8B44FhZtYjoUyqbZPOaOBpd38p2kc3Ew4QRwKVhANJv6hp74No20E4SB9oZiXu/oW7v5rh+5BGoOTevHyUOGJmB5vZc2b2sZltAiYB3dK8/uOE51tIfxI1Vdl9EuNwdyfUdJPKMMaM1kWocabzGDAmen5ONB6L4zQze9XMPjOzDYRac7ptFbN3uhjM7AIzeyNq/tgAHJzhciG8v6rlufsm4HNg34QyddlnqZa7k7CP9nX3pcCPCPvh06iZb6+o6IVAX2Cpmb1mZqdk+D6kESi5Ny81LwP8LaG2eoC7dwJ+Rmh2aExrCc0kAJiZUT0Z1bQ7Ma4F9ksYr+1SzZnAiWa2L6EG/1gUY1vgceAmQpNJF+BPGcbxcaoYzGx/4F7gcqAkWu47Ccut7bLNNYSmntjyOhKaf1ZnEFddltuCsM9WA7j7VHcfQmiSKSJsF9x9qbuPJjS9/RJ4wsza7GYsUk9K7s1bR2Aj8KWZ9QEuzcI6nwUGmdnpZtYS+CHQvZFinAlcZWb7mlkJcG26wu7+MfAK8BCw1N2XRbNaA62AdcAOMzsNOKEOMUwwsy4W/gfw/YR5HQgJfB3hOPddQs095hOgR+wEchLTgYvNrL+ZtSYk2ZfdPeUvoTrEfIaZHRet+z8J50leNbM+ZjY8Wt/WaNhJeAPnmlm3qKa/MXpvO3czFqknJffm7UfA+YQv7m8JJz4blbt/AnwHuAOoAL4O/JNwXX5Dx3gvoW18MeFk3+MZvOYxwgnSqiYZd98AXA08STgpeRbhIJWJGwi/IFYAzwOPJCx3EXA38FpU5iAgsZ36z8Ay4BMzS2xeib3+BULzyJPR63sS2uF3i7u/Rdjm9xIOPCOAM6L299bArYTzJB8TfilMjF56CrDEwtVYtwPfcfdtuxuP1I+FJk+R3DCzIkIzwFnu/nKu4xEpFKq5S9aZ2YiomaI18FPCVRav5TgskYKi5C65cAzwPuEn/zeAUe6eqllGROpBzTIiIgVINXcRkQKUs47DunXr5qWlpblavYhIXpo/f/56d093+TCQw+ReWlrKvHnzcrV6EZG8ZGa1/dMaULOMiEhBUnIXESlASu4iIgVId2ISaSa2b9/OqlWr+Oqrr3IdimSgTZs29OjRg+LiVF0LpafkLtJMrFq1io4dO1JaWkrojFOaKnenoqKCVatW0bt379pfkEReNctMmwalpdCiRXicVqdbPos0b1999RUlJSVK7HnAzCgpKdmtX1l5U3OfNg3GjYMtW8L4ypVhHKB8t/vBE2kelNjzx+7uq7ypuU+cGE/sMVu2hOkiIlJd3iT3Dz+s23QRaVoqKioYMGAAAwYMYK+99mLfffetGt+2LbNu3y+88EKWLl2atsw999zDtAZqsz3mmGNYuHBhgywr2/KmWaZnz9AUk2y6iDS8adPCL+MPPwzfs8mTd68JtKSkpCpR3njjjXTo0IEf//jH1cq4O+5OixbJ650PPvhgreu54oor6h9kAam15m5m+5nZHDN728zeMrMfJiljZjbFzJab2SIzG9TQgU6eDO3aVZ/Wrl2YLiINK3aOa+VKcI+f42qMixiWL19O3759KS8vp1+/fqxdu5Zx48ZRVlZGv379mDRpUlXZWE26srKSLl26MH78eA477DCOPvpoPv30UwCuv/567rrrrqry48ePZ/DgwRx00EH8/e9/B+DLL7/kzDPPpG/fvpx11lmUlZXVWkOfOnUqhx56KIcccggTJkwAoLKyknPPPbdq+pQpUwC488476du3L/3792fs2LENvs0ykUnNvRL4kbsviG7AO9/M/uzubyeUORk4MBqOJNye68iGDDRWY2jImoSIJJfuHFdjfOfeeecdHnnkEcrKygC4+eab6dq1K5WVlQwfPpyzzjqLvn37VnvNxo0bGTZsGDfffDPXXHMNDzzwAOPHj99l2e7Oa6+9xtNPP82kSZN44YUXuPvuu9lrr7144okneOONNxg0KH19dNWqVVx//fXMmzePzp07c+KJJ/Lss8/SvXt31q9fz+LFiwHYsGEDALfeeisrV66kVatWVdOyrdaau7uvdfcF0fMvgCXserf6kcAjHvwD6GJmezd0sOXlsGIF7NwZHpXYRRpHts9xff3rX69K7ADTp09n0KBBDBo0iCVLlvD222/v8pq2bdty8sknA3D44YezYsWKpMv+1re+tUuZV155hdGjRwNw2GGH0a9fv7Txvfrqqxx//PF069aN4uJizjnnHObOncsBBxzA0qVLufLKK5k9ezadO3cGoF+/fowdO5Zp06bV+09Iu6tOJ1TNrBQYSPWb+EJI9h8ljK9i1wMAZjbOzOaZ2bx169bVLVIRyZpU57Ia6xxX+/btq54vW7aMX/3qV7z00kssWrSIESNGJL3eu1WrVlXPi4qKqKysTLrs1q1b11qmvkpKSli0aBFDhw7lnnvu4dJLLwVg9uzZXHbZZbz++usMHjyYHTt2NOh6M5FxcjezDsATwFXuvqk+K3P3+929zN3LunevtTtiEcmRXJ7j2rRpEx07dqRTp06sXbuW2bNnN/g6hgwZwsyZMwFYvHhx0l8GiY488kjmzJlDRUUFlZWVzJgxg2HDhrFu3TrcnW9/+9tMmjSJBQsWsGPHDlatWsXxxx/Prbfeyvr169lSs40rCzK6WsbMigmJfZq7/78kRVYD+yWM94imiUgeyuU5rkGDBtG3b18OPvhgevXqxZAhQxp8HT/4wQ8477zz6Nu3b9UQa1JJpkePHvz85z/nuOOOw905/fTTOfXUU1mwYAEXX3wx7o6Zccstt1BZWck555zDF198wc6dO/nxj39Mx44dG/w91KbWe6ha+JvUw8Bn7n5VijKnAt8HTiGcSJ3i7oPTLbesrMx1sw6R7FmyZAl9+vTJdRhNQmVlJZWVlbRp04Zly5Zx0kknsWzZMlq2bFpXhyfbZ2Y2393LUrykSibvZAhwLrDYzGLXCk0AegK4+33ALEJiXw5sAS7MOHoRkSzbvHkzJ5xwApWVlbg7v/3tb5tcYt9dtb4bd38FSNvJgYfqv/45ICJ5oUuXLsyfPz/XYTSqvOl+QEREMqfkLiJSgJTcRUQKkJK7iEgBUnIXkawYPnz4Ln9Iuuuuu7j88svTvq5Dhw4ArFmzhrPOOitpmeOOO47aLq2+6667qv2Z6JRTTmmQfl9uvPFGbr/99t1eTkNTcheRrBgzZgwzZsyoNm3GjBmMGTMmo9fvs88+PP744/Vef83kPmvWLLp06VLv5TV1Su4ikhVnnXUWzz33XNWNOVasWMGaNWsYOnRo1XXngwYN4tBDD+Wpp57a5fUrVqzgkEMOAWDr1q2MHj2aPn36MGrUKLZu3VpV7vLLL6/qLviGG24AYMqUKaxZs4bhw4czfPhwAEpLS1m/fj0Ad9xxB4cccgiHHHJIVXfBK1asoE+fPnz3u9+lX79+nHTSSdXWk8zChQs56qij6N+/P6NGjeLzzz+vWn+sC+BYh2V/+9vfqm5WMnDgQL744ot6b9tkCuuqfRHJyFVXQUPfYGjAAIjyYlJdu3Zl8ODBPP/884wcOZIZM2Zw9tlnY2a0adOGJ598kk6dOrF+/XqOOuoozjjjjJT3Eb333ntp164dS5YsYdGiRdW67J08eTJdu3Zlx44dnHDCCSxatIgrr7ySO+64gzlz5tCtW7dqy5o/fz4PPvggr776Ku7OkUceybBhw9hjjz1YtmwZ06dP53e/+x1nn302TzzxRNr+2c877zzuvvtuhg0bxs9+9jP+67/+i7vuuoubb76ZDz74gNatW1c1Bd1+++3cc889DBkyhM2bN9OmTZs6bO3aqeYuIlmT2DST2CTj7kyYMIH+/ftz4oknsnr1aj755JOUy5k7d25Vku3fvz/9+/evmjdz5kwGDRrEwIEDeeutt2rtFOyVV15h1KhRtG/fng4dOvCtb32Ll19+GYDevXszYMAAIH23whD6l9+wYQPDhg0D4Pzzz2fu3LlVMZaXlzN16tSqf8IOGTKEa665hilTprBhw4YG/4esau4izVC6GnZjGjlyJFdffTULFixgy5YtHH744QBMmzaNdevWMX/+fIqLiyktLU3azW9tPvjgA26//XZef/119thjDy644IJ6LScm1l0whC6Da2uWSeW5555j7ty5PPPMM0yePJnFixczfvx4Tj31VGbNmsWQIUOYPXs2Bx98cL1jrUk1dxHJmg4dOjB8+HAuuuiiaidSN27cyJ577klxcTFz5sxhZbIbJic49thjeeyxxwB48803WbRoERC6C27fvj2dO3fmk08+4fnnn696TceOHZO2aw8dOpQ//vGPbNmyhS+//JInn3ySoUOH1vm9de7cmT322KOq1v/oo48ybNgwdu7cyUcffcTw4cO55ZZb2LhxI5s3b+a9997j0EMP5dprr+WII47gnXfeqfM601HNXUSyasyYMYwaNaralTPl5eWcfvrpHHrooZSVldVag7388su58MIL6dOnD3369Kn6BXDYYYcxcOBADj74YPbbb79q3QWPGzeOESNGsM8++zBnzpyq6YMGDeKCCy5g8ODQke0ll1zCwIED0zbBpPLwww9z2WWXsWXLFvbff38efPBBduzYwdixY9m4cSPuzpVXXkmXLl346U9/ypw5c2jRogX9+vWruqtUQ6m1y9/Goi5/RbJLXf7mn93p8lfNMiIiBUjJXUSkACm5izQjuWqGlbrb3X2l5C7STLRp04aKigol+Dzg7lRUVOzWH5t0tYxIM9GjRw9WrVrFunXrch2KZKBNmzb06NGj3q9XchdpJoqLi+ndu3euw5AsUbOMiEgBUnIXESlASu4iIgVIyV1EpAApuYuIFCAldxGRAqTkLiJSgJTcRUQKkJK7iEgBUnIXESlASu4iIgVIyV1EpAApuYuIFCAldxGRAqTkLiJSgJTcRUQKkJK7iEgBUnIXESlASu4iIgVIyV1EpADVmtzN7AEz+9TM3kwx/zgz22hmC6PhZw0fpoiI1EXLDMo8BPwaeCRNmZfd/bQGiUhERHZbrTV3d58LfJaFWEREpIE0VJv70Wb2hpk9b2b9GmiZIiJST5k0y9RmAdDL3Teb2SnAH4EDkxU0s3HAOICePXs2wKpFRCSZ3a65u/smd98cPZ8FFJtZtxRl73f3Mncv6969++6uWkREUtjt5G5me5mZRc8HR8us2N3liohI/dXaLGNm04HjgG5mtgq4ASgGcPf7gLOAy82sEtgKjHZ3b7SIRUSkVrUmd3cfU8v8XxMulRQRkSZC/1AVESlASu4iIgUoL5O7exhERCS5vEvuM2dCixawZEmuIxERabryLrm3jE4Bb9+e2zhERJqyvEvurVqFRyV3EZHU8i65FxeHx23bchuHiEhTlrfJXTV3EZHU8i65q1lGRKR2eZfc1SwjIlK7vE3uqrmLiKSWd8k91iyjmruISGp5l9xVcxcRqV3eJXedUBURqV3eJXedUBURqV3eJnfV3EVEUsu75K5mGRGR2uVdclezjIhI7fIuuavmLiJSu7xL7qq5i4jULu+SuxkUFanmLiKSTt4ldwhNM0ruIiKp5WVyB7jvvnC7vdJSmDYt19GIiDQtLXMdQF1NmwZbt8bHV66EcePC8/Ly3MQkItLU5F3NfeLEXadt2ZJ8uohIc5V3yf3DD+s2XUSkOcq75N6zZ92mi4g0R3mX3CdPDpdDJmrXLkwXEZEg75J7eTnstx+0bRuSfK9ecP/9OpkqIpIo766WAdhzTzjkEHjuuVxHIiLSNOVdzR1CFwTqfkBEJLW8TO76h6qISHp5mdyLi5XcRUTSydvkrmYZEZHU8jK5q1lGRCS9vEzuapYREUkvb5O7mmVERFLLy+SuZhkRkfTyMrmr5i4ikl5eJnfV3EVE0qs1uZvZA2b2qZm9mWK+mdkUM1tuZovMbFDDh1mdTqiKiKSXSc39IWBEmvknAwdGwzjg3t0PKz01y4iIpFdrcnf3ucBnaYqMBB7x4B9AFzPbu6ECTEbNMiIi6TVEm/u+wEcJ46uiabsws3FmNs/M5q1bt67eKywuhspKcK/3IkREClpWT6i6+/3uXubuZd27d6/3coqLw6Nq7yIiyTVEcl8N7Jcw3iOa1mhatQqPSu4iIsk1RHJ/GjgvumrmKGCju69tgOWmFKu566SqiEhytd6JycymA8cB3cxsFXADUAzg7vcBs4BTgOXAFuDCxgo2RjV3EZH0ak3u7j6mlvkOXNFgEWVAbe4iIunl5T9U1SwjIpJeXiZ3NcuIiKSXl8ldzTIiIunlZXKP1dzVLCMiklxeJnfV3EVE0svr5K6au4hIcnmZ3HVCVUQkvbxM7mqWERFJL6+Tu5plRESSy8vkHmuWuegiaNECSkth2rSchiQi0qTU2v1AUzR7dniMdQm/ciWMGxeel5fnJiYRkaYkL2vuv/rVrtO2bIGJE7Mfi4hIU5SXyX3NmuTTP/wwu3GIiDRVeZnc9016Ez/o2TO7cYiINFV5mdyvv37Xae3aweTJ2Y9FRKQpysvkPibqYX6PPcAMevWC++/XyVQRkZi8vFomdinktdeGQUREqsvLmrv+oSoikl5eJveiovCo5C4iklxeJnez0DSj7gdERJLLy+QOoWlGNXcRkeTyOrmr5i4iklzeJvdWrVRzFxFJJW+Tu5plRERSy9vkrhOqIiKp5W1yV81dRCQ1JXcRkQKUt8m9dWv46qtwB6bSUt2RSUQkUV72LQPQtSssXRruwLRlS5imOzKJiAR5W3Pv3h1WrIgn9hjdkUlEJM+Te2Vl8nm6I5OINHd5ndxT0R2ZRKS5y/vk3qZN9em6I5OISAEk95/+NNyJSXdkEhGJy9urZWLJffBgmDAht7GIiDQ1eVtz79YtPK5bl9s4RESaorxN7rGau5K7iMiu8ja5d+0a/pWq5C4isqu8Te4tWkBJiZK7iEgyGSV3MxthZkvNbLmZjU8y/wIzW2dmC6PhkoYPdVfduyu5i4gkU2tyN7Mi4B7gZKAvMMbM+iYp+gd3HxANv2/gOJNKTO7qQExEJC6TmvtgYLm7v+/u24AZwMjGDSsz3bvD+vUhkY8bFzoOc493IKYELyLNVSbJfV/go4TxVdG0ms40s0Vm9riZ7ZdsQWY2zszmmdm8dQ3QnhKruU+cqA7EREQSNdQJ1WeAUnfvD/wZeDhZIXe/393L3L2se7rOYTLUvTtUVISaejLqQExEmqtMkvtqILEm3iOaVsXdK9z9X9Ho74HDGya89Lp1C80wPXokn68OxESkucokub8OHGhmvc2sFTAaeDqxgJntnTB6BrCk4UJMLVb5/8EPQodhidSBmIg0Z7Umd3evBL4PzCYk7Znu/paZTTKzM6JiV5rZW2b2BnAlcEFjBZwoltyPPDJ0GFZSEp/Xtm02IhARaZoy6jjM3WcBs2pM+1nC8+uA6xo2tNrV7IJg69b4vIoK3XJPRJqvvP2HKlRP7rpiRkQkLq+Te2LPkKmujNEVMyLSHOV1ci8uhi5dQnJPdWWMrpgRkeYor5M7xP/INHmyrpgREYkpiOS+fn04aXr//eFWewBFRfE2d3VDICLNTd7eZi+mWzdYsSI8j10VM25c/ORqrJ+ZxPkiIoWuIGruid3U6KoZEZECSe7r14duCEBXzYiIQIEk9+3bYePGMK6rZkRECiS5Q7xpJtlVM2ZwyinZjUtEJJcKLrmXl8P554eEHuMODz+sq2ZEpPkouOQOMGtWvA0+ZsuWkPSV4EWkOSjI5J7q5OmOHbr9nog0D3mf3BP7l4lJd/JUl0WKSHOQ98m9XbswJCb3ZCdVE6W6LZ+ISKHI++QO8WvdY2JdERQVJS9vpqYZESlsed/9AITk/umn1afFuho499xdT666h5OrieVERApJQdTc+/aF116Dysrq08vLd03sMTq5KiKFrCCS+8iR8Pnn8PLLu86L9RKZjE6uikihKojkftJJ0Lo1PPXUrvMyObnaogWUlqoWLyKFoyCSe4cOcOKJIbnXbIap7eQqhNfEugZWgheRQlAQyR3gm98M/bq/8cau88rLQ/cDiV0SJKN/sYpIoSio5N6qFTzwQPL56U6uJtqxI1xh873vNWx8IiLZVDDJvVs3+Pa3Qw198+bkZdKdXE3kDvfdpxq8iOSvgknuEGrbmzbB9OnJ59d2cjWRu66kEZH8VVDJ/eij4bDDwgnUZGInV0tKMlveypXhF4Fq8CKSbwoquZvBmDEwbx6sWpW8THl56Kpg6tTMmmkqKmDsWCV5EckvBZXcIfyhCeDpp9OXKy8PV9e4w+WX134lTSzJm+maeBFp+gouuR90EBx4YO3JPdFvfgOPPpp5+ZUrwxU1SvQi0lQVXHI3C7X3l14KJ1czVV6e+dU0EL+scuVKNduISNNTcMkdwjXv27fDL39Zt9dNngzFxfVbZ0WFavMimVixItcRNA8Fmdz//d9Dop00CaZMyfx15eXw4IOZX01TU83afM1EP21aGFdfNlIIbroJXnihbq+57z7o3RtmzkxfLpM/HCazdWv18Xnz4MUX67esRC+/DN/9bur/0DRJ7p6T4fDDD/fGtG2b+xlnuIP7RRe5b9lS92Vcfrm7WVhGYw8lJe5Tpzb8dpDGt3On+6OPun/xRRhfvdp9x47MX79pk/tXXzVObI3l3XfD57ZFC/cHH8zsNQsWuLduHV53/PGpyz37rPuee7rfe2/dYvrrX92Li91fey2Mf/KJe7duYUjcH1u3hlgWL85suXff7V5UFOJuCt9RYJ5nkGMLNrm7u2/f7j5xYniXZ55Zty9czNSp7r16ZSfBJyb5qVPD89j09u3DuFmIpyl8yJqrxx93f+yx+PicOWEfjR/v/uab7i1buv/mN5kt64sv3EtL3Xv0cH/kEfe333b/17/qH1vNz/iOHeHgU1Nl5e6t57bbwnv+938Pn8m//jVMX7nSfdw49wceqF5+61b3gw9233df9yuvDK996SX3b37T/dxzwzZ1d//tb8PyWrRw797dffPmzGMaOTIs93vfC+/5zDPj35/XXw9lVq92/9rX4tPHjHH/9a/dZ86svu22bw8VxMcfD+XOOCMccMaMyTye7dszL1sXSu4Jbr89vNMTT3Tv29f9vPPcFy6s+3KmTnVv1y57ib6utf3EA1GsplFSEg4M+oXQMCorQ3Jo3dr9gw/CtKuvDtu2c2f3006LJz139z/+0f2669xvvjkkuJp+8pNQvk+f+D7ad9+Q5LZtC2V27HC/9Vb3IUPcBw0Ktc5Nm9yPOsr9iCPC+r/6yv2qq9x79nT/7LPwum3b3P/t39x/8IMw/uWXIeHs2BFPyn36uL//fur3u3mz+4sv7hr7Mce4H3ZYmP/1r4cD1LXXxmvmRUXuTz0V3vv117v/8Idh+uzZ7h9+GNZt5t6pk/tee4V5EyaEmveIEe5/+UuYdtttme2XlSvDAaG4OBwUZs4Mr//Rj8J6fv7zUO6aa0JsjzwS4orFG8sP777r/ve/h9p+p07ubduG7fzVV+7nn+++xx7xpP3ZZ+7r1sVj2LYtHChuvtn9nHPcW7UK7z/ZwXV3KLkn2Lkz1Cbat3c/4YTwWFTkfued1Tf8pk2hFpZuZ8QSaKx2kesE31AHhmx4/fWG+aCnqo1mYufO2pvoFixwv/BC9w0b3P/0p7C/Z80K8158Mb4Nv/OdsLwDDnDff//49L33Do9Tp4bHli3D46BBIbG5h8Rw//1h3oUXhoPG3/4Wks7mfBDcAAAMIklEQVTRR4fyBxwQEtApp4TxI45w79DBffTo0GQBIclCSOKx9U+YENbx6KPxaY88EuIaPjz86gD3sWNDArriipCYpk8P34GYO+90b9Mm/l4/+ywkuLvuCp/9G24I5V55Jd58OXas+1tvuR90UBhv0SI+74IL4sseOTJ8BhcuDL8ghg8PZXr0cF+/PpT5j/9w79rVfdGi5Pvp7rvd//u/wz64+uqwrjvvDMtp2zYcfLZvdy8rCwfGdetC5ezcc+PL2LgxNN/cd1+84lZcHA5Yl1wSmo9Wrw5l/+d/wvyXXw6fwYEDQy65886Q/K+4Ir69O3VyP/bY8HzixPq1GqSi5F7Dzp3xhPDZZ+HnIIRa/JYt7p9+Gr58EI76iTZvdr/nHvclS6pPb2o1+foOmZxXSDwY3HGH+9y58e2waFGoqcQSQ+IBMNaE9Oc/h+U8+mjy/fP886EWm2rf3XST++TJIants0/4si5fXr3cQw+FGm+i5cvD6265JSznnHNCAklMYom+/NL9wANDrOeeG2qksQT92GPxSsJ//meYftNN4fE3v3E/7riQfP/+93iS6NEjLPOpp8K8U08N695vv1CmX7/w2av5fp9+OiQnCAn2jjvC9B/+MCz3wAPdBwwI0x54ICS2UaPczz47fCbXrg3zDzoobC9w79gxHlffvuGAcv754f1cemmYN2CA+5o14QBXVOR+0kmh5g/VmzMglImZPdt9/vz4+JIlYfvNmxfKXXed++efV9/Oifvg88/dL7443l7u7v7OOyH2Ll1C8r7ttjDNPd4UBuGXDoSD3tatIbGC+//+byh7/fVh+5x8cvhMvv128n2/Zk34rJSXh4Rf04YN4XNwzTXuTz4Z338QavQQfil8+WU4YO3YEQ4QEA4Sjz0WtsXuVnCU3GuxY4f7pEleVdPq1Cl8iQ44IHyIYz9td+4MH5rYB+m009w//ji+nMREVlJSvZ1cQ92H2IGmc+fUB86iojCvbVv3ww8PbaGJ87t1C8v52tfizVMQmiJiz0ePjie92C+wXr1CkwDEHyEk2mOPDcts2za0u27e7H7kkfEyH34YkkOslhlLzIknGydPDtNiJ/qfe67uX/QlS+LrvO+++PSVK0MtNdYEGRsuucT9mWdCEvrnP0N7NITk5B6STazsN74REn379uE7sdde7hUV4bty2mlhm7/wQqjoXHFFwzc3JLNiRWgW6dAhHuewYeHg1ru3+5QpYVvfe2+8GevOO8N3O+aVV8Lr2rQJB/ndceaZ4XOw114hV2zfHg5ssXMHlZXVy+/c6f7731ePv29f92nT6h+DknuGnn021HjGjXP/v/+L11j69w9flNiX4brr3H/xi/AB6dYt1HQmTgw79J57QsLfti2UHzw4tL317Jn7ZKmh4YdsX0GV7FwK5OZXY+xAWPNcTiblYxWfVOeDkr33RB99FGrvsTb6n/wkvl0S90mydXboEH4B1Fx/XS5SmDo1NBPF1tO+ferX1LwgIhZf+/bxZrr6XhjRoMkdGAEsBZYD45PMbw38IZr/KlBa2zKbSnJPZsYM90MOie+Ys8+Ot5m9+WaoxXTsGD4ksZ+qLVrEP2itWoUTVf37h/GLL3a/8cZwdP/Od+LtmBB+nieeTNOgQUPzGdq1q3uCb7DkDhQB7wH7A62AN4C+Ncp8D7gvej4a+ENty23Kyd09/JxasyZcqpbqUrLYz8DFi0O73jHHuD/8cDjT36ZNuPRr+vTkr9+wwX3p0tA2t2lTvM2u5jB0aDj7HquNtGqV+w+kBg0aGm7o1atuuakhk/vRwOyE8euA62qUmQ0cHT1vCawHLN1ym3py312ff75r+1s6H34YTppddlk4q//BB6GmX/OkoXs40sd+mib7OdqzZ2iz7tSp+k/ObP4pS4MGDZkNZnXLLZkmdwtlUzOzs4AR7n5JNH4ucKS7fz+hzJtRmVXR+HtRmfU1ljUOGAfQs2fPw1euXJl23VJ/lZWhm4MWNTqYmDYt3GFq5UooKgr3jDULHzMRyb5everW346ZzXf3strKZbVvGXe/393L3L2se/fu2Vx1s9Oy5a6JHar3Y19ZGR537kxft5g6te797bRvH39NUVF4rK3PfJHmpl270GFhY8gkua8G9ksY7xFNS1rGzFoCnYGKhghQci9296q6/NjcvDn+mkwPIrEDSa9e4UDQq1cYT3egiR3ASkrCYBZ/DqkPKLHX9eoVbtYS6+45kwNQ7LVN9WCVeGAthPdTqEpKwm0/y8sbaQW1tdsQ2tDfB3oTP6Har0aZK6h+QnVmbcst9DZ3kd2R7PLHXr3CeZOafxDLRVx1uXywZvnallHzMsJUQ+L/E5Ito+b/T2LrS9yGiZdL1uXPfIn7J9XrEi/nrOt2S4eGanMHMLNTgLsIV8484O6TzWxStJKnzawN8CgwEPgMGO3u76dbZllZmc+bN68ehyMRkeYr0zb3lpkszN1nAbNqTPtZwvOvgG/XNUgREWkcBXmzDhGR5k7JXUSkACm5i4gUICV3EZEClNHVMo2yYrN1QH3/otqN0MVBU9RUY1NcddNU44KmG5viqpv6xtXL3Wv9F2jOkvvuMLN5mVwKlAtNNTbFVTdNNS5ourEprrpp7LjULCMiUoCU3EVEClC+Jvf7cx1AGk01NsVVN001Lmi6sSmuumnUuPKyzV1ERNLL15q7iIikoeQuIlKA8i65m9kIM1tqZsvNbHwO49jPzOaY2dtm9paZ/TCafqOZrTazhdFwSg5iW2Fmi6P1z4umdTWzP5vZsuhxjxzEdVDCdlloZpvM7KpcbDMze8DMPo3uIhablnQbWTAl+swtMrNBWY7rNjN7J1r3k2bWJZpeamZbE7bbfVmOK+V+M7Prou211My+0VhxpYntDwlxrTCzhdH0bG6zVDkiO5+zTPoFbioDGdysO4ux7A0Mip53BN4F+gI3Aj/O8XZaAXSrMe1WYHz0fDxwSxPYlx8DvXKxzYBjgUHAm7VtI+AU4HnAgKOAV7Mc10lAy+j5LQlxlSaWy8H2Srrfou/BG0Brwn0g3gOKshlbjfm/BH6Wg22WKkdk5XOWbzX3wcByd3/f3bcBM4CRuQjE3de6+4Lo+RfAEmDfXMSSoZHAw9Hzh4Fv5jAWgBOA99w9JzfSdfe5hHsPJEq1jUYCj3jwD6CLme2drbjc/U/uXhmN/oNwN7SsSrG9UhkJzHD3f7n7B8Bywnc367GZmQFnA9Mba/2ppMkRWfmc5Vty3xf4KGF8FU0goZpZKeFGJa9Gk74f/ax6IBfNH4ADfzKz+RZuSg7wNXdfGz3/GPhaDuJKNJrqX7hcbzNIvY2a0ufuIkLtLqa3mf3TzP5mZkNzEE+y/daUttdQ4BN3X5YwLevbrEaOyMrnLN+Se5NjZh2AJ4Cr3H0TcC/wdWAAsJbwkzDbjnH3QcDJwBVmdmziTA+/AXN2DayZtQLOAP4nmtQUtlk1ud5GyZjZRKASmBZNWgv0dPeBwDXAY2bWKYshNbn9lsQYqlcisr7NkuSIKo35Ocu35J7JzbqzxsyKCTttmrv/PwB3/8Tdd7j7TuB3NOLP0VTcfXX0+CnwZBTDJ7GfeNHjp9mOK8HJwAJ3/wSaxjaLpNpGOf/cmdkFwGlAeZQQiJo9KqLn8wlt2/+WrZjS7Lecby8AM2sJfAv4Q2xatrdZshxBlj5n+ZbcXwcONLPeUe1vNPB0LgKJ2vL+G1ji7nckTE9sIxsFvFnztY0cV3sz6xh7TjgZ9yZhO50fFTsfeCqbcdVQrTaV622WINU2eho4L7qa4ShgY8LP6kZnZiOAnwBnuPuWhOndzawoer4/cCDhZvbZiivVfnsaGG1mrc2sdxTXa9mKK8GJwDvuvio2IZvbLFWOIFufs2ycNW7IgXBG+V3CEXdiDuM4hvBzahGwMBpOIdwofHE0/Wlg7yzHtT/hSoU3gLdi2wgoAV4ElgF/AbrmaLu1ByqAzgnTsr7NCAeXtcB2Qtvmxam2EeHqhXuiz9xioCzLcS0ntMXGPmf3RWXPjPbxQmABcHqW40q534CJ0fZaCpyc7X0ZTX8IuKxG2Wxus1Q5IiufM3U/ICJSgPKtWUZERDKg5C4iUoCU3EVECpCSu4hIAVJyFxEpQEruIiIFSMldRKQA/X+sR7oDd7vGeQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "acc = history.history['acc']\n",
    "val_acc = history.history['val_acc']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(len(acc))\n",
    "\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.legend()\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<div class=\"alert alert-block alert-info\">\n",
    "<div style=\"direction:rtl;text-align:right;font-family:B Lotus, B Nazanin, Tahoma\"> دانشگاه تربیت دبیر شهید رجایی<br>مباحث ویژه - آشنایی با بینایی کامپیوتر<br>علیرضا اخوان پور<br>96-97<br>\n",
    "</div>\n",
    "<a href=\"https://www.srttu.edu/\">SRTTU.edu</a> - <a href=\"http://class.vision\">Class.Vision</a> - <a href=\"http://AkhavanPour.ir\">AkhavanPour.ir</a>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda env:virtual_platform]",
   "language": "python",
   "name": "conda-env-virtual_platform-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  },
  "nbpresent": {
   "slides": {
    "300ee14f-a043-486e-b274-7ff253907cd7": {
     "id": "300ee14f-a043-486e-b274-7ff253907cd7",
     "prev": "cb74e0bc-4513-4d13-b7f1-14c3078a7927",
     "regions": {
      "26dc3f39-a230-447c-af4c-f5e5b2fb7835": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "c58440a5-3f8f-4f37-9c79-6bf766209406",
        "part": "whole"
       },
       "id": "26dc3f39-a230-447c-af4c-f5e5b2fb7835"
      }
     }
    },
    "878aa53a-1444-4100-8f50-7a408191c579": {
     "id": "878aa53a-1444-4100-8f50-7a408191c579",
     "prev": null,
     "regions": {
      "a6c6843a-5ea6-4fbc-b890-3b4b8ae475b3": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "588ee1fa-64b5-453b-ade7-8e6b2515821c",
        "part": "whole"
       },
       "id": "a6c6843a-5ea6-4fbc-b890-3b4b8ae475b3"
      }
     }
    },
    "96ffe88e-7b50-43de-afdd-942e564f4e3e": {
     "id": "96ffe88e-7b50-43de-afdd-942e564f4e3e",
     "prev": "878aa53a-1444-4100-8f50-7a408191c579",
     "regions": {
      "b7e52e12-489a-468d-b10c-af2024fd2856": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "de829a92-1fb6-44ad-a2c6-fc1001e1f6e1",
        "part": "whole"
       },
       "id": "b7e52e12-489a-468d-b10c-af2024fd2856"
      }
     }
    },
    "cb74e0bc-4513-4d13-b7f1-14c3078a7927": {
     "id": "cb74e0bc-4513-4d13-b7f1-14c3078a7927",
     "prev": "96ffe88e-7b50-43de-afdd-942e564f4e3e",
     "regions": {
      "444878ee-68f3-4abb-acff-a7079b21e86d": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "25f3f538-1ee8-4d98-a6bb-14cbeb7a702d",
        "part": "whole"
       },
       "id": "444878ee-68f3-4abb-acff-a7079b21e86d"
      }
     }
    }
   },
   "themes": {}
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}