t81_558_class_09_5_transfer_feature_eng.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://colab.research.google.com/github/jeffheaton/t81_558_deep_learning/blob/master/t81_558_class_09_5_transfer_feature_eng.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# T81-558: Applications of Deep Neural Networks\n",
    "**Module 9: Transfer Learning**\n",
    "* Instructor: [Jeff Heaton](https://sites.wustl.edu/jeffheaton/), McKelvey School of Engineering, [Washington University in St. Louis](https://engineering.wustl.edu/Programs/Pages/default.aspx)\n",
    "* For more information visit the [class website](https://sites.wustl.edu/jeffheaton/t81-558/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Module 9 Material\n",
    "\n",
    "* Part 9.1: Introduction to Keras Transfer Learning [[Video]](https://www.youtube.com/watch?v=WLlP6S-Z8Xs&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_09_1_keras_transfer.ipynb)\n",
    "* Part 9.2: Popular Pretrained Neural Networks for Keras [[Video]](https://www.youtube.com/watch?v=ctVA1_46YEE&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_09_2_popular_transfer.ipynb)\n",
    "* Part 9.3: Transfer Learning for Computer Vision and Keras [[Video]](https://www.youtube.com/watch?v=61vMUm_XBMI&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_09_3_transfer_cv.ipynb)\n",
    "* Part 9.4: Transfer Learning for Languages and Keras [[Video]](https://www.youtube.com/watch?v=ajmAAg9FxXA&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_09_4_transfer_nlp.ipynb)\n",
    "* **Part 9.5: Transfer Learning for Keras Feature Engineering** [[Video]](https://www.youtube.com/watch?v=Dttxsm8zpL8&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_09_5_transfer_feature_eng.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Google CoLab Instructions\n",
    "\n",
    "The following code ensures that Google CoLab is running the correct version of TensorFlow."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "    %tensorflow_version 2.x\n",
    "    COLAB = True\n",
    "    print(\"Note: using Google CoLab\")\n",
    "except:\n",
    "    print(\"Note: not using Google CoLab\")\n",
    "    COLAB = False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 9.5: Transfer Learning for Keras Feature Engineering\n",
    "\n",
    "It is also common to use transfer learning to generate features from complex input.  We will see this technique in later modules when we make use of Word2Vec. In this section, we will learn that we can take a pretrained neural network and strip off the dense layers.  The output from the final convolutional layers becomes vectors of features about the source images. These vectors will contain information about the subcomponents of the images and might be useful as input to other models.\n",
    "\n",
    "\n",
    "The following code extracts 1,024 sized vectors of features for images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\jeffh\\miniconda3\\envs\\tensorflow\\lib\\site-packages\\keras_applications\\mobilenet.py:207: UserWarning: `input_shape` is undefined or non-square, or `rows` is not in [128, 160, 192, 224]. Weights for input shape (224, 224) will be loaded as the default.\n",
      "  warnings.warn('`input_shape` is undefined or non-square, '\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'___________________________________________________________________________________________'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOAAAADgCAIAAACVT/22AABL4klEQVR4nO29edxsV1Um/Ky19zk1vfOdb3KT3AwkBEgCYUgIiLSIzEgjKMokLWD7/ZpBW9tutT9tu9Xu9uvWdgC1RVpBREUFhGbGBJCQAIEAIfPNeJM7v2NVnbP3Wuv7Y59zqt4L2rYSUu+99fwg933rrTp1qs5z1l7rWcMmM8MUU0wq+OE+gSmm+PswJegUE40pQaeYaEwJOsVEY0rQKSYaU4JOMdGYEnSKicaUoFNMNKYEnWKiMSXoFBONKUGnmGhMCTrFRGNK0CkmGlOCTjHRmBJ0ionGlKBTTDSmBJ1iojEl6BQTjSlBp5hoTAk6xURjStApJhpTgk4x0ZgSdIqJxpSgU0w0pgSdYqIxJegUE40pQaeYaEwJOsVEY0rQKSYaU4JOMdGYEnSKicaUoFNMNKYEnWKiMSXoFBONKUGnmGhMCTrFRGNK0CkmGlOCTjHRmBJ0ionGlKBTTDSmBJ1iojEl6BQTjSlBp5hoTAk6xURjStApJhpTgk4x0fAP9wlsPZjZSXuYE1Hz3+Y51X/TM6u/0/hzpviHgKb7xf9DkEj5LWFYOs635KxOB0wJ+n/ASXyKMa6srBw/fvz48eMnTpxYXl5eWVkZDofD4TCUQxExgJm991mWdTqd+bm5pcXFbdu279ixffv2HfOLSw/jZ9mKmBL074AZxnh577333n77rQcOHLj33vuPHTu6trYcyxCjqkRV1eprNJiZChGZQSQSp7Xft1qtbre9Y8fOc88978qnPPniR17MRMTM7NLyP8XfhSlBvznMVERiCEeOHP7MZ/72C9dff+jI0RjKmn9mRGbkIAYYsaqQSjAmKIxURFXATJzcAiJm71yWZdsWl5729O98+tOf7r0jQp53yHnv3HTd/6aYEnQzDCIhxKAhSIwPHj78oQ99+Ktf/UpZFIMixFhCRMzUSGIEgc2MDMSmyoCRM4KZqpiqsmODMdI3TADyLO/meXumd+VVVz3vuc/pdTsGEDsi553PMu+zbGpTxzGN4iuoagxBYhQNRGrwx5ZPfOGLn7/1tls3+v3BcBjKUsogMFFl02hGIAYpJUIREVEsjEkJMCViVSA5AAQ1AxCKcuCoPRx87EMf2VhZ/b7vf8nsbA9mRFKWZVkSM2V5K8/bzO5h/kYmA1OCQiTGGGIMMDARO8fkNvrrd91113XXf+HIkSNra2shBIkipmSG9A8TDGwQpAUfDLBBCZYMKrOZEeDYiWn62cgQ3TBKL8qnP/3pYPLyH3pZq9X2PvPOiYqqDQf9YjjMsnbeanl/ul+g0/rzi8QYo8RARExkUIMxcwxy++13feADH7zttlvXN/plGUwNZmbKTCqqZid5jMws1VKehE+nEsmUyQeOSRVVwIxUA7GPZTkYDj72sY8zu1e+4gczZpB33puqqplRWQ5C6HvfarU73mff9u9mUnB6+qCmKqEsYwxEiZykKmbqHGsMd9554O1/+M4bPv+FYTEsRVRMNBJAIDAZzBGZgZlVtRFHDQQQk6oBRkQKcgQyE2dqZpEAYzMlEENBZOy898999rN/5HWvXVxcUBEAqqoKwMxExIiQt9qdzgzz6Zj2Ox0taFkWoSwAMHPyHQFzjsw8gTb6q3/53vd/6lOf0ihFKFUNCrAAzMRmyuwMYKLIpgBDAQCUZCmBJbkJAJGokmmk9JAZYCYc2UwjgVJs/6d/8u75+YUf+ZHXsOMkuzqCEky9cwZoUQyLsuy0251O73QLoU4vgorE4XCoIo4dMYjMYESJbw6wsiw+efU173vfB/r9jRCDc0xmHgpzqiImRhRiJCYidgYmUiYFQI6hVbrJDIYUIJlCWVWVxEzFVMRYTAhgxxKlCME790u/8svs+HWvf10MUdQQgxGIHBGZkWc2s2F/oyyK3sys9/nD/UV++3DaENSsKIuiGBLBOSayZIiYQMwwJmOQnDh6+M/+8B0P3nWgH4PP82F/kLVaeauTZ5xlvtfplmUJZuecqgYJAIjZQMRKAGBQEGCqClOjKMJmqhJViEhUTYOpDQb9tbWNPTt3XnTmvoOHD198yaV33X7n7bfcetbZZxOYvIsxxBAcV8KTQdm5KLa6strptDrdmdPElJ4WPqiKDAcDtchMdWEHAEfEzGZgAqBqJhLD8pETx44dvf7aayH0xS9+aXHntk9d8+mV9RVjuvWOu/efc0YRhJmz3GfexygxBjITVQE5AhuiqYEgQtAgiCl2D2VZDPMs72/0z9ix64XPf87a2saznvucJzzpcmPavXuviA6KwmAxljBy3pkhhAAgz7MqOwAzhYhmWdbrzbrTIMY/9QkawmA4KAjOeaTgJa3DzMzszEhVk2dJBJD6rE2bqhBl9cTy8tFjdx84cP1nrnUd/2v/4y17du8+ePRQUQwX5uZnep0yRHMeUpZRYlQzhSGKxFAQZ6Esj5848eTHP3Hj6JGXv+qVj7r0kv3nnnv+oy8EcgASAxlCGJpBiQ1GJgnDYdHrzkSJZsE5T8TVtUoJVXC7N9dqtx6OL/Xbh1OcoEUxKIvCMRM7IgMUYMCYPRGLlAAxE5EDsZmSmZgCSMJ60pK8T+usAyDlxp233SEx/uk7/vTzn/v8ug2u/fyXLr304lAWUbTfH6pqjEGiRHDb89333Puyf/6ixz3m0qd/93ef/Yj923bsBBwsDgZDVSMzctUloKQDVDUAZmZRIkQNIEcWhAjkvAGAEaCqItTt9Trd7sP3BT/kOGUJaobhcBjC0LFjx6jCaiXyKXrXGKJKnufMZMqJjpaKOFOgA1jSLs2oCsKZncvzZLQsDAY3fOFLH/7AhyLFX/7P//0RFz1iYWHu4MH7Qyj7w8HasbUfe/2/eOxjLnn2C1+w88x9gEiIIZamBiIiBkCU7LcAAKi+HMRMqZhUVUIIJuoyr6JqwuyS3pQunEjM8/bM7DwRnZJe6alJUDPt99dV1DlOhcRNTTGzC2VBROy881We3EBkZjAQEcjqS53cvpquqOuVjGAK8s75PAPcg/cceM+7//Jjf/PxL99809r6+vLhY0+76sk/8JLve8krXjG/tE2kLAZDdmAyAwNkRgAIZDAzqSr6zGCUaqIIAJJRj4CYSEyVfABUmSvtFgCMQ4hZK5ubnyc6BbOjpyBBVa3fX1ONmc+oSpNzLaezqhSDjVanyz6HabKbRiCzJKETu2Q8AQBkMDSWLVHCDDA1RxZNRImyvOV9/t53/clPvOknYoZf/oX/9Jzvff78tm0a+sMyOOeR2AcHKGpJP6WmACUiNSNVIpa0yteo81VmqsOiVDNHBCiM2DFXyj+ilC5vzc0tnXpi/qlGUDNdX98AzDtHSTsnIzDgYijV1GeZd56IzDQt6EyUzChXQhGTWUNNq6X8lDQCyExTIik5AESkUVRiZ2bhxus+d2LlxNO++1lRhlIEpiS1OgWlNzFTMwO5pNmnDJRVwn5yQpwZTA2kgKKiMokJmUQRieIcK3EMIXeOHcFg4CjifD4/v3CKcfSUIqiZ9fvrpuqcJzaCADmREchAG2vLLnPtmQUWJU5qehWdNKE9Gh+0PqKa0dglp01vJ2mxNhMyDSGsF+Vst+t9vuloAFDxr34Vo4qHDHCoTsUMoxsDhlRiBTDBAZaulUqQKKJKTCKSO8feqYEVUYPL8vmF7adSaekpdbf1+30Vdc6h4p5nNoBDkBjL2bn5bneeRMCVq5deldiJb9L1VqlR6XFmZiIee6TugiMiMmLO8jDo9zc2fJY17CQyIq0MJwjmYK7O3Ve5T0AAQUXY2l6QEjHMV85qkkHBzC7zDKiZOu9jjEVRmBnImFxZxpXVlVPI5pw6BLVhf8M0eO+Sy8kEhiP4EIqN/nqWt1JOM4UjJ3UaJaaNaDpKWALJDWBOlNRKgAKI6i46I2JmDyNmZ2pUUzelAZLqSuQMMFIw1a5liuXNwEoMMoYk04pEaFPnyAA1TWUqBDGYErJWi0BSBpdl7H1RFMOyFJjzLhbr62snvu3f/0OFU4Sgw+GglJI4A5gBMBt5AFECs5tfWKhLNarCDgLIRrz8xk5iJKbWnG5W/NHjibVEIE48Y+eGa8sxFKjNp5mBGOBKVAKN+RREtTCLSvlkM28gMzVTgImUEDwTmakqVTETpQRt1spd5ouy9M5nWRaTIKWR2Q2H/f7G2rfli3/IsdUJagBCKENZZuwdKXNlAh1Tf315WPRTLjsZsfo1VkVFADUr6hhZm6M3+lTzYDp4HTClE0hPUBXpLuzozs6pSgq/KlZh8zGbZnk0R9bqvVgJUr+pmTpVgkUmn9hcG2MiYhhled5utYb9gXNZO++YmYgmag821opi+NB97982bHWCQjWWRd85B2IQJzXTgUIsW71utzMDVaaR1UI9UkGbBA6Amk8nHbyavFC/pBnEUAvsVXCjlhL6PFhd6a+vp8rRivE6Imf6qQ6aqoMSFLVcYJV1ZBAToFAjUrCREKBGICPSkUOixs7l7Va/v2HQPM9VNcaUEbPV1eUYwkN+AR5ibG2CmtlwOAAcVRVEKiJGWF5dVhHncqLG4UPFv1q3H1u3bfyH6lezxk/E2DSRhtWjQAng5DOQOs95K6/JmfxTGI182bHjV+eTnN1E91SpLMwwrs9q3AMZt7tJ6jIzc47b7bzf3yjKIZOXGMtQqKlKXF1d3uoqzdYuhwllUFFiVjMySlrNYFi2Wq0sq7SeTbF5s2qbaYpfxqL4BOKUF6WkN43oW5NVU3dRenKtYQKAaavdYXaqYrV+R/XohyYsIyI1hYFhBqrSRjCgbq1TgBzABkm5A9TphPqdGCQEV2di4ZzvzczcdONXYpTFxfn5hcXoM8eA6Mb66szs/EN7GR5KbGGCioSi6APQqKLa7/cfePCB4WB4wXnnbNuxQyWFLtWTG+tFRFa3VJykK43zL6U/R1ERgMp2oWGtbZbxI2z5+PH5paW5hdT2sel9MWaqyWhkVlHHXs0DRGYKU04BPoyq11KqdAE0uQiJ8GYmIjOzs+Vg8KY3vPG8iy956lOfeuljL9m7e/fMTFdWVvJWu64f2HrYqgQ1s35/IKKmury8/OUvf/mzn732KzfeyKZve/vbGE4gbIxRcrJ6mW42qw0/UhOmNdYxLb1NKFRb05SZGon5RDBVUyMDuNXr9mZm6v4OY7Jx81zdAKrVg2ZIddNVrWflU9SOav2q6gQaQ6vV0wGYJp2BmQf9/qWPvXRpYf6zn/vcwfvvuummyx5z2ePOP/+sc885n9nt2r2HtmaGaasStCiKsiyOHzt2+223f+GLX/zqV75y4PY7iuHwX7z65Ys794QQKPX3gFLAcrIUb4ZK0bSkAY2bWN78NANEtaohAmpWV0VPyTk0g0bL81yTc2lavXqzC9i0w9VCwshyJ3G1eh5Z8lmSMmUmI9tZPa3xRyvpKYQwO794yaWXfP2uB9b6w6/fcvPy2tp9d+87/Mgjlz32spnZuZnZ2W/9ZXjosfUImph09NjRW2+5+Ss3fvnWW2994IEHDh06PCyHsSx27d6T+WxQFs6ovobfZB1PvzARbNOhT34O6jCImRr3tBbqx8JyJjNCXFte8Vmr15tlqMHEqE64A02UlrKZlRElAjeKQhXAAdEAGNeZAkq1qlUlwOgEFY4BBqJFUwX5RzziIiv/QkJ3MBisLi/fo7a+sn7fwfuvOH7iGc/47vFbb6tgixE0XZyv3fTVT11z9T13333o8KHV1ZXjy8fX1tZDWc62/SMueqSZEQMnBeD1Aj1CXenUHLl+uApotHZYK3GqZiSNOQbJDBIRQR3Z3Pzc3NycqgKkqYoUlvzRcdIrweqSO6ttO8ZkV+9ItYmuKh81nUIlkZpLIpdCzZRgZGYa9p+/v5XnRRnaUQb9PsyEaCBhbWV5ptd98lVPPfn2m3hsJYKmL/eBg/e998///PDxY6srqxvrG2trG2trG0VZglyEo9rROil4TyFRHW188/z7pvdC42SOnMgxfR6NzWtiJlUV1XJYtLteLM2/UYzEqNojaOTXypEdyfiNNW/eyEZ/TOWjrCaAEvn6gOkYINAwhF27ds3Mzq4VZSyDmBWhXF1dZaajKn/5V3+1/9zz9uzZu7U4uvUc5w/97/99z333LK8sr62t9/v9/vqGFBFqZRnOOmv/3r1nioTUVNRc26Ya9BvRuJ6N/WxCperPTca9Npla3yqVZVVThRkUvH7ieAhl05rXEM6w2ZWoBQEzUzM2TUbezNREUwVglZU1QImVyGCVNTUwERgCGLNLCVliMpG81XYExDLGUlXNUJbFxsb6YFjef889H/rg//6WXopvB7YMQZNRueP22667/rphWQzW10Moh2VRhjKqiJnP/YP33xuKorZKm/Qdqtf6hqsNYxvBHCcJouPxTVMgMqZrVoCpRtVgpvOLS72ZWR1LODXkruPuynymStJqzg0AVHFVOqc6eosVt43NCFCDMMMxpYrR+pnpjVhiXFxaOmPv3lJEo2iIDIJZf6M/6Pe1GH76mqvvuONOGrF/C2DLEDTx6SMf+9jKylo5LMsQYgyxKGMMpkKwclhcdPFFizu2hTKMyjXq146vayNqjR25zouPHIDx1zaqJtXV9zpKjWryD41JYoxlOX58QyqMbo5PozNI5XzERq56AoFS8x5gcACn3ujqbasbBFDhRNC0ulvFciZnhsGgMO+joZQYNSYGlyrq/Mra2kc++mFUOdWtga1B0GQ+77/vnhs+fx3ByjKGGMsyxDKIqomkTp3BoDByY40ZmziaDtUwdXzNb8IgqhND1pTVWbVA64jZquklSTpNtUcgb1hfPTYYbGw+85pIQJWaRFVxBSJiTknaJl2qBoVytQpwHd+LVYpSSqyywaXIjKApG6UwkaHPMs9EsQRMRQiW4iqRIBJyx1+49m8P3nsPb14rJhlbg6AJV199zYnlFVEJsVBBCFKomjohNjIjbWWe6xzO+PKKsWT6mHY5ysg3uSUeWdKKqmKaviOqXcnmcDBTVVEAMIjAZrbtyjs9idKcc/P2VS/c2M3TRGzpRAxVwG9wZFK9ps5hEY2tAMTCXN0foNSIx8wAt1ut3Xt3Az7lJFSVU6m1QkMk5tWV5WuuuRqjt550bAGCmhkzr6+vX3fdF8A+loWIBCljLAyqMCI2OI1xx46dWavViOEjHtTHQRPf1M+o3qPxR5vGjJo1I1EpOQBmVCeKrHojNRNTi2KqVcbSzCr3spY2qym39eG5Fq0agz3mcxjAVtnLmsFVfYASLI1CMZjWhc+pfIpdZkSHDh9VIxOVWEoZWIQkmGoABRHO8ms/9/mNjX7KnD1kF+1bhq1BUAA33njjwYP3EZlEM6MYpAouVEWEmVpZduvXb1YRbmrhmsxMHR6NH7B+uF5167dLJjZ1rVut8ifCUR3ONK9NU0lEYhlCWRSHDx4c9Pt19T0171U1fFiVVq+7PVJ2KHm1dRxmRgpND9TnAwPBQZ3CA8R1EFZ9KFM2ZcCgWZbtP+ccE4tm0RBBARQU0VREQox5p/PgA/fe+OUvYYsY0S1A0HSlP/vZz4pGqEbRKKoKkzS42JiYDDHIoy6+mL0/qaxzzE5+8/Ao5TC1CWtqSqXjVD0elUReRfHjbkOMEoKUReyvrt1z93394fDIkSNFUdiYMUZavqs4nWCQ0S1E1T9px4XqvtDKaQVMjRggIRbiyqZTXamiKR6vOkZIFGLKzkhhihhDiFEBEQ0xBlGNmjG++IXrmo8/4Zh0oT4ZiaNHj958yy3e+VAWYtG0ZBMjUhVHJKAoqsCwHCbrVYci1bhN2yxNn6TA25h9bZ6PWqtqkkYAFJXrmR6OImknkH6//+CDR77w+es+/pGPXvO3f/vYyx77pCc9aX5pKW/5VquNWlVVGOoqp3HZK8lgBqrukiakqp9qo7I9S8ehdCBzsMTR5AinRMaRYRSfWcowpSJZgqmkZFPstNoHDhw4fvzY0tK2k76ZCcTWIOhXv/qV5RPHmSiqEXuVUgyqxABBxURMnfNf+epNG6trWbeTRhWPH8FO0tvHlsjxlS71yDeRyUncTYdLLBWVGEIZwvr6+nve/Z6Pf+Jjh44cGQyHN3z5xo986COXXfLYV/3wDz/y0guZmchV29GkUSJjB6dao0Uy1gSAAVIAplWPJ1XOKKWRJON3DMVUBmBIJabqHO3eNU8qahCNqlAzUSUYmFVCVInUPnZi+fbbb3viE6cE/ScjfX033ngjTJ2l2jaDmiE5Wik5bmaaZfnS0jZYmohcR+uptm3zMU9a4jfZzrEQm2pFvfEZ6gyQRZEYYzkcHj9+/I/f9Wd/8vb/1ZrpUJYBIM8bZf8TV388qDz/e5//HU97qhnmZme9H+2EZPXBNxls1Ml51IqsVTkwsxR8UdU3r0iJI1MYGVGqbIaCQgjLx5eZAIIoylKcEyJycKoaQhQpyXITvuWWW5/4xCsmnJ2YcIKm67e+vn7nbbeRc1Fj6kEyMxWBKKmZKFQdcRRZW1stiyLrddWUCKpVsb2p8pgRHT948/NYlgiqxo3YNGY+yaCmUaUcFjGGW2657W8+es0nP/GxvMNwbjgcigiZqqlv5Vdf/YnrPv/517zmNd/z7Ke3W+eaeeccN2lQAOkuwvgKPvrsm86tyg+oUdXbaQqldKoxdTIpIsPCsDh67ASISCNZBo0inpi0biwZDAYzM71ut3vgwIFhUbRbk17IvAUIetdddx05csibDmI0IYoGKJuxWmkazYxZowC448B9dx64+/Ldu/vlIFXbqcg4IcyqmotGzaxj5FEspTquOMLGOi1URVRjiIP+xg03fPHXf/U3b7v15pnFGXPtWJZVVagk64fOzMz6xvp73vMXpPqEK1YffemjGK7TaTMzo6qop1qLrW0z7KSSFFSeZeWyKhQg0uSvkim08hokRmJ/YvnwsdU15zMG1FRN2cQZW9I6zIx8GaVrevTo0Qfuv3//uedqXeo6mZjcM2tw++23DQcDqV0tgUW1IBLJxKpUosKI0C8Gt3z9diKvKjaa0QGkjE7SLCHY9BcoLOVb0q9U171XRKuzmiGEoixXVlaPHj3y3vd98A/e/u7b77pjdvt8MAoaDRCFKkVRUVWzoiy9d4cOP/Brv/7rP/1T//ajH/zo8tEjIhJCiKbaRE7VG9XxWNOO16hOI/EBBgWkkrfERE0SDWOMZejM9r7yxRsOHjrKjoOqmYmqRlNDMA0iMJhIf2NDi3K4MThw14GH/ur9UzHRBE1X6o7bb09z4NN8V1GJadSGGalRNERJJI2wz3z6M2apSSmSjodKqipJPmoutzVa0kh7oir7U+mfFUGrgD3G9Y2Vv/iL9/7nX/7VG774uW6vEwIohdJqqkFiIRLSiK+kd4LQnuncdc/dv/2W3/njd//Z3ffcE2MshgNVbdTQRrU1M6im1LoBKYmfzm3sOazKqioqUSyKRdEIco4fvPfAb/3GbxUhspGKiigAAWJdkCIxKmBq/VAEi/fefQ8w6WLT5C7xaf0tiuH9991LzldDZ1JxLqpG3WgmsBSocqTubO/LX/vSHbfcvH3XXhUjP1bWpArmuuw3GafRGwHVWttE1Q17DFCRoij6GxsH77/v7X/w7i99+fp27rI8CwKCIyIIJEoU1SgqogwyiJJjJmKJkndbB+458Nbf+p3rrr/h9a97zWWXXcLOKUXn/CaJACCiqrqqPjkyTSpsyoMmplYtyqJEGmIcDMoTx4+gDM/87u+558FDh4+ttDp5uqPER6/OjNUsmqoImQ9BWh168OADoQxZnk1yLD+5BE04euTIieVl730a4BpUVZXUoBKjxfQfS6EDtdr5V79+66c++okXv/xlqydWspzzVifvdFIRBkSompdk40OKx1LyjQKlpim+1hBiWYbhcHDrLbf+3lt/7+qrr5md63LeimbsnBlFEdWYgiczVYKopvtATJ0yMxPIO+cy++y1nxIJr/ihl/+zZ3xHu90yg/ebOJpMO2s9bCRFTmp1giAJmxEgNSuKIhaD5ePHZrq9/efsz/L2FU//ride+YSXvPTl6/3B/MxMjDGKZFHgUxOKkgRDJ5LTKEcOHTyxfGLnzp0P3+X9P2NyCZpu60OHDxfFsLIrySckUhU1TWYxbT7MzAYcu+fQf/qZn/2h172WyFqd9mBtdWN1dXVtJeNsdmGBmEWkEYtOlutHbwzUliyEMBgMVlaWP/qJq9/xv/742OEHZpfmHDuAlB2Mk3YvplGDqtYzv1Bt8EUkVKUMQFRE7XQ6N3zpi7ffetv1X3jB97/s+y+84LwYA7FzzlW6WGp1h0M9SVSrXRhjKkAmpihaDIdx2D9x7Oj8tl3nXfQYn5EZLMqhg/c9/Vnf87a3/e7P/vx/OHz4mDqNMcYsa4sQM1kWYsxU2gYzW+tvHD784M6dO6cW9B+PI0eOqJpzKRsZYVAjIRKDipqoioBAzq2eWPmVX/j3r//xNw/LIak6ptmlpdlt2war6/211WMPHlTF0u6dIAeQahqMiCRxnhTGNuwsi3D48IN//f4PvfMdf7IxXGbOHflIcHCkzkw0NT6risRUPVTr+WiCNBFRM3ZMzKbw3g/K4p3veMfnr7v21T/8I8953jNb7ZaZeedqf4QMAlMxAqQeeGZmFmIoBsON9fXh2tq+/ft3n3k2M0WJaTdRR5R5d+zI0e992Us1yktf9cOLS4uuZMmy6Nilan2mEEJRbLRzX4APHT786G/j1fxHYNIJunzsGDMDRqrOjGEaBVFNLcZoqjDizA3X1//9T/3U63/8zRvrK8zOmDUqQgRx3u125+cHa+sb6+vraxvD9dWlnXuyVg5iESFSAld6fso6qqlpWcaN/vrNN9/y1t9+y41fvsm0dJw532ZKWW5OZSpmac8Yg6Z58mhaNMeKPznlDOrJzqYW2p3sjgN3/PzP/8JNN339ta9/1a5duyPITFNZqJKYmGhqe+c46JexjGVRbKzOzi+dcda+dqsNUBQJRQwxqEaYCDt2jjL34H2Hnvm8F7z6pd/3rve9P8t8GQvyzkmWOzXKLNXbx5B7t7qy8jBf4P8TJpegadFZWTnhnFPTlENhGAEiUVSS2+dzf/zQ4Z/+yZ9488/82/W1ldQ1R6pSd2PGEEJZujzbsWd3KIrVLA/l4Oihg/Pzi935eSKnokgVJ5xqREQk9ger137m+t/6zd+65bZbW62W85nzOTlnVfmnGEwYIlrXNVd573qZ3pSmUlUGCwTVIFyOouw8s73jne84sXz89a9/zfnnX0BMBmVypholDjb6EiMRmwRiXty2vX3m2XkrE4kSo4ioaowqEtPOTAYzCIOZtIyD//iff+ULX/zyrfffm/n5GGJwUmqUyC7LmFgMRjTo9zHZgfykE7Q/GDqfSVFGTYKSiko0TdeHiNf761c84fJ/9cY3DvsbVE3q2JQlqkIgs3I4IOZtu3YD0XtfDIvl48cQpbew4LPcYCpiBhFZX19/61ve/md//heD4Uqr3fLeZ75FzptBqSpHMlEy46Q1iqilXeTNqq3hU+977eCqiYmpsnONMK6iUPUZfeLjH73l5q9913c9+1/8yCvmF2ai6HBtrYhBQtHKOzt27szbrRTFq1oxKGBJCpUopiqp9EnZp1S/wsGh3+8vbd/+khd/7y/9+m8obDAYeO9Lz20FGUQEpq0sL4oCU4L+o6GiRGh7iv3goYEoqqpExBCjMFFRlOcs7vi93/+f+cxMMeizc7VKNEIia0rPsKEshwQsbNthQCgGJw4d6a8uc5b1enNgB7K1jZX/9l//2zvf+a5Otw2wucxlLWOXWK5MWnWxKdSqeXomUYNEMENUoEibgY60AmoKQjRt4s3MIIgZqQUpD9x151t++zcI4QUvfMGO7QuIYdu27bMLC2yQGEOMVUbCTGKAwtipsVlQVSImJjKtP20BZefyE8srP/rGH7v/4L1/8lfv73TaZTHMstzByDSakfPGzG7Sd66ZbKGeeX5+vt1qZ3lO3qdLFERCjDFG51wI5Ytf/H37zrugv77eDFdq0Byn+dlqCbEsy1iWWZbv2X/e7jPPyFy2cuzYxtrKPbfd8vu/9dsf+OBHOt0cBCaX+ZzIkbEppVYkZ0SiSfq30WEVHEOMKa2VCvPSKgzAtKrvVK3cABVNUX+iOIDuTOutv/v7n/ybq3fv3nvWeefPzi+EshwUw1JiVBG1KBajqnEk1iTpp5BqrD+AiAgMMMhp1LzTffGLX4QiqtSCWYwGZKmwlGlhYeHbe0n/rzG5BE1258yzzm71ZrM8dwwixOTgK7H3h48e/YGXvvSN/+4nN9ZXneNU5DRiTFWrvqm+WJskjZmZhRCKwboobdu5a8++M7rt1vLx4x/+wIfX11bVOEYj75MApFUxKJEYRFNfZ2IY1FTEtJogYgpTqKTE/CgJpFJnsNJDUK2yWwoiFRRFcf5557/gBc9v93pFKaEIElVFTUSjiYS0m7eaQCNZ2tfLEVE6LiOVk6SiJzNE57C6vHrF0/7Zf/rFf+9ggyIMy7IsijJGcmywTqe7tLQ04S3Ik0vQ5Bft379/7949Ls+JnaXtiGMso0BtvjfzQz/4srzVilGsKuOtPk/SxlNJiGFT6r3ia8UTUzEJYTDoi6E3v3Dldz3jlT/6L33mYZpK3FE3GUdYAEqrQxKGmUmMAFKFfDKKUIKBOEm2SJ4oxoJ6s8q+Iu3TBEQTYwyG8tLvf9GjHnVxf30DIibVNk7pqJYYncrtwQomKCBpFnMquyeqbhEySdtAmbq1/vDZL3rRttkFIpYQB6WEslRTdm5paWlxcVHHEsITiMklKABVWVpauuiii3q9GeczAKYaYgTzyonlf/dTP/GkK5+0traWVFKgWssT/5qeybqk0sae0phVAymIUhtTLKMMi+c95xnPffZzhkXhPDHqifCETM2LpDjfqqxByutrEj3roCyZME31AoS0rTLXOnxFTdIqW4uqOD8uLs5d/MhHl3EYU5SuSb9KR68mOaS6O6tm4hkDDHVpYxwQLEt9fkIMIkcwCmU5mFtc/JX/8ktdYG2wrhIsBo3RZ9nZ+/b1et1xX2gCMdEENTPHfOGFF5591j7vHXE9xKMoz9q+87uf+V0b5RB1GERAU1CZSkKaUW60uYUNY9nNNKNLU/6HLJp2Z2YkxDRbqVmjoRJNzCIldd40iqgZyAyaGpYMcM6BzUgAcuxSy12MEZsjZTWLqlJ5IgazUITLH3f5xY+6qBiUolKIxGSQo4kYLBFVNXXCQWAR9SB8IwKco7Q5Xfq8ic7Vz8VgeMnll19+xRMX5uaHxSAKRGzb0tLZZ5+VefcNUeVkYaIJCsDMFhYXL7n00vm5OQJrFKhurCz/wn/8+bPPf0RZBK79QauHb4xS66NCtZoKqOhYhxSOmmljdUd8r9u74sor5uYWVUxR+awCoN6fow5xBKIQExExhWNjquvoquA9Gc7RaNymXHpTAAdmUqX9556/fccOUWmKViwlDZr6kOaDAACnT6DIQGTEWtl61MWljLRNI7gIsT03/9rXvCorCo1aSiDH55xzzp69eye5EjRhos8vrY9EdOFFF51/3nnMDEM5GDzlyiuu/I6r+oOBY64ng9UeZ3VdU8VHcznrsQtjc74NJhKTB9bEMgoLMV76uEvO2Ls7liUSRYTIHGkyTFrxVJWo3pMDRgpRNVWqGZVo1ShN1ZmkUcy1ep9OTUT37dv7lKuuVImmlX+C6myNKG0DogQBEciDXP0somqXr1RVXdVAp5oF1TSGFEQ66Pcfdcmll1z6GDU1tT179lxyyaXtdquasjPBmGiCIpWXA71u9wlXXrG4fZt5LsvyFa9+xfbde2KItLmRQ9OYubE1vU4tWmrzHXdG0TRyEFVxuiobGTA7NzszOwPATMxiWpODxQiNpqpGakoIiecVY6MnYkvzbJiIyHHyI8fLPRXJKI4CJiKI2hl79j3q4kc2pjL9W60DFY2dUlJXBWajxpGKl+mTkipB4aqtaAkQaCQVCWFmceFFL/7nxWDYnu099alP3bNnT/oSJnym7WQTtF6xAFxw/gXPfOYzvcvOO2vfd37n09bX+27zbAyqv+sUxqSZiKMn1CSuNjUE0GwVZ0ZpSBcRCKq6a/eu7/zO7yDOpJKlTJPEk0RMQvoFIKuXbCIyrYLs1KvOIEecoifH3BhO29TxDOfYoj7mksftO3tfzTYFYr0AsIEt7UpjBPNEfjQI3BSUmkA1NUqnUU31berQfGSyoigf85hLHnHWmc959rOf8ITH5608bYE34bPrJzqTlJofVNWYvXPf8dTvOHjvveXaiYWlHev1AA+MpWpQX/v0mwIQYSJw00ZRtW5Ww7+pHhJG9S7tRMwcyvIJj7987569Bx+4L/etpCily5gWUSVyYEWMpGmvTpORCJuG1pgqs0seiJoRU7olkt6Uzsc5VrWFhcUrnvQ4VanK6ap6ajKYmqRQqP50SIsyIypyUFr6q9uy0kRVUY+BUiMjFhAzlWW5tHvXm/7NTz7q0scuzM2zY7PInE9ynhMTTtDRTCMARN12+wd+8IfyzA+KktJuQ7WT2qxzwOh6prENagaRNLkD9VScqg9NjYlTYxOjMp8AmLk303PeA1ATNlcZclNK5jnJ4UpQpGb6+j2JmcFILU4GOGYYgiozXLoN6tupcUvO3LfvokdekKpOyJI5TsG1MhFSGFf5JwIyAkOVORqS6lTdj2lcGAFpA57UYGdkqgKXQYmdv/IpT82cI1YCqYLZTThBJ9q8pzBJkmdp8Hm+MDeXt9pWzTjY5HqSpfbc8XUd9YjDxMbRQC/UhrMR0muCpQWRztx31pOe+CQFoomaIHmxVOniqeWEklnlWueqlK40ZyElCIwMZPDMLo1UUuNaN0ghlIheeNGF5+zfX4tkdaxTbUczGn5becxpUa/ipPRB0ygA07TOJ0eBXCr9YmLHTiUyG0zZwEzEPuVjnfMjEzCRmHCCwjk2s5hC1notbmYib7p4lc0CxqJmqqUXqqsiGk2qoi2QnpbcOleLAp1W66qrnjA/O6/RYMQgz0zKlZVr3r0+TzNzznnvR9aRiIjETKhqbk6RtzS7hBIRkffu0ksvyXKnKk1yoXFSm0dqJH+UDayjKX1URfSNMRxF9PXLmKv7mYzgUm80gdIqMcmYeIKm/V+MtJHTx1JzJy9P1ozgHD0yEh3r3mJqaDoe9QNIvUiqIGLPnU6HyQhRLVqjqxIimZARTAj1iGVTpJCrSholL8ApmKiazoh69k59FzFzjHLGGfse/ahHahSqRzM1Zj5RdXQjEQHGhCQ5MXGt/rrkb1efPkVmOlIPkskkYosx9TPBlAwM9j5LX9O38pp9SzHxBHVpOwtRE61b2sfVzc3YpHoCaPhXy4aEmkRAUgmb7Hz1SmZ2zMz0yIsvvvDCi0XNoEzEYGMiQ27kYGZKpGRgJQYxO5hCYxo/x4BPGq3BodqeqArgas8kkfURFzziUY96lFkji25aGVAb1ERTgsICpe+hcqUZpGYmIqgWek1CQHpRFftV4oA5zsykHrRn3k0t6D8NzjtmEgWbkiFZt5GyOPbM0Vpf05crc7IJVM9CSgUlTQa9HiKXZoFDVHudzjOe/p3tvKMiogGAAzErkWNzQKqv0mgKImdmleavydFkZmJOxc1W70pXGdEUBzFD7JJLHzM3Pzc+MrI2e44xnuhKOTBEY4OrzrbaRUnqQrvqAzIRE7iu3k5lfUhhVGXCjUiJyVdL/OS6oZNOUGZmdjADGKnEou4cTyL4OFOra9SojDTqXEPtZTaOo6pRFYokz3bckhKIfZ4Nhv1QlkwwFaXKmulYptJQZcAtZWxstL9R6pOuU68Yz1cSERGryo5t2y55zKNTb9MISHVOQiy15FmXZVX/T5Vb0SBmAsA5Tt5DSlEAKbQnpuYUACBGTQ8RmIScy12WPWSX7luDSScoQM67JEmLmplyM6Wm/uJtc72DVRFVvQtC/WjKOdKIx2m80Zj1qM0vGRyRz7PzL7qwNzeX6t6hKbbOqjC9ufbg2uZVonfV0aFaTTiz2opWpU4gkGM2sfPPv+DRj3601dvLpsrlMaI2t0IVByWbnwpZzDiV9qHW9K0ae8tmDOP0dObqxjJVidGTk+pjSp75WjeYXEw+QeF9ZsaAUr07JhFUlOpZhRhf39Ov9Y7V1GThx//XjFM0Mx5RfBQEp+sm+uiLH3n5Yx+nQQCyFGVDKa2nXOUR0nqaXsWO6zFLVVp1NIyciJC2liGqgj931ZOv2rVnh4gkATU9PyljzGkgPSqBE0C1EkejlOzk2itNo8XJCNX9W/1BVUlFAWIjU2m120aAKpgV1G63vx3X75+GLUFQBxKMNtYiwGmSrDcnkMY10CoQHgs1rDJClVzfvKp6fvOq2g0gIlWJsSAyJVWIVqXGo3epwx12zjlLuXgmx6Mxi2NhuxEFhhLAZGR5u3XeIy7w7KzOp6fxE9WT0wAdMiJPVMnpliZCIE39iUqmcAAzhEzNGKRqUS3ChEBpyzoSIViMMZ08MTOUiLNWp/4+JxdbgKDO5d5lyRxa8gbTTFezuiloJJU3TE2GEWNmtdJrapo20tKYQ9lQtiph7s709l9wLnkHILUhESmxMZQBkBHDOThHROScg3eOGUSWCMqcjHEVA6UMFJFjr6rnnX/upZc9OsaYEj9W7eCt0EhoioxSS73V3c3JwVSu5M+6fHlsf5Ja+NXUG21ESqaE/mDoajFY1XyW+4l3QLElCEpEzmeJNSnqSTGBmaWRCONPHv91k02tYykbs6aNC/sNRpEAMHPm3BWXP27H4pJGSSUiVjUi1c0XY6eZSow01d9RU5haS6PpZyJm9hmb6WWPfdzuvbslRiYHJOGoziBV2SiqNFAaCUUKNngFKG2QqBFmatXAh+QMpyMlJ4OY2TkR6XS74CrTq2btdvtkFXkisQUICiBvtUbxLwCAiNN4QdScG42dqR/ctIJvijxsZECt3gSmeVrKLwJEzN6fefZZ5z3iPAMcQSVEEU1zE0KoWuJFTJVIU2KTiTKjDFWpsqtNaSKoc845x44cZ+ecvT/LsirDVC3zRswgJ1rH7M2/AACGcVWXmpzytGt3sqpJ8aVaIkV1r6VUvYgjVlS+jRk6ne5DfNG+NdgaBM2y3DlXXzETkSQQqm7i6EnY5I9+w9PS5FcCUT0ubOxpagA7JtCO7Tuf/4IXzs3NpjnioFRMZAQjwLHz3ifh0bNzzJ4dcp/SnllC/UOe5+wdiES03e485tEXM2DETVqyuaOqAeSp8w2pkNrIzMCpRYko/UwGNlNY5GpP7nrbnJQvqnac1ZXVFec4kVhEnM9a7c5DcKG+9dgaBCWiLMtlTEpMnWjNdGI02ctNNnIsMK8N8CgMstTrZpIi6/olDGKk0Z7kne92uk98/OOf+z3P8i4TsWJYFEVZhhijpHHJEktS1WAhpOGQGsrQ/LWIsYwxhKhROFpGbqbb27199/d8zzMvfOSFIQSMlTBXH6YJmkwMqbpj9AGTFqomZkQwT8ZJWDBHtbElSs4AcSo5jDY/N8cwjWX6MmZmZia/2SNhy2wqGkJYPnE8LZVm5pxTNYkxy3NL+wYZpezeSQs9Nqfsx9KMm3mcBt6NtdqZmUgIYqsnTgzL4ovX3nDLzbeWFgfDYjgoTMK999/f6bYIOHL4mPdZt9c6cuSYgXbs2B6LIKZzc3PtLO/OzezYubPbbuc+2760OLc4v7A0tzQ3d9Z5F3iXxFlF1fBUJQG08T9s0wepqkCSebQqhRujgthxJXMo1S/QNE4ZRRlz75igJo69Ee3ac2Z7i1jQSU/FNvBZluVZDKGJbJihbKrKBDLD+EzaGiMlv/4BzbDFWofiugw4uaWNApUqMaTse6YzzzrrnLPPGmz0iXljWMB4Y31ZFUyu3183CJHv9zeKYcwy353pECjzWafbIVin28nzvNobkxgmpPbAwQcHG6vzC9tEYnI+K9NeWXEA0LRHfH2PpUW/ShaZwCylAJwjI7Jql5rq0zZVewYUxaCVz5kKk1dDq91ptbaAApqwZQhKQLvVXisLVG6Vep8mfQqxs021zSPV6RvXh/FQCXVlE5qCklq0FIlmZqblcDi3sJizj6rtbs9Ufd5StdnZbiKW6k4imFqUYAaJwXnHzmHUUlINnqeKRsyOOp32kUMPLCxuSyl2Y0skpfQCI62idmJirbsAk+Ws9wCrEloEUpU0zQGUnqNp0y8CSK3bbaPK1pOY9mZmtkT8nrBllngApnr8+BEzJXJAqgZHWRTsPDnPm7Zxo8bjbMxn86dv/HlEWaqbTEwBWltdzryfW9yWkk9abxRvdeMojS2+lMqfQZUSVaV0xCzNxt9UDuKYT5w4MVg/sXvPGUpZLclW4/eNXbKDBKcqhipZD+IkB9efoXZWx1wBVFsiCQA2iyGQcyCkimn2vPfMc9zEFzE12BqecgIxt9s91WbhVgDOu9T5eVIHbWMpT2LhScZjFDyN7aeRWuOKYRlCnJmdracrVUJ6Pf+rUYfGDqUmUhUD1wcfnUm9AoPJRMrFbduczw8dPuIyD6kmKSf9MoU6qYejoR+BExvrxaK+Taq4P/V/UFr3ydgBKjGKsPPpSSoyM7uwhdiJrUVQAJ1ul8g1o41VzTlnJhpj7XZtiuLHBU7UP4z/jLE0KcakAFUdrJ2Yn5+nlMRKuw3Wf6pyO1oNp1NJDZ/a+K9j79U026VpjaoqMaoppCh27d6bMd13x63kXO1dNFvUp8qPuroKXFWmAKkSJulcbJX2aamlxaJINIlkMQ0kydotmDHITL33s3MLD/lF+pZiixGUmTvdzphErwB5nwUpqioejHLfGF+7N9vO8UcS52hklZSINtZO+Nx1uj0TjaqNEnTylI+Tcv1ESEPpR4JX9fymUikNxVO1GMuyHC7u3B3Kcv3EceczVF5Dc4bf2JBU9Tm5tAlIVXetIMDqQipG2ochRiGwM4KJYxdVe/OL3vsx4X8LYIsRFEC322XHaX57Y0QdKEqoJ4ak6ypppW0Mm40nnGhUAld7q8kKCYj6/fWNtdWFxe1puCKq5Xs0cGH8gCACMzaPekz+qNUi7Un3SVrkyWIaN77v/AuOHTl05PAD3mXViBAIgdO+ucn7YFQzHQEliqmSy5Qkzcwf1TubRoEZM0mMzqXiQIoqLsvmFxarE9462HoEZXa93oxapGpevZhq5tP2hym4ScyoNBraXOuUGGT15i6bHq6mdtqJEycWt+9il1td5HGSrwnU7XgYsbJKczWBWsqgEqVReM3z079KTjhXYwvBM+8888yV5eNl2bdqWw9TS/0tWpMv7RtaNTRX3S9WKfhOI1kwlVBad3a20+LVtXXnMiKLZAQykcWFbc65yS7+/CbYegQF0On28rwl1QRNFVViB5CIAGwxUqV7n7wWn8wza/bwtOQhgnjlxHK31ZqdX5Bq68RNy3p67cmMb4QqGhlLq41h3QE1QsqQAy7NLB8Oinanu3fvmXffcVuMQs4nvStNECWQaZpFT0DarDMVlmhVBggy86E08u2lbXNf+Ju/+bmf/plDR4/4LDOkiePSanXm5heq995S2JIEJdDszIKZEAmRqkpUYXZra32Ng3ZvLoRQSe/ftGC0/iGKWL01Ryq6CCEMh/2l7dtUQ5od23i0J7mwIyKmFFDtdY7MqmjjUY5cS1AqM2JHImWMUYhMxaJ0erOzc7OrRw+DvRAnRcmqxnoCuTRLvBKS4NXYTEwlSETG89vnVo8f+sV/+7Ov+JHXfPiTnzl66IhAzIxExGxpx87x9qYthC150gDyPO92ZiSizv4IEW1srL/lN996w6f/Ztu2beSzEAOaglEdtRJZpRUJ6p4eAGrKoJXDhxbm5/N2L42yMa0Fz29Q+JtDNY8rqo6OlONPe24n20tVv1ATAKmZ5Llj8mZgx2aI5XDX3rMkxgfvuT1nJtVKq4IZoObSiZIaRM2ixBhE8m5nbr536O473/Ebb3nTG3/yf/zPt7dnl7ZvX4yV/IQQw8z8Yqc78xBfkIcKW0kSOwkzs/NlGSyWRCSGGIZFMTx47NiP/NiPvvQF3/v6N7xx1759y6troSgy75q+OAAgNo0MMsdUx+4E9Id9I5pbWJQQDaQmqeoSVflwI/hbk83ieoCZmVGdSk+PMJKt3FS9TykTRFUKnp2ZpNqk6IhFbemMsx68+/b15aO9mXkzSTt51dMVyQgKRRSwn52foSifv/baz3zq6o995JM3fPWmmW2Lu888wxHml5a2b99OaiLqWq0d2yd6N86/H1uYoOx4bmHu+JHDABk5NVpbW29l2eKeM/7nu9519d9e+6LnPfelr/ihbWfuW11Zi6H0zZYrWs33qmoxK3rJ6tGjO/ecQc5J8hAS8SA1retGp9GoskoB0LHZjqltYzwU+buTrlXo70ijGakySma/Z+8Zd9554KxzO865KnOZnicSTPN2qzszs7G88r4/+bMvfO76T1xz9W333bdtafv2M/ZmbHnufat1zjnnzM3Ophtj1869zJO+18zfgy1MUAB53p6Zm19bWyEoEcUQRWRhdo7OPe++I4f/31/9bx+9+poXvfB5z3rhi7bt2tHvD0JREsDkKNHIqoJLMK+trHZ6s51eJ4ay2oGgqhyqNrEl42Z8RxPIV+s7gawxkFZNqR/zWVMrHCrBNPUVEXPa5I4IuWcSLU2FQXmrvX3H9oN3Htj3iAslBoCIInvfmpllRwdvv+OP/uzPP/rJq6+78WuRaGF+5owzziAm71utjDud7uzMzLn7z+12u0Hizl27O93ew3RxvjXY2gQFMDMzH8o4GG4A6HQ73W631+vFGHfYjvnZ+Zvuuvuan/25v/rz973qta++5HGXn3XOuQobDtZjWQBsSDMgJEo5WFs749xzVQUiVhX9MsGgVtfmpTesEzcnRfQAgQ0RYCMijIoBmlO1KtfP1XGQZi06qZK0pMYwLkJcWNreHwwGJ47vOWefqUrA2okTn/n0Rz758U9+7G8+devdd7dnu3PblxwTMbeyPM8yl+V5K+u02zt27dp39lmOaGZubnFp+7i934rY8gQFYX5hMRyNMRStdnt2brbT7YpoVDHCPPV63XO/fOdt//KNb969tOMl3/eiSx572eMee9mefWcIqCgCAEAP3Xf3tu3bnMtiqHdlqH1Tq0c1JgJqk+0ZH1CD6rlEDINCFak4w06mqamBK81UDVW3sVS5TYKZEHEQ2Xf2OW/5td944P7DC4vd++594MDtd95w89cCMD87u3P3LmZmtsxnma/+47O8nWdz8/Pbtu9YmF+Y6c3u3rX3G0oPth62PkEBdry4tHTsyOFWqzXTm+11ZiVYlGBp70OUi0uLM3OzK2trv/a7b7VSz9t3zpOffMUFF+y/6JLHhKJ84NDhnUuLz3jec/obQ4zmHRKgBgM7pMGcSF2/lgacYCy3iZSdIQBsEII1AXuKosbcUDLT2uimMaNEVds6DMYwmGos24tzDxw5/Dt/9Ad55gC0Wp25pW2tzKW+u8w555z37LOW81me+SzznU67NzOzsLC4uLRt9759zmebBK6tiVOBoACyLFtYWuz3N3ZuX7qn5UMrk9hhw9Bo4Hx/OCRyMzPotNsidvDEsT/8s3cz0a69ewcb/XvvvfcXfvbnnvOi792Iq0TN5IVI8HU9ncFc6kaqbeHmfpKKkQRIyhaZVaN1mhg/uaHjAVO1xUE6QjVMkVRUFezcxvLyg3fdPTcz05vpEpEZeeerpifnvXfe+cw7EGfes+NW3up2e61Wa2lp6bLLLmu1WuP3z9bFKUJQM2u3u7v27N1/7nkH7r5nMBy0Q+ZSc7BjAGVRwjSYAtbptXrdtqiura74LGv3erfefkva+IhT7V413mi0OnNd4MmcZKN04dMIr8ZeSnp2aroYr2kar+fHKMZKM04YqfGNAAObRZXZpcU/+O3f/tinPtNdmIkieZZ5753jzDlyzmXeO86dcz4DIc+8z1u9Tqeddzrd3vOe+5z5+Xmt91Te6jhFCJrq0Hu9mcsf/8S777nn+NGjbLoBa1f7XIGYjGGeXFAqOcRoRMxOos7Nz378Ex/7/Gf/9tIrn7Ry9ATV9T4jckIJZtUMpqqyBABSvhF1IRSYzAiW2NxMv9l0kiOaVnVOaf9CMyUjNVEgb2crhw6/44/+1LeyjIi8T0YyyzJ23jtOLM2dd3luRFmWtdrtVp5luXvlK1/9yIsfdcqwE1s3k/RNQDCzVrv95Kuump2fa3U7s7Pz3Xa722p1O51Ot9fr9BbavXa33e60Oq08846ZPVM7c/0i/vgbfvLQPfd3uz1EYQaq3YnAMJdio6rg0tLyrclWUlNql/KlzQhTBYE2m8xRKmq0YQ5XKXUYwTlyRByK8sff9BN3PXBfu9cl8jn7zPksy7x33vuqmzlvkfeJnXm77b1n71/7utc/+corVE4dduKUImit+Jx99v4Xv+QlLsvZ++5sr9Pr9Lrtbqvd7nbzTrvVanXa7U6n1WrlrbyV5zkTL8zPfuXOW9/0Y/9P2V/L2nmIAepM2aqadhinHZSqEIqrcjqtR88oUrKHUtdbylGKWdrusCpUric0aP0/AdTUpRFMZrEIYefSwh///tve/9GPtWfnmMhljvOKlHne8j7127ec8y7L8zzvdNreu26n/YY3/fhTnvo0VWV3al3Tb8xwbHWoCrP7+k1fe/vbfm9tfd3neX99o98fDoZFUQ5CGUIIoSyHZVkUAaJlLEMo1WjQHz7+Eef/ztt/v7WwGIZDdpw6fLlOOlVmVY3T9m/VlDnUOzIwUgFcKvkDSdrFYBRbVa4qjA0hWQcVYRCAGMvZ+fkPvvvdP/sffqlwzMze5e1WlmXO+cw757OM2Xnn095GrbzV63ZDlF07d7zpzW++5NLHpg/+8H3xDwlOQYICEFXHfN+9B972u7/74OGjrXZ7Y319faMfyrIIg2FZhI1yEENZliZaRgkhhGIYTY8cPvz9z3ver/7afw/MppHZVe0+AGnSl1J9aeqho6qFDmqJtNVOhxCA0j5aaoxIZGKkaehxUgMqzzM5Ei6Gcm5p6abrP/d93//y4HyrlWdZlrdarcxnmfM+c0zkvHMucw7MebvdyVvDQf+CCy/61//6J/fv3y8q7pRjJ05VgqKu5FhdXfmjP/xfN33tK628tbq23h8OihAHw43QH/aHIcagUogglKEoi1CWAjt26OgLnvnPfvsPfn9YSuPPVXXBAEzBaTs3AxOqll8i8mrKasbJ+yRo1anOAEwFacqNELipNUnlqFFkptu9+7Zb3/iv/tUt9z3YarWIqN1ut1qtPM+8Y+9cyzuwY++99612y/tssLHxlKc85Q1vevPCwqKINJPrTjGcsgTFqNpI3/++9374wx/yzINSVtf7ZbExHAyKoiiLoYYQRINIGUJZFmVRhqj9jf6rXvKif/Mz/w55z6TwjqMawCCDpcGLlOrpapU0PWQgp5B6y5k0ZiHJCAozNQKZMwYpTERhoBhiZ6a3dvSBV7z0FV+/5965+Rkz3+60Op229z7PMu8zYvLe+Sz3znU6HdOYOXrxS37gZT/4Q8TuVIrZvxGnMkExVtXx1a98+d3veueRo8dc1lpZWdvorw8HG2VRlEEkBhFRkRDCcFgMizKIHLz//p/+0df/3K/84vLquqmSzwCCiZqljXEARt3orI0knvKgoyQnkNSF1OWGSCCXgiqyKBqC9GZnjt1/7xt+7E3X33LzfLdDhqzd7XRbnXabncu9Z+8JcM61Wi3nXIjx3P37X/u61z3u8ic0n+5h+34fepziBE1INmZldeW97/nTz33u2sFQhsVwfX1tMBiYImrUGGOUGGNRFP3hsCxLFZGieP6zvus//tdfLUVNwa5OUFKdeLe60BN1nSksjQ6FKqWdDA1cF/WlAtGUeTcRMZ2fm7/79tte/7ofu/n2A72lOah18lan1223W3mWs3PeO+dd5rNWlqeN5L7n2c965StffSpJ8X8/TguCouYogC998fq//PP3fP2WW5URy2DDMLQYU2N7lHI43BgOBoNBGeLGYGih/PmfeNNLXvWqiLQ5rNe02whMwVVjZbVZZ1W8b6jqi42cgVilqjUxNZASsUgEkaNeu/XBv/yrX/qV//rg8eVOr8NGrpX3uu1up5uMJTtOoicRhRgvuvDCl7/8lU960pPGP84pj9OFoKi65IyYB8P+Rz/ykQ/89V/fd/99uc/S7AU1kRg1yjCEtfX1jY3+sCgd0dEH7n/NK1/5K//fr66trw+K0mcZJbaRY9N6a66KgEgxOQEgBSUFPlWI1FMZkieK2F//jV//H3/0J39KWafdaYOsl+etTrfV6XbaSVjKfOZBFGPcuXPn85//guc//wWdTqfJ6T+s3+W3D6cRQRMa23Pk8OH3vf+9H/3IR44eO9putfNWS0ViWRQhhiCD9fXltZXBYDgohrOdzrO+6+lveNOb9p519vLqGpmRqTK5lIwnroYzMFKtSd2ZRKmvSKiOpghkVsbYybPf/c23/Jff/I1tSzsYxJ5brdbMTG92dq7dahNXm4xFKRcXl77z6c944QtfsGvXbpxOhrPBaUfQhOZK33PP3X/9/vddc801Gxv9mV4XJoOykGgS46C/sb6ytry+tjEcHjp4+JlPffIrX/2KK55ylXOemcl7iLJjraeCSdXTaWxmRgpyGpS5sa4pX6oiIdjNt9326c9cc8tNXw9qM73e/Pw8e18URSiGIQQRWVhcuOqqq571rOeeeeY+1DX5p4/hbHCaEhR1MXKi6YMPPvC3n/r0566/7v4HDvbXVwaDoZnLvfNMhUi/DK0sixKfetWVL37Ri2CU55mEfqczG8ph1mlrNGYmxxpKdh5pFDy5pNkT0shjAoiYoIhmZkoxDgaDYAZwKMtjx44++OCDhw8fZeceefHFV1xxxY4dO3EaUzPh9CVowjhNB4PBTV/7+mc/e829994vhl4337Nz19LS9navN7+4uLiwuGP7trrHToeDQSv368trM/PzxaCfsjvDwaCaVZuGPTFX3zCbQVVSoFRvhg1lIMlbKgKiVndmcWnHtm3b0rmd5tRMON0JmjBOUwCqura+NuivF8NCRbKMGeSYoqXt5RRGGUhEyXEaNqZp6g57gbEIG2Il18coSkaWJuAYmZaqUEWIpZl573u9udmF+d7MXDOWe0rNBlOCjtD0GDVMLYvhcDgohsNhfyOGICZpDl5KVhI5cDW028zYALXISB0hYmoKMYGIGqKKxghVJXbsWq1Wb2Z2ZnZmZma2XU9VOB2E9/9bTAn6zXFSv4SqFkVRDvuDYb8YDsvhMIQgMUaN0QxqrNWOtiqa9uMmNTNS5pyJnPd53m3l7Xa3PTvb6/RSoujverspGkwJ+o+BmYUYJYayLFQthFJCFFVRsbTVIru0N5JzLstb3rksyzdXao5V4E2p+XdjStD/S/wT+sybr3pqLP/hmBL0n4i/8/urTeOWb/x9eDEl6BQTjdMrbzbFlsOUoFNMNKYEnWKiMSXoFBONKUGnmGicIqNvTnOcwgrr1IJOMdGYEnTLo6lxOSUxJegUE40pQbc8Tu1KqClBtzZO4cU9YUrQrY1T2wHFlKBbGs0GYg/3iTyEmOqgWxjN/t7jv55imBJ0C+OUZORJmC7xpwhOVbJOCTrFRGNK0FMBp6r5xLTlY4oJx9SCTjHRmBJ0ionG6Sgz/UO8mlPYq9taOK190ClTJx+nNUGnmHxMfdApJhpTgk4x0ZgSdIqJxpSgU0w0pgSdYqIxJegUE40pQaeYaEwJOsVEY0rQKSYaU4JOMdGYEnSKicaUoFNMNKYEnWKiMSXoFBONKUGnmGhMCTrFRGNK0CkmGlOCTjHRmBJ0ionGlKBTTDSmBJ1iojEl6BQTjSlBp5hoTAk6xURjStApJhpTgk4x0ZgSdIqJxpSgU0w0pgSdYqLx/wPkI1FlG/VRgwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<PIL.Image.Image image mode=RGB size=224x224 at 0x201BC317F08>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1, 7, 7, 1024)\n",
      "[[[[0.         0.         0.         ... 0.         0.\n",
      "    0.        ]\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    0.        ]\n",
      "   [0.         0.         0.         ... 0.26740852 0.\n",
      "    0.        ]\n",
      "   ...\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    0.        ]\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    0.        ]\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    0.        ]]\n",
      "\n",
      "  [[0.         0.         0.         ... 0.         0.\n",
      "    0.        ]\n",
      "   [0.         0.         0.         ... 6.         0.\n",
      "    0.        ]\n",
      "   [0.         0.         0.         ... 5.2920036  0.\n",
      "    0.        ]\n",
      "   ...\n",
      "   [0.         0.         0.         ... 0.36225504 0.\n",
      "    0.        ]\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    1.6294715 ]\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    0.        ]]\n",
      "\n",
      "  [[0.         0.         0.         ... 0.         0.\n",
      "    0.        ]\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    0.5091744 ]\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    1.1348834 ]\n",
      "   ...\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    0.        ]\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    3.2059414 ]\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    0.        ]]\n",
      "\n",
      "  ...\n",
      "\n",
      "  [[0.         0.         0.         ... 0.         0.\n",
      "    0.        ]\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    0.        ]\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    0.        ]\n",
      "   ...\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    0.        ]\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    0.        ]\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    0.        ]]\n",
      "\n",
      "  [[0.         0.         0.         ... 0.         0.\n",
      "    0.        ]\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    0.        ]\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    0.        ]\n",
      "   ...\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    0.        ]\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    0.        ]\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    0.        ]]\n",
      "\n",
      "  [[0.         0.         0.         ... 0.         0.\n",
      "    0.        ]\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    0.        ]\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    0.        ]\n",
      "   ...\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    0.        ]\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    0.        ]\n",
      "   [0.         0.         0.         ... 0.         0.\n",
      "    0.        ]]]]\n"
     ]
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import os\n",
    "import tensorflow.keras\n",
    "import matplotlib.pyplot as plt\n",
    "from tensorflow.keras.layers import Dense,GlobalAveragePooling2D\n",
    "from tensorflow.keras.applications import MobileNet\n",
    "from tensorflow.keras.preprocessing import image\n",
    "from tensorflow.keras.applications.mobilenet import preprocess_input\n",
    "from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
    "from tensorflow.keras.models import Model\n",
    "from tensorflow.keras.optimizers import Adam\n",
    "from PIL import Image, ImageFile\n",
    "from matplotlib.pyplot import imshow\n",
    "import requests\n",
    "import numpy as np\n",
    "from io import BytesIO\n",
    "from IPython.display import display, HTML\n",
    "from tensorflow.keras.applications.mobilenet import decode_predictions\n",
    "\n",
    "model = MobileNet(weights='imagenet',include_top=False)\n",
    "\n",
    "IMAGE_WIDTH = 224\n",
    "IMAGE_HEIGHT = 224\n",
    "IMAGE_CHANNELS = 3\n",
    "\n",
    "ROOT = \"https://data.heatonresearch.com/data/t81-558/images/\"\n",
    "\n",
    "images = [\n",
    "    ROOT+\"soccer_ball.jpg\"\n",
    "]\n",
    "\n",
    "\n",
    "def make_square(img):\n",
    "    cols,rows = img.size\n",
    "    \n",
    "    if rows>cols:\n",
    "        pad = (rows-cols)/2\n",
    "        img = img.crop((pad,0,cols,cols))\n",
    "    else:\n",
    "        pad = (cols-rows)/2\n",
    "        img = img.crop((0,pad,rows,rows))\n",
    "    \n",
    "    return img\n",
    "        \n",
    "for url in images:\n",
    "    x = []\n",
    "    ImageFile.LOAD_TRUNCATED_IMAGES = False\n",
    "    response = requests.get(url)\n",
    "    img = Image.open(BytesIO(response.content))\n",
    "    img.load()\n",
    "    img = img.resize((IMAGE_WIDTH,IMAGE_HEIGHT),Image.ANTIALIAS)\n",
    "    \n",
    "    x = image.img_to_array(img)\n",
    "    x = np.expand_dims(x, axis=0)\n",
    "    x = preprocess_input(x)\n",
    "    pred = model.predict(x)\n",
    "    \n",
    "    display(\"_____________________________________________\")\n",
    "    display(img)\n",
    "    print(pred.shape)\n",
    "    print(pred)\n",
    "    \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"mobilenet_1.00_224\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_1 (InputLayer)         [(None, None, None, 3)]   0         \n",
      "_________________________________________________________________\n",
      "conv1_pad (ZeroPadding2D)    (None, None, None, 3)     0         \n",
      "_________________________________________________________________\n",
      "conv1 (Conv2D)               (None, None, None, 32)    864       \n",
      "_________________________________________________________________\n",
      "conv1_bn (BatchNormalization (None, None, None, 32)    128       \n",
      "_________________________________________________________________\n",
      "conv1_relu (ReLU)            (None, None, None, 32)    0         \n",
      "_________________________________________________________________\n",
      "conv_dw_1 (DepthwiseConv2D)  (None, None, None, 32)    288       \n",
      "_________________________________________________________________\n",
      "conv_dw_1_bn (BatchNormaliza (None, None, None, 32)    128       \n",
      "_________________________________________________________________\n",
      "conv_dw_1_relu (ReLU)        (None, None, None, 32)    0         \n",
      "_________________________________________________________________\n",
      "conv_pw_1 (Conv2D)           (None, None, None, 64)    2048      \n",
      "_________________________________________________________________\n",
      "conv_pw_1_bn (BatchNormaliza (None, None, None, 64)    256       \n",
      "_________________________________________________________________\n",
      "conv_pw_1_relu (ReLU)        (None, None, None, 64)    0         \n",
      "_________________________________________________________________\n",
      "conv_pad_2 (ZeroPadding2D)   (None, None, None, 64)    0         \n",
      "_________________________________________________________________\n",
      "conv_dw_2 (DepthwiseConv2D)  (None, None, None, 64)    576       \n",
      "_________________________________________________________________\n",
      "conv_dw_2_bn (BatchNormaliza (None, None, None, 64)    256       \n",
      "_________________________________________________________________\n",
      "conv_dw_2_relu (ReLU)        (None, None, None, 64)    0         \n",
      "_________________________________________________________________\n",
      "conv_pw_2 (Conv2D)           (None, None, None, 128)   8192      \n",
      "_________________________________________________________________\n",
      "conv_pw_2_bn (BatchNormaliza (None, None, None, 128)   512       \n",
      "_________________________________________________________________\n",
      "conv_pw_2_relu (ReLU)        (None, None, None, 128)   0         \n",
      "_________________________________________________________________\n",
      "conv_dw_3 (DepthwiseConv2D)  (None, None, None, 128)   1152      \n",
      "_________________________________________________________________\n",
      "conv_dw_3_bn (BatchNormaliza (None, None, None, 128)   512       \n",
      "_________________________________________________________________\n",
      "conv_dw_3_relu (ReLU)        (None, None, None, 128)   0         \n",
      "_________________________________________________________________\n",
      "conv_pw_3 (Conv2D)           (None, None, None, 128)   16384     \n",
      "_________________________________________________________________\n",
      "conv_pw_3_bn (BatchNormaliza (None, None, None, 128)   512       \n",
      "_________________________________________________________________\n",
      "conv_pw_3_relu (ReLU)        (None, None, None, 128)   0         \n",
      "_________________________________________________________________\n",
      "conv_pad_4 (ZeroPadding2D)   (None, None, None, 128)   0         \n",
      "_________________________________________________________________\n",
      "conv_dw_4 (DepthwiseConv2D)  (None, None, None, 128)   1152      \n",
      "_________________________________________________________________\n",
      "conv_dw_4_bn (BatchNormaliza (None, None, None, 128)   512       \n",
      "_________________________________________________________________\n",
      "conv_dw_4_relu (ReLU)        (None, None, None, 128)   0         \n",
      "_________________________________________________________________\n",
      "conv_pw_4 (Conv2D)           (None, None, None, 256)   32768     \n",
      "_________________________________________________________________\n",
      "conv_pw_4_bn (BatchNormaliza (None, None, None, 256)   1024      \n",
      "_________________________________________________________________\n",
      "conv_pw_4_relu (ReLU)        (None, None, None, 256)   0         \n",
      "_________________________________________________________________\n",
      "conv_dw_5 (DepthwiseConv2D)  (None, None, None, 256)   2304      \n",
      "_________________________________________________________________\n",
      "conv_dw_5_bn (BatchNormaliza (None, None, None, 256)   1024      \n",
      "_________________________________________________________________\n",
      "conv_dw_5_relu (ReLU)        (None, None, None, 256)   0         \n",
      "_________________________________________________________________\n",
      "conv_pw_5 (Conv2D)           (None, None, None, 256)   65536     \n",
      "_________________________________________________________________\n",
      "conv_pw_5_bn (BatchNormaliza (None, None, None, 256)   1024      \n",
      "_________________________________________________________________\n",
      "conv_pw_5_relu (ReLU)        (None, None, None, 256)   0         \n",
      "_________________________________________________________________\n",
      "conv_pad_6 (ZeroPadding2D)   (None, None, None, 256)   0         \n",
      "_________________________________________________________________\n",
      "conv_dw_6 (DepthwiseConv2D)  (None, None, None, 256)   2304      \n",
      "_________________________________________________________________\n",
      "conv_dw_6_bn (BatchNormaliza (None, None, None, 256)   1024      \n",
      "_________________________________________________________________\n",
      "conv_dw_6_relu (ReLU)        (None, None, None, 256)   0         \n",
      "_________________________________________________________________\n",
      "conv_pw_6 (Conv2D)           (None, None, None, 512)   131072    \n",
      "_________________________________________________________________\n",
      "conv_pw_6_bn (BatchNormaliza (None, None, None, 512)   2048      \n",
      "_________________________________________________________________\n",
      "conv_pw_6_relu (ReLU)        (None, None, None, 512)   0         \n",
      "_________________________________________________________________\n",
      "conv_dw_7 (DepthwiseConv2D)  (None, None, None, 512)   4608      \n",
      "_________________________________________________________________\n",
      "conv_dw_7_bn (BatchNormaliza (None, None, None, 512)   2048      \n",
      "_________________________________________________________________\n",
      "conv_dw_7_relu (ReLU)        (None, None, None, 512)   0         \n",
      "_________________________________________________________________\n",
      "conv_pw_7 (Conv2D)           (None, None, None, 512)   262144    \n",
      "_________________________________________________________________\n",
      "conv_pw_7_bn (BatchNormaliza (None, None, None, 512)   2048      \n",
      "_________________________________________________________________\n",
      "conv_pw_7_relu (ReLU)        (None, None, None, 512)   0         \n",
      "_________________________________________________________________\n",
      "conv_dw_8 (DepthwiseConv2D)  (None, None, None, 512)   4608      \n",
      "_________________________________________________________________\n",
      "conv_dw_8_bn (BatchNormaliza (None, None, None, 512)   2048      \n",
      "_________________________________________________________________\n",
      "conv_dw_8_relu (ReLU)        (None, None, None, 512)   0         \n",
      "_________________________________________________________________\n",
      "conv_pw_8 (Conv2D)           (None, None, None, 512)   262144    \n",
      "_________________________________________________________________\n",
      "conv_pw_8_bn (BatchNormaliza (None, None, None, 512)   2048      \n",
      "_________________________________________________________________\n",
      "conv_pw_8_relu (ReLU)        (None, None, None, 512)   0         \n",
      "_________________________________________________________________\n",
      "conv_dw_9 (DepthwiseConv2D)  (None, None, None, 512)   4608      \n",
      "_________________________________________________________________\n",
      "conv_dw_9_bn (BatchNormaliza (None, None, None, 512)   2048      \n",
      "_________________________________________________________________\n",
      "conv_dw_9_relu (ReLU)        (None, None, None, 512)   0         \n",
      "_________________________________________________________________\n",
      "conv_pw_9 (Conv2D)           (None, None, None, 512)   262144    \n",
      "_________________________________________________________________\n",
      "conv_pw_9_bn (BatchNormaliza (None, None, None, 512)   2048      \n",
      "_________________________________________________________________\n",
      "conv_pw_9_relu (ReLU)        (None, None, None, 512)   0         \n",
      "_________________________________________________________________\n",
      "conv_dw_10 (DepthwiseConv2D) (None, None, None, 512)   4608      \n",
      "_________________________________________________________________\n",
      "conv_dw_10_bn (BatchNormaliz (None, None, None, 512)   2048      \n",
      "_________________________________________________________________\n",
      "conv_dw_10_relu (ReLU)       (None, None, None, 512)   0         \n",
      "_________________________________________________________________\n",
      "conv_pw_10 (Conv2D)          (None, None, None, 512)   262144    \n",
      "_________________________________________________________________\n",
      "conv_pw_10_bn (BatchNormaliz (None, None, None, 512)   2048      \n",
      "_________________________________________________________________\n",
      "conv_pw_10_relu (ReLU)       (None, None, None, 512)   0         \n",
      "_________________________________________________________________\n",
      "conv_dw_11 (DepthwiseConv2D) (None, None, None, 512)   4608      \n",
      "_________________________________________________________________\n",
      "conv_dw_11_bn (BatchNormaliz (None, None, None, 512)   2048      \n",
      "_________________________________________________________________\n",
      "conv_dw_11_relu (ReLU)       (None, None, None, 512)   0         \n",
      "_________________________________________________________________\n",
      "conv_pw_11 (Conv2D)          (None, None, None, 512)   262144    \n",
      "_________________________________________________________________\n",
      "conv_pw_11_bn (BatchNormaliz (None, None, None, 512)   2048      \n",
      "_________________________________________________________________\n",
      "conv_pw_11_relu (ReLU)       (None, None, None, 512)   0         \n",
      "_________________________________________________________________\n",
      "conv_pad_12 (ZeroPadding2D)  (None, None, None, 512)   0         \n",
      "_________________________________________________________________\n",
      "conv_dw_12 (DepthwiseConv2D) (None, None, None, 512)   4608      \n",
      "_________________________________________________________________\n",
      "conv_dw_12_bn (BatchNormaliz (None, None, None, 512)   2048      \n",
      "_________________________________________________________________\n",
      "conv_dw_12_relu (ReLU)       (None, None, None, 512)   0         \n",
      "_________________________________________________________________\n",
      "conv_pw_12 (Conv2D)          (None, None, None, 1024)  524288    \n",
      "_________________________________________________________________\n",
      "conv_pw_12_bn (BatchNormaliz (None, None, None, 1024)  4096      \n",
      "_________________________________________________________________\n",
      "conv_pw_12_relu (ReLU)       (None, None, None, 1024)  0         \n",
      "_________________________________________________________________\n",
      "conv_dw_13 (DepthwiseConv2D) (None, None, None, 1024)  9216      \n",
      "_________________________________________________________________\n",
      "conv_dw_13_bn (BatchNormaliz (None, None, None, 1024)  4096      \n",
      "_________________________________________________________________\n",
      "conv_dw_13_relu (ReLU)       (None, None, None, 1024)  0         \n",
      "_________________________________________________________________\n",
      "conv_pw_13 (Conv2D)          (None, None, None, 1024)  1048576   \n",
      "_________________________________________________________________\n",
      "conv_pw_13_bn (BatchNormaliz (None, None, None, 1024)  4096      \n",
      "_________________________________________________________________\n",
      "conv_pw_13_relu (ReLU)       (None, None, None, 1024)  0         \n",
      "=================================================================\n",
      "Total params: 3,228,864\n",
      "Trainable params: 3,206,976\n",
      "Non-trainable params: 21,888\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3.6 (tensorflow)",
   "language": "python",
   "name": "tensorflow"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}