CNN_Project_Dog breed_Identification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "2kIWaR5ZpKlJ"
   },
   "source": [
    "## Dog Breed Classification\n",
    "\n",
    "In this project we will use traditional CNN, CNN with data augmentation and finally transfer Learning by VGG16 model with weights pre-trained on Imagenet to solve the dog breed classification problem"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "F7MDmaAw2xGO"
   },
   "source": [
    "### Load Dataset Files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "fVhB9OopxFbX",
    "outputId": "7a9eb737-0086-4047-ecf1-53ba34b4218c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n"
     ]
    }
   ],
   "source": [
    "from pandas import (\n",
    "    DataFrame, HDFStore\n",
    ")\n",
    "import pandas as pd\n",
    "from google.colab import drive\n",
    "drive.mount('/content/drive')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "1q2zzIaUprk_"
   },
   "source": [
    "Now, upload the given dataset file shared with you in your google drive and give its path for the below given `project_path` variable. For example, a path is given below according to the file path in our google drive. You need to change this to match the path of yours."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Tp6FvAToxUFs"
   },
   "outputs": [],
   "source": [
    "project_path =  \"/content/drive/My Drive/DogBreed_Classification/\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "rydR_j8lqUei"
   },
   "source": [
    "Run the below code to extract all the images in the train.zip files given in the dataset. We are going to use these images as train and validation sets and their labels in further steps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "3350WZM4w4EL"
   },
   "outputs": [],
   "source": [
    "from zipfile import ZipFile\n",
    "with ZipFile(project_path+'train.zip', 'r') as z:\n",
    "  z.extractall()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "3NHq1iBCfFjE"
   },
   "source": [
    "Repeat the same step for test.zip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "_fxzynvB2YCb"
   },
   "outputs": [],
   "source": [
    "from zipfile import ZipFile\n",
    "with ZipFile(project_path+'test.zip', 'r') as z:\n",
    "  z.extractall()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "jnUMhQrDfJmz"
   },
   "source": [
    "Repeat the same step for sample_submission.csv.zip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "4PyTxE8q2jLf"
   },
   "outputs": [],
   "source": [
    "from zipfile import ZipFile\n",
    "with ZipFile(project_path+'sample_submission.csv.zip', 'r') as z:\n",
    "  z.extractall()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "2G9RIxB-fOLT"
   },
   "source": [
    "Repeat the same step for labels.csv.zip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "rXtnEoEixbgi"
   },
   "outputs": [],
   "source": [
    "from zipfile import ZipFile\n",
    "with ZipFile(project_path+'labels.csv.zip', 'r') as z:\n",
    "  z.extractall()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "sJc1lVrW_jmL"
   },
   "source": [
    "After this process, we will have 4 files - Train folder, test folder and labels.csv and sample_submission.csv as part of your google drive"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "aYmJKmDqqpng"
   },
   "source": [
    "### Read labels.csv file using pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WmlJ2VMY96IZ"
   },
   "outputs": [],
   "source": [
    "labels = pd.read_csv('./labels.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 363
    },
    "colab_type": "code",
    "id": "hPvb1RSc96If",
    "outputId": "6696639d-b1ea-4cba-89ff-f1e303f6607a"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>breed</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>000bec180eb18c7604dcecc8fe0dba07</td>\n",
       "      <td>boston_bull</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>001513dfcb2ffafc82cccf4d8bbaba97</td>\n",
       "      <td>dingo</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>001cdf01b096e06d78e9e5112d419397</td>\n",
       "      <td>pekinese</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>00214f311d5d2247d5dfe4fe24b2303d</td>\n",
       "      <td>bluetick</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0021f9ceb3235effd7fcde7f7538ed62</td>\n",
       "      <td>golden_retriever</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>002211c81b498ef88e1b40b9abf84e1d</td>\n",
       "      <td>bedlington_terrier</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>00290d3e1fdd27226ba27a8ce248ce85</td>\n",
       "      <td>bedlington_terrier</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>002a283a315af96eaea0e28e7163b21b</td>\n",
       "      <td>borzoi</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>003df8b8a8b05244b1d920bb6cf451f9</td>\n",
       "      <td>basenji</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0042188c895a2f14ef64a918ed9c7b64</td>\n",
       "      <td>scottish_deerhound</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                 id               breed\n",
       "0  000bec180eb18c7604dcecc8fe0dba07         boston_bull\n",
       "1  001513dfcb2ffafc82cccf4d8bbaba97               dingo\n",
       "2  001cdf01b096e06d78e9e5112d419397            pekinese\n",
       "3  00214f311d5d2247d5dfe4fe24b2303d            bluetick\n",
       "4  0021f9ceb3235effd7fcde7f7538ed62    golden_retriever\n",
       "5  002211c81b498ef88e1b40b9abf84e1d  bedlington_terrier\n",
       "6  00290d3e1fdd27226ba27a8ce248ce85  bedlington_terrier\n",
       "7  002a283a315af96eaea0e28e7163b21b              borzoi\n",
       "8  003df8b8a8b05244b1d920bb6cf451f9             basenji\n",
       "9  0042188c895a2f14ef64a918ed9c7b64  scottish_deerhound"
      ]
     },
     "execution_count": 8,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "labels.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "QP8YAzQvqyK-"
   },
   "source": [
    "### Print the count of each category of Dogs given in the dataset\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 225
    },
    "colab_type": "code",
    "id": "3L2naXlr96Im",
    "outputId": "d7762c65-3358-4974-947f-4b23c157b683"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "scottish_deerhound      126\n",
      "maltese_dog             117\n",
      "afghan_hound            116\n",
      "entlebucher             115\n",
      "bernese_mountain_dog    114\n",
      "                       ... \n",
      "komondor                 67\n",
      "golden_retriever         67\n",
      "brabancon_griffon        67\n",
      "eskimo_dog               66\n",
      "briard                   66\n",
      "Name: breed, Length: 120, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "print(labels.breed.value_counts())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "WI94_Qcc0D4M"
   },
   "source": [
    "### Get one-hot encodings of labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "j_5Y5ssN180m",
    "outputId": "0736b99f-aac4-41d7-f3fb-7d0404ee1c42"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TensorFlow 1.x selected.\n"
     ]
    }
   ],
   "source": [
    "%tensorflow_version 1.x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "Lw5G8ROw4AD0",
    "outputId": "5b40e85a-b231-4ba9-de18-bb747b71c911"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.15.2\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "print(tf.__version__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Q48iAcY196I3"
   },
   "outputs": [],
   "source": [
    "labels_ec = tf.keras.utils.to_categorical(labels['breed'].factorize()[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "9nlWmRNM96I8",
    "outputId": "91a3711b-3896-4850-c2fd-7a766f7750f9"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10222, 120)"
      ]
     },
     "execution_count": 14,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "labels_ec.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "VWaJ9naXfoiU"
   },
   "source": [
    "## Preparing training dataset\n",
    "1. Write a code which reads each and every id from labels.csv file and loads the corresponding image (in RGB - 128, 128, 3) from the train folder. <br>\n",
    "2. Create 2 variables <br> \n",
    "     a.  x_train - Should have all the images of the dogs from train folder <br>\n",
    "     b.  y_train - Corresponding label of the dog <br>\n",
    "<u>Note:</u> The id of the dog images and its corresponding labels are available in labels.csv file   \n",
    "<u>Hint:</u> Watch the video shared on \"Preparing the training dataset\" if you face issue on creating the training dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "aC2f9ecR0XGR",
    "outputId": "e9510ee7-8e93-4256-b2c8-6ed2a0731c46"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 10222/10222 [00:26<00:00, 379.76it/s]\n"
     ]
    }
   ],
   "source": [
    "from tqdm import tqdm\n",
    "import cv2\n",
    "img_rows = 128\n",
    "img_cols = 128\n",
    "x_train = []\n",
    "y_train = []\n",
    "\n",
    "for f,img in tqdm(labels.values):\n",
    "  train_img = cv2.imread('./train/{}.jpg'.format(f),1)\n",
    "  train_img_resize = cv2.resize(train_img,(img_rows, img_cols))\n",
    "  x_train.append(train_img_resize)\n",
    "  y_train.append(img)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 52
    },
    "colab_type": "code",
    "id": "nkkZEpOe0ipk",
    "outputId": "bfb90199-f6a1-4c51-e360-9d975dd2f1d8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "saluki\n",
      "(128, 128, 3)\n"
     ]
    }
   ],
   "source": [
    "print(y_train[110])\n",
    "print(x_train[110].shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "6ioWDEgElBOs"
   },
   "source": [
    "Normalize the training data and convert into 4 dimensions so that it can be used as an input to conv layers in the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "ARn76j3U1CDa",
    "outputId": "5be3e5a6-29c0-41be-e5c9-f37da491fd4b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(128, 128, 3)"
      ]
     },
     "execution_count": 17,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dum = pd.get_dummies(y_train)\n",
    "encoded_labels = dum\n",
    "y_train = dum\n",
    "\n",
    "import numpy as np\n",
    "y_train = np.array(y_train)\n",
    "\n",
    "x_train = np.array(x_train)\n",
    "x_train[0].shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bdCXuAE11gZL"
   },
   "source": [
    "### Split the training and validation data from `x_train_data` and `y_train_data` obtained from above step"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 52
    },
    "colab_type": "code",
    "id": "kpWx-pgV96Jv",
    "outputId": "98bed980-a56b-40ee-c5dc-4d13b0e3dad8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8177\n",
      "2045\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "x_train2, x_val, y_train2, y_val = train_test_split(x_train, y_train, test_size=0.2, random_state=2)\n",
    "print (len(x_train2))\n",
    "print (len(x_val))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "qg7Oth8A2Wmy",
    "outputId": "a57a7c90-1383-4c55-9e53-8bc3185a8d05"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(128, 128, 3)"
      ]
     },
     "execution_count": 19,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_train2[0].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 86
    },
    "colab_type": "code",
    "id": "Wfk9eM0g2ZXj",
    "outputId": "739f7ae8-3a1d-4428-89b9-89565732c52e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(8177, 128, 128, 3)\n",
      "(2045, 128, 128, 3)\n",
      "(8177, 120)\n",
      "(2045, 120)\n"
     ]
    }
   ],
   "source": [
    "x_train2 = x_train2.reshape(x_train2.shape[0],128,128,3)\n",
    "x_val = x_val.reshape(x_val.shape[0],128,128,3)\n",
    "x_train2 = x_train2/255.\n",
    "\n",
    "x_val = x_val/255.\n",
    "\n",
    "print (x_train2.shape)\n",
    "print (x_val.shape)\n",
    "\n",
    "\n",
    "print(y_train2.shape)\n",
    "print(y_val.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "XkL-N1jDsU8m"
   },
   "source": [
    "### Loading the test data\n",
    "Read the id column from the samples_submission.csv and store it in test_img"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 430
    },
    "colab_type": "code",
    "id": "DnpXdpd9b3E7",
    "outputId": "d0d41e0e-5bc5-4828-f94a-f9061ccc6f00"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>affenpinscher</th>\n",
       "      <th>afghan_hound</th>\n",
       "      <th>african_hunting_dog</th>\n",
       "      <th>airedale</th>\n",
       "      <th>american_staffordshire_terrier</th>\n",
       "      <th>appenzeller</th>\n",
       "      <th>australian_terrier</th>\n",
       "      <th>basenji</th>\n",
       "      <th>basset</th>\n",
       "      <th>beagle</th>\n",
       "      <th>bedlington_terrier</th>\n",
       "      <th>bernese_mountain_dog</th>\n",
       "      <th>black-and-tan_coonhound</th>\n",
       "      <th>blenheim_spaniel</th>\n",
       "      <th>bloodhound</th>\n",
       "      <th>bluetick</th>\n",
       "      <th>border_collie</th>\n",
       "      <th>border_terrier</th>\n",
       "      <th>borzoi</th>\n",
       "      <th>boston_bull</th>\n",
       "      <th>bouvier_des_flandres</th>\n",
       "      <th>boxer</th>\n",
       "      <th>brabancon_griffon</th>\n",
       "      <th>briard</th>\n",
       "      <th>brittany_spaniel</th>\n",
       "      <th>bull_mastiff</th>\n",
       "      <th>cairn</th>\n",
       "      <th>cardigan</th>\n",
       "      <th>chesapeake_bay_retriever</th>\n",
       "      <th>chihuahua</th>\n",
       "      <th>chow</th>\n",
       "      <th>clumber</th>\n",
       "      <th>cocker_spaniel</th>\n",
       "      <th>collie</th>\n",
       "      <th>curly-coated_retriever</th>\n",
       "      <th>dandie_dinmont</th>\n",
       "      <th>dhole</th>\n",
       "      <th>dingo</th>\n",
       "      <th>doberman</th>\n",
       "      <th>...</th>\n",
       "      <th>norwegian_elkhound</th>\n",
       "      <th>norwich_terrier</th>\n",
       "      <th>old_english_sheepdog</th>\n",
       "      <th>otterhound</th>\n",
       "      <th>papillon</th>\n",
       "      <th>pekinese</th>\n",
       "      <th>pembroke</th>\n",
       "      <th>pomeranian</th>\n",
       "      <th>pug</th>\n",
       "      <th>redbone</th>\n",
       "      <th>rhodesian_ridgeback</th>\n",
       "      <th>rottweiler</th>\n",
       "      <th>saint_bernard</th>\n",
       "      <th>saluki</th>\n",
       "      <th>samoyed</th>\n",
       "      <th>schipperke</th>\n",
       "      <th>scotch_terrier</th>\n",
       "      <th>scottish_deerhound</th>\n",
       "      <th>sealyham_terrier</th>\n",
       "      <th>shetland_sheepdog</th>\n",
       "      <th>shih-tzu</th>\n",
       "      <th>siberian_husky</th>\n",
       "      <th>silky_terrier</th>\n",
       "      <th>soft-coated_wheaten_terrier</th>\n",
       "      <th>staffordshire_bullterrier</th>\n",
       "      <th>standard_poodle</th>\n",
       "      <th>standard_schnauzer</th>\n",
       "      <th>sussex_spaniel</th>\n",
       "      <th>tibetan_mastiff</th>\n",
       "      <th>tibetan_terrier</th>\n",
       "      <th>toy_poodle</th>\n",
       "      <th>toy_terrier</th>\n",
       "      <th>vizsla</th>\n",
       "      <th>walker_hound</th>\n",
       "      <th>weimaraner</th>\n",
       "      <th>welsh_springer_spaniel</th>\n",
       "      <th>west_highland_white_terrier</th>\n",
       "      <th>whippet</th>\n",
       "      <th>wire-haired_fox_terrier</th>\n",
       "      <th>yorkshire_terrier</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3004</th>\n",
       "      <td>49de8ff5d5e6ab332356610bcb042948</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>...</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9088</th>\n",
       "      <td>e1fa4eb8a57184773ac11efe0de11e92</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>...</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4468</th>\n",
       "      <td>6cf486696a155c9e41397e18aa58f144</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>...</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1317</th>\n",
       "      <td>1f503f2c112adff1bd9ccc23c1377c59</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>...</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10165</th>\n",
       "      <td>fb757be9d9a6676284ea6d67d497160b</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>...</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5187</th>\n",
       "      <td>7eb470b7b4efcbcdb88cd8a249fbafcf</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>...</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6683</th>\n",
       "      <td>a444cfda52fc5918caa524535df959be</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>...</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5182</th>\n",
       "      <td>7e8b64637f260d3e678356839ae94fb2</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>...</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1920</th>\n",
       "      <td>2e0aeaa29b6fe3dcd092c020e6fd45c9</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>...</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2045</th>\n",
       "      <td>30f4be30c5f35ee6ba22524f02bddb2d</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>...</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "      <td>0.008333</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10 rows × 121 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                     id  ...  yorkshire_terrier\n",
       "3004   49de8ff5d5e6ab332356610bcb042948  ...           0.008333\n",
       "9088   e1fa4eb8a57184773ac11efe0de11e92  ...           0.008333\n",
       "4468   6cf486696a155c9e41397e18aa58f144  ...           0.008333\n",
       "1317   1f503f2c112adff1bd9ccc23c1377c59  ...           0.008333\n",
       "10165  fb757be9d9a6676284ea6d67d497160b  ...           0.008333\n",
       "5187   7eb470b7b4efcbcdb88cd8a249fbafcf  ...           0.008333\n",
       "6683   a444cfda52fc5918caa524535df959be  ...           0.008333\n",
       "5182   7e8b64637f260d3e678356839ae94fb2  ...           0.008333\n",
       "1920   2e0aeaa29b6fe3dcd092c020e6fd45c9  ...           0.008333\n",
       "2045   30f4be30c5f35ee6ba22524f02bddb2d  ...           0.008333\n",
       "\n",
       "[10 rows x 121 columns]"
      ]
     },
     "execution_count": 21,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_img = pd.read_csv('./sample_submission.csv')\n",
    "test_img.sample(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "DEJqZIMbm0Jo"
   },
   "source": [
    "Run the below code to load the test image files in x_test_feature"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "zf7n4WG-b3Hv",
    "outputId": "99898a67-a473-495a-8827-dc7863ed187e"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 10357/10357 [00:26<00:00, 387.37it/s]\n"
     ]
    }
   ],
   "source": [
    "x_test_feature = []\n",
    "img_rows = 128\n",
    "img_cols = 128\n",
    "i = 0 # initialisation\n",
    "for f in tqdm(test_img.values): # f for format ,jpg\n",
    "    img = cv2.imread('./test/{}.jpg'.format(f[0]), 1)\n",
    "    img_resize = cv2.resize(img, (img_rows, img_cols)) \n",
    "    x_test_feature.append(img_resize)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "9My6qSyDnE-_"
   },
   "source": [
    "Normalize the test data and convert it into 4 dimensions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "93n-IntMnJGI",
    "outputId": "04dadf5d-5544-4eda-8c88-76f625c632c6"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(128, 128, 3)"
      ]
     },
     "execution_count": 24,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_test_feature = np.array(x_test_feature)\n",
    "x_test_feature[0].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "Gv8bEyNX2kRD",
    "outputId": "93d493ff-03e0-482a-aacf-973a2e07df83"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2045, 128, 128, 3)\n"
     ]
    }
   ],
   "source": [
    "x_test_feature = x_test_feature.reshape(x_test_feature.shape[0],128,128,3)\n",
    "x_test_feature = x_val/255.\n",
    "print (x_test_feature.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "zKezNJVMsocP"
   },
   "source": [
    "### Build a basic conv neural network with 2 conv layers (kernel sizes - 5 and 3) add layers as mentioned below for classification.\n",
    "\n",
    "1. Add a Dense layer with 256 neurons with `relu` activation\n",
    "\n",
    "2. Add a Dense layer with 120 neurons as final layer (as there are 120 classes in the given dataset) with `softmax` activation for classifiaction. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "D2jxTY2S96J4"
   },
   "outputs": [],
   "source": [
    "tf.keras.backend.clear_session()\n",
    "model = tf.keras.models.Sequential()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 89
    },
    "colab_type": "code",
    "id": "f_BAvCzo96J6",
    "outputId": "7ccad6e4-74fa-4cad-ace4-4552f3d8b815"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "If using Keras pass *_constraint arguments to layers.\n"
     ]
    }
   ],
   "source": [
    "model.add(tf.keras.layers.Conv2D(filters=32, kernel_size=(5,5),input_shape=(128, 128, 3), activation='relu'))\n",
    "model.add(tf.keras.layers.Conv2D(filters=32, kernel_size=(3,3), activation='relu'))\n",
    "model.add(tf.keras.layers.Flatten())\n",
    "model.add(tf.keras.layers.Dense(256, activation='relu'))\n",
    "model.add(tf.keras.layers.Dense(120, activation='softmax'))\n",
    "model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ui8EXw6_oqpR"
   },
   "source": [
    "### Use batch_size = 128 and epochs = 10 and execute the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 86
    },
    "colab_type": "code",
    "id": "IriIc37NozbK",
    "outputId": "40b84a5e-9ca9-4bc4-8dc5-124d88ded10c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(8177, 128, 128, 3)\n",
      "(2045, 128, 128, 3)\n",
      "(8177, 120)\n",
      "(2045, 120)\n"
     ]
    }
   ],
   "source": [
    "print (x_train2.shape)\n",
    "print (x_val.shape)\n",
    "print(y_train2.shape)\n",
    "print(y_val.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 399
    },
    "colab_type": "code",
    "id": "0PR9j5_Xozmd",
    "outputId": "938722e4-9cc0-41c4-ae08-5c057880d5b5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 8177 samples, validate on 2045 samples\n",
      "Epoch 1/10\n",
      "8177/8177 [==============================] - 19s 2ms/sample - loss: 6.5151 - acc: 0.0111 - val_loss: 4.7846 - val_acc: 0.0103\n",
      "Epoch 2/10\n",
      "8177/8177 [==============================] - 9s 1ms/sample - loss: 4.7305 - acc: 0.0201 - val_loss: 4.7400 - val_acc: 0.0166\n",
      "Epoch 3/10\n",
      "8177/8177 [==============================] - 10s 1ms/sample - loss: 4.0708 - acc: 0.1375 - val_loss: 4.8528 - val_acc: 0.0328\n",
      "Epoch 4/10\n",
      "8177/8177 [==============================] - 10s 1ms/sample - loss: 1.7370 - acc: 0.6131 - val_loss: 6.5872 - val_acc: 0.0264\n",
      "Epoch 5/10\n",
      "8177/8177 [==============================] - 10s 1ms/sample - loss: 0.2581 - acc: 0.9502 - val_loss: 9.2650 - val_acc: 0.0293\n",
      "Epoch 6/10\n",
      "8177/8177 [==============================] - 10s 1ms/sample - loss: 0.0631 - acc: 0.9941 - val_loss: 9.5597 - val_acc: 0.0274\n",
      "Epoch 7/10\n",
      "8177/8177 [==============================] - 10s 1ms/sample - loss: 0.0226 - acc: 0.9980 - val_loss: 10.2662 - val_acc: 0.0284\n",
      "Epoch 8/10\n",
      "8177/8177 [==============================] - 10s 1ms/sample - loss: 0.0270 - acc: 0.9982 - val_loss: 8.9769 - val_acc: 0.0337\n",
      "Epoch 9/10\n",
      "8177/8177 [==============================] - 10s 1ms/sample - loss: 0.0195 - acc: 0.9984 - val_loss: 9.7681 - val_acc: 0.0318\n",
      "Epoch 10/10\n",
      "8177/8177 [==============================] - 10s 1ms/sample - loss: 0.0127 - acc: 0.9991 - val_loss: 9.4433 - val_acc: 0.0293\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f681112ae80>"
      ]
     },
     "execution_count": 29,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(x_train2,\n",
    "          y_train2,          \n",
    "          validation_data=(x_val,y_val),\n",
    "          epochs=10,\n",
    "          batch_size=128)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Z8hWaKmjoz69"
   },
   "source": [
    "#The model accuracy is very poor !!!!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "agJKkc6xtKiq"
   },
   "source": [
    "### Use Data Augmentation in the above model to see if the accuracy improves\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "31Mn8qnZb3Ru",
    "outputId": "896a66f2-d797-45a7-f59d-1274ba7006d3"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from keras.preprocessing.image import ImageDataGenerator\n",
    "datagen = ImageDataGenerator(\n",
    "    featurewise_center=False, \n",
    "    samplewise_center=False,  \n",
    "    featurewise_std_normalization=False,  \n",
    "    samplewise_std_normalization=False,  \n",
    "    zca_whitening=False,  \n",
    "    rotation_range=50,  \n",
    "    width_shift_range=0.2, \n",
    "    height_shift_range=0.2,  \n",
    "    horizontal_flip=True,  \n",
    "    vertical_flip=True)  \n",
    "datagen.fit(x_train2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 399
    },
    "colab_type": "code",
    "id": "gDLQVFDP96KI",
    "outputId": "c0ab0674-1ef9-4da1-a228-9c4b75ff7881"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 8177 samples, validate on 2045 samples\n",
      "Epoch 1/10\n",
      "8177/8177 [==============================] - 10s 1ms/sample - loss: 0.0152 - acc: 0.9989 - val_loss: 8.8782 - val_acc: 0.0298\n",
      "Epoch 2/10\n",
      "8177/8177 [==============================] - 10s 1ms/sample - loss: 0.0109 - acc: 0.9990 - val_loss: 10.2616 - val_acc: 0.0259\n",
      "Epoch 3/10\n",
      "8177/8177 [==============================] - 10s 1ms/sample - loss: 0.0119 - acc: 0.9991 - val_loss: 9.3449 - val_acc: 0.0264\n",
      "Epoch 4/10\n",
      "8177/8177 [==============================] - 10s 1ms/sample - loss: 0.0080 - acc: 0.9990 - val_loss: 10.3343 - val_acc: 0.0284\n",
      "Epoch 5/10\n",
      "8177/8177 [==============================] - 10s 1ms/sample - loss: 0.0078 - acc: 0.9991 - val_loss: 9.1303 - val_acc: 0.0269\n",
      "Epoch 6/10\n",
      "8177/8177 [==============================] - 10s 1ms/sample - loss: 0.0061 - acc: 0.9991 - val_loss: 9.4497 - val_acc: 0.0274\n",
      "Epoch 7/10\n",
      "8177/8177 [==============================] - 10s 1ms/sample - loss: 0.0113 - acc: 0.9988 - val_loss: 9.2374 - val_acc: 0.0269\n",
      "Epoch 8/10\n",
      "8177/8177 [==============================] - 10s 1ms/sample - loss: 0.0065 - acc: 0.9991 - val_loss: 9.5107 - val_acc: 0.0254\n",
      "Epoch 9/10\n",
      "8177/8177 [==============================] - 10s 1ms/sample - loss: 0.0065 - acc: 0.9990 - val_loss: 9.4652 - val_acc: 0.0274\n",
      "Epoch 10/10\n",
      "8177/8177 [==============================] - 10s 1ms/sample - loss: 0.0057 - acc: 0.9991 - val_loss: 9.2403 - val_acc: 0.0254\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f67b43e24e0>"
      ]
     },
     "execution_count": 31,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(x_train2,\n",
    "          y_train2,          \n",
    "          validation_data=(x_val,y_val),\n",
    "          epochs=10,\n",
    "          batch_size=128)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "6sssbaTfxlkk"
   },
   "source": [
    "### Using the above objects, create the image generators with variable names `train_generator` and `val_generator`\n",
    "\n",
    "You need to use train_datagen.flow() and val_datagen.flow()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "sehaRgT-96KQ"
   },
   "outputs": [],
   "source": [
    "train_generator = datagen.flow(x_train2,y_train2,batch_size=32)\n",
    "val_generator = datagen.flow(x_val,y_val,batch_size=32)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "TVFQJZw3x4-C"
   },
   "source": [
    "### Fit the model using fit_generator() using `train_generator` and `val_generator` from the above step with 10 epochs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 555
    },
    "colab_type": "code",
    "id": "J1K2MqHbuPUa",
    "outputId": "28a7ee29-31c2-464c-926d-c6f6b24dbcec"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "8176/8177 [============================>.] - ETA: 0s - loss: 4.5476 - acc: 0.0306Epoch 1/10\n",
      "8177/8177 [==============================] - 986s 121ms/step - loss: 4.5476 - acc: 0.0306 - val_loss: 4.4362 - val_acc: 0.0494\n",
      "Epoch 2/10\n",
      "8176/8177 [============================>.] - ETA: 0s - loss: 4.1453 - acc: 0.0731Epoch 1/10\n",
      "8177/8177 [==============================] - 980s 120ms/step - loss: 4.1453 - acc: 0.0731 - val_loss: 4.4448 - val_acc: 0.0538\n",
      "Epoch 3/10\n",
      "8176/8177 [============================>.] - ETA: 0s - loss: 3.9265 - acc: 0.1007Epoch 1/10\n",
      "8177/8177 [==============================] - 977s 119ms/step - loss: 3.9265 - acc: 0.1007 - val_loss: 4.4449 - val_acc: 0.0660\n",
      "Epoch 4/10\n",
      "8176/8177 [============================>.] - ETA: 0s - loss: 3.7690 - acc: 0.1222Epoch 1/10\n",
      "8177/8177 [==============================] - 980s 120ms/step - loss: 3.7690 - acc: 0.1222 - val_loss: 4.4168 - val_acc: 0.0748\n",
      "Epoch 5/10\n",
      "8176/8177 [============================>.] - ETA: 0s - loss: 3.6525 - acc: 0.1399Epoch 1/10\n",
      "8177/8177 [==============================] - 984s 120ms/step - loss: 3.6524 - acc: 0.1399 - val_loss: 4.4323 - val_acc: 0.0841\n",
      "Epoch 6/10\n",
      "8176/8177 [============================>.] - ETA: 0s - loss: 3.5659 - acc: 0.1532Epoch 1/10\n",
      "8177/8177 [==============================] - 982s 120ms/step - loss: 3.5659 - acc: 0.1532 - val_loss: 4.7046 - val_acc: 0.0822\n",
      "Epoch 7/10\n",
      "8176/8177 [============================>.] - ETA: 0s - loss: 3.4974 - acc: 0.1646Epoch 1/10\n",
      "8177/8177 [==============================] - 979s 120ms/step - loss: 3.4974 - acc: 0.1646 - val_loss: 4.7281 - val_acc: 0.0758\n",
      "Epoch 8/10\n",
      "8176/8177 [============================>.] - ETA: 0s - loss: 3.4415 - acc: 0.1741Epoch 1/10\n",
      "8177/8177 [==============================] - 984s 120ms/step - loss: 3.4415 - acc: 0.1741 - val_loss: 4.7597 - val_acc: 0.0792\n",
      "Epoch 9/10\n",
      "8176/8177 [============================>.] - ETA: 0s - loss: 3.3998 - acc: 0.1814Epoch 1/10\n",
      "8177/8177 [==============================] - 984s 120ms/step - loss: 3.3997 - acc: 0.1814 - val_loss: 4.9201 - val_acc: 0.0817\n",
      "Epoch 10/10\n",
      "8176/8177 [============================>.] - ETA: 0s - loss: 3.3590 - acc: 0.1878Epoch 1/10\n",
      "8177/8177 [==============================] - 986s 121ms/step - loss: 3.3590 - acc: 0.1878 - val_loss: 4.9387 - val_acc: 0.0875\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f67b28fb828>"
      ]
     },
     "execution_count": 33,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit_generator(train_generator,steps_per_epoch=x_train2.shape[0],\n",
    "                    epochs=10,\n",
    "                    validation_data=val_generator)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Q2zmLztqo5DY"
   },
   "source": [
    "# Model accuracy is still poor!!!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "rSTATrhsAo7L"
   },
   "source": [
    "### Lets use Transfer Learning\n",
    "\n",
    "Download the vgg wieght file from here : https://github.com/MinerKasch/applied_deep_learning/blob/master/vgg16_weights_tf_dim_ordering_tf_kernels_notop.h5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "zy5JdbW6pIvD"
   },
   "source": [
    "Use the below code to load VGG16 weights trained on ImageNet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "frdQGG6kBtEV"
   },
   "outputs": [],
   "source": [
    "tl_path = \"/content/drive/My Drive/Colab Notebooks/\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 52
    },
    "colab_type": "code",
    "id": "yrqs0zg7ApNw",
    "outputId": "b65783b6-25a4-4b90-90b5-7ef6d7fcf24b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/keras/backend/tensorflow_backend.py:4070: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from keras.applications.vgg16 import VGG16, preprocess_input\n",
    "# Instantiate the model with the pre-trained weights (no top)\n",
    "base_model= VGG16(weights=(tl_path+'vgg16_weights_tf_dim_ordering_tf_kernels_notop.h5'),\n",
    "                 include_top=False, pooling='avg')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "EItOlRBGpV_A"
   },
   "source": [
    "Print the summary of the base_model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 851
    },
    "colab_type": "code",
    "id": "lQsEBgnlpHjH",
    "outputId": "4d3945dc-30e6-4823-f5d5-0c70e1f74027"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"vgg16\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_1 (InputLayer)         (None, None, None, 3)     0         \n",
      "_________________________________________________________________\n",
      "block1_conv1 (Conv2D)        (None, None, None, 64)    1792      \n",
      "_________________________________________________________________\n",
      "block1_conv2 (Conv2D)        (None, None, None, 64)    36928     \n",
      "_________________________________________________________________\n",
      "block1_pool (MaxPooling2D)   (None, None, None, 64)    0         \n",
      "_________________________________________________________________\n",
      "block2_conv1 (Conv2D)        (None, None, None, 128)   73856     \n",
      "_________________________________________________________________\n",
      "block2_conv2 (Conv2D)        (None, None, None, 128)   147584    \n",
      "_________________________________________________________________\n",
      "block2_pool (MaxPooling2D)   (None, None, None, 128)   0         \n",
      "_________________________________________________________________\n",
      "block3_conv1 (Conv2D)        (None, None, None, 256)   295168    \n",
      "_________________________________________________________________\n",
      "block3_conv2 (Conv2D)        (None, None, None, 256)   590080    \n",
      "_________________________________________________________________\n",
      "block3_conv3 (Conv2D)        (None, None, None, 256)   590080    \n",
      "_________________________________________________________________\n",
      "block3_pool (MaxPooling2D)   (None, None, None, 256)   0         \n",
      "_________________________________________________________________\n",
      "block4_conv1 (Conv2D)        (None, None, None, 512)   1180160   \n",
      "_________________________________________________________________\n",
      "block4_conv2 (Conv2D)        (None, None, None, 512)   2359808   \n",
      "_________________________________________________________________\n",
      "block4_conv3 (Conv2D)        (None, None, None, 512)   2359808   \n",
      "_________________________________________________________________\n",
      "block4_pool (MaxPooling2D)   (None, None, None, 512)   0         \n",
      "_________________________________________________________________\n",
      "block5_conv1 (Conv2D)        (None, None, None, 512)   2359808   \n",
      "_________________________________________________________________\n",
      "block5_conv2 (Conv2D)        (None, None, None, 512)   2359808   \n",
      "_________________________________________________________________\n",
      "block5_conv3 (Conv2D)        (None, None, None, 512)   2359808   \n",
      "_________________________________________________________________\n",
      "block5_pool (MaxPooling2D)   (None, None, None, 512)   0         \n",
      "_________________________________________________________________\n",
      "global_average_pooling2d_1 ( (None, 512)               0         \n",
      "=================================================================\n",
      "Total params: 14,714,688\n",
      "Trainable params: 14,714,688\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "base_model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "fHpeOyW0qauW"
   },
   "source": [
    "### Add the following classification layers to the imported VGG Model <br>\n",
    "1. Flatten Layer\n",
    "2. Dense layer with 1024 neurons with activation as Relu\n",
    "3. Dense layer with 256 neurons with activation as Relu\n",
    "4. Dense layer with 120 neurons with activation as Softmax"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "0BpT4MLkqoaO"
   },
   "outputs": [],
   "source": [
    "from keras.layers import Flatten, Dense\n",
    "x = base_model.output\n",
    "x = Flatten()(x)\n",
    "x = Dense(1024, activation=\"relu\")(x)\n",
    "x = Dense(256, activation=\"relu\")(x)\n",
    "predictions = Dense(120, activation=\"softmax\")(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "LeQem0pHITIj"
   },
   "source": [
    "### Make all the layers in the base_model (VGG16) to be non-trainable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "C7w9CSPvIRnX"
   },
   "outputs": [],
   "source": [
    "for layer in base_model.layers[:]:\n",
    "    layer.trainable = False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "kj-BwqgfIkdv"
   },
   "source": [
    "### Fit and compile the model with batch_size = 128 and epochs = 10 and execute the model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "YD5fAgVQIpKZ"
   },
   "source": [
    "Try to get training and validation accuracy to be more than 90%"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 558
    },
    "colab_type": "code",
    "id": "SZk2SWvjIoRP",
    "outputId": "95486a70-1a9d-4bc1-daec-25582732e555"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:9: UserWarning: Update your `Model` call to the Keras 2 API: `Model(inputs=Tensor(\"in..., outputs=Tensor(\"de...)`\n",
      "  if __name__ == '__main__':\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/keras/backend/tensorflow_backend.py:422: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.\n",
      "\n",
      "Train on 10222 samples, validate on 2045 samples\n",
      "Epoch 1/10\n",
      "10222/10222 [==============================] - 27s 3ms/step - loss: 19.3410 - accuracy: 0.0094 - val_loss: 4.8378 - val_accuracy: 0.0054\n",
      "Epoch 2/10\n",
      "  128/10222 [..............................] - ETA: 13s - loss: 17.8651 - accuracy: 0.0156"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/tensorflow-1.15.2/python3.6/keras/callbacks/callbacks.py:707: RuntimeWarning: Can save best model only with val_acc available, skipping.\n",
      "  'skipping.' % (self.monitor), RuntimeWarning)\n",
      "/tensorflow-1.15.2/python3.6/keras/callbacks/callbacks.py:846: RuntimeWarning: Early stopping conditioned on metric `val_acc` which is not available. Available metrics are: val_loss,val_accuracy,loss,accuracy\n",
      "  (self.monitor, ','.join(list(logs.keys()))), RuntimeWarning\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10222/10222 [==============================] - 17s 2ms/step - loss: 17.7589 - accuracy: 0.0087 - val_loss: 4.8334 - val_accuracy: 0.0064\n",
      "Epoch 3/10\n",
      "10222/10222 [==============================] - 17s 2ms/step - loss: 16.6960 - accuracy: 0.0093 - val_loss: 4.8298 - val_accuracy: 0.0068\n",
      "Epoch 4/10\n",
      "10222/10222 [==============================] - 18s 2ms/step - loss: 15.8795 - accuracy: 0.0089 - val_loss: 4.8267 - val_accuracy: 0.0059\n",
      "Epoch 5/10\n",
      "10222/10222 [==============================] - 18s 2ms/step - loss: 15.2051 - accuracy: 0.0083 - val_loss: 4.8240 - val_accuracy: 0.0064\n",
      "Epoch 6/10\n",
      "10222/10222 [==============================] - 18s 2ms/step - loss: 14.6195 - accuracy: 0.0089 - val_loss: 4.8216 - val_accuracy: 0.0059\n",
      "Epoch 7/10\n",
      "10222/10222 [==============================] - 18s 2ms/step - loss: 14.0957 - accuracy: 0.0089 - val_loss: 4.8195 - val_accuracy: 0.0073\n",
      "Epoch 8/10\n",
      "10222/10222 [==============================] - 18s 2ms/step - loss: 13.6197 - accuracy: 0.0089 - val_loss: 4.8177 - val_accuracy: 0.0059\n",
      "Epoch 9/10\n",
      "10222/10222 [==============================] - 18s 2ms/step - loss: 13.1823 - accuracy: 0.0091 - val_loss: 4.8160 - val_accuracy: 0.0068\n",
      "Epoch 10/10\n",
      "10222/10222 [==============================] - 18s 2ms/step - loss: 12.7769 - accuracy: 0.0095 - val_loss: 4.8145 - val_accuracy: 0.0068\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.callbacks.History at 0x7f67a055ff28>"
      ]
     },
     "execution_count": 43,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from keras import applications\n",
    "from keras.models import Sequential, Model \n",
    "from keras.layers import Dropout, Flatten, Dense\n",
    "from keras import backend as k \n",
    "from keras.callbacks import ModelCheckpoint, EarlyStopping\n",
    "from keras import optimizers\n",
    "model_final = Model(input = base_model.input, output = predictions)\n",
    "sgd = optimizers.SGD(lr=0.000001, decay=1e-6, momentum=0.9, nesterov=True)\n",
    "model_final.compile(loss = \"categorical_crossentropy\", optimizer = sgd, metrics=[\"accuracy\"])\n",
    "checkpoint = ModelCheckpoint(\"vgg16_best.h5\", monitor='val_acc', verbose=1, save_best_only=True, mode='auto')\n",
    "early = EarlyStopping(monitor='val_acc', min_delta=0, patience=5, verbose=1, mode='auto')\n",
    "epochs=10\n",
    "model_final.fit(x_train, y_train, batch_size = 128, epochs = epochs, validation_data=(x_val, y_val), callbacks = [checkpoint, early])"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "CV_Project2_Dog_Breed_Classification_Questions.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}