mnist-with-pytorch-catalyst-amp-nvidia-apex.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Jme-NJAJYvnW",
    "outputId": "1ed720e9-d7cd-4a68-a5c7-9f52281e6442"
   },
   "source": [
    "![](https://raw.githubusercontent.com/catalyst-team/catalyst-pics/master/pics/catalyst_logo.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Introduction\n",
    "\n",
    "In this Kernel I will develop my take on the classic competition of Digit Recognizer using the nice Python module called [Catalyst](https://github.com/catalyst-team/catalyst). It's good because you can write modular and reproducible code to future experiments.<br>\n",
    "Let's start installing catalyst module.<br>\n",
    "Now import all required modules."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "_kg_hide-input": true,
    "id": "FYQC9cqwY9Xy",
    "outputId": "7809cf7f-9fcf-465c-8b90-e4836faa629e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['train.csv', 'test.csv', 'sample_submission.csv']\n"
     ]
    }
   ],
   "source": [
    "# Standard Modules\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "# Graphic Plots\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "plt.style.use('ggplot')\n",
    "import seaborn as sns\n",
    "sns.set_style('darkgrid')\n",
    "# Extra Tools\n",
    "import os\n",
    "import warnings\n",
    "import math\n",
    "import random\n",
    "from sklearn.model_selection import train_test_split\n",
    "warnings.filterwarnings('ignore')\n",
    "# PyTorch and Catalyst\n",
    "import collections\n",
    "import torch\n",
    "import torchvision\n",
    "import torchvision.transforms as transforms\n",
    "from torch.utils.data import DataLoader, Dataset, TensorDataset\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "from catalyst.dl import utils\n",
    "from catalyst.dl.runner import SupervisedRunner\n",
    "from catalyst.dl.callbacks import AccuracyCallback\n",
    "from catalyst.utils import set_global_seed, prepare_cudnn\n",
    "# For better plot visualization\n",
    "%config InlineBackend.figure_format = 'retina'\n",
    "# Enviroment PATH\n",
    "PATH = '../input/digit-recognizer/'\n",
    "print(os.listdir(PATH))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And set the enviroment settings for reprodutibility."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "SEED = 42\n",
    "os.environ['CUDA_VISIBLE_DEVICES'] = \"0\"\n",
    "set_global_seed(SEED)                       \n",
    "prepare_cudnn(deterministic=True)           "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's use [NVIDIA Apex](https://github.com/NVIDIA/apex) to work with [Mixed Precision Training](https://docs.nvidia.com/deeplearning/sdk/mixed-precision-training/index.html), also called Automatic Mixed Precision (AMP).\n",
    "Set it to use FP16 and install Apex."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "_kg_hide-input": true
   },
   "outputs": [],
   "source": [
    "FP16_PARAMS = None"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "_kg_hide-input": true,
    "_kg_hide-output": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing setup.sh\n"
     ]
    }
   ],
   "source": [
    "%%writefile setup.sh\n",
    "git clone https://github.com/NVIDIA/apex\n",
    "cd apex\n",
    "pip install -v --no-cache-dir --global-option=\"--cpp_ext\" --global-option=\"--cuda_ext\" ./\n",
    "rm -rf ./apex"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "!sh setup.sh\n",
    "FP16_PARAMS = dict(opt_level=\"O1\") "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Load Datasets and Explore the Images\n",
    "\n",
    "Load the datasets in Pandas Dataframes and take a look on the Train Set to check the dataset format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "_kg_hide-input": true,
    "id": "QBegVYWtwTRn"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>pixel0</th>\n",
       "      <th>pixel1</th>\n",
       "      <th>pixel2</th>\n",
       "      <th>pixel3</th>\n",
       "      <th>pixel4</th>\n",
       "      <th>pixel5</th>\n",
       "      <th>pixel6</th>\n",
       "      <th>pixel7</th>\n",
       "      <th>pixel8</th>\n",
       "      <th>...</th>\n",
       "      <th>pixel774</th>\n",
       "      <th>pixel775</th>\n",
       "      <th>pixel776</th>\n",
       "      <th>pixel777</th>\n",
       "      <th>pixel778</th>\n",
       "      <th>pixel779</th>\n",
       "      <th>pixel780</th>\n",
       "      <th>pixel781</th>\n",
       "      <th>pixel782</th>\n",
       "      <th>pixel783</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 785 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   label  pixel0  pixel1  pixel2  pixel3  pixel4  pixel5  pixel6  pixel7  \\\n",
       "0      1       0       0       0       0       0       0       0       0   \n",
       "1      0       0       0       0       0       0       0       0       0   \n",
       "2      1       0       0       0       0       0       0       0       0   \n",
       "3      4       0       0       0       0       0       0       0       0   \n",
       "4      0       0       0       0       0       0       0       0       0   \n",
       "\n",
       "   pixel8  ...  pixel774  pixel775  pixel776  pixel777  pixel778  pixel779  \\\n",
       "0       0  ...         0         0         0         0         0         0   \n",
       "1       0  ...         0         0         0         0         0         0   \n",
       "2       0  ...         0         0         0         0         0         0   \n",
       "3       0  ...         0         0         0         0         0         0   \n",
       "4       0  ...         0         0         0         0         0         0   \n",
       "\n",
       "   pixel780  pixel781  pixel782  pixel783  \n",
       "0         0         0         0         0  \n",
       "1         0         0         0         0  \n",
       "2         0         0         0         0  \n",
       "3         0         0         0         0  \n",
       "4         0         0         0         0  \n",
       "\n",
       "[5 rows x 785 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_raw = pd.read_csv(PATH+'train.csv')\n",
    "test_raw = pd.read_csv(PATH+'test.csv')\n",
    "submission = pd.read_csv(PATH+'sample_submission.csv')\n",
    "train_raw.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Although this is an image competition, is provided tabular data, where each feature column represents an image pixel. <br>\n",
    "Let's see how many samples (images) it have."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "_kg_hide-input": true,
    "id": "iRtK3DzM2eBz",
    "outputId": "caed51e5-676c-4143-d4a1-67990bb3017f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train Samples: 42000\n",
      "Test Samples: 28000\n"
     ]
    }
   ],
   "source": [
    "print(f'Train Samples: {train_raw.shape[0]}')\n",
    "print(f'Test Samples: {test_raw.shape[0]}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check the label distribution to find any disbalance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "_kg_hide-input": true,
    "id": "FlN2yv66273H",
    "outputId": "758822f1-3325-4dce-af3e-0e23b6ff2d39"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABNoAAAMMCAYAAACFS8r2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAWJQAAFiUBSVIk8AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdeZyXZb0//tewDNsgA+aSAurxKAlhIop8KyQRozRJQDQVyCWXSk+anQ6klufESTla2gMrzMIFFFtcj5K4p5mEghpKqEcFxQUwGXYZlvn94YP5ObIz9zAMPp9/zdz3dd33+3Pf94ecV9d13SVVVVVVAQAAAABqpVF9FwAAAAAAOwJBGwAAAAAUQNAGAAAAAAUQtAEAAABAAQRtAAAAAFAAQRsAAAAAFEDQBgAAAAAFELQBAAAAQAEEbQAAAABQAEEbAAAAABRA0AYAAAAABRC0AQAAAEABBG0AAAAAUABBGwCwxTp16pROnTpl+PDhdXqe0aNHV59rzpw5dXqu7cmcOXOqP/fo0aPru5wk2+6eb42G+pxs6j5vj8/BltqenxsAqAtN6rsAAKBuzJkzJ0ceeeR697Vs2TJlZWUpLy/Pfvvtl86dO+eII47Ivvvuu42r3P4MHTo0U6ZMSZI89NBDad++fT1XtOMYPnx47rjjjnW2N2nSJGVlZSkrK8see+yRzp0756CDDkqfPn3SrFmzeqgUAGDrGNEGAB9Dy5Yty7x58/LSSy/l3nvvzRVXXJGjjz46J510Up566qn6Lm+zNdSRTNS0atWqVFRUZM6cOZkyZUpuuOGGnH/++enVq1euuOKKLF++vN5q84zV5HoAwMYZ0QYAHwOf/vSnc9lll1X/vmrVqixcuDBz587Nc889l/vvvz/vvvtupk2blmHDhuWss87KBRdcsMHjvfjii9ui7Jx33nk577zztsm52LZ+8pOfpGvXrkmSqqqqLF26NIsWLcrMmTPzt7/9LU8++WQWLlyY3/zmN3nwwQfzy1/+coMjLhvqc9K+fftt9l2qLzv65wOAjxK0AcDHQMuWLbP//vuvd99xxx2XESNG5JZbbsmVV16ZlStXZsyYMSkrK8uZZ565jSvl46J9+/brfSa/8IUv5Jxzzskrr7ySESNG5LnnnsusWbNy5pln5ve//30+8YlP1EO1AACbx9RRACClpaU59dRT88tf/jKNGn3wnwdXXXVVXn311XqujI+rfffdNzfffHN69OiRJHnzzTdz+eWX13NVAAAbZ0QbAFDt8MMPzymnnJJx48Zl9erVGTNmTP7nf/5nnXadOnVKkgwYMGCj4cdTTz2VcePGZdq0aVm4cGF23nnndOvWLcOGDUu3bt0yevToXHPNNUnW/+KBDe2//fbbM2LEiBpt1/fih3PPPXebTylcvXp1Jk+enL/85S959tlnM2vWrCxevDhNmzbNrrvumoMPPjgnnHBCunXrtkXHXXstn3vuubz33nspLy/PoYcemlNPPTUHHnjgJvuvWbMm9913X+67775Mnz49//znP9OkSZPsscce6dmzZ4YOHZq99tpraz92nWjatGmuvPLK9O3bN5WVlZk4cWLOPffc7L333jXabeo5SpK33347N998c/7617/m9ddfz/Lly1NWVpZ27dqlQ4cO6dmzZ4488sjqa7A1z9iHX0CydvvUqVNz6623ZurUqZk/f34qKyvz1FNPZaeddlpv+03Z2udgc7+zH/7cN910Uw477LCtvh5bct633nor48aNy1//+te8+eabWbFiRdq2bZsDDzwwX/nKV9KvX7+UlJRsds0PP/xwbr311syYMSMVFRVp165devTokbPOOmuDo3sBoAiCNgCghlNPPTXjx49PVVVVHnzwwaxevTqNGzfe4uNcffXVGTNmTKqqqqq3vf3223n77bdz33335cILLyyy7O3G5Zdfnptuummd7StXrsysWbMya9as3H777Rk6dGguuuiiDYYHHzZmzJhcffXVNa7lvHnzcu+99+ZPf/pTvvvd7250mu+bb76Z8847Ly+88EKN7StWrMjLL7+cl19+ORMmTMiIESMyZMiQLfi0dW+33XbLV77yldx+++1ZvXp1HnzwwXzjG9/YomM88sgjueCCC9Z5qUJFRUUqKiry6quv5s9//nNeeeWV/Pd//3dhtf/iF7/I6NGja9y32qjtc7C9+sMf/pD/+q//SmVlZY3tc+fOzQMPPJAHHnggPXr0yOjRo1NeXr7RY61ZsyYXX3xx/vCHP6xzrP/93//NpEmTcs0116R3796Ffw4ASARtAMBHtG/fPvvss09effXVLF26NDNmzKhetH5zjR8/Pr/61a+SJK1bt87pp5+eHj16pLS0NDNmzMhvfvObXHnllZs1Emt9+vbtm09/+tO55ZZbMmHChCTJb3/72+y666412u28885bdfzaWL16dXbbbbf06dMnBx10UDp06JAWLVpk/vz5mTlzZsaNG5f58+dn3Lhx2X333TcZGv35z3/O9OnTs+eee+bMM89Mly5dUllZmSeeeCI33HBDli1bliuvvDK77LJLjjvuuHX6z5s3LyeeeGLmz5+fxo0b55hjjsnhhx+e9u3bp6SkJM8//3xuuummzJ49Oz/+8Y/TqlWrDBgwoK4uz1bp1atXbr/99iQfjOjakqDtvffey4UXXpjly5enRYsWGTx4cD73uc9l5513TlVVVebNm5cXXnghjz76aI1+tX3GHnzwwcycOTP77LNPTj311HzqU5/KmjVr8uyzz6Zp06Zb8Ok/UNvnoLbq6jt3zz335OKLL06SNGvWLEOGDMnhhx+eVq1a5ZVXXsmNN96YGTNmZMqUKfnGN76RCRMmbPT6jR49OlOnTs3nP//5DBo0KB07dsySJUty3333ZcKECamsrMz3v//93H///WnTps0WXgUA2DRBGwCwji5dulSvz/baa69tUdD23nvv5ac//WmSpLy8PBMmTMi//Mu/VO9fOxVsyJAhee6557aqvp122ik77bRTjT/q99577/VOGdzWTjvttFx00UXrHQXYu3fvfP3rX8+ZZ56ZKVOmZMyYMTnppJPSqlWrDR5v+vTp2X///XPzzTdnp512qt5+yCGHpG/fvhkyZEiWLVuWn/zkJ+nbt2/Kyspq9B8xYkTmz5+fnXfeOddff331VL61DjrooAwcODBnnHFGpk2blssuuyxHHXXUOsepT126dKn++bXXXtuivo888kiWLl2aJNXTUD+qb9+++c53vpMFCxZUb6vtMzZz5swccsgh+e1vf5vmzZtXbz/44IO3qP61avsc1FZdfOcWL16cSy+9NEnSokWL3HDDDTnooIOq93ft2jVf+cpXcu655+aRRx7J9OnTM3bs2Jx99tkbPObUqVNzzjnnrPPW5J49e6Zt27b55S9/mYqKivzv//7vdjd6E4Adg5chAADraNu2bfXPFRUVW9T3zjvvzLJly5Ik5513Xo2Qba2ysrL8+Mc/rl2R26kOHTpsdKpt8+bN84Mf/CDJB0HDk08+ucljjhw5ska4slaXLl1y1llnJUkWLlyYu+66q8b+v//97/nLX/6S5IPA7aMh21otW7bMf/7nf1Yf57777ttkTdtSbZ7H+fPnV//cs2fPzT5PbTVq1CiXXXZZjZCttrb2Odhe3XbbbVm8eHGS5Oyzz64Rsq3VpEmT/OQnP6kODteuH7khBxxwQM4///z17jv99NOrR8NNmTKltuUDwHoJ2gCAdbRs2bL657WjgTbXE088kSRp3Lhx+vfvv8F2Xbt2/VgsSr506dLMmTMnL7/8cl566aW89NJLNfbPmDFjo/3322+/fOYzn9ng/sGDB1f//Pjjj9fYd//99yf54KUC/fr12+h59t9//+r1r6ZNm7bRtttabZ7H3Xbbrfrnj67bVZcOOuigdOzYsbDj1eY52F6tDYFLSkpy4oknbrBdu3bt8uUvfznJB8Hpiy++uMG2/fv33+C6h61bt65+kcbrr7++lVUDwMaZOgoArOPDYcaWTkFbGyR17NhxvaNvPqxr167rBE87gtmzZ+fGG2/Mo48+mjfffHOjbT88XXF9NrWO3Sc+8YnsueeeefPNN9e5ln//+9+TfPAihi2Z/vvhUWDbg9o8j0ceeWTKy8tTUVGRyy+/PHfffXeOPPLIdO/ePZ/5zGdqhHhFOuCAAwo9Xm2eg+3V2sCsY8eOadeu3UbbduvWrToonTlzZjp37rzedvvuu+9Gj7N2XbYlS5ZsabkAsFkEbQDAOj4c/mzpguFrp/ZtzqLo9fGygrp2zz33ZPjw4Vm5cuVmtX///fc3un9zrtEuu+ySN998c53Q7r333tusGra0pm3tw59jS5/HnXbaKdddd12+973vZfbs2ZkxY0b1KMImTZrk05/+dL74xS9m8ODBmwyGt/S8RarNc7C9WvtvxSc+8YlNtt1ll13W6bc+m5qq26jRBxN61qxZszklAsAWE7QBAOt4/vnnq3/e1AiR2qiqqqqzY9eH2bNnZ8SIEVm5cmVatGiRr3/96+nVq1f22muvtGnTJqWlpUk++CN/7YinTV2DDU2D+7ANHWPVqlVJPggpxo4du9mfo0WLFpvddlt44YUXqn9e35p/m3LggQdm4sSJefTRR/Poo49m2rRpefXVV7Nq1ao8++yzefbZZ/PrX/86V111VT772c8WUvPG1unbGrV5DrZ3m/PZatMeALYlQRsAUMMbb7yR2bNnJ/lgTaMtnQLXpk2bzJ8/P//85z832XZz2jQkt912WyorK5Mko0ePTq9evdbbbksW9H/33Xc32WbtdfzoYv7t2rXLa6+9lgULFqRjx46FLsy/LX14zbEePXps1TGaNGmSvn37Vr91dMGCBfnb3/6WO+64I48++mgqKipy7rnn5sEHH9zkNMb6UJvnIPkgnKqqqtrkSK7ly5dvXYFboby8PPPmzdusqcof/vxbOqoRALYlL0MAAGq4/vrrq0fGHHXUUdVTrTbX2hccvP7661m0aNFG23545NzW2N5GtqxdG6u8vHyDIVuSTJ8+fbOPuXadtQ355z//Wb0O3EdfLtGlS5ckH4xs++tf/7rZ59yezJ07NxMnTkzywSixo446qpDjtm3bNl/60pdy7bXX5uSTT07ywVpwf/7zn2u0216esdo8B0nSqlWrJNnkd/LVV1/d6P4ir8fat+C+/vrrm5zm/Mwzz1T//KlPfaqwGgCgaII2AKDaY489lgkTJiT5INQ455xztvgYa6ferV69OnffffcG202fPr3Wi7avnYqZpHokWX1aO1VzxYoVWb169Qbb3XLLLZt9zJdffnmjIcsf//jH6mD085//fI19X/ziF6t//vWvf73RmrZHK1euzPe+973qe3vsscemQ4cOhZ/nw9fto4HP9vKM1eY5SFL9BtTnn39+g1NMV6xYkUmTJm20jiKvx9o6q6qqNvpG2IqKivzpT39K8sE06LUBHQBsjwRtAEAqKytzww035Fvf+lb11LJ///d/z1577bXFxxowYED1Gl+jR49e7wiZJUuW5JJLLqld0Ul222236p/XTnetT3vvvXeSD6bf3XPPPettM3bs2Dz66KNbdNyLL744ixcvXmf7P/7xj4wZMybJB4vvf/WrX62x/9BDD03Pnj2TfDAi6NJLL60OA9dn5cqVue222zZrmmJde+WVVzJkyJBMmTIlSdKhQ4f8x3/8xxYf57HHHsvbb7+90TYfnpr60SBve3rGtvY5SJLDDjssyQdvlL3rrrvW2b9mzZpceumlm5zGWeT1GDhwYFq3bp0kGTNmzHpHeq5atSo/+MEPqj/30KFDC1//DgCKZI02APgYWLZsWY3RY6tWrcqiRYvyzjvv5LnnnssDDzxQ/Qd2o0aN8s1vfjOnnXbaVp1r5513zgUXXJCf/OQnqaioyIknnpjTTz89PXr0SNOmTTNjxoz89re/zeuvv54DDzywepTO1kxJO/jgg6vXnrrqqqvSqFGjdOjQofoP8TZt2qS8vHyrPkeSTJo0ab3rXX1Uz549s8cee2TAgAEZP358qqqqcvHFF+cf//hHevXqlTZt2uSNN97InXfemUcffTTdu3fP1KlTN6uGrl27Zvr06RkwYEC+8Y1vpHPnzqmsrMyTTz6ZsWPHZtmyZUmSESNGVIcWH3bFFVfkhBNOyNtvv53f//73mTJlSk444YR07do1rVu3ztKlS/P666/nmWeeyQMPPJAFCxbk/vvv36w3QdbGnDlzalzbpUuXZuHChXnxxRczefLkPPnkk9Ujr/bZZ5/86le/2qq10yZOnJi77747hx56aD7/+c+nU6dOadeuXVatWpW33norEydOzAMPPJDkg5Ctd+/eNfrX9TO2uWr7HJx44okZP358Vq5cmYsvvjivvfZaevfunaZNm+aVV17JhAkT8uyzz+bggw/OtGnTNlhHkddjp512yqWXXpoLL7wwy5Yty5AhQzJ06NAcfvjhadmyZf7v//4vN910U/XLMD796U/n9NNP39JLBwDblKANAD4Gnn/++Rx77LGbbNe9e/dceOGF6d69e63O9/Wvfz3vvvtufv3rX2fRokW5+uqra+wvKSnJBRdckBUrVlQHbc2aNdvi83To0CEDBgzI7bffnhdffDFnnXVWjf3nnntuzjvvvK3+HP/zP/+zWe1+8YtfZI899kiXLl3yve99L1deeWUqKytz/fXX5/rrr6/RtkuXLhk9evRmv92yd+/eOfLII/Pzn/88P/rRj9bZ36hRo5x//vkZOHDgevvvuuuu+d3vfpfvfe97mTJlSmbNmrXRz1VaWlpjemBd+cEPfrDJNuXl5TnhhBPy7W9/u1Yvcli9enUmT56cyZMnb7DNXnvtlV/96lfrPId1/Yxtrto+B/vss09++MMf5oc//GFWrlyZMWPGVI+CSz74Tn7rW99Khw4dNhq0FX09vvKVr2T58uX5r//6r7z//vu57rrrct11163TrkePHhk9enSaNm262ccGgPogaAOAj6EWLVqkrKws5eXl2X///dOlS5d84QtfyL777lvYOS688ML06tUr48aNyzPPPJOKioq0a9cuBx10UE455ZQcdthhGTlyZHX79Y3C2RwjR45M165dM3HixLz88stZsmTJRqdH1rVvfOMb6dq1a2688cY8++yzWbhwYdq0aZN/+Zd/yZe//OWccMIJWxwWfPOb30y3bt1y880359lnn82CBQtSXl6eQw45JKeeemoOOuigjfbfbbfdMm7cuPz1r3/NPffck2eeeSbz5s3L8uXL07Jly3zyk59Mp06d8tnPfjZ9+/bNTjvtVJtLsMWaNGmSVq1apaysLHvuuWc6d+6cbt26pU+fPrUO/S666KL07ds3kydPzvTp06vfiLt69eq0a9cunTp1St++fTNgwIANnmt7ecZq+xyccMIJ2XfffXP99ddn2rRpWbRoUcrLy9OtW7cMGTIkhx12WG6//fZN1lH09Rg8eHA+97nPZdy4cXniiSfy5ptvZsWKFWnXrl26du2aY489Nv369dtuXkwBABtTUrWh1VABAOrYsGHD8re//S177LFHHnnkkfouBwAAasXLEACAevHmm29Wr1PWrVu3eq4GAABqT9AGABRu5cqVmTNnzgb3L1myJN///verp5sNGjRoW5UGAAB1xhptAEDhli9fnn79+qVXr17p3bt39ttvv7Rq1SoLFy7Ms88+m1tvvTVvv/12kqRv37753Oc+V88VAwBA7VmjDQAo3KJFi3LooYdusl3v3r3zs5/9LGVlZdugKgAAqFuCNgCgcGvWrMnDDz+cv/zlL3nuuefy7rvvpqKiIo0bN87OO++cgw46KP3790/v3r3ru1QAACiMoA0AAAAACuBlCAAAAABQAEEbAAAAABRA0AYAAAAABRC0AQAAAEABBG0AAAAAUABBGwAAAAAUoEl9F0DtzZ+/uL5LAAAAANgh7LJL663ua0QbAAAAABRA0AYAAAAABRC0AQAAAEABBG0AAAAAUABBGwAAAAAUQNAGAAAAAAUQtAEAAABAAQRtAAAAAFAAQRsAAAAAFEDQBgAAAAAFELQBAAAAQAEEbQAAAABQAEEbAAAAABRA0AYAAAAABRC0AQAAAEABBG0AAAAAUABBGwAAAAAUQNAGAAAAAAUQtAEAAABAAQRtAAAAAFAAQRsAAAAAFEDQBgAAAAAFELQBAAAAQAEEbQAAAABQAEEbAAAAABRA0AYAAAAABRC0AQAAAEABBG0AAAAAUABBGwAAAAAUoEl9FwBsnrZtStOktFl9l7FDW1W5IgsWVtZ3GQAAADRQgjZoIJqUNsvMX3y1vsvYoX3q23clEbQBAACwdUwdBQAAAIACCNoAAAAAoACCNgAAAAAogKANAAAAAAogaAMAAACAAgjaAAAAAKAAgjYAAAAAKICgDQAAAAAKIGgDAAAAgAII2gAAAACgAII2AAAAACiAoA0AAAAACiBoAwAAAIACCNoAAAAAoACCNgAAAAAogKANAAAAAAogaAMAAACAAgjaAAAAAKAAgjYAAAAAKICgDQAAAAAKIGgDAAAAgAII2gAAAACgAII2AAAAACiAoA0AAAAACiBoAwAAAIACCNoAAAAAoACCNgAAAAAogKANAAAAAAogaAMAAACAAgjaAAAAAKAAgjYAAAAAKICgDQAAAAAKIGgDAAAAgAII2gAAAACgAII2AAAAACiAoA0AAAAACtCkvgsAAABg+9CmvFVKmxqPUZcqV67Jwoql9V0GUEcEbQAAACRJSps2yi/umFvfZezQvj1gt/ouAahD/q8KAAAAACiAoA0AAAAACiBoAwAAAIACCNoAAAAAoACCNgAAAAAogLeOAsBGtClvmtKmzeu7jB1a5cr3s7BiZX2XAQAAtSZoA4CNKG3aPJf+vl99l7FDu/SESUkEbQAANHymjgIAAABAAQRtAAAAAFAAQRsAAAAAFEDQBgAAAAAFELQBAAAAQAEEbQAAAABQAEEbAAAAABRA0AYAAAAABRC0AQAAAEABBG0AAAAAUABBGwAAAAAUQNAGAAAAAAUQtAEAAABAAQRtAAAAAFAAQRsAAAAAFEDQBgAAAAAFELQBAAAAQAEEbQAAAABQAEEbAAAAABSgSX0XAAAAAPBx1q5NyzQubVzfZezwVleuznsLl9XpOQRtAAAAAPWocWnjzL16an2XscPb7fzudX4OQRtAHStvU5qmpc3qu4wd2srKFalYWFnfZQAAAB9zgjaAOta0tFn+eP2X6ruMHdrxp92XRNAGAADULy9DAAAAAIACCNoAAAAAoACCNgAAAAAogDXaAACAQrUub5HmTf2pUZfeX7kqiyuW13cZAHyE//X7GGnXpnkalzat7zJ2aKsrV+a9he/XdxkAAPWqedMmGXDbX+q7jB3aHYM+n8X1XQQA6xC0fYw0Lm2a+b8aX99l7NB2+eaQJII2AAAA+DiyRhsAAAAAFEDQBgAAAAAFELQBAAAAQAEEbQAAAABQAEEbAAAAABRA0AYAAAAABWhS3wUAANSF1uXN0rxpaX2XscN7f2VlFlesqO8yAAC2C4I2AGCH1LxpaY6+Y2R9l7HDmzjg4iyOoA1ge9C2Tas0KTVxrS6tqlyTBQuX1ncZbMcEbQAAALADaFLaKM/8Zl59l7FD6/aNXeu7BLZzO2TUfeedd6ZTp07p1KlT/vCHP6y3zSOPPJKhQ4eme/fu6datWwYPHpw77rhjo8e94447cvzxx6dbt27p3r17hg4dmkceeWSD7VevXp0bbrghxx57bA488MD06NEjZ555ZqZNm1arzwcAAADA9meHC9refvvtjBw5Mi1bttxgm/Hjx+ecc87JSy+9lP79+2fw4MGZN29ehg8fnlGjRq23z6hRozJ8+PDMnz8/gwcPTv/+/fPSSy/lnHPOyfjx49dpX1VVlQsuuCCXXXZZVq5cmVNOOSV9+/bN008/nSFDhuTBBx8s7DMDAAAAUP92qKmjVVVVGTFiRMrLy3PUUUdl7Nix67SZM2dORo0alfLy8tx2221p3759kuTb3/52jj/++IwdOzZf/OIX061bt+o+06ZNy9ixY9OxY8f88Y9/TJs2bZIkZ5xxRgYNGpRRo0blC1/4QvWxkuTee+/NpEmT0q1bt9x4441p1qxZkuRrX/taTj755FxyySXp2bNnysrK6vKSAAAAALCN7FAj2m666aZMnjw5l1122QZHtN12222prKzMKaecUiMYa9OmTc4+++wkya233lqjz9rfzznnnOqQLUnat2+fk08+OZWVlbn99ttr9JkwYUKS5Pzzz68O2ZLkwAMPzNFHH5333nsvkyZNqsWnBQAAAGB7ssMEba+88kp++tOfZtiwYTn00EM32G7y5MlJkl69eq2z7/DDD6/RZmv7VFZW5plnnkmLFi1yyCGHbPZ5AAAAAGi4doipo6tWrcq///u/55Of/GS++93vbrTta6+9liTZe++919m36667pmXLlnnnnXeyfPnytGjRIsuWLcvcuXPTsmXL7Lrrum8X2WuvvZIks2bNqt42e/bsrF69Oh06dEiTJute4vX1qY3y8g2vR8e25340bO5fw+XeNWzuX8Pm/kH98N1ruNy7hs39a9jq+v7tEEHbL37xi/zjH//ILbfckubNm2+07ZIlS5IkrVu3Xu/+srKyLFu2LIsXL06LFi2yePHijbZfu33RokXV29b22dD6a+vrAwAAAEDD1uCDtr///e+59tprc9ppp9V4gcHWqqqqSpKUlJRsUb8tab+159iQioplm9Vul13WHxZSrM29H1vK/ds26uL+uXfbhu9ew+a717DV1fePhsv3b9vwb2fD5b9bGjbfvYZtc+5fbe5Hg16jbe2U0b333jvnn3/+ZvVZO8ps7aizj1q6dGmNdmtHn22o/fpGvK39ee3ouY/a1Kg6AAAAABqeBj2ibdmyZdXrnHXt2nW9bS6++OJcfPHFGTZsWC666KLss88+WbBgQWbNmpW2bdvWaDtv3rwsW7Ysu+++e1q0aJEkadmyZXbbbbfMnTs38+bNW2edttmzZyepuebbXnvtlcaNG+eNN97IqlWr1lmnbX19AAAAAGjYGonmlIEAACAASURBVHTQVlpamuOPP369+2bMmJEZM2ake/fu2Weffaqnlfbs2TPTpk3L448/vs5U08cee6y6zYf17Nkzd911Vx5//PEMGjRok31KS0vTrVu3PP3003n66afXOd6GzgMAAABAw9Wgg7bmzZvnv//7v9e7b/To0ZkxY0YGDBiQwYMHV28fOHBgfvOb3+Tmm2/OwIED0759+yTJwoULc+211yZJvva1r9U41te+9rXcddddGTNmTPr27Zs2bdokSebMmZNbbrklpaWlGThwYI0+J510Up5++ulcffXVufHGG9OsWbMkH6wpN3HixLRr1y79+vUr5kIAAAAAUO8adNC2NTp06JDvf//7GTlyZAYNGpSjjz46TZs2zaRJk/LOO+/k9NNPX2ek28EHH5zTTjst119/ffr3759+/fpl5cqVmThxYioqKnLJJZdUB3ZrHXPMMbn//vszadKkHHfccTniiCNSUVGRP/3pT1mzZk1+/OMfb/CtpAAAAAA0PB+7oC1Jhg4dmj333DNjx47NnXfemaqqquy77745//zzM2DAgPX2GT58eDp16pTx48fn97//fUpKStKlS5ecccYZOeKII9ZpX1JSkp/97GcZP358brvttowfPz7NmjXLIYcckm9+85s5+OCD6/pjAgAAALAN7bBB23nnnZfzzjtvg/v79OmTPn36bNExBwwYsMEgbn2aNGmSU089NaeeeuoWnQcAAACAhqdRfRcAAAAAADsCQRsAAAAAFEDQBgAAAAAFELQBAAAAQAEEbQAAAABQAEEbAAAAABRA0AYAAAAABRC0AQAAAEABBG0AAAAAUABBGwAAAAAUQNAGAAAAAAUQtAEAAABAAQRtAAAAAFAAQRsAAAAAFEDQBgAAAAAFELQBAAAAQAEEbQAAAABQAEEbAAAAABSgSX0XAAAAH9W6vHmaN21a32Xs0N5fuTKLK96v7zIAYIciaAMAYLvTvGnTfOWPN9d3GTu0e44/JYsjaAOAIpk6CgAAAAAFELQBAAAAQAEEbQAAAABQAEEbAAAAABRA0AYAAAAABRC0AQAAAEABBG0AAAAAUABBGwAAAAAUQNAGAAAAAAUQtAEAAABAAQRtAAAAAFAAQRsAAAAAFEDQBgAAAAAFELQBAAAAQAEEbQAAAABQAEEbAAAAABRA0AYAAAAABRC0AQAAAEABBG0AAAAAUABBGwAAAAAUQNAGAAAAAAUQtAEAAABAAQRtAAAAAFAAQRsAAAAAFEDQBgAAAAAFELQBAAAAQAEEbQAAAABQAEEbAAAAABRA0AYAAAAABRC0AQAAAEABBG0AAAAAUABBGwAAAAAUQNAGAAAAAAUQtAEAAABAAQRtAAAAAFAAQRsAAAAAFEDQBgAAAAAFELQBAAAAQAEEbQAAAABQAEEbAAAAABRA0AYAAAAABRC0AQAAAEABBG0AAAAAUABBGwAAAAAUQNAGAAAAAAUQtAEAAABAAQRtAAAAAFAAQRsAAAAAFEDQBgAAAAAFELQBAAAAQAEEbQAAAABQAEEbAAAAABRA0AYAAAAABRC0AQAAAEABBG0AAAAAUABBGwAAAAAUQNAGAAAAAAUQtAEAAABAAQRtAAAAAFAAQRsAAAAAFEDQBgAAAAAFELQBAAAAQAEEbQAAAABQAEEbAAAAABRA0AYAAAAABRC0AQAAAEABBG0AAAAAUABBGwAAAAAUQNAGAAAAAAUQtAEAAABAAQRtAAAAAFAAQRsAAAAAFEDQBgAAAAAFELQBAAAAQAEEbQAAAABQAEEbAAAAABRA0AYAAAAABRC0AQAAAEABBG0AAAAAUABBGwAAAAAUQNAGAAAAAAUQtAEAAABAAQRtAAAAAFAAQRsAAAAAFEDQBgAAAAAFELQBAAAAQAEEbQAAAABQAEEbAAAAABRA0AYAAAAABRC0AQAAAEABBG0AAAAAUABBGwAAAAAUQNAGAAAAAAUQtAEAAABAAQRtAAAAAFAAQRsAAAAAFEDQBgAAAAAFaFLfBRThiiuuyPPPP59Zs2ZlwYIFad68efbYY4/07ds3p5xyStq2bbtOn2nTpuVXv/pVnnvuuaxYsSIdO3bMoEGDMnTo0DRu3Hi953nkkUcyduzYzJgxI2vWrMm//uu/5uSTT86AAQM2WNsdd9yRm2++Oa+88koaNWqUzp075/TTT88RRxxR2OcHAAAAoP7tECPabrzxxixfvjyf/exnM2zYsBx77LFp3LhxRo8enf79++ftt9+u0f7BBx/MkCFD8vTTT6dv3745+eSTs3Llylx22WW54IIL1nuO8ePH55xzzslLL72U/v37Z/DgwZk3b16GDx+eUaNGrbfPqFGjMnz48MyfPz+DBw9O//7989JLL+Wcc87J+PHjC78OAAAAANSfHWJE29SpU9OsWbN1tl911VUZM2ZMrr322lx66aVJkiVLluSSSy5Jo0aNctNNN6Vr165JkvPPPz9f//rXM2nSpNx777055phjqo8zZ86cjBo1KuXl5bntttvSvn37JMm3v/3tHH/88Rk7dmy++MUvplu3btV9pk2blrFjx6Zjx4754x//mDZt2iRJzjjjjAwaNCijRo3KF77whepjAQAAANCw7RAj2tYXsiXJl7/85STJ7Nmzq7fdd999ee+993LMMcdUh2xrj/Gd73wnSTJhwoQax7nttttSWVmZU045pUYw1qZNm5x99tlJkltvvbVGn7W/n3POOdUhW5K0b98+J598ciorK3P77bdv8WcFAAAAYPu0QwRtG/Lwww8nSTp16lS9bfLkyUmSXr16rdP+0EMPTYsWLfLMM8+ksrJys/ocfvjhNdrUpg8AAAAADdcOMXV0rd/+9rdZtmxZFi9enOeffz5Tp05Np06dctZZZ1W3ee2115Ike++99zr9mzRpkvbt2+fll1/OG2+8kX333XeTfXbddde0bNky77zzTpYvX54WLVpk2bJlmTt3blq2bJldd911nT577bVXkmTWrFm1/MQAAAAAbC92qKBt7Nixeffdd6t/79WrVy6//PK0a9euetuSJUuSJK1bt17vMcrKypIkixYt2qI+awO+Fi1aZPHixRttv3b7h89RG+XlLQs5DsVwPxo296/hcu8aNvevYXP/Gi73rmFz/xou965hc/8atrq+fzvU1NEnnngiL774Yp544olcc801eeONN3LcccflhRde2OJjlZSUbHbbqqqqLe6zNe0BAAAA2H7tUCPa1vrEJz6Ro446Kp07d06/fv3yH//xH7nnnnuS/P8j1taOOvuo9Y1eKysry4IFC7J48eK0bdt2nT5Lly6tcey1fTd0jk2NeNtSFRXLNqvdLrsUcz42bnPvx5Zy/7aNurh/7t224bvXsPnuNWzuX8Pl386GzXev4fLda9h89xq2zbl/tbkfO9SIto/ac88986//+q95+eWX89577yVJ9tlnnyTrXx9t1apVmTNnTpo0aZIOHTpUb99Yn3nz5mXZsmXZfffd06JFiyRJy5Yts9tuu2XZsmWZN2/eOn3WvgV1fWu+AQAAANAw7dBBW5LqoKtx48ZJkp49eyZJHn/88XXaPvXUU1m+fHm6deuW0tLS6u0b6/PYY4/VaFObPgAAAAA0XA0+aHvllVcyf/78dbavWbMmV111Vf75z3+mW7duadOmTZLkS1/6Utq2bZt7770306dPr26/YsWK/PznP0+SnHTSSTWONXDgwJSWlubmm2/OnDlzqrcvXLgw1157bZLka1/7Wo0+a38fM2ZMFi5cWL19zpw5ueWWW1JaWpqBAwfW5qMDAAAAsB1p8Gu0Pf7447niiityyCGHpGPHjikvL8+7776bp556Km+88UZ22WWXjBw5srp9WVlZRo4cmX/7t3/LsGHDcvTRR6dNmzZ5+OGH89prr6Vfv345+uija5yjQ4cO+f73v5+RI0dm0KBBOfroo9O0adNMmjQp77zzTk4//fR069atRp+DDz44p512Wq6//vr0798//fr1y8qVKzNx4sRUVFTkkksuSfv27bfJNQIAAACg7jX4oO2zn/1sTjjhhEybNi0zZ87M4sWL06JFi+y999756le/mqFDh6a8vLxGn759+2bcuHEZM2ZM7r///qxYsSJ77bVXRowYkaFDh673baBDhw7NnnvumbFjx+bOO+9MVVVV9t1335x//vkZMGDAemsbPnx4OnXqlPHjx+f3v/99SkpK0qVLl5xxxhk54ogj6uR6AAAAAFA/GnzQtv/+++dHP/rRFvfr3r17rrvuui3q06dPn/Tp02eL+gwYMGCDQRwAAAAAO44Gv0YbAAAAAGwPBG0AAAAAUABBGwAAAAAUQNAGAAAAAAUQtAEAAABAAQRtAAAAAFAAQRsAAAAAFEDQBgAAAAAFELQBAAAAQAEEbQAAAABQAEEbAAAAABRA0AYAAAAABRC0AQAAAEABBG0AAAAAUABBGwAAAAAUQNAGAAAAAAUQtAEAAABAAQRtAAAAAFAAQRsAAAAAFEDQBgAAAAAFELQBAAAAQAEEbQAAAABQAEEbAAAAABRA0AYAAAAABRC0AQAAAEABBG0AAAAAUABBGwAAAAAUQNAGAAAAAAUQtAEAAABAAQRtAAAAAFAAQRsAAAAAFEDQBgAAAAAFKDxoe+uttzJ37tzNbj937ty89dZbRZcBAAAAANtUk6IP2KdPn+yyyy55/PHHN6v9SSedlHfeeSczZswouhQAAAAA2GbqZOpoVVVVnbYHAAAAgO1Nva/R9v7776dx48b1XQYAAAAA1Eq9Bm2zZ8/OggULsssuu9RnGQAAAABQa7Veo+3BBx/MQw89VGPbkiVLMmLEiI32W7RoUaZOnZokOeyww2pbBgAAAADUq1oHbTNnzswdd9xRY9v777+/zrYN6dixY77zne/UtgwAAAAAqFe1Dtp69OiRc889t/r3a665Ji1btszpp5++wT4lJSUpKyvLfvvtlx49eqRJk8JffgoAAAAA21QhQVuPHj2qf18btH04fAMAAACAHV3hQ8keeughbxEFAAAA4GOn8KBtzz33LPqQAAAAALDdq9PF0VatWpXZs2dn0aJFWbVq1UbbHnrooXVZCgAAAADUqToJ2t5444387Gc/y8MPP5zKyspNti8pKcmMGTPqohQAAAAA2CYKD9pmz56dE088MQsXLkxVVVVKSkqy8847p7S0tOhTAQAAAMB2o/Cg7ec//3kqKiqy++675wc/+EH69OmTJk3qdIYqAAAAANS7whOwyZMnp6SkJD/96U/TvXv3og8PAAAAANulRkUfcOnSpWnevLmQDQAAAICPlcKDtk9+8pNZs2ZNqqqqij40AAAAAGy3Cg/ajjnmmFRWVubJJ58s+tAAAAAAsN0qPGg766yz8qlPfSo//OEP88YbbxR9eAAAAADYLhX+MoQ//elPGThwYEaPHp3+/funX79+6dq1a1q1arXRfscdd1zRpQAAAADANlN40DZ8+PCUlJRUr9F211135a677tpkP0EbAAAAAA1Z4UHboYceWvQhAQAAAGC7V3jQNm7cuKIPCQAAAADbvcJfhgAAAAAAH0eCNgAAAAAogKANAAAAAApQ+BptBxxwwBb3KSkpyYwZM4ouBQAAAAC2mcKDtqqqqm3SBwAAAAC2J4UHbQ899NBG9y9evDjTp0/PTTfdlHnz5uWyyy5Lp06dii4DAAAAALapwoO2Pffcc5NtPvWpT+WrX/1qzjzzzFx00UW5/fbbiy4DAAAAALapensZQmlpaS6++OIsWLAg11xzTX2VAQAAAACFqNe3ju63334pKyvL448/Xp9lAAAAAECtFT51dEtUVlbm/fffT2VlZX2WAQAAAAC1Vq8j2u65556sWrUqu+66a32WAQAAAAC1VviItrfeemuj+1esWJF33nknDz30UP7whz+kpKQkX/rSl4ouAwAAAAC2qcKDtiOPPHKz21ZVVeUzn/lMvvWtbxVdBgAAAABsU4UHbVVVVRvd37hx47Ru3Tr7779/vvzlL2fw4MFp0qRel4oDAAAAgForPOGaOXNm0YcEAAAAgO1evb4MAQAAAAB2FII2AAAAAChAnS6OtnTp0vz5z3/OjBkz8t577yVJ2rVrl86dO6d3795p1apVXZ4eAAAAALaZOgnaqqqqcu211+a6667LsmXL1tumZcuWOfvss3PmmWempKSkLsoAAAAAgG2mToK24cOH5+67705VVVWaNWuWLl26ZPfdd0+SvPPOO3nhhReydOnSXHXVVXnllVcyatSouigDAAAAALaZwoO2+++/P3fddVdKSkqqR6yVlZXVaLNkyZL8+te/znXXXZe77747ffv2zVFHHVV0KQAAAACwzRT+MoTf/e53KSkpyQUXXJALLrhgnZAtScrKyvLd73433/nOd1JVVZXf/e53RZcBAAAAANtU4UHbCy+8kMaNG2fo0KGbbDts2LA0btw4zz//fNFlAAAAAMA2VXjQtnTp0rRq1SotWrTYZNuWLVumrKwsS5cuLboMAAAAANimCg/adt555yxatChz587dZNu5c+dm0aJFadeuXdFlAAAAAMA2VXjQdsghhyRJLr/88lRVVW207WWXXZYk6dGjR9FlAAAAAMA2VXjQdsYZZ6SkpCT33Xdfhg4dmsceeyzLly+v3r9gwYLcd999GTRoUCZNmpRGjRrl9NNPL7oMAAAAANimmhR9wAMOOCA/+tGP8p//+Z+ZOnVqzj777JSUlKR169aprKzM+++/nySpqqpKo0aN8sMf/jAHHHBA0WUAAAAAwDZV+Ii2JDnxxBMzfvz46imha9asycKFC7N8+fLq6aQ9e/bMzTffnBNPPLEuSgAAAACAbarwEW1rHXzwwbnxxhuzcOHCzJgxIwsWLEiStG3bNp07d06bNm3q6tQAAAAAsM3VWdC2Vps2bfL//t//q+vTAAAAAEC9Knzq6AsvvJBhw4Zl1KhRm2w7cuTIDBs2LDNnziy6DAAAAADYpgoP2u6444489dRT6dKlyybb7r///pkyZUruvPPOossAAID/r717j/Kqrvc//uLOjCigjJCMgJfgmCcNL5mlokKScNIAKdKQxHXSUpI8lVC6KsODpC4DJNTOQUTwQiniSQUhNUFTQRQv4MHwBqaCIghyE+H3hz9mnYlBIDfMjD4e/yjfvT/feX9nrxmY5+zv3gAAu1Thoe2xxx5L8uHNDrblhBNOSJI8+uijRY8BAAAAALtU4aHtjTfeSOPGjdOiRYtt7ltWVpbGjRvn9ddfL3oMAAAAANilCg9tGzZsSJ06dbZ/gLp1s3bt2qLHAAAAAIBdqvDQtvfee2fNmjV58cUXt7nviy++mNWrV6esrKzoMQAAAABglyo8tB111FHZtGlTRo4cuc19R4wYkTp16uSoo44qegwAAAAA2KUKD239+vVLvXr1MmXKlPzkJz/JkiVLtthnyZIl+fGPf5wpU6akbt266devX9FjAAAAAMAuVb/oJzzggAMyaNCgXHbZZfnTn/6Ue++9Nx06dMg+++yTJHnttdeyYMGCfPDBB0mSn/zkJ2nfvn3RYwAAAADALlV4aEuSvn37pkWLFrn88svz5ptv5rnnnstzzz1XaZ+WLVvmoosuSrdu3XbGCAAAAACwS+2U0JYkJ598cr761a/mr3/9a+bOnZu33normzZtSllZWQ499NAcffTRqV9/p314AAAAANildmrpql+/fo499tgce+yxO/PDAAAAAEC1K/xmCAAAAADwaSS0AQAAAEABhDYAAAAAKIDQBgAAAAAFENoAAAAAoABCGwAAAAAUQGgDAAAAgAIIbQAAAABQAKENAAAAAAogtAEAAABAAYQ2AAAAACiA0AYAAAAABRDaAAAAAKAAQhsAAAAAFEBoAwAAAIACCG0AAAAAUAChDQAAAAAKILQBAAAAQAGENgAAAAAoQP3qHuDjeueddzJ9+vQ8+OCDWbBgQd588800aNAg7du3T8+ePdOrV6/UrbtlT5wzZ05Gjx6duXPnZt26dWnTpk169eqVvn37pl69elV+rAceeCBjxozJvHnzsnHjxhx44IE5/fTT06NHj63ON2nSpEyYMCELFy5M3bp187nPfS79+/fPCSecUNjnAAAAAIDqV+vPaJsyZUouvvjizJ07N4ccckj69euXk046KS+88EIuvvjiDBw4MJs2baq0Zvr06fnOd76T2bNnp0uXLjn99NPz/vvvZ+jQofnRj35U5ccZP358zj333CxYsCCnnHJKevfunSVLlmTQoEEZNmxYlWuGDRuWQYMGZenSpendu3dOOeWULFiwIOeee27Gjx9f+OcCAAAAgOpT689oa9euXUaPHp3jjz++0plrF154YXr37p2pU6fmvvvuS9euXZMkq1atyiWXXJK6detm3Lhx+fznP58kGThwYPr165epU6fm7rvvTvfu3Suea/HixRk2bFiaNWuW22+/PeXl5UmS8847L6eddlrGjBmTk046KR07dqxYM2fOnIwZMyZt2rTJH//4xzRt2jRJcvbZZ6dXr14ZNmxYjj/++IrnAgAAAKB2q/VntB199NE58cQTt3h7aFlZWfr06ZMkefzxxysenzJlSpYtW5bu3btXRLYkadSoUS644IIkyS233FLpuW6//fasX78+Z5xxRqUw1rRp05xzzjlJkltvvbXSms1/PvfccysiW5KUl5fn9NNPz/r163PHHXf8068bAAAAgJql1oe2j1K//ocn7P3fa649+uijSZJjjz12i/2PPPLIlJSU5Mknn8z69eu3a81xxx1XaZ+PswYAAACA2qvWv3V0azZs2JDJkycnqRy7XnrppSQfvuX0H9WvXz/l5eV54YUXsmjRohxwwAHbXLP33nuntLQ0b7zxRtasWZOSkpKsXr06b775ZkpLS7P33ntvsaZt27ZJkpdffvnjvMQKzZqVFvI8FMPxqN0cv9rLsavdHL/azfGrvRy72s3xq70cu9rN8avddvbx+8Se0XbVVVdlwYIF6dSpU6XQtmrVqiTJ7rvvXuW6Jk2aJEnefffdHV6zcuXKSv/d2v6bH/+/HwMAAACA2u0TeUbbuHHjMmbMmOy///75zW9+8089R506dbZ73813Nd2RNf/M/luzfPnq7dqvrKzq8Eextvd47CjHb9fYGcfPsds1fO3Vbr72ajfHr/byvbN287VXe/naq9187dVu23P8Ps7x+MSd0TZhwoRcdtllOfDAAzNu3Lg0a9as0vZ/PPvsH1V19tq21rz33nuV9tu8dmv7b+uMNwAAAABqn09UaBs7dmwuvfTStG/fPuPGjUtZWdkW++y3335Jqr4+2oYNG7J48eLUr18/++6773atWbJkSVavXp1WrVqlpKQkSVJaWpqWLVtm9erVWbJkyRZrXnnllSRVX/MNAAAAgNrpExParr/++gwdOjQHHXRQbrzxxuy1115V7velL30pSTJjxowtts2aNStr1qxJx44d07Bhw+1a89BDD1Xa5+OsAQAAAKD2+kSEtlGjRuWqq67KwQcfnLFjx2bPPffc6r5f+9rX0rx589x999155plnKh5ft25dhg8fniT59re/XWlNz54907Bhw0yYMCGLFy+ueHzFihW57rrrkiR9+vSptGbzn6+99tqsWLGi4vHFixfn5ptvTsOGDdOzZ89/8hUDAAAAUNPU+pshTJo0KSNGjEi9evVyxBFH5Kabbtpin9atW1dErSZNmmTIkCH54Q9/mDPPPDPdunVL06ZNc//99+ell15K165d061bt0rr99133/z0pz/NkCFD0qtXr3Tr1i0NGjTI1KlT88Ybb6R///7p2LFjpTWHHXZYzjrrrNxwww055ZRT0rVr17z//vu55557snz58lxyySUpLy/feZ8YAAAAAHapWh/aNp9h9sEHH+TGG2+scp8vfvGLlc4e69KlS2666aZce+21ue+++7Ju3bq0bds2gwcPTt++fau8G2jfvn3TunXrjBkzJnfeeWc2bdqUAw44IAMHDkyPHj2q/LiDBg1Khw4dMn78+EycODF16tTJwQcfnLPPPjsnnHBCAa8eAAAAgJqi1oe2AQMGZMCAATu87vDDD8/vf//7HVpz4okn5sQTT9yhNT169NhqiAMAAADgk+MTcY02AAAAAKhuQhsAAAAAFEBoAwAAAIACCG0AAAAAUAChDQAAAAAKILQBAAAAQAGENgAAAAAogNAGAAAAAAUQ2gAAAACgAEIbAAAAABRAaAMAAACAAghtAAAAAFAAoQ0AAAAACiC0AQAAAEABhDYAAAAAKIDQBgAAAAAFENoAAAAAoABCGwAAAAAUQGgDAAAAgAIIbQAAAABQAKENAAAAAAogtAEAAABAAYQ2AAAAACiA0AYAAAAABRDaAAAAAKAAQhsAAAAAFEBoAwAAAIACCG0AAAAAUAChDQAAAAAKILQBAAAAQAGENgAAAAAogNAGAAAAAAUQ2gAAAACgAEIbAAAAABRAaAMAAACAAghtAAAAAFAAoQ0AAAAACiC0AQAAAEABhDYAAAAAKIDQBgAAAAAFENoAAAAAoABCGwAAAAAUQGgDAAAAgAIIbQAAAABQAKENAAAAAAogtAEAAABAAYQ2AAAAACiA0AYAAAAABRDaAAAAAKAAQhsAAAAAFEBoAwAAAIACCG0AAAAAUAChDQAAAAAKILQBAAAAQAGENgAAAAAogNAGAAAAAAUQ2gAAAACgAEIbAAAAABRAaAMAAACAAghtAAAAAFAAoQ0AAAAACiC0AQAAAEABhDYAAAAAKIDQBgAAAAAFENoAAAAAoABCGwAAAAAUQGgDAAAAgAIIbQAAAABQAKENAAAAAAogtAEAAABAAYQ2AAAAACiA0AYAAAAABRDaAAAAAKAAQhsAAAAAFEBoAwAAAIACCG0AAAAAUAChDQAAAAAKILQBAAAAQAGENgAAAAAogNAGAAAAR/N24gAAIABJREFUAAUQ2gAAAACgAEIbAAAAABRAaAMAAACAAghtAAAAAFAAoQ0AAAAACiC0AQAAAEABhDYAAAAAKIDQBgAAAAAFENoAAAAAoABCGwAAAAAUQGgDAAAAgAIIbQAAAABQAKENAAAAAAogtAEAAABAAYQ2AAAAACiA0AYAAAAABRDaAAAAAKAAQhsAAAAAFEBoAwAAAIACCG0AAAAAUAChDQAAAAAKILQBAAAAQAGENgAAAAAogNAGAAAAAAUQ2gAAAACgAEIbAAAAABRAaAMAAACAAghtAAAAAFAAoQ0AAAAACiC0AQAAAEABhDYAAAAAKIDQBgAAAAAFENoAAAAAoABCGwAAAAAUQGgDAAAAgAIIbQAAAABQAKENAAAAAAogtAEAAABAAepX9wBFmDJlSmbNmpX58+fn+eefz3vvvZevf/3rufLKK7e6Zs6cORk9enTmzp2bdevWpU2bNunVq1f69u2bevXqVbnmgQceyJgxYzJv3rxs3LgxBx54YE4//fT06NFjqx9n0qRJmTBhQhYuXJi6devmc5/7XPr3758TTjjhY79uAAAAAGqOT8QZbaNHj8748eMzf/78tGzZcpv7T58+Pd/5zncye/bsdOnSJaeffnref//9DB06ND/60Y+qXDN+/Pice+65WbBgQU455ZT07t07S5YsyaBBgzJs2LAq1wwbNiyDBg3K0qVL07t375xyyilZsGBBzj333IwfP/5jvWYAAAAAapZPxBltgwcPTqtWrdK2bds8/vjjOfPMM7e676pVq3LJJZekbt26GTduXD7/+c8nSQYOHJh+/fpl6tSpufvuu9O9e/eKNYsXL86wYcPSrFmz3H777SkvL0+SnHfeeTnttNMyZsyYnHTSSenYsWPFmjlz5mTMmDFp06ZN/vjHP6Zp06ZJkrPPPju9evXKsGHDcvzxx1c8FwAAAAC12yfijLYvfelLadeuXerUqbPNfadMmZJly5ale/fuFZEtSRo1apQLLrggSXLLLbdUWnP77bdn/fr1OeOMMyqFsaZNm+acc85Jktx6662V1mz+87nnnlsR2ZKkvLw8p59+etavX5877rhjB18pAAAAADXVJyK07YhHH300SXLsscduse3II49MSUlJnnzyyaxfv3671hx33HGV9vk4awAAAACovT4Rbx3dES+99FKSpF27dltsq1+/fsrLy/PCCy9k0aJFOeCAA7a5Zu+9905paWneeOONrFmzJiUlJVm9enXefPPNlJaWZu+9995iTdu2bZMkL7/8ciGvqVmz0kKeh2I4HrWb41d7OXa1m+NXuzl+tZdjV7s5frWXY1e7OX61284+fp+6M9pWrVqVJNl9992r3N6kSZMkybvvvrvDa1auXFnpv1vbf/Pj//djAAAAAFC7ferOaNte23O9t802bdq0w2v+mf23Zvny1du1X1lZ1eGPYm3v8dhRjt+usTOOn2O3a/jaq9187dVujl/t5Xtn7eZrr/bytVe7+dqr3bbn+H2c4/GpO6PtH88++0dVnb22rTXvvfdepf02r93a/ts64w0AAACA2udTF9r222+/JFVfH23Dhg1ZvHhx6tevn3333Xe71ixZsiSrV69Oq1atUlJSkiQpLS1Ny5Yts3r16ixZsmSLNa+88kqSqq/5BgAAAEDt9KkLbV/60peSJDNmzNhi26xZs7JmzZp07NgxDRs23K41Dz30UKV9Ps4aAAAAAGqvT11o+9rXvpbmzZvn7rvvzjPPPFPx+Lp16zJ8+PAkybe//e1Ka3r27JmGDRtmwoQJWbx4ccXjK1asyHXXXZck6dOnT6U1m/987bXXZsWKFRWPL168ODfffHMaNmyYnj17FvviAAAAAKg2n4ibIUyfPj3Tp09PkixdujRJ8tRTT2XQoEFJkubNm+eiiy5K8uF11IYMGZIf/vCHOfPMM9OtW7c0bdo0999/f1566aV07do13bp1q/T8++67b376059myJAh6dWrV7p165YGDRpk6tSpeeONN9K/f/907Nix0prDDjssZ511Vm644Yaccsop6dq1a95///3cc889Wb58eS655JKUl5fv7E8NAAAAALvIJyK0zZ8/P5MmTar02KJFi7Jo0aIkSevWrStCW5J06dIlN910U6699trcd999WbduXdq2bZvBgwenb9++Vd4NtG/fvmndunXGjBmTO++8M5s2bcoBBxyQgQMHpkePHlXONWjQoHTo0CHjx4/PxIkTU6dOnRx88ME5++yzc8IJJxT4GQAAAACgun0iQtuAAQMyYMCAHVpz+OGH5/e///0OrTnxxBNz4okn7tCaHj16bDXEAQAAAPDJ8am7RhsAAAAA7AxCGwAAAAAUQGgDAAAAgAIIbQAAAABQAKENAAAAAAogtAEAAABAAYQ2AAAAACiA0AYAAAAABRDaAAAAAKAAQhsAAAAAFEBoAwAAAIACCG0AAAAAUAChDQAAAAAKILQBAAAAQAGENgAAAAAogNAGAAAAAAUQ2gAAAACgAEIbAAAAABRAaAMAAACAAghtAAAAAFAAoQ0AAAAACiC0AQAAAEABhDYAAAAAKIDQBgAAAAAFENoAAAAAoABCGwAAAAAUQGgDAAAAgAIIbQAAAABQAKENAAAAAAogtAEAAABAAYQ2AAAAACiA0AYAAAAABRDaAAAAAKAAQhsAAAAAFEBoAwAAAIACCG0AAAAAUAChDQAAAAAKILQBAAAAQAGENgAAAAAogNAGAAAAAAUQ2gAAAACgAEIbAAAAABRAaAMAAACAAghtAAAAAFAAoQ0AAAAACiC0AQAAAEABhDYAAAAAKIDQBgAAAAAFENoAAAAAoABCGwAAAAAUQGgDAAAAgAIIbQAAAABQAKENAAAAAAogtAEAAABAAYQ2AAAAACiA0AYAAAAABRDaAAAAAKAAQhsAAAAAFEBoAwAAAIACCG0AAAAAUAChDQAAAAAKILQBAAAAQAGENgAAAAAogNAGAAAAAAUQ2gAAAACgAEIbAAAAABRAaAMAAACAAghtAAAAAFAAoQ0AAAAACiC0AQAAAEABhDYAAAAAKIDQBgAAAAAFENoAAAAAoABCGwAAAAAUQGgDAAAAgAIIbQAAAABQAKENAAAAAAogtAEAAABAAYQ2AAAAACiA0AYAAAAABRDaAAAAAKAAQhsAAAAAFEBoAwAAAIACCG0AAAAAUAChDQAAAAAKILQBAAAAQAGENgAAAAAogNAGAAAAAAUQ2gAAAACgAEIbAAAAABRAaAMAAACAAghtAAAAAFAAoQ0AAAAACiC0AQAAAEABhDYAAAAAKIDQBgAAAAAFENoAAAAAoABCGwAAAAAUQGgDAAAAgAIIbQAAAABQAKENAAAAAAogtAEAAABAAYQ2AAAAACiA0AYAAAAABRDaAAAAAKAAQhsAAAAAFEBoAwAAAIACCG0AAAAAUAChDQAAAAAKILQBAAAAQAGENgAAAAAogNAGAAAAAAUQ2gAAAACgAEIbAAAAABRAaAMAAACAAghtAAAAAFAAoQ0AAAAAClC/ugf4pHvjjTcyfPjwzJgxI8uXL8/ee++dzp075/zzz0/Tpk2rezwAAAAACiK07USvvvpq+vTpk7fffjudO3fO/vvvn6effjrjxo3LjBkzcsstt6R58+bVPSYAAAAABRDadqJf/epXefvtt3PxxRenb9++FY8PHTo0Y8eOzdVXX51LL720GicEAAAAoCiu0baTLFq0KDNnzkzr1q1zxhlnVNo2YMCAlJaW5q677srq1auraUIAAAAAiiS07SSPPvpokuSYY45J3bqVP81NmjTJYYcdljVr1mTu3LnVMR4AAAAABRPadpIXX3wxSdKuXbsqt7dt2zZJ8tJLL+2qkQAAAADYieps2rRpU3UP8Ul0ySWXZOLEiRkyZEh69+69xfarr7461157bS688MKcc8451TAhAAAAAEVyRls12dw369SpU82TAAAAAFAEoW0nadKkSZJk5cqVVW5ftWpVpf0AAAAAqN2Etp1k//33T5K8/PLLVW5/5ZVXkiT77bffrhoJAAAAgJ1IaNtJjjrqqCTJzJkzs3HjxkrbVq1alTlz5qRx48Y59NBDq2M8AAAAAAomtO0kbdq0yTHHHJPXXnstEyZMqLRt5MiRWb16dU499dSUlpZW04QAAAAAFMldR3eiV199NX369Mnbb7+dzp0754ADDsjcuXPz2GOPpV27drn11lvTvHnz6h4TAAAAgAIIbTvZ66+/nhEjRmTGjBlZvnx5ysrK0rlz55x//vlp1qxZdY8HAAAAQEGENgAAAAAogGu0AQAAAEABhDYAAAAAKIDQBgAAAAAFENoAAAAAoABCGwAAAAAUQGgDAAAAgAIIbQAAAABQgPrVPQD8ozfeeCPDhw/PjBkzsnz58uy9997p3Llzzj///DRt2rS6x2MrpkyZklmzZmX+/Pl5/vnn89577+XrX/96rrzyyuoejW145513Mn369Dz44INZsGBB3nzzzTRo0CDt27dPz54906tXr9St6/cyNdkVV1yRZ599Ni+//HLeeeedNG7cOPvss0+6dOmSM844I82bN6/uEdkBd955Zy666KIkyZAhQ9K7d+9qnoitOfHEE/Paa69Vua1FixZ5+OGHd/FE7KjZs2fnxhtvzJNPPpnly5enWbNmad++ffr165dOnTpV93hU4Y477sjgwYM/cp+6detm/vz5u2gidtSDDz6YcePG5W9/+1uWL1+esrKyHHzwwTnrrLPSsWPH6h6Pj7Bp06bcfvvtmThxYl544YVs3Lgx++23X3r27Jkzzjgj9erVq+4RawShjRrl1VdfTZ8+ffL222+nc+fO2X///fP0009n3LhxmTFjRm655RY/MNZQo0ePzvPPP5/S0tK0atUqL774YnWPxHaaMmVKfvnLX6asrCxHHXVU9tlnn7z11luZNm1aLr744syYMSPDhw9PnTp1qntUtuLGG2/M5z73uXz5y1/OXnvtlTVr1uSpp57KyJEjc9ttt2XixIn5zGc+U91jsh1ef/31DBkyJKWlpVm9enV1j8N22H333dOvX78tHi8tLa2GadgRv/vd7zJ8+PA0b948J5xwQsrKyvLOO+9k3rx5eeyxx4S2Guqggw7K+eefX+W22bNn59FHH81xxx23i6die11xxRX5r//6rzRr1ixdunRJ8+bN8+qrr+b+++/Pfffdl2HDhuXUU0+t7jHZiosuuiiTJ0/OXnvtlW7duqWkpCR//etfc9lll2X27Nl+Zvj/hDZqlF/96ld5++23c/HFF6dv374Vjw8dOjRjx47N1VdfnUsvvbQaJ2RrBg8enFatWqVt27Z5/PHHc+aZZ1b3SGyndu3aZfTo0Tn++OMrnbl24YUXpnfv3pk6dWruu+++dO3atRqn5KM88cQTadSo0RaPX3311bn22mtz3XXX5Ze//OWuH4wdsmnTpgwePDjNmjXLV7/61YwZM6a6R2I77LHHHhkwYEB1j8EOuvfeezN8+PB8+ctfzsiRI9OkSZNK299///1qmoxtOeigg3LQQQdVue1b3/pWkuSb3/zmrhyJ7bR06dKMGTMmLVq0yF133ZW99tqrYtujjz6afv36ZcSIEUJbDTV9+vRMnjw55eXl+cMf/pA999wzyYffLwcOHJipU6dm0qRJ6dmzZzVPWv28F4gaY9GiRZk5c2Zat26dM844o9K2AQMGpLS0NHfddZff8NdQX/rSl9KuXTu/waiFjj766Jx44olbvD20rKwsffr0SZI8/vjj1TEa26mqyJYkJ598cpLklVde2ZXj8E8aN25cHn300QwdOtTZULATbdy4MVdeeWVKSkpy1VVXbRHZkqRBgwbVMBkfx4IFC/LUU0+lZcuWOf7446t7HKrw97//PRs3bswhhxxSKbIlH/4ssdtuu2XZsmXVNB3bct999yVJ+vfvXxHZkg+/X15wwQVJkvHjx1fLbDWNM9qoMR599NEkyTHHHLPFD/xNmjTJYYcdlpkzZ2bu3Lk5+uijq2NE+NSpX//DvyZcb6F2uv/++5MkHTp0qOZJ2JaFCxfmqquuyplnnpkjjzyy4u9Ear7169dn8uTJef3111NSUpIOHTrkyCOP9H2zBpszZ04WL16crl27Zo899qi4RmmjRo1yyCGHuEZULXXbbbclSU477TRffzVU27Zt06BBgzzzzDNZtmxZpVgza9asvPfee+nSpUs1TshHeeutt5Ik5eXlW2zbd999kyTPPfdc3n333eyxxx67dLaaRmijxth8Ta927dpVub1t27aZOXNmXnrpJaENdoENGzZk8uTJSZJjjz22mqdhe/z3f/93Vq9enZUrV+bZZ5/NE088kQ4dOuR73/tedY/GR9iwYUN+8pOf5DOf+UwuvPDC6h6HHbR06dL89Kc/rfRYeXl5hg4dmi9+8YvVNBUf5dlnn03y4Q0revTokQULFlTafuSRR2bEiBGVIgA129q1a3PXXXelbt26biBTgzVr1iw//vGPc/nll6d79+7p0qVLmjVrVnGNtq985SsuE1SDbb5W+uLFi7fYtmjRoor/f/HFF/OFL3xhl81VEwlt1BirVq1K8uFFhauy+fGVK1fuspng0+yqq67KggUL0qlTJ6GtlhgzZkzFbxuTDwPp5Zdf7ofFGm7UqFGZP39+br755jRu3Li6x2EH9OzZM4cffng++9nPZrfddsuiRYsyfvz4TJw4Mf/+7/+e2267Lf/yL/9S3WPyD95+++0kya233pry8vKMHTs2hxxySP7+97/n8ssvz8yZM3PBBRfkpptuquZJ2V733ntv3n333Rx//PFu/lPDffe73015eXl+9rOfZeLEiRWPt23bNj169NjiLaXUHMcff3z+9Kc/ZezYsenevXuaNWuW5MNfGI4cObJivxUrVlTXiDWGa7RRa2zatClJXAMMdoFx48ZlzJgx2X///fOb3/ymusdhOz388MP53//93zz88MO55pprsmjRonzjG9/Ic889V92jsRVPP/10rrvuupx11lnerlYLnX/++Tn66KPTokWLlJSUpH379rn00ktz1llnZe3atZV+8KDm+OCDD5J8+G/LESNG5Oijj85uu+2Wz372sxk1alRatWqVxx9/PE8++WQ1T8r22vy20c03Q6Dm+v3vf58f/vCH6dGjR6ZPn56nnnoqd9xxR/bdd9/8+Mc/9u/OGqx79+457rjj8uqrr6Z79+655JJLctlll+Ub3/hG/vKXv1S8M81bt4U2apDNF6Ld2hlrm894q+qCtUBxJkyYkMsuuywHHnhgxo0bV/HbKmqPFi1aVNy1cvny5bnooouqeySqsPkto+3atcvAgQOrexwKtPlGMrNnz67mSahK06ZNk3x4TaF/POOwcePGOeaYY5J8GMKp+f72t7/lySefTKtWrdKpU6fqHoeP8Nhjj+XKK6/MiSeemMGDB2ffffdNSUlJDj744FxzzTVp2bJlbrjhhkpvQ6TmqFu3bkaPHp2LLrooLVq0yOTJk3P77benZcuWufnmmyt+ZnBWoreOUoPsv//+SZKXX365yu2b75q333777aqR4FNn7NixGTp0aNq3b5+xY8f6i7KWa926dQ488MDMnz9/i4sOU/1Wr15d8Xfe5z//+Sr3ufjii3PxxRfnzDPPzM9//vNdOB0fx+bvne6UXjNt/rfk1i5Xsvki3uvWrdtlM/HPu/XWW5O4CUJt8OCDDyZJjjrqqC22lZSU5JBDDsm0adMyb968iovrU7PUr18//fv3T//+/Ss9vnbt2syfPz+NGzfOgQceWE3T1RxCGzXG5m+4M2fOzMaNGyvdeXTVqlWZM2dOGjdunEMPPbS6RoRPtOuvvz5XXXVVDjrooIwZM0aU+YRYsmRJEqfx10QNGzbMaaedVuW2efPmZd68eTn88MOz3377eVtpLbP5LYd+UKyZjjjiiNSvXz+vvPJK1q9fn4YNG1ba/sILLyT58JcV1Gzr1q2ruAnC1r6fUnOsX78+SbJs2bIqt29+vEGDBrtsJooxefLkrFu3Lj169HD8IrRRg7Rp0ybHHHNMZs6cmQkTJqRv374V20aOHJnVq1fnW9/6VkpLS6txSvhkGjVqVEaMGJGDDz44Y8aM8XbRWmThwoXZY489UlZWVunxjRs3Zvjw4Xn77bfTsWPHirdKUXM0btw4l112WZXbRo4cmXnz5qVHjx7uoFdDvfDCCykrK9vi++Vrr72WX//610mSU045pTpGYxv23HPPnHzyyfmf//mfjBo1Kj/60Y8qtj388MOZOXNmdt99dzcCqgXuvfferFixIieccIKbINQChx9+eMUNY/r06ZOWLVtWbPvLX/6SOXPmpFGjRn65VIOtWrVqi0s5Pf3007nqqqtSWlqa8847r5omq1mENmqUX/ziF+nTp0+GDBmSv/71rznggAMyd+7cPPbYY2nXrl2lfwhRs0yfPj3Tp09PkixdujRJ8tRTT2XQoEFJPrwdtOtE1UyTJk3KiBEjUq9evRxxxBFV3mWtdevW6dmzZzVMx7bMmDEjV1xxRY444oi0adMmzZo1y1tvvZVZs2Zl0aJFKSsry5AhQ6p7TPjEmTJlSq6//vocddRRKS8vr7jr6IMPPph169alU6dOW7y1hppj8ODBefrpp3Pttddm9uzZOeSQQ/Laa69l+vTpqVevXn79619XvIWUmmvzXSu/+c1vVvMkbI+vfe1r+cMf/pBHHnkkJ598cr761a+mRYsWWbhwYR588MFs2rQp//Ef/5HmzZtX96hsxVlnnZXGjRtX3G37hRdeyEMPPZSGDRtm5MiRzuT+/4Q2apQ2bdrk9ttvz4gRIzJjxow89NBDKSsrS9++fXP++ec7y6YGmz9/fiZNmlTpsUWLFlVczLR169ZCWw21ePHiJB/ehe3GG2+scp8vfvGLQlsN9eUvfznf/OY3M2fOnDz//PNZuXJlSkpK0q5du5x66qnp27ev752wExx11FF56aWXMm/evDz11FNZs2ZNdt999xx++OE59dRTc+qpp7pTeg221157ZeLEiRk9enSmTZuWuXPnZrfddkunTp1yzjnn5Atf+EJ1j8g2LFy4ME888YSbINQidevWzfXXX58JEybknnvuybRp07J27do0bdo0nTp1St++fStuRkLN1LVr19xzzz256667snbt2uy999457bTT8r3vfS/l5eXVPV6NUWfTpk2bqnsIAAAAAKjt6m57FwAAAABgW4Q2AAAAACiA0AYAAAAABRDaAAAAAKAAQhsAAAAAFEBoAwAAAIACCG0AAAAAUAChDQAAAAAKILQBAAAAQAGENgAAAAAogNAGAAAAAAUQ2gAAPsX69u2bDh065I477ijk+UaOHJkOHTpk0KBBhTzf9rrjjjvSoUOH9O3bd5d+XACA/0toAwAAAIACCG0AAAAAUAChDQAAAAAKILQBAAAAQAHqV/cAAADUPAsWLMi0adPy8MMP5+9//3veeuut7Lbbbmnfvn1OOeWU9OzZM/Xq1fvI59i4cWPGjRuXO+64I6+++moaNWqUjh075gc/+EEOOeSQj1x31113ZfLkyZk/f35WrVqVZs2a5YgjjshZZ52VQw89tOiXCwBQCKENAIAt9O3bN8uXL0+S1KtXL6WlpVm+fHkef/zxPP7445k2bVp+97vfpX79qv85uWnTplxwwQW57777Ur9+/ZSUlGT58uV54IEH8tBDD+XKK69Mt27dtli3atWqDBgwII888kiSpE6dOtltt92ydOnS3HvvvZk6dWp+/vOf5zvf+c7Oe/EAAP8kbx0FAGALRx55ZIYMGZIHHnggTz/9dGbPnp0nn3wyv/nNb1JWVpa//OUvGTt27FbX//nPf86f//znDB48OE888URmz56dadOm5Stf+Uo++OCDDB48OK+++uoW6y666KI88sgj6dChQ66//vo89dRTeeKJJzJr1qxceOGFqVevXi677LI88cQTO/HVAwD8c4Q2AAC2cM0116R3797ZZ599Ks5aKy0tzamnnprf/va3SZKbb755q+tXrlyZAQMG5Lvf/W4aN26cJGnTpk1Gjx6d/fbbL2vXrs11111Xac0jjzyS6dOnp3Xr1hk3blw6depUsXaPPfbIOeeckwsuuCAbN27M9ddfvzNeNgDAxyK0AQCwQ4444ojsscceee211/Lmm29WuU9JSUn69eu3xeONGjVK//79kyT33XdfNm3aVLFt0qRJSZKePXumWbNmVT7v17/+9STJY489lg8++OBjvQ4AgKK5RhsAAFWaMmVK7rrrrsybNy/Lli3LunXrtthnyZIladmy5RaP/+u//mtKS0urfN4jjzwySfLuu+9m8eLF2XfffZMkTz75ZJJk7NixueWWWz5ytjVr1mT58uXZa6+9dug1AQDsTEIbAACVbNiwIQMHDsy0adMqHmvYsGGaN29ecafRZcuWZePGjVmzZk2Vz1FVfKtq27JlyypC29KlS5N8+LbTlStXbnPOrX1sAIDqIrQBAFDJxIkTM23atJSUlOTCCy/MSSedlFatWlXap1OnTnnjjTcqvfVze21tzcaNG5Mkv/vd79K5c+cdHxwAoJq5RhsAAJVMmTIlSfKDH/wgZ5555haR7YMPPsg777zzkc+xZMmSrW7bfOZakuy5554V/9+iRYskycKFC3d4ZgCAmkBoAwCgks03ODjooIOq3D5nzpwqr9f2fz3zzDNbfWvnrFmzknx4J9Hy8vKKx7/whS8kSaZOnbrDMwMA1ARCGwAAlTRp0iRJsmDBgi22bdiwIb/97W+3+Rxr1qzJuHHjtnh8/fr1ueGGG5IkXbt2TZ06dSq29ejRI0ny7LPP5s477/zI51+xYsU2ZwAA2NWENgAAKvnKV76S5MNrpU2fPj0ffPBBkg/f0nnuuefm6aef3uodRTfbfffdM3z48Nx4441Zu3Z82dCFAAAB30lEQVRtkmTRokX5/ve/n4ULF6ZRo0b53ve+V2nNcccdl5NOOilJ8rOf/SwjRoyo9BbUFStWZPr06fn+97+fyy+/vLDXCwBQFDdDAACgkv79++fee+/Nq6++mvPOOy8NGjRIo0aNsmrVqtSrVy9DhgzJNddck9WrV2/1OTp37pz33nsv//mf/5krrrgiJSUleffdd5Mk9erVy9ChQ9OmTZst1g0bNiwbN27M9OnTM2rUqIwaNSq77757Nm3alFWrVlXs17Nnz+JfOADAx+SMNgAAKmnWrFluu+22fPvb3664EULjxo3TpUuX3HTTTdsVuerUqZPhw4dn8ODB2X///fP++++nadOmOeGEE3Lrrbeme/fuVa4rLS3NqFGjct111+Wkk05Ky5Yts3bt2rz//vtp27Zt/u3f/i0jRozIL37xi0JfMwBAEeps+mfuyQ4AAAAAVOKMNgAAAAAogNAGAAAAAAUQ2gAAAACgAEIbAAAAABRAaAMAAACAAghtAAAAAFAAoQ0AAAAACiC0AQAAAEABhDYAAAAAKIDQBgAAAAAFENoAAAAAoABCGwAAAAAUQGgDAAAAgAIIbQAAAABQAKENAAAAAAogtAEAAABAAYQ2AAAAACiA0AYAAAAABfh/OCQ8xzKh/XMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 390,
       "width": 621
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(10,6))\n",
    "sns.countplot(x='label', data=train_raw)\n",
    "plt.title('Digit Label Distribution');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Time to see how the samples look like images. For this, they will reshaped for images with 28x28 pixels, because the product of this results in 784, the number of columns/pixels in the dataset (except label column). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "_kg_hide-input": true,
    "id": "r3FT-bHc1GWB",
    "outputId": "1cd9b37f-8e82-4c45-daad-15c643125f70"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABHgAAALLCAYAAACCfDWiAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAWJQAAFiUBSVIk8AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdd3iW5d0//k9Cwl4yRHFhrRtx46oLaq1WW7WOKqJotdo6q3WAfequaLVOWq2jtY46saj1qYKKtioiriJuHkH2FDCGkZD8/vBLflJzIoSsk7xex+Fx3Lnf1/g0z3NzJ+9c93UWVFZWVgYAAAAA2Sps6AEAAAAAWD0KHgAAAIDMKXgAAAAAMqfgAQAAAMicggcAAAAgcwoeAAAAgMwpeAAAAAAyp+ABAAAAyJyCBwAAACBzCh4AAACAzCl4AAAAADKn4AEAAADInIIHAAAAIHMKHgCA/+fCCy+MzTffPG6++eZaPW7//v1j8803j6FDh9bqcQEAlilq6AEAoCm68MIL47HHHouIiKKionjxxRejc+fOye1HjBgRp512WtXXV111VRx22GHLbdOnT5+YMmVKRESccMIJceGFFyaPt9dee8WMGTNWeJzTTz89zjjjjK/tu3Dhwnj44Ydj+PDh8dFHH0VJSUm0adMmunbtGj169Iidd945dt9999h0000j4styY/To0d/wHaneBx98sFLbbb755l97rmXLltGuXbvo0qVLbLXVVtG7d+/4/ve/Hy1btqzRLHVhwYIFcffdd0dEVPu9BgBYWQoeAGhg5eXl8cQTT8SAAQOS2/z9739fpWPef//9ccIJJ0S3bt1Wc7rlTZw4MU466aT49NNPq55r3bp1lJWVxUcffRQfffRRDB8+PLbYYosYNmxYRER06NAhunTp8rVjlZaWRmlpaRQWFkanTp1qZb62bdtWFTjl5eUxd+7cmDVrVrz33nvx6KOPxpVXXhlnnnlmHHvssVFQUPC1/bt27Robb7xxrLXWWrUyzzLrrrtubLzxxtGuXbvlnl+wYEHccsstEaHgAQBWj4IHABpQ9+7dY+rUqTFs2LBkwTNv3rwYOXJktG7dOpo3bx7z5s37xuMuXrw4/vCHP8Sll15aa7OWl5fHaaedFp9++ml07do1TjvttDjggAOiY8eOVXO+8cYb8fTTT8eECROq9ltWYPy3m2++OW655ZZYd91147nnnquVGS+66KKvXZE0ceLEGDVqVNxzzz3x0UcfxRVXXBHvvvtuXHXVVV/b/9xzz41zzz23Vmb5qmuuuabWjwkA8FXuwQMADWi77baLDTfcMN5999346KOPqt3mH//4R5SVlcX+++8fLVq0+MZj7rXXXhER8eijj8akSZNqbdaXX365asY//vGPcfTRR1eVOxERHTt2jD59+sTVV19d9bGjxmCjjTaKo446Kh577LE48sgjIyJi6NChcf/99zfwZAAAtUfBAwAN7Ec/+lFEpD+GteyjTsu2+yZ9+vSJbbfdNsrKypJXz9TEhx9+GBERXbp0iW222WaF2zam+9wsU1xcHJdccklst912EfFlSbVkyZLltvmmmyzPnz8/fvvb30afPn2iZ8+esffee8dFF10U06ZNi1dffTU233zz6NOnz9f2q+4my/3794++fftWfb355psv999XZ6ioqIihQ4dG//79Y5dddomtt946dt111/jBD34QAwcOjBdffHG1vjcAQP4UPADQwJYVN0888URUVFQsl33yySfx9ttvx7rrrhu77LLLSh/zrLPOqjrm+PHja2/Y+LLkWLx4ca0es740a9YsTj755IiImDlzZrz66qsrve/06dPj0EMPjbvvvjumTJkSzZo1iwULFsQjjzwShx122HL3JVoZHTp0WO5eP126dFnuv9atW1dl5513XgwcODBGjx4d8+bNi1atWkVJSUl8/PHHMXTo0Fot8gCAPCl4AKCBbbDBBrHDDjvEjBkzYtSoUctly67qOfjgg6OwcOXftvfYY4/o3bt3LF26NG666aZambNnz54REVFWVhYXX3xxlJSU1Mpx69see+wRRUVf3oZwzJgxK73feeedF1OmTIkuXbrEbbfdFm+++Wa8+eab8cADD0SHDh1W+T47t9xySzzyyCNVX7/00kvL/ffTn/40IiJee+21ePLJJ6OwsDAGDhwYr7/+eowZMybGjh0b//rXv2Lw4MGx4447rtK5AYA1j4IHABqBQw45JCKW/5hWZWVlPPHEE8vlq+Lss8+OiIinn3463nvvvdWecdddd43evXtHRMRjjz0We+65Z5xyyikxZMiQePHFF2PBggWrfY760KpVq1h//fUjImLy5Mkrtc+oUaNi9OjRUVBQEDfddFPss88+VYXb9ttvH3fcccfXPu5VW956662I+LKYGjBgQLRt2zYiIgoKCmLttdeOQw89NC644II6OTcAkA8FDwA0AgcccEC0aNEihg8fHqWlpRERMXr06JgyZUr07NkzNtlkk1U+5o477hh77rlnVFZWxg033FArc/7xj3+MQw45JAoLC6O0tDRGjhwZN910U5x88smxyy67RP/+/eOFF16olXPVpQ4dOkTElx83WxnDhw+PiIgddtih2qtl1l9//TjwwANrb8CvWFbozJ0792sf4QMAWEbBAwCNQPv27WPfffeN0tLSeOaZZyLi/7+apyZX7yyz7CqekSNHxptvvrnac7Zt2zauvvrqePbZZ2PgwIHxve99L9Zbb72I+PJGwKNHj46f/exnMXjw4NU+V30oKChYqe3efffdiIgVfhRqp512qpWZ/tvuu+8excXFMW7cuOjfv38MGzYsZsyYUSfnAgDypeABgEZiWZEzbNiwWLRoUTzzzDNRXFwcP/jBD2p8zJ49e8Z+++0XEVFrV/FERHTv3j0GDBgQN998czz33HPx/PPPx8CBA6uWTf/zn/8cI0aMqLXz1bZlV+60b99+pbb/7LPPIiKia9euyW3WXnvt1R+sGhtttFFccskl0bJlyxgzZkycf/75sddee0WfPn3i4osvriqfAICmTcEDAI3EnnvuGZ07d45Ro0bFvffeGyUlJbHnnntGp06dVuu4Z511VhQWFsaoUaPilVdeqaVpl7es8HnggQeiVatWERHx6KOP1sm5VtfChQur7r2zwQYbrNQ+lZWVdTnSNzr88MPj2WefjUGDBkXfvn2jY8eOMWXKlHjggQfisMMOi1tvvbVB5wMAGp6CBwAaiaKiojjwwAOjoqKi6mqbZUuor45NN9206v4wtXkVT3U23njjqo8xTZgwoU7PVVMvvfRSlJeXR8TKf6xqWck2a9as5DYrympDly5d4vjjj48//OEPMWrUqHj44Ydjv/32i8rKyrjxxhvj/fffr9PzAwCNm4IHABqRZR/TKisriw4dOkSfPn1q5bhnnnlmFBUVxVtvvRXPP/98rRwzZdkVPMXFxXV6nppYunRp3H777RER0a1bt6pVwb7JVlttFRERr7/+enKbVVlyfZllK3FFrNpVQgUFBdGrV6+48cYbY5111omKiooVzgYArPkUPADQiPTs2TPOOOOMOPHEE2PQoEHRvHnzWjnuRhttVFUe3XjjjTX6yNGHH374jVepzJ49O0aNGhUREVtuueWqD1qHysvL49JLL61advy0005b6e/vd7/73YiIeOONN6q9WfXUqVPjqaeeWuWZlq2QFRHJZeZXtPx6s2bNoqioKCK+LAUBgKZLwQMAjczpp58eF1xwwWqtnlWd0047LYqLi+O9996LmTNnrvL+o0ePjr59+8Z5550Xzz33XMybN68qKykpiccffzyOOeaY+Pzzz6OwsDD69etXm+PX2KRJk+Khhx6KQw89NB588MGI+PKeNkcdddRKH2PXXXeNnXbaKSorK+OMM86IF154oaoke+utt+Kkk06q0RVL7du3r7o589ChQ6vd5vrrr48zzzwzRowYsdz3fPbs2XHFFVfE5MmTo6CgIHbfffdVPj8AsOYoaugBAID60b179zjqqKPi3nvvrdH+RUVFsXjx4nj88cfj8ccfj4iI1q1bR0FBQXzxxRdV27Vo0SIuvfTS6NWrV63MvSquvPLKuO666yLiyyt2SkpKqu63E/FloXL22WevcvlUUFAQv/vd76Jfv34xderU+NnPfhYtW7aMwsLCKC0tjS5dusT5558fv/71r1e56DniiCNiyJAhMXjw4LjppptirbXWioiI4447LgYMGBDl5eXx9NNPx9NPPx0RX171U1lZudz3/Oyzz47NNttslc4LAKxZFDwA0ISceuqp8eijj8bChQtXed+f/OQn0atXr3jxxRfj9ddfj/Hjx8esWbOioqIiOnbsGD169Ihdd901jjjiiFh//fXrYPpvVlJSEiUlJRHxZdHUoUOH6Nq1a2y11Vax6667xv777x8tW7as0bG7d+8eQ4cOjSFDhsSzzz4bs2bNio4dO8ZBBx0Up59+eowdOzYiVn7p9WVOO+20aNWqVTzxxBPx6aefxpQpUyIi4vPPP4+IiAEDBsSGG24Yr7zyStX3fMmSJbHuuuvG9ttvH/369Vvpm0UDAGuugsqGXvcTAGANcMMNN8Qf//jHOPTQQ2Pw4MENPQ4A0MS4Bw8AwGqaN29ePPLIIxER7oUDADQIBQ8AwEp4++234/LLL4+xY8fG4sWLI+LL+/y88sorcdxxx8WsWbNivfXWi/3337+BJwUAmiIf0QIAWAkvv/xynHDCCVVfd+jQIUpLS6uWJ+/YsWPcfvvtDXJzaQAABQ8AwEqYO3duPPjgg/Hyyy/HpEmTYs6cOVFUVBTrrbde7LnnnnHCCSdULXkOAFDfFDwAAAAAmXMPHgAAAIDMKXgAAAAAMqfgAQAAAMicggcAAAAgcwoeAAAAgMwpeAAAAAAyp+ABAAAAyJyCBwAAACBzCh4AAACAzCl4AAAAADKn4AEAAADInIIHAAAAIHMKHgAAAIDMKXgAAAAAMqfgAQAAAMicggcAAAAgcwoeAAAAgMwpeAAAAAAyp+ABAAAAyJyCBwAAACBzCh4AAACAzCl4AAAAADKn4AEAAADInIIHAAAAIHMKHgAAAIDMKXgAAAAAMqfgAQAAAMicggcAAAAgcwoeAAAAgMwpeAAAAAAyp+ABAAAAyJyCBwAAACBzCh4AAACAzCl4AAAAADKn4AEAAADInIIHAAAAIHMKHgAAAIDMKXgAAAAAMqfgAQAAAMicggcAAAAgcwoeAAAAgMwpeAAAAAAyp+ABAAAAyJyCBwAAACBzCh4AAACAzCl4AAAAADKn4AEAAADInIIHAAAAIHMKHgAAAIDMKXgAAAAAMqfgAQAAAMicggcAAAAgcwoeAAAAgMwpeAAAAAAyp+ABAAAAyJyCBwAAACBzCh4AAACAzCl4AAAAADKn4AEAAADInIIHAAAAIHMKHgAAAIDMKXgAAAAAMqfgAQAAAMicggcAAAAgcwoeAAAAgMwpeAAAAAAyp+ABAAAAyJyCBwAAACBzCh4AAACAzCl4AAAAADKn4AEAAADInIIHAAAAIHMKHgAAAIDMKXgAAAAAMqfgAQAAAMicggcAAAAgcwoeAAAAgMwpeAAAAAAyp+ABAAAAyJyCBwAAACBzCh4AAACAzBU19ACsvHnz5sV9990XI0aMiMmTJ0dZWVl069Yt9thjj+jXr19ssskmDT0iNCklJSXx7rvvxjvvvBPvvPNOjBs3LiZOnBiVlZUREfHBBx808ITQ9EycODH+9a9/xWuvvRYffPBBzJgxI8rKyqJDhw6x6aabxt577x0//vGPo3379g09KjQZZWVl8cILL8S4ceNi3LhxMXny5Pjss89iwYIF0bJly1hnnXWiV69e8cMf/jB22223hh4XiIi33347fvKTn0RFRUVERPTu3TvuueeeBp6Kb1JQuew3ERq1N954I84888yYNWtWtXnz5s1j0KBBcfTRR9fzZNB0HXLIIfHee+8lcwUP1K8LL7wwHnvssW/crnPnzjF48ODYa6+96mEqYPr06bH33nuv1Lb77LNPXHfdddG2bds6ngpIWbJkSRx66KHx8ccfVz2n4MmDK3gyMHHixDj11FNj/vz5UVhYGMccc0zst99+0aJFixgzZkzcfvvtMX/+/Lj00kujU6dOsf/++zf0yNAkfLUfb9euXWy55ZbxySefJItYoG5Nnz49IiJat24d3/3ud2OXXXaJHj16ROvWrWPy5Mnx2GOPxXPPPRdz5syJX/ziF3HXXXdF7969G3hqaBo6duwYvXv3jm222SbWW2+96NKlS7Rp0yZmz54d7777bjz00EMxbdq0GDlyZPziF7+Iu+++OwoKChp6bGiSbrnllvj444+jS5cuMXv27IYeh1XgCp4MnHLKKTFy5MiIiLjqqqvisMMOWy7/8MMP48gjj4yFCxdG165d45lnnonWrVs3wKTQtPz1r3+NTp06Rc+ePWOjjTaKgoKC6N+/f4wePToiXMED9e2CCy6IbbbZJg477LDk++Bdd90VV199dUREbLLJJvHUU0/V54jQJC37iEdhYfr2n6WlpXH88cfHf/7zn4iI+NOf/rTSV/0AtWfcuHFx5JFHRkVFRVx77bVxzjnnRIQreHLhJsuN3IcfflhV7vTu3ftr5U5ExGabbRY//elPIyJi1qxZK3V5OrD6jjvuuDjooIOiR48e/soIjcDVV18dxx577Ar/yHHiiSfG1ltvHRER48ePj/fff7++xoMmq7CwcIXlTsSXV94NGDCg6utRo0bV8VTAfysrK4uBAwdGeXl59OvXL7bddtuGHolVpOBp5J5++umqx0cccURyu8MPP7zq8T//+c86nQkAcrbLLrtUPZ4wYULDDQIsp02bNlWPlyxZ0oCTQNN06623xgcffBDdu3ePX/7ylw09DjWg4Gnkln3UI2L5H0j/27rrrhsbbrhhRES8+eabUV5eXuezAUCOysrKqh5/01UFQP154oknqh5/61vfasBJoOl5//3347bbbouIiEsuuWS5wpV8+KmmkRs/fnxERLRt2za6deu2wm2XLZNeVlYWEydOrPPZACBHr776atXjZe+dQP1bunRpzJgxI1588cU45ZRT4sknn4yIL1e6+9GPftTA00HTUV5eHoMGDYqysrI4+OCD3f8qY1bRasSWLFkSc+bMiYgvr9D5Jl/dZtq0aX5oBYD/8swzz8SHH34YERFbbrml90qoZ5MnT46+ffsm865du8Ytt9ximXSoR3fccUeMGzcu1lprrRg0aFBDj8NqUPA0YiUlJVWPV2ZVrK9u88UXX9TJTACQq5kzZ8Zll11W9fV5553XgNMAX1VUVBQ///nPo3///tGhQ4eGHgeajPHjx8eQIUMiImLQoEHRqVOnBp6I1aHgacQWL15c9bi4uPgbt2/evHnV40WLFtXJTACQo0WLFsXpp58es2bNiogvV8HbY489GngqaHq6detWda+dpUuXxpw5c+K1116LBx98MG699dYYP358XHzxxdGxY8cGnhTWfBUVFTFw4MBYsmRJ7LXXXvHDH/6woUdiNSl4GrEWLVpUPf7qDSFTvrraQMuWLetkJgDITVlZWZx11lnx9ttvR0TEd77znTj//PMbeCpomoqLi2OzzTZb7rnvfOc7cfzxx8cJJ5wQTz31VIwdOzb+9re/RdeuXRtoSmga/vznP8fbb78drVu3jksvvbShx6EWuMlyI/bVzx6XlpZ+4/Zf3cZdzwHgyysEzj333Bg5cmRERPTu3TuGDBmyUlfGAvWnU6dOcc0110RExKRJk+J3v/tdA08Ea7YJEybETTfdFBER55xzTnTv3r2BJ6I2uIKnEWvevHl06tQp5s6dG9OmTfvG7b+6zcrclBkA1mQVFRVx/vnnx9NPPx0RETvssEPcdtttrnKFRmrzzTePHj16xIQJE2L48OFRXl4eRUV+XYG6cNddd8WiRYuiY8eO0bFjx/jHP/7xtW3mzp1b9XjOnDlV23Tr1i122mmnepuVledfzEbu29/+dowePTpKSkpixowZK1wqfdmS6sXFxbHRRhvV14gA0Ogsu6/AsmWXt99++7j99ttXatECoOGstdZaMWHChCgtLY25c+fG2muv3dAjwRpp2e095s2bF7/61a++cfvx48fHOeecExERffv2VfA0Uj6i1cjtvPPOVY9fffXV5HbTpk2LTz/9NCK+/CHWXzsAaKoqKyvj17/+dfz973+PiIhtt9027rjjDssuQwZmzJhR9dgtBwBWjRagkfve975XtWzdww8/nLyz+SOPPFL1eP/996+X2QCgsamsrIyLL744Hn300YiI2GabbeLOO+9U7kAG3nzzzZg6dWpERKy//voKHqhDgwcPjsGDB69wm8mTJ0ffvn0j4st72N1zzz31MRqrwRU8jdwWW2wRe+21V0REjB49OoYOHfq1bT766KO48847IyKiS5cuceihh9brjADQWFx++eXx4IMPRkTE1ltvHXfddVe0a9eugaeCpu2JJ56IefPmrXCbSZMmxQUXXFD19WGHHVbXYwGscVzBk4FBgwbFW2+9FQsWLIiLLroo3n333dhvv/2iRYsWMWbMmPjTn/4UCxcujIKCgrjooov8tQPqycSJE+P1119f7rlZs2ZVPf7vQrZLly5VhS1Q+6655pq47777IiJi7bXXjkGDBsX06dNj+vTpyX06d+4cnTt3rq8RoUl66KGH4te//nXss88+0bt379hkk02iffv2UV5eHlOnTo1XXnklHn/88aoVYbfZZps4+eSTG3hqgPwUVFZWVjb0EHyzMWPGxFlnnRWzZ8+uNi8uLo6BAwdGv3796nkyaLqGDh0aAwcOXOntXdoKdatPnz4xZcqUVdrn9NNPjzPOOKOOJgIiIvr37x+jR49eqW0POuiguPTSS32sEhoBH9HKjyt4MrHTTjvFk08+Gffdd1+MGDEiJk+eHGVlZdGtW7fYfffd49hjj41vf/vbDT0mAAAs59prr41XX301Ro8eHe+9917MmTMn5s6dG5WVldG+ffvo0aNHbL/99nHQQQfFFlts0dDjAmTLFTwAAAAAmXOTZQAAAIDMKXgAAAAAMqfgAQAAAMicggcAAAAgcwoeAAAAgMwpeAAAAAAyp+ABAAAAyJyCBwAAACBzCh4AAACAzCl4AAAAADJXVNMdmxV3r805IDtLy6Y29AjV8tqkqfPahMbJaxMaJ69NaJxq8tp0BQ8AAABA5hQ8AAAAAJlT8AAAAABkTsEDAAAAkDkFDwAAAEDmFDwAAAAAmVPwAAAAAGROwQMAAACQOQUPAAAAQOYUPAAAAACZU/AAAAAAZE7BAwAAAJA5BQ8AAABA5hQ8AAAAAJlT8AAAAABkTsEDAAAAkDkFDwAAAEDmFDwAAAAAmVPwAAAAAGROwQMAAACQOQUPAAAAQOYUPAAAAACZU/AAAAAAZE7BAwAAAJA5BQ8AAABA5hQ8AAAAAJlT8AAAAABkrqihB1iTrNu2UzJbq3nbZPZwh7VW+VwbnNgtmd34h7JkNqOwPJkNmfrvZFZZWblygwFZWPDrfZJZ819cnsxu2+HiZHbW9OdWZyQAAGA1uIIHAAAAIHMKHgAAAIDMKXgAAAAAMqfgAQAAAMicggcAAAAgc1bRqkanVu2S2S877pTMfvXyBcmsoHmr1ZppVfxyq2HJrLDnXsnswuNmJLMDPl6SzMbOmbBScwH1a8Fl+yWz4v7nJrPKpenV9ipWayLWVLt03TyZHdFsvXqcJO1nv2iWzJqf9Jtk9mmfnyezx+etncxuWTgumU2Yn36/BSAfJY+nf/8rf+bZZNbt1v8ks8Xl6d+7ctd2Bb8Tn9qldzK7duoLdTHOGskVPAAAAACZU/AAAAAAZE7BAwAAAJA5BQ8AAABA5hQ8AAAAAJlT8AAAAABkrqCysrKyJjs2K+5e27PUq7XbdExmH57w7WTWcuB1dTFOo1f+f28ks35H3pvMXpr/cTKbXTp/tWZqaEvLpjb0CNXK/bXJqnmh827JbOdXL0pmBS3aJLPSc05NZj0en5TMPl9cmszqk9dm3ejRoVsyG3vLj5JZ0T5H18U4jd6S69JL53a4YXQ9TtJ4eG02Lhu075rMhrbeKJltfmqHZNb85N+s8hyVSxYms0W/Tb+OJj2d/jv1i4vXWuU5IiIuK0n/vPvZopJkVra0vEbnayy8Nmtu87XWT2ZvvvqHZLbxDscnsxkln63WTI3ZZmutl8xev3yPZNbu9IfqYpxGryavTVfwAAAAAGROwQMAAACQOQUPAAAAQOYUPAAAAACZU/AAAAAAZE7BAwAAAJC5ooYeoKEc2XGbZNZy4BX1OEkeir61QzJ7cEw6e2i7S5LZcaXPr85I0GQM7L5PMtt55DnJbEVLoZcP/2sy2/TJ9JKMjWUpdOrfW8dvmMya6lLoK9Lsh4elwya6TDo19+2O6eWiLyrePJkddlJ6+e4Wp9fs592KhZ8ns/K3hq/6AYuaJ6NWl9yUzDa7JH3IzVZ9ioiIOGkFWdlTdySzERd8msyOnP9S+piZL69OxAefTU6Hi79IRm/t0CWZrfvimrtM+ooUH3paMuv/20nJ7J6pr9TFONlyBQ8AAABA5hQ8AAAAAJlT8AAAAABkTsEDAAAAkDkFDwAAAEDmFDwAAAAAmVujl0k/pvuuyey3p7Ssx0lq7vFeF1f7/DstCpL7nHdex2RWfOTZqz3Tqjj8+VOT2SffS+938VRLqNO0bNlpg2Q26PKNkllBq3bJrGLq+8ns0l//XzKbu4IlcAGof29et18yK/7+iTU65qIr0j8TjnqwbTK7uNnMZPbqrA9WeY7D1t05md37rx2S2YO7XZfM7m82d5XniIjot7RTMjvszOJkdsDYy5LZ67ulv8+9Pn1z5QYjSyXn/U8ya3v8HsmsxcsTk9ni8iWrNVOumkX6d1+W5woeAAAAgMwpeAAAAAAyp+ABAAAAyJyCBwAAACBzCh4AAACAzCl4AAAAADK3Ri+TPmS/kmTW/KdX1/r5yh4bkszm3PGfGh3zj0XV/5/o+anvJPe5cWCbZNb98jHJbPTFvZNZ8eFnJrMVKVxrnWR2Rt/0MpsX31Oj00Gj9oYiYzMAACAASURBVKN1d0xm9161bTIr6tu/Rue76ZCHktm1016o0TFpuh59oF0yO+a8ehwEmqDbLpqUzL513m+S2ZWFU5LZ67M/Xq2Zakv3wpbJ7I/f+X0yO3fW87U+y/AVZGdfmV46fvLO6Vk2viL93t/8hOp/nl+ytGwFk5CL8a93Smbb33liMlu79VPJbNKCWas1U0MrLV+czJbO+KQeJ1lzuYIHAAAAIHMKHgAAAIDMKXgAAAAAMqfgAQAAAMicggcAAAAgcwoeAAAAgMxlv0x6QUFBOiyq/f7qhp0uTWYfF6SXfbtj6vu1PkvK/EVf1Ch798p1k1mvQ8uTWUGzmv2/UfH390xm5zzbJZn9fqrlnWm8rl+nTzL7+ZiLk1llRUUyq5iZXjZy8e+vTWY3fD4tmcGqOr9kTDI7alz63+VmW+9do/OVj/xbMht5Vvo9dWJx9e9JPx3zPzWaAxqDX01/rqFHqDO3TPlXvZ7v++tsl8yub5N+L97w0V8ls2ZdN0pmv9h5YDKzHPqa7dHiVsls+3qcozGZ/PnsZLZ0+MP1OMmayxU8AAAAAJlT8AAAAABkTsEDAAAAkDkFDwAAAEDmFDwAAAAAmVPwAAAAAGQu+2XS+3bbJpm1uuzmWj/f3Ys/SmbvzZ1U6+erT7vMHJ3MfrPrFcls0GuX1Oh8xfscnczO7XFSMvv91BqdDmpVjw7dqn3+pGu+VevnKhn422TW7ZmPa/18UJ05pQuS2aEDHktmQy/5MJmNHDQ5mf1syTvJbFrJ3HS2z7eTGZCXVsUtktk/2u2YzHr/Zc9kVrTjgcmsYv7MZFY+7K5kttHlLyez2aXzkxlrtpKCpQ09whpjUPv0zyB/8bvhclzBAwAAAJA5BQ8AAABA5hQ8AAAAAJlT8AAAAABkTsEDAAAAkDkFDwAAAEDmsl8mfefCTrV+zKWzJiazRUvLav18Obiv9P1kdsEKvl/Num5UF+NAvVi7TcdkNvbGH1T7fNG+/Wp0roo5k5LZTe+sv4I9LZNOwxs+/e1ktvE5E5JZXSwf3O7SU2v9mDWx9OknG3oEmojWzVsmszvb75rMWlZU1vos77ZI/+14q8UV1T/fZU5yn/UG75/MCnvtm8yWXPc/yeyXp/47mf3v5x8ks8mfz05mUJ2ZFYuTWWVZOuPrut9yTDr87qD6GyQDruABAAAAyJyCBwAAACBzCh4AAACAzCl4AAAAADKn4AEAAADInIIHAAAAIHPZL5M+NWp/ibnyB29PZtNLP6v18+Vg/LxpyWzR5VcmszY3/alG5+vwm/RS0x0Pubba5+ctKqnRuSClY/M2yayob/9aPdcGu52WzOYu/LxWzwX1qaZLoffstFEyG96rOJkVdk3vV9vK/nJVMtvhtvH1NgdN2y+67JLMDr5tt2RWtMP3a32WA2qwT/k7LySzocc+n8yGxJPJ7NVZ6eXOob4MnfZaMls67sVkNqpnx2S28avzktmSpWUrN1gj9drV6f9tuz63fjLr1KpdMmuKP0O7ggcAAAAgcwoeAAAAgMwpeAAAAAAyp+ABAAAAyJyCBwAAACBz2ayi1bl1+2qfH/LQT2r9XC1OvyKZ9bjt2GT23txJtT5LDq4Y1S2ZpdcXWbGibfZNZi2KbqzhUeHr1m/XJZm9fk2fZFZQuOr9eNmD1yezheVLVvl4sCZ75fd9k1ltr2K3IpUlc5NZl0vSK/zkvpoJ+bh2anoVqtsOT6/i06Pt3+pinKS7iqt/v93ipt2T+xzyq9bJ7K/Xt1jtmaChnH1y+v3j5peHJLMde5+ZzF6Z+f5qzdTQ3mmefk3vsfbGyeyITtsms9um/Hu1ZsqRK3gAAAAAMqfgAQAAAMicggcAAAAgcwoeAAAAgMwpeAAAAAAyp+ABAAAAyFw2y6Q3L6x+1KKNt6vnSfhv7yyd39AjQI29fWB6mfSi75+YzCorKqp9vuxv1yX36XbR8GS2sGxxMoM1VfNmxemwddt6m2NFS6GP2/eqZGYpdBq7zxeXJrOxiyfU3yARsXNUf74Wx/wnuc/f2u+WzP4+7IRkVvaXPyezbn95L5l5TVNf7pj6UjK7cdpHyeyfx3dKZh1+t1ojNbirv3g7mZ08f2Y9TpI3V/AAAAAAZE7BAwAAAJA5BQ8AAABA5hQ8AAAAAJlT8AAAAABkTsEDAAAAkLlslkmf8cW8ap///NSTkvu0u/WOuhoHyMj67dJLoTffe7saHbNy8RfVPn/JDZ8n97EUOizv2Y47JrOi3Q6ttzk69zommXndUl++u06vZDauZHIym1Yyty7GqTeLy5cks8PmvpDMdvvxjGQ2/IEByWxO77HJbK+LRiWzN2ePT2ZQX5ZO/qyhR6gzqd/3IyKW/vPeZHbNUeXJ7P4hrZPZ54tLV26wzLiCBwAAACBzCh4AAACAzCl4AAAAADKn4AEAAADInIIHAAAAIHMKHgAAAIDMZbNMekVlRbXP3//aBsl9TqmDOUZf3DuZbXBBetm6eYtK6mCa+rN2m47JbOjg9DK3NTXrkJ8ls5krWEKPpmvD9msns3fO3CqZFR2c/peicgVLz96275Bqn//99PSSrtAU7bH2lsls+zv2rrc5KmZ+ks4qK+ttDki9Xw17LP1+tP+Pqn/Pich/mfSaemXm+8nsuH6PJrO7r03/3Pqvv52QzPoc89dkNnrWh8kMVtX8825IZu36p38XLX7oo2RWtjS9lHjKtzt2T2bfb7NJMutftjiZbXXJt9InbNkqGRXvPyC93wqM/Nu5yWzHaWNqdMzGzhU8AAAAAJlT8AAAAABkTsEDAAAAkDkFDwAAAEDmFDwAAAAAmVPwAAAAAGQum2XSUy5f8HoyO/GpO5JZ8YEn1eh8xYefmcwmNWuWzM767eRkdtfUl2s0S23boH3XZPb6d9LLpBfvd3yNzlexYHYyO2D8omRWaTlbqnFZi62TWfEpv6nRMcsfvjWZ/XL6czU6JqyJ9l9nu2T26J0HJbNmvfrWxTjVuueAe5PZ4vIl9TYHnN+qZ7XPf37R9cl9/j3z47oaZ400dNpryez9M2Yks1dvPSSZjTglvWR092smJbOSJQuTGVTnV//XOZndddDPktlLv/kwmU0rSy9Bvs8p1f9uVXzc2cl9Clq1S2Zldw1OZr+5cmoym1iRfq1ceulZyWzTl29MZn8pbJ3M1lSu4AEAAADInIIHAAAAIHMKHgAAAIDMKXgAAAAAMqfgAQAAAMicggcAAAAgc9kvkz67dH4yO/ni9FJxt7f+WzIr3ufoGs1SfOhpyeyGz9PLxU26cdtVPtes8s+TWZvCFsmsdWHzZDZ08I7JrKZLoa/Iot+ml65+Z+7EWj8f+Ru8bp9kdsT/DqjRMcvuvzaZbf/7d2p0TGhqBpR3TGb1uRR6RET58/dV+/xtFfX7vtK8WXEy691503qcJGJ86fRqn59WMrde52DFFkxt2dAjNAnvzv00md1w1thkdt6YS5PZ0XcNSma3T31p5QaD/+eZee8ls/KXH0tmWz12XDpbwfm+uKT6ZcaH3XFLcp+HitK/gz857Y0VnK1mJnfdLJm9WOtny5sreAAAAAAyp+ABAAAAyJyCBwAAACBzCh4AAACAzCl4AAAAADKn4AEAAADIXPbLpK/IA1NfTWZb/6p1Mjt3ePXLiUZEFK61To1maX7chcnsifSKdknlE/+TzArbr53Oajh/XRgyIj1nxLh6m4PGpUvrDsns9Nt2TmaFnTeo0fmuuvGLZDZ+3rQaHRNoOEX79qv2+eePTC/bOv6xnWp9jg4dFyaz9Z5JLz1bF5ZcP7Da59e95a3kPqVLFtXVOE3eu0Vl1T5//OnphYy7nD8pmc0uTS9XzKq5YlZ6SfOz//VwMrt4g1nJ7PapqzUSTdCKXtNtj7ihHif5uB7PtWKTFs5u6BGy4QoeAAAAgMwpeAAAAAAyp+ABAAAAyJyCBwAAACBzCh4AAACAzCl4AAAAADK3Ri+TviL/M+35ZLbdXkuT2X5jL6+LcVZZ0Ua9GnqEKktnfJLMfv+Du5PZtZ+NrotxyNz/tN8xmTXb/vu1fr51KvTcsLrea16QzH44J728c2HnDepinGq1GHhdMtuq+lXEs1Kxgu9zQbfO1T7fvDD9Y2Dpak9Eyj2zX6/2+Wu2vTK5z1kd0stwX7zwhWRWUVmx8oMRi8uXJLPKDz9IZh1+tlv6oK9+uDojAawSv9kAAAAAZE7BAwAAAJA5BQ8AAABA5hQ8AAAAAJlT8AAAAABkTsEDAAAAkLkmu0z6ihy/8M1k9sl1FySzFudeXRfjNAoVn01PZucefEcyu3Xav+tiHNZgC9OrLUcsLUtnzYrTWfniZPSdggXfPBSwQpdNfT6ZnTeiXTIrPOqXdTFO1pZ+/FoyG3bE/yaz95pXJrMrpo5cjYmobZ8vrn4R+tuPeCK5z7n/PjeZ7b9HSTLbe+5/ktnCsvR7Y1P1z057JrNmPzg6mT2z3911MQ7w/8xZ+HkyK3sq/bvoroubJbObV2uixssVPAAAAACZU/AAAAAAZE7BAwAAAJA5BQ8AAABA5hQ8AAAAAJlT8AAAAABkrqCysjK9ruYKNCvuXtuzZKF185bJ7NQuuySzM7rMTGbr/u+t6RMWJDq4yopV3ycipn3/Z8lsj0/SMy5ZWp7MZpfOT8+yBltaNrWhR6jWmvza/PyB05NZQfMWyeyun7+ZzM6c8dxqzUTj47XZuGzbeeNk9sLRnZNZ8cnpJdQLOzX+72XlCt4bl9x8RTL70cOLktkLM95ZrZkamtdmzd24Tp9kdvK/z0lmS/89NJkNuujjZHbPnDeS2bxF6WXZG4vvrL1VMru3e/pn6G6PD0lmH++ZXqp+t2kfJLOSJQuTWWPhtUnO5g/cM5kten1aMuv2TPrfwMaiJq9NV/AAAAAAZE7BAwAAAJA5BQ8AAABA5hQ8AAAAAJlT8AAAAABkzipaUENWHIDGyWtzzfDL9fZKZt0qipLZL85uVe3zf7ihfley+bwgvVLPlVNH1tscjYnXZt3Yb51tk9l9232RzFqdeGAyK+i2YTL74vL06q+/fS/9vZxXWVbt8ztUVP+ajYg4/FuTklmHXx+dzJptuXsyK3/1yWQ27PRxyezY2c8ns9x5bdLYtShqnsxm33FsMnvo/AnJ7KczG/9KulbRAgAAAGiCFDwAAAAAmVPwAAAAAGROwQMAAACQOQUPAAAAQOYUPAAAAACZs0w61JAlJaFx8tqExslrs3Hp0LJNMhveYatktsUp7ZJZswN+nMwq58+q9vmiXn2T+yy5+7fJ7JM/TE9mf6lIz/iHGaOS2eLyJclsTea1CY2TZdIBAAAAmiAFDwAAAEDmFDwAAAAAmVPwAAAAAGROwQMAAACQOQUPAAAAQOYskw41ZElJaJy8NqFx8tqExslrExony6QDAAAANEEKHgAAAIDMKXgAAAAAMqfgAQAAAMicggcAAAAgcwoeAAAAgMwpeAAAAAAyp+ABAAAAyJyCBwAAACBzCh4AAACAzCl4AAAAADKn4AEAAADInIIHAAAAIHMFlZWVlQ09BAAAAAA15woeAAAAgMwpeAAAAAAyp+ABAAAAyJyCBwAAACBzCh4AAACAzCl4AAAAADKn4AEAAADInIIHAAAAIHMKHgAAAIDMKXgAAAAAMqfgAQAAAMicggcAAAAgcwoeAAAAgMwpeAAAAAAyp+ABAAAAyJyCBwAAACBzCh4AAACAzCl4AAAAADKn4AEAAADInIIHAAAAIHMKHgAAAIDMKXgAAAAAMqfgAQAAAMicggcAAAAgcwoeAAAAgMwpeAAAAAAyp+ABAAAAyJyCBwAAACBzCh4AAACAzCl4AAAAADKn4AEAAADInIIHAAAAIHMKHgAAAIDMKXgAAAAAMqfgAQAAAMicggcAAAAgcwoeAAAAgMwpeAAAAAAyp+ABAAAAyJyCBwAAACBzCh4AAACAzCl4AAAAADKn4AEAAADInIIHAAAAIHMKHgAAAIDMKXgAAAAAMqfgAQAAAMicggcAAAAgcwoeAAAAgMwpeAAAAAAyp+ABAAAAyJyCBwAAACBzCh4AAACAzCl4AAAAADKn4AEAAADInIIHAAAAIHMKHgAAAIDMKXgAAAAAMqfgAQAAAMicggcAAAAgcwoeAAAAgMwpeAAAAAAyp+ABAAAAyJyCBwAAACBzCh4AAACAzCl4AAAAADKn4AEAAADInIIHAAAAIHMKHgAAAIDMKXgAAAAAMqfgAQAAAMicggcAAAAgcwoeAAAAgMwpeAAAAAAyp+ABAAAAyJyCBwAAACBzCh4AAACAzCl4AAAAADKn4AEAAADInIIHAAAAIHMKHgAAAIDMKXgAAAAAMqfgAQAAAMicggcAAAAgcwoeAAAAgMwpeAAAAAAyp+ABAAAAyJyCBwAAACBzCh4AAACAzCl4AAAAADKn4AEAAADInIIHAAAAIHMKHgAAAIDMKXgAAAAAMqfgAQAAAMicggcAAAAgcwoeAAAAgMwpeAAAAAAyp+ABAAAAyJyCBwAAACBzCh4AAACAzCl4AAAAADKn4AEAAADInIIHAAAAIHMKHgAAAIDMKXgAAAAAMqfgAQAAAMicggcAAAAgcwoeAAAAgMwpeAAAAAAyp+ABAAAAyJyCBwAAACBzCh4AAACAzCl4AAAAADKn4AEAAADInIIHAAAAIHMKHgAAAIDMKXgAAAAAMqfgAQAAAMicggcAAAAgcwoeAAAAgMwpeAAAAAAyp+ABAAAAyJyCBwAAACBzCh4AAACAzCl4AAAAADKn4AEAAADInIIHAAAAIHMKHgAAAIDMKXgAAAAAMlfU0APwzfr06RNTpkxZqW2HDBkS3/3ud+t4IuCrFi1aFMOGDYvhw4fHxx9/HHPmzIlWrVpF586dY6uttopdd901fvjDH0aLFi0aelRY41144YXx2GOPrdI+6623Xjz33HN1NBHwVS+++GIMGzYsxo4dGzNnzozy8vJo3759bLrpprHvvvvG4YcfHm3btm3oMaHJKS8vj6eeeiqeeuqpeOedd2LevHnRpk2b2GCDDaJv375x9NFHR8eOHRt6TL6BggdgNYwaNSoGDRr0tRJ2yZIlMX/+/Pi///u/ePLJJ2O33XaL9ddfv4GmBFbkW9/6VkOPAGu80tLSOOecc+L555//WjZnzpyYM2dOjBo1Ku6888646aabYvvtt2+AKaFpmjJlSpxxxhkxbty45Z6fN29ezJs3L8aOHRv33Xdf/P73v4/evXs30JSsjILKysrKhh6CFVt2BU/Pnj3jqquuWuG23bt391cPqCcjR46MM844I5YsWRItW7aMH//4x7HHHntEt27doqSkJKZPnx5vvPFGjBgxIh566CEFD9SDGTNmxPz5879xu8suuyxee+21iIi4/vrr48ADD6zr0aBJO+2002LEiBEREdGhQ4cYMGBA9OrVK9q1axeTJk2Khx9+OEaNGhUREW3bto1//OMfsc466zTkyNAkzJ8/P4488siYMGFCRETsuOOO0a9fv+jRo0d88cUX8dJLL8Vf//rXKC0tjTZt2sT9998fW2yxRcMOTZKCJwPLCp7evXvHPffc09DjAPHlXzoOPvjg+OKLL2LDDTeMu+66KzbYYINqt62oqIjK/6+9O4+3az73B/6cc3LiZKokkohIkJYoUa2iqCGmuqpRUvNFS/mhintv9VKKUkO1SvX302jDpdpLaZpGinSgpS03xkoNoaYYIiKTzImc6fdHXs5L2/3sm+ycaZ3zfv+1rc/+rv3k2OvsvZ+9znqam6OmpqadqwRKWbhwYey1115RX18f/fv3jz//+c/Rs2fPji4LuqwZM2bEuHHjIiJi4MCBMXny5JLNm6uuuipuvPHGiIj4/Oc/H1//+tfbtU7ojq688sq4+eabIyJi7NixcdVVV0V19d9fqvfZZ5+NY489NlauXBk77LBD3H777R1RKmvBRZYBKnDZZZfF8uXLo7a2NsaPH582dyIiqqurNXegE7nrrruivr4+IiIOPvhgzR1oY0888UTL7SOPPDI9M+f000+P2traf1oDtI2mpqaW69b16tUrLrroon9q7kREjB49Oo4//viIiHjyySdj2rRp7Vona0+DB2AdvfHGGy3XEDj44INjq6226uCKgHXxy1/+suX24Ycf3oGVQPewfPnyltubbrpper8+ffq0XMT1vSYs0HZmzpwZixYtioiIHXbYITbccMP0vmPGjGm5PXXq1Davjcq4yDLAOpo6dWq899etY8eObdm+cuXKmDt3btTV1cWgQYOctQOd0LPPPhvPP/98RKz5RtJ1BKDtbbHFFi23y02GXb58ecuHzZEjR7Z1WdDtvf+adRtttFHZ+w4aNKjl9qOPPtpmNbF+NHgK5JVXXomjjjoqXnnllVi5cmX0798/ttxyy9h9993jyCOPLNtxBVrPk08+GRERVVVV8bGPfSymTZsW48ePj8cee6yl8dO3b9/Yc88947TTTvMBEjqRSZMmtdw+7LDDOrAS6D723nvvGDJkSMydOzcmTpwY//qv/xobb7zxP93vhz/8YcuZO8ccc0x7lwndTu/evVtuL126tOx9lyxZ0nL79ddfj1WrVkVdXV2b1UZl/IlWgcyfPz+mT58eS5Ysifr6+pg3b15MmzYtvvvd78Z+++3nVDloJy+++GJErLlQ5G233RYnnnhiPProo/H+a9YvW7Ysfv3rX8fhhx/+dx8ogY6zevXquOeeeyIiomfPnn93Bh7Qdurq6uIHP/hBDBo0KBYsWBCf/exn4/rrr4+HHnoonnrqqZg6dWp88YtfjAkTJkRExJlnnhm77bZbB1cNXd9mm23Wct2r6dOnx+rVq9P7vjd5MmLNtXvmzJnT5vWx7pzBUwA9evSIMWPGxO677x6jRo2K/v37x8qVK+OFF16IO++8M5588slYunRpfOUrX4mmpiZvWKGNvXc669KlS+O73/1u1NbWxmmnnRaHHnpoDBkyJN5+++34xS9+ETfccEPU19fHhRdeGMOHD49ddtmlgyuH7u2+++5r+fOPT33qU858hXa0/fbbx+TJk+O2226LW265Ja699tp/us8ee+wRp5xyitdLaCe9e/eOPfbYI+6///5YtGhRTJgwIc4444x/ut+8efNaJm295/3X1qLzMCa9ABYvXlz2TehNN90U3/72tyNizZ+F3HvvvTFw4MD2Kg+6nW233TYaGxtb/vuaa66Jz3zmM/90v4kTJ8YFF1wQEWuu9fH+C7sC7e+kk06KBx98MCLWvHbuvvvuHVwRdC+33357/OQnP4mXX365ZF5XVxcHHHBAnHPOOTF48OB2rg66p+eeey6OPPLIlrN3jj766DjuuONiiy22iBUrVsRDDz0UV199dcyaNStqa2tb/ozy1ltvjZ122qkjS6cEDZ4u4pxzzokpU6ZERMTZZ58dp5xySgdXBF3XDjvsECtWrGi5ffvtt6f3PeSQQ1ou6HrPPffElltu2S41An9vzpw5sc8++0RTU1MMGzYsfv/735ccBQu0vubm5vja174Wd955Z0SsmcZz0kknxejRo6Nnz57x5ptvxt133x0TJkyI1atXx9ChQ+Omm26KD33oQx1cOXQPv/vd7+Kcc86JlStXpvfZd999o6mpKR544IGIiJgyZYrrTHZC3tl0Eccee2zL7UceeaQDK4Gur0+fPi233z8yspS999675fZf//rXtioJ+F9Mnjw5mpqaIiJi3LhxmjvQju64446W5s7BBx8cEyZMiF122SX69u0bPXv2jJEjR8aZZ54Z1113XUSsaciee+65HVkydCsHHHBATJkyJQ4//PAYMGDA32Uf/OAH46KLLorx48fHggULWrb7M+fOyTV4uoitttqq5bYLXkHbGjZsWMybNy8iIjbZZJOy931//v4XRaB9TZ48OSLWTL/73Oc+18HVQPfy85//vOX22Wefnd5vzJgxseuuu8bDDz8cTz/9dDz//PPOEIB2svnmm8fll18el112WcyfPz+WLVsWAwYMiP79+0dERENDQ8ugkX79+sXQoUM7slwSvr7qIqqqqkreBlrf+xuq778WTynvz2tqatqsJiD32GOPxWuvvRYREbvssksMHz68gyuC7uW9D4WDBg36X78Y2W677Vpuv/LKK21aF/DPqqqqYvDgwTFy5MiW5k5ExNNPPx2rVq2KiIiPfvSjPnN2Uho8XcR7L5wREUOGDOnASqDr+8QnPtFy+/XXXy973/fnvumAjjFp0qSW24cddlgHVgLdU48ea/5ooKGh4X+97/vv8946oOO992eWEVFyuAidgwZPF3Hbbbe13DZaEtrWfvvtFxtssEFErLkoXXat+oaGhrj33nsjYs23ITvvvHO71QissXz58vjtb38bEWtOKT/ggAM6uCLoft47a27RokV/96VkKY8++mjL7REjRrRpXcDamTlzZsuXJYMGDdLg6cQ0eDq5P/7xjy3TejI333xzy7UFevfu7dtJaGN9+/aN4447LiLWnD5+/fXXl7zftddeG2+++WZErLl4nbProP1NnTq15XX0M5/5TNTV1XVwRdD97L///i23L7300nj33XdL3u+nP/1pzJgxIyIiNt10U9ffgXby3vvVUmbPnh2nnXZay3j0iy66qOWLTjofY9I7ueOPPz6ee+652HfffWOnnXaKLbbYIvr16xerVq2KF154Ie688874y1/+EhFrzhC48sor49BDD+3gqqHrW7ZsWRx11FHx0ksvRUTEgQceGOPGjYshQ4bEnDlzYtKkSXHfffdFRMRGG20UkyZN+l+vOwC0vmOOOabldXLixImx/fbbd3BF0P0sWbIkxo4dG2+//XZErJnKc9xxx7WMSZ81a1bcfffdLWfbRUR8//vfjwMPPLCjSoZuZcyYMTFs2LA46KCD4sMf/nD06dMn5s+fH9OmTYs77rgjli9fHhERRQ+T+QAAGapJREFUp5xyStkLpdPxNHg6ueOPP/7vTlXN9O/fP77xjW/EQQcd1A5VARFrJtadfvrp8eyzz6b3GTFiRIwfPz5GjRrVjpUBEWtOKX/vA+KoUaPirrvu6uCKoPt6+eWX46yzzmr5YiRTV1cX559/fhx11FHtVBmw1157tTRgS+ndu3f827/9W5xwwgntVxQV0eDp5J555pl4+OGH46mnnoqZM2fGO++8E4sWLYoePXrEgAEDYuutt44999wzDjnkkOjbt29HlwvdTkNDQ0yePDmmTp0aL774YixatCj69OkTo0aNigMOOCCOPPJIp7FCB7n66qtjwoQJERFx3nnneWMKHWz16tUxderUuPfee+O5556LBQsWRGNjY/Tt2zdGjhwZu+22WxxxxBHOeIV2dt9998WDDz4Y06dPj7lz58aSJUuiX79+MWLEiBgzZkwcdthhhoUUhAYPAAAAQMG5yDIAAABAwWnwAAAAABScBg8AAABAwWnwAAAAABScBg8AAABAwWnwAAAAABScBg8AAABAwWnwAAAAABScBg8AAABAwWnwAAAAABRcj0oX1tQOa806oHAa62d3dAklOTbp7hyb0Dk5NqFzcmxC51TJsekMHgAAAICC0+ABAAAAKDgNHgAAAICC0+ABAAAAKDgNHgAAAICC0+ABAAAAKDgNHgAAAICC0+ABAAAAKDgNHgAAAICC0+ABAAAAKDgNHgAAAICC0+ABAAAAKDgNHgAAAICC69HRBQDQ/qqr8v7+FUP3TrMvX755ms28eHrJ7du//uRa1wUAAFTGGTwAAAAABafBAwAAAFBwGjwAAAAABafBAwAAAFBwGjwAAAAABafBAwAAAFBwxqQDdFE9a2rT7ItDd0mzf3/0G2lWP+X6NNt1znNrVxgAANDqnMEDAAAAUHAaPAAAAAAFp8EDAAAAUHAaPAAAAAAFp8EDAAAAUHAaPAAAAAAFZ0w6QIGNGrBpmj1+5tZp1vPUfBR6w0uP5Y93zu/SbMXqVWkGAAC0LWfwAAAAABScBg8AAABAwWnwAAAAABScBg8AAABAwWnwAAAAABScKVqd3O5Dtim5/WcjmtI1tf0a06zXdv3TrO68q9e+sPep/+V1afbwN2an2bdql6TZH95+uqJaoCvaesDwNHvi5mPSrMcnPptm9fdMSLP+p96WZo1N+e8XAACg4ziDBwAAAKDgNHgAAAAACk6DBwAAAKDgNHgAAAAACk6DBwAAAKDgNHgAAAAACs6Y9HYysFe/NPvL1pum2eBbLii5vXrgsIrqqKrOe3rNTfno9XJ6HHp6mu1xaL7u7mUL02zZf5ybZkN+89Ja1QWdTe+edWn2rYGfTLOTfjkuzaqHjEyzVVf8R5rtcdu8NDMKHYBK9KrdIM2G9h6wzvv7z7pt0+zExy9c5/2tj6qa/GPTqiu/kmZH/Hx1mt0756/rVRO0pUG9N0yzHT6wRZpdP6AhzQYfOjDNao8/K81mH3Zxmh369rI0e2bha2nWVTmDBwAAAKDgNHgAAAAACk6DBwAAAKDgNHgAAAAACk6DBwAAAKDgNHgAAAAACq6qubm5uZKFNbWVjenuyrYbuHmaPXzbiWlWM3rMOj/Wu1edk2bNy9/NF1ZX5VlTRU+FitUels9Qr95ypzSb87nSI9S3ef7ldM27DfmIyko11s9u9X22Bsdm5zVz+23SbJNf/zDNmuvzY/rbn/xWml08+/61K6yLcWxW7oJhe6fZaZvlP9cr3tg4zV5tzMeXlvP0illptlndoDTbsLquoserxKmr+6XZPmf3TrMld72SZge8sCrNZix8fe0K66Qcm13DtgM3S7Npp+bvhXt+6eJ1f7CqMt9FNzet+/7WR4W1NL7wSJrtfOT1Jbc/t/CNtS6rNTg2u7eLhu1Tcvt/XjEyXdNjv+PzHbbzsVl/y7fT7IPfyo+/+SsWt0U5raqSY9MZPAAAAAAFp8EDAAAAUHAaPAAAAAAFp8EDAAAAUHAaPAAAAAAFp8EDAAAAUHA9OrqAoulVu0GaTTv/o2nW4yOlx89FRDQ35aPklp91asntQ+58KV3T1N5jIyvU/9Z8TOz3+u2cZp87YUDJ7RvOzEfSzm2DMel0bwcN3aHk9ls/3ZCuqbvk2jSrv/eWNDvz/OfT7Mez/yfNYF3t/27+u3KjiT9Ks6vLve5UOlr4mT+mWfVmo/OH23DIOj9WxaOYK1zXf58Z+bqxV+UZtJNPDB6VZlO2aU6zikaht7NyY8sv/fzv0uzCG/ZIs5oy7/NrRu2SZrfUTi65/RPRvmPS6Rp61tSm2VuHfjDN6i46ueT26gFD17umf7TyvDPTrHFxfZr1Hf/DNKv9wrlp9l/fuyTNDlnxpzQrMmfwAAAAABScBg8AAABAwWnwAAAAABScBg8AAABAwWnwAAAAABScBg8AAABAwRmTvo5e3HHzNOtx1H9UtM9lXzolzbZ/YEHJ7UUZhV7OolXL0uzEVffnmQmytKLqMmOOfztgtzTb49FvltxeVZP/Wn3r06el2Z4z56XZrKXz0wxaU1Xk449v/3jp53xExI1Vc9Ls5ObWH7Ma8ec22GdpY3eZlWb9fnRjmpV7mZ79xXzk/IyFr69VXbC+LhmWj/Y++xefS7Oa4du2RTntpvn56Wn2w4WPpdl5v12eZuXGpJezuL5nRevo2sqNOz984x3T7IZJx6dZ9aYfXuc6mpfk7z8f3uv7aXZJ7ZI0+/Pc59Ns034bpdnfli9Ks6o+/dNsw6qGNOuqnMEDAAAAUHAaPAAAAAAFp8EDAAAAUHAaPAAAAAAFp8EDAAAAUHAaPAAAAAAFZ0z6Ohrw/XwUelV13i9b/IWT0mz0/+TjkeevWLx2hQGpTfoOTLO7+30wzUY/fkWaNb49s+T2GQddl645elU+4tEodDqDj1+6eZq98o230+yhuc/lWeRZESz+8G5p1tyUz0Jv+NMdabbvm/lYeWhNe208Os2+OvXENKveaES+0+b8eV8ENfsekWa/7vdavm6rzVq9lg9/KHntX9jqD0WBnL3x7ml20cMXVrTPciPPl593Qcntxz/WJ13zm4XTK6qjnLfLjEL/2ZgfpNncmuY0u3TpX9arpiJyBg8AAABAwWnwAAAAABScBg8AAABAwWnwAAAAABScBg8AAABAwWnwAAAAABScMekl/Nume6VZ9ZCRaVZuXGp7jkIf0qd/mvWszv+XN0U+Ym720gXrVRO0tY37Dkizl+7/TprVDNsqzbJR6BERJ4wdX3L7xDmPp2s6k+H9BqXZxhvkv0OemP9SW5RDAey+Uf46FmWiItiwLh8FW/3xHdKsqjr/nuxHZ7+QZrOW5uNqYV31qt0gze65Zs80KzcKvaomf7/Y3NiwdoWtpcbXnsrDd+ak0YEnTUmzB+fOSLOfDN4nzY74y5VpVunPpGHqjWn2yb+Zh95d/feg/Hk4btLhFe2z4YGfpdl1576cZue99WJFj1eJcp+zL/vqRmlWvc+4NPv0v3w7zVasXrV2hXUhzuABAAAAKDgNHgAAAICC0+ABAAAAKDgNHgAAAICC0+ABAAAAKDgNHgAAAICC67Zj0suNlCw3oi165OvKqXQU+of6b1Jy++11m6Zrtp1yUprVDN82zZqW5zXO+dy5abbN8/nYvXcbVqcZlFJbZgzpSUN3TbPv3rBvmlUP3izNVt96VZqddm0+ynji7EfTrBLbDsxrvDxGptnW/d9JsxFfLzPeebN8PHz10A+l2auHfCvNtn2lzKhbOo3dh2yTZjV75WNIH/rG221RTqewe/9RadZjj8PSbOX5X06zS955Zb1qgrU1rM/ANKsaMCRf2NyUR+VGoZdZV07D/0wuuX3Uybela95aVtkY8Y8N+mCajfvBR/KFFf5MGp++P82+cPELafbGknl5LXRpB59Un2bVm344zcqNQt/2jClpVu65lr337lNbl64Z/YERafbbSaekWc2wrdMsynw+L2dgda+K1nVVzuABAAAAKDgNHgAAAICC0+ABAAAAKDgNHgAAAICC0+ABAAAAKDgNHgAAAICC67Zj0jeq65dmPQ4/q6J9vnvFV9LshiH5COdjJuyYZlVblB7lWL1RPpquUlW98p/JJr8en2a/3+G8NNtj/sPrVRPdz6u75CPBN5p4eZo1/G1amp31yUvS7IbZD61dYWup3GjWXwzOxzhuenc+rr2qrk+aNbyWjyZv+OlNaRZPPptGdeddnWbDPz843+fFeUQxNL2ePy+6sjuu3T0Pq/Lvwu6cmh8PS999Zn1KgrX28qK30uzWEx5Ms+P//PE0q+q94XrVVMolX3m65PZyo9B7lRmb/IUhO6fZVV8fnmY9dj0kzcpZdVH++WCnyfPTrNz/H7qv2uO+XNG6qr75sfnI9v3LrMyzPruVfi3reealZQopc55Ic1OZOsosW70yzep/dEWa/WX56xU9XlflDB4AAACAgtPgAQAAACg4DR4AAACAgtPgAQAAACg4DR4AAACAgqtqbm5urmRhTe2w1q6lXfWsqU2z+d/cL81qP39uRY9XVZ330pqb1v1K4w2/vC7Nlk3Kp+qU84Hv5dOwqofkk40aX3oszbY7NJ/G8+rit9eusE6qsX52R5dQUhGOzZOH5RNr/t8jl6VZw70/SbMDvpZP0Zo29/m1K+wfbDdw8zQ7s8eHSm4/9tZ8Yl5VuePoN7el2e+/vSzNTl71ZJotWLEkzTbpOzDNnj9zdJrVnnB2mn1nn++X3H7x7PvTNW3BsVm54f0GVbRu1tJ8gkwRrHyjzHO0zDSQ23fMf1+dOK99n/dF4NjsXPYYsm2a3fvED/KFFU7IaZrzcsntEz/zs3TNQTu+kWb9JvyoojoaH5+aZvf8nyfS7KgFD1T0eEXg2Gx/S3/8xTTrse+x7VhJhdpgilb97d9Lsw+cc3dF+yy6So5NZ/AAAAAAFJwGDwAAAEDBafAAAAAAFJwGDwAAAEDBafAAAAAAFJwGDwAAAEDB9ejoAjrK6sb6NDvtB4vS7L+OXplmVXV90qxx/qw0W3jyN9PsX14u/XgzFr6erqnU0penp1nN0NIjoSMieozaJc127pOPmS76mHTKKzcK/fu/Pj3NVnz1S2k2YvKr+brVq9aqrn909LD8+ftfd+e1VA8u/dxeeuop6ZqDHsvreHzei3lYoXIjr6fvn49J3+CMfPTzLz56cZpdPN9Y6KIr+rjzco4dtmuaVVXn33eVm/Z6Y9Wc9SkJOtSDc2ek2dzP5q9/Q6aUGaFeRnXyXvKoJy7IF5UZxdz4av6+dcq4X6XZWSvyUegLVizJa4FWtOUZk9Nsj35Pp9m3Nsw/i/7mnSFpduCAuWl28ZK+JbePjLp0zQWPfCPNymn4w61ptu2lD1e0T/6eM3gAAAAACk6DBwAAAKDgNHgAAAAACk6DBwAAAKDgNHgAAAAACk6DBwAAAKDguu2Y9HJum52PaPvbLmelWY+qmjRb2piPtGuLkecVacpnwTaXyZpm5WM2n1tlFHp3dfXxVWlWM2h4mn3zwXzE44rVz1dUy14bj06zG2/6dJrVbDwyzf7wkdJjXQ9a2Prjzmuq898tp2+Sj6O/4qIRadbjoJPT7N2rz02z81a/nGbQme1f3yvNyr3GrTz/jDR7aO5z61UTdFZ7v7IozWaUGV3e2qpq8o8qyy/Nx7WfvPiNNFtZ/+561QSt4e1l76TZpGWP5dlblT5gHm1Y16fk9tdOHJWuKXdsNrz4SJpde97MNJu1dH6asfacwQMAAABQcBo8AAAAAAWnwQMAAABQcBo8AAAAAAWnwQMAAABQcBo8AAAAAAVnTPo6emL+Sx1dwnrZfcg2aVa9zW4V7bNh0n+n2TMLX6tonxTHEcM+UXJ7zy9fkq75n4+cl2b/951pFdWx/9Dt02zyJR/JF/bqm0ZH7PSVNLvnnSfXqq61dfAmO6bZTz/bmGZ1F12aZg2P3Z1mf/rohWn2LwsfTjMoqrG7zkqzqur8+66b7t24zF6fXY+KoGPttfHoNPt6/YB8YXNTGjUtfDPNqnr2Kr2978D8oRob0qzPhV9Os9FPjk+zx+e9mGbQVfWvy9/v3tVnu5LbNzjv8nRN0+K5afaFo+9Is0lv5SPgaR3O4AEAAAAoOA0eAAAAgILT4AEAAAAoOA0eAAAAgILT4AEAAAAoOA0eAAAAgIIzJr2b+c2FpcfgRURUDxxW0T5/eUtdpeXQBezfUHrsYlVN/uvl7eqeaVYVVfmDlYkubcyfh7VjT0mz03bOR7b/du5Tabb5B4aU3H5F7YfTNWPP759mPcadlmZRU5tGq//7O2m205XT0+yFd/JRttAV1X16xzRrbsrHPt8XC9uiHGg1w/ptlGZP7Ztnvb+Tj0Cu6r1hmq3+4cVpdtR/LU6zcU2lx6Ef/+ezKqqjZvPt02zHnhun2eNhTDrdz5+GbJVmWz2Y/y7IXL3vdWlmFHrHcgYPAAAAQMFp8AAAAAAUnAYPAAAAQMFp8AAAAAAUnAYPAAAAQMFp8AAAAAAUnDHpXdBdA/dKs9rPnZFm5cbEvnNMPmb61IWvrlVddE1Xvjuj5PbjXn8mXTPur5ek2eKfX5tmDdPz0aa9rrgqzcr5/hdq0uyHZ9xX0T4zjWV+Ji/slY9rv6Ah78Xf9dYT61UTdDUHDd2h5PYeB5+arqmq9n0XxfWFfh9Js7r/HJdmVb36pVnDXT9Ks5NvXplmv5kzPc+S7bt96p10zVYPXZNm5Ry1Mn9Pm//LoNjOHbZ3mm35h6+u8/4aZ5V+jx8RceFb96/z/mgf3tEAAAAAFJwGDwAAAEDBafAAAAAAFJwGDwAAAEDBafAAAAAAFJwGDwAAAEDBGZNeUMcO2zXNPvXYpfnCqryn17RwVprtP+PdNFvdWJ8/Hl3ezMVzSm6/Z+zEdM2B50xNs+o9D0yzukPzMceV2uCMy9KsafHcNHtmn++U3P61WJWueWHFW2k2a+n8NANaQXM+NrlMBJ3CXhuPTrML7//3NCs7Cn3y+DQbfm421Dxi8arlaVaJWcv6ptlWFe7zjl6+w6b7ufCOQ9KsqmevNGv41Y9Kbh/x1fz9Op2X334AAAAABafBAwAAAFBwGjwAAAAABafBAwAAAFBwGjwAAAAABafBAwAAAFBwxqR3Av026J1mv+jz8ZLb95h8bLqmuanMvNfV+WjLifvfmGYzFr6e7xNKOHLBA3l4bh6N3PCJNPt033xg6jWP5uPOK9VcZnT51JrSY13/MPuxVq8DaAVV+XdaVdW+76Jzm7Tz6jSr6r1hmtX/+FtpNuLyh9KstUehl/PJf12ZZlU1+UeV5saGtigHOtzwfoPS7OmjN02z6k3y98nlvHrVCyW3L1q1rKL90bG8owEAAAAoOA0eAAAAgILT4AEAAAAoOA0eAAAAgILT4AEAAAAoOA0eAAAAgIIzJr2EQzfZKc1GV5UejRwRcfnsB9Ls5sH7pNnh1+Qj7XrsfUyaVeLFfb6eZifMe7JVHwsqMXPxnDQbXy7bdExblAN0Fc1NaVT/x4lp9tA7pcfHQmvrWVObZrWb9csXlnluP/K9fNx5uVHo5WrZa/A2eS1lXNevufRjnX5uuqbsKPQy/24osmt6bpdmdRdeWNE+l5x8aprt9ObrFe2TzskZPAAAAAAFp8EDAAAAUHAaPAAAAAAFp8EDAAAAUHAaPAAAAAAFp8EDAAAAUHDGpJewdZlR6F//3ZfS7PylR6RZzWb5uLvmpnUf89j48JQ0e/DUJ9LswIVGoQPQdU3YIhn9XJV/p/Xtr85Is6XvrljfkmCt1FTnz9GqDftUtM/dfv6ZNFv829JjyyMiqvrlj1d7wtcqqqW1NS18M81ea1zWjpXAuvvoRiPT7KAbdmz1x+u15wfT7KlX60pu/+SsfHz6ghVL1rsm2oYzeAAAAAAKToMHAAAAoOA0eAAAAAAKToMHAAAAoOA0eAAAAAAKzhStEv7S9E4e1uQ/surh21b0eA2/+3GaPf2150tuP6FhVrrmhXfyqQIA0JX1v+bM0kFzPrHystkPtEktsC56lnmPOf9XC9Js4/0eSbOaUbtUlJWbOlfuWGptt+x4aZrd2PxGmj0+78W2KAdazc0bDEyzmp0OavXHqxoyOM3GL19VcvuCFc+0eh20PWfwAAAAABScBg8AAABAwWnwAAAAABScBg8AAABAwWnwAAAAABScBg8AAABAwRmTXsK9c/6aZr23OawdKwEA/tGID+TjXnts8bGS25sbG9qqHGgVi1ctT7MPPTsjzT513B1pdm2vPBv5q6+lWVX/oWnW+NpTafba53+cZpmjlyxOs6cXvLrO+4Puas7YM9JsxxdnpdnClUvbohw6iDN4AAAAAApOgwcAAACg4DR4AAAAAApOgwcAAACg4DR4AAAAAApOgwcAAACg4IxJBwAKpbm5Oc+ScegNf8rHRUOR3Tvnr2k2utzCjxzT6rUAa+fjs5/Iw832a79C6HKcwQMAAABQcBo8AAAAAAWnwQMAAABQcBo8AAAAAAWnwQMAAABQcBo8AAAAAAVnTDoAUCizls5Ps14j9mnHSgAAOg9n8AAAAAAUnAYPAAAAQMFp8AAAAAAUnAYPAAAAQMFp8AAAAAAUnAYPAAAAQMFVNTc3N3d0EQAAAABUzhk8AAAAAAWnwQMAAABQcBo8AAAAAAWnwQMAAABQcBo8AAAAAAWnwQMAAABQcBo8AAAAAAWnwQMAAABQcBo8AAAAAAWnwQMAAABQcBo8AAAAAAWnwQMAAABQcBo8AAAAAAX3/wFjgnJiNdhHmwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x432 with 10 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 357,
       "width": 572
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "label_index = []\n",
    "for i in range(10):\n",
    "  label_index.append(train_raw.index[train_raw['label'] == i][0])\n",
    "\n",
    "fig = plt.figure(figsize=(10,6))\n",
    "\n",
    "idx = 1\n",
    "\n",
    "for i in label_index:\n",
    "  plt.subplot(2, 5, idx)\n",
    "  plt.imshow(train_raw.iloc[i,1:].values.reshape(28, 28))\n",
    "  plt.axis('off')\n",
    "  plt.suptitle('MNIST Digits')\n",
    "  plt.title(str(train_raw['label'].loc[i]))\n",
    "  idx = idx+1\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Data Preparation for Catalyst\n",
    "\n",
    "We start splitting the train dataframe in train and validations sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "_kg_hide-input": true,
    "id": "zToQ4BwoyEMX"
   },
   "outputs": [],
   "source": [
    "x_train, x_val, y_train, y_val = train_test_split(train_raw.iloc[:, 1:], \n",
    "                                                  train_raw['label'], \n",
    "                                                  test_size=0.3, \n",
    "                                                  random_state=42)\n",
    "\n",
    "x_train.reset_index(drop=True, inplace=True)\n",
    "y_train.reset_index(drop=True, inplace=True)\n",
    "x_val.reset_index(drop=True, inplace=True)\n",
    "y_val.reset_index(drop=True, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we have a Pandas Dataframe with pixel values, it's needed to transform in tensors, they will be input of the Network. <br>\n",
    "The pixels are normalized with the transform.\n",
    "Let's just write how the transform will work."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "_kg_hide-input": true,
    "id": "N8szhSG8K876"
   },
   "outputs": [],
   "source": [
    "data_transform = transforms.Compose([transforms.ToTensor(),transforms.Normalize((0.5,), (0.5,))])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the Catalyst, we have to write a class called dataset to prepare the data for the network. <br>\n",
    "On the Dataset class is defined the input, label and any process, like tranformations, in the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "_kg_hide-input": true,
    "id": "Vr1Yz5XAC132"
   },
   "outputs": [],
   "source": [
    "class MnistDataset(Dataset):\n",
    "# Get input, labels and transforms\n",
    "  def __init__(self, images, labels=None, transform=None):\n",
    "    self.x = images\n",
    "    self.y = labels\n",
    "    self.transform = transform\n",
    "# Get data length\n",
    "  def __len__(self):\n",
    "    return len(self.x)\n",
    "# Get data index\n",
    "  def __getitem__(self, i):\n",
    "    data = np.array(self.x.iloc[i, :], dtype='uint8').reshape([28, 28, 1])\n",
    "\n",
    "    if self.transform is not None:\n",
    "      data = self.transform(data)\n",
    "    if self.y is not None:\n",
    "      target = self.y.iloc[i]\n",
    "      return data, target\n",
    "    else:\n",
    "      return data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, we define the parameters of the experiment, like Batch Size, Iterations, etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "_kg_hide-input": false,
    "id": "yRi8KADSnXjN"
   },
   "outputs": [],
   "source": [
    "num_class = 10\n",
    "batch_size = 32\n",
    "num_workers = 4\n",
    "n_iters = 5000\n",
    "n_epochs = int(n_iters/(len(x_train)/batch_size))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's a personal choice, I create fake labels for the test set for avoid any error on Catalyst."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "_kg_hide-input": false,
    "id": "75Hbwo-eVTC5"
   },
   "outputs": [],
   "source": [
    "fake_labels = np.zeros(test_raw.shape[0])\n",
    "fake_labels = pd.Series(fake_labels)\n",
    "fake_labels = fake_labels.astype(int)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Time to convert the data on Catalyst Dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "_kg_hide-input": false,
    "id": "YHOkWYXNMEn4"
   },
   "outputs": [],
   "source": [
    "train = MnistDataset(x_train, y_train, transform=data_transform)\n",
    "val = MnistDataset(x_val, y_val, transform=data_transform)\n",
    "test = MnistDataset(test_raw, fake_labels, transform=data_transform)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And get these datasets and put they on Loaders. <br>\n",
    "Loaders in Catalyst provide parameters like batch size together the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "_kg_hide-input": true,
    "id": "qERBENjSy_aO"
   },
   "outputs": [],
   "source": [
    "train_loader = DataLoader(train, batch_size=batch_size, shuffle=False, num_workers=num_workers)\n",
    "val_loader = DataLoader(val, batch_size=batch_size, shuffle=False, num_workers=num_workers)\n",
    "test_loader = DataLoader(test, batch_size=batch_size, shuffle=False, num_workers=num_workers)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Put the train and validation loaders in dictionares to feed the network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "_kg_hide-input": true,
    "id": "MBCs3SqJNVwe"
   },
   "outputs": [],
   "source": [
    "loaders = collections.OrderedDict()\n",
    "loaders[\"train\"] = train_loader\n",
    "loaders[\"valid\"] = val_loader"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have the advantage of use GPU, let's check for any available and save it for use on Catalyst."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "_kg_hide-input": true,
    "id": "SAlSzLgI8CR2",
    "outputId": "04b70d0b-e60a-4f82-b80e-2f3665a1c507"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cuda\n"
     ]
    }
   ],
   "source": [
    "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n",
    "print(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. Model\n",
    "\n",
    "Time to build the model! This is a nice part on Catalyst, it's modular, is easy to change any part like the network. <br>\n",
    "If you see on this competition, you will find a lot of interesting model networks, I think that the model that is provided [here](https://www.kaggle.com/juiyangchang/cnn-with-pytorch-0-995-accuracy) is good enough."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "id": "JE3YsTaV9htk"
   },
   "outputs": [],
   "source": [
    "class Net(nn.Module):    \n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "          \n",
    "        self.features = nn.Sequential(\n",
    "            nn.Conv2d(1, 32, kernel_size=3, stride=1, padding=1),\n",
    "            nn.BatchNorm2d(32),\n",
    "            nn.ReLU(inplace=True),\n",
    "            nn.Conv2d(32, 32, kernel_size=3, stride=1, padding=1),\n",
    "            nn.BatchNorm2d(32),\n",
    "            nn.ReLU(inplace=True),\n",
    "            nn.MaxPool2d(kernel_size=2, stride=2),\n",
    "            nn.Conv2d(32, 64, kernel_size=3, padding=1),\n",
    "            nn.BatchNorm2d(64),\n",
    "            nn.ReLU(inplace=True),\n",
    "            nn.Conv2d(64, 64, kernel_size=3, padding=1),\n",
    "            nn.BatchNorm2d(64),\n",
    "            nn.ReLU(inplace=True),\n",
    "            nn.MaxPool2d(kernel_size=2, stride=2)\n",
    "        )\n",
    "          \n",
    "        self.classifier = nn.Sequential(\n",
    "            nn.Dropout(p = 0.5),\n",
    "            nn.Linear(64 * 7 * 7, 512),\n",
    "            nn.BatchNorm1d(512),\n",
    "            nn.ReLU(inplace=True),\n",
    "            nn.Dropout(p = 0.5),\n",
    "            nn.Linear(512, 512),\n",
    "            nn.BatchNorm1d(512),\n",
    "            nn.ReLU(inplace=True),\n",
    "            nn.Dropout(p = 0.5),\n",
    "            nn.Linear(512, 10),\n",
    "        )\n",
    "          \n",
    "        for m in self.features.children():\n",
    "            if isinstance(m, nn.Conv2d):\n",
    "                n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels\n",
    "                m.weight.data.normal_(0, math.sqrt(2. / n))\n",
    "            elif isinstance(m, nn.BatchNorm2d):\n",
    "                m.weight.data.fill_(1)\n",
    "                m.bias.data.zero_()\n",
    "        \n",
    "        for m in self.classifier.children():\n",
    "            if isinstance(m, nn.Linear):\n",
    "                nn.init.xavier_uniform(m.weight)\n",
    "            elif isinstance(m, nn.BatchNorm1d):\n",
    "                m.weight.data.fill_(1)\n",
    "                m.bias.data.zero_()\n",
    "                \n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.features(x)\n",
    "        x = x.view(x.size(0), -1)\n",
    "        x = self.classifier(x)\n",
    "        \n",
    "        return x    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we assemble each component created before and run the expirement on Catalyst. <br>\n",
    "Let's start training the network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "_kg_hide-input": false,
    "_kg_hide-output": true,
    "id": "ANJTmLtc_Yvg",
    "outputId": "9d67fe03-37be-4d1c-d2b5-148a5b6bcb43"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/lib/python3.6/site-packages/ipykernel_launcher.py:45: UserWarning:\n",
      "\n",
      "nn.init.xavier_uniform is now deprecated in favor of nn.init.xavier_uniform_.\n",
      "\n",
      "/opt/conda/lib/python3.6/site-packages/ipykernel_launcher.py:45: UserWarning:\n",
      "\n",
      "nn.init.xavier_uniform is now deprecated in favor of nn.init.xavier_uniform_.\n",
      "\n",
      "/opt/conda/lib/python3.6/site-packages/ipykernel_launcher.py:45: UserWarning:\n",
      "\n",
      "nn.init.xavier_uniform is now deprecated in favor of nn.init.xavier_uniform_.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Selected optimization level O1:  Insert automatic casts around Pytorch functions and Tensor methods.\n",
      "\n",
      "Defaults for this optimization level are:\n",
      "enabled                : True\n",
      "opt_level              : O1\n",
      "cast_model_type        : None\n",
      "patch_torch_functions  : True\n",
      "keep_batchnorm_fp32    : None\n",
      "master_weights         : None\n",
      "loss_scale             : dynamic\n",
      "Processing user overrides (additional kwargs that are not None)...\n",
      "After processing overrides, optimization options are:\n",
      "enabled                : True\n",
      "opt_level              : O1\n",
      "cast_model_type        : None\n",
      "patch_torch_functions  : True\n",
      "keep_batchnorm_fp32    : None\n",
      "master_weights         : None\n",
      "loss_scale             : dynamic\n",
      "Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 32768.0\n",
      "Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 16384.0\n",
      "[2020-04-18 14:15:45,662] \n",
      "1/5 * Epoch 1 (_base): lr=0.0010 | momentum=0.9000\n",
      "1/5 * Epoch 1 (train): _timer/_fps=1373.2228 | _timer/batch_time=0.0257 | _timer/data_time=0.0183 | _timer/model_time=0.0074 | accuracy01=0.9084 | accuracy03=0.9759 | accuracy05=0.9900 | loss=0.2912\n",
      "1/5 * Epoch 1 (valid): _timer/_fps=2256.1089 | _timer/batch_time=0.0177 | _timer/data_time=0.0121 | _timer/model_time=0.0055 | accuracy01=0.9827 | accuracy03=0.9990 | accuracy05=0.9999 | loss=0.0518\n",
      "[2020-04-18 14:16:16,595] \n",
      "2/5 * Epoch 2 (_base): lr=0.0010 | momentum=0.9000\n",
      "2/5 * Epoch 2 (train): _timer/_fps=1390.3130 | _timer/batch_time=0.0244 | _timer/data_time=0.0180 | _timer/model_time=0.0064 | accuracy01=0.9655 | accuracy03=0.9964 | accuracy05=0.9993 | loss=0.1130\n",
      "2/5 * Epoch 2 (valid): _timer/_fps=2013.1495 | _timer/batch_time=0.0210 | _timer/data_time=0.0138 | _timer/model_time=0.0071 | accuracy01=0.9869 | accuracy03=0.9992 | accuracy05=0.9998 | loss=0.0439\n",
      "[2020-04-18 14:16:47,020] \n",
      "3/5 * Epoch 3 (_base): lr=0.0010 | momentum=0.9000\n",
      "3/5 * Epoch 3 (train): _timer/_fps=1362.4998 | _timer/batch_time=0.0248 | _timer/data_time=0.0183 | _timer/model_time=0.0065 | accuracy01=0.9719 | accuracy03=0.9974 | accuracy05=0.9997 | loss=0.0935\n",
      "3/5 * Epoch 3 (valid): _timer/_fps=2224.7190 | _timer/batch_time=0.0186 | _timer/data_time=0.0127 | _timer/model_time=0.0059 | accuracy01=0.9893 | accuracy03=0.9994 | accuracy05=0.9998 | loss=0.0327\n",
      "[2020-04-18 14:17:18,163] \n",
      "4/5 * Epoch 4 (_base): lr=0.0010 | momentum=0.9000\n",
      "4/5 * Epoch 4 (train): _timer/_fps=1367.3403 | _timer/batch_time=0.0249 | _timer/data_time=0.0184 | _timer/model_time=0.0065 | accuracy01=0.9770 | accuracy03=0.9983 | accuracy05=0.9998 | loss=0.0746\n",
      "4/5 * Epoch 4 (valid): _timer/_fps=2057.7356 | _timer/batch_time=0.0201 | _timer/data_time=0.0133 | _timer/model_time=0.0068 | accuracy01=0.9905 | accuracy03=0.9997 | accuracy05=1.0000 | loss=0.0304\n",
      "[2020-04-18 14:17:48,184] \n",
      "5/5 * Epoch 5 (_base): lr=0.0010 | momentum=0.9000\n",
      "5/5 * Epoch 5 (train): _timer/_fps=1381.2889 | _timer/batch_time=0.0245 | _timer/data_time=0.0181 | _timer/model_time=0.0064 | accuracy01=0.9796 | accuracy03=0.9986 | accuracy05=0.9998 | loss=0.0677\n",
      "5/5 * Epoch 5 (valid): _timer/_fps=2241.2340 | _timer/batch_time=0.0182 | _timer/data_time=0.0123 | _timer/model_time=0.0059 | accuracy01=0.9901 | accuracy03=0.9995 | accuracy05=1.0000 | loss=0.0306\n",
      "Top best models:\n",
      "logs/mnist/checkpoints/train.4.pth\t0.0304\n",
      "CPU times: user 1min 40s, sys: 6.7 s, total: 1min 47s\n",
      "Wall time: 2min 40s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# Experiment\n",
    "logdir = './logs/mnist'\n",
    "\n",
    "model = Net().to(device)\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "optimizer = torch.optim.Adam(model.parameters())\n",
    "\n",
    "runner = SupervisedRunner()\n",
    "\n",
    "runner.train(\n",
    "    model=model,\n",
    "    criterion=criterion,\n",
    "    optimizer=optimizer,\n",
    "    loaders=loaders,\n",
    "    callbacks=[\n",
    "        AccuracyCallback(num_classes=num_class),\n",
    "    ],\n",
    "    logdir=logdir,\n",
    "    num_epochs=n_epochs,\n",
    "    fp16=FP16_PARAMS,\n",
    "    verbose=False\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's possible to check the saved models on the `logdir` directory defined early."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "_kg_hide-input": true,
    "id": "MsmCzDB9rs3J",
    "outputId": "22958127-4803-476b-94ad-4a49983da58d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_metrics.json  best_full.pth  last_full.pth  train.4_full.pth\r\n",
      "best.pth       last.pth       train.4.pth\r\n"
     ]
    }
   ],
   "source": [
    "!ls ./logs/mnist/checkpoints"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. Predictions\n",
    "\n",
    "Ok, let's run our prediction. Remember the fake labels? I used here to don't get errors on predict."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "_kg_hide-input": false,
    "id": "u-rjPyEJrEBr",
    "outputId": "639ac2e9-74d4-448e-8586-b90cb5f2a52b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Selected optimization level O0:  Pure FP32 training.\n",
      "\n",
      "Defaults for this optimization level are:\n",
      "enabled                : True\n",
      "opt_level              : O0\n",
      "cast_model_type        : torch.float32\n",
      "patch_torch_functions  : False\n",
      "keep_batchnorm_fp32    : None\n",
      "master_weights         : False\n",
      "loss_scale             : 1.0\n",
      "Processing user overrides (additional kwargs that are not None)...\n",
      "After processing overrides, optimization options are:\n",
      "enabled                : True\n",
      "opt_level              : O0\n",
      "cast_model_type        : torch.float32\n",
      "patch_torch_functions  : False\n",
      "keep_batchnorm_fp32    : None\n",
      "master_weights         : False\n",
      "loss_scale             : 1.0\n",
      "=> loading checkpoint ./logs/mnist/checkpoints/best.pth\n",
      "loaded checkpoint ./logs/mnist/checkpoints/best.pth (global epoch 4, epoch 4, stage train)\n"
     ]
    }
   ],
   "source": [
    "runner_out = runner.predict_loader(model=model, loader=test_loader, resume=f\"{logdir}/checkpoints/best.pth\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And save the results on submission file for the competition."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "_kg_hide-input": true,
    "id": "eMtbpoLDoX5y"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predictions made and Submission file saved!\n"
     ]
    }
   ],
   "source": [
    "_,results = torch.topk(torch.from_numpy(runner_out),1)\n",
    "submission.columns \n",
    "submission_final = pd.DataFrame({'ImageId':submission.ImageId,'Label':np.squeeze(results.numpy())},columns=['ImageId','Label'])\n",
    "submission_final.to_csv('submission_final.csv',index=False)\n",
    "print(\"Predictions made and Submission file saved!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 6. Conclusion\n",
    "\n",
    "Catalyst is awesome module to work with Deep Learning, it is based on the great PyTorch, but without use any complicated code lines and is friendly user. <br>\n",
    "Another advantage, due the PyTorch, is that easy to change and classification and segmentation tasks. Catalyst is really worth to give a try!\n",
    "About the Mixed Precision Training, it is so good, because the task gain more power to deal with big datasets, like images."
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "MNIST_Catalyst",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}