17_accel_sgd.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide_input": false
   },
   "outputs": [],
   "source": [
    "#hide\n",
    "from utils import *"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "[[chapter_accel_sgd]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Variants of SGD"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that you know all about how the architectures are put together, it's time to start exploring the training process.\n",
    "\n",
    "We explained earlier the basis of Stochastic Gradient Descent: pass a minibatch in the model, compare it to our target with the loss function then compute the gradients of this loss function with regards to each weight before updating the weights with the formula:\n",
    "\n",
    "```python\n",
    "new_weight = weight - lr * weight.grad\n",
    "```\n",
    "\n",
    "We implemented this from scratch in a training loop, and also saw that Pytorch provides a simple `nn.SGD` class that does this calculation for each parameter for us. Let's now build some faster optimizers, using a flexible foundation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let's start with SGD"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, we'll create a baseline, using plain SGD, and compare it to fastai's default optimizer. We'll start by grabbing Imagenette with the same `get_data` we used in <<chapter_resnet>>:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#hide_input\n",
    "def get_data(url, presize, resize):\n",
    "    path = untar_data(url)\n",
    "    return DataBlock(\n",
    "        blocks=(ImageBlock, CategoryBlock), get_items=get_image_files, \n",
    "        splitter=GrandparentSplitter(valid_name='val'),\n",
    "        get_y=parent_label, item_tfms=Resize(presize),\n",
    "        batch_tfms=[*aug_transforms(min_scale=0.5, size=resize),\n",
    "                    Normalize.from_stats(*imagenet_stats)],\n",
    "    ).dataloaders(path, bs=128)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dls = get_data(URLs.IMAGENETTE_160, 160, 128)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll create a ResNet34 without pretraining, and pass along any arguments received:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_learner(**kwargs):\n",
    "    return cnn_learner(dls, resnet34, pretrained=False,\n",
    "                    metrics=accuracy, **kwargs).to_fp16()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's the default fastai optimizer, with the usual 3e-3 learning rate:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>accuracy</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>2.571932</td>\n",
       "      <td>2.685040</td>\n",
       "      <td>0.322548</td>\n",
       "      <td>00:11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>1.904674</td>\n",
       "      <td>1.852589</td>\n",
       "      <td>0.437452</td>\n",
       "      <td>00:11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>1.586909</td>\n",
       "      <td>1.374908</td>\n",
       "      <td>0.594904</td>\n",
       "      <td>00:11</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn = get_learner()\n",
    "learn.fit_one_cycle(3, 0.003)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's try plain SGD. We can pass `opt_func` (optimization function) to `cnn_learner` to get fastai to use any optimizer:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "learn = get_learner(opt_func=SGD)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first thing to look at is `lr_find`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "(0.017378008365631102, 3.019951861915615e-07)"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXhc9X3v8fdXI412a7HlfTfGYDYbGxNKYxxCEhIIJCHNpU3a0NBCkiYkN0/TNM29tOU+Cbmla5qnpQQuoQFSEghroEAbtkAM2JjF7LaxZEm2JWvfpdF87x8zkoUsyZLRmTmj+byeZx7PnDkz58NIzFe/81uOuTsiIpK9ctIdQERE0kuFQEQky6kQiIhkORUCEZEsp0IgIpLlVAhERLJcbroDTNWcOXN8+fLl6Y4hIpJRtm/ffsjdq8Z6LuMKwfLly9m2bVu6Y4iIZBQzqx7vOZ0aEhHJcioEIiJZToVARCTLqRCIiGQ5FQIRkSynQiAikuVUCEREMsCjrx1kV0NnIO+tQiAiEnLuzpdv285dL9QG8v4qBCIiIdfRF2Ng0Kksigby/ioEIiIh19LVD0BFsQqBiEhWakoWgtkqBCIi2UktAhGRLKcWgYhIllOLQEQkyzV39RPNzaE4Ggnk/VUIRERCrrmrn8qiKGYWyPurEIiIhFxzVz+VAZ0WAhUCEZHQa+5WIRARyWpqEYiIZDkVAhGRLDYwGKejN6ZCICKSrYKeQwAqBCIiodbcHeysYlAhEBEJtebOZIsgoCWoQYVARCTUhlsEJSoEIiJZqblLLQIRkax2uBDkBXYMFQIRkRBr7uqnrDCP3EhwX9cqBCIiIdbc1R/oiCFQIRARCbXmrv5A5xCACoGISKgFvbwEqBCIiITa0LUIgqRCICISUu5OS3c/lQHOIQAVAhGR0OrsizEw6GoRiIhkq+YULDgHkBvkm5vZXqADGARi7r5x1PNbgHuBd5KbfuHu1wSZSUQkUwwVgqCHjwZaCJI+4O6HJnj+KXe/MAU5REQySqpaBDo1JCISUqlqEQRdCBx4xMy2m9kV4+xzlpm9ZGYPmdlJAecREckYM6KPADjb3evNbC7wqJm94e5Pjnj+BWCZu3ea2ceAe4DVo98kWUSuAFi6dGnAkUVEwqG5u59obg7F0Uigxwm0ReDu9cl/G4C7gU2jnm93987k/QeBPDObM8b73ODuG919Y1VVVZCRRURCo7kzMZnMzAI9TmCFwMyKzax06D7wYWDnqH3mW/K/0Mw2JfM0BZVJRCSTtHQHv7wEBHtqaB5wd/J7Phe43d3/08y+CODu1wOfBr5kZjGgB7jU3T3ATCIiGaMpBesMQYCFwN33AKeNsf36Efd/CPwwqAwiIpmspaufJRVFgR9Hw0dFREIqVS0CFQIRkRAaGIzT0RtTIRARyVYtKZpDACoEIiKh1NydmlnFoEIgIhJKzZ3JFkHAS1CDCoGISCg1dPQBUFWaH/ixVAhERELoYHsvAPPLCgI/lgqBiEgIHWjvpTgaoSQ/+KsFqBCIiIRQQ3sf81LQGgAVAhGRUDrQ3sv8WSoEIiJZ62B7L/NUCEREspO7J04NqRCIiGSn5q5++gfjzJsV/NBRUCEQEQmdg+2JOQTqIxARyVJDcwjmqhCIiGSnVE4mAxUCEZHQOZAsBFUl6iMQEclKB9t7mVMSJZqbmq9oFQIRkZA5mMKho6BCICISOgfaUjeZDFQIRERCp6FDhUBEJGv1x+Ic6uxP2WQyUCEQEQmVxs7UTiYDFQIRkVA50JYYOqpTQyIiWWpoMpkKgYhIljpcCNRHICKSlQ609xKN5FBZHE3ZMVUIRERCpKG9j7mz8jGzlB1ThUBEJERSPZkMVAhERELlYAqvVTxEhUBEJEQOtvcyN4UdxaBCICISGh29A3T1D6pFICKSrYYuUak+AhGRLJWOyWQQcCEws71m9oqZvWhm28Z43szsB2a2y8xeNrPTg8wjIhJmQ8tLpOoSlUNyU3CMD7j7oXGe+yiwOnk7E/jX5L8iIlnnYEfqZxVD+k8NXQz8uydsBcrNbEGaM4mIpMXBtl5K83Mpiqbib/TDgi4EDjxiZtvN7Ioxnl8E7BvxuDa57V3M7Aoz22Zm2xobGwOKKiKSXnWtvSwsL0z5cYMuBGe7++kkTgH9iZltHvX8WHOo/YgN7je4+0Z331hVVRVEThGRtKtv7WFRxQwrBO5en/y3Abgb2DRql1pgyYjHi4H6IDOJiIRVXWsPC8tT21EMARYCMys2s9Kh+8CHgZ2jdrsP+IPk6KH3AW3uvj+oTCIiYdXZF6OtZ4BF5UUpP3aQPRLzgLuTK+jlAre7+3+a2RcB3P164EHgY8AuoBv4wwDziIiEVn1rD0BaWgSBFQJ33wOcNsb260fcd+BPgsogIpIp6pKFYPFM6yMQEZHJqWsZahGoEIiIZKX61h5yc4y5pTOos1hERCavrrWHBeUFRHJSd2WyISoEIiIhUNfSw8Ky1J8WAhUCEZFQSNdkMlAhEBFJu4HBOAfae1mUho5iUCEQEUm7g+29xB0VAhGRbJXOoaOgQiAiknb1bYlCoD4CEZEsNdwi0KghEZHsVNfay+ziKIXRSFqOP6lCYGarzCw/eX+LmV1lZuXBRhMRyQ51aRw6CpNvEdwFDJrZccBNwArg9sBSiYhkkfrW9E0mg8kXgri7x4BPAv/o7v8T0LWFRUTeI3enriUzWgQDZva7wOeBB5Lb8oKJJCKSPVq7B+gZGEzb0FGYfCH4Q+As4Lvu/o6ZrQBuDS6WiEh2GLoOQbomk8EkL0zj7q8BVwGYWQVQ6u7fDzKYiEg2CEMhmOyoocfNbJaZVQIvATeb2d8HG01EZOYbmkOQCX0EZe7eDnwKuNndNwDnBRdLRCQ71Lf2UJCXQ0VR+rpdJ1sIcs1sAfAZDncWi4jIe1TX2sOi8kLMUn9BmiGTLQTXAA8Du939eTNbCbwdXCwRkeyQmExWlNYMkyoE7v5zdz/V3b+UfLzH3S8JNpqIyMzm7uw91MXiNPYPwOQ7ixeb2d1m1mBmB83sLjNbHHQ4EZGZbM+hLtp7Y5y2uCytOSZ7auhm4D5gIbAIuD+5TUREjtGOmlYA1i+tSGuOyRaCKne/2d1jyduPgaoAc4mIzHg7aloozc/luKqStOaYbCE4ZGafM7NI8vY5oCnIYCIiM92OmlbWLS0nJyd9I4Zg8oXgCySGjh4A9gOfJrHshIiIHIOuvhhvHGhn/ZL0r+g/2VFDNe5+kbtXuftcd/8EicllIiJyDF6ubSPu6e8fgPd2hbJvTFsKEZEss2NfCwDrMqVFMI70ntQSEclgO2paWTmnmIriaLqjvKdC4NOWQkQki7g7O2paWLc0/a0BOMoy1GbWwdhf+AakdyqciEiGqm3p4VBnfyj6B+AohcDdS1MVREQkW7xQk+gfOD0kLYL3cmpoUpLzDnaY2RGrlprZZWbWaGYvJm9/FHQeEZF021HTSmFehDXzwvG39qSuUPYefQ14HZg1zvN3uPtXUpBDRCQUdtS0cOriMnIjgf8tPimBpkguTHcBcGOQxxERyRS9A4O8Wt8emv4BCP7U0D8CfwbEJ9jnEjN72czuNLMlY+1gZleY2TYz29bY2BhIUBGRVHjjQAexuLNuSXpXHB0psEJgZhcCDe6+fYLd7geWu/upwH8Bt4y1k7vf4O4b3X1jVZXWuhORzFXd1AXAyjQvNDdSkC2Cs4GLzGwv8B/AuWZ268gd3L3J3fuSD38EbAgwj4hI2lU3dQOwtDK9VyUbKbBC4O7fdvfF7r4cuBT4lbt/buQ+yesgD7mIRKeyiMiMVd3UzbxZ+RTkRdIdZVgqRg29i5ldA2xz9/uAq8zsIiAGNAOXpTqPiEgq1TR3sayyON0x3iUlhcDdHwceT96/esT2bwPfTkUGEZEwqG7qZvPx4errDMcgVhGRLNDTP0hDRx/LQtQ/ACoEIiIpU9Oc7CierUIgIpKVhguBWgQiItlpaA7Bstnh6ixWIRARSZGa5m5K83OpKMpLd5R3USEQEUmR6qZuls4uwixcF3hUIRARSZGa5m6WhayjGFQIRERSYjDu1LZ0szRkk8lAhUBEJCXqW3sYGHS1CEREstXQ0NGwTSYDFQIRkZQYXnVULQIRkexU09xNXsRYUFaY7ihHUCEQEUmBmuYuFlcUEckJ19BRUCEQEUmJ6qbu0C0tMUSFQEQkYO5OTVM45xCACoGISOBaugfo6IupRSAikq3CutjcEBUCEZGADc8h0KkhEZHsVJOcQ7CkQoVARCQrVTd3M7c0n8JoJN1RxqRCICISsDCPGAIVAhGRwNU0h3PV0SEqBCIiAeodGORAe29oh46CCoGISKBqW8I9YghUCEREAjW06ugStQhERLJT2OcQgAqBiEigqpu6KYpGmF0cTXeUcakQiIgEaF9zYtVRs/AtPz1EhUBEJEDVzeFdfnqICoGISEDicWdfc7gnk4EKgYhIYBo6+uiLxdUiEBHJVkMjhpaGdPnpISoEIiIBGb4OQba3CMwsYmY7zOyBMZ7LN7M7zGyXmT1rZsuDziMikir7mrvJMVhYXpjuKBNKRYvga8Dr4zx3OdDi7scB/wD83xTkERFJiermbhaWFxLNDffJl0DTmdli4ALgxnF2uRi4JXn/TuCDFubBtiIiU1CTAUNHIfgWwT8CfwbEx3l+EbAPwN1jQBswe/ROZnaFmW0zs22NjY1BZRURmVZhvw7BkMAKgZldCDS4+/aJdhtjmx+xwf0Gd9/o7hurqqqmLaOISFA6+2I0dfWHerG5IUG2CM4GLjKzvcB/AOea2a2j9qkFlgCYWS5QBjQHmElEJCWGrlO8LMQXpBkSWCFw92+7+2J3Xw5cCvzK3T83arf7gM8n7386uc8RLQIRkUwzPIcgA1oEuak+oJldA2xz9/uAm4CfmNkuEi2BS1OdR0QkCDXNiTkESzOgjyAlhcDdHwceT96/esT2XuB3UpFBRCSVapq7KSvMo6wwL91Rjircg1tFRDJUdYaMGAIVAhGRaefuvHmgg5Vzwt9RDCoEIiLTbm9TNw0dfZyxojLdUSZFhUBEZJo9u6cJgDNXHDE/NpRUCI7RM7sO8YP/fpvY4HiTpkUkWz37TjNzSvJZVZUZp4ZSPnw0jHY1dNLTP8gpi8uOum9XX4xrH3qdW7fWANDc1c9fXXRS0BFFJEO4O8/uaeLMFZWhvk7xSFlfCNydK3+yjeqmbv7ls6fz4ZPmj7vvM7sP8a27Xqa2pYc/+u0VxOLOj5/Zy/LZRVx29ooUphaRsKpt6aG+rZcvrsyM/gFQIWB7dQu7G7uoKMrjy7e9MGYxqG/t4XsPvs4DL+9n2ewifnblWZyxvJLBuFPb0sM1D7zGstnFnLGikmd2HeKZ3U3MKYly8bpFGbHOiIhMn60Z1j8AWVYIDrT1Mr+s4F3b7nh+H8XRCA99bTNX3rqdL9/2An/z6VNZUFZIY2cfb+xv5+an9xJ352sfXM0Xz1lFYTQCQCTH+KdL1/GZf/sNV966nXjcicWdgrwcegfi/O0jb3Hmiko+eOJcygujFOVHKC+McubKSvIi6p4RmYm27mmmoiiP1XNL0h1l0rKmENz7Yh3f/PnL3PuVszlxwSwgsTrgL1/Zz8dPXcj8sgJ+cvkmfv+m5/jGz15612vPP2k+37ngxDH/ui/Oz+Wmz5/B1ffuZGVVCZuPn8OGZRU0tPdxz446frGjju89+Ma7XrNsdhFfPXc1n1i3kNxIDu7OvuYeork5RxQqEcksz77TxKYVleTkZEb/AIBl2hpvGzdu9G3btk35dS1d/XzoH55gYXkhd3/5bCI5xh3P1/Ctu17hri/9FhuWVQCJzuCn3m6ktCCPuaX5zJ1V8J6miLs7bT0DdPUP0tUXY1dDJz/81S5e29/OijnFVJXk8/r+djr6YkRyjE+uX8RV564ed32Srr4YDpTkZ00NF8kYda09nP39X3H1hWv5wm+Hq9/QzLa7+8axnsuab5OK4ih/+fGT+OpPd3Dz0+/wR+9fyR3P72NVVTGnLy0f3q84P5fzT14wbcc1M8qLopQnv9ePn1fKR0+ez8OvHuRHT+1h0J1PrF/E2oWz2NXQya1bq7lnRx0fP20hK+cUU1aUR1E0l7cOdvDsniZ21rcDsGFpBeesqeKc46tYu2BWRv31ITJTDc8fyKCOYsiiQgBw4akLuGdHHX/3yFusqirhhZpW/uJjJ6R8iJeZcf7J8zn/5CNHKF25eSX/8vhu7nqhlo7e2PD2aCSHdUvK+fKWVQzGnSffbuS6h9/kuoffZE5JPuccX8WWNYlbaUH4F7kSmYme3dNMWWEeJ86fle4oU5I1p4aG1Lf28KG/f4KBQSfuzta/+CBzSvKnMeH06Y/Fae8doKM3xoKyAgryIu96vrGjjyffauTxtxp58q1G2noGiObm8IE1VVxw6kLOO3EuRdGsqvUiabXlusc4bm4pN35+zDMwaaVTQyMsLC/kWx89gavvfZWPnDQvtEUAIJqbw5yS/HEzVpXmc8mGxVyyYTGxwTg79rXyy5f38+Ar+3n41YPMKsjlMxuX8AdnLR/uc4jHnY6+GLMKcjNmsotIJth7qIu9Td187n3L0h1lyrKuEAB87sxlNHf189Fp7AtIt9xIDmcsr+SM5ZX87wvX8tw7zdz2bDU/fmYvNz39DqcsKqOlu5+DbX30D8ZZWFbA+1bO5n2rZrOgrID+WJyBwTgVRVE2TTAjsrW7n7cOdtLWM8DskihVJflUFkeJJPsozCA/NzLma0Vmsuuf2E00N4ePn7Yw3VGmLCsLQU6O8fXzjk93jMBEcoyzVs3mrFWzOdDWy61bq9le3cKKOcXML0uMgnq1rp0n3mrkFzvqjnj9mnmlXP7+FVy8biH7mnt48q1Gfr3rEK/Wt3Gwve+ox1+/tJzLfms5HztlwTHPl3B3Drb3sauhk5wcKIrmUhyNMKswj/KivGkvNv2xOF19MXpjg/QOxDESlxgc3QnfOzCIO8NzSYIy1HIryc8dLrISXvuau7lzey2fPXMp82Zl3hDwrOsjkMPcnV0Nib/u8yI5RHNzeLW+nRuf2sMbBzqI5ubQH0ssqrdyTjHrl1awZn4Jx88rpaIoSlNXH4c6+mnp7iee/DXq6Y9x30v17G3qZt6sfNYtKWd/Wy/1rT209QywtLKIVVUlrKgqpiA3gieC0DcYp7M3RldfjIaOPl7f305L98C42YuiEeaW5nPc3FKOn1fCqqoSZhXmUZCXQ0FehKbOfmqau6hp7qatJ0ZhXg6FeREiOTm0dvfT2NlHU2c/rd39tPYM0N0/eMQxSvJzOXVxGSctnEVjRx+v7W9nd2MXg3GnrDCPBWUFLCovZNXcEo6rKmHV3GLmzSpgTkn+Ef05Y+mPxbn/pXrebuiktTvxOTZ19rO/rZeD7b3E4k6OQWVxlMriKHmRHOKe+Ln1x+J09sXo7IvhDicvmsX6pRWsW1LOovJCKoqilBfnUZQXwcwwEq01nQ4Mxl/c/Qp3bqvliT/bwoKywnTHGdNEfQQqBHIEd+fpXU08/OoBTlhQyubVVVNaKiMed554q5FbfrOX2pYeFpYXsqi8gNKCPKqbutjd2EV1UxcDg4d/96K5OZTm51Kcn0tFcZQT5pWyduEsVs8rwTC6+2N09Q/S3jNAS1c/Ld0D7G/r4e2GTvYe6iIWH/v3uLwoj4qiKL0Dg/QMDDIQi1NRHGV2ST5ziqNUFEcpT7YyivNzKciLUJCXKICv1LXx0r42Xt/fTlVpPmsXzGLtwlkU5EU40NbL/rZealu62XOoa7hgDinJz2VheQHLZhezYk7idsL8UtbMLyWSY/x8Wy3/+vhu6lp7yIskhhhXFOVRWRxlYVkh88sKqCyO0tYzwKHOfpo6+xiMO2ZGjkHeiM9rMO68XNvKzrp2+idYDTdx2i6H/NwIxdEIpy0pZ/PxVWw+vopF5eH88soE9a09nHPdY3xm4xK++8lT0h1nXCoEEjpDv3fT8RdqfyxObUs33f2JL/vegUEqiqIsqSyaluvFxuM+4TyNwbizr7mbPYc6aezo41BnP4c6+6ht6aG6KdGBOFQozKAkmktHX4z1S8v5+nnHs3n1nGn5HPpig7x5oIOG9j5auvtp7R6gJ3kqy3EG44mWRO/AIG09A2zd08yB9l4g0aL49OmLuXjdIiqKo+85Sza5+t6d/PS5Gh770y0srgjv2mIqBCJpFE8uTvj6gXZe399OXUsPHz9tIe+fpgJwrNydtxs6eeLNRu55sY5X69vJixjvWzmbxRWFzC0tYH5ZAacsKuOE+aXkan2sIxxo62Xz3zzGJRsWce2nTk13nAmpEIjIUb1W386d22vZuqeJho4+mrr6GPp6KMyLcNqSMtYuKGNlVTErq4pZM6+U2SEefh00d+eKn2znsTcaeOxPt4R+pWHNIxCRo1q7cBZXL1w7/Dg2GGd/Wy8v7mtle3ULL9S08NPnaugZONyxvnJOMRuXV7BpxWzOO3Eu5UXZc1rp5qf38uhrB/lf4yxImUlUCERkTLmRHJZUFrGksmh4bHw87hxo72VPYxc769vYtreZR147yM+21ZIXMbasmcsn1i3inDVVM3phxJf2tXLtQ69z3olzuTxki8sdC50aEpH3JB53Xq1v594X67jvpXoaOvqI5BgnLyrjfSsqOX1ZBSfML2VJxZHzMjJRW88AF/zgKdzhl1f9dsa0gtRHICIpMRh3nnunmWd2H2LrniZe3Nc6PEy4KBphzfxSNiytYOPySjYurwj1Ei9jOdTZx1dv38Hze5u548qzhpevzwQqBCKSFj39g7xxoJ03D3TwxoEOXq1v46XatuHhtMtmF7F+SXlysmIpC8sKmVeWH8plSp7edYiv3/EibT0DXPvJU7hkw+J0R5oSdRaLSFoURiOsX1rB+qWH/3Luiw2ys66d7dXNvFDdyjO7m7jnxfp3vW5BWQEbllWwaUUlm1ZUsmZeadqG2vbFBvnBf7/Nvzy+m5Vzivn3L2wavsrhTKFCICIplZ8bYcOyiuHTKu5OfVsvexo72d+amLG9q7GT599p5oGX9wOwem4Jv3fmUj61fjFlRam73sZvdjfxnXteYU9jF7+zYTF/ffFJM3Jpd50aEpFQck9MxHvq7UPc8XwNL9W2UZCXw4fWzudDa+exZU0VswK6CNNbBzv4tyf2cNcLtSypLOT/XHwyW9bMDeRYqaI+AhHJeDvr2vjpczU8/OoBDnX2kxcxfmvVHD51+iI+ctL8SS30N5H61h7ueH4fD76yn7cbOsnNMa7YvJKvnrs68NVmU0GFQERmjMG4s6OmhUdfO8gDL++nrrWH0oJcLjx1AZtWVA7Pfp7KEuhPvd3IV27fQXvvAJuWV3LBqQs4/6T5zM3AJaXHo0IgIjNSPO5sfaeJO7fV8tDOA8OznqORHE5eNIszV85m04pKTl9SMWbfgrtz06/f4XsPvs7quaVc//sbWDGnONX/GSmRlkJgZgXAk0A+iU7pO939L0ftcxlwHTB0dZQfuvuNE72vCoGIjCU2GOedQ128tr+dnXVtbK9u4eXatuElyiuLoyybXcSi8sLh1sLB9l6e2d3ER0+ez9/+zmkUz+DZ0OkaPtoHnOvunWaWB/zazB5y962j9rvD3b8SYA4RyQK5kRxWzytl9bxSLl63CIDu/hg7alrZWddGdXM31U1d7KxrG76QUo7BNz+yhi+ds2pGzHo+VoEVAk80NTqTD/OSt8w6DyUiGa0omsvZx83h7OPmpDtKqAW6wLiZRczsRaABeNTdnx1jt0vM7GUzu9PMlozzPleY2TYz29bY2BhkZBGRrBNoIXD3QXdfBywGNpnZyaN2uR9Y7u6nAv8F3DLO+9zg7hvdfWNVVVWQkUVEsk5KLjnk7q3A48D5o7Y3uXtf8uGPgA2pyCMiIocFVgjMrMrMypP3C4HzgDdG7bNgxMOLgNeDyiMiImMLctTQAuAWM4uQKDg/c/cHzOwaYJu73wdcZWYXATGgGbgswDwiIjIGTSgTEckCE80jSEkfgYiIhJcKgYhIlsu4U0Nm1ghUJx+WAW0T3B+9LQ84NMVDjnyPyTw3ett4jyfKO2eKOSfKeCw5J8p2rBmPlnM6Mw5t0897cjkz9ec9Vt7p/Cxn2s+73N3HHn/v7hl7A26Y6P7obSQ6qY/5GJN5bvS28R5PlHeqOSfKeCw5j5LtmDJO92epn7d+3kF/ljP15z3WLdNPDd1/lPvjPX+sx5jMc6O3jff4aHmn4mivm2rOibIda8ajvXY6Mx7tWBPRz3vsf49F0D/vkff1855424TvkXGnht4LM9vm4/Sah0km5FTG6ZMJOTMhI2RGzjBmzPQWwVTdkO4Ak5QJOZVx+mRCzkzICJmRM3QZs6pFICIiR8q2FoGIiIyiQiAikuVUCEREspwKQZKZvd/MrjezG83smXTnGYuZ5ZjZd83sn83s8+nOMx4z22JmTyU/zy3pzjMeMys2s+1mdmG6s4zHzE5Mfo53mtmX0p1nLGb2CTP7kZnda2YfTneesZjZSjO7yczuTHeWkZK/g7ckP7/PpivHjCgEZvb/zKzBzHaO2n6+mb1pZrvM7M8neg93f8rdvwg8wDgXyEl3RuBiYBEwANROd8ZpzDl0mdKCIHJOU0aAbwE/m+58I/JMx+/l68nfy88A0z7kcJoy3uPuf0xi9eD/EdKMe9z98unONpYp5v0UcGfy87soFfnGNNXZgmG8AZuB04GdI7ZFgN3ASiAKvASsBU4h8WU/8jZ3xOt+BswKY0bgz4Erk6+9M6yfJZCTfN084LaQZjwPuJTEl9eFYf0sk6+5CHgG+L2wZky+7u+A00OeMZD/b95D3m8D65L73B50tvFuQV6PIGXc/UkzWz5q8yZgl7vvATCz/wAudvdrgTFPBZjZUqDN3dvDmNHMaoH+5MPB6c44XTlHaAHyw5jRzD4AFJP4n7HHzB5093jYcibf5z7gPjP7JXB72DKamQHfBx5y9xemM990ZUylqeQl0WJeDLxIGs/QzIhCMI5FwL4Rj2uBM4/ymsuBmwNLdKSpZvwF8M9m9n7gySCDjTKlnGb2KQeorNkAAARLSURBVOAjQDnww2CjDZtSRnf/DoCZXQYcmu4iMIGpfpZbSJw+yAceDDTZYVP9vfwqiRZWmZkd5+7XBxkuaaqf42zgu8B6M/t2smCk0nh5fwD80Mwu4L0t5/GezORCYGNsm3D2nLv/ZUBZxjOljO7eTaJYpdpUc/6CRNFKpSn/vAHc/cfTH2VCU/0sHydxve9UmmrGH5D4QkulqWZsAr4YXJyjGjOvu3cBf5jqMKPNiM7icdQCS0Y8XgzUpynLeDIhI2RGzkzICJmRUxmnX6jzzuRC8Dyw2sxWmFmURMfgfWnONFomZITMyJkJGSEzcirj9At33nT1Uk9zL/1Pgf0cHlZ5eXL7x4C3SPTWf0cZZ0bOTMiYKTmVUXndXYvOiYhku5l8akhERCZBhUBEJMupEIiIZDkVAhGRLKdCICKS5VQIRESynAqBzAhm1pni491oZmun6b0GzexFM9tpZvebWflR9i83sy9Px7FFQBevlxnCzDrdvWQa3y/X3WPT9X5HOdZwdjO7BXjL3b87wf7LgQfc/eRU5JOZTy0CmbHMrMrM7jKz55O3s5PbN5nZM2a2I/nvmuT2y8zs52Z2P/CIJa609rglrg72hpndllxymeT2jcn7nZa4ctxLZrbVzOYlt69KPn7ezK6ZZKvlNyRWqsTMSszsv83sBTN7xcwuTu7zfWBVshVxXXLfbyaP87KZ/fU0foySBVQIZCb7J+Af3P0M4BLgxuT2N4DN7r4euBr43ojXnAV83t3PTT5eD3ydxHULVgJnj3GcYmCru59GYnnwPx5x/H9KHv+oC4yZWQT4IIfXoOkFPunupwMfAP4uWYj+HNjt7uvc/ZuWuDzkahJr3q8DNpjZ5qMdT2TITF6GWuQ8YG3yj3iAWWZWCpQBt5jZahJLF+eNeM2j7t484vFz7l4LYGYvAsuBX486Tj+JK2EBbAc+lLx/FvCJ5P3bgb8dJ2fhiPfeDjya3G7A95Jf6nESLYV5Y7z+w8nbjuTjEhKFIZXXrJAMpkIgM1kOcJa794zcaGb/DDzm7p9Mnm9/fMTTXaPeo2/E/UHG/n9mwA93to23z0R63H2dmZWRKCh/QmJ9/88CVcAGdx8ws70krgM9mgHXuvu/TfG4IoBODcnM9gjwlaEHZrYuebcMqEvevyzA428lcUoKEssOT8jd24CrgD81szwSORuSReADwLLkrh1A6YiXPgx8wcyGOpwXmdncafpvkCygQiAzRZGZ1Y64fYPEl+rGZAfqaxy+QtXfANea2dMkLioelK8D3zCz54AFQNvRXuDuO0hc2PxS4DYS+beRaB28kdynCXg6Odz0Ond/hMSpp9+Y2SvAnby7UIhMSMNHRQJiZkUkTvu4mV0K/K67X3y014mkmvoIRIKzgcSFyQ1oBb6Q5jwiY1KLQEQky6mPQEQky6kQiIhkORUCEZEsp0IgIpLlVAhERLKcCoGISJb7/3JpPASwEr3ZAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn.lr_find()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It looks like we'll need to use a higher learning rate than we normally use:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>accuracy</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>2.969412</td>\n",
       "      <td>2.214596</td>\n",
       "      <td>0.242038</td>\n",
       "      <td>00:09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>2.442730</td>\n",
       "      <td>1.845950</td>\n",
       "      <td>0.362548</td>\n",
       "      <td>00:09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>2.157159</td>\n",
       "      <td>1.741143</td>\n",
       "      <td>0.408917</td>\n",
       "      <td>00:09</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn.fit_one_cycle(3, 0.03, moms=(0,0,0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(Because accelerated SGD using momentum with is such a good idea, fastai uses it by default in `fit_one_cycle`, so we turn it off with `moms=(0,0,0)`; we'll be learning about momentum shortly.)\n",
    "\n",
    "Clearly, plain SGD isn't training as fast as we'd like. So let's learn the tricks to get accelerated training!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A generic optimizer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to build up our accelerated SGD tricks, we'll need to start with a nice flexible optimizer foundation. No library prior to fastai provided such a foundation, but during fastai's development we realized that all optimizer improvements we'd seen in the academic literature could be handled using *optimizer callbacks*. These are small pieces of code that an optimizer can add to the optimizer `step`. They are called by fastai's `Optimizer` class. This is a small class (less than a screen of code); these are the definitions in `Optimizer` of the two key methods that we've been using in this book:\n",
    "\n",
    "```python\n",
    "def zero_grad(self):\n",
    "    for p,*_ in self.all_params():\n",
    "        p.grad.detach_()\n",
    "        p.grad.zero_()\n",
    "\n",
    "def step(self):\n",
    "    for p,pg,state,hyper in self.all_params():\n",
    "        for cb in self.cbs:\n",
    "            state = _update(state, cb(p, **{**state, **hyper}))\n",
    "        self.state[p] = state\n",
    "```\n",
    "\n",
    "As we saw when training an MNIST model from scratch, `zero_grad` just loops through the parameters of the model and sets the gradients to zero. It also calls `detach_`, which removes any history of gradient computation, since it won't be needed after `zero_grad`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The more interesting method is `step`, which loops through the callbacks (`cbs`) and calls them to update the parameters (the `_update` function just calls `state.update` if there's anything returned by `cb(...)`). As you can see, `Optimizer` doesn't actually do any SGD steps itself. Let's see how we can add SGD to `Optimizer`.\n",
    "\n",
    "Here's an optimizer callback that does a single SGD step, by multiplying `-lr` by the gradients, and adding that to the parameter (when `Tensor.add_` in PyTorch is passed two parameters, they are multiplied together before the addition): "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sgd_cb(p, lr, **kwargs): p.data.add_(-lr, p.grad.data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can pass this to `Optimizer` using the `cbs` parameter; we'll need to use `partial` since `Learner` will call this function to create our optimizer later:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "opt_func = partial(Optimizer, cbs=[sgd_step])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see if this trains:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>accuracy</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>2.730918</td>\n",
       "      <td>2.009971</td>\n",
       "      <td>0.332739</td>\n",
       "      <td>00:09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>2.204893</td>\n",
       "      <td>1.747202</td>\n",
       "      <td>0.441529</td>\n",
       "      <td>00:09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>1.875621</td>\n",
       "      <td>1.684515</td>\n",
       "      <td>0.445350</td>\n",
       "      <td>00:09</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn = get_learner(opt_func=opt_func)\n",
    "learn.fit(3, 0.03)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's working! So that's how we create SGD from scratch in fastai."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Momentum"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "SGD is the idea of taking a step in the direction of the steepest slope at each point of time. But what if we have a ball rolling down the mountain? It won't, at each given point, exactly follow the direction of the gradient, as it will have *momentum*. A ball with more momentum (for instance, a heavier ball) will skip over little bumps and holes, and be more likely to get to the bottom of a bumpy mountain. A ping pong ball, on the other hand, will get stuck in every little crevice.\n",
    "\n",
    "So how could we bring this idea over to SGD? We can use a moving average, instead of only the current gradient, to make our step:\n",
    "\n",
    "```python\n",
    "weight.avg = beta * weight.avg + (1-beta) * weight.grad\n",
    "new_weight = weight - lr * weight.avg\n",
    "```\n",
    "\n",
    "Here `beta` is some number we choose which defines how much momentum to use. If `beta` is zero, then the first equation above becomes  `weight.avg = weight.grad`, so we end up with plain SGD. But if it's a number close to one, then the main direction chosen is an average of previous steps. (If you have done a bit of statistics, you may recognize in the first equation an *exponentially weighted moving average*, which is very often used to denoise data and get the underlying tendency.)\n",
    "\n",
    "Note that we are writing `weight.avg` to highlight the fact we need to store thoe moving averages for each parameter of the model (and they all their own independent moving averages).\n",
    "\n",
    "Here is an example of noisy data for a single parameter, with the momentum curve plotted in red, and the gradients of the parameter plotted in blue. The gradients increase, and then decrease, and the momentum does a good job of following the general trend, without getting too influenced by noise:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide_input": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3hU1dbA4d8mBEhACEhNAEFRBGnRWDCCFAUuIgYVBXvnqlgRDVe5dkWxtw8VvIKiwBUMCCgXBKQjwURRBKkKoYUSigQIyf7+2DNhMpkWppwp630enpCZkzkrgaw5Z++111Zaa4QQQkS/SlYHIIQQIjQk4QshRIyQhC+EEDFCEr4QQsQISfhCCBEjKlsdgDt169bVzZo1szoMIYSIKCtXrtytta7n6rmwTfjNmjUjOzvb6jCEECKiKKX+dPecDOkIIUSMkIQvhBAxQhK+EELECEn4QggRIyThCyFEjJCEL4QQMUISvhBCxIiwrcMXIhpl5eQxctZathUUkpyUwNCeLclITbE6LBEjJOELESJZOXkMm7KKwqJiAPIKChk2ZRWAJH0REjKkI0SIjJy1tjTZ2xUWFTNy1lqLIhKxRhK+ECGyraCwQo8LEWiS8IUIkeSkhAo9LkSgScIXIkSG9mxJQnxcmccS4uMY2rOlRRGJWCOTtkKEiH1iVqp0hFUk4YuoECnljhmpKWEZl4gNkvBFxPO13DFS3hSECBZJ+CLieSp3tCf0k62BlzcJEU1k0lZEPF/KHU+mBt7+JpFXUIjmxJtEVk5eQOIWItTkCl9EvOSkBPJcJH3Hckdf3hScr+b/Pnrc452DXP2LSCNX+CLi+VLu6K0G3tXVfEFhkcuv2VZQyFNZq3hkYu7JXf0fOQK//+7T9yZEIEnCFxEvIzWFl69uS0pSAgpISUrg5avblrna9vam4GrIx51aCfGMX/YX2ulxd0NEWTl5pI+YS/PMGXR6aTbrOnaH1q358MJrSH/5exkiEiETkCEdpdQnQB9gl9a6jYvnFfA20Bs4DNymtf4pEOcWkSHYwx+O5Y72cz0yMbfcudzF4Gt7g4T4OJSiXLK3c34d58nigd98xJm5S1jWpA2DfpxCXs16DD3Yl2e/+Y2Cw0Vefzb27y2voJA4pSjWmhQZThI+CtQY/qfAe8A4N8//AzjT9udC4P9sH0UMCGWXSFfnemRiLtl/7uWFjLZuz+duHqB2YjyJVSqXeZN4ZGKu2/M7Dx053jn0XrOI+5Z9xfgOvXiqx318PPl5npo7hl8anUVucsvSeN39bJy/t2KtvX6NEI4CMqSjtV4A7PVwyFXAOG0sA5KUUo0CcW4R/nytkHEc+kgfMfekhjpcnUsD45f95fH13A35PH3lOSzO7MamEVewOLMbGakppUn9lKN/8/ScD2m6bzsAyvY6juxX/Gflb2bkzLdYmXw2z3YfhFaVGHLFo+w85VTezxpB2tbfuGRTDmjtdmjI07CTdN0UvgjVGH4KsMXh8622x8pQSt2jlMpWSmXn5+eHKDQRbL5WyASiBNLduTR4TIi+zAPYDe3ZkobHD/PPZV9x+8pv+O4/g2l0YDc3XtTUHF9cDFu2wKJF3LZpEfcvmcjoyc/zd5UE7s0YxrHK8QDsTziF+67KpO7hfXw1/gk+nzSct6a/RtXjx1x+H96GnVzdoZyMQLzxivAUqrJM5eKxcsOgWuuPgI8A0tLS3A2TigjjS9mkL4un3HGcH6hkG9d2pWhrHgweDNdfD506lXveY9uDbdtg4UJYsICMBQvI+PVX85qV4tCV4pgzdTjV17SARzebZH/8OABP2758a816/DPjX+w65dQyL7uq0Zncct3zNDi0h9P2bWfIovEkH8jnmdteKBeCu5+jXZxy9WtWMbJJS3QL1RX+VqCJw+eNgW0hOrewmC9lkyfbK975zsBdsu+yIZtZnz4I778PXbvC66+D87EbNsBNN5k3hG0O/z0ffhhSUmDAABg3DpKT4YUXYMEC4g//TfXZ31G9/qlw9Ch07AhDh8KHH8J338GaNUxbso7rh00gp3ErUpISuOmipqV3EkkJ8fzUvB3TWnfh3fSBDO77OO23r2PS2Edh3TqPP8fKxcepd+jESKq7793xZ+Xtyl02aYluobrCnwYMVkpNwEzW7tdabw/RuYXFfOkS6ctdgCveyinji4t4bMFnDPpxCvtbnA2ffQuvvgqPPQbLlsEnn5hE/fzz8H//B/Hx5o1g1ix480247TaYNw/S0szzHTpAZadfm06dICfHbQx9gb4dW7h93vEOJadjT5ZndKTz0Lvgootg6lS45BLgxM9xyKSfKdaaN6e/zpVrFnLuA+PZm1iLFA8/K1+v3GWTlugWqLLML4EuQF2l1FbMnWw8gNZ6FDATU5K5HlOWeXsgzisih7cukUN7tiyTkMC3XvGeElFaSQHDxz9L++3r2Nj/Fk4fOwoSEmDyZHjtNcjMhJUrYc8eOHQI7roLnnnG/P3OO+GOO2DiRNi0CW65xSR9J4EoN3X5s+nUBq64Arp3h/Hj4dprS48FmD7yU65csxCAW1dOZ1S3Wzz+rHwdMjvZN14RGQKS8LXWA708r4H7A3EuEZ1Otle8Y4KqVXiQ/dVqgFK0jj/GV58/BYUFMHkyp1999YkvUsoMu5x/vknkl14KI0ZA69Ynjpk/Hz74wLwp/P03NG1a7txBHe9u0QKWLIG+feGGG6BOHejWzbx2w0r0mv0OGxs0Iy+hNrflTOf0kc9ypYdz+nrlfrJvvCIyKO1l3M8qaWlpOjs72+owRJizJ924QwdZ/v4tvN/xOj7tdD3zZ79M/Z9XmInW888/+RNs2mSGdh59FJo1K/NU+oi5Lq+GU5ISWJzZzef4Pb7JFRRAejrk5cGiRVC/PnTpAn/9Zd4QDh40Qz7vvAMPPOD2PPZYqxYdJb6kmENVE93GKj2CIptSaqXWuvztKNI8TUQ4eyKa8XEW1YuOcP/yr+jcrwv1f1xkkqA/yR6geXPzOi74O97t0x1CUhJ8+60Zz+/d23y+ebN5rF07c8zFF8Mbb0CbNrBrl/mzc+eJv+/axawt22DXLmocK6QExZLT2jG9XXcuefyecnHJJi3RS67wRXQYO9ZMsIJJiocOQX6++XuQ+HuFX6Gvz8mBzp2hqAimT4fLLjvx3NSpkJFR9vhKlaBePXNHYPuzgUTm7NEcP3iIvmsW0mTfdg5Urc7ETv1p9O8n6NPpbJ++bxHe5ApfRL81a0yFzYAB8NlnJiEGMdmD/+PdFbpDSE2FxYtNff+555Z9rm9fmD3bVA/ZE3ydOibpOzjD9icrJ48ek3+h9eZfuefHKdw951P2LZ7CvDGT6Dqwp0+xi8gkCV9EhzVr4MwzTX381KlmotMLX8aqPR3j76bkFa6IsQ/hOFOq7BW/FyNnraXweAkrG7dmUOPWtNmxntGTn6PRg4PosmEUD1/hvueQiGyS8EVA+ZtET9qaNabKpmlTM25dpYrXOL2Nn/tyjD/j3VZVxDjfQfzasAWZvR7g06+epf+MTxh29A5AVtZGI+mHLwLGl344Ad828PBhePBBk/BTU81jVauaq14PfFlRGuxVpxXp3xNIru4g5p9xPhPbXs4/l0/mrD9X8/0HE6B6dbNGQUQNucIXAePL4h6vx+zfbxJ2tWreT7h8uamj/+MPk/SHDPE5Vl/Gz0Ox6tTTHUKwyiNd3VkAvND9Li7ZnMvrM94kr1Z982b6/fdw3XV+n1OEB7nCFwHjdxI9cAAaNYLERDjjDLPSdMgQGD3a1KDv3m0OPnYMnnzSlCMeOWKS0ttvm1W0PvK25aGvxwSLqzuhRybm0iwAHSwd7ywcHaxanSf+8SAt9m7l0k22/Yk2bvTjuxDhRhK+CBi/k+iuXVBYCH36mPr5vDyz2vXuu02/mnr1oG5dswr1pZfg1lvhl19KV6BWhC8N3Xw5Jljc9fWHAAyDYZL+4sxuvHV9hzLf46LmqUw4t/eJAyXhRxVJ+CJg/E6iBw6YB+66CyZMgNxcU0+/cSPMnAlvvMGmS3uxJDGZu68eTvpZN5G18VCZ1/K1l7sv4+dWjbGD92GjQM0luPoea7z9xolVxfPnw759fp9HhAcZwxcB40uZYkZqCmjNitc/5mjBAZZ2urL0DWHwqPm8BzzwzTq6N8kzx8bFmdWuzZuT1bAdw/a2ovBM25WvU9VMRXvb+FJhY9WqU2+97+HEm4K/Y/0uv8cpU0wfofnzTcnn7Nmmtl9ENEn4MS7QE4NeE+QPP5Dx+ONk/Pij+XzIlWQBw6asIn1PAQCbiiq7TNTeJnz92UQl3LibWHWUnJQQvAZuqammRfTMmdCvn0n6c+ZI0o9wMqQTwwJeIunN3Xebxl/btpmJ2BYt4K67yH/8Ke79/lOu/+V/AByqkuByyMLbpHA09XJ3nlh1LjK1D4MFfcOS3r0hKwtWrzZJf6+nratFuJOEH8NCtbtRVk4enV+cDaNHM6ddF6ZNmmf6zY8ZA7t3c/ecTxm8dBKdN/3EptqN2GHbBtA5UXubFLayqiYY7BOrm0dcwZvXd3A5lxCSN7l//ONE0u/eXWrzI5gM6cSwUCQL+11Etf3mynBh/ZZMmrmekmoJZHTuDHv30unVeWw5cKzc1zonam8rU6O5l7u7obKQbVjSq5dpWXHVVSbpz5ljKqZERJEr/BgWiiti+11E0hFTTVNQrUbpXURWTh7pry1gy4FjbocsHHmrmrGyqsYqIS0d7dkTpk2DtWvhyisD//oi6OQKP4aF4orYfreQVHgQgP3VTgFOzBfYz60x49Qak6jdTR57mxSOtV7u/jZwc8ftZH6PHmYbyMxM2LEDGjYMwHchQkUSfgwLVrJwZB9yqHXEJPyCBJPw45RyubCoIrtFCSPQb3JeK38uvtgcuHKlWQ0tIoYk/BgX7Cti+12E45BOQnyc23LDSKyoiTZey1tTU02v/RUrJOFHGBnDF0FlH1dvhknk1RrWd9nHxS5SK2qiidfJ/Bo1oFUrkB3pIo4kfBF0GakpPHxuXVCK756+kozUFEv71AjPfJrM79jRrL59+WWz7aKICDKkI1zyZwWu89eOPGU7F48bZyb44kySD8X8gTg5Pk3mv/SS6bHzr3+ZvkejR/u/YbwIOtnEXJTjPGkH5hfelxJHx69NPrCLp74fTe8/lnCoaXNqjPsPXHppsMMXAeDzG35WFtx/v6nYefZZ07bay+YzIrhkE3NRIf70pBk5ay2df13IwJ9nccHWX1EaXu18CzMvH8h8SfYRoUJ3dxkZ0LUr3HcfDB9uupu+/LIk/TAlCV+U427SLq+gkPQRcz0mgk4/ZPHSd++xtVZ9Zp3Zkdc630JerfqoQ+6bgInwcVLN2GrVgs8+g5o14ZVXzKY0b77pNekHa0cv4Z4kfFGOu+X6Ckofd5kI3nqLEd+9y/zm5/HPfsM4En9im0KpvokMJ3N3V5q4a17BiPR8rn/7bfMG0LAhNGgADRqwQVVn9l7ITWjAhg4dubD9aUxemRf4Lp/CI0n4ohxXk3b2VbCOShNBh2R48UUYPpy87r158PxBHNEnKnCk+iZyVLS/Upk7AqV4Iv02fq7dlLsS9nB68SHYuZNDi5fRYOdO/nnMvMbRbyqz5LT2HG2ZzldtL0MrUywYqa2sI4kkfFGOqwoad5txbNt32Cyzf/VVuOUWUsaM4blVO+VWPUJ5a8bmPAxz+NjxsncESvHFOd35wWHFdM8Rc8krKKRa0RHab1/H5euW0XPdMkZ++w6/NWjB6ganl365LLwLLkn4wiXnFbjptl9aR0qX8PoPo2H5NDNp9+67UKlSzPWziSaeSjJdje+742rj+iPx1VjetC3Lm7Zl1lkd+e8XmdQ5vL/M18nQX3BJwo9SzldiXc+ux7w1+Sd91T20Z0vGffA1PX6Zx28NzuDblum8NutdMlZ9D48/DiNGSGVGFPC0PiJ9xFyPO3A50piLhKE9W7q8a9hfrQYA1/z6PU327+TLDr0A6Hp2vcB9M6IcqcOPQq7q6J15ras/doyfXnyXw5O+4ufaTbl06yrabPmdEqWopDV7ayRR51ABvPCCWXwjyT7qNc+cUW4ex5uE+DiuOS+lzAQtQP2De/jxg1tLP7+/7xPMaNVJmucFgKc6fGmtEIVcVVo4c7uz1aFD8MYbFDY5jXOfe4xL1izj/qWTSPj7AC/1GMS33//CT0+/zoGEmjxz2SDS4zqSlbstSN+JCCeehlvi3LzhFxYVM29Nfrn+SQUJNTkcX5Ufmp/LqgZn8PJ379KkYIeM4QeZDOlEIV9/acoct3u3GYN/913Yt4/fT2/PW5feyx91T6PJ/h2saHwOKEXSgm0cLW5N4R0fmK+TcrqYMbRnSx6emOvyuRKtXVZygfl/Zp/XKb37BC69ZzR7EmvS6OAeZv7nAd6b+goP3v9uML+FmCdX+FHI14mv0uPefReaNoXnnjOtD5Yu5Zr+L7Lg9PPYUbMuK5q0KR2yKSgsCsk+uCL8ZKSmUDsx3uVzyUkJPjVds3dPTUqIJ79GbUoqxZFXqz5Dez9M+x3rGLNmclBiF4Yk/CjkqhOls9LaeK3NDkYdOphNqr/+Gi66qMLVEnIrHhuevvIct11Ofe2AmpGaQu7TPXjLYWP23y7oxoYBt9Ni/MfwzTfB/jZilgzpRCFXlRZuq3Q2b4a9e+GWW0yPcxt35XnV4iux73D5drhSThcbfOly6usajHLlu0fTYW0u3HYb5OZCkybB/FZiklTpxLrJk+Haa2H5crjggjJPuep1Apx0J00hvFq3Ds491wwxzpgBzZpZHVHECXq3TKVUL+BtIA4YrbUe4fT8bcBIIM/20Hta69GBOLc4OfZkfsO0SQyqVImZJafS1+kYTwuoZCWtCIozz4Rp0+Dqq+HCC0375Y4drY4qavid8JVSccD7wOXAVmCFUmqa1nq106ETtdaD/T2f8J9jnX6bHetZd2pTnpixjpKq1XxK3LKSVvjLY6fMrl1h6VLo08f8/dNPYcAAS+ONFoGYtL0AWK+13qi1PgZMAK4KwOuKICmt09eac3ZuYFXDFlJpI0LGfsGRV1CI5kSnzKycvBMHnX02LFtmhhkHDjQVZGE6/BxJApHwU4AtDp9vtT3m7Bql1C9Kqa+UUi5nY5RS9yilspVS2fn5+QEITbhir6hpdHA3dQ/v59cGZ5R5XIhg8tSCuYy6dc2+uTffDE8/DY89FsIoo1MgEr6rJXbOb8XfAM201u2AOcBYVy+ktf5Ia52mtU6rV096agSLvaKm7Y71APzasEWZx4UIpgq1YK5aFcaOhUGD4I03YMmSIEcX3QKR8LcCjlfsjYEya+211nu01kdtn34MnBeA8woPsnLySB8xl+aZM0gfMbfM7fLTbRO5M3cGDy6ZQLGqxOr6zaVnvQgZXxZolaEUvPaaKdO85x44diyI0UW3QCT8FcCZSqnmSqkqwABgmuMBSqlGDp/2BX4PwHmFG+7GSH/4dBq0bUuPPh0ZPuv/qFl8lFcvvZVT69WWskoRMr4u0CqjRg14/3347TeT/MVJ8btKR2t9XCk1GJiFKcv8RGv9m1LqOSBbaz0NeFAp1Rc4DuwFbvP3vMI9V2OkiQV7OOfBh6BODXNrfMUVND3rLIYBw6wJU8QoXxZvuXTllWbNyHPPQf/+poRTVIgsvIpCzm1slS5h7KSnuWDrb1RbuQLatbMsNiH8sm2bWRGelgZz5khbbhekPXKMcR4LvW/pf+m8OYe3+g6WZC8iW3Ky2Wxn7lyzUbqoELnCD1P2hSl5BYXEKUWx1qT4eOvruLDqwr9W8cWEJ5nZujPF4z4j49zGIfoOhAiSkhLo1AnWroU1a0z5pigV9NYKIrCcd6wqtr0p5/nYe97+3KivV/DONyPJOzUZNWqUJHsR9tz1byo33v/hh5CaCkOGmLJN4RO5wg9DrjYMd+TzNnD2xmjz5kGXLoELUIggcLU1Z3wlBQqKik/kqdJmfV99AC+9BD/8AJ07WxFyWJIx/AjjbcWrzytit241H9u29TMiIYLPVXVZUYkuk+zBYVXuk0+arpoPPADHj4cy1IglCT8MeVvx6ul5xwVXX0xeTHHVqlCnTqBDFCLgKtLaY1tBISQmwptvwi+/wKhRQYwsekjCD0OedqzytEDFecHVKfk72Jp4qmwyLiJCRVp7lB7brx9cfjkMHw67dgUpsughCT8M2ff9TLH9p46z1RqnJCV4XBHrfEvc6OBu8k6pK10wRURwdaETX0kRH1e21r7MRY9S8M47cOgQPPVUqEKNWFKlE6Yq2nM+Kyev3ERvw4O7Wd6kjXTBFBHB3QpcV4+V+d04+2y4/Xb44guT/KtVsyL8iCAJPwrYh3IcVSoppsGhvWyvWU+6YIqI4e5Cx+vFT0YGfPyxqdjp2TNI0UU+GdKJAq6qG5IP5BNfUszupPrSBVNEv65dISHB7IMr3JKEHwVcDdn0XrsYgIv/OUC6YIrol5AA3bvD9OmyM5YHkvADyFMP+mByNWTT77d5/NqkFT36pockBiEs16cPbNpk2i0IlyThB4hP+3QGiXN1w9m7NtEqfzPFN9wY9HMLETZ69zYfp0+3No4wJgk/QHzepzMIHMs4FXDThkWUVK5M+yH3BP3cQoSNJk2gfXv4739lWMcNqdIJkArt0xkEpdUNR47A6XebSgXZF1jEmvvvN9sgTpgAAwdaHU3YkSv8AKnwPp3B8vHHsH07PPpoaM8rhEUc58465Tej4Oy2MHSoWYwlypCEHyAntU+nr/bt8+24wkLTPfDSS02ZmhBRznnubMuBY9x70e2Ql2d+F0QZkvADxHkc3VsbBJ/Nm2ean82Z4/6YzZtNE6kuXWDHDnj2Wdn6TcQEV3NnSxucxbepl8Prr8Mff1gUWXiSMfwAqmg7BJ/MnGk+LlkCl11m/q41rFoFWVnw9deQm2seb9sW3n7bXOELEQPczZH9++Jb+MefK+G668zvTmJiiCMLT5Lww92ePeZjUREsXmwSfFYWbNhgruI7doSRI83S8hYtrI1ViBCx74zlrhanSuNkGD/elGrefTd8+CHUqBHSGMOR7HhlMVdbupW5S+jYEZYtO/F5fLxZUdivH/TtCw0bhj5oISzkamcsR6U7YqWmwAsvmNbJtWrBXXfB4MHQrFloAw4xTzteScK3kKv/uGX+s5aUQM2a8Pff5tY0I8NcsdSqZWHUQljL0xagKa4umpYuNUOdX31lhkP79oWHHjJDn1E41yVbHIahrJw8hkz62fNirU2bTLIfPRomTjR1xZLsRYxzN26vgMWZ3UqTfWm55tS9pHe4h++mL4XMTFi40FSxdegAY8aY6rYYIQnfAvYr+2I3d1el/6Htdzjt2oUoMiHCny9rXly1Onlk8R6yrr0PtmwxF1FghnmaNIGpU0MQufUk4VvAVSmZo+SkBFN7/8QTZrxREr4QpXxZ8+Kx1UlCAtx5p6lumzfPDPNMnhyS2K0mVToW8NRuISE+jqE9zjKVBXl5sGgRVK0awuiECG/udsZyHLd3N8Zf5ndPKbN2pVkz2Ls3iBGHD0n4FkhOSnD5HzJOKTNhu/wbc8Xx6qtkVWnMyBFz3VfxCBGDXK15sVe8uUv24GY4qE6dmEn4MqRjAXe3pK9f156MuD3w8MPQqxdZ3QZY1nJZiEjiOGbvjgLXrU4k4YtgctuG4awkuP56qF0bxo5l5Ox1lrVcFiKSeJsXA9C42Rs3hhK+DOlYxGUbhjvvhLVrYfZsqF/f8pbLQkQKX34nUtx1rrUnfK2jsi7fkVzhh4svvoBPPoEnnzQraQmjlstChDlvvxPuOtdm5eTxbu4eKC7m8menR/1wqST8IPJ5j9v162HQIEhPh6efLn04qC2XhYgirn5X7Nfq7jrX2sf9/9LVACjcmR/1c2QypBMkzm0T7BOu4DSOqDXccIPpkfPFF1D5xD+JL+VnQoiT+12xj/sXJJwCQK3Cg2y1zZFF6++YJPwg8bTwo8x/ppwcWLECRo2Cpk3LvU5QWi4LEYUq+rtiH/ffX8100Uw6cqjM49FIhnSCxOcJ18mTIS4OrrkmBFEJIezs4/4FtoRfy5bwo3mOTBJ+kPg84TpliunaV7duCKISQtjZx/1PXOEfjPo5Mkn4QeLThOuqVbBmDVx9dYijE0LY18NUTW7E7sRa3PHzTF7p3SKqh1ADkvCVUr2UUmuVUuuVUpkunq+qlJpoe365UqpZIM4bzrzucbt6NfTpY3bhkeEcISyRkZrCgid7UHfyBFrs2ETf/7xqdUhB5fekrVIqDngfuBzYCqxQSk3TWq92OOxOYJ/WuoVSagDwCnC9v+cOd24nkRYuNJswVK0KP/wgu1YJYbVevWDYMHj5ZTPEeuONVkcUFIGo0rkAWK+13giglJoAXAU4JvyrgGdsf/8KeE8ppXS4brcVQM5bGL5ZaR0XDH8QmjeH776L+u3WhIgYzz1nLsYGDYK0NGgZfWP5gRjSSQG2OHy+1faYy2O01seB/cCpzi+klLpHKZWtlMrOz88PQGjWct6EodfsL7ngiX+yp3V7WLJEkr0QYaB0geRTs8i4ZDBH46tC//5RuRNWIBK+q+YTzlfuvhyD1vojrXWa1jqtXr16AQjNWvZafKVLGP79xwyfO5qZZ13MtVc/Z/p3CCEs5XxRlqtrMPgfj5iCiocesjq8gAtEwt8KNHH4vDGwzd0xSqnKQC0g6tvTbSso5Ibcb5k1ZjB3Zk9lTNpV3J+Ryea/PXf1E0KEhqsFkrObpjKuy0D4+GMYP96iyIIjEAl/BXCmUqq5UqoKMACY5nTMNOBW29+vBebGwvh9Zs4UXpr1PoerVOOhPkN4vvvdaFUpqhd2CBFJ3C2QfO78AXDJJWY8f230tCP3O+HbxuQHA7OA34FJWuvflFLPKaX62g4bA5yqlFoPPAqUK92MOm+9xaD/fcLUtt3pd/NrTD2nKyDNz4QIJ+4uvhrUqQFffmn2v+3fHw4fDnFkwRGQOnyt9Uyt9Vla6zO01i/aHvu31nqa7e9HtNb9tdYttNYX2Ct6ooljZ8xXrn4UHnnE1NePGUNy7equa/GFEJbyuECycWMYO9aM548bZ1GEgVKGykgAABLXSURBVKXCdWQlLS1NZ2dnWx2GTxw7Y/Zes4j3pr7CwhZpFIyfyFUXNLc6PCGEB86l02W6bGoNLVpAq1Ywfbq1gfpIKbVSa53m6jnplhkAjhM/N+bOZHPtRtzTN5O6czdJwhcizHnssqmUWRH/0UdmWCcxMbTBBZj00gkAx4mf6scK+SupEUfjq0Z1m1UhYkafPnDkCMyda3UkfpOEHwCOEz+Jx45yOL5quceFEBGqc2fT8ypChnQ8kYQfAI4TP4lFhRRWqSbVOEJEi6pVTa+dSZNg926ro/GLJPwAcOyMmVh0FGrUkGocIaLJM8/AwYPw+ONWR+IXSfgBkpGawuLMbtQpOco1nWTfWSEihWNJdfqIua43MT/nHHjsMfjPf2DBgtAHGSCS8AOpuBiOmit8IUT4c+6lk1dQyLApq1wn/eHDTcPDe++FY8dKv97rm0UYkYQfSH//bT5Wr25tHEIIn7jqpVNYVMzIWS7aKSQmwnvvmc2L/v1v5n8+g8/en0LVDevQWnt+swgTUodfAR4XaAAcMpsgyxW+EJHBXem025LqK64wW5K+8gpdeIUutof73zCCFU3alL5ZhOuQriR8HzmupoUTt37Zf+5l3pp8thUUcsHxPUwEucIXIkIkJyWQ5yK5eyyp/vJLmD+fOz5eQq3Cg7w54w1a7v6LFU3aAB7eLMKADOn4yN2t3/hlf5WO/x3aUwDA8l1HLYhQCFFRHnvpuFOlCvTowdrzOpN1TheOxsXTpGBH6dPhvP5GEr4PsnLyXF4FQNldXBKKjgDw5W97QhCVEMJfjiXVFW1wOLRnS6pViWdrrQalCT/c19/IkI4X9qEcXyQeMwn/r2PyPipEpPDYSwf3c3f2r9k1qRFN9u8kxdW8XpiRhO+Fq6EcO0XZK/xE2xV+jTq1gh+YECLo3M3dgcMbRfc0mDCBxZndrAzVJ3Ip6oWnCZgbL2paZvyvTuEBAG66vE3Q4xJCBJ9PZZunnw779kFBQYijqzhJ+F64m4BJSUrghYy2Zcb/rtz0I4cbptCj94WhDVIIERQ+lW02t7VA37QpBBH5RxK+F95m8e0tFTY9fB4d168k8Y5boZL8WIWIBu4u+Mo8Lgk/evg8i//ll1BSAjfdZEmcQojA63p2PZTTYwozll/aSuH0080TG8N/51ZJ+D6wX8W/eX0HAB6ZmFu+b8bnn8N555mt0IQQES8rJ4/JK/Nw3gTW/nlpK4VNf5uk//33oQ6xwiTh+8hjk6XVq+Gnn+Dmm60OUwgRIJ4q9OxKJ3CvucYk/H37QhTdyZGE7yOPs/Wffw5xcTBwoEXRCSECzdcWCdsKCuHaa6GoCL75JshR+UcSvo/c/ePnFRSy6pNJrDytLVl5RSGOSggRLL62SEhOSoDzz4cmTWDy5CBH5R9J+D5y949fregIrXZtYkmDlmHfGlUI4TtXFXrOSiv2lDLDOrNmwYEDIYqw4iTh+8jdP36bnRuorEvITT7LfR9tIUTEcVWhd9NFTd1X7F17rdkAado0K8P2SFor+Mj+j2rvqaG1pse6ZTy68HOKVSVyG5m6/HBujSqEqBhvfXbK6NgRTjsNxo0L2/JsSfgVkJGaQkaHZJgxgzWDHuXsbevYWDuZ+67KZE/1JCC8W6MKIYKoUiW49VZ4/nnYssWM6YcZGdKpCK2hWze48kqaVDpGZt8hXH7X/zGr5cVA+LdGFUIE2W23QeXK0K8f7N1rdTTlSMKnAhsRHz8O8+fD7bdTfeM6LnrmERrWqVHhPtpCiCjVvDlMmQKrVpmLw/x8qyMqI+aHdLy1Py1D29bYtWgB8fEVG98TQsSGPn1MPf5VV0HXrjBnDjRsaHVUgFzhV2zXeu28yFoIIVzo0QNmzDAN1bp0gbzwKNeO+YRfoV3r7QlfObdTEkIIp+HhH2HhO5+ZZH/ppfDXX1aHJwnfp/anziThCyGcuOq3dc/mRH54bzzs3m2SvsUtlGM+4Vdo1/oVK8zH+vVDEJkQIpK4Gx7+1/YaprHa/v0m6a9fb1GEkvArtmv9iy9CvXowYEDI4xRChDePw8PnnQfz5kFhIXTuDGvWhDg6I+ardMDH1XTZ2aZPxssvQ2JiaAITQkSMWgnxFBSWb6BYOjzcvr0p6+7e3Vzp//QTpIS2yi/mr/B99vrrUKsW3Hef1ZEIIcJMVk4efx87Xu7x+Eqq7PDwOefA3LlmeGfo0BBGaEjC98XBg5CVZfpj1KxpdTRCiDAzctZaiorLl23XqFa5/OhB69bw+ONmW9QFC0IUoeFXwldK1VFKzVZKrbN9rO3muGKlVK7tT/i2knMnKwuOHIEbbrA6EiFEmMnKySPPzfh9wWE3e2RkZkLTpvDAA2YFf4j4e4WfCXyvtT4T+N72uSuFWusOtj99/Txn6I0fD82amW54QghhYy/FdMdteXdiohkm/uUXGDUqSNGV52/CvwoYa/v7WCDDz9cLGZ/75+zcaZZGDxwo9fdCiDI87XvrtZniNdeYCdzhw6GgIEgRluVvwm+gtd4OYPvorkC9mlIqWym1TCnl9k1BKXWP7bjs/CA2HfK4Ibmz//4XiovhxhuDFo8QIjJ52v/CazNFpUzVX0EBTJwYhOjK85rwlVJzlFK/uvhzVQXO01RrnQbcALyllDrD1UFa64+01mla67R69epV4OUrpkL9c774Atq1M7PrQgjhwN2QTUpSgm+NFdPSzCTuuHEBjsw1rwlfa32Z1rqNiz9TgZ1KqUYAto+73LzGNtvHjcB8IDVg38FJKPeurDUZv83jWN62so9v3AhLl8pkrRDCpQqt1HdFKbNpypIlIVmB6++QzjTgVtvfbwWmOh+glKqtlKpq+3tdIB1Y7ed5/eL8rpz+58+8Nf11bt60pOyBEyaYj7KyVgjhQoVW6rtz440m8YfgKt/flbYjgElKqTuBv4D+AEqpNOCfWuu7gFbAh0qpEswbzAittaUJf2jPlmV64N+77L8AXN60+omDtDbVOZdcYvapFEIIF/zeFyMlBS67DD77DJ55xmyVGCR+JXyt9R6gu4vHs4G7bH9fArT15zyB5rghed3ff+aSP38GoFVS/ImDVq2C1avhgw+sCFEIEUtuuQVuvhmWLYOLLw7aaWJ2pW1GagqLM7sx9dAiSEqChAQ4fPjEAV98Yfam7N/fuiCFELGhVy/zMcgrb2M24QOmY93XX8P995uWCYW2ydySErPsuUcPqFvX2hiFENGvbl1o2RIWLQrqaWI74b/6KlSrBg89ZK7w7Ql/yRKzO43U3gshQiU93eSekpKgnSJ22yPv2gWffw6DBpke94mJJ4Z0xo83n/eNvC4QQojIkJWTx8hZa9lWUEhyUgLvNjmHc/d9YkYeWrcOyjlj9wp/6VIoKjItE+DEFf6xYzBpktlxvkYNa2MUQkQlV6v9n9xl68S7eHHQzhu7CX/lSlP+1KEDWTl5/LznKEt/3cJj97wGe/fKYishRNC4Wu3/+ykN2Vc9Kajj+LGb8LOz4ZxzyFq7j2FTVrGfeKoVHeOSFf+jIOEUptaXVgpCiMCyN2102U5ZKVYkny1X+IFS2iHziensXbiUP5u1Kn2nPRJfldqFB+ixbhkzWqbz6lxrd5cXQkQXx2Ecd/5o0R42bDBdeoMgZhK+4w+74cHd1DlUwLhjp5b+8AsrV6VZwXYSi44ytXUXj13whBCiojy1UgbTg6ftdb3NJ0G6yo+ZKh3HH3bbHaZJ0U/1ziBOKYq15kjlKgDknVKPFY1bu9+4QAghToKni8iUpASG9mzJpa3rwrPNzJ63QRAzV/iOP+w2O9ZzXFVidf3mFGtNQnwcR+JNwv+mdWeqVYn3vdudEEL4wFMr5aE9WzJy1lqaPz2H9EFjyOrQIygxxEzCd/xht9uxnnV1m3I0vmppd7vKNUzjtKUX9qx4tzshhPDCXSvlrmfX831DJj/FTMIv/WFrTZud61nVsEVp3+qM1BRufO0xeOopxr51lyR7IUTAuWulPG9Nvu8bMvkpZsbw7Ul87MSF1D28ny3NW5W9kj//fPNHCCGCxFUr5Ucm5ro8NhiFIzGT8MH2w95cFYAh/7oR5EpeCGGx5KQEl6WawSgcicohndJ6+8wZpI+YW3YsbOVKiIsz+9QKIYTF/N4msQKi7grfXm9vHxOzT4CAbVhn5UqzIXmClF0KIaznuCGTvZGafW4x0KIu4bta3GCfAMlIrmx2lOnXz6LohBCiPL+3SfRR1A3puJvoOLx9J1x+uemGee+9IY5KCCGsF3UJ39VER80jh/hy8jPwxx8wbZpU4wghYlLUJXznCZDqRw8zdvKznLVzE0yZAt3L7bkuhBAxIerG8B0nQPbm7+OzaS/SfvsfVJo0CXr3tjg6IYSwTtQlfLBNgLSua7Yo3PSL2crw6qutDksIISwVlQkfrc3Whf/7H4wZI7tXCSEEUTiGD8COHfD11zBsGNxxh9XRCCFEWIjOhL9nj/mYmmptHEIIEUaiM+Hv3Ws+1qljbRxCCBFGJOELIUSMiM5JW0n4QogIlJWTF9SeOpLwhRAiDHht/BgA0TukU7ky1KhhdSRCCOETT40fAyXqEn5WTh5Z369id5UapL8yLyj7QgohRKC5a/wYyJ2voirh22+JKu8voKBajaBuBiyEEIHkboerQO58FVUJ335LlHTkIAUJpwDB2wxYCCECKRQ7X0XVpK391mfUhddSueR4uceFECJchWLnq6hK+PbNgBc1Ty33uBBChLtg73wVVUM6odwMWAghIk1UXeGHcjNgIYSINH4lfKVUf+AZoBVwgdY6281xvYC3gThgtNZ6hD/n9SRUmwELIUSk8XdI51fgamCBuwOUUnHA+8A/gNbAQKVUaz/PK4QQooL8usLXWv8OoJTydNgFwHqt9UbbsROAq4DV/pxbCCFExYRi0jYF2OLw+VbbY+Uope5RSmUrpbLz8/NDEJoQQsQOr1f4Sqk5QEMXTz2ptZ7qwzlcXf5rVwdqrT8CPgJIS0tzeYwQQoiT4zXha60v8/McW4EmDp83Brb5+ZpCCCEqKBRDOiuAM5VSzZVSVYABwLQQnFcIIYQDpfXJj5wopfoB7wL1gAIgV2vdUymVjCm/7G07rjfwFqYs8xOt9Ys+vHY+8OdJhFUX2H0SXxds4RoXhG9sElfFhWtsElfFnWxsp2mt67l6wq+EH46UUtla6zSr43AWrnFB+MYmcVVcuMYmcVVcMGKLqtYKQggh3JOEL4QQMSIaE/5HVgfgRrjGBeEbm8RVceEam8RVcQGPLerG8IUQQrgWjVf4QgghXJCEL4QQMSKqE75S6jGllFZK1bU6FgCl1PNKqV+UUrlKqf/Z1itYTik1Uim1xhbb10qpJKtjslNK9VdK/aaUKlFKWV4+p5TqpZRaq5Rar5TKtDoeO6XUJ0qpXUqpX62OxZFSqolSap5S6nfbv+NDVscEoJSqppT6USn1sy2uZ62OyZFSKk4plaOUmh7I143ahK+UagJcDvxldSwORmqt22mtOwDTgX9bHZDNbKCN1rod8AcwzOJ4HHltwR0qYd7q+1Ogl9VBuHAcGKK1bgVcBNwfJj+zo0A3rXV7oAPQSyl1kcUxOXoI+D3QLxq1CR94E3gcN43arKC1PuDwaXXCJDat9f+01vZd35dh+h2FBa3171rrtVbHYVPa6ltrfQywt/q2nNZ6AbDX6jicaa23a61/sv39ICaJWb5DkTYO2T6Nt/0Ji99HpVRj4ApgdKBfOyoTvlKqL5Cntf7Z6licKaVeVEptAW4kfK7wHd0BfGt1EGHK51bfojylVDMgFVhubSSGbdgkF9gFzNZah0VcmDY0jwMlgX7hiN3T1lPbZuBfQI/QRmR4ayettX4SeFIpNQwYDDwdDnHZjnkScws+PhQxVSS2MOFzq29RllKqBjAZeNjpTtcyWutioINtzuprpVQbrbWlcyBKqT7ALq31SqVUl0C/fsQmfHdtm5VSbYHmwM+2nbgaAz8ppS7QWu+wKi4XvgBmEKKE7y0updStQB+guw7x4owAtOAOFWn1fRKUUvGYZD9eaz3F6nicaa0LlFLzMXMgVk96pwN9bQ0nqwE1lVKfa61vCsSLR92QjtZ6lda6vta6mda6GeaX9NxQJHtvlFJnOnzaF1hjVSyObJvMPwH01VoftjqeMCatvitImauuMcDvWus3rI7HTilVz16NppRKAC4jDH4ftdbDtNaNbblrADA3UMkeojDhh7kRSqlflVK/YIacwqJEDXgPOAWYbSsZHWV1QHZKqX5Kqa1AR2CGUmqWVbHYJrYHA7Mwk4+TtNa/WRWPI6XUl8BSoKVSaqtS6k6rY7JJB24Gutn+b+Xarl6t1giYZ/tdXIEZww9oCWQ4ktYKQggRI+QKXwghYoQkfCGEiBGS8IUQIkZIwhdCiBghCV8IIWKEJHwhhIgRkvCFECJG/D93auf9SR44sAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(-4, 4, 100)\n",
    "y = 1 - (x/3) ** 2\n",
    "x1 = x + np.random.randn(100) * 0.1\n",
    "y1 = y + np.random.randn(100) * 0.1\n",
    "plt.scatter(x1,y1)\n",
    "idx = x1.argsort()\n",
    "beta,avg,res = 0.7,0,[]\n",
    "for i in idx:\n",
    "    avg = beta * avg + (1-beta) * y1[i]\n",
    "    res.append(avg/(1-beta**(i+1)))\n",
    "plt.plot(x1[idx],np.array(res), color='red');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It works particularly well if the loss function has narrow canyons we need to navigate: vanilla SGD would send us from one side to the other while SGD with momentum will average those to roll down inside. The parameter `beta` determines the strength of that momentum we are using: with a small beta we stay closer to the actual gradient values whereas with a high beta, we will mostly go in the direction of the average of the gradients and it will take a while before any change in the gradients makes that trend move.\n",
    "\n",
    "With a large beta, we might miss that the gradients have changed directions and roll over a small local minima which is a desired side-effect: intuitively, when we show a new picture/text/data to our model, it will look like something in the training set but won't be exactly like it. That means it will correspond to a point in the loss function that is closest to the minimum we ended up with at the end of training, but not exactly *at* that minimum. We then would rather end up training in a wide minimum, where nearby points have approximately the same loss (or if you prefer, a point where the loss is as flat as possible). Here's how the above chart varies as we change beta:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide_input": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtEAAAHiCAYAAAAuz5CZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3iTVRvH8e9pKTRllQ0tG9kgFCpDcLBBVkV8AVFBRUDEiSjgHkABxYELRMEByrSioAwBlVFZBZElG9qyocxC13n/eJKSpkm60qRp78919Wrz5El6CumvJ+c55z5Ka40QQgghhBAi83w83QAhhBBCCCG8jXSihRBCCCGEyCLpRAshhBBCCJFF0okWQgghhBAii6QTLYQQQgghRBZJJ1oIIYQQQogskk608ApKqSNKqY6ebocQQoiMSWaLgkA60SJfy60gV4ZJSqlz5o/JSinl4Ny7lVIpSqkrVh+DXN0mIYTwdnkks8fZ5HW8OcPLurpdwrsV8nQDhPBSQ4EwoAmggZXAIeBzB+fHaq0ru6ltQggh0sp0ZmutJwATLLeVUm8Ad2qtz7qlpcJryEi08Ca3KaV2K6UuKKVmKaX8AZRSPZRS25VScUqpDUqpW83HvwWqAj+bRxNeNB9foJQ6qZS6qJT6UynVMBttGQS8p7WO1lrHAO8Bg13yUwohRP7g9ZltHq1+CPg6G99T5HPSiRbeZCDQBagF1AFeUUo1A74ChgFlgOnAEqVUEa31Q8AxoKfWupjWerL5eX4FagPlgW3AHMs3UEqNMQe73Q+rtjQEdljd3mE+5kh5pdQppdRhpdT7SqmiOfh3EEIIb+DNmW1xB1ABWJTln17ke0pr7ek2CJEhpdQRIFxr/bn59j3ANGAFcFZr/arVufuAoVrrP8yPG6K1XuXgeQOBC0Cg1vpiFtqTDDTUWu81364N/Af4aJtfKqVURaA0sBeohjGisUdrPSyz308IIbyJN2e2zeO+BHy11oMz+71EwSEj0cKbHLf6+igQhNEpHWUz8lDFfF86SilfpVS4UuqgUuoScMR8V1YXjFwBSljdLgFcsRfGWuuTWuvdWusUrfVh4EWgbxa/nxBCeBuvzGyr720C7kemcggHpBMtvEkVq6+rArEYIT1eax1o9RGgtf7efJ5tQD4A9AY6AiWB6ubjCuyuyk7zYfU8uzAWqFg0MR/LDG35fkIIkY95e2b3Ac4DazPzw4qCRzrRwps8qZSqrJQqDYwD5gFfAMOVUi2VoahSqrtSqrj5MaeAmlbPURy4AZwDArBagQ3GqmzzXDy7H1anfgM8r5QKVkoFAaOA2fYarYwSd1XN7asChAM/5fQfQwgh8jivzGwrg4BvnI1Wi4JNOtHCm8zFmE93yPzxjtZ6C/A48DHGPLkDpF1xPRFjMUucUuoFjCA9CsQAu4HIbLZlOvAzsBP4F1hqPgaAeRTkDvPNZsBG4CqwwXz+09n8vkII4S28NbNRSgUD7c3fXwi7ZGGhEEIIIYQQWSQj0UIIIYQQQmSRdKKFEEIIIYTIIpd0opVSXymlTiul/nVw/0Cl1D/mjw1KqSZW9x1RSu1Uxu5FW1zRHiGEEI5JZgshRM65aiR6NtDVyf2Hgbu01rcCbwMzbO5vp7VuqrUOdVF7hBBCODYbyWwhhMiRQq54Eq31n0qp6k7u32B1MxKo7IrvK4QQIusks4UQIudc0onOoseAX61ua2CFUkoD07XWtiMe6ZQtW1ZXr149l5onhBC5Z+vWrWe11uU83Y4skMwWQhRYzjLbrZ1opVQ7jEBua3W4jdY6VilVHliplNqrtf7TzmOHAkMBqlatypYtMhVPCOF9lFJHPd2GzJLMFkIUdM4y223VOZRStwIzgd5a63OW41rrWPPn08CPQAt7j9daz9Bah2qtQ8uV86ZBHCGE8D6S2UII4ZxbOtFKqarAYuAhrfV/VseLWrb6VEoVBTpj7CQkhBDCQySzhRAiYy6ZzqGU+h64GyirlIoGXgf8ALTWnwOvAWWAT5VSAEnmVd0VgB/NxwoBc7XWv7miTUIIIeyTzBZCiJxzVXWOARncPwQYYuf4IaBJ+kcIIYTILZLZQgiRc7JjoRBCCCGEEFnkiRJ3QniViKgYpizfR2xcPEGBJkZ3qUtYSLCnmyWEEMIOyWzhLtKJFsKJiKgYxi7eSXxiMgAxcfGMXbwTQEJZCCHyGMls4U4ynUMIJ6Ys35caxhbxiclMWb7PQy0SQgjhiGS2cCfpRAvhRGxcfJaOCyGE8BzJbOFO0okWwomgQFOWjgshhPAcyWzhTtKJFsKJ0V3qYvLzTXPM5OfL6C51PdQiIYQQjkhmC3eShYVCOGFZiOJopbesAhdCiLxDMlu4k3SihchAWEiw3ZCVVeBCCJH3SGYLd5FOtBDZ5GwVeFYCWUZGhBAi90lmC1eTTrQQ2eSKVeAyMiKEEO4hmS1cTRYWigIpIiqGNuGrqTFmKW3CVxMRFeP8AbNmwe7daQ65YhW41DQVQojMyXJu25DMFq4mnWhR4FhGEmLi4tHcHElwGMjR0fDoo9CkCbzwAly6BLhmFbjUNBVCiIxlObftkMwWruaSTrRS6iul1Gml1L8O7ldKqY+UUgeUUv8opZpZ3TdIKbXf/DHIFe0RBVdmRiqyPJKQmGh8rlcPpk41Ps+ZQ1jTICb2aUxwoAkFBAeamNincZYu6UlNU+EJktkir8js6LIrRoDDQoIls4VLuWpO9GzgY+AbB/d3A2qbP1oCnwEtlVKlgdeBUEADW5VSS7TWF1zULlGAZHauWpZHErQ2Pr/wAtSvD08+CQ8+CNOnE/bxx4SNaZ/tNo/uUjdNm0Fqmgq3mI1ktvCwrMwvdtUIsKPKHZklmS2suWQkWmv9J3DeySm9gW+0IRIIVEpVAroAK7XW580hvBLo6oo2iYInsyMVWR5JSEkxPvv4QIsW8PffMGOGMUe6SRMoXhyWLs1Wm10xMiJEVklmi7wgK6PLWc7t8eNhwAB45x1YvBj27YOkpBy3WTJbWHNXdY5g4LjV7WjzMUfHhXDKXomhzI5U2BtJUBijIG3CV6crV7Ry1wk6Ac/N38GmmGDj/scfZ1Xp2nTs2w6uXCGpVy92vhxOyFujM91ey/fI6ciIELlAMlu4VE4yG7KW2xFRMbSe8iGBV+Io8sMPqecn+xXmcJlg9pUMJrJZO0Kfe4zet1XLUpsteS2ZLcB9nWhl55h2cjz9Eyg1FBgKULVqVde1TOR5tkHWrl45Fm2NSXcJMDDAjwvXEtM93nakwnpHq5i4eBQ3X3S2lxMjomKYvnwfnYBkpVLv33L0PKs2nqGj+XG7ytci5O0X2XsylnrTp4JSqW1/Y8ku4uJvtismLp7n5m1ny9HzvBPW2DhomTKi7P1KCOF2ktkiR6xzu6TJj6sJSSQmGy+VrGY2ZD63AcYu3sma5GQWN2zHWx2G0vBiDP2LXSZu83aGbFzILScP033fOmKXfs6/gx+n0RsvQKlSadpt73uky21R4LmrOkc0UMXqdmUg1snxdLTWM7TWoVrr0HLlyuVaQ0XeYm9F9pzIY3YvAWptzE2rcPksShtTMBzNVQsLCWb9mPYEB5rS9QCsLydOWb6PhETLJUCVev/3fx/nrCqc+pgH+o9nfuOO1PviA3jsMThwgF9XbeeNeVuIu5aQ7vtrYE7ksZuLaEaMgHbtICH9uUJ4gGS2yDbb3I6LT0ztQFtYZ7Y1Z/OLM5PblikiPlqjlQ/xhf3ZUq4WLwWE8M6dg1PP/7RVXw6XqkSjjyZA5crQtSvH7+nDpWFP0veXL6l17ni675Eut0WB565O9BLgYfOK71bARa31CWA50FkpVUopVQrobD4mCoDsVtKwO+wFXIxPZFqzANZNH8KgbUszNVct9sI1msTuY8yar5iy9APKXI0zjpsvJ549e4mmsf+l+77JWpPke/NCztUiAbzY7Rk+aDPAqClduzbdOoWwfVIY/717L712/wGAb0oyQZdOpz7flOX74MQJmDkT/vgD3n7b2T+ZEO4imS3SyUklDXsuxidma36xs2kglvt8dArJPje7OMk67V+OL0PDGNh/Avc88pExd/r8eXz+jiTsn1U8t34usxa8gU+K8TOUvnYx9XGpuR0ZCcOHQ4x0qAsyl0znUEp9D9wNlFVKRWOs3vYD0Fp/DiwD7gEOANeAR8z3nVdKvQ1sNj/VW1prZ4tdRD6R00oaALecPUbn/ZEsq9uGI6WDCQo00fG3OZCcxBtHV/PGi58aiwEdWbWKDTOGUCnuFAk+hdBKERK7l6d7vUiby8ehz5dELf2VgITrXPXz51Dpm+3yVSpdKKMUszsNZk/j1hQ7fpiAhHiKJcTz8NZfCNu1hmV12zBj8TvceXgbYQ9P5d+Ktxg/31dfGQteOneGCROgRw9o2TIb/6pCZI5ktsgqV1TSsBUUaDLmF9csBgEB4OeX6cedPXuRRicPsLdcda4WCQCgnl8CdU4coOLB3RRLiCfFanqcJbM/at2PfjtXcq5ooNHWanVpU6QhsWXvRZsLLXXdt57PIyYybNNiWh/9hzuPRPF5y/sIv2swKEXKsePQazScOQPz58Pnn8P//peptov8RWnbjoAXCA0N1Vu2bPF0M0QOhLy1wu5cuOBAE+utSsa1CV9NjINAfmf5Jzy4/VcAIqs04tfG7Ri3/HPOlihL8IUTsGIFdOpkvwF//QVdunCpUhUmNOzBspotqHvmCF8tfJPiCcb3iy9fkZN3dGRSodqsDW7IdT9/wLjceF/zYBZtjWHPO90AqP7SL/j5KFCku2z5+qrp9N+xgpm3hfHUxnlcLmziaKkgej88laBAE39NHwK1a8OiRXDrreDvD1FRxh8VnC9KFN5HKbVVax3q6Xa4k2S298tsZoPz3LYW4OeD3+VLrJw1Er8qwZTatCE19+zSGtau5egH0ym9/GeK37jGDV8/tgXXo0rcKSqbr/IBRJcoz5sdh7Kydqs0mW09Qu4os31Sklk7YyhVL57ign9xooLq0v7QFuY07cpb7R9n8YJXaHjhOMyda1T/2LQJRo+GSZNAKcnsfMZZZrtrYaEQqSKiYuyGMWS+koYGgi+d5kDpyixq3IF+O1bw5rJpAAwJG8t3817lxsT3CLLXid6yBbp3h6pVKfHnn7SKSWT5z7vY7F+Mvg9Ooe2R7URWbcyh4NpMvO9WugI77QRiaLXSvHB8AgeTCxMcaOJaQpLdn+uPGs15ZOvPPLVxHr/Uu4Olddvw2U/hDNu2hDt63wnHjsG77xJx6Aq/d36aaTNfYH6nByn8sfHzZHb0RwghckNWMhvs5zYAWlPjQizJyocrRQK4UjiA11d/SenL5/HZdY7osP5U/i3C/hXErVuNWv1r11KteHG23d6Rr4rXo3nMHm6L3k1UUF1+CO1B6/s7EVevERMiTxMbF0+wTWZbd24dZXaKjy+vdB5By+P/8kWLe4nzL87oP7/hycgFtDu0zZiSt3Ah9OzJTxVvJfmpp+kzZQq/rtrOxnHhLNhxWjK7gJCRaOF2zkYp7I1qRETF8MmSKO788yfqx58l8HQMwZfOUOtcNGtqhTKszyugNa2O76T4jWusrN2KF/+YzbC/F+N77KixaMTi33/hrrugRAljNNp8n6M22WuPIzXGLLU7X9s/8To7PhxAoq8ffZ6ayX7f4nz78wRaHY6iUJMmcOQIPy3ZyJifjXmEr6+aziNbf2bQg+H8Uzsk06M/wjvISLTwNlnNbEh7Bc2Si20PR/Hd/FfTnftZy75cMBVn3NpZMG6cMbprmYpx/Di8/DJ8+y2ULQuvvw6PPUabDzfmWmaDMVhjqQa1Zu8ZYuPiGb0jghG/zYQXX4RJk25OcUlI4qkNPzBq3RzW1GzOiN5jiS/sn+12ibxFRqJFnpJm5ELrNGXdHFXSCJv2KqyeBSVLst9UhiMlKxBZtTELG3UwTlKKyKq3pj5mbtNuDI9cBB9/DOHhxsH9+6FjR2O6xO+/p+lcu2I3rKBAk91QL1OuFEU+mEqRKlVY0bu3cXBkE2jQwNi4ZexYJq8+nDpyMemuQdx1aCvjl0yl66MfQ5H0lzezukuXEEJkl7O8cVZJwzLyaumE1z99GIAxXUbil5JE8RvXuF6oCHNCunHD149a52PoN2FC6tU5PvoIpk41/k689BKMHQslSzptkysy23GHtzscfy31b0fqAkqlmNZmAGeKlmL8ik/5/odxvNjtac4WLcX5AOftFd5NOtHC7ayDa/aCNzAl3aDfA+EEmvzsX+46ftwYhRg5EqZNY5fNAhd7oktWYFWT9nSePNnYZbB5c+jQAZKTYc0aqFnTYZtsjzuSUf1qMOZPt6tXjjZ7GxC7MZ6gPZZNAaoYfySefx6GDiX2812pj7nu588L3Z9jwZyXeGX1TMZ0ezpL7RJCCFdylI8OM9vixg3YupXpJ37n1G+rue3oTs6ZSvBDU/ubXE7rN5p+fdsao81z5hid54EDjd0Hq6XdFCXXMzt8tf05zVVuVni07Rj/0LQr5wNKMm3JZFZ8NZLzphI80/MF/qrRTDI7n3JXiTshUo3uUheTny+B8Ze4+/BWWh7/F5OfL2/0apjmPEs5pS/7PEVSimZ51weB9NuulgrwMxaIWDH5+XLj08+NKhcPPGBM4bh0yVhsWL++wzbZPoejURZ79asXbY3hvubBaco1WRazWJ83dvFOozTUsGFw/jxUr54uYLcF12d6yz70/2cFvQ5GsnTW0zy5YV6G7RJCCFdzlI+OMrvGmKUMHfExiWXLQZs2NPpoAi1unObPRnfwfI9RDjP7hW714dVX4ddfjbJzmzbBd9+l60A7a1OuZrYNex3jFXVaM7D/eJbVuZ1rfv58Pf91noucz+hOte22S3g3GYkWbmd5R79v3PjUY1N/mkRQUidW/3cbrx+E45cSUEDgtYsM2PEbPzW4i1ciLzAxKMbutqv2VkP3DAlm6YQvaNTvHsrGnuK5xyZzD+UJc9KmzK6otlcHNT4xmTV7z6SrLmLvvDd/3pXhiMjndz/EwLP/8v5Pk/FNTKDh6UMklq9Ag5eflQUqQgi3seSN9e6r/n7GGJy9Hf6K37jKq3Pf4ZRvADHvvU/LB3tSvHx5egA9zM/pqIJFRFQMU6L8iK3yAEG/X2Z0oRi7eefuzLbegCujkex6fbsyvlFzLpy+wPtrPuOZP76BN87DN9+k7owo8gdZWCg8ImJbNPW73UG187GsqtWC22J2U+GKUW72UpGi7CtbjeuFClPq+mUanTpIp0c/YX+5aplenGG93XaxG9coceMKsSXKY/LzzVQx/4w4WpCigMPh3TM8z5alBJNlAUvqHwR1Bm67zVitfscdsHYtLFtm1JQWXkkWFgpvZFsnGhyUiNOaaUsm023feu4fOJnTDUOynNnW8lJmm/x803WY7ea2dVu1hk8/heeegwoVjFJ4jz4KxYrl6OcR7iMLC0WeYD3y0Dx2LwtPH+HlziOYE3IPaE3li6e4LXo3t0XvpsaFGAISr6OB6S36sL+ccTkvM4szbMP+SpEArpgX51lGFHIayJmdj+foPFv2RkQMwTB9Opw6BU8+aXSk+/aFdeuMmtJCCJFLrDPbx7xZScVLZxkRuYDCyWk7u34pSZS7coFKl89S+9xxJt/5MFHB9VDZyGxreSWzfZXK1Eh2OkoZ2R0aCqNGwTPPGHO+n3gCnnoKKlXK3g8k8gTpRAu3sA3Jh7cs4VKRovzYsJ1xglJEB1YkOrAiPzZyHEiZWZyR0Zazrlglba8Oqr35eA7rpWalXY8+evPrpUuNed7duxvbzgan/8Mihf6FEDllm9mWHVqfX/cd/9u5ipPFSqO5Oa852ceHs0VLcbBMZX5qcBeft7wPyB+ZbTsCna22tWxpDH5ERsJ77xkbs7z3HgwcyOouA3j1kA+xF69LZnsZ6UQLt7AOyQqXz9Jt33pmN+/JtcKZX7Gc2QV1GYWaK1ZJZ3Y+nr3zrt5ISnfJMtPtqlzZ6EjfcYcRyq+/DoMHp26Xm5WteYUQwhF7HdsS16/Qc89fzG3SlXFdR2b4HPkls0d3qZs67zvHbWvVChYsgIMH4YMPSJr5Je1nzSK0cADfNbuHyXcNlsz2ItKJFm5hHZIPRS3DR2u+btYjzTn25tel7k6YhXfnzqZQuLKyhe3ixsyeZ+/SZZba1bSpUef6mWdg6FCYMgXeegv+9z+Hi2dccTlUCFFw2OvY9t35O6akG3wXcg9QcDLbIke5batWLZg2jZ6B7Wm6eTXtD25iRORC/q7SmD9qNpfM9hLSiRY5Zq/2pu1CC0tIFkm8wQPbf2PVLS2IDqyIr1KkaJ16HmR+tbUjjqZQlArw4/WeDT0eTFlZVe5wakaLFrBhA/zyi7Gb14ABMHEi9WqHUa1QYcJ/m8aMFn34rpmxYEYK/QshLLKS2RaFkpN4eNsvbAuux54KNVM7yZD/Mxsyn9tZnU63N7Ewe5p2ZVGjDiyb9RTjl39Ml0c/ITYuV38c4SJSnUPkiLMFIRaWFcxLIw8yfM23DNu0mAH9J7D9lhCXrLp21C5vnxfsaMQ63WrwTrUJ+2+dUV/14MHUc08VK03b4V+S6OuXYVUT6zJVvuYFRFkZSRKZJ9U5hCdlJbOty7cN3rKEN36fwcYPZtP6mUG50q4Ck9lWP5v1lurNo3ezYM5LfN28BzP7PiuZnUfkenUOpVRX4EPAF5iptQ63uf99wLyCjACgvNY60HxfMrDTfN8xrXUvV7RJuJ69kMtoQQiAvnaV8l98ysa/F+F/7gzL6rbh2K0tmNi1Xq79smf2sl1e5mhqxpzIY6klmGLi4hkbsQv63Imau4od73xA+dPR7KhYm89+CueevetY0bSj00uOjhYQyXzq/Esyu2DIbmZbqk5M7NOYKcv3ce3EKZ7f8D2nW91J66cfzpW2FqjMXrwz9f5rCUmpX2+t3IBvmnVn0NZfqDXyMYffRzI778jxSLRSyhf4D+gERAObgQFa690Ozn8KCNFaP2q+fUVrnaWCiTKq4X6O3mE7Hc1IuM7A7csY9vdiyl2Lg44djYVwbdu6o8leL7M1psHYfvdGUkrq/4fSKayaOYLr/gHs//l3wppVdvhY65EQezJbm1tkjqdHoiWzC4bsZLa1NPWTn3rKqHW8Ywc0apQLrc0fcpLZFkE+iayaNZKAiuUgKsruYyWz3ctZZrti2+8WwAGt9SGtdQLwA9DbyfkDgO9d8H2FGzl6h+2rVLpzCyUn8ejmn/hz+hBeWfMVe8tV54lhH8DKldKBzoKsrPqOi09M8/+jlQ+zQnvRMPY/wq4fc/rYjOZLy3zqfEcyuwDISmbbExRogosX4aOP4LPPYNgw6UBnICeZbaFKlCBgyCPwzz8QZ39itGR23uGKTnQwcNzqdrT5WDpKqWpADWC11WF/pdQWpVSkUsrejszCTSKiYmgTvpoaY5bSJnw1EVExqfc5+qVM1tpYoW3W+ugOfp31FK+t/oJ95arRd+Akhj40kS7D+uZ6+/Ob0V3qYvLzTXMsc3/+DIsatudikaLwwQdOz0sNfq0plJxEu4ObqXYhNv39Ir+QzM4nXJHZ9jQ9f5TvNn9l1KF/5hljIfNbb7m07flRTjMbzP9vtWtDSgqULm1sqjV8OHz9NezfD+aF+FUvnOD5P7/lzZWfUfXCCeqcOZL6HJLZ7uOKOdH2XiOOrmj0BxZqra3fflXVWscqpWoCq5VSO7XWB20fqJQaCgwFqFq1ak7bLMysFydYShNB+nlblt2qbAWa/LiakETn/zbSf8dy2h/awtHAijz9wJv8XKU5QYEmJspCh2yxtxq8Xb1yaRb7gHGJ1t/PhwvX0taeji/sz4/N7+HBRYu4a8QsqFrN7qKT0V3qMmHuRr78dgw3ChWmaew+tlZuQL8Hwl1aXkrkGZLZXsxVmW3NR0FZP83tUWt5dOdv3Hp0F/j7wwMPGDvrhRaodbDZltPMBihp8qNtdBDV+73D3ef20/PqESr88IOxcy1A2bIsLVuRwL3/pj6m/44VJPoWotfD73OiYjXJbDdyxZzo1sAbWusu5ttjAbTWE+2cGwU8qbXe4OC5ZgO/aK0XOvueMr/ONV6J2JlmwYM9juZtwc0gKHX8MKtnDgdgatuBTG95H2XLlpQ5WbnE3mIhSF/D1M9HUenSaVZ/+hizQnsxvv0QTH6+6SuiXL/O2bbtKLs1Ms33eWzEJ/Qc0lveALlYHpgTLZntpVyV2dadN5+UZMJ2r+XFdXOoePG0MQr6xBMwaJAxEipyLCuZbVt32+Tny8SwhoT5XzLKmm7cCPv3s7tJG94wNWLa588Q7+dP8etXuFCyDHsWraDn7be4/WfMz3K7OsdmoLZSqgYQgzFy8YCdRtQFSgEbrY6VAq5prW8opcoCbYDJLmhTgZPV8kARUTEZhjGQZmc9/8TrhP82jU1VGjG/aVcm9mnMc/O28/a6OVwvVJgOQz4npmR5QOZk5SZnq9itXwPXEpI4llKOv2qE8PjmCHrs+YvDpYM5tbIa9GvHxkJleP+oYvAv07lnXyQbuz9A66VzWVa3DXce2c6bB5ZTOWSEm3864QaS2XmAOzLbmq9SqZltceehrYxdO4v6Z46wo2JtKs77Bjp1Ah9XzPQUFlnJ7HRXFBOTmbJyP2Fj2hOREMiUc7WILX9zlHtI4ekcjFd0vbCf974exy1fhcPtM93xYwlc0InWWicppUYCyzHKJX2ltd6llHoL2KK1XmI+dQDwg0479F0fmK6USsGYnx3uaIW4cCw7Wz1PWb4v06uILZ6IXETY7j8I2/0H9+38neYPLiDiWgw99v7FR637pXagQeZkeYJtUNcYsxSA1zsOZ/Pev6h1Ppoa52Not2MN/P0zrYHW5nPfbj+Erxr24tFrJpY0uJvHNv/I4ysjWLE0ks7dW7n/hxG5RjLb89yV2dZStCYsJDh1Kkj7A5v4atFbHAmsxJO9XmJHy46s69IxB99BZJWjzLYVGxdv9zXzXeQxwASFYXGFxtS5vR/Dv/wS7rwTHs6dUoQiLdlsJR9wVO7GWZmbzJTisb70VyXuJKtmPsGK2q1YUyuU19Z8SWDCNa5WDCbx7DnuHDCyqVgAACAASURBVDaTS/7FUh+XW5uoiMxz+Loo6U/Ja5cwHT5AzfMxXPQvxoo6rdOcU/HSWf6a/hg/turN/9YvcleTCwRPT+fwBMnstNyR2Y6eOyIqhtfmb2Xel89QLCGe9o9/TiF/f8nsPMDZ6wJwWtYOwDclmQWLXqPZyf2waRM0bJgr7SxocrvEnfAwR1MnnE2pyGikODjQxMQ+jXm9Z0NMhXx4dfVMknx8Gd/uMX4N6cyGn9dxtFsfikYf5dNW93PVVDzN4ySMPc/eSnGTny+ju9ZjT2JhtlZuwIJbO6XrQAOcLFGWn+vfSffNy+DCBXc1WYgCIdcz297vfZe6qVNInljzLfXPHOG1TsMpX6aEZHYe4TCzu9TN1BTJZB9fhncbBcWLw/33w5UrudVUYSad6HzAUbg6C11HpXgebFWVI+HdWT+mvXGpqZqJVZs+pfP+SKbd3h/fqlWY2KcxCaVK07XJI7QcMZsZLfqQrHXqL7uEcd4QFhLMfc2DU+vC+irFfc2Ny4eZmW4z87Z7KZp4HWbMyO2mClGg5GpmhwQzsU9jggNNKOAWE3zSyJeiRw8x7atV1Nq2jmF/L2Juky5E1mslmZ2H5DSzAfwqB8PcubB3L4wYAV4428CbyHSOfMDRzlQZjS5kuLBlzRp46CE4dcqoEfrii+BrhHh2LkcK93L2uoD0K8Ntmfx8+WPlBMofPwiHD0Phwrne5oJApnOIXMtsW1evGnWGDx1Kc/hYyQp0e2QaV4sESGbnIa7I7NTX0FtvGTsEz5gBjz+e623Pz3K7OofwMHu1KTMzuuBwxXBCArzyCrz7rlHuaOPGdHVCs3M5UriXox3Lpizfl/pH07ae6Zq9Z9K8hsq3GQfdusEPPzhcqJLlP+xCFHAuz2xHJk40OtDTpvHMiqMUTkqgcHISa2qFcrVIACCZnZe4IrNTXx8vvwzr1xuj0TVrQocOqc8pme060onOJ7IcrtYiImDWLPjpJzh40JhLFRUFQ4fC1KlQtGi6hwQFmuyOREtVjrwjozc6mXrN6CBjq9933zWuSthsGZydKgNCiBxmdmbs3w9TpsCDD8LIkWy5Yv/qoWR23uGSzLbw9YV586BtW+jTx6gx3bChZLaLyZxoAQsWwJIlxqW/554zRi4iIowdkux0oMH5AgiRN2Rn3mU6SsGoUbBzJ6xcme5uZyMnQggPOXsWhgyBIkVgslHGWzI773NJZlsLDIRlyyA5GaZNAySzXU060QL+NW8fGhtrzIMeMAB693b6ENvFK1KVI+9x2R/NAQOgYkV47710d8m0HiHymIULjdJmGzYYHadKlQDJbG+QK290qlY1dp68cQOQzHY1mc5R0CUmwp49xtdLlxolcazmTjmT65cjRY5kd95lOkWKwNNPw7hxrJnzK68cL5L6fIEBfnZr0solYiHc7NQpePJJWLQImjUzrhzdemuaUySz8zaXZbaViKgYWl5OYP2W47wfvtrI7KsJVLp8lktFiqbOjZfMzh7pRBd0//1ndKTBWDwGcPfdHmuOcC2X/dEcPpyEiZNoPTiMh5r14N07HyYmLh4/H4WfryIx+WaVH7lELIQbaW2UNHv6aWMQZMIEGD0aCsmfd2/kyjc6lvnPyzVUuHyODr/Pp0XMHppH76bS5bMsr92KYX1ekczOAfktK+gsUzkA/v4bmjaFsmU91x6RN5UqxQPDP+GhX75g+KbF7KxYm6X17yAxRVO6iA8Ddq/kp4qN0dWqy0pvIdzp66/hkUegVSv46iuoX9/TLRJ5hGX+c6KvH3cc3c4dR7cTW7ws26s14tL5aBqeOkSwVOfIEelEF3Q7d4KPD6SkGLczOZVDFDxbfQI5cdcgeu/5g4BEY/6cKeE6kxdNpuOBTYx++WUY86iHWylEAaI1fPihMW1j3brUOv5CwM15zmO6jqTS5bNsqdyA2BLlUcBh0xZ4803WP3s7+Pt7tqFeTBYWFmRaG/PmGja8WYVDOtHCgaBAEynmEndKQ7krF/jh+7G0O7jF2Ijl5EkPt1CIAuLwYbjjDvjlF9i+HYYNkw60SMcyz3lzlUYsaXA3sSXK3zxep47RBzh40JNN9HrSiS7IVq+GTZuMYuzlyxtz6O64w9OtEnnU6C51KWJeOV773DEWf/cCtc8dY/PUL41a0idOeLiFQhQQH35ojDz36gUmEwwc6OkWiTzIabWPOnWMA//954GW5R/SiS7I3nkHgoJg8GCoXt0oyl6smKdbJfKosJBgxnRvAMDjmyMISE5g06zFtHp2sFECT0aihXCPixdvft2/P5Qs6bm2iDzLaVnD2rWNk6QTnSMumROtlOoKfAj4AjO11uE29w8GpgAx5kMfa61nmu8bBLxiPv6O1vprV7RJZGDdOli7Ft5/35gPNWeOMTdaCCe6Nq9ufFG/PmWWLePu6ubbFSsal5WF15Dc9mJRUTe/HjbMc+0QeZ7Dah8lSkCFCsbOliLbctyJVkr5Ap8AnYBoYLNSaonWerfNqfO01iNtHlsaeB0IBTSw1fzYCzltV4E0YQL8/DMsX278gjgzfjyUKwePP27cNhfkF8KpMmVgxQq47TZjNyyLihWNOrUpKfJmzAtIbnux69dh1y64/Xbjo0ULT7dIeKs6dWQkOodc8deuBXBAa31Ia50A/AA43+7upi7ASq31eXMArwS6uqBNBY9lW8/ISBg06Ga1DXu2bIHffoPnn3e4rbcQDnXqlLYDDUYnOjnZ2G5YeAPJbW+1cyckJRn5PWUKmBf7CpFldeoYZW6vXvV0S7yWKzrRwcBxq9vR5mO27lNK/aOUWqiUqpLFx6KUGqqU2qKU2nLmzBkXNNs7RUTF0CZ8NTXGLKVN+GoiosxXWjdsMOakduwIEREwaZLjJxk/3ugEjRjhnkaLfMPh689yJUPmRXuLXM9tyWyDw9+Z7PrlF+Nzs2Y5b5zI95y+/gYPhgsX4LXXPNY+b+eKTrS9t8Ha5vbPQHWt9a3AKsAyfy4zjzUOaj1Dax2qtQ4tV65cthvrzSy7D8XExaOBmLh4xi7eafxSLFxozG1evBgGDICXXzYuu9vaudPoZD/zTMZTPoSw4vT1V7GicZJ0or1Frue2ZHYGvzPZ8eefxiDI/fdDjRoubavIfzJ8/bVtC8OHwwcfwObNHm2rt3JFJzoaqGJ1uzIQa32C1vqc1vqG+eYXQPPMPlbcZNl9yFp8YjLv/roHFi2Crl2heHH44guj5NiAAUY9UWsTJhgVOJ5+OvWQy0dKRL4TERXDqPk77L7+pizfJ51o7yO57QaOMnvK8n1Zf7LTp41KHDVr8stTb0lmC6cyzGyL8HAjv4cMgcREN7fS+7miE70ZqK2UqqGUKgz0B5ZYn6CUsl611gvYY/56OdBZKVVKKVUK6Gw+Juyw7D5kq/ye7RATA337GgeKFoUffzTmRffpA9euGcf37YN58+DJJ6F0aSAXRkpEvmN5jSRruxeJjNeldKK9jeS2GzjKbEfHHUpOhgcegAsXWP32J4xecUQyWziUqcy2KFkSPvsM/vkHJk92Uwvzjxx3orXWScBIjBDdA8zXWu9SSr2llOplPu1ppdQupdQO4GlgsPmx54G3MQJ9M/CW+Ziww7L7kK37j/xt7BjXo8fNg7VqGWXrduzgaP9HaBO+mgV9R3K9UGGWdRqQeppLR0pEvmTvNWItKNBkXN0oWlQ2XPESktvu4SizHR23sL06uPeJF+D33+Hjj3n1sK9ktnAqU5ltrVcv+N//4K23YO/eXG5d/uKSWlRa62Va6zpa61pa6/HmY69prZeYvx6rtW6otW6itW6ntd5r9divtNa3mD9muaI9+ZXd3YcK+dD7YCR07py+4P4993D4vgep9vN8fA8f4t5dq5l7axdGrT2ROmrhspESkW85ey2k7n4FxuLCWLmq7y0kt3Of0x3jHLC9Olgzaj11vviQYz3vh0cflcwWGcp0Zlv76CNjIOTxx51X9xJpSEFXL2Jv96HP6msCTkTfnMphY8UlPwCeXT+XZB8fZrTok2bUIrsjJaLgcPRa8FXq5u5XAKGh8OuvUuZOCDOnO8Y5YD2KWPHSWT74+V3+K1uVwaGDQSnJbJGhTGe2tQoVYOpUYyO2r2XvpMySTrSXCQsJZv2Y9hwO7876Me25e+efUKiQcTnGjgsJxjvKPrvWMLdpN06WKAvcfKeanZESUbA4eo28978macP41VeNeqMTJ7q5hULkXbaZneZ3JiXFqNNrWbfCzWwulJzEx0smUSQ5kRFhYzl8zZjfKpktMpLpzLY1aJAxGDJ+vFGLXGTIJdt+Cw/R2iht16EDlCqV7u6IqBi2VGnEumpNmH9rJ36pd0fqfZZ3qpZfqCnL9xEbF09QoInRXeo6/0UTBUqmXyMNGsDDD8Mnn8Czz0KVKnaeTQiR6sMPjU1TfHygbl1o2pRnzgbwb/Egeuz9k9CYPYzs9SKHylQmWDJbZFK2XyNKwdixcN99Rt+if383tNa7Ke1g9WZeFhoaqrds2eLpZnje9u0QEmKUtBsyJM1dEVExjF6wg8SU9P+/Jj/fDC8pCpEtR48au2A99BDMnOnp1uRJSqmtWutQT7fDnSSz7bh+3aj1XK2aUZ50+3biNmwi8MzNxblT2w7kozYD8PNVTOmbwSiiEK6QkgINGxrFCrZvlx0xcZ7ZMhLtzRYuBF9fCAtLd9cbS3bZ7UArhXSgRe6pVg2eeMLYgv6FF6BePU+3SIi86euvjZKQc+ZA+/ZERMXw3LztlLt8jmfXz2X+rZ3ZHmRM0ShauJBktnAPHx9jNHrQIFi2DLp393SL8jSZE+2ttIYFC+Duu6Fs2XR3x8XbL5quNRLGIneNGwcmkzFHWgiRXlISTJoELVtCu3aAceldA6eLl2Fc16dSO9AAFx3kuRC5YsAAY0Bk/Hij0yAckk60t0lIgN69jbl0//3nsCqHEB5TvjyMGmVcKdm61dOtESLvmTfP2E123LjUy+XOypJJ5Q3hVn5+8OKLsHGjsdW8cEg60d7mt99gyRJ47jkjfO+91+5ppQL8snRcCJcaNQrKlDE6CUKIm1JSjAo2jRql2SDLUUdZgVTeEO73yCPGgMiECZ5uSZ4mneg8znbnqpiPvyDJ3wjbyMoNaTNrl93tXl/v2RA/37QLAvx8Fa/3bOiWdosCrkQJowO9YoUxmiFEAWGb2RFRMWmOvTR4POzaBWPGGPNPzeyVJVPAwFZVZQqecD+TyRisW7FCdjF0QhYW5mGWnasshfcvnjpHmTUr+L5JZ/aVqcru8jWIiYtn7OKdQNq5zlIGSXjc0KHwxhvw6afQurWnWyNErrPN7Ji4eEYv2AEKEpM1aM2AVd9xPLAi2+q0pbfVYyWzRZ4zcKCxyDAiwnjTJ9KRTnQeZr1zFUDX/zbgn5TAj/XvYltw/dTjlh0IbcM2LCRYAli4RURUjP0//pZSd++/b3cBrBD5iW1mA2mqJN1+dAdNT/zHuC5P8sfvB+l9W7U050pmC3dxmNnWqlQxNl+RTrRDMp0jD7NdaNJ711qOBlZkW1D6smHOFqUIkZsso28xcfFoSL06EhEVY5S7S0iA2bM93Uwhcl1GOfxk5HxOFSvNokYdJLOFxzjNbFthYfD333DsmNvb6Q2kE52HWS80KXflPLcf+4eIBnfbLX4uq7eFp9gbfbNcHaFRI2jbFj7/3FhQJUQ+5iyHm8buo83Rf/jitjBuFCosmS08xmlm2xo40OhzfPWVm1rnXVzSiVZKdVVK7VNKHVBKpRvzV0o9r5TarZT6Ryn1u1KqmtV9yUqp7eaPJa5ojzeytxjFeqFJrz1/4qtTWNqoXboFgyY/X1m9LTzG0Yha6vEnnoCDB2HVKje2SjgjmZ1zGWW2hZ+PIjApnjdWfU6cfzG+b9JVMlt4VIaZba16dejcGb780qhvLtLIcSdaKeULfAJ0AxoAA5RSDWxOiwJCtda3AguByVb3xWutm5o/euW0Pd7I0aUVMHYXDA400Xv3WvYG12HE8O5M6duE4EATCggONMkOhMKjHI2o+ShFjTFLuftQaW6UKgOffebmlgl7JLNzLjOZbcnnKfc34detM2l06iAv3vMsgRXKSGYLj3KU2SVNfuneGALGIvHoaKPErkjDFQsLWwAHtNaHAJRSPwC9gd2WE7TWa6zOjwQedMH3zTecXVpZP6Y9YabLMPYATJ1KPXPwSgCLvGJ0l7ppKhJYJJt3ujpyJZlv6rdnyJJFqOhoqFzZE80UN0lm51CGmW2dz1u3wtrlMH48M6RuusgD7GW2n4/iakJS6m7HaSp/9ewJFSrAjBlpapsL10znCAaOW92ONh9z5DHgV6vb/kqpLUqpSKVUmKMHKaWGms/bcubMmZy1OA+wvhQYk9GllblzjXqi/fu7sYVCZE5YSHCa0TdfO3P2v27cBZ2i+WrQuPSjHMLdJLOzIUuZbW3SJChZEkaOzOUWCpE5tpkdHGiimH8howyjlfjEZEbN30GNV1fwTf326KVL4cgRj7Q5r3LFSHT6v5hgd7N1pdSDQChwl9XhqlrrWKVUTWC1Umqn1vpguifUegYwAyA0NNSrN3O3rSXqiOVy+OpZ31CsWSvKVarkphYKkTXWpblqjFma7v7owIqsrdmc/n/NZ3vRCixpcLfd+ubCLSSzsyirmZ1aMqx4PCxaZGyhXKKEm1orRMZsyynay224eUXxs/qd6bt+MZcHPkKFdavtFjgoiFwxEh0NVLG6XRmItT1JKdUReBnopbW+YTmutY41fz4ErAVCXNCmPM3epUB7krWm6oVYapw+yhelG8vInfAKjubbje06kj3lahD+2zSK3rjmeDW4yG2S2VmUlcy2niN9eMyb4OcHzzyT+40UIgcyqhZzokQ5Jt85iAob1sK337qlTd7AFZ3ozUBtpVQNpVRhoD+QZsW2UioEmI4RxqetjpdSShUxf10WaIPVvLz8yll9UNvL4Z32RwKwrGYL6XAIr2CvQgHAqeJleaf9EAISb9B97zpA6pt7iGR2FmUlsy2KXThL0E/zYNAgqFgxF1snRM45ym1rXzfvwebgBvDss3DypJtalrfluBOttU4CRgLLgT3AfK31LqXUW0opy8rtKUAxYIFNWaT6wBal1A5gDRCutc73gezoHV9woInD4d1J0TevfHba/ze7y9cgumQF6XAIr2Bvvl2pAD8AooLqcqB0Ze7faZS7k1q57ieZnXVZyWyLwVuX4JeUBC+8kNvNEyLHMrO2RSsfpvZ7Ea5dgxEjwM7rvqBxybbfWutlwDKbY69Zfd3RweM2AI1d0QZvYm9lrHXd0KBAEzFx8ZS6dpHQmD183Lpf6nEhvIHtfDvrOaULbu3I2LWzqX/pBMP6dcvS82Zqq1qRIcnsrMlsZlsUu3GNh6KWsbbRHbSvXdvt7RUiO6xz2946AJOfL/36dYTybxrbgC9cCPff7/Q583tmy46FHmBvpM66bqjlskrHA5vw1SmsqN1SivMLr2b9mv+xYXuSfHz48Pr2LIVplraqFcKFMpvZFn13rqLEjav4vPSih1osRM44fc2PGgXNm8Pw4fD11w53oy0Ima20Fw7Hh4aG6i1btni6GbnC8q7tRkwsv8x+liuFTQx6YTaju9bLV+/eRAHXowds22bsZGjK3BWWNuGr7ZYWCw40sX5Me1e3MNcopbZqrUM93Q53KgiZHRMXj69SJKeksPKbZyhbujildm33dPOEyB3798ODD8KmTRASAu+9B+3apTmlIGS2jETnIZZ3bSfPX2HaksmUvH6FUfeNkw608Hq2WyT/FTYYTpyAiRMz/RxZ2qpWCDewHmkDozpH83NHqH3yEKWeHOrh1gmRffa2tU+jdm3YuNHYx+LcOWjfHnr1gr17U08pCJntkjnRIuts5wm92rQ4tz3Qm1VJSST7+FD14ime7/4cO0pXZcryfdKJFl7Ldm5dTFw8Q68WY0W3e6kyaRI89JARyBmwnXdqfVyI3GZvbqe90nc9o5Zzo5AfRQYM8FBLhcgZe5ltt66/jw8MGAD33gsffggTJkDjxvDNNzBgQGpmF79xlat+/qT4GFOe8lNmy0i0B9ibJ7Rr8mcEXzjB31Ubsad8Dd5uP4TFjToA+etdmyh4HG2R3KdmH64oP049/HimVnnbK8EkawWEOzia22n7pq5IUgJhu9eyvHZrKFXKM40VIoccZfaz87bbH5X294eXXoIDB6BtWxg4EGbOZFKps8z8cTzbPxzAY5t/AvJfZstItAfYe4He8+8atgXV4/keo9Kdn5/etYmCx9GbwDPFSjHljgd5c9V0Nr07gxajhzl9nrCQYIof2Eul559lePfRJFevke9Weou8yVGnwlep1B3dAPrvWE7g9SusadWdXrZPIoSXcDZw53BUGqBcOVi6FO67Dx5/nLbAjcDSXCtiolnsXoKlOodwBcsL9Jazx1A6hVvOHqP+mSMsqX+njLSJfMfZm8DvQu5hV/maVH/nFbh82fkTXb9Oh7efo0H0Pv68K4D1Y9rnqzAWeZejTkWy1qmZXevcccasnc2ftUK5a+QD7myeEC6V0cCd091mAwIgIgJefRVmzaLIiRiKd+1EN98L+TKzpRPtAUGBJqqfj2HVlyO469A2eu3+g2Tlw9aWnZyWURLCGznbCSvZx5dXOz9B+UtnjZ3dliyBU6fsL2oZNw52GiMgxOSfEkki73O22crEPo2pWtyPqb9MJcGvMFc/nU5Ys8pubqEQrpOZ3Qtt31imyez31xNx7zAYPNiY6lG/vlHNIyEhF1vtGTKdw80iomK4eiOJkAuxAJS5dpFee/7k7+pNeKxv63SbVAjh7SyvZ0sZMFvbguvzzd0DeHjJfPjxRwBCS1ZgTKU6/F21Mb/Ua8ui8A2EzX+fRa1603vHSgrFxrr1ZxAFW7t65ZgTeQzrmfuWq4Rht5QgbPsXcHI/LFxIt87NPNZOIVwho8yGtG8s7S1EfG7edp6dt53gQBMfBlQkNCnJmDPdoEHu/wBuJCPRbmR5ocXFJxJ86QwAodG7qB53Av+HBkrnWeRbYSHBrB/Tng/6NbU7ZanE1Clw6RL89RfT7hnG9oq1aRazl3dWfMqmjx/mk4iJ7C9ThXG3P0y0qRTRuw566CcRBU1EVAyLtsak6UAr4L7mwYSZLkPLlrB4Mbz7rjEXVIh8IKPMtp5mam/NgOX3JSYunolHzF3NPXtys8keISPRbhIRFcOo+TtSF6EEXToNwP07V5Hg60ez5x7zZPOEcAvrEQ6728C2bcvUxhfRjXsCUP/0Ie7b+Tutj+1k9D3PcsOvCCeLlsL/v8PIBXORm6w3UbGlgcgdx2DoI0ZlmZUrjTq5QuQzGWY2GVcQ212ikvmL3fnujaZ0ot3AMgJtvYo7yDwSXUinsKJmCzoHBnqqeUK4le2UJctcOktABwb4ceFaIgB7ytfknQ410zz+VLEyNDnxn1vbLAoW28vT9rRfuxjOnDE2nGjVyo2tE8K9spLZ9sQX9ud4yQpU2b3bHc11K5nO4Qb2LnVYOtEA60I7urtJQuQJ9urvXrmehJ+vcviYU8VKU+HKeWq89Iv9mqVC5JC9zLZWJPEGw7b8CB06SAdaFCjZyWyAA2Wr8N/KDfkus13SiVZKdVVK7VNKHVBKjbFzfxGl1Dzz/X8rpapb3TfWfHyfUqqLK9qT19i71GHdib5t5CB3NkeIPMNeZyUxRVO0cCGCzQtXbKP5VLHSmJJuUPzG1dSapfkplN1FctuxjC5PP7j7d8pcuQAvv+ymFgmRN2QnswHWVW1CnXPHqLVtXb7K7Bx3opVSvsAnQDegATBAKWW7/PIx4ILW+hbgfWCS+bENgP5AQ6Ar8Kn5+fIV2/JIvinJVLx8LvV2z9tvcXeThMgTHHVWLsYnsn5Me46Ed+f9fk1Tyz76KsXpYqUBqGD+HXJas1TYJbntnLM6uVWL+zFq+0/QujXcfbf7GiVEHpCdzAb4pnkPThQrw4jIBfkqs10xEt0COKC1PqS1TgB+AHrbnNMb+Nr89UKgg1JKmY//oLW+obU+DBwwP1++YltzsfyV8xTSKR5skRB5g6POivVxyyrxw+HdSdGak8XLAFDhyvnUczIaORTpSG474WiL+Q/6NeXPoBgCTsbAK6+Acn4JW4j8JjuZDZDo68fMFvfS6vi/NIvZk28y2xWd6GDguNXtaPMxu+dorZOAi0CZTD4WAKXUUKXUFqXUljNnztg7Jc8KCwlOs4lKk5RLnm6SEHmCo86Ko106gwJNnCpmdKKnLp1qVEYg4x22RDq5ntv5KbNTN74qD7z2GoSEQLdunm6mEG6Xncy2+L5JFy74F+eJyIX5JrNdUZ3D3ltxnclzMvNY46DWM4AZAKGhoXbPycvSrG6dPRume7Q5QuQJmSmfZG10l7q8Oc94E1r+6gUanD7M4cq1HQa4cCjXcztfZTYYb9h8zONOU6fKKLQokLKT2ZZKN9cKm/i6eQ+eXf89PvlkFqsrOtHRQBWr25UB2+3ELOdEK6UKASWB85l8bP6zdClUqgQnTni6JUJ4XFZ26Uw9b5Lx6X9HNxH4dB/ZqCjrJLez6vzN6UO0aeO5dgjhYdnJbEune3m7+xm5NYIOS2ZDX++vre6K6RybgdpKqRpKqcIYC06W2JyzBLCUoOgLrNZaa/Px/uZV4DWA2sAmF7Qp74qPh19/hd620w+FEJlhHd6DT26VDnT2SG5n1YEDN78uXNhz7RDCy1jPkf717XspNHwYzJ0LR496umk5luNOtHmu3EhgObAHmK+13qWUeksp1ct82pdAGaXUAeB5YIz5sbuA+cBu4DfgSa214+Kc+cGqVXD1Ktx7r6dbIoT3unABwsONbWTz4VayuU1yOxsOWm01L51oIbJv1ChjatS773q6JTmmtPa6qWqEhobqLVu2eLoZ2fPoo7B4MZw+DSNHwrVr8N13nm6VMGyuXwAAIABJREFUEN4nJgYqV4Z33vGqer1Kqa1a61BPt8OdvDqzLd5+21hUCEZum/LHwighPOLRR+H7743R6PLlPd0ap5xltuxY6E5JSbBkCfToYYxkzJghHWghsis42KjVu2iRp1siCgLrkWg/P8+1Q4j8YMwYSEiA997zdEtyRDrR7rRuHZw7J1M5hHCVPn0gKgoOH/Z0S0R+d/Ag1K8Py5ZBIVesyReiAKtTB/r1g08+gbNnPd2abJNOtDvNnWtcAuza1dMtESJ/uO8+4/PixZ5th8j/DhyAVq2kPrQQrvLyy8bUqPff93RLsk060e5y4QLMmQMPPABFi3q6NULkDzVqGBtfyJQOkZuuXoWTJ+GWfFLcVoi8oGFD42riZ58Z0129kHSi3WXWLOMd18iRnm6JEPlLnz6wcaOx0FCI3LBhg/G5cWPPtkOI/Ob++41BRi9deCyd6FwSERVDm/DV1BizlDsmrOTq+x9B27bQtKmnmyZE/mKZ0hER4dl2CK9mndltwlcTEWX1pmzRIuMKYseOnmugEPlRhw7G7p8rVni6JdkinehcEBEVw9jFO4mJi0cDt0Stp2j0UTb1GOjppgmR/9Svb3z88IOxNbMQWWSb2TFx8YxdvNPoSCcnG3Pue/SQsnZCuFrZstC8uXSixU1Tlu8jPvHm3gODt/7MqWKleSGppgdbJUQ+NmyYUf3myy893RLhhWwzGyA+MZkpy/fBX3/BmTPQt6+HWidEPte5M0RGwsWLnm5JlkmdnlwQGxef+nWN8zHcdXgbU9sO5PjlRA+2Soj8ISIqhinL9xEbF09QoInRXeoS9tRTRg32Z54xpk3Vq5e1x8vW4QWadWanO77wV2MEWqpyCJEtGWZu584wYQKsXQu9e2f98R4knegcsvefGxRoIsYcyg9tW0qCTyG+b9KVoEC5FChETlguu1tGDS2X3QH8X5rM7WHtWBc2hPGD37IbtM4en1dCWeSujDLbWnCJIvD1IrjnHqmqJEQ2OMtcMK4CnTl3ie2F/Tk550dq2nSi83pmy3SOHHA0j65dvXKY/HwJSIin785VLKvXhiulyjK6S900j3W4iEUIYZejy+5vLNnFc3+dYXWNZoTE7E07pzUTj5+yfF+ut114XkaZbc3k58ukUmeM0nb33SeZLUQ2OMtsy+9igq8fG6o0xnfVSq/LbBmJzgFH/7lr9p5hYp/G7H9zCiUSrrHsrr5M7NM49V1TXn9nJURe5eiye1y8MVXqn4p1CNv9B+WunOdMsdJMWb4vze+U08v2It/LKLPTjFB3vIU2Q8OgalWWVL9NMluIbMgosy3WVQ+h48HNLHhjMnStB5cvw6VLDPntH4rdiKdowjXiTMV5ucuTaOWTZzJbOtE54OwPcljTINi/Epo3Z8YnTxolXMycvbOSQBbCMUeX3S3+qWRshnHryf38fkvLdL+jjh4vU60KBqeZHRKcNn9nzTK2lP/+eyb9cUwyW4hsyCizLdbUCuXV1TN5YclHsMR8UCn6Fgngsp8Jv5Qkyl2N4/22AzlTrHSeyewcTedQSpVWSq1USu03fy5l55ymSqmNSqldSql/lFL9rO6brZQ6rJTabv7Is0WU7V3Kc/SfGBRogilTYPdueOqpNB1okNEwIbJrdJe6di+7lwrwA2BX+VokKx9uPXEASN85dvR466lW+V1Bye0sZ7a1K1dg3Dho3Rr69ZPMFiKbMspsi6Olgrhr6Az6Pz8bYmON38HkZH7fsI8Oz3zLpLsGA1AkOTFPZXZO50SPAX7XWtcGfjfftnUNeFhr3RDoCnyglAq0un+01rqp+WN7DtuTK7I6j27GiVXw0kvwv//BwPS1oTMd5EKINMJCgpnYpzHBgSYUEBxoYmKfxrzesyEmP1/iC/uzv0wVbj35n92gdfT4AjaamO9zO6uZne4P8qRJxlzo998HpSSzhcimjDLb2rlywfR/sCNUqmQs5FUq9fFFixsLe6sE+OapzM7pdI7ewN3mr78G1gIvWZ+gtf7P6utYpdRpoBwQl8Pv7TZZmUc3I2YlDT8Oh/794dtvoVD6f+LRXeqmmV8HBW80TIjsSnfZ3cqU5fv4p1JtOh3azMR7G9k9LywkmLDEGEg2/Z+9Ow+PsrrfP/7+EIIGUQKKKGF1Q0FUNKIWN3BBqyJ112rVn4haW5daFNzXSkVrXb6tW9XWXasiiooLWK2KAiKrYhVlCS7IJkuEEM7vjzOByWRmMssza+7Xdc2V5JlnZg4w3DlznnM+x480Nj1Fn9tJzX2OrOIybx7cfjucdhrssw+gzBZJR2OZ3VjpukG9Kxh00p7wDDx1Zm/YLT860JB+J7q9c+5bAOfct2a2dbyTzawP0AL4KuzwLWZ2LaEREefcmhiPHQIMAejcuXOazU5OwvPobrkF7h3hw/ef/4zagYaNC1Hyte6hSCGJLFnW/ah+tBnxFoO2rG148qpV/jL93Xf7XQ5nzcp+g3MvK7mddmY//zy8/DI8+mjSD01q7nOk4cP911tv3XBImS0SnMjMvvPkPRr/v7TJJv7rmqhdxJxptBNtZm8B20S566pkXsjMtgUeA850zq0PHR4OfIcP6AfwoyE3Rnu8c+6B0DlUVlZmdW/fhBYj3XgjXHcdnH66D/2Skgbnh2s0yEWkUdEq3dy8aDOeA5g4Ebp02Xjyf/8LZ58NX37pt5pdvTonbc6GfMjttDN77lw/GHHTTdCpU1IPTXkB6UcfwZNPwlVXQUTHX5ktkr6Uq5PlaSe60TnRzrlDnXO7Rrm9BHwfCtm6sP0h2nOY2RbAGOBq59yEsOf+1nlrgEeAPkH8oYLW6GKkm27yHejf/CahDrSIBCPaZfupbbpQU9IcJk3yB6qr4bLL4MADobYWxo+HgQOhpnh3EC2K3D7kEP913LikH5rSAtJVq/xC8G22gWHRpomLSLpSrvtcqJ3oRowGzgx9fybwUuQJZtYCeBH4l3PuuYj76oLcgEHAjDTbkxFxFyPNn+870KeeCg8/rA60SBZFu2y/tnkpn7Xr6keiP/wQ9tgD/vIXuOACmDYNDj6Yr5evZeny1U1144zCyO1evfwVg7ffTvqhSS8gnT8fDjgAJk+Gu+6CVq3Sa7uIRJVqpZt3vlkOwNn3vZdXmZ3unOgRwLNmdg4wDzgRwMwqgfOdc4OBk4ADgS3N7KzQ484Kreh+wszaAQZ8CpyfZnsyJualvKeeAuf8aLQ60CJZFeuy/ZzOO7Pbe2Nh//2hY0d4660NI5ujplSxcu5yjq5dV69yAzSZjTMKI7ebNYN+/fxItHMNSoU2JuHpFx99BIMG+ZHoV16BI49MscEi0phUplqNmlLFo+/N42Bgq1XLGJ9HmZ3WSLRzbrFz7hDn3I6hr0tCxyeFghjn3OPOudKwckgbSiI55/o753qFLjOe7pxbmf4fKcsef9yv8N9++1y3RKTJiXXZvuJXv/TTNQYPhunTN04NwF9O/JlmNF+/8ZJiPm0jm2kFlduHHAJVVfDFF42fm4onn4SDDoKyMn/VQh1okYxKZarVyLGzmb9Ja1a0KOPW1+9hn3nT8yaz053O0bRNm+Z/QUepBS0imRfrsv3eV5wP334L998PW2xR7zELl1WzrqQ5pbXrGhyXPNO/v/+awpSOuNavh6uv9tm9zz7w8cfQs2ewryEiDaRSq3/hsmoWb1bOQec9RFXrrRnx+t1sWvNzXmS2tv1Ox+OP+zJ2J52U65aINFkxL9tvE604hb9sWNOseb2R6Lrjkmd22MFX5hg3Dn772+Ce98IL4b774Jxz4G9/gxYtgntuEYkr2Uo3dVNAlrRszbAjfs9TT1/FJf99ksd+dWEGW5kYjUSnYNSUKvb/01t8+/eH+e8OlYxasDbXTRKRBA0d0B1r3pwStx4LVW3Txhl5ysxP6Rg/3o8epyh8C/AzL37Qd6B//3t48EF1oEXyXPgUkA+77M5Tux3OuRNHcUvH3I9EqxOdpLoah52mT2TblYt5eqcDGf7C9LxZKSoi8Q3qXUH/Xn4U5Ia37qei9aZ5tY2sROjfH5YsgalTU3p45BbgZ4x5kOWbtmLM8eclvVhRRLIvcgrII7+6kLVbtePg24bD2twOYmo6RzI++4za3w7j5hUr2PX7r1jZooy3dujDz6EJ7volLFIYenTZEoDffDKG39w9HPR/N3/VLQp9+23o3Tvph4fXpd1zwWcc+tVEbjvwN7z04fccdZDmQYsUggZTQPYyX+9/xAi49tqctUsj0YmqqYETTuCoSa/TZ8EszMH/7XcSP5duCmhRkkhBCS9HeeeduWuHNK5DB9h555QXF27IZue4/N1/smizch7Za6AyW6SQHXOM35/j5pthRu62GNFIdKLuuw9mzeK6M27kmQ57Nrhbi5JECsjsUGmk8nJ48UX4+mvo1i23bZLY+vf3W4CvXZv0HOYO5WWs+G4RJ057k33nz+DaQ8+jusWmVCizRQrbXXfBm2/6BcIffJCTvTo0Eh0SvvCkwW44P/7oLxcceij7XXJ28tvJikh+2W47//WZZ/ymHvfem9v2SHyHHOI3Q/n443qH4+Y2wNdf89wbtzHp3tO5Zvw/+HTbHXl69yOU2SLFoF07uPtunwt33ZWTJphzLicvnI7Kyko3adKkwJ6vbuFJ+H7uZaUlGxcbXXihrzc7dSr07MmoKVWMHDubhcuq6VBextAB3TUfWiRPJPT/c906mDvXb5J02mkwZgwsWACbb57x9pnZZOdcZcZfKI+kndlLlkD79nDyyb60KAnk9qxZcNhhsGoVXx51IiPKe/N2q850aNNSmS2SZ1LuVzkHxx7rd6WdNs2XxQxYvMxuOp3oF17w85pPPrnBXX1HjIu6DWVFeRnvH9UO9tjDd6TvvjvVJotIFjTasYpm4kTo0wf++le4+OKMt1Gd6BRdey3cdBOMHg3HHBM/tw/dAo44AkpL4Y03oFev9F5bRDImpdwOV1UFPXrAnnv6mvIBV92Jl9lNZzrHfffBn/4U9a5YC0wWLl3tf6mWl8P112ewcSIShPBKDHUa3R52772hb19/ObC2NvZ5kltXXw277QZDhsB338XM7U7TPvZzqDffHN57Tx1okTyXUm6Hq6iA22+Hd96Bhx8OvoFxNJ1O9AEH+C26ly1rcFesRYGnLPzEF/m/6SZo2zbTLRSRNMX8QNxYJYZLL/WLC0ePzkCrJBAtWsC//gU//QRHHcX2UWK731cT+edz10HHjvDf/2bk0q6IBCvl3A43eDDssw/ccktWB0OaTid6//393JkPPmhwV/huOHVaWy1Xj/8H7LqrH/kQkbwX6wNxo9VzBg2Crl1V7i7f7b47PPccTJ3KE2/czg4rF/Hg8zdy+BcfMnDWf3jghZtZvePO8O67fnRKRPJeyrkdzgyGDvWDIU88EVDLGpdWJ9rM2prZm2b2v9DXNjHOqzWzT0O30WHHu5nZR6HHP2Nmmdt/dZ99/Py4995rcFfkbjgV5WU8ufJDNqua5+dJNlclQJFCEO0DcUKVGEpK4KKLfD5MnpzBFuZeQeV2NL/8Jfz977T/4B3GPng+h335MX8bdSt/feV2lvfemzYT3oOttspqk0QkdSnndqRBg2Dfff2Vxe+/D7CFsaU7Ej0MeNs5tyPwdujnaKqdc3uEbgPDjv8ZuDP0+KXAOWm2J7aWLWGvvTZ0oiNLIwG8P6w/dw3ama1++pGu9/+V//Tcn1Ftd85Yk0QkWNE+ECe8OOWcc/w82jvvbLx0WmErnNwOU+/fZPH2fHbBHylpvzXvPfgcH+20N692358Tj72WUV+tyEZzRCQgaeV2mFHTvuPX+57LmuU/8fZhJ2clt9OqzmFms4GDnXPfmtm2wDvOuQYfHcxspXOuVcQxAxYB2zjn1pnZfsD1zrkBjb1uyiu9hw6Fu+9m9H9mccWY/zVYCfrHTRZy9jWDWdB6a9qvXMyhg+/jx3YVKf1jikgBuvRS1t97L/1++whzyzYO0Ca1UrwRua7OkYvcTrc6R6zV+8fv2YHnP1mY+qp+ESkK4Rnx2w+f5fJ3/8VFJ1xN/yvPTzsLMlmdo71z7luA0NetY5y3qZlNMrMJZjYodGxLYJlzbl3o5wVAzD+pmQ0JPcekRYsWpdbaAw6AtWt5/ZGXo64EdS+/Qm2zZmyzYjH37XMi88u3SW6FqIgUtosuwtWu56SP6y8wLLIcyEpuB5LZIbFW7z/18YL0VvWLSFEIz4gH+hzHjPbbc81r93LfixMz+rqNdqLN7C0zmxHldmwSr9M51Is/DfirmW0PRCvkF3NY3Dn3gHOu0jlX2a5duyReOkzfvgB0+3xK1Lsr58/kkw47s8fFT3Hn/qdtOJ7UClERKVzduvHmjvtw2qevs2nNz34xckgh5UA+5HYgmR0S6+++NsaV1EL6txKR9IX/n19X0pzLj7yY8uoVnPviPRl93UY70c65Q51zu0a5vQR8H7ocSOjrDzGeY2Ho6xzgHaA38CNQbmZ1q/Y6AgvT/hPFs+WW0KMH+3/3WYO7Wq6tZtfvvmRip56sblFWr1h3UitERSSvNTbfefTBJ9Hm5xX8bdQIptx9GhXLfawVUg4UVW4T++++JMamCoX0byUijWsstyP/z89qvx1/3+cEjp8xDl57LWPtSnc6x2jgzND3ZwIvRZ5gZm3MbJPQ91sBfYFZzk/GHg+cEO/xgTvgACoXfs5m9ReCsufC2TR365nYsWe94ymtEBWRvFQ3b65qWTUOqFpWzfAXptcL5MPPO4EZ2+5I/zmTaPPzCi7575PFlgMFl9vRVu8bfiQ6shtdZP9WIk1eIrkdLSMeOug0ftpuJzj33Kh7hAQh3U70COAwM/sfcFjoZ8ys0sweCp2zCzDJzKbiw3eEc25W6L4rgD+Y2Zf4uXb/SLM9jTvgAEpXruDeXqVc8MXbjP7nJdwx5i8M+eh5aq0Zkyt22RDKqa4QFZH8lMjOWIP27MjPF/4egMVlW3DczHHcu3uLYsqBgsvt8NX74DvQdRM5HCizRYpYQrkdpcLHjSftxRbPPAHffedLmGZAWgWQnXOLgUOiHJ8EDA59/wEQdd/V0GXCPum0IWn77w9Av4Uz6DfhaX5ctZb2K5fQfuUSPu7Yg5WbtAT8P8D7w/pntWkiklmJ7oxVecUFsOlatjz6aOjTh0OeuAdOKI48KMjcxv+SHNS7gr4jxlEV8e/lUGaLFKtEc7suI+qrgKuvhhtu8HWkjzsu0LY1vV1EunSBTp1gxAhYtIgrjr+Wt3foQ9vVy/m5+cY9A7QwRaT4dCgva9ABqzteT/PmcNll/vuhQ+Gaa2DCBF/IX3IqkC2CRaRgJJzbsVx1FbzyCpx3nq/SluZC53BNZ9vvcAccAIsWQZcufLGnH5le0rK1X1AYooUpIsUnpZ2xLrkEtt4ahg2rV61DciOQLYJFpGCkvaNhaSn8619+QKRt20Db1nQ70QBDhnDZkT2C2W5SRPJeSjtjtWrlR6L/8x94442stVWiC2yLYBEpCIHsaNijB1x+OZSUNH5uEtLasTBX0t39ikWLYPhwuO02aNuWUVOqGDl2NguXVdOhvIyhA7prYYqIbLR2LXTvDuXlMHkyNEt9/CHXOxbmQtqZHUGZLSLZEi+zm2YnWkQkWY8/DmecAfffD0OGpPw06kSLiBSOTG77LSLSNJx2GvziF3Deeby61wB2v+SZqEX/RUQkfzS2UUs61IkWEUlEs2aM/stj3Nf3FA6f8hZPPTWcqqWrGxT9FxGR/JDIRi3paHol7kREUvTn8d9Qtf/pvLzDvrSt/gnMNhT915xcEZH8Em+jliAyW51oEZEE1dUinrnNDlGPi4hI/sh0XXlN5xARSZBqFIuIFI5MZ7Y60SIiCVKNYhGRwpHpzNZ0DhGRBNTVJq6uqaXEjFrnqFCNYhGRvJSNzFYnWkSkEXUrvOsWqNQ6t2E0Qx1oEZH8kq3MTqsTbWZtgWeArsA3wEnOuaUR5/QD7gw7tDNwinNulJk9ChwELA/dd5Zz7tN02iQikq7IHfFWr12X0RXe2aTcFpFik6vMTndO9DDgbefcjsDboZ/rcc6Nd87t4ZzbA+gPrAbeCDtlaN39CmIRybVodUWXrq6Jem6BVuVQbotI0chlZqc7neNY4ODQ9/8E3gGuiHP+CcBrzrnVab5uyiI/rehyrIiEi1ZXNJYCrcqh3BaRopHLzE53JLq9c+5bgNDXrRs5/xTgqYhjt5jZNDO708w2SbM9cWV65xoRKXyJjlQUcFUO5baIFI1cZnajnWgze8vMZkS5HZvMC5nZtkAvYGzY4eH4uXZ7A22JMxpiZkPMbJKZTVq0aFEyL71BvJ1rREQg9khFeVkpFeVlGFBRXsatx/XK29HQfMjtIDIblNsiEl8uM7vR6RzOuUNj3Wdm35vZts65b0Nh+0OcpzoJeNE5t2GiSt1oCLDGzB4B/hinHQ8ADwBUVla6xtodTaZ3rhGRwjd0QPd6q7rBj2BcP7Bn3naaI+VDbgeR2aDcFpH4cpnZ6U7nGA2cGfr+TOClOOeeSsQlwVCAY2YGDAJmpNmeuLTbmIg0ZlDvCm49rlfBjDqnQLktIkUjl5md7sLCEcCzZnYOMA84EcDMKoHznXODQz93BToB/4l4/BNm1g4w4FPg/DTbE1esTysFOq9RRDJkUO+KYuo0R1Jui0hRyVVmp9WJds4tBg6JcnwSMDjs52+ABn8651z/dF4/WXV/wVrlLSJNlXJbRCQYTW7HwiIfYRIRKTrKbRHJR02iE60aoyIihUOZLSKFoOg70ZH7p9fVGAUUyiIieUaZLSKFIt3qHHlPNUZFRAqHMltECkXRd6JVY1REpHAos0WkUBR9J1o1RkVECocyW0QKRdF3oocO6E5ZaUm9Y6oxKiKSn5TZIlIoin5hoWqMiogUDmW2iBSKou9Eg2qMiogUEmW2iBSCop/OISIiIiISNHWiRURERESSpE60iIiIiEiS1IkWEREREUmSOtEiIiIiIklSJ1pEREREJEnmnMt1G5JmZouAuUk+bCvgxww0J1n50g7In7aoHQ3lS1vUjobSbUsX51y7oBpTCJTZgcmXtuRLOyB/2qJ2NJQvbclYZhdkJzoVZjbJOVepdmyUL21ROxrKl7aoHQ3lU1uKWb78PedLOyB/2pIv7YD8aYva0VC+tCWT7dB0DhERERGRJKkTLSIiIiKSpKbUiX4g1w0IyZd2QP60Re1oKF/aonY0lE9tKWb58vecL+2A/GlLvrQD8qctakdD+dKWjLWjycyJFhEREREJSlMaiRYRERERCUST7ESb2R/NzJnZVjl6/ZvMbJqZfWpmb5hZhxy1Y6SZfR5qy4tmVp6LdoTacqKZzTSz9WaW9dW8ZnaEmc02sy/NbFi2Xz+sHQ+b2Q9mNiNXbQi1o5OZjTezz0L/LhfnqB2bmtnHZjY11I4bctGOsPaUmNkUM3sll+1oapTZ9dqSF7mtzN7QDmV2/XY0qcxucp1oM+sEHAbMy2EzRjrndnPO7QG8Alybo3a8CezqnNsN+AIYnqN2AMwAjgPezfYLm1kJ8H/AkUAP4FQz65HtdoQ8ChyRo9cOtw64zDm3C7AvcGGO/k7WAP2dc7sDewBHmNm+OWhHnYuBz3L4+k2OMruBfMltZbb3KMrscE0qs5tcJxq4E7gcyNlkcOfcT2E/bpartjjn3nDOrQv9OAHomIt2hNrymXNudo5evg/wpXNujnNuLfA0cGwuGuKcexdYkovXjmjHt865T0Lfr8CHUEUO2uGccytDP5aGbjn5/2JmHYGjgIdy8fpNmDK7flvyIreV2Z4yu0E7mlRmN6lOtJkNBKqcc1PzoC23mNl84NfkdlSjzv8DXst1I3KkApgf9vMCchA++crMugK9gY9y9PolZvYp8APwpnMuJ+0A/orvzK3P0es3OcrsRjXV3FZmx6HM3iDjmd08U0+cK2b2FrBNlLuuAq4EDs91O5xzLznnrgKuMrPhwO+A63LRjtA5V+EvBT2RiTYk05YcsSjHVLYGMLNWwPPAJRGjcVnjnKsF9gjN/XzRzHZ1zmV1/qGZHQ384JybbGYHZ/O1i50yO/m2hM7JeG4rswuPMtvLVmYXXSfaOXdotONm1gvoBkw1M/CXwD4xsz7Oue+y1Y4ongTGkKFAbqwdZnYmcDRwiMtwvcMk/k6ybQHQKeznjsDCHLUlb5hZKT6Mn3DOvZDr9jjnlpnZO/j5h9lexNMXGGhmvwQ2BbYws8edc6dnuR1FR5mdfFuyldvK7MKizK4nK5ndZKZzOOemO+e2ds51dc51xf8n3DMTYdwYM9sx7MeBwOfZbkOoHUcAVwADnXOrc9GGPDER2NHMuplZC+AUYHSO25RT5nst/wA+c879JYftaFdXfcDMyoBDycH/F+fccOdcx1B2nAKMUwc6s5TZMdui3FZmN6DMri9bmd1kOtF5ZoSZzTCzafhLlTkpRQPcC2wOvBkq3XRfjtqBmf3KzBYA+wFjzGxstl47tEjnd8BY/GKMZ51zM7P1+uHM7CngQ6C7mS0ws3Ny0Q78p/gzgP6h98anoU/02bYtMD70f2Uifn6dystJtuVLZkOe5LYy21NmN9CkMls7FoqIiIiIJEkj0SIiIiIiSVInWkREREQkSepEi4iIiIgkSZ1oEREREZEkqRMtIiIiIpIkdaJFRERERJKkTrSIiIiISJLUiZaCYGbfmFm+bkErItKkKaOlKVInWopapoLdvD+b2eLQ7bbQtquxzr3KzOaZ2U9m9rSZbRF0m0RECk0hZLSZVZjZS2a2JLQr4flBt1cKkzrRIqkZAgwCdgd2A44Gzotx7m/w27H2BToAZcA9WWijiEhTFWRGPw58DbQHjgL+ZGb9MtNsKSTqREsh2dvMZpnZUjN7xMw2BTCzo83sUzNbZmYfmNluoeOPAZ2Bl81spZldHjr+nJl9Z2bLzewBUw9nAAAgAElEQVRdM+uZQlvOBO5wzi1wzlUBdwBnxTj3GOAfzrn5zrmVwJ+Bk82sZQqvKyKSr4ouo82sFXAwcItzrsY5NxX4N/D/UmiTFBl1oqWQ/BoYAGwP7ARcbWZ7Ag/jRxi2BO4HRpvZJs65M4B5wDHOuVbOudtCz/MasCOwNfAJ8ETdC5jZsFDQR72FtaUnMDXs56mhY9FY6Bb+8yahNoiIFItizGgLOxZ+/64J/H1IkVMnWgrJvaGRgiXALcCpwLnA/c65j5xztc65fwJrgH1jPYlz7mHn3Arn3BrgemB3M2sdum+Ec6481i3saVoBy8N+Xg60ijHn7jVgsJl1Db3OFaHjGokWkWJSdBntnFsBvA9cY2abhj4UHI/yW1AnWgrL/LDv5+LnrnUBLosYiegUuq8BMysxsxFm9pWZ/QR8E7prqyTbshIIXxy4BbDSOeeinPsw8BTwDjATGB86viDJ1xQRyWfFmtG/Brrh/3x/x4+MK79FnWgpKJ3Cvu8MLMSH2i0RoxEtnXNPhc6LDMzTgGOBQ4HWQNfQcQMwsytDc/Oi3sKeZyZ+wUqd3UPHGnDOrXfOXeec6+qc6xg6ryp0ExEpFkWZ0c65uc65o51z7Zxz++CnpXycxN+LFCl1oqWQXGhmHc2sLXAl8AzwIHC+me0TKlO0mZkdZWabhx7zPbBd2HNsjr+UuBh/Oe5P4S/gnPtTaG5e1FvYqf8C/hAqfdQBuAx4NFqjzaytmW0fal8P4C/Ajc659en+hYiI5JGizGgz28XMNjezFmZ2OnB46Bxp4tSJlkLyJPAGMCd0u9k5Nwk/5+5eYCnwJfVXYN+KX9yyzMz+iA/WufgRhlnAhBTbcj/wMjAdmAGMCR0DIDQqckDox62AV4FV+Ll3DzvnHkjxdUVE8lWxZvSA0J9nKXA+cIRzblGK7ZIiYtGnB4mIiIiISCwaiRYRERERSZI60SIiIiIiSVInWkREREQkSYF0os3sYTP7wcxmxLj/12Y2LXT7wMx2D7vvGzObbn5L0ElBtEdERGJTZouIpC+okehHgSPi3P81cJBzbjfgJiCyMkE/59wezrnKgNojIiKxPYoyW0QkLc2DeBLn3Ltm1jXO/R+E/TgB6JjO62211Vaua9eYLycikrcmT578o3OuXS7boMwWEUlMvMwOpBOdpHPwdRjrOOANM3PA/bHq55rZEGAIQOfOnZk0SVcRRaTwmNncXLchScpsEWmy4mV2VjvRZtYPH8j7hx3u65xbaGZbA2+a2efOuXcjHxsK6gcAKisrVdxaRCTDlNkiIrFlrTqHme0GPAQc65xbXHfcObcw9PUH4EWgT7baJCIi0SmzRUTiy0on2sw6Ay8AZzjnvgg7vpmZbV73PX4/+qirxUVEJDuU2SIijQtkOoeZPQUcDGxlZguA64BSAOfcfcC1wJbA38wMYF1oVXd74MXQsebAk86514Nok4iIRKfMFhFJX1DVOU5t5P7BwOAox+cAuzd8hIiIZIoyW0QkfbmoziFSUEZNqWLk2NksXFZNh/Iyhg7ozqDeFbluloiIRKHMlmxRJ1okjlFTqhj+wnSqa2oBqFpWzfAXpgMolEVE8owyW7Ipa9U5RArRyLGzN4RxneqaWkaOnZ2jFomISCzKbMkmdaJF4li4rDqp4yIikjvKbMkmdaJF4uhQXpbUcRERyR1ltmSTOtEicQwd0J2y0pJ6x8pKSxg6oHuOWiQiIrEosyWbtLBQJI66hSixVnprFbiISP5QZks2qRMt0ohBvSuihqxWgYuI5B9ltmSLOtEiKYq3CjyZQNbIiIhI5imzJWjqRIukKIhV4BoZERHJDmW2BE0LC0VSFMQqcNU0FRHJDmW2BE2daGmSRk2pou+IcXQbNoa+I8YxakpV0s8RxCpw1TQVEUlMurmtzJagaTqHNDlBXY5rbBV4IjqUl1EVJXxV01REZKMgcluZLUELpBNtZg8DRwM/OOd2jXK/AXcBvwRWA2c55z4J3XcmcHXo1Judc/8Mok3SNCWy4COoxSUQexV4ooYO6F7vFwOopqlknjJb8kWii/SCym1ltgQpqOkcjwJHxLn/SGDH0G0I8HcAM2sLXAfsA/QBrjOzNgG1SZqYupGKqmXVODaOVERe8suny3GDeldw63G9qCgvw4CK8jJuPa6XFqhIpj2KMltyLNHMhvzJbWW2hAtkJNo5966ZdY1zyrHAv5xzDphgZuVmti1wMPCmc24JgJm9iQ/2p4JolxSvaKMXiY5UxLoc54C+I8Y1GAkJopxRvOdId2REJFnKbMm2dDIbksvtoErQxXoeZbbUydac6ApgftjPC0LHYh1vwMyG4EdE6Ny5c2ZaKXkpMsj67dyO5ydXNZgbFxnGdSJHKqJdjqsTOc8u3jw8aHxu3agpVVw/eibLqmvqvcalz3zKpLlLuHlQrxT+RkQyTpktaQnP7dZlpaxau46aWgckn9mQeG4DaWf2yLGzqVpWjeE76XXPo9yWSNnqRFuUYy7O8YYHnXsAeACgsrIy6jlSfKJ1Yp+YMK/Bm6S6ppYSM2pdw7dG5IKP8MUl0UY2wkdCYo2U3PDyTH6uWR93kUtk28M54IkJ86js0lYjGpKPlNmSssjsCx9EqJNMZkPiuV33feR9qWR2ZMuU2xIpWyXuFgCdwn7uCCyMc1yagETKFUXrxMb6bVzrXMLliwb1ruD9Yf2j9ghg40hIrPl2S1fXNForNFrbw7nQOSJ5SJktDSRaYq6x7KuTTGZDYrmdycwG5bbUl61O9GjgN+btCyx3zn0LjAUON7M2ocUph4eOSZFLdxFgNHULPJJZ8NFY8f1kyxZVLave8Esm2mhJJNUWlTylzJZ6glgEGCmVzIb4uZ3pzAbltmwUSCfazJ4CPgS6m9kCMzvHzM43s/NDp7wKzAG+BB4EfgsQWpxyEzAxdLuxbsGKFLcbXp6Z0K5PyQTi0lVruPSZTwG48+Q9eH9Y/0bDOFrxfYBVa9YxakpVzOL85WWlUZ/PYMMvmUSotqjkgjJbkpVoZkPiuZZKZkP03Dag387tMp7ZoNyWjYKqznFqI/c74MIY9z0MPBxEO6QwjJpSxdLVDefIQWKLAMMXe4RbXbMeSK4If939N7w8s16bllXXMPyF6dx6XC9uPa5Xg8UoQMLtClex/Ad2//YLZrbfjh/adWTo4TvBuHFw990sn/k5s5ptwRet2vNZj73Z/9wTOfrAXQJbaS5SR5ktyUgmsyH+IsBwqWR23TmT5i6ptz7GAc9PrqKyS9tAMztS5HSTaPkM6W3oIoVDOxZK1sWbTxZvEWBdICVyyS2ZIvx1Cwgjf0nUPUe80ZE7XptF3/depjPVrF71M6W1tZSur6Ht6p9ov3Ix7VcuYcvVy5m2zY60Wl/D3t98SvP1/hfHz223YtPn2sIXX7CmzZZM3moH2i3/kRO+ns5mn7zCuiduYG733Zm/dU923Hp71rbfniraprS7oohIqpLJbGiY24l0VJPdOGX854uiLjBvLLMT/V1isKEa1PjPF0XtEEdb+D70ualgNKhEAsrsYqROtGRdvPlksRYBhodP3xHjAp9vnEoh/0G92jNo4hh4/Z56x9c2a87Sllvwfau2LNx8K+ZW7MCR38+Eli3hiivgmGNg2jQ2ff99qKqCK67giKoOfL3KB3FpbQ29F87mgK+ncOA3U7jw8ydoFvp1Mb399ryzXSWv/diXQff+FkoaTkUREQlSspkN9XM7bzI7wd8lFeVlvD+sf6NtiLYQsWZ9w48Mqe6IK/lPnWjJulgjAOVlpQnPh0vkUmEy89ZitWnDczz2GEybBvPnb7itX7iQZrW1rGhRxq+ufI5f9NyW5z79nup16zc8vqy0hOP3quDmupGMLcoYumlnBp23H5x33obzvhk2ZsP3NSWlfNxpVz7utCt3HHgG5dU/scPi+fSZP5OD50ziggnP0fzDZ+Cx4bDnnlBZCXvv7b/usANYrLXrIiLJK8jMjqKxPQfAZ3a/ndvRd8S4RqdjBNHpl8KmTrRkXbRALSst4fqBPeudF2+3KNh4Wa68ZSkrf15XbwQgXpmkZNo0dEB3P1r8m9/AJptAp07QqRPz9tiXMV2as652PR936smX1VA19QeOr+xY79JfrI1hoP6lvVi/EErMWFa2BZM69mRSx578bb+T2OLnlRz7w0xuarccJk6Ev/8d7rzTP6C8HH7xC/j1r+H4432bRUTSUHCZHUW0qRfPT67i+L0qUspsiJ3b0WgxYnEyF6XQeb6rrKx0kyZNynUzJA2RO/m1aVnKdcf4QI62WxT4gIxV/ihWeCezKC/muV9+CTvu6EejTz8dSPwyYKzz2rQspWWL5o2OiBy/V0XU4/X+Htatg5kzYdIk36l+7TWYNw922QUefhj23Tf6P4LkhJlNds5V5rod2aTMLnwFldlRpJvZFWFblcfL7dJmVm9OdGN/D5L/4mW2OtGSE9F284sWPpESnasWbbttSDDMampg1izfIZ04ET78EKZPh2eegZNOAqDbsDFRF8sY8PWIozb8HOu8SHUd5mgLWJKuzrF+PYwZAxde6EfRL70UbroJyjQSkg/UiZZClNeZnYAgMrustCTqQEdkboOqcxSTeJmt6RySNeGdwWZRtnuNtiAjUiLzyuJttx1zgUd1Nbz0Ejz5JLz5Jvz8sz9eXu7nGh97LBxxxIbTE52Pl0w1kfGfL4r6yyZyMUyjmjXzixcPOgguvxzuuAOWL4cHH0z8OUSkycvrzE5SupldYha1Tna83Jbip060ZEVkSEaGcaISmVfW2Nat9UJ93jw/p/jBB2HxYqiogHPP9VMg9t475kK9ROfjJbqgpkG70lBv5LrrSTzf9wu2ef55Pz969939bdddfbUQEZEo8jazU5ROZkeOQAfdNu0FULjUiZasaCwkE5HowpPGQm3v2qV+Id7o0fDuu/7gscf66Q/9+vmR3EZEq18dLfiinbdqzboGlywhmIUn0RbPXN/xYG5ZuYIt//lPWLnSn9ismZ/nvfvusNtu0KOHn0O9444qmycieZXZQWRjOpldNxc62WogiYiW2aorXTjUiZasSOTTerT5dXULVSqS+HQeeTmu2fpadv/2fxz65UcM+PIjdvhxnr+jVy+48koYPBi6dEn2j5TwNIvI86Jdukx2ZXos0X7xvd51L6bvsT/vX34wfP01TJ268fbxx/DssxtPPvJIP59aZfJEmrRcZna4oLIRUs/sOpnI7WiZrbrShUOdaElbtNqbkQst4s0zW+9coAsybt16OR+9NYbOP8yj+49z2enHubSsWcO6Zs1Yutd+cPUfYOBA6NYtkD9/shIdEYHkL/PF3YCgWTPYfnt/O+64jXeuWAGffw7PPQcjR8KAAXDOOXDiiQmNyotIYcm3zI417a2uAkg+dCYTze1AM1vynqpzSFriLQipk3CptiC88IKvjwwsbtWG2W07Mb/Tjmx3VD/2Pv80aNMmuNfKsFgj1rGqeEDqO3CNmlLFX16dxakvP8AvZ/+XLsu+Y3aHHVh85fX84renaWQ6QKrOIbmUd5kd1q5Cnxec7cyum2JSElr0mczovyROJe4kYxLdzjVajc3A/7PPmgX77+9HWseOhbZtg3vuHIj1dxurFivADS/PZOnq5EpERQZ/s/W1DPzsXS5773E6Lf+eRZX70e7eO2GffQL5czV16kRLLuVVZheZXGV2Mo+V5GW8xJ2ZHQHcBZQADznnRkTcfyfQL/RjS2Br51x56L5aYHrovnnOuYFBtEmCF22kINFLTguXVSdfqi0R//ufn4bw73/DlCmw+ebw1FMF34GG2JfzIj/2VtfUcv3omaxZt75BqJaXlXL9wPiXQyPn5K1vVsKonv14tfv+nDr1dS758BlfreRXv/ILMlOYPy75RZndNORlZhexXGV25HNrPnX2pN2JNrMS4P+Aw4AFwEQzG+2cm1V3jnPu0rDzfw/0DnuKaufcHum2QzIr1gri8palDT5FR5ORLU/vvRd+/3v//b77wu23w8knQ8eOwb9WDiSzpWy0ah8Am23SvNEwjRX8a5uX8s+9juH5XQ9hRtvP4bbbYL/94I03fIk8KUjK7KYhLzO7yOU6sxO9X4ITxKqhPsCXzrk5zrm1wNPAsXHOPxV4KoDXlYCNmlJF3xHj6DZsDH1HjGPUlKoN98VaQexcaIV2HEGurq7n3XehUydf6/nDD+Gyy4qmAw1+sU1Zaf1yc8nOTE4kTBv7Zdm6/ZZwzTUwYYKfG33AAfDVV0m2RPKIMrtIFFxmF7l8yWx9AMqeIKZzVADzw35eAESdPGlmXYBuwLiww5ua2SRgHTDCOTcqxmOHAEMAOnfuHECzBeovTgiftxVeq7Lu52iWVddQWlI/JpoZtC4rZdnqmszMo/vpJz9lY/x4vyFKp07BPXceibYavN/O7aIu9tm0tFnU0aVEwjTehjD1fpn27Akvvwx77QX/+Y+fey6FSJldwAoys5uIvMtsybggOtHRPmjFWq14CvBv51z4v3xn59xCM9sOGGdm051zDYa5nHMPAA+AX6SSbqMFrh41nScmzNvwjxVv3lYsJWb1aoQCrHfQskVzplx7eHCNdQ4mTYIHHvAd6FWrfJ3na64J7jXyULQ5iZVd2jaY5wjRa5j227kdfUeMS3hzgUZXem+9tf+6enUG/rSSJcrsAlVQmd1E5V1mS0YF0YleAIQPBXYEFsY49xTgwvADzrmFoa9zzOwd/Nw7XStOUrLlgUZNqaoXxrHEmrcFmd8KdQPn/FSNO+/0W1WfeqrfmrtPnyZZei3eYp94IyCRO2HFqxUbM4w328x//f3v4cYb/eh0jx5+jvQppxRUCcEmTJmdB4o6s6WenGa2ZFQQneiJwI5m1g2owofuaZEnmVl3oA3wYdixNsBq59waM9sK6AvcFkCbmpRUtg0dOXZ2o2HcmFuP65WxrVDr+dOffAf6ggtgxAjYYovgnruIRAZ13xHjYu6EBTR4zzw+Yd6G82K+h9q08fPPP/zQlxScORMef9xPsfniC//vJPlOmZ1jRZ/ZkpCsZLZkVNoLC51z64DfAWOBz4BnnXMzzexGMwsvfXQq8LSrX5h6F2CSmU0FxuPn181CkhJv29BYEhl1KCstoU3L0qj3VZSXMah3RdSFFIHNyVq7Fm65Ba6+Gk4/3VfjUAc6YfF2wopXIqlOzPfQvvvCpZfCgw/CBx/AsmVw9NHw/PP+qoHkNWV27hVtZktaMpbZkjGB1Il2zr0KvBpx7NqIn6+P8rgPgF5BtKEpS2Xb0MZK8VQ0Mm9r6IDuGy4tVdfUBjsnyzl49VX4wx/86OYJJ8DDD2sL6iTF+jfuUF6WVK3YRpn5f6NXXoHJk6GySe0jUpCU2blVdJktgchaZktg1CspArEuw8W7PBerFM/p+3bmmxFH8f6w/hsuNd16XC8qysswfFDX7bQ0/IXpG/7D1zq3IahTDuNvvoE//xl69/Yjm+A7Zs8+C6XRR1cktqEDujdYhV9aYgwd0D3hS7cJX+I95hgoKYEnnki2mSJNTtFktgQqq5ktgQhkJFpyK1q5m8Yuz0UrxRMrTKMtiog3dyupQF6xAh55xFfcmDDBH9t3X/jb32DwYHWe0xU5uyL0c7wSSXWSusTbti2cdBL89a/QqhXccIOuHIjEUNCZLZmVrcyWQKgTXQSSCdfIx6Uanqlcjozqz3/285732MMvGjzpJOjWLaU2SX0jx86mZn39RK5Z7xg5djbvD+u/4ZxoK72TqRVbd4l4UcVJ3FG5lGNuvtlPw3n0USjTqIhIpILObMmYbGd2so+ThtSJLhLphGsq4s3dSsp330GHDjBlSkAtkzqN/dIM4j1Tr8pASSm/738hX2yxLZc9+wjMnQsvvQTt26f1GiLFqGAzWzIm65mNqnqkS9dbJSWBrPB2DiZOLKqtuvNJKvMuk9VgxbgZ9+x9PFf++nqYNs3X8p47N7DXE5HUqCpH/stJZqOqHulQJ1pSEmvxSlKfZF991Xe0fvvbjLWzKcvGL81YIydPdayEd9+F5cv9osMVKwJ7TRFJXiCZLRmVy8zWtJ7UaDqHpCytS0vOwU03QdeucFqDfR4kAKnOu0xG3EvElZXw3HNw5JF+l8mXXvIVPEQkJ7I9hUSSk/PMlqSpEy258cAD8NFHcP/9qsCRQUH/0oy27Wz4NrUQMXJy2GFwzz3+asPll8MddwTWFhGRYpPzzJakaDqHZN+kSXDRRX6EcvDgXLdGElS3IKVqWTUOvyDl+clVHL9XRfxLxBdc4P+9//IXv8uhiIhkXMqZLQnTSLRk1+LFfne7bbaBxx5TLeECEmtByvjPF20ovxTTHXfAzJnwxz/CmWdCixYZbKmIiKSV2ZIQ9WAkOz7/HC65BHbYARYu9HNlt9wy162SJKS1IKV5c7j4YvjpJ3jnnWAbJiIiDWgRYeapEy2ZU1PjO8v9+8Muu/hdCI84Av77X1/6TApK2uWXDj3Ub74yenSArRIRkWiyUTKvqVMnWoK3Zo3fhbBzZ78D4Zw58Kc/wfz5fntvdaALUtrll8rKYMAAePZZWLkyAy0UEZE6qg2eeYF0os3sCDObbWZfmtmwKPefZWaLzOzT0G1w2H1nmtn/Qrczg2iP5NCUKbD33nD11bDnnvDKK/DVVzB8uHauK3CB1JkdOhQWLYI778xYOyUxym2R4qba4JlnzrnGz4r3BGYlwBfAYcACYCJwqnNuVtg5ZwGVzrnfRTy2LTAJqAQcMBnYyzm3NN5rVlZWukmTJqXVbglYTY0fbb75ZmjXzpewO/roXLdK8tFxx8Gbb/oPV1tvnevWZJ2ZTXbOVea4DVnNbWW2iBSqeJkdxEh0H+BL59wc59xa4Gng2AQfOwB40zm3JBTAbwJHBNAmybbbb4frr4dTToEZM9SBlthuvRWqq/1mO5Irym0RkTQFUeKuApgf9vMCYJ8o5x1vZgfiRz8udc7Nj/HYqNcZzGwIMASgc+fOATS7MEUWTg96N6OUffgh9Ozpy9ZJ0Qrk/de9OwwZ4heaHnIIDBqUmcZKPBnPbWW2l7eZLU2C3n+ZFcRItEU5FjlH5GWgq3NuN+At4J9JPNYfdO4B51ylc66yXbt2KTe2kEUrnD78hemMmlKV24atWgUTJ0KvXrlth2RUoO+/kSP9tuCnngrvvRd4W6VRGc9tZXYeZ7Y0CXr/ZV4QnegFQKewnzsCC8NPcM4tds6tCf34ILBXoo+VjWIVTh85dnZazztqShV9R4yj27Ax9B0xLvn/YBddBN9/D+eem1Y7JH+NmlLFZc9ODe79t9lmMGYMdOkCxxwD06cH1FJJkHI7C/I2s6XoBZ7ZElUQneiJwI5m1s3MWgCnAPUKwZrZtmE/DgQ+C30/FjjczNqYWRvg8NAxiSIThdPT/qT6+OPw8MNw5ZW+HrQUnbr3SG2MRcgpv/+22greeANatfKl7775JvVGSrKU21mQl5ktRS9jmS0NpN2Jds6tA36HD9HPgGedczPN7EYzGxg67SIzm2lmU4GLgLNCj10C3IQP9InAjaFjEkWqhdPjjVqkPFLy009www1+busBB/hFhVKUor1HwqVVuL9zZ3j9db/QcP/94bXXUn8uSZhyOzvyKrOlychoZks9gdSJds696pzbyTm3vXPultCxa51zo0PfD3fO9XTO7e6c6+ec+zzssQ8753YI3R4Joj3FKpXC6Y2NWiQ9UrJ6tZ/Put12vuN8xBF+84zmQaxRlXwUb9QikML9u+4K48ZB69bwy1/CWWfB0rhVLiUAyu3My4vMliYn45ktG2jHwgKSSuH0xkYtkhopWbQIevSAyy/3i8ImToQXXoBttkn5zyT5L9Z7pMQsuML9vXvDJ5/AVVf5KUI9esBLL6X/vCI5lPPMliYpK5ktQDAl7iSLBvWuSOo/QGOjFkMHdGf4C9PrhXbMT6qXXAILF/qNMg49NLmGS8GK9R4JPIw32cRv1nPccXD22b703dlnw0MPQTN93pfClNPMliYpa5ktGokuZqOmVNHMolWj2vhJNeGRktdegyef9AsI1YFuUrK+deyee/qrHMOGwSOP+I18RJqI8palUY8nndnSZOk9kj1pb/udC9pCtnGjplQx9Lmp1Kxv+O+b9CfSGTPgqKN8WbIpU/yIoUimOQcnnwwvvuhrSe+7b65bFIh82PY725TZibl61HQenzCvwfHSEmPkCburEySSA5ne9lvy0PWjZ0btQJuReAd62TI/hWOPPWDFCj8qqA60ZIsZPPAAdOzot5OfMyfXLRLJmFFTqngiSgcaYLMWzdWBFslD6kQXqWXVNVGPO0fjYbx+va/9vNNOcPfdfhOV//0P9om2K7BIBpWXwzPPwOLFflv5W26BNWsaf5xIgRk5dnb07XqB5THyXERyS51oqW/mTNhvPzjnHNhxR5g0Cf7+d9hyy1y3TJqqPn3gs8/g6KPh6qt9JY/vv891q0QCFa8smSpviOQnVecoUm1alrJ0dcPRizYxFq0AsHatr4ywdCn8619w+un+krpIrnXsCM89B6NHw7HH+kWul16a61aJBKZDeRlVUTrSBqq8IfE5539/r1u38VZT478uWQITJvireStWbLytXOkf17IllJZuvLVo4Y9tvrm/tW4NFRXQoQNsu63fYVb9gg3Uic5zo6ZUMXLsbBYuq6ZDedmGMI08FjlF47pjejL031Opqd14gbC0xLjumJ6xX+zuu+GLL2DMGL/phUi+GTjQT+t49VV1oiUvpZrZ0cqSGfDrfTtrPrQ0tGyZn2Y5erSvrf/NN40/pnlz3zFu1cp/Bb9bbE3NxtvatbBqle9gR1Na6q9Mt23rb61a1b9tvrnvbHfoAG3abDy2ww7+sUVG1TnyWN3OVeGhWtrMwKjXOY5VbSNamMcM4+++83OgDzwQXnklI38ekUBcfjn89a9+hJS3kGUAACAASURBVKVVq1y3JmmqzlG8sprZUnyc81eCZ83yVYkWL/YjxpG3pUt9/oEfFR4wAPr29aPIpaW+s1z3tXVrfyWvZ0/fmU1kFNk5vztx3WtVVcG33/rb4sUbb0uX+g53XbtWrYKffvIj4JHatPELxI86CrbfHrp1K5hCBfEyW53oPNZ3xLiol/eiqSgv4/1h/VN/sbPPhiee8HOid9wx9eeRJimrv/zHj4f+/WHUKD+1o8CoE128sprZUlhqazd2PJct81/rvp87F8aO9Z3ntWs3PqZz540jx5Gjvd26+d/Ve+zhv09SxjJ7/Xr48Ue/Mdvy5b5zvWSJ32vi6ac3jnCbQZcucMABPs8PO8xPG8lD8TJb0znyWLyFJumc28DEifDoozB0qDrQkrTI0beqZdUMf2E6kEAlmFT07et/ibz2WkF2oqV4ZS2zJb+tWePLc44du3EE9/vvfQczmpIS2H9/X1J2221hm218NawUOseJyGhmN2sGW2/tb+HOOAPuucdPGf3qK3+bMQNefx0ee8yf0707dOrkR8+32GLj17rvt9vOd7jziDrReSzWQpNY56bs73/3b9Crr079OaTJGjl2dr3L1wDVNbWMHDs7M53oFi38roYzZgT/3CJpyFpmS+6tX+8z6D//8Qv3qqr8tMjvv/ejywA9ekDXrr6iUIcOvmPZtq2f2lBe7r/W3Vq0yFrTs57Zdbbc0lf/2m+/jcec83+Pb7zh/y5//NF/6Fi+3E8NWbGi/vzsceOgX7/MtTFJgXSizewI4C6gBHjIOTci4v4/AIOBdcAi4P855+aG7qsFpodOneecGxhEmwpNtEsr0RaaxJpfl/Lqbef8J8EBA/ynPZEkxRpRy+hIW7duPnQlJcrs9OUssyU3Fi/20x2nTYN33vG3xYv9fR07+kzq1ctPS2jfHnbZxVe7ysNKFjnJ7FjM/N9br15w2WUN71+/3k8J+fJLOOQQPxK9xRZ+KkjdrWvXjd/37p3VBYxpz4k2sxLgC+AwYAEwETjVOTcr7Jx+wEfOudVmdgFwsHPu5NB9K51zSa0OKrb5ddEWo9QtPIGGq7qjHUv50+O0abD77n5zlbPPTvvPIk1PrHmgJWasdy4zc6TvvBP+8Af497/h+OODe94syPWcaGV2+nKa2ZI9zvnSmn/8I8yfv/F4p06+M9evHxx8sO+8FZBYmV1eVspmmzTP3/fpjz/6qR9z5vh55HW35cs3ntOnD1xxhe9Md+0ayIeYjC4sNLP9gOudcwNCPw8HcM7dGuP83sC9zrm+oZ+bfCDHekNnZeHJiBEwfLi/FNWhQ2ZfS4pStA5FpFjVCFK2Zo3/5TVjBnz0kb9sWiDyoBOtzE5TTjNbMm/1ar9O6O67YfZs2GsvOPVUX+GiZ08/8pyHI8yJSreKTN6pW5z50Ud+cGXVKn+8vNx3pvff38/JTnHNV6Y70ScARzjnBod+PgPYxzn3uxjn3wt855y7OfTzOuBT/GXDEc65UTEeNwQYAtC5c+e95s6dm1a7cy38UmCsfwEDvh5xVGYasH69LxM2bJgPiA8/zMzrSJMQ/n5uZkZtlFwJfJSjqsq/d7fYwi+Obd06jT9B9uRBJ1qZnYKcZ7YEyzk/svnFF/72+ed+Z9TPPvMjnevX+1HNCy/0pdmyOGc5GyKnI61euy7qBm0ZvaKYCdXVMH06TJnib598ApMn+3/PX/wCXn7Zz0tPQqarc0T7OBY1Y8zsdKASOCjscGfn3EIz2w4YZ2bTnXNfNXhC5x4AHgA/qpF+s3MnkZE7yODCk+++g7PO8iuHBw2Chx7KzOtIkzGod8WGcO02bEzUc5ZV17Cs2od0IKvBKyr8pdb+/f0ow6hRfmW4NEaZnaScZ7YEZ/VqvxvvuHH1pwG0aOH3Sujd2486H3qoL79WwCPO8YRnNsTO7boBkYxXXQpKWZn/8NOnz8ZjCxf6Er7vv+8XcQYoiE70AqBT2M8dgYWRJ5nZocBVwEHOuTV1x51zC0Nf55jZO0BvoEEgF5NoK2MjlTYzVq9dR7dhY4L9BPjaa74D/dNPvirHeecVbUhIbiRaoSCQ1eAHHAB/+QtcdBHccgtcc03qz9V0KLOTlNPMlvSsW+c7UfPn+9v99/sqEIMH+6kZO+3kb126+M1JmqhEcjsrFTwyoUMHX8J36NDAnzqIYZuJwI5m1s3MWgCnAKPDTwjNqbsfGOic+yHseBsz2yT0/VZAX2AWRS7eCljDX/bGYOnqGhwbPwGOmlKV+ouuWeNrUP7yl37l8OTJcP756kBL4IYO6E5ZaUlC5wayGvx3v/MjRzfdBAsWpP98xU+ZnaScZLak5+KL/dzlTTbxHeT99/c58c47cO21vo7zxRfDkUf6HfSacAcaEs9t1TevL+1OtHNuHfA7YCzwGfCsc26mmd1oZnWlj0YCrYDnzOxTM6sL7F2ASWY2FRiPn19X9IEc65JfRXkZX484is02aV5vcj9s/ASYkrpFWHfd5UfsPv64oBZiSWEZ1LuCW4/rRUV5GYZ/X7dpGb3kUCCXv83g1lv9HMfbbkv/+YqcMjt5Wc9sSdyqVf4K6z33+EVlgwb5ilN33+3XTVx1le8wv/aanyu7bBlcf32uW513InO7JMYAm6Ys1RfIRy/n3KvAqxHHrg37/tAYj/sA6BVEGwpJtFqi4XVDA6/heP31vhj8k0/6T+IiGRY53y5WSbBka+XG3Kq2Sxf4zW/gwQfhyiv9jl8SkzI7OVnPbInNOb8I8N134e23YcwYP88ZoGVLX6+5Wzc46CC44AJfr1kSEp7bGc/sItG0r1/kSN0bKNYbK9bcpJQ+AU6c6EfnBg9WB1pyprH3fCIa3ap2+HBfluqOO2DkyMD/DNJ0ZTWzxc9jnjkTvvnG38JrAs+ZA0uX+vO22QbOPNPXit91V78joKYoBiIrmV0E1InOkciRujqjplSxas26BsdT3uFq/Hhf2mXEiMbPFcmgWO/5RDW6Ve0OO8BJJ/mFQ9dcox04JVBZy+ymas4ceOstvxPpW2/Vr5xRVrZxR7rKSl954YAD/P95dZozJuOZXQTUic4jscootWlZynXH9Ez+Tbdsma+TCOpQSE4FcUkvoUvml10GTz/tyzb+4Q/pNFmkUYFndlOzerXvNF9/PUyd6o917AgnnuhLV+6wg991bqut1FnOsqxldoFTJzpHor1BY5VRatmieXJv3k8+8eXrnnjCFx4fMKDJrzyW3Anqkl5Cl8wrK+HAAzcuotX7XgKS0cxuKqqr4fbbfXWoGTP86LNzfie5u+6Cww+H7t3VYc6xrGZ2gdPOBDlQ9watCu18VfcGjVWjMeFPbe++C/vt53dxe/JJX1D+k0/g9dcVSpIz8S7pJSNaCaaol8z/8AeYNw+efz6l9opEylhmF7tVq/zV0Kef9qPNe+/ty8vNmgV77umPvfSSn/980UWw8876XZUHsp7ZBUzDNDkQ6w1aEmO75IQ+tb33HhxxhF9ocdddvlJBeXlQTRZJWawORdWyavqOGJfwZcKEF7occ4wf2Ro5Eo4+GjbbLO0/gzRtGcnsYrVggZ+i8cwzvnpGbejvzcxXzbjqKrj55ty2UeLKemYXMHWicyDWG7TWOcpKS5IvKTNpEhx1lF908e670K5dkM0VSUu8nbCSvUyY0EKXZs3giit8RZqtt4aBA+G00/y0phYtkm6/SOCZXeicgx9+8NMx5syBr77yZefef99fBQLYbjv44x/9FKvu3f0H2003zW27JSFZz+wCpukcORCvcH/kJhW3Htcr/htw5kw/Ar3llvDmm+pAS95pbCesjGxKcc45fmvfM87wo2IDB/qrNOeeC+PGbRwdE0lAoJld6D75BNq29f+ffvELP23wuuv81dA+feDOO/05X37pq0KdcAL06qUOdAHJSWYXKI1EZ1lj5ZCS+tQ2Zw4cdpgfXXvrLb+qWSTPhF/SS3QOafgirvKWpTgHy6trkrsceOCB/nbPPf4D5lNP+dtDD8G228KwYX4epkgj+u3cjicmzCN84kZKmV3IvvkG/v1vuOUWX/nphhv8+pvttvMVNMqa8BSWIpOzzC5A6kRnUaDlkKqq4JBDYO1aP+K2/fYBt1YkOHUdjb4jxjW6Wjvy/8nS1TUb7ktplXhpKfzyl/62erXf4ezaa+Gvf1UnWho1akoVz0+uqteBNuD4vYqs81xTA/Pn+znNVVX1b59/7rfMBr8T4IgRsO++uW2vZFROM7uAqBOdRYGVQ6qt9XOgFy/2m6n07BlgK0Uyp7HtkyH2/5M6aRXrb9nS16AdNw5eeCH5x0uTE+396IDxny/KTYOC8vPPcN99fk3N9Onw2We+Ix2uZUt/hbNLFzjrLDj2WA3YNDE5z+w8p050loyaUhVcOaQXX/SF6Z96yl9OEykQiazWTuT/Q9olxFq1gpUr03sOKWp1l6eLqoxdTY2fljFuHPzjHzBxIlRUwG67wZFHwk47+U5zRYW/tW6tknNNXN5kdp5SJzoL6i51xJJUOaT16+HWW33YnXhiAK0Tya7IOaSjplTRd8S4DQFd3rK03uXAaJqZ0W3YmNTn27Vq5ad2rF/vq3mIhIk19S5c3pexq62FCRN8HeZPP/UVNObO3biotls3P6Xp4otz207Je3mR2XkqkE60mR0B3AWUAA8550ZE3L8J8C9gL2AxcLJz7pvQfcOBc4Ba4CLn3Ngg2pRP4l3qSLoc0vDhfuXzY49BSezVsyKFINrOWKXNjNISo6a2Yf3dOnW1eVOeb1dXO3r1at+hboKU27E1dnk6b8rYOec3Lvnvf/185ro5zXVfq6v9moDddvOVM0491U/H6NMHevTQKLMkLWeZnafS7kSbWQnwf8BhwAJgopmNds7NCjvtHGCpc24HMzsF+DNwspn1AE4BegIdgLfMbCfnXFHVn4p3GSOpckgPPwy33QYXXAC//nVArRPJnWidlZr1jvKyUjbbpHmDld7NomxukdJ8u80391+XLGmSnWjldnzxMrsiH0bSVq70m2o99JCfngF+UKVDBz8do3dvv+lQnz6+BGrr1rlrqxSVnGV2ngpiJLoP8KVzbg6AmT0NHAuEh/GxwPWh7/8N3GtmFjr+tHNuDfC1mX0Zer4PA2hX3ohVuLyivCzxN9E778B558Hhh8Pdd2sEQYpCrM7K8uoaPr3u8AbHuw0bk9TzxFS3luC995rqB1LldhzxMvv9Yf2z04iff/ZTML79FhYu3Hj79lv/vv3+e//7YNgwX+q0c2dorhmaklk5y+w8FcT/uApgftjPC4B9Yp3jnFtnZsuBLUPHJ0Q8tvA/mkRIZHVrXP/7Hxx3nN/x6ZlnFJRSNGJ1VmLNN032/Jj23BO22grGjm2qnej8z+333/cLqHv39tMRsrh9e9qZnSjnfM3lJUtg6VJ/mzMH/u//YPZsX8I0XOvWvsb5Pvv4qX0qMydZlrPMzlNB9MaiDYlGToyJdU4ij/VPYDYEGALQuXPnZNqXc2ntH19b67csbtYMXnkFyssz3FqR7Em2sxJY56ZZMz+K99JL8PHH/rJ305Lx3E47s0eNgttv9983a+a3ju7b19cp3m03//MmmyT/vAlIK7MjOQc//ugX+U2b5hf31d3mzfPzlqM57TQ/JaOiwnect902qx8kRKLJWWbnqSA60QuATmE/dwQWxjhngZk1B1oDSxJ8LADOuQeAB+D/t3f3UVbV9R7H31950MF0DSoiDKLY9VIqCDK37LJuElpoGiD5eK1LZcvL8mFpGQmXHoyWS5LlxcryRmqWV1NbKlIuQwWpVsvMURBBRb2WykCBISiC6MD3/vHbA2dmzvOZc/be53xea501c87ZZ/aXOcxnvrP37/fb0Nramnv0ekKVfVWrhQvDOp533hmuDCVSR0ptVnq1uZk7NzQ2EybAHXfAmWeW+89Io6rndsWZfd11cNllsGJFmEz91FPwq1+FccCwt7E+6SQ44wxobQ2Xlt5333CrcMhbUZm9ezds3br3aPKGDWFS39q18OKLYbzyq6+GCaydDj00DL047rhwAaBhw+Dgg2HgwL23wYNh0KCK6hephlgzO4HMvbJ+NArXF4GTgXbgSeDf3X1NxjaXAKPcfUY0QWWau59jZscCdxLG0w0FlgJHF5qg0tra6m1tbRXVnQobN4ZfEmPHwtKlGgct0ts2bYLJk+GJJ+D66+GKK6r+c2ZmT7l7a1V3UriGmuZ2r2X2rl2wZk1YkeK550Jj/Yc/wNtv99y2f//QTA8YEI7itrSE5vSAA8Jtv/3CyhV9+4Zb5+fu0NERbrt2hbWV33kH3norNPObNoXn/vGPMPwi2+/QAQPCMqQjRoRLYh95JIwaFYZhDBhQ+fdBRGomX2ZXfCQ6Git3KbCEsFTSre6+xszmAm3uvhi4Bbg9moCymTCzm2i7ewiTWTqAS+pphndFOjrgi18M4f2jH6mBFqmGQYPChSc+/3n46lfDRK4bbqj7eQepze0+fcJQjtGj9z723nvhPXzlFdi5M0zIy/y4bVs4QtzeHo5qb9sWmu5SDyDtv39oivv2Dfs/+OBwO+igMMxu4MDQrA8dGm5af1yk7lV8JDoOdX8k2h0uvjhckvWmm2DGjLgrEqlvu3eHVQ7mz4fTT4e77qra0ndJOBJda4nLbPdwhLmjY+/Hzs/32Wfv0enO2377qSkWaVBVPRItvcwdrr46NNCzZqmBFqmFffYJY3CPOgouuQQ+/vEwkXfo0Lgrk2owC8M9+vePuxIRSbHG+NPaHRYsCGMek+zdd2H69DDhafp0uOaauCsSaSwzZoTm+aWXwvjV116LuyIREUmoxmiizeDxx+Gb30zuL8XNm+Hkk8PlvL/7XfjZz3T6UCQOp50WLm60bh3cc0/c1YiISEI1Tpc2f344Ij1zZtyV9LR5c7jiVFtb+KX9jW9oIqFInMaNC8uQJWkcr4iIJErjNNFHHAFXXRWa1N/9Lu5q9upsoFevDhcXOPvsuCsSEQjrDquJFhGRHBqniQb4+tfh8MPh8svD+p9VtGhFO+PnLWPErAcZP28Zi1a099yoewN92mlVrUlEStDaGpa8e/bZuCuRGigqs0VEMjRWEz1gQLiM7DPPwE9/WrXdLFrRzuz7nqV9yw4caN+yg9n3PdszlL/znfALWg20SPJMnx7W/T399LDOsNStojNbRCRDYzXREIZLnHRSGHf85ptV2cX8JWu7XCceYMf7u5i/ZG3XDV94AT74QTXQIkk0dGhYqWPz5nBZ6W3b4q5IqqTozBYRydB460Sbwfe/Hy6lfe21YW3YXrZ+y47Cjz/3HDz8cK/vW6TeLVrRzvwla1m/ZQdDm5uYOWkkU8e2VOf1J5wQLrwyZQqcf344a9SnTy/9SyQpispsESlLTTO7xhqviQY4/vhwmd8f/AAuuyyMky5Ttjd3aHMT7VnCd2hz0947d90VPs6eXfa+RRpN52n3zqOGnafdOxUK2nyvzxnKZ5wBP/xhuAjLFVeE3NDqOalVdmaLSMliyewaarzhHJ3mzg1L3s2ZU/aXyDWO7hMfGkRTv65Hq5r69WHmpJF77r91x908PeJ4Ruwer0ksIkXKddr96sVrihrTWvZp+4svhiuvhBtvhJ/8pDf+KRKDSjJbEw9FShdbZtdI4zbRRxwRfinefjv8+MdlfYlcb+5jL2zi2mmjaGluwoCW5iaunTZqz19Nj967nANfeZEHPniiJrGIlCDX6fUtO94vKmgrOm1/3XXhgkhz5sDWrcUVLIlSbmZr4qFIeWLN7BpozOEcnebODatjXHYZDBsGkyeX9PJ8b+7UsS3ZTzXs3MmhV17Ku33789A//+uehzv/8yTh9IRIUuU67Z5L95/Rik7b77NPaKTHjYMFC+Dqq4uuQ5KhrMwm/9EwZbZIbrFmdg1UdCTazA4ys0fM7KXo48As24wxs8fNbI2ZrTKzczOeu83M/mJmK6PbmErqKVnfvmFs8rhxcN558MQTOTfNdiov15uY8811h4suYvSra7jy019h4wEHd3k6KX9ZiSTVzEkjs552HzigX9btu/8s5np95mn7vE44AT77Wbj+enjjjeILT5DU53aReiWzI0k/GiaSVLFndpVVOpxjFrDU3Y8Glkb3u9sO/Ie7HwucCtxgZs0Zz8909zHRbWWF9ZRu//3h178O68FOmQLtPU/PVTKObo9168Jkxl/8gptPmc6DH/63Hpsk5S8rkaSaOrYl62n3b3/m2KJ+FnO9vqSjiXPnwjvvwPe+V/k/KB7pz+0CeiWzM5TbfIs0ukRkdhVVOpxjCjAh+vznwHLgqswN3P3FjM/Xm9lGYBCwpcJ9957Bg2HxYvjoR+Gss8Jlwfv33/N0oXF0eWeXbt0aftkuWBCORH/jGxxy5n/SdP/qLl8zSX9ZiSRZodPuhZZByvf6ohxzDHzuc/Dqq+FnOn0rddRHbudRUWZnMXPSyC4rBIAyW6RYsWd2FVXaRA929w0A7r7BzA7Nt7GZfQToD/xfxsPXmNm3iI6IuPvOCmsqz7HHwq23wrnnwk03hUuDR0oeR9fRAS+/DL/9LVxzTTjte8EF4fMjjmAqgFli1z0USZPuS5YtOHdM9X+WbrkF+mU/HZkC9ZPbOZQ79jmXzu2V2SKViyWzq6RgE21mjwKHZXmqpLXhzGwIcDsw3d13Rw/PBv5GCOiFhKMhc3O8/iLgIoDhw4eXsuvinX023HxzuBz3mDGwaxds384Frz/Jji1vs1/HTt7p38Tyo8axpenAcCpv584wlvrpp2HVqnBbswbefTd8zYkTYf78MJYyQ5L/shJJi9jWEE14A52E3K5JZudQjclIymyRyiV93edSmbuX/2KztcCE6GjGEGC5u/c4v2VmBxJOGV7r7r/K8bUmAF9z9zMK7be1tdXb2trKrjuvVavC1Qx37865SYftQ9vwUYxoGcjglX+G7dvDE4MHw+jRe29jx8Jxx6XxdK9IKoyftyxrs9TS3MQfZ02MoaLCzOwpd2+Ncf81z+2qZnYW3X9RQxh+kaSxlCKNqN4yu9LhHIuB6cC86OMDWXbeH7gf+EX3IDazIVGQGzAVWF1hPZUbPTocWX7jDRgwAJqaYMAAHn5lKzc+3g7t7Ux7vY1pr7Vx4Oa/wZe+BKecAieeGJpoEakZrZpQlvrL7W40/EIkmeotsyttoucB95jZhcBrwNkAZtYKzHD3LwPnAB8HDjazL0Sv+0I0o/sOMxsEGLASmFFhPb2jtecfHJ86Fj71mRhqEZGcyj1tn+3Szw3UYNVnbnej4RciyVNvmV3RcI641PrUoIgkUzmn7eM+1R/3cI44KLNFBOovsxv3st8iknrlrCGa7+pzIiJSPfWW2Y192W8RSb1ST9vX25g8EZE0qafMVhNdhqSOzRGRwqqx/JkkmzJbJL2SnNkazlGiXJeTXbSi5+XCRSR5Zk4aWdalnyWdlNki6ZbkzFYTXaIkj80RkcLKGZMn6aXMFkm3JGe2hnOUKMljc0SkOFr+rHEos0XSL6mZrSPRJco1BicJY3NERKQrZbaIVIua6MiiFe2Mn7eMEbMeZPy8ZTnHyyV5bI6ISCMpJreV2SJSLRrOQc+FvDsnngA9Th/ocrIiIvErNreV2SJSLWqiyT/xJFvQJnVsjohoObNGUUpuK7NFki2tua0mGk08EakXpZxVknRTbovUhzTntsZEo4knIvVCy5k1DuW2SH1Ic26riUYTT0TqhY5ONg7ltkh9SHNuV9REm9lBZvaImb0UfRyYY7tdZrYyui3OeHyEmT0Rvf5uM+tfST3lSvJC3iJSPB2dLEy5LSJJkubcrvRI9CxgqbsfDSyN7mezw93HRLfJGY9/D1gQvf5N4MIK6yla96WRAP44ayILzh0DwFfuXpl3qTsRSZ7eOjpZ7JKXKZXK3M72nkwd28LMSSMZ2tzE+i07mL9kbb29VyJ1L825be5e/ovN1gIT3H2DmQ0Blrt7j3+1mW1z9w90e8yATcBh7t5hZh8Drnb3SYX229ra6m1tbWXX3X0QO4Q37LPjWrj3qfYej+vohkh6VDrLO1c+9FYOmNlT7t5a8Rcqf/81z21ltojkk+TczpfZlTbRW9y9OeP+m+7e49SgmXUAK4EOYJ67LzKzQ4A/ufs/RdscDjzk7scV2m+lgTx+3jLas4y16WPGrizfj5bmJv44a2LZ+xOR9MiVD72VAwloomue28psEammauZ2vswuuMSdmT0KHJblqTkl1DDc3deb2VHAMjN7Fngry3Y5O3ozuwi4CGD48OEl7LqnXIPVs4Vxvu1FpP6keZJLpyTktjJbRGolrtwu2ES7+ym5njOzv5vZkIzTghtzfI310cdXzGw5MBa4F2g2s77u3gEMA9bnqWMhsBDCUY1CdecztLmppKMaaRjcLiLFKXTaMFc+pCkHkpDbymwR6S1Jze1KJxYuBqZHn08HHui+gZkNNLN9o88PAcYDz3kYR/IYcFa+11dDtkHsRjiqYd221ZJJIvWjc9xc+5YdOHsX9c+cgNIAS6elLreV2SKNK8m5XWkTPQ/4pJm9BHwyuo+ZtZrZzdE2HwbazOwZQvjOc/fnoueuAr5qZi8DBwO3VFhPUTKXRoIQxp3HMjy6D1oySaTeFLOofwMsnZa63FZmizSuJOd2RRML41LpJJVM1Z5EJCLJMWLWg1kH8Brwl3mn16SGuCcWxkGZLSLliju382V2w1+xsB4mEYlIcdK8qL8EymyRxpLk3G74JjrJb46I9K4GGO9c95TZIo0lybnd8E10kt8cEeldDTDeue4ps0UaS5Jzu+ASd/Wu802o5Eo5IpIeU8e26Oc7xZTZIo0nqbnd8E00JPfNERGRnpTZIpIEaqJFREpQaNF/ERFJjmpmtppoEZEidS7637lmaeei/4AaaRGRhKl2Zjf8xEIRkWIVs+i/iIgkQ7UzW020iEiR998rQgAACEhJREFUtEaxiEh6VDuz1USLiBRJaxSLiKRHtTNbTbSISJG0RrGISHpUO7M1sVBEpAidM7x3vL+LPmbscqdFq3OIiCRSLTJbTbSISAHdZ3jvct9zNEMNtIhIstQqsytqos3sIOBu4Ejgr8A57v5mt20+ASzIeOhDwHnuvsjMbgNOArZGz33B3VdWUpOISKW6ryu6/b2OnDO809ZEK7dFpN7EldmVjomeBSx196OBpdH9Ltz9MXcf4+5jgInAduDhjE1mdj6vIBaRuHUewWjfsgMnrCv65vb3s26b0lU5lNsiUjfizOxKh3NMASZEn/8cWA5clWf7s4CH3H17hfstm642JiL5ZFtXNJeUrsqh3BaRuhFnZld6JHqwu28AiD4eWmD784BfdnvsGjNbZWYLzGzfCuvJK9tfK7Pve5ZFK9qruVsRSZFij1SkeFUO5baI1I04M7tgE21mj5rZ6iy3KaXsyMyGAKOAJRkPzyaMtfsX4CDyHA0xs4vMrM3M2jZt2lTKrvfQ1cZEpJBcRyqam/rR0tyEAS3NTVw7bVRij4YmIbd7I7NBuS0i+cWZ2QWHc7j7KbmeM7O/m9kQd98Qhe3GPF/qHOB+d98zUKXzaAiw08x+BnwtTx0LgYUAra2tXqjubHS1MREpZOakkV1mdUM4gnH15GMT2zR3l4Tc7o3MBuW2iOQXZ2ZXOpxjMTA9+nw68ECebc+n2ynBKMAxMwOmAqsrrCcvXW1MRAqZOraFa6eNSs1R5zIot0WkbsSZ2ZVOLJwH3GNmFwKvAWcDmFkrMMPdvxzdPxI4HPhdt9ffYWaDAANWAjMqrCevXH+tpHRco4hUydSxLfXUNHen3BaRuhJXZlfURLv7P4CTszzeBnw54/5fgR7/OnefWMn+S9X5DdYsbxFpVMptEZHe0XBXLKzzI0wiInVHuS0iSdQQTbTWGBURSQ9ltoikQd030d2vn965xiigUBYRSRhltoikRaWrcySe1hgVEUkPZbaIpEXdN9FaY1REJD2U2SKSFnXfRGuNURGR9FBmi0ha1H0TPXPSSJr69enymNYYFRFJJmW2iKRF3U8s1BqjIiLpocwWkbSo+yYatMaoiEiaKLNFJA3qfjiHiIiIiEhvUxMtIiIiIlIiNdEiIiIiIiVSEy0iIiIiUiI10SIiIiIiJVITLSIiIiJSInP3uGsomZltAl4t8WWHAG9UoZxSJaUOSE4tqqOnpNSiOnqqtJYj3H1QbxWTBsrsXpOUWpJSBySnFtXRU1JqqVpmp7KJLoeZtbl7q+rYKym1qI6eklKL6ugpSbXUs6R8n5NSBySnlqTUAcmpRXX0lJRaqlmHhnOIiIiIiJRITbSIiIiISIkaqYleGHcBkaTUAcmpRXX0lJRaVEdPSaqlniXl+5yUOiA5tSSlDkhOLaqjp6TUUrU6GmZMtIiIiIhIb2mkI9EiIiIiIr2iIZtoM/uambmZHRLT/r9rZqvMbKWZPWxmQ2OqY76ZvRDVcr+ZNcdRR1TL2Wa2xsx2m1nNZ/Oa2almttbMXjazWbXef0Ydt5rZRjNbHVcNUR2Hm9ljZvZ89L5cHlMd+5nZn83smaiO78RRR0Y9fcxshZn9Js46Go0yu0stichtZfaeOpTZXetoqMxuuCbazA4HPgm8FmMZ8919tLuPAX4DfCumOh4BjnP30cCLwOyY6gBYDUwDfl/rHZtZH+BHwGnAMcD5ZnZMreuI3AacGtO+M3UAV7r7h4ETgUti+p7sBCa6+/HAGOBUMzsxhjo6XQ48H+P+G44yu4ek5LYyO7gNZXamhsrshmuigQXA14HYBoO7+1sZd/ePqxZ3f9jdO6K7fwKGxVFHVMvz7r42pt1/BHjZ3V9x9/eAu4ApcRTi7r8HNsex7251bHD3p6PP3yaEUEsMdbi7b4vu9otusfy8mNkw4HTg5jj238CU2V1rSURuK7MDZXaPOhoqsxuqiTazyUC7uz+TgFquMbPXgQuI96hGpy8BD8VdRExagNcz7q8jhvBJKjM7EhgLPBHT/vuY2UpgI/CIu8dSB3ADoZnbHdP+G44yu6BGzW1ldh7K7D2qntl9q/WF42JmjwKHZXlqDvBfwKfirsPdH3D3OcAcM5sNXAp8O446om3mEE4F3VGNGkqpJSaW5TEtWwOY2QeAe4Eruh2Nqxl33wWMicZ+3m9mx7l7TccfmtkZwEZ3f8rMJtRy3/VOmV16LdE2Vc9tZXb6KLODWmV23TXR7n5KtsfNbBQwAnjGzCCcAnvazD7i7n+rVR1Z3Ak8SJUCuVAdZjYdOAM42au83mEJ35NaWwccnnF/GLA+ploSw8z6EcL4Dne/L+563H2LmS0njD+s9SSe8cBkM/s0sB9woJn9r7t/rsZ11B1ldum11Cq3ldnposzuoiaZ3TDDOdz9WXc/1N2PdPcjCT+EJ1QjjAsxs6Mz7k4GXqh1DVEdpwJXAZPdfXscNSTEk8DRZjbCzPoD5wGLY64pVha6lluA5939v2OsY1Dn6gNm1gScQgw/L+4+292HRdlxHrBMDXR1KbNz1qLcVmb3oMzuqlaZ3TBNdMLMM7PVZraKcKoylqVogBuBA4BHoqWb/iemOjCzM81sHfAx4EEzW1KrfUeTdC4FlhAmY9zj7mtqtf9MZvZL4HFgpJmtM7ML46iD8Ff854GJ0f+NldFf9LU2BHgs+ll5kjC+TsvLSa0lJbMhIbmtzA6U2T00VGbrioUiIiIiIiXSkWgRERERkRKpiRYRERERKZGaaBERERGREqmJFhEREREpkZpoEREREZESqYkWERERESmRmmgRERERkRKpiRYRERERKdH/AxLAPez3VQ2aAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 864x576 with 4 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(-4, 4, 100)\n",
    "y = 1 - (x/3) ** 2\n",
    "x1 = x + np.random.randn(100) * 0.1\n",
    "y1 = y + np.random.randn(100) * 0.1\n",
    "_,axs = plt.subplots(2,2, figsize=(12,8))\n",
    "betas = [0.5,0.7,0.9,0.99]\n",
    "idx = x1.argsort()\n",
    "for beta,ax in zip(betas, axs.flatten()):\n",
    "    ax.scatter(x1,y1)\n",
    "    avg,res = 0,[]\n",
    "    for i in idx:\n",
    "        avg = beta * avg + (1-beta) * y1[i]\n",
    "        res.append(avg)#/(1-beta**(i+1)))\n",
    "    ax.plot(x1[idx],np.array(res), color='red');\n",
    "    ax.set_title(f'beta={beta}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see in these examples that a beta that's too high results in the overall changes in gradient getting ignored. In SGD with momentum, a value of `beta` that is often used is 0.9.\n",
    "\n",
    "`fit_one_cycle` by default starts with a beta of 0.95, gradually adjusts it to 0.85, and then gradually moves it back to 0.95 at the end of training. Let's see how our training goes with momentum added to plain SGD:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to add momentum to our optimizer, we'll first need to keep track of the moving average gradient, which we can do with another callback. When an optimizer callback returns a dict, it is used to update the state of the optimizer, and is passed back to the optimizer on the next step. So this callback will keep track of the gradient averages in a parameter called `grad_avg`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def average_grad(p, mom, grad_avg=None, **kwargs):\n",
    "    if grad_avg is None: grad_avg = torch.zeros_like(p.grad.data)\n",
    "    return {'grad_avg': grad_avg*mom + p.grad.data}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To use it, we just have to replace `p.grad.data` with `grad_avg` in our step function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def momentum_step(p, lr, grad_avg, **kwargs): p.data.add_(-lr, grad_avg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "opt_func = partial(Optimizer, cbs=[average_grad,momentum_step], mom=0.9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Learner` will automatically schedule `mom` and `lr`, so fit_one_cycle will even work with our custom Optimizer:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>accuracy</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>2.856000</td>\n",
       "      <td>2.493429</td>\n",
       "      <td>0.246115</td>\n",
       "      <td>00:10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>2.504205</td>\n",
       "      <td>2.463813</td>\n",
       "      <td>0.348280</td>\n",
       "      <td>00:10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>2.187387</td>\n",
       "      <td>1.755670</td>\n",
       "      <td>0.418853</td>\n",
       "      <td>00:10</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn = get_learner(opt_func=opt_func)\n",
    "learn.fit_one_cycle(3, 0.03)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAt0AAAD4CAYAAAAwyVpeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3yV9fn/8deVTfYkjEz2JpAQQETcolVRBAGto9pqtWqdraNaZx1UqbXan7TiqBNwoYI4EBVkJIywAyEJJKwkZJKdnM/vjxz8pjRAgJzcZ1zPx+M8PLnPfd95H4STK/d9fT4fMcaglFJKKaWUchwvqwMopZRSSinl7rToVkoppZRSysG06FZKKaWUUsrBtOhWSimllFLKwbToVkoppZRSysF8rA7QGaKjo01SUpLVMZRS6oStWbOmxBgTY3WOzqSf2UopV3Wsz2yPKLqTkpLIzMy0OoZSSp0wEdlldYbOpp/ZSilXdazPbG0vUUoppZRSysG06FZKKaWUUsrBtOhWSimllFLKwbToVkoppZRSysG06FZKKaWUUsrBHFp0i8hEEckWkRwRub+N1/1F5AP766tEJMm+PV1E1tsfWSJyeXvPqZRSSimllLNxWNEtIt7Ay8CFwCBghogMOmK3G4EyY0wfYBbwrH37JiDNGJMCTAReFRGfdp5TKaWUUkopp+LIebrTgRxjTC6AiLwPTAK2tNpnEvCo/fl84B8iIsaYmlb7BADmBM6pOlh9UzMrc0vJ3l9JTUMzgX7eBPn7EOzvQ3igHwmRgfQM74Kfj3YrKaWsMevr7UQG+TEsLoyU+HBExOpISikXUVHTyP7KOg5U1lFUVU9pdT01Dc3MSE8gNjSgw76PI4vunkBBq68LgdFH28cY0yQiFUAUUCIio4E5QCJwjf319pwTABG5CbgJICEh4dTfjQdqthneWbWLv32zg9LqhmPu6yXQPawL/buFMCI+nBEJEQyPDyMkwLeT0iqlPFVjs43ZP+RS29gMwIBuITx66WDG9IqyOJlSytkUVdWxMreUzXsr2LK3kq37qig5VN/mvuP7xrhM0d3WZQbT3n2MMauAwSIyEHhTRBa185zYj58NzAZIS0trcx91dBW1jdz5/jq+yy5mbK8ofnNGMqmJkQT7+1Db2Ex1fRPV9U2UHGpgd2kNuw9Ws6u0hs17K1myrQgAEUiJD+fcgbGcNyiWvl2D9eqTUqrD+Xp7seXxCyiuqmfJtiJeXprD9Nkruff8fvzurD76uaOUB2tqtrEi9yDfZxezLKeEbfurAPDz9qJvbDBn9o+hf2wI3cMDiA0NoGuIP5FBfnTx9cbHu2Pv4Duy6C4E4lt9HQfsPco+hSLiA4QBpa13MMZsFZFqYEg7z6lOUWVdI9e8toqt+yp54rIh/HJ0wn/90Aq2t5YA9IqB9OTI/zq+oraRrIJyMneVsTS7iJmLs5m5OJuEyEAuS+nB1LR44iMDO/U9KaXcm4jQNTSA6ekJXJrSg4c+3sRfv9pOVV0TD1w00Op4SqlOZIxh7e5yFqzfwxcb91FyqAE/Hy9GJUXwx4kDOL1PNAO6h+DbwUX18Tiy6M4A+opIMrAHmA5cdcQ+C4DrgBXAFGCJMcbYjymwt5QkAv2BfKC8HedUp6Cx2cZNb2WyZW8lr16TyjkDY0/4HGFdfDmjXwxn9Ivh7vP6caCyjm+3FrFo0z5e+i6Hl77LYVzvaKaNimfikG6d/pdeKeXeAv18eOHK4YQE+PDqD7l0Dwvg+nHJVsdSSjlYdX0TH64t5I3l+eSWVOPv48W5A2O5ZHgPJvSLoYuft6X5HFZ02wvm24DFgDcwxxizWUQeBzKNMQuA14D/iEgOLVe4p9sPPx24X0QaARtwqzGmBKCtczrqPXiiZxdtY2VuKS9cOfykCu62xIYGcNXoBK4ancCe8lrmZRYwL7OQ299bR1xEF246oxdTU+Mt/8eglHIfIsKfLxnMvoo6nvhiK8PiwxmZEGF1LKWUA+wtr+X15Xm8n1FAVV0Tw+PDmTllGBOHdHOqsWVijPu3O6elpZnMzEyrYzi9H7YXc+2c1Vw3NpHHJg1x6Pey2Qzfbivin0tzWLu7nKggP244PZkbxiVr8a1UKyKyxhiTZnWOztSRn9mVdY1c9OKPiMCXvz+DIH9H3uBVSnWmoso6Xlm6k3dX7abZGC4c0o0bTk+29BfsY31m66ePAqCqrpH7P9xA75igTul/9PISzhsUy7kDu5KRX8YrS3OYuTibt1bkc/d5/ZiSGo+3lw5+UkqdmtAAX2ZNS2Hq/1vB377ZzkO/0KUdlHJ1FTWNvLI0hzdX5NPYbJiaGsdtZ/chLsK5x4tp0a0AeGlJDvsq65j/29MI8O28K80iQnpyJOnJ6WTkl/KXhVv544cbmbMsn4d+MZAz+sV0WhallHsalRTJ9FHxzFmezxWpcQzoFmp1JKXUSbDZDHMzC3hucTZlNQ1cltKT35/Tl6ToIKujtYuOYFPklVTz+vI8poyMIzXRulsyo5Ii+eiW03jl6pHUNTVz7ZzV3PHeOoqr2p4/Uyml2uuPEwcQ7O/D0wu3WR1FKXUSsgrKufyV5dz/0UZ6xwTxxe3jmTUtxWUKbtCiWwEzF2/Dz9uL+yb2tzoKIsJFQ7vz1V1ncOe5ffly037OeX4p76/ejc3m/uMPlFKOERHkx61n9ub77cX8tLPE6jhKqXaqa2zmic+3cNkry9lbUcesacOZe/NYBvVwvTtWWnR7uM17K1i4cT83np5M15COW3XpVPn7eHPnuf1Y+PvxDOweyv0fbeSaOavYV1FrdTSllIu67rQkuocF8PxX2/GESQSUcnVrd5dx0Ys/8tqyPK5KT2DJPRO4fEScyy54pUW3h/vbNzsIDfDhxvG9rI7Spj5dg3n/pjE8PXko63aXM/FvP/LFhn1Wx1JKuaAAX29+O6E3a3aVsTqv9PgHKKUs0dBk45lF25jyz5+ob7Lx9o2jeeryoU41/d/J0KLbg+UWH+LrLQe4flwyYV2c9y+yiDAjPYGFd4wnOTqI3727lrvnrudQfZPV0ZRSLmbaqHiig/34x3c5VkdRSrWhoLSGqa+u4P99v5Mr0+L58s7xnN432upYHUKLbg/2+vJ8/Ly9uGZMotVR2iUpOoh5vx3L78/pyyfr9nDpP5ax40CV1bGUUi4kwNebX41L5scdJeQU6eeHUs7kq837+cXffyS36BCvXD2SZ64Y5vJXt1vTottDVdQ0Mn9NIZem9CAmxN/qOO3m6+3FXef1451fj6GytpFJLy/ns6y9VsdSSrmQaaPi8fP24u2Vu62OopQCmpptPPn5Fm76zxoSo4L4/I7TuWhod6tjdTgtuj3Uexm7qW1s5oZxyVZHOSlje0fx+e0tgyxvf28dj3+2haZmm9WxlFIuIDrYnwuHduPDNYXUNGibmlJWqqhp5FdvZPDvZXlcOzaR+beMJTHKdaYBPBFadHugxmYbb/6Uz9heUS455c5h3cICeO83Y7j+tCTmLM/jV29kUFnXaHUspZQLuGZMIlX1TXy6Xu+UKWWVnKIqJr28jJW5B3nuimE8PmkI/j6dt0BfZ9Oi2wN9veUA+yrquPF017zK3ZqfjxePXjqYZyYPZcXOg1zxyk8UlNZYHUsptyIiE0UkW0RyROT+Nl5PFJFvRWSDiCwVkbgjXg8VkT0i8o/OS31sqYkRDOgWwn9W7NLpA5WywNLsIi5/+ScO1Tfx3m/GcOWoeKsjOZwW3R5oXmYB3cMCOGtAV6ujdJjp6Qm8dUM6ByrruPyV5azdXWZ1JKXcgoh4Ay8DFwKDgBkiMuiI3f4KvGWMGQY8Djx9xOtPAN87OuuJEBF+OSaRLfsqWVdQbnUcpTzK3IwCbnwzk/jIQD697XTSkiKtjtQptOj2MEWVdXy/vZjJI3vi7eWak8sfzWl9ovno1nEE+fswY/ZKvtlywOpISrmDdCDHGJNrjGkA3gcmHbHPIOBb+/PvWr8uIqlALPBVJ2Q9IZeN6EmgnzdzMwqsjqKURzDG8OI3O/jDhxs4rXcUc387lp7hXayO1Wm06PYwn6zfg83A5JFxx9/ZBfXpGszHt45jQLcQbn57DR+uKbQ6klKurifQuiottG9rLQu4wv78ciBERKJExAt4HrjveN9ERG4SkUwRySwuLu6A2McX7O/DBYO7sWjTfuqbmjvleyrlqZqabTz48UZmfbOdySN7Muf6UQT7+1gdq1Np0e1BjDHMX1PIyIRwescEWx3HYSKD/HjnN2MY0yuSe+Zl8e8fc62OpJQra+uW2JFN0PcCE0RkHTAB2AM0AbcCC40xx72UbIyZbYxJM8akxcTEnGrmdrs0pQcVtY18n905hb5SnqihycZt767jvdUF3HZWH56fOhxfb88rQT3vHXuwjXsq2H7gEFNS3X+wQrC/D3OuH8VFQ7vx5Bdbmbl4mw6WUurkFAKtPzTigP+a8sMYs9cYM9kYMwJ4yL6tAhgL3CYi+bT0fV8rIs90Sup2Or1PNJFBfnyq8/0r5RB1jc3c9J9Mvty8n0cuHsS9F/RHxL3aW9vLs67re7gP1xTi5+PFL4a534TzbfH38ealGSMJ67KJl7/bSWOz4YELB3jsP3alTlIG0FdEkmm5gj0duKr1DiISDZQaY2zAA8AcAGPM1a32uR5IM8b8z+wnVvL19uIXQ7szN7OAQ/VNHne7WylHqq5v4tdvZrIy7yBPTx7KjPQEqyNZSq90e4jGZhsLsvZy/qBYwrq4z5Kqx+PtJfzl8iFcNzaR2T/k8sTnW/WKt1InwBjTBNwGLAa2AnONMZtF5HERudS+25lAtohsp2XQ5FOWhD1Jl43oQX2Tja8277c6ilJuo7KukWteW8Xq/FJmXZni8QU36JVuj7Eqt5SymkYuHd7D6iidTkR49NLBeHkJc5bn0Wyz8eilg/WKt1LtZIxZCCw8YtsjrZ7PB+Yf5xxvAG84IN4pG5kQQVxEFz5Zv9dtB5kr1Zmq65v41esZbCis4OWrRjBxiGfcYT8eLbo9xBcb9xHk580Z/TpvgJIzEREeuXgQPl7Cv37MA9DCWykFtHw+XDK8B7N/yKW0uoHIID+rIynlsmobmrnxzQzWF5TzjxlacLem7SUeoKm55bbp2QNjCfB13+VVj0dEePCigfxmfDJvrtjFzMXZVkdSSjmJC4d0o9lmWLKtyOooSrmsw4MmV+WV8sKVw7lwqBbcrWnR7QFW55dysLqBi4Z0szqK5Q4X3leNTuCVpTt5+bscqyMppZzA0J5hdAsN4Ost2tet1MloaLJx6ztr+XFHCc9dMYxJKUdO56+0vcQDLNq4ny6+3pzZ332WfT8VIsKTk4ZQU9/EzMXZhAT4cO3YJKtjKaUsJCKcNyiW+WsKqWts9ui7gkqdqMZmG7e/t5Yl24r4y+VDmZrm/lMTnwyHXukWkYkiki0iOSLyP9NEiYi/iHxgf32ViCTZt58nImtEZKP9v2e3Omap/Zzr7Q+tJI+h2Wb4cvN+zhoQQxc//SFymJeXMHPqcM4bFMsjn25mvq5cqZTHO39wLLWNzSzbUWJ1FKVchjGG+z/cyOLNB/jzJYO4arTOUnI0Diu6RcQbeBm4EBgEzBCRQUfsdiNQZozpA8wCnrVvLwEuMcYMBa4D/nPEcVcbY1LsD23AO4bM/FKKq+q5SPuq/oevtxcvzRjB6X2i+cP8LBZt3Gd1JKWUhUYnRxHi78NX2mKiVLvNXJzNh2sLuevcfvxqXLLVcZyaI690pwM5xphcY0wD8D4w6Yh9JgFv2p/PB84RETHGrDPGHF4ebDMQICL+Dszqtr7acgA/Hy/O0taSNgX4ejP72lRGJETw+w/Wk5FfanUkpZRF/Hy8OGtAV77dWkSzTefzV+p43liexytLd3LV6ATuOKeP1XGcniOL7p5AQauvC+3b2tzHvgBDBRB1xD5XAOuMMfWttr1uby15WI4y55uI3CQimSKSWVxcfCrvw6Ut2VbE2F5RBOkqa0cV6OfDv69NIy6iC79+M5OcokNWR1JKWeT8wbEcrG5g7e4yq6Mo5dQ+37CXxz7fwvmDYnli0hCdgrcdHFl0t/Wnf+Slg2PuIyKDaWk5ubnV61fb207G2x/XtPXNjTGzjTFpxpi0mBjPnJs6t/gQeSXVnDNQr3IfT0SQH2/+Kh1fby+um7Oaoqo6qyMppSwwoV8Mft5efL3lgNVRlHJaP+0s4e4PshiVGMnfZ4zA20sL7vZwZNFdCLQevhoH7D3aPiLiA4QBpfav44CPgWuNMTsPH2CM2WP/bxXwLi1tLKoNh+eb1daS9omPDGTO9WmU1TRwwxsZVNc3WR1JKdXJQgJ8GZUcwffZnnuHVKlj2bqvkpveWkNSdCD/ujZNZ/o5AY4sujOAviKSLCJ+wHRgwRH7LKBloCTAFGCJMcaISDjwBfCAMWb54Z1FxEdEou3PfYGLgU0OfA8u7dutRfSPDSE+MtDqKC5jWFw4L181kq37qrj1nbU0NtusjqSU6mRn9utK9oEq9lXUWh1FKadSVFnHjW9kEOzvw5s3pBMW6Gt1JJfisKLb3qN9G7AY2ArMNcZsFpHHReRS+26vAVEikgPcDRyeVvA2oA/w8BFTA/oDi0VkA7Ae2AP8y1HvwZVV1DaSkV/K2dpacsLOGtCVpy4bwvfbi3n4k00YowOqlPIkE/q3tCTq1W6l/k9dYzO/eSuT8tpGXrs+je5hXayO5HIcOrrOGLMQWHjEtkdaPa8DprZx3JPAk0c5bWpHZnRXP+4opslmOGeAFt0nY3p6AoVltfzjuxz6xoZw4+k6DZJSnqJv12C6hwXw/fZipqfrnMNK2WyGe+ZmsWFPBbOvSWNwjzCrI7kkXQbeTS3ZWkR4oC8jEiKsjuKy7j6vHxcMjuWpL7bwXbZOB6+UpxARJvSLYdmOEm0xUwqY9c12vti4jwcvHMh5g2KtjuOytOh2Q802w3fZRZzVv6uOKD4FXl7CrGkpDOgWyh3vriOnqMrqSEqpTjKhXwxV9U2sLyi3OopSlvp4XSEvLclh+qh4fj1e7/qeCi263dCmPRWU1TQyoZ9nTpXYkQL9fPjXdWn4+3pz45uZlFU3WB1JKdUJxvWNxttLtK9bebQ1u0r54/yNjO0VxeM6F/cp06LbDf24o+WHxOl9oy1O4h56hnfh1WtS2Vdexy3vrNHbzUp5gNAAX1ITIli6XVvLlGfaX1HHzf9ZS4/wAP75y5H4+WjJeKr0T9AN/bCjhME9QokO9rc6ittITYzgmSuGsjK3lEcXbLY6jlKqE0zoH8OmPZWUHKo//s5KuZG6xmZufnsNtQ1N/OvaNMID/ayO5Ba06HYzh+qbWLurjPF9tbWko00eGcfNE3rxzqrdfJCx2+o4SikHG9en5W7hTzsPWpxEqc5jjOGRTzeRVVDO81em0Dc2xOpIbkOLbjezcudBmmyGM/ppa4kj/OGCAZzeJ5qHP91Mlg6wUsqtDe0ZRkiADyt2llgdRalO8/bKXczNLOT2s/swcUg3q+O4FS263cyPO4rp4utNaqJOFegI3l7C32eMICbYn1veXsNBve2slNvy9hLG9IpieY5e6VaeYXVeKY99toWzB3TlrnP7WR3H7WjR7WZ+3FHCmF6R+Pt4Wx3FbUUG+fHqNakcrG7g9vfW0aQDK5VyW+N6R7G7tIaC0hqroyjlUPsqarn1nTXERwYya1oKXjrlcIfTotuNFJTWkFtSrf3cnWBIzzCeunwoP+08yMzF2VbHUUo5yGk/93Vri4lyXw1NNm59Zy21Dc3MviaVsC6+VkdyS1p0u5FlOS0/FLSfu3NMSY3jmjGJvPpDLl9s2Gd1HKWUA/TtGkxMiL8OplRu7ZlF21i3u5znpgzXgZMOpEW3G1mWU0K30AB6xwRbHcVjPHzxIEYmhHPf/Cx2Fh+yOo5SqoOJCKf1juKnnQcxxlgdR6kOt2jjPuYsz+P605L4xbDuVsdxa1p0uwljDKtyDzK2d5SuGNWJ/Hy8ePnqkQT4evO7d9ZS19hsdSSlVAcb1zua4qp6dhTpL9bKveSXVPOH+RsYHh/OgxcNtDqO29Oi203sKDpEyaEGxvaKsjqKx+ke1oUXrhzOtv1VPPaZLpyjlLsZ27vlc3V5jvZ1K/dR19jMLe+sxctLePmqEbriZCfQP2E3scLeb3j4h4PqXGf278qtZ/bmvdUFfLJuj9VxlFIdKD4ykITIwJ8/Z5VyB48u2MzWfZXMmjacuIhAq+N4BC263cSKnQfpGd6F+Ej9h2OVu8/rR3pSJA9+vJEcvQ2tlFsZnRxJRn4pNpv2dSvX9+GaQt7PKODWM3tz9oBYq+N4DC263YDNZliZd1CvclvMx9uLv88YQRd7f3dtg/Z3K+Uu0pMjKatpJEcHTCsXl72/ioc+2cjo5EjuPk8XwOlMWnS7gW37qyivaeQ0Lbot1y0sgFnTUtheVMWjC7S/W7kHEZkoItkikiMi97fxeqKIfCsiG0RkqYjE2beniMgKEdlsf21a56fvGKOTWz5fV+WVWpxEqZNX09DEre+sIdjfl5dmjMDHW8vAzqR/2m7g8KINeqXbOZzRL4bfndmHDzIL+GhtodVxlDolIuINvAxcCAwCZojIoCN2+yvwljFmGPA48LR9ew1wrTFmMDAR+JuIhHdO8o4VH9mFbqEBrNaiW7mwxxZsIbekmr9PT6FraIDVcTyOFt1uYGXuQZKiAuke1sXqKMruznP7Mjo5koc+3qT93crVpQM5xphcY0wD8D4w6Yh9BgHf2p9/d/h1Y8x2Y8wO+/O9QBHgkkvmigjpyZGsytX5upVr+ixrLx9ktvRxH15pVXUuLbpdXLPNsCqvVK9yO5mf+7v9vLnjvXXUN2l/t3JZPYGCVl8X2re1lgVcYX9+ORAiIv/1oSQi6YAfsNNBOR1udK9Iiqrq2XWwxuooSp2QgtIaHvxoIyMSwrnzXO3jtooW3S5u894KquqaGKPzczud2NAAnrtiGFv2VfLcl9lWx1HqZLW12taRl3rvBSaIyDpgArAHaPr5BCLdgf8AvzLG2Nr8JiI3iUimiGQWFxd3TPIONjo5EkBbTJRLaWy2ccf76wD4+/QR+Goft2X0T97F/Tw/txbdTuncQbFcNzaR15blsTS7yOo4Sp2MQiC+1ddxwN7WOxhj9hpjJhtjRgAP2bdVAIhIKPAF8CdjzMqjfRNjzGxjTJoxJi0mxjk7UHrHBBMZ5KeDKZVLefGbHazbXc5fJg/VaYUtpkW3i1uRe5DeMUE6IMKJPXDRQPrHhnDvvCyKq+qtjqPUicoA+opIsoj4AdOBBa13EJFoETn88+QBYI59ux/wMS2DLOd1YmaHEBHSkyJZna+L5CjX8FNOCS8vzWFaWjyXDO9hdRyP59Ciux3TTPmLyAf211eJSJJ9+3kiskZENtr/e3arY1Lt23NE5O8i0tatT4/Q2GwjQ/u5nV6Arzd/nzGCqrom7p2XpYtrKJdijGkCbgMWA1uBucaYzSLyuIhcat/tTCBbRLYDscBT9u1XAmcA14vIevsjpXPfQcdKT46koLSWveW1VkdR6phKqxu484P1JEcH8edLj5xwSFnBYUV3O6eZuhEoM8b0AWYBz9q3lwCXGGOGAtfR0gt42D+Bm4C+9sdER70HZ7dxTwXVDc2c1ltHITu7/t1C+NMvBvL99mJe/ynf6jhKnRBjzEJjTD9jTG9jzFP2bY8YYxbYn883xvS17/NrY0y9ffvbxhhfY0xKq8d6K9/LqUrXvm7lAowx3Dcvi/KaRl6aMYJAPx+rIykce6W7PdNMTQLetD+fD5wjImKMWWefXgpgMxBgvyreHQg1xqwwLXM2vQVc5sD34NRW5bZ86B/+IaCc2y/HJHLuwFieXbSNzXsrrI6jlDoJA7uHEhLgo33dyqm9vXIX324r4oGLBjC4R5jVcZSdI4vu9kwz9fM+9luYFcCRvRJXAOvsV0562s9zrHMCrjES/lRl5pfSKyaI6GB/q6OodhARnpsyjPBAX+54bx01DU3HP0gp5VS8vYRRSZGsztO+buWccooO8eQXW5nQL4brT0uyOo5qxZFFd3ummTrmPiIymJaWk5tP4JwtG11gJPypsNkMmbvKSE/Sq9yuJDLIj1nTUsgtqebJL7ZaHUcpdRLSkyPZWVxNySEdGK2cS0OTjTs/WEegnzczpwzDg4e9OSVHFt3HnWaq9T4i4gOEAaX2r+NoGfV+rTFmZ6v9445zTo+wo+gQFbWNpGnR7XLG9YnmN+N78e6q3SzZdsDqOEqpE6R93cpZvfjtdjbtqeTpycN0VjMn5Mii+7jTTNm/vs7+fAqwxBhjRCSclnldHzDGLD+8szFmH1AlImPss5ZcC3zqwPfgtFbn2/u5teh2Sfec348B3UL4w/yNHNSrZUq5lKE9wwjw9SIzv8zqKEr9LCO/lH8u3cmVaXFMHNLN6jiqDQ4ruts5zdRrQJSI5AB3A4enFbwN6AM83Gqaqa72124B/g3k0LKc8CJHvQdnlplfStcQf+Iju1gdRZ0Efx9vZk1LobK2kQc+2kjLuGCllCvw9fZiWFw4a3bplW7lHKrqGrnrg/XERQTyyCWDrY6jjsKhc8gYYxYCC4/Y9kir53XA1DaOexJ48ijnzASGdGxS15ORV8qo5Ejt13JhA7uHcu8F/fjLwm3MW1PIlWnxxz9IKeUU0hIjmP1DLrUNzXTx87Y6jvJwj322hb3ltcz77WkE++v0gM5KV6R0QXvKa9lbUceoxAiro6hTdOPpvRidHMljCzZTUFpjdRylVDulJkbQZDNkFZZbHUV5uEUb9zF/TSG3ndWHVK0LnJoW3S4owz54Z5TOz+3yvL2E568cjpcId89dT7OuVqmUSxiZ0FLcrNmlfd3KOgcq63jg440Mjwvj9nP6Wh1HHYcW3S4oI7+UEH8fBnQLtTqK6gBxEYE8NmkwGfllzP4h1+o4Sql2iAjyo3dMkBbdyjI2m+HeeVnUNTbzwrQUfL21pHN2+n/IBWXklzIyMQJvL+3ndheXj+jJRUO78cLX2Wzao6tVKuUK0hIjWbOrDJveoVIWeGtFPj/uKOFPvxhE75hgq+OodtCi2/LRC9gAACAASURBVMWU1zSw/cAhRiVp35Y7ERGeumwoEYF+3PXBeuoam62OpJQ6jtTECCpqG8ktOWR1FOVhdhYf4ulF2zirfwxXj06wOo5qJy26XczheWFH6fzcbiciyI/npgxjR9EhZi7OtjqOUuo4Uu0XP3S+btWZmppt3DM3iwBfb569QleddCVadLuYjF2l+HoLw+PDrY6iHODM/l25Zkwiry3L46edJVbHUUodQ6/oICICfbWvW3Wq2T/msr6gnCcuG6KrTroYLbpdTEZeKcPiwgnw1Xlh3dWDFw0kKSqQ++ZtoKqu0eo4SqmjEBFSEyO06FadJnt/FX/7egcXDe3GJcO6Wx1HnSAtul1IXWMzG/dUkKb93G6ti583z185nH0Vtfxl4Var4yiljmFkYgS5JdWUVjdYHUW5ucZmG3fPXU9IgA9PTBqibSUuSItuF7K+oJzGZkO69nO7vdTESH4zvhfvrS5gaXaR1XGUUkeRltjyeaxXu5WjvfxdDpv3VvLU5UOJCva3Oo46CVp0u5DDi+LoilOe4a7z+tG3azB//HADFTXaZqKUMxoWF4avt2jRrRxq054K/rEkh8tSejBxSDer46iTpEW3C8nYVUb/2BDCA/2sjqI6QYBvS5tJyaEGHvtss9VxlFJtCPD1ZnCPMNbsKrU6inJT9U3N3D13PVHBfjx26RCr46hToEW3i2i2GdbuKtN+bg8zLC6c353Zm4/W7WHx5v1Wx1FKtSE1MYKswgoammxWR1Fu6G/f7GD7gUM8c8UwwgJ9rY6jToEW3S5i675KDtU3kZ6s/dye5raz+zKoeygPfbxRB2sp5YTSEiNoaLKxaa+uJqs61trdZbz6/U6mpcVzVv+uVsdRp0iLbheRkd9y6zJNB1F6HD8fL16YNpyK2kYe/mST1XGUUkc4PM5mrfZ1qw5U29DMvXOz6B7WhT9dPNDqOKoDaNHtIjLzy+gZ3oWe4V2sjqIsMKBbKHee248vNu7js6y9VsdRSrXSNTSA+MguujKl6lAzF2eTW1LNc1OGERKgbSXuQItuF2CMYXV+qfZze7ibz+jF8PhwHv50E0VVdVbHUUq1kpoQwZrdZRhjrI6i3MCq3IO8/lMe145NZFyfaKvjqA6iRbcL2F1aQ3FVPaO0tcSj+Xh78fzU4dQ2NPPgRxv1h7tSTiQ1KZLiqnoKSmutjqJcXHV9E/fOzyIhMpD7LxxgdRzVgbTodgGr7fNz6yBK1adrMPdd0J9vthbx4do9VsdRLkhEvEXkUhG5Q0TuPvywOperS01ouRO5ZrdOHahOzV8WbqWwrJa/Th1OoJ+P1XFUB9Ki2wVk5pcR1sWXPjHBVkdRTuCGccmkJ0Xy2ILN7C3Xq2rqhH0GXA9EASGtHuoU9O8WQrC/j/Z1q1Py445i3lm1m1+fnqx3t92Q/grlAjLySxmVFIGXl1gdRTkBLy9h5tRhTPzbj/zxww28dUM6Ivp3Q7VbnDFmmNUh3I23lzAiIZy1u8utjqJcVGVdI3+Yv4HeMUHcc35/q+MoB9Ar3U6uuKqe3JJqnSpQ/ZfEqCAe/MVAftxRwrurd1sdR7mWRSJyvtUh3NHIhAiy91dSVddodRTlgp74bAsHKut4/soUAny9rY6jHECLbid3eGlhvc2kjvTL0Qmc3ieap77Yyu6DNVbHUa5jJfCxiNSKSKWIVIlIpdWh3EFqYgQ2A+sL9Gq3OjHfbj3AvDWF3HJmb1Liw62OoxxEi24ntzqvDH8fL4b2DLM6inIyIsKzU4bhLcK987Ow2XQ2E9UuzwNjgUBjTKgxJsQYE2p1KHeQkhCOCKzRRXLUCSirbuD+jzYyoFsId5zT1+o4yoEcWnSLyEQRyRaRHBG5v43X/UXkA/vrq0Qkyb49SkS+E5FDIvKPI45Zaj/nevvDrddFzdxVSkp8OH4++vuR+l89w7vw8CWDWJ1Xyhs/5VsdR7mGHcAmcwJzTrbjszxRRL4VkQ32z+i4Vq9dJyI77I/rOug9OKXQAF/6x4Zo0a1OyJ8XbKasuoHnrxyOv4+2lbgzh1VyIuINvAxcCAwCZojIoCN2uxEoM8b0AWYBz9q31wEPA/ce5fRXG2NS7I+ijk/vHKrrm9i8t1JbS9QxTU2N45wBXXlu8TZyiw9ZHUc5v33AUhF5oD1TBrbzs/yvwFv2AZqPA0/bj40E/gyMBtKBP4uIW6/ylZoYwfrd5TTrnSfVDos27mNB1l7uOKcvg3voHW13d9yiW0S8RGTTSZw7HcgxxuQaYxqA94FJR+wzCXjT/nw+cI6IiDGm2hizjJbi22Ots39wj9L5udUxiAhPTx6Kv48398zL0h/26njygG8BP9o3ZWB7PssH2c8J8F2r1y8AvjbGlBpjyoCvgYkd8i6cVGpiBFX1TewoqrI6inJyJYfqeeiTTQztGcYtZ/a2Oo7qBMctuo0xNiBLRBJO8Nw9gYJWXxfat7W5jzGmCaigZe7Y43nd3lrysBxlrjQRuUlEMkUks7i4+ASjO4fV+aV4CYxM0EEV6ti6hgbw+KTBrNtdzuwfcq2Oo5yYMeYxY8xjwAvA862+Ppr2fJZnAVfYn18OhIhIVDuPBdzjMxtaim7Qvm51bMYYHvp4I4fqmnj+yuH4emsLqSdo7//l7sBme8/egsOP4xzTVjF85CW49uxzpKuNMUOB8fbHNW3tZIyZbYxJM8akxcTEHOeUzikzv5SB3UMJCfC1OopyAZcO78GFQ7ox6+vtZO/Xq2yqbSIyRETWAZto+VxfIyKDj3VIG9uO/Jy+F5hgP+8EYA/Q1M5jWza6wWc2QEJkINHBflp0q2P6dP1eFm8+wD3n96NfrK5N5SnaW3Q/BlxMS6/e860ex1IIxLf6Og7Ye7R9RMQHCAOOuYauMWaP/b9VwLu03Pp0O43NNtbtLtd+btVuIsKTlw0hJMCHe+atp7HZZnUk5ZxmA3cbYxKNMYnAPcC/jrH/cT/LjTF7jTGTjTEjgIfs2yrac6y7ERFGJkSwVotudRQHKut45NNNpCZG8OvxvayOozpRu4puY8z3bT2Oc1gG0FdEkkXED5gOHHl1fAFweDT7FGDJsUbUi4iPiETbn/vS8ovAyfSbO71NeyqobWwmLcmtxxypDhYV7M9Tlw9h055KXvlup9VxlHMKMsZ8d/gLY8xSIOgY+x/3s1xEokXk8M+TB4A59ueLgfNFJMI+gPJ8+za3lpoYQf7BGoqr6q2OopyMMYb7P9xAQ7ONv04djreuNO1Rjll0H140oY3HcRdTsPdo30bLB+xWYK4xZrOIPC4il9p3ew2IEpEc4G7g56moRCSflp7D60Wk0D5a3h9YLCIbgPW03MI81hUal5WZ33KVJF2vdKsTNHFIdyal9OClJTvYtKfC6jjK+eTax8Mk2R9/omVwZZva+Vl+JpAtItuBWOAp+7GlwBO0FO4ZwOP2bW7tcF/32t16tVv9t7mZBXyXXcwfJw4gOfpYv+sqd+RzrBeNMafUaGSMWQgsPGLbI62e1wFTj3Js0lFOm3oqmVzF6vxSEqMC6RoaYHUU5YIeu3QwK3Ye5J65WSy4fZzO/apau4GWlsEPaem5/gG4/lgHtOOzfD4tM1C1dewc/u/Kt0cY0jMMP28v1u4q44LB3ayOo5xEYVkNT3y+lTG9IrlubJLVcZQFdLisE7LZDJn5pdrPrU5aeKAfz1wxlOwDVbz4zQ6r4yjn0puWPmsvwBc4h5bCW3WQAF9vhvQM1cGU6mc2m+EP8zdgjGHmlOF4aVuJR9Ki2wntLD5EWU2jtpaoU3L2gFiuTIvj/32/k3V6m1v9n3doufI8mZZxMRcDl1iayA2lJkawYU8F9U3NVkdRTuDtVbv4aedB/nTxIOIjA62OoyyiRbcTWp3f0vKoi+KoU/WniwfRLTSAe+ZlUdeoP/wVAMXGmM+MMXnGmF2HH1aHcjepiRE0NNnYvPeYw5+UB8gvqebphduY0C+G6aPij3+AcltadDuhjLxSooP9SYrS34bVqQkN8OW5KcPJLa5m5uJsq+Mo5/BnEfm3iMwQkcmHH1aHcjcjE+yDKbXFxKM12wz3zc/Cx1t45oqhHGU9P+UhtOh2Qhn5ZaQnR+g/TtUhTu8bzS/HJDBneR6r89x+4gh1fL8CUmhZjv0S++NiSxO5oa6hAcRHdtG+bg83Z1keGfllPHbpYLqHdbE6jrKYFt1OZk95LXvKa3UQpepQD1w4kPiIQO6dl0V1fZPVcZS1httXfrzOGPMr++MGq0O5o9SECDJ3lXGM5SeUG8spqmLmV9mcNyiWy0f0tDqOcgJadDuZDPuVSC26VUcK8vfhr1OHU1BWwzOLtlkdR1lrpX3dA+VgqYkRFFfVU1hWa3UU1cmamm3cMzeLID9v/nK5tpWoFlp0O5nV+aWE+PswsHuo1VGUm0lPjuSGccn8Z+Uulu0osTqOss7pwHoRyRaRDSKy0b7gmOpgI+2L5GiLief5f9/vJKuwgicvG0pMiL/VcZST0KLbyWTklTIyMUKXhlUOcd8F/ekVE8Qf5mdRWddodRxljYlAX1qWZD/cz61TBjrAgG6hBPl5a9HtYbbsreTFb3dw8bDu/GJYd6vjKCeiRbcTKatuYEfRIdJ1qkDlIAG+3jw/dTj7K+t48vMtVsdRFmg9TaBOGehY3l7CiIQILbo9SEOTjXvmZRHWxY8nJg2xOo5yMlp0O5GMfO3nVo43IiGC307ozdzMQpZsO2B1HKXc2sjECLbtr+SQDmD2CC8t2cHWfZU8M3koEUF+VsdRTkaLbieSkV+Kn7cXw+LCrI6i3Nzvz+3LgG4h3P/hRsprGqyOo5TbSk2MwGYgq6Dc6ijKwbIKynll6U6uGBnHuYNirY6jnJAW3U4kI7+M4fFhBPh6Wx1FuTl/H2/+OnU4pdUNPLpgs9VxlHJbKfHhiOhgSndX19jMPfOy6BrizyOX6ORAqm1adDuJmoYmNu2pIE1bS1QnGdIzjNvO7sMn6/fy5aZ9VsdRyi2FdfGlX9cQLbrd3Atfbyen6BDPXjGMsC6+VsdRTkqLbiexfnc5TTZDuhbdqhP97qw+DOkZykMfb+LgoXqr4yjllkYmRrB2dxk2my6S445W55Xyrx9zuWp0Amf0i7E6jnJiWnQ7idX5pYj837yuSnUGX28vnp+aQlVdE3/6ZJOunKeUA6QmRlBV18SOokNWR1EdrKqukbvnrichMpCHLhpodRzl5LTodhIZ+aUM6Baqt6VUp+vfLYS7zuvHok37+XT9XqvjKOV2UnWRHLf1+Gdb2FteywtXphDk72N1HOXktOh2Ao3NNtbuKic9Sa9yK2vcdEYvUhMjePjTTewp1yWrlepISVGBRAX5adHtZr7ctI95awr53Vl9fv7FSqlj0aLbCWzeW0ltYzOjdFEcZRFvL2HWlSnYbIZ75q7X3lOlOpCI/NzXrdxDUVUdD3y0kaE9w7jjnL5Wx1EuQotuJ7Ay9yCArkSpLJUQFcifLx3MytxS/r0s1+o4SrmV1MQI8kqqdcCyGzDG8Mf5G6hpaGbWtBR8vbWUUu2jf1OcwMrcg/SOCaJrSIDVUZSHm5oaxwWDY5m5OJsteyutjqOU2zjcfrB2ty6S4+reXb2b77KLefCigfTpGmx1HOVCtOi2WGOzjYy8Usb2jrI6ilKICE9PHkZ4oB93frCOusZmqyMp5RaG9gzD11u0r9vF5RYf4snPtzK+bzTXjEm0Oo5yMVp0W2zjngqqG5oZ2yva6ihKARAZ5MfMKcPYfuAQMxdnWx1HKbcQ4OvN4B5hrNWi22U1Ndu4a24Wfj5ezJwyHC8vsTqScjFadFvscD/36F7az62cx5n9u3Lt2EReW5bHsh0lVsdRyi2kJkaQVVhOQ5PN6ijqJLz83U6yCsp56vIhdAvTdlB14hxadIvIRBHJFpEcEbm/jdf9ReQD++urRCTJvj1KRL4TkUMi8o8jjkkVkY32Y/4uIi79q+aKnQfpFxtMdLC/1VGU+i8PXDiQ3jFB3Dsvi/KaBqvjKOXyUhMjqG+ysXlvhdVR1AnKKijn70t2cFlKDy4e1sPqOMpFOazoFhFv4GXgQmAQMENEBh2x241AmTGmDzALeNa+vQ54GLi3jVP/E7gJ6Gt/TOz49J2joclGZn4ZY3tpP7dyPl38vHlx+ghKDtXzkK5WqdQpS7MPpszM1xYTV1Jd38SdH6wnNsSfxyYNsTqOcmGOvNKdDuQYY3KNMQ3A+8CkI/aZBLxpfz4fOEdExBhTbYxZRkvx/TMR6Q6EGmNWmJYK4C3gMge+B4faUFhObWOzDqJUTmtIzzDuOq8fX2zYxyfr91gdRymX1jU0gMSoQFbnl1odRZ2ARxdsJv9gNbOmpeiq0eqUOLLo7gkUtPq60L6tzX2MMU1ABXCsCrSn/TzHOicAInKTiGSKSGZxcfEJRu8cK3MPIgKjk7XoVs7rtxN6k5YYwSOfbKagtMbqOEq5tNHJkWTkl+oCVC7is6y9zFtTyG1n9WG03pVWp8iRRXdbvdZHfsq0Z5+T2t8YM9sYk2aMSYuJiTnGKa2zIvcgA7qFEhHkZ3UUpY7K20uYNS0FBG5/bx2NzToITKmTNTo5ivKaRrYXVVkdRR1HQWkND368kZEJ4fxeV51UHcCRRXchEN/q6zhg79H2EREfIAw41n23Qvt5jnVOl1Df1ExmfhljdNYS5QLiIwN5evJQ1heUM+vr7VbHUcplHV55eFWutpg4s6ZmG3d+sB4MvDh9BD666qTqAI78W5QB9BWRZBHxA6YDC47YZwFwnf35FGCJOcZoLWPMPqBKRMbYZy25Fvi046M7XlZBBfVNNh1EqVzGxcN6MH1UPP/8fifLc3QaQU/TjtmoEuyzTq0TkQ0icpF9u6+IvGmfdWqriDzQ+emdR3xkID3Du7A6T4tuZ/bSkhzW7CrjycuHEB8ZaHUc5SYcVnTbe7RvAxYDW4G5xpjNIvK4iFxq3+01IEpEcoC7gZ8/yEUkH3gBuF5EClvNfHIL8G8gB9gJLHLUe3CkFTu1n1u5nkcuGUSv6CDu+mA9Bw/VWx1HdZJ2zkb1J1o+50fQcpHlFfv2qYC/MWYokArcfHh6WE+VnhzJqryDOiOQk1qdV8pLS3YweWRPJqW0OWxMqZPi0PslxpiFxph+xpjexpin7NseMcYssD+vM8ZMNcb0McakG2NyWx2bZIyJNMYEG2PijDFb7NszjTFD7Oe87VhXxp3ZitwSBvcIJSxQR0Ir1xHo58NLM0ZSXtvIffM3aNHgOdozG5UBQu3Pw/i/1j8DBNlbCLsADUCl4yM7r9HJkZQcaiC3pNrqKOoIFTWN3Pn+OuIjA3lcpwdUHUyblCxQ19jM2t3ljNGr3MoFDeoRykMXDWTJtiJeX55vdRzVOdozG9WjwC9FpBBYCNxu3z4fqAb2AbuBvxpjPLq3Qvu6nZMxhgc/3khRVT0vTh9BsL+P1ZGUm9Gi2wKZ+WU0NNk4rY8W3co1XTs2kXMHduWZRdvYtEdX1/MA7Zk5agbwhjEmDrgI+I+IeNFylbwZ6AEkA/eISK//+QYuMM1rR0mODiImxJ/VeQetjqJaeWfVbr7YuI+7z+9HSny41XGUG9Ki2wI/5hTj6y2M0UGUykWJCM9NGU5EkC93vLeO6vomqyMpx2rPbFQ3AnMBjDErgAAgGrgK+NIY02iMKQKWA2lHfgNXmOa1o4iIva+7VFu0nMSmPRU8/vkWJvSL4bdn9LY6jnJTWnRb4MftJaQmRhDop7eulOuKDPLjb9NGkHewmod1mXh3157ZqHYD5wCIyEBaiu5i+/azpUUQMAbY1mnJndSY5Ej2VdRRWFZrdRSPV1XXyG3vriUy0I8XrhyOl1dbN3aUOnVadHeykkP1bNlXyfi+7n0lR3mGsb2juPOcfny0bg/vZxQc/wDlkto5G9U9wG9EJAt4D7jePtD9ZSAY2ERL8f66MWZDp78JJ5NuH9OzMldbTKxkjOH+jzZSUFbLS1eNICrY3+pIyo3ppdZOdnh+4/F9oy1OolTHuP3sPmTuKuXPCzYztGcYQ3qGWR1JOYAxZiEtAyRbb3uk1fMtwLg2jjtEy7SBqpW+XYOJCPRldV4pU9Pij3+Acoi3V+7iiw37+OPEAYxK0sXqlGPple5O9uOOEsIDfRncQwsT5R68vIS/TUshMtCP3727lsq6RqsjKeX0vLyEUUktfd3KGpv2VPDE51s5q38MN5/xP2N7lepwWnR3ImMMP+4oZlyfaLy1Z0y5kahgf/5x1QgKy2q5b16W9ncr1Q6je0Wxu7SGfRXa193ZKusaufWdtUQF+/H8lSnax606hRbdnSin6BAHKusZ30dbS5T7SUuK5P6JA1i8+QCvLcuzOo5STm9Mr5Z2hhU7ta+7MxljuP/DDewpr+WlGSOIDPKzOpLyEFp0d6IfdrT0c5+u/dzKTf16fDLnD4rlmUXbWLOrzOo4Sjm1gd1CiQzyY5l9rI/qHK8vz2fhxv384YL+pGkft+pEWnR3ou+3F9MrJoi4iECroyjlECLCzKnD6RHehdveXUtpdYPVkZRyWl5ewtheUfyUc1BbsjrJqtyD/GXhVs4dGMtvxmsft+pcWnR3kpqGJlbmHuSs/l2tjqKUQ4V18eWVq0dy8FADv39/Hc02LSaUOppxfaLZX1lHbkm11VHc3v6KOn737joSIgN5YZrOx606nxbdnWR5zkEammycPUCLbuX+hvQM4/FJg/lxRwnPLfb4dVCUOqpxfVrm616uLSYOVd/UzC3vrKGmoYlXr0klNMDX6kjKA2nR3UmWbCsi2N9H5wFVHmN6egJXjU7g1e9z+SzryBXDlVIACZGB9AzvokW3gz3x+RbW7S5n5pTh9I0NsTqO8lBadHcCYwzfbStifN9o/Hz0j1x5jkcvGUxaYgR/mL+BLXsrrY6jlNMREcb1iWLFzoPaiuUgczMLeHvlbm4+oxe/GNbd6jjKg2kF2Am27Ktkf2UdZ2lrifIwfj5evPLLkYR28eHmtzMp04GVSv2PcX2iqaxrYtOeCqujuJ0NheX86ZNNnNY7ivsu6G91HOXhtOjuBN9tKwLgzP4xFidRqvN1DQng//0ylQMV9dzx/jqamm1WR1LKqZzWu2Ua2eU7tcWkIx2orOM3b2USE+zPSzNG4OOtJY+ylv4N7ARLthUxLC6MriEBVkdRyhIjEiJ44rLDAyuzrY6jlFOJCfGnf2wIy3Zo0d1R6hqbuemtTKrqmvjXtWlEBftbHUkpLbodrbS6gXUF5TpVoPJ400YlcM2YRGb/kMvcjAKr4yjlVM7oF01mfhnV9U1WR3F5xhj+MH8DWYUVzJqWwqAeoVZHUgrQotvhvtlyAGPg3IGxVkdRynKPXDKI8X2jefDjjbr0tVKtnNm/Kw3NNn7Sfxen7JWlO1mQtZf7LujPBYO7WR1HqZ9p0e1gizbtIy6iC0N66m/aSvl6e/GPq0aSGBXILe+sIU8XBFEKgLSkCAL9vPl+e5HVUVzal5v2M3NxNpel9ODWM3tbHUep/6JFtwNV1jWyLKeEiYO7IaIrXykFLStWzrl+FALc+EYG5TU6o4lS/j7enNY7mqXZxbok/EnavLeCuz5YT0p8OM9cMUx/7iqno0W3Ay3ZWkRjs+HCoXp7S6nWEqOCmH1tGoVltdzy9loamnRGE6XO7B9DYVktO4v1DtCJ2lNeyw1vZBAe6Mvsa1IJ8PW2OpJS/0OLbgdatGkfsaH+jIiPsDqKUk5nVFIkT08eyorcg/zpk416dU95vMPTyi7N1haTE1FR08j1c1ZT09DMG79Kp2uozhSmnJNDi24RmSgi2SKSIyL3t/G6v4h8YH99lYgktXrtAfv2bBG5oNX2fBHZKCLrRSTTkflPRU1DE99vL+aCwd3w8tJbXEq15YrUOG47qw9zMwv52zc7rI6jlKXiIgLp0zWY77cXWx3FZdQ3NXPTfzLJP1jNq9ek0r+bLvGunJePo04sIt7Ay8B5QCGQISILjDFbWu12I1BmjOkjItOBZ4FpIjIImA4MBnoA34hIP2NMs/24s4wxTj2h6dLsYuoabUwcoq0lSh3LPef3Y39lHS9+u4OYEH9+OSbR6khKWWZCvxj+s2IXNQ1NBPo57Ee0W7DZDPfO28CqvFJenJ7y8yJDSjkrR17pTgdyjDG5xpgG4H1g0hH7TALetD+fD5wjLSMfJgHvG2PqjTF5QI79fC5j4cZ9RAb5kZ4UaXUUpZyaiPD05KGc1T+GRz7dxJeb9lsdSSnLnGWfOnB5jk4deDzPfrmNz7L28seJA5iU0tPqOEodlyOL7p5A6xUwCu3b2tzHGNMEVABRxznWAF+JyBoRuelo31xEbhKRTBHJLC7u3Ft1VXWNfLP1ABcN7abLzirVDr7eXrx89UiGxYVzx/vryMgvtTqSUpZIT44kJMCHrzbrL5/H8vryPF79IZdfjkngtxN6WR1HqXZxZEXYViPzkSOljrbPsY4dZ4wZCVwI/E5EzmjrmxtjZhtj0owxaTExMe3N3CEWbdpPXaONySPjOvX7KuXKAv18mHP9KOIiunDjGxlk76+yOpJSnc7Px4uzB3Tlm60HaGrWWX3aMi+zgMc+28L5g2J59JLBOjWgchmOLLoLgfhWX8cBe4+2j4j4AGFA6bGONcYc/m8R8DFO2Hby0dpCkqODGBEfbnUUpVxKZJAfb/4qnQBfb66ds4rdB2usjqRUp7tgcDfKahrJ3FVmdRSn88WGffzxww2M7xvNS1eN0LvJyqU48m9rBtBXRJJFxI+WgZELjthnAXCd/fkUYIlpmTdsATDdPrtJMtAXWC0iQSISAiAiQcD5wCYHvocTVlhWw8rcUiaP6Km/fSt1EuIjA3nrxnTqm2zM+NdKCsu08FaeZUK/GPx8vFisLSb/5bttRdz5wTpGJETw6jWp+PvoXNzKtTis6Lb3aN8GtR0GdwAAEytJREFULAa2AnONMZtF5HERudS+22tAlIjkAHcD99uP3QzMBbYAXwK/s89cEgssE5EsYDXwhTHmS0e9h5Pxybo9AFw2Qgd1KHWyBnQL5e0bR1NZ18jV/17F/oo6qyMp1WmC/H0Y3yearzYf0Pnr7VbmHuS3b6+hX2wIc64fpTO7KJfk0PsyxpiFxph+xpjexpin7NseMcYssD+vM8ZMNcb0McakG2NyWx37lP24/saYRfZtucaY4fbH4MPndBbGGD5au4f05EjiIwOtjqOUSxvSM4y3bkinpKqeq/69kuKqeqsjKdVpLvj/7d15eBX1vcfx95esBAgQwhKSsAgBwr7EyFptsQh4FdEioBa0WrRUra2tt9Y+bZ8+2tv23tba4gIFRdRKuW1RKhRXKlZZlR0Ewh6IBBJ2QkKS3/3jjNc0JIiQk0nOfF7Pc54zZ87MOd8vM+fHNzO/+U2PNuw/WsSmA8f9DsV3a/cd5c7Zq0Jnwb6RTdOGMX6HJHJR1BmqBn209yg7D5/ipv46yi1SE/q1a85zd2STd/QMt81cQeGpEr9DEqkVwzNb0cAI/Cgma/cd5euzVpDUOJYX77yCFo3j/A5J5KKp6K5Bc5btpklcNNf2but3KCIRI7tjErMmZ7G74JQKbwmMFo3juLxDEos2fhLYLiYf7jnC12euoHlCLHOnDKJNU93eXeo3Fd015ODxMyxcn8e4rHQax6mvmUhNGtw5mRmTsthx6CQTZiwj/7j6eEvku65PW3LyT7I5L3hdTFbtLmTSrBW0aBzLn+8eSGqzhn6HJHLJVHTXkJdW7KXMOSYN0i2sRcLhyi4tee72y8k9UsT4Gcs5cLTI75BEwuraXinERNn/X6AfFMt2FDD52ZW0Toznz3cPIqWpCm6JDCq6a0BxaRl/WrGHr3RtRYfkRn6HIxKxBndO5oU7QxdXjntmGXsKTvkdUiCY2Ugz22pmOWb2wyreb2dmS8xsjZmtN7PRFd7rbWbLzGyTmW0wM/URuEDNG8VyZZdWvLr2AGXlwehi8vqmT5j83EpSmzVk7pSBtE7U7iKRQ0V3DVi4Po/DJ0u4fUgHv0MRiXgD2ifx8pSBnC4p5WvPLGPTgWN+hxTRzCwKeJLQXYC7AxPNrHulxX5MaFjYfoTuyfCUt2408CJwj3OuB3AVcLaWQo8IY/ulkn+imOU7C/wOJezmrdrHt178kO4picy7exCtVHBLhFHRfYmcc8x8bxedWjZiaOdkv8MRCYSeqU2Zd/cgYhoY46cv51/bD/sdUiTLBnK8IVtLgLnAmErLOCDRm27KZ3cfHgGsd86tA3DOFXj3XJALNDyzFU3iopkf4V1Mnnl3Bw/9dT1DOifz0l1X0LxRrN8hidQ4Fd2X6K0t+WzOO849V3bSHShFalFG6yb8beoQ0po35PbnVjJ/Ta7fIUWqVGBfhde53ryKfgbcZma5wCLgPm9+F8CZ2etm9pGZPVTdl5jZFDNbbWarDx06VHPR13PxMVGM6tWGxRs/oagk8v5eKS0r56evbuSX//iY6/q0Zdbky2mkwQgkQqnovgTOOZ54exvtWyQwVnegFKl1bZrGM++eQVzeIYnv/nkd097ZHtjh1cKoqqMJlf+RJwKznXNpwGjgBTNrAEQDQ4FbveexZja8qi9xzs1wzmU557JatmxZc9FHgBv6pXKyuJQ3NkfWmN0nzpzlrjmreX7ZHr45rCNPjO9LbLTKEolc2rsvwWvr89i4/zj3frkz0VH6pxTxQ2J8DLO/cTlj+rblf97Yxv1z10bkEUEf5QLpFV6n8Vn3kU/dCcwDcM4tA+KBZG/dd51zh51zpwkdBe8f9ogjzMCOLWjfIoEXlu3xO5Qas/9oEeOeWcZ72w/zi7G9eOTa7jRooLPFEtlUKV6kM2fL+NXij+nWpgk39k/zOxyRQIuLjuJ34/vy0MiuvLb+ADdPX0beMQ0pWENWARlm1tHMYgldKLmg0jJ7geEAZpZJqOg+BLwO9DazBO+iyiuBzbUWeYRo0MD4+sD2rN5zhI376/+Fwyt3FTJm2vvsP1rE83dkc8sV7fwOSaRWqOi+SDOW7iT3SBE/vrY7UfrrXMR3ZsbUqzozc1IWuw6f4ro/vM+q3YV+h1XvOedKgXsJFdBbCI1SssnMfm5m13uLPQh808zWAS8Dt7uQI8BvCRXua4GPnHMLaz+L+m/cgHQaxkTV66PdoYEHdjLxj8tJjI9m/tTBDM3QAAQSHCq6L0JO/gmmvZPDdX3aqsEQqWOGZ7Zm/tTBNI6LYsKM5Tz1zxzKAzLGcbg45xY557o45zo55x7z5v3EObfAm97snBvinOvjnOvrnHujwrovOud6OOd6OueqvZBSzq9pQgw39EvllbX7OXKqxO9wvrCTxaXc+6c1PLpwC1dntuLVe4fQuVUTv8MSqVUqur+g4tIyvjdvHQlxUfzkPyoPVSsidUFG6yb8/b6hjOrZhl8v3sods1dRcLLY77BELsmkQe0pLi1n3up9n79wHbIh9xjXT/sX/9iYx8OjuvHMbQNoEh/jd1gitU5F9xf02MItrM89xq9v6k3LJnF+hyMi1WgSH8MfJvbj0Rt6smxnAaN//x5Lt2koOqm/MlMSye6YxJxlezhbVu53OJ+rrNzx5JIcxj71PqeLy3jxriu4W8PrSoCp6P4Cpr+7gzne0EYjerTxOxwR+Rxmxm0D23vdTaKZ9OxKfjR/AyeLS/0OTeSiTBl2GfuPFvFKHb9Zzr7C00ycsZz/fn0r1/Row+IHhjG4k7pjSrCp6L4Azjkef3Mb/+UN3v/wqEy/QxKRL6BH26YsvH8YU750GS+v3Ms1jy/lgxzdxVLqn+GZreieksiTS3IorYNHu0vLypn53k5GPL6UzXnH+e3NfZh2Sz+aJegOkyIquj/Hxv3HuHXmCp54eztfG5DGb8b10ViiIvVQfEwUPxqdyV/uGURsdANumbmCB+auIf/4Gb9DE7lgZsZ3rs5gd8Fp/vJh3boL6/rco4x58n0eXbiFQZ1asPiBYdzYP03dSUQ8utdqFbZ+coJHF25mT8Fp9haeJjE+msfG9uSW7HZqPETquQHtk1h0/zCe+mcO09/dyZubD/LA1V24fUgHYnSTK6kHRnRvzYD2zfnNm9u4rk9b32+bnn/iDL97aztzV+4luXEcT9/an5E92+j/S5FKVHRXISbKOF50lp6pidw1rCPX92mrU2MiEaRhbBQPjujKTf3T+Plrm3ls0RZeXrmX7361C9f2StHZLKnTzIxHrs3kxqc+YNqSHP5zZDdf4jhdUsofl+5i+tIdlJSWM2lQB743oguJGplEpErmXOSPX5uVleVWr17tdxgiUke9veUgv1r8MdsOnqRbmyY8OKIrV2e2qhNH6szsQ+dclt9x1Ca12Rfm+/+7jlfW7Ofv9w0lMyWx1r73VHEpc1ftY/q7O8g/Ucyonm14aGQ3OiY3qrUYROqq87XZOtItIoE3PLM1V3VtxWvrD/C7t7bzzTmryUxJ5I4hHbi+T1viY6L8DlHkHI+MzmTJx/l8b9465k8dHPb9tPBUCbM/2M3zH+zmWNFZBl6WxNO39WdA+6Swfq9IpNCRbhGRCkrLyvnbmv3Mem8XWw+eoEWjWG69oh3js9uR2qxhrcejI91yPu98fJBvzF7NxOx2/GJszxo/O+OcY8WuQuat2seijXmcOVvOV7u35ltXdaJ/u+Y1+l0ikUBHukVELlB0VANuzkpn3IA0PthRwHPv7+IPS3L4/Ts5ZHdI4vq+bRndK4WkRrrOQ/z3lW6hAvjpf+4gPakhU6/qfMmf6Zxj28GTvLHpE/76US67C07TJC6aG/unccfgDmS01u3bRS5GWItuMxsJPAFEATOdc7+s9H4cMAcYABQA451zu733HgbuBMqA+51zr1/IZ4qI1AQzY0jnZIZ0TmZf4WkWrDvAK2v28+NXNvLTBZvol96MoRnJDMtIpndaM418Ir75wYiuHDhaxK8Xb6WopIzvXt3lC18MfPR0CWv2HmXp9kO8teUg+wqLAMjumMR9X8lgdK8UGsaqm5XIpQhb9xIziwK2AV8FcoFVwETn3OYKy0wFejvn7jGzCcBY59x4M+sOvAxkA22Bt4Au3mrn/cyq6FSliNQE5xxb8k6wcMMB3tt+mA37j+EcJMRG0a1NEzJTEslMSSSjVWNaJ8bTKjGOhNhLO7ah7iVyIc6WlfPI/A3MW51Ldockvn9NV7LaN/+34ts5x6mSMvYWnGZv4Sn2Fp5m6ycnWbP3CDsPnwIgLroBQzsnMzyzNcMzW9E6Md6vlETqJb+6l2QDOc65nV4Qc4ExQMUCeQzwM2/6L8A0C3VIGwPMdc4VA7vMLMf7PC7gM0VEwsLM6N42ke5tE/nBNaGjgx/sKGDlrkI25x1nwdoDvLRi77+t0yQumqYJMTSOi+Yf3xlWJ0ZEkcgTE9WAX93Um6z2Sfxy8cfcPH0ZifHRJDWKpaS0nJPFpZwuKaO0/N8PtCU3jqVvenNuGpBGv3bN6Jve7JL/UBSRqoXzl5UK7KvwOhe4orplnHOlZnYMaOHNX15p3VRv+vM+EwAzmwJMAWjXrt3FZSAich7NEmIZ3SuF0b1SgNCRxNwjRew6fIr8E8XknzhD/vFijhWdpbTcqeCWsDIzbr48ndG9U3h7y0FW7irk+JlS4qIb0DgumkZxUSTGx5DWPIH2LRJIb55A0wSNqS1SW8JZdFf1v0vlvizVLVPd/Ko6TVbZP8Y5NwOYAaFTldWHKSJSM8yM9KQE0pMS/A5FAqxxXDRj+qYypm/q5y8sIrUmnFf+5ALpFV6nAQeqW8bMooGmQOF51r2QzxQRERERqVPCWXSvAjLMrKOZxQITgAWVllkATPamvwa840JXdi4AJphZnJl1BDKAlRf4mSIiIiIidUrYupd4fbTvBV4nNLzfs865TWb2c2C1c24BMAt4wbtQspBQEY233DxCF0iWAt92zpUBVPWZ4cpBRERERKQmhPUSZefcImBRpXk/qTB9BhhXzbqPAY9dyGeKiIiIiNRlupuDiIiIiEiYqegWEREREQkzFd0iIiIiImGmoltEREREJMwsNEJfZDOzQ8Cei1g1GThcw+HUF8o9uIKcf13Mvb1zrqXfQdQmtdkXJci5Q7DzD3LuUPfyr7bNDkTRfbHMbLVzLsvvOPyg3IOZOwQ7/yDnHgmCvP2CnDsEO/8g5w71K391LxERERERCTMV3SIiIiIiYaai+/xm+B2Aj5R7cAU5/yDnHgmCvP2CnDsEO/8g5w71KH/16RYRERERCTMd6RYRERERCTMV3SIiIiIiYaaiuwpmNtLMtppZjpn90O94ws3MdpvZBjNba2arvXlJZvammW33npv7HWdNMbNnzSzfzDZWmFdlvhbye29fWG9m/f2LvGZUk//PzGy/tw+sNbPRFd572Mt/q5ld40/UNcPM0s1siZltMbNNZvYdb35gtn8kClqbDcFqt9Vmq82OlDZbRXclZhYFPAmMAroDE82su79R1YovO+f6Vhjr8ofA2865DOBt73WkmA2MrDSvunxHARneYwrwdC3FGE6zOTd/gMe9faCvc24RgLfvTwB6eOs85f1G6qtS4EHnXCYwEPi2l2OQtn9ECXCbDcFpt2ejNlttdgS02Sq6z5UN5DjndjrnSoC5wBifY/LDGOB5b/p54AYfY6lRzrmlQGGl2dXlOwaY40KWA83MLKV2Ig2PavKvzhhgrnOu2Dm3C8gh9Bupl5xzec65j7zpE8AWIJUAbf8IpDb7MxHZbqvNVpvtTdf7NltF97lSgX0VXud68yKZA94wsw/NbIo3r7VzLg9COz3Qyrfoakd1+QZpf7jXOx33bIXT0hGbv5l1APoBK9D2r8+Cuo2C3m7rN6s2u95tfxXd57Iq5kX6uIpDnHP9CZ2W+baZfcnvgOqQoOwPTwOdgL5AHvAbb35E5m9mjYG/Ag84546fb9Eq5tX7/CNMULeR2u2qBWV/UJtdzaJVzKsz+avoPlcukF7hdRpwwKdYaoVz7oD3nA/MJ3Qq6uCnp2S853z/IqwV1eUbiP3BOXfQOVfmnCsH/shnpyMjLn8ziyHUeL/knPubNzvQ27+eC+Q2Ursd7N+s2uz6uf1VdJ9rFZBhZh3NLJbQBQkLfI4pbMyskZk1+XQaGAFsJJTzZG+xycCr/kRYa6rLdwEwybsieiBw7NNTWpGkUp+3sYT2AQjlP8HM4sysI6GLU1bWdnw1xcwMmAVscc79tsJbgd7+9Vyg2mxQu+0J9G9WbXY93f7OOT0qPYDRwDZgB/CI3/GEOdfLgHXeY9On+QItCF0RvN17TvI71hrM+WVCp+POEvqr+M7q8iV0qupJb1/YAGT5HX+Y8n/By289oUYrpcLyj3j5bwVG+R3/JeY+lNCpxvXAWu8xOkjbPxIfQWqzvXwD1W6rzVabHSlttm4DLyIiIiISZupeIiIiIiISZiq6RURERETCTEW3iIiIiEiYqegWEREREQkzFd0iIiIiImGmoltEREREJMxUdIuIiIiIhNn/AZgklY9og79ZAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 864x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn.recorder.plot_sched()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We're still not getting great results, so let's see what else we can do."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## RMSProp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "RMSProp is another variant of SGD introduced by Geoffrey Hinton in [Lecture 6e of his Coursera class](http://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf). The main difference with SGD is that it uses an adaptive learning rate: instead of using the same learning rate for every parameter, each parameter gets it's own specific learning rate controlled by a global learning rate. That way we can speed up training by giving a high learning rate to the weights that needs to change a lot while the ones that are good enough get a lower learning rate.\n",
    "\n",
    "How do we decide which parameter should have a high learning rate and which should not? We can look at the gradients to get an idea. Not just the one we computed, but all of them: if they have been close to 0 for a while, it means this parameter will need a higher learning rate because the loss is very flat. On the opposite, if they are all over the place, we should probably be careful and pick a low learning rate to avoid divergence. We can't just average the gradients to see if they're changing a lot, since the average of a large positive and a large negative number is close to zero. So we can use the usual trick of either taking the absolute value, or the squared values (and then taking the square root after the mean).\n",
    "\n",
    "Once again, to pick the general tendency behind the noise, we will use a moving average, specifically the moving average of the gradients squared. Then, we will update the corresponding weight by using the current gradient (for the direction) divided by the square root of this moving average (that way if it's low, the effective learning rate will be higher, and if it's big, the effective learning rate will be lower).\n",
    "\n",
    "```python\n",
    "w.square_avg = alpha * w.square_avg + (1-alpha) * (w.grad ** 2)\n",
    "new_w = w - lr * w.grad / math.sqrt(w.square_avg + eps)\n",
    "```\n",
    "\n",
    "The `eps` (*epsilon*) is added for numerical stability (usually set at 1e-8) and the default value for `alpha` is usually 0.99."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can add this to `Optimizer` by doing much the same thing we did for `avg_grad`, but with an extra `**2`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def average_sqr_grad(p, sqr_mom, sqr_avg=None, **kwargs):\n",
    "    if sqr_avg is None: sqr_avg = torch.zeros_like(p.grad.data)\n",
    "    return {'sqr_avg': sqr_avg*sqr_mom + p.grad.data**2}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And we can define our step function and optimizer as before:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def rms_prop_step(p, lr, sqr_avg, eps, grad_avg=None, **kwargs):\n",
    "    denom = sqr_avg.sqrt().add_(eps)\n",
    "    p.data.addcdiv_(-lr, p.grad, denom)\n",
    "\n",
    "opt_func = partial(Optimizer, cbs=[average_sqr_grad,rms_prop_step],\n",
    "                   sqr_mom=0.99, eps=1e-7)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's try it out:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>accuracy</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>2.766912</td>\n",
       "      <td>1.845900</td>\n",
       "      <td>0.402548</td>\n",
       "      <td>00:11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>2.194586</td>\n",
       "      <td>1.510269</td>\n",
       "      <td>0.504459</td>\n",
       "      <td>00:11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>1.869099</td>\n",
       "      <td>1.447939</td>\n",
       "      <td>0.544968</td>\n",
       "      <td>00:11</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn = get_learner(opt_func=opt_func)\n",
    "learn.fit_one_cycle(3, 0.003)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Much better! Now we just have to bring these ideas together, and we have Adam, fastai's default optimizer."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Adam"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Adam mixes the ideas of SGD with momentum and RMSProp together: it uses the moving average of the gradients as a direction and divides by the square root of the moving average of the gradients squared to give an adaptive learning rate to each parameter.\n",
    "\n",
    "There is one other difference with how Adam calculates moving averages, is that it takes the *unbiased* moving average which is:\n",
    "\n",
    "``` python\n",
    "w.avg = beta * w.avg + (1-beta) * w.grad\n",
    "unbias_avg = w.avg / (1 - (beta**(i+1)))\n",
    "```\n",
    "\n",
    "if we are the `i`-th iteration (starting at 0 like python does). This divisor of `1 - (beta**(i+1))` makes sure the unbiased average looks more like the gradients at the beginning (since `beta < 1` the denominator is very quickly very close to 1).\n",
    "\n",
    "Putting everything together, our update step looks like:\n",
    "``` python\n",
    "w.avg = beta1 * w.avg + (1-beta1) * w.grad\n",
    "unbias_avg = w.avg / (1 - (beta1**(i+1)))\n",
    "w.sqr_avg = beta2 * w.sqr_avg + (1-beta2) * (w.grad ** 2)\n",
    "new_w = w - lr * unbias_avg / sqrt(w.sqr_avg + eps)\n",
    "```\n",
    "\n",
    "Like for RMSProp, `eps` is usually set to 1e-8, and the default for `(beta1,beta2)` suggested by the literature `(0.9,0.999)`. \n",
    "\n",
    "In fastai, Adam is the default optimizer we use since it allows faster training, but we found that `beta2=0.99` is better suited for the type of schedule we are using. `beta1` is the momentum parameter, which we specify with the argument `moms` in our call to `fit_one_cycle`. As for `eps`, fastai uses a default of 1e-5. `eps` is not just useful for numerical stability. A higher `eps` limits the maximum value of the adjusted learning rate. To take an extreme example, if `eps` is 1, then the adjusted learning will never be higher than the base learning rate. \n",
    "\n",
    "Rather than show all the code for this in the book, we'll let you look at the optimizer notebook in fastai's GitHub repository--you'll see all the code we've seen so far, along with Adam and other optimizers, and lots of examples and tests."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Decoupled weight_decay"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We've discussed weight decay before, which is equivalent to (in the case of vanilla SGD) updating the parameters\n",
    "with:\n",
    "\n",
    "``` python\n",
    "new_weight = weight - lr*weight.grad - lr*wd*weight\n",
    "```\n",
    "\n",
    "This last formula explains why the name of this technique is weight decay, as each weight is decayed by a factor `lr * wd`. \n",
    "\n",
    "However, this only works correctly  for standard SGD, because we have seen that with momentum, RMSProp or in Adam, the update has some additional formulas around the gradient. In those cases, the formula that comes from L2 regularization:\n",
    "\n",
    "``` python\n",
    "weight.grad += wd*weight\n",
    "```\n",
    "\n",
    "is different than weight decay:\n",
    "\n",
    "``` python\n",
    "new_weight = weight - lr*weight.grad - lr*wd*weight\n",
    "```\n",
    "\n",
    "Most libraries use the first formulation, but it was pointed out in [Decoupled Weight Regularization](https://arxiv.org/pdf/1711.05101.pdf) by Ilya Loshchilov and Frank Hutter, second one is the only correct approach with the Adam optimizer or momentum, which is why fastai makes it its default.\n",
    "\n",
    "Now you know everything that is hidden behind the line `learn.fit_one_cycle`!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Questionnaire"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. What is the equation for a step of SGD, in math or code (as you prefer)?\n",
    "1. What do we pass to `cnn_learner` to use a non-default optimizer?\n",
    "1. What are optimizer callbacks?\n",
    "1. What does `zero_grad` do in an optimizer?\n",
    "1. What does `step` do in an optimizer? How is it implemented in the general optimizer?\n",
    "1. Rewrite `sgd_cb` to use the `+=` operator, instead of `add_`.\n",
    "1. What is momentum? Write out the equation.\n",
    "1. What's a physical analogy for momentum? How does it apply in our model training settings?\n",
    "1. What does a bigger value for momentum do to the gradients?\n",
    "1. What are the default values of momentum for 1cycle training?\n",
    "1. What is RMSProp? Write out the equation.\n",
    "1. What do the squared values of the gradients indicate?\n",
    "1. How does Adam differ from momentum and RMSProp?\n",
    "1. Write out the equation for Adam.\n",
    "1. Calculate the value of `unbias_avg` and `w.avg` for a few batches of dummy values.\n",
    "1. What's the impact of having a high eps in Adam?\n",
    "1. Read through the optimizer notebook in fastai's repo, and execute it.\n",
    "1. In what situations do dynamic learning rate methods like Adam change the behaviour of weight decay?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Further research"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Look up the \"rectified Adam\" paper and implement it using the general optimizer framework, and try it out. Search for other recent optimizers that work well in practice, and pick one to implement."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "jupytext": {
   "split_at_heading": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": true,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}