05_sklearn/05_sklearn_notes.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "STAT 479: Machine Learning (Fall 2018)  \n",
    "Instructor: Sebastian Raschka (sraschka@wisc.edu)  \n",
    "Course website: http://pages.stat.wisc.edu/~sraschka/teaching/stat479-fs2018/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# L05 - Data Preprocessing and Machine Learning with Scikit-Learn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sebastian Raschka \n",
      "\n",
      "CPython 3.6.6\n",
      "IPython 6.5.0\n",
      "\n",
      "numpy 1.15.1\n",
      "scipy 1.1.0\n",
      "matplotlib 2.2.3\n",
      "sklearn 0.19.2\n",
      "mlxtend 0.14.0dev\n"
     ]
    }
   ],
   "source": [
    "%load_ext watermark\n",
    "%watermark -v -a 'Sebastian Raschka' -p numpy,scipy,matplotlib,sklearn,mlxtend"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this lecture, we are closing the \"Computational Foundation\" section by introducing yet another Python library, pandas, which is extremely handy for data (pre)processing. The second focus of this lecture is on the [Scikit-learn](http://scikit-learn.org) machine learning library, which is widely considered as the most mature and most well-designed general machine learning library."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pandas -- A Python Library for Working with Data Frames"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Pandas is probably the most popular and convenient data wrangling library for Python (official website: https://pandas.pydata.org) \n",
    "- Pandas stands for PANel-DAta-S.\n",
    "- Relativ similar to data frames in R.\n",
    "- How is it different from NumPy arrays? \n",
    "    - Allows for heterogenous data (columns can have different data types)\n",
    "    - Adds some more convenient functions on top that are handy for data processing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading Tabular Datasets from Text Files"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Here, we are working with structured data, data which is organized similar to a \"design matrix\" (see lecture 1) -- that is, examples as rows and features as columns (in contrast: unstructured data such as text or images, etc.).\n",
    "- CSV stands for \"comma separated values\" (also common: TSV, tab seperated values).\n",
    "- The `head` command is a Linux/Unix command that shows the first 10 rows by default; the `!` denotes that Jupyter/the IPython kernel should execute it as a shell command (`!`-commands may not work if you are on Windows, but it is not really important)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Id,SepalLength[cm],SepalWidth[cm],PetalLength[cm],PetalWidth[cm],Species\n",
      "1,5.1,3.5,1.4,0.2,Iris-setosa\n",
      "2,4.9,3.0,1.4,0.2,Iris-setosa\n",
      "3,4.7,3.2,1.3,0.2,Iris-setosa\n",
      "4,4.6,3.1,1.5,0.2,Iris-setosa\n",
      "5,5.0,3.6,1.4,0.2,Iris-setosa\n",
      "6,5.4,3.9,1.7,0.4,Iris-setosa\n",
      "7,4.6,3.4,1.4,0.3,Iris-setosa\n",
      "8,5.0,3.4,1.5,0.2,Iris-setosa\n",
      "9,4.4,2.9,1.4,0.2,Iris-setosa\n"
     ]
    }
   ],
   "source": [
    "!head iris.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- We use the `read_csv` command to load the CSV file into a pandas data frame object f of the class `DataFrame`.\n",
    "- Data frames also have a `head` command; here it shows the first 5 rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Id</th>\n",
       "      <th>SepalLength[cm]</th>\n",
       "      <th>SepalWidth[cm]</th>\n",
       "      <th>PetalLength[cm]</th>\n",
       "      <th>PetalWidth[cm]</th>\n",
       "      <th>Species</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>5.1</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>4.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>4.7</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4.6</td>\n",
       "      <td>3.1</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>5.0</td>\n",
       "      <td>3.6</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Id  SepalLength[cm]  SepalWidth[cm]  PetalLength[cm]  PetalWidth[cm]  \\\n",
       "0   1              5.1             3.5              1.4             0.2   \n",
       "1   2              4.9             3.0              1.4             0.2   \n",
       "2   3              4.7             3.2              1.3             0.2   \n",
       "3   4              4.6             3.1              1.5             0.2   \n",
       "4   5              5.0             3.6              1.4             0.2   \n",
       "\n",
       "       Species  \n",
       "0  Iris-setosa  \n",
       "1  Iris-setosa  \n",
       "2  Iris-setosa  \n",
       "3  Iris-setosa  \n",
       "4  Iris-setosa  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "\n",
    "df = pd.read_csv('iris.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pandas.core.frame.DataFrame"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- It is always good to double check the dimensions and see if they are what we expect. \n",
    "- The `DataFrame` `shape` attribute works the same way as the NumPy array `shape` attribute (Lecture 04)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(150, 6)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Basic Data Handling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- The `apply` method offers a convenient way to manipulate pandas `DataFrame` entries along the column axis.\n",
    "- We can use a regular Python or lambda function as input to the apply method.\n",
    "- In this context, assume that our goal is to transform class labels from a string representation (e.g., \"Iris-Setosa\") to an integer representation (e.g., 0), which is a historical convention and a recommendation for compatibility with various machine learning tools."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Id</th>\n",
       "      <th>SepalLength[cm]</th>\n",
       "      <th>SepalWidth[cm]</th>\n",
       "      <th>PetalLength[cm]</th>\n",
       "      <th>PetalWidth[cm]</th>\n",
       "      <th>Species</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>5.1</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>4.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>4.7</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4.6</td>\n",
       "      <td>3.1</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>5.0</td>\n",
       "      <td>3.6</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Id  SepalLength[cm]  SepalWidth[cm]  PetalLength[cm]  PetalWidth[cm]  \\\n",
       "0   1              5.1             3.5              1.4             0.2   \n",
       "1   2              4.9             3.0              1.4             0.2   \n",
       "2   3              4.7             3.2              1.3             0.2   \n",
       "3   4              4.6             3.1              1.5             0.2   \n",
       "4   5              5.0             3.6              1.4             0.2   \n",
       "\n",
       "  Species  \n",
       "0       0  \n",
       "1       0  \n",
       "2       0  \n",
       "3       0  \n",
       "4       0  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Species'] = df['Species'].apply(lambda x: 0 if x=='Iris-setosa' else x)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Digression: Lambda Functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- If you are not familiar with \"lambda functions,\" they are basically the same as \"regular function but can be written more compactly as a one-liner."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Hello World 123'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def some_func(x):\n",
    "    return 'Hello World ' + str(x)\n",
    "\n",
    "some_func(123)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Hello World 123'"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f = lambda x: 'Hello World ' + str(x)\n",
    "f(123)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### .map vs. .apply"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- If we want to map column values from one value to another, it is often more convenient to use the `map` method instead of apply.\n",
    "- The achieve the following with the `apply` method, we would have to call `apply` three times."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Id</th>\n",
       "      <th>SepalLength[cm]</th>\n",
       "      <th>SepalWidth[cm]</th>\n",
       "      <th>PetalLength[cm]</th>\n",
       "      <th>PetalWidth[cm]</th>\n",
       "      <th>Species</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>5.1</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>4.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>4.7</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4.6</td>\n",
       "      <td>3.1</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>5.0</td>\n",
       "      <td>3.6</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Id  SepalLength[cm]  SepalWidth[cm]  PetalLength[cm]  PetalWidth[cm]  \\\n",
       "0   1              5.1             3.5              1.4             0.2   \n",
       "1   2              4.9             3.0              1.4             0.2   \n",
       "2   3              4.7             3.2              1.3             0.2   \n",
       "3   4              4.6             3.1              1.5             0.2   \n",
       "4   5              5.0             3.6              1.4             0.2   \n",
       "\n",
       "   Species  \n",
       "0        0  \n",
       "1        0  \n",
       "2        0  \n",
       "3        0  \n",
       "4        0  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d = {'Iris-setosa': 0,\n",
    "     'Iris-versicolor': 1,\n",
    "     'Iris-virginica': 2}\n",
    "\n",
    "df = pd.read_csv('iris.csv')\n",
    "df['Species'] = df['Species'].map(d)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- The `tail` method is similar to `head` but shows the last five rows by default; we use it to double check that the last class label  (Iris-Virginica) was also successfully transformed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Id</th>\n",
       "      <th>SepalLength[cm]</th>\n",
       "      <th>SepalWidth[cm]</th>\n",
       "      <th>PetalLength[cm]</th>\n",
       "      <th>PetalWidth[cm]</th>\n",
       "      <th>Species</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>146</td>\n",
       "      <td>6.7</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.2</td>\n",
       "      <td>2.3</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>146</th>\n",
       "      <td>147</td>\n",
       "      <td>6.3</td>\n",
       "      <td>2.5</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1.9</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>147</th>\n",
       "      <td>148</td>\n",
       "      <td>6.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.2</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>148</th>\n",
       "      <td>149</td>\n",
       "      <td>6.2</td>\n",
       "      <td>3.4</td>\n",
       "      <td>5.4</td>\n",
       "      <td>2.3</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>149</th>\n",
       "      <td>150</td>\n",
       "      <td>5.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.1</td>\n",
       "      <td>1.8</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Id  SepalLength[cm]  SepalWidth[cm]  PetalLength[cm]  PetalWidth[cm]  \\\n",
       "145  146              6.7             3.0              5.2             2.3   \n",
       "146  147              6.3             2.5              5.0             1.9   \n",
       "147  148              6.5             3.0              5.2             2.0   \n",
       "148  149              6.2             3.4              5.4             2.3   \n",
       "149  150              5.9             3.0              5.1             1.8   \n",
       "\n",
       "     Species  \n",
       "145        2  \n",
       "146        2  \n",
       "147        2  \n",
       "148        2  \n",
       "149        2  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- It's actually not a bad idea to check if all row entries of the `Species` column got transformed correctly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1, 2])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "\n",
    "np.unique(df['Species'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### NumPy Arrays"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Pandas' data frames are built on top of NumPy arrays.\n",
    "- While many machine learning-related tools also support pandas `DataFrame` objects as inputs now, by convention, we usually use NumPy arrays most tasks.\n",
    "- We can access the NumPy array that is underlying a `DataFrame` via the `values` attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
       "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
       "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
       "       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
       "       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = df['Species'].values\n",
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- There are many different ways to access columns and rows in a pandas `DataFrame`, which we won't discuss here; a good reference documentation can be found at https://pandas.pydata.org/pandas-docs/stable/indexing.html\n",
    "- The `iloc` attribute allows for integer-based indexing and slicing, which is similar to how we use indexing on NumPy arrays (Lecture 04).\n",
    "The following expression will select column 1, 2, 3, and 4 (sepal length, sepal width, petal length, petal width) from the `DataFrame` and then assign the underlying NumPy array to `X`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df.iloc[:, 1:5].values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Just as a quick check, we show the first 5 rows in the NumPy array:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[5.1, 3.5, 1.4, 0.2],\n",
       "       [4.9, 3. , 1.4, 0.2],\n",
       "       [4.7, 3.2, 1.3, 0.2],\n",
       "       [4.6, 3.1, 1.5, 0.2],\n",
       "       [5. , 3.6, 1.4, 0.2]])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exploratory Data Analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Occasionally, we will use the MLxtend library (http://rasbt.github.io/mlxtend/) -- MLxtend stands for \"machine learning extensions\" and contains some convenience functions for machine learning and data science tasks.\n",
    "- In particular, we will use the `scatterplotmatrix` function to display a scatter plot matrix of the dataset, which is useful to get a quick overview of the dataset (to inspect the relationship between features, look for outliers, etc.).\n",
    "- I just added the `scatterplotmatrix` functions a few days ago, hence it is not in mlxtend's latest release version; however, you can install the development version (currently 0.14dev) directly from GitHub by un-commenting and executing the next cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "#!pip install git+git://github.com/rasbt/mlxtend.git"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsgAAAI4CAYAAAB3OR9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzs3X2c3HV57//XNTM7szezs0nI7YaERAhLkCJiQEG0YkSx1lr6syLnqKk/K21R6021taf9Vfs7p5W29CH9eU6sHKumarFqpUe0ctMUBQoIBBACIQQI5GZzf7Ozs7tzf/3++M5MZjeb3cnO7Mxu8n4+HvOYme98v9/PNWF8eOWT63N9zN0REREREZFAqNUBiIiIiIjMJEqQRURERESqKEEWEREREamiBFlEREREpIoSZBERERGRKkqQRURERESqKEEWEREREamiBFlEREREpIoSZBERERGRKpFWB1CL+fPn+4oVK1odhjTRpk2bDrr7glbHISIiIqefWZEgr1ixgkcffbTVYUgTmdnLrY5BRERETk8qsRARERERqaIEWURERESkihJkEREREZEqs6IGebp98e7njjv2yavObUEkIiIiItJqmkEWEREREamiBFlEREREpIoSZBERERGRKkqQRURERESqKEEWEREREamiBFlEREREpIoSZBERERGRKkqQRURERESqKEEWEREREamiBFlEREREpIoSZBERERGRKtOWIJvZ18xsv5ltrjo2z8zuNrNtpee50zW+iIiIiMhUTOcM8jeAq8cc+yyw0d1XARtL70VEREREZoxpS5Dd/V7g8JjD7wI2lF5vAH59usYXEREREZmKSJPHW+TuewDcfY+ZLTzRiWZ2PXA9wPLly5sUXoPc84UTf3blHzcvjirrn1hf03k3XHTDNEciIiIiMrPN2EV67n6Lu69x9zULFixodTgiIiIicppodoK8z8yWAJSe9zd5fBERERGRCTU7Qf4hsK70eh3wf5o8voiIiIjIhKazzdutwINAn5ntMrMPATcCV5nZNuCq0nsRERERkRlj2hbpuft1J/ho7XSNKSIiIiJSrxm7SE9EREREpBWUIIuIiIiIVGl2H2SZBrX2OBYRERGRyWkGWURERESkihJkEREREZEqSpBFRERERKooQRYRERERqaIEWURERESkihJkEREREZEqavN2Al+8+7njjn3yqnOnd9B7vjDq7fqjTwYvVr5hescVERERkYqaZpDN7PW1HBMRERERme1qLbH4Uo3HRERERERmtQlLLMzsMuByYIGZfarqowQQns7ARERERERaYbIa5CgQL53XXXU8Cbx7uoISEREREWmVCRNkd/8Z8DMz+4a7v9ykmEREREREWqbWLhYxM7sFWFF9jbu/eTqCEhERERFplVoT5O8Bfw98FShMXzgiIiIiIq1Va4Kcd/cvT2skIiIiIiIzQK1t3m43sxvMbImZzSs/pjUyEREREZEWqHUGeV3p+TNVxxx4RWPDERERERFprZoSZHdfOd2BiIiIiIjMBDUlyGb2gfGOu/s/NjYcEREREZHWqrXE4pKq1+3AWuAxQAmyiIiIiJxSai2x+Fj1ezPrAb45LRGJiIiIiLRQrV0sxhoGVjUyEBERERGRmaDWGuTbCbpWAISB1cB3pyuoRvri3c8dd+yTV53bmJvf84XG3OcUtf6J9ZOec8NFNzQhEhEREZHa1VqDfFPV6zzwsrvvmoZ4RERERERaqqYSC3f/GfAs0A3MBbLTGZSIiIiISKvUlCCb2XuAh4HfBN4D/NzM3j2dgYmIiIiItEKtJRZ/Alzi7vsBzGwB8O/A96cyqJm9BAwCBSDv7mumch8RERERkUarNUEOlZPjkkNMvQNG2ZXufrDOe4iIiIiINFStCfIdZnYncGvp/bXAv01PSCIiIiIirTNhgmxm5wCL3P0zZvYbwBWAAQ8C365jXAfuMjMHvuLut4wz9vXA9QDLly+vY6jjjdf6baoefPHQqPePz9s98QW3XXfyg2y/78SfrXzDyV83wTVqzSYiIiKnu8nKJG4mqBXG3X/g7p9y908SzB7fXMe4r3f3i4G3Ax8xszeOPcHdb3H3Ne6+ZsGCBXUMJSIiIiJSu8kS5BXu/uTYg+7+KLBiqoO6e3/peT9wG3DpVO8lIiIiItJIkyXI7RN81jGVAc2sy8y6y6+BtwKbp3IvEREREZFGmyxBfsTMPjz2oJl9CNg0xTEXAfeb2S8Ieiv/2N3vmOK9REREREQaarIuFp8AbjOz/8qxhHgNEAWumcqA7v4i8KqpXCsiIiIiMt0mTJDdfR9wuZldCVxQOvxjd/+PaY9MRERERKQFauqD7O73APdMcywiIiIiIi1X60YhM9LYfsafvOrcKd3ndTuOa8Nc8dDy60843ut2HBp7uoiIiIjMcvVuFy0iIiIickpRgiwiIiIiUkUJsoiIiIhIFSXIIiIiIiJVlCCLiIiIiFRRgiwiIiIiUmVWt3mbLXYeHTnu2LI5HY25+fb7GnvNyjdMPRYRERGRU4BmkEVEREREqihBFhERERGpogRZRERERKSKEmQRERERkSpKkEVEREREqihBFhERERGpogRZRERERKSK+iA32Hg9j0VERERk9tAMsoiIiIhIFSXIIiIiIiJVlCCLiIiIiFRRgiwiIiIiUkUJsoiIiIhIFSXIIiIiIiJVlCCLiIiIiFQ5pfogf/Hu56Z03Q9Dz5/4w11/WHn5a8VzpnT/U836J9a3OgQRERGRaaMZZBERERGRKkqQRURERESqKEEWEREREanSkgTZzK42s61m9ryZfbYVMYiIiIiIjKfpCbKZhYH/BbwdOB+4zszOb3YcIiIiIiLjacUM8qXA8+7+ortnge8A72pBHCIiIiIix2lFm7elwM6q97uA1449ycyuB64vvU2Z2dYmxFY2Hzg49uCNLRy7eb7T1PE/wkeq31aPfVazYhARERGp1ooE2cY55scdcL8FuGX6wzmemT3q7mtOt7FbPX6rv7uIiIgItKbEYhewrOr9mUB/C+IQERERETlOKxLkR4BVZrbSzKLAe4EftiAOEREREZHjNL3Ewt3zZvZR4E4gDHzN3Z9udhyTaElpxwwYu9Xjt/q7i4iIiGDux5X/ioiIiIictrSTnoiIiIhIFSXIIiIiIiJVlCBXMbOXzOwpM3vCzB5twfhzzOz7ZvasmW0xs8uaNG5f6TuXH0kz+0Qzxq6K4ZNm9rSZbTazW82svZnji4iIiJSpBrmKmb0ErHH3lmzUYWYbgPvc/aulDh+d7n60yTGEgd3Aa9395SaNuRS4Hzjf3UfM7LvAv7n7N5oxvoiIiEi1VmwUIuMwswTwRuC3AErbcGdbEMpa4IVmJcdVIkCHmeWATtQbW0RERFpEJRajOXCXmW0qbXXdTK8ADgBfN7PHzeyrZtbV5Bgg6Et9azMHdPfdwE3ADmAPMODudzUzBhEREZEyJcijvd7dLwbeDnzEzN7YxLEjwMXAl9391cAQ8Nkmjk+prOPXgO81edy5wLuAlUAv0GVm72tmDCIiIiJlSpCruHt/6Xk/cBtwaROH3wXscvefl95/nyBhbqa3A4+5+74mj/sWYLu7H3D3HPAD4PImxyAiIiICKEGuMLMuM+suvwbeCmxu1vjuvhfYaWZ9pUNrgWeaNX7JdTS5vKJkB/A6M+s0MyP47ltaEIeIiIiIuliUmdkrCGaNISh3+Cd3/4smx3AR8FUgCrwIfNDdjzRp7E5gJ/AKdx9oxphjxv9z4FogDzwO/La7Z5odh4iIiIgSZBERERGRKiqxEBERERGpogRZRERERKSKEmQRERERkSpKkEVEREREqihBFhERERGpctomyGb2J2b2tJk9aWZPmNlrG3jvN5nZj0qvf8vM/mej7j3OWHPM7Ibxxh7n3JfM7CkzW9OAcf/GzPaa2afrvZeIiIjITBJpdQCtYGaXAb8KXOzuGTObT9B7eDaaA9wArK/x/Cvd/WC9g7r7Z8xsqN77iIiIiMw0p+sM8hLgYHkjCnc/6O79ZvYaM/uZmW0yszvNbAmAmf3UzG42swfMbLOZXVo6fmnp2OOl574JxhzFzN5qZg+a2WNm9j0zi5eOv2Rmf146/pSZnVc6vsDM7i4d/4qZvVxK7G8Ezi7Ngv9N6fZxM/u+mT1rZt8u7U43XgyXlOL+hZk9bGbdpRnvfzWz281su5l91Mw+VfqOD5nZvKn+oYuIiIjMBqdrgnwXsMzMnjOz9Wb2y2bWBnwJeLe7vwb4GlC9k16Xu19OMFv7tdKxZ4E3uvurgT8D/rKWwUuJ7Z8Cb3H3i4FHgU9VnXKwdPzLQLmE4XPAf5SO3wYsLx3/LPCCu1/k7p8pHXs18AngfOAVwOvHiSEK/DPwcXd/FfAWYKT08QXAfwEuLf0ZDJe+44PAB2r5jiIiIiKz1WlZYuHuKTN7DfAG4EqCRPF/ECSGd5cmXMPAnqrLbi1de6+ZJcxsDtANbDCzVYADbTWG8DqC5PU/S2NFCZLPsh+UnjcBv1F6fQVwTSmGO8xsoi2oH3b3XQBm9gSwArh/zDl9wB53f6R0z2TpfIB73H0QGDSzAeD20jVPARfW+B1FREREZqXTMkEGcPcC8FPgp2b2FPAR4Gl3v+xEl4zz/r8TJJPXmNmK0v1qYcDd7n7dCT7PlJ4LHPtvNG6ZxCTXj73H2BhOtM949fXFqvfFE9xLRERE5JRxWpZYmFlfada37CJgC7CgtIAPM2szs1dWnXNt6fgVwIC7DwA9wO7S5791EiE8BLzezM4p3bPTzM6d5Jr7gfeUzn8rMLd0fJBgJvtkPQv0mtklpXt2m5mSXxERETntnZYJMhAnKI14xsyeJCh3+DPg3cBfmdkvgCeAy6uuOWJmDwB/D3yodOyvgS+Y2X8SlGScyG+Z2a7yA4gRJNS3lsZ/CDhvkpj/HHirmT0GvJ2g/GPQ3Q8RlGpsrlqkNyl3zxIk/V8qfd+7gfZarxcRERE5VZn7if6VXcrM7KfAp9390RbGEAMK7p4vzXJ/2d0vOsl7vASsaUSbt9L9Pg+k3P2mRtxPREREZCY4XWeQZ6PlwCOl2d7/D/jwFO5xANjYqI1CgPcB6oUsIiIipxTNIIuIiIiIVNEMsoiIiIhIFSXIIiIiIiJVlCCLiIiIiFRRgiwiIiIiUkUJsoiIiIhIFSXIIiIiIiJVlCCLiIiIiFRRgiwiIiIiUkUJsoiIiIhIFSXIIiIiIiJVlCCLiIiIiFRRgiwiIiIiUkUJsoiIiIhIlUirA6jF/PnzfcWKFa0OQ5po06ZNB919QavjaAT9fk8/p9LvV0TkdNSSBNnMPgn8NuDAU8AH3T19ovNXrFjBo48+2qzwZAYws5dbHUOj6Pd7+jmVfr8iIqejppdYmNlS4PeBNe5+ARAG3tvsOERERERExtOqEosI0GFmOaAT6G9RHFKy9fBWNu7YSH+qn954L2uXr6VvXl+rwxIRERFpuqYnyO6+28xuAnYAI8Bd7n7X2PPM7HrgeoDly5c3N8jTzNbDW9nw9AYS0QSLuxaTzCTZ8PQG1r1ynZJkGdf6J9ZP+PkNF93QpEhEREQarxUlFnOBdwErgV6gy8zeN/Y8d7/F3de4+5oFC7TWZTpt3LGRRDRBIpYgZCESsQSJaIKNOza2OjQRERGRpmtFm7e3ANvd/YC754AfAJe3IA4p6U/1E4/GRx2LR+P0p1T5IiIiIqefViTIO4DXmVmnmRmwFtjSgjikpDfeSyqbGnUslU3RG+9tUUStZ2ZfM7P9Zra56tjnzWy3mT1RevxKK2MUERGR6TGlGmQz+1QNpw25+1fGHnT3n5vZ94HHgDzwOHDLVOJotloWsjXqnGZau3wtX3r8Sxw+eJhMPkMsEmNe+zw+tupjLYtpBvgG8D+Bfxxz/IvuflPzwxEREZFmmeoM8meAONA9weMPTnSxu3/O3c9z9wvc/f3unpliHE1TXsiWzCRHLWTbenhrw89pBXcPulIb4KX3pzF3vxc43Oo4REREpPmm2sXim+7+/050gpl1TfHeM1L1Qjag8rxxx8bK7G+t5xS8wNYjWxnMDtId7WZh58JR5zTbxh0bWda9jFfOf2XlWDKTbGlMM9hHzewDwKPAH7j7kfFOUheWyX3x7udqOu+TV507zZGIiIiMNqUZZHf/w0acM5vUspCtlnOePfws2w5vI5PPEG+Lk8ln2HZ4G88efnZ6v8AEtEivZl8GzgYuAvYAf3uiE9WFRUREZPaqqw+ymc0BPgCsqL6Xu/9+fWHNPL3xXpKZZGVWGI5fyFbLOYPZQUIWIhaJARCLxMgWsgxmB5vwLcZXS9wC7r6v/NrM/jfwoxaGIyIiItOk3i4W/0aQHD8FbKp6nHLWLl9LMpskmUlS9CLJTJJkNsna5WtP6px4W5wiRTKFDO5OppChSJF4W3y8YZuilrgFzGxJ1dtrgM0nOldERERmr3p30mt391o6Wsx6ffP6WPfKdaO6T1yz6ppRNbq1nLP6jNV0RjrZP7K/UoO8LL6Ms3rOasXXqjnu042Z3Qq8CZhvZruAzwFvMrOLCJYzvgT8TssCFBERkWlTb4L8TTP7MME/NVc6Ubj7Kbn6v29eX91J49rla9nw9Ab65vYRj8ZJZVMzYra2Ed/tVOLu141z+B+aHoiIiIg0Xb0lFlngb4AHOVZe8Wi9Qc1WtbRwK8/WJmIJ9g7tJRFLsO6V65ScioiIiMwQ9c4gfwo4x90PNiKY2W68Nm9HMke48eEbWRpfOmpTECXEIiIiIjNTvTPITwPDjQjkVDC2XdrBkYNsO7yNw+nDM2pTEBERERE5sXpnkAvAE2Z2D6NrkE+5Nm+1GNsubfvAdkIWoifWQ8hC424cIiIiIiIzS70J8r+WHsKxBXgQbLRxaOQQkVCElYmVlXPi0ThbDm1h/RPrKx0jymUX1bYe3jqqq8R459SiUfcREREROV3UW2LxfeBb7r7B3TcA3wK+V39Ys9PYBXhndJzBqjmrmN85v3LOzuRO+lP9Ey7kq2WxXy0adR8RERGR00m9M8gbgbcAqdL7DuAu4PI67ztrVS/Aq05Qyy3dnj/6PAs7F7L1yNZKH+SFnQtHlV2Mt9ivfPxkZn8bdR8RERGR00m9M8jt7l5Ojim97qzznqeM8Vq6JWIJ9g3tI5PPEG+Lk8ln2HZ4G88efrZy3djFfhCUZvSn+k9q/EbdR0REROR0Uu8M8pCZXezujwGY2WuAkfrDOnWMben2wTs+SMhCxCIxAGKRGNlClsHsYOWc3ngvySPbSRzdBZkkxBKk5pxJ79yVx91/ImMXDQKksil64711fisRERGRU1e9M8ifAL5nZveZ2X3APwMfrT+sU1e8LU6RIplCBncnU8hQpEi87dhM79rOs0j2P0oyO0gxGieZHSTZ/yhrO09uO+q1y9eSzCZJZpIUvUgyk5wRu/aJiIiIzGR1Jcju/ghwHvB7wA3Aanff1IjATlWrz1jNqjmriIVjpHIpYuEYq+asYvUZqyvn9O1+knXx80hEu9lbTJOIdrMufh59u588qbG0a5+IiIjIyZtSiUV1WYW754DNE50jx5RbwfXN7ass3Etmk3QMD/DBW9/Mvtwgi3JZ3th5FrR3Hbsw1gUDO096PO3aJyey/on1E35+w0U3NCkSERGRmWWqM8hfN7O5ZjbvRA/gHxoZ6KlivFndVeFuvvv8vzCYT7Mg0sWBkPP36e08ldrN4lAHyWKWDUc3s7U9PvkAIiIiIlKXqS7S6wE2ATbBOQemeO/Zbe9m2HJ7MNvbswxWvxMWXzDhJXe89BO6LEp3pB2AbDhKqJDhifwRDqSdbjcWurEx3onmgkVERESm15QSZHdf0eA4Tg17N8MDX4L2OZBYCiNHg/eXf6ySJJd7IyeiicrmHS/lkiwPH5sdHvICuZBRdCdeKJAJt7GtvYvhzKFWfTMRERGR00a9bd4ws8uBFdX3cvd/rPe+rTalLZq33B4kxx1zgvfl5y23VxLkjTs2Uhg6xNZdDzGYH6Y70kkMo78wQFfWyeKMmEHIiFkEW9BHDMhmBnk5+TIfvOOD7Bvex6LORby37728beXbpu8PQUREROQ0VFcXCzP7JnATcAVwSemxpgFxtdSUt2ge2AntidHH2hOjFtdt2f0w2/Y/TqaQIR7uIFPIECrkGSwWGcGJYDhOzp0uiwat4PIZjmaO0j/Uz2BmkAXtCxjMDHLzYzdz5/Y7p+FPQEREROT0Ve8M8hrgfHf3RgQzU0x5i+aeZUFZRXnmGCCdDI6XpJI7CFmEWLi0UUg4BjhxdzoIMYTTjtFeNEKkSeVSdEe7yRazxCNxumPdAJXn72z9jmaRZeq233fiz8Z0sfji3c8B8Lodt7Az9Py4l+xKvIbLzj6jYeGJiIi0Qr0J8mZgMbCnAbHMGP2pfhZ3LR51rKYtmle/M6g5hmDmOJ2EIy9BoRf+9QboWUZ3Ls0ABTKZo0SLBbKhMFmceUX4lUgPAAfJ8wQZ8oUCb1r2JlLZFM8ceobl3ctHDdfV1sW+4X2N+toiIiIiwhRLLMzsdjP7ITAfeMbM7jSzH5YfjQ2x+XrjvaSyqVHHatqiefEFwYK8jjmQ3A2FLOAQjlYW7Z03MsSq9DAxd1KhEDF3FuULJIrHJuHnE2FVAc4IRSut4FYkVjB2on4oN8SizkWN+tqnLDN7fS3Hxjnna2a238w2Vx2bZ2Z3m9m20vPcRscrIiIirTXVGuSbgL8FPg/8OvCXpfflx6zWsC2aD26FWCJImC0EHXNYW4iy3fI8TZ4XCJ7zoRBzC0WSuZFgvNwIQ15g5YILK7e6esXVDOWHGMwMUiwWGcwMMpQf4r19723wtz8lfanGY2N9A7h6zLHPAhvdfRWwsfReRERETiFTbfP2MwAz+yt3/6Pqz8zsr4CfNSC2lilv5lHdxeKaVddM3sVibJu3XY/C8BGIdUN8IQAPRPK8GImAQRgoGuxpi/CGcDdxIvTnBom2deJzVhJd0Me8aJxkJsmu7C7ec+57uHf3vZUuFh++8MOqP56AmV0GXA4sMLNPVX2UIPjjn5C732tmK8YcfhfwptLrDcBPgT9CREREThn11iBfxfHJwdvHOTbr1LJF83Gt4Pa+xEs+zHcGtrHvyAiLugq8dzjN256/O0ia2xPcGsnTYdAR7qzcZ6SY4yfhAr954TpI9bM9tZv2XJatL95VaQW3cO4qRrrP5OtXf326v/qpJArECX7n3VXHk8C7p3jPRe6+B8Dd95jZwvFOMrPrgesBli9fPt4pIiIiMkNNKUE2s98DbgBeYWZPVn3UDfxnIwKb6cbb8OMv99/PbsszJxRlgbUzGMpxc3seMhneFj0TRgYYaC/STQi8GJRdeJEQRY54odJW7pFdDzCYOcQia6cn0kmmkGHb/scZLmSO6ywgJ1b6l46fmdk33P3lJo99C3ALwJo1a06pLi8iIiKnuqnOIP8T8BPgC4yuwRx098N1RzULjNcKbpflKBQLdEeCXfG63YEw3+nu4G3ZFHT00BNKM0KRDguB58EiDBEiGo5U7uW5IcKEGTJnjhmxcIysO6nkjlZ93dkuZma3cPyGNm+ewr32mdmS0uzxEmB/g2IUERGRGWKqNcgDwICZfWTsZ2bW5u65uiObTns3B7vbDewMehSvfmdlp7uyrc/9mI1Pf4v+kQP0dixg7SvfR9+576h83p/qpy3UxiN7H2EwO0h3tJshM0Lk2Z0bJGtOtJglZrAn1safzj2D3nAnb8m38y/DL0AkSizcTaaQIZ/LsOaMYwvyYl4kDaS9gANZL1A0g+wQ62+7rhLT2SvW8oJlT263v9PT94C/B74KFOq81w+BdcCNpef/U+f9REREZIapayc94DHgAPAcsK30eruZPWZmr6k3uGlRXkg3crTSeo0HvhQcL9n63I/Z8MhNJLODLG4/g2R2kA2P3MTW535cOScairJp36ZgR7y2OJlChkwhR9pC5IG2YpF0yNgVhrAbi0MdJItZjhaG+b86z6ajrYPB3CAdbR1csugSzpl7TuXe89u6SVgb7RYm5TliFmZRwUkW0pWYXh45yM1P/T0v7/vFye32d3rKu/uX3f1hd99Ufkx2kZndCjwI9JnZLjP7EEFifJWZbSOowb9xekMXERGRZqt3kd4dwG3ufieAmb2VoC3Wd4H1wGvrvH/9xs4Wp/YHC+bKu92Vn7fcXplF3vj0t0hEukhEg3Vd5eeNT3/r2CyyAV56EDy3hdrIFDOkwoZ7mFyxADhzCnlCh7eRsAhEYsTPfh13XfH5SojlemYINiRZOG8Ve/dt4tXhuSxrn0cqN8QjhYOc0za3Est+KxD2EI/vf5wXMofojnazsGPh5Lv9nZ5uN7MbgNuATPngZOVA7n7dCT46yX5/IiIiMpvUO4O8ppwcA7j7XcAb3f0hIFbnves33mzxi/dAPj36vPZEkECX9I8cIN7WNeqUeFsX/SMHKu+zhSyvWfwaYpEYqVyKWCTGnPY5tFkbZkaRIhD8IThWSaTjhOgfOTjq3uW2colYgr1Dezlr0av4xC/9Lmd1zGdv+hCJaDe94S6WVe3ud6AwwqAVGCnmKjPY245uY8uhLQ34gzvlrAM+AzwAbCo9Hm1pRCIiIjJj1TuDfNjM/gj4Tun9tcARMwtDKUNspS23Hz9b3DEP9j4F3VVbSaeTwexySW/HApLZwcpsLUAqN0Q02s36J9bTn+pnd2o3CzsWcsniSyrn7EjuIB6KsDKXg0KW3eaMhIxMiKDhcThCKhIhemDbqFricn3zcTO/r/7tysv1t103KqasFyi60xGKYuWFfPksqdzoHQAF3H1lq2MQERGR2aPeGeT/ApwJ/CvBYqXlpWNh4D113rt+AzuD2eFqi38JRg4Hs8leDJ7TR4OFeiVrX/k+kvkhktnBYGe77CC7sgPs6+iutGJb2LmQJw8+ycsDL1d227Ncmq7MEJliDg+30VUoUCgWCBWKFMNtJAtZdg7vY39q14T1zeMZG1PYIU+Bro55uDuZfIaiF+mOdk94n9ORmX1gvEer4xIREZGZqa4ZZHc/CHzsBB8/f6LrzGwOQUeBCwiKD/5vd3+wnljG1bOHoeGRAAAgAElEQVQsSIDLM8gAkXZ4xZXBsXJd8sXvH9XFou/cd7AORnWxWDj/HKLxxZVWbGclzgJg//B+2sJt9MZ7eWO2wJCH2R8OM4gzp+gszudJh4rsjRbpDUVZlB0kSmji+uZxjI1pcVucFd3nko60VbpoLEssq8Qlo1xS9bqdoIb4MeAfWxOOiIiIzGR1Jchmdi7waU6+v+zfAXe4+7vNLAp0TnL+1Kx+J1vv/m9szO6l3/P0WoS10cX0XfWXx7V1G6vv3HeMSlj/9P4/ZV40PuqcZdZOW+ow/yPdAbkutiaTbEjE6SNMHCOVT5HEWXf4EH1teQhH+NOOIh5p45H0PgY9R7e1sSIcH1XfXEtMWw9vZcOmvyOR3Ec8O0Iq5yTDHaxdrvVjY7n7qL/EmVkP8M0WhSMiIiIzXL0lFt8DHgf+lGARVPlxQmaWAN4I/AOAu2fd/WidcYxra/JlNvghkjiLgSTOBj/E1uTJb6rWG+8lla2q703tJ7XzAXqLocoCwL5slnWpLAkPsdcLJAiz7uggfQWvdL2IFp1NESPjBeJEyHiBTZn9RNviJxx7PH3ZHOuODJAoFNkbiZAoFFl3ZIC+7MxuQT1DDAOrWh2EiIiIzEz1LtLLu/uXT/KaVxD0S/66mb2KoKPAx919qM5YjrPx6W+RiM0l0V0qZwDIDk5azjCetcvXjmrFltr3FMlQmGsSq4ItozvmwOKL6Nv1MH3hLojNhcF9kM8HZRyJJZDPQHE/GS8ylDlCASeMEQmFYd6K0QNOtpnJltvpi59JX3X5yMjRUe3qJGBmt3OsIV8YWE3QilCmyfon1p/wsxu0XbqIiMxw9c4g325mN5jZEjObV35Mck0EuBj4sru/Ghhi9HbVAJjZ9Wb2qJk9euDA5OUH46mlXVutxrZiS+TSrJtzAX2xucdOWvrqYBFgRw8M7YNwG5x1BfT0QmYQ2to50NFDBMfNgzbK5kQwDqSrJtFr2Mxk3AWIY9rVScVNwN+WHn9J0IrwuN+ciIiICNQ/g7yu9FxdVuEEs8QnsgvY5e4/L73/PuMkyO5+C3ALwJo1a3zs57U4Ubu23o4FU7kdfdkcfUcGYOAwjDh3ph7iRj/KPiuyyEO8N7qEFUtezcbFK4Ltnw+8wNrQHPoSy4+Nv/27dIQiLIydUTk2mE+TSu44NtB47enKx8uzw+MtQBzTrq7RtuwZ4I7N+9h9dISlczq4+oJFrF7SM23jNYq7/8zMFnFssd62VsYjIiIiM1tdM8juvnKcx0TJMe6+F9hpZuWmv2uBZ+qJ40TGa9eWzA+x9pXvO/mbjZnVvbOQ5GYOMugFFhBikCJ/nd/NF44+VmkFl5xzJhtSz7I1uaPSUq67kKcYbifjBRzIeIGiGd3FwrGxapkdXv3OoD3dBO3qGmnLngFuuXc7AyM5lvS0MzCS45Z7t7Nlz8C0jNdIZvYe4GHgNwnaD/7czN7d2qhERERkpqq3i0Un8Clgubtfb2argD53/9Ekl34M+Hapg8WLwAfrieNExmvXds2rf++k64+B42Z1v+NH6XKjG8CdbgtxMGTsTB/g0lIruMTcYH+KjUd30Zc06FnGeYmVdBTT7Cdf6WKxLBzjrPZjM8o1zQ4vvgAu/9joOuUx7eoa6Y7N++jpaKOnoy0IsfR8x+Z9s2EW+U+AS9x9P4CZLQD+neBfL0RERERGqbfE4usEi+wuL73fRdDZYsIE2d2fANbUOXZNxrZrq9XWw1vZuGNjUCoR72XtoS30RefCS/dBOsm+SIEFZoBBWzsARbLkinl44ttBzXGsm/jcs+kvDEMkqFVeu/JX2PDsN+nLZYl7gZSFSbZFWfuajx8bfPU7g9lqCGaO08lgdvji948OcvEFDUuIJyuf2H10hCU97aOu6W6PsPvoSMPHmgahcnJccoj66+9FRETkFFVvgny2u19rZtcBuPuImVkD4mqprYe3suHpDSSiiaBUIpNkQ24P6/Y9RV/bXIh1syh3gEGDbjv2RxgqOm1ehFwaonHIpIJWcN1nwrJLg1Zw/U+wLtfORnL0A70Y19gZ9FVv8NHk2eFy+URPR9uo8onr37iykrgundPBwEiuMnMMMJjOs3ROR8PHmgZ3mNmdwK2l99cC/zZdg50y7vnCqLev23GopssefGHi8zIHnptySCIiIs1Qb4KcNbMOSi20zOxsIFN3VC22ccdGEvkciYO/gEwy2D0vn2djpEi5cPq9+Sg3R3PgebpyIwwB4ZBzhoe4v83JMEIsXGBuMczv798Owz8JZoOHD9OXS9Pn+cosM/Ojx7dna+Ds8GRqKZ+4+oJF3HLvdiCYOR5M5xkYyXHtJWc2fKxGMbNzgEXu/hkz+w3gCoKO1A8C327oYCIiInLKqPefmT8H3AEsM7NvAxuBP6w7qhbrP/AM8T1PQT4dJLD5NPHhg/R3zgnKKTKDvC08l08MQ7c7B8zpxnhfcpiFVp5hdSgWMIpQyAX3yaXh8EvBrHB5ljmXhp0/h5fub9n33X10hO720X9XGls+sXpJD9e/cSU9HW3sGUjT09E2pVnfWsZqoJuBQQB3/4G7f8rdP0kwe3zzdAwoIiIis19dM8jufreZPQa8jmBm7uNAtBGBTavxNuE4+Dw8cgsM7qG3LUcy1kUiOxhs7hGJkbIIvcODEC31VR45zNvCnbwtuhhWvAGA9Vu+yZmZNOd3lhbcZYdIeoGNXZ30WalWuZgDvFK3HBwrwNGT391vPFOp7621fGL1kp5J7zXZ+I0q1ajRCnd/cuxBd3/UzFZMx4AiIiIy+9W9UMndD7n7j939R+5+EHioAXFNn/E24fjxp+HOP4aRAehaxNpkkuTIQZLZJMVwG8lClqRnWXt0X3BONB6UR6QOQOexnsr93QuIF3LBrLA7uBMvFumPdQTvc2nAwQ2K+eBYMQ+hUNCqrU5TbcV29QWLGBjJMTCSo+heeX31BYsaPn6jxqpR+wSfTUtGLiIiIrNfvTXI45nZi/TG24TjyEtBotoRzHT2FZ11yWE2xo3+tk56Q1GuGTpKn7UH56STQclEOAbDB4DzAOiNzSM5J0SikK3snJeKdtFrEUgPBjXI7XOCGeNQpDI7TaQTEovr/mpTre8tl09Uz/xee8mZJ10+Ucv4jRqrRo+Y2Yfd/X9XHzSzDxF0X5l9xiycG+XKP25eHDPQF++ubfHfJ686d5ojERGR2W46EuQp7XrXNAM7g5njavkRqOpGQShCXzZL39EB6FoVJLKZDHTNr5RTkNoPOx+CoYPB7G86yVo62bB4CSSWE4/GSR19mWT/o1wTPw+6zwwS62IRDm07NpucS0O4AJdcf9Jf5cdP7mbDgzvYl0yzKNFOyOCSFaN3+h6vvvdEZRDVSeqPn9zN5374TOXe6y5bzjsuHPPnNkatreBqKdVokE8At5nZf+VYQryGoAzommYEICIiIrPPlBJkM/sS4yfCBswZ5/jMMd4mHJGOYAa5LNoFhXzwbTKlmd/EUmir+lf5+EJYcD6k9kByN/Qso+/i97Mu2nasf/LclVyz9E307X7yWL3za38Xfv4VOLI9SMwjHTBnBcw/56S+xo+f3M2NP9lKVyzCwniU5EiO/qMjtEdCvGrZ3Mp5Y+t7a2mzNt69b/zJVoAJk+Qm1xdPyt33AZeb2ZVAuSXIj939P1oSkIiIiMwKU51BfnSKn7XeeJtwzF0RLJIbGQhKJ0JR8AIsuxyWvjo458h2wILkunxdOAJv/+tR7dj6gL55faPHrN6o5J4vwJIL4RVvPHZs5Ojxbd4mseHBHXTFIlXlDCHS2TxP7U6yYn78hK3Y7ti8j0KhyDN7kqTSeeLtERZ3x0aVQYx37/LxiRLkRrWCazR3vwe4p1H3M7OXCLpjFIC8uzdl0xsRERFpjiklyO6+odGBNM14m3C846ZRXSxILIHX/Bbkhqo26viz4Pp6N+8Yr8SjPREcPwn7kmkWxkc3DFnQHSOdH6Gno+2E9b1P9w+w6/AIsbYQ8ViYTK7Ac/tSDOcKE967OxZmXzI9YUxNri9utStLi1JFRETkFDPVEovbmaDW2N1/bcoRNcN4m3AsvgAu+PXari3buzmYEa5uFzdZwjxeiUc6GRw/CYsS7RxIpskUimTzRaKRELFwiPnx2ITXJdNB6Uh7WxgInjP5YnC86t7JkVxl5hhgMFNgUWJ0fXEttcwiIiIis81USyxuamgUs1G5XVz7nGPt4h74UjA7PVGSPF6JR/poMBt9EtaeN5+/2/g80XCYjjZjOFPgUD7LOfO7jmuzVl1f3NMRYWA4SzpXIBYJkckXcXd6Oo79FNZdtrxSc9wdCzOYKTCUyfOxN59dOadFW0bPFA7cZWYOfMXdbxl7gpldD1wPsHz58iaHd2qptTuFiIhIo0y1xOJnjQ5k1hmvXVz5+EQJ8nglHlMo1RjOOpeeNZdtB4ZIZfLEYxF62iNEwqEJ26ydv6SHzrYwe5MZkukcifY2VpzRyYr58cq9y3XG1R0yPvbms0fVHzdzy+gZ6PXu3m9mC4G7zexZd7+3+oRS0nwLwJo1a2Z2ZxcREREZpa42b2a2CvgCcD5VmzK4+yvqjKv1xtttrzqJraeWeLwSjxpUlzQ8059k+bx2ls7trCy22z8wwsFUhu9t2slQJk9XLMKrliZIZY7VE199wSJuunOAdK6Au5POFUhlCgymM1x+48ZKF4p1ly3nu79z2QljqbWl26nI3ftLz/vN7DbgUuDeia8SERGR2aLenfS+DnwZyANXAv8IfLPeoFpuvN32HvhScLysZ1lQHlFtCrXEtRq7S12uUOD+5w8xMJytLLY7kMqw92iaTK5IVzRMJlfkvm2HGM7kRt2r6MGEppX2dNl+YJBvP7yL4UyBRCzMcKbA3218nq/8bNsJ41k6p4PBqrplaG1Lt2Yxsy4z6y6/Bt4KbJ74KhEREZlN6t0opMPdN5qZufvLwOfN7D7gcw2IbXKTzfJO9bpayifqqCU+0eK2ic45MJge1Z7t6HCObL7ACwdShMyIhI1MrkgBSI5kKTqESnsaPvzSYT79vV+wdE4HBwfTnHVGFxeeeWyR4FfvewHc6YoFC/e6YkAmKLH4nV9eNe53mKkt3ZpgEcHmIxD87+ef3P2O1oYkIiIijVTvDHLazELANjP7qJldAyxsQFyTq2WWd6rXDewMkt5qY8snyrXEHXOCjUI65ky+QI/jZ4LLi9u27BmY8JyNz+7nyd0DZHIF4rEww9k82QLkixAOGe5QKFW6ugePQjH4PDl87D73PX+ITH70zG+24NiYKtmONmNgZPTMc7VyS7eejjb2DKTp6Wg7LRboufuL7v6q0uOV7v4XrY5JREREGqveGeRPAJ3A7wP/HXgzsK7eoGoy1UVyJ7rukX8Idscb2AlHXmJr5ggbo9BfGKY33Mna0Bz6esZ0I5hCLfEdm/dRLBbZsidZWSS3OBHjL25/hhcPDzMwksOAvkVxrlgV/F2jp6MNHFLpfKXVWq7gGBAJGfO6ghrj3UeDPsUd0WAmOJ0vki84YITM6OloY25nG8/0D7Kw71gpRDRs5IujM+SRnI/aEW88aunWeuuPPnnszW3XjfrshjkXBi+u/OMmRiQiIjL71TWD7O6PuHsKSAK/7+6/4e4PNSa0SdQyy1vrdfk0vHhPZVZ5a0eCDQObSQ4fYHGonWR2kA2pZ9m69MK6w35mzwDP7h0knSvQHYuQzhW4b9sBHth+uFIDnM0X+MWuATa9dKhyXWcsTCZfqCyuK5YSWjNwd/KFYqmiGPJFx4FC6ZxI5Nh/5tVLujkynGNgJEfRnYGRHCvO6AIzhjIFisUiQ5kC2UKBdZepPZmIiIicfupKkM1sjZk9BTwJPGVmvzCz1zQmtElMdZHceNftfQo65gWzyRZiY8xIdC4kkRshlE2RiHaT6F3DxuGX6w57YCSPmdHeFq48J9N5DOiKhQmFQsQiYUJmPLHrWNlFor2N3p4O2tvCDGbytEfDdLSFiEVCZAtOJBwiGjbaQkHJRa5QJBwyOqIh5nQemwlub4twxTlnjCqNuOk9r+JTbzmHzliYZKZAZyzMx9eec8L6YxEREZFTWb0lFl8DbnD3+wDM7AqCzhb1T7VOZqqL5Ma7buQwrPzlyin9hWEWd86HbArOewcAcS/Sn+qvO+xEe4Sdh4bYcXiIQtEJh4x8ESIGAyO5yqwvOOlckaI7g+k887qihOLGsnmddLdHeGrXUR544RD5YpFMvkA4FCIcgng0yoLuGN2xMAeHsuxLplm1oKtyn4GR3Li1wquX9CghFhEREaH+RXqD5eQYwN3vBwbrvGdtprhIbtzrXnElRI719O0Nd5LKDUHsWClGKpuiN95bd9htoWDxmzsYwWI6gHxpYV04FNQLFwrQFrbKLO8fXt3Hp992bmXmtzMapj1ihEMhKBVXtLeFeddFi0l0tLE/lWVBdzsfX3sOFy2fd1otpBMRERGpR70zyA+b2VeAWwm2370W+KmZXQzg7o/Vef+JTXHDjeOuK3e2AGhPsDY0hw35vbBwNXEvksqmSGaTXLPqmrpD3nVkhHAoRGc0TDhkFIpOOpfBgXyxSMSMvDtuEDZ45KXD7Ei0c+6iLt5x4dJKcvuerzxI79yuUQvpBkZybO5PTbjBh5yG7vnC+MerF/g1yJnJTeMe35VoTuWViIhII9SbIF9Ueh7b9/hygoT5zXXevznGbP/c17Ocdat/lY3DL9Of6qc33ss1q66hb15f3UMlM3mWz2vn6EieTL5ILBIi0R4mnSsSDYdI54uEzQjhhMMhFsajJEdy3PiTrcCxbaD3JdMsjEdH3bs7FmZfMl13jCIiIiKns7oSZHe/slGBtNyYWeW+0qPRFiXaSY7kOHNuZ+XYC/tTREIhzlnUTTKdY+fhYQyjWHS2HxomFgnRFjI2PLijkiAvSrRzYDBNNl+sJNrRSKjSBq6slk1JREREROSYertYLDKzfzCzn5Ten29mH2pMaKemdZctZygTLJYrFotBPTLgBC3X4tEwI5kC6XzQ5zgaNvKFIoOZPDsODVXus/a8+exLphnKFGgLwVCmwL5kmrXnza+cU8umJCIiIiIyWr2L9L4B3AmUV689R7B5iJzAOy5cymff3ldZSJfoaOPSFXO54pwFJDraGMoG7dnCBkXAzIiEQxSKECrvHQ0MZ51LzppLZyzMcM7pjIW55Ky5DGePbfhxx+Z99HS00dPRVtkopKejjTs272vBNxcRERGZHeqtQZ7v7t81sz8GcPe8mRUaENe0anXZwTsuXFoplQD49Pd+wZKedlYuiAOQHAnas2XzRdydTN4pFIssn9dduWb30REuXDaXi5bPqxwrurP76Mioc5b0jC656G6PjDpHREREREarN0EeMrMzCBbkYWavA2b0v9+Xyw56OtpGlR20sv3Z0jkdDIzkKh0peud2UnQnmc4zlC3QFYvwS0sTvGrZ3BNeAzCYzrN0TsdJnSMz0Im6Tpyk9XV2qRh7/c7Q1P9iVd3dYv+YThdjO1xcnLj2hPd5LPnPE44z0bVlX7z7uUnPqdUnrzq3YfcSEZGZo94E+VPAD4Gzzew/gQXAu+uOahpVlx0Alec7Nu+blgR5vNnq8njlY+cu6uLftxwAghnexd0xXj40xNKeDiwEsUiYcChUuRbg6gsWccu92yvXlDcBufaSM0/qHBEREREZbUo1yGZ2iZktLvU5/mXgvwEZ4C5gVwPja7jdR0fobh/994LpKjsYb5HcX9+xlZvufG7UsX/fcoC3rF5Q2QQk3h7hrHmdtEfDWGkTkKL7qHuvXtLD9W9cOWrL6LGz4LWcIyIiIiKjTXUG+SvAW0qvLwf+BPgYQV/kW5jBs8jNLDsYb7b68FCWkWyBVCZPMp0j0d7G4kSM5/YNVf659ot3P0c0Ej5uE5Cxs9yrl/RMmuzWco6IiIiIHDPVLhZhdz9cen0tcIu7/4u7/z/AOY0JbXpcfcEiBkZyQZs198rr6vKFRhlvtjqZzrFnYIR0rkB3LEI6V+DZvYM8U9V6rZmz3CIiIiIy2pQTZDMrZ3Brgf+o+qzeuuZp1cyyg6VzOhhM50cdG84UiEXCtLeFMbPK88BIfsLrtLhOREREpDmmmszeCvzMzA4CI8B9AGZ2DjO8iwU0r+xgvEVyGMTbg5njWCREJl8Eh0TVjLEW150mauhWMV4nip01/kvCshr/QlXr/Zplsk4VIiIi021KM8ju/hfAHxBsFHKFe2UFWYigFlkYf7b6Lect5MKlPcTawqQyBWJtYc5dFOeVvVpcJyIiIjITTLkcwt0fGudY4xqMniLGzlaXO1ucvyQxanZ4bA20FtfNXGZ2NfB3QBj4qrvf2OKQREREpIHq3WpaTpJmh2c3MwsD/wt4O3A+cJ2Znd/aqERERKSRZvSCulOVZodntUuB5939RQAz+w7wLuCZlkYlIiIiDWM+ZgOKmcjMDgAvT3DKfOBgk8JpJMV9Yme5+4JpHuOkmdm7gavd/bdL798PvNbdPzrmvOuB60tv+4Ct49xutv73B8U+mRn5+xURkdrMihnkyf6Pxswedfc1zYqnURT3rGTjHDvub5nufgvBpjknvtEs/nNU7CIicipTDbLIydkFLKt6fybQ36JYREREZBooQRY5OY8Aq8xspZlFgfcCP2xxTCIiItJAs6LEogYT/lP2DKa4Zxl3z5vZR4E7Cdq8fc3dn57i7Wbzn6NiFxGRU9asWKQnIiIiItIsKrEQEREREamiBFlEREREpMqsTpDNbJmZ3WNmW8zsaTP7eKtjqoWZtZvZw2b2i1Lcf97qmGplZmEze9zMftTqWGaj2fqbLZvNv90y/YZFRGQys32RXh74A3d/zMy6gU1mdre7z/RdzTLAm909ZWZtwP1m9hN3f6jVgdXg48AWINHqQGap2fqbLZvNv90y/YZFRGRCs3oG2d33uPtjpdeDBP+nt7S1UU3OA6nS27bSY8avljSzM4F3AF9tdSyz1Wz9zZbN1t9umX7DIiJSi1mdIFczsxXAq4GftzaS2pT+mfcJYD9wt7vPhrhvBv4QKLY6kFPBbPvNls3S326ZfsMiIjKpUyJBNrM48C/AJ9w92ep4auHuBXe/iGAntkvN7IJWxzQRM/tVYL+7b2p1LKeC2fibLZttv90y/YZFRKRWsz5BLtVB/gvwbXf/QavjOVnufhT4KXB1i0OZzOuBXzOzl4DvAG82s2+1NqTZabb/Zstm0W+3TL9hERGpyazeKMTMDNgAHHb3T7Q6nlqZ2QIg5+5HzawDuAv4K3efFavqzexNwKfd/VdbHctsM1t/s2Wz/bdbpt+wiIhMZLbPIL8eeD/BTNATpcevtDqoGiwB7jGzJ4FHCOo4Z1WCIVM2W3+zZfrtiojIKW9WzyCLiIiIiDTabJ9BFhERERFpKCXIIiIiIiJVlCCLiIiIiFRRgiwiIiIiUkUJsoiIiIhIFSXIgJn9iZk9bWZPltpuvbaB936Tmf3IAgfNbG7p+BIzczO7ourcA2Z2hpn9rpl9YJx7rTCzzaXXF1W3BzOzz5vZp09wzUhpa+BGfJ97zCxlZmsacT+pn36/J/V99PsVEZFJRVodQKuZ2WXArwIXu3vGzOYD0UaP4+5uZj8HLgP+DbgceLz0fL/Z/8/eu4fHdZX3/p9377lKc5FlyZIsX2M7ihOThFwgBEibuJRQmp7mnJaEtjSlPeShacPt11Io/A4tpy1pSwvn8JyU5hwubnoJlFIOLiRpakICJCE3ktiOI+fiuyzJsqS5ae77PX+sGWkkS7KtkTSSvT555tmz1957zTvSjvXutb7r+0oPMKSqJ4EvnkGXlwNXVfo6Ha9WSgPXjapeLyLfn4++LPVj79+zw96/FovFYjkT7AiyKXwwpKp5AFUdUtU+EblSRB4RkWdE5EER6QIQke+LyOdF5DER2SMib6i0v6HS9pPKtmeaz/oRJqGgsv1rTMJR3X+s0tf4aFoljudF5HHgdyptAeDTwC2VEcNbKn1cXInvNRH5wExfWER+vTLa+LyI3Ftp+6qI/E1lhO01EfkpEfmyiOwTka/O6SdrWQzs/Yu9fy0Wi8Uyv9gE2ZTKXSsi+0Xk7sofVj/wBeCXVPVK4MvAn9Zc06yq1wJ3VI4BvARcp6qvB/4b8GfTfNZjTCQYbwC+Bayt7F+LSUCm8hXgA6paTURQ1ULlM76mqper6tcqhy4C3l7p+1OV7zEJEbkE+ARwg6peBnyw5vAK4Abgw8BO4HPAJcDrRGReRvAs8469fyew96/FYrFY5oXzXmKhqmkRuRJ4K3A98DXgT4BtwEMiAuACx2su+6fKtY+KSExEWoAosENEtgAKnPLHHXgSeL2INAP+yme/JiKbMQnGX9WeLCJxoEVVH6k03Qu8Y5av853KSGJeRAaBjmnOuQH4hqoOVb7DcM2xnZWp9N3AgKrursSxF9gAzIsO1DJ/2PvX3r8Wi8VimX/O+wQZQFXLwPeB71f+uP4OsLd21GvqJdPs/3fgYVW9WUQ2VPqb+jljIvIK8JvAs5XmJ4CfA1YBvVMukWk+azbyNe/LTP/7na3P6vXelL68GfqyLAHs/XvK9fb+tVgsFktdnPcSCxHpqYyaVbkc2Ae0i1kAhYj4K1O7VW6ptL8FSKhqAogDxyrHf2OWj/wR8CHg8cr+45hp4idUddIfflUdBRIy4RTwqzWHU5hRv7NlF/AuEVlZ+Q6tc+jDskSw96+9fy0Wi8Uy/5z3CTIQwUwtvygiLwAXY/SRvwT8uYg8j5mavbbmmhEReQyzWv+3Km1/AXxGRH6EmdKeiR8BFzCRYDwLrKGywGka3gv8r8oip2xN+8OYRU21i5xOi6ruxehRH6l8t78+02stSxJ7/1osFovFMs/IlEEfy2kQYxH1e6r6dKNjORMq0+X/pqrb5rHP77OMfgaWCZbb787evxaLxWJpBIFRczsAACAASURBVHYE+dynDMRlHgstYEYQi/PRn8VyGuz9a7FYLJZFx44gWywWi8VisVgsNdgRZIvFYrFYLBaLpQabIFssFovFYrFYLDXYBNlisVgsFovFYqnBJsgWi8VisVgsFksNNkG2WCwWi8VisVhqsAmyxWKxWCwWi8VSg02QLRaLxWKxWCyWGmyCbLFYLBaLxWKx1GATZIvFYrFYLBaLpQabIFssFovFYrFYLDXYBNlisVgsFovFYqnBJsgWi8VisVgsFksNNkG2WCwWi8VisVhq8DU6gDOhra1NN2zY0OgwLIvIM888M6Sq7Y2OYz6w9+/5h71/LcuZc+n+tVjmSkMSZBH5MPBfAQV2A+9V1dxM52/YsIGnn356scKzLAFE5FCjY5gv7P17/mHvX8ty5ly6fy2WubLoCbKIdAMfAC5W1ayIfB24FfjqYsdisSxl9h1P8MCeAY6NZuluCXPjtg62dsUbHZbFYrFYLOc8jZJY+ICwiBSBJqCvQXFY5pHe4V52Hd5FX7qP1ZHVbF+3nZ7WnkaHtSzZdzzBPY8eIB720xUPkcgWuefRA9x+3UabJFssFovFssAs+iI9VT0GfBY4DBwHEqr671PPE5HbReRpEXn6xIkTix2m5SzpHe5lx94dJPNJOps7SeaT7Ni7g97h3kaHtix5YM8A8bCfeNiPIzL+/oE9A40OzWI5Yz730P5Gh2CxWCxzYtETZBFZAfwnYCOwGmgWkV+bep6q3qOqV6nqVe3tdq3AUmfX4V3EAjFiwRiOOMSCMWKBGLsO72p0aMuSY6NZoqHJEzzRkI9jo9kGRWSxWCwWy/lDI2zefgY4oKonVLUIfBO4tgFxWOaRvnQfkUBkUlskEKEvbdUzc6G7JUwqV5rUlsqV6G4JNygii8VisVjOHxqRIB8GrhGRJhERYDuwrwFxWOaR1ZHVpAvpSW3pQprVkdUNimh5c+O2DhLZIolsEU91/P2N2zoaHZrFYrFYLOc8c1qkJyIfOYPTMqr6t1MbVfXHIvIN4FmgBPwEuGcucVgaw3SL8bav286OvTsAM3KcLqRJFpJ0NHXw3gfey8DYAB1NHVzXfR3ZctYu5DsNW7vi3H7dxkkuFrdcvcYu0LNYLBaLZRGY6wjy7wMRIDrL6/+b6WJV/ZSqXqSq21T1Paqan2MclkVmpsV4ALddchuxYIz+TD+xYIwtLVv4+v6vk8qnaA+1c2LsBF98/ovsPrHbLuSbhX3HE3zuof186YcHAfitt2zgw2+70CbHFovFYrEsEnO1ebtXVT892wki0jzHvi1LmNrFeMD4dtfhXdxx+R3jo8G9w73c+b07SRfSlLWM67gUvAIBN8CriVd5XfvrJl27mKPIInLFGZxWVNXdCx7MFKy9m8VisVgsjWdOCbKqfnQ+zrEsP/rSfXQ2d05qm7oYrzrKnCqkaPY1U/bK9Gf6KWuZkBsiU8zMeO0i8QjwFCCznLMR2LAo0dRQa+8GjG8f2DNgE2SLxWKxWBaJugqFiEgL8OuYRGK8L1X9QH1hWZYqqyOrSeaT46O/cOpivOooczQQpVAqEPAFACiUCuTLeZr9zTNeu0g8pao3zHaCiHxvsYKp5dholq54aFKbtXezWCwWi2VxqdfF4ruY5Hg38EzNy3KOsn3ddpKFJMl8Ek89kvkkyUKS7eu2j59TtXzb1rqNglegUCogCJ7nUSgX2BTfNOO1i8HpkuMzPWchqLV3G0rneOK1k3x393EOD4+x73iiESFZLBaLxXLeUW+p6ZCqnomjheUcoae1h9suuW2Si8XNW26epCGujjJvbt0MwJ7hPaQKKeKhOO/uefckF4up1y42InIpp86AfLNR8dy4rYN7Hj3ASCbPS/0pRAS/49AZDVotssVisVgsi0S9CfK9IvI+4N+AcScKVR2us1/LMqbW8u2CFRewqnkVyUKS2y65bUlZuonIl4FLgb2AV2lWTPGahlC1d/vUt1+k7EFrxM/m9mbao2bBntUiWywWi8Wy8NSbIBeAvwQ+gUksqGwvqLNfyxKlugAvFohNsmqrTX7PZJR5iXCNql48lwtFxAWeBo6p6s/XE8S+44lJfsc3butgXWsTb9zYiiMT6witFtlisVgslsWh3gT5I8BmVR2aj2AsS5/pbN5G8iPc9eRddEe6JxX/WIIJ8VQeF5GLVfXFOVz7QUwFyNjpTpyNmWzdmvwOqVxp3MUCbKlpy/Lhcw/tb3QIS5K7n7t70v4dl9/RoEgsFsvpqHeR3l5gbD4CsSwPqgvwqgxlh3h5+GWGc8PLsfjHDkyS3CsiL4jIbhF54XQXicga4J3A/6k3gFpbN0dk/L2CLTVtsVgsFkuDqHcEuQw8JyIPM1mDbG3ezlGm2rwdSBzAEYd4MI4jTsOKf8yRLwPvwbiweKc5t5bPAx/FVIycFhG5HbgdYN26dTN2NJOt2/FEyZaatlgsFoulQdSbIH+r8rKcg/QO907SEW+Kb2JobIhHjz2Kpx5hN0yyYJLlS2OXjl/XoOIfc+Gwqn77bC4QkZ8HBlX1GRH56ZnOU9V7gHsArrrqKp3pvO6WMIlskXjYz4lUjldOZBhOF2iNGO/oD7/tQmBCp/ylHx4c1ynbZNlisVgsloWh3gT5G0BOVcswvnApWHdUloYzdTHeoeQhdr66kw2xDQQkwFh5jNHyKK64lMqlSdc2qPjHXHhJRP4R2MnkGZDZXCzeDPyCiPwcEAJiIvL3qvprcwmgaus2nM6zfyANAq4DXbEJWzfAlp+2WCwWi2URqVeDvAuoXTUUBv6jzj4tS4DaxXjDuWF+MvgTEvkEz514jqA/yMaWjayPraeruYugG+Sl4ZcaWvxjjoQxifHPAjdVXrM6Uqjqx1V1japuAG4FvjfX5BgmbN36U3mKnkc87OeqDSvY0BYhHvbzwJ6BGXXKD+wZmOvHWiwWi8VimYX5KBSSru6oalpEmurs07IE6Ev30dncyVB2iOcHnydbytLka2IkP8JwdpiAEyDsC5Mup7my80p2D+2mP9O/lC3dTkFV39voGMAkydPZuuWKJb7fO8DIWBEB/K5DJOijPRZiRdjHjw9kJ1nD2dHk+pgqKaq6sczUbrFYLJZzl3oT5IyIXKGqzwKIyJWANWptFP17YN9OSByB+FrYehN0bptTV9XFeAcSBwi6QcK+MIVSgYAbQBBG8iO4jks0ECXoBrlh3Q3LzrJIRHYAH1TV0cr+CuCvVPU3z+R6Vf0+8P35iKVWiwxwIpXjh/uHSBfLuCKkckVEhELZo+R5vNhXZHU8bCUX88RM/t4/teaneOToI7P6flssFovl3KNeicWHgH8WkR+IyA+ArwG/W39YlrOmfw889gXIjkKs22wf+4JpnwPb120nWUhyMnsSv+On2ddMwSvQEmjBU490IU2ulGNVeNVyklRM5dJqcgygqiPA6xsRyI3bOibZuu3tS5IulOmIBvE5guM4uI5QLHsMZwoIQtDnWMnFPFErKaq6scQCMe7rvW/a9l2HdzU6ZIvFYrEsIHUlyKr6FHAR8NvAHcBWVX1mPgKznCX7dkKoBcItII7ZhlpM+xyoVsNbGV7JaGGUllAL13ReQ2ekk+ZAM5FAhO5IN+vj65fzaJpTGTUGQERaqX9WZU5UtcjxsJ/jiRzFsrKi2SS+HhAL+XAdoVT2KHmwrjVESSfMMWyVvfqY6u8Nxo1lYGxg2vZl4tJisVgsljkyp2SgVlahqkXglGHK2nMsi0DiiBk5rpIehKGXINUPQG/3pewaO3TG+spqezQQJZlPsiq8irWxtaQLabqau5ZzUlzLXwGPicg3MCXS3wX8aaOC2doVH5dIfO6h/XztqUP85PAYJQ8ECPsdWpoC+FyHsoqtsjePTPX3BuPG0tHUQbqQPqV9Obi0iMiXMYtOB1V1W6WtFTPTtwE4CLyrMnNisVgslhrmOoL8FRFZISKtM72AL81noJbTEF8LuaR5nx6Eo09CNgmRTnqTh9nx1GdJjhyYpKN88MCD7Ni7g2Q+OWP7hSsuZPOKzbwy+gq9w73EgrFzJTlGVf8O+C/AAHAC+M+qem9jozKkcnkGU4Xx5NgDMkWPfMljS3szmXyJzljQVtmbJ6qSomQ+OcmN5daeW6dtXyaSoq8CN05p+xiwS1W3YFyIPrbYQVksFstyYK7TyXHgGczf7pk4Mce+LXNh601Gcwxw7GlIDYBXBHcDu3LHiPmaiY0ehRUbx0fD7uu9j54VPcSCMYayQxxIHOBk9iRPDzzN61e9fvy8Zl8zQTfI8cxxtq7c2qhvuCCo6ovAi42OYyoP7B2kOeBS8pRi2UMUVCFXLHP5ulbedfVa9g9kbJW9eaIqKaqdTam6sWyIb5i2famjqo+KyIYpzf8J+OnK+x2YRaZ/sGhBWSwWyzJhTglyxQPWspTo3AbX3glPfQmGX4NABGKrQVz6UkfpjKyGfHL89Kq+8sqOK8et3IJukBXBFewf3c/Lwy/T7G8GhedPPE/ADaCq58QqfhF5VlWvqPecuVKtijeTRdu+4wlOpPI4KAGfSzQUwO86eJ5HMl8er673zoUI7jxgJllR9TWVmdqXKR2qehxAVY+LyKrpTjrTUumWCg9/ZvL+9R9vTBwWi2XeqHtBkohci9GzjfdVmbq2LBSz2bkN9YIbZHxw3yuxupQjOfwysaY2hk7u50AxwXBumJJX4kjyCIPZQYJukKAvyGjOmDocSR3hQPIAnnqAGUVeG1s7Pqq86/Cu5Zw0bBWRF2Y5LphZknln3/HErFXxqscDrlDywFOjL46GfBRKOkl3bDl7ZrJzW84PfAvBmZZKt1gslnOVuhJkEbkX2AQ8B5QrzQrYBHmhqNq5hVom27ld+A7Yfz9khkx74iiMHACE7cEAO8IuI66Pl489htPcjusG2RTfxAtDL+B5Hp3NnYzmRhkYGyAgAdLeeP0XBCFdStOf6WcoO0RrqHW5r+K/6AzOKZ/+lLOntioeML59YM8AW7vi48cvX9PCkwdHKOHhAIlsEdcRfvtNGxcirPOGWjs34Fx54DsbBkSkqzJ63AUMNjogi8ViWYrUO4J8FXCxqtoRhsWi1s4NJrZP3QMdr4PmNijmYMU6OPkKeGV6vCZuc1dyl8+lVHJYWciyccMbaQu3EQvG+MngTxgtjJItZelo6qB/rB+pjEAr5lfrd/zkyjkOJA4QcALLYhX/TKjqoep7EXGBDibPgBxeqM8+NpqlKx6a1Fa1aPvOC8f4hx8fYqxQplTyKHpqVudhhrQvWxPlugunnRG3nCHVCpG1nGe2bd8GbgPuqmz/b2PDsVgslqVJvQnyHqATOD4PsVjOhMQRcPxw8AfGtSIUg/BKGHgRsglwfZBPmSTaFwL1oKmVnq430J1/jSuD7TiFNITbAFgbXUu2lCXij7D35F5igRiHUyY/DLkhFKVQLuDiUvJKnMyeJNmc5OYtNzfypzAviMidwKcwLhaVVBQFLl2oz5xaMQ+MhCKTK3LX/b3kC2XGCuVTVr9Ggy7JXJnPPrif33v7hXZB3hyZyc5tOT/wzYSI/BNmQV6biBzF3Ot3AV8Xkd8CDgO/3LgILRaLZekyJ5s3EdkpIt8G2oAXReRBEfl29TW/IVom4Qbh0I/MKHEwCmPD8NrDIC74AmYLoGVje+AGYe0bIbKK1W4T6WIGpiQHF7VexG2X3EZrqJWR/AgBN4Df8SMiOOIQdIOoKA4OK8MrzyW95geBHlW9RFVfV3ktWHIMp1bMq74/OpKlOeijrIojk+1hHIGSp0RDPobSeVsxrw5msnNbyrZtIvLmM2mbiqq+W1W7VNWvqmtU9UuqelJVt6vqlsp2eGGitlgsluXNXEeQPzuvUVjmTrZSKVk9OPkaBMIQbDGJ8pqrAQU3QG/+JEO5UR4p9IMXIXzgu8T8MVaEVvCBLR8AYGNsI/2ZfloCLQxlh8iX8igmMRZH2BzfzMfe8LFzJTkGOAIkFvMDqxXzal0sbrl6De//+2dZFQlQViXoc8gVvfFrgj5nvD2ZK9qKeXUwm53bEuYLwFRHlenaLBaLxTJPzNXm7REAEflzVZ3koSkifw48Mg+xWaajnIf1b4aTLxuJRTkHOCAC8XWQOQGpPoi0wy/8TwB6n/s7dgz/hLIvRCi8koxXYnhsGH+zn1Zp5WDiII8cfYRYIMY1XdfQO9xLspCkrGUUxef4aAu20d7U3tjvPk+IyEcqb18Dvi8i3wHy1eOq+tcL+fm1FfOqdMRCJLNF/K5DvuhRK+rPVpLlpw6O0BRwiIUDfOeFY5N8kKdaxS0LZnNjOUOmWrZtim/i1cSrp1i41bJcbNtE5E3AtUB7zT0LEAPcxkRlWWjufu7uU9ruuPyOBkRisZzfzLWSXpW3TdP2jjr7tMxE/x4YOQiHHzf7rZvAK0Mpa4qCOA6svMAkG5EOk2x0bmNX5wZim97GYGwV8eZVbGzZSEuohf6xfp4ZeIY/evyPOJI8Qu9IL48de4wDyQOMFcfwu36uX3M97932Xm7achNro2vZdXhXQ38E80S08joMPAQEatoijQjotjetI5Mv4XfFLM6bBgUyBY/UWI677u/l4FB6klXcvuOLOhheH1U3luzoZDeW/lOq1s9I1bKtWgnyUPIQn3/28xxKHJpk4dY73LuAX2RBCWDuRx8T92cUSAK/1MC4LBaL5ZxnTiPIIvLbwB3ABVP8ZKPAj+YjMMsUqglFpAuyI6ZS3vHdJjEWARwYPWpGjkUgODGaWF25nyqkiPgjDGeHGRgzOtYVwRUM5YbYfXI3Lf4WksUkjjh4eBRKBZ7ofwKAza2bz5nV/qr6xwAi8suq+s+1x0SkIYuW3nlpNwB/8C+7cQXKs/jCHB3N072iif5kng1tkVOs4pYFM7mx7Nt5xqPIUy3bBscGafY1M5gdZH18/bK3cKvM1D0iIl+tdV6xWCwWy8IzVw3yPwL3A58BPlbTnrKLPhaI2oQiFINXHjIjxuKHUCtQhsIYFNKw5o3Qtnn80urK/WggSr6cZzA7iCD4XT8eHo44iAojhRGCbhDXMY4VCAScAHuG97C5dfO5uNr/48A/n0HbovDOS7v5iwf30+x3eLE/PeN5mUIZR5RkrjjeVrWKWzYkjpiR41pKOXjpO2csuZhq2ZYqpGj2N5MqpHhl5BX2nNxDppjB7/jZFN90Ssno6eQXS5SgiNzDqQWZbmhYRBaLxXKOM1cNcgJIiMjvTD0mIn5VLU5zmaUepiYUpbyxe9MyuC6EVkKoCMlj0PcMvcUEu772C+wrJhh0lH7KqLjkvTzZUhYfPvw+P2WvTMAJUPJKeOpR9srkyjnjf6yQL+cpapFkPsnR9FFWlVfxyR9+crklGJMQkXcAPwd0i8j/rDkUA0qNicoQCfo4MjyGADMNIrsCR4dzrFvZNN6WypXobgkvSozzQnytkVVUR47Tg8adJRibLLm49s4Zk+Splm3RQJRUPkXBK/DE8ScIOAECjimR/pdP/yXdzd1sXbl1OVbQ+2fgi8D/YYEK2FgsFotlMvVqkJ8FTgD7gZcr7w+IyLMicmW9wVlqiK81i/LSg3D0SXBcY+PmC5rjuSSMHgRfgN62TezIvMyh5AGOaZ5MKUeumKLsFVBVBEFFcR2XzuZOYsEYAScAQEELADiV/zKlDGWvTKFybcANnAv6zj7gGSBX2VZf3wbe3sC4WN8axvOUkG+qE7JBgJWRAB5KseRNsoq7cVvH4gZbD1tvgtyoSYTVg+PPm/auy0CcykxJi5k5mYGplm2rmlaRKWUYzY3iFz+u4+KpR0dzB6VyiaPpo8SCMRxxiAVjxAKx5aKpL6nq36jqk6r6TPXV6KAsFovlXKbeQiEPAP+qqg8CiMjPAjcCXwfuBt5YZ/+WKltvgl2fhuMvGCcLHChmgGYojUCpAOEVsOVtfG3oMQ66Qp8ITnkMdQKE8RPwlLaKVGJwbBBVxfM8xgpjjJXHcHEpU8bBAQEXFw+TeLSF2wg4gXOiRK+qPg88LyL/sNRmO5qCft66ZSXPH0tSTuUp1IiRfQIdsSBrWptZ0eTj0HCW44ncuFXcstEfgxkVvvbOCReLcsG4s0RqKgWGYubYDEy1bFsfW88Na2/g0098GldcXMelLdxGk78JTz2K5cm/6mWkqd8pIncA/8pktxUrZ1soHv7M5P3rP96YOCpMdbawrhYWy8JTd6lpVX1/dUdV/11E/kxVPyIiwTr7tpyCmESiksCaxXlq9r2iGT0upXhcs7Q41YIhSlKLRMVPwSuaaWc3wJu63sST/U9yOH2YZn8zV7VexbODz1auAVEh4A/QFmqjpKVzqkSviOymomAQOXWkdqGLhUxl3/EED+wZ4MevDbGnL0mx7OF3HdqjQVqbAhwbzaIKWzqjbG5vpj1qnCt6OuN8+G0XLmaoC0coBoljxr4wNWAeAsWFeDf076E34J/Vzu09F79n/EHtvt77ODF2goJXYGBsAEHIlDIAfOuVbxFwArQ3tbMqvIr18fWN/NZnym2V7e/XtClwQQNisVgslvOCehPkYRH5A+C+yv4twIiIuEyU7rWcLdP5w+7bCSs2QCFlquhlBgEBfwj8EfAKkDrOrsP/wYpwgKy45H0+igIeSkoLrHTCFLwC0UCUjuYOLmy9kJ4VPeOjwYfTh0kX0ihK0A0ScAMoSkdTx7lWovfnK9uqhv7eyvZXgbHFDGTf8QT3PHqAkXSOF/uSlMsehZLieWX6C2VOZvI4CLGwj2S2yLOHRrmwI4LrOtxy9ZrFDHV+qbqyhFqM5jg1CIcfM7MgpRx4HnglWLmF3kc/w44VcWKxdcbOLXGIna/u5NL2S1kbXXuKnvi67uv44vNfJOAGcHBIFpOUtISLS7aYJS95UOjP9HPDuqW/zk1VNzY6BovFYjnfqFeD/CvAGuBbwP8F1lXaXOBddfZ9fjKjP+wLZpStrcckEIUxcANGe5zqMyWli1n6XKGj5DHgCo5XNgPNKAUU/E3kSjlWhVeRLBhXi0hgwvZ3fXQ9Y8UxCuUCPvGRLWY5MXaC67qvW5YlemdCVQ9VbLPerKofVdXdldfHWGQN8gN7BoiH/bx8IkPA5+JzHfwuVItNl8vQvSJMezREPOyn6Hn0p/Lcft3G5SWpmEqtK4s4UBqDSCfkk0ZbH2w2o8elMXYxRiw5MK4fHsxW7NzGBqfVE2fLWa5YdQVhf5hMOYPP9RHxRYgGo4T9YTz1KGiBS9sv5dXEqw3+QZweEfn16V6NjstisVjOZeoaQVbVIeDOGQ6/Uk/f5y0z+cMmjphkOLIK1r7R2LzlUoCaRDp5DBw/q8vKD/0uHWUl4zjgKSXXh9/1g89Pd6Sb9fH1bF+3nV2Hd00aFS5piVVNq0gVU2TLWZr9zWxr20a2nF2uJXpPR7OIvEVVfwggItcCzYsZwLHRLF3xEOl8iSa/y1gR/K6Dp5AvKkVVXh408oBVET83bO2g5LG8k+P+PcbODYVQ3Dz05ZIQaDb3caiiziqXYOhl+spR/ChPHXJIiTKUHaI91E6qkGJobIgDyQOcyJ4gnU/z5PEnOZg4SCQYoTvSTdANsjK0kgPJA6DQHelGVUkX06yNrl0uEqGra96HgO2YBdJ/15hwLBaL5dynrgRZRC4Efo+z9OcUkRaMZdE2jJbuN1X18XpiOWdIHDH2bQd/YJKGUAxWbjGFP3Kj5pzmNlh7Dbz2sDnn5KtGYiE+tntBvunz4Ql4jovrKfiaafE1ES5k+d2RBD1OAjqLsG47O/buAIye+GT2JJFAhLd0v4W2pjYAPPXGk4jlUqL3LPgt4MsiUs02R4HfXMwAulvCJLJFIkEf+aKHK1DyoFT2TtEoDaaLfPf5Pn7+8u5p+1oWVGdI3IAZKS7mjCtLMW9kQ64PcKGYNVrkYJSACE87ZSKJI0TiaxkWh75MHyvDK3n+xPN46jGaHUVV2T+yHxFhNGucLJL5JD7x4RNfdVB+XGa0XCRCqjppEKJyv947w+kWi8VimQfqlVj8M/AT4JOYBSTV1+n4H8ADqnoRcBmwr844zh3coPGDLeYgGDXbQz+CaKdZ9R9uMaNs5WJlfZ5bWawnxhO5XCIEeAhFVTIOqJbw50YJeMoOHaE3eRge+wI9hSK3XXIbsWCM/kw/K8Mr2dKyZTw5hmWtMz4tFbusy4BLgctU9XJVfXYxY7hxWweJbJEt7c0USmVUoewpxRkU/MmCx/QGcMuE6gxJ12UVNxZMspwdMXZvTe2gJSMhAsx97Zn/LxwfjJ0kHohT1jKj+VECboBkIYmIEPAFxr2PRYRkIWkqRWaH8Dk+Qm6IVD41SWa0HCVCGJ38lkYHYbFYLOcy9S7SK6nq35zNBSISA64DfgNAVQtAoc44zm1KBTj2jEko4mvhmjvg/o+apNlxjQa5VIR8gl2+IpcUfLzidxkVCPoCeOUiJ1DWSIi9hWHuKmX4WFMPPft20nP9x8dHhXuHe9mxdwfJfJJIIEK6kCZZSHLzlpsb/AOYX0Tk11T170XkI1PaAVDVv16sWLZ2xbn9uo08sGeAXMnj8EiWfLHMyYyxJKv6lNRu87PVoV7qVAveiGOkQkO9kE2Yh7sLboDskBk5Lo6Bvwkch0JzO1f6mjhYSpIqjdESWs+G2AaeHHiSQrlAWct0RboYHBvE7/gpanF833Vc4sE4b+h6A0PZofFy61WZ0XKYERGRnUzUjXGBrRgrzSXH5x7aP2PbsnZcOVvbtyVmE2exWM6eehPkufhzXoApKPIVEbkMU6Dhg6qaqT1JRG4HbgdYt25dnWEuI8p54wd78mUjn3DcyiKm3ORFeyOHTLLsOMBqyA6DevT5XNYWizSLy/fCAVRcKBUouw6uOLTg46SXY0fhKLedzFCbHpyjOuPpqOqMo3O5WERCwKNAEPP/0DdU9VNn20/V3u3YaJbuljDvedN69g9k+N5LA+MJ8tRU2BGWV8W8VqE04gAAIABJREFU/j3wyF/A4cfNrIcbgBUbTfXHkcOQT5gHQPXg2FPQupHeta9n12iIl8oZUq7LieIo/vwggXKZlOtw7MReygKu47IqvIpYMMZAZoBkIUnJK+FzfDT5mtgY30jIDfHS8Es8ePBB/OLnilVX8PaNb+fVxKvc++K9s1aE7B3uXSqlqT9b874EHFLVo40IxGKxWM4X6k2Q5+LP6QOuAO5U1R+LyP8APgb8/7Unqeo9wD0AV1111axDZlMTjRu3dSzfRUxuEA49DtmTxuaqlAcEmlZAZmiikIII5FMQjkMwUnklWC0Jku0baAtE2ZgbIK9lBsoj5NTjpeIIJRQfQlwddjVHmPrn/hzUGU/H1wFU9Y/neH0euEFV0yLiB34oIver6hNn2kHV3i0e9tMVD3HgRJpvPnuUaNBh/2DmlFLT1fdBn8OFHYu6jnDu9O+B7/w+DOw2FR/dgHGpOPokBFugmDZ2bpTBDUF6kF6fy45yH2V/E0e9Mo7jkitlGRTwHCiJh+tlQUwS/Ozgs7SF2ujL9CGV/9RT+tJ9uOJyfOw4Dg7NvmY8PH7Q9wOeGXyGa1ZfM61FXJXqbEosEGt4aWpVfUREOphYrPfyogZgsVgs5yF1aZBVdeM0r9OZ1x8Fjqrqjyv738AkzHOimmgkskW64qaAwj2PHmDf8cRcu2wc/Xtg8CVIHjWjavmMmWouVYomHH3SlJoOxSC+HgppMz3teWZbSLM92EnSEZJegQ2+GGmvyKhjRItlFAczTb+7nODRxbX8XUr0isheEfnfIvIblcWmZ4wa0pVdf+V1VrqHqr1bPOzHEaE/lac56OPlwTFcR4gE3VO0xqsifn5mawf7BzLT9rnk2LcTRg6ALwT+MPgCZkbE8UF+1DzkOY5JnH1+CEbZ5eSIeTCoeUKxbqJqfrAhgbJj3gdwacYh4DMFP45njxPyhQj5Q4TcytYX4ljmGEEJEg1E8fv8BH1BRIWx0tiMFnFVdh3eRSwQWxKlqUXkXcCTwC9j7DN/LCK/tOiBWCwWy3lEvS4WTcBHgHWqeruIbAF6VPXfZrpGVftF5IiI9KhqL8ay6MW5xlCbaADj2wf2DCy/UeR9Oys64zUV2zYH8BmJhZbN6PJQr0koNr4F2rfCU/dA6jhEu+C636fnxD5uSx5mlzdKn45xVaiDI5kUIuAiGI8AlwJwKDvY4C/cGFR1VSUpvrby+j0RaQeeAH6kqn9xuj4qxXCeATYD/6vmga96fFaJUNXerUo6VyIadCmpEkTIlbxxzXHQJ5QV3v3GDXiqHBvNzvGbzyPTFbPp3Db5nMQRKGVB/JBLGGcKr6bcswRAc8bOrVwAx0dfJEaneqQ8JRKIQj5F2RMExdESPiDqBFGvSKFcoKupi6Ppo/Ss6MFxHMaKY4zkR8gVc4yVxlBHccWt+Uwoe2VShdR4U76c53uHvzdJSrHEKkd+ArhaVQcBKvfqf2AGFywWi8WyANQrsfgKJkm4trJ/FONsMWOCXOFO4B9EJAC8Brx3rgFMTTQAoiEfe/sSfO6h/eOyiws7mtk/kGm8DGO2xCJxxCTI4RUwdtJMOxcz5pU4Csnj4PrNiPE7K7LEDW+d6KttM7RtpuexL9ATWgehGL2po3yTV8zUs7io+PAcB/HKFMvFmeM8x1HV/cB+4Ksisgn4OeCDwM8Cp02QVbUMXF6xLPxXEdmmqntqjs8qEarau1Uf6CIhUynPFciXFJ8j4yPI+ZIS8pvJnlSu1HgN8tQqeFVd/LV3Tk6S42vNzEehUvzDm3K/lVI1OwJekdWlEknJEfWFySePEBQ/Lori4ENQFNSjLC4BN0CmmCHsC5MpZnAdl/5MPz7Hh4jgiku+nMdX9hHyVf6NUKNdjgaM/HwoO8Qz/c8QCUQmSSlCboh0Ib1UKkc61eS4wknqdyCyWCwWyyzU+4/spspoWxFAVbNwehcqVX1OVa9S1UtV9RdVdWSuAXS3hEnlSpPaDg1lODqSHZddHDiR5q77ezk4lG6sDGPGKnmVvCq+1owSV3XH+ZSRWgBUEgPUg9FD8Mr3pu8Lxu3geod72VE4StAJ4mBG0YpaRFXxO35aw62L+/2XCCJyrYj8noj8i4g8Cfwpxh3g14CzempS1VHg+8CNZ3Nd1d4tkS3iqdIZDZLJl4gHzTNr2TM5dTWzXtnkHz//xm0dZ/NR88/UKnjhFrO/b+fk87beZKQVnndqcnzKPxPmm27PFkk6DqtwySGkvCJBhQDQhA8Bslqk7AsQkACZUoabN91MppRhIDOAg0PZK1P0imxdsRXXcckUMhRLRfKlPCpKk6+JVU2r8NTjpeGXQOCi1osmSSkQllLlyAdE5MGKHOg3gO8A321EIJYKD39m8stisZxz1JsgF0QkTOWvW2UkLj/7JfPL1EQjkS2yfyBNT0fkFH1nfzKPIzIuyXhgz8Bihjo5scgMmcVLfT8xlm39e0xC4QaNbjM7AuUcaDWxqCzbcnzG/uqpe8zU9MBu6L3fbMsl8xmd2+i97D9zV8Rlr9+hJbQSDw9XXPzix1MPTz3e3fPuxf3+S4cfArcC/wL8tKreqqqfV9UnKraDsyIi7ZWRYyr3/88AL51NAFV7t3jYz/FEjo3tET72jh5WtYRpi/jxOYLrQNjv0N0SxPW5xMP+pVFiOnHE6OBrCcVMey2d26DrUmjdONHm+CEQM4l1dQJLfICAE6DHg9tkBes9hzWBGFFx2dT2Oq4Nd3O9v4WLA22Egy34fWHam9v50BUf4qNv/CgfuuJDABS0QMAX4JrOa3jL2rfw5tVvJuwPU9ACgvDW1W/lD9/4h6yPrac/00+hVODKjisneX9HAhEK5cIkj/BYMLboC/REZLOIvFlVfx/4Wyp+3cDjVGYnLBaLxbIw1Cux+BTwALBWRP4BeDMVf+PFotZHtiqfWLsyzLqVEyv9q/rOZG5iFCsa8i2+lrPqAZsehCM/NouXwq0mWX7sC3DhOyAUheYOswDvFMQ4W2RHjOzixIvgC08UFDnxIhTHxlfgn8yeZEVwBUV/kVw5R7aUpaQlfOLj/Ze9n/e+bs7KluXOaib0x+8XER+mdO/jwOOq+tppru8CdlR0yA7w9dl09zOxtSt+SrK7fyAzSXoBjO8vGR/Z+FozY1Etgw7GkjC+dmK/KiU6sR8KGeNigUCwkhxXbTpKOTPKXMpCuWyuTRyFpiBoHvwBAEbbLuBY+hipYorupg5u7bmVt298+/jHVa3bdg/u5tXkq/x44MfsGd7DptgmfmXrr3DH5XeMn9s73MuriVcBiAajjBUnL1atSimqji5Vu7fT2cItAJ8H/hBAVb8JfBNARK6qHLtpMYKwWCyW85G6EmRVfUhEngWuwfzJ+yBmNnRRqSYaVbu3VwfTvHAkQVPApT0WQgRS+fKkpKMhWs5qYjHUW1nZH4LMsHGqOPw49H4XQitMMqG15SGY0B6rGj/ZUgGSfSY5Drcam7dS3hQLqazAXxleSb6UJ+gL0h3tJugG6VnRQywYO5+TY1S1H5NsVBOOJkyJ6T8GNgLuzFeDqr4AvH4hYrtxWwf3PHoAMA9xqVyJRLbILVevWYiPmxtbb5qQ84RiJjnOjcIV7zFtVSlRuQSoub9xzEK87MiEq8XYkCmhXkiDVwbK9LouO5r9lB2Ho1LCcQIcGXqepOPicwN0R7pJ5VN8/tnPA0xKksNumGcHnyXgBgi7YbLFLM8OPssbut4wfs5U+7aiV+SFEy8AsDa29pTiOA22e9tQudcmoapPi8iGhf5wi8ViOZ+pe6GHqp5U1e+o6r+p6hDGCWDRqdq9HRxK43nKWKHMyFiRkXSOxFiRkUyezlhwXIbREC3n1ptMIpEZMk4UmWFTBc/xQT4NhSxkTsLwgVM1m+UioCaRKGUrBURcM3KcPGb6Ug9CcfrSfUQCETbGN5Iv58mX8vjFz3BueDmX1503RCQuIjeKyKdF5D+AI8B7gJ3ALY2Mbar0YsnIKmrp3Da57Hm4ZfICvaqUKH3cbFs3QihupEG+AKBmu/oqCDabZNlxAZddzWFiCoM+h5DrJ6qQxEO9ImFfmEQhQTQYpdnXzH29900K69Fjj9Le1E7YH6akJcL+MO1N7Tx67NHxc6bat62PrefS9ksZzA5OK6VosN1baJZjdT3di8hBEdktIs+JyNP19GWxWCznIvVKLKbjtIv0FoKq3du+40niTQGiYT+DyRwjWTNS3N0SYkNbZFyGccvVaxY/6agmFvd/1CTJxTGIrTYjaOJAIAyFynSvUxkxxgE8xgsNOw40rzIl1QJRKKTMNcU0rL0GVm5idSROMp+kLdzGZasu40DiACezJ1kZXtmQQgdLkFcwD3KPAf8deLKywHRJMJ30YsnRue1UWzeAPd+Cp79sRovLBQi3Gy19PmUe4Np7IN5tnFpi3UY/H18LJ18Bx0+fv0CnEyTlgOO4DGmBnJi7v5gdRIFU5gTRQJzjY8f55A8/OS57GBgboD3UjuNMPPd7nsfA2MRag+ns29ZG1+J3/PzJW/7klK/TYLu3p0Tkfar6v2sbReS3MO5B9XJ9ZVDDYrFYLFNYiAT5rAomzBfHRrP4XXj1RBpVxVPIFUqUFPyuMFYoMZQpMpDMcTgW4sKO5sYkIZ3b4B1/Yaag+35SGWkbNEUTol0w9DIgZoTZy1VGiv0muVi5xWiPA83m/cAeM40tYpLkfBK23sT2gJ8de3cA0BpqJeAESDYnbXJcQVXbGx3DOcmeb8F/fMrIgNxgZXbjkHnYc3zmXh45aB4IN/6UkWaEYuY8nzl/tQdJKePio48SAXFQPFBMeWmgQJn+wkkibniS7CHqj5IpZogGJyqIZ4oZOpomZopWR1aTzCfP2L7tbM+fZz6EsRD8VSYS4qswMrabFyMAi8ViOV+ZU4IsIl9g+kRYgJZp2hecoCs8/towrgjZUpl0vowqhPwOiWyR/kSerliJ7hVhktkid93fC8A7L+1e/GCnjiQHmsxocNMKGD08Iafwh422OJcBnwOtG0zBhUjXZBcBz5tYBIUpF33bJbex6/Cu8eIHN2+52SbHFURkJ7M8yKnqLyxiOOcOT90DgciE9EIqk0leseK+EjYPek5lmUJuFKKrYWCvaSsn2e5F2eErURAxcnvHQTwPrRGDlSu/OrdcGpc9AHQ3d7NvZB8Azf5mMsUMmVKG9136vvFrt6/bPv7wGAlETtEcT+Vsz59PVHUAuFZErgeqw/XfUdXvzUf3wL+LiAJ/W/HtHud0hW4sp+fu0Rr5+HN3z2/f0/RXuxDVYrHUz1xHkGfTrDVEz1Zd0hYP+xlK5/HU7OdLHlp5n8yXWOs4xMPmr+2Oxw83JkGGySPJXtkkCdkENK2E7LAZdWtZbxIKXxh+5o9h2y9OLIA6/rxZ4BRqMcVF1rzBjDpXbN6qK/At0/LZRgdwTpI6bhxYHMfIJ4b2M77Q1B82r9AKc0+X8+Yhcd9OMyuSS0C0ix5RbqPAJzhJi+NQcFwcSrgYoVFVdBTFxat5xokEIqSLaT50xYe4r/c+BsYG6Gjq4H2Xvm/SQr6zfXhcCg+bqvow8PA8d/tmVe0TkVXAQyLykqqOi7VPV+jGYrFYznXmlCCr6o75DqReCmXl6o0reG1oDB2cEEILlT+qArliefz8aNBlIJlrRKgTVEeSa5OElnXGsWL0kJFMVEpIs+0XJ1/zr+8H1Cx+6roMIqtM4jHVi9ZyCqr6SKNjWLbMVgky2mUe8sJxI7NwfGYUWXzmvg40GzlFccxILZ642/Rx/Scm6Zl7gBueu3tc2vCt3V+l4JVxHQe3sq446xUJOxP/fFVlDxviG7i68+rxZHZDfMMpX+FsHx5rz2+g5du8oqp9le2giPwr8Abg0dmvslgslvOHuUosltwUdbV075suWMnuo6Nkix4A5UqUZQW/TKwfTOXLdMRmWyS+SExd7FRNQIKRUxOQ2msueufpvWgtsyIiW4DPABdT4xigqhc0LKilzOlKTF99u9EgF7OQT5h/IRTw+WHkEEQ6THLslWDVJbOWqa6VNlyychuPn3gGx3PodpvJegUSlNnWshVPvXHZw+tXvX5BLdkabPk2b4hIM6Z8dary/meBTzc4LIvFYllSzFViseSmqKv+scPpPCXPm+EsxfM8UvkymXyJO2/YtKgxnpbTJSC1nM6L1nImfAVT7OZzwPXAe2mQC8uyoLYSJExsK7Ke8VmOhz5pPLnDLdB9pZEMJY6abWw1tG6Gts3T91GhVtqQjq3mWhGODe8nVc7R4Y/yX9bfTLYpPkn2UGvJBoxvdx3eNS8J7EL3v4h0YBb/gfkb8I+q+kBjQ7JYLJalxVwlFktuirrqH/upb7+I503OcqTm3WC6QEcsxJ03bGqc/ngmTpeA1FIrz6hOd1/xnumttywzEVbVXSIiqnoI+CMR+QEmaT7/mE0+AROVIGuZWmK6bbO5h50AeAVIHjEjx1tvmvD2rvTRmx9hV/YIfaUMq5Nltg/3Tko0ewpFekYSPDj4Evd5w6T8QTpWbj6lil6Ve1+89xRLtnw5z/cOf288ka5HEtFgy7d5o1Ip8rJGx2GxWCxLmbps3pbaFPXWrjjrWpt47vDweNVamNCCOChbVkW4ZLWxd/vcQ/vHfZFv3NZRv+3b6RKM03EmCUgtM3nRWs6UnIg4wMsi8rvAMWBVg2NqDGcye3G6EtPVPrxSZaGpz8gtnACkfwQb3mq08tlReh1lR2ofMSdApzok/f7JcoVKXw/qGJ8vHqVZHdqLSsodnLaKHpxqyTaUHeKZ/meIBCLzIolosOWbxWKxWBaReivpfQX4G6CEmaL+O+DeeoOqh7F8kUJ5hoMKR4ezPHd4mLvu7+XgUJqueIhEtsg9jx5g3/HE3D+4mhxkRycnGP17zryP+FqTcNRidcULyYeAJuADwJWYanq3NTSiRlE7eyGO2YZaTHuVaiXI7KhZEJodNftbb5rchxswi/Ncn1mglxs5pY9dyZeJOX5inodTzhPreN3kCnWVvu4rHKNZfER9IRzXT7SQnbaKHhjdcrKQJJlP4qnHS8MvgcBFrRfNSxW8qf0n80lbmfIs+NxD+/ncQ/sbHUZjOPCDya8Kdz939ykvi8WyNKi3UMiSmKLedzzBA3sG2NuX4EevzFwYqqSQGMuTzBeJhfz0J/NsaIsQD/sBU41vzqPIM8kjnvqSGTWrjiq3b4UT+6YfZba64kVFVZ8CqIwif0BVUw0OqXGcyezF6WQ91T7Ug9gakxh7OfAE1r/ZWLtV+uj74SfoLBQnubBE1JuQK1T6GhjJ0i6VySnHB+U8zf7mSdXxqky1ZCuUClzZcSVtTW3j59QjiVgKlm8Wi8ViWRzqTZAbPkW973iCex49QLnscXQ4S6440wI9Q3+qQNDn0B0PkcwVx9ujIR/HRuuoNjxdglHKwYFHYMvbzbGTr8Lz90H3VbDyglOnsa2ueFERkaswsyDRyn4C+E1VnY8yvsuL2eQTs0mH+vfAw5+B/hegfzfjpdEDEWPtVsyBPwS+EIQr+t3Obaze/PbZ5QqVeDqcMCmvQFQCRrrhBk+pjldLrSXb3RWruFrqlURYf3GLxWI5P6hXYtHwKeoH9gwQD/vpT+UJ+h3kNB4EAhRKxskiFvKPt6dyJbpbwnMPZDp5RP9uCLdOTFun+kzikD4+8zR25za4/uPwi3ebrU2OF5IvA3eo6gZV3QD8DiZhPv+YST7RvnVm6VBVVnTyVRg+aMpL50bBU0j2QarfaJCjqydLMTgDuUIlnlsD3WS0RKqUwysXSQXCZEoZbu259bRfyUoiLBaLxTJX6hpBXgpT1MdGs3TFQ6RzJSJBF0cET2cu/CRiPJEz+RI9HRE8VVK5EolskVuuXjP3QKaTR2SHoeN1cPAHZj89AJHOyYn0bIvwTkO1aMF8rNA/T0mp6rggUFV/KCLnp8xiptmL2ZxVwBwb2G2q5DWtMEVuCulKhccCtPfAyk2nLFg9rVyhEs/b9+2EwSz3ecMMuA4dzat43wwuFlOxkgiLxWKxzJV6XSwaPkVdLRASCfnIF8sEXKHkzVIZVSESdPnYO3rYP5AZd7G45eo19blYTJdgdF4Kgy9CMAbBKGSGTIW81o0T181xEd65UrSgwTwpIn8L/BPG7OQW4PsicgWAqj67mMFUtfTz6qxyNkznivLE3absefUhLxSDlVsmHupi3aY9GDX74RVGK9x9JRx7xuxPpSLZ6EkcoSe+FrbeOv1MSSWetwOnT4enx0oiLBaLxTIX6tUgV6eofwAgIm/BJMyX1hvYmVItENIZDbJ/IH3aKg+uK9x5wyYuaI+wfyAzv8FMTTB2fnjy8XCLGV0r5s00dh2L8M6hogWN5PLKduqi0msxCfMNixVIVUsfD/snOavcft3GxU2Sp+IGTXJcfcgr5uDQZMs2QrEJrXEpD45rzgnGTrWMgzMvhmOxLBQPf2by/vUfb0wci8x0Lhl3XH5HAyKxWJY+9SbIDZ+irhYIeWDPAGPFMkdHZk96Q34ju16UZKScN6v3T75skuGmVjP6NnIAksfqWoR3rhQtaCSqen2jY6hS1dJXHVXmxVlloanKiqKrYWCvSY7xjAYZjDtFVWsPk2UZZ1IMx2KxWCyWBlFvgrwkpqi3dsXZ2hVn3/EEv/zFkzOe5wArm4PsePwwP3tx58InI1VngA1vnWjLjkLHxXWPWNiiBfUjIh3AnwGrVfUdInIx8CZV/dJix1LV0tdSt7PKfDD1IS8Ug87XTbJsY99OKGYgl4Bg3Mgvut9oRpir1Grtz6YYjsVisVgsDaDeBHnJTVF7Om40dQohv5DIFskWy0RDk7/6giQjC+hrvH3ddnbs3QGYkeN0IU2ykOTmLTfX3fd5xFcxkqBPVPb3A18DFj1Brmrpqw9rMA/OKvPBTA95NZZtp4z8PvwZGHoFTtboliNdpgz1+PUtkB6EoV6jzW9uM9rkaUaR7WJUi8VisSw2ddm8qer1s7wWLTmGiSnqliY/sbAP/5RvJsCK5iCegusIqVxp0vEFSUaqI2zhFiOpCLfMm9ayukI/FozRn+knFozZBXpnT5uqfp3K85SqloCZ6jAuKDdu6yCRLZLIFvFUx9/fuG16v99F43TV86ajfSscexqyCWNrmE2Y/fatE/2dfBUOP2GOOa5JoKepPFldjJrMJyctRu0d7l3gL26xWCyW85l6XSwaMkU93Wr/6hT1Zd1xHu4dpDzFyKI5IOSLHp7nsbGtmUTWFAmJhnzzY/M2E9ONsM0TdoV+3WREZCVmtgMRuQaoo9743KnV0s+bs8p8MJfiNSf2wZqrje93LgnhOKyqVJDc9oumv/s/Clo21nBtPRML/qZoke1iVIvFYrE0gnolFl9lkaeoZ1rtH/Y746PCnk6sE6oyVlCaAnDFuhYuW9fKjds6ll4yYllsPgJ8G9gkIj8C2vl/7J15eFvVmf8/79Vm2bKcxY4dJ3ESliRACCUkFCilgUBo2SkwgTJlm5YyzATalJlfaRnIdJl2BqZNF2iboUCmhYallBY6Zakh0JRQshCyNHEWstqJ7cSLFmvX+f1xJVmy5X2RHJ/P8+iRdO+5976Sj4/e+573fF+4PlfGJHPp846+3uS1HjSlDMef2L5NxdvzjCtmw9hpZm6zpE31ZMlF1otRNRqNRpMLBuoglyqlnhOR+8GcohaRIZ2i7mq1fyQaY/8xP+/vPUY0prAmfncthgEoLCJMHONkrKsgpS+bl86IZsgRkfnAQaXURhH5FPAl4DrgdeBQTo07HuiubHVf2qAXo44UfvDGzn4d85VLZgyBNVnoKOvW1/0jgI4Sblq+TaMZGAMtNT3sU9S1LYGsC+zqvSEMESIxRTJ4bDEMCm0GVkOIxBXhWDz3urKafODnQDjx+jzMGZBHgWZgRa6MOm7oTd5yL3ObdblojUaj0eSCgTrIHaeo/xdYMmCrumHSGGfWBXaeYJRYLIZSprcejUM4GscfjmE1DMYV2ll0aoV2jjUAFqVUU+L1YmCFUuo3Sql/A07q7kARmSIib4nIdhHZJiL3Drm1I43eLE7t5QJWvRhVo9FoNLmgXykWuZyiTlbOg8wFdoFwlB11fqwGRBIabwoIxxQxFWXSmILcKwJo8gWLiFgTqhULgTvT9vX0PxEFvpro+8XABhF5Qyn1t6EyNq9IlIlOLdg75cpuy0R3Sy9zm/ViVI1Go9EMN/2NIOdsijq52r/EaeNwa5ASp407L5jOMX8YwzCwWgzsFjJKTk8qcXJm1VgdPdYk+TXwtoj8DggAyVLpJ9FDipBS6nCyAI5SygtsByZ1d8xxw5GtphRboCWzTHQHaTaNRqPRaEY6/V2kl3WKGviNiGwaHNO6Jr1y3qtb6/nFmn14AhEMgUhMYbdYsFkUsbjCajG4+sxJHG4NDrVZmhGCUuo7IlINTAReV0ol09YN+pAiJCLTgDOBvw62jXnJ9pd1mWiNRqPRjAr67SAPYIp6UOgo91Zkt+ILmU5yNK6wWQSr1aAokYaR84pkmrxCKfVelm29XoovIi7gN8CXlVKeLPvvJPF/UVVVNQBL84jWg7pMtCZFf5QrhoUcKFI81rJ52K/ZEx1VLTQaTd/ob4pFv6eoB4t0uTdDhLlVYxAR7BaDQruBxTBQCk4uK8qPimSa4wYRsWE6x08rpV7M1kYptUIpNU8pNa+srGx4DRwqSqaYUmzpZJFm02g0Go1mpNOvaO9gTVEPhGTlvCQzKtzElWLDgRbsFgMEpoxx8rFEURCdf6wZDEREMAvhbFdKfT/X9gwrp1xp5hyDGTkOekxptrmfz61dGo1Go9EMMv1OhxjoFPVAmTTGSWsgkioUAjBxTCE3TywZPvF5zWjkE8DngS1p+fZfV0r9Xw5tGh76U3Zak5eIyKeBHwIW4HGl1PdybJJGo9HkFcOSLzwUdCX3tnj+5BxbpjkASoo4AAAgAElEQVSeUUqtIVMkZXTR17LTmrxDRCyYqkOXYMpyrhOR348aqUKNRqPpBQMtFJIzupJ706kUGo1G0y1nA7uVUh8ppcLAKuDqHNuk0Wg0ecWIjSBDu9ybRqPRaHrNJCBdeuQQ8PEc2aLRaDR5ibSvr8tfRKQR2D/ElykFjg7xNfpKPtoEw2PXVKXUcSH/MIr7L+SnXaO6/4rIDcClSqkvJN5/HjhbKbUkrU1KphCYCdQMsVn52E+S5KttQ2lX3vZfjWa4GBER5OH4RxWR9UqpeUN9nb6QjzZB/tqVr4zW/gv5aVc+2jTMHALStfkmA3XpDZRSKxjiqqjp5PPfJF9ty1e7NJrjhRGbg6zRaDSafrEOOFlEpouIHbgR+H2ObdJoNJq8YkREkDUajUYzOCiloiLyz8BrmDJvTyiltuXYLI1Go8krtIPczrBNJ/aBfLQJ8teu0Uy+/k3y0a58tGlYSeh255N2dz7/TfLVtny1S6M5LhgRi/Q0Go1Go9FoNJrhQucgazQajUaj0Wg0aYxqB1lEpojIWyKyXUS2ici9ubYpHRGxiMgHIvJKrm1JIiJjROQFEdmR+N7OzbVNo5l87sO6/2p6Ip/7L+RnHwbdjzWa4WC05yBHga8qpTaKSDGwQUTeyKOSq/cC2wF3rg1J44fAq0qp6xMr4AtzbdAoJ5/7sO6/mp7I5/4L+dmHQfdjjWbIGdURZKXUYaXUxsRrL+ZAOCm3VpmIyGTgcuDxXNuSRETcwAXALwCUUmGlVEturRrd5Gsf1v1X0xvytf9CfvZh0P1YoxkuRrWDnI6ITAPOBP6aW0tSLAf+FYjn2pA0TgAagScT046Pi0hRro3SmORZH9b9V9Mn8qz/Qn72YdD9WKMZFrSDDIiIC/gN8GWllCcP7LkCaFBKbci1LR2wAnOBnyqlzgT8wNdya5IG8qsP6/6r6Sv51H8T9uRrHwbdjzWaYWHUO8giYsMcmJ9WSr2Ya3sSfAK4SkT2AauAi0TkV7k1CTBL1B5SSiUjPC9gDtSaHJKHfVj3X02vycP+C/nbh0H3Y41mWBjVDrKICGYe13al1PdzbU8SpdT9SqnJSqlpmGVg31RK/X2OzUIpdQQ4KCIzE5sWAvmymGZUko99WPdfTW/Jx/4L+duHQfdjjWa4GO0qFp8APg9sEZFNiW1fT1SZ0mRnCfB0YuX0R8DtObZntKP7cN/Q/Te/0P23f+h+rNEMMbqSnkaj0Wg0Go1Gk8aoTrHQaDQajUaj0Wg6oh1kjUaj0Wg0Go0mDe0gazQajUaj0Wg0aWgHWaPRaDQajUajSUM7yBqNRqPRaDQaTRqj2kEWkZiIbBKRrSLyvIgU9tD+67087z4RKU289g2Grd1c6zYRqcx27Q7tlolIrYh8cxCu+UkR+ZuIbB3ouTT9R/fffl9T9988QPfffl9T91+NZhgY1Q4yEFBKfUwpNRsIA3f10L5XA/QwcxtQ2VOjBD9QSj040Asqpf4MXDbQ82gGjO6//UD337xB999+oPuvRjM8jHYHOZ0/AycBiMjfi8j7iejGz0XEIiLfA5yJbU8n2r0kIhtEZJuI3NnbC4lImYj8RkTWJR6fSGxfJiJPiMhqEflIRO5JO+bfRGSHiLwhIr8WkftE5HpgHqZg/CYRcSaaLxGRjSKyRURmdWGDS0SeTLTZLCLXJbb7ROQ/E5/rTyJydpo9V/Xje9UMD7r/ovvvCEb3X3T/1WjyCqXUqH0AvsSzFfgd8I/AKcDLgC2x7zHglvT2acePSzw7ga3A+MT7fUBptmMS254Bzk+8rsIsswqwDHgXcAClwDHAhjkIb0pcpxjYBdyXOGY1MC/t3PuAJYnXdwOPp537vrR2/wksT3s/NvGsgM8kXv8WeD1hwxnAprT204Ctuf4bjuaH7r+6/47kh+6/uv/qh37k82O0l5p2Snt50z8DvwDuBM4C1okImINiQxfH3yMi1yZeTwFOxhxUe+Ji4NTE+QHcIlKceP0HpVQICIlIA1AOnA/8TikVABCRl3s4/4uJ5w3AZ7ux4cbkG6VUc+JlGHg18XoLEFJKRURkC+agrMkfdP9NoPvviET33wS6/2o0+cdod5ADSqmPpW8Qc9RcqZS6v7sDRWQB5iB3rlKqTURWAwW9vK6ROC7Q4ZwAobRNMcy/kdA3kudIHp8NwYxWdCSilEpujyfPpZSKi8ho7y/5hu6/ndH9d+Sg+29ndP/VaPIEnYPcmWrgehGZACAi40RkamJfRERsidclQHNicJ4FnNOHa7wO/HPyjYh8rJu2AGuAK0WkQERcwOVp+7yY0359paMNY/txDk3+ofuvZiSj+69Go8kLtIPcAaXU34AHgNdFZDPwBjAxsXsFsDmxSORVwJpo8y3gvS5OWSgih9IeS4F7gHmJxRl/o4fV20qpdcDvgQ8xp+/WA62J3U8BP+uwSKQ3fBsYK6bE0ofAhX04VpOn6P6rGcno/qvRaPIFaZ/N0eQzIuJSSvnE1Ap9B7hTKbWxD8cvw1yw8sgg2TMNeEWZEk0aTbfo/qsZyej+q9GMPnQEeeSwIrGgZSPwm74Mzgl8wJ0ySEL1mCvNjw70XJpRg+6/mpGM7r8azShDR5A1Go1Go9FoNJo0dARZo9FoNBqNRqNJQzvIGo1Go9FoNBpNGtpB1mg0Go1Go9Fo0tAOskaj0Wg0Go1Gk4Z2kDUajUaj0Wg0mjS0g6zRaDQajUaj0aShHWSNRqPRaDQajSYN7SBrNBqNRqPRaDRpaAdZo9FoNBqNRqNJQzvIGo1Go9FoNBpNGtpB1mg0Go1Go9Fo0tAOskaj0Wg0Go1Gk4Z2kDUajUaj0Wg0mjSsuTagN5SWlqpp06bl2gzNMLJhw4ajSqmyXNsxGOj+O/rQ/VczktH9VzPSGYw+nBMHWUS+AnwBUMAW4HalVLCr9tOmTWP9+vXDZZ4mDxCR/bm2YbDQ/Xf0ofuvZiSj+69mpDMYfXjYHWQRmQTcA5yqlAqIyHPAjcBTw22LRjPS2H64lVe31lPbEmDSGCefnl3OKRNLcm2WRqPRaDTHFblKsbACThGJAIVAXY7s0GhGDNsPt7Linb2UOG1MLCmgNRBhxTt7ufOC6dpJ1mhGOTVNNVQfqKbOV0elq5KFVQuZOW5mrs3SaEYsw+4gK6VqReQR4AAQAF5XSr3esZ2I3AncCVBVVTW8Ro5y9ECbn7y6tZ4Sp40Spw0g9fzq1nrtIGs0o5iaphpWbluJ2+6moqgCT8jDym0rufW0W/XY3Qd+8MZOAL5yyYwcW6LJB3KRYjEWuBqYDrQAz4vI3yulfpXeTim1AlgBMG/ePDXcdo5W9ECbv9S2BJhYUpCxrbjASm1LIEcWaTSafKD6QDVuuxu3ww2Qeq4+UK3H7TwiEolw6NAhgsEul1xp+khBQQGTJ0/GZrMN+rlzkWJxMbBXKdUIICIvAucBv+r2KM2woAfa/GXSGCetgUgqcgzgDUaZNMaZQ6s0Gs1g0tsZvPR2O5p2cHrp6Rn7XXYXdT6dvZhPHDp0iOLiYqZNm4aI5NqcEY9SimPHjnHo0CGmT58+6OfPhQ7yAeAcESkUs4csBLbnwA5NFup8dbjsroxteqDNDz49u5zWQITWQIS4UqnXn55dnmvTNBrNIJCcwfOEPBkzeDVNNd22sxt2NhzZwNHA0VQbX9hHpatyuD+CphuCwSDjx4/XzvEgISKMHz9+yCLy/Yogi8jSXjTzK6V+3nGjUuqvIvICsBGIAh+QSKXQ5J5KVyX7PftpaGvAG/ZSbC9mQuEEprqnZrTLFuXY17qPVTWrqG+rp7ywnBtn3sil0y/N0Sc5/jhlYgl3XjA9Q8Vi8fzJOv9YozlO6GoG79maZyl1lqbG26OBoxntZo2bxfr69exo2sF5lefhC/vwhD1ce/K1Ofssmuxo53hwGcrvs78pFv8C/BTozrK7gE4OMoBS6iHgoX5eWzOEnFhyIi/veZkiaxFFtiK8IS9H/Ee4aMpFqTbZ8pT/473/oNZfyxj7GMoKyvCGvCzfuBxAO8m9oLfybadMLNEOsUZznFLnq6OiqCJjWygW4q+H/8oFky9Ijbfv1b3Hxyd+PNWmtLCUs8rPYkvjFo74j1DpquTak6/VaXEazQDor4P8S6XUN7trICJF/Ty3Jofsad3DnNI5NAQSEWRHMSeOOZE9rXtSbbJFOQ75DhGLxyh2FAOknlfVrNIOcg9o+TZNXxGRzb1o1qiUWjjkxmgGjUpXJZ6QJzW2AtQ01zDGMSZjvB3jGENNcw1lhe2FwhwWBxdNvYi7P3b3sNut6R9J1YzBojfqG9/5znd45plnsFgsGIbBz3/+cz7+8Y9nbfvUU0+xaNEiKitHZ6pOvxxkpdS/DkYbTf5R56tjinsKU0vaUyriKs6Oph08tumxLheEBGNBLFgythXZiqhvqx8Wu/uCiMztRbOIUmrLkBuDlm/T9AsLcFk3+wX4/TDZohkkFlYt5Ecbf0RzYzOheAiH4aC+rZ4LJl+Q0W7muJm8d/g9PCEPLrtLp1RoesXatWt55ZVX2LhxIw6Hg6NHjxIOh7ts/9RTTzF79mztIPcHERkD3AJMSz+XUuqegZmlyRXZIhgHvQep9dZSWVRJRVEFH7V8xIYjG5g3cR6lzlIACiwFxOKxjHP5I37KC/NyAdnbwDq6TxGajtmvhxwt36bpB19SSnVbSlVEdCgxz+iNQoWImCOTAgScVidtkbaMNg6Lg3Mnnovb4U6dS6dUaHri8OHDlJaW4nA4ACgtNX+/N2zYwNKlS/H5fJSWlvLUU0/xl7/8hfXr13PzzTfjdDpZu3Yt7777Lvfddx/RaJT58+fz05/+FIfDwde+9jV+//vfY7VaWbRoEY888ggvv/wy3/72twmHw4wfP56nn36a8vK89Ae6ZKAyb/8HvAdsAeIDN0eTaxZWLWTltpUAqcjE7ubdnDTmpG4XhEx2TabWX4s35KXIVoQ/4scf9fPFOV/M5cfpinVKqYu6ayAibw6lAek5xwea2ohEY0wrbVcP0fJtmu5QSq0ZjDaa4aM3GvPVB6qZ7JrMqeNPTR2337Of3c27GVswNiNarLXpNX1l0aJFfPOb32TGjBlcfPHFLF68mPPOO48lS5bwu9/9jrKyMp599lm+8Y1v8MQTT/CTn/yERx55hHnz5hEMBrntttuorq5mxowZ3HLLLfz0pz/llltu4be//S07duxARGhpaQHg/PPP57333kNEePzxx/mv//ov/vu//zvH30DfGKiDXKCU6o2ihWaEMHPcTG497daMKEelq5Ip7impNtkWhHz9nK93UrH44pwv5mX+cU/OcW/b9JeOOcfhSIyNB8xBpWp8Ed5glNZAhMXzJw+VCZrjBBG5AvgWMBVzPBdAKaXc3R6oGXZ6ozGfbZHelOIpBKIBHS3WDBiXy8WGDRv485//zFtvvcXixYt54IEH2Lp1K5dccgkAsViMiRMndjq2pqaG6dOnM2OGmed866238uijj/LP//zPFBQU8IUvfIHLL7+cK664AjA1nxcvXszhw4cJh8NDolM81AzUQf6liHwReAUIJTcqpZoGeF7NMNHVlF/64PvYpsc6pV1kWxAyc9zMvHSIu0NE5tA5RejFobxmx5zj6WVm5PiwJ4TNatHybZq+sBz4LLBFKaUrjuYx2Zxfl93F9mPbU+s7an21ROKRDFlNX9jHrHGzelx815v0jd4WIdEcv1gsFhYsWMCCBQs4/fTTefTRRznttNNYu3Ztt8d1NbxYrVbef/99qqurWbVqFT/5yU948803WbJkCUuXLuWqq65i9erVLFu2bAg+zdAy0EIhYeBhYC2wIfFYP1CjNMNDb0XpF1YtxBP24Al5iKs4npAHT9jDwqqRvUBeRJ4AngCuA65MPK7o5bEWEflARF7p63VrWwIUF2TemxYVWPAFo309lUZzENjaV+dYRMaIyAsiskNEtovIuUNknyZBpauSg56DrDuyjjcPvMm6I+vY1riNOl9dagye4JzA5sbN7Pfs79NY25uxvKaphh9/8GPW1K5h+7HtrKldw48/+HGn8V5z/FJTU8OuXbtS7zdt2sQpp5xCY2NjykGORCJs27YNgOLiYrxeLwCzZs1i37597N69G4Bf/vKXfOpTn8Ln89Ha2spll13G8uXL2bRpEwCtra1MmjQJgJUrVw7bZxxMBhpBXgqcpJQ62mNLTd7R27LS2dIujpMpvnOUUqf23Cwr92JWgOzzVHbHktFHfUH++lEzxQVWLfOm6Sv/CvyfiLxN5ize93s47ofAq0qp60XEDhQOoY0asmvM1/hrOKPsjNTYm1QPamhrwGbYej3W9mYsf7bmWQ60HsBld1FsLyYcC3Og9QDP1jzLg+c+OFQfW9MNvZFlG0x8Ph9LliyhpaUFq9XKSSedxIoVK7jzzju55557aG1tJRqN8uUvf5nTTjuN2267jbvuuiu1SO/JJ5/khhtuSC3Su+uuu2hqauLqq68mGAyilOIHP/gBAMuWLeOGG25g0qRJnHPOOezdu3dYP+tgMFAHeRvQ1mMrTd6QPsWWTa6tq7LSHdMujhPWisipSqm/9eUgEZkMXA58B/MmsU98enY5K94xB4viAitbaz0IcFqlG0NEy7xp+sJ3AB9QANh7c4CIuIELgNsAlFJhzNlAzRCyp3UPc8rmtFcpdRRTEi0hEM1Uq5ninoLNYuPb5387ta2n1Ig6Xx02w8a6I+tSFVCnuadljOWbGzdTZCvCYTUVDBxWB0opNjf2RlJbczxw1lln8e6773baXlpayjvvvNNp+3XXXcd1112Xer9w4UI++OCDjDYTJ07k/fff73Ts1VdfzdVXXz0IVueOgTrIMWCTiLxFZvRCy7zlIR1XUWeTa/OFfVS6Ro3m4UpMJ/kIZv9NLnCa08NxyzEjd8VdNRCRO4E7AaqqqjL2dSwZHY7FmT99LGXF7VJvWuZN00vGKaUW9fGYE4BG4EkROQMzNe5epZQ/2aC7/qvJTm+c2EJrZqC+2F5MUzBzyU7HMbgr9YtPTf4Ue1r3UOerY1fzLlpDrYxzjsNlcxGKhdhQv4F55fPaT6zonFQpaP0pjaYLBuogv5R4aPKAngbo6gPVxOIxappr8Ia9WMRCOB7OkGsbarH5PFsk8gTwefogU5hQDWhQSm0QkQVdtVNKrQBWAMybN69Tfmh6yegfvLGT1kAkY39XMm9/2FzLyrUHqPcEKXcXcOu5VVw+Z1JvTNccn/xJRBYppV7vwzFWYC6wRCn1VxH5IfA14N+SDXrqv5pMeiPhZrfYWX94PS67y3RioyFaA63YrLZuC35kS59oDjazYvMK5lfMp6Kogm1Ht9ESasFpc2K3201nOKGjnGRO2RzW169HRLAbdsLxMP6IP9OJ1mg0KQbqIL8ABJVSMTAXLgGOAVul6TO9GaC3H9tOra+WAmsBLpuLcDyMIQaekCcl1zaUucW9sXGYOaCU6mu1sU8AV4nIZZjT2m4R+ZVS6u/7a0THlIuuZN7+sLmW7/2xhiKHlQkuO55AhO/90Vxgo53kUcs/Af8qIiEgQu9k3g4Bh5RSf028fwHTQdb0k16t50g6rEmnVcBhc3BSyUndSrhlS5/wh/1E4pHUdSyGhbGOsRzxH6GhrYEiWxGnjTuNcKw9c2bxrMXsadnDId8hgrEgBZYCJrsms3jW4qH+ejSaEclAHeRq4GLMHDgAJ/A6cN4Az6vpI70ZoH0RHwYGDksiB83ioNBaSLGjOCPfLRuDEfnt7aLAYWSHiDwDvExmilCXMm9KqfuB+wESEeT7BuIcQ+eUi65k3lauPUCRw5pWktpIbdcO8uhEKdVlmk83xxwRkYMiMlMpVQMsBPqUhz/a6E36RDYJt/Qc4HA8zIklJ7KtaRv+iD/lxBbaC7uVcLMbdv5S+xeiKkpMxWgONuMNeykvLE85zccCxwhGg1gNKyeOOZFwLMyelj2UF5Vz+6u3U99WT7GtGIWivKicUDSEw+qg2NHn7qPRjBoGo1BI0jlGKeUTEb0aOgf0ZoAuthfTGmwlFA1ht9gJx8LEVZxie/eD5GBFfntj4zDjxHSM03M4FTCkOsi9pWM6xe56DxXuAg41txGKxnFYDUoKrNR7grk2Na8YTXqwInIt8KZSqjXxfgywQCnVU+rbEuDphILFR8DtQ2vpyKU341+lq7KTVnzHXGK7xc5f6v9CW7yNmIoRjUfZVL+JC6dd2O31W0ItNIeaUSiUUogIoViIhrYGih3FuGwu6v31hGIhbIZ584yAP+rnw8YPqSquoqygjP2e/QRjQc6rPI+Txp4EgCfkyWWAQqPJawaqg+wXkbnJNyJyFqBXFuWASlclvrAvY1vHAXrWuFmcPO5kHFYHvogPh9XByeNOZta4Wd2eOz3ya4iB2+HGbXdTfaB60G0cTpRSt2d53NGH41crpXqlm9wdycp6rYFISubtod9t45sv/w1PIJJKp/AEouw95icai2O3CNFYnEMtQVyOgd7nHj/0Vg+2N/rfI4SHks4xgFKqBXiop4OUUpuUUvOUUnOUUtcopZqH1MoRTG/Gv95oxR9sPUhzpJlYPIYFC7F4jOZIMwdbD3Z7/V0tu7CIBUMMRARDDHP9SCxs3s4DiOmAWwyLObZbHMTiMayGlWJHMYZhmCkdhoOtx7amzp3jAIVGk9cM9Jf1y8DzIpL8D5sI6ISmwebIVtj+MrQehJIpcMqVUDE7o8nCqoWs3GaKcXe12CPZZubYmRltehKhH6zIb29sHE5EZCXm6v2WxPuxwH/3xUkeDDpW1itx2th3zE8sTkY6RYHdwB+KU2gHiwHROMTjiqnjOi/mG630Jo0nD1N9BkK2IIe+YxpEejP+9UYrflfLLopsRcRVnFg8hs1iwyEOdrXsoju8YS8GpoOrUBhigAJDDBxWB96wF6fVicvqIhBrj0+1RdsosbenadktdiJE8EdSYiWjTbUo/3jru4N7vgvv73b3ggULuP/++7n00vaKt8uXL2fnzp089thj/b7sgw8+yAUXXMDFF1/cp+NWr17NI488wiuv9Lne1rAwoIFUKbVORGYBMzGXHuxQSkV6OEzTF45shXd/DAVjwD0JAi3m+/OWZDjJvRmg+1vwozfTh70hDwuOzEk6xwBKqWYROXO4jahtCTCxpCBjWzASw26xZGwrsFqJxSI4bAb+UJQih5Wzp42h0GEbTnPzmt44M3mY6jMQ1ovI94FHMeOJSzBl2zSDRG/Hv5604iMqgl3s2Gzt/6+RaIRgPJgqNZ0t3ccmNjwxDxbDggULURUlThybYWN+xXwAdjfvZk3tGhSKeDzO0bajROOZlTnHOsZyyHsIh9VBXMVzHqDQDD833XQTq1atynCQV61axcMPP9zjsUqZKT6G0fme/Jvf/Oag2tkV0WgUq3X47v/7dSURmauU2giQcIi3dtdGMwC2v2w6x84x5vvk87pfgGtCRlR5ZsXsHp3N/hT8WFi1kB9t/BHNjc2E4iEchoOxBWO55+RMuevX9r7GqppV1LfVU15Yzo0zb+TS6ZdmtMmzgiOGiIxNTi+LyDhyEH2bNMbJhwea2NXoxxeKmikTqn32NIkCnHYrU8YW4glGcBfYsFstWeXgRiu9cWYG64YvT1iCKc/2bOL968ADuTPn+GOwZr4muyZz0HsQI26mSMRUjLZYGzbD1indJz2/eZxzHN6IF4UipmIYYmA37Ngt9pQ83F7PXhQKp8WJGIJFLGa1vogXb8hLka2IWDxGgbWAMyacMSyqRQNFRKYA/wtUYMpwrlBK/TAxTj8LTAP2AX+nU4R6x/XXX88DDzxAKBTC4XCwb98+6urqOP/883n44Yd57rnnCIVCXHvttfz7v/87+/bt4zOf+QwXXngha9eu5aWXXuKhhx5i/XpTLvCOO+7gK1/5CrfddhtXXHEF119/PevWrePee+/F7/fjcDiorq7GZrPxj//4j6xfvx6r1cr3v/99LrwwM/e+qamJO+64g48++ojCwkJWrFjBnDlzWLZsGXV1dezbt4/S0lKeeeaZYfu++usMPJlYwS/dtPkFMOzRuOOO1oNm5DidaBD2vg0nX9ptVHkwEZGEgBQgifdpvLb3NZZvXE6RtYiygjK8IS/LNy4H6OQk5xH/DbwrIi9gfrK/w6xMNqwU2oX39zdjt1gotBn4QzGC0TigaA1EKHZY8IZiqHgcMYzUttZAhLqWAItOKx9uk/OWvqQadddmpJAo7qEl2oaQwZr5umP2HTy8/mGisSgRFTFzihFOH396J43j773/PSa5JlHpqqTQVsi4gnGEYiGiKopVrDgsDsqLylPycM3BZkoLSokRIxwLYzEsjLeNJxANUOwoTgUtvjjni/k8HnckCnxVKbVRRIqBDSLyBmYFyGql1PdE5GuY/f//5dDOEcP48eM5++yzefXVV7n66qtZtWoVixcv5o033mDXrl28//77KKW46qqreOedd6iqqqKmpoYnn3ySxx57jA0bNlBbW8vWrWZMtKWlJeP84XCYxYsX8+yzzzJ//nw8Hg9Op5Mf/vCHAGzZsoUdO3awaNEidu7cmXHsQw89xJlnnslLL73Em2++yS233MKmTZsA2LBhA2vWrMHpHN5gUH8d5BLMabzuHOTGfp5bk07JFDi6G3yHIeiBAjf4m8A5rnNUefvLQ+IgVx+opshWhD/iJxwL47A4KLIVZeRsrqpZhQULnoiHo8Gj2C127GJnVc2qvB2QlVL/KyLrgYsw+/Jn+1p2ejCo3nGU8uICQrE44WicQoeFsYU2LIbgdtpSKhYnlBZiNQyOeEP4glHcThszJrjYWe/n8uE2Ok8ZylSjfEJElimllg20jaZ3dJz5qmmq6ZwWEY50u1bk0umXUuer49c1v6Y11EqJo4QJhRMosBbw0u6X8Ef8pgqFgpiK4Ql5+Kj1IzwhD1PdUwnGgt5N7dwAACAASURBVCkd5AmFE5jqnpqSh7vixSs4FjxGXMWJqziBaACvmG3nV8xP2TmtZNpwf3X9Ril1GDiceO0Vke3AJOBqYEGi2UpgNdpB7jXJNIukg/zEE0/wzDPP8Prrr3PmmWZM0+fzsWvXLqqqqpg6dSrnnHMOACeccAIfffQRS5Ys4fLLL2fRoswinjU1NUycOJH5883UH7fbvPFbs2YNS5YsAWDWrFlMnTq1k4O8Zs0afvOb3wBw0UUXcezYMVpbzfXHV1111bA7x9BPB1kpNW2Q7dB0Rdkp8OEqsLvAUQyBVvAcgBM6LKwrcJsD8xCwo2kHhzyHUgVGQtEQu5p2EYi2Lwg56DlIJB7BYliwGTai8SiheIhgnkuQJRzinGrA1nuCWEThDUQIx+LYLQYFxXYicXjuS+em2t33/IdMLClgepkrtS2ulC5J3YHepPH0N9Unj+ThviAinm72C3AjsGx4zBk9ZJV92/BDbm1uZaZrcpezejVNNexq2cVFUy5KzVxUH6hmb+teimxFFFoKaQo1EY6HKbYVU2wvJhwLE4lHOOg9yPmTzu9ycbUhRkoH2YKZvhGKhYjH492mb4wURGQa5oz0X4HyhPOMUuqwiEzIoWkjjmuuuYalS5eyceNGAoEAc+fO5emnn+b+++/nS1/6Ukbbffv2UVRUlHo/duxYPvzwQ1577TUeffRRnnvuOZ544onU/qQMYUeU6rkYZ7Y2yXOl2zCcDFTmDRE5T0Q+JyK3JB+DYZgmQeN2mDwfnCUQ9pnPJVXgqc1sF/SYUYshwBv2plZMiwgOqwNDDLxhb6qNYRjEVRyrYUVEsBpW4iqeNaE/14hIj7nxvWkzWNgM4UBTgGhcYTMMonHFgaYANiNzoJk0xok3mLnwpquS1JrBJ8/k4f4HKO7m4Uq00Qwy1QeqiakYNc01rD64mprmGmLNB6imzZzNE8N8LhhjrhV567vw0t1Uv70MdzSSIRcXipppE4FYgNZwK5G4ucY9EA2wz7OPo8GjZl6xCG6HmyP+I7gd7k5OrkLhMMxxOU7cTN9QgsWwDFieM9eIiAv4DfBlpVR3N4Xpx9wpIutFZH1jo57MTsflcrFgwQLuuOMObrrpJgAuvfRSnnjiCXw+U4a1traWhoaGTscePXqUeDzOddddx7e+9S02bsz8mZw1axZ1dXWsW7cOAK/XSzQa5YILLuDpp58GYOfOnRw4cICZMzNv0tLbrF69mtLS0lQEOlcMaEGSiPwSOBHYBMQSmxVmYr1mMGg9CPYOd0/jZ0DtOtj1BsRCYHFAUSnMfTCzXRZ5uBq7rc8RMJfNRUNbA02eJlPD07DgtDqZaJuYajPJNYltjdsIRUOpXOWYilFoK+SBNQ+krgXkQwTuFBHZ3M1+wUwjGhaUUkgiW0klluYJ0umOurclqTVDQz7Jwyml/n1YL6hJsaNpB4catlEQbMYVixCy2NhFnICzw1qADmtF6hr24j9Yy5v16/CrCEW2InwRXypKppRK/f9HVZS2SJuZKoGX8c7x3VbbcxgOKlwVtEXbCMfC2C12vHixWTIVbkaaWouI2DCd46fTKpzWi8jERPR4ItDJk1NKrQBWAMybN6/n8GWu6EGWbai46aab+OxnP8uqVasAWLRoEdu3b+fcc80ZS5fLxa9+9SssHZSUamtruf3224nH4wB897uZMnV2u51nn32WJUuWEAgEcDqd/OlPf+Luu+/mrrvu4vTTT8dqtfLUU0/hcDgyjl22bBm33347c+bMobCwkJUrVw7Vx+81A12xPw84VfUmfq7pHxYH7PszONxmikUkCM0bQRIdN/XNd5jWyCIPV/POd1k5tgS3u6pPU25lzjJ2Nu1EEEQEQYjGopQ5y1Jtprunc8BzAF/ERyQWSYnaF1oKU9f60cYfISJMdk3O9ZRf95VRTGI9Nxkcogomjy2g0RcmElPYLEKF20G0w39Vb0tSa4aGfJSHE5EZwH2YK/pT47lS6qJc2XRckhZs8Ho2YoS9OAwrWOw44lHCsTBe6SCkcGRLxlqRNhHeUc3EgxAXaAt7iAJWrJQ4zP/hen89cUznQxDiKk4kbo6n3TGnbA7r69dT6izFbtgJx8MEogHG2cdltBtJai1i3jn8AtiulPp+2q7fA7cC30s8/y4H5o1orr322k4BmHvvvZd77723U9vkgjyAM844o1PUGOCpp55KvZ4/fz7vvfdet22SLFiwgAULFgAwbtw4fve7zn/KZcuWdfEphp6BOshbMSVYDg+CLZok6ZHfI1sh5DXTK2JRsFgh7Ded3pMvaT8m0JK5SC+LPFy1byduTxB3mdmm1xEwAYfFwXj7+FSJal/Yl+mTCxRZiygvLMdusXPQe5BANIDdak9N7zU3NoPAqeNP7dv1Bxml1P6U2SIWoJxM5+LAsBkDlLsL+KjBQzgSJ6oUKi54gxFKnHb+7udrU4v0bj23isvnTOrRId5+uDXDif707HLtRA8CeSoP9zzwM+BxhvGmblTRIdjgql9Nq6EIAXYUYcNCXFlxBb3mOFzgNlPeAk0w/VOp02wL1hOJg4iZ2xhPbI+qKOFoOCPaK0iqKIjFYklFlrti8azF1LfV0xxsxhvx4jAcnFhyIkW2opQU3AhUa/kE8Hlgi4hsSmz7OqZj/JyI/ANwALghR/ZpjnP6q4P8Mmbsshj4m4i8D4SS+5VSVw2OeaOQjpHfvWsg6AWbEwwxv3UVh2g487iOi/SyyMPVSYyKcChjW28iYOFYmLMqzmKfZ19qFfXMcTPNUqddtInFY0xyTSKm2n+zQ/FQJ3HfXEbgRGQJZlneetp/rxQwZzjtmOi28f7eKAJYBCJxxVF/FH84hs1qSZWa/t4fzVzXy+dM6vJcybLVJU5bqmz1inf2cucF07WTPEDyVB4uqpT6aS4NGPH0VKl0+8vgPwb7/wIhL6cUKAptDhqsCq/FoBiDKdiYGo1A/RbwHobiiVAxB6ztBYCOxsNYUmqZggWIJwZEu9WOP+I3NY7FjtViSrnZLXYKrYVYje5/qmeOm8k9c+/plL4G5FytRUQ+oZT6S0/bOqKUWkPXSlndl3/VaAaB/kaQHxlUKzTtdIz8EgerA+xOGFNlbmrcCaFWM/UiKf1WXAnjT2w/T8kUM5qROg9UKgv7LTEajqxrlwtyTmBqydQMEzoW/BjrGMuEwgmpqk0A+z37aWhrSOUX2y127IY91WbdkXV4Q16KHcWpYxyGo9Nwl+MI3L3ATKXUsVwZALBufyuFdoNITBGPK6wWIRpTRKIqo9Q0wMq1B7p1kLOVrU5u1w7ywMgnebhEsQSAl0XkbuC3ZAYpmobdqJFItkql1d8Cd6W5vqNkCux8DRprAAUiLPRHWFliYWYojAsLPsOCxwixMBCF6afD1E+Y43LzXmjeB0yDArd5AwzYxAKpNQdR4pBStqjeX81h/2EqnBWUFJSkZuvmlPd8z96VMkseKFb8GJjbi20aTV7RX5m3twFE5D+VUhn6gyLyn8Dbg2Db6KRj5NdaAOE286EURENgsUOwyZR8S0q/tdbCrCvajzvlSnPgh9SU34kx4WVLkKJEZSVvyMsR/xEuqmpPV8xW8OOA5wBTiqcwa9wsXHYXB70H2dy4mTmlc1K5xPX+ejO/mMm47C4mFE7giP8IJ445MVXWdGzBWEQkn6b8DgKtubp4ktZAhHA4TlKfIhZLLtSBQ81thKJxHFaDMU4rHzX6sqZdJMlWtrq4wDoypOB6iuQNMdkk3KDzotLuFksNIxtIle0B4F/S9inghGG3aCTSMSARC0PTHvA3wkkLTYe5YRvEFdgcgDAzHOTWVi/VRU7q7A4qIxGubfUxs6Csgzb9dPN8zjHQepBKLByUGAYKA0nlGleIM1Xw48zyMylpKSEcD+MNe3FYHVSVVLF45uKcfD0DQUTOBc4DykRkadouN+a9gkaT1ww0B/kSOgt0fybLNk1v6Rj5dU0wUypiYTMXucBtPhxuU/It6DGfy2aZknBcYx5XMdvU4ExzOPa45zAnHqChrcGMIDuKOXHsiexp3ZO6/KqaVRRZi1KR3+RzIBpIDeINbQ3MKZ2Tijy7HW6mMIVwLJxqM9U9lYumXMSe1j0p5yJZmjrXEbi0wfojYLWI/IHM6Nv3sx44RIQjMaJd7IvGFHaLQTSm2NPoJxYHTyDSZdrFpDFOWgORVOQYRogUXLZI3hBXh0wnm7btjz/4MUopphRPyfWi0k4opaYDiEiBUipDbFxECrIfpelEx4DE0RoQq7m95o/mWBvvmNqtmBmJMLMlAiRVxwQsbZmzeuNPpibcTPWUk6mz+TnJcRbHGtYTRhEmhgVwY2Hp6Xdy6ce+kDp7HmltDxQ7ptygFTMdM4kHuD4nFmk0faC/Ocj/CNwNnNBBLqsY6DavSNMDp1wJ1d+EQ0fNKb5YDCIBc9pu3HRz8N31GpSfDoG0zACHq3OhkIrZGc5F3ZoHKLQWZjQJR8O8eeDN1GB8yHuIyqLMlIciWxF1/vY8YW/Iyxj7GDONIpGqMc09jaOBo5QWlqbaTSuZlrWKXh4M9snB+kDiYU88oFOW9NATiXe9LxZXGGI+h2MKp9XoNu1ixErBZVlUmto+DA5yNgm3pqNNoOC00tNS25Jt86APJ3mXzlPV2bZpstGxUqnnEISDYFjAdwT8R82ZO+LmLJ65CCTLiRQEW0yVoYTaUM3Bd1g5rhR3QjfbZ3NxUqiVYFsD3miAclsxN876HJee+YWMM/W3iE2+kZhpfltEnkpfGK3RjBT6G0F+Bvgj8F3MOuhJvDr3bTBIzJoqwF4IzrHmoOupNQf0ijnQ8LdM6bf9f4Fpn+z2rHbDzvr69bjsLlw2Fy3BFrYf287EwompCFk4FuZY8Bhlhe0SbseCxwjHwqkCCduObmPtkbVMLJxIiaOEUCzE2rq1WMTCJNekvIu2dSSpISsiNyilnk/fJyJ5tSLabjPwh2IUOSw4LILDlin3VOywUJ9WrXDESsFlWVQ6lNUhO5JNwi2l6Z1GrmXdkohIBWbZXaeInEm7pW6gsMsDNZl0rFQabjNn6yxuU2JTdYgeG1ZIFPPoTKbjXG2N48aacdN1ysR5uB3ufEnTGS4cIrICLUXYicc2PTao5+upXy1YsID777+fSy9tD1wtX76czZs34/F4eOGFF/p0vS984QssXbqUU089tcs2P/vZzygsLOSWW0ZeDbn+5iC3Aq0i8k8d94mITSnV1Qii6YntL5uDddhrDtS2AnBNNFMtkqLiL3+FmniQ6mgbdXFFZVxYGDeY2XbMrNrUVQ6nmIvi6tvqicajxFUcUUJLuIXVB1dTbC+mqriKPa17KLAUUGQrwh/x0xRs4oyyM1IDvd1ix8DAE/aY+p0KfBEfZYVlGT8GzcFmvvf+95jkmpRvhUKS3I8pk9XTtiElXfKpIzec1V4dcdX7B4jFM3+Ej/rDRGJx7nv+wwxJt7x3iDtSMgWO7QFvXdcLTwdAT9PW2STcHFZHp2DhtqPb2NO6h8tevIzywnJunHlj1lmSYeBS4DZgMpCeEuTFlMLS9IZkpdJkv0veZ4S9EPGZVfGS/52GJUu6RQdsBan+W1cwgYoOd1j5coM1zGgpwjzhpptuYtWqVRkO8qpVq3j44Yf55Cc7B9ii0ShWa9du4uOPP97jNe+6667+GZsHDLQO8EagEdgJ7Eq83isiG0XkrIEaNyo5vBka/5YxVUfj38ztCWpa97Ky0MBjQEVM4TFgZYGi5vA6M3czPYfzSLvI997WvfgiPpQy9TWjKkqECG3RNlw2F6FYCH/Uz5TiKRQ7imkMNlLsMCXdktPMYFbIS0q4+SI+HFYHYxxjMsTsj7YdZVfLLo4FjmXkdP5o449yXqpXRD4jIj8GJonIj9IeT0GX6cBDxplTspfTLHEYtAYixJWiNRBhyjgnVouY2+JxGrxB6j1BTi4rypB023445+sO+07ZKXBonbng1O4ynw+tM7cPkN6UiF5YtRBP2IMn5CGu4nhCHsYVjGNswdjUti2NW9hYvxG7YU8tYF2+cTmv7X1twDb2FaXUSqXUhcBtSqkL0x5XpVUc0/REx0qlKoZ5VxQ3X6eixQYUV5gVS7tEzFm8WZfDtE9SaXHhs2fm/ueBbnYuiCqlfqqUel8ptSH5yLVRo5Hrr7+eV155hVDIXHKzb98+6urqmDx5MrNnm8G0p556ihtuuIErr7ySRYsWEY/HufvuuznttNO44ooruOyyy1KR5gULFrB+/XrArL73jW98gzPOOINzzjmH+vp6wCz08cgjpvDZ7t27ufjiiznjjDOYO3cue/bswefzsXDhQubOncvpp5+etVhIrhiog/wqcJlSqlQpNR5zgd5zmPnJgzt3MFoItUIkBP4GOLbbfI6EzO0JqiONuBW4MTBEcGPgjkaotouZuymG+VwwxoxIJ6j11WIVKxbDgiAYiT9/JB5BRHBYHBgYFNoKefLTT/J/n/0/nvz0k8yvmG8WBklQbC9GKcX0kulcVHUR8yvm43a4icfirDuyjjcPvMmaujWEo2HGO8enCoU0BZtoDjbjdrhT29x2N9UHqofv+zWpw1QBCCaek4/fY0bmhpXzZ5RTWWLP2FZZYueKMyZR4rRxuDVIidPGt66Zzb9dcQpup40GX5hQNM78qWP5WNU4DJGUvNurW+uH+yMMnMbtMGmeueA07DOfJ81LLDwdGOn5xV31u6SEm9vh5oj/CG6HmyVnLuGeufektu1p3UNZYRllhWUYhkGxo5giaxGralYN2MYBMFVElnZ4/IOIfCyXRo0YLA4zPS0ZkIin3x8L7ZkrypTZLD0Z7NlvaHFVmIEJFYdACwspxOMuz7jp8oQ9qZm0UcTLInK3iEwUkXHJR66NGo2MHz+es88+m1dffRUwo8eLFy9OlTtPsnbtWlauXMmbb77Jiy++yL59+9iyZQuPP/44a9euzXpuv9/POeecw4cffsgFF1zA//zP/3Rqc/PNN/NP//RPfPjhh7z77rtMnDiRgoICfvvb37Jx40beeustvvrVr3aq8pcrBlxqWimVip8rpV4Xkf9QSi0VEUd3B2q6QqDtqCnlZrGbA3fMA+6JqRZ1xKgIBcwpP8MKsSiuaJi6gkwnq2MOZygSIhQPYYiRco4BYvEYSinCsTBxFafYXpxxmo4FEiY4ExJuY9sl3OyGnfpIPRbD0r6oT8GMcTPar58nOZ1KqQ+BD0Xk6XxIB6ptCeByWDEIE8e8a3U5rIRiiq9cMiOj7SkTS1IL8u57/sORI+nWUcKt7BTT+U2+P7IZymdD6Untx6h4pxzkmp1/oHrbr6gLNFLpLGPhaX/PzBmXZ7bpkE6xo2kHM8Zmfo/Z+t27te/y0u6XaA21UuIowWlxcvvpt6dSMS578TLGF4zPOKbIVkR9W05vSOYlHsk74cuBdcBdIvK8Uuq/cmZZrumXbKDq+nVSVtPqgLBB+4I9A8aeACdckJJ0o2QKM+d+nlvttpyr9uQBtyaetRRhHpBMs7j66qtZtWoVTzzxRKc2l1xyCePGmfcwa9as4YYbbsAwDCoqKrjwwguzntdut3PFFabU7FlnncUbb7yRsd/r9VJbW8u115qyrgUF5m9XJBLh61//Ou+88w6GYVBbW0t9fT0VFZlrQnLBQB3kJhH5f0AyhLIYaE6U7+1mbb6maxQUlZkKFtGQmdNWUEL6YF2JBY/DiTseN3PiLFZ82KmMdfjKgx7zhyGBw+YgHA+DmGkSFsOCxAXDMPBFfBTbi5ninsJUd2bhkI4FEqaWTOWiqkwJtxPGnEB5UXlKQs5lc2E37DSHmtuvnyWnMxdTjiKyhYQlHe+cAZRSw1pJb8uhZnY2tJm2Yf7j7Gxoy2pbOiNG0q2jhNvR3ebCqMnzTWWWQAs07wdbUWbOcYf+W7PzD6xc9whuaxEVBePxhL2sXPcIt0LKSc4m11brrcVpdWb064797sktT/KzD3+G3WKn2FZMIBLgZx/+DIDbT78dgPLC8k7Fb/wRP+WF5UPxrfWW8cBcpZQPQEQeAl4ALsCcFcnqICfG6PVArVLqimxtRjRHtpoFP/yN5jjasAPqNsHCf8t0kmMhKDsVDm80ZTS7xDBnNbyHzVrRs66A8rSFSSpuLqJOrhNJMJO8UO3JKUlJQk1+cM0117B06VI2btxIIBBg7ty57Nu3L6NNUVF72lFvo7k2my31m2WxWIhGM7MVuzrP008/TWNjIxs2bMBmszFt2jSCwWDWtsPNQB3kz2GW6n0J87d9TWKbBfi7AZ57dNAxyoEkdI+jprxQLEqNClHt91K3Yg6VtmJOjER52zDA4sBlWPHFo3gsBtf6I6azkSgMQrAF5n4+dalJrklsC2wz3whIXFAoxjjGsGDKglThjmxTgD1JDz2w5gGmFLc710cDR9lUv4mmYFMqyjyuYBxKqXwoFJJ0CJKLTH+ZeL4ZaBtuY3bX+4H24LpZirZ9e1eMGEm3jhJuvsNmnrG3znSInWPaI8qF47vsv9XbfoXbWoQ7McORfK7e9quUg5xNru2kMSexu3k3Yx1ju+x3v675NXaLHafNvLlwGs7U9qSDfOPMG1m+cTlAagGrP+rni3O+OFTfXG+oAtLrzkeAqUqpgIiEujgGzCqS2zFVL44/1v3CLPjhSOjGR0Pm+7f/Cyac0j7ehvxw+ANzvDVs3Z9z2ifN45r3mVHkdDrczGnaEZGs8gVKqf8dbls0Zq7wggULuOOOO7jpppt6bH/++eezcuVKbr31VhobG1m9ejWf+9zn+nxdt9vN5MmTeemll7jmmmsIhULEYjFaW1uZMGECNpuNt956i/3780cRcEAOslLqKLCki927B3LuUUG24gjewxBpw/RgoUaFWGkX3GKlwlaEJxrg7ZifTxll7DEUdbEAlRYn1xadyMyKiRlTfMz9fEa0ZLp7Ogc8B/BFfERiEWxWG0VGEWXOMo74jwxoCrCjCkCps5STx51MQ1tD6txLTja7Sq6nHJOanCLyCaXUJ9J2fU1E/gJ8czjtSaZVpKefiOp5CmbESLp1lHALesyp6qCnfdv4E8x+303/rQs0UtEhxcFlK6Iu0NjeJotc2xT3FNqibakiNtn6XWuolWJbZmqRw+KgNS33P6lWkV6G/YtzvpgrFYskzwDviUhyZcuVwK9FpAj4W7YDRGQyZirGd4Cl2dqMeA6tA3uxOQMH5nPIDx+9DcUT28fb2vUQaDJnLyz2bk4Yb18AHQmZ54d2bfoON3OaDOanvS4AFmIu8B/1DnKu5P5uuukmPvvZz7JqVc/rJ6677jqqq6uZPXs2M2bM4OMf/zglJf37jfnlL3/Jl770JR588EFsNhvPP/88N998M1deeSXz5s3jYx/7GLNmzerXuYeCATnIIjIDuI8+6huKyBhMyZfZmMGyO5RS2TO/RzI95cBlK45g2CHWDGOng9VBtW8nbqVwGxYQA7fNCSrKX8MNlDoSx8UjEPGbx+x61XSyi/eZUbkOMm9WrLhsLmLWGBaxYMXKaaWn8eC5Dw7oo3bMU/aFfVjEwtfO/lonBziPphyLROR8pdQaABE5Dyjq4ZhB4edv72Ll2gO0BiLtpQc6zEDZjO5TLICRIenWsRhDsAU89RAPwl9/bha5GT8TpszrNEWdTqWzjP2BozRIDK+KUCw2JigLU53tmt3Z5Np8YR9laW2ymugowRfyoUQRiyfSj5RQaC3ksU2PZcjDPfnpJwf+nQwSSqlvicgfgU9g3mLdpZRan9h9cxeHLQf+lczqZhmIyJ3AnQBVVVWDZ/Bw0vHfJ5RYQFe/pV1KMOQFS4HpQEe7C7jTPk4n8+S9dWC1Z72Z07SjlMoIoolICe2zdpoccO2112akPEybNo2tW03Fq9tuu43bbrsttc8wDB555BFcLhfHjh3j7LPP5vTTTwdg9erVqXY+X/tC/uuvv57rrzeLJS5btiy1/eSTT+bNN9/sZE9XC/9yzUBVLJ4HPgAewEzATz564ofAq0qpWcAZmFN9xxfJ6HA3smu0HjQH6XQMi1kYxFYAIS91FsFlZEY2QoaF9+wWPCgqFHhQrAzXUrP+p+ZCkqJy8/lPD8HWl1LHNbY1YrVaUSiUUigUVquVxrZGBko2FYB8LBLSgX8AHhWRfSKyD1N55Y6hvujP397FD6t30xaK4XZYOv2OJ5kzqUv/ZWRRdkoiUpeQcAuHINBg5s/bCs3I3sG1ZhSvG06ctpDN0Wa80SBFWPBGg2yONnPitPaUoGxybQe9B2kINHQr83bxlItpi7YRjoYRhHA0jD/qx2Vz5VyWsBd8gDkWvwg0iEiXHq2IXAE09CSzpZRaoZSap5SaV1bW/c1FXjJ5Pvga4dhHcHSn+dzWDIaRKaH5/9s79/g4z+rOf887d2k0ki+ybPkeO1YMTsjFBpKUAHHZhgBtsssWWsIG6DYU2oSUwi6ly7aU3U0+uxQKtHTjXSgu8IGGcmmzlEAwCQFiiEkIiRNjx0kc32RZli3NSJr7e/aP552bPJJGl9HMyM/Xn/nMzHubZ+SjV+c5zzm/k0sZ5YqCQgW+0jUkYNpOAzgTUiqWXQRLNsBNnzWTOuscz4Rx4OJGD8JSO2984xu5/PLLedWrXsVHPvKRpiigWwjmmoOcU9W/m8kJIhLDFJC8A0BVM1Tm0C0Oammd27nWOM6FfWBy2/yhYle83l99mbibJeaU/qsOapouhJg/CJom5gvC+BB7AtAX8aKJhed9u2DbTYBp5iG5NP70KK7m8IsfwRTozQet1iLVcxJe5tmkeA1w6s7uvUcJ+ny0h8wf44BPyOQrw8ftAfBNIdDeUkxsxuCmIBA1zoqbhVA7+Jaa1Y9fu2PSyzwnGS7ruYrTZ58lkRunw9/GpqWX8pyUbh8TC0p7o730uD0EnWBFXjJUtoyOhqJcuvxSDo8cJp1PE/KHWOJfQkewY8rzGo2IPkFIIwAAIABJREFU3I6pAxnANGEopLBPVmh6LfCbInIjZrk7JiJfUtVbFmK8C8bGV8Oh+z3ZNgXNAa6ZoJWnXfgjplAvmzL33Y6VkDjhXaSQ5CTQe0Xl9W3Occ2IyH2U1sd8wFaMHKylRSiPFF9IzPUv8H0i8l7gm0BxfWqadtMXYRqK/L2IvAxTaf0+Va2oSGr5Jb5aWudufZOJKhf2peJGwQItFtvtbNvA7rGD4DhE1WU0l+acKK/MOeDkTN5cPkc0m+JkYEKUI9Rh0i0KpMc4mxwkiJ+A+Mi4eUaTg3SHLixJShG5RVW/JCLvn7AdAFX9RNUTS8eFgYeBEOZ36J9U9c9r/fyRZBa/wOlEDtdVyn1jR8AvQldbgKNDY3zygUPF/OJCl7ymoloaEVRu63/SyBEOH4X0KGTGAD9oxmgeO35oX8nBsRPsKU9naFtP34kni9c5mTvB0Uycg7lhcm4Ofy5DXyZOYBqZwMHxQZLZJE+ffZpULkXYH2Zjx0YymqmQglseWc5AcgAn69AeaEddJe1WLrs3YSe09wF9qjpUy8Gq+qeYbpGIyGuADyw65xjMpGz9tZWdGc8+bxzhgjOcS5vVuoLGezpuHGTxQXLIOM6BNrj49eYXc4oCaMuUfLzsdQ54UVWPN2owjUZVp1UostROPTWT5+ogz0bf0A9cCdyuqj8TkU8BHwI+Un6Qqu4CdgFs3769OVSjZ0K16PDEqMPKbXDN7ZXOxJXej8Hb1rfuWm51r2LP0e9zMpugN9DB1dkcQZ8ffN5/n8/PqOOjNzdB0jedMAUpBcbPsFTCZAQy5AmKn6j6je7yhUVhLX+2OQxp4HpVHRWRAPBjEfmOqv60lpODjsO5ZBa/A44I+bJfcAfIq3JiJENbwHTSK++Sd9t1G5vHSa5WZLrnY4CafPjCtlNPlgqhAm3GuSjMp8UPbp6DyX52R1cRK6QznHuB3U9/jVujl9AXWwPJYZ4f3Md+xnG8f3k3z/6z+5GyJJVqMm/7z+xnKDVE0AkS8oXI5DM8dfYpeiI9xWMOnDnASGaENn8bbb42MrkM8Uwcv6/yFtmEndCOAS3YOrHOjByDXMablCUg1WGK9nyZinbQrNluAgnRFeYcX8jItS3ZWOkMb3l9pW63zTmuGVX9oYj0UCrWe7aR42kk4XCYoaEhli1bZp3keUBVGRoaKmoqzzdzVbGYjb7hceC4qv7Me/9PGAd5ThzoH6mo5m94tG3rm2DPX8LxMyYS4QuZNqVXTiiGO3MYjvyosrBu200VN98+oI+PFt8f/NJN7E4dgXymJPMWDHPz8DD4PTH7dMJERq4rzV063Dwjjp+Y4ycoPjKaZzA7zrOZc9z4jRvpaevhrX1vnbeq/IkNG3au29ksS9P3AqjqR6c7sBpqpqyFvJSA96h5EtcedBhOGhW/iacV75kKOdctahwXnu/fP9A8DnK1NKLjXj57YUk60mUidm7ORIqlkAHgYdSo2dPeRiyXKaUzDB8Hfzt73GH6ZB1EujhMCgEccRCkmEt/eKQkmFNN5i2RSSAIPsektLjqIgjxTLzYWS+dT5e0wcUcG/QFTfezxssSTsXzwEMi8m0qV/GmXAXxjnkIeKhuI2sk6TF4/gfgD5u0imwKxoagbSn0XFrp/O74vdL99sG7zKrcxNS4wQNTFpBaJkdEfhv4XxhbE+AzIvJBVf2nhg6sAaxZs4bjx48zODj3up9JSU2YL4eb5O9FnQiHw6xZUx9p07mqWLRhZILWqeptInIxZrnv/012jqqeEpFjItKnqgcxki9V5Yhq5UD/CLsefoHOSKDJom2et6MT3hfY/y1TSBeMVhbWQTFvuBp9q1/BrQNh9iSPlWTeOjfR1xU00eBEv4kcX/fBiutc0tZLJDXEaXIkNEvWzTNMhnYJ0B3uJpFOFHVe5+okV4vk7X56d7MU7h0UkUHgEeAnwCOqemgmF/AaLTwGbAb+tmzCV9g/aYqQz++jNxZkcDRLbsLykKvgOELAB+6EfU3XJa9aGlGVboloDvxtZsUjl6aUJgvggvg4GYywMp8vneM5pSfzJVnqDEoQcEWKy5RBgqb5jUc1mbecm8Pv+HHEIa/mM4ISJKclIfs8eUJOCJ/jI5PPEPQFWduxluH08JTycE3AUe8R9B4WgHMvgDgVq2z4AibneAopwZpS4ywz5c+AHap6GkBEuoHvY4JjFxSBQICNG+vcN+XBuyrf24ndrJlrisXfY5yEa7z3xzHV1JM6yB63A18WkSAmAvLOuQzi/v0DdEYC50Xbvrj3Rbo7wsWo8paedg4NjC1MlPnAfV7FfgLy3rJeMFpZpLdvl8l3S4/A2GmTF+cE4XsfgQf+CyTPmRy5HbdVFi9tfRN9547QF1tfGQm55vYpl/12vvQWdu/7OH3+dqKBdr6ROEwQ6ImuwnGcYoewz+//fEWXvNlEfqtF8grbG+1gqOoKT6LwGu/xAe+m/VPgJ7W051XVPHC5J1n4TRHZpqr7y/ZPmiLUEwsTT2ZZsywKwC9ePEfWVUQg6HfwOUIm6xLwV4rMNF2XvMmKTM94DRkKCs+OH4LlChWegyyOZ/N+enN54qFwqWtFKMZoJkFvWdvzCA5J8qibx8XFUQcXt9jcA6rLvIX8IfJuvthCPZFJkMllCPlKOfthX5i85FkdLTlHiXSCNR1rGqZVWguFVRARaZ9Yx3FBk0mYJiGjA6B5c5+N9gD5qR2GWlLjLDPFKTjHHkPMXUHLYqk7czXSTZ4zkQVQ1STnx4/OQ1Wf8CSELlPVm1T13HTnTMWJ4SQd4UpfP5XN8ePDQ8UcziNnRrn7Owd5YXC0Isp8oL9O6Xv9T8LgM5WSQoPPmO0Fzh0xzrHrFdu5ORg5CfGjZokw1Gmef3g3/PjTpfMKucuRLpMvF+ma1jkG04731h0fIBbs4FRqiLQove1raWtbXjxGRDgSPzJnaauToyeJBqMV25qpwElVD6nqF1T1NuC3gP8GvBSYUdqFqg5jlg5vqPWcW69ex1jadL1zXZdIUFDML46Dksm5ZF0t2qmrykgyy0gyyw3bGtrWuJKtbzITs6SnL5schuETkBmmpADggpuB1Dljy07AOMzgBZGN7NbO0TjxzrUlebauNcRzY+x0uorXvpx28oDrXdvFJU+ey5dfXhxSNZm39bH1pu41m8R1TXpFXvNs7txcPGZNdA1+n59EOoHruiTSCcZyY7y1760L+AOdOSJytYg8gyeVKSIvE5HPNnhYjUf8MHrK2Jo/Yp5HT5Vk2yajmk2nhkvFp5bZcL+IfFdE3iEi7wC+Dfxrg8dksUzLXB3kjIhEKGQSimyiLA9uoVjdFSGRquz7faA/wZI2E1V2RDgVT9Me8nMqkcYRKUac798/UJ9BpUcAx0SORTxpIcdUVT94F3zrvSZPOJcp5WY6fiOBBUbUPnGiJG6/b9e8DKtvyxt4781f4b/97vfZsuJlaKAyuX1wfJCQL8TBcwd56NhDHDx3kLybZ8/RPTP6nN5oL6OZSvm4ZilwEpFrROQDIvJ1EXkU01HMB9wCTLukICLdXuQYz/5/HfhVrZ//hstW86HX9xGLBDg9mqE9HGTD0hBtQR8ZF0J+hyvXxnjFxmV0RgL0j6TojASaIGVoAtUmaqkhTBdIp+zZI9RuuuUFwhCIefZuVlf6VlzBrV2XlnS0l2zk1h0foC+2rnjtVFcvMX8Mx7ttOTjE/DFShd8Zqutxf+zaj/GHV/whkUCERDZBNBjldy/5Xa5efXXxmA+/8sN8cPsH6Qh1MJgapCPUwZ1X3tnoLnm18NfAb2CicqjqLzEymhc2IqUsHilLdZuuMGqWwQfL+YjIZq9b6QeBezDSgy8D9uKtrlkszcxcUyz+HLgfWCsiX8ZobL5jroOaKTds62HXwy8AJk8zkcpxbjzLtZtL8mXxVJaOkI/RMke6rjmd4U4jTF8uKZQdh1S61Dwk0A5jg6Y5SLDdy8/07uqqgGOec0mzVFigmnrAI5+Z8Y38rX1vLeYctwfaGcuOMZodpSvURTqfJhqIks6neXb4WcZz49NcrZJqnfWaqMDpx5hWp58AvqWqM/tysArY7eUhO8C9U+XdV+MNl63mDZeZ5fwPfO2XrOoM45T98XZV6R9J8cev2zLDoTWYou5suR17dK0zy9WjAxyMRNkjSU6G2uj1RdjZuZq+1Ch9E9MZtryh+HLgGzfSG+1lJDtSzBPuDHRyLH6sottdxBdh36l9DIwPcGL0BJs6N3HN6mtI5pPFYzZ1buK5kecqPuo3Nv7GeQ5xExeaFlHVYxMq4vOTHXvB4GZNI4+xAROE8AfNvdLNTnsqK7dZh3h++GvgwwCq+g1MIxtEZLu3z4blLU3NXFUsHhCRx4FXYlaI30cDCkW2rurktus2VqhYvGrzMgL+UlekWDjASDJLzMtPhjrndK68zDjA5Tqc2ZDJKS7kty27yOTGZeJGcSJUpjpW/IMnxsnIl0XIa2lCUgMFZ+CrB7/KwPgAPW099EZ7i3JYgJHFymVm3EykWsOGJipw6qWUf/wHIuLHOMx7gb2q+vxUJ6vqk8AVUx0zE1Z3RRhJZou589CE+cbVqDZRm0rMw0s3Ojh+it0SJyYBVvpCxN0Mu4ce49blO5jKOjoCHRxPHCfkDxFwAuTcHEcTRwn4AsWUoKcGn+LxgcfpjnTTHTGFp/9z3/9kbcdaLll6CSvbV/LiyIvc99x9XNZ9GWs71k5aQNrkhaYFjnkt0tWr6biDxdiZdKZ0rDJFzysvLW1LjphgxIN3Vep2W2e4Xmzw7pUVqOrPRWTDwg/HYpkZc27V5QnUf7vwXkSOAgve2eP5wVH2Pj/EQDzF0ViYnZcs51enTM1KR9jPyliIk8NJtqyI4qqSSJkc0LfsqI88SFHmLZsC1DynRmDZxUbWLRU3y8+CycvsWGUizYVqaXUrr1e+VD2PldYTo2Z3/OAOjsePk86lCfqCZPIZXHWLBU4zoVk766nqKUw0oxDRaMO0mP4osJGKfrP1p9oKSF1tc76oOlFzKOUfV2dPyEfMzRBzAIQYDijsyZye0kFe3bGaF+Mv4rouPp8P13XJulmWtS0rFuU9N/KcsVvNFAtPzyTPcDRxlJevejkAp5Onafe3c3r8NOtj6yctIG3mQtMy/gD4FLAaUyT9PaB5qwoXih23lRSBCrKX40OwdGNpBW+WK2+WmplKnHba2b+IfB4otEbf5m1bCvwjsAE4Avz2XGuYLLXz2Seqlzc0cyHzXKhHL9sFV7/+9pMnuPs7B2kP+VkRDRJPZvnST49xyyvXMp5RTgwn2bA8yr95aU+FisVbdqypc07nBJk3ceDkL4wofajDpGCMD5lIc/Gn5sc0G5pAuFSVX89K60uWXkLEH+H0+GkSmQQdwQ7WxtaaQqdFgoh0AldTiiJfARwG7sPIvi0o1VZA6m+bNVCtS95ESSwnUJrwhWPGxidO7sCslIydhnSCkx0BVvrajbpAPgP+ENH2tZxMn5syutfmb2Pbsm08ffZpRjIjhP1hOkOdjGXG+OIzXySdT5Nzc0R8kaLmMYCipHKlPOVEJkF7oJ1EJlHcVq2AtJpkXDMVmgKo6hngbeXbROROzBL2hUtB3nLfrpLs5bLNplPeHFfeLDWzT0R+X1X/T/lGEfk9jPrVdHwB+BvgH8q2fQjYo6p3i8iHvPf/eZ7Ga7FUUA8HecG73u3ee5S863L83DjZvEvA5xAL+9nzqzPc++6ri8fd88Nn+dpjx4rL2W1Bqa/M25IN0FuqsOeZf67MJU4NG/WKrjWwxRNBOH0QcqOAr8zZcE0XqAK1NiGZBYXc4b4lfRW5wzvX7ZzztZuIwxhJt0eAjwGPegosDWPrqs7GO8Tl1JLn7gsZ5zgUKym1qAuIJ+vmybllxkyaUPsK6FxLb+pF4m6GWLgTlm4GYHTsNL3p8Smje0FfkNPjp9kY21hc3Th49iAZzRByQgQkQJYs4/lxnLIVF0EI+0vBrI5gB4l0oihrCNULSKtJxjVLoek0vJ8L3UEG4ySX68l/672VgQawGsf15U6MBObbKDnE2zFpmNMWo6jqw1VSMX4LeI33ejdGQcg6yJa6MCsHWUQ+Q3VHWICuKtvryguDowyPZ/D7HAI+h7yrDCbSZPOlId7zw2f51J7DBH0+YiEf4+k8n9pjOnC9+9UXz/+gqqVBiA8iS0utTjUPXevBLaup8fmANlPspHmTMxft9bZXXMw8TdaEZJY0ee7wvKCq3Y0eQ9Mz2zx3X9BM2vJe5zw3V9rusVPa2c0I5HNE1WU0O0Y8dZabY5dM/XkFLbyy9PysZotd8gQh6JjGIeO5cVzXZSw7hs/xFXONo8EoKyIrODV2ik1LNuGqO2kBaZMXmk6F7WFbDatxvKCo6gBwjYi8FijcNL6tqj+Yw2V7VLXfu36/iKyodtBUjZosEyhvLGKbilQw2wjyz2e5ry4ks3lTx5ZXsnnFwQSsktmS47l771GCPh/tIeNotoeAtNleFwd5siYK/hBseJV5f+RHpnAkUhY5DLYZ56K7TL1g4jHVotPJ4XlbKmzW3OH5QkTuY4qVDlX9zQUcTnNSS557Pg3rr4WhZ0spFp1rzOt8CnIp0+o3FDP7vIlhX7ibW4Pr2DP+IidTQ/RGurk5sp6+7m1Tfl7GzXBVz1UciR8ppv844uCoY7rkuXmCviABCZDMJxlMDdLT1sPvX/b7bOjcUJz0re9cz/Xrrq9ohlNtEtjCk8UFX8VrCba+yaxKQGWDpSvf3thxLXJU9UHgwQX+zEkbNVkstTIrB1lVd8/3QOaCg5JXk2soGI0jV832AiPJLLFQZRQ2EhBGkjXI/syGajfj9m5Mx4Jhsy26CkZOwIqtZmk6FYclG+HsC8YpLhSXZEZN2+jil7HtUOfIxxs9gKancy2cOQyj/SXnN7oKlm+uPObYz2H4KKRHIRU1jRi61sLFrysd9+wDHJQse7o6OZkP0OtrY6fTxXt7t5ciFg/eNW10r5DysGPljuK2p4eexnUri0iT2SSxcIx//beVvQimc2wnk3RrRodYRBJMvorX5PInDaKgcVyeVz+x1bSl2RkQkVVe9HgVcHraMyyWWTLbFIumisBFI0FybpqcC3lX8TlCyDHbC3RGAoyn8yZy7JHMaoW01rxS9Wb8EbOvsG35ZuNIDx4oHXPj/zKOSXlxyXUfrMyls0uFc0JVf9joMTQ93Vvhl1817dFDHWbCNnKisqNYoB2O7TW5yIE20ykvO240wAuTwFScg5Eou3WIWCbBykA78UyC3blT3Lr1jSXVihqie9VSHjZ0bOD5kedJZpOEfCHS+TSZfIZ3bXvXjL5ui0i6FVHVmcvKWKzGcevzL8CtwN3e8z83djiWxcxsUywaHoE70D9SrPp3gGxOUS/zzlUl78K6pW3F42+9ep3JOU6byHEyq2Tyed5z9cb6DXKym/F52246f3+5Q3xqf2V1f/dWOPQds88uFc4aEbkYuAt4CWWSRKp6UcMG1SwMHoA1O0o63pFOs9IxeICivT57v5nA5dOmyU2oHdqWAmomb5697tl4JbHsGLHh41AoeluxlT3jL5Yc5Bqie9VSHj72ax/jkROP8JWDX2EkPUJnqJN3bXsX77z0nTP6ui0i6VbEk7uaFFU9u1BjsVjqgYh8BVOQt1xEjmMak90N3OspYRwF/n3jRmhZ7Mw2xaKhEbgD/SPsevgFOiMBVnWG8TlCTr3KlGIhj7C0rRQdLuQZ7957tKhi8Z6rN9Yn/3g+qaYmcOg7sOX1lZFnu1Q4G/4ec9P9JPBa4J3YAifDyDGjGbtsU2mbunDkx/D3N5rVjZGTxva6yopgXPe8VJ+TyTOsXH6JSR/yiCYGOHn4u3Bkf6WkW5kNHzx7kD1lHfImU1KZ2CWvN9pb0Vmvlu53rSDpNoHHKN3tJqKAneRZWhpV/Z1Jdi0qSSVL8zInmbdGReDu3z9AZyRQTI8Yy5hKeddL+hBPaerJ4yP89j17GYin6ImF2dYbZc2SNgI+837NkraKSPTqrgg3bOupr9zWdNqyE5lMTWDwgK04nTsRVd0jIqKqLwJ/ISI/wjjNFzbV0nhO/AJOPWX2tfdA/BQMHzFdH6OeMMjYoIkml8m19Y48R9wfIVZwkEdPM3rsEXr90Ukl3aqlPHz68U8jIqyJrpl024txr0ve8stYG5u8S95EWk3STVXruPRlsVgslrnqIDckAndiOMmqzpKuaXw8W+rF4T2n88rx4RTRcIAV0SDHhkZ57MhZemIhVndFiCez/OX/O8D6pW28pLeTVZ1hRpJZdj38Arddt7E+TnIt2rITsQV59SQlIg7wrIj8EXACqCobdMFRLSf41BNGb7ugqNK1FoaeN7bYtswUlI4NwuodFRO6ndkN7D7zK2hbZnKHB54i7vi4OXax0fquIulWLeXh3OA5EHjJspdMuu30uNclL3ma9Z2Td8mbSAtLuiEiS4CLqQxSPNy4EVksFsvsaZaOfXN1kBsSgVvdFSmmSQDkyhp3TawcTKSynBlNEx/PgsLgaIZMXgn6HVKZPC8OjXP1puUAxet9ce+LdHeE5x5V3v+tymK7tu6Zd3KyBXn15E6gDbgD0zDkekzhh6VaTrAvZJp9FGhfblIq4sdgbMDYeLgL1lxZcam+jjXcenaMPaGYSXvIpri5axt9oSWlgyZM+qqlPKTd9Hm/4BO31dolbyKtKukmIv8ReB+wBngCeCWwF2PLFovFYpklc3WQGxKBu2FbD7sefgGAjrCfKo1ti+TyLkGfkFPFBdy8EvQ55PPKeCZH3q38i5vK5njkubNcf8mKuUWV938Lvv/nRgWgvceoAPT/Ei7aWensThcNttqddUNV9wF4NnyHqiamOeXC4sxho9Wd6IeOI6Y73thp0x46lzaa3r4grL4K3ulJqk0i19a3bCt9hdl/4ZhyJpF0K095CDmh89anJm6rtUteNZpV0m0a3gfsAH6qqq8VkUuAjzZ4TBaLxdLyzNVBbkgEbuuqTm67bmMxd3gq/D6n4r1Xv4ffJziOoBNCUgf6EyxpK+U3F57v3z8wMwd53y7jHBeWoyOdMBqBE49Cz9bScdNFg612Z90Qke2YNKEO7/0I8C5VfWzKEy8Eqk3wEqdMO+lgOwQiRtYtfxaufEfpvFomdLOUdFsSXoKIFDviVdu2os3rktc1dZe8RURKVVMigoiEVPVXIjKlly8ia4F/AFYCLrBLVT+1EIO1WFqBTz5wqPj6j1+3peZ9daO82x3Mrf5o4rUskzInB7mREbiHD53ma48dm7bRRyrrEvIL4vnBjoDrmo57fkcIB3yMJLN0hP0kUjnOjWe5dnOlglJH2D+tI34eiX4IRE0ThUK0LdxlGi+UacTWFA2epXbnZI0PLEU+D7xXVX8EICK/hnGYL2voqJqBqhO8AZPOEGo3jUFCUVh+FWTHSufVMqGbpaTbHRffATDltvWx9Vy/dvoueYuI4yLSBXwLeEBEzgHTSW/kgD9R1cdFpAN4TEQeUNVn6j1Yi8ViaRXmqmLRkAjcPT98lk/tOUzQ5yMW8jGWzk96bCjgMJbOEQn58DtC3lXGs3miIT/bepewekkbnZFAMd/4VZuXEfBXdtxLpHKs7pphc6pQzHTEC7SZZeh8DlKnoaO3QiO2XtHgVmt80CASBecYQFV/7HUoawoWXGGlnGoTvHzGzDAvf1vpOD1f1u281IzurefbeA2TvslSHmrddiGgqoXQ+F+IyINAJ/Cdac7pB/q91wkROQCsBubkINdirw21aYvFYpkBc02xaEgEbvfeo/hEyLkuw8nJ26xH/EJ3NERnOICryvB4hldsXMb65e0kUjlGklluuXp9xQ26oLEMFKPKI8ksb9mxZmaD7FoPQ8+BmzOV+m7OOBMrti6IPFurNT5oEI+KyD3AVzCx0bcAD4nIlQCq+nijBjZR67vuCisTCXbAuSMQ9CZ4bg5yKRNVLmdiilC11IzvezW75c1vLPOCiHxRVd8OJX16EfkiUFORgohsAK4AfjaXcdRirw23aYuljEKqxIKlSVxgTKZEMZGZKFNUu2Y9lS2c6Q+ZkvMicEDdI3DnxrJkci6q4HMmV5Ur7FOUcMDH+mXtRMN++kdSdEYCVW/MhfzmzkhgyuOmJRSFi14LgbBpvxsIm/eh6PTnzgMnR08SneDMNHnjg0ZwObAFo7ryF8BW4Brgr2hwt8hyrW9HpPj6/v0DCzOAJRvNhC5vNMbJ58Dxmdzj5LDZlxw2KULl7afLUzMcxzwHo2a7pR68tPyNiPiAq2o5UUSiwNeBO1U1PmHfbSLycxH5+eDg4LTXqsVeG27TFovFMgPmGkFuSATO50DeNau9ExHvoUAq5/KavpKoRkEabroZ49ZVnXOPaHSuheAw9LyktG1idX8dabXGB41AVV/b6DFMxkStb5hlLvxsCbXDRddD/+NG3zjUAeuuAc1OnSKU6DeR44prdZjtlnlDRP4U+DAQEZE4JS2PDDDtbEREAhjn+Muq+o2J+1V1V+E627dvn3yZzuPEcJKAD376fJx4KkssHGBpm59HX0gV0yme6R9heTQ45TE25cJisTQLc3WQL/eeJ+oeX4PxUeuixblxWRu/OpUgk3fxy/lesnqP/AT9twV1MBosz9bKjQ8WChHpAf4H0KuqrxeRlwBXq+rnGjy087S+YZa58LOlcy3knjNtpFNxY8P+ICybJkWoY5VJq4iUOTnphNlexsFD32bP01/iZHKQ3kg3O196C31b3lCnL7P4UNW7gLtE5C5VnVHOlogI8DnggKp+Yj7GE/IJ33j8GMlc5fYtK6LFdIrDA6M8cSyHg6kFGRhJkczmWLe0vSLl4te3dnNoYMw6zRZLLTSBKkWt6RStxpxSLFT1tVM86iZUf/3WlVy2OkbQ55Bxzw9uVPGZgQV2MAqV+pEuiJ8wz1N1zJtnCipnoE9tAAAP5UlEQVQAsVCMU2OniIVitkDvfL4AfBcohNUPYaQLG84N23oYSWYZSWZxVYuvb9jWM/3J80H3Vji+zzi7wah5Pr7PbJ+KHbdBZtQc77rmOTNqtnscPPRtdu/7OPFMgpXhZcQzCXbv+zgHD327zl9qUfJnInKLiHwEjISbiLx8mnOuxeQoXy8iT3iPG+cyiB8dOn2ecwzQPzxaTKfIuS7D41nyrhLwCePZHLm80aovHJPPu3zmB88xksxWOM0H+kfmMjyLxWKZMXNVsWhIBO6GbT0cPTvOpWuX0hH2878fOkzWixYroGWSbg8dPE06lyfk97E8GuItv7GACfmzlGebL1q08cFCslxV7/WWq1HVnIhMLomygEzU+l7dFeEtO9YsXCRt8ACs3m5kCVNxExHuvsRsZ4piu0IhXnkHyes+WFGgt+fpLxHztxMLmmYehec9T3/JRpFnzt9itIyvx2jRj3rbdkx2glcrMnnxRg385X1Pce/PT5DKuoQDDqOTKAklMvDAMwNEw37iyRyqytmxTLG5U9CBkVTJsz4VT5HNu3PXobdYLJY5MtcUiy9gVCv+zHt/CPhHzPJd3ZjoPIT8DtnM+f30fN6fAPH+Frg6bSqd5cJiTESW4TUrFpFXAk0TqpqXXPjZMnIMll0EyzeXtlWTdKvGtpumVKw4mRxkZXhZxbZooJ2TyemLwSzn8QpVvVJEfgGgqudEJFjPD/zL+57iC48cLbb4Hp9CZhPgV6fiOAI57xbtE/OHJ6eQcWGszEE+O5ZlWXvl8Bc0Nc5isVg85uogNywCV+487HnmFODid0oFejkXXIVQwEc2r4QCPjpCfhuJsJTzfuBfgE0i8hOgG3hzY4fUJHSurdoyesqujzXSG+kmnkkUI8cAo9kxeiPdc772BUjWU64oTPK6gfOjBfPIVx49jir4HSnecN0qqW4FHEy2TfG9mPN8rpJXyOZdXFUSqRx+R84rTl3Q1DiLxWLxmKvMW1NE4DKu0hn24YjgqrkBO0BeIZ3NEw35SGfzHBoY5emTTRMgtDQIEdkhIis9lZVXY9QA0sD3gOMNHVyzsPVNpqh0Kkm3WbLzpbcQz40RzyRw1SWeSRDPjbHzpbfMw8AvOD4NfBNYISL/HfgxJu2tbqRyLg6m1kOYvOajQE4rPfa818lU1VvlE4qSmrfv3ITjOI3LvbdYLBaPuUaQmyIC1xkJMJRIkcubG7HmFRfwOxAOmK544YCPdM4lnqpSSWK50LgH+HXv9TWYFKHbMaosu7BR5NpaRs+Svi1v4FaoULG4+Yr32PzjWaCqXxaRx4CdGH/1JlU9UM/P9IlZnauVgv9cOMXngIMgKK4LS9qDfPzfv6x4/EXd0cbl3lssFovHrBxkEdkBHFPVx0Xk1cC7gX9HgyJwW7rbeGg4VXxfiFb4BVLZPCG/Qzrnoqp0RuY6J7AsAnyqetZ7/RZgl6p+Hfi6iDwx1Ykishb4B2AlxtR2qeqn6jraRUjfljdYh3gOiEgY+ANgM/AUcI+qLsjsf9uqGE+ciJPLK04NzrJ4aRiFwxxH8It4kWWlb0VlQ6OG5t5bLK3AHKTdPjv85Hnb3tt1WfXrLkDX32aWiJttisU9GEF6KEXg/hY4Rw0i9fPNocFxQn6jWiGYZ7ME6BAO+Eikc4QDPi5Z2cFL7I3XAj4RKcyUdgI/KNs33QwqB/yJqm4FXgn8oafesrg4td/oeCeHIbbaPD/yGbPd0gzsBrZjnOPXs4CdH+9682WsjplCulwNkWS3LMXCB0V5zqDP4RUblrBySVvdxmqxWCyzZbbh1FlH4OaLA/0jxWW4wUSaWNhPOFD6OqlMjpFUnq2rYnSE/SRSOZvLZinwFeCHInIGSAI/AhCRzUyTQ6+q/UC/9zohIgeA1cAzdR3xQnPgPgh3lYr0Cs8H7muodKGlyEtU9VIAEfkc8OhCfvglvV0sj6WLEpq/PD75r40j4BPB70AsEuStL19X3DexGY7FYrE0C7N2kEXE7y3p7QRuK9tX9xyGA/0j7Hr4BTojAVZ1hgn6hHgyhyMOQb8JiudVWNIWoDMSsLlslgpU9b+LyB5gFfA91aL+n4PJRa4JEdkAXAH8rMq+2/B+L9atWzdxd/MzcsxEjssJx2qTebMsBNnCC089aME++P79A6xd2sa21aV76TMnRshOEk2+al0XiXSe4fEMa5aYDpHlQYu37FizQCO3WCyW2pmtMzvrCNx8cP/+ATojgWLk4Yq1Xex94SzxVJalbQGSWSWTz/Oe12zm3a++uN7DsbQgqvrTKtsO1Xq+iESBrwN3qmq8yrV24aUbbd++vfUEuOso82aZF14mIgW7EyDivRdAVTVWrw8+MZw8T4pt7bIIz585X6t4SSTA6dEMPbEwt1+/yRbgWSyWlmFWDvJ8ReBmy8Qb9FUbluGq8otjI8TTeTojAd5z9UbrHFvqgogEMM7xl1X1G40eT13Y+iaTcwwmcpyKG5m3K9/e2HFZAFBVX6M+e3VX5LzUCEd8XLQsQjqvjKZzREN+Ll0dIxquVKgArENssVhaglmnQ8w1AjcXqt2gt6zsZMfG5fzx6xawlbTlgkPMWvbngAOq+olGj6du1FHmzdLa3LCth10PvwBQTJXwO0JfbxcblpcUKWx+scViaWVaUvOs2g3a5rJZFohrgbcDT5UVpH5YVf+1gWOqDyu3WYfYch5bV3Vy23UbK1Ilbt+5ie8fGLT5xZYFQURuAD6FEUb5v6p6dyPH88kHzo8NtlKwrpr0GwDf/J2pT9z4qvkfTBPRkg5ytRu0zWWzLASq+mNKvQ8slguSalrFNr/YshB4rdX/Fngdpu/CPhH5F1VdXEpClobTkg4yWDF5i8ViaSbsPdmyQLwcOKyqzwOIyFeB32KxSW1aGo6U6uuaFxEZBF4ElgNnGjyc2WDHPXPWq2p3gz57XrH22zCs/c4D1n4bhrXfKojIm4EbVPU/eu/fDrxCVf+o7JiizCbQBwzRmjZQoFVtGBo39jnbcEtEkAtfUkR+rqrbGz2emWLHfWFj7bcxtOq4mw1rv42hVce9AFRLcauI9JXLbELr/yxbefytPPbZtpq2WCwWi8ViWWiOA+WC7GuAkw0ai2URYx1ki8VisVgsrcI+4GIR2SgiQeCtwL80eEyWRUhLpFiUsWv6Q5oSO24LtO7P047bAq3787TjXkR4rdX/CPguRubt86r69DSntfrPspXH37Jjb4kiPYvFYrFYLBaLZaGwKRYWi8VisVgsFksZ1kG2WCwWi8VisVjKaBkHWURuEJGDInJYRD7U6PHUgoh8XkROi8j+Ro9lJojIWhF5UEQOiMjTIvK+Ro+p1bH2u3BY+51/rP0uHNZ+55dWtN0CrWrDsDjsuCVykL3Wkocoay0J/E6zt5YUkeuAUeAfVHVbo8dTKyKyClilqo+LSAfwGHBTs/+8mxVrvwuLtd/5xdrvwmLtd/5oVdst0Ko2DIvDjlslglxsLamqGaDQWrKpUdWHgbONHsdMUdV+VX3ce50ADgCrGzuqlsba7wJi7Xfesfa7gFj7nVda0nYLtKoNw+Kw41ZxkFcDx8reH6fFftCtiohsAK4AftbYkbQ01n4bhLXfecHab4Ow9jtnrO02Aa1qx63iIE/bWtIy/4hIFPg6cKeqxhs9nhbG2m8DsPY7b1j7bQDWfucFa7sNppXtuFUcZNtacoERkQDGqL+sqt9o9HhaHGu/C4y133nF2u8CY+133rC220Ba3Y5bxUG2rSUXEBER4HPAAVX9RKPHswiw9ruAWPudd6z9LiDWfucVa7sNYjHYcUs4yKqaAwqtJQ8A99bQWrLhiMhXgL1An4gcF5Hfa/SYauRa4O3A9SLyhPe4sdGDalWs/S441n7nEWu/C46133miVW23QAvbMCwCO24JmTeLxWKxWCwWi2WhaIkIssVisVgsFovFslBYB9lisVgsFovFYinDOsgWi8VisVgsFksZ1kG2WCwWi8VisVjKsA6yxWKxWCwWi8VShnWQLRaLxWKxWCyWMqyD7CEieU+nb7+IfE1E2qY5/sM1XveIiCwXkU+KyJ1l278rIv+37P1ficj7RaRXRP5pkms9JCLbJ36+iGwQkf1TnHNQRH6zlvFO813+WESOisjfzPValvnF2m9N38Xab5Ni7bem72Ltt0mx9lvTd2k5+7UOcomkql6uqtuADPAH0xxfk4GX8QhwDYCIOMBy4KVl+68BfqKqJ1X1zTVcbyaf/zZVnXP3IFX9JPBf53odS12w9jsN1n6bGmu/02Dtt6mx9jsNrWi/1kGuzo+AzQAicouIPOrNDu8REZ+I3A1EvG1f9o77log8JiJPi8htVa75EzwDxxj2fiAhIktEJARsBX5RPpsTkYiIfFVEnhSRfwQi3vbzPh/wicj/8T7/eyISqfbFRGSziHxfRH4pIo+LyCYReY2I/FBE7hWRQyJyt4i8zfveT4nIpvn4oVoWDGu/1n5bGWu/1n5bGWu/i8V+VdU+TDfBUe/ZD/wz8B6M0d0HBLx9nwX+Q/nxZecv9Z4jGONd5r0/Aiwve70OeDdmhvkx4EZMS8aHvWM2APu91+8HPu+9vgzIAdsnfr53Tg643Ht/L3CL9/qhwjne+58BN3uvw0Ab8BpgGFgFhIATwEe9Y94H/HXZ+e8A/qbR/1/2Ye3X2u/ieVj7tfbbyg9rv4vTfv1YCkRE5Anv9Y+AzwG3AVcB+0QEjPGenuT8O0TkZu/1WuBiYGjCMYVZ4DXAJ4DV3usRzBLKRK4DPg2gqk+KyJNTjP8FVS2M/zGM0VcgIh3AalX9pnfNlLcdYJ+q9nvvnwO+5532FPDaKT7X0hxY+7X228pY+7X228pY+12E9msd5BJJVb28fIOY//ndqvqnU50oIq8Bfh24WlXHReQhzOxqIoU8oksxs8RjwJ8AceDzk1xeaxx/uux1Hm85ZeJQazzfLXvvYu2kFbD2W8Lab+th7beEtd/Ww9pviUVjvzYHeWr2AG8WkRUAIrJURNZ7+7IiEvBedwLnPOO+BHjlJNf7CfBG4Kyq5lX1LNAFXA3srXL8w8DbvM/ehlkmKVD++TWhqnHguIjc5F0zJNNU21paGmu/llbG2q+llbH22+JYB3kKVPUZ4L8A3/OWJx7A5NkA7AKe9JLc7wf83jEfA346ySWfwlSf/nTCthFVPVPl+L8Dot51/xPwaNm+8s+fCW/HLOc8iZmRrpzh+ZYWwdqvpZWx9mtpZaz9tj7iJU5bFinecs0HVPXn83S9d2CS9v9oPq5nsUyFtV9LK2Pt19LKXOj2ayPIi5+zwBdknoS+gT/F5DxZLAuBtV9LK2Pt19LKXND2ayPIFovFYrFYLBZLGTaCbLFYLBaLxWKxlGEdZIvFYrFYLBaLpQzrIFssFovFYrFYLGVYB9lisVgsFovFYinj/wO24DXIL4w97gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x576 with 16 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "from mlxtend.data import iris_data\n",
    "from mlxtend.plotting import scatterplotmatrix\n",
    "\n",
    "\n",
    "names = df.columns[1:5]\n",
    "\n",
    "fig, axes = scatterplotmatrix(X[y==0], figsize=(10, 8), alpha=0.5)\n",
    "fig, axes = scatterplotmatrix(X[y==1], fig_axes=(fig, axes), alpha=0.5)\n",
    "fig, axes = scatterplotmatrix(X[y==2], fig_axes=(fig, axes), alpha=0.5, names=names)\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.legend(labels=['Setosa', 'Versicolor', 'Virginica'])\n",
    "plt.savefig('images/eda.pdf')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Splitting a Dataset into Train, Validation, and Test Subsets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- The following code cells in this section illustrate the process of splitting a dataset into several subsets.\n",
    "- One important step, prior to splitting a dataset, is shuffling it, otherwise, we may end up with unrepresentative class distributions if the dataset was sorted prior to splitting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 72, 112, 132,  88,  37, 138,  87,  42,   8,  90, 141,  33,  59,\n",
       "       116, 135, 104,  36,  13,  63,  45,  28, 133,  24, 127,  46,  20,\n",
       "        31, 121, 117,   4, 130, 119,  29,   0,  62,  93, 131,   5,  16,\n",
       "        82,  60,  35, 143, 145, 142, 114, 136,  53,  19,  38, 110,  23,\n",
       "         9,  86,  91,  89,  79, 101,  65, 115,  41, 124,  95,  21,  11,\n",
       "       103,  74, 122, 118,  44,  51,  81, 149,  12, 129,  56,  50,  25,\n",
       "       128, 146,  43,   1,  71,  54, 100,  14,   6,  80,  26,  70, 139,\n",
       "        30, 108,  15,  18,  77,  22,  10,  58, 107,  75,  64,  69,   3,\n",
       "        40,  76, 134,  34,  27,  94,  85,  97, 102,  52,  92,  99, 105,\n",
       "         7,  48,  61, 120, 137, 125, 147,  39,  84,   2,  67,  55,  49,\n",
       "        68, 140,  78, 144, 111,  32,  73,  47, 148, 113,  96,  57, 123,\n",
       "       106,  83,  17,  98,  66, 126, 109])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "\n",
    "indices = np.arange(X.shape[0])\n",
    "rng = np.random.RandomState(123)\n",
    "permuted_indices = rng.permutation(indices)\n",
    "permuted_indices"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "97 22 31\n"
     ]
    }
   ],
   "source": [
    "train_size, valid_size = int(0.65*X.shape[0]), int(0.15*X.shape[0])\n",
    "test_size = X.shape[0] - (train_size + valid_size)\n",
    "print(train_size, valid_size, test_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_ind = permuted_indices[:train_size]\n",
    "valid_ind = permuted_indices[train_size:(train_size + valid_size)]\n",
    "test_ind = permuted_indices[(train_size + valid_size):]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(97, 4)"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train, y_train = X[train_ind], y[train_ind]\n",
    "X_valid, y_valid = X[valid_ind], y[valid_ind]\n",
    "X_test, y_test = X[test_ind], y[test_ind]\n",
    "\n",
    "X_train.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Python Classes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- This section illustrates the concept of \"classes\" in Python, which is relevant for understanding how the scikit-learn API works on a fundamental level later in this lecture.\n",
    "- Note that Python is an object oriented language, and everything in Python is an object.\n",
    "- Classes are \"templates\" for creating objects (this is called \"instantiating\" objects).\n",
    "- An object is a collection of special \"functions\" (a \"function\" of an object or class is called \"method\") and attributes.\n",
    "- Note that the `self` attribute is a special keyword for referring to a class or an instantiated object of a class, \"itself.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "class VehicleClass():\n",
    "    \n",
    "    def __init__(self, horsepower):\n",
    "        \"This is the 'init' method\"\n",
    "        # this is a class attribute:\n",
    "        self.horsepower = horsepower\n",
    "        \n",
    "    def horsepower_to_torque(self, rpm):\n",
    "        \"This is a regular method\"\n",
    "        numerator = self.horsepower * 33000\n",
    "        denominator = 2* np.pi * 5000\n",
    "        return numerator/denominator\n",
    "    \n",
    "    def tune_motor(self):\n",
    "        self.horsepower *= 2\n",
    "    \n",
    "    def _private_method(self):\n",
    "        print('this is private')\n",
    "    \n",
    "    def __very_private_method(self):\n",
    "        print('this is very private')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "123\n"
     ]
    }
   ],
   "source": [
    "# instantiate an object:\n",
    "car1 = VehicleClass(horsepower=123)\n",
    "print(car1.horsepower)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "129.20198280200063"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "car1.horsepower_to_torque(rpm=5000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "258.40396560400126"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "car1.tune_motor()\n",
    "car1.horsepower_to_torque(rpm=5000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "this is private\n"
     ]
    }
   ],
   "source": [
    "car1._private_method()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Python has the motto \"we are all adults here,\" which means that a user can do the same things as a developer (in contrast to other programming languages, e.g., Java).\n",
    "- A preceding underscore is an indicator that a method is considered \"private\" -- this means, this method is meant to be used internally but not by the user directly (also, it does not show up in the \"help\" documentation)\n",
    "- a preceding double-underscore is a \"stronger\" indicator for methods that are supposed to be private, and while users can access these (adhering to the \"we are all adults here\" moto), we have to refer to \"name mangling.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Excecuting the following would raise an error:\n",
    "# car1.__very_private_method()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "this is very private\n"
     ]
    }
   ],
   "source": [
    "# If we use \"name mangling\" we can access this private method:\n",
    "car1._VehicleClass__very_private_method()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Another useful aspect of using classes is the concept of \"inheritance.\"\n",
    "- Using inheritance, we can \"inherit\" methods and attributes from a parent class for re-use.\n",
    "- For instance, consider the `VehicleClass` as a more general class than the `CarClass` -- i.e., a car, truck, or motorbike are specific cases of a vehicle.\n",
    "- Below is an example of a `CarClass` that inherits the methods from the `VehicleClass` and adds a specific `self.num_wheels=4` attribute -- if we were to create a `BikeClass`, we could set this to `self.num_wheels=2`, for example.\n",
    "- All-in-all, this is a very simple demonstration of class inheritance, however, it's a concept that is very useful for writing \"clean code\" and structuring projects -- the scikit-learn machine learning library makes heavy use of this concept internally (we, as users, don't have to worry about it too much though, it is useful to know though in case you would like to modify or contribute to the library)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of wheels: 4\n",
      "Horsepower: 123\n",
      "Horsepower: 246\n"
     ]
    }
   ],
   "source": [
    "class CarClass(VehicleClass):\n",
    "\n",
    "    def __init__(self, horsepower):\n",
    "        super(CarClass, self).__init__(horsepower)\n",
    "        self.num_wheels = 4\n",
    "    \n",
    "new_car = CarClass(horsepower=123)\n",
    "print('Number of wheels:', new_car.num_wheels)\n",
    "print('Horsepower:', new_car.horsepower)\n",
    "new_car.tune_motor()\n",
    "print('Horsepower:', new_car.horsepower)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## K-Nearest Neighbors Implementation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Below is a very simple implementation of a K-nearest Neighbor classifier.\n",
    "- This is a very slow and inefficient implementation, and in real-world problems, it is always recommended to use established libraries (like scikit-learn) instead of implementing algorithms from scratch.\n",
    "- The scikit-learn library, for example, implements *k*NN much more efficiently and robustly -- using advanced data structures (KD-Tree and Ball-Tree, which we briefly discussed in Lecture 02).\n",
    "- A scenario where it is useful to implement algorithms from scratch is for learning and teaching purposes, or if we want to try out new algorithms, hence, the implementation below, which gently introduces how things are implemented in scikit-learn."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "class KNNClassifier(object):\n",
    "    def __init__(self, k, dist_fn=None):\n",
    "        self.k = k\n",
    "        if dist_fn is None:\n",
    "            self.dist_fn = self._euclidean_dist\n",
    "    \n",
    "    def _euclidean_dist(self, a, b):\n",
    "        dist = 0.\n",
    "        for ele_i, ele_j in zip(a, b):\n",
    "            dist += ((ele_i - ele_j)**2)\n",
    "        dist = dist**0.5\n",
    "        return dist\n",
    "        \n",
    "    def _find_nearest(self, x):\n",
    "        dist_idx_pairs = []\n",
    "        for j in range(self.dataset_.shape[0]):\n",
    "            d = self.dist_fn(x, self.dataset_[j])\n",
    "            dist_idx_pairs.append((d, j))\n",
    "            \n",
    "        sorted_dist_idx_pairs = sorted(dist_idx_pairs)\n",
    "\n",
    "        return sorted_dist_idx_pairs\n",
    "    \n",
    "    def fit(self, X, y):\n",
    "        self.dataset_ = X.copy()\n",
    "        self.labels_ = y.copy()\n",
    "        self.possible_labels_ = np.unique(y)\n",
    "\n",
    "    def predict(self, X):\n",
    "        predictions = np.zeros(X.shape[0], dtype=int)\n",
    "        for i in range(X.shape[0]):\n",
    "            k_nearest = self._find_nearest(X[i])[:self.k]\n",
    "            indices = [entry[1] for entry in k_nearest]\n",
    "            k_labels = self.labels_[indices]\n",
    "            counts = np.bincount(k_labels,\n",
    "                                 minlength=self.possible_labels_.shape[0])\n",
    "            pred_label = np.argmax(counts)\n",
    "            predictions[i] = pred_label\n",
    "        return predictions\n",
    "    \n",
    "    \n",
    "\n",
    "knn_model = KNNClassifier(k=3)\n",
    "knn_model.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0 1 2 1 1 1 0 0 1 2 0 0 1 1 1 2 1 1 1 2 0 0]\n"
     ]
    }
   ],
   "source": [
    "print(knn_model.predict(X_valid))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that there are class attributes with a `_` suffix in the implementation above -- this is not a typo.\n",
    "- The trailing `_` (e.g., here: `self.dataset_`) is a scikit-learn convention and indicates that these are \"fit\" attributes -- that is, attributes that are available only *after* calling the `fit` method."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Scikit-Learn Estimator API"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Below is an overview of the scikit-learn estimator API, which is used for implementing classification and regression models/algorithms.\n",
    "- We have seen the methods in the context of the *k*NN implementation earlier; however, one interesting, additional method we have not covered yet is `score`.\n",
    "- The `score` method simply runs `predict` on the features (`X`) internally and then computes the performance by comparing the predicted targets to the true targets `y`.\n",
    "- In the case of classification models, the `score` method computes the classification accuracy (in the range [0, 1]) -- i.e., the proportion of correctly predicted labels.\n",
    "In the case of regression models, the `score` method computes the coefficient of determination ($R^2$)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "class SupervisedEstimator(...):\n",
    "    \n",
    "    def __init__(self, hyperparam_1, ...):\n",
    "        self.hyperparm_1\n",
    "        ...\n",
    "    \n",
    "    def fit(self, X, y):\n",
    "        ...\n",
    "        self.fit_attribute_\n",
    "        return self\n",
    "    \n",
    "    def predict(self, X):\n",
    "        ...\n",
    "        return y_pred\n",
    "    \n",
    "    def score(self, X, y):\n",
    "        ...\n",
    "        return score\n",
    "    \n",
    "    def _private_method(self):\n",
    "        ...\n",
    "    ...\n",
    "    \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- The graphic below summarizes the useage of the `SupervisedEstimator` API that scikit-learn uses for implementing classification and regression algorithms/models."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/estimator-api.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- For 2D datasets (which we usually only have in teaching/learning contexts), we can plot the decision regions using a convenient wrapper function in  mlxtend as shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAEKCAYAAADuEgmxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xd8VHX2//HXmfRCAilAqAECSpEiiCgq9oqKii52Vl27rut+dy27P1e36uoW21pxBVdF7IANVBBRQRCpgjSBhAAhEAKhpM35/TGTEELKJJnMnXKej8c8mHJz75uWk3vP/Xw+oqoYY4wxvnA5HcAYY0zosKJhjDHGZ1Y0jDHG+MyKhjHGGJ9Z0TDGGOMzKxrGGGN85ljREJF4EflWRJaIyAoReaiObcaLyHYRWex93OBEVmOMMR7RDh67FDhVVUtEJAaYKyIfqeq8Wtu9oaq3O5DPGGNMLY4VDfWMKizxvozxPmykoTHGBDEnzzQQkSjgOyAHeFpV59ex2SUichKwGviVqubWsZ8bgRsB/vTo40NHjh7XiqnDw/L5sxmWvJ3jjurtdBS/mfDhHDJHDiEzK9PpKMaEnOO6nCS+bCfBMI2IiLQF3gXuUNXlNd5PB0pUtVREbgYuU9VTG9rXN+t26PLNxa0bOAyoKktfvo+nfnGi01H85kBpOTe98DHDfn4WmZ0ynI5jTEgZ1/9an4pGUNw9paq7gNnA2bXe36Gqpd6XLwBDAxwtbIkIu/eVU1npdjqK38THxfDCTefw1XMfUFlR6XQcY8KSk3dPZXrPMBCRBOB0YFWtbbJqvLwAWBm4hOEv+4zxPPvhIqdj+FVsTDTnDenFirnLG9/YGNNkTp5pZAGzRGQpsACYqarTReSPInKBd5s7vbfjLgHuBMY7lDUspWd1Z+ee0sY3DDFXnzKQ9E3bWPHlMqejGBN2nLx7aikwpI73H6jx/D7gvkDmiiQJyW2YtzuBTVt30q1jmtNx/OrXF47gZ09Pp98JAxDx6VKtMS0mKiSRQpwrDiH4/t0pSqm7lL3sRqV5/WxH754yzhIROg4YyZYdK8OuaADcPmoALz4zjbNvOd8KhwmIJFJISUwBlxKENQMU4txxsA9KaN4NQ0HRCDfO6ZB9BJPmrHM6Rqs4cUA2w1IS2LJhq9NRTISIc8UFb8EATy6XenI2kxWNCJfesTO7Y8N3XMMVJw1g/v8+Y9+efU5HMRFAkOAtGFWEFl06s6JhSErP4usfDhszGRbSU5N4ZvxpzHrxY6ejGBMWrGgYjj7vWiZ8sdHpGK0mLSWJLnFR5K7a5HQUY1rd/NnfctWp47li1DW8+p/X/b5/KxoGEaHCDRVhPCDuL5ePYtP0eWzfvN3pKMa0msrKSv79wJP8/eW/MnHmBD6bOosNa/z7A6HdPWUA6H3O9Tzy1sv8btxxTkdpFS6XixtPHcgTny/mtKvPcDqOMdw89tfs2nV4r61t20SefesfzdrnysU/0rl7Jzp16wTAqeefzNwZX5Hdu3uLstZkRcMA0KFbDp9PK2N70R4y27VxOk6rOKpXJxI+X0pZaRmxcbFOxzERbteuffS5+d+Hvb/62buavc/CbYW079S++nVmViYrF69q4Cuazi5PmWq9zxzPrKXhfd3/5jOH8NGT79vcVCYs1TkBrZ/v5rKiYaolJKewfOMOp2O0qn7dO/DAmUOYPWmm01GM8bvMjpkU5BdUv96+ZTsZ7dP9egwrGqZaRlYXNtKRgqI9TkdpVX27t6d82052Fe5yOooxfnXkoCPI27CZLblbKC8r5/Npsxl5xvF+PYYVDXOIhNQMinaH/0C4/1x3Jl8+M439e/c7HcUYv4mOjuKuP97B/11zL9ecfh2njB5Fjz7Z/j2GX/dmQt6AUy/hzxN+xyt3dXA6SqtKjI/lkqG9WbDgRwadPNjpOCYCtW2bWGfTu23bxBbtd8QpxzLilGNbtI+GWNEwh4hLSCQuxb/XQIPVRcf3ZfmUuayKj+XIEf2cjmMiTHNvq3WaXZ4yh0k78lie/3iJ0zFanYjwwM9OZN0cW3fDGF9Z0TCH6T38DFZuCf++RpUrhvXms5c+qft2RWPMIZxc7jVeRL4VkSXe1fkeqmObOBF5Q0TWish8EckOfNLIVLBP2VLYvPn2Q805Q3MYmRzLumXrnY5iTNBz8kyjFDhVVQcBg4GzRWRErW2uB4pUNQf4F/BIgDNGrCFj7+J/s/07kjSYnTe8D0unzuPAvvBb/tYYf3KsaKhHifdljPdR+/rAhcBE7/O3gNPElmALiLiEBHILdzsdI2Ay27XhiStG8emz052OYkxQc7SnISJRIrIYKABmqur8Wpt0BnIBVLUCKAYOu7VHRG4UkYUisvC9yRNrf2yaIT4xmf0dhrB8fb7TUQKmY3oKXeKi2bJ+i9NRjGm2h3/zKBcOHcv4M29olf07WjRUtVJVBwNdgOEiMqDWJnWdVRzWrVTV51V1mKoOGzPu2taIGpGSMjqza09kDX7787iTWP32HDav3ex0FGOa5ZyxZ/HoxL+12v6D4u4pVd0FzAbOrvVRHtAVQESigVRgZ0DDRbCcISN5YuZap2MEVHR0FE9dfxY/vfcVm2zRJhMAu3YW84df3E9xkX9uPBl07EDapLbeTNVO3j2VKSJtvc8TgNOB2p3XqUDVqcNY4HO1+yIDJi4+gYS0Tk7HCLioKBdPXH8my9760ukoJgLMfPNDKnJXM2PKh05H8YmTZxpZwCwRWQoswNPTmC4ifxSRC7zbTADSRWQtcDdwr0NZI1ano0byxLRFTscIOJfLxXmDezL//W+cjmLC2K6dxSz4cCaPX5LFgg9n+u1sozU5effUUlUdoqoDVXWAqv7R+/4DqjrV+/yAql6qqjmqOlxV7Ub6AOs55ERWba9wOoYjxp3Yn6O1km/etjMO0zpmvvkh5+cIvTvEc36OhMTZRlD0NExwi+S7nK8+ZSDHxUUzd8oXTkcxYabqLOPKoSkAXDk0JSTONqxomEZVJnfki2WR2xQed2J/Tk2JZ85rnzsdxYSRqrOM9GTPvLHpydF+Odt46I6/cOvFd7JpfS5jR4zjgzc+8kfcahJufeVv1u3Q5ZuDu1KHGlVlzSv38Oh1o5yO4qhp83/k8/3lHHuhfxe1MeEjQzoQmxTj07a/v/Y37Mw/fBxUWqdO/Hnio/6OdoiyveUU6rZD3hvX/1qfLinY1OimUSIS0Zeoqpx/7BG88u/3KT/nGGJiffvGYEx9WrswtBa7PGV8smm3sm1n5EwrUp/Hxp3ItEffpOxAmdNRjHGEFQ3jk95nXMuXy3OdjuG4bh3T+NfPTmD6Y2/a5IYmIlnRMD6xy1MHdc5syxNXjGLWBP82GI0JBVY0jE/aZnbg48V5VFa6nY4SFDqmp9CuspKtG7Y6HcWYgLKiYXzSpm06rs6DKCwuaXzjCPGny05k4fTaEzMbE97s7injM5fLfsaoqU1SPEfGRrF5fT6de0beHF0mOBXkF/CXux9h5/YiXC7h/MvPY+x1F/tt//ZdwDRJmA3rabHLR/Zl8QffOh3DmGpR0VHc9vubeeWzl3jm3Sd595X32bBmo9/2b0XD+Kzb0Sfz8DuRN3lhQ3p1ySTL7WZfSWStO2L8Y97sb/nNjfdw1TnX8psb72He7Jb/AJLePp0+A3oDkJicSPde3di+tbDF+61iRcP4LLNzD/ZGpzodI+jcc9FxfP7cB07HMCFm3uxveeG550k5NY5jHxxAyqlxvPDc834pHFW25G5lzQ9r6Tf4SL/t04qGaRq7PHWY9NQkuifGkbfaxrEY37392ttkX9iJtJxUXFEu0nJSyb6wE2+/9rZf9r9v734euOUh7njgVpLaJPlln2BFwzRRuz7DefGTZU7HCDp/Gnci302d53QME0K25G6lbY9DV9hr26MNW3Jbfht3RXkFD9z8IKePOY2Tzj6xxfuryYqGaZLeI85kRf4ep2MEHZfLxfEd27L2+8haHtc0X1bXjuz66dD/S7t+2kNW144t2q+q8sg9j9E9pzs/u2Fsi/ZVFyeXe+0qIrNEZKWIrBCRX9axzckiUiwii72PB5zIag5ld1DV7Wcn9GPZp9/hdtsAyEBaPHcpD9/5GL8eey8P3/kYi+cudTqSTy654hI2vJ/PzrXFuCvd7FxbzIb387nkiktatN9lC5cz451PWfTN91x/zk1cf85NzJvlv/FETo7TqAB+raqLRKQN8J2IzFTVH2pt96WqjnYgn6nHnrj2LFy9hWF9spyOElTat2vD/acP5p//mcY5t1/odJyIsHjuUt747xSyx2SRnd2X4g0lvPHfKQAMPmGgw+kaNuLk4YCnt/Fj7kayunbkFzfdWP1+cw085ii+2PCpPyLWybGioapbgC3e53tEZCXQGahdNEyQ6X/W1Xwx53ErGnUY1DOLtDkr2F20h5R2bRr/AtMiH0+ZQfaYLNr18qx+165XCozxvB/sRQM8haOlRSLQgqKnISLZwBCgrnOo40RkiYh8JCL96/n6G0VkoYgsfG/yxFZMakzj/njpCXz58kynY0SE7fmFpGYnH/JeanYy2/P9Ny7BHMrxoiEiycDbwF2qWnvBhkVAd1UdBDwJvFfXPlT1eVUdpqrDxoy7tnUDGwBKyyqcjhC02iTF0z0hhnWL1zkdJexldsqgeMOh86EVbyghs1OGI3kUDf7b0tWbs5kcnXtKRGLwFIxXVfWd2p/XLCKq+qGI/EdEMlTVfoxwUHJqOxbsiievoIgu7ds5HSco/WncSfzu1dmsqayk99A+Tsfxi8Vzl/LxlBlszy8ks1MGZ192ZosvAbV0n2dfdqanhzHGc4ZRvKGEDe9t4Wc/v6xFuZqr1F1KnDsOXArBuJqAAm6h1F3a7HyOFQ3xLNAwAVipqv+sZ5uOwDZVVREZjufMaEcAY5o6iAgd+g6noOgnKxr1EBH+cuXJPDh5DqsqKjny2L5OR2qR1mg4+2OfVdt9PGUGq/NzyeyUwc9+fplj/Yy97IZ9EOeKQ4KwaihKqbvUk7OZnDzTGAlcDSwTkcXe9+4HugGo6rPAWOAWEakA9gPjVO2Gz2BRXlHpdISgJiI8dPkoxj/3UcgXjdZoOPtrn4NPGBg0TW8VpYRiSoL5u1QLa5ljPQ1VnauqoqoDVXWw9/Ghqj7rLRio6lOq2l9VB6nqCFX92qm85lC9hozkXx/96HSMkHDp0Bw+nzjD6Rgt0hoNZ2tihybHG+EmNMUnJhOfZrfc+uK8Yb0Z2y2TmRM+JlRPlFuj4RxsTWzjG1uEyTSbKlRWuomKsp89GnPm0b2IiXbx8qNv0unoHAadfrTTkZrk7MvO5JVn/kf6yW2Iy4yidHslW2cUkRzbhl+PvZfMThn06d+b1SvW+NzUbk4T25fGeWs07JsjWHL4W9SDDz7odAa/yiva/2DBnlKnY0SE2HYd+WT6VE4f3M3pKCGhR4d2XDQsh7XLf+K7VXmktG9LfGK807F8snXTNr6fu5g9uXspmL+LPev2cqD4AL2u6EyfS7uxc0cR3838nk7npdPn0m5IuvLV5PlkpGfQsVuHOvfZsVsHMtIzWD7tR9ZM34Rreyxjrrqg3m+sVY3zrPPa0XNM5zqP4cs2gRAsOZpiQPvBD/mynZ1pmGbr2L03S+bEOh0j5Nx81lDmLN/A4/98i0v/er3TcXzy8ZQZ9B3fo7ppPe8fS+h+aQck1Y0rykXhD0X0GJdV/drXpnZTmti+NM6DZYR4sORoDXZdwRgHnDQgm6tPPopvp37jdBSf1G5a79t+gNScZCrKy+t8Df5vavvSOA+W5nqw5GgNVjRMi5TsK6Wy0mZ1bY4xxx5J0apNuEPgz6920zoxM57itSVEx8TU+Rr839T2pXEeLM31YMnRGqynYVokJqMr82d9xAn9uzgdJST1zkjhxVdn0Xv4kbhcwfszXHKbZL6aPJ/4DrHEpcZQurucDR/kk9knnaSMhMNe7/ppDxve28KYqy5o8Br+4rlLefmxV3h3wlS+n7uY5DbJh2xf8/PKskoKlheS1DmBuNSYOo9RO6evOZqqsdyByuFPvvY0rGiYFmnTNoPNCz/hjCHdnY4Sktq3Teao9qk8N+lTcoYfiStI70Sr3bROdqdw7MjhbPtuR52vG2tqQ+PN4tqfx3eJoWBhEQfWVrBx5tY6j9HU5npz+NLkDkQOf/O1aEio3jden2/W7dDlm4udjhFRFj33S5657UynY4S0dXmF3D/tW0bffQkxsTGNf0EYePjOx2h3RkJ1sxigaN1uimbu594n/q/Rz50SrLlaalz/a30aKx6cP9aYkOLuOpyvlm9wOkZI69Ulg0fGjGDa36ewZPaSkB0E2BSNNYuDtZkcrLkCxYqGabHMnEEU7NrndIyQl52VxovXnc5xpQeY8fwHYV84GmsWB2szOVhzBYqN0zAt5hIXxXutj+QPKUkJXDCiL2ltEnnhmWmcfcv5eCaEDj11jYgGqt+Li4sj9/lcotoIpSXlxCXHUL7TTXqHNH499l4SkuLZ+vJW+o7vERTTnlcJtunYA82KhmmxLjl9+WDGXsaeWEpyYpzTccLCCf27Ex3l4qkn3+Pc2y8M6jur6lLXtOevPPM/Kg+46Tu+B9nZfVk/I4/CWeV0HNWexA6x7NtWRu70AhIHxjDgzB4Ubyhh9eR9bJxcQGmp89OeVwm26dgDzYqGaTERIa1LDvtLy6xo+NGII7sSHRXFvx5/l/N+eVFIFY66RkSnn9yGrbN2VL9X+EMRPa/IIjEtgY7dOrA1fhsxl0dR8FkROed0o12vFPqM6+5tMPt0Y0/ABNN07IEWOv8KTVBTEfYeKHM6RtgZ1rsTvz15ANP+9XZIDAKsUlezOC4zitKSgyPGa48irygvJzUnmX3bD1RvE0kN5lBR75mGiPgyDWe5qi5rzoFFpCswCegIuIHnVfXxWtsI8DhwLrAPGK+qi5pzPNO6+p96KQ9NeoD/3nm601HCzqCeWfy/KBd/+ufbjP7VxURFR/n9GP6ePbaqWVzzttTS7ZXEJR+8nbhqFHliWgIA0TExnteZBydxDFSDuakz0obrDLa+aOjy1BfAAhpe56kHkN3MY1cAv1bVRSLSBvhORGaq6g81tjkH6O19HAs84/3VBJnE5DbEtEl3OkbY6te9Aw+eczQP/OMtzr/7EqJj/Hdl2ZdlV5u6NGtdzeIds/fgKo2maN1uUrOTyejXjp8mb6bflTm4K91osYufJm+h+6jOuCvdAWswN/X31hpL34aShv7lLVDVUxv6YhH5vLkHVtUtwBbv8z0ishLoDNQsGhcCk7xLvM4TkbYikuX9WmMiyhFdM/nr6GO4/7E3Ofeui4lL8E//qDVmj62rWXz1LVcd9t5Z553F6q/WsODNlQdfr1jDgodWBqzB3NTfWzjPYOuLeotGYwXD1218ISLZwBBgfq2POgO5NV7ned87pGiIyI3AjQC//fM/yDlxjD9imSY6UKHs3L2XtJQkp6OErV5dMvj7Rcfx5PPTKUpK5PTrz27xLbnb8wvJzj50DfPU7GRW5+c2aZva6msWB9s31qb+3przZxFOfGqEi8hAEblARC6uevgrgIgkA28Dd6nq7tof1/Elh414UtXnVXWYqg4bM+5af0UzTTR07C/505SFTscIe92z0nhs/Blcc2Rn3vrLa6z4slltxWqhNHtsa2jq7y2c/yx80eiFURF5CRgIrMDTsAbPN+53WnpwEYnBUzBeVdW69pcHdK3xuguQ39LjmtaR2CYFoiJj3qRgMOqobEYdlc2LM7/no0en0O/c4XTvn93k/dTVf1j58k8kJycfspTrZ89/jjuukvK9FcQkReMqjeIX9wTfIlJNbVI3dbCeDe5r3AhV7efvA3vvjJoArFTVf9az2VTgdhGZjKcBXmz9jOC2c/c+3G53SI0pCHU3nDGE61X519T5vPPWHE697QLaZrT1+etr9x/i4uKIinfRdWxm9TfF2ZO+YP++A2Sf14H4DrEc2FZG3vRC1q/4KaguNzWnSd3UwXqRPriv0VluRWQC8I9adzW1/MAiJwBfAss4eAZzP9ANQFWf9RaWp4Cz8dxy+3NVbfD6h81y66wNP3xHry2fcN1Zg5yOEpH2HSjj5hc+Jq5re066/JRmzZhb1yyuK2etZuucIgbfk1P9XtGqPWx8ZTvPfPSEX7L7Q7jOQBsIvs5y68uZxkTgGxHZCpTi6TOoqraorKrqXBq+nRfvXVO3teQ4JrDSOnZnz3ob5OeUxPhYJt5+Phu37uSex97kvLvHEhvftHXc62r0xneIpWJf5SHvpeYkc2BvcDV/I71JHQi+XEN4Cbgaz0/75wOjvb8ac5io6Gi27CxpfEPTakSE7Kx0/nXZiUx/7E0O7GvaZJJ1NXoPbCsjOvHQQYXFa0uIT4onmER6kzoQfDnT2KSqU1s9iQkLSSlt2ZbQizW5BfTu2t7pOBGtU2YqT145ijv+8Sbn3H0JCUkJ1Z8tnruU6a99xIaVG+jRN5u+g45k9Yo11bPPbp9cQJ9x3WsMzNvL/m2lLH5kLRX7KolOjKJsRznnXXpukzLVblL36d+7+rh1Na1bu6ltms6XorFKRF4DpuG5PAVAPXc7GUNqVg/27t/odAwDdEhL4c9jRvD4B99y4mWjgIPN4oyT2zDozM5sX1LCJx98Qr8rczimX9/qu6dy39rO6r2eRu/Jp49i9qdfkH5SUnUjfMfsvfTs38PnLLWb1Otn5B123JpN60A0tU3T+VI0EvAUi5rrefrllltjTOvL6ZKB+6PvyFuTR5feXfh4ygy6nd+eiuh9dGsby4+rt9FjXBaS6sYV5aJdrxT6ju9B0cz9/Hnig4Cnwdz/ml4HG8z9oKjb7iaNgq49krrwh6LDjltzZHVzR15H8gy0gdBoT0NVf17H47pAhDOhKbFdJl//uNXpGMbL5XLx9C/OYuPUr8ldlcv2/EJcqUJqPMRFC+W7yknOTqCivBx1KztW7Tpsdll/LHFaex+1Z7mtvc9IX1Y1WDVaNERkooi0rfG6nXfAnzF16tF/KHM2VTgdw9Tgcrl4/Loz2fzRfBIS4tn5YyFpiZ7//m0yYtm9tgRXdBSrXl9H7id57Fxd7PcR4bX3UTXLbXTMwduCQ2G510jny91TA1V1V9ULVS3CM0+UMfWKT0h0OoKpxeVy8a+fn0HC3gq2TdvGrk0HcFcqnfoksuGNrexdt5/4WBcJqbH89G4+J517Io/f8U/27Crh7MvOZMN7Wyhatxt3pZuidbvZ8N6W6iVcfVF7H55Zbregxa469+mPYxr/86Wn4RKRdt5igYik+fh1JoLFJKexZN0WBvXKcjqKqUFEyEyIJm+L8uVz+ZRXuImJdhFX6WLt/zbTqUMK2R3TyGzXnuL87bjyNzD3ndmcc91ooGUN5rqa1LVnua25T2tqBydfRoRfA9wHvIWnAX4Z8BdVfaX14zWdjQgPDm63mwUv3suLt45yOoqpwxtzVzCnpJQTf3Zy9XtfPPEez1zrmbj6huc/ZvGi73l6dBK3Td/LTc/8jjZtk+vZmwkHvo4I96URPgm4BNgGbAcuDtaCYYKHy+UiPiGh8Q2NI352Qn9OS03gi1c9S+Is+nghJ/U4OK5m1aoNXJDjoneHOC7IcTH3ndkOJTXBxqdZ5VT1B1V9SlWf9PccVCZ8Fe+voLhkv9MxTD0uPq4v57VPYfpT77P2m+VcftIAAAp3lbDpp01cPtRzZnH50GRWzPyKPbtspL9poGiISKNrcfuyjYlcQ8f9hr+/vcDpGMarcFcJl9z7LDuK91a/N3p4Hx47dygv3nBW9XuTPviaoR2jeP6LnWzbXU56crRjZxuL5y7l4Tsf49dj7+XhOx9j8dylAc9gDtVQQ7uviDT0NyRAqp/zmDCSlNKWfHfj25nAmPTB1xRtzWXi9K+4+8qDdyB1TE85ZLvZi1aTXxLF2qIyXluST0qKZ36plO0rqhvigRDpa3EHq4aKxpE+fH1l45sYY5xWuKuE6V8s4JmLM7hl+gKuHT2S9NS6l+Wd+o/bAdi4dSf/+CGP4y86IZBRq0X6WtzBqt7LU6q6seqBZwW9cjx3T6nnY92oqnkBymlC1KadB9i91/oaTpv0wdeMznFxRPs4Rue4mDj9K6cjNcpGhAcnX0aE34HnzqmZwAfex/RWzmXCRI/TxzNryQanY0S0qrOMa472nFlcc3QS079YcEhvoza3282O4r141kFzho0ID06+3D31S+AIVe2vqkd5H345NxSRl0SkQESW1/P5ySJSLCKLvY8H/HFcEzgJySms3ryr8Q1Ni9Ructd8XXWWkRrvYm3edtomuOo826j6mvfmLuesByZx4d/+x5RJ71c3oAPdlLYR4cHJl5HduUBrjZZ7Gc9yrpMa2OZLVQ1c9834VVb3Xnz+WQK79+4nJcnGbbSW2k3umq9nL1pNfkEpz84rZv/+AyQk7CclKZ5O21Yf0hCf9MHXLF65jjWlleyq2E3OdV2q16R45Zn/UXnATd/xPQLWlLYR4cGp3qIhInd7n64HZovIBxy6nsY/W3pwVZ0jItkt3Y8Jbsnt2rNnX6kVjVZSu8k9+sTBh7x+89G7UFUu++3jPDO6A7dM38ebj951SCO8cFcJ02Z/yyl9knhn3SaOuqPfIQ3o9JPbsHXWjoA3pW2a8+DT0OWpNt7HJjz9jNga7wVyPoHjRGSJiHwkIv3r2kBEbhSRhSKy8L3JEwMYzfii36ljeWDyd07HCFu1m9z3PPXmYU3vxhrhkz74mvNyXHRIiSaaSly1bqaPy4yitKT8kPesKR2ZGrp76iFVfQj4oep5jfdWBijfIqC7qg4CngTeqyfr86o6TFWHjRl3bYCiGV8lp7YjKrmd0zHCUu0m93lHxLNs1TquGOyZZfiao5N4//Nvefez+fU2wqv3McTz+REd4tj5YyHuyoODbEq3VxKXHFPz0NaUjlC+NMLv8/E9v1PV3apa4n3+IRAjIvav1ISUukZi+2ufz7wzm9E5LjKSPVeaP1hZwhUDonGX7atueo/qXEZGzAHKK5Wzn1lHpVsPOduoOgup2scdx6Wx45PtbF68tboBvWP2Hlyl0dbWtVMmAAAgAElEQVSUNg32NM4BzgU6i8gTNT5KAQKywo6IdAS2qaqKyHA8RW5HII5t/Ku03B2xzfD6RmL7Y59v5xUQLW5eW1YAwObCPbhwM+H7AlwoCQn7OVBWTnklDHt8E5nxlRz9701kpbepboRXNcpfXVpAbnEp6WvdyN5o1v8vj/z3i8jslMHVt1wFWFPaNHz3VD7wHXCB99cqe4Bf+ePgIvI6cDKQISJ5wB+AGABVfRYYC9wiIhXAfmCcNjaXuwlKgy+5k4cm/41/XB9ZU6U3ZSR28/a5jzcf/fVhTW1P0zuxuulduKuE8+54hEkXJ3PxlH28/tdb6d3VM6tt1QhwVeWSf73Hxb+/kuiYur81WJEw9RYNVV0CLBGRV1W1vL7tWkJVL2/k86fw3JJrQlxyajvUFdP4hmHm0Ab0Ab+cbTS2z7o+/3LJGq4YEM3AjjFcMSCa3z75Ju/+/bZD9isi3HxCX+Ys+JEBx9d5z4kxDc5yu8w7YeF3IrK09iOAGY0JSc0Zid3Sfdb1+Vsz5rFs1TpuGe6ZePCW4Z5m+ZrcgsP2nxQfi53Mm4Y01AgfDZwPfOx9XOl9fIhnFT9jTANqN5gzkqMZnePiwRenkXbGr/j2hw2Ab43y6sb327MbHN1d1zFTXSVcdEQUWclRAGQlR1WfbRjTVA1dntoIICIjVXVkjY/uFZGvgD+2djgTXgorEvgxdztHdM10OkpAVDWYq5rUVfK2f0t2inLT3ybx/SsP+NQor258524j2qX1ju6u65g/bS3l+83w2vJDp3OJic097Dg5XTJY+8Zc+h/XD5fLpzXaTITxZRqRJBE5QVXnAojI8UDLOnkmIh11/i+YMf/ZiCkaVQ3mmuYt+4lx9/ybly5MZMzkHcyYv7LRRnntxvezv/8FN//5hTpHd9d1zKbokJZCv7Rkyg6UEZ8Y36J9mfDky48S1wNPi8gGEdkA/Ae4rlVTmfDk4IypweKWv7/ClQNjGNwxmisHxnDT3yY2OmW5LyO+/UmwvydTv0aLhqp+5x2RPRAYpKqDVdWWeTVNlpicwrx1O9l3oMzpKI6Yt+wninbs4LZjYgG47ZhY3GX7GdDe89+wrkZ57cb25YMSWbZqHaOPjK/3a/yh5mhwY2pq6O6pq7y/3u2dvPAG4Poar41pkti4eDIGn8HavO1ORwmoqib2TQ9P4sqBMXRq42lId0x2cdVRMdzzgefPo6pR/vSbsw6b1ryqsS0V+7liQDRvfF/MJf/NRUT8frZx5ci+zP7vJ37bnwkvDfU0qi6stglEEBMhIvASVVUTO3drCRO2w4RFhw57OlChDHv6YOO6wv0daTFlh0xrXtXYLijaQ2Wlm7KKXXRNFY55Mo+0NgmHTXPeEr26ZJBqTXBTj4bunnrO+/QRVT0QoDwmzGV07c3rc+cwMKez01EC4tAmduJhU5LXtb1nNHdK9bTmdTXGa4/4bukoc2N85cuPE8tF5CsReVhEzhWR1Ma/xJi6dczuQ35F5PwTaura3L5sH4j1vksPlFFZUen3/ZrQ50sjPAe4HFiGZ8DfEhFZ3NrBTPgSV2RcomrqiHBftm+NUeZ1+fXZRzPr5Rl+3acJD40WDRHpAowETgSGACuAN1o5lwlnqmE7VUVda3PXHhFe35mBL9s3dZ/NdUTXTLS0VaacMyHOl8F9m4AFwF9V9eZWzmMiQLdjzuTht6Zy36XHOh3F7+pam7v2iPD6mta+bN/UfRrjb74UjSHACcAVInIvsAb4QlUntGoyE7a69hvGku+mOR3D72pPg97UBrUvo7lbOuK7Kfbs3E1lRSVR0VEBO6YJfr70NJYAE4H/Ap8Do4D/18q5TJgLx8tTgWhQB9Ldpw9i4ccLnI5hgowvPY2FwDfARcAq4CRVzW7lXCbcZfTi44XrnU7hN4FqUAdSVlobKssDskinCSG+3HJ7jqoepao3qeorVbPf+oOIvCQiBSKyvJ7PRUSeEJG13nU8jvbXsY2zBp97LbOWHT7LaiD5MiX5jxu30WPM/XWuPVFzH7XX6m6oQd0aa4a3hvjYaIrybXVlcyhfLk+15pwPLwNnN/D5OUBv7+NG4JlWzGICzOlLVDWb1vW59+m3SIveX+/aE9VTln++iNeWlTLs6YLqx2vLSpm9aHWzjhsMOqSlcGRsFNs3R9a0L6ZhvjTCW42qzhGR7AY2uRCY5F0XfJ6ItBWRLFXdEpCAplXl7oslr6CILu3bBfzYvqzd/ePGbSxbtY53Lkvi4imele6q1tU+fB+Hr9Xd3OMGky7pKewos0tU5qBgn2CmM1DzGkae971DiMiNIrJQRBa+N3liwMKZlsk+6VIWr9vmyLF9aVrf+/Rbh62r3dR9NOe4xgSzhma5vbihR4Dy1TV0+LBrGqr6vKoOU9VhY8ZdG4BYJpT50rSuOsuob13t5jS+Q7FZ3qtjO9bOX+V0DBNEGjrTOL+Bx+jWjwZ4ziy61njdBcgP0LFNAJQ5cHeOL6Oqq84y6ltXuzkjswM1mtufTujfHdlc6HQME0QamuX254EMUo+pwO0iMhk4Fii2fkb46NanP6//5xXOG1FOXGxMwI7ry6jq73/M5duyciZ8X/e62s0ZmR2qo7ljA/h3Y4KfT41wETkP6A9ULxqsqn9s6cFF5HXgZCBDRPKAPwAx3v0/C3wInAusBfYBwVDIjJ9ER8fQtnMO+0sDWzR8GVW94f2HW7wPf3yNMcGm0aIhIs8CicApwIvAWOBbfxxcVS9v5HMFbvPHsUxwUoRym4I7qI3MyeKbafM45vwRTkcxQcCXu6eOV9VrgCJVfQg4jkP7DMY02xGjLuL/vW5TVQSzscf3Zc9GZ+5yM8HHl6Kx3/vrPhHpBJQDPVovkokkbTM6oAltnY5hGhGBq/SaevhSNKaLSFvgUWARsAGY3JqhTGSJbd+TafPXOh3DNCAzykXuyk1OxzBBwJei8XdV3aWqbwPdgSOBP7duLBNJBp51Je/O3+D4tCKmfn+49AR+mLPU6RgmCPhSNL6peqKqpapaXPM9Y/wh/ZgLeWP2D07HMPWwq1OmSkMjwjuKyFAgQUSGiMjR3sfJeO6mMsZvuh91LNO+z2N/aZnTUUwdoqJc7N9aREkQj143gdHQLbdnAePxjML+Z433dwP3t2ImE4Hi4hPIGHQ66zZvY0DPTk7HMbWICLee3J8ZKzcyYEQ/p+MYB9V7pqGqE1X1FGC8qp5S43Ghqr4TwIwmQiS0zWDBGru1M1ilpySxdXWe0zGMw3zpaXwlIhNE5CMAEeknIte3ci4TgXoNOp5PVu+3hniQ6t8zi+TCYsrsEmJE86Vo/Bf4BKi6ZrAauKvVEpmI1ueUS/jrlPlOxzD1SE1KQN1W1COZL0UjQ1WnAG4AVa0AbN4H0yq6Hnk0W/fZvTrBqn1KApvXbHY6hnGQLxMW7hWRdLzrWIjICKC4VVOZFvnb7ZdTUrLnsPeTk9tw31OvO5CoaUpIYuXGAvp2b9/4xiagbj1nGOOf+4ieA3s6HcU4xJeicTeeKcp7ichXQCaeSQtNkCop2UPPG5487P31L97hQJqmG3HF//H0hF/z1K1nOB3F1CIixEVF4a5044oK9oU/TWto9G9dVRcBo4DjgZuA/qpqQ0NNq3FFRbE/sz/f2506Qen20wYx+5VPnY5hHNJo0RCReOBO4E/AQ8Bt3veMaTV9Rp7Hy7N+tDupgtDAnh2p2HvA6RjGIb6cX07CswDTk8BTQD/gFX8cXETOFpEfRWStiNxbx+fjRWS7iCz2Pm7wx3FN8Etr34k9ydns2WffnIJRWWm50xGMQ3zpaRyhqoNqvJ4lIktaemARiQKeBs7Asxb4AhGZqqq1JyB6Q1VtybMW2Jq7nsrKSooKC/jd+IPLuwd7Yzw2uR2r83Yy7IjOTkcxtZzUOY2V81fS99i+TkcxAeZL0fheREao6jwAETkW+MoPxx4OrFXV9d79TgYuBGzWuhZKTm5zSNO7qLCAmOQ04jO60POGf1S/H+yN8SHnXMnjz93DK1Y0gs7pR3XniTVbnY5hHOBL0TgWuEZEqibT7wasFJFleFZkHdjMY3cGcmu8zvMeq7ZLROQkPIMKf6WquXVsY2qoffbwu/Gj67ybKti5XC4y+h7P5DkrGXeS/UQbTETELlFFKF96GmfjWalvlPfRAzgXGA2c34Jj1zWCq3bXcxqQ7S1MnwIT69yRyI0islBEFr43uc5NTIjqd/JFfLQoD7fb7XQUU0N2VhqV6/PZXXT4eCAT3ny55XZjQ48WHDuPQ9ca7wLk1zr2DlUt9b58ARhaT8bnVXWYqg4bM+7aFkQywajTqCt49fPlTscwNYgIx+Z0Yu9umyo90vhyeaq1LAB6i0gPYDMwDrii5gYikqWqW7wvLwBWBjZicGhshPctZw1Go6IOfqAKCLgrSM/qys6Crez69w24YuLJPO+X1ZsV7yj0+RhOatuhCzs22F1UweaEvl358/vfkHXnRU5HMQHkWNFQ1QoRuR3PZIhRwEuqukJE/ggsVNWpwJ0icgFQAezEs75HxGlshLdGRdH19v9Vv19WuInYjG7kv3Q7PW94kui1PyCuGLa98Xt2TD+4NIq6K3w+hpPaZnTk881urijaQ/t2bZyOY7z6dMkg3ekQJuCcPNNAVT8EPqz13gM1nt8H3BfoXOEmKjqGuIxuxLZJY+Dtz1S/HwwFwRcul4ucE87njS8/5I4LjnE6jjERzSaPMSEhe8Bw5uc3vp0JrONyslj4gU1lH0kcPdMwvtm1fStLn7ql+nVpcQHiikLdldw6ejio55IUIkSndABAa9xttP2Dx1F3BZUlRYfsp2LPwZ5GKIiKS2Dn7r2kpSQ5HcV4XTayH59OmuV0DBNAVjRCgIqLTuP/Xf0696U7yRr/OGUFP5HYsSebXryN2IxulBVuwhUTC4C4Dp5EussP0Om6pygvzCWhfbfq9zc+dU3gfhN+MPTiW/n9aw/xn5tPcTqKqcG9/wB7dpXQpm2y01FMAFjRCAEucVFauKn6tborcZcfXHJTouPIf+l2KvfuQlyu6rMMraxg/Yt34N5XTFnBT4Aesh+XHCwstUeR13w/WCQktaGsTRcWrM7nmD6dGv8CExB/v/wk/u/NOZz5i3OdjmICwIpGCEhNz6Bzdu/q19sAV0ws4vL89WWcexcxGV3Jf+l2jr7rxert1r94B395eTq/Gz+abjmHj6guTc+ofu70bbW+OmbsHbzz+v1WNIJIckLc4cNyTdiyRrgJKeJysalwD5WVNkI8WES5XOzasgO3/Z1EBDvTCLDGBtHdecGxVLgP/bHNXVlJ8VO3kX72bZ7X+/d4Gt+VFezbsg7wNMLd+z37/fZPY6rPQm4+xzOIPnrNDyhKtLfnAaEzuK8mESFjxKXM+v57Th92hNNxDBAV5eKaYb1YtGg1/Y450uk4ppVZ0QiwxgbRVbiV7rdPOuSzfVvWUfDWg+z46AkAtKKUwql/B0Bc0ai7ElDc5aWsf/EOxBVNl9smgYCIi62v38+Oj5+gYnchcamZ1fsNlcF9tSWmtGNbwX6nY5ga2iXF2YJZEcKKRiiIisaVkELWtZ47qPJfup1O1z1VPfK7Su6TV/GXl6dz8zlDD7l7qsO4vyAi5D55VUgO7qutR99BvPvZ64w9qZy42Bin4xggLSWJrSvy6D/cZiMOd9bTMCFHREjt2JXS8orGNzYBcVSvTiRs20l5mU2XHu6saJiQ1OPYs/njGwucjmFqSE9JtFlvI4AVDYdtzV3P5g1rqpdidbsrKS8rpaKstN6vqW6EuyspK9xEWcFP1eMwbh09HPCMCFcN37tZOnTLoURsZHgwueu8Y/hywidOxzCtzHoaAdbYUqw7/3YZm5/7BaqVuFye6c7dlRVoZSW5T10FgFZUeBvh3inQAbzbdr99Euv/9TPynj58tLdWlh9y7JoD90JhcF9tmpTJ1z/kcny/ro1vbFpdanIC7ZLinY5hWpmE2x0P36zbocs3Fzsdw2f1LcVaNTDPl6/ZtHYlse17sGXiXXT9+ePV72986hr+M/1b/4cOEqrKDxPv4983nOh0FOP1ixc+4ZS7L8HlsosYoWZc/2vrWk31MPY3a0KWiBAVZf+Eg8ltpw5k9iufOh3DtCL7H2dC2pbdFWy3daqDxuBeWVTssVUWw5mjRUNEzhaRH0VkrYjcW8fncSLyhvfz+SKSHfiUJpj1u/AW3pu31ukYpoZ0rSR/nS1+Eq4ca4SLSBTwNHAGkAcsEJGpqvpDjc2uB4pUNUdExgGPAD8LfNrW05wGdO2v2VmwxTNfXGXlIdOdR7t8ukQZ0hIS27B+2y6nY5gabjp1IE//sJFOvWxSyXDk5N1Tw4G1qroeQEQmAxcCNYvGhcCD3udvAU+JiGgYde+bM69TMM0F5bSUtAyWx/Viw5YdZGfZitXBInz+h5ranLw81RnIrfE6z/tenduoagVQDIevZS8iN4rIQhFZ+N7kia0U1wSr5IyOFJfYdfRg0TkzlW1L1nFgX/1jjUzocrJo1HXtpPbPJ75sg6o+r6rDVHXYmHHX+iWcCR19jj2Tv0//ofENTUAkxMVy2pFdKC4MnVvfje+cLBp5QM1RWV2A2t2z6m1EJBpIBXYGJJ0JGfGJScSltnc6hqkhp2M71sxf6XQM0wqcLBoLgN4i0kNEYoFxwNRa20wFqk4dxgKfh1M/w/hPxpHDee6jJU7HMF4nD+yBO3e70zFMK3CsaHh7FLcDnwArgSmqukJE/igiF3g3mwCki8ha4G7gsNtyjQHoPfx0Vmy1vkYwiYuJcjqCaQWOzj2lqh8CH9Z674Eazw8AlwY6lwlNO/cr23bupkNaitNRDNAhOYGNKzbQvX+201GMH9mIcBM2hl/5W56abpeogsU9Fx3HqhnfOR3D+JkVDRM2YuMSDr+1zjhGRIi1lRXDjhUNEzbE5WL9tt1UVFQ6HcV4VZTso6S4xOkYxo+saJiw4XK5yDzuUuYsWed0FOP1yLiTmPf2XKdjGD+yomHCSmKbtmwp2ud0DOOVkhRPic1CHFasaJiw0qPfYN5Ztpuy8gqnoxggNiaa4elt2Lhqk9NRjJ9Y0TBhRURIzexEWbn1NYLFoK7p7CvZ73QM4ydWNEzY6XXC+Tzw2jynYxivmOgo9hXvdTqG8RMrGibstO/Sk31RSU7HMF4nDOzJli+X4650Ox3F+IEVDROekjvw5fLcxrczrc7lctEtM5XKSrtkGA6saJiwdPSYm3jn2w1OxzBel4/sy6yXPnE6hvEDKxomLIkI24v3U24D/YLCUT06kmh/F2HBioYJW91P/zmTZ61wOobxskUNwoMVDRO2kttmULjHbvUMFiO6Z7B45iKnY5gWsqJhwlZ6x87ML4hm287dTkcxwNWjjqJgdZ7TMUwLWdEwYUtEyOzRj917bXGmYGGXqEKfI0VDRNJEZKaIrPH+2q6e7SpFZLH3UXspWGMa5cbFvgPlTscwXmmVFeSvy3c6hmkBp8407gU+U9XewGfUv4zrflUd7H1cUM82xtSr7/Fn88i0H5yOYbxuOnUgeTYPVUhzqmhcCEz0Pp8IjHEohwlzcQmJxLfNdDqG8UpPTSJv+UbcbhsdHqqcKhodVHULgPfX9vVsFy8iC0VknojUW1hE5Ebvdgvfmzyxvs1MhMrsO4JnPvze6RgGT9E4qUsahfk7nI5imim6tXYsIp8CHev46HdN2E03Vc0XkZ7A5yKyTFUPW2FHVZ8Hngf4Zt0OXb65uFmZTXjKOeY0lr0y2+kYxis2Jhq7NSF0tVrRUNXT6/tMRLaJSJaqbhGRLKCgnn3ke39dLyKzgSGALctmmqzogLClsJisjFSno0S8nI7teHfeKtqPtcuGocipy1NTgWu9z68F3q+9gYi0E5E47/MMYCRgHU3TLMMu/w3PfrzU6RgGOHVQTyo3bnM6hmkmp4rGw8AZIrIGOMP7GhEZJiIverfpCywUkSXALOBhVbWiYZolNj6egl22DGywiIt2UVJc4nQM0wyOFA1V3aGqp6lqb++vO73vL1TVG7zPv1bVo1R1kPfXCU5kNeEhOjqGym4jWLjSbvcMBg+OHckXL81wOoZpBhsRbiJGSlZPivbY2UYwSE1OICE2yukYphmsaJiI0aPfYJ7/YiOVtoJcUNi/Z7+N1whBVjRMxIiNiye5Yw8qbAW5oHDnqQP5+u0vnY5hmsiKhok4FXamERT6dcvkgE1dH3KsaJiI0mfUxfy/V+c5HcMALpewb/dep2OYJrKiYSJKRqfu7I9KdjqGARLiYjk6NZFNq3OdjmKawIqGiTwpHZm1ZIPTKQwwqEsGe3fbHW2hxIqGiThDLvgF07+zn26DwbH9u/HDB/NRW50pZFjRMBFHRNi6az9l5RVOR4l4KUkJ9EpPQd1WNEKFFQ0TkbLPuI4pX9isNMHCxmuEDisaJiIlt01ju93uGRQuH9mXz1/62OkYxkdWNExESu/YhW+3xVBQtMfpKBFvUM8sEsptwGWosKJhIpKIkN79SHbvteWAgkF6TBSbbDLJkCDhdtfCsrxiXbvdfno0jSvM38y22RP4/VWnOR0l4qkqv5v6JaOuPNvpKBHrrF7niy/bhV3REJEbvcu/Br1QyRoqOSF0slpO/wuVrKGSsz7heHnqRqcDNEGoZA2VnBA6WS2n/4VK1lDJWadwLBrGGGNaiRUNY4wxPgvHohFK1wpDJWuo5ITQyWo5/S9UsoZKzjqFXSPcGGNM6wnHMw1jjDGtxIqGMcYYn4VV0RCRs0XkRxFZKyL3Op2nPiLykogUiMhyp7M0RES6isgsEVkpIitE5JdOZ6qLiMSLyLcissSb8yGnMzVERKJE5HsRme50loaIyAYRWSYii0VkodN56iMibUXkLRFZ5f23epzTmeoiIkd4/yyrHrtF5C6nczVV2PQ0RCQKWA2cAeQBC4DLVTXopjIVkZOAEmCSqg5wOk99RCQLyFLVRSLSBvgOGBNsf6YiIkCSqpaISAwwF/ilqgbluq4icjcwDEhR1dFO56mPiGwAhqlqodNZGiIiE4EvVfVFEYkFElV1l9O5GuL9frUZOFZVNzqdpynC6UxjOLBWVderahkwGbjQ4Ux1UtU5wE6nczRGVbeo6iLv8z3ASqCzs6kOpx4l3pcx3kdQ/jQkIl2A84AXnc4SDkQkBTgJmACgqmXBXjC8TgPWhVrBgPAqGp2Bmsux5RGE3+BClYhkA0OA+c4mqZv3ks9ioACYqapBmRP4N/BbIBQWkFBghoh8JyLBOoq5J7Ad+K/3kt+LIpLkdCgfjANedzpEc4RT0ahrsq2g/Gkz1IhIMvA2cJeq7nY6T11UtVJVBwNdgOEiEnSX/URkNFCgqt85ncVHI1X1aOAc4DbvZdVgEw0cDTyjqkOAvUDQ9jMBvJfQLgDedDpLc4RT0cgDutZ43QXIdyhL2PD2CN4GXlXVd5zO0xjvpYnZQDBOlzoSuMDbK5gMnCoi/3M2Uv1UNd/7awHwLp5LwMEmD8ircWb5Fp4iEszOARap6jangzRHOBWNBUBvEenhreTjgKkOZwpp3gbzBGClqv7T6Tz1EZFMEWnrfZ4AnA6scjbV4VT1PlXtoqrZeP59fq6qVzkcq04ikuS9+QHv5Z4zgaC7209VtwK5InKE963TgKC6UaMOlxOil6bAc2oXFlS1QkRuBz4BooCXVHWFw7HqJCKvAycDGSKSB/xBVSc4m6pOI4GrgWXefgHA/ar6oYOZ6pIFTPTekeICpqhqUN/OGgI6AO96fm4gGnhNVYN1TdY7gFe9PyyuB37ucJ56iUginjs8b3I6S3OFzS23xhhjWl84XZ4yxhjTyqxoGGOM8ZkVDWOMMT6zomGMMcZnVjSMMcb4zIqGiVgiMl5EOvmw3csiMtbX9/2Q6/4az7Prmw1ZRGZ7Z3W+wA/H/JWIbBKRp1q6LxPerGiYSDYeaLRoOOD+xjepdqWqtngQq6r+C3igpfsx4c+KhgkL3p/IV4nIRBFZ6l1fIdH72VAR+cI78d4nIpLlPUMYhmdQ2GIRSRCRB0RkgYgsF5HnvSPifT3+Ycfwvj9bRB7xrvexWkRO9L6fKCJTvFnfEJH5IjJMRB4GEryZXvXuPkpEXhDPWiEzvKPe68qQIyKfimddkUUi0ktETvbmmuI9/sMicqU3zzIR6dWSP3cTeaxomHByBPC8qg4EdgO3eufOehIYq6pDgZeAv6jqW8BCPD+pD1bV/cBTqnqMd42TBMCntS7qO0aNTaJVdThwF/AH73u3AkXerH8ChgKo6r3Afm+mK73b9gaeVtX+wC7gknqivOrdbhBwPLDF+/4g4JfAUXhG+Pfx5nkRz2hqY3wWNtOIGAPkqupX3uf/A+4EPgYGADO9Jw5RHPxmWtspIvJbIBFIA1YA03w47hGNHKNqosfvgGzv8xOAxwFUdbmILG1g/z+patU0LjX3Uc07T1RnVX3Xu88D3vcBFqjqFu/rdcAM75ctA07x4fdnTDUrGiac1J4TR/FMmb9CVRtcAlRE4oH/4FmpLldEHgTifTxuY8co9f5aycH/cz5f+qrx9VX7qOvyVEP7q/n17hqv3dj3ANNEdnnKhJNucnB96MvxLPv6I5BZ9b6IxIhIf+82e4A23udVBaLQu35IU+6KaugY9ZkLXObdvh+eS0dVyr2XvHzmXeckT0TGePcZV9XTMcafrGiYcLISuNZ7qScNz8I8ZXgKwCMisgRYjOd6P8DLwLPeGXxLgRfwXLJ5D89U+z5p5Bj1+Q+eQrMUuAdYChR7P3seWFqjEe6rq4E7vfv8GujYxK83plE2y60JC+JZjna6t4kd9LzTuMeo6gHvHUyf4WlQlzVhH7OB/1PVhX7KNB7P5bnb/bE/E57sTMMYZyQCc71nJu8CtzSlYHjtBF721+A+4D48d50ZUy870zDGGOMzO9MwxhjjM9QIepwAAAAgSURBVCsaxhhjfGZFwxhjjM+saBhjjPGZFQ1jjDE++/9cc4GkDGd2IAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from mlxtend.plotting import plot_decision_regions\n",
    "\n",
    "\n",
    "knn_model = KNeighborsClassifier(n_neighbors=3)\n",
    "knn_model.fit(X_train[:, 2:], y_train)\n",
    "plot_decision_regions(X_train[:, 2:], y_train, knn_model)\n",
    "plt.xlabel('petal length[cm]')\n",
    "plt.ylabel('petal width[cm]')\n",
    "plt.savefig('images/decisionreg.pdf')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stratification"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Previously, we wrote our own code to shuffle and split a dataset into training, validation, and test subsets, which had one considerable downside.\n",
    "- If we are working with small datasets and split it randomly into subsets, it will affect the class distribution in the samples -- this is problematic since machine learning algorithms/models assume that training, validation, and test samples have been drawn from the same distributions to produce reliable models and estimates of the generalization performance."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/iris-subsampling.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- The method of ensuring that the class label proportions are the same in each subset after splitting, we use an approach that is usually referred to as \"stratification.\"\n",
    "- Stratification is supported in scikit-learn's `train_test_split` method if we pass the class label array to the `stratify` parameter as shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([40, 40, 40])"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "\n",
    "X_temp, X_test, y_temp, y_test = \\\n",
    "        train_test_split(X, y, test_size=0.2, \n",
    "                         shuffle=True, random_state=123, stratify=y)\n",
    "np.bincount(y_temp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train size (96, 4) class proportions [32 32 32]\n",
      "Valid size (24, 4) class proportions [8 8 8]\n",
      "Test size (30, 4) class proportions [10 10 10]\n"
     ]
    }
   ],
   "source": [
    "X_train, X_valid, y_train, y_valid = \\\n",
    "        train_test_split(X_temp, y_temp, test_size=0.2,\n",
    "                         shuffle=True, random_state=123, stratify=y_temp)\n",
    "\n",
    "print('Train size', X_train.shape, 'class proportions', np.bincount(y_train))\n",
    "print('Valid size', X_valid.shape, 'class proportions', np.bincount(y_valid))\n",
    "print('Test size', X_test.shape, 'class proportions', np.bincount(y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Scaling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- In the case of the Iris dataset, all dimensions were measured in centimeters, hence \"scaling\" features would not be necessary in the context of *k*NN -- unless we want to weight features differently.\n",
    "- Whether or not to scale features depends on the problem at hand and requires your judgement.\n",
    "- However, there are several algorithms (especially gradient-descent, etc., which we will cover later in this course), which work much better (are more robust, numerically stable, and converge faster) if the data is centered and has a smaller range.\n",
    "- There are many different ways for scaling features; here, we only cover to of the most common \"normalization\" schemes: min-max scaling and z-score standardization."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Normalization -- Min-max scaling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Min-max scaling squashes the features into a [0, 1] range, which can be achieved via the following equation for a single input $i$:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$x^{[i]}_{\\text{norm}} = \\frac{x^{[i]} - x_{\\text{min}} }{ x_{\\text{max}} - x_{\\text{min}} }$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Below is an example of how we can implement and apply min-max scaling on 6 data instances given a 1D input vector (1 feature) via NumPy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0., 1., 2., 3., 4., 5.])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = np.arange(6).astype(float)\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0. , 0.2, 0.4, 0.6, 0.8, 1. ])"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_norm = (x - x.min()) / (x.max() - x.min())\n",
    "x_norm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Standardization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Z-score standardization is a useful standardization scheme if we are working with certain optimization methods (e.g., gradient descent, later in this course). \n",
    "- After standardizing a feature, it will have the properties of a standard normal distribution, that is, unit variance and zero mean ($N(\\mu=0, \\sigma^2=1)$); however, this does not transform a feature from not following a normal distribution to a normal distributed one.\n",
    "- The formula for standardizing a feature is shown below, for a single data point $x^{[i]}$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$x^{[i]}_{\\text{std}} = \\frac{x^{[i]} - \\mu_x }{ \\sigma_{x} }$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0., 1., 2., 3., 4., 5.])"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = np.arange(6).astype(float)\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-1.46385011, -0.87831007, -0.29277002,  0.29277002,  0.87831007,\n",
       "        1.46385011])"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_std = (x - x.mean()) / x.std()\n",
    "x_std"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Conveniently, NumPy and Pandas both implement a `std` method, which computes the standard devation.\n",
    "- Note the different results shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.1690451944500122"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame([1, 2, 1, 2, 3, 4])\n",
    "df[0].std()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0671873729054748"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[0].values.std()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- The results differ because Pandas computes the \"sample\" standard deviation ($s_x$), whereas NumPy computes the \"population\" standard deviation ($\\sigma_x$)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$s_x = \\sqrt{ \\frac{1}{n-1} \\sum^{n}_{i=1} (x^{[i]} - \\bar{x})^2 }$$\n",
    "\n",
    "$$\\sigma_x = \\sqrt{ \\frac{1}{n} \\sum^{n}_{i=1} (x^{[i]} - \\mu_x)^2 }$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- In the context of machine learning, since we are typically working with large datasets, we typically don't care about Bessel's correction (subtracting one degree of freedom in the denominator).\n",
    "- Further, the goal here is not to model a particular distribution or estimate distribution parameters accurately; however, if you like, you can remove the extra degree of freedom via NumPy's `ddof` parameters -- it's not necessary in practice though."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.1690451944500122"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[0].values.std(ddof=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- A concept that is very important though is how we use the estimated normalization parameters (e.g., mean and standard deviation in z-score standardization).\n",
    "- In particular, it is important that we re-use the parameters estimated from the training set to transfrom validation and test sets -- re-estimating the parameters is a common \"beginner-mistake\" which is why we discuss it in more detail."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "mu, sigma = X_train.mean(axis=0), X_train.std(axis=0)\n",
    "\n",
    "X_train_std = (X_train - mu) / sigma\n",
    "X_valid_std = (X_valid - mu) / sigma\n",
    "X_test_std = (X_test - mu) / sigma"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Again, if we standardize the training dataset, we need to keep the parameters (mean and standard deviation for each feature). Then, we’d use these parameters to transform our test data and any future data later on\n",
    "- Let’s assume we have a simple training set consisting of 3 samples with 1 feature column (let’s call the feature column “length in cm”):\n",
    "\n",
    "- example1: 10 cm -> class 2\n",
    "- example2: 20 cm -> class 2\n",
    "- example3: 30 cm -> class 1\n",
    "\n",
    "Given the data above, we estimate the following parameters from this training set:\n",
    "\n",
    "- mean: 20\n",
    "- standard deviation: 8.2\n",
    "\n",
    "If we use these parameters to standardize the same dataset, we get the following z-score values:\n",
    "\n",
    "- example1: -1.21 -> class 2\n",
    "- example2: 0 -> class 2\n",
    "- example3: 1.21 -> class 1\n",
    "\n",
    "Now, let’s say our model has learned the following hypotheses: It classifies samples with a standardized length value < 0.6 as class 2 (and class 1 otherwise). So far so good. Now, let’s imagine we have 3 new unlabeled data points that you want to classify.\n",
    "\n",
    "- example4: 5 cm -> class ?\n",
    "- example5: 6 cm -> class ?\n",
    "- example6: 7 cm -> class ?\n",
    "\n",
    "If we look at the non-standardized \"length in cm\" values in the training datast, it is intuitive to say that all of these examples (5, 6, and 7) are likely belonging to class 2  because they are smaller than anything in the training set. However, if we standardize these by re-computing the standard deviation and and mean from the new data, we will get similar values as before (i.e., properties of a standard normal distribtion) in the training set and our classifier would (probably incorrectly) assign the “class 2” label to the samples 4 and 5.\n",
    "\n",
    "- example5: -1.21 -> class 2\n",
    "- example6: 0 -> class 2\n",
    "- example7: 1.21 -> class 1\n",
    "\n",
    "However, if we use the parameters from the \"training set standardization,\" we will get the following standardized values\n",
    "\n",
    "- example5: -18.37\n",
    "- example6: -17.15\n",
    "- example7: -15.92\n",
    "\n",
    "Note that these values are more negative than the value of example1 in the original training set, which makes much more sense now!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Scikit-Learn Transformer API"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- The transformer API in scikit-learn is very similar to the estimator API; the main difference is that transformers are typically \"unsupervised,\" meaning, they don't make use of class labels or target values."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/transformer-api.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Typical examples of transformers in scikit-learn are the `MinMaxScaler` and the `StandardScaler`, which can be used to perform min-max scaling and z-score standardization as discussed earlier.\\"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import MinMaxScaler\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "\n",
    "scaler = StandardScaler()\n",
    "scaler.fit(X_train)\n",
    "X_train_std = scaler.transform(X_train)\n",
    "X_valid_std = scaler.transform(X_valid)\n",
    "X_test_std = scaler.transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Categorical Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- When we preprocess a dataset as input to a machine learning algorithm, we have to be careful how we treat categorical variables.\n",
    "- There are two broad categories of categorical variables: nominal (no order implied) and ordinal (order implied)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>color</th>\n",
       "      <th>size</th>\n",
       "      <th>price</th>\n",
       "      <th>classlabel</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>green</td>\n",
       "      <td>M</td>\n",
       "      <td>10.1</td>\n",
       "      <td>class1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>red</td>\n",
       "      <td>L</td>\n",
       "      <td>13.5</td>\n",
       "      <td>class2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>blue</td>\n",
       "      <td>XXL</td>\n",
       "      <td>15.3</td>\n",
       "      <td>class1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   color size  price classlabel\n",
       "0  green    M   10.1     class1\n",
       "1    red    L   13.5     class2\n",
       "2   blue  XXL   15.3     class1"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('categoricaldata.csv')\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- In the example above, 'size' would be an example of an ordinal variable; i.e., if the letters refer to T-shirt sizes, it would make sense to come up with an ordering like M < L < XXL.\n",
    "- Hence, we can assign increasing values to a ordinal values; however, the range and difference between categories depends on our domain knowledge and judgement.\n",
    "- To convert ordinal variables into a proper representation for numerical computations via machine learning algorithms, we can use the now familiar `map` method in Pandas, as shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>color</th>\n",
       "      <th>size</th>\n",
       "      <th>price</th>\n",
       "      <th>classlabel</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>green</td>\n",
       "      <td>2</td>\n",
       "      <td>10.1</td>\n",
       "      <td>class1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>red</td>\n",
       "      <td>3</td>\n",
       "      <td>13.5</td>\n",
       "      <td>class2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>blue</td>\n",
       "      <td>5</td>\n",
       "      <td>15.3</td>\n",
       "      <td>class1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   color  size  price classlabel\n",
       "0  green     2   10.1     class1\n",
       "1    red     3   13.5     class2\n",
       "2   blue     5   15.3     class1"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mapping_dict = {'M': 2,\n",
    "                'L': 3,\n",
    "                'XXL': 5}\n",
    "\n",
    "df['size'] = df['size'].map(mapping_dict)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Machine learning algorithms do not assume an ordering in the case of class labels.\n",
    "- Here, we can use the `LabelEncoder` from scikit-learn to convert class labels to integers as an alternative to using the `map` method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>color</th>\n",
       "      <th>size</th>\n",
       "      <th>price</th>\n",
       "      <th>classlabel</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>green</td>\n",
       "      <td>2</td>\n",
       "      <td>10.1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>red</td>\n",
       "      <td>3</td>\n",
       "      <td>13.5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>blue</td>\n",
       "      <td>5</td>\n",
       "      <td>15.3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   color  size  price  classlabel\n",
       "0  green     2   10.1           0\n",
       "1    red     3   13.5           1\n",
       "2   blue     5   15.3           0"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "\n",
    "le = LabelEncoder()\n",
    "df['classlabel'] = le.fit_transform(df['classlabel'])\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Representing nominal variables properly is a bit more tricky.\n",
    "- Since machine learning algorithms usually assume an order if a variable takes on integer values, we need to apply a \"trick\" here such that the algorithm would not make this assumption.\n",
    "- this \"trick\" is also called \"one-hot\" encoding -- we binarize a nominal variable, as shown below for the color variable (again, we do this because some ordering like orange < red < blue would not make sense in many applications)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      A     B     C    D\n",
       "0   1.0   2.0   3.0  4.0\n",
       "1   5.0   6.0   NaN  8.0\n",
       "2  10.0  11.0  12.0  NaN"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.get_dummies(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Note that executing the code above produced 3 new variables for \"color,\" each of which takes on binary values.\n",
    "- However, there is some redundancy now (e.g., if we know the values for `color_green` and `color_red`, we automatically know the value for `color_blue`).\n",
    "- While collinearity may cause problems (i.e., the matrix inverse doesn't exist in e.g., the context of the closed-form of linear regression), again, in machine learning we typically would not care about it too much, because most algorithms can deal with collinearity (e.g., adding constraints like regularization penalties to regression models, which we learn via gradient-based optimization).\n",
    "- However, removing collinearity if possible is never a bad idea, and we can do this conveniently by dropping e.g., one of the columns of the one-hot encoded variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>size</th>\n",
       "      <th>price</th>\n",
       "      <th>classlabel</th>\n",
       "      <th>color_green</th>\n",
       "      <th>color_red</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2</td>\n",
       "      <td>10.1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3</td>\n",
       "      <td>13.5</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5</td>\n",
       "      <td>15.3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   size  price  classlabel  color_green  color_red\n",
       "0     2   10.1           0            1          0\n",
       "1     3   13.5           1            0          1\n",
       "2     5   15.3           0            0          0"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.get_dummies(df, drop_first=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Missing Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- There are many different ways for dealing with missing data.\n",
    "- The simplest approaches are removing entire columns or rows.\n",
    "- Another simple approach is to impute missing values via the feature means, medians, mode, etc.\n",
    "- There is no rule or best practice, and the choice of the approprite missing data imputation method depends on your judgement and domain knowledge.\n",
    "- Below are some examples for dealing with missing data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      A     B     C    D\n",
       "0   1.0   2.0   3.0  4.0\n",
       "1   5.0   6.0   NaN  8.0\n",
       "2  10.0  11.0  12.0  NaN"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('missingdata.csv')\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "A    0\n",
       "B    0\n",
       "C    1\n",
       "D    1\n",
       "dtype: int64"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# missing values per column:\n",
    "\n",
    "df.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     A    B    C    D\n",
       "0  1.0  2.0  3.0  4.0"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# drop rows with missing values:\n",
    "\n",
    "df.dropna(axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5.0</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10.0</td>\n",
       "      <td>11.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      A     B\n",
       "0   1.0   2.0\n",
       "1   5.0   6.0\n",
       "2  10.0  11.0"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# drop columns with missing values:\n",
    "\n",
    "df.dropna(axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1. ,  2. ,  3. ,  4. ],\n",
       "       [ 5. ,  6. ,  7.5,  8. ],\n",
       "       [10. , 11. , 12. ,  6. ]])"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.preprocessing import Imputer\n",
    "\n",
    "\n",
    "imputer = Imputer(missing_values='NaN', strategy='mean', axis=0)\n",
    "X = df.values\n",
    "X = imputer.fit_transform(df.values)\n",
    "X"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature Transformation, Extraction, and Selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have already covered very simple cases of feature transformation, i.e., normalization, that is, min-max scaling and standardization. There are many other cases, but an extensive coverage of feature preprocessing is beyond the scope of a machine learning class. However, we will will look at some popular feature selection (sequential feature selection) and feature extraction (e.g., principal component analysis) techniques later in this course."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Scikit-Learn Pipelines"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Scikit-learn pipelines are an extremely convenient and powerful concept -- one of the things that sets scikit-learn apart from other machine learning libraries.\n",
    "- Pipelines basically let us define a series of perprocessing steps together with fitting an estimator.\n",
    "- Pipelines will automatically take care of pitfalls like estimating feature scaling parameters from the training set and applying those to scale new data (which we discussed earlier in the context of z-score standardization).\n",
    "- Below is an visualization of how pipelines work."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/sklearn-pipeline.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Below is an example pipeline that combines the feature scaling step with the *k*NN classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.pipeline import make_pipeline\n",
    "\n",
    "\n",
    "pipe = make_pipeline(StandardScaler(),\n",
    "                     KNeighborsClassifier(n_neighbors=3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pipeline(memory=None,\n",
       "     steps=[('standardscaler', StandardScaler(copy=True, with_mean=True, with_std=True)), ('kneighborsclassifier', KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
       "           metric_params=None, n_jobs=1, n_neighbors=3, p=2,\n",
       "           weights='uniform'))])"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pipe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 0, 2, 2, 0, 0, 2, 1, 2, 0, 0, 2, 2, 1, 2, 1, 0, 0, 0, 0, 0, 2,\n",
       "       2, 1, 2, 2, 1, 1, 1, 1])"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pipe.fit(X_train, y_train)\n",
    "pipe.predict(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- As you can see above, the Pipeline itself follows the scikit-learn estimator API."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(Also see the [FunctionTransformer](http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.FunctionTransformer.html) in scikit-learn, which allows creating a transformer class from an arbitrary callable or function.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Intro Model Selection -- Pipelines and Grid Search"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- In machine learning practice, we often need to experiment with an machine learning algorithm's hyperparameters to find a good setting.\n",
    "- The process of tuning hyperparameters and comparing and selecting the resulting models is also called \"model selection\" (in contrast to \"algorithm selection\").\n",
    "- We will cover topics such as \"model selection\" and \"algorithm selection\" in more detail later in this course.\n",
    "- For now, we are introducing the simplest way of performing model selection: using the \"holdout method.\"\n",
    "- In the holdout method, we split a dataset into 3 subsets: a training, a validation, and a test datatset.\n",
    "- To avoid biasing the estimate of the generalization performance, we only want to use the test dataset once, which is why we use the validation dataset for hyperparameter tuning (model selection).\n",
    "- Here, the validation dataset serves as an estimate of the generalization performance, too, but it becomes more biased than the final estimate on the test data because of its repeated re-use during model selection (think of \"multiple hypothesis testing\").\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/holdout-tuning.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/sebastian/miniconda3/lib/python3.6/site-packages/sklearn/model_selection/_search.py:762: DeprecationWarning: The grid_scores_ attribute was deprecated in version 0.18 in favor of the more elaborate cv_results_ attribute. The grid_scores_ attribute will not be available from 0.20\n",
      "  DeprecationWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[mean: 0.90000, std: 0.00000, params: {'kneighborsclassifier__n_neighbors': 1, 'kneighborsclassifier__p': 1},\n",
       " mean: 0.96667, std: 0.00000, params: {'kneighborsclassifier__n_neighbors': 1, 'kneighborsclassifier__p': 2},\n",
       " mean: 0.96667, std: 0.00000, params: {'kneighborsclassifier__n_neighbors': 3, 'kneighborsclassifier__p': 1},\n",
       " mean: 0.93333, std: 0.00000, params: {'kneighborsclassifier__n_neighbors': 3, 'kneighborsclassifier__p': 2},\n",
       " mean: 0.90000, std: 0.00000, params: {'kneighborsclassifier__n_neighbors': 5, 'kneighborsclassifier__p': 1},\n",
       " mean: 0.90000, std: 0.00000, params: {'kneighborsclassifier__n_neighbors': 5, 'kneighborsclassifier__p': 2}]"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "from mlxtend.evaluate import PredefinedHoldoutSplit\n",
    "from sklearn.pipeline import make_pipeline\n",
    "from sklearn.datasets import load_iris\n",
    "\n",
    "iris = load_iris()\n",
    "X, y = iris.data, iris.target\n",
    "\n",
    "train_ind, valid_ind = train_test_split(np.arange(X.shape[0]),\n",
    "                                        test_size=0.2, shuffle=True,\n",
    "                                        random_state=123, stratify=y)\n",
    "\n",
    "pipe = make_pipeline(StandardScaler(),\n",
    "                     KNeighborsClassifier())\n",
    "\n",
    "params = {'kneighborsclassifier__n_neighbors': [1, 3, 5],\n",
    "          'kneighborsclassifier__p': [1, 2]}\n",
    "\n",
    "split = PredefinedHoldoutSplit(valid_indices=valid_ind)\n",
    "\n",
    "grid = GridSearchCV(pipe,\n",
    "                    param_grid=params,\n",
    "                    cv=split)\n",
    "\n",
    "grid.fit(X, y)\n",
    "grid.grid_scores_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reading Assignment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Python Machine Learning 2nd ed.: Ch04 up to \"Selecting Meaningful Features\" (pg 107-123)\n",
    "- Python Machine Learning 2nd ed.: Ch06 up to \"Debugging algorithms with learning and validation curves\" (pg 185-194)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Further Resources"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Scikit-learn documentation: http://scikit-learn.org/stable/documentation.html"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  },
  "toc-autonumbering": false
 },
 "nbformat": 4,
 "nbformat_minor": 2
}