NPL-Algoritms-Final-Solution.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "196fa89b",
   "metadata": {},
   "source": [
    "## Insert Packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "7ecee356",
   "metadata": {},
   "outputs": [],
   "source": [
    "#handling\n",
    "import csv\n",
    "import numpy as np\n",
    "from numpy import genfromtxt\n",
    "import pandas as pd\n",
    "from time import time\n",
    "\n",
    "#Models\n",
    "from sklearn import svm\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.ensemble import AdaBoostClassifier\n",
    "from sklearn.ensemble import BaggingClassifier\n",
    "from sklearn.ensemble import GradientBoostingClassifier\n",
    "from sklearn.naive_bayes import GaussianNB\n",
    "from sklearn.discriminant_analysis import LinearDiscriminantAnalysis\n",
    "from sklearn.neural_network import MLPClassifier\n",
    "\n",
    "#Scores\n",
    "from sklearn.metrics import f1_score, accuracy_score, log_loss\n",
    "\n",
    "#FS  !pip install mlxtend\n",
    "from mlxtend.feature_selection import SequentialFeatureSelector as sfs\n",
    "from mlxtend.plotting import plot_sequential_feature_selection as plot_sfs\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "#Graphs\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "173fa99a",
   "metadata": {},
   "source": [
    "## Insert Data Extracted on previous process"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "370bd9b0",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Training Data\n",
    "training_set = genfromtxt('training_features_scaled.csv', delimiter=',') #,skip_header=1\n",
    "labels = genfromtxt('labels.csv', delimiter=',') #labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "c7db3d9a",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Testing Data\n",
    "testing_set = genfromtxt('testing_features_scaled.csv', delimiter=',')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "fb06ab8b",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Solutions to compare\n",
    "with open(\"solutions.csv\", \"r\") as s:\n",
    "    reader = csv.reader(s)    \n",
    "    solutions  = list(reader)\n",
    "sol = [element[1] for element in solutions] \n",
    "sol = sol [1:len(sol)] \n",
    "for i in range(0, len(sol)):\n",
    "    sol[i] = int(sol[i])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "89031dda",
   "metadata": {},
   "source": [
    "## Pearson correlation coefficient"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9844a6fc",
   "metadata": {},
   "source": [
    "Define the Pearson correlation coefficient between certain variables to see if the result of feature selection make sense ( certain features are being eliminated due to their correlation with the chosen ones)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "b6ca576b",
   "metadata": {},
   "outputs": [],
   "source": [
    "df=pd.DataFrame(training_set,columns=['overlap_title','temp_diff','comm_auth', 'overlap_journ','overlap_abstr','cos_sim_abstr', 'cos_sim_title', 'cos_sim_author', 'cos_sim_journal','jac_sim', 'pref_attac', 'adam_index','com_neigh', 'same_cluster','commun_ra'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "7bd10108",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The correlation between overlap_journ and cos_sim_journal is : 0.6961093120203985\n",
      "\n",
      "The correlation between cos_sim_abstr and overlap_abstr is : 0.657352002806121\n",
      "\n",
      "The correlation between cos_sim_author and comm_auth is : 0.6591472522652183\n",
      "\n",
      "The correlation between adam_index and jac_sim is : 0.5917660360358773\n",
      "\n",
      "The correlation between adam_index and com_neigh is : 0.9916100524582747\n"
     ]
    }
   ],
   "source": [
    "print('The correlation between overlap_journ and cos_sim_journal is :',(df['overlap_journ']).corr(df['cos_sim_journal']))\n",
    "print('\\nThe correlation between cos_sim_abstr and overlap_abstr is :',(df['cos_sim_abstr']).corr(df['overlap_abstr']))\n",
    "print('\\nThe correlation between cos_sim_author and comm_auth is :',(df['cos_sim_author']).corr(df['comm_auth']))\n",
    "print('\\nThe correlation between adam_index and jac_sim is :',(df['adam_index']).corr(df['jac_sim']))\n",
    "print('\\nThe correlation between adam_index and com_neigh is :',(df['adam_index']).corr(df['com_neigh']))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a62ca39d",
   "metadata": {},
   "source": [
    "We expect one feature of each pair to be excluded. If the whole pair is excluded then that happens due to poor performance."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1a0c6dc",
   "metadata": {},
   "source": [
    "## Feature Selection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "92ba2ac6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# train data into a dataframe\n",
    "df=pd.DataFrame(training_set,columns=['overlap_title','temp_diff','comm_auth', 'overlap_journ','overlap_abstr','cos_sim_abstr', 'cos_sim_title', 'cos_sim_author', 'cos_sim_journal','jac_sim', 'pref_attac', 'adam_index','com_neigh', 'same_cluster','commun_ra'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "4cb2f12c",
   "metadata": {},
   "outputs": [],
   "source": [
    "def fs(number_f,kind):\n",
    "    if kind==True:\n",
    "                \n",
    "    #Sequential Forward Selection \n",
    "        SFS = sfs(svm.LinearSVC(),\n",
    "                  k_features=number_f,  # Number of features to be selected\n",
    "                  forward=True,   \n",
    "                  floating=False,      # Do not exclude features from the previous iteration\n",
    "                  scoring='f1',\n",
    "                  cv = None)             # None, to use the default 5-fold cross validation\n",
    "\n",
    "        # Place all the feature names into a list\n",
    "        feature_names=['overlap_title','temp_diff','comm_auth', 'overlap_journ','overlap_abstr','cos_sim_abstr', 'cos_sim_title', 'cos_sim_author', 'cos_sim_journal','jac_sim', 'pref_attac', 'adam_index','com_neigh', 'same_cluster','commun_ra']    \n",
    "        # Apply the fit\n",
    "        SFS.fit(training_set, labels,custom_feature_names=feature_names) \n",
    "        \n",
    "        \n",
    "        #Plot results\n",
    "        fig = plot_sfs(SFS.get_metric_dict(), kind='std_err')  # standard errors of the cross-validation scores\n",
    "        plt.ylim([0.8, 1])\n",
    "        plt.title('Sequential Forward Selection (with standard errors)')\n",
    "        plt.grid()\n",
    "        plt.show()\n",
    "        \n",
    "    else:\n",
    "        \n",
    "        #Sequential Backward Selection \n",
    "        SFS = sfs(svm.LinearSVC(),\n",
    "                  k_features=number_f,    # Number of features to be selected\n",
    "                  forward=False,   \n",
    "                  floating=False,        # Do not include features from the previous iteration\n",
    "                  scoring='f1',\n",
    "                  cv = None)   # None, to use the default 5-fold cross validation,\n",
    "\n",
    "        # Place all the feature names into a list\n",
    "        feature_names=['overlap_title','temp_diff','comm_auth', 'overlap_journ','overlap_abstr','cos_sim_abstr', 'cos_sim_title', 'cos_sim_author', 'cos_sim_journal','jac_sim', 'pref_attac', 'adam_index','com_neigh', 'same_cluster','commun_ra']    \n",
    "        # Apply the fit\n",
    "        SFS.fit(training_set, labels,custom_feature_names=feature_names) \n",
    "\n",
    "        \n",
    "        #Plot results\n",
    "        fig = plot_sfs(SFS.get_metric_dict(), kind='std_err')  # standard errors of the cross-validation scores\n",
    "        plt.ylim([0.8, 1])\n",
    "        plt.title('Sequential Backward Selection (with standard errors)')\n",
    "        plt.grid()\n",
    "        plt.show()\n",
    "\n",
    "        \n",
    "    #Create a dataframe for the SFS results \n",
    "    df_SFS_results = pd.DataFrame(SFS.subsets_).transpose()\n",
    "    print('\\n',df_SFS_results)\n",
    "    \n",
    "\n",
    "    print('\\nIndex of features : ',SFS.k_feature_idx_)  #index of feature selected features      \n",
    "    print('\\n',SFS.k_score_)             # CV Score\n",
    "\n",
    "    return(SFS.k_feature_names_)     #name of features selected  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3deee244",
   "metadata": {},
   "source": [
    "In this phase forward and backward feature selection were executed (with the sampling dataset) and the results pointed that the optimal number of features were 6. Due to run time, the whole dataset is going to be tested on backward feature selection for 6 features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "69d8b240",
   "metadata": {},
   "outputs": [],
   "source": [
    "#%%time\n",
    "#fs(1,False)  # Backward Feature Selection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "883cc0e4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# %%time\n",
    "# fs(15,True)  # Forward Feature Selection"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6575d274",
   "metadata": {},
   "source": [
    "Optimal number of features is 6. Run backward feature selection to view them. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "2652c0b0",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\numpy\\core\\_methods.py:261: RuntimeWarning: Degrees of freedom <= 0 for slice\n",
      "  ret = _var(a, axis=axis, dtype=dtype, out=out, ddof=ddof,\n",
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\numpy\\core\\_methods.py:253: RuntimeWarning: invalid value encountered in double_scalars\n",
      "  ret = ret.dtype.type(ret / rcount)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAvEklEQVR4nO3de7xVdZ3/8ddbEAEBwbQTKoIXRmUctY4/rMlRiS7oWGrpjE6ZmoZO0b0ZL82vaIrGmsrpV/4iM7wrWYlhMl5+htrNETQUEElEUsS8E+IN0c/vj+/3xDqbvc/e63D2ORt5Px+P9Th7re/6ftdn7b3O+uz1XZetiMDMzKxRW/V1AGZmtnlx4jAzs1KcOMzMrBQnDjMzK8WJw8zMSnHiMDOzUpw4XickrZW0ewPzjZEUkvr3RlyF5V4s6au9ucwqMRwmaWUT2m3Keypp1/y59uvJdgvtXyXp6JJ1zpF0YRflJ0v69SYH12Q9HaekqZIu76n2mknSNZImbUobW3zikHSwpN9K+rOkZyT9RtL/6uu4uiLpVkmnFadFxJCIWN4Dba+Q9GLeYT0r6XpJoza13c2BpKMkLZC0RtJTkm6RNKYXl79C0js7xiPi4fy5vtqEZe0H7A/8vEy9iPhaRJyW22jal5DNaUe8GToXmLYpDWzRiUPSMOAXwHeB7YGdgS8DL/dlXC3gvRExBBgJPE56fzYrZXdmkvYELgU+B2wH7Ab8X+C1no+uJZwOXBG+A7i0Zh6tV2u7G9tyl/NHxJ3AMEkHlgzvL7boxAH8FUBEXBURr0bEixFxU0Tc2zGDpI9IWpK/fd8oaXSh7F2S7s9HK9+TdFvHkUDlN6bKb2eStpP0I0mPSXpU0lc7uiQ6DqMlfTMv9yFJh+eyacDfAd/LRwXfy9Mj7/yQ9PeSfp+/OT8iaWp33pyIeAn4KTCusB5dtl04gludy0+ubFfSUElzJf0fSbvlebfKZRdKeqIw7+WSPp1fn5I/i+ckLZd0emG+wyStlHSmpD8BF0kapNRF9qyk+4CujiQPAB6KiFsieS4ifhYRD+f2t5J0lqQHJT0t6WpJ21drqKvPNpd/tLAe90l6i6TLgF2B6/Ln+q9VtpmdJM1WOjJeJumjhTan5pguze0urrNjOBy4rVD/j5La8+sP5eWOy+OnSbq2sJyO7fr2/Hd1jvlthfY22nZrvFdn5vfoOUlLJU1U6kY5B/jH3O49ed5GPv/PSXoiv/enFMrfkN+7NZLuBPaoiOM7eXtdI+kuSX9X8d7+NG+La4CT83Z7W47lZmCHLt5rJB2pdDS7Ov9/7FcoW5Hfh3uB5yXtmd//UyU9DPwyb3//lj+nJ/LnvF2uP6bK/ANzvE/nZc6T1FYI6Vbg77uKuUsRscUOwDDgaeAS0j/SiIryo4FlwD5Af+DfgN/msh2ANcCxwNbAZ4D1wGm5fCpweaGtMUAA/fP4tcAPgG2BNwJ3AqfnspOBV4CPAv2AfwZWAcrlt3Ysp9B+AHvm14cBf0P6YrAf6ajh6GpxVHlPVgDvzK8H5/fm0kJ5V23vCjwHnJDfkzcAB+Syi4Gv5ml3Al8ttPkw0J5fLwWWA/sUyt6cX/896R9ewKHAC8BbCnGtB74ObAMMIh2S/4p0NDkKWASsrLHeuwMvAecBE4AhFeWfBu4Adsnt/wC4qhuf7XHAo6QkJmBPYHTle1+j3dtIR0EDSYnuSWBiYXt7CTiCtM38B3BHjXXdNre7Y2HapcDn8usLgAeBfy6UfaZyu66Mr5FttyKOvYBHgJ0K7e1R7f+nxOf/76Rt74hcPiKXzwSuzuu+b/4Mfl1o+0OkbbM/6ajzT8DAQiyvkPYHW5G2rd8B3yZtC4eQtvvLa7zfbwGeAA7K78lJ+bPepvC5LyBto4MK7+ulOd5BwEdI+6LdgSHANcBlFZ9Dcf7TgetI/8P9gHZgWCGmzwLXdHvf2Vc77VYZSEnhYmBl3vBmA2257L+BUwvzbpU3xtHAhyn8Y+aNeSUNJA6gjdQdNqhQfgIwt/DPt6xQNjjXfVMev5UuEkeVdfwv4Lxa/+wV864A1gKr8/uxCvibLt6/YttnA7NqzHcxMIO08/6XirLL8ob8JlLi+AZwBqm7aDWwVY02rwU+lV8fBqwj/7PnacuBSYXxydRIHLn8raSdy5OknfDF5AQCLCHvpPP4SNLOpH/Jz/bGjphrvPdVEwdpp/IqMLRQ/h/AxYXt7f8VysYBL9ZYzs653eJ7dSowu7CupwEz8/gf2bCDnkr9xFFz262IY0/SDvWdwNYVZX9ZThefV+Xn/2JFLE/kz7Rf/qz2LpR9jULiqNL2s8D+hVhuL5TtSvrf2LYw7cpa8QLfB75SMW0pcGjhc/9Ilc9998K0W4CPFcb3YuPtrzj/R4DfAvvViOmjwC+7en+7Grb0rioiYklEnBwRu5C+iexE2hlCShDfyYd6q4FnSAli5zzfI4V2ojhex2jSt6LHCm3/gPTttMOfCm2/kF8OaaRxSQcpdQU9KenPpJ1wl4fSFY6OiOGkb1NTgNskvamBtkeRvqnW8vekb0PTK6bfRvrHP4TU/XEr6RvlocCvIuK1vOzDJd2Ru2pWk75VFtfryUjdax06fUakHWBNEXFHRPxDROxI6g48BPhCLh4NzCp8XktIO/K2imbqfbb13qNadgKeiYjnKtZn58L4nwqvXwAGqnp/9+r8d2hh2m3A3+XPuR/wY+DtShcHbEf6RtyohrbdiFhGOpKbCjwhaaaknWo12sDn/3RErC+Mv5CXuyNpB1tzW8hdXEuUup1Xk9a52Hax7k7AsxHxfK32KowGPtexPeT2R+V2qrVfa5nFZfyRDV9Uqs1/GelLykxJqyR9Q9LWhfKhbNgOStviE0dRRNxP+pa5b570CKmLYXhhGBQRvwUeI334AEhScRx4nvRtq8ObCq8fIX0r3aHQ7rCI+OtGQ61TfiXpyGlURGxH2lGrwbY3LCSd97mGtIM8uIG2H6Gi77jCD4EbgDmSti1Mv420oz4sv/418HZS4rgNQNI2wM+Ab5KOCIcDcyrWq/J96fQZkb4pNiQi5pG6A4rbwuEV28LAiHi0omq9z7ar96irz3UVsL2k4s5+V1KXSyl5h/cg+RxfnraMtKP9JOnb9XOkBDCZ9M282kUC9bbDRmK5MiIOJu1cg9TVuFHbDX7+tTxJOkKoui3k8xlnAv9A6toaDvyZ2tvWY8CIim24q23rEWBaxbYzOCKuqtF+tWmrSO9RcXnrSV3FG80fEa9ExJcjYhzwt8CRpF6SDvsA93QRc5e26MQhae/8TWOXPD6K1K1wR55lOnC2pL/O5dtJOi6XXQ/8taT35291n6RzclgAHKJ0Lf52pG4cACLiMeAm4FuShuUTX3tIOrTB0B8n9XXWMpT07fQlSeOBf2qw3U6UHAWMIH3Drtf2FcA7Jf2DpP75hOQBFc1OIR2m/0LSIICIeIDUzfAh0k5rTV7HD7DhBO4A0hHQk8B6pROu766zCleTPr8R+TP+RBfrerDSSes35vG9gffReVuYpnxxhKQd83vTSQOf7YXA5yW15/d3T2244KLm5xoRj5C6Hv4jn/jcj9S9dEWd96CWOaTEXHQb+Qgzj99aMV7pSdJVZ3XvH6pG0l6S3pGTwkukbaDj0uPHgTHKF03Qvc8fSF+ASF8CpkoarHTS/6TCLENJO+Engf6Svkg6/1mrvT8C84EvSxog6WDgvV2E8EPgjHy0LknbKl1kMrSLOpWuAj6jdFJ+CKmr7ccVR1h/IWmCpL9RuihjDalbq3hZ96Gkrvhu2aITB+mE1kHA/0h6nrSTWEQ6OUZEzCJ9A5qpdDXFItJJdCLiKdKJznNJJ9jHAr/paDgibiYd7t8L3EW67Lfow6R/hvtI/ak/JfWbN+I7wLFKV638nyrlHwP+XdJzwBdJO9AyrpO0lrTBTQNOiojF9dqOdAXSEaT37xlS8ty/2HDu0ptM+hb2c0kDc9FtpK6GhwvjAn6f6z1HSs5Xk96vfyId+XTly6RD+odIO/PLuph3NSlRLMzrfgMwi3S+BdJ7Phu4Ka/7HaRtp5qan21E/IT0nl5J2v6uJZ28h3TO4t9yd8bnq7R7Aqk/e1WO7Ut5O+uOC4AP5iPlDreRdqK31xjvJHdDTQN+k2N+a8kYtiH9/zxFOrp5I+lqKoCf5L9PS7q7m59/0RRSt9WfSL0KFxXKbiTtRP9A2l5eon638z+RPv9ngC+RTkxXFRHzSecUvpdjX0Y6F1TGDNL2eztpe36JLr4Ikb7E/pT0P7yE9FleDqB0n9rzkS7L7ZaOq3SsB0i6lXSCrOadtWatQtKVwNURcW1fx2K9R9LPgB9FxJzuttGrj50ws9YREd3qwrTNW0R8YFPbaFpXlaQZSjeqLKpRLqUbwJZJulfSWwplk5RuBlom6azC9O0l3Szpgfx3RLPiNzOz6pp5juNioKsHaR1OOi8wltTn/X2AfDLn/Fw+Djghn8wCOAu4JSLGkq5rPquy0b4UEYe5m8rMXu+aljgi4nbSiaNajiLdkRwRcQcwXNJIYDzpBqLlEbGOdMfnUYU6l+TXl5Du5DQzs17Ul+c4dqbzlQsr87Rq0zuuXmnLlzsSEY91XDpZjaTJpCMZBg0a1D5qVPce8Praa6+x1VZ9f/GZ42i9OFohBsfhOJoZxx/+8Ien8g2xnXX3lvNGBtKlg4tqlF0PHFwYv4X0PJXjgAsL008Evptfr65o49lG4mhvb4/umjt3brfr9iTH0VkrxNEKMUQ4jkqOo7NNiQOYHy32yJGVdL6TcxfS9em1pgM8nruzyH+fwMzMelVfJo7ZwIfz1VVvBf4cqRtqHjA23yE5ADieDTf6zGbDHZ8nUfJHaMzMbNM18wdJriI9e2gHpZ/r/BLp4W9ExHTSIw+OIN1F+QJwSi5bL2kK6W7OfsCM2HDX8rnA1ZJOJT1u+zjMzKxXNS1xRMQJdcoD+HiNsjmkxFI5/WlgYo8EaGZm3dL3p/zNzGyz4sRhZmalOHGYmVkpThxmZlaKE4eZmZXixGFmZqU4cZiZWSlOHGZmVooTh5mZleLEYWZmpThxmJlZKU4cZmZWihOHmZmV4sRhZmalOHGYmVkpThxmZlaKE4eZmZXixGFmZqU0NXFImiRpqaRlks6qUj5C0ixJ90q6U9K+efpekhYUhjWSPp3Lpkp6tFB2RDPXwczMOmvab45L6gecD7wLWAnMkzQ7Iu4rzHYOsCAijpG0d55/YkQsBQ4otPMoMKtQ77yI+GazYjczs9qaecQxHlgWEcsjYh0wEziqYp5xwC0AEXE/MEZSW8U8E4EHI+KPTYzVzMwa1MzEsTPwSGF8ZZ5WdA/wfgBJ44HRwC4V8xwPXFUxbUru3pohaUTPhWxmZvUoIprTsHQc8J6IOC2PnwiMj4hPFOYZBnwHeDOwENgbOC0i7snlA4BVwF9HxON5WhvwFBDAV4CREfGRKsufDEwGaGtra585c2a31mPt2rUMGTKkW3V7kuNovThaIQbH4TiaGceECRPuiogDNyqIiKYMwNuAGwvjZwNndzG/gBXAsMK0o4CbuqgzBlhUL5b29vborrlz53a7bk9yHJ21QhytEEOE46jkODrblDiA+VFln9rMrqp5wFhJu+Ujh+OB2cUZJA3PZQCnAbdHxJrCLCdQ0U0laWRh9BhgUY9HbmZmNTUtcUTEemAKcCOwBLg6IhZLOkPSGXm2fYDFku4HDgc+1VFf0mDSFVnXVDT9DUkLJd0LTAA+06x1sNZzxRUwZgy84x2HMmZMGt8SY3AcjqNP46h2GPJ6GzbnrqrLL48YPTpCei1Gj07jW2ocl18eMXhwBGwYBg/u3VhaIQbH4Th6Kw5qdFU17eR4KznwwANj/vz53ap76623cthhh/VsQA264gqYPBleeGHDtMGD4YIL4IMfbM4yI2D9+s7Dj38Mn/0svPjihvkGDoQvfhHe856N5+/u8MorXZdfdRU8//zGMQ8eDEceuSH+4rpUrlu91/XKbrqp8/vQYdAgmDixZ5fVVRu//jW89NLGcQwcCH/7t421uSnxdIzPnw8vv7xxHAMGQHt7uWVsynuzaFHafiptvTXsu+/G05ulVhwDBsCb3wwSbLVV+lt83ei0Rutce231/5XRo2HFisbXR1LVk+NNuwFwc3fFFfCFL8DDDx/KrrvCtGk9s7N+9dWUCJ5/fsNQOd4x7Utf6pw0II2ffjpcf33jO9xG51m/PsXXiJdegnPOSUNPktI/e//+Gw/V/hEgvSf33tu5jWqvuyprdL5qSaNj+qpVPbusrsqqJY2O6S+/3Hib1ZZRpm61pAGwbh0UL+Tp6c+hcvz3v68exyuvwC6VF/g3Ua041q2D4cPhtddSouv42/F6/fqNp1Wbr5Hy116r/b/y8MM9s54+4qii2jf9bbaBT3wCxo+vv9PvKhnU+ocva+zY6jvXWjvdTZnn05+uHoMEs2Y11l6jw1ZdnHUbMwb+WOU20LLfojZFK8TgOBxHb8XhI44SvvCFjb/pv/wyfLPGQ04GD07Dttt2HtraNp5WOV+1eh3T3vxmeOSRjZc3ejT84Q89v961nHde9Y1w113hqMpnATTRtGnVu+6mTduyYnAcjqPP46h24uP1NpQ9OS51PqnUMUgRixZFLF8e8fjjEWvXRrz6aqmmS3m9nWjrqVha4SR9X8fgOBxHb8RBjZPjfb5T742hbOIYPbp64hg9ulQzPeL1tBH2pL6+2q1VYohwHJUcR2eb2w2Am61p09JhXVFfHG5COiG/YgX88pe3sWJF866m2lziMLO+58RRxQc/mC55HT0apGD06OZeAmtmtjlx4qjB37DNzKpz4jAzs1KcOMzMrBQnDjMzK8WJw8zMSnHiMDOzUpw4zMysFCcOMzMrxYnDzMxKceIwM7NSnDjMzKyUpiYOSZMkLZW0TNJZVcpHSJol6V5Jd0rat1C2QtJCSQskzS9M317SzZIeyH9HNHMdzMyss6YlDkn9gPOBw4FxwAmSxlXMdg6wICL2Az4MfKeifEJEHBCdf4HqLOCWiBgL3JLHzcyslzTziGM8sCwilkfEOmAmUPl7ceNIO38i4n5gjKS2Ou0eBVySX18CHN1jEZuZWV1N+81xSccCkyLitDx+InBQREwpzPM1YGBEfFbSeOC3eZ67JD0EPAsE8IOIuCDXWR0RwwttPBsRG3VXSZoMTAZoa2trnzlzZrfWY+3atQwZMqRbdXuS42i9OFohBsfhOJoZx4QJE6r+5njTfnUPOA64sDB+IvDdinmGARcBC4DLgHnA/rlsp/z3jcA9wCF5fHVFG8/Wi6XsLwAWvR5+xasnOY7WiiHCcVRyHJ014xcA+3crDTVmJTCqML4LsKo4Q0SsAU4BkCTgoTwQEavy3yckzSJ1fd0OPC5pZEQ8Jmkk8EQT18HMzCo08xzHPGCspN0kDQCOB2YXZ5A0PJcBnAbcHhFrJG0raWieZ1vg3cCiPN9s4KT8+iTg501cBzMzq9C0I46IWC9pCnAj0A+YERGLJZ2Ry6cD+wCXSnoVuA84NVdvA2algxD6A1dGxA257FzgakmnAg+TusTMzKyXNLOrioiYA8ypmDa98Pp3wNgq9ZYD+9do82lgYs9GamZmjfKd42ZmVooTh5mZleLEYWZmpThxmJlZKU4cZmZWihOHmZmV4sRhZmalOHGYmVkpThxmZlaKE4eZmZXixGFmZqU4cZiZWSlOHGZmVooTh5mZleLEYWZmpThxmJlZKU4cZmZWihOHmZmV0tTEIWmSpKWSlkk6q0r5CEmzJN0r6U5J++bpoyTNlbRE0mJJnyrUmSrpUUkL8nBEM9fBzMw6a9pvjkvqB5wPvAtYCcyTNDsi7ivMdg6wICKOkbR3nn8isB74XETcLWkocJekmwt1z4uIbzYrdjMzq62hIw5JgyX9b0k/zONjJR1Zp9p4YFlELI+IdcBM4KiKecYBtwBExP3AGEltEfFYRNydpz8HLAF2bnitzMysaRQR9WeSfgzcBXw4IvaVNAj4XUQc0EWdY4FJEXFaHj8ROCgiphTm+RowMCI+K2k88Ns8z12FecYAtwP7RsQaSVOBk4E1wHzSkcmzVZY/GZgM0NbW1j5z5sy661nN2rVrGTJkSLfq9iTH0XpxtEIMjsNxNDOOCRMm3BURB25UEBF1B2B+/vv7wrR76tQ5DriwMH4i8N2KeYYBFwELgMuAecD+hfIhpIT1/sK0NqAf6WhpGjCjXvzt7e3RXXPnzu123Z7kODprhThaIYYIx1HJcXS2KXF07Psrh0bPcazLRxkBIGkP4OU6dVYCowrjuwCrKpLWGuCU3KaAh/KApK2BnwFXRMQ1hTqPd7zOXWe/aHAdzMysBzR6VdWXgBuAUZKuIJ2X+Nc6deYBYyXtJmkAcDwwuziDpOG5DOA04PZI3VECfgQsiYhvV9QZWRg9BljU4DqYmVkPaOiIIyJulnQ38FZAwKci4qk6ddZLmgLcSOpamhERiyWdkcunA/sAl0p6FbgPODVXfzupa2uhpAV52jkRMQf4hqQDSEc/K4DTG1xXMzPrAQ0lDknHAL+MiOvz+HBJR0fEtV3Vyzv6ORXTphde/w4YW6Xer0kJqlqbJzYSs5mZNUfDXVUR8eeOkYhYTeq+MjOzLUyjiaPafE27edDMzFpXo4ljvqRvS9pD0u6SziNdJmtmZluYRhPHJ4B1wI+BnwAvAR9vVlBmZta6Gr2q6nlgo4cUmpnZlqfRq6r+Cvg8MKZYJyLe0ZywzMysVTV6gvsnwHTgQuDV5oVjZmatrtHEsT4ivt/USMzMbLPQ6Mnx6yR9TNJISdt3DE2NzMzMWlKjRxwn5b//UpgWwO49G46ZmbW6Rq+q2q3ZgZiZ2eah4bu/8++BjwMGdkyLiEubEZSZmbWuRi/H/RJwGClxzAEOB34NOHGYmW1hGj05fiwwEfhTRJwC7A9s07SozMysZTWaOF6MiNeA9ZKGAU/gE+NmZlukRs9xzJc0HPgh6eGGa4E7mxWUmZm1rkavqvpYfjld0g3AsIi4t3lhmZlZqypzVdV+FJ5VJWnPiLimSXGZmVmLaugch6QZwAzgA8B783BkA/UmSVoqaZmkjZ6uK2mEpFmS7pV0Z77kt8u6+a71myU9kP+OaGQdzMysZzR6xPHWiBhXpmFJ/YDzgXcBK4F5kmZHxH2F2c4BFkTEMZL2zvNPrFP3LOCWiDg3J5SzgDPLxGZmZt3X6FVVv5NUKnEA44FlEbE8ItYBM4GjKuYZB9wCEBH3A2MktdWpexRwSX59CXB0ybjMzGwTKCLqzyQdAlwH/Al4GRAQEbFfF3WOBSZFxGl5/ETgoIiYUpjna8DAiPispPHAb4GDgN1q1ZW0OiKGF9p4NiI26q6SNBmYDNDW1tY+c+bMuutZzdq1axkyZEi36vYkx9F6cbRCDI7DcTQzjgkTJtwVEQduVBARdQdgGfA+0g59dMdQp85xwIWF8ROB71bMMwy4CFgAXAbMI91cWLMusLqijWfrxd/e3h7dNXfu3G7X7UmOo7NWiKMVYohwHJUcR2ebEgcwP6rsUxs9x/FwRMwumaxWAqMK47sAq4ozRMQa4BQASQIeysPgLuo+LmlkRDwmaSTpZkQzM+sljZ7juF/SlZJOkPT+jqFOnXnAWEm7SRoAHA90Sj6ShucygNOA23My6arubDY85v0k4OcNroOZmfWARo84BpHObby7MC2AmvdxRMR6SVOAG4F+wIyIWCzpjFw+HdgHuFTSq8B9wKld1c1NnwtcLelU4GFSt5aZmfWSuokjXxr7VET8S715K0XEHNLTdIvTphde/w4Y22jdPP1p0gMXzcysD9TtqoqIV4G39EIsZma2GWi0q2qBpNnAT4DnOyaGHzliZrbFaTRxbA88DbyjMK3LcxxmZvb61OjTcU9pdiBmZrZ5aPQhh7vkhxE+IelxST+TtEuzgzMzs9bT6H0cF5Hun9gJ2Jn0+JGLmhWUmZm1rkYTx44RcVFErM/DxcCOTYzLzMxaVKOJ4ylJH5LULw8fIp0sNzOzLUyjieMjwD+Qno77GHBsnmZmZluYLq+qkvT1iDiT9Ejz9/VSTGZm1sLqHXEcIWlr4OzeCMbMzFpfvfs4bgCeAraVtIb8A05s+CGnYU2Oz8zMWkyXRxwR8S8RsR1wfUQMi4ihxb+9FKOZmbWQuifH89Nxt+2FWMzMbDPQ6NNxX5C0XS/EY2ZmLa7Rhxy+BCyUdDOdn477yaZEZWZmLavRxHF9HszMbAvX6NNxL5E0CNg1IpY2OSYzM2thjT4d973AAtLluUg6IP+wU716kyQtlbRM0llVyreTdJ2keyQtlnRKnr6XpAWFYY2kT+eyqZIeLZQd0fjqmpnZpmq0q2oqMB64FSAiFkjarasK+Wqs84F3ASuBeZJmR8R9hdk+DtwXEe+VtCOwVNIV+ajmgEI7jwKzCvXOi4hvNhi7mZn1oEafVbU+Iv5cMS3q1BkPLIuI5RGxDpgJHFWljaGSBAwBngHWV8wzEXgwIv7YYKxmZtZEiqi3/wdJPwJuAc4CPgB8Etg6Is7oos6xwKSIOC2Pn0h65tWUwjxDSb/zsTcwFPjHiLi+op0ZwN0R8b08PhU4GVgDzAc+FxHPVln+ZGAyQFtbW/vMmTPrrmc1a9euZciQId2q25McR+vF0QoxOA7H0cw4JkyYcFdEHLhRQUTUHYDBwDRgXh6+CgysU+c44MLC+InAdyvmORY4j/QIkz2Bh4BhhfIBpEeetBWmtQH9SEdL04AZ9eJvb2+P7po7d2636/Ykx9FZK8TRCjFEOI5KjqOzTYkDmB9V9qn1no47EDgj79QXAm+LiMqupFpWAqMK47sAqyrmOQU4Nwe4TNJDpKOPO3P54aSjjcc7KhRfS/oh8IsG4zEzsx5Q7xzHJcCBpKRxOFDmhPQ8YKyk3SQNAI4ndUsVPUw6h4GkNmAvYHmh/ATgqmIFSSMLo8cAi0rEZGZmm6jeVVXjIuJv4C/nOe6sM/9fRMR6SVOAG0ldSzMiYrGkM3L5dOArwMWSFpK6q86MiKfy8gaTrsg6vaLpb0g6gHRifUWVcjMza6J6ieOVjhc5EZRqPCLmAHMqpk0vvF4FvLtG3ReAN1SZfmKpIMzMrEfVSxz759/hgHREMKj4uxzhR6ubmW1xukwcEdGvtwIxM7PNQ6M3AJqZmQFOHGZmVpITh5mZleLEYWZmpThxmJlZKU4cZmZWihOHmZmV4sRhZmalOHGYmVkpThxmZlaKE4eZmZXixGFmZqU4cZiZWSlOHGZmVooTh5mZleLEYWZmpTQ1cUiaJGmppGWSzqpSvp2k6yTdI2mxpFMKZSskLZS0QNL8wvTtJd0s6YH8d0Qz18HMzDprWuKQ1A84HzgcGAecIGlcxWwfB+6LiP2Bw4BvSRpQKJ8QEQdExIGFaWcBt0TEWOCWPG5mZr2kmUcc44FlEbE8ItYBM4GjKuYJYKgkAUOAZ4D1ddo9Crgkv74EOLrHIjYzs7oUEc1pWDoWmBQRp+XxE4GDImJKYZ6hwGxgb2Ao8I8RcX0uewh4lpRcfhARF+TpqyNieKGNZyNio+4qSZOByQBtbW3tM2fO7NZ6rF27liFDhnSrbk9yHK0XRyvE4DgcRzPjmDBhwl0VPT5JRDRlAI4DLiyMnwh8t2KeY4HzAAF7Ag8Bw3LZTvnvG4F7gEPy+OqKNp6tF0t7e3t019y5c7tdtyc5js5aIY5WiCHCcVRyHJ1tShzA/KiyT21mV9VKYFRhfBdgVcU8pwDX5BiX5cSxN0BErMp/nwBmkbq+AB6XNBIg/32iaWtgZmYbaWbimAeMlbRbPuF9PKlbquhhYCKApDZgL2C5pG1zNxaStgXeDSzKdWYDJ+XXJwE/b+I6mJlZhf7Najgi1kuaAtwI9ANmRMRiSWfk8unAV4CLJS0kdVedGRFPSdodmJXOmdMfuDIibshNnwtcLelUUuI5rlnrYGZmG2ta4gCIiDnAnIpp0wuvV5GOJirrLQf2r9Hm0+SjFDMz632+c9zMzEpx4jAzs1KcOMzMrBQnDjMzK8WJw8zMSnHiMDOzUpw4zMysFCcOMzMrxYnDzMxKceIwM7NSnDjMzKwUJw4zMyvFicPMzEpx4jAzs1KcOMzMrBQnDjMzK8WJw8zMSnHiMDOzUpqaOCRNkrRU0jJJZ1Up307SdZLukbRY0il5+ihJcyUtydM/VagzVdKjkhbk4YhmroOZmXXWtN8cl9QPOB94F7ASmCdpdkTcV5jt48B9EfFeSTsCSyVdAawHPhcRd0saCtwl6eZC3fMi4pvNit3MzGpr5hHHeGBZRCyPiHXATOCoinkCGCpJwBDgGWB9RDwWEXcDRMRzwBJg5ybGamZmDWpm4tgZeKQwvpKNd/7fA/YBVgELgU9FxGvFGSSNAd4M/E9h8hRJ90qaIWlETwduZma1KSKa07B0HPCeiDgtj58IjI+ITxTmORZ4O/BZYA/gZmD/iFiTy4cAtwHTIuKaPK0NeIp0tPIVYGREfKTK8icDkwHa2traZ86c2a31WLt2LUOGDOlW3Z7kOFovjlaIwXE4jmbGMWHChLsi4sCNCiKiKQPwNuDGwvjZwNkV81wP/F1h/Jek5AKwNXAj8NkuljEGWFQvlvb29uiuuXPndrtuT3IcnbVCHK0QQ4TjqOQ4OtuUOID5UWWf2syuqnnAWEm7SRoAHA/MrpjnYWAi/OVIYi9geT7n8SNgSUR8u1hB0sjC6DHAoibFb2ZmVTTtqqqIWC9pCumooR8wIyIWSzojl08ndTVdLGkhIODMiHhK0sHAicBCSQtyk+dExBzgG5IOIHVVrQBOb9Y6mJnZxpqWOADyjn5OxbTphdergHdXqfdrUiKp1uaJPRymmZmV4DvHzcysFCcOMzMrxYnDzMxKceIwM7NSnDjMzKwUJw4zMyvFicPMzEpx4jAzs1KcOMzMrBQnDjMzK8WJw8zMSnHiMDOzUpw4zMysFCcOMzMrxYnDzMxKceIwM7NSnDjMzKwUJw4zMyvFicPMzEppauKQNEnSUknLJJ1VpXw7SddJukfSYkmn1KsraXtJN0t6IP8d0cx1MDOzzpqWOCT1A84HDgfGASdIGlcx28eB+yJif+Aw4FuSBtSpexZwS0SMBW7J42Zm1kuaecQxHlgWEcsjYh0wEziqYp4AhkoSMAR4Blhfp+5RwCX59SXA0U1cBzMzq9C/iW3vDDxSGF8JHFQxz/eA2cAqYCjwjxHxmqSu6rZFxGMAEfGYpDdWW7ikycDkPLpW0tJurscOwFPdrNuTHEdnrRBHK8QAjqOS4+hsU+IYXW1iMxOHqkyLivH3AAuAdwB7ADdL+lWDdbsUERcAF5SpU42k+RFx4Ka24zhef3G0QgyOw3H0RRzN7KpaCYwqjO9COrIoOgW4JpJlwEPA3nXqPi5pJED++0QTYjczsxqamTjmAWMl7SZpAHA8qVuq6GFgIoCkNmAvYHmdurOBk/Lrk4CfN3EdzMysQtO6qiJivaQpwI1AP2BGRCyWdEYunw58BbhY0kJS99SZEfEUQLW6uelzgaslnUpKPMc1ax2yTe7u6iGOo7NWiKMVYgDHUclxdNbjcSii1KkDMzPbwvnOcTMzK8WJw8zMSnHi6IKk4ZJ+Kul+SUskva0PYthL0oLCsEbSp/sgjs/kx8IsknSVpIG9HUOO41M5hsW9+T5ImiHpCUmLCtN6/fE3NeI4Lr8fr0nqlcs/a8Txn/l/5V5JsyQN76M4vpJjWCDpJkk79XYMhbLPSwpJOzQzhlpxSJoq6dHC/uOInliWE0fXvgPcEBF7A/sDS3o7gIhYGhEHRMQBQDvwAjCrN2PIN2R+EjgwIvYlXbBwfG/GkOPYF/go6ckC+wNHShrbS4u/GJhUMa0vHn9TLY5FwPuB23th+V3FcTOwb0TsB/wBOLuP4vjPiNgv/8/8AvhiH8SApFHAu0gX8fSGqnEA53XsQyJiTk8syImjBknDgEOAHwFExLqIWN2nQaVLlx+MiD/2wbL7A4Mk9QcGs/E9Ob1hH+COiHghItYDtwHH9MaCI+J20iNxinr98TfV4oiIJRHR3Scj9GQcN+XPBeAO0v1XfRHHmsLotpS8ebgnYsjOA/612ctvII4e58RR2+7Ak8BFkn4v6UJJ2/ZxTMcDV/X2QiPiUeCbpG9OjwF/joibejsO0jfrQyS9QdJg4Ag63yja2zo9/gao+vibLdRHgP/uq4VLmibpEeCDNP+Io9ry3wc8GhH39Payq5iSu+5m9FR3qhNHbf2BtwDfj4g3A8/Th0/izTdCvg/4SR8sewTp2/VuwE7AtpI+1NtxRMQS4OukLpEbgHtID8W0FiLpC6TP5Yq+iiEivhARo3IMU3pz2flLzRfog4RVxfdJj3M6gPSl71s90agTR20rgZUR8T95/KekRNJXDgfujojH+2DZ7wQeiognI+IV4Brgb/sgDiLiRxHxlog4hHRY/kBfxJH58TcVJJ0EHAl8MFrjJrErgQ/08jL3IH3JukfSClKX3d2S3tTLcRARj0fEqxHxGvBD0vnBTebEUUNE/Al4RNJeedJE4L4+DOkE+qCbKnsYeKukwfkR+BPpgwsFADqehixpV9IJ4b56T8CPv+lE0iTgTOB9EfFCH8ZRvGDifcD9vbn8iFgYEW+MiDERMYb0JfQteZ/Sqzq+2GTHkLp7N11EeKgxkA7v5gP3AtcCI/oojsHA08B2ffhefJn0D7gIuAzYpo/i+BUpgd8DTOzF5V5FOtR/hbQjOBV4A+lqqgfy3+37KI5j8uuXgceBG/sojmWkn0NYkIfpfRTHz/J2ei9wHbBzb8dQUb4C2KGP3ovLgIX5vZgNjOyJZfmRI2ZmVoq7qszMrBQnDjMzK8WJw8zMSnHiMDOzUpw4zMysFCcO26zlJ49+qzD+eUlTe6jtiyUd2xNt1VnOcfnpy3Mrpo+R9GLF05EHdKP9k5v9hFjbsjhx2ObuZeD9vfHY6jIk9Ssx+6nAxyJiQpWyB2PDk00PiIh13QjnZNKjYhqWH2ZpVpUTh23u1pN+U/kzlQWVRwyS1ua/h0m6TdLVkv4g6VxJH5R0p6SFkvYoNPNOSb/K8x2Z6/fLvz0xLz887vRCu3MlXUm66aoynhNy+4skfT1P+yJwMDBd0n82ssKS3i3pd5LulvQTSUM62soxLZJ0gZJjgQOBK/IRyyBJKzoSraQDJd2aX0/N9W4CLpW0o6Sf5TbnSXp7nu/QwhHQ7yUNbSRuex1p9t2MHjw0cwDWAsNId+duB3wemJrLLgaOLc6b/x4GrAZGAtsAjwJfzmWfAv6rUP8G0hessaS7cQcCk4F/y/NsQ3q6wG653eeB3arEuRPp0S07kh6g+Uvg6Fx2K+m3TirrjAFeZMOd2OcDO5B+d2PbPM+ZwBfz6+0LdS8D3lutfQp3MpOSyq359VTgLmBQHr8SODi/3hVYkl9fB7w9vx4C9O/r7cBD7w4+HLXNXkSskXQp6cemXmyw2rzIj0SX9CDQ8Zj4hUCxy+jqSA+Ie0DScmBv4N3AfoWjme1IiWUdcGdEPFRlef+LtIN+Mi/zCtLvvVxbJ84HI/0gEbnekcA44DfpsWEMAH6XiydI+lfSI2q2BxaTdvJlzI6IjvfwncC4vByAYfno4jfAt/M6XBMRK0suwzZzThz2evFfwN3ARYVp68ndsfnhjMUTyy8XXr9WGH+Nzv8Xlc/kCUDAJyLixmKBpMNIRxzVqMb0sgTcHBEnVCx7IPB/SUcWj+QLBGr9vO9f3pcq8xTj3wp4WyGRdDhX0vWk30O5Q9I7I6JXHyRofcvnOOx1ISKeAa4mnWjusIL0c7uQfk9k6240fZykrfJ5j92BpcCNwD9L2hpA0l+p/o98/Q9wqKQd8onzE0i/YFjWHcDbJe2Zlz1Y0l+xIQE8lc95FK8Gew4onodYwYb3patHjt9E4bcsJB2Q/+4R6QmwXyd10+3djfWwzZgTh72efIt0DqDDD0k76zuBg6h9NNCVpaQd/H8DZ0TES8CFpCf03i1pEfAD6hy9526xs4G5pCf73h0RpR/Dnru6TgauknQvKZHsHelnjX9I6mq7FphXqHYx6eT7AkmDSE86/o6kXwGvdrG4TwIH5gsA7gPOyNM/nU/A30PqGuyzX/qzvuGn45qZWSk+4jAzs1KcOMzMrBQnDjMzK8WJw8zMSnHiMDOzUpw4zMysFCcOMzMr5f8Dxs2BL0Yh0RwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "                                           feature_idx             cv_scores  \\\n",
      "15  (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13,...  [0.9689402935952911]   \n",
      "14    (0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14)  [0.9691098308695342]   \n",
      "13       (0, 1, 2, 3, 4, 5, 6, 7, 10, 11, 12, 13, 14)  [0.9691070775607927]   \n",
      "12          (0, 1, 3, 4, 5, 6, 7, 10, 11, 12, 13, 14)  [0.9690728532087123]   \n",
      "11             (0, 1, 3, 5, 6, 7, 10, 11, 12, 13, 14)  [0.9690410322792119]   \n",
      "10                (0, 1, 5, 6, 7, 10, 11, 12, 13, 14)  [0.9690034828399942]   \n",
      "9                    (1, 5, 6, 7, 10, 11, 12, 13, 14)  [0.9689672704644967]   \n",
      "8                        (1, 5, 6, 7, 11, 12, 13, 14)  [0.9684770489462476]   \n",
      "7                           (1, 5, 7, 11, 12, 13, 14)  [0.9679777918579262]   \n",
      "6                              (1, 5, 11, 12, 13, 14)  [0.9666643864249611]   \n",
      "\n",
      "   avg_score                                      feature_names  \n",
      "15   0.96894  (overlap_title, temp_diff, comm_auth, overlap_...  \n",
      "14   0.96911  (overlap_title, temp_diff, comm_auth, overlap_...  \n",
      "13  0.969107  (overlap_title, temp_diff, comm_auth, overlap_...  \n",
      "12  0.969073  (overlap_title, temp_diff, overlap_journ, over...  \n",
      "11  0.969041  (overlap_title, temp_diff, overlap_journ, cos_...  \n",
      "10  0.969003  (overlap_title, temp_diff, cos_sim_abstr, cos_...  \n",
      "9   0.968967  (temp_diff, cos_sim_abstr, cos_sim_title, cos_...  \n",
      "8   0.968477  (temp_diff, cos_sim_abstr, cos_sim_title, cos_...  \n",
      "7   0.967978  (temp_diff, cos_sim_abstr, cos_sim_author, ada...  \n",
      "6   0.966664  (temp_diff, cos_sim_abstr, adam_index, com_nei...  \n",
      "\n",
      "Index of features :  (1, 5, 11, 12, 13, 14)\n",
      "\n",
      " 0.9666643864249611\n",
      "Wall time: 1h 55min 11s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "feature_names=fs(6,False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "5107be10",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('temp_diff',\n",
       " 'cos_sim_abstr',\n",
       " 'adam_index',\n",
       " 'com_neigh',\n",
       " 'same_cluster',\n",
       " 'commun_ra')"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_names"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ab6ef79",
   "metadata": {},
   "source": [
    "<b> End up in 6 Features: </b>\n",
    "\n",
    "1) Temporal distance between the papers  \n",
    "2) Cosine similarity of adbstract    \n",
    "3) Adamic Adar Index   \n",
    "4) Common neighbours    \n",
    "5) Same Cluster (community)   \n",
    "6) Community resource allocation    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "14689415",
   "metadata": {},
   "outputs": [],
   "source": [
    "# New train set with 6 optimal features\n",
    "train_fs=np.array([df['temp_diff'],df['cos_sim_abstr'],df['adam_index'],df['com_neigh'],df['same_cluster'],df['commun_ra']]).T"
   ]
  },
  {
   "cell_type": "raw",
   "id": "9092141f",
   "metadata": {},
   "source": [
    "# Save the new train csv with header for someone to be aware of the features.\n",
    "\n",
    "header=feature_names\n",
    "with open(\"fs_training_features_h.csv\",\"w\") as fs_feat:\n",
    "    csv_out = csv.writer(fs_feat)\n",
    "    csv_out.writerow(header)\n",
    "    for row in train_fs:\n",
    "        csv_out.writerow(row)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "5826be6c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save the new train csv\n",
    "\n",
    "with open(\"fs_training_features.csv\",\"w\") as fs_feat:\n",
    "    csv_out = csv.writer(fs_feat)    \n",
    "    for row in train_fs:\n",
    "        csv_out.writerow(row)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "9be43bfc",
   "metadata": {},
   "outputs": [],
   "source": [
    "df=pd.DataFrame(testing_set,columns=['overlap_title','temp_diff','comm_auth', 'overlap_journ','overlap_abstr','cos_sim_abstr', 'cos_sim_title', 'cos_sim_author', 'cos_sim_journal','jac_sim', 'pref_attac', 'adam_index','com_neigh', 'same_cluster','commun_ra'])\n",
    "# New test set with 6 optimal features\n",
    "test_fs=np.array([df['temp_diff'],df['cos_sim_abstr'],df['adam_index'],df['com_neigh'],df['same_cluster'],df['commun_ra']]).T"
   ]
  },
  {
   "cell_type": "raw",
   "id": "4d34bb83",
   "metadata": {},
   "source": [
    "# Save the new test csv with header for someone to be aware of the features.\n",
    "\n",
    "header=feature_names \n",
    "with open(\"fs_testing_features_h.csv\",\"w\") as fs_feat:\n",
    "    csv_out = csv.writer(fs_feat)\n",
    "    csv_out.writerow(header)\n",
    "    for row in test_fs:\n",
    "        csv_out.writerow(row)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "31cf043f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save the new test csv\n",
    "\n",
    "with open(\"fs_testing_features.csv\",\"w\") as fs_feat:\n",
    "    csv_out = csv.writer(fs_feat)    \n",
    "    for row in test_fs:\n",
    "        csv_out.writerow(row)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e41faf86",
   "metadata": {},
   "source": [
    "<b> Open new train and test set </b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "89ecfcbd",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Training Data\n",
    "training_set = genfromtxt('fs_training_features.csv', delimiter=',')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "539b21e1",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Testing Data\n",
    "testing_set = genfromtxt('fs_testing_features.csv', delimiter=',')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ae8b321",
   "metadata": {},
   "source": [
    "# Algorithms  \n",
    "\n",
    "- SVM  \n",
    "- KNN  \n",
    "- Decision Trees  \n",
    "- Radom Forest Classifier  \n",
    "- AdaBoost\n",
    "- Bagging Classifier\n",
    "- Gradient Boosting Classifier\n",
    "- Gausian NB\n",
    "- Linear Discriminant Analysis\n",
    "- MLP Classifier\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd1c72b7",
   "metadata": {},
   "source": [
    "From the above algorithms, 4 were excluded due to poor performance (comparing with the others).\n",
    "This decision was made on the sampling datasets so as to decrease the run time when the whole dataset was going to be used. The excluded algorithms are LinearDiscriminantAnalysis, DecisionTreeClassifier, GaussianNB and BaggingClassifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "52792c0e",
   "metadata": {},
   "outputs": [],
   "source": [
    "#name the models\n",
    "l_models =[\"SVM\", \"KNN\", \"RFC\", \"ADAB\", \"GBC\", \"MLP\"] # \"DT\", \"BC\", \"GNB\", \"LDA\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "e76d0265",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Create a function to train each model\n",
    "def TrainTestClassifier(C):\n",
    "    \n",
    "    if C == \"SVM\":\n",
    "        classifier = svm.LinearSVC() #Create an SVM model\n",
    "    elif C == \"KNN\":\n",
    "        classifier = KNeighborsClassifier(3) #Create an KNN model\n",
    "#     elif C == \"DT\":\n",
    "#         classifier = DecisionTreeClassifier() #Create a Decision Tree Classifier    \n",
    "    elif C == \"RFC\":\n",
    "        classifier = RandomForestClassifier() #Create a Random Forest\n",
    "\n",
    "    elif C == \"ADAB\":\n",
    "        classifier = AdaBoostClassifier() #Create a Bagging Classifier\n",
    "#     elif C == \"BC\":\n",
    "#         classifier = BaggingClassifier() #Create a Bagging Classifier\n",
    "    elif C == \"GBC\":\n",
    "        classifier = GradientBoostingClassifier() #Create a Gradient Boosting Classifier\n",
    "#     elif C == \"GNB\":\n",
    "#         classifier = GaussianNB() #Create a GaussianNB\n",
    "#     elif C == \"LDA\":\n",
    "#         classifier = LinearDiscriminantAnalysis() #Create a Linear Discriminant Analysis   \n",
    "    elif C == \"MLP\":\n",
    "        classifier = MLPClassifier()  #Create a MLPC lassifier\n",
    "   \n",
    "\n",
    "\n",
    "    classifier.fit(training_set, labels) #Train the model selected\n",
    "    predictions = classifier.predict(testing_set) # Perform predictions\n",
    "    \n",
    "    # Scores\n",
    "    f1_sc = f1_score(y_true = sol, y_pred = predictions) # Compare prediction with actula values\n",
    "    accuracy = accuracy_score(y_true = sol, y_pred = predictions)\n",
    "    logloss = log_loss(y_true = sol, y_pred = predictions)\n",
    "\n",
    "    return [C, f1_sc, accuracy, logloss]  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "310e7d29",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:985: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\"Liblinear failed to converge, increase \"\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 7min 29s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "RESULTS = []\n",
    "for model in l_models:\n",
    "    RESULTS.append(TrainTestClassifier(model))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "7567debe",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Classifier</th>\n",
       "      <th>F1 Score</th>\n",
       "      <th>Accuracy</th>\n",
       "      <th>LogLos</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>SVM</td>\n",
       "      <td>0.962613</td>\n",
       "      <td>0.960518</td>\n",
       "      <td>1.363657</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>KNN</td>\n",
       "      <td>0.969790</td>\n",
       "      <td>0.967563</td>\n",
       "      <td>1.120341</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>RFC</td>\n",
       "      <td>0.964262</td>\n",
       "      <td>0.961376</td>\n",
       "      <td>1.334045</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ADAB</td>\n",
       "      <td>0.960477</td>\n",
       "      <td>0.958589</td>\n",
       "      <td>1.430303</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>GBC</td>\n",
       "      <td>0.973861</td>\n",
       "      <td>0.972066</td>\n",
       "      <td>0.964824</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>MLP</td>\n",
       "      <td>0.974011</td>\n",
       "      <td>0.972066</td>\n",
       "      <td>0.964826</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Classifier  F1 Score  Accuracy    LogLos\n",
       "0        SVM  0.962613  0.960518  1.363657\n",
       "1        KNN  0.969790  0.967563  1.120341\n",
       "2        RFC  0.964262  0.961376  1.334045\n",
       "3       ADAB  0.960477  0.958589  1.430303\n",
       "4        GBC  0.973861  0.972066  0.964824\n",
       "5        MLP  0.974011  0.972066  0.964826"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_results = pd.DataFrame(RESULTS, columns=[\"Classifier\", 'F1 Score', 'Accuracy', 'LogLos'])\n",
    "display(df_results)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20c41439",
   "metadata": {},
   "source": [
    "# Graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "22f010a9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAEWCAYAAACe8xtsAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAan0lEQVR4nO3de7hcdX3v8feHmyESQAFREAleqmgKEeIFCjSK2kqx4KUKeAFF0RarHERtOT6o9JyjVfGCylFEsGrBegEVj1ZqNYJHrCQYCBdRICCIWhAJEC5K+PaPtVLG7d7Zs5I9Mzs779fz7Cezfmv91nznl2Q++7fWmjWpKiRJ6mKjURcgSVr/GB6SpM4MD0lSZ4aHJKkzw0OS1JnhIUnqzPDQei3JO5J8doD7vzzJwvZxkpyR5DdJfphk3yRXDeq5penM8NC0l+SwJIuT3JnkF0m+kWSfYTx3VT2pqha1i/sAzwYeWVVPraoLqurxU/2cbSBWkqdO9b6lqWJ4aFpLcizwQeD/ANsDjwJOAQ4aQTk7A9dV1cp13VGSTSZoD/By4Fbg8HV9nqmoSRqP4aFpK8lWwInA0VV1dlWtrKrfVdW5VfXmCfp8Ickvk6xIcn6SJ/WsOyDJFUnuSPLzJMe17dsm+VqS25LcmuSCJBu1665L8qwkRwKnAXu1M6B3JlmY5Mae/e+Q5EtJbk6yPMkbeta9I8kXk3w2ye3AERO87H2BHYA3Aock2axnH5snOSnJ9e3r+16Szdt1+yT5fvsabkhyRNu+KMmre/ZxRJLv9SxXkqOT/BT4adv2oXYftydZkmTfnu03TnJ8kmvacVySZKckH01y0pi/i3OTHDPB69R6zvDQdLYXMAs4p0OfbwCPAx4GXAz8c8+6TwKvrao5wDzg2237m4Abge1oZjfHA793356q+iTwOuDCqtqiqt7eu74Nm3OBS4Adgf2BY5L8Wc9mBwFfBLYeU1evw9v9/Eu7fGDPuvcBewJ7Aw8F3gLcn+RR7ev+cPsa5gNLJ9j/eA4GngY8sV2+qN3HQ4EzgS8kmdWuOxY4FDgA2BJ4FXAX8E/AoT2huy3NGJzVoQ6tRwwPTWfbALdU1X39dqiq06vqjqq6F3gHsHs7gwH4HfDEJFtW1W+q6uKe9kcAO7czmwuq+03fngJsV1UnVtVvq+pa4BPAIT3bXFhVX66q+6vq7rE7SDIb+CvgzKr6HU3QHN6u24jmjfqNVfXzqlpVVd9vX+dLgW9V1Vlt/b+uqqUdan9XVd26uqaq+my7j/uq6iTgQcDqczuvBt5WVVdV45J22x8CK2gCg/Z1L6qqX3WoQ+sRw0PT2a+Bbfs9Ft8eUnl3e0jlduC6dtW27Z8vpPmN+fok302yV9v+XuBq4Lwk1yb5u7WodWdgh/aw0W1JbqOZwWzfs80Nk+zj+cB9wNfb5X8Gnptku/Y1zAKuGaffThO09+v36krypiRXtofGbgO24oExXNNz/RPwsvbxy4DPrENNmuYMD01nFwL30BxW6cdhNIeGnkXzhje3bQ9AVV1UVQfRHNL6MvD5tv2OqnpTVT0aeB5wbJL96eYGYHlVbd3zM6eqDujZZrLZzOHAFsDPkvwS+AKwKc1holtoxuIxEzz3eO0AK4HZPcsPH2eb/66rPb/xVuDFwEOqamuaGUX6eK7PAgcl2R3YlWaMNUMZHpq2qmoFcALw0SQHJ5mdZNMkz03ynnG6zAHupZmxzKa5QguAJJsleWmSrdpDQrcDq9p1ByZ5bHul0+r2VR3L/SFwe5K3tie2N04yL8lT+umcZPV5kgNpzjfMB3YH/hE4vKruB04H3t+emN84yV5JHkQzQ3lWkhcn2STJNknmt7teCrygHbvHAkdOUsocmtnPzcAmSU6gObex2mnAPyR5XBq7JdkGoKpupDlf8hngS+MdmtPMYXhoWquq99OcpH0bzRvaDcDrGf+32k8D1wM/B64AfjBm/cuB69pDWq/jgUMsjwO+BdxJM9s5peezHf3WuYpm1jIfWE4zUziNZgbUj5cDS6vqvKr65eof4GRgtyTzgOOAZTRv0LfSBMtGVfUzmsNxb2rbl9IED8AHgN8Cv6I5rDTRifrVvklz8v0nNGN5D79/WOv9NDO282iC9pPA5j3r/wn4YzxkNePFL4OSNFWS7Edz+GpuO1vSDOXMQ9KUSLIpzedTTjM4Zj7DQ9I6S7IrcBvNJc8fHGkxGgoPW0mSOnPmIUnqbMbfCG3bbbetuXPnjroMSVqvLFmy5Jaq2m6i9TM+PObOncvixYtHXYYkrVeSXL+m9R62kiR1ZnhIkjqb8Yetlt90B4edsGjUZUjSwJ154sKhPZczD0lSZ4aHJKkzw0OS1JnhIUnqzPCQJHVmeEiSOjM8JEmdGR6SpM4MD0lSZ4aHJKmzkYRHkv+Z5PIklyZZmuQbSd41Zpv5Sa5sH1+X5IIx65cmuWyYdUuSGkMPjyR7AQcCe1TVbsCzgHcDLxmz6SHAmT3Lc5Ls1O5j12HUKkka3yhmHo8AbqmqewGq6paq+i5wW5Kn9Wz3YuBzPcuf54GAORQ4axjFSpL+0CjC4zxgpyQ/SXJKkj9t28+imW2Q5OnAr6vqpz39vgi8oH38PODciZ4gyVFJFidZfM9dK6b+FUjSBm7o4VFVdwJ7AkcBNwP/kuQImlnGi5JsRBMiY2cWtwK/SXIIcCVw1xqe49SqWlBVC2bN3moAr0KSNmwj+T6PqloFLAIWJVkGHF5Vn0pyHfCnwAuBvcbp+i/AR4EjhlOpJGk8Qw+PJI8H7u85JDUfWP1duWcBHwCuqaobx+l+Ds05k28COwy4VEnSBEYx89gC+HCSrYH7gKtpDmEBfAH4EPC343WsqjuAfwRIMvBCJUnjG3p4VNUSYO8J1t0MbDpO+9xx2q4D5k1xeZKkPvgJc0lSZ4aHJKkzw0OS1JnhIUnqzPCQJHVmeEiSOjM8JEmdGR6SpM4MD0lSZyO5MeIw7bLDHM48ceGoy5CkGcWZhySpM8NDktSZ4SFJ6szwkCR1ZnhIkjozPCRJnc34S3WX33QHh52waNRlSNLQDOPjCc48JEmdGR6SpM4MD0lSZ4aHJKkzw0OS1JnhIUnqzPCQJHVmeEiSOjM8JEmdGR6SpM4MD0lSZyMJjyR39jw+IMlPkzwqyTuS3JXkYRNsW0lO6lk+Lsk7hla4JAkY8cwjyf7Ah4E/r6qftc23AG+aoMu9wAuSbDuM+iRJ4xtZeCTZF/gE8BdVdU3PqtOBlyR56Djd7gNOBf7HEEqUJE1gVOHxIOArwMFV9eMx6+6kCZA3TtD3o8BLk2w10c6THJVkcZLF99y1YkoKliQ9YFTh8Tvg+8CRE6w/GTg8yZZjV1TV7cCngTdMtPOqOrWqFlTVglmzJ8wYSdJaGlV43A+8GHhKkuPHrqyq24Azgb+ZoP8HaYLnwQOqT5K0BiM751FVdwEH0hyCGm8G8n7gtYzzbYdVdSvweSaeuUiSBmikV1u1IfDnwNuSHDRm3S3AOTTnR8ZzEuBVV5I0AiP5DvOq2qLn8Q3ALu3iV8Zsdyxw7AT9fgXMHmylkqTx+AlzSVJnhockqTPDQ5LUmeEhSerM8JAkdWZ4SJI6MzwkSZ0ZHpKkzkbyIcFh2mWHOZx54sJRlyFJM4ozD0lSZ4aHJKkzw0OS1JnhIUnqzPCQJHVmeEiSOpvxl+ouv+kODjth0ajLkKSBG+bHEpx5SJI6MzwkSZ0ZHpKkzgwPSVJnhockqTPDQ5LUmeEhSerM8JAkdWZ4SJI6MzwkSZ0ZHpKkzkYaHklWJVma5LIk5ybZum2fm+Tudt3qn83adc9NsjjJlUl+nOR9o3wNkrQhmjQ8kmyU5LIBPf/dVTW/quYBtwJH96y7pl23+ue3SeYBHwFeVlW7AvOAawdUmyRpApOGR1XdD1yS5FEDruVCYMdJtnkL8L+r6sdtbfdV1SkDrkuSNEa/t2R/BHB5kh8CK1c3VtVfTkURSTYG9gc+2dP8mCRL28f/v6qOpplpnNTH/o4CjgKYvdX2U1GiJKlHv+HxzgE9/+ZtQMwFlgD/1rPumqqavzY7rapTgVMBttnh8bVuJUqSxurrhHlVfRe4Dti0fXwRcPEUPP/dbUDsDGzG75/zGM/lwJ5T8LySpHXQV3gkeQ3wReDjbdOOwJenqoiqWgG8ATguyaZr2PS9wPFJ/qita6Mkx05VHZKk/vR7qe7RwJ8AtwNU1U+Bh01lIVX1I+AS4JA1bHMpcAxwVpIrgctozsdIkoao33Me97aXygKQZBNgnc8lVNUWY5af17M4b4I+XwO+tq7PLUlae/3OPL6b5HiaE9zPBr4AnDu4siRJ01m/4fF3wM3AMuC1wNeBtw2qKEnS9NbXYav2g4KfaH8kSRu4NYZHks9X1YuTLGOccxxVtdvAKpMkTVuTzTyOaf88cMB1SJLWI5OFx9eAPYD/VVUvH0I9kqT1wGThsVmSw4G9k7xg7MqqOnswZUmSprPJwuN1wEuBrYHnjVlXgOEhSRugVE3+Wb8kR1bVJyfdcBpasGBBLV68eNRlSNJ6JcmSqlow0frJrrZ6ZlV9G/iNh60kSatNdtjqT4Fv84eHrMDDVpK0wVpjeFTV29s/XzmcciRJ64N+b8n+xiRbpnFakouTPGfQxUmSpqd+7231qqq6HXgOza3YXwm8e2BVSZKmtX7DI+2fBwBnVNUlPW2SpA1Mv+GxJMl5NOHxzSRzgPsHV5YkaTrr98ugjgTmA9dW1V1JHkpz6GraW37THRx2wqJRlyFJA3XmiQuH+nz9zjz2Aq6qqtuSvIzmuzxWDK4sSdJ01m94/F/griS7A28Brgc+PbCqJEnTWr/hcV819zE5CPhQVX0ImDO4siRJ01m/5zzuSPL3wMuA/ZJsDGw6uLIkSdNZvzOPlwD3AkdW1S+BHYH3DqwqSdK01u93mP8SeH/P8s/wnIckbbD6vT3J05NclOTOJL9NsiqJV1tJ0gaq38NWHwEOBX4KbA68GvjooIqSJE1v/Z4wp6quTrJxVa0Czkjy/QHWJUmaxvoNj7uSbAYsTfIe4BfAgwdXliRpOuv3sNXLgY2B1wMrgZ2AF07WKcnzk1SSJ7TLc5PcneRHSa5M8sMkh4/T75IkZ41p+1SS5UmWJvlxkrf3WbskaYr1e7XV9e3Du4F3dtj/ocD3gEOAd7Rt11TVkwGSPBo4O8lGVXVG27YrTajtl+TBVbWyZ39vrqovJpkFXJHk01W1vEM9kqQpMNl3mC+j+brZcVXVbmvouwXwJ8AzgK/yQHj09r82ybHAScAZbfNhwGeAXYG/BM4a2w+Y1f65cpx1kqQBm2zm8QJge+CGMe07AzdN0vdg4F+r6idJbk2yB3DrONtdDDyhZ/klwLOBx9McJusNj/cmeRvwWODkqvrP8Z44yVHAUQCzt9p+kjIlSV1Nds7jA8DtVXV97w9wV7tuTQ4FPtc+/ly7PJ7//lKpJE8Bbm6f49+BPZI8pGfbN1fVfODhwP5J9h5vh1V1alUtqKoFs2ZvNUmZkqSuJpt5zK2qS8c2VtXiJHMn6pRkG+CZwLwkRXOyvYBTxtn8ycCV7eNDgSckua5d3pLmxPxpY57/ziSLgH0ALxmWpCGbbOYxaw3rNl/DuhcBn66qnatqblXtBCwHHtm7URtA7wM+nGQj4K+A3do+c2nu4vsHM5YkmwBPA66ZpH5J0gBMFh4XJXnN2MYkRwJL1tDvUOCcMW1fAo4HHrP6Ul3g88CH2yut9gN+XlU/7+lzPvDEJI9ol9+bZClwKbAMOHuS+iVJAzDZYatjgHOSvJQHwmIBsBnw/Ik6VdXCcdpOBk5eQ59FwNPHtK0CVgfHEZPUKkkakjWGR1X9Ctg7yTOAeW3z/6uqbw+8MknStNXvhwS/A3xnwLVIktYT/d6eRJKk/2Z4SJI6MzwkSZ0ZHpKkzgwPSVJnhockqTPDQ5LUWd/fYb6+2mWHOZx54sJRlyFJM4ozD0lSZ4aHJKkzw0OS1JnhIUnqzPCQJHVmeEiSOpvxl+ouv+kODjth0ajLkKShGvRHFJx5SJI6MzwkSZ0ZHpKkzgwPSVJnhockqTPDQ5LUmeEhSerM8JAkdWZ4SJI6MzwkSZ2NLDySbJ/kzCTXJlmS5MIkz0+yMMmKJEuTXJrkW0ke1tPvFUkuS3J5kiuSHDeq1yBJG6qRhEeSAF8Gzq+qR1fVnsAhwCPbTS6oqvlVtRtwEXB02++5wDHAc6rqScAewIohly9JG7xRzTyeCfy2qj62uqGqrq+qD/du1IbMHOA3bdPfA8dV1U1tn3uq6hNDqlmS1BrVXXWfBFy8hvX7JlkKbAOsBI5v2+cBSybbeZKjgKMAZm+1/ToVKkn6Q9PihHmSjya5JMlFbdPqw1Y7AWcA7+myv6o6taoWVNWCWbO3mvJ6JWlDN6rwuJzmfAUAVXU0sD+w3TjbfhXYr6ffngOvTpK0RqMKj28Ds5L8dU/b7Am23Qe4pn38LuA9SR4OkORBSd4wuDIlSeMZyTmPqqokBwMfSPIW4GaacxtvbTdZfc4jNFdTvbrt9/Uk2wPfak+mF3D6kMuXpA3eyL6Gtqp+QXN57ngmPFFRVWfQnAeRJI3ItDhhLklavxgekqTODA9JUmeGhySpM8NDktSZ4SFJ6szwkCR1ZnhIkjozPCRJnY3sE+bDsssOczjzxIWjLkOSZhRnHpKkzgwPSVJnhockqTPDQ5LUmeEhSerM8JAkdTbjL9VdftMdHHbColGXIUlDNeiPKDjzkCR1ZnhIkjozPCRJnRkekqTODA9JUmeGhySpM8NDktSZ4SFJ6szwkCR1ZnhIkjozPCRJnY0sPJJUks/0LG+S5OYkX2uXj0jykXH6XZdkWZJLkpyX5OHDrFuSNNqZx0pgXpLN2+VnAz/vs+8zqmp3YDFw/CCKkyRNbNSHrb4B/EX7+FDgrI79zwceO6UVSZImNerw+BxwSJJZwG7Af3TsfyCwbGxjkqOSLE6y+J67VkxBmZKkXiMNj6q6FJhLM+v4eoeu30myFNgSeNc4+z21qhZU1YJZs7eailIlST2mw5dBfRV4H7AQ2KbPPs+oqlsGVpEkaY2mQ3icDqyoqmVJFo64FklSH0Z9zoOqurGqPjTB6iOS3Njz88ihFidJGtfIZh5VtcU4bYuARe3jTwGfGqfr3MFVJUnqx8hnHpKk9Y/hIUnqzPCQJHVmeEiSOjM8JEmdGR6SpM4MD0lSZ4aHJKmz6XB7koHaZYc5nHniwlGXIUkzijMPSVJnhockqTPDQ5LUWapq1DUMVJI7gKtGXcc0sC2woX8HimPQcBwcg9XWNA47V9V2E3Wc8SfMgauqasGoixi1JIs39HFwDBqOg2Ow2rqMg4etJEmdGR6SpM42hPA4ddQFTBOOg2OwmuPgGKy21uMw40+YS5Km3oYw85AkTTHDQ5LU2YwJjyR/nuSqJFcn+btx1ifJye36S5PsMYo6B62PcXhp+/ovTfL9JLuPos5BmmwMerZ7SpJVSV40zPqGoZ8xSLIwydIklyf57rBrHIY+/j9sleTcJJe04/DKUdQ5SElOT/KfSS6bYP3avTdW1Xr/A2wMXAM8GtgMuAR44phtDgC+AQR4OvAfo657ROOwN/CQ9vFzZ9o49DMGPdt9G/g68KJR1z2CfwdbA1cAj2qXHzbqukc0DscD/9g+3g64Fdhs1LVP8TjsB+wBXDbB+rV6b5wpM4+nAldX1bVV9Vvgc8BBY7Y5CPh0NX4AbJ3kEcMudMAmHYeq+n5V/aZd/AHwyCHXOGj9/FsA+FvgS8B/DrO4IelnDA4Dzq6qnwFU1YY6DgXMSRJgC5rwuG+4ZQ5WVZ1P87omslbvjTMlPHYEbuhZvrFt67rN+q7razyS5jeOmWTSMUiyI/B84GNDrGuY+vl38EfAQ5IsSrIkySuGVt3w9DMOHwF2BW4ClgFvrKr7h1PetLFW740z5fYkGadt7DXI/Wyzvuv7NSZ5Bk147DPQioavnzH4IPDWqlrV/MI54/QzBpsAewL7A5sDFyb5QVX9ZNDFDVE/4/BnwFLgmcBjgH9LckFV3T7g2qaTtXpvnCnhcSOwU8/yI2l+k+i6zfqur9eYZDfgNOC5VfXrIdU2LP2MwQLgc21wbAsckOS+qvryUCocvH7/P9xSVSuBlUnOB3YHZlJ49DMOrwTeXc3B/6uTLAeeAPxwOCVOC2v13jhTDltdBDwuyS5JNgMOAb46ZpuvAq9oryx4OrCiqn4x7EIHbNJxSPIo4Gzg5TPst8zVJh2DqtqlquZW1Vzgi8DfzKDggP7+P3wF2DfJJklmA08DrhxynYPWzzj8jGb2RZLtgccD1w61ytFbq/fGGTHzqKr7krwe+CbNFRanV9XlSV7Xrv8YzVU1BwBXA3fR/MYxo/Q5DicA2wCntL9531cz6O6ifY7BjNbPGFTVlUn+FbgUuB84rarGvZRzfdXnv4V/AD6VZBnN4Zu3VtWMulV7krOAhcC2SW4E3g5sCuv23ujtSSRJnc2Uw1aSpCEyPCRJnRkekqTODA9JUmeGhySpM8ND6lN7B96lPT9zk2yT5DtJ7kzykTX0PTDJj9q7t16R5LXDrF2aal6qK/UpyZ1VtcWYtgcDTwbmAfOq6vXj9NsUuB54alXdmORBwNyqumodagnN/98N7T5MmiaceUjroKpWVtX3gHvWsNkcmg/k/rrtc+/q4EiyfZJz2hnJJUn2btuPTXJZ+3NM2zY3yZVJTgEuBnZK8uYkF7Xfw/DOAb5U6fcYHlL/Nu85ZHVOv52q6laaW0Bcn+SsNF/Itfr/3snAd6tqd5rvXLg8yZ40n/J9Gs33K7wmyZPb7R9Pc/vsJ7ePH0dz6/H5wJ5J9lv3lylNbkbcnkQakrurav7adKyqVyf5Y+BZwHHAs4EjaO7m+op2m1XAiiT7AOe0Ny0kydnAvrQB1H7nAsBz2p8ftctb0ITJ+WtTo9SF4SENSVUtA5Yl+QywnCY8xrOm+8SvHLPdu6rq41NTodQ/D1tJA5ZkiyQLe5rm05xAB/h34K/b7TZOsiXNzOHgJLPbE/LPBy4YZ9ffBF6VZIu2/45JHjaQFyGN4cxDWkdJrgO2BDZLcjDwnKq6oncT4C1JPg7cTTN7OKJd90bg1CRHAquAv66qC5N8ige+U+K0qvpRkrm9z1tV5yXZleaLnADuBF7GzPxqXU0zXqorSerMw1aSpM4MD0lSZ4aHJKkzw0OS1JnhIUnqzPCQJHVmeEiSOvsv+osfN7A3eDAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAEWCAYAAACe8xtsAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAa5UlEQVR4nO3deZgddb3n8feHzRCJoIAoiASXq2gGEIMKblHUq1wUXK4SEEHxonNxYXC9jA8id2ZccUMYBxEQFNxRcdxH43JRIeEGAuICBARxAZGwBNCE7/xRFT223elTps85nc779Tz99KlfLedbP8L59K+qTlWqCkmSutho1AVIktY/hockqTPDQ5LUmeEhSerM8JAkdWZ4SJI6Mzy0XktyXJKPD3D7lyVZ0L5OktOT/CHJBUmemORng3pvaTozPDTtJTkoyeIktyX5dZKvJnnCMN67qh5ZVYvayScATwceUFWPqarvV9XDpvo920CsJI+Z6m1LU8Xw0LSW5Gjg/cD/ArYDHgicDOw/gnJ2Aq6uqtvXdUNJNpmgPcAhwE3Aoev6PlNRkzQew0PTVpItgeOBI6vq81V1e1X9qarOq6o3TLDOZ5L8JsmKJN9L8sieefsm+UmSW5P8Ksnr2/Ztknw5yc1Jbkry/SQbtfOuTvK0JIcDpwJ7tSOgtyVZkOS6nu1vn+RzSW5IsjzJa3rmHZfks0k+nuQW4LAJdvuJwPbAa4EDk2zWs43Nk5yQ5Jp2/36QZPN23hOSnN/uw7VJDmvbFyV5ec82Dkvyg57pSnJkkl8Av2jbPtBu45YkS5I8sWf5jZMck+TKth+XJNkxyUlJThjz3+K8JEdNsJ9azxkems72AmYB53ZY56vAQ4H7AhcBn+iZ91HgFVU1B5gHfLttfx1wHbAtzejmGOCv7ttTVR8FXgn8sKq2qKq39s5vw+Y84GJgB2Af4Kgk/9iz2P7AZ4GtxtTV69B2O59qp/frmfce4NHA3sB9gDcCdyd5YLvfJ7b7sDuwdILtj+cA4LHAI9rpC9tt3Ac4G/hMklntvKOBhcC+wL2AlwErgY8BC3tCdxuaPjinQx1ajxgems62Bm6sqlX9rlBVp1XVrVV1F3AcsFs7ggH4E/CIJPeqqj9U1UU97fcHdmpHNt+v7jd92xPYtqqOr6o/VtVVwEeAA3uW+WFVfaGq7q6qO8ZuIMls4J+Bs6vqTzRBc2g7byOaD+rXVtWvqmp1VZ3f7ufBwLeq6py2/t9X1dIOtb+9qm5aU1NVfbzdxqqqOgG4B7Dm3M7LgbdU1c+qcXG77AXACprAoN3vRVX12w51aD1ieGg6+z2wTb/H4ttDKu9oD6ncAlzdztqm/f18mr+Yr0ny3SR7te3vBq4AvpHkqiRv/jtq3QnYvj1sdHOSm2lGMNv1LHPtJNt4LrAK+Eo7/QngWUm2bfdhFnDlOOvtOEF7v/6qriSvS3J5e2jsZmBL/tKHa3uvjwEvbl+/GDhrHWrSNGd4aDr7IXAnzWGVfhxEc2joaTQfeHPb9gBU1YVVtT/NIa0vAJ9u22+tqtdV1YOAZwNHJ9mHbq4FllfVVj0/c6pq355lJhvNHApsAfwyyW+AzwCb0hwmupGmLx48wXuP1w5wOzC7Z/p+4yzz57ra8xtvAl4I3LuqtqIZUaSP9/o4sH+S3YBdaPpYM5ThoWmrqlYAxwInJTkgyewkmyZ5VpJ3jbPKHOAumhHLbJortABIslmSg5Ns2R4SugVY3c7bL8lD2iud1rSv7ljuBcAtSd7UntjeOMm8JHv2s3KSNedJ9qM537A7sBvwTuDQqrobOA14b3tifuMkeyW5B80I5WlJXphkkyRbJ9m93fRS4Hlt3z0EOHySUubQjH5uADZJcizNuY01TgX+PclD09g1ydYAVXUdzfmSs4DPjXdoTjOH4aFprareS3OS9i00H2jXAq9i/L9qzwSuAX4F/AT40Zj5hwBXt4e0XslfDrE8FPgWcBvNaOfknu929FvnappRy+7AcpqRwqk0I6B+HAIsrapvVNVv1vwAHwR2TTIPeD2wjOYD+iaaYNmoqn5JczjudW37UprgAXgf8EfgtzSHlSY6Ub/G12lOvv+cpi/v5K8Pa72XZsT2DZqg/Siwec/8jwH/BQ9ZzXjxYVCSpkqSJ9EcvprbjpY0QznykDQlkmxK8/2UUw2Omc/wkLTOkuwC3ExzyfP7R1qMhsLDVpKkzhx5SJI6m/E3Qttmm21q7ty5oy5DktYrS5YsubGqtp1o/owPj7lz57J48eJRlyFJ65Uk16xtvoetJEmdGR6SpM5m/GGr5dffykHHLhp1GZI0cGcfv2Bo7+XIQ5LUmeEhSerM8JAkdWZ4SJI6MzwkSZ0ZHpKkzgwPSVJnhockqTPDQ5LUmeEhSepsJOGR5L8nuSzJJUmWJvlqkrePWWb3JJe3r69O8v0x85cmuXSYdUuSGkMPjyR7AfsBe1TVrsDTgHcALxqz6IHA2T3Tc5Ls2G5jl2HUKkka3yhGHvcHbqyquwCq6saq+i5wc5LH9iz3QuCTPdOf5i8BsxA4ZxjFSpL+1ijC4xvAjkl+nuTkJE9u28+hGW2Q5HHA76vqFz3rfRZ4Xvv62cB5E71BkiOSLE6y+M6VK6Z+DyRpAzf08Kiq24BHA0cANwCfSnIYzSjjBUk2ogmRsSOLm4A/JDkQuBxYuZb3OKWq5lfV/FmztxzAXkjShm0kz/OoqtXAImBRkmXAoVV1RpKrgScDzwf2GmfVTwEnAYcNp1JJ0niGHh5JHgbc3XNIandgzbNyzwHeB1xZVdeNs/q5NOdMvg5sP+BSJUkTGMXIYwvgxCRbAauAK2gOYQF8BvgA8OrxVqyqW4F3AiQZeKGSpPENPTyqagmw9wTzbgA2Had97jhtVwPzprg8SVIf/Ia5JKkzw0OS1JnhIUnqzPCQJHVmeEiSOjM8JEmdGR6SpM4MD0lSZ4aHJKmzkdwYcZh23n4OZx+/YNRlSNKM4shDktSZ4SFJ6szwkCR1ZnhIkjozPCRJnRkekqTOZvylusuvv5WDjl006jIkaWiG8fUERx6SpM4MD0lSZ4aHJKkzw0OS1JnhIUnqzPCQJHVmeEiSOjM8JEmdGR6SpM4MD0lSZ4aHJKmzkYRHktt6Xu+b5BdJHpjkuCQrk9x3gmUryQk9069PctzQCpckASMeeSTZBzgReGZV/bJtvhF43QSr3AU8L8k2w6hPkjS+kYVHkicCHwH+qaqu7Jl1GvCiJPcZZ7VVwCnAfxtCiZKkCYwqPO4BfBE4oKp+OmbebTQB8toJ1j0JODjJlhNtPMkRSRYnWXznyhVTUrAk6S9GFR5/As4HDp9g/geBQ5Pca+yMqroFOBN4zUQbr6pTqmp+Vc2fNXvCjJEk/Z1GFR53Ay8E9kxyzNiZVXUzcDbwrxOs/36a4LnngOqTJK3FyM55VNVKYD+aQ1DjjUDeC7yCcZ52WFU3AZ9m4pGLJGmARnq1VRsCzwTekmT/MfNuBM6lOT8ynhMAr7qSpBEYyTPMq2qLntfXAju3k18cs9zRwNETrPdbYPZgK5UkjcdvmEuSOjM8JEmdGR6SpM4MD0lSZ4aHJKkzw0OS1JnhIUnqzPCQJHU2ki8JDtPO28/h7OMXjLoMSZpRHHlIkjozPCRJnRkekqTODA9JUmeGhySpM8NDktTZjL9Ud/n1t3LQsYtGXYYkDdwwv5bgyEOS1JnhIUnqzPCQJHVmeEiSOjM8JEmdGR6SpM4MD0lSZ4aHJKkzw0OS1JnhIUnqzPCQJHU20vBIsjrJ0iSXJjkvyVZt+9wkd7Tz1vxs1s57VpLFSS5P8tMk7xnlPkjShmjS8EiyUZJLB/T+d1TV7lU1D7gJOLJn3pXtvDU/f0wyD/gQ8OKq2gWYB1w1oNokSROYNDyq6m7g4iQPHHAtPwR2mGSZNwL/s6p+2ta2qqpOHnBdkqQx+r0l+/2By5JcANy+prGqnjMVRSTZGNgH+GhP84OTLG1f/0dVHUkz0jihj+0dARwBMHvL7aaiRElSj37D420Dev/N24CYCywBvtkz78qq2v3v2WhVnQKcArD19g+rdStRkjRWXyfMq+q7wNXApu3rC4GLpuD972gDYidgM/76nMd4LgMePQXvK0laB32FR5J/AT4L/J+2aQfgC1NVRFWtAF4DvD7JpmtZ9N3AMUn+oa1royRHT1UdkqT+9Hup7pHA44FbAKrqF8B9p7KQqvpP4GLgwLUscwlwFHBOksuBS2nOx0iShqjfcx53tZfKApBkE2CdzyVU1RZjpp/dMzlvgnW+DHx5Xd9bkvT363fk8d0kx9Cc4H468BngvMGVJUmazvoNjzcDNwDLgFcAXwHeMqiiJEnTW1+HrdovCn6k/ZEkbeDWGh5JPl1VL0yyjHHOcVTVrgOrTJI0bU028jiq/b3fgOuQJK1HJguPLwN7AP+jqg4ZQj2SpPXAZOGxWZJDgb2TPG/szKr6/GDKkiRNZ5OFxyuBg4GtgGePmVeA4SFJG6BUTf5dvySHV9VHJ11wGpo/f34tXrx41GVI0nolyZKqmj/R/MmutnpqVX0b+IOHrSRJa0x22OrJwLf520NW4GErSdpgrTU8quqt7e+XDqccSdL6oN9bsr82yb3SODXJRUmeMejiJEnTU7/3tnpZVd0CPIPmVuwvBd4xsKokSdNav+GR9ve+wOlVdXFPmyRpA9NveCxJ8g2a8Ph6kjnA3YMrS5I0nfX7MKjDgd2Bq6pqZZL70By6mvaWX38rBx27aNRlSNJAnX38gqG+X78jj72An1XVzUleTPMsjxWDK0uSNJ31Gx7/G1iZZDfgjcA1wJkDq0qSNK31Gx6rqrmPyf7AB6rqA8CcwZUlSZrO+j3ncWuSfwNeDDwpycbApoMrS5I0nfU78ngRcBdweFX9BtgBePfAqpIkTWv9PsP8N8B7e6Z/iec8JGmD1e/tSR6X5MIktyX5Y5LVSbzaSpI2UP0etvoQsBD4BbA58HLgpEEVJUma3vo9YU5VXZFk46paDZye5PwB1iVJmsb6DY+VSTYDliZ5F/Br4J6DK0uSNJ31e9jqEGBj4FXA7cCOwPMnWynJc5NUkoe303OT3JHkP5NcnuSCJIeOs97FSc4Z03ZGkuVJlib5aZK39lm7JGmK9Xu11TXtyzuAt3XY/kLgB8CBwHFt25VV9SiAJA8CPp9ko6o6vW3bhSbUnpTknlV1e8/23lBVn00yC/hJkjOranmHeiRJU2CyZ5gvo3nc7Liqate1rLsF8HjgKcCX+Et49K5/VZKjgROA09vmg4CzgF2A5wDnjF0PmNX+vn2ceZKkAZts5PE8YDvg2jHtOwHXT7LuAcDXqurnSW5Ksgdw0zjLXQQ8vGf6RcDTgYfRHCbrDY93J3kL8BDgg1X1u/HeOMkRwBEAs7fcbpIyJUldTXbO433ALVV1Te8PsLKdtzYLgU+2rz/ZTo/nzw+VSrIncEP7Hv8P2CPJvXuWfUNV7Q7cD9gnyd7jbbCqTqmq+VU1f9bsLScpU5LU1WQjj7lVdcnYxqpanGTuRCsl2Rp4KjAvSdGcbC/g5HEWfxRweft6IfDwJFe30/eiOTF/6pj3vy3JIuAJgJcMS9KQTTbymLWWeZuvZd4LgDOraqeqmltVOwLLgQf0LtQG0HuAE5NsBPwzsGu7zlyau/j+zYglySbAY4ErJ6lfkjQAk4XHhUn+ZWxjksOBJWtZbyFw7pi2zwHHAA9ec6ku8GngxPZKqycBv6qqX/Ws8z3gEUnu306/O8lS4BJgGfD5SeqXJA3AZIetjgLOTXIwfwmL+cBmwHMnWqmqFozT9kHgg2tZZxHwuDFtq4E1wXHYJLVKkoZkreFRVb8F9k7yFGBe2/x/q+rbA69MkjRt9fslwe8A3xlwLZKk9US/tyeRJOnPDA9JUmeGhySpM8NDktSZ4SFJ6szwkCR1ZnhIkjrr+xnm66udt5/D2ccvGHUZkjSjOPKQJHVmeEiSOjM8JEmdGR6SpM4MD0lSZ4aHJKmzGX+p7vLrb+WgYxeNugxJGqpBf0XBkYckqTPDQ5LUmeEhSerM8JAkdWZ4SJI6MzwkSZ0ZHpKkzgwPSVJnhockqTPDQ5LU2cjCI8l2Sc5OclWSJUl+mOS5SRYkWZFkaZJLknwryX171ntJkkuTXJbkJ0leP6p9kKQN1UjCI0mALwDfq6oHVdWjgQOBB7SLfL+qdq+qXYELgSPb9Z4FHAU8o6oeCewBrBhy+ZK0wRvVyOOpwB+r6sNrGqrqmqo6sXehNmTmAH9om/4NeH1VXd+uc2dVfWRINUuSWqO6q+4jgYvWMv+JSZYCWwO3A8e07fOAJZNtPMkRwBEAs7fcbp0KlST9rWlxwjzJSUkuTnJh27TmsNWOwOnAu7psr6pOqar5VTV/1uwtp7xeSdrQjSo8LqM5XwFAVR0J7ANsO86yXwKe1LPeowdenSRprUYVHt8GZiX5rz1tsydY9gnAle3rtwPvSnI/gCT3SPKawZUpSRrPSM55VFUlOQB4X5I3AjfQnNt4U7vImnMeobma6uXtel9Jsh3wrfZkegGnDbl8SdrgjewxtFX1a5rLc8cz4YmKqjqd5jyIJGlEpsUJc0nS+sXwkCR1ZnhIkjozPCRJnRkekqTODA9JUmeGhySpM8NDktSZ4SFJ6mxk3zAflp23n8PZxy8YdRmSNKM48pAkdWZ4SJI6MzwkSZ0ZHpKkzgwPSVJnhockqbMZf6nu8utv5aBjF426DEkaqkF/RcGRhySpM8NDktSZ4SFJ6szwkCR1ZnhIkjozPCRJnRkekqTODA9JUmeGhySpM8NDktSZ4SFJ6mxk4ZGkkpzVM71JkhuSfLmdPizJh8ZZ7+oky5JcnOQbSe43zLolSaMdedwOzEuyeTv9dOBXfa77lKraDVgMHDOI4iRJExv1YauvAv/Uvl4InNNx/e8BD5nSiiRJkxp1eHwSODDJLGBX4Mcd198PWDa2MckRSRYnWXznyhVTUKYkqddIw6OqLgHm0ow6vtJh1e8kWQrcC3j7ONs9parmV9X8WbO3nIpSJUk9psPDoL4EvAdYAGzd5zpPqaobB1aRJGmtpkN4nAasqKplSRaMuBZJUh9Gfc6Dqrquqj4wwezDklzX8/OAoRYnSRrXyEYeVbXFOG2LgEXt6zOAM8ZZde7gqpIk9WPkIw9J0vrH8JAkdWZ4SJI6MzwkSZ0ZHpKkzgwPSVJnhockqTPDQ5LU2XS4PclA7bz9HM4+fsGoy5CkGcWRhySpM8NDktSZ4SFJ6ixVNeoaBirJrcDPRl3HNLAN4DNQ7AewD9awH9beBztV1bYTrTjjT5gDP6uq+aMuYtSSLLYf7AewD9awH9atDzxsJUnqzPCQJHW2IYTHKaMuYJqwHxr2g32whv2wDn0w40+YS5Km3oYw8pAkTTHDQ5LU2YwJjyTPTPKzJFckefM485Pkg+38S5LsMYo6B62Pfji43f9LkpyfZLdR1DlIk/VBz3J7Jlmd5AXDrG9Y+umHJAuSLE1yWZLvDrvGQevj/4ctk5yX5OK2D146ijoHLclpSX6X5NIJ5nf/fKyq9f4H2Bi4EngQsBlwMfCIMcvsC3wVCPA44MejrntE/bA3cO/29bNmWj/00wc9y30b+ArwglHXPaJ/C1sBPwEe2E7fd9R1j6APjgHe2b7eFrgJ2GzUtQ+gL54E7AFcOsH8zp+PM2Xk8Rjgiqq6qqr+CHwS2H/MMvsDZ1bjR8BWSe4/7EIHbNJ+qKrzq+oP7eSPgAcMucZB6+ffAsCrgc8BvxtmcUPUTz8cBHy+qn4JUFUzrS/66YMC5iQJsAVNeKwabpmDV1Xfo9m3iXT+fJwp4bEDcG3P9HVtW9dl1ndd9/Fwmr82ZpJJ+yDJDsBzgQ8Psa5h6+ffwj8A906yKMmSJC8ZWnXD0U8ffAjYBbgeWAa8tqruHk5500rnz8eZcnuSjNM29hrkfpZZ3/W9j0meQhMeTxhoRcPXTx+8H3hTVa1u/uCckfrph02ARwP7AJsDP0zyo6r6+aCLG5J++uAfgaXAU4EHA99M8v2qumXAtU03nT8fZ0p4XAfs2DP9AJq/JLous77rax+T7AqcCjyrqn4/pNqGpZ8+mA98sg2ObYB9k6yqqi8MpcLh6Pf/iRur6nbg9iTfA3YDZkp49NMHLwXeUc2B/yuSLAceDlwwnBKnjc6fjzPlsNWFwEOT7JxkM+BA4EtjlvkS8JL2qoLHASuq6tfDLnTAJu2HJA8EPg8cMoP+wuw1aR9U1c5VNbeq5gKfBf51hgUH9Pf/xBeBJybZJMls4LHA5UOuc5D66YNf0oy8SLId8DDgqqFWOT10/nycESOPqlqV5FXA12musDitqi5L8sp2/odprqrZF7gCWEnzF8eM0mc/HAtsDZzc/uW9qmbQnUX77IMZr59+qKrLk3wNuAS4Gzi1qsa9lHN91Oe/hX8HzkiyjObQzZuqasbdpj3JOcACYJsk1wFvBTaFv//z0duTSJI6mymHrSRJQ2R4SJI6MzwkSZ0ZHpKkzgwPSVJnhofUpyTPTVJJHj7qWqRRMzyk/i0EfkDzZbOBSLLxoLYtTSXDQ+pDki2Ax9PcD+zAtm3jJO9Jsqx9BsKr2/Y922elXJzkgiRzkhyW5EM92/tykgXt69uSHJ/kx8BeSY5NcmGSS5Oc0t7xlSQPSfKtdrsXJXlwkrOS7N+z3U8kec6w+kUbLsND6s8BwNfaW7rc1D4s5whgZ+BRVbUr8In2Nhifork7627A04A7Jtn2PWmes/DYqvoB8KGq2rOq5tHcsHC/drlPACe1290b+DXNPcpeCs2Djdr2r0zVTksTMTyk/iykeR4E7e+FNMHw4apaBVBVN9HcG+nXVXVh23bLmvlrsZrm2SJrPCXJj9tbZjwVeGSSOcAOVXVuu907q2plVX0XeEiS+7Y1fa6P95PW2Yy4t5U0SEm2pvkQn5ekaO6TVMASxr/1/3j3/FnFX/+xNqvn9Z1Vtbp9r1nAycD8qro2yXHtsmu7d/xZwME0h9Ne1uduSevEkYc0uRfQPGVtp/ZuvDsCy4GLgFcm2QQgyX2AnwLbJ9mzbZvTzr8a2D3JRkl2pHnK3XjWhMqN7XmWF0AzggGuS3JAu917tHfCBTgDOKpd7rIp22tpLQwPaXILgXPHtH0O2J7mlt6XJLkYOKh93OmLgBPbtm/SBMJ/0ATOMuA9NMHzN6rqZuAj7XJfoLmt+BqHAK9JcglwPnC/dp3f0txK/fR13E+pb95VV1rPtSOQZcAeVbVi1PVow+DIQ1qPJXkazaGyEw0ODZMjD0lSZ448JEmdGR6SpM4MD0lSZ4aHJKkzw0OS1Nn/B+KSRxVvAFIjAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAaQklEQVR4nO3deZglZZmm8fthL6QolEJEQKpFRZRGGgtbaJdSXHEBdxBtcVC0B1tpVFTGUexxRi0UBZR2EIGme4B2V9wXLGAaFAosNhFlKaRwA9kKKFHgnT9O1HBIMitPVOVZMuv+XVdeeeKL+CLec66MfPKLiIxIVSFJUq/WGXYBkqTpxeCQJLVicEiSWjE4JEmtGBySpFYMDklSKwaHpr0kRyT59z6u//IkC5rXSXJSkluSnJ/k6Umu7Ne2pVFkcGhaSPLaJIuT3JHkt0m+k+Rpg9h2VT2xqhY1k08DngtsU1VPqapzqmqHqdpWkpOTfHiq1jfBNvoatJr5DA6NvCSHAp8C/hewJfAo4Dhg7yGUsx2wtKruXNMVJVlvCuqRBs7g0EhLMgf4Z+DgqvpKVd1ZVX+pqjOq6t0T9Plikt8luS3J2Ume2DVvryQ/T7I8yQ1J3tW0z03yzSS3Jrk5yTlJ1mnmLU3ynCQHAicAuzcjnw8lWZBkWdf6H5nky0luTHJtkrd3zTsiyZeS/HuS24EDWn4Wb05yVVPfN5I8smve85Jc2bzn45KcleRNbdbfrOelzaG5W5MsSrJj17z3NJ/Z8mZbezbtT2lGg7cn+X2So9puV9OLwaFRtzuwEfDVFn2+AzwWeDhwEfB/uuZ9HnhLVc0GdgLObNrfCSwDtqAzqjkceMD9eKrq88BbgfOqapOq+mD3/CZozgAuBrYG9gQOSfL8rsX2Br4EbDamrlVK8mzgI8Crga2A64DTm3lzm3W+D9gcuBLYo9d1d23jccBpwCF0PodvA2ck2SDJDsDbgN2az+75wNKm69HA0VW1KbA98IW229b0YnBo1G0O3FRV9/TaoapOrKrlVXU3cATwpGbkAvAX4AlJNq2qW6rqoq72rYDtmhHNOdX+Rm67AVtU1T9X1Z+r6hrgc8C+XcucV1Vfq6r7qmpFi3XvD5xYVRc17+t9dEY+84C9gMubEdk9wDHA71rWDvAa4FtV9YOq+gvwcWAWnRC6F9iQzme3flUtraqrm35/AR6TZG5V3VFVP1mNbWsaMTg06v4IzO31fECSdZN8NMnVzeGgpc2suc33V9D5RXtdczhn96b9SOAq4PtJrkny3tWodTvgkc1hnluT3Epn5LJl1zLXr8Z6AR5JZ5QBQFXdQeez2bqZd33XvKIzelrTbdzXrHfrqrqKzkjkCOAPSU7vOlR2IPA44BdJLkjy4tXYtqYRg0Oj7jzgT8A+PS7/WjqHg54DzAHmNe0BqKoLqmpvOoexvkZzWKUZobyzqh4NvAQ4dOUx/BauB66tqs26vmZX1V5dy6zu7ah/QyeYOm8meQid0dgNwG+BbbrmpXt6DbYRYNtmG1TVqVX1tGaZAj7WtP+qqvaj85l+DPhSU59mKINDI62qbgM+AHwmyT5JNk6yfpIXJlk4TpfZwN10/hrfmM6VWAA0x+r3TzKnORRzO51DMCR5cZLHNL8sV7bf27Lc84Hbm5PIs5rRz05Jdmu5nnWTbNT1tQFwKvDGJLsk2bB5Xz+tqqXAt4C/bj6f9YCDgUdMso11xmxjQzoh+qIkeyZZn855n7uBc5PskOTZzXJ/AlZw/2f3uiRbNCOUW5v1t/3sNI0YHBp5VXUUcCjwfuBGOn/Zv43OiGGsU+gcbrkB+Dkw9nj764GlzWGstwKva9ofC/wQuIPOKOe4rv/d6LXOe+mMVnYBrgVuonMV1pxVdBvPe+n8Yl75dWZV/Qj478CX6Ywwtqc5d1JVNwGvAhbSCcwnAIvp/NKfyH5jtnF1VV1J5/M4tqn9JcBLqurPdM5vfLRp/x2d0cXhzbpeAFye5A46J8r3rao/tXzPmkbig5ykmaW5umsZsH9V/XjY9WjmccQhzQBJnp9ks+ZQ0uF0zul4dZP6wuCQZobdgau5/xDTPi0v95V65qEqSVIrjjgkSa3M+JuszZ07t+bNmzfsMiRpWrnwwgtvqqotxps344Nj3rx5LF68eNhlSNK0kuS6ieZ5qEqS1IrBIUlqZcYfqlq2/HoOW3TosMuQpNYWLhjNR5s44pAktWJwSJJaMTgkSa0YHJKkVgwOSVIrBockqRWDQ5LUisEhSWrF4JAktWJwSJJaGUpwJPlvSS5PckmSJUm+k+QjY5bZJckVzeulSc4ZM39JkssGWbckaQjBkWR34MXArlW1M/Ac4KPAa8Ysui9watf07CTbNuvYcRC1SpIebBgjjq2Am6rqboCquqmqzgJuTfK3Xcu9Gji9a/oL3B8u+wGnDaJYSdIDDSM4vg9sm+SXSY5L8sym/TQ6owySPBX4Y1X9qqvfl4CXN69fApwx0QaSHJRkcZLFK25bMfXvQJLWYgMPjqq6A3gycBBwI/AfSQ6gM7p4ZZJ16ATI2BHFzcAtSfYFrgDuWsU2jq+q+VU1f9acWX14F5K09hrK8ziq6l5gEbAoyaXAG6rq5CRLgWcCrwB2H6frfwCfAQ4YTKWSpLEGHhxJdgDu6zoMtQuw8tm2pwGfBK6uqmXjdP8qnXMk3wMe2edSJUnjGMaIYxPg2CSbAfcAV9E5bAXwReBo4B/H61hVy4GPASTpe6GSpAcbeHBU1YXAHhPMuxFYf5z2eeO0LQV2muLyJEmT8D/HJUmtGBySpFYMDklSKwaHJKkVg0OS1IrBIUlqxeCQJLVicEiSWjE4JEmtDOUmh4O0zextWbjgqGGXIUkzhiMOSVIrBockqRWDQ5LUisEhSWrF4JAktWJwSJJamfGX4y5bfj2HLTp02GVI6iMvuR8sRxySpFYMDklSKwaHJKkVg0OS1IrBIUlqxeCQJLVicEiSWjE4JEmtGBySpFYMDklSKwaHJKmVoQRHkju6Xu+V5FdJHpXkiCR3JXn4BMtWkk90Tb8ryREDK1ySNNwRR5I9gWOBF1TVr5vmm4B3TtDlbuDlSeYOoj5J0oMNLTiSPB34HPCiqrq6a9aJwGuSPGycbvcAxwP/NIASJUnjGFZwbAh8Hdinqn4xZt4ddMLjHRP0/Qywf5I5E608yUFJFidZvOK2FVNSsCSpY1jB8RfgXODACeYfA7whyaZjZ1TV7cApwNsnWnlVHV9V86tq/qw5s6aiXklSY1jBcR/wamC3JIePnVlVtwKnAv91gv6fohM6D+lTfZKkCQztHEdV3QW8mM5hp/FGHkcBb2GcpxRW1c3AF5h4xCJJ6pOhXlXVBMALgPcn2XvMvJuAr9I5HzKeTwBeXSVJAzaUZ45X1SZdr68H/qqZ/PqY5Q4FDp2g3++BjftbqSRpLP9zXJLUisEhSWrF4JAktWJwSJJaMTgkSa0YHJKkVgwOSVIrBockqZWh/APgIG0ze1sWLjhq2GVI0ozhiEOS1IrBIUlqxeCQJLVicEiSWjE4JEmtGBySpFZm/OW4y5Zfz2GLDp18QUkakun2LwOOOCRJrRgckqRWDA5JUisGhySpFYNDktSKwSFJasXgkCS1YnBIkloxOCRJrRgckqRWDA5JUitDDY4k9yZZkuSyJGck2axpn5dkRTNv5dcGzbwXJlmc5Iokv0jy8WG+B0la20waHEnWSXJZn7a/oqp2qaqdgJuBg7vmXd3MW/n15yQ7AZ8GXldVOwI7Adf0qTZJ0jgmDY6qug+4OMmj+lzLecDWkyxzGPA/q+oXTW33VNVxfa5LktSl19uqbwVcnuR84M6VjVX10qkoIsm6wJ7A57uat0+ypHn9n1V1MJ0Rxid6WN9BwEEAm245eypKlCQ1eg2OD/Vp+7OacJgHXAj8oGve1VW1y+qstKqOB44HeMQOW9aalShJ6tbTyfGqOgtYCqzfvL4AuGgKtr+iCYftgA144DmO8VwOPHkKtitJWk09BUeSNwNfAv5307Q18LWpKqKqbgPeDrwryfqrWPRI4PAkj2vqWieJj/eTpAHq9XLcg4G/A24HqKpfAQ+fykKq6mfAxcC+q1jmEuAQ4LQkVwCX0Tn/IkkakF7PcdzdXA4LQJL1gDU+d1BVm4yZfknX5E4T9Pkm8M013bYkafX0OuI4K8nhdE5mPxf4InBG/8qSJI2qXoPjvcCNwKXAW4BvA+/vV1GSpNHV06Gq5p8AP9d8SZLWYqsMjiRfqKpXJ7mUcc5pVNXOfatMkjSSJhtxHNJ8f3Gf65AkTROTBcc3gV2BD1fV6wdQjyRpxE0WHBskeQOwR5KXj51ZVV/pT1mSpFE1WXC8Fdgf2Ax4yZh5BRgckrSWSdXk/8eX5MCq+vykC46g+fPn1+LFi4ddhiRNK0kurKr5482b7KqqZ1fVmcAtHqqSJMHkh6qeCZzJgw9TgYeqJGmttMrgqKoPNt/fOJhyJEmjrtfbqr8jyabpOCHJRUme1+/iJEmjp9d7Vf2XqrodeB6d26m/Efho36qSJI2sXoMjzfe9gJOq6uKuNknSWqTX4LgwyffpBMf3kswG7utfWZKkUdXrg5wOBHYBrqmqu5I8jM7hqpG3bPn1HLbIp8tKWrssXHBU39bd64hjd+DKqro1yevoPIvjtr5VJUkaWb0Gx78AdyV5EnAYcB1wSt+qkiSNrF6D457q3Jtkb+DoqjoamN2/siRJo6rXcxzLk7wPeB3wjCTrAuv3ryxJ0qjqdcTxGuBu4MCq+h2wNXBk36qSJI2sXp85/jvgqK7pX+M5DklaK/V6y5GnJrkgyR1J/pzk3iReVSVJa6FeD1V9GtgP+BUwC3gT8Jl+FSVJGl29nhynqq5Ksm5V3QuclOTcPtYlSRpRvQbHXUk2AJYkWQj8FnhI/8qSJI2qXg9VvR5YF3gbcCewLfCKyToleVmSSvL4ZnpekhVJfpbkiiTnJ3nDOP0uTnLamLaTk1ybZEmSXyT5YI+1S5KmUK9XVV3XvFwBfKjF+vcD/i+wL3BE03Z1Vf0NQJJHA19Jsk5VndS07Ugn0J6R5CFVdWfX+t5dVV9KshHw8ySnVNW1LeqRJK2hyZ45fimdR8SOq6p2XkXfTYC/A54FfIP7g6O7/zVJDgU+AZzUNL8W+DdgR+ClwGlj+wEbNd/vHGeeJKmPJhtxvBzYErh+TPt2wG8m6bsP8N2q+mWSm5PsCtw8znIXAY/vmn4N8FxgBzqHxrqD48gk7wceAxxTVX8Yb8NJDgIOAth0S++MIklTabJzHJ8Ebq+q67q/gLuaeauyH3B68/r0Zno8//+BUEl2A25stvEjYNckD+1a9t1VtQvwCGDPJHuMt8KqOr6q5lfV/FlzZk1SpiSpjclGHPOq6pKxjVW1OMm8iTol2Rx4NrBTkqJzYr2A48ZZ/G+AK5rX+wGPT7K0md6Uzkn4E8Zs/44ki4CnAV4WLEkDNNmIY6NVzFvVn/KvBE6pqu2qal5VbQtcC2zTvVATPh8Hjk2yDvAqYOemzzw6d+N90EglyXrA3wJXT1K/JGmKTRYcFyR589jGJAcCF66i337AV8e0fRk4HNh+5eW4wBeAY5srqp4B3FBVN3T1ORt4QpKtmukjkywBLgEuBb4ySf2SpCk22aGqQ4CvJtmf+4NiPrAB8LKJOlXVgnHajgGOWUWfRcBTx7TdC6wMjQMmqVWSNACrDI6q+j2wR5JnATs1zd+qqjP7XpkkaST1+g+APwZ+3OdaJEnTQK+3HJEkCTA4JEktGRySpFYMDklSKwaHJKkVg0OS1IrBIUlqpednjk9X28zeloULjhp2GZI0YzjikCS1YnBIkloxOCRJrRgckqRWDA5JUisGhySplRl/Oe6y5ddz2KJDh12GNCN5qfvayRGHJKkVg0OS1IrBIUlqxeCQJLVicEiSWjE4JEmtGBySpFYMDklSKwaHJKkVg0OS1MrQgiPJlklOTXJNkguTnJfkZUkWJLktyZIklyT5YZKHd/X7+ySXJbk8yc+TvGtY70GS1kZDCY4kAb4GnF1Vj66qJwP7Ats0i5xTVbtU1c7ABcDBTb8XAocAz6uqJwK7ArcNuHxJWqsNa8TxbODPVfXZlQ1VdV1VHdu9UBMws4Fbmqb3Ae+qqt80ff5UVZ8bUM2SJIZ3d9wnAhetYv7TkywBNgfuBA5v2ncCLpxs5UkOAg4C2HTL2WtUqCTpgUbi5HiSzyS5OMkFTdPKQ1XbAicBC9usr6qOr6r5VTV/1pxZU16vJK3NhhUcl9M5PwFAVR0M7AlsMc6y3wCe0dXvyX2vTpI0oWEFx5nARkn+oatt4wmWfRpwdfP6I8DCJI8ASLJhkrf3r0xJ0lhDOcdRVZVkH+CTSQ4DbqRzLuM9zSIrz3GEzlVTb2r6fTvJlsAPmxPnBZw44PIlaa02tEfHVtVv6VyCO545q+h3Ep3zHpKkIRiJk+OSpOnD4JAktWJwSJJaMTgkSa0YHJKkVgwOSVIrBockqRWDQ5LUisEhSWplaP85PijbzN6WhQuOGnYZkjRjOOKQJLVicEiSWjE4JEmtGBySpFYMDklSKwaHJKmVGX857rLl13PYokOHXYY0I3mp+9rJEYckqRWDQ5LUisEhSWrF4JAktWJwSJJaMTgkSa0YHJKkVgwOSVIrBockqRWDQ5LUisEhSWplaMGRpJL8W9f0ekluTPLNZvqAJJ8ep9/SJJcmuTjJ95M8YpB1S9LabpgjjjuBnZLMaqafC9zQY99nVdWTgMXA4f0oTpI0vmEfqvoO8KLm9X7AaS37nw08ZkorkiSt0rCD43Rg3yQbATsDP23Z/8XApWMbkxyUZHGSxStuWzEFZUqSVhpqcFTVJcA8OqONb7fo+uMkS4BNgY+Ms97jq2p+Vc2fNWfWgzpLklbfKDzI6RvAx4EFwOY99nlWVd3Ut4okSRMaheA4Ebitqi5NsmDItUiSJjHscxxU1bKqOnqC2QckWdb1tc1Ai5MkPcjQRhxVtck4bYuARc3rk4GTx+k6r39VSZImM/QRhyRpejE4JEmtGBySpFYMDklSKwaHJKkVg0OS1IrBIUlqxeCQJLUyCrcc6attZm/LwgVHDbsMSZoxHHFIkloxOCRJrRgckqRWUlXDrqGvkiwHrhx2HS3NBabb80amW83TrV6YfjVPt3rBmrttV1VbjDdjxp8cB66sqvnDLqKNJIutub+mW70w/WqebvWCNffKQ1WSpFYMDklSK2tDcBw/7AJWgzX333SrF6ZfzdOtXrDmnsz4k+OSpKm1Now4JElTyOCQJLUyY4IjyQuSXJnkqiTvHWd+khzTzL8kya7DqHNMTZPVvH9T6yVJzk3ypGHU2VXPKuvtWm63JPcmeeUg65uglklrTrIgyZIklyc5a9A1jqllsp+JOUnOSHJxU+8bh1HnmJpOTPKHJJdNMH+k9r0e6h2p/a6paZU1dy03mH2vqqb9F7AucDXwaGAD4GLgCWOW2Qv4DhDgqcBPp0HNewAPbV6/cJg191Jv13JnAt8GXjkNPuPNgJ8Dj2qmHz7i9R4OfKx5vQVwM7DBkD/nZwC7ApdNMH/U9r3J6h2Z/a7Xmrt+fgay782UEcdTgKuq6pqq+jNwOrD3mGX2Bk6pjp8AmyXZatCFdpm05qo6t6puaSZ/Amwz4Bq79fIZA/wj8GXgD4MsbgK91Pxa4CtV9WuAqhpm3b3UW8DsJAE2oRMc9wy2zDEFVZ3d1DGRkdr3Jqt3xPY7oKfPGAa4782U4NgauL5relnT1naZQWpbz4F0/moblknrTbI18DLgswOsa1V6+YwfBzw0yaIkFyb5+4FV92C91PtpYEfgN8ClwDuq6r7BlLfaRm3fa2PY+11PBr3vzZRbjmSctrHXGfeyzCD1XE+SZ9H5AX5aXytatV7q/RTwnqq6t/MH8dD1UvN6wJOBPYFZwHlJflJVv+x3cePopd7nA0uAZwPbAz9Ick5V3d7n2tbEqO17PRmR/a5Xn2KA+95MCY5lwLZd09vQ+Yus7TKD1FM9SXYGTgBeWFV/HFBt4+ml3vnA6c0P7lxgryT3VNXXBlLhg/X6c3FTVd0J3JnkbOBJwDCCo5d63wh8tDoHta9Kci3weOD8wZS4WkZt35vUCO13vRrsvjfskz5TdOJoPeAa4K+4/6TiE8cs8yIeeILu/GlQ86OAq4A9psNnPGb5kxn+yfFePuMdgR81y24MXAbsNML1/gtwRPN6S+AGYO4I/HzMY+KTzSO17/VQ78jsd73WPGa5vu97M2LEUVX3JHkb8D06VxacWFWXJ3lrM/+zdK402IvOD8RddP5yG5oea/4AsDlwXPOXxD01pDt39ljvSOml5qq6Isl3gUuA+4ATqmqVlzwOs17gfwAnJ7mUzi/i91TVUG8DnuQ0YAEwN8ky4IPA+jCa+14P9Y7MfrdSDzUPtp4moSRJ6slMuapKkjQgBockqRWDQ5LUisEhSWrF4JAktWJwSD1KcscUr2/eZHc7lUaRwSFJasXgkNZAkl2S/KR5dsNXkzy0ad+taTsvyZFtRhZJ9kzysySXNs9h2LBp/2iSnzfr/XjT9qoklzXP5zi7P+9SeiCDQ1ozp9D57+2d6dyt9oNN+0nAW6tqd+DeXleWZCM6t4x4TVX9NZ3bkPxDkofRufvpE5ttfbjp8gHg+VX1JOClU/B+pEkZHNJqSjIH2KyqVj418F+BZyTZDJhdVec27ae2WO0OwLV1/915/5XOQ3xuB/4EnJDk5XRu3QHwn3RuQfJmOrcpkfrO4JCm3prc13rcvlV1D50HPX0Z2Af4btP+VuD9dO4+uyTJ5muwbaknBoe0mqrqNuCWJE9vml4PnFWdp8ctT/LUpn3fFqv9BTAvyWO615lkE2BOVX0bOATYBSDJ9lX106r6AHATD7x9udQXM+LuuNKAbNzcmXSlo4A3AJ9NsjGdW6KvvPPrgcDnktwJLAJum2CdO4xZ5z816/hikvWAC+g81e1hwNebcyBplgM4Msljm7Yf0bkVu9RX3h1X6oMkm1TVHc3r9wJbVdU7hlyWNCUccUj98aIk76Ozj10HHDDccqSp44hDktSKJ8clSa0YHJKkVgwOSVIrBockqRWDQ5LUyv8DVh3U04rOJoEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.set_color_codes(\"muted\")\n",
    "\n",
    "sns.barplot(x='F1 Score', y='Classifier', data=df_results, color=\"b\")\n",
    "\n",
    "plt.xlabel('F1 Score')\n",
    "plt.title('Classifier Accuracy')\n",
    "plt.show()\n",
    "\n",
    "sns.barplot(x='Accuracy', y='Classifier', data=df_results, color=\"b\")\n",
    "\n",
    "plt.xlabel('Accuracy')\n",
    "plt.title('Classifier Accuracy')\n",
    "plt.show()\n",
    "\n",
    "sns.set_color_codes(\"muted\")\n",
    "sns.barplot(x='LogLos', y='Classifier', data=df_results, color=\"g\")\n",
    "\n",
    "plt.xlabel('Log Loss')\n",
    "plt.title('Classifier Log Loss')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b318ba8",
   "metadata": {},
   "source": [
    "<b> Try different parameters on classifiers </b>"
   ]
  },
  {
   "cell_type": "raw",
   "id": "9648d5a5",
   "metadata": {},
   "source": [
    "Comparing the classifiers with each other, the following classifiers are chosen due to better performance: RFC, ADAB, GBC, MLP"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f2d4bca",
   "metadata": {},
   "source": [
    "The base_estimator of the AdaBoostClassifier was experimented with Support Vector Classifier and a Voting Classifier (with Decision Tree and Support Vector) on sampling. The results were worse and the execution time was highly increased. So this method is just being stated and will not be included here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "72e76918",
   "metadata": {},
   "outputs": [],
   "source": [
    "# from sklearn.svm import SVC\n",
    "# svc=SVC(probability=True, kernel='linear')\n",
    "# classifier = AdaBoostClassifier(base_estimator=svc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "4e9ed77e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# from sklearn.ensemble import VotingClassifier\n",
    "# votingClf = VotingClassifier([('clf1',SVC(probability=True)),('clf2',DecisionTreeClassifier())],voting='soft')\n",
    "# classifier = AdaBoostClassifier(base_estimator=votingClf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "a64d1385",
   "metadata": {},
   "outputs": [],
   "source": [
    "#name the models\n",
    "l_models =[\"RFC\", \"ADAB\",  \"GBC\",  \"MLP\"]\n",
    "\n",
    "#Create a function to train each model\n",
    "def ImprovedClassifiers(C):\n",
    "    \n",
    "    if C == \"RFC\":\n",
    "        classifier = RandomForestClassifier(n_jobs=1, n_estimators=700, criterion=\"entropy\", min_samples_split=10,\n",
    "                                            min_samples_leaf=2, max_features=\"sqrt\", max_depth=10) \n",
    "    elif C == \"ADAB\":\n",
    "        classifier = AdaBoostClassifier(n_estimators=5,  random_state=42,\n",
    "                                        base_estimator=RandomForestClassifier(max_depth=6, random_state=42)) \n",
    "    elif C == \"GBC\":\n",
    "        classifier = GradientBoostingClassifier(loss='exponential',min_samples_split=50,max_depth=5) \n",
    "    elif C == \"MLP\":\n",
    "        classifier = MLPClassifier(hidden_layer_sizes=(250,100,50), max_iter=100,activation = 'relu',solver='adam',\n",
    "                                   random_state=1)   \n",
    "\n",
    "\n",
    "    classifier.fit(training_set, labels) #Train the model selected\n",
    "    predictions = classifier.predict(testing_set) # Perform predictions\n",
    "    \n",
    "    # Scores\n",
    "    f1_sc = f1_score(y_true = sol, y_pred = predictions) # Compare prediction with actula values\n",
    "    accuracy = accuracy_score(y_true = sol, y_pred = predictions)\n",
    "    logloss = log_loss(y_true = sol, y_pred = predictions)\n",
    "\n",
    "    return [C, f1_sc, accuracy, logloss]  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "80b954ce",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Classifier</th>\n",
       "      <th>F1 Score</th>\n",
       "      <th>Accuracy</th>\n",
       "      <th>LogLos</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>RFC</td>\n",
       "      <td>0.974645</td>\n",
       "      <td>0.972831</td>\n",
       "      <td>0.938377</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ADAB</td>\n",
       "      <td>0.974670</td>\n",
       "      <td>0.972862</td>\n",
       "      <td>0.937319</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>GBC</td>\n",
       "      <td>0.973412</td>\n",
       "      <td>0.971637</td>\n",
       "      <td>0.979634</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MLP</td>\n",
       "      <td>0.974406</td>\n",
       "      <td>0.972586</td>\n",
       "      <td>0.946840</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Classifier  F1 Score  Accuracy    LogLos\n",
       "0        RFC  0.974645  0.972831  0.938377\n",
       "1       ADAB  0.974670  0.972862  0.937319\n",
       "2        GBC  0.973412  0.971637  0.979634\n",
       "3        MLP  0.974406  0.972586  0.946840"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 31min 12s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "RESULTS = []\n",
    "for model in l_models:\n",
    "    RESULTS.append(ImprovedClassifiers(model))\n",
    "df_results = pd.DataFrame(RESULTS, columns=[\"Classifier\", 'F1 Score', 'Accuracy', 'LogLos'])\n",
    "display(df_results)    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "990f601f",
   "metadata": {},
   "source": [
    "## Graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "109eb341",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAXjUlEQVR4nO3deZRlZX3u8e9DYwfabsALiDIWDlG0Iwg4o2nE4cLVCw65IjigKGowSogjy4XKTaJRo1GBq4jAwlzBRIEoV6MxiOASI80MIgnQoIAGEGRsIDS/+8fehYeyqutsus7Q1d/PWmfVOe8e6rff7jrPeffeZ+9UFZIk9Wu9URcgSVq7GBySpE4MDklSJwaHJKkTg0OS1InBIUnqxODQWi/JR5L8/QDXf1mSZe3zJDk+ya1Jfprk+UmuGNTvlsaRwaG1QpL9kixPcmeSXyX5TpLdhvG7q+qpVXVm+3I34MXA1lX1zKo6u6qeNFe/K8kJSe5rt3Py8Zp22jvbPrg3yQmzrGdhkr9Ncl27jhVJPjNXdWrdZnBo7CU5FPg74K+BLYBtgaOBvUdQznbANVV115quKMn6M0z6RFUt7nl8rW2/AfhL4Lg+Vv9BYFfgmcASYHfggjWtuddq6tc8Z3BorCXZGDgCOLiqTqmqu6rqv6rqW1X13hmW+cckv05yW5Kzkjy1Z9peSX6W5I4k1yd5T9u+WZLTk/w2yS1Jzk6yXjvtmiQvSnIgcCzwnPZT/EeTLEtyXc/6t0zyjSQ3tZ/y39Uz7SNJvp7k75PcDhzQpS/a7T8N+E0fsz8DOLWqbqjGNVV1Yk8t2yQ5pa3zN0mObNvXS/KhJNcmuTHJie2/AUkmklSSA5P8AjijbX9zksvb3XffTbJdl+3S2sfg0Lh7DrABcGqHZb4DPBF4NHA+8H97pn0ZeFtVLQGW0r75AX8BXAdsTjOqOQx4yPV4qurLwNuBc9qRwId7p7dB8y3gImArYA/gkCQv7Zltb+DrwCZT6pprPwEOTfKnSf4oSXrqXACcDlwLTLS1ntxOPqB97A48DlgMHDll3X8M7AC8NMk+NH31Spq+Oxs4aRAbpPFhcGjcbQrcXFX397tAVR1XVXdU1b3AR4AdJz81A/8FPCXJRlV1a1Wd39P+WGC7dkRzdnW/kNszgM2r6oiquq+qrga+BOzbM885VXVaVT1QVStnWM972pHPb5Pc3LGGSR8D/gbYH1gOXJ/kje20ZwJbAu9tR3D3VNWP2mn7A5+uqqur6k6aXV77Ttkt9ZF2uZXA24CPVdXl7b/RXwM7OeqY3wwOjbvfAJv1uz89yYIkH09yVbs76Jp20mbtz1cBewHXJvlhkue07Z8ErgS+l+TqJB94GLVuB2zZ86b/W5pP41v0zPPLPtbzqarapH1sNvvsv6+qVlXVUVX1PJrRzV8BxyXZAdgGuHaGMN6SZiQy6Vpg/dVsw3bAZ3u29xYgNKMYzVMGh8bdOcA9wD59zr8fze6gFwEb0+yKgebNjKo6t6r2ptmNdRrwD237HVX1F1X1OODlNLt59uhY6y+BFT1v+ptU1ZKq2qtnnqFfjrqqVlbVUcCtwFPaOredIYxvoAmDSdsC9wP/2bvKnue/pNn117vNG1bVj+d2KzRODA6Ntaq6DTgcOCrJPkkWJXlEkj2TfGKaRZYA99KMVBbR7DoBHjxFdf8kG1fVfwG3A6vaaS9L8oT2WMBk+6qO5f4UuD3J+5Ns2I5+liZ5Rtftnk6S9ZNsACwAFiTZYKaRWJJD2gP3G7bLvZGmby5o6/wV8PEkj2zX87x20ZOAP0+yfZLFNP33tdXsKvwC8MHJExCSbJzkT+ZiezW+DA6Nvar6NHAo8CHgJppPue+kGTFMdSLN7pXrgZ/RHCTu9XrgmnY31tuB17XtTwS+D9xJM8o5uue7G/3WuYpmtLITsAK4meYsrI1Xs1gXHwJWAh+gqXtl2zadlcDfAr9u6zgYeFV77GKyzicAv6A5KeA17XLHAV8Bzmq34R7gz2YqqKpOpTmWcnLbp5cCez78TdTaIN7ISZLUhSMOSVInBockqRODQ5LUicEhSepk3l+kbLPNNquJiYlRlyFJa5Xzzjvv5qrafLpp8z44JiYmWL58+ajLkKS1SpJrZ5rmripJUicGhySpE4NDktSJwSFJ6mTeHxxfccMd7Hf4maMuQ5KG6qtHLBvYuh1xSJI6MTgkSZ0YHJKkTgwOSVInBockqRODQ5LUicEhSerE4JAkdWJwSJI6MTgkSZ0YHJKkTgwOSVInBockqRODQ5LUicEhSerE4JAkdWJwSJI6GWlwJFmV5MIklyb5VpJN2vaJJCvbaZOPhe20PZMsT3J5kp8n+dQot0GS1jWjHnGsrKqdqmopcAtwcM+0q9ppk4/7kiwFjgReV1U7AEuBq0dQtySts0YdHL3OAbaaZZ73AX9VVT8HqKr7q+rogVcmSXrQWARHkgXAHsA3e5of37Ob6qi2bSlw3tALlCQ9aP0R//4Nk1wITNAEwr/0TLuqqnZ6OCtNchBwEMCijbdYswolSQ8x6hHHyjYctgMW8tBjHNO5DNhltpVW1TFVtWtV7brBoo3XvEpJ0oNGHRwAVNVtwLuA9yR5xGpm/SRwWJI/BEiyXpJDh1GjJKkxFsEBUFUXABcB+65mnouBQ4CTklwOXAo8digFSpKAER/jqKrFU16/vOfl0hmWOR04fZB1SZJmNjYjDknS2sHgkCR1YnBIkjoxOCRJnRgckqRODA5JUicGhySpE4NDktSJwSFJ6sTgkCR1YnBIkjoxOCRJnRgckqRODA5JUicGhySpE4NDktTJSG/kNAzbb7mErx6xbNRlSNK84YhDktSJwSFJ6sTgkCR1YnBIkjoxOCRJnRgckqRODA5JUicGhySpE4NDktSJwSFJ6mTeX3JkxQ13sN/hZ466DEkaqkFeaskRhySpE4NDktSJwSFJ6sTgkCR1YnBIkjoxOCRJnRgckqRODA5JUicGhySpE4NDktSJwSFJ6sTgkCR1YnBIkjoxOCRJnRgckqRODA5JUicGhySpE4NDktTJQIMjySuSVJInt68nkqxMckGSy5P8NMkbp1nuoiQnTWk7IcmKJBcm+XmSDw+ydknS9AY94ngt8CNg3562q6rq6VW1Q9v+50neNDkxyQ5tXS9I8sgp63tvVe0E7AS8Mcn2gyxekvT7BhYcSRYDzwMO5KHB8aCquho4FHhXT/N+wFeA7wH/c4bVb9D+vGtOipUk9W2QI459gH+uqn8Hbkmy8wzznQ88uef1a4CvASfRjFh6fTLJhcB1wMlVdeN0K0xyUJLlSZbfc/dta7AJkqSpBhkcrwVObp+fzO+HwKQ8+CR5BnBTVV0L/Cuwc5JH9cw7uavqMcAeSZ473Qqr6piq2rWqdt1g0cZruBmSpF7rD2KlSTYFXggsTVLAAqCAo6eZ/enA5e3z1wJPTnJN+3oj4FXAsb0LVNWdSc4EdgN+PNf1S5JmNqgRx6uBE6tqu6qaqKptgBXA1r0zJZkAPgV8Psl6wJ8AT2uXmQD2ZpqRSpL1gWcBVw2ofknSDAYy4qB5s//4lLZvAIcBj09yAc0B7juAz1fV8UmWAddX1fU9y5wFPCXJY9vXn0zyIWAhza6sUwZUvyRpBgMJjqpaNk3b54DPrWaZM4FnT2lbBUyGxgFzVqAk6WHzm+OSpE5mDY4k6yW5dBjFSJLG36zBUVUPABcl2XYI9UiSxly/xzgeC1yW5Kf0fFu7qmb6ZrckaZ7qNzg+OtAqJElrjb6Co6p+mGQ74IlV9f0ki2i+1CdJWsf0dVZVkrcCXwe+2DZtBZw2oJokSWOs39NxD6a50u3tAFX1H8CjB1WUJGl89Rsc91bVfZMv2kt+1GBKkiSNs36D44dJDgM2TPJi4B+Bbw2uLEnSuOo3OD4A3ARcArwN+DbwoUEVJUkaX/2eVfUA8KX2IUlah602OJL8Q1X9rySXMM0xjap62sAqkySNpdlGHIe0P1824DokSWuJ2YLjdGBn4C+r6vVDqEeSNOZmC46FSd4IPDfJK6dOrCpvpCRJ65jZguPtwP7AJsDLp0wr1oI78G2/5RK+esSyUZchSfPGaoOjqn4E/CjJ8qr68pBqkiSNsdnOqnphVZ0B3OquKkkSzL6r6o+BM/j93VSwluyqkiTNrdl2VX24/fmm4ZQjSRp3/V5W/d1JNkrj2CTnJ3nJoIuTJI2ffq9V9eaquh14Cc3l1N8EfHxgVUmSxla/wZH2517A8VV1UU+bJGkd0m9wnJfkezTB8d0kS4AHBleWJGlc9XV1XOBAYCfg6qq6O8l/o9ldJUlax/Q74ngOcEVV/TbJ62juxXHb4MqSJI2rfkcc/wfYMcmOwPuALwMn0nzPY6ytuOEO9jv8zFGXIUlDNchLLfU74ri/qgrYG/hsVX0WWDKwqiRJY6vfEccdST4IvA54QZIFwCMGV5YkaVz1O+J4DXAvcGBV/RrYCvjkwKqSJI2tfu85/mvg0z2vf0FzjEOStI7p95Ijz05ybpI7k9yXZFUSz6qSpHVQv7uqjgReC/wHsCHwFuCoQRUlSRpf/R4cp6quTLKgqlYBxyf58QDrkiSNqX6D4+4kC4ELk3wC+BXwyMGVJUkaV/3uqno9sAB4J3AXsA3wqkEVJUkaX/2eVXVt+3Ql8NHBlSNJGnez3XP8EppbxE6rqp425xVJksbabCOOVwJbAL+c0r4dcMNAKpIkjbXZjnF8Bri9qq7tfQB3t9MkSeuY2YJjoqountpYVcuBiYFUJEkaa7MFxwarmbbhXBYiSVo7zBYc5yZ569TGJAcC5w2mJEnSOJvt4PghwKlJ9ud3QbErsBB4xQDrkiSNqdUGR1X9J/DcJLsDS9vm/1dVZ6zpL06yBc0B9mcDtwL3AZ9on/8TsIJmRHQjsF9V3dgu9waauxCmfRxXVZ9a03okSf3p65vjVfWDqvp8+5iL0AhwGnBWVT2uqnYB9gW2bmc5u6p2ar8nci5wcLvcnjSjoJdU1VOBnfHe55I0VP1ecmSuvRC4r6q+MNnQnur7+d6Z2oBZQjMKAfgg8J6quqFd5p6q+tKQapYk0eHquHPsqcD5q5n+/CQXApvSXBvrsLZ9KX0clE9yEHAQwKKNt1ijQiVJDzWqEcdDJDkqyUVJzm2bJndVbQMcT3Pso29VdUxV7VpVu26waOM5r1eS1mWjCo7LaI5PAFBVBwN7AJtPM+83gRf0LLfLwKuTJM1oVMFxBrBBknf0tC2aYd7dgKva5x8DPpHkMQBJ/iDJuwZXpiRpqpEc46iqSrIP8Jkk7wNuojmW8f52lsljHKE5a+ot7XLfbk/j/X574LyA44ZcviSt00Z1cJyq+hXNKbjTmfHARFUdT3PcQ5I0AmNxcFyStPYwOCRJnRgckqRODA5JUicGhySpE4NDktSJwSFJ6sTgkCR1YnBIkjoxOCRJnRgckqRODA5JUicGhySpE4NDktSJwSFJ6sTgkCR1MrIbOQ3L9lsu4atHLBt1GZI0bzjikCR1YnBIkjoxOCRJnRgckqRODA5JUicGhySpE4NDktSJwSFJ6sTgkCR1YnBIkjqZ95ccWXHDHex3+JmjLkOShmqQl1pyxCFJ6sTgkCR1YnBIkjoxOCRJnRgckqRODA5JUicGhySpE4NDktSJwSFJ6sTgkCR1YnBIkjoxOCRJnRgckqRODA5JUicGhySpE4NDktSJwSFJ6mRkwZGkknyl5/X6SW5Kcnr7+oAkR06z3DVJLklyUZLvJXnMMOuWpHXdKEccdwFLk2zYvn4xcH2fy+5eVTsCy4HDBlGcJGl6o95V9R3gf7TPXwuc1HH5s4AnzGlFkqTVGnVwnAzsm2QD4GnAv3Vc/mXAJXNelSRpRiMNjqq6GJigGW18u8OiP0hyIbAR8LGpE5MclGR5kuX33H3bXJQqSWqtP+oCgG8CnwKWAZv2uczuVXXzTBOr6hjgGIBNt3xSrWmBkqTfGYfgOA64raouSbJsxLVIkmYx6mMcVNV1VfXZGSYfkOS6nsfWQy1OkvR7RjbiqKrF07SdCZzZPj8BOGGaRScGV5UkaTYjH3FIktYuBockqRODQ5LUicEhSerE4JAkdWJwSJI6MTgkSZ0YHJKkTgwOSVInBockqRODQ5LUicEhSerE4JAkdWJwSJI6MTgkSZ0YHJKkTsbh1rEDtf2WS/jqEctGXYYkzRuOOCRJnRgckqRODA5JUicGhySpE4NDktRJqmrUNQxUkjuAK0ZdxxjYDLh51EWMmH3QsB/sg0mr64ftqmrz6SbM+9NxgSuqatdRFzFqSZav6/1gHzTsB/tg0sPtB3dVSZI6MTgkSZ2sC8FxzKgLGBP2g30wyX6wDyY9rH6Y9wfHJUlza10YcUiS5pDBIUnqZN4ER5L/nuSKJFcm+cA005Pkc+30i5PsPIo6B62Pfti/3f6Lk/w4yY6jqHOQZuuDnvmekWRVklcPs75h6KcPkixLcmGSy5L8cNg1DkMffw8bJ/lWkovafnjTKOocpCTHJbkxyaUzTO/+3lhVa/0DWABcBTwOWAhcBDxlyjx7Ad8BAjwb+LdR1z2ifngu8Kj2+Z7zrR/66YOe+c4Avg28etR1j+D/wSbAz4Bt29ePHnXdI+qHw4C/aZ9vDtwCLBx17XPcDy8AdgYunWF65/fG+TLieCZwZVVdXVX3AScDe0+ZZ2/gxGr8BNgkyWOHXeiAzdoPVfXjqrq1ffkTYOsh1zho/fxfAPgz4BvAjcMsbkj66YP9gFOq6hcAVbWu9kMBS5IEWEwTHPcPt8zBqqqzaLZrJp3fG+dLcGwF/LLn9XVtW9d51nZdt/FAmk8a88msfZBkK+AVwBeGWNcw9fP/4A+BRyU5M8l5Sd4wtOqGp59+OBLYAbgBuAR4d1U9MJzyxkbn98b5csmRTNM29TzjfuZZ2/W9jUl2pwmO3QZa0fD10wd/B7y/qlY1HzTnnX76YH1gF2APYEPgnCQ/qap/H3RxQ9RPP7wUuBB4IfB44F+SnF1Vtw+4tnHS+b1xvgTHdcA2Pa+3pvkE0XWetV1f25jkacCxwJ5V9Zsh1TYs/fTBrsDJbWhsBuyV5P6qOm0oFQ5ev38PN1fVXcBdSc4CdgTmU3D00w9vAj5ezc7+K5OsAJ4M/HQ4JY6Fzu+N82VX1bnAE5Nsn2QhsC/wzSnzfBN4Q3sGwbOB26rqV8MudMBm7Yck2wKnAK+fZ58uJ83aB1W1fVVNVNUE8HXgT+dRaEB/fw//BDw/yfpJFgHPAi4fcp2D1k8//IJm1EWSLYAnAVcPtcrR6/zeOC9GHFV1f5J3At+lOZPiuKq6LMnb2+lfoDl7Zi/gSuBumk8a80qf/XA4sClwdPuJ+/6aR1cJ7bMP5rV++qCqLk/yz8DFwAPAsVU17emaa6s+/y/8b+CEJJfQ7LJ5f1XNq8utJzkJWAZsluQ64MPAI+Dhvzd6yRFJUifzZVeVJGlIDA5JUicGhySpE4NDktSJwSFJ6sTgkPrQXkX3wp7HRJJNk/wgyZ1JjlzNsi9LckF7BdafJXnbMGuX5pqn40p9SHJnVS2e0vZI4OnAUmBpVb1zmuUeAVwLPLOqrkvyB8BEVV2xBrWE5m93XbumksaEIw7pYaqqu6rqR8A9q5ltCc0XbX/TLnPvZGgk2SLJqe1I5KIkz23bD01yafs4pG2bSHJ5kqOB84Ftkrw3ybntPRQ+OsBNlR7C4JD6s2HPbqpT+12oqm6huaTDtUlOSnMjrcm/u88BP6yqHWnul3BZkl1ovrn7LJp7I7w1ydPb+Z9Ec/nrp7fPn0hz6fCdgF2SvGDNN1Oa3by45Ig0BCuraqeHs2BVvSXJHwEvAt4DvBg4gOaKrG9o51kF3JZkN+DU9uKDJDkFeD5t+LT3SwB4Sfu4oH29mCZIzno4NUpdGBzSEFTVJcAlSb4CrKAJjums7jrvd02Z72NV9cW5qVDqn7uqpAFKsjjJsp6mnWgOlgP8K/COdr4FSTaiGTHsk2RRe/D9FcDZ06z6u8Cbkyxul98qyaMHshHSFI44pDWQ5BpgI2Bhkn2Al1TVz3pnAd6X5IvASppRwwHttHcDxyQ5EFgFvKOqzklyAr+7H8SxVXVBkone31tV30uyA80NmADuBF7H/LwVrsaMp+NKkjpxV5UkqRODQ5LUicEhSerE4JAkdWJwSJI6MTgkSZ0YHJKkTv4/fkMgeJeiIBUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAEWCAYAAACe8xtsAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAYXElEQVR4nO3deZRkZZ3m8e/DJgWUooCgUFJu3aI1gAgKNDooaA8MNri0soiA2LQz2Bajttocj1vPjLbiUrKMgwgq2mArSguDig6NyCBCgVUUiCirBaKCyGKB0hS/+ePe1KDMrIxLZWREZn0/58TJuO9d4hcvRTz53hv53lQVkiR1sc6wC5AkzTyGhySpM8NDktSZ4SFJ6szwkCR1ZnhIkjozPDSjJXlfki8M8PjXJNmzfZ4kpyX5TZLLkrwwyXWDem1plBkeGnlJDk6yOMlvk9ye5BtJ9piO166q51TVhe3iHsBLgW2q6vlV9b2q+vOpfs02ECvJ86f62NJUMTw00pK8FfgE8D+BLYGnACcB+w+hnG2Bm6tqxZoeKMl6E7QHOBS4CzhsTV9nKmqSxmN4aGQleRzwAeDoqvpqVa2oqn+vqnOq6u8n2OfLSX6R5J4kFyV5Ts+6fZP8KMl9SW5L8va2ffMk5ya5O8ldSb6XZJ123c1J9k5yJHAKsFs7Anp/kj2T3Npz/CcnOSvJHUluSvKWnnXvS/KVJF9Ici9w+ARv+4XAk4GFwIFJNug5xpwkH01yS/v+Lk4yp123R5JL2vewPMnhbfuFSd7Yc4zDk1zcs1xJjk7yU+Cnbdui9hj3JrkiyQt7tl83ybFJbmj78Yok85KcmOSjq/y3OCfJMRO8T81whodG2W7AhsDXOuzzDeCZwBOBK4Ev9qz7DPC3VTUXWABc0La/DbgV2IJmdHMs8Ih5e6rqM8CbgO9X1SZV9d7e9W3YnAMsBbYG9gKOSfKXPZvtD3wF2HSVunod1h7nS+3yfj3rjgOeB+wOPAF4B/Bwkqe07/v49j3sCCyZ4PjjOQB4AfDsdvny9hhPAP4Z+HKSDdt1bwUOAvYFHgu8Abgf+BxwUE/obk7TB2d0qEMziOGhUbYZcGdVPdTvDlV1alXdV1W/B94H7NCOYAD+HXh2ksdW1W+q6sqe9icB27Yjm+9V90nfdgG2qKoPVNWDVXUj8GngwJ5tvl9VZ1fVw1X1wKoHSLIR8NfAP1fVv9MEzWHtunVoPqgXVtVtVbWyqi5p3+chwHeq6oy2/l9X1ZIOtX+wqu4aq6mqvtAe46Gq+ijwGGDs2s4bgXdX1XXVWNpuexlwD01g0L7vC6vqlx3q0AxieGiU/RrYvN9z8e0plQ+1p1TuBW5uV23e/nwVzW/MtyT5bpLd2vaPANcD5ye5Mcm7HkWt2wJPbk8b3Z3kbpoRzJY92yyf5BivAB4CzmuXvwjsk2SL9j1sCNwwzn7zJmjv1yPqSvK2JNe2p8buBh7HH/twda/1OeB17fPXAaevQU0acYaHRtn3gd/RnFbpx8E0p4b2pvnAm9+2B6CqLq+q/WlOaZ0N/Evbfl9Vva2qnga8HHhrkr3oZjlwU1Vt2vOYW1X79mwz2WjmMGAT4GdJfgF8GVif5jTRnTR98fQJXnu8doAVwEY9y1uNs80f6mqvb7wTeA3w+KralGZEkT5e6wvA/kl2ALaj6WPNUoaHRlZV3QO8BzgxyQFJNkqyfpJ9knx4nF3mAr+nGbFsRPMNLQCSbJDkkCSPa08J3QusbNftl+QZ7TedxtpXdiz3MuDeJO9sL2yvm2RBkl362TnJ2HWS/WiuN+wI7AD8E3BYVT0MnAp8rL0wv26S3ZI8hmaEsneS1yRZL8lmSXZsD70EeGXbd88AjpyklLk0o587gPWSvIfm2saYU4B/TPLMNLZPshlAVd1Kc73kdOCs8U7NafYwPDTSqupjNBdp303zgbYceDPj/1b7eeAW4DbgR8Clq6w/FLi5PaX1Jv54iuWZwHeA39KMdk7q+duOfutcSTNq2RG4iWakcArNCKgfhwJLqur8qvrF2AP4JLB9kgXA24FlNB/Qd9EEyzpV9TOa03Fva9uX0AQPwMeBB4Ff0pxWmuhC/Zhv0Vx8/wlNX/6OR57W+hjNiO18mqD9DDCnZ/3ngP+Ap6xmvXgzKElTJcmLaE5fzW9HS5qlHHlImhJJ1qf5+5RTDI7Zz/CQtMaSbAfcTfOV508MtRhNC09bSZI6c+QhSeps1k+Etvnmm9f8+fOHXYYkzShXXHHFnVW1xUTrZ314zJ8/n8WLFw+7DEmaUZLcsrr1nraSJHVmeEiSOjM8JEmdGR6SpM5m/QXzFcuXc+nChcMuQ5Km1a6LFg30+I48JEmdGR6SpM4MD0lSZ4aHJKkzw0OS1JnhIUnqzPCQJHVmeEiSOjM8JEmdGR6SpM4MD0lSZ4aHJKkzw0OS1JnhIUnqzPCQJHVmeEiSOjM8JEmdDTU8kqxMsiTJ1UnOSbJp2z4/yQPturHHBu26fZIsTnJtkh8nOW6Y70GS1kbDHnk8UFU7VtUC4C7g6J51N7Trxh4PJlkAnAC8rqq2AxYANw6hbklaqw07PHp9H9h6km3eAfyPqvoxQFU9VFUnDbwySdIjjER4JFkX2Av4ek/z03tOWZ3Yti0Arpj2AiVJj7DekF9/TpIlwHyaUPh2z7obqmrHR3PQJEcBRwFsNXfumlUoSfoTwx55PNAGxLbABjzymsd4rgGeN9lBq+rkqtq5qnbedM6cNa9SkvQIww4PAKrqHuAtwNuTrL+aTT8CHJvkzwCSrJPkrdNRoyTpj0YiPACq6ofAUuDA1WxzFXAMcEaSa4GrgSdNS4GSpD8Y6jWPqtpkleWX9ywumGCfc4FzB1mXJGn1RmbkIUmaOQwPSVJnhockqTPDQ5LUmeEhSerM8JAkdWZ4SJI6MzwkSZ0ZHpKkzgwPSVJnhockqTPDQ5LUmeEhSerM8JAkdWZ4SJI6MzwkSZ0N9WZQ02HjefPYddGiYZchSbOKIw9JUmeGhySpM8NDktSZ4SFJ6szwkCR1ZnhIkjozPCRJnRkekqTODA9JUmeGhySps1k/PcmK5cu5dOHCYZchSdNq0NMyOfKQJHVmeEiSOjM8JEmdGR6SpM4MD0lSZ4aHJKkzw0OS1JnhIUnqzPCQJHVmeEiSOjM8JEmdGR6SpM4MD0lSZ4aHJKkzw0OS1JnhIUnqzPCQJHVmeEiSOhtoeCR5RZJK8qx2eX6SB5L8MMm1SS5Lctg4+y1NcsYqbZ9NclOSJUl+nOS9g6xdkjSxQY88DgIuBg7sabuhqp5bVdu17f8tyRFjK5Ns19b1oiQbr3K8v6+qHYEdgcOSPHWQxUuSxjew8EiyCfAXwJE8Mjz+oKpuBN4KvKWn+WDgdOB84K8mOPyG7c8VU1KsJKmTQY48DgC+WVU/Ae5KstME210JPKtn+bXAl4AzaEYuvT6SZAlwK3BmVf1qvAMmOSrJ4iSL737ggTV4C5Kk8QwyPA4Czmyfn8mfBsGY/OFJsgtwR1XdAvxfYKckj+/Zduy01VbAXkl2H++AVXVyVe1cVTtvOmfOGr4NSdKq1hvEQZNsBrwEWJCkgHWBAk4aZ/PnAte2zw8CnpXk5nb5scCrgFN6d6iq3ya5ENgDuGSq65ckrd6gRh6vBj5fVdtW1fyqmgfcBGzTu1GS+cBxwPFJ1gH+Gti+3Wc+sD/jjFiSrAe8ALhhQPVLklZjICMPmg/8D63SdhZwLPD0JD+kueh9H3B8VZ2WZE/gtqq6rWefi4BnJ3lSu/yRJO8GNqA5rfXVAdUvSVqNgYRHVe05TtsngU+uZp8LgV1XaVsJjAXH4VNWoCRpjfgX5pKkziYNjyTrJLl6OoqRJM0Mk4ZHVT0MLE3ylGmoR5I0A/R7zeNJwDVJLqPnr7qraqK/AJckzWL9hsf7B1qFJGlG6Ss8quq7SbYFnllV30myEc0f/kmS1kJ9fdsqyd8AXwH+d9u0NXD2gGqSJI24fr+qezTNDLn3AlTVT4EnDqooSdJo6zc8fl9VD44ttNOD1GBKkiSNun7D47tJjgXmJHkp8GXgnMGVJUkaZf2Gx7uAO4BlwN8C5wHvHlRRkqTR1u+3rR4GPt0+JElrudWGR5J/qarXJFnGONc4qmr7gVUmSRpZk408jml/7jfgOiRJM8hk4XEusBPw36vq0GmoR5I0A0wWHhskOQzYPckrV11ZVd6MSZLWQpOFx5uAQ4BNgZevsq6YAXfy23jePHZdtGjYZUjSrLLa8Kiqi4GLkyyuqs9MU02SpBE32betXlJVFwC/8bSVJGnMZKet/iNwAX96ygpmyGkrSdLUm+y01Xvbn0dMTzmSpJmg3ynZFyZ5bBqnJLkyycsGXZwkaTT1O7fVG6rqXuBlNFOxHwF8aGBVSZJGWr/hkfbnvsBpVbW0p02StJbpNzyuSHI+TXh8K8lc4OHBlSVJGmV9zaoLHAnsCNxYVfcneQLNqStJ0lqo35HHbsB1VXV3ktfR3MvjnsGVJUkaZf2OPP4XsEOSHYB3AJ8BPk/zdyAjbcXy5Vy6cOGwy5CkaTXoaZn6HXk8VFUF7A8sqqpFwNzBlSVJGmX9jjzuS/IPwOuAFyVZF1h/cGVJkkZZvyOP1wK/B46sql8AWwMfGVhVkqSR1u89zH8BfKxn+Wc01zwkSWuhfqcn2TXJ5Ul+m+TBJCuT+G0rSVpL9Xva6gTgIOCnwBzgjcCJgypKkjTa+r1gTlVdn2TdqloJnJbkkgHWJUkaYf2Gx/1JNgCWJPkwcDuw8eDKkiSNsn5PWx0KrAu8GVgBzANeNaiiJEmjrd9vW93SPn0AeP/gypEkzQST3cN8Gc3tZsdVVdtPeUWSpJE32cjjlcCWwPJV2rcFfj6QiiRJI2+yax4fB+6tqlt6H8D97TpJ0lposvCYX1VXrdpYVYuB+QOpSJI08iYLjw1Xs27OVBYiSZo5JguPy5P8zaqNSY4ErhhMSZKkUTfZBfNjgK8lOYQ/hsXOwAbAKwZYlyRphK02PKrql8DuSV4MLGib/09VXbCmL5xkS5qL7rsCvwEeBD7cPv9X4CaakdGvgIOr6lftfq+nuZth2sepVXXcmtYjSepfX39hXlX/VlXHt4+pCI4AZwMXVdXTqup5wIHANu0m36uqHdu/I7kcOLrdbx+a0dDLquo5wE54L3VJmnb9Tk8y1V4CPFhVnxpraL8GfHzvRm3IzKUZjQD8A/D2qvp5u8/vqurT01SzJKnV96y6U+w5wJWrWf/CJEuAzWjm0jq2bV9AHxfqkxwFHAWw1VxvtS5JU21YI49HSHJikqVJLm+bxk5bzQNOo7kW0reqOrmqdq6qnTed4zeKJWmqDSs8rqG5XgFAVR0N7AVsMc62Xwde1LPf8wZenSRptYYVHhcAGyb5Lz1tG02w7R7ADe3zDwIfTrIVQJLHJHnL4MqUJI1nKNc8qqqSHAB8PMk7gDtorm28s91k7JpHaL5N9cZ2v/Par/h+p72YXsCp01y+JK31hnXBnKq6nebrueN53Gr2O43mOogkaUhG4oK5JGlmMTwkSZ0ZHpKkzgwPSVJnhockqTPDQ5LUmeEhSerM8JAkdWZ4SJI6MzwkSZ0ZHpKkzgwPSVJnhockqTPDQ5LUmeEhSerM8JAkdTa0m0FNl43nzWPXRYuGXYYkzSqOPCRJnRkekqTODA9JUmeGhySpM8NDktSZ4SFJ6szwkCR1ZnhIkjozPCRJnRkekqTOZv30JCuWL+fShQuHXYYkTatBT8vkyEOS1JnhIUnqzPCQJHVmeEiSOjM8JEmdGR6SpM4MD0lSZ4aHJKkzw0OS1JnhIUnqzPCQJHVmeEiSOjM8JEmdGR6SpM4MD0lSZ4aHJKkzw0OS1NnQwiNJJTm9Z3m9JHckObddPjzJCePsd3OSZUmWJjk/yVbTWbckabgjjxXAgiRz2uWXArf1ue+Lq2oHYDFw7CCKkyRNbNinrb4B/Of2+UHAGR33vwh4xpRWJEma1LDD40zgwCQbAtsDP+i4/37AsimvSpK0WkMNj6q6CphPM+o4r8Ou/5ZkCfBY4IOrrkxyVJLFSRbf/cADU1GqJKnHesMuAPg6cBywJ7BZn/u8uKrunGhlVZ0MnAyw3ZZb1poWKEl6pFEIj1OBe6pqWZI9h1yLJKkPw77mQVXdWlWLJlh9eJJbex7bTGtxkqRxDW3kUVWbjNN2IXBh+/yzwGfH2XX+4KqSJPVj6CMPSdLMY3hIkjozPCRJnRkekqTODA9JUmeGhySpM8NDktSZ4SFJ6szwkCR1ZnhIkjozPCRJnRkekqTODA9JUmeGhySpM8NDktSZ4SFJ6mwUbkM7UBvPm8euiya6UaEk6dFw5CFJ6szwkCR1ZnhIkjozPCRJnRkekqTOUlXDrmGgktwHXDfsOkbA5sCdwy5iBNgP9sEY+2H1fbBtVW0x0Y6z/qu6wHVVtfOwixi2JIvtB/sB7IMx9sOa9YGnrSRJnRkekqTO1obwOHnYBYwI+6FhP9gHY+yHNeiDWX/BXJI09daGkYckaYoZHpKkzmZNeCT5T0muS3J9kneNsz5JPtmuvyrJTsOoc9D66IdD2vd/VZJLkuwwjDoHabI+6NlulyQrk7x6OuubLv30Q5I9kyxJck2S7053jYPWx/8Pj0tyTpKlbR8cMYw6BynJqUl+leTqCdY/us/GqprxD2Bd4AbgacAGwFLg2atssy/wDSDArsAPhl33kPphd+Dx7fN9Zls/9NMHPdtdAJwHvHrYdQ/p38KmwI+Ap7TLTxx23UPog2OBf2qfbwHcBWww7NqnuB9eBOwEXD3B+kf12ThbRh7PB66vqhur6kHgTGD/VbbZH/h8NS4FNk3ypOkudMAm7YequqSqftMuXgpsM801Dlo//xYA/g44C/jVdBY3jfrph4OBr1bVzwCqarb1RT99UMDcJAE2oQmPh6a3zMGqqoto3tdEHtVn42wJj62B5T3Lt7ZtXbeZ6bq+xyNpfuOYTSbtgyRbA68APjWNdU23fv4t/Bnw+CQXJrkiyeunrbrp0U8fnABsB/wcWAYsrKqHp6e8kfGoPhtny/QkGadt1e8g97PNTNf3e0zyYprw2GOgFU2/fvrgE8A7q2pl8wvnrNRPP6wHPA/YC5gDfD/JpVX1k0EXN0366YO/BJYALwGeDnw7yfeq6t4B1zZKHtVn42wJj1uBeT3L29D8JtF1m5mur/eYZHvgFGCfqvr1NNU2Xfrpg52BM9vg2BzYN8lDVXX2tFQ4Pfr9f+LOqloBrEhyEbADMFvCo58+OAL4UDUn/69PchPwLOCy6SlxJDyqz8bZctrqcuCZSZ6aZAPgQODrq2zzdeD17TcLdgXuqarbp7vQAZu0H5I8BfgqcOgs+g2z16R9UFVPrar5VTUf+ArwX2dZcEB//0/8K/DCJOsl2Qh4AXDtNNc5SP30wc9oRl4k2RL4c+DGaa1y+B7VZ+OsGHlU1UNJ3gx8i+YbFqdW1TVJ3tSu/xTNt2r2Ba4H7qf5jWNW6bMf3gNsBpzU/ub9UM2imUX77INZr59+qKprk3wTuAp4GDilqsb9OudM1Oe/hX8EPptkGc3pm3dW1ayapj3JGcCewOZJbgXeC6wPa/bZ6PQkkqTOZstpK0nSNDI8JEmdGR6SpM4MD0lSZ4aHJKkzw0OaRJJXJKkkzxp2LdKoMDykyR0EXEzzR2YDk2TdQR5fmkqGh7QaSTYB/oJmHrADe9rXTXJckmXtPRD+rm3fpb1PytIklyWZm+TwJCf07Htukj3b579N8oEkPwB2S/KeJJcnuTrJye1sryR5RpLvtMe9MsnTk5yeZP+e434xyV9NR79Ihoe0egcA32yncrmr50Y5RwFPBZ5bVdsDX2ynwPgSzcysOwB7Aw9McvyNae6z8IKquhg4oap2qaoFNJMV7tdu90XgxPa4uwO308xPdgQ0NzVq28+bijctTcbwkFbvIJr7QND+PKh9vjfwqap6CKCq7qKZF+n2qrq8bbt3bP1qrKS5r8iYFyf5QTtdxkuA5ySZC2xdVV9rj/u7qrq/qr4LPCPJE9u6zurj9aQpMSvmtpIGIclmNB/gC5IUzfxIleQdNPMgjTft/3jz/TzEI39R27Dn+e+qamX7ehsCJwE7V9XyJO9rt13dvPGnA4fQnFJ7Q59vTVpjjjykib2a5g5r27az8M4DbqK5B8r5wJuSrAeQ5AnAj4EnJ9mlbZvbrr8Z2DHJOknm0dzhbjxjoXJne63l1dCMYIBbkxzQHvcx7Sy4AJ8Fjmm3u2aq3rg0GcNDmthBwNdWaTuL5vatp9BM531VkqXAwe2tTl8LHN+2fZsmEP4fTegsA44DrhzvxarqbuDT7XZn00wpPuZQ4C1JrgIuAbZq9/klzTTqp63ZW5W6cVZdaQZrRyDLgJ2q6p5h16O1hyMPaYZKsjfNqbLjDQ5NN0cekqTOHHlIkjozPCRJnRkekqTODA9JUmeGhySps/8P4YJWOa2M9rwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAXbUlEQVR4nO3deZhldX3n8feHpaWRplEaEWmkFBWXDhAEIwyaFqJGXMAtgrgxKCGDo4QYVB7HJeOMCDwkboyDCMRkhIk7KBoXbNEBlQZZRSKrNEqAyNJgKwLf+eOchktZ1XVPd926t6rfr+e5T937O8v93t/TfT/1O+fU76SqkCSpXxsMuwBJ0uxicEiSOjE4JEmdGBySpE4MDklSJwaHJKkTg0OzXpL3J/nnAe7/iiRL2+dJcmqS25P8OMlzklw1qPeWRpHBoVkhyWuTLE9yd5JfJfl6kr1m4r2r6hlVtax9uRfwfGBxVT2rqr5fVTtO13slOS3JB6drf5O8x0CDVnOfwaGRl+RI4B+A/wlsDTweOBHYbwjlbA9cX1X3rOuOkmw0DfVIM87g0EhLshD4O+DwqvpiVd1TVb+vqrOq6m8n2eZzSW5OcmeSc5M8o2fZvkl+mmRlkpuSvKNtX5Tkq0nuSPLrJN9PskG77Pokf5bkEOBkYI925POBJEuTrOjZ/+OSfCHJrUmuS/K2nmXvT/L5JP+c5C7gTR374i1Jrm7rOzPJ43qWvSDJVe1nPjHJ95K8ucv+2/28rD00d0eSZUme1rPsnW2frWzfa5+2/VntaPCuJP+e5ISu76vZxeDQqNsD2AT4Uodtvg48GXgMcBHwf3qWfRr4y6paACwBzmnb/wZYAWxFM6o5GnjYfDxV9WngMOD8qtqsqt7Xu7wNmrOAS4BtgX2AI5K8sGe1/YDPA1uMq2uNkuwNfAj4C2Ab4AbgjHbZonaf7wa2BK4C9ux33z3v8RTgdOAImn44GzgrybwkOwJvBXZv++6FwPXtph8BPlJVmwM7AP/S9b01uxgcGnVbArdV1X39blBVp1TVyqr6HfB+YOd25ALwe+DpSTavqtur6qKe9m2A7dsRzfer+0RuuwNbVdXfVdW9VXUt8CnggJ51zq+qL1fVA1W1qsO+DwJOqaqL2s/1bpqRzxiwL3BFOyK7D/gocHPH2gFeA3ytqr5VVb8Hjgfm04TQ/cAjaPpu46q6vqquabf7PfCkJIuq6u6q+uFavLdmEYNDo+4/gEX9ng9IsmGSY5Jc0x4Our5dtKj9+UqaL9ob2sM5e7TtxwFXA99Mcm2Sd61FrdsDj2sP89yR5A6akcvWPevcuBb7BXgczSgDgKq6m6Zvtm2X3dizrGhGT+v6Hg+0+922qq6mGYm8H7glyRk9h8oOAZ4C/CzJBUleshbvrVnE4NCoOx/4LbB/n+u/luZw0J8BC4Gxtj0AVXVBVe1Hcxjry7SHVdoRyt9U1ROBlwJHrj6G38GNwHVVtUXPY0FV7duzztpOR/1LmmBqPkzySJrR2E3Ar4DFPcvS+3od3iPAdu17UFWfraq92nUK+HDb/vOqOpCmTz8MfL6tT3OUwaGRVlV3Au8FPpFk/ySbJtk4yYuSHDvBJguA39H8Nr4pzZVYALTH6g9KsrA9FHMXzSEYkrwkyZPaL8vV7fd3LPfHwF3tSeT57ehnSZLdO+5nwySb9DzmAZ8FDk6yS5JHtJ/rR1V1PfA14I/a/tkIOBx47BTvscG493gETYi+OMk+STamOe/zO+C8JDsm2btd77fAKh7qu9cl2aododzR7r9r32kWMTg08qrqBOBI4D3ArTS/2b+VZsQw3mdoDrfcBPwUGH+8/fXA9e1hrMOA17XtTwa+DdxNM8o5sedvN/qt836a0couwHXAbTRXYS1cw2YTeRfNF/PqxzlV9R3gvwFfoBlh7EB77qSqbgNeDRxLE5hPB5bTfOlP5sBx73FNVV1F0x8fa2t/KfDSqrqX5vzGMW37zTSji6Pbff05cEWSu2lOlB9QVb/t+Jk1i8QbOUlzS3t11wrgoKr67rDr0dzjiEOaA5K8MMkW7aGko2nO6Xh1kwbC4JDmhj2Aa3joENP+HS/3lfrmoSpJUieOOCRJncz5SdYWLVpUY2Njwy5DkmaVCy+88Laq2mqiZXM+OMbGxli+fPmwy5CkWSXJDZMt81CVJKkTg0OS1InBIUnqxOCQJHUy50+Or1h5I0ctO3LYZUjSWjt26WjdVNERhySpE4NDktSJwSFJ6sTgkCR1YnBIkjoxOCRJnRgckqRODA5JUicGhySpE4NDktSJwSFJ6sTgkCR1YnBIkjoxOCRJnRgckqRODA5JUicGhySpk6EGR5L7k1yc5PIkZyXZom0fS7KqXbb6Ma9d9qIky5NcmeRnSY4f5meQpPXNsEccq6pql6paAvwaOLxn2TXtstWPe5MsAT4OvK6qngYsAa4dQt2StN4adnD0Oh/Ydop1jgL+R1X9DKCq7quqEwdemSTpQSMRHEk2BPYBzuxp3qHnMNUn2rYlwIUzXqAk6UEbDfn95ye5GBijCYRv9Sy7pqp2WZudJjkUOBRg860XrFuFkqSHGfaIY1UbDtsD83j4OY6JXAE8c6qdVtVJVbVbVe02f+H8da9SkvSgYQcHAFV1J/A24B1JNl7DqscBRyd5CkCSDZIcORM1SpIaIxEcAFX1E+AS4IA1rHMpcARwepIrgcuBbWakQEkSMORzHFW12bjXL+15uWSSbb4KfHWQdUmSJjcyIw5J0uxgcEiSOjE4JEmdGBySpE4MDklSJwaHJKkTg0OS1InBIUnqxOCQJHVicEiSOjE4JEmdGBySpE4MDklSJwaHJKkTg0OS1InBIUnqZKg3cpoJixdsx7FLTxh2GZI0ZzjikCR1YnBIkjoxOCRJnRgckqRODA5JUicGhySpE4NDktSJwSFJ6sTgkCR1YnBIkjqZ81OOrFh5I0ctO3LYZUjSWhu1aZMccUiSOjE4JEmdGBySpE4MDklSJwaHJKkTg0OS1InBIUnqxOCQJHVicEiSOjE4JEmdGBySpE4MDklSJwaHJKkTg0OS1InBIUnqxOCQJHVicEiSOjE4JEmdDDQ4krw8SSV5avt6LMmqJD9JcmWSHyd54wTbXZLk9HFtpyW5LsnFSX6W5H2DrF2SNLFBjzgOBH4AHNDTdk1V/XFVPa1t/+skB69emORpbV3PTfLIcfv726raBdgFeGOSJwyyeEnSHxpYcCTZDPhPwCE8PDgeVFXXAkcCb+tpfi3wT8A3gZdNsvtN2p/3TEuxkqS+DXLEsT/wjar6N+DXSXadZL2LgKf2vH4N8H+B02lGLL2OS3IxsAI4o6pumWiHSQ5NsjzJ8lV3rlqHjyBJGm+QwXEgcEb7/Az+MARWy4NPkt2BW6vqBuA7wK5JHtWz7upDVY8F9kmy50Q7rKqTqmq3qtpt/sL56/gxJEm9NhrETpNsCewNLElSwIZAASdOsPofA1e2zw8Enprk+vb15sArgZN7N6iqu5MsA/YCzpvu+iVJkxvUiONVwGeqavuqGquq7YDrgMW9KyUZA44HPpZkA+DVwE7tNmPAfkwwUkmyEfAnwDUDql+SNImBjDhovuyPGdf2BeBoYIckP6E5wb0S+FhVnZpkKXBTVd3Us825wNOTbNO+Pi7Je4B5NIeyvjig+iVJkxhIcFTV0gnaPgp8dA3bLAOePa7tfmB1aLxp2gqUJK01/3JcktTJlMGRZIMkl89EMZKk0TdlcFTVA8AlSR4/A/VIkkZcv+c4tgGuSPJjev5au6om+8tuSdIc1W9wfGCgVUiSZo2+gqOqvpdke+DJVfXtJJvS/FGfJGk909dVVUneAnwe+N9t07bAlwdUkyRphPV7Oe7hNDPd3gVQVT8HHjOooiRJo6vf4PhdVd27+kU75UcNpiRJ0ijrNzi+l+RoYH6S5wOfA84aXFmSpFHVb3C8C7gVuAz4S+Bs4D2DKkqSNLr6varqAeBT7UOStB5bY3Ak+Zeq+osklzHBOY2q2mlglUmSRtJUI44j2p8vGXAdkqRZYqrg+CqwK/DBqnr9DNQjSRpxUwXHvCRvBPZM8orxC6vKGylJ0npmquA4DDgI2AJ46bhlxSy4A9/iBdtx7NIThl2GJM0ZawyOqvoB8IMky6vq0zNUkyRphE11VdXeVXUOcLuHqiRJMPWhqj8FzuEPD1PBLDlUJUmaXlMdqnpf+/PgmSlHkjTq+p1W/e1JNk/j5CQXJXnBoIuTJI2efueq+s9VdRfwAprp1A8GjhlYVZKkkdVvcKT9uS9walVd0tMmSVqP9BscFyb5Jk1w/GuSBcADgytLkjSq+podFzgE2AW4tqp+k+TRNIerJEnrmX5HHHsAV1XVHUleR3MvjjsHV5YkaVT1O+L4X8DOSXYGjgI+DXyG5u88RtqKlTdy1LIjh12GJM2oQU611O+I476qKmA/4CNV9RFgwcCqkiSNrH5HHCuTvBt4HfDcJBsCGw+uLEnSqOp3xPEa4HfAIVV1M7AtcNzAqpIkjax+7zl+M3BCz+tf0JzjkCStZ/qdcuTZSS5IcneSe5Pcn8SrqiRpPdTvoaqPAwcCPwfmA28GPjGooiRJo6vfk+NU1dVJNqyq+4FTk5w3wLokSSOq3+D4TZJ5wMVJjgV+BTxycGVJkkZVv4eqXg9sCLwVuAfYDnjloIqSJI2ufq+quqF9ugr4wODKkSSNuqnuOX4ZzS1iJ1RVO017RZKkkTbViOMVwNbAjePatwd+OZCKJEkjbapzHH8P3FVVN/Q+gN+0yyRJ65mpgmOsqi4d31hVy4GxgVQkSRppUwXHJmtYNn86C5EkzQ5TBccFSd4yvjHJIcCFgylJkjTKpjo5fgTwpSQH8VBQ7AbMA14+wLokSSNqjcFRVf8O7JnkecCStvlrVXXOur5xkq1pTrA/G7gduBc4tn3+FeA6mhHRLcBrq+qWdrs30NyFMO3jlKo6fl3rkST1p6+/HK+q71bVx9rHdIRGgC8D51bVE6vqmcABwOJ2le9X1S7t34lcABzebvcimlHQC6rqGcCueO9zSZpR/U45Mt32Bu6tqk+ubmgv9f1Y70ptwCygGYUAvBt4R1X9st3mt1X1qRmqWZJEh9lxp9kzgIvWsPw5SS4GtqSZG+votn0JfZyUT3IocCjA5lt7a3RJmk7DGnE8TJJPJLkkyQVt0+pDVdsBp9Kc++hbVZ1UVbtV1W7zF3rVsCRNp2EFxxU05ycAqKrDgX2ArSZY90zguT3bPXPg1UmSJjWs4DgH2CTJX/W0bTrJunsB17TPPwQcm+SxAEkekeRtgytTkjTeUM5xVFUl2R/4+yRHAbfSnMt4Z7vK6nMcoblq6s3tdme3l/F+uz1xXsApM1y+JK3XhnVynKr6Fc0luBNZuIbtTqU57yFJGoKRODkuSZo9DA5JUicGhySpE4NDktSJwSFJ6sTgkCR1YnBIkjoxOCRJnRgckqRODA5JUicGhySpE4NDktSJwSFJ6sTgkCR1YnBIkjoxOCRJnQztRk4zZfGC7Th26QnDLkOS5gxHHJKkTgwOSVInBockqRODQ5LUicEhSerE4JAkdWJwSJI6MTgkSZ0YHJKkTgwOSVInc37KkRUrb+SoZUcOuwxJmhajMIWSIw5JUicGhySpE4NDktSJwSFJ6sTgkCR1YnBIkjoxOCRJnRgckqRODA5JUicGhySpE4NDktSJwSFJ6sTgkCR1YnBIkjoxOCRJnRgckqRODA5JUidDC44kleSfel5vlOTWJF9tX78pyccn2O76JJcluSTJN5M8dibrlqT13TBHHPcAS5LMb18/H7ipz22fV1U7A8uBowdRnCRpYsM+VPV14MXt8wOB0ztufy7wpGmtSJK0RsMOjjOAA5JsAuwE/Kjj9i8BLpv2qiRJkxpqcFTVpcAYzWjj7A6bfjfJxcDmwIfGL0xyaJLlSZavunPVdJQqSWptNOwCgDOB44GlwJZ9bvO8qrptsoVVdRJwEsBjd9y61rVASdJDRiE4TgHurKrLkiwdci2SpCkM+xwHVbWiqj4yyeI3JVnR81g8o8VJkv7A0EYcVbXZBG3LgGXt89OA0ybYdGxwVUmSpjL0EYckaXYxOCRJnRgckqRODA5JUicGhySpE4NDktSJwSFJ6sTgkCR1YnBIkjoxOCRJnRgckqRODA5JUicGhySpE4NDktSJwSFJ6sTgkCR1Mgq3jh2oxQu249ilJwy7DEmaMxxxSJI6MTgkSZ0YHJKkTgwOSVInBockqZNU1bBrGKgkK4Grhl3HCFgE3DbsIobMPrAPwD6A/vpg+6raaqIFc/5yXOCqqtpt2EUMW5Ll63s/2Af2AdgHsO594KEqSVInBockqZP1IThOGnYBI8J+sA/APgD7ANaxD+b8yXFJ0vRaH0YckqRpZHBIkjqZM8GR5M+TXJXk6iTvmmB5kny0XX5pkl2HUecg9dEHB7Wf/dIk5yXZeRh1DtJUfdCz3u5J7k/yqpmsbyb00wdJlia5OMkVSb430zUOWh//FxYmOSvJJW0fHDyMOgcpySlJbkly+STL1/47sapm/QPYELgGeCIwD7gEePq4dfYFvg4EeDbwo2HXPYQ+2BN4VPv8RetjH/Ssdw5wNvCqYdc9hH8HWwA/BR7fvn7MsOseQh8cDXy4fb4V8Gtg3rBrn+Z+eC6wK3D5JMvX+jtxrow4ngVcXVXXVtW9wBnAfuPW2Q/4TDV+CGyRZJuZLnSApuyDqjqvqm5vX/4QWDzDNQ5aP/8OAP4r8AXglpksbob00wevBb5YVb8AqKq51g/99EEBC5IE2IwmOO6b2TIHq6rOpflck1nr78S5EhzbAjf2vF7RtnVdZzbr+vkOofltYy6Zsg+SbAu8HPjkDNY1k/r5d/AU4FFJliW5MMkbZqy6mdFPH3wceBrwS+Ay4O1V9cDMlDcy1vo7ca5MOZIJ2sZfZ9zPOrNZ358vyfNogmOvgVY08/rpg38A3llV9ze/bM45/fTBRsAzgX2A+cD5SX5YVf826OJmSD998ELgYmBvYAfgW0m+X1V3Dbi2UbLW34lzJThWANv1vF5M85tE13Vms74+X5KdgJOBF1XVf8xQbTOlnz7YDTijDY1FwL5J7quqL89IhYPX7/+F26rqHuCeJOcCOwNzJTj66YODgWOqOdh/dZLrgKcCP56ZEkfCWn8nzpVDVRcAT07yhCTzgAOAM8etcybwhvZKgmcDd1bVr2a60AGasg+SPB74IvD6OfTbZa8p+6CqnlBVY1U1Bnwe+C9zKDSgv/8LXwGek2SjJJsCfwJcOcN1DlI/ffALmhEXSbYGdgSundEqh2+tvxPnxIijqu5L8lbgX2muqDilqq5Icli7/JM0V9DsC1wN/IbmN445o88+eC+wJXBi+xv3fTWHZgntsw/mtH76oKquTPIN4FLgAeDkqprwks3ZqM9/B/8dOC3JZTSHbN5ZVXNqqvUkpwNLgUVJVgDvAzaGdf9OdMoRSVInc+VQlSRphhgckqRODA5JUicGhySpE4NDktSJwSH1Kcnd07y/sclmLpVGmcEhSerE4JDWQZJdkvywvZ/Bl5I8qm3fvW07P8lxXUYWSfZJ8pMkl7X3VHhE235Mkp+2+z2+bXt1ksvb+0qcO5hPKT2cwSGtm8/Q/NXxTjSzrL6vbT8VOKyq9gDu73dnSTYBTgNeU1V/RDO7w18leTTNrL7PaN/rg+0m7wVeWFU7Ay+bhs8jTcngkNZSkoXAFlW1+g56/wg8N8kWwIKqOq9t/2yH3e4IXNczl9g/0tyQ5y7gt8DJSV5BM0UEwP+jmTrjLTTTa0gDZ3BI029d5mufcNuquo/mBkVfAPYHvtG2Hwa8h2aW04uTbLkO7y31xeCQ1lJV3QncnuQ5bdPrge+1d1lc2c44Cs3srP36GTCW5Em9+0yyGbCwqs4GjgB2AUiyQ1X9qKreC9zGw6fJlgZiTsyOK82QTdtZRlc7AXgj8Ml2evJreWiG0UOATyW5B1gG3DnJPncct8+/bvfxuSQb0UwR/kng0cBX2nMgadcDOC7Jk9u279DcX1saKGfHlQYgyWZVdXf7/F3ANlX19iGXJU0LRxzSYLw4ybtp/o/dALxpuOVI08cRhySpE0+OS5I6MTgkSZ0YHJKkTgwOSVInBockqZP/D8vI8p2gMr1SAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.set_color_codes(\"muted\")\n",
    "\n",
    "sns.barplot(x='F1 Score', y='Classifier', data=df_results, color=\"b\")\n",
    "\n",
    "plt.xlabel('F1 Score ')\n",
    "plt.title('Classifier F1 Score')\n",
    "plt.show()\n",
    "\n",
    "sns.barplot(x='Accuracy', y='Classifier', data=df_results, color=\"r\")\n",
    "\n",
    "plt.xlabel('Accuracy ')\n",
    "plt.title('Classifier Accuracy')\n",
    "plt.show()\n",
    "\n",
    "sns.set_color_codes(\"muted\")\n",
    "sns.barplot(x='LogLos', y='Classifier', data=df_results, color=\"g\")\n",
    "\n",
    "plt.xlabel('Log Loss')\n",
    "plt.title('Classifier Log Loss')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1197ea4",
   "metadata": {},
   "source": [
    "<b>Further thoughs</b>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "240d19bd",
   "metadata": {},
   "source": [
    "RFC and ADAB have improved results, while GBC and MLP have similar ones. For this phase a lot of parameters were tested so as to conclude to the above results. One could try further experimenting with the parameters for all the classifiers to see if they improve or if they get better results from the ones that we had considered as optimal ones. Also, combining the classifiers or applying a pipeline with grid search, might be something that could give better results. Moreover, including more features in the training phase could be useful, due to the fact that the feature selection may provide with different results. \n",
    "\n",
    "While running feature selection with the sampling dataset, the results showed that the optimal number of features were 6. This could happen with the whole dataset so as to see if the optimal number of features are still 6 (due to run time, this couldn't happen here)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}