ML0101EN-Proj-Loan-py-v1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<a href=\"https://www.bigdatauniversity.com\"><img src=\"https://ibm.box.com/shared/static/cw2c7r3o20w9zn8gkecaeyjhgw3xdgbj.png\" width=\"400\" align=\"center\"></a>\n",
    "\n",
    "<h1 align=\"center\"><font size=\"5\">Classification with Python</font></h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "In this notebook we try to practice all the classification algorithms that we learned in this course.\n",
    "\n",
    "We load a dataset using Pandas library, and apply the following algorithms, and find the best one for this specific dataset by accuracy evaluation methods.\n",
    "\n",
    "Lets first load required libraries:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "import itertools\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.ticker import NullFormatter\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.ticker as ticker\n",
    "from sklearn import preprocessing\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### About dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "This dataset is about past loans. The __Loan_train.csv__ data set includes details of 346 customers whose loan are already paid off or defaulted. It includes following fields:\n",
    "\n",
    "| Field          | Description                                                                           |\n",
    "|----------------|---------------------------------------------------------------------------------------|\n",
    "| Loan_status    | Whether a loan is paid off on in collection                                           |\n",
    "| Principal      | Basic principal loan amount at the                                                    |\n",
    "| Terms          | Origination terms which can be weekly (7 days), biweekly, and monthly payoff schedule |\n",
    "| Effective_date | When the loan got originated and took effects                                         |\n",
    "| Due_date       | Since it’s one-time payoff schedule, each loan has one single due date                |\n",
    "| Age            | Age of applicant                                                                      |\n",
    "| Education      | Education of applicant                                                                |\n",
    "| Gender         | The gender of applicant                                                               |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets download the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2020-05-10 12:10:14--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv\n",
      "Resolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\n",
      "Connecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 23101 (23K) [text/csv]\n",
      "Saving to: ‘loan_train.csv’\n",
      "\n",
      "loan_train.csv      100%[===================>]  22.56K  70.6KB/s    in 0.3s    \n",
      "\n",
      "2020-05-10 12:10:16 (70.6 KB/s) - ‘loan_train.csv’ saved [23101/23101]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget -O loan_train.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Load Data From CSV File  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>10/7/2016</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>10/7/2016</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>9/22/2016</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/9/2016</td>\n",
       "      <td>10/8/2016</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/9/2016</td>\n",
       "      <td>10/8/2016</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30       9/8/2016   \n",
       "1           2             2     PAIDOFF       1000     30       9/8/2016   \n",
       "2           3             3     PAIDOFF       1000     15       9/8/2016   \n",
       "3           4             4     PAIDOFF       1000     30       9/9/2016   \n",
       "4           6             6     PAIDOFF       1000     30       9/9/2016   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0  10/7/2016   45  High School or Below    male  \n",
       "1  10/7/2016   33              Bechalor  female  \n",
       "2  9/22/2016   27               college    male  \n",
       "3  10/8/2016   28               college  female  \n",
       "4  10/8/2016   29               college    male  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('loan_train.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(346, 10)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Convert to date time object "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30     2016-09-08   \n",
       "1           2             2     PAIDOFF       1000     30     2016-09-08   \n",
       "2           3             3     PAIDOFF       1000     15     2016-09-08   \n",
       "3           4             4     PAIDOFF       1000     30     2016-09-09   \n",
       "4           6             6     PAIDOFF       1000     30     2016-09-09   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0 2016-10-07   45  High School or Below    male  \n",
       "1 2016-10-07   33              Bechalor  female  \n",
       "2 2016-09-22   27               college    male  \n",
       "3 2016-10-08   28               college  female  \n",
       "4 2016-10-08   29               college    male  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['due_date'] = pd.to_datetime(df['due_date'])\n",
    "df['effective_date'] = pd.to_datetime(df['effective_date'])\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Data visualization and pre-processing\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Let’s see how many of each class is in our data set "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PAIDOFF       260\n",
       "COLLECTION     86\n",
       "Name: loan_status, dtype: int64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['loan_status'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "260 people have paid off the loan on time while 86 have gone into collection \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets plot some columns to underestand data better:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fetching package metadata ...........\n",
      "Solving package specifications: ....\n",
      "\n",
      "\n",
      "UnsatisfiableError: The following specifications were found to be in conflict:\n",
      "  - python 3.4*\n",
      "  - seaborn\n",
      "Use \"conda info <package>\" to see the dependencies for each package.\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# notice: installing seaborn might takes a few minutes\n",
    "!conda install -c anaconda seaborn -y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAG1NJREFUeJzt3XucFOWd7/HPV5w4KsYLTBRBnJGMqCgZcFbjdQmsLGq8rYnBuAobzyEajWETj8bkvHQTX2tiJDHJUYlk5ZBNFDVkRZckGrxwlCReBhy5eCFEBx0WENBNYhAC+Dt/VM2kwQZ6pnuma3q+79erXlP1VNVTv66ZZ35dT1c/pYjAzMwsa3YrdwBmZmb5OEGZmVkmOUGZmVkmOUGZmVkmOUGZmVkmOUGZmVkmOUF1EUkHSrpH0quSFkj6raTzSlT3KElzSlFXd5A0T1JjueOw8qiktiCpRtIzkp6XdEoXHuedrqq7J3GC6gKSBMwGnoyIwyLiWGA8MKhM8exejuOaVWBbGAO8HBEjIuKpUsRkO+YE1TVGA3+JiB+0FUTEioj4PwCS+ki6RdJzkhZJ+mxaPiq92pgl6WVJd6cNHEnj0rL5wD+01Stpb0nTJT2bvqs7Jy2fKOmnkv4T+FUxL0bSDElTJT2RvgselR7zJUkzcrabKqlJ0lJJX9tBXWPTd9AL0/j6FhObZV7FtAVJDcC3gDMkNUvac0d/z5JaJN2UrmuSNFLSI5J+L+mydJu+kh5L913cFm+e4/6vnPOTt11VrIjwVOIJuAq4dSfrJwH/O53fA2gC6oBRwB9I3l3uBvwWOBmoBt4A6gEB9wNz0v1vAv4xnd8PWAbsDUwEWoEDdhDDU0Bznunv8mw7A7g3PfY5wB+BY9IYFwAN6XYHpD/7APOA4enyPKAR6A88Ceydll8LXF/u35enrpsqsC1MBG5L53f49wy0AJen87cCi4B9gBpgTVq+O/DBnLqWA0qX30l/jgWmpa91N2AOcGq5f6/dNbnrpxtIup2kcf0lIv6G5I9uuKRPpJvsS9Lg/gI8GxGt6X7NQC3wDvBaRPwuLf8JScMmretsSVeny9XA4HR+bkS8lS+miOho//l/RkRIWkzSwBansSxNY2wGLpA0iaThDQCOImmYbT6alv06fTP8AZJ/PNZLVEhbaLOrv+eH0p+Lgb4R8SfgT5I2SdoP+DNwk6RTgfeAgcCBwOqcOsam0/Ppcl+S8/NkJ2PuUZygusZS4Py2hYi4QlJ/kneHkLwb+nxEPJK7k6RRwKacoq3s+nck4PyIeGW7uo4naQD5d5KeInlHt72rI+LRPOVtcb23XYzvAbtLqgOuBv4mIt5Ou/6q88Q6NyIu3MnrscpSiW0h93g7+3veaZsBLiK5ojo2IjZLaiF/m/lGRNy5kzgqlj+D6hqPA9WSLs8p2ytn/hHgcklVAJIOl7T3Tup7GaiVNCRdzm0QjwCfz+mfH1FIgBFxSkQ05Jl21iB35oMk/wT+IOlA4PQ82zwNnCTpw2mse0s6vJPHs56hkttCsX/P+wJvpsnpY8ChebZ5BPhMzmdbAyV9qAPH6NGcoLpAJJ3H5wJ/K+k1Sc8CPyLpowb4N+BFYKGkJcCd7OTdYURsJOnG+Hn6wfCKnNU3AlXAorS77cZSv55CRMQLJN0QS4HpwK/zbLOWpA9/pqRFJN0hR3RjmNbNKrktlODv+W6gUVITydXUy3mO8SvgHuC3aff6LPJf7VWktg/kzMzMMsVXUGZmlklOUGZmlklOUGZmlklOUGZmlkmZSFDjxo0LwJOnSppKxu3DU4VNBctEglq3bl25QzDLLLcP660ykaDMzMy25wRlZmaZ5ARlZmaZ5MFizayibN68mdbWVjZu3FjuUHq16upqBg0aRFVVVafrcIIys4rS2trKPvvsQ21tLem4sdbNIoL169fT2tpKXV1dp+txF5+ZVZSNGzfSr18/J6cykkS/fv2Kvop1grJe49ABA5BUkunQAQPK/XJsJ5ycyq8UvwN38Vmv8frq1bQePKgkdQ36r9aS1GNmO+YrKDOraKW8ci706rlPnz40NDRw9NFH88lPfpINGza0r5s9ezaSePnlvz7+qaWlhaOPPhqAefPmse+++zJixAiGDh3Kqaeeypw5c7apf9q0aRxxxBEcccQRHHfcccyfP7993ahRoxg6dCgNDQ00NDQwa9asbWJqm1paWoo5rd3CV1BmVtFKeeUMhV0977nnnjQ3NwNw0UUX8YMf/IAvfvGLAMycOZOTTz6ZmTNn8rWvfS3v/qecckp7Umpububcc89lzz33ZMyYMcyZM4c777yT+fPn079/fxYuXMi5557Ls88+y0EHHQTA3XffTWNj4w5j6il2eQUlabqkN9OnXbaV/YuklZKa0+mMnHXXSVou6RVJf99VgZuZ9QSnnHIKy5cvB+Cdd95h/vz53HXXXdx7770F7d/Q0MD111/PbbfdBsDNN9/MLbfcQv/+/QEYOXIkEyZM4Pbbb++aF1BGhXTxzQDG5Sm/NSIa0ukXAJKOAsYDw9J97pDUp1TBmpn1JFu2bOGXv/wlxxxzDAAPPvgg48aN4/DDD6dfv34sWLCgoHpGjhzZ3iW4dOlSjj322G3WNzY2snTp0vbliy66qL0rb/369QC8++677WXnnXdeKV5el9tlF19EPCmptsD6zgHujYhNwGuSlgPHAb/tdIRmZj1MWzKA5Arq0ksvBZLuvS984QsAjB8/npkzZ74v2eQT0aFBwCumi6+Yz6CulHQJ0AR8KSLeBgYCT+ds05qWvY+kScAkgMGDBxcRhlnlcfvo2fIlg7feeovHH3+cxYsXI4mtW7ciiVtuuWWX9T3//PMceeSRABx11FEsWLCA0aNHt69fsGABw4YNK+2LyIDO3sU3FRgCNACrgG93tIKImBYRjRHRWFNT08kwzCqT20flmTVrFhdffDErVqygpaWFN954g7q6Op566qmd7rdo0SJuvPFGrrjiCgCuueYarr322vauu+bmZmbMmMHnPve5Ln8N3a1TV1ARsaZtXtIPgbZ7IFcCh+RsOigtMzMri8EHHVTS760NTu+U66iZM2dy7bXXblN2/vnn5y1/6qmnGDFiBBs2bOBDH/oQ3//+9xkzZgwAZ599NitXruTEE09EEvvssw8/+clPGFCBXx5XIX2b6WdQcyLi6HR5QESsSuf/GTg+IsZLGgbcQ/K508HAY0B9RGzdWf2NjY3R1NRUzOsw2yVJJf2i7i7aTsmGMnD76JiXXnqpvTvMymsHv4uC28Yur6AkzQRGAf0ltQI3AKMkNZA8vrcF+CxARCyVdD/wIrAFuGJXycnMzCyfQu7iuzBP8V072f5fgX8tJigzMzMPdWRmZpnkBGVmZpnkBGVmZpnkBGVmZpnkBGVmFe3gQYNL+riNgwcVNrLH6tWrGT9+PEOGDOGoo47ijDPOYNmyZSxdupTRo0czdOhQ6uvrufHGG9u/sjBjxgyuvPLK99VVW1vLunXrtimbMWMGNTU12zxC48UXXwRg2bJlnHHGGdTX13PkkUdywQUXcN9997Vv17dv3/ZHclxyySXMmzePj3/84+11z549m+HDh3PkkUdyzDHHMHv27PZ1EydOZODAgWzatAmAdevWUVtb26HfSaH8uA0zq2irVr7B8dc/XLL6nvl6vrGztxURnHfeeUyYMKF91PLm5mbWrFnDxIkTmTp1KmPHjmXDhg2cf/753HHHHe0jRXTEpz71qfZRztts3LiRM888k+985zucddZZADzxxBPU1NS0D780atQopkyZ0j5e37x589r3f+GFF7j66quZO3cudXV1vPbaa5x22mkcdthhDB8+HEieLTV9+nQuv/zyDsfcEb6CMjMrsSeeeIKqqiouu+yy9rKGhgaWLVvGSSedxNixYwHYa6+9uO222/jmN79ZsmPfc889nHDCCe3JCeBjH/tY+wMRd2XKlCl85Stfoa6uDoC6ujquu+66bcYMnDx5MrfeeitbtmwpWdz5OEGZmZXYkiVL8o5Snu9RGUOGDOGdd97hj3/8Y4ePk9tt19DQwLvvvrvDYxeqkMd5DB48mJNPPpkf//jHnT5OIdzFZ2bWQ+Xr4usu1113Heeccw5nnnlmlx3DV1BmZiU2bNiwvA8jbHtURq5XX32Vvn378sEPfrBLj12ofDHme5xHfX09DQ0N3H///Z0+1q44QZmZldjo0aPZtGkT06ZNay977rnnqK+vZ/78+Tz66KNA8mDDq666imuuuaZkx/70pz/Nb37zG37+85+3lz388MMsXry4oP2vvvpqvvGNb9DS0gJAS0sLN910E1/60pfet+1Xv/pVpkyZUpK483EXn5lVtAEDDynozruO1LcrknjggQeYPHkyN998M9XV1dTW1vLd736XBx98kM9//vNcccUVbN26lYsvvnibW8tnzJixzW3dTz+dPAN2+PDh7LZbck1xwQUXMHz4cO677z7mz5/fvu0dd9zBiSeeyJw5c5g8eTKTJ0+mqqqK4cOH873vfa+g19fQ0MDNN9/MWWedxebNm6mqquJb3/pW+xOCcw0bNoyRI0eycOHCguruqIIet9HV/DgB6w5+3Ebv4MdtZEexj9vYZRefpOmS3pS0JKfsFkkvS1ok6QFJ+6XltZLeldScTj8oNBAzM7NchXwGNQPY/vp4LnB0RAwHlgHX5az7fUQ0pNNlmJmZdcIuE1REPAm8tV3ZryKi7RtaT5M82t3MLBOy8NFFb1eK30Ep7uL7DPDLnOU6Sc9L+n+STtnRTpImSWqS1LR27doShGFWOdw+Oq+6upr169c7SZVRRLB+/Xqqq6uLqqeou/gkfZXk0e53p0WrgMERsV7SscBsScMi4n1fkY6IacA0SD4ELiYOs0rj9tF5gwYNorW1FSf28qqurmbQoOI61zqdoCRNBD4OjIn0rUpEbAI2pfMLJP0eOBzwLUhm1i2qqqrax5Gznq1TXXySxgHXAGdHxIac8hpJfdL5w4B64NVSBGpmZr3LLq+gJM0ERgH9JbUCN5DctbcHMFcSwNPpHXunAl+XtBl4D7gsIt7KW7GZmdlO7DJBRcSFeYrv2sG2PwN+VmxQZmZmHovPzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyqaAEJWm6pDclLckpO0DSXEm/S3/un5ZL0vclLZe0SNLIrgrezMwqV6FXUDOAcduVfRl4LCLqgcfSZYDTSZ6kWw9MAqYWH6aZmfU2BSWoiHgS2P7JuOcAP0rnfwScm1P+75F4GthP0oBSBGtmZr1HMZ9BHRgRq9L51cCB6fxA4I2c7VrTsm1ImiSpSVLT2rVriwjDrPK4fZiV6CaJiAggOrjPtIhojIjGmpqaUoRhVjHcPsyKS1Br2rru0p9vpuUrgUNythuUlpmZmRWsmAT1EDAhnZ8APJhTfkl6N99HgT/kdAWamZkVZPdCNpI0ExgF9JfUCtwAfBO4X9KlwArggnTzXwBnAMuBDcA/lThmMzPrBQpKUBFx4Q5WjcmzbQBXFBOUmZmZR5IwM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMKmg083wkDQXuyyk6DLge2A/4n0Dbc6q/EhG/6HSEZmbWK3U6QUXEK0ADgKQ+JE/NfYDk+U+3RsSUkkRoZma9Uqm6+MYAv4+IFSWqz8zMerlSJajxwMyc5SslLZI0XdL++XaQNElSk6SmtWvX5tvErNdy+zArQYKS9AHgbOCnadFUYAhJ998q4Nv59ouIaRHRGBGNNTU1xYZhVlHcPsxKcwV1OrAwItYARMSaiNgaEe8BPwSOK8ExzMyslylFgrqQnO49SQNy1p0HLCnBMczMrJfp9F18AJL2Bk4DPptT/C1JDUAALdutMzMzK0hRCSoi/gz0267s4qIiMjMzwyNJmJlZRjlBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJhV1m7lZT6I+VQz6r9aS1WVmXcsJynqN2LqZ469/uCR1PfP1cSWpx8x2zF18ZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSUXfZi6pBfgTsBXYEhGNkg4A7gNqSZ4JdUFEvF3ssczMrPco1RXUxyKiISIa0+UvA49FRD3wWLpsvcyhAwYgqejp0AEDdn0wM6s4XfVF3XOAUen8j4B5wLVddCzLqNdXr6b14EFF11Oq0R/MrGcpxRVUAL+StEDSpLTswIhYlc6vBg7cfidJkyQ1SWpau3ZtCcIwqxxuH2alSVAnR8RI4HTgCkmn5q6MiCBJYmxXPi0iGiOisaampgRhmFUOtw+zEiSoiFiZ/nwTeAA4DlgjaQBA+vPNYo9jZma9S1EJStLekvZpmwfGAkuAh4AJ6WYTgAeLOY6ZmfU+xd4kcSDwgKS2uu6JiIclPQfcL+lSYAVwQZHHMTOzXqaoBBURrwIfyVO+HhhTTN1mZta7eSQJMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPL5BOwu+qJumZm1oNk8QnYvoIyM7NM6nSCknSIpCckvShpqaQvpOX/ImmlpOZ0OqN04ZqZWW9RTBffFuBLEbEwfWjhAklz03W3RsSU4sMzM7PeqtMJKiJWAavS+T9JegkYWKrAzMysdyvJZ1CSaoERwDNp0ZWSFkmaLmn/HewzSVKTpKa1a9eWIgyziuH2YVaCBCWpL/AzYHJE/BGYCgwBGkiusL6db7+ImBYRjRHRWFNTU2wYZhXF7cOsyAQlqYokOd0dEf8BEBFrImJrRLwH/BA4rvgwzcystynmLj4BdwEvRcR3cspzv6V1HrCk8+GZmVlvVcxdfCcBFwOLJTWnZV8BLpTUAATQAny2qAjNzKxXKuYuvvmA8qz6RefDMTMzS3gkCTMzyySPxWddRn2qSjIul/pUlSAaM+tpnKCsy8TWzRx//cNF1/PM18eVIBoz62ncxWdmZpnkBGVmZpnkBGVmZpnkBGVmZpnkBGVm1s2y+Hj1LPJdfGZm3SyLj1fPIl9BmZlZJjlBmZlZJrmLz8zMMjnyixOUmZllcuQXd/GZmVkmdVmCkjRO0iuSlkv6crH1+bZMM7PepUu6+CT1AW4HTgNageckPRQRL3a2Tt+WaWbWu3TVZ1DHAcsj4lUASfcC5wCdTlBZc+iAAby+enXR9Qw+6CBWrFpVgogqm5Tv2ZiWRW4bu1aqGxJ261NV0W1DEVH6SqVPAOMi4n+kyxcDx0fElTnbTAImpYtDgVdKHkjh+gPrynj8Yjj28thV7OsiotOfFmeofVTy7yjLKjn2gttG2e7ii4hpwLRyHT+XpKaIaCx3HJ3h2Mujq2PPSvvw76g8HHuiq26SWAkckrM8KC0zMzMrSFclqOeAekl1kj4AjAce6qJjmZlZBeqSLr6I2CLpSuARoA8wPSKWdsWxSqTsXSlFcOzl0ZNj74ie/Dode3mULPYuuUnCzMysWB5JwszMMskJyszMMqnXJChJfSQ9L2lOulwn6Zl0KKb70ps5kLRHurw8XV9b5rj3kzRL0suSXpJ0gqQDJM2V9Lv05/7ptpL0/TT2RZJGljn2f5a0VNISSTMlVWf1vEuaLulNSUtyyjp8niVNSLf/naQJ3fkaOsttoyyxu20UoNckKOALwEs5yzcDt0bEh4G3gUvT8kuBt9PyW9Ptyul7wMMRcQTwEZLX8GXgsYioBx5LlwFOB+rTaRIwtfvDTUgaCFwFNEbE0SQ3y4wnu+d9BrD9lwc7dJ4lHQDcABxPMprKDW0NN+PcNrqR20YH2kZEVPxE8j2sx4DRwBxAJN903j1dfwLwSDr/CHBCOr97up3KFPe+wGvbH59kVIEB6fwA4JV0/k7gwnzblSH2gcAbwAHpeZwD/H2WzztQCyzp7HkGLgTuzCnfZrssTm4bbhsFxlyWttFbrqC+C1wDvJcu9wP+OyK2pMutJH808Nc/HtL1f0i3L4c6YC3wf9MumH+TtDdwYES0DVK2GjgwnW+PPZX7urpVRKwEpgCvA6tIzuMCesZ5b9PR85yZ898BbhvdzG1jm/KdqvgEJenjwJsRsaDcsXTC7sBIYGpEjAD+zF8vpQGI5O1I5r4rkF6+n0Pyj+RgYG/e303QY2T1PBfDbaM83DYKV/EJCjgJOFtSC3AvSVfG94D9JLV9UTl3KKb2YZrS9fsC67sz4BytQGtEPJMuzyJplGskDQBIf76Zrs/SEFN/B7wWEWsjYjPwHyS/i55w3tt09Dxn6fwXwm2jPNw2Cjz/FZ+gIuK6iBgUEbUkH0Q+HhEXAU8An0g3mwA8mM4/lC6Trn88fYfQ7SJiNfCGpKFp0RiSR5bkxrh97Jekd9J8FPhDzmV4d3sd+KikvSSJv8ae+fOeo6Pn+RFgrKT903fJY9OyTHLbcNsoQve0je78oK3cEzAKmJPOHwY8CywHfgrskZZXp8vL0/WHlTnmBqAJWATMBvYn6X9+DPgd8ChwQLqtSB4U+XtgMcldQuWM/WvAy8AS4MfAHlk978BMks8DNpO8O7+0M+cZ+Ez6GpYD/1Tuv/kOvH63je6N3W2jgGN7qCMzM8ukiu/iMzOznskJyszMMskJyszMMskJyszMMskJyszMMskJKsMkbZXUnI54/FNJe+1gu19I2q8T9R8saVYR8bVI6t/Z/c06y22jd/Bt5hkm6Z2I6JvO3w0siIjv5KwXye/wvR3V0cXxtZB8z2FdOY5vvZfbRu/gK6ie4yngw5JqlTz75g5gIXBI27u1nHU/VPKsmV9J2hNA0oclPSrpBUkLJQ1Jt1+Srp8o6UFJD0t6RdINbQeWNFvSgrTOSWV59WY75rZRoZygeoB0/K3TSb6ZDTAU+PeIGBERK7bbvB64PSKGAf8NnJ+W352WfwQ4keSb4ds7DriI5Bv6n5TUmJZ/JiKOBRqBqySVeyRlM8Bto9I5QWXbnpKaSYZzeR24Ky1fERFP72Cf1yKiOZ1fANRK2gcYGBEPAETExojYkGffuRGxPiLeJRnA8uS0/CpJLwBPkwz4WF/0KzMrjttGL7D7rjexMno3IhpyC5Kudf68k3025cxvBfbswPG2/0AyJI0iGX35hIjYIGkeydhgZuXkttEL+AqqF4iIPwGtks4FkLTHDu56Ok3SAWnf/LnAr0mG9n87bYBHAB/ttsDNupjbRrY5QfUeF5N0RywCfgMclGeb+SQjKzcDP4uIJuBhYPd0vxtJujLMKonbRkb5NnMDkjuVSG6LvbLcsZhlidtG+fgKyszMMslXUGZmlkm+gjIzs0xygjIzs0xygjIzs0xygjIzs0xygjIzs0z6/9xZU5Bl6cnFAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x216 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import seaborn as sns\n",
    "\n",
    "bins = np.linspace(df.Principal.min(), df.Principal.max(), 10)\n",
    "g = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\n",
    "g.map(plt.hist, 'Principal', bins=bins, ec=\"k\")\n",
    "\n",
    "g.axes[-1].legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAGdNJREFUeJzt3XuUVOW55/HvT2htFC8RW2ltsVuDICppsI9X9CBEhoM3PBriiWNwDRNG440VHY3JLNd4nGVCdHnJeIlMcOFKFDHkCB6yokGEE0niBbQFAaNGG20jclGjjEIEn/mjdve02tDV3bu6dlX/PmvtVbUv9e5nd9XbT+13v/VuRQRmZmZZs0uxAzAzM2uPE5SZmWWSE5SZmWWSE5SZmWWSE5SZmWWSE5SZmWWSE1RKJB0g6UFJr0taLulPks5JqezRkhakUVZPkLREUkOx47DiK6d6IalK0jOSXpB0cgH3s7lQZZcaJ6gUSBIwD/h9RBwaEccA5wM1RYqnbzH2a9ZWGdaLscDLETEiIp5KIybbOSeodIwB/h4RP2tZEBFrI+J/A0jqI+lmSc9JWiHpvyXLRydnG3MlvSzpgaRSI2l8smwp8M8t5UraQ9J9kp5NvsmdnSy/SNKvJP078LvuHIykWZLukbQ4+eY7OtnnGkmz2mx3j6RlklZJumEHZY1LvjU/n8TXvzuxWUkpm3ohqR74CTBBUqOkfjv6bEtqknRTsm6ZpJGSHpf0F0kXJ9v0l7Qoee3Klnjb2e9/b/P3abeOlbWI8NTNCbgCuG0n66cC/yN5vhuwDKgDRgN/I/eNchfgT8AooBJ4CxgMCHgYWJC8/ibgPyfP9wFeAfYALgKagX13EMNTQGM709fb2XYW8FCy77OBD4GjkxiXA/XJdvsmj32AJcDwZH4J0ADsB/we2CNZfi1wfbHfL089M5VhvbgIuDN5vsPPNtAEXJI8vw1YAewJVAHvJsv7Anu1Kes1QMn85uRxHDAjOdZdgAXAKcV+X3tyclNQAUi6i1yF+ntE/AO5D9pwSeclm+xNrpL9HXg2IpqT1zUCtcBm4I2IeDVZ/ktylZmkrLMkXZ3MVwKDkucLI+K99mKKiM62mf97RISkleQq1cokllVJjI3AJElTyVW2amAYucrY4vhk2R+SL8C7kvtnY71QmdSLFh19th9NHlcC/SPiI+AjSVsl7QP8X+AmSacAnwEHAQcA69qUMS6ZXkjm+5P7+/y+izGXHCeodKwCzm2ZiYhLJe1H7hsh5L4BXR4Rj7d9kaTRwNY2i7bT8Xsi4NyI+PMXyjqO3Ie+/RdJT5H7FvdFV0fEE+0sb4nrsy/E+BnQV1IdcDXwDxHxftL0V9lOrAsj4l92cjxWvsqxXrTd384+2zutP8AF5M6ojomITyU10X79+VFE3LuTOMqar0Gl40mgUtIlbZbt3ub548AlkioAJB0uaY+dlPcyUCvpsGS+bSV4HLi8TZv8iHwCjIiTI6K+nWlnlXBn9iJX8f8m6QDgn9rZ5mngJElfTWLdQ9LhXdyflZ5yrhfd/WzvDaxPktOpwCHtbPM48F/aXNs6SNL+ndhHyXOCSkHkGownAv8o6Q1JzwL3k2uXBvg5sBp4XtJLwL3s5BthRGwh13Txm+Ri8No2q28EKoAVSXPbjWkfTz4i4kVyTQ+rgPuAP7SzzQZy7fazJa0g1wQytAfDtCIq53qRwmf7AaBB0jJyZ1Mvt7OP3wEPAn9Kmtrn0v7ZXtlquShnZmaWKT6DMjOzTHKCMjOzTHKCMjOzTHKCMjOzTOrRBDV+/PgAPHkq56nbXE889YIpLz2aoDZu3NiTuzMrSa4nZjlu4jMzs0xygjIzs0xygjIzs0zyYLFmVnY+/fRTmpub2bJlS7FD6dUqKyupqamhoqKiS693gjKzstPc3Myee+5JbW0tyfix1sMigk2bNtHc3ExdXV2XynATn5mVnS1btjBgwAAnpyKSxIABA7p1FusE1YMOqa5GUirTIdXVxT4cs0xzciq+7r4HbuLrQW+uW0fzgTWplFXz1+ZUyjEzyyqfQZlZ2Uuz9SLfFow+ffpQX1/PUUcdxTe+8Q0+/vjj1nXz5s1DEi+//P9vA9XU1MRRRx0FwJIlS9h7770ZMWIEQ4YM4ZRTTmHBggWfK3/GjBkMHTqUoUOHcuyxx7J06dLWdaNHj2bIkCHU19dTX1/P3LlzPxdTy9TU1NSdP2vB+QzKzMpemq0XkF8LRr9+/WhsbATgggsu4Gc/+xnf+973AJg9ezajRo1i9uzZ3HDDDe2+/uSTT25NSo2NjUycOJF+/foxduxYFixYwL333svSpUvZb7/9eP7555k4cSLPPvssAwcOBOCBBx6goaFhhzGVAp9BmZkV2Mknn8xrr70GwObNm1m6dCkzZ87koYceyuv19fX1XH/99dx5550ATJ8+nZtvvpn99tsPgJEjRzJ58mTuuuuuwhxAkThBmZkV0LZt2/jtb3/L0UcfDcD8+fMZP348hx9+OAMGDGD58uV5lTNy5MjWJsFVq1ZxzDHHfG59Q0MDq1atap2/4IILWpvyNm3aBMAnn3zSuuycc85J4/AKyk18ZmYF0JIMIHcGNWXKFCDXvHfllVcCcP755zN79uwvJZv2ROQ9CDhQHk18eSUoSU3AR8B2YFtENEjaF5gD1AJNwKSIeL8wYZqZlZb2ksF7773Hk08+ycqVK5HE9u3bkcTNN9/cYXkvvPACRxxxBADDhg1j+fLljBkzpnX98uXLOfLII9M9iCLrTBPfqRFRHxEtKfn7wKKIGAwsSubNzGwH5s6dy4UXXsjatWtpamrirbfeoq6ujqeeemqnr1uxYgU33ngjl156KQDXXHMN1157bWvTXWNjI7NmzeK73/1uwY+hJ3Wnie9sYHTy/H5gCXBtN+MxM0vdoIEDU/3t4KCkp1xnzZ49m2uv/fy/yXPPPbfd5U899RQjRozg448/Zv/99+enP/0pY8eOBeCss87i7bff5sQTT0QSe+65J7/85S+pLrMf8Cufdk1JbwDvk7sT4r0RMUPSBxGxT7JewPst81947VRgKsCgQYOOWbt2bZrxlxRJqf5Qt7Nt0tYjuvTTedeTdK1Zs6a1OcyKawfvRV71JN8mvlERMRL4J+BSSae0XRm5/5Tt/reMiBkR0RARDVVVVXnuzqx3cT0x+7K8ElREvJ08rgceAY4F3pVUDZA8ri9UkGZm1vt0mKAk7SFpz5bnwDjgJeBRYHKy2WRgfqGCNDOz3iefThIHAI8ko9L2BR6MiMckPQc8LGkKsBaYVLgwzcyst+kwQUXE68DX2lm+CRhbiKDMzMw81JGZmWWSE5SZlb0DawaleruNA2sG5bXfdevWcf7553PYYYcxbNgwJkyYwCuvvMKqVasYM2YMQ4YMYfDgwdx4442tPxuZNWsWl1122ZfKqq2tZePGjZ9bNmvWLKqqqj53C43Vq1cD8MorrzBhwgQGDx7MEUccwaRJk5gzZ07rdv3792+9Jce3v/1tlixZwhlnnNFa9rx58xg+fDhHHHEERx99NPPmzWtdd9FFF3HQQQexdetWADZu3EhtbW2n3pN8eCy+PBxSXc2b69YVOwwz66J33n6L465/LLXynvnX8R1uExGcc845TJ48uXXU8sbGRt59910uuugi7rnnHsaNG8fHH3/Mueeey9133906UkRnfPOb32wd5bzFli1bOP3007n11ls588wzAVi8eDFVVVWtwy+NHj2aW265pXW8viVLlrS+/sUXX+Tqq69m4cKF1NXV8cYbb3Daaadx6KGHMnz4cCB3b6n77ruPSy65pNMx58sJKg9p3UvGd8E16z0WL15MRUUFF198ceuy+vp6Zs6cyUknncS4ceMA2H333bnzzjsZPXp0lxJUex588EFOOOGE1uQEcOqpp+b9+ltuuYUf/OAH1NXVAVBXV8d1113HzTffzC9+8QsApk2bxm233cZ3vvOdVGJuj5v4zMwK4KWXXmp3lPL2bpVx2GGHsXnzZj788MNO76dts119fT2ffPLJDvedr3xu5zFo0CBGjRrVmrAKwWdQZmYlrL0mvp5y3XXXcfbZZ3P66acXpHyfQZmZFcCRRx7Z7s0IW26V0dbrr79O//792WuvvQq673y1F2N7t/MYPHgw9fX1PPzww13e1844QZmZFcCYMWPYunUrM2bMaF323HPPMXjwYJYuXcoTTzwB5G5seMUVV3DNNdektu9vfetb/PGPf+Q3v/lN67LHHnuMlStX5vX6q6++mh/96Ec0NTUB0NTUxE033cRVV131pW1/+MMfcsstt6QS9xe5ic/Myl71QQfn1fOuM+V1RBKPPPII06ZNY/r06VRWVlJbW8vtt9/O/Pnzufzyy7n00kvZvn07F1544ee6ls+aNetz3bqffvppAIYPH84uu+TOKyZNmsTw4cOZM2cOS5cubd327rvv5sQTT2TBggVMmzaNadOmUVFRwfDhw7njjjvyOr76+nqmT5/OmWeeyaeffkpFRQU/+clPWu8Q3NaRRx7JyJEjef755/MquzPyut1GWhoaGmLZsmU9tr+0pHWbjJq/Nvt2G+WvS7fbaKtU60mW+HYb2dETt9swMzPrUU5QZmaWSU5QZlaW3ARefN19D5ygzKzsVFZWsmnTJiepIooINm3aRGVlZZfLcC8+Mys7NTU1NDc3s2HDhmKH0qtVVlZSU9P1jmFOUCVqN3K9C9MwaOBA1r7zTiplmWVBRUVF6zhyVrqcoErUVki1y7qZWdbkfQ1KUh9JL0hakMzXSXpG0muS5kjatXBhmplZb9OZThJXAmvazE8HbouIrwLvA1PSDMzMzHq3vBKUpBrgdODnybyAMcDcZJP7gYmFCNDMzHqnfM+gbgeuAT5L5gcAH0TEtmS+GTiovRdKmippmaRl7lFj1j7XE7Mv6zBBSToDWB8RXRq7PSJmRERDRDRUVVV1pQizsud6YvZl+fTiOwk4S9IEoBLYC7gD2EdS3+QsqgZ4u3BhmplZb9PhGVREXBcRNRFRC5wPPBkRFwCLgfOSzSYD8wsWpZmZ9TrdGeroWuB7kl4jd01qZjohmZmZdfKHuhGxBFiSPH8dODb9kMzMzDxYrJmZZZQTlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZVKHCUpSpaRnJb0oaZWkG5LldZKekfSapDmSdi18uGZm1lvkcwa1FRgTEV8D6oHxko4HpgO3RcRXgfeBKYUL08zMepsOE1TkbE5mK5IpgDHA3GT5/cDEgkRoZma9Ul7XoCT1kdQIrAcWAn8BPoiIbckmzcBBO3jtVEnLJC3bsGFDGjGblR3XE7MvyytBRcT2iKgHaoBjgaH57iAiZkREQ0Q0VFVVdTFMs/LmemL2ZZ3qxRcRHwCLgROAfST1TVbVAG+nHJuZmfVi+fTiq5K0T/K8H3AasIZcojov2WwyML9QQZqZWe/Tt+NNqAbul9SHXEJ7OCIWSFoNPCTpfwEvADMLGKeZmfUyHSaoiFgBjGhn+evkrkeZmZmlziNJmJlZJjlBmZlZJjlBmZlZJjlBmZlZJpVtgjqkuhpJqUxmZtbz8ulmXpLeXLeO5gNrUimr5q/NqZRjZmb5K9szKDMzK21OUGZmlklOUGZmlklOUGZmlklOUGZmlklOUGZmlklOUGZmlklOUGZmlklOUGZmlklOUGZmlklOUGZmlkkdJihJB0taLGm1pFWSrkyW7ytpoaRXk8evFD5cMzPrLfI5g9oGXBURw4DjgUslDQO+DyyKiMHAomTezMwsFR0mqIh4JyKeT55/BKwBDgLOBu5PNrsfmFioIM3MrPfp1DUoSbXACOAZ4ICIeCdZtQ44YAevmSppmaRlGzZs6EaoZuXL9cTsy/JOUJL6A78GpkXEh23XRUQA0d7rImJGRDRERENVVVW3gjUrV64nZl+WV4KSVEEuOT0QEf+WLH5XUnWyvhpYX5gQzcysN8qnF5+AmcCaiLi1zapHgcnJ88nA/PTDs56wG3R42/t8pkOqq4t9KGZWRvK55ftJwIXASkmNybIfAD8GHpY0BVgLTCpMiFZoW4HmA2u6XU7NX5u7H4yZWaLDBBURSwHtYPXYdMPJJvWpSOWfr/rumto/cfWpSKUcM7OsyucMqteL7Z9y3PWPdbucZ/51fCrltJRlZlbOPNSRmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllUtmOJJHW8ERmZlYcZZug0hqeCDyskJlZMbiJz8zMMskJyszMMskJyszMMqlsr0GVuzQ7gfjeUpY1h1RX8+a6dd0up98uffjks+0pRASDBg5k7TvvpFKW5ccJqkS5E4iVszfXrUvtLs9plNNSlvWsDpv4JN0nab2kl9os21fSQkmvJo9fKWyYZmbW2+RzDWoW8MWv2N8HFkXEYGBRMm+93G6ApFSmQ6qri304ZlZkHTbxRcTvJdV+YfHZwOjk+f3AEuDaFOOyErQV3JxiZqnpai++AyKi5WrhOuCAHW0oaaqkZZKWbdiwoYu7Mytv5VBPDqmuTu0M2gxS6CQRESEpdrJ+BjADoKGhYYfbmfVm5VBP0urYAD6DtpyunkG9K6kaIHlcn15IZmZmXU9QjwKTk+eTgfnphGNmZpaTTzfz2cCfgCGSmiVNAX4MnCbpVeDrybyZmVlq8unF9y87WDU25VjMzMxaZWosPvcCMjOzFpka6si9gMzMrEWmEpQVR1oDz3rQWTNLkxOUpTbwrAedNbM0ZeoalJmZWQsnKDMzyyQnKDMzyyQnKDMzyyQnKMsk31uqZ/i3h5Zl7sVnmeR7S/UM//bQsswJylKT1u+pWsoys97NCcpSk9bvqcC/qTIzX4MyM7OM8hmUZVKazYW79KlI5SL+oIEDWfvOOylEVJ5SbeLtu6uH38rDIdXVvLluXSplZfHz7QRlmZR2c2EaHQHcCWDn0n7PPPxWx8q9k4ub+MzMLJMydQaVZhOBmZmVtkwlKPcCMzOzFt1KUJLGA3cAfYCfR8SPU4nKLEXleL+rNC+OW37S6mwDsEvfCj7b9mkqZZWzLicoSX2Au4DTgGbgOUmPRsTqtIIzS0M53u8qrYvjblLP32fuuNPjutNJ4ljgtYh4PSL+DjwEnJ1OWGZm1tspIrr2Quk8YHxE/Ndk/kLguIi47AvbTQWmJrNDgD93PdxW+wEbUygnC3ws2dTVY9kYEZ0+1SpQPQG/J1nV248lr3pS8E4SETEDmJFmmZKWRURDmmUWi48lm3r6WApRT8DvSVb5WPLTnSa+t4GD28zXJMvMzMy6rTsJ6jlgsKQ6SbsC5wOPphOWmZn1dl1u4ouIbZIuAx4n1838vohYlVpkO5d6U0gR+ViyqVyOpVyOA3wsWVWwY+lyJwkzM7NC8lh8ZmaWSU5QZmaWSZlPUJIOlrRY0mpJqyRdmSzfV9JCSa8mj18pdqwdkVQp6VlJLybHckOyvE7SM5JekzQn6XSSeZL6SHpB0oJkviSPA0BSk6SVkholLUuWlcxnzPUk28qlrvR0Pcl8ggK2AVdFxDDgeOBSScOA7wOLImIwsCiZz7qtwJiI+BpQD4yXdDwwHbgtIr4KvA9MKWKMnXElsKbNfKkeR4tTI6K+zW86Sukz5nqSbeVUV3qunkRESU3AfHLj//0ZqE6WVQN/LnZsnTyO3YHngePI/Qq7b7L8BODxYseXR/w1yYdxDLAAUCkeR5vjaQL2+8Kykv2MuZ5kZyqnutLT9aQUzqBaSaoFRgDPAAdERMv9idcBBxQprE5JTvUbgfXAQuAvwAcRsS3ZpBk4qFjxdcLtwDXAZ8n8AErzOFoE8DtJy5Nhh6B0P2O1uJ5kSTnVlR6tJ5m6H9TOSOoP/BqYFhEfth32PiJCUkn0l4+I7UC9pH2AR4ChRQ6p0ySdAayPiOWSRhc7npSMioi3Je0PLJT0ctuVpfIZcz3JljKsKz1aT0oiQUmqIFfpHoiIf0sWvyupOiLekVRN7ptWyYiIDyQtJnd6v4+kvsk3qlIYMuok4CxJE4BKYC9y9wUrteNoFRFvJ4/rJT1CbrT+kvqMuZ5kUlnVlZ6uJ5lv4lPuK+BMYE1E3Npm1aPA5OT5ZHJt7pkmqSr5RoikfuSuEawBFgPnJZtl/lgi4rqIqImIWnJDXD0ZERdQYsfRQtIekvZseQ6MA16ihD5jrifZVE51pSj1pNgX3fK4KDeKXLvnCqAxmSaQa8ddBLwKPAHsW+xY8ziW4cALybG8BFyfLD8UeBZ4DfgVsFuxY+3EMY0GFpTycSRxv5hMq4AfJstL5jPmepL9qdTrSjHqiYc6MjOzTMp8E5+ZmfVOTlBmZpZJTlBmZpZJTlBmZpZJTlBmZpZJTlBmZpZJTlBmZpZJTlAlTtK8ZODGVS2DN0qaIukVSUsk/R9JdybLqyT9WtJzyXRScaM36xmuJ6XJP9QtcZL2jYj3kiFhngP+E/AHYCTwEfAk8GJEXCbpQeDuiFgqaRC5If6PKFrwZj3E9aQ0lcRgsbZTV0g6J3l+MHAh8B8R8R6ApF8Bhyfrvw4MazPC9V6S+kfE5p4M2KwIXE9KkBNUCUuG7/86cEJEfCxpCfAysKNve7sAx0fElp6J0Kz4XE9Kl69Blba9gfeTSjeU3K2+9wD+UdJXJPUFzm2z/e+Ay1tmJNX3aLRmxeF6UqKcoErbY0BfSSuAG4Gnyd1X5iZyd1N9AlgN/C3Z/gqgQdIKSauBi3s+ZLMe53pSotxJogy1tJcn3wwfAe6LiEeKHZdZlrieZJ/PoMrT/5TUSO5eOm8A84ocj1kWuZ5knM+gzMwsk3wGZWZmmeQEZWZmmeQEZWZmmeQEZWZmmeQEZWZmmfT/ADDLQneOBIAaAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x216 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "bins = np.linspace(df.age.min(), df.age.max(), 10)\n",
    "g = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\n",
    "g.map(plt.hist, 'age', bins=bins, ec=\"k\")\n",
    "\n",
    "g.axes[-1].legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Pre-processing:  Feature selection/extraction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Lets look at the day of the week people get the loan "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAGb5JREFUeJzt3XuYFPWd7/H3R5gsKngDoiMTYKIIgrIDTjSKeBASHoIXcDWGxBDImvVovHEMq9HkmDWeYySyXhKvPGpwI6KGKGTJRoMKEeKVywgihriKOoZRIDHGIAj6PX90zZwBB6aHqaZrej6v5+mHruqqX31rZr58u35d/fspIjAzM8uaPYodgJmZWVNcoMzMLJNcoMzMLJNcoMzMLJNcoMzMLJNcoMzMLJNcoFIi6UBJ90l6VdISSU9LOi2ltodJmptGW7uDpAWSqosdhxVfKeWFpO6SnpW0TNLQAh7n/UK13da4QKVAkoDZwJMR8dmIOAoYB1QUKZ6OxTiuWWMlmBcjgJcjYlBELEwjJts5F6h0DAc+jIjb61dExOsR8VMASR0kXSfpeUnLJf3PZP2w5GpjlqSXJc1IkhpJo5J1i4B/qm9X0t6S7pb0XPJObkyyfqKkX0j6T+C3rTkZSdMl3SZpfvLOd1hyzFWSpjfa7jZJiyWtlHTVDtoambxrXprE17k1sVmbUjJ5IakK+DEwWlKNpD139LctaY2ka5LXFksaLOlRSf8t6dxkm86SHk/2XVEfbxPH/ddGP58mc6ykRYQfrXwAFwE37OT1c4DvJ8//AVgMVALDgL+Se0e5B/A0cDzQCXgT6AMIeBCYm+x/DfD15Pl+wGpgb2AiUAscsIMYFgI1TTy+0MS204H7k2OPAd4DjkxiXAJUJdsdkPzbAVgADEyWFwDVQDfgSWDvZP1lwJXF/n35sXseJZgXE4Gbk+c7/NsG1gDnJc9vAJYDXYDuwNvJ+o7APo3aegVQsvx+8u9IYFpyrnsAc4ETiv173Z0PdwUVgKRbyCXUhxHxOXJ/aAMlnZFssi+5JPsQeC4iapP9aoDewPvAaxHxx2T9veSSmaStUyVNTpY7AT2T5/Mi4s9NxRQRLe0z/8+ICEkryCXViiSWlUmMNcCZks4hl2zlQH9yyVjv88m63ydvgD9F7j8ba4dKJC/qNfe3/avk3xVA54j4G/A3SZsl7Qf8HbhG0gnAx0AP4ECgrlEbI5PHsmS5M7mfz5O7GHOb4wKVjpXA6fULEXG+pG7k3hFC7h3QhRHxaOOdJA0DNjda9RHN/04EnB4Rf9iurWPI/dE3vZO0kNy7uO1NjojHmlhfH9fH28X4MdBRUiUwGfhcRPwl6frr1ESs8yLiqzs5HytdpZgXjY+3s7/tneYPcBa5K6qjImKLpDU0nT8/iog7dhJHSfNnUOl4Augk6bxG6/Zq9PxR4DxJZQCSDpO0907aexnoLemQZLlxEjwKXNioT35QPgFGxNCIqGrisbMk3Jl9yCX+XyUdCHypiW2eAYZIOjSJdW9Jh+3i8aztKeW8aO3f9r7AO0lxOhHo1cQ2jwL/3OizrR6SPt2CY7R5LlApiFyH8Vjgf0h6TdJzwD3k+qUB7gReApZKehG4g528I4yITeS6Ln6dfBj8eqOXrwbKgOVJd9vVaZ9PPiLiBXJdDyuBu4HfN7HNOnL99jMlLSfXBdJvN4ZpRVTKeZHC3/YMoFrSYnJXUy83cYzfAvcBTydd7bNo+mqvZNV/KGdmZpYpvoIyM7NMcoEyM7NMcoEyM7NMcoEyM7NM2q0FatSoUQH44UcpP1rNeeJHO3jkZbcWqPXr1+/Ow5m1Sc4Tsxx38ZmZWSa5QJmZWSa5QJmZWSZ5sFgzKzlbtmyhtraWTZs2FTuUdq1Tp05UVFRQVla2S/u7QJlZyamtraVLly707t2bZPxY280igg0bNlBbW0tlZeUuteEuPjMrOZs2baJr164uTkUkia5du7bqKtYFykper/JyJLX60au8vNinYi3g4lR8rf0duIvPSt4bdXXUHlzR6nYq/lSbQjRmli9fQZlZyUvrKrolV9MdOnSgqqqKI444gi9/+cts3Lix4bXZs2cjiZdf/v/TQK1Zs4YjjjgCgAULFrDvvvsyaNAg+vbtywknnMDcuXO3aX/atGn069ePfv36cfTRR7No0aKG14YNG0bfvn2pqqqiqqqKWbNmbRNT/WPNmjWt+bEWXF5XUJL+F/AtckNUrAC+CZQD9wNdgSXA+Ij4sEBxmpntsrSuouvlczW95557UlNTA8BZZ53F7bffziWXXALAzJkzOf7445k5cyZXXXVVk/sPHTq0oSjV1NQwduxY9txzT0aMGMHcuXO54447WLRoEd26dWPp0qWMHTuW5557joMOOgiAGTNmUF1dvcOY2oJmr6Ak9QAuAqoj4gigAzAOmALcEBGHAn8Bzi5koGZmbdXQoUN55ZVXAHj//fdZtGgRd911F/fff39e+1dVVXHllVdy8803AzBlyhSuu+46unXrBsDgwYOZMGECt9xyS2FOoEjy7eLrCOwpqSOwF7AWGE5uCmLITeM8Nv3wzMzatq1bt/Kb3/yGI488EoA5c+YwatQoDjvsMLp27cqSJUvyamfw4MENXYIrV67kqKOO2ub16upqVq5c2bB81llnNXTlbdiwAYAPPvigYd1pp52WxukVVLNdfBHxlqSpwBvAB8BvyXXpvRsRW5PNaoEeTe0v6RzgHICePXumEbNZyXGelJ76YgC5K6izz851Ms2cOZOLL74YgHHjxjFz5sxPFJumROQ9CDhQGl18zRYoSfsDY4BK4F3gF8CofA8QEdOAaQDV1dUt+wmbtRPOk9LTVDH485//zBNPPMGKFSuQxEcffYQkrrvuumbbW7ZsGYcffjgA/fv3Z8mSJQwfPrzh9SVLljBgwIB0T6LI8uni+wLwWkSsi4gtwEPAEGC/pMsPoAJ4q0AxmpmVhFmzZjF+/Hhef/111qxZw5tvvkllZSULFy7c6X7Lly/n6quv5vzzzwfg0ksv5bLLLmvouqupqWH69Ol8+9vfLvg57E753MX3BvB5SXuR6+IbASwG5gNnkLuTbwIwp1BBmpm1Rs+DDkr1e2w9kzvlWmrmzJlcdtll26w7/fTTm1y/cOFCBg0axMaNG/n0pz/NT37yE0aMGAHAqaeeyltvvcVxxx2HJLp06cK9995LeYl9mVz59GtKugr4CrAVWEbulvMe5IrTAcm6r0fE5p21U11dHYsXL25tzGYtIim1L+rmkS+tHr7AedJ6q1ataugOs+Lawe8irzzJ63tQEfED4AfbrX4VODqf/c3MzFrKI0mYmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZWck7uKJnqtNtHFyR33BUdXV1jBs3jkMOOYT+/fszevRoVq9ezcqVKxk+fDh9+/alT58+XH311Q1fYZg+fToXXHDBJ9rq3bs369ev32bd9OnT6d69+zZTaLz00ksArF69mtGjR9OnTx8OP/xwzjzzTB544IGG7Tp37twwJcc3vvENFixYwMknn9zQ9uzZsxk4cCCHH344Rx55JLNnz254beLEifTo0YPNm3PfLFq/fj29e/du0e8kH56w0MxK3tq33uSYKx9Jrb1nf9j8aG8RwWmnncaECRMaRi2vqanh7bffZuLEidx2222MHDmSjRs3cvrpp3Prrbc2jBTREl/5ylcaRjmvt2nTJk466SSuv/56TjnlFADmz59P9+7dG4ZfGjZsGFOnTm0Yr2/BggUN+7/wwgtMnjyZefPmUVlZyWuvvcYXv/hFPvvZzzJw4EAgN7fU3XffzXnnndfimPPlKygzswKYP38+ZWVlnHvuuQ3rqqqqWL16NUOGDGHkyJEA7LXXXtx8881ce+21qR37vvvu49hjj20oTgAnnnhiw4SIzZk6dSpXXHEFlZWVAFRWVnL55ZdvM2bgpEmTuOGGG9i6deuOmmk1FygzswJ48cUXmxylvKmpMg455BDef/993nvvvRYfp3G3XVVVFR988MEOj52vfKbz6NmzJ8cffzw///nPd/k4zXEXn5lZG9ZUF9/ucvnllzNmzBhOOumkgrTvKygzswIYMGBAk5MR1k+V0dirr75K586d2WeffQp67Hw1FWNT03n06dOHqqoqHnzwwV0+1s64QJmZFcDw4cPZvHkz06ZNa1j3/PPP06dPHxYtWsRjjz0G5CY2vOiii7j00ktTO/bXvvY1nnrqKX796183rHvkkUdYsWJFXvtPnjyZH/3oR6xZswaANWvWcM011/Cd73znE9t+73vfY+rUqanEvT138ZlZySvv8Zm87rxrSXvNkcTDDz/MpEmTmDJlCp06daJ3797ceOONzJkzhwsvvJDzzz+fjz76iPHjx29za/n06dO3ua37mWeeAWDgwIHssUfuuuLMM89k4MCBPPDAAyxatKhh21tvvZXjjjuOuXPnMmnSJCZNmkRZWRkDBw7kpptuyuv8qqqqmDJlCqeccgpbtmyhrKyMH//4xw0zBDc2YMAABg8ezNKlS/NquyXymm4jLZ5GwIrB0220P55uIztaM92Gu/jMzCyTMlWgepWXp/ZN714lNrOkmVl7k6nPoN6oq0ulKwZIdXpnM2t7IgKp1T2u1gqt/QgpU1dQZmZp6NSpExs2bGj1f5C26yKCDRs20KlTp11uI1NXUGZmaaioqKC2tpZ169YVO5R2rVOnTlRU7HqvmAuUmZWcsrKyhnHkrO1yF5+ZmWWSC5SZmWWSC5SZmWWSC5SZmWWSC5SZmWVSXgVK0n6SZkl6WdIqScdKOkDSPEl/TP7dv9DBmplZ+5HvFdRNwCMR0Q/4R2AV8F3g8YjoAzyeLJuZmaWi2QIlaV/gBOAugIj4MCLeBcYA9ySb3QOMLVSQZmbW/uRzBVUJrAN+JmmZpDsl7Q0cGBFrk23qgAOb2lnSOZIWS1rsb3WbNc15YvZJ+RSojsBg4LaIGAT8ne268yI34FWTg15FxLSIqI6I6u7du7c2XrOS5Dwx+6R8ClQtUBsRzybLs8gVrLcllQMk/75TmBDNzKw9arZARUQd8KakvsmqEcBLwK+ACcm6CcCcgkRoZmbtUr6DxV4IzJD0KeBV4JvkituDks4GXgfOLEyIZq2jDmWpzA+mDmUpRGNm+cqrQEVEDVDdxEsj0g3HLH3x0RaOufKRVrfz7A9HpRCNmeXLI0mYmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkm5V2gJHWQtEzS3GS5UtKzkl6R9ICkTxUuTDMza29acgV1MbCq0fIU4IaIOBT4C3B2moGZmVn7lleBklQBnATcmSwLGA7MSja5BxhbiADNzKx9yvcK6kbgUuDjZLkr8G5EbE2Wa4EeTe0o6RxJiyUtXrduXauCNStVzhOzT2q2QEk6GXgnIpbsygEiYlpEVEdEdffu3XelCbOS5zwx+6SOeWwzBDhV0migE7APcBOwn6SOyVVUBfBW4cI0M7P2ptkrqIi4PCIqIqI3MA54IiLOAuYDZySbTQDmFCxKMzNrd1rzPajLgEskvULuM6m70gnJzMwsvy6+BhGxAFiQPH8VODr9kMzMzDyShJmZZZQLlJmZZZILlJmZZZILlJmZZZILlJmZZZILlJmZZZILlJmZZZILlJmZZZILlJmZZZILlJmZZZILlJmZZZILlJmZZZILlJmZZZILlJmZZZIL1G7Uq7wcSak8epWXF/t0zMwKqkXzQVnrvFFXR+3BFam0VfGn2lTaMTPLKl9BmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJjVboCR9RtJ8SS9JWinp4mT9AZLmSfpj8u/+hQ/XzMzai3yuoLYC34mI/sDngfMl9Qe+CzweEX2Ax5NlMzOzVDRboCJibUQsTZ7/DVgF9ADGAPckm90DjC1UkGZm1v606DMoSb2BQcCzwIERsTZ5qQ44cAf7nCNpsaTF69ata0WoZqXLeWL2SXkXKEmdgV8CkyLivcavRUQA0dR+ETEtIqojorp79+6tCtasVDlPzD4prwIlqYxccZoREQ8lq9+WVJ68Xg68U5gQzcysPcrnLj4BdwGrIuL6Ri/9CpiQPJ8AzEk/PDMza6/ymbBwCDAeWCGpJll3BXAt8KCks4HXgTMLE6KZmbVHzRaoiFgEaAcvj0g3HDMzK4Ze5eW8UVeXSls9DzqI19eubX7DZnjKdzMz4426OmoPrkilrYo/1abSjoc6skzqVV6OpFQepSitn0+v8vJin4rZDvkKyjIpi+/msiStn08p/mysdPgKyszMMqlkr6D+AVLr3knrAz/LnzqU+d29WTtXsgVqM7iLqA2Lj7ZwzJWPpNLWsz8clUo7ZrZ7uYvPzMwyyQXKzMwyyQXKzMwyyQXKzMwyyQXKzMwyyQXKzMwyyQXKzMwyyQXKzMwyyQXKzMwyyQXKzMwyqWSHOjIzs/ylOf6lOpSl0o4LlJmZZXL8S3fxmbVj9aP+e/JDyyJfQZm1Yx7137LMV1BmZpZJLlCWmoMreqbWXWRm5i4+S83at97M3IesZtZ2ZapAZfE2RzPb/XqVl/NGXV2r2+l50EG8vnZtChFZMWSqQGXxNsesqr/7Kg1OYsuaN+rqUrl5wzdutG2tKlCSRgE3AR2AOyPi2lSismb57iszK3W7fJOEpA7ALcCXgP7AVyX1TyswM7PWyur3vHqVl6cS014dOpb0jUmtuYI6GnglIl4FkHQ/MAZ4KY3AzMxaK6s9DWl2YWbx/NKiiNi1HaUzgFER8a1keTxwTERcsN125wDnJIt9gT/spNluwPpdCqht8Pm1bfmc3/qIaPEHoC3Mk3xjact8fm1bc+eXV54U/CaJiJgGTMtnW0mLI6K6wCEVjc+vbSvk+bUkTwodSxb4/Nq2tM6vNV/UfQv4TKPlimSdmZlZq7WmQD0P9JFUKelTwDjgV+mEZWZm7d0ud/FFxFZJFwCPkrvN/O6IWNnKePLu4mijfH5tW5bOL0uxFILPr21L5fx2+SYJMzOzQvJgsWZmlkkuUGZmlkmZKVCSRkn6g6RXJH232PGkSdJnJM2X9JKklZIuLnZMaZPUQdIySXOLHUshSNpP0ixJL0taJenYIsXhPGnjSjlX0s6TTHwGlQybtBr4IlBL7g7Br0ZESYxKIakcKI+IpZK6AEuAsaVyfgCSLgGqgX0i4uRix5M2SfcACyPizuSu1b0i4t3dHIPzpASUcq6knSdZuYJqGDYpIj4E6odNKgkRsTYilibP/wasAnoUN6r0SKoATgLuLHYshSBpX+AE4C6AiPhwdxenhPOkjSvlXClEnmSlQPUA3my0XEuJ/WHWk9QbGAQ8W9xIUnUjcCnwcbEDKZBKYB3ws6Rr5k5JexchDudJ21fKuZJ6nmSlQLULkjoDvwQmRcR7xY4nDZJOBt6JiCXFjqWAOgKDgdsiYhDwd6CkPv/JklLME2gXuZJ6nmSlQJX8sEmSysgl3YyIeKjY8aRoCHCqpDXkupyGS7q3uCGlrhaojYj6d/OzyCXi7uY8adtKPVdSz5OsFKiSHjZJuclW7gJWRcT1xY4nTRFxeURURERvcr+3JyLi60UOK1URUQe8KalvsmoExZlWxnnShpV6rhQiTzIx5XuBhk3KkiHAeGCFpJpk3RUR8V9FjMla5kJgRlIYXgW+ubsDcJ5YG5BqnmTiNnMzM7PtZaWLz8zMbBsuUGZmlkkuUGZmlkkuUGZmlkkuUGZmlkkuUBkg6d8kTU6xvX6SapLhRg5Jq91G7S+QVJ12u2Y74zxpf1ygStNYYE5EDIqI/y52MGYZ5TzJOBeoIpH0vWRen8eAvsm6f5H0vKQXJP1S0l6Sukh6LRkCBkn71C9LqpL0jKTlkh6WtL+k0cAk4FvJ3Dr/KumiZN8bJD2RPB8uaUbyfKSkpyUtlfSLZCw0JB0l6XeSlkh6NJkOofE57CFpuqT/s9t+cNauOE/aNxeoIpB0FLmhTgYB/wR8LnnpoYj4XET8I7mpBs5Oph1YQG6IfpL9HoqILcB/AJdFxEBgBfCD5Fv3twM3RMSJwEJgaLJvNdA5SeKhwJOSugHfB74QEYOBxcAlyTY/Bc6IiKOAu4H/2+g0OgIzgD9GxPdT/PGYAc4Ty8hQR+3QUODhiNgIIKl+PLUjkndZ+wGdyQ1pA7m5Yy4FZpMbOuRflJt7Zb+I+F2yzT3AL5o41hLgKEn7AJuBpeQScChwEfB5oD/w+9xQaHwKeJrcu9UjgHnJ+g7A2kbt3gE8GBGNk9EsTc6Tds4FKlumk5tB9AVJE4FhABHxe0m9JQ0DOkTEi0niNSsitkh6DZgIPAUsB04EDiX37vMQYF5EfLXxfpKOBFZGxI6mbH4KOFHSv0fEphadpVnrTMd50i64i684ngTGStpTuamtT0nWdwHWJt0GZ223z38A9wE/A4iIvwJ/kVTfLTEe+B1NWwhMTo67EDgXWBa5gRifAYZIOhRA0t6SDgP+AHSXdGyyvkzSgEZt3gX8F/CgJL/RsUJwnrRzLlBFkExr/QBQQ27um4XJS/+b3Ayi84CXt9ttBrA/MLPRugnAdZKWA1XAD3dwyIVAOfB0RLwNbKo/ZkSsI/eucWbSztNAv2RK8TOAKZJeSGI9brvzuB5YBvxckv+WLFXOE/No5m2EpDOAMRExvtixmGWV86S0+JKzDZD0U+BLwOhix2KWVc6T0uMrKDMzyyT3h5qZWSa5QJmZWSa5QJmZWSa5QJmZWSa5QJmZWSb9P6FJufwFllRXAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x216 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df['dayofweek'] = df['effective_date'].dt.dayofweek\n",
    "bins = np.linspace(df.dayofweek.min(), df.dayofweek.max(), 10)\n",
    "g = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\n",
    "g.map(plt.hist, 'dayofweek', bins=bins, ec=\"k\")\n",
    "g.axes[-1].legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "We see that people who get the loan at the end of the week dont pay it off, so lets use Feature binarization to set a threshold values less then day 4 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "      <th>weekend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>male</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>female</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30     2016-09-08   \n",
       "1           2             2     PAIDOFF       1000     30     2016-09-08   \n",
       "2           3             3     PAIDOFF       1000     15     2016-09-08   \n",
       "3           4             4     PAIDOFF       1000     30     2016-09-09   \n",
       "4           6             6     PAIDOFF       1000     30     2016-09-09   \n",
       "\n",
       "    due_date  age             education  Gender  dayofweek  weekend  \n",
       "0 2016-10-07   45  High School or Below    male          3        0  \n",
       "1 2016-10-07   33              Bechalor  female          3        0  \n",
       "2 2016-09-22   27               college    male          3        0  \n",
       "3 2016-10-08   28               college  female          4        1  \n",
       "4 2016-10-08   29               college    male          4        1  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['weekend'] = df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Convert Categorical features to numerical values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets look at gender:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Gender  loan_status\n",
       "female  PAIDOFF        0.865385\n",
       "        COLLECTION     0.134615\n",
       "male    PAIDOFF        0.731293\n",
       "        COLLECTION     0.268707\n",
       "Name: loan_status, dtype: float64"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(['Gender'])['loan_status'].value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "86 % of female pay there loans while only 73 % of males pay there loan\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets convert male to 0 and female to 1:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "      <th>weekend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30     2016-09-08   \n",
       "1           2             2     PAIDOFF       1000     30     2016-09-08   \n",
       "2           3             3     PAIDOFF       1000     15     2016-09-08   \n",
       "3           4             4     PAIDOFF       1000     30     2016-09-09   \n",
       "4           6             6     PAIDOFF       1000     30     2016-09-09   \n",
       "\n",
       "    due_date  age             education  Gender  dayofweek  weekend  \n",
       "0 2016-10-07   45  High School or Below       0          3        0  \n",
       "1 2016-10-07   33              Bechalor       1          3        0  \n",
       "2 2016-09-22   27               college       0          3        0  \n",
       "3 2016-10-08   28               college       1          4        1  \n",
       "4 2016-10-08   29               college       0          4        1  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## One Hot Encoding  \n",
    "#### How about education?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "education             loan_status\n",
       "Bechalor              PAIDOFF        0.750000\n",
       "                      COLLECTION     0.250000\n",
       "High School or Below  PAIDOFF        0.741722\n",
       "                      COLLECTION     0.258278\n",
       "Master or Above       COLLECTION     0.500000\n",
       "                      PAIDOFF        0.500000\n",
       "college               PAIDOFF        0.765101\n",
       "                      COLLECTION     0.234899\n",
       "Name: loan_status, dtype: float64"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(['education'])['loan_status'].value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Feature befor One Hot Encoding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>education</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>High School or Below</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>Bechalor</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>college</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>college</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>college</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender             education\n",
       "0       1000     30   45       0  High School or Below\n",
       "1       1000     30   33       1              Bechalor\n",
       "2       1000     15   27       0               college\n",
       "3       1000     30   28       1               college\n",
       "4       1000     30   29       0               college"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[['Principal','terms','age','Gender','education']].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Use one hot encoding technique to convert categorical varables to binary variables and append them to the feature Data Frame "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>weekend</th>\n",
       "      <th>Bechalor</th>\n",
       "      <th>High School or Below</th>\n",
       "      <th>college</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n",
       "0       1000     30   45       0        0         0                     1   \n",
       "1       1000     30   33       1        0         1                     0   \n",
       "2       1000     15   27       0        0         0                     0   \n",
       "3       1000     30   28       1        1         0                     0   \n",
       "4       1000     30   29       0        1         0                     0   \n",
       "\n",
       "   college  \n",
       "0        0  \n",
       "1        0  \n",
       "2        1  \n",
       "3        1  \n",
       "4        1  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Feature = df[['Principal','terms','age','Gender','weekend']]\n",
    "Feature = pd.concat([Feature,pd.get_dummies(df['education'])], axis=1)\n",
    "Feature.drop(['Master or Above'], axis = 1,inplace=True)\n",
    "Feature.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Feature selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets defind feature sets, X:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>weekend</th>\n",
       "      <th>Bechalor</th>\n",
       "      <th>High School or Below</th>\n",
       "      <th>college</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n",
       "0       1000     30   45       0        0         0                     1   \n",
       "1       1000     30   33       1        0         1                     0   \n",
       "2       1000     15   27       0        0         0                     0   \n",
       "3       1000     30   28       1        1         0                     0   \n",
       "4       1000     30   29       0        1         0                     0   \n",
       "\n",
       "   college  \n",
       "0        0  \n",
       "1        0  \n",
       "2        1  \n",
       "3        1  \n",
       "4        1  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = Feature\n",
    "X[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "What are our lables?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF'],\n",
       "      dtype=object)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = df['loan_status'].values\n",
    "y[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Normalize Data "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Data Standardization give data zero mean and unit variance (technically should be done after train test split )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.51578458,  0.92071769,  2.33152555, -0.42056004, -1.20577805,\n",
       "        -0.38170062,  1.13639374, -0.86968108],\n",
       "       [ 0.51578458,  0.92071769,  0.34170148,  2.37778177, -1.20577805,\n",
       "         2.61985426, -0.87997669, -0.86968108],\n",
       "       [ 0.51578458, -0.95911111, -0.65321055, -0.42056004, -1.20577805,\n",
       "        -0.38170062, -0.87997669,  1.14984679],\n",
       "       [ 0.51578458,  0.92071769, -0.48739188,  2.37778177,  0.82934003,\n",
       "        -0.38170062, -0.87997669,  1.14984679],\n",
       "       [ 0.51578458,  0.92071769, -0.3215732 , -0.42056004,  0.82934003,\n",
       "        -0.38170062, -0.87997669,  1.14984679]])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X= preprocessing.StandardScaler().fit(X).transform(X)\n",
    "X[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Classification "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Now, it is your turn, use the training set to build an accurate model. Then use the test set to report the accuracy of the model\n",
    "You should use the following algorithm:\n",
    "- K Nearest Neighbor(KNN)\n",
    "- Decision Tree\n",
    "- Support Vector Machine\n",
    "- Logistic Regression\n",
    "\n",
    "\n",
    "\n",
    "__ Notice:__ \n",
    "- You can go above and change the pre-processing, feature selection, feature-extraction, and so on, to make a better model.\n",
    "- You should use either scikit-learn, Scipy or Numpy libraries for developing the classification algorithms.\n",
    "- You should include the code of the algorithm in the following cells."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# K Nearest Neighbor(KNN)\n",
    "Notice: You should find the best k to build the model with the best accuracy.  \n",
    "**warning:** You should not use the __loan_test.csv__ for finding the best k, however, you can split your train_loan.csv into train and test to find the best __k__."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train set: (276, 8) (276,)\n",
      "Test set: (70, 8) (70,)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=4)\n",
    "print ('Train set:', X_train.shape,  y_train.shape)\n",
    "print ('Test set:', X_test.shape,  y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn import metrics\n",
    "Ks = 10\n",
    "mean_acc = np.zeros((Ks-1))\n",
    "std_acc = np.zeros((Ks-1))\n",
    "ConfustionMx = [];\n",
    "for n in range(1,Ks):\n",
    "    \n",
    "    #Train Model and Predict  \n",
    "    neigh = KNeighborsClassifier(n_neighbors = n).fit(X_train,y_train)\n",
    "    yhat=neigh.predict(X_test)\n",
    "    mean_acc[n-1] = metrics.accuracy_score(y_test, yhat)\n",
    "\n",
    "    \n",
    "    std_acc[n-1]=np.std(yhat==y_test)/np.sqrt(yhat.shape[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xd8U+X3wPHPSdJBBxsnCIjgxqIVByggWwXEASgO3F/3FvXrXj9Q3JuvA1ABAQVBZS8BkSmiDJUpqMjubtIkz++PpLWWlqZtkptx3q9XX7Q3N/eeljYnz3PPPY8YY1BKKaUijc3qAJRSSqnyaIJSSikVkTRBKaWUikiaoJRSSkUkTVBKKaUikiYopZRSEUkTlFJKqYikCUoppVRE0gSllFIqIjmsDiBYGjZsaJo1a2Z1GEoppSqxYsWK3caYRpXtFzMJqlmzZixfvtzqMJRSSlVCRLYGsp9O8SmllIpImqCUUkpFJE1QSimlIlLMXINSSqmaKioqYvv27RQWFlodSkxITk6mcePGJCQkVOv5mqCUUspv+/btpKen06xZM0TE6nCimjGGPXv2sH37dpo3b16tY+gUn1JK+RUWFtKgQQNNTkEgIjRo0KBGo1FNUEopVYomp+Cp6c9SE5RSSqmIpAlKKRXz3B4vLrfX6jACNmnSJESE9evXWx2KpTRBKaViXlZBEfvyXWTlF+HxGqvDqdSYMWNo3749Y8aMCel5PB5PSI9fU1rFp5SKablON25/Uip0e3DmekhJcpCaaD/oNZK7p93Nqh2rghpLxmEZvNrj1YPHm5vLwoULmTt3Lr169eKpp54qeWzo0KF88skn2Gw2evbsyZAhQ9iwYQP/+c9/2LVrF3a7nfHjx7Nt2zaGDRvGV199BcDtt99OZmYmgwYNolmzZlx33XXMmDGD22+/nZycHIYPH47L5eKYY47h448/JiUlhb///pv//Oc/bNq0CYB33nmHadOmUb9+fe6++24A/vvf/3LIIYdw1113BfXnVEwTlFIqZhV5vOQ53f/aZoA8p5sCl4f0ZAfJCXZrgqvAl19+SY8ePWjVqhUNGjRgxYoVnHbaaUydOpUvv/ySJUuWkJKSwt69ewEYOHAgDz30EH379qWwsBCv18u2bdsOeo7k5GQWLlwIwJ49e7jxxhsBePTRR/nggw+44447uPPOO+nQoQMTJ07E4/GQm5vLEUccwcUXX8zdd9+N1+tl7NixLF26NGQ/C01QSqmYZIwhq6Cowse9/sfzXR7SkhwkOv59xaOykU6ojBkzpmREMmDAAMaMGcNpp53GrFmzuPbaa0lJSQGgfv365OTk8Mcff9C3b1/Al3gC0b9//5LPf/75Zx599FH2799Pbm4u3bt3B2DOnDmMGjUKALvdTp06dahTpw4NGjTghx9+4O+//6ZNmzY0aNAgaN97WZqglFIxKcfpDuh6U5HHy758F8kOO8biy1N79+5lzpw5/PTTT4gIHo8HEeHFF1+s0nEcDgde7z9FIWXvRUpNTS35fNCgQUyaNIlTTjmFESNGMG/evIMe+4YbbmDEiBHs2LGD6667rkpxVZUWSSilYo7T7aHAVbUCgEK3B68xeLwGY1GmmjBhAldddRVbt25ly5YtbNu2jebNm7NgwQK6du3KRx99RH5+PuBLZunp6TRu3JhJkyYB4HQ6yc/Pp2nTpqxduxan08n+/fuZPXt2hefMycnh8MMPp6ioiE8//bRke+fOnXnnnXcAXzFFVlYWAH379mXatGksW7asZLQVKpqglFIxxes1ZBe4K9+xoucbg9tr8FpQ7TdmzJiS6bpil1xyCWPGjKFHjx707t2bzMxMMjIyGDZsGAAff/wxr7/+Oq1bt+bss89mx44dNGnShH79+tG6dWuuuuoq2rRpU+E5n3nmGc444wy6du3KcccdV7L9tddeY+7cuZx88smcdtpprF27FoDExEQ6depEv379sNtDe/1OrHqnEGyZmZlGFyxUSmXlF1Horl759K5tmzi21Iu0ADabYNPuEiW8Xi+nnnoq48ePp2XLlpXuv27dOo4//vh/bRORFcaYzMqeqyMopVTMKCzyVDs5lccAHq/B7fVaNu0XSdauXcsxxxxD586dA0pONaVFEkqpmODxGrILK67aqwljwG0MNgGbxG+/vhNOOKHkvqhw0BGUUiomZBcUhbwKz8rrU/FIE5RSKurlu9y4POHrtecxBrfHi1en/UJKE5RSKqq5PV5yC6tftVdden0q9PQalFIqqmUXurEyPej1qdDREZRSKmrlOt0UhXBq7+/swoA//soq4I/9Bfy1v6DCfYLl+++/L+mfV5F3332Xk08+mYyMDNq3b19yH1NVbNmyhdGjR1f4eMeOHQnl7T2aoJRSUanI4yXfGf6pvcoUX5+q6bTfvHnzGDRoULmPTZ06lR49ehz0+VdccQU//fQTq1at4sEHH+Tee++tcgyVJahQ0wSllIo6xY1gI/XKjwHcXoMnRNenZs+eTZcuXQ66T+3atUs+z8vLK5l6fOWVV0p66P3000+cdNJJ5OfnM3/+fDIyMsjIyKBNmzbk5OTw0EMPsWDBAjIyMnjllVcoKChgwIABtG7dmv79+1NQUBD07600vQallIo6gTaCtZrX+ErTbQJ2W3CuTe3evZuEhATq1KlT6b5vvfUWL7/8Mi6Xizlz5gBw11130bFjRyZOnMhzzz3He++9R0pKCsOGDeOtt96iXbt25ObmkpyczJAhQ/61rtTLL79MSkoKq1evZvXq1Zx66qlB+Z4qoiMopVRUqU4jWKt5jaEowLL0M844g4yMDG644QYmT55cMqqZPn06ADNmzKBbt24Bnfe2225j48aNDB06lGeffRYAm83GiBEjuOqqq+jQoQPt2rUDoF27dtx77728/vrr7N+/H4fjwPHLt99+y5VXXglA69atad26dUBxVJcmKKVU1KhpI1irebyVX59asmQJq1at4v3336d3796sWrWKVatWlXQOL3396dprryUjI4Pzzz//oOcdMGBAScdzgN9++420tDT+/PPPkm0PPfQQ77//PgUFBZx55pmsX7++Jt9qUGiCUkpFjRynO+pvjq3J9SljDKtXryYjIwOAjz76iFWrVvHNN98csO9vv/1W8vnXX39d0jsvKyuLO++8k2+//ZY9e/YwYcIEADZu3MjJJ5/M4MGDyczMZP369aSnp5OTk1NynHPPPbekaOLnn39m9erVVYq/qvQalFIqKhQWeSgsCu/U3qG1A1uhtjqqc31qxYoVtGnTJqB7rd58801mzZpFQkIC9erVY+TIkQDcc8893HbbbbRq1YoPPviATp06ce655/Lqq68yd+5cbDYbJ554Ij179sRms2G32znllFMYNGgQt9xyC9deey2tW7cmIyODtm3b1uhnUBldbkMpFfG8XsPuPGfIe+2VXW4jnOwBLOvx7LPPcswxxzBgwIAwRVVzNVluQ0dQSkUor9eQ63KTnuSI++4E2YWhbwRrNY/X4KmkcH7ww48AhPTm5MrYRIJWkVjpucJyFqVUleUUuilwedifXxTX3bMLXB6cbutekFVZ4ftd1ASlVAQqvfCey+Nlb74rKu77CTaP15ATojWeKhIrlz0iQU1/lpqglIow3nIW3vN4DXvynJZO7Vgh3N0iHIlJ7N2zV5NUEBhj2LNnD8nJ1S800WtQSkWYiq63GAP78lzUrpVAcoI9/IGFWV6IG8GWp3aDQ9m/5292794V1vNGExEqLeYolpycTOPGjat9rpAmKBHpAbwG2IH3jTFDyjx+FDASqOvf5yFjzDf+xx4Grgc8wJ3GmOmhjFWpSFDZ9RaDb1ThNYaUxNh9f1nk8ZJnQSNYu8NBvUOPDPt5o0mtRDu1kxPCcq6QTfGJiB14C+gJnABcLiInlNntUWCcMaYNMAB42//cE/xfnwj0AN72H0+pmFWV6y05he6wX5sJF2OMb/l2qwNRlgvlNai2wAZjzCZjjAsYC/Qps48Bilvu1gGK+270AcYaY5zGmM3ABv/xlIpZVX1Rznd5yMovirnrJblON+44LAhRBwplgjoS2Fbq6+3+baU9CVwpItuBb4A7qvBcROQmEVkuIst37dI5YxW98l1uXNW43lLo9rAvhsrQXW4v+VHWCFaFjtVVfJcDI4wxjYHzgY9FJOCYjDHDjTGZxpjMRo0ahSxIpULJ7fGSW1j96y1FMVKGXrzGk1LFQpmg/gCalPq6sX9badcD4wCMMYuBZKBhgM9VKiZkF7prfL0lFsrQswujvxGsCq5QJqhlQEsRaS4iifiKHiaX2ed3oDOAiByPL0Ht8u83QESSRKQ50BJYGsJYlbJEbhBLqYvL0MPdUDUYrGgEqyJfyOpUjTFuEbkdmI6vhPxDY8waEXkaWG6MmQzcB/xPRO7BVzAxyPiu+K4RkXHAWsAN3GaM0d9eFVOKPF7yg1xKHY1l6OXdmKwUaDdzpSxhjGFvniuk1WopiXbSw3S/Sk3sz3dpr70oEoz7oALtZm51kYRScSkcpdTRUIaujWDVwWiCUirMwllKHcll6FY0glXRRROUUmFkTPivt0RqGXq4G8Gq6KMJSqkwynG6LUkUxWXorgiZTrOiEayKPpqglAoTp9tDgYVdEozxFSRYXc7ttqgRrIo+mqCUCgOv15BdYP2LcnEZer7LmliKu0WEewyZ7cwivyg/zGdVNaUJSqkwyImwLgk5hW5L7j2yohFsriuHzp+cwRkfnsDE9Z9FdFWj+jdNUEqFWOnl2yNJgcvD/nxX2F6wrWoE+/yix9me/TsNUxpxy9RBXD6xN1v2bwp7HKrqNEEpFUKR3iXB6faGpQzdiupFgGV/LuajVe9xQ5tbmTXwe57t+BLL/1pCx1Gn8eqSobg8rrDHpAKnCUqpEKpo+fZIUlyG7g5hVV12YfirF51uJ/fOvJUjazfhobOfxG6zc0ObW1lwzQ90ObonQ757ks6fnMHi7QvDGpcKnCYopUIkmrokeLyGvfmukJShW9UI9rWlL/Db3vW82PlNUhPTSrYfnnYk7184mo/7fEGBO5++47ty94yb2VuwJ+wxqoPTBKVUCHi8hhxn5E7tlScUZehWTXGu272GN5a9yKXHX0GnZl3L3afr0T2Zf/VKbsu8l/FrP6X9yAw+W/OJFlFEEE1QSoVAdkHkT+2Vp7gMPVj3KVkxxenxerhv5i2kJ9XhqQ5DD7pvakIqj53zHLOu/J6j67bgrhk3csmEHvy295cwRasORhOUUkFW3eXbI0mus+Zl6IVF1kxxfrjqHVbuWMZzHYfRoFbDgJ5zfMOTmNx/Di92fpM1u1bT+ZO2vPDdMxS6C0McrToYTVBKBVFNl2+PJDUpQ/dYNLX3e9ZWnl/0BJ2b9+CiY/tV6bk2sXFV6+tZeM0qLmzZl5eXPM95H5/Ogt/nhihaVRlNUEoFUTCWb48k1S1Dt2KK0xjD4Nl3YBMbQ897DRGp1nEapR7K2z1H8NnFX2EwXPb5+dw+9Tp25e8McsSqMpqglAqSWG2AWuTxsicv8DJ0q6Y4J6wbw9ytM/lv+2doXPuoGh+vQ9POzLlqGfec8RBf/jqBc0Zk8MlPH+I1sfd/HKk0QSkVBEUx3gDVawIrQ7dqinNX/k4en/8Apx9+JoNOuSlox63lqMXgs59g9pVLOa7hidw/6zYuGteFdbvXBO0cqmKaoJSqIWOMb0rL6kBCrLIydKsawQI8Pu8B8opyeanrO9gk+C9rrRocx8TLZvBqt+Fs2PsrXT89k+cWPqYNaENME5RSNWRFA1SrHKwMPc/lseTnMHPTVCb+Mo672z5EqwbHhew8IsKAE69iwaBVXHLc5byxbBgdR53G7M3TQ3bOeKcJSqkasKoBqtXKlqG73NZMcea6chg8+w6Oa3Ait59+X1jO2aBWQ17rPpwvLptBkiOJgZMu4savBrIj98+wnD+eaIJSqpqsaoAaKYrL0K1siPvcwsf4K/dPXu76Non2xLCe++zG5zBr4BIGn/0EMzZ9zTkj2/DhqnfxeOPvDUuoaIJSQZfrdEfM0uKhZNXy7ZHE6fayO9dpyc9h6R/fMeLH4dzQ5lZOPbxt2M8PkORI4p4zHmLe1Stoc1gmj8y9hwvGduDnnT9aEk+s0QSlgirf5SbP6WZfvous/KKYfQG3evn2SGLF/7DT7eS+Wf90Krda87ot+Ozir3i750dsz9lG99HteGL+YPJcuVaHFtU0QamgKSzykFOqxLjQ7WFPrpM8pzumGnD6qvZit6Q8Gry6dCi/7f3lgE7lVhIRLj5uAAuvWcXAk67lvZWvc+6oNkzbOMXq0KKWJigVFEUeL9kFB16HMPim/PbkBbdLtpWyI2z59nizbvfPlXYqt1Ld5Hq80OUNpvSfQ3piHQZN7segyZfxR842q0OLOpqgVI15vYb9+Qe//8Xj9d0jsz/EC+OFmlVrGykfj9fDvTNvoU5SPZ7u8ILV4RzU6UecxcyBi3m0/bPM3zqbc0a24b2Vr+P26ug7UJqgVI0YY9hfUBTwiMLp9rI3z0VOYVHUTftF+vLt8eCDVW/zw47lPNvxRerXamB1OJVKsCdw++n3Mf/qHzir8Tk8MX8wPUa354cdy60OLSpoglI1kl1Q9f5zBsh3edidG13TftGwfHss25q1hf9b9CRdmvescqdyqx1Vpymf9PmC/13wKbvzd3L+mHN5eM7dZDuzrA4tommCUtWW63RT6K5+gvH6W+PszXNFfJNVq9Y2Uj7/6lTeufqdyq0kIvRqdTELrlnFdRn/YcSPwzlnZAaTf/086mYTwkUTlKqWwiJP0DoHFHl8037ZhVVf1iEcrFrbSP1j/LrRzNs6i/+2f4Yj05tYHU6NpCfV5rlOL/PN5d9ySOph3PT1lQyc1JetWVusDi3iSKxk7szMTLN8uc7rhoPL7fUtZBeCY4tAelICtRLtITh69ezLc1myfMSO3D/5bvu39D22f1SOGIJlV/5Ozh3Zhpb1j2VSv1khaQZrFbfXzYer3mHod0/jNR6uOvl66ibXtzqsg0qwCwNb9+f4RsdX+xgissIYk1nZfo5qn0HFJY/XsL8gNMkJfB2zswuLyHe5SU9OINFh7YuRVWsbeY2Xm76+kqV/Lubb3+cyrMtbOGzx+ef62Lz7ySvKZViXt2MqOQE4bA5uOvUOLmzZl8fm3c/wH960OqSAnHpE6xolqEDF52+8qhZjDPvyXWEpFHB7fedKTrCTnuTAZgv/CMLK5dtH/DicpX8uplOzboxdM4ocZxZv9xxJkiPJknisMmPTN0z6ZTwPnvV4SDuVW+2I9MZ80GtsVPTxq5Vop26t8PwextbbERVSWQXhb11UWORht0XdKKxavv2PnG08t/AxOhzVmdEXTeKZDi/y9YYvufrLS+KqdU6OM5uHZt8Z1k7lVrPb7FHxEa4p55AmKBHpISK/iMgGEXmonMdfEZFV/o9fRWR/qcc8pR6bHMo4VeWyC4ssq2Ir3Y3CWYOqwaqwavl2X7XaXXiNlxe6vImIcOOpt/Nat/+xcNs8+n1xAfsL94U9Lis8t+hxyzqVq8gQsik+EbEDbwFdge3AMhGZbIxZW7yPMeaeUvvfAbQpdYgCY0xGqOJTgct3uSOiMarH37EiyeEhPTkBe4im/dwWLt/+5a/jmbV5Kk91GErTOs1Ktvc/8UpqJ9Xm5m+uou/4boztO5lD0w63JMZwWPLHIkb8+B43tbndsk7lynqhHEG1BTYYYzYZY1zAWKDPQfa/HBgTwnhUNTjd/24AGwmcbi97cp3khmDaz8ply/cW7OG/c+8n49DTuCHjtgMe73lMbz69aBJbszbTZ1yXmC1LLnQXcv+s22hSuymD2z1pdTjKQqFMUEcCpbsjbvdvO4CINAWaA3NKbU4WkeUi8r2IXBS6MFVF3B4vWeU0gI0EBt80XLC7UVi1bDnA4/MfJMu5j5e7voPdVn6Z/TlHdWLCpVPJcu6nz2fnsX732nL3i2avFXcq7/ImqQmpVoejLBQpRRIDgAnGmNKvNE39dfJXAK+KSIuyTxKRm/xJbPmuXbvCFWtc8HoN+/Ijv7VPcTeKfXk1b0Jr1bLlAHO3zGTCutHccfr9nNDo5IPue+php/PFZTMwGPqO7xpTfd3W7vqJN5YN47LjB9KxaRerw1EWC2WC+gMofct3Y/+28gygzPSeMeYP/7+bgHn8+/pU8T7DjTGZxpjMRo0aBSNmRdUbwEYCl8fLnho0obVy+fY8Vy4PzL6dY+q14q62gwN6zvENT2RyvznUTqrDpRN6snDb/BBHGXqlO5U/1WGo1eGoCBDKBLUMaCkizUUkEV8SOqAaT0SOA+oBi0ttqyciSf7PGwLtgNiby4hQ2YXWVLAFQ77Lw65cZ5WLOqxcvn3Id0+yPft3Xur6NsmO5ICf17Rucyb3m02T2k0ZOLFP1C+M9/6qt1j19wqe6zQsKjqVq9ALWYIyxriB24HpwDpgnDFmjYg8LSK9S+06ABhr/v2293hguYj8CMwFhpSu/lOhk+d0R1WH8fIUd6MItAmtlcu3r/xrKe//8DaDTrmZM45sV+XnH5p2OBP7zeDERq25fsrljF87OgRRht7WrC0MWfQUXZufT59Wl1kdjooQ2otPlSgs8kRsUURNHKwbhTGG3bkuS6YzXR4X3T49m2znfuZfvZL0pNrVPlaeK5dBk/uxYNtcnu34Eje0uTWIkYaWMYYBX/RixY6lzL96RdQ3g411tRLt1E5OqNExAu3FFylFEspiFS3ZHgsKizzsznOS7zqwAMLK5dvfWDaM9XvWMKTz6zVKTgCpiWl8ctFEzj+mN4/Ou4+Xvn8+apZwGLf2U+b/PjsmOpWr4NIEpUpugI2Ol7PqMQZyCt3syXWWdKOwcvn2X/as49UlQ7jo2MvodvT5QTlmkiOJ4Rd8Sv8TruLFxc/wxPwH8ZrIvpa4K+9vnpj/IG2POItrWt9odTgqwmiz2DhnjGF/vjVTXFZw+5NxssOL02NNcvIaL/fPvJW0xHSe6TgsqMd22By80u1d6iTVYfgPb5LlzOKlrm9HbCf0R+fdT747j5e6vhNzncpVzUXmb60Km6yCIstuTLVSTVYCrqkRPw5n2V/f83r392mUckjQj28TG091eIE6yfV4cfEz5LiyeKfnqIjrhD5j49d8+esEBp/9BC3rH2t1OCoCVfqWRUTuEJF64QhGhVeOhQ1g49X27N95buFjdGzahcuOvyJk5xER7jvzEZ7tOIxvNkzmqi8vjqhO6NnOLAbPuZPjG57EbZn3Wh2OilCBjKkPxdfodZy/O3n8Lu0ZQwpcHvIjoAFsPDHGMHiOv1N55zfDsmTBDW1u4/Xu77No23wu+/x89hXuDfk5A/Hcwsf5O28HL2mncnUQlSYoY8yjQEvgA2AQ8JuIPF9e6yEVHXwNYGOzYi+STfplHLM3T+Phdk9yVJ2mYTtvvxMG8kGvMfy860f6juvG37l/he3c5fl++0JGrh7OjW1u49TDTg/LOcX/oaJLQFcl/TfR7vB/uPF1fpggIi+EMDYVAsUNYOPvqpO19hTs5r/z7qfNYZlcnxH+e5R6tOjF6Iu+ZFv2VnqP68zW/ZvDHgP8u1P5g2c/EbbzpiU7aJCWRKJdCzGiSSDXoO4SkRXAC8Ai4GRjzC3AacAlIY5PBZHX6+uxFycFexHliXkPku3cf9BO5aHW/qiOjL/0G7KdWfQeZ00n9FeXDGHDvl/D2qncYRNSEh3YbUK91ETSkx06mooSgbydqA9cbIzpbowZb4wpAjDGeIELQxqdCpridY6s6jcXz+ZsmcGE9WO44/QHOL7hSZbGcuphpzPxspkIQt/xXVm5Y1nYzr1210+8ufwl+p1wZVg7laeX6XqQkqijqWgRyP/QVKDkyqqI1BaRMwCMMetCFZgKruxCN64obQAbzfJcuTww63Za1j+WuwPsVB5qxzU8gS/7+zqhXzbhfBb+Pi/k5/R4Pdwz8z/USarHk+cOCfn5iiUn2El0HPgyp6Op6BBIgnoHKF2fmuvfpqJELDSAjVb/t+gJ/szZzktd3o6o+5Ca1mlW0gn9ikl9mLrhgIUGgup/P7zJj3+v5LlOL4WtU7kIpCcd/FbP4tFUgo6mIlIg/ytSutO4f2pPb/CNEoVFHnItWoQv3q34awkfrHqHQafcRNsjz7Y6nAMUd0I/+ZAMbvjqCsat/TQk59m6fzNDv3uKbkdfQJ9Wl4bkHOVJq6BBcFl2m1BfR1MRKZAEtUlE7hSRBP/HXcCmUAemai6WG8BGOpfHxb0zb+XwtCN4pN3TVodToXrJ9Rl38de0a9KBO6ffwPs/vBXU4xtjeGD27dhtDoac92pY7v2CfwojqiIl0UH91EQdTUWQQP4n/gOcjW813O3AGcBNoQzKKtHS/TkQ8dAANpK9vvRFftmzlqGd36hxp/JQS01M4+M+X3DBMX14dN79DFv8XND+Fj5b+wnf/j6HR9s/yxHpjYNyzECULYwIlMNu09FUBKn0LYYxZie+RQVjmjGGXTlOEh02khx2khy2gKYHIlG8NYCNNL/sWcdrS4fS99h+dD26p9XhBCTJkcR7F3zCfTNvZdj3z5Ll3MdTHV6oUQPXXXl/8+T8wZxxxNlc3fqGIEZ7cBUVRlRFSqKDRLstqleXjgWVJigRSQauB04EStajNsZcF8K4LGEAp9tb0p8uwW7zJyxbVA37swvccdkANhJ4vB7um3kLaYnpPN3xRavDqZLiTuh1k+vy3so3yHJm8XLXd6rdCf2/8+4j353HsK5vh61TeSCFEYEqHk3lOd3kOd06G2GBQP4nPwbWA92Bp4GB+JZwj3lFHi9FHi95TrCJkJRgI9HuS1iR2pIwp7DI0k7d8W7E6uEs/2sJb3T/ICSdykPNJjaePHcodZLq8cLip8lxZvHO+aNIdiRX/uRSpm/8ism/fs5DZz8Z1k7lgRZGVEVqkoMkh46mrBDI25pjjDGPAXnGmJHABcDJoQ0r8niNocDlWxJ9V46T/fku8l3uiLrxVRvAWmt79u88v/BxOjbtwqXHX251ONUmItx75sM82/Elpm6cwpWT+pLrygn4+dnOLAbP9nUqvzXznhDBod8uAAAgAElEQVRG+m/VKYwI+Nj+0VRakl6bCqdAElRxGdh+ETkJqAM0C1lEUaB4KjCn0M3uXCd7cp3kOt24LFy6wuX2agNYCxljeHD2nRhM2DqVh9oNbW7lje4fsHj7gip1Qn924WPszP877J3Kq1sYURWpSVrpF06B/JSH+9eDehSYDKwFhoY0qijj9hrynG725bvYmVNIVkERhUWesFUFeryG/QUunSO30MRfPmPOluk8fHZ4O5WH2mUnXMGHvcaydtdPXDSuKzty/zzo/ou3L2TU6v+FtVM5BKcwIlA6mgofOdiLqIjYgEuNMePCF1L1ZGZmmuXLl1f7+cYYduY4gxiRr71/gt1GUoKvMtAegqpAr9ewN98VUVON8WZPwW7OGdmGZnWaM6X/XMuawYbSwm3zuebLS2lQqyHjL/mGpnWbH7BPobuQzp+0pchTxNyrl4etGawINExNsqTqtnh1gHgqSqqVaKd2DUerIrLCGJNZ2X4Hfcvh7xpxe40iiWMGcHn+mQrcneskp7AoqFOB2gDWeo/Pe4AcZxYvWdipPNTaN+nAhEunkuPKpve481i3e80B+7yy5P/YuO+3sHYqh9AURgTKYbfRIC2JVB1NhUQgY+KZInK/iDQRkfrFHyGPLAZ5vIZ8l+efqcD8mk0FZhcWaQNYi83ePJ3P14/1dyo/0epwQqrNYZlM6jfL1wl9XFdW/rW05LE1u1bz1vKX6XfClXRo2jlsMYWyMKIq0pIc1EtNxBGl905GqoNO8QGISHkrmxljzNGhCal6InGKL1DFU4HF91w5ArgAm+9yk1OoPfaslOvKocOo00hJSGXWwO8jqhlsKG3N2kL/zy9gZ/7fjOw9nrMan8MFYzvwR852FlzzA/WSw/f+NRILFnKdbvJj+L6pcE7xBdJJ4sDJZhVUxVOBLo+XXKeveWWSw1bS1aKswiKPJqcIUNypfHL/2XGTnMDXCf3LfrMYMLE3AyddRI8Wvfjx75W8d/7HYU1OyQn2iEtO4BtNJTlsZMfZtalQCKSTxNXlbTfGjAp+OAr+mQrMd3kQKSLJbi+5SdhrTEQ3gN247zeun3I5XY/uyT1nPExKQorVIYXE8j+/58NV73Jtxs2cfsRZVocTdoemHc4Xl03nykl9mfzr53Q7+gJ6twrfAtvB7BgRCgnFXShcnpgeTYVaIFN8b5T6MhnoDKw0xoSvb34AonmKryoEIvaXPceZzfljO7A9eysF7gKOqt2MIZ1f47xm3awOLaicbiddPz2LvKIc5l+9krTEdKtDskxeUR4jfxzOZScMDGvnjNrJCdRKjI6ClOJVBWJlNBVpU3x3lDlwXWBkDWJTNRCpv+Je4+WO6dezad9vfHbJ1wjC4Nl3cMXEPvRudQnPdHiRQ9MOtzrMoHhj2TB+3buOj/t8EdfJCSA1ITWs3SLANzqJluQE/x5N5enabFVSnQncPKBVsANR0e2VJUOYtvErnjx3CO2bdKBdk3OZfeVSHjzrcaZv/Ir2IzP4cNW7eLzR3Ypp/e61UdepPNakJ0fu1F5FRIQ0fxeKUNwPGasqTVAiMkVEJvs/vgJ+Ab4MfWgqWkzf+BUvLn6Gy44fyA1tbivZnuRI4t4zH2buVctpc1gmj8y9hws/68jPO3+0MNrq83g93DfrFtITa/NMx2FWhxOXaiVGZmFEoBLsNhqkJpISRSNAKwXyVqT0X6Ib2GqM2R6ieFSU+XXPem6bdh2nHHoqL3R5o9wedEfXO4bPLv6Kib98xuPzB9N9dDtuaHMbD571GKmJaRZEXT0f/fgeK/5ayps9PqRhSiOrw4k7IpAWAfc81ZSIkJ6cQHKCXW+0r0Qgb0V+B5YYY+YbYxYBe0SkWUijUlEh25nFtVP6keyoxYe9xlLLUavCfUWEi48bwMJrVnHFSYN4b+XrnDuqDdM2TgljxNW3LXsrzy96nE5Nu3LJcTG/fmdESk9KiNpFRMujo6nKBZKgxgOl2xV4/NtUHPMaL7dOvZatWZt5/4JPOTK9SUDPq5tcjxe7vMmU/nNIT6zDoMn9uHZyP/7I2RbiiKuvuFM5UOEoUYVWtBVGBKp4NFUvRa9NlSeQ8bLDGOMq/sIY4xKR8PXQVxHpxcXPMGvzVP7vvFc5s3H7Kj//9CPOYubAxby78jVe/v55zh15Kg+e/RjXZ9xa7RVcQ+WL9WOZu2UGz3YcRpPa4e1UnuSwlazwHM+isTCiKhIdvtGU0+0lTIsgVFs4E2kg90HNBN4wxkz2f90HuNMYE76GWwGIl/ugIsFXv03khq+u4IqTBvFSl7drPKLYmrWFR+bew+zN0zj5kAxe6PwGbQ6r9BaJsNidv4tzR7ahWd2jw96pvPh+k+zCIgrieCHKYNx3oyJLULqZ+/0HeEREfheR34HBwM0BBtFDRH4RkQ0i8lA5j78iIqv8H7+KyP5Sj10jIr/5P64J5Hwq9NbtXsOd02/ktMPb8n+dXg3KdFfTOs34pM8X/O+CT9mZt4Pzx5zLI3PvIduZFYSIa+bx+Q+Q48rm5a7vhjU5lS4IiKabUoMtVgojVPVUOoIq2VEkDcAYkxvg/nbgV6ArsB1YBlxujFlbwf53AG2MMdf5u6UvBzLx3Zu6AjjNGLOvovPpCCr09hfuo8fo9uQX5TNj4CIOSzsi6OfIdmYx5Lsn+WjVexySeijPdnyJC1v2teS6z6zN07hyUl/uO/O/PHDWo2E9d3lJKR5HUvGcnGNZ0EZQIvK8iNQ1xuQaY3JFpJ6IPBtADG2BDcaYTf5rWGOBPgfZ/3JgjP/z7sBMY8xef1KaCfQI4JwqRDxeD7d8cw1/5Gzjw15jQpKcAGon1eH5Tq/wzeXfckjqYdz49UCu/PJifs/aGpLzVSTXlcPg2XfQsv5x3Hn6A2E9d0UFAfH2Yh2rhREqcIFM8fU0xpRMvfkTxvkBPO9IoHRp1nb/tgOISFOgOTCnqs9V4fF/i55g7taZ/N95r5J5xJkhP1+bwzKZevkCnu7wAou3L6DDqDa8sWwYRZ7wNMr1dSr/g1e6vhP2TuW1D1IQEE9JKtYLI1TlAklQdhEp+QsVkVpAsP9iBwATjDFVmr8QkZtEZLmILN+1a1eQQ1LFJv0ynjeXv8TVrW/kypOvC9t5HTYHN516Bwuu+YGOTbvw3MLH6PrpWSz7c3FIz1u6U3k4knFpKYn2StcDq52cEPP3zkR7xwgVHIH8BnwKzBaR60XkenzTbYEstfEHUPrmmMb+beUZwD/TewE/1xgz3BiTaYzJbNRI7+wPhTW7VnPPjJtpe8RZPGtRe58j05vwUe9xjOw9nhxXFr0+O48HZt3O/sIKL0lWm9Pt5J6Zt3BE+pE80u7poB//YGz+fm2BSI/hJKWFEapYpQnKGDMUeBY43v/xjH9bZZYBLUWkuf++qQHA5LI7ichxQD2g9Nvi6UA3//WuekA3/zYVRnsL9jBocj/qJtfj/QtHk2i39va37i0u5Nurf+DmU+9k9M8jaD8ygy/WjyXQQp9AvL7sRX7bu54XOr8Z9k7l6cmOKhWDxGqSirWOEar6AhpDG2OmGWPuN8bcD+SJyFsBPMcN3I4vsawDxhlj1ojI0yLSu9SuA4CxptSrjDFmL/AMviS3DHjav02Fidvr5uavr2Jn3g4+7PUZh6QeZnVIAKQmpvFUh6FMv2IRTWofxa1Tr6X/Fxeyef/GGh97/e61vL70BS4+rj+dm3cPQrSBS7TbSE6oerKJtSSlhRGqtIDKzEWkDb4qu37AZuALY8wbB39WeGmZeXA9MX8w7618nVe7DWfAiVdZHU65PF4Po356n+cXPo7L4+SutoO5LfPeahU1eLween3WiS37N/HtNT+EtRmsAPVTEyu99nQwOYVF5MdACXr91ES99hQHalxmLiKtROQJEVkPvIGvqk6MMZ0iLTmp4JqwbgzvrXyd6zNuidjkBGC32bn2lJtZeM0qure4kBcWP02XT8/gu+0Lqnysj358l5U7lvFMxxfD3qk8JclRo+QEvpFUagQvgR4ILYxQZR3st2E9cB5woTGmvT8pRf9bNHVQP/69kvtn3spZjc/hyXMDudRovUPTDmf4BZ8wuu+XON1OLh7fjbum38Segt0BPd/XqfwJzmvWnYvD3KncJkJqkKa00pIcUZuktDBCledgCepi4C9groj8T0Q645uNUDFqV/5OrpvSnwYpjfjfBZ+SYI+u/mfnNevGvKtXcOfpD/D5+jGcMyKDMWtGHbSIwhjDg7PuAGBo59fC3rGiqoURlYnWJKWFEao8FSYoY8wkY8wA4DhgLnA3cIiIvCMi3cIVoAqPIk8RN319JXvyd/NRr8+idkG+lIQUHmn/NDMHfs8x9Vtxz4yb6Tu+G7/uWV/u/p+vH8vcrTN5pN3TlnQqr05hRGWiLUlpYYSqSCBl5nnGmNHGmF747kf6AV/DWBVDnvx2MIu3L+Clrm/T+tA2VodTY8c3PJFJ/WbxUpe3Wb97DZ0/acvQ756iwF1Qss/u/F08Pu8BTju8LdeeElD/46ARfNeNQiWakpR2jFAVqdIVSWPMPv/NsRG11IaqmTFrRvHBqne4+dQ7ueT4y60OJ2hsYmPgydeyYNAq+rS6lFeWDKHTqEzmb50N/NOp/KUu74S1UzlAapIj5OvqREOS0sIIdTD6mxHnVv61lMGz7+CcJp147JznrA4nJBqlHMKbPT9k/CXfYBMb/b+4kMsmnM8X6z/jzrYPclzDE8Iaj90mYbt3KS3JEXB3inDTwghVGU1QcWxn3g6umzKAQ1MP570LPo64lWyD7ZyjOjHnqmXcd+Z/WfLnIlrVPz7sncoh+IURlUmN0CRVO1kLI9TBRd5vrQoLl8fFDV9dQZYzi68GzKV+rQZWhxQWyY5kHjjrUQaeNIhkR62wdypPdthJcoS/IKB4qi/X6Q77ucuTUM3OGSq+aIKKU4/Ou5+lfy7mvfM/5sRGra0OJ+yOSG8c9nMKkGZhQUAkJSktjFCB0Cm+OPTx6g8Ytfp/3J55H32OvdTqcOJGOAojAonB6uk+LYxQgdLfkjiz7M/FPDL3Hjo17crD7Z6yOpy44bBJxFTUpSY5LBvB2ERIj5Cfg4p8mqDiyI7cP7l+yhUcmd6Ed84fGfbS6ngWynueqiMl0ZokFe4CERXd9K1MnHC6nVw3ZQC5RTmMu+Rr6ibXszqkuJGcYCfREXnvBVP8Jd45heG5JqWFEaqqNEHFAWMMD825i5U7lvHBhWPCft9PPBMhoqe0wpmkamthhKqiyHtbp4JuxOrhjFkzknvOeIgLWl5kdThxJS3JEfH3+oRjui8l0V7jJUVU/NHfmBi3ePtCHpt3P12bn88DZz1mdThxxWGTkhFKpAtlkrKJWF45qKKTJqgY9kfONm78eiDN6hzNWz0/xCb63x1OtWtFVmFEZVISHdQOQTGHFkao6tK3NTGqwF3AtZMHUOgu4KPLZlA7qY7VIcWVaL3Xp3jZi+zCoqAcL1ELI1QNaILym/zLZNyeZM48sl3U96QrXoRv9c6VjOo9gZb1j7U6pLgS7U1Qg5WkfEuKRO/PQVlPf3v8Hp/3OKv/Xk295Pp0bt6Dni160bFZV1ITUq0Orcre/+Etxq/7lAfOeoxuLS6wOpy4EwurwwYjSdXSwghVQ5qg/BZdu4jxa75m6oYpzNo8lQnrRpNsT+aco86j5zG96Hr0+TRKOcTqMCu1cNt8nvz2IXq26MU9ZzxkdThxJ5ZWh61JktLCCBUMYoyxOoagyMzMNMuXL6/2840x7MxxAuD2ulnyx3dM2ziZaRu/Ylv2VgTh9CPOpEeLXvRo0Yuj6x0TrNCD5vesrfQY3Y6GKYfwzeXzSUtMtzqkuNMgNTHmRg2FRR6yCqqWpOrUStBrT6pCIrLCGJNZ6X6aoHxKJ6iy29fu/olpG79i2sYp/LRzFQCt6h9PjxYX0uOYXmQceprlFXL5Rfn0/uw8fs/ewrTLF0ZkAo11KYn2iGtpFCxVSVKJdhv1UhNDHJGKZpqgqqiiBFXWtuytTN/4NdM2TmHx9gV4jIfDUg+nW4sL6dmiF+2adCDRHt4/TmMMt04dxKRfxvPJRRPp3Lx7WM+vfFNaDdMSY7qcOpAkJUD9GBxFquDSBFVFgSao0vYV7mX25ulM3TCZuVtnkl+UR3pibc5r1o2ex/TivGbdw1Le/fbyV3h6wSP8t93T3NE2/CvEqviZ0qosScXyKFIFjyaoKqpOgiqt0F3Igt/nMm3jFKZv+prd+TtJsCXQrkkHerToRfcWF3B42pHVPn5F5m+dzeUTe3PBMRcx/IJPYvodfKSKtymtipJUPIwiVXBogqqimiao0jxeDyt3LC25brVx328AZBx6WkmRxbENjq/xH/LW/ZvpProdh6cfyVf955KamBaM8FUVxOuUVnlJKl5GkarmNEFVUTATVFm/7f2FaRsmM3XjFFbuWAZA87ot6O6/bpV5+JlVXpspz5XLhZ914q+cP5h+xSKa1m0eitBVJSJhhVqrlE5S8TaKVDWjCaqKQpmgStuR+yczNn3D1I1TWLRtHi6Piwa1GtHt6PPp2aIX5zQ9j1qOWpXGeuPXA/lmw5eM6TuZDk07hzxudSCd0vIlqeyCorgcRarq0wRVReFKUKXlOLOZs2UG0zd9xazN08h2ZlHLkUKnZl3ofnQvuh7dk/q1GhzwvNeXvsjzix7niXP/j1tOuzusMQciyWGjyGPwxsjvVkV0SsvH4zXYo7xzhgovTVBVZEWCKs3lcbF4+wKmbZzCtI1T+Cv3T+xi54wj29GjxYV0b9GLpnWaMWvzNK6adDF9j+vHWz0+irh37wl2G/VTE/F6DTlON4VFHqtDCokkh426KTqlpVR1aIKqIqsTVGnGGFbv/IFpG6cwdcMU1u9ZA8AJDU9me87vHFW7GZP7zyElIcXiSP9NBBqkJv3r3bTT7SG7wB1ToykBGqQl6ahBqWrSBFVFkZSgytqyf5N/ZPUVO3L/ZPyl39CkdlOrwzpA3ZQEkhwHTnkZ4xtNFbhiYzSVluQgNU4LI5QKBk1QVRTJCSoaBPKi7XJ7yS4swuON3t85u01okBrfhRFK1VSgCUrLblSNJTlsAY0oEh02GqQmkhLF3b51dVilwiekCUpEeojILyKyQUTKXftBRPqJyFoRWSMio0tt94jIKv/H5FDGqarPJlKlZcJFhPTkBOqnJkbdNZxkh73cKUylVGiEbCJdROzAW0BXYDuwTEQmG2PWltqnJfAw0M4Ys09ESi+4VGCMyQhVfKrmBN91p+oszpdg942m8lwe8p1uIn3ST4A0XR1WqbAK5QiqLbDBGLPJGOMCxgJ9yuxzI/CWMWYfgDFmZwjjUUGWluwgoQY3Z4p/Ubt6qYk4Inw0lZrkiLoRn1LRLpQJ6khgW6mvt/u3ldYKaCUii0TkexHpUeqxZBFZ7t9+UQjjVNWQ7LCTkhicEUWC3UaDtCTSkhxEYgpw2ESr9pSygNV/dQ6gJdARaAx8KyInG2P2A02NMX+IyNHAHBH5yRizsfSTReQm4CaAo446KryRxzG7TahdK/i/OqlJDpIcNrIL3RR5vEE/fnXp8hFKWSOUI6g/gCalvm7s31badmCyMabIGLMZ+BVfwsIY84f/303APKBN2RMYY4YbYzKNMZmNGjUK/negDiBA3VoJIatkc/g7UaQnR8ZoKjnBTqJDi12VskIo//KWAS1FpLmIJAIDgLLVeJPwjZ4QkYb4pvw2iUg9EUkqtb0dsBZludq1EsLSFDQl0UGDtCQSLWxAKgLpOrWnlGVC9tdnjHGLyO3AdMAOfGiMWSMiTwPLjTGT/Y91E5G1gAd4wBizR0TOBt4TES++JDqkdPWfskatRHtYm6PabUK91EQKXB5yCovCXumXluSoVoWiUio4tJOEn3aSOLgEu416KaGb2quMx2vIKSzC6Q7PtSmHTWiQlhSWcykVb7SThAoaEd/SElZ2ULDbhLopif44Qn++2rW0MEIpq+kEu6pUnVoJEXMPUHKCnUS7jZxCN4Xu0DSfrZVor9H9XUqp4NC/QnVQvtLvyGrvY7MJdVISQjKaEoG0IN3fpZSqGf1LVBVKtNtIi+AqtpLRVBAXRkxPql7rJqVU8OkISpXLJkKdKLgOY7P54qybkoCthsOpBLuNWlHcaV2pWKMJSh1A8F13iqaRRJLDTsO0xBolmNraDFapiKIJSh0gLdkRld0TxL/0R92Uqhd1pCTaw3IDslIqcPoXqf4lmE1grZLksNMgNfDRlM3fVV0pFVk0QakSoWoCa4Xi0VS9lMoXRtRVcpWKTJqgFBD6JrBWqWyZ+US7Laztm5RSgdMEpYDwNYG1Qull5ksvjCj4Rk9KqcgUm69IqkqSE8LbBNYqxQsjpvoXRkxJcsRsUlYqFujbxzjnsEnclVenJTlIdtgipn2TUqp88fXKpP5FBOqmJMbcdadA6MhJqcinf6VxrHZy5DSBVUqpsjRBxamUMC8+qJRSVaUJKg4l2m2kJ0d+nz2lVHzTBBVnoqUJrFJKaYKKM9HWBFYpFb80QcWRtKTobAKrlIpP+moVJ5IcNlK1IapSKopogooDNn/jVKWUiiaaoGKcgG+1Wb3upJSKMpqgYlx6cgIJ2jVBKRWF9JUrhiUn2Gu0BLpSSllJE5SfiBBLk2Dx2ARWKRVb9BWslIZpSbg8XgqLPLjcXozVAVWT4LvfKR6bwCqlYocmqFJsNiHZ5utRZ4zB6fbiLPLidHuiKlnF8uKDSqn4oQmqAiJSspCfMY5/kpXHg4ngbKVNYJVSsUITVADKJivfNKB/ZBVBySpBm8AqpWKIJqgqEhGSHHaSHHYgAafbUzK68lqYrUTQJrBKqZiiCaqGSpJVMrjcvlFVoQXJqk4tXXxQKRVbNEEFUaLDRqLDRnoyFHm8ON2+ikCPN7TJKjXJ4R/RKaVU7NAEFSIJdhsJdhtpSQ7cHi+Fbi/OIg/uICerJIfvHEopFWv0lS0MHHYbaaWSldPt+yjyeGt0XG0Cq5SKZZqgwsxht+Gw20hNAo/XlFyzqmqy0iawSqlYF9K7OUWkh4j8IiIbROShCvbpJyJrRWSNiIwutf0aEfnN/3FNKOO0it0mpCQ6qJ+aSKO0JNKTHSQGeINtWrJDm8AqpWJayEZQImIH3gK6AtuBZSIy2RizttQ+LYGHgXbGmH0icoh/e33gCSATMMAK/3P3hSpeq9n8ySolEbxefxcLd/ktl5IddlISdfCrlIptoXwL3hbYYIzZZIxxAWOBPmX2uRF4qzjxGGN2+rd3B2YaY/b6H5sJ9AhhrBHFZhNqJdqpm5JIo/Qk6tRKIMlhQ/CNumrX0uSklIp9oXylOxLYVurr7cAZZfZpBSAiiwA78KQxZloFzz0ydKFGrn93sTAYgzaBVUrFBavfijuAlkBHoDHwrYicHOiTReQm4CaAo446KhTxRRQRQXOTUipehHKK7w+gSamvG/u3lbYdmGyMKTLGbAZ+xZewAnkuxpjhxphMY0xmo0aNghq8Ukopa4UyQS0DWopIcxFJBAYAk8vsMwnf6AkRaYhvym8TMB3oJiL1RKQe0M2/TSmlVJwI2RSfMcYtIrfjSyx24ENjzBoReRpYboyZzD+JaC3gAR4wxuwBEJFn8CU5gKeNMXtDFatSSqnIIyaS1ouogczMTLN8+XKrw1BKKVUJEVlhjMmsbD+901MppVRE0gSllFIqImmCUkopFZE0QSmllIpImqCUUkpFpJip4hORXcDWGh6mIbA7COGEUjTECNERp8YYPNEQp8YYPDWNs6kxptLuCjGToIJBRJYHUvpopWiIEaIjTo0xeKIhTo0xeMIVp07xKaWUikiaoJRSSkUkTVD/NtzqAAIQDTFCdMSpMQZPNMSpMQZPWOLUa1BKKaUiko6glFJKRSRNUEoppSKSJihARD4UkZ0i8rPVsVRERJqIyFwRWSsia0TkLqtjKktEkkVkqYj86I/xKatjqoiI2EXkBxH5yupYKiIiW0TkJxFZJSIR2apfROqKyAQRWS8i60TkLKtjKktEjvX/DIs/skXkbqvjKktE7vH/3fwsImNEJNnqmMoSkbv88a0Jx89Qr0EBInIukAuMMsacZHU85RGRw4HDjTErRSQdWAFcZIxZa3FoJUREgFRjTK6IJAALgbuMMd9bHNoBROReIBOobYy50Op4yiMiW4BMY0zE3rgpIiOBBcaY9/0Lk6YYY/ZbHVdFRMSOb3XuM4wxNb2xP2hE5Eh8fy8nGGMKRGQc8I0xZoS1kf1DRE4CxgJtARcwDfiPMWZDqM6pIyjAGPMtENELIhpj/jLGrPR/ngOsA460Nqp/Mz65/i8T/B8R9w5IRBoDFwDvWx1LNBOROsC5wAcAxhhXJCcnv87AxkhKTqU4gFoi4gBSgD8tjqes44Elxph8Y4wbmA9cHMoTaoKKQiLSDGgDLLE2kgP5p85WATuBmcaYiIsReBV4EPBaHUglDDBDRFaIyE1WB1OO5sAu4CP/dOn7IpJqdVCVGACMsTqIsowxfwDDgN+Bv4AsY8wMa6M6wM/AOSLSQERSgPOBJqE8oSaoKCMiacDnwN3GmGyr4ynLGOMxxmQAjYG2/mmBiCEiFwI7jTErrI4lAO2NMacCPYHb/FPRkcQBnAq8Y4xpA+QBD1kbUsX8U5C9gfFWx1KWiNQD+uBL+kcAqSJypbVR/ZsxZh0wFJiBb3pvFeAJ5Tk1QUUR/3Wdz4FPjTFfWB3PwfineuYCPayOpYx2QG//9Z2xwHki8om1IZXP/64aY8xOYCK+uf9Ish3YXmqUPAFfwopUPYGVxpi/rQ6kHF2AzcaYXcaYIuAL4GyLYzqAMeYDY8xpxphzgX3Ar6E8nyaoKOEvQPgAWGeMednqeMojIo1EpK7/81pAV2C9tVH9mzHmYWNMY2NMM3zTPXOMMRH1ThVARFL9xTD4p8264fdcV8kAAAWkSURBVJtiiRjGmB3ANhE51r+pMxAxRTvluJwInN7z+x04U0RS/H/rnfFdZ44oInKI/9+j8F1/Gh3K8zlCefBoISJjgI5AQxHZDjxhjPnA2qgO0A64CvjJf40H4BFjzDcWxlTW4cBIf6WUDRhnjInYMu4Idygw0fdahQMYbYyZZm1I5boD+NQ/fbYJuNbieMrlT/JdgZutjqU8xpglIjIBWAm4gR+IzLZHn4tIA6AIuC3URTFaZq6UUioi6RSfUkqpiKQJSimlVETSBKWUUioiaYJSSikVkTRBKaWUikiaoFRUEhEjIi+V+vp+EXkySMceISKXBuNYlZznMn8H8Llltjfzf393lNr2pogMquR4/xGRq/2fzxORzJAEHgARubtULCU/TxGp72+LdK3/vrlILJ1XEUITlIpWTuBiEWlodSCl+Rt9Bup64FZjTKdyHtsJ3OW/vyggxph3jTGjqnD+KhGfSl8z/D+D6yhzE6e/uex0YLgx5iNjzC7gLxFpF5KAVdTTBKWilRvfjYz3lH2g7AhIRHL9/3YUkfkiMk5EfhWRISIyUHxrWP0kIi1KHaaLiCzw73eh//l2EXlRRJaJyGoRubnUceeKyGhgdTnxXO4//s8iMtS/7XGgPfCuiLxYzve3C5gNXFPO8VqIyDR/E9kFInKcf/uTInJ/qV2vFJHv/Odt69+nvohM8sf/vYi0Lu+5/uc083+sE5G38d1E2sT/8/3Z/z0d8PMHzsPXUshdalsaMBXfDcfvlNo+CRhYzjGU0k4SKqq9BawWkReq8JxT8C0bsBdf54P3jTFtxbcA5B1A8SJszYAOQAtgrogcA1yNr8v06SKSBCwSkeKO022Bk4wxm0ufTESOwNdg8zR8vctmiMhFxpinReQ84H5jTEWLEQ4FporIh2W2D8e3Ds9vInIG8Da+pFBWqjHmbPE1mf0QOAl4CvjBGHOR//yjgIyD/8g4FrjWGHOriJwGHFm8bpr4W1uV0Q7femWlvYzvZ/1Kme3LgWcrOb+KUzqCUlHL3819FHBnFZ62zL+2lhPYiK8zM8BP+JJSsXHGGK8x5jd8iew4fP3wrva3mloCNABa+vdfWjY5+Z0OzPM3AXUDn+JbQymQ72+T/zxXFG8TXzf7s4Hx/jjew9diqjxj/Mf5FqjtTybtgY/92+cADUSkdiWhbC216OQm4GgReUNEegDlddQ/HN8IsLQ5QJ/iXm6l7MTXvVupA+gISkW7V/FNPX1Uapsb/5sv/zWT0tdxnKU+95b62su//x7K9gAzgAB3GGOml35ARDriW2oiFJ7H1yV8vv9rG7Dfv6RJZcr7HipS8jPzK73ceMn3ZozZJyKnAN2B24B++K43lVZQ5vng6xy/CPhGRDr5F90sPk/Bwb4JFb90BKWimjFmLzAOX8FBsS34ptTAt/5PQjUOfZmI2PzXpY4GfsF3gf8W8S17goi0ksoX6FsKdBCRhuJrons5/ySbShlj1uPrEN7L/3U2sFlELvPHIP6EUZ7+/n3a45uazAIW4L/m40+su/3H3IJ/qQwRORXfukQH8Bel2IwxnwOPUf7yGuuAY8r5Xl7Bd13ti1LFH62IsC7tKnJoglKx4CWgdDXf//AlhaXAGVRvdPMLvkQyFd/1nkJ8S8SvBVaKyM/4ptcOOgthjPkL3yJ+c4EfgRXGmC+rGMtz+BaALDYQuF5EfgTW4Fvorjz7ROQ74F3+SeBPAqeJyGpgCP8UYXwO1BeRH4BbqHidnyOBef7pxRHAw+XsM5UKpjGNMYPxrSP1sX902wn4uoJzqTin3cyVUkEnIhOBB/3X8A6237dAH2PMvvBEpqKJJiilVNCJbxHDQ/0FGhXt0whoZ4yZFL7IVDTRBKWUUioi6TUopZRSEUkTlFJKqYikCUoppVRE0gSllFIqImmCUkopFZH+H7tAKd7WYvxlAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(range(1,Ks),mean_acc,'g')\n",
    "plt.fill_between(range(1,Ks),mean_acc - 1 * std_acc,mean_acc + 1 * std_acc, alpha=0.10)\n",
    "plt.legend(('Accuracy ', '+/- 3xstd'))\n",
    "plt.ylabel('Accuracy ')\n",
    "plt.xlabel('Number of Neibours (K)')\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The best accuracy was with 0.7857142857142857 with k= 7\n"
     ]
    }
   ],
   "source": [
    "print( \"The best accuracy was with\", mean_acc.max(), \"with k=\", mean_acc.argmax()+1) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "knn_model = KNeighborsClassifier(n_neighbors = 7).fit(X_train,y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Decision Tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeClassifier(ccp_alpha=0.0, class_weight=None, criterion='entropy',\n",
       "                       max_depth=4, max_features=None, max_leaf_nodes=None,\n",
       "                       min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "                       min_samples_leaf=1, min_samples_split=2,\n",
       "                       min_weight_fraction_leaf=0.0, presort='deprecated',\n",
       "                       random_state=None, splitter='best')"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier\n",
    "tree = DecisionTreeClassifier(criterion=\"entropy\", max_depth = 4)\n",
    "tree.fit(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Support Vector Machine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=1.0, break_ties=False, cache_size=200, class_weight=None, coef0=0.0,\n",
       "    decision_function_shape='ovr', degree=3, gamma='scale', kernel='rbf',\n",
       "    max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
       "    tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn import svm\n",
    "clf = svm.SVC(kernel='rbf')\n",
    "clf.fit(X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['loan_status'].replace(to_replace=['PAIDOFF','COLLECTION'], value=[0,1],inplace=True)\n",
    "y = np.asarray(df['loan_status'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "LR = LogisticRegression(C=0.01, solver='liblinear').fit(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model Evaluation using Test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import jaccard_similarity_score\n",
    "from sklearn.metrics import f1_score\n",
    "from sklearn.metrics import log_loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, download and load the test set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2020-05-10 12:11:53--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv\n",
      "Resolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\n",
      "Connecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 3642 (3.6K) [text/csv]\n",
      "Saving to: ‘loan_test.csv’\n",
      "\n",
      "loan_test.csv       100%[===================>]   3.56K  --.-KB/s    in 0s      \n",
      "\n",
      "2020-05-10 12:11:55 (23.1 MB/s) - ‘loan_test.csv’ saved [3642/3642]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget -O loan_test.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Load Test set for evaluation "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>10/7/2016</td>\n",
       "      <td>50</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>9/9/2016</td>\n",
       "      <td>9/15/2016</td>\n",
       "      <td>35</td>\n",
       "      <td>Master or Above</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>21</td>\n",
       "      <td>21</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/10/2016</td>\n",
       "      <td>10/9/2016</td>\n",
       "      <td>43</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/10/2016</td>\n",
       "      <td>10/9/2016</td>\n",
       "      <td>26</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>9/11/2016</td>\n",
       "      <td>9/25/2016</td>\n",
       "      <td>29</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           1             1     PAIDOFF       1000     30       9/8/2016   \n",
       "1           5             5     PAIDOFF        300      7       9/9/2016   \n",
       "2          21            21     PAIDOFF       1000     30      9/10/2016   \n",
       "3          24            24     PAIDOFF       1000     30      9/10/2016   \n",
       "4          35            35     PAIDOFF        800     15      9/11/2016   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0  10/7/2016   50              Bechalor  female  \n",
       "1  9/15/2016   35       Master or Above    male  \n",
       "2  10/9/2016   43  High School or Below  female  \n",
       "3  10/9/2016   26               college    male  \n",
       "4  9/25/2016   29              Bechalor    male  "
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df = pd.read_csv('loan_test.csv')\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_df['due_date'] = pd.to_datetime(test_df['due_date'])\n",
    "test_df['effective_date'] = pd.to_datetime(test_df['effective_date'])\n",
    "test_df['dayofweek'] = test_df['effective_date'].dt.dayofweek\n",
    "test_df['weekend'] = test_df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\n",
    "test_df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\n",
    "Feature = test_df[['Principal','terms','age','Gender','weekend']]\n",
    "Feature = pd.concat([Feature,pd.get_dummies(test_df['education'])], axis=1)\n",
    "Feature.drop(['Master or Above'], axis = 1,inplace=True)\n",
    "X_test = preprocessing.StandardScaler().fit(Feature).transform(Feature)\n",
    "y_test = test_df['loan_status'].values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "jaccard index value for KNN : 0.67\n",
      "f1_score value for KNN : 0.63\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.7/dist-packages/sklearn/metrics/_classification.py:664: FutureWarning: jaccard_similarity_score has been deprecated and replaced with jaccard_score. It will be removed in version 0.23. This implementation has surprising behavior for binary and multiclass classification tasks.\n",
      "  FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "#K-NN\n",
    "knn_yhat = knn_model.predict(X_test)\n",
    "#calculating jaccard index\n",
    "print(\"jaccard index value for KNN : %.2f\" % jaccard_similarity_score(y_test, knn_yhat))\n",
    "#calculating f1-score\n",
    "print(\"f1_score value for KNN : %.2f\" % f1_score(y_test, knn_yhat, average='weighted'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "jaccard index value for DT : 0.78\n",
      "f1_score value for DT : 0.73\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.7/dist-packages/sklearn/metrics/_classification.py:664: FutureWarning: jaccard_similarity_score has been deprecated and replaced with jaccard_score. It will be removed in version 0.23. This implementation has surprising behavior for binary and multiclass classification tasks.\n",
      "  FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "#Decision Tree\n",
    "dt_yhat = tree.predict(X_test)\n",
    "#calculating jaccard index\n",
    "print(\"jaccard index value for DT : %.2f\" % jaccard_similarity_score(y_test, dt_yhat))\n",
    "print(\"f1_score value for DT : %.2f\" % f1_score(y_test, dt_yhat, average='weighted'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "jaccard index value for SVM : 0.72\n",
      "f1_score value for SVM : 0.62\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.7/dist-packages/sklearn/metrics/_classification.py:664: FutureWarning: jaccard_similarity_score has been deprecated and replaced with jaccard_score. It will be removed in version 0.23. This implementation has surprising behavior for binary and multiclass classification tasks.\n",
      "  FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "#SVM\n",
    "svm_yhat = clf.predict(X_test)\n",
    "#calculating jaccard index\n",
    "print(\"jaccard index value for SVM : %.2f\" % jaccard_similarity_score(y_test, svm_yhat))\n",
    "print(\"f1_score value for SVM : %.2f\" % f1_score(y_test, svm_yhat, average='weighted'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "jaccard index value for LR : 0.74\n",
      "f1_score value for LR : 0.63\n",
      "Log Loss value for LR : 0.56\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.7/dist-packages/sklearn/metrics/_classification.py:664: FutureWarning: jaccard_similarity_score has been deprecated and replaced with jaccard_score. It will be removed in version 0.23. This implementation has surprising behavior for binary and multiclass classification tasks.\n",
      "  FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "test_df['loan_status'].replace(to_replace=['PAIDOFF','COLLECTION'], value=[0,1],inplace=True)\n",
    "y_test = np.asarray(test_df['loan_status'])\n",
    "#Logistic Regresion\n",
    "lr_yhat = LR.predict(X_test)\n",
    "lr_yhat_prob = LR.predict_proba(X_test)\n",
    "print(\"jaccard index value for LR : %.2f\" % jaccard_similarity_score(y_test, lr_yhat))\n",
    "print(\"f1_score value for LR : %.2f\" % f1_score(y_test, lr_yhat, average='weighted'))\n",
    "print(\"Log Loss value for LR : %.2f\" % log_loss(y_test, lr_yhat_prob))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Report\n",
    "You should be able to report the accuracy of the built model using different evaluation metrics:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Algorithm          | Jaccard | F1-score | LogLoss |\n",
    "|--------------------|---------|----------|---------|\n",
    "| KNN                | 0.67    | 0.63     | NA      |\n",
    "| Decision Tree      | 0.78    | 0.73     | NA      |\n",
    "| SVM                | 0.72    | 0.62     | NA      |\n",
    "| LogisticRegression | 0.74    | 0.63     | 0.56    |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<h2>Want to learn more?</h2>\n",
    "\n",
    "IBM SPSS Modeler is a comprehensive analytics platform that has many machine learning algorithms. It has been designed to bring predictive intelligence to decisions made by individuals, by groups, by systems – by your enterprise as a whole. A free trial is available through this course, available here: <a href=\"http://cocl.us/ML0101EN-SPSSModeler\">SPSS Modeler</a>\n",
    "\n",
    "Also, you can use Watson Studio to run these notebooks faster with bigger datasets. Watson Studio is IBM's leading cloud solution for data scientists, built by data scientists. With Jupyter notebooks, RStudio, Apache Spark and popular libraries pre-packaged in the cloud, Watson Studio enables data scientists to collaborate on their projects without having to install anything. Join the fast-growing community of Watson Studio users today with a free account at <a href=\"https://cocl.us/ML0101EN_DSX\">Watson Studio</a>\n",
    "\n",
    "<h3>Thanks for completing this lesson!</h3>\n",
    "\n",
    "<h4>Author:  <a href=\"https://ca.linkedin.com/in/saeedaghabozorgi\">Saeed Aghabozorgi</a></h4>\n",
    "<p><a href=\"https://ca.linkedin.com/in/saeedaghabozorgi\">Saeed Aghabozorgi</a>, PhD is a Data Scientist in IBM with a track record of developing enterprise level applications that substantially increases clients’ ability to turn data into actionable knowledge. He is a researcher in data mining field and expert in developing advanced analytic methods like machine learning and statistical modelling on large datasets.</p>\n",
    "\n",
    "<hr>\n",
    "\n",
    "<p>Copyright &copy; 2018 <a href=\"https://cocl.us/DX0108EN_CC\">Cognitive Class</a>. This notebook and its source code are released under the terms of the <a href=\"https://bigdatauniversity.com/mit-license/\">MIT License</a>.</p>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}