model-optimization-with-random-forest.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "view-in-github"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/dphi-official/Data_Science_Bootcamp/blob/master/Week4/Model_Optimization_with_Random_Forest.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jn2WK-EnvdaQ"
   },
   "source": [
    "## Introduction\n",
    "The dataset used in this notebook is of '[IEEE-CIS Fraud Detection](https://www.kaggle.com/c/ieee-fraud-detection/data)'. This notebook will introduce you to class imbalance problem.\n",
    "\n",
    "Data set link: [Fraud Dataset](https://drive.google.com/file/d/1q8SYcjOJULdSkETv5S_gd7xNq1GrBHAO/view)\n",
    "\n",
    "#### Imbalanced Problem\n",
    "Imbalanced classes are a common problem in machine learning classification where there are a disproportionate ratio of observations in each class. Class imbalance can be found in many different areas including medical diagnosis, spam filtering, and fraud detection.\n",
    "\n",
    "#### Agenda\n",
    "*  Loading Libraries\n",
    "*  Loading Data\n",
    "*  Getting Basic Idea About Data\n",
    "*  Missing Values and Dealing with Missing Values\n",
    "*  One Hot Encoding (Creating dummies for categorical columns)\n",
    "*  Standardization / Normalization\n",
    "*  Splitting the dataset into train and test data\n",
    "*  Dealing with Imbalanced Data\n",
    "    *  Generate Synthetic Samples\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "89IQeG9VuUpM"
   },
   "source": [
    "## Loading Libraries\n",
    "All Python capabilities are not loaded to our working environment by default (even they are already installed in your system). So, we import each and every library that we want to use.\n",
    "\n",
    "In data science, numpy and pandas are most commonly used libraries. Numpy is required for calculations like means, medians, square roots, etc. Pandas is used for data processin and data frames. We chose alias names for our libraries for the sake of our convenience (numpy --> np and pandas --> pd)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 71
    },
    "id": "L_lWWBjGvNP1",
    "outputId": "39b19f43-5dae-48a0-eaa6-a16e4ef04af8"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.6/dist-packages/statsmodels/tools/_testing.py:19: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.\n",
      "  import pandas.util.testing as tm\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd                  # A fundamental package for linear algebra and multidimensional arrays\n",
    "import numpy as np                   # Data analysis and data manipulating tool\n",
    "import random                        # Library to generate random numbers\n",
    "from collections import Counter      # Collection is a Python module that implements specialized container datatypes providing \n",
    "                                     # alternatives to Python’s general purpose built-in containers, dict, list, set, and tuple.\n",
    "                                     # Counter is a dict subclass for counting hashable objects\n",
    "# Visualization libraries\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "# To ignore warnings in the notebook\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2jIwY1OGvr7A"
   },
   "source": [
    "## Loading Data\n",
    "Pandas module is used for reading files. We have our data in '.csv' format. We will use 'read_csv()' function for loading the data.\n",
    "\n",
    "**Disclaimer:** Loading fraud data might take time due to the nature of its size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "G_bYGjBvvkVv"
   },
   "outputs": [],
   "source": [
    "fraud_data = pd.read_csv(\"https://raw.githubusercontent.com/dphi-official/Imbalanced_classes/master/fraud_data.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kvyUn8YG8wyC"
   },
   "source": [
    "### Getting Basic Idea About Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 321
    },
    "id": "8CmggKjE8ZUk",
    "outputId": "adbfe24d-340f-4a0c-b756-5a5fb350fb9f"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>TransactionID</th>\n",
       "      <th>isFraud</th>\n",
       "      <th>TransactionDT</th>\n",
       "      <th>TransactionAmt</th>\n",
       "      <th>ProductCD</th>\n",
       "      <th>card1</th>\n",
       "      <th>card2</th>\n",
       "      <th>card3</th>\n",
       "      <th>card4</th>\n",
       "      <th>card5</th>\n",
       "      <th>card6</th>\n",
       "      <th>addr1</th>\n",
       "      <th>addr2</th>\n",
       "      <th>dist1</th>\n",
       "      <th>dist2</th>\n",
       "      <th>P_emaildomain</th>\n",
       "      <th>R_emaildomain</th>\n",
       "      <th>C1</th>\n",
       "      <th>C2</th>\n",
       "      <th>C3</th>\n",
       "      <th>C4</th>\n",
       "      <th>C5</th>\n",
       "      <th>C6</th>\n",
       "      <th>C7</th>\n",
       "      <th>C8</th>\n",
       "      <th>C9</th>\n",
       "      <th>C10</th>\n",
       "      <th>C11</th>\n",
       "      <th>C12</th>\n",
       "      <th>C13</th>\n",
       "      <th>C14</th>\n",
       "      <th>D1</th>\n",
       "      <th>D2</th>\n",
       "      <th>D3</th>\n",
       "      <th>D4</th>\n",
       "      <th>D5</th>\n",
       "      <th>D6</th>\n",
       "      <th>D7</th>\n",
       "      <th>D8</th>\n",
       "      <th>D9</th>\n",
       "      <th>...</th>\n",
       "      <th>id_01</th>\n",
       "      <th>id_02</th>\n",
       "      <th>id_03</th>\n",
       "      <th>id_04</th>\n",
       "      <th>id_05</th>\n",
       "      <th>id_06</th>\n",
       "      <th>id_07</th>\n",
       "      <th>id_08</th>\n",
       "      <th>id_09</th>\n",
       "      <th>id_10</th>\n",
       "      <th>id_11</th>\n",
       "      <th>id_12</th>\n",
       "      <th>id_13</th>\n",
       "      <th>id_14</th>\n",
       "      <th>id_15</th>\n",
       "      <th>id_16</th>\n",
       "      <th>id_17</th>\n",
       "      <th>id_18</th>\n",
       "      <th>id_19</th>\n",
       "      <th>id_20</th>\n",
       "      <th>id_21</th>\n",
       "      <th>id_22</th>\n",
       "      <th>id_23</th>\n",
       "      <th>id_24</th>\n",
       "      <th>id_25</th>\n",
       "      <th>id_26</th>\n",
       "      <th>id_27</th>\n",
       "      <th>id_28</th>\n",
       "      <th>id_29</th>\n",
       "      <th>id_30</th>\n",
       "      <th>id_31</th>\n",
       "      <th>id_32</th>\n",
       "      <th>id_33</th>\n",
       "      <th>id_34</th>\n",
       "      <th>id_35</th>\n",
       "      <th>id_36</th>\n",
       "      <th>id_37</th>\n",
       "      <th>id_38</th>\n",
       "      <th>DeviceType</th>\n",
       "      <th>DeviceInfo</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2994681</td>\n",
       "      <td>0</td>\n",
       "      <td>242834</td>\n",
       "      <td>25.000</td>\n",
       "      <td>H</td>\n",
       "      <td>9803</td>\n",
       "      <td>583.0</td>\n",
       "      <td>150.0</td>\n",
       "      <td>visa</td>\n",
       "      <td>226.0</td>\n",
       "      <td>credit</td>\n",
       "      <td>269.0</td>\n",
       "      <td>87.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>yahoo.com</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.791666</td>\n",
       "      <td>0.791666</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>82050.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>100.0</td>\n",
       "      <td>NotFound</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-300.0</td>\n",
       "      <td>Found</td>\n",
       "      <td>Found</td>\n",
       "      <td>166.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>529.0</td>\n",
       "      <td>178.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Found</td>\n",
       "      <td>Found</td>\n",
       "      <td>Mac OS X 10.12</td>\n",
       "      <td>firefox 56.0</td>\n",
       "      <td>24.0</td>\n",
       "      <td>1920x1080</td>\n",
       "      <td>match_status:2</td>\n",
       "      <td>T</td>\n",
       "      <td>F</td>\n",
       "      <td>T</td>\n",
       "      <td>T</td>\n",
       "      <td>desktop</td>\n",
       "      <td>rv:56.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3557242</td>\n",
       "      <td>0</td>\n",
       "      <td>15123000</td>\n",
       "      <td>117.000</td>\n",
       "      <td>W</td>\n",
       "      <td>7919</td>\n",
       "      <td>194.0</td>\n",
       "      <td>150.0</td>\n",
       "      <td>mastercard</td>\n",
       "      <td>166.0</td>\n",
       "      <td>debit</td>\n",
       "      <td>181.0</td>\n",
       "      <td>87.0</td>\n",
       "      <td>483.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>gmail.com</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>63.0</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3327470</td>\n",
       "      <td>0</td>\n",
       "      <td>8378575</td>\n",
       "      <td>73.773</td>\n",
       "      <td>C</td>\n",
       "      <td>12778</td>\n",
       "      <td>500.0</td>\n",
       "      <td>185.0</td>\n",
       "      <td>mastercard</td>\n",
       "      <td>224.0</td>\n",
       "      <td>credit</td>\n",
       "      <td>284.0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>hotmail.com</td>\n",
       "      <td>hotmail.com</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>54.0</td>\n",
       "      <td>54.0</td>\n",
       "      <td>35.0</td>\n",
       "      <td>54.0</td>\n",
       "      <td>35.0</td>\n",
       "      <td>54.0</td>\n",
       "      <td>35.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3118781</td>\n",
       "      <td>0</td>\n",
       "      <td>2607840</td>\n",
       "      <td>400.000</td>\n",
       "      <td>R</td>\n",
       "      <td>12316</td>\n",
       "      <td>548.0</td>\n",
       "      <td>150.0</td>\n",
       "      <td>visa</td>\n",
       "      <td>195.0</td>\n",
       "      <td>credit</td>\n",
       "      <td>441.0</td>\n",
       "      <td>87.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>371.0</td>\n",
       "      <td>anonymous.com</td>\n",
       "      <td>anonymous.com</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>-5.0</td>\n",
       "      <td>183599.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.0</td>\n",
       "      <td>-11.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>100.0</td>\n",
       "      <td>NotFound</td>\n",
       "      <td>49.0</td>\n",
       "      <td>-300.0</td>\n",
       "      <td>New</td>\n",
       "      <td>NotFound</td>\n",
       "      <td>166.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>352.0</td>\n",
       "      <td>280.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>New</td>\n",
       "      <td>NotFound</td>\n",
       "      <td>iOS 10.3.3</td>\n",
       "      <td>mobile safari generic</td>\n",
       "      <td>32.0</td>\n",
       "      <td>1136x640</td>\n",
       "      <td>match_status:2</td>\n",
       "      <td>T</td>\n",
       "      <td>F</td>\n",
       "      <td>T</td>\n",
       "      <td>F</td>\n",
       "      <td>mobile</td>\n",
       "      <td>iOS Device</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3459772</td>\n",
       "      <td>0</td>\n",
       "      <td>12226544</td>\n",
       "      <td>31.950</td>\n",
       "      <td>W</td>\n",
       "      <td>9002</td>\n",
       "      <td>453.0</td>\n",
       "      <td>150.0</td>\n",
       "      <td>visa</td>\n",
       "      <td>226.0</td>\n",
       "      <td>debit</td>\n",
       "      <td>264.0</td>\n",
       "      <td>87.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>yahoo.com</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>33</td>\n",
       "      <td>3</td>\n",
       "      <td>54.0</td>\n",
       "      <td>54.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>405.0</td>\n",
       "      <td>32.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 434 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   TransactionID  isFraud  TransactionDT  ...  id_38 DeviceType  DeviceInfo\n",
       "0        2994681        0         242834  ...      T    desktop     rv:56.0\n",
       "1        3557242        0       15123000  ...    NaN        NaN         NaN\n",
       "2        3327470        0        8378575  ...    NaN        NaN         NaN\n",
       "3        3118781        0        2607840  ...      F     mobile  iOS Device\n",
       "4        3459772        0       12226544  ...    NaN        NaN         NaN\n",
       "\n",
       "[5 rows x 434 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fraud_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 102
    },
    "id": "jwNQSckh83QZ",
    "outputId": "ea49d8b3-fff8-487c-e97f-7cda744f10da"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 59054 entries, 0 to 59053\n",
      "Columns: 434 entries, TransactionID to DeviceInfo\n",
      "dtypes: float64(385), int64(18), object(31)\n",
      "memory usage: 195.5+ MB\n"
     ]
    }
   ],
   "source": [
    "fraud_data.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6FjL68CPABH7"
   },
   "source": [
    "There are 434 columns with 59054 observations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 346
    },
    "id": "1mqIdBU-M_mT",
    "outputId": "d92a24e6-b367-4775-eb34-e3f7c253f2b3"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>TransactionID</th>\n",
       "      <th>isFraud</th>\n",
       "      <th>TransactionDT</th>\n",
       "      <th>TransactionAmt</th>\n",
       "      <th>card1</th>\n",
       "      <th>card2</th>\n",
       "      <th>card3</th>\n",
       "      <th>card5</th>\n",
       "      <th>addr1</th>\n",
       "      <th>addr2</th>\n",
       "      <th>dist1</th>\n",
       "      <th>dist2</th>\n",
       "      <th>C1</th>\n",
       "      <th>C2</th>\n",
       "      <th>C3</th>\n",
       "      <th>C4</th>\n",
       "      <th>C5</th>\n",
       "      <th>C6</th>\n",
       "      <th>C7</th>\n",
       "      <th>C8</th>\n",
       "      <th>C9</th>\n",
       "      <th>C10</th>\n",
       "      <th>C11</th>\n",
       "      <th>C12</th>\n",
       "      <th>C13</th>\n",
       "      <th>C14</th>\n",
       "      <th>D1</th>\n",
       "      <th>D2</th>\n",
       "      <th>D3</th>\n",
       "      <th>D4</th>\n",
       "      <th>D5</th>\n",
       "      <th>D6</th>\n",
       "      <th>D7</th>\n",
       "      <th>D8</th>\n",
       "      <th>D9</th>\n",
       "      <th>D10</th>\n",
       "      <th>D11</th>\n",
       "      <th>D12</th>\n",
       "      <th>D13</th>\n",
       "      <th>D14</th>\n",
       "      <th>...</th>\n",
       "      <th>V323</th>\n",
       "      <th>V324</th>\n",
       "      <th>V325</th>\n",
       "      <th>V326</th>\n",
       "      <th>V327</th>\n",
       "      <th>V328</th>\n",
       "      <th>V329</th>\n",
       "      <th>V330</th>\n",
       "      <th>V331</th>\n",
       "      <th>V332</th>\n",
       "      <th>V333</th>\n",
       "      <th>V334</th>\n",
       "      <th>V335</th>\n",
       "      <th>V336</th>\n",
       "      <th>V337</th>\n",
       "      <th>V338</th>\n",
       "      <th>V339</th>\n",
       "      <th>id_01</th>\n",
       "      <th>id_02</th>\n",
       "      <th>id_03</th>\n",
       "      <th>id_04</th>\n",
       "      <th>id_05</th>\n",
       "      <th>id_06</th>\n",
       "      <th>id_07</th>\n",
       "      <th>id_08</th>\n",
       "      <th>id_09</th>\n",
       "      <th>id_10</th>\n",
       "      <th>id_11</th>\n",
       "      <th>id_13</th>\n",
       "      <th>id_14</th>\n",
       "      <th>id_17</th>\n",
       "      <th>id_18</th>\n",
       "      <th>id_19</th>\n",
       "      <th>id_20</th>\n",
       "      <th>id_21</th>\n",
       "      <th>id_22</th>\n",
       "      <th>id_24</th>\n",
       "      <th>id_25</th>\n",
       "      <th>id_26</th>\n",
       "      <th>id_32</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>5.905400e+04</td>\n",
       "      <td>59054.000000</td>\n",
       "      <td>5.905400e+04</td>\n",
       "      <td>59054.000000</td>\n",
       "      <td>59054.000000</td>\n",
       "      <td>58139.000000</td>\n",
       "      <td>58896.000000</td>\n",
       "      <td>58610.000000</td>\n",
       "      <td>52326.000000</td>\n",
       "      <td>52326.000000</td>\n",
       "      <td>23701.000000</td>\n",
       "      <td>3872.000000</td>\n",
       "      <td>59054.000000</td>\n",
       "      <td>59054.000000</td>\n",
       "      <td>59054.000000</td>\n",
       "      <td>59054.000000</td>\n",
       "      <td>59054.000000</td>\n",
       "      <td>59054.000000</td>\n",
       "      <td>59054.000000</td>\n",
       "      <td>59054.000000</td>\n",
       "      <td>59054.000000</td>\n",
       "      <td>59054.000000</td>\n",
       "      <td>59054.000000</td>\n",
       "      <td>59054.000000</td>\n",
       "      <td>59054.000000</td>\n",
       "      <td>59054.000000</td>\n",
       "      <td>58933.000000</td>\n",
       "      <td>31100.000000</td>\n",
       "      <td>32839.000000</td>\n",
       "      <td>42208.000000</td>\n",
       "      <td>28075.000000</td>\n",
       "      <td>7421.000000</td>\n",
       "      <td>3947.000000</td>\n",
       "      <td>7553.000000</td>\n",
       "      <td>7553.000000</td>\n",
       "      <td>51514.000000</td>\n",
       "      <td>30902.000000</td>\n",
       "      <td>6580.000000</td>\n",
       "      <td>6324.000000</td>\n",
       "      <td>6317.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>8211.000000</td>\n",
       "      <td>8211.000000</td>\n",
       "      <td>8211.000000</td>\n",
       "      <td>8211.000000</td>\n",
       "      <td>8211.000000</td>\n",
       "      <td>8211.000000</td>\n",
       "      <td>8211.000000</td>\n",
       "      <td>8211.000000</td>\n",
       "      <td>8211.000000</td>\n",
       "      <td>8211.000000</td>\n",
       "      <td>8211.000000</td>\n",
       "      <td>8211.000000</td>\n",
       "      <td>8211.000000</td>\n",
       "      <td>8211.000000</td>\n",
       "      <td>8211.000000</td>\n",
       "      <td>8211.000000</td>\n",
       "      <td>8211.000000</td>\n",
       "      <td>14555.000000</td>\n",
       "      <td>14194.000000</td>\n",
       "      <td>6697.000000</td>\n",
       "      <td>6697.000000</td>\n",
       "      <td>13820.000000</td>\n",
       "      <td>13820.000000</td>\n",
       "      <td>525.000000</td>\n",
       "      <td>525.000000</td>\n",
       "      <td>7553.000000</td>\n",
       "      <td>7553.000000</td>\n",
       "      <td>14205.000000</td>\n",
       "      <td>12887.000000</td>\n",
       "      <td>7963.000000</td>\n",
       "      <td>14061.00000</td>\n",
       "      <td>4511.000000</td>\n",
       "      <td>14059.000000</td>\n",
       "      <td>14054.000000</td>\n",
       "      <td>525.000000</td>\n",
       "      <td>525.000000</td>\n",
       "      <td>487.000000</td>\n",
       "      <td>524.000000</td>\n",
       "      <td>525.000000</td>\n",
       "      <td>7715.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>3.282166e+06</td>\n",
       "      <td>0.033952</td>\n",
       "      <td>7.368220e+06</td>\n",
       "      <td>134.142888</td>\n",
       "      <td>9910.366360</td>\n",
       "      <td>362.438054</td>\n",
       "      <td>153.264551</td>\n",
       "      <td>199.104231</td>\n",
       "      <td>290.653939</td>\n",
       "      <td>86.806616</td>\n",
       "      <td>115.655837</td>\n",
       "      <td>215.529442</td>\n",
       "      <td>13.213212</td>\n",
       "      <td>14.233447</td>\n",
       "      <td>0.005571</td>\n",
       "      <td>3.673773</td>\n",
       "      <td>5.480120</td>\n",
       "      <td>8.574305</td>\n",
       "      <td>2.476784</td>\n",
       "      <td>4.551275</td>\n",
       "      <td>4.422173</td>\n",
       "      <td>4.642226</td>\n",
       "      <td>9.597893</td>\n",
       "      <td>3.551428</td>\n",
       "      <td>31.658753</td>\n",
       "      <td>7.955397</td>\n",
       "      <td>94.243548</td>\n",
       "      <td>168.784855</td>\n",
       "      <td>28.215689</td>\n",
       "      <td>139.992561</td>\n",
       "      <td>42.073375</td>\n",
       "      <td>70.105107</td>\n",
       "      <td>41.393970</td>\n",
       "      <td>145.864171</td>\n",
       "      <td>0.552242</td>\n",
       "      <td>124.267325</td>\n",
       "      <td>147.232800</td>\n",
       "      <td>54.019149</td>\n",
       "      <td>18.445130</td>\n",
       "      <td>57.257084</td>\n",
       "      <td>...</td>\n",
       "      <td>14.844477</td>\n",
       "      <td>10.334673</td>\n",
       "      <td>0.061259</td>\n",
       "      <td>0.891609</td>\n",
       "      <td>0.318232</td>\n",
       "      <td>0.367556</td>\n",
       "      <td>1.470710</td>\n",
       "      <td>0.857752</td>\n",
       "      <td>849.716295</td>\n",
       "      <td>1577.078680</td>\n",
       "      <td>1152.623919</td>\n",
       "      <td>12.852454</td>\n",
       "      <td>67.041535</td>\n",
       "      <td>34.014493</td>\n",
       "      <td>59.909633</td>\n",
       "      <td>171.761661</td>\n",
       "      <td>111.076848</td>\n",
       "      <td>-10.241498</td>\n",
       "      <td>174517.019938</td>\n",
       "      <td>0.048828</td>\n",
       "      <td>-0.054950</td>\n",
       "      <td>1.564110</td>\n",
       "      <td>-6.731766</td>\n",
       "      <td>13.460952</td>\n",
       "      <td>-38.022857</td>\n",
       "      <td>0.084337</td>\n",
       "      <td>-0.274196</td>\n",
       "      <td>99.753982</td>\n",
       "      <td>47.978273</td>\n",
       "      <td>-345.713927</td>\n",
       "      <td>189.89958</td>\n",
       "      <td>14.238750</td>\n",
       "      <td>351.767622</td>\n",
       "      <td>404.594777</td>\n",
       "      <td>385.257143</td>\n",
       "      <td>15.748571</td>\n",
       "      <td>12.739220</td>\n",
       "      <td>326.225191</td>\n",
       "      <td>148.794286</td>\n",
       "      <td>26.499028</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1.702573e+05</td>\n",
       "      <td>0.181107</td>\n",
       "      <td>4.612063e+06</td>\n",
       "      <td>233.112295</td>\n",
       "      <td>4893.704524</td>\n",
       "      <td>157.360648</td>\n",
       "      <td>11.395609</td>\n",
       "      <td>41.296438</td>\n",
       "      <td>101.796538</td>\n",
       "      <td>2.639572</td>\n",
       "      <td>366.462377</td>\n",
       "      <td>506.824710</td>\n",
       "      <td>123.135469</td>\n",
       "      <td>142.344325</td>\n",
       "      <td>0.163114</td>\n",
       "      <td>63.702441</td>\n",
       "      <td>25.570422</td>\n",
       "      <td>66.567581</td>\n",
       "      <td>56.502781</td>\n",
       "      <td>87.486345</td>\n",
       "      <td>16.593728</td>\n",
       "      <td>87.825362</td>\n",
       "      <td>87.212257</td>\n",
       "      <td>79.301070</td>\n",
       "      <td>124.218136</td>\n",
       "      <td>46.514295</td>\n",
       "      <td>157.679479</td>\n",
       "      <td>177.314255</td>\n",
       "      <td>62.528194</td>\n",
       "      <td>190.990250</td>\n",
       "      <td>88.257774</td>\n",
       "      <td>142.669131</td>\n",
       "      <td>98.446942</td>\n",
       "      <td>233.615671</td>\n",
       "      <td>0.319701</td>\n",
       "      <td>183.034325</td>\n",
       "      <td>185.927214</td>\n",
       "      <td>123.116200</td>\n",
       "      <td>69.076225</td>\n",
       "      <td>134.707158</td>\n",
       "      <td>...</td>\n",
       "      <td>113.784292</td>\n",
       "      <td>78.037190</td>\n",
       "      <td>0.312194</td>\n",
       "      <td>4.115381</td>\n",
       "      <td>1.434882</td>\n",
       "      <td>1.684518</td>\n",
       "      <td>9.305046</td>\n",
       "      <td>4.987901</td>\n",
       "      <td>6698.443610</td>\n",
       "      <td>11954.433804</td>\n",
       "      <td>8490.237746</td>\n",
       "      <td>366.868543</td>\n",
       "      <td>566.486726</td>\n",
       "      <td>390.038930</td>\n",
       "      <td>657.909569</td>\n",
       "      <td>1144.071698</td>\n",
       "      <td>823.586157</td>\n",
       "      <td>14.412998</td>\n",
       "      <td>158198.544249</td>\n",
       "      <td>0.607971</td>\n",
       "      <td>0.659661</td>\n",
       "      <td>5.071657</td>\n",
       "      <td>16.586186</td>\n",
       "      <td>11.546822</td>\n",
       "      <td>24.980822</td>\n",
       "      <td>0.932670</td>\n",
       "      <td>2.668107</td>\n",
       "      <td>1.106252</td>\n",
       "      <td>11.914148</td>\n",
       "      <td>91.661528</td>\n",
       "      <td>30.34787</td>\n",
       "      <td>1.524658</td>\n",
       "      <td>141.600677</td>\n",
       "      <td>152.201538</td>\n",
       "      <td>213.565534</td>\n",
       "      <td>6.496154</td>\n",
       "      <td>2.275238</td>\n",
       "      <td>97.662855</td>\n",
       "      <td>31.168092</td>\n",
       "      <td>3.739140</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>2.987019e+06</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>8.673000e+04</td>\n",
       "      <td>0.292000</td>\n",
       "      <td>1008.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>13.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-90.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-33.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-100.000000</td>\n",
       "      <td>43.000000</td>\n",
       "      <td>-12.000000</td>\n",
       "      <td>-12.000000</td>\n",
       "      <td>-32.000000</td>\n",
       "      <td>-100.000000</td>\n",
       "      <td>-31.000000</td>\n",
       "      <td>-100.000000</td>\n",
       "      <td>-26.000000</td>\n",
       "      <td>-100.000000</td>\n",
       "      <td>90.000000</td>\n",
       "      <td>11.000000</td>\n",
       "      <td>-660.000000</td>\n",
       "      <td>100.00000</td>\n",
       "      <td>11.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>114.000000</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>11.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>3.135748e+06</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.074217e+06</td>\n",
       "      <td>42.950000</td>\n",
       "      <td>6019.000000</td>\n",
       "      <td>215.000000</td>\n",
       "      <td>150.000000</td>\n",
       "      <td>166.000000</td>\n",
       "      <td>204.000000</td>\n",
       "      <td>87.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>25.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.958333</td>\n",
       "      <td>0.208333</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-10.000000</td>\n",
       "      <td>67806.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-6.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>-48.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>49.000000</td>\n",
       "      <td>-360.000000</td>\n",
       "      <td>166.00000</td>\n",
       "      <td>13.000000</td>\n",
       "      <td>266.000000</td>\n",
       "      <td>256.000000</td>\n",
       "      <td>252.000000</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>11.000000</td>\n",
       "      <td>321.000000</td>\n",
       "      <td>119.000000</td>\n",
       "      <td>24.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>3.282062e+06</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>7.288450e+06</td>\n",
       "      <td>68.017000</td>\n",
       "      <td>9749.000000</td>\n",
       "      <td>361.000000</td>\n",
       "      <td>150.000000</td>\n",
       "      <td>226.000000</td>\n",
       "      <td>299.000000</td>\n",
       "      <td>87.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>96.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>25.000000</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>36.875000</td>\n",
       "      <td>0.666666</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>44.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-5.000000</td>\n",
       "      <td>126886.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>-34.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>52.000000</td>\n",
       "      <td>-300.000000</td>\n",
       "      <td>166.00000</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>339.000000</td>\n",
       "      <td>484.000000</td>\n",
       "      <td>252.000000</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>11.000000</td>\n",
       "      <td>321.000000</td>\n",
       "      <td>147.000000</td>\n",
       "      <td>24.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>3.429699e+06</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.123918e+07</td>\n",
       "      <td>117.000000</td>\n",
       "      <td>14223.000000</td>\n",
       "      <td>512.000000</td>\n",
       "      <td>150.000000</td>\n",
       "      <td>226.000000</td>\n",
       "      <td>330.000000</td>\n",
       "      <td>87.000000</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>199.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>13.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>121.000000</td>\n",
       "      <td>273.000000</td>\n",
       "      <td>27.000000</td>\n",
       "      <td>255.000000</td>\n",
       "      <td>32.000000</td>\n",
       "      <td>44.000000</td>\n",
       "      <td>17.000000</td>\n",
       "      <td>183.791672</td>\n",
       "      <td>0.833333</td>\n",
       "      <td>198.000000</td>\n",
       "      <td>275.000000</td>\n",
       "      <td>13.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>25.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-5.000000</td>\n",
       "      <td>228324.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>-23.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>52.000000</td>\n",
       "      <td>-300.000000</td>\n",
       "      <td>225.00000</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>427.000000</td>\n",
       "      <td>533.000000</td>\n",
       "      <td>554.000000</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>361.000000</td>\n",
       "      <td>169.000000</td>\n",
       "      <td>32.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>3.577536e+06</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.581105e+07</td>\n",
       "      <td>5279.950000</td>\n",
       "      <td>18390.000000</td>\n",
       "      <td>600.000000</td>\n",
       "      <td>229.000000</td>\n",
       "      <td>237.000000</td>\n",
       "      <td>536.000000</td>\n",
       "      <td>102.000000</td>\n",
       "      <td>6818.000000</td>\n",
       "      <td>7104.000000</td>\n",
       "      <td>4669.000000</td>\n",
       "      <td>5624.000000</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>2241.000000</td>\n",
       "      <td>349.000000</td>\n",
       "      <td>2241.000000</td>\n",
       "      <td>2243.000000</td>\n",
       "      <td>3318.000000</td>\n",
       "      <td>205.000000</td>\n",
       "      <td>3245.000000</td>\n",
       "      <td>3171.000000</td>\n",
       "      <td>3171.000000</td>\n",
       "      <td>2904.000000</td>\n",
       "      <td>1418.000000</td>\n",
       "      <td>640.000000</td>\n",
       "      <td>640.000000</td>\n",
       "      <td>819.000000</td>\n",
       "      <td>869.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>849.000000</td>\n",
       "      <td>730.000000</td>\n",
       "      <td>1376.541626</td>\n",
       "      <td>0.958333</td>\n",
       "      <td>869.000000</td>\n",
       "      <td>664.000000</td>\n",
       "      <td>632.000000</td>\n",
       "      <td>820.000000</td>\n",
       "      <td>869.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1400.000000</td>\n",
       "      <td>971.000000</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>43.000000</td>\n",
       "      <td>16.000000</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>88.000000</td>\n",
       "      <td>53.000000</td>\n",
       "      <td>108800.000000</td>\n",
       "      <td>145167.000000</td>\n",
       "      <td>108800.000000</td>\n",
       "      <td>30250.000000</td>\n",
       "      <td>41690.000000</td>\n",
       "      <td>30250.000000</td>\n",
       "      <td>51200.000000</td>\n",
       "      <td>51200.000000</td>\n",
       "      <td>51200.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>991867.000000</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>52.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>61.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>17.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>64.000000</td>\n",
       "      <td>600.000000</td>\n",
       "      <td>225.00000</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>670.000000</td>\n",
       "      <td>660.000000</td>\n",
       "      <td>854.000000</td>\n",
       "      <td>43.000000</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>548.000000</td>\n",
       "      <td>216.000000</td>\n",
       "      <td>32.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 403 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       TransactionID       isFraud  ...       id_26        id_32\n",
       "count   5.905400e+04  59054.000000  ...  525.000000  7715.000000\n",
       "mean    3.282166e+06      0.033952  ...  148.794286    26.499028\n",
       "std     1.702573e+05      0.181107  ...   31.168092     3.739140\n",
       "min     2.987019e+06      0.000000  ...  100.000000     0.000000\n",
       "25%     3.135748e+06      0.000000  ...  119.000000    24.000000\n",
       "50%     3.282062e+06      0.000000  ...  147.000000    24.000000\n",
       "75%     3.429699e+06      0.000000  ...  169.000000    32.000000\n",
       "max     3.577536e+06      1.000000  ...  216.000000    32.000000\n",
       "\n",
       "[8 rows x 403 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fraud_data.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 68
    },
    "id": "GcnguA4n9IsZ",
    "outputId": "6c26e9c0-f80f-4837-fc12-8ce52005130d"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    57049\n",
       "1     2005\n",
       "Name: isFraud, dtype: int64"
      ]
     },
     "execution_count": 6,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Taking a look at the target variable\n",
    "fraud_data.isFraud.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1L-PiTGd9d8b"
   },
   "source": [
    "We can notice, of 57049 observations / records only 2005 were fraud transactions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 68
    },
    "id": "92IJJSUf9cgq",
    "outputId": "d5fb80a7-b0fa-4758-9e54-991602d400bb"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    0.966048\n",
       "1    0.033952\n",
       "Name: isFraud, dtype: float64"
      ]
     },
     "execution_count": 7,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fraud_data.isFraud.value_counts(normalize = True)      # Normalize = True will find the proportion of fraud transaction and not fraud transaction "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 296
    },
    "id": "sOl5jgYE9wLg",
    "outputId": "d8074c55-e2cd-476c-9548-9e9ab39cf8b3"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7f464636ec18>"
      ]
     },
     "execution_count": 8,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAEGCAYAAACkQqisAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAARvklEQVR4nO3de7Bd5V3G8e/ThApaESgRaYINY+MF0VaIlHobbUcI9UJHaaVeiJVpHEuddnRaqX+IUpmxY7WWXnDSkpJ0tClakVipMVIq07HQHCxytcMRQZKBJm24lGFoG/z5x34P2YaTsPMme58czvczs2av9Xvftfa7ZpJ5Zq317nVSVUiS1ON5cz0ASdL8ZYhIkroZIpKkboaIJKmbISJJ6rZ4rgcwaccff3wtX758rochSfPGLbfc8uWqWjJb24ILkeXLlzM1NTXXw5CkeSPJ/ftq83aWJKmbISJJ6maISJK6GSKSpG6GiCSpmyEiSepmiEiSuhkikqRuhogkqduC+8X6wTr9bRvmegg6DN3ypxfM9RCkOeGViCSpmyEiSepmiEiSuhkikqRuhogkqZshIknqZohIkroZIpKkboaIJKmbISJJ6maISJK6GSKSpG6GiCSpmyEiSepmiEiSuhkikqRuhogkqZshIknqZohIkroZIpKkboaIJKnbWEMkyX1Jbk9ya5KpVjsuyZYk97TPY1s9SS5PMp3ktiSnDR1ndet/T5LVQ/XT2/Gn274Z5/lIkv6/SVyJ/FRVvayqVrbti4Hrq2oFcH3bBjgHWNGWNcAVMAgd4BLg5cAZwCUzwdP6vHFov1XjPx1J0oy5uJ11LrC+ra8HXjNU31ADNwHHJDkROBvYUlW7quphYAuwqrUdXVU3VVUBG4aOJUmagHGHSAH/nOSWJGta7YSqerCtPwSc0NaXAg8M7but1fZX3zZL/RmSrEkylWRq586dB3M+kqQhi8d8/B+rqu1Jvh3YkuQ/hxurqpLUmMdAVa0F1gKsXLly7N8nSQvFWK9Eqmp7+9wBXMPgmcaX2q0o2ueO1n07cNLQ7stabX/1ZbPUJUkTMrYQSfItSb51Zh04C7gD2ATMzLBaDVzb1jcBF7RZWmcCj7bbXpuBs5Ic2x6onwVsbm2PJTmzzcq6YOhYkqQJGOftrBOAa9qs28XAX1fVPyXZClyd5ELgfuB1rf91wKuBaeAJ4A0AVbUryTuBra3fpVW1q62/CbgKOAr4VFskSRMythCpqnuBl85S/wrwqlnqBVy0j2OtA9bNUp8CTj3owUqSuviLdUlSN0NEktTNEJEkdTNEJEndDBFJUjdDRJLUzRCRJHUzRCRJ3QwRSVI3Q0SS1M0QkSR1M0QkSd0MEUlSN0NEktTNEJEkdTNEJEndDBFJUjdDRJLUzRCRJHUzRCRJ3QwRSVI3Q0SS1M0QkSR1M0QkSd0MEUlSN0NEktTNEJEkdRt7iCRZlOQLST7Ztk9OcnOS6SQfT/L8Vv+mtj3d2pcPHeMdrf7FJGcP1Ve12nSSi8d9LpKk/28SVyJvAe4e2n4X8J6qegnwMHBhq18IPNzq72n9SHIKcD7w/cAq4IMtmBYBHwDOAU4BXt/6SpImZKwhkmQZ8DPAh9t2gFcCf9u6rAde09bPbdu09le1/ucCG6vqa1X138A0cEZbpqvq3qr6OrCx9ZUkTci4r0T+Ang78L9t+4XAI1W1u21vA5a29aXAAwCt/dHW/+n6Xvvsqy5JmpCxhUiSnwV2VNUt4/qOAxjLmiRTSaZ27tw518ORpOeMcV6J/Cjw80nuY3Cr6ZXAe4FjkixufZYB29v6duAkgNb+bcBXhut77bOv+jNU1dqqWllVK5csWXLwZyZJAsYYIlX1jqpaVlXLGTwY/3RV/QpwA3Be67YauLatb2rbtPZPV1W1+vlt9tbJwArg88BWYEWb7fX89h2bxnU+kqRnWvzsXQ653wM2Jvlj4AvAla1+JfDRJNPALgahQFXdmeRq4C5gN3BRVT0FkOTNwGZgEbCuqu6c6JlI0gI3kRCpqs8An2nr9zKYWbV3nyeB1+5j/8uAy2apXwdcdwiHKkk6AP5iXZLUzRCRJHUzRCRJ3QwRSVI3Q0SS1M0QkSR1M0QkSd0MEUlSN0NEktTNEJEkdTNEJEndDBFJUjdDRJLUzRCRJHUzRCRJ3QwRSVI3Q0SS1M0QkSR1M0QkSd0MEUlSt5FCJMn1o9QkSQvL4v01JjkS+Gbg+CTHAmlNRwNLxzw2SdJhbr8hAvwm8FbgRcAt7AmRx4D3j3FckqR5YL8hUlXvBd6b5Ler6n0TGpMkaZ54tisRAKrqfUl+BFg+vE9VbRjTuCRJ88BIIZLko8B3AbcCT7VyAYaIJC1gI4UIsBI4papqnIORJM0vo/5O5A7gO8Y5EEnS/DNqiBwP3JVkc5JNM8v+dkhyZJLPJ/mPJHcm+aNWPznJzUmmk3w8yfNb/Zva9nRrXz50rHe0+heTnD1UX9Vq00kuPtCTlyQdnFFvZ/1hx7G/Bryyqh5PcgTw2SSfAn4HeE9VbUzyl8CFwBXt8+GqekmS84F3Ab+U5BTgfOD7GUw1/pck392+4wPATwPbgK1JNlXVXR1jlSR1GHV21r8e6IHb85PH2+YRbSnglcAvt/p6BgF1BXAue8Lqb4H3J0mrb6yqrwH/nWQaOKP1m66qewGSbGx9DRFJmpBRX3vy1SSPteXJJE8leWyE/RYluRXYAWwB/gt4pKp2ty7b2PPL96XAAwCt/VHghcP1vfbZV322caxJMpVkaufOnaOcsiRpBCOFSFV9a1UdXVVHA0cBvwh8cIT9nqqqlwHLGFw9fO/BDLZXVa2tqpVVtXLJkiVzMQRJek464Lf41sDfA2c/a+c9+zwC3AC8AjgmycxttGXA9ra+HTgJoLV/G/CV4fpe++yrLkmakFF/bPgLQ5vPY/C7kSefZZ8lwDeq6pEkRzF4AP4uBmFyHrARWA1c23bZ1LY/19o/XVXVZoH9dZI/Z/BgfQXweQbv8VqR5GQG4XE+e561SJImYNTZWT83tL4buI/BQ+z9ORFYn2QRg+C5uqo+meQuYGOSPwa+AFzZ+l8JfLQ9ON/FIBSoqjuTXM3ggflu4KKqegogyZuBzcAiYF1V3Tni+UiSDoFRZ2e94UAPXFW3AT80S/1e9syuGq4/Cbx2H8e6DLhslvp1wHUHOjZJ0qEx6uysZUmuSbKjLZ9Ismzcg5MkHd5GfbD+EQbPLF7Uln9oNUnSAjZqiCypqo9U1e62XAU4V1aSFrhRQ+QrSX61/XhwUZJfZTD9VpK0gI0aIr8BvA54CHiQwRTcXx/TmCRJ88SoU3wvBVZX1cMASY4D3s0gXCRJC9SoVyI/OBMgAFW1i1mm70qSFpZRQ+R5SY6d2WhXIqNexUiSnqNGDYI/Az6X5G/a9muZ5cd/kqSFZdRfrG9IMsXgb4EA/IJ//EmSNPItqRYaBock6WkH/Cp4SZJmGCKSpG6GiCSpmyEiSepmiEiSuhkikqRuhogkqZshIknqZohIkroZIpKkboaIJKmbISJJ6maISJK6GSKSpG6GiCSpmyEiSeo2thBJclKSG5LcleTOJG9p9eOSbElyT/s8ttWT5PIk00luS3La0LFWt/73JFk9VD89ye1tn8uTZFznI0l6pnFeiewGfreqTgHOBC5KcgpwMXB9Va0Arm/bAOcAK9qyBrgCBqEDXAK8HDgDuGQmeFqfNw7tt2qM5yNJ2svYQqSqHqyqf2/rXwXuBpYC5wLrW7f1wGva+rnAhhq4CTgmyYnA2cCWqtpVVQ8DW4BVre3oqrqpqgrYMHQsSdIETOSZSJLlwA8BNwMnVNWDrekh4IS2vhR4YGi3ba22v/q2Weqzff+aJFNJpnbu3HlQ5yJJ2mPsIZLkBcAngLdW1WPDbe0KosY9hqpaW1Urq2rlkiVLxv11krRgjDVEkhzBIED+qqr+rpW/1G5F0T53tPp24KSh3Ze12v7qy2apS5ImZJyzswJcCdxdVX8+1LQJmJlhtRq4dqh+QZuldSbwaLvttRk4K8mx7YH6WcDm1vZYkjPbd10wdCxJ0gQsHuOxfxT4NeD2JLe22u8DfwJcneRC4H7gda3tOuDVwDTwBPAGgKraleSdwNbW79Kq2tXW3wRcBRwFfKotkqQJGVuIVNVngX39buNVs/Qv4KJ9HGsdsG6W+hRw6kEMU5J0EPzFuiSpmyEiSepmiEiSuhkikqRuhogkqZshIknqZohIkroZIpKkboaIJKmbISJJ6maISJK6GSKSpG6GiCSpmyEiSepmiEiSuhkikqRuhogkqZshIknqZohIkroZIpKkboaIJKmbISJJ6maISJK6GSKSpG6GiCSpmyEiSepmiEiSuo0tRJKsS7IjyR1DteOSbElyT/s8ttWT5PIk00luS3La0D6rW/97kqweqp+e5Pa2z+VJMq5zkSTNbpxXIlcBq/aqXQxcX1UrgOvbNsA5wIq2rAGugEHoAJcALwfOAC6ZCZ7W541D++39XZKkMRtbiFTVjcCuvcrnAuvb+nrgNUP1DTVwE3BMkhOBs4EtVbWrqh4GtgCrWtvRVXVTVRWwYehYkqQJmfQzkROq6sG2/hBwQltfCjww1G9bq+2vvm2WuiRpgubswXq7gqhJfFeSNUmmkkzt3LlzEl8pSQvCpEPkS+1WFO1zR6tvB04a6res1fZXXzZLfVZVtbaqVlbVyiVLlhz0SUiSBiYdIpuAmRlWq4Frh+oXtFlaZwKPtttem4GzkhzbHqifBWxubY8lObPNyrpg6FiSpAlZPK4DJ/kY8JPA8Um2MZhl9SfA1UkuBO4HXte6Xwe8GpgGngDeAFBVu5K8E9ja+l1aVTMP69/EYAbYUcCn2iJJmqCxhUhVvX4fTa+apW8BF+3jOOuAdbPUp4BTD2aMkqSD4y/WJUndDBFJUjdDRJLUzRCRJHUzRCRJ3QwRSVI3Q0SS1M0QkSR1M0QkSd0MEUlSN0NEktTNEJEkdTNEJEndDBFJUjdDRJLUzRCRJHUzRCRJ3QwRSVI3Q0SS1M0QkSR1M0QkSd0MEUlSN0NEktTNEJEkdVs81wOQdOj8z6U/MNdD0GHoO//g9rEd2ysRSVI3Q0SS1M0QkSR1m/chkmRVki8mmU5y8VyPR5IWknkdIkkWAR8AzgFOAV6f5JS5HZUkLRzzOkSAM4Dpqrq3qr4ObATOneMxSdKCMd+n+C4FHhja3ga8fO9OSdYAa9rm40m+OIGxLQTHA1+e60EcDvLu1XM9BD2T/z5nXJKDPcKL99Uw30NkJFW1Flg71+N4rkkyVVUr53oc0mz89zkZ8/121nbgpKHtZa0mSZqA+R4iW4EVSU5O8nzgfGDTHI9JkhaMeX07q6p2J3kzsBlYBKyrqjvneFgLibcIdTjz3+cEpKrmegySpHlqvt/OkiTNIUNEktTNEFEXXzejw1WSdUl2JLljrseyEBgiOmC+bkaHuauAVXM9iIXCEFEPXzejw1ZV3QjsmutxLBSGiHrM9rqZpXM0FklzyBCRJHUzRNTD181IAgwR9fF1M5IAQ0Qdqmo3MPO6mbuBq33djA4XST4GfA74niTbklw412N6LvO1J5Kkbl6JSJK6GSKSpG6GiCSpmyEiSepmiEiSuhki0iGQ5N+epf2+JLcnubUtPzKGMXwmycpDfVxpf+b1n8eVDhdVNUoo/FRVfXm2hiSLquqpQzwsaey8EpEOgSSPt88Tk9zYrjbuSPLj+9snyZ8l+Q/gFUn+IMnWtt/aJGn9nr7CSHJ8kvva+lFJNia5O8k1wFFjP1FpL4aIdGj9MrC5ql4GvBS4dajthhYuN7ftbwFurqqXVtVngfdX1Q9X1akMAuFnn+W7fgt4oqq+D7gEOP2Qnok0Am9nSYfWVmBdkiOAv6+q4RDZ+3bWU8AnhtuTvB34ZuA44E7gH/bzXT8BXA5QVbclue1QnIB0ILwSkQ6h9geRfoLBW42vSnLBfro/OfMcJMmRwAeB86rqB4APAUe2frvZ83/1yGccRZpDhoh0CCV5MfClqvoQ8GHgtBF3nQmHLyd5AXDeUNt97LlVNVy/kcHtM5KcCvxg57Clbt7Okg6tnwTeluQbwOPA/q5EnlZVjyT5EHAH8BCD22Iz3g1cnWQN8I9D9SuAjyS5m8HblG85+OFLB8a3+EqSunk7S5LUzRCRJHUzRCRJ3QwRSVI3Q0SS1M0QkSR1M0QkSd3+D2dg0FMESNyhAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# we can also use countplot form seaborn to plot the above information graphically.\n",
    "sns.countplot(fraud_data.isFraud)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WBTv2idc-l1a"
   },
   "source": [
    "There are only 3% of the data which are fraud and the rest 97% are not fraud. This is clearly a class imbalance problem. In this notebook we will look to solve this type of problems."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zks_JnKq_6tN"
   },
   "source": [
    "### Missing values\n",
    "Generally datasets always have some missing values. May be done during data collection, or due to some data validation rule.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "drHmFMx_ANjt"
   },
   "outputs": [],
   "source": [
    "def miss_val_info(df):\n",
    "  \"\"\"\n",
    "  This function will take a dataframe and calculates the frequency and percentage of missing values in each column.\n",
    "  \"\"\"\n",
    "  missing_count = df.isnull().sum().sort_values(ascending = False)\n",
    "  missing_percent = round(missing_count / len(df) * 100, 2)\n",
    "  missing_info = pd.concat([missing_count, missing_percent], axis = 1, keys=['Missing Value Count','Percent of missing values'])\n",
    "  return missing_info[missing_info['Missing Value Count'] != 0]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 419
    },
    "id": "EHgTEqHR-Kk2",
    "outputId": "12627aea-4503-4183-94a1-90a6bea21e4b"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Missing Value Count</th>\n",
       "      <th>Percent of missing values</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>id_24</th>\n",
       "      <td>58567</td>\n",
       "      <td>99.18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id_25</th>\n",
       "      <td>58530</td>\n",
       "      <td>99.11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id_27</th>\n",
       "      <td>58529</td>\n",
       "      <td>99.11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id_21</th>\n",
       "      <td>58529</td>\n",
       "      <td>99.11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id_22</th>\n",
       "      <td>58529</td>\n",
       "      <td>99.11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>V309</th>\n",
       "      <td>3</td>\n",
       "      <td>0.01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>V308</th>\n",
       "      <td>3</td>\n",
       "      <td>0.01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>V307</th>\n",
       "      <td>3</td>\n",
       "      <td>0.01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>V306</th>\n",
       "      <td>3</td>\n",
       "      <td>0.01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>V290</th>\n",
       "      <td>3</td>\n",
       "      <td>0.01</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>414 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       Missing Value Count  Percent of missing values\n",
       "id_24                58567                      99.18\n",
       "id_25                58530                      99.11\n",
       "id_27                58529                      99.11\n",
       "id_21                58529                      99.11\n",
       "id_22                58529                      99.11\n",
       "...                    ...                        ...\n",
       "V309                     3                       0.01\n",
       "V308                     3                       0.01\n",
       "V307                     3                       0.01\n",
       "V306                     3                       0.01\n",
       "V290                     3                       0.01\n",
       "\n",
       "[414 rows x 2 columns]"
      ]
     },
     "execution_count": 10,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "miss_val_info(fraud_data)      # Display the frequency and percentage of data missing in each column"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "02bqVJEbBxQc"
   },
   "source": [
    "Out of 434 columns, 414 have some missing values."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qjdy3vG5BmER"
   },
   "source": [
    "### Dealing with Missing Values\n",
    "*  Filling the missing values with right technique can change our results drastically. \n",
    "*  Also, there is no fixed rule of filling the missing values.\n",
    "*  No method is perfect for filling the missing values. We need to use our common sense, our logic, or may need to see what works for that particular data set.\n",
    "\n",
    "### Ways of dealing with missing values:\n",
    "\n",
    "**Default value:** One can fill the missing value by default value on the basis of one's 1) understanding of variable, 2) context / data insight or 3) common sense / logic. \n",
    "\n",
    "**Deleting:** Suppose in our dataset we have too many missing values in\n",
    "\n",
    "*  Column, we can drop the column\n",
    "*  Row, drop the row. Usually we do this for a large enough dataset.\n",
    "\n",
    "**Mean/Median/Mode - Imputation:** We fill missing values by mean or median or mode(i.e. maximum occuring value). Generally we use mean but if there are some outliers, we fill missing values with median. Mode is used to fill missing values for categorical column."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "eZzsrECfAaSV"
   },
   "source": [
    "Eliminate columns with more than 20% missing values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "qv5kehT_ALOO"
   },
   "outputs": [],
   "source": [
    "fraud_data = fraud_data[fraud_data.columns[fraud_data.isnull().mean() < 0.2]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ysp1sghyDU1s"
   },
   "source": [
    "Here we will fill missing values with mean value for the numerical column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "KSvRk1xJBAyR"
   },
   "outputs": [],
   "source": [
    "# filling missing values of numerical columns with mean value.\n",
    "num_cols = fraud_data.select_dtypes(include=np.number).columns      # getting all the numerical columns\n",
    "\n",
    "fraud_data[num_cols] = fraud_data[num_cols].fillna(fraud_data[num_cols].mean())   # fills the missing values with mean"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yK7ANXA5Eoeh"
   },
   "source": [
    "Filling categorical columns with mode (i.e. maximum occuring element in the column)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "UeUz0v1ZEhAY"
   },
   "outputs": [],
   "source": [
    "cat_cols = fraud_data.select_dtypes(include = 'object').columns    # getting all the categorical columns\n",
    "\n",
    "fraud_data[cat_cols] = fraud_data[cat_cols].fillna(fraud_data[cat_cols].mode().iloc[0])  # fills the missing values with maximum occuring element in the column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ajSX2MhlFbWz",
    "outputId": "319fd42e-f10a-421a-c1b7-d14e404bc751"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Missing Value Count</th>\n",
       "      <th>Percent of missing values</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [Missing Value Count, Percent of missing values]\n",
       "Index: []"
      ]
     },
     "execution_count": 14,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's have a look if there still exist any missing values\n",
    "miss_val_info(fraud_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LW9XOIz6CGBj"
   },
   "source": [
    "Notice, now we don't have any column with missing value."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qKO21Qg8NSQc"
   },
   "source": [
    "### One Hot Encoding (Creating dummies for categorical columns)\n",
    "In this strategy, each category value is converted into a new column and assigned a 1 or 0 (notation for true/false) value to the column. In Python there is a class 'OneHotEncoder' in 'sklearn.preprocessing' to do this task, but here we will use pandas function 'get_dummies()'. This get_dummies() does the same work as done by 'OneHotEncoder' form sklearn.preprocessing.\n",
    "\n",
    "So, let's create dummy variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ydD6ZRZrMFFu"
   },
   "outputs": [],
   "source": [
    "fraud_data = pd.get_dummies(fraud_data, columns=cat_cols)    # earlier we have collected all the categorical columns in cat_cols"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ulgnsjelE3I6",
    "outputId": "ac9b4811-5e38-4ad8-9339-57d6ccb7f1ee"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>TransactionID</th>\n",
       "      <th>isFraud</th>\n",
       "      <th>TransactionDT</th>\n",
       "      <th>TransactionAmt</th>\n",
       "      <th>card1</th>\n",
       "      <th>card2</th>\n",
       "      <th>card3</th>\n",
       "      <th>card5</th>\n",
       "      <th>addr1</th>\n",
       "      <th>addr2</th>\n",
       "      <th>C1</th>\n",
       "      <th>C2</th>\n",
       "      <th>C3</th>\n",
       "      <th>C4</th>\n",
       "      <th>C5</th>\n",
       "      <th>C6</th>\n",
       "      <th>C7</th>\n",
       "      <th>C8</th>\n",
       "      <th>C9</th>\n",
       "      <th>C10</th>\n",
       "      <th>C11</th>\n",
       "      <th>C12</th>\n",
       "      <th>C13</th>\n",
       "      <th>C14</th>\n",
       "      <th>D1</th>\n",
       "      <th>D10</th>\n",
       "      <th>D15</th>\n",
       "      <th>V12</th>\n",
       "      <th>V13</th>\n",
       "      <th>V14</th>\n",
       "      <th>V15</th>\n",
       "      <th>V16</th>\n",
       "      <th>V17</th>\n",
       "      <th>V18</th>\n",
       "      <th>V19</th>\n",
       "      <th>V20</th>\n",
       "      <th>V21</th>\n",
       "      <th>V22</th>\n",
       "      <th>V23</th>\n",
       "      <th>V24</th>\n",
       "      <th>...</th>\n",
       "      <th>P_emaildomain_hotmail.com</th>\n",
       "      <th>P_emaildomain_hotmail.de</th>\n",
       "      <th>P_emaildomain_hotmail.es</th>\n",
       "      <th>P_emaildomain_hotmail.fr</th>\n",
       "      <th>P_emaildomain_icloud.com</th>\n",
       "      <th>P_emaildomain_juno.com</th>\n",
       "      <th>P_emaildomain_live.com</th>\n",
       "      <th>P_emaildomain_live.com.mx</th>\n",
       "      <th>P_emaildomain_live.fr</th>\n",
       "      <th>P_emaildomain_mac.com</th>\n",
       "      <th>P_emaildomain_mail.com</th>\n",
       "      <th>P_emaildomain_me.com</th>\n",
       "      <th>P_emaildomain_msn.com</th>\n",
       "      <th>P_emaildomain_netzero.com</th>\n",
       "      <th>P_emaildomain_netzero.net</th>\n",
       "      <th>P_emaildomain_optonline.net</th>\n",
       "      <th>P_emaildomain_outlook.com</th>\n",
       "      <th>P_emaildomain_outlook.es</th>\n",
       "      <th>P_emaildomain_prodigy.net.mx</th>\n",
       "      <th>P_emaildomain_protonmail.com</th>\n",
       "      <th>P_emaildomain_ptd.net</th>\n",
       "      <th>P_emaildomain_q.com</th>\n",
       "      <th>P_emaildomain_roadrunner.com</th>\n",
       "      <th>P_emaildomain_rocketmail.com</th>\n",
       "      <th>P_emaildomain_sbcglobal.net</th>\n",
       "      <th>P_emaildomain_sc.rr.com</th>\n",
       "      <th>P_emaildomain_servicios-ta.com</th>\n",
       "      <th>P_emaildomain_suddenlink.net</th>\n",
       "      <th>P_emaildomain_twc.com</th>\n",
       "      <th>P_emaildomain_verizon.net</th>\n",
       "      <th>P_emaildomain_web.de</th>\n",
       "      <th>P_emaildomain_windstream.net</th>\n",
       "      <th>P_emaildomain_yahoo.co.jp</th>\n",
       "      <th>P_emaildomain_yahoo.co.uk</th>\n",
       "      <th>P_emaildomain_yahoo.com</th>\n",
       "      <th>P_emaildomain_yahoo.com.mx</th>\n",
       "      <th>P_emaildomain_yahoo.de</th>\n",
       "      <th>P_emaildomain_yahoo.es</th>\n",
       "      <th>P_emaildomain_yahoo.fr</th>\n",
       "      <th>P_emaildomain_ymail.com</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2994681</td>\n",
       "      <td>0</td>\n",
       "      <td>242834</td>\n",
       "      <td>25.000</td>\n",
       "      <td>9803</td>\n",
       "      <td>583.0</td>\n",
       "      <td>150.0</td>\n",
       "      <td>226.0</td>\n",
       "      <td>269.0</td>\n",
       "      <td>87.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>124.267325</td>\n",
       "      <td>163.966616</td>\n",
       "      <td>0.55965</td>\n",
       "      <td>0.597624</td>\n",
       "      <td>0.999398</td>\n",
       "      <td>0.124619</td>\n",
       "      <td>0.125551</td>\n",
       "      <td>0.137374</td>\n",
       "      <td>0.13883</td>\n",
       "      <td>0.814731</td>\n",
       "      <td>0.845542</td>\n",
       "      <td>0.133239</td>\n",
       "      <td>0.135394</td>\n",
       "      <td>1.035702</td>\n",
       "      <td>1.058825</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3557242</td>\n",
       "      <td>0</td>\n",
       "      <td>15123000</td>\n",
       "      <td>117.000</td>\n",
       "      <td>7919</td>\n",
       "      <td>194.0</td>\n",
       "      <td>150.0</td>\n",
       "      <td>166.0</td>\n",
       "      <td>181.0</td>\n",
       "      <td>87.0</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>63.0</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3327470</td>\n",
       "      <td>0</td>\n",
       "      <td>8378575</td>\n",
       "      <td>73.773</td>\n",
       "      <td>12778</td>\n",
       "      <td>500.0</td>\n",
       "      <td>185.0</td>\n",
       "      <td>224.0</td>\n",
       "      <td>284.0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>54.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>54.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3118781</td>\n",
       "      <td>0</td>\n",
       "      <td>2607840</td>\n",
       "      <td>400.000</td>\n",
       "      <td>12316</td>\n",
       "      <td>548.0</td>\n",
       "      <td>150.0</td>\n",
       "      <td>195.0</td>\n",
       "      <td>441.0</td>\n",
       "      <td>87.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>124.267325</td>\n",
       "      <td>163.966616</td>\n",
       "      <td>0.55965</td>\n",
       "      <td>0.597624</td>\n",
       "      <td>0.999398</td>\n",
       "      <td>0.124619</td>\n",
       "      <td>0.125551</td>\n",
       "      <td>0.137374</td>\n",
       "      <td>0.13883</td>\n",
       "      <td>0.814731</td>\n",
       "      <td>0.845542</td>\n",
       "      <td>0.133239</td>\n",
       "      <td>0.135394</td>\n",
       "      <td>1.035702</td>\n",
       "      <td>1.058825</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3459772</td>\n",
       "      <td>0</td>\n",
       "      <td>12226544</td>\n",
       "      <td>31.950</td>\n",
       "      <td>9002</td>\n",
       "      <td>453.0</td>\n",
       "      <td>150.0</td>\n",
       "      <td>226.0</td>\n",
       "      <td>264.0</td>\n",
       "      <td>87.0</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>33</td>\n",
       "      <td>3</td>\n",
       "      <td>54.0</td>\n",
       "      <td>71.000000</td>\n",
       "      <td>620.000000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 250 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   TransactionID  isFraud  ...  P_emaildomain_yahoo.fr  P_emaildomain_ymail.com\n",
       "0        2994681        0  ...                       0                        0\n",
       "1        3557242        0  ...                       0                        0\n",
       "2        3327470        0  ...                       0                        0\n",
       "3        3118781        0  ...                       0                        0\n",
       "4        3459772        0  ...                       0                        0\n",
       "\n",
       "[5 rows x 250 columns]"
      ]
     },
     "execution_count": 16,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fraud_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kAkHAuGaE_Xo"
   },
   "source": [
    "If you notice, a lot of dummy variables are created like; **P_emaildomain_hotmail.com, P_emaildomain_hotmail.de,** etc."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DQPCxfdDn0d5"
   },
   "source": [
    "#### Separate Input Features and Output Features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "UQWaD295Lr7p"
   },
   "outputs": [],
   "source": [
    "# Separate input features and output feature\n",
    "X = fraud_data.drop(columns = ['isFraud'])       # input features\n",
    "Y = fraud_data.isFraud      # output feature\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# Split randomly into 70% train data and 30% test data\n",
    "X_train, X_Test, Y_train, Y_Test = train_test_split(X, Y, test_size = 0.3, random_state = 123)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0myjC1VxpHyx"
   },
   "source": [
    "# Dealing with Imbalanced Data\n",
    "Most machine learning algorithms work best when the number of samples in each class are about equal. This is because most algorithms are designed to maximize accuracy and reduce error.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "u9vPjbAMvDFX"
   },
   "source": [
    "3. **Generate Synthetic Samples:** Here we will use imblearn’s SMOTE or Synthetic Minority Oversampling Technique. SMOTE uses a nearest neighbors algorithm to generate new and synthetic data we can use for training our model.\n",
    "\n",
    "Again, it’s important to generate the new samples only in the training set to ensure our model generalizes well to unseen data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "FWkp26p-uwF2"
   },
   "outputs": [],
   "source": [
    "# import SMOTE \n",
    "from imblearn.over_sampling import SMOTE\n",
    "\n",
    "sm = SMOTE(random_state = 25, sampling_strategy = 1.0)   # again we are eqalizing both the classes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Yf7sHze_wFJp"
   },
   "outputs": [],
   "source": [
    "# fit the sampling\n",
    "X_train, Y_train = sm.fit_resample(X_train, Y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "id": "15XOWmqPwULH",
    "outputId": "16229543-6fce-41f4-fd49-815cb829d8fb"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([0, 1]), array([39944, 39944]))"
      ]
     },
     "execution_count": 20,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.unique(Y_train, return_counts=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "j8m9UNkRw1LV"
   },
   "source": [
    "The count of both the classes are equal."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "--K223wHPMZe"
   },
   "source": [
    "### Building Random Forest Model\n",
    "A Random Forest 🌲🌲🌲 is actually just a bunch of Decision Trees 🌲 bundled together (ohhhhh that’s why it’s called a forest). In this notebook we will learn how to build Random Forest Model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "p8M-21Xp4EhV"
   },
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "rfc = RandomForestClassifier(criterion='entropy')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 153
    },
    "id": "I1zxcn6P5Rud",
    "outputId": "1d4f9890-5da7-4494-e499-ac64e053151d"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestClassifier(bootstrap=True, ccp_alpha=0.0, class_weight=None,\n",
       "                       criterion='entropy', max_depth=None, max_features='auto',\n",
       "                       max_leaf_nodes=None, max_samples=None,\n",
       "                       min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "                       min_samples_leaf=1, min_samples_split=2,\n",
       "                       min_weight_fraction_leaf=0.0, n_estimators=100,\n",
       "                       n_jobs=None, oob_score=False, random_state=None,\n",
       "                       verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 22,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rfc.fit(X_train, Y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "id": "9S9hvP8N4RDw",
    "outputId": "a1b2d410-ab8f-441f-de5d-129012a9db02"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9753908675283626"
      ]
     },
     "execution_count": 23,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rfc.score(X_Test, Y_Test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CFooIZrR4q4B"
   },
   "source": [
    "### Feature Selection\n",
    "Feature selection is the process of reducing the number of input variables when developing a predictive model.\n",
    "\n",
    "It is desirable to reduce the number of input variables to both reduce the computational cost of modeling and, in some cases, to improve the performance of the model.\n",
    "\n",
    "Here we are using 'slectKBest' from 'sklearn.feature_selection' which selects features according to the k highest scores. It takes two parameters:\n",
    "\n",
    "* **score_func:** **callable**\n",
    "\n",
    "  Function taking two arrays X and y, and returning a pair of arrays (scores, pvalues) or a single array with scores. Default is f_classif (see below “See also”). The default function only works with classification tasks.\n",
    "\n",
    "* **k:  optional, default=10**\n",
    "  \n",
    "  Number of top features to select.\n",
    "\n",
    "\n",
    "When we have numerical features and categorical output we can use 'f_classif' from 'sklearn.feature_selection'.\n",
    "\n",
    "Further reading about feature selection: https://machinelearningmastery.com/feature-selection-with-real-and-categorical-data/\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "KwvlPSXR4ckY"
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_selection import SelectKBest, f_classif"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "TdjhE2Ri4p5U"
   },
   "outputs": [],
   "source": [
    "selector = SelectKBest(f_classif, k=10)     # Let's say we select 10 best features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "2BNBrOEO5ev2"
   },
   "outputs": [],
   "source": [
    "X_new = selector.fit_transform(X, Y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "t7ETZ0cT6sDw"
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, Y_train, Y_test = train_test_split(X_new, Y, test_size = 0.2, random_state = 42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 153
    },
    "id": "hdcjRY2m57Yk",
    "outputId": "97200dd0-10f8-4336-dd6b-35ac254f5262"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestClassifier(bootstrap=True, ccp_alpha=0.0, class_weight=None,\n",
       "                       criterion='entropy', max_depth=None, max_features='auto',\n",
       "                       max_leaf_nodes=None, max_samples=None,\n",
       "                       min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "                       min_samples_leaf=1, min_samples_split=2,\n",
       "                       min_weight_fraction_leaf=0.0, n_estimators=100,\n",
       "                       n_jobs=None, oob_score=False, random_state=None,\n",
       "                       verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 28,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rfc.fit(X_train, Y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "id": "8Yg0g4H86MOV",
    "outputId": "49fdaeb7-159f-4dd3-e416-f9e499540a69"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.969265938531877"
      ]
     },
     "execution_count": 29,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rfc.score(X_test, Y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0DIxQVy_guKV"
   },
   "source": [
    "**Note:** Obviously we can notice there is 0.7% decrease in accuracy score. The point to note is that we had got 97.5% accuracy with 249 features, then after that we selected only 10 features of 249 features and still able to get 96.9% accurate results. The conclusion is that we have reduced a lot of computational complexities which is very good as our aim is not only to increase the performance of the model at any cost but also to reduce the computational complexity of our model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "no3aR-9LimRd"
   },
   "source": [
    "### Cross - Validation\n",
    "Usually, our data is divided into Train and Test Sets. The Train set is further divided into Train and Validation set.\n",
    "\n",
    "The Validation Set helps us in selecting good parameters/tune the parameters for our model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "TWZDs96oi1mh"
   },
   "outputs": [],
   "source": [
    "# We will use here k - fold cross validation technique\n",
    "from sklearn.model_selection import cross_validate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 187
    },
    "id": "_JuwMQb1i1Wh",
    "outputId": "3bef6842-bc9c-484b-871d-a9ff84017e5f"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'fit_time': array([1.53913021, 1.56077433, 1.57155609, 1.60067868, 1.6161437 ,\n",
       "        1.58456111, 1.54124928, 1.52417064, 1.50520635, 1.52491522]),\n",
       " 'score_time': array([0.05374193, 0.05819058, 0.05322051, 0.05408406, 0.05415249,\n",
       "        0.05401015, 0.05401635, 0.0528028 , 0.05400944, 0.05233979]),\n",
       " 'test_accuracy': array([0.96918388, 0.96867592, 0.96952252, 0.96867592, 0.96765453,\n",
       "        0.96900931, 0.96816257, 0.96867062, 0.96985605, 0.96900931]),\n",
       " 'test_precision': array([0.85185185, 0.78571429, 0.81818182, 0.80769231, 0.62857143,\n",
       "        0.75757576, 0.7       , 0.71428571, 0.78947368, 0.8       ]),\n",
       " 'test_recall': array([0.11442786, 0.10945274, 0.13432836, 0.10447761, 0.11      ,\n",
       "        0.125     , 0.105     , 0.125     , 0.15      , 0.11940299])}"
      ]
     },
     "execution_count": 31,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cv_results = cross_validate(rfc, X_new, Y, cv=10, scoring=[\"accuracy\", \"precision\", \"recall\"])\n",
    "cv_results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "id": "NPQhL-Wdi1Fr",
    "outputId": "cd639aa6-a000-454a-b1a4-8f9372c53677"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy:  0.968842065059342\n"
     ]
    }
   ],
   "source": [
    "print(\"Accuracy: \", cv_results[\"test_accuracy\"].mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bn_XEArIjm9A"
   },
   "source": [
    "Even with cross validation we are getting approx 96.9% of accurate results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Fh2gSYBB7q9t"
   },
   "source": [
    "### Hyper parameter tunning\n",
    "Hyperparameters are important parts of the ML model and can make the model gold or trash. Here we have discussed one of the popular hyperparameter tunning method i.e. using Grid Search CV."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "iJULs32W7_2x"
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import GridSearchCV"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "9H2stTDwkRbl"
   },
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "rfc = RandomForestClassifier()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "M2itoOa7mddr"
   },
   "source": [
    "To know about different parameters of random forest visit here: https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "RSZj1jrx9XQg"
   },
   "outputs": [],
   "source": [
    "# Different parameters in random forest\n",
    "\n",
    "criterion = ['gini', 'entropy']        # what criteria to consider\n",
    "\n",
    "n_estimators = [100, 200, 300]       # Number of trees in random forest\n",
    "\n",
    "max_features = ['auto', 'sqrt']       # Number of features to consider at every split\n",
    "\n",
    "max_depth = [10, 20]      # Maximum number of levels in tree. Hope you remember linspace function from numpy session\n",
    "\n",
    "max_depth.append(None)     # also appendin 'None' in max_depth i.e. no maximum depth to be considered.\n",
    "\n",
    "params = {'criterion': criterion,\n",
    "          'n_estimators': n_estimators,\n",
    "          'max_features': max_features,\n",
    "          'max_depth': max_depth}\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 85
    },
    "id": "QbMaahyDnneZ",
    "outputId": "5a0ba0c1-4098-49d4-bb2e-73d47c809041"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'criterion': ['gini', 'entropy'],\n",
       " 'max_depth': [10, 20, None],\n",
       " 'max_features': ['auto', 'sqrt'],\n",
       " 'n_estimators': [100, 200, 300]}"
      ]
     },
     "execution_count": 36,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "params"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "XwlLlPFN9ZnF"
   },
   "outputs": [],
   "source": [
    "gs = GridSearchCV(rfc, param_grid=params, n_jobs=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 408
    },
    "id": "zFDZSMKB9hnT",
    "outputId": "e45aee54-21a9-4772-e208-4ab10d7d3b70"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=None, error_score=nan,\n",
       "             estimator=RandomForestClassifier(bootstrap=True, ccp_alpha=0.0,\n",
       "                                              class_weight=None,\n",
       "                                              criterion='gini', max_depth=None,\n",
       "                                              max_features='auto',\n",
       "                                              max_leaf_nodes=None,\n",
       "                                              max_samples=None,\n",
       "                                              min_impurity_decrease=0.0,\n",
       "                                              min_impurity_split=None,\n",
       "                                              min_samples_leaf=1,\n",
       "                                              min_samples_split=2,\n",
       "                                              min_weight_fraction_leaf=0.0,\n",
       "                                              n_estimators=100, n_jobs=None,\n",
       "                                              oob_score=False,\n",
       "                                              random_state=None, verbose=0,\n",
       "                                              warm_start=False),\n",
       "             iid='deprecated', n_jobs=2,\n",
       "             param_grid={'criterion': ['gini', 'entropy'],\n",
       "                         'max_depth': [10, 20, None],\n",
       "                         'max_features': ['auto', 'sqrt'],\n",
       "                         'n_estimators': [100, 200, 300]},\n",
       "             pre_dispatch='2*n_jobs', refit=True, return_train_score=False,\n",
       "             scoring=None, verbose=0)"
      ]
     },
     "execution_count": 38,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gs.fit(X_train, Y_train)    # this will take a lot of time to execute; have some patience"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 85
    },
    "id": "z6FNdBsL9nB6",
    "outputId": "8ab6d8a2-bb3d-4955-c248-7233876935e7"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'criterion': 'gini',\n",
       " 'max_depth': 10,\n",
       " 'max_features': 'sqrt',\n",
       " 'n_estimators': 100}"
      ]
     },
     "execution_count": 39,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gs.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "id": "K0l9DDOE_QxJ",
    "outputId": "c36979ec-8efc-4f36-d720-e7ae189e68cf"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.968418616846677"
      ]
     },
     "execution_count": 40,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gs.best_score_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "id": "hkoDdv8wev4g",
    "outputId": "93f40867-3092-4d2e-cf0c-8c7115932257"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9689272711878757"
      ]
     },
     "execution_count": 42,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gs.score(X_test, Y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3xj73WQFfVVA"
   },
   "source": [
    "**Conclusion**\n",
    "*  We observed that the dataset contained missing values. We removed some columns and filled missing values for numerical column with mean and categorical column with mode (i.e. the maximum occuring value).\n",
    "*  We observed that the dataset was imblanced. We used 'SMOTE' to generate new data to deal with the problem of imbalanced data.\n",
    "*  We build Random Forest model, got accuracy score of 97.53%.\n",
    "*  Then we selected only 10 most important features using selectKBest and f_classif. Here the model complexity is reduced a lot (which is very good) with very little decrease in accuracy\n",
    "*  Cross Validation and Hyper parameter tunning gave nearly 96.8% of accurate results which is not bad. Most of the times the default values for hyper parameters of the models are same that we get through the hyper parameter tunning. That's the reason there is not much difference in normal model and tunned model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "j8ydaZR7timZ"
   },
   "source": [
    "References:\n",
    "1. [Dealing with Imbalanced Data by Tara Boyle](https://towardsdatascience.com/methods-for-dealing-with-imbalanced-data-5b761be45a18)\n",
    "2. [Data Pre-processing - Handling missing values and dealing with class imbalance by Bharat Ram Ammu](https://www.youtube.com/watch?v=vksQx1JNo8Y)\n",
    "3. https://www.kaggle.com/drgilermo/a-tutorial-for-complete-beginners\n",
    "4. https://machinelearningmastery.com/feature-selection-with-real-and-categorical-data/"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [
    "qjdy3vG5BmER",
    "qKO21Qg8NSQc",
    "DQPCxfdDn0d5",
    "--K223wHPMZe",
    "CFooIZrR4q4B",
    "Fh2gSYBB7q9t"
   ],
   "name": "model-optimization-with-random-forest-db41a665dbb340d99d00e2821ea36a32.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}