AML_Project_LearnerNotebook_LowCode.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "9EaJ8AGwpM-2",
      "metadata": {
        "id": "9EaJ8AGwpM-2"
      },
      "source": [
        "## Problem Statement"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "x3-QehJxbp0t",
      "metadata": {
        "id": "x3-QehJxbp0t"
      },
      "source": [
        "### Business Context\n",
        "\n",
        "The Thera bank recently saw a steep decline in the number of users of their credit card, credit cards are a good source of income for banks because of different kinds of fees charged by the banks like annual fees, balance transfer fees, and cash advance fees, late payment fees, foreign transaction fees, and others. Some fees are charged to every user irrespective of usage, while others are charged under specified circumstances.\n",
        "\n",
        "Customers’ leaving credit cards services would lead bank to loss, so the bank wants to analyze the data of customers and identify the customers who will leave their credit card services and reason for same – so that bank could improve upon those areas\n",
        "\n",
        "You as a Data scientist at Thera bank need to come up with a classification model that will help the bank improve its services so that customers do not renounce their credit cards\n",
        "\n",
        "### Data Description\n",
        "\n",
        "* CLIENTNUM: Client number. Unique identifier for the customer holding the account\n",
        "* Attrition_Flag: Internal event (customer activity) variable - if the account is closed then \"Attrited Customer\" else \"Existing Customer\"\n",
        "* Customer_Age: Age in Years\n",
        "* Gender: Gender of the account holder\n",
        "* Dependent_count: Number of dependents\n",
        "* Education_Level: Educational Qualification of the account holder - Graduate, High School, Unknown, Uneducated, College(refers to college student), Post-Graduate, Doctorate\n",
        "* Marital_Status: Marital Status of the account holder\n",
        "* Income_Category: Annual Income Category of the account holder\n",
        "* Card_Category: Type of Card\n",
        "* Months_on_book: Period of relationship with the bank (in months)\n",
        "* Total_Relationship_Count: Total no. of products held by the customer\n",
        "* Months_Inactive_12_mon: No. of months inactive in the last 12 months\n",
        "* Contacts_Count_12_mon: No. of Contacts in the last 12 months\n",
        "* Credit_Limit: Credit Limit on the Credit Card\n",
        "* Total_Revolving_Bal: Total Revolving Balance on the Credit Card\n",
        "* Avg_Open_To_Buy: Open to Buy Credit Line (Average of last 12 months)\n",
        "* Total_Amt_Chng_Q4_Q1: Change in Transaction Amount (Q4 over Q1)\n",
        "* Total_Trans_Amt: Total Transaction Amount (Last 12 months)\n",
        "* Total_Trans_Ct: Total Transaction Count (Last 12 months)\n",
        "* Total_Ct_Chng_Q4_Q1: Change in Transaction Count (Q4 over Q1)\n",
        "* Avg_Utilization_Ratio: Average Card Utilization Ratio\n",
        "\n",
        "#### What Is a Revolving Balance?\n",
        "\n",
        "- If we don't pay the balance of the revolving credit account in full every month, the unpaid portion carries over to the next month. That's called a revolving balance\n",
        "\n",
        "\n",
        "##### What is the Average Open to buy?\n",
        "\n",
        "- 'Open to Buy' means the amount left on your credit card to use. Now, this column represents the average of this value for the last 12 months.\n",
        "\n",
        "##### What is the Average utilization Ratio?\n",
        "\n",
        "- The Avg_Utilization_Ratio represents how much of the available credit the customer spent. This is useful for calculating credit scores.\n",
        "\n",
        "\n",
        "##### Relation b/w Avg_Open_To_Buy, Credit_Limit and Avg_Utilization_Ratio:\n",
        "\n",
        "- ( Avg_Open_To_Buy / Credit_Limit ) + Avg_Utilization_Ratio = 1"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "NbHOIdlwcrqR",
      "metadata": {
        "id": "NbHOIdlwcrqR"
      },
      "source": [
        "## **Please read the instructions carefully before starting the project.**\n",
        "This is a commented Jupyter IPython Notebook file in which all the instructions and tasks to be performed are mentioned.\n",
        "* Blanks '_______' are provided in the notebook that\n",
        "needs to be filled with an appropriate code to get the correct result. With every '_______' blank, there is a comment that briefly describes what needs to be filled in the blank space.\n",
        "* Identify the task to be performed correctly, and only then proceed to write the required code.\n",
        "* Fill the code wherever asked by the commented lines like \"# write your code here\" or \"# complete the code\". Running incomplete code may throw error.\n",
        "* Please run the codes in a sequential manner from the beginning to avoid any unnecessary errors.\n",
        "* Add the results/observations (wherever mentioned) derived from the analysis in the presentation and submit the same.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "v_-uuGqH-qTt",
      "metadata": {
        "id": "v_-uuGqH-qTt"
      },
      "source": [
        "## Importing necessary libraries"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Installing the libraries with the specified version.\n",
        "# uncomment and run the following line if Google Colab is being used\n",
        "# !pip install scikit-learn==1.2.2 seaborn==0.13.1 matplotlib==3.7.1 numpy==1.25.2 pandas==1.5.3 imbalanced-learn==0.10.1 xgboost==2.0.3 -q --user"
      ],
      "metadata": {
        "id": "nEmTDPI8kYd4"
      },
      "id": "nEmTDPI8kYd4",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Installing the libraries with the specified version.\n",
        "# uncomment and run the following lines if Jupyter Notebook is being used\n",
        "# !pip install scikit-learn==1.2.2 seaborn==0.13.1 matplotlib==3.7.1 numpy==1.25.2 pandas==1.5.3 imblearn==0.12.0 xgboost==2.0.3 -q --user\n",
        "# !pip install --upgrade -q threadpoolctl"
      ],
      "metadata": {
        "id": "JGuls9cUfoy8"
      },
      "id": "JGuls9cUfoy8",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Note**: *After running the above cell, kindly restart the notebook kernel and run all cells sequentially from the start again*."
      ],
      "metadata": {
        "id": "wIPQd1MFrD3B"
      },
      "id": "wIPQd1MFrD3B"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "9285c399",
      "metadata": {
        "id": "9285c399"
      },
      "outputs": [],
      "source": [
        "# Libraries to help with reading and manipulating data\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "\n",
        "# To suppress scientific notations\n",
        "pd.set_option(\"display.float_format\", lambda x: \"%.3f\" % x)\n",
        "\n",
        "# Libaries to help with data visualization\n",
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns\n",
        "\n",
        "# To tune model, get different metric scores, and split data\n",
        "from sklearn import metrics\n",
        "from sklearn.metrics import (\n",
        "    f1_score,\n",
        "    accuracy_score,\n",
        "    recall_score,\n",
        "    precision_score,\n",
        "    confusion_matrix,\n",
        "    roc_auc_score,\n",
        ")\n",
        "from sklearn.model_selection import train_test_split, StratifiedKFold, cross_val_score\n",
        "\n",
        "# To be used for data scaling and one hot encoding\n",
        "from sklearn.preprocessing import StandardScaler, MinMaxScaler, OneHotEncoder\n",
        "\n",
        "# To impute missing values\n",
        "from sklearn.impute import SimpleImputer\n",
        "\n",
        "# To oversample and undersample data\n",
        "from imblearn.over_sampling import SMOTE\n",
        "from imblearn.under_sampling import RandomUnderSampler\n",
        "\n",
        "# To do hyperparameter tuning\n",
        "from sklearn.model_selection import RandomizedSearchCV\n",
        "\n",
        "# To define maximum number of columns to be displayed in a dataframe\n",
        "pd.set_option(\"display.max_columns\", None)\n",
        "\n",
        "# To supress scientific notations for a dataframe\n",
        "pd.set_option(\"display.float_format\", lambda x: \"%.3f\" % x)\n",
        "\n",
        "# To help with model building\n",
        "from sklearn.tree import DecisionTreeClassifier\n",
        "from sklearn.ensemble import (\n",
        "    AdaBoostClassifier,\n",
        "    GradientBoostingClassifier,\n",
        "    RandomForestClassifier,\n",
        "    BaggingClassifier,\n",
        ")\n",
        "from xgboost import XGBClassifier\n",
        "\n",
        "# To supress warnings\n",
        "import warnings\n",
        "warnings.filterwarnings(\"ignore\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "xxhpZv9y-qTw",
      "metadata": {
        "id": "xxhpZv9y-qTw"
      },
      "source": [
        "## Loading the dataset"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "e2ad2a10",
      "metadata": {
        "id": "e2ad2a10"
      },
      "outputs": [],
      "source": [
        "churn = pd.read_csv(\"BankChurners.csv\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "M1TEog7cAs5m",
      "metadata": {
        "id": "M1TEog7cAs5m"
      },
      "source": [
        "## Data Overview"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "W-5jUOgu-qTz",
      "metadata": {
        "id": "W-5jUOgu-qTz"
      },
      "source": [
        "The initial steps to get an overview of any dataset is to:\n",
        "- observe the first few rows of the dataset, to check whether the dataset has been loaded properly or not\n",
        "- get information about the number of rows and columns in the dataset\n",
        "- find out the data types of the columns to ensure that data is stored in the preferred format and the value of each property is as expected.\n",
        "- check the statistical summary of the dataset to get an overview of the numerical columns of the data"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "KQi5ygTC-qT1",
      "metadata": {
        "id": "KQi5ygTC-qT1"
      },
      "source": [
        "### Checking the shape of the dataset"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "7Y8epHpjbp0z",
      "metadata": {
        "id": "7Y8epHpjbp0z"
      },
      "outputs": [],
      "source": [
        "# Checking the number of rows and columns in the training data\n",
        "churn.'_______' ##  Complete the code to view dimensions of the train data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "45153b44",
      "metadata": {
        "id": "45153b44"
      },
      "outputs": [],
      "source": [
        "# let's create a copy of the data\n",
        "data = churn.copy()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "961px703qhLV",
      "metadata": {
        "id": "961px703qhLV"
      },
      "source": [
        "### Displaying the first few rows of the dataset"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "l9RnN7Twbp03",
      "metadata": {
        "id": "l9RnN7Twbp03"
      },
      "outputs": [],
      "source": [
        "# let's view the first 5 rows of the data\n",
        "data.'_______' ##  Complete the code to view top 5 rows of the data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "N9Ir8YPgbp04",
      "metadata": {
        "id": "N9Ir8YPgbp04"
      },
      "outputs": [],
      "source": [
        "# let's view the last 5 rows of the data\n",
        "data.'_______' ##  Complete the code to view last 5 rows of the data"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5TcqcxbK-qT3",
      "metadata": {
        "id": "5TcqcxbK-qT3"
      },
      "source": [
        "### Checking the data types of the columns for the dataset"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "EXBYJoKkbp04",
      "metadata": {
        "id": "EXBYJoKkbp04"
      },
      "outputs": [],
      "source": [
        "# let's check the data types of the columns in the dataset\n",
        "data.info()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "xNr4bWoM-qT5",
      "metadata": {
        "id": "xNr4bWoM-qT5"
      },
      "source": [
        "### Checking for duplicate values"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "V0EmBHNmbp04",
      "metadata": {
        "id": "V0EmBHNmbp04"
      },
      "outputs": [],
      "source": [
        "# let's check for duplicate values in the data\n",
        "data.'_______' ##  Complete the code to check duplicate entries in the data"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "Ch_TjRfF-qT5",
      "metadata": {
        "id": "Ch_TjRfF-qT5"
      },
      "source": [
        "### Checking for missing values"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "SlwFZm-Jbp05",
      "metadata": {
        "id": "SlwFZm-Jbp05"
      },
      "outputs": [],
      "source": [
        "# let's check for missing values in the data\n",
        "data.'_______' ##  Complete the code to check missing entries in the train data"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "nUCorhch-qT4",
      "metadata": {
        "id": "nUCorhch-qT4"
      },
      "source": [
        "### Statistical summary of the dataset"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "J6lzvHKCbp06",
      "metadata": {
        "id": "J6lzvHKCbp06"
      },
      "outputs": [],
      "source": [
        "# let's view the statistical summary of the numerical columns in the data\n",
        "data.'_______' ##  Complete the code to print the statitical summary of the train data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "7b047856",
      "metadata": {
        "id": "7b047856"
      },
      "outputs": [],
      "source": [
        "data.describe(include=[\"object\"]).T"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "2c87adfc",
      "metadata": {
        "id": "2c87adfc"
      },
      "outputs": [],
      "source": [
        "for i in data.describe(include=[\"object\"]).columns:\n",
        "    print(\"Unique values in\", i, \"are :\")\n",
        "    print(data[i].value_counts())\n",
        "    print(\"*\" * 50)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "fc5c4135",
      "metadata": {
        "id": "fc5c4135"
      },
      "outputs": [],
      "source": [
        "# CLIENTNUM consists of uniques ID for clients and hence will not add value to the modeling\n",
        "data.drop([\"CLIENTNUM\"], axis=1, inplace=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "d06a79d8",
      "metadata": {
        "id": "d06a79d8"
      },
      "outputs": [],
      "source": [
        "## Encoding Existing and Attrited customers to 0 and 1 respectively, for analysis.\n",
        "data[\"Attrition_Flag\"].replace(\"Existing Customer\", 0, inplace=True)\n",
        "data[\"Attrition_Flag\"].replace(\"Attrited Customer\", 1, inplace=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "DhPuzWO7hmV8",
      "metadata": {
        "id": "DhPuzWO7hmV8"
      },
      "source": [
        "## Exploratory Data Analysis"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "-YyWJgFlKlWM",
      "metadata": {
        "id": "-YyWJgFlKlWM"
      },
      "source": [
        "#### The below functions need to be defined to carry out the Exploratory Data Analysis."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "QIP4bI3Zbp07",
      "metadata": {
        "id": "QIP4bI3Zbp07"
      },
      "outputs": [],
      "source": [
        "# function to plot a boxplot and a histogram along the same scale.\n",
        "\n",
        "\n",
        "def histogram_boxplot(data, feature, figsize=(12, 7), kde=False, bins=None):\n",
        "    \"\"\"\n",
        "    Boxplot and histogram combined\n",
        "\n",
        "    data: dataframe\n",
        "    feature: dataframe column\n",
        "    figsize: size of figure (default (12,7))\n",
        "    kde: whether to the show density curve (default False)\n",
        "    bins: number of bins for histogram (default None)\n",
        "    \"\"\"\n",
        "    f2, (ax_box2, ax_hist2) = plt.subplots(\n",
        "        nrows=2,  # Number of rows of the subplot grid= 2\n",
        "        sharex=True,  # x-axis will be shared among all subplots\n",
        "        gridspec_kw={\"height_ratios\": (0.25, 0.75)},\n",
        "        figsize=figsize,\n",
        "    )  # creating the 2 subplots\n",
        "    sns.boxplot(\n",
        "        data=data, x=feature, ax=ax_box2, showmeans=True, color=\"violet\"\n",
        "    )  # boxplot will be created and a triangle will indicate the mean value of the column\n",
        "    sns.histplot(\n",
        "        data=data, x=feature, kde=kde, ax=ax_hist2, bins=bins, palette=\"winter\"\n",
        "    ) if bins else sns.histplot(\n",
        "        data=data, x=feature, kde=kde, ax=ax_hist2\n",
        "    )  # For histogram\n",
        "    ax_hist2.axvline(\n",
        "        data[feature].mean(), color=\"green\", linestyle=\"--\"\n",
        "    )  # Add mean to the histogram\n",
        "    ax_hist2.axvline(\n",
        "        data[feature].median(), color=\"black\", linestyle=\"-\"\n",
        "    )  # Add median to the histogram"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "5021de33",
      "metadata": {
        "id": "5021de33"
      },
      "outputs": [],
      "source": [
        "# function to create labeled barplots\n",
        "\n",
        "\n",
        "def labeled_barplot(data, feature, perc=False, n=None):\n",
        "    \"\"\"\n",
        "    Barplot with percentage at the top\n",
        "\n",
        "    data: dataframe\n",
        "    feature: dataframe column\n",
        "    perc: whether to display percentages instead of count (default is False)\n",
        "    n: displays the top n category levels (default is None, i.e., display all levels)\n",
        "    \"\"\"\n",
        "\n",
        "    total = len(data[feature])  # length of the column\n",
        "    count = data[feature].nunique()\n",
        "    if n is None:\n",
        "        plt.figure(figsize=(count + 1, 5))\n",
        "    else:\n",
        "        plt.figure(figsize=(n + 1, 5))\n",
        "\n",
        "    plt.xticks(rotation=90, fontsize=15)\n",
        "    ax = sns.countplot(\n",
        "        data=data,\n",
        "        x=feature,\n",
        "        palette=\"Paired\",\n",
        "        order=data[feature].value_counts().index[:n].sort_values(),\n",
        "    )\n",
        "\n",
        "    for p in ax.patches:\n",
        "        if perc == True:\n",
        "            label = \"{:.1f}%\".format(\n",
        "                100 * p.get_height() / total\n",
        "            )  # percentage of each class of the category\n",
        "        else:\n",
        "            label = p.get_height()  # count of each level of the category\n",
        "\n",
        "        x = p.get_x() + p.get_width() / 2  # width of the plot\n",
        "        y = p.get_height()  # height of the plot\n",
        "\n",
        "        ax.annotate(\n",
        "            label,\n",
        "            (x, y),\n",
        "            ha=\"center\",\n",
        "            va=\"center\",\n",
        "            size=12,\n",
        "            xytext=(0, 5),\n",
        "            textcoords=\"offset points\",\n",
        "        )  # annotate the percentage\n",
        "\n",
        "    plt.show()  # show the plot"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "c08fe5b8",
      "metadata": {
        "id": "c08fe5b8"
      },
      "outputs": [],
      "source": [
        "# function to plot stacked bar chart\n",
        "\n",
        "def stacked_barplot(data, predictor, target):\n",
        "    \"\"\"\n",
        "    Print the category counts and plot a stacked bar chart\n",
        "\n",
        "    data: dataframe\n",
        "    predictor: independent variable\n",
        "    target: target variable\n",
        "    \"\"\"\n",
        "    count = data[predictor].nunique()\n",
        "    sorter = data[target].value_counts().index[-1]\n",
        "    tab1 = pd.crosstab(data[predictor], data[target], margins=True).sort_values(\n",
        "        by=sorter, ascending=False\n",
        "    )\n",
        "    print(tab1)\n",
        "    print(\"-\" * 120)\n",
        "    tab = pd.crosstab(data[predictor], data[target], normalize=\"index\").sort_values(\n",
        "        by=sorter, ascending=False\n",
        "    )\n",
        "    tab.plot(kind=\"bar\", stacked=True, figsize=(count + 1, 5))\n",
        "    plt.legend(\n",
        "        loc=\"lower left\", frameon=False,\n",
        "    )\n",
        "    plt.legend(loc=\"upper left\", bbox_to_anchor=(1, 1))\n",
        "    plt.show()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "e90985c5",
      "metadata": {
        "id": "e90985c5"
      },
      "outputs": [],
      "source": [
        "### Function to plot distributions\n",
        "\n",
        "def distribution_plot_wrt_target(data, predictor, target):\n",
        "\n",
        "    fig, axs = plt.subplots(2, 2, figsize=(12, 10))\n",
        "\n",
        "    target_uniq = data[target].unique()\n",
        "\n",
        "    axs[0, 0].set_title(\"Distribution of target for target=\" + str(target_uniq[0]))\n",
        "    sns.histplot(\n",
        "        data=data[data[target] == target_uniq[0]],\n",
        "        x=predictor,\n",
        "        kde=True,\n",
        "        ax=axs[0, 0],\n",
        "        color=\"teal\",\n",
        "    )\n",
        "\n",
        "    axs[0, 1].set_title(\"Distribution of target for target=\" + str(target_uniq[1]))\n",
        "    sns.histplot(\n",
        "        data=data[data[target] == target_uniq[1]],\n",
        "        x=predictor,\n",
        "        kde=True,\n",
        "        ax=axs[0, 1],\n",
        "        color=\"orange\",\n",
        "    )\n",
        "\n",
        "    axs[1, 0].set_title(\"Boxplot w.r.t target\")\n",
        "    sns.boxplot(data=data, x=target, y=predictor, ax=axs[1, 0], palette=\"gist_rainbow\")\n",
        "\n",
        "    axs[1, 1].set_title(\"Boxplot (without outliers) w.r.t target\")\n",
        "    sns.boxplot(\n",
        "        data=data,\n",
        "        x=target,\n",
        "        y=predictor,\n",
        "        ax=axs[1, 1],\n",
        "        showfliers=False,\n",
        "        palette=\"gist_rainbow\",\n",
        "    )\n",
        "\n",
        "    plt.tight_layout()\n",
        "    plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "Lv7Bs8aUbp07",
      "metadata": {
        "id": "Lv7Bs8aUbp07"
      },
      "source": [
        "### Univariate analysis"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "s7MRxT3t7qga",
      "metadata": {
        "id": "s7MRxT3t7qga"
      },
      "source": [
        "`Customer_Age`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "EM_sgrCl7qga",
      "metadata": {
        "id": "EM_sgrCl7qga"
      },
      "outputs": [],
      "source": [
        "histogram_boxplot(df1, \"Customer_Age\", kde=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "GcqAgmVV7qgb",
      "metadata": {
        "id": "GcqAgmVV7qgb"
      },
      "source": [
        "`Months_on_book`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "mSc92tz07qgb",
      "metadata": {
        "id": "mSc92tz07qgb"
      },
      "outputs": [],
      "source": [
        "histogram_boxplot('_______')  ## Complete the code to create histogram_boxplot for 'New_Price'"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "YUId8FxdW8Ym",
      "metadata": {
        "id": "YUId8FxdW8Ym"
      },
      "source": [
        "`Credit_Limit`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "qYvp9BGYZE_3",
      "metadata": {
        "id": "qYvp9BGYZE_3"
      },
      "outputs": [],
      "source": [
        "histogram_boxplot('_______')  ## Complete the code to create histogram_boxplot for 'New_Price'"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "IbhqMK1hXKIA",
      "metadata": {
        "id": "IbhqMK1hXKIA"
      },
      "source": [
        "`Total_Revolving_Bal`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "Dgd2jalyZF5x",
      "metadata": {
        "id": "Dgd2jalyZF5x"
      },
      "outputs": [],
      "source": [
        "histogram_boxplot('_______')  ## Complete the code to create histogram_boxplot for 'New_Price'"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ffd7AGe5XMRe",
      "metadata": {
        "id": "ffd7AGe5XMRe"
      },
      "source": [
        "`Avg_Open_To_Buy`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "-DcMrUshZGq-",
      "metadata": {
        "id": "-DcMrUshZGq-"
      },
      "outputs": [],
      "source": [
        "histogram_boxplot('_______')  ## Complete the code to create histogram_boxplot for 'New_Price'"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1jOTv-2YXPXU",
      "metadata": {
        "id": "1jOTv-2YXPXU"
      },
      "source": [
        "`Total_Trans_Ct`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "g9xu5JCUZHr0",
      "metadata": {
        "id": "g9xu5JCUZHr0"
      },
      "outputs": [],
      "source": [
        "histogram_boxplot('_______')  ## Complete the code to create histogram_boxplot for 'New_Price'"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "HSic71g1XSZ0",
      "metadata": {
        "id": "HSic71g1XSZ0"
      },
      "source": [
        "`Total_Amt_Chng_Q4_Q1`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "VrTWxn3wZIeC",
      "metadata": {
        "id": "VrTWxn3wZIeC"
      },
      "outputs": [],
      "source": [
        "histogram_boxplot('_______')  ## Complete the code to create histogram_boxplot for 'New_Price'"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "bt3CAWMcwCMD",
      "metadata": {
        "id": "bt3CAWMcwCMD"
      },
      "source": [
        "**Let's see total transaction amount distributed**"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "sjabW8-MXVqS",
      "metadata": {
        "id": "sjabW8-MXVqS"
      },
      "source": [
        "`Total_Trans_Amt`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "BObS4YoyZJQQ",
      "metadata": {
        "id": "BObS4YoyZJQQ"
      },
      "outputs": [],
      "source": [
        "histogram_boxplot('_______')  ## Complete the code to create histogram_boxplot for 'New_Price'"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "Ofev5koBXYqo",
      "metadata": {
        "id": "Ofev5koBXYqo"
      },
      "source": [
        "`Total_Ct_Chng_Q4_Q1`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "QXMe62QnZKMc",
      "metadata": {
        "id": "QXMe62QnZKMc"
      },
      "outputs": [],
      "source": [
        "histogram_boxplot('_______')  ## Complete the code to create histogram_boxplot for 'New_Price'"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "RgHzZOF0XcXM",
      "metadata": {
        "id": "RgHzZOF0XcXM"
      },
      "source": [
        "`Avg_Utilization_Ratio`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "bnVP5PITZLA7",
      "metadata": {
        "id": "bnVP5PITZLA7"
      },
      "outputs": [],
      "source": [
        "histogram_boxplot('_______')  ## Complete the code to create histogram_boxplot for 'New_Price'"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ZHlOYUwiXfee",
      "metadata": {
        "id": "ZHlOYUwiXfee"
      },
      "source": [
        "`Dependent_count`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "n8NU88SFYeML",
      "metadata": {
        "id": "n8NU88SFYeML"
      },
      "outputs": [],
      "source": [
        "labeled_barplot(data, \"Dependent_count\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "6MCn6s5TXmml",
      "metadata": {
        "id": "6MCn6s5TXmml"
      },
      "source": [
        "`Total_Relationship_Count`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "FeLFvGWeYjjr",
      "metadata": {
        "id": "FeLFvGWeYjjr"
      },
      "outputs": [],
      "source": [
        "labeled_barplot('_______') ## Complete the code to create labeled_barplot for 'Total_Relationship_Count'"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "_1p6YMsqXpRq",
      "metadata": {
        "id": "_1p6YMsqXpRq"
      },
      "source": [
        "`Months_Inactive_12_mon`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "I6Zw9FW5Ykr-",
      "metadata": {
        "id": "I6Zw9FW5Ykr-"
      },
      "outputs": [],
      "source": [
        "labeled_barplot('_______') ## Complete the code to create labeled_barplot for 'Months_Inactive_12_mon'"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ngVaFiyUXsCh",
      "metadata": {
        "id": "ngVaFiyUXsCh"
      },
      "source": [
        "`Contacts_Count_12_mon`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "oq0NREc8Ylhq",
      "metadata": {
        "id": "oq0NREc8Ylhq"
      },
      "outputs": [],
      "source": [
        "labeled_barplot('_______') ## Complete the code to create labeled_barplot for 'Contacts_Count_12_mon'"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "bQJndzkrXvYk",
      "metadata": {
        "id": "bQJndzkrXvYk"
      },
      "source": [
        "`Gender`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "HMpLKSURYmWs",
      "metadata": {
        "id": "HMpLKSURYmWs"
      },
      "outputs": [],
      "source": [
        "labeled_barplot('_______') ## Complete the code to create labeled_barplot for 'Gender'"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "4JEp3-biwVbB",
      "metadata": {
        "id": "4JEp3-biwVbB"
      },
      "source": [
        "**Let's see the distribution of the level of education of customers**"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "QoAM-ZO5XyVO",
      "metadata": {
        "id": "QoAM-ZO5XyVO"
      },
      "source": [
        "`Education_Level`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "DH6nG9K-YnOg",
      "metadata": {
        "id": "DH6nG9K-YnOg"
      },
      "outputs": [],
      "source": [
        "labeled_barplot('_______') ## Complete the code to create labeled_barplot for 'Education_Level'"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "7BEbgoC-X1L9",
      "metadata": {
        "id": "7BEbgoC-X1L9"
      },
      "source": [
        "`Marital_Status`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "n-6izcDzYoFp",
      "metadata": {
        "id": "n-6izcDzYoFp"
      },
      "outputs": [],
      "source": [
        "labeled_barplot('_______') ## Complete the code to create labeled_barplot for 'Marital_Status'"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "GWtkQYBBwg2F",
      "metadata": {
        "id": "GWtkQYBBwg2F"
      },
      "source": [
        "**Let's see the distribution of the level of income of customers**"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "Og8k_ygXX4Hy",
      "metadata": {
        "id": "Og8k_ygXX4Hy"
      },
      "source": [
        "`Income_Category`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "k9SZ_hz8Yo9-",
      "metadata": {
        "id": "k9SZ_hz8Yo9-"
      },
      "outputs": [],
      "source": [
        "labeled_barplot('_______') ## Complete the code to create labeled_barplot for 'Income_Category'"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ny8qxs6LX6__",
      "metadata": {
        "id": "ny8qxs6LX6__"
      },
      "source": [
        "`Card_Category`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "50wMutrcYwwG",
      "metadata": {
        "id": "50wMutrcYwwG"
      },
      "outputs": [],
      "source": [
        "labeled_barplot('_______') ## Complete the code to create labeled_barplot for 'Card_Category'"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "cDGfxaCoX97-",
      "metadata": {
        "id": "cDGfxaCoX97-"
      },
      "source": [
        "`Attrition_Flag`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "pMwj3aL5Yx0J",
      "metadata": {
        "id": "pMwj3aL5Yx0J"
      },
      "outputs": [],
      "source": [
        "labeled_barplot('_______') ## Complete the code to create labeled_barplot for 'Attrition_Flag'"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "rbY-TRxSZoOr",
      "metadata": {
        "id": "rbY-TRxSZoOr"
      },
      "outputs": [],
      "source": [
        "# creating histograms\n",
        "data.hist(figsize=(14, 14))\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "nEDRlDTm7qgg",
      "metadata": {
        "id": "nEDRlDTm7qgg"
      },
      "source": [
        "### Bivariate Distributions"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "NkY7FMkKxHw1",
      "metadata": {
        "id": "NkY7FMkKxHw1"
      },
      "source": [
        "**Let's see the attributes that have a strong correlation with each other**"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "CWhkGrqh7qgg",
      "metadata": {
        "id": "CWhkGrqh7qgg"
      },
      "source": [
        "**Correlation Check**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "9cY3QC0eYKs5",
      "metadata": {
        "id": "9cY3QC0eYKs5"
      },
      "outputs": [],
      "source": [
        "plt.figure(figsize=(15, 7))\n",
        "sns.heatmap(data.corr(), annot=True, vmin=-1, vmax=1, fmt=\".2f\", cmap=\"Spectral\")\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "svKq-pcwEXVL",
      "metadata": {
        "id": "svKq-pcwEXVL"
      },
      "source": [
        "`Attrition_Flag vs Gender`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "5l1GW5UNEUrm",
      "metadata": {
        "id": "5l1GW5UNEUrm"
      },
      "outputs": [],
      "source": [
        "stacked_barplot(data, \"Gender\", \"Attrition_Flag\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "_elOBDmmEbD8",
      "metadata": {
        "id": "_elOBDmmEbD8"
      },
      "source": [
        "`Attrition_Flag vs Marital_Status`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "NcB-8bdXEiaj",
      "metadata": {
        "id": "NcB-8bdXEiaj"
      },
      "outputs": [],
      "source": [
        "stacked_barplot(data,\"_____\", \"_____\") ## Complete the code to create distribution_plot for Attrition_Flag vs Marital_Status"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ajwwgwfhElXx",
      "metadata": {
        "id": "ajwwgwfhElXx"
      },
      "source": [
        "`Attrition_Flag vs Education_Level`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "yd-kgU9LEkpv",
      "metadata": {
        "id": "yd-kgU9LEkpv"
      },
      "outputs": [],
      "source": [
        "stacked_barplot(data,\"_____\", \"_____\") ## Complete the code to create distribution_plot for Attrition_Flag vs Education_Level"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1tv5SBX3ErDY",
      "metadata": {
        "id": "1tv5SBX3ErDY"
      },
      "source": [
        "`Attrition_Flag vs Income_Category`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "2hA530NfEvhJ",
      "metadata": {
        "id": "2hA530NfEvhJ"
      },
      "outputs": [],
      "source": [
        "stacked_barplot(data,\"_____\", \"_____\") ## Complete the code to create distribution_plot for Attrition_Flag vs Income_Category"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "vMeOF3zZF6ML",
      "metadata": {
        "id": "vMeOF3zZF6ML"
      },
      "source": [
        "`Attrition_Flag vs Contacts_Count_12_mon`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "m4mgfZ9CEzeU",
      "metadata": {
        "id": "m4mgfZ9CEzeU"
      },
      "outputs": [],
      "source": [
        "stacked_barplot(data,\"_____\", \"_____\") ## Complete the code to create distribution_plot for Attrition_Flag vs Income_Category"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "Y8vZSWIFw8IW",
      "metadata": {
        "id": "Y8vZSWIFw8IW"
      },
      "source": [
        "**Let's see the number of months a customer was inactive in the last 12 months (Months_Inactive_12_mon) vary by the customer's account status (Attrition_Flag)**"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "Mt_-F0eeF_bQ",
      "metadata": {
        "id": "Mt_-F0eeF_bQ"
      },
      "source": [
        "`Attrition_Flag vs Months_Inactive_12_mon`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "OodGfS8uGB6B",
      "metadata": {
        "id": "OodGfS8uGB6B"
      },
      "outputs": [],
      "source": [
        "stacked_barplot(data,\"_____\", \"_____\") ## Complete the code to create distribution_plot for Attrition_Flag vs Months_Inactive_12_mon"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "rzqmPfo7GHZp",
      "metadata": {
        "id": "rzqmPfo7GHZp"
      },
      "source": [
        "`Attrition_Flag vs Total_Relationship_Count`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "0Z3LDWbKEzSe",
      "metadata": {
        "id": "0Z3LDWbKEzSe"
      },
      "outputs": [],
      "source": [
        "stacked_barplot(data,\"_____\", \"_____\") ## Complete the code to create distribution_plot for Attrition_Flag vs Total_Relationship_Count"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "lSUXfkNDGUyq",
      "metadata": {
        "id": "lSUXfkNDGUyq"
      },
      "source": [
        "`Attrition_Flag vs Dependent_count`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "e8HhKTzSEzF5",
      "metadata": {
        "id": "e8HhKTzSEzF5"
      },
      "outputs": [],
      "source": [
        "stacked_barplot(data,\"_____\", \"_____\") ## Complete the code to create distribution_plot for Attrition_Flag vs Dependent_count"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "zeyTrAyxSTYL",
      "metadata": {
        "id": "zeyTrAyxSTYL"
      },
      "source": [
        "`Total_Revolving_Bal` vs `Attrition_Flag`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "d349fbf9",
      "metadata": {
        "id": "d349fbf9"
      },
      "outputs": [],
      "source": [
        "distribution_plot_wrt_target(data, \"Total_Revolving_Bal\", \"Attrition_Flag\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "MtLrPoayRW0G",
      "metadata": {
        "id": "MtLrPoayRW0G"
      },
      "source": [
        "`Attrition_Flag vs Credit_Limit`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "qphgukcMSMBm",
      "metadata": {
        "id": "qphgukcMSMBm"
      },
      "outputs": [],
      "source": [
        "distribution_plot_wrt_target(data, \"_____\", \"_____\") ## Complete the code to create distribution_plot for Attrition_Flag vs Credit_Limit"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "XLjEv417GiD1",
      "metadata": {
        "id": "XLjEv417GiD1"
      },
      "source": [
        "`Attrition_Flag vs Customer_Age`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "MPLLhKTKGt3z",
      "metadata": {
        "id": "MPLLhKTKGt3z"
      },
      "outputs": [],
      "source": [
        "distribution_plot_wrt_target(data, \"_____\", \"_____\") ## Complete the code to create distribution_plot for Attrition_Flag vs Customer_Age"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "8ddx0DPFaAY1",
      "metadata": {
        "id": "8ddx0DPFaAY1"
      },
      "source": [
        "`Total_Trans_Ct` vs `Attrition_Flag`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "e34b2b16",
      "metadata": {
        "id": "e34b2b16"
      },
      "outputs": [],
      "source": [
        "distribution_plot_wrt_target(data, \"_____\", \"_____\") ## Complete the code to create distribution_plot for Total_Trans_Ct vs Attrition_Flag"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "7YtgTTtWaI7e",
      "metadata": {
        "id": "7YtgTTtWaI7e"
      },
      "source": [
        "`Total_Trans_Amt` vs `Attrition_Flag`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "7d7e711c",
      "metadata": {
        "id": "7d7e711c"
      },
      "outputs": [],
      "source": [
        "distribution_plot_wrt_target(data, \"_____\", \"_____\") ## Complete the code to create distribution_plot for Total_Trans_Amt vs Attrition_Flag"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "jI148oudwx4Q",
      "metadata": {
        "id": "jI148oudwx4Q"
      },
      "source": [
        "**Let's see the change in transaction amount between Q4 and Q1 (total_ct_change_Q4_Q1) vary by the customer's account status (Attrition_Flag)**"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "gnRnCGgfaUUc",
      "metadata": {
        "id": "gnRnCGgfaUUc"
      },
      "source": [
        "`Total_Ct_Chng_Q4_Q1` vs `Attrition_Flag`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "c99ed6bf",
      "metadata": {
        "id": "c99ed6bf"
      },
      "outputs": [],
      "source": [
        "distribution_plot_wrt_target(data, \"_____\", \"_____\") ## Complete the code to create distribution_plot for Total_Ct_Chng_Q4_Q1 vs Attrition_Flag"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "9YO1V6OnaaFM",
      "metadata": {
        "id": "9YO1V6OnaaFM"
      },
      "source": [
        "`Avg_Utilization_Ratio` vs `Attrition_Flag`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "2e9d6fe5",
      "metadata": {
        "id": "2e9d6fe5"
      },
      "outputs": [],
      "source": [
        "distribution_plot_wrt_target(data, \"_____\", \"_____\") ## Complete the code to create distribution_plot for Avg_Utilization_Ratio vs Attrition_Flag"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "0UlATp75RNES",
      "metadata": {
        "id": "0UlATp75RNES"
      },
      "source": [
        "\n",
        "`Attrition_Flag vs Months_on_book`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "0rd_MfSQSAW0",
      "metadata": {
        "id": "0rd_MfSQSAW0"
      },
      "outputs": [],
      "source": [
        "distribution_plot_wrt_target(data, \"_____\", \"_____\") ## Complete the code to create distribution_plot for Attrition_Flag vs Months_on_book"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "09Lyk8g4SGfa",
      "metadata": {
        "id": "09Lyk8g4SGfa"
      },
      "source": [
        "`Attrition_Flag vs Total_Revolving_Bal`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "QK4QPGTrSuhZ",
      "metadata": {
        "id": "QK4QPGTrSuhZ"
      },
      "outputs": [],
      "source": [
        "distribution_plot_wrt_target(data, \"_____\", \"_____\") ## Complete the code to create distribution_plot for Attrition_Flag vs Total_Revolving_Bal"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "sgB2nOz3SyVb",
      "metadata": {
        "id": "sgB2nOz3SyVb"
      },
      "source": [
        "`Attrition_Flag vs Avg_Open_To_Buy`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "iTFLp4F_S17K",
      "metadata": {
        "id": "iTFLp4F_S17K"
      },
      "outputs": [],
      "source": [
        "distribution_plot_wrt_target(data, \"_____\", \"_____\") ## Complete the code to create distribution_plot for Attrition_Flag vs Avg_Open_To_Buy"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "QHo0SRqVp9yG",
      "metadata": {
        "id": "QHo0SRqVp9yG"
      },
      "source": [
        "## Data Preprocessing"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "p9TAxpMQaldR",
      "metadata": {
        "id": "p9TAxpMQaldR"
      },
      "source": [
        "### Outlier Detection"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "617e8bd0",
      "metadata": {
        "id": "617e8bd0"
      },
      "outputs": [],
      "source": [
        "Q1 = data.quantile(0.25)  # To find the 25th percentile\n",
        "Q3 = data.quantile(0.75)  # To find the 75th percentile\n",
        "\n",
        "IQR = Q3 - Q1  # Inter Quantile Range (75th perentile - 25th percentile)\n",
        "\n",
        "# Finding lower and upper bounds for all values. All values outside these bounds are outliers\n",
        "lower = (Q1 - 1.5 * IQR)\n",
        "upper = (Q3 + 1.5 * IQR)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "dc9689d8",
      "metadata": {
        "id": "dc9689d8"
      },
      "outputs": [],
      "source": [
        "# checking the % outliers\n",
        "((data.select_dtypes(include=[\"float64\", \"int64\"]) < lower) | (data.select_dtypes(include=[\"float64\", \"int64\"]) > upper)).sum() / len(data) * 100"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "rAiIh-pRqcIt",
      "metadata": {
        "id": "rAiIh-pRqcIt"
      },
      "source": [
        "### Train-Test Split"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "70b8307d",
      "metadata": {
        "id": "70b8307d"
      },
      "outputs": [],
      "source": [
        "# creating the copy of the dataframe\n",
        "data1 = data.copy()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "e25f10b0",
      "metadata": {
        "id": "e25f10b0"
      },
      "outputs": [],
      "source": [
        "data1[\"Income_Category\"].replace(\"_____\", np.nan, inplace=True) ### complete the code to replace the anomalous values with NaN"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "8a9274c8",
      "metadata": {
        "id": "8a9274c8"
      },
      "outputs": [],
      "source": [
        "data1.isna().sum()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "93e65425",
      "metadata": {
        "id": "93e65425"
      },
      "outputs": [],
      "source": [
        "# creating an instace of the imputer to be used\n",
        "imputer = SimpleImputer(strategy=\"most_frequent\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "f54a957a",
      "metadata": {
        "id": "f54a957a"
      },
      "outputs": [],
      "source": [
        "# Dividing train data into X and y\n",
        "\n",
        "X = data1.drop([\"Attrition_Flag\"], axis=1)\n",
        "y = data1[\"Attrition_Flag\"]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "72d57cb2",
      "metadata": {
        "id": "72d57cb2"
      },
      "outputs": [],
      "source": [
        "# Splitting data into training and validation set:\n",
        "\n",
        "X_train, X_temp, y_train, y_temp = train_test_split('_______') ## Complete the code to split the data into train test in the ratio 80:20\n",
        "\n",
        "X_test, X_val, y_test, y_val = train_test_split('_______') ## Complete the code to split the data into train test in the ratio 75:25\n",
        "\n",
        "print(X_train.shape, X_val.shape, X_test.shape)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "081ec1f8",
      "metadata": {
        "id": "081ec1f8"
      },
      "source": [
        "### Missing value imputation"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "1eedeab5",
      "metadata": {
        "id": "1eedeab5"
      },
      "outputs": [],
      "source": [
        "reqd_col_for_impute = [\"Education_Level\", \"Marital_Status\", \"Income_Category\"]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "6a9ea1cb",
      "metadata": {
        "id": "6a9ea1cb"
      },
      "outputs": [],
      "source": [
        "# Fit and transform the train data\n",
        "X_train[reqd_col_for_impute] = imputer.fit_transform(X_train[reqd_col_for_impute])\n",
        "\n",
        "# Transform the validation data\n",
        "X_val[reqd_col_for_impute]  =  '_______' ## Complete the code to impute missing values in X_val\n",
        "\n",
        "# Transform the test data\n",
        "X_test[reqd_col_for_impute] = '_______' ## Complete the code to impute missing values in X_test"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "c86c92e4",
      "metadata": {
        "id": "c86c92e4"
      },
      "outputs": [],
      "source": [
        "# Checking that no column has missing values in train or test sets\n",
        "print(X_train.isna().sum())\n",
        "print(\"-\" * 30)\n",
        "print(X_val.isna().sum())\n",
        "print(\"-\" * 30)\n",
        "print(X_test.isna().sum())"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "29d06e61",
      "metadata": {
        "id": "29d06e61"
      },
      "outputs": [],
      "source": [
        "cols = X_train.select_dtypes(include=[\"object\", \"category\"])\n",
        "for i in cols.columns:\n",
        "    print(X_train[i].value_counts())\n",
        "    print(\"*\" * 30)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "136c6022",
      "metadata": {
        "id": "136c6022"
      },
      "outputs": [],
      "source": [
        "cols = X_val.select_dtypes(include=[\"object\", \"category\"])\n",
        "for i in cols.columns:\n",
        "    print(X_val[i].value_counts())\n",
        "    print(\"*\" * 30)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "09ce8c1f",
      "metadata": {
        "id": "09ce8c1f"
      },
      "outputs": [],
      "source": [
        "cols = X_test.select_dtypes(include=[\"object\", \"category\"])\n",
        "for i in cols.columns:\n",
        "    print(X_train[i].value_counts())\n",
        "    print(\"*\" * 30)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1db1b914",
      "metadata": {
        "id": "1db1b914"
      },
      "source": [
        "### Encoding categorical variables"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "bc60ffe4",
      "metadata": {
        "id": "bc60ffe4"
      },
      "outputs": [],
      "source": [
        "X_train = pd.get_dummies(X_train, drop_first=True)\n",
        "X_val = '_______'  ## Complete the code to impute missing values in X_val\n",
        "X_test = '_______'  ## Complete the code to impute missing values in X_val\n",
        "print(X_train.shape, X_val.shape, X_test.shape)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "b2c34f29",
      "metadata": {
        "id": "b2c34f29"
      },
      "source": [
        "* After encoding there are 29 columns."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "5e41bd76",
      "metadata": {
        "id": "5e41bd76"
      },
      "outputs": [],
      "source": [
        "# check the top 5 rows from the train dataset\n",
        "X_train.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "OzOa9FGA6WtG",
      "metadata": {
        "id": "OzOa9FGA6WtG"
      },
      "source": [
        "## Model Building"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "YZqmoqz7bp0-",
      "metadata": {
        "id": "YZqmoqz7bp0-"
      },
      "source": [
        "### Model evaluation criterion"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "l2ORUgmUjDZC",
      "metadata": {
        "id": "l2ORUgmUjDZC"
      },
      "source": [
        "Model can make wrong predictions as:\n",
        "\n",
        "- Predicting a customer will attrite and the customer doesn't attrite\n",
        "- Predicting a customer will not attrite and the customer attrites\n",
        "\n",
        "Which case is more important?\n",
        "\n",
        "- Predicting that customer will not attrite but he attrites i.e. losing on a valuable customer or asset.\n",
        "\n",
        "**How to reduce this loss i.e need to reduce False Negatives??**\n",
        "\n",
        "- Bank would want Recall to be maximized, greater the Recall higher the chances of minimizing false negatives. Hence, the focus should be on increasing Recall or minimizing the false negatives or in other words identifying the true positives(i.e. Class 1) so that the bank can retain their valuable customers by identifying the customers who are at risk of attrition."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "VUs837UXbp0-",
      "metadata": {
        "id": "VUs837UXbp0-"
      },
      "source": [
        "**Let's define a function to output different metrics (including recall) on the train and test set and a function to show confusion matrix so that we do not have to use the same code repetitively while evaluating models.**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "28675acc",
      "metadata": {
        "id": "28675acc"
      },
      "outputs": [],
      "source": [
        "# defining a function to compute different metrics to check performance of a classification model built using sklearn\n",
        "def model_performance_classification_sklearn(model, predictors, target):\n",
        "    \"\"\"\n",
        "    Function to compute different metrics to check classification model performance\n",
        "\n",
        "    model: classifier\n",
        "    predictors: independent variables\n",
        "    target: dependent variable\n",
        "    \"\"\"\n",
        "\n",
        "    # predicting using the independent variables\n",
        "    pred = model.predict(predictors)\n",
        "\n",
        "    acc = accuracy_score(target, pred)  # to compute Accuracy\n",
        "    recall = recall_score(target, pred)  # to compute Recall\n",
        "    precision = precision_score(target, pred)  # to compute Precision\n",
        "    f1 = f1_score(target, pred)  # to compute F1-score\n",
        "\n",
        "    # creating a dataframe of metrics\n",
        "    df_perf = pd.DataFrame(\n",
        "        {\"Accuracy\": acc, \"Recall\": recall, \"Precision\": precision, \"F1\": f1,},\n",
        "        index=[0],\n",
        "    )\n",
        "\n",
        "    return df_perf"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "8d3adba0",
      "metadata": {
        "id": "8d3adba0"
      },
      "outputs": [],
      "source": [
        "def confusion_matrix_sklearn(model, predictors, target):\n",
        "    \"\"\"\n",
        "    To plot the confusion_matrix with percentages\n",
        "\n",
        "    model: classifier\n",
        "    predictors: independent variables\n",
        "    target: dependent variable\n",
        "    \"\"\"\n",
        "    y_pred = model.predict(predictors)\n",
        "    cm = confusion_matrix(target, y_pred)\n",
        "    labels = np.asarray(\n",
        "        [\n",
        "            [\"{0:0.0f}\".format(item) + \"\\n{0:.2%}\".format(item / cm.flatten().sum())]\n",
        "            for item in cm.flatten()\n",
        "        ]\n",
        "    ).reshape(2, 2)\n",
        "\n",
        "    plt.figure(figsize=(6, 4))\n",
        "    sns.heatmap(cm, annot=labels, fmt=\"\")\n",
        "    plt.ylabel(\"True label\")\n",
        "    plt.xlabel(\"Predicted label\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "cdea3372",
      "metadata": {
        "id": "cdea3372"
      },
      "source": [
        "### Model Building - Original Data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "be50c6d3",
      "metadata": {
        "id": "be50c6d3"
      },
      "outputs": [],
      "source": [
        "models = []  # Empty list to store all the models\n",
        "\n",
        "# Appending models into the list\n",
        "models.append((\"Bagging\", BaggingClassifier(random_state=1)))\n",
        "models.append((\"Random forest\", RandomForestClassifier(random_state=1)))\n",
        "'_______' ## Complete the code to append remaining 3 models in the list models\n",
        "\n",
        "print(\"\\n\" \"Training Performance:\" \"\\n\")\n",
        "for name, model in models:\n",
        "    model.fit(X_train, y_train)\n",
        "    scores = recall_score(y_train, model.predict(X_train))\n",
        "    print(\"{}: {}\".format(name, scores))\n",
        "\n",
        "print(\"\\n\" \"Validation Performance:\" \"\\n\")\n",
        "\n",
        "for name, model in models:\n",
        "    model.fit(X_train, y_train)\n",
        "    scores_val = recall_score(y_val, model.predict(X_val))\n",
        "    print(\"{}: {}\".format(name, scores_val))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "d6cf8a19",
      "metadata": {
        "id": "d6cf8a19"
      },
      "source": [
        "### Model Building - Oversampled Data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "6b25ae18",
      "metadata": {
        "id": "6b25ae18"
      },
      "outputs": [],
      "source": [
        "print(\"Before Oversampling, counts of label 'Yes': {}\".format(sum(y_train == 1)))\n",
        "print(\"Before Oversampling, counts of label 'No': {} \\n\".format(sum(y_train == 0)))\n",
        "\n",
        "sm = SMOTE(\n",
        "    sampling_strategy=1, k_neighbors=5, random_state=1\n",
        ")  # Synthetic Minority Over Sampling Technique\n",
        "X_train_over, y_train_over = sm.fit_resample(X_train, y_train)\n",
        "\n",
        "\n",
        "print(\"After Oversampling, counts of label 'Yes': {}\".format(sum(y_train_over == 1)))\n",
        "print(\"After Oversampling, counts of label 'No': {} \\n\".format(sum(y_train_over == 0)))\n",
        "\n",
        "\n",
        "print(\"After Oversampling, the shape of train_X: {}\".format(X_train_over.shape))\n",
        "print(\"After Oversampling, the shape of train_y: {} \\n\".format(y_train_over.shape))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "e5a457f5",
      "metadata": {
        "id": "e5a457f5"
      },
      "outputs": [],
      "source": [
        "models = []  # Empty list to store all the models\n",
        "\n",
        "# Appending models into the list\n",
        "models.append((\"Bagging\", BaggingClassifier(random_state=1)))\n",
        "models.append((\"Random forest\", RandomForestClassifier(random_state=1)))\n",
        "'_______' ## Complete the code to append remaining 3 models in the list models\n",
        "\n",
        "\n",
        "print(\"\\n\" \"Training Performance:\" \"\\n\")\n",
        "for name, model in models:\n",
        "    model.fit(X_train_over, y_train_over)\n",
        "    scores = recall_score(_____, model.predict(_____))  ## Complete the code to build models on oversampled data\n",
        "    print(\"{}: {}\".format(name, scores))\n",
        "\n",
        "print(\"\\n\" \"Validation Performance:\" \"\\n\")\n",
        "\n",
        "for name, model in models:\n",
        "    model.fit(X_train_over, y_train_over)\n",
        "    scores = recall_score(y_val, model.predict(X_val))\n",
        "    print(\"{}: {}\".format(name, scores))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "62f1f452",
      "metadata": {
        "id": "62f1f452"
      },
      "source": [
        "### Model Building - Undersampled Data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "f3a65f4c",
      "metadata": {
        "id": "f3a65f4c"
      },
      "outputs": [],
      "source": [
        "rus = RandomUnderSampler(random_state=1)\n",
        "X_train_un, y_train_un = rus.fit_resample(X_train, y_train)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "580b5abc",
      "metadata": {
        "id": "580b5abc"
      },
      "outputs": [],
      "source": [
        "print(\"Before Under Sampling, counts of label 'Yes': {}\".format(sum(y_train == 1)))\n",
        "print(\"Before Under Sampling, counts of label 'No': {} \\n\".format(sum(y_train == 0)))\n",
        "\n",
        "print(\"After Under Sampling, counts of label 'Yes': {}\".format(sum(y_train_un == 1)))\n",
        "print(\"After Under Sampling, counts of label 'No': {} \\n\".format(sum(y_train_un == 0)))\n",
        "\n",
        "print(\"After Under Sampling, the shape of train_X: {}\".format(X_train_un.shape))\n",
        "print(\"After Under Sampling, the shape of train_y: {} \\n\".format(y_train_un.shape))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "e674701b",
      "metadata": {
        "id": "e674701b"
      },
      "outputs": [],
      "source": [
        "models = []  # Empty list to store all the models\n",
        "\n",
        "# Appending models into the list\n",
        "models.append((\"Bagging\", BaggingClassifier(random_state=1)))\n",
        "models.append((\"Random forest\", RandomForestClassifier(random_state=1)))\n",
        "'_______' ## Complete the code to append remaining 3 models in the list models\n",
        "\n",
        "\n",
        "print(\"\\n\" \"Training Performance:\" \"\\n\")\n",
        "for name, model in models:\n",
        "    model.fit(X_train_un, y_train_un)\n",
        "    scores = recall_score(____, model.predict(____))  ## Complete the code to build models on undersampled data\n",
        "    print(\"{}: {}\".format(name, scores))\n",
        "\n",
        "print(\"\\n\" \"Validation Performance:\" \"\\n\")\n",
        "\n",
        "for name, model in models:\n",
        "    model.fit(X_train_un, y_train_un)\n",
        "    scores = recall_score(y_val, model.predict(X_val))\n",
        "    print(\"{}: {}\".format(name, scores))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "zLZlKa99bp1C",
      "metadata": {
        "id": "zLZlKa99bp1C"
      },
      "source": [
        "### Hyperparameter Tuning"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "tFEScXRsl9NM",
      "metadata": {
        "id": "tFEScXRsl9NM"
      },
      "source": [
        "#### **Note**\n",
        "\n",
        "1. Sample parameter grids have been provided to do necessary hyperparameter tuning. These sample grids are expected to provide a balance between model performance improvement and execution time. One can extend/reduce the parameter grid based on execution time and system configuration.\n",
        "  - Please note that if the parameter grid is extended to improve the model performance further, the execution time will increase\n",
        "2. The models chosen in this notebook are based on test runs. One can update the best models as obtained upon code execution and tune them for best performance.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "K-xL2RGU_SdH",
      "metadata": {
        "id": "K-xL2RGU_SdH"
      },
      "source": [
        "#### Tuning AdaBoost using original data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "7f179c6d",
      "metadata": {
        "id": "7f179c6d"
      },
      "outputs": [],
      "source": [
        "%%time\n",
        "\n",
        "# defining model\n",
        "Model = AdaBoostClassifier(random_state=1)\n",
        "\n",
        "# Parameter grid to pass in RandomSearchCV\n",
        "param_grid = {\n",
        "    \"n_estimators\": np.arange(50,110,25),\n",
        "    \"learning_rate\": [0.01,0.1,0.05],\n",
        "    \"base_estimator\": [\n",
        "        DecisionTreeClassifier(max_depth=2, random_state=1),\n",
        "        DecisionTreeClassifier(max_depth=3, random_state=1),\n",
        "    ],\n",
        "}\n",
        "\n",
        "# Type of scoring used to compare parameter combinations\n",
        "scorer = metrics.make_scorer(metrics.recall_score)\n",
        "\n",
        "#Calling RandomizedSearchCV\n",
        "randomized_cv = RandomizedSearchCV(estimator=Model, param_distributions=param_grid, n_jobs = -1, n_iter=50, scoring=scorer, cv=5, random_state=1)\n",
        "\n",
        "#Fitting parameters in RandomizedSearchCV\n",
        "randomized_cv.'_______' ## Complete the code to fit the model on original data\n",
        "\n",
        "print(\"Best parameters are {} with CV score={}:\" .format(randomized_cv.best_params_,randomized_cv.best_score_))"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Creating new pipeline with best parameters\n",
        "tuned_adb = AdaBoostClassifier( random_state=___,\n",
        "    n_estimators= _______, learning_rate= _______, base_estimator= DecisionTreeClassifier(max_depth=_______, random_state=1)\n",
        ") ## Complete the code with the best parameters obtained from tuning\n",
        "\n",
        "tuned_ada.'_______' ## Complete the code to fit the model on original data"
      ],
      "metadata": {
        "id": "kdB4rlv28eHs"
      },
      "id": "kdB4rlv28eHs",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "adb_train = model_performance_classification_sklearn(tuned_adb, ______, _____) ## Complete the code to check the performance on training set\n",
        "adb_train"
      ],
      "metadata": {
        "id": "nEhn8ao98d_C"
      },
      "id": "nEhn8ao98d_C",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Checking model's performance on validation set\n",
        "adb_val =  model_performance_classification_sklearn(tuned_adb, ______, _____) ## Complete the code to check the performance on validation set\n",
        "adb_val"
      ],
      "metadata": {
        "id": "oTpzexpz8mOa"
      },
      "id": "oTpzexpz8mOa",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "id": "6eaa2bb2",
      "metadata": {
        "id": "6eaa2bb2"
      },
      "source": [
        "#### Tuning Ada Boost using undersampled data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "2qG1EcwH_SdI",
      "metadata": {
        "id": "2qG1EcwH_SdI"
      },
      "outputs": [],
      "source": [
        "# Creating new pipeline with best parameters\n",
        "tuned_ada2 = AdaBoostClassifier( random_state=___,\n",
        "    n_estimators= _______, learning_rate= _______, base_estimator= DecisionTreeClassifier(max_depth=_______, random_state=1)\n",
        ") ## Complete the code with the best parameters obtained from tuning\n",
        "\n",
        "tuned_ada2.'_______' ## Complete the code to fit the model on undersampled data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "oaGGaBXf_SdI",
      "metadata": {
        "id": "oaGGaBXf_SdI",
        "scrolled": true
      },
      "outputs": [],
      "source": [
        "adb2_train = '_______' ## Complete the code to check the performance on training set\n",
        "adb2_train"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "oyaP_jkM_SdJ",
      "metadata": {
        "id": "oyaP_jkM_SdJ",
        "scrolled": false
      },
      "outputs": [],
      "source": [
        "# Checking model's performance on validation set\n",
        "adb2_val =  '_______' ## Complete the code to check the performance on validation set\n",
        "adb2_val"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "XoLT8ewJ5V2d",
      "metadata": {
        "id": "XoLT8ewJ5V2d"
      },
      "source": [
        "#### Tuning Gradient Boosting using undersampled data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "a8619e93",
      "metadata": {
        "id": "a8619e93"
      },
      "outputs": [],
      "source": [
        "%%time\n",
        "\n",
        "#Creating pipeline\n",
        "Model = GradientBoostingClassifier(random_state=1)\n",
        "\n",
        "#Parameter grid to pass in RandomSearchCV\n",
        "param_grid = {\n",
        "    \"init\": [AdaBoostClassifier(random_state=1),DecisionTreeClassifier(random_state=1)],\n",
        "    \"n_estimators\": np.arange(50,110,25),\n",
        "    \"learning_rate\": [0.01,0.1,0.05],\n",
        "    \"subsample\":[0.7,0.9],\n",
        "    \"max_features\":[0.5,0.7,1],\n",
        "}\n",
        "\n",
        "# Type of scoring used to compare parameter combinations\n",
        "scorer = metrics.make_scorer(metrics.recall_score)\n",
        "\n",
        "#Calling RandomizedSearchCV\n",
        "randomized_cv = RandomizedSearchCV(estimator=Model, param_distributions=param_grid, n_iter=50, scoring=scorer, cv=5, random_state=1, n_jobs = -1)\n",
        "\n",
        "#Fitting parameters in RandomizedSearchCV\n",
        "randomized_cv.'_______' ## Complete the code to fit the model on under sampled data\n",
        "\n",
        "\n",
        "print(\"Best parameters are {} with CV score={}:\" .format(randomized_cv.best_params_,randomized_cv.best_score_))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "j_61Omhq5KkB",
      "metadata": {
        "id": "j_61Omhq5KkB"
      },
      "outputs": [],
      "source": [
        "# Creating new pipeline with best parameters\n",
        "tuned_gbm1 = GradientBoostingClassifier(\n",
        "    max_features=_______,\n",
        "    init=AdaBoostClassifier(random_state=1),\n",
        "    random_state=1,\n",
        "    learning_rate=_______,\n",
        "    n_estimators=_______,\n",
        "    subsample=_______,\n",
        ")## Complete the code with the best parameters obtained from tuning\n",
        "\n",
        "tuned_gbm1.fit(X_train_un, y_train_un)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "k8Oxv7PIJTmF",
      "metadata": {
        "id": "k8Oxv7PIJTmF"
      },
      "outputs": [],
      "source": [
        "gbm1_train = '_______' ## Complete the code to check the performance on undersampled train set\n",
        "gbm1_train"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "wpcbbrBLJjS5",
      "metadata": {
        "id": "wpcbbrBLJjS5"
      },
      "outputs": [],
      "source": [
        "gbm1_val = '_______' ## Complete the code to check the performance on validation set\n",
        "gbm1_val"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "X0fOfEEdBXay",
      "metadata": {
        "id": "X0fOfEEdBXay"
      },
      "source": [
        "#### Tuning Gradient Boosting using original data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "d3e4048d",
      "metadata": {
        "id": "d3e4048d"
      },
      "outputs": [],
      "source": [
        "%%time\n",
        "\n",
        "#defining model\n",
        "Model = GradientBoostingClassifier(random_state=1)\n",
        "\n",
        "#Parameter grid to pass in RandomSearchCV\n",
        "param_grid = {\n",
        "    \"init\": [AdaBoostClassifier(random_state=1),DecisionTreeClassifier(random_state=1)],\n",
        "    \"n_estimators\": np.arange(50,110,25),\n",
        "    \"learning_rate\": [0.01,0.1,0.05],\n",
        "    \"subsample\":[0.7,0.9],\n",
        "    \"max_features\":[0.5,0.7,1],\n",
        "}\n",
        "\n",
        "# Type of scoring used to compare parameter combinations\n",
        "scorer = metrics.make_scorer(metrics.recall_score)\n",
        "\n",
        "#Calling RandomizedSearchCV\n",
        "randomized_cv = RandomizedSearchCV(estimator=Model, param_distributions=param_grid, n_iter=50, scoring=scorer, cv=5, random_state=1, n_jobs = -1)\n",
        "\n",
        "#Fitting parameters in RandomizedSearchCV\n",
        "randomized_cv.'_______' ## Complete the code to fit the model on original data\n",
        "\n",
        "\n",
        "print(\"Best parameters are {} with CV score={}:\" .format(randomized_cv.best_params_,randomized_cv.best_score_))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "2YlugBs8Bewz",
      "metadata": {
        "id": "2YlugBs8Bewz"
      },
      "outputs": [],
      "source": [
        "# Creating new pipeline with best parameters\n",
        "tuned_gbm2 = GradientBoostingClassifier(\n",
        "    max_features=_______,\n",
        "    init=AdaBoostClassifier(random_state=1),\n",
        "    random_state=1,\n",
        "    learning_rate=_______,\n",
        "    n_estimators=_______,\n",
        "    subsample=_______,\n",
        ")## Complete the code with the best parameters obtained from tuning\n",
        "\n",
        "tuned_gbm2.fit(X_train, y_train)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "687f5e20",
      "metadata": {
        "id": "687f5e20"
      },
      "source": [
        "#### Tuning Gradient Boosting using over sampled data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ilYVyajmBew0",
      "metadata": {
        "id": "ilYVyajmBew0"
      },
      "outputs": [],
      "source": [
        "gbm2_train = '_______' ## Complete the code to check the performance on oversampled train set\n",
        "gbm2_train"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ptkIcRnjBew0",
      "metadata": {
        "id": "ptkIcRnjBew0"
      },
      "outputs": [],
      "source": [
        "gbm2_val = '_______' ## Complete the code to check the performance on validation set\n",
        "gbm2_val"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "fPGqokBQL7RN",
      "metadata": {
        "id": "fPGqokBQL7RN"
      },
      "source": [
        "#### Tuning XGBoost Model with Original data"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Note**: This section is optional. You can choose not to build XGBoost if you are facing issues with installation or if it is taking more time to execute.\n",
        "\n"
      ],
      "metadata": {
        "id": "20KUhfedFXLA"
      },
      "id": "20KUhfedFXLA"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "E6GtcXaGLz9S",
      "metadata": {
        "id": "E6GtcXaGLz9S"
      },
      "outputs": [],
      "source": [
        "%%time\n",
        "\n",
        "# defining model\n",
        "Model = XGBClassifier(random_state=1,eval_metric='logloss')\n",
        "\n",
        "#Parameter grid to pass in RandomSearchCV\n",
        "param_grid={'n_estimators':np.arange(50,110,25),\n",
        "            'scale_pos_weight':[1,2,5],\n",
        "            'learning_rate':[0.01,0.1,0.05],\n",
        "            'gamma':[1,3],\n",
        "            'subsample':[0.7,0.9]\n",
        "           }\n",
        "from sklearn import metrics\n",
        "\n",
        "# Type of scoring used to compare parameter combinations\n",
        "scorer = metrics.make_scorer(metrics.recall_score)\n",
        "\n",
        "#Calling RandomizedSearchCV\n",
        "randomized_cv = RandomizedSearchCV(estimator=Model, param_distributions=param_grid, n_iter=50, n_jobs = -1, scoring=scorer, cv=5, random_state=1)\n",
        "\n",
        "#Fitting parameters in RandomizedSearchCV\n",
        "randomized_cv.'_______' ## Complete the code to fit the model on original data\n",
        "\n",
        "print(\"Best parameters are {} with CV score={}:\" .format(randomized_cv.best_params_,randomized_cv.best_score_))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "PaQ9bLyyMIV6",
      "metadata": {
        "id": "PaQ9bLyyMIV6"
      },
      "outputs": [],
      "source": [
        "tuned_xgb = XGBClassifier(\n",
        "    random_state=1,\n",
        "    eval_metric=\"logloss\",\n",
        "    subsample=____,\n",
        "    scale_pos_weight=____,\n",
        "    n_estimators=____,\n",
        "    learning_rate=______,\n",
        "    gamma=1,\n",
        ")## Complete the code with the best parameters obtained from tuning\n",
        "\n",
        "tuned_xgb.fit(X_train, y_train)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "Uv5YEDfLMaTR",
      "metadata": {
        "id": "Uv5YEDfLMaTR"
      },
      "outputs": [],
      "source": [
        "xgb_train = '_______' ## Complete the code to check the performance on original train set\n",
        "xgb_train"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "6c-2O5N5Mjhu",
      "metadata": {
        "id": "6c-2O5N5Mjhu"
      },
      "outputs": [],
      "source": [
        "xgb_val = '_______' ## Complete the code to check the performance on validation set\n",
        "xgb_val"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "D9JNnpxa4jau",
      "metadata": {
        "id": "D9JNnpxa4jau"
      },
      "source": [
        "## Model Comparison and Final Model Selection"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "G4OokQOFSjJ6",
      "metadata": {
        "id": "G4OokQOFSjJ6"
      },
      "source": [
        "**Note**: If you want to include XGBoost model for final model selection, you need to add **xgb_train.T** in the training performance comparison list and **xgb_val.T** in the validation performance comparison list below."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "1c501f92",
      "metadata": {
        "id": "1c501f92"
      },
      "outputs": [],
      "source": [
        "# training performance comparison\n",
        "\n",
        "models_train_comp_df = pd.concat(\n",
        "    [\n",
        "        gbm1_train.T,\n",
        "        gbm2_train.T,\n",
        "        adb2_train.T,\n",
        "    ],\n",
        "    axis=1,\n",
        ")\n",
        "models_train_comp_df.columns = [\n",
        "    \"Gradient boosting trained with Undersampled data\",\n",
        "    \"Gradient boosting trained with Original data\",\n",
        "    \"AdaBoost trained with Undersampled data\",\n",
        "]\n",
        "print(\"Training performance comparison:\")\n",
        "models_train_comp_df"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "0FgTXpsXbp1F",
      "metadata": {
        "id": "0FgTXpsXbp1F"
      },
      "outputs": [],
      "source": [
        "# validation performance comparison\n",
        "\n",
        "'_______' ## Write the code to compare the performance on validation set"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "PYS5m_mcbp1F",
      "metadata": {
        "id": "PYS5m_mcbp1F"
      },
      "source": [
        "**Now we have our final model, so let's find out how our final model is performing on unseen test data.**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "rqgvz7e7bp1F",
      "metadata": {
        "id": "rqgvz7e7bp1F"
      },
      "outputs": [],
      "source": [
        "# Let's check the performance on test set\n",
        "'_______' ## Write the code to check the performance of best model on test data"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "J49s-TEB41JQ",
      "metadata": {
        "id": "J49s-TEB41JQ"
      },
      "source": [
        "### Feature Importances"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "7Zk2dFRzbp1F",
      "metadata": {
        "id": "7Zk2dFRzbp1F"
      },
      "outputs": [],
      "source": [
        "feature_names = X_train.columns\n",
        "importances =  '_______' ## Complete the code to check the feature importance of the best model\n",
        "indices = np.argsort(importances)\n",
        "\n",
        "plt.figure(figsize=(12, 12))\n",
        "plt.title(\"Feature Importances\")\n",
        "plt.barh(range(len(indices)), importances[indices], color=\"violet\", align=\"center\")\n",
        "plt.yticks(range(len(indices)), [feature_names[i] for i in indices])\n",
        "plt.xlabel(\"Relative Importance\")\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "gxFmwam_bp1M",
      "metadata": {
        "id": "gxFmwam_bp1M"
      },
      "source": [
        "# Business Insights and Conclusions\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "RZN2YkUhq77C",
      "metadata": {
        "id": "RZN2YkUhq77C"
      },
      "source": [
        "-\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "MyLBn7uE6LhP",
      "metadata": {
        "id": "MyLBn7uE6LhP"
      },
      "source": [
        "***"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [
        "9EaJ8AGwpM-2",
        "xxhpZv9y-qTw",
        "M1TEog7cAs5m",
        "KQi5ygTC-qT1",
        "961px703qhLV",
        "5TcqcxbK-qT3",
        "xNr4bWoM-qT5",
        "Ch_TjRfF-qT5",
        "nUCorhch-qT4",
        "DhPuzWO7hmV8",
        "QHo0SRqVp9yG",
        "p9TAxpMQaldR",
        "rAiIh-pRqcIt",
        "081ec1f8",
        "1db1b914",
        "OzOa9FGA6WtG",
        "YZqmoqz7bp0-",
        "cdea3372",
        "d6cf8a19",
        "62f1f452",
        "K-xL2RGU_SdH",
        "6eaa2bb2",
        "XoLT8ewJ5V2d",
        "X0fOfEEdBXay",
        "687f5e20",
        "fPGqokBQL7RN",
        "D9JNnpxa4jau",
        "J49s-TEB41JQ",
        "gxFmwam_bp1M"
      ],
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.8"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}