venture_funding_with_deep_learning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Venture Funding with Deep Learning\n",
    "\n",
    "You work as a risk management associate at Alphabet Soup, a venture capital firm. Alphabet Soup’s business team receives many funding applications from startups every day. This team has asked you to help them create a model that predicts whether applicants will be successful if funded by Alphabet Soup.\n",
    "\n",
    "The business team has given you a CSV containing more than 34,000 organizations that have received funding from Alphabet Soup over the years. With your knowledge of machine learning and neural networks, you decide to use the features in the provided dataset to create a binary classifier model that will predict whether an applicant will become a successful business. The CSV file contains a variety of information about these businesses, including whether or not they ultimately became successful.\n",
    "\n",
    "## Instructions:\n",
    "\n",
    "The steps for this challenge are broken out into the following sections:\n",
    "\n",
    "* Prepare the data for use on a neural network model.\n",
    "\n",
    "* Compile and evaluate a binary classification model using a neural network.\n",
    "\n",
    "* Optimize the neural network model.\n",
    "\n",
    "### Prepare the Data for Use on a Neural Network Model \n",
    "\n",
    "Using your knowledge of Pandas and scikit-learn’s `StandardScaler()`, preprocess the dataset so that you can use it to compile and evaluate the neural network model later.\n",
    "\n",
    "Open the starter code file, and complete the following data preparation steps:\n",
    "\n",
    "1. Read the `applicants_data.csv` file into a Pandas DataFrame. Review the DataFrame, looking for categorical variables that will need to be encoded, as well as columns that could eventually define your features and target variables.   \n",
    "\n",
    "2. Drop the “EIN” (Employer Identification Number) and “NAME” columns from the DataFrame, because they are not relevant to the binary classification model.\n",
    " \n",
    "3. Encode the dataset’s categorical variables using `OneHotEncoder`, and then place the encoded variables into a new DataFrame.\n",
    "\n",
    "4. Add the original DataFrame’s numerical variables to the DataFrame containing the encoded variables.\n",
    "\n",
    "> **Note** To complete this step, you will employ the Pandas `concat()` function that was introduced earlier in this course. \n",
    "\n",
    "5. Using the preprocessed data, create the features (`X`) and target (`y`) datasets. The target dataset should be defined by the preprocessed DataFrame column “IS_SUCCESSFUL”. The remaining columns should define the features dataset. \n",
    "\n",
    "6. Split the features and target sets into training and testing datasets.\n",
    "\n",
    "7. Use scikit-learn's `StandardScaler` to scale the features data.\n",
    "\n",
    "### Compile and Evaluate a Binary Classification Model Using a Neural Network\n",
    "\n",
    "Use your knowledge of TensorFlow to design a binary classification deep neural network model. This model should use the dataset’s features to predict whether an Alphabet Soup&ndash;funded startup will be successful based on the features in the dataset. Consider the number of inputs before determining the number of layers that your model will contain or the number of neurons on each layer. Then, compile and fit your model. Finally, evaluate your binary classification model to calculate the model’s loss and accuracy. \n",
    " \n",
    "To do so, complete the following steps:\n",
    "\n",
    "1. Create a deep neural network by assigning the number of input features, the number of layers, and the number of neurons on each layer using Tensorflow’s Keras.\n",
    "\n",
    "> **Hint** You can start with a two-layer deep neural network model that uses the `relu` activation function for both layers.\n",
    "\n",
    "2. Compile and fit the model using the `binary_crossentropy` loss function, the `adam` optimizer, and the `accuracy` evaluation metric.\n",
    "\n",
    "> **Hint** When fitting the model, start with a small number of epochs, such as 20, 50, or 100.\n",
    "\n",
    "3. Evaluate the model using the test data to determine the model’s loss and accuracy.\n",
    "\n",
    "4. Save and export your model to an HDF5 file, and name the file `AlphabetSoup.h5`. \n",
    "\n",
    "### Optimize the Neural Network Model\n",
    "\n",
    "Using your knowledge of TensorFlow and Keras, optimize your model to improve the model's accuracy. Even if you do not successfully achieve a better accuracy, you'll need to demonstrate at least two attempts to optimize the model. You can include these attempts in your existing notebook. Or, you can make copies of the starter notebook in the same folder, rename them, and code each model optimization in a new notebook. \n",
    "\n",
    "> **Note** You will not lose points if your model does not achieve a high accuracy, as long as you make at least two attempts to optimize the model.\n",
    "\n",
    "To do so, complete the following steps:\n",
    "\n",
    "1. Define at least three new deep neural network models (the original plus 2 optimization attempts). With each, try to improve on your first model’s predictive accuracy.\n",
    "\n",
    "> **Rewind** Recall that perfect accuracy has a value of 1, so accuracy improves as its value moves closer to 1. To optimize your model for a predictive accuracy as close to 1 as possible, you can use any or all of the following techniques:\n",
    ">\n",
    "> * Adjust the input data by dropping different features columns to ensure that no variables or outliers confuse the model.\n",
    ">\n",
    "> * Add more neurons (nodes) to a hidden layer.\n",
    ">\n",
    "> * Add more hidden layers.\n",
    ">\n",
    "> * Use different activation functions for the hidden layers.\n",
    ">\n",
    "> * Add to or reduce the number of epochs in the training regimen.\n",
    "\n",
    "2. After finishing your models, display the accuracy scores achieved by each model, and compare the results.\n",
    "\n",
    "3. Save each of your models as an HDF5 file.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Imports\n",
    "import pandas as pd\n",
    "from pathlib import Path\n",
    "import tensorflow as tf\n",
    "from tensorflow.keras.layers import Dense\n",
    "from tensorflow.keras.models import Sequential\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import StandardScaler,OneHotEncoder"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Prepare the data to be used on a neural network model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1: Read the `applicants_data.csv` file into a Pandas DataFrame. Review the DataFrame, looking for categorical variables that will need to be encoded, as well as columns that could eventually define your features and target variables.  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>EIN</th>\n",
       "      <th>NAME</th>\n",
       "      <th>APPLICATION_TYPE</th>\n",
       "      <th>AFFILIATION</th>\n",
       "      <th>CLASSIFICATION</th>\n",
       "      <th>USE_CASE</th>\n",
       "      <th>ORGANIZATION</th>\n",
       "      <th>STATUS</th>\n",
       "      <th>INCOME_AMT</th>\n",
       "      <th>SPECIAL_CONSIDERATIONS</th>\n",
       "      <th>ASK_AMT</th>\n",
       "      <th>IS_SUCCESSFUL</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10520599</td>\n",
       "      <td>BLUE KNIGHTS MOTORCYCLE CLUB</td>\n",
       "      <td>T10</td>\n",
       "      <td>Independent</td>\n",
       "      <td>C1000</td>\n",
       "      <td>ProductDev</td>\n",
       "      <td>Association</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>N</td>\n",
       "      <td>5000</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10531628</td>\n",
       "      <td>AMERICAN CHESAPEAKE CLUB CHARITABLE TR</td>\n",
       "      <td>T3</td>\n",
       "      <td>Independent</td>\n",
       "      <td>C2000</td>\n",
       "      <td>Preservation</td>\n",
       "      <td>Co-operative</td>\n",
       "      <td>1</td>\n",
       "      <td>1-9999</td>\n",
       "      <td>N</td>\n",
       "      <td>108590</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10547893</td>\n",
       "      <td>ST CLOUD PROFESSIONAL FIREFIGHTERS</td>\n",
       "      <td>T5</td>\n",
       "      <td>CompanySponsored</td>\n",
       "      <td>C3000</td>\n",
       "      <td>ProductDev</td>\n",
       "      <td>Association</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>N</td>\n",
       "      <td>5000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10553066</td>\n",
       "      <td>SOUTHSIDE ATHLETIC ASSOCIATION</td>\n",
       "      <td>T3</td>\n",
       "      <td>CompanySponsored</td>\n",
       "      <td>C2000</td>\n",
       "      <td>Preservation</td>\n",
       "      <td>Trust</td>\n",
       "      <td>1</td>\n",
       "      <td>10000-24999</td>\n",
       "      <td>N</td>\n",
       "      <td>6692</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10556103</td>\n",
       "      <td>GENETIC RESEARCH INSTITUTE OF THE DESERT</td>\n",
       "      <td>T3</td>\n",
       "      <td>Independent</td>\n",
       "      <td>C1000</td>\n",
       "      <td>Heathcare</td>\n",
       "      <td>Trust</td>\n",
       "      <td>1</td>\n",
       "      <td>100000-499999</td>\n",
       "      <td>N</td>\n",
       "      <td>142590</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34294</th>\n",
       "      <td>996009318</td>\n",
       "      <td>THE LIONS CLUB OF HONOLULU KAMEHAMEHA</td>\n",
       "      <td>T4</td>\n",
       "      <td>Independent</td>\n",
       "      <td>C1000</td>\n",
       "      <td>ProductDev</td>\n",
       "      <td>Association</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>N</td>\n",
       "      <td>5000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34295</th>\n",
       "      <td>996010315</td>\n",
       "      <td>INTERNATIONAL ASSOCIATION OF LIONS CLUBS</td>\n",
       "      <td>T4</td>\n",
       "      <td>CompanySponsored</td>\n",
       "      <td>C3000</td>\n",
       "      <td>ProductDev</td>\n",
       "      <td>Association</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>N</td>\n",
       "      <td>5000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34296</th>\n",
       "      <td>996012607</td>\n",
       "      <td>PTA HAWAII CONGRESS</td>\n",
       "      <td>T3</td>\n",
       "      <td>CompanySponsored</td>\n",
       "      <td>C2000</td>\n",
       "      <td>Preservation</td>\n",
       "      <td>Association</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>N</td>\n",
       "      <td>5000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34297</th>\n",
       "      <td>996015768</td>\n",
       "      <td>AMERICAN FEDERATION OF GOVERNMENT EMPLOYEES LO...</td>\n",
       "      <td>T5</td>\n",
       "      <td>Independent</td>\n",
       "      <td>C3000</td>\n",
       "      <td>ProductDev</td>\n",
       "      <td>Association</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>N</td>\n",
       "      <td>5000</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34298</th>\n",
       "      <td>996086871</td>\n",
       "      <td>WATERHOUSE CHARITABLE TR</td>\n",
       "      <td>T3</td>\n",
       "      <td>Independent</td>\n",
       "      <td>C1000</td>\n",
       "      <td>Preservation</td>\n",
       "      <td>Co-operative</td>\n",
       "      <td>1</td>\n",
       "      <td>1M-5M</td>\n",
       "      <td>N</td>\n",
       "      <td>36500179</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>34299 rows × 12 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "             EIN                                               NAME  \\\n",
       "0       10520599                       BLUE KNIGHTS MOTORCYCLE CLUB   \n",
       "1       10531628             AMERICAN CHESAPEAKE CLUB CHARITABLE TR   \n",
       "2       10547893                 ST CLOUD PROFESSIONAL FIREFIGHTERS   \n",
       "3       10553066                     SOUTHSIDE ATHLETIC ASSOCIATION   \n",
       "4       10556103           GENETIC RESEARCH INSTITUTE OF THE DESERT   \n",
       "...          ...                                                ...   \n",
       "34294  996009318              THE LIONS CLUB OF HONOLULU KAMEHAMEHA   \n",
       "34295  996010315           INTERNATIONAL ASSOCIATION OF LIONS CLUBS   \n",
       "34296  996012607                                PTA HAWAII CONGRESS   \n",
       "34297  996015768  AMERICAN FEDERATION OF GOVERNMENT EMPLOYEES LO...   \n",
       "34298  996086871                           WATERHOUSE CHARITABLE TR   \n",
       "\n",
       "      APPLICATION_TYPE       AFFILIATION CLASSIFICATION      USE_CASE  \\\n",
       "0                  T10       Independent          C1000    ProductDev   \n",
       "1                   T3       Independent          C2000  Preservation   \n",
       "2                   T5  CompanySponsored          C3000    ProductDev   \n",
       "3                   T3  CompanySponsored          C2000  Preservation   \n",
       "4                   T3       Independent          C1000     Heathcare   \n",
       "...                ...               ...            ...           ...   \n",
       "34294               T4       Independent          C1000    ProductDev   \n",
       "34295               T4  CompanySponsored          C3000    ProductDev   \n",
       "34296               T3  CompanySponsored          C2000  Preservation   \n",
       "34297               T5       Independent          C3000    ProductDev   \n",
       "34298               T3       Independent          C1000  Preservation   \n",
       "\n",
       "       ORGANIZATION  STATUS     INCOME_AMT SPECIAL_CONSIDERATIONS   ASK_AMT  \\\n",
       "0       Association       1              0                      N      5000   \n",
       "1      Co-operative       1         1-9999                      N    108590   \n",
       "2       Association       1              0                      N      5000   \n",
       "3             Trust       1    10000-24999                      N      6692   \n",
       "4             Trust       1  100000-499999                      N    142590   \n",
       "...             ...     ...            ...                    ...       ...   \n",
       "34294   Association       1              0                      N      5000   \n",
       "34295   Association       1              0                      N      5000   \n",
       "34296   Association       1              0                      N      5000   \n",
       "34297   Association       1              0                      N      5000   \n",
       "34298  Co-operative       1          1M-5M                      N  36500179   \n",
       "\n",
       "       IS_SUCCESSFUL  \n",
       "0                  1  \n",
       "1                  1  \n",
       "2                  0  \n",
       "3                  1  \n",
       "4                  1  \n",
       "...              ...  \n",
       "34294              0  \n",
       "34295              0  \n",
       "34296              0  \n",
       "34297              1  \n",
       "34298              0  \n",
       "\n",
       "[34299 rows x 12 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Read the applicants_data.csv file from the Resources folder into a Pandas DataFrame\n",
    "applicant_data_df = pd.read_csv(Path(\"Resources/applicants_data.csv\"))\n",
    "                                     \n",
    "\n",
    "# Review the DataFrame\n",
    "applicant_data_df\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 34299 entries, 0 to 34298\n",
      "Data columns (total 12 columns):\n",
      " #   Column                  Non-Null Count  Dtype \n",
      "---  ------                  --------------  ----- \n",
      " 0   EIN                     34299 non-null  int64 \n",
      " 1   NAME                    34299 non-null  object\n",
      " 2   APPLICATION_TYPE        34299 non-null  object\n",
      " 3   AFFILIATION             34299 non-null  object\n",
      " 4   CLASSIFICATION          34299 non-null  object\n",
      " 5   USE_CASE                34299 non-null  object\n",
      " 6   ORGANIZATION            34299 non-null  object\n",
      " 7   STATUS                  34299 non-null  int64 \n",
      " 8   INCOME_AMT              34299 non-null  object\n",
      " 9   SPECIAL_CONSIDERATIONS  34299 non-null  object\n",
      " 10  ASK_AMT                 34299 non-null  int64 \n",
      " 11  IS_SUCCESSFUL           34299 non-null  int64 \n",
      "dtypes: int64(4), object(8)\n",
      "memory usage: 3.1+ MB\n"
     ]
    }
   ],
   "source": [
    "# Review the data types associated with the columns\n",
    "applicant_data_df.info()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "EIN                       0\n",
       "NAME                      0\n",
       "APPLICATION_TYPE          0\n",
       "AFFILIATION               0\n",
       "CLASSIFICATION            0\n",
       "USE_CASE                  0\n",
       "ORGANIZATION              0\n",
       "STATUS                    0\n",
       "INCOME_AMT                0\n",
       "SPECIAL_CONSIDERATIONS    0\n",
       "ASK_AMT                   0\n",
       "IS_SUCCESSFUL             0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Check for null values in the dataset:\n",
    "applicant_data_df.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check for duplicate rows in the dataframe:\n",
    "applicant_data_df.duplicated().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: Drop the “EIN” (Employer Identification Number) and “NAME” columns from the DataFrame, because they are not relevant to the binary classification model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>APPLICATION_TYPE</th>\n",
       "      <th>AFFILIATION</th>\n",
       "      <th>CLASSIFICATION</th>\n",
       "      <th>USE_CASE</th>\n",
       "      <th>ORGANIZATION</th>\n",
       "      <th>STATUS</th>\n",
       "      <th>INCOME_AMT</th>\n",
       "      <th>SPECIAL_CONSIDERATIONS</th>\n",
       "      <th>ASK_AMT</th>\n",
       "      <th>IS_SUCCESSFUL</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>T10</td>\n",
       "      <td>Independent</td>\n",
       "      <td>C1000</td>\n",
       "      <td>ProductDev</td>\n",
       "      <td>Association</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>N</td>\n",
       "      <td>5000</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>T3</td>\n",
       "      <td>Independent</td>\n",
       "      <td>C2000</td>\n",
       "      <td>Preservation</td>\n",
       "      <td>Co-operative</td>\n",
       "      <td>1</td>\n",
       "      <td>1-9999</td>\n",
       "      <td>N</td>\n",
       "      <td>108590</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>T5</td>\n",
       "      <td>CompanySponsored</td>\n",
       "      <td>C3000</td>\n",
       "      <td>ProductDev</td>\n",
       "      <td>Association</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>N</td>\n",
       "      <td>5000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>T3</td>\n",
       "      <td>CompanySponsored</td>\n",
       "      <td>C2000</td>\n",
       "      <td>Preservation</td>\n",
       "      <td>Trust</td>\n",
       "      <td>1</td>\n",
       "      <td>10000-24999</td>\n",
       "      <td>N</td>\n",
       "      <td>6692</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>T3</td>\n",
       "      <td>Independent</td>\n",
       "      <td>C1000</td>\n",
       "      <td>Heathcare</td>\n",
       "      <td>Trust</td>\n",
       "      <td>1</td>\n",
       "      <td>100000-499999</td>\n",
       "      <td>N</td>\n",
       "      <td>142590</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34294</th>\n",
       "      <td>T4</td>\n",
       "      <td>Independent</td>\n",
       "      <td>C1000</td>\n",
       "      <td>ProductDev</td>\n",
       "      <td>Association</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>N</td>\n",
       "      <td>5000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34295</th>\n",
       "      <td>T4</td>\n",
       "      <td>CompanySponsored</td>\n",
       "      <td>C3000</td>\n",
       "      <td>ProductDev</td>\n",
       "      <td>Association</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>N</td>\n",
       "      <td>5000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34296</th>\n",
       "      <td>T3</td>\n",
       "      <td>CompanySponsored</td>\n",
       "      <td>C2000</td>\n",
       "      <td>Preservation</td>\n",
       "      <td>Association</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>N</td>\n",
       "      <td>5000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34297</th>\n",
       "      <td>T5</td>\n",
       "      <td>Independent</td>\n",
       "      <td>C3000</td>\n",
       "      <td>ProductDev</td>\n",
       "      <td>Association</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>N</td>\n",
       "      <td>5000</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34298</th>\n",
       "      <td>T3</td>\n",
       "      <td>Independent</td>\n",
       "      <td>C1000</td>\n",
       "      <td>Preservation</td>\n",
       "      <td>Co-operative</td>\n",
       "      <td>1</td>\n",
       "      <td>1M-5M</td>\n",
       "      <td>N</td>\n",
       "      <td>36500179</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>34299 rows × 10 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      APPLICATION_TYPE       AFFILIATION CLASSIFICATION      USE_CASE  \\\n",
       "0                  T10       Independent          C1000    ProductDev   \n",
       "1                   T3       Independent          C2000  Preservation   \n",
       "2                   T5  CompanySponsored          C3000    ProductDev   \n",
       "3                   T3  CompanySponsored          C2000  Preservation   \n",
       "4                   T3       Independent          C1000     Heathcare   \n",
       "...                ...               ...            ...           ...   \n",
       "34294               T4       Independent          C1000    ProductDev   \n",
       "34295               T4  CompanySponsored          C3000    ProductDev   \n",
       "34296               T3  CompanySponsored          C2000  Preservation   \n",
       "34297               T5       Independent          C3000    ProductDev   \n",
       "34298               T3       Independent          C1000  Preservation   \n",
       "\n",
       "       ORGANIZATION  STATUS     INCOME_AMT SPECIAL_CONSIDERATIONS   ASK_AMT  \\\n",
       "0       Association       1              0                      N      5000   \n",
       "1      Co-operative       1         1-9999                      N    108590   \n",
       "2       Association       1              0                      N      5000   \n",
       "3             Trust       1    10000-24999                      N      6692   \n",
       "4             Trust       1  100000-499999                      N    142590   \n",
       "...             ...     ...            ...                    ...       ...   \n",
       "34294   Association       1              0                      N      5000   \n",
       "34295   Association       1              0                      N      5000   \n",
       "34296   Association       1              0                      N      5000   \n",
       "34297   Association       1              0                      N      5000   \n",
       "34298  Co-operative       1          1M-5M                      N  36500179   \n",
       "\n",
       "       IS_SUCCESSFUL  \n",
       "0                  1  \n",
       "1                  1  \n",
       "2                  0  \n",
       "3                  1  \n",
       "4                  1  \n",
       "...              ...  \n",
       "34294              0  \n",
       "34295              0  \n",
       "34296              0  \n",
       "34297              1  \n",
       "34298              0  \n",
       "\n",
       "[34299 rows x 10 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Drop the 'EIN' and 'NAME' columns from the DataFrame\n",
    "applicant_data_df = applicant_data_df.drop(['EIN', 'NAME'], axis=1)\n",
    "\n",
    "# Review the DataFrame\n",
    "applicant_data_df\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 34299 entries, 0 to 34298\n",
      "Data columns (total 10 columns):\n",
      " #   Column                  Non-Null Count  Dtype \n",
      "---  ------                  --------------  ----- \n",
      " 0   APPLICATION_TYPE        34299 non-null  object\n",
      " 1   AFFILIATION             34299 non-null  object\n",
      " 2   CLASSIFICATION          34299 non-null  object\n",
      " 3   USE_CASE                34299 non-null  object\n",
      " 4   ORGANIZATION            34299 non-null  object\n",
      " 5   STATUS                  34299 non-null  int64 \n",
      " 6   INCOME_AMT              34299 non-null  object\n",
      " 7   SPECIAL_CONSIDERATIONS  34299 non-null  object\n",
      " 8   ASK_AMT                 34299 non-null  int64 \n",
      " 9   IS_SUCCESSFUL           34299 non-null  int64 \n",
      "dtypes: int64(3), object(7)\n",
      "memory usage: 2.6+ MB\n"
     ]
    }
   ],
   "source": [
    "# Review the data types associated with the columns\n",
    "applicant_data_df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Encode the dataset’s categorical variables using `OneHotEncoder`, and then place the encoded variables into a new DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a list of categorical variables \n",
    "categorical_variables = list(applicant_data_df.dtypes[applicant_data_df.dtypes == \"object\"].index)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['APPLICATION_TYPE',\n",
       " 'AFFILIATION',\n",
       " 'CLASSIFICATION',\n",
       " 'USE_CASE',\n",
       " 'ORGANIZATION',\n",
       " 'INCOME_AMT',\n",
       " 'SPECIAL_CONSIDERATIONS']"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Display the categorical variables list\n",
    "categorical_variables\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-1 {color: black;background-color: white;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>OneHotEncoder(sparse_output=False)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">OneHotEncoder</label><div class=\"sk-toggleable__content\"><pre>OneHotEncoder(sparse_output=False)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "OneHotEncoder(sparse_output=False)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a OneHotEncoder instance\n",
    "enc = OneHotEncoder(sparse_output=False)\n",
    "enc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Encode the categorcal variables using OneHotEncoder\n",
    "encoded_data = enc.fit_transform(applicant_data_df[categorical_variables])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(34299, 114)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# The column names should match those of the encoded variables\n",
    "encoded_data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>APPLICATION_TYPE_T10</th>\n",
       "      <th>APPLICATION_TYPE_T12</th>\n",
       "      <th>APPLICATION_TYPE_T13</th>\n",
       "      <th>APPLICATION_TYPE_T14</th>\n",
       "      <th>APPLICATION_TYPE_T15</th>\n",
       "      <th>APPLICATION_TYPE_T17</th>\n",
       "      <th>APPLICATION_TYPE_T19</th>\n",
       "      <th>APPLICATION_TYPE_T2</th>\n",
       "      <th>APPLICATION_TYPE_T25</th>\n",
       "      <th>APPLICATION_TYPE_T29</th>\n",
       "      <th>...</th>\n",
       "      <th>INCOME_AMT_1-9999</th>\n",
       "      <th>INCOME_AMT_10000-24999</th>\n",
       "      <th>INCOME_AMT_100000-499999</th>\n",
       "      <th>INCOME_AMT_10M-50M</th>\n",
       "      <th>INCOME_AMT_1M-5M</th>\n",
       "      <th>INCOME_AMT_25000-99999</th>\n",
       "      <th>INCOME_AMT_50M+</th>\n",
       "      <th>INCOME_AMT_5M-10M</th>\n",
       "      <th>SPECIAL_CONSIDERATIONS_N</th>\n",
       "      <th>SPECIAL_CONSIDERATIONS_Y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34294</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34295</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34296</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34297</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34298</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>34299 rows × 114 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       APPLICATION_TYPE_T10  APPLICATION_TYPE_T12  APPLICATION_TYPE_T13  \\\n",
       "0                       1.0                   0.0                   0.0   \n",
       "1                       0.0                   0.0                   0.0   \n",
       "2                       0.0                   0.0                   0.0   \n",
       "3                       0.0                   0.0                   0.0   \n",
       "4                       0.0                   0.0                   0.0   \n",
       "...                     ...                   ...                   ...   \n",
       "34294                   0.0                   0.0                   0.0   \n",
       "34295                   0.0                   0.0                   0.0   \n",
       "34296                   0.0                   0.0                   0.0   \n",
       "34297                   0.0                   0.0                   0.0   \n",
       "34298                   0.0                   0.0                   0.0   \n",
       "\n",
       "       APPLICATION_TYPE_T14  APPLICATION_TYPE_T15  APPLICATION_TYPE_T17  \\\n",
       "0                       0.0                   0.0                   0.0   \n",
       "1                       0.0                   0.0                   0.0   \n",
       "2                       0.0                   0.0                   0.0   \n",
       "3                       0.0                   0.0                   0.0   \n",
       "4                       0.0                   0.0                   0.0   \n",
       "...                     ...                   ...                   ...   \n",
       "34294                   0.0                   0.0                   0.0   \n",
       "34295                   0.0                   0.0                   0.0   \n",
       "34296                   0.0                   0.0                   0.0   \n",
       "34297                   0.0                   0.0                   0.0   \n",
       "34298                   0.0                   0.0                   0.0   \n",
       "\n",
       "       APPLICATION_TYPE_T19  APPLICATION_TYPE_T2  APPLICATION_TYPE_T25  \\\n",
       "0                       0.0                  0.0                   0.0   \n",
       "1                       0.0                  0.0                   0.0   \n",
       "2                       0.0                  0.0                   0.0   \n",
       "3                       0.0                  0.0                   0.0   \n",
       "4                       0.0                  0.0                   0.0   \n",
       "...                     ...                  ...                   ...   \n",
       "34294                   0.0                  0.0                   0.0   \n",
       "34295                   0.0                  0.0                   0.0   \n",
       "34296                   0.0                  0.0                   0.0   \n",
       "34297                   0.0                  0.0                   0.0   \n",
       "34298                   0.0                  0.0                   0.0   \n",
       "\n",
       "       APPLICATION_TYPE_T29  ...  INCOME_AMT_1-9999  INCOME_AMT_10000-24999  \\\n",
       "0                       0.0  ...                0.0                     0.0   \n",
       "1                       0.0  ...                1.0                     0.0   \n",
       "2                       0.0  ...                0.0                     0.0   \n",
       "3                       0.0  ...                0.0                     1.0   \n",
       "4                       0.0  ...                0.0                     0.0   \n",
       "...                     ...  ...                ...                     ...   \n",
       "34294                   0.0  ...                0.0                     0.0   \n",
       "34295                   0.0  ...                0.0                     0.0   \n",
       "34296                   0.0  ...                0.0                     0.0   \n",
       "34297                   0.0  ...                0.0                     0.0   \n",
       "34298                   0.0  ...                0.0                     0.0   \n",
       "\n",
       "       INCOME_AMT_100000-499999  INCOME_AMT_10M-50M  INCOME_AMT_1M-5M  \\\n",
       "0                           0.0                 0.0               0.0   \n",
       "1                           0.0                 0.0               0.0   \n",
       "2                           0.0                 0.0               0.0   \n",
       "3                           0.0                 0.0               0.0   \n",
       "4                           1.0                 0.0               0.0   \n",
       "...                         ...                 ...               ...   \n",
       "34294                       0.0                 0.0               0.0   \n",
       "34295                       0.0                 0.0               0.0   \n",
       "34296                       0.0                 0.0               0.0   \n",
       "34297                       0.0                 0.0               0.0   \n",
       "34298                       0.0                 0.0               1.0   \n",
       "\n",
       "       INCOME_AMT_25000-99999  INCOME_AMT_50M+  INCOME_AMT_5M-10M  \\\n",
       "0                         0.0              0.0                0.0   \n",
       "1                         0.0              0.0                0.0   \n",
       "2                         0.0              0.0                0.0   \n",
       "3                         0.0              0.0                0.0   \n",
       "4                         0.0              0.0                0.0   \n",
       "...                       ...              ...                ...   \n",
       "34294                     0.0              0.0                0.0   \n",
       "34295                     0.0              0.0                0.0   \n",
       "34296                     0.0              0.0                0.0   \n",
       "34297                     0.0              0.0                0.0   \n",
       "34298                     0.0              0.0                0.0   \n",
       "\n",
       "       SPECIAL_CONSIDERATIONS_N  SPECIAL_CONSIDERATIONS_Y  \n",
       "0                           1.0                       0.0  \n",
       "1                           1.0                       0.0  \n",
       "2                           1.0                       0.0  \n",
       "3                           1.0                       0.0  \n",
       "4                           1.0                       0.0  \n",
       "...                         ...                       ...  \n",
       "34294                       1.0                       0.0  \n",
       "34295                       1.0                       0.0  \n",
       "34296                       1.0                       0.0  \n",
       "34297                       1.0                       0.0  \n",
       "34298                       1.0                       0.0  \n",
       "\n",
       "[34299 rows x 114 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a DataFrame with the encoded variables\n",
    "encoded_df = pd.DataFrame(\n",
    "    encoded_data,\n",
    "    columns = enc.get_feature_names_out(categorical_variables)\n",
    ")\n",
    "\n",
    "# Display the DataFrame\n",
    "encoded_df\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 34299 entries, 0 to 34298\n",
      "Data columns (total 10 columns):\n",
      " #   Column                  Non-Null Count  Dtype \n",
      "---  ------                  --------------  ----- \n",
      " 0   APPLICATION_TYPE        34299 non-null  object\n",
      " 1   AFFILIATION             34299 non-null  object\n",
      " 2   CLASSIFICATION          34299 non-null  object\n",
      " 3   USE_CASE                34299 non-null  object\n",
      " 4   ORGANIZATION            34299 non-null  object\n",
      " 5   STATUS                  34299 non-null  int64 \n",
      " 6   INCOME_AMT              34299 non-null  object\n",
      " 7   SPECIAL_CONSIDERATIONS  34299 non-null  object\n",
      " 8   ASK_AMT                 34299 non-null  int64 \n",
      " 9   IS_SUCCESSFUL           34299 non-null  int64 \n",
      "dtypes: int64(3), object(7)\n",
      "memory usage: 2.6+ MB\n"
     ]
    }
   ],
   "source": [
    "# Review the data types associated with the columns\n",
    "applicant_data_df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 4: Add the original DataFrame’s numerical variables to the DataFrame containing the encoded variables.\n",
    "\n",
    "> **Note** To complete this step, you will employ the Pandas `concat()` function that was introduced earlier in this course. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>STATUS</th>\n",
       "      <th>ASK_AMT</th>\n",
       "      <th>IS_SUCCESSFUL</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>5000</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>108590</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>5000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>6692</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>142590</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34294</th>\n",
       "      <td>1</td>\n",
       "      <td>5000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34295</th>\n",
       "      <td>1</td>\n",
       "      <td>5000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34296</th>\n",
       "      <td>1</td>\n",
       "      <td>5000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34297</th>\n",
       "      <td>1</td>\n",
       "      <td>5000</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34298</th>\n",
       "      <td>1</td>\n",
       "      <td>36500179</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>34299 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       STATUS   ASK_AMT  IS_SUCCESSFUL\n",
       "0           1      5000              1\n",
       "1           1    108590              1\n",
       "2           1      5000              0\n",
       "3           1      6692              1\n",
       "4           1    142590              1\n",
       "...       ...       ...            ...\n",
       "34294       1      5000              0\n",
       "34295       1      5000              0\n",
       "34296       1      5000              0\n",
       "34297       1      5000              1\n",
       "34298       1  36500179              0\n",
       "\n",
       "[34299 rows x 3 columns]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a DataFrame with the columnns containing numerical variables from the original dataset\n",
    "numerical_variables_df = applicant_data_df.drop(columns = categorical_variables)\n",
    "\n",
    "# Review the DataFrame\n",
    "numerical_variables_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>STATUS</th>\n",
       "      <th>ASK_AMT</th>\n",
       "      <th>IS_SUCCESSFUL</th>\n",
       "      <th>APPLICATION_TYPE_T10</th>\n",
       "      <th>APPLICATION_TYPE_T12</th>\n",
       "      <th>APPLICATION_TYPE_T13</th>\n",
       "      <th>APPLICATION_TYPE_T14</th>\n",
       "      <th>APPLICATION_TYPE_T15</th>\n",
       "      <th>APPLICATION_TYPE_T17</th>\n",
       "      <th>APPLICATION_TYPE_T19</th>\n",
       "      <th>...</th>\n",
       "      <th>INCOME_AMT_1-9999</th>\n",
       "      <th>INCOME_AMT_10000-24999</th>\n",
       "      <th>INCOME_AMT_100000-499999</th>\n",
       "      <th>INCOME_AMT_10M-50M</th>\n",
       "      <th>INCOME_AMT_1M-5M</th>\n",
       "      <th>INCOME_AMT_25000-99999</th>\n",
       "      <th>INCOME_AMT_50M+</th>\n",
       "      <th>INCOME_AMT_5M-10M</th>\n",
       "      <th>SPECIAL_CONSIDERATIONS_N</th>\n",
       "      <th>SPECIAL_CONSIDERATIONS_Y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>5000</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>108590</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>5000</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>6692</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>142590</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 117 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   STATUS  ASK_AMT  IS_SUCCESSFUL  APPLICATION_TYPE_T10  APPLICATION_TYPE_T12  \\\n",
       "0       1     5000              1                   1.0                   0.0   \n",
       "1       1   108590              1                   0.0                   0.0   \n",
       "2       1     5000              0                   0.0                   0.0   \n",
       "3       1     6692              1                   0.0                   0.0   \n",
       "4       1   142590              1                   0.0                   0.0   \n",
       "\n",
       "   APPLICATION_TYPE_T13  APPLICATION_TYPE_T14  APPLICATION_TYPE_T15  \\\n",
       "0                   0.0                   0.0                   0.0   \n",
       "1                   0.0                   0.0                   0.0   \n",
       "2                   0.0                   0.0                   0.0   \n",
       "3                   0.0                   0.0                   0.0   \n",
       "4                   0.0                   0.0                   0.0   \n",
       "\n",
       "   APPLICATION_TYPE_T17  APPLICATION_TYPE_T19  ...  INCOME_AMT_1-9999  \\\n",
       "0                   0.0                   0.0  ...                0.0   \n",
       "1                   0.0                   0.0  ...                1.0   \n",
       "2                   0.0                   0.0  ...                0.0   \n",
       "3                   0.0                   0.0  ...                0.0   \n",
       "4                   0.0                   0.0  ...                0.0   \n",
       "\n",
       "   INCOME_AMT_10000-24999  INCOME_AMT_100000-499999  INCOME_AMT_10M-50M  \\\n",
       "0                     0.0                       0.0                 0.0   \n",
       "1                     0.0                       0.0                 0.0   \n",
       "2                     0.0                       0.0                 0.0   \n",
       "3                     1.0                       0.0                 0.0   \n",
       "4                     0.0                       1.0                 0.0   \n",
       "\n",
       "   INCOME_AMT_1M-5M  INCOME_AMT_25000-99999  INCOME_AMT_50M+  \\\n",
       "0               0.0                     0.0              0.0   \n",
       "1               0.0                     0.0              0.0   \n",
       "2               0.0                     0.0              0.0   \n",
       "3               0.0                     0.0              0.0   \n",
       "4               0.0                     0.0              0.0   \n",
       "\n",
       "   INCOME_AMT_5M-10M  SPECIAL_CONSIDERATIONS_N  SPECIAL_CONSIDERATIONS_Y  \n",
       "0                0.0                       1.0                       0.0  \n",
       "1                0.0                       1.0                       0.0  \n",
       "2                0.0                       1.0                       0.0  \n",
       "3                0.0                       1.0                       0.0  \n",
       "4                0.0                       1.0                       0.0  \n",
       "\n",
       "[5 rows x 117 columns]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Add the numerical variables from the original DataFrame to the one-hot encoding DataFrame\n",
    "# Using the Pandas concat function, combine the DataFrames the contain the encoded categorical data and the numerical data\n",
    "numerical_encoded_df = pd.concat(\n",
    "    [\n",
    "        numerical_variables_df,\n",
    "        encoded_df\n",
    "    ],\n",
    "    axis=1\n",
    ")\n",
    "\n",
    "# Reveiw the DataFrame\n",
    "numerical_encoded_df.head()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 34299 entries, 0 to 34298\n",
      "Columns: 117 entries, STATUS to SPECIAL_CONSIDERATIONS_Y\n",
      "dtypes: float64(114), int64(3)\n",
      "memory usage: 30.6 MB\n"
     ]
    }
   ],
   "source": [
    "numerical_encoded_df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 5: Using the preprocessed data, create the features (`X`) and target (`y`) datasets. The target dataset should be defined by the preprocessed DataFrame column “IS_SUCCESSFUL”. The remaining columns should define the features dataset. \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    1\n",
       "1    1\n",
       "2    0\n",
       "3    1\n",
       "4    1\n",
       "5    1\n",
       "6    1\n",
       "7    1\n",
       "8    1\n",
       "9    0\n",
       "Name: IS_SUCCESSFUL, dtype: int64"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define the target set y using the IS_SUCCESSFUL column\n",
    "y = numerical_encoded_df[\"IS_SUCCESSFUL\"]\n",
    "\n",
    "# Display a sample of y\n",
    "y[:10]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>STATUS</th>\n",
       "      <th>ASK_AMT</th>\n",
       "      <th>APPLICATION_TYPE_T10</th>\n",
       "      <th>APPLICATION_TYPE_T12</th>\n",
       "      <th>APPLICATION_TYPE_T13</th>\n",
       "      <th>APPLICATION_TYPE_T14</th>\n",
       "      <th>APPLICATION_TYPE_T15</th>\n",
       "      <th>APPLICATION_TYPE_T17</th>\n",
       "      <th>APPLICATION_TYPE_T19</th>\n",
       "      <th>APPLICATION_TYPE_T2</th>\n",
       "      <th>...</th>\n",
       "      <th>INCOME_AMT_1-9999</th>\n",
       "      <th>INCOME_AMT_10000-24999</th>\n",
       "      <th>INCOME_AMT_100000-499999</th>\n",
       "      <th>INCOME_AMT_10M-50M</th>\n",
       "      <th>INCOME_AMT_1M-5M</th>\n",
       "      <th>INCOME_AMT_25000-99999</th>\n",
       "      <th>INCOME_AMT_50M+</th>\n",
       "      <th>INCOME_AMT_5M-10M</th>\n",
       "      <th>SPECIAL_CONSIDERATIONS_N</th>\n",
       "      <th>SPECIAL_CONSIDERATIONS_Y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>5000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>108590</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>5000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>6692</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>142590</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 116 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   STATUS  ASK_AMT  APPLICATION_TYPE_T10  APPLICATION_TYPE_T12  \\\n",
       "0       1     5000                   1.0                   0.0   \n",
       "1       1   108590                   0.0                   0.0   \n",
       "2       1     5000                   0.0                   0.0   \n",
       "3       1     6692                   0.0                   0.0   \n",
       "4       1   142590                   0.0                   0.0   \n",
       "\n",
       "   APPLICATION_TYPE_T13  APPLICATION_TYPE_T14  APPLICATION_TYPE_T15  \\\n",
       "0                   0.0                   0.0                   0.0   \n",
       "1                   0.0                   0.0                   0.0   \n",
       "2                   0.0                   0.0                   0.0   \n",
       "3                   0.0                   0.0                   0.0   \n",
       "4                   0.0                   0.0                   0.0   \n",
       "\n",
       "   APPLICATION_TYPE_T17  APPLICATION_TYPE_T19  APPLICATION_TYPE_T2  ...  \\\n",
       "0                   0.0                   0.0                  0.0  ...   \n",
       "1                   0.0                   0.0                  0.0  ...   \n",
       "2                   0.0                   0.0                  0.0  ...   \n",
       "3                   0.0                   0.0                  0.0  ...   \n",
       "4                   0.0                   0.0                  0.0  ...   \n",
       "\n",
       "   INCOME_AMT_1-9999  INCOME_AMT_10000-24999  INCOME_AMT_100000-499999  \\\n",
       "0                0.0                     0.0                       0.0   \n",
       "1                1.0                     0.0                       0.0   \n",
       "2                0.0                     0.0                       0.0   \n",
       "3                0.0                     1.0                       0.0   \n",
       "4                0.0                     0.0                       1.0   \n",
       "\n",
       "   INCOME_AMT_10M-50M  INCOME_AMT_1M-5M  INCOME_AMT_25000-99999  \\\n",
       "0                 0.0               0.0                     0.0   \n",
       "1                 0.0               0.0                     0.0   \n",
       "2                 0.0               0.0                     0.0   \n",
       "3                 0.0               0.0                     0.0   \n",
       "4                 0.0               0.0                     0.0   \n",
       "\n",
       "   INCOME_AMT_50M+  INCOME_AMT_5M-10M  SPECIAL_CONSIDERATIONS_N  \\\n",
       "0              0.0                0.0                       1.0   \n",
       "1              0.0                0.0                       1.0   \n",
       "2              0.0                0.0                       1.0   \n",
       "3              0.0                0.0                       1.0   \n",
       "4              0.0                0.0                       1.0   \n",
       "\n",
       "   SPECIAL_CONSIDERATIONS_Y  \n",
       "0                       0.0  \n",
       "1                       0.0  \n",
       "2                       0.0  \n",
       "3                       0.0  \n",
       "4                       0.0  \n",
       "\n",
       "[5 rows x 116 columns]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define features set X by selecting all columns but IS_SUCCESSFUL\n",
    "X = numerical_encoded_df.drop(columns=[\"IS_SUCCESSFUL\"])\n",
    "\n",
    "# Review the features DataFrame\n",
    "X.head()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>STATUS</th>\n",
       "      <th>ASK_AMT</th>\n",
       "      <th>APPLICATION_TYPE_T10</th>\n",
       "      <th>APPLICATION_TYPE_T12</th>\n",
       "      <th>APPLICATION_TYPE_T13</th>\n",
       "      <th>APPLICATION_TYPE_T14</th>\n",
       "      <th>APPLICATION_TYPE_T15</th>\n",
       "      <th>APPLICATION_TYPE_T17</th>\n",
       "      <th>APPLICATION_TYPE_T19</th>\n",
       "      <th>APPLICATION_TYPE_T2</th>\n",
       "      <th>...</th>\n",
       "      <th>INCOME_AMT_1-9999</th>\n",
       "      <th>INCOME_AMT_10000-24999</th>\n",
       "      <th>INCOME_AMT_100000-499999</th>\n",
       "      <th>INCOME_AMT_10M-50M</th>\n",
       "      <th>INCOME_AMT_1M-5M</th>\n",
       "      <th>INCOME_AMT_25000-99999</th>\n",
       "      <th>INCOME_AMT_50M+</th>\n",
       "      <th>INCOME_AMT_5M-10M</th>\n",
       "      <th>SPECIAL_CONSIDERATIONS_N</th>\n",
       "      <th>SPECIAL_CONSIDERATIONS_Y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>5000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>108590</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>5000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>6692</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>142590</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34294</th>\n",
       "      <td>1</td>\n",
       "      <td>5000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34295</th>\n",
       "      <td>1</td>\n",
       "      <td>5000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34296</th>\n",
       "      <td>1</td>\n",
       "      <td>5000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34297</th>\n",
       "      <td>1</td>\n",
       "      <td>5000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34298</th>\n",
       "      <td>1</td>\n",
       "      <td>36500179</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>34299 rows × 116 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       STATUS   ASK_AMT  APPLICATION_TYPE_T10  APPLICATION_TYPE_T12  \\\n",
       "0           1      5000                   1.0                   0.0   \n",
       "1           1    108590                   0.0                   0.0   \n",
       "2           1      5000                   0.0                   0.0   \n",
       "3           1      6692                   0.0                   0.0   \n",
       "4           1    142590                   0.0                   0.0   \n",
       "...       ...       ...                   ...                   ...   \n",
       "34294       1      5000                   0.0                   0.0   \n",
       "34295       1      5000                   0.0                   0.0   \n",
       "34296       1      5000                   0.0                   0.0   \n",
       "34297       1      5000                   0.0                   0.0   \n",
       "34298       1  36500179                   0.0                   0.0   \n",
       "\n",
       "       APPLICATION_TYPE_T13  APPLICATION_TYPE_T14  APPLICATION_TYPE_T15  \\\n",
       "0                       0.0                   0.0                   0.0   \n",
       "1                       0.0                   0.0                   0.0   \n",
       "2                       0.0                   0.0                   0.0   \n",
       "3                       0.0                   0.0                   0.0   \n",
       "4                       0.0                   0.0                   0.0   \n",
       "...                     ...                   ...                   ...   \n",
       "34294                   0.0                   0.0                   0.0   \n",
       "34295                   0.0                   0.0                   0.0   \n",
       "34296                   0.0                   0.0                   0.0   \n",
       "34297                   0.0                   0.0                   0.0   \n",
       "34298                   0.0                   0.0                   0.0   \n",
       "\n",
       "       APPLICATION_TYPE_T17  APPLICATION_TYPE_T19  APPLICATION_TYPE_T2  ...  \\\n",
       "0                       0.0                   0.0                  0.0  ...   \n",
       "1                       0.0                   0.0                  0.0  ...   \n",
       "2                       0.0                   0.0                  0.0  ...   \n",
       "3                       0.0                   0.0                  0.0  ...   \n",
       "4                       0.0                   0.0                  0.0  ...   \n",
       "...                     ...                   ...                  ...  ...   \n",
       "34294                   0.0                   0.0                  0.0  ...   \n",
       "34295                   0.0                   0.0                  0.0  ...   \n",
       "34296                   0.0                   0.0                  0.0  ...   \n",
       "34297                   0.0                   0.0                  0.0  ...   \n",
       "34298                   0.0                   0.0                  0.0  ...   \n",
       "\n",
       "       INCOME_AMT_1-9999  INCOME_AMT_10000-24999  INCOME_AMT_100000-499999  \\\n",
       "0                    0.0                     0.0                       0.0   \n",
       "1                    1.0                     0.0                       0.0   \n",
       "2                    0.0                     0.0                       0.0   \n",
       "3                    0.0                     1.0                       0.0   \n",
       "4                    0.0                     0.0                       1.0   \n",
       "...                  ...                     ...                       ...   \n",
       "34294                0.0                     0.0                       0.0   \n",
       "34295                0.0                     0.0                       0.0   \n",
       "34296                0.0                     0.0                       0.0   \n",
       "34297                0.0                     0.0                       0.0   \n",
       "34298                0.0                     0.0                       0.0   \n",
       "\n",
       "       INCOME_AMT_10M-50M  INCOME_AMT_1M-5M  INCOME_AMT_25000-99999  \\\n",
       "0                     0.0               0.0                     0.0   \n",
       "1                     0.0               0.0                     0.0   \n",
       "2                     0.0               0.0                     0.0   \n",
       "3                     0.0               0.0                     0.0   \n",
       "4                     0.0               0.0                     0.0   \n",
       "...                   ...               ...                     ...   \n",
       "34294                 0.0               0.0                     0.0   \n",
       "34295                 0.0               0.0                     0.0   \n",
       "34296                 0.0               0.0                     0.0   \n",
       "34297                 0.0               0.0                     0.0   \n",
       "34298                 0.0               1.0                     0.0   \n",
       "\n",
       "       INCOME_AMT_50M+  INCOME_AMT_5M-10M  SPECIAL_CONSIDERATIONS_N  \\\n",
       "0                  0.0                0.0                       1.0   \n",
       "1                  0.0                0.0                       1.0   \n",
       "2                  0.0                0.0                       1.0   \n",
       "3                  0.0                0.0                       1.0   \n",
       "4                  0.0                0.0                       1.0   \n",
       "...                ...                ...                       ...   \n",
       "34294              0.0                0.0                       1.0   \n",
       "34295              0.0                0.0                       1.0   \n",
       "34296              0.0                0.0                       1.0   \n",
       "34297              0.0                0.0                       1.0   \n",
       "34298              0.0                0.0                       1.0   \n",
       "\n",
       "       SPECIAL_CONSIDERATIONS_Y  \n",
       "0                           0.0  \n",
       "1                           0.0  \n",
       "2                           0.0  \n",
       "3                           0.0  \n",
       "4                           0.0  \n",
       "...                         ...  \n",
       "34294                       0.0  \n",
       "34295                       0.0  \n",
       "34296                       0.0  \n",
       "34297                       0.0  \n",
       "34298                       0.0  \n",
       "\n",
       "[34299 rows x 116 columns]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Review the features DataFrame\n",
    "X\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 6: Split the features and target sets into training and testing datasets.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Split the preprocessed data into a training and testing dataset\n",
    "# Assign the function a random_state equal to 10\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=10)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 7: Use scikit-learn's `StandardScaler` to scale the features data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a StandardScaler instance\n",
    "scaler = StandardScaler()\n",
    "\n",
    "# Fit the scaler to the features training dataset\n",
    "X_scaler = scaler.fit(X_train)\n",
    "\n",
    "# Fit the scaler to the features training dataset\n",
    "X_train_scaled = X_scaler.transform(X_train)\n",
    "X_test_scaled = X_scaler.transform(X_test)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Compile and Evaluate a Binary Classification Model Using a Neural Network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1: Create a deep neural network by assigning the number of input features, the number of layers, and the number of neurons on each layer using Tensorflow’s Keras.\n",
    "\n",
    "> **Hint** You can start with a two-layer deep neural network model that uses the `relu` activation function for both layers.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "116"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define the the number of inputs (features) to the model\n",
    "number_input_features = len(X_train.iloc[0])\n",
    "\n",
    "# Review the number of features\n",
    "number_input_features\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the number of neurons in the output layer\n",
    "number_output_neurons = 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define the number of hidden nodes for the first hidden layer\n",
    "# We are going to use 5 hidden nodes based on the fact that initially there were 8 features in the original dataframe.\n",
    "hidden_nodes_layer1 = 5\n",
    "\n",
    "# Review the number hidden nodes in the first layer\n",
    "hidden_nodes_layer1\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define the number of hidden nodes for the second hidden layer\n",
    "hidden_nodes_layer2 = 5 \n",
    "\n",
    "# Review the number hidden nodes in the second layer\n",
    "hidden_nodes_layer2\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the Sequential model instance\n",
    "nn = Sequential()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add the first hidden layer specifying the number of inputs, the number of hidden nodes, and the activation function\n",
    "nn.add(Dense(units=hidden_nodes_layer1, input_dim=number_input_features, activation=\"relu\"))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add the second hidden layer\n",
    "nn.add(Dense(units=hidden_nodes_layer2, activation=\"relu\"))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add the output layer to the model specifying the number of output neurons and activation function\n",
    "nn.add(Dense(units=number_output_neurons, activation=\"sigmoid\"))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " dense (Dense)               (None, 5)                 585       \n",
      "                                                                 \n",
      " dense_1 (Dense)             (None, 5)                 30        \n",
      "                                                                 \n",
      " dense_2 (Dense)             (None, 1)                 6         \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 621\n",
      "Trainable params: 621\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# Display the Sequential model summary\n",
    "nn.summary()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: Compile and fit the model using the `binary_crossentropy` loss function, the `adam` optimizer, and the `accuracy` evaluation metric.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compile the Sequential model\n",
    "nn.compile(loss=\"binary_crossentropy\", optimizer=\"adam\", metrics=[\"accuracy\"])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/50\n",
      "804/804 [==============================] - 3s 2ms/step - loss: 0.6424 - accuracy: 0.6607\n",
      "Epoch 2/50\n",
      "804/804 [==============================] - 2s 2ms/step - loss: 0.5866 - accuracy: 0.7203\n",
      "Epoch 3/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5683 - accuracy: 0.7280\n",
      "Epoch 4/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5620 - accuracy: 0.7300\n",
      "Epoch 5/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5591 - accuracy: 0.7289\n",
      "Epoch 6/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5571 - accuracy: 0.7292\n",
      "Epoch 7/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5561 - accuracy: 0.7302\n",
      "Epoch 8/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5549 - accuracy: 0.7311\n",
      "Epoch 9/50\n",
      "804/804 [==============================] - 1s 891us/step - loss: 0.5543 - accuracy: 0.7313\n",
      "Epoch 10/50\n",
      "804/804 [==============================] - 1s 859us/step - loss: 0.5536 - accuracy: 0.7312\n",
      "Epoch 11/50\n",
      "804/804 [==============================] - 1s 932us/step - loss: 0.5529 - accuracy: 0.7325\n",
      "Epoch 12/50\n",
      "804/804 [==============================] - 1s 864us/step - loss: 0.5526 - accuracy: 0.7319\n",
      "Epoch 13/50\n",
      "804/804 [==============================] - 1s 841us/step - loss: 0.5521 - accuracy: 0.7326\n",
      "Epoch 14/50\n",
      "804/804 [==============================] - 1s 927us/step - loss: 0.5515 - accuracy: 0.7319\n",
      "Epoch 15/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5514 - accuracy: 0.7330\n",
      "Epoch 16/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5509 - accuracy: 0.7324\n",
      "Epoch 17/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5505 - accuracy: 0.7329\n",
      "Epoch 18/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5503 - accuracy: 0.7321\n",
      "Epoch 19/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5493 - accuracy: 0.7335\n",
      "Epoch 20/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5487 - accuracy: 0.7331\n",
      "Epoch 21/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5480 - accuracy: 0.7334\n",
      "Epoch 22/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5474 - accuracy: 0.7331\n",
      "Epoch 23/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5469 - accuracy: 0.7326\n",
      "Epoch 24/50\n",
      "804/804 [==============================] - 1s 897us/step - loss: 0.5466 - accuracy: 0.7342\n",
      "Epoch 25/50\n",
      "804/804 [==============================] - 1s 873us/step - loss: 0.5463 - accuracy: 0.7342\n",
      "Epoch 26/50\n",
      "804/804 [==============================] - 1s 909us/step - loss: 0.5463 - accuracy: 0.7337\n",
      "Epoch 27/50\n",
      "804/804 [==============================] - 1s 867us/step - loss: 0.5459 - accuracy: 0.7350\n",
      "Epoch 28/50\n",
      "804/804 [==============================] - 1s 878us/step - loss: 0.5458 - accuracy: 0.7343\n",
      "Epoch 29/50\n",
      "804/804 [==============================] - 1s 898us/step - loss: 0.5455 - accuracy: 0.7344\n",
      "Epoch 30/50\n",
      "804/804 [==============================] - 1s 941us/step - loss: 0.5458 - accuracy: 0.7348\n",
      "Epoch 31/50\n",
      "804/804 [==============================] - 2s 2ms/step - loss: 0.5455 - accuracy: 0.7339\n",
      "Epoch 32/50\n",
      "804/804 [==============================] - 2s 2ms/step - loss: 0.5455 - accuracy: 0.7345\n",
      "Epoch 33/50\n",
      "804/804 [==============================] - 2s 2ms/step - loss: 0.5453 - accuracy: 0.7349\n",
      "Epoch 34/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5452 - accuracy: 0.7352\n",
      "Epoch 35/50\n",
      "804/804 [==============================] - 2s 2ms/step - loss: 0.5450 - accuracy: 0.7353\n",
      "Epoch 36/50\n",
      "804/804 [==============================] - 2s 2ms/step - loss: 0.5451 - accuracy: 0.7354\n",
      "Epoch 37/50\n",
      "804/804 [==============================] - 2s 2ms/step - loss: 0.5449 - accuracy: 0.7354\n",
      "Epoch 38/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5450 - accuracy: 0.7350\n",
      "Epoch 39/50\n",
      "804/804 [==============================] - 2s 2ms/step - loss: 0.5448 - accuracy: 0.7354\n",
      "Epoch 40/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5449 - accuracy: 0.7356\n",
      "Epoch 41/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5447 - accuracy: 0.7362\n",
      "Epoch 42/50\n",
      "804/804 [==============================] - 2s 2ms/step - loss: 0.5444 - accuracy: 0.7352\n",
      "Epoch 43/50\n",
      "804/804 [==============================] - 2s 2ms/step - loss: 0.5443 - accuracy: 0.7358\n",
      "Epoch 44/50\n",
      "804/804 [==============================] - 2s 2ms/step - loss: 0.5448 - accuracy: 0.7355\n",
      "Epoch 45/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5442 - accuracy: 0.7359\n",
      "Epoch 46/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5444 - accuracy: 0.7348\n",
      "Epoch 47/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5442 - accuracy: 0.7361\n",
      "Epoch 48/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5444 - accuracy: 0.7357\n",
      "Epoch 49/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5443 - accuracy: 0.7353\n",
      "Epoch 50/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5443 - accuracy: 0.7354\n"
     ]
    }
   ],
   "source": [
    "# Fit the model using 50 epochs and the training data\n",
    "fit_model = nn.fit(X_train_scaled, y_train, epochs=50)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Evaluate the model using the test data to determine the model’s loss and accuracy.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "268/268 - 0s - loss: 0.5565 - accuracy: 0.7236 - 325ms/epoch - 1ms/step\n",
      "Loss: 0.5564946532249451, Accuracy: 0.7236151695251465\n"
     ]
    }
   ],
   "source": [
    "# Evaluate the model loss and accuracy metrics using the evaluate method and the test data\n",
    "model_loss, model_accuracy = nn.evaluate(X_test_scaled,y_test,verbose=2)\n",
    "\n",
    "# Display the model loss and accuracy results\n",
    "print(f\"Loss: {model_loss}, Accuracy: {model_accuracy}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 4: Save and export your model to an HDF5 file, and name the file `AlphabetSoup.h5`. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the model's file path\n",
    "file_path = Path(\"AlphabetSoup.h5\")\n",
    "\n",
    "# Export your model to an HDF5 file\n",
    "nn.save(file_path)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Optimize the neural network model\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1: Define at least three new deep neural network models (resulting in the original plus 3 optimization attempts). With each, try to improve on your first model’s predictive accuracy.\n",
    "\n",
    "> **Rewind** Recall that perfect accuracy has a value of 1, so accuracy improves as its value moves closer to 1. To optimize your model for a predictive accuracy as close to 1 as possible, you can use any or all of the following techniques:\n",
    ">\n",
    "> * Adjust the input data by dropping different features columns to ensure that no variables or outliers confuse the model.\n",
    ">\n",
    "> * Add more neurons (nodes) to a hidden layer.\n",
    ">\n",
    "> * Add more hidden layers.\n",
    ">\n",
    "> * Use different activation functions for the hidden layers.\n",
    ">\n",
    "> * Add to or reduce the number of epochs in the training regimen.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Alternative Model 1 - Increase size of neural network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "116"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define the the number of inputs (features) to the model\n",
    "number_input_features_A1 = len(X_train.iloc[0])\n",
    "\n",
    "# Review the number of features\n",
    "number_input_features_A1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the number of neurons in the output layer\n",
    "number_output_neurons_A1 = 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# for this model will try 3 hidden layers and use the number of nodes \n",
    "# equal to the number of input features divided by 4**2 = 16\n",
    "# We will end up with 7 nodes per hidden layer.\n",
    "# Define the number of hidden nodes for the first hidden layer\n",
    "hidden_nodes_layer_A1 = (number_input_features_A1 // 4**2)\n",
    "\n",
    "# Review the number of hidden nodes in the first layer\n",
    "hidden_nodes_layer_A1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the Sequential model instance\n",
    "nn_A1 = Sequential()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_1\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " dense_3 (Dense)             (None, 7)                 819       \n",
      "                                                                 \n",
      " dense_4 (Dense)             (None, 7)                 56        \n",
      "                                                                 \n",
      " dense_5 (Dense)             (None, 7)                 56        \n",
      "                                                                 \n",
      " dense_6 (Dense)             (None, 7)                 56        \n",
      "                                                                 \n",
      " dense_7 (Dense)             (None, 1)                 8         \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 995\n",
      "Trainable params: 995\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# we will setup 4 hidden layer using 7 nodes each :\n",
    "nn_A1.add(Dense(units=hidden_nodes_layer_A1, input_dim=number_input_features_A1, activation=\"relu\"))\n",
    "nn_A1.add(Dense(units=hidden_nodes_layer_A1, input_dim=number_input_features_A1, activation=\"relu\"))\n",
    "nn_A1.add(Dense(units=hidden_nodes_layer_A1, input_dim=number_input_features_A1, activation=\"relu\"))\n",
    "nn_A1.add(Dense(units=hidden_nodes_layer_A1, input_dim=number_input_features_A1, activation=\"relu\"))\n",
    "\n",
    "# Output layer\n",
    "nn_A1.add(Dense(units=number_output_neurons_A1, activation=\"sigmoid\"))\n",
    "\n",
    "# Check the structure of the model\n",
    "nn_A1.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compile the Sequential model\n",
    "nn_A1.compile(loss=\"binary_crossentropy\", optimizer=\"adam\", metrics=[\"accuracy\"])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/50\n",
      "804/804 [==============================] - 2s 1ms/step - loss: 0.6106 - accuracy: 0.6869\n",
      "Epoch 2/50\n",
      "804/804 [==============================] - 1s 940us/step - loss: 0.5619 - accuracy: 0.7267\n",
      "Epoch 3/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5555 - accuracy: 0.7280\n",
      "Epoch 4/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5515 - accuracy: 0.7322\n",
      "Epoch 5/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5504 - accuracy: 0.7329\n",
      "Epoch 6/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5494 - accuracy: 0.7329\n",
      "Epoch 7/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5482 - accuracy: 0.7331\n",
      "Epoch 8/50\n",
      "804/804 [==============================] - 1s 992us/step - loss: 0.5479 - accuracy: 0.7331\n",
      "Epoch 9/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5473 - accuracy: 0.7342\n",
      "Epoch 10/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5469 - accuracy: 0.7339\n",
      "Epoch 11/50\n",
      "804/804 [==============================] - 1s 928us/step - loss: 0.5462 - accuracy: 0.7339\n",
      "Epoch 12/50\n",
      "804/804 [==============================] - 1s 937us/step - loss: 0.5458 - accuracy: 0.7350\n",
      "Epoch 13/50\n",
      "804/804 [==============================] - 1s 984us/step - loss: 0.5453 - accuracy: 0.7347\n",
      "Epoch 14/50\n",
      "804/804 [==============================] - 1s 979us/step - loss: 0.5454 - accuracy: 0.7338\n",
      "Epoch 15/50\n",
      "804/804 [==============================] - 1s 981us/step - loss: 0.5448 - accuracy: 0.7352\n",
      "Epoch 16/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5448 - accuracy: 0.7343\n",
      "Epoch 17/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5449 - accuracy: 0.7352\n",
      "Epoch 18/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5447 - accuracy: 0.7353\n",
      "Epoch 19/50\n",
      "804/804 [==============================] - 2s 2ms/step - loss: 0.5444 - accuracy: 0.7365\n",
      "Epoch 20/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5441 - accuracy: 0.7358\n",
      "Epoch 21/50\n",
      "804/804 [==============================] - 2s 2ms/step - loss: 0.5441 - accuracy: 0.7352\n",
      "Epoch 22/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5438 - accuracy: 0.7355\n",
      "Epoch 23/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5438 - accuracy: 0.7360\n",
      "Epoch 24/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5436 - accuracy: 0.7358\n",
      "Epoch 25/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5435 - accuracy: 0.7348\n",
      "Epoch 26/50\n",
      "804/804 [==============================] - 1s 953us/step - loss: 0.5436 - accuracy: 0.7350\n",
      "Epoch 27/50\n",
      "804/804 [==============================] - 1s 999us/step - loss: 0.5433 - accuracy: 0.7363\n",
      "Epoch 28/50\n",
      "804/804 [==============================] - 1s 950us/step - loss: 0.5434 - accuracy: 0.7356\n",
      "Epoch 29/50\n",
      "804/804 [==============================] - 1s 924us/step - loss: 0.5432 - accuracy: 0.7361\n",
      "Epoch 30/50\n",
      "804/804 [==============================] - 1s 992us/step - loss: 0.5429 - accuracy: 0.7359\n",
      "Epoch 31/50\n",
      "804/804 [==============================] - 1s 953us/step - loss: 0.5432 - accuracy: 0.7364\n",
      "Epoch 32/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5435 - accuracy: 0.7356\n",
      "Epoch 33/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5432 - accuracy: 0.7361\n",
      "Epoch 34/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5430 - accuracy: 0.7355\n",
      "Epoch 35/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5427 - accuracy: 0.7365\n",
      "Epoch 36/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5426 - accuracy: 0.7370\n",
      "Epoch 37/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5425 - accuracy: 0.7362\n",
      "Epoch 38/50\n",
      "804/804 [==============================] - 1s 975us/step - loss: 0.5426 - accuracy: 0.7356\n",
      "Epoch 39/50\n",
      "804/804 [==============================] - 1s 987us/step - loss: 0.5425 - accuracy: 0.7366\n",
      "Epoch 40/50\n",
      "804/804 [==============================] - 1s 958us/step - loss: 0.5423 - accuracy: 0.7366\n",
      "Epoch 41/50\n",
      "804/804 [==============================] - 1s 953us/step - loss: 0.5421 - accuracy: 0.7377\n",
      "Epoch 42/50\n",
      "804/804 [==============================] - 1s 977us/step - loss: 0.5424 - accuracy: 0.7364\n",
      "Epoch 43/50\n",
      "804/804 [==============================] - 1s 996us/step - loss: 0.5421 - accuracy: 0.7367\n",
      "Epoch 44/50\n",
      "804/804 [==============================] - 1s 918us/step - loss: 0.5418 - accuracy: 0.7363\n",
      "Epoch 45/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5418 - accuracy: 0.7364\n",
      "Epoch 46/50\n",
      "804/804 [==============================] - 1s 981us/step - loss: 0.5418 - accuracy: 0.7359\n",
      "Epoch 47/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5420 - accuracy: 0.7361\n",
      "Epoch 48/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5417 - accuracy: 0.7360\n",
      "Epoch 49/50\n",
      "804/804 [==============================] - 2s 2ms/step - loss: 0.5415 - accuracy: 0.7367\n",
      "Epoch 50/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5411 - accuracy: 0.7360\n"
     ]
    }
   ],
   "source": [
    "# Fit the model using 50 epochs and the training data\n",
    "fit_model_A1 = nn_A1.fit(X_train_scaled, y_train, epochs=50)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Evaluate the model using the test data to determine the model’s loss and accuracy.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "268/268 - 1s - loss: 0.5574 - accuracy: 0.7202 - 525ms/epoch - 2ms/step\n",
      "Loss: 0.5574216842651367, Accuracy: 0.7202332615852356\n"
     ]
    }
   ],
   "source": [
    "# Evaluate the model loss and accuracy metrics using the evaluate method and the test data\n",
    "model_loss_A1, model_accuracy_A1 = nn_A1.evaluate(X_test_scaled,y_test,verbose=2)\n",
    "\n",
    "# Display the model loss and accuracy results\n",
    "print(f\"Loss: {model_loss_A1}, Accuracy: {model_accuracy_A1}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the model's file path\n",
    "file_path = Path(\"AlphabetSoup_A1.h5\")\n",
    "\n",
    "# Export your model to an HDF5 file\n",
    "nn_A1.save(file_path)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "### Alternative Model 2 - Dropout layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the `Dropout` layer\n",
    "from keras.layers import Dropout\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "116"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define the the number of inputs (features) to the model\n",
    "number_input_features_A2 = len(X_train.iloc[0])\n",
    "\n",
    "# Review the number of features\n",
    "number_input_features_A2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the number of neurons in the output layer\n",
    "number_output_neurons_A2 = 1\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# for this model will use the number of nodes as in alternative_1 model,\n",
    "# but we will add dropout lary overfitting: \n",
    "# Define the number of hidden nodes for the first hidden layer\n",
    "hidden_nodes_layer_A2 = (number_input_features // 4**2)\n",
    "\n",
    "# Review the number of hidden nodes in the first layer\n",
    "hidden_nodes_layer_A2\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Create the Sequential model instance\n",
    "nn_A2 = Sequential()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.25"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# for the dropout layers, we will calculate the dropout_probability, we will use the \n",
    "# probability of keeping a node in the layer as 75%:\n",
    "keep_probability = 0.75\n",
    "dropout_probability = (1 - keep_probability)\n",
    "dropout_probability"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_2\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " dense_8 (Dense)             (None, 7)                 819       \n",
      "                                                                 \n",
      " dropout (Dropout)           (None, 7)                 0         \n",
      "                                                                 \n",
      " dense_9 (Dense)             (None, 7)                 56        \n",
      "                                                                 \n",
      " dense_10 (Dense)            (None, 1)                 8         \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 883\n",
      "Trainable params: 883\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# We will create the 3 hidden layers with Dropout(), and we will use the same number\n",
    "# of nodes per layer.\n",
    "# layer 1:\n",
    "nn_A2.add(Dense(units=hidden_nodes_layer_A2, input_dim=number_input_features_A2, activation=\"relu\"))\n",
    "# Now add a dropout layer\n",
    "# layer 2:\n",
    "nn_A2.add(Dropout(dropout_probability))\n",
    "\n",
    "# layer 3:\n",
    "nn_A2.add(Dense(units=hidden_nodes_layer_A2, input_dim=number_input_features_A2, activation=\"relu\"))\n",
    "\n",
    "# Output layer\n",
    "nn_A2.add(Dense(units=number_output_neurons_A2, activation=\"sigmoid\"))\n",
    "\n",
    "# Check the structure of the model\n",
    "nn_A2.summary()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compile the Sequential model\n",
    "nn_A2.compile(loss=\"binary_crossentropy\", optimizer=\"adam\", metrics=[\"accuracy\"])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/50\n",
      "804/804 [==============================] - 2s 1ms/step - loss: 0.6294 - accuracy: 0.6854\n",
      "Epoch 2/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5920 - accuracy: 0.7126\n",
      "Epoch 3/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5815 - accuracy: 0.7198\n",
      "Epoch 4/50\n",
      "804/804 [==============================] - 1s 991us/step - loss: 0.5726 - accuracy: 0.7220\n",
      "Epoch 5/50\n",
      "804/804 [==============================] - 1s 889us/step - loss: 0.5713 - accuracy: 0.7191\n",
      "Epoch 6/50\n",
      "804/804 [==============================] - 1s 934us/step - loss: 0.5710 - accuracy: 0.7180\n",
      "Epoch 7/50\n",
      "804/804 [==============================] - 1s 956us/step - loss: 0.5673 - accuracy: 0.7170\n",
      "Epoch 8/50\n",
      "804/804 [==============================] - 1s 929us/step - loss: 0.5664 - accuracy: 0.7177\n",
      "Epoch 9/50\n",
      "804/804 [==============================] - 1s 867us/step - loss: 0.5642 - accuracy: 0.7189\n",
      "Epoch 10/50\n",
      "804/804 [==============================] - 1s 920us/step - loss: 0.5620 - accuracy: 0.7214\n",
      "Epoch 11/50\n",
      "804/804 [==============================] - 1s 997us/step - loss: 0.5635 - accuracy: 0.7207\n",
      "Epoch 12/50\n",
      "804/804 [==============================] - 1s 914us/step - loss: 0.5605 - accuracy: 0.7205\n",
      "Epoch 13/50\n",
      "804/804 [==============================] - 1s 920us/step - loss: 0.5597 - accuracy: 0.7241\n",
      "Epoch 14/50\n",
      "804/804 [==============================] - 1s 933us/step - loss: 0.5604 - accuracy: 0.7241\n",
      "Epoch 15/50\n",
      "804/804 [==============================] - 1s 974us/step - loss: 0.5584 - accuracy: 0.7250\n",
      "Epoch 16/50\n",
      "804/804 [==============================] - 1s 879us/step - loss: 0.5595 - accuracy: 0.7248\n",
      "Epoch 17/50\n",
      "804/804 [==============================] - 1s 969us/step - loss: 0.5584 - accuracy: 0.7259\n",
      "Epoch 18/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5593 - accuracy: 0.7254\n",
      "Epoch 19/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5596 - accuracy: 0.7271\n",
      "Epoch 20/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5577 - accuracy: 0.7282\n",
      "Epoch 21/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5583 - accuracy: 0.7265\n",
      "Epoch 22/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5582 - accuracy: 0.7261\n",
      "Epoch 23/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5574 - accuracy: 0.7260\n",
      "Epoch 24/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5568 - accuracy: 0.7275\n",
      "Epoch 25/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5575 - accuracy: 0.7259\n",
      "Epoch 26/50\n",
      "804/804 [==============================] - 1s 986us/step - loss: 0.5570 - accuracy: 0.7261\n",
      "Epoch 27/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5580 - accuracy: 0.7251\n",
      "Epoch 28/50\n",
      "804/804 [==============================] - 1s 991us/step - loss: 0.5575 - accuracy: 0.7260\n",
      "Epoch 29/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5567 - accuracy: 0.7279\n",
      "Epoch 30/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5569 - accuracy: 0.7278\n",
      "Epoch 31/50\n",
      "804/804 [==============================] - 1s 982us/step - loss: 0.5556 - accuracy: 0.7295\n",
      "Epoch 32/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5557 - accuracy: 0.7282\n",
      "Epoch 33/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5575 - accuracy: 0.7267\n",
      "Epoch 34/50\n",
      "804/804 [==============================] - 2s 3ms/step - loss: 0.5583 - accuracy: 0.7273\n",
      "Epoch 35/50\n",
      "804/804 [==============================] - 2s 2ms/step - loss: 0.5553 - accuracy: 0.7276\n",
      "Epoch 36/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5556 - accuracy: 0.7299\n",
      "Epoch 37/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5559 - accuracy: 0.7289\n",
      "Epoch 38/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5553 - accuracy: 0.7280\n",
      "Epoch 39/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5561 - accuracy: 0.7279\n",
      "Epoch 40/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5547 - accuracy: 0.7277\n",
      "Epoch 41/50\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5546 - accuracy: 0.7280\n",
      "Epoch 42/50\n",
      "804/804 [==============================] - 1s 950us/step - loss: 0.5555 - accuracy: 0.7276\n",
      "Epoch 43/50\n",
      "804/804 [==============================] - 1s 979us/step - loss: 0.5565 - accuracy: 0.7284\n",
      "Epoch 44/50\n",
      "804/804 [==============================] - 1s 895us/step - loss: 0.5559 - accuracy: 0.7272\n",
      "Epoch 45/50\n",
      "804/804 [==============================] - 1s 976us/step - loss: 0.5566 - accuracy: 0.7283\n",
      "Epoch 46/50\n",
      "804/804 [==============================] - 1s 989us/step - loss: 0.5564 - accuracy: 0.7270\n",
      "Epoch 47/50\n",
      "804/804 [==============================] - 1s 919us/step - loss: 0.5548 - accuracy: 0.7276\n",
      "Epoch 48/50\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5556 - accuracy: 0.7301\n",
      "Epoch 49/50\n",
      "804/804 [==============================] - 2s 2ms/step - loss: 0.5555 - accuracy: 0.7271\n",
      "Epoch 50/50\n",
      "804/804 [==============================] - 1s 914us/step - loss: 0.5554 - accuracy: 0.7303\n"
     ]
    }
   ],
   "source": [
    "# Fit the model using 50 epochs and the training data\n",
    "fit_model_A2 = nn_A2.fit(X_train_scaled, y_train, epochs=50)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "268/268 - 0s - loss: 0.5752 - accuracy: 0.7258 - 243ms/epoch - 908us/step\n",
      "Loss: 0.5752180814743042, Accuracy: 0.7258309125900269\n"
     ]
    }
   ],
   "source": [
    "# Evaluate the model loss and accuracy metrics using the evaluate method and the test data\n",
    "model_loss_A2, model_accuracy_A2 = nn_A2.evaluate(X_test_scaled,y_test,verbose=2)\n",
    "\n",
    "# Display the model loss and accuracy results\n",
    "print(f\"Loss: {model_loss_A2}, Accuracy: {model_accuracy_A2}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the model's file path\n",
    "file_path = Path(\"AlphabetSoup_A2.h5\")\n",
    "\n",
    "# Export your model to an HDF5 file\n",
    "nn_A2.save(file_path)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "### Alternative Model 3 - Make model smaller and decrease number of epochs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the nodes on the first layer are: 4, on the second layer we have 3 nodes.\n"
     ]
    }
   ],
   "source": [
    "# Define the the number of inputs (features) to the model\n",
    "number_input_features_A3 = len(X_train.iloc[0])\n",
    "\n",
    "# Define the number of neurons in the output layer\n",
    "number_output_neurons_A3 = 1\n",
    "\n",
    "# Review the number of features# Define the number of hidden nodes for the first hidden layer\n",
    "hidden_nodes_layer1_A3 = 4\n",
    "hidden_nodes_layer2_A3 = 3\n",
    "\n",
    "# Review the number of hidden nodes in the first layer\n",
    "print(f\"the nodes on the first layer are: {hidden_nodes_layer1_A3}, on the second layer we have {hidden_nodes_layer2_A3} nodes.\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Create the Sequential model instance\n",
    "nn_A3 = Sequential()\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_3\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " dense_11 (Dense)            (None, 4)                 468       \n",
      "                                                                 \n",
      " dense_12 (Dense)            (None, 3)                 15        \n",
      "                                                                 \n",
      " dense_13 (Dense)            (None, 1)                 4         \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 487\n",
      "Trainable params: 487\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# we will setup 2 hidden layer using 4 nodes per layer:\n",
    "nn_A3.add(Dense(units=hidden_nodes_layer1_A3, input_dim=number_input_features_A3, activation=\"relu\"))\n",
    "nn_A3.add(Dense(units=hidden_nodes_layer2_A3, input_dim=number_input_features_A3, activation=\"relu\"))\n",
    "\n",
    "# Output layer\n",
    "nn_A3.add(Dense(units=number_output_neurons_A3, activation=\"sigmoid\"))\n",
    "\n",
    "# Check the structure of the model\n",
    "nn_A3.summary()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compile the Sequential model\n",
    "nn_A3.compile(loss=\"binary_crossentropy\", optimizer=\"adam\", metrics=[\"accuracy\"])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/25\n",
      "804/804 [==============================] - 1s 917us/step - loss: 0.6252 - accuracy: 0.6788\n",
      "Epoch 2/25\n",
      "804/804 [==============================] - 1s 854us/step - loss: 0.5669 - accuracy: 0.7225\n",
      "Epoch 3/25\n",
      "804/804 [==============================] - 1s 873us/step - loss: 0.5579 - accuracy: 0.7298\n",
      "Epoch 4/25\n",
      "804/804 [==============================] - 1s 949us/step - loss: 0.5550 - accuracy: 0.7310\n",
      "Epoch 5/25\n",
      "804/804 [==============================] - 1s 861us/step - loss: 0.5535 - accuracy: 0.7307\n",
      "Epoch 6/25\n",
      "804/804 [==============================] - 1s 838us/step - loss: 0.5522 - accuracy: 0.7317\n",
      "Epoch 7/25\n",
      "804/804 [==============================] - 1s 919us/step - loss: 0.5512 - accuracy: 0.7322\n",
      "Epoch 8/25\n",
      "804/804 [==============================] - 1s 891us/step - loss: 0.5504 - accuracy: 0.7324\n",
      "Epoch 9/25\n",
      "804/804 [==============================] - 1s 851us/step - loss: 0.5501 - accuracy: 0.7322\n",
      "Epoch 10/25\n",
      "804/804 [==============================] - 1s 876us/step - loss: 0.5494 - accuracy: 0.7324\n",
      "Epoch 11/25\n",
      "804/804 [==============================] - 1s 919us/step - loss: 0.5491 - accuracy: 0.7327\n",
      "Epoch 12/25\n",
      "804/804 [==============================] - 1s 879us/step - loss: 0.5493 - accuracy: 0.7322\n",
      "Epoch 13/25\n",
      "804/804 [==============================] - 1s 840us/step - loss: 0.5486 - accuracy: 0.7325\n",
      "Epoch 14/25\n",
      "804/804 [==============================] - 1s 951us/step - loss: 0.5487 - accuracy: 0.7327\n",
      "Epoch 15/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5484 - accuracy: 0.7324\n",
      "Epoch 16/25\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5483 - accuracy: 0.7322\n",
      "Epoch 17/25\n",
      "804/804 [==============================] - 2s 3ms/step - loss: 0.5483 - accuracy: 0.7323\n",
      "Epoch 18/25\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5482 - accuracy: 0.7330\n",
      "Epoch 19/25\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5482 - accuracy: 0.7329\n",
      "Epoch 20/25\n",
      "804/804 [==============================] - 2s 2ms/step - loss: 0.5479 - accuracy: 0.7331\n",
      "Epoch 21/25\n",
      "804/804 [==============================] - 1s 2ms/step - loss: 0.5478 - accuracy: 0.7328\n",
      "Epoch 22/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5479 - accuracy: 0.7329\n",
      "Epoch 23/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5477 - accuracy: 0.7331\n",
      "Epoch 24/25\n",
      "804/804 [==============================] - 1s 1ms/step - loss: 0.5475 - accuracy: 0.7329\n",
      "Epoch 25/25\n",
      "804/804 [==============================] - 1s 894us/step - loss: 0.5475 - accuracy: 0.7336\n"
     ]
    }
   ],
   "source": [
    "# For this alternative model, we will use a smaller number of epochs,\n",
    "# we will use 25 epochs:\n",
    "fit_model_A3 = nn_A3.fit(X_train_scaled, y_train, epochs=25)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "268/268 - 0s - loss: 0.5572 - accuracy: 0.7245 - 247ms/epoch - 923us/step\n",
      "Loss: 0.5571622252464294, Accuracy: 0.7245481014251709\n"
     ]
    }
   ],
   "source": [
    "# Evaluate the model loss and accuracy metrics using the evaluate method and the test data\n",
    "model_loss_A3, model_accuracy_A3 = nn_A3.evaluate(X_test_scaled,y_test,verbose=2)\n",
    "\n",
    "# Display the model loss and accuracy results\n",
    "print(f\"Loss: {model_loss_A3}, Accuracy: {model_accuracy_A3}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the model's file path\n",
    "file_path = Path(\"AlphabetSoup_A3.h5\")\n",
    "\n",
    "# Export your model to an HDF5 file\n",
    "nn_A3.save(file_path)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: After finishing your models, display the accuracy scores achieved by each model, and compare the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Original Model Results\n",
      "268/268 - 0s - loss: 0.5565 - accuracy: 0.7236 - 186ms/epoch - 692us/step\n",
      "Loss: 0.5564946532249451, Accuracy: 0.7236151695251465\n"
     ]
    }
   ],
   "source": [
    "print(\"Original Model Results\")\n",
    "\n",
    "# Evaluate the model loss and accuracy metrics using the evaluate method and the test data\n",
    "model_loss, model_accuracy = nn.evaluate(X_test_scaled,y_test,verbose=2)\n",
    "\n",
    "# Display the model loss and accuracy results\n",
    "print(f\"Loss: {model_loss}, Accuracy: {model_accuracy}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Alternative Model 1 Results\n",
      "268/268 - 0s - loss: 0.5574 - accuracy: 0.7202 - 183ms/epoch - 681us/step\n",
      "Loss: 0.5574216842651367, Accuracy: 0.7202332615852356\n"
     ]
    }
   ],
   "source": [
    "print(\"Alternative Model 1 Results\")\n",
    "\n",
    "# Evaluate the model loss and accuracy metrics using the evaluate method and the test data\n",
    "model_loss, model_accuracy = nn_A1.evaluate(X_test_scaled,y_test,verbose=2)\n",
    "\n",
    "# Display the model loss and accuracy results\n",
    "print(f\"Loss: {model_loss}, Accuracy: {model_accuracy}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Alternative Model 2 Results\n",
      "268/268 - 0s - loss: 0.5752 - accuracy: 0.7258 - 182ms/epoch - 677us/step\n",
      "Loss: 0.5752180814743042, Accuracy: 0.7258309125900269\n"
     ]
    }
   ],
   "source": [
    "print(\"Alternative Model 2 Results\")\n",
    "\n",
    "# Evaluate the model loss and accuracy metrics using the evaluate method and the test data\n",
    "model_loss, model_accuracy = nn_A2.evaluate(X_test_scaled,y_test,verbose=2)\n",
    "\n",
    "# Display the model loss and accuracy results\n",
    "print(f\"Loss: {model_loss}, Accuracy: {model_accuracy}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Alternative Model 3 Results\n",
      "268/268 - 0s - loss: 0.5572 - accuracy: 0.7245 - 187ms/epoch - 700us/step\n",
      "Loss: 0.5571622252464294, Accuracy: 0.7245481014251709\n"
     ]
    }
   ],
   "source": [
    "print(\"Alternative Model 3 Results\")\n",
    "\n",
    "# Evaluate the model loss and accuracy metrics using the evaluate method and the test data\n",
    "model_loss, model_accuracy = nn_A3.evaluate(X_test_scaled,y_test,verbose=2)\n",
    "\n",
    "# Display the model loss and accuracy results\n",
    "print(f\"Loss: {model_loss}, Accuracy: {model_accuracy}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Comparing the results of the 4 models:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We made significant changes in all the models and we can notice some changes in the losses and accuracy results, but these changes are small.\n",
    "\n",
    "in the Losses resutl, I could see more changes than in the accuracy results. \n",
    "\n",
    "We we noticed is that we reduced the number of epochs by half in the last alternative and this did not make significant changes either. We notice that on the alternative models with 50 epochs, the losses and accuracy parameters stabilized also around the 25th epoch and did not change significantly after that epoch. So we can conclude that we do can reduce the number of epochs significantly and obtain the same results that with larger number of epochs and this have a significant impact in the time it takes to handle large data models.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Set the file path for the second alternative model\n",
    "file_path = # YOUR CODE HERE\n",
    "\n",
    "# Export your model to a HDF5 file\n",
    "# YOUR CODE HERE\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Set the file path for the third alternative model\n",
    "file_path = # YOUR CODE HERE\n",
    "\n",
    "# Export your model to a HDF5 file\n",
    "# YOUR CODE HERE\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}