DZ_33.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Задание 1**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Импортируем библиотеки pandas и numpy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Загружаем набор данных. Создаем датафреймы X и y из этих данных."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_boston\n",
    "boston = load_boston()\n",
    "data = boston[\"data\"]\n",
    "\n",
    "feature_names = boston[\"feature_names\"]\n",
    "X = pd.DataFrame(data, columns=feature_names)\n",
    "\n",
    "target = boston[\"target\"]\n",
    "y= pd.DataFrame(target,columns=[\"price\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Разделим эти датафреймы на тренировочные (X_train, y_train) и тестовые (X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.35, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Создадим модель линейной регрессии"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "lr=LinearRegression()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Обучим модель на тренировочных данных (используйте все признаки) и сделайте предсказание на тестовых."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "lr.fit(X_train, y_train)\n",
    "y_pred = lr.predict(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Вычислим R2 полученных предказаний"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.661492992776259"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.metrics import r2_score\n",
    "r2_score(y_pred,y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Задание 2**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Создаем модель под названием model с помощью RandomForestRegressor из модуля sklearn.ensemble."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestRegressor\n",
    "model=RandomForestRegressor(n_estimators=1000, max_depth=12,random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Обучим модель на тренировочных данных"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestRegressor(max_depth=12, n_estimators=1000, random_state=42)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(X_train,y_train.values[:, 0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Сделаем предсказание на тестовых данных и посчитаем R2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8126231105547794"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_pred = model.predict(X_test)\n",
    "r2_score(y_pred,y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Как видим при использовании Случайных лесов коэффциент детерминации у нас вырос до 0.8, что указывает на его лучшую роботоспособность в данном случае."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Задание 3**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Вызовем документацию для класса RandomForestRegressor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    " RandomForestRegressor?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "С помощью feature_importances_ найдем сумму всех показателей важности"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum=np.sum(model.feature_importances_)\n",
    "sum"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Установим, какие два признака показывают наибольшую важность."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.03218709, 0.00298766])"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.feature_importances_.sort()\n",
    "model.feature_importances_[0:2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Задание 4**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Импортируйте из соответствующих модулей RandomForestClassifier, GridSearchCV и train_test_split."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Загрузим датасет creditcard.csv и создайте датафрейм df."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('creditcard.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "С помощью метода value_counts с аргументом normalize=True убеждаемся в том, что выборка несбалансирована"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    0.998273\n",
       "1    0.001727\n",
       "Name: Class, dtype: float64"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"Class\"].value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Используя метод info, проверяем, все ли столбцы содержат числовые данные и нет ли в них пропусков"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 284807 entries, 0 to 284806\n",
      "Data columns (total 31 columns):\n",
      " #   Column  Non-Null Count   Dtype  \n",
      "---  ------  --------------   -----  \n",
      " 0   Time    284807 non-null  float64\n",
      " 1   V1      284807 non-null  float64\n",
      " 2   V2      284807 non-null  float64\n",
      " 3   V3      284807 non-null  float64\n",
      " 4   V4      284807 non-null  float64\n",
      " 5   V5      284807 non-null  float64\n",
      " 6   V6      284807 non-null  float64\n",
      " 7   V7      284807 non-null  float64\n",
      " 8   V8      284807 non-null  float64\n",
      " 9   V9      284807 non-null  float64\n",
      " 10  V10     284807 non-null  float64\n",
      " 11  V11     284807 non-null  float64\n",
      " 12  V12     284807 non-null  float64\n",
      " 13  V13     284807 non-null  float64\n",
      " 14  V14     284807 non-null  float64\n",
      " 15  V15     284807 non-null  float64\n",
      " 16  V16     284807 non-null  float64\n",
      " 17  V17     284807 non-null  float64\n",
      " 18  V18     284807 non-null  float64\n",
      " 19  V19     284807 non-null  float64\n",
      " 20  V20     284807 non-null  float64\n",
      " 21  V21     284807 non-null  float64\n",
      " 22  V22     284807 non-null  float64\n",
      " 23  V23     284807 non-null  float64\n",
      " 24  V24     284807 non-null  float64\n",
      " 25  V25     284807 non-null  float64\n",
      " 26  V26     284807 non-null  float64\n",
      " 27  V27     284807 non-null  float64\n",
      " 28  V28     284807 non-null  float64\n",
      " 29  Amount  284807 non-null  float64\n",
      " 30  Class   284807 non-null  int64  \n",
      "dtypes: float64(30), int64(1)\n",
      "memory usage: 67.4 MB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Просмотрите первые 10 строк датафрейма df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Time</th>\n",
       "      <th>V1</th>\n",
       "      <th>V2</th>\n",
       "      <th>V3</th>\n",
       "      <th>V4</th>\n",
       "      <th>V5</th>\n",
       "      <th>V6</th>\n",
       "      <th>V7</th>\n",
       "      <th>V8</th>\n",
       "      <th>V9</th>\n",
       "      <th>V10</th>\n",
       "      <th>V11</th>\n",
       "      <th>V12</th>\n",
       "      <th>V13</th>\n",
       "      <th>V14</th>\n",
       "      <th>V15</th>\n",
       "      <th>V16</th>\n",
       "      <th>V17</th>\n",
       "      <th>V18</th>\n",
       "      <th>V19</th>\n",
       "      <th>V20</th>\n",
       "      <th>V21</th>\n",
       "      <th>V22</th>\n",
       "      <th>V23</th>\n",
       "      <th>V24</th>\n",
       "      <th>V25</th>\n",
       "      <th>V26</th>\n",
       "      <th>V27</th>\n",
       "      <th>V28</th>\n",
       "      <th>Amount</th>\n",
       "      <th>Class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.0</td>\n",
       "      <td>-1.359807</td>\n",
       "      <td>-0.072781</td>\n",
       "      <td>2.536347</td>\n",
       "      <td>1.378155</td>\n",
       "      <td>-0.338321</td>\n",
       "      <td>0.462388</td>\n",
       "      <td>0.239599</td>\n",
       "      <td>0.098698</td>\n",
       "      <td>0.363787</td>\n",
       "      <td>0.090794</td>\n",
       "      <td>-0.551600</td>\n",
       "      <td>-0.617801</td>\n",
       "      <td>-0.991390</td>\n",
       "      <td>-0.311169</td>\n",
       "      <td>1.468177</td>\n",
       "      <td>-0.470401</td>\n",
       "      <td>0.207971</td>\n",
       "      <td>0.025791</td>\n",
       "      <td>0.403993</td>\n",
       "      <td>0.251412</td>\n",
       "      <td>-0.018307</td>\n",
       "      <td>0.277838</td>\n",
       "      <td>-0.110474</td>\n",
       "      <td>0.066928</td>\n",
       "      <td>0.128539</td>\n",
       "      <td>-0.189115</td>\n",
       "      <td>0.133558</td>\n",
       "      <td>-0.021053</td>\n",
       "      <td>149.62</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>1.191857</td>\n",
       "      <td>0.266151</td>\n",
       "      <td>0.166480</td>\n",
       "      <td>0.448154</td>\n",
       "      <td>0.060018</td>\n",
       "      <td>-0.082361</td>\n",
       "      <td>-0.078803</td>\n",
       "      <td>0.085102</td>\n",
       "      <td>-0.255425</td>\n",
       "      <td>-0.166974</td>\n",
       "      <td>1.612727</td>\n",
       "      <td>1.065235</td>\n",
       "      <td>0.489095</td>\n",
       "      <td>-0.143772</td>\n",
       "      <td>0.635558</td>\n",
       "      <td>0.463917</td>\n",
       "      <td>-0.114805</td>\n",
       "      <td>-0.183361</td>\n",
       "      <td>-0.145783</td>\n",
       "      <td>-0.069083</td>\n",
       "      <td>-0.225775</td>\n",
       "      <td>-0.638672</td>\n",
       "      <td>0.101288</td>\n",
       "      <td>-0.339846</td>\n",
       "      <td>0.167170</td>\n",
       "      <td>0.125895</td>\n",
       "      <td>-0.008983</td>\n",
       "      <td>0.014724</td>\n",
       "      <td>2.69</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>-1.358354</td>\n",
       "      <td>-1.340163</td>\n",
       "      <td>1.773209</td>\n",
       "      <td>0.379780</td>\n",
       "      <td>-0.503198</td>\n",
       "      <td>1.800499</td>\n",
       "      <td>0.791461</td>\n",
       "      <td>0.247676</td>\n",
       "      <td>-1.514654</td>\n",
       "      <td>0.207643</td>\n",
       "      <td>0.624501</td>\n",
       "      <td>0.066084</td>\n",
       "      <td>0.717293</td>\n",
       "      <td>-0.165946</td>\n",
       "      <td>2.345865</td>\n",
       "      <td>-2.890083</td>\n",
       "      <td>1.109969</td>\n",
       "      <td>-0.121359</td>\n",
       "      <td>-2.261857</td>\n",
       "      <td>0.524980</td>\n",
       "      <td>0.247998</td>\n",
       "      <td>0.771679</td>\n",
       "      <td>0.909412</td>\n",
       "      <td>-0.689281</td>\n",
       "      <td>-0.327642</td>\n",
       "      <td>-0.139097</td>\n",
       "      <td>-0.055353</td>\n",
       "      <td>-0.059752</td>\n",
       "      <td>378.66</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>-0.966272</td>\n",
       "      <td>-0.185226</td>\n",
       "      <td>1.792993</td>\n",
       "      <td>-0.863291</td>\n",
       "      <td>-0.010309</td>\n",
       "      <td>1.247203</td>\n",
       "      <td>0.237609</td>\n",
       "      <td>0.377436</td>\n",
       "      <td>-1.387024</td>\n",
       "      <td>-0.054952</td>\n",
       "      <td>-0.226487</td>\n",
       "      <td>0.178228</td>\n",
       "      <td>0.507757</td>\n",
       "      <td>-0.287924</td>\n",
       "      <td>-0.631418</td>\n",
       "      <td>-1.059647</td>\n",
       "      <td>-0.684093</td>\n",
       "      <td>1.965775</td>\n",
       "      <td>-1.232622</td>\n",
       "      <td>-0.208038</td>\n",
       "      <td>-0.108300</td>\n",
       "      <td>0.005274</td>\n",
       "      <td>-0.190321</td>\n",
       "      <td>-1.175575</td>\n",
       "      <td>0.647376</td>\n",
       "      <td>-0.221929</td>\n",
       "      <td>0.062723</td>\n",
       "      <td>0.061458</td>\n",
       "      <td>123.50</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2.0</td>\n",
       "      <td>-1.158233</td>\n",
       "      <td>0.877737</td>\n",
       "      <td>1.548718</td>\n",
       "      <td>0.403034</td>\n",
       "      <td>-0.407193</td>\n",
       "      <td>0.095921</td>\n",
       "      <td>0.592941</td>\n",
       "      <td>-0.270533</td>\n",
       "      <td>0.817739</td>\n",
       "      <td>0.753074</td>\n",
       "      <td>-0.822843</td>\n",
       "      <td>0.538196</td>\n",
       "      <td>1.345852</td>\n",
       "      <td>-1.119670</td>\n",
       "      <td>0.175121</td>\n",
       "      <td>-0.451449</td>\n",
       "      <td>-0.237033</td>\n",
       "      <td>-0.038195</td>\n",
       "      <td>0.803487</td>\n",
       "      <td>0.408542</td>\n",
       "      <td>-0.009431</td>\n",
       "      <td>0.798278</td>\n",
       "      <td>-0.137458</td>\n",
       "      <td>0.141267</td>\n",
       "      <td>-0.206010</td>\n",
       "      <td>0.502292</td>\n",
       "      <td>0.219422</td>\n",
       "      <td>0.215153</td>\n",
       "      <td>69.99</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Time        V1        V2        V3        V4        V5        V6        V7  \\\n",
       "0   0.0 -1.359807 -0.072781  2.536347  1.378155 -0.338321  0.462388  0.239599   \n",
       "1   0.0  1.191857  0.266151  0.166480  0.448154  0.060018 -0.082361 -0.078803   \n",
       "2   1.0 -1.358354 -1.340163  1.773209  0.379780 -0.503198  1.800499  0.791461   \n",
       "3   1.0 -0.966272 -0.185226  1.792993 -0.863291 -0.010309  1.247203  0.237609   \n",
       "4   2.0 -1.158233  0.877737  1.548718  0.403034 -0.407193  0.095921  0.592941   \n",
       "\n",
       "         V8        V9       V10       V11       V12       V13       V14  \\\n",
       "0  0.098698  0.363787  0.090794 -0.551600 -0.617801 -0.991390 -0.311169   \n",
       "1  0.085102 -0.255425 -0.166974  1.612727  1.065235  0.489095 -0.143772   \n",
       "2  0.247676 -1.514654  0.207643  0.624501  0.066084  0.717293 -0.165946   \n",
       "3  0.377436 -1.387024 -0.054952 -0.226487  0.178228  0.507757 -0.287924   \n",
       "4 -0.270533  0.817739  0.753074 -0.822843  0.538196  1.345852 -1.119670   \n",
       "\n",
       "        V15       V16       V17       V18       V19       V20       V21  \\\n",
       "0  1.468177 -0.470401  0.207971  0.025791  0.403993  0.251412 -0.018307   \n",
       "1  0.635558  0.463917 -0.114805 -0.183361 -0.145783 -0.069083 -0.225775   \n",
       "2  2.345865 -2.890083  1.109969 -0.121359 -2.261857  0.524980  0.247998   \n",
       "3 -0.631418 -1.059647 -0.684093  1.965775 -1.232622 -0.208038 -0.108300   \n",
       "4  0.175121 -0.451449 -0.237033 -0.038195  0.803487  0.408542 -0.009431   \n",
       "\n",
       "        V22       V23       V24       V25       V26       V27       V28  \\\n",
       "0  0.277838 -0.110474  0.066928  0.128539 -0.189115  0.133558 -0.021053   \n",
       "1 -0.638672  0.101288 -0.339846  0.167170  0.125895 -0.008983  0.014724   \n",
       "2  0.771679  0.909412 -0.689281 -0.327642 -0.139097 -0.055353 -0.059752   \n",
       "3  0.005274 -0.190321 -1.175575  0.647376 -0.221929  0.062723  0.061458   \n",
       "4  0.798278 -0.137458  0.141267 -0.206010  0.502292  0.219422  0.215153   \n",
       "\n",
       "   Amount  Class  \n",
       "0  149.62      0  \n",
       "1    2.69      0  \n",
       "2  378.66      0  \n",
       "3  123.50      0  \n",
       "4   69.99      0  "
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.options.display.max_columns = 100\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Создайте датафрейм X из датафрейма df, исключив столбец Class.\n",
    "Создайте объект Series под названием y из столбца Class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = df[\"Class\"]\n",
    "X = df.drop(\"Class\", axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Разобьем X и y на тренировочный и тестовый наборы данных"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=100, stratify=y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Просмотрите информацию о их форме"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [],
   "source": [
    "parameters = [{'n_estimators': [10, 15],\n",
    "'max_features': np.arange(3, 5),\n",
    "'max_depth': np.arange(4, 7)}]\n",
    "\n",
    "clf = GridSearchCV(\n",
    "    estimator=RandomForestClassifier(random_state=100),\n",
    "    param_grid=parameters,\n",
    "    scoring='accuracy',\n",
    "    cv=3\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Обучим модель на тренировочном наборе данных"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=3, estimator=RandomForestClassifier(random_state=100),\n",
       "             param_grid=[{'max_depth': array([4, 5, 6]),\n",
       "                          'max_features': array([3, 4]),\n",
       "                          'n_estimators': [10, 15]}],\n",
       "             scoring='accuracy')"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Просмотрим параметры лучшей модели"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'max_depth': 6, 'max_features': 4, 'n_estimators': 10}"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf.best_params_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Предскажим вероятности классов с помощью полученнной модели "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.0009625 , 0.00031616, 0.00026808, ..., 0.00026808, 0.0004142 ,\n",
       "       0.20069477])"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_pred_proba=clf.predict_proba(X_test)[:,1]\n",
    "y_pred_proba"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Вычислите AUC на тестовых данных и сравните с результатом,полученным на тренировочных данных"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9364198655540469"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.metrics import roc_auc_score\n",
    "auc=roc_auc_score(y_test, y_pred_proba)\n",
    "auc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Дополнительное задание**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_wine\n",
    "wine = load_wine()\n",
    "data = wine[\"data\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Создадим список data_keys, содержащий ее ключи."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['alcohol',\n",
       " 'malic_acid',\n",
       " 'ash',\n",
       " 'alcalinity_of_ash',\n",
       " 'magnesium',\n",
       " 'total_phenols',\n",
       " 'flavanoids',\n",
       " 'nonflavanoid_phenols',\n",
       " 'proanthocyanins',\n",
       " 'color_intensity',\n",
       " 'hue',\n",
       " 'od280/od315_of_diluted_wines',\n",
       " 'proline']"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_keys=wine[\"feature_names\"]\n",
    "data_keys"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Просмотрим данные, описание и названия признаков в датасете. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".. _wine_dataset:\n",
      "\n",
      "Wine recognition dataset\n",
      "------------------------\n",
      "\n",
      "**Data Set Characteristics:**\n",
      "\n",
      "    :Number of Instances: 178 (50 in each of three classes)\n",
      "    :Number of Attributes: 13 numeric, predictive attributes and the class\n",
      "    :Attribute Information:\n",
      " \t\t- Alcohol\n",
      " \t\t- Malic acid\n",
      " \t\t- Ash\n",
      "\t\t- Alcalinity of ash  \n",
      " \t\t- Magnesium\n",
      "\t\t- Total phenols\n",
      " \t\t- Flavanoids\n",
      " \t\t- Nonflavanoid phenols\n",
      " \t\t- Proanthocyanins\n",
      "\t\t- Color intensity\n",
      " \t\t- Hue\n",
      " \t\t- OD280/OD315 of diluted wines\n",
      " \t\t- Proline\n",
      "\n",
      "    - class:\n",
      "            - class_0\n",
      "            - class_1\n",
      "            - class_2\n",
      "\t\t\n",
      "    :Summary Statistics:\n",
      "    \n",
      "    ============================= ==== ===== ======= =====\n",
      "                                   Min   Max   Mean     SD\n",
      "    ============================= ==== ===== ======= =====\n",
      "    Alcohol:                      11.0  14.8    13.0   0.8\n",
      "    Malic Acid:                   0.74  5.80    2.34  1.12\n",
      "    Ash:                          1.36  3.23    2.36  0.27\n",
      "    Alcalinity of Ash:            10.6  30.0    19.5   3.3\n",
      "    Magnesium:                    70.0 162.0    99.7  14.3\n",
      "    Total Phenols:                0.98  3.88    2.29  0.63\n",
      "    Flavanoids:                   0.34  5.08    2.03  1.00\n",
      "    Nonflavanoid Phenols:         0.13  0.66    0.36  0.12\n",
      "    Proanthocyanins:              0.41  3.58    1.59  0.57\n",
      "    Colour Intensity:              1.3  13.0     5.1   2.3\n",
      "    Hue:                          0.48  1.71    0.96  0.23\n",
      "    OD280/OD315 of diluted wines: 1.27  4.00    2.61  0.71\n",
      "    Proline:                       278  1680     746   315\n",
      "    ============================= ==== ===== ======= =====\n",
      "\n",
      "    :Missing Attribute Values: None\n",
      "    :Class Distribution: class_0 (59), class_1 (71), class_2 (48)\n",
      "    :Creator: R.A. Fisher\n",
      "    :Donor: Michael Marshall (MARSHALL%PLU@io.arc.nasa.gov)\n",
      "    :Date: July, 1988\n",
      "\n",
      "This is a copy of UCI ML Wine recognition datasets.\n",
      "https://archive.ics.uci.edu/ml/machine-learning-databases/wine/wine.data\n",
      "\n",
      "The data is the results of a chemical analysis of wines grown in the same\n",
      "region in Italy by three different cultivators. There are thirteen different\n",
      "measurements taken for different constituents found in the three types of\n",
      "wine.\n",
      "\n",
      "Original Owners: \n",
      "\n",
      "Forina, M. et al, PARVUS - \n",
      "An Extendible Package for Data Exploration, Classification and Correlation. \n",
      "Institute of Pharmaceutical and Food Analysis and Technologies,\n",
      "Via Brigata Salerno, 16147 Genoa, Italy.\n",
      "\n",
      "Citation:\n",
      "\n",
      "Lichman, M. (2013). UCI Machine Learning Repository\n",
      "[https://archive.ics.uci.edu/ml]. Irvine, CA: University of California,\n",
      "School of Information and Computer Science. \n",
      "\n",
      ".. topic:: References\n",
      "\n",
      "  (1) S. Aeberhard, D. Coomans and O. de Vel, \n",
      "  Comparison of Classifiers in High Dimensional Settings, \n",
      "  Tech. Rep. no. 92-02, (1992), Dept. of Computer Science and Dept. of  \n",
      "  Mathematics and Statistics, James Cook University of North Queensland. \n",
      "  (Also submitted to Technometrics). \n",
      "\n",
      "  The data was used with many others for comparing various \n",
      "  classifiers. The classes are separable, though only RDA \n",
      "  has achieved 100% correct classification. \n",
      "  (RDA : 100%, QDA 99.4%, LDA 98.9%, 1NN 96.1% (z-transformed data)) \n",
      "  (All results using the leave-one-out technique) \n",
      "\n",
      "  (2) S. Aeberhard, D. Coomans and O. de Vel, \n",
      "  \"THE CLASSIFICATION PERFORMANCE OF RDA\" \n",
      "  Tech. Rep. no. 92-01, (1992), Dept. of Computer Science and Dept. of \n",
      "  Mathematics and Statistics, James Cook University of North Queensland. \n",
      "  (Also submitted to Journal of Chemometrics).\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(wine[\"DESCR\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Кол-во классов в целевая переменной датасета\n",
    "Вывод названия классов."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(3,)\n",
      "['class_0' 'class_1' 'class_2']\n"
     ]
    }
   ],
   "source": [
    "print(np.unique(wine[\"target\"]).shape)\n",
    "print(wine[\"target_names\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>CRIM</th>\n",
       "      <th>ZN</th>\n",
       "      <th>INDUS</th>\n",
       "      <th>CHAS</th>\n",
       "      <th>NOX</th>\n",
       "      <th>RM</th>\n",
       "      <th>AGE</th>\n",
       "      <th>DIS</th>\n",
       "      <th>RAD</th>\n",
       "      <th>TAX</th>\n",
       "      <th>PTRATIO</th>\n",
       "      <th>B</th>\n",
       "      <th>LSTAT</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>14.23</td>\n",
       "      <td>1.71</td>\n",
       "      <td>2.43</td>\n",
       "      <td>15.6</td>\n",
       "      <td>127.0</td>\n",
       "      <td>2.80</td>\n",
       "      <td>3.06</td>\n",
       "      <td>0.28</td>\n",
       "      <td>2.29</td>\n",
       "      <td>5.64</td>\n",
       "      <td>1.04</td>\n",
       "      <td>3.92</td>\n",
       "      <td>1065.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>13.20</td>\n",
       "      <td>1.78</td>\n",
       "      <td>2.14</td>\n",
       "      <td>11.2</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2.65</td>\n",
       "      <td>2.76</td>\n",
       "      <td>0.26</td>\n",
       "      <td>1.28</td>\n",
       "      <td>4.38</td>\n",
       "      <td>1.05</td>\n",
       "      <td>3.40</td>\n",
       "      <td>1050.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>13.16</td>\n",
       "      <td>2.36</td>\n",
       "      <td>2.67</td>\n",
       "      <td>18.6</td>\n",
       "      <td>101.0</td>\n",
       "      <td>2.80</td>\n",
       "      <td>3.24</td>\n",
       "      <td>0.30</td>\n",
       "      <td>2.81</td>\n",
       "      <td>5.68</td>\n",
       "      <td>1.03</td>\n",
       "      <td>3.17</td>\n",
       "      <td>1185.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>14.37</td>\n",
       "      <td>1.95</td>\n",
       "      <td>2.50</td>\n",
       "      <td>16.8</td>\n",
       "      <td>113.0</td>\n",
       "      <td>3.85</td>\n",
       "      <td>3.49</td>\n",
       "      <td>0.24</td>\n",
       "      <td>2.18</td>\n",
       "      <td>7.80</td>\n",
       "      <td>0.86</td>\n",
       "      <td>3.45</td>\n",
       "      <td>1480.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>13.24</td>\n",
       "      <td>2.59</td>\n",
       "      <td>2.87</td>\n",
       "      <td>21.0</td>\n",
       "      <td>118.0</td>\n",
       "      <td>2.80</td>\n",
       "      <td>2.69</td>\n",
       "      <td>0.39</td>\n",
       "      <td>1.82</td>\n",
       "      <td>4.32</td>\n",
       "      <td>1.04</td>\n",
       "      <td>2.93</td>\n",
       "      <td>735.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    CRIM    ZN  INDUS  CHAS    NOX    RM   AGE   DIS   RAD   TAX  PTRATIO  \\\n",
       "0  14.23  1.71   2.43  15.6  127.0  2.80  3.06  0.28  2.29  5.64     1.04   \n",
       "1  13.20  1.78   2.14  11.2  100.0  2.65  2.76  0.26  1.28  4.38     1.05   \n",
       "2  13.16  2.36   2.67  18.6  101.0  2.80  3.24  0.30  2.81  5.68     1.03   \n",
       "3  14.37  1.95   2.50  16.8  113.0  3.85  3.49  0.24  2.18  7.80     0.86   \n",
       "4  13.24  2.59   2.87  21.0  118.0  2.80  2.69  0.39  1.82  4.32     1.04   \n",
       "\n",
       "      B   LSTAT  \n",
       "0  3.92  1065.0  \n",
       "1  3.40  1050.0  \n",
       "2  3.17  1185.0  \n",
       "3  3.45  1480.0  \n",
       "4  2.93   735.0  "
      ]
     },
     "execution_count": 134,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = pd.DataFrame(wine.data, columns=feature_names)\n",
    "X.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Выясним размер датафрейма X и установите, имеются ли в нем пропущенные значения"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(178, 13)"
      ]
     },
     "execution_count": 135,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "CRIM       0\n",
       "ZN         0\n",
       "INDUS      0\n",
       "CHAS       0\n",
       "NOX        0\n",
       "RM         0\n",
       "AGE        0\n",
       "DIS        0\n",
       "RAD        0\n",
       "TAX        0\n",
       "PTRATIO    0\n",
       "B          0\n",
       "LSTAT      0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 136,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.isnull().astype(\"int\").sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Проупщенных значенией нет"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Добавим в датафрейм поле с классами вин в виде чисел, имеющих тип данных numpy.int64. Название поля - 'target'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [],
   "source": [
    "X[\"target\"]=wine[\"target\"].astype(np.int64)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Построим матрицу корреляций для всех полей X. Дайте полученному датафрейму название X_corr."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_corr=X.corr()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Создадим список high_corr из признаков, корреляция которых с полем target по абсолютному значению превышает 0.5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['CHAS', 'RM', 'AGE', 'PTRATIO', 'B', 'LSTAT']"
      ]
     },
     "execution_count": 156,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_corr[\"target\"]\n",
    "high_corr=X_corr[\"target\"]\n",
    "high_corr=high_corr[np.abs(high_corr)>0.5].drop(\"target\", axis=0)\n",
    "high_corr=list(high_corr.index)\n",
    "high_corr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Удалим из датафрейма X поле с целевой переменной. Для всех признаков, названия которых содержатся в списке high_corr, вычислим квадрат их значений и добавьте в датафрейм X соответствующие поля с суффиксом '_2', добавленного к первоначальному названию признака."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>CRIM</th>\n",
       "      <th>ZN</th>\n",
       "      <th>INDUS</th>\n",
       "      <th>CHAS</th>\n",
       "      <th>NOX</th>\n",
       "      <th>RM</th>\n",
       "      <th>AGE</th>\n",
       "      <th>DIS</th>\n",
       "      <th>RAD</th>\n",
       "      <th>TAX</th>\n",
       "      <th>PTRATIO</th>\n",
       "      <th>B</th>\n",
       "      <th>LSTAT</th>\n",
       "      <th>CHAS_2</th>\n",
       "      <th>RM_2</th>\n",
       "      <th>AGE_2</th>\n",
       "      <th>PTRATIO_2</th>\n",
       "      <th>B_2</th>\n",
       "      <th>LSTAT_2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>14.23</td>\n",
       "      <td>1.71</td>\n",
       "      <td>2.43</td>\n",
       "      <td>15.6</td>\n",
       "      <td>127.0</td>\n",
       "      <td>2.80</td>\n",
       "      <td>3.06</td>\n",
       "      <td>0.28</td>\n",
       "      <td>2.29</td>\n",
       "      <td>5.64</td>\n",
       "      <td>1.04</td>\n",
       "      <td>3.92</td>\n",
       "      <td>1065.0</td>\n",
       "      <td>243.36</td>\n",
       "      <td>7.8400</td>\n",
       "      <td>9.3636</td>\n",
       "      <td>1.0816</td>\n",
       "      <td>15.3664</td>\n",
       "      <td>1134225.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>13.20</td>\n",
       "      <td>1.78</td>\n",
       "      <td>2.14</td>\n",
       "      <td>11.2</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2.65</td>\n",
       "      <td>2.76</td>\n",
       "      <td>0.26</td>\n",
       "      <td>1.28</td>\n",
       "      <td>4.38</td>\n",
       "      <td>1.05</td>\n",
       "      <td>3.40</td>\n",
       "      <td>1050.0</td>\n",
       "      <td>125.44</td>\n",
       "      <td>7.0225</td>\n",
       "      <td>7.6176</td>\n",
       "      <td>1.1025</td>\n",
       "      <td>11.5600</td>\n",
       "      <td>1102500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>13.16</td>\n",
       "      <td>2.36</td>\n",
       "      <td>2.67</td>\n",
       "      <td>18.6</td>\n",
       "      <td>101.0</td>\n",
       "      <td>2.80</td>\n",
       "      <td>3.24</td>\n",
       "      <td>0.30</td>\n",
       "      <td>2.81</td>\n",
       "      <td>5.68</td>\n",
       "      <td>1.03</td>\n",
       "      <td>3.17</td>\n",
       "      <td>1185.0</td>\n",
       "      <td>345.96</td>\n",
       "      <td>7.8400</td>\n",
       "      <td>10.4976</td>\n",
       "      <td>1.0609</td>\n",
       "      <td>10.0489</td>\n",
       "      <td>1404225.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>14.37</td>\n",
       "      <td>1.95</td>\n",
       "      <td>2.50</td>\n",
       "      <td>16.8</td>\n",
       "      <td>113.0</td>\n",
       "      <td>3.85</td>\n",
       "      <td>3.49</td>\n",
       "      <td>0.24</td>\n",
       "      <td>2.18</td>\n",
       "      <td>7.80</td>\n",
       "      <td>0.86</td>\n",
       "      <td>3.45</td>\n",
       "      <td>1480.0</td>\n",
       "      <td>282.24</td>\n",
       "      <td>14.8225</td>\n",
       "      <td>12.1801</td>\n",
       "      <td>0.7396</td>\n",
       "      <td>11.9025</td>\n",
       "      <td>2190400.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>13.24</td>\n",
       "      <td>2.59</td>\n",
       "      <td>2.87</td>\n",
       "      <td>21.0</td>\n",
       "      <td>118.0</td>\n",
       "      <td>2.80</td>\n",
       "      <td>2.69</td>\n",
       "      <td>0.39</td>\n",
       "      <td>1.82</td>\n",
       "      <td>4.32</td>\n",
       "      <td>1.04</td>\n",
       "      <td>2.93</td>\n",
       "      <td>735.0</td>\n",
       "      <td>441.00</td>\n",
       "      <td>7.8400</td>\n",
       "      <td>7.2361</td>\n",
       "      <td>1.0816</td>\n",
       "      <td>8.5849</td>\n",
       "      <td>540225.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    CRIM    ZN  INDUS  CHAS    NOX    RM   AGE   DIS   RAD   TAX  PTRATIO  \\\n",
       "0  14.23  1.71   2.43  15.6  127.0  2.80  3.06  0.28  2.29  5.64     1.04   \n",
       "1  13.20  1.78   2.14  11.2  100.0  2.65  2.76  0.26  1.28  4.38     1.05   \n",
       "2  13.16  2.36   2.67  18.6  101.0  2.80  3.24  0.30  2.81  5.68     1.03   \n",
       "3  14.37  1.95   2.50  16.8  113.0  3.85  3.49  0.24  2.18  7.80     0.86   \n",
       "4  13.24  2.59   2.87  21.0  118.0  2.80  2.69  0.39  1.82  4.32     1.04   \n",
       "\n",
       "      B   LSTAT  CHAS_2     RM_2    AGE_2  PTRATIO_2      B_2    LSTAT_2  \n",
       "0  3.92  1065.0  243.36   7.8400   9.3636     1.0816  15.3664  1134225.0  \n",
       "1  3.40  1050.0  125.44   7.0225   7.6176     1.1025  11.5600  1102500.0  \n",
       "2  3.17  1185.0  345.96   7.8400  10.4976     1.0609  10.0489  1404225.0  \n",
       "3  3.45  1480.0  282.24  14.8225  12.1801     0.7396  11.9025  2190400.0  \n",
       "4  2.93   735.0  441.00   7.8400   7.2361     1.0816   8.5849   540225.0  "
      ]
     },
     "execution_count": 157,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X=X.drop(\"target\", axis=1)\n",
    "for i in high_corr:\n",
    " X[i+\"_2\"]=X[i]**2\n",
    "X.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}