Ankur_ML_KNN.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The following data is for predicting whether the cancer is benign or maligant.\n",
    "### To predict the class of cancer we will use a classification alogrithm: KNN "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Features are computed from a digitized image of a fine needle aspirate (FNA) of a breast mass. They describe characteristics of the cell nuclei present in the image.\n",
    "n the 3-dimensional space is that described in: [K. P. Bennett and O. L. Mangasarian: \"Robust Linear Programming Discrimination of Two Linearly Inseparable Sets\", Optimization Methods and Software 1, 1992, 23-34].\n",
    "\n",
    "This database is also available through the UW CS ftp server:\n",
    "ftp ftp.cs.wisc.edu\n",
    "cd math-prog/cpo-dataset/machine-learn/WDBC/\n",
    "\n",
    "Also can be found on UCI Machine Learning Repository: https://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+%28Diagnostic%29\n",
    "\n",
    "**This machine learning project is about predicting the type of tumor — Malignant or Benign.**\n",
    "\n",
    "## Attribute Information:\n",
    "\n",
    "1) ID number\n",
    "2) Diagnosis (M = malignant, B = benign)\n",
    "3-32)\n",
    "\n",
    "Ten real-valued features are computed for each cell nucleus:\n",
    "\n",
    "a) radius (mean of distances from center to points on the perimeter)\n",
    "b) texture (standard deviation of gray-scale values)\n",
    "c) perimeter\n",
    "d) area\n",
    "e) smoothness (local variation in radius lengths)\n",
    "f) compactness (perimeter^2 / area - 1.0)\n",
    "g) concavity (severity of concave portions of the contour)\n",
    "h) concave points (number of concave portions of the contour)\n",
    "i) symmetry\n",
    "j) fractal dimension (\"coastline approximation\" - 1)\n",
    "\n",
    "The mean, standard error and \"worst\" or largest (mean of the three\n",
    "largest values) of these features were computed for each image,\n",
    "resulting in 30 features. For instance, field 3 is Mean Radius, field\n",
    "13 is Radius SE, field 23 is Worst Radius.\n",
    "\n",
    "All feature values are recoded with four significant digits.\n",
    "\n",
    "Missing attribute values: none\n",
    "\n",
    "Class distribution: 357 benign, 212 malignant\n",
    "\n",
    "```\n",
    "Breast cancer is a disease in which cells in the breast grow out of control. There are different kinds of breast cancer.The kind of breast cancer depends on which cells in the breast turn into cancer.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![cancer](http://triplesteptowardthecure.org/images/slide1.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Importing the relevant packages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "sns.set()\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 569 entries, 0 to 568\n",
      "Data columns (total 33 columns):\n",
      "id                         569 non-null int64\n",
      "diagnosis                  569 non-null object\n",
      "radius_mean                569 non-null float64\n",
      "texture_mean               569 non-null float64\n",
      "perimeter_mean             569 non-null float64\n",
      "area_mean                  569 non-null float64\n",
      "smoothness_mean            569 non-null float64\n",
      "compactness_mean           569 non-null float64\n",
      "concavity_mean             569 non-null float64\n",
      "concave points_mean        569 non-null float64\n",
      "symmetry_mean              569 non-null float64\n",
      "fractal_dimension_mean     569 non-null float64\n",
      "radius_se                  569 non-null float64\n",
      "texture_se                 569 non-null float64\n",
      "perimeter_se               569 non-null float64\n",
      "area_se                    569 non-null float64\n",
      "smoothness_se              569 non-null float64\n",
      "compactness_se             569 non-null float64\n",
      "concavity_se               569 non-null float64\n",
      "concave points_se          569 non-null float64\n",
      "symmetry_se                569 non-null float64\n",
      "fractal_dimension_se       569 non-null float64\n",
      "radius_worst               569 non-null float64\n",
      "texture_worst              569 non-null float64\n",
      "perimeter_worst            569 non-null float64\n",
      "area_worst                 569 non-null float64\n",
      "smoothness_worst           569 non-null float64\n",
      "compactness_worst          569 non-null float64\n",
      "concavity_worst            569 non-null float64\n",
      "concave points_worst       569 non-null float64\n",
      "symmetry_worst             569 non-null float64\n",
      "fractal_dimension_worst    569 non-null float64\n",
      "Unnamed: 32                0 non-null float64\n",
      "dtypes: float64(31), int64(1), object(1)\n",
      "memory usage: 144.5+ KB\n"
     ]
    }
   ],
   "source": [
    "data =  pd.read_csv(r'Kaggle_BreastCancer.csv')\n",
    "data.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Descriptive analyis of the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>radius_mean</th>\n",
       "      <th>texture_mean</th>\n",
       "      <th>perimeter_mean</th>\n",
       "      <th>area_mean</th>\n",
       "      <th>smoothness_mean</th>\n",
       "      <th>compactness_mean</th>\n",
       "      <th>concavity_mean</th>\n",
       "      <th>concave points_mean</th>\n",
       "      <th>symmetry_mean</th>\n",
       "      <th>...</th>\n",
       "      <th>texture_worst</th>\n",
       "      <th>perimeter_worst</th>\n",
       "      <th>area_worst</th>\n",
       "      <th>smoothness_worst</th>\n",
       "      <th>compactness_worst</th>\n",
       "      <th>concavity_worst</th>\n",
       "      <th>concave points_worst</th>\n",
       "      <th>symmetry_worst</th>\n",
       "      <th>fractal_dimension_worst</th>\n",
       "      <th>Unnamed: 32</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>5.690000e+02</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>3.037183e+07</td>\n",
       "      <td>14.127292</td>\n",
       "      <td>19.289649</td>\n",
       "      <td>91.969033</td>\n",
       "      <td>654.889104</td>\n",
       "      <td>0.096360</td>\n",
       "      <td>0.104341</td>\n",
       "      <td>0.088799</td>\n",
       "      <td>0.048919</td>\n",
       "      <td>0.181162</td>\n",
       "      <td>...</td>\n",
       "      <td>25.677223</td>\n",
       "      <td>107.261213</td>\n",
       "      <td>880.583128</td>\n",
       "      <td>0.132369</td>\n",
       "      <td>0.254265</td>\n",
       "      <td>0.272188</td>\n",
       "      <td>0.114606</td>\n",
       "      <td>0.290076</td>\n",
       "      <td>0.083946</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1.250206e+08</td>\n",
       "      <td>3.524049</td>\n",
       "      <td>4.301036</td>\n",
       "      <td>24.298981</td>\n",
       "      <td>351.914129</td>\n",
       "      <td>0.014064</td>\n",
       "      <td>0.052813</td>\n",
       "      <td>0.079720</td>\n",
       "      <td>0.038803</td>\n",
       "      <td>0.027414</td>\n",
       "      <td>...</td>\n",
       "      <td>6.146258</td>\n",
       "      <td>33.602542</td>\n",
       "      <td>569.356993</td>\n",
       "      <td>0.022832</td>\n",
       "      <td>0.157336</td>\n",
       "      <td>0.208624</td>\n",
       "      <td>0.065732</td>\n",
       "      <td>0.061867</td>\n",
       "      <td>0.018061</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>8.670000e+03</td>\n",
       "      <td>6.981000</td>\n",
       "      <td>9.710000</td>\n",
       "      <td>43.790000</td>\n",
       "      <td>143.500000</td>\n",
       "      <td>0.052630</td>\n",
       "      <td>0.019380</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.106000</td>\n",
       "      <td>...</td>\n",
       "      <td>12.020000</td>\n",
       "      <td>50.410000</td>\n",
       "      <td>185.200000</td>\n",
       "      <td>0.071170</td>\n",
       "      <td>0.027290</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.156500</td>\n",
       "      <td>0.055040</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>8.692180e+05</td>\n",
       "      <td>11.700000</td>\n",
       "      <td>16.170000</td>\n",
       "      <td>75.170000</td>\n",
       "      <td>420.300000</td>\n",
       "      <td>0.086370</td>\n",
       "      <td>0.064920</td>\n",
       "      <td>0.029560</td>\n",
       "      <td>0.020310</td>\n",
       "      <td>0.161900</td>\n",
       "      <td>...</td>\n",
       "      <td>21.080000</td>\n",
       "      <td>84.110000</td>\n",
       "      <td>515.300000</td>\n",
       "      <td>0.116600</td>\n",
       "      <td>0.147200</td>\n",
       "      <td>0.114500</td>\n",
       "      <td>0.064930</td>\n",
       "      <td>0.250400</td>\n",
       "      <td>0.071460</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>9.060240e+05</td>\n",
       "      <td>13.370000</td>\n",
       "      <td>18.840000</td>\n",
       "      <td>86.240000</td>\n",
       "      <td>551.100000</td>\n",
       "      <td>0.095870</td>\n",
       "      <td>0.092630</td>\n",
       "      <td>0.061540</td>\n",
       "      <td>0.033500</td>\n",
       "      <td>0.179200</td>\n",
       "      <td>...</td>\n",
       "      <td>25.410000</td>\n",
       "      <td>97.660000</td>\n",
       "      <td>686.500000</td>\n",
       "      <td>0.131300</td>\n",
       "      <td>0.211900</td>\n",
       "      <td>0.226700</td>\n",
       "      <td>0.099930</td>\n",
       "      <td>0.282200</td>\n",
       "      <td>0.080040</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>8.813129e+06</td>\n",
       "      <td>15.780000</td>\n",
       "      <td>21.800000</td>\n",
       "      <td>104.100000</td>\n",
       "      <td>782.700000</td>\n",
       "      <td>0.105300</td>\n",
       "      <td>0.130400</td>\n",
       "      <td>0.130700</td>\n",
       "      <td>0.074000</td>\n",
       "      <td>0.195700</td>\n",
       "      <td>...</td>\n",
       "      <td>29.720000</td>\n",
       "      <td>125.400000</td>\n",
       "      <td>1084.000000</td>\n",
       "      <td>0.146000</td>\n",
       "      <td>0.339100</td>\n",
       "      <td>0.382900</td>\n",
       "      <td>0.161400</td>\n",
       "      <td>0.317900</td>\n",
       "      <td>0.092080</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>9.113205e+08</td>\n",
       "      <td>28.110000</td>\n",
       "      <td>39.280000</td>\n",
       "      <td>188.500000</td>\n",
       "      <td>2501.000000</td>\n",
       "      <td>0.163400</td>\n",
       "      <td>0.345400</td>\n",
       "      <td>0.426800</td>\n",
       "      <td>0.201200</td>\n",
       "      <td>0.304000</td>\n",
       "      <td>...</td>\n",
       "      <td>49.540000</td>\n",
       "      <td>251.200000</td>\n",
       "      <td>4254.000000</td>\n",
       "      <td>0.222600</td>\n",
       "      <td>1.058000</td>\n",
       "      <td>1.252000</td>\n",
       "      <td>0.291000</td>\n",
       "      <td>0.663800</td>\n",
       "      <td>0.207500</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 32 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 id  radius_mean  texture_mean  perimeter_mean    area_mean  \\\n",
       "count  5.690000e+02   569.000000    569.000000      569.000000   569.000000   \n",
       "mean   3.037183e+07    14.127292     19.289649       91.969033   654.889104   \n",
       "std    1.250206e+08     3.524049      4.301036       24.298981   351.914129   \n",
       "min    8.670000e+03     6.981000      9.710000       43.790000   143.500000   \n",
       "25%    8.692180e+05    11.700000     16.170000       75.170000   420.300000   \n",
       "50%    9.060240e+05    13.370000     18.840000       86.240000   551.100000   \n",
       "75%    8.813129e+06    15.780000     21.800000      104.100000   782.700000   \n",
       "max    9.113205e+08    28.110000     39.280000      188.500000  2501.000000   \n",
       "\n",
       "       smoothness_mean  compactness_mean  concavity_mean  concave points_mean  \\\n",
       "count       569.000000        569.000000      569.000000           569.000000   \n",
       "mean          0.096360          0.104341        0.088799             0.048919   \n",
       "std           0.014064          0.052813        0.079720             0.038803   \n",
       "min           0.052630          0.019380        0.000000             0.000000   \n",
       "25%           0.086370          0.064920        0.029560             0.020310   \n",
       "50%           0.095870          0.092630        0.061540             0.033500   \n",
       "75%           0.105300          0.130400        0.130700             0.074000   \n",
       "max           0.163400          0.345400        0.426800             0.201200   \n",
       "\n",
       "       symmetry_mean  ...  texture_worst  perimeter_worst   area_worst  \\\n",
       "count     569.000000  ...     569.000000       569.000000   569.000000   \n",
       "mean        0.181162  ...      25.677223       107.261213   880.583128   \n",
       "std         0.027414  ...       6.146258        33.602542   569.356993   \n",
       "min         0.106000  ...      12.020000        50.410000   185.200000   \n",
       "25%         0.161900  ...      21.080000        84.110000   515.300000   \n",
       "50%         0.179200  ...      25.410000        97.660000   686.500000   \n",
       "75%         0.195700  ...      29.720000       125.400000  1084.000000   \n",
       "max         0.304000  ...      49.540000       251.200000  4254.000000   \n",
       "\n",
       "       smoothness_worst  compactness_worst  concavity_worst  \\\n",
       "count        569.000000         569.000000       569.000000   \n",
       "mean           0.132369           0.254265         0.272188   \n",
       "std            0.022832           0.157336         0.208624   \n",
       "min            0.071170           0.027290         0.000000   \n",
       "25%            0.116600           0.147200         0.114500   \n",
       "50%            0.131300           0.211900         0.226700   \n",
       "75%            0.146000           0.339100         0.382900   \n",
       "max            0.222600           1.058000         1.252000   \n",
       "\n",
       "       concave points_worst  symmetry_worst  fractal_dimension_worst  \\\n",
       "count            569.000000      569.000000               569.000000   \n",
       "mean               0.114606        0.290076                 0.083946   \n",
       "std                0.065732        0.061867                 0.018061   \n",
       "min                0.000000        0.156500                 0.055040   \n",
       "25%                0.064930        0.250400                 0.071460   \n",
       "50%                0.099930        0.282200                 0.080040   \n",
       "75%                0.161400        0.317900                 0.092080   \n",
       "max                0.291000        0.663800                 0.207500   \n",
       "\n",
       "       Unnamed: 32  \n",
       "count          0.0  \n",
       "mean           NaN  \n",
       "std            NaN  \n",
       "min            NaN  \n",
       "25%            NaN  \n",
       "50%            NaN  \n",
       "75%            NaN  \n",
       "max            NaN  \n",
       "\n",
       "[8 rows x 32 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(569, 33)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "id                           0\n",
       "diagnosis                    0\n",
       "radius_mean                  0\n",
       "texture_mean                 0\n",
       "perimeter_mean               0\n",
       "area_mean                    0\n",
       "smoothness_mean              0\n",
       "compactness_mean             0\n",
       "concavity_mean               0\n",
       "concave points_mean          0\n",
       "symmetry_mean                0\n",
       "fractal_dimension_mean       0\n",
       "radius_se                    0\n",
       "texture_se                   0\n",
       "perimeter_se                 0\n",
       "area_se                      0\n",
       "smoothness_se                0\n",
       "compactness_se               0\n",
       "concavity_se                 0\n",
       "concave points_se            0\n",
       "symmetry_se                  0\n",
       "fractal_dimension_se         0\n",
       "radius_worst                 0\n",
       "texture_worst                0\n",
       "perimeter_worst              0\n",
       "area_worst                   0\n",
       "smoothness_worst             0\n",
       "compactness_worst            0\n",
       "concavity_worst              0\n",
       "concave points_worst         0\n",
       "symmetry_worst               0\n",
       "fractal_dimension_worst      0\n",
       "Unnamed: 32                569\n",
       "dtype: int64"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>radius_mean</th>\n",
       "      <th>texture_mean</th>\n",
       "      <th>perimeter_mean</th>\n",
       "      <th>area_mean</th>\n",
       "      <th>smoothness_mean</th>\n",
       "      <th>compactness_mean</th>\n",
       "      <th>concavity_mean</th>\n",
       "      <th>concave points_mean</th>\n",
       "      <th>symmetry_mean</th>\n",
       "      <th>...</th>\n",
       "      <th>texture_worst</th>\n",
       "      <th>perimeter_worst</th>\n",
       "      <th>area_worst</th>\n",
       "      <th>smoothness_worst</th>\n",
       "      <th>compactness_worst</th>\n",
       "      <th>concavity_worst</th>\n",
       "      <th>concave points_worst</th>\n",
       "      <th>symmetry_worst</th>\n",
       "      <th>fractal_dimension_worst</th>\n",
       "      <th>Unnamed: 32</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>5.690000e+02</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>3.037183e+07</td>\n",
       "      <td>14.127292</td>\n",
       "      <td>19.289649</td>\n",
       "      <td>91.969033</td>\n",
       "      <td>654.889104</td>\n",
       "      <td>0.096360</td>\n",
       "      <td>0.104341</td>\n",
       "      <td>0.088799</td>\n",
       "      <td>0.048919</td>\n",
       "      <td>0.181162</td>\n",
       "      <td>...</td>\n",
       "      <td>25.677223</td>\n",
       "      <td>107.261213</td>\n",
       "      <td>880.583128</td>\n",
       "      <td>0.132369</td>\n",
       "      <td>0.254265</td>\n",
       "      <td>0.272188</td>\n",
       "      <td>0.114606</td>\n",
       "      <td>0.290076</td>\n",
       "      <td>0.083946</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1.250206e+08</td>\n",
       "      <td>3.524049</td>\n",
       "      <td>4.301036</td>\n",
       "      <td>24.298981</td>\n",
       "      <td>351.914129</td>\n",
       "      <td>0.014064</td>\n",
       "      <td>0.052813</td>\n",
       "      <td>0.079720</td>\n",
       "      <td>0.038803</td>\n",
       "      <td>0.027414</td>\n",
       "      <td>...</td>\n",
       "      <td>6.146258</td>\n",
       "      <td>33.602542</td>\n",
       "      <td>569.356993</td>\n",
       "      <td>0.022832</td>\n",
       "      <td>0.157336</td>\n",
       "      <td>0.208624</td>\n",
       "      <td>0.065732</td>\n",
       "      <td>0.061867</td>\n",
       "      <td>0.018061</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>8.670000e+03</td>\n",
       "      <td>6.981000</td>\n",
       "      <td>9.710000</td>\n",
       "      <td>43.790000</td>\n",
       "      <td>143.500000</td>\n",
       "      <td>0.052630</td>\n",
       "      <td>0.019380</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.106000</td>\n",
       "      <td>...</td>\n",
       "      <td>12.020000</td>\n",
       "      <td>50.410000</td>\n",
       "      <td>185.200000</td>\n",
       "      <td>0.071170</td>\n",
       "      <td>0.027290</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.156500</td>\n",
       "      <td>0.055040</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>8.692180e+05</td>\n",
       "      <td>11.700000</td>\n",
       "      <td>16.170000</td>\n",
       "      <td>75.170000</td>\n",
       "      <td>420.300000</td>\n",
       "      <td>0.086370</td>\n",
       "      <td>0.064920</td>\n",
       "      <td>0.029560</td>\n",
       "      <td>0.020310</td>\n",
       "      <td>0.161900</td>\n",
       "      <td>...</td>\n",
       "      <td>21.080000</td>\n",
       "      <td>84.110000</td>\n",
       "      <td>515.300000</td>\n",
       "      <td>0.116600</td>\n",
       "      <td>0.147200</td>\n",
       "      <td>0.114500</td>\n",
       "      <td>0.064930</td>\n",
       "      <td>0.250400</td>\n",
       "      <td>0.071460</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>9.060240e+05</td>\n",
       "      <td>13.370000</td>\n",
       "      <td>18.840000</td>\n",
       "      <td>86.240000</td>\n",
       "      <td>551.100000</td>\n",
       "      <td>0.095870</td>\n",
       "      <td>0.092630</td>\n",
       "      <td>0.061540</td>\n",
       "      <td>0.033500</td>\n",
       "      <td>0.179200</td>\n",
       "      <td>...</td>\n",
       "      <td>25.410000</td>\n",
       "      <td>97.660000</td>\n",
       "      <td>686.500000</td>\n",
       "      <td>0.131300</td>\n",
       "      <td>0.211900</td>\n",
       "      <td>0.226700</td>\n",
       "      <td>0.099930</td>\n",
       "      <td>0.282200</td>\n",
       "      <td>0.080040</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>8.813129e+06</td>\n",
       "      <td>15.780000</td>\n",
       "      <td>21.800000</td>\n",
       "      <td>104.100000</td>\n",
       "      <td>782.700000</td>\n",
       "      <td>0.105300</td>\n",
       "      <td>0.130400</td>\n",
       "      <td>0.130700</td>\n",
       "      <td>0.074000</td>\n",
       "      <td>0.195700</td>\n",
       "      <td>...</td>\n",
       "      <td>29.720000</td>\n",
       "      <td>125.400000</td>\n",
       "      <td>1084.000000</td>\n",
       "      <td>0.146000</td>\n",
       "      <td>0.339100</td>\n",
       "      <td>0.382900</td>\n",
       "      <td>0.161400</td>\n",
       "      <td>0.317900</td>\n",
       "      <td>0.092080</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>9.113205e+08</td>\n",
       "      <td>28.110000</td>\n",
       "      <td>39.280000</td>\n",
       "      <td>188.500000</td>\n",
       "      <td>2501.000000</td>\n",
       "      <td>0.163400</td>\n",
       "      <td>0.345400</td>\n",
       "      <td>0.426800</td>\n",
       "      <td>0.201200</td>\n",
       "      <td>0.304000</td>\n",
       "      <td>...</td>\n",
       "      <td>49.540000</td>\n",
       "      <td>251.200000</td>\n",
       "      <td>4254.000000</td>\n",
       "      <td>0.222600</td>\n",
       "      <td>1.058000</td>\n",
       "      <td>1.252000</td>\n",
       "      <td>0.291000</td>\n",
       "      <td>0.663800</td>\n",
       "      <td>0.207500</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 32 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 id  radius_mean  texture_mean  perimeter_mean    area_mean  \\\n",
       "count  5.690000e+02   569.000000    569.000000      569.000000   569.000000   \n",
       "mean   3.037183e+07    14.127292     19.289649       91.969033   654.889104   \n",
       "std    1.250206e+08     3.524049      4.301036       24.298981   351.914129   \n",
       "min    8.670000e+03     6.981000      9.710000       43.790000   143.500000   \n",
       "25%    8.692180e+05    11.700000     16.170000       75.170000   420.300000   \n",
       "50%    9.060240e+05    13.370000     18.840000       86.240000   551.100000   \n",
       "75%    8.813129e+06    15.780000     21.800000      104.100000   782.700000   \n",
       "max    9.113205e+08    28.110000     39.280000      188.500000  2501.000000   \n",
       "\n",
       "       smoothness_mean  compactness_mean  concavity_mean  concave points_mean  \\\n",
       "count       569.000000        569.000000      569.000000           569.000000   \n",
       "mean          0.096360          0.104341        0.088799             0.048919   \n",
       "std           0.014064          0.052813        0.079720             0.038803   \n",
       "min           0.052630          0.019380        0.000000             0.000000   \n",
       "25%           0.086370          0.064920        0.029560             0.020310   \n",
       "50%           0.095870          0.092630        0.061540             0.033500   \n",
       "75%           0.105300          0.130400        0.130700             0.074000   \n",
       "max           0.163400          0.345400        0.426800             0.201200   \n",
       "\n",
       "       symmetry_mean  ...  texture_worst  perimeter_worst   area_worst  \\\n",
       "count     569.000000  ...     569.000000       569.000000   569.000000   \n",
       "mean        0.181162  ...      25.677223       107.261213   880.583128   \n",
       "std         0.027414  ...       6.146258        33.602542   569.356993   \n",
       "min         0.106000  ...      12.020000        50.410000   185.200000   \n",
       "25%         0.161900  ...      21.080000        84.110000   515.300000   \n",
       "50%         0.179200  ...      25.410000        97.660000   686.500000   \n",
       "75%         0.195700  ...      29.720000       125.400000  1084.000000   \n",
       "max         0.304000  ...      49.540000       251.200000  4254.000000   \n",
       "\n",
       "       smoothness_worst  compactness_worst  concavity_worst  \\\n",
       "count        569.000000         569.000000       569.000000   \n",
       "mean           0.132369           0.254265         0.272188   \n",
       "std            0.022832           0.157336         0.208624   \n",
       "min            0.071170           0.027290         0.000000   \n",
       "25%            0.116600           0.147200         0.114500   \n",
       "50%            0.131300           0.211900         0.226700   \n",
       "75%            0.146000           0.339100         0.382900   \n",
       "max            0.222600           1.058000         1.252000   \n",
       "\n",
       "       concave points_worst  symmetry_worst  fractal_dimension_worst  \\\n",
       "count            569.000000      569.000000               569.000000   \n",
       "mean               0.114606        0.290076                 0.083946   \n",
       "std                0.065732        0.061867                 0.018061   \n",
       "min                0.000000        0.156500                 0.055040   \n",
       "25%                0.064930        0.250400                 0.071460   \n",
       "50%                0.099930        0.282200                 0.080040   \n",
       "75%                0.161400        0.317900                 0.092080   \n",
       "max                0.291000        0.663800                 0.207500   \n",
       "\n",
       "       Unnamed: 32  \n",
       "count          0.0  \n",
       "mean           NaN  \n",
       "std            NaN  \n",
       "min            NaN  \n",
       "25%            NaN  \n",
       "50%            NaN  \n",
       "75%            NaN  \n",
       "max            NaN  \n",
       "\n",
       "[8 rows x 32 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Since our target valiable is a classifier we will convert it into numerical variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "data['diagnosis']=data['diagnosis'].map({'M':0,'B':1})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1], dtype=int64)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['diagnosis'].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Dropping the irrelevant data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "data=data.drop(['Unnamed: 32','id'],axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Defining our Dependent and Independent variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "X=data.drop('diagnosis',axis=1)\n",
    "Y=data['diagnosis']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>radius_mean</th>\n",
       "      <th>texture_mean</th>\n",
       "      <th>perimeter_mean</th>\n",
       "      <th>area_mean</th>\n",
       "      <th>smoothness_mean</th>\n",
       "      <th>compactness_mean</th>\n",
       "      <th>concavity_mean</th>\n",
       "      <th>concave points_mean</th>\n",
       "      <th>symmetry_mean</th>\n",
       "      <th>fractal_dimension_mean</th>\n",
       "      <th>...</th>\n",
       "      <th>radius_worst</th>\n",
       "      <th>texture_worst</th>\n",
       "      <th>perimeter_worst</th>\n",
       "      <th>area_worst</th>\n",
       "      <th>smoothness_worst</th>\n",
       "      <th>compactness_worst</th>\n",
       "      <th>concavity_worst</th>\n",
       "      <th>concave points_worst</th>\n",
       "      <th>symmetry_worst</th>\n",
       "      <th>fractal_dimension_worst</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>17.99</td>\n",
       "      <td>10.38</td>\n",
       "      <td>122.80</td>\n",
       "      <td>1001.0</td>\n",
       "      <td>0.11840</td>\n",
       "      <td>0.27760</td>\n",
       "      <td>0.30010</td>\n",
       "      <td>0.14710</td>\n",
       "      <td>0.2419</td>\n",
       "      <td>0.07871</td>\n",
       "      <td>...</td>\n",
       "      <td>25.380</td>\n",
       "      <td>17.33</td>\n",
       "      <td>184.60</td>\n",
       "      <td>2019.0</td>\n",
       "      <td>0.16220</td>\n",
       "      <td>0.66560</td>\n",
       "      <td>0.7119</td>\n",
       "      <td>0.2654</td>\n",
       "      <td>0.4601</td>\n",
       "      <td>0.11890</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>20.57</td>\n",
       "      <td>17.77</td>\n",
       "      <td>132.90</td>\n",
       "      <td>1326.0</td>\n",
       "      <td>0.08474</td>\n",
       "      <td>0.07864</td>\n",
       "      <td>0.08690</td>\n",
       "      <td>0.07017</td>\n",
       "      <td>0.1812</td>\n",
       "      <td>0.05667</td>\n",
       "      <td>...</td>\n",
       "      <td>24.990</td>\n",
       "      <td>23.41</td>\n",
       "      <td>158.80</td>\n",
       "      <td>1956.0</td>\n",
       "      <td>0.12380</td>\n",
       "      <td>0.18660</td>\n",
       "      <td>0.2416</td>\n",
       "      <td>0.1860</td>\n",
       "      <td>0.2750</td>\n",
       "      <td>0.08902</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>19.69</td>\n",
       "      <td>21.25</td>\n",
       "      <td>130.00</td>\n",
       "      <td>1203.0</td>\n",
       "      <td>0.10960</td>\n",
       "      <td>0.15990</td>\n",
       "      <td>0.19740</td>\n",
       "      <td>0.12790</td>\n",
       "      <td>0.2069</td>\n",
       "      <td>0.05999</td>\n",
       "      <td>...</td>\n",
       "      <td>23.570</td>\n",
       "      <td>25.53</td>\n",
       "      <td>152.50</td>\n",
       "      <td>1709.0</td>\n",
       "      <td>0.14440</td>\n",
       "      <td>0.42450</td>\n",
       "      <td>0.4504</td>\n",
       "      <td>0.2430</td>\n",
       "      <td>0.3613</td>\n",
       "      <td>0.08758</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>11.42</td>\n",
       "      <td>20.38</td>\n",
       "      <td>77.58</td>\n",
       "      <td>386.1</td>\n",
       "      <td>0.14250</td>\n",
       "      <td>0.28390</td>\n",
       "      <td>0.24140</td>\n",
       "      <td>0.10520</td>\n",
       "      <td>0.2597</td>\n",
       "      <td>0.09744</td>\n",
       "      <td>...</td>\n",
       "      <td>14.910</td>\n",
       "      <td>26.50</td>\n",
       "      <td>98.87</td>\n",
       "      <td>567.7</td>\n",
       "      <td>0.20980</td>\n",
       "      <td>0.86630</td>\n",
       "      <td>0.6869</td>\n",
       "      <td>0.2575</td>\n",
       "      <td>0.6638</td>\n",
       "      <td>0.17300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>20.29</td>\n",
       "      <td>14.34</td>\n",
       "      <td>135.10</td>\n",
       "      <td>1297.0</td>\n",
       "      <td>0.10030</td>\n",
       "      <td>0.13280</td>\n",
       "      <td>0.19800</td>\n",
       "      <td>0.10430</td>\n",
       "      <td>0.1809</td>\n",
       "      <td>0.05883</td>\n",
       "      <td>...</td>\n",
       "      <td>22.540</td>\n",
       "      <td>16.67</td>\n",
       "      <td>152.20</td>\n",
       "      <td>1575.0</td>\n",
       "      <td>0.13740</td>\n",
       "      <td>0.20500</td>\n",
       "      <td>0.4000</td>\n",
       "      <td>0.1625</td>\n",
       "      <td>0.2364</td>\n",
       "      <td>0.07678</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>564</th>\n",
       "      <td>21.56</td>\n",
       "      <td>22.39</td>\n",
       "      <td>142.00</td>\n",
       "      <td>1479.0</td>\n",
       "      <td>0.11100</td>\n",
       "      <td>0.11590</td>\n",
       "      <td>0.24390</td>\n",
       "      <td>0.13890</td>\n",
       "      <td>0.1726</td>\n",
       "      <td>0.05623</td>\n",
       "      <td>...</td>\n",
       "      <td>25.450</td>\n",
       "      <td>26.40</td>\n",
       "      <td>166.10</td>\n",
       "      <td>2027.0</td>\n",
       "      <td>0.14100</td>\n",
       "      <td>0.21130</td>\n",
       "      <td>0.4107</td>\n",
       "      <td>0.2216</td>\n",
       "      <td>0.2060</td>\n",
       "      <td>0.07115</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>565</th>\n",
       "      <td>20.13</td>\n",
       "      <td>28.25</td>\n",
       "      <td>131.20</td>\n",
       "      <td>1261.0</td>\n",
       "      <td>0.09780</td>\n",
       "      <td>0.10340</td>\n",
       "      <td>0.14400</td>\n",
       "      <td>0.09791</td>\n",
       "      <td>0.1752</td>\n",
       "      <td>0.05533</td>\n",
       "      <td>...</td>\n",
       "      <td>23.690</td>\n",
       "      <td>38.25</td>\n",
       "      <td>155.00</td>\n",
       "      <td>1731.0</td>\n",
       "      <td>0.11660</td>\n",
       "      <td>0.19220</td>\n",
       "      <td>0.3215</td>\n",
       "      <td>0.1628</td>\n",
       "      <td>0.2572</td>\n",
       "      <td>0.06637</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>566</th>\n",
       "      <td>16.60</td>\n",
       "      <td>28.08</td>\n",
       "      <td>108.30</td>\n",
       "      <td>858.1</td>\n",
       "      <td>0.08455</td>\n",
       "      <td>0.10230</td>\n",
       "      <td>0.09251</td>\n",
       "      <td>0.05302</td>\n",
       "      <td>0.1590</td>\n",
       "      <td>0.05648</td>\n",
       "      <td>...</td>\n",
       "      <td>18.980</td>\n",
       "      <td>34.12</td>\n",
       "      <td>126.70</td>\n",
       "      <td>1124.0</td>\n",
       "      <td>0.11390</td>\n",
       "      <td>0.30940</td>\n",
       "      <td>0.3403</td>\n",
       "      <td>0.1418</td>\n",
       "      <td>0.2218</td>\n",
       "      <td>0.07820</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>567</th>\n",
       "      <td>20.60</td>\n",
       "      <td>29.33</td>\n",
       "      <td>140.10</td>\n",
       "      <td>1265.0</td>\n",
       "      <td>0.11780</td>\n",
       "      <td>0.27700</td>\n",
       "      <td>0.35140</td>\n",
       "      <td>0.15200</td>\n",
       "      <td>0.2397</td>\n",
       "      <td>0.07016</td>\n",
       "      <td>...</td>\n",
       "      <td>25.740</td>\n",
       "      <td>39.42</td>\n",
       "      <td>184.60</td>\n",
       "      <td>1821.0</td>\n",
       "      <td>0.16500</td>\n",
       "      <td>0.86810</td>\n",
       "      <td>0.9387</td>\n",
       "      <td>0.2650</td>\n",
       "      <td>0.4087</td>\n",
       "      <td>0.12400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568</th>\n",
       "      <td>7.76</td>\n",
       "      <td>24.54</td>\n",
       "      <td>47.92</td>\n",
       "      <td>181.0</td>\n",
       "      <td>0.05263</td>\n",
       "      <td>0.04362</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.1587</td>\n",
       "      <td>0.05884</td>\n",
       "      <td>...</td>\n",
       "      <td>9.456</td>\n",
       "      <td>30.37</td>\n",
       "      <td>59.16</td>\n",
       "      <td>268.6</td>\n",
       "      <td>0.08996</td>\n",
       "      <td>0.06444</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.2871</td>\n",
       "      <td>0.07039</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>569 rows × 30 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     radius_mean  texture_mean  perimeter_mean  area_mean  smoothness_mean  \\\n",
       "0          17.99         10.38          122.80     1001.0          0.11840   \n",
       "1          20.57         17.77          132.90     1326.0          0.08474   \n",
       "2          19.69         21.25          130.00     1203.0          0.10960   \n",
       "3          11.42         20.38           77.58      386.1          0.14250   \n",
       "4          20.29         14.34          135.10     1297.0          0.10030   \n",
       "..           ...           ...             ...        ...              ...   \n",
       "564        21.56         22.39          142.00     1479.0          0.11100   \n",
       "565        20.13         28.25          131.20     1261.0          0.09780   \n",
       "566        16.60         28.08          108.30      858.1          0.08455   \n",
       "567        20.60         29.33          140.10     1265.0          0.11780   \n",
       "568         7.76         24.54           47.92      181.0          0.05263   \n",
       "\n",
       "     compactness_mean  concavity_mean  concave points_mean  symmetry_mean  \\\n",
       "0             0.27760         0.30010              0.14710         0.2419   \n",
       "1             0.07864         0.08690              0.07017         0.1812   \n",
       "2             0.15990         0.19740              0.12790         0.2069   \n",
       "3             0.28390         0.24140              0.10520         0.2597   \n",
       "4             0.13280         0.19800              0.10430         0.1809   \n",
       "..                ...             ...                  ...            ...   \n",
       "564           0.11590         0.24390              0.13890         0.1726   \n",
       "565           0.10340         0.14400              0.09791         0.1752   \n",
       "566           0.10230         0.09251              0.05302         0.1590   \n",
       "567           0.27700         0.35140              0.15200         0.2397   \n",
       "568           0.04362         0.00000              0.00000         0.1587   \n",
       "\n",
       "     fractal_dimension_mean  ...  radius_worst  texture_worst  \\\n",
       "0                   0.07871  ...        25.380          17.33   \n",
       "1                   0.05667  ...        24.990          23.41   \n",
       "2                   0.05999  ...        23.570          25.53   \n",
       "3                   0.09744  ...        14.910          26.50   \n",
       "4                   0.05883  ...        22.540          16.67   \n",
       "..                      ...  ...           ...            ...   \n",
       "564                 0.05623  ...        25.450          26.40   \n",
       "565                 0.05533  ...        23.690          38.25   \n",
       "566                 0.05648  ...        18.980          34.12   \n",
       "567                 0.07016  ...        25.740          39.42   \n",
       "568                 0.05884  ...         9.456          30.37   \n",
       "\n",
       "     perimeter_worst  area_worst  smoothness_worst  compactness_worst  \\\n",
       "0             184.60      2019.0           0.16220            0.66560   \n",
       "1             158.80      1956.0           0.12380            0.18660   \n",
       "2             152.50      1709.0           0.14440            0.42450   \n",
       "3              98.87       567.7           0.20980            0.86630   \n",
       "4             152.20      1575.0           0.13740            0.20500   \n",
       "..               ...         ...               ...                ...   \n",
       "564           166.10      2027.0           0.14100            0.21130   \n",
       "565           155.00      1731.0           0.11660            0.19220   \n",
       "566           126.70      1124.0           0.11390            0.30940   \n",
       "567           184.60      1821.0           0.16500            0.86810   \n",
       "568            59.16       268.6           0.08996            0.06444   \n",
       "\n",
       "     concavity_worst  concave points_worst  symmetry_worst  \\\n",
       "0             0.7119                0.2654          0.4601   \n",
       "1             0.2416                0.1860          0.2750   \n",
       "2             0.4504                0.2430          0.3613   \n",
       "3             0.6869                0.2575          0.6638   \n",
       "4             0.4000                0.1625          0.2364   \n",
       "..               ...                   ...             ...   \n",
       "564           0.4107                0.2216          0.2060   \n",
       "565           0.3215                0.1628          0.2572   \n",
       "566           0.3403                0.1418          0.2218   \n",
       "567           0.9387                0.2650          0.4087   \n",
       "568           0.0000                0.0000          0.2871   \n",
       "\n",
       "     fractal_dimension_worst  \n",
       "0                    0.11890  \n",
       "1                    0.08902  \n",
       "2                    0.08758  \n",
       "3                    0.17300  \n",
       "4                    0.07678  \n",
       "..                       ...  \n",
       "564                  0.07115  \n",
       "565                  0.06637  \n",
       "566                  0.07820  \n",
       "567                  0.12400  \n",
       "568                  0.07039  \n",
       "\n",
       "[569 rows x 30 columns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((569, 30), (569,))"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape , Y.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.metrics import classification_report,accuracy_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train, x_test, y_train, y_test = train_test_split(X, Y , test_size=0.2 , random_state=200)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "sc = StandardScaler()\n",
    "X_train = sc.fit_transform(x_train)\n",
    "X_test = sc.transform(x_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Why do we need to Standardize our data?\n",
    "**For classification algorithms like KNN, we measure the distances between pairs of samples and these distances are influenced by the measurement units also. For example: Let’s say, we are applying KNN on a data set having 3 features.First feature ranging from 1-10, second from 1-20 and the last one ranging from 1-1000. In this case, most of the clusters will be generated based on the last feature as the difference between 1 to 10 and 1-20 are smaller as compared to 1-1000. To avoid this miss classification, we should normalize the feature variables.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.13398313, -0.37790596, -0.18649353, ..., -0.36899428,\n",
       "        -0.62916077, -0.7991975 ],\n",
       "       [-0.62521863,  0.35662947, -0.58575874, ..., -0.00165682,\n",
       "        -0.71291944,  0.84431363],\n",
       "       [-0.75640083, -0.75694507, -0.78155613, ..., -0.74464994,\n",
       "        -0.1029035 ,  0.1816865 ],\n",
       "       ...,\n",
       "       [-0.20096979, -0.11658085, -0.19941212, ...,  0.40236984,\n",
       "         0.95119139,  0.12837168],\n",
       "       [-1.09412523, -1.06771006, -1.05688361, ..., -1.11213595,\n",
       "        -0.76191036,  0.03207857],\n",
       "       [-0.36843644, -0.29315187, -0.37098716, ..., -0.53996291,\n",
       "        -0.21668886, -0.27801787]])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 4.46479573e-02, -1.35964081e+00,  5.26681024e-03, ...,\n",
       "        -1.02663479e-01,  2.30550805e-01, -1.28229511e+00],\n",
       "       [ 1.17216837e-01,  9.52262368e-01,  6.42078824e-02, ...,\n",
       "         3.01363172e-01, -6.93955213e-01, -8.66657077e-01],\n",
       "       [-1.15273856e+00, -4.03803039e-01, -1.12914698e+00, ...,\n",
       "        -7.51928366e-01, -3.49483593e-02, -4.16201195e-01],\n",
       "       ...,\n",
       "       [ 5.57240663e-03,  3.02481027e-01,  1.22975051e-03, ...,\n",
       "         2.71655330e-01, -6.03875139e-01, -3.10659600e-01],\n",
       "       [-6.53129734e-01, -1.28352253e-01, -5.83336500e-01, ...,\n",
       "         5.79131495e-01,  2.91556914e+00,  3.04164613e+00],\n",
       "       [-4.27049762e-01, -2.53129105e-01, -3.91172456e-01, ...,\n",
       "         3.72661993e-01,  4.96049969e-01,  1.03417970e+00]])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier(n_neighbors=10)"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "classifier = KNeighborsClassifier(n_neighbors =10, metric = 'minkowski', p = 2)\n",
    "classifier.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = classifier.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 0, 1,\n",
       "       1, 0, 0, 1, 1, 0, 1, 0, 1, 1, 0, 1, 0, 0, 1, 1, 1, 0, 1, 1, 0, 1,\n",
       "       0, 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0,\n",
       "       1, 1, 1, 0, 1, 1, 1, 1, 0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 1,\n",
       "       0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1,\n",
       "       1, 1, 0, 1], dtype=int64)"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_pred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.97      0.92      0.95        38\n",
      "           1       0.96      0.99      0.97        76\n",
      "\n",
      "    accuracy                           0.96       114\n",
      "   macro avg       0.97      0.95      0.96       114\n",
      "weighted avg       0.97      0.96      0.96       114\n",
      "\n"
     ]
    }
   ],
   "source": [
    "predicted = classifier.predict(X_test)\n",
    "report = classification_report(y_test, y_pred)\n",
    "print(report)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## From this model based on KNN we get an accuracy of : 0.97 and,\n",
    "## F1-score: 0.96 !\n",
    "\n",
    "### What can we do to imporove our model ?\n",
    "\n",
    "1. Hyperparameter-tuning.\n",
    "2. Use K-folds method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}