cnn.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cardiovascular Disease (CVD) is the \"number one cause of death worldwide\", having \"almost 17.9 million deaths occurring\" per year. CVD in many cases if treatment is given early the risk factor is greatly reduced. CVD also has many causing factors and therefore to make a accurate study huge amount of data needs to be collected (Princy, R.J.P. et al) (Ali, M.M. et al).\n",
    "\n",
    "It is a perfect opportunity for machine learning, which becomes more accurate with more data. I have chosen to illustrate a convulsion neural network (CNN) by using a reduced heart disease dataset from \"Rousseauw et al, 1983, South African Medical Journal\"."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Method"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Install Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install tensorflow==2.8.0\n",
    "%pip install liac-arff==2.5.0\n",
    "%pip install pandas==1.4.2\n",
    "%pip install scikit-learn==1.0.2\n",
    "%pip install matplotlib==3.5.1\n",
    "%pip install seaborn==0.11.2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports\n",
    "First we will import required libraries.\n",
    "\n",
    "- liac-arff is used to process arff files\n",
    "- matplotlib & seaborn is used to output result graphs\n",
    "- scikit-learn is used to assist with dataset preparation for machine learning\n",
    "- tensorflow is used to build, train and test models\n",
    "- pandas is used to load csv data into a dataframe for in-memory manipulation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "from urllib.request import urlopen\n",
    "\n",
    "import arff\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import accuracy_score, confusion_matrix\n",
    "import seaborn as sns\n",
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Dense\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Variables\n",
    "Here we will set up some constant variables to be shared with various processes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "DATA_SET_URL = \"https://www.openml.org/data/download/1592290/phpgNaXZe\"\n",
    "ARFF_FILE_NAME = \"dataset.arff\"\n",
    "CSV_FILE_NAME = \"dataset.csv\"\n",
    "CSV_HEADER = \"sbp,tobacco,ldl,adiposity,famhist,type,obesity,alcohol,age,chd\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Download Dataset\n",
    "First we will need to download the dataset from OpenML. This section of code will download it to write it to disk.\n",
    "\n",
    "When downloading the dataset a few extra white space characters are added; the code below removes them using the built-in `strip()` function.\n",
    "\n",
    "Before downloading a check will be run to skip this if the file already exists."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "if not Path(ARFF_FILE_NAME).exists():\n",
    "    with urlopen(DATA_SET_URL) as response:\n",
    "        content = response.read()\n",
    "        with open(ARFF_FILE_NAME, \"wb\") as fo:\n",
    "            fo.write(content.strip())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load Dataset\n",
    "\n",
    "The following code will load the arff data into memory using the \"liac-arff\" library. This is required as the data is given as an arff file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(ARFF_FILE_NAME) as fo:\n",
    "    arff_dataset = arff.load(fo)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Convert To CSV\n",
    "To use this dataset with tensorflow we need to convert it to a csv file. This code creates a csv file places a predefined header (which matches the attribute names). Then we access the \"data\" field of the dataset, which is provided as a list which we then join into a string separating each column with a \",\" and ending each row with \"\\n\", which is the newline character. This csv file will then be written to disk.\n",
    "\n",
    "Before processing a new csv file; a check will be run to skip this if the file already exists."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "if not Path(CSV_FILE_NAME).exists():\n",
    "    data = arff_dataset[\"data\"]\n",
    "\n",
    "    with open(CSV_FILE_NAME, \"wt\") as fo:\n",
    "        fo.write(CSV_HEADER + \"\\n\")\n",
    "        for row in data:\n",
    "            fo.write(\",\".join([str(col) for col in row]) + \"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load CSV\n",
    "To load it into tensorflow we need to read the csv data into a pandas data-frame, this allows for the data to be manipulated in memory easily.\n",
    "\n",
    "To load the \"x\" coordinate data, which will be the inputs, we will use pandas `get_dummies()` method, which will convert the data into a format for tensor flow and dropping the output column.\n",
    "\n",
    "Then we will store the output column as \"y\" which has the header name of \"chd\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(CSV_FILE_NAME)\n",
    "\n",
    "x = pd.get_dummies(df.drop(['chd'], axis=1))\n",
    "y = df[\"chd\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As our data is now loaded, we can use sklearn's `train_test_split()` method, which will allow us to split our data into two sections, one for training and the other for testing. I have selected to keep 20% for testing leaving 80% for training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a sample output of our training data \"x\" rows"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sbp</th>\n",
       "      <th>tobacco</th>\n",
       "      <th>ldl</th>\n",
       "      <th>adiposity</th>\n",
       "      <th>famhist</th>\n",
       "      <th>type</th>\n",
       "      <th>obesity</th>\n",
       "      <th>alcohol</th>\n",
       "      <th>age</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>282</th>\n",
       "      <td>206.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>4.17</td>\n",
       "      <td>33.23</td>\n",
       "      <td>2</td>\n",
       "      <td>69.0</td>\n",
       "      <td>27.36</td>\n",
       "      <td>6.17</td>\n",
       "      <td>50.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>105</th>\n",
       "      <td>158.0</td>\n",
       "      <td>3.60</td>\n",
       "      <td>2.97</td>\n",
       "      <td>30.11</td>\n",
       "      <td>2</td>\n",
       "      <td>63.0</td>\n",
       "      <td>26.64</td>\n",
       "      <td>108.00</td>\n",
       "      <td>64.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>288</th>\n",
       "      <td>123.0</td>\n",
       "      <td>0.05</td>\n",
       "      <td>4.61</td>\n",
       "      <td>13.69</td>\n",
       "      <td>2</td>\n",
       "      <td>51.0</td>\n",
       "      <td>23.23</td>\n",
       "      <td>2.78</td>\n",
       "      <td>16.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>327</th>\n",
       "      <td>128.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>2.98</td>\n",
       "      <td>12.59</td>\n",
       "      <td>2</td>\n",
       "      <td>65.0</td>\n",
       "      <td>20.74</td>\n",
       "      <td>2.06</td>\n",
       "      <td>19.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>376</th>\n",
       "      <td>114.0</td>\n",
       "      <td>3.60</td>\n",
       "      <td>4.16</td>\n",
       "      <td>22.58</td>\n",
       "      <td>2</td>\n",
       "      <td>60.0</td>\n",
       "      <td>24.49</td>\n",
       "      <td>65.31</td>\n",
       "      <td>31.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       sbp  tobacco   ldl  adiposity  famhist  type  obesity  alcohol   age\n",
       "282  206.0     0.00  4.17      33.23        2  69.0    27.36     6.17  50.0\n",
       "105  158.0     3.60  2.97      30.11        2  63.0    26.64   108.00  64.0\n",
       "288  123.0     0.05  4.61      13.69        2  51.0    23.23     2.78  16.0\n",
       "327  128.0     0.00  2.98      12.59        2  65.0    20.74     2.06  19.0\n",
       "376  114.0     3.60  4.16      22.58        2  60.0    24.49    65.31  31.0"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a sample output of our training data \"y\" rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "282    2\n",
       "105    1\n",
       "288    1\n",
       "327    1\n",
       "376    1\n",
       "Name: chd, dtype: int64"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create Model\n",
    "Now our data is loaded and processed, we need to make our CNN models. As our dataset is numerical I am using the `Sequential` model from Keras.\n",
    "\n",
    "The first layer of our network will be the input, it will have two nodes as that is the number of classes in our dataset. The number of inputs will match the length of our columns from the dataset.\n",
    "\n",
    "We will use one hidden layer as we are working with linear relationships. We will also set the number of nodes to sixteen.\n",
    "\n",
    "The last layer will be our output, because we want a true/false output the number of nodes has been set to one. It will also use a sigmoid function as we want a predication value of whether it is likely true or false."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = Sequential()\n",
    "model.add(Dense(units=2, activation='relu', input_dim=len(x_train.columns)))\n",
    "model.add(Dense(units=16, activation='relu'))\n",
    "model.add(Dense(units=1, activation='sigmoid'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we compile the model. We will calculate loss using `binary_crossentropy`. We have chosen to use Binary Cross Entropy as it calculates the loss between predicted true and false labels. As our output layer of our model is either true/false it is well suited.\n",
    "\n",
    "We will use `sgd` for the optimizer, which will allow the changing of weights and learning rates to reduce losses. I have chosen to use the stochastic gradient descent optimizer as it requires less memory and converges in less time due to frequent updates to model parameters.\n",
    "\n",
    "We will also request that metric data for accuracy to be shown during fitting so we can see the progress of the training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(loss='binary_crossentropy', optimizer='sgd', metrics='accuracy')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we can see a summary of how our model is constructed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_4\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " dense_9 (Dense)             (None, 2)                 20        \n",
      "                                                                 \n",
      " dense_10 (Dense)            (None, 16)                48        \n",
      "                                                                 \n",
      " dense_11 (Dense)            (None, 1)                 17        \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 85\n",
      "Trainable params: 85\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train Model\n",
    "Here we train the model, specifying that we want 4 epochs as we don't have much data to train with and we don't want to over-fit our model.\n",
    "\n",
    "Multiprocessing is also specified which will allow full utilisation of all CPU cores."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/4\n",
      "12/12 [==============================] - 0s 680us/step - loss: nan - accuracy: 0.0000e+00\n",
      "Epoch 2/4\n",
      "12/12 [==============================] - 0s 705us/step - loss: nan - accuracy: 0.0000e+00\n",
      "Epoch 3/4\n",
      "12/12 [==============================] - 0s 684us/step - loss: nan - accuracy: 0.0000e+00\n",
      "Epoch 4/4\n",
      "12/12 [==============================] - 0s 790us/step - loss: nan - accuracy: 0.0000e+00\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f904c7b0430>"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(x_train, y_train, epochs=4, use_multiprocessing=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test Model\n",
    "Now the model has been trained we can test it using our test data. We will use the `predict()` method to pass in our test data and get out a predication. We will then convert the accuracy values to true/false."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_predication = model.predict(x_test)\n",
    "y_predication = [0 if val < 0.5 else 1 for val in y_predication]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Calculate Accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "63.4409 % Accurate\n"
     ]
    }
   ],
   "source": [
    "accuracy = accuracy_score(y_test, y_predication)\n",
    "\n",
    "print(round(accuracy * 100, 4), \"% Accurate\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWgAAAEiCAYAAADUJkjfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAfx0lEQVR4nO3de5xd49n/8c93ZhIJCUVlpBKHikNDJRR1KHVo1bGi4qzSFlEtT5+qKuWnKH1Kn2q1VBtVDU1VVR3j+AvqUNUIqcRZCSJHGhEJkkyu54+1JnbGzN57JnvtvfbM9+21XrPW2mvf65rJuObe17rXvRQRmJlZ/jTUOgAzM2ufE7SZWU45QZuZ5ZQTtJlZTjlBm5nllBO0mVlOOUHbSpPUV9KtkuZLun4l2jlK0t2VjK0WJN0haVSt47D65wTdg0g6UtJjkt6RNDNNJJ+pQNMjgWZg7Yg4pKuNRMS4iNirAvGsQNJukkLSjW32D0v3319mO+dI+kOp4yJin4gY28VwzZZzgu4hJJ0C/Bz4EUkyXR/4FXBgBZrfAHg+IpZWoK2szAV2lLR2wb5RwPOVOoES/n/KKsa/TD2ApDWA84BvRsRfI2JhRCyJiFsj4rvpMatI+rmkGenyc0mrpK/tJmm6pO9ImpP2vr+avnYucDZwWNozP7ZtT1PShmlPtSnd/oqklyQtkPSypKMK9j9U8L6dJE1MSycTJe1U8Nr9kn4o6eG0nbslfbTIj2ExcBNwePr+RuAwYFybn9Ulkl6T9LakSZJ2SffvDXy/4Pv8V0EcF0h6GFgEfDzdd1z6+uWSbiho/0JJEySp3H8/67mcoHuGHYE+wI1FjjkT2AEYDgwDtgfOKnh9XWANYD3gWOAySWtGxA9IeuXXRUS/iLiyWCCSVgN+AewTEf2BnYDJ7Ry3FjA+PXZt4GJgfJse8JHAV4EBQG/g1GLnBq4GjknXvwBMBWa0OWYiyc9gLeCPwPWS+kTEnW2+z2EF7/kyMBroD7zSpr3vAJ9M//jsQvKzGxWeY8HK4ATdM6wNvFGiBHEUcF5EzImIucC5JImn1ZL09SURcTvwDrBZF+NZBmwpqW9EzIyIp9o5Zj/ghYi4JiKWRsS1wLPAAQXHXBURz0fEu8CfSRJrhyLi78BakjYjSdRXt3PMHyLizfScPwVWofT3+fuIeCp9z5I27S0i+TleDPwBODkippdozwxwgu4p3gQ+2lpi6MDHWLH390q6b3kbbRL8IqBfZwOJiIUkpYWvAzMljZe0eRnxtMa0XsH2rC7Ecw1wErA77XyikHSqpGfSsspbJJ8aipVOAF4r9mJEPAq8BIjkD4lZWZyge4ZHgPeBEUWOmUFysa/V+nz443+5FgKrFmyvW/hiRNwVEZ8HBpL0iq8oI57WmF7vYkytrgG+Adye9m6XS0sQpwGHAmtGxEeA+SSJFaCjskTRcoWkb5L0xGek7ZuVxQm6B4iI+SQX8i6TNELSqpJ6SdpH0kXpYdcCZ0laJ73YdjbJR/KumAzsKmn99ALlGa0vSGqWdGBai36fpFSyrJ02bgc2TYcGNkk6DBgK3NbFmACIiJeBz5LU3NvqDywlGfHRJOlsYPWC12cDG3ZmpIakTYHzgaNJSh2nSRreteitp3GC7iHSeuopJBf+5pJ8LD+JZGQDJEnkMeBJYArweLqvK+e6B7gubWsSKybVhjSOGcB/SJLlie208SawP8lFtjdJep77R8QbXYmpTdsPRUR7nw7uAu4kGXr3CvAeK5YvWm/CeVPS46XOk5aU/gBcGBH/iogXSEaCXNM6QsasGPlisplZPrkHbWaWU07QZmY55QRtZpZTTtBmZjnlBG1mllPF7iyrqb5bn+ThJfYh8yZeWusQLIf6NLHSk091Jue8+8SlVZnsKrcJ2sysqhoaax3BhzhBm5kB5HAqbydoMzOAHE7R7QRtZgbuQZuZ5ZZ70GZmOeUetJlZTnkUh5lZTrnEYWaWUy5xmJnllHvQZmY55R60mVlOOUGbmeVUo0dxmJnlk2vQZmY55RKHmVlOuQdtZpZT7kGbmeWUb/U2M8splzjMzHLKJQ4zs5xyD9rMLKfcgzYzyyknaDOznPIoDjOznHIN2swsp1ziMDPLKfegzczySU7QZmb5pIbKJWhJ04AFQAuwNCK2lbQWcB2wITANODQi5hVrJ39FFzOzGpBU9lKm3SNieERsm26fDkyIiE2ACel2UU7QZmZkkqDbOhAYm66PBUaUeoMTtJkZFU/QAdwtaZKk0em+5oiYma7PAppLNeIatJkZnbtImCbd0QW7xkTEmILtz0TE65IGAPdIerbw/RERkqLUeZygzcwAOlG5SJPxmCKvv55+nSPpRmB7YLakgRExU9JAYE6p87jEYWYGNDQ0lL0UI2k1Sf1b14G9gKnALcCo9LBRwM2lYnIP2syMio6DbgZuTNtrAv4YEXdKmgj8WdKxwCvAoaUacoI2M6NyCToiXgKGtbP/TWDPzrTlBG1mBp2qQVeLE7SZGb7V28wst5ygzcxyqpJzcVSKE7SZGe5Bm5nlVh4TdKY3qkhaVdL/k3RFur2JpP2zPKeZWVdUYbKkTsv6TsKrgPeBHdPt14HzMz6nmVmn9cQEvXFEXAQsAYiIReRytKGZ9XRqUNlLtWRdg14sqS/J1HtI2pikR21mlit5rEFnnaB/ANwJDJY0DtgZ+ErG5zQz67Qel6Aj4h5JjwM7kJQ2vhURb2R5TjOzLslffs42QUvaGZgcEeMlHQ18X9IlEfFKluetd8+OP5cFC9+nZdkylrYs4zNHXcQnN12PX555OKv1XYVXZrzJV88cy4KF79U6VKuRhx98gAt/fAHLWpZx0MGHcOzxo0u/yYrqcT1o4HJgmKRhwCnAlcDVwGczPm/d23v0Jbz51sLl25effSSn/+xGHpr0IsccuAPfHrUn5/1qfA0jtFppaWnhRxecx2+uuIrm5maOPGwku+2+BxsPGVLr0OpaHhN01qM4lkZEkDws8bKIuAzon/E5u6Uh6w/goUkvAnDvP55lxJ7DaxuQ1czUKU8yePAGDBo8mF69e7P3vvtx/30Tah1W3avUhP0VjSnj9hdIOgM4GhgvqQHolfE5615EcOuvTuLhcafxtS/tDMAzL83kgN22AuBLn9+GQc1r1jJEq6E5s2ez7sB1l28PaG5m9uzZNYyom1AnlirJusRxGHAkcGxEzJK0PvCTjg4ufBBj06DdaProFhmHl097fvVnzJg7n3XW7Mdtvz6J56bN4oRzxvHT00Zy+vF7M/5vU1i8pKXWYZp1K3kscWQ9imMWcHHB9qskNeiOjl/+IMa+W59U8om33dWMufMBmDvvHW6590m222JDfn7NBA74xmVAUu7YZ5ee+cfLkh7zrJmzlm/PmT2b5ubmGkbUPeQxQWdS4pC0QNLb7SwLJL2dxTm7i1X79KbfqqssX//cjpvz1L9nsM6a/YDkl+j047/AFX95qJZhWg1tseUnefXVaUyf/hpLFi/mztvH89nd96h1WHVPKn+plkx60BHhC4FdNGDt/lx38fEANDU2ct0dj3HP35/hm0fsxgmH7QrAzfdO5uqb/1HLMK2GmpqaOOPMszlx9HEsW9bCiIMOZsiQTWodVt3LYw9aySCLjE8iDQD6tG6npY6ienKJwzo2b+KltQ7BcqhP08pfutvse3eVnXOeu/ALVcnmWU83+kVJLwAvA38DpgF3ZHlOM7OuyGOJI+thdj8kuc37+YjYiOSR4/5sbma509CgspeqxZRx+0si4k2gQVJDRNwHbJvxOc3MOi2PPeisx0G/Jakf8AAwTtIcYGGJ95iZVV0eLxJmNcxu/XT1QGAR8G2SaUf/DRyQxTnNzFZGHkscWfWgbwK2iYiFkm6IiIOBsRmdy8xspeWxB51Vgi78Tj+e0TnMzComh/k5s4uE0cG6mVkuVfqhsZIaJT0h6bZ0eyNJj0p6UdJ1knqXaiOrBD2s9dZuYCvf6m1meZfBKI5vAc8UbF8I/CwihgDzgGNLNZBJgo6IxohYPSL6R0RTut66vXoW5zQzWxmV7EFLGgTsB/w23RawB/CX9JCxwIhS7WQ9zM7MrC5UeHTGz4HT+OABJWsDb0XE0nR7OrBeyZgqGZGZWb3qTIlD0mhJjxUsoz9oR/sDcyJi0srG5B60mRmdG2ZXOHd9O3YGvihpX5JJ4lYHLgE+Iqkp7UUPAl4vdR73oM3MqNxFwog4IyIGRcSGwOHAvRFxFHAfMDI9bBRwc6mYnKDNzKj8MLt2fA84RdKLJDXpK0u9wSUOMzOyuVElIu4H7k/XXwK278z7naDNzKj4KI6KcII2M6NnzcVhZlZX8pigS14klHSRpNUl9ZI0QdJcSUdXIzgzs2rJ44T95Yzi2Csi3gb2J3mm4BDgu1kGZWZWbVUYxdFp5ZQ4Wo/ZD7g+Iubn8aOAmdnKqNeLhLdJehZ4FzhR0jrAe9mGZWZWXXnsd5ZM0BFxuqSLgPkR0SJpEcmjrMzMuo2GHGboci4Srgp8A7g83fUx/GRuM+tm6vUi4VXAYmCndPt14PzMIjIzq4E8XiQsJ0FvHBEXAUsAImIRKz5z0Mys7jWo/KVayrlIuFhSX9JnC0raGHg/06jMzKqsXkdx/AC4ExgsaRzJXKdfyTIoM7NqUw4LA+WM4rhH0uPADiSljW9FxBuZR2ZmVkU57ECXTtCSdk1XF6Rfh0oiIh7ILiwzs+rK4w145ZQ4Cm/r7kMyn+kkkifUmpl1CznMz2WVOA4o3JY0mOSJtWZm3UZjDmscXZludDrwiUoHYmZWS3VZ4pD0S9IhdiTjpocDj2cYk5lZ1eUwP5fVg36sYH0pcG1EPJxRPGZmNZHHuTjKqUGPrUYgZma1lL/0XCRBS5rCB6WNFV4CIiK2yiwqM7Mqq7ca9P5Vi8LMrMbqahRHRLxSzUDMzGophx3osuaD3kHSREnvSFosqUXS29UIzsysWvI43Wg5ozguBQ4HrieZqP8YYNMsgzIzq7YcVjjKmg+aiHgRaIyIloi4Ctg727DMzKqrXnvQiyT1BianzyacSZmJ3cysXuSwA91xopW0Xbr65fS4k4CFwGDg4OxDMzOrnsYGlb0UI6mPpH9K+pekpySdm+7fSNKjkl6UdF3a8S2qWE94jKQXgOOAj0fE2xFxbkSckpY8zMy6jQqWON4H9oiIYSRTY+wtaQfgQuBnETEEmAccW6qhDhN0RGxNMhZ6KfCX9K/B6ZI2LOebNTOrJ5V6qnck3kk3e6VLkEzR/Jd0/1hgRKmYitaSI+K5tNc8lGT0xhrABEmei8PMupUGqeylFEmNkiYDc4B7gH8Db0XE0vSQ6cB6JWMqJ3BJDcAAoBlYLT2pmVm30ZketKTRkh4rWEYXtpWOeBsODCJ5yMnmXYmp6CgOSbsAR5B0xacAfwK+HRHzu3Kyztj35K9mfQozs+UaOzF8LiLGAGPKOO4tSfcBOwIfkdSU9qIHAa+Xen+xURyvAf8DPA0Mj4gvRMRV1UjOZmbVVqmLhJLWkfSRdL0v8HngGeA+YGR62Cjg5lIxFetBf8bzcZhZT1HBOwkHAmMlNZJ0gv8cEbdJehr4k6TzgSeAK0s15MmSzMyoXIKOiCeBrdvZ/xJJPbpsXXkmoZlZt1Nv80GbmfUYeZwsqdgTVQofFvshEfFfmURkZlYDdTVhPys+LNbMrFvL4wxwxS4S+mGxZtZj5LAEXboGLWkd4HvAUKBP6/6I2CPDuMzMqqqcW7irrZxe/TiSQdYbAecC04CJGcZkZlZ1lZosqZLKSdBrR8SVwJKI+FtEfI1kViYzs26jQeUv1VLOMLsl6deZkvYDZgBrZReSmVn11dsojlbnS1oD+A7wS2B14NuZRmVmVmU5zM+lE3RE3Jauzgd2zzYcM7PaUA6fSljOKI6raOeGlbQWbWbWLdRlDxq4rWC9D3AQSR3azKzbqMsEHRE3FG5LuhZ4KLOIzMxqoF4vEra1Ccnjr8zMuo0c3qdSVg16ASvWoGeR3FloZtZt5PFOwnJKHP2rEYiZWS3lsMJR+k5CSRPK2WdmVs/yeKt3sfmg+wCrAh+VtCYsHyS4OrBeFWIzM6uahjobB30C8N/Ax4BJfJCg3wYuzTYsM7PqaszhhNDF5oO+BLhE0skR8csqxmRmVnV5vEhYzt+MZZI+0rohaU1J38guJDOz6stjDbqcBH18RLzVuhER84DjM4vIzKwGGqSyl2op50aVRkmKiACQ1Aj0zjYsM7PqymGFo6wEfSdwnaTfpNsnpPvMzLqNHF4jLCtBfw8YDZyYbt8DXJFZRGZmNVCXFwkjYllE/DoiRkbESOBpkon7zcy6jXqtQSNpa+AI4FDgZeCvWQZlZlZt+es/F7+TcFOSpHwE8AZwHaCI8FNVzKzbqVTHWNJg4GqgmWSiuTERcYmktUjy6IbANODQdFRch4qVOJ4leXr3/hHxmfRmlZaVD9/MLH8klb2UsBT4TkQMBXYAvilpKHA6MCEiNgEmpNtFFUvQXwJmAvdJukLSnuTzU4CZ2UprlMpeiomImRHxeLq+AHiGZP6iA4Gx6WFjgRGlYuowQUfETRFxOLA5cB/JvBwDJF0uaa+S362ZWR1RZxZptKTHCpbR7bYpbQhsDTwKNEfEzPSlWSQlkKLKmQ96IfBH4I/prHaHkAy9u7vUe83M6kUZpYvlImIMMKZEe/2AG4D/joi3C9uPiJD0oYdxt9WpsdkRMS8ixkTEnp15n5lZ3jV0YilFUi+S5DwuIlpHvc2WNDB9fSAwp5yYzMx6vEpdJFRywJXAMxFxccFLtwCj0vVRwM2lYurKQ2PNzLqdCo6A2Bn4MjBF0uR03/eBHwN/lnQs8ArJfSVFOUGbmUHJ0RnlioiH6Djfd6o87ARtZkb9zmZnZtbtKYe3eThBm5nhHrSZWW7V21O9zcx6jIYcDjrOLCQljpZ0drq9vqTtszqfmdnKUCf+q5Ys/2b8CtiRZLpSgAXAZRmez8ysyxpU/lItWZY4Ph0R20h6ApLbxCX5YbNmlks9bRTHkvQJ4K1PA18HWJbh+czMuqynjeL4BXAjyRSlFwAjgbMyPF+30KtR/HDfzejVmMw7+8i0eVz3xMzlr3/t04PZY9O1OfqaybUL0mru4Qcf4MIfX8CylmUcdPAhHHt8u7NdWif0qB50RIyTNInk1kYBIyLimazO110saQnOueN53lu6jEbB+ftvzuPT3+aFuQvZeO1V6bdKY61DtBpraWnhRxecx2+uuIrm5maOPGwku+2+BxsPGVLr0OpapW71rqQsR3GsDywCbiWZxWlhus9KeG9pUglqbBBNEhA0CI7ZfhBXT5xe2+Cs5qZOeZLBgzdg0ODB9Ordm7333Y/775tQ67DqnlT+Ui1ZljjGk9SfBfQBNgKeA7bI8JzdQoPgoi9+gnVXX4U7n5nLC3MXsd/QAUx89S3eendprcOzGpszezbrDlx3+faA5mamPPlkDSPqHvLXf862xPHJwm1J2wDfyOp83cmygFNvfoZVezfyvT03ZmhzP3bcaE3Ovv25Wodm1m019KQSR1vpQxQ/XeyYwud8vfy3vxY7tEdYtLiFqTMXsMXA/qzbfxUuG7kllx+yJas0NXDpSH8Q6akGNDcza+as5dtzZs+mubnk4+2shM48k7BaMutBSzqlYLMB2AaYUew9hc/5Ovh3k0o+r6s7Wr1PE0uXBYsWt9C7UWz1sf7cNGU2x/3pg4+wf/jycE76y1M1jNJqaYstP8mrr05j+vTXaB7QzJ23j+d/fvLTWodV//LXgc60Bt2/YH0pSU36hgzP1y2s2bcXJ+26IY1KHsHz95fnMem1+bUOy3KkqamJM848mxNHH8eyZS2MOOhghgzZpNZh1b08ljgUUfmOanqDyoURcWpX2+ipPWgrbtwxn6p1CJZDfZpWvv878aX5Zeec7T6+RlWyecV70JKaImKppJ0r3baZWWby14HOpMTxT5J682RJtwDXAwtbXyx4BLmZWW70qDsJScY+vwnswQfjoQNwgjaz3MlhCTqTBD0gHcExlQ8ScyvXlc0sl3pKgm4E+tF+RccJ2sxyqaeUOGZGxHkZtGtmlpme0oPO4bdpZlZcHhNXFgl6zwzaNDPLVg4zdMUTdET8p9JtmpllrafUoM3M6k41HwZbrqrNZmdmlmsVnM5O0u8kzZE0tWDfWpLukfRC+nXNUu04QZuZkZQ4yv2vDL8H9m6z73RgQkRsAkxIt4tygjYzo7KPvIqIB4C21+MOBMam62OBEaXacYI2M6NzFY7Ch4ukSzmPVW+OiJnp+iyg5FMWfJHQzAw6Ncyu8OEiXRERIankndVO0GZmVGXC/tmSBkbETEkDgTklY8o6IjOzelCFZxLeAoxK10cBN5d6gxO0mRlUepjdtcAjwGaSpks6Fvgx8HlJLwCfS7eLconDzIzK3kkYEUd08FKnpsJwgjYzo+fMZmdmVnecoM3McsqTJZmZ5ZR70GZmOZXD/OwEbWYG5DJDO0GbmeEatJlZbuVxwn4naDMzfJHQzCzH8pehnaDNzHAP2swst3KYn52gzczAPWgzs9xSDjO0E7SZGS5xmJnlVg470E7QZmbgOwnNzPIrf/nZCdrMDHyrt5lZbrnEYWaWU3m8SNhQ6wDMzKx97kGbmZHPHrQTtJkZrkGbmeWWR3GYmeWVE7SZWT65xGFmllN5vEjoYXZmZiQVjnKXkm1Je0t6TtKLkk7vakxO0GZmULEMLakRuAzYBxgKHCFpaFdCconDzAxoqFyNY3vgxYh4CUDSn4ADgac721BuE/QNX/tUDitCtSFpdESMqXUcli/+vaisPk3lXyWUNBoYXbBrTMG/xXrAawWvTQc+3ZWYXOKoD6NLH2I9kH8vaiQixkTEtgVLJn8onaDNzCrrdWBwwfagdF+nOUGbmVXWRGATSRtJ6g0cDtzSlYZyW4O2FbjOaO3x70UORcRSSScBdwGNwO8i4qmutKWIqGhwZmZWGS5xmJnllBO0mVlOuQZdI5JagCkFu0ZExLQOjn0nIvpVJTCrKUlrAxPSzXWBFmBuur19RCyuSWBWE65B10hnkq4TdM8k6RzgnYj434J9TRGxtHZRWTW5xJETkvpJmiDpcUlTJB3YzjEDJT0gabKkqZJ2SffvJemR9L3XS3Iy70Yk/V7SryU9Clwk6RxJpxa8PlXShun60ZL+mf6O/CadF8LqlBN07fRN/yeaLOlG4D3goIjYBtgd+Kn0ockBjgTuiojhwDBgsqSPAmcBn0vf+xhwStW+C6uWQcBOEdHhv62kTwCHATunvyMtwFHVCc+y4Bp07byb/k8EgKRewI8k7QosI7mfvxmYVfCeicDv0mNviojJkj5LMmPWw2k+7w08Up1vwaro+ohoKXHMnsCngInp70JfYE7WgVl2nKDz4yhgHeBTEbFE0jSgT+EBEfFAmsD3A34v6WJgHnBPRBxR7YCtqhYWrC9lxU+/rb8nAsZGxBlVi8oy5RJHfqwBzEmT8+7ABm0PkLQBMDsirgB+C2wD/APYWdKQ9JjVJG1axbit+qaR/NsjaRtgo3T/BGCkpAHpa2ulvzNWp9yDzo9xwK2SppDUkZ9t55jdgO9KWgK8AxwTEXMlfQW4VtIq6XFnAc9nH7LVyA3AMZKeAh4l/beOiKclnQXcLakBWAJ8E3ilZpHaSvEwOzOznHKJw8wsp5ygzcxyygnazCynnKDNzHLKCdrMLKecoM3McsoJ2swsp5ygzcxyygnazCynnKDNzHLKCdrMLKecoM3McsoJ2swsp5ygzcxyygnazCynnKDNzHLKCdpWIKklfdL4VEnXS1p1Jdr6vaSR6fpvJQ0tcuxuknbqwjmmpU82L9x3laQT2uwbIemOcmI1ywsnaGvr3YgYHhFbAouBrxe+KKlLj0mLiOMi4ukih+wGdDpBd+Ba4PA2+w5P95vVDSdoK+ZBYEjau31Q0i3A05IaJf1E0kRJT7b2VpW4VNJzkv4/MKC1IUn3S9o2Xd9b0uOS/iVpgqQNSf4QfDvtve8iaR1JN6TnmChp5/S9a0u6W9JTkn5L8iTrtiYAm0samL5nNeBzwE2Szk7bmyppjKQPvb+wVy5pW0n3t7Yj6XeS/inpCUkHpvu3SPdNTn8em1Tih2/mBG3tSnvK+wBT0l3bAN+KiE2BY4H5EbEdsB1wvKSNgIOAzYChwDG00yOWtA5wBXBwRAwDDomIacCvgZ+lvfcHgUvS7e2Ag0meYg7wA+ChiNgCuBFYv+05IqKF5MGqh6a7DgDuj4i3gUsjYrv0E0JfYP9O/FjOBO6NiO2B3YGfpMn/68AlETEc2BaY3ok2zTrkp3pbW30lTU7XHwSuJEm0/4yIl9P9ewFbFdRs1wA2AXYFrk0T5AxJ97bT/g7AA61tRcR/Oojjc8DQgg7u6pL6pef4Uvre8ZLmdfD+a4H/JUn0hwPXpPt3l3QasCqwFvAUcGsHbbS1F/BFSaem231I/kA8ApwpaRDw14h4ocz2zIpygra23k17gsulSXJh4S7g5Ii4q81x+1YwjgZgh4h4r51YyvF3YKCkYSR/YA6X1Af4FbBtRLwm6RySJNvWUj74dFn4ukh6/s+1Of4ZSY8C+wG3SzohItr742TWKS5xWFfcBZwoqReApE3Tj/oPAIelNeqBJGWAtv4B7JqWRJC0Vrp/AdC/4Li7gZNbNyQNT1cfAI5M9+0DrNlegBERwHXAWOCONNG3Jts30t54R6M2pgGfStcPbvN9n9xat5a0dfr148BLEfEL4GZgqw7aNesUJ2jrit8CTwOPS5oK/Ibk09iNwAvpa1eTfPRfQUTMBUYDf5X0L5IkCkmZ4aDWi4TAfwHbphfdnuaD0STnkiT4p0hKHa8WifNaYFj6lYh4i6T+PZUk2U7s4H3nApdIegxoKdj/Q6AX8GR6/h+m+w8FpqaloS3T791spSnpaJiZWd64B21mllNO0GZmOeUEbWaWU07QZmY55QRtZpZTTtBmZjnlBG1mllNO0GZmOfV/bvmI6ICdMYgAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "confusion = confusion_matrix(y_test, y_predication)\n",
    "\n",
    "ax = sns.heatmap(confusion, annot=True, cmap='Blues')\n",
    "ax.set_title(\"Confusion Matrix\")\n",
    "ax.set_xlabel('\\nPredicted Values')\n",
    "ax.set_ylabel('Actual Values ');\n",
    "ax.xaxis.set_ticklabels(['False','True'])\n",
    "ax.yaxis.set_ticklabels(['False','True'])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In conclusion we can see that our model is ~60% accurate, while this is quite low we don't have much data only around 400 rows. As we have used a reduced dataset, accuracy could be improved by simply increasing the amount of rows of data which we could get in the same format from the original (unreduced) dataset. We chose to use the reduced dataset as a large amount of data would take longer to train and for testing purposes a smaller amount makes it easier for experimenting with different hidden layer configurations. We could also change to use a different dataset which could include more attributes allowing a more realistic predication. For example other studies such as Beunza, J.J. et al used the \"Framingham Heart Study\" which had \"4000+\" observations and got an accuracy value of ~85% on some configurations. Sharma, V. et al who used the Cleveland dataset and used 14 attributes and 1025 instances got similar results depending on the configuration and algorithm used.\n",
    "\n",
    "There are many machine learning methods that can be used that may be better for accuracy other than a Artificial Neural Network. For example Decision Tree (DT) matches groups of data into a tree structure, Random Forest (RF) also creates a tree like structure. K-Nearest Neighbour (KNN) makes predictions based on neighbours. Lots of experiments by different people have been carried out to find out which one is most accurate. Ali's experiments found that they all produced higher than 80%. However, most of these algorithms (DT, RF) performed badly without a large dataset, apart from KNN which performed better with small amounts of data (Princy, R.J.P. et al) (Ali, M.M. et al).\n",
    "\n",
    "When using machine learning with medical data, there are many ethical and legal considerations. One of these is racial & gender injustice which is when datasets are biased to a certain race/group or gender which causing inaccurate predications, this could be prevented by ensuring that captured data is diverse to include everyone to prevent any bias. A legal issue is the collecting of personal medical data and using it for processing, this can be easily be prevented by ensuring everyone signs that they are willing to give their information away (Chen, I.Y. et al)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# References\n",
    "- Princy, R.J.P., Parthasarathy, S., Jose, P.S.H., Lakshminarayanan, A.R. and Jeganathan, S., 2020, May. Prediction of cardiac disease using supervised machine learning algorithms. In 2020 4th International Conference on Intelligent Computing and Control Systems (ICICCS) (pp. 570-575). IEEE.\n",
    "- Ali, M.M., Paul, B.K., Ahmed, K., Bui, F.M., Quinn, J.M. and Moni, M.A., 2021. Heart disease prediction using supervised machine learning algorithms: performance analysis and comparison. Computers in Biology and Medicine, 136, p.104672.\n",
    "- Beunza, J.J., Puertas, E., García-Ovejero, E., Villalba, G., Condes, E., Koleva, G., Hurtado, C. and Landecho, M.F., 2019. Comparison of machine learning algorithms for clinical event prediction (risk of coronary heart disease). Journal of biomedical informatics, 97, p.103257.\n",
    "- Sharma, V., Yadav, S. and Gupta, M., 2020, December. Heart disease prediction using machine learning techniques. In 2020 2nd International Conference on Advances in Computing, Communication Control and Networking (ICACCCN) (pp. 177-181). IEEE.\n",
    "- Chen, I.Y., Pierson, E., Rose, S., Joshi, S., Ferryman, K. and Ghassemi, M., 2021. Ethical machine learning in healthcare. Annual Review of Biomedical Data Science, 4, pp.123-144."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Appendices\n",
    "## Source Code\n",
    "- <https://github.com/enchanted-code/python-cnn>\n"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "85d0591f7722192431602c588f28923d555d94d033b5eac611b7a2ed79e64804"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}