DataCheckpoint_Group141_WI24.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# COGS 108 - Data Checkpoint"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Names\n",
    "\n",
    "- Shivangi Gupta\n",
    "- Joseph Hwang\n",
    "- Zijun Yang\n",
    "- Johnny Gonzales\n",
    "- Tanishq Rathore"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Research Question"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Utilizing clinical MRI Data and personal details of an individual, can we predict via machine learning model whether an individual will have an onset of Alzheimer's disease? Features the model will be trained on include variables such as Mini Mental State Examination (MMSE), visit number, Clinical Dementia Rating (CDR), gender, age, years of education, socioeconomic status, Estimated total intracranial volume (eTIV), Normalize Whole Brain Volume (nWBV), and Atlas Scaling Factor (ASF)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Background and Prior Work"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Advancements in healthcare, improvements in living conditions, and\n",
    "breakthroughs in medicine have collectively contributed to longer life\n",
    "expectancies worldwide; simultaneously, developed countries are also\n",
    "experiencing declining fertility\n",
    "rates.[<sup>1</sup>](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4255510/)\n",
    "The combination of these two circumstances has resulted in the\n",
    "proportion of older people within populations to steadily increase. The\n",
    "World Health Organization (WHO) reported that “in 2020, the number of\n",
    "people aged 60 and older outnumbered children younger than 5\n",
    "years”.[<sup>2</sup>](https://www.who.int/news-room/fact-sheets/detail/ageing-and-health)\n",
    "In addition, they also state that “between 2015 and 2050, the proportion\n",
    "of the world’s population over 60 years will nearly double from 12% to\n",
    "22%”. As a result, it is reasonable that we examine common health\n",
    "conditions associated with older age, one being Alzheimer’s disease.\n",
    "\n",
    "So what is Alzheimer’s disease? Alzheimer’s disease is a progressive\n",
    "neurodegenerative brain disorder that impairs memory and cognitive\n",
    "functions. It is the most common cause of dementia and affects about 6.5\n",
    "million people in the United States who are aged 65 and\n",
    "older.[<sup>3</sup>](https://www.mayoclinic.org/diseases-conditions/alzheimers-disease/symptoms-causes/syc-20350447)\n",
    "At the moment, there are no cures for the disease but medicines may\n",
    "improve or slow the progression of\n",
    "symptoms.[<sup>3</sup>](https://www.mayoclinic.org/diseases-conditions/alzheimers-disease/symptoms-causes/syc-20350447)\n",
    "As such, it is our project to create a model that is able to predict\n",
    "Alzheimer’s disease based on clinical data that include factors that\n",
    "show risk and progression of the disease.\n",
    "\n",
    "There are several other projects that have asked similar questions and\n",
    "approached similar problems for other diseases. For instance, one study\n",
    "tried to use machine learning methods to predict risk of cardiovascular\n",
    "disease based on major contributing\n",
    "factors.[<sup>4</sup>](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10036320/)\n",
    "Similarly, another paper used machine learning and ranker-based feature\n",
    "selection methods to predict eye diseases based on\n",
    "symptoms.[<sup>5</sup>](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9854513/)\n",
    "Lastly, there was a paper that predicted thyroid disease using selective\n",
    "features and machine learning\n",
    "techniques.[<sup>6</sup>](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9405591/)\n",
    "All three papers seem to have been relatively successful in predicting\n",
    "the disease based on distinct features. Evidently, training machine\n",
    "learning models based on datasets which contain factors and indicators\n",
    "for a given disease is not a novel format of question and method; we\n",
    "hope to achieve similarly for Alzheimer’s disease.\n",
    "\n",
    "<u>In-Depth Study Analysis</u>\n",
    "\n",
    "Our group analyzed two studies published in the National Institute of\n",
    "Health’s (NIH) journal database. The first study[<sup>7</sup>](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8927715/) employed\n",
    "machine learning models to predict early-stage Alzheimer's Disease using\n",
    "Open Access Series of Imaging Studies (OASIS) data, focusing on metrics\n",
    "like precision, recall, accuracy, and F1-score. The authors, with\n",
    "backgrounds in technology and health research, aimed to enhance early\n",
    "diagnosis, potentially lowering Alzheimer's mortality rates. The study\n",
    "demonstrated that machine learning techniques such as decision trees,\n",
    "random forests, SVM, gradient boosting, and voting classifiers can\n",
    "effectively predict early-stage Alzheimer's Disease with an accuracy of\n",
    "up to 83%. This achievement highlights the critical role of data science\n",
    "in identifying Alzheimer's at an early phase, leveraging feature\n",
    "selection and advanced algorithms to enhance diagnostic accuracy. Early\n",
    "detection is crucial for timely intervention, potentially mitigating the\n",
    "disease's progression and impact on patients and their families (Kavitha\n",
    "et al.).\n",
    "\n",
    "The second study[<sup>8</sup>](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6138240/) titled\n",
    "“Application of machine learning methods for diagnosis of dementia based\n",
    "on the 10/66 battery of cognitive function tests in south India”\n",
    "investigated the use of machine learning for diagnosing dementia in\n",
    "South India, employing the culturally and educationally fair 10/66\n",
    "battery of cognitive function tests designed for use in low and\n",
    "middle-income countries. Through the analysis of neuropsychological\n",
    "data, demographic information, and normative data, the research applied\n",
    "Jrip classification algorithm among others, achieving high diagnostic\n",
    "accuracy. This approach demonstrates the potential to streamline the\n",
    "diagnostic process, making it quicker and more accessible for clinicians\n",
    "and patients in India, thereby addressing the significant healthcare\n",
    "challenge of efficiently identifying dementia in community settings\n",
    "(Bhagyashree et al).\n",
    "\n",
    "<u>In-Depth Analysis of Similar Projects</u>\n",
    "\n",
    "Our group also delved into actual Kaggle projects that are directly\n",
    "associated with the dataset we’ve chosen to use, delving into EDA and\n",
    "prediction models using Scikit-Learn and Tensorflow. The first\n",
    "project[<sup>9</sup>](https://www.kaggle.com/code/shreyaspj/alzheimer-s-analysis-using-mri)\n",
    "I will discuss starts with an introduction to Alzheimer's disease and\n",
    "the problem statement of estimating the Clinical Dementia Rating (CDR)\n",
    "using MRI dataset features. It progresses through data loading and\n",
    "preprocessing, including null value handling and normalization, and\n",
    "employs machine learning techniques, specifically mentioning model\n",
    "training with hyperparameter tuning for XGBClassifier and\n",
    "GradientBoostingClassifier. The notebook concludes with predictions and\n",
    "performance evaluation, indicated by confusion matrix and classification\n",
    "report visualizations, and the model was able to reach a final accuracy\n",
    "of \\~80%.\n",
    "\n",
    "The second project[<sup>10</sup>](https://www.kaggle.com/code/andrew32bit/predict-alzheimer-disease-sl-and-tf) tried to predict the Clinical Rating of Alzheimer's disease (CRA) by\n",
    "integrating data loading, visualization, and extensive machine learning,\n",
    "including the use of TensorFlow for neural network models. It explored\n",
    "various machine learning models, with a special emphasis on model\n",
    "training and evaluation, culminating in the finding that the\n",
    "DecisionTreeClassifier performed the best among the models tested. The\n",
    "conclusion stressed the need for more data to enhance the precision of\n",
    "Alzheimer's disease predictions, highlighting the challenge of data\n",
    "scarcity in achieving accurate diagnostic models.\n",
    "\n",
    "**References**\n",
    "\n",
    "1.  [^](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4255510/) Nargund G. (2009) Declining birth rate in Developed Countries: A\n",
    "     radical policy re-think is required. *Facts, views & vision in\n",
    "     ObGyn, 1(3), 191–193.*\n",
    "     [<u>https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4255510/</u>](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4255510/)\n",
    "\n",
    "2.  [^](https://www.who.int/news-room/fact-sheets/detail/ageing-and-health) World Health Organization. (1 Oct 2022) Ageing and health. *World\n",
    "     Health Organization*.\n",
    "     [<u>https://www.who.int/news-room/fact-sheets/detail/ageing-and-health</u>](https://www.who.int/news-room/fact-sheets/detail/ageing-and-health)\n",
    "\n",
    "3.  [^](https://www.mayoclinic.org/diseases-conditions/alzheimers-disease/symptoms-causes/syc-20350447) Mayo Foundation for Medical Education and Research. (30\n",
    "     August 2023) Alzheimer’s disease. *Mayo Clinic*.\n",
    "     [<u>https://www.mayoclinic.org/diseases-conditions/alzheimers-disease/symptoms-causes/syc-20350447</u>](https://www.mayoclinic.org/diseases-conditions/alzheimers-disease/symptoms-causes/syc-20350447)\n",
    "\n",
    "4.  [^](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10036320/) Peng, M., Hou, F., Cheng, Z., Shen, T., Liu, K., Zhao, C., &\n",
    "     Zheng, W. (23 Mar 2023) Prediction of cardiovascular disease risk\n",
    "     based on major contributing features. *Scientific reports, 13(1),\n",
    "     4778*.\n",
    "     [<u>https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10036320/</u>](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10036320/)\n",
    "\n",
    "5.  [^](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9854513/) Marouf, A. A., Mottalib, M. M., Alhajj, R., Rokne, J., &\n",
    "     Jafarullah, O. (24 Dec 2022) An Efficient Approach to Predict Eye\n",
    "     Diseases from Symptoms Using Machine Learning and Ranker-Based\n",
    "     Feature Selection Methods. *Bioengineering (Basel, Switzerland),\n",
    "     10(1), 25*.\n",
    "     [<u>https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9854513/</u>](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9854513/)\n",
    "\n",
    "6.  [^](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9405591/) Chaganti, R., Rustam, F., De La Torre Díez, I., Mazón, J. L. V.,\n",
    "     Rodríguez, C. L., & Ashraf, I. (13 Aug 2022). Thyroid Disease\n",
    "     Prediction Using Selective Features and Machine Learning\n",
    "     Techniques. *Cancers, 14(16), 3914*.\n",
    "     [<u>https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9405591/</u>](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9405591/)\n",
    "\n",
    "7.  [^](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8927715/) Kavitha C, Mani V, Srividhya SR, Khalaf OI, Tavera Romero CA.\n",
    "     Early-Stage Alzheimer's Disease Prediction Using Machine Learning\n",
    "     Models. Front Public Health. 2022 Mar 3;10:853294. doi:\n",
    "     10.3389/fpubh.2022.853294. PMID: 35309200; PMCID: PMC8927715.\n",
    "     [<u>https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8927715/</u>](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8927715/)\n",
    "\n",
    "8.  [^](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6138240/) Bhagyashree SIR, Nagaraj K, Prince M, Fall CHD, Krishna M.\n",
    "     Diagnosis of Dementia by Machine learning methods in\n",
    "     Epidemiological studies: a pilot exploratory study from south\n",
    "     India. Soc Psychiatry Psychiatr Epidemiol. 2018 Jan;53(1):77-86.\n",
    "     doi: 10.1007/s00127-017-1410-0. Epub 2017 Jul 11. PMID: 28698926;\n",
    "     PMCID: PMC6138240.\n",
    "     [<u>https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6138240/</u>](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6138240/)\n",
    "\n",
    "9.  [^](https://www.kaggle.com/code/shreyaspj/alzheimer-s-analysis-using-mri) Reddy, Shreyas. 2021. Alzheimer's analysis using MRI, February\n",
    "     8, 2024.\n",
    "     [<u>https://www.kaggle.com/code/shreyaspj/alzheimer-s-analysis-using-mri</u>](https://www.kaggle.com/code/shreyaspj/alzheimer-s-analysis-using-mri)\n",
    "\n",
    "10. [^](https://www.kaggle.com/code/andrew32bit/predict-alzheimer-disease-sl-and-tf) Andrew. 2017. Predict alzheimer disease sl and tf, February 8, 2024.\n",
    "     [<u>https://www.kaggle.com/code/andrew32bit/predict-alzheimer-disease-sl-and-tf</u>](https://www.kaggle.com/code/andrew32bit/predict-alzheimer-disease-sl-and-tf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Hypothesis\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our project's hypothesis is the following: \"It is possible to predict the onset of Alzheimers based on the combination of (1) clinical data* and (2) personal features such as gender, age, years of education, and socioeconomic status.\" We believe that we will be able to successfully train a model that is able to predict the onset of Alzheimer's disease because, as mentioned in the background portion of the proposal, there has been numerous successful machine learning models trained on clinical data to predict the onset of a disease. The clinical data provide variables that capture both the cognitive and structural changes associated with the disease's progression. Incorporating personal features such as gender and age in the prediction model is justified by extensive research indicating that these factors can influence the risk and progression rate of Alzheimer's disease. Together, we believe that this will be enough data to train a model to predict the onset of Alzheimer's disease.\n",
    "\n",
    "*Clinical data include Mini Mental State Examination (MMSE), visit number, Clinical Dementia Rating (CDR), Estimated total intracranial volume (eTIV), Normalize Whole Brain Volume (nWBV), and the Atlas Scaling Factor (ASF)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data overview\n",
    "\n",
    "- Dataset #1\n",
    "  - Dataset Name: OASIS-2: Longitudinal MRI Data in Nondemented and Demented Older Adults\n",
    "  - Link to the dataset:https://www.oasis-brains.org/#data\n",
    "  - Number of observations: 373\n",
    "  - Number of variables: 15\n",
    "\n",
    "The dataset provided by The Open Access Series of Imaging Studies (OASIS) contains a collection of 150 subjects aged 60 to 96. Each subject was scanned on 2 or more visits, separated by one year for a total of 373 imaging sessions. All subjects were right-handed and included both men and women. 72 subjects were charaterized as nondemented throughout the study. 64 subjects subjects were characterized as demented at their initial vists and remained so for subsequent scans. Lastly, 14 subjects were characterized as nondemented at their initial vist but were subsequently characterized as demented in later visits. Important variables are years of education (EDUC), socioeconomic status (SES; 1 to 5), mini mental state examination (MMSE score), clinical dementia rating (CDR rating scale), estimated total intracranial volume (eTIV), normalize whole brain volume(nWBV), and atlas scaling factor (ASF). \n",
    "\n",
    "The variables may be proxies for Alzheimers/cognitive state (conjecture):\n",
    "- __EDUC__: Represents years of education which may proxy for cognitive reserve. Higher education levels may result in better cognitive function and reduced risk of dementia. \n",
    "- __SES__: Ranging from 1 to 5, is a proxy for income, education level, and occupation, which represents environmental influences on cognitive health. \n",
    "- __MMSE__: A widely used screening tool for cognitive impairment. Scores range from 1 to 30 with higher scores indicating better cognitive function. The examination assesses domains such as orientation, memory, attention, and language. \n",
    "- __CDR__: CDR scale is also commonly used to assess severity of dementia symptoms. The scale ranges from 0 to 3 where 0 indicates no dementia, 0.5 is questionable dementia, 1 is mild dementia, 2 is moderate dementia, and 3 is severe dementia.\n",
    "- __eTIV__: Indicates the total volume inside the skull (including brain tissue, cerebrospinal fluid, etc.). It is measured in cubic centimeters (cc) and can help establish baseline brain size. Decrease/varying sizes may proxy for cognitive function and Alzheimer's risk.\n",
    "- __nWBV__: Represents the volume of the brain normalized to the subject's eTIV. It is expressed as a percentage, reflecting the proportion of the brain occupying the eTIV (nWBV = brain volume / eTIV). Changes in nWBV may be indicative of brain atrophy which is a common feature of Alzheimer's and dementia.\n",
    "- __ASF__: A scaling factor used in brain imaging to adjust for individual differences in brain size and shape. It accounts for variability in brain morphology allowing for accurate comparisons of brain structures.\n",
    "\n",
    "The dataset provided by OASIS is relatively clean and organized. However, some cleaning may be required, such as changing M/F and group to numeric values, dropping columns such as MRI ID, Visit, and Hand which will not be used in our predictions. All of this will be done by manipulating the dataframe. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset #1: oasis_longitudinal.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "%config InlineBackend.figure_format ='retina'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Subject ID</th>\n",
       "      <th>MRI ID</th>\n",
       "      <th>Group</th>\n",
       "      <th>Visit</th>\n",
       "      <th>MR Delay</th>\n",
       "      <th>M/F</th>\n",
       "      <th>Hand</th>\n",
       "      <th>Age</th>\n",
       "      <th>EDUC</th>\n",
       "      <th>SES</th>\n",
       "      <th>MMSE</th>\n",
       "      <th>CDR</th>\n",
       "      <th>eTIV</th>\n",
       "      <th>nWBV</th>\n",
       "      <th>ASF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>OAS2_0001</td>\n",
       "      <td>OAS2_0001_MR1</td>\n",
       "      <td>Nondemented</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>M</td>\n",
       "      <td>R</td>\n",
       "      <td>87</td>\n",
       "      <td>14</td>\n",
       "      <td>2.0</td>\n",
       "      <td>27.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1987</td>\n",
       "      <td>0.696</td>\n",
       "      <td>0.883</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>OAS2_0001</td>\n",
       "      <td>OAS2_0001_MR2</td>\n",
       "      <td>Nondemented</td>\n",
       "      <td>2</td>\n",
       "      <td>457</td>\n",
       "      <td>M</td>\n",
       "      <td>R</td>\n",
       "      <td>88</td>\n",
       "      <td>14</td>\n",
       "      <td>2.0</td>\n",
       "      <td>30.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2004</td>\n",
       "      <td>0.681</td>\n",
       "      <td>0.876</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>OAS2_0002</td>\n",
       "      <td>OAS2_0002_MR1</td>\n",
       "      <td>Demented</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>M</td>\n",
       "      <td>R</td>\n",
       "      <td>75</td>\n",
       "      <td>12</td>\n",
       "      <td>NaN</td>\n",
       "      <td>23.0</td>\n",
       "      <td>0.5</td>\n",
       "      <td>1678</td>\n",
       "      <td>0.736</td>\n",
       "      <td>1.046</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>OAS2_0002</td>\n",
       "      <td>OAS2_0002_MR2</td>\n",
       "      <td>Demented</td>\n",
       "      <td>2</td>\n",
       "      <td>560</td>\n",
       "      <td>M</td>\n",
       "      <td>R</td>\n",
       "      <td>76</td>\n",
       "      <td>12</td>\n",
       "      <td>NaN</td>\n",
       "      <td>28.0</td>\n",
       "      <td>0.5</td>\n",
       "      <td>1738</td>\n",
       "      <td>0.713</td>\n",
       "      <td>1.010</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>OAS2_0002</td>\n",
       "      <td>OAS2_0002_MR3</td>\n",
       "      <td>Demented</td>\n",
       "      <td>3</td>\n",
       "      <td>1895</td>\n",
       "      <td>M</td>\n",
       "      <td>R</td>\n",
       "      <td>80</td>\n",
       "      <td>12</td>\n",
       "      <td>NaN</td>\n",
       "      <td>22.0</td>\n",
       "      <td>0.5</td>\n",
       "      <td>1698</td>\n",
       "      <td>0.701</td>\n",
       "      <td>1.034</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>368</th>\n",
       "      <td>OAS2_0185</td>\n",
       "      <td>OAS2_0185_MR2</td>\n",
       "      <td>Demented</td>\n",
       "      <td>2</td>\n",
       "      <td>842</td>\n",
       "      <td>M</td>\n",
       "      <td>R</td>\n",
       "      <td>82</td>\n",
       "      <td>16</td>\n",
       "      <td>1.0</td>\n",
       "      <td>28.0</td>\n",
       "      <td>0.5</td>\n",
       "      <td>1693</td>\n",
       "      <td>0.694</td>\n",
       "      <td>1.037</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>369</th>\n",
       "      <td>OAS2_0185</td>\n",
       "      <td>OAS2_0185_MR3</td>\n",
       "      <td>Demented</td>\n",
       "      <td>3</td>\n",
       "      <td>2297</td>\n",
       "      <td>M</td>\n",
       "      <td>R</td>\n",
       "      <td>86</td>\n",
       "      <td>16</td>\n",
       "      <td>1.0</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0.5</td>\n",
       "      <td>1688</td>\n",
       "      <td>0.675</td>\n",
       "      <td>1.040</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>370</th>\n",
       "      <td>OAS2_0186</td>\n",
       "      <td>OAS2_0186_MR1</td>\n",
       "      <td>Nondemented</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>F</td>\n",
       "      <td>R</td>\n",
       "      <td>61</td>\n",
       "      <td>13</td>\n",
       "      <td>2.0</td>\n",
       "      <td>30.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1319</td>\n",
       "      <td>0.801</td>\n",
       "      <td>1.331</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>371</th>\n",
       "      <td>OAS2_0186</td>\n",
       "      <td>OAS2_0186_MR2</td>\n",
       "      <td>Nondemented</td>\n",
       "      <td>2</td>\n",
       "      <td>763</td>\n",
       "      <td>F</td>\n",
       "      <td>R</td>\n",
       "      <td>63</td>\n",
       "      <td>13</td>\n",
       "      <td>2.0</td>\n",
       "      <td>30.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1327</td>\n",
       "      <td>0.796</td>\n",
       "      <td>1.323</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>372</th>\n",
       "      <td>OAS2_0186</td>\n",
       "      <td>OAS2_0186_MR3</td>\n",
       "      <td>Nondemented</td>\n",
       "      <td>3</td>\n",
       "      <td>1608</td>\n",
       "      <td>F</td>\n",
       "      <td>R</td>\n",
       "      <td>65</td>\n",
       "      <td>13</td>\n",
       "      <td>2.0</td>\n",
       "      <td>30.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1333</td>\n",
       "      <td>0.801</td>\n",
       "      <td>1.317</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>373 rows × 15 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    Subject ID         MRI ID        Group  Visit  MR Delay M/F Hand  Age  \\\n",
       "0    OAS2_0001  OAS2_0001_MR1  Nondemented      1         0   M    R   87   \n",
       "1    OAS2_0001  OAS2_0001_MR2  Nondemented      2       457   M    R   88   \n",
       "2    OAS2_0002  OAS2_0002_MR1     Demented      1         0   M    R   75   \n",
       "3    OAS2_0002  OAS2_0002_MR2     Demented      2       560   M    R   76   \n",
       "4    OAS2_0002  OAS2_0002_MR3     Demented      3      1895   M    R   80   \n",
       "..         ...            ...          ...    ...       ...  ..  ...  ...   \n",
       "368  OAS2_0185  OAS2_0185_MR2     Demented      2       842   M    R   82   \n",
       "369  OAS2_0185  OAS2_0185_MR3     Demented      3      2297   M    R   86   \n",
       "370  OAS2_0186  OAS2_0186_MR1  Nondemented      1         0   F    R   61   \n",
       "371  OAS2_0186  OAS2_0186_MR2  Nondemented      2       763   F    R   63   \n",
       "372  OAS2_0186  OAS2_0186_MR3  Nondemented      3      1608   F    R   65   \n",
       "\n",
       "     EDUC  SES  MMSE  CDR  eTIV   nWBV    ASF  \n",
       "0      14  2.0  27.0  0.0  1987  0.696  0.883  \n",
       "1      14  2.0  30.0  0.0  2004  0.681  0.876  \n",
       "2      12  NaN  23.0  0.5  1678  0.736  1.046  \n",
       "3      12  NaN  28.0  0.5  1738  0.713  1.010  \n",
       "4      12  NaN  22.0  0.5  1698  0.701  1.034  \n",
       "..    ...  ...   ...  ...   ...    ...    ...  \n",
       "368    16  1.0  28.0  0.5  1693  0.694  1.037  \n",
       "369    16  1.0  26.0  0.5  1688  0.675  1.040  \n",
       "370    13  2.0  30.0  0.0  1319  0.801  1.331  \n",
       "371    13  2.0  30.0  0.0  1327  0.796  1.323  \n",
       "372    13  2.0  30.0  0.0  1333  0.801  1.317  \n",
       "\n",
       "[373 rows x 15 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('oasis_longitudinal.csv')\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(373, 15)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 373 entries, 0 to 372\n",
      "Data columns (total 15 columns):\n",
      " #   Column      Non-Null Count  Dtype  \n",
      "---  ------      --------------  -----  \n",
      " 0   Subject ID  373 non-null    object \n",
      " 1   MRI ID      373 non-null    object \n",
      " 2   Group       373 non-null    object \n",
      " 3   Visit       373 non-null    int64  \n",
      " 4   MR Delay    373 non-null    int64  \n",
      " 5   M/F         373 non-null    object \n",
      " 6   Hand        373 non-null    object \n",
      " 7   Age         373 non-null    int64  \n",
      " 8   EDUC        373 non-null    int64  \n",
      " 9   SES         354 non-null    float64\n",
      " 10  MMSE        371 non-null    float64\n",
      " 11  CDR         373 non-null    float64\n",
      " 12  eTIV        373 non-null    int64  \n",
      " 13  nWBV        373 non-null    float64\n",
      " 14  ASF         373 non-null    float64\n",
      "dtypes: float64(5), int64(5), object(5)\n",
      "memory usage: 43.8+ KB\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "print(df.info())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAwkAAAILCAYAAACjJNAzAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAABYlAAAWJQFJUiTwAAAnwUlEQVR4nO3de5RtV10n+u+PRJIQzIk8FFrBJFzyuGALCfI4aBKCcgMoIIRLfCDSEBFRRKBbBg8JNN4Ljchb4GJjQGwTiZcgGANKEg8kIE2CptFAgOSAtjwMBxIgD5Iw+4+9qqlZqTpVdc6qvevU/nzG2GPWXmuuteaaYyanvjXXo1prAQAAWHCbWTcAAADYXIQEAACgIyQAAAAdIQEAAOgICQAAQEdIAAAAOkICAADQERIAAICOkAAAAHSEBAAAoCMkAAAAHSEBAADo7D/rBsyjqroqySFJds64KQAAbF2HJbm2tXb4ejcUEmbjkIMOOugOxxxzzB1m3RAAALamyy+/PNdff/0ebSskzMbOY4455g6XXHLJrNsBAMAWddxxx+XSSy/duSfbuicBAADoCAkAAEBHSAAAADpCAgAA0BESAACAjpAAAAB0hAQAAKAjJAAAAB0hAQAA6AgJAABAR0gAAAA6QgIAANAREgAAgI6QAAAAdIQEAACgIyQAAAAdIQEAAOjsP+sGAOOqqlk3YU1aa7NuAgCwAjMJAABAx0wCbFFPeMvFs27Css562vZZNwEAWIWZBAAAoCMkAAAAHSEBAADoCAkAAEBHSAAAADr7XEioqlOq6vVV9aGquraqWlW9c5VttlfVuVW1q6quq6rLqupZVbXfbrZ5UlV9rKq+WVXXVNWFVfXT458RAABsLvtcSEjywiS/nuQ+Sf7napWr6tFJdiQ5Psm7k7wxyW2TvDrJmSts83tJzkhy1yRvTfLOJD+S5L1V9et7ewIAALCZ7Ysh4beSHJnkkCRP313Fqjokk1/yb0lyYmvtKa21/5hJwPhIklOq6tQl22xP8pwkn0vy71trv9Vae0aS45LsSvJ7VXXYqGcEAACbyD4XElprF7TWPtNaa2uofkqSOyc5s7X28UX7uCGTGYnk1kHjV4fyd1trX1u0zc5MZiEOSPLkPWw+AABselv9jcsnDeV5y6zbkeS6JNur6oDW2o1r2OavkrxoqPPi1Q5eVZessOro1bYFAIBZ2edmEtbpqKG8YumK1trNSa7KJCgdkSRVdXCSH0zyzdbaF5fZ32eG8sjxmwoAAJvDVp9J2DaU16ywfmH5oXtYf7daa8ctt3yYYTh2LfsAAIBp2+ozCaupoVzL/Q2Lrbc+AADsM7Z6SFj4y/+2FdYfsqTeavVXm2kAAIB93lYPCZ8eylvdQ1BV+yc5PMnNSa5MktbatzJ598Ltq+quy+zvnkN5q3scAABgq9jqIeH8oTx5mXXHJ7ldkosXPdlotW0evqQOAABsOVs9JJyd5Ookp1bV/RYWVtWBSV42fH3Tkm3ePJQvqKrvW7TNYUmekeTGJH+0UQ0GAIBZ2+eeblRVj0nymOHrXYbyQVV1xvDz1a215yZJa+3aqjotk7BwYVWdmclbkx+VyeNRz05y1uL9t9YurqrfT/LsJJdV1dlJbpvkCUnukOQ3hherAQDAlrTPhYQk90nypCXLjhg+SfL5JM9dWNFaO6eqTkjygiSPS3Jgks9mEgJet9ybm1trz6mqy5L8epJfSfKdJJcmeWVr7X2jng0AAGwy+1xIaK2dnuT0dW5zUZJHrHObtyd5+3q2AQCArWCr35MAAACsk5AAAAB0hAQAAKAjJAAAAB0hAQAA6AgJAABAR0gAAAA6QgIAANAREgAAgI6QAAAAdIQEAACgIyQAAAAdIQEAAOgICQAAQEdIAAAAOkICAADQERIAAICOkAAAAHSEBAAAoCMkAAAAHSEBAADoCAkAAEBHSAAAADpCAgAA0BESAACAjpAAAAB0hAQAAKAjJAAAAB0hAQAA6AgJAABAR0gAAAA6QgIAANAREgAAgI6QAAAAdIQEAACgIyQAAAAdIQEAAOgICQAAQGf/WTeA6amqWTdhzVprs24CAMDcMpMAAAB0zCTMoSe85eJZN2FFZz1t+6ybAAAw98wkAAAAHSEBAADoCAkAAEBHSAAAADpCAgAA0BESAACAjpAAAAB0hAQAAKAjJAAAAB0hAQAA6AgJAABAR0gAAAA6QgIAANAREgAAgI6QAAAAdIQEAACgIyQAAAAdIQEAAOgICQAAQEdIAAAAOkICAADQ2X/WDQBg66mqWTdhzVprs24CwKZjJgEAAOiYSQBgwzzhLRfPugkrOutp22fdBIBNa25mEqrqkVX1gar6l6q6vqqurKp3VdWDVqi/varOrapdVXVdVV1WVc+qqv2m3XYAAJimuQgJVfWKJO9LcmyS85K8NsmlSR6d5KKq+sUl9R+dZEeS45O8O8kbk9w2yauTnDm9lgMAwPRt+cuNquouSZ6b5MtJ/n1r7SuL1j0kyflJXprkncOyQ5K8NcktSU5srX18WP6ioe4pVXVqa01YAABgS5qHmYQfzuQ8/25xQEiS1toFSb6R5M6LFp8yfD9zISAMdW9I8sLh69M3tMUAADBD8xASPpPk20nuX1V3Wryiqo5P8r1J/mbR4pOG8rxl9rUjyXVJtlfVARvQVgAAmLktf7lRa21XVf12kt9P8k9VdU6Srya5R5JHJfnrJE9btMlRQ3nFMvu6uaquSnKvJEckuXx3x66qS1ZYdfR6zgEAAKZpy4eEJGmtvaaqdiZ5W5LTFq36bJIzllyGtG0or1lhdwvLDx2zjQAAsFnMw+VGqar/lOTsJGdkMoNwcJLjklyZ5E+q6r+sZ3dDueorOltrxy33SfKpdZ0AAABM0ZYPCVV1YpJXJPmL1tqzW2tXttaua61dmuRnk/zPJM+pqiOGTRZmCrbdamcThyypBwAAW8qWDwlJfnooL1i6orV2XZKPZdIP9x0Wf3ooj1xav6r2T3J4kpszmYUAAIAtZx5CwsJTiO68wvqF5d8eyvOH8uRl6h6f5HZJLm6t3ThO8wAAYHOZh5DwoaH8lar6wcUrqurhSR6c5IYkFw+Lz05ydZJTq+p+i+oemORlw9c3bWiLAQBghubh6UZnZ/IehJ9McnlVvTvJl5Ick8mlSJXkea21ryZJa+3aqjpt2O7Cqjozya5MHpd61LD8rKmfBQAATMmWDwmtte9U1SOSPCPJqZncrHy7TH7xPzfJ61prH1iyzTlVdUKSFyR5XJIDM3lc6rOH+qs+2QgAAPZVWz4kJElr7aYkrxk+a93moiSP2KAmAQDApjUP9yQAAADrICQAAAAdIQEAAOgICQAAQEdIAAAAOkICAADQERIAAICOkAAAAHSEBAAAoCMkAAAAHSEBAADoCAkAAEBHSAAAADpCAgAA0Nl/1g0AANgIVTXrJqxZa23WTYCOmQQAAKBjJgEA2NKe8JaLZ92EFZ31tO2zbgIsy0wCAADQERIAAICOkAAAAHSEBAAAoCMkAAAAHSEBAADoCAkAAEBHSAAAADpCAgAA0BESAACAjpAAAAB0hAQAAKAjJAAAAB0hAQAA6AgJAABAR0gAAAA6QgIAANAREgAAgI6QAAAAdIQEAACgIyQAAAAdIQEAAOgICQAAQEdIAAAAOkICAADQERIAAICOkAAAAHSEBAAAoCMkAAAAHSEBAADoCAkAAEBHSAAAADpCAgAA0BESAACAjpAAAAB0hAQAAKAjJAAAAB0hAQAA6AgJAABAR0gAAAA6QgIAANAREgAAgI6QAAAAdIQEAACgIyQAAAAdIQEAAOgICQAAQEdIAAAAOkICAADQmauQUFU/UVV/XlVfrKobh/IDVfWIZepur6pzq2pXVV1XVZdV1bOqar9ZtB0AAKZl/1k3YFqq6oVJ/nOSq5O8L8kXk9wpyX2TnJjk3EV1H53kz5PckOSsJLuS/EySVyd5cJLHT7HpAAAwVXMREqrq8ZkEhL9J8tjW2jeWrP+eRT8fkuStSW5JcmJr7ePD8hclOT/JKVV1amvtzGm1HwAApmnUy42q6u7DL9m7q/O9VXX3MY+7yvFuk+QVSa5L8vNLA0KStNZuWvT1lCR3TnLmQkAY6tyQ5IXD16dvXIsBAGC2xp5JuCrJ6Zn81X4lz0zy0iTTurZ/e5LDk5yd5GtV9cgk987kUqKPtdY+sqT+SUN53jL72pFJ2NheVQe01m7c3YGr6pIVVh291sYDAMC0jR0SavhsJj82lF9OcmmSH1m8sqp2JDmltfZvw6KjhvKKpTtqrd1cVVcluVeSI5JcviEtBgCAGZrFPQk/kORbUzze9w/lr2Yy0/GTSf4uyQ8neVWS/yvJuzK5eTlJtg3lNSvsb2H5oasduLV23HLLhxmGY1fbHgAAZmGvQ0JV/dKSRfdZZlkyubzo7kmemOR/7O1x12HhsqbKZMbgH4bv/1hVP5vJjMEJVfWgZS49Ws7CTEkbuZ0AALApjDGTcEa++wtzS/Lo4bPUwi/X1yV5yQjHXauvDeWViwJCkqS1dn1VvT/JU5LcP8lH8t2Zgm1Z3sKN2SvNNAAAwD5tjJDw5KGsJG9Lck6S9yxT75YkX03ykdba10c47lp9eihXOuZCiDhoUf37JTkySXfjcVXtn8lN0DcnuXLUVgIAwCax1yGhtfb2hZ+r6klJzmmtvWNv9zuiHZn8Un/Pqrpta+3bS9bfeyh3DuX5SX4hyclJ/nRJ3eOT3C7JjtWebAQAAPuqUd+T0Fp7yCYLCGmtXZ3JW5O3Jfmdxeuq6qcyuXH5mnz3kadnZ/JW5lOr6n6L6h6Y5GXD1zdtcLMBAGBm5uKNy0meneQBSV5QVccn+VgmTzf62Uwugzpt4RKo1tq1VXVaJmHhwqo6M8muJI/K5PGoZ2cSOgAAYEsadSYhSarqhKp6X1V9papuqqpblvncPPZxd6e19pVMQsKrk9wtkxe6nZTkL5P8RGvtXUvqn5PkhEwuVXpckt9IclMmYePU1ponGwEAsGWNOpMwvM34nEweO/qFTG4CnmogWElrbVcmv+Q/e431L0ryiA1tFAAAbEJjX250eiZ/cX9ka+0DI+8bAACYgrEvN7p3krMEBAAA2HeNHRK+mclNvgAAwD5q7JDwwSQPGnmfAADAFI0dEn47yT2q6oVVVSPvGwAAmIKxb1x+cZJ/TPKSJP+hqv4+ydeXqddaa08Z+dgAAMAIxg4Jv7zo58OGz3JaEiEBAAA2obFDwuEj7w8AAJiyUUNCa+3zY+4PAACYvrFvXAYAAPZxo84kVNXd11q3tfaFMY8NAACMY+x7EnZmclPyatoGHBsAABjB2L+ovyPLh4RDk9wnyQ8nuTCJexcAAGCTGvvG5V9eaV1V3SbJi5L8apInjXlcAABgPFO7cbm19p3W2ksyuSTp5dM6LgAAsD6zeLrRxUkeNoPjAgAAazCLkHCHJAfP4LgAAMAaTDUkVNVPJnlCkk9O87gAAMDajf2ehPN3c5y7JVl4j8JLxzwuAAAwnrEfgXriCstbkq8leX+S32utrRQmAACAGRv7EaizuMcBAAAYkV/qAQCAztiXG3Wq6pAk25Jc01q7diOPBQAAjGP0mYSq2q+qnldVn83kPoSdSb5WVZ8dlm9oMAEAAPbO2E83um2S85KckMnNyv+c5ItJ7prksCS/m+TkqnpYa+3bYx4bAAAYx9gzCc/O5AlHf5nkmNbaYa21B7XWDktyVJL3JvmJoR4AALAJjR0Sfj6TF6U9prX2mcUrWmufS/LYJP+Y5BdGPi4AADCSsUPC/5Hkr1pr31lu5bD8r5LcY+TjAgAAIxk7JHw7ye1XqXNwkptGPi4AADCSsUPCZUlOqao7L7eyqu6U5JQk/zDycQEAgJGMHRLekOTOST5WVU+pqiOq6qCqOryqnpzk74b1bxj5uAAAwEhGfQRqa+3Pquo+SZ6X5P9bpkol+S+ttT8b87gAAMB4Rn+xWWvt+VX1F0mekuS+Gd64nOQTSd7WWvvI2McEAADGsyFvP26tfTTJRzdi3wAAwMba63sSquqAqvpYVX2wqr5nN/VuO9T56O7qAQAAszXGjcu/kOS4JK9qra34aNPW2reTvDLJ/eNlagAAsGmNERIem+TK1tq5q1VsrZ2X5DNJHj/CcQEAgA0wRki4b5IL11F/R5L7jHBcAABgA4wREu6U5MvrqP/lJHcc4bgAAMAGGCMkXJ/k9uuof/skN4xwXAAAYAOMERL+OcmPraP+/ZJ8YYTjAgAAG2CMkHBhkgdW1f1Wq1hVxyXZnuSCEY4LAABsgDFCwhuStCTvqqpjVqpUVUcneVeSW5L8wQjHBQAANsBev3G5tfbpqnppktOTfKKqzk5yfpJ/ySQ8/FCShyZ5XJIDkvxOa+3Te3tcAABgY+x1SEiS1tpLq+rmJC9O8vNJfm5JlUpyU5IXtNb+3zGOCQAAbIxRQkKStNb+n6r6kyT/IcmDk9w1k3Dwr0k+nOSPWmufH+t4AADAxhgtJCTJEAJePOY+AQCA6RrjxmUAAGALERIAAICOkAAAAHSEBAAAoCMkAAAAHSEBAADoCAkAAEBHSAAAADpCAgAA0BESAACAjpAAAAB0hAQAAKAjJAAAAB0hAQAA6AgJAABAR0gAAAA6QgIAANAREgAAgM5choSqemJVteHz1BXqbK+qc6tqV1VdV1WXVdWzqmq/abcXAACmae5CQlXdLcnrk3xzN3UenWRHkuOTvDvJG5PcNsmrk5w5hWYCAMDMzFVIqKpK8kdJvprkzSvUOSTJW5PckuTE1tpTWmv/Mcl9knwkySlVdep0WgwAANM3VyEhyTOTnJTkyUm+tUKdU5LcOcmZrbWPLyxsrd2Q5IXD16dvZCMBAGCW5iYkVNUxSV6e5LWttR27qXrSUJ63zLodSa5Lsr2qDhi5iQAAsCnsP+sGTENV7Z/kj5N8IcnzV6l+1FBesXRFa+3mqroqyb2SHJHk8lWOe8kKq45epQ0AADAzcxESkvxOkvsm+fHW2vWr1N02lNessH5h+aEjtAsAADadLR8Squr+mcwevKq19pExdjmUbbWKrbXjVmjTJUmOHaEtAAAwui19T8Kiy4yuSPKiNW62MFOwbYX1hyypBwAAW8qWDglJbp/kyCTHJLlh0QvUWpIXD3XeOix7zfD900N55NKdDaHj8CQ3J7lyQ1sOAAAzstUvN7oxyX9dYd2xmdyn8OFMgsHCpUjnJ/mFJCcn+dMl2xyf5HZJdrTWbhy9tQAAsAls6ZAw3KT81OXWVdXpmYSEt7fW/nDRqrOTvCLJqVX1+oV3JVTVgUleNtR504Y1GgAAZmxLh4Q90Vq7tqpOyyQsXFhVZybZleRRmTwe9ewkZ82wiQAAsKG2+j0Je6S1dk6SEzJ5edrjkvxGkpuSPDvJqa21VZ9sBAAA+6q5nUlorZ2e5PTdrL8oySOm1R4AANgszCQAAAAdIQEAAOgICQAAQEdIAAAAOkICAADQERIAAICOkAAAAHSEBAAAoCMkAAAAHSEBAADoCAkAAEBHSAAAADpCAgAA0BESAACAjpAAAAB0hAQAAKAjJAAAAB0hAQAA6AgJAABAR0gAAAA6QgIAANAREgAAgI6QAAAAdIQEAACgIyQAAAAdIQEAAOgICQAAQEdIAAAAOkICAADQERIAAICOkAAAAHSEBAAAoCMkAAAAHSEBAADoCAkAAEBHSAAAADpCAgAA0BESAACAjpAAAAB0hAQAAKAjJAAAAB0hAQAA6AgJAABAR0gAAAA6QgIAANAREgAAgI6QAAAAdIQEAACgIyQAAAAdIQEAAOgICQAAQEdIAAAAOkICAADQERIAAICOkAAAAHSEBAAAoCMkAAAAHSEBAADoCAkAAEBHSAAAADpCAgAA0BESAACAjpAAAAB0hAQAAKAjJAAAAJ0tHxKq6o5V9dSqendVfbaqrq+qa6rqw1X1lKpatg+qantVnVtVu6rquqq6rKqeVVX7TfscAABgmvafdQOm4PFJ3pTki0kuSPKFJD+Q5LFJ/jDJw6vq8a21trBBVT06yZ8nuSHJWUl2JfmZJK9O8uBhnwAAsCXNQ0i4Ismjkvxla+07Cwur6vlJPpbkcZkEhj8flh+S5K1JbklyYmvt48PyFyU5P8kpVXVqa+3MqZ4FAABMyZa/3Ki1dn5r7b2LA8Kw/EtJ3jx8PXHRqlOS3DnJmQsBYah/Q5IXDl+fvnEtBgCA2dryIWEVNw3lzYuWnTSU5y1Tf0eS65Jsr6oDNrJhAAAwK/NwudGyqmr/JL80fF0cCI4ayiuWbtNau7mqrkpyryRHJLl8lWNcssKqo9fXWgAAmJ55nkl4eZJ7Jzm3tfb+Rcu3DeU1K2y3sPzQDWoXAADM1FzOJFTVM5M8J8mnkjxxvZsPZdttrSStteNWOP4lSY5d53EBAGAq5m4moaqekeS1Sf4pyUNaa7uWVFmYKdiW5R2ypB4AAGwpcxUSqupZSd6Q5JOZBIQvLVPt00N55DLb75/k8ExudL5yg5oJAAAzNTchoap+O5OXof19JgHhKytUPX8oT15m3fFJbpfk4tbajaM3EgAANoG5CAnDi9BenuSSJA9trV29m+pnJ7k6yalVdb9F+zgwycuGr2/aqLYCAMCsbfkbl6vqSUlemskblD+U5JlVtbTaztbaGUnSWru2qk7LJCxcWFVnJtmVyVubjxqWnzWd1gMAwPRt+ZCQyT0ESbJfkmetUOdvk5yx8KW1dk5VnZDkBUkel+TAJJ9N8uwkr2utrfpkIwAA2Fdt+ZDQWjs9yel7sN1FSR4xdnsAAGCzm4t7EgAAgLUTEgAAgI6QAAAAdIQEAACgIyQAAAAdIQEAAOgICQAAQEdIAAAAOkICAADQERIAAICOkAAAAHSEBAAAoCMkAAAAHSEBAADoCAkAAEBHSAAAADpCAgAA0BESAACAjpAAAAB0hAQAAKAjJAAAAB0hAQAA6AgJAABAR0gAAAA6QgIAANAREgAAgI6QAAAAdIQEAACgIyQAAAAdIQEAAOgICQAAQEdIAAAAOkICAADQERIAAICOkAAAAHSEBAAAoCMkAAAAHSEBAADoCAkAAEBHSAAAADpCAgAA0BESAACAzv6zbgAAAOypqpp1E9astTbrJqyZmQQAAKBjJgEAgH3eE95y8aybsKKznrZ91k1YNzMJAABAR0gAAAA6QgIAANAREgAAgI6QAAAAdIQEAACgIyQAAAAdIQEAAOgICQAAQEdIAAAAOkICAADQERIAAICOkAAAAHSEBAAAoCMkAAAAHSEBAADoCAkAAEBHSAAAADpCAgAA0BESAACAjpAAAAB0hIQVVNUPVdXbqupfq+rGqtpZVa+pqu+bddsAAGAj7T/rBmxGVXWPJBcn+f4k70nyqST3T/KbSU6uqge31r46wyYCAMCGMZOwvD/IJCA8s7X2mNba81prJyV5dZKjkvzuTFsHAAAbSEhYoqqOSPKwJDuTvHHJ6hcn+VaSJ1bVwVNuGgAATIWQcGsnDeUHWmvfWbyitfaNJBcluV2SB067YQAAMA3VWpt1GzaVqnplkucmeW5r7VXLrH9Dkmck+bXW2ptW2dclK6z60YMOOmi/Y445Zq/bux6XXnrpVI8HAMB3HXvssVM93uWXX57rr79+V2vtjuvd1o3Lt7ZtKK9ZYf3C8kP34hi3XH/99ddceumlO/diH3vi6KH81JSPu6/SX+ujv9ZHf62P/lo/fbY++mt99Nf6HJ0kl1566bT767Ak1+7JhkLC+tVQrjoF01o7boPbsi4LMxubrV2blf5aH/21PvprffTX+umz9dFf66O/1mdf7C/3JNzawkzBthXWH7KkHgAAbClCwq19eiiPXGH9PYfyiim0BQAApk5IuLULhvJhVdX1T1V9b5IHJ7k+yUen3TAAAJgGIWGJ1trnknwgkxs9nrFk9UuSHJzkHa21b025aQAAMBVuXF7eryW5OMnrquqhSS5P8oAkD8nkMqMXzLBtAACwobwnYQVVdbckL01ycpI7JvliknOSvKS1tmuGTQMAgA0lJAAAAB33JAAAAB0hAQAA6AgJAABAR0gAAAA6QgIAANAREgAAgI6QsI+qqlOq6vVV9aGquraqWlW9cw/39UNV9baq+tequrGqdlbVa6rq+8Zu96yM1V9D37QVPl/aiLbPQlXdsaqeWlXvrqrPVtX1VXVNVX24qp5SVev6f8dWH2Nj9tccjbFXVNUHq+qfh/7aVVWfqKoXV9Ud17mvLT2+kvH6a17G13Kq6omLzvWp69x2y4+xpfa0v+ZhjI15jpt5bHnj8r7rhUl+NMk3k/xLkqP3ZCdVdY9M3i79/Unek+RTSe6f5DeTnFxVD26tfXWUFs/WKP01uCbJa5ZZ/s292Odm8/gkb8rkJYIXJPlCkh9I8tgkf5jk4VX1+LaGF63MyRgbrb8G8zDGfivJpUn+OslXkhyc5IFJTk/yK1X1wNbaP6+2kzkZX8lI/TWYh/HVqckLUl+fyTnefp3bzssY+9/2pr8G8zDG9vocN/3Yaq357IOfJA9Jcs8kleTEJC3JO/dgP+8ftv2NJct/f1j+5lmf6ybrr51Jds76fKbQXycl+Zkkt1my/C6Z/ALckjxujfva8mNs5P6alzF24ArLf3forz9Y4362/Pgaub/mYnwtOedK8jdJPpfklUN/PXUd28/FGBuxv7b8GBvrHDf72HK50T6qtXZBa+0zbRhNe6KqjkjysEwG+xuXrH5xkm8leWJVHbzHDd0kxuivedJaO7+19t7W2neWLP9SkjcPX09cbT/zMsbG6q950lq7YYVVfzaU91xtH/MyvpJx+muOPTOTIP/kTMbEms3TGFtkj/uLtdsXxpbLjebbSUP5gWV+uflGVV2UyQB+YJIPTrtxm9gBVfWLSe6eyX/ElyXZ0Vq7ZbbNmpqbhvLmNdQ1xtbXXwvmeYz9zFBetoa6xtf6+mvB3IyvqjomycuTvLa1tqOqTlptmyXmaoyN0F8L5mGM7e05bvqxJSTMt6OG8ooV1n8mkwF6ZLbA//xGdJckf7xk2VVV9eTW2t/OokHTUlX7J/ml4et5a9hkrsfYHvTXgrkZY1X13Eyued6W5H5JfjyTf2xfvobN52587WV/LZiL8TX89/fHmVzy9/w93M3cjLGR+mvBPIyxvT3HTT+2XG4037YN5TUrrF9YfujGN2Wf8UdJHprJ/xwOTvIjSd6S5LAkf1VVPzq7pk3Fy5PcO8m5rbX3r6H+vI+x9fZXMn9j7LmZTK0/K5NfeM9L8rDW2r+tYdt5HF9701/JfI2v30ly3yS/3Fq7fg/3MU9jbIz+SuZjjI1xjpt+bAkJ7E4Npev4B621lwzXn3+5tXZda+2TrbVfzeQmo4MyedLIllRVz0zynEyevvDEsXY7lFtujO1pf83bGGut3aW1Vpn8Y/vYJEck+URVHTvC7rfc+Nrb/pqX8VVV98/kr+Gvaq19ZCMPNZT79Bgbs7/mYYxN6RxnPraEhPm2kFK3rbD+kCX1WNnCzanHz7QVG6SqnpHktUn+KclDWmu71rjpXI6xveiv3dnSY2z4x/bdmUyv3zHJO9aw2VyOr2SP+2t3tsz4WnTZzBVJXrSXu9vyY2zk/tqdLTPGdmM957jpx5aQMN8+PZRHrrB+4WkZK10vx3d9ZSi30hMukiRV9awkb0jyyUx+4V3Pi2LmboztZX/tzpYdY4u11j6fSbi6V1XdaZXqcze+llpnf+3OVhpft89kTByT5IbFL7rK5FKtJHnrsOw1q+xrHsbYmP21O1tpjK1kPee46ceWG5fn2wVD+bCqus3iu+ur6nuTPDjJ9Uk+OovG7WMeNJRXzrQVI6uq387kuvq/T/JTrbWr17mLuRpjI/TX7mzJMbaCfzeUqz0lZK7G126stb92ZyuNrxuT/NcV1h2byXX3H87kl7TVLq2ZhzE2Zn/tzlYaYytZzzlu/rE1zZcy+GzMJ6u8HCzJ92TyhuF7LLNuU7/IYzP1V5J7JbnDMvV/OJOnELQkz5/1+Y3YTy8azunjy523MTZ+f83LGBvO/S7LLL9NvvtysIuMr3H7a17G1yp9eXqWeTnYvI+xsfprHsbYes9xXx5bZhL2UVX1mCSPGb7eZSgfVFVnDD9f3Vp77vDzDya5PMnnM7nzfrFfy+SV4K+rqocO9R6QyRuKr0jygvFbP30j9dfjkzyvqi5IclWSbyS5R5JHJjkwyblJfm9DTmDKqupJSV6ayV8mP5TkmVW1tNrO1toZw89zPcZG7K95GWMnJ3llVe3I5K2uX03yA0lOyORG3C8lOW1R/bkeXxmvv+ZlfO2JeR9j6zXPY2y957jvjq1ZJzKfPfvku+l+pc/ORXUPW7psyb7ulsnjvL6Y5NuZDOTXZpW/hu5LnzH6K5N/kP80k6fVfD2Tl2T9W5K/zuRZ+DXr85xif7UkFxpj4/bXvIyxTB4L+8ZMLsu6OpMXzV2T5L8PfXmHJfXnfXyN0l/zMr5W6cuF/1aX/mV8rsfYWP01D2Nsvee4L4+tGhoIAACQxNONAACAJYQEAACgIyQAAAAdIQEAAOgICQAAQEdIAAAAOkICAADQERIAAICOkAAAAHSEBAAAoCMkAAAAHSEBAADoCAkATEVV7VdVp1XV31bVrqq6qaq+UlWXVdUfVtWjFtU9saraap9ljvFjVfUnVfX5qrqxqq6tqs9V1Xur6j9V1cHTPWuAfdP+s24AAFtfVe2X5H1JTk7y9SR/meRfktwhyT2S/HySo5P8xZJNP5/kjDUe4xeTvD1JJTk/ybuT3JLk8CT3S/LTSf7/JJ/dm3MBmAdCAgDT8HOZBIR/SHJCa+2axSur6nZJHrDMdjtba6evtvNh+zcmaUke1lr74DJ1tie5ev1NB5g/QgIA07B9KM9YGhCSpLV2XZIL9mL/905ySJJ/WC4gDMe4eC/2DzBX3JMAwDR8dSiP3OD9/zv3HQDsvWrtVvd9AcCoquq+Sf4ukxnsP8nkfoFLWmufX6H+iZnMLOzunoRPtdbOHOrXsP8fy+SSprcmuTjJP7bWvj3WeQDMCyEBgKmoqv87yWuT3GXR4l1JdiR5W2vtvYvqnpjVLz96T2vtMYu2uXsmNy6fuKjOTUk+kckNy29qrV27xycAMEeEBACmpqq+J8lDkvx4kvsO5aHD6nck+eXWWlsUEv62tXbiOo9xTJKfyuSJRvdPctSw6gtJTmytXbVXJwEwB4QEAGZmeDTq45K8LcnBSX62tXbO3oSEZY5x9LD/B2XJ7AMAy3PjMgAz01q7pbX2Z0lePSw6aQOO8akkT9yo/QNsRUICAJvBN4ay9tH9A2wpQgIAG66qfq6qfqqqbvXvTlXdJclpw9cde7j/w6vqmVW1bZl1leQFe7N/gHnjZWoATMMDkvxmki9V1YeTLNw8fHiSRyY5KMl7kpy9ZLvDqur03ez3Na21ryfZlsmTk15ZVRcl+WQmswffn8klRkck+UqS54xxMgBbnRuXAdhwVXW3JI9K8pNJ/s8kd01yYCYvQftEkv+W5L+11r4z1D8xa3sD8+GttZ1VdUCShyd5WJIHJvnBJHdIcl2SzyY5L5NA8W/jnRXA1iUkAAAAHfckAAAAHSEBAADoCAkAAEBHSAAAADpCAgAA0BESAACAjpAAAAB0hAQAAKAjJAAAAB0hAQAA6AgJAABAR0gAAAA6QgIAANAREgAAgI6QAAAAdIQEAACgIyQAAACd/wUqKrRz2vD27QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 261,
       "width": 388
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# slightly skewed distribution of SES\n",
    "sns.histplot(df['SES'].dropna(), kde=False)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['SES'].fillna(df['SES'].median(), inplace=True) # impute missing socioeconomic status columns using median"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAwkAAAILCAYAAACjJNAzAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAABYlAAAWJQFJUiTwAAApzElEQVR4nO3debRlZ10n/O+PlFQoQoIMQjmGIJAsUIHQChVNQkQbQZkSutNOjBEQpcPgq4tBAo1vo7aCATva4UUGXSYSGlohAg0hhlEWCYhomEwCDgEMRRIOlQQq+b1/nHP17pt7q+pW7Tt/Pmud9dTZz7Of/Zy7a1fd79n72bu6OwAAAHNus9YDAAAA1hchAQAAGBASAACAASEBAAAYEBIAAIABIQEAABgQEgAAgAEhAQAAGBASAACAASEBAAAYEBIAAIABIQEAABjYttYD2Iqq6sokRya5ao2HAgDA5nV0kuu7+x7LXVFIWBtH3u52t7vTcccdd6e1HggAAJvT5ZdfnhtuuOGg1hUS1sZVxx133J0uvfTStR4HAACb1PHHH5/LLrvsqoNZ15wEAABgQEgAAAAGhAQAAGBASAAAAAaEBAAAYEBIAAAABoQEAABgQEgAAAAGhAQAAGBASAAAAAaEBAAAYEBIAAAABoQEAABgQEgAAAAGhAQAAGBASAAAAAaEBAAAYGDbWg8AAADWm507d2YymYzW3xFHHJGrr756tP5WmpAAAAALTCaTTCaTbNu+45D72nvTnhFGtLqEBAAAWMS27Tty6tnvPuR+3vysh40wmtVlTgIAADAgJAAAAANCAgAAMCAkAAAAA0ICAAAwICQAAAADQgIAADAgJAAAAANCAgAAMCAkAAAAA0ICAAAwICQAAAADQgIAADAgJAAAAANCAgAAMCAkAAAAA0ICAAAwICQAAAADQgIAADAgJAAAAANCAgAAMCAkAAAAA0ICAAAwICQAAAADQgIAADAgJAAAAANCAgAAMCAkAAAAA0ICAAAwICQAAAADQgIAADAgJAAAAANCAgAAMCAkAAAAA0ICAAAwICQAAAADQgIAADAgJAAAAANCAgAAMCAkAAAAA0ICAAAwICQAAAADQgIAADCw4UJCVZ1WVa+qqvdV1fVV1VX1x/tZZ1dVXVhVu6tqT1V9oqrOrKrD9rHOE6rqI1U1qarrquriqvrJ8T8RAACsLxsuJCR5YZJfSnL/JP+8v8ZV9egklyQ5Mclbkvx+ktsmeUWS85ZY538keV2SnUnOTfLHSb4vyV9U1S8d6gcAAID1bCOGhGcnuXeSI5M8Y18Nq+rITH/JvznJyd39lO7+lUwDxoeSnFZVpy9YZ1eS5yb5hyTf393P7u5nJjk+ye4k/6Oqjh71EwEAwDqy4UJCd7+3uz/b3X0AzU9Lctck53X3R+f1cWOmZySSWweNp8/K3+jur85b56pMz0JsT/Kkgxw+AACsexsuJCzTKbPyHYvUXZJkT5JdVbX9ANf5ywVtAABg09m21gNYYfeZlZ9ZWNHde6vqyiT3TXJMksur6vZJviPJpLuvXqS/z87Kex/Ixqvq0iWqjj2Q9QEAYC1s9jMJR83K65aon1t+x4NsDwAAm85mP5OwPzUrD2R+w3wH1L67j190o9MzDA9c5jYBAGBVbPYzCXPf/B+1RP2RC9rtr/3+zjQAAMCGt9lDwqdn5a3mEFTVtiT3SLI3yRVJ0t1fz/TZC0dU1c5F+rvXrLzVHAcAANgsNntIuGhWPnyRuhOT7Ejywe6+6QDX+YkFbQAAYNPZ7CHhgiTXJDm9qh40t7CqDk/ystnbcxas8wez8gVV9a3z1jk6yTOT3JTkj1ZqwAAAsNY23MTlqnpMksfM3t59Vj6kql43+/M13f28JOnu66vqjEzDwsVVdV6mT01+VKa3R70gyfnz++/uD1bV7yZ5TpJPVNUFSW6b5D8nuVOSX549WA0AADalDRcSktw/yRMWLDtm9kqSzyd53lxFd7+1qk5K8oIkpyY5PMnnMg0BZy/25Obufm5VfSLJLyX5hSS3JLksyW9399tG/TQAALDObLiQ0N1nJTlrmet8IMkjlrnO65O8fjnrAADAZrDZ5yQAAADLJCQAAAADQgIAADAgJAAAAANCAgAAMCAkAAAAA0ICAAAwICQAAAADQgIAADAgJAAAAANCAgAAMCAkAAAAA0ICAAAwICQAAAADQgIAADAgJAAAAANCAgAAMCAkAAAAA0ICAAAwICQAAAADQgIAADAgJAAAAANCAgAAMLBtrQcAAABj2LlzZyaTySh9TSaT1Lbto/S1EQkJAABsCpPJJJPJJNu27xipxx6pn41HSAAAYNPYtn1HTj373Yfcz/lP2zXCaDYucxIAAIABIQEAABgQEgAAgAEhAQAAGBASAACAASEBAAAYEBIAAIABIQEAABgQEgAAgAEhAQAAGBASAACAASEBAAAYEBIAAIABIQEAABgQEgAAgAEhAQAAGBASAACAASEBAAAYEBIAAIABIQEAABgQEgAAgAEhAQAAGBASAACAASEBAAAYEBIAAIABIQEAABgQEgAAgAEhAQAAGBASAACAASEBAAAYEBIAAIABIQEAABjYMiGhqh5ZVe+qqn+qqhuq6oqqelNVPWSJ9ruq6sKq2l1Ve6rqE1V1ZlUdttpjBwCA1bQlQkJV/WaStyV5YJJ3JPm9JJcleXSSD1TVzy5o/+gklyQ5Mclbkvx+ktsmeUWS81Zv5AAAsPq2rfUAVlpV3T3J85J8Kcn3d/eX59U9NMlFSV6a5I9ny45Mcm6Sm5Oc3N0fnS1/0aztaVV1encLCwAAbEpb4UzC92T6Of96fkBIku5+b5KvJbnrvMWnzd6fNxcQZm1vTPLC2dtnrOiIAQBgDW2FkPDZJN9I8oNVdZf5FVV1YpI7JHn3vMWnzMp3LNLXJUn2JNlVVdtXYKwAALDmNv3lRt29u6p+NcnvJvn7qnprkq8kuWeSRyX5v0meNm+V+8zKzyzS196qujLJfZMck+TyfW27qi5dourY5XwGAABYTZs+JCRJd7+yqq5K8tokZ8yr+lyS1y24DOmoWXndEt3NLb/jmGMEAID1YitcbpSq+n+SXJDkdZmeQbh9kuOTXJHkT6rqt5bT3azs/TXs7uMXeyX51LI+AAAArKJNHxKq6uQkv5nkz7v7Od19RXfv6e7Lkjw2yT8neW5VHTNbZe5MwVG36mzqyAXtAABgU9n0ISHJT87K9y6s6O49ST6S6c/hAbPFn56V917Yvqq2JblHkr2ZnoUAAIBNZyuEhLm7EN11ifq55d+YlRfNyocv0vbEJDuSfLC7bxpneAAAsL5shZDwvln5C1X1HfMrquonkpyQ5MYkH5wtviDJNUlOr6oHzWt7eJKXzd6es6IjBgCANbQV7m50QabPQXhYksur6i1JvpjkuEwvRaokv9bdX0mS7r6+qs6YrXdxVZ2XZHemt0u9z2z5+av+KQAAYJVs+pDQ3bdU1SOSPDPJ6ZlOVt6R6S/+FyY5u7vftWCdt1bVSUlekOTUJIdnervU58za7/fORgAAsFFt+pCQJN39zSSvnL0OdJ0PJHnECg0JAADWra0wJwEAAFgGIQEAABgQEgAAgAEhAQAAGBASAACAASEBAAAYEBIAAIABIQEAABgQEgAAgAEhAQAAGBASAACAASEBAAAYEBIAAIABIQEAABgQEgAAgAEhAQAAGBASAACAASEBAAAYEBIAAIABIQEAABgQEgAAgAEhAQAAGBASAACAASEBAAAYEBIAAIABIQEAABgQEgAAgAEhAQAAGBASAACAASEBAAAYEBIAAIABIQEAABgQEgAAgAEhAQAAGBASAACAASEBAAAYEBIAAIABIQEAABgYNSRU1XdX1ZH7aXOHqvruMbcLAACMZ+wzCVcm+a/7afOsWTsAAGAdGjsk1OwFAABsUGsxJ+FuSb6+BtsFAAAOwLZD7aCqfn7BovsvsixJDkvy3Ul+LsnfHup2AQCAlXHIISHJ65L07M+d5NGz10JzlyHtSfKSEbYLAACsgDFCwpNmZSV5bZK3Jvk/i7S7OclXknyou68dYbsAAMAKOOSQ0N2vn/tzVT0hyVu7+w2H2i8AALA2xjiT8G+6+6Fj9gcAAKw+T1wGAAAGRg8JVXVSVb2tqr5cVd+sqpsXee0de7sAAMA4Rr3cqKoemenE5cOSfCHJp5MIBAAAsIGMGhKSnJXkm0ke2d3vGrlvAABgFYx9udH9kpwvIAAAwMY1dkiYJNk9cp8AAMAqGjskvCfJQ0buEwAAWEVjh4RfTXLPqnphVdXIfQMAAKtg7InLL07yd0lekuTJVfXxJNcu0q67+ykjbxsAABjB2CHhifP+fPTstZhOIiQAAMA6NHZIuMfI/QEAAKts1JDQ3Z8fs7+xVdWPJDkzya4kd8r0Tkx/m+SV3X3hgra7krwwyYOTHJ7kc0lem+RV3X3zKg4bAABW1dhnEtatqnphkv+W5Jokb0tydZK7JHlAkpOTXDiv7aOTvDnJjUnOzzRM/FSSVyQ5IcnjV3HoAACwqkYNCVX13Qfatru/MOa296WqHp9pQHh3ksd199cW1H/LvD8fmeTcJDcnObm7Pzpb/qIkFyU5rapO7+7zVmv8AACwmsY+k3BVppOS96dXYNuLqqrbJPnNJHuS/PTCgJAk3f3NeW9PS3LXJG+YCwizNjfOzka8J8kzkggJAABsSmP/ov6GLB4S7pjk/km+J8nFSVZz7sKuTCdUX5Dkq1X1yCT3y/RSoo9094cWtD9lVr5jkb4uyTRs7Kqq7d190wqNGQAA1szYE5efuFTd7Bv9FyV5epInjLnd/fgPs/JLSS5L8n0LxnVJktO6+19ni+4zKz+zsKPu3ltVVya5b5Jjkly+rw1X1aVLVB17YEMHAIDVN/YTl5fU3bd090syvSTp5au13STfNiufnuR2SR6W5A6Znk14Z5ITk7xpXvujZuV1S/Q3t/yOo44SAADWibW4u9EHk/z8Km7vsFlZmZ4x+JvZ+7+rqsdmesbgpKp6yCKXHi2mZuV+51509/GLdjA9w/DAA9gWAACsulU7kzDPnZLcfhW399VZecW8gJAk6e4bMj2bkCQ/OCvnzhQclcUduaAdAABsKqsaEqrqYUn+c5JPruJmPz0rr12ifi5E3G5B+3svbFhV2zKdBL03yRUjjQ8AANaVsZ+TcNE+tvNdSeaeo/DSMbe7H5dk+kv9varqtt39jQX195uVV83Ki5L8TJKHJ/nTBW1PTLIjySXubAQAwGY19pmEk5d4nZDpZOF3Jvmx7n7byNtdUndfk+lTk49K8uvz66rqx5L8x0wvHZq75ekFmT6V+fSqetC8tocnedns7TkrPGwAAFgzY98CdS3mOByI5yT5oSQvqKoTk3wk02c2PDbTJyuf0d3XJkl3X19VZ2QaFi6uqvOS7E7yqExvj3pBpqEDAAA2pfX6S/2ouvvLmYaEV2R62dOzMn1o2tuT/Eh3v2lB+7cmOSnTS5VOTfLLSb6Zadg4vbsP5KnSAACwIa3oLVCr6shML/O5rruvX8lt7U937870l/znHGD7DyR5xIoOCgAA1qHRzyRU1WFV9WtV9blM7xx0VZKvVtXnZsvX4tkMAADAARr77ka3zXQC8EmZPmzsH5NcnWRnkqOT/EaSh1fVjy9ylyEAAGAdGPtMwnMyvZvR25Mc191Hd/dDuvvoTCf9/kWSH8kBXvIDAACsvrFDwk9n+qC0x3T3Z+dXdPc/JHlckr/L9DkEAADAOjR2SPjeJH/Z3bcsVjlb/pdJ7jnydgEAgJGMHRK+keSI/bS5faa3EwUAANahsUPCJ5KcVlV3Xayyqu6S5LQkfzPydgEAgJGMHRJeneSuST5SVU+pqmOq6nZVdY+qelKSv57Vv3rk7QIAACMZ9Rao3f1nVXX/JL+W5H8t0qSS/FZ3/9mY2wUAAMYz+oPNuvv5VfXnSZ6S5AGZPXE5yceSvLa7PzT2NgEAgPGsyNOPu/vDST68En0DAAAr65DnJFTV9qr6SFW9p6q+ZR/tbjtr8+F9tQMAANbWGBOXfybJ8Ul+p7uXvLVpd38jyW8n+cF4mBoAAKxbY4SExyW5orsv3F/D7n5Hks8mefwI2wUAAFbAGCHhAUkuXkb7S5Lcf4TtAgAAK2CMkHCXJF9aRvsvJbnzCNsFAABWwBgh4YYkRyyj/RFJbhxhuwAAwAoYIyT8Y5L/sIz2D0ryhRG2CwAArIAxQsLFSR5cVQ/aX8OqOj7JriTvHWG7AADAChgjJLw6SSd5U1Udt1Sjqjo2yZuS3Jzkf46wXQAAYAUc8hOXu/vTVfXSJGcl+VhVXZDkoiT/lGl4+M4kP5rk1CTbk/x6d3/6ULcLAACsjEMOCUnS3S+tqr1JXpzkp5P8lwVNKsk3k7ygu//7GNsEAABWxighIUm6+/+tqj9J8uQkJyTZmWk4+Jck70/yR939+bG2BwAArIzRQkKSzELAi8fsEwAAWF1jTFwGAAA2ESEBAAAYEBIAAIABIQEAABgQEgAAgAEhAQAAGBASAACAASEBAAAYEBIAAIABIQEAABgQEgAAgAEhAQAAGBASAACAASEBAAAYEBIAAIABIQEAABgQEgAAgAEhAQAAGBASAACAASEBAAAY2LbWAwAAYOvauXNnJpPJKH1NJpPUtu2j9LXVCQkAAKyZyWSSyWSSbdt3jNRjj9TP1iYkAACwprZt35FTz373Ifdz/tN2jTAaEnMSAACABYQEAABgQEgAAAAGhAQAAGBASAAAAAaEBAAAYEBIAAAABoQEAABgQEgAAAAGhAQAAGBgS4aEqvq5qurZ66lLtNlVVRdW1e6q2lNVn6iqM6vqsNUeLwAArKYtFxKq6ruSvCrJZB9tHp3kkiQnJnlLkt9Pctskr0hy3ioMEwAA1syWCglVVUn+KMlXkvzBEm2OTHJukpuTnNzdT+nuX0ly/yQfSnJaVZ2+OiMGAIDVt6VCQpJnJTklyZOSfH2JNqcluWuS87r7o3MLu/vGJC+cvX3GSg4SAADW0pYJCVV1XJKXJ/m97r5kH01PmZXvWKTukiR7kuyqqu0jDxEAANaFbWs9gNVQVduSvDHJF5I8fz/N7zMrP7Oworv3VtWVSe6b5Jgkl+9nu5cuUXXsfsYAAABrZkuEhCS/nuQBSX64u2/YT9ujZuV1S9TPLb/jCOMCAIB1Z9OHhKr6wUzPHvxOd39ojC5nZe+vYXcfv8SYLk3ywBHGAgAAo9vUcxLmXWb0mSQvOsDV5s4UHLVE/ZEL2gEAwKayqUNCkiOS3DvJcUlunPcAtU7y4lmbc2fLXjl7/+lZee+Fnc1Cxz2S7E1yxYqOHAAA1shmv9zopiT/3xJ1D8x0nsL7Mw0Gc5ciXZTkZ5I8PMmfLljnxCQ7klzS3TeNPloAAFgHNnVImE1SfupidVV1VqYh4fXd/Zp5VRck+c0kp1fVq+aelVBVhyd52azNOSs2aAAAWGObOiQcjO6+vqrOyDQsXFxV5yXZneRRmd4e9YIk56/hEAEAYEVt9jkJB6W735rkpEwfnnZqkl9O8s0kz0lyenfv985GAACwUW3ZMwndfVaSs/ZR/4Ekj1it8QAAwHrhTAIAADAgJAAAAANCAgAAMCAkAAAAA0ICAAAwICQAAAADQgIAADAgJAAAAANCAgAAMCAkAAAAA0ICAAAwICQAAAADQgIAADAgJAAAAANCAgAAMCAkAAAAA0ICAAAwICQAAAADQgIAADCwba0HAADAxrJz585MJpNR+ppMJqlt20fpi/EICQAALMtkMslkMsm27TtG6rFH6oexCAkAACzbtu07curZ7z7kfs5/2q4RRsPYzEkAAAAGhAQAAGBASAAAAAaEBAAAYEBIAAAABoQEAABgQEgAAAAGhAQAAGBASAAAAAaEBAAAYEBIAAAABoQEAABgQEgAAAAGhAQAAGBASAAAAAaEBAAAYEBIAAAABoQEAABgQEgAAAAGhAQAAGBASAAAAAaEBAAAYEBIAAAABoQEAABgQEgAAAAGhAQAAGBASAAAAAaEBAAAYEBIAAAABoQEAABgQEgAAAAGhAQAAGBASAAAAAaEBAAAYEBIAAAABoQEAABgYNOHhKq6c1U9tareUlWfq6obquq6qnp/VT2lqhb9GVTVrqq6sKp2V9WeqvpEVZ1ZVYet9mcAAIDVtG2tB7AKHp/knCRXJ3lvki8kuVuSxyV5TZKfqKrHd3fPrVBVj07y5iQ3Jjk/ye4kP5XkFUlOmPUJAACb0lYICZ9J8qgkb+/uW+YWVtXzk3wkyamZBoY3z5YfmeTcJDcnObm7Pzpb/qIkFyU5rapO7+7zVvVTAADAKtn0lxt190Xd/RfzA8Js+ReT/MHs7cnzqk5Lctck580FhFn7G5O8cPb2GSs3YgAAWFubPiTsxzdn5d55y06Zle9YpP0lSfYk2VVV21dyYAAAsFa2wuVGi6qqbUl+fvZ2fiC4z6z8zMJ1untvVV2Z5L5Jjkly+X62cekSVccub7QAAIdm586dmUwmo/Q1mUxS23xfuplt2ZCQ5OVJ7pfkwu5+57zlR83K65ZYb275HVdoXAAAo5tMJplMJtm2fcdIPfb+m7BhbcmQUFXPSvLcJJ9K8nPLXX1W7vfI6O7jl9j+pUkeuMztAgAckm3bd+TUs999yP2c/7RdI4yG9WzLzUmoqmcm+b0kf5/kod29e0GTuTMFR2VxRy5oBwAAm8qWCglVdWaSVyf5ZKYB4YuLNPv0rLz3IutvS3KPTCc6X7FCwwQAgDW1ZUJCVf1qpg9D+3imAeHLSzS9aFY+fJG6E5PsSPLB7r5p9EECAMA6sCVCwuxBaC9PcmmSH+3ua/bR/IIk1yQ5vaoeNK+Pw5O8bPb2nJUaKwAArLVNP3G5qp6Q5KWZPkH5fUmeVVULm13V3a9Lku6+vqrOyDQsXFxV5yXZnelTm+8zW37+6oweAABW36YPCZnOIUiSw5KcuUSbv0ryurk33f3WqjopyQuSnJrk8CSfS/KcJGd3t3t+AQCwaW36kNDdZyU56yDW+0CSR4w9HgAAWO+2xJwEAADgwAkJAADAgJAAAAAMCAkAAMCAkAAAAAwICQAAwICQAAAADAgJAADAgJAAAAAMCAkAAMCAkAAAAAwICQAAwICQAAAADAgJAADAgJAAAAAMCAkAAMCAkAAAAAwICQAAwICQAAAADGxb6wEAAGwmO3fuzGQyGa2/I444IldfffVo/cGBEBIAAEY0mUwymUyybfuOQ+5r7017RhgRLJ+QAAAwsm3bd+TUs999yP28+VkPG2E0sHzmJAAAAAPOJADAJrQVrovfCp8R1oqQAACb0Fa4Ln4rfEZYK0ICAGxSW+G6+K3wGWEtmJMAAAAMCAkAAMCAkAAAAAwICQAAwICQAAAADAgJAADAgJAAAAAMCAkAAMCAkAAAAAwICQAAwICQAAAADGxb6wEAAFM7d+7MZDIZpa/JZJLatn2Uvsa0FT4jbAZCAgCsE5PJJJPJJNu27xipxx6pn/Fshc8Im4GQAADryLbtO3Lq2e8+5H7Of9quEUazMrbCZ4SNzpwEAABgQEgAAAAGhAQAAGBASAAAAAZMXAZg3RvztplJcsQRR+Tqq68erT+AzUZIAGDdG/O2mXtv2jPCiAA2NyEBgA1hrNtmvvlZDxthNACbmzkJAADAgDMJAMCWt/emPZnclNzhDnc45L4mk0lq2/YRRgVrR0gAAJi58Zu3jNRTj9QPrA0hAQAgSW277SjzXs5/2q4RRgNry5wEAABgwJkEAP6N5xEsz9g/L9eyA+uFkADAv/E8guUZ8+f171zLDqw9IQGAAc8jWJ6xfl6Ja9mB9UNIANjgxrzkZStc7uJWlwD7JyQAbHDjX/KyNS53catLgKUJCQCbwFiXvGyVy13c6hJg39wCdQlV9Z1V9dqq+pequqmqrqqqV1bVt6712AAAYCU5k7CIqrpnkg8m+bYk/yfJp5L8YJL/muThVXVCd39lDYd4ULbCrQ23wmcEAFhpQsLi/memAeFZ3f2quYVV9btJnp3kN5I8fY3GdtC2wq0Nt8JnBABYaULCAlV1TJIfT3JVkt9fUP3iJL+Q5Oeq6rnd/fVVHt4h2wq3NtwKnxEAYCWZk3Brp8zKd3X34NYX3f21JB9IsiPJg1d7YAAAsBqq263b5quq307yvCTP6+7fWaT+1UmemeQXu/uc/fR16RJVP3C7293usOOOO+6Qx7scH//4x3PLLbckVYfe2ezvzW1us75y5i23zHLdJv6MsNBK/L1fb8fQev+Mo/Q1dn/25abqy89+4/d1m9vcJve///0Pva9luPzyy3PDDTfs7u47L3ddlxvd2lGz8rol6ueW3/EQtnHzDTfccN1ll1121SH0cfBGDIb/9g9Ocuys/NRonR+KlfmMW9H62q/s24H9vT+wfbpej6Exv9har30tv79971P7ciP2dat96me/4fs69pZbbslll1222v+fHp3k+oNZUUhYvrk4ud+/Md19/AqPZd2YO2uylT7zVmC/bj726eZjn24+9unmsxH3qesobm3uTMFRS9QfuaAdAABsKkLCrX16Vt57ifp7zcrPrMJYAABg1QkJt/beWfnjVTX4+VTVHZKckOSGJB9e7YEBAMBqEBIW6O5/SPKuTCd6PHNB9UuS3D7JGzbiMxIAAOBAmLi8uF9M8sEkZ1fVjya5PMkPJXloppcZvWANxwYAACvKcxKWUFXfleSlSR6e5M5Jrk7y1iQv6e7dazg0AABYUUICAAAwYE4CAAAwICQAAAADQgIAADAgJAAAAANCAgAAMCAkAAAAA0ICh6SqrqqqXuL1xbUeH4urqtOq6lVV9b6qun62v/54P+vsqqoLq2p3Ve2pqk9U1ZlVddhqjZulLWefVtXR+zhuu6rOW+3xM1RVd66qp1bVW6rqc1V1Q1VdV1Xvr6qnVNWi/387Tte35e5Xx+rGUFW/WVXvqap/nO3T3VX1sap6cVXdeYl11v2x6onLjOG6JK9cZPlklcfBgXthkh/IdB/9U5Jj99W4qh6d5M1JbkxyfpLdSX4qySuSnJDk8Ss5WA7IsvbpzN9k+pDIhT453rA4SI9Pck6mD/J8b5IvJLlbkscleU2Sn6iqx/e8hx05TjeEZe/XGcfq+vbsJJcl+b9Jvpzk9kkenOSsJL9QVQ/u7n+ca7xRjlUPU+OQVNVVSdLdR6/tSFiOqnpopr9Ifi7JSZn+Z/Un3f2zi7Q9ctbuqCQndPdHZ8sPT3JRkock+S/d7RutNbTMfXp0kiuTvL67n7iKw+QAVdUpmf6i8fbuvmXe8rsn+UiS70pyWne/ebbccboBHMR+PTqO1XWvqg7v7hsXWf4bSZ6f5Jzu/sXZsg1zrLrcCLag7n5vd392kW+rFnNakrsmOW/uH7NZHzdm+u11kjxjBYbJMixzn7LOdfdF3f0X83+RnC3/YpI/mL09eV6V43QDOIj9ygawWECY+bNZea95yzbMsepyI8awvap+Nsl3J/l6kk8kuaS7b17bYTGSU2blOxapuyTJniS7qmp7d9+0esNiBN9eVU9LcuckX0nyoe7+xBqPif375qzcO2+Z43TjW2y/znGsbkw/NSvn76sNc6wKCYzh7kneuGDZlVX1pO7+q7UYEKO6z6z8zMKK7t5bVVcmuW+SY5JcvpoD45D92Oz1b6rq4iRP6O4vrMmI2Keq2pbk52dv5/+S4TjdwPaxX+c4VjeAqnpekiMyvZToQUl+ONOA8PJ5zTbMsepyIw7VHyX50UyDwu2TfF+SP0xydJK/rKofWLuhMZKjZuV1S9TPLb/jyg+FkexJ8t+SHJ/kW2evuXkMJyd5T1Xdfs1Gx768PMn9klzY3e+ct9xxurEttV8dqxvL85K8OMmZmQaEdyT58e7+13ltNsyxKiRwSLr7JbNrLL/U3Xu6+5Pd/fQkv5vkdpnO7Gdzq1npWvgNoru/3N2/3t2Xdfe1s9clSX48yV8n+d4kT13bUbJQVT0ryXOTfCrJzy139VnpOF1n9rVfHasbS3ffvbsr0y9OH5fp2YCPVdUDl9HNujlWhQRWytwErBPXdBSMYe5bjaOWqD9yQTs2qO7em+ltGBPH7rpSVc9M8ntJ/j7JQ7t794ImjtMN6AD266Icq+vb7IvTt2Qa5u6c5A3zqjfMsSoksFK+PCudBt34Pj0r772wYnYd7T0ynWh3xWoOihUzd1rcsbtOVNWZSV6d6T3xHzq7E85CjtMN5gD36744Vte57v58pgHwvlV1l9niDXOsCgmslIfMyjX/S84hu2hWPnyRuhOT7EjywbW+CwOjefCsdOyuA1X1q5k+YOnjmf4i+eUlmjpON5Bl7Nd9caxuDN8+K+fu+LhhjlUhgYNWVfetqjstsvx7Mv12JEn+eHVHxQq4IMk1SU6vqgfNLZw9+OVls7fnrMXAODhV9UNVddtFlp+S6ZNDE8fumquqF2U6ofXSJD/a3dfso7njdINYzn51rK5/VXXs7GF4C5ffZvYwtW/L9Jf+r86qNsyx6onLHLSqOivJr2V6l4Urk3wtyT2TPDLJ4UkuTPLY7v7GWo2RxVXVY5I8Zvb27kn+Y6bfRr1vtuya7n7egvYXZPoI+fMyfYT8ozK9ldsFSf6Th3itreXs09mtE++b5OJMn9KcJN+ff79/94u6e+4/K9ZAVT0hyesy/fbxVVn8+uSruvt189Z5TByn69py96tjdf2bXTb225k+4+AfMn2Oxd0yvQvVMUm+mGkY/Pt56zwmG+BYFRI4aFV1UpKnJ3lA/v0WqNdmevr0jUneuB7+knNrs4D34n00+Xx3H71gnROSvCDTS8kOz/Sx8q9NcrYH56295ezTqnpKksdmesvFuyT5liRfSvKhJK/u7vct1Qmr4wD2Z5L8VXefvGA9x+k6ttz96lhd/6rqfpk+IfmEJN+Z6a1Lv57pcxDenumxd6sJ6RvhWBUSAACAAXMSAACAASEBAAAYEBIAAIABIQEAABgQEgAAgAEhAQAAGBASAACAASEBAAAYEBIAAIABIQEAABgQEgAAgAEhAQAAGBASADgoVdWz1y1Vdc99tHvvvLZPXFD3unl1L9lHH0+Y1+7iRervWFUvraqPV9Wkqm6qqn+uqg9X1e9U1QMWtD9rXn9LvW61HYCtYttaDwCADW1vpv+XPCXJ8xdWVtW9kpw0r92++nlyVb20u29epP6Mpfqoqm9P8oEkRye5IsmfJNmd5DuSHJvkzCQ3JPnYIv3+VZKLlxjTVfsYL8CmJiQAcCi+lOTqJE+qql/v7r0L6p+apJK8Lclj9tHPXP3Dk7x9fkVVHZfkhCRvSfLYRdZ9aaYB4bVJntrdvWD9nUl2LrHdi7v7rH2MC2BLcrkRAIfq3CR3T/KT8xdW1bckeUKSDyb5u/308SeZftt/xiJ1c8tes8S6u2blqxYGhCTp7qu7+7L9bB+AeYQEAA7Vnyb5eqZnDeZ7VJK7ZRoi9ufaJG9K8sjZN/9JkqranuTnM70s6DNLrPuVWXnvAx8yAPviciMADkl3f62qzkvyxKr6zu7+p1nVGUmuT/JnWWS+wiLOzTQQPDHJf58te2ySO2ffQeP8JD+c5DVV9aAk70ryse7+yj7WmXNyVZ21RN07uvvDB9AHwKYjJAAwhnMznbz85CQvrarvSfJjSf6wu/dU1X476O73V9XlSZ5aVS+fXTp0RpKvJnlzku9cYtXfT/LtSZ6d5Fdmr1TVVUneneTV3f03S6x70uy1mGuTCAnAluRyIwAOWXf/dZK/zfQORbfJ9NKj2+TALjWa7zVJjklySlV9b5KHJnljd9+4j213dz8/08nJpyd5ZZJLZu+fmuTSqlpsrkOSvKS7a4nXK5c5doBNQ0gAYCznJvmeTO9Q9KQkl3b3Yrcd3Zc3JLkp01/u5+6MdEBBo7uv7e7zu/vZ3X1SppcpvSzJYUleVVV3W+ZYALYsIQGAsbwx0zsU/WGmzyj4X8vtoLuvyb/f6vTJST7U3Z88mMF099e7+0VJ3p9ke6a3UQXgAAgJAIyiu69NckGmcwe+nuldjw7GuZn+Un/XLP9ypcV8bVbuf2IEAElMXAZgXC9M8r+T/Gt3f21/jZfw3iSPzvSLrHfur3FV/UqSC7v7Vs9iqKofznRew94kHzrI8QBsOUICAKPp7i8k+cIh9tFJ/nwZq/xMkt+qqk9lejeiq5PcPsl9k5yS6RmE53b3vyyy7r5ugXqtycvAViUkALDRPSnJIzMNBCdn+vTnSvLPmV7ydE53v3+Jdfd1C9TPZ3qnJIAtpxZ5gj0AALCFmbgMAAAMCAkAAMCAkAAAAAwICQAAwICQAAAADAgJAADAgJAAAAAMCAkAAMCAkAAAAAwICQAAwICQAAAADAgJAADAgJAAAAAMCAkAAMCAkAAAAAwICQAAwICQAAAADPz/OCQ+RaR4ECQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 261,
       "width": 388
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# left-skewed distribution of MMSE\n",
    "sns.histplot(df['MMSE'].dropna(), kde=False)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['MMSE'].fillna(df['MMSE'].median(), inplace=True) # impute missing MMSE columns using the median"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Ensure no more Missing Values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['SES'].isna().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['MMSE'].isna().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Subject ID    0\n",
       "MRI ID        0\n",
       "Group         0\n",
       "Visit         0\n",
       "MR Delay      0\n",
       "M/F           0\n",
       "Hand          0\n",
       "Age           0\n",
       "EDUC          0\n",
       "SES           0\n",
       "MMSE          0\n",
       "CDR           0\n",
       "eTIV          0\n",
       "nWBV          0\n",
       "ASF           0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Cleaning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our data cleaning approach was quite straightforward. Once we imputed the missing values in the 'SES' and 'MMSE' columns using the medians because of their non-normal distributions, we simply had to ensure our relevant columns were of the correct data type, encoded numerically, and renamed for easier understanding. We chose to use a subset of the data, only subjects' first visits as that is the most appropriate for our prediction task for determing the early onset of Alzheimers/Dementia based on an initial visit. Then, we renamed the 'M/F' column and encoded it to be binary, updated the target variable, 'Group,' to have binary values of (1,0) representing the final categorization of Demented or Nondemented, respectively. Lastly, we dropped extraneous columns such as 'Hand' and 'MRI ID' that will not be used in our prediction models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['Nondemented', 'Demented', 'Converted'], dtype=object)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Group'].unique() # 'Converted' are those individuals who were initially Nondemented but then became Demented"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.options.mode.copy_on_write = True\n",
    "\n",
    "#Only use date from subjects' first visits\n",
    "df = df.loc[df['Visit']==1]\n",
    "\n",
    "#Convert M/F to numeric values and rename\n",
    "df['Gender'] = df['M/F'].map({'F': 0, 'M': 1})\n",
    "\n",
    "#Change 'Converted' to 'Demented'\n",
    "df['Group'] = df['Group'].replace('Converted', 'Demented')\n",
    "\n",
    "#Convert Group to numeric values (these are our target variables)\n",
    "df['Group'] = df['Group'].replace(['Demented', 'Nondemented'], [1,0])\n",
    "\n",
    "#Drop variables that will not be used in predictions\n",
    "df = df.drop(['MRI ID', 'Visit', 'Hand', 'M/F'], axis=1)\n",
    "\n",
    "df = df.reset_index(drop=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Subject ID</th>\n",
       "      <th>Group</th>\n",
       "      <th>MR Delay</th>\n",
       "      <th>Age</th>\n",
       "      <th>EDUC</th>\n",
       "      <th>SES</th>\n",
       "      <th>MMSE</th>\n",
       "      <th>CDR</th>\n",
       "      <th>eTIV</th>\n",
       "      <th>nWBV</th>\n",
       "      <th>ASF</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>OAS2_0001</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>87</td>\n",
       "      <td>14</td>\n",
       "      <td>2.0</td>\n",
       "      <td>27.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1987</td>\n",
       "      <td>0.696</td>\n",
       "      <td>0.883</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>OAS2_0002</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>75</td>\n",
       "      <td>12</td>\n",
       "      <td>2.0</td>\n",
       "      <td>23.0</td>\n",
       "      <td>0.5</td>\n",
       "      <td>1678</td>\n",
       "      <td>0.736</td>\n",
       "      <td>1.046</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>OAS2_0004</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>88</td>\n",
       "      <td>18</td>\n",
       "      <td>3.0</td>\n",
       "      <td>28.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1215</td>\n",
       "      <td>0.710</td>\n",
       "      <td>1.444</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>OAS2_0005</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>80</td>\n",
       "      <td>12</td>\n",
       "      <td>4.0</td>\n",
       "      <td>28.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1689</td>\n",
       "      <td>0.712</td>\n",
       "      <td>1.039</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>OAS2_0007</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>71</td>\n",
       "      <td>16</td>\n",
       "      <td>2.0</td>\n",
       "      <td>28.0</td>\n",
       "      <td>0.5</td>\n",
       "      <td>1357</td>\n",
       "      <td>0.748</td>\n",
       "      <td>1.293</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>OAS2_0182</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>73</td>\n",
       "      <td>12</td>\n",
       "      <td>2.0</td>\n",
       "      <td>23.0</td>\n",
       "      <td>0.5</td>\n",
       "      <td>1661</td>\n",
       "      <td>0.698</td>\n",
       "      <td>1.056</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>146</th>\n",
       "      <td>OAS2_0183</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>66</td>\n",
       "      <td>13</td>\n",
       "      <td>2.0</td>\n",
       "      <td>30.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1495</td>\n",
       "      <td>0.746</td>\n",
       "      <td>1.174</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>147</th>\n",
       "      <td>OAS2_0184</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>72</td>\n",
       "      <td>16</td>\n",
       "      <td>3.0</td>\n",
       "      <td>24.0</td>\n",
       "      <td>0.5</td>\n",
       "      <td>1354</td>\n",
       "      <td>0.733</td>\n",
       "      <td>1.296</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>148</th>\n",
       "      <td>OAS2_0185</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>80</td>\n",
       "      <td>16</td>\n",
       "      <td>1.0</td>\n",
       "      <td>28.0</td>\n",
       "      <td>0.5</td>\n",
       "      <td>1704</td>\n",
       "      <td>0.711</td>\n",
       "      <td>1.030</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>149</th>\n",
       "      <td>OAS2_0186</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>61</td>\n",
       "      <td>13</td>\n",
       "      <td>2.0</td>\n",
       "      <td>30.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1319</td>\n",
       "      <td>0.801</td>\n",
       "      <td>1.331</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>150 rows × 12 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    Subject ID  Group  MR Delay  Age  EDUC  SES  MMSE  CDR  eTIV   nWBV  \\\n",
       "0    OAS2_0001      0         0   87    14  2.0  27.0  0.0  1987  0.696   \n",
       "1    OAS2_0002      1         0   75    12  2.0  23.0  0.5  1678  0.736   \n",
       "2    OAS2_0004      0         0   88    18  3.0  28.0  0.0  1215  0.710   \n",
       "3    OAS2_0005      0         0   80    12  4.0  28.0  0.0  1689  0.712   \n",
       "4    OAS2_0007      1         0   71    16  2.0  28.0  0.5  1357  0.748   \n",
       "..         ...    ...       ...  ...   ...  ...   ...  ...   ...    ...   \n",
       "145  OAS2_0182      1         0   73    12  2.0  23.0  0.5  1661  0.698   \n",
       "146  OAS2_0183      0         0   66    13  2.0  30.0  0.0  1495  0.746   \n",
       "147  OAS2_0184      1         0   72    16  3.0  24.0  0.5  1354  0.733   \n",
       "148  OAS2_0185      1         0   80    16  1.0  28.0  0.5  1704  0.711   \n",
       "149  OAS2_0186      0         0   61    13  2.0  30.0  0.0  1319  0.801   \n",
       "\n",
       "       ASF  Gender  \n",
       "0    0.883       1  \n",
       "1    1.046       1  \n",
       "2    1.444       0  \n",
       "3    1.039       1  \n",
       "4    1.293       1  \n",
       "..     ...     ...  \n",
       "145  1.056       1  \n",
       "146  1.174       0  \n",
       "147  1.296       0  \n",
       "148  1.030       1  \n",
       "149  1.331       0  \n",
       "\n",
       "[150 rows x 12 columns]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Check to ensure all Correct Data Types"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Subject ID     object\n",
       "Group           int64\n",
       "MR Delay        int64\n",
       "Age             int64\n",
       "EDUC            int64\n",
       "SES           float64\n",
       "MMSE          float64\n",
       "CDR           float64\n",
       "eTIV            int64\n",
       "nWBV          float64\n",
       "ASF           float64\n",
       "Gender          int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Ethics & Privacy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ethics & Privacy Considerations:\n",
    "\n",
    "Biases/Privacy/Terms of Use Issues with Proposed Data:\n",
    "\n",
    "1.  The potential datasets considered, such as OASIS MRI, UK Biobank,\n",
    "     and NACC, may have biases and privacy considerations. For\n",
    "     instance, the OASIS MRI project's terms of use and participant\n",
    "     selection could introduce biases. UK Biobank, despite its size,\n",
    "     might not include a diverse representation of certain populations,\n",
    "     leading to potential biases in the dataset.\n",
    "\n",
    "Potential Biases in Dataset Composition and Collection:\n",
    "\n",
    "2.  Biases may arise in dataset composition and collection, affecting\n",
    "     the equitable analysis of Alzheimer's prediction. For example, if\n",
    "     the data predominantly includes participants from specific\n",
    "     demographic groups, it could introduce biases in the model.\n",
    "     Additionally, variations in data collection methods across\n",
    "     different research centers, as in the case of NACC, may impact\n",
    "     standardization, potentially leading to biases.\n",
    "\n",
    "Detection and Mitigation of Biases:\n",
    "\n",
    "3.  To detect biases, the group will conduct a thorough review of the\n",
    "     dataset sources, including participant demographics and\n",
    "     recruitment methods. During data preprocessing, the team will\n",
    "     analyze variables for potential biases, ensuring a balanced\n",
    "     representation. The group plans to collaborate with experts in the\n",
    "     field and seek external input to validate the fairness and\n",
    "     inclusivity of the dataset.\n",
    "\n",
    "Other Issues Related to Privacy and Equitable Impact:\n",
    "\n",
    "4.  Privacy concerns arise from the sensitive nature of medical data,\n",
    "     especially in Alzheimer's research. Ensuring participant anonymity\n",
    "     and adhering to privacy regulations are paramount. Equitable\n",
    "     impact considerations involve understanding if the model's\n",
    "     predictions could disproportionately affect certain groups. It is\n",
    "     essential to communicate findings responsibly, avoiding\n",
    "     reinforcing existing biases or stigmatizing specific populations.\n",
    "\n",
    "Handling Identified Issues:\n",
    "\n",
    "5.  The group commits to transparently communicating any identified\n",
    "     biases throughout the research process. Mitigation strategies will\n",
    "     be implemented during data preprocessing and model development.\n",
    "     The team will consider alternative datasets or additional sampling\n",
    "     methods if biases persist. Ethical review boards will be\n",
    "     consulted, and the group aims to publish findings with a clear\n",
    "     acknowledgment of potential limitations and biases, promoting\n",
    "     responsible and equitable use of the predictive model.\n",
    "\n",
    "In summary, the group is dedicated to addressing ethical concerns\n",
    "comprehensively, from data collection to analysis and post-analysis.\n",
    "Transparency, collaboration with experts, and continuous evaluation of\n",
    "potential biases will guide the research, ensuring responsible and\n",
    "ethical development of the Alzheimer's prediction model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Team Expectations "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Clear communication and relatively reasonable responsiveness to messages.\n",
    "2. Shared Responsibility and Accountability: finish all tasked work by the designated due date.\n",
    "3. Maintain quality work and attention to detail.\n",
    "4. Attendance and Participation: during designated days in which we meet, everyone should be present unless notified previously. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Project Timeline Proposal"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Meeting Date  | Meeting Time| Completed Before Meeting  | Discuss at Meeting |\n",
    "|---|---|---|---|\n",
    "| 2/24  | 6 PM  | Import & Wrangle Data (Joseph); | Review/Edit wrangling; Discuss Analysis Plan and EDA   |\n",
    "| 3/9  | 12 PM  | Finalize wrangling/EDA; Begin Analysis (Shivangi; Zijun) | Discuss/edit Analysis; Complete project check-in |\n",
    "| 3/13  | 12 PM  | Complete analysis; Draft results/conclusion/discussion (Johnny; Tanishq)| Discuss/edit full project |\n",
    "| 3/20  | Before 11:59 PM  | NA | Turn in Final Project & Group Project Surveys |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}