DataCheckpoint_Group070_WI24.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**If you lost points on the last checkpoint you can get them back by responding to TA/IA feedback**  \n",
    "\n",
    "Update/change the relevant sections where you lost those points, make sure you respond on GitHub Issues to your TA/IA to call their attention to the changes you made here.\n",
    "\n",
    "Please update your Timeline... no battle plan survives contact with the enemy, so make sure we understand how your plans have changed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# COGS 108 - Data Checkpoint"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Names\n",
    "\n",
    "- Deena Pederson\n",
    "- Vedika Harnathka\n",
    "- Paola Sanchez\n",
    "- Anne Pham\n",
    "- Zoe Ludena"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Research Question"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How have abortion rates changed in response to changes in various factors - such as the representation of female politicians, access to obstetric and gynecological (OB/GYN) care, and income levels - in the USA from 2014-2017? How will these factors help us predict future abortion rates in the foreseeable future?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Background and Prior Work"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "To be more specific in our research, the definition in which we are abiding by the term “women in governing bodies” are women who are in the respective current state legislature. Throughout our data we will look at all U.S. territories excluding Guam, U.S. Virgin Islands, and the District of Columbia. However, we will be using 3 states as markers and general comparisons to the data: 1) having the highest percentage of women in their state legislature, 2) having the lowest percentage of women in their state legislature, and 3) having the median percentage of women in their state legislature. While proceeding with our research, if we have the time and data to do so, we would love to delve deeper into any possible outliers within the states.\n",
    "\n",
    "One of the first considerations to be made is which states have higher female representation and which have less. We can note that in all 50 states males are over represented while females are underrepresented.[^1] This data splits the measures into two separate categories; state legislatures. Overall Vermont, Maine, and Rhode Island notably have the smallest disparities, while North Dakota, South Dakota, and Nebraska have the highest.\n",
    "\n",
    "A study done by the National Partnership for Women & Families[^2] states with a higher representation of female legislators are more likely to adopt abortion protection policies. This study considers 2 categories of representation in legislation: states where women make up more than a third of the members, and states where women make up less than a third of members. They found that of the prior, 84% of these states have policies that support abortion access while the former only has 15%. They also consider how the ethnic diversity in these women may also correlate.\n",
    "\n",
    "Subsequently, female health is a very general statement, therefore we decided to focus on OB/GYN access. We believe that this may yield some type of correlation between abortion rates. We believe that there will be some type of existing correlation between this accessibility and abortion rates. Additionally, another factor that we must consider are average income rates. In the article, *Rural-urban differences in access to hospital obstetric and neonatal care: how far is the closest one?*, their general conclusion is that: “Women in communities that are already socioeconomically disadvantaged face increasing and substantial travel distances to access perinatal care.[^3] (Hung et al.)\n",
    "\n",
    "[^1]: Wolf, C. (2022, August 25). [Best States for Gender Equality: Representation and Power](https://www.usnews.com/news/best-states/articles/best-states-for-gender-equality-representation-power#:~:text=Vermont%2C%20Maine%20and%20Maryland%20have,women%20compared%20to%2038.1%25%20men)\n",
    "\n",
    "[^2]: Lack of Representation by Women in State Legislatures Threatens Abortion Access. (n.d.). National Partnership for Women & Families. Retrieved February 12, 2024, from [source](https://nationalpartnership.org/news_post/lack-of-representation-women-state-legislatures-threatens-abortion-access/#:~:text=Of%20the%2024%20states%20and)\n",
    "\n",
    "[^3]: Hung, P., Casey, M. M., Kozhimannil, K. B., Karaca-Mandic, P., & Moscovice, I. S. (2018). [Rural-urban differences in access to hospital obstetric and neonatal care: how far is the closest one?](https://doi.org/10.1038/s41372-018-0063-5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Hypothesis\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Proportionately higher representation of females in state government, lower income level, and higher concentration of OBGYN  correlates to states with higher rates of abortion.\n",
    "We think that with higher representation and advocacy from women in governmental positions of power, this will be reflected through policies allowing for increased access to abortion. \n",
    "Additionally, areas with lower income may have financial burdens not suitable for raising a child, as well as limited access to affordable healthcare and contraceptives. Finally, a higher concentration of OBGYN will correlate to higher access to abortion services and healthcare professionals. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data overview\n",
    "\n",
    "For each dataset include the following information\n",
    "- Dataset #1\n",
    "  - Dataset Name:\n",
    "  - Link to the dataset:\n",
    "  - Number of observations:\n",
    "  - Number of variables:\n",
    "- Dataset #2 (if you have more than one!)\n",
    "  - Dataset Name:\n",
    "  - Link to the dataset:\n",
    "  - Number of observations:\n",
    "  - Number of variables:\n",
    "- etc\n",
    "\n",
    "Now write 2 - 5 sentences describing each dataset here. Include a short description of the important variables in the dataset; what the metrics and datatypes are, what concepts they may be proxies for. Include information about how you would need to wrangle/clean/preprocess the dataset\n",
    "\n",
    "If you plan to use multiple datasets, add a few sentences about how you plan to combine these datasets."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Dataset #1: Abortion Incidence**\n",
    "\n",
    "\n",
    "Link: https://www.guttmacher.org/report/abortion-incidence-service-availability-us-2017 \n",
    "\n",
    "\n",
    "Table 1 contains the number of reported abortions, abortion rate, and abortion ratio in the United States from the years 2000-2017. To clean this table we removed the brackets and commas so that the No. of abortions (in 000’s), Rate, and Ratio could be converted to floats. \n",
    "- `aisau1` Number of Observations: 18\n",
    "- `aisau1` Number of Variables: 4\n",
    "\n",
    "Table 2 contains the number of reported abortions and abortion rate in 2014, 2016, and 2017 by region and state. There is also a percentage change from 2014 to 2017. The only cleaning required for this dataset was removing the commas in the data. \n",
    "- `aisau2` Number of Observations: 56\n",
    "- `aisau2` Number of Variables: 8\n",
    "\n",
    "Table 3 provides information about the number and percentage of abortion providers by type and caseload for the years 2014 and 2017. We cleaned this data by removing the commas from the 'Facilities No. 2014', 'Facilities No. 2017', 'Abortions No. 2014',  and 'Abortions No. 2017' columns.\n",
    "- `aisau3` Number of Observations: 10\n",
    "- `aisau3` Number of Variables: 9\n",
    "\n",
    "Table 4 holds the total number of abortion clinics, the percentage of counties without a clinic, and percentage of women living in these counties, by state and region in the years 2014 and 2017. This dataset did not require any further cleaning since all the values were integers.\n",
    "- `aisau4` Number of Observations: 56\n",
    "- `aisau4` Number of Variables: 6\n",
    "\n",
    "Table 5 provides information about the number of abortion clinics in 2014 and 2017 and the percentage change between these years. Additionally, it contains the abortion rate in 2014 and 2017 and the change between these years. It is focused on the states with the proportionately largest decrease in number of clinics and increase in number of clinics. 'Abortion rate 2014', 'Abortion rate 2017' were cleaned by removing commas. \n",
    "- `aisau5` Number of Observations: 21\n",
    "- `aisau5` Number of Variables: 9\n",
    "\n",
    "Table 6 contains the number of medication abortions, among facilities offering only early medication abortions. We removed the commas and switched the objects to floats.  \n",
    "- `aisau6` Number of Observations: 8\n",
    "- `aisau6` Number of Variables: 4\n",
    "\n",
    "Some of the notable variables in this dataset include the abortion rate by state and region, as well as the number of abortion providers by time and region."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Dataset #2: Median Household Income by State**\n",
    "- Link: https://nces.ed.gov/programs/digest/d16/tables/dt16_102.30.asp\n",
    "- Number of Observations: 52\n",
    "- `mhis` (1990-2015) Number of Variables: 17\n",
    "- `mhis2`  (2015-2021) Number of Variables: 19\n",
    "\n",
    "This dataset contains information about the median household income in the United States as a whole, as well as broken down by individual states. It contains data for the years 1990- 2015, and 2015-2021 (excluding 2020) with the median for each year as well as the standard error for each year. To clean this dataset, we needed to remove the dollar signs (present in doubles $$) as well as the parentheses to convert appropriate variables to floats. Some of the key information we will utilize from this dataset is from the years 2014-2017 since that is when our first dataset is focused on."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Dataset #3: History of Women in the US Congress**\n",
    "\n",
    "- Link: https://cawp.rutgers.edu/facts/levels-office/congress/history-women-us-congress\n",
    "- `congress` Number of Observations: 54\n",
    "- `congress` Number of Variables: 6\n",
    "\n",
    "This dataset provides information on the number of women in the senate, women in the house, and total women in congress over 2 year terms. The information provided was an object with the number of women in each of these categories and then in parentheses specified if these were made up of democrats or republicans. We cleaned this data by splitting each of these 3 data points into 3 more subcategories to identify the number of democrat, republican, and total women in the Senate, House, and Congress.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Dataset #4: OB/GYN Rate Per State, 2019-2021**\n",
    "\n",
    "- Link: https://data.hrsa.gov/topics/health-workforce/ahrf\n",
    "- `obgyn1` Number of Observations: 60\n",
    "- `obgyn1` Number of Variables: 5\n",
    "- `obgyn2` Number of Observations: 60\n",
    "- `obgyn2` Number of Variables: 5\n",
    "- `obgyn3` Number of Observations: 60\n",
    "- `obgyn3` Number of Variables: 5\n",
    "\n",
    "This dataset provides information on the rate of OB/GYNs per state from 2019 to 2021. This rate is per 100,000 population, from the HRSA (Health Resources and Services Organization). The data from each year was downloaded as a separate csv file, cleaned to include only the state name and the rate for that year, and then merged into one dataset with all three years.\n",
    "\n",
    "*Note*: The years are not consistent with the rest of the data currently, but we have requested the source for data for years 2014-2017. The data will look very similar so the cleaning/merging should look exactly the same as it does right now."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**How we will combine the data:**\n",
    "\n",
    "We plan on combining the data by using `.merge()`. We will isolate the years we want to use (which will be 2014-2017 because we have consistent data for those years) and format them in a way that allows for `.merge()` to happen. This means we will expect to have repeating datapoints for the smaller tables. We might have to `.transpose()` DataFrames. This is because some of our columns are years and some of our rows are years. Another way we could work around this issue is by saving the Series or rows into arrays and re-assigning them via `.iloc()` or `.assign()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import os"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get the current working directory (allows for everyone to pull it without formatting issues)\n",
    "current_dir = os.getcwd()\n",
    "data_folder = os.path.join(current_dir, 'data')\n",
    "\n",
    "# The different CSV files we will be using file paths\n",
    "aisau1 = \"abortion-incidence-service-availability-us-2017-table_1.csv\"\n",
    "aisau2 = \"abortion-incidence-service-availability-us-2017-table_2.csv\"\n",
    "aisau3 = \"abortion-incidence-service-availability-us-2017-table_3.csv\"\n",
    "aisau4 = \"abortion-incidence-service-availability-us-2017-table_4.csv\"\n",
    "aisau5 = \"abortion-incidence-service-availability-us-2017-table_5.csv\"\n",
    "aisau6 = \"abortion-incidence-service-availability-us-2017-table_6.csv\"\n",
    "mhis = \"median-household-income-by-state.csv\"\n",
    "mhis2 = \"tabn102.30.csv\"\n",
    "congress = \"paragraph_1009_field_table_und_0.csv\"\n",
    "obgyn1 = \"AHRFDashboard_2019.csv\"\n",
    "obgyn2 = \"AHRFDashboard_2020.csv\"\n",
    "obgyn3 = \"AHRFDashboard_2021.csv\"\n",
    "\n",
    "# Using os to make the file path\n",
    "\n",
    "#These are from the Abortion Rates link\n",
    "aisau1_fp = os.path.join(data_folder, aisau1)\n",
    "aisau2_fp = os.path.join(data_folder, aisau2)\n",
    "aisau3_fp = os.path.join(data_folder, aisau3)\n",
    "aisau4_fp = os.path.join(data_folder, aisau4)\n",
    "aisau5_fp = os.path.join(data_folder, aisau5)\n",
    "aisau6_fp = os.path.join(data_folder, aisau6)\n",
    "\n",
    "#This is from the Median Household Income by State link\n",
    "mhis_fp = os.path.join(data_folder, mhis)\n",
    "mhis_fp2 = os.path.join(data_folder, mhis2)\n",
    "\n",
    "#This is from https://cawp.rutgers.edu/facts/levels-office/congress/history-women-us-congress\n",
    "congress_fp = os.path.join(data_folder, congress)\n",
    "\n",
    "#We still need more congress data and the OBGYN data\n",
    "obgyn1_fp = os.path.join(data_folder, obgyn1)\n",
    "obgyn2_fp = os.path.join(data_folder, obgyn2)\n",
    "obgyn3_fp = os.path.join(data_folder, obgyn3)\n",
    "\n",
    "# Making DataFrames\n",
    "aisau1 = pd.read_csv(aisau1_fp, sep = \"|\")\n",
    "aisau2 = pd.read_csv(aisau2_fp, sep = \"|\")\n",
    "aisau3 = pd.read_csv(aisau3_fp, sep = \"|\")\n",
    "aisau4 = pd.read_csv(aisau4_fp, sep = \"|\")\n",
    "aisau5 = pd.read_csv(aisau5_fp, sep = \"|\")\n",
    "aisau6 = pd.read_csv(aisau6_fp, sep = \"|\")\n",
    "mhis = pd.read_csv(mhis_fp, sep = \"|\")\n",
    "mhis2 = pd.read_csv(mhis_fp2, sep = \"|\")\n",
    "congress = pd.read_csv(congress_fp)\n",
    "obgyn1 = pd.read_csv(obgyn1_fp)\n",
    "obgyn2 = pd.read_csv(obgyn2_fp)\n",
    "obgyn3 = pd.read_csv(obgyn3_fp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cleaning Functions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def remove_commas(df, columns): \n",
    "    #removes commas from specified columns in a df & converts them to float\n",
    "    for column in columns: \n",
    "        df[column] = df[column].replace(',', '', regex = True).astype(float)\n",
    "        \n",
    "#use as such: remove_commas(df, ['column1', 'column2'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def remove_dollar_sign(df, columns): \n",
    "    #removes dollar signs ('$') from specified columns in a df \n",
    "    for column in columns: \n",
    "        df[column] = df[column].replace('[\\$,]', '', regex = True)\n",
    "\n",
    "#use as such: remove_dollar_sign(df, ['column1', 'column2'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def remove_parentheses(df, columns): \n",
    "    #removes parantheses from specified columns in a df\n",
    "    for column in columns: \n",
    "        df[column] = df[column].replace('[\\(\\)]', '', regex=True)\n",
    "        \n",
    "#use as such: remove_parentheses(df, ['column1', 'column2'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def remove_brackets(df, columns): \n",
    "    #removes brackets from specified columns in df \n",
    "    for column in columns: \n",
    "        df[column] = df[column].replace('[\\[\\]]', '', regex = True)\n",
    "        \n",
    "#use as such: remove_brackets(df, ['column1', 'column2'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset #1 Abortion Incidence"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Table 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(18, 4)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau1.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Year</th>\n",
       "      <th>No. (in 000s)</th>\n",
       "      <th>Rate</th>\n",
       "      <th>Ratio</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2000</td>\n",
       "      <td>1,313.0</td>\n",
       "      <td>21.3</td>\n",
       "      <td>24.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2001</td>\n",
       "      <td>[1,291.0]</td>\n",
       "      <td>[20.9]</td>\n",
       "      <td>[24.4]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2002</td>\n",
       "      <td>[1,269.0]</td>\n",
       "      <td>[20.5]</td>\n",
       "      <td>[23.8]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2003</td>\n",
       "      <td>[1,250.0]</td>\n",
       "      <td>[20.2]</td>\n",
       "      <td>[23.3]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2004</td>\n",
       "      <td>1,222.1</td>\n",
       "      <td>19.7</td>\n",
       "      <td>22.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Year No. (in 000s)    Rate   Ratio\n",
       "0  2000       1,313.0    21.3    24.5\n",
       "1  2001     [1,291.0]  [20.9]  [24.4]\n",
       "2  2002     [1,269.0]  [20.5]  [23.8]\n",
       "3  2003     [1,250.0]  [20.2]  [23.3]\n",
       "4  2004       1,222.1    19.7    22.9"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau1.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Year              int64\n",
       "No. (in 000s)    object\n",
       "Rate             object\n",
       "Ratio            object\n",
       "dtype: object"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau1.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "remove_brackets(aisau1, ['No. (in 000s)', 'Rate', 'Ratio'])\n",
    "remove_commas(aisau1, ['No. (in 000s)', 'Rate', 'Ratio'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Year</th>\n",
       "      <th>No. (in 000s)</th>\n",
       "      <th>Rate</th>\n",
       "      <th>Ratio</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2000</td>\n",
       "      <td>1313.0</td>\n",
       "      <td>21.3</td>\n",
       "      <td>24.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2001</td>\n",
       "      <td>1291.0</td>\n",
       "      <td>20.9</td>\n",
       "      <td>24.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2002</td>\n",
       "      <td>1269.0</td>\n",
       "      <td>20.5</td>\n",
       "      <td>23.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2003</td>\n",
       "      <td>1250.0</td>\n",
       "      <td>20.2</td>\n",
       "      <td>23.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2004</td>\n",
       "      <td>1222.1</td>\n",
       "      <td>19.7</td>\n",
       "      <td>22.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Year  No. (in 000s)  Rate  Ratio\n",
       "0  2000         1313.0  21.3   24.5\n",
       "1  2001         1291.0  20.9   24.4\n",
       "2  2002         1269.0  20.5   23.8\n",
       "3  2003         1250.0  20.2   23.3\n",
       "4  2004         1222.1  19.7   22.9"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau1.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Year               int64\n",
       "No. (in 000s)    float64\n",
       "Rate             float64\n",
       "Ratio            float64\n",
       "dtype: object"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau1.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# aisau1[\"No. (in 000s)\"] = aisau1[\"No. (in 000s)\"].apply(remove_brackets)\n",
    "# aisau1[\"Rate\"] = aisau1[\"Rate\"].apply(remove_brackets)\n",
    "# aisau1[\"Ratio\"] = aisau1[\"Ratio\"].apply(remove_brackets)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Table 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(56, 8)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau2.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Region and State</th>\n",
       "      <th>No. 2014</th>\n",
       "      <th>No. 2016</th>\n",
       "      <th>No. 2017</th>\n",
       "      <th>Rate 2014</th>\n",
       "      <th>Rate 2016</th>\n",
       "      <th>Rate 2017</th>\n",
       "      <th>% change 2014-2017</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>U.S. total</td>\n",
       "      <td>926,190</td>\n",
       "      <td>874,080</td>\n",
       "      <td>862,320</td>\n",
       "      <td>14.6</td>\n",
       "      <td>13.7</td>\n",
       "      <td>13.5</td>\n",
       "      <td>-8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Northeast</td>\n",
       "      <td>240,320</td>\n",
       "      <td>232,040</td>\n",
       "      <td>224,310</td>\n",
       "      <td>21.8</td>\n",
       "      <td>21.2</td>\n",
       "      <td>20.5</td>\n",
       "      <td>-6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Connecticut</td>\n",
       "      <td>13,140</td>\n",
       "      <td>12,210</td>\n",
       "      <td>11,910</td>\n",
       "      <td>19.2</td>\n",
       "      <td>18.1</td>\n",
       "      <td>17.7</td>\n",
       "      <td>-8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Maine</td>\n",
       "      <td>2,220</td>\n",
       "      <td>2,060</td>\n",
       "      <td>2,040</td>\n",
       "      <td>9.5</td>\n",
       "      <td>8.9</td>\n",
       "      <td>8.8</td>\n",
       "      <td>-7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Massachusetts</td>\n",
       "      <td>21,020</td>\n",
       "      <td>19,200</td>\n",
       "      <td>18,590</td>\n",
       "      <td>15.3</td>\n",
       "      <td>14.0</td>\n",
       "      <td>13.5</td>\n",
       "      <td>-12</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Region and State No. 2014 No. 2016 No. 2017  Rate 2014  Rate 2016  \\\n",
       "0       U.S. total  926,190  874,080  862,320       14.6       13.7   \n",
       "1        Northeast  240,320  232,040  224,310       21.8       21.2   \n",
       "2      Connecticut   13,140   12,210   11,910       19.2       18.1   \n",
       "3            Maine    2,220    2,060    2,040        9.5        8.9   \n",
       "4    Massachusetts   21,020   19,200   18,590       15.3       14.0   \n",
       "\n",
       "   Rate 2017  % change 2014-2017  \n",
       "0       13.5                  -8  \n",
       "1       20.5                  -6  \n",
       "2       17.7                  -8  \n",
       "3        8.8                  -7  \n",
       "4       13.5                 -12  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau2.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Region and State       object\n",
       "No. 2014               object\n",
       "No. 2016               object\n",
       "No. 2017               object\n",
       "Rate 2014             float64\n",
       "Rate 2016             float64\n",
       "Rate 2017             float64\n",
       "% change 2014-2017      int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau2.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "remove_commas(aisau2, [\"No. 2014\", \"No. 2016\", \"No. 2017\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Region and State</th>\n",
       "      <th>No. 2014</th>\n",
       "      <th>No. 2016</th>\n",
       "      <th>No. 2017</th>\n",
       "      <th>Rate 2014</th>\n",
       "      <th>Rate 2016</th>\n",
       "      <th>Rate 2017</th>\n",
       "      <th>% change 2014-2017</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>U.S. total</td>\n",
       "      <td>926190.0</td>\n",
       "      <td>874080.0</td>\n",
       "      <td>862320.0</td>\n",
       "      <td>14.6</td>\n",
       "      <td>13.7</td>\n",
       "      <td>13.5</td>\n",
       "      <td>-8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Northeast</td>\n",
       "      <td>240320.0</td>\n",
       "      <td>232040.0</td>\n",
       "      <td>224310.0</td>\n",
       "      <td>21.8</td>\n",
       "      <td>21.2</td>\n",
       "      <td>20.5</td>\n",
       "      <td>-6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Connecticut</td>\n",
       "      <td>13140.0</td>\n",
       "      <td>12210.0</td>\n",
       "      <td>11910.0</td>\n",
       "      <td>19.2</td>\n",
       "      <td>18.1</td>\n",
       "      <td>17.7</td>\n",
       "      <td>-8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Maine</td>\n",
       "      <td>2220.0</td>\n",
       "      <td>2060.0</td>\n",
       "      <td>2040.0</td>\n",
       "      <td>9.5</td>\n",
       "      <td>8.9</td>\n",
       "      <td>8.8</td>\n",
       "      <td>-7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Massachusetts</td>\n",
       "      <td>21020.0</td>\n",
       "      <td>19200.0</td>\n",
       "      <td>18590.0</td>\n",
       "      <td>15.3</td>\n",
       "      <td>14.0</td>\n",
       "      <td>13.5</td>\n",
       "      <td>-12</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Region and State  No. 2014  No. 2016  No. 2017  Rate 2014  Rate 2016  \\\n",
       "0       U.S. total  926190.0  874080.0  862320.0       14.6       13.7   \n",
       "1        Northeast  240320.0  232040.0  224310.0       21.8       21.2   \n",
       "2      Connecticut   13140.0   12210.0   11910.0       19.2       18.1   \n",
       "3            Maine    2220.0    2060.0    2040.0        9.5        8.9   \n",
       "4    Massachusetts   21020.0   19200.0   18590.0       15.3       14.0   \n",
       "\n",
       "   Rate 2017  % change 2014-2017  \n",
       "0       13.5                  -8  \n",
       "1       20.5                  -6  \n",
       "2       17.7                  -8  \n",
       "3        8.8                  -7  \n",
       "4       13.5                 -12  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau2.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Region and State       object\n",
       "No. 2014              float64\n",
       "No. 2016              float64\n",
       "No. 2017              float64\n",
       "Rate 2014             float64\n",
       "Rate 2016             float64\n",
       "Rate 2017             float64\n",
       "% change 2014-2017      int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau2.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Table 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10, 9)"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau3.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed</th>\n",
       "      <th>Facilities No. 2014</th>\n",
       "      <th>Facilities No. 2017</th>\n",
       "      <th>Facilities % 2014</th>\n",
       "      <th>Facilities % 2017</th>\n",
       "      <th>Abortions No. 2014</th>\n",
       "      <th>Abortions No. 2017</th>\n",
       "      <th>Abortions % 2014</th>\n",
       "      <th>Abortions % 2017</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Facility type</th>\n",
       "      <td>Total</td>\n",
       "      <td>1,671</td>\n",
       "      <td>1,587</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>926,190</td>\n",
       "      <td>862,320</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Facility type</th>\n",
       "      <td>Hospital</td>\n",
       "      <td>638</td>\n",
       "      <td>518</td>\n",
       "      <td>38</td>\n",
       "      <td>33</td>\n",
       "      <td>34,410</td>\n",
       "      <td>28,760</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Facility type</th>\n",
       "      <td>Abortion clinic</td>\n",
       "      <td>272</td>\n",
       "      <td>253</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>547,130</td>\n",
       "      <td>519,180</td>\n",
       "      <td>59</td>\n",
       "      <td>60</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Facility type</th>\n",
       "      <td>Nonspecialized clinic</td>\n",
       "      <td>517</td>\n",
       "      <td>555</td>\n",
       "      <td>31</td>\n",
       "      <td>35</td>\n",
       "      <td>331,790</td>\n",
       "      <td>302,860</td>\n",
       "      <td>36</td>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Facility type</th>\n",
       "      <td>Physicians' office</td>\n",
       "      <td>244</td>\n",
       "      <td>261</td>\n",
       "      <td>15</td>\n",
       "      <td>16</td>\n",
       "      <td>12,870</td>\n",
       "      <td>11,510</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                             Unnamed Facilities No. 2014 Facilities No. 2017  \\\n",
       "Facility type                  Total               1,671               1,587   \n",
       "Facility type               Hospital                 638                 518   \n",
       "Facility type        Abortion clinic                 272                 253   \n",
       "Facility type  Nonspecialized clinic                 517                 555   \n",
       "Facility type     Physicians' office                 244                 261   \n",
       "\n",
       "               Facilities % 2014  Facilities % 2017 Abortions No. 2014  \\\n",
       "Facility type                100                100            926,190   \n",
       "Facility type                 38                 33             34,410   \n",
       "Facility type                 16                 16            547,130   \n",
       "Facility type                 31                 35            331,790   \n",
       "Facility type                 15                 16             12,870   \n",
       "\n",
       "              Abortions No. 2017  Abortions % 2014  Abortions % 2017  \n",
       "Facility type            862,320               100               100  \n",
       "Facility type             28,760                 4                 3  \n",
       "Facility type            519,180                59                60  \n",
       "Facility type            302,860                36                35  \n",
       "Facility type             11,510                 1                 1  "
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau3.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Unnamed                object\n",
       "Facilities No. 2014    object\n",
       "Facilities No. 2017    object\n",
       "Facilities % 2014       int64\n",
       "Facilities % 2017       int64\n",
       "Abortions No. 2014     object\n",
       "Abortions No. 2017     object\n",
       "Abortions % 2014        int64\n",
       "Abortions % 2017        int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau3.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "remove_commas(aisau3, ['Facilities No. 2014', 'Facilities No. 2017', \n",
    "                       'Abortions No. 2014', 'Abortions No. 2017'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed</th>\n",
       "      <th>Facilities No. 2014</th>\n",
       "      <th>Facilities No. 2017</th>\n",
       "      <th>Facilities % 2014</th>\n",
       "      <th>Facilities % 2017</th>\n",
       "      <th>Abortions No. 2014</th>\n",
       "      <th>Abortions No. 2017</th>\n",
       "      <th>Abortions % 2014</th>\n",
       "      <th>Abortions % 2017</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Facility type</th>\n",
       "      <td>Total</td>\n",
       "      <td>1671.0</td>\n",
       "      <td>1587.0</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>926190.0</td>\n",
       "      <td>862320.0</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Facility type</th>\n",
       "      <td>Hospital</td>\n",
       "      <td>638.0</td>\n",
       "      <td>518.0</td>\n",
       "      <td>38</td>\n",
       "      <td>33</td>\n",
       "      <td>34410.0</td>\n",
       "      <td>28760.0</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Facility type</th>\n",
       "      <td>Abortion clinic</td>\n",
       "      <td>272.0</td>\n",
       "      <td>253.0</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>547130.0</td>\n",
       "      <td>519180.0</td>\n",
       "      <td>59</td>\n",
       "      <td>60</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Facility type</th>\n",
       "      <td>Nonspecialized clinic</td>\n",
       "      <td>517.0</td>\n",
       "      <td>555.0</td>\n",
       "      <td>31</td>\n",
       "      <td>35</td>\n",
       "      <td>331790.0</td>\n",
       "      <td>302860.0</td>\n",
       "      <td>36</td>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Facility type</th>\n",
       "      <td>Physicians' office</td>\n",
       "      <td>244.0</td>\n",
       "      <td>261.0</td>\n",
       "      <td>15</td>\n",
       "      <td>16</td>\n",
       "      <td>12870.0</td>\n",
       "      <td>11510.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                             Unnamed  Facilities No. 2014  \\\n",
       "Facility type                  Total               1671.0   \n",
       "Facility type               Hospital                638.0   \n",
       "Facility type        Abortion clinic                272.0   \n",
       "Facility type  Nonspecialized clinic                517.0   \n",
       "Facility type     Physicians' office                244.0   \n",
       "\n",
       "               Facilities No. 2017  Facilities % 2014  Facilities % 2017  \\\n",
       "Facility type               1587.0                100                100   \n",
       "Facility type                518.0                 38                 33   \n",
       "Facility type                253.0                 16                 16   \n",
       "Facility type                555.0                 31                 35   \n",
       "Facility type                261.0                 15                 16   \n",
       "\n",
       "               Abortions No. 2014  Abortions No. 2017  Abortions % 2014  \\\n",
       "Facility type            926190.0            862320.0               100   \n",
       "Facility type             34410.0             28760.0                 4   \n",
       "Facility type            547130.0            519180.0                59   \n",
       "Facility type            331790.0            302860.0                36   \n",
       "Facility type             12870.0             11510.0                 1   \n",
       "\n",
       "               Abortions % 2017  \n",
       "Facility type               100  \n",
       "Facility type                 3  \n",
       "Facility type                60  \n",
       "Facility type                35  \n",
       "Facility type                 1  "
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau3.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Unnamed                 object\n",
       "Facilities No. 2014    float64\n",
       "Facilities No. 2017    float64\n",
       "Facilities % 2014        int64\n",
       "Facilities % 2017        int64\n",
       "Abortions No. 2014     float64\n",
       "Abortions No. 2017     float64\n",
       "Abortions % 2014         int64\n",
       "Abortions % 2017         int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau3.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Table 4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(56, 6)"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau4.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Region and state</th>\n",
       "      <th>No. of clinics 2014</th>\n",
       "      <th>No. of clinics 2017</th>\n",
       "      <th>% change 2014-2017</th>\n",
       "      <th>% of counties without a clinic, 2017</th>\n",
       "      <th>% of women in counties with no clinic, 2017</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>U.S. total</td>\n",
       "      <td>789</td>\n",
       "      <td>808</td>\n",
       "      <td>2</td>\n",
       "      <td>89</td>\n",
       "      <td>38</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Northeast</td>\n",
       "      <td>212</td>\n",
       "      <td>245</td>\n",
       "      <td>16</td>\n",
       "      <td>54</td>\n",
       "      <td>21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Connecticut</td>\n",
       "      <td>25</td>\n",
       "      <td>26</td>\n",
       "      <td>4</td>\n",
       "      <td>13</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Maine</td>\n",
       "      <td>4</td>\n",
       "      <td>16</td>\n",
       "      <td>300</td>\n",
       "      <td>31</td>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Massachusetts</td>\n",
       "      <td>14</td>\n",
       "      <td>19</td>\n",
       "      <td>36</td>\n",
       "      <td>43</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Region and state  No. of clinics 2014  No. of clinics 2017  \\\n",
       "0       U.S. total                  789                  808   \n",
       "1        Northeast                  212                  245   \n",
       "2      Connecticut                   25                   26   \n",
       "3            Maine                    4                   16   \n",
       "4    Massachusetts                   14                   19   \n",
       "\n",
       "   % change 2014-2017  % of counties without a clinic, 2017  \\\n",
       "0                   2                                    89   \n",
       "1                  16                                    54   \n",
       "2                   4                                    13   \n",
       "3                 300                                    31   \n",
       "4                  36                                    43   \n",
       "\n",
       "   % of women in counties with no clinic, 2017  \n",
       "0                                           38  \n",
       "1                                           21  \n",
       "2                                            5  \n",
       "3                                           24  \n",
       "4                                           13  "
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau4.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Region and state                               object\n",
       "No. of clinics 2014                             int64\n",
       "No. of clinics 2017                             int64\n",
       "% change 2014-2017                              int64\n",
       "% of counties without a clinic, 2017            int64\n",
       "% of women in counties with no clinic, 2017     int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau4.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Table 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(21, 9)"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau5.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Prop Num of Clinics Decline/Increase</th>\n",
       "      <th>Location</th>\n",
       "      <th>No. of clinics 2014</th>\n",
       "      <th>No. of clinics 2017</th>\n",
       "      <th>No. of clinics % change</th>\n",
       "      <th>Abortion rate 2014</th>\n",
       "      <th>Abortion rate 2017</th>\n",
       "      <th>Abortion rate % change</th>\n",
       "      <th>Total no. of new restrictions in effect 2014–2017</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>U.S. total</td>\n",
       "      <td>789</td>\n",
       "      <td>808</td>\n",
       "      <td>2</td>\n",
       "      <td>14.6</td>\n",
       "      <td>13.5</td>\n",
       "      <td>-8</td>\n",
       "      <td>196</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10 states with proportionately largest decline...</td>\n",
       "      <td>Kentucky</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>-50</td>\n",
       "      <td>4.1</td>\n",
       "      <td>3.8</td>\n",
       "      <td>-9</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10 states with proportionately largest decline...</td>\n",
       "      <td>West Virginia</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>-50</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4.4</td>\n",
       "      <td>-26</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10 states with proportionately largest decline...</td>\n",
       "      <td>Indiana</td>\n",
       "      <td>9</td>\n",
       "      <td>6</td>\n",
       "      <td>-33</td>\n",
       "      <td>6.3</td>\n",
       "      <td>5.9</td>\n",
       "      <td>-6</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10 states with proportionately largest decline...</td>\n",
       "      <td>Iowa</td>\n",
       "      <td>12</td>\n",
       "      <td>8</td>\n",
       "      <td>-33</td>\n",
       "      <td>7.5</td>\n",
       "      <td>6.3</td>\n",
       "      <td>-15</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                Prop Num of Clinics Decline/Increase       Location  \\\n",
       "0                                                NaN     U.S. total   \n",
       "1  10 states with proportionately largest decline...       Kentucky   \n",
       "2  10 states with proportionately largest decline...  West Virginia   \n",
       "3  10 states with proportionately largest decline...        Indiana   \n",
       "4  10 states with proportionately largest decline...           Iowa   \n",
       "\n",
       "   No. of clinics 2014  No. of clinics 2017  No. of clinics % change  \\\n",
       "0                  789                  808                        2   \n",
       "1                    2                    1                      -50   \n",
       "2                    2                    1                      -50   \n",
       "3                    9                    6                      -33   \n",
       "4                   12                    8                      -33   \n",
       "\n",
       "   Abortion rate 2014  Abortion rate 2017  Abortion rate % change  \\\n",
       "0                14.6                13.5                      -8   \n",
       "1                 4.1                 3.8                      -9   \n",
       "2                 6.0                 4.4                     -26   \n",
       "3                 6.3                 5.9                      -6   \n",
       "4                 7.5                 6.3                     -15   \n",
       "\n",
       "   Total no. of new restrictions in effect 2014–2017  \n",
       "0                                                196  \n",
       "1                                                  3  \n",
       "2                                                  6  \n",
       "3                                                 20  \n",
       "4                                                  9  "
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau5.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Prop Num of Clinics Decline/Increase                  object\n",
       "Location                                              object\n",
       "No. of clinics 2014                                    int64\n",
       "No. of clinics 2017                                    int64\n",
       "No. of clinics % change                                int64\n",
       "Abortion rate 2014                                   float64\n",
       "Abortion rate 2017                                   float64\n",
       "Abortion rate % change                                 int64\n",
       "Total no. of new restrictions in effect 2014–2017      int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau5.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "remove_commas(aisau5, ['Abortion rate 2014', 'Abortion rate 2017'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Prop Num of Clinics Decline/Increase</th>\n",
       "      <th>Location</th>\n",
       "      <th>No. of clinics 2014</th>\n",
       "      <th>No. of clinics 2017</th>\n",
       "      <th>No. of clinics % change</th>\n",
       "      <th>Abortion rate 2014</th>\n",
       "      <th>Abortion rate 2017</th>\n",
       "      <th>Abortion rate % change</th>\n",
       "      <th>Total no. of new restrictions in effect 2014–2017</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>U.S. total</td>\n",
       "      <td>789</td>\n",
       "      <td>808</td>\n",
       "      <td>2</td>\n",
       "      <td>14.6</td>\n",
       "      <td>13.5</td>\n",
       "      <td>-8</td>\n",
       "      <td>196</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10 states with proportionately largest decline...</td>\n",
       "      <td>Kentucky</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>-50</td>\n",
       "      <td>4.1</td>\n",
       "      <td>3.8</td>\n",
       "      <td>-9</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10 states with proportionately largest decline...</td>\n",
       "      <td>West Virginia</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>-50</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4.4</td>\n",
       "      <td>-26</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10 states with proportionately largest decline...</td>\n",
       "      <td>Indiana</td>\n",
       "      <td>9</td>\n",
       "      <td>6</td>\n",
       "      <td>-33</td>\n",
       "      <td>6.3</td>\n",
       "      <td>5.9</td>\n",
       "      <td>-6</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10 states with proportionately largest decline...</td>\n",
       "      <td>Iowa</td>\n",
       "      <td>12</td>\n",
       "      <td>8</td>\n",
       "      <td>-33</td>\n",
       "      <td>7.5</td>\n",
       "      <td>6.3</td>\n",
       "      <td>-15</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                Prop Num of Clinics Decline/Increase       Location  \\\n",
       "0                                                NaN     U.S. total   \n",
       "1  10 states with proportionately largest decline...       Kentucky   \n",
       "2  10 states with proportionately largest decline...  West Virginia   \n",
       "3  10 states with proportionately largest decline...        Indiana   \n",
       "4  10 states with proportionately largest decline...           Iowa   \n",
       "\n",
       "   No. of clinics 2014  No. of clinics 2017  No. of clinics % change  \\\n",
       "0                  789                  808                        2   \n",
       "1                    2                    1                      -50   \n",
       "2                    2                    1                      -50   \n",
       "3                    9                    6                      -33   \n",
       "4                   12                    8                      -33   \n",
       "\n",
       "   Abortion rate 2014  Abortion rate 2017  Abortion rate % change  \\\n",
       "0                14.6                13.5                      -8   \n",
       "1                 4.1                 3.8                      -9   \n",
       "2                 6.0                 4.4                     -26   \n",
       "3                 6.3                 5.9                      -6   \n",
       "4                 7.5                 6.3                     -15   \n",
       "\n",
       "   Total no. of new restrictions in effect 2014–2017  \n",
       "0                                                196  \n",
       "1                                                  3  \n",
       "2                                                  6  \n",
       "3                                                 20  \n",
       "4                                                  9  "
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau5.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Table 6"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(8, 4)"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau6.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed</th>\n",
       "      <th>No. of medication abortions</th>\n",
       "      <th>Offering only early medication abortion No.</th>\n",
       "      <th>Offering only early medication abortion % of all nonhospital providers</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>NaN</th>\n",
       "      <td>U.S. total</td>\n",
       "      <td>339,640</td>\n",
       "      <td>269</td>\n",
       "      <td>25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Facility type</th>\n",
       "      <td>Physicians' office</td>\n",
       "      <td>5,020</td>\n",
       "      <td>30</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Facility type</th>\n",
       "      <td>Nonspecialized clinic</td>\n",
       "      <td>153,350</td>\n",
       "      <td>229</td>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Facility type</th>\n",
       "      <td>Abortion clinic</td>\n",
       "      <td>181,280</td>\n",
       "      <td>10</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Facility caseload</th>\n",
       "      <td>1-29</td>\n",
       "      <td>1,230</td>\n",
       "      <td>57</td>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                 Unnamed No. of medication abortions  \\\n",
       "NaN                           U.S. total                     339,640   \n",
       "Facility type         Physicians' office                       5,020   \n",
       "Facility type      Nonspecialized clinic                     153,350   \n",
       "Facility type            Abortion clinic                     181,280   \n",
       "Facility caseload                   1-29                       1,230   \n",
       "\n",
       "                   Offering only early medication abortion No.  \\\n",
       "NaN                                                        269   \n",
       "Facility type                                               30   \n",
       "Facility type                                              229   \n",
       "Facility type                                               10   \n",
       "Facility caseload                                           57   \n",
       "\n",
       "                   Offering only early medication abortion % of all nonhospital providers  \n",
       "NaN                                                               25                       \n",
       "Facility type                                                     11                       \n",
       "Facility type                                                     41                       \n",
       "Facility type                                                      4                       \n",
       "Facility caseload                                                 24                       "
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau6.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Unnamed                                                                   object\n",
       "No. of medication abortions                                               object\n",
       "Offering only early medication abortion No.                                int64\n",
       "Offering only early medication abortion % of all nonhospital providers     int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau6.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "remove_commas(aisau6, ['No. of medication abortions'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed</th>\n",
       "      <th>No. of medication abortions</th>\n",
       "      <th>Offering only early medication abortion No.</th>\n",
       "      <th>Offering only early medication abortion % of all nonhospital providers</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>NaN</th>\n",
       "      <td>U.S. total</td>\n",
       "      <td>339640.0</td>\n",
       "      <td>269</td>\n",
       "      <td>25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Facility type</th>\n",
       "      <td>Physicians' office</td>\n",
       "      <td>5020.0</td>\n",
       "      <td>30</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Facility type</th>\n",
       "      <td>Nonspecialized clinic</td>\n",
       "      <td>153350.0</td>\n",
       "      <td>229</td>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Facility type</th>\n",
       "      <td>Abortion clinic</td>\n",
       "      <td>181280.0</td>\n",
       "      <td>10</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Facility caseload</th>\n",
       "      <td>1-29</td>\n",
       "      <td>1230.0</td>\n",
       "      <td>57</td>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                 Unnamed  No. of medication abortions  \\\n",
       "NaN                           U.S. total                     339640.0   \n",
       "Facility type         Physicians' office                       5020.0   \n",
       "Facility type      Nonspecialized clinic                     153350.0   \n",
       "Facility type            Abortion clinic                     181280.0   \n",
       "Facility caseload                   1-29                       1230.0   \n",
       "\n",
       "                   Offering only early medication abortion No.  \\\n",
       "NaN                                                        269   \n",
       "Facility type                                               30   \n",
       "Facility type                                              229   \n",
       "Facility type                                               10   \n",
       "Facility caseload                                           57   \n",
       "\n",
       "                   Offering only early medication abortion % of all nonhospital providers  \n",
       "NaN                                                               25                       \n",
       "Facility type                                                     11                       \n",
       "Facility type                                                     41                       \n",
       "Facility type                                                      4                       \n",
       "Facility caseload                                                 24                       "
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau6.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Unnamed                                                                    object\n",
       "No. of medication abortions                                               float64\n",
       "Offering only early medication abortion No.                                 int64\n",
       "Offering only early medication abortion % of all nonhospital providers      int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aisau6.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset #2 Median household income, by state: Selected years, 1990 through 2015"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Up to 2015"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(52, 17)"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mhis.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>State</th>\n",
       "      <th>1990</th>\n",
       "      <th>2000</th>\n",
       "      <th>2005</th>\n",
       "      <th>2005 (SE)</th>\n",
       "      <th>2010</th>\n",
       "      <th>2010 (SE)</th>\n",
       "      <th>2011</th>\n",
       "      <th>2011 (SE)</th>\n",
       "      <th>2012</th>\n",
       "      <th>2012 (SE)</th>\n",
       "      <th>2013</th>\n",
       "      <th>2013 (SE)</th>\n",
       "      <th>2014</th>\n",
       "      <th>2014 (SE)</th>\n",
       "      <th>2015</th>\n",
       "      <th>2015 (SE)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>United States</td>\n",
       "      <td>$$55,500</td>\n",
       "      <td>$$59,800</td>\n",
       "      <td>$$56,100</td>\n",
       "      <td>($$80)</td>\n",
       "      <td>$$54,400</td>\n",
       "      <td>($$40)</td>\n",
       "      <td>$$53,200</td>\n",
       "      <td>($$50)</td>\n",
       "      <td>$$53,000</td>\n",
       "      <td>($$30)</td>\n",
       "      <td>$$53,200</td>\n",
       "      <td>($$40)</td>\n",
       "      <td>$$53,700</td>\n",
       "      <td>($$60)</td>\n",
       "      <td>$$55,800</td>\n",
       "      <td>($$50)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Alabama</td>\n",
       "      <td>$$43,600</td>\n",
       "      <td>$$48,600</td>\n",
       "      <td>$$44,800</td>\n",
       "      <td>($$390)</td>\n",
       "      <td>$$44,000</td>\n",
       "      <td>($$310)</td>\n",
       "      <td>$$43,600</td>\n",
       "      <td>($$350)</td>\n",
       "      <td>$$42,900</td>\n",
       "      <td>($$300)</td>\n",
       "      <td>$$43,600</td>\n",
       "      <td>($$400)</td>\n",
       "      <td>$$42,900</td>\n",
       "      <td>($$360)</td>\n",
       "      <td>$$44,800</td>\n",
       "      <td>($$450)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Alaska</td>\n",
       "      <td>$$76,400</td>\n",
       "      <td>$$73,400</td>\n",
       "      <td>$$68,200</td>\n",
       "      <td>($$1,330)</td>\n",
       "      <td>$$70,200</td>\n",
       "      <td>($$1,380)</td>\n",
       "      <td>$$71,500</td>\n",
       "      <td>($$1,250)</td>\n",
       "      <td>$$69,900</td>\n",
       "      <td>($$1,190)</td>\n",
       "      <td>$$73,500</td>\n",
       "      <td>($$1,170)</td>\n",
       "      <td>$$71,700</td>\n",
       "      <td>($$1,090)</td>\n",
       "      <td>$$73,400</td>\n",
       "      <td>($$1,410)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Arizona</td>\n",
       "      <td>$$50,800</td>\n",
       "      <td>$$57,700</td>\n",
       "      <td>$$53,700</td>\n",
       "      <td>($$480)</td>\n",
       "      <td>$$50,900</td>\n",
       "      <td>($$350)</td>\n",
       "      <td>$$49,200</td>\n",
       "      <td>($$350)</td>\n",
       "      <td>$$49,400</td>\n",
       "      <td>($$380)</td>\n",
       "      <td>$$49,400</td>\n",
       "      <td>($$360)</td>\n",
       "      <td>$$50,100</td>\n",
       "      <td>($$290)</td>\n",
       "      <td>$$51,500</td>\n",
       "      <td>($$260)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Arkansas</td>\n",
       "      <td>$$39,000</td>\n",
       "      <td>$$45,800</td>\n",
       "      <td>$$42,500</td>\n",
       "      <td>($$440)</td>\n",
       "      <td>$$41,600</td>\n",
       "      <td>($$420)</td>\n",
       "      <td>$$40,800</td>\n",
       "      <td>($$490)</td>\n",
       "      <td>$$41,400</td>\n",
       "      <td>($$310)</td>\n",
       "      <td>$$41,200</td>\n",
       "      <td>($$440)</td>\n",
       "      <td>$$41,300</td>\n",
       "      <td>($$400)</td>\n",
       "      <td>$$42,000</td>\n",
       "      <td>($$290)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           State      1990      2000      2005  2005 (SE)      2010  \\\n",
       "0  United States  $$55,500  $$59,800  $$56,100     ($$80)  $$54,400   \n",
       "1        Alabama  $$43,600  $$48,600  $$44,800    ($$390)  $$44,000   \n",
       "2         Alaska  $$76,400  $$73,400  $$68,200  ($$1,330)  $$70,200   \n",
       "3        Arizona  $$50,800  $$57,700  $$53,700    ($$480)  $$50,900   \n",
       "4       Arkansas  $$39,000  $$45,800  $$42,500    ($$440)  $$41,600   \n",
       "\n",
       "   2010 (SE)      2011  2011 (SE)      2012  2012 (SE)      2013  2013 (SE)  \\\n",
       "0     ($$40)  $$53,200     ($$50)  $$53,000     ($$30)  $$53,200     ($$40)   \n",
       "1    ($$310)  $$43,600    ($$350)  $$42,900    ($$300)  $$43,600    ($$400)   \n",
       "2  ($$1,380)  $$71,500  ($$1,250)  $$69,900  ($$1,190)  $$73,500  ($$1,170)   \n",
       "3    ($$350)  $$49,200    ($$350)  $$49,400    ($$380)  $$49,400    ($$360)   \n",
       "4    ($$420)  $$40,800    ($$490)  $$41,400    ($$310)  $$41,200    ($$440)   \n",
       "\n",
       "       2014  2014 (SE)      2015  2015 (SE)  \n",
       "0  $$53,700     ($$60)  $$55,800     ($$50)  \n",
       "1  $$42,900    ($$360)  $$44,800    ($$450)  \n",
       "2  $$71,700  ($$1,090)  $$73,400  ($$1,410)  \n",
       "3  $$50,100    ($$290)  $$51,500    ($$260)  \n",
       "4  $$41,300    ($$400)  $$42,000    ($$290)  "
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mhis.head() #SE means standard error"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "State        object\n",
       "1990         object\n",
       "2000         object\n",
       "2005         object\n",
       "2005 (SE)    object\n",
       "2010         object\n",
       "2010 (SE)    object\n",
       "2011         object\n",
       "2011 (SE)    object\n",
       "2012         object\n",
       "2012 (SE)    object\n",
       "2013         object\n",
       "2013 (SE)    object\n",
       "2014         object\n",
       "2014 (SE)    object\n",
       "2015         object\n",
       "2015 (SE)    object\n",
       "dtype: object"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mhis.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "remove_dollar_sign(mhis, ['1990', '2000', '2005', '2005 (SE)', '2010', \n",
    "                         '2010 (SE)', '2011', '2011 (SE)', '2012', '2012 (SE)', \n",
    "                         '2013', '2013 (SE)', '2014', '2014 (SE)', '2015', '2015 (SE)'])\n",
    "\n",
    "remove_parentheses(mhis, ['2005 (SE)', '2010 (SE)', '2011 (SE)', '2012 (SE)', \n",
    "                          '2013 (SE)', '2014 (SE)', '2015 (SE)'])\n",
    "\n",
    "mhis = mhis.astype(float, errors = 'ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>State</th>\n",
       "      <th>1990</th>\n",
       "      <th>2000</th>\n",
       "      <th>2005</th>\n",
       "      <th>2005 (SE)</th>\n",
       "      <th>2010</th>\n",
       "      <th>2010 (SE)</th>\n",
       "      <th>2011</th>\n",
       "      <th>2011 (SE)</th>\n",
       "      <th>2012</th>\n",
       "      <th>2012 (SE)</th>\n",
       "      <th>2013</th>\n",
       "      <th>2013 (SE)</th>\n",
       "      <th>2014</th>\n",
       "      <th>2014 (SE)</th>\n",
       "      <th>2015</th>\n",
       "      <th>2015 (SE)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>United States</td>\n",
       "      <td>55500.0</td>\n",
       "      <td>59800.0</td>\n",
       "      <td>56100.0</td>\n",
       "      <td>80.0</td>\n",
       "      <td>54400.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>53200.0</td>\n",
       "      <td>50.0</td>\n",
       "      <td>53000.0</td>\n",
       "      <td>30.0</td>\n",
       "      <td>53200.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>53700.0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>55800.0</td>\n",
       "      <td>50.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Alabama</td>\n",
       "      <td>43600.0</td>\n",
       "      <td>48600.0</td>\n",
       "      <td>44800.0</td>\n",
       "      <td>390.0</td>\n",
       "      <td>44000.0</td>\n",
       "      <td>310.0</td>\n",
       "      <td>43600.0</td>\n",
       "      <td>350.0</td>\n",
       "      <td>42900.0</td>\n",
       "      <td>300.0</td>\n",
       "      <td>43600.0</td>\n",
       "      <td>400.0</td>\n",
       "      <td>42900.0</td>\n",
       "      <td>360.0</td>\n",
       "      <td>44800.0</td>\n",
       "      <td>450.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Alaska</td>\n",
       "      <td>76400.0</td>\n",
       "      <td>73400.0</td>\n",
       "      <td>68200.0</td>\n",
       "      <td>1330.0</td>\n",
       "      <td>70200.0</td>\n",
       "      <td>1380.0</td>\n",
       "      <td>71500.0</td>\n",
       "      <td>1250.0</td>\n",
       "      <td>69900.0</td>\n",
       "      <td>1190.0</td>\n",
       "      <td>73500.0</td>\n",
       "      <td>1170.0</td>\n",
       "      <td>71700.0</td>\n",
       "      <td>1090.0</td>\n",
       "      <td>73400.0</td>\n",
       "      <td>1410.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Arizona</td>\n",
       "      <td>50800.0</td>\n",
       "      <td>57700.0</td>\n",
       "      <td>53700.0</td>\n",
       "      <td>480.0</td>\n",
       "      <td>50900.0</td>\n",
       "      <td>350.0</td>\n",
       "      <td>49200.0</td>\n",
       "      <td>350.0</td>\n",
       "      <td>49400.0</td>\n",
       "      <td>380.0</td>\n",
       "      <td>49400.0</td>\n",
       "      <td>360.0</td>\n",
       "      <td>50100.0</td>\n",
       "      <td>290.0</td>\n",
       "      <td>51500.0</td>\n",
       "      <td>260.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Arkansas</td>\n",
       "      <td>39000.0</td>\n",
       "      <td>45800.0</td>\n",
       "      <td>42500.0</td>\n",
       "      <td>440.0</td>\n",
       "      <td>41600.0</td>\n",
       "      <td>420.0</td>\n",
       "      <td>40800.0</td>\n",
       "      <td>490.0</td>\n",
       "      <td>41400.0</td>\n",
       "      <td>310.0</td>\n",
       "      <td>41200.0</td>\n",
       "      <td>440.0</td>\n",
       "      <td>41300.0</td>\n",
       "      <td>400.0</td>\n",
       "      <td>42000.0</td>\n",
       "      <td>290.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           State     1990     2000     2005  2005 (SE)     2010  2010 (SE)  \\\n",
       "0  United States  55500.0  59800.0  56100.0       80.0  54400.0       40.0   \n",
       "1        Alabama  43600.0  48600.0  44800.0      390.0  44000.0      310.0   \n",
       "2         Alaska  76400.0  73400.0  68200.0     1330.0  70200.0     1380.0   \n",
       "3        Arizona  50800.0  57700.0  53700.0      480.0  50900.0      350.0   \n",
       "4       Arkansas  39000.0  45800.0  42500.0      440.0  41600.0      420.0   \n",
       "\n",
       "      2011  2011 (SE)     2012  2012 (SE)     2013  2013 (SE)     2014  \\\n",
       "0  53200.0       50.0  53000.0       30.0  53200.0       40.0  53700.0   \n",
       "1  43600.0      350.0  42900.0      300.0  43600.0      400.0  42900.0   \n",
       "2  71500.0     1250.0  69900.0     1190.0  73500.0     1170.0  71700.0   \n",
       "3  49200.0      350.0  49400.0      380.0  49400.0      360.0  50100.0   \n",
       "4  40800.0      490.0  41400.0      310.0  41200.0      440.0  41300.0   \n",
       "\n",
       "   2014 (SE)     2015  2015 (SE)  \n",
       "0       60.0  55800.0       50.0  \n",
       "1      360.0  44800.0      450.0  \n",
       "2     1090.0  73400.0     1410.0  \n",
       "3      290.0  51500.0      260.0  \n",
       "4      400.0  42000.0      290.0  "
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mhis.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "State         object\n",
       "1990         float64\n",
       "2000         float64\n",
       "2005         float64\n",
       "2005 (SE)    float64\n",
       "2010         float64\n",
       "2010 (SE)    float64\n",
       "2011         float64\n",
       "2011 (SE)    float64\n",
       "2012         float64\n",
       "2012 (SE)    float64\n",
       "2013         float64\n",
       "2013 (SE)    float64\n",
       "2014         float64\n",
       "2014 (SE)    float64\n",
       "2015         float64\n",
       "2015 (SE)    float64\n",
       "dtype: object"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mhis.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2015 to 2021 (Missing 2020)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(52, 19)"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mhis2.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>State</th>\n",
       "      <th>1990</th>\n",
       "      <th>2000</th>\n",
       "      <th>2005</th>\n",
       "      <th>2005 (SE)</th>\n",
       "      <th>2010</th>\n",
       "      <th>2010 (SE)</th>\n",
       "      <th>2015</th>\n",
       "      <th>2015 (SE)</th>\n",
       "      <th>2016</th>\n",
       "      <th>2016 (SE)</th>\n",
       "      <th>2017</th>\n",
       "      <th>2017 (SE)</th>\n",
       "      <th>2018</th>\n",
       "      <th>2018 (SE)</th>\n",
       "      <th>2019</th>\n",
       "      <th>2019 (SE)</th>\n",
       "      <th>2021</th>\n",
       "      <th>2021 (SE)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>United States</td>\n",
       "      <td>$63,600</td>\n",
       "      <td>$68,500</td>\n",
       "      <td>$64,300</td>\n",
       "      <td>(90)</td>\n",
       "      <td>$62,300</td>\n",
       "      <td>(50)</td>\n",
       "      <td>$63,800</td>\n",
       "      <td>(60)</td>\n",
       "      <td>$65,100</td>\n",
       "      <td>(80)</td>\n",
       "      <td>$66,700</td>\n",
       "      <td>(60)</td>\n",
       "      <td>$66,800</td>\n",
       "      <td>(60)</td>\n",
       "      <td>$69,600</td>\n",
       "      <td>(80)</td>\n",
       "      <td>$69,700</td>\n",
       "      <td>(80)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Alabama</td>\n",
       "      <td>$49,900</td>\n",
       "      <td>$55,700</td>\n",
       "      <td>$51,300</td>\n",
       "      <td>(450)</td>\n",
       "      <td>$50,400</td>\n",
       "      <td>(360)</td>\n",
       "      <td>$51,200</td>\n",
       "      <td>(520)</td>\n",
       "      <td>$52,200</td>\n",
       "      <td>(460)</td>\n",
       "      <td>$53,200</td>\n",
       "      <td>(520)</td>\n",
       "      <td>$53,800</td>\n",
       "      <td>(510)</td>\n",
       "      <td>$54,800</td>\n",
       "      <td>(390)</td>\n",
       "      <td>$53,900</td>\n",
       "      <td>(470)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Alaska</td>\n",
       "      <td>$87,600</td>\n",
       "      <td>$84,100</td>\n",
       "      <td>$78,200</td>\n",
       "      <td>(1,530)</td>\n",
       "      <td>$80,400</td>\n",
       "      <td>(1,580)</td>\n",
       "      <td>$83,900</td>\n",
       "      <td>(1,610)</td>\n",
       "      <td>$86,300</td>\n",
       "      <td>(1,530)</td>\n",
       "      <td>$80,900</td>\n",
       "      <td>(1,770)</td>\n",
       "      <td>$80,200</td>\n",
       "      <td>(1,500)</td>\n",
       "      <td>$80,000</td>\n",
       "      <td>(1,740)</td>\n",
       "      <td>$77,800</td>\n",
       "      <td>(1,310)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Arizona</td>\n",
       "      <td>$58,300</td>\n",
       "      <td>$66,200</td>\n",
       "      <td>$61,600</td>\n",
       "      <td>(550)</td>\n",
       "      <td>$58,300</td>\n",
       "      <td>(400)</td>\n",
       "      <td>$58,900</td>\n",
       "      <td>(300)</td>\n",
       "      <td>$60,500</td>\n",
       "      <td>(440)</td>\n",
       "      <td>$62,500</td>\n",
       "      <td>(350)</td>\n",
       "      <td>$63,900</td>\n",
       "      <td>(480)</td>\n",
       "      <td>$65,800</td>\n",
       "      <td>(290)</td>\n",
       "      <td>$69,100</td>\n",
       "      <td>(420)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Arkansas</td>\n",
       "      <td>$44,700</td>\n",
       "      <td>$52,500</td>\n",
       "      <td>$48,700</td>\n",
       "      <td>(510)</td>\n",
       "      <td>$47,700</td>\n",
       "      <td>(490)</td>\n",
       "      <td>$48,000</td>\n",
       "      <td>(330)</td>\n",
       "      <td>$50,100</td>\n",
       "      <td>(630)</td>\n",
       "      <td>$50,700</td>\n",
       "      <td>(510)</td>\n",
       "      <td>$50,800</td>\n",
       "      <td>(470)</td>\n",
       "      <td>$51,900</td>\n",
       "      <td>(560)</td>\n",
       "      <td>$52,500</td>\n",
       "      <td>(610)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           State     1990     2000     2005 2005 (SE)     2010 2010 (SE)  \\\n",
       "0  United States  $63,600  $68,500  $64,300      (90)  $62,300      (50)   \n",
       "1        Alabama  $49,900  $55,700  $51,300     (450)  $50,400     (360)   \n",
       "2         Alaska  $87,600  $84,100  $78,200   (1,530)  $80,400   (1,580)   \n",
       "3        Arizona  $58,300  $66,200  $61,600     (550)  $58,300     (400)   \n",
       "4       Arkansas  $44,700  $52,500  $48,700     (510)  $47,700     (490)   \n",
       "\n",
       "      2015 2015 (SE)     2016 2016 (SE)     2017 2017 (SE)     2018 2018 (SE)  \\\n",
       "0  $63,800      (60)  $65,100      (80)  $66,700      (60)  $66,800      (60)   \n",
       "1  $51,200     (520)  $52,200     (460)  $53,200     (520)  $53,800     (510)   \n",
       "2  $83,900   (1,610)  $86,300   (1,530)  $80,900   (1,770)  $80,200   (1,500)   \n",
       "3  $58,900     (300)  $60,500     (440)  $62,500     (350)  $63,900     (480)   \n",
       "4  $48,000     (330)  $50,100     (630)  $50,700     (510)  $50,800     (470)   \n",
       "\n",
       "      2019 2019 (SE)     2021 2021 (SE)  \n",
       "0  $69,600      (80)  $69,700      (80)  \n",
       "1  $54,800     (390)  $53,900     (470)  \n",
       "2  $80,000   (1,740)  $77,800   (1,310)  \n",
       "3  $65,800     (290)  $69,100     (420)  \n",
       "4  $51,900     (560)  $52,500     (610)  "
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mhis2.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "State        object\n",
       "1990         object\n",
       "2000         object\n",
       "2005         object\n",
       "2005 (SE)    object\n",
       "2010         object\n",
       "2010 (SE)    object\n",
       "2015         object\n",
       "2015 (SE)    object\n",
       "2016         object\n",
       "2016 (SE)    object\n",
       "2017         object\n",
       "2017 (SE)    object\n",
       "2018         object\n",
       "2018 (SE)    object\n",
       "2019         object\n",
       "2019 (SE)    object\n",
       "2021         object\n",
       "2021 (SE)    object\n",
       "dtype: object"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mhis2.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "remove_dollar_sign(mhis2, ['1990', '2000', '2005', '2005 (SE)', '2010', '2010 (SE)', \n",
    "                          '2015', '2015 (SE)', '2016', '2016 (SE)', '2017', '2017 (SE)', \n",
    "                          '2018', '2018 (SE)', '2019', '2019 (SE)', '2021', '2021 (SE)'])\n",
    "\n",
    "remove_parentheses(mhis2, ['2005 (SE)', '2010 (SE)', '2015 (SE)','2016 (SE)','2017 (SE)', \n",
    "                          '2018 (SE)','2019 (SE)','2021 (SE)'])\n",
    "\n",
    "mhis2 = mhis2.astype(float, errors = 'ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>State</th>\n",
       "      <th>1990</th>\n",
       "      <th>2000</th>\n",
       "      <th>2005</th>\n",
       "      <th>2005 (SE)</th>\n",
       "      <th>2010</th>\n",
       "      <th>2010 (SE)</th>\n",
       "      <th>2015</th>\n",
       "      <th>2015 (SE)</th>\n",
       "      <th>2016</th>\n",
       "      <th>2016 (SE)</th>\n",
       "      <th>2017</th>\n",
       "      <th>2017 (SE)</th>\n",
       "      <th>2018</th>\n",
       "      <th>2018 (SE)</th>\n",
       "      <th>2019</th>\n",
       "      <th>2019 (SE)</th>\n",
       "      <th>2021</th>\n",
       "      <th>2021 (SE)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>United States</td>\n",
       "      <td>63600.0</td>\n",
       "      <td>68500.0</td>\n",
       "      <td>64300.0</td>\n",
       "      <td>90.0</td>\n",
       "      <td>62300.0</td>\n",
       "      <td>50.0</td>\n",
       "      <td>63800.0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>65100.0</td>\n",
       "      <td>80.0</td>\n",
       "      <td>66700.0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>66800.0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>69600.0</td>\n",
       "      <td>80.0</td>\n",
       "      <td>69700.0</td>\n",
       "      <td>80.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Alabama</td>\n",
       "      <td>49900.0</td>\n",
       "      <td>55700.0</td>\n",
       "      <td>51300.0</td>\n",
       "      <td>450.0</td>\n",
       "      <td>50400.0</td>\n",
       "      <td>360.0</td>\n",
       "      <td>51200.0</td>\n",
       "      <td>520.0</td>\n",
       "      <td>52200.0</td>\n",
       "      <td>460.0</td>\n",
       "      <td>53200.0</td>\n",
       "      <td>520.0</td>\n",
       "      <td>53800.0</td>\n",
       "      <td>510.0</td>\n",
       "      <td>54800.0</td>\n",
       "      <td>390.0</td>\n",
       "      <td>53900.0</td>\n",
       "      <td>470.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Alaska</td>\n",
       "      <td>87600.0</td>\n",
       "      <td>84100.0</td>\n",
       "      <td>78200.0</td>\n",
       "      <td>1530.0</td>\n",
       "      <td>80400.0</td>\n",
       "      <td>1580.0</td>\n",
       "      <td>83900.0</td>\n",
       "      <td>1610.0</td>\n",
       "      <td>86300.0</td>\n",
       "      <td>1530.0</td>\n",
       "      <td>80900.0</td>\n",
       "      <td>1770.0</td>\n",
       "      <td>80200.0</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>80000.0</td>\n",
       "      <td>1740.0</td>\n",
       "      <td>77800.0</td>\n",
       "      <td>1310.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Arizona</td>\n",
       "      <td>58300.0</td>\n",
       "      <td>66200.0</td>\n",
       "      <td>61600.0</td>\n",
       "      <td>550.0</td>\n",
       "      <td>58300.0</td>\n",
       "      <td>400.0</td>\n",
       "      <td>58900.0</td>\n",
       "      <td>300.0</td>\n",
       "      <td>60500.0</td>\n",
       "      <td>440.0</td>\n",
       "      <td>62500.0</td>\n",
       "      <td>350.0</td>\n",
       "      <td>63900.0</td>\n",
       "      <td>480.0</td>\n",
       "      <td>65800.0</td>\n",
       "      <td>290.0</td>\n",
       "      <td>69100.0</td>\n",
       "      <td>420.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Arkansas</td>\n",
       "      <td>44700.0</td>\n",
       "      <td>52500.0</td>\n",
       "      <td>48700.0</td>\n",
       "      <td>510.0</td>\n",
       "      <td>47700.0</td>\n",
       "      <td>490.0</td>\n",
       "      <td>48000.0</td>\n",
       "      <td>330.0</td>\n",
       "      <td>50100.0</td>\n",
       "      <td>630.0</td>\n",
       "      <td>50700.0</td>\n",
       "      <td>510.0</td>\n",
       "      <td>50800.0</td>\n",
       "      <td>470.0</td>\n",
       "      <td>51900.0</td>\n",
       "      <td>560.0</td>\n",
       "      <td>52500.0</td>\n",
       "      <td>610.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           State     1990     2000     2005  2005 (SE)     2010  2010 (SE)  \\\n",
       "0  United States  63600.0  68500.0  64300.0       90.0  62300.0       50.0   \n",
       "1        Alabama  49900.0  55700.0  51300.0      450.0  50400.0      360.0   \n",
       "2         Alaska  87600.0  84100.0  78200.0     1530.0  80400.0     1580.0   \n",
       "3        Arizona  58300.0  66200.0  61600.0      550.0  58300.0      400.0   \n",
       "4       Arkansas  44700.0  52500.0  48700.0      510.0  47700.0      490.0   \n",
       "\n",
       "      2015  2015 (SE)     2016  2016 (SE)     2017  2017 (SE)     2018  \\\n",
       "0  63800.0       60.0  65100.0       80.0  66700.0       60.0  66800.0   \n",
       "1  51200.0      520.0  52200.0      460.0  53200.0      520.0  53800.0   \n",
       "2  83900.0     1610.0  86300.0     1530.0  80900.0     1770.0  80200.0   \n",
       "3  58900.0      300.0  60500.0      440.0  62500.0      350.0  63900.0   \n",
       "4  48000.0      330.0  50100.0      630.0  50700.0      510.0  50800.0   \n",
       "\n",
       "   2018 (SE)     2019  2019 (SE)     2021  2021 (SE)  \n",
       "0       60.0  69600.0       80.0  69700.0       80.0  \n",
       "1      510.0  54800.0      390.0  53900.0      470.0  \n",
       "2     1500.0  80000.0     1740.0  77800.0     1310.0  \n",
       "3      480.0  65800.0      290.0  69100.0      420.0  \n",
       "4      470.0  51900.0      560.0  52500.0      610.0  "
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mhis2.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "State         object\n",
       "1990         float64\n",
       "2000         float64\n",
       "2005         float64\n",
       "2005 (SE)    float64\n",
       "2010         float64\n",
       "2010 (SE)    float64\n",
       "2015         float64\n",
       "2015 (SE)    float64\n",
       "2016         float64\n",
       "2016 (SE)    float64\n",
       "2017         float64\n",
       "2017 (SE)    float64\n",
       "2018         float64\n",
       "2018 (SE)    float64\n",
       "2019         float64\n",
       "2019 (SE)    float64\n",
       "2021         float64\n",
       "2021 (SE)    float64\n",
       "dtype: object"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mhis2.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset #3: History of Women in the U.S. Congress"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(54, 6)"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "congress.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "congress = congress.drop(columns = [\"0\"], axis = 1) #Removing index provided"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Congress</th>\n",
       "      <th>Dates</th>\n",
       "      <th>Women in the Senate</th>\n",
       "      <th>Women in the House</th>\n",
       "      <th>Total Women</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>65th</td>\n",
       "      <td>1917-1919</td>\n",
       "      <td>0 (0D, 0R)</td>\n",
       "      <td>1 (0D, 1R)</td>\n",
       "      <td>1 (0D, 1R)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>66th</td>\n",
       "      <td>1919-1921</td>\n",
       "      <td>0 (0D, 0R)</td>\n",
       "      <td>0 (0D, 0R)</td>\n",
       "      <td>0 (0D, 0R)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>67th</td>\n",
       "      <td>1921-1923</td>\n",
       "      <td>1 (1D, 0R)</td>\n",
       "      <td>3 (0D, 3R)</td>\n",
       "      <td>4 (1D, 3R)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>68th</td>\n",
       "      <td>1923-1925</td>\n",
       "      <td>0 (0D, 0R)</td>\n",
       "      <td>1 (0D, 1R)</td>\n",
       "      <td>1 (0D, 1R)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>69th</td>\n",
       "      <td>1925-1927</td>\n",
       "      <td>0 (0D, 0R)</td>\n",
       "      <td>3 (1D, 2R)</td>\n",
       "      <td>3 (1D, 2R)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Congress      Dates Women in the Senate Women in the House Total Women\n",
       "0     65th  1917-1919          0 (0D, 0R)         1 (0D, 1R)  1 (0D, 1R)\n",
       "1     66th  1919-1921          0 (0D, 0R)         0 (0D, 0R)  0 (0D, 0R)\n",
       "2     67th  1921-1923          1 (1D, 0R)         3 (0D, 3R)  4 (1D, 3R)\n",
       "3     68th  1923-1925          0 (0D, 0R)         1 (0D, 1R)  1 (0D, 1R)\n",
       "4     69th  1925-1927          0 (0D, 0R)         3 (1D, 2R)  3 (1D, 2R)"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "congress.head() #D means Democrat and R means Republican"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Congress               object\n",
       "Dates                  object\n",
       "Women in the Senate    object\n",
       "Women in the House     object\n",
       "Total Women            object\n",
       "dtype: object"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "congress.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Removing these rows because they are not needed for our analysis and caused some cleaning problems, as they were not uniform\n",
    "congress.drop(53, inplace = True)\n",
    "congress.drop(52, inplace = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "congress[\"Start Date\"] = congress[\"Dates\"].apply(lambda string: int(string.split(\"-\")[0]))\n",
    "congress[\"End Date\"] = congress[\"Dates\"].apply(lambda string: int(string.split(\"-\")[1]))\n",
    "# congress = congress.drop(columns = [\"Dates\"], axis = 1)\n",
    "congress[\"Total Women in Senate\"] = congress[\"Women in the Senate\"].apply(lambda string: int(string.split(\" \")[0]))\n",
    "congress[\"Democratic Women in Senate\"] = congress[\"Women in the Senate\"].apply(lambda string:\n",
    "                                                                    int(string[string.find(\"(\") + 1:string.find(\")\")].split(\",\")[0].replace(\"D\", \"\")))\n",
    "congress[\"Republican Women in Senate\"] = congress[\"Women in the Senate\"].apply(lambda string:\n",
    "                                                                    int(string[string.find(\"(\") + 1:string.find(\")\")].split(\",\")[-1].replace(\"R\", \"\")))\n",
    "congress[\"Total Women in House\"] = congress[\"Women in the House\"].apply(lambda string: int(string.split(\" \")[0]))\n",
    "congress[\"Democratic Women in House\"] = congress[\"Women in the House\"].apply(lambda string:\n",
    "                                                                    int(string[string.find(\"(\") + 1:string.find(\")\")].split(\",\")[0].replace(\"D\", \"\")))\n",
    "congress[\"Republican Women in House\"] = congress[\"Women in the House\"].apply(lambda string:\n",
    "                                                                    int(string[string.find(\"(\") + 1:string.find(\")\")].split(\",\")[-1].replace(\"R\", \"\")))\n",
    "congress[\"Total Women in Congress\"] = congress[\"Total Women\"].apply(lambda string: int(string.split(\" \")[0]))\n",
    "congress[\"Total Democratic Women\"] = congress[\"Total Women\"].apply(lambda string:\n",
    "                                                                    int(string[string.find(\"(\") + 1:string.find(\")\")].split(\",\")[0].replace(\"D\", \"\")))\n",
    "congress[\"Total Republican Women\"] = congress[\"Total Women\"].apply(lambda string:\n",
    "                                                                    int(string[string.find(\"(\") + 1:string.find(\")\")].split(\",\")[-1].replace(\"R\", \"\")))\n",
    "congress = congress.drop(columns = [\"Dates\", \"Women in the Senate\", \"Women in the House\", \"Total Women\"], axis = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Congress</th>\n",
       "      <th>Start Date</th>\n",
       "      <th>End Date</th>\n",
       "      <th>Total Women in Senate</th>\n",
       "      <th>Democratic Women in Senate</th>\n",
       "      <th>Republican Women in Senate</th>\n",
       "      <th>Total Women in House</th>\n",
       "      <th>Democratic Women in House</th>\n",
       "      <th>Republican Women in House</th>\n",
       "      <th>Total Women in Congress</th>\n",
       "      <th>Total Democratic Women</th>\n",
       "      <th>Total Republican Women</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>65th</td>\n",
       "      <td>1917</td>\n",
       "      <td>1919</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>66th</td>\n",
       "      <td>1919</td>\n",
       "      <td>1921</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>67th</td>\n",
       "      <td>1921</td>\n",
       "      <td>1923</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>68th</td>\n",
       "      <td>1923</td>\n",
       "      <td>1925</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>69th</td>\n",
       "      <td>1925</td>\n",
       "      <td>1927</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Congress  Start Date  End Date  Total Women in Senate  \\\n",
       "0     65th        1917      1919                      0   \n",
       "1     66th        1919      1921                      0   \n",
       "2     67th        1921      1923                      1   \n",
       "3     68th        1923      1925                      0   \n",
       "4     69th        1925      1927                      0   \n",
       "\n",
       "   Democratic Women in Senate  Republican Women in Senate  \\\n",
       "0                           0                           0   \n",
       "1                           0                           0   \n",
       "2                           1                           0   \n",
       "3                           0                           0   \n",
       "4                           0                           0   \n",
       "\n",
       "   Total Women in House  Democratic Women in House  Republican Women in House  \\\n",
       "0                     1                          0                          1   \n",
       "1                     0                          0                          0   \n",
       "2                     3                          0                          3   \n",
       "3                     1                          0                          1   \n",
       "4                     3                          1                          2   \n",
       "\n",
       "   Total Women in Congress  Total Democratic Women  Total Republican Women  \n",
       "0                        1                       0                       1  \n",
       "1                        0                       0                       0  \n",
       "2                        4                       1                       3  \n",
       "3                        1                       0                       1  \n",
       "4                        3                       1                       2  "
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "congress.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Congress                      object\n",
       "Start Date                     int64\n",
       "End Date                       int64\n",
       "Total Women in Senate          int64\n",
       "Democratic Women in Senate     int64\n",
       "Republican Women in Senate     int64\n",
       "Total Women in House           int64\n",
       "Democratic Women in House      int64\n",
       "Republican Women in House      int64\n",
       "Total Women in Congress        int64\n",
       "Total Democratic Women         int64\n",
       "Total Republican Women         int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "congress.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset #4 OB/GYN Rate Per State, 2019-2021"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(60, 5)"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "obgyn1.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(60, 5)"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "obgyn2.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(60, 5)"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "obgyn3.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 1</th>\n",
       "      <th>Unnamed: 2</th>\n",
       "      <th>Unnamed: 3</th>\n",
       "      <th>Unnamed: 4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>data.HRSA.gov</td>\n",
       "      <td>data.HRSA.gov</td>\n",
       "      <td>data.HRSA.gov</td>\n",
       "      <td>data.HRSA.gov</td>\n",
       "      <td>data.HRSA.gov</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Selected Filters: 2020-2021 / M.D., Obstetrics...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>State/Territory</td>\n",
       "      <td>M.D., Obstetrics and Gynecology (County Level ...</td>\n",
       "      <td>Population, All (County Level File)</td>\n",
       "      <td>Rate (per 100,000 population)</td>\n",
       "      <td>Source</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Alabama</td>\n",
       "      <td>518</td>\n",
       "      <td>4921532</td>\n",
       "      <td>10.53</td>\n",
       "      <td>AMA Physician Masterfile 2019; Census County P...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Alaska</td>\n",
       "      <td>87</td>\n",
       "      <td>731158</td>\n",
       "      <td>11.90</td>\n",
       "      <td>AMA Physician Masterfile 2019; Census County P...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                          Unnamed: 0  \\\n",
       "0                                      data.HRSA.gov   \n",
       "1  Selected Filters: 2020-2021 / M.D., Obstetrics...   \n",
       "2                                    State/Territory   \n",
       "3                                            Alabama   \n",
       "4                                             Alaska   \n",
       "\n",
       "                                          Unnamed: 1  \\\n",
       "0                                      data.HRSA.gov   \n",
       "1                                                NaN   \n",
       "2  M.D., Obstetrics and Gynecology (County Level ...   \n",
       "3                                                518   \n",
       "4                                                 87   \n",
       "\n",
       "                            Unnamed: 2                     Unnamed: 3  \\\n",
       "0                        data.HRSA.gov                  data.HRSA.gov   \n",
       "1                                  NaN                            NaN   \n",
       "2  Population, All (County Level File)  Rate (per 100,000 population)   \n",
       "3                              4921532                          10.53   \n",
       "4                               731158                          11.90   \n",
       "\n",
       "                                          Unnamed: 4  \n",
       "0                                      data.HRSA.gov  \n",
       "1                                                NaN  \n",
       "2                                             Source  \n",
       "3  AMA Physician Masterfile 2019; Census County P...  \n",
       "4  AMA Physician Masterfile 2019; Census County P...  "
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "obgyn1.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 1</th>\n",
       "      <th>Unnamed: 2</th>\n",
       "      <th>Unnamed: 3</th>\n",
       "      <th>Unnamed: 4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>data.HRSA.gov</td>\n",
       "      <td>data.HRSA.gov</td>\n",
       "      <td>data.HRSA.gov</td>\n",
       "      <td>data.HRSA.gov</td>\n",
       "      <td>data.HRSA.gov</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Selected Filters: 2021-2022 / M.D., Obstetrics...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>State/Territory</td>\n",
       "      <td>M.D., Obstetrics and Gynecology (County Level ...</td>\n",
       "      <td>Population, All (County Level File)</td>\n",
       "      <td>Rate (per 100,000 population)</td>\n",
       "      <td>Source</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Alabama</td>\n",
       "      <td>508</td>\n",
       "      <td>5039877</td>\n",
       "      <td>10.08</td>\n",
       "      <td>AMA Physician Masterfile 2020; Census County P...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Alaska</td>\n",
       "      <td>88</td>\n",
       "      <td>732673</td>\n",
       "      <td>12.01</td>\n",
       "      <td>AMA Physician Masterfile 2020; Census County P...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                          Unnamed: 0  \\\n",
       "0                                      data.HRSA.gov   \n",
       "1  Selected Filters: 2021-2022 / M.D., Obstetrics...   \n",
       "2                                    State/Territory   \n",
       "3                                            Alabama   \n",
       "4                                             Alaska   \n",
       "\n",
       "                                          Unnamed: 1  \\\n",
       "0                                      data.HRSA.gov   \n",
       "1                                                NaN   \n",
       "2  M.D., Obstetrics and Gynecology (County Level ...   \n",
       "3                                                508   \n",
       "4                                                 88   \n",
       "\n",
       "                            Unnamed: 2                     Unnamed: 3  \\\n",
       "0                        data.HRSA.gov                  data.HRSA.gov   \n",
       "1                                  NaN                            NaN   \n",
       "2  Population, All (County Level File)  Rate (per 100,000 population)   \n",
       "3                              5039877                          10.08   \n",
       "4                               732673                          12.01   \n",
       "\n",
       "                                          Unnamed: 4  \n",
       "0                                      data.HRSA.gov  \n",
       "1                                                NaN  \n",
       "2                                             Source  \n",
       "3  AMA Physician Masterfile 2020; Census County P...  \n",
       "4  AMA Physician Masterfile 2020; Census County P...  "
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "obgyn2.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 1</th>\n",
       "      <th>Unnamed: 2</th>\n",
       "      <th>Unnamed: 3</th>\n",
       "      <th>Unnamed: 4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>data.HRSA.gov</td>\n",
       "      <td>data.HRSA.gov</td>\n",
       "      <td>data.HRSA.gov</td>\n",
       "      <td>data.HRSA.gov</td>\n",
       "      <td>data.HRSA.gov</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Selected Filters: 2022-2023 / M.D., Obstetrics...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>State/Territory</td>\n",
       "      <td>M.D., Obstetrics and Gynecology (County Level ...</td>\n",
       "      <td>Population, All (County Level File)</td>\n",
       "      <td>Rate (per 100,000 population)</td>\n",
       "      <td>Source</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Alabama</td>\n",
       "      <td>511</td>\n",
       "      <td>5039877</td>\n",
       "      <td>10.14</td>\n",
       "      <td>AMA Physician Masterfile 2021; Census County P...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Alaska</td>\n",
       "      <td>94</td>\n",
       "      <td>732673</td>\n",
       "      <td>12.83</td>\n",
       "      <td>AMA Physician Masterfile 2021; Census County P...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                          Unnamed: 0  \\\n",
       "0                                      data.HRSA.gov   \n",
       "1  Selected Filters: 2022-2023 / M.D., Obstetrics...   \n",
       "2                                    State/Territory   \n",
       "3                                            Alabama   \n",
       "4                                             Alaska   \n",
       "\n",
       "                                          Unnamed: 1  \\\n",
       "0                                      data.HRSA.gov   \n",
       "1                                                NaN   \n",
       "2  M.D., Obstetrics and Gynecology (County Level ...   \n",
       "3                                                511   \n",
       "4                                                 94   \n",
       "\n",
       "                            Unnamed: 2                     Unnamed: 3  \\\n",
       "0                        data.HRSA.gov                  data.HRSA.gov   \n",
       "1                                  NaN                            NaN   \n",
       "2  Population, All (County Level File)  Rate (per 100,000 population)   \n",
       "3                              5039877                          10.14   \n",
       "4                               732673                          12.83   \n",
       "\n",
       "                                          Unnamed: 4  \n",
       "0                                      data.HRSA.gov  \n",
       "1                                                NaN  \n",
       "2                                             Source  \n",
       "3  AMA Physician Masterfile 2021; Census County P...  \n",
       "4  AMA Physician Masterfile 2021; Census County P...  "
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "obgyn3.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "obgyn1 = obgyn1.drop([0,1,2, 59])\n",
    "obgyn1 = obgyn1.drop(columns = ['Unnamed: 1', 'Unnamed: 2', 'Unnamed: 4'])\n",
    "obgyn1.columns = ['State', 'Rate (2019)']\n",
    "\n",
    "obgyn2 = obgyn2.drop([0,1,2, 59])\n",
    "obgyn2 = obgyn2.drop(columns = ['Unnamed: 1', 'Unnamed: 2', 'Unnamed: 4'])\n",
    "obgyn2.columns = ['State', 'Rate (2020)']\n",
    "\n",
    "obgyn3 = obgyn3.drop([0,1,2, 59])\n",
    "obgyn3 = obgyn3.drop(columns = ['Unnamed: 1', 'Unnamed: 2', 'Unnamed: 4'])\n",
    "obgyn3.columns = ['State', 'Rate (2021)']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "obgyn4 = pd.merge(obgyn1, obgyn2, on='State')\n",
    "obgyn = pd.merge(obgyn4, obgyn3, on='State')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [],
   "source": [
    "obgyn = obgyn.drop([54,55,56,57,58,59,60,61])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>State</th>\n",
       "      <th>Rate (2019)</th>\n",
       "      <th>Rate (2020)</th>\n",
       "      <th>Rate (2021)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Alabama</td>\n",
       "      <td>10.53</td>\n",
       "      <td>10.08</td>\n",
       "      <td>10.14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Alaska</td>\n",
       "      <td>11.90</td>\n",
       "      <td>12.01</td>\n",
       "      <td>12.83</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Arizona</td>\n",
       "      <td>9.74</td>\n",
       "      <td>10.12</td>\n",
       "      <td>10.06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Arkansas</td>\n",
       "      <td>8.74</td>\n",
       "      <td>8.89</td>\n",
       "      <td>8.59</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>California</td>\n",
       "      <td>12.52</td>\n",
       "      <td>12.59</td>\n",
       "      <td>12.57</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        State Rate (2019) Rate (2020) Rate (2021)\n",
       "0     Alabama       10.53       10.08       10.14\n",
       "1      Alaska       11.90       12.01       12.83\n",
       "2     Arizona        9.74       10.12       10.06\n",
       "3    Arkansas        8.74        8.89        8.59\n",
       "4  California       12.52       12.59       12.57"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "obgyn.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Ethics & Privacy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In all of the datasets we have utilized, we have accessed these specific ones from public institutions or have already been labeled public-use datasets. Therefore, the subjects used in the research have already consented and answered all, if any, anonymity agreements or anything else necessary to protect their privacy. For the intent of use in this project, there are no additional steps for us to take to further participants’ privacies.\n",
    "Beyond privacy concerns, we must consider the reliability of our data. If we are considering abortion rates, we must consider the possibility of underreporting and illegal abortions. Additionally it is possible that political agendas or compromises in data collecting infrastructure causing unreliable data. We will do the best we can to find the most complete and reliable data, while still considering the possibilities of bias or possible underreporting. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Team Expectations "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Read over the [COGS108 Team Policies](https://github.com/COGS108/Projects/blob/master/COGS108_TeamPolicies.md) individually. Then, include your group’s expectations of one another for successful completion of your COGS108 project below. Discuss and agree on what all of your expectations are. Discuss how your team will communicate throughout the quarter and consider how you will communicate respectfully should conflicts arise. By including each member’s name above and by adding their name to the submission, you are indicating that you have read the COGS108 Team Policies, accept your team’s expectations below, and have every intention to fulfill them. These expectations are for your team’s use and benefit — they won’t be graded for their details.\n",
    "\n",
    "- We expect to meet at least once a week in person and another time again online, especially if there is a checkpoint in the next few days.\n",
    "- In the case that our meetings are cut short, we expect to delegate work amongst each other respective to our availability and team contributions.\n",
    "- We will do our best to stay on track, however we expect each other to be understanding of personal schedules and will do our best to accommodate one another."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Project Timeline Proposal"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Subject to change\n",
    "\n",
    "| Meeting Date  | Meeting Time| Completed Before Meeting  | Discuss at Meeting |\n",
    "|---|---|---|---|\n",
    "| 2/3  |  4:30 PM | NA | Discuss possible topics, Work on Group Project Review | \n",
    "| 2/5  |  2:30 PM |  Brainstorm for possible topics | Discuss possible topics, determine what kind of datasets we should look for| \n",
    "| 2/8  | 3:30 PM  | Search for datasets, think about ways in which our research question could be elaborated | Discuss research question, look for datasets, delegate sections for project proposal  |\n",
    "| 2/13  | 2 PM  | Submit Proposal, do more research on datasets | choose a few datasets, discuss how we will clean the data, delegate roles/work |\n",
    "| 2/15  | 3:30 PM  | Wrangle Data | Check in to discuss and clarify our edits/ questions |\n",
    "| 2/19  | 2 PM  | Continue to edit and wrangle data | Check-in to discuss and clarify our edits/ questions |\n",
    "| 2/22  | 3:30 PM  | Review requirements for Checkpoint #1| Finish SetUp and Data Cleaning parts of the Checkpoint, delegate tasks |\n",
    "| 2/26  | 2 PM  | Submit Checkpoint on the 25th | Meet to discuss next steps, delegate Tasks |\n",
    "| 3/4  | 2 PM  | Data Analysis | Meet to Analyze the Data, delegate tasks for the Checkpoint |\n",
    "| 3/7  | 3:30 PM  | Review and work on requirements for Checkpoint #2 |Check-in to discuss Data Analysis portion |\n",
    "| 3/14  | 2 PM  | Submit Checkpoint #2 on the 10th, start working on the final report| Meet to discuss Final Report and film Video |\n",
    "| 3/20 | Before 11:59 PM  | Continue to touch-up Final Report and Submit| Turn in Final Project & Group Project Survey |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}