18_param_estimation_bootstrap.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e7fca645",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# ICS 434: DATA SCIENCE FUNDAMENTALS\n",
    "\n",
    "## Parameter Estimation: Bootstrap Confidence Interval\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "9e75dc7e",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9af80eaf",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Bootstrapping\n",
    "\n",
    "* Bootstrapping consists of sampling <ins>with replacement</ins> from observed dataset\n",
    "\n",
    "* _Basic idea_: inference about a population from a sample of the population can be modeled by resampling from the sample data and performing inference on the resampled sample\n",
    "\n",
    "* In bootstrap resamples, the 'population' is in fact the sample\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eeb825f9-f8e1-4cf6-abdb-1728257f8d1c",
   "metadata": {},
   "source": [
    "### How Bootstrapiing Used\n",
    "\n",
    "* The bootstrap data shows us the extent of sampling variability\n",
    "\n",
    "* it's used to, among other things, infer resampling distribution of a statistic (like the mean or median) by repeatedly sampling with replacement from the data set.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b612a599",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Using Bootstrap to Estimate the Population Mean\n",
    "\n",
    "* In the following example, we draw a sample of size 100 from Normal with $\\mu=50$ and $\\sigma=10$\n",
    "\n",
    "```python\n",
    "np.random.seed(22)\n",
    "data = np.random.normal(50, 10, 100)\n",
    "```\n",
    "\n",
    "* Are 100 records sufficient to obtain a good measure of the population mean using bootstrap?\n",
    "\n",
    "* What is the population's range of possible means that can be estimated from samples of this size?\n",
    "  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "2f41a31f-1075-41b1-94fd-3a1811c3dd82",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "44.15263117075237"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = np.random.normal(50, 1, 100)\n",
    "data.mean()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "b3d107f5",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "49.46324599387749"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Given the following\n",
    "\n",
    "np.random.seed(22)\n",
    "\n",
    "data = np.random.normal(50, 10, 100)\n",
    "data.mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "61dfb120",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([49.0805008 , 35.36649347, 60.81791679, 47.60674832, 45.08870864,\n",
       "       39.9772799 , 59.18821503, 38.96367899, 56.26493456, 44.38486219,\n",
       "       50.28855118, 47.69232812, 55.87752215, 57.52317579, 39.41497446,\n",
       "       60.55972414, 57.47750266, 60.64676588, 65.20129588, 35.11397064,\n",
       "       68.59989892, 34.01386973, 43.53926387, 53.37325002, 60.46728725,\n",
       "       56.2914334 , 53.63059095, 55.55749697, 39.11450468, 50.23694772,\n",
       "       74.99176641, 25.09969607, 47.65137613, 49.0243654 , 41.13470658,\n",
       "       48.63286762, 51.01978927, 47.49070464, 49.21187696, 39.14835834,\n",
       "       55.94928451, 43.61091928, 38.92163142, 71.06145756, 44.3261223 ,\n",
       "       45.20050207, 30.76770144, 53.99588542, 39.51816129, 43.06121297,\n",
       "       57.45339147, 55.36909995, 42.67278223, 55.5571563 , 54.32289301,\n",
       "       48.64196022, 40.58893908, 54.84767363, 34.67178547, 54.0497889 ,\n",
       "       50.12826125, 37.68284441, 39.46914977, 75.16221824, 29.60971496,\n",
       "       50.94487309, 46.85176665, 54.90314717, 53.54976188, 59.50711346,\n",
       "       57.60371799, 50.11503626, 36.23943042, 47.28445316, 55.42252051,\n",
       "       55.35522208, 61.5999804 , 48.34994951, 38.20597435, 44.54701049,\n",
       "       52.7050267 , 59.80237559, 60.06888399, 57.82156255, 37.51408341,\n",
       "       45.76166554, 55.52608154, 53.29603425, 58.6311829 , 37.72621974,\n",
       "       56.24874027, 22.30467045, 45.07822406, 50.69148198, 46.49997428,\n",
       "       58.69345174, 47.83235794, 50.17118495, 56.89713394, 41.16856848])"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "b5599290",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2, 3, 3, 3, 2, 1, 2, 1, 2, 2])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Selecting data from a sample\n",
    "# Draw to instance from the given array \n",
    "\n",
    "np.random.choice([1, 2, 3], 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "2c28cac2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "66"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(set(np.random.choice(data, 100)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "dc20e9a7",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "bootstrap_means = []\n",
    "\n",
    "for i in range(10_000):\n",
    "    returns_data_100_bootstrap = np.random.choice(data, 100)\n",
    "    bootstrap_mean = returns_data_100_bootstrap.mean()\n",
    "    bootstrap_means.append(bootstrap_mean)\n",
    "    \n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "df1e686a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[50.57385448510077,\n",
       " 49.53641037803769,\n",
       " 50.55427621601572,\n",
       " 47.29643327348919,\n",
       " 48.95876521752653,\n",
       " 49.92982451344803,\n",
       " 47.22595130710195,\n",
       " 49.98014167201929,\n",
       " 50.77783363049219,\n",
       " 49.03002220125765,\n",
       " 50.23380516658277,\n",
       " 50.98130616472651,\n",
       " 50.04615174675433,\n",
       " 49.791136470467265,\n",
       " 49.02299607565522,\n",
       " 49.92917466823551,\n",
       " 49.33774684843988,\n",
       " 47.90850118167857,\n",
       " 46.336865887675174,\n",
       " 48.47666260818641,\n",
       " 51.797774428855625,\n",
       " 50.04999110107246,\n",
       " 49.165593398521686,\n",
       " 49.373480437347354,\n",
       " 51.60266977893916,\n",
       " 49.685622042805065,\n",
       " 50.17800213807037,\n",
       " 49.74520178385228,\n",
       " 49.43938366278381,\n",
       " 48.60245096099747,\n",
       " 47.00424931864226,\n",
       " 48.965304283597504,\n",
       " 49.24518164949317,\n",
       " 48.831126461860585,\n",
       " 48.89110677512726,\n",
       " 50.27456958308707,\n",
       " 49.28090313955237,\n",
       " 48.1960706785353,\n",
       " 48.16468969687005,\n",
       " 49.609700807421575,\n",
       " 49.298333003814896,\n",
       " 48.203765275341695,\n",
       " 49.74117226410623,\n",
       " 48.19076428502297,\n",
       " 48.30426586607183,\n",
       " 49.45124637925022,\n",
       " 50.02185028748993,\n",
       " 48.08138771934734,\n",
       " 49.98450737056892,\n",
       " 49.65242244979583,\n",
       " 49.46141274566048,\n",
       " 50.413439449809765,\n",
       " 50.46435730194166,\n",
       " 50.59621703349999,\n",
       " 48.6233764053169,\n",
       " 48.14761807307961,\n",
       " 48.68920639037012,\n",
       " 50.96035717789134,\n",
       " 51.77274437262842,\n",
       " 48.19495538161627,\n",
       " 50.616085594393866,\n",
       " 48.34957846466103,\n",
       " 50.20749652484321,\n",
       " 52.14566933868125,\n",
       " 48.31496994524615,\n",
       " 47.34895188895086,\n",
       " 47.890653436614826,\n",
       " 48.05112906144972,\n",
       " 49.29133692506146,\n",
       " 49.82168463302459,\n",
       " 50.61523276089715,\n",
       " 48.03198942279701,\n",
       " 51.017195465879944,\n",
       " 50.71507689909376,\n",
       " 50.81752913039165,\n",
       " 51.13266072064503,\n",
       " 50.18177682765807,\n",
       " 50.25912256653999,\n",
       " 49.304580719793684,\n",
       " 48.69708449184736,\n",
       " 50.70573523750216,\n",
       " 49.068631515939884,\n",
       " 49.25092057071571,\n",
       " 48.41477100790344,\n",
       " 49.61887981910919,\n",
       " 49.88499738161353,\n",
       " 49.66021129862053,\n",
       " 50.11636264770122,\n",
       " 50.567278258380156,\n",
       " 49.35164007861623,\n",
       " 48.66028766672207,\n",
       " 49.81268962712771,\n",
       " 49.43566489004871,\n",
       " 49.21626053224014,\n",
       " 50.5867834662865,\n",
       " 50.63726137676866,\n",
       " 50.54511754084735,\n",
       " 48.191308386757456,\n",
       " 48.55650338473089,\n",
       " 48.20746419627855]"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bootstrap_means[0:100]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "7a009c52",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlsAAAESCAYAAADKehBIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAoY0lEQVR4nO3df1RVdb7/8dcR5aAmOEodYAnImIZmOQQpP0Lv3BRyKu2HV8YmnO7FMRZWEmvmFldbmd3EpgTURGWuxXXNREw3Ha1xRmmtQhm4/UBwmnLSWxZcPUdCryLNCAn7+4ffzurIDz1HNhvx+Vhrr+X57M/+8H4vXMuXn7PPPjbDMAwBAADAFIOsLgAAAGAgI2wBAACYiLAFAABgIsIWAACAiQhbAAAAJiJsAQAAmIiwBQAAYKLBVhfQWzo6OnTs2DGNGDFCNpvN6nIAAMAAZhiGzpw5o7CwMA0a1PPe1YAJW8eOHVN4eLjVZQAAgKtIQ0ODxowZ0+OcARO2RowYIel804GBgRZXAwAABrLm5maFh4e780dPBkzY+vatw8DAQMIWAADoE5dy6xI3yAMAAJiIsAUAAGAiwhYAAICJCFsAAAAmImwBAACYiLAFAABgIsIWAACAiQbMc7YAwBelpaU6efKk1WVctlGjRmnBggVWlwGgC4QtAFe1kydP6q33D2lY4EirS/HZ35pP6a6pE6wuA0A3CFsArnrDAkfqtjk/sboMn1Xu/I3VJQDoAfdsAQAAmIiwBQAAYCLCFgAAgIm4ZwuAzwbCJ/kqKip0rD3I6jIADGCELQA+Gwif5Dv4ZaOGOYZZXQaAAYywBeCyXOmf5Pvszx9YXQKAAY57tgAAAExE2AIAADARYQsAAMBEPoWtoqIiRUVFKSAgQLGxsdq3b1+3cysrK5WUlKTRo0dr6NChio6OVkFBgceckpIS2Wy2TsfZs2d9KQ8AAKDf8PoG+bKyMmVnZ6uoqEhJSUnavHmzZs+erU8++UQRERGd5g8fPlyPPPKIbr75Zg0fPlyVlZV6+OGHNXz4cC1evNg9LzAwUJ9++qnHtQEBAT60BAAA0H94Hbby8/OVkZGhRYsWSZIKCwu1e/dubdy4UXl5eZ3mx8TEKCYmxv167Nix2rZtm/bt2+cRtmw2m0JCQi65jtbWVrW2trpfNzc3e9sKAACA6bx6G7GtrU01NTVKSUnxGE9JSVFVVdUlrVFbW6uqqirNmDHDY7ylpUWRkZEaM2aM7rrrLtXW1va4Tl5enoKCgtxHeHi4N60AAAD0Ca/CVlNTk9rb2+VwODzGHQ6HXC5Xj9eOGTNGdrtdcXFxWrJkiXtnTJKio6NVUlKinTt3qrS0VAEBAUpKStLhw4e7XS83N1enT592Hw0NDd60AgAA0Cd8eqipzWbzeG0YRqexC+3bt08tLS367//+bz355JO6/vrrtWDBAklSfHy84uPj3XOTkpJ0yy23aP369Vq3bl2X69ntdtntdl/KBwAA6DNeha3g4GD5+fl12sVqbGzstNt1oaioKEnSTTfdpOPHj2vFihXusHWhQYMG6dZbb+1xZwsAAOBK4NXbiP7+/oqNjVV5ebnHeHl5uRITEy95HcMwPG5u7+p8XV2dQkNDvSkPAACg3/H6bcScnBylp6crLi5OCQkJKi4uVn19vTIzMyWdv5fq6NGj2rp1qyRpw4YNioiIUHR0tKTzz9168cUX9eijj7rXfOaZZxQfH6/x48erublZ69atU11dnTZs2NAbPQIAAFjG67CVlpamEydOaOXKlXI6nZo8ebJ27dqlyMhISZLT6VR9fb17fkdHh3Jzc3XkyBENHjxY48aN0+rVq/Xwww+755w6dUqLFy+Wy+VSUFCQYmJitHfvXk2dOrUXWgQAALCOTzfIZ2VlKSsrq8tzJSUlHq8fffRRj12srhQUFHR6qjwAAMBAwHcjAgAAmIiwBQAAYCLCFgAAgIkIWwAAACYibAEAAJiIsAUAAGAiwhYAAICJCFsAAAAmImwBAACYiLAFAABgIsIWAACAiQhbAAAAJiJsAQAAmIiwBQAAYCLCFgAAgIkIWwAAACYibAEAAJiIsAUAAGAin8JWUVGRoqKiFBAQoNjYWO3bt6/buZWVlUpKStLo0aM1dOhQRUdHq6CgoNO8N954Q5MmTZLdbtekSZO0fft2X0oDAADoV7wOW2VlZcrOztayZctUW1ur5ORkzZ49W/X19V3OHz58uB555BHt3btXBw8e1PLly7V8+XIVFxe751RXVystLU3p6ek6cOCA0tPTNX/+fL333nu+dwYAANAPeB228vPzlZGRoUWLFmnixIkqLCxUeHi4Nm7c2OX8mJgYLViwQDfeeKPGjh2rBx98UKmpqR67YYWFhZo1a5Zyc3MVHR2t3Nxc3X777SosLPS5MQAAgP7Aq7DV1tammpoapaSkeIynpKSoqqrqktaora1VVVWVZsyY4R6rrq7utGZqamqPa7a2tqq5udnjAAAA6G+8CltNTU1qb2+Xw+HwGHc4HHK5XD1eO2bMGNntdsXFxWnJkiVatGiR+5zL5fJ6zby8PAUFBbmP8PBwb1oBAADoEz7dIG+z2TxeG4bRaexC+/bt04cffqhNmzapsLBQpaWll7Vmbm6uTp8+7T4aGhq87AIAAMB8g72ZHBwcLD8/v047To2NjZ12pi4UFRUlSbrpppt0/PhxrVixQgsWLJAkhYSEeL2m3W6X3W73pnwAAIA+59XOlr+/v2JjY1VeXu4xXl5ersTExEtexzAMtba2ul8nJCR0WnPPnj1erQkAANAfebWzJUk5OTlKT09XXFycEhISVFxcrPr6emVmZko6//be0aNHtXXrVknShg0bFBERoejoaEnnn7v14osv6tFHH3WvuXTpUk2fPl3PP/+85s6dqx07dujtt99WZWVlb/QIAABgGa/DVlpamk6cOKGVK1fK6XRq8uTJ2rVrlyIjIyVJTqfT45lbHR0dys3N1ZEjRzR48GCNGzdOq1ev1sMPP+yek5iYqNdee03Lly/XU089pXHjxqmsrEzTpk3rhRYBAACs43XYkqSsrCxlZWV1ea6kpMTj9aOPPuqxi9WdefPmad68eb6UAwAA0G/x3YgAAAAmImwBAACYiLAFAABgIsIWAACAiQhbAAAAJiJsAQAAmIiwBQAAYCLCFgAAgIkIWwAAACYibAEAAJiIsAUAAGAiwhYAAICJCFsAAAAmImwBAACYiLAFAABgIsIWAACAiQhbAAAAJiJsAQAAmMinsFVUVKSoqCgFBAQoNjZW+/bt63butm3bNGvWLF177bUKDAxUQkKCdu/e7TGnpKRENput03H27FlfygMAAOg3vA5bZWVlys7O1rJly1RbW6vk5GTNnj1b9fX1Xc7fu3evZs2apV27dqmmpkY//OEPdffdd6u2ttZjXmBgoJxOp8cREBDgW1cAAAD9xGBvL8jPz1dGRoYWLVokSSosLNTu3bu1ceNG5eXldZpfWFjo8XrVqlXasWOH3nzzTcXExLjHbTabQkJCvC0HAACgX/NqZ6utrU01NTVKSUnxGE9JSVFVVdUlrdHR0aEzZ85o1KhRHuMtLS2KjIzUmDFjdNddd3Xa+bpQa2urmpubPQ4AAID+xquw1dTUpPb2djkcDo9xh8Mhl8t1SWusWbNGX3/9tebPn+8ei46OVklJiXbu3KnS0lIFBAQoKSlJhw8f7nadvLw8BQUFuY/w8HBvWgEAAOgTPt0gb7PZPF4bhtFprCulpaVasWKFysrKdN1117nH4+Pj9eCDD2rKlClKTk7Wb3/7W02YMEHr16/vdq3c3FydPn3afTQ0NPjSCgAAgKm8umcrODhYfn5+nXaxGhsbO+12XaisrEwZGRl6/fXXNXPmzB7nDho0SLfeemuPO1t2u112u/3SiwcAALCAVztb/v7+io2NVXl5ucd4eXm5EhMTu72utLRUDz30kF599VXdeeedF/05hmGorq5OoaGh3pQHAADQ73j9acScnBylp6crLi5OCQkJKi4uVn19vTIzMyWdf3vv6NGj2rp1q6TzQWvhwoVau3at4uPj3btiQ4cOVVBQkCTpmWeeUXx8vMaPH6/m5matW7dOdXV12rBhQ2/1CQAAYAmvw1ZaWppOnDihlStXyul0avLkydq1a5ciIyMlSU6n0+OZW5s3b9a5c+e0ZMkSLVmyxD3+05/+VCUlJZKkU6dOafHixXK5XAoKClJMTIz27t2rqVOnXmZ7AAAA1vI6bElSVlaWsrKyujz3bYD61rvvvnvR9QoKClRQUOBLKQAAAP0a340IAABgIsIWAACAiQhbAAAAJiJsAQAAmIiwBQAAYCLCFgAAgIkIWwAAACYibAEAAJiIsAUAAGAiwhYAAICJCFsAAAAmImwBAACYiLAFAABgIsIWAACAiQhbAAAAJiJsAQAAmIiwBQAAYCLCFgAAgIkIWwAAACYa7MtFRUVFeuGFF+R0OnXjjTeqsLBQycnJXc7dtm2bNm7cqLq6OrW2turGG2/UihUrlJqa6jHvjTfe0FNPPaXPPvtM48aN03PPPad7773Xl/KAfq+0tFQnT560uozLVlFRoWPtQVaXAQD9mtdhq6ysTNnZ2SoqKlJSUpI2b96s2bNn65NPPlFERESn+Xv37tWsWbO0atUqjRw5Uq+88oruvvtuvffee4qJiZEkVVdXKy0tTc8++6zuvfdebd++XfPnz1dlZaWmTZt2+V0C/czJkyf11vuHNCxwpNWlXJaDXzZqmGOY1WUAQL/mddjKz89XRkaGFi1aJEkqLCzU7t27tXHjRuXl5XWaX1hY6PF61apV2rFjh95880132CosLNSsWbOUm5srScrNzVVFRYUKCwtVWlrqbYnAFWFY4EjdNucnVpdxWT778wdWlwAA/Z5X92y1tbWppqZGKSkpHuMpKSmqqqq6pDU6Ojp05swZjRo1yj1WXV3dac3U1NQe12xtbVVzc7PHAQAA0N94FbaamprU3t4uh8PhMe5wOORyuS5pjTVr1ujrr7/W/Pnz3WMul8vrNfPy8hQUFOQ+wsPDvegEAACgb/j0aUSbzebx2jCMTmNdKS0t1YoVK1RWVqbrrrvustbMzc3V6dOn3UdDQ4MXHQAAAPQNr+7ZCg4Olp+fX6cdp8bGxk47UxcqKytTRkaGXn/9dc2cOdPjXEhIiNdr2u122e12b8oHAADoc17tbPn7+ys2Nlbl5eUe4+Xl5UpMTOz2utLSUj300EN69dVXdeedd3Y6n5CQ0GnNPXv29LgmAADAlcDrTyPm5OQoPT1dcXFxSkhIUHFxserr65WZmSnp/Nt7R48e1datWyWdD1oLFy7U2rVrFR8f797BGjp0qIKCzj+fZ+nSpZo+fbqef/55zZ07Vzt27NDbb7+tysrK3uoTAADAEl7fs5WWlqbCwkKtXLlSP/jBD7R3717t2rVLkZGRkiSn06n6+nr3/M2bN+vcuXNasmSJQkND3cfSpUvdcxITE/Xaa6/plVde0c0336ySkhKVlZXxjC0AAHDF8+kJ8llZWcrKyuryXElJicfrd99995LWnDdvnubNm+dLOQAAAP0W340IAABgIsIWAACAiXx6GxEA0H80HPqLKo77WV1Grxg1apQWLFhgdRlAryJsAcAVru3s3/XX9mFq/2uj1aVclr81n9JdUydYXQbQ6whbADAA+A+75or/YvPKnb+xugTAFNyzBQAAYCLCFgAAgIkIWwAAACYibAEAAJiIsAUAAGAiwhYAAICJCFsAAAAmImwBAACYiLAFAABgIsIWAACAiQhbAAAAJiJsAQAAmIiwBQAAYCKfwlZRUZGioqIUEBCg2NhY7du3r9u5TqdTDzzwgG644QYNGjRI2dnZneaUlJTIZrN1Os6ePetLeQAAAP2G12GrrKxM2dnZWrZsmWpra5WcnKzZs2ervr6+y/mtra269tprtWzZMk2ZMqXbdQMDA+V0Oj2OgIAAb8sDAADoV7wOW/n5+crIyNCiRYs0ceJEFRYWKjw8XBs3buxy/tixY7V27VotXLhQQUFB3a5rs9kUEhLicQAAAFzpvApbbW1tqqmpUUpKisd4SkqKqqqqLquQlpYWRUZGasyYMbrrrrtUW1vb4/zW1lY1Nzd7HAAAAP2NV2GrqalJ7e3tcjgcHuMOh0Mul8vnIqKjo1VSUqKdO3eqtLRUAQEBSkpK0uHDh7u9Ji8vT0FBQe4jPDzc558PAABgFp9ukLfZbB6vDcPoNOaN+Ph4Pfjgg5oyZYqSk5P129/+VhMmTND69eu7vSY3N1enT592Hw0NDT7/fAAAALMM9mZycHCw/Pz8Ou1iNTY2dtrtuhyDBg3Srbfe2uPOlt1ul91u77WfCQAAYAavdrb8/f0VGxur8vJyj/Hy8nIlJib2WlGGYaiurk6hoaG9tiYAAIAVvNrZkqScnBylp6crLi5OCQkJKi4uVn19vTIzMyWdf3vv6NGj2rp1q/uauro6Sedvgv/qq69UV1cnf39/TZo0SZL0zDPPKD4+XuPHj1dzc7PWrVunuro6bdiwoRdaBAAAsI7XYSstLU0nTpzQypUr5XQ6NXnyZO3atUuRkZGSzj/E9MJnbsXExLj/XFNTo1dffVWRkZH64osvJEmnTp3S4sWL5XK5FBQUpJiYGO3du1dTp069jNYAAACs53XYkqSsrCxlZWV1ea6kpKTTmGEYPa5XUFCggoICX0oBAADo1/huRAAAABMRtgAAAExE2AIAADARYQsAAMBEhC0AAAATEbYAAABMRNgCAAAwEWELAADARIQtAAAAExG2AAAATETYAgAAMBFhCwAAwESELQAAABMRtgAAAExE2AIAADARYQsAAMBEhC0AAAATEbYAAABMRNgCAAAwkU9hq6ioSFFRUQoICFBsbKz27dvX7Vyn06kHHnhAN9xwgwYNGqTs7Owu573xxhuaNGmS7Ha7Jk2apO3bt/tSGgAAQL/iddgqKytTdna2li1bptraWiUnJ2v27Nmqr6/vcn5ra6uuvfZaLVu2TFOmTOlyTnV1tdLS0pSenq4DBw4oPT1d8+fP13vvvedteQAAAP2K12ErPz9fGRkZWrRokSZOnKjCwkKFh4dr48aNXc4fO3as1q5dq4ULFyooKKjLOYWFhZo1a5Zyc3MVHR2t3Nxc3X777SosLOy2jtbWVjU3N3scAAAA/Y1XYautrU01NTVKSUnxGE9JSVFVVZXPRVRXV3daMzU1tcc18/LyFBQU5D7Cw8N9/vkAAABm8SpsNTU1qb29XQ6Hw2Pc4XDI5XL5XITL5fJ6zdzcXJ0+fdp9NDQ0+PzzAQAAzDLYl4tsNpvHa8MwOo2Zvabdbpfdbr+snwkAAGA2r3a2goOD5efn12nHqbGxsdPOlDdCQkJ6fU0AAID+wKuw5e/vr9jYWJWXl3uMl5eXKzEx0eciEhISOq25Z8+ey1oTAACgP/D6bcScnBylp6crLi5OCQkJKi4uVn19vTIzMyWdv5fq6NGj2rp1q/uauro6SVJLS4u++uor1dXVyd/fX5MmTZIkLV26VNOnT9fzzz+vuXPnaseOHXr77bdVWVnZCy0CAABYx+uwlZaWphMnTmjlypVyOp2aPHmydu3apcjISEnnH2J64TO3YmJi3H+uqanRq6++qsjISH3xxReSpMTERL322mtavny5nnrqKY0bN05lZWWaNm3aZbQGAABgPZ9ukM/KylJWVlaX50pKSjqNGYZx0TXnzZunefPm+VIOAABAv8V3IwIAAJiIsAUAAGAiwhYAAICJCFsAAAAmImwBAACYiLAFAABgIsIWAACAiQhbAAAAJiJsAQAAmIiwBQAAYCLCFgAAgIkIWwAAACYibAEAAJhosNUFAN4qLS3VyZMnrS7jslRUVOhYe5DVZQAA+gBhC1eckydP6q33D2lY4EirS/HZwS8bNcwxzOoyAAB9gLCFK9KwwJG6bc5PrC7DZ5/9+QOrSwAA9BHu2QIAADARYQsAAMBEPoWtoqIiRUVFKSAgQLGxsdq3b1+P8ysqKhQbG6uAgAB9//vf16ZNmzzOl5SUyGazdTrOnj3rS3kAAAD9htdhq6ysTNnZ2Vq2bJlqa2uVnJys2bNnq76+vsv5R44c0Y9+9CMlJyertrZW//Zv/6bHHntMb7zxhse8wMBAOZ1OjyMgIMC3rgAAAPoJr2+Qz8/PV0ZGhhYtWiRJKiws1O7du7Vx40bl5eV1mr9p0yZFRESosLBQkjRx4kR9+OGHevHFF3X//fe759lsNoWEhPjYBgAAQP/kVdhqa2tTTU2NnnzySY/xlJQUVVVVdXlNdXW1UlJSPMZSU1O1ZcsWffPNNxoyZIgkqaWlRZGRkWpvb9cPfvADPfvss4qJiem2ltbWVrW2trpfNzc3e9MKAKCfaTj0F1Uc97O6jMs2atQoLViwwOoy0I94FbaamprU3t4uh8PhMe5wOORyubq8xuVydTn/3LlzampqUmhoqKKjo1VSUqKbbrpJzc3NWrt2rZKSknTgwAGNHz++y3Xz8vL0zDPPeFM+AKAfazv7d/21fZja/9podSk++1vzKd01dYLVZaCf8ek5WzabzeO1YRidxi42/7vj8fHxio+Pd59PSkrSLbfcovXr12vdunVdrpmbm6ucnBz36+bmZoWHh3vXCACgX/Efds0V/Qy9yp2/sboE9ENeha3g4GD5+fl12sVqbGzstHv1rZCQkC7nDx48WKNHj+7ymkGDBunWW2/V4cOHu63FbrfLbrd7Uz4AAECf8+rTiP7+/oqNjVV5ebnHeHl5uRITE7u8JiEhodP8PXv2KC4uzn2/1oUMw1BdXZ1CQ0O9KQ8AAKDf8frRDzk5OfqP//gPvfzyyzp48KAef/xx1dfXKzMzU9L5t/cWLlzonp+Zmakvv/xSOTk5OnjwoF5++WVt2bJFP//5z91znnnmGe3evVuff/656urqlJGRobq6OveaAAAAVyqv79lKS0vTiRMntHLlSjmdTk2ePFm7du1SZGSkJMnpdHo8cysqKkq7du3S448/rg0bNigsLEzr1q3zeOzDqVOntHjxYrlcLgUFBSkmJkZ79+7V1KlTe6FFAAAA6/h0g3xWVpaysrK6PFdSUtJpbMaMGdq/f3+36xUUFKigoMCXUgAAAPo1vhsRAADARIQtAAAAExG2AAAATETYAgAAMBFhCwAAwESELQAAABMRtgAAAExE2AIAADARYQsAAMBEhC0AAAATEbYAAABMRNgCAAAwEWELAADARIQtAAAAExG2AAAATDTY6gLQt0pLS3Xy5Emry7gsFRUVOtYeZHUZAABcEsLWVebkyZN66/1DGhY40upSfHbwy0YNcwyzugwA6KTh0F9UcdzP6jIu26hRo7RgwQKryxgwCFtXoWGBI3XbnJ9YXYbPPvvzB1aXAABdajv7d/21fZja/9podSk++1vzKd01dYLVZQwohC0AAHqR/7Brruj/0Fbu/I3VJQw4Pt0gX1RUpKioKAUEBCg2Nlb79u3rcX5FRYViY2MVEBCg73//+9q0aVOnOW+88YYmTZoku92uSZMmafv27b6UBgAA0K94vbNVVlam7OxsFRUVKSkpSZs3b9bs2bP1ySefKCIiotP8I0eO6Ec/+pF+9rOf6de//rX+9Kc/KSsrS9dee63uv/9+SVJ1dbXS0tL07LPP6t5779X27ds1f/58VVZWatq0aZffZS8YCDeWS9xcDgBAX7MZhmF4c8G0adN0yy23aOPGje6xiRMn6p577lFeXl6n+U888YR27typgwcPuscyMzN14MABVVdXS5LS0tLU3NysP/zhD+45d9xxh773ve+ptLS0yzpaW1vV2trqfn369GlFRESooaFBgYGB3rR0SYqLi/WHDw73+rp9reF//qIhI4I17oaJVpfis09rKjX4mtH00A8MhD7oof8YCH0MhB7+3nxK15w9rqSkJKtLuWyLFy82be3m5maFh4fr1KlTCgq6yCaG4YXW1lbDz8/P2LZtm8f4Y489ZkyfPr3La5KTk43HHnvMY2zbtm3G4MGDjba2NsMwDCM8PNzIz8/3mJOfn29ERER0W8vTTz9tSOLg4ODg4ODgsOxoaGi4aH7y6m3EpqYmtbe3y+FweIw7HA65XK4ur3G5XF3OP3funJqamhQaGtrtnO7WlKTc3Fzl5OS4X3d0dOjkyZMaPXq0bDabN231W9+mZrN26/or+qbvq8XV2jt90/dAYBiGzpw5o7CwsIvO9enTiBeGGcMwegw4Xc2/cNzbNe12u+x2u8fYyJEje6z7ShUYGDig/oJeKvq+ulytfUtXb+/0fXUZiH1f9O3D/8+rTyMGBwfLz8+v045TY2Njp52pb4WEhHQ5f/DgwRo9enSPc7pbEwAA4ErhVdjy9/dXbGysysvLPcbLy8uVmJjY5TUJCQmd5u/Zs0dxcXEaMmRIj3O6WxMAAOBK4fXbiDk5OUpPT1dcXJwSEhJUXFys+vp6ZWZmSjp/L9XRo0e1detWSec/efjSSy8pJydHP/vZz1RdXa0tW7Z4fMpw6dKlmj59up5//nnNnTtXO3bs0Ntvv63KyspeavPKZLfb9fTTT3d6u3Sgo2/6vlpcrb3TN31fbbx+9IN0/qGmv/zlL+V0OjV58mQVFBRo+vTpkqSHHnpIX3zxhd599133/IqKCj3++OP6+OOPFRYWpieeeMIdzr71X//1X1q+fLk+//xzjRs3Ts8995zuu+++y+sOAADAYj6FLQAAAFwan76uBwAAAJeGsAUAAGAiwhYAAICJCFsAAAAmImz1M3l5ebLZbMrOzvYYP3jwoObMmaOgoCCNGDFC8fHxqq+vt6ZIE3TVt81m6/J44YUXrCu0l3XVd0tLix555BGNGTNGQ4cO1cSJEz2++H0g6Krv48eP66GHHlJYWJiGDRumO+64Q4cPX/lf/r5ixYpOf4dDQkLc5w3D0IoVKxQWFqahQ4fqH/7hH/Txxx9bWHHvuFjf27ZtU2pqqoKDg2Wz2VRXV2ddsb2op76/+eYbPfHEE7rppps0fPhwhYWFaeHChTp27JjFVV++i/2+V6xYoejoaA0fPlzf+973NHPmTL333nsWVty3CFv9yAcffKDi4mLdfPPNHuOfffaZbrvtNkVHR+vdd9/VgQMH9NRTTykgIMCiSntXd307nU6P4+WXX5bNZtP9999vUaW9q7u+H3/8cf3xj3/Ur3/9ax08eFCPP/64Hn30Ue3YscOiSntXV30bhqF77rlHn3/+uXbs2KHa2lpFRkZq5syZ+vrrry2stnfceOONHn+XP/roI/e5X/7yl8rPz9dLL72kDz74QCEhIZo1a5bOnDljYcW9o6e+v/76ayUlJWn16tUWVmiO7vr+29/+pv379+upp57S/v37tW3bNh06dEhz5syxuOLe0dPve8KECXrppZf00UcfqbKyUmPHjlVKSoq++uorCyvuQxf9qmr0iTNnzhjjx483ysvLjRkzZhhLly51n0tLSzMefPBB64ozUU99X2ju3LnGP/7jP/ZdcSbqqe8bb7zRWLlypcf8W265xVi+fHkfV9n7uuv7008/NSQZf/nLX9xzz507Z4waNcr41a9+ZVG1vePpp582pkyZ0uW5jo4OIyQkxFi9erV77OzZs0ZQUJCxadOmPqrQHD31/V1HjhwxJBm1tbWm19QXLrXvb73//vuGJOPLL780r6g+4G3fp0+fNiQZb7/9tnlF9SPsbPUTS5Ys0Z133qmZM2d6jHd0dOj3v/+9JkyYoNTUVF133XWaNm2afve731lTaC/rru8LHT9+XL///e+VkZHRR5WZq6e+b7vtNu3cuVNHjx6VYRh65513dOjQIaWmplpQae/qru/W1lZJ8tit9fPzk7+//4D4JonDhw8rLCxMUVFR+vGPf6zPP/9cknTkyBG5XC6lpKS459rtds2YMUNVVVVWldtruut7oPOm79OnT8tms2nkyJF9V6BJLrXvtrY2FRcXKygoSFOmTOnjKq1B2OoHXnvtNe3fv195eXmdzjU2NqqlpUWrV6/WHXfcoT179ujee+/Vfffdp4qKCguq7T099X2h//zP/9SIESMGxLcKXKzvdevWadKkSRozZoz8/f11xx13qKioSLfddlsfV9q7euo7OjpakZGRys3N1f/93/+pra1Nq1evlsvlktPptKDa3jNt2jRt3bpVu3fv1q9+9Su5XC4lJibqxIkTcrlckiSHw+FxjcPhcJ+7UvXU90DmTd9nz57Vk08+qQceeECBgYEWVNt7LqXvt956S9dcc40CAgJUUFCg8vJyBQcHW1h1H7J6a+1qV19fb1x33XVGXV2de+y7b68cPXrUkGQsWLDA47q7777b+PGPf9yXpfaqi/V9oRtuuMF45JFH+qg681xK3y+88IIxYcIEY+fOncaBAweM9evXG9dcc41RXl5uQcW941L6/vDDD40pU6YYkgw/Pz8jNTXVmD17tjF79mwLKjZPS0uL4XA4jDVr1hh/+tOfDEnGsWPHPOYsWrTISE1NtahCc3y37+8aaG8jXqi7vtva2oy5c+caMTExxunTpy2qzjxd9d3S0mIcPnzYqK6uNv7lX/7FGDt2rHH8+HELq+w7hC2Lbd++3f2Py7eHJMNmsxl+fn7G2bNnjcGDBxvPPvusx3X/+q//aiQmJlpU9eW7WN/nzp1zz927d68hyeMf6ivVxfpuaWkxhgwZYrz11lse12VkZFzR//h68/s+deqU0djYaBiGYUydOtXIysqyqmzTzJw508jMzDQ+++wzQ5Kxf/9+j/Nz5swxFi5caFF15vm27+8a6GHLMDr33dbWZtxzzz3GzTffbDQ1NVlYmbm6+n1/1/XXX2+sWrWqDyuyzmALNtPwHbfffrvHJzYk6Z//+Z8VHR2tJ554Qna7Xbfeeqs+/fRTjzmHDh1SZGRkX5baqy7Wt5+fn3t8y5Ytio2NHRDv7V+s7/b2dn3zzTcaNMjzHX4/Pz91dHT0Zam9ypvfd1BQkKTz9398+OGHevbZZ/u0VrO1trbq4MGDSk5OVlRUlEJCQlReXq6YmBhJ5+9nqaio0PPPP29xpb3ru31fTS7s+5tvvtH8+fN1+PBhvfPOOxo9erTFFZrjUn7fhmG479cc6AhbFhsxYoQmT57sMTZ8+HCNHj3aPf6LX/xCaWlpmj59un74wx/qj3/8o9588029++67FlTcOy6lb0lqbm7W66+/rjVr1vR1iaa4lL5nzJihX/ziFxo6dKgiIyNVUVGhrVu3Kj8/34qSe8Wl9P3666/r2muvVUREhD766CMtXbpU99xzj8fN41ein//857r77rsVERGhxsZG/fu//7uam5v105/+1P2ssVWrVmn8+PEaP368Vq1apWHDhumBBx6wuvTL0lPfknTy5EnV19e7nzH17X8oQ0JCPJ7PdKXpqe9z585p3rx52r9/v9566y21t7e7780bNWqU/P39La7edz31/fXXX+u5557TnDlzFBoaqhMnTqioqEj/+7//q3/6p3+yuvS+YfXWGjrr6t6lLVu2GNdff70REBBgTJkyxfjd735nTXEm6qrvzZs3G0OHDjVOnTplTVF94MK+nU6n8dBDDxlhYWFGQECAccMNNxhr1qwxOjo6rCvSBBf2vXbtWmPMmDHGkCFDjIiICGP58uVGa2urdQX2krS0NCM0NNQYMmSIERYWZtx3333Gxx9/7D7f0dFhPP3000ZISIhht9uN6dOnGx999JGFFfeOi/X9yiuvGJI6HU8//bR1RfeCnvr+9i3Tro533nnH2sIvU099//3vfzfuvfdeIywszPD39zdCQ0ONOXPmGO+//77FVfcdm2EYhkU5DwAAYMDj0Q8AAAAmImwBAACYiLAFAABgIsIWAACAiQhbAAAAJiJsAQAAmIiwBQAAYCLCFgAAgIkIWwAAACYibAEAAJiIsAUAAGCi/weAznyXf9mZ/QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 700x300 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(7, 3))\n",
    "_ = plt.hist(bootstrap_means, density=True, edgecolor='black', linewidth=1.2, alpha=0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "f3f3506f",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([47.49108953, 51.3568865 ])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.percentile(bootstrap_means, (2.5, 97.5))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff3b87c0",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Interpreting the Bootstrap Values\n",
    "\n",
    "* The bootstrap data shows us the extent of sampling variability\n",
    "\n",
    "* The 95% confidence interval contains our population mean\n",
    "  * Values within 95% confidence interval are not a fluke\n",
    "  * We cannot discredit the fact that any of those values are possible means of the population\n",
    "\n",
    "* Conclusion: even with only 100 samples, we can observe that population mean is included in the interval [47.49, 51.36]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66810eaa",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### How Confident Are We in the Bootstrap Confidence Interval?\n",
    "\n",
    "* The estimate 95% confidence interval captures the population parameter\n",
    "\n",
    "  * Was that a fluke?\n",
    "\n",
    "* To see how frequently the interval contains the parameter, we have to run the entire process over and over again\n",
    "\n",
    "* We will repeat the following process a number of times (say 100 times):\n",
    "\n",
    "  * Generate a data sample of size 100\n",
    "  * Generate 10,000 bootstrap samples from the data sample\n",
    "  * Generate the 95% confidence of the mean\n",
    "  \n",
    "* We will end up with 100 intervals, and count how many of them contain the population mean\n",
    "  * How many of these 95 confidence intervals will include our mean?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "aa0b438f",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "def comput_conf_interval(data, nb_bootstrap_iters = 10_000):\n",
    "    bootstrap_means = []\n",
    "    for i in range(nb_bootstrap_iters):\n",
    "        bootstrap_sample = np.random.choice(data, 100, replace=True)\n",
    "        bootstrap_means.append(bootstrap_sample.mean())\n",
    "    return np.percentile(bootstrap_means, (2.5, 97.5))\n",
    "    \n",
    "lower_bound = []\n",
    "upper_bound = []\n",
    "for i in range(100):\n",
    "    data = np.random.normal(50, 10, 100)\n",
    "    conf_interval = comput_conf_interval(data)\n",
    "    lower_bound.append(conf_interval[0])\n",
    "    upper_bound.append(conf_interval[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "3d5e128e",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>lower</th>\n",
       "      <th>upper</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>48.857784</td>\n",
       "      <td>53.082003</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>48.448264</td>\n",
       "      <td>52.205987</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>49.248013</td>\n",
       "      <td>52.893485</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>49.127204</td>\n",
       "      <td>52.737125</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>47.576569</td>\n",
       "      <td>51.452966</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       lower      upper\n",
       "0  48.857784  53.082003\n",
       "1  48.448264  52.205987\n",
       "2  49.248013  52.893485\n",
       "3  49.127204  52.737125\n",
       "4  47.576569  51.452966"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "conf_ints_95 = pd.DataFrame({\"lower\": lower_bound, \"upper\": upper_bound})\n",
    "conf_ints_95.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "63627654",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(100, 2)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "conf_ints_95.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "fc556f89",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "97"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum((conf_ints_95[\"lower\"] < 50) & (conf_ints_95[\"upper\"] > 50))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "8e6d8260",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAz8AAAH5CAYAAACve4DDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAwlUlEQVR4nO3de5CV5X0H8N+CYQPMQiKpu2xdkXQWL6CWMSlGTdBRUWvVhmmN4hhjbUrAG5IQRZoWOrhbSIfZtt7AZoTUEvNHqrVpo2CqMA6m8VJTm1i0CVGirjuxDAvRLAqnf1BWF3aXvZxz3tvnM+OMvHuAx5kzPud3vu/3eWtKpVIpAAAAcm5E0gsAAACoBsMPAABQCIYfAACgEAw/AABAIRh+AACAQjD8AAAAhWD4AQAACuGIpBcwFPv27YvXX3896urqoqamJunlAAAACSmVSrFr165obGyMESP6z3YyOfy8/vrr0dTUlPQyAACAlNi+fXscffTR/b4mk8NPXV1dROz/Dxw3blzCqwEAAJLS2dkZTU1N3TNCfzI5/By41W3cuHGGHwAAYEB1GAceAAAAhWD4AQAACsHwAwAAFILhBwAAKATDDwAAUAiGHwAAoBAMPwAAQCEYfgAAgEIw/AAAAIVg+AEAAArB8AMAABSC4QcAACgEww8AAFAIhh8AAKAQDD8AAEAhGH4AAIBCMPwAAACFYPgBAAAK4YikFwBA8bz52BWx560f9bg2asIpUX/utxJaEQBFIPkBAAAKwfADAAAUguEHAAAoBMMPAABQCIMefjZv3hwXX3xxNDY2Rk1NTTz00EM9fl4qlWLp0qXR2NgYo0ePjrPOOit+/OMf93hNV1dX3HDDDfGxj30sxo4dG5dcckn84he/GNZ/CAAAQH8GPfz86le/ilNOOSXuuOOOXn++cuXKWLVqVdxxxx3x9NNPR0NDQ5x33nmxa9eu7tcsWLAgHnzwwXjggQfiySefjN27d8fv/d7vxd69e4f+XwIAANCPQR91feGFF8aFF17Y689KpVK0tbXFkiVLYvbs2RERsW7duqivr4/169fH3LlzY+fOnfGNb3wj/v7v/z7OPffciIi4//77o6mpKR577LE4//zzh/GfAwAA0Luydn62bdsW7e3tMWvWrO5rtbW1MXPmzNiyZUtERDz77LPx7rvv9nhNY2NjTJs2rfs1B+vq6orOzs4e/wAAAAxGWYef9vb2iIior6/vcb2+vr77Z+3t7TFq1Kj46Ec/2udrDtba2hrjx4/v/qepqamcywYAAAqgIqe91dTU9Ph1qVQ65NrB+nvN4sWLY+fOnd3/bN++vWxrBQAAiqGsw09DQ0NExCEJTkdHR3ca1NDQEHv27IkdO3b0+ZqD1dbWxrhx43r8AwAAMBiDPvCgP5MnT46GhobYuHFjTJ8+PSIi9uzZE5s2bYoVK1ZERMSpp54aH/rQh2Ljxo1x2WWXRUTEG2+8Ef/1X/8VK1euLOdyyKi9eyNKpaRXAVTSe3tHxHt7R/a4NmLviHjvvYQWlEM1NREjRx7+dQBFMujhZ/fu3fE///M/3b/etm1bPP/883HkkUfGMcccEwsWLIiWlpZobm6O5ubmaGlpiTFjxsScOXMiImL8+PFx7bXXxpe//OWYMGFCHHnkkfGVr3wlTjrppO7T3yi2u+6K6OMkdSAn9r799Sjt7epxrWZkbYwck9CCcuj66yNuuCHpVQCkS02pNLjv2J944ok4++yzD7l+9dVXx9q1a6NUKsWyZcti9erVsWPHjpgxY0bceeedMW3atO7X/vrXv45FixbF+vXr45133olzzjkn7rrrrgEfZNDZ2Rnjx4+PnTt3ugUuhyQ/kH9vfv/K2PPWf/a4NmrCyVF/zj8ktKL8kfwARTGY2WDQw08aGH4Asu3Nx66IPW/9qMe1URNOifpzv5XQigDIqsHMBmXt/AAMhHQPnR+oPmkgGH6ABOh1ofMD1acHBm57AxIg+UHnB6pP8kNeue0NSDWbL0eM3Bf7Ru495NoRdiUAKsg2AwybJIfB0vmhUqQbQH8MP8Cw6fAwWDo/VIpeC9AfnR9g2CQ/DJbOD5Ui+YHi0fkBqsoHDQZL5weAJNhm6Obbe6BadH6GT8IBMHiGH7rpbQDVovMzfLotAIOn80M3yQ9QLTo/wyf5AdhP54chsYkC1aLzA0ASbDNArkgws0HnB7JJ4kjWGX6AXNFdywadH8gmXTOyTucHyBXJTzbo/EA2SX5II50foLBsytmg8wNAEmwzAGUkeRoYnR/gYFIlqsHwA1BGOkcDo/MDHEyfiGrQ+QEoI8nPwOj8AAeT/DBUOj8ACbFxD4zODwBJsM0AkB2iNYC+ic8Oy/ADQHYoVQH0TXHqsAw/AGTH/PkR8+YlvQqAdKqpSXoFqWf4ASA73M4BwDAYfoBcUAXJFs/5AYZKrYXhMPwAuaAKki2e8wMMlVoLw+E5P0AuSH6yxXN+gKGS/HAwz/kBCsdGmC2e8wNAEkYkvQAAAIBq8B0bUAhui0sXBx5AermtjDwz/ACF4ECEdHHgAaSXAwXIMwceAIUg+UkXBx5Aekl+yBoHHgAcxEaeLg48ACAJthmgIiQt9EfnB4ZHOgNDY/gBKkLHhv7o/MDw6OXA0Oj8ABUh+aE/Oj8wPJIfeJ/OD5A4mzL90fkBIAm2GRIjGYDi0vk5PN/sA5Sf4YfE6IRAcen8HJ5OB0D56fyQGMkPFJfOz+FJfgAGRueHTLCpQ3Hp/ACQBNsMpIAUjKLR+aEvEi+gkgw/kAL6TxSNzg990XUCKknnB1JA8kPR6PzQF8kPMFg6P5AxNnqKRucHgCTYZig8qQtUn84PfZH8AJVk+KHw9G2g+nR+6IvOD1BJOj8UnuQHqk/nh75IfoDB0vmBQbDJQvXp/ACQBNsMMCySM4ZC5wd6J/mCyjL8AMOiM8VQ6PxA73SeoLJ0foBhkfwwFDo/0DvJDwyezg9QNTZphkLnB4Ak2Gag4CQ3JEHnhzSTvkB+GX6g4HR2SILOD2mmdwP5pfMDBSf5IQk6P6SZ5AeyRecHGDAbPEnQ+QEgCbYZCkG6Aemi85Mukg6gKAw/FIJeC6SLzk+66LgARaHzQyFIfiBddH7SRfIDZJnODxzEpg7povMDQBJGJL0AAACAavAdG1AYbn9MDwceQP65nZI0MvwAheHgi/Rw4AHkn4M0SCMHHgCFIflJDwceQP5JfqgWBx4A9MImnB4OPAAgCbYZgIQVMZHS+QEOkBBRTYYfgIQVsYuk8wMcoBtENen8ACSsiMmPzg9wgOSH4dL5AciQIm76Oj8AJME2A2VSxG/vYah0fvLNN/lAWpV9+Hnvvfdi6dKl8Q//8A/R3t4eEydOjC984Qvxp3/6pzFixIiIiCiVSrFs2bJYs2ZN7NixI2bMmBF33nlnTJ06tdzLgaopYm8DhkrnJ990OIC0Kvvws2LFirjnnnti3bp1MXXq1HjmmWfimmuuifHjx8dNN90UERErV66MVatWxdq1a2PKlCmxfPnyOO+882Lr1q1RV1dX7iVBVcyfHzFvXtKrgGx48/uLdH5yrKYm6RUA9K7sw89TTz0Vl156aVx00UUREXHsscfGt771rXjmmWciYn/q09bWFkuWLInZs2dHRMS6deuivr4+1q9fH3Pnzi33kqAq3OIBA6fzA0ASyr7NnHnmmXHPPffESy+9FFOmTIkf/ehH8eSTT0ZbW1tERGzbti3a29tj1qxZ3b+ntrY2Zs6cGVu2bOl1+Onq6oqurvdvj+js7Cz3soEq05EqNp0fqCy9K+hd2YefW265JXbu3BnHH398jBw5Mvbu3Ru33357XHHFFRER0d7eHhER9fX1PX5ffX19vPLKK73+ma2trbFs2bJyLxVIkI5Usen8QGXpXUHvyj78fPvb3477778/1q9fH1OnTo3nn38+FixYEI2NjXH11Vd3v67moBuCS6XSIdcOWLx4cSxcuLD7152dndHU1FTupQNVpCNVbDo/UFl6V9C7sg8/ixYtiltvvTUuv/zyiIg46aST4pVXXonW1ta4+uqro6GhISKi+yS4Azo6Og5Jgw6ora2N2traci8VSJDbMYpN5weAJJR9m3n77be7j7Q+YOTIkbFv376IiJg8eXI0NDTExo0bY/r06RERsWfPnti0aVOsWLGi3MsBck53KJt0fqBYdJBIi7IPPxdffHHcfvvtccwxx8TUqVPjP/7jP2LVqlXxR3/0RxGx/3a3BQsWREtLSzQ3N0dzc3O0tLTEmDFjYs6cOeVeDpBzukPZpPMDxaKDRFrUlErl/c50165d8bWvfS0efPDB6OjoiMbGxrjiiiviz/7sz2LUqFER8f5DTlevXt3jIafTpk0b0N/R2dkZ48ePj507d8a4cePKuXwgYyQ/2fTm96/U+YECkfxQSYOZDco+/FSD4Qcg29587IrY89aPelwbNeGUqD/3WwmtCICsGsxsoFoKCZJaUFQ6PxwgEQCqyfADCdJXoah0fjhAFwSoJre9QYIkPxSVzg8HSH6A4XLbG2SEDZ+i8pwfAJJgm2FQJBVAOej8pJckBsgzww+DoqMClIPOT3rp4AB5pvPDoEh+gHLQ+UkvyQ+QNTo/VIwNESgHnR8AkmCbAQpJipksnR9IloSPojL8AIWkv5YsnR9Ilm4XRaXzAxSS5CdZOj+QLMkPeaLzA3AYNv1k6fwAkATbDBSEpIM00fkhTaQgUByGHygIHRfSROeHNNF/geLQ+YGCkPyQJjo/pInkB7JN5wc4hI2dNNH5ASAJI5JeAAAAQDX4jg3ol9vlqAQHHpBHbp+D9DP8AP1yUAKV4MAD8sjBCZB+DjwA+iX5oRIceEAeSX4gGQ48AMrGRk4lOPAAgCTYZgAoHpEmwPBlMO40/ABQPMpsAMOXwaKb4QeA4pk/P2LevKRXAZBtNTVJr2DQDD8AFE/GbtMAoDwMP8CAqEhQTp7zA+mXwToHHJbhBxgQFQnKyXN+IP0yWOeAw/KcH2BAJD+Uk+f8QPpJfsgKz/kBys4GSDl5zg8ASbDNwCBJQGD4dH7gfRIWqB7DDwyS7gsMn84PvE+3BqpH5wcGSfIDw6fzA++T/MDw6PxABdmgYPh0fgBIgm0GSCUJW77p/ED5SZDg8Aw/QCrpVuWbzg+Un+4QHJ7OD5BKkp980/mB8pP8UFQ6P0Dm2cDzTecHgCTYZuD/SRqgenR+sk3CAGSV4Qf+n44JVI/OT7bplgBZpfMD/0/yA9Wj85Ntkh8gTXR+YAhs5FA9Oj8AJME2AwyLxIyh0PkhjyRikH6GH2BYdKUYCp0f8kgXCtJP5wcYFskPQ6HzQx5JfiAZOj9A1djoGQqdHwCSYJth2HzzDwxW2jo/vrEHKAbDD8Om8wEMVto6P7oaAMWg88OwSX6AwUpb50fyA5BdOj9UlQ8MwGDp/ACQhBFJLwAAAKAafMdGN7evAdWStgMPPsgtcAD5Zfihm4MLgGpJ24EHH+TwA4D8cuAB3SQ/QLWk7cCDD5L8AGSLAw8YEps9UC0OPAAgCbYZIDHSxuJKc+cHqkXKCNVn+AESo2dWXGnu/EC16JdB9en8AImR/BRXmjs/UC2SHygPnR8gE2z6xaXzA0ASbDNAoUmfkqHzA/km1SKtDD9AoekdJUPnB/JNn4m00vkBCk3ykwydH8g3yQ/VpPMDMEA252To/ACQBNsMVIA0Afqn8zM8vlUHGBrDD1SAHgn0T+dnePQpAIZG5wcqQPID/dP5GR7JD8D7dH4gYT6UQP90fgBIgm0GIIOyni7q/AB5I5HNBsMPQAZlvVem8wPkjS5eNuj8AGRQ1pMfnR8gbyQ/ydH5Aci5rG+wOj8AJME2A/D/sp6mZInOD6SP5IIiqMjw89prr8Utt9wS3/ve9+Kdd96JKVOmxDe+8Y049dRTIyKiVCrFsmXLYs2aNbFjx46YMWNG3HnnnTF16tRKLAdgQLLeo8kSnR9IH50ViqDsw8+OHTvijDPOiLPPPju+973vxVFHHRU//elP4yMf+Uj3a1auXBmrVq2KtWvXxpQpU2L58uVx3nnnxdatW6Ourq7cSwIYkPnzI+bNS3oVxfDm9xfp/EDK1NQkvQKovLIPPytWrIimpqa47777uq8de+yx3f9eKpWira0tlixZErNnz46IiHXr1kV9fX2sX78+5s6dW+4lAQyI2z2qR+cHgCSUfZt5+OGH4/zzz48//MM/jE2bNsVv/uZvxvz58+OLX/xiRERs27Yt2tvbY9asWd2/p7a2NmbOnBlbtmzpdfjp6uqKrq73b4/o7Ows97KBDNHNyT6dH8g+HSGyqOzDz89+9rO4++67Y+HChXHbbbfFD3/4w7jxxhujtrY2Pv/5z0d7e3tERNTX1/f4ffX19fHKK6/0+me2trbGsmXLyr1UIKN0c7JP5weyT0eILCr7c35GjRoVn/jEJ2LLli3d12688cZ4+umn46mnnootW7bEGWecEa+//npMnDix+zVf/OIXY/v27fHII48c8mf2lvw0NTV5zg8UlOQn+zznB7JP8kNaJPqcn4kTJ8aJJ57Y49oJJ5wQ3/nOdyIioqGhISIi2tvbeww/HR0dh6RBB9TW1kZtbW25lwpklM02+3R+AEhC2beZM844I7Zu3drj2ksvvRSTJk2KiIjJkydHQ0NDbNy4MaZPnx4REXv27IlNmzbFihUryr0chsC36kClJdX58U01QLGVffi5+eab4/TTT4+Wlpa47LLL4oc//GGsWbMm1qxZExERNTU1sWDBgmhpaYnm5uZobm6OlpaWGDNmTMyZM6fcy2EI9CmASkuq86OjAFBsZe/8RER897vfjcWLF8fLL78ckydPjoULF3af9hbx/kNOV69e3eMhp9OmTRvQnz+Y+/oYPMkPUGlJdX4kPwD5M5jZoCLDT6UZfgCy7c3Hrog9b/2ox7VRE06J+nO/ldCKAMiqRA88ALJL6ke1eM4PWSQ5hOwz/ADd9L2oFs/5IYt0xiD73PYGdJP8UC2e80MWSX4gndz2BgyJTZ1q8ZwfAJIwIukFAAAAVIPv2IDMcFtefjjwAIbHLXgwNIYfIDMcyJAfDjyA4XH4AgyNAw+AzJD85IcDD2B4JD/wPgceALlko88PBx4AkATbDJAYSU5x6fxQRNIaSJ7hB0iMDk9x6fxQRHo6kDydHyAxkp/i0vmhiCQ/UBk6P0Am+BBQXDo/ACTBNgPkhiQpO3R+YPAkRzB8hh8gN3SIskPnBwZPZwiGT+cHyA3JT3bo/MDgSX6gdzo/QCH5UJAdOj8AJME2A+SG5Cc7dH4gfSRLFIHhB8gNnZ/s0PmB9NEpogh0foDckPxkh84PpI/kh6zS+QEKyaadHTo/ACTBNgNVJJmA/XR+8kt6AKSZ4QeqSCcF9tP5yS+9ESDNdH6giiQ/sJ/OT35JfoBq0/mBlPKBAPbT+QEgCbYZKkLCAfRH5+dQEhOAyjP8UBG6LUB/dH4OpSsDUHk6P1SE5Afoj87PoSQ/AEOj80PibOBAf3R+AEiCbQYGQaIF5aHzw0BIw4ByM/zAIOgyQXno/DAQelBAuen8wCBIfqA8dH4YCMkPMBA6P1AhNmEoD50fAJJgmwEqQkpGf3R+yCppFGSb4QeoCP0o+qPzQ1bpIUG26fwAFSH5oT86P2SV5AfSR+cHSJwPB/RH5weAJIxIegEAAADV4Ds2CsttWZAcBx4Ui1vFgLQw/FBYCvmQHAceFItDAoC0cOABhSX5geQ48KBYJD9AJTnwAAbARgzJceABAEmwzQBkVJbTS50foJKkjfTF8AOQUVnuren8AJWkZ0ZfdH4AMirLyY/OD1BJkp9i0fkBKIAsb+w6PwAkwTZD5mT5225gP52f8vNNN8DhGX7InCz3HID9dH7KT8cB4PB0fsgcyQ9kn85P+Ul+gKLS+SHXbO6QfTo/ACTBNgNUjdSOA3R+SDtJGuST4QeoGn0tDtD5Ie10qCCfdH6AqpH8cIDOD2kn+YHs0PkBUskHCQ7Q+QEgCbYZcke6AOmn83N4kgeA8jP8kDt6JZB+Oj+Hp3MCUH46P+SO5AfST+fn8CQ/AAOj80Oh+bAA6afzA0ASbDMAAyBRLC+dH0iWZJGiMvwADIAuWXnp/ECydMooKp0fgAGQ/JSXzg8kS/JDnuj8AJSZDwnlpfMDQBJsM0AuSGayRecHOJg0imow/AC5oJOTLTo/wMH0kKgGnR8gFyQ/2aLzAxxM8sNQ6fwAhWPDzBadHwCSYJsBMk3ik006P5A+kheKwPADZJquTzbp/ED66NxQBDo/QKZJfrJJ5wfSR/JDVun8AIVho84mnR8AkmCbAfgASVJ16PxAMUmXSJrhB+ADdIiqQ+cHikmviKRVvPPT2toat912W9x0003R1tYWERGlUimWLVsWa9asiR07dsSMGTPizjvvjKlTpw7oz9T5ASpF8lMdOj9QTJIfKiE1nZ+nn3461qxZEyeffHKP6ytXroxVq1bF2rVrY8qUKbF8+fI477zzYuvWrVFXV1fJJQH0y6ZcHTo/ACRhRKX+4N27d8eVV14Z9957b3z0ox/tvl4qlaKtrS2WLFkSs2fPjmnTpsW6devi7bffjvXr11dqOQAAQMFV7Du26667Li666KI499xzY/ny5d3Xt23bFu3t7TFr1qzua7W1tTFz5szYsmVLzJ0795A/q6urK7q63r83vLOzs1LLBigbt9D1zYEHwMHcEkc1VGT4eeCBB+K5556Lp59++pCftbe3R0REfX19j+v19fXxyiuv9Prntba2xrJly8q/UIAKcnhC3xx4ABzMYQhUQ9mHn+3bt8dNN90UGzZsiA9/+MN9vq6mpqbHr0ul0iHXDli8eHEsXLiw+9ednZ3R1NRUngUDVMj8+RHz5iW9inR68/uLHHgA9NDHx0Aoq7IPP88++2x0dHTEqaee2n1t7969sXnz5rjjjjti69atEbE/AZo4cWL3azo6Og5Jgw6ora2N2traci8VoKLcvtE3Bx4AkISybzPnnHNOvPDCCz2uXXPNNXH88cfHLbfcEh//+MejoaEhNm7cGNOnT4+IiD179sSmTZtixYoV5V4OQOoVsRuk8wOkkd5R/pV9+Kmrq4tp06b1uDZ27NiYMGFC9/UFCxZES0tLNDc3R3Nzc7S0tMSYMWNizpw55V4OQOoVsRuk8wOkkd5R/iVyg8FXv/rVeOedd2L+/PndDzndsGGDZ/wAhVTEbpDOD5BGekf5V1MqZe9mi8E8xRWA9HnzsStiz1s/6nFt1IRTov7cbyW0IgCyajCzgWophewbAMmqZOfHPfsA9MXwQyH7BkCyKtn5cc8+AH1x2xuSH6Dq3vz+lRXr/Eh+AIrFbW8Mig8JQLV5zg8ASbDNAEMiMWQ4POcH+ifBhMow/ABDoivGcHjOD/RPdw0qQ+cHGBLJD8NRyc4P5IHkBwZO5weoOJsyw6HzA0ASbDNAxUiH6IvOD1SXJAn2M/wAFaMXRF90fqC6dIhgP50foGIkP/RF5weqS/JDnun8AKlgo6UvOj8AJME2A2SORCn7dH6A/kiqqBTDD5A5ukTZp/MD9EdHiUrR+QEyR/KTfTo/QH8kPwyGzg+QazbE7NP5ASAJthkoAEkJaaPzQ5FIMSA9DD9QADoypI3OD0WivwLpofMDBSD5IW10figSyQ9Uls4P0INNl7TR+QEgCbYZKkriAPRG52fopAgAQ2f4oaJ0TYDe6PwMnf4IwNDp/FBRkh+gNzo/Qyf5AehJ54fUsEEDvdH5ASAJI5JeAAAAQDX4ji3n3HYGpFFaDjxwCxlAsRh+cs6BA0AapeXAA4cHABSLAw9yTvIDpFFaDjyQ/ABknwMP6GZTB9LIgQcAJME2A5BTaU5+09L5AYpN+ls8hh+AnEpz5y8tnR+g2PT+ikfnByCn0pz8pKXzAxSb5CcfdH4ASPWGrvMDQBJsM0CqpTm9YOh0fiB/pChkgeEHSLU091YYOp0fyB/9GbJA5wdINclPPun8QP5IfkiKzg+QGzbSfNL5ASAJtply8fU0UCS+4gUggww/5aKYABSJm/sByCDDT7nMnx8xb17SqwCojpqapFcAAINm+CkXt38AAECqGX6AilKHozee80NWqLdBvhh+gIpSh6M3nvNDVqi3Qb54zg9QUZIfeuM5P2SF5AfSz3N+gNTwoYHeeM4PAEmwzeSQb9qBtNP5GT6JBMDgGX5ySMcCSDudn+HTRQEYPJ2fHJL8AGmn8zN8kh+A/XR+Cs5mCKSdzg8ASbDNAP2SJFIJOj9Ui4QM+CDDD9AvHTIqQeeHatGNAj5I5wfol+SHStD5oVokP5B/Oj9A2fjQQCXo/ACQBNsM0C/JD5Wg8wMDJ72C8jH8AP3S+aESdH5g4PSWoHx0foB+SX6oBJ0fGDjJD/RP5wcoGxsulaDzA0ASbDMwDFIRGBqdn2ySQABZZ/iBYdCHgaHR+ckm3RMg63R+YBgkPzA0Oj/ZJPkB0kjnB6rEhwAYGp0fAJIwIukFAAAAVIPv2IBCcItiujjwgDxzeyCkl+EHKASHU6SLAw/IMwdDQHo58AAoBMlPujjwgDyT/EB1OfAA4CA+iKSLAw8ASIJthmHxbTowFJXq/PjGHYD+GH4YFj0KYCgq1fnRtQCgPzo/DIvkBxiKSnV+JD8AxaPzQ9X4kAEMhc4PAEmwzUCGSNrIC8/54XCkeEAllH34aW1tjX/8x3+M//7v/47Ro0fH6aefHitWrIjjjjuu+zWlUimWLVsWa9asiR07dsSMGTPizjvvjKlTp5Z7OZArOlbkhef8cDj6W0AllL3zc8EFF8Tll18en/zkJ+O9996LJUuWxAsvvBA/+clPYuzYsRERsWLFirj99ttj7dq1MWXKlFi+fHls3rw5tm7dGnV1dYf9O3R+KCrJD3nhOT8cjuQHGKhEOz+PPPJIj1/fd999cdRRR8Wzzz4bn/nMZ6JUKkVbW1ssWbIkZs+eHRER69ati/r6+li/fn3MnTu33EuC3PBBgLzQ+QEgCRXfZnbu3BkREUceeWRERGzbti3a29tj1qxZ3a+pra2NmTNnxpYtW3odfrq6uqKr6/3bIzo7Oyu8avJIagLpofMzdBIRgKGr6PBTKpVi4cKFceaZZ8a0adMiIqK9vT0iIurr63u8tr6+Pl555ZVe/5zW1tZYtmxZJZdKAejLQHro/AydLgzA0FV0+Ln++uvjP//zP+PJJ5885Gc1NTU9fl0qlQ65dsDixYtj4cKF3b/u7OyMpqam8i6W3Js/P2LevKRXAUREvPn9RTo/Q9THVgnAAFRs+Lnhhhvi4Ycfjs2bN8fRRx/dfb2hoSEi9idAEydO7L7e0dFxSBp0QG1tbdTW1lZqqRSE20QgPXR+AEhC2beZUqkUN9xwQzz44IPxxBNPxOTJk3v8fPLkydHQ0BAbN26M6dOnR0TEnj17YtOmTbFixYpyLwcoKB2vdNP5AYZK743hKPvwc91118X69evjn/7pn6Kurq674zN+/PgYPXp01NTUxIIFC6KlpSWam5ujubk5WlpaYsyYMTFnzpxyLwcoKB2vdNP5AYZK743hKPtzfvrq7dx3333xhS98ISLef8jp6tWrezzk9MChCIfjOT/A4Uh+0s1zfoChkvxwsESf8zOQWaqmpiaWLl0aS5cuLfdfDxARNsa00/kBIAm2GSARkpli0/mBnqQZUB2GHyAROjnFpvMDPemxQHWUvfNTDTo/kH2Sn2LT+YGeJD8wdIl2fgAGwiZfbDo/ACTBNgMFIGUhbXR+8k2KAaSV4QcKQL+GtNH5yTf9FSCtdH6gACQ/pI3OT75JfoBq0vkBevAhhLTR+QEgCbaZnPONP5BGeev8SDoAssHwk3O6HkAa5a3zo+MCkA06Pzkn+QHSKG+dH8kPQHJ0fuhmMwbSSOcHgCSMSHoBAAAA1eA7NgbNrXTAcOXlwAO3uwFki+GHQXOIAjBceTnwwEEHANniwAMGTfIDDFdeDjyQ/AAkz4EHVJSNHhguBx4AkATbTI5IZICsyEvnZzikRgDVZ/jJEV0cICvy0vkZDn0hgOrT+ckRyQ+QFXnp/AyH5AegPHR+CsomCmSFzg8ASbDNAAyDxHVodH6AwZCUUi6GH4Bh0LUbGp0fYDB05CgXnR+AYZD8DI3ODzAYkh/6o/MDUCU246HR+QEgCbYZINMkL9mk8wPZI30hDww/QKbp3GSTzg9kj94NeaDzA2Sa5CebdH4geyQ/pJXOD1AYNuJs0vkBIAm2GQD6VYl0TecHyAqJV74YfgDoVyV6VTo/QFboOuWLzg8A/apE8qPzA2SF5Cf9dH4AKJtKbPo6PwAkwTaTQU63ArKu3J0f38wCMBCGnwzyXBMg68rd+XFPPgADofOTQZIfIOvK3fmR/AAUl85PztnggazT+QEgCbYZKDhJIknwnB/KSfIHDJThBwpOh4wkeM4P5aTzBQyUzg8UnOSHJHjOD+Uk+YFi0/kBBswHBpKg8wNAEmwzkELSGPJO54dqkQoBH2T4gRTSwyHvdH6oFn0g4IN0fiCFJD/knc4P1SL5gfzT+YGMs1GTdzo/ACTBNlMFvsUH6Ongzk9Njf9JAlB5hp8q0N8A6Ongzs/V56+PuVf+V4IrAqAIDD9VMH9+xLx5Sa8CID3e/P6iHp2f/cnPScktCIBCMPxUgf4GQE+9dX4AoNJGJL0AAACAapD85IiDFYCsyNJDTh2VDJAfhp8ccbACkBVZesiph2QC5IeHnOaI5AfIiiw95FTyA5BuHnJaUDZnICs85BSAJNhmgFSSZOZbljo/kBVSSjg8ww+QSjps+Zalzg9khX4aHJ7OD5BKkp98y1LnB7JC8kNR6fwAmWcDzzedHwCSYJuBMpFUwMDp/BSXdAJIkuEHykRHBQZO56e49FKAJOn8QJlIfmDgdH6KS/IDlJvODyTAZg4Dp/MDQBJsMwAVIAnsn84PZIe0jjwx/ABUgA5Y/3R+IDv0tMgTnR+ACpD89E/nB7JD8kPa6fwAJMwHhf7p/ACQBNsMQMKKmBLp/ED+SYxII8MPQMKK2A/S+YH80xUijXR+ABJWxORH5wfyT/JDtej8AGRIET8c6PwAkATbDFBVRUw5OJTOD0UlDYFkJTr83HXXXfH1r3893njjjZg6dWq0tbXFpz/96SSXBFRYEfstHErnh6LSg4FkJdb5+fa3vx1XXXVV3HXXXXHGGWfE6tWr4+/+7u/iJz/5SRxzzDH9/l6dH8guyQ8ROj8Ul+QHyi8TnZ9Vq1bFtddeG3/8x38cERFtbW3x6KOPxt133x2tra1JLQuoMJs+ETo/ACQjkW1mz5498eyzz8att97a4/qsWbNiy5Yth7y+q6srurrevz2is7Oz4msE8k8KlRydH+BwpGRUQiLDzy9/+cvYu3dv1NfX97heX18f7e3th7y+tbU1li1bVq3lAQWhf5Qkt7cB/dOPohISvcGgpqamx69LpdIh1yIiFi9eHAsXLuz+dWdnZzQ1NVV8fUC+zZ8fMW9e0qsAoDe9fCSEYUtk+PnYxz4WI0eOPCTl6ejoOCQNioiora2N2traai0PKAi3UwBAsYxI4i8dNWpUnHrqqbFx48Ye1zdu3Binn356EksCAAByLrHb3hYuXBhXXXVVfOITn4hPfepTsWbNmnj11VfjS1/6UlJLAgAAciyx4edzn/tcvPXWW/EXf/EX8cYbb8S0adPiX//1X2PSpElJLQkAAMixxB5yOhwecgoAAEQMbjZIpPMDAABQbYYfAACgEAw/AABAIRh+AACAQjD8AAAAhWD4AQAACsHwAwAAFILhBwAAKATDDwAAUAiGHwAAoBAMPwAAQCEYfgAAgEIw/AAAAIVwRNILGIpSqRQREZ2dnQmvBAAASNKBmeDAjNCfTA4/u3btioiIpqamhFcCAACkwa5du2L8+PH9vqamNJARKWX27dsXr7/+etTV1UVNTU3V/t7Ozs5oamqK7du3x7hx46r298Jgea+SFd6rZIH3KVlR1PdqqVSKXbt2RWNjY4wY0X+rJ5PJz4gRI+Loo49O7O8fN25cod5QZJf3KlnhvUoWeJ+SFUV8rx4u8TnAgQcAAEAhGH4AAIBCMPwMQm1tbfz5n/951NbWJr0U6Jf3KlnhvUoWeJ+SFd6rh5fJAw8AAAAGS/IDAAAUguEHAAAoBMMPAABQCIYfAACgEAw/AABAIRh+DqO1tTVqampiwYIFPa6/+OKLcckll8T48eOjrq4uTjvttHj11VeTWSRE7+/V3bt3x/XXXx9HH310jB49Ok444YS4++67k1skhbR06dKoqanp8U9DQ0P3z0ulUixdujQaGxtj9OjRcdZZZ8WPf/zjBFdMUfX3Xn333XfjlltuiZNOOinGjh0bjY2N8fnPfz5ef/31hFdN0Rzu/6kfNHfu3KipqYm2trbqLjLFjkh6AWn29NNPx5o1a+Lkk0/ucf2nP/1pnHnmmXHttdfGsmXLYvz48fHiiy/Ghz/84YRWStH19V69+eab4/HHH4/7778/jj322NiwYUPMnz8/Ghsb49JLL01otRTR1KlT47HHHuv+9ciRI7v/feXKlbFq1apYu3ZtTJkyJZYvXx7nnXdebN26Nerq6pJYLgXW13v17bffjueeey6+9rWvxSmnnBI7duyIBQsWxCWXXBLPPPNMUsuloPr7f+oBDz30UPz7v/97NDY2VnNpqWf46cPu3bvjyiuvjHvvvTeWL1/e42dLliyJ3/3d342VK1d2X/v4xz9e7SVCRPT/Xn3qqafi6quvjrPOOisiIv7kT/4kVq9eHc8884zhh6o64ogjev1mslQqRVtbWyxZsiRmz54dERHr1q2L+vr6WL9+fcydO7faS6Xg+nqvjh8/PjZu3Njj2t/+7d/G7/zO78Srr74axxxzTLWWCH2+Tw947bXX4vrrr49HH300LrrooiquLP3c9taH6667Li666KI499xze1zft29f/Mu//EtMmTIlzj///DjqqKNixowZ8dBDDyWzUAqvr/dqRMSZZ54ZDz/8cLz22mtRKpXi8ccfj5deeinOP//8BFZKkb388svR2NgYkydPjssvvzx+9rOfRUTEtm3bor29PWbNmtX92tra2pg5c2Zs2bIlqeVSYH29V3uzc+fOqKmpiY985CPVWyBE/+/Tffv2xVVXXRWLFi2KqVOnJrjKdDL89OKBBx6I5557LlpbWw/5WUdHR+zevTv+8i//Mi644ILYsGFDfPazn43Zs2fHpk2bElgtRdbfezUi4m/+5m/ixBNPjKOPPjpGjRoVF1xwQdx1111x5plnVnmlFNmMGTPim9/8Zjz66KNx7733Rnt7e5x++unx1ltvRXt7e0RE1NfX9/g99fX13T+DaunvvXqwX//613HrrbfGnDlzYty4cQmslqI63Pt0xYoVccQRR8SNN96Y8ErTyW1vB9m+fXvcdNNNsWHDhl47PPv27YuIiEsvvTRuvvnmiIj47d/+7diyZUvcc889MXPmzKqul+I63Hs1Yv/w84Mf/CAefvjhmDRpUmzevDnmz58fEydO7DUpgkq48MILu//9pJNOik996lPxW7/1W7Fu3bo47bTTIiKipqamx+8plUqHXINK6++9unDhwu6fvfvuu3H55ZfHvn374q677kpiqRRYf+/TmTNnxl//9V/Hc8895/+hfTD8HOTZZ5+Njo6OOPXUU7uv7d27NzZv3hx33HFH/OpXv4ojjjgiTjzxxB6/74QTTognn3yy2sulwA73Xt25c2fcdttt8eCDD3bf73vyySfH888/H3/1V39l+CExY8eOjZNOOilefvnl+P3f//2IiGhvb4+JEyd2v6ajo+OQNAiq7YPv1QPefffduOyyy2Lbtm3xb//2b1IfEvfB9+mIESOio6OjRwdt79698eUvfzna2tri5z//eXILTQnDz0HOOeeceOGFF3pcu+aaa+L444+PW265JWpra+OTn/xkbN26tcdrXnrppZg0aVI1l0rBHe69unfv3nj33XdjxIied7eOHDmyO8GEJHR1dcWLL74Yn/70p2Py5MnR0NAQGzdujOnTp0dExJ49e2LTpk2xYsWKhFdK0X3wvRrx/uDz8ssvx+OPPx4TJkxIeIXQ83161VVXHfLl5vnnnx9XXXVVXHPNNQmtMF0MPwepq6uLadOm9bg2duzYmDBhQvf1RYsWxec+97n4zGc+E2effXY88sgj8c///M/xxBNPJLBiimog79WZM2fGokWLYvTo0TFp0qTYtGlTfPOb34xVq1YlsWQK6itf+UpcfPHFccwxx0RHR0csX748Ojs74+qrr+5+NlVLS0s0NzdHc3NztLS0xJgxY2LOnDlJL52C6e+9+t5778Uf/MEfxHPPPRff/e53Y+/evd29tCOPPDJGjRqV8Oopiv7epxMmTDhkKP/Qhz4UDQ0NcdxxxyW04nQx/AzBZz/72bjnnnuitbU1brzxxjjuuOPiO9/5jhI5qfPAAw/E4sWL48orr4z//d//jUmTJsXtt98eX/rSl5JeGgXyi1/8Iq644or45S9/Gb/xG78Rp512WvzgBz/oTsu/+tWvxjvvvBPz58+PHTt2xIwZM2LDhg2e8UPV9fde/fnPfx4PP/xwROzv+n7Q448/3v1IAai0w/0/lf7VlEqlUtKLAAAAqDRHXQMAAIVg+AEAAArB8AMAABSC4QcAACgEww8AAFAIhh8AAKAQDD8AAEAhGH4AAIBCMPwAAACFYPgBAAAKwfADAAAUwv8BoPVwqdAHPOcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1000x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(10, 6))\n",
    "plt.vlines(50, 0, 102, color=\"#e2a829\", linewidth=4)\n",
    "\n",
    "for i in range(100):\n",
    "    c = \"blue\"\n",
    "    if lower_bound[i] > 50 or upper_bound[i] < 50:\n",
    "        c = \"red\"\n",
    "    plt.hlines(i, lower_bound[i], upper_bound[i], color=c, alpha=0.85, linewidth=0.75)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a8e58cc",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### How Confident Are We in the Bootstrap Confidence Interval? -- Cont'd\n",
    "\n",
    "* If an interval doesn't cover the parameter, it's a failure \n",
    "  * There are very few of them in this case\n",
    "* Any method based on sampling has the possibility of being off\n",
    " * If we have a 95% confidence interval, we should expect to be wrong 5% of the time\n",
    "   * We are here about statistical expectation over a long run of experiments\n",
    "   *  if you were to construct confidence intervals an extremely large number of times under the same conditions, 95% of those intervals would contain the true parameter value, leaving 5% that do not.\n",
    "* The beauty of methods based on random sampling is that we can quantify how often they are likely to be off\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08b259be-1e45-4e09-89ae-32a04a60375f",
   "metadata": {},
   "source": [
    "### Misinterpretations and Misunderstandings or Confidence Intervals\n",
    "  * A 95% confidence interval does not mean there is a 95% chance that a given interval contains the true parameter value.\n",
    "     * The probability of a wrong interval containing the data is 0. \n",
    "  * It means that 95% of such intervals constructed from repeated sampling will contain the true parameter.\n",
    "  This distinction is subtle but fundamental to understanding statistical confidence.\n",
    "\n",
    "* Once a specific interval has been calculated from a specific sample, it either does contain the true parameter or it doesn't.\n",
    "*  the interval itself doesn't have a probability attached to it because it's fixed.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "abfbca11",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### The Bootstrap Confidence Interval\n",
    "\n",
    "* Our interval of estimates is called a *95% confidence interval* for the parameter\n",
    "\n",
    "* 95% is called the *confidence level* of the interval\n",
    "\n",
    "* We say that we are *95% confident* that the process results in a good interval"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c79c3e3",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Why is the Bootstrap a Good Idea? \n",
    "\n",
    "* Similarity by transitivity\n",
    "  *  By the law of averages, the distribution of the original sample is likely to resemble the population. \n",
    "  * So, the distributions of all the \"resamples\" are likely to resemble the original sample.\n",
    "  * So the distributions of all the resamples are likely to resemble the population as well. \n",
    "* Treat the original sample as if it were the entire population.\n",
    "* Draw from the sample at random **with** replacement.\n",
    "  * The resample (bootstrap sample) should be of the same size as the original one.\n",
    "    * Avoids any discrepencies that may be due to size alone.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a20d068",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<img src=\"https://www.dropbox.com/scl/fi/xaxy7lnz2xyktcdwi3e0e/bootstrap_pic.png?rlkey=btyevb4egdfsv1dn12c29vt7q&dl=1\" alt=\"drawing\" style=\"width:950px\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c680d52",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "### Care in Using the Bootstrap ###\n",
    "\n",
    "* The bootstrap is an elegant and powerful method to assess the accuracy of the estimate\n",
    "\n",
    " * The bootstrap uses a different approach to estimating the standard error\n",
    "\n",
    "\n",
    "* What you need to know before using it:\n",
    "\n",
    "  * Start with a large random sample\n",
    "\n",
    "  * The Law of Averages says that this is likely to be true provided the random sample is large\n",
    "\n",
    "\n",
    "* To approximate the probability distribution of a statistic, it is a good idea to replicate the resampling procedure as many times as possible\n",
    "\n",
    "  * 10,000, in general, is a good estimate\n",
    "\n",
    "* The bootstrap percentile method works well for estimating the population median or mean based on a large random sample"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8145c4c",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "### When To Not Use the Bootstrap\n",
    "\n",
    "* The bootstrap is not useful in the following situations:\n",
    "\n",
    "    * The goal is to estimate the minimum or maximum value in the population, or a very low or very high percentile, or parameters that are greatly influenced by rare elements of the population\n",
    "    * The probability distribution of the statistic is not roughly bell-shaped\n",
    "      * Can withstand some skewness in the data\n",
    "    * The original sample is very small, say less than 20 or 25\n",
    "      * Sample is not representative of the population\n",
    "      * Bootstrap samples are not representative of samples from the population"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}