power

book/_build/html/_sources/more.ipynb

@@ -6,7 +6,7 @@
    "metadata": {},
    "source": [
     "(more)=\n",
-    "# More About Tests of Significance\n",
+    "# More About Significance\n",
     "\n",
     "```{admonition} Important Readings\n",
     ":class: seealso\n",
@@ -86,7 +86,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "638ae2f5",
+   "id": "1798fe33",
    "metadata": {},
    "outputs": [],
    "source": []

book/_build/html/_sources/power.ipynb

@@ -16,9 +16,23 @@
     "\n",
     "A **Type I error** is made by rejecting the null hypothesis when the null hypothesis is true. A **Type II Error** is made when failing to reject the null hypothesis when the null hypothesis is false. While some mistakes, like snooping, are a matter of bad practice and can be avoided, Type I and II errors are unavoidable.\n",
     "\n",
-    "Hypothesis testing involves a binary decision. If we compare this to the deliberations of a judge, it is the null hypothesis that is on trial. Rejecting the hypothesis is akin to judging it to be guilty. Failing to reject the null hypothesis is akin to acquitting it, and this might be considered a negative result. This corresponds to the taxonomy in the table below. \n",
     "\n",
-    "Continuing the judicial analogy, a Type I error is convicting the true, innocent null hypothesis. A Type II error lets the false, crooked null hypothesis off the hook. \n",
+    "```{list-table}\n",
+    ":header-rows: 1\n",
+    "\n",
+    "* - \n",
+    "  - $H_0$ is true\n",
+    "  - $H_0$ is false\n",
+    "* - Don't Reject\n",
+    "  - True Negative\n",
+    "  - <span style=\"color:red;\">False Negative (Type II Error)</span>\n",
+    "* - Reject\n",
+    "  - <span style=\"color:red;\">False Positive (Type I Error)</span>\n",
+    "  - True Positive\n",
+    "```\n",
+    "\n",
+    "\n",
+    "Hypothesis testing involves a binary decision. If we compare this to the deliberations of a judge, it is the null hypothesis that is on trial. Rejecting the hypothesis is akin to judging it to be guilty. Failing to reject the null hypothesis is akin to acquitting it, and this might be considered a negative result. Continuing the judicial analogy, a Type I error is convicting the true, innocent null hypothesis. A Type II error lets the false, crooked null hypothesis off the hook. \n",
     "\n",
     "Statisticians use $\\alpha$ and $\\beta$ to denote the Type I and II conditional error rates. I call them *conditional* error rates to emphasize that each is a conditional probability. \n",
     "\n",
@@ -49,13 +63,20 @@
     ":width: 80%\n",
     ":name: power_stacked_vary_alpha\n",
     "\n",
-    "As $\\alpha$ increases, $\\beta$ decreases. \n",
+    "As $\\alpha$ increases, $\\beta$ decreases and power increases. The power is the unshaded region under the orange curve. \n",
     "```\n",
     "\n",
-    "Suppose you wanted 95% power in the illustration above ($\\beta = 0.05$). According to the null hypothesis, a $z$-statistic will be drawn from a standard normal distribution (the top panel). We simplify the world to consider a single alternative hypothesis, under which the $z$-statistic is actually drawn from a distribution centered at two. To force $\\beta = 0.05$, the vertical line must be 1.645 standard deviations away from the mean of the alternative distribution. Accordingly our critical value is $z^\\star = 2-1.645 = 0.355$. A $z$-table helps show this corresponds to $\\alpha \\approx 0.361$.\n",
+    "Above, we simplify the world to consider a single alternative hypothesis, under which the $z$-statistic is actually drawn from a distribution centered at two. Suppose you wanted 95% power in the illustration above ($\\beta = 0.05$). According to the null hypothesis, a $z$-statistic will be drawn from a standard normal distribution (the top panel). To force $\\beta = 0.05$, the vertical line must be 1.645 standard deviations away from the mean of the alternative distribution. Accordingly our critical value is $z^\\star = 2-1.645 = 0.355$. A $z$-table helps show this corresponds to $\\alpha \\approx 0.361$.\n",
     "\n",
-    "Second, $n$ increases power by lowering the standard errors and thus making the sampling distribution for the sample mean more narrow. This is because the standard error for such a distribution is $\\text{SE} = \\frac{\\text{SD}}{\\sqrt{n}}$. With less overlapping area, greater power is achieved. \n",
+    "Second, this analysis was based on an **effect size** corresponding to the alternative distribution being centered at two. A bigger effect size will spread the two distributions further apart. What if we have a smaller effect size? The power will be lower for a fixed $\\alpha$. What $\\alpha$ is required if the test statistic was actually drawn from a distribution centered at 1.645? Why is this unrealistic? \n",
+    "\n",
+    "```{dropdown} Smaller Effect Size\n",
+    "This would require $\\alpha = 0.5$ and $z^\\star = 0$. This is not realistic because $\\alpha = 0.5$ is unreasonably high. If the null is true, you expect the test statistic to equal the corresponding critical value. \n",
+    "```\n",
     "\n",
+    "Third, higher $n$ increases power by lowering the standard errors and thus making the sampling distribution for the sample mean more narrow. This is because the standard error for such a distribution is $\\text{SE} = \\frac{\\text{SD}}{\\sqrt{n}}$. With less overlapping area, greater power is achieved. \n",
+    "\n",
+    "In real-world applications, a *power analysis* is usually done before launching an experiment. While 95% is the typical confidence level, 80% is the typical power chosen. A power analysis takes these targets as given and then finds the required number of observations, $n$. \n",
     "\n",
     "\n",
     "#### Is power relevant in the world of big data? \n",
@@ -69,7 +90,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "09c9b3d1",
+   "id": "bda0df37",
    "metadata": {},
    "source": [
     "## Exercises\n",
@@ -83,6 +104,14 @@
     "```{exercise-end}\n",
     "```"
    ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7947ac30",
+   "metadata": {},
+   "outputs": [],
+   "source": []
   }
  ],
  "metadata": {

book/_build/html/_sources/sampling.md

@@ -215,10 +215,19 @@ A user researcher at a large ecommerce company wants to have a representative sa
 ```{exercise-end}
 ```
 
+
+```{exercise-start}
+:label: netflix
+```
+Netflix replaced the original 1-5 star moving-rating system with a thumbs up/thumbs down scale. The goal was to have users rate more movies so they could increase the quality of their recommendations. Is this change better explained as an effort to combat selection bias, response bias, or non-response bias? 
+
+```{exercise-end}
+```
+
 ```{exercise-start}
 :label: rivera
 ```
-{cite}`rivera2019scaling` finds that "that the number of scale points used in faculty teaching evaluations—whether instructors were rated on a scale of 6 versus a scale of 10—significantly affected the size of the gender gap in evaluations in the most male-dominated fields." That is, women receive better ratings relative to men when rated out of 6 instead of out of 10. Is this difference, driven by the scales, better described as selection bias, response bias, or non-response bias? 
+{cite}`rivera2019scaling` finds that "that the number of scale points used in faculty teaching evaluations—whether instructors were rated on a scale of 6 versus a scale of 10—significantly affected the size of the gender gap in evaluations in the most male-dominated fields." That is, women receive better ratings relative to men when rated out of 6 instead of out of 10. Is this difference, driven by the scales, better explained by selection bias, response bias, or non-response bias? 
 
 ```{exercise-end}
 ```

book/_build/html/confidenceintervals.html

@@ -201,7 +201,7 @@
 <li class="toctree-l1"><a class="reference internal" href="hypothesistesting.html">Hypothesis Testing</a></li>
 <li class="toctree-l1"><a class="reference internal" href="twosample.html">Two-Sample Tests</a></li>
 <li class="toctree-l1"><a class="reference internal" href="chi2.html">The Chi-Square Test</a></li>
-<li class="toctree-l1"><a class="reference internal" href="more.html">More About Tests of Significance</a></li>
+<li class="toctree-l1"><a class="reference internal" href="more.html">More About Significance</a></li>
 <li class="toctree-l1"><a class="reference internal" href="bibliography.html">Bibliography</a></li>
 </ul>
 <p aria-level="2" class="caption" role="heading"><span class="caption-text">Google Sheets (optional)</span></p>
@@ -549,14 +549,14 @@ <h2>The Accuracy of Averages<a class="headerlink" href="#the-accuracy-of-average
 <h2>Exercises<a class="headerlink" href="#exercises" title="Permalink to this heading">#</a></h2>
 <div class="exercise admonition" id="CIicecream">
 
-<p class="admonition-title"><span class="caption-number">Exercise 42 </span></p>
+<p class="admonition-title"><span class="caption-number">Exercise 43 </span></p>
 <section id="exercise-content">
 <p>In a survey of 96 randomly selected Americans, <span class="math notranslate nohighlight">\(\hat{p} = 0.60\)</span> said they think ice cream should be banned. Find the 95% confidence interval for the proportion.</p>
 </section>
 </div>
 <div class="exercise admonition" id="CInarrow">
 
-<p class="admonition-title"><span class="caption-number">Exercise 43 </span></p>
+<p class="admonition-title"><span class="caption-number">Exercise 44 </span></p>
 <section id="exercise-content">
 <p>You are interested in the 95% confidence interval of a sample mean. Which of the following makes this interval more narrow?</p>
 <ol class="arabic simple">
@@ -570,7 +570,7 @@ <h2>Exercises<a class="headerlink" href="#exercises" title="Permalink to this he
 </div>
 <div class="exercise admonition" id="CImc">
 
-<p class="admonition-title"><span class="caption-number">Exercise 44 </span></p>
+<p class="admonition-title"><span class="caption-number">Exercise 45 </span></p>
 <section id="exercise-content">
 <p>Consider a multiple choice exam with choices A, B, C, D, and E for each question. Assume that the probability that a particular question has the answer <span class="math notranslate nohighlight">\(C\)</span> is <span class="math notranslate nohighlight">\(p = 0.2\)</span> and that all letter answers are determined independently. Find a 90% confidence interval for the proportion of questions that will have a correct answer of C on a 1600-question exam.</p>
 </section>

book/_build/html/genindex.html

@@ -196,7 +196,7 @@
 <li class="toctree-l1"><a class="reference internal" href="hypothesistesting.html">Hypothesis Testing</a></li>
 <li class="toctree-l1"><a class="reference internal" href="twosample.html">Two-Sample Tests</a></li>
 <li class="toctree-l1"><a class="reference internal" href="chi2.html">The Chi-Square Test</a></li>
-<li class="toctree-l1"><a class="reference internal" href="more.html">More About Tests of Significance</a></li>
+<li class="toctree-l1"><a class="reference internal" href="more.html">More About Significance</a></li>
 <li class="toctree-l1"><a class="reference internal" href="bibliography.html">Bibliography</a></li>
 </ul>
 <p aria-level="2" class="caption" role="heading"><span class="caption-text">Google Sheets (optional)</span></p>

book/_build/html/hypothesistesting.html

@@ -201,7 +201,7 @@
 <li class="toctree-l1 current active"><a class="current reference internal" href="#">Hypothesis Testing</a></li>
 <li class="toctree-l1"><a class="reference internal" href="twosample.html">Two-Sample Tests</a></li>
 <li class="toctree-l1"><a class="reference internal" href="chi2.html">The Chi-Square Test</a></li>
-<li class="toctree-l1"><a class="reference internal" href="more.html">More About Tests of Significance</a></li>
+<li class="toctree-l1"><a class="reference internal" href="more.html">More About Significance</a></li>
 <li class="toctree-l1"><a class="reference internal" href="bibliography.html">Bibliography</a></li>
 </ul>
 <p aria-level="2" class="caption" role="heading"><span class="caption-text">Google Sheets (optional)</span></p>
@@ -1064,7 +1064,7 @@ <h3>Example<a class="headerlink" href="#example" title="Permalink to this headin
 <h2>Exercises<a class="headerlink" href="#exercises" title="Permalink to this heading">#</a></h2>
 <div class="exercise admonition" id="daycare">
 
-<p class="admonition-title"><span class="caption-number">Exercise 45 </span></p>
+<p class="admonition-title"><span class="caption-number">Exercise 46 </span></p>
 <section id="exercise-content">
 <p>The owner of a daycare is tired of parents arriving late to pick up their children. Suppose it’s known that, under the status quo, <span class="math notranslate nohighlight">\(p=0.2\)</span> of parents will arrive late. The owner starts fining parents and wants to test if this makes a significant difference using a 95% confidence level and proposing hypotheses:</p>
 <div class="math notranslate nohighlight">

book/_build/html/intro.html

@@ -198,7 +198,7 @@
 <li class="toctree-l1"><a class="reference internal" href="hypothesistesting.html">Hypothesis Testing</a></li>
 <li class="toctree-l1"><a class="reference internal" href="twosample.html">Two-Sample Tests</a></li>
 <li class="toctree-l1"><a class="reference internal" href="chi2.html">The Chi-Square Test</a></li>
-<li class="toctree-l1"><a class="reference internal" href="more.html">More About Tests of Significance</a></li>
+<li class="toctree-l1"><a class="reference internal" href="more.html">More About Significance</a></li>
 <li class="toctree-l1"><a class="reference internal" href="bibliography.html">Bibliography</a></li>
 </ul>
 <p aria-level="2" class="caption" role="heading"><span class="caption-text">Google Sheets (optional)</span></p>
@@ -482,7 +482,7 @@ <h2>More about 1101<a class="headerlink" href="#more-about-1101" title="Permalin
 <li class="toctree-l1"><a class="reference internal" href="hypothesistesting.html">Hypothesis Testing</a></li>
 <li class="toctree-l1"><a class="reference internal" href="twosample.html">Two-Sample Tests</a></li>
 <li class="toctree-l1"><a class="reference internal" href="chi2.html">The Chi-Square Test</a></li>
-<li class="toctree-l1"><a class="reference internal" href="more.html">More About Tests of Significance</a></li>
+<li class="toctree-l1"><a class="reference internal" href="more.html">More About Significance</a></li>
 <li class="toctree-l1"><a class="reference internal" href="bibliography.html">Bibliography</a></li>
 </ul>
 </div>

book/_build/html/more.html

@@ -9,7 +9,7 @@
     <meta charset="utf-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.18.1: http://docutils.sourceforge.net/" />
 
-    <title>More About Tests of Significance &#8212; Intro Stats</title>
+    <title>More About Significance &#8212; Intro Stats</title>
 
 
 
@@ -77,7 +77,7 @@
     <link rel="shortcut icon" href="_static/norm_favicon.ico"/>
     <link rel="index" title="Index" href="genindex.html" />
     <link rel="search" title="Search" href="search.html" />
-    <link rel="next" title="Statistical Power" href="power.html" />
+    <link rel="next" title="Bibliography" href="bibliography.html" />
     <link rel="prev" title="The Chi-Square Test" href="chi2.html" />
   <meta name="viewport" content="width=device-width, initial-scale=1"/>
   <meta name="docsearch:language" content="en"/>
@@ -201,8 +201,7 @@
 <li class="toctree-l1"><a class="reference internal" href="hypothesistesting.html">Hypothesis Testing</a></li>
 <li class="toctree-l1"><a class="reference internal" href="twosample.html">Two-Sample Tests</a></li>
 <li class="toctree-l1"><a class="reference internal" href="chi2.html">The Chi-Square Test</a></li>
-<li class="toctree-l1 current active"><a class="current reference internal" href="#">More About Tests of Significance</a></li>
-<li class="toctree-l1"><a class="reference internal" href="power.html">Statistical Power</a></li>
+<li class="toctree-l1 current active"><a class="current reference internal" href="#">More About Significance</a></li>
 <li class="toctree-l1"><a class="reference internal" href="bibliography.html">Bibliography</a></li>
 </ul>
 <p aria-level="2" class="caption" role="heading"><span class="caption-text">Google Sheets (optional)</span></p>
@@ -483,7 +482,7 @@
 
 
 <div id="jb-print-docs-body" class="onlyprint">
-    <h1>More About Tests of Significance</h1>
+    <h1>More About Significance</h1>
     <!-- Table of contents -->
     <div id="print-main-content">
         <div id="jb-print-toc">
@@ -511,8 +510,8 @@ <h2> Contents </h2>
 <div id="searchbox"></div>
                 <article class="bd-article" role="main">
 
-  <section class="tex2jax_ignore mathjax_ignore" id="more-about-tests-of-significance">
-<span id="more"></span><h1>More About Tests of Significance<a class="headerlink" href="#more-about-tests-of-significance" title="Permalink to this heading">#</a></h1>
+  <section class="tex2jax_ignore mathjax_ignore" id="more-about-significance">
+<span id="more"></span><h1>More About Significance<a class="headerlink" href="#more-about-significance" title="Permalink to this heading">#</a></h1>
 <div class="seealso admonition">
 <p class="admonition-title">Important Readings</p>
 <ul class="simple">
@@ -615,11 +614,11 @@ <h2>Subjectivity and Argument<a class="headerlink" href="#subjectivity-and-argum
       </div>
     </a>
     <a class="right-next"
-       href="power.html"
+       href="bibliography.html"
        title="next page">
       <div class="prev-next-info">
         <p class="prev-next-subtitle">next</p>
-        <p class="prev-next-title">Statistical Power</p>
+        <p class="prev-next-title">Bibliography</p>
       </div>
       <i class="fa-solid fa-angle-right"></i>
     </a>