Merge pull request #11 from ProjectPythia/update-intro

Update intro and readme

README.md

@@ -10,6 +10,7 @@ This Project Pythia Cookbook covers empirical orthogonal function analysis and i
 
 ## Motivation
 
+Empirical orthogonal function (EOF) analysis is an essential tool for studying the variability of the atmosphere-ocean system. Meteorological and oceanographic data is noisy and multidimensional, but an EOF analysis allows us to pull out patterns from the data that might otherwise be difficult to find. The goal of this cookbook is to provide background and context to the analysis alongside practical examples of carrying out the analysis using Python packages.
 
 ## Authors
 
@@ -23,14 +24,13 @@ This Project Pythia Cookbook covers empirical orthogonal function analysis and i
 
 ## Structure
 
+This cookbook currently has one section that covers the basics of EOF analysis.
 
 ### Foundations
 
-
-
-### Applications
-
-
+This section includes two notebooks:
+- Introduction to EOFs
+- Finding Climate Modes with EOFs
 
 ## Running the Notebooks
 

notebooks/climate-modes-xeofs.ipynb

@@ -480,7 +480,7 @@
     "In this notebook, we demonstrated a basic workflow for performing an EOF analysis on gridded SST data using the `xeofs` package. We plotted the PCs associated with ENSO and PDO using deseasonalized, detrended SSTs.\n",
     "\n",
     "### What's next?\n",
-    "The next section will focus on applications of EOF analysis to answer scientific questions. (Coming soon!)"
+    "In the future, additional notebooks may use EOFs to recreate published figures, give an overview of other EOF packages, or explore variations of the EOF method."
    ]
   },
   {

notebooks/eof-intro.ipynb

@@ -7,16 +7,6 @@
     "# Introduction to EOFs"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "<div class=\"admonition alert alert-info\">\n",
-    "    <p class=\"admonition-title\" style=\"font-weight:bold\">Note</p>\n",
-    "    This content is under construction.\n",
-    "</div>"
-   ]
-  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -61,12 +51,10 @@
    ]
   },
   {
-   "cell_type": "code",
-   "execution_count": null,
+   "cell_type": "markdown",
    "metadata": {},
-   "outputs": [],
    "source": [
-    "import numpy"
+    "N/A"
    ]
   },
   {
@@ -80,6 +68,41 @@
     "Each EOF-PC pair can be thought of as a *mode of variability* in the data, but it is not necessary that these modes represent something physical. Each mode comes with a corresponding variance fraction that tells us how important the mode is. This is often phrased something like \"the first mode explains 25% of the variance in the data,\" and we will discuss what this means mathematically in the following sections. If a mode explains a large fraction of the total variance, it is more likely to be capturing some *physical* mode of variability, like the [El Niño Southern Oscillation (ENSO)](https://www.ncei.noaa.gov/access/monitoring/enso/) or [North Atlantic Oscillation (NAO)](https://www.ncei.noaa.gov/access/monitoring/nao/)."
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Examples of use\n",
+    "\n",
+    "EOF analysis (also called principal component analysis, PCA) has been applied to meteorological and oceanographic data since the 1940s and 1970s, respectively (Preisendorfer 1988). Here are a few examples of its use:\n",
+    "\n",
+    "### Identifying climate modes: the Pacific Decadal Oscillation\n",
+    "\n",
+    "The Pacific Decadal Oscillation (PDO) is a mode of climate variability characterized by decadal-scale sea surface temperature (SST) anomalies in the North Pacific. As typically defined, the positive/warm phase of PDO occurs when the SSTs off the west coast of North America are anomalously warm and the eastern/interior North Pacific is anomalously cold (and vice versa for the negative/cold phase). Since its identification and naming in the 1990s, PDO has been defined as the leading (i.e., first) EOF in North Pacific SST anomalies (Mantua and Hare 2002). This is the same definition used by [NOAA in their PDO index](https://www.ncei.noaa.gov/access/monitoring/pdo/).\n",
+    "\n",
+    "| ![PDO](images/mantua_hare_1.PNG) |\n",
+    "|:--:|\n",
+    "| Adapted from Mantua and Hare (2002). The three plots on the top show the EOFs for SST, sea level pressure, and surface wind stress, while the time series at the bottom is the associated PC. |\n",
+    "\n",
+    "Besides the modes of variability already mentioned, an EOF analysis can also be used to identify the Northern and Southern Annular Modes (NAM and SAM), the Atlantic Multidecadal Oscillation/Variability (AMO/V), the zonal wave 3 pattern (ZW3) of both hemispheres, and others.\n",
+    "\n",
+    "### Identifying patterns across an ensemble\n",
+    "\n",
+    "While an EOF analysis is usually applied to data that is a function of space and time, it can also be used with data that is a function of space and *ensemble member*. Then the analysis may be able to identify the most important patterns that characterize the differences between ensemble members. Tokinaga et al. (2012) perform an EOF analysis on an ensemble of 100 realizations of observed SSTs in order to understand the uncertainty in the zonal SST gradient in the tropical Pacific.\n",
+    "\n",
+    "| ![Pacific EOF](images/tokinaga_eof.PNG)![Pacific PC vs. SST gradient](images/tokinaga_pcs.PNG) |\n",
+    "|:--:|\n",
+    "| Adapted from Tokinaga et al. (2012). The left plot shows the leading EOF (with 83.6% variance fraction) of tropical Pacific SST across 100 realizations of observed SSTs. The right plot shows the PC (in this case, a single value instead of a time series) associated with the EOF plotted against the zonal SST gradient in the tropical Pacific for each ensemble member. |\n",
+    "\n",
+    "### Isolating the global warming signal\n",
+    "\n",
+    "To isolate the global warming signal from internal variability, it is common to run a climate model multiple times&mdash;with the same anthropogenic emissions and slightly different initial conditions&mdash;and take the average response across the ensemble. With enough ensemble members, the internal variability should mostly cancel out, identifying the ensemble average with the model's response to the emissions. Unfortunately, we cannot use this method on Earth's actual climate, which only has one realization. As you will see in [Finding Climate Modes with EOFs](climate-modes-xeofs), the long-term warming trend will be apparent in an EOF analysis of observed SSTs if the data is not detrended, but it is not cleanly separated from internal variability. Wills et al. (2018) develop a method, which they call low-frequency component analysis (LFCA), that can more cleanly separate out the warming trend. After perfoming an EOF analysis, their method finds a linear combination of the EOFs that maximizes the variance fraction at low frequencies (see their paper for more details).\n",
+    "\n",
+    "| ![LFCA](images/wills_1.PNG) |\n",
+    "|:--:|\n",
+    "| Adapted from Wills et al. (2018). Three leading low frequency patterns (LFPs, similar to EOFs) and their corresponding low frequency components (LFCs, similar to PCs) for Pacific SST anomalies. This method is able to separate the global warming trend from PDO and ENSO. |"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -101,11 +124,17 @@
    "source": [
     "## Deriving the eigenvalue problem\n",
     "\n",
-    "It is not obvious why solving the eigenvalue problem for the covariance matrix results in spatial patterns that align well with the data. The following explanation follows those given by Preisendorfer (1988), Peixoto et al. (1992), and Björnsson and Venegas (1997).\n",
+    "As an aside, it is not obvious why solving the eigenvalue problem for the covariance matrix results in spatial patterns that align well with the data. The following explanation follows those given by Preisendorfer (1988), Peixoto et al. (1992), and Björnsson and Venegas (1997).\n",
     "\n",
     "Let $f(x, t)$ be a geophysical field recorded over locations $x_1, x_2, \\dots, x_M$ and times $t_1, t_2, \\dots, t_N$. In matrix notation, we can write each map of $M$ locations as the column vector $\\mathbf{f}_n=\\begin{bmatrix}f_{1n} & f_{2n} & \\cdots & f_{Mn}\\end{bmatrix}^\\mathrm T$, where $n=1, \\dots, N$. We then have an $M\\times N$ matrix that can be written as $\\mathbf F=\\begin{bmatrix}\\mathbf{f}_1 & \\mathbf{f}_2 & \\cdots & \\mathbf{f}_N\\end{bmatrix}$. \n",
     "\n",
-    "Now, imagine that we only have three locations to collect data. Then each $\\mathbf{f}_n$ would be a 3-dimensional vector, and the whole set could be represented as a linear combination of three arbitrary basis vectors. This generalizes such that the vector space $\\mathbf F$ is spanned by an arbirary unit basis $\\{\\mathbf u_1,\\mathbf u_2,\\dots,\\mathbf u_M\\}$. It is very likely that some of the vectors $\\mathbf f_n$ are correlated as a result of some physical process. The goal of the EOF analysis is then to find an orthogonal unit basis $\\{\\mathbf e_1,\\mathbf e_2,\\dots,\\mathbf e_M\\}$ that aligns well with these vector \"clusters\". This is now an optimization problem. To maximize the alignment between the basis and observations, we will maximize the projection of the vectors $\\mathbf f_n$ onto each basis vector using the sum of squares:\n",
+    "Now, imagine that we only have three locations to collect data. Then each $\\mathbf{f}_n$ would be a 3-dimensional vector, and the whole set could be represented as a linear combination of three arbitrary basis vectors. This generalizes such that the vector space $\\mathbf F$ is spanned by an arbirary unit basis $\\{\\mathbf u_1,\\mathbf u_2,\\dots,\\mathbf u_M\\}$. It is very likely that some of the vectors $\\mathbf f_n$ are correlated as a result of some physical process. The goal of the EOF analysis is then to find an orthogonal unit basis $\\{\\mathbf e_1,\\mathbf e_2,\\dots,\\mathbf e_M\\}$ that aligns well with these vector \"clusters\". \n",
+    "\n",
+    "| ![EOF clustering](images/peixoto_b1.PNG) |\n",
+    "|:--:|\n",
+    "| Adapted from Peixoto et al. (1992). |\n",
+    "\n",
+    "This is now an optimization problem. To maximize the alignment between the basis and observations, we will maximize the projection of the vectors $\\mathbf f_n$ onto each basis vector using the sum of squares:\n",
     "\\begin{equation}\n",
     "\\sum_{n=1}^N(\\mathbf f_n \\cdot \\mathbf e_m)^2\n",
     "\\end{equation}\n",
@@ -119,13 +148,8 @@
     "&=\\mathbf e_m^\\mathrm T \\mathbf R\\mathbf e_m\n",
     "\\end{aligned}\n",
     "\\end{equation}\n",
-    "where $\\mathbf R=\\mathbf F\\mathbf F^\\mathrm T$ is the covariance matrix of $\\mathbf F$."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
+    "where $\\mathbf R=\\mathbf F\\mathbf F^\\mathrm T$ is the covariance matrix of $\\mathbf F$.\n",
+    "\n",
     "We now seek the extrema of $\\psi(\\mathbf e)$ (dropping the subscript $m$ for convenience), which satisfy $\\psi(\\mathbf e+\\delta\\mathbf e)=\\psi(\\mathbf e)$ for a small change $\\delta\\mathbf e$ in the direction of $\\mathbf e$. Then\n",
     "\\begin{equation}\n",
     "\\begin{aligned}\n",
@@ -161,7 +185,7 @@
     "In this notebook, we covered some background on EOFs, the steps required to carry out an EOF analysis, and some of the math behind the analysis.\n",
     "\n",
     "### What's next?\n",
-    "An example of carrying out an EOF analysis on idealized data using NumPy. (Coming soon!)"
+    "In the next notebook, we will use a package called `xeofs` to find climate modes in SST data."
    ]
   },
   {
@@ -172,9 +196,15 @@
     "\n",
     "Björnsson, H., & Venegas, S. A. (1997). A manual for EOF and SVD analyses of climatic data. *CCGCR Report*, *97*(1), 112-134.\n",
     "\n",
+    "Mantua, N. J., & Hare, S. R. (2002). The Pacific Decadal Oscillation. *Journal of Oceanography*, *58*, 35-44.\n",
+    "\n",
     "Peixoto, J. P., Oort, A. H., & Lorenz, E. N. (1992). *Physics of climate* (Vol. 520). New York: American Institute of Physics.\n",
     "\n",
-    "Preisendorfer, R. Ä. N. Ü. (1988). Principal component analysis in meteorology and oceanography. *Elsevier Sci. Publ.*, *17*, 425."
+    "Preisendorfer, R. Ä. N. Ü. (1988). Principal component analysis in meteorology and oceanography. *Elsevier Sci. Publ.*, *17*, 425.\n",
+    "\n",
+    "Tokinaga, H., Xie, S. P., Deser, C., Kosaka, Y., & Okumura, Y. M. (2012). Slowdown of the Walker circulation driven by tropical Indo-Pacific warming. *Nature*, *491*(7424), 439-443.\n",
+    "\n",
+    "Wills, R. C., Schneider, T., Wallace, J. M., Battisti, D. S., & Hartmann, D. L. (2018). Disentangling global warming, multidecadal variability, and El Niño in Pacific temperatures. *Geophysical Research Letters*, *45*(5), 2487-2496."
    ]
   }
  ],