notes_06_numpy_and_linear_algebra.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Linear algebra\n",
    "\n",
    "* Writing and reading arrays to/from files.\n",
    "* Linear algebra and matrix methods.\n",
    "* Multidimensional root finding and minimization.\n",
    "* Curve fitting."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matrices\n",
    "\n",
    "#### Creating Arrays in Numpy\n",
    "* Creating arrays:\n",
    "  * Online documentation:  \n",
    "    http://docs.scipy.org/doc/numpy/reference/routines.array-creation.html\n",
    "* Writing to a file:\n",
    "  * Online documentation:  \n",
    "    http://docs.scipy.org/doc/numpy/reference/generated/numpy.savetxt.html\n",
    "  * Python help:\n",
    "  `help(numpy.savetxt)`\n",
    "* Loading from a file:\n",
    "  * Online documentation:  \n",
    "  http://docs.scipy.org/doc/numpy/reference/generated/numpy.loadtxt.html\n",
    "  * Python help:  \n",
    "  `help(numpy.loadtxt)`\n",
    "* There are other fileIO functions:\n",
    "  e.g. `numpy.load` and `numpy.save`.\n",
    "* and other formats (e.g., `hdf`) and frameworks (e.g., `astropy.Tables`)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Examples: Arrays and Files\n",
    "* Setup:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "A = np.array([[0.0, 2.5, 3.2],\n",
    "              [1.0, 5.6, 8.9],\n",
    "              [2.0, 7.1, 3.7],\n",
    "              [3.0, 4.2, 9.3]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Save $\\mathbf{A}$ to a file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    " np.savetxt('A_test.csv', A,\n",
    "            delimiter=',', fmt='%g')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Load the array back from the file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "B = np.loadtxt('_test.csv',\n",
    "               delimiter=',')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0.   2.5  3.2]\n",
      " [ 1.   5.6  8.9]\n",
      " [ 2.   7.1  3.7]\n",
      " [ 3.   4.2  9.3]]\n"
     ]
    }
   ],
   "source": [
    "print(B)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Let's take a moment to look at the file. [...] "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "foo = {\"PHYS\" : \"Physics\", \"ASTRON\" : \"Astronomy\"}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "ename": "KeyError",
     "evalue": "'Physics'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-3-59f5b77800a9>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfoo\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'Physics'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m: 'Physics'"
     ]
    }
   ],
   "source": [
    "print(foo['Physics'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PHYS Physics\n",
      "ASTRON Astronomy\n"
     ]
    }
   ],
   "source": [
    "for key, value in foo.items():\n",
    "    print(key, value)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Linear Algebra\n",
    "\n",
    "#### Linear Algebra Problems\n",
    "* The basic linear algebra problem is a set of $m$ simultaneous linear equations with $n$ unknowns:\n",
    "\\begin{equation*}\n",
    "\\begin{matrix}\n",
    " \\alpha_{0,0}x_0   &+& \\alpha_{0,1}x_1   &+& \\cdots &+& \\alpha_{0,n-1}x_{n-1}  &=& b_0 \\\\\n",
    " \\alpha_{1,0}x_0   &+& \\alpha_{1,1}x_1   &+& \\cdots &+& \\alpha_{1,n-1}x_{n-1}  &=& b_1 \\\\\n",
    " \\vdots            &+& \\vdots            &+& \\ddots &+& \\vdots                 &=& \\vdots \\\\\n",
    " \\alpha_{m-1,0}x_0 &+& \\alpha_{m-1,1}x_1 &+& \\cdots &+& \\alpha_{m-1,n-1}x_{n-1}&=& b_{m-1} \\\\\n",
    "\\end{matrix}\n",
    "\\end{equation*}\n",
    "* Typically, we know $\\alpha_{i,j}$ and $b_i$, and want to find $x_j$.\n",
    "* If $m>n$, the system is overdetermined.\n",
    "* If $m<n$, the system is underdetermined.\n",
    "* We will look primarily at the case $n=m$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Matrix Representation\n",
    "* Computers are better able to handle equations as matrix equations.\n",
    "* Matrix representation:\n",
    "\n",
    "\\begin{equation*}\n",
    "\\begin{pmatrix}\n",
    "\\alpha_{0,0}   & \\alpha_{0,1}   & \\cdots & \\alpha_{0,n-1}   \\\\\n",
    "\\alpha_{1,0}   & \\alpha_{1,1}   & \\cdots & \\alpha_{1,n-1}   \\\\\n",
    "\\vdots         & \\vdots         & \\ddots & \\vdots           \\\\\n",
    "\\alpha_{m-1,0} & \\alpha_{m-1,1} & \\cdots & \\alpha_{m-1,n-1} \\\\\n",
    "\\end{pmatrix}\n",
    "\\begin{pmatrix}\n",
    "x_0 \\\\\n",
    "x_1 \\\\\n",
    "\\vdots \\\\\n",
    "x_{n-1} \\\\\n",
    "\\end{pmatrix}\n",
    "=\n",
    "\\begin{pmatrix}\n",
    "b_0 \\\\\n",
    "b_1 \\\\\n",
    "\\vdots \\\\\n",
    "b_{m-1} \\\\\n",
    "\\end{pmatrix}\n",
    "\\end{equation*}\n",
    "\n",
    "* We can write this set of equations in a compact form, writing the matrix as $\\mathbf{A}$:\n",
    "\n",
    "\\begin{equation*}\n",
    "\\mathbf{A}\\vec{x} = \\vec{b}\n",
    "\\end{equation*}\n",
    "\n",
    "* Here $\\mathbf{A}$ is a square matrix."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Classes of Matrix Problems\n",
    "$\\mathbf{A}\\vec{x}=\\vec{b}$\n",
    "* $\\mathbf{A}$ is a known $N \\times N$ matrix.\n",
    "* $\\vec{x}$ is an unknown vector of length $N$.\n",
    "* $\\vec{b}$ is a known vector of length $N$.\n",
    "* Solve with Gaussian elimination\n",
    " or lower-upper (LU) decomposition.\n",
    "* Slower: solve by finding $\\mathbf{A}^{-1}$, then $\\vec{x}=\\mathbf{A}^{-1}\\vec{b}$.\n",
    "$\\mathbf{A}\\vec{x}=\\lambda\\vec{x}$\n",
    "* Eigenvector $\\vec{x}$ is an unknown vector of length $N$.\n",
    "* Eigenvalue $\\lambda$ is an unknown parameter.\n",
    "* $\\mathbf{A}^{-1}$ doesn't help! Need specialized solver.\n",
    "* Can shown that $\\textrm{det}[\\mathbf{A}-\\lambda\\mathbf{I}] = 0$ for eigenvalues $\\lambda$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Linear Algebra Routines\n",
    "* We *will* solve linear algebra problems with \"canned\" routines.\n",
    "* Eigensystems, matrix multiplication, inverses, determinants, etc.\n",
    "* Many tested and optimized packages available:\n",
    "NETLIB, LAPACK, SLATEC, BLAS, \\ldots\n",
    "* Writing custom solvers \"from scratch\" is not usually worthwhile for these.\n",
    "* NumPy and SciPy wrap some of these.\n",
    "* We will primarily use the NumPy routines in `numpy.linalg`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Linear Algebra in Numpy\n",
    "Common linear algebra functions available in `numpy`:\n",
    "* Online documentation:\n",
    "  http://docs.scipy.org/doc/numpy/reference/routines.linalg.html\n",
    "* Python help:\n",
    "  `help(numpy.linalg)`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Examples: Solving Linear Systems\n",
    "* Setup matrices:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "A = np.array([[ 1,  2,   3],\n",
    "              [22, 32,  42],\n",
    "              [55, 66, 100]])\n",
    "b = np.array([1, 2, 3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Solve $\\mathbf{A}\\vec{x}=\\vec{b}$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-1.4057971  -0.1884058   0.92753623]\n"
     ]
    }
   ],
   "source": [
    "x = np.linalg.solve(A, b)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Check accuracy of the solution by calculating $\\mathbf{A}\\vec{x}-\\vec{b}$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  4.44089210e-16,   1.77635684e-15,  -3.55271368e-15])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.dot(A, x) - b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Less efficient direct solution, $\\vec{x}=\\mathbf{A}^{-1}\\vec{b}$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[  1.00000000e+00  -1.33226763e-15  -1.77635684e-15]\n",
      " [  8.88178420e-16   1.00000000e+00   0.00000000e+00]\n",
      " [ -4.44089210e-16   4.44089210e-16   1.00000000e+00]]\n"
     ]
    }
   ],
   "source": [
    "A_inverse = np.linalg.inv(A)\n",
    "\n",
    "print(np.dot(A_inverse, A))\n",
    "x = np.dot(A_inverse, b)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-1.4057971  -0.1884058   0.92753623]\n"
     ]
    }
   ],
   "source": [
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Examples: Solving Eigensystems\n",
    "Solve for the principle axes of a cube,\n",
    "where the moment of inertia tensor $\\mathbf{I}$ is diagonal.\n",
    "* Solve $\\mathbf{I}\\vec{\\omega}= \\lambda\\vec{\\omega}$ for eigenvectors $\\vec{\\omega}$ and eigenvalues $\\lambda$.\n",
    "* Setup:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "I = np.array([[ 2./3,-1./4,-1./4],\n",
    "              [-1./4, 2./3,-1./4],\n",
    "              [-1./4,-1./4, 2./3]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Solve eigensystem:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "evalues, evectors = np.linalg.eig(I)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Eigenvectors: \n",
      "[[ 0.81649658 -0.57735027  0.43514263]\n",
      " [-0.40824829 -0.57735027 -0.81589244]\n",
      " [-0.40824829 -0.57735027  0.38074981]]\n",
      "Eigenvalues: \n",
      "[ 0.91666667  0.16666667  0.91666667]\n"
     ]
    }
   ],
   "source": [
    "print(\"Eigenvectors: \")\n",
    "print(evectors)\n",
    "\n",
    "print(\"Eigenvalues: \")\n",
    "print(evalues)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Evaluate difference between RHS and LHS of $\\mathbf{I}\\vec{\\omega}= \\lambda\\vec{\\omega}$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  1.11022302e-16,  -1.11022302e-16,  -1.66533454e-16])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "LHS = np.dot(I, evectors[:, 0])\n",
    "RHS = evalues[0] * evectors[:, 0]\n",
    "LHS - RHS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Excellent, the LHS and RHS look equal to something approaching machine precision."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Multidimensional Nonlinear Root Finding\n",
    "The general problem of finding roots of multiple nonlinear simultaneous equations $f(x, y)$ and $g(x, y)$ is difficult:\n",
    "\n",
    "![](figures/multidimensional_root_finding.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Multidimensional Newton-Raphson\n",
    "Consider $N$ nonlinear functions of $N$ variables, $F_i(x_1, x_2, \\ldots , x_N)$.  \n",
    "Find the $\\vec{x}$ which gives $F_i(\\vec{x}) = 0$\n",
    " for all $i$.\n",
    "* Taylor expansion:\n",
    "\\begin{equation*}\n",
    "\\vec{F}(\\vec{x}+ \\delta\\vec{x}) = \\vec{F}(\\vec{x}) + \\mathbf{J}\\cdot \\delta\\vec{x} + \\mathcal{O}(\\delta\\vec{x}^2)\n",
    "\\end{equation*}\n",
    "where $\\mathbf{J}$ is the Jacobian matrix with $J_{ij} = \\frac{\\partial F_i}{\\partial x_j}$.\n",
    "* Set $\\vec{F}(\\vec{x}+ \\delta\\vec{x})=0$ to get an approximate solution:\n",
    "\\begin{equation*}\n",
    "\\mathbf{J}\\cdot \\delta\\vec{x} = -\\vec{F}(\\vec{x})\n",
    "\\end{equation*}\n",
    "* Solve for $\\delta\\vec{x}$ to get the Newton-Raphson step:\n",
    "\\begin{equation*}\n",
    "\\vec{x}_{\\rm new} = \\vec{x}_{\\rm old} + \\delta\\vec{x}\n",
    "\\end{equation*}\n",
    "* Need to use matrix mathods (e.g. LU decomposition) to get $\\delta\\vec{x}$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Function Minimization\n",
    "What if the problem is to minimize a function instead of finding a zero?\n",
    "* There are routines similar to the bisection and Newton-Raphson methods for minimization.\n",
    "* Multidimensional minimization (optimization) is better behaved than multidimensional root finding.\n",
    "* Many Scipy optimization routines:\n",
    "  * Online documentation:\n",
    "  http://docs.scipy.org/doc/scipy/reference/optimize.html\n",
    "  * Python help:\n",
    "  `help(scipy.optimize)`\n",
    "  * Many trade-offs between speed, memory use, robustness, \\ldots\n",
    "  * Ideal technique is problem-dependent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on package scipy.optimize in scipy:\n",
      "\n",
      "NAME\n",
      "    scipy.optimize\n",
      "\n",
      "DESCRIPTION\n",
      "    =====================================================\n",
      "    Optimization and root finding (:mod:`scipy.optimize`)\n",
      "    =====================================================\n",
      "    \n",
      "    .. currentmodule:: scipy.optimize\n",
      "    \n",
      "    Optimization\n",
      "    ============\n",
      "    \n",
      "    Local Optimization\n",
      "    ------------------\n",
      "    \n",
      "    .. autosummary::\n",
      "       :toctree: generated/\n",
      "    \n",
      "       minimize - Unified interface for minimizers of multivariate functions\n",
      "       minimize_scalar - Unified interface for minimizers of univariate functions\n",
      "       OptimizeResult - The optimization result returned by some optimizers\n",
      "    \n",
      "    The `minimize` function supports the following methods:\n",
      "    \n",
      "    .. toctree::\n",
      "    \n",
      "       optimize.minimize-neldermead\n",
      "       optimize.minimize-powell\n",
      "       optimize.minimize-cg\n",
      "       optimize.minimize-bfgs\n",
      "       optimize.minimize-newtoncg\n",
      "       optimize.minimize-lbfgsb\n",
      "       optimize.minimize-tnc\n",
      "       optimize.minimize-cobyla\n",
      "       optimize.minimize-slsqp\n",
      "       optimize.minimize-dogleg\n",
      "       optimize.minimize-trustncg\n",
      "    \n",
      "    The `minimize_scalar` function supports the following methods:\n",
      "    \n",
      "    .. toctree::\n",
      "    \n",
      "       optimize.minimize_scalar-brent\n",
      "       optimize.minimize_scalar-bounded\n",
      "       optimize.minimize_scalar-golden\n",
      "    \n",
      "    The specific optimization method interfaces below in this subsection are\n",
      "    not recommended for use in new scripts; all of these methods are accessible\n",
      "    via a newer, more consistent interface provided by the functions above.\n",
      "    \n",
      "    General-purpose multivariate methods:\n",
      "    \n",
      "    .. autosummary::\n",
      "       :toctree: generated/\n",
      "    \n",
      "       fmin - Nelder-Mead Simplex algorithm\n",
      "       fmin_powell - Powell's (modified) level set method\n",
      "       fmin_cg - Non-linear (Polak-Ribiere) conjugate gradient algorithm\n",
      "       fmin_bfgs - Quasi-Newton method (Broydon-Fletcher-Goldfarb-Shanno)\n",
      "       fmin_ncg - Line-search Newton Conjugate Gradient\n",
      "    \n",
      "    Constrained multivariate methods:\n",
      "    \n",
      "    .. autosummary::\n",
      "       :toctree: generated/\n",
      "    \n",
      "       fmin_l_bfgs_b - Zhu, Byrd, and Nocedal's constrained optimizer\n",
      "       fmin_tnc - Truncated Newton code\n",
      "       fmin_cobyla - Constrained optimization by linear approximation\n",
      "       fmin_slsqp - Minimization using sequential least-squares programming\n",
      "       differential_evolution - stochastic minimization using differential evolution\n",
      "    \n",
      "    Univariate (scalar) minimization methods:\n",
      "    \n",
      "    .. autosummary::\n",
      "       :toctree: generated/\n",
      "    \n",
      "       fminbound - Bounded minimization of a scalar function\n",
      "       brent - 1-D function minimization using Brent method\n",
      "       golden - 1-D function minimization using Golden Section method\n",
      "    \n",
      "    Equation (Local) Minimizers\n",
      "    ---------------------------\n",
      "    \n",
      "    .. autosummary::\n",
      "       :toctree: generated/\n",
      "    \n",
      "       leastsq - Minimize the sum of squares of M equations in N unknowns\n",
      "       nnls - Linear least-squares problem with non-negativity constraint\n",
      "    \n",
      "    Global Optimization\n",
      "    -------------------\n",
      "    \n",
      "    .. autosummary::\n",
      "       :toctree: generated/\n",
      "    \n",
      "       basinhopping - Basinhopping stochastic optimizer\n",
      "       brute - Brute force searching optimizer\n",
      "       differential_evolution - stochastic minimization using differential evolution\n",
      "    \n",
      "    Rosenbrock function\n",
      "    -------------------\n",
      "    \n",
      "    .. autosummary::\n",
      "       :toctree: generated/\n",
      "    \n",
      "       rosen - The Rosenbrock function.\n",
      "       rosen_der - The derivative of the Rosenbrock function.\n",
      "       rosen_hess - The Hessian matrix of the Rosenbrock function.\n",
      "       rosen_hess_prod - Product of the Rosenbrock Hessian with a vector.\n",
      "    \n",
      "    Fitting\n",
      "    =======\n",
      "    \n",
      "    .. autosummary::\n",
      "       :toctree: generated/\n",
      "    \n",
      "       curve_fit -- Fit curve to a set of points\n",
      "    \n",
      "    Root finding\n",
      "    ============\n",
      "    \n",
      "    Scalar functions\n",
      "    ----------------\n",
      "    .. autosummary::\n",
      "       :toctree: generated/\n",
      "    \n",
      "       brentq - quadratic interpolation Brent method\n",
      "       brenth - Brent method, modified by Harris with hyperbolic extrapolation\n",
      "       ridder - Ridder's method\n",
      "       bisect - Bisection method\n",
      "       newton - Secant method or Newton's method\n",
      "    \n",
      "    Fixed point finding:\n",
      "    \n",
      "    .. autosummary::\n",
      "       :toctree: generated/\n",
      "    \n",
      "       fixed_point - Single-variable fixed-point solver\n",
      "    \n",
      "    Multidimensional\n",
      "    ----------------\n",
      "    \n",
      "    General nonlinear solvers:\n",
      "    \n",
      "    .. autosummary::\n",
      "       :toctree: generated/\n",
      "    \n",
      "       root - Unified interface for nonlinear solvers of multivariate functions\n",
      "       fsolve - Non-linear multi-variable equation solver\n",
      "       broyden1 - Broyden's first method\n",
      "       broyden2 - Broyden's second method\n",
      "    \n",
      "    The `root` function supports the following methods:\n",
      "    \n",
      "    .. toctree::\n",
      "    \n",
      "       optimize.root-hybr\n",
      "       optimize.root-lm\n",
      "       optimize.root-broyden1\n",
      "       optimize.root-broyden2\n",
      "       optimize.root-anderson\n",
      "       optimize.root-linearmixing\n",
      "       optimize.root-diagbroyden\n",
      "       optimize.root-excitingmixing\n",
      "       optimize.root-krylov\n",
      "       optimize.root-dfsane\n",
      "    \n",
      "    Large-scale nonlinear solvers:\n",
      "    \n",
      "    .. autosummary::\n",
      "       :toctree: generated/\n",
      "    \n",
      "       newton_krylov\n",
      "       anderson\n",
      "    \n",
      "    Simple iterations:\n",
      "    \n",
      "    .. autosummary::\n",
      "       :toctree: generated/\n",
      "    \n",
      "       excitingmixing\n",
      "       linearmixing\n",
      "       diagbroyden\n",
      "    \n",
      "    :mod:`Additional information on the nonlinear solvers <scipy.optimize.nonlin>`\n",
      "    \n",
      "    Linear Programming\n",
      "    ==================\n",
      "    \n",
      "    Simplex Algorithm:\n",
      "    \n",
      "    .. autosummary::\n",
      "       :toctree: generated/\n",
      "    \n",
      "       linprog -- Linear programming using the simplex algorithm\n",
      "    \n",
      "    The `linprog` function supports the following methods:\n",
      "    \n",
      "    .. toctree::\n",
      "    \n",
      "       optimize.linprog-simplex\n",
      "    \n",
      "    \n",
      "    Utilities\n",
      "    =========\n",
      "    \n",
      "    .. autosummary::\n",
      "       :toctree: generated/\n",
      "    \n",
      "       approx_fprime - Approximate the gradient of a scalar function\n",
      "       bracket - Bracket a minimum, given two starting points\n",
      "       check_grad - Check the supplied derivative using finite differences\n",
      "       line_search - Return a step that satisfies the strong Wolfe conditions\n",
      "    \n",
      "       show_options - Show specific options optimization solvers\n",
      "       LbfgsInvHessProduct - Linear operator for L-BFGS approximate inverse Hessian\n",
      "\n",
      "PACKAGE CONTENTS\n",
      "    _basinhopping\n",
      "    _cobyla\n",
      "    _differentialevolution\n",
      "    _lbfgsb\n",
      "    _linprog\n",
      "    _minimize\n",
      "    _minpack\n",
      "    _nnls\n",
      "    _root\n",
      "    _slsqp\n",
      "    _spectral\n",
      "    _trustregion\n",
      "    _trustregion_dogleg\n",
      "    _trustregion_ncg\n",
      "    _tstutils\n",
      "    _zeros\n",
      "    cobyla\n",
      "    lbfgsb\n",
      "    linesearch\n",
      "    minpack\n",
      "    minpack2\n",
      "    moduleTNC\n",
      "    nnls\n",
      "    nonlin\n",
      "    optimize\n",
      "    setup\n",
      "    slsqp\n",
      "    tnc\n",
      "    zeros\n",
      "\n",
      "CLASSES\n",
      "    builtins.UserWarning(builtins.Warning)\n",
      "        scipy.optimize.optimize.OptimizeWarning\n",
      "    builtins.dict(builtins.object)\n",
      "        scipy.optimize.optimize.OptimizeResult\n",
      "    scipy.sparse.linalg.interface.LinearOperator(builtins.object)\n",
      "        scipy.optimize.lbfgsb.LbfgsInvHessProduct\n",
      "    \n",
      "    class LbfgsInvHessProduct(scipy.sparse.linalg.interface.LinearOperator)\n",
      "     |  Linear operator for the L-BFGS approximate inverse Hessian.\n",
      "     |  \n",
      "     |  This operator computes the product of a vector with the approximate inverse\n",
      "     |  of the Hessian of the objective function, using the L-BFGS limited\n",
      "     |  memory approximation to the inverse Hessian, accumulated during the\n",
      "     |  optimization.\n",
      "     |  \n",
      "     |  Parameters\n",
      "     |  ----------\n",
      "     |  sk : array_like, shape=(n_corr, n)\n",
      "     |      Array of `n_corr` most recent updates to the solution vector.\n",
      "     |      (See [1]).\n",
      "     |  yk : array_like, shape=(n_corr, n)\n",
      "     |      Array of `n_corr` most recent updates to the gradient. (See [1]).\n",
      "     |  \n",
      "     |  References\n",
      "     |  ----------\n",
      "     |  .. [1] Nocedal, Jorge. \"Updating quasi-Newton matrices with limited\n",
      "     |     storage.\" Mathematics of computation 35.151 (1980): 773-782.\n",
      "     |  \n",
      "     |  Method resolution order:\n",
      "     |      LbfgsInvHessProduct\n",
      "     |      scipy.sparse.linalg.interface.LinearOperator\n",
      "     |      builtins.object\n",
      "     |  \n",
      "     |  Methods defined here:\n",
      "     |  \n",
      "     |  __init__(self, sk, yk)\n",
      "     |      Construct the operator.\n",
      "     |  \n",
      "     |  todense(self)\n",
      "     |      Return a dense array representation of this operator.\n",
      "     |      \n",
      "     |      Returns\n",
      "     |      -------\n",
      "     |      arr : ndarray, shape=(n, n)\n",
      "     |          An array with the same shape and containing\n",
      "     |          the same data represented by this `LinearOperator`.\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Methods inherited from scipy.sparse.linalg.interface.LinearOperator:\n",
      "     |  \n",
      "     |  __add__(self, x)\n",
      "     |  \n",
      "     |  __call__(self, x)\n",
      "     |  \n",
      "     |  __mul__(self, x)\n",
      "     |  \n",
      "     |  __neg__(self)\n",
      "     |  \n",
      "     |  __pow__(self, p)\n",
      "     |  \n",
      "     |  __repr__(self)\n",
      "     |  \n",
      "     |  __rmul__(self, x)\n",
      "     |  \n",
      "     |  __sub__(self, x)\n",
      "     |  \n",
      "     |  adjoint(self)\n",
      "     |      Hermitian adjoint.\n",
      "     |      \n",
      "     |      Returns the Hermitian adjoint of self, aka the Hermitian\n",
      "     |      conjugate or Hermitian transpose. For a complex matrix, the\n",
      "     |      Hermitian adjoint is equal to the conjugate transpose.\n",
      "     |      \n",
      "     |      Can be abbreviated self.H instead of self.adjoint().\n",
      "     |      \n",
      "     |      Returns\n",
      "     |      -------\n",
      "     |      A_H : LinearOperator\n",
      "     |          Hermitian adjoint of self.\n",
      "     |  \n",
      "     |  dot(self, x)\n",
      "     |      Matrix-matrix or matrix-vector multiplication.\n",
      "     |      \n",
      "     |      Parameters\n",
      "     |      ----------\n",
      "     |      x : array_like\n",
      "     |          1-d or 2-d array, representing a vector or matrix.\n",
      "     |      \n",
      "     |      Returns\n",
      "     |      -------\n",
      "     |      Ax : array\n",
      "     |          1-d or 2-d array (depending on the shape of x) that represents\n",
      "     |          the result of applying this linear operator on x.\n",
      "     |  \n",
      "     |  matmat(self, X)\n",
      "     |      Matrix-matrix multiplication.\n",
      "     |      \n",
      "     |      Performs the operation y=A*X where A is an MxN linear\n",
      "     |      operator and X dense N*K matrix or ndarray.\n",
      "     |      \n",
      "     |      Parameters\n",
      "     |      ----------\n",
      "     |      X : {matrix, ndarray}\n",
      "     |          An array with shape (N,K).\n",
      "     |      \n",
      "     |      Returns\n",
      "     |      -------\n",
      "     |      Y : {matrix, ndarray}\n",
      "     |          A matrix or ndarray with shape (M,K) depending on\n",
      "     |          the type of the X argument.\n",
      "     |      \n",
      "     |      Notes\n",
      "     |      -----\n",
      "     |      This matmat wraps any user-specified matmat routine or overridden\n",
      "     |      _matmat method to ensure that y has the correct type.\n",
      "     |  \n",
      "     |  matvec(self, x)\n",
      "     |      Matrix-vector multiplication.\n",
      "     |      \n",
      "     |      Performs the operation y=A*x where A is an MxN linear\n",
      "     |      operator and x is a column vector or 1-d array.\n",
      "     |      \n",
      "     |      Parameters\n",
      "     |      ----------\n",
      "     |      x : {matrix, ndarray}\n",
      "     |          An array with shape (N,) or (N,1).\n",
      "     |      \n",
      "     |      Returns\n",
      "     |      -------\n",
      "     |      y : {matrix, ndarray}\n",
      "     |          A matrix or ndarray with shape (M,) or (M,1) depending\n",
      "     |          on the type and shape of the x argument.\n",
      "     |      \n",
      "     |      Notes\n",
      "     |      -----\n",
      "     |      This matvec wraps the user-specified matvec routine or overridden\n",
      "     |      _matvec method to ensure that y has the correct shape and type.\n",
      "     |  \n",
      "     |  rmatvec(self, x)\n",
      "     |      Adjoint matrix-vector multiplication.\n",
      "     |      \n",
      "     |      Performs the operation y = A^H * x where A is an MxN linear\n",
      "     |      operator and x is a column vector or 1-d array.\n",
      "     |      \n",
      "     |      Parameters\n",
      "     |      ----------\n",
      "     |      x : {matrix, ndarray}\n",
      "     |          An array with shape (M,) or (M,1).\n",
      "     |      \n",
      "     |      Returns\n",
      "     |      -------\n",
      "     |      y : {matrix, ndarray}\n",
      "     |          A matrix or ndarray with shape (N,) or (N,1) depending\n",
      "     |          on the type and shape of the x argument.\n",
      "     |      \n",
      "     |      Notes\n",
      "     |      -----\n",
      "     |      This rmatvec wraps the user-specified rmatvec routine or overridden\n",
      "     |      _rmatvec method to ensure that y has the correct shape and type.\n",
      "     |  \n",
      "     |  transpose(self)\n",
      "     |      Transpose this linear operator.\n",
      "     |      \n",
      "     |      Returns a LinearOperator that represents the transpose of this one.\n",
      "     |      Can be abbreviated self.T instead of self.transpose().\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Static methods inherited from scipy.sparse.linalg.interface.LinearOperator:\n",
      "     |  \n",
      "     |  __new__(cls, *args, **kwargs)\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Data descriptors inherited from scipy.sparse.linalg.interface.LinearOperator:\n",
      "     |  \n",
      "     |  H\n",
      "     |      Hermitian adjoint.\n",
      "     |      \n",
      "     |      Returns the Hermitian adjoint of self, aka the Hermitian\n",
      "     |      conjugate or Hermitian transpose. For a complex matrix, the\n",
      "     |      Hermitian adjoint is equal to the conjugate transpose.\n",
      "     |      \n",
      "     |      Can be abbreviated self.H instead of self.adjoint().\n",
      "     |      \n",
      "     |      Returns\n",
      "     |      -------\n",
      "     |      A_H : LinearOperator\n",
      "     |          Hermitian adjoint of self.\n",
      "     |  \n",
      "     |  T\n",
      "     |      Transpose this linear operator.\n",
      "     |      \n",
      "     |      Returns a LinearOperator that represents the transpose of this one.\n",
      "     |      Can be abbreviated self.T instead of self.transpose().\n",
      "     |  \n",
      "     |  __dict__\n",
      "     |      dictionary for instance variables (if defined)\n",
      "     |  \n",
      "     |  __weakref__\n",
      "     |      list of weak references to the object (if defined)\n",
      "    \n",
      "    class OptimizeResult(builtins.dict)\n",
      "     |  Represents the optimization result.\n",
      "     |  \n",
      "     |  Attributes\n",
      "     |  ----------\n",
      "     |  x : ndarray\n",
      "     |      The solution of the optimization.\n",
      "     |  success : bool\n",
      "     |      Whether or not the optimizer exited successfully.\n",
      "     |  status : int\n",
      "     |      Termination status of the optimizer. Its value depends on the\n",
      "     |      underlying solver. Refer to `message` for details.\n",
      "     |  message : str\n",
      "     |      Description of the cause of the termination.\n",
      "     |  fun, jac, hess, hess_inv : ndarray\n",
      "     |      Values of objective function, Jacobian, Hessian or its inverse (if\n",
      "     |      available). The Hessians may be approximations, see the documentation\n",
      "     |      of the function in question.\n",
      "     |  nfev, njev, nhev : int\n",
      "     |      Number of evaluations of the objective functions and of its\n",
      "     |      Jacobian and Hessian.\n",
      "     |  nit : int\n",
      "     |      Number of iterations performed by the optimizer.\n",
      "     |  maxcv : float\n",
      "     |      The maximum constraint violation.\n",
      "     |  \n",
      "     |  Notes\n",
      "     |  -----\n",
      "     |  There may be additional attributes not listed above depending of the\n",
      "     |  specific solver. Since this class is essentially a subclass of dict\n",
      "     |  with attribute accessors, one can see which attributes are available\n",
      "     |  using the `keys()` method.\n",
      "     |  \n",
      "     |  Method resolution order:\n",
      "     |      OptimizeResult\n",
      "     |      builtins.dict\n",
      "     |      builtins.object\n",
      "     |  \n",
      "     |  Methods defined here:\n",
      "     |  \n",
      "     |  __delattr__ = __delitem__(self, key, /)\n",
      "     |      Delete self[key].\n",
      "     |  \n",
      "     |  __getattr__(self, name)\n",
      "     |  \n",
      "     |  __repr__(self)\n",
      "     |  \n",
      "     |  __setattr__ = __setitem__(self, key, value, /)\n",
      "     |      Set self[key] to value.\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Data descriptors defined here:\n",
      "     |  \n",
      "     |  __dict__\n",
      "     |      dictionary for instance variables (if defined)\n",
      "     |  \n",
      "     |  __weakref__\n",
      "     |      list of weak references to the object (if defined)\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Methods inherited from builtins.dict:\n",
      "     |  \n",
      "     |  __contains__(self, key, /)\n",
      "     |      True if D has a key k, else False.\n",
      "     |  \n",
      "     |  __delitem__(self, key, /)\n",
      "     |      Delete self[key].\n",
      "     |  \n",
      "     |  __eq__(self, value, /)\n",
      "     |      Return self==value.\n",
      "     |  \n",
      "     |  __ge__(self, value, /)\n",
      "     |      Return self>=value.\n",
      "     |  \n",
      "     |  __getattribute__(self, name, /)\n",
      "     |      Return getattr(self, name).\n",
      "     |  \n",
      "     |  __getitem__(...)\n",
      "     |      x.__getitem__(y) <==> x[y]\n",
      "     |  \n",
      "     |  __gt__(self, value, /)\n",
      "     |      Return self>value.\n",
      "     |  \n",
      "     |  __init__(self, /, *args, **kwargs)\n",
      "     |      Initialize self.  See help(type(self)) for accurate signature.\n",
      "     |  \n",
      "     |  __iter__(self, /)\n",
      "     |      Implement iter(self).\n",
      "     |  \n",
      "     |  __le__(self, value, /)\n",
      "     |      Return self<=value.\n",
      "     |  \n",
      "     |  __len__(self, /)\n",
      "     |      Return len(self).\n",
      "     |  \n",
      "     |  __lt__(self, value, /)\n",
      "     |      Return self<value.\n",
      "     |  \n",
      "     |  __ne__(self, value, /)\n",
      "     |      Return self!=value.\n",
      "     |  \n",
      "     |  __new__(*args, **kwargs) from builtins.type\n",
      "     |      Create and return a new object.  See help(type) for accurate signature.\n",
      "     |  \n",
      "     |  __setitem__(self, key, value, /)\n",
      "     |      Set self[key] to value.\n",
      "     |  \n",
      "     |  __sizeof__(...)\n",
      "     |      D.__sizeof__() -> size of D in memory, in bytes\n",
      "     |  \n",
      "     |  clear(...)\n",
      "     |      D.clear() -> None.  Remove all items from D.\n",
      "     |  \n",
      "     |  copy(...)\n",
      "     |      D.copy() -> a shallow copy of D\n",
      "     |  \n",
      "     |  fromkeys(iterable, value=None, /) from builtins.type\n",
      "     |      Returns a new dict with keys from iterable and values equal to value.\n",
      "     |  \n",
      "     |  get(...)\n",
      "     |      D.get(k[,d]) -> D[k] if k in D, else d.  d defaults to None.\n",
      "     |  \n",
      "     |  items(...)\n",
      "     |      D.items() -> a set-like object providing a view on D's items\n",
      "     |  \n",
      "     |  keys(...)\n",
      "     |      D.keys() -> a set-like object providing a view on D's keys\n",
      "     |  \n",
      "     |  pop(...)\n",
      "     |      D.pop(k[,d]) -> v, remove specified key and return the corresponding value.\n",
      "     |      If key is not found, d is returned if given, otherwise KeyError is raised\n",
      "     |  \n",
      "     |  popitem(...)\n",
      "     |      D.popitem() -> (k, v), remove and return some (key, value) pair as a\n",
      "     |      2-tuple; but raise KeyError if D is empty.\n",
      "     |  \n",
      "     |  setdefault(...)\n",
      "     |      D.setdefault(k[,d]) -> D.get(k,d), also set D[k]=d if k not in D\n",
      "     |  \n",
      "     |  update(...)\n",
      "     |      D.update([E, ]**F) -> None.  Update D from dict/iterable E and F.\n",
      "     |      If E is present and has a .keys() method, then does:  for k in E: D[k] = E[k]\n",
      "     |      If E is present and lacks a .keys() method, then does:  for k, v in E: D[k] = v\n",
      "     |      In either case, this is followed by: for k in F:  D[k] = F[k]\n",
      "     |  \n",
      "     |  values(...)\n",
      "     |      D.values() -> an object providing a view on D's values\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Data and other attributes inherited from builtins.dict:\n",
      "     |  \n",
      "     |  __hash__ = None\n",
      "    \n",
      "    class OptimizeWarning(builtins.UserWarning)\n",
      "     |  Method resolution order:\n",
      "     |      OptimizeWarning\n",
      "     |      builtins.UserWarning\n",
      "     |      builtins.Warning\n",
      "     |      builtins.Exception\n",
      "     |      builtins.BaseException\n",
      "     |      builtins.object\n",
      "     |  \n",
      "     |  Data descriptors defined here:\n",
      "     |  \n",
      "     |  __weakref__\n",
      "     |      list of weak references to the object (if defined)\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Methods inherited from builtins.UserWarning:\n",
      "     |  \n",
      "     |  __init__(self, /, *args, **kwargs)\n",
      "     |      Initialize self.  See help(type(self)) for accurate signature.\n",
      "     |  \n",
      "     |  __new__(*args, **kwargs) from builtins.type\n",
      "     |      Create and return a new object.  See help(type) for accurate signature.\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Methods inherited from builtins.BaseException:\n",
      "     |  \n",
      "     |  __delattr__(self, name, /)\n",
      "     |      Implement delattr(self, name).\n",
      "     |  \n",
      "     |  __getattribute__(self, name, /)\n",
      "     |      Return getattr(self, name).\n",
      "     |  \n",
      "     |  __reduce__(...)\n",
      "     |  \n",
      "     |  __repr__(self, /)\n",
      "     |      Return repr(self).\n",
      "     |  \n",
      "     |  __setattr__(self, name, value, /)\n",
      "     |      Implement setattr(self, name, value).\n",
      "     |  \n",
      "     |  __setstate__(...)\n",
      "     |  \n",
      "     |  __str__(self, /)\n",
      "     |      Return str(self).\n",
      "     |  \n",
      "     |  with_traceback(...)\n",
      "     |      Exception.with_traceback(tb) --\n",
      "     |      set self.__traceback__ to tb and return self.\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Data descriptors inherited from builtins.BaseException:\n",
      "     |  \n",
      "     |  __cause__\n",
      "     |      exception cause\n",
      "     |  \n",
      "     |  __context__\n",
      "     |      exception context\n",
      "     |  \n",
      "     |  __dict__\n",
      "     |  \n",
      "     |  __suppress_context__\n",
      "     |  \n",
      "     |  __traceback__\n",
      "     |  \n",
      "     |  args\n",
      "\n",
      "FUNCTIONS\n",
      "    anderson(F, xin, iter=None, alpha=None, w0=0.01, M=5, verbose=False, maxiter=None, f_tol=None, f_rtol=None, x_tol=None, x_rtol=None, tol_norm=None, line_search='armijo', callback=None, **kw)\n",
      "        Find a root of a function, using (extended) Anderson mixing.\n",
      "        \n",
      "        The Jacobian is formed by for a 'best' solution in the space\n",
      "        spanned by last `M` vectors. As a result, only a MxM matrix\n",
      "        inversions and MxN multiplications are required. [Ey]_\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        F : function(x) -> f\n",
      "            Function whose root to find; should take and return an array-like\n",
      "            object.\n",
      "        x0 : array_like\n",
      "            Initial guess for the solution\n",
      "        alpha : float, optional\n",
      "            Initial guess for the Jacobian is (-1/alpha).\n",
      "        M : float, optional\n",
      "            Number of previous vectors to retain. Defaults to 5.\n",
      "        w0 : float, optional\n",
      "            Regularization parameter for numerical stability.\n",
      "            Compared to unity, good values of the order of 0.01.\n",
      "        iter : int, optional\n",
      "            Number of iterations to make. If omitted (default), make as many\n",
      "            as required to meet tolerances.\n",
      "        verbose : bool, optional\n",
      "            Print status to stdout on every iteration.\n",
      "        maxiter : int, optional\n",
      "            Maximum number of iterations to make. If more are needed to\n",
      "            meet convergence, `NoConvergence` is raised.\n",
      "        f_tol : float, optional\n",
      "            Absolute tolerance (in max-norm) for the residual.\n",
      "            If omitted, default is 6e-6.\n",
      "        f_rtol : float, optional\n",
      "            Relative tolerance for the residual. If omitted, not used.\n",
      "        x_tol : float, optional\n",
      "            Absolute minimum step size, as determined from the Jacobian\n",
      "            approximation. If the step size is smaller than this, optimization\n",
      "            is terminated as successful. If omitted, not used.\n",
      "        x_rtol : float, optional\n",
      "            Relative minimum step size. If omitted, not used.\n",
      "        tol_norm : function(vector) -> scalar, optional\n",
      "            Norm to use in convergence check. Default is the maximum norm.\n",
      "        line_search : {None, 'armijo' (default), 'wolfe'}, optional\n",
      "            Which type of a line search to use to determine the step size in the\n",
      "            direction given by the Jacobian approximation. Defaults to 'armijo'.\n",
      "        callback : function, optional\n",
      "            Optional callback function. It is called on every iteration as\n",
      "            ``callback(x, f)`` where `x` is the current solution and `f`\n",
      "            the corresponding residual.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        sol : ndarray\n",
      "            An array (of similar array type as `x0`) containing the final solution.\n",
      "        \n",
      "        Raises\n",
      "        ------\n",
      "        NoConvergence\n",
      "            When a solution was not found.\n",
      "        \n",
      "        References\n",
      "        ----------\n",
      "        .. [Ey] V. Eyert, J. Comp. Phys., 124, 271 (1996).\n",
      "    \n",
      "    approx_fprime(xk, f, epsilon, *args)\n",
      "        Finite-difference approximation of the gradient of a scalar function.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        xk : array_like\n",
      "            The coordinate vector at which to determine the gradient of `f`.\n",
      "        f : callable\n",
      "            The function of which to determine the gradient (partial derivatives).\n",
      "            Should take `xk` as first argument, other arguments to `f` can be\n",
      "            supplied in ``*args``.  Should return a scalar, the value of the\n",
      "            function at `xk`.\n",
      "        epsilon : array_like\n",
      "            Increment to `xk` to use for determining the function gradient.\n",
      "            If a scalar, uses the same finite difference delta for all partial\n",
      "            derivatives.  If an array, should contain one value per element of\n",
      "            `xk`.\n",
      "        \\*args : args, optional\n",
      "            Any other arguments that are to be passed to `f`.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        grad : ndarray\n",
      "            The partial derivatives of `f` to `xk`.\n",
      "        \n",
      "        See Also\n",
      "        --------\n",
      "        check_grad : Check correctness of gradient function against approx_fprime.\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        The function gradient is determined by the forward finite difference\n",
      "        formula::\n",
      "        \n",
      "                     f(xk[i] + epsilon[i]) - f(xk[i])\n",
      "            f'[i] = ---------------------------------\n",
      "                                epsilon[i]\n",
      "        \n",
      "        The main use of `approx_fprime` is in scalar function optimizers like\n",
      "        `fmin_bfgs`, to determine numerically the Jacobian of a function.\n",
      "        \n",
      "        Examples\n",
      "        --------\n",
      "        >>> from scipy import optimize\n",
      "        >>> def func(x, c0, c1):\n",
      "        ...     \"Coordinate vector `x` should be an array of size two.\"\n",
      "        ...     return c0 * x[0]**2 + c1*x[1]**2\n",
      "        \n",
      "        >>> x = np.ones(2)\n",
      "        >>> c0, c1 = (1, 200)\n",
      "        >>> eps = np.sqrt(np.finfo(np.float).eps)\n",
      "        >>> optimize.approx_fprime(x, func, [eps, np.sqrt(200) * eps], c0, c1)\n",
      "        array([   2.        ,  400.00004198])\n",
      "    \n",
      "    basinhopping(func, x0, niter=100, T=1.0, stepsize=0.5, minimizer_kwargs=None, take_step=None, accept_test=None, callback=None, interval=50, disp=False, niter_success=None)\n",
      "        Find the global minimum of a function using the basin-hopping algorithm\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        func : callable ``f(x, *args)``\n",
      "            Function to be optimized.  ``args`` can be passed as an optional item\n",
      "            in the dict ``minimizer_kwargs``\n",
      "        x0 : ndarray\n",
      "            Initial guess.\n",
      "        niter : integer, optional\n",
      "            The number of basin hopping iterations\n",
      "        T : float, optional\n",
      "            The \"temperature\" parameter for the accept or reject criterion.  Higher\n",
      "            \"temperatures\" mean that larger jumps in function value will be\n",
      "            accepted.  For best results ``T`` should be comparable to the\n",
      "            separation\n",
      "            (in function value) between local minima.\n",
      "        stepsize : float, optional\n",
      "            initial step size for use in the random displacement.\n",
      "        minimizer_kwargs : dict, optional\n",
      "            Extra keyword arguments to be passed to the minimizer\n",
      "            ``scipy.optimize.minimize()`` Some important options could be:\n",
      "        \n",
      "                method : str\n",
      "                    The minimization method (e.g. ``\"L-BFGS-B\"``)\n",
      "                args : tuple\n",
      "                    Extra arguments passed to the objective function (``func``) and\n",
      "                    its derivatives (Jacobian, Hessian).\n",
      "        \n",
      "        take_step : callable ``take_step(x)``, optional\n",
      "            Replace the default step taking routine with this routine.  The default\n",
      "            step taking routine is a random displacement of the coordinates, but\n",
      "            other step taking algorithms may be better for some systems.\n",
      "            ``take_step`` can optionally have the attribute ``take_step.stepsize``.\n",
      "            If this attribute exists, then ``basinhopping`` will adjust\n",
      "            ``take_step.stepsize`` in order to try to optimize the global minimum\n",
      "            search.\n",
      "        accept_test : callable, ``accept_test(f_new=f_new, x_new=x_new, f_old=fold, x_old=x_old)``, optional\n",
      "            Define a test which will be used to judge whether or not to accept the\n",
      "            step.  This will be used in addition to the Metropolis test based on\n",
      "            \"temperature\" ``T``.  The acceptable return values are True,\n",
      "            False, or ``\"force accept\"``.  If the latter, then this will\n",
      "            override any other tests in order to accept the step.  This can be\n",
      "            used, for example, to forcefully escape from a local minimum that\n",
      "            ``basinhopping`` is trapped in.\n",
      "        callback : callable, ``callback(x, f, accept)``, optional\n",
      "            A callback function which will be called for all minima found.  ``x``\n",
      "            and ``f`` are the coordinates and function value of the trial minimum,\n",
      "            and ``accept`` is whether or not that minimum was accepted.  This can be\n",
      "            used, for example, to save the lowest N minima found.  Also,\n",
      "            ``callback`` can be used to specify a user defined stop criterion by\n",
      "            optionally returning True to stop the ``basinhopping`` routine.\n",
      "        interval : integer, optional\n",
      "            interval for how often to update the ``stepsize``\n",
      "        disp : bool, optional\n",
      "            Set to True to print status messages\n",
      "        niter_success : integer, optional\n",
      "            Stop the run if the global minimum candidate remains the same for this\n",
      "            number of iterations.\n",
      "        \n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        res : OptimizeResult\n",
      "            The optimization result represented as a ``OptimizeResult`` object.  Important\n",
      "            attributes are: ``x`` the solution array, ``fun`` the value of the\n",
      "            function at the solution, and ``message`` which describes the cause of\n",
      "            the termination. See `OptimizeResult` for a description of other attributes.\n",
      "        \n",
      "        See Also\n",
      "        --------\n",
      "        minimize :\n",
      "            The local minimization function called once for each basinhopping step.\n",
      "            ``minimizer_kwargs`` is passed to this routine.\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        Basin-hopping is a stochastic algorithm which attempts to find the global\n",
      "        minimum of a smooth scalar function of one or more variables [1]_ [2]_ [3]_\n",
      "        [4]_.  The algorithm in its current form was described by David Wales and\n",
      "        Jonathan Doye [2]_ http://www-wales.ch.cam.ac.uk/.\n",
      "        \n",
      "        The algorithm is iterative with each cycle composed of the following\n",
      "        features\n",
      "        \n",
      "        1) random perturbation of the coordinates\n",
      "        \n",
      "        2) local minimization\n",
      "        \n",
      "        3) accept or reject the new coordinates based on the minimized function\n",
      "           value\n",
      "        \n",
      "        The acceptance test used here is the Metropolis criterion of standard Monte\n",
      "        Carlo algorithms, although there are many other possibilities [3]_.\n",
      "        \n",
      "        This global minimization method has been shown to be extremely efficient\n",
      "        for a wide variety of problems in physics and chemistry.  It is\n",
      "        particularly useful when the function has many minima separated by large\n",
      "        barriers. See the Cambridge Cluster Database\n",
      "        http://www-wales.ch.cam.ac.uk/CCD.html for databases of molecular systems\n",
      "        that have been optimized primarily using basin-hopping.  This database\n",
      "        includes minimization problems exceeding 300 degrees of freedom.\n",
      "        \n",
      "        See the free software program GMIN (http://www-wales.ch.cam.ac.uk/GMIN) for\n",
      "        a Fortran implementation of basin-hopping.  This implementation has many\n",
      "        different variations of the procedure described above, including more\n",
      "        advanced step taking algorithms and alternate acceptance criterion.\n",
      "        \n",
      "        For stochastic global optimization there is no way to determine if the true\n",
      "        global minimum has actually been found. Instead, as a consistency check,\n",
      "        the algorithm can be run from a number of different random starting points\n",
      "        to ensure the lowest minimum found in each example has converged to the\n",
      "        global minimum.  For this reason ``basinhopping`` will by default simply\n",
      "        run for the number of iterations ``niter`` and return the lowest minimum\n",
      "        found.  It is left to the user to ensure that this is in fact the global\n",
      "        minimum.\n",
      "        \n",
      "        Choosing ``stepsize``:  This is a crucial parameter in ``basinhopping`` and\n",
      "        depends on the problem being solved.  Ideally it should be comparable to\n",
      "        the typical separation between local minima of the function being\n",
      "        optimized.  ``basinhopping`` will, by default, adjust ``stepsize`` to find\n",
      "        an optimal value, but this may take many iterations.  You will get quicker\n",
      "        results if you set a sensible value for ``stepsize``.\n",
      "        \n",
      "        Choosing ``T``: The parameter ``T`` is the temperature used in the\n",
      "        metropolis criterion.  Basinhopping steps are accepted with probability\n",
      "        ``1`` if ``func(xnew) < func(xold)``, or otherwise with probability::\n",
      "        \n",
      "            exp( -(func(xnew) - func(xold)) / T )\n",
      "        \n",
      "        So, for best results, ``T`` should to be comparable to the typical\n",
      "        difference in function values between local minima.\n",
      "        \n",
      "        .. versionadded:: 0.12.0\n",
      "        \n",
      "        References\n",
      "        ----------\n",
      "        .. [1] Wales, David J. 2003, Energy Landscapes, Cambridge University Press,\n",
      "            Cambridge, UK.\n",
      "        .. [2] Wales, D J, and Doye J P K, Global Optimization by Basin-Hopping and\n",
      "            the Lowest Energy Structures of Lennard-Jones Clusters Containing up to\n",
      "            110 Atoms.  Journal of Physical Chemistry A, 1997, 101, 5111.\n",
      "        .. [3] Li, Z. and Scheraga, H. A., Monte Carlo-minimization approach to the\n",
      "            multiple-minima problem in protein folding, Proc. Natl. Acad. Sci. USA,\n",
      "            1987, 84, 6611.\n",
      "        .. [4] Wales, D. J. and Scheraga, H. A., Global optimization of clusters,\n",
      "            crystals, and biomolecules, Science, 1999, 285, 1368.\n",
      "        \n",
      "        Examples\n",
      "        --------\n",
      "        The following example is a one-dimensional minimization problem,  with many\n",
      "        local minima superimposed on a parabola.\n",
      "        \n",
      "        >>> from scipy.optimize import basinhopping\n",
      "        >>> func = lambda x: np.cos(14.5 * x - 0.3) + (x + 0.2) * x\n",
      "        >>> x0=[1.]\n",
      "        \n",
      "        Basinhopping, internally, uses a local minimization algorithm.  We will use\n",
      "        the parameter ``minimizer_kwargs`` to tell basinhopping which algorithm to\n",
      "        use and how to set up that minimizer.  This parameter will be passed to\n",
      "        ``scipy.optimize.minimize()``.\n",
      "        \n",
      "        >>> minimizer_kwargs = {\"method\": \"BFGS\"}\n",
      "        >>> ret = basinhopping(func, x0, minimizer_kwargs=minimizer_kwargs,\n",
      "        ...                    niter=200)\n",
      "        >>> print(\"global minimum: x = %.4f, f(x0) = %.4f\" % (ret.x, ret.fun))\n",
      "        global minimum: x = -0.1951, f(x0) = -1.0009\n",
      "        \n",
      "        Next consider a two-dimensional minimization problem. Also, this time we\n",
      "        will use gradient information to significantly speed up the search.\n",
      "        \n",
      "        >>> def func2d(x):\n",
      "        ...     f = np.cos(14.5 * x[0] - 0.3) + (x[1] + 0.2) * x[1] + (x[0] +\n",
      "        ...                                                            0.2) * x[0]\n",
      "        ...     df = np.zeros(2)\n",
      "        ...     df[0] = -14.5 * np.sin(14.5 * x[0] - 0.3) + 2. * x[0] + 0.2\n",
      "        ...     df[1] = 2. * x[1] + 0.2\n",
      "        ...     return f, df\n",
      "        \n",
      "        We'll also use a different local minimization algorithm.  Also we must tell\n",
      "        the minimizer that our function returns both energy and gradient (jacobian)\n",
      "        \n",
      "        >>> minimizer_kwargs = {\"method\":\"L-BFGS-B\", \"jac\":True}\n",
      "        >>> x0 = [1.0, 1.0]\n",
      "        >>> ret = basinhopping(func2d, x0, minimizer_kwargs=minimizer_kwargs,\n",
      "        ...                    niter=200)\n",
      "        >>> print(\"global minimum: x = [%.4f, %.4f], f(x0) = %.4f\" % (ret.x[0],\n",
      "        ...                                                           ret.x[1],\n",
      "        ...                                                           ret.fun))\n",
      "        global minimum: x = [-0.1951, -0.1000], f(x0) = -1.0109\n",
      "        \n",
      "        \n",
      "        Here is an example using a custom step taking routine.  Imagine you want\n",
      "        the first coordinate to take larger steps then the rest of the coordinates.\n",
      "        This can be implemented like so:\n",
      "        \n",
      "        >>> class MyTakeStep(object):\n",
      "        ...    def __init__(self, stepsize=0.5):\n",
      "        ...        self.stepsize = stepsize\n",
      "        ...    def __call__(self, x):\n",
      "        ...        s = self.stepsize\n",
      "        ...        x[0] += np.random.uniform(-2.*s, 2.*s)\n",
      "        ...        x[1:] += np.random.uniform(-s, s, x[1:].shape)\n",
      "        ...        return x\n",
      "        \n",
      "        Since ``MyTakeStep.stepsize`` exists basinhopping will adjust the magnitude\n",
      "        of ``stepsize`` to optimize the search.  We'll use the same 2-D function as\n",
      "        before\n",
      "        \n",
      "        >>> mytakestep = MyTakeStep()\n",
      "        >>> ret = basinhopping(func2d, x0, minimizer_kwargs=minimizer_kwargs,\n",
      "        ...                    niter=200, take_step=mytakestep)\n",
      "        >>> print(\"global minimum: x = [%.4f, %.4f], f(x0) = %.4f\" % (ret.x[0],\n",
      "        ...                                                           ret.x[1],\n",
      "        ...                                                           ret.fun))\n",
      "        global minimum: x = [-0.1951, -0.1000], f(x0) = -1.0109\n",
      "        \n",
      "        \n",
      "        Now let's do an example using a custom callback function which prints the\n",
      "        value of every minimum found\n",
      "        \n",
      "        >>> def print_fun(x, f, accepted):\n",
      "        ...         print(\"at minimum %.4f accepted %d\" % (f, int(accepted)))\n",
      "        \n",
      "        We'll run it for only 10 basinhopping steps this time.\n",
      "        \n",
      "        >>> np.random.seed(1)\n",
      "        >>> ret = basinhopping(func2d, x0, minimizer_kwargs=minimizer_kwargs,\n",
      "        ...                    niter=10, callback=print_fun)\n",
      "        at minimum 0.4159 accepted 1\n",
      "        at minimum -0.9073 accepted 1\n",
      "        at minimum -0.1021 accepted 1\n",
      "        at minimum -0.1021 accepted 1\n",
      "        at minimum 0.9102 accepted 1\n",
      "        at minimum 0.9102 accepted 1\n",
      "        at minimum 2.2945 accepted 0\n",
      "        at minimum -0.1021 accepted 1\n",
      "        at minimum -1.0109 accepted 1\n",
      "        at minimum -1.0109 accepted 1\n",
      "        \n",
      "        \n",
      "        The minimum at -1.0109 is actually the global minimum, found already on the\n",
      "        8th iteration.\n",
      "        \n",
      "        Now let's implement bounds on the problem using a custom ``accept_test``:\n",
      "        \n",
      "        >>> class MyBounds(object):\n",
      "        ...     def __init__(self, xmax=[1.1,1.1], xmin=[-1.1,-1.1] ):\n",
      "        ...         self.xmax = np.array(xmax)\n",
      "        ...         self.xmin = np.array(xmin)\n",
      "        ...     def __call__(self, **kwargs):\n",
      "        ...         x = kwargs[\"x_new\"]\n",
      "        ...         tmax = bool(np.all(x <= self.xmax))\n",
      "        ...         tmin = bool(np.all(x >= self.xmin))\n",
      "        ...         return tmax and tmin\n",
      "        \n",
      "        >>> mybounds = MyBounds()\n",
      "        >>> ret = basinhopping(func2d, x0, minimizer_kwargs=minimizer_kwargs,\n",
      "        ...                    niter=10, accept_test=mybounds)\n",
      "    \n",
      "    bisect(f, a, b, args=(), xtol=1e-12, rtol=4.4408920985006262e-16, maxiter=100, full_output=False, disp=True)\n",
      "        Find root of a function within an interval.\n",
      "        \n",
      "        Basic bisection routine to find a zero of the function `f` between the\n",
      "        arguments `a` and `b`. `f(a)` and `f(b)` cannot have the same signs.\n",
      "        Slow but sure.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        f : function\n",
      "            Python function returning a number.  `f` must be continuous, and\n",
      "            f(a) and f(b) must have opposite signs.\n",
      "        a : number\n",
      "            One end of the bracketing interval [a,b].\n",
      "        b : number\n",
      "            The other end of the bracketing interval [a,b].\n",
      "        xtol : number, optional\n",
      "            The routine converges when a root is known to lie within `xtol` of the\n",
      "            value return. Should be >= 0.  The routine modifies this to take into\n",
      "            account the relative precision of doubles.\n",
      "        rtol : number, optional\n",
      "            The routine converges when a root is known to lie within `rtol` times\n",
      "            the value returned of the value returned. Should be >= 0. Defaults to\n",
      "            ``np.finfo(float).eps * 2``.\n",
      "        maxiter : number, optional\n",
      "            if convergence is not achieved in `maxiter` iterations, an error is\n",
      "            raised.  Must be >= 0.\n",
      "        args : tuple, optional\n",
      "            containing extra arguments for the function `f`.\n",
      "            `f` is called by ``apply(f, (x)+args)``.\n",
      "        full_output : bool, optional\n",
      "            If `full_output` is False, the root is returned.  If `full_output` is\n",
      "            True, the return value is ``(x, r)``, where x is the root, and r is\n",
      "            a `RootResults` object.\n",
      "        disp : bool, optional\n",
      "            If True, raise RuntimeError if the algorithm didn't converge.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        x0 : float\n",
      "            Zero of `f` between `a` and `b`.\n",
      "        r : RootResults (present if ``full_output = True``)\n",
      "            Object containing information about the convergence.  In particular,\n",
      "            ``r.converged`` is True if the routine converged.\n",
      "        \n",
      "        See Also\n",
      "        --------\n",
      "        brentq, brenth, bisect, newton\n",
      "        fixed_point : scalar fixed-point finder\n",
      "        fsolve : n-dimensional root-finding\n",
      "    \n",
      "    bracket(func, xa=0.0, xb=1.0, args=(), grow_limit=110.0, maxiter=1000)\n",
      "        Bracket the minimum of the function.\n",
      "        \n",
      "        Given a function and distinct initial points, search in the\n",
      "        downhill direction (as defined by the initital points) and return\n",
      "        new points xa, xb, xc that bracket the minimum of the function\n",
      "        f(xa) > f(xb) < f(xc). It doesn't always mean that obtained\n",
      "        solution will satisfy xa<=x<=xb\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        func : callable f(x,*args)\n",
      "            Objective function to minimize.\n",
      "        xa, xb : float, optional\n",
      "            Bracketing interval. Defaults `xa` to 0.0, and `xb` to 1.0.\n",
      "        args : tuple, optional\n",
      "            Additional arguments (if present), passed to `func`.\n",
      "        grow_limit : float, optional\n",
      "            Maximum grow limit.  Defaults to 110.0\n",
      "        maxiter : int, optional\n",
      "            Maximum number of iterations to perform. Defaults to 1000.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        xa, xb, xc : float\n",
      "            Bracket.\n",
      "        fa, fb, fc : float\n",
      "            Objective function values in bracket.\n",
      "        funcalls : int\n",
      "            Number of function evaluations made.\n",
      "    \n",
      "    brent(func, args=(), brack=None, tol=1.48e-08, full_output=0, maxiter=500)\n",
      "        Given a function of one-variable and a possible bracketing interval,\n",
      "        return the minimum of the function isolated to a fractional precision of\n",
      "        tol.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        func : callable f(x,*args)\n",
      "            Objective function.\n",
      "        args : tuple, optional\n",
      "            Additional arguments (if present).\n",
      "        brack : tuple, optional\n",
      "            Triple (a,b,c) where (a<b<c) and func(b) <\n",
      "            func(a),func(c).  If bracket consists of two numbers (a,c)\n",
      "            then they are assumed to be a starting interval for a\n",
      "            downhill bracket search (see `bracket`); it doesn't always\n",
      "            mean that the obtained solution will satisfy a<=x<=c.\n",
      "        tol : float, optional\n",
      "            Stop if between iteration change is less than `tol`.\n",
      "        full_output : bool, optional\n",
      "            If True, return all output args (xmin, fval, iter,\n",
      "            funcalls).\n",
      "        maxiter : int, optional\n",
      "            Maximum number of iterations in solution.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        xmin : ndarray\n",
      "            Optimum point.\n",
      "        fval : float\n",
      "            Optimum value.\n",
      "        iter : int\n",
      "            Number of iterations.\n",
      "        funcalls : int\n",
      "            Number of objective function evaluations made.\n",
      "        \n",
      "        See also\n",
      "        --------\n",
      "        minimize_scalar: Interface to minimization algorithms for scalar\n",
      "            univariate functions. See the 'Brent' `method` in particular.\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        Uses inverse parabolic interpolation when possible to speed up\n",
      "        convergence of golden section method.\n",
      "    \n",
      "    brenth(f, a, b, args=(), xtol=1e-12, rtol=4.4408920985006262e-16, maxiter=100, full_output=False, disp=True)\n",
      "        Find root of f in [a,b].\n",
      "        \n",
      "        A variation on the classic Brent routine to find a zero of the function f\n",
      "        between the arguments a and b that uses hyperbolic extrapolation instead of\n",
      "        inverse quadratic extrapolation. There was a paper back in the 1980's ...\n",
      "        f(a) and f(b) cannot have the same signs. Generally on a par with the\n",
      "        brent routine, but not as heavily tested.  It is a safe version of the\n",
      "        secant method that uses hyperbolic extrapolation. The version here is by\n",
      "        Chuck Harris.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        f : function\n",
      "            Python function returning a number.  f must be continuous, and f(a) and\n",
      "            f(b) must have opposite signs.\n",
      "        a : number\n",
      "            One end of the bracketing interval [a,b].\n",
      "        b : number\n",
      "            The other end of the bracketing interval [a,b].\n",
      "        xtol : number, optional\n",
      "            The routine converges when a root is known to lie within xtol of the\n",
      "            value return. Should be >= 0.  The routine modifies this to take into\n",
      "            account the relative precision of doubles.\n",
      "        rtol : number, optional\n",
      "            The routine converges when a root is known to lie within `rtol` times\n",
      "            the value returned of the value returned. Should be >= 0. Defaults to\n",
      "            ``np.finfo(float).eps * 2``.\n",
      "        maxiter : number, optional\n",
      "            if convergence is not achieved in maxiter iterations, an error is\n",
      "            raised.  Must be >= 0.\n",
      "        args : tuple, optional\n",
      "            containing extra arguments for the function `f`.\n",
      "            `f` is called by ``apply(f, (x)+args)``.\n",
      "        full_output : bool, optional\n",
      "            If `full_output` is False, the root is returned.  If `full_output` is\n",
      "            True, the return value is ``(x, r)``, where `x` is the root, and `r` is\n",
      "            a RootResults object.\n",
      "        disp : bool, optional\n",
      "            If True, raise RuntimeError if the algorithm didn't converge.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        x0 : float\n",
      "            Zero of `f` between `a` and `b`.\n",
      "        r : RootResults (present if ``full_output = True``)\n",
      "            Object containing information about the convergence.  In particular,\n",
      "            ``r.converged`` is True if the routine converged.\n",
      "        \n",
      "        See Also\n",
      "        --------\n",
      "        fmin, fmin_powell, fmin_cg,\n",
      "               fmin_bfgs, fmin_ncg : multivariate local optimizers\n",
      "        \n",
      "        leastsq : nonlinear least squares minimizer\n",
      "        \n",
      "        fmin_l_bfgs_b, fmin_tnc, fmin_cobyla : constrained multivariate optimizers\n",
      "        \n",
      "        basinhopping, differential_evolution, brute : global optimizers\n",
      "        \n",
      "        fminbound, brent, golden, bracket : local scalar minimizers\n",
      "        \n",
      "        fsolve : n-dimensional root-finding\n",
      "        \n",
      "        brentq, brenth, ridder, bisect, newton : one-dimensional root-finding\n",
      "        \n",
      "        fixed_point : scalar fixed-point finder\n",
      "    \n",
      "    brentq(f, a, b, args=(), xtol=1e-12, rtol=4.4408920985006262e-16, maxiter=100, full_output=False, disp=True)\n",
      "        Find a root of a function in given interval.\n",
      "        \n",
      "        Return float, a zero of `f` between `a` and `b`.  `f` must be a continuous\n",
      "        function, and [a,b] must be a sign changing interval.\n",
      "        \n",
      "        Description:\n",
      "        Uses the classic Brent (1973) method to find a zero of the function `f` on\n",
      "        the sign changing interval [a , b].  Generally considered the best of the\n",
      "        rootfinding routines here.  It is a safe version of the secant method that\n",
      "        uses inverse quadratic extrapolation.  Brent's method combines root\n",
      "        bracketing, interval bisection, and inverse quadratic interpolation.  It is\n",
      "        sometimes known as the van Wijngaarden-Dekker-Brent method.  Brent (1973)\n",
      "        claims convergence is guaranteed for functions computable within [a,b].\n",
      "        \n",
      "        [Brent1973]_ provides the classic description of the algorithm.  Another\n",
      "        description can be found in a recent edition of Numerical Recipes, including\n",
      "        [PressEtal1992]_.  Another description is at\n",
      "        http://mathworld.wolfram.com/BrentsMethod.html.  It should be easy to\n",
      "        understand the algorithm just by reading our code.  Our code diverges a bit\n",
      "        from standard presentations: we choose a different formula for the\n",
      "        extrapolation step.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        f : function\n",
      "            Python function returning a number.  f must be continuous, and f(a) and\n",
      "            f(b) must have opposite signs.\n",
      "        a : number\n",
      "            One end of the bracketing interval [a,b].\n",
      "        b : number\n",
      "            The other end of the bracketing interval [a,b].\n",
      "        xtol : number, optional\n",
      "            The routine converges when a root is known to lie within xtol of the\n",
      "            value return. Should be >= 0.  The routine modifies this to take into\n",
      "            account the relative precision of doubles.\n",
      "        rtol : number, optional\n",
      "            The routine converges when a root is known to lie within `rtol` times\n",
      "            the value returned of the value returned. Should be >= 0. Defaults to\n",
      "            ``np.finfo(float).eps * 2``.\n",
      "        maxiter : number, optional\n",
      "            if convergence is not achieved in maxiter iterations, an error is\n",
      "            raised.  Must be >= 0.\n",
      "        args : tuple, optional\n",
      "            containing extra arguments for the function `f`.\n",
      "            `f` is called by ``apply(f, (x)+args)``.\n",
      "        full_output : bool, optional\n",
      "            If `full_output` is False, the root is returned.  If `full_output` is\n",
      "            True, the return value is ``(x, r)``, where `x` is the root, and `r` is\n",
      "            a RootResults object.\n",
      "        disp : bool, optional\n",
      "            If True, raise RuntimeError if the algorithm didn't converge.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        x0 : float\n",
      "            Zero of `f` between `a` and `b`.\n",
      "        r : RootResults (present if ``full_output = True``)\n",
      "            Object containing information about the convergence.  In particular,\n",
      "            ``r.converged`` is True if the routine converged.\n",
      "        \n",
      "        See Also\n",
      "        --------\n",
      "        multivariate local optimizers\n",
      "          `fmin`, `fmin_powell`, `fmin_cg`, `fmin_bfgs`, `fmin_ncg`\n",
      "        nonlinear least squares minimizer\n",
      "          `leastsq`\n",
      "        constrained multivariate optimizers\n",
      "          `fmin_l_bfgs_b`, `fmin_tnc`, `fmin_cobyla`\n",
      "        global optimizers\n",
      "          `basinhopping`, `brute`, `differential_evolution`\n",
      "        local scalar minimizers\n",
      "          `fminbound`, `brent`, `golden`, `bracket`\n",
      "        n-dimensional root-finding\n",
      "          `fsolve`\n",
      "        one-dimensional root-finding\n",
      "          `brentq`, `brenth`, `ridder`, `bisect`, `newton`\n",
      "        scalar fixed-point finder\n",
      "          `fixed_point`\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        `f` must be continuous.  f(a) and f(b) must have opposite signs.\n",
      "        \n",
      "        \n",
      "        References\n",
      "        ----------\n",
      "        .. [Brent1973]\n",
      "           Brent, R. P.,\n",
      "           *Algorithms for Minimization Without Derivatives*.\n",
      "           Englewood Cliffs, NJ: Prentice-Hall, 1973. Ch. 3-4.\n",
      "        \n",
      "        .. [PressEtal1992]\n",
      "           Press, W. H.; Flannery, B. P.; Teukolsky, S. A.; and Vetterling, W. T.\n",
      "           *Numerical Recipes in FORTRAN: The Art of Scientific Computing*, 2nd ed.\n",
      "           Cambridge, England: Cambridge University Press, pp. 352-355, 1992.\n",
      "           Section 9.3:  \"Van Wijngaarden-Dekker-Brent Method.\"\n",
      "    \n",
      "    broyden1(F, xin, iter=None, alpha=None, reduction_method='restart', max_rank=None, verbose=False, maxiter=None, f_tol=None, f_rtol=None, x_tol=None, x_rtol=None, tol_norm=None, line_search='armijo', callback=None, **kw)\n",
      "        Find a root of a function, using Broyden's first Jacobian approximation.\n",
      "        \n",
      "        This method is also known as \\\"Broyden's good method\\\".\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        F : function(x) -> f\n",
      "            Function whose root to find; should take and return an array-like\n",
      "            object.\n",
      "        x0 : array_like\n",
      "            Initial guess for the solution\n",
      "        alpha : float, optional\n",
      "            Initial guess for the Jacobian is ``(-1/alpha)``.\n",
      "        reduction_method : str or tuple, optional\n",
      "            Method used in ensuring that the rank of the Broyden matrix\n",
      "            stays low. Can either be a string giving the name of the method,\n",
      "            or a tuple of the form ``(method, param1, param2, ...)``\n",
      "            that gives the name of the method and values for additional parameters.\n",
      "        \n",
      "            Methods available:\n",
      "        \n",
      "                - ``restart``: drop all matrix columns. Has no extra parameters.\n",
      "                - ``simple``: drop oldest matrix column. Has no extra parameters.\n",
      "                - ``svd``: keep only the most significant SVD components.\n",
      "                  Takes an extra parameter, ``to_retain``, which determines the\n",
      "                  number of SVD components to retain when rank reduction is done.\n",
      "                  Default is ``max_rank - 2``.\n",
      "        \n",
      "        max_rank : int, optional\n",
      "            Maximum rank for the Broyden matrix.\n",
      "            Default is infinity (ie., no rank reduction).\n",
      "        iter : int, optional\n",
      "            Number of iterations to make. If omitted (default), make as many\n",
      "            as required to meet tolerances.\n",
      "        verbose : bool, optional\n",
      "            Print status to stdout on every iteration.\n",
      "        maxiter : int, optional\n",
      "            Maximum number of iterations to make. If more are needed to\n",
      "            meet convergence, `NoConvergence` is raised.\n",
      "        f_tol : float, optional\n",
      "            Absolute tolerance (in max-norm) for the residual.\n",
      "            If omitted, default is 6e-6.\n",
      "        f_rtol : float, optional\n",
      "            Relative tolerance for the residual. If omitted, not used.\n",
      "        x_tol : float, optional\n",
      "            Absolute minimum step size, as determined from the Jacobian\n",
      "            approximation. If the step size is smaller than this, optimization\n",
      "            is terminated as successful. If omitted, not used.\n",
      "        x_rtol : float, optional\n",
      "            Relative minimum step size. If omitted, not used.\n",
      "        tol_norm : function(vector) -> scalar, optional\n",
      "            Norm to use in convergence check. Default is the maximum norm.\n",
      "        line_search : {None, 'armijo' (default), 'wolfe'}, optional\n",
      "            Which type of a line search to use to determine the step size in the\n",
      "            direction given by the Jacobian approximation. Defaults to 'armijo'.\n",
      "        callback : function, optional\n",
      "            Optional callback function. It is called on every iteration as\n",
      "            ``callback(x, f)`` where `x` is the current solution and `f`\n",
      "            the corresponding residual.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        sol : ndarray\n",
      "            An array (of similar array type as `x0`) containing the final solution.\n",
      "        \n",
      "        Raises\n",
      "        ------\n",
      "        NoConvergence\n",
      "            When a solution was not found.\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        This algorithm implements the inverse Jacobian Quasi-Newton update\n",
      "        \n",
      "        .. math:: H_+ = H + (dx - H df) dx^\\dagger H / ( dx^\\dagger H df)\n",
      "        \n",
      "        which corresponds to Broyden's first Jacobian update\n",
      "        \n",
      "        .. math:: J_+ = J + (df - J dx) dx^\\dagger / dx^\\dagger dx\n",
      "        \n",
      "        \n",
      "        References\n",
      "        ----------\n",
      "        .. [1] B.A. van der Rotten, PhD thesis,\n",
      "           \\\"A limited memory Broyden method to solve high-dimensional\n",
      "           systems of nonlinear equations\\\". Mathematisch Instituut,\n",
      "           Universiteit Leiden, The Netherlands (2003).\n",
      "        \n",
      "           http://www.math.leidenuniv.nl/scripties/Rotten.pdf\n",
      "    \n",
      "    broyden2(F, xin, iter=None, alpha=None, reduction_method='restart', max_rank=None, verbose=False, maxiter=None, f_tol=None, f_rtol=None, x_tol=None, x_rtol=None, tol_norm=None, line_search='armijo', callback=None, **kw)\n",
      "        Find a root of a function, using Broyden's second Jacobian approximation.\n",
      "        \n",
      "        This method is also known as \"Broyden's bad method\".\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        F : function(x) -> f\n",
      "            Function whose root to find; should take and return an array-like\n",
      "            object.\n",
      "        x0 : array_like\n",
      "            Initial guess for the solution\n",
      "        alpha : float, optional\n",
      "            Initial guess for the Jacobian is ``(-1/alpha)``.\n",
      "        reduction_method : str or tuple, optional\n",
      "            Method used in ensuring that the rank of the Broyden matrix\n",
      "            stays low. Can either be a string giving the name of the method,\n",
      "            or a tuple of the form ``(method, param1, param2, ...)``\n",
      "            that gives the name of the method and values for additional parameters.\n",
      "        \n",
      "            Methods available:\n",
      "        \n",
      "                - ``restart``: drop all matrix columns. Has no extra parameters.\n",
      "                - ``simple``: drop oldest matrix column. Has no extra parameters.\n",
      "                - ``svd``: keep only the most significant SVD components.\n",
      "                  Takes an extra parameter, ``to_retain``, which determines the\n",
      "                  number of SVD components to retain when rank reduction is done.\n",
      "                  Default is ``max_rank - 2``.\n",
      "        \n",
      "        max_rank : int, optional\n",
      "            Maximum rank for the Broyden matrix.\n",
      "            Default is infinity (ie., no rank reduction).\n",
      "        iter : int, optional\n",
      "            Number of iterations to make. If omitted (default), make as many\n",
      "            as required to meet tolerances.\n",
      "        verbose : bool, optional\n",
      "            Print status to stdout on every iteration.\n",
      "        maxiter : int, optional\n",
      "            Maximum number of iterations to make. If more are needed to\n",
      "            meet convergence, `NoConvergence` is raised.\n",
      "        f_tol : float, optional\n",
      "            Absolute tolerance (in max-norm) for the residual.\n",
      "            If omitted, default is 6e-6.\n",
      "        f_rtol : float, optional\n",
      "            Relative tolerance for the residual. If omitted, not used.\n",
      "        x_tol : float, optional\n",
      "            Absolute minimum step size, as determined from the Jacobian\n",
      "            approximation. If the step size is smaller than this, optimization\n",
      "            is terminated as successful. If omitted, not used.\n",
      "        x_rtol : float, optional\n",
      "            Relative minimum step size. If omitted, not used.\n",
      "        tol_norm : function(vector) -> scalar, optional\n",
      "            Norm to use in convergence check. Default is the maximum norm.\n",
      "        line_search : {None, 'armijo' (default), 'wolfe'}, optional\n",
      "            Which type of a line search to use to determine the step size in the\n",
      "            direction given by the Jacobian approximation. Defaults to 'armijo'.\n",
      "        callback : function, optional\n",
      "            Optional callback function. It is called on every iteration as\n",
      "            ``callback(x, f)`` where `x` is the current solution and `f`\n",
      "            the corresponding residual.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        sol : ndarray\n",
      "            An array (of similar array type as `x0`) containing the final solution.\n",
      "        \n",
      "        Raises\n",
      "        ------\n",
      "        NoConvergence\n",
      "            When a solution was not found.\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        This algorithm implements the inverse Jacobian Quasi-Newton update\n",
      "        \n",
      "        .. math:: H_+ = H + (dx - H df) df^\\dagger / ( df^\\dagger df)\n",
      "        \n",
      "        corresponding to Broyden's second method.\n",
      "        \n",
      "        References\n",
      "        ----------\n",
      "        .. [1] B.A. van der Rotten, PhD thesis,\n",
      "           \"A limited memory Broyden method to solve high-dimensional\n",
      "           systems of nonlinear equations\". Mathematisch Instituut,\n",
      "           Universiteit Leiden, The Netherlands (2003).\n",
      "        \n",
      "           http://www.math.leidenuniv.nl/scripties/Rotten.pdf\n",
      "    \n",
      "    brute(func, ranges, args=(), Ns=20, full_output=0, finish=<function fmin at 0x111316840>, disp=False)\n",
      "        Minimize a function over a given range by brute force.\n",
      "        \n",
      "        Uses the \"brute force\" method, i.e. computes the function's value\n",
      "        at each point of a multidimensional grid of points, to find the global\n",
      "        minimum of the function.\n",
      "        \n",
      "        The function is evaluated everywhere in the range with the datatype of the\n",
      "        first call to the function, as enforced by the ``vectorize`` NumPy\n",
      "        function.  The value and type of the function evaluation returned when\n",
      "        ``full_output=True`` are affected in addition by the ``finish`` argument\n",
      "        (see Notes).\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        func : callable\n",
      "            The objective function to be minimized. Must be in the\n",
      "            form ``f(x, *args)``, where ``x`` is the argument in\n",
      "            the form of a 1-D array and ``args`` is a tuple of any\n",
      "            additional fixed parameters needed to completely specify\n",
      "            the function.\n",
      "        ranges : tuple\n",
      "            Each component of the `ranges` tuple must be either a\n",
      "            \"slice object\" or a range tuple of the form ``(low, high)``.\n",
      "            The program uses these to create the grid of points on which\n",
      "            the objective function will be computed. See `Note 2` for\n",
      "            more detail.\n",
      "        args : tuple, optional\n",
      "            Any additional fixed parameters needed to completely specify\n",
      "            the function.\n",
      "        Ns : int, optional\n",
      "            Number of grid points along the axes, if not otherwise\n",
      "            specified. See `Note2`.\n",
      "        full_output : bool, optional\n",
      "            If True, return the evaluation grid and the objective function's\n",
      "            values on it.\n",
      "        finish : callable, optional\n",
      "            An optimization function that is called with the result of brute force\n",
      "            minimization as initial guess.  `finish` should take `func` and\n",
      "            the initial guess as positional arguments, and take `args` as\n",
      "            keyword arguments.  It may additionally take `full_output`\n",
      "            and/or `disp` as keyword arguments.  Use None if no \"polishing\"\n",
      "            function is to be used. See Notes for more details.\n",
      "        disp : bool, optional\n",
      "            Set to True to print convergence messages.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        x0 : ndarray\n",
      "            A 1-D array containing the coordinates of a point at which the\n",
      "            objective function had its minimum value. (See `Note 1` for\n",
      "            which point is returned.)\n",
      "        fval : float\n",
      "            Function value at the point `x0`. (Returned when `full_output` is\n",
      "            True.)\n",
      "        grid : tuple\n",
      "            Representation of the evaluation grid.  It has the same\n",
      "            length as `x0`. (Returned when `full_output` is True.)\n",
      "        Jout : ndarray\n",
      "            Function values at each point of the evaluation\n",
      "            grid, `i.e.`, ``Jout = func(*grid)``. (Returned\n",
      "            when `full_output` is True.)\n",
      "        \n",
      "        See Also\n",
      "        --------\n",
      "        basinhopping, differential_evolution\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        *Note 1*: The program finds the gridpoint at which the lowest value\n",
      "        of the objective function occurs.  If `finish` is None, that is the\n",
      "        point returned.  When the global minimum occurs within (or not very far\n",
      "        outside) the grid's boundaries, and the grid is fine enough, that\n",
      "        point will be in the neighborhood of the gobal minimum.\n",
      "        \n",
      "        However, users often employ some other optimization program to\n",
      "        \"polish\" the gridpoint values, `i.e.`, to seek a more precise\n",
      "        (local) minimum near `brute's` best gridpoint.\n",
      "        The `brute` function's `finish` option provides a convenient way to do\n",
      "        that.  Any polishing program used must take `brute's` output as its\n",
      "        initial guess as a positional argument, and take `brute's` input values\n",
      "        for `args` as keyword arguments, otherwise an error will be raised.\n",
      "        It may additionally take `full_output` and/or `disp` as keyword arguments.\n",
      "        \n",
      "        `brute` assumes that the `finish` function returns either an\n",
      "        `OptimizeResult` object or a tuple in the form:\n",
      "        ``(xmin, Jmin, ... , statuscode)``, where ``xmin`` is the minimizing\n",
      "        value of the argument, ``Jmin`` is the minimum value of the objective\n",
      "        function, \"...\" may be some other returned values (which are not used\n",
      "        by `brute`), and ``statuscode`` is the status code of the `finish` program.\n",
      "        \n",
      "        Note that when `finish` is not None, the values returned are those\n",
      "        of the `finish` program, *not* the gridpoint ones.  Consequently,\n",
      "        while `brute` confines its search to the input grid points,\n",
      "        the `finish` program's results usually will not coincide with any\n",
      "        gridpoint, and may fall outside the grid's boundary.\n",
      "        \n",
      "        *Note 2*: The grid of points is a `numpy.mgrid` object.\n",
      "        For `brute` the `ranges` and `Ns` inputs have the following effect.\n",
      "        Each component of the `ranges` tuple can be either a slice object or a\n",
      "        two-tuple giving a range of values, such as (0, 5).  If the component is a\n",
      "        slice object, `brute` uses it directly.  If the component is a two-tuple\n",
      "        range, `brute` internally converts it to a slice object that interpolates\n",
      "        `Ns` points from its low-value to its high-value, inclusive.\n",
      "        \n",
      "        Examples\n",
      "        --------\n",
      "        We illustrate the use of `brute` to seek the global minimum of a function\n",
      "        of two variables that is given as the sum of a positive-definite\n",
      "        quadratic and two deep \"Gaussian-shaped\" craters.  Specifically, define\n",
      "        the objective function `f` as the sum of three other functions,\n",
      "        ``f = f1 + f2 + f3``.  We suppose each of these has a signature\n",
      "        ``(z, *params)``, where ``z = (x, y)``,  and ``params`` and the functions\n",
      "        are as defined below.\n",
      "        \n",
      "        >>> params = (2, 3, 7, 8, 9, 10, 44, -1, 2, 26, 1, -2, 0.5)\n",
      "        >>> def f1(z, *params):\n",
      "        ...     x, y = z\n",
      "        ...     a, b, c, d, e, f, g, h, i, j, k, l, scale = params\n",
      "        ...     return (a * x**2 + b * x * y + c * y**2 + d*x + e*y + f)\n",
      "        \n",
      "        >>> def f2(z, *params):\n",
      "        ...     x, y = z\n",
      "        ...     a, b, c, d, e, f, g, h, i, j, k, l, scale = params\n",
      "        ...     return (-g*np.exp(-((x-h)**2 + (y-i)**2) / scale))\n",
      "        \n",
      "        >>> def f3(z, *params):\n",
      "        ...     x, y = z\n",
      "        ...     a, b, c, d, e, f, g, h, i, j, k, l, scale = params\n",
      "        ...     return (-j*np.exp(-((x-k)**2 + (y-l)**2) / scale))\n",
      "        \n",
      "        >>> def f(z, *params):\n",
      "        ...     return f1(z, *params) + f2(z, *params) + f3(z, *params)\n",
      "        \n",
      "        Thus, the objective function may have local minima near the minimum\n",
      "        of each of the three functions of which it is composed.  To\n",
      "        use `fmin` to polish its gridpoint result, we may then continue as\n",
      "        follows:\n",
      "        \n",
      "        >>> rranges = (slice(-4, 4, 0.25), slice(-4, 4, 0.25))\n",
      "        >>> from scipy import optimize\n",
      "        >>> resbrute = optimize.brute(f, rranges, args=params, full_output=True,\n",
      "        ...                           finish=optimize.fmin)\n",
      "        >>> resbrute[0]  # global minimum\n",
      "        array([-1.05665192,  1.80834843])\n",
      "        >>> resbrute[1]  # function value at global minimum\n",
      "        -3.4085818767\n",
      "        \n",
      "        Note that if `finish` had been set to None, we would have gotten the\n",
      "        gridpoint [-1.0 1.75] where the rounded function value is -2.892.\n",
      "    \n",
      "    check_grad(func, grad, x0, *args, **kwargs)\n",
      "        Check the correctness of a gradient function by comparing it against a\n",
      "        (forward) finite-difference approximation of the gradient.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        func : callable ``func(x0, *args)``\n",
      "            Function whose derivative is to be checked.\n",
      "        grad : callable ``grad(x0, *args)``\n",
      "            Gradient of `func`.\n",
      "        x0 : ndarray\n",
      "            Points to check `grad` against forward difference approximation of grad\n",
      "            using `func`.\n",
      "        args : \\*args, optional\n",
      "            Extra arguments passed to `func` and `grad`.\n",
      "        epsilon : float, optional\n",
      "            Step size used for the finite difference approximation. It defaults to\n",
      "            ``sqrt(numpy.finfo(float).eps)``, which is approximately 1.49e-08.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        err : float\n",
      "            The square root of the sum of squares (i.e. the 2-norm) of the\n",
      "            difference between ``grad(x0, *args)`` and the finite difference\n",
      "            approximation of `grad` using func at the points `x0`.\n",
      "        \n",
      "        See Also\n",
      "        --------\n",
      "        approx_fprime\n",
      "        \n",
      "        Examples\n",
      "        --------\n",
      "        >>> def func(x):\n",
      "        ...     return x[0]**2 - 0.5 * x[1]**3\n",
      "        >>> def grad(x):\n",
      "        ...     return [2 * x[0], -1.5 * x[1]**2]\n",
      "        >>> from scipy.optimize import check_grad\n",
      "        >>> check_grad(func, grad, [1.5, -1.5])\n",
      "        2.9802322387695312e-08\n",
      "    \n",
      "    curve_fit(f, xdata, ydata, p0=None, sigma=None, absolute_sigma=False, check_finite=True, **kw)\n",
      "        Use non-linear least squares to fit a function, f, to data.\n",
      "        \n",
      "        Assumes ``ydata = f(xdata, *params) + eps``\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        f : callable\n",
      "            The model function, f(x, ...).  It must take the independent\n",
      "            variable as the first argument and the parameters to fit as\n",
      "            separate remaining arguments.\n",
      "        xdata : An M-length sequence or an (k,M)-shaped array\n",
      "            for functions with k predictors.\n",
      "            The independent variable where the data is measured.\n",
      "        ydata : M-length sequence\n",
      "            The dependent data --- nominally f(xdata, ...)\n",
      "        p0 : None, scalar, or N-length sequence, optional\n",
      "            Initial guess for the parameters.  If None, then the initial\n",
      "            values will all be 1 (if the number of parameters for the function\n",
      "            can be determined using introspection, otherwise a ValueError\n",
      "            is raised).\n",
      "        sigma : None or M-length sequence, optional\n",
      "            If not None, the uncertainties in the ydata array. These are used as\n",
      "            weights in the least-squares problem\n",
      "            i.e. minimising ``np.sum( ((f(xdata, *popt) - ydata) / sigma)**2 )``\n",
      "            If None, the uncertainties are assumed to be 1.\n",
      "        absolute_sigma : bool, optional\n",
      "            If False, `sigma` denotes relative weights of the data points.\n",
      "            The returned covariance matrix `pcov` is based on *estimated*\n",
      "            errors in the data, and is not affected by the overall\n",
      "            magnitude of the values in `sigma`. Only the relative\n",
      "            magnitudes of the `sigma` values matter.\n",
      "        \n",
      "            If True, `sigma` describes one standard deviation errors of\n",
      "            the input data points. The estimated covariance in `pcov` is\n",
      "            based on these values.\n",
      "        check_finite : bool, optional\n",
      "            If True, check that the input arrays do not contain nans of infs,\n",
      "            and raise a ValueError if they do. Setting this parameter to\n",
      "            False may silently produce nonsensical results if the input arrays\n",
      "            do contain nans.\n",
      "            Default is True.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        popt : array\n",
      "            Optimal values for the parameters so that the sum of the squared error\n",
      "            of ``f(xdata, *popt) - ydata`` is minimized\n",
      "        pcov : 2d array\n",
      "            The estimated covariance of popt. The diagonals provide the variance\n",
      "            of the parameter estimate. To compute one standard deviation errors\n",
      "            on the parameters use ``perr = np.sqrt(np.diag(pcov))``.\n",
      "        \n",
      "            How the `sigma` parameter affects the estimated covariance\n",
      "            depends on `absolute_sigma` argument, as described above.\n",
      "        \n",
      "        Raises\n",
      "        ------\n",
      "        OptimizeWarning\n",
      "            if covariance of the parameters can not be estimated.\n",
      "        \n",
      "        ValueError\n",
      "            if ydata and xdata contain NaNs.\n",
      "        \n",
      "        See Also\n",
      "        --------\n",
      "        leastsq\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        The algorithm uses the Levenberg-Marquardt algorithm through `leastsq`.\n",
      "        Additional keyword arguments are passed directly to that algorithm.\n",
      "        \n",
      "        Examples\n",
      "        --------\n",
      "        >>> import numpy as np\n",
      "        >>> from scipy.optimize import curve_fit\n",
      "        >>> def func(x, a, b, c):\n",
      "        ...     return a * np.exp(-b * x) + c\n",
      "        \n",
      "        >>> xdata = np.linspace(0, 4, 50)\n",
      "        >>> y = func(xdata, 2.5, 1.3, 0.5)\n",
      "        >>> ydata = y + 0.2 * np.random.normal(size=len(xdata))\n",
      "        \n",
      "        >>> popt, pcov = curve_fit(func, xdata, ydata)\n",
      "    \n",
      "    diagbroyden(F, xin, iter=None, alpha=None, verbose=False, maxiter=None, f_tol=None, f_rtol=None, x_tol=None, x_rtol=None, tol_norm=None, line_search='armijo', callback=None, **kw)\n",
      "        Find a root of a function, using diagonal Broyden Jacobian approximation.\n",
      "        \n",
      "        The Jacobian approximation is derived from previous iterations, by\n",
      "        retaining only the diagonal of Broyden matrices.\n",
      "        \n",
      "        .. warning::\n",
      "        \n",
      "           This algorithm may be useful for specific problems, but whether\n",
      "           it will work may depend strongly on the problem.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        F : function(x) -> f\n",
      "            Function whose root to find; should take and return an array-like\n",
      "            object.\n",
      "        x0 : array_like\n",
      "            Initial guess for the solution\n",
      "        alpha : float, optional\n",
      "            Initial guess for the Jacobian is (-1/alpha).\n",
      "        iter : int, optional\n",
      "            Number of iterations to make. If omitted (default), make as many\n",
      "            as required to meet tolerances.\n",
      "        verbose : bool, optional\n",
      "            Print status to stdout on every iteration.\n",
      "        maxiter : int, optional\n",
      "            Maximum number of iterations to make. If more are needed to\n",
      "            meet convergence, `NoConvergence` is raised.\n",
      "        f_tol : float, optional\n",
      "            Absolute tolerance (in max-norm) for the residual.\n",
      "            If omitted, default is 6e-6.\n",
      "        f_rtol : float, optional\n",
      "            Relative tolerance for the residual. If omitted, not used.\n",
      "        x_tol : float, optional\n",
      "            Absolute minimum step size, as determined from the Jacobian\n",
      "            approximation. If the step size is smaller than this, optimization\n",
      "            is terminated as successful. If omitted, not used.\n",
      "        x_rtol : float, optional\n",
      "            Relative minimum step size. If omitted, not used.\n",
      "        tol_norm : function(vector) -> scalar, optional\n",
      "            Norm to use in convergence check. Default is the maximum norm.\n",
      "        line_search : {None, 'armijo' (default), 'wolfe'}, optional\n",
      "            Which type of a line search to use to determine the step size in the\n",
      "            direction given by the Jacobian approximation. Defaults to 'armijo'.\n",
      "        callback : function, optional\n",
      "            Optional callback function. It is called on every iteration as\n",
      "            ``callback(x, f)`` where `x` is the current solution and `f`\n",
      "            the corresponding residual.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        sol : ndarray\n",
      "            An array (of similar array type as `x0`) containing the final solution.\n",
      "        \n",
      "        Raises\n",
      "        ------\n",
      "        NoConvergence\n",
      "            When a solution was not found.\n",
      "    \n",
      "    differential_evolution(func, bounds, args=(), strategy='best1bin', maxiter=None, popsize=15, tol=0.01, mutation=(0.5, 1), recombination=0.7, seed=None, callback=None, disp=False, polish=True, init='latinhypercube')\n",
      "        Finds the global minimum of a multivariate function.\n",
      "        Differential Evolution is stochastic in nature (does not use gradient\n",
      "        methods) to find the minimium, and can search large areas of candidate\n",
      "        space, but often requires larger numbers of function evaluations than\n",
      "        conventional gradient based techniques.\n",
      "        \n",
      "        The algorithm is due to Storn and Price [1]_.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        func : callable\n",
      "            The objective function to be minimized.  Must be in the form\n",
      "            ``f(x, *args)``, where ``x`` is the argument in the form of a 1-D array\n",
      "            and ``args`` is a  tuple of any additional fixed parameters needed to\n",
      "            completely specify the function.\n",
      "        bounds : sequence\n",
      "            Bounds for variables.  ``(min, max)`` pairs for each element in ``x``,\n",
      "            defining the lower and upper bounds for the optimizing argument of\n",
      "            `func`. It is required to have ``len(bounds) == len(x)``.\n",
      "            ``len(bounds)`` is used to determine the number of parameters in ``x``.\n",
      "        args : tuple, optional\n",
      "            Any additional fixed parameters needed to\n",
      "            completely specify the objective function.\n",
      "        strategy : str, optional\n",
      "            The differential evolution strategy to use. Should be one of:\n",
      "        \n",
      "                - 'best1bin'\n",
      "                - 'best1exp'\n",
      "                - 'rand1exp'\n",
      "                - 'randtobest1exp'\n",
      "                - 'best2exp'\n",
      "                - 'rand2exp'\n",
      "                - 'randtobest1bin'\n",
      "                - 'best2bin'\n",
      "                - 'rand2bin'\n",
      "                - 'rand1bin'\n",
      "        \n",
      "            The default is 'best1bin'.\n",
      "        maxiter : int, optional\n",
      "            The maximum number of times the entire population is evolved.\n",
      "            The maximum number of function evaluations is:\n",
      "            ``maxiter * popsize * len(x)``\n",
      "        popsize : int, optional\n",
      "            A multiplier for setting the total population size.  The population has\n",
      "            ``popsize * len(x)`` individuals.\n",
      "        tol : float, optional\n",
      "            When the mean of the population energies, multiplied by tol,\n",
      "            divided by the standard deviation of the population energies\n",
      "            is greater than 1 the solving process terminates:\n",
      "            ``convergence = mean(pop) * tol / stdev(pop) > 1``\n",
      "        mutation : float or tuple(float, float), optional\n",
      "            The mutation constant.\n",
      "            If specified as a float it should be in the range [0, 2].\n",
      "            If specified as a tuple ``(min, max)`` dithering is employed. Dithering\n",
      "            randomly changes the mutation constant on a generation by generation\n",
      "            basis. The mutation constant for that generation is taken from\n",
      "            ``U[min, max)``. Dithering can help speed convergence significantly.\n",
      "            Increasing the mutation constant increases the search radius, but will\n",
      "            slow down convergence.\n",
      "        recombination : float, optional\n",
      "            The recombination constant, should be in the range [0, 1]. Increasing\n",
      "            this value allows a larger number of mutants to progress into the next\n",
      "            generation, but at the risk of population stability.\n",
      "        seed : int or `np.random.RandomState`, optional\n",
      "            If `seed` is not specified the `np.RandomState` singleton is used.\n",
      "            If `seed` is an int, a new `np.random.RandomState` instance is used,\n",
      "            seeded with seed.\n",
      "            If `seed` is already a `np.random.RandomState instance`, then that\n",
      "            `np.random.RandomState` instance is used.\n",
      "            Specify `seed` for repeatable minimizations.\n",
      "        disp : bool, optional\n",
      "            Display status messages\n",
      "        callback : callable, `callback(xk, convergence=val)`, optional\n",
      "            A function to follow the progress of the minimization. ``xk`` is\n",
      "            the current value of ``x0``. ``val`` represents the fractional\n",
      "            value of the population convergence.  When ``val`` is greater than one\n",
      "            the function halts. If callback returns `True`, then the minimization\n",
      "            is halted (any polishing is still carried out).\n",
      "        polish : bool, optional\n",
      "            If True (default), then `scipy.optimize.minimize` with the `L-BFGS-B`\n",
      "            method is used to polish the best population member at the end, which\n",
      "            can improve the minimization slightly.\n",
      "        init : string, optional\n",
      "            Specify how the population initialization is performed. Should be\n",
      "            one of:\n",
      "        \n",
      "                - 'latinhypercube'\n",
      "                - 'random'\n",
      "        \n",
      "            The default is 'latinhypercube'. Latin Hypercube sampling tries to\n",
      "            maximize coverage of the available parameter space. 'random' initializes\n",
      "            the population randomly - this has the drawback that clustering can\n",
      "            occur, preventing the whole of parameter space being covered.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        res : OptimizeResult\n",
      "            The optimization result represented as a `OptimizeResult` object.\n",
      "            Important attributes are: ``x`` the solution array, ``success`` a\n",
      "            Boolean flag indicating if the optimizer exited successfully and\n",
      "            ``message`` which describes the cause of the termination. See\n",
      "            `OptimizeResult` for a description of other attributes. If `polish`\n",
      "            was employed, then OptimizeResult also contains the `jac` attribute.\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        Differential evolution is a stochastic population based method that is\n",
      "        useful for global optimization problems. At each pass through the population\n",
      "        the algorithm mutates each candidate solution by mixing with other candidate\n",
      "        solutions to create a trial candidate. There are several strategies [2]_ for\n",
      "        creating trial candidates, which suit some problems more than others. The\n",
      "        'best1bin' strategy is a good starting point for many systems. In this\n",
      "        strategy two members of the population are randomly chosen. Their difference\n",
      "        is used to mutate the best member (the `best` in `best1bin`), :math:`b_0`,\n",
      "        so far:\n",
      "        \n",
      "        .. math::\n",
      "        \n",
      "            b' = b_0 + mutation * (population[rand0] - population[rand1])\n",
      "        \n",
      "        A trial vector is then constructed. Starting with a randomly chosen 'i'th\n",
      "        parameter the trial is sequentially filled (in modulo) with parameters from\n",
      "        `b'` or the original candidate. The choice of whether to use `b'` or the\n",
      "        original candidate is made with a binomial distribution (the 'bin' in\n",
      "        'best1bin') - a random number in [0, 1) is generated.  If this number is\n",
      "        less than the `recombination` constant then the parameter is loaded from\n",
      "        `b'`, otherwise it is loaded from the original candidate.  The final\n",
      "        parameter is always loaded from `b'`.  Once the trial candidate is built\n",
      "        its fitness is assessed. If the trial is better than the original candidate\n",
      "        then it takes its place. If it is also better than the best overall\n",
      "        candidate it also replaces that.\n",
      "        To improve your chances of finding a global minimum use higher `popsize`\n",
      "        values, with higher `mutation` and (dithering), but lower `recombination`\n",
      "        values. This has the effect of widening the search radius, but slowing\n",
      "        convergence.\n",
      "        \n",
      "        .. versionadded:: 0.15.0\n",
      "        \n",
      "        Examples\n",
      "        --------\n",
      "        Let us consider the problem of minimizing the Rosenbrock function. This\n",
      "        function is implemented in `rosen` in `scipy.optimize`.\n",
      "        \n",
      "        >>> from scipy.optimize import rosen, differential_evolution\n",
      "        >>> bounds = [(0,2), (0, 2), (0, 2), (0, 2), (0, 2)]\n",
      "        >>> result = differential_evolution(rosen, bounds)\n",
      "        >>> result.x, result.fun\n",
      "        (array([1., 1., 1., 1., 1.]), 1.9216496320061384e-19)\n",
      "        \n",
      "        Next find the minimum of the Ackley function\n",
      "        (http://en.wikipedia.org/wiki/Test_functions_for_optimization).\n",
      "        \n",
      "        >>> from scipy.optimize import differential_evolution\n",
      "        >>> import numpy as np\n",
      "        >>> def ackley(x):\n",
      "        ...     arg1 = -0.2 * np.sqrt(0.5 * (x[0] ** 2 + x[1] ** 2))\n",
      "        ...     arg2 = 0.5 * (np.cos(2. * np.pi * x[0]) + np.cos(2. * np.pi * x[1]))\n",
      "        ...     return -20. * np.exp(arg1) - np.exp(arg2) + 20. + np.e\n",
      "        >>> bounds = [(-5, 5), (-5, 5)]\n",
      "        >>> result = differential_evolution(ackley, bounds)\n",
      "        >>> result.x, result.fun\n",
      "        (array([ 0.,  0.]), 4.4408920985006262e-16)\n",
      "        \n",
      "        References\n",
      "        ----------\n",
      "        .. [1] Storn, R and Price, K, Differential Evolution - a Simple and\n",
      "               Efficient Heuristic for Global Optimization over Continuous Spaces,\n",
      "               Journal of Global Optimization, 1997, 11, 341 - 359.\n",
      "        .. [2] http://www1.icsi.berkeley.edu/~storn/code.html\n",
      "        .. [3] http://en.wikipedia.org/wiki/Differential_evolution\n",
      "    \n",
      "    excitingmixing(F, xin, iter=None, alpha=None, alphamax=1.0, verbose=False, maxiter=None, f_tol=None, f_rtol=None, x_tol=None, x_rtol=None, tol_norm=None, line_search='armijo', callback=None, **kw)\n",
      "        Find a root of a function, using a tuned diagonal Jacobian approximation.\n",
      "        \n",
      "        The Jacobian matrix is diagonal and is tuned on each iteration.\n",
      "        \n",
      "        .. warning::\n",
      "        \n",
      "           This algorithm may be useful for specific problems, but whether\n",
      "           it will work may depend strongly on the problem.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        F : function(x) -> f\n",
      "            Function whose root to find; should take and return an array-like\n",
      "            object.\n",
      "        x0 : array_like\n",
      "            Initial guess for the solution\n",
      "        alpha : float, optional\n",
      "            Initial Jacobian approximation is (-1/alpha).\n",
      "        alphamax : float, optional\n",
      "            The entries of the diagonal Jacobian are kept in the range\n",
      "            ``[alpha, alphamax]``.\n",
      "        iter : int, optional\n",
      "            Number of iterations to make. If omitted (default), make as many\n",
      "            as required to meet tolerances.\n",
      "        verbose : bool, optional\n",
      "            Print status to stdout on every iteration.\n",
      "        maxiter : int, optional\n",
      "            Maximum number of iterations to make. If more are needed to\n",
      "            meet convergence, `NoConvergence` is raised.\n",
      "        f_tol : float, optional\n",
      "            Absolute tolerance (in max-norm) for the residual.\n",
      "            If omitted, default is 6e-6.\n",
      "        f_rtol : float, optional\n",
      "            Relative tolerance for the residual. If omitted, not used.\n",
      "        x_tol : float, optional\n",
      "            Absolute minimum step size, as determined from the Jacobian\n",
      "            approximation. If the step size is smaller than this, optimization\n",
      "            is terminated as successful. If omitted, not used.\n",
      "        x_rtol : float, optional\n",
      "            Relative minimum step size. If omitted, not used.\n",
      "        tol_norm : function(vector) -> scalar, optional\n",
      "            Norm to use in convergence check. Default is the maximum norm.\n",
      "        line_search : {None, 'armijo' (default), 'wolfe'}, optional\n",
      "            Which type of a line search to use to determine the step size in the\n",
      "            direction given by the Jacobian approximation. Defaults to 'armijo'.\n",
      "        callback : function, optional\n",
      "            Optional callback function. It is called on every iteration as\n",
      "            ``callback(x, f)`` where `x` is the current solution and `f`\n",
      "            the corresponding residual.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        sol : ndarray\n",
      "            An array (of similar array type as `x0`) containing the final solution.\n",
      "        \n",
      "        Raises\n",
      "        ------\n",
      "        NoConvergence\n",
      "            When a solution was not found.\n",
      "    \n",
      "    fixed_point(func, x0, args=(), xtol=1e-08, maxiter=500)\n",
      "        Find a fixed point of the function.\n",
      "        \n",
      "        Given a function of one or more variables and a starting point, find a\n",
      "        fixed-point of the function: i.e. where ``func(x0) == x0``.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        func : function\n",
      "            Function to evaluate.\n",
      "        x0 : array_like\n",
      "            Fixed point of function.\n",
      "        args : tuple, optional\n",
      "            Extra arguments to `func`.\n",
      "        xtol : float, optional\n",
      "            Convergence tolerance, defaults to 1e-08.\n",
      "        maxiter : int, optional\n",
      "            Maximum number of iterations, defaults to 500.\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        Uses Steffensen's Method using Aitken's ``Del^2`` convergence acceleration.\n",
      "        See Burden, Faires, \"Numerical Analysis\", 5th edition, pg. 80\n",
      "        \n",
      "        Examples\n",
      "        --------\n",
      "        >>> from scipy import optimize\n",
      "        >>> def func(x, c1, c2):\n",
      "        ...    return np.sqrt(c1/(x+c2))\n",
      "        >>> c1 = np.array([10,12.])\n",
      "        >>> c2 = np.array([3, 5.])\n",
      "        >>> optimize.fixed_point(func, [1.2, 1.3], args=(c1,c2))\n",
      "        array([ 1.4920333 ,  1.37228132])\n",
      "    \n",
      "    fmin(func, x0, args=(), xtol=0.0001, ftol=0.0001, maxiter=None, maxfun=None, full_output=0, disp=1, retall=0, callback=None)\n",
      "        Minimize a function using the downhill simplex algorithm.\n",
      "        \n",
      "        This algorithm only uses function values, not derivatives or second\n",
      "        derivatives.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        func : callable func(x,*args)\n",
      "            The objective function to be minimized.\n",
      "        x0 : ndarray\n",
      "            Initial guess.\n",
      "        args : tuple, optional\n",
      "            Extra arguments passed to func, i.e. ``f(x,*args)``.\n",
      "        callback : callable, optional\n",
      "            Called after each iteration, as callback(xk), where xk is the\n",
      "            current parameter vector.\n",
      "        xtol : float, optional\n",
      "            Relative error in xopt acceptable for convergence.\n",
      "        ftol : number, optional\n",
      "            Relative error in func(xopt) acceptable for convergence.\n",
      "        maxiter : int, optional\n",
      "            Maximum number of iterations to perform.\n",
      "        maxfun : number, optional\n",
      "            Maximum number of function evaluations to make.\n",
      "        full_output : bool, optional\n",
      "            Set to True if fopt and warnflag outputs are desired.\n",
      "        disp : bool, optional\n",
      "            Set to True to print convergence messages.\n",
      "        retall : bool, optional\n",
      "            Set to True to return list of solutions at each iteration.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        xopt : ndarray\n",
      "            Parameter that minimizes function.\n",
      "        fopt : float\n",
      "            Value of function at minimum: ``fopt = func(xopt)``.\n",
      "        iter : int\n",
      "            Number of iterations performed.\n",
      "        funcalls : int\n",
      "            Number of function calls made.\n",
      "        warnflag : int\n",
      "            1 : Maximum number of function evaluations made.\n",
      "            2 : Maximum number of iterations reached.\n",
      "        allvecs : list\n",
      "            Solution at each iteration.\n",
      "        \n",
      "        See also\n",
      "        --------\n",
      "        minimize: Interface to minimization algorithms for multivariate\n",
      "            functions. See the 'Nelder-Mead' `method` in particular.\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        Uses a Nelder-Mead simplex algorithm to find the minimum of function of\n",
      "        one or more variables.\n",
      "        \n",
      "        This algorithm has a long history of successful use in applications.\n",
      "        But it will usually be slower than an algorithm that uses first or\n",
      "        second derivative information. In practice it can have poor\n",
      "        performance in high-dimensional problems and is not robust to\n",
      "        minimizing complicated functions. Additionally, there currently is no\n",
      "        complete theory describing when the algorithm will successfully\n",
      "        converge to the minimum, or how fast it will if it does.\n",
      "        \n",
      "        References\n",
      "        ----------\n",
      "        .. [1] Nelder, J.A. and Mead, R. (1965), \"A simplex method for function\n",
      "               minimization\", The Computer Journal, 7, pp. 308-313\n",
      "        \n",
      "        .. [2] Wright, M.H. (1996), \"Direct Search Methods: Once Scorned, Now\n",
      "               Respectable\", in Numerical Analysis 1995, Proceedings of the\n",
      "               1995 Dundee Biennial Conference in Numerical Analysis, D.F.\n",
      "               Griffiths and G.A. Watson (Eds.), Addison Wesley Longman,\n",
      "               Harlow, UK, pp. 191-208.\n",
      "    \n",
      "    fmin_bfgs(f, x0, fprime=None, args=(), gtol=1e-05, norm=inf, epsilon=1.4901161193847656e-08, maxiter=None, full_output=0, disp=1, retall=0, callback=None)\n",
      "        Minimize a function using the BFGS algorithm.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        f : callable f(x,*args)\n",
      "            Objective function to be minimized.\n",
      "        x0 : ndarray\n",
      "            Initial guess.\n",
      "        fprime : callable f'(x,*args), optional\n",
      "            Gradient of f.\n",
      "        args : tuple, optional\n",
      "            Extra arguments passed to f and fprime.\n",
      "        gtol : float, optional\n",
      "            Gradient norm must be less than gtol before successful termination.\n",
      "        norm : float, optional\n",
      "            Order of norm (Inf is max, -Inf is min)\n",
      "        epsilon : int or ndarray, optional\n",
      "            If fprime is approximated, use this value for the step size.\n",
      "        callback : callable, optional\n",
      "            An optional user-supplied function to call after each\n",
      "            iteration.  Called as callback(xk), where xk is the\n",
      "            current parameter vector.\n",
      "        maxiter : int, optional\n",
      "            Maximum number of iterations to perform.\n",
      "        full_output : bool, optional\n",
      "            If True,return fopt, func_calls, grad_calls, and warnflag\n",
      "            in addition to xopt.\n",
      "        disp : bool, optional\n",
      "            Print convergence message if True.\n",
      "        retall : bool, optional\n",
      "            Return a list of results at each iteration if True.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        xopt : ndarray\n",
      "            Parameters which minimize f, i.e. f(xopt) == fopt.\n",
      "        fopt : float\n",
      "            Minimum value.\n",
      "        gopt : ndarray\n",
      "            Value of gradient at minimum, f'(xopt), which should be near 0.\n",
      "        Bopt : ndarray\n",
      "            Value of 1/f''(xopt), i.e. the inverse hessian matrix.\n",
      "        func_calls : int\n",
      "            Number of function_calls made.\n",
      "        grad_calls : int\n",
      "            Number of gradient calls made.\n",
      "        warnflag : integer\n",
      "            1 : Maximum number of iterations exceeded.\n",
      "            2 : Gradient and/or function calls not changing.\n",
      "        allvecs  :  list\n",
      "            `OptimizeResult` at each iteration.  Only returned if retall is True.\n",
      "        \n",
      "        See also\n",
      "        --------\n",
      "        minimize: Interface to minimization algorithms for multivariate\n",
      "            functions. See the 'BFGS' `method` in particular.\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        Optimize the function, f, whose gradient is given by fprime\n",
      "        using the quasi-Newton method of Broyden, Fletcher, Goldfarb,\n",
      "        and Shanno (BFGS)\n",
      "        \n",
      "        References\n",
      "        ----------\n",
      "        Wright, and Nocedal 'Numerical Optimization', 1999, pg. 198.\n",
      "    \n",
      "    fmin_cg(f, x0, fprime=None, args=(), gtol=1e-05, norm=inf, epsilon=1.4901161193847656e-08, maxiter=None, full_output=0, disp=1, retall=0, callback=None)\n",
      "        Minimize a function using a nonlinear conjugate gradient algorithm.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        f : callable, ``f(x, *args)``\n",
      "            Objective function to be minimized.  Here `x` must be a 1-D array of\n",
      "            the variables that are to be changed in the search for a minimum, and\n",
      "            `args` are the other (fixed) parameters of `f`.\n",
      "        x0 : ndarray\n",
      "            A user-supplied initial estimate of `xopt`, the optimal value of `x`.\n",
      "            It must be a 1-D array of values.\n",
      "        fprime : callable, ``fprime(x, *args)``, optional\n",
      "            A function that returns the gradient of `f` at `x`. Here `x` and `args`\n",
      "            are as described above for `f`. The returned value must be a 1-D array.\n",
      "            Defaults to None, in which case the gradient is approximated\n",
      "            numerically (see `epsilon`, below).\n",
      "        args : tuple, optional\n",
      "            Parameter values passed to `f` and `fprime`. Must be supplied whenever\n",
      "            additional fixed parameters are needed to completely specify the\n",
      "            functions `f` and `fprime`.\n",
      "        gtol : float, optional\n",
      "            Stop when the norm of the gradient is less than `gtol`.\n",
      "        norm : float, optional\n",
      "            Order to use for the norm of the gradient\n",
      "            (``-np.Inf`` is min, ``np.Inf`` is max).\n",
      "        epsilon : float or ndarray, optional\n",
      "            Step size(s) to use when `fprime` is approximated numerically. Can be a\n",
      "            scalar or a 1-D array.  Defaults to ``sqrt(eps)``, with eps the\n",
      "            floating point machine precision.  Usually ``sqrt(eps)`` is about\n",
      "            1.5e-8.\n",
      "        maxiter : int, optional\n",
      "            Maximum number of iterations to perform. Default is ``200 * len(x0)``.\n",
      "        full_output : bool, optional\n",
      "            If True, return `fopt`, `func_calls`, `grad_calls`, and `warnflag` in\n",
      "            addition to `xopt`.  See the Returns section below for additional\n",
      "            information on optional return values.\n",
      "        disp : bool, optional\n",
      "            If True, return a convergence message, followed by `xopt`.\n",
      "        retall : bool, optional\n",
      "            If True, add to the returned values the results of each iteration.\n",
      "        callback : callable, optional\n",
      "            An optional user-supplied function, called after each iteration.\n",
      "            Called as ``callback(xk)``, where ``xk`` is the current value of `x0`.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        xopt : ndarray\n",
      "            Parameters which minimize f, i.e. ``f(xopt) == fopt``.\n",
      "        fopt : float, optional\n",
      "            Minimum value found, f(xopt).  Only returned if `full_output` is True.\n",
      "        func_calls : int, optional\n",
      "            The number of function_calls made.  Only returned if `full_output`\n",
      "            is True.\n",
      "        grad_calls : int, optional\n",
      "            The number of gradient calls made. Only returned if `full_output` is\n",
      "            True.\n",
      "        warnflag : int, optional\n",
      "            Integer value with warning status, only returned if `full_output` is\n",
      "            True.\n",
      "        \n",
      "            0 : Success.\n",
      "        \n",
      "            1 : The maximum number of iterations was exceeded.\n",
      "        \n",
      "            2 : Gradient and/or function calls were not changing.  May indicate\n",
      "                that precision was lost, i.e., the routine did not converge.\n",
      "        \n",
      "        allvecs : list of ndarray, optional\n",
      "            List of arrays, containing the results at each iteration.\n",
      "            Only returned if `retall` is True.\n",
      "        \n",
      "        See Also\n",
      "        --------\n",
      "        minimize : common interface to all `scipy.optimize` algorithms for\n",
      "                   unconstrained and constrained minimization of multivariate\n",
      "                   functions.  It provides an alternative way to call\n",
      "                   ``fmin_cg``, by specifying ``method='CG'``.\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        This conjugate gradient algorithm is based on that of Polak and Ribiere\n",
      "        [1]_.\n",
      "        \n",
      "        Conjugate gradient methods tend to work better when:\n",
      "        \n",
      "        1. `f` has a unique global minimizing point, and no local minima or\n",
      "           other stationary points,\n",
      "        2. `f` is, at least locally, reasonably well approximated by a\n",
      "           quadratic function of the variables,\n",
      "        3. `f` is continuous and has a continuous gradient,\n",
      "        4. `fprime` is not too large, e.g., has a norm less than 1000,\n",
      "        5. The initial guess, `x0`, is reasonably close to `f` 's global\n",
      "           minimizing point, `xopt`.\n",
      "        \n",
      "        References\n",
      "        ----------\n",
      "        .. [1] Wright & Nocedal, \"Numerical Optimization\", 1999, pp. 120-122.\n",
      "        \n",
      "        Examples\n",
      "        --------\n",
      "        Example 1: seek the minimum value of the expression\n",
      "        ``a*u**2 + b*u*v + c*v**2 + d*u + e*v + f`` for given values\n",
      "        of the parameters and an initial guess ``(u, v) = (0, 0)``.\n",
      "        \n",
      "        >>> args = (2, 3, 7, 8, 9, 10)  # parameter values\n",
      "        >>> def f(x, *args):\n",
      "        ...     u, v = x\n",
      "        ...     a, b, c, d, e, f = args\n",
      "        ...     return a*u**2 + b*u*v + c*v**2 + d*u + e*v + f\n",
      "        >>> def gradf(x, *args):\n",
      "        ...     u, v = x\n",
      "        ...     a, b, c, d, e, f = args\n",
      "        ...     gu = 2*a*u + b*v + d     # u-component of the gradient\n",
      "        ...     gv = b*u + 2*c*v + e     # v-component of the gradient\n",
      "        ...     return np.asarray((gu, gv))\n",
      "        >>> x0 = np.asarray((0, 0))  # Initial guess.\n",
      "        >>> from scipy import optimize\n",
      "        >>> res1 = optimize.fmin_cg(f, x0, fprime=gradf, args=args)\n",
      "        >>> print('res1 = ', res1)\n",
      "        Optimization terminated successfully.\n",
      "                 Current function value: 1.617021\n",
      "                 Iterations: 2\n",
      "                 Function evaluations: 5\n",
      "                 Gradient evaluations: 5\n",
      "        res1 =  [-1.80851064 -0.25531915]\n",
      "        \n",
      "        Example 2: solve the same problem using the `minimize` function.\n",
      "        (This `myopts` dictionary shows all of the available options,\n",
      "        although in practice only non-default values would be needed.\n",
      "        The returned value will be a dictionary.)\n",
      "        \n",
      "        >>> opts = {'maxiter' : None,    # default value.\n",
      "        ...         'disp' : True,    # non-default value.\n",
      "        ...         'gtol' : 1e-5,    # default value.\n",
      "        ...         'norm' : np.inf,  # default value.\n",
      "        ...         'eps' : 1.4901161193847656e-08}  # default value.\n",
      "        >>> res2 = optimize.minimize(f, x0, jac=gradf, args=args,\n",
      "        ...                          method='CG', options=opts)\n",
      "        Optimization terminated successfully.\n",
      "                Current function value: 1.617021\n",
      "                Iterations: 2\n",
      "                Function evaluations: 5\n",
      "                Gradient evaluations: 5\n",
      "        >>> res2.x  # minimum found\n",
      "        array([-1.80851064 -0.25531915])\n",
      "    \n",
      "    fmin_cobyla(func, x0, cons, args=(), consargs=None, rhobeg=1.0, rhoend=0.0001, iprint=1, maxfun=1000, disp=None, catol=0.0002)\n",
      "        Minimize a function using the Constrained Optimization BY Linear\n",
      "        Approximation (COBYLA) method. This method wraps a FORTRAN\n",
      "        implentation of the algorithm.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        func : callable\n",
      "            Function to minimize. In the form func(x, \\*args).\n",
      "        x0 : ndarray\n",
      "            Initial guess.\n",
      "        cons : sequence\n",
      "            Constraint functions; must all be ``>=0`` (a single function\n",
      "            if only 1 constraint). Each function takes the parameters `x`\n",
      "            as its first argument, and it can return either a single number or\n",
      "            an array or list of numbers.\n",
      "        args : tuple, optional\n",
      "            Extra arguments to pass to function.\n",
      "        consargs : tuple, optional\n",
      "            Extra arguments to pass to constraint functions (default of None means\n",
      "            use same extra arguments as those passed to func).\n",
      "            Use ``()`` for no extra arguments.\n",
      "        rhobeg : float, optional\n",
      "            Reasonable initial changes to the variables.\n",
      "        rhoend : float, optional\n",
      "            Final accuracy in the optimization (not precisely guaranteed). This\n",
      "            is a lower bound on the size of the trust region.\n",
      "        iprint : {0, 1, 2, 3}, optional\n",
      "            Controls the frequency of output; 0 implies no output.  Deprecated.\n",
      "        disp : {0, 1, 2, 3}, optional\n",
      "            Over-rides the iprint interface.  Preferred.\n",
      "        maxfun : int, optional\n",
      "            Maximum number of function evaluations.\n",
      "        catol : float, optional\n",
      "            Absolute tolerance for constraint violations.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        x : ndarray\n",
      "            The argument that minimises `f`.\n",
      "        \n",
      "        See also\n",
      "        --------\n",
      "        minimize: Interface to minimization algorithms for multivariate\n",
      "            functions. See the 'COBYLA' `method` in particular.\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        This algorithm is based on linear approximations to the objective\n",
      "        function and each constraint. We briefly describe the algorithm.\n",
      "        \n",
      "        Suppose the function is being minimized over k variables. At the\n",
      "        jth iteration the algorithm has k+1 points v_1, ..., v_(k+1),\n",
      "        an approximate solution x_j, and a radius RHO_j.\n",
      "        (i.e. linear plus a constant) approximations to the objective\n",
      "        function and constraint functions such that their function values\n",
      "        agree with the linear approximation on the k+1 points v_1,.., v_(k+1).\n",
      "        This gives a linear program to solve (where the linear approximations\n",
      "        of the constraint functions are constrained to be non-negative).\n",
      "        \n",
      "        However the linear approximations are likely only good\n",
      "        approximations near the current simplex, so the linear program is\n",
      "        given the further requirement that the solution, which\n",
      "        will become x_(j+1), must be within RHO_j from x_j. RHO_j only\n",
      "        decreases, never increases. The initial RHO_j is rhobeg and the\n",
      "        final RHO_j is rhoend. In this way COBYLA's iterations behave\n",
      "        like a trust region algorithm.\n",
      "        \n",
      "        Additionally, the linear program may be inconsistent, or the\n",
      "        approximation may give poor improvement. For details about\n",
      "        how these issues are resolved, as well as how the points v_i are\n",
      "        updated, refer to the source code or the references below.\n",
      "        \n",
      "        \n",
      "        References\n",
      "        ----------\n",
      "        Powell M.J.D. (1994), \"A direct search optimization method that models\n",
      "        the objective and constraint functions by linear interpolation.\", in\n",
      "        Advances in Optimization and Numerical Analysis, eds. S. Gomez and\n",
      "        J-P Hennart, Kluwer Academic (Dordrecht), pp. 51-67\n",
      "        \n",
      "        Powell M.J.D. (1998), \"Direct search algorithms for optimization\n",
      "        calculations\", Acta Numerica 7, 287-336\n",
      "        \n",
      "        Powell M.J.D. (2007), \"A view of algorithms for optimization without\n",
      "        derivatives\", Cambridge University Technical Report DAMTP 2007/NA03\n",
      "        \n",
      "        \n",
      "        Examples\n",
      "        --------\n",
      "        Minimize the objective function f(x,y) = x*y subject\n",
      "        to the constraints x**2 + y**2 < 1 and y > 0::\n",
      "        \n",
      "            >>> def objective(x):\n",
      "            ...     return x[0]*x[1]\n",
      "            ...\n",
      "            >>> def constr1(x):\n",
      "            ...     return 1 - (x[0]**2 + x[1]**2)\n",
      "            ...\n",
      "            >>> def constr2(x):\n",
      "            ...     return x[1]\n",
      "            ...\n",
      "            >>> from scipy.optimize import fmin_cobyla\n",
      "            >>> fmin_cobyla(objective, [0.0, 0.1], [constr1, constr2], rhoend=1e-7)\n",
      "        \n",
      "               Normal return from subroutine COBYLA\n",
      "        \n",
      "               NFVALS =   64   F =-5.000000E-01    MAXCV = 1.998401E-14\n",
      "               X =-7.071069E-01   7.071067E-01\n",
      "            array([-0.70710685,  0.70710671])\n",
      "        \n",
      "        The exact solution is (-sqrt(2)/2, sqrt(2)/2).\n",
      "    \n",
      "    fmin_l_bfgs_b(func, x0, fprime=None, args=(), approx_grad=0, bounds=None, m=10, factr=10000000.0, pgtol=1e-05, epsilon=1e-08, iprint=-1, maxfun=15000, maxiter=15000, disp=None, callback=None)\n",
      "        Minimize a function func using the L-BFGS-B algorithm.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        func : callable f(x,*args)\n",
      "            Function to minimise.\n",
      "        x0 : ndarray\n",
      "            Initial guess.\n",
      "        fprime : callable fprime(x,*args), optional\n",
      "            The gradient of `func`.  If None, then `func` returns the function\n",
      "            value and the gradient (``f, g = func(x, *args)``), unless\n",
      "            `approx_grad` is True in which case `func` returns only ``f``.\n",
      "        args : sequence, optional\n",
      "            Arguments to pass to `func` and `fprime`.\n",
      "        approx_grad : bool, optional\n",
      "            Whether to approximate the gradient numerically (in which case\n",
      "            `func` returns only the function value).\n",
      "        bounds : list, optional\n",
      "            ``(min, max)`` pairs for each element in ``x``, defining\n",
      "            the bounds on that parameter. Use None or +-inf for one of ``min`` or\n",
      "            ``max`` when there is no bound in that direction.\n",
      "        m : int, optional\n",
      "            The maximum number of variable metric corrections\n",
      "            used to define the limited memory matrix. (The limited memory BFGS\n",
      "            method does not store the full hessian but uses this many terms in an\n",
      "            approximation to it.)\n",
      "        factr : float, optional\n",
      "            The iteration stops when\n",
      "            ``(f^k - f^{k+1})/max{|f^k|,|f^{k+1}|,1} <= factr * eps``,\n",
      "            where ``eps`` is the machine precision, which is automatically\n",
      "            generated by the code. Typical values for `factr` are: 1e12 for\n",
      "            low accuracy; 1e7 for moderate accuracy; 10.0 for extremely\n",
      "            high accuracy.\n",
      "        pgtol : float, optional\n",
      "            The iteration will stop when\n",
      "            ``max{|proj g_i | i = 1, ..., n} <= pgtol``\n",
      "            where ``pg_i`` is the i-th component of the projected gradient.\n",
      "        epsilon : float, optional\n",
      "            Step size used when `approx_grad` is True, for numerically\n",
      "            calculating the gradient\n",
      "        iprint : int, optional\n",
      "            Controls the frequency of output. ``iprint < 0`` means no output;\n",
      "            ``iprint == 0`` means write messages to stdout; ``iprint > 1`` in\n",
      "            addition means write logging information to a file named\n",
      "            ``iterate.dat`` in the current working directory.\n",
      "        disp : int, optional\n",
      "            If zero, then no output.  If a positive number, then this over-rides\n",
      "            `iprint` (i.e., `iprint` gets the value of `disp`).\n",
      "        maxfun : int, optional\n",
      "            Maximum number of function evaluations.\n",
      "        maxiter : int, optional\n",
      "            Maximum number of iterations.\n",
      "        callback : callable, optional\n",
      "            Called after each iteration, as ``callback(xk)``, where ``xk`` is the\n",
      "            current parameter vector.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        x : array_like\n",
      "            Estimated position of the minimum.\n",
      "        f : float\n",
      "            Value of `func` at the minimum.\n",
      "        d : dict\n",
      "            Information dictionary.\n",
      "        \n",
      "            * d['warnflag'] is\n",
      "        \n",
      "              - 0 if converged,\n",
      "              - 1 if too many function evaluations or too many iterations,\n",
      "              - 2 if stopped for another reason, given in d['task']\n",
      "        \n",
      "            * d['grad'] is the gradient at the minimum (should be 0 ish)\n",
      "            * d['funcalls'] is the number of function calls made.\n",
      "            * d['nit'] is the number of iterations.\n",
      "        \n",
      "        See also\n",
      "        --------\n",
      "        minimize: Interface to minimization algorithms for multivariate\n",
      "            functions. See the 'L-BFGS-B' `method` in particular.\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        License of L-BFGS-B (FORTRAN code):\n",
      "        \n",
      "        The version included here (in fortran code) is 3.0\n",
      "        (released April 25, 2011).  It was written by Ciyou Zhu, Richard Byrd,\n",
      "        and Jorge Nocedal <nocedal@ece.nwu.edu>. It carries the following\n",
      "        condition for use:\n",
      "        \n",
      "        This software is freely available, but we expect that all publications\n",
      "        describing work using this software, or all commercial products using it,\n",
      "        quote at least one of the references given below. This software is released\n",
      "        under the BSD License.\n",
      "        \n",
      "        References\n",
      "        ----------\n",
      "        * R. H. Byrd, P. Lu and J. Nocedal. A Limited Memory Algorithm for Bound\n",
      "          Constrained Optimization, (1995), SIAM Journal on Scientific and\n",
      "          Statistical Computing, 16, 5, pp. 1190-1208.\n",
      "        * C. Zhu, R. H. Byrd and J. Nocedal. L-BFGS-B: Algorithm 778: L-BFGS-B,\n",
      "          FORTRAN routines for large scale bound constrained optimization (1997),\n",
      "          ACM Transactions on Mathematical Software, 23, 4, pp. 550 - 560.\n",
      "        * J.L. Morales and J. Nocedal. L-BFGS-B: Remark on Algorithm 778: L-BFGS-B,\n",
      "          FORTRAN routines for large scale bound constrained optimization (2011),\n",
      "          ACM Transactions on Mathematical Software, 38, 1.\n",
      "    \n",
      "    fmin_ncg(f, x0, fprime, fhess_p=None, fhess=None, args=(), avextol=1e-05, epsilon=1.4901161193847656e-08, maxiter=None, full_output=0, disp=1, retall=0, callback=None)\n",
      "        Unconstrained minimization of a function using the Newton-CG method.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        f : callable ``f(x, *args)``\n",
      "            Objective function to be minimized.\n",
      "        x0 : ndarray\n",
      "            Initial guess.\n",
      "        fprime : callable ``f'(x, *args)``\n",
      "            Gradient of f.\n",
      "        fhess_p : callable ``fhess_p(x, p, *args)``, optional\n",
      "            Function which computes the Hessian of f times an\n",
      "            arbitrary vector, p.\n",
      "        fhess : callable ``fhess(x, *args)``, optional\n",
      "            Function to compute the Hessian matrix of f.\n",
      "        args : tuple, optional\n",
      "            Extra arguments passed to f, fprime, fhess_p, and fhess\n",
      "            (the same set of extra arguments is supplied to all of\n",
      "            these functions).\n",
      "        epsilon : float or ndarray, optional\n",
      "            If fhess is approximated, use this value for the step size.\n",
      "        callback : callable, optional\n",
      "            An optional user-supplied function which is called after\n",
      "            each iteration.  Called as callback(xk), where xk is the\n",
      "            current parameter vector.\n",
      "        avextol : float, optional\n",
      "            Convergence is assumed when the average relative error in\n",
      "            the minimizer falls below this amount.\n",
      "        maxiter : int, optional\n",
      "            Maximum number of iterations to perform.\n",
      "        full_output : bool, optional\n",
      "            If True, return the optional outputs.\n",
      "        disp : bool, optional\n",
      "            If True, print convergence message.\n",
      "        retall : bool, optional\n",
      "            If True, return a list of results at each iteration.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        xopt : ndarray\n",
      "            Parameters which minimize f, i.e. ``f(xopt) == fopt``.\n",
      "        fopt : float\n",
      "            Value of the function at xopt, i.e. ``fopt = f(xopt)``.\n",
      "        fcalls : int\n",
      "            Number of function calls made.\n",
      "        gcalls : int\n",
      "            Number of gradient calls made.\n",
      "        hcalls : int\n",
      "            Number of hessian calls made.\n",
      "        warnflag : int\n",
      "            Warnings generated by the algorithm.\n",
      "            1 : Maximum number of iterations exceeded.\n",
      "        allvecs : list\n",
      "            The result at each iteration, if retall is True (see below).\n",
      "        \n",
      "        See also\n",
      "        --------\n",
      "        minimize: Interface to minimization algorithms for multivariate\n",
      "            functions. See the 'Newton-CG' `method` in particular.\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        Only one of `fhess_p` or `fhess` need to be given.  If `fhess`\n",
      "        is provided, then `fhess_p` will be ignored.  If neither `fhess`\n",
      "        nor `fhess_p` is provided, then the hessian product will be\n",
      "        approximated using finite differences on `fprime`. `fhess_p`\n",
      "        must compute the hessian times an arbitrary vector. If it is not\n",
      "        given, finite-differences on `fprime` are used to compute\n",
      "        it.\n",
      "        \n",
      "        Newton-CG methods are also called truncated Newton methods. This\n",
      "        function differs from scipy.optimize.fmin_tnc because\n",
      "        \n",
      "        1. scipy.optimize.fmin_ncg is written purely in python using numpy\n",
      "            and scipy while scipy.optimize.fmin_tnc calls a C function.\n",
      "        2. scipy.optimize.fmin_ncg is only for unconstrained minimization\n",
      "            while scipy.optimize.fmin_tnc is for unconstrained minimization\n",
      "            or box constrained minimization. (Box constraints give\n",
      "            lower and upper bounds for each variable separately.)\n",
      "        \n",
      "        References\n",
      "        ----------\n",
      "        Wright & Nocedal, 'Numerical Optimization', 1999, pg. 140.\n",
      "    \n",
      "    fmin_powell(func, x0, args=(), xtol=0.0001, ftol=0.0001, maxiter=None, maxfun=None, full_output=0, disp=1, retall=0, callback=None, direc=None)\n",
      "        Minimize a function using modified Powell's method. This method\n",
      "        only uses function values, not derivatives.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        func : callable f(x,*args)\n",
      "            Objective function to be minimized.\n",
      "        x0 : ndarray\n",
      "            Initial guess.\n",
      "        args : tuple, optional\n",
      "            Extra arguments passed to func.\n",
      "        callback : callable, optional\n",
      "            An optional user-supplied function, called after each\n",
      "            iteration.  Called as ``callback(xk)``, where ``xk`` is the\n",
      "            current parameter vector.\n",
      "        direc : ndarray, optional\n",
      "            Initial direction set.\n",
      "        xtol : float, optional\n",
      "            Line-search error tolerance.\n",
      "        ftol : float, optional\n",
      "            Relative error in ``func(xopt)`` acceptable for convergence.\n",
      "        maxiter : int, optional\n",
      "            Maximum number of iterations to perform.\n",
      "        maxfun : int, optional\n",
      "            Maximum number of function evaluations to make.\n",
      "        full_output : bool, optional\n",
      "            If True, fopt, xi, direc, iter, funcalls, and\n",
      "            warnflag are returned.\n",
      "        disp : bool, optional\n",
      "            If True, print convergence messages.\n",
      "        retall : bool, optional\n",
      "            If True, return a list of the solution at each iteration.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        xopt : ndarray\n",
      "            Parameter which minimizes `func`.\n",
      "        fopt : number\n",
      "            Value of function at minimum: ``fopt = func(xopt)``.\n",
      "        direc : ndarray\n",
      "            Current direction set.\n",
      "        iter : int\n",
      "            Number of iterations.\n",
      "        funcalls : int\n",
      "            Number of function calls made.\n",
      "        warnflag : int\n",
      "            Integer warning flag:\n",
      "                1 : Maximum number of function evaluations.\n",
      "                2 : Maximum number of iterations.\n",
      "        allvecs : list\n",
      "            List of solutions at each iteration.\n",
      "        \n",
      "        See also\n",
      "        --------\n",
      "        minimize: Interface to unconstrained minimization algorithms for\n",
      "            multivariate functions. See the 'Powell' `method` in particular.\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        Uses a modification of Powell's method to find the minimum of\n",
      "        a function of N variables. Powell's method is a conjugate\n",
      "        direction method.\n",
      "        \n",
      "        The algorithm has two loops. The outer loop\n",
      "        merely iterates over the inner loop. The inner loop minimizes\n",
      "        over each current direction in the direction set. At the end\n",
      "        of the inner loop, if certain conditions are met, the direction\n",
      "        that gave the largest decrease is dropped and replaced with\n",
      "        the difference between the current estiamted x and the estimated\n",
      "        x from the beginning of the inner-loop.\n",
      "        \n",
      "        The technical conditions for replacing the direction of greatest\n",
      "        increase amount to checking that\n",
      "        \n",
      "        1. No further gain can be made along the direction of greatest increase\n",
      "           from that iteration.\n",
      "        2. The direction of greatest increase accounted for a large sufficient\n",
      "           fraction of the decrease in the function value from that iteration of\n",
      "           the inner loop.\n",
      "        \n",
      "        References\n",
      "        ----------\n",
      "        Powell M.J.D. (1964) An efficient method for finding the minimum of a\n",
      "        function of several variables without calculating derivatives,\n",
      "        Computer Journal, 7 (2):155-162.\n",
      "        \n",
      "        Press W., Teukolsky S.A., Vetterling W.T., and Flannery B.P.:\n",
      "        Numerical Recipes (any edition), Cambridge University Press\n",
      "    \n",
      "    fmin_slsqp(func, x0, eqcons=(), f_eqcons=None, ieqcons=(), f_ieqcons=None, bounds=(), fprime=None, fprime_eqcons=None, fprime_ieqcons=None, args=(), iter=100, acc=1e-06, iprint=1, disp=None, full_output=0, epsilon=1.4901161193847656e-08, callback=None)\n",
      "        Minimize a function using Sequential Least SQuares Programming\n",
      "        \n",
      "        Python interface function for the SLSQP Optimization subroutine\n",
      "        originally implemented by Dieter Kraft.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        func : callable f(x,*args)\n",
      "            Objective function.\n",
      "        x0 : 1-D ndarray of float\n",
      "            Initial guess for the independent variable(s).\n",
      "        eqcons : list, optional\n",
      "            A list of functions of length n such that\n",
      "            eqcons[j](x,*args) == 0.0 in a successfully optimized\n",
      "            problem.\n",
      "        f_eqcons : callable f(x,*args), optional\n",
      "            Returns a 1-D array in which each element must equal 0.0 in a\n",
      "            successfully optimized problem.  If f_eqcons is specified,\n",
      "            eqcons is ignored.\n",
      "        ieqcons : list, optional\n",
      "            A list of functions of length n such that\n",
      "            ieqcons[j](x,*args) >= 0.0 in a successfully optimized\n",
      "            problem.\n",
      "        f_ieqcons : callable f(x,*args), optional\n",
      "            Returns a 1-D ndarray in which each element must be greater or\n",
      "            equal to 0.0 in a successfully optimized problem.  If\n",
      "            f_ieqcons is specified, ieqcons is ignored.\n",
      "        bounds : list, optional\n",
      "            A list of tuples specifying the lower and upper bound\n",
      "            for each independent variable [(xl0, xu0),(xl1, xu1),...]\n",
      "            Infinite values will be interpreted as large floating values.\n",
      "        fprime : callable `f(x,*args)`, optional\n",
      "            A function that evaluates the partial derivatives of func.\n",
      "        fprime_eqcons : callable `f(x,*args)`, optional\n",
      "            A function of the form `f(x, *args)` that returns the m by n\n",
      "            array of equality constraint normals.  If not provided,\n",
      "            the normals will be approximated. The array returned by\n",
      "            fprime_eqcons should be sized as ( len(eqcons), len(x0) ).\n",
      "        fprime_ieqcons : callable `f(x,*args)`, optional\n",
      "            A function of the form `f(x, *args)` that returns the m by n\n",
      "            array of inequality constraint normals.  If not provided,\n",
      "            the normals will be approximated. The array returned by\n",
      "            fprime_ieqcons should be sized as ( len(ieqcons), len(x0) ).\n",
      "        args : sequence, optional\n",
      "            Additional arguments passed to func and fprime.\n",
      "        iter : int, optional\n",
      "            The maximum number of iterations.\n",
      "        acc : float, optional\n",
      "            Requested accuracy.\n",
      "        iprint : int, optional\n",
      "            The verbosity of fmin_slsqp :\n",
      "        \n",
      "            * iprint <= 0 : Silent operation\n",
      "            * iprint == 1 : Print summary upon completion (default)\n",
      "            * iprint >= 2 : Print status of each iterate and summary\n",
      "        disp : int, optional\n",
      "            Over-rides the iprint interface (preferred).\n",
      "        full_output : bool, optional\n",
      "            If False, return only the minimizer of func (default).\n",
      "            Otherwise, output final objective function and summary\n",
      "            information.\n",
      "        epsilon : float, optional\n",
      "            The step size for finite-difference derivative estimates.\n",
      "        callback : callable, optional\n",
      "            Called after each iteration, as ``callback(x)``, where ``x`` is the\n",
      "            current parameter vector.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        out : ndarray of float\n",
      "            The final minimizer of func.\n",
      "        fx : ndarray of float, if full_output is true\n",
      "            The final value of the objective function.\n",
      "        its : int, if full_output is true\n",
      "            The number of iterations.\n",
      "        imode : int, if full_output is true\n",
      "            The exit mode from the optimizer (see below).\n",
      "        smode : string, if full_output is true\n",
      "            Message describing the exit mode from the optimizer.\n",
      "        \n",
      "        See also\n",
      "        --------\n",
      "        minimize: Interface to minimization algorithms for multivariate\n",
      "            functions. See the 'SLSQP' `method` in particular.\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        Exit modes are defined as follows ::\n",
      "        \n",
      "            -1 : Gradient evaluation required (g & a)\n",
      "             0 : Optimization terminated successfully.\n",
      "             1 : Function evaluation required (f & c)\n",
      "             2 : More equality constraints than independent variables\n",
      "             3 : More than 3*n iterations in LSQ subproblem\n",
      "             4 : Inequality constraints incompatible\n",
      "             5 : Singular matrix E in LSQ subproblem\n",
      "             6 : Singular matrix C in LSQ subproblem\n",
      "             7 : Rank-deficient equality constraint subproblem HFTI\n",
      "             8 : Positive directional derivative for linesearch\n",
      "             9 : Iteration limit exceeded\n",
      "        \n",
      "        Examples\n",
      "        --------\n",
      "        Examples are given :ref:`in the tutorial <tutorial-sqlsp>`.\n",
      "    \n",
      "    fmin_tnc(func, x0, fprime=None, args=(), approx_grad=0, bounds=None, epsilon=1e-08, scale=None, offset=None, messages=15, maxCGit=-1, maxfun=None, eta=-1, stepmx=0, accuracy=0, fmin=0, ftol=-1, xtol=-1, pgtol=-1, rescale=-1, disp=None, callback=None)\n",
      "        Minimize a function with variables subject to bounds, using\n",
      "        gradient information in a truncated Newton algorithm. This\n",
      "        method wraps a C implementation of the algorithm.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        func : callable ``func(x, *args)``\n",
      "            Function to minimize.  Must do one of:\n",
      "        \n",
      "            1. Return f and g, where f is the value of the function and g its\n",
      "               gradient (a list of floats).\n",
      "        \n",
      "            2. Return the function value but supply gradient function\n",
      "               separately as `fprime`.\n",
      "        \n",
      "            3. Return the function value and set ``approx_grad=True``.\n",
      "        \n",
      "            If the function returns None, the minimization\n",
      "            is aborted.\n",
      "        x0 : array_like\n",
      "            Initial estimate of minimum.\n",
      "        fprime : callable ``fprime(x, *args)``, optional\n",
      "            Gradient of `func`. If None, then either `func` must return the\n",
      "            function value and the gradient (``f,g = func(x, *args)``)\n",
      "            or `approx_grad` must be True.\n",
      "        args : tuple, optional\n",
      "            Arguments to pass to function.\n",
      "        approx_grad : bool, optional\n",
      "            If true, approximate the gradient numerically.\n",
      "        bounds : list, optional\n",
      "            (min, max) pairs for each element in x0, defining the\n",
      "            bounds on that parameter. Use None or +/-inf for one of\n",
      "            min or max when there is no bound in that direction.\n",
      "        epsilon : float, optional\n",
      "            Used if approx_grad is True. The stepsize in a finite\n",
      "            difference approximation for fprime.\n",
      "        scale : array_like, optional\n",
      "            Scaling factors to apply to each variable.  If None, the\n",
      "            factors are up-low for interval bounded variables and\n",
      "            1+|x| for the others.  Defaults to None.\n",
      "        offset : array_like, optional\n",
      "            Value to subtract from each variable.  If None, the\n",
      "            offsets are (up+low)/2 for interval bounded variables\n",
      "            and x for the others.\n",
      "        messages : int, optional\n",
      "            Bit mask used to select messages display during\n",
      "            minimization values defined in the MSGS dict.  Defaults to\n",
      "            MGS_ALL.\n",
      "        disp : int, optional\n",
      "            Integer interface to messages.  0 = no message, 5 = all messages\n",
      "        maxCGit : int, optional\n",
      "            Maximum number of hessian*vector evaluations per main\n",
      "            iteration.  If maxCGit == 0, the direction chosen is\n",
      "            -gradient if maxCGit < 0, maxCGit is set to\n",
      "            max(1,min(50,n/2)).  Defaults to -1.\n",
      "        maxfun : int, optional\n",
      "            Maximum number of function evaluation.  if None, maxfun is\n",
      "            set to max(100, 10*len(x0)).  Defaults to None.\n",
      "        eta : float, optional\n",
      "            Severity of the line search. if < 0 or > 1, set to 0.25.\n",
      "            Defaults to -1.\n",
      "        stepmx : float, optional\n",
      "            Maximum step for the line search.  May be increased during\n",
      "            call.  If too small, it will be set to 10.0.  Defaults to 0.\n",
      "        accuracy : float, optional\n",
      "            Relative precision for finite difference calculations.  If\n",
      "            <= machine_precision, set to sqrt(machine_precision).\n",
      "            Defaults to 0.\n",
      "        fmin : float, optional\n",
      "            Minimum function value estimate.  Defaults to 0.\n",
      "        ftol : float, optional\n",
      "            Precision goal for the value of f in the stoping criterion.\n",
      "            If ftol < 0.0, ftol is set to 0.0 defaults to -1.\n",
      "        xtol : float, optional\n",
      "            Precision goal for the value of x in the stopping\n",
      "            criterion (after applying x scaling factors).  If xtol <\n",
      "            0.0, xtol is set to sqrt(machine_precision).  Defaults to\n",
      "            -1.\n",
      "        pgtol : float, optional\n",
      "            Precision goal for the value of the projected gradient in\n",
      "            the stopping criterion (after applying x scaling factors).\n",
      "            If pgtol < 0.0, pgtol is set to 1e-2 * sqrt(accuracy).\n",
      "            Setting it to 0.0 is not recommended.  Defaults to -1.\n",
      "        rescale : float, optional\n",
      "            Scaling factor (in log10) used to trigger f value\n",
      "            rescaling.  If 0, rescale at each iteration.  If a large\n",
      "            value, never rescale.  If < 0, rescale is set to 1.3.\n",
      "        callback : callable, optional\n",
      "            Called after each iteration, as callback(xk), where xk is the\n",
      "            current parameter vector.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        x : ndarray\n",
      "            The solution.\n",
      "        nfeval : int\n",
      "            The number of function evaluations.\n",
      "        rc : int\n",
      "            Return code, see below\n",
      "        \n",
      "        See also\n",
      "        --------\n",
      "        minimize: Interface to minimization algorithms for multivariate\n",
      "            functions. See the 'TNC' `method` in particular.\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        The underlying algorithm is truncated Newton, also called\n",
      "        Newton Conjugate-Gradient. This method differs from\n",
      "        scipy.optimize.fmin_ncg in that\n",
      "        \n",
      "        1. It wraps a C implementation of the algorithm\n",
      "        2. It allows each variable to be given an upper and lower bound.\n",
      "        \n",
      "        The algorithm incoporates the bound constraints by determining\n",
      "        the descent direction as in an unconstrained truncated Newton,\n",
      "        but never taking a step-size large enough to leave the space\n",
      "        of feasible x's. The algorithm keeps track of a set of\n",
      "        currently active constraints, and ignores them when computing\n",
      "        the minimum allowable step size. (The x's associated with the\n",
      "        active constraint are kept fixed.) If the maximum allowable\n",
      "        step size is zero then a new constraint is added. At the end\n",
      "        of each iteration one of the constraints may be deemed no\n",
      "        longer active and removed. A constraint is considered\n",
      "        no longer active is if it is currently active\n",
      "        but the gradient for that variable points inward from the\n",
      "        constraint. The specific constraint removed is the one\n",
      "        associated with the variable of largest index whose\n",
      "        constraint is no longer active.\n",
      "        \n",
      "        Return codes are defined as follows::\n",
      "        \n",
      "            -1 : Infeasible (lower bound > upper bound)\n",
      "             0 : Local minimum reached (|pg| ~= 0)\n",
      "             1 : Converged (|f_n-f_(n-1)| ~= 0)\n",
      "             2 : Converged (|x_n-x_(n-1)| ~= 0)\n",
      "             3 : Max. number of function evaluations reached\n",
      "             4 : Linear search failed\n",
      "             5 : All lower bounds are equal to the upper bounds\n",
      "             6 : Unable to progress\n",
      "             7 : User requested end of minimization\n",
      "        \n",
      "        References\n",
      "        ----------\n",
      "        Wright S., Nocedal J. (2006), 'Numerical Optimization'\n",
      "        \n",
      "        Nash S.G. (1984), \"Newton-Type Minimization Via the Lanczos Method\",\n",
      "        SIAM Journal of Numerical Analysis 21, pp. 770-778\n",
      "    \n",
      "    fminbound(func, x1, x2, args=(), xtol=1e-05, maxfun=500, full_output=0, disp=1)\n",
      "        Bounded minimization for scalar functions.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        func : callable f(x,*args)\n",
      "            Objective function to be minimized (must accept and return scalars).\n",
      "        x1, x2 : float or array scalar\n",
      "            The optimization bounds.\n",
      "        args : tuple, optional\n",
      "            Extra arguments passed to function.\n",
      "        xtol : float, optional\n",
      "            The convergence tolerance.\n",
      "        maxfun : int, optional\n",
      "            Maximum number of function evaluations allowed.\n",
      "        full_output : bool, optional\n",
      "            If True, return optional outputs.\n",
      "        disp : int, optional\n",
      "            If non-zero, print messages.\n",
      "                0 : no message printing.\n",
      "                1 : non-convergence notification messages only.\n",
      "                2 : print a message on convergence too.\n",
      "                3 : print iteration results.\n",
      "        \n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        xopt : ndarray\n",
      "            Parameters (over given interval) which minimize the\n",
      "            objective function.\n",
      "        fval : number\n",
      "            The function value at the minimum point.\n",
      "        ierr : int\n",
      "            An error flag (0 if converged, 1 if maximum number of\n",
      "            function calls reached).\n",
      "        numfunc : int\n",
      "          The number of function calls made.\n",
      "        \n",
      "        See also\n",
      "        --------\n",
      "        minimize_scalar: Interface to minimization algorithms for scalar\n",
      "            univariate functions. See the 'Bounded' `method` in particular.\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        Finds a local minimizer of the scalar function `func` in the\n",
      "        interval x1 < xopt < x2 using Brent's method.  (See `brent`\n",
      "        for auto-bracketing).\n",
      "    \n",
      "    fsolve(func, x0, args=(), fprime=None, full_output=0, col_deriv=0, xtol=1.49012e-08, maxfev=0, band=None, epsfcn=None, factor=100, diag=None)\n",
      "        Find the roots of a function.\n",
      "        \n",
      "        Return the roots of the (non-linear) equations defined by\n",
      "        ``func(x) = 0`` given a starting estimate.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        func : callable ``f(x, *args)``\n",
      "            A function that takes at least one (possibly vector) argument.\n",
      "        x0 : ndarray\n",
      "            The starting estimate for the roots of ``func(x) = 0``.\n",
      "        args : tuple, optional\n",
      "            Any extra arguments to `func`.\n",
      "        fprime : callable(x), optional\n",
      "            A function to compute the Jacobian of `func` with derivatives\n",
      "            across the rows. By default, the Jacobian will be estimated.\n",
      "        full_output : bool, optional\n",
      "            If True, return optional outputs.\n",
      "        col_deriv : bool, optional\n",
      "            Specify whether the Jacobian function computes derivatives down\n",
      "            the columns (faster, because there is no transpose operation).\n",
      "        xtol : float, optional\n",
      "            The calculation will terminate if the relative error between two\n",
      "            consecutive iterates is at most `xtol`.\n",
      "        maxfev : int, optional\n",
      "            The maximum number of calls to the function. If zero, then\n",
      "            ``100*(N+1)`` is the maximum where N is the number of elements\n",
      "            in `x0`.\n",
      "        band : tuple, optional\n",
      "            If set to a two-sequence containing the number of sub- and\n",
      "            super-diagonals within the band of the Jacobi matrix, the\n",
      "            Jacobi matrix is considered banded (only for ``fprime=None``).\n",
      "        epsfcn : float, optional\n",
      "            A suitable step length for the forward-difference\n",
      "            approximation of the Jacobian (for ``fprime=None``). If\n",
      "            `epsfcn` is less than the machine precision, it is assumed\n",
      "            that the relative errors in the functions are of the order of\n",
      "            the machine precision.\n",
      "        factor : float, optional\n",
      "            A parameter determining the initial step bound\n",
      "            (``factor * || diag * x||``).  Should be in the interval\n",
      "            ``(0.1, 100)``.\n",
      "        diag : sequence, optional\n",
      "            N positive entries that serve as a scale factors for the\n",
      "            variables.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        x : ndarray\n",
      "            The solution (or the result of the last iteration for\n",
      "            an unsuccessful call).\n",
      "        infodict : dict\n",
      "            A dictionary of optional outputs with the keys:\n",
      "        \n",
      "            ``nfev``\n",
      "                number of function calls\n",
      "            ``njev``\n",
      "                number of Jacobian calls\n",
      "            ``fvec``\n",
      "                function evaluated at the output\n",
      "            ``fjac``\n",
      "                the orthogonal matrix, q, produced by the QR\n",
      "                factorization of the final approximate Jacobian\n",
      "                matrix, stored column wise\n",
      "            ``r``\n",
      "                upper triangular matrix produced by QR factorization\n",
      "                of the same matrix\n",
      "            ``qtf``\n",
      "                the vector ``(transpose(q) * fvec)``\n",
      "        \n",
      "        ier : int\n",
      "            An integer flag.  Set to 1 if a solution was found, otherwise refer\n",
      "            to `mesg` for more information.\n",
      "        mesg : str\n",
      "            If no solution is found, `mesg` details the cause of failure.\n",
      "        \n",
      "        See Also\n",
      "        --------\n",
      "        root : Interface to root finding algorithms for multivariate\n",
      "        functions. See the 'hybr' `method` in particular.\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        ``fsolve`` is a wrapper around MINPACK's hybrd and hybrj algorithms.\n",
      "    \n",
      "    golden(func, args=(), brack=None, tol=1.4901161193847656e-08, full_output=0)\n",
      "        Return the minimum of a function of one variable.\n",
      "        \n",
      "        Given a function of one variable and a possible bracketing interval,\n",
      "        return the minimum of the function isolated to a fractional precision of\n",
      "        tol.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        func : callable func(x,*args)\n",
      "            Objective function to minimize.\n",
      "        args : tuple, optional\n",
      "            Additional arguments (if present), passed to func.\n",
      "        brack : tuple, optional\n",
      "            Triple (a,b,c), where (a<b<c) and func(b) <\n",
      "            func(a),func(c).  If bracket consists of two numbers (a,\n",
      "            c), then they are assumed to be a starting interval for a\n",
      "            downhill bracket search (see `bracket`); it doesn't always\n",
      "            mean that obtained solution will satisfy a<=x<=c.\n",
      "        tol : float, optional\n",
      "            x tolerance stop criterion\n",
      "        full_output : bool, optional\n",
      "            If True, return optional outputs.\n",
      "        \n",
      "        See also\n",
      "        --------\n",
      "        minimize_scalar: Interface to minimization algorithms for scalar\n",
      "            univariate functions. See the 'Golden' `method` in particular.\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        Uses analog of bisection method to decrease the bracketed\n",
      "        interval.\n",
      "    \n",
      "    leastsq(func, x0, args=(), Dfun=None, full_output=0, col_deriv=0, ftol=1.49012e-08, xtol=1.49012e-08, gtol=0.0, maxfev=0, epsfcn=None, factor=100, diag=None)\n",
      "        Minimize the sum of squares of a set of equations.\n",
      "        \n",
      "        ::\n",
      "        \n",
      "            x = arg min(sum(func(y)**2,axis=0))\n",
      "                     y\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        func : callable\n",
      "            should take at least one (possibly length N vector) argument and\n",
      "            returns M floating point numbers. It must not return NaNs or\n",
      "            fitting might fail.\n",
      "        x0 : ndarray\n",
      "            The starting estimate for the minimization.\n",
      "        args : tuple, optional\n",
      "            Any extra arguments to func are placed in this tuple.\n",
      "        Dfun : callable, optional\n",
      "            A function or method to compute the Jacobian of func with derivatives\n",
      "            across the rows. If this is None, the Jacobian will be estimated.\n",
      "        full_output : bool, optional\n",
      "            non-zero to return all optional outputs.\n",
      "        col_deriv : bool, optional\n",
      "            non-zero to specify that the Jacobian function computes derivatives\n",
      "            down the columns (faster, because there is no transpose operation).\n",
      "        ftol : float, optional\n",
      "            Relative error desired in the sum of squares.\n",
      "        xtol : float, optional\n",
      "            Relative error desired in the approximate solution.\n",
      "        gtol : float, optional\n",
      "            Orthogonality desired between the function vector and the columns of\n",
      "            the Jacobian.\n",
      "        maxfev : int, optional\n",
      "            The maximum number of calls to the function. If `Dfun` is provided\n",
      "            then the default `maxfev` is 100*(N+1) where N is the number of elements\n",
      "            in x0, otherwise the default `maxfev` is 200*(N+1).\n",
      "        epsfcn : float, optional\n",
      "            A variable used in determining a suitable step length for the forward-\n",
      "            difference approximation of the Jacobian (for Dfun=None). \n",
      "            Normally the actual step length will be sqrt(epsfcn)*x\n",
      "            If epsfcn is less than the machine precision, it is assumed that the \n",
      "            relative errors are of the order of the machine precision.\n",
      "        factor : float, optional\n",
      "            A parameter determining the initial step bound\n",
      "            (``factor * || diag * x||``). Should be in interval ``(0.1, 100)``.\n",
      "        diag : sequence, optional\n",
      "            N positive entries that serve as a scale factors for the variables.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        x : ndarray\n",
      "            The solution (or the result of the last iteration for an unsuccessful\n",
      "            call).\n",
      "        cov_x : ndarray\n",
      "            Uses the fjac and ipvt optional outputs to construct an\n",
      "            estimate of the jacobian around the solution. None if a\n",
      "            singular matrix encountered (indicates very flat curvature in\n",
      "            some direction).  This matrix must be multiplied by the\n",
      "            residual variance to get the covariance of the\n",
      "            parameter estimates -- see curve_fit.\n",
      "        infodict : dict\n",
      "            a dictionary of optional outputs with the key s:\n",
      "        \n",
      "            ``nfev``\n",
      "                The number of function calls\n",
      "            ``fvec``\n",
      "                The function evaluated at the output\n",
      "            ``fjac``\n",
      "                A permutation of the R matrix of a QR\n",
      "                factorization of the final approximate\n",
      "                Jacobian matrix, stored column wise.\n",
      "                Together with ipvt, the covariance of the\n",
      "                estimate can be approximated.\n",
      "            ``ipvt``\n",
      "                An integer array of length N which defines\n",
      "                a permutation matrix, p, such that\n",
      "                fjac*p = q*r, where r is upper triangular\n",
      "                with diagonal elements of nonincreasing\n",
      "                magnitude. Column j of p is column ipvt(j)\n",
      "                of the identity matrix.\n",
      "            ``qtf``\n",
      "                The vector (transpose(q) * fvec).\n",
      "        \n",
      "        mesg : str\n",
      "            A string message giving information about the cause of failure.\n",
      "        ier : int\n",
      "            An integer flag.  If it is equal to 1, 2, 3 or 4, the solution was\n",
      "            found.  Otherwise, the solution was not found. In either case, the\n",
      "            optional output variable 'mesg' gives more information.\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        \"leastsq\" is a wrapper around MINPACK's lmdif and lmder algorithms.\n",
      "        \n",
      "        cov_x is a Jacobian approximation to the Hessian of the least squares\n",
      "        objective function.\n",
      "        This approximation assumes that the objective function is based on the\n",
      "        difference between some observed target data (ydata) and a (non-linear)\n",
      "        function of the parameters `f(xdata, params)` ::\n",
      "        \n",
      "               func(params) = ydata - f(xdata, params)\n",
      "        \n",
      "        so that the objective function is ::\n",
      "        \n",
      "               min   sum((ydata - f(xdata, params))**2, axis=0)\n",
      "             params\n",
      "    \n",
      "    line_search = line_search_wolfe2(f, myfprime, xk, pk, gfk=None, old_fval=None, old_old_fval=None, args=(), c1=0.0001, c2=0.9, amax=50)\n",
      "        Find alpha that satisfies strong Wolfe conditions.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        f : callable f(x,*args)\n",
      "            Objective function.\n",
      "        myfprime : callable f'(x,*args)\n",
      "            Objective function gradient.\n",
      "        xk : ndarray\n",
      "            Starting point.\n",
      "        pk : ndarray\n",
      "            Search direction.\n",
      "        gfk : ndarray, optional\n",
      "            Gradient value for x=xk (xk being the current parameter\n",
      "            estimate). Will be recomputed if omitted.\n",
      "        old_fval : float, optional\n",
      "            Function value for x=xk. Will be recomputed if omitted.\n",
      "        old_old_fval : float, optional\n",
      "            Function value for the point preceding x=xk\n",
      "        args : tuple, optional\n",
      "            Additional arguments passed to objective function.\n",
      "        c1 : float, optional\n",
      "            Parameter for Armijo condition rule.\n",
      "        c2 : float, optional\n",
      "            Parameter for curvature condition rule.\n",
      "        amax : float, optional\n",
      "            Maximum step size\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        alpha : float or None\n",
      "            Alpha for which ``x_new = x0 + alpha * pk``,\n",
      "            or None if the line search algorithm did not converge.\n",
      "        fc : int\n",
      "            Number of function evaluations made.\n",
      "        gc : int\n",
      "            Number of gradient evaluations made.\n",
      "        new_fval : float or None\n",
      "            New function value ``f(x_new)=f(x0+alpha*pk)``,\n",
      "            or None if the line search algorithm did not converge.\n",
      "        old_fval : float\n",
      "            Old function value ``f(x0)``.\n",
      "        new_slope : float or None\n",
      "            The local slope along the search direction at the\n",
      "            new value ``<myfprime(x_new), pk>``,\n",
      "            or None if the line search algorithm did not converge.\n",
      "        \n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        Uses the line search algorithm to enforce strong Wolfe\n",
      "        conditions.  See Wright and Nocedal, 'Numerical Optimization',\n",
      "        1999, pg. 59-60.\n",
      "        \n",
      "        For the zoom phase it uses an algorithm by [...].\n",
      "    \n",
      "    linearmixing(F, xin, iter=None, alpha=None, verbose=False, maxiter=None, f_tol=None, f_rtol=None, x_tol=None, x_rtol=None, tol_norm=None, line_search='armijo', callback=None, **kw)\n",
      "        Find a root of a function, using a scalar Jacobian approximation.\n",
      "        \n",
      "        .. warning::\n",
      "        \n",
      "           This algorithm may be useful for specific problems, but whether\n",
      "           it will work may depend strongly on the problem.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        F : function(x) -> f\n",
      "            Function whose root to find; should take and return an array-like\n",
      "            object.\n",
      "        x0 : array_like\n",
      "            Initial guess for the solution\n",
      "        alpha : float, optional\n",
      "            The Jacobian approximation is (-1/alpha).\n",
      "        iter : int, optional\n",
      "            Number of iterations to make. If omitted (default), make as many\n",
      "            as required to meet tolerances.\n",
      "        verbose : bool, optional\n",
      "            Print status to stdout on every iteration.\n",
      "        maxiter : int, optional\n",
      "            Maximum number of iterations to make. If more are needed to\n",
      "            meet convergence, `NoConvergence` is raised.\n",
      "        f_tol : float, optional\n",
      "            Absolute tolerance (in max-norm) for the residual.\n",
      "            If omitted, default is 6e-6.\n",
      "        f_rtol : float, optional\n",
      "            Relative tolerance for the residual. If omitted, not used.\n",
      "        x_tol : float, optional\n",
      "            Absolute minimum step size, as determined from the Jacobian\n",
      "            approximation. If the step size is smaller than this, optimization\n",
      "            is terminated as successful. If omitted, not used.\n",
      "        x_rtol : float, optional\n",
      "            Relative minimum step size. If omitted, not used.\n",
      "        tol_norm : function(vector) -> scalar, optional\n",
      "            Norm to use in convergence check. Default is the maximum norm.\n",
      "        line_search : {None, 'armijo' (default), 'wolfe'}, optional\n",
      "            Which type of a line search to use to determine the step size in the\n",
      "            direction given by the Jacobian approximation. Defaults to 'armijo'.\n",
      "        callback : function, optional\n",
      "            Optional callback function. It is called on every iteration as\n",
      "            ``callback(x, f)`` where `x` is the current solution and `f`\n",
      "            the corresponding residual.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        sol : ndarray\n",
      "            An array (of similar array type as `x0`) containing the final solution.\n",
      "        \n",
      "        Raises\n",
      "        ------\n",
      "        NoConvergence\n",
      "            When a solution was not found.\n",
      "    \n",
      "    linprog(c, A_ub=None, b_ub=None, A_eq=None, b_eq=None, bounds=None, method='simplex', callback=None, options=None)\n",
      "        Minimize a linear objective function subject to linear\n",
      "        equality and inequality constraints.\n",
      "        \n",
      "        Linear Programming is intended to solve the following problem form:\n",
      "        \n",
      "        Minimize:     c^T * x\n",
      "        \n",
      "        Subject to:   A_ub * x <= b_ub\n",
      "                      A_eq * x == b_eq\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        c : array_like\n",
      "            Coefficients of the linear objective function to be minimized.\n",
      "        A_ub : array_like, optional\n",
      "            2-D array which, when matrix-multiplied by x, gives the values of the\n",
      "            upper-bound inequality constraints at x.\n",
      "        b_ub : array_like, optional\n",
      "            1-D array of values representing the upper-bound of each inequality\n",
      "            constraint (row) in A_ub.\n",
      "        A_eq : array_like, optional\n",
      "            2-D array which, when matrix-multiplied by x, gives the values of the\n",
      "            equality constraints at x.\n",
      "        b_eq : array_like, optional\n",
      "            1-D array of values representing the RHS of each equality constraint\n",
      "            (row) in A_eq.\n",
      "        bounds : sequence, optional\n",
      "            ``(min, max)`` pairs for each element in ``x``, defining\n",
      "            the bounds on that parameter. Use None for one of ``min`` or\n",
      "            ``max`` when there is no bound in that direction. By default\n",
      "            bounds are ``(0, None)`` (non-negative)\n",
      "            If a sequence containing a single tuple is provided, then ``min`` and\n",
      "            ``max`` will be applied to all variables in the problem.\n",
      "        method : str, optional\n",
      "            Type of solver.  At this time only 'simplex' is supported\n",
      "            :ref:`(see here) <optimize.linprog-simplex>`.\n",
      "        callback : callable, optional\n",
      "            If a callback function is provide, it will be called within each\n",
      "            iteration of the simplex algorithm. The callback must have the signature\n",
      "            `callback(xk, **kwargs)` where xk is the current solution vector\n",
      "            and kwargs is a dictionary containing the following::\n",
      "        \n",
      "                \"tableau\" : The current Simplex algorithm tableau\n",
      "                \"nit\" : The current iteration.\n",
      "                \"pivot\" : The pivot (row, column) used for the next iteration.\n",
      "                \"phase\" : Whether the algorithm is in Phase 1 or Phase 2.\n",
      "                \"basis\" : The indices of the columns of the basic variables.\n",
      "        \n",
      "        options : dict, optional\n",
      "            A dictionary of solver options. All methods accept the following\n",
      "            generic options:\n",
      "        \n",
      "                maxiter : int\n",
      "                    Maximum number of iterations to perform.\n",
      "                disp : bool\n",
      "                    Set to True to print convergence messages.\n",
      "        \n",
      "            For method-specific options, see `show_options('linprog')`.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        A `scipy.optimize.OptimizeResult` consisting of the following fields:\n",
      "        \n",
      "            x : ndarray\n",
      "                The independent variable vector which optimizes the linear\n",
      "                programming problem.\n",
      "            slack : ndarray\n",
      "                The values of the slack variables.  Each slack variable corresponds\n",
      "                to an inequality constraint.  If the slack is zero, then the\n",
      "                corresponding constraint is active.\n",
      "            success : bool\n",
      "                Returns True if the algorithm succeeded in finding an optimal\n",
      "                solution.\n",
      "            status : int\n",
      "                An integer representing the exit status of the optimization::\n",
      "        \n",
      "                     0 : Optimization terminated successfully\n",
      "                     1 : Iteration limit reached\n",
      "                     2 : Problem appears to be infeasible\n",
      "                     3 : Problem appears to be unbounded\n",
      "        \n",
      "            nit : int\n",
      "                The number of iterations performed.\n",
      "            message : str\n",
      "                A string descriptor of the exit status of the optimization.\n",
      "        \n",
      "        See Also\n",
      "        --------\n",
      "        show_options : Additional options accepted by the solvers\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        This section describes the available solvers that can be selected by the\n",
      "        'method' parameter. The default method is :ref:`Simplex <optimize.linprog-simplex>`.\n",
      "        \n",
      "        Method *Simplex* uses the Simplex algorithm (as it relates to Linear\n",
      "        Programming, NOT the Nelder-Mead Simplex) [1]_, [2]_. This algorithm\n",
      "        should be reasonably reliable and fast.\n",
      "        \n",
      "        .. versionadded:: 0.15.0\n",
      "        \n",
      "        References\n",
      "        ----------\n",
      "        .. [1] Dantzig, George B., Linear programming and extensions. Rand\n",
      "               Corporation Research Study Princeton Univ. Press, Princeton, NJ, 1963\n",
      "        .. [2] Hillier, S.H. and Lieberman, G.J. (1995), \"Introduction to\n",
      "               Mathematical Programming\", McGraw-Hill, Chapter 4.\n",
      "        .. [3] Bland, Robert G. New finite pivoting rules for the simplex method.\n",
      "               Mathematics of Operations Research (2), 1977: pp. 103-107.\n",
      "        \n",
      "        Examples\n",
      "        --------\n",
      "        Consider the following problem:\n",
      "        \n",
      "        Minimize: f = -1*x[0] + 4*x[1]\n",
      "        \n",
      "        Subject to: -3*x[0] + 1*x[1] <= 6\n",
      "                     1*x[0] + 2*x[1] <= 4\n",
      "                                x[1] >= -3\n",
      "        \n",
      "        where:  -inf <= x[0] <= inf\n",
      "        \n",
      "        This problem deviates from the standard linear programming problem.\n",
      "        In standard form, linear programming problems assume the variables x are\n",
      "        non-negative.  Since the variables don't have standard bounds where\n",
      "        0 <= x <= inf, the bounds of the variables must be explicitly set.\n",
      "        \n",
      "        There are two upper-bound constraints, which can be expressed as\n",
      "        \n",
      "        dot(A_ub, x) <= b_ub\n",
      "        \n",
      "        The input for this problem is as follows:\n",
      "        \n",
      "        >>> c = [-1, 4]\n",
      "        >>> A = [[-3, 1], [1, 2]]\n",
      "        >>> b = [6, 4]\n",
      "        >>> x0_bounds = (None, None)\n",
      "        >>> x1_bounds = (-3, None)\n",
      "        >>> from scipy.optimize import linprog\n",
      "        >>> res = linprog(c, A_ub=A, b_ub=b, bounds=(x0_bounds, x1_bounds),\n",
      "        ...               options={\"disp\": True})\n",
      "        >>> print(res)\n",
      "        Optimization terminated successfully.\n",
      "             Current function value: -11.428571\n",
      "             Iterations: 2\n",
      "        status: 0\n",
      "        success: True\n",
      "        fun: -11.428571428571429\n",
      "        x: array([-1.14285714,  2.57142857])\n",
      "        message: 'Optimization terminated successfully.'\n",
      "        nit: 2\n",
      "        \n",
      "        Note the actual objective value is 11.428571.  In this case we minimized\n",
      "        the negative of the objective function.\n",
      "    \n",
      "    linprog_verbose_callback(xk, **kwargs)\n",
      "        A sample callback function demonstrating the linprog callback interface.\n",
      "        This callback produces detailed output to sys.stdout before each iteration\n",
      "        and after the final iteration of the simplex algorithm.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        xk : array_like\n",
      "            The current solution vector.\n",
      "        **kwargs : dict\n",
      "            A dictionary containing the following parameters:\n",
      "        \n",
      "            tableau : array_like\n",
      "                The current tableau of the simplex algorithm.\n",
      "                Its structure is defined in _solve_simplex.\n",
      "            phase : int\n",
      "                The current Phase of the simplex algorithm (1 or 2)\n",
      "            nit : int\n",
      "                The current iteration number.\n",
      "            pivot : tuple(int, int)\n",
      "                The index of the tableau selected as the next pivot,\n",
      "                or nan if no pivot exists\n",
      "            basis : array(int)\n",
      "                A list of the current basic variables.\n",
      "                Each element contains the name of a basic variable and its value.\n",
      "            complete : bool\n",
      "                True if the simplex algorithm has completed\n",
      "                (and this is the final call to callback), otherwise False.\n",
      "    \n",
      "    minimize(fun, x0, args=(), method=None, jac=None, hess=None, hessp=None, bounds=None, constraints=(), tol=None, callback=None, options=None)\n",
      "        Minimization of scalar function of one or more variables.\n",
      "        \n",
      "        In general, the optimization problems are of the form:\n",
      "        \n",
      "        minimize f(x)\n",
      "        \n",
      "        subject to:\n",
      "        \n",
      "            ``g_i(x) >= 0``, i = 1,...,m\n",
      "            ``h_j(x)  = 0``, j = 1,...,p\n",
      "        \n",
      "        Where x is a vector of one or more variables.\n",
      "        ``g_i(x)`` are the inequality constraints.\n",
      "        ``h_j(x)`` are the equality constrains.\n",
      "        \n",
      "        Optionally, the lower and upper bounds for each element in x can also be specified \n",
      "        using the `bounds` argument.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        fun : callable\n",
      "            Objective function.\n",
      "        x0 : ndarray\n",
      "            Initial guess.\n",
      "        args : tuple, optional\n",
      "            Extra arguments passed to the objective function and its\n",
      "            derivatives (Jacobian, Hessian).\n",
      "        method : str or callable, optional\n",
      "            Type of solver.  Should be one of\n",
      "        \n",
      "                - 'Nelder-Mead' :ref:`(see here) <optimize.minimize-neldermead>`\n",
      "                - 'Powell'      :ref:`(see here) <optimize.minimize-powell>`\n",
      "                - 'CG'          :ref:`(see here) <optimize.minimize-cg>`\n",
      "                - 'BFGS'        :ref:`(see here) <optimize.minimize-bfgs>`\n",
      "                - 'Newton-CG'   :ref:`(see here) <optimize.minimize-newtoncg>`\n",
      "                - 'L-BFGS-B'    :ref:`(see here) <optimize.minimize-lbfgsb>`\n",
      "                - 'TNC'         :ref:`(see here) <optimize.minimize-tnc>`\n",
      "                - 'COBYLA'      :ref:`(see here) <optimize.minimize-cobyla>`\n",
      "                - 'SLSQP'       :ref:`(see here) <optimize.minimize-slsqp>`\n",
      "                - 'dogleg'      :ref:`(see here) <optimize.minimize-dogleg>`\n",
      "                - 'trust-ncg'   :ref:`(see here) <optimize.minimize-trustncg>`\n",
      "                - custom - a callable object (added in version 0.14.0),\n",
      "                  see below for description.\n",
      "        \n",
      "            If not given, chosen to be one of ``BFGS``, ``L-BFGS-B``, ``SLSQP``,\n",
      "            depending if the problem has constraints or bounds.\n",
      "        jac : bool or callable, optional\n",
      "            Jacobian (gradient) of objective function. Only for CG, BFGS,\n",
      "            Newton-CG, L-BFGS-B, TNC, SLSQP, dogleg, trust-ncg.\n",
      "            If `jac` is a Boolean and is True, `fun` is assumed to return the\n",
      "            gradient along with the objective function. If False, the\n",
      "            gradient will be estimated numerically.\n",
      "            `jac` can also be a callable returning the gradient of the\n",
      "            objective. In this case, it must accept the same arguments as `fun`.\n",
      "        hess, hessp : callable, optional\n",
      "            Hessian (matrix of second-order derivatives) of objective function or\n",
      "            Hessian of objective function times an arbitrary vector p.  Only for\n",
      "            Newton-CG, dogleg, trust-ncg.\n",
      "            Only one of `hessp` or `hess` needs to be given.  If `hess` is\n",
      "            provided, then `hessp` will be ignored.  If neither `hess` nor\n",
      "            `hessp` is provided, then the Hessian product will be approximated\n",
      "            using finite differences on `jac`. `hessp` must compute the Hessian\n",
      "            times an arbitrary vector.\n",
      "        bounds : sequence, optional\n",
      "            Bounds for variables (only for L-BFGS-B, TNC and SLSQP).\n",
      "            ``(min, max)`` pairs for each element in ``x``, defining\n",
      "            the bounds on that parameter. Use None for one of ``min`` or\n",
      "            ``max`` when there is no bound in that direction.\n",
      "        constraints : dict or sequence of dict, optional\n",
      "            Constraints definition (only for COBYLA and SLSQP).\n",
      "            Each constraint is defined in a dictionary with fields:\n",
      "                type : str\n",
      "                    Constraint type: 'eq' for equality, 'ineq' for inequality.\n",
      "                fun : callable\n",
      "                    The function defining the constraint.\n",
      "                jac : callable, optional\n",
      "                    The Jacobian of `fun` (only for SLSQP).\n",
      "                args : sequence, optional\n",
      "                    Extra arguments to be passed to the function and Jacobian.\n",
      "            Equality constraint means that the constraint function result is to\n",
      "            be zero whereas inequality means that it is to be non-negative.\n",
      "            Note that COBYLA only supports inequality constraints.\n",
      "        tol : float, optional\n",
      "            Tolerance for termination. For detailed control, use solver-specific\n",
      "            options.\n",
      "        options : dict, optional\n",
      "            A dictionary of solver options. All methods accept the following\n",
      "            generic options:\n",
      "                maxiter : int\n",
      "                    Maximum number of iterations to perform.\n",
      "                disp : bool\n",
      "                    Set to True to print convergence messages.\n",
      "            For method-specific options, see :func:`show_options()`.\n",
      "        callback : callable, optional\n",
      "            Called after each iteration, as ``callback(xk)``, where ``xk`` is the\n",
      "            current parameter vector.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        res : OptimizeResult\n",
      "            The optimization result represented as a ``OptimizeResult`` object.\n",
      "            Important attributes are: ``x`` the solution array, ``success`` a\n",
      "            Boolean flag indicating if the optimizer exited successfully and\n",
      "            ``message`` which describes the cause of the termination. See\n",
      "            `OptimizeResult` for a description of other attributes.\n",
      "        \n",
      "        \n",
      "        See also\n",
      "        --------\n",
      "        minimize_scalar : Interface to minimization algorithms for scalar\n",
      "            univariate functions\n",
      "        show_options : Additional options accepted by the solvers\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        This section describes the available solvers that can be selected by the\n",
      "        'method' parameter. The default method is *BFGS*.\n",
      "        \n",
      "        **Unconstrained minimization**\n",
      "        \n",
      "        Method :ref:`Nelder-Mead <optimize.minimize-neldermead>` uses the\n",
      "        Simplex algorithm [1]_, [2]_. This algorithm has been successful\n",
      "        in many applications but other algorithms using the first and/or\n",
      "        second derivatives information might be preferred for their better\n",
      "        performances and robustness in general.\n",
      "        \n",
      "        Method :ref:`Powell <optimize.minimize-powell>` is a modification\n",
      "        of Powell's method [3]_, [4]_ which is a conjugate direction\n",
      "        method. It performs sequential one-dimensional minimizations along\n",
      "        each vector of the directions set (`direc` field in `options` and\n",
      "        `info`), which is updated at each iteration of the main\n",
      "        minimization loop. The function need not be differentiable, and no\n",
      "        derivatives are taken.\n",
      "        \n",
      "        Method :ref:`CG <optimize.minimize-cg>` uses a nonlinear conjugate\n",
      "        gradient algorithm by Polak and Ribiere, a variant of the\n",
      "        Fletcher-Reeves method described in [5]_ pp.  120-122. Only the\n",
      "        first derivatives are used.\n",
      "        \n",
      "        Method :ref:`BFGS <optimize.minimize-bfgs>` uses the quasi-Newton\n",
      "        method of Broyden, Fletcher, Goldfarb, and Shanno (BFGS) [5]_\n",
      "        pp. 136. It uses the first derivatives only. BFGS has proven good\n",
      "        performance even for non-smooth optimizations. This method also\n",
      "        returns an approximation of the Hessian inverse, stored as\n",
      "        `hess_inv` in the OptimizeResult object.\n",
      "        \n",
      "        Method :ref:`Newton-CG <optimize.minimize-newtoncg>` uses a\n",
      "        Newton-CG algorithm [5]_ pp. 168 (also known as the truncated\n",
      "        Newton method). It uses a CG method to the compute the search\n",
      "        direction. See also *TNC* method for a box-constrained\n",
      "        minimization with a similar algorithm.\n",
      "        \n",
      "        Method :ref:`dogleg <optimize.minimize-dogleg>` uses the dog-leg\n",
      "        trust-region algorithm [5]_ for unconstrained minimization. This\n",
      "        algorithm requires the gradient and Hessian; furthermore the\n",
      "        Hessian is required to be positive definite.\n",
      "        \n",
      "        Method :ref:`trust-ncg <optimize.minimize-trustncg>` uses the\n",
      "        Newton conjugate gradient trust-region algorithm [5]_ for\n",
      "        unconstrained minimization. This algorithm requires the gradient\n",
      "        and either the Hessian or a function that computes the product of\n",
      "        the Hessian with a given vector.\n",
      "        \n",
      "        **Constrained minimization**\n",
      "        \n",
      "        Method :ref:`L-BFGS-B <optimize.minimize-lbfgsb>` uses the L-BFGS-B\n",
      "        algorithm [6]_, [7]_ for bound constrained minimization.\n",
      "        \n",
      "        Method :ref:`TNC <optimize.minimize-tnc>` uses a truncated Newton\n",
      "        algorithm [5]_, [8]_ to minimize a function with variables subject\n",
      "        to bounds. This algorithm uses gradient information; it is also\n",
      "        called Newton Conjugate-Gradient. It differs from the *Newton-CG*\n",
      "        method described above as it wraps a C implementation and allows\n",
      "        each variable to be given upper and lower bounds.\n",
      "        \n",
      "        Method :ref:`COBYLA <optimize.minimize-cobyla>` uses the\n",
      "        Constrained Optimization BY Linear Approximation (COBYLA) method\n",
      "        [9]_, [10]_, [11]_. The algorithm is based on linear\n",
      "        approximations to the objective function and each constraint. The\n",
      "        method wraps a FORTRAN implementation of the algorithm. The\n",
      "        constraints functions 'fun' may return either a single number\n",
      "        or an array or list of numbers.\n",
      "        \n",
      "        Method :ref:`SLSQP <optimize.minimize-slsqp>` uses Sequential\n",
      "        Least SQuares Programming to minimize a function of several\n",
      "        variables with any combination of bounds, equality and inequality\n",
      "        constraints. The method wraps the SLSQP Optimization subroutine\n",
      "        originally implemented by Dieter Kraft [12]_. Note that the\n",
      "        wrapper handles infinite values in bounds by converting them into\n",
      "        large floating values.\n",
      "        \n",
      "        **Custom minimizers**\n",
      "        \n",
      "        It may be useful to pass a custom minimization method, for example\n",
      "        when using a frontend to this method such as `scipy.optimize.basinhopping`\n",
      "        or a different library.  You can simply pass a callable as the ``method``\n",
      "        parameter.\n",
      "        \n",
      "        The callable is called as ``method(fun, x0, args, **kwargs, **options)``\n",
      "        where ``kwargs`` corresponds to any other parameters passed to `minimize`\n",
      "        (such as `callback`, `hess`, etc.), except the `options` dict, which has\n",
      "        its contents also passed as `method` parameters pair by pair.  Also, if\n",
      "        `jac` has been passed as a bool type, `jac` and `fun` are mangled so that\n",
      "        `fun` returns just the function values and `jac` is converted to a function\n",
      "        returning the Jacobian.  The method shall return an ``OptimizeResult``\n",
      "        object.\n",
      "        \n",
      "        The provided `method` callable must be able to accept (and possibly ignore)\n",
      "        arbitrary parameters; the set of parameters accepted by `minimize` may\n",
      "        expand in future versions and then these parameters will be passed to\n",
      "        the method.  You can find an example in the scipy.optimize tutorial.\n",
      "        \n",
      "        .. versionadded:: 0.11.0\n",
      "        \n",
      "        References\n",
      "        ----------\n",
      "        .. [1] Nelder, J A, and R Mead. 1965. A Simplex Method for Function\n",
      "            Minimization. The Computer Journal 7: 308-13.\n",
      "        .. [2] Wright M H. 1996. Direct search methods: Once scorned, now\n",
      "            respectable, in Numerical Analysis 1995: Proceedings of the 1995\n",
      "            Dundee Biennial Conference in Numerical Analysis (Eds. D F\n",
      "            Griffiths and G A Watson). Addison Wesley Longman, Harlow, UK.\n",
      "            191-208.\n",
      "        .. [3] Powell, M J D. 1964. An efficient method for finding the minimum of\n",
      "           a function of several variables without calculating derivatives. The\n",
      "           Computer Journal 7: 155-162.\n",
      "        .. [4] Press W, S A Teukolsky, W T Vetterling and B P Flannery.\n",
      "           Numerical Recipes (any edition), Cambridge University Press.\n",
      "        .. [5] Nocedal, J, and S J Wright. 2006. Numerical Optimization.\n",
      "           Springer New York.\n",
      "        .. [6] Byrd, R H and P Lu and J. Nocedal. 1995. A Limited Memory\n",
      "           Algorithm for Bound Constrained Optimization. SIAM Journal on\n",
      "           Scientific and Statistical Computing 16 (5): 1190-1208.\n",
      "        .. [7] Zhu, C and R H Byrd and J Nocedal. 1997. L-BFGS-B: Algorithm\n",
      "           778: L-BFGS-B, FORTRAN routines for large scale bound constrained\n",
      "           optimization. ACM Transactions on Mathematical Software 23 (4):\n",
      "           550-560.\n",
      "        .. [8] Nash, S G. Newton-Type Minimization Via the Lanczos Method.\n",
      "           1984. SIAM Journal of Numerical Analysis 21: 770-778.\n",
      "        .. [9] Powell, M J D. A direct search optimization method that models\n",
      "           the objective and constraint functions by linear interpolation.\n",
      "           1994. Advances in Optimization and Numerical Analysis, eds. S. Gomez\n",
      "           and J-P Hennart, Kluwer Academic (Dordrecht), 51-67.\n",
      "        .. [10] Powell M J D. Direct search algorithms for optimization\n",
      "           calculations. 1998. Acta Numerica 7: 287-336.\n",
      "        .. [11] Powell M J D. A view of algorithms for optimization without\n",
      "           derivatives. 2007.Cambridge University Technical Report DAMTP\n",
      "           2007/NA03\n",
      "        .. [12] Kraft, D. A software package for sequential quadratic\n",
      "           programming. 1988. Tech. Rep. DFVLR-FB 88-28, DLR German Aerospace\n",
      "           Center -- Institute for Flight Mechanics, Koln, Germany.\n",
      "        \n",
      "        Examples\n",
      "        --------\n",
      "        Let us consider the problem of minimizing the Rosenbrock function. This\n",
      "        function (and its respective derivatives) is implemented in `rosen`\n",
      "        (resp. `rosen_der`, `rosen_hess`) in the `scipy.optimize`.\n",
      "        \n",
      "        >>> from scipy.optimize import minimize, rosen, rosen_der\n",
      "        \n",
      "        A simple application of the *Nelder-Mead* method is:\n",
      "        \n",
      "        >>> x0 = [1.3, 0.7, 0.8, 1.9, 1.2]\n",
      "        >>> res = minimize(rosen, x0, method='Nelder-Mead')\n",
      "        >>> res.x\n",
      "        [ 1.  1.  1.  1.  1.]\n",
      "        \n",
      "        Now using the *BFGS* algorithm, using the first derivative and a few\n",
      "        options:\n",
      "        \n",
      "        >>> res = minimize(rosen, x0, method='BFGS', jac=rosen_der,\n",
      "        ...                options={'gtol': 1e-6, 'disp': True})\n",
      "        Optimization terminated successfully.\n",
      "                 Current function value: 0.000000\n",
      "                 Iterations: 52\n",
      "                 Function evaluations: 64\n",
      "                 Gradient evaluations: 64\n",
      "        >>> res.x\n",
      "        array([ 1.  1.  1.  1.  1.])\n",
      "        >>> print(res.message)\n",
      "        Optimization terminated successfully.\n",
      "        >>> res.hess_inv\n",
      "        [[ 0.00749589  0.01255155  0.02396251  0.04750988  0.09495377]\n",
      "         [ 0.01255155  0.02510441  0.04794055  0.09502834  0.18996269]\n",
      "         [ 0.02396251  0.04794055  0.09631614  0.19092151  0.38165151]\n",
      "         [ 0.04750988  0.09502834  0.19092151  0.38341252  0.7664427 ]\n",
      "         [ 0.09495377  0.18996269  0.38165151  0.7664427   1.53713523]]\n",
      "        \n",
      "        \n",
      "        Next, consider a minimization problem with several constraints (namely\n",
      "        Example 16.4 from [5]_). The objective function is:\n",
      "        \n",
      "        >>> fun = lambda x: (x[0] - 1)**2 + (x[1] - 2.5)**2\n",
      "        \n",
      "        There are three constraints defined as:\n",
      "        \n",
      "        >>> cons = ({'type': 'ineq', 'fun': lambda x:  x[0] - 2 * x[1] + 2},\n",
      "        ...         {'type': 'ineq', 'fun': lambda x: -x[0] - 2 * x[1] + 6},\n",
      "        ...         {'type': 'ineq', 'fun': lambda x: -x[0] + 2 * x[1] + 2})\n",
      "        \n",
      "        And variables must be positive, hence the following bounds:\n",
      "        \n",
      "        >>> bnds = ((0, None), (0, None))\n",
      "        \n",
      "        The optimization problem is solved using the SLSQP method as:\n",
      "        \n",
      "        >>> res = minimize(fun, (2, 0), method='SLSQP', bounds=bnds,\n",
      "        ...                constraints=cons)\n",
      "        \n",
      "        It should converge to the theoretical solution (1.4 ,1.7).\n",
      "    \n",
      "    minimize_scalar(fun, bracket=None, bounds=None, args=(), method='brent', tol=None, options=None)\n",
      "        Minimization of scalar function of one variable.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        fun : callable\n",
      "            Objective function.\n",
      "            Scalar function, must return a scalar.\n",
      "        bracket : sequence, optional\n",
      "            For methods 'brent' and 'golden', `bracket` defines the bracketing\n",
      "            interval and can either have three items `(a, b, c)` so that `a < b\n",
      "            < c` and `fun(b) < fun(a), fun(c)` or two items `a` and `c` which\n",
      "            are assumed to be a starting interval for a downhill bracket search\n",
      "            (see `bracket`); it doesn't always mean that the obtained solution\n",
      "            will satisfy `a <= x <= c`.\n",
      "        bounds : sequence, optional\n",
      "            For method 'bounded', `bounds` is mandatory and must have two items\n",
      "            corresponding to the optimization bounds.\n",
      "        args : tuple, optional\n",
      "            Extra arguments passed to the objective function.\n",
      "        method : str or callable, optional\n",
      "            Type of solver.  Should be one of\n",
      "        \n",
      "                - 'Brent'     :ref:`(see here) <optimize.minimize_scalar-brent>`\n",
      "                - 'Bounded'   :ref:`(see here) <optimize.minimize_scalar-bounded>`\n",
      "                - 'Golden'    :ref:`(see here) <optimize.minimize_scalar-golden>`\n",
      "                - custom - a callable object (added in version 0.14.0),\n",
      "                  see below\n",
      "        tol : float, optional\n",
      "            Tolerance for termination. For detailed control, use solver-specific\n",
      "            options.\n",
      "        options : dict, optional\n",
      "            A dictionary of solver options.\n",
      "                maxiter : int\n",
      "                    Maximum number of iterations to perform.\n",
      "                disp : bool\n",
      "                    Set to True to print convergence messages.\n",
      "        \n",
      "            See :func:`show_options()` for solver-specific options.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        res : OptimizeResult\n",
      "            The optimization result represented as a ``OptimizeResult`` object.\n",
      "            Important attributes are: ``x`` the solution array, ``success`` a\n",
      "            Boolean flag indicating if the optimizer exited successfully and\n",
      "            ``message`` which describes the cause of the termination. See\n",
      "            `OptimizeResult` for a description of other attributes.\n",
      "        \n",
      "        See also\n",
      "        --------\n",
      "        minimize : Interface to minimization algorithms for scalar multivariate\n",
      "            functions\n",
      "        show_options : Additional options accepted by the solvers\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        This section describes the available solvers that can be selected by the\n",
      "        'method' parameter. The default method is *Brent*.\n",
      "        \n",
      "        Method :ref:`Brent <optimize.minimize_scalar-brent>` uses Brent's\n",
      "        algorithm to find a local minimum.  The algorithm uses inverse\n",
      "        parabolic interpolation when possible to speed up convergence of\n",
      "        the golden section method.\n",
      "        \n",
      "        Method :ref:`Golden <optimize.minimize_scalar-golden>` uses the\n",
      "        golden section search technique. It uses analog of the bisection\n",
      "        method to decrease the bracketed interval. It is usually\n",
      "        preferable to use the *Brent* method.\n",
      "        \n",
      "        Method :ref:`Bounded <optimize.minimize_scalar-bounded>` can\n",
      "        perform bounded minimization. It uses the Brent method to find a\n",
      "        local minimum in the interval x1 < xopt < x2.\n",
      "        \n",
      "        **Custom minimizers**\n",
      "        \n",
      "        It may be useful to pass a custom minimization method, for example\n",
      "        when using some library frontend to minimize_scalar.  You can simply\n",
      "        pass a callable as the ``method`` parameter.\n",
      "        \n",
      "        The callable is called as ``method(fun, args, **kwargs, **options)``\n",
      "        where ``kwargs`` corresponds to any other parameters passed to `minimize`\n",
      "        (such as `bracket`, `tol`, etc.), except the `options` dict, which has\n",
      "        its contents also passed as `method` parameters pair by pair.  The method\n",
      "        shall return an ``OptimizeResult`` object.\n",
      "        \n",
      "        The provided `method` callable must be able to accept (and possibly ignore)\n",
      "        arbitrary parameters; the set of parameters accepted by `minimize` may\n",
      "        expand in future versions and then these parameters will be passed to\n",
      "        the method.  You can find an example in the scipy.optimize tutorial.\n",
      "        \n",
      "        .. versionadded:: 0.11.0\n",
      "        \n",
      "        Examples\n",
      "        --------\n",
      "        Consider the problem of minimizing the following function.\n",
      "        \n",
      "        >>> def f(x):\n",
      "        ...     return (x - 2) * x * (x + 2)**2\n",
      "        \n",
      "        Using the *Brent* method, we find the local minimum as:\n",
      "        \n",
      "        >>> from scipy.optimize import minimize_scalar\n",
      "        >>> res = minimize_scalar(f)\n",
      "        >>> res.x\n",
      "        1.28077640403\n",
      "        \n",
      "        Using the *Bounded* method, we find a local minimum with specified\n",
      "        bounds as:\n",
      "        \n",
      "        >>> res = minimize_scalar(f, bounds=(-3, -1), method='bounded')\n",
      "        >>> res.x\n",
      "        -2.0000002026\n",
      "    \n",
      "    newton(func, x0, fprime=None, args=(), tol=1.48e-08, maxiter=50, fprime2=None)\n",
      "        Find a zero using the Newton-Raphson or secant method.\n",
      "        \n",
      "        Find a zero of the function `func` given a nearby starting point `x0`.\n",
      "        The Newton-Raphson method is used if the derivative `fprime` of `func`\n",
      "        is provided, otherwise the secant method is used.  If the second order\n",
      "        derivate `fprime2` of `func` is provided, parabolic Halley's method\n",
      "        is used.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        func : function\n",
      "            The function whose zero is wanted. It must be a function of a\n",
      "            single variable of the form f(x,a,b,c...), where a,b,c... are extra\n",
      "            arguments that can be passed in the `args` parameter.\n",
      "        x0 : float\n",
      "            An initial estimate of the zero that should be somewhere near the\n",
      "            actual zero.\n",
      "        fprime : function, optional\n",
      "            The derivative of the function when available and convenient. If it\n",
      "            is None (default), then the secant method is used.\n",
      "        args : tuple, optional\n",
      "            Extra arguments to be used in the function call.\n",
      "        tol : float, optional\n",
      "            The allowable error of the zero value.\n",
      "        maxiter : int, optional\n",
      "            Maximum number of iterations.\n",
      "        fprime2 : function, optional\n",
      "            The second order derivative of the function when available and\n",
      "            convenient. If it is None (default), then the normal Newton-Raphson\n",
      "            or the secant method is used. If it is given, parabolic Halley's\n",
      "            method is used.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        zero : float\n",
      "            Estimated location where function is zero.\n",
      "        \n",
      "        See Also\n",
      "        --------\n",
      "        brentq, brenth, ridder, bisect\n",
      "        fsolve : find zeroes in n dimensions.\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        The convergence rate of the Newton-Raphson method is quadratic,\n",
      "        the Halley method is cubic, and the secant method is\n",
      "        sub-quadratic.  This means that if the function is well behaved\n",
      "        the actual error in the estimated zero is approximately the square\n",
      "        (cube for Halley) of the requested tolerance up to roundoff\n",
      "        error. However, the stopping criterion used here is the step size\n",
      "        and there is no guarantee that a zero has been found. Consequently\n",
      "        the result should be verified. Safer algorithms are brentq,\n",
      "        brenth, ridder, and bisect, but they all require that the root\n",
      "        first be bracketed in an interval where the function changes\n",
      "        sign. The brentq algorithm is recommended for general use in one\n",
      "        dimensional problems when such an interval has been found.\n",
      "    \n",
      "    newton_krylov(F, xin, iter=None, rdiff=None, method='lgmres', inner_maxiter=20, inner_M=None, outer_k=10, verbose=False, maxiter=None, f_tol=None, f_rtol=None, x_tol=None, x_rtol=None, tol_norm=None, line_search='armijo', callback=None, **kw)\n",
      "        Find a root of a function, using Krylov approximation for inverse Jacobian.\n",
      "        \n",
      "        This method is suitable for solving large-scale problems.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        F : function(x) -> f\n",
      "            Function whose root to find; should take and return an array-like\n",
      "            object.\n",
      "        x0 : array_like\n",
      "            Initial guess for the solution\n",
      "        rdiff : float, optional\n",
      "            Relative step size to use in numerical differentiation.\n",
      "        method : {'lgmres', 'gmres', 'bicgstab', 'cgs', 'minres'} or function\n",
      "            Krylov method to use to approximate the Jacobian.\n",
      "            Can be a string, or a function implementing the same interface as\n",
      "            the iterative solvers in `scipy.sparse.linalg`.\n",
      "        \n",
      "            The default is `scipy.sparse.linalg.lgmres`.\n",
      "        inner_M : LinearOperator or InverseJacobian\n",
      "            Preconditioner for the inner Krylov iteration.\n",
      "            Note that you can use also inverse Jacobians as (adaptive)\n",
      "            preconditioners. For example,\n",
      "        \n",
      "            >>> from scipy.optimize.nonlin import BroydenFirst, KrylovJacobian\n",
      "            >>> from scipy.optimize.nonlin import InverseJacobian\n",
      "            >>> jac = BroydenFirst()\n",
      "            >>> kjac = KrylovJacobian(inner_M=InverseJacobian(jac))\n",
      "        \n",
      "            If the preconditioner has a method named 'update', it will be called\n",
      "            as ``update(x, f)`` after each nonlinear step, with ``x`` giving\n",
      "            the current point, and ``f`` the current function value.\n",
      "        inner_tol, inner_maxiter, ...\n",
      "            Parameters to pass on to the \\\"inner\\\" Krylov solver.\n",
      "            See `scipy.sparse.linalg.gmres` for details.\n",
      "        outer_k : int, optional\n",
      "            Size of the subspace kept across LGMRES nonlinear iterations.\n",
      "            See `scipy.sparse.linalg.lgmres` for details.\n",
      "        iter : int, optional\n",
      "            Number of iterations to make. If omitted (default), make as many\n",
      "            as required to meet tolerances.\n",
      "        verbose : bool, optional\n",
      "            Print status to stdout on every iteration.\n",
      "        maxiter : int, optional\n",
      "            Maximum number of iterations to make. If more are needed to\n",
      "            meet convergence, `NoConvergence` is raised.\n",
      "        f_tol : float, optional\n",
      "            Absolute tolerance (in max-norm) for the residual.\n",
      "            If omitted, default is 6e-6.\n",
      "        f_rtol : float, optional\n",
      "            Relative tolerance for the residual. If omitted, not used.\n",
      "        x_tol : float, optional\n",
      "            Absolute minimum step size, as determined from the Jacobian\n",
      "            approximation. If the step size is smaller than this, optimization\n",
      "            is terminated as successful. If omitted, not used.\n",
      "        x_rtol : float, optional\n",
      "            Relative minimum step size. If omitted, not used.\n",
      "        tol_norm : function(vector) -> scalar, optional\n",
      "            Norm to use in convergence check. Default is the maximum norm.\n",
      "        line_search : {None, 'armijo' (default), 'wolfe'}, optional\n",
      "            Which type of a line search to use to determine the step size in the\n",
      "            direction given by the Jacobian approximation. Defaults to 'armijo'.\n",
      "        callback : function, optional\n",
      "            Optional callback function. It is called on every iteration as\n",
      "            ``callback(x, f)`` where `x` is the current solution and `f`\n",
      "            the corresponding residual.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        sol : ndarray\n",
      "            An array (of similar array type as `x0`) containing the final solution.\n",
      "        \n",
      "        Raises\n",
      "        ------\n",
      "        NoConvergence\n",
      "            When a solution was not found.\n",
      "        \n",
      "        See Also\n",
      "        --------\n",
      "        scipy.sparse.linalg.gmres\n",
      "        scipy.sparse.linalg.lgmres\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        This function implements a Newton-Krylov solver. The basic idea is\n",
      "        to compute the inverse of the Jacobian with an iterative Krylov\n",
      "        method. These methods require only evaluating the Jacobian-vector\n",
      "        products, which are conveniently approximated by a finite difference:\n",
      "        \n",
      "        .. math:: J v \\approx (f(x + \\omega*v/|v|) - f(x)) / \\omega\n",
      "        \n",
      "        Due to the use of iterative matrix inverses, these methods can\n",
      "        deal with large nonlinear problems.\n",
      "        \n",
      "        Scipy's `scipy.sparse.linalg` module offers a selection of Krylov\n",
      "        solvers to choose from. The default here is `lgmres`, which is a\n",
      "        variant of restarted GMRES iteration that reuses some of the\n",
      "        information obtained in the previous Newton steps to invert\n",
      "        Jacobians in subsequent steps.\n",
      "        \n",
      "        For a review on Newton-Krylov methods, see for example [1]_,\n",
      "        and for the LGMRES sparse inverse method, see [2]_.\n",
      "        \n",
      "        References\n",
      "        ----------\n",
      "        .. [1] D.A. Knoll and D.E. Keyes, J. Comp. Phys. 193, 357 (2003).\n",
      "        .. [2] A.H. Baker and E.R. Jessup and T. Manteuffel,\n",
      "                 SIAM J. Matrix Anal. Appl. 26, 962 (2005).\n",
      "    \n",
      "    nnls(A, b)\n",
      "        Solve ``argmin_x || Ax - b ||_2`` for ``x>=0``. This is a wrapper\n",
      "        for a FORTAN non-negative least squares solver.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        A : ndarray\n",
      "            Matrix ``A`` as shown above.\n",
      "        b : ndarray\n",
      "            Right-hand side vector.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        x : ndarray\n",
      "            Solution vector.\n",
      "        rnorm : float\n",
      "            The residual, ``|| Ax-b ||_2``.\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        The FORTRAN code was published in the book below. The algorithm\n",
      "        is an active set method. It solves the KKT (Karush-Kuhn-Tucker)\n",
      "        conditions for the non-negative least squares problem.\n",
      "        \n",
      "        References\n",
      "        ----------\n",
      "        Lawson C., Hanson R.J., (1987) Solving Least Squares Problems, SIAM\n",
      "    \n",
      "    ridder(f, a, b, args=(), xtol=1e-12, rtol=4.4408920985006262e-16, maxiter=100, full_output=False, disp=True)\n",
      "        Find a root of a function in an interval.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        f : function\n",
      "            Python function returning a number.  f must be continuous, and f(a) and\n",
      "            f(b) must have opposite signs.\n",
      "        a : number\n",
      "            One end of the bracketing interval [a,b].\n",
      "        b : number\n",
      "            The other end of the bracketing interval [a,b].\n",
      "        xtol : number, optional\n",
      "            The routine converges when a root is known to lie within xtol of the\n",
      "            value return. Should be >= 0.  The routine modifies this to take into\n",
      "            account the relative precision of doubles.\n",
      "        rtol : number, optional\n",
      "            The routine converges when a root is known to lie within `rtol` times\n",
      "            the value returned of the value returned. Should be >= 0. Defaults to\n",
      "            ``np.finfo(float).eps * 2``.\n",
      "        maxiter : number, optional\n",
      "            if convergence is not achieved in maxiter iterations, an error is\n",
      "            raised.  Must be >= 0.\n",
      "        args : tuple, optional\n",
      "            containing extra arguments for the function `f`.\n",
      "            `f` is called by ``apply(f, (x)+args)``.\n",
      "        full_output : bool, optional\n",
      "            If `full_output` is False, the root is returned.  If `full_output` is\n",
      "            True, the return value is ``(x, r)``, where `x` is the root, and `r` is\n",
      "            a RootResults object.\n",
      "        disp : bool, optional\n",
      "            If True, raise RuntimeError if the algorithm didn't converge.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        x0 : float\n",
      "            Zero of `f` between `a` and `b`.\n",
      "        r : RootResults (present if ``full_output = True``)\n",
      "            Object containing information about the convergence.\n",
      "            In particular, ``r.converged`` is True if the routine converged.\n",
      "        \n",
      "        See Also\n",
      "        --------\n",
      "        brentq, brenth, bisect, newton : one-dimensional root-finding\n",
      "        fixed_point : scalar fixed-point finder\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        Uses [Ridders1979]_ method to find a zero of the function `f` between the\n",
      "        arguments `a` and `b`. Ridders' method is faster than bisection, but not\n",
      "        generally as fast as the Brent rountines. [Ridders1979]_ provides the\n",
      "        classic description and source of the algorithm. A description can also be\n",
      "        found in any recent edition of Numerical Recipes.\n",
      "        \n",
      "        The routine used here diverges slightly from standard presentations in\n",
      "        order to be a bit more careful of tolerance.\n",
      "        \n",
      "        References\n",
      "        ----------\n",
      "        .. [Ridders1979]\n",
      "           Ridders, C. F. J. \"A New Algorithm for Computing a\n",
      "           Single Root of a Real Continuous Function.\"\n",
      "           IEEE Trans. Circuits Systems 26, 979-980, 1979.\n",
      "    \n",
      "    root(fun, x0, args=(), method='hybr', jac=None, tol=None, callback=None, options=None)\n",
      "        Find a root of a vector function.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        fun : callable\n",
      "            A vector function to find a root of.\n",
      "        x0 : ndarray\n",
      "            Initial guess.\n",
      "        args : tuple, optional\n",
      "            Extra arguments passed to the objective function and its Jacobian.\n",
      "        method : str, optional\n",
      "            Type of solver.  Should be one of\n",
      "        \n",
      "                - 'hybr'             :ref:`(see here) <optimize.root-hybr>`\n",
      "                - 'lm'               :ref:`(see here) <optimize.root-lm>`\n",
      "                - 'broyden1'         :ref:`(see here) <optimize.root-broyden1>`\n",
      "                - 'broyden2'         :ref:`(see here) <optimize.root-broyden2>`\n",
      "                - 'anderson'         :ref:`(see here) <optimize.root-anderson>`\n",
      "                - 'linearmixing'     :ref:`(see here) <optimize.root-linearmixing>`\n",
      "                - 'diagbroyden'      :ref:`(see here) <optimize.root-diagbroyden>`\n",
      "                - 'excitingmixing'   :ref:`(see here) <optimize.root-excitingmixing>`\n",
      "                - 'krylov'           :ref:`(see here) <optimize.root-krylov>`\n",
      "                - 'df-sane'          :ref:`(see here) <optimize.root-dfsane>`\n",
      "        \n",
      "        jac : bool or callable, optional\n",
      "            If `jac` is a Boolean and is True, `fun` is assumed to return the\n",
      "            value of Jacobian along with the objective function. If False, the\n",
      "            Jacobian will be estimated numerically.\n",
      "            `jac` can also be a callable returning the Jacobian of `fun`. In\n",
      "            this case, it must accept the same arguments as `fun`.\n",
      "        tol : float, optional\n",
      "            Tolerance for termination. For detailed control, use solver-specific\n",
      "            options.\n",
      "        callback : function, optional\n",
      "            Optional callback function. It is called on every iteration as\n",
      "            ``callback(x, f)`` where `x` is the current solution and `f`\n",
      "            the corresponding residual. For all methods but 'hybr' and 'lm'.\n",
      "        options : dict, optional\n",
      "            A dictionary of solver options. E.g. `xtol` or `maxiter`, see\n",
      "            :obj:`show_options()` for details.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        sol : OptimizeResult\n",
      "            The solution represented as a ``OptimizeResult`` object.\n",
      "            Important attributes are: ``x`` the solution array, ``success`` a\n",
      "            Boolean flag indicating if the algorithm exited successfully and\n",
      "            ``message`` which describes the cause of the termination. See\n",
      "            `OptimizeResult` for a description of other attributes.\n",
      "        \n",
      "        See also\n",
      "        --------\n",
      "        show_options : Additional options accepted by the solvers\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        This section describes the available solvers that can be selected by the\n",
      "        'method' parameter. The default method is *hybr*.\n",
      "        \n",
      "        Method *hybr* uses a modification of the Powell hybrid method as\n",
      "        implemented in MINPACK [1]_.\n",
      "        \n",
      "        Method *lm* solves the system of nonlinear equations in a least squares\n",
      "        sense using a modification of the Levenberg-Marquardt algorithm as\n",
      "        implemented in MINPACK [1]_.\n",
      "        \n",
      "        Method *df-sane* is a derivative-free spectral method. [3]_\n",
      "        \n",
      "        Methods *broyden1*, *broyden2*, *anderson*, *linearmixing*,\n",
      "        *diagbroyden*, *excitingmixing*, *krylov* are inexact Newton methods,\n",
      "        with backtracking or full line searches [2]_. Each method corresponds\n",
      "        to a particular Jacobian approximations. See `nonlin` for details.\n",
      "        \n",
      "        - Method *broyden1* uses Broyden's first Jacobian approximation, it is\n",
      "          known as Broyden's good method.\n",
      "        - Method *broyden2* uses Broyden's second Jacobian approximation, it\n",
      "          is known as Broyden's bad method.\n",
      "        - Method *anderson* uses (extended) Anderson mixing.\n",
      "        - Method *Krylov* uses Krylov approximation for inverse Jacobian. It\n",
      "          is suitable for large-scale problem.\n",
      "        - Method *diagbroyden* uses diagonal Broyden Jacobian approximation.\n",
      "        - Method *linearmixing* uses a scalar Jacobian approximation.\n",
      "        - Method *excitingmixing* uses a tuned diagonal Jacobian\n",
      "          approximation.\n",
      "        \n",
      "        .. warning::\n",
      "        \n",
      "            The algorithms implemented for methods *diagbroyden*,\n",
      "            *linearmixing* and *excitingmixing* may be useful for specific\n",
      "            problems, but whether they will work may depend strongly on the\n",
      "            problem.\n",
      "        \n",
      "        .. versionadded:: 0.11.0\n",
      "        \n",
      "        References\n",
      "        ----------\n",
      "        .. [1] More, Jorge J., Burton S. Garbow, and Kenneth E. Hillstrom.\n",
      "           1980. User Guide for MINPACK-1.\n",
      "        .. [2] C. T. Kelley. 1995. Iterative Methods for Linear and Nonlinear\n",
      "            Equations. Society for Industrial and Applied Mathematics.\n",
      "            <http://www.siam.org/books/kelley/>\n",
      "        .. [3] W. La Cruz, J.M. Martinez, M. Raydan. Math. Comp. 75, 1429 (2006).\n",
      "        \n",
      "        Examples\n",
      "        --------\n",
      "        The following functions define a system of nonlinear equations and its\n",
      "        jacobian.\n",
      "        \n",
      "        >>> def fun(x):\n",
      "        ...     return [x[0]  + 0.5 * (x[0] - x[1])**3 - 1.0,\n",
      "        ...             0.5 * (x[1] - x[0])**3 + x[1]]\n",
      "        \n",
      "        >>> def jac(x):\n",
      "        ...     return np.array([[1 + 1.5 * (x[0] - x[1])**2,\n",
      "        ...                       -1.5 * (x[0] - x[1])**2],\n",
      "        ...                      [-1.5 * (x[1] - x[0])**2,\n",
      "        ...                       1 + 1.5 * (x[1] - x[0])**2]])\n",
      "        \n",
      "        A solution can be obtained as follows.\n",
      "        \n",
      "        >>> from scipy import optimize\n",
      "        >>> sol = optimize.root(fun, [0, 0], jac=jac, method='hybr')\n",
      "        >>> sol.x\n",
      "        array([ 0.8411639,  0.1588361])\n",
      "    \n",
      "    rosen(x)\n",
      "        The Rosenbrock function.\n",
      "        \n",
      "        The function computed is::\n",
      "        \n",
      "            sum(100.0*(x[1:] - x[:-1]**2.0)**2.0 + (1 - x[:-1])**2.0\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        x : array_like\n",
      "            1-D array of points at which the Rosenbrock function is to be computed.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        f : float\n",
      "            The value of the Rosenbrock function.\n",
      "        \n",
      "        See Also\n",
      "        --------\n",
      "        rosen_der, rosen_hess, rosen_hess_prod\n",
      "    \n",
      "    rosen_der(x)\n",
      "        The derivative (i.e. gradient) of the Rosenbrock function.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        x : array_like\n",
      "            1-D array of points at which the derivative is to be computed.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        rosen_der : (N,) ndarray\n",
      "            The gradient of the Rosenbrock function at `x`.\n",
      "        \n",
      "        See Also\n",
      "        --------\n",
      "        rosen, rosen_hess, rosen_hess_prod\n",
      "    \n",
      "    rosen_hess(x)\n",
      "        The Hessian matrix of the Rosenbrock function.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        x : array_like\n",
      "            1-D array of points at which the Hessian matrix is to be computed.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        rosen_hess : ndarray\n",
      "            The Hessian matrix of the Rosenbrock function at `x`.\n",
      "        \n",
      "        See Also\n",
      "        --------\n",
      "        rosen, rosen_der, rosen_hess_prod\n",
      "    \n",
      "    rosen_hess_prod(x, p)\n",
      "        Product of the Hessian matrix of the Rosenbrock function with a vector.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        x : array_like\n",
      "            1-D array of points at which the Hessian matrix is to be computed.\n",
      "        p : array_like\n",
      "            1-D array, the vector to be multiplied by the Hessian matrix.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        rosen_hess_prod : ndarray\n",
      "            The Hessian matrix of the Rosenbrock function at `x` multiplied\n",
      "            by the vector `p`.\n",
      "        \n",
      "        See Also\n",
      "        --------\n",
      "        rosen, rosen_der, rosen_hess\n",
      "    \n",
      "    show_options(solver=None, method=None, disp=True)\n",
      "        Show documentation for additional options of optimization solvers.\n",
      "        \n",
      "        These are method-specific options that can be supplied through the\n",
      "        ``options`` dict.\n",
      "        \n",
      "        Parameters\n",
      "        ----------\n",
      "        solver : str\n",
      "            Type of optimization solver. One of 'minimize', 'minimize_scalar',\n",
      "            'root', or 'linprog'.\n",
      "        method : str, optional\n",
      "            If not given, shows all methods of the specified solver. Otherwise,\n",
      "            show only the options for the specified method. Valid values\n",
      "            corresponds to methods' names of respective solver (e.g. 'BFGS' for\n",
      "            'minimize').\n",
      "        disp : bool, optional\n",
      "            Whether to print the result rather than returning it.\n",
      "        \n",
      "        Returns\n",
      "        -------\n",
      "        text\n",
      "            Either None (for disp=False) or the text string (disp=True)\n",
      "        \n",
      "        Notes\n",
      "        -----\n",
      "        The solver-specific methods are:\n",
      "        \n",
      "        `scipy.optimize.minimize`\n",
      "        \n",
      "        - :ref:`Nelder-Mead <optimize.minimize-neldermead>`\n",
      "        - :ref:`Powell      <optimize.minimize-powell>`\n",
      "        - :ref:`CG          <optimize.minimize-cg>`\n",
      "        - :ref:`BFGS        <optimize.minimize-bfgs>`\n",
      "        - :ref:`Newton-CG   <optimize.minimize-newtoncg>`\n",
      "        - :ref:`L-BFGS-B    <optimize.minimize-lbfgsb>`\n",
      "        - :ref:`TNC         <optimize.minimize-tnc>`\n",
      "        - :ref:`COBYLA      <optimize.minimize-cobyla>`\n",
      "        - :ref:`SLSQP       <optimize.minimize-slsqp>`\n",
      "        - :ref:`dogleg      <optimize.minimize-dogleg>`\n",
      "        - :ref:`trust-ncg   <optimize.minimize-trustncg>`\n",
      "        \n",
      "        `scipy.optimize.root`\n",
      "        \n",
      "        - :ref:`hybr              <optimize.root-hybr>`\n",
      "        - :ref:`lm                <optimize.root-lm>`\n",
      "        - :ref:`broyden1          <optimize.root-broyden1>`\n",
      "        - :ref:`broyden2          <optimize.root-broyden2>`\n",
      "        - :ref:`anderson          <optimize.root-anderson>`\n",
      "        - :ref:`linearmixing      <optimize.root-linearmixing>`\n",
      "        - :ref:`diagbroyden       <optimize.root-diagbroyden>`\n",
      "        - :ref:`excitingmixing    <optimize.root-excitingmixing>`\n",
      "        - :ref:`krylov            <optimize.root-krylov>`\n",
      "        - :ref:`df-sane           <optimize.root-dfsane>`\n",
      "        \n",
      "        `scipy.optimize.minimize_scalar`\n",
      "        \n",
      "        - :ref:`brent       <optimize.minimize_scalar-brent>`\n",
      "        - :ref:`golden      <optimize.minimize_scalar-golden>`\n",
      "        - :ref:`bounded     <optimize.minimize_scalar-bounded>`\n",
      "        \n",
      "        `scipy.optimize.linprog`\n",
      "        \n",
      "        - :ref:`simplex     <optimize.linprog-simplex>`\n",
      "\n",
      "DATA\n",
      "    __all__ = ['LbfgsInvHessProduct', 'OptimizeResult', 'OptimizeWarning',...\n",
      "    absolute_import = _Feature((2, 5, 0, 'alpha', 1), (3, 0, 0, 'alpha', 0...\n",
      "    division = _Feature((2, 2, 0, 'alpha', 2), (3, 0, 0, 'alpha', 0), 8192...\n",
      "    print_function = _Feature((2, 6, 0, 'alpha', 2), (3, 0, 0, 'alpha', 0)...\n",
      "\n",
      "FILE\n",
      "    /sw/lib/python3.4/site-packages/scipy/optimize/__init__.py\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import scipy.optimize\n",
    "help(scipy.optimize)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Curve Fitting\n",
    "\n",
    "#### Curve Fitting\n",
    "Find the parameters $\\vec{a}=(a_1, a_2, \\ldots, a_M)$ that make the function $f(x, \\vec{a})$ fit the data $(x_i, y_i)$ with standard deviations (errors) $\\sigma_i$ as well as possible.\n",
    "* A common approach is to adjust $\\vec{a}$ to minimize the sum of the squares of the weighted errors, called $\\chi^2$:\n",
    "\\begin{equation*}\n",
    "\\chi^2 = \\sum_{i=0}^{N} \\left[\\frac{y_i - f(x_i, \\vec{a})}{\\sigma_i}\\right]^2\n",
    "\\end{equation*}\n",
    "* For normally distributed data,\n",
    " minimum $\\chi^2 \\Rightarrow$ maximum likelihood.\n",
    "* $1/\\sigma_i^2$ = weighting $\\Rightarrow$ large errors contribute least.\n",
    "* Smaller $\\chi^2 \\Rightarrow$ better fit.\n",
    "* $\\chi^2 \\approx N - M =$ \\# degrees freedom, good.\n",
    "* Good fit: misses $\\sim 1/3$ of points.\n",
    "* $\\chi^2 = 0 \\Rightarrow$ theory passes through all data points."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example: Linear Regression\n",
    "Fitting data to a line (called Simple Linear Regression) is particularly easy.   \n",
    "e.g. fit a line, $y = m * x + c * 1$, through some noisy data-points:\n",
    "* Setup and make some data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "x = np.array([0, 1, 2, 3])\n",
    "y = np.array([-1, 0.2, 0.9, 2.1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* By inspecting or plotting, we see that the line should have a slope of roughly $1$ and cut the y-axis at $-1$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(-2, 3)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEACAYAAACqOy3+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADR9JREFUeJzt3V2IXVcZxvHnadKSSCKCvbAfEyNNBc2IbS5aq0Kn1NJk\naC29qFoQsZLcaJNQRKQV7BS8ULywTbxLP6iIbUGhWJ2mDTRDFUtaNdEkTW0TqJlWGsGPkNCEfuT1\n4pyYmWQ+zszes9Ze+/x/cMg5++zZ+2WRPNms/e41jggBAMpzXu4CAADzQ4ADQKEIcAAoFAEOAIUi\nwAGgUAQ4ABSqUoDbXmJ7l+09tvfZHqmpLgDALFy1D9z2ByLibduLJf1e0uaI2FVLdQCAaVWeQomI\nt7tvL5B0vqRTVY8JAJhd5QC3fZ7tPZKOSHo2Il6qXhYAYDZ1XIGfiogrJF0q6Wrbq6uXBQCYzeK6\nDhQRR23vlLRW0v7T222z2AoAzENEeKbvKwW47QslvRcR/7W9VNINkn441yL6he2xiBjKXUcT2B6J\niJHcdTQBY3EGY3FGLxe/Va/AL5L0qO1F6kzHPBERoxWP2WbX5i4AQHtUCvCI2CtpTU21AADmgCcx\nkctY7gIaZCx3AQ0ylruAklR+kGfWE9jBHHgHYwGgV73kBVfgAFAoAjyt+3IXAKA9mEIBgAZiCgUA\nWowAB4BCEeAAUCgCHAAKRYAnxG8sAlAnulASYiwA9IouFABoMQIcAApFgANAoQhwACgUAZ4Wa6EA\nqA1dKADQQHShAECLEeAAUCgCHAAKRYADQKEI8IRYCwVAnehCSYixANArulAAoMUIcAAoFAEOAIUi\nwAGgUAR4WqyFAqA2dKEAQAPRhQIALUaAA0ChCHAAKBQBDgCFIsATYi0UYGr24LC9brv9pbHOn4PD\nuWsqAV0oCTEWwLk6YX3NA9K2VWe2bjgovbA5Yt9ovsryogsFQAEGNk0Ob6nzecXGPPWUo1KA2x6w\nvdP2ftv7bG+qqzAA/WL5kqm3L1uato7yLK748+9Kuisi9theJulPtndExIEaagPQF46dnHr78RNp\n6yhPpSvwiHgrIvZ03x+XdEDSxXUUBqBfjG/pzHlPtP6QdHhrnnrKUfUK/P9sr5R0paRddR2zhVgL\nBThLxL5Re1DS8MbOtMnxE9Lhrf18A7NXtXShdKdPxiT9ICKePOu70OTgGouIsconBYAWsT0kaWjC\npntn60KpHOC2z5f0G0lPR8T9U3xP6xwAzFEv2VkpwG1b0qOS/hURd823CADAZCkC/POSnpf0V0mn\nD3R3RGyfSxEAgMkWPMDrKgIAMBlPYjYMa6EAqBNX4AkxFgB6xRU4ALQYAQ4AhSLAAaBQBDgAFIoA\nT4u1UADUhi4UAGggulAAoMUIcAAoFAEOAIUiwAGgUAR4QqyFAqBOdKEkxFgA6BVdKADQYgQ4ABSK\nAAeAQhHgAFAoAjwt1kIBUBu6UACggehCAYAWI8ABoFAEOAAUigAHgEIR4AmxFgqAOtGFkhBjAaBX\ndKEAQIsR4ABQKAIcAApFgANAoQjwtFgLBUBt6EIBgAaiCwUAWowAB4BCEeAAUKjKAW77YdtHbO+t\noyAAQG/quAJ/RNLaGo7TeqyFAqBOtXSh2F4p6amI+NQU39GF0sVYAOgVXSgA0GIEOAAUanGKk5w1\n9zsWEWMpzgsApbA9JGloTj/DHHg6jAUmsgeHpYFN0vIl0rGT0viWiH2juetCM/SSF5WvwG0/Jula\nSR+2PS7p+xHxSNXjthRroUDS6fC+5gFp26ozWzdcZg+KEEevWAsFyMBet116+sZzvxneHjG6Ln1F\naBq6UIDGWr5k6u3LlqatAyUjwIEsjp2cevvxE2nrQMkIcCCL8S3ShoOTt60/JB3emqcelIg5cCCT\nzo3MFRs70ybHT0iHt3IDE6f1kp0EeEK2RyJiJHcdAJqPAG8YxgJAr+hCAYAWI8ABoFAEOAAUigAH\ngEIR4GmxFgqA2tCFAgANRBcKALQYAQ4AhSLAAaBQBDgAFIoAT+is3w0KAJXQhZIQYwGgV3ShAECL\nEeAAUCgCHAAKRYADQKEI8LRYCwVAbehCAYAGogsFAFqMAAeAQhHgAFAoAhwACkWAJ8RaKADqRBdK\nQoyFZA8OSwObpOVLpGMnpfEtEftGc9cFNE0vebE4VTFAJ7yveUDaturM1g2X2YMixIG5YwoFCQ1s\nmhzeUufzio156gHKRoAjoeVLpt6+bGnaOoB2IMCR0LGTU28/fiJtHUA7EOBp9flaKONbpA0HJ29b\nf0g6vDVPPUDZ6EJBUp0bmSs2dqZNjp+QDm/lBiZwrl6ys3KA214r6X5JiyQ9GBE/mmsRAIDJFjzA\nbS+S9DdJX5D0pqSXJN0eEQfmUgQAYLIUqxFeJelgRLweEe9KelzSLRWPCQDoQdUAv0TS+ITPb3S3\nAQAWWNUAX9g7oC3DWigA6lT1Ufo3JQ1M+DygzlX4JGcF11hEjFU8b6nulTSSuwgAzWN7SNLQnH6m\n4k3MxercxLxe0j8kvShuYk6LsQDQqwVfzCoi3rN9p6Rn1GkjfGhieAMAFg4P8iTEWADoFb/UGABa\njABPq8/XQgFQJ6ZQAKCBmEIBgBYjwAGgUAQ4ABSKAAeAQhHgCbEWCoA60YWSEGMBoFd0oQBAixHg\nAFAoAhwACkWAA0ChCPC0WAsFQG3oQgGABqILBQBajAAHgEIR4ABQKAIcAApFgCfEWigA6kQXSkKM\nBYBe0YUCAC1GgANAoQhwACgUAQ4AhSLA02ItFAC1oQsFABqILhQAaDECHAAKRYADQKEIcAAoFAGe\nEGuhAKgTXSgJMRYAekUXCgC0GAEOAIUiwAGgUPMOcNu32d5v+33ba+osCgAwuypX4Hsl3Srp+Zpq\naS17cNhet11a9Xd73XZ7cDh3TQDKt3i+PxgRr0iSTVPFTDphfc0D0rZV3U0flTZcZg8qYt9o1uIA\nFI058AU3sGlCeHdtWyWt2JinHgCtERHTviTtUGeq5OzXzRP22SlpzQzHiGleI9PsP9LO/e8NKSa8\nbhsrq372Z3/2X8j9JQ11/zz9iqmOMfFV+UEe2zslfTsi/jzN9xF9/PBKZ+776RvP/WZ4e8TouvQV\nAShBygd5+jagZze+RdpwcPK29Yekw1vz1AOgLeZ9BW77VklbJF0o6aik3RFxzhVlv1+BS6dvZK7Y\nKL32CenyA9LhrdzABDCTXrKTtVASYiwA9Iq1UACgxQhwACgUAQ4AhSLAAaBQBHha9+UuAEB70IUC\nAA1EFwoAtBgBDgCFIsABoFAEOAAUigBPyPZI7hoAtAddKAkxFgB6RRcKALQYAQ4AhSLAAaBQBDgA\nFIoAT4u1UADUhi4UAGggulAAoMUIcAAoFAEOAIUiwAGgUAR4QqyFAqBOdKEkxFgA6BVdKADQYgQ4\nABSKAAeAQhHgAFAoAjwt1kIBUBu6UACggehCAYAWI8ABoFAEOAAUigAHgEIR4AmxFgqAOs27C8X2\njyXdJOkdSYck3RERR6fYjy6ULsYCQK8WugvlWUmrI+LTkl6VdHeFY6HP2B7KXUNTMBZnMBZzM+8A\nj4gdEXGq+3GXpEvrKQl9Yih3AQ0ylLuABhnKXUBJ6poD/4ak0ZqOBQDoweKZvrS9Q9JHpvjqnoh4\nqrvP9yS9ExG/WID6AADTqPQove2vS9og6fqIODnNPgv7rD4AtNRsNzFnvAKfie21kr4j6drpwruX\nAgAA81OljfA1SRdI+nd30wsR8c26CgMAzGzBVyMEACyMJE9i2r7N9n7b79tek+KcTWJ7re1XbL9m\n+7u568nJ9sO2j9jem7uWnGwP2N7Z/Xexz/am3DXlYnuJ7V2293THYiR3TbnZXmR7t+2nZtov1aP0\neyXdKun5ROdrDNuLJP1U0lpJn5R0u+1P5K0qq0fUGYt+966kuyJitaTPSPpWv/696N5Duy4irpB0\nhaS1tq/OXFZumyW9LGnGKZIkAR4Rr0TEqynO1UBXSToYEa9HxLuSHpd0S+aasomI30n6T+46couI\ntyJiT/f9cUkHJF2ct6p8IuLt7tsLJJ0v6dQMu7ea7UslDUt6UBK/0CGzSySNT/j8RncbIEmyvVLS\nleo80dyXbJ9ne4+kI5KejYiXcteU0U/U6fCb9T+x2gLc9g7be6d43VzXOQrFXWJMy/YySb+UtLl7\nJd6XIuJUdwrlUklX216du6YcbN8k6Z8RsVuzXH1LFfrAzxYRN9R1rJZ5U9LAhM8D6lyFo8/ZPl/S\nryT9PCKezF1PE0TEUds71blPsj93PRl8VtIXbQ9LWiLpg7Z/FhFfm2rnHFMo/fZgzx8lXW57pe0L\nJH1Z0q8z14TMbFvSQ5Jejoj7c9eTk+0LbX+o+36ppBvUuSfQdyLinogYiIiPSfqKpOemC28pXRvh\nrbbH1bnb/lvbT6c4bxNExHuS7pT0jDp3lZ+IiL78yylJth+T9AdJH7c9bvuO3DVl8jlJX5V0Xbdd\nbHf36eZ+dJGk52z/RdKL6syBszhex4xTsDzIAwCFogsFAApFgANAoQhwACgUAQ4AhSLAAaBQBDgA\nFIoAB4BCEeAAUKj/AbAzaOoa7vjvAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x112a82e10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(x,       y, linestyle='' , marker='o', color='blue', label='data')\n",
    "plt.axhline(-1, linestyle='dashed', color='black')\n",
    "plt.axvline(0, linestyle='dashed', color='black')\n",
    "plt.xlim(-1,4)\n",
    "plt.ylim(-2,3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* We can rewrite the line equation as $\\mathbf{A}\\vec{p} = \\vec{y}$,  \n",
    " where `A = [[x 1]]` and `p = [[m], [c]]`.  \n",
    "Now use `lstsq` to solve for `p`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "A = np.column_stack([x, np.ones(len(x))])\n",
    "m, c = np.linalg.lstsq(A, y)[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 1.  , -0.95])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.linalg.lstsq(A, y)[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example: Linear Regression, Plotting\n",
    "* Plot the data along with the fitted line:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(-2, 3)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAEPCAYAAACqZsSmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFDxJREFUeJzt3X+wXGd93/H3R/6BFCx+xUkgIENBuCFWEiDhhxpSKyEE\nW46TgRgoGYZxiJVpQiQnLUwTKEUdEiZpMwmS3MlQgSm0NQ62kwymsrFbEJDwyxDs2MYE2x0i4RST\nFnClIhHb+vaPs7Kurq6u9kr33ufsnvdrRqPdc87ufrWj+93Pfc6z50lVIUkalhWtC5AkLT+bvyQN\nkM1fkgbI5i9JA2Tzl6QBsvlL0gA1a/5JVib5TJJbk9yRZGurWiRpaNJynn+S76qqbyc5HfgL4PKq\n+kyzgiRpIJoO+1TVt0c3zwTOAA41LEeSBqNp80+yIsmtwP3ATVV1S8t6JGkoWif/Q1X1bOApwAuS\nnNeyHkkaitNbFwBQVQ8k+ShwAXDn4e1JvPCQJJ2Eqsp8+5s1/yRnAw9V1beSrAJeAvze7ONO9A8Y\niiRbq2pr6zr6wPfiCN+LI3wvjhgnOLdM/k8C3pvkNLrhpz+pql0N65GkwWjW/KvqduC5rV5fkobM\nb/hOjt2tC+iR3a0L6JHdrQvokd2tC5gkTb/kdSJJyjF/SVqYcXpnL2b7LJSzgNrzQ1mabBPZ/MHm\n05IfvtLkc8xfkgbI5i9JA2Tzl6QBsvkvsST/KcnbWtchSTPZ/Jdejf7MK8nuJL+8DPVI0uTO9jlZ\nybqNsGYLrF4J+w7C3u1Vdyz1ZSXGmZnkDBpJy2ZQyb9r/Ou3wQ0vhQ+c3/29flu3fbFeI89J8ldJ\n/m+Sq4GVo+2PS/KhJF9P8o0k1yd58mjf7wI/AVyRZF+S7aPt25LsSfJAks8ledFi1Slp2AbV/LvE\nv3Pt0dt2roVzNi/Gsyc5E/hz4L3A44FrgF+gS/UrgHcD54z+HACuAKiqNwOfAF5fVaurasvoKT8L\n/Mjoua4Crhm9hiSdkoE1/9Ur595+1qpFeoEXAqdX1baqeriqrgNuAaiqb1TVn1XVwaraD7wdOH/W\n448aHqqq/1pV3xwtevOHwKOAf7xItUoasIE1/30H596+/8AivcD3A/fN2va3AElWJXlnkq8keQD4\nGPDYJDMb/lHj/knekOSLSb6V5JvAY4GzF6lWSQM2sOa/dztsuufobZfdC3t2LNIL/C/gybO2PZUu\n0b8BOBd4flU9li71hyNpf3bj/wngjcArqupxVfV44AHGO3ksSfMa1Gyfqjt2JeuAjZu7oZ79B2DP\njkWc7fNJ4KEkW4A/Bi4Gngd8BDiLbpz/gSRPAN4667H3A8+YcX818BDwv0fj/L8FPGaR6pQ0cBN5\nSec+X+o5yY8CO4G1wC66RH833YfBVcCP0Q0N/eFo2xlVdSjJC+lOFH8P8D7gX4ye5xLg/wF/BPwq\ncFlVfWQ5/02z9fn9lzTez6jNXwvm+y/12zg/owMb85ckgc1fkgbJ5i9JA2Tzl6QBsvlL0gDZ/CVp\ngGz+kjRANn9JGiCb/wRZyJKQowvIvfg4+1aN1hP4VpIPJPnFJB9e3Gol9dmgru0zBcZaEnKMYy8B\nvhd4QlUdGm276vDOJIeAtVX1P0+2UEn9ZvKfPItxWYWnAl+e0fiX6nUk9ZTNf5GNhlvekOS2JPuT\nvCvJ9yW5YbS0481JHjfj+J9LcmeSbyb5aJIfmLFvziUhZ+z/2SS3jh77l0l+aIz6/i3wFuBVoyUj\nX5fk0iSfGO3/+OjQ20b7X7EY74ukfrH5L74CXg78NN31+y+mu7rnb9FdsXMFsAUgybl0wy1b6BZp\n2QVcn+T0EywJSZLn0C0LuQl4AvBO4INJzpi3uKq30q0idvVoycgrZ+3/p6ObPzzaf83JvxWS+mp6\nmn9SS/5nfDuq6u+r6u/o1ub9dFXdVlXfAf4MeM7ouFcBH6qq/1FVDwN/AKwCfpx5loQc+RXgnVV1\nS3XeB3xn9LgTvls4rCMN2vSc8O3XJYbvn3H7wKz7B+kWdoFu2cc9h3dUVSXZS7ca2MMcZ0nIkacC\nr00yc/H5M0bPKUnzmp7m32/H+2C6D3hknH60nu8a4KujTXMtCXl4Gco9wO9W1dtPop7+LuIgLVCy\nbiOs2QKrV3brdO/dvoir802t6Rn2mUzXABcl+anRWP2/pPvN4JPApxktCZnkjCQvp1sS8rCdwD9P\n8vx0Hp3koiRnHfMqxzrRb0mzl5SUeqlr/Ou3wQ0vhQ+c3/29flu3XfOx+S+PmnW7AKrqb4DXADuA\nvwcuAi6uqoeq6h/oThxfCvwf4JXAdY88SdXn6U72XgF8g26pyNcyXqqf/R2A2fe3Au8dzSK6ZNx/\npLT81myBnWuP3rZzLZyzee7jdVizZRyTrKFbq/Z76RrPf6yq7bOOcRnHHvL9V18kr9zdJf7ZXvmx\nqg9sWO56+qLvyzg+CPxmVZ1HN0Pl9Ume1bAeSRNn38FwiF/jP/A9fH3G9v0HmpU0IZo1/6r6WlXd\nOrq9H7gLZ6pIWoDXcPs1n+KJB1/N+1nF4X5/2b2wZ0fTwiZAs2Gfo4pIngZ8DDhv9EFweLvDPj3k\n+6/mkhXAZuAtN/Ho6y7iRec8xGNWdYl/z46hz/YZ52e0+VTP0eyUa4HLZzb+Gfu3zri7u6p2L1Np\nkvooWQtcSTdysf5nav/dDzYuqbUkG4ANC3pMy+Q/mt74IeCGqnrHHPtN/j3k+68mZqR94HeAHXTf\njNcsvU7+oy80vRv44lyNX5IeMSvtU3V344omXsupni8CPg78NUfmmP92Vd0445jjJv/lqVLHY/LX\nsjDtn5Rxkn8vTvgej8ML0oAdnfZ/ybQ/vr7P85ekYyUrSC6nu8TJnwLn2/gXX/PZPpL0CMf2l43J\nX1J7pv1lZ/KX1JZpvwmTv6Q2TPtNmfwlLb8u7b+Hbm0J034DJn9Jy+fotH8dpv1mTP6Slodpv1dM\n/pKWlmm/l0z+kpaOab+3TP6SFp9pv/dM/pIWl2l/Ipj8JS0O0/5EMflLOnWm/Ylj8pd08kz7E8vk\nL+nkHEn7YNqfOCZ/SQtzdNq/FtP+RDL5SxqfaX9qmPwlnZhpf+qY/CXNz7Q/lUz+kuZm2p9qJn9J\nxzLtTz2Tv6QjTPuDYfKX1DHtD4rJXxo60/4gmfylITPtD5bJXxqiLu3/Bqb9wTL5S0OTPBO4cnTP\ntD9QJn9pKI6k/U9h2h88k780BKZ9zWLyl6aZaV/HYfKXppVpX/Mw+UvTxrSvMZj8pWli2teYTP7S\nNDDta4GaJv8kVwIXAV+vqh9qWYs0sUz7Ogmtk/97gAsa1yBNJtO+TkHT5F9Vn0jytJY1SBPJtK9T\n1Dr5S1oI074WibN9pElh2tci6n3zT7J1xt3dVbW7USlSG8kKYAvwr4G3ATuoOtS2KPVJkg3AhgU9\npqqWpJixC+jG/K+fa7ZPkqqqLHtRUl8cnfZfdzjtJ+s2wpotsHol7DsIe7dX3bGrXaHqk3F6Z+up\nnu8Hzge+O8le4N9U1XtO8DBp+s2T9rvGv34b7Fx75AGbnpGsww8Ajat58p+PyV+DdJy0f2T3hTfC\nDS899oEbb6zadeHSF6i+G6d3OttH6ouxZ/KsXjn3E5y1agmr05Tp/QlfaRAWNJNn38G5t+8/sNhl\naXqZ/KWWTmre/t7tsOmeo7dddi/s2bFEVWoKOeYvtXKCsf35H7puI5yzuRvq2X8A9uzwZK8OG6d3\n2vyl5ea8fS2x3k/1lAbHb+mqJxzzl5aD1+RRz5j8paVm2lcPmfylpWLaV4+Z/KWlYNpXz5n8pcVk\n2teEMPlLi8W0rwli8pdOlWlfE8jkL50K074mlMlfOhmmfU04k7+0UMla4PCiQ6Z9TaQTJv8kW5I8\nfjmKkXqtS/uXA5/GtK8JN07y/z7gliR/RTe2+eHq89XgpKVg2teUOWHyr6o3A+fSNf5LgbuTvD3J\nM5a4Nqk9076m1Fhj/lV1KMnXgPuBh4HHA9cm+e9V9calLFBqxrSvKTbOmP/lST4P/DvgL4F1VfWr\nwI8CL1/i+qTlZ9rXAIyT/J8AvLyq/nbmxtFvAxcvTVlSI6Z9DYQreUlweHWtzcBbcHUtTThX8pLG\nYdrXAPkNXw2XY/saMJO/hsm0r4Ez+WtYTPsSYPLXkJj2pUeY/DX9TPvSMUz+mhjJuo2wZgusXgn7\nDsLe7VV37DrBg0z70hxs/poIXeNfvw12rj2yddMzknXM+QHgvH1pXn7JSxMhufBGuOGlx+7ZeGPV\nrgtnHTwz7b/OtK+hGad3OuavCbF65dzbz1r1yE3H9qWxOeyjCbHv4Nzb9x8AHNuXFsjkrwmxdzts\nuufobZfdeyZfucK0Ly2cY/6aGN1J33M2d0M9+w+8hr++9j9z36Wj3Y7tSyPj9M6mzT/JBcA7gNOA\nd1XV78/ab/PXsZzJI82r180/yWnA3wA/DdwH3AK8uqrumnGMzV9HcyaPdEJ9n+3zfOCeqvpKVT0I\nXA38fMN61GdHz+S5Dsf2pVPScrbPk4G9M+5/FXhBo1rUZ87kkRZdy+Tf3zPN6gfn7UtLpmXyvw9Y\nM+P+Grr0f5QkW2fc3V1Vu5e2LPWCaV8aW5INwIYFPabhCd/T6U74vhj4O+CzeMJXzuSRTlmv1/Ct\nqoeS/DrwYbqpnu+e2fg1QKZ9adn4JS+1Z9qXFlWvk78EmPalRry2j9pwJo/UlMlfy8+0LzVn8tfy\nMe1LvWHy1/Iw7Uu9YvLX0jLtS71k8tfSMe1LvWXy1+Iz7Uu9Z/LX4jLtSxPB5K/FYdqXJorJX6fO\ntC9NHJO/Tp5pX5pYJn+dHNO+NNFM/loY0740FUz+Gp9pX5oaJn+dmGlfmjomf83PtC9NJZO/5mba\nl6aayV/HMu1LU8/kryNM+9JgmPzVMe1Lg2LyHzrTvjRIJv8hM+1Lg2XyHyLTvjR4Jv+hMe1LwuQ/\nHKZ9STOY/IfAtC9pFpP/NDPtSzoOk/+0Mu1LmofJf9qY9iWNweQ/TUz7ksZk8p8Gpn1JC2Tyn3Sm\nfUknweQ/qUz7kk6ByX8SmfYlnSKT/yQx7UtaJE2af5JXJLkzycNJntuihonTpf2PAZfQpf1tVB1q\nXJWkCdUq+d8OvAz4eKPXnxjJuo1vyrl3fIsz7vodnn72o3jW75n2JZ2qJmP+VfUlgCQtXn5iJOs2\nwvptn+eStc/j6dzDM38ANr0jWVdVd+xqXZ+kyeUJ315bswV2rr3pqG0718LGzYDNX9JJW7Lmn+Rm\n4Ilz7HpTVV2/gOfZOuPu7qrafYqlTZDVK+feftaq5a1DUp8l2QBsWMhjlqz5V9VLFul5ti7G80ym\nfQfn3r7/wPLWIanPRqF49+H7Sd56osf0YaqnA//HtXc7bLrn6G2X3Qt7drSpR9K0SFUt/4smLwO2\nA2cDDwBfqKoL5ziuqmrQHw7dSd9zNndDPfsPwJ4dnuyVNJ9xemeT5j8um78kLdw4vbMPwz6SpGVm\n85ekAbL5S9IA2fwlaYBs/pI0QDZ/SRogm78kDZDNX5IGyOYvSQNk85ekAbL5S9IA2fwlaYBs/pI0\nQDZ/SRogm78kDZDNX5IGyOYvSQNk85ekAbL5S9IA2fwlaYBs/pI0QDZ/SRogm78kDZDNX5IGyOYv\nSQNk85ekAbL5S9IA2fwlaYBs/pI0QDZ/SRogm78kDZDNX5IGyOYvSQNk85ekAbL5S9IANWn+Sf59\nkruS3JbkT5M8tkUdkjRUrZL/TcB5VfUjwJeB325Ux8RIsqF1DX3he3GE78URvhcL06T5V9XNVXVo\ndPczwFNa1DFhNrQuoEc2tC6gRza0LqBHNrQuYJL0Ycz/dcCu1kVI0pCcvlRPnORm4Ilz7HpTVV0/\nOubNwD9U1VVLVYck6VipqjYvnFwKbAJeXFUHj3NMm+IkacJVVebbv2TJfz5JLgDeCJx/vMYPJy5e\nknRymiT/JHcDZwLfGG36VFX92rIXIkkD1WzYR5LUTh9m+8wrySuS3Jnk4STPbV3PcktyQZIvJbk7\nyb9qXU9LSa5Mcn+S21vX0lKSNUk+Ovq5uCPJltY1tZJkZZLPJLl19F5sbV1Ta0lOS/KFJNfPd1zv\nmz9wO/Ay4OOtC1luSU4DrgAuAH4QeHWSZ7Wtqqn30L0XQ/cg8JtVdR7wQuD1Q/1/MTpn+JNV9Wzg\n2cAFSV7QuKzWLge+CMw7rNP75l9VX6qqL7euo5HnA/dU1Veq6kHgauDnG9fUTFV9Avhm6zpaq6qv\nVdWto9v7gbuA729bVTtV9e3RzTOBM4BD8xw+1ZI8BdgIvAuYd8JM75v/wD0Z2Dvj/ldH2yQAkjwN\neA7dN+UHKcmKJLcC9wM3VdUtrWtq6I/oZlKe8AOwF80/yc1Jbp/jz8Wta2vMs/E6riRnAdcCl49+\nAxikqjo0GvZ5CvCCJOe1rqmFJD8LfL2qvsAJUj80muc/W1W9pHUNPXUfsGbG/TV06V8Dl+QM4Drg\nv1TVn7eupw+q6oEkH6U7L3Rn63oa+CfAzyXZCKwEHpPkfVX12rkO7kXyX4Chfenrc8AzkzwtyZnA\nq4APNq5JjSUJ8G7gi1X1jtb1tJTk7CSPG91eBbyE7hzI4FTVm6pqTVX9I+CfAR85XuOHCWj+SV6W\nZC/drIb/luSG1jUtl6p6CPh14MN0Z+//pKoG+R8bIMn7gU8C5ybZm+SXWtfUyI8DrwF+cjSl7wuj\nb80P0ZOAjyS5Dfgs3Zi/F4rszDts7Je8JGmAep/8JUmLz+YvSQNk85ekAbL5S9IA2fwlaYBs/pI0\nQDZ/SRogm78kDZDNX1qAJM9LcluSRyV59GgBkR9sXZe0UH7DV1qgJG+ju3DWKmBvVf1+45KkBbP5\nSws0uqLm54ADwPryh0gTyGEfaeHOBh4NnEWX/qWJY/KXFijJB4GrgKcDT6qqzY1LkhasF4u5SJMi\nyWuB71TV1UlWAJ9MsqGqdjcuTVoQk78kDZBj/pI0QDZ/SRogm78kDZDNX5IGyOYvSQNk85ekAbL5\nS9IA2fwlaYD+P6kF2YKSWdj2AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x112abb1d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(x,       y, linestyle='' , marker='o', color='blue', label='data')\n",
    "plt.plot(x, m*x + c, linestyle='-', color='red', label='model fit')\n",
    "plt.xlabel('x')\n",
    "plt.ylabel('y')\n",
    "plt.legend(numpoints=1, loc='upper left')\n",
    "plt.xlim(-1,4)\n",
    "plt.ylim(-2,3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Linear Regression means linear in the *coefficients*.\n",
    "### It doesn't have to be a straight line."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's fit $y = a + b x + c x^2$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "x = np.array([0, 0.5, 1, 1.2, 2, 2.7, 3])\n",
    "(a, b, c) = (1.5, 2, 4)\n",
    "y = np.polyval([c,b,a], x)  # polyval expects coefficients from largest to smallest power of x\n",
    "# Add some noise\n",
    "(mu, sigma) = (0, 1)\n",
    "y += np.random.normal(mu, sigma, size=len(x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[  0.24946239   4.00398772   7.68805203  10.41142068  21.24199763\n",
      "  36.22010636  43.72138877]\n"
     ]
    }
   ],
   "source": [
    "print(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "A = np.column_stack([x**2, x, np.ones(len(x))])\n",
    "(fit_c, fit_b, fit_a) = np.linalg.lstsq(A, y)[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.60595950093 3.4173196633 0.733048821144\n"
     ]
    }
   ],
   "source": [
    "print(fit_c, fit_b, fit_a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(-1, 4)"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEPCAYAAABFpK+YAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuclWW9/vHPBYqgkIc8pnhkmymkVJrmgclSEQ+h5jFP\neawU2JmlWSZpJ21rIZbbFE0rNc00caOpKUU/f6UWknjIMt2iKR5Sg4RS+O4/7mdiOQ7DzLCeda81\nz/V+vebFWs9ai7layXfu+a77oIjAzMyqo1/uAGZm1lgu/GZmFePCb2ZWMS78ZmYV48JvZlYxLvxm\nZhVTeuGX1F/STElTi/sTJT1dXJspaXTZGczMbIkVGvA9JgAPA0OK+wFcEBEXNOB7m5lZB6WO+CVt\nAIwBLgPUfrnmtpmZNVjZrZ5vAZ8FFtdcC2CcpFmSpkhareQMZmZWo7TCL2lv4PmImMmbR/gXA5sA\n2wDPAueXlcHMzN5KZe3VI+lrwBHAG8BA4G3ADRFxZM1zNgamRsSITl7vTYTMzHohIrpsp5dW+N/0\nTaRRwKkRsY+k9SLi2eL6p4FtI+KwTl4TywpfFZImRsTE3Dmagd+LJfxeLOH3Yonu1M5GzOqB1Opp\n/wlznqSti/tPACc2KIOZmdGgwh8R04Hpxe0jGvE9zazvk4aPgaHjYdgW0p7bw5wLI2ZPy52r2TVq\nxG/LZ3ruAE1keu4ATWR67gA5paK/wyS4dFh6K9o2guM3k4bj4t+1hvT4e8M9fjPrirTnbXDrHm99\nZMxtEdP2bHyi5tBMPf668Wyf/PwD2ZrDkIEAm/I4f2FTlswaHzwoX6bW0HKFH1x4cvIPXmse8xa+\ng2e4l+0YzmyeY73i+vwFWWO1AO/OaWYtas6F57PL3y/hxJqif9zj8NTkrLFaQEuO+M3MgoeeW4gW\nbcC9d8BBA9JI/6nJ/mB32Vruw11/6JuX339rCpKAu4Grifhe7jjNpDv/Rt3qKZGk70s6J3cOsz7o\nI8DbgctzB2lFLvzlCpasWF4qSdMlHduAPGatTxoAfBP4DBFv5I7TiirV41+yym/IQJi3sEGr/LrT\nFmnOfptZc/oU8BgRt+cO0qoqM+Jfssrv1j3gulHpzx0mpev1+h4aKen3kv4u6VrSrqRIWk3SLZKe\nl/Q3SVMlrV889lVgZ+AiSfMkXVhcnyTpKUmvSrpf0k71ymnWsqS3A2eQzvmwXqpM4U8j/UuHvfna\npcNgw3H1+NuVfv28CbgSWB24HjiANJrvB0wBNiy+FgAXAUTEF4AZwEkRMSQixhd/5b3A1sXfdTVw\nffE9zKrsS8D1RDycO0grq1DhT6v83qpuq/y2B1aIiEkRsSgibgDuA4iIv0XEjRGxMCLmA18DRnV4\n/ZtaQhHxo4h4OSIWF+cTrwS8s05ZzVqP9E7gY8DEzElaXoUK/7yFnV+v2yq/dwDPdLj2vwCSBkm6\nRNKTkl4FfgmsqjQlrd2b+vySTpX0sKRXJL0MrAqsWaesZq3oPOBcIl7IHaTVVajwz7kQjv/zm6/V\ndZXfs8D6Ha5tRBrJnwpsDmwXEauSRvu1h853LPo7k3qYB0bEahGxOvAq3fug2KzvkXYFRgAX5o7S\nF1RmVk/E7GnScGDMuNTeqfsqv3uANySNJ50rvA+wLXAXMJjU139V0hrAWR1eOxfYrOb+ENKRlS8W\nff3TSUdXmlWP1J90NvfniPhn7jh9Qekjfkn9Jc2UNLW4v4akOyQ9Jul2SauVnaFdxOxpEdP2jLiu\nLf1Zv6mcEfE6sD9wNPAScBBwA2k0/21gEPAi6QfErbx5lD8J+Ggx4+fbwG3F12PAk6QfGk/VK6tZ\nizkK+Afp35PVQelbNkg6BXgvMCQi9pV0HvBiRJwn6TRg9Yg4vZPXecuGJuT33xpKGkwaAI0l4t7c\ncVpB9i0bJG0AjAEuY0l/el/SlEeKP8eWmcHMWtrngLtc9Our7B7/t0gfUtb2p9eJiLnF7bnAOiVn\nMLNWJA0FTgJG5o7S15RW+CXtDTwfETMltXX2nIiIrg72kDSx5u704tB2M6uGrwLfJcKfb3WhqK9t\nPXpNWT1+SV8DjiDNThlIGvX/lDTTpS0inpO0HnB3RGzRyevd429Cfv+tIaRtgZ8Bm5MWPVo3Ze3x\nR8QZETE0IjYBDgHuiogjgJtJn9JT/HlTWRnMrAWlhY0XAGe66JejkQu42n+1+Aawm6THgF2L+2Zm\n7fYndQi+nzlHn+UTuKxH/P5bqaSVgIeBE4j4Re44rSj7dE4zsx46GXjIRb9cLvwtoifHOBabwX1o\nKY8NKs4DeEXSdZIOk/Tz+qY16wVpLdL2JN5rv2SV2aunD+jWMY7deO5HgbWBNSJicXHt6vYHJS0G\nhkXEX3ob1KyXzgKuIeKPuYP0dS78raUevfWNgMdqin5Z38es+6R3AQcDb5nabfXnVk8dFS2WUyXN\nkjRf0mWS1pF0a3Ec4x21m9JJ2lfSQ5JelnS3pC1qHuv0GMeax/eW9EDx2v8naUQ38n0ZOBM4uDjm\n8RhJR0uaUTz+q+Kps4rHD6zH+2LWDd8Evk7ES7mDVIELf30FaSrah0n77+8DTCP1Ldcivd/jASRt\nTmqxjCcdsDINmCpphWUc44ikkaSjHI8H1gAuAW6WtGKX4SLOIp3+dW1xzOPlHR7fpbj57uLx63v/\nVph1k7QbaaT/ndxRqqJvFH4pSv/qvskR8UJE/JV0lu5vImJWpH3Eb2TJviMHA7dExC8iYhHwX6St\nm3eki2McCycAl0TEfZFcBfyzeN0y3y3cyrFmsWSv/c96r/3G6Rs9/uaaVz635vaCDvcXkg5lgXRU\n47/3ICn2LZpDOsVrEUs5xrGwEXCkpNqD4lcs/k6zVnIM8DJewd9QfaPwN7el/VB6hnSUXHpSWqY+\nFHi6uNTZMY7tR0c+BXw1Ir7WizzNuWLPqkcaApwN7E2zriTto/pGq6c1XQ/sJWnXojf/GdJvBPcA\nv6E4xlHSipL2J21u1+5S4BOStlOyiqS9lA6tWJZl/XbU8RhIs7KcDtxOxO9yB6kaF/7yRYfbARBp\nrvLhwGTgBWAvYJ+IeCMi/kXnxzhSvPZ3pA92LwL+BvwJOJLujeY7zvHveH8icGUxW+ij3f0fadYj\n0kbAJ4Av5I5SRd6rx3rE778tD2n4GBg6/n94YOQTDJh3MkPG1/Psa/NePWbWRFLR32HS+zlrj3fT\nf+3P8fBmsMOkdN0ayYXfzBpk6HhxybALOIUv8hVeYxXg0mGw4bhlvtTqyoXfzBpkyMATuYR+LOYH\nHFFzffCgbJEqytM5zawhtuRZncOZ7MwMFtO/5pH5C7KFqqhSR/ySBkr6bbGnzOz2w9MlTZT0tKSZ\nxdfoMnOYWWaS7uQ3b/sOG770KO+qeeC4x+GpydlyVVTps3okrRwRr0laAfg1MAEYDcyLiAu6eJ1n\n9TQhv//WK9IJwHFvY4uz57HJSam9M38BPDXZs3rqqzv/Rktv9UTEa8XNAaRtBdp/0vS6eKhne+eY\nWU5pzv5Xgba/xyMPAbdkTlR5pRd+Sf2A35NWg14UEfdK2hMYJ+lI4H7gMxHxSnf+Po82zVpI2ork\nUuBbRDyUO44lDVvAJWlV0u6U40grVV8oHjoHWC8iju3w/AC+XHNpekRMb0BUM6sX6Vjgk8D2RLyR\nO05fJKkNaKu5dNayBsgNXbkr6UzgtYg4v+baxsDUiBjR4bnuJZu1Mmko6bf9XYl4MHecqsi+clfS\nmu0nTkkaBOwGPCJp3Zqn7Qf4PwqzviS1eL4HXOii33zK7vGvR9rwqz/ph8yPI2KapKskbUP6oPcJ\n4MSSc5hZYx0NrAt8I3MO60TLbdJmZk1OWh+YCexGxKzccaome6vHzComtXguAb7rot+8vGWDmdXT\nEaST5PbPHcSWzq0eM6sPaT1gFrAHETNzx6kqt3rMrDGWtHj+20W/+bnVY2b1cBiwCeDjOluAWz1m\ntnzSupxZwF5E3J87TtW51WNm5UotnouBy1z0W4dbPWa2PA4GNgcOyR3Eus+F38x6R1oHmATsQ8Q/\nc8ex7nOrx8x66yLgCiLuzR3EesYjfjPrOelAYDi86dR0axEu/GbWM9JawGRgLBELc8exnvN0TjPr\nGelaYA4Rn80dxd6qKc7cNbM+RDoAGAl8PHcU6z0XfjPrHmlN0ge6HyViQe441ntu9ZhZ90g/AuYS\ncUruKLZ0WVs9kgYCvwRWKr7PTyJioqQ1gB8DGwFPAgdFxCtl5TCzOpDGAtsBW+eOYsuv1BG/pJUj\n4jVJKwC/BiYABwAvRsR5kk4DVo+I0zt5rUf8Zs0gDdZmAwcTMSN3HOta9r16IuK14uYAYEXSGbv7\nAlcW168ExpaZwcyW2yTgehf9vqPUD3cl9QN+D2wGXBQR90paJyLmFk+ZC6xTZgYzWw7SPsAHgHfn\njmL1U2rhj4jFwDaSVgVulDS8w+Mhaam9JkkTa+5Oj4jppQQ1s7eSViftvHk4Ef/IHcc6J6kNaOvR\naxo1q0fSmcBrwPFAW0Q8p3RU290RsUUnz3eP3ywn6fvAfCJOzh3Fui9rj1/SmpJWK24PAnYDHgFu\nBo4qnnYUcFNZGcysl6QxwC7AWyZeWOsrbcQvaQTpw9v+pB8wP46IrxTTOa8DNqSL6Zwe8ZtlkgZs\nDwJHEXFX7jjWM92pnV7AZWZvJk0B/kXEJ3NHsZ7zXj1m1jPSHsCHgBG5o1h5XPjNLEmz7y4FjiFi\nXu44Vh63eswskb4HQMQJmZPYcnCrx8y6R9oN2AO3eCrBZ+6aVZ00hNTiOYGIv+eOY+Vzq8es6qSL\ngQFEHJs7ii0/t3rMrGvSrsDeuMVTKW71mFWVNBi4DDgRn4lRKW71mFWVdBEwmIijc0ex+nGrx8w6\nl3Z0HItbPJXkVo9Z1UirAFOATxLxcu441nhu9ZhVjTQJWIOII3JHsfpzq8fM3kzaBTgQGL6sp1rf\n5VaPWVVIK7OkxfO33HEsH7d6zKpCugBYh4iP5Y5i5cl6AlcRYKikuyU9JGm2pPHF9YmSnpY0s/ga\nXWYOs8qTdgQOBcbnjmL5lTril7QusG5EPKC0WOR3pClkBwHzIuKCLl7rEb9ZPaSjTx8APk/ET3PH\nsXJl/3A3Ip4Dnituz5f0CLB+e74yv7eZ/dvZwEwXfWvXsA93JW0MjAR+U1waJ2mWpCnth7KbWZ1J\nOwCHA+NyR7Hm0ZDCX7R5fgJMiIj5wMXAJsA2wLPA+Y3IYVYp0kDgcmA8ES/kjmPNo/R5/JJWBG4A\nfhgRNwFExPM1j18GTF3KayfW3J0eEdPLS2rW50wEZhNxfe4gVh6l7TfaevSakj/cFXAl8FJEfLrm\n+noR8Wxx+9PAthFxWIfX+sNds96StiMNqEZQM9Cyvq87tbPswr8T8CvgD0D7NzqDNK1sm+LaE8CJ\nETG3w2td+M16I7V4fgecQ8S1ueNYY2Uv/MvDhd+sl6SvAVsAB9Cs/8CtNNmnc5pZg0nvA44FtnbR\nt6XxXj1mfYW0EnAFcAppDY1Zp1z4zfqOLwJ/Aa7OHcSam1s9Zn2B9B7gRNzisW7wiN+s1UkDSC2e\nUymmSZt1xYXfrPWdAcwBfpA7iLWGZRZ+SeMlrd6IMGbWQ9LWwKeAE93ise7qzoh/HeA+SddJGl2s\nxjWz3NJ2KN8HTiPimcxprIV0awGXpH7A7sDRwPuA64ApEfF4acG8gMusa9KZwAeAMR7tW7u6ncAV\nEYtJ++rPBRYBqwM/kfTN5U5pZj0njSCdpnWCi7711DJH/JImAEcCLwGXATdGxOvFbwF/iojNSgnm\nEb9Z59I25zOA7xBxWe441lzqNeJfA9g/InaPiOsi4nX4928B+9Qhp5l1lzTgr6ww/SbWXqsfHz1c\n2vM2afiY3LGstXiTNrNWIfX7EwPueIK13j+GJ1dZ9O/1l8f/Gf7/hIjZ07Lms6ZQtx6/mTWFcxcy\n4D1jeaym6ANcOgw29NGK1m0u/GatQDoF2Gs3PvjQAlbu5AmDBzU6krUuF36zZid9DPhPYI+5vD6/\n8yfNX9DISNbaSi38koZKulvSQ5JmSxpfXF9D0h2SHpN0u6TVysxh1rKk3YELgD2JmANzLkw9/VrH\nPQ5PTc4Rz1pT2UcvrgusGxEPKE1B+x0wFvg48GJEnCfpNGD1iDi9w2v94a5VWzpUZRqwPxG/XnJ5\n+JjU0x88KI30n5rsD3atXdMdvSjpJuCi4mtURMwtfjhMj4gtOjzXhd+qS/oP4JfAJ4i4OXccax1N\nNatH0sbASOC3wDo1h6vPJe0HZGYAaTB0G/AlF30rQ0MKf9HmuQGYEBHzah+L9CtHcy4mMGs06W3A\nrcD3vSrXylL6CVxKOwjeAPwgIm4qLs+VtG5EPCdpPeD5pbx2Ys3d6RExvdSwZjmlM3NvBO4BvpI5\njbUISW1AW49eU/KHuwKuBF6KiE/XXD+vuHaupNOB1fzhrlVa2vvqGqA/cDARizInshaV/cNdSTsB\nvwL+wJJ2zueBe0lbO28IPAkcFBGvdHitC79VQxogTQLeDYwmYmHmRNbCshf+5eHCb5UhfR44FNiF\nDgMgs57qTu0svcdvZl2QPg6cAOzoom+N4sJvlou0N/B1YBQRf80dx6rDhd8sB2kH4ApgbyL+mDuO\nVYs3aTNrNOldpGmbRxLx29xxrHpc+M0aSVqftEDrNCJuzR3HqsmF36xRpNVJWzF8l4grc8ex6vJ0\nTrNGkAYBPyftUHsKzfoPz1qe5/GbNQOpP3A9sBA4nIjFmRNZH+Z5/Ga5pVW53wWGAIe46FszcOE3\nK9dZwPuANiL+lTuMGbjwm5VH+gRwOGlV7rxlPd2sUVz4zcog7Qd8CdiZJYcOmTUFF36zepN2AS4h\n7bT5eO44Zh15Hr9ZPUkjSDN4DiPi97njmHXGhd+sXqSNgGnABCLuzB3HbGlc+M3qQXo7aVXu+URc\nmzuOWVdKLfySLpc0V9KDNdcmSnpa0szia3SZGcxKJ60C3ALcTMS3c8cxW5ayj17cGZgPXBURI4pr\nZwHzIuKCZbzWK3et+UkrknbafAk42lsxWG7dqZ2ljvgjYgbwcicPuaBb60urci8h/Ts6zkXfWkWu\nHv84SbMkTZG0WqYMZsvrq8CWwIFEvJ47jFl35ZjHfzFwdnH7HOB84NjOnihpYs3d6RExvdRkZt0l\njQP2B3Yi4h+541h1SWoD2nr0mrJ/O5W0MTC1vcffg8fc47fmJB0M/Bep6P9v7jhmtbL3+Dsjab2a\nu/sBDy7tuWZNR/oQMBnYy0XfWlWprR5J1wCjgDUlzSHtVNgmaRsggCeAE8vMYFY30kjgGlJP/w+5\n45j1lg9iMesOaVNgBjCeiBtyxzFbmqZs9Zi1HGlt0rGJX3HRt77Ahd+sK9IQ0v471xBxce44ZvXg\nVo/Z0kgDSFsxPAmc6AVa1gp82LpZb0n9gB8Ag4EDiHgjcyKzbvFh62a9901gI2A3F33ra1z4zTqS\nTgVGk45NXJA7jlm9ufCb1ZKOAMaRDkj/W+44ZmVw4Tdrl86G+CbwQSKezh3HrCwu/GYA0rbAVcBY\nIh7JHcesTJ7HbyZtDtwMHEvEPbnjmJXNhd+qLW0aeBvwRSKm5o5j1ggu/FZd0qrArcAUIqbkjmPW\nKF7AZdUkDSQV/YeAcV6Va32FV+6adUbqD1xb3DuEiEU545jVk1fumnWUDkifBKwJ7Omib1Xkwm99\nljR8DAwdD0MGwryFMOfCgJHATsAoIhbmzmiWQ9kncF0O7AU8336urqQ1gB+T9kF5EjgoIl4pM4dV\nTyr6O0yCS4e1XzuZD4z8B1q8CvEeIl7Nmc8sp7Jn9VxB2vOk1unAHRGxOfCL4r5ZnQ0dX1v092Yq\nZ/DE2juzw2NEPJszmVlupRb+iJgBvNzh8r7AlcXtK4GxZWawqhoysP3WDtzD5RzDR/gZM1m/OWcz\nmDVQjnn860TE3OL2XGCdDBmsz5u3EGA4D/JT9ucIfsB9bAfM926bVnlZP9yNiJC01BGYpIk1d6dH\nxPTSQ1kfMefCsew24hJmvWMck/k5o4HjHoenJudOZlZPktqAth69pux5/JI2BqbWfLj7KNAWEc8p\nLZe/OyK26OR1nsdvvZOmbJ68EH35QN7z2C1sujCN9J+aHDF7Wu54ZmVq1nn8NwNHAecWf96UIYP1\nVdKKpHn6owYS750a9z+RO5JZsyl1xC/pGmAUabHMXOBLwM+A64AN6WI6p0f81mPSasD1wBukFbme\nsmmV4y0brDqkYcBU4HbgMz4n16qqO7XTu3Na65NGAb8GJhExwUXfrGvessFam3QM8HXgY0TcmTuO\nWStw4bfWlHbY/AZpAeAoIh7NnMisZbjwW+uRBgM/AlYFtifipcyJzFqKC7+1hPadNjeHVW9h5a0W\n8cY9W/Cv3Yn4V+5sZq3GH+5a02vfaXM7Ju5xF3/Y/r85e8iWHLGZ2OrDubOZtSKP+K3pDeQdE05j\n/WEnsTfHMoWp7AswDMaMA7wS16yHXPituUnbPsgqOz7CSmzDA/yV9WseHDwoWy6zFuZWjzUnaRDS\necAtF7Lh4/tyc4eiD95p06x3XPit+Ug7A7NI23qMmEy/z8MJf37zk7zTpllvecsGax7SENJirP2A\nk4i4aclDw8fAhuNSe8c7bZotjffqsdYh7Q58D7gbOIWIjie3mVk3NOu2zGZLSKsD5wO7AicS8fPM\nicz6PPf4LR/pI8CDwAJghIu+WWN4xG+NJ60FTAbeCxxGxK8yJzKrFI/4rXEkIR1KGuXPAbZ20Tdr\nvGwjfklPAn8HFgGvR8R2ubJYA0jrAxcDmwL7EnFv5kRmlZVzxB+kQ9dHuuj3YWmUfyzwADATeK+L\nvlleuXv8nq7Zl0mbkKZorgF8mIhZmROZGflH/HdKul/S8RlzWL1J/ZDGAfcBdwLvd9E3ax45R/w7\nRsSzSjM87pD0aETMqH2CpIk1d6dHxPRGBrRekN4JXEYaVOxIxB8zJzLr0yS1AW09ek0zrNyVdBYw\nPyLOr7nmlbutRFoB+AzwWeBs4DtELMobyqx6mnblrqSVgf4RMU/SKsDuwJdzZLGutZ98BUMGwryF\nMOfCt+yRI70buBx4BdiWiCdyZDWz7snV6lkHuFFSe4YfRcTtmbLYUrSffAWXDlty9fjNpOFEzJ6G\nNAD4AvAp4HTgcprhV0gz61KWwh9pRLhNju9tPTF0/JuLPqT7Y8YhvUAa5af/LyOeyRDQzHoh93RO\na2pDBna8MpAFXMQjWwG3AP8JXOtRvllr8ZYN1oV5C2vv7cQMZrE1Q5m3AmlTtWtc9M1ajwu/dWHO\nhXD8n9/Oi0zmZK7lEM5i/ef2YN3jiHg+dzoz6x23eqxzUv+ARU/w2LNrccXG01jvhfeyxUNzmfst\nn3xl1tqaYh5/ZzyPPxNpY+DjxdfzwBTgGiJeyZjKzLqpaefxW5ORBgJjgWOBkcDVwD7eZsGsb3Lh\nrzJpa1KxP4y0c+YU4CYiFnb5OjNraS78VSOtBhxKKvhrA1cA7yPiyZyxzKxx3OOvgrREehSp2O8D\n3E4a3d/p/XTM+pbu1E4X/r4snXp1FHAMsJBU7H9IxAtZc5lZafzhbhVJKwJ7k0b3OwLXAx8D7vVi\nKzMDF/6+Q9qCVOyPAP5EGt0fTMQ/suYys6bjwt/KpMHAQaSCvylwFTDKh5+YWVfc42816YPa7UnF\n/gDgV6RdMqcR8XrOaGaWn3v8fYm0NqmNcyzp/7cpwJZEPJs1l5m1nGybtEkaLelRSX+SdFquHE1P\n2gPpBuAxYARwIvBOIs510Tez3shS+CX1By4CRgNbAodKeleOLM1MGj7mMjb47sFstO1a7Hq/2Oo6\nImZUeXZOcbC04feilt+Lnsk14t8O+HNEPBmpL30t8JFMWZpS+7GHxzNn0+s4euiL/OJDsMOkdL3S\n2nIHaCJtuQM0kbbcAVpJrsK/PjCn5v7TxTX7t6Ude7jhuDx5zKyvyFX4K9uq6L63HnuYDB7U2Bxm\n1tdkmc4paXtgYkSMLu5/HlgcEefWPMc/HMzMeqEp9+qRtALwR+BDwF+Be4FDI+KRhocxM6uYLPP4\nI+INSScDPwf6A1Nc9M3MGqNpV+6amVk5si3gWhZJB0p6SNIiSe/JnScHL3JbQtLlkuZKejB3lpwk\nDZV0d/FvY7ak8bkz5SJpoKTfSnqgeC8m5s6Um6T+kmZKmtrV85q28AMPAvuR9qKpHC9ye4srSO9F\n1b0OfDoitiLt2XRSVf+7iHRE6AcjYhtgG2C0pPdnjpXbBOBhljFzsmkLf0Q8GhGP5c6RkRe51YiI\nGcDLuXPkFhHPRcQDxe35wCPAO/KmyiciXituDgBWBBZnjJOVpA2AMcBlQJezepq28JsXuVnXJG0M\njAR+mzdJPpL6SXoAmAvcHhH35c6U0beAz9KNH35ZC7+kOyQ92MnXPjlzNQl/6m5LpXQWw0+ACcXI\nv5IiYnHR6tkAeL+krXJnykHS3sDzETGTZYz2IfO2zBGxW87v3+SeAYbW3B9KGvVbxSkdr3kD8MOI\nuCl3nmYQEa9Kupv0OdBDufNk8AFgX0ljgIHA2yRdFRFHdvbkVmn1VPFAlvuB/5C0saQBwMHAzZkz\nWWZKB/FMAR6OiG/nzpOTpDUlrVbcHgTsRvrMo3Ii4oyIGBoRmwCHAHctrehDExd+SftJmkOaufA/\nkm7NnamRIuINoH2R28PAj6u8yE3SNcA9wOaS5kj6eO5MmewIHA58sJi2N1NSVWc7rQfcJWkWafX/\n7RExLXOmZtFlq9gLuMzMKqZpR/xmZlYOF34zs4px4TczqxgXfjOzinHhNzOrGBd+M7OKceE3M6sY\nF34zs4px4TfrJknbSpolaSVJqxSHf2yZO5dZT3nlrlkPSDqHtAnWIGBORJybOZJZj7nwm/VAsTPm\n/cACYIfwPyBrQW71mPXMmsAqwGDSqN+s5XjEb9YDkm4GrgY2BdaLiHGZI5n1WNaDWMxaiaQjgX9G\nxLWS+gH3SGqLiOmZo5n1iEf8ZmYV4x6/mVnFuPCbmVWMC7+ZWcW48JuZVYwLv5lZxbjwm5lVjAu/\nmVnFuPAxG6ySAAAACklEQVSbmVXM/wHXnM4fJ5jlqQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11332ffd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fit_y = np.polyval([fit_c, fit_b, fit_a], x)\n",
    "plt.plot(x,     y, linestyle='' , marker='o', color='blue', label='data')\n",
    "plt.plot(x, fit_y, linestyle='-', color='red', label='model fit')\n",
    "plt.xlabel('x')\n",
    "plt.ylabel('y')\n",
    "plt.legend(numpoints=1, loc='upper left')\n",
    "plt.xlim(-1,4)\n",
    "# plt.ylim(-2,3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And just to really drive the point home, let's fit $y = a + b x + c \\sin(x)$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "def f_b_sin(x, a, b, c):\n",
    "    return a + b*x + c * np.sin(x)\n",
    "\n",
    "x = np.linspace(-10, 10, 101)\n",
    "(a, b, c) = (1.5, 1, 2)\n",
    "y = f_b_sin(x, a, b, c)\n",
    "\n",
    "# Add some noise\n",
    "(mu, sigma) = (0, 1)\n",
    "y += np.random.normal(mu, sigma, size=len(x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.95922207317 1.00043620918 1.44201214501\n"
     ]
    }
   ],
   "source": [
    "A = np.column_stack([np.sin(x), x, np.ones(len(x))])\n",
    "(fit_c, fit_b, fit_a) = np.linalg.lstsq(A, y)[0]\n",
    "print(fit_c, fit_b, fit_a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x1133687b8>"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEPCAYAAABY9lNGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XeYVOXZx/HvTVOaAoKICIJiC4tiRWxgoewaExsYjbGC\nmkSwvBoLb2KLUaPiK5oYxY4tqLHGBdGwrAU7FlCMIMLSLShdBe73jzPjzs7O7M7szu6Z8vtc17nY\nOW2eHddzz1Pu5zF3R0REJF6TsAsgIiLZSQFCREQSUoAQEZGEFCBERCQhBQgREUlIAUJERBIKNUCY\n2b1mtszMPorZd6WZLTSzGZFtaJhlFBEpVGHXIO4D4gOAA2Pdfc/INimEcomIFLxQA4S7vwKsSHDI\nGrssIiJSVdg1iGRGmdkHZnaPmbULuzAiIoUoGwPEHUBPoC+wBLg53OKIiBSmZmEXIJ67L4/+bGZ3\nA8/Fn2NmmkBKRKQO3D3lJvysCxBm1sXdl0ReHgN8lOi8dH5JqZmZXenuV4ZdjnyhzzOz9HlmTrpf\nrkMNEGb2KDAA6GhmFcAVwEAz60swmmkecHaIRRQRKVihBgh3PzHB7nsbvSAiIlJNNnZSS+MrC7sA\neaYs7ALkmbKwC1CoLBcXDDIzVx+EiEh60n12Zl0ndX1odFN2UPAWyQ95FSBAD6ewKUiL5A/1QYiI\nSEIKECIikpAChIiIJKQAESIzu9/Mrgm7HCIiiShAhMsjW43MrMzMzmyE8oiI/CTvRjHVlVlRCXQb\nDW03h1XroWKc+8wXGuOtUzhHI4NEpNGpBkE0OPS/FUqHwMQBwb/9bw32Z/J9bE8ze8/MVprZY8Dm\nkf3tzOx5M1tuZt+Y2XNm1jVy7FrgYOB2M1tlZuMi+281swVm9p2ZvWNmB2WyrCIiChBAUHMY36vq\nvvG9oPuoTL2DmbUAngYeANoDjwPHEdQOmgD3AN0j2zrgdgB3HwO8Avze3du6++jILd8C9ojc6xHg\n8ch7iIhkhAIEEDQrJdKmZQbfZH+gmbvf6u4b3f1J4G0Ad//G3Z9y9/Xuvhr4C8Est7GqNEW5+8Pu\nvsLdN7n7WGAzYJcMlldECpz6IICgzyGR1esy+CbbAovi9s0HMLOWwP8BQwhqBABtLDJxSuR1lX4I\nM7sIOCNyXwe2ADpmsLwikoUas79UAQKAinEwcseqzUwj5sKC2zL4JkuArnH7tgfmAhcBOwP7ufvy\nyHoY7xHUGqqNdDKzg4GLgcPcfVZk3zek1uEtIo0okw/0yv7S2GfVyB3NimiIIKEAAbjPfMGsCCgZ\nFTQrrV4HC27L8Af+OrDBzEYTrLt9FLAv8B+gDUG/w3dm1oFg4aRYy4AdY163BTYAX0X6HS4lqEGI\nSBbJ/AM9vr+0HOjcC3aZYFb8dsZrE+6ec1tQ7NT3Z8sG7E1QM1gJPAY8ClwNdAGmAquA2cBZwEag\nSeS6/YFPgW8ImqKindrfAYsJahOfE9Qowv4ds/q/gTZtjbnB0EngXn0rLq3b/YaVVd5jmsPlcfcd\n8Rn0Lkl+PZ7O+6kG0Yjc/V1grySHD417fVfMdW9QvQP6zMgWdWO9CygiGZb6AJjUmqJi+0tfBK6N\nu8v4XkFLCBmpRShAiIg0mNQGwKTeFBXbX5rs8Z250ZcKECIiDSb5AJiqNYZuvWF83CjE6rWBqv2l\nTfpROeoxRuZGXypAiIg0kGQDYIKjsTWGK5PcoXptIFKjeCEIMCNvbcjRlwoQIiINKPpAj91nVjyp\n6oN9Q5Krk9cGGmP0pQKEiEiji++8HgyMoWqnc+21gUTBJ5MUIERE6in9ZLj4zutDIv8WfwVtZzVQ\nLlbaFCBEROqhbslwiTqvH5wLFaPDDgqxLJI8kVMiUxRVm1Yi2X5pPPpvIIUm6E8oHVL9SMkk9xeK\nq54bW9NY2iaY8b/d6vgaQ0PNt5Tu/5+qQeQJM7sfqHD3P6Zw7hfAme7+coJjLYGJBGtQvEgwRfmp\n7p7gfwARSTUZzqyoZAd6/+0wjuixJzPYmk1sR/m6Xfl6STs2fQH8GrPDH2cL35Ghx83lnz0qr264\n+ZZqogCRP1JavjSFc48HtgY6uPumyL5HogfNbBPQy90/r2tBRfJLfH9COcF3q419zIonDWT2g1P5\novc8Nj+vNctaT6Y5b9KPMgbyJb9vCZcuncpb1wHbANu2p9mFrzO18xp68iKDmcBveI27esGRGcuQ\nTpUCRH7JRNPO9sB/Y4JDQ72PSFZKv3kntj+hHJgMXMu2LOpwMTcOOY3/DJrJZi+Mot+n0/jPXl5t\nGZ7tf8TffCn6apANL4F/dt6V2fySZ7iLs9iM73iEVgO6289fq2DjqsZaElkLBjUiM/vCzC4ysw/M\nbLWZ3W1mnc2sNLIM6RQzaxdz/i/MbJaZrTCzqWa2a8yxhMuXxhz/uZm9H7n2NTPrk0L5rgL+CJwQ\nWd70DDM7zcxeiRwvj5z6QeT4sEx8LiLZoi7LDwcP6unnQckk+OsK4xou5GY+ZHc20pSfMa9JHw5r\nVkbLL6sHB6ie67BqPRiz2Y0buJTe3MEJDKETh7f8gNcOuIvthvRhl79neknkJL9c+DMepj+jYeIZ\nCZPtz5YNmEcw7XcngoV+lgHvEiwduhnwMvCnyLk7A6uBw4GmBDO2fkZQ62tBsNjQeZFjxwE/AFdH\nrt0zcu99Cb7tnxJ57+Yx5Ug48yvBVOMPxrw+DXgl5vUmYId0/9to05YLW31nX92FIdOnMsDLOch7\n8HnM9cPKoHdJMNtq7H3PnBM/+2r188b8dH5HlvufuNKXsrW/TIelDrun9/vh6Zyf/01MZg0/TCu9\nUTu3ufuXAJFv5svc/YPI66cIAgLACcDzHulINrObCALCgQT9B83c/dbIuU+a2dsx73EWcKe7R/c9\naGaXE0wb/kot5TPUhCQFqx7LD5sd8wbN9v4L13Iz/8MmmsYcXL0u1czn6udt7AN0APiKTlzNFdzI\nxVzOPqsP45vJmL0GXIX7R1WLU72pLJ1PAgqhDyL7hlwui/l5Xdzr9QSLB0FQw1gQPeDubmYVBKvS\nbSTJ8qUR2wOnmNmomH3NI/cUkaSSz75aY9+E2bnAZX9hq4tu5LNR0DTh/EipZj7HnhcMo6XKKMJ1\ntOKP9Pjsf/nkeOC3wBTMphEEio+T5WbArFQ/CKAQAkT2SxbAFgE/9RuYmQHdgIWRXYmWL50T+XkB\ncK27/6UO5cm9xBiRjEk6++obiR64zaw3G/j4IIJm3oP+6kvn3WhFczI7P1INSyK7rwFuwuwO4PfA\nVMxe7k+/7tOrnA/B9Xen9c4KENnrceBSMzuMoFnoPIIaxusEgwsSLV8azWsYDzxlZi8BbwOtgIHA\nNHdfXcv71lbjii5/qmGukneSz74av9QnGHf2epwudxJ8mTsQ96+i9yCDw1FTapoKAsVfo4HiWWYc\nN52juI7LmM4BdX5vBYjwedzP0Z6kT83sZOA2gtrCDOAod98AYGbHEgSCPxP8MT75003c3zWzkcDt\nwE4ETVmvAGUplidhmSKuBB6IJNSNdPcnUv1FRXJB4tlXh/8h7izGciE9+X4LYNfIA7pRy5TkxFXA\n9T1t0GEnUzLoIU7mKzoynpE8xq+o7dthvFCn2jCze4EjgeXu3ieyrwPwT4Imky+A4e7+bdx17ppq\nIyvpv4Hko/jpNMbwZ4bxOIPo+NJyf3lQmGVLJNoH0YR/9BrKJEYynkN5YVM7fmySzv+fYedB3AcM\njdt3KTDF3XcmaDK5tNFLJSJSRcU4GDkH4Bzu4DTu5+fs9sWXLLsl7JIlEs3N2MRRk17ggWnH8MOk\nE+hyarr3CX2yPjPrATwXU4OYDQxw92Vmtg1Q5u67xl2jGkSW0n8DySfxk+v9im+3HMen3Q9jv+kz\nWfHXbJp5NRX5MFlfZ3ePDv1cBnQOszAiUpjih4ruzKfczh4bx7HlHz7yV8aGXb50VAa69GRjgPhJ\nZOx/wiqOmV0Z87LM3csapVAiUiAqRy61ZSVPczSXclvTu3lq0FUwtqGm5K6P6mV6eyqs3R12Ggr9\nOsCktO6XjQFimZlt4+5LzawLsDzRSe5+ZeMWS0QKS5BVbWziQU5hGgO4m5HAlJZ1WySoYSVPjpu/\nEl7sELy+Kq17ht1JncizQLQz5VSC9QhERBpZkFV9CTfQiS8ZTXSmitXrEuVFBK+7jyI0ycrUtkdd\n7xhqDcLMHgUGAB0j00j8CbgemGhmZxIZ5prmPZUJLCIZUDFuAEf2Pp93ttubd/mRFlROm/GzPyS+\nJoU5mxpMsnmkvq/zoJFQA4S7n5jk0BF1vJ9Gz4hIRjizXlvDx83Oo/d7i7hwVWwGs1lxkg7f+Km7\nG1OyeaRWzoWR7arXLmoX+jDXutBQShFpUMHcZw8BK3H/bfXDidr7R8yFN0ZnVx9EUKbg5+6joHRo\nOs9OBQgRkXhmpwCXAPvivjbxKUUlwUM3U5Py1V9tZUr32akAISIFLX5o6LHMeuxJKm4EDsf9w7DL\nl0kKECIiKareLOOU02XdJlb9c4CvOT3c0mVePmRSi4g0kqpDQ8/mTlqwfcuDad/lhzCLlSWyMQ9C\nRKSRVA4N7cE8ruGPnMb9/MgWSYaMFhYFCBEpYMHQUGMT93IG13Mps9mNcIerZg81MYlIAQuW8zyb\nvXptxvf8H+cTu4Z0oVMntYgUtKNs+99MYPE9RzDg/Xdp8XU2DFdtKBrFJCKSqiAh7lngLdyvCbs4\nDU2jmEREUncC0AM4LuRyZCXVIESkMJltBcwEjsb9zbCL0xjUxCQikgqzB4BvcL8g7KI0FjUxiYjU\nxuwIgqUGisIuSjZTHoSIFBazVsCdwO9wXx12cbKZAoSIFJo/EYxaysuhrJmkJiYRKRxmfYEzgD5h\nFyUXqAYhIoXBrCkwHrgM92VhFycXqAYhIjktfj0HqBgXLAtadf+LtF4wiDWrgHvDLnOuUIAQkZyV\neJnNkTuaFe0L/U+O7u/BPPZm102/oetZE3xh7o3tD4nyIEQkZ5kVT4LSIdWPFH8FpR2Dn53JDOEl\njuBGyia5v1DcuKXMHuk+O9UHISI5rG2SdRvaNI/+9Bsm0JGvGMuFBGs1S6rUxCQiOSxYz6G61T8C\ndGI5N3IxxZSykWZonYf0qAYhIjmsYhyMnFN134i5UHG7MWLO3/kd93MaM9gLrfOQPvVBiOSxZCN8\nwi5XJgW/Y/dRQfPR6nXR9RwmWLvrDub7Ub0pfm8t69fk8zoPqdJkfSICJB3hMwemn5evD8poQNyO\npm0/YPI+t9Dh0mt82S3JzsvnwJmIAoSIADWN8CnJy5E8sQFxIsP4nB24lG+qBcRCDJxRGsUkIhFJ\nR/jk6UiebqNhfK9hTKSImVzBVQRBoPuoROdV3ZfoPNEoJpG8lXSET56O5Gm7eVcWchujOIrn+J5o\nfAwCYmWzUpv9E1+fr4Gz7lSDEMlBZkUlZsWTzIaXBf8WlVQ/K9kIn/wcydOM79Y/yCmMYzRvs1/M\nkdXrKpuVSofALlsmvkO+Bs66Uw1CJMfUML0EsW3okfmIgJJqI3xCKHaDe5FXF7Wi9frrmRLTtBYN\niLHNSoOBMcC1VD9PYqmTWiTHFFrnc0rM9gZKj6PbH/5F0QnxAdFseBlMHFB5QTkwBZj9HayZns+B\nM5aWHBXJe4XW+VwLsy2AR4DznvQFjwL3Vz8pvj/mkMhWMr1gg2oK1AchknMKrfO5BmYG3AdMxf3R\n5CcWVn9MpqiJSSTHVO+DKAfGroUmc2Hd4lxN+qpT8prZJcAxwADcv6/9/tUzrjP4K2S9vEmUM7Mv\ngJXARuBHd98v5pgChBS0yofdhq6w3Y5wb6vKo7mX9FWn5DWzI4AJwL64L2yUgua4fEqUc2Cgu+8Z\nGxxEJBihFLSdN11cNThAbiZ9pZ68ZlZUUmQHTfuaFi8cS98Ko/fujVXKQpPtndSqJYjUqLLDeiu+\nYi/eYy/eox8f9MVsAsH/QwYsmMgWze9klwPeZLsNa1i3NrYJJ/y5iVLreDcrKtmGPW5/knd6XsHN\nPMW5+8LIW+OH+EpmZHOAcOAlM9sI3Onu48MukEi26cjXG47nDkYynh2ZGwkPe/EWbb46JhjH6YC9\nRstBLWh/9Fi+b9OTl5jKobzMFnseYT0ufJk2K1LJq2hYqXW8b8E2FzzNnJ5Pchx/49zI3vG9glwP\nFCAyLJsDxIHuvsTMOgFTzGy2u78SdqFEsoJZZ+CqRdBvMp+tvoR72rzM4ThNCEbnNL/kOvefHpgH\nWfFJUNoGoD3fUEwpv+DZra/lrfvfZYtv/sKwrV9iU+R6iD50g0S7KjWL6dCtf+ZrGhXjYOSOVYNU\n3Cgjs+bP03Gvj/k5Y6okuUHBDvFtYFnbSR3LzK4AVrv7zZHXDlwVc0qZu5eFUTaRRmXWFDib4O//\nQeBGo/detY3OqZ4oFmjH0a+ew9ytT6Dpzq1Zwy1cwH3sxHqmAe+sgW2tso+jHJjwI4xvXnmHoCM5\n+Ll+TVQ1jjIyaw48+BrtDh/I8k4baB53dQEnCdbAzAYCA2N2XZHWAB93z7oNaAW0jfzcGngNGBxz\n3MMuozZtjb7Brg5vO5Q7FCU6B3qXwNBJMKws+Ld3SbB/6CRwr74VlwbHNvkBvOrPcIAvoo1fxF+9\nDRfFnTsm7vW0yL5BK+H0NVWPjfgs+t4Z+L1bOTzv8Px27Hx0cO/Y9zpzTsbeK8+3dJ+doRc4yS/R\nE3g/ss0ELqvPL6lNW85vcKTDcoezPVLzj9+C4BD/8Awe1ImPBQ/WqsfGeB8+8Ec5wZfS2i/mBm/F\n6sj5V8QFh8uTBI7oVlxa2++VLKBFj3Xi0Jfeo923/6bj4nbselTlNcWlwTXFpQoOqW95ESAy/Utq\n05azG5jDZQ6LHA6o6dyaagnB8eQP1spjx38bvW43zvLHGO5L6OwXcLO35A9JahNXJAkQw8pqLm/N\nAe1nHPfF++zutzLKjY2ZrZUU6JbuszOb8yBEClvQ33APcCzQD/fXa76g5qGi0dwJ94kDozOcRqcL\nD46/UAyr34he9Qm/5lf0YjAvciCvMZe7uZh+G1uxhqrjWzYkKU9tU38kz304n43XlzFt+7sZwXnc\nGuk8z8X8jtyWzaOYRAqXWRNgPEFz60Dc19R+UWpDRWuaLhyIGU10CAAf8ae1x9Nk7oHstPJWPmh/\nKe173UaPjbdxbsuv6Ujdp8+uHtDasYIHeH/P3fmu7RG8zofsEXeGRis1JgUIkWxTGRx2BEpSCw6Q\n0lBRIPk395JR7i8U17qGhFnvE5h/8/lsO+gxTm9yO+cykyHA0ZH5oNYvir0mQRJeZKhs058yoFuz\nmtGM4wJuYSrNvt+D/h+tZI8jqv+OBTghYYgUIESySTA76T+AnUgrOKSzQFDtTVHUlHTmPmtXGDrA\nep44nElXv8QD3VbT5IeF2JMDWHs17vMqf51EEwtOOCwYKlvOLpzNcXTnXG6njIEcyiELZvHpb2Ep\nMLJH7cFOGpIChEh2uRzYEzgU99XpXlzrwx3I1HTh03zeo8CjmDXpDAftCCcBb2K2DpgOTD+NolPm\ncEqvFczEMbpxB905tPnOXMTPeZ42fMPTbM+R7PbdDL6bHlfzoFBWw8tWOZEoF0+zuUpeMhsG3EzQ\nIb2k4d4mUR/EiLnwxuh6P4CDGtBOQH+gfzkdhzVhlw7tWUETNlHBD1QwkHn0ZDJDeJe9Ix3Qw6cF\nnefSkPJmuu+aKEBI3jHbD3geGIz7+w3/do2zNkL15VH/F/hzgjOVCd0YFCBEco3ZdsCbwG9xfzbs\n4mRSkj6IuOk6MlR7kVopQIjkkmCOoTLg37j/JeTSNIgEtZU3oPv+6ltofAoQIjnkHWs58Uc2P/Qg\njpi1idUhrMPQMMJfX0ISSffZqVFMIiG5wzpc8UvsmD34b7NNdIrMtNrY6zBkXk2JeLn8exUi1SBE\nwmDWbQXN5hzF1BavcVDMgXLguq+g7SxY2gY2N2i3Kpe+hVfvmI5SR3TYVIMQyXbBHEsP3c0Oi17j\noJ6VB8qByUBpRygfEPx8bcyxmw82O3YurFuc3cEiteVDJfspQIg0vgsAu5we/yWYayniRSoDQuzP\n0cDxTCugT7Blc5NNZhLxJHyazVWkMQXpwZcAp25g0bhgRbao2O9rsT/HBouobJ7ZtCLu9wJNk5Gb\nVIMQaSxmLYCHgEtwn+cwr+p0Eqt6Ax2Dk2On0E72v2l2NtmkPieUZLtaA4SZjQYmuPuKRiiPSD67\nAlgA3BfdETt3UjD6Z2Rk9E/sFNp1XW8hPKnNCSXZLpUaRGfgbTN7D7gXmOy5OPRJJESXW+fzLqLZ\nRX0Z8k4FG0uxomqdzNW/eS9tDYMNaAFn7Aj3tqo8u7CabJRXEY6UhrlaMD/9YOA0YB9gInCPu89t\n0NIlL4+GuUrO6Gi7/uJVlj5+FXe0eIwTI3tHzoHp56X6kGusuZOyUZK8irQ+Pwk0WCa1mfUFTgeG\nAv8B9gdecveL61LQ+lCAkFzyN9t+Tjf22PGXPAPE/tkqLyAVyqvInIznQZjZecApwNfA3cBF7v5j\npFbxGdDoAUIkLGk3dZjtfhLNuxfxOpXBoZxgZFLr/sHDT80lNVNeRVhS6YPoABzr7vNjd7r7JjM7\nqmGKJZJ90p5CwqwZcO8N9Px0MV2Lgp3RnIZrAbYEhmR3TkM2UF5FWGrNg3D3K+KDQ8yxjzNfJJFs\nlWwt56T5CBcA395Es0sq8wJyLachGyivIizKgxBJWRpNHWY7ECTE9dvgs+ZWjk5q3Z+g5lD7PQRQ\nXkWYFCBEUpZiU0ew7OY/gL8SGekXzQsI+hxI0OGq5pKaKK8iHJpqQyRlKTd1nAx0AsbW4x4iodN0\n3yJpqDUfwawT8BFwJO7v1ukeIg1EK8qJhMlsArAc9/8Juygi8bQehEhYzIYABwFFYRdFJBPUByGS\nCWatCTqmf4v7mrCLI5IJamISyQSzm4AuuP867KKIJKMmJmlwmlmzkllRyaG0+t8naL5PXw54tcKK\nSiLj9vUZSc5TgJC0pD3dRB4zKyppRr9bb2JGrwsYTwWnHgoju5kV7Qv9T9ZnJLlOTUySFs2sWcms\neNIlDBhyGP9hCJOpnIyv+Cso7Vj9isL7jCS7qIlJGphm1ozaiw0dLuIm9uEdqk7j3aZ54isK7zOS\n3KYAIWnSzJoAmDW9nzY7Xcl1zKdHZGd0Gu+1rRJfVGCfkeS8rBzmamZDzWy2mX1mZpeEXR6JVf+p\nIsyKSsyKJ5kNLwv+LSrJdCnrI8XynduZdYvu4P3IZxGdxvvPwCXNg/WkY2k6Dck9WdcHYWZNgU+B\nI4BFwNvAie7+Scw56oMIUX2mikjcyX3sYli9FNqtCnvET0rLWwYztb4FHGD0jkzV3aQfPN++8ppy\nYArw8Tew7i1NpyHZIOen2jCz/sAV7j408vpSAHe/PuYcBYgckGioZ/A6tpO7ygI6EeGtN1xrJ3zw\nBWYq8CzuN1VeN7wMJg6oft3wae4TBzZUeUXSkQ+d1F2BipjXC4F+IZVF6ijZcFj4Oq4dPtkCOiWj\nCGV651o74S8EHLil6nH1zUj+ycY+iOyq0kgdJVt9bV2XqvuSfUcJa8RPogd9ObCq9wF22Nvf0uza\nYWx3H+4bq56jabwl/2RjDWIR0C3mdTeCWkQVZnZlzMsydy9r2GLlh8bL8E32TXzjEhj5bWXw2JDk\n+rC+eVeMC2o60fKVAxN+bMHTHe9gv44XcidPMH2MWdHy2M9Nq55JNjKzgcDAOl+fhX0QzQg6qQ8H\nFhN0BqqTOgNS6oDN2Hslb8sPvlVHO7mXtoZeXeHemJrFiLnwxuhwO6qj5VvVG0o7Xsel7MYnHM3T\nBDkPSnqT3JPzfRDuvsHMziXouWwK3BMbHKQ+kjX71NzeX7daR/w3cYg2ucQvHxncP3u+eceWz2x4\n2RAmDTiZh9iL96hMiFPSm+S/rAsQAO5eCpSGXY78k34WdF3nXkqnySWb1xveiaV+P6cxnIl8ydYx\nR9T5LPkvKwOENJS6jLSpW60DsvvBnxKzZq/StNNd9Pn6FQ7ZqvKAOp+lMChAFJTkzT7JrynouZeu\n3pqNi67h+0ug5NxsaQITaSwKEAWkbiNtUq915NUaCGbHAycD+3zvHy8H/h1yiUQanQJEgUm/2Se1\nWkderRNhti9wBzAY9+VhF0ckLFk3zDUVGuaaukx8qzcrKtmSLucfxLpOvfjGjmH+jAGsXQmsBVYA\nK45mzxFTeGW/tbSOuzrHhoOadQPeAH6H+zNhF0ckk3J+mKtkTn2+1ZsVlWzD1v9zIt90ncbi9v2Z\n1bY5fAh8AXwJLAVaAp2AXa5h7m4PszWz2ZUyBvJPTuBt9iWn+irM2gDPAbcoOIioBpHX6rr62+7W\n6/hTaHXn6SzqUMZAnuIYJjH586+ZMSpZYDErnrQZTw3Zi/cYwmRO5FGasIkp/DD3t1QMxL1aNnxW\nMWsNPA/8FziHXPwfQ6QW6T47s3EuJsmYNEcgmTXH7OJpzH94K/busA/vcDxP8jAn8zUTdgiyi5Op\nGPc9o+ZM5wCu5Cp24VNOYbeFB7J8FvAhZo9E2vZDk3SdB7NWBDWH+QRNSwoOIqiJKc+lkfdg1gN4\nBFh1BAM/eI/7EjzMkzcXJRohNZ0Ft+3u61/AbEvgTOAJzD4HrqKR585Ksg5Fn7Z22LKXad9zOc3W\nnkDHsWv8443J7yJSWBQg8lqKeQ9mw4C/ATcAt7xHsyT9EzVnDycdIeX+HTAWs9uAXwN3Y7YQuDKd\nQFG/Dvf4hL9yOrD9tk/yzbafciSncUa7TfzfP82OnQvrFuf0EF2RDFGAyGO15j2YGXA9cBxQgvs7\nwf6ilBPq0npou/8I3I/ZQ8BJwHjMKoA/4f5qTb9L/YfRVm1u68MEnuZlHmcYlzOUTbwEPNMK6BNs\nOTpEVyST3D3ntqDY4Zcjpzdo6nC3w5sOW1X/jHuXQHEpDCsL/u1dkvicEZ+Be+U24rPk5w6dFNxv\n6CToXeLQzOF0h3kOLzocnPy/+dBJVd8nuhWXpvY3U3n9sTzhy2nlJ/Jw5B5jEtw39Xtr05YrW7rP\nTtUgCpFFQCfyAAAQS0lEQVTZZsDDQDvgcNxXB7tjawPd1sOCWppZUpunKem3fzjPfeZ9mD0MnEpQ\nu1hEsMTci9G/6EB9p/yoGNeVk3a+jqY9D+YVhvIr3uOkyLFsW7RIJDsoQBQas+bAv4D1ndll3HJ6\nPoEN3xyWtoH9tq26LkNtzSypPrRrCSTuPxA0N90HnADcDGzC7A7gIdxX1WtJT7OmDj2/5+OtHmbb\neX3ZZ9F3fLY5nBFZhyLbFi0SyQ4a5lpIgj6He4BNW7Drfcs5+OYgT2LiADhk76rBAYKHeE1DW1N9\naKcYSNw34P4wsDtwAcGiUfMxG/833ny9JafPrXp9LRMNmnV91Vo9uIwWq9+m/Z8Pod+HZ9Lu3G/9\n6YPdy/eFt0YECxi99RGcsTate4sUANUg8lANHcfXATsBh6+ix7+qfquvSzNLqrPDpvnt330T8DLw\nMmbbAr/+HSuOGsn9XV7nmeXv0GnNpzT/Fr658y6WvIHZVgRfdtoCewB7Af1+gP5z2G7jaB7bfAZ7\nbQ4cBCNvjdaKqi4MlF2LFolkhbA7TRqjo6WQtmQdx5Np/Q+H2dEO6aCzOPacunXU1r0z+8w5ic6t\ncYOtHE5yuGEBzd5YyGZrVtJsw3c0/WE99p3DAofnHa52OLodh09R57M2bZVbus9O1SDyTvX2/mKO\n6bUXD3QDdsH962Bv/Lf6wcAYgv7hqNqbWVKZHbZu04wnqgn1Huc+8xGzom+h/7EwvlXk1KYwcn78\n2trf2vDzE99Znc8iqVCAyDtV2/t3YC73cxq/od/Hpf7K/Moj8c1DhwC3LIHBi6Hd6kw3s6Q7zXhN\neQ+pr3JXj45tEVGAyD+VD8WWrOVJjuMa/sgkSpfFnlXXb/WNp6Yg0CbF0VN1WUFPRKIUIPJO9KF4\nV69/cA4zKeJvvJ/woZjda0bXNPIptZpB9gdBkeymAJFnog/FMezy1/4s6bE3/V9zFt+aew/FmoJA\nxW1VawblwNi1sHnXYIrzyuk+sjsIimQ3BYg85MyaB3QG9l7pL34adnnqJnnzUNWawYausN2O8LTm\nURLJMC0YlG+CaTTeAO7A/a6wi1MfQUd19xqbh+q6KJJIIdKSo/JngmVBx4dcjnpLrXkoUV9FOdCk\nn9nwsrquwy0iChD5xewI4ESgL7lYNayT+L6KcmAy8Hx7YECwT01OInWhuZjyRbBq273A6bh/FXZx\nGk/FOBg5p/L1i1RN9oPa55QSkURUg8gfY4F/4z4l7ILUR7qrxlUfyrqxD9Ch+pnKnhZJlwJEPjAr\nBg4jmAU1Z9V11biqk+4VTwISdFore1okXWpiynVB09JdwAjcV4VdnPpJlj2dTvNQfJMTKHtapG5U\ng8h90aall8MuSP3Vd9U4ZU+LZJICRJarsU3ebBBwBFAUaiEzJjOT6yl7WiQzFCCyWI1t8syaBtwJ\nnJP7TUtRmlxPJJsokzqL1ZglTOnHQGfcT270gjWgVLKnRaRulEmdVxK3yR/K2q3XY/vvxID3FuZZ\ntrCah0SyhwJEVqveJt+MH7mDt3YbxYErFzL1sMojyhYWkczKumGuZnalmS00sxmRbWjYZQpP9SGb\nf2K/b5az2Y93U9656rnKFhaRzMrGGoQDY919bNgFCVv8kM2+LLY/8EHfvgz+BKxf9SuULSwimZON\nAQIg7zugU/VTm7yZAS8BV83GBic+W9nCIpI5WdfEFDHKzD4ws3vMrF3YhckSpwJbAuMSZwsfswQ2\nbG02vMyseFIwGkhEpO5CGeZqZlOAbRIcGkOw2M2XkdfXAF3c/cy46x24KmZXmbuXNUBRs4PZ1sBH\nwFDcZwS7YoeDLm0NvbrCvV0qLxo5B6afp05rkcJlZgOBgTG7rkhnmGtW50GYWQ/gOXfvE7e/IPIg\nfmL2MLAY94sTH9aqaiJSu5zPgzCzLu6+JPLyGIJvzoXL7EigP8F6y0nUfw4jEZF4WRcggBvMrC/B\naKZ5wNkhlyc8Zm2BvwNn4r4m+YmZmcNIRCRW1gUIdz8l7DJkkeuAl3F/qebTNIeRiGReVvdBJFMQ\nfRBmBwKPA71xX1H76ZrDSERqlu6zUwEiG5ltBrwP/BH3J8Iujojkh3SfndmaB1HorgQ+AZ4MuRwi\nUsCyrg+i4Jn1A04H9iAXq3cikjcUIEKScKU4Zk0F7gdG4b4s5CKKSIFTH0QIkqwUN+ddHvqgHRu3\n3pHD1yZcYlREpB5yPlGuMHQbXTU4wAGc1qsb929fxPAKeHiHyiNa50FEwqFO6lBUzXzegu94iJM5\nh92/X14lOIDWeRCRsChAhCI289n5B8dSSgf+RU9NmSEiWUMBIhSV03Wfwhj68CH/w6vArkma/IIp\nM8yKSoKpvDWlt4g0PHVSh8SsqKQv7S6ZwlsHH8q7NpM+QDkwGbg25swRc+GN0cHP1Tu2NaW3iKSq\noDKpEw4VTfCwTPW8RmXWEnj1j/Ru+2dm7lR5oByYAsz+DtZMj06ZoSm9RaS+CmYUU5KhotVG/KR6\nXuOUNwhSxsr1s2nRbGd++PRatusAxASIQyJbyfSqD35N6S0ijSuH+yCqDxVNPOIn/rxyoHMv2GVC\nY7XjVwap0iEwccAoiof8SKuDe7DT487CBMuHJpqJVVN6i0jjytkaROrfqGPPq9LG3wEYkqw2kYlm\nqcp79NgPxrcHGMhULuM69mdGi/n87iz3F4rNioCSWmZi1ZTeItK4cjhApPqNOva8F6naAQzBA7dk\nFJDRZqmq97gSgJ58ziOcxK95mPn0IBrMIves8b5BP0QqgUREJDNyOECk+o069rxEv2450KSf2fCy\naE0hefNV1UBSs9h7bGAbljCFQVzFFfyHwyPnpNc8lEogERHJlJwNEFNZfN9zPDdlHEPnbGCLlrC0\nNWzeBH72B7Pi0dEmoarfvJv0A9pX3iXa5PR8e2BAsG/kjvB1kgd39Q7h5E1RlU1b7ejPZPbgXs7j\nTs6J7FXzkIhkOXfPuQ3wEdzlM+iwfg325VRa3duO38wF98ptxGfQu6Tqdb1Lgv3Rc8Z41Wui29Av\nE+8vLq35fpXvC0MngXsrVvurHOA3Myzyfsd+DcWl8WXTpk2btobegkd+6ufnbB4EBOU+hP1fu4S5\nO+yHd7mLs7iV81hO58iZ1XMEqi7NubEPPNmh+jsM/gi2b1m9+eqN0ZEaSaTW0HS/SO0jTskkWHBb\nB/a87UkqdpjP9pzOfThn/XSPTH0WIiKpKpg8iKhyum8op/t/e3J9l4u4iU/YjbsZwY1czFcJmoRi\n2/GD5DMSJJ81WwTTb0vUIZyo87m6Ni2dWR98yydNSuk8/0w6fuH8XJ3KIpJTcjgPImr1Oli1fh47\n8Hv+zu58SGvWMJtdGc/0bph1TH5tRVwOQjlw9FrYvGtQQ1hwm/vEge4vFFc+2Kt2PieyLwubAK+1\nY9M/TmRJzw3+RNw9RESyX47XIGI7eoORSovYjnP5G2P5cv6zPPsp8ClmdwJjcf8q9uqqHdgbusJ2\nO8LTrYA+wZZoaGtsXsVgYAyVQ2edERy87Hbe2INgVbgHG+gXFxFpcDncB1E8KbbJpmrfQkyOgNn2\nwGXAMOBe4FbcF1a/Z2pzHVU/L5g7qSMzVjzGG9/vw4oftmTT0bjPyPCvLSJSLwU1WV+aF20PnA+c\nCjwP3BL7EA/yICYOqH7h8GnuEwdWnlc1ia45P3AOA768gTebtMQf2Jady5eww2+zamJAEREKsJM6\nZe7zgQswuxo4G3gas2+BB4BHYGhKmdnRZqk2DD7vAhbv8Hs+224jmxa0xEcZvdtnw8SAIiKZUDg1\niOo3aUIwbeqpwDEraPLVRHbeqpTr2s2kiKVswxrOD4alMmvSvuxw/GZ0Gj2IVZ0Hsbjdvnzbunkw\nL/e1uL8V3FJTcotI9lINIlXum4AyoAyzs9uzaZ/dWHBWX047sis/tOnI9y2asalrM7hvE2z1EhW2\nhOZNXuUgxjKUMp75/Gtm3Ok+863Km2pKbhHJH4UbIGK5/wC8fgi8/tM+MwPaAq1bM+j+9bw4uOpF\nx+9QfW4mTcktIvkjD/IgGkiQa74S9yXrabdZ4pPiawbxeRWgOZdEJFepBpGS1GoGmpJbRPJJ4XZS\np/V+idaHGKF5lUQkpygPosHeM0kinohIjlCA+Omc+i8ZKiKSTzTMlcwsGSoiUuhCGcVkZsPMbJaZ\nbTSzveKOXWZmn5nZbDMbnOweNUu2ZGj3UXUts4hIoQlrmOtHwDEEM939xMx+BpwA/AwYCvzdgozn\nNClhLR1mNjDsMuQTfZ6Zpc8zPKEECHef7e7/TXDol8Cj7v6ju38BzAH2S/8dlLCWpoFhFyDPDAy7\nAHlmYNgFKFTZlii3LRA7FfdCoGv6t1HCmohIfTVYJ7WZTQG2SXDocnd/Lo1bpT3MSglrIiL112AB\nwt0H1eGyRUC3mNfbRfZVEywaVJtZsS+GBtMrSSJmdkXYZcgn+jwzS59nOLJhmGvsU/tZ4BEzG0vQ\ntLQT8Fb8BY2dJCciUojCGuZ6jJlVAPsD/zazUgB3/xiYCHwMlAK/81zM5BMRyQM5mUktIiINL9tG\nMdWo4RPsCpeZXWlmC81sRmQbGnaZco2ZDY38/X1mZpeEXZ5cZ2ZfmNmHkb/Hak3NUjMzu9fMlpnZ\nRzH7OpjZFDP7r5m9aGbtarpHTgUIGjzBrqA5MNbd94xsk8IuUC4xs6bA7QR/fz8DTjSz3cItVc5z\nYGDk77EO+VAF7z6Cv8dYlwJT3H1n4OXI66Ry6iHa8Al2BU+d/3W3HzDH3b9w9x+Bxwj+LqV+9DdZ\nR+7+CrAibvcvgAciPz8AHF3TPXIqQNQgQwl2BW+UmX1gZvfUVvWUaroCFTGv9TdYfw68ZGbvmNnI\nsAuTJzq7+7LIz8uAzjWdnA3DXKsIM8Eu39Xw2Y4B7gCujry+BrgZOLORipYP9PeWeQe6+xIz6wRM\nMbPZkW/FkgHu7rXlk2VdgGjoBLtClupna2Z3A+kEY6n+N9iNqrVaSZO7L4n8+6WZPUXQjKcAUT/L\nzGwbd19qZl2A5TWdnMtNTPEJdr8ysxZm1pMkCXaSXOSPJeoYggEBkrp3gJ3MrIeZtSAYNPFsyGXK\nWWbWyszaRn5uDQxGf5OZ8CxwauTnU4Gnazo562oQNTGzY4BxQEeCBLsZ7l7s7h+bWTTBbgNKsKuL\nG8ysL0FTyTzg7JDLk1PcfYOZnQtMBpoC97j7JyEXK5d1Bp6KTI/TDHjY3V8Mt0i5xcweBQYAHSOJ\nyX8CrgcmmtmZwBfA8BrvoeeoiIgkkstNTCIi0oAUIEREJCEFCBERSUgBQkREElKAEBGRhBQgREQk\nIQUIERFJSAFCREQSUoAQyQAz2zcyE+5mZtbazGZG1ikRyVnKpBbJEDO7BtgcaAlUuPsNIRdJpF4U\nIEQyxMyaE0zatw7or/nAJNepiUkkczoCrYE2BLUIkZymGoRIhpjZs8AjwA5AF3cfFXKRROolp6b7\nFslWZnYK8L27P2ZmTYDXzWygu5eFXDSROlMNQkREElIfhIiIJKQAISIiCSlAiIhIQgoQIiKSkAKE\niIgkpAAhIiIJKUCIiEhCChAiIpLQ/wNlikSCU0Pz7QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x113368ba8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fit_y = f_b_sin(x, fit_a, fit_b, fit_c)\n",
    "plt.plot(x,     y, linestyle='' , marker='o', color='blue', label='data')\n",
    "plt.plot(x, fit_y, linestyle='-', color='red', label='model fit')\n",
    "\n",
    "plt.xlabel('x')\n",
    "plt.ylabel('y')\n",
    "plt.legend(numpoints=1, loc='upper left')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Nonlinear Curve Fitting\n",
    "\n",
    "### If a function is non-linear in one of the parameters to be *fit*\n",
    "\n",
    "In this case we need to use something more involved, such as `scipy.optimize`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$y = a + bx + e^{ax}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import scipy.optimize"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* The function to fit:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def f(x, a, b, c):\n",
    "    return np.exp(a*x) + b*x + c"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "* Generate some simulated data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = np.linspace(0.0, 4.0, 50)\n",
    "y = f(x, 2.5, 1.3, 0.5)\n",
    "y += 0.2*np.random.normal(size=len(x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Fit the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(array([ 2.50000016,  1.30367848,  0.4842005 ]), array([[  3.67102199e-12,  -3.86676399e-08,   4.54488780e-08],\n",
      "       [ -3.86676399e-08,   9.31772365e-04,  -1.52767623e-03],\n",
      "       [  4.54488780e-08,  -1.52767623e-03,   3.38833522e-03]]))\n"
     ]
    }
   ],
   "source": [
    "r = scipy.optimize.curve_fit(f, x, y)\n",
    "print(r)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.50000015628 1.30367848114 0.48420050002\n"
     ]
    }
   ],
   "source": [
    "# Extract the best-fit parameters\n",
    "(a, b, c) = r[0]\n",
    "print(a,b,c)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x1131b0f98>"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZUAAAEPCAYAAACKplkeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuYVmW9//H3hxlgRhlBUAE5eGAgRTTYqOSZtBLIU2qa\nZepO7WCBO3ftsvwFdvDatne6wbZuQ02tNDWtsAAzFTMT0BJPeAAVhVExQRGEAQa+vz/WGnkY5zzP\nM+uZmc/ruuZiPfc6fZ+lzJf7sO5bEYGZmVk+dMs6ADMz6zycVMzMLG+cVMzMLG+cVMzMLG+cVMzM\nLG+cVMzMLG8KllQkDZH0gKRnJD0taUpaPk3SCkmPpz8Tc865WNISSc9J+kRO+VhJT6X7pueU95R0\nW1o+X9Iehfo+ZmbWtELWVDYDX4+I/YCPAF+VtC8QwBURMSb9mQMgaSRwOjASmABcLUnpta4Bzo2I\n4cBwSRPS8nOBVWn5lcDlBfw+ZmbWhIIllYh4IyIWpdvrgGeBQelu1XPKicCtEbE5IpYBS4FxkgYC\nFRGxMD3uZuCkdPsE4KZ0+07gmLx/ETMza7Z26VORtCcwBpifFk2W9ISk6yX1Sct2B1bknLaCJAnV\nLa9iW3IaBCwHiIgaYI2kvoX4DmZm1rSCJxVJvYDfABemNZZrgL2A0cDrwE8KHYOZmbWP0kJeXFJ3\nkmapX0bE7wAi4s2c/dcBd6cfq4AhOacPJqmhVKXbdctrzxkKvCapFOgdEavricMTnJmZtVBE1NdV\n0eRJBfkh6Te5GbiyTvnAnO2vA7ek2yOBRUAPkprMi4DSfQuAcek1ZwMT0vILgGvS7c8Av24glijU\n98zj85qWdQyO03E6TseZE2O05rxC1lQOA84EnpT0eFr2HeAMSaNJRoG9DHwpjX6xpNuBxUANcEGk\n34wkedwIlAOzI2JuWn498AtJS4BVJInFzMwyUrCkEhF/pf4+mzmNnHMZcFk95X8H9q+nfCNwWhvC\nNDOzPPIb9cVjXtYBNNO8rANopnlZB9BM87IOoJnmZR1AM83LOoBmmpd1AIWibS1MnZekiNZ0OJmZ\ndVGt/b1Z0NFfxc6jwrLnZG/WuXTppAL+pZYlJ3Wzzsd9KmZmljdOKmZmljdOKmZmljdOKh2EpBsl\n/SDrOMzMGuOk0nFE+tMoSfMkndsO8ZiZfUCXH/3VGtKoSTBkClSUwdpqWD4j4unZ7XHrZhzjEVVm\nlhnXVFooSSiHTIc5x8LtRyV/HjI9Kc/nfTRG0j8kvSvp10BZWt5H0h8kvSlptaS7JQ1K9/0IOAL4\nqaS1kmak5dMlvSppjaTHJB2ez1jNzGo5qbTYkCkws3L7spmVMHRyvu4gqQfwO5JVLXcG7gBOIamF\ndCOZSHNo+rMB+ClARHwXeAj4akRURMSU9JILgQ+n17oFuCO9h5lZXjmptFhFWf3lvcrzeJOPAKUR\nMT0itkTEncCjABGxOiJ+GxHVkSx6dhlwVJ3zt2smi4hfRcTbEbE1Iq4AegIfymO8ZmaAk0orrK2u\nv3zdhjzeZHeSBchyvQIgqVzStZKWSVoDPAj0lpSbSLbrV5H0DUmLJb0j6W2gN7BLHuM1MwOcVFph\n+Qw4f+n2Zee9CK9elcebvA4MqlO2B0kN5BvACODgiOhNUksR22ondRPKEcA3gU9HRJ+I2BlYQ/M6\n/c3MWsSjv1oo4unZ0ihg0uSkyWvdBnj1qjyP/vobUCNpCnANcDxwEHA/0IukH2WNpL7A1DrnrgSG\n5XyuIFn07K20H+XbwE55jNXM7H1deur7Yp4SX9JYYCZQSbKEcgBLSJLMLcCBJE1kV6Rl3SNiq6SP\nkHTw70qynPNF6XVOBd4DrgS+ApwXEfe353eqq5ifv1lX19q/n04q/qWWGT9/s+LV2r+f7lMxM7O8\ncZ+KmZm9b9uMIa3jpGJmZkDujCEzK1s7QNTNX2ZmlkpmDCmj9a/dOamYmVkqmTFkf55q9RWcVMzM\nLJXMGDKaRa2+gvtUzMwstXwGnD9sDKWVTR9bPycVMzMDts0YcgSv/Aro05pruPmrC2rJ0sTpxJXH\nNLCvPF3P5R1Jt0v6rKR78hutmbWn4Jl7RrGue2vPd02la2rW0sTNOPZUYDegb0RsTctuqd0paStQ\nGREvtTZQM2t3lcCbwF6tOdk1la4rH9Oj7AG8kJNQCnUfM2s/o4HHW3uyk0qRSpudviHpCUnrJF0n\nqb+kOekSw/dK6pNz/AmSnpH0tqQHJO2Ts6/epYlz9h8naVF67sOS9m9GfJcC/w84PV26+AuSzpH0\nULr/L+mhT6T7P52P52JmBTcaWj/8y0mleAVwMvAxkvVTjieZrfjbJDMQdwOmAEgaQdLsNIVk8a3Z\nwN2SSptYmhhJY0iWJz4f6AtcC8yS1GibakRMJVl18tfp0sU31Nl/ZLp5QLr/jtY/CjNrR2NwUskT\nKQr+0zJXRcQ/I+I1krXn50fEExGxEfgtyX98gNOBP0TEfRGxBfhvoBw4jEaWJk59Ebg2Ih6NxM3A\nxvS8Jp8Ybt4y62zaVFNxR32u4puGfWXO9oY6n6tJFuyCZPnhV2t3RERIWk6yeuQWGliaOLUHcJak\nyTll3dNrmllXIg0g+fu/orWXcFLpWBpKelXA+/0g6Xr1Q9j2P0Z9SxPXLon8KvCjiLisFfF0/sV4\nzLqWpJM+IpAnlOzK7gA+KenotC/k30lqMn8D5pMuTSypu6STSZYmrjUT+LKkg5XYUdInJfX6wF0+\nqKn/6+oubWxmxa1NTV/gpNLRRJ3tAIiI54EzgauAfwKfBI6PiJqI2ETS4X8OsAo4Dbjz/YtE/J2k\nk/6nwGqSJYvPonm1kLrvsNT9PA24KR1Vdmpzv6SZZabNSaVgywlLGkKyRvpuJL9ofhYRMyT1BW4j\naYJZBpwWEe+k51wMfIGkH2BKRPwpLR8L3EgyFHZ2RFyYlvdM7/EvJL8wT4+I3P6C2li8nHAR8vM3\nKzLS88ApRDxdjMsJbwa+HhH7kYwk+qqkfUmGxN4bESOA+9LPSBpJMoppJDABuDrtGwC4Bjg3IoYD\nwyVNSMvPBVal5VcClxfw+5iZdV5Jk/cQ4Pm2XKZgSSUi3oiIRen2OuBZkg7jE0jemSD986R0+0Tg\n1ojYHBHLSDqSx0kaCFRExML0uJtzzsm91p1AvXNUmZlZk/YHniFic1su0i59KpL2JHmnYgHQPyJq\nh8auBPqn27uz/TC2FSRJqG55FdtGMw0ClgNERA2wJm1eMzOzlmlzfwq0w5DidBTRncCFEbFWOcPU\n0vcp2mVYqqRpOR/nRcS89rivmVlH8EuY+Gso+eP2vytbrKBJJR3eeifwi4j4XVq8UtKAiHgjbdp6\nMy2vImnPqzWYpIZSlW7XLa89ZyjwmqRSoHdErK4vloiYloevZGbWKZ0J/c+Ei4h4GEDS1NZcp2DN\nX2kn+/XA4oj4n5xds4Cz0+2zSealqi3/jKQekvYChgMLI+IN4F1J49Jrfh74fT3XOpWk49/MzFoi\n+Uf5KODJNl+qgEOKDwf+QhJk7U0uBhYCt5PUMJax/ZDi75AMKa4haS67Jy2vHVJcTjKkuHYixZ7A\nL0j6a1YBn0k7+evG0uCQ4vx8W2stDyk2KwLJ6Nvfk4ykTYtaN6S4YEmlmPh9CDOzRkifBU4i4rRt\nRcX3noqZmXUMeRn5BU4qZmbmpGJmZnmRDIByUjEzs7yoXTvp9XxczEnFzKxr27aGSh44qZiZdW15\na/oCJxUzs67OScXMzPImr0nFLz+amXVV0k4kHfQ7EbFl+11++dHMzFrmAODpugmlLZxUzMy6IGnU\npEsZ9rPbGDxEmjhXGjUpH9ct+HoqZmZWXJIEcsj0QUTlAxwIfHkgnD9MGkXE07Pbcm3XVMzMupwh\nU2Bm5WgWsYjRadnMShg6ua1XdlIxM+tyKsp6sJF9eZan2D+nvFd5W6/spGJm1uWsrR7L33mOfVjP\njjnl6za09cpOKmZmXc7yGR9n8lsPc1hO2XkvwqtXtfXKfk/FzKwLelXdH7mcEWVXs9+apIby6lW5\nnfRe+bERTipmZjmS6e5XAv9CxIoGDvHLj2Zm1izDgQ0NJZS2cFIxM+t6DgMeLsSFnVTMzLqew3FS\nMTOzPHFNxczM8kDaBRgIPFWIyzupmJl1LYcC8/M5M3EuJxUzs66lYE1f4KRiZtbVFDSp+OVHM7Ou\nQuoJrAIGELGu8UP98qOZmTVuLPB8UwmlLZxUzMy6joI2fYGTiplZV+KkYmZmeZBMIumkYmZmeTEC\nWF+ISSRzOamYmXUNBa+lgJOKmVlX4aRiZmZ50/GTiqQbJK2U9FRO2TRJKyQ9nv5MzNl3saQlkp6T\n9Imc8rGSnkr3Tc8p7ynptrR8vqQ9Cvl9zMw6pAJPIpmr0DWVnwMT6pQFcEVEjEl/5gBIGgmcDoxM\nz7layWgFgGuAcyNiODBcUu01zwVWpeVXApcX9uuYmXVIhwILCjWJZK6CJpWIeAh4u55d9b36fyJw\na0RsjohlwFJgnKSBQEVELEyPuxk4Kd0+Abgp3b4TOCZfsZuZdSKHAX9tjxtl1acyWdITkq6X1Cct\n2x3IHeq2AhhUT3lVWk7653KAiKgB1kjqW9DIzcw6nnbpT4Fskso1wF7AaOB14CcZxGBm1jUkk0iO\nBha0x+1K2+MmuSLizdptSdcBd6cfq4AhOYcOJqmhVKXbdctrzxkKvCapFOgdEavru6+kaTkf50XE\nvNZ/CzOz4ieNmjSRsd/7CS/ESA77DRo1I+Lp2fUfq/HA+Lbes92TiqSBEfF6+vFTbBuNMAu4RdIV\nJM1aw4GFERGS3pU0DlgIfB6YkXPO2cB84FTgvobuGxHT8v1dzMyKlTRqEhwyfT9GVN7LcmDGsXD+\nMGkU9SWW9B/a87adr6mtuW9Bk4qkW4GjgF0kLQemAuMljSYZBfYy8CWAiFgs6XZgMVADXBDbFnu5\nALgRKAdmR8TctPx64BeSlpCsEfCZQn4fM7OOY8gUmFl5OCdyC59Ny2ZWwqTJQL21lXzwIl1mZp2Q\ndNq8Em456p/syr48y0oGpHtOezDi9vFNn+9FuszM7H1rqz/CfF5mr5yEArBuQyHv6qRiZtYpLZ9x\nCl95e+5275+f9yK8elUh79ruHfVmZlZ4EU/PXq2SVefQ4wVYWp3UUF69qqHRX/niPhUzs85IGgA8\nC+xGxOaWn+4+FTMz2+YTwH2tSSht4aRiZtY5TQTmNnlUnrn5y8yss5FKgJXA6NYuH+zmLzMzq3UQ\n8Hqh16Ovj5OKmVnnM4EMmr7AScXMrDOaCMzJ4sbuUzEz60ySpYNfJBlKvLH1l3GfipmZJUOJ57Ul\nobSFk4qZWeeSWX8KuPnLzKzzkLqRrKj7ESJebtul3PxlZtbVjQHebmtCaQsnFTOzziOzUV+1mkwq\nkqZI2rk9gjEzszbJtD8FmldT6Q88Kul2SRMkuW/CzKzYJP/4PwB4MMswmkwqEfFdYARwA3AOsETS\nZZKGFTg2MzNrvo8BfyWiOssgmtWnEhFbgTdIJijbAuwM/EbSfxUwNjMza77M+1OgeX0qF0r6O/Bj\n4GFgVER8BRgLnFzg+MzMrBHSqEndNGHuW/T43BgO/bQ0alKW8TRnOeG+wMkR8UpuYURslXR8YcIy\nM7OmJAnkkOn787XKdziFRTx8BJw/UBpFoZcNbkhz+lSm1k0oOfsW5z8kMzNrniFTYGblBOYylwlp\n2cxKGDo5q4j8noqZWYdVUQbwKX7LHzgup7xXeUYBOamYmXVca6v34iX25iXu45ic8nUbsorIScXM\nrMNaPuMLnLLqN5xKDd3TsvNehFevyioiTyhpZtaBrVHJsrMY/cYshlUnNZRXr8pHJ31rf286qZiZ\ndVTSKJJ3U/YgeZ8wj5f2LMVmZl3NGcBt+U4obdGc91TMzKzYJPMwfgY4LetQcrmmYmbWMR1EMm3W\nP7IOJJeTiplZx3QGcCtF1jHu5i8zs45GKgFOh+1eTikKrqmYmXU8RwJvEvFs1oHU5aRiZtbxJE1f\nRcjvqZiZdSRSD+A1YCwNTPabn9sU4Xsqkm6QtFLSUzllfSXdK+kFSX+S1Cdn38WSlkh6TtIncsrH\nSnoq3Tc9p7ynpNvS8vmS9ijk9zEzKwIfB54vZEJpi0I3f/0c3p+Puda3gXsjYgRwX/oZSSNJOp5G\npudcrWQcNsA1wLkRMRwYLqn2mucCq9LyK4HLC/llzMyKQNE2fUGBk0pEPAS8Xaf4BOCmdPsm4KR0\n+0Tg1ojYHBHLgKXAOEkDgYqIWJged3POObnXupMiHAlhZpY30g7AccAdWYfSkCw66vtHxMp0eyXQ\nP93eHViRc9wKYFA95VVpOemfywEiogZYI6lvgeI2M8vaccBCtv0OLTqZjv6KZJRA5x8pYGbWBtKo\nSdLEufez6//+ByMGZr0OfWOyePlxpaQBEfFG2rT1ZlpeBQzJOW4wSQ2lKt2uW157zlDgNUmlQO+I\nWF3fTSVNy/k4LyLmtfWLmJkVWu069L35r8oDGcrJLNgFvjk93+vQSxoPjG/rdbJIKrOAs0k61c8G\nfpdTfoukK0iatYYDCyMiJL0raRywEPg8MKPOteYDp5J0/NcrIqbl/6uYmRVasg79ydzA/RzNGvqQ\nrEM/aTKQt6SS/kN7Xu1nSVNbc52CJhVJtwJHAbtIWg58D/hP4HZJ5wLLSGfYjIjFkm4HFgM1wAWx\n7SWaC4AbgXJgdkTMTcuvB34haQmwimTGTjOzTqSiDIKvcA3f53s55dmtQ9+YgiaViDijgV0fa+D4\ny4DL6in/O7B/PeUbKbJpn83M8mtt9TgW0I9VzCa3KyW7degb42lazMyK2vIZ/8EZa/+Xr7KVkrQs\n23XoG+NpWszMipk0YBMsGcrR81fSr3s+16Fv/LZeo75BTipm1mElHea7E/Gl9r2tk0qDnFTMrENK\nJo9cBhxLxFNNHJ3nWxfhhJJmZtYmp5BMHtmuCaUtnFTMzIrXZKAoO+Qb4qRiZlaMpLEkM4jMyjqU\nlnBSMTMrTpOBq0kmy+0w3FFvZlZspF2BJUAlEW9lE4I76s3MOovzgbuySiht4ZqKmVkxSWZcfxk4\ngYjHswvDNRUzsw6rds2U8xn7xJP06iX2G5h1TK3hpGJmlrHaNVNgzrFnsuPIH3FdHzhkejEvxtUQ\nJxUzs8wla6Z8mEVUspS7OJlkzZShk7OOrKWcVMzMMldRBjCVS7mCi6ihe1penGumNCaLlR/NzGw7\na6sPYiEH8Sif5Zac8uJcM6UxrqmYmWVu+Yz/5oQNP+D/UU1t5aR410xpjIcUm5llTfroOvSrfnzi\nyU3sVNZea6Y0HpKnvm+Qk4qZFS1JwMMkU7L8Mutwavk9FTOzjumTwE7ArVkHkg9OKmZmWZG6AT8E\nLiFiS9bh5IOTiplZdj4NbAJ+n3Ug+eIhxWZmWUjm+Po+8DU6Uee2aypmZtk4C3gN+HPWgeSTaypm\nZu0kmctryJRyystfpsfB/0fvS6bGm52mlgKuqZiZtYvcSSPP5aNHPsrHy6Zx4pc74qSRjfF7KmZm\n7UCaOBfmHNuLtbzACCYyhycYDUyaGzF7Ytbx1eX3VMzMiloyaeQPuYR7ODZNKNARJ41sjPtUzMza\nxdrqg1nAadzOKJ7OKe94k0Y2xjUVM7N20I+Xr76JYzd9nStZTb+0tGNOGtkY96mYmbUH6ZIVlB4/\nlI+tDirKi2HSyMZ4QslGOKmYWaakfYCHgLFEvJp1OM3hjnozs2KUzO81E7i0oySUtnBSMTMrrC8C\nJcA1WQfSHtz8ZWZWKNIgYBEwnohnsg6nJdz8ZWZWJKRRk7ppwtx57PrE1QxZI/bbI+uY2ktmSUXS\nMklPSnpc0sK0rK+keyW9IOlPkvrkHH+xpCWSnpP0iZzysZKeSvdNz+K7mJnVqp2O5WTOO7Y//fp9\nnSXD4JDpnW06loZkWVMJYHxEjImIg9OybwP3RsQI4L70M5JGAqcDI4EJwNVKluCEpJ3y3IgYDgyX\nNKE9v4SZ2faGTOnPDypnMIXzuI5N9ARmVsLQyVlH1h6ybv6q2153AnBTun0TcFK6fSJwa0Rsjohl\nwFJgnKSBQEVELEyPuznnHDOzdteDHcpv5zR+xhf5G4fl7Olc07E0JOuayp8lPSbp/LSsf0SsTLdX\nAv3T7d2BFTnnrgAG1VNelZabmWXieh7ZYy0VfJ/v1dnTuaZjaUiWc38dFhGvS9oVuFfSc7k7IyIk\n5W1omqRpOR/nRcS8fF3bzAwA6YyTUI896ftS0G3vbTuKfzoWSeOB8W29TmZJJSJeT//8p6TfAgcD\nKyUNiIg30qatN9PDq4AhOacPJqmhVKXbueVVDdxvWn6/gZlZDml/YEYv4phVPD4YJk1OmryKezqW\nWuk/tOfVfpY0tTXXySSpSNoBKImItZJ2BD4BXArMAs4GLk///F16yizgFklXkDRvDQcWprWZdyWN\nAxYCnwdmtO+3MbMuLxmpehfwb0Q8GfAkUNRJpFCyqqn0B36bDuAqBX4VEX+S9Bhwu6RzgWXAaQAR\nsVjS7cBioAa4ILa9tXkBcCNQDsyOiLnt+UXMrGuqXRq4G73K5rHzyN6sf+SAqP5V1nFlzW/Um5m1\n0LalgWdWXsIPOJZ7OIYRSzex8MJib+Zqrtb+3vQiXWZmLTZkCsysnMAcvsz/cSCPsYmBlUk/Stds\n9qrlpGJm1mIVZQfyKDdxNidzF28wMC3vGu+iNCbrlx/NzDqcw3ml+x84jnO5noc5PGdP13gXpTGu\nqZiZtYQ08l607xc58o0/cPyAbTuK/12U9uCOejOz5pIqSd7l+LbYb3Uyn1fHeRelJdxRb2aWZ7XD\nhqGibF9e5zG07w7EJUT8Mv3neKdJIvnipGJmVo/cYcO7U8XdHMn3OfCfl7O+qvO377SeO+rNzOqV\nDBvejZXcxzH8jC9yOQt37SpT2LeWk4qZWb0qyoaxlL9wJLdyBj/mW2m5hw03xknFzKweJ7C0/K8c\nzk/4d75P7tyKHjbcGPepmJnVJZ19GxpxKh+v+iNfzFmjycOGm+IhxWbWpeWO8OrGu9WP8OCag6ke\nCxwn9tu7Mw8bboyHFJuZtVDuCK9y1nMzZwEV1Ycy8F//Fi89F/AcHjbcIu5TMbMuLBnhNZDXeJCj\nWM8OHMHyskfY5+ysI+uonFTMrAurKDuVO3icMdzFyZzNTWyiJx7h1Xpu/jKzrkna+Y/sMmIYT3A8\nd/MoB+fs9Aiv1nJSMbNOL7czHtZW/4RH/3IRfHkwax8byydHvsfBw7Yd7RFebeHRX2bWqdXtjP9P\nvs2pXFdzB+XTLoxVP0r2d80RXo3x6C8zs3olnfFH8iDX8iX+zlj2o6r0HT53+IVAmkC6fBLJFycV\nM+vUDmDLztP4FGN4nH/nJ9zFKeked8YXgpOKmXUKdftNDuSFGx7lpUP+QumYy/ghZ3ArGynLOcOd\n8YXgpGJmHV5uv0kpm7mAq/keD3zsGXrecwEDz/4LS78PZZXbznBnfKE4qZhZJzBkSnf+t/Kz3Mh3\nuIwXGcaR/L1kMd/sFjH7VmnUGpjkzvh24KRiZh1K3Wau/Vly3WV0H/559mYxI/kK13A/x6RHJ/0m\n7oxvP04qZtZh1F2N8UKmcz73f/wv9H7neO5hEWPqnOF+k/bmaVrMrOhIoyZJE+dKp81L/hw1CUAM\nnnIo51T+nHN4iv3pwSZG80K3k9jnpUVcvXT7q7jfJAuuqZhZUcmtjdSW7c3pH3pIOz70CnHkWs7n\nRs7hIq7gbfqmRwx4Dx6Z6n6T7DmpmFmRSV5WLGc9J3MX53Ajo1m0533s1O1SBi+6j4WHQN0Xvddt\ncL9JcXDzl5llot4mLmm3i3htz7v4FK8zkDO4lWv5EoNZwWc44uX72PBD+KKbuYqYaypmVjB1R2rB\n8hkRT8/e1sT1s8p9eI4TmMWp/PdRm2HLRKrW3sg3OY/rWE2/nKsltRFpFG7mKl5OKmZWEPX1jcD5\nwz6tIbtdxI4XjaKmcjx7U0oNsziBS/hl2UNccc8GqmbAg9PhzHpfVnQzV3HzLMVm1iYN10YmzhV/\nPLaSpRzK3xjPPMYzj95UbVpAvzW/Z+quD3IUz7Iv2/pITnsw4vbxnjk4e56l2MwKpulmrJmVEOzJ\nMo7gXz/8mMr/uoDycSPoyzv0YQHjeICPcjnf4nm+90iwrhq+cuwH75S8V+LaSMflpGLWxTSUIBra\nl5yV24wVDOTMEZdrl2unsvPZu9KjciQf5QCeZAPlPMaBAx6m/4cX0PvFe/nz2LfYtU4E722A5VfB\n+cO2bxpzh3tn4KRi1gk1r2ZR6/xhSec35NY6+rKakZwzch9erNmNPfcaxheoZCkjWUwJW/ZazpZv\nPUd19cOM4Ld8iqcZxUoGpNc57TVY/GP4zvT6Eoc73DuvTtGnImkC8D9ACXBdRFxeZ7/7VKxotbTm\n0NS+BhLHUnjkwuT4OceWs55BVDGYFQxmBWP43guDqC4r56ChQ1jO3rxEIF5kGK/y+qbnOavHS+zN\niwxjMSN5gwHA6Q8m951TTzPWpLkRsye6b6Tj6rJ9KpJKgJ8CHwOqgEclzYqIZ7ONrGUkjY+IeVnH\n0ZQs42zZL9hHH4h46/J8/8LO9z54Y3/41HlN1xwa2xf0Yh39Oe9D39SAK05jwFnimMq+XM1uvEl/\nVtKfVZXDeeVXO/NSWW8q6M5mqhiUppTBrEflz9Bv45N8gRUM5iX25m12JulAP+49mNADxtf5L7Ku\nyWas9u4b8d+j7HX4pAIcDCyNiGUAkn4NnAhsl1SkiXO3/5dcsf2iqdxHmvhcccdYUQZDB0ujprR3\njM1vtqn14YOkUWVwyJlt+4VdmH1iKzvyhcrgz736cUn/HVnMDqyngrX04oTKvXj4sgpqSqvZp7IX\nl7IT79KbNfRmTeUerLi5FzWlJWzu3Yf+9OEdaihlNX33rGbtj95m05ZXuJNV9OOf7Moz7Mf9HM0G\n3lz2GlvXvsicI95lJ7Z/K33SMxCCk3Jir/Xui3Dh3vBE321lRduMNR6Yl9G9W2I8HSPOFusMSWUQ\nsDzn8wqFi4T8AAAHSElEQVRg3AcPm3Ns+hf8oOL8RTMNmLZHcccISZxV09s/xiFT4GeVYislbKEb\nWylhemVPTvy3UrZ2q+HHlSW8RQlbKGEL6zi67678Y0pwcd8SllDCFkqpoZSvVvblye92Z2u3dZxT\nWcqDaXkN3TmxcgDzf1DK1m5rOLqylF/Qnc30YBPdOaByEPf/pDtbu61hSGUPLqEHm+jJRnpSUrk7\n//xZD7Z2e4/VA3tyHD3ZSBnVlFFduQsv3daDrd2Cd3Yopx/lbKCMajZQPuxSqvkqR7KeHVjPDqyl\ngnX0ooS3B66jdMtyqlhLBa8zkOfYhzX0BlZUvU2PmuVc+y9r6M079KGa2pVxT/tHkozvqKdJ6udv\nJDWIbwxsuIO8vlrH6qnw0udgUt/6EodHalmuDt+nIukUYEJEnJ9+PhMYFxGTc46J2UwAoJRHN9dw\nUHcAse2792H+WwDv8JFdPrhvwVsBWsO4fnX39WbhKoA1HNwvtzw5r3bfQfXse2x1sm9sXxHcyMuc\nw16IoJQnNtfw4Q/E2JvH3wZ4l9E75+4TwU48+TaE1nJAn7r7Knj6HYHWsl/v3HIR9OLZNQK9x4d2\n+uC+JWsB1jOsorbsalbyNXalhFe2bGVoSW157c+OLN8AsJGB5bVl3diKCMp4cyNADX175u7rxla6\n825NN2ALO5TWltX+lLA5BKqdU6gmTR1b6UaweWtAbKKiZCvd0pRSwn/xHv9GTWyhv2rLNtOdLZTQ\ng5fXbUGxhlEVNZS+v6+GUnbg0VU1KFbx0V1qyzbRg030oIK7V2yiW6zkc0M2052NaUrZSE925P+e\n30i3ra/z3X030YMNlCcphTJK+PZj6ynZupobDq6mjPfYkWrKSGoKw6thSe4at6lJc5OaQ/39FY3v\ne/WqDybn816E+Tk1zPr7ORraJ2laREz74P2Ki+PMn9b2qXSGpPIRYFpETEg/Xwxsze2sl9Sxv6SZ\nWQa6alIpBZ4HjgFeAxYCZ3S0jnozs86gw/epRESNpK8B95AMKb7eCcXMLBsdvqZiZmbFo9OspyJp\ngqTnJC2R9K0GjpmR7n9CUt3FrNtFU3FKGi9pjaTH059LMojxBkkrJT3VyDHF8CwbjbMYnmUaxxBJ\nD0h6RtLTkqY0cFymz7Q5cWb9TCWVSVogaVEa47QGjsv6WTYZZ9bPsk4sJWkMdzewv/nPMyI6/A9J\ns9dSYE+gO7AI2LfOMZOA2en2OGB+kcY5HpiV8fM8AhgDPNXA/syfZTPjzPxZpnEMAEan271I+gCL\n8f/P5sSZ+TMFdkj/LAXmk4z2LKpn2cw4M3+WObFcBPyqvnha+jw7S03l/RcgI2IzUPsCZK4TgJsA\nImIB0EdS//YNs1lxwgfXSm1XEfEQ8HYjhxTDs2xOnJDxswSIiDciYlG6vY7kxdzd6xyW+TNtZpyQ\n/f+f69PNHiT/ONta55DMn2V676bihCL4/1PSYJLEcR31x9Oi59lZkkp9L0AOasYxgwscV13NiTOA\nQ9Nq5mxJI9stuuYrhmfZHEX3LCXtSVK7WlBnV1E900bizPyZSuomaRGwEvhTRDxa55CieJbNiDPz\nZ5m6Evgm9Sc9aOHz7CxJpbmjDepm4fYepdCc+/0DGBIRHwauAn5X2JBaLetn2RxF9Swl9QJ+A1yY\n1gQ+cEidz5k80ybizPyZRsTWiBhN8ottnKT96jks82fZjDgzf5aSjgPejIjHabzW1Ozn2VmSShUw\nJOfzEJJs2tgxg9Oy9tRknBGxtrbaHBFzgO6S+lJciuFZNqmYnqWk7sCdwC8jor5fHkXxTJuKs5ie\naUSsAR6AdLqMbYriWdZqKM4ieZaHAidIehm4FTha0s11jmnR8+wsSeUxYLikPSX1AE4HZtU5ZhZw\nFrz/Fv47EbGyfcNsOk5J/SUp3T6YZNj36naOsynF8CybVCzPMo3hemBxRPxPA4dl/kybE2fWz1TS\nLpL6pNvlwMepM3ksxfEsm4wz62cJEBHfiYghEbEX8Bng/og4q85hLXqeHf7lR2j4BUhJX0r3XxsR\nsyVNkrQUeA/412KMEzgV+IqkGmA9yX/odiXpVuAoYBdJy4GpJB2NRfMsmxMnRfAsU4cBZwJPSno8\nLfsOMBSK6pk2GSfZP9OBwE1KlrzoBtyWPrui+rvenDjJ/lnWJwDa8jz98qOZmeVNZ2n+MjOzIuCk\nYmZmeeOkYmZmeeOkYmZmeeOkYmZmeeOkYmZmeeOkYmZmeeOkYmZmeeOkYtbOJB2UzkzbU9KO6SJO\nmc+gbJYPfqPeLAOSfgCUAeXA8oi4POOQzPLCScUsA+lswI8BG4BDwn8RrZNw85dZNnYBdiRZtrc8\n41jM8sY1FbMMSJoF3ALsDQyMiMkZh2SWF51i6nuzjkTSWcDGiPi1pG7A3ySNj4h5GYdm1mauqZiZ\nWd64T8XMzPLGScXMzPLGScXMzPLGScXMzPLGScXMzPLGScXMzPLGScXMzPLGScXMzPLm/wNdwcwF\nXDnh0wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11298ca20>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(x, y, linestyle='' , marker='o', color='blue', label='data')\n",
    "plt.plot(x, f(x, a, b, c), linestyle='-', color='red' , label='model fit')\n",
    "plt.xlabel('x')\n",
    "plt.ylabel('y')\n",
    "plt.legend(numpoints=1, loc='upper left')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Uncertainties in Fit Parameters\n",
    "How do we estimate the uncertainties in the fit parameters $\\vec{a}=(a_1, a_2, \\ldots, a_M)$?\n",
    "* `scipy.optimize.curve_fit` returns:\n",
    "  * `popt`: array\n",
    "Optimal values for the parameters so that the sum of the squared error is minimized.\n",
    "  * `pcov`: 2d array\n",
    "The estimated covariance of `popt`. The diagonals provide the variance of the parameter estimate.\n",
    "* The standard deviation of $a_i$ is $\\sigma_i = \\sqrt{C_{ii}}$.\n",
    "  * The 68% confidence interval for $a_i$ is $\\pm \\sigma_i$.\n",
    "  * The 95% confidence interval for $a_i$ is $\\pm 2\\sigma_i$.\n",
    "  * etc."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example: Nonlinear Curve Fitting, Continued\n",
    "The results of the fit:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0: 2.5      +/- 2.40323e-06\n",
      "1: 1.33016  +/- 0.0382858\n",
      "2: 0.503916 +/- 0.0730082\n"
     ]
    }
   ],
   "source": [
    "popt, pcov = scipy.optimize.curve_fit(f, x, yn)\n",
    "for i in range(len(popt)):\n",
    "    params = (i, popt[i], np.sqrt(pcov[i,i]))\n",
    "    print('%d: %-8g +/- %g' % params)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[  5.77549120e-12  -6.08383097e-08   7.15178560e-08]\n",
      " [ -6.08383097e-08   1.46580140e-03  -2.40323648e-03]\n",
      " [  7.15178560e-08  -2.40323648e-03   5.33019536e-03]]\n"
     ]
    }
   ],
   "source": [
    "print(pcov) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[  5.77549120e-12   1.46580140e-03   5.33019536e-03]\n"
     ]
    }
   ],
   "source": [
    "print(np.diag(pcov))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}