线性回归/06. NFL定理和过拟合.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于一堆输入数据，我们可以用线性回归来拟合它，也可以用二次回归，三次回归，或者 N 次回归来拟合它。在函数逼近理论里，有一个定理叫做 **Weierstrass定理**，该定理阐述了在预先给定的精度下，可以用多项式逼近任意给定的闭区间上的连续函数。\n",
    "\n",
    "> Weierstrass第一逼近定理：假设 $f(x)$ 是闭区间 [a,b] 上的连续函数，对于任意的 $\\epsilon>0$，则存在多项式 $P(x)$ 使得对于所有的 $x\\in [a,b]$，有 $|f(x)-P(x)|<\\epsilon$。 \n",
    "\n",
    "也就是说使用多项式回归，随便什么样的输入数据，都可以逼近，因为回归问题的输入本身就是连续且在特定区间内的。多项式回归的表达式如下：\n",
    "\n",
    "$$\n",
    "y = \\theta_1\\phi_1(x) + \\theta_2\\phi_2(x) + ... + \\theta_k\\phi_k(x)\n",
    "$$\n",
    "\n",
    "其中，$\\phi(x) = (1, x, x_2, ..., x^{k-1})^T$，在实际求解中，这里的 k 值不可能取无穷大，我们可以从 k=2 开始（也就是线性模型），得到一个模型，评估模型是否满足条件，不满足的话，取 k=3，依次迭代，直到模型满足条件为止。\n",
    "\n",
    "我们再来看看之前的一元线性回归的数据："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD8CAYAAACb4nSYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAEXVJREFUeJzt3H+MnVd95/H3p7YDo7R0CHWixI6UsLUcVlslzs6iVJaqNqlqSKvGWxEJbbtYyJJ3JRax0sol3n/aSqwaZLUBtFK0XlJqWFqIsokTQYQbxaD+RdrxOsQFY8VlWeKxG7slTttllg3h2z/mDIydcXzv/MidmfN+SVf3ec49z+R7dJzPfebM8zypKiRJffiJURcgSXrjGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0ZKPSTjCd5JMk3k5xI8vNJrknyVJLn2/tbW98k+USSU0meS3L78g5BkjSoQc/0Pw58qapuAW4FTgD3AU9X1Rbg6bYP8G5gS3vtAR5c0oolSQuWK92cleQtwNeAt9eczklOAr9YVWeTXA98paq2JvlvbftPL+23bKOQJA1k/QB93g6cBz6V5FbgKPAh4LrZIG/Bf23rvwl4Yc7xp1vbRaGfZA8zvwlw9dVX/8tbbrllMeOQpO4cPXr0b6tq4zDHDBL664HbgQ9W1TNJPs6Pl3Lmk3naXvPrRFUdAA4ATExM1OTk5AClSJJmJfk/wx4zyJr+aeB0VT3T9h9h5kvgxbasQ3s/N6f/jXOO3wycGbYwSdLSu2LoV9XfAC8k2dqa7gK+ATwB7Gptu4DH2/YTwPvaVTx3AC+7ni9JK8MgyzsAHwQ+m+Qq4FvA+5n5wng4yW7gO8C9re+TwN3AKeB7ra8kaQUYKPSr6llgYp6P7pqnbwEfWGRdkqRl4B25ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwYK/STfTnI8ybNJJlvbNUmeSvJ8e39ra0+STyQ5leS5JLcv5wAkSYMb5kz/l6rqtqqaaPv3AU9X1Rbg6bYP8G5gS3vtAR5cqmIlSYuzmOWde4CDbfsgsHNO+6drxleB8STXL+K/I0laIoOGfgF/luRokj2t7bqqOgvQ3q9t7ZuAF+Yce7q1XSTJniSTSSbPnz+/sOolSUNZP2C/7VV1Jsm1wFNJvvk6fTNPW72moeoAcABgYmLiNZ9LkpbeQGf6VXWmvZ8DHgPeCbw4u2zT3s+17qeBG+ccvhk4s1QFS5IW7oqhn+TqJD81uw38CvBXwBPArtZtF/B4234CeF+7iucO4OXZZSBJ0mgNsrxzHfBYktn+f1JVX0ryl8DDSXYD3wHubf2fBO4GTgHfA96/5FVLkhbkiqFfVd8Cbp2n/e+Au+ZpL+ADS1KdJGlJeUeuJHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0JekjqwfdQGSdKlDx6bYf/gkZy5Mc8P4GHt3bGXntk2jLmtNMPQlrSiHjk2x79HjTL/yKgBTF6bZ9+hxAIN/Cbi8I2lF2X/45I8Cf9b0K6+y//DJEVW0thj6klaUMxemh2rXcAx9SSvKDeNjQ7VrOIa+pBVl746tjG1Yd1Hb2IZ17N2xdUQVrS3+IVfSijL7x1qv3lkehr6kFWfntk2G/DIZeHknybokx5J8oe3fnOSZJM8n+XySq1r7m9r+qfb5TctTuiRpWMOs6X8IODFn/6PAA1W1BXgJ2N3adwMvVdXPAg+0fpJWiEPHpth+/xFuvu+LbL//CIeOTY26JL2BBgr9JJuBXwU+2fYD3Ak80rocBHa27XvaPu3zu1p/SSM2e+PT1IVpih/f+GTw92PQM/2PAb8N/LDtvw24UFU/aPungdkFuE3ACwDt85db/4sk2ZNkMsnk+fPnF1i+pGF445OuGPpJfg04V1VH5zbP07UG+OzHDVUHqmqiqiY2btw4ULGSFscbnzTI1TvbgV9PcjfwZuAtzJz5jydZ387mNwNnWv/TwI3A6STrgZ8GvrvklUsa2g3jY0zNE/De+NSPK57pV9W+qtpcVTcB7wWOVNVvAl8G3tO67QIeb9tPtH3a50eq6jVn+pLeeN74pMVcp/9h4HNJPgIcAx5q7Q8Bn0lyipkz/PcurkRJS8Ubn5SVcBI+MTFRk5OToy5DklaVJEeramKYY3z2jiR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SerI+lEXIK0Uh45Nsf/wSc5cmOaG8TH27tjKzm2bRl2WtKQMfYmZwN/36HGmX3kVgKkL0+x79DiAwa81xeUdCdh/+OSPAn/W9Cuvsv/wyRFVJC0PQ18CzlyYHqpdWq0MfQm4YXxsqHZptbpi6Cd5c5K/SPK1JF9P8nut/eYkzyR5Psnnk1zV2t/U9k+1z29a3iFIi7d3x1bGNqy7qG1swzr27tg6ooqk5THImf73gTur6lbgNuBdSe4APgo8UFVbgJeA3a3/buClqvpZ4IHWT1pWh45Nsf3+I9x83xfZfv8RDh2bGur4nds28fu/8XNsGh8jwKbxMX7/N37OP+Jqzbni1TtVVcA/tt0N7VXAncC/ae0Hgd8FHgTuadsAjwD/NUnaz5GW3FJdebNz2yZDXmveQGv6SdYleRY4BzwF/DVwoap+0LqcBmb/b9kEvADQPn8ZeNs8P3NPkskkk+fPn1/cKNQ1r7yRBjfQdfpV9SpwW5Jx4DHgHfN1a+95nc/m/swDwAGAiYkJfwvQRYa5Ucorb6TBDXX1TlVdAL4C3AGMJ5n90tgMnGnbp4EbAdrnPw18dymKVR9ml2umLkxT/Hi55nLr9F55Iw1ukKt3NrYzfJKMAb8MnAC+DLynddsFPN62n2j7tM+PuJ6vYQy7XOOVN9LgBlneuR44mGQdM18SD1fVF5J8A/hcko8Ax4CHWv+HgM8kOcXMGf57l6FurWHDLtfMLvv43Bzpyga5euc5YNs87d8C3jlP+/8D7l2S6tSlG8bHmJon4F9vucYrb6TBeEeuVhyXa6Tl41M2teK4XCMtH0NfK5LLNdLycHlHkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOXDH0k9yY5MtJTiT5epIPtfZrkjyV5Pn2/tbWniSfSHIqyXNJbl/uQUiSBjPImf4PgP9UVe8A7gA+kOSfA/cBT1fVFuDptg/wbmBLe+0BHlzyqjVyh45Nsf3+I9x83xfZfv8RDh2bGnVJkgZwxdCvqrNV9b/a9j8AJ4BNwD3AwdbtILCzbd8DfLpmfBUYT3L9kleukTl0bIp9jx5n6sI0BUxdmGbfo8cNfmkVGGpNP8lNwDbgGeC6qjoLM18MwLWt2ybghTmHnW5tl/6sPUkmk0yeP39++Mo1MvsPn2T6lVcvapt+5VX2Hz45oookDWrg0E/yk8D/BP5jVf3963Wdp61e01B1oKomqmpi48aNg5ahFeDMhemh2iWtHAOFfpINzAT+Z6vq0db84uyyTXs/19pPAzfOOXwzcGZpytVKcMP42FDtklaOQa7eCfAQcKKq/nDOR08Au9r2LuDxOe3va1fx3AG8PLsMpLVh746tjG1Yd1Hb2IZ17N2xdUQVSRrU+gH6bAf+LXA8ybOt7T8D9wMPJ9kNfAe4t332JHA3cAr4HvD+Ja1YI7dz28yfaPYfPsmZC9PcMD7G3h1bf9QuaeVK1WuW299wExMTNTk5OeoyJGlVSXK0qiaGOcY7ciWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI5cMfST/FGSc0n+ak7bNUmeSvJ8e39ra0+STyQ5leS5JLcvZ/GSpOEMcqb/x8C7Lmm7D3i6qrYAT7d9gHcDW9prD/Dg0pQpSVoKVwz9qvpz4LuXNN8DHGzbB4Gdc9o/XTO+CownuX6pipUkLc76BR53XVWdBaiqs0mube2bgBfm9Dvd2s5e+gOS7GHmtwGA789dPlqDfgb421EXsYzW8vjW8tjA8a12W4c9YKGhfzmZp63m61hVB4ADAEkmq2piiWtZMRzf6rWWxwaOb7VLMjnsMQu9eufF2WWb9n6utZ8GbpzTbzNwZoH/DUnSElto6D8B7Grbu4DH57S/r13Fcwfw8uwykCRp9K64vJPkT4FfBH4myWngd4D7gYeT7Aa+A9zbuj8J3A2cAr4HvH/AOg4MV/aq4/hWr7U8NnB8q93Q40vVvEvukqQ1yDtyJakjhr4kdeQND/0kb07yF0m+luTrSX6vtd+c5Jn2aIfPJ7nqja5tsV5nbH+c5H8neba9bht1rYuRZF2SY0m+0PZX/dzNNc/41sz8Jfl2kuNtHJOtbd7HqqxGlxnf7yaZmjN/d4+6zoVIMp7kkSTfTHIiyc8vZO5Gcab/feDOqroVuA14V7vS56PAA+3RDi8Bu0dQ22JdbmwAe6vqtvZ6dnQlLokPASfm7K+FuZvr0vHB2pq/X2rjmL1+/XKPVVmtLh0fzPz7nJ2/J0dW2eJ8HPhSVd0C3MrMv9Gh5+4ND/32iIZ/bLsb2quAO4FHWvvcRzusGq8ztjUjyWbgV4FPtv2wBuZu1qXj68TlHquiFSLJW4BfAB4CqKr/X1UXWMDcjWRNv/36/CwzN3U9Bfw1cKGqftC6zD6+YdW5dGxV9Uz76L+0J48+kORNIyxxsT4G/Dbww7b/NtbI3DWXjm/WWpm/Av4sydH2KBS45LEqwLWXPXrlm298AP+hzd8frdLlq7cD54FPtaXHTya5mgXM3UhCv6perarbmLlj953AO+br9sZWtTQuHVuSfwHsA24B/hVwDfDhEZa4YEl+DThXVUfnNs/TdVXO3WXGB2tk/prtVXU7M0/E/UCSXxh1QUtsvvE9CPwzZpZczwJ/MML6Fmo9cDvwYFVtA/4vC1yGG+nVO+3Xk68AdzDzRM7Zm8VW/eMb5oztXVV1ti39fB/4FDNfdKvRduDXk3wb+BwzyzofY+3M3WvGl+R/rKH5o6rOtPdzwGPMjOVyj1VZdeYbX1W92E7Gfgj8d1bn/J0GTs9ZOXiEmS+BoeduFFfvbEwy3rbHgF9m5g8SXwbe07rNfbTDqnGZsX1zzqSEmTW3VflE0araV1Wbq+om4L3Akar6TdbA3MFlx/dba2X+klyd5Kdmt4FfYWYsl3usyqpyufHl4se7/2tW4fxV1d8ALySZfarmXcA3WMDcLfVTNgdxPXAwyTpmvnQerqovJPkG8LkkHwGO0f5gscpcbmxHkmxkZinkWeDfj7LIZfBhVv/cvZ7PrpH5uw54bOa7i/XAn1TVl5L8JfM/VmW1udz4PtMusy3g28C/G12Ji/JBZv4tXgV8i5nH3PwEQ86dj2GQpI54R64kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR35J0xtlV+NFPsrAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.xlim(30, 60)\n",
    "plt.ylim(100, 600)\n",
    "\n",
    "X = np.array([39.93, 42.05, 43.18, 44.68, 49.87, 53.57])\n",
    "Y = np.array([199,   290,   298,   310,   399,   420])\n",
    "plt.scatter(X, Y)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从图中很显然能看出这是一条直线，我们使用线性回归来求解（这里只是一个简单的示例，在实际问题的求解中，如果输入的维度非常大，很难通过图形直观的看出数据是不是线性的）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a = [[-360.87880256]\n",
      " [  14.93439994]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "X = np.matrix([\n",
    "    [1, 1, 1, 1, 1, 1], \n",
    "    [39.93, 42.05, 43.18, 44.68, 49.87, 53.57]]).T\n",
    "Y = np.matrix([199,   290,   298,   310,   399,   420]).T\n",
    "\n",
    "a = (X.T * X).I * X.T * Y\n",
    "print(\"a = {0}\".format(a))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "得到回归模型：\n",
    "\n",
    "$$\n",
    "y = 14.93x - 360.88\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD8CAYAAACb4nSYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl4VeW5/vHvQyZCGMI8JISZhCgiEHCoWgSRuWqPY9Wi9Wh7aqejh1pafwpixXmqFUur1qFWPdYqYZBBxKHOiIqZIEwhCRCmhCFz8v7+yMopYpAk7GRP9+e6cu29115r53mvBXdW3jxrLXPOISIi4aGNvwsQEZHWo9AXEQkjCn0RkTCi0BcRCSMKfRGRMKLQFxEJI40KfTOLN7NXzCzbzLLM7Awz62JmK81so/fY2VvXzOxRM8s1sy/NbFTLDkFERBqrsUf6jwBvOOdSgBFAFvAb4E3n3BDgTe81wBRgiPd1A7DApxWLiEiz2fFOzjKzjsAXwEB3xMpmlgOMc87tMLPewBrnXLKZ/cl7/vej12uxUYiISKNENmKdgcBu4GkzGwGsBX4J9KwPci/4e3jrJwDbj9g+31v2tdA3sxuo+02AuLi40SkpKScyDhGRsLN27do9zrnuTdmmMaEfCYwCfu6c+8jMHuHfUzkNsQaWfePXCefcQmAhQFpamvv0008bUYqIiNQzs21N3aYxc/r5QL5z7iPv9SvU/RDY5U3r4D0WHbF+3yO2TwQKm1qYiIj43nFD3zm3E9huZsneoglAJrAImOktmwm87j1fBPzQ6+I5HSjRfL6ISGBozPQOwM+Bv5lZNLAZuJa6Hxgvm9l1QB5wibfuUmAqkAuUeuuKiEgAaFToO+c+B9IaeGtCA+s64MYTrEtERFqAzsgVEQkjCn0RkTCi0BcRCSMKfRGRMKLQFxEJIwp9EZEwotAXEQkjCn0RkTCi0BcRCSMKfRGRMKLQFxEJIwp9EZEwotAXEQkjCn0RkTCi0BcRCSMKfRGRMKLQFxEJIwp9EZEwotAXEQkjCn0RkTCi0BcRCSMKfRGRIFRcWtms7SJ9XIeIiLSgmlrHCx/n8eCKnGZtr9AXEQkSH27ey5xFGWTvPMhpA7rweTM+Q6EvIhLg8veXMn9pNkvW7yAhPpbHrxzFlJN78fJPmv5ZCn0RkQBVVlnDE29v4om3N2EG/33eUH783YG0jYpo9mcq9EVEAoxzjsVf7mD+0iwKS8qZfkpvZk8dRkJ87Al/tkJfRCSAZBSWMDc9k4+37CO1d0ceuuxUThvY1Wefr9AXEQkA+w5Xcv+KHF78OI/4dtHcddFwLhvTl4g25tPvo9AXEfGjqppanv9wGw+t3MDhyhpmntmfX00YSqd2US3y/RT6IiJ+8t7GPcxNz2Bj0SHOGtyN22akMrRnhxb9ngp9EZFWlre3lDuXZLIicxd9u8Typ6tHc35qT8x8O5XTEIW+iEgrOVxRzeNrcvnzu1uIbGPMmpTMdWcNOKEWzKZS6IuItDDnHK9/Xsj8ZVnsOlDBRSMTuGVyCr06tW31WhT6IiItaH1+CXPSM1i7bT/DEzrx+JWjGN2vi9/qaVTom9lW4CBQA1Q759LMrAvwEtAf2Apc6pzbb3WTUo8AU4FS4Brn3Ge+L11EJHDtOVTBfW/k8PLa7XSNi+ae/xjOJaP70sbHLZhN1ZQj/XOdc3uOeP0b4E3n3N1m9hvv9S3AFGCI93UasMB7FBEJeZXVtTz7wVYeWbWRsqoa/vOsAfx8whA6tm2ZFsymOpHpnQuAcd7zZ4A11IX+BcCzzjkHfGhm8WbW2zm340QKFREJdGtyirhjcSabdx9mXHJ3/t/0VAZ1b+/vsr6msaHvgBVm5oA/OecWAj3rg9w5t8PMenjrJgDbj9g231v2tdA3sxuAGwCSkpKaPwIRET/bsucw8xZnsjq7iAHd4njqmjTGp/T0d1kNamzof8c5V+gF+0ozy/6WdRuasHLfWFD3g2MhQFpa2jfeFxEJdAfLq3hsdS5P/WsLMZERzJ6SwrXfGUB0ZODelLBRoe+cK/Qei8zsn8BYYFf9tI2Z9QaKvNXzgb5HbJ4IFPqwZhERv6qtdfzjs3zuXZ7D7oMVXDI6kVmTk+nRofVbMJvquKFvZnFAG+fcQe/5+cAdwCJgJnC39/i6t8ki4Gdm9iJ1f8At0Xy+iISKdXn7mZOeyRfbizm1bzx//mEap/aN93dZjdaYI/2ewD+904MjgRecc2+Y2SfAy2Z2HZAHXOKtv5S6ds1c6lo2r/V51SIirazoQDl3v5HNq58V0L1DDA9cMoKLRib4vQWzqY4b+s65zcCIBpbvBSY0sNwBN/qkOhERP6uoruGp97by2OqNVNU4fvLdQfxs/GDaxwTnua3BWbWISAtzzvFmVhF3Lslk695SzhvWk1unDaN/tzh/l3ZCFPoiIkfJLTrEHYszeWfDbgb3aM+zPxrLOUO7+7ssn1Doi4h4SsqqeGTVRp79YCux0RHcNj2Vq8/oR1RE4LZgNpVCX0TCXk2t4+VPt3P/8hz2lVZy+Zgk/uf8oXRtH+Pv0nxOoS8iYe2TrfuYsyiDjMIDjOnfmWdmjOXkhE7+LqvFKPRFJCztKClj/tJsFn1RSO9ObXn0ipHMOKV3q9y9yp8U+iISVsqravjzO5t5fM0mapzjF+MH85Nxg2gXHR5xGB6jFJGw55xjecZO7lySRf7+Mqac3IvfTh1G3y7t/F1aq1Loi0jIy9l5kLnpGby/aS/JPTvwwn+expmDu/m7LL9Q6ItIyCoureShlRt4/qM82sdEcscFJ/GDsUlEhlALZlMp9EUk5FTX1PL3T7bz4IocSsqquPK0ftw0cSid46L9XZrfKfRFJKR8sGkvc9MzyN55kNMHduH2GScxrHdHf5cVMBT6IhIS8veXctfSLJau30lCfCyPXzmKKSf3CvkWzKZS6ItIUCurrGHB25v409ubMIP/Pm8oP/7uQNpGRfi7tICk0BeRoOScY/GXO5i/NIvCknJmjOjD7Ckp9ImP9XdpAU2hLyJBJ6OwhLnpmXy8ZR+pvTvy8OUjGTugi7/LCgoKfREJGvsOV3L/ihxe/DiP+HbR3HXRcC4b05eIILt7lT8p9EUk4FXV1PL8h9t4aOUGDlfWMPPM/vxqwlA6tYvyd2lBR6EvIgHtvY17mJuewcaiQ5w9pBu3TU9lSM8O/i4raCn0RSQg5e0t5c4lmazI3EVSl3YsvHo0E1N7qgXzBCn0RSSgHK6o5vE1ufz53S1EtjFmTUrmurMGqAXTRxT6IhIQnHO8/nkh85dlsetABd8fmcAtU1Lo2bGtv0sLKQp9EfG79fklzEnPYO22/ZyS2InHrxzN6H6d/V1WSFLoi4jf7D5Ywf3Lc3h57Xa6xsVw78WncPGoRNqoBbPFKPRFpNVVVtfyzPtbefTNjZRX13D92QP5+fjBdGirFsyWptAXkVb1Vk4R8xZnsnn3Yc5N7s7/m57KwO7t/V1W2FDoi0ir2LLnMPMWZ7I6u4gB3eJ4+poxnJvSo8F1X1tXwH3LcygsLqNPfCyzJiVz4ciEVq44NCn0RaRFHSyv4rHVuTz1ry3EREbw26kpXHPmAKIjG7571WvrCpj96nrKqmoAKCguY/ar6wEU/D6g0BeRFlFb6/jHZ/nc80YOew5VcMnoRGZNTqZHh29vwbxvec7/BX69sqoa7lueo9D3AYW+iPjcurz9zEnP5IvtxYxMiufJmWmM6BvfqG0Li8uatFyaRqEvIj6z60A59yzL5tV1BfToEMODl47gwlMTmtSC2Sc+loIGAl7XyfcNhb6InLCK6hqefG8Lj63OpbrG8dNxg7jx3MHExTQ9YmZNSv7anD5AbFQEsyYl+7LksKXQF5Fmc86xKquIO5dksm1vKRNTe3LrtGH06xrX7M+sn7dX907LUOiLSLPkFh1kbnom727cw+Ae7XnuurGcPaS7Tz77wpEJCvkW0ujQN7MI4FOgwDk33cwGAC8CXYDPgKudc5VmFgM8C4wG9gKXOee2+rxyEfGLkrIqHl61gWc/2Ea76Ahum57K1Wf0Iyqi4RZMCSxNOdL/JZAFdPRe3wM85Jx70cyeAK4DFniP+51zg83scm+9y3xYs4icgOae+FRT63jpk+3cvyKH/aWVXDE2iZsnDqVr+5hWqFp8pVE/ms0sEZgG/MV7bcB44BVvlWeAC73nF3iv8d6fYLrrgUhAqD/xqaC4DMe/T3x6bV3Bt273ydZ9fO+x9/jtP9czqHsc6T87i7suGq7AD0KNPdJ/GPg1UH+Psq5AsXOu2nudD9QfKiQA2wGcc9VmVuKtv+fIDzSzG4AbAJKSkppbv4g0QVNPfNpRUsb8pdks+qKQ3p3a8ugVI5lxSm/dvSqIHTf0zWw6UOScW2tm4+oXN7Cqa8R7/17g3EJgIUBaWto33hcR32vsiU/lVTX8+Z3NPL5mEzXO8Yvxg/nJuEG0i1bvR7BrzB78DvA9M5sKtKVuTv9hIN7MIr2j/USg0Fs/H+gL5JtZJNAJ2OfzykWkyY534pNzjuUZO7lzSRb5+8uYcnIvfjt1GH27tGvtUqWFHHdO3zk32zmX6JzrD1wOrHbOXQm8BVzsrTYTeN17vsh7jff+auecjuRFAsCsScnEHnWv2foTn7J3HuDKv3zET57/jLjoSF64/jQWXDVagR9iTuR3tVuAF83sTmAd8KS3/EngOTPLpe4I//ITK1FEfKWhE59+Om4Q6/L2c9PLn9OhbRR3XHASPxibRKRaMEOSBcJBeFpamvv000/9XYZIWKmuqeXvH+fxwMoNHCir4srT+nHTxKF0jov2d2nSSGa21jmX1pRt9FcZkTD0waa9zE3PIHvnQU4f2IXbZ5zEsN4dj7+hBD2FvkgYyd9fyl1Ls1i6ficJ8bEsuHIUk0/upRbMMKLQFwkDZZU1LHh7E396exNmcNPEodxwzkDaHvVHXQl9Cn2REOacI/3LHcxfmsWOknJmjOjD7CkpujZ9GFPoi4SorwpKuCM9k4+37iO1d0ceuXwkYwd08XdZ4mcKfZEQs+9wJfevyOHvH+fRuV00d100nMvG9CWiCXevktCl0BcJEVU1tTz/4TYeWrmBw5U1XHNmf341YSid2kX5uzQJIAp9kRDw3sY9zE3PYGPRIc4e0o3bpqcypGeH428oYUehLxLE8vaWMm9JJiszd5HUpR0Lrx7NxNSeasGUY1LoiwShwxXV/PGtXP7y7hYiI4xZk5K57qwBasGU41LoiwQR5xyvfV7A3cuy2XWggotGJnDL5BR6dWrr79IkSCj0RYLEl/nFzFmUwWd5xQxP6MTjV45mdL/O/i5LgoxCXyTA7T5YwX3Ls/nftfl0jYvh3otP4eJRibRRC6Y0g0JfJEBVVtfy1/e38OibuVRU13D92QP5+fjBdGirFkxpPoW+SAB6K7uIeYsz2bznMONTenDrtGEM7N7e32VJCFDoiwSQzbsPMW9xJm/l7GZgtzievnYM5yb38HdZEkIU+iIB4GB5FX9YncvT/9pCTGQEv5s6jJln9ic6UnevEt9S6Iv4UW2t45XP8rn3jRz2Hq7gktGJzJqUQvcOMf4uTUKUQl/ETz7L28/cRRl8kV/CqKR4nromjVMS4/1dloQ4hb5IK9t1oJx7lmXz6roCenSI4cFLR3DhqQlqwZRWodAXaSUV1TU8+d4WHludS3WN46fjBnHjuYOJi9F/Q2k9+tcm0sKcc6zKKuLOJZls21vKxNSe3DptGP26xvm7NAlDCn2RFpRbdJC56Zm8u3EPg3u057nrxnL2kO7+LkvCmEJfpAWUlFXx8KoNPPvBNtpFR3Db9FSuPqMfURFqwRT/UuiL+FBNreOlT7Zz/4oc9pdWcsXYJG6eOJSu7dWCKYFBoS/iI59s3cecRRlkFB5gbP8u3P69VE7q08nfZYl8jUJf5AQVFpcxf1k26V8U0qdTW/5wxUimn9Jbd6+SgKTQF2mm8qoaFr6zmQVrNlHrHL8YP5j/GjeY2GjdvUoCl0JfpImcc7zx1U5+vzSL/P1lTB3ei9lThtG3Szt/lyZyXAp9kSbI3nmAuYsy+WDzXpJ7duCF60/jzEHd/F2WSKMp9EU8r60r4L7lORQWl9EnPpZZk5K5cGQCAMWllTy4cgPPf7iNjrFRzLvgJK4Ym0SkWjAlyCj0RagL/NmvrqesqgaAguIyZr+6nppaR2llNQ+s3MCBsiquPK0fN00cSue4aD9XLNI8Cn0R4L7lOf8X+PXKqmq45R9fUl3rOGNgV27/XiopvTr6qUIR31Doi1DXdtmQ6lrHgitHMfnkXmrBlJCgCUkRoE98bMPLO7VlynD13EvoOG7om1lbM/vYzL4wswwzm+stH2BmH5nZRjN7ycyiveUx3utc7/3+LTsEkRPjnGPCsB4cHeuxURH8enKKX2oSaSmNmd6pAMY75w6ZWRTwnpktA24CHnLOvWhmTwDXAQu8x/3OucFmdjlwD3BZC9UvAnx75823+aqghDvSM/l46z4S42Mpr65h76HKJn2GSDA5bug75xxwyHsZ5X05YDzwA2/5M8Ac6kL/Au85wCvAY2Zm3ueI+NyxOm+AY4b23kMV3L9iAy9+kkfndtHM//5wLk3rS4TuXiUhrlF/yDWzCGAtMBj4I7AJKHbOVXur5AP1/7sSgO0AzrlqMysBugJ7jvrMG4AbAJKSkk5sFBLWjtV5c9/ynG+EflVNLc99sI2HVm2grLKGa88cwC/PG0Kn2KjWLFnEbxoV+s65GuBUM4sH/gkMa2g177GhQ6VvHOU75xYCCwHS0tL0W4B8TVOma47VeXP08nc27OaOxZnkFh3i7CHduH1GKoN7dPB57SKBrEktm865YjNbA5wOxJtZpHe0nwgUeqvlA32BfDOLBDoB+3xXsoS6pk7X9ImPpaCB4K/vyNm29zB3LsliZeYu+nVtx59/mMZ5w3qoI0fCUmO6d7p7R/iYWSxwHpAFvAVc7K02E3jde77Ie433/mrN50tTfNt0TUNmTUomNurrV7aMjYrgF+MHc+8b2Ux88B3+lbuHX09OZsV/n8PE1J4KfAlbjTnS7w08483rtwFeds4tNrNM4EUzuxNYBzzprf8k8JyZ5VJ3hH95C9QtIayx0zX16o/+66eDendqy/iUHjy4agO7DlTw/ZEJ3DIlhZ4d27ZYzSLBojHdO18CIxtYvhkY28DycuASn1QnYel40zUNuXBkAheOTOCL7cXMTc/g+Y/yOCWxEwuuGs2opM4tWa5IUNFlGCTgzJqU/LU5faibrpk1KfmY2xQdLOe+N3L437X5dGsfw70Xn8LFoxJpoxZMka9R6EvAOXq65tu6dyqra/nr+1t49M1cKqpr+PE5A/nZ+MF0aKsWTJGGKPQlINVP13ybt7KLmLc4k817DjM+pQe3ThvGwO7tW6lCkeCk0Jegs3n3IeYtzuStnN0M7BbH09eO4dzkHv4uSyQoKPQlaBwsr+IPq3N5+l9biImM4HdThzHzzP5ER+pisSKNpdCXgFdb63hlbT73Ls9m7+FKLhmdyKxJKXTvEOPv0kSCjkJfAtpnefuZuyiDL/JLGJUUz1PXjOGUxHh/lyUStBT6EpB2HSjnnmXZvLqugJ4dY3joshFceGqCzqQVOUEKfQkoFdU1PPneFh5bnUt1jeOn4wZx47mDiYvRP1URX9D/JAkIzjlWZRVx55JMtu0tZWJqT26dNox+XeP8XZpISFHoi9/lFh1kbnom727cw+Ae7XnuurGcPaS7v8sSCUkKffGbkrIqHl61gWc/2Ea76Ahum57K1Wf0IypCLZgiLUWhL62uptbx0ifbuX9FDvtLK7libBI3TxxK1/ZqwRRpaQp9aVUfb9nH3PQMMgoPMLZ/F27/Xion9enk77JEwoZCX1pFYXEZ85dlk/5FIX06teUPV4xk+im91YIp0soU+tKiyqtqWPjOZh5fk4tz8IsJQ/iv7w4iNjri+BuLiM8p9KVFOOdY9tVOfr8ki4LiMqYO78XsKcPo26Wdv0sTCWsKffG57J0HmLsokw827yWlVwdeuP40zhzUzd9liQgKffGh/YcreXDlBv720TY6xkYx78KTuWJMXyLVgikSMBT6csKqa2p54eM8Hly5gYPl1Vx1ej9umjiU+HbR/i5NRI6i0JcT8v6mPdyRnkn2zoOcOagrt884ieReHfxdlogcg0JfmmX7vlLuWprFsq92khAfy4IrRzH55F5qwRQJcAp9aZLSymqeWLOJP72zmTZm3DxxKNefM5C2UWrBFAkGCn1pFOcc6V/uYP7SLHaUlDNjRB9mT0mhT3ysv0sTkSZQ6MtxfVVQwh3pmXy8dR8n9enIo1eMZEz/Lv4uS0SaQaEvx7T3UAX3r9jAi5/k0bldNPO/P5xL0/oS0Ubz9iLBSqEv31BVU8uzH2zj4VUbKKus4dozB/DL84bQKTbK36WJyAlS6MvXvLNhN3csziS36BBnD+nG7TNSGdxDLZgioUKhLwBs23uYeYuzWJW1i35d2/GXH6YxYVgPtWCKhBiFfpg7XFHNY2/l8uS7W4iKMG6ZnMKPzupPTKRaMEVCkUI/TNXWOl77vIC7l2VTdLCC749K4JbJKfTs2NbfpYlIC1Loh6EvthczJz2DdXnFjEjsxIKrRjO6X2d/lyUirUChH0Z2H6zg3jey+d+1+XRrH8N9F5/Cf4xKpI1aMEXChkI/DFRW1/LX97fw6Ju5VFTX8ONzBvKz8YPp0FYtmCLh5rihb2Z9gWeBXkAtsNA594iZdQFeAvoDW4FLnXP7ra7d4xFgKlAKXOOc+6xlypfjeSu7iHmLM9m85zDjU3pw67RhDOze3t9liYifNOZIvxq42Tn3mZl1ANaa2UrgGuBN59zdZvYb4DfALcAUYIj3dRqwwHuUVrR59yHmLc7krZzdDOwWx9PXjuHc5B4++/zX1hVw3/IcCovL6BMfy6xJyVw4MsFnny8iLeO4oe+c2wHs8J4fNLMsIAG4ABjnrfYMsIa60L8AeNY554APzSzezHp7nyMt7GB5FX9YncvT/9pC28gIfjd1GDPP7E90pO/uXvXaugJmv7qesqoaAAqKy5j96noABb9IgGvSnL6Z9QdGAh8BPeuD3Dm3w8zqDyMTgO1HbJbvLfta6JvZDcANAElJSc0oXY5UW+t4ZW0+9y7PZu/hSi4ZncisSSl07xDj8+913/Kc/wv8emVVNdy3PEehLxLgGh36ZtYe+AfwK+fcgW85U7OhN9w3Fji3EFgIkJaW9o33pfHWbtvP3PQMvswvYVRSPE/OHMOIvvEt9v0Ki8uatFxEAkejQt/MoqgL/L855171Fu+qn7Yxs95Akbc8H+h7xOaJQKGvCpZ/23WgnHuWZfPqugJ6dozhoctGcOGpCS1+6YQ+8bEUNBDwura+SOA77kSv143zJJDlnHvwiLcWATO95zOB149Y/kOrczpQovl83yqvquGPb+Vy7v1rWPzlDm48dxCrbx7HRSMTW+VaObMmJRN71J2yYqMimDUpucW/t4icmMYc6X8HuBpYb2afe8t+C9wNvGxm1wF5wCXee0upa9fMpa5l81qfVhzGnHOszNzFnUuyyNtXyvmpPbl1WipJXdu1ah318/bq3hEJPlbXZONfaWlp7tNPP/V3GQFt466D3LE4k3c37mFIj/bcPuMkzhrSzd9liYgfmdla51xaU7bRGbkBrqS0iodWbeC5D7cRFx3B7TNSuer0fkRF+K4FU0TCh0I/QNXUOl78JI8HVmyguLSSK8YmcfP5yXSJi/Z3aSISxBT6AeijzXuZm55J5o4DjB3QhdtnpHJSn07+LktEQoBCP4AUFJcxf2kWi7/cQZ9ObXnsByOZNry37l4lIj6j0A8A5VU1PPH2Jp54exPOwS8nDOEn3x1EbLTuXiUivqXQ9yPnHEvX7+SupVkUFJcxbXhvZk9NIbFz67Zgikj4UOj7SdaOA8xNz+DDzftI6dWBv19/OmcM6urvskQkxCn0W9n+w5U8sDKHFz7Ko2NsFPMuOIkrxiYRqRZMEWkFCv1WUl1Ty98+yuPBlRs4WF7FVaf346aJQ4lvpxZMEWk9Cv1W8H7uHuamZ5Kz6yBnDurKbTNSSenV0d9liUgYUui3oO37Svn9kizeyNhJYudYnrhqFJNO6qUWTBHxG4V+CyitrOaJNZt44p3NRJhx88ShXH/OQNpGqQVTRPxLoe9DzjnSv9zB/KVZ7Cgp53sj+jB7agq9O+k68yISGBT6PvJVQQlzFmXw6bb9nNSnI49eMZIx/bv4uywRka9R6J+gvYcquH9FDi9+sp3O7aKZ//3hXJrWl4g2mrcXkcCj0G+mqppanv1gGw+v2kBZZQ0/+s4AfjFhCJ1io/xdmojIMSn0m+GdDbu5Y3EmuUWHOGdod26bPozBPTr4uywRkeNS6DfBtr2Hmbc4i1VZu+jXtR1/+WEaE4b1UAumiAQNhX4jHK6o5rG3cnny3S1ERRi3TE7hR2f1JyZSLZgiElwU+t+ittbx2ucF3L0sm6KDFXx/VAK3TE6hZ8e2/i5NRKRZFPrH8MX2YuakZ7Aur5gRiZ144urRjErq7O+yREROiEL/KEUHy7n3jRxeWZtPt/Yx3HfxKfzHqETaqAVTREKAQt9TWV3LX9/fwqNv5lJRXcOPzxnIz8YPpkNbtWCKSOhQ6AOrs3cxb3EWW/YcZnxKD26dNoyB3dv7uywREZ8L69DftPsQ8xZnsiZnNwO7xfH0NWM4N6WHv8sSEWkxYRn6B8qr+MObG3n6X1uJjYrgd1OHMfPM/kRH6u5VIhLawir0a2sdr6zN597l2ew9XMmlo/vyP5OS6d4hxt+liYi0irAJ/bXb9jFnUSbrC0oY3a8zT18zluGJnfxdlohIqwr50N9ZUs7dy7J47fNCenaM4eHLTuWCU/vo0gkiEpZCNvTLq2p48r0t/PGtXKprHTeeO4ifjhtMXEzIDllE5LhCLgGdc6zM3MWdS7LI21fK+ak9uXVaKkld2/m7NBERvwup0N+46yB3LM7k3Y17GNqzPc9fdxpnDenm77JERAJGSIR+SWkVD63awHMfbiMuOoLbZ6Ry1enM3X4RAAAFgUlEQVT9iIpQC6aIyJGCOvRrah0vfpLHAys2UFxayRVjk7j5/GS6xEX7uzQRkYAUtKH/8ZZ9zFmUQeaOA4wd0IXbZ6RyUh+1YIqIfJvjhr6ZPQVMB4qccyd7y7oALwH9ga3Apc65/VbXB/kIMBUoBa5xzn3my4ILi8u4a2kWi7/cQZ9ObXnsByOZNry3WjBFRBqhMZPefwUmH7XsN8CbzrkhwJvea4ApwBDv6wZggW/KrGvBfGTVRsY/sIaVmbv45YQhvHnzOKafop57EZHGOu6RvnPuHTPrf9TiC4Bx3vNngDXALd7yZ51zDvjQzOLNrLdzbkdzC3TOseyrnfx+SRYFxWVMG96b2VNTSOysFkwRkaZq7px+z/ogd87tMLP6S1MmANuPWC/fW/aN0DezG6j7bQCgwsy+asw3ftz7CjLdgD3+LqIFhfL4QnlsoPEFu+SmbuDrP+Q2NM/iGlrRObcQWAhgZp8659J8XEvA0PiCVyiPDTS+YGdmnzZ1m+Y2su8ys97eN+0NFHnL84G+R6yXCBQ283uIiIiPNTf0FwEzveczgdePWP5Dq3M6UHIi8/kiIuJbjWnZ/Dt1f7TtZmb5wO3A3cDLZnYdkAdc4q2+lLp2zVzqWjavbWQdC5tWdtDR+IJXKI8NNL5g1+TxWV2jjYiIhANdnEZEJIwo9EVEwkirh76ZtTWzj83sCzPLMLO53vIBZvaRmW00s5fMLOiumvYtY/urmW0xs8+9r1P9XeuJMLMIM1tnZou910G/747UwPhCZv+Z2VYzW++N41NvWRczW+ntv5Vm1tnfdTbXMcY3x8wKjth/U/1dZ3N4J7u+YmbZZpZlZmc0Z9/540i/AhjvnBsBnApM9jp97gEe8i7tsB+4zg+1nahjjQ1glnPuVO/rc/+V6BO/BLKOeB0K++5IR48PQmv/neuNo75//ViXVQlWR48P6v591u+/pX6r7MQ8ArzhnEsBRlD3b7TJ+67VQ9/VOeS9jPK+HDAeeMVb/gxwYWvXdqK+ZWwhw8wSgWnAX7zXRgjsu3pHjy9MXEDdfoMg33+hysw6AucATwI45yqdc8U0Y9/5ZU7f+/X5c+pO6loJbAKKnXPV3ir1l28IOkePzTn3kffW783sSzN7yMxi/FjiiXoY+DVQ673uSojsO8/R46sXKvvPASvMbK13KRQ46rIqQI9jbh34GhofwM+8/fdUkE5fDQR2A097U49/MbM4mrHv/BL6zrka59yp1J2xOxYY1tBqrVuVbxw9NjM7GZgNpABjgC7UXZwu6JhZ/SW21x65uIFVg3LfHWN8ECL7z/Md59wo6q6Ie6OZnePvgnysofEtAAZRN+W6A3jAj/U1VyQwCljgnBsJHKaZ03B+7d7xfj1ZA5wOxJtZ/cliQX/5hiPGNtk5t8Ob+qkAnqbuB10w+g7wPTPbCrxI3bTOw4TOvvvG+Mzs+RDafzjnCr3HIuCf1I3lWJdVCToNjc85t8s7GKsF/kxw7r98IP+ImYNXqPsh0OR954/une5mFu89jwXOo+4PEm8BF3urHXlph6BxjLFlH7FTjLo5t0ZdUTTQOOdmO+cSnXP9gcuB1c65KwmBfQfHHN9VobL/zCzOzDrUPwfOp24sx7qsSlA51vjq95/nIoJw/znndgLbzaz+qpoTgEyase/8cbvE3sAzZhZB3Q+dl51zi80sE3jRzO4E1uH9wSLIHGtsq82sO3VTIZ8DP/FnkS3gFoJ/332bv4XI/usJ/LPuZxeRwAvOuTfM7BMavqxKsDnW+J7z2mwddXf6+7H/SjwhP6fu32I0sJm6y9y0oYn7TpdhEBEJIzojV0QkjCj0RUTCiEJfRCSMKPRFRMKIQl9EJIwo9EVEwohCX0QkjPx/YKIvT4ibuPoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.xlim(30, 60)\n",
    "plt.ylim(100, 600)\n",
    "\n",
    "X = np.array([39.93, 42.05, 43.18, 44.68, 49.87, 53.57])\n",
    "Y = np.array([199,   290,   298,   310,   399,   420])\n",
    "plt.scatter(X, Y)\n",
    "\n",
    "x = np.linspace(30, 60)\n",
    "y = 14.93*x-360.88\n",
    "plt.plot(x, y)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "那么这个模型是不是最好的呢？我们不妨再来求解二次回归和三次回归，甚至可以求出十次回归："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a2 = [[-2560.63241005]\n",
      " [  109.75507269]\n",
      " [   -1.01064275]]\n",
      "a3 = [[-8453.71313725]\n",
      " [  494.12237385]\n",
      " [   -9.32082522]\n",
      " [    0.0595445 ]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "# 不使用科学计数法\n",
    "np.set_printoptions(suppress=True)\n",
    "\n",
    "X2 = np.matrix([\n",
    "    [1, 1, 1, 1, 1, 1], \n",
    "    [39.93, 42.05, 43.18, 44.68, 49.87, 53.57],\n",
    "    [39.93**2, 42.05**2, 43.18**2, 44.68**2, 49.87**2, 53.57**2]]).T\n",
    "X3 = np.matrix([\n",
    "    [1, 1, 1, 1, 1, 1], \n",
    "    [39.93, 42.05, 43.18, 44.68, 49.87, 53.57],\n",
    "    [39.93**2, 42.05**2, 43.18**2, 44.68**2, 49.87**2, 53.57**2],\n",
    "    [39.93**3, 42.05**3, 43.18**3, 44.68**3, 49.87**3, 53.57**3]]).T\n",
    "Y = np.matrix([199,   290,   298,   310,   399,   420]).T\n",
    "\n",
    "a2 = (X2.T * X2).I * X2.T * Y\n",
    "a3 = (X3.T * X3).I * X3.T * Y\n",
    "print(\"a2 = {0}\".format(a2))\n",
    "print(\"a3 = {0}\".format(a3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "得到二次模型：\n",
    "\n",
    "$$\n",
    "y = 109.75507269x - 1.01064275x^2 - 2560.63241005\n",
    "$$\n",
    "\n",
    "和三次模型：\n",
    "\n",
    "$$\n",
    "y = 494.12237385x - -9.32082522x^2 + 0.0595445x^3 - 8453.71313725\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD8CAYAAACb4nSYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl4VNX9x/H3NythCWEJ+44ICLJGRVEU0SK44Y67Fotaq7ZarFurtdpf1VattVqpqLiBiBsqioAgdQEFw75I2BO2EEkIZJ85vz/mRiMESEKSm2Q+r+eZZ2bu3Jl8z3PhM3fOPfdcc84hIiLhIcLvAkREpPoo9EVEwohCX0QkjCj0RUTCiEJfRCSMKPRFRMJImULfzBLMbKqZrTazVWZ2opk1NbOZZrbWu2/irWtm9rSZpZjZUjMbULVNEBGRsirrnv4/gU+ccz2AvsAq4G5gtnOuGzDbew4wAujm3cYCz1VqxSIiUmF2uJOzzCweWAJ0cSVWNrM1wGnOuW1m1hqY65zrbmbPe48n7b9elbVCRETKJKoM63QB0oGXzKwvsAi4HWhZHORe8Lfw1m8LbCnx/lRv2c9C38zGEvolQIMGDQb26NHjSNohIhJ2Fi1atMs5l1ie95Ql9KOAAcCtzrkFZvZPfurKKY2VsuyAnxPOufHAeICkpCS3cOHCMpQiIiLFzGxTed9Tlj79VCDVObfAez6V0JfADq9bB+9+Z4n125d4fztga3kLExGRynfY0HfObQe2mFl3b9EwYCUwDbjWW3Yt8L73eBpwjTeKZxCQpf58EZGaoSzdOwC3Aq+bWQywHrie0BfGFDMbA2wGLvHWnQ6MBFKAHG9dERGpAcoU+s65xUBSKS8NK2VdB9xyhHWJiEgV0Bm5IiJhRKEvIhJGFPoiImFEoS8iEkYU+iIiYUShLyISRhT6IiJhRKEvIhJGFPoiImFEoS8iEkYU+iIiYUShLyISRhT6IiJhRKEvIhJGFPoiImFEoS8iEkYU+iIiYUShLyISRhT6IiJhRKEvIhJGFPoiImFEoS8iEkYU+iIiYUShLyISRhT6IiJhRKEvIhJGFPoiImFEoS8iEkYU+iIiYUShLyISRhT6IiJhRKEvIhJGFPoiImFEoS8iEkYU+iIiYaRMoW9mG81smZktNrOF3rKmZjbTzNZ690285WZmT5tZipktNbMBVdkAEREpu/Ls6Q91zvVzziV5z+8GZjvnugGzvecAI4Bu3m0s8FxlFSsiIp68rAq97Ui6d84HJnqPJwKjSix/xYXMBxLMrPUR/B0RESlp83x47uQKvbWsoe+AT81skZmN9Za1dM5tA/DuW3jL2wJbSrw31Vv2M2Y21swWmtnC9PT0ChUvIhJWAkUw56/w0ggwq9BHlDX0BzvnBhDqurnFzIYcYt3SKnEHLHBuvHMuyTmXlJiYWMYyRETCVMa6UNh//ij0uQxu+qJCHxNVlpWcc1u9+51m9i5wPLDDzFo757Z53Tc7vdVTgfYl3t4O2Fqh6kREwl0wCN/+F2Y+AJExcNEEOPbiCn/cYff0zayBmTUqfgz8AlgOTAOu9Va7FnjfezwNuMYbxTMIyCruBhIRkXLYvRFeOQ8+vgs6nQy3zD+iwIey7em3BN61UP9RFPCGc+4TM/sWmGJmY4DNwCXe+tOBkUAKkANcf0QVioiEG+dg4Yvw6R/BIuC8Z6D/VRXuxy/psKHvnFsP9C1leQYwrJTlDrjliCsTEQlH6d/Dh7+FTV9Cl9NCgZ/Q/nDvKrMy9emLiEgVK8qH/z0BXzwB0XFw7tMw4JpK2bsvSaEvIuK3jV+G9u53fQ/HXgLD/woNWxz+fRWg0BcR8Uv2dpj1ICyZBAkd4aq34agzqvRPKvRFRKpbUQEseA4+fwwCBXDyHTBkHMTUr/I/rdAXEalOa2fBJ3+AjBQ4egSc9Vdo2qXa/rxCX0SkOmxfDrMegJRZ0OwouHIqdDuz2stQ6IuIVKWsNJjzCCx+A+o1hl88AsePhagYX8pR6IuIVIXcTPjynzD/WXBBOOk3cMqdENfE17IU+iIilSlvDyx4Hr7+V2jO+2MvhdPvhyYd/a4MUOiLiFSO/L3wzfPw1b8gdzd0Hwmn3Q2tD5jQwFcKfRGRI1GwD759IdSVk5MB3YaHwr5tzbxSrEJfRKQicnfDtxNgwX9gX3ropKrT7oV2A/2u7JAU+iIi5ZGVFjo4u+hlKNgbCvshd0GHE/yurEwU+iIiZZG+JtSFs3RKaDRO74tg8O3QqrfflZWLQl9E5GCcg/VzYMF4+P5jiK4Px42BQb+uMaNxykuhLyKyv/xsWDIZvhkfmvmyQSKcdg8c9yto0Mzv6o6IQl9EpFjGulDQL34D8vdAmwFwwfPQ6wKIivW7ukqh0BeR8FZUAGumw3cTYd1nEBEdCvkTboR2SX5XV+kU+iISnnatDQX94kmQswvi24W6cAZeD41a+l1dlVHoi0j4KNgHqz6ARRNh81cQEQVHnwUDr4Oup0NEpN8VVjmFvojUbcEAbJgHS98MBX7B3tD89Wc8CH2vqNN79aVR6ItI3bRjRWgEzrK3IHsbxMaH+ur7joaOgyv9guO1hUJfROqOPdtg+VRY8ibsWBbqvjnqjNCFxruPgOg4vyv0nUJfRGq3Pdtg1TRY8S5sng+40FDLEY+Fzppt0NzvCmsUhb6I1D4/Bv17sPlrwEGLY0Kjb3pfCM27+V1hjaXQF5Ha4VBB32sUJHb3u8JaQaEvIjXOe8lpPD5jDTFZ67mowVJGxy+j+Q/JgIPEngr6I6DQF5GaIxjk8zmfsPPzybzMQrrFpkERrMroSHqPX9Nz2DXQooffVdZqCn0R8VWwIJe9q2ZTtOpDGmycxal56Qy2CBYEe/J64TBmBQeS6hKJWxnJ4IJs8osWUFAUBEKjLg3DDCIjjHrRkcQV32IiqR8TSUL9aJrUjwndGkTTtEEsreLrERdT90/EKo1CX0SqlHOO9L35bMrI8W772LlzG213zKPPvi85LpBMvOWz19VjZrAvMwOXMCfYjz00/Nnn5BYG2JaVR2xUBFGREaHPDoIjiHNQFHSkZ+eTVxggtzBAbkGAfQUBAkFXal0J9aNpFV+P1o3r0Tohjk7N6tOpWQO6JDagfdP6xEbVzS8Fhb6IVJqs3EJWb9vDmh3ZrNnu3XZkk51XRAfbwZkRC/lF5CKSIr4nkiCZUYmsaX4Ou9qejut4Ci0TGvH169+xJzv/gM9umxDHR7edUq56nHNk5xeRua+Q3TkF7M4p4Id9BWzLymN7Vl7ofk8ui7dksjun8Mf3RRi0bRJH95bxHNO6ET1ax9OzdTwdm9YnIqJ2n9Sl0BeRCskvCrBy6x6WpmaxZEsmi1MzWZ++78fXG9eLYGTTHdzachF99n5Jk33rAAi2OIaIHndC95EktOnPgP3OjL13ZE/ueWcZuYWBH5fFRUcybnj5D9qaGfH1oomvF02HZvUPuW5WTiEbMvaxYddeNuzKYV36XtZsz+az1Tso/rFQPyaSY9s2pl+HBPq3b8KADgm0iK9X7rr8ZM6V/tOnOiUlJbmFCxf6XYaIHEJuQYDkzbuZv+EHFqzPIHlL5o9964mNYunXPoH+beI4OXIlR/0wj7iNM7HsbWCR0PEk6D4ydFZs086H/VvFo3e2ZubSJiGOccO7M6p/26puYqnyCgOs3bGXVdv2sGJrFotTs1i5NYvCQCg72ybEcXznppzYpRmDujSjfdM4rJqmeDCzRc65cs3/rNAXkVIFg46V2/bw+ffpfP59Osmbd1MYcEQY9GrTmBM6NyWpUxP6JULL7fOwNR9ByuzQhGbRDeCoYdDjbOj2C6jf1O/mVKq8wgArt+0heXMm323ezYL1GezaWwCEvgQGdWnGkKObM6RbIk0axFRZHVUa+mYWCSwE0pxz55hZZ2Ay0BT4DrjaOVdgZrHAK8BAIAO4zDm38VCfrdAXqRmy8wqZuyadOWt2Mu/7XezaG+pb7902nsFHNWdQ52YM7NSE+LztoQuPrP4QNn0FwSJo2DK0J9/9bOg8BKJrV7fHkXDOkbJzL1+vz+DrdRnMX5/B7pxCIgz6d2jC0O6JnNa9Bb3axFfqr4CqDv07gCQg3gv9KcA7zrnJZvYfYIlz7jkz+zXQxzl3k5mNBi5wzl12qM9W6ItUn/27Tm46tQtRkRHMWLGdr1IyKAgEaVI/miFHJ3Lq0Ymc0i2RxIYxsH3ZT0G/fVnow5p3hx4jQ0HfdiBERPjbuBoiEHQsTc1kzpp05q7ZydLULCD0K2B4r1aMOLYVAzs0OeKDwlUW+mbWDpgIPALcAZwLpAOtnHNFZnYi8KBzbriZzfAef21mUcB2INEd4g8p9EWqx3vJaQccJC3Wvmkcw49pxVm9W9G/QxMiXVFoL371R7DmY8jaDBi0P+GnoG9+VPU3ohZKz85n7pqdzFixnXlrd1FQFCSxUSzDe7Xk3D5tOK5T0wp9AVQk9Ms6eucp4C6gkfe8GZDpnCvynqcCxUdZ2gJbALwvhCxv/V37FTsWGAvQoUOH8tQsIhWQU1DEnz9YUWrgt2gUy7xxQ7FAIayfC9PeC4V9XiZE1YMuQ+HUcaGrTDVsUf3F13KJjWK5JKk9lyS1Z29+EZ+t3skny7fx9qI0Xpu/mbYJcZzfrw0X9G9Lt5aNDv+BR+CwoW9m5wA7nXOLzOy04sWlrOrK8NpPC5wbD4yH0J5+maoVkXJxzrE0NYvJ325h2uI09hUcGPgxFNJn3yLsvamwejrkZ4UuONJ9JPQ8J3QZwZgGPlRfNzWMjeK8vm04r28bcgqKmLlyB+8mp/H8vPU8O3cdvdrEc/HAdlzYvx2N60dX+t8vy57+YOA8MxsJ1APiCe35J5hZlLe33w7Y6q2fCrQHUr3uncbAD5VeuYgcVFZuIe8vTmPSN1tYtW0P9aIjOKdPG+au3smufQXEUsCQiKWMiPyGMyIWEW+5sKZxKOSPOR+6nAZRsX43o86rHxPF+f3acn6/tqRn5/Ph0q28810af/5gJX/7eDUjj23N6OPac3znppV2ALhcQza9Pf3fewdy3wLeLnEgd6lz7lkzuwU4tsSB3Audc5ce6nPVpy9SOb7bvJvX5m9i+rJt5BUG6dUmntHHd+D8fm2Ij4avP32LXV+/xmn2HY0sl0zXgNnueFqfdBknnXERRFXd8EIpuxVbs5j8zRbeS04jO7+ILokNuPKEjlyS1I74ej/t/Vf5OP39Qr8LPw3ZTAaucs7lm1k94FWgP6E9/NHOufWH+lyFvkjFFQWCfLx8OxO+2MDiLZmh7oN+bbj8uA4c2zYeUheGLgq+4h3IyaAgujGfBI/j7dyBbGiUxB1n9fLtxCc5tJyCIj5auo1J32zmu82ZNIiJ5OKB7bjmpE50TWyok7NEwklWbiGTv9nMxK82sjUrj07N6nP94M5cPLAdDbI3wrIpsHQK7N4QOhjbfQT0uQy6DtMefS20LDWLl77awIdLtlEQCHLq0Ym8MuYEhb5IXbflhxxe+N963lqUSk5BgEFdmjLm5C4M6xhDxIqpsGQSpC0CLHSSVJ/LoOe5UC/e79KlEqRn5/PGgs28tmATC+8/U6EvUldt2LWPf89J4d3kNCIMzu3bhl+e1JHeBUsh+VVYOQ0C+dDyWOh7Weii4PFt/C5bqkhBUZDY6MgqG6cvIj5J2ZnNM5+lMG3JVqIjI7h6UEduHhhHy3Vvw9uvwe6NENsYBlwDA66G1n39LlmqQUxUxc5+VuiL1FCrtu3hmc9SmL58G/WiIrnh5M7c3GELTZY/Ai98HLqCSKdTYOh9oe6b6Di/S5ZaQKEvUsNsytjHPz79nmlLttIwNorfntySGxrNp8GS++HbtVC/OQy+PbRn37SL3+VKLaPQF6kh0rPzeeaztby+YDNRkcafjocrI6YTu/gtKNwH7Y6DC8ZDr1E6cUoqTKEv4rO9+UWMn7eeF/63nvyiAPf22MFVgfeJXTo3NNSy98Vw/A3Qpr/fpUodoNAX8UlRIMjrCzbz9Oy1ZO3L4b6Oq7ki8D6x61eE5qYf9icYeH2duwCJ+EuhL+KDL1N28ecPVpC2I50/tFjA6HofEbMjLTQ//XnPQJ9L1YUjVUKhL1KNtvyQwyMfreLLFeu5rdFnXNfoI6L3ZEHHwXDeE6FLC+pCJFKFFPoi1SC3IMBzc1N4fd5yro34hKcafky9wmw4egQM+T20K9f5NSIVptAXqULOOT5Zvp0nPviWEfve53+xn1A/uBe6joRT79LBWal2Cn2RKrI1M5eH311I55RXeCfmIxpF74NuZ3th38/v8iRMKfRFKlkg6HjlixQ2zBrPA/YWLaN3E+w2AobeoykSxHcKfZFKtCItk3cnT+DSrAlcH5FGfqskGPkIER0G+V2aCKDQF6kUuQUB3nzvXXouf5z7I1azt1FH3NmvENvzPKiky9yJVAaFvsgRWrZ6Ndun3sV1RZ+THdOE3KGP0XDQLyGy8i9qLXKkFPoiFZSfn8vXrz9M0qYX6G5FpPa+mXbn3gexjfwuTeSgFPoiFbBx/jQiP72b04JprIwfTIcrnqJd66P9LkvksBT6IuVQmLmVja/dSrdds9hEaxYP+S/9Tr/U77JEykyhL1IWwSC75o2n3ucP0SFYwAeJN3Dy1Q/SsbG6cqR2UeiLHIbbuYqMSTfTfHcyC+hNzvC/c+5JJ/pdlkiFKPRFDiZQSP7cvxP5v78T6erx74Q7ueC639OmSX2/KxOpMIW+SGl2riZnyq+ov2spHwROZPtJD3HT8OOIjNCYe6ndFPoiJQUDBL96huBnD5MXiOWRmLsYdd3NnNtJFzKRukGhL1Lsh/UUTh1L9NZv+TRwHJ91vZv7Lj2VxvV1kpXUHQp9EYAlkwl8cAe5RXB34DcMOPtXPHpCR0xTKEgdo9CX8Ja3Bzf9TmzpFL4L9uBvcXfw4NVncWy7xn5XJlIlFPoSvtIWEZw6BnZv4snCi1ne9QYmjB5IQv0YvysTqTIKfQk/zsHX/8bNeoB014TfFNzPkGHn8cLQo4jQ6Byp4xT6El7y9sD7t8Cqacxyx/OXiJt55PpTOKVbIu8lp/H4jDVszcylTUIc44Z3Z1T/tn5XLFKpFPoSPnauwr15FS5jA38tvJJvW13OpKuTaJsQx3vJadzzzjJyCwMApGXmcs87ywAU/FKnRPhdgEi1WDYV99/T2ZP5A6Pz7yWjz1jevOkk2ibEAfD4jDU/Bn6x3MIAj89Y40e1IlVGe/pStwWKYOYfYf6zrIw6hutzbuGXZ53IjUO6/Gw45tbM3FLffrDlIrWVQl/qrtxMmHo9rPuMSTaSRwuv4olrkzi9R8sDVm2TEEdaKQHfxvslIFJXHLZ7x8zqmdk3ZrbEzFaY2Z+95Z3NbIGZrTWzN80sxlse6z1P8V7vVLVNECnFrhR44QwC6+dxb9FY/lN/LFNvGVJq4AOMG96duOjIny2Li45k3PDu1VGtSLUpy55+PnC6c26vmUUDX5jZx8AdwJPOuclm9h9gDPCcd7/bOXeUmY0GHgUuq6L6RQB+NvLm/EZreIwnKXIRXJt3LzFdB/P+FQMOOf6++GCtRu9IXXfY0HfOOWCv9zTauzngdOAKb/lE4EFCoX++9xhgKvCMmZn3OSKVruTIm6siZ/JgwUTWubaMKfw9Q44fyEPn9SIq8vBjFkb1b6uQlzqvTKN3zCzSzBYDO4GZwDog0zlX5K2SChT/b2kLbAHwXs8CmpXymWPNbKGZLUxPTz+yVkhYe3zGGvIKC/lD1CQejn6JucG+XFjwIHtiW/PIqN5lCnyRcFGmA7nOuQDQz8wSgHeBnqWt5t2XdkrjAXv5zrnxwHiApKQk/QqQnynPiVLpmdk8Ef08F0R+yWtFw3ig6DoCRGJ5RZowTWQ/5Rq945zLNLO5wCAgwcyivL35dsBWb7VUoD2QamZRQGPgh8orWeq6cp0olZfFG/UfJym4jMcKL+PZwHkU73do5I3IgcoyeifR28PHzOKAM4BVwBzgYm+1a4H3vcfTvOd4r3+m/nwpjzKfKJWVBi+OoH9wJb8ruJlnA+dTHPgaeSNSurLs6bcGJppZJKEviSnOuQ/NbCUw2cweBpKBCd76E4BXzSyF0B7+6CqoW+qwMp0olbEOXjmfgr0/MKbgLrY0OYFWhUF27MnTyBuRQyjL6J2lQP9Slq8Hji9leR5wSaVUJ2HpsCdK7ViBe2UUufkFXJJzL626n8DHVwwgLibygPeIyM9pWIPUOIc8USp1Ee6lkWQXBDl333306H8Kz189UIEvUkaahkFqnIOeKJWwHvfKaHa5eC7YexcjTj6Be0b01Bz4IuWg0Jca6YATpb6fgXv9GtJoyYV772LMiBO58dSu/hUoUksp9KXmW/MJ7s2rWBfRkcv2/Z4/XHwylya197sqkVpJoS812/ef4qZczVo6ckXe3fzfVafwi16t/K5KpNZS6EvNtXYm7s0rWevac03RPTx9/Wmc1LW531WJ1GoKfamZ1s7CTb6S74Pt+GXwPv49ZigDOzb1uyqRWk+hLzVPyiyCk69gbbANN7j7ee5Xw+jTLsHvqkTqBIW+1CybviI4+UrWBtpwU8SfGD/2DHq2jve7KpE6Q6EvNcfWxQReu5RNRc24NeqP/HfsmRzVopHfVYnUKQp9qRnSv6folQvYWRjLnbEPMv5XZ9GpeQO/qxKpcxT64r/MzRS8fB578gLcWe8v/OvGc2jXpL7fVYnUSQp98dfeneS/eB75+/bw+3p/4e83XURbzYMvUmU04Zr4Jz+b3JdGEdyzlXEx9/PwjZcr8EWqmEJf/BEoZO+rVxCdsYr7ou/i/puuU5eOSDVQ6Ev1c47MKb+mYeo8Ho26md/e9GvaN1Xgi1QHhb5Uu10fPUTCmim8EHEpV950Hx2aKfBFqotCX6rVjs8n0HzhE3wYMZRhNz+pYZki1UyhL9VmZ/J0ms0Zx3zrQ8+xL9E5saHfJYmEHYW+VIuMDUto8P4vSaEdTa6bTNdWTfwuSSQsaZy+VLnMXdvJf/VSgi6WotGT6d2x7eHfJCJVQnv6UqWy9+Ww+fmLaRbIYNuIF+jd8xi/SxIJawp9qTK5+UXM//cY+hQuY+0Jj9Bn0Jl+lyQS9hT6UiUKioJMff4BzsyZztpuN9B75I1+lyQiKPSlCgSCjv+8NIHLM54ltcVpdLv8Mb9LEhGPQl8qlXOOp96ayTWpD7CnYWfajXkNIiL9LktEPBq9I5Xq2ZnLGb5iHPWijXpjpkKsLoIiUpMo9KXSTFqwiRbz7qV31EaCl0yGpl38LklE9qPuHakUn67YzrJp/+SSqHkEThlHRI8RfpckIqXQnr4csW83/sD4SVOZFD2RQJfTiRx6j98lichBKPTliKzZns2dL8/mregniWjUmsiLJ+jArUgNptCXCkvLzOX6CV/zhP2LFhHZ2OipUL+p32WJyCEo9KVCsnILue7Fbxhd+DaDWAIjn4Y2/f0uS0QOQwdypdwKioLc/Noimv2wiFvtLeh9MQy4xu+yRKQMDhv6ZtbezOaY2SozW2Fmt3vLm5rZTDNb69038ZabmT1tZilmttTMBlR1I6T6OOe4551lrFi3iRcbPo8ldIBzngQzv0sTkTIoy55+EXCnc64nMAi4xcyOAe4GZjvnugGzvecAI4Bu3m0s8FylVy2+eXp2Cm9/t4VXmr5MdN4uzts+hsFPLeS95DS/SxORMjhs6DvntjnnvvMeZwOrgLbA+cBEb7WJwCjv8fnAKy5kPpBgZq0rvXKpdu8mp/LkrO/5Y+L/6JvzFf9XeDlLXRfSMnO5551lCn6RWqBcffpm1gnoDywAWjrntkHoiwFo4a3WFthS4m2p3rL9P2usmS00s4Xp6enlr1yq1dfrMrhr6lIub/cDV2e/wKxAf14MnPXj67mFAR6fscbHCkWkLMoc+mbWEHgb+K1zbs+hVi1lmTtggXPjnXNJzrmkxMTEspYhPkjZmc2Nry6ke9MIHg48QYaLZ1zhjey/qbdm5vpToIiUWZlC38yiCQX+6865d7zFO4q7bbz7nd7yVKB9ibe3A7ZWTrlS3dKz87nupW+JiYpgcscPidy9gYdjf8du4g9Yt01CnA8Vikh5lGX0jgETgFXOuSdKvDQNuNZ7fC3wfonl13ijeAYBWcXdQFK75BUGuPHVhezam8+UoXtouPxVGHwbZ464iLjon591Gxcdybjh3X2qVETKqiwnZw0GrgaWmdlib9m9wN+AKWY2BtgMXOK9Nh0YCaQAOcD1lVqxVAvnHPe+s4zvNmfy34s60WXuKGjZG4bex6ioWAAen7GGrZm5tEmIY9zw7ozqrwuei9R0hw1959wXlN5PDzCslPUdcMsR1iU++8/n63knOY3fDevGmev+DHmZcM174AX+qP5tFfIitZDOyJUDzFy5g8dmrOacPq25rdkCWP0hDPsTtOzld2kicoQU+vIzq7bt4fbJyRzbtjH/OKMx9snd0OkUGKQfbyJ1gSZckx/t2pvPDRMX0qheFP+9qj+xb18IFgGjnoUI7R+I1AUKfQEgvyjATa8uImNfPlNuPJGWq16GLfNh1H8goYPf5YlIJVHoC8457nt3OQs37eaZK/rTJy4DZv8Fjj4L+o72uzwRqUQKfeHFLzcydVEqtw/rxjm9W8HEcyAyRrNnitRBCv0w99W6Xfx1+ip+cUxLbh/WDRa+AJu+hPOegfg2fpcnIpVMR+fCWOruHH7zRjKdmzfgicv6EZG1CWY+AF2HQf+r/C5PRKqAQj9M5RUGuOm1RRQGgoy/eiANYyJh2m2h0Trn/lPdOiJ1lLp3wtCPV7/auocJ1ybRJbEhLHoZNnwe6sdPaH/YzxCR2kl7+mHoxS838m5yGneccTSn92gJe7bCjPuh8xAYqKmSROoyhX6YKT5wO7xXS24ZelRo4cd3QbAQzn1a3ToidZxCP4yUPHD7j0v7ERFhsOZjWPUBnPoHaNrZ7xJFpIop9MPEAQduY6Mgfy9MHweJPeGkW/0uUUSqgQ7khokHp61gedoeXrjGO3ALMPf/IGsL/HIGREb7W6CIVAvt6YeBKQu3MPnbLdwytCtnHNMytHDbEpj/HAx0LXyFAAAH+klEQVS8DjoM8rU+Eak+Cv06bsXWLP743nJO6tqMO870LmcYDMAHv4X6TeGMB/0sT0Sqmbp36rCs3EJ+/fp3JNSP5unL+xMZ4Y3M+XYCbP0OLnwB4pr4W6SIVCuFfh3lnOP3by0hbXcub944iOYNQ5c5JHs7zH4IupwGx17sZ4ki4gN179RRz89bz8yVO7h3ZE8Gdmz60wsz/wSBfDj7CY3JFwlDCv066Ot1GTz2yWrO7tOa6wd3+umFTV/D0jdDwzObdfWtPhHxj0K/jtmxJ49bJyXTqXkDHr2oD1a8Nx8MwMfjIL4tnHKnv0WKiG/Up1+HFAWC3DYpmX35RbzxqxNCJ2AVW/QSbF8GF78EMQ38K1JEfKXQr0Oenr2WBRt+4IlL+3J0y0Y/vbDPu/xhp1Og1wX+FSgivlP3Th3xZcou/jUnhYsHtuPCAe1+/uJnf4H8bBjxmA7eioQ5hX4dsDM7j9snL6ZrYkMeOr/Xz1/cmhyaK/+EG6HlMb7UJyI1h7p3arlA0PG7NxezN7+Q1284gfoxJTZpMAjT74IGzeG0u/0rUkRqDIV+LffsnBS+TMng0YuOpXurRj9/cflUSP0Gzv831GvsT4EiUqOoe6cWW7A+gydnfc/5/dpwadJ+lzgsyIFZD0LrvtD3Cl/qE5GaR3v6tVTG3nxum5xMx2YNeOSCY38aj19s/r9hTxpcOB4i9N0uIiFKg1ooGHTc+dYSducU8swV/X8+Hh8gewd88RT0OAc6nexPkSJSIyn0a6EJX2xg7pp0/nh2T3q1KaWvfs4jUJQHZz5U/cWJSI2m0K9llqdl8diM1Qzv1ZKrBnU8cIXtyyH5VTh+rObXEZEDKPRrkZyCIm6blEyzBrH87cI+B/bjOwef3g+x8TBknD9FikiNptCvRR76YCUbMvbxxGV9adIg5sAVUmbB+jmhMfn1mx74uoiEvcOGvpm9aGY7zWx5iWVNzWymma317pt4y83MnjazFDNbamYDqrL4cDJ92TYmf7uFm0/tykldmx+4QqAIZtwHTbtC0pjqL1BEaoWy7Om/DJy137K7gdnOuW7AbO85wAigm3cbCzxXOWWGt62Zudz99lL6tmvM7848uvSVkl+FXWtCB2+jSvkVICJCGULfOTcP+GG/xecDE73HE4FRJZa/4kLmAwlm1rqyig1HgaDjt28uJhB0/HN0f6IjS9lkBTnw+aPQ/gTocXb1FykitUZFT85q6ZzbBuCc22ZmLbzlbYEtJdZL9ZZt2/8DzGwsoV8DAPklu4/qoObAriP9kM5/Odwaa+AGXw7TVEr7aqi63DZQ+2q77uV9Q2WfkVvavL2utBWdc+OB8QBmttA5l1TJtdQYal/tVZfbBmpfbWdmC8v7noruFu4o7rbx7nd6y1OBkpPAtAO2VvBviIhIJato6E8DrvUeXwu8X2L5Nd4onkFAVnE3kIiI+O+w3TtmNgk4DWhuZqnAA8DfgClmNgbYDFzirT4dGAmkADnA9WWsY3z5yq511L7aqy63DdS+2q7c7TPnSu1yFxGROkhn5IqIhBGFvohIGKn20Dezemb2jZktMbMVZvZnb3lnM1vgTe3wppnVutNKD9G2l81sg5kt9m79/K71SJhZpJklm9mH3vNav+1KKqV9dWb7mdlGM1vmtWOht6zUaVVqo4O070EzSyux/Ub6XWdFmFmCmU01s9VmtsrMTqzItvNjTz8fON051xfoB5zljfR5FHjSm9phN1AbJ5A5WNsAxjnn+nm3xf6VWCluB1aVeF4Xtl1J+7cP6tb2G+q1o3j8+sGmVamt9m8fhP59Fm+/6b5VdmT+CXzinOsB9CX0b7Tc267aQ9+bomGv9zTauzngdGCqt7zk1A61xiHaVmeYWTvgbOAF77lRB7Zdsf3bFyYONq2K1BBmFg8MASYAOOcKnHOZVGDb+dKn7/18XkzopK6ZwDog0zlX5K1SPH1DrbN/25xzC7yXHvFmHn3SzGJ9LPFIPQXcBQS9582oI9vOs3/7itWV7eeAT81skTcVCuw3rQrQ4qDvrvlKax/Ab7zt92It7b7qAqQDL3ldjy+YWQMqsO18CX3nXMA514/QGbvHAz1LW616q6oc+7fNzHoD9wA9gOOApsAffCyxwszsHGCnc25RycWlrFort91B2gd1ZPt5BjvnBhCaEfcWMxvid0GVrLT2PQd0JdTlug34h4/1VVQUMAB4zjnXH9hHBbvhfB294/08mQsMIjQjZ/HJYrV++oYSbTvLObfN6/rJB14i9EVXGw0GzjOzjcBkQt06T1F3tt0B7TOz1+rQ9sM5t9W73wm8S6gtB5tWpdYprX3OuR3ezlgQ+C+1c/ulAqkleg6mEvoSKPe282P0TqKZJXiP44AzCB2QmANc7K1WcmqHWuMgbVtdYqMYoT63WjmjqHPuHudcO+dcJ2A08Jlz7krqwLaDg7bvqrqy/cysgZk1Kn4M/IJQWw42rUqtcrD22c+nd7+AWrj9nHPbgS1mVjyr5jBgJRXYdpU9y2ZZtAYmmlkkoS+dKc65D81sJTDZzB4GkvEOWNQyB2vbZ2aWSKgrZDFwk59FVoE/UPu33aG8Xke2X0vg3dB3F1HAG865T8zsW0qfVqW2OVj7XvWG2TpgI3CjfyUekVsJ/VuMAdYTmuYmgnJuO03DICISRnRGrohIGFHoi4iEEYW+iEgYUeiLiIQRhb6ISBhR6IuIhBGFvohIGPl/lxVElb9n9CgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.xlim(30, 60)\n",
    "plt.ylim(100, 600)\n",
    "\n",
    "X = np.array([39.93, 42.05, 43.18, 44.68, 49.87, 53.57])\n",
    "Y = np.array([199,   290,   298,   310,   399,   420])\n",
    "plt.scatter(X, Y)\n",
    "\n",
    "x = np.linspace(30, 60, 200)\n",
    "y = 109.75507269*x-1.01064275*x**2-2560.63241005\n",
    "plt.plot(x, y)\n",
    "\n",
    "x = np.linspace(30, 60, 200)\n",
    "y = 494.12237385*x-9.32082522*x**2+0.0595445*x**3-8453.71313725\n",
    "plt.plot(x, y)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "蓝色曲线是二次回归的结果，橙色曲线是三次回归的结果，可以看出拟合程度也很高，而且随着多项式回归的维数的增加，回归线可以无限的接近输入的数据。那么，我们到底该使用哪个回归模型呢？\n",
    "\n",
    "根据 **奥卡姆剃刀原则**（Occam's razor）我们可能会认为最简单的往往是最有效的，既然一元线性回归能解决问题，为什么要使用多项式回归呢？虽然这个原则在大多数情况下很实用，但是在很多时候，“最简单”是一个模棱两可的指标，什么是“最简单”并不好判定。\n",
    "\n",
    "其实，更多的时候，我们并不是要追求最简单的模型，而是希望我们的模型能适应更多训练集之外的情景，也就是模型要有举一反三的能力，即泛化能力。从直观上来看，在上面的场景中，线性回归和多项式回归比较起来，我们更希望线性回归更好一点。遗憾的是，虽然一元线性回归能在训练集上得到很好的拟合结果，但是在训练集之外的样本数据上，完全可能出现二次曲线或三次曲线的情况，而且谁也不能保证在训练集之外的数据是什么样子的。\n",
    "\n",
    "而且可以证明的是（证明过程可以参见周志华《机器学习》1.4节 归纳偏好）：学习算法在样本上的总误差与学习算法本身无关，任意两个学习算法它们的期望性能是相同的，也就是说，你所见过的所有机器学习算法，简单的线性回归也好，高级的决策树、支持向量机也好，甚至神经网络、深度学习，他们的期望性能和随机瞎猜是一样的。这个定理被称为 **NFL 定理**（No Free Lunch Theorem，没有免费的午餐定理）。\n",
    "\n",
    "那么既然如此，还要搞这么多的机器学习算法干什么呢？实际上，NFL 定理有一个很重要的前提：所有的问题出现的机会相同，或者说所有的问题同等重要。很显然，事实并非如此，我们一般只关注自己正在解决的问题，对其他问题并不关心。所以，NFL 定理的真实意图是告诉我们，判断模型的好坏要结合具体的问题，脱离具体问题的模型毫无意义，就譬如上面的几个散点数据，如果你不知道这是什么数据，不知道在解决什么问题，只是抽象的数字，得到的模型也无法得知孰好孰坏。\n",
    "\n",
    "回到我们上面的问题，现在有三个模型对数据集的拟合都很好，能看出二次回归和三次回归对数据的误差要更小，根据上面介绍的 Weierstrass 定理，我们甚至可以计算出一个多项式回归模型，使得误差趋近于 0，这样的模型是不是最好的呢？\n",
    "\n",
    "这里要引入几个概念，我们把模型预测结果和样本的真实结果之间的差异称为 **误差**（error），训练集上得到的误差称为 **训练误差**（training error）或 **经验误差**（empirical error），在新样本上的误差称为 **泛化误差**（generalization error），上面我们用复杂的模型把误差降的很低，其实降低的是训练误差，而我们显然是希望模型在新样本上表现的很好，也就是泛化误差要小，当我们把模型训练的太复杂时，模型的泛化能力反而会下降。这种现象被称为 **过拟合**（overfitting）。\n",
    "\n",
    "过拟合是机器学习算法中一个非常重要，也非常棘手的问题，各种机器学习算法都避免不了，有多种因素可能会导致过拟合，其中最常见的情况是由于学习能力过于强大，以至于把训练样本所包含的不太一般的特性都学习到了。与之相反的一种场景是 **欠拟合**（underfitting），通常是由于学习能力低下造成的。\n",
    "\n",
    "针对这个问题我们后面还会学习如何尽可能的避免过拟合。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}