逻辑回归/04. 使用梯度下降解逻辑回归.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 梯度下降法\n",
    "\n",
    "根据上一节得到的参数 w 和 b 的更新公式，和线性回归一样，为了保证求值稳定，我们在求和的基础上乘以 $\\frac{1}{n}$：\n",
    "\n",
    "$$\n",
    "w := w - \\eta \\frac{\\partial loss}{\\partial w} = w + \\eta \\frac{1}{n} \\sum_{i=1}^n (y_i - \\pi(x_i))x_i\n",
    "$$\n",
    "\n",
    "$$\n",
    "b := b - \\eta \\frac{\\partial loss}{\\partial b} = b + \\eta \\frac{1}{n} \\sum_{i=1}^n (y_i - \\pi(x_i))\n",
    "$$\n",
    "\n",
    "我们就可以用代码实践一把了。首先把最开始的例子散点图画出来："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAD8CAYAAABkbJM/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAFKpJREFUeJzt3X+QXeV93/H3pxI/ZJpUgGQbBLJxqmI3/gH2HWyXjoMbA7LHgxjqFNGmxg6uZjKhruMJDdRpmJJ4TEpnbHfs2laIbMg0kJoCUX/YCsbxuNOElJXBCOTIKMI1iwhSjMHjogGEv/3jno0uy670rO7V7mr3/Zq5s+c853nu/d5nrvaj8+OeTVUhSdKh/K25LkCSdHQwMCRJTQwMSVITA0OS1MTAkCQ1MTAkSU1GEhhJNiXZk+TBabafl+TpJPd3j98c2LY2yY4kO5NcPYp6JEmjl1F8DyPJO4AfAzdX1eun2H4e8GtV9d5J7UuA7wLnA+PAvcBlVbV96KIkSSM1kj2Mqvom8ORhDD0H2FlVu6rqOeBWYN0oapIkjdbSWXyttyf5NrCb/t7GQ8Aq4NGBPuPAW6canGQDsAHghBNOeMtrX/vaI1yuJC0sW7du/euqWnm442crML4FvKqqfpzkPcCdwBogU/Sd8hhZVW0ENgL0er0aGxs7UrVK0oKU5P8OM35WrpKqqh9V1Y+75f8JHJNkBf09itMHup5Gfw9EkjTPzEpgJHllknTL53Sv+wP6J7nXJDkjybHAemDzbNQkSZqZkRySSnILcB6wIsk4cC1wDEBVfR54H/DLSfYD+4D11b88a3+SK4EtwBJgU3duQ5I0z4zkstrZ5jkMSZq5JFurqne44/2mtySpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqMpLASLIpyZ4kD06z/Z8leaB7/GmSNw1s+16SbUnuT+LfXZWkeWpUexhfAtYeZPsjwM9V1RuB3wI2Ttr+zqo6a5i/NStJOrKWjuJJquqbSV59kO1/OrB6D3DaKF5XkjR75uIcxhXAVwbWC/jjJFuTbJiDeiRJDUayh9EqyTvpB8Y/HGg+t6p2J3k5cFeSv6iqb04xdgOwAWD16tWzUq8k6YBZ28NI8kbgRmBdVf1gor2qdnc/9wB3AOdMNb6qNlZVr6p6K1eunI2SJUkDZiUwkqwGbgf+eVV9d6D9hCQ/NbEMXABMeaWVJGlujeSQVJJbgPOAFUnGgWuBYwCq6vPAbwInA/8pCcD+7oqoVwB3dG1LgT+oqq+OoiZJ0miN6iqpyw6x/UPAh6Zo3wW86aUjJEnzjd/0liQ1MTAkSU0MDElSEwNDktTEwJAkNTEwJElNDAxJUhMDQ5LUxMCQJDUxMCRJTQwMSVITA0OS1MTAkCQ1MTAkSU0MDElSEwNDktTEwJAkNTEwJElNDAxJUpOR/E3vJJuA9wJ7qur1U2wP8GngPcAzwAeq6lvdtsuB3+i6/nZV3TSKmqTF6M77HuOGLTvY/dQ+Tl2+jKsuPJOLz1616GqYT3UsJCMJDOBLwGeAm6fZ/m5gTfd4K/A54K1JTgKuBXpAAVuTbK6qH46oLmnRuPO+x7jm9m3se/4FAB57ah/X3L4NYNZ+Uc6HGuZTHQvNSA5JVdU3gScP0mUdcHP13QMsT3IKcCFwV1U92YXEXcDaUdQkLTY3bNnxN78gJ+x7/gVu2LJjUdUwn+pYaGbrHMYq4NGB9fGubbr2l0iyIclYkrG9e/cesUKlo9Xup/bNqH2h1jCf6lhoZiswMkVbHaT9pY1VG6uqV1W9lStXjrQ4aSE4dfmyGbUv1BrmUx0LzWwFxjhw+sD6acDug7RLmqGrLjyTZccseVHbsmOWcNWFZy6qGuZTHQvNbAXGZuD96Xsb8HRVPQ5sAS5IcmKSE4ELujZJM3Tx2av4xCVvYNXyZQRYtXwZn7jkDbN6knc+1DCf6lhoUjXlEaCZPUlyC3AesAJ4gv6VT8cAVNXnu8tqP0P/hPYzwAeraqwb+0vAv+me6uNV9cVDvV6v16uxsbGh65akxSTJ1qrqHe74kVxWW1WXHWJ7Ab8yzbZNwKZR1CFJOnL8prckqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJajKSwEiyNsmOJDuTXD3F9k8mub97fDfJUwPbXhjYtnkU9UiSRm/ov+mdZAnwWeB8YBy4N8nmqto+0aeqfnWg/78Ezh54in1VddawdUiSjqxR7GGcA+ysql1V9RxwK7DuIP0vA24ZwetKkmbRKAJjFfDowPp41/YSSV4FnAF8faD5+CRjSe5JcvF0L5JkQ9dvbO/evSMoW5I0E6MIjEzRVtP0XQ/cVlUvDLStrqoe8E+BTyX5makGVtXGqupVVW/lypXDVSxJmrFRBMY4cPrA+mnA7mn6rmfS4aiq2t393AV8gxef35AkzROjCIx7gTVJzkhyLP1QeMnVTknOBE4E/myg7cQkx3XLK4Bzge2Tx0qS5t7QV0lV1f4kVwJbgCXApqp6KMl1wFhVTYTHZcCtVTV4uOp1wBeS/IR+eF0/eHWVJGn+yIt/fx8der1ejY2NzXUZknRUSbK1O2d8WPymtySpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqMpLASLI2yY4kO5NcPcX2DyTZm+T+7vGhgW2XJ3m4e1w+inokSaO3dNgnSLIE+CxwPjAO3Jtkc1Vtn9T1D6vqykljTwKuBXpAAVu7sT8cti5J0miNYg/jHGBnVe2qqueAW4F1jWMvBO6qqie7kLgLWDuCmiRJIzaKwFgFPDqwPt61TfaPkzyQ5LYkp89wLEk2JBlLMrZ3794RlC1JmolRBEamaKtJ6/8NeHVVvRH4GnDTDMb2G6s2VlWvqnorV6487GIlSYdnFIExDpw+sH4asHuwQ1X9oKqe7VZ/F3hL61hJ0vwwisC4F1iT5IwkxwLrgc2DHZKcMrB6EfCdbnkLcEGSE5OcCFzQtUmS5pmhr5Kqqv1JrqT/i34JsKmqHkpyHTBWVZuBDye5CNgPPAl8oBv7ZJLfoh86ANdV1ZPD1iRJGr1UTXnKYF7r9Xo1NjY212VI0lElydaq6h3ueL/pLUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKajCQwkqxNsiPJziRXT7H9o0m2J3kgyd1JXjWw7YUk93ePzaOoR5I0ekuHfYIkS4DPAucD48C9STZX1faBbvcBvap6JskvA/8euLTbtq+qzhq2DknSkTWKPYxzgJ1VtauqngNuBdYNdqiqP6mqZ7rVe4DTRvC6kqRZNIrAWAU8OrA+3rVN5wrgKwPrxycZS3JPkounG5RkQ9dvbO/evcNVLEmasaEPSQGZoq2m7Jj8ItADfm6geXVV7U7yGuDrSbZV1V++5AmrNgIbAXq93pTPL0k6ckaxhzEOnD6wfhqwe3KnJO8CPgZcVFXPTrRX1e7u5y7gG8DZI6hJkjRiowiMe4E1Sc5IciywHnjR1U5Jzga+QD8s9gy0n5jkuG55BXAuMHiyXJI0Twx9SKqq9ie5EtgCLAE2VdVDSa4DxqpqM3AD8LeBLycB+H5VXQS8DvhCkp/QD6/rJ11dJUmaJ1J19J0O6PV6NTY2NtdlSNJRJcnWquod7ni/6S1JamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmgz9N70BkqwFPk3/b3rfWFXXT9p+HHAz8BbgB8ClVfW9bts1wBXAC8CHq2rLKGrSzNx532PcsGUHu5/ax6nLl3HVhWdy8dmr5rqsJrNZ+9E8T9Kwhg6MJEuAzwLnA+PAvUk2V9X2gW5XAD+sqr+bZD3wO8ClSf4+sB74WeBU4GtJ/l5VvTBsXWp3532Pcc3t29j3fH/aH3tqH9fcvg1g3v8ynM3aj+Z5kkZhFIekzgF2VtWuqnoOuBVYN6nPOuCmbvk24OeTpGu/taqerapHgJ3d82kW3bBlx9/8Epyw7/kXuGHLjjmqqN1s1n40z5M0CqMIjFXAowPr413blH2qaj/wNHBy41gAkmxIMpZkbO/evSMoWxN2P7VvRu3zyWzWfjTPkzQKowiMTNFWjX1axvYbqzZWVa+qeitXrpxhiTqYU5cvm1H7fDKbtR/N8ySNwigCYxw4fWD9NGD3dH2SLAX+DvBk41gdYVddeCbLjlnyorZlxyzhqgvPnKOK2s1m7UfzPEmjMIrAuBdYk+SMJMfSP4m9eVKfzcDl3fL7gK9XVXXt65Mcl+QMYA3wf0ZQk2bg4rNX8YlL3sCq5csIsGr5Mj5xyRuOihO5s1n70TxP0iik/3t7yCdJ3gN8iv5ltZuq6uNJrgPGqmpzkuOB3wfOpr9nsb6qdnVjPwb8ErAf+EhVfeVQr9fr9WpsbGzouiVpMUmytap6hz1+FIEx2wwMSZq5YQPDb3pLkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCZDBUaSk5LcleTh7ueJU/Q5K8mfJXkoyQNJLh3Y9qUkjyS5v3ucNUw9kqQjZ9g9jKuBu6tqDXB3tz7ZM8D7q+pngbXAp5IsH9h+VVWd1T3uH7IeSdIRMmxgrANu6pZvAi6e3KGqvltVD3fLu4E9wMohX1eSNMuGDYxXVNXjAN3Plx+sc5JzgGOBvxxo/nh3qOqTSY4bsh5J0hGy9FAdknwNeOUUmz42kxdKcgrw+8DlVfWTrvka4K/oh8hG4NeB66YZvwHYALB69eqZvLQkaQQOGRhV9a7ptiV5IskpVfV4Fwh7pun308D/AH6jqu4ZeO7Hu8Vnk3wR+LWD1LGRfqjQ6/XqUHVLkkZr2ENSm4HLu+XLgT+a3CHJscAdwM1V9eVJ207pfob++Y8Hh6xHknSEDBsY1wPnJ3kYOL9bJ0kvyY1dn38CvAP4wBSXz/7nJNuAbcAK4LeHrEeSdISk6ug7utPr9WpsbGyuy5Cko0qSrVXVO9zxftNbktTEwJAkNTEwJElNDAxJUhMDQ5LUxMCQJDUxMCRJTQwMSVITA0OS1MTAkCQ1MTAkSU0MDElSEwNDktTEwJAkNTEwJElNDAxJUhMDQ5LUxMCQJDUxMCRJTYYKjCQnJbkrycPdzxOn6fdCkvu7x+aB9jOS/Hk3/g+THDtMPZKkI2fYPYyrgburag1wd7c+lX1VdVb3uGig/XeAT3bjfwhcMWQ9kqQjZNjAWAfc1C3fBFzcOjBJgH8E3HY44yVJs2vpkONfUVWPA1TV40lePk2/45OMAfuB66vqTuBk4Kmq2t/1GQdWTfdCSTYAG7rVZ5M8OGTtC8UK4K/nuoh5wrk4wLk4wLk44MxhBh8yMJJ8DXjlFJs+NoPXWV1Vu5O8Bvh6km3Aj6boV9M9QVVtBDZ2NY1VVW8Gr79gORcHOBcHOBcHOBcHdP9xP2yHDIyqetdBXvyJJKd0exenAHumeY7d3c9dSb4BnA38V2B5kqXdXsZpwO7DeA+SpFkw7DmMzcDl3fLlwB9N7pDkxCTHdcsrgHOB7VVVwJ8A7zvYeEnS/DBsYFwPnJ/kYeD8bp0kvSQ3dn1eB4wl+Tb9gLi+qrZ3234d+GiSnfTPafxe4+tuHLLuhcS5OMC5OMC5OMC5OGCouUj/P/qSJB2c3/SWJDUxMCRJTeZ9YCT5XpJt3W1Fxrq2pluSLDRJlie5LclfJPlOkrcvxrlIcubArWbuT/KjJB9ZjHMBkORXkzyU5MEktyQ5frHedifJv+rm4aEkH+naFsXnIsmmJHsGv6M23XtP339MsjPJA0ne3PIa8z4wOu/sbisycS116y1JFppPA1+tqtcCbwK+wyKci6raMXGrGeAtwDPAHSzCuUiyCvgw0Kuq1wNLgPUswtvuJHk98C+Ac+j/+3hvkjUsns/Fl4C1k9qme+/vBtZ0jw3A55peoarm9QP4HrBiUtsO4JRu+RRgx1zXOQvz8NPAI3QXKizmuZj0/i8A/vdinQv6d0d4FDiJ/veq/jtwIf1vNi/t+rwd2DLXtc7CXPwCcOPA+r8F/vVi+lwArwYeHFif8r0DXwAum6rfwR5Hwx5GAX+cZGt3exCYdEsSYLpbkiwkrwH2Al9Mcl+SG5OcwOKci0HrgVu65UU3F1X1GPAfgO8DjwNPA1uZwW13FpAHgXckOTnJy4D3AKezCD8XA6Z77xP/0ZjQ9Bk5GgLj3Kp6M/1dqF9J8o65LmiOLAXeDHyuqs4G/h8Ld9e6SXdc/iLgy3Ndy1zpjkmvA84ATgVOoP9vZbIFf/18VX2H/qG4u4CvAt+mf/86vVSmaDvkZ2TeB0YduK3IHvrHqc8BnuhuRcLBbkmywIwD41X15936bfQDZDHOxYR3A9+qqie69cU4F+8CHqmqvVX1PHA78A/obrvT9Vk0t92pqt+rqjdX1TuAJ4GHWZyfiwnTvfdx+ntfE5o+I/M6MJKckOSnJpbpH69+kIZbkiw0VfVXwKNJJu42+fPAdhbhXAy4jAOHo2BxzsX3gbcleVn3JwMmPheL8rY7E3fMTrIauIT+52Mxfi4mTPfeNwPv766Wehvw9MShq4OZ19/07u5ue0e3uhT4g6r6eJKTgf8CrKb/D+YXqurJOSpz1iQ5C7gROBbYBXyQfugvxrl4Gf1jsK+pqqe7tsX6ufh3wKX0D7/cB3yI/vHoW+mfDL8P+MWqenbOipwlSf4X/dsMPQ98tKruXiyfiyS3AOfRv537E8C1wJ1M8d67/1x8hv5VVc8AH6yqQ97Jdl4HhiRp/pjXh6QkSfOHgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmvx/hYgEHyerDbYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.xlim(50, 100)\n",
    "plt.ylim(-0.5, 1.5)\n",
    "\n",
    "X = np.array([80, 85, 74, 63, 90, 70])\n",
    "Y = np.array([1, 1, 0, 0, 1, 0])\n",
    "plt.scatter(X, Y)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然后定义 $\\pi(x)$ 函数和参数 w 和 b 的更新公式："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def pi(x, w, b):\n",
    "    return np.exp(w*x+b) / (1 + np.exp(w*x+b))\n",
    "\n",
    "# 参数 w 的更新公式\n",
    "def update_w(X, Y, w, b, eta):\n",
    "    s = 0\n",
    "    for i in range(X.size):\n",
    "        s += (Y[i] - pi(X[i], w, b))*X[i]\n",
    "    return w + eta * s * (1./Y.size)\n",
    "\n",
    "# 参数 b 的更新公式\n",
    "def update_b(X, Y, w, b, eta):\n",
    "    s = 0\n",
    "    for i in range(X.size):\n",
    "        s += (Y[i] - pi(X[i], w, b))\n",
    "    return b - eta * s * (1./Y.size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们选取参数的初始值 $(w, b) = (0, 0)$，并令学习率 $\\eta = 0.01$，然后进行第一次迭代："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w = 0.004, b = 0.0\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAEllJREFUeJzt3X+M5Hddx/Hn+3ZvrlxbaOEOrXdXruoVadDQZmk5m+DqlaQ/kp4QJFdDBFO5SKiKirFEU7GGlB8alKSAJ5RCja2lMXAhhwXPrqjZ1u4J1N6VytEGbmmhC7alSLy923v7x8yVudnZne/sze139tPnI9nsfL/z2ZnXfG729f0xszeRmUiSyrKq7gCSpMGz3CWpQJa7JBXIcpekAlnuklQgy12SCmS5S1KBLHdJKpDlLkkFGq3rjtetW5ebN2+u6+4laUXat2/fdzNzfa9xtZX75s2bmZqaquvuJWlFiohvVBnnaRlJKpDlLkkFstwlqUCWuyQVyHKXpAL1LPeIuCUinoiIBxe4PiLigxFxMCIeiIiLBh9TktSPKnvutwKXL3L9FcCW1tdO4MMnH0uqx+TkJDfddBOTk5N1R5lnWLMNa67nup7vc8/ML0bE5kWGbAc+mc3P67s3Is6KiHMy8/EBZZSWxeTkJNu2bWN2dpZGo8HevXvZunVr3bGA4c02rLk0mHPuG4BDbcvTrXXzRMTOiJiKiKmZmZkB3LU0OBMTE8zOzjI3N8fs7CwTExN1R3rWsGYb1lwaTLlHl3VdP3U7M3dl5lhmjq1f3/OvZ6VlNT4+TqPRYGRkhEajwfj4eN2RnjWs2YY1lwbz3w9MA5valjcCjw3gdqVltXXrVvbu3cvExATj4+NDdXphWLMNay5BNE+V9xjUPOf+2cx8eZfrrgKuA64ELgE+mJkX97rNsbGx9P+WkaT+RMS+zBzrNa7nnntE3A6MA+siYhr4E2A1QGZ+BNhDs9gPAj8Efn3psSVJg1Dl3TLX9Lg+gbcNLJEk6aT5F6qSVCDLXZIKZLlLUoEsd0kqkOUuSQWy3CWpQJa7JBXIcpekAlnuklQgy12SCmS5S1KBLHdJKpDlLkkFstwlqUCWuyQVyHKXpAJZ7pJUIMtdkgpkuUtSgSx3SSqQ5S5JBbLcJalAlrskFchyl6QCWe6SVCDLXZIKZLlLUoEsd0kqkOUuSQWy3CWpQJXKPSIuj4iHI+JgRFzf5fpzI+KeiPhSRDwQEVcOPqokqaqe5R4RI8DNwBXABcA1EXFBx7A/Bu7MzAuBHcCHBh1UklRdlT33i4GDmflIZs4CdwDbO8Yk8PzW5RcAjw0uoiSpX6MVxmwADrUtTwOXdIx5F/D5iPgt4HTgsoGkkyQtSZU99+iyLjuWrwFuzcyNwJXAbREx77YjYmdETEXE1MzMTP9pJUmVVCn3aWBT2/JG5p92uRa4EyAzJ4HTgHWdN5SZuzJzLDPH1q9fv7TEkqSeqpT7/cCWiDgvIho0XzDd3THmm8A2gIh4Gc1yd9dckmrSs9wz8yhwHXA38BDNd8Xsj4gbI+Lq1rDfB94SEV8BbgfenJmdp24kScukyguqZOYeYE/HuhvaLh8ALh1sNEnSUvkXqpJUIMtdkgpkuUtSgSx3SSqQ5S5JBbLcJalAlrskFchyl6QCWe6SVCDLXZIKZLlLUoEsd0kqkOUuSQWy3CWpQJa7JBXIcpekAlnuklQgy12SCmS5S1KBLHdJKpDlLkkFstwlqUCWuyQVyHKXpAJZ7pJUIMtdkgpkuUtSgSx3SSqQ5S5JBbLcJalAlrskFahSuUfE5RHxcEQcjIjrFxjzhog4EBH7I+LvBhtTktSP0V4DImIEuBl4DTAN3B8RuzPzQNuYLcA7gUsz88mIePGpCixJ6q3KnvvFwMHMfCQzZ4E7gO0dY94C3JyZTwJk5hODjSlJ6keVct8AHGpbnm6ta3c+cH5E/HtE3BsRl3e7oYjYGRFTETE1MzOztMSSpJ6qlHt0WZcdy6PAFmAcuAb4aEScNe+HMndl5lhmjq1fv77frJKkiqqU+zSwqW15I/BYlzGfycwjmfko8DDNspck1aBKud8PbImI8yKiAewAdneM+TTwiwARsY7maZpHBhlUklRdz3LPzKPAdcDdwEPAnZm5PyJujIirW8PuBr4XEQeAe4A/yMzvnarQkqTFRWbn6fPlMTY2llNTU7XctyStVBGxLzPHeo3zL1QlqUCWuyQVyHKXpAJZ7pJUIMtdkgpkuUtSgSx3SSqQ5S5JBbLcJalAlrskFchyl6QCWe6SVCDLXZIKZLlLUoEsd0kqkOUuSQWy3CWpQJa7JBXIcpekAlnuklQgy12SCmS5S1KBLHdJKpDlLkkFstwlqUCjdQfo1659u3jPv72HNaNrWDOyhsZIgzWjre9dlhcbc7LXja4aJSLqnhJJmmfFlfuGMzdw6bmXcvjoYWbnZjk81/p+9DDfP/z9Zy93Xnf88iAF0XPj0nNjUWUDVeH2O5dXr1rthkd6Dltx5X7V+Vdx1flXLelnM5Mjx47M2wB02xgstHHo9nMnLB/r/vPPHH5m0ds4PHd4wDMFjZFG3xuF5djwNEYarArPCEqn0oor95MREc8W3hmNM+qOc4LM5Oixo4tuYLptcDo3MJV+tmP9k0ee7Plzx/LYQB/v6KrRU77hWWx5sTGeblMJnlPlPswigtUjq1k9srruKF3NHZurdGRTdcM0b3mBI54fzP6g5+0cOXZkoI+1n9Nt7cuD2OhU2Qh51KMqKpV7RFwO/BUwAnw0M9+zwLjXA58CXpmZUwNLqdqNrBph7aq1rF29tu4o8xzLYxyZO7L0I5kFlhc72jm+/MzhZ3reVpIDfbxLPeqpcuRTeSO2wBhf6xkePcs9IkaAm4HXANPA/RGxOzMPdIw7E/ht4L5TEVRayKpY1XzRenQNrKk7zYkyk7mcG+hGZ95GaAlHPcfXDfqoB6q/1tN13ao+x/e5EXounXKrsud+MXAwMx8BiIg7gO3AgY5xfwa8D3jHQBNKK1hEMBqjjDZGOZ3T644zT2YuuNHp++imx8ao87qnjzzdc/zRY0cH+niPn3Jbykaj1ym4ftZtOHMDZz/v7IE+tk5Vyn0DcKhteRq4pH1ARFwIbMrMz0aE5S6tEBHx7FHPmZxZd5x55o7NceTYkUobk5Ne13G7Tx15qufGbC7nlvS4PnTlh3jrK9864Nk6UZVy73YM8+xJxIhYBXwAeHPPG4rYCewEOPfcc6sllPScNbJqhJFVI5w2elrdUbqqsvHptu6icy465dmqlPs0sKlteSPwWNvymcDLgYnWuawfB3ZHxNWdL6pm5i5gF8DY2NhgX2WSpGU2zBufKu+puh/YEhHnRUQD2AHsPn5lZj6dmesyc3NmbgbuBeYVuyRp+fQs98w8ClwH3A08BNyZmfsj4saIuPpUB5Qk9a/S+9wzcw+wp2PdDQuMHT/5WJKkk+GfuklSgSx3SSqQ5S5JBbLcJalAlrskFchyl6QCWe6SVCDLXZIKZLlLUoEsd0kqkOUuSQWy3CWpQJa7JBXIcpekAlnuklQgy12SCmS5S1KBLHdJKpDlLkkFstwlqUCWuyQVyHKXpAJZ7pJUIMtdkgpkuUtSgSx3SSqQ5S5JBbLcJalAlrskFchyl6QCWe6SVKBK5R4Rl0fEwxFxMCKu73L970XEgYh4ICL2RsRLBh9VklRVz3KPiBHgZuAK4ALgmoi4oGPYl4CxzPw54C7gfYMOKkmqrsqe+8XAwcx8JDNngTuA7e0DMvOezPxha/FeYONgY0qS+lGl3DcAh9qWp1vrFnIt8LluV0TEzoiYioipmZmZ6iklSX2pUu7RZV12HRjxRmAMeH+36zNzV2aOZebY+vXrq6eUJPVltMKYaWBT2/JG4LHOQRFxGfBHwC9k5uHBxJMkLUWVPff7gS0RcV5ENIAdwO72ARFxIfDXwNWZ+cTgY0qS+tGz3DPzKHAdcDfwEHBnZu6PiBsj4urWsPcDZwCfiogvR8TuBW5OkrQMqpyWITP3AHs61t3QdvmyAeeSJJ0E/0JVkgpkuUtSgSx3SSqQ5S5JBbLcJalAlrskFchyl6QCWe6SVCDLXZIKZLlLUoEsd0kqkOUuSQWy3CWpQJa7JBXIcpekAlnuklQgy12SCmS5S1KBLHdJKpDlLkkFstwlqUCWuyQVyHKXpAJZ7pJUIMtdkgpkuUtSgSx3SSqQ5S5JBbLcJalAlrskFahSuUfE5RHxcEQcjIjru1y/JiL+vnX9fRGxedBBJUnV9Sz3iBgBbgauAC4AromICzqGXQs8mZk/DXwAeO+gg0qSqquy534xcDAzH8nMWeAOYHvHmO3AJ1qX7wK2RUQMLubymJyc5KabbmJycrLuKANT52MqcT6llWK0wpgNwKG25WngkoXGZObRiHgaeBHw3UGEXA6Tk5Ns27aN2dlZGo0Ge/fuZevWrXXHOil1PqYS51NaSarsuXfbA88ljCEidkbEVERMzczMVMm3bCYmJpidnWVubo7Z2VkmJibqjnTS6nxMJc6ntJJUKfdpYFPb8kbgsYXGRMQo8ALgfzpvKDN3ZeZYZo6tX79+aYlPkfHxcRqNBiMjIzQaDcbHx+uOdNLqfEwlzqe0kkTmvB3sEwc0y/q/gW3At4D7gV/NzP1tY94G/Gxm/mZE7ABel5lvWOx2x8bGcmpq6mTzD9Tk5CQTExOMj48XcwqhzsdU4nxKdYuIfZk51nNcr3Jv3diVwF8CI8AtmfnuiLgRmMrM3RFxGnAbcCHNPfYdmfnIYrc5jOUuScOuarlXeUGVzNwD7OlYd0Pb5f8DfqXfkJKkU8O/UJWkAlnuklQgy12SCmS5S1KBLHdJKlClt0KekjuOmAG+scQfX8dw/tcG5uqPufo3rNnM1Z+TyfWSzOz5V6C1lfvJiIipKu/zXG7m6o+5+jes2czVn+XI5WkZSSqQ5S5JBVqp5b6r7gALMFd/zNW/Yc1mrv6c8lwr8py7JGlxK3XPXZK0iBVR7hFxVkTcFRFfjYiHImJrRLwwIr4QEV9rfT97SHK9KyK+FRFfbn1ducyZXtp231+OiO9HxNvrnq9FctU6X61svxsR+yPiwYi4PSJOi4jzWh/2/rXWh783hiTXrRHxaNt8vaKGXL/TyrQ/It7eWjcMv4/dci378ysibomIJyLiwbZ1Xecnmj4YEQcj4oGIuGhgQTJz6L9ofj7rb7QuN4CzgPcB17fWXQ+8d0hyvQt4R91z1so0AnwbeMkwzNcCuWqdL5ofEfko8LzW8p3Am1vfd7TWfQR465DkuhV4fY3z9XLgQWAtzf9V9p+ALXU/vxbJtezPL+DVwEXAg23rus4PcCXwOZqfZvcq4L5B5Rj6PfeIeD7NyfoYQGbOZuZTnPih3J8AfnlIcg2TbcDXM/Mb1DxfHdpzDYNR4HmtD6ZZCzwO/BLND3uH+uarM1fnJ6DV4WXAvZn5w8w8CvwL8Frqf34tlGvZZeYXmf9JdAvNz3bgk9l0L3BWRJwziBxDX+7ATwIzwMcj4ksR8dGIOB34scx8HKD1/cVDkgvgutYh1i11HJ622QHc3rpc93y1a88FNc5XZn4L+HPgmzRL/WlgH/BUqySg+TGSG+rOlZmfb1397tZ8fSAi1ixnLpp7x6+OiBdFxFqae56bqP/5tVAuGI7fx4XmZwNwqG3cwJ5rK6HcR2ke4nw4My8E/pfmYU3dFsr1YeCngFfQ/KX8izrCtc4RXw18qo77X0iXXLXOV+uXfTtwHvATwOnAFV2GLuvbyrrliog3Au8EfgZ4JfBC4A+XM1dmPgS8F/gC8I/AV4Cji/7QMlgk11D8Pi4iuqwbyHNtJZT7NDCdmfe1lu+iWarfOX740vr+xDDkyszvZOZcZh4D/ga4eJlzHXcF8J+Z+Z3Wct3z1TXXEMzXZcCjmTmTmUeAfwB+nubh8fFPKuv2ofC15MrMx1uH8IeBj1PD8yszP5aZF2Xmq2mefvgaQ/D86pZrCJ5fxy00P9P86AgDBvhcG/pyz8xvA4ci4qWtVduAA8Bu4E2tdW8CPjMMuTrOl72W5uFiHa7hxFMftc5XmxNyDcF8fRN4VUSsjYjgR8+ve4DXt8bUMV/dcj3UVhBB87ztsj+/IuLFre/nAq+j+e9Z+/OrW64heH4dt9D87AZ+rfWumVfRPP32+EDucTlfRV7qF81DqingAeDTwNnAi4C9NPca9gIvHJJctwH/1Vq3Gzinhlxrge8BL2hbNwzz1S3XMMzXnwJfpfmLfxuwhuZrKv8BHKR5CmnNkOT659Z8PQj8LXBGDbn+leYG8CvAtiF6fnXLtezPL5obu8eBIzT3zK9daH5onpa5Gfh6K+fYoHL4F6qSVKChPy0jSeqf5S5JBbLcJalAlrskFchyl6QCWe6SVCDLXZIKZLlLUoH+H6Mr8u8qzlgpAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "w = 0\n",
    "b = 0\n",
    "eta = 0.001\n",
    "\n",
    "w_new = update_w(X, Y, w, b, eta)\n",
    "b_new = update_b(X, Y, w, b, eta)\n",
    "print(\"w = {0}, b = {1}\".format(w_new, b_new))\n",
    "\n",
    "x = np.linspace(60, 100, 10)\n",
    "y = 1.0 / (1 + np.exp(w_new*x+b_new))\n",
    "plt.plot(X, Y, 'k.')\n",
    "plt.plot(x, y, 'g-')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "显然，一次迭代并不能得到很好的分类效果，我们把这个过程继续执行 10 万遍，直到参数收敛："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w = -1.2710404934301518, b = 102.07895720830781\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAG09JREFUeJzt3X1wHPWd5/H3V5LlZ2xsy88yNsHgB/wgn3AiwybKmb0CthbfpnIspLKEFAl1Kbg8bEKF1N3mcqS2TEJYjtQ5JN6EOGQ3cCR1tetKeS+XM3H2wDJY7rGNHzAY21jCM7b8CH7U0/f+mJEykUfWSB6pe3o+ryqVNDOtmY/bo49av+7+tbk7IiISL2VhBxARkcJTuYuIxJDKXUQkhlTuIiIxpHIXEYkhlbuISAyp3EVEYkjlLiISQyp3EZEYqgjrhSdNmuSzZ88O6+VFRIrStm3bjrt7VV/LhVbus2fPprGxMayXFxEpSmb2bj7LaVhGRCSGVO4iIjGkchcRiSGVu4hIDKncRURiqM9yN7PnzOyYme3q5XEzs++b2X4z22lmywofU0RE+iOfLfd1wB1XePxOYG7m4yHg2auPJRKOhoYGVq9eTUNDQ9hRLhPVbFHNVer6PM7d3f/VzGZfYZFVwPOevl7fFjMbb2bT3D1ZoIwiQ6KhoYGVK1fS2tpKZWUlGzdupK6uLuxYwMCyBcmATYc24e446ctpdl1WM9/bfS3T1NTEup+to6Ojg/KXy7n//vuZOXPmZc8hf+zPb/xzbplxy6C+RiFOYpoBNGXdbs7cd1m5m9lDpLfumTVrVgFeWqRwNm3aRGtrKx0dHbS2trJp06bIlHu+2dydTYc2sfqV1fz2wG+HJtxt6U/ttPPcgefgwB8eMmxoMhSZ6WOnF0W55/rfy/kr293XAmsBamtr9WtdIqW+vp7KysrureP6+vqwI3XrK1und7J+33pWv7Ka1997nSmjp/DEyif4bM1nGVkxEgCz9I9qV+H2dTufZbZs2cLtt9/enevljS9H5hdiqStEuTcD1Vm3ZwJHCvC8IkOqrq6OjRs3smnTJurr6yNVUr1la+to44VdL/CdV7/DnpY9zBk/h2f/7FkeWPoAIypGDHquW1fcyssbX47kOit11jV+dsWF0mPuv3b3m3M89mfAI8BdwIeB77v78r6es7a21jW3jMjAnG87z0+Cn/C9hu9x+MxhFk1exGO3PcY9C++hoiy0KaNkCJjZNnev7Wu5Pt8FZvYCUA9MMrNm4L8CwwDc/YfABtLFvh84D3x24LFF5EpOXTjFD7b+gGdee4aW8y3cWn0rP7jrB9w1967uIRMRyO9omfv6eNyBhwuWSEQuk/wgydNbnuaHjT/kg9YPuPOGO/nGbd/gT677k7CjSUTp7zeRCHvn5Ds8uflJ1m1fR1tnG/csvIev3/p1lk5dGnY0iTiVu0gE7Ujt4IlXn+Cl3S9RUVbBA0se4NFbH+WGCTeEHU2KhMpdJEJeOfwKq19ZzYa3NzCmcgxfrfsqX/nIV5g2dlrY0aTIqNxFQububHh7A6tfWc2rTa8yadQkvv3xb/PwLQ9z7chrw44nRUrlLhKS9s52frn7lzzx6hPsPLqTWeNm8f07vs+Dyx5k1LBRYceTIqdyFxliF9svsm77Op7c/CQHTh1g/qT5rFu1jk8t+hTDyoeFHU9iQuUuMkTev/Q+z259lqe3PM3Rc0dZPmM5T/27p7j7prspM11aQQpL5S4yyI6dO8YzW55hzdY1nLl0htuvv51f3PYLPj774zrxSAaNyl1kkBw6fYinNj/FjxM/5lL7JT4x/xM8dttj1E7v88xxkaumchcpsLOtZ3l4w8P8485/pMzK+KvFf8Wjtz7KvEnzwo4mJUTlLlJgP038lOd3PM8Xl3+Rr634GtXjqvv+JpECU7mLFNi25DamjpnKM3c+E3YUKWHaRS9SYIlUgpqpNWHHkBKnchcpoIvtF9l9bDfLpi0LO4qUOJW7SAG9cfQNOrxD5S6hU7mLFFCQDAA0LCOhU7mLFFAilWD8iPHMHj877ChS4lTuIgUUJAOWTVumM08ldCp3kQJp62hj59GdLJuq8XYJn8pdpED2Ht/LpY5L1EzTeLuET+UuUiCJZAJAR8pIJKjcRQokSAaMHjaauRPmhh1FROUuUihBKmDp1KWUl5WHHUVE5S5SCJ3eyfbUdh3fLpGhchcpgP0n93O29azG2yUyVO4iBdB1ZqrKXaJC5S5SAEEyoLK8kgVVC8KOIgKo3EUKIpFKsGjyIoaVDws7igigche5au7ePe2ASFSo3EWu0uEzhzl54aTKXSIlr3I3szvMbJ+Z7Tezx3I8PsvMfmdmCTPbaWZ3FT6qSDRpZ6pEUZ/lbmblwBrgTmABcJ+Z9dxr9F+Al9y9BrgX+EGhg4pEVSKVoNzKWTR5UdhRRLrls+W+HNjv7gfcvRV4EVjVYxkHrsl8PQ44UriIItEWJAPmV81n5LCRYUcR6ZZPuc8AmrJuN2fuy/Yt4NNm1gxsAP5TQdKJFAHtTJUoyqfcc111wHvcvg9Y5+4zgbuAn5vZZc9tZg+ZWaOZNba0tPQ/rUjEpM6mSJ5NatoBiZx8yr0ZqM66PZPLh10eBF4CcPcGYAQwqecTuftad69199qqqqqBJRaJEE3zK1GVT7lvBeaa2RwzqyS9w3R9j2UOAysBzGw+6XLXprnEXteRMkunLg05icgf67Pc3b0deAT4DbCX9FExu83scTO7O7PYV4HPm9kO4AXgAXfvOXQjEjtBKmDuhLlcM/yavhcWGUIV+Szk7htI7yjNvu+bWV/vAW4tbDSR6AuSActnLA87hshldIaqyACdunCKQ6cP6YLYEkkqd5EBSqS0M1WiS+UuMkBdO1NrpukwSIkelbvIAAXJgOprqpk06rKjfkVCp3IXGaBEKqEhGYkslbvIAJxtPcu+4/tU7hJZKneRAdiR2oHjKneJLJW7yAB0HSmjOWUkqlTuIgMQJAMmj57M9LHTw44ikpPKXWQAuqb5Ncs1aapI+FTuIv10qf0Su1t268xUiTSVu0g/7Tq2i/bOdp28JJGmchfpJ10QW4qByl2kn4JkwLjh45gzfk7YUUR6pXIX6acgFVAzrUY7UyXSVO4i/dDe2c7Oozu1M1UiT+Uu0g9vHn+Ti+0XNd4ukadyF+kH7UyVYqFyF+mHIBkwatgobpx4Y9hRRK5I5S7SD4lUgiVTllBeVh52FJErUrmL5KnTO0kkNYe7FAeVu0ie3jn5Dh+0fqByl6KgchfJU/c1UzXNrxQBlbtInhKpBMPKhrFw8sKwo4j0SeUukqcgGbBoyiIqyyvDjiLSJ5W7SB7cPT2Hu85MlSKhchfJQ9P7TZy4cELT/ErRULmL5CGRTF8zVUfKSLFQuYvkIUgGlFkZi6csDjuKSF5U7iJ5CFIB8yfNZ9SwUWFHEcmLyl0kD4lkQuPtUlTyKnczu8PM9pnZfjN7rJdl7jGzPWa228x+UdiYIuE5evYo733wno6UkaJS0dcCZlYOrAH+FGgGtprZenffk7XMXOAbwK3ufsrMJg9WYJGhlkhpZ6oUn3y23JcD+939gLu3Ai8Cq3os83lgjbufAnD3Y4WNKRKermkHlk5dGnISkfzlU+4zgKas282Z+7LdCNxoZq+a2RYzuyPXE5nZQ2bWaGaNLS0tA0ssMsQSqQQfuvZDjBsxLuwoInnLp9xzXQXYe9yuAOYC9cB9wI/NbPxl3+S+1t1r3b22qqqqv1lFQhEkAw3JSNHJp9ybgeqs2zOBIzmW+Wd3b3P3g8A+0mUvUtROXTjFgVMHVO5SdPIp963AXDObY2aVwL3A+h7L/BPwcQAzm0R6mOZAIYOKhGF7ajugaX6l+PRZ7u7eDjwC/AbYC7zk7rvN7HEzuzuz2G+AE2a2B/gd8Ki7nxis0CJDpetIGR3jLsWmz0MhAdx9A7Chx33fzPragb/OfIjERpAMmHnNTCaP1tG9Ulx0hqrIFWhnqhQrlbtIL861nuPN429qvF2KkspdpBc7j+7EcW25S1FSuYv0ouvMVJW7FCOVu0gvgmRA1agqZozteUK2SPSp3EV6EaQCaqbVYJbrJG2RaFO5i+Rwqf0Su4/t1jS/UrRU7iI57G7ZTVtnm8bbpWip3EVy0M5UKXYqd5EcgmTANcOvYc61c8KOIjIgKneRHBKpBDVTaygz/YhIcdI7V6SH9s52dqR2aEhGiprKXaSHfcf3caH9gspdiprKXaSH7ml+NaeMFDGVu0gPQTJgZMVIbpp0U9hRRAZM5S7SQ5AMWDJ1CRVleV3uQCSSVO4iWTq9s/tIGZFipnIXyXLw1EHev/S+dqZK0VO5i2TRmakSFyp3kSxBMmBY2TAWVi0MO4rIVVG5i2QJUgELJy9keMXwsKOIXBWVu0iGu5NIJjTNr8SCyl0k470P3qPlfIvG2yUWVO4iGdqZKnGichfJCJIBhrF4yuKwo4hcNZW7SEYilWDepHmMrhwddhSRq6ZyF8kIkoGGZCQ2VO4iwLFzx2h+v1nlLrGhchcBEklN8yvxonIXIWsO92kqd4mHvMrdzO4ws31mtt/MHrvCcp80Mzez2sJFFBl8QTLg+muvZ/yI8WFHESmIPsvdzMqBNcCdwALgPjNbkGO5scAXgdcKHVJksGlnqsRNPlvuy4H97n7A3VuBF4FVOZb7NvBd4GIB84kMujMXz/DOqXc03i6xkk+5zwCasm43Z+7rZmY1QLW7/7qA2USGxPbUdkBnpkq85FPuluM+737QrAx4Gvhqn09k9pCZNZpZY0tLS/4pRQZR17QD2nKXOMmn3JuB6qzbM4EjWbfHAjcDm8zsEPARYH2unaruvtbda929tqqqauCpRQooSAVMHzudKWOmhB1FpGDyKfetwFwzm2NmlcC9wPquB939jLtPcvfZ7j4b2ALc7e6Ng5JYpMASyYSGZCR2+ix3d28HHgF+A+wFXnL33Wb2uJndPdgBRQbT+bbz7D2+V3O4S+xU5LOQu28ANvS475u9LFt/9bFEhsbOozvp9E5tuUvs6AxVKWndO1N1ZqrEjMpdSloimWDiyIlUX1Pd98IiRUTlLiUtSKXPTDXLdcSvSPFSuUvJau1o5Y2jb2i8XWJJ5S4la/ex3bR1tunkJYkllbuUrK5pfrXlLnGkcpeSFSQDxlaO5UMTPhR2FJGCU7lLyQqSATXTaigz/RhI/OhdLSWpo7ODHUd3aLxdYkvlLiXprRNvcb7tvMbbJbZU7lKSus5MVblLXKncpSQFyYARFSOYN2le2FFEBoXKXUpSkApYPGUxFWV5zZ0nUnRU7lJy3D09h7um+ZUYU7lLyTl4+iBnLp3ReLvEmspdSo6m+ZVSoHKXkhMkAyrKKrh58s1hRxEZNCp3KTmJVIKFVQsZUTEi7Cgig0blLiXF3dl2ZJvG2yX2VO5SUo58cISW8y2adkBiT+UuJUXT/EqpULlLSQmSAYaxZOqSsKOIDCqVu5SUIBlw06SbGFM5JuwoIoNK5S4lJUgGGm+XkqByl5Jx/Pxxmt5v0ni7lASVu5SMRFI7U6V0qNylZHRPO6BhGSkBKncpGUEqYPb42Vw78tqwo4gMOpW7lIxEMqEhGSkZKncpCe9fep+3T76tOdylZKjcpSRsT20HtDNVSkde5W5md5jZPjPbb2aP5Xj8r81sj5ntNLONZnZd4aOKDJzmcJdS02e5m1k5sAa4E1gA3GdmC3oslgBq3X0x8Cvgu4UOKnI1EqkE08ZMY+qYqWFHERkS+Wy5Lwf2u/sBd28FXgRWZS/g7r9z9/OZm1uAmYWNKXJ1gmSgIRkpKfmU+wygKet2c+a+3jwI/EuuB8zsITNrNLPGlpaW/FOKXIXzbefZ07JH5S4lJZ9ytxz3ec4FzT4N1AJP5nrc3de6e62711ZVVeWfUuQqvHH0DTq9UycvSUmpyGOZZqA66/ZM4EjPhczsduA/Ax9z90uFiSdy9TSHu5SifLbctwJzzWyOmVUC9wLrsxcwsxrgR8Dd7n6s8DFFBi5IBkwYOYFZ42aFHUVkyPRZ7u7eDjwC/AbYC7zk7rvN7HEzuzuz2JPAGOCXZrbdzNb38nQiQ65rml+zXCOMIvGUz7AM7r4B2NDjvm9mfX17gXOJFERbRxtvHHuDL334S2FHERlSOkNVYm1Pyx5aO1o13i4lR+UusdZ1ZqrKXUqNyl1iLUgGjKkcww0Tbgg7isiQUrlLrCVSCZZOXUqZ6a0upUXveImtjs4Otqe2a5pfKUkqd4mtt0++zbm2cxpvl5KkcpfY0jS/UspU7hJbiWSC4eXDmT9pfthRRIacyl1iK0gFLJ6ymGHlw8KOIjLkVO4SS+6uOdylpKncJZYOnT7E6YunNc2vlCyVu8SSpvmVUqdyl1gKkgHlVs6iKYvCjiISCpW7xFKQDFg4eSEjKkaEHUUkFCp3iaWuOdxFSpXKXWIn+UGSo+eOarxdSprKXWJH0/yKqNwlhoJkgGEsmbIk7CgioVG5S+wEqYC5E+cydvjYsKOIhEblLrGTSCY0JCMlT+UusXLi/AnePfOu5nCXkqdyl1jpOjNV0/xKqVO5S6x0z+GuY9ylxKncJVYSqQTXjbuOiaMmhh1FJFQqd4kVTfMrkqZyl1ho7Whlc9Nm3jrxloZkRICKsAOIDETqbIqGpgYamtMfjUcaudh+EcP42OyPhR1PJHQqd4m8to42dh7d2V3kDU0NHDx9EIDK8kqWTVvGF2q/wIrqFayoXsH0sdNDTiwSPpW7RE7LuZbuEm9obmDrka2cbzsPwPSx06mbWcfDtzzMiuoV1Eyr0bS+Ijmo3CVUHZ0d7Dq2i81Nm7u3zPef3A9ARVkFNVNr+FzN56irrqNuZh2zxs3CzEJOLRJ9eZW7md0BPAOUAz929yd6PD4ceB74N8AJ4C/d/VBho0ocnLxwki3NW7rL/PX3Xuds61kAJo+ezIrqFXx+2eepm1lH7fRaRg4bGXJikeLUZ7mbWTmwBvhToBnYambr3X1P1mIPAqfc/QYzuxf4DvCXgxFYikend7KnZQ8NTQ1sbt5MQ1MD+07sA6Dcylk8ZTGfWfIZ6mbWUVddx5zxc7RVLlIg+Wy5Lwf2u/sBADN7EVgFZJf7KuBbma9/BfwPMzN39wJmHXQNDQ1s2rSJ+vp66urqwo5TEEP5bzp98TSvNb9GQ3MDm5s2s/nwZs61nwNg4siJ1FXXcf+S+1lRvYJbpt/C6MrRg5pHpJTlU+4zgKas283Ah3tbxt3bzewMMBE4XoiQ2Z5LPMdTDU8V+mm5cP4Ch949hLtjvzBmvzqbkaOiPySQ6/enk77vwoULHD58OP1vesGofqWakSNHdj+ez3P0tkzPx9s722k604TjlFkZ14++notbL2KHjcqjlaz/5XpWrFgxoH+jiPRfPuWe6+/knm2QzzKY2UPAQwCzZs3K46UvN3HkRBZULRjQ917Jm3vfxI85OLg5o6tGM++6eQV/ncFgOVa/mbF3715Ikf6fMBhXNY4FcxZ0P57Pc1xpmZ6P3zjhRuqq61g+Yzlr/m4Nf/Prv8E7nPbydn7/+9+r3EWGUD7l3gxUZ92eCRzpZZlmM6sAxgEnez6Ru68F1gLU1tYOaMhm1bxVrJq3aiDfekUNDQ2sfGIlra2tVFZWsvbLa4t+aKahoYGVq//wb/rRV340ZP+m+vp6Kisru1+7vr5+SF5XRNKsr2HxTFm/BawE3gO2Ap9y991ZyzwMLHL3/5jZofoJd7/nSs9bW1vrjY2NV5u/oDTmHp/XFokrM9vm7rV9LpfPPk8zuwv476QPhXzO3f/WzB4HGt19vZmNAH4O1JDeYr+3awdsb6JY7iIiUZdvued1nLu7bwA29Ljvm1lfXwT+Q39DiojI4NCskCIiMaRyFxGJIZW7iEgMqdxFRGJI5S4iEkN5HQo5KC9s1gK8O8Bvn8QgTG1QAMrVP8rVf1HNplz9czW5rnP3qr4WCq3cr4aZNeZznOdQU67+Ua7+i2o25eqfocilYRkRkRhSuYuIxFCxlvvasAP0Qrn6R7n6L6rZlKt/Bj1XUY65i4jIlRXrlruIiFxBUZS7mY03s1+Z2ZtmttfM6sxsgpn91szezny+NiK5vmVm75nZ9szHXUOc6aas195uZu+b2ZfDXl9XyBXq+spk+4qZ7TazXWb2gpmNMLM5ZvZaZn39TzOrjEiudWZ2MGt9LQ0h15cymXab2Zcz90Xh5zFXriF/f5nZc2Z2zMx2Zd2Xc/1Y2vfNbL+Z7TSzZQUL4u6R/wB+Bnwu83UlMB74LvBY5r7HgO9EJNe3gK+Fvc4ymcpJX4vpuiisr15yhbq+SF8i8iAwMnP7JeCBzOd7M/f9EPhCRHKtAz4Z4vq6GdgFjCI9q+z/BeaG/f66Qq4hf38BHwWWAbuy7su5foC7gH8hfTW7jwCvFSpH5Lfczewa0ivrJwDu3urup0lflPtnmcV+Bvz7iOSKkpXAO+7+LiGvrx6yc0VBBTAyc2GaUUAS+LekL/YO4a2vnrl6XgEtDPOBLe5+3t3bgd8Df0H476/ecg05d/9XLr8SXW/rZxXwvKdtAcab2bRC5Ih8uQPXAy3AT80sYWY/NrPRwBR3TwJkPk+OSC6ARzJ/Yj0Xxp+nWe4FXsh8Hfb6ypadC0JcX+7+HvA94DDpUj8DbANOZ0oC0peRnBF2Lnf/P5mH/zazvp42s+FDmYv01vFHzWyimY0iveVZTfjvr95yQTR+HntbPzOApqzlCvZeK4ZyryD9J86z7l4DnCP9Z03Yesv1LPAhYCnpH8qnwgiXGSO+G/hlGK/fmxy5Ql1fmR/2VcAcYDowGrgzx6JDelhZrlxm9mngG8A84BZgAvD1oczl7nuB7wC/Bf43sANov+I3DYEr5IrEz+MVXH5l+gK914qh3JuBZnd/LXP7V6RL9WjXny+Zz8eikMvdj7p7h7t3An8PLB/iXF3uBAJ3P5q5Hfb6ypkrAuvrduCgu7e4exvwv4AVpP887rpSWa6LwoeSy92TmT/hLwE/JYT3l7v/xN2XuftHSQ8/vE0E3l+5ckXg/dWlt/XTzB/+woACvtciX+7ungKazOymzF0rgT3AeuAzmfs+A/xzFHL1GC/7C9J/LobhPv546CPU9ZXlj3JFYH0dBj5iZqPMzPjD++t3wCczy4SxvnLl2ptVEEZ63HbI319mNjnzeRbwCdL/n6G/v3LlisD7q0tv62c9cH/mqJmPkB5+SxbkFYdyL/JAP0j/SdUI7AT+CbgWmAhsJL3VsBGYEJFcPwfeyNy3HpgWQq5RwAlgXNZ9UVhfuXJFYX39N+BN0j/4PweGk96n8jqwn/QQ0vCI5Ho5s752Af8AjAkh1/8j/QtwB7AyQu+vXLmG/P1F+pddEmgjvWX+YG/rh/SwzBrgnUzO2kLl0BmqIiIxFPlhGRER6T+Vu4hIDKncRURiSOUuIhJDKncRkRhSuYuIxJDKXUQkhlTuIiIx9P8BdHp3wLMJYAIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "for i in range(100000):\n",
    "    w_new = update_w(X, Y, w, b, eta)\n",
    "    b_new = update_b(X, Y, w, b, eta)\n",
    "    w = w_new\n",
    "    b = b_new\n",
    "    \n",
    "print(\"w = {0}, b = {1}\".format(w_new, b_new))\n",
    "\n",
    "x = np.linspace(60, 100, 10)\n",
    "y = 1.0 / (1 + np.exp(w_new*x+b_new))\n",
    "plt.plot(X, Y, 'k.')\n",
    "plt.plot(x, y, 'g-')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "根据上面的参数 w 和 b，令：\n",
    "\n",
    "$$\n",
    "wx+b = 0\n",
    "$$\n",
    "\n",
    "所以有 $x = 80.31$，这就是该问题的分类决策边界，可以看出一元分类问题的决策边界就是一个点，接下来我们看看多元分类问题。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}