支持向量机/06. Scikit Learn 中的 SVM.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这一节我们使用 sklearn 中提供的 SVM 方法来解决一个简单的二分类问题。sklearn 自带了一个鸢尾花的数据集，这个数据集是 Fisher 在 1936 年收集整理的，是一类多重变量分析的数据集，数据集共包含 150 条数据记录，分为 3 类，每类 50 个数据，每条数据包含 4 个属性。可通过花萼长度，花萼宽度，花瓣长度，花瓣宽度 4 个属性预测鸢尾花卉属于（Setosa，Versicolour，Virginica）三个种类中的哪一类。\n",
    "\n",
    "由于我们想使用 SVM 解决二分类问题，所以这里我们只取鸢尾花的两种类型，每种类型只取两个特征："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAFz1JREFUeJzt3W+MXFd5x/Hv45kUMDREIqsSxX9WFYi2oBDiVQhNhUJsqhAs8wKqplpKg1q5eEMJLRV/GilVLSFUIVGgYKMlqEpqt4QGaANKaZsAhb4g1TokgWBaBWonTmizSUrS1G0q209f3Lt4d3Z25p6ZOTPnnPl9pKuduXNy9zn33n1yfe5zz5i7IyIiZdk06QBERGT0lNxFRAqk5C4iUiAldxGRAim5i4gUSMldRKRASu4iIgVSchcRKZCSu4hIgdpNG5pZC1gCHnH33R2fXQt8GHikXvUJd7+p1/bOP/98n52dDQpWRGTaHTly5HF3n+nXrnFyB64HjgLnbvD5re7+zqYbm52dZWlpKeDXi4iImR1v0q7RsIyZbQHeCPS8GhcRkTQ0HXP/KPBe4EyPNm82s/vN7DYz29qtgZntNbMlM1taXl4OjVVERBrqm9zNbDfwmLsf6dHsS8Csu18E3Anc3K2Ruy+6+5y7z83M9B0yEhGRATW5cr8c2GNmx4DPAlea2aHVDdz9CXd/tn77aWDHSKMUEZEgfZO7u3/A3be4+yxwDfBVd3/r6jZmdsGqt3uobryKiMiEhFTLrGFm+4Eld78deJeZ7QFOAU8C144mPBERGUTQQ0zu/vWVGnd3v7FO7CtX9y9391e6++vc/fsxghWZiMOHYXYWNm2qfh4+POmIRPoa+MpdZCocPgx798LJk9X748er9wDz85OLS6QPTT8g0ssNN5xN7CtOnqzWiyRMyV2kl4ceClsvkggld5Fetm0LWy+SCCV3kV4++EHYvHntus2bq/UiCVNyF+llfh4WF2H7djCrfi4u6maqJE/VMiL9zM8rmUt2dOUuIlIgJXcRkQIpuYuIFEjJXUSkQEruIiIFUnIXESmQkruISIGU3EVECqTkLiJSICV3KYe+VEPkJzT9gJRBX6ohsoau3KUM+lINkTWU3KUM+lINkTWU3KUM+lINkTWU3KUM+lINkTWU3KUM+lINkTVULSPl0JdqiPyErtxleKovF0mOrtxlOKovF0mSrtxlOKovF0mSkrsMR/XlIklScpfhqL5cJElK7jIc1ZeLJEnJXYaj+nKRJDWuljGzFrAEPOLuuzs+ew5wC7ADeAL4VXc/NsI4JWWqLxdJTsiV+/XA0Q0++03gP939JcCfAH88bGAiWVLNvySiUXI3sy3AG4GbNmjyJuDm+vVtwE4zs+HDE8nISs3/8ePgfrbmXwleJqDplftHgfcCZzb4/ELgYQB3PwU8Bbxo6OhEcqKaf0lI3+RuZruBx9z9SK9mXdZ5l23tNbMlM1taXl4OCFMkA6r5l4Q0uXK/HNhjZseAzwJXmtmhjjYngK0AZtYGXgg82bkhd1909zl3n5uZmRkqcJHkqOZfEtI3ubv7B9x9i7vPAtcAX3X3t3Y0ux34jfr1W+o2667cRYqmmn9JyMB17ma238z21G8/A7zIzB4Efg94/yiCE8mKav4lITapC+y5uTlfWlqayO8WEcmVmR1x97l+7fSEqqRrYQHa7eoquN2u3otII5rPXdK0sAAHD559f/r02fcHDkwmJpGM6Mpd0rS4GLZeRNZQcpc0nT4dtl5E1lBylzS1WmHrRWQNJXdJ08r3sDZdLyJr6IaqpGnlpuniYjUU02pViV03U0UaUXKXdB04oGQuMiANy0h3u3ZV9eUry65dk45ocjRHu2RIyV3W27UL7rpr7bq77prOBK852iVTmn5A1uv1PSvTNh/c7GyV0Dtt3w7Hjo07GhFNPyAyEpqjXTKl5C7Si+Zol0wpuct6O3eGrS+Z5miXTCm5y3p33rk+ke/cWa2fNpqjXTKlG6oiIhnRDVUZTqza7pDtqr5cZGB6QlXWW6ntPnmyer9S2w3DDUeEbDdWDCJTQsMysl6s2u6Q7aq+XKQrDcvI4GLVdodsV/XlIkNRcpf1YtV2h2xX9eUiQ1Fyl/Vi1XaHbFf15SJDUXKX9WLVdodsV/XlIkPRDVURkYzohmpsKdRgh8aQQswiMhaqcx9ECjXYoTGkELOIjI2GZQaRQg12aAwpxCwiQ9OwTEwp1GCHxpBCzCIyNkrug0ihBjs0hhRiFpGxUXIfRAo12KExpBCziIyNkvsgUqjBDo0hhZhFZGz63lA1s+cC3wCeQ1Vdc5u7/2FHm2uBDwOP1Ks+4e439dpu1jdURUQmZJQ3VJ8FrnT3VwIXA1eZ2WVd2t3q7hfXS8/ELhOysADtdnXl3m5X70fRNpX6+VTiEElA3zp3ry7tn6nfnlMvk6mflMEtLMDBg2ffnz599v2BA4O3TaV+PpU4RBLRqM7dzFrAEeAlwCfd/X0dn18LfAhYBv4V+F13f7jXNjUsM2btdpWkO7VacOrU4G1TqZ9PJQ6RyEZa5+7up939YmALcKmZvaKjyZeAWXe/CLgTuHmDoPaa2ZKZLS0vLzf51TIq3ZL1RutD2qZSP59KHCKJCKqWcfcfA18HrupY/4S7P1u//TSwY4P/ftHd59x9bmZmZoBwZWCtVvP1IW1TqZ9PJQ6RRPRN7mY2Y2bn1a+fB+wCvt/R5oJVb/cAR0cZpIzAyvhzk/UhbVOpn08lDpFUuHvPBbgI+DZwP/Bd4MZ6/X5gT/36Q8ADwH3A14Cf67fdHTt2uIzZvn3urZY7VD/37RtN20OH3Ldvdzerfh46NOrIm0klDpGIgCXvk1/dXROHiYjkRBOHxRarpjqkvjzmtkP6l+O+yIxK+CVYk8v7GEvWwzKHDrlv3lwNWawsmzcPPwywb9/aba4svYZEYmw7pH857ovMxNrFkic0LBNRrJrqkPrymNsO6V+O+yIzKuGX1ZoOyyi5D2LTpuoCqpMZnDkz+HbNNv5s2OMUsu2Q/uW4LzITaxdLnjTmHlOsmuqQ+vKY2w7pX477IjMq4ZdBKLkPIlZNdUh9ecxth/Qvx32RGZXwy0CaDMzHWLK+oeoer6Y6pL485rZD+pfjvsiMSvhlBbqhKiJSHo25y3op1K5L1nRa5KPvfO5SiJD5zjU3unSh0yIvGpaZFinUrkvWdFqkQcMyslbIfOeaG1260GmRFyX3aZFC7bpkTadFXpTcp0UKteuSNZ0WeVFynxbz87C4WA2QmlU/Fxe73wkLaStTQ6dFXnRDVUQkI7qhuiJWYW7IdlOZl1xFykkp/XCU3r8QE9kXTR5jjbGMZfqBWBNhh2w3lXnJNSl4Uko/HKX3L8So9wWafoB4hbkh201lXnIVKSel9MNRev9CjHpfaD53iDcRdsh2U5mXXJOCJ6X0w1F6/0KMel9ozB3iFeaGbDeVeclVpJyU0g9H6f0LMal9UXZyj1WYG7LdVOYlV5FyUko/HKX3L8TE9kWTgfkYy9jmc481EXbIdlOZl1yTgiel9MNRev9CjHJfoBuqIiLl0Zh7bKqfF8lCrD+T5Ov4m1zex1iy/po91c+LZCHWn8kk6/jRsExEqp8XyUKsP5NJ1vFrWCamWBNbh2y32xnba73IFIr1Z5LD3PZK7oNQ/bxIFmL9meRQx6/kPgjVz4tkIdafSRZ1/E0G5mMsWd9QdVf9vEgmYv2ZTKqOH91QFREpz8huqJrZc83sn83sPjN7wMz+qEub55jZrWb2oJndbWazg4XdQGhxafLFqB1CinIL3xcxw425m5uK2b/MDnWQwk/70el3aQ8Y8IL69TnA3cBlHW0WgE/Vr68Bbu233YGGZUKLS3ObVDqkKLfwfREz3Ji7uamY/cvsUAcp/LRvhIbDMkHj5MBm4B7g1R3r/w54Tf26DTxOPZ3wRstAyX379u5/ldu3j6b9pK0MDHYurdb6toXvi5jhxtzNTcXsX2aHOkjhp30jTZN7ozF3M2sBR4CXAJ909/d1fP5d4Cp3P1G//0H9P4DHO9rtBfYCbNu2bcfxbk8B9BI6MXJuk0qHzP1e+L6IGW7M3dxUzP5ldqiDFH7aNzLSh5jc/bS7XwxsAS41s1d0/r5u/1mX7Sy6+5y7z83MzDT51WuFFpfmUIy6WkhRbuH7Ima4MXdzUzH7l9mhDlL4aT9SQXXu7v5j4OvAVR0fnQC2AphZG3gh8OQI4lsrtLg0i2LUVUKKcgvfFzHDjbmbm4rZv8wOdZDCT/vR6jduA8wA59Wvnwd8E9jd0eY61t5Q/Vy/7Q5c5x5aXJrbpNIhRbmF74uY4cbczU3F7F9mhzpI4ad9X4xqzN3MLgJuBlpUV/qfc/f9Zra//iW3m9lzgT8HXkV1xX6Nu/+w13ZV5y4iEq7pmHu7XwN3v58qaXeuv3HV6/8FfiU0SBERiaP8uWWm9gkG6SXktEjhFIr54E5uD2mlcDyy0GTsJsYylrllSnyCQYYWclqkcArFfHAnt4e0Ujgek4bmlmGyM+pLskJOixROodAYUuhfbtvNSdMx97KTe4lPMMjQQk6LFE6hmA/u5PaQVgrHY9L0TUww3U8wyIZCTosUTqGYD+7k9pBWCscjF2Un96l+gkE2EnJapHAKxXxwJ7eHtFI4HtloMjAfYxnbl3WU9gSDjETIaZHCKRTzwZ3cHtJK4XhMErqhKiJSHo25i4xIyBd7pCK3mFOpXU8ljpFocnkfY8n+O1RlKoR8sUcqcos5ldr1VOLoBw3LiAyv3YbTp9evb7Xg1Knxx9NEbjGnUrueShz9aFhGZAS6Jcle61OQW8wPPRS2vvQ4RkXJXaSHkC/2SEVuMadSu55KHKOi5C7SQ8gXe6Qit5hTqV1PJY6RaTIwH2PRDVXJRcgXe6Qit5hTqV1PJY5e0A1VEZHy6IaqjE2OtcGxYo5VX57jPpYJa3J5H2PRsEwZcqkNXi1WzLHqy3PcxxIPGpaRccilNni1WDHHqi/PcR9LPBqWkbHIsTY4Vsyx6stz3McyeUruMpQca4NjxRyrvjzHfSyTp+QuQ8mxNjhWzLHqy3Pcx5KAJgPzMRbdUC1HDrXBnWLFHKu+PMd9LHGgG6oiIuXRDVWZOrFqwUO2q3p0SUV70gGIjMLhw9XY9smT1fvjx8+Odc/Pj2e7sWIQGYSGZaQIsWrBQ7arenQZBw3LyFSJVQsesl3Vo0tKlNylCLFqwUO2q3p0SYmSuxQhVi14yHZVjy4pUXKXIszPw+JiNb5tVv1cXBz+RmbIdmPFIDKIvjdUzWwrcAvwYuAMsOjuH+tocwXwN8C/1au+4O77e21XN1RFRMKN8obqKeA97v7zwGXAdWb2C13afdPdL66Xnold0pdjvbbq0ePTfstIk8dYVy9UV+iv71h3BfDlkO1o+oF05Th/eEjMOfYvBdpvaSDG9ANmNgt8A3iFuz+9av0VwOeBE8CjwO+7+wO9tqVhmXTlWK+tevT4tN/S0HRYpnFyN7MXAP8IfNDdv9Dx2bnAGXd/xsyuBj7m7i/tso29wF6Abdu27Tje7UyRidu0qbou62QGZ86MP54mQmLOsX8p0H5Lw0gfYjKzc6iuzA93JnYAd3/a3Z+pX98BnGNm53dpt+juc+4+NzMz0+RXywTkWK+tevT4tN/y0je5m5kBnwGOuvtHNmjz4rodZnZpvd0nRhmojE+O9dqqR49P+y0z/QblgV8CHLgfuLdergbeAbyjbvNO4AHgPuBbwC/2265uqKYtx/nDQ2LOsX8p0H6bPDSfu4hIeTRx2BRQzfFaCwvQblc3+Nrt6r3ItNJ87pnS3OFrLSzAwYNn358+ffb9gQOTiUlkkjQskynVHK/VblcJvVOrBadOjT8ekVg0LFM4zR2+VrfE3mu9SOmU3DOlmuO1Wq2w9SKlU3LPlGqO11q539B0vUjplNwzpbnD1zpwAPbtO3ul3mpV73UzVaaVbqiKiGREN1QHUXjheOHdK75/KdA+zkiTx1hjLMlNP1D4ZNWFd6/4/qVA+zgNaPqBQIUXjhfeveL7lwLt4zSMfD73UUsuuRc+WXXh3Su+fynQPk6DxtxDFV44Xnj3iu9fCrSP86LkvqLwwvHCu1d8/1KgfZwXJfcVhReOF9694vuXAu3jvGjMXUQkIxpzFylIzPpy1a6XSfO5iyQu5tz9+l6AcmlYRiRxMevLVbueHw3LiBQi5tz9+l6Acim5iyQuZn25atfLpeQukriY9eWqXS+XkrtI4mLWl6t2vVy6oSoikhHdUBURmWJK7iIiBVJyFxEpkJK7iEiBlNxFRAqk5C4iUiAldxGRAim5i4gUqG9yN7OtZvY1MztqZg+Y2fVd2piZfdzMHjSz+83skjjhyjA0b7fI9Ggyn/sp4D3ufo+Z/TRwxMz+wd2/t6rNG4CX1surgYP1T0mE5u0WmS59r9zd/Ufufk/9+r+Ao8CFHc3eBNzilW8B55nZBSOPVgZ2ww1nE/uKkyer9SJSnqAxdzObBV4F3N3x0YXAw6ven2D9/wAws71mtmRmS8vLy2GRylA0b7fIdGmc3M3sBcDngXe7+9OdH3f5T9bNSObui+4+5+5zMzMzYZHKUDRvt8h0aZTczewcqsR+2N2/0KXJCWDrqvdbgEeHD09GRfN2i0yXJtUyBnwGOOruH9mg2e3A2+qqmcuAp9z9RyOMU4akebtFpkuTapnLgV8HvmNm99br/gDYBuDunwLuAK4GHgROAm8ffagyrPl5JXORadE3ubv7P9F9TH11GweuG1VQIiIyHD2hKiJSICV3EZECKbmLiBRIyV1EpEBK7iIiBVJyFxEpkJK7iEiBrCpRn8AvNlsGjk/kl/d3PvD4pIOISP3LV8l9A/Wvie3u3ndyrokl95SZ2ZK7z006jljUv3yV3DdQ/0ZJwzIiIgVSchcRKZCSe3eLkw4gMvUvXyX3DdS/kdGYu4hIgXTlLiJSoKlO7mbWMrNvm9mXu3x2rZktm9m99fJbk4hxGGZ2zMy+U8e/1OVzM7OPm9mDZna/mV0yiTgH0aBvV5jZU6uO342TiHNQZnaemd1mZt83s6Nm9pqOz7M9dtCof9kePzN72aq47zWzp83s3R1toh+/Jl/WUbLrgaPAuRt8fqu7v3OM8cTwOnffqK72DcBL6+XVwMH6Zy569Q3gm+6+e2zRjNbHgK+4+1vM7KeAji9JzP7Y9esfZHr83P1fgIuhuoAEHgG+2NEs+vGb2it3M9sCvBG4adKxTNCbgFu88i3gPDO7YNJBTTszOxd4LdXXW+Lu/+fuP+5olu2xa9i/UuwEfuDunQ9sRj9+U5vcgY8C7wXO9Gjz5vqfTLeZ2dYe7VLlwN+b2REz29vl8wuBh1e9P1Gvy0G/vgG8xszuM7O/NbOXjzO4If0ssAz8WT1seJOZPb+jTc7Hrkn/IN/jt9o1wF92WR/9+E1lcjez3cBj7n6kR7MvAbPufhFwJ3DzWIIbrcvd/RKqfwJeZ2av7fi829cn5lI+1a9v91A9pv1K4E+Bvx53gENoA5cAB939VcB/A+/vaJPzsWvSv5yPHwD1cNMe4K+6fdxl3UiP31Qmd6ov/d5jZseAzwJXmtmh1Q3c/Ql3f7Z++2lgx3hDHJ67P1r/fIxqzO/SjiYngNX/ItkCPDqe6IbTr2/u/rS7P1O/vgM4x8zOH3uggzkBnHD3u+v3t1Elw842WR47GvQv8+O34g3APe7+H10+i378pjK5u/sH3H2Lu89S/bPpq+7+1tVtOsa/9lDdeM2GmT3fzH565TXwy8B3O5rdDrytvnN/GfCUu/9ozKEGa9I3M3uxmVn9+lKqc/2Jccc6CHf/d+BhM3tZvWon8L2OZlkeO2jWv5yP3yq/RvchGRjD8Zv2apk1zGw/sOTutwPvMrM9wCngSeDaScY2gJ8Bvlj/fbSBv3D3r5jZOwDc/VPAHcDVwIPASeDtE4o1VJO+vQXYZ2angP8BrvG8ntj7HeBw/U/7HwJvL+TYrejXv6yPn5ltBl4P/PaqdWM9fnpCVUSkQFM5LCMiUjoldxGRAim5i4gUSMldRKRASu4iIgVSchcRKZCSu4hIgZTcRUQK9P/zUmS5eqGLXgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn import datasets\n",
    "\n",
    "iris = datasets.load_iris();\n",
    "\n",
    "X = iris.data\n",
    "y = iris.target\n",
    "\n",
    "X = X[y < 2, :2]\n",
    "y = y[y < 2]\n",
    "\n",
    "plt.scatter(X[y == 0, 0], X[y == 0, 1], color='red')\n",
    "plt.scatter(X[y == 1, 0], X[y == 1, 1], color='blue')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "图中可以看出数据非常明显的分为两类。通过 sklearn 中提供的 `LinearSVC` 类，可以非常方便的训练出模型，这里的参数 C 取一个很大的数字，这样得到的接近于硬间隔支持向量机。我们输入两个测试数据：`(6,2)` 和 `(4,3)` 得到的结果正好是 1 和 0。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1 0]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/aneasystone/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:922: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  \"the number of iterations.\", ConvergenceWarning)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.svm import LinearSVC\n",
    "\n",
    "svc = LinearSVC(C=1e9)\n",
    "svc.fit(X, y)\n",
    "results = svc.predict([[6,2], [4, 3]])\n",
    "print(results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "不过这里我们看到了警告信息：`Liblinear failed to converge, increase the number of iterations`，这句话的意思是算法没有达到收敛，需要增加迭代次数，sklearn 默认的迭代次数是 1000，我们将其加到 40000 才收敛："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1 0]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.svm import LinearSVC\n",
    "\n",
    "svc = LinearSVC(C=1e9, max_iter=40000)\n",
    "svc.fit(X, y)\n",
    "results = svc.predict([[6,2], [4, 3]])\n",
    "print(results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "迭代次数越多很显然算法性能就越低，解决这个问题的令一个办法是数据归一化，使用 `preprocessing` 模块提供的 `scale` 方法将数据移到靠近中心的位置，让其符合正态分布，可以看出不用增加迭代次数算法也可以很快收敛："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1 0]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAE1BJREFUeJzt3WGI5Vd5x/Hfs3di241CXnRoSpLZtbU1SYMgOxSD0IK74DaEFgtCyyBChYFdWyJY0LBv2hfSF4L0hW7sgKEtGWwFDWm1JWaDEgRjOysxJGwiqThxUXBFioZ5UXb36Ys7k7szc2fmf+495/6fc+73A5fN/c/dM8//zJ1n/znn+T/X3F0AgHYc6zsAAEBeJHYAaAyJHQAaQ2IHgMaQ2AGgMSR2AGjM1IndzH7VzP7LzL5nZi+b2d/mCAwAMBmbto7dzEzS7e7+hpndJulbkh5x9+dzBAgASLMw7QA+/Jfhje2nt20/uOsJAHoydWKXJDMbSLos6R2SPufu3xnzmlVJq5J0++23n7r33ntzfGsAmBuXL1/+mbsvHvW6qZdidg1mdoekJyX9lbu/dNDrlpeXfWNjI9v3BYB5YGaX3X35qNdlrYpx9/+V9E1JZ3OOCwDoLkdVzOL2lbrM7NcknZH0yrTjAgAmk2ON/Tcl/dP2OvsxSV9y969mGBcAMIEcVTEvSnp3hlgAABlw5ykANIbEDgCNIbEDB1lfl06elI4dG/65vt53REAnWW5QApqzvi6trkpbW8Pnm5vD55K0stJfXEAHXLED41y4MErqO7a2hseB4EjswDivv552HAiExA6Ms7SUdhwIhMQOjPOpT0nHj+8+dvz48DgQHIkdGGdlRVpbk06ckMyGf66tsXGKKlAVAxxkZYVEjipxxQ4AjSGxA0BjSOwA0BgSOwA0hsQOAI0hsQNAY0jsANAYEjsANIbEjvrRNx3YhTtPUTf6pgP7cMWOutE3HdiHxI660Tcd2IfEjrrRNx3Yh8SOutE3HdiHxI660Tcd2IeqGNSPvunALlyxY3LUjwMhccWOyVA/DoTFFTsmQ/04EBaJHZOhfhwIa+rEbmb3mNk3zOyKmb1sZo/kCAzBUT8OhJXjiv26pI+7+32S3iPpo2Z2f4ZxERn140BYUyd2d/+Ju393+79/KemKpLumHRfBUT8OhGXunm8ws5OSnpP0gLv/Ys/XViWtStLS0tKpzc3NbN8XAOaBmV129+WjXpdt89TM3irpy5I+tjepS5K7r7n7srsvLy4u5vq2QBzU9SOILHXsZnabhkl93d2/kmNMoCrU9SOQHFUxJukLkq64+2emDwmoEHX9CCTHUsx7JX1I0vvM7IXtx0MZxgXqQV0/Apl6KcbdvyXJMsQC1Gtpabj8Mu44MGPceQrkQF0/AiGxAzlQ149A6O4I5EJfeATBFTtiOn9eWlgYXv0uLAyfA+iEK3bEc/689Nhjo+c3boyeX7zYT0xARbhiRzxra2nHAexCYkc8N26kHQewC4kd8QwGaccB7EJiRzw7PVa6HgewC5uniGdng3Rtbbj8MhgMkzobp0AnJHbEdPEiiRyYEEsx2O3MmWHt+M7jzJm+I+oP/dVRKRI7Rs6ckZ59dvexZ5+dz+S+0199c1NyH/VXJ7mjAlk/Gq+r5eVl39jYmPn3xRHskCadPbxPenXy5PhujSdOSD/84ayjAST18NF4QFPor46KkdiBcQ7qo05/dVSAxI6R06fTjreM/uqoGIkdI5cu7U/ip08Pj88b+qujYmyeAkAl2DwFgDlFYsduJW/KSRmbm4OAidFSACM7N+VsbQ2f79yUI02/tpwydsk4gDnAGjtGSt6UkzI2NwcBY7HGjnQlb8pJGZubg4CpkNgxUvKmnJSxuTkImAqJHSMlb8pJGZubg4CpkNgxUvKmnJSxuTkImAqbpwBQCTZPS4lSX01NOIADZKljN7PHJT0s6afu/kCOMUOKUl9NTTiAQ2RZijGzP5D0hqR/7pLYq12KiVJfTU04MJdmuhTj7s9J+nmOsUKLUl9NTTiAQ7DGniJKfTU14QAOMbPEbmarZrZhZhvXrl2b1bfNK0p9NTXhAA4xs8Tu7mvuvuzuy4uLi7P6tnlFqa+mJhzAIbLVsZvZSUlfbXrzFAB6NNPNUzP7oqRvS3qnmV01s4/kGBeZnD8vLSwMr9gXFobPc7w2Sn18lDiAKNx95o9Tp045ZuTcOXdp/+Pcuele+8QT7seP737d8ePD47MUJQ5gBiRteIccS0uB1i0sSDdu7D8+GEjXr0/+2ij18VHiAGaAlgIYGpeoDzqe8too9fFR4gACIbG3bjDofjzltVHq46PEAQRCYm/dTl+YLsdTXhulPj5KHEAkXRbicz/YPJ2xc+fcB4PhxuJgMH4zdJLXPvGE+4kT7mbDP/vasIwSB1CY2DwFgLaweVpKyZrplBryUuNG6fNeai4qRJk+knW5rM/9qHYppmTNdEoNealxU86vxrmoEGX6uJVYiimgZM10Sg15qXGj9HkvNRcVokwft+q6FENiT3Hs2PCiaS8z6ebN6cY2O/hr0/yMUsZNOb8a56JCJacZ9WGNvYSSNdMpNeSlxo3S573UXFSIMn1MgsSeomTNdEoNealxo/R5LzUXFaJMHxPpshCf+1Ht5ql72ZrplBryUuOmnF+Nc1EhyvSxQ2yeAkBbWGPHSJTadFSNt0Y9FvoOAIWtrw/Xpre2hs83N0dr1Xs/Hi/ltZgrvDXqwlJM66LUpqNqvDViYCkGQyn9yultjgPw1qgLib11UWrTUTXeGnUhsbcuSm06qsZboy4k9tatrEhra8PFULPhn2tr43e8Ul6LucJboy5sngJAJdg8LVV0mzpuhL7iFCCH0/qPpPXzS9HLXHS5PTX3o3hLgVJNrFPHjdBXnIbe4bT+I2n9/FLkngvNdUuBUkW3qeNG6CtOAXI4rf9IWj+/FLnnYr77sZdqYp06boS+4jT0Dqf1H0nr55ci91zM9xp7qaLb1HEj9BWnADmc1n8krZ9fir7mos3EXqroNnXcCH3FKUAOp/UfSevnl6K3ueiyEJ/7MZN+7KWaWKeOG6GvOA29w2n9R9L6+aXIORea681TAGjQTNfYzeysmb1qZq+Z2SdzjBlWyaLUlLEj1McDgZX8FQlfp9/lsv6wh6SBpP+R9FuS3iLpe5LuP+zvVPvReCULdFPGjlAfDwRW8lekzzp9zWopxswelPQ37v7+7eePbv+D8XcH/Z1ql2JKFuimjB2hPh4IrOSvSJ91+rNcirlL0o9ueX51+9jegFbNbMPMNq5du5bh2/agZFPqlLHHvWMPOw7MmZK/IjX0ps+R2MfdhbPvfwPcfc3dl919eXFxMcO37UHJotSUsSPUxwOBlfwVqaFOP0divyrpnlue3y3pxxnGjadkUWrK2BHq44HASv6KVFGn32Uh/rCHhh+I/QNJb9do8/T3Dvs71W6eupct0E0ZO0J9PBBYyV+Rvur0Ncs6djN7SNLfa1gh87i7H/pvV7WbpwDQo5nWsbv7f7j777r7bx+V1KeSUjwavtB0jJTC28bnolTIpaY4VZSPC6hN42/7fLpc1ud+TLQUk1I8WmND6JTC28bnolTIpaY4VZSPC6hN42/7TtRxKaaexH7ixPjfyhMnpnttFDuLgXsfg8H+1zY+F6VCLjXFqUqNXeGPOknjb/tOuib2enrFpDQ2rrEhdErv9sbnolTIpaY4VZSPC6hN42/7Ttrrx55SPFpDoeleKYW3jc9FqZBLTXGqKB8XUJvG3/ZZ1ZPYU4pHqyg03SOl8LbxuSgVcqkpThXl4wJq0/jbPq8u6zW5HxPXsacUj9bYEDql8LbxuSgVcqkpThXl4wJq0/jb/khqbo0dAOZce2vsqea6iBUHqbEOulTM1Ok3rMtlfe5H8ZYCrRaxYio11kGXipk6/Tpprpdi+myYjLBS3hZR3kKlYo7y0QIRxq1J16WYNhN7q0WsmEqNddClYqZOv07zvcY+70WsGKvGOuhSMVOn37Y2E/vcF7FinBrroEvFTJ1+47osxOd+zKQfe4tFrJhajXXQpWKmTr8+muvNUwBo0HyvsQMZpPRuj6LGmCPUpkeIIasul/W5H1V/NB7mQkrv9ihqjDlCbXqEGLoSSzHA5BYWpBs39h8fDKTr12cfTxc1xhyhNj1CDF2xFANMYVyCPOx4BDXG/PrracdbjSE3EjswRkrv9ihqjDlCbXqEGHIjsQNjpPRuj6LGmCPUpkeIIbsuC/G5H2yeogYpvdujqDHmCLXpEWLoQmyeAkBb2DwFgDlFYsfEarypo2TMpW4OqnGe0bMu6zW5H6yx16+mmzp2lIy51M1BNc4zyhFr7Cippps6dpSMudTNQTXOM8phjR1F1XhTR8mYS90cVOM8o38kdkykxps6SsZc6uagGucZ/SOxYyI13tRRMuZSNwfVOM8IoMtC/EEPSR+U9LKkm5KWu/49Nk/bUMtNHbcqGXOpm4NqnGeUoVlsnprZfdtJ/R8k/bW7d9oRZfMUANLNZPPU3a+4+6vTjAFMq1Sdd+q41JsjioVZfSMzW5W0KklL7Pwgk/X14Tr21tbw+ebmaF17ZWV245aKA5jEkUsxZnZJ0p1jvnTB3Z/afs03xVIMelCqzjt1XOrNMQtdl2KOvGJ39zN5QgLyK1XnnTou9eaIhHJHVK1UnXfquNSbI5KpEruZfcDMrkp6UNLXzOzpPGEB3ZSq804dl3pzRDJtVcyT7n63u/+Ku/+Gu78/V2BAFysr0tracC3bbPjn2tr0G5ap45aKA5gETcAAoBI0AcNEaqzFTom5xvOLgrmrSJfbU3M/aCkQU429v1NirvH8omDuYhD92JGqxlrslJhrPL8omLsYui7FkNjxpmPHhtdie5lJN2/OPp4uUmKu8fyiYO5iYI0dyWqsxU6Jucbzi4K5qwuJHW+qsRY7JeYazy8K5q4yXRbicz/YPI2rxt7fKTHXeH5RMHf9E5unANAW1tgbR03xyPnz0sLCcCNvYWH4HJhnM+vHjnzo/T1y/rz02GOj5zdujJ5fvNhPTEDfWIqpEDXFIwsLw2S+12AgXb8++3iAkliKaRi9v0fGJfXDjgPzgMReIWqKRwaDtOPAPCCxV4ia4pGdvYWux4F5QGKvEL2/Ry5elM6dG12hDwbD52ycYp6xeQoAlWDzNEXjReGNn17z5xcF81yRLren5n6EainQeKPpxk+v+fOLgnmOQbQU6KjxovDGT6/584uCeY6BfuxdNd5ouvHTa/78omCeY2CNvavGi8IbP73mzy8K5rkuJPbGi8IbP73mzy8K5rkyXRbicz9CbZ66N99ouvHTa/78omCe+yc2TwGgLayxA40oVT9OXXq76McOBFaq9z49/dvGUgwQWKn6cerS68RSDNCAUr336enftqkSu5l92sxeMbMXzexJM7sjV2AAytWPU5fetmmv2J+R9IC7v0vS9yU9On1IAHaUqh+nLr1tUyV2d/+6u+98suTzku6ePiQAO0r13qenf9uybZ6a2b9L+ld3f+KAr69KWpWkpaWlU5vjdm4AAAfqunl6ZLmjmV2SdOeYL11w96e2X3NB0nVJB1bCuvuapDVpWBVz1PcFAEzmyMTu7mcO+7qZfVjSw5JOex+1kwCAXaa6QcnMzkr6hKQ/dPetPCEBAKYxbVXMZyW9TdIzZvaCmX0+Q0wAgClMdcXu7u/IFQgAIA/uPAWAxpDYAaAxJHYAaAyJfQ7QdxuYL/Rjbxx9t4H5wxV74y5cGCX1HVtbw+MA2kRibxx9t4H5Q2JvHH23gflDYm8cfbeB+UNibxx9t4H5Q1XMHFhZIZED84QrdgBoDIkdABpDYgeAxpDYAaAxJHYAaAyJHQAaQ2IHgMaQ2AGgMebus/+mZtckbd5y6Ncl/WzmgdSBuTkc83Mw5uZwNc7PCXdfPOpFvST2fUGYbbj7ct9xRMTcHI75ORhzc7iW54elGABoDIkdABoTJbGv9R1AYMzN4ZifgzE3h2t2fkKssQMA8olyxQ4AyITEDgCNCZPYzezTZvaKmb1oZk+a2R19xxSFmX3QzF42s5tm1mR5ViozO2tmr5rZa2b2yb7jicTMHjezn5rZS33HEo2Z3WNm3zCzK9u/U4/0HVMJYRK7pGckPeDu75L0fUmP9hxPJC9J+lNJz/UdSARmNpD0OUl/JOl+SX9uZvf3G1Uo/yjpbN9BBHVd0sfd/T5J75H00RbfO2ESu7t/3d2vbz99XtLdfcYTibtfcfdX+44jkN+X9Jq7/8Dd/0/Sv0j6k55jCsPdn5P0877jiMjdf+Lu393+719KuiLprn6jyi9MYt/jLyT9Z99BIKy7JP3oludX1eAvJ8oys5OS3i3pO/1Gkt9MP8zazC5JunPMly64+1Pbr7mg4f8urc8ytr51mRu8ycYco24XnZnZWyV9WdLH3P0XfceT20wTu7ufOezrZvZhSQ9LOu1zVmB/1Nxgl6uS7rnl+d2SftxTLKiMmd2mYVJfd/ev9B1PCWGWYszsrKRPSPpjd9/qOx6E9t+SfsfM3m5mb5H0Z5L+reeYUAEzM0lfkHTF3T/TdzylhEnskj4r6W2SnjGzF8zs830HFIWZfcDMrkp6UNLXzOzpvmPq0/Ym+19KelrDza8vufvL/UYVh5l9UdK3Jb3TzK6a2Uf6jimQ90r6kKT3beeZF8zsob6Dyo2WAgDQmEhX7ACADEjsANAYEjsANIbEDgCNIbEDQGNI7ADQGBI7ADTm/wFmtHNBW1/OsAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.svm import LinearSVC\n",
    "from sklearn import preprocessing\n",
    "\n",
    "X_scaled = preprocessing.scale(X)\n",
    "svc = LinearSVC(C=1e9)\n",
    "svc.fit(X_scaled, y)\n",
    "results = svc.predict(preprocessing.scale([[6,2], [4, 3]]))\n",
    "print(results)\n",
    "\n",
    "plt.scatter(X_scaled[y == 0, 0], X_scaled[y == 0, 1], color='red')\n",
    "plt.scatter(X_scaled[y == 1, 0], X_scaled[y == 1, 1], color='blue')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最后我们在图中画出划分超平面，也就是决策边界，决策边界为一条直线：\n",
    "\n",
    "$$\n",
    "w_1x + w_2y + b = 0\n",
    "$$\n",
    "\n",
    "这里的 $w$ 为特征系数，可以通过 `svc.coef_` 取得，$b$ 为截距，可以通过 `intercept_` 取得："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w =  [[ 4.03244052 -2.50699934]]\n",
      "b =  [0.92733476]\n"
     ]
    }
   ],
   "source": [
    "print('w = ', svc.coef_)\n",
    "print('b = ', svc.intercept_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从而得到直线方程：\n",
    "\n",
    "$$\n",
    "y = -\\frac{w_1x+b}{w_2}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl4VEX2N/BvkQAaFhcEZWR9HRjpBASMKOLCJogg6k9EFEaFJoGwCCKCTHRcICgiCIY1IQxo4qDIvggMyGoIISGEhE0QQTYVZJMJEJKc94/ACBjSnb73dvXt/n6epx/t5d46N2mPlVO3qpSIgIiI/Ecp3QEQEZG5mNiJiPwMEzsRkZ9hYici8jNM7EREfoaJnYjIzzCxExH5GSZ2IiI/w8RORORngnU0etttt0mtWrV0NE1EZFvp6enHRaSyq89pSey1atVCWlqajqaJiGxLKXXAnc+xFENE5GeY2ImI/AwTOxGRn2FiJyLyM0zsRER+homdiMjPMLETEfkZJnYiIi/47bffMHDgQJw+fdrytpjYiYgsJCKYPXs2HA4HJk6ciHXr1lneJhM7EZFFjh49iv/7v/9D586dUb16daSnp+PJJ5+0vF0mdiIik4kIpk+fjnr16mHZsmX46KOPkJKSggYNGnilfS1rxRAR+at9+/YhMjISq1atwiOPPIJp06ahTp06Xo2BPXYiIhPk5+dj3LhxqF+/PlJTUzF58mSsXr3a60kdYI+diMiwHTt2wOl0IiUlBU888QSmTJmC6tWra4uHPXYiIg/l5uZi+PDhaNSoEfbs2YPExEQsXrxYa1IH2GMnIvJIWloanE4ntm3bhi5dumD8+PGoUqWK7rAAsMdORFQiOTk5GDJkCO6//34cP34cCxYswL///W+fSeoAe+xERG5bu3Ytevbsib179yIiIgKjR4/GTTfdpDusP2GPnYjIhTNnziAqKgrNmzdHQUEBVq1ahbi4OJ9M6gATOxFRsZYsWYLQ0FDExcVh0KBByMrKQsuWLXWHVSwmdiKiIhw/fhzdunVDhw4dcNNNNyE5ORljxoxBSEiI7tBcYmInIrqCiGDWrFmoV68evvrqK7z77rvYsmUL7r//ft2huY2Dp0RElxw+fBhRUVFYtGgRmjRpgoSEBISFhekOq8TYYyeigCciiI+Ph8PhwMqVKzFmzBgkJyfbMqkD7LETUYD74YcfEBERgdWrV6NFixaIj4/HXXfdpTssQ9hjJ6KAlJ+fj7Fjx6J+/fpIT09HXFwcVq1aZfukDrDHTkQBKDs7G06nE6mpqXjyyScxefJk3HnnnbrDMg177EQUMHJzc/Huu++icePG+PHHHzFr1iwsWLDAr5I6wB47EQWI1NRU9OjRA9u3b0fXrl0xbtw43HbbbbrDsoRpPXalVJBSKkMptdiscxIRGZWTk4PXX38dTZs2xenTp7F48WIkJib6bVIHzO2xDwCwE0BFE89JROSx1atXo2fPnti3bx969+6NUaNGoWJF/09RpvTYlVLVALQHMM2M8xERGXH69GlERkaiZcuWKFWqFNasWYPJkycHRFIHzCvFjAMwBECBSecjIvLIwoUL4XA4kJCQgDfeeAOZmZl49NFHdYflVYYTu1KqA4BfRSTdxecilVJpSqm0Y8eOGW2WiOgqv/76K7p06YKnnnoKlSpVwqZNm/DRRx/ZYtEus5nRY28GoKNSaj+AWQBaKqUSr/2QiMSJSLiIhFeuXNmEZomICpcDSEpKgsPhwLx58zB8+HCkpaUhPDxcd2jaGE7sIjJMRKqJSC0AXQB8KyLdDEdGROTCwYMH8eSTT6Jbt26oU6cOMjIy8NZbb6FMmTK6Q9OKE5SIyHYKCgowZcoUhIaGYvXq1Rg3bhw2bNgAh8OhOzSfYOoEJRFZA2CNmeckIrrSnj17EBERgbVr16J169aIi4tD7dq1dYflU9hjJyJbyMvLw+jRo9GgQQNs3boVCQkJWLFiBZN6EbikABH5vMzMTDidTqSnp+Ppp5/GxIkT8Ze//EV3WD6LPXYi8lkXLlzA22+/jfDwcBw8eBBfffUV5s6dy6TuAnvsROSTNm7cCKfTiZ07d+Kll17C2LFjUalSJd1h2QJ77ETkU/773/9i4MCBaNasGc6ePYulS5di5syZTOolwB47EfmMlStXIiIiAvv370ffvn3xwQcfoEKFCrrDsh322IlIu5MnT8LpdOKxxx5DmTJlsG7dOkyYMIFJ3UNM7ESk1bx58+BwODBz5ky8+eabyMzMxMMPP6w7LFtjKYaItPjll1/Qv39/zJ49Gw0bNsSSJUvQuHFj3WH5BfbYicirRASfffYZ6tWrhwULFiAmJgapqalM6iZij52IvOann35Cr169sGzZMjz44INISEjA3XffrTssv8MeOxFZrqCgABMnTkRoaCjWr1+P2NhYrF+/nkndIuyxE5Gldu/ejZ49e2LDhg1o06YNpk6dilq1aukOy6+xx05Elrh48SI+/PBD3HPPPdi+fTtmzJiBZcuWMal7AXvsRGS6jIwMOJ1OZGRk4Nlnn8WECRNwxx136A4rYLDHTkSmOX/+PKKjo3HffffhyJEj+Prrr/H1118zqXsZe+xEZIrvvvsOTqcTu3fvxiuvvIIxY8bg1ltv1R1WQGKPnYgM+f3339G/f388/PDDOH/+PJYvX45//etfTOoaMbETmS0pCahVCyhVqvCfSUm6I7LM8uXLERYWhokTJ6J///7Izs5GmzZtdIcV8FiKITJTUhIQGQnk5BQ+P3Cg8DkAdO2qLy6TnThxAoMGDcLMmTNx9913Y/369WjWrJnusOgS9tiJzBQd/UdSvywnp/B1PzFnzhw4HA4kJiYiOjoaGRkZTOo+hj12IjP99FPJXreRo0ePol+/fpg7dy4aNWqEZcuWoWHDhrrDoiKwx05kpho1Sva6DYgIZsyYAYfDgSVLluDDDz9Eamoqk7oPY2InMlNMDBAScvVrISGFr9vQ/v370bZtW3Tv3h3169fHtm3bMHToUAQH8499X8bETmSmrl2BuDigZk1AqcJ/xsXZbuA0Pz8fn376KcLCwrBx40ZMnDgRa9asQd26dXWHRm7g/3aJzNa1q+0S+ZV27tyJnj17Ijk5GY8//jimTp2KGjYuJQUi9tiJrhVA96Ff6eLFi4iJiUHDhg2xa9cufPbZZ1i6dCmTug2xx050pQC5D/1aW7ZsQY8ePZCZmYnOnTvj008/xe233647LPIQe+xEVwqA+9CvdO7cObz55pto0qQJfv31V8ybNw9ffvklk7rNscdOdCU/vg/9WuvWrUPPnj2xZ88eOJ1OfPzxx7j55pt1h0UmYI+d6Ep+eB/6tc6cOYO+ffvi0UcfRV5eHlauXIlp06YxqfsRJnayHysHN/3sPvRrffPNNwgLC8PkyZMxcOBAZGVloVWrVrrDIpMZTuxKqepKqdVKqZ1Kqe1KqQFmBEZUpMuDmwcOACJ/DG6aldz95D70a/3222946aWX8MQTT6BChQpITk7GJ598gnLlyukOjSygRMTYCZSqCqCqiGxRSlUAkA7gaRHZcb1jwsPDJS0tzVC7FKBq1SpM5teqWRPYv9/b0fg8EcHs2bPRr18/nDx5EsOGDUN0dDTKli2rOzTygFIqXUTCXX3O8OCpiBwFcPTSv/+ulNoJ4E4A103sRB4LoMFNo44cOYI+ffpgwYIFCA8Px8qVK9GgQQPdYZEXmFpjV0rVAtAIwCYzz0v0P3Yf3PTC5CcRQUJCAhwOB5YvX47Ro0dj48aNTOoBxLTErpQqD2AOgIEicqaI9yOVUmlKqbRjx46Z1SwFGjsPblo9PgBg3759aN26NXr27ImGDRsiKysLgwcP5qJdgUZEDD8AlAawHMAgdz5/7733CpHHEhNFatYUUarwn4mJuiNyT82aIoUp/epHzZqGT52XlyeffPKJhISESIUKFWTKlCmSn59v+LzkWwCkiRs51ozBUwVgJoATIjLQnWM4eEoBqVSpwlR+LaWAggKPT7t9+3Y4nU5s2rQJ7du3x5QpU1CtWjUDgZKvcnfw1IxSTDMAfwfQUim19dLjCRPOS2SNPn2A4ODChBocXPjcG0weH8jNzcX777+PRo0aYe/evUhKSsKiRYuY1MmUu2I2AFAmxEJkvT59gMmT/3ien//H80mTrG07JubqBcYAj8cHNm/eDKfTiaysLLzwwgsYP348KleubGKwZGeceUqBJS6uZK+byYTJTzk5OXjjjTfwwAMP4MSJE1i4cCG++OILJnW6CofKKbDk55fsdbMZ2IRjzZo1iIiIwN69exEZGYmPPvoIN910k8kBkj9gj50CS1BQyV73AadPn0bv3r3RokULiAi+/fZbTJ06lUmdrouJnXxP69aFpYrLj9atzTv35U0z3H3dEyZOQlq8eDFCQ0MRHx+P119/Hdu2bUOLFi1MC5X8lDv3RJr94H3sdF2tWhV9r3erVua1ERUlEhRUeN6goMLnZklMFAkJuTr2kJAS32v/66+/ygsvvCAAJCwsTDZt2mRejGRb8NZ97J7gfex0XaqYG6w0fFdLzOAiZSKCWbNm4dVXX8Xp06fx1ltv4c0330SZMmVMD5Xsx2uLgBHRFQwsUnbo0CFERUVh8eLFaNKkCRISEhAWFmZygBQIWGMn/+Oqxm3lQlweTEIqKChAXFwcQkNDsWrVKowdOxbJyclM6uQ5d+o1Zj9YY6frcjiKrrE7HO4d76rGbVIN3OP2r7Fnzx5p3ry5AJAWLVrI3r17zYmD/BLcrLEzsZNvMbpQlqvjLVyI63/cWKTs4sWL8vHHH8uNN94oFStWlPj4eCkoKDAvBvJL7iZ2Dp6SbzG6UJar4y1aiKsksrKy4HQ6sXnzZnTs2BGTJk3CnXfe6ZW2yd68uQgY2Y3Vmz0YqXEbXSjL1fEaN+q4cOEC3nnnHTRu3Bj79+/HrFmzMH/+fCZ1Mp873XqzHyzFaKS7xmx1DVx3jf06UlJSJDQ0VABIt27d5NixY5a2R/4JrLFTkayuMZtR4za6kYar4724UcfZs2fltddeE6WUVKtWTRYvXmxZW+T/3E3srLEHGqtrzDaocXvLt99+i4iICOzbtw9RUVH48MMPUbFiRd1hkY2xxk5Fs7rGbEaN29VGGLrHCFw4deoUIiIi0KpVK5QqVQpr1qzBpEmTmNTJe9zp1pv9YClGI901dldrwURFFf3+5fVcdMfvwvz586Vq1apSqlQpGTJkiOTk5JgTF5Gwxk7FsbrGXNz5Ly++de0jKMi993WPEVzHL7/8Is8//7wAkAYNGsjmzZvNiYfoCu4mdtbYybtcLfLl6n3dYwR/CkmQlJSEAQMG4OzZs3j77bcxdOhQlC5d2ngsRNdgjZ18k6uNLly9r3uM4AoHDx5Ehw4d8Pe//x1169ZFRkYG3nrrLSZ10o6JPRAZHXx0NbhZHFcbXbh6PyYGuDZxli5dsg2hi4s/JqZwg+krXbPhdEFBASZPnozQ0FCsWbMG48aNw4YNG+BwONyPwUNWjxuTn3CnXmP2gzV2jYwOProa3DR6fGKiSHDw1e8FB189wahMmavfL1PG3PiLGSPYvXu3PPzwwwJAWrduLfv27XOvXRNomltFPgQcPKUiGR18dDW4afR4qxfx8jD+ixcvyqhRo+SGG26Qm2++WaZPn+71Rbu8sX4Z+TZ3EzsHTwON0cFHozscGR0c1RB/ZmYmevTogS1btuCZZ57BxIkTUbVqVddtmSyA5nbRdXDwlIpmdPDR1eCm0eOtXsSrBPFfuHABb7/9NsLDw3Ho0CHMnj0bc+bM0ZLUAe+sX8Yavn9gYg80bgwOFsvV4KYrzZsX//oTTxT9/uXXvRT/xo0b0ahRI4wYMQIvvvgiduzYgU6dOkEV1+O3mNFLdyUpqfDHcOBA4V8GBw4UPmdytyF36jVmP1hj18zoBKWoqD9q1UFB7g+civjGImHFxP/777/LgAEDRCklNWrUkG+++aZk57aYlXPLWMP3fWCNnXySDy8S9p///AeRkZHYv38/+vXrh5EjR6JChQqWtulLWMP3fayx+zOjhVBXxxu5T90VH9wI4+TJk+jRowfatGmDsmXLYv369YiNjdWS1HXWuN350du9Bm/3+N3mTrfe7AdLMQZYvRGF0fvUXdG9yNc15s6dK3fccYcEBQXJsGHD5Ny5c5a04w7d96n76B4lprF7/CLul2KY2O3G6s2ejd6nbkb8XtgI4+jRo9KpUycBIA0bNpT09HTT2ygpX6hxF/ej94X4jLB7/CLuJ3bW2O3G6s2ejd6nbrR9i4kIPvvsM7z22mvIycnBO++8g8GDB/vE+i6+XuP29fhcsXv8gJdr7Eqpx5VSu5VSe5VSb5pxTroOqzd7NnqfutH2LXTgwAG0a9cOr7zyCurVq4etW7di2LBhPpHUAa0/Grf4enyu2D3+EnGnW1/cA0AQgB8A/D8AZQBkAnAUdwxLMQaYUWMvbq0Vg2up/O8c17sdUkOhMz8/X2JjY6VcuXJSrlw5iY2Nlfz8fMva85Sv14C9EZ+RO2kvx3i9r6av/3zdAW/V2AE0BbD8iufDAAwr7hgmdoOM1KATE0VKl77621269NXnMJKYzfgfg4l27dolzZo1EwDStm1b2b9/v2VtmcGLPxqPWBmf0XF7dxK3r/98XfFmYu8EYNoVz/8OYEJxxzCxa2T3wVc35ebmysiRI6Vs2bJyyy23yIwZM7y+aBeVjNGvjj8MjrribmIPNqGaU9Ro25+GKJRSkQAiAaCGXxa1bOKnn0r2ekmPz88v+v3rvW6BjIwMOJ1OZGRkoFOnToiNjcUdd9zhtfbJM0a/Oka/2v7EjMHTQwCqX/G8GoAj135IROJEJFxEwitXrmxCs37M1SwKI7MsfGHw1aJZIufPn8c//vEP3HfffThy5AjmzJmD2bNn/ympW73PiJHzWz33TLfi4jM6bh9Qg6OuuNOtL+4BIBjAPgC18cfgaWhxx7AUUwyrZ4kYPb5Vq6L/3m3Vyr33LRrBWr9+vdStW1cASPfu3eXEiROWXL6V86t0/2qt5io+V18do+f3B/DmBCUATwD4HoV3x0S7+jwTezGs3mhCxNgIku6NMq5x5swZ6du3rwCQWrVqyfLly4v9vNX7dBg5v9XDH7r5+lfbDtxN7Jyg5Gt8eJGs/7VzPSJejX/58uWIjIzEwYMH0b9/f8TExKB8+fLFHmP1Ph1Gzm/13DPdfP2rbQdcBMyufHCRrKvo3igDwIkTJ/Dyyy/j8ccfR0hICDZs2IDx48e7TOpmNG/08otjdBEubyziZeXwju6vtl9xp1tv9oOlmGLoXqTLFaNFZgOF0IKCApk9e7ZUqVJFgoODJTo6usSLdlk9xGDk12P0R2f1+mq+PD4RKMBFwGzM11dicjU90FWh04NC6JEjR+SZZ54RANK4cWPJyMjwOHwjdVir68RGfvVWx+aN4/29Rm6Uu4mdNXa7CbBCpIhgxowZGDRoEM6fP4/33nsPgwYNQnCwGVMwSk5nndho27pr+AH21bUEa+z+KoAKkT/++CPatGmDHj16oH79+sjMzMSQIUO0JXVAb53YaNtWT2Gw+ngqAXe69WY/WIoxIAAKkXl5eTJ+/HgJCQmR8uXLy6RJk3xm0S536uDX3hIZFGTOr8edtkuVuvr9UqW8OwXC6kW4Ar1UA9bY/Zgff7t37NghTZs2FQDSrl07OXDggO6Q/qS4H7/VY9tG2zb61TGauI2uX+fnfRqX3E3srLGTT7h48SJGjRqF4cOHo3z58hg/fjy6du0KVdyN4z4oOLjotU2CgoC8PP9tGyi8/fHAgT+/XrMmsH+/75/fDtytsesrVhJdkp6ejh49emDbtm3o3LkzYmNjUaVKFd1heUTnGmi611+zehEuLvLlPg6ekjbnzp3D0KFD0aRJExw7dgzz5s3Dl19+6TKp617oyshCVq4WELOybatZPTjKwdcScKdeY/aDNXZau3at1KlTRwBIz5495eTJk24dp7vOamSSkNUbSeieu2b170b3794XgIOn5ItOnz4tUVFRAkBq164tK1euLNHxuudnudP+9eZveWMjCaNbyxll9bi+H9834BZ3EzsHT8lrli5dil69euHw4cMYOHAghg8fjnLlypXoHLonuRhp39UCYla2Tf6BE5TIZxw/fhzdunVD+/btUbFiRSQnJ2Ps2LElTuqAd+qsVi20xY0kyGvc6dab/WApJjAUFBTIrFmzpHLlyhIcHCz//Oc/5fz584bOqbuOa+R9biRBRoE1dtLp8OHD0rFjRwEg4eHhkpmZadq5rayzGl2oqrjjuZEEGeVuYmeNnUwlIkhISMDgwYNx4cIFjBgxAgMGDNC6vktJWLnQFcAaORnDGjt53b59+9C6dWtERESgYcOGyMrKwuuvv17ipK7zPnUrN7uwQ41c9xwBMok73XqzHyzF+Je8vDwZO3as3HjjjVKhQgWZOnWqx4t26a4jW7mPiO5rc8XX4yPW2MlLsrKypEmTJgJA2rdvLwcPHjR0Pl+/T91oDd6Xa+S6f/bkmruJnTV28khubi4++OADxMTE4KabbsKnn36KLl26GF60S/e92oG84bI/X5u/YI2dLLN582bce++9ePfdd/Hcc89hx44deOGFF0xZiVF3HTqQN1z252sLNEzs5LacnBwMHjwYDzzwAE6ePImFCxciKSkJlStXvupzRgbgYmKAkJCrXwsJKXzdG2JiChfnulJw8B/t647PSv58bQHHnXqN2Q/W2O1n9erVctdddwkA6dWrl5w6darIz9l9lxxvbFbhy/z52vwBWGMnM5w+fRpDhgxBXFwc7rrrLsTHx6NFixbX/bzdN0PQvVkFUXFYYyfDFi1aBIfDgWnTpmHw4MHYtm1bsUkdsP9mCLo3qyAyAxO7PzI4y+TYsWN48cUX0bFjR1SqVAkpKSkYPXo0Qq4twBbB6AQf3XRvVqGbL/9uqATcqdeY/WCN3UIGitwFBQWSlJQklSpVktKlS8t7770nFy5cMLV5X58EY3ShLjvz9d8NuV9jZ2L3Nx7OMjl48KB06NBBAMj9998v2dnZHofg6SJZvsDX47NSIF+7Xbib2Dl46m9KOMukoKAA8fHxeOONN5CXl4eYmBi8+uqrCLKo9uDrk2B8PT4rBfK12wUHTwNVCWaZ7NmzBy1btkTv3r1x3333ITs7G6+99pplSb2E4Wnh6/EZrYEb3USEbMKdbr3ZD5ZiLORGofTixYsyevRoueGGG6RixYoSHx8vBQUFvhKeVr4cn9HY7D7+QV6qsQMYDWAXgG0A5gG42Z3jmNgtVkyROzMzU8LDwwWAdOzYUQ4dOuRL4fkEX43PaA3c6AJmpJ+7id1QjV0p1QbAtyKSp5QadekvgKGujmON3fsuXLiAkSNHYuTIkbjlllswYcIEPPfcc6as70LeYeUmIKyh24NXauwiskJELs/HSwFQzcj5yBopKSlo3Lgx3n//fXTp0gU7d+5E586dmdRtxmgNnDX0wGHm4GkPAN+YeD4y6L///S8GDRqEBx98EGfOnMGSJUvw+eefo1KlSrpDIw8YXaSLi3wFEFe1GgArAWQX8Xjqis9Eo7DGroo5TySANABpNWrU8EI1KrCtXLlSateuLQAkKipKTp8+rTskMoHRGjhr6PYGb93HrpR6GUBvAK1EJMedY1hjt86pU6cwePBgJCQkoE6dOpg2bRoeeeQR3WERkQm8UmNXSj0OYCiAju4mdbLOggUL4HA4MGPGDAwdOhSZmZlM6kQByGiNfQKACgD+o5TaqpSaYkJMVEK//PILnn/+eTz99NOoUqUKNm3ahA8//BA33nij7tCKxIWmiKwV7Poj1ycifzUrECo5EUFiYiIGDhyIs2fPYsSIERgyZAhKly6tO7TrSkoCIiOBnEt/3x04UPgcALp21RcXkT/hkgI29dNPP6F9+/Z46aWX8Le//Q1bt25FdHS0Tyd1AIiO/iOpX5aTU/g6EZmDid1mCgoKMGnSJISGhmLt2rUYP3481q9fj3r16ukOzS1234iDyA6Y2G3k+++/R/PmzdG3b1888MADyM7OtnQlRitwkgyR9ZjYbSAvLw+jRo1CgwYNkJWVhenTp2PFihWoXbu27tBKjJNkiKxnaPCUrJeZmYkePXpgy5YteOaZZzBx4kRUrVpVd1geuzxAGh1dWH6pUaMwqXPglMg8TOw+6vz58xgxYgRGjRqFSpUq4euvv8azzz6rOyxTdO3KRE5kJSZ2H5ScnAyn04ldu3bh5ZdfxtixY3HrrbfqDouIbII1dh9y9uxZvPrqq3jooYeQk5ODZcuWYcaMGUzqRFQiTOw+YsWKFQgLC8OECRPQt29fZGdno23btrrDIiIbYmLX7OTJk+jevTvatm2LG264AevWrUNsbCwqVKigOzQisikmdo3mzp0Lh8OBzz//HMOGDcPWrVvx0EMP6Q6LiGyOg6ca/Pzzz+jXrx/mzJmDhg0bYunSpWjUqJHusIjIT7DH7kUighkzZsDhcGDx4sUYOXIkUlNTmdSJyFTssXvJ/v370atXL6xYsQLNmjXDtGnTcPfdd+sOi4j8EHvsFisoKEBsbCzCwsKQnJyMCRMmYN26dUzqRGQZ9tgttGvXLvTs2RPfffcd2rZti6lTp6JmzZq6wyIiP8ceuwUuXryIkSNH4p577sGOHTswc+ZMfPPNN0zqROQV7LGbbMuWLXA6ndi6dSs6deqECRMm4Pbbb9cdFhEFEPbYTXLu3DkMGzYMTZo0wc8//4w5c+Zg9uzZTOpE5HXssZtgw4YNcDqd+P7779G9e3eMGTMGt9xyi+6wiChAscduwO+//45+/frh4YcfRm5uLlasWIHp06czqRORVkzsHlq2bBnCwsIwadIkDBgwAFlZWXjsscd0h0VExMReUr/99htefvlltGvXDuXKlcN3332HcePGoXz58rpDIyICwMTuNhHB7Nmz4XA48MUXX+Ctt95CRkYGmjZtqjs0IqKrcPDUDUePHkWfPn0wf/583HvvvVixYgXuuece3WERERWJPfZiiAimT5+OevXqYdmyZRg1ahRSUlKY1InIp7HHfh0//vgjIiP6A1ExAAAGwklEQVQjsXLlSjzyyCOIj49H3bp1dYdFROQSe+zXyM/Px/jx4xEWFoZNmzZh8uTJWL16NZM6EdkGe+xX2LFjB5xOJ1JSUtCuXTtMnToV1atX1x0WEVGJsMcOIDc3F8OHD0ejRo2wZ88eJCYmYsmSJUzqRGRLAd9jT0tLg9PpxLZt2/D888/j008/RZUqVXSHRUTksYDtsZ87dw5DhgzB/fffj+PHj2P+/PmYNWsWkzoR2Z4piV0pNVgpJUqp28w4n9XWrl2LBg0aYPTo0XA6ndi+fTueeuop3WEREZnCcGJXSlUH8BiAn4yHY60zZ84gKioKzZs3R0FBAVatWoW4uDjcfPPNukMjIjKNGT32TwAMASAmnMsyS5YsQWhoKOLi4jBo0CBs27YNLVu21B0WEZHpDCV2pVRHAIdFJNOkeEx3/PhxdOvWDR06dEDFihWRnJyMMWPGoFy5crpDIyKyhMu7YpRSKwHcUcRb0QD+AaCNOw0ppSIBRAJAjRo1ShCiZ0QEX375Jfr3749Tp07hnXfewbBhw1C2bFnL2yYi0kmJeFZBUUrVB7AKQM6ll6oBOAKgiYj8XNyx4eHhkpaW5lG77jh8+DD69OmDhQsX4r777kNCQgLq169vWXtERN6glEoXkXBXn/P4PnYRyQLwv3sDlVL7AYSLyHFPz2mUiGDatGkYPHgwLl68iI8//hgDBw5EUFCQrpCIiLzObyYo/fDDD4iIiMDq1avRvHlzxMfH469//avusIiIvM60CUoiUktHbz0/Px9jx45F/fr1kZ6ejqlTp2LVqlVM6kQUsGzdY8/OzobT6URqaio6dOiAyZMno1q1arrDIiLSypZLCuTm5uK9995D48aNsW/fPnzxxRdYuHAhkzoREWzYY09NTYXT6UR2djZefPFFjBs3DpUrV9YdFhGRz7BVj33EiBFo2rQpTp48iUWLFiEpKYlJnYjoGrZK7HfddRciIiKwfft2dOjQQXc4REQ+yeMJSkZYPUGJiMgfuTtByVY9diIico2JnYjIzzCxExH5GSZ2IiI/w8RORORnmNiJiPwMEzsRkZ9hYici8jNaJigppY4BOODh4bcB0LaZh8l4Lb7HX64D4LX4KiPXUlNEXK6joiWxG6GUSnNn5pUd8Fp8j79cB8Br8VXeuBaWYoiI/AwTOxGRn7FjYo/THYCJeC2+x1+uA+C1+CrLr8V2NXYiIiqeHXvsRERUDFsmdqXUcKXUNqXUVqXUCqXUX3TH5Cml1Gil1K5L1zNPKXWz7pg8oZR6Tim1XSlVoJSy5d0LSqnHlVK7lVJ7lVJv6o7HU0qp6UqpX5VS2bpjMUIpVV0ptVoptfPSd2uA7pg8pZS6QSmVqpTKvHQt71nanh1LMUqpiiJy5tK/vwrAISK9NYflEaVUGwDfikieUmoUAIjIUM1hlZhSqh6AAgBTAQwWEVvtpKKUCgLwPYDHABwCsBnACyKyQ2tgHlBKPQLgLIDPRCRMdzyeUkpVBVBVRLYopSoASAfwtE1/JwpAORE5q5QqDWADgAEikmJFe7bssV9O6peUA2C//ztdIiIrRCTv0tMUANV0xuMpEdkpIrt1x2FAEwB7RWSfiOQCmAXgKc0xeURE1gE4oTsOo0TkqIhsufTvvwPYCeBOvVF5RgqdvfS09KWHZXnLlokdAJRSMUqpgwC6Avin7nhM0gPAN7qDCFB3Ajh4xfNDsGkS8UdKqVoAGgHYpDcSzymlgpRSWwH8CuA/ImLZtfhsYldKrVRKZRfxeAoARCRaRKoDSALQT2+0xXN1LZc+Ew0gD4XX45PcuQ4bU0W8Ztu/BP2JUqo8gDkABl7z17qtiEi+iDRE4V/lTZRSlpXJgq06sVEi0trNj34BYAmAdywMxxBX16KUehlABwCtxIcHPUrwO7GjQwCqX/G8GoAjmmKhSy7Vo+cASBKRubrjMYOInFJKrQHwOABLBrh9tsdeHKVUnSuedgSwS1csRimlHgcwFEBHEcnRHU8A2wygjlKqtlKqDIAuABZqjimgXRpwTACwU0TG6o7HCKVU5ct3vCmlbgTQGhbmLbveFTMHwN9QeBfGAQC9ReSw3qg8o5TaC6AsgN8uvZRixzt8lFLPAIgFUBnAKQBbRaSt3qhKRin1BIBxAIIATBeRGM0heUQp9W8AzVG4iuAvAN4RkQStQXlAKfUQgPUAslD43zoA/ENEluqLyjNKqQYAZqLwu1UKwFci8r5l7dkxsRMR0fXZshRDRETXx8RORORnmNiJiPwMEzsRkZ9hYici8jNM7EREfoaJnYjIzzCxExH5mf8P6jX0NJQWAaQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def plot_svc_decision_boundary(model):\n",
    "    w = model.coef_[0]\n",
    "    b = model.intercept_[0]\n",
    "    x = np.linspace(-3, 3, 200)\n",
    "    y = - (w[0]*x+b) / w[1]\n",
    "    plt.plot(x, y, color='black')\n",
    "\n",
    "plot_svc_decision_boundary(svc)\n",
    "plt.scatter(X_scaled[y == 0, 0], X_scaled[y == 0, 1], color='red')\n",
    "plt.scatter(X_scaled[y == 1, 0], X_scaled[y == 1, 1], color='blue')\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}