Lab03.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8PsD0Lw0g_mL"
   },
   "source": [
    "# Lab 3: Minimizing Cost"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UPg1vs52g_mQ"
   },
   "source": [
    "Author: Seungjae Lee (이승재)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "A1VVTA6Xg_mR"
   },
   "source": [
    "<div class=\"alert alert-warning\">\n",
    "    We use elemental PyTorch to implement linear regression here. However, in most actual applications, abstractions such as <code>nn.Module</code> or <code>nn.Linear</code> are used.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HSf_efZxg_mS"
   },
   "source": [
    "## Theoretical Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9hOXEnwKg_mS"
   },
   "source": [
    "$$ H(x) = Wx $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "juqvz7hSg_mT"
   },
   "source": [
    "$$ cost(W) = \\frac{1}{m} \\sum^m_{i=1} \\left( Wx^{(i)} - y^{(i)} \\right)^2 $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VFNYs7EBg_mU"
   },
   "source": [
    " - $H(x)$: 주어진 $x$ 값에 대해 예측을 어떻게 할 것인가\n",
    " - $cost(W)$: $H(x)$ 가 $y$ 를 얼마나 잘 예측했는가"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UPBuQBPJg_mV"
   },
   "source": [
    "Note that it is simplified, without the bias $b$ added to $H(x)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "l9OqqeHgg_mX"
   },
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "_EzXq4xBg_mY"
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import torch.optim as optim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "KCD_9kMcg_ma",
    "outputId": "0c5c2cf3-04bf-4039-d356-e240a5a1aa97"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x22711916770>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# For reproducibility\n",
    "torch.manual_seed(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DX-yid2yg_mb"
   },
   "source": [
    "## Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HMDL0D3Kg_mc"
   },
   "source": [
    "We will use fake data for this example."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SWh1HBeqg_mc"
   },
   "source": [
    "기본적으로 PyTorch는 NCHW 형태이다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "GycghV9Og_mc"
   },
   "outputs": [],
   "source": [
    "x_train = torch.FloatTensor([[1], [2], [3]])\n",
    "y_train = torch.FloatTensor([[1], [2], [3]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "sdyERsQEg_md",
    "outputId": "4a3e17ad-eb2b-42df-b819-19b8b0407168"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x227120685c8>]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXhU9dn/8fcNhH3f1xCQfRUMoGLdZRNFxLb4WKq2lra2v6erAq4oWlH72NpWi1SpYl0fEhAVRaxaUBRZxCTsYRFC2JcQIIEs9++PGXxiTGCASSYz+byuK1dmvuc7M3cOh09Ozpy5j7k7IiISu6pEugARESlbCnoRkRinoBcRiXEKehGRGKegFxGJcdUiXUBJmjZt6gkJCZEuQ0QkaixfvnyvuzcraVmFDPqEhASWLVsW6TJERKKGmX1V2jIduhERiXEKehGRGKegFxGJcQp6EZEYp6AXEYlxCnoRkRh3yqA3s5pm9rmZfWlmq8zsgRLm1DCz18ws3cyWmFlCkWWTguPrzGxoeMsXEZFTCeU8+mPA5e5+2MzigI/N7B13/6zInB8DB9y9k5mNBR4Fvm9mPYCxQE+gNfC+mXVx94Iw/xwiIlFpzhfbeXz+OjIP5tC6YS3uGNqV6/q1CetrnHKP3gMOB+/GBb+KN7EfBbwQvD0LuMLMLDj+qrsfc/fNQDowMCyVi4hEuTlfbGdScirbD+bgwPaDOUxKTmXOF9vD+johHaM3s6pmthLYDSxw9yXFprQBtgG4ez6QBTQpOh6UERwTEan0Hnt3LTl53zzAkZNXwOPz14X1dUIKencvcPdzgbbAQDPrVWyKlfSwk4x/i5mNN7NlZrZsz549oZQlIhK10rZnkZmVW+KyzIM5YX2t0zrrxt0PAh8Bw4otygDaAZhZNaABsL/oeFBbILOU557u7onuntisWYl9eUREol5uXgGPvruWUU99QpWSdoWB1g1rhfU1QznrppmZNQzergVcCawtNm0ucHPw9g3ABx64GO1cYGzwrJwOQGfg83AVLyISTZZs2sfwJxfx9482cn2/NkwZ1YtacVW/MadWXFXuGNo1rK8bylk3rYAXzKwqgV8Mr7v7W2b2ILDM3ecCzwEvmlk6gT35sQDuvsrMXgdWA/nAL3TGjYhUNtm5eUx9Zy0vLdlK20a1ePHHA/lO58CRizo1qpX5WTcW2PGuWBITE11tikUkFvx7zS7umZPGzkO53HphB34/tAu1q4e/Q7yZLXf3xJKWVch+9CIi0W7f4WM88OZq5n6ZSZcWdXnqpgvpH98oIrUo6EVEwsjdeWNlJg+8uYrDx/L59ZWduf3STlSvFrmOMwp6EZEw2X4wh7tnp/LRuj2c264hj93Qhy4t6kW6LAW9iMjZKix0/rXkKx59Zy2FDveO7MEtFyZQtbTzJ8uZgl5E5Cyk7z7MpOQUlm45wEWdmvLI9b1p17h2pMv6BgW9iMgZyCsoZPrCTTz5/gZqxlXh8Rv6cMN5bQm0+apYFPQiIqcpNSOLO5NSWLPjECN6t2TytT1pXq9mpMsqlYJeRCREOccL+PP76/nHok00rVuDZ8adx9CeLSNd1ikp6EVEQvDpxn1MSk5hy76jjB3QjkkjutOgVlykywqJgl5E5CQO5ebxyLy1vPL5VuIb1+bl2wZxYaemkS7rtCjoRURKsWD1Lu6Zk8qe7GOMv7gjv7myC7WqVz31AysYBb2ISDF7so8x+c1VvJ2yg24t6zF9XCJ92zWMdFlnTEEvIhLk7iSv2M6Ut1dz9FgBv7uqCz+95JyIti8IBwW9iAiwbf9R7pqdyqINezmvfSMeHdObTs0j374gHBT0IlKpFRQ6Mz/d8vV1Wh+4tifjzm9PlQrSviAcFPQiUmlt2JXNnUkpfLH1IJd0acbDo3vRtlHFal8QDqcMejNrB8wEWgKFwHR3f7LYnDuAm4o8Z3egmbvvN7MtQDZQAOSX1hhfRKS8HM8v5O8fbeSpD9OpXaMqT3yvL6P7tamQ7QvCIZQ9+nzgd+6+wszqAcvNbIG7rz4xwd0fBx4HMLNrgN+4+/4iz3GZu+8NZ+EiImdi5baDTJiVwrpd2Yzs04rJ1/akad0akS6rTJ0y6N19B7AjeDvbzNYAbQhcB7YkNwKvhK1CEZEwOHo8nyfeW8+MTzbTrF4N/vHDRK7q0SLSZZWL0zpGb2YJQD9gSSnLawPDgF8WGXbgPTNz4Bl3n17KY8cD4wHi4+NPpywRkZP6JH0vE5NT2LY/h/8aFM/E4d2oXzM62heEQ8hBb2Z1gSTg1+5+qJRp1wCfFDtsM9jdM82sObDAzNa6+8LiDwz+ApgOgYuDh/wTiIiUIutoHg/PW83ryzJIaFKbV8efz/kdm0S6rHIXUtCbWRyBkH/J3ZNPMnUsxQ7buHtm8PtuM5sNDAS+FfQiIuH0btoO7n1jFfuPHOenlwTaF9SMi772BeEQylk3BjwHrHH3J04yrwFwCfCDImN1gCrBY/t1gCHAg2ddtYhIKXZn53L/G6t4J20nPVrV55+3DKBXmwaRLiuiQtmjHwyMA1LNbGVw7C4gHsDdpwXHRgPvufuRIo9tAcwOnrJUDXjZ3d8NR+EiIkW5O/+7PIOH3lpNbn4hdwztyviLOxJXNbrbF4RDKGfdfAyc8uRSd38eeL7Y2Cag7xnWJiISkq37Au0LPk7fy4CERkwd04dzmtWNdFkVhj4ZKyJRq6DQ+ecnm/mf99ZTxWDKdb24aWB8TLUvCAcFvYhEpXU7s5mQlMLKbQe5vFtzHrquF60b1op0WRWSgl5Eosqx/AKe/nAjT3+UTr2acTw59lyu7ds6ZtsXhIOCXkSixoqtB5gwK4UNuw8z6tzW3DeyB01ivH1BOCjoRaTCO3Isnz++t47nF2+hZf2azLglkcu7VY72BeGgoBeRCm3h+j1MSk5l+8Ecxp3fnjuHdaVeJWpfEA4KehGpkA4ePc6Ut9aQtCKDjk3r8PpPL2Bgh8aRLisqKehFpEJxd+al7uT+uWkcOJrH7Zeew39f0bnSti8IBwW9iFQYuw7lcu+cNN5bvYteberzwo8G0rN15W5fEA4KehGJOHfntaXbeHjeGo7nFzJxeDduu6gD1dS+ICwU9CISUVv2HmFSciqfbtrHoA6NmTqmDx2a1ol0WTFFQS8iEZFfUMiMTzbzxIL1xFWpwsOje3HjALUvKAsKehEpd2t2HGJCUgopGVlc2b05U67rRasGal9QVhT0IlJujuUX8LcP0vn7RxtpUCuOv97Yj5F9Wql9QRlT0ItIuVi2ZT8TklLYuOcI1/dvw71X96BRneqRLqtSOOVb2mbWzsw+NLM1ZrbKzH5VwpxLzSzLzFYGv+4rsmyYma0zs3QzmxjuH0BEKrbDx/K5/400vvvMp+TmFfL8rQN44nvnKuTLUSh79PnA79x9hZnVA5ab2QJ3X11s3iJ3H1l0wMyqAk8BVwEZwFIzm1vCY0UkBn20bjd3z04jMyuHmy9I4PdDu1K3hg4klLdQrjC1A9gRvJ1tZmuANkAoYT0QSA9eaQozexUYFeJjRSRKHThynClvrSb5i+2c06wOs352Aee1V/uCSDmtX61mlgD0A5aUsPgCM/sSyAR+7+6rCPxC2FZkTgYwqJTnHg+MB4iPjz+dskSkgnB33krZweS5q8jKyeO/L+/ELy7vRI1qal8QSSEHvZnVBZKAX7v7oWKLVwDt3f2wmY0A5gCdKflas17S87v7dGA6QGJiYolzRKTi2pGVw71z0nh/zW76tG3Av24bRPdW9SNdlhBi0JtZHIGQf8ndk4svLxr87j7PzJ42s6YE9uDbFZnalsAev4jEiMJC55WlW5k6by15hYXcPaI7tw5OUPuCCuSUQW+BE1yfA9a4+xOlzGkJ7HJ3N7OBBM7m2QccBDqbWQdgOzAW+K9wFS8ikbVpz2EmJaeyZPN+LujYhKljetO+idoXVDSh7NEPBsYBqWa2Mjh2FxAP4O7TgBuAn5tZPpADjHV3B/LN7JfAfKAqMCN47F5Eolh+QSH/WLSZP72/nhrVqvDomN58L7GdPvhUQVkgjyuWxMREX7ZsWaTLEJESpG3PYkJSCqsyDzGkRwumXNeLFvVrRrqsSs/Mlrt7YknLdEKriIQkN6+AJ/+9gekLN9GodnWevqk/w3u11F58FFDQi8gpfb55PxOTUti09wg3nNeWe67uTsPa+mRrtFDQi0ipsnPzePTdtfzrs620bVSLmT8ayMVdmkW6LDlNCnoRKdEHa3dx9+w0dh7K5UeDO/C7IV2oo/YFUUn/aiLyDfsOH+OBN1cz98tMurSoy1M3XUj/+EaRLkvOgoJeRIBA+4I3VmbywJurOHwsn19f2ZnbL+1E9Wr64FO0U9CLCJkHc7h7diofrtvDue0a8tgNfejSol6ky5IwUdCLVGKFhc5LS75i6jtrKXS4d2QPbrkwgaq6bmtMUdCLVFIb9xxmYlIKS7cc4KJOTXnk+t60a1w70mVJGVDQi1QyeQWFTF+4iSf/vYGa1arw+A19uOG8tvrgUwxT0ItUIqkZWdyZlMKaHYcY0bslk6/tSfN6al8Q6xT0IpVAbl4Bf3p/Pc8u2kzjOtWZ9oPzGNarZaTLknKioBeJcZ9u3Mek5BS27DvK9xPbcdeI7jSoHRfpsqQcKehFYtSh3DwembeWVz7fSnzj2rx02yAGd2oa6bIkAhT0IjFowepd3DMnlT3Zx/jJdzrw26u6Uqu6rttaWSnoRWLInuxjTH5zFW+n7KBby3pMH5dI33YNI12WRFgolxJsB8wEWgKFwHR3f7LYnJuACcG7h4Gfu/uXwWVbgGygAMgvrTG+iJw5dyd5xXamvL2ao8cK+O1VXfjZJeeofYEAoe3R5wO/c/cVZlYPWG5mC9x9dZE5m4FL3P2AmQ0HpgODiiy/zN33hq9sETkh48BR7pqdxsL1e+gf35BHx/Shs9oXSBGnDHp33wHsCN7ONrM1QBtgdZE5i4s85DOgbZjrFJFiCgqdFz/dwmPz1wEw+ZoejLtA7Qvk207rGL2ZJQD9gCUnmfZj4J0i9x14z8wceMbdp5fy3OOB8QDx8fGnU5ZIpbNhVzYTklJYsfUgF3dpxh9G96JtI7UvkJKFHPRmVhdIAn7t7odKmXMZgaC/qMjwYHfPNLPmwAIzW+vuC4s/NvgLYDoELg5+Gj+DSKVxPL+Qaf/ZyN8+SKd2jao88b2+jO7XRu0L5KRCCnoziyMQ8i+5e3Ipc/oAzwLD3X3fiXF3zwx+321ms4GBwLeCXkRO7sttB5mQlMLandmM7NOKydf2pGndGpEuS6JAKGfdGPAcsMbdnyhlTjyQDIxz9/VFxusAVYLH9usAQ4AHw1K5SCWRc7yAJxas47mPN9OsXg3+8cNErurRItJlSRQJZY9+MDAOSDWzlcGxu4B4AHefBtwHNAGeDv4JeeI0yhbA7OBYNeBld383rD+BSAxbnL6XicmpbN1/lP8aFM/E4d2oX1PtC+T0hHLWzcfASQ8AuvttwG0ljG8C+p5xdSKVVFZOHn94ew2vLdtGQpPavDr+fM7v2CTSZUmU0idjRSqYd9N2ct8baew7cpyfXtKR31zZhZpxal8gZ05BL1JB7M7OZfLcVcxL3Un3VvV57uYB9G7bINJlSQxQ0ItEmLsza3kGD729hpy8Au4Y2pXxF3ckrqraF0h4KOhFImjb/qPcNTuVRRv2MiChEVPH9OGcZnUjXZbEGAW9SAQUFDrPL97CH+evo4rBlFE9uWlQe6qofYGUAQW9SDlbvyubO2elsHLbQS7r2oyHRvemTcNakS5LYpiCXqScHM8v5OmP0nnqw3Tq1qjGk2PP5dq+rdW+QMqcgl6kHKzYeoCJSSms33WYUee25r6RPWii9gVSThT0ImXoyLF8/vjeOp5fvIWW9Wsy45ZELu+m9gVSvhT0ImVk0YY9TEpOJeNADj84P54Jw7pRT+0LJAIU9CJhdvDocR56ew2zlmfQsWkdXht/PoPUvkAiSEEvEibuzjtpO7nvjVUcOHqc2y89h/++orPaF0jEKehFwmDXoVzunZPGe6t30atNfV740QB6tlb7AqkYFPQiZ8HdeW3pNh6et4bj+YVMHN6N2y7qQDW1L5AKREEvcoa+2neEScmpLN64j0EdGjN1TB86NK0T6bJEviWUK0y1A2YCLYFCYLq7P1lsjgFPAiOAo8At7r4iuOxm4J7g1Ifc/YXwlS9SPuZ8sZ3H568j82AOrRrUZEBCY+av3klclSo8PLoXNw6IV/sCqbBC2aPPB37n7ivMrB6w3MwWuPvqInOGA52DX4OAvwODzKwxcD+QCHjwsXPd/UBYfwqRMjTni+1MSk4lJ68AgMysXN74MpOerevz7M2JtGqg9gVSsZ3yQKK77zixd+7u2cAaoE2xaaOAmR7wGdDQzFoBQ4EF7r4/GO4LgGFh/QlEytjj89d9HfJFHThyXCEvUeG03jEyswSgH7Ck2KI2wLYi9zOCY6WNl/Tc481smZkt27Nnz+mUJVKmth/MKXF8R1ZuOVcicmZCDnozqwskAb9290PFF5fwED/J+LcH3ae7e6K7JzZr1izUskTKzJFj+Uyeu6rU5a3VcVKiREhBb2ZxBEL+JXdPLmFKBtCuyP22QOZJxkUqtI/W7WbInxbywqdb+E6nptSs9s3/KrXiqnLH0K6RKU7kNJ0y6INn1DwHrHH3J0qZNhf4oQWcD2S5+w5gPjDEzBqZWSNgSHBMpEI6cOQ4v31tJbf8cyk146ow62cX8OJtg5g6pg9tGtbCgDYNa/HI9b25rl+JRyFFKpxQzroZDIwDUs1sZXDsLiAewN2nAfMInFqZTuD0yluDy/ab2RRgafBxD7r7/vCVLxIe7s5bKTuYPHcVWTl5/L/LO/GLyzp93b7gun5tFOwStU4Z9O7+MSUfay86x4FflLJsBjDjjKoTKQc7s3K5Z04a76/ZRZ+2DfjXbYPo3qp+pMsSCRt9MlYqrcJC55WlW5k6by15hYXcPaI7tw5OUPsCiTkKeqmUNu89wsSkFJZs3s8FHZswdUxv2jdR+wKJTQp6qVTyCwp59uPN/GnBeqpXq8LU63vz/QHtdN1WiWkKeqk0VmVmMSEphbTthxjSowVTrutFi/o1I12WSJlT0EvMy80r4C//3sAzCzfRqHZ1nr6pP8N7tdRevFQaCnqJaZ9v3s/EpBQ27T3CDee15Z6ru9OwdvVIlyVSrhT0EpOyc/N47N11vPjZV7RtVIuZPxrIxV3UWkMqJwW9xJwP1u7i7tlp7DyUy48Gd+B3Q7pQp4Y2dam8tPVLzNh3+BgPvrWaN1Zm0rl5XZJ+fiH94xtFuiyRiFPQS9Rzd+Z+mckDb64mOzePX13RmdsvO4ca1apGujSRCkFBL1Et82AO98xJ44O1u+nbriGPjelD15b1Il2WSIWioJeoVFjovPT5Vh59Zy0Fhc69I3twy4UJVNV1W0W+RUEvUWfjnsNMSkrl8y37uahTUx65vjftGteOdFkiFZaCXqJGXkEh0xdu4sl/b6BmtSo8dkMfvnteW33wSeQUFPQSFdK2Z3HnrBRW7zjE8F4teWBUT5rXU/sCkVCcMujNbAYwEtjt7r1KWH4HcFOR5+sONAtedGQLkA0UAPnunhiuwqVyyM0r4E/vr+fZRZtpXKc6037Qn2G9WkW6LJGoEsoe/fPA34CZJS1098eBxwHM7BrgN8WuInWZu+89yzqlEvps0z4mJaeyee8Rvp/YjrtGdKdB7bhIlyUSdUK5wtRCM0sI8fluBF45m4JEDuXmMfWdtby8ZCvxjWvz0m2DGNypaaTLEolaYTtGb2a1gWHAL4sMO/CemTnwjLtPP8njxwPjAeLj48NVlkSZBat3cc+cVPZkH+O2izrw2yFdqF1dbyWJnI1w/g+6Bvik2GGbwe6eaWbNgQVmttbdF5b04OAvgekAiYmJHsa6JArsPXyMyXNX8VbKDrq1rMf0cYn0bdcw0mWJxIRwBv1Yih22cffM4PfdZjYbGAiUGPRSObk7s7/YzoNvrebosQJ+e1UXfnbJOVSvpuu2ioRLWILezBoAlwA/KDJWB6ji7tnB20OAB8PxehIbMg4c5a7ZaSxcv4f+8Q15dEwfOrdQ+wKRcAvl9MpXgEuBpmaWAdwPxAG4+7TgtNHAe+5+pMhDWwCzgx9mqQa87O7vhq90iVaFhc7MT7fw2Px1AEy+pgfjLlD7ApGyEspZNzeGMOd5AqdhFh3bBPQ908IkNqXvzmZCUirLvzrAxV2a8YfRvWjbSO0LRMqSTmeQcnE8v5Bn/rORv36QTu0aVXnie30Z3a+N2heIlAMFvZS5L7cdZEJSCmt3ZjOyTyvuv6YnzerViHRZIpWGgl7KTM7xAp5YsI7nPt5Ms3o1+McPE7mqR4tIlyVS6SjopUwsTt/LxORUtu4/yo0D45k0ohv1a6p9gUgkKOglrLJy8nhk3hpeXbqNhCa1eeUn53PBOU0iXZZIpaagl7CZv2on985JY+/hY/z0ko785sou1IzTdVtFIk1BL2dtd3Yuk+euYl7qTrq3qs9zNw+gd9sGkS5LRIIU9HLG3J1ZyzN46O015OQVcMfQroy/uCNxVdW+QKQiUdDLGdm2/yh3zU5l0Ya9JLZvxNQxfejUvG6kyxKREijo5bQUFDovLN7C4/PXUcVgyqie3DSoPVXUvkCkwlLQS8jW78rmzlkprNx2kMu6NuOh0b1p07BWpMsSkVNQ0MspHc8v5OmP0nnqw3Tq1qjGk2PP5dq+rdW+QCRKKOjlpL7YeoAJSSms33WYUee25r6RPWhSV+0LRKKJgl5KdPR4Pn+cv55/Lt5My/o1mXFLIpd3U/sCkWikoJdv+XjDXiYmp5BxIIcfnB/PhGHdqKf2BSJRS0EvX8s6msdDb6/mf5dn0LFpHV4bfz6DOqp9gUi0O+UnW8xshpntNrO0UpZfamZZZrYy+HVfkWXDzGydmaWb2cRwFi7h9U7qDq544j8kf7Gd2y89h3m/+o5CXiRGhLJH/zzwN2DmSeYscveRRQfMrCrwFHAVkAEsNbO57r76DGuVMrD7UC73vpHG/FW76Nm6Ps/fOoBebdS+QCSWhHIpwYVmlnAGzz0QSA9eUhAzexUYBSjoKwB35/Vl23jo7TUczy9kwrBu/OQ7Haim9gUiMSdcx+gvMLMvgUzg9+6+CmgDbCsyJwMYVNoTmNl4YDxAfHx8mMqSkny17wiTklNZvHEfAzs0Zur1venYTO0LRGJVOIJ+BdDe3Q+b2QhgDtAZKOnTNF7ak7j7dGA6QGJiYqnz5MwVFDozPt7M/yxYR7UqVXh4dC9uHBCv9gUiMe6sg97dDxW5Pc/MnjazpgT24NsVmdqWwB6/RMDanYeYMCuFLzOyuKJbcx4a3YtWDdS+QKQyOOugN7OWwC53dzMbSOBMnn3AQaCzmXUAtgNjgf8629eT03Msv4CnPkjn6Y820qBWHH+5sR/X9Gml9gUilcgpg97MXgEuBZqaWQZwPxAH4O7TgBuAn5tZPpADjHV3B/LN7JfAfKAqMCN47F7KyfKv9jMhKZX03YcZ3a8N947sQeM61SNdloiUMwtkcsWSmJjoy5Yti3QZUevIsXwen7+OFz7dQqv6NXn4+t5c1rV5pMsSkTJkZsvdPbGkZfpkbIz5z/o93JWcSmZWDj88vz13DOtG3Rr6ZxapzJQAMeLAkeNMeXs1ySu2c06zOvzvTy8gMaFxpMsSkQpAQR/l3J23UnbwwJurOHg0j/93eSd+cVknasZVjXRpIlJBKOij2M6sXO6Zk8b7a3bRp20DXvzxILq3qh/pskSkglHQR6HCQufVpdt4ZN4a8goLuXtEd24dnKD2BSJSIgV9lNm89wgTk1JYsnk/F3RswtQxvWnfpE6kyxKRCkxBHyXyCwp59uPN/GnBeqpXq8LU63vz/QHt9MEnETklBX0UWJWZxYSkFNK2H2JIjxZMua4XLerXjHRZIhIlFPQVWG5eAX/9YAPT/rOJRrXjePqm/gzv1VJ78SJyWhT0FdTSLfuZkJTCpj1HuOG8ttxzdXca1lb7AhE5fQr6CiY7N4/H3l3Hi599RdtGtZj5o4Fc3KVZpMsSkSimoK9APly7m7tnp7LjUC63Dk7g90O6UkftC0TkLClFKoD9R47z4JurmLMyk87N6zLrZxdyXvtGkS5LRGKEgj6C3J25X2bywJuryc7N41dXdOb2y86hRjW1LxCR8FHQR0jmwRzumZPGB2t307ddQx4b04euLetFuiwRiUGhXHhkBjAS2O3uvUpYfhMwIXj3MPBzd/8yuGwLkA0UAPml9UquTAoLnZc+38qj76yloNC55+ru3Dq4A1V13VYRKSOh7NE/D/wNmFnK8s3AJe5+wMyGE7jA96Aiyy9z971nVWWM2LTnMBOTUvl8y34Gd2rCI6P7EN+kdqTLEpEYd8qgd/eFZpZwkuWLi9z9jMBFwKWIvIJC/rFoE39+fwM1q1XhsRv68N3z2uqDTyJSLsJ9jP7HwDtF7jvwnpk58Iy7Ty/tgWY2HhgPEB8fH+ayIidtexZ3zkph9Y5DDO/VkgdG9aR5PbUvEJHyE7agN7PLCAT9RUWGB7t7ppk1BxaY2Vp3X1jS44O/BKZD4Jqx4aorUnLzCvjz+xv4x6JNNK5TnWk/6M+wXq0iXZaIVEJhCXoz6wM8Cwx3930nxt09M/h9t5nNBgYCJQZ9LPls0z4mJaeyee8Rvp/YjrtGdKdB7bhIlyUildRZB72ZxQPJwDh3X19kvA5Qxd2zg7eHAA+e7etVZIdy85j6zlpeXrKV+Ma1eem2QQzu1DTSZYlIJRfK6ZWvAJcCTc0sA7gfiANw92nAfUAT4Ongm4snTqNsAcwOjlUDXnb3d8vgZ6gQ3l+9i3vmpLE7O5fbLurAb4d0oXZ1fUxBRCIvlLNubjzF8tuA20oY3wT0PfPSosPew8eYPHcVb6XsoGuLekwbdx7ntmsY6bJERL6mXc4z5O7M/mI7D761miPH8vntVV342SXnUL2artsqIhWLgv4MZBw4yt2z0/jP+j30j2/Io2P60LmF2heISMWkoD8NhYXOi599xTnp/YkAAAixSURBVKPvrgXg/mt68MMLEtS+QEQqNAV9iNJ3ZzMhKZXlXx3gO52b8ofRvWnXWO0LRKTiU9CfQl5BIdM+2shfP0inVvWq/M93+3J9/zZqXyAiUUNBfxIpGQe5c1YKa3dmc3WfVky+pifN6tWIdFkiIqdFQV+CnOMF/On99Ty7aBPN6tVg+rjzGNKzZaTLEhE5Iwr6YhZv3Muk5FS+2neUGwe2Y+Lw7jSopfYFIhK9FPRBWTl5PDJvDa8u3UZCk9q8/JNBXHiO2heISPRT0APzV+3k3jlp7D18jJ9e0pHfXNmFmnG6bquIxIZKHfR7sgPtC95O3UH3VvV57uYB9G7bINJliYiEVaUMencnacV2pry1mpy8Au4Y2pXxF3ckrqraF4hI7Kl0Qb9t/1Hump3Kog17SWzfiKlj+tCped1IlyUiUmYqTdAXFDovLN7C4/PXUcVgyqie3DSoPVXUvkBEYlylCPr1u7K5c1YKK7cd5NKuzXh4dG/aNKwV6bJERMpFSAelzWyGme02s7RSlpuZ/cXM0s0sxcz6F1l2s5ltCH7dHK7CQ3E8v5A/v7+eq/+yiK/2HeHP3z+Xf94yQCEvIpVKqHv0zwN/A2aWsnw40Dn4NQj4OzDIzBoTuCJVIuDAcjOb6+4Hzqbo0sz5YjuPz19H5sEcmtatQbWqxo6sXK7t25r7r+lBk7pqXyAilU9IQe/uC80s4SRTRgEz3d2Bz8ysoZm1InAJwgXuvh/AzBYAw4BXzqboksz5YjuTklPJySsAYM/hYwD85DsduPvqHuF+ORGRqBGu8wnbANuK3M8IjpU2HnaPz1/3dcgXNS91Z1m8nIhI1AhX0Jd06oqfZPzbT2A23syWmdmyPXv2nHYBmQdzTmtcRKSyCFfQZwDtitxvC2SeZPxb3H26uye6e2KzZs1Ou4DWpbzBWtq4iEhlEa6gnwv8MHj2zflAlrvvAOYDQ8yskZk1AoYEx8LujqFdqVWsP02tuKrcMbRrWbyciEjUCOnNWDN7hcAbq03NLIPAmTRxAO4+DZgHjADSgaPArcFl+81sCrA0+FQPnnhjNtyu6xc49H/irJvWDWtxx9CuX4+LiFRWFjhRpmJJTEz0ZcuWRboMEZGoYWbL3T2xpGXq4iUiEuMU9CIiMU5BLyIS4xT0IiIxTkEvIhLjFPQiIjGuQp5eaWZ7gK/O4imaAnvDVE44qa7QVcSaQHWdjopYE8RuXe3dvcS2AhUy6M+WmS0r7XzSSFJdoauINYHqOh0VsSaonHXp0I2ISIxT0IuIxLhYDfrpkS6gFKordBWxJlBdp6Mi1gSVsK6YPEYvIiL/J1b36EVEJEhBLyIS46Iq6M1shpntNrO0Upabmf3FzNLNLMXM+hdZdrOZbQh+3VzOdd0UrCfFzBabWd8iy7aYWaqZrTSzsPZmDqGuS80sK/jaK83sviLLhpnZuuC6nFiONd1RpJ40Mysws8bBZWW5rtqZ2YdmtsbMVpnZr0qYU67bV4g1lfu2FWJdkdi2Qqmr3LcvM6tpZp+b2ZfBuh4oYU4NM3stuE6WmFlCkWWTguPrzGzoGRXh7lHzBVwM9AfSSlk+AniHwLVqzweWBMcbA5uC3xsFbzcqx7ouPPF6wPATdQXvbwGaRmh9XQq8VcJ4VWAj0BGoDnwJ9CiPmorNvQb4oJzWVSugf/B2PWB98Z+5vLevEGsq920rxLoisW2dsq5IbF/B7aVu8HYcsAQ4v9ic24FpwdtjgdeCt3sE11ENoENw3VU93Rqiao/e3RcCJ7tC1Shgpgd8BjQ0s1bAUGCBu+939wPAAmBYedXl7ouDrwvwGYFr55a5ENZXaQYC6e6+yd2PA68SWLflXdONwCvheN1Tcfcd7r4ieDsbWAMUvzxZuW5fodQUiW0rxHVVmrLctk63rnLZvoLby+Hg3bjgV/GzYEYBLwRvzwKuMDMLjr/q7sfcfTOBq/gNPN0aoiroQ9AG2FbkfkZwrLTxSPgxgb3CExx4z8yWm9n4CNRzQfBPynfMrGdwLOLry8xqEwjLpCLD5bKugn829yOw51VUxLavk9RUVLlvW6eoK2Lb1qnWV3lvX2ZW1cxWArsJ7BSUum25ez6QBTQhTOsrpGvGRhErYcxPMl6uzOwyAv8ZLyoyPNjdM82sObDAzNYG93rLwwoC/TEOm9kIYA7QmYqxvq4BPvFvXmO4zNeVmdUl8J//1+5+qPjiEh5S5tvXKWo6Mafct61T1BWxbSuU9UU5b1/uXgCca2YNgdlm1svdi75PVabbVqzt0WcA7YrcbwtknmS83JhZH+BZYJS77zsx7u6Zwe+7gdmcwZ9lZ8rdD534k9Ld5wFxZtaUCrC+CByn/Maf1WW9rswsjkBAvOTuySVMKfftK4SaIrJtnaquSG1boayvoHLfvoLPfRD4iG8f2vt6vZhZNaABgUOc4Vlf4X7joay/gARKf3Pxar75ZtnnwfHGwGYCb5Q1Ct5uXI51xRM4tnZhsfE6QL0itxcDw8qxrpb834fmBgJbg+uuGoE3FDvwf2+Y9SyPmoLLT2zkdcprXQV/7pnAn08yp1y3rxBrKvdtK8S6yn3bCqWuSGxfQDOgYfB2LWARMLLYnF/wzTdjXw/e7sk334zdxBm8GRtVh27M7BUC7+Y3NbMM4H4Cb2zg7tOAeQTOjEgHjgK3BpftN7MpwNLgUz3o3/yTrazruo/A8banA++vkO+BLnUtCPwZB4H/AC+7+7vlWNcNwM/NLB/IAcZ6YOvKN7NfAvMJnCUxw91XlVNNAKOB99z9SJGHlum6AgYD44DU4LFUgLsIBGmktq9QaorEthVKXeW+bYVYF5T/9tUKeMHMqhI4ivK6u79lZg8Cy9x9LvAc8KKZpRP4JTQ2WPMqM3sdWA3kA7/wwGGg06IWCCIiMS7WjtGLiEgxCnoRkRinoBcRiXEKehGRGKegFxGJcQp6EZEYp6AXEYlx/x/A7ImP/Y+jMgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Data\n",
    "plt.scatter(x_train, y_train)\n",
    "# Best-fit line\n",
    "xs = np.linspace(1, 3, 1000)\n",
    "plt.plot(xs, xs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jblHIh7Zg_md"
   },
   "source": [
    "## Cost by W"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Q6GXN7TCg_md"
   },
   "source": [
    "$$ H(x) = Wx $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "nQ44HY6hg_me"
   },
   "outputs": [],
   "source": [
    "W_l = np.linspace(-5, 7, 1000)\n",
    "cost_l = []\n",
    "for W in W_l:\n",
    "    hypothesis = W * x_train\n",
    "    cost = torch.mean((hypothesis - y_train) ** 2)\n",
    "\n",
    "    cost_l.append(cost.item())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "XY-t2-wtg_me",
    "outputId": "915cc4cd-4d78-4eca-97c2-0b57b8e97e1d"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAELCAYAAAA2mZrgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd3zV5dnH8c+VTQYjJEAMCWGFIZuwFBwoiqPgQnHi40Dbal2trXbZYevTVlutrduKiihuFDdOlBWQPQMkkBBCCISEhOzr+SOHPCkGSUJO7nNyrvfrdV7J+Z1zcr6H1nzzG/d9i6pijDHGAAS5DmCMMcZ3WCkYY4ypY6VgjDGmjpWCMcaYOlYKxhhj6lgpGGOMqeO1UhCRZ0Vkj4isrbftFRFZ6bllishKz/YUETlU77HHvZXLGGPM0YV48Wc/BzwKPH94g6pedvh7EXkQOFDv+VtVdZgX8xhjjDkGr5WCqn4pIikNPSYiAlwKTDye94iLi9OUlAbfwhhjzFEsX758r6rGN/SYN/cUvs8EIE9Vt9Tb1lNEvgWKgF+p6lfH+iEpKSmkp6d7K6MxxrRJIpJ1tMdclcLlwJx693OBZFUtEJGRwFsicqKqFh35QhGZCcwESE5ObpWwxhgTKFr96iMRCQEuAl45vE1Vy1W1wPP9cmArkNrQ61X1SVVNU9W0+PgG936MMcY0k4tLUs8ENqpq9uENIhIvIsGe73sBfYFtDrIZY0xA8+YlqXOARUA/EckWkes9D03nvw8dAZwCrBaRVcBrwM2qus9b2YwxxjTMm1cfXX6U7dc2sO114HVvZTHGGNM4NqLZGGNMHSsFY4wxdQKyFHIPHOL++evZe7DcdRRjjGmyZxdu56N1u73yswOyFIrLqnjqq+28uSLHdRRjjGmSg+VV/O2jTSzYsMcrPz8gSyG1awwjkjsyZ9kObI1qY4w/mb96F6UV1Vw6KskrPz8gSwFg+uhktuWXkJ6133UUY4xptFeW7aR3fBQjkjt65ecHbCmcPySB6PAQ5izd4TqKMcY0yua8YlbsKGT6qGRq5xVteQFbCpFhIUwZdgLvrcnlwKFK13GMMeaY5izdQVhwEBeP7O619wjYUgC4fFQyZZU1zFtpJ5yNMb6trLKaN7/N4awTuxIbFea19wnoUhiU2J6BCe2Zs3SnnXA2xvi0D9ftprC0kstHe3d26IAuBRHh8tFJrM8tYm3Od2bpNsYYn/HSkh0kx0Yyrldnr75PQJcCwJRhiUSEBjFnmZ1wNsb4pm35B1myfR+XjUoiKMg7J5gPC/hS6NAulHMHJzBv5S5KK6pcxzHGmO94ZdlOgoOEaV48wXxYwJcCwPRRyRwsr+Ld1bmuoxhjzH+pqKrhteXZnNG/C13aR3j9/awUgFEpnegdH8Ury3a6jmKMMf/lkw15FJRUeP0E82FWCtSecJ4+KpnlWfvZnFfsOo4xxtSZs3QHJ3SI4JTU1ll+2ErB46IRiYQGi+0tGGN8xs59pSzM2Mu0tCSCvXyC+TArBY/O0eGcNbAbb6zIpryq2nUcY4xhbnrtH6nemvyuIVYK9Vw2Kon9pZV8uC7PdRRjTICrqq5hbvpOTk2NJ7Fju1Z7XyuFesb3iSM5NpLZi7NcRzHGBLjPN+WTV1TO9FGtc4L5MCuFeoKChCvGJLNk+z622AlnY4xDLy3dQXxMOGcM6NKq7+u1UhCRZ0Vkj4isrbftPhHJEZGVntu59R67R0QyRGSTiJztrVzHMm1kd8KCg5i9xEY4G2Pc2LmvlM827eHyUUmEBrfu3+7efLfngMkNbP+7qg7z3N4DEJGBwHTgRM9r/i0iwV7MdlSdo8M5Z3A3Xl+RbSOcjTFOvLR0B0LtYmCtzWuloKpfAvsa+fSpwMuqWq6q24EMYLS3sh3LVWN7UFxWxTurdrmKYIwJUOVV1cxdtpMzBnTlhFY8wXyYi3MKt4jIas/hpU6ebYlA/QEC2Z5t3yEiM0UkXUTS8/PzvRIwrUcnUrtG8+JiO4RkjGldH6zdTUFJBVeN7eHk/Vu7FB4DegPDgFzgQc/2hkZlNLjAgao+qappqpoWH++dEX4iwlVje7Am5wCrswu98h7GGNOQ2Yt30KNzJBP6xDl5/1YtBVXNU9VqVa0BnuL/DxFlA/VHZ3QHnB67uXB4IpFhwbxol6caY1rJpt3FLM3cx5Vjkr0+RfbRtGopiEhCvbsXAoevTJoHTBeRcBHpCfQFlrZmtiPFRIQydVgi81bt4kCpreFsjPG+FxdnERYSxLSRrTeC+UjevCR1DrAI6Cci2SJyPfAXEVkjIquB04E7AFR1HTAXWA98APxYVZ3PNXHlmNo1nF9fke06ijGmjSspr+LNb3M4f3ACnby4BvOxhHjrB6vq5Q1sfuZ7nn8/cL+38jTHoMQODEvqyOwlWfzPySmIuNmdM8a0fW+tzOFgeRVXOjrBfJiNaD6Gq8b2YGt+CYu3NfbqWmOMaRpV5cXFOxiQ0J4RyR2dZrFSOIbzhyTQoV0oLy6xE87GGO9YsaOQDblFXDU22fkRCSuFY4gIDeaSkd35cO1u9hSXuY5jjGmDZi/OIjo8hAuGNTg8q1VZKTTCVWN7UFWjzFliC/AYY1rWvpIK3l2dy0UjEokK99pp3kazUmiEnnFRnJoaz+wlWVRU1biOY4xpQ+am76SiuoYrx7g9wXyYlUIjXXtyCnuKy3l/ba7rKMaYNqKquoYXFmUxrldn+nWLcR0HsFJotFP7xtMzLopZ32S6jmKMaSM+2ZBHTuEhrj05xXWUOlYKjRQUJFw9tgcrdhSyJvuA6zjGmDbguW8ySezYjjMHdHUdpY6VQhNcktadyLBgnrO9BWPMcdqQW8Tibfu4ZlwPgh3Nc9QQK4UmaB8RysUjuvPO6l0UHCx3HccY48eeX5RJRGgQl41yN89RQ6wUmmjGST2oqKrh5WV2eaoxpnn2l1Tw5rc5XDg8kY6R7uY5aoiVQhP16RLD+D5xvLAoi8pquzzVGNN0r6TvpKyyhhknpbiO8h1WCs0w46QUdheV8dG6PNdRjDF+pv5lqP27tXcd5zusFJphYv8uJMW2s8tTjTFN9smGPeQUHvLJvQSwUmiW4CDhmrEpLM3cx/pdRa7jGGP8yKy6y1C7uI7SICuFZro0LYl2ocG2t2CMabSNu4tYtK2Aa8b1ICTYN3/9+mYqP9AhMpQLhify1socuzzVGNMos77xzctQ67NSOA7XnZxCeVUNLy3Z4TqKMcbHFZb67mWo9VkpHIe+XWM4rV88sxZlUV7lfElpY4wPm71kB2WVNVx7Uk/XUb6XlcJxumF8L/YeLGfeyl2uoxhjfFRFVQ2zvslkQt84n5kN9Wi8Vgoi8qyI7BGRtfW2/VVENorIahF5U0Q6eraniMghEVnpuT3urVwt7eQ+nenfLYZnFm5HVV3HMcb4oHdW7WJPcTk3TujlOsoxeXNP4Tlg8hHbPgYGqeoQYDNwT73HtqrqMM/tZi/malEiwvXje7JxdzFfZxS4jmOM8TGqytMLt5PaNZoJfeNcxzkmr5WCqn4J7Dti20eqWuW5uxjo7q33b01Thp1AXHQ4Ty/c5jqKMcbHLNpawIbcIm4Y3wsR35kN9WhcnlO4Dni/3v2eIvKtiHwhIhNchWqO8JBgZozrweeb8tmSV+w6jjHGhzz11TbiosOZOvwE11EaxUkpiMgvgSpgtmdTLpCsqsOBO4GXRKTBSUFEZKaIpItIen5+fusEboQrx/YgPCSIZ7/e7jqKMcZHZOwp5rNN+VwzrgfhIcGu4zRKq5eCiMwAzgeuVM+ZWVUtV9UCz/fLga1AakOvV9UnVTVNVdPi4+NbK/YxxUaFcdGI7ry+wgazGWNqPbNwO+EhQVw5Jtl1lEZr1VIQkcnAz4Epqlpab3u8iAR7vu8F9AX87gD99eNTqKiqYbYNZjMm4BUcLOf1FTlcPLI7naPDXcdpNG9ekjoHWAT0E5FsEbkeeBSIAT4+4tLTU4DVIrIKeA24WVX3NfiDfVifLjGc3i+e5xdlUlZpg9mMCWQvLt5BRVUN153s24PVjhTirR+sqpc3sPmZozz3deB1b2VpTTdM6MWVTy9h3qpdXJrmu/ObGGO8p6yymhcWZzKxfxf6dIl2HadJbERzCzupd+1gtqe/2kZNjQ1mMyYQvb0yh70HK7hhgn/tJYCVQosTEW46tReb8w7y2aY9ruMYY1pZTY3yxJfbGJjQnnG9OruO02RWCl5w/pATSOzYjse/2Oo6ijGmlX28IY9t+SXcfFpvvxisdiQrBS8IDQ7ihgk9WZa5n+VZfne+3BjTTKrKY59vJSm2HecO6uY6TrNYKXjJZaOS6BQZymOf+92VtcaYZlqyfR8rdxYy85TePruy2rH4Z2o/EBkWwjXjUvhkQx6bbeoLYwLC419sJS46jGkj/XdaNysFL5pxUgoRoUE88YXtLRjT1q3fVcTnm/K59qQUIkL9Y0qLhlgpeFFsVBjTRyXz9socdhUech3HGONFT3y5laiwYK4em+I6ynGxUvCy68f3RKmdA8UY0zbt3FfKu6tzuWJMMh0iQ13HOS5WCl6WFBvJD4YkMGfpDgpLK1zHMcZ4wVNfbSNI4Prxvr+y2rFYKbSCm07tTWlFNS8synIdxRjTwgoOljM3fScXDk+kW4cI13GOm5VCKxiQ0J7T+sXz3DeZHKqwifKMaUtmfZNJeVUNM0/p7TpKi7BSaCU/PLU3BSUVvLLMptU2pq04WF7FrEVZTBrQ1e8mvjsaK4VWMqZXZ0anxPLEl9sor7K9BWPaghcXZ3HgUCU/Or2P6ygtxkqhFd0ysQ+5B8p4fXmO6yjGmON0qKKap7/axoS+cQxL6ug6TouxUmhFE/rGMTSpI//+PIPK6hrXcYwxx2HO0h3sPVjBrRP7uo7SoqwUWpGIcOvpfcjef4h5K3e5jmOMaaayymqe+HIrY3rGMrpnrOs4LcpKoZWdMaALAxLa86/PMqi2RXiM8UuvLs8mr6icn5zRtvYSwEqh1YkIt07sw7a9Jby3Jtd1HGNME1VU1fD451sZntyRk3r73yI6x2Kl4MDkE7vRp0s0j36aYUt2GuNn3vo2h5zCQ/xkYl+/XETnWLxaCiLyrIjsEZG19bbFisjHIrLF87WTZ7uIyCMikiEiq0VkhDezuRQUJNxyeh825RXz8YY813GMMY1UVV3Dvz7PYHBiB07rF+86jld4e0/hOWDyEdt+ASxQ1b7AAs99gHOAvp7bTOAxL2dz6vwhCaR0juTRTzNQtb0FY/zBO6t3kVVQyi0T+7TJvQTwcimo6pfAketRTgVmeb6fBVxQb/vzWmsx0FFEEryZz6WQ4CB+dFof1uQc4PPN+a7jGGOOobpGefTTDPp1jWHSgK6u43iNi3MKXVU1F8DztYtneyKws97zsj3b2qwLRySS2LEdjyzYYnsLxvi499fmsjW/hFsm9iEoqG3uJYBvnWhu6F/5O78pRWSmiKSLSHp+vn//hR0aHMQtE/vw7Y5C21swxodV1yiPLNhCr/gozh3cZg9gAG5KIe/wYSHP1z2e7dlAUr3ndQe+M8JLVZ9U1TRVTYuP9/8TPZeM7E5SbDv+/vFm21swxkfNX5PL5ryD3H5mKsFteC8B3JTCPGCG5/sZwNv1tl/juQppLHDg8GGmtiw0OIhbJ/ZldfYBPtmw59gvMMa0qqrqGv7xyWZSu0ZzfhvfSwDvX5I6B1gE9BORbBG5HngAmCQiW4BJnvsA7wHbgAzgKeBH3szmSy4ankhK50ge+nizjVswxsfMW7WLbfkl3HFmaps+l3BYiDd/uKpefpSHzmjguQr82Jt5fFVIcBC3ndmXO15ZxYfrdnNOAPw1Yow/qKyu4eEFWxiY0J6zT+zmOk6r8KUTzQFtytBEesdH8fdPNtucSMb4iDdX5JBVUMqdkwJjLwGsFHxGcJBw+5mpbM47yHybE8kY5yqqavcShnbvwBkDuhz7BW2ElYIPOW9wAqldo/nHJ5upsvUWjHFqbvpOcgoPccek1DY7erkhVgo+JChIuOPMVLbllzBvla23YIwrZZXV/OuzDEb26MSpqf5/6XtTWCn4mLNP7MbAhPY8vGCLrc5mjCMvL91B7oEy7gywvQRoZCmIyAuN2WaOX1CQcMekVLIKSnltebbrOMYEnNKKKh79rHZVtba4XsKxNHZP4cT6d0QkGBjZ8nEMwJkDujAiuSP/+GQzhyqqXccxJqA8u3A7ew+Wc/fk/gG3lwDHKAURuUdEioEhIlLkuRVTOzXF29/3WtN8IsLPJ/cnr6ic577JdB3HmICxv6SCJ77YxqSBXRnZo5PrOE58bymo6p9VNQb4q6q299xiVLWzqt7TShkD0phenTm9XzyPfZ7BgdJK13GMCQj/+iyDkooq7j67n+sozjT28NG7IhIFICJXichDItLDi7kMcPfk/hSXV/HYF1tdRzGmzcspPMTzi7K4eER3+naNcR3HmcaWwmNAqYgMBe4GsoDnvZbKADAgoT1Th57Af77ezu4DZa7jGNOm/f3jzSBw+6RU11GcamwpVHnmJpoKPKyqDwOBW6Wt6K6z+lGjysMLNruOYkybtTmvmDdWZHPN2B4kdmznOo5TjS2FYhG5B7gamO+5+ijUe7HMYUmxkVw5pgdz07PZmn/QdRxj2qS/friJqLAQfnx6H9dRnGtsKVwGlAPXqepuapfJ/KvXUpn/csvEPoSHBPHgR5tcRzGmzVmetY+P1+dx06m96BQV5jqOc40qBU8RzAY6iMj5QJmq2jmFVhIXHc4NE3rx3prdrNxZ6DqOMW2GqvK/728iLjqc68b3dB3HJzR2RPOlwFJgGnApsERELvFmMPPfbpzQk7joMP40f4Mt22lMC/lofR5LM/dx25l9iQzz6vIyfqOxh49+CYxS1Rmqeg0wGvi192KZI8VEhHL7makszdzHh+t2u45jjN+rqKrhz+9toE+XaC4flXTsFwSIxpZCkKrWX0C4oAmvNS1k+qgk+naJ5s/vb6SiyibLM+Z4vLg4i8yCUn557gBCgu3X2WGN/Zf4QEQ+FJFrReRaYD61ayqbVhQSHMS95w0gq6CU5xdluo5jjN8qLK3g4QVbmNA3jtP6BdbU2MdyrLmP+ojIyar6M+AJYAgwFFgEPNkK+cwRTkuNZ0LfOP75aQaFpRWu4xjjl/75aQZFZZXce+6AgJz07vsca0/hH0AxgKq+oap3quod1O4l/MPb4cx3iQi/PG8AxWWVPLxgi+s4xvidzL0lPL8ok8vSkhiQ0N51HJ9zrFJIUdXVR25U1XQgpTlvKCL9RGRlvVuRiNwuIveJSE697ec25+cHgv7d2nPZqCReWJTFNhvQZkyTPPD+RkKDg7jzrMCezuJojlUKEd/zWLPGgqvqJlUdpqrDqF2ToRR40/Pw3w8/pqp2zuJ73DEplfCQIB54f6PrKMb4jSXbCvhg3W5+eGpvusR836+3wHWsUlgmIjceuVFErgeWt8D7nwFsVdWsFvhZAaVLTAQ/PK03H63PY/G2AtdxjPF5NTXK/e9tIKFDBDdM6OU6js86VincDvyPiHwuIg96bl8ANwC3tcD7Twfm1Lt/i4isFpFnRSQwV7hoghsm9OKEDhH8/p31VNfYgDZjvs/rK7JZnX2An53dj3Zhwa7j+KxjLbKTp6onAb8DMj2336nqOM/UF80mImHAFOBVz6bHgN7AMCAXePAor5spIukikp6fn388EfxeRGgw9543gPW5Rby0dIfrOMb4rKKySv73g42MSO7IBcMSXcfxaY2d++gzVf2n5/ZpC733OcAKVc3zvEeeqlarag3wFLWjphvK8qSqpqlqWny8XV983uAExvXqzIMfbWJ/iV2iakxDHvlkCwUlFfxuyiCCguwS1O/jchjf5dQ7dCQiCfUeuxBY2+qJ/JCIcN+UEykuq+JvNouqMd+RsaeY577JZPqoJAZ37+A6js9zUgoiEglMAt6ot/kvIrJGRFYDpwN3uMjmj/p1i+HqsT14aekO1uYccB3HGJ+hqtw3bz2RYcH89KzAXXe5KZyUgqqWqmpnVT1Qb9vVqjpYVYeo6hRVzXWRzV/dMSmV2Mgw7pu3zmZRNcbjw3V5LMzYy11n9aNzdLjrOH7BZoFqIzq0C+Xuyf1Iz9rPWytzXMcxxrmyymr+OH89/bvFcOWYZNdx/IaVQhsybWQSQ7t34M/vbeRgeZXrOMY49cQX28jef4j7ppxos6A2gf1LtSFBQbUnnfcUl/OIzYtkAtiOglL+/XkG5w9JYGyvzq7j+BUrhTZmeHInLktL4tmF29m4u8h1HGNanary67fXEhocxK/OG+g6jt+xUmiDfnFOf9q3C+XeN9ZQYyOdTYB5b81uvticz11npdKtg81v1FRWCm1Qp6gw7j13ACt2FPLysp2u4xjTaorKKvndO+sYlNiea8aluI7jl6wU2qiLRyQytlcsD7y/gfzictdxjGkVD320mfyD5fzpwsEE28jlZrFSaKNEhD9eMJhDldXcP3+96zjGeN3q7EJmLcrkmrE9GNK9o+s4fstKoQ3r0yWaH57am7dW7mLhlr2u4xjjNdU1yr1vriE+Opy7zraRy8fDSqGN+9HpfUjpHMmv315LWWW16zjGeMXzizJZm1PEb34wkPYRoa7j+DUrhTYuIjSYP14wmO17S/jXZxmu4xjT4nIKD/G3DzdxSmo85w1OOPYLzPeyUggA4/vGceHwRB77fCvrd9nYBdN2qCr3vLEGBe6/YBAidnL5eFkpBIjfnD+QjpGh3P36Kqqqa1zHMaZFvL4ihy835/Pzyf1Jio10HadNsFIIEJ2iwvj91EGszSniya+2uY5jzHHbU1zGH95dz6iUTlw9tofrOG2GlUIAOXdwAucM6sY/PtlCxp6DruMY02yqyq/fWsuhymoeuHiIrabWgqwUAszvpp5IZFgwd7+2imqbAsP4qffW7ObDdXnccWYqveOjXcdpU6wUAkyXmAh++4OBrNhRyKxvMl3HMabJ9pdU8Nt5axmc2IEbJ/R0HafNsVIIQBcMS+T0fvH89cNN7CgodR3HmCb53TvrKCyt5H8vHmLrJHiB/YsGIBHhTxcNJiRI+OmrdhjJ+I/31uTy1spd/Pj0Pgw8ob3rOG2SlUKASujQjvumnMjSzH08bVcjGT+wp6iMX765hiHdO3DLxD6u47RZzkpBRDJFZI2IrBSRdM+2WBH5WES2eL52cpUvEFw0IpHJJ3bjwY82syHXBrUZ36Wq/Pz11ZRWVPPQpcMItcNGXuP6X/Z0VR2mqmme+78AFqhqX2CB577xksOHkdq3C+WOV1ZSXmVzIxnfNGfpTj7blM8vzulPny52tZE3uS6FI00FZnm+nwVc4DBLQIiNCuMvlwxm4+5iHvp4s+s4xnxHVkEJf5y/npP7dGaGLZzjdS5LQYGPRGS5iMz0bOuqqrkAnq9dnKULIBP7d+WKMck8+eU2lmwrcB3HmDrVNcpdc1cRHCT89ZKhNkitFbgshZNVdQRwDvBjETmlMS8SkZkiki4i6fn5+d5NGEB+ee4AkmMjuevVVRSXVbqOYwwAj3+xlfSs/fxh6iBO6NjOdZyA4KwUVHWX5+se4E1gNJAnIgkAnq97Gnjdk6qapqpp8fHxrRm5TYsKD+GhS4exq/AQv35rLap2mapxa8WO/Tz08WbOG5LA1GEnuI4TMJyUgohEiUjM4e+Bs4C1wDxghudpM4C3XeQLVCN7dOKOM1N5a+UuXlue7TqOCWAHDlXykznfktAhgj9fNNimxG5FIY7etyvwpud/6BDgJVX9QESWAXNF5HpgBzDNUb6A9aPT+/DN1gJ+8/Y6hid3sis9TKtTrV1aM/dAGa/ePM5WUmtlTvYUVHWbqg713E5U1fs92wtU9QxV7ev5us9FvkAWHCT8Y/ow2oUFc8tLK2wJT9PqXlm2k/mrc7nrrFRGJNtQpdbma5ekGh/QtX0ED04bysbdxdw/f4PrOCaAbMkr5r531jG+Txw3n9LbdZyAZKVgGnR6/y7cOKEnLyzO4oO1ua7jmABQVlnNrXO+JSoshIcutctPXbFSMEf1s7P7M7R7B+5+bTU799lsqsZ7VJVfvbWWjbuLefDSoXRpH+E6UsCyUjBHFRYSxD8vH4ECP5y93M4vGK95edlOXluezU8m9uG0fjZm1SUrBfO9kjtH8vdLh7E2p4jfvG3jF0zLW51dyG/fXseEvnHcdmaq6zgBz0rBHNOZA7ty68Q+zE3P5uVlO13HMW3I/pIKfvjiCuJjwnl4+nCC7TyCc1YKplFuPzOVCX3j+O3b61i1s9B1HNMGVNcot7+ykvzicv595Qhio8JcRzJYKZhGCg4SHpk+nPiYcH744nL2lVS4jmT83CMLtvDF5nx+O2UgQ5M6uo5jPKwUTKN1igrjsatGsLekglvnrKCyusZ1JOOn3l+Ty8MLtnDJyO5cMTrZdRxTj5WCaZIh3Tty/wWD+DqjgD+8u951HOOH1u06wJ1zVzE8uSN/vGCQzWvkY1zNfWT82LS0JLbsOciTX26jb9cYrh7bw3Uk4yfyi8u5cVY6HSNDeeLqkUSEBruOZI5gewqmWX4+uT9n9O/CffPW8XXGXtdxjB8or6rm5heXs6+0gqeuSaNLjA1Q80VWCqZZDk+c1zs+ih/NXsH2vSWuIxkfpqr86s21LM/az9+mDWVQYgfXkcxRWCmYZouJCOWZGaMIDhKuf24ZB0ptxTbTsCe+3MarnhHL5w+xBXN8mZWCOS5JsZE8ftVIdu4vZeYL6ZRX2VQY5r+9vTKHB97fyHlDErjdRiz7PCsFc9xG94zlb9OGsmT7Pu6cu4qaGpsKw9RavK2An726mtE9Y3lwms186g/s6iPTIqYOSySvqIw/vbeRbu0j+PX5A11HMo5tzitm5vPpJHeO5Kmr0+xKIz9hpWBazI0TepF7oIxnFm4noUMEN0zo5TqScSSvqIxrn11KeGgwz/3PKDpE2pKa/sJKwbQYEeHX5w0kr6iMP87fQHxMOFOHJbqOZVpZYWkF1zyzlAOHKnnlpnF07xTpOpJpAjunYFpUUJDw0KXDGJ0Sy09fXcVnG/e4jo94wAQAAA37SURBVGRa0cHyKmb8Zxnb95bw5DVpdumpH7JSMC0uIjSYp2ak0a9bDDe/uJxvttrgtkBQVlnNjbPSWZtzgEevGM7JfeJcRzLN0OqlICJJIvKZiGwQkXUicptn+30ikiMiKz23c1s7m2k5HdqF8vx1Y0iOjeSGWeksz9rvOpLxosrqGm55aQWLtxfw4LShnHViN9eRTDO52FOoAu5S1QHAWODHInL4UpW/q+owz+09B9lMC4qNCmP2DWPoEhPOtf9ZytqcA64jGS+oqq7hzrmr+GTDHv4wdRAXDLfzSP6s1UtBVXNVdYXn+2JgA2D/L2qjurSPYPaNY2kfEco1zy5l4+4i15FMC6qqruGOuat4Z9UufnFOf66yyRH9ntNzCiKSAgwHlng23SIiq0XkWRHp5CyYaVGJHdsx+4YxhAUHcfmTi22PoY2oqq7htldW1hXCzaf2dh3JtABnpSAi0cDrwO2qWgQ8BvQGhgG5wINHed1MEUkXkfT8/PxWy2uOT0pcFK/cNJbIsBCueGqxLenp5yqra/jJy98yf3Uu955rhdCWOCkFEQmlthBmq+obAKqap6rVqloDPAWMbui1qvqkqqapalp8fHzrhTbHrUfnKF6eOZYOkaFc9fQSO/nspyqqarj1pW95b81ufnXeAGaeYoXQlri4+kiAZ4ANqvpQve0J9Z52IbC2tbMZ70uKjeSVmePoHB3GNc8sYdHWAteRTBOUlFdx/axlfLBuN785f6CNWm+DXOwpnAxcDUw84vLTv4jIGhFZDZwO3OEgm2kFJ3Rsxys3jSOhYztm/GcpH6zNdR3JNML+kgqufHoJX2fs5S+XDOG68T1dRzJeIKr+O6NlWlqapqenu45hmml/SQXXz1rGtzsL+f3UQbaspw/bVXiIa55dyo59pTx6+XAbh+DnRGS5qqY19JiNaDbOdIoKY/YNY5nYrwu/fmstD320CX/+I6WtythTzCWPfcPuA2U8f91oK4Q2zkrBONUuLJgnrh7JpWndeeTTDH7++moqqmpcxzIeX23J58J/f0NFtfLyzLGM7dXZdSTjZTZLqnEuJDiI/714CN06tOORBVvILCjl8atGEhsV5jpaQHtxcRa/nbeOvl2ieXpGms12GiBsT8H4BBHhzkmpPDx9GCt3FjLl0YVs2l3sOlZAqq5R/vDuen711lpO6RvHqzfb9NeBxErB+JSpwxKZe9M4KqpquOjfX/PJ+jzXkQLKvpIKrv3PUp5ZuJ1rT0rhqWvSiImwBXICiZWC8TnDkjoy75bx9IqP5sYX0nnwo01U27rPXrdqZyE/+OdClmzfxwMXDea+KScSEmy/IgKN/S9ufFK3DhG8evM4po3szj8/zeCqp5ewp7jMdaw2SVWZs3QH0x5fBMDrN5/E9NHJjlMZV6wUjM+KCA3mL5cM5a+XDOHbnfs575GFNgK6hR04VMltL6/knjfWMLZ3Z969dTyDu9tqaYHMSsH4vGlpSbz94/HERIRwxdOL+fP7GyivqnYdy+8t3b6Pcx/+ivlrcrlrUir/uXYUneyKr4BnpWD8Qr9uMbxzy3imj0riiS+2MfXRr21thmaqrK7hwY82Mf3JRYQEC6/dPI5bz+hLcJC4jmZ8gJWC8RtR4SH8+aIhPDMjjb0HK5jyz695/IutVFXbYLfGWp1dezL5n59mcPGI7sz/yQSGJ9vSJeb/2dxHxi8VHCzn3jfX8OG6PAYmtOdPFw1mWFJH17F81qGKah76eBPPLNxOfEw4f7xgMJMGdnUdyzjyfXMfWSkYv6WqfLhuN/fNW09ecRlXj+3BT8/uR3u7rr6OqrJgwx5+/+56duwr5YoxyfzinP72bxTgvq8UbJoL47dEhMmDEji5TxwPfrSZ5xdl8t6a3dw5KZVL07oH/DX2W/KK+f276/lqy156x0fZ3EWmUWxPwbQZq7ML+f0760nP2k9q12juOXcAp6XGU7uuU+DYe7CcRz/N4IXFWUSFBXPHpFSuGtuD0AAvSfP/7PCRCRiHDyk98P5GMgtKOal3Z247oy9jAuAv5MLSCp74chuzvsmkrLKaK8Ykc+ekfjaxoPkOKwUTcCqqanhxcRb//nwrew+WM6ZnLD85oy8n9e7c5vYc8ovLeX5RJs99ncnBiip+MOQEbjuzL73jo11HMz7KSsEErLLKauYs3cHjX2wlr6icod07cO3JKZw7OIHwkGDX8Y5Lxp5inv5qO298m0NldQ1nD+zGHZNS6dctxnU04+OsFEzAK6us5tXl2fxn4Xa27S0hLjqMK0YnMy0tiaRY/5kWuqyymg/X7WZu+k6+ziggPCSIS0Z25/rxPellewamkawUjPGoqVEWZuzl+UWZLNi4B1UYnRLLhSMSOXdQAh0ife9SzeoaZXnWfuav3sVbK3dx4FAl3Tu1Y/qoJK4Y08POGZgm86tSEJHJwMNAMPC0qj5wtOdaKZjjkb2/lLdX7uKNFdlszS8hNFgY07MzZwzowhn9u5Lc2d0exKGKapZm7uPj9bv5cF0e+cXlhIUEcfaJ3Zg+KolxvToTZNNSmGbym1IQkWBgMzAJyAaWAZer6vqGnm+lYFqCqrI2p4h3V+/ikw15bM0vAaBnXBSjUjoxKiWW0T1jSY6N9NpJ6gOllazddYDlWfv5OmMv3+4opKK6hnahwZzeP57JgxKY2L8L0eE2tMgcP38qhXHAfap6tuf+PQCq+ueGnm+lYLwhc28JCzbuYdHWvSzL3M+BQ5UAxISHkNothn7dYujbJZoTOrbjhA7t6NYhgs5RYd/7l7uqUlRWRWFpBbsKy8gqKCFrXymZe0tYt6uIHftKARCBE09oz8m94zi5TxyjUmJpF+bfJ8SN7/GnEc2JwM5697OBMY6ymACVEhfF9eN7cv34ntTUKBn5B0nP3M+G3CI25RUzf3VuXVHU1y40mKjwEKLCa3+JV9coNTVKeVUNBw5VUnXE6nEhQUL3Tu0YlNie6aOTGJzYgcGJHegYaecIjDu+VgoN/an1X/8lichMYCZAcrKtDmW8KyhISO0aQ2rX/7/MU1UpKKkgt7CMXQcOkVt4iP2llZSUV1FSUU1pRRUAwSIEBQlhIUF0igylU2QYnSLD6No+gh6dI0noEBHwU3EY3+NrpZANJNW73x3YVf8Jqvok8CTUHj5qvWjG1BIR4qLDiYsOt1XKTJvja3+mLAP6ikhPEQkDpgPzHGcyxpiA4VN7CqpaJSK3AB9Se0nqs6q6znEsY4wJGD5VCgCq+h7wnuscxhgTiHzt8JExxhiHrBSMMcbUsVIwxhhTx0rBGGNMHSsFY4wxdXxq7qOmEpF8IMt1jmOIA/a6DtFC2spnaSufA+yz+Cpf/yw9VDW+oQf8uhT8gYikH23iKX/TVj5LW/kcYJ/FV/nzZ7HDR8YYY+pYKRhjjKljpeB9T7oO0ILaymdpK58D7LP4Kr/9LHZOwRhjTB3bUzDGGFPHSqEVichPRURFJM51luYQkb+KyEYRWS0ib4pIR9eZmkpEJovIJhHJEJFfuM7TXCKSJCKficgGEVknIre5znQ8RCRYRL4VkXddZzkeItJRRF7z/HeywbPEsF+xUmglIpIETAJ2uM5yHD4GBqnqEGAzcI/jPE0iIsHAv4BzgIHA5SIy0G2qZqsC7lLVAcBY4Md+/FkAbgM2uA7RAh4GPlDV/sBQ/PAzWSm0nr8Dd3PE8qL+RFU/UtUqz93F1K6M509GAxmquk1VK4CXgamOMzWLquaq6grP98XU/vJJdJuqeUSkO3Ae8LTrLMdDRNoDpwDPAKhqhaoWuk3VdFYKrUBEpgA5qrrKdZYWdB3wvusQTZQI7Kx3Pxs//UVan4ikAMOBJW6TNNs/qP2DqcZ1kOPUC8gH/uM5FPa0iES5DtVUPrfIjr8SkU+Abg089EvgXuCs1k3UPN/3OVT1bc9zfknt4YvZrZmtBUgD2/x2zw1ARKKB14HbVbXIdZ6mEpHzgT2qulxETnOd5ziFACOAW1V1iYg8DPwC+LXbWE1jpdBCVPXMhraLyGCgJ7BKRKD2kMsKERmtqrtbMWKjHO1zHCYiM4DzgTPU/65nzgaS6t3vDuxylOW4iUgotYUwW1XfcJ2nmU4GpojIuUAE0F5EXlTVqxznao5sIFtVD++xvUZtKfgVG6fQykQkE0hTVV+eLKtBIjIZeAg4VVXzXedpKhEJofYE+RlADrAMuMIf1wGX2r8wZgH7VPV213lagmdP4aeqer7rLM0lIl8BN6jqJhG5D4hS1Z85jtUktqdgmuJRIBz42LPXs1hVb3YbqfFUtUpEbgE+BIKBZ/2xEDxOBq4G1ojISs+2ez1rnBt3bgVmi0gYsA34H8d5msz2FIwxxtSxq4+MMcbUsVIwxhhTx0rBGGNMHSsFY4wxdawUjDHG1LFSMMYYU8dKwRhjTB0rBWNagIgEiUhhvfvDPGtnpHruR4tIjmeuImN8lpWCMS1AVWuonX3i8KR7N1E7r1J7z/0rgHmqetBFPmMay6a5MKbllACRnmKYALwJxHgeuxG4wVUwYxrLSsGYllNEbQlMAV4Bwqid9XM4UNnG1tMwbZQdPjKm5RwuheuoXUWs2HP/JuAJh7mMaTQrBWNaThEwkdo59XOpLYUEYDIw12UwYxrLSsGYllME3Ak87rlfTO25hDdV9ZCzVMY0gZWCMS3nALX/TS3w3C8G+mKHjowfsfUUjDHG1LE9BWOMMXWsFIwxxtSxUjDGGFPHSsEYY0wdKwVjjDF1rBSMMcbUsVIwxhhTx0rBGGNMnf8DKqCLHXyr3VcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(W_l, cost_l)\n",
    "plt.xlabel('$W$')\n",
    "plt.ylabel('Cost')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6Caylv3xg_mf"
   },
   "source": [
    "## Gradient Descent by Hand"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "5pRl-n4rg_mf"
   },
   "outputs": [],
   "source": [
    "W = 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "YCAMFpKjg_mf"
   },
   "source": [
    "$$ cost(W) = \\frac{1}{m} \\sum^m_{i=1} \\left( Wx^{(i)} - y^{(i)} \\right)^2 $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1ZpkeAeqg_mf"
   },
   "source": [
    "$$ \\nabla W = \\frac{\\partial cost}{\\partial W} = \\frac{2}{m} \\sum^m_{i=1} \\left( Wx^{(i)} - y^{(i)} \\right)x^{(i)} $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "Vn8RhN7ng_mg",
    "outputId": "c726bdbd-42f7-4ee5-db1a-5a17b631d4b5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(-14.)\n"
     ]
    }
   ],
   "source": [
    "gradient = torch.sum((W * x_train - y_train) * x_train)\n",
    "print(gradient)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xv6NEYi_g_mg"
   },
   "source": [
    "$$ W := W - \\alpha \\nabla W $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "id": "f4zRrlSJg_mg",
    "outputId": "7f45ff63-b630-4c0e-fa1e-131c1b1db52a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(1.4000)\n"
     ]
    }
   ],
   "source": [
    "lr = 0.1\n",
    "W -= lr * gradient\n",
    "print(W)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "s6lQjuziKJBh"
   },
   "source": [
    "Q1. 경사하강법(Gradient Descent) W:=W−α∇W에서 -를 쓰는 이유는?\n",
    "\n",
    "=> 기울기가 음수면 W의 값을 증가시켜야 하고, 양수이면 W의 값을 감소시켜야 하기 때문이다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qi8YmCKTg_mg"
   },
   "source": [
    "## Training"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gCwk2_NTjr6p"
   },
   "source": [
    "위의 내용을 참고해 코드를 완성시키세요."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "id": "lYYQHd6sg_mh",
    "outputId": "8b250d4e-b233-4f6c-8b6f-c1d854e27bae"
   },
   "outputs": [
    {
     "ename": "AttributeError",
     "evalue": "'int' object has no attribute 'item'",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-14-d825392eecbc>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m     18\u001b[0m     \u001b[0mgradient\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mtorch\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msum\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mW\u001b[0m \u001b[1;33m*\u001b[0m \u001b[0mx_train\u001b[0m \u001b[1;33m-\u001b[0m \u001b[0my_train\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;33m*\u001b[0m \u001b[0mx_train\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;31m#Gradient Descent by Hand를 이용하세요.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     19\u001b[0m     print('Epoch {:4d}/{} W: {:.3f}, Cost: {:.6f}'.format(\n\u001b[1;32m---> 20\u001b[1;33m         \u001b[0mepoch\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mnb_epochs\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mW\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mitem\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcost\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mitem\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     21\u001b[0m     ))\n\u001b[0;32m     22\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mAttributeError\u001b[0m: 'int' object has no attribute 'item'"
     ]
    }
   ],
   "source": [
    "# 데이터\n",
    "x_train = torch.FloatTensor([[1], [2], [3]])\n",
    "y_train = torch.FloatTensor([[1], [2], [3]])\n",
    "# 모델 초기화\n",
    "W = 0  #가중치 W를 0으로 초기화하세요. Gradient Descent by Hand이기 때문에 학습을 통해 값이 변경되는 것이 아닙니다!\n",
    "# learning rate 설정\n",
    "lr = 0.1\n",
    "\n",
    "nb_epochs = 10\n",
    "for epoch in range(nb_epochs + 1):\n",
    "    \n",
    "    # H(x) 계산\n",
    "    hypothesis = W*x_train # H(x)=Wx를 이용하세요.\n",
    "    \n",
    "    # cost gradient 계산\n",
    "    cost = torch.mean((hypothesis - y_train) ** 2)\n",
    "    #파이토치 코드 상으로 선형 회귀의 비용 함수에 해당되는 평균 제곱 오차를 선언하세요.\n",
    "    gradient = torch.sum((W * x_train - y_train) * x_train) #Gradient Descent by Hand를 이용하세요.\n",
    "    print('Epoch {:4d}/{} W: {:.3f}, Cost: {:.6f}'.format(\n",
    "        epoch, nb_epochs, W.item(), cost.item()\n",
    "    ))\n",
    "\n",
    "    # cost gradient로 H(x) 개선\n",
    "    W -= lr * gradient #W:=W−α∇W을 이용하세요."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9iaJEB1Eg_mh"
   },
   "source": [
    "## Training with `optim`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KLki9JjIkyEL"
   },
   "source": [
    "위의 내용을 참고해 코드를 완성시키세요."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "id": "ZVBbc9t9g_mh",
    "outputId": "8f8397eb-5c46-4232-8f89-045dad4851f0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch    0/10 W: 0.000 Cost: 4.666667\n",
      "Epoch    1/10 W: 1.400 Cost: 0.746667\n",
      "Epoch    2/10 W: 0.840 Cost: 0.119467\n",
      "Epoch    3/10 W: 1.064 Cost: 0.019115\n",
      "Epoch    4/10 W: 0.974 Cost: 0.003058\n",
      "Epoch    5/10 W: 1.010 Cost: 0.000489\n",
      "Epoch    6/10 W: 0.996 Cost: 0.000078\n",
      "Epoch    7/10 W: 1.002 Cost: 0.000013\n",
      "Epoch    8/10 W: 0.999 Cost: 0.000002\n",
      "Epoch    9/10 W: 1.000 Cost: 0.000000\n",
      "Epoch   10/10 W: 1.000 Cost: 0.000000\n"
     ]
    }
   ],
   "source": [
    "# 데이터\n",
    "x_train = torch.FloatTensor([[1], [2], [3]])\n",
    "y_train = torch.FloatTensor([[1], [2], [3]])\n",
    "# 모델 초기화\n",
    "W = torch.zeros(1,requires_grad=True) \n",
    "#가중치 W를 0으로 초기화하고 학습을 통해 값이 변경될 수 있도록 하세요.\n",
    "# optimizer 설정\n",
    "optimizer = optim.SGD([W],lr=0.15)\n",
    "#SGD optimizer를 사용하고 learning rate는 0.15로 적용하세요.\n",
    "\n",
    "nb_epochs = 10\n",
    "for epoch in range(nb_epochs + 1):\n",
    "    \n",
    "    # H(x) 계산\n",
    "    hypothesis = x_train*W # H(x)=Wx를 이용하세요.\n",
    "    \n",
    "    # cost 계산\n",
    "    cost = torch.mean((hypothesis-y_train)**2) #파이토치 코드 상으로 선형 회귀의 비용 함수에 해당되는 평균 제곱 오차를 선언하세요.\n",
    "\n",
    "    print('Epoch {:4d}/{} W: {:.3f} Cost: {:.6f}'.format(\n",
    "        epoch, nb_epochs, W.item(), cost.item()\n",
    "    ))\n",
    "\n",
    "    # cost로 H(x) 개선\n",
    "    optimizer.zero_grad()\n",
    "    cost.backward()\n",
    "    optimizer.step()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "Lab03.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}