digit_classifier_Part2.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "digit_classifier_Part2.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "E4ySFSR4GxzE",
        "YJJ8RohfGqz7"
      ],
      "authorship_tag": "ABX9TyN+8nx/TpdlTzdNInZJxdVX",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/ozaanme/MNIST_fastai/blob/main/digit_classifier_Part2.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Intro"
      ],
      "metadata": {
        "id": "E4ySFSR4GxzE"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "Hi everyone, this is part 2 for MNIST digit classifier. This part differs from the first one in the way of creating our model. In first part, we used ResNet to train our data. In this part, we are going to learn how to use SGD for training data. In addition, I'd like to say that this isn't a content created entirely by me. I wrote this because I think that it helps me to understand concepts in fast.ai book."
      ],
      "metadata": {
        "id": "jf4u0TOo_l4r"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "-AEh23Dz_j2c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a3f90655-4165-46d3-f679-1028e2bb94d1"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[K     |████████████████████████████████| 197 kB 4.3 MB/s \n",
            "\u001b[K     |████████████████████████████████| 59 kB 6.2 MB/s \n",
            "\u001b[?25hCollecting fastbook\n",
            "  Downloading fastbook-0.0.24-py3-none-any.whl (719 kB)\n",
            "\u001b[K     |████████████████████████████████| 719 kB 4.2 MB/s \n",
            "\u001b[?25hRequirement already satisfied: fastai>=2.6 in /usr/local/lib/python3.7/dist-packages (from fastbook) (2.6.3)\n",
            "Requirement already satisfied: ipywidgets in /usr/local/lib/python3.7/dist-packages (from fastbook) (7.7.0)\n",
            "Collecting sentencepiece\n",
            "  Downloading sentencepiece-0.1.96-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)\n",
            "\u001b[K     |████████████████████████████████| 1.2 MB 39.4 MB/s \n",
            "\u001b[?25hRequirement already satisfied: packaging in /usr/local/lib/python3.7/dist-packages (from fastbook) (21.3)\n",
            "Requirement already satisfied: pip in /usr/local/lib/python3.7/dist-packages (from fastbook) (21.1.3)\n",
            "Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from fastbook) (1.3.5)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from fastbook) (2.23.0)\n",
            "Requirement already satisfied: graphviz in /usr/local/lib/python3.7/dist-packages (from fastbook) (0.10.1)\n",
            "Requirement already satisfied: fastprogress>=0.2.4 in /usr/local/lib/python3.7/dist-packages (from fastai>=2.6->fastbook) (1.0.2)\n",
            "Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from fastai>=2.6->fastbook) (3.2.2)\n",
            "Requirement already satisfied: fastcore<1.5,>=1.3.27 in /usr/local/lib/python3.7/dist-packages (from fastai>=2.6->fastbook) (1.4.3)\n",
            "Requirement already satisfied: pyyaml in /usr/local/lib/python3.7/dist-packages (from fastai>=2.6->fastbook) (3.13)\n",
            "Requirement already satisfied: fastdownload<2,>=0.0.5 in /usr/local/lib/python3.7/dist-packages (from fastai>=2.6->fastbook) (0.0.5)\n",
            "Requirement already satisfied: pillow>6.0.0 in /usr/local/lib/python3.7/dist-packages (from fastai>=2.6->fastbook) (7.1.2)\n",
            "Requirement already satisfied: scikit-learn in /usr/local/lib/python3.7/dist-packages (from fastai>=2.6->fastbook) (1.0.2)\n",
            "Requirement already satisfied: spacy<4 in /usr/local/lib/python3.7/dist-packages (from fastai>=2.6->fastbook) (2.2.4)\n",
            "Requirement already satisfied: torch<1.12,>=1.7.0 in /usr/local/lib/python3.7/dist-packages (from fastai>=2.6->fastbook) (1.11.0+cu113)\n",
            "Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from fastai>=2.6->fastbook) (1.4.1)\n",
            "Requirement already satisfied: torchvision>=0.8.2 in /usr/local/lib/python3.7/dist-packages (from fastai>=2.6->fastbook) (0.12.0+cu113)\n",
            "Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai>=2.6->fastbook) (2.0.6)\n",
            "Requirement already satisfied: thinc==7.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai>=2.6->fastbook) (7.4.0)\n",
            "Requirement already satisfied: numpy>=1.15.0 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai>=2.6->fastbook) (1.21.6)\n",
            "Requirement already satisfied: plac<1.2.0,>=0.9.6 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai>=2.6->fastbook) (1.1.3)\n",
            "Requirement already satisfied: wasabi<1.1.0,>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai>=2.6->fastbook) (0.9.1)\n",
            "Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai>=2.6->fastbook) (3.0.6)\n",
            "Requirement already satisfied: srsly<1.1.0,>=1.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai>=2.6->fastbook) (1.0.5)\n",
            "Requirement already satisfied: catalogue<1.1.0,>=0.0.7 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai>=2.6->fastbook) (1.0.0)\n",
            "Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai>=2.6->fastbook) (4.64.0)\n",
            "Requirement already satisfied: blis<0.5.0,>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai>=2.6->fastbook) (0.4.1)\n",
            "Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai>=2.6->fastbook) (1.0.7)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai>=2.6->fastbook) (57.4.0)\n",
            "Requirement already satisfied: importlib-metadata>=0.20 in /usr/local/lib/python3.7/dist-packages (from catalogue<1.1.0,>=0.0.7->spacy<4->fastai>=2.6->fastbook) (4.11.3)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=0.20->catalogue<1.1.0,>=0.0.7->spacy<4->fastai>=2.6->fastbook) (3.8.0)\n",
            "Requirement already satisfied: typing-extensions>=3.6.4 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=0.20->catalogue<1.1.0,>=0.0.7->spacy<4->fastai>=2.6->fastbook) (4.2.0)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->fastbook) (1.24.3)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->fastbook) (2021.10.8)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->fastbook) (3.0.4)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->fastbook) (2.10)\n",
            "Requirement already satisfied: ipython>=4.0.0 in /usr/local/lib/python3.7/dist-packages (from ipywidgets->fastbook) (5.5.0)\n",
            "Requirement already satisfied: widgetsnbextension~=3.6.0 in /usr/local/lib/python3.7/dist-packages (from ipywidgets->fastbook) (3.6.0)\n",
            "Requirement already satisfied: ipykernel>=4.5.1 in /usr/local/lib/python3.7/dist-packages (from ipywidgets->fastbook) (4.10.1)\n",
            "Requirement already satisfied: traitlets>=4.3.1 in /usr/local/lib/python3.7/dist-packages (from ipywidgets->fastbook) (5.1.1)\n",
            "Requirement already satisfied: ipython-genutils~=0.2.0 in /usr/local/lib/python3.7/dist-packages (from ipywidgets->fastbook) (0.2.0)\n",
            "Requirement already satisfied: nbformat>=4.2.0 in /usr/local/lib/python3.7/dist-packages (from ipywidgets->fastbook) (5.3.0)\n",
            "Requirement already satisfied: jupyterlab-widgets>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from ipywidgets->fastbook) (1.1.0)\n",
            "Requirement already satisfied: jupyter-client in /usr/local/lib/python3.7/dist-packages (from ipykernel>=4.5.1->ipywidgets->fastbook) (5.3.5)\n",
            "Requirement already satisfied: tornado>=4.0 in /usr/local/lib/python3.7/dist-packages (from ipykernel>=4.5.1->ipywidgets->fastbook) (5.1.1)\n",
            "Requirement already satisfied: pickleshare in /usr/local/lib/python3.7/dist-packages (from ipython>=4.0.0->ipywidgets->fastbook) (0.7.5)\n",
            "Requirement already satisfied: prompt-toolkit<2.0.0,>=1.0.4 in /usr/local/lib/python3.7/dist-packages (from ipython>=4.0.0->ipywidgets->fastbook) (1.0.18)\n",
            "Requirement already satisfied: pygments in /usr/local/lib/python3.7/dist-packages (from ipython>=4.0.0->ipywidgets->fastbook) (2.6.1)\n",
            "Requirement already satisfied: decorator in /usr/local/lib/python3.7/dist-packages (from ipython>=4.0.0->ipywidgets->fastbook) (4.4.2)\n",
            "Requirement already satisfied: pexpect in /usr/local/lib/python3.7/dist-packages (from ipython>=4.0.0->ipywidgets->fastbook) (4.8.0)\n",
            "Requirement already satisfied: simplegeneric>0.8 in /usr/local/lib/python3.7/dist-packages (from ipython>=4.0.0->ipywidgets->fastbook) (0.8.1)\n",
            "Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.7/dist-packages (from nbformat>=4.2.0->ipywidgets->fastbook) (4.3.3)\n",
            "Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.7/dist-packages (from nbformat>=4.2.0->ipywidgets->fastbook) (2.15.3)\n",
            "Requirement already satisfied: jupyter-core in /usr/local/lib/python3.7/dist-packages (from nbformat>=4.2.0->ipywidgets->fastbook) (4.10.0)\n",
            "Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.7/dist-packages (from jsonschema>=2.6->nbformat>=4.2.0->ipywidgets->fastbook) (0.18.1)\n",
            "Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.7/dist-packages (from jsonschema>=2.6->nbformat>=4.2.0->ipywidgets->fastbook) (21.4.0)\n",
            "Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.7/dist-packages (from jsonschema>=2.6->nbformat>=4.2.0->ipywidgets->fastbook) (5.7.1)\n",
            "Requirement already satisfied: six>=1.9.0 in /usr/local/lib/python3.7/dist-packages (from prompt-toolkit<2.0.0,>=1.0.4->ipython>=4.0.0->ipywidgets->fastbook) (1.15.0)\n",
            "Requirement already satisfied: wcwidth in /usr/local/lib/python3.7/dist-packages (from prompt-toolkit<2.0.0,>=1.0.4->ipython>=4.0.0->ipywidgets->fastbook) (0.2.5)\n",
            "Requirement already satisfied: notebook>=4.4.1 in /usr/local/lib/python3.7/dist-packages (from widgetsnbextension~=3.6.0->ipywidgets->fastbook) (5.3.1)\n",
            "Requirement already satisfied: nbconvert in /usr/local/lib/python3.7/dist-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->fastbook) (5.6.1)\n",
            "Requirement already satisfied: terminado>=0.8.1 in /usr/local/lib/python3.7/dist-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->fastbook) (0.13.3)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.7/dist-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->fastbook) (2.11.3)\n",
            "Requirement already satisfied: Send2Trash in /usr/local/lib/python3.7/dist-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->fastbook) (1.8.0)\n",
            "Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from jupyter-client->ipykernel>=4.5.1->ipywidgets->fastbook) (2.8.2)\n",
            "Requirement already satisfied: pyzmq>=13 in /usr/local/lib/python3.7/dist-packages (from jupyter-client->ipykernel>=4.5.1->ipywidgets->fastbook) (22.3.0)\n",
            "Requirement already satisfied: ptyprocess in /usr/local/lib/python3.7/dist-packages (from terminado>=0.8.1->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->fastbook) (0.7.0)\n",
            "Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from jinja2->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->fastbook) (2.0.1)\n",
            "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fastai>=2.6->fastbook) (1.4.2)\n",
            "Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fastai>=2.6->fastbook) (3.0.8)\n",
            "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fastai>=2.6->fastbook) (0.11.0)\n",
            "Requirement already satisfied: bleach in /usr/local/lib/python3.7/dist-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->fastbook) (5.0.0)\n",
            "Requirement already satisfied: entrypoints>=0.2.2 in /usr/local/lib/python3.7/dist-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->fastbook) (0.4)\n",
            "Requirement already satisfied: testpath in /usr/local/lib/python3.7/dist-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->fastbook) (0.6.0)\n",
            "Requirement already satisfied: defusedxml in /usr/local/lib/python3.7/dist-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->fastbook) (0.7.1)\n",
            "Requirement already satisfied: pandocfilters>=1.4.1 in /usr/local/lib/python3.7/dist-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->fastbook) (1.5.0)\n",
            "Requirement already satisfied: mistune<2,>=0.8.1 in /usr/local/lib/python3.7/dist-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->fastbook) (0.8.4)\n",
            "Requirement already satisfied: webencodings in /usr/local/lib/python3.7/dist-packages (from bleach->nbconvert->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->fastbook) (0.5.1)\n",
            "Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas->fastbook) (2022.1)\n",
            "Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->fastai>=2.6->fastbook) (3.1.0)\n",
            "Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->fastai>=2.6->fastbook) (1.1.0)\n",
            "Installing collected packages: sentencepiece, fastbook\n",
            "Successfully installed fastbook-0.0.24 sentencepiece-0.1.96\n"
          ]
        }
      ],
      "source": [
        "! [ -e /content ] && pip install -Uqq fastai  # upgrade fastai on colab\n",
        "! pip install fastbook\n",
        "from fastai.vision.all import * \n",
        "from fastbook import * \n",
        "import ipywidgets as widgets\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "As you can remember from the first after we install fastai on colab, we need to download data."
      ],
      "metadata": {
        "id": "3DQe05fBAMSr"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "path = untar_data(URLs.MNIST)"
      ],
      "metadata": {
        "id": "Is6a_Ij6ASE0",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "outputId": "b03b31a0-7322-4355-bacc-c63fc944da50"
      },
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "<style>\n",
              "    /* Turns off some styling */\n",
              "    progress {\n",
              "        /* gets rid of default border in Firefox and Opera. */\n",
              "        border: none;\n",
              "        /* Needs to be in here for Safari polyfill so background images work as expected. */\n",
              "        background-size: auto;\n",
              "    }\n",
              "    .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {\n",
              "        background: #F44336;\n",
              "    }\n",
              "</style>\n"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "    <div>\n",
              "      <progress value='15687680' class='' max='15683414' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      100.03% [15687680/15683414 00:02<00:00]\n",
              "    </div>\n",
              "    "
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        ""
      ],
      "metadata": {
        "id": "pSWX66tWDS4Y"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "After we download our data lets take a look whats in it."
      ],
      "metadata": {
        "id": "3gnC-9beAEt1"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "(path/'training').ls()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IIbyZM0NAgPp",
        "outputId": "225e5e49-5bd9-49d8-f0eb-591e5b475e33"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(#10) [Path('/root/.fastai/data/mnist_png/training/3'),Path('/root/.fastai/data/mnist_png/training/2'),Path('/root/.fastai/data/mnist_png/training/0'),Path('/root/.fastai/data/mnist_png/training/9'),Path('/root/.fastai/data/mnist_png/training/4'),Path('/root/.fastai/data/mnist_png/training/7'),Path('/root/.fastai/data/mnist_png/training/5'),Path('/root/.fastai/data/mnist_png/training/8'),Path('/root/.fastai/data/mnist_png/training/1'),Path('/root/.fastai/data/mnist_png/training/6')]"
            ]
          },
          "metadata": {},
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "As you can see, there are folders contain images of each number. We can also look into these ones(using sorted to ensure we all get the same order of files)."
      ],
      "metadata": {
        "id": "WcK8NZ7eBaxA"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "zeros = (path/'training'/'0').ls().sorted()\n",
        "ones = (path/'training'/'1').ls().sorted()\n",
        "twos = (path/'training'/'2').ls().sorted()\n",
        "threes = (path/'training'/'3').ls().sorted()\n",
        "fours = (path/'training'/'4').ls().sorted()\n",
        "fives = (path/'training'/'5').ls().sorted()\n",
        "sixes = (path/'training'/'6').ls().sorted()\n",
        "sevens = (path/'training'/'7').ls().sorted()\n",
        "eights = (path/'training'/'8').ls().sorted()\n",
        "nines = (path/'training'/'9').ls().sorted()\n",
        "zeros\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Z6mahGewB2-F",
        "outputId": "2b2ddad9-7dfb-4264-96a2-5e3752671050"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(#5923) [Path('/root/.fastai/data/mnist_png/training/0/1.png'),Path('/root/.fastai/data/mnist_png/training/0/1000.png'),Path('/root/.fastai/data/mnist_png/training/0/10005.png'),Path('/root/.fastai/data/mnist_png/training/0/10010.png'),Path('/root/.fastai/data/mnist_png/training/0/10022.png'),Path('/root/.fastai/data/mnist_png/training/0/10025.png'),Path('/root/.fastai/data/mnist_png/training/0/10026.png'),Path('/root/.fastai/data/mnist_png/training/0/10045.png'),Path('/root/.fastai/data/mnist_png/training/0/10069.png'),Path('/root/.fastai/data/mnist_png/training/0/10071.png')...]"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "As expected, it is full of images. Let’s take a look at one now."
      ],
      "metadata": {
        "id": "2VIpnPkbCV0o"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "im5_path = fives[5]\n",
        "im5 = Image.open(im5_path)\n",
        "im5"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "5dG_KOZ5Cdor",
        "outputId": "b119673e-8d15-4b41-aa4e-aed9fe9aae6c"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<PIL.PngImagePlugin.PngImageFile image mode=L size=28x28 at 0x7FE684A5DFD0>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAAAxUlEQVR4nGNgGDaAEY3vxhAsz3DGhIGh4gKqpOD8rwoW/xkYGBj/MzC8kWBgYEGSlPOBMg6eZlgUimqixN+/fz+XaGG3XuLPnzVGuNwm8eePL06Hs7b/OWSMU1aq88UnnJIMDB2/v+C0lYEh5O9aXnSxnYvdoGKdf1LQJef/+XM9nYGBgYFB4s92dEnxy3/+/LlbosDA0IApycCgMPfj/79///79/3ctNrdopV/78+fP31OSCCHkWOG1YGBguPIct18GHAAApodAzc31jCsAAAAASUVORK5CYII=\n"
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "In this part, it is important to understand what images means to computer. In computers, everything is represented as numbers. In order to review the numbers that create this image, we have to convert it to NumPy array or PyTorch tensor.\n",
        "Let's first examine NumPy array:"
      ],
      "metadata": {
        "id": "m75oOxHACp0J"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "array(im5)[5:15,5:15]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pH1Qz0FUDLVU",
        "outputId": "7633fa8f-5a94-4dd3-d83f-b3e2d335dc1a"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,  17, 176, 165, 197],\n",
              "       [  0,   0,   0,   0,   0,   0,  47, 252, 252, 252],\n",
              "       [  0,   0,   0,   0,   0,  24, 253, 253, 253, 243],\n",
              "       [  0,   0,   0,   0,   0,  24, 252, 252, 172,  50],\n",
              "       [  0,   0,   0,   0,   0,  24, 252, 252,  77,   0],\n",
              "       [  0,   0,   0,   0,   0,   5, 135, 252, 194,  51],\n",
              "       [  0,   0,   0,   0,   0,   0,  26, 137, 232, 242]], dtype=uint8)"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "5:15 means that we only request the part that includes these columns and rows. And we can also make same thing for a tensor."
      ],
      "metadata": {
        "id": "0MR1F7VgD8Lv"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "tensor(im5)[5:15,5:15]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dqPsGLHfELeS",
        "outputId": "8fc79bd9-b0c0-4739-8e58-7ca6337db56f"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "        [  0,   0,   0,   0,   0,   0,  17, 176, 165, 197],\n",
              "        [  0,   0,   0,   0,   0,   0,  47, 252, 252, 252],\n",
              "        [  0,   0,   0,   0,   0,  24, 253, 253, 253, 243],\n",
              "        [  0,   0,   0,   0,   0,  24, 252, 252, 172,  50],\n",
              "        [  0,   0,   0,   0,   0,  24, 252, 252,  77,   0],\n",
              "        [  0,   0,   0,   0,   0,   5, 135, 252, 194,  51],\n",
              "        [  0,   0,   0,   0,   0,   0,  26, 137, 232, 242]], dtype=torch.uint8)"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can pick specific part of this tensor, and then use Pandas DataFrame to color-code the values using a gradient."
      ],
      "metadata": {
        "id": "1nN2C7bTFY0d"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "im5_t = tensor(im5)\n",
        "df = pd.DataFrame(im5_t[10:25,5:20]) ## This indexes shows us the bottom part of image.\n",
        "df.style.set_properties(**{'font-size':'6pt'}).background_gradient('Greys')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "u7N5rbXkFjsT",
        "outputId": "eb81b81a-52e3-425e-8ee9-324f59867b26"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<pandas.io.formats.style.Styler at 0x7fe68375a810>"
            ],
            "text/html": [
              "<style type=\"text/css\">\n",
              "#T_c9b29_row0_col0, #T_c9b29_row0_col1, #T_c9b29_row0_col2, #T_c9b29_row0_col3, #T_c9b29_row0_col4, #T_c9b29_row0_col12, #T_c9b29_row0_col13, #T_c9b29_row0_col14, #T_c9b29_row1_col0, #T_c9b29_row1_col1, #T_c9b29_row1_col2, #T_c9b29_row1_col3, #T_c9b29_row1_col4, #T_c9b29_row1_col10, #T_c9b29_row1_col11, #T_c9b29_row1_col12, #T_c9b29_row1_col13, #T_c9b29_row1_col14, #T_c9b29_row2_col0, #T_c9b29_row2_col1, #T_c9b29_row2_col2, #T_c9b29_row2_col3, #T_c9b29_row2_col4, #T_c9b29_row2_col9, #T_c9b29_row2_col10, #T_c9b29_row2_col11, #T_c9b29_row2_col12, #T_c9b29_row2_col13, #T_c9b29_row2_col14, #T_c9b29_row3_col0, #T_c9b29_row3_col1, #T_c9b29_row3_col2, #T_c9b29_row3_col3, #T_c9b29_row3_col4, #T_c9b29_row3_col10, #T_c9b29_row3_col11, #T_c9b29_row3_col12, #T_c9b29_row3_col13, #T_c9b29_row3_col14, #T_c9b29_row4_col0, #T_c9b29_row4_col1, #T_c9b29_row4_col2, #T_c9b29_row4_col3, #T_c9b29_row4_col4, #T_c9b29_row4_col5, #T_c9b29_row4_col10, #T_c9b29_row4_col11, #T_c9b29_row4_col12, #T_c9b29_row4_col13, #T_c9b29_row4_col14, #T_c9b29_row5_col0, #T_c9b29_row5_col1, #T_c9b29_row5_col2, #T_c9b29_row5_col3, #T_c9b29_row5_col4, #T_c9b29_row5_col5, #T_c9b29_row5_col6, #T_c9b29_row5_col7, #T_c9b29_row5_col12, #T_c9b29_row5_col13, #T_c9b29_row5_col14, #T_c9b29_row6_col0, #T_c9b29_row6_col1, #T_c9b29_row6_col2, #T_c9b29_row6_col3, #T_c9b29_row6_col4, #T_c9b29_row6_col5, #T_c9b29_row6_col6, #T_c9b29_row6_col7, #T_c9b29_row6_col8, #T_c9b29_row6_col13, #T_c9b29_row6_col14, #T_c9b29_row7_col4, #T_c9b29_row7_col5, #T_c9b29_row7_col6, #T_c9b29_row7_col7, #T_c9b29_row7_col8, #T_c9b29_row7_col9, #T_c9b29_row7_col13, #T_c9b29_row7_col14, #T_c9b29_row8_col5, #T_c9b29_row8_col6, #T_c9b29_row8_col7, #T_c9b29_row8_col8, #T_c9b29_row8_col9, #T_c9b29_row8_col13, #T_c9b29_row8_col14, #T_c9b29_row9_col8, #T_c9b29_row9_col9, #T_c9b29_row9_col13, #T_c9b29_row9_col14, #T_c9b29_row10_col0, #T_c9b29_row10_col1, #T_c9b29_row10_col13, #T_c9b29_row10_col14, #T_c9b29_row11_col0, #T_c9b29_row11_col1, #T_c9b29_row11_col2, #T_c9b29_row11_col3, #T_c9b29_row11_col13, #T_c9b29_row11_col14, #T_c9b29_row12_col0, #T_c9b29_row12_col1, #T_c9b29_row12_col2, #T_c9b29_row12_col3, #T_c9b29_row12_col4, #T_c9b29_row12_col5, #T_c9b29_row12_col12, #T_c9b29_row12_col13, #T_c9b29_row12_col14, #T_c9b29_row13_col0, #T_c9b29_row13_col1, #T_c9b29_row13_col2, #T_c9b29_row13_col3, #T_c9b29_row13_col4, #T_c9b29_row13_col5, #T_c9b29_row13_col6, #T_c9b29_row13_col7, #T_c9b29_row13_col8, #T_c9b29_row13_col9, #T_c9b29_row13_col10, #T_c9b29_row13_col11, #T_c9b29_row13_col12, #T_c9b29_row13_col13, #T_c9b29_row13_col14, #T_c9b29_row14_col0, #T_c9b29_row14_col1, #T_c9b29_row14_col2, #T_c9b29_row14_col3, #T_c9b29_row14_col4, #T_c9b29_row14_col5, #T_c9b29_row14_col6, #T_c9b29_row14_col7, #T_c9b29_row14_col8, #T_c9b29_row14_col9, #T_c9b29_row14_col10, #T_c9b29_row14_col11, #T_c9b29_row14_col12, #T_c9b29_row14_col13, #T_c9b29_row14_col14 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #ffffff;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_c9b29_row0_col5, #T_c9b29_row1_col5, #T_c9b29_row2_col5, #T_c9b29_row8_col10 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #f4f4f4;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_c9b29_row0_col6, #T_c9b29_row0_col7, #T_c9b29_row0_col8, #T_c9b29_row7_col0, #T_c9b29_row8_col1, #T_c9b29_row8_col2, #T_c9b29_row8_col12, #T_c9b29_row9_col2, #T_c9b29_row9_col3, #T_c9b29_row9_col4, #T_c9b29_row9_col12, #T_c9b29_row10_col5, #T_c9b29_row10_col6, #T_c9b29_row10_col7, #T_c9b29_row10_col8, #T_c9b29_row10_col9, #T_c9b29_row10_col10, #T_c9b29_row10_col11 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #000000;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row0_col9 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #0c0c0c;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row0_col10 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #a4a4a4;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row0_col11, #T_c9b29_row11_col4 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #e9e9e9;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_c9b29_row1_col6, #T_c9b29_row1_col7, #T_c9b29_row2_col6, #T_c9b29_row2_col7, #T_c9b29_row3_col7, #T_c9b29_row7_col11, #T_c9b29_row8_col11, #T_c9b29_row9_col11, #T_c9b29_row11_col7, #T_c9b29_row11_col8, #T_c9b29_row11_col9 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #010101;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row1_col8 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #646464;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row1_col9, #T_c9b29_row5_col11 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #e3e3e3;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_c9b29_row2_col8 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #cdcdcd;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_c9b29_row3_col5 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #fdfdfd;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_c9b29_row3_col6 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #8d8d8d;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row3_col8 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #4b4b4b;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row3_col9 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #e2e2e2;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_c9b29_row4_col6, #T_c9b29_row12_col11 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #f3f3f3;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_c9b29_row4_col7 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #8a8a8a;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row4_col8 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #181818;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row4_col9, #T_c9b29_row5_col10, #T_c9b29_row10_col4 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #0d0d0d;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row5_col8, #T_c9b29_row7_col10 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #8c8c8c;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row5_col9, #T_c9b29_row6_col10, #T_c9b29_row11_col10 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #020202;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row6_col9 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #c7c7c7;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_c9b29_row6_col11 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #636363;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row6_col12 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #f7f7f7;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_c9b29_row7_col1 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #6d6d6d;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row7_col2 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #d3d3d3;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_c9b29_row7_col3, #T_c9b29_row12_col6 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #f9f9f9;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_c9b29_row7_col12 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #898989;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row8_col0 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #292929;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row8_col3 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #2c2c2c;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row8_col4 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #b3b3b3;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_c9b29_row9_col0, #T_c9b29_row9_col7, #T_c9b29_row10_col2 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #f0f0f0;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_c9b29_row9_col1 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #323232;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row9_col5 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #282828;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row9_col6 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #a3a3a3;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row9_col10 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #959595;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row10_col3 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #737373;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row10_col12 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #0f0f0f;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row11_col5 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #b4b4b4;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_c9b29_row11_col6 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #2f2f2f;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row11_col11 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #404040;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_c9b29_row11_col12 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #eeeeee;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_c9b29_row12_col7, #T_c9b29_row12_col8, #T_c9b29_row12_col9, #T_c9b29_row12_col10 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #d4d4d4;\n",
              "  color: #000000;\n",
              "}\n",
              "</style>\n",
              "<table id=\"T_c9b29_\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr>\n",
              "      <th class=\"blank level0\" >&nbsp;</th>\n",
              "      <th class=\"col_heading level0 col0\" >0</th>\n",
              "      <th class=\"col_heading level0 col1\" >1</th>\n",
              "      <th class=\"col_heading level0 col2\" >2</th>\n",
              "      <th class=\"col_heading level0 col3\" >3</th>\n",
              "      <th class=\"col_heading level0 col4\" >4</th>\n",
              "      <th class=\"col_heading level0 col5\" >5</th>\n",
              "      <th class=\"col_heading level0 col6\" >6</th>\n",
              "      <th class=\"col_heading level0 col7\" >7</th>\n",
              "      <th class=\"col_heading level0 col8\" >8</th>\n",
              "      <th class=\"col_heading level0 col9\" >9</th>\n",
              "      <th class=\"col_heading level0 col10\" >10</th>\n",
              "      <th class=\"col_heading level0 col11\" >11</th>\n",
              "      <th class=\"col_heading level0 col12\" >12</th>\n",
              "      <th class=\"col_heading level0 col13\" >13</th>\n",
              "      <th class=\"col_heading level0 col14\" >14</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th id=\"T_c9b29_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
              "      <td id=\"T_c9b29_row0_col0\" class=\"data row0 col0\" >0</td>\n",
              "      <td id=\"T_c9b29_row0_col1\" class=\"data row0 col1\" >0</td>\n",
              "      <td id=\"T_c9b29_row0_col2\" class=\"data row0 col2\" >0</td>\n",
              "      <td id=\"T_c9b29_row0_col3\" class=\"data row0 col3\" >0</td>\n",
              "      <td id=\"T_c9b29_row0_col4\" class=\"data row0 col4\" >0</td>\n",
              "      <td id=\"T_c9b29_row0_col5\" class=\"data row0 col5\" >24</td>\n",
              "      <td id=\"T_c9b29_row0_col6\" class=\"data row0 col6\" >253</td>\n",
              "      <td id=\"T_c9b29_row0_col7\" class=\"data row0 col7\" >253</td>\n",
              "      <td id=\"T_c9b29_row0_col8\" class=\"data row0 col8\" >253</td>\n",
              "      <td id=\"T_c9b29_row0_col9\" class=\"data row0 col9\" >243</td>\n",
              "      <td id=\"T_c9b29_row0_col10\" class=\"data row0 col10\" >116</td>\n",
              "      <td id=\"T_c9b29_row0_col11\" class=\"data row0 col11\" >42</td>\n",
              "      <td id=\"T_c9b29_row0_col12\" class=\"data row0 col12\" >0</td>\n",
              "      <td id=\"T_c9b29_row0_col13\" class=\"data row0 col13\" >0</td>\n",
              "      <td id=\"T_c9b29_row0_col14\" class=\"data row0 col14\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_c9b29_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
              "      <td id=\"T_c9b29_row1_col0\" class=\"data row1 col0\" >0</td>\n",
              "      <td id=\"T_c9b29_row1_col1\" class=\"data row1 col1\" >0</td>\n",
              "      <td id=\"T_c9b29_row1_col2\" class=\"data row1 col2\" >0</td>\n",
              "      <td id=\"T_c9b29_row1_col3\" class=\"data row1 col3\" >0</td>\n",
              "      <td id=\"T_c9b29_row1_col4\" class=\"data row1 col4\" >0</td>\n",
              "      <td id=\"T_c9b29_row1_col5\" class=\"data row1 col5\" >24</td>\n",
              "      <td id=\"T_c9b29_row1_col6\" class=\"data row1 col6\" >252</td>\n",
              "      <td id=\"T_c9b29_row1_col7\" class=\"data row1 col7\" >252</td>\n",
              "      <td id=\"T_c9b29_row1_col8\" class=\"data row1 col8\" >172</td>\n",
              "      <td id=\"T_c9b29_row1_col9\" class=\"data row1 col9\" >50</td>\n",
              "      <td id=\"T_c9b29_row1_col10\" class=\"data row1 col10\" >0</td>\n",
              "      <td id=\"T_c9b29_row1_col11\" class=\"data row1 col11\" >0</td>\n",
              "      <td id=\"T_c9b29_row1_col12\" class=\"data row1 col12\" >0</td>\n",
              "      <td id=\"T_c9b29_row1_col13\" class=\"data row1 col13\" >0</td>\n",
              "      <td id=\"T_c9b29_row1_col14\" class=\"data row1 col14\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_c9b29_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
              "      <td id=\"T_c9b29_row2_col0\" class=\"data row2 col0\" >0</td>\n",
              "      <td id=\"T_c9b29_row2_col1\" class=\"data row2 col1\" >0</td>\n",
              "      <td id=\"T_c9b29_row2_col2\" class=\"data row2 col2\" >0</td>\n",
              "      <td id=\"T_c9b29_row2_col3\" class=\"data row2 col3\" >0</td>\n",
              "      <td id=\"T_c9b29_row2_col4\" class=\"data row2 col4\" >0</td>\n",
              "      <td id=\"T_c9b29_row2_col5\" class=\"data row2 col5\" >24</td>\n",
              "      <td id=\"T_c9b29_row2_col6\" class=\"data row2 col6\" >252</td>\n",
              "      <td id=\"T_c9b29_row2_col7\" class=\"data row2 col7\" >252</td>\n",
              "      <td id=\"T_c9b29_row2_col8\" class=\"data row2 col8\" >77</td>\n",
              "      <td id=\"T_c9b29_row2_col9\" class=\"data row2 col9\" >0</td>\n",
              "      <td id=\"T_c9b29_row2_col10\" class=\"data row2 col10\" >0</td>\n",
              "      <td id=\"T_c9b29_row2_col11\" class=\"data row2 col11\" >0</td>\n",
              "      <td id=\"T_c9b29_row2_col12\" class=\"data row2 col12\" >0</td>\n",
              "      <td id=\"T_c9b29_row2_col13\" class=\"data row2 col13\" >0</td>\n",
              "      <td id=\"T_c9b29_row2_col14\" class=\"data row2 col14\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_c9b29_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
              "      <td id=\"T_c9b29_row3_col0\" class=\"data row3 col0\" >0</td>\n",
              "      <td id=\"T_c9b29_row3_col1\" class=\"data row3 col1\" >0</td>\n",
              "      <td id=\"T_c9b29_row3_col2\" class=\"data row3 col2\" >0</td>\n",
              "      <td id=\"T_c9b29_row3_col3\" class=\"data row3 col3\" >0</td>\n",
              "      <td id=\"T_c9b29_row3_col4\" class=\"data row3 col4\" >0</td>\n",
              "      <td id=\"T_c9b29_row3_col5\" class=\"data row3 col5\" >5</td>\n",
              "      <td id=\"T_c9b29_row3_col6\" class=\"data row3 col6\" >135</td>\n",
              "      <td id=\"T_c9b29_row3_col7\" class=\"data row3 col7\" >252</td>\n",
              "      <td id=\"T_c9b29_row3_col8\" class=\"data row3 col8\" >194</td>\n",
              "      <td id=\"T_c9b29_row3_col9\" class=\"data row3 col9\" >51</td>\n",
              "      <td id=\"T_c9b29_row3_col10\" class=\"data row3 col10\" >0</td>\n",
              "      <td id=\"T_c9b29_row3_col11\" class=\"data row3 col11\" >0</td>\n",
              "      <td id=\"T_c9b29_row3_col12\" class=\"data row3 col12\" >0</td>\n",
              "      <td id=\"T_c9b29_row3_col13\" class=\"data row3 col13\" >0</td>\n",
              "      <td id=\"T_c9b29_row3_col14\" class=\"data row3 col14\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_c9b29_level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
              "      <td id=\"T_c9b29_row4_col0\" class=\"data row4 col0\" >0</td>\n",
              "      <td id=\"T_c9b29_row4_col1\" class=\"data row4 col1\" >0</td>\n",
              "      <td id=\"T_c9b29_row4_col2\" class=\"data row4 col2\" >0</td>\n",
              "      <td id=\"T_c9b29_row4_col3\" class=\"data row4 col3\" >0</td>\n",
              "      <td id=\"T_c9b29_row4_col4\" class=\"data row4 col4\" >0</td>\n",
              "      <td id=\"T_c9b29_row4_col5\" class=\"data row4 col5\" >0</td>\n",
              "      <td id=\"T_c9b29_row4_col6\" class=\"data row4 col6\" >26</td>\n",
              "      <td id=\"T_c9b29_row4_col7\" class=\"data row4 col7\" >137</td>\n",
              "      <td id=\"T_c9b29_row4_col8\" class=\"data row4 col8\" >232</td>\n",
              "      <td id=\"T_c9b29_row4_col9\" class=\"data row4 col9\" >242</td>\n",
              "      <td id=\"T_c9b29_row4_col10\" class=\"data row4 col10\" >0</td>\n",
              "      <td id=\"T_c9b29_row4_col11\" class=\"data row4 col11\" >0</td>\n",
              "      <td id=\"T_c9b29_row4_col12\" class=\"data row4 col12\" >0</td>\n",
              "      <td id=\"T_c9b29_row4_col13\" class=\"data row4 col13\" >0</td>\n",
              "      <td id=\"T_c9b29_row4_col14\" class=\"data row4 col14\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_c9b29_level0_row5\" class=\"row_heading level0 row5\" >5</th>\n",
              "      <td id=\"T_c9b29_row5_col0\" class=\"data row5 col0\" >0</td>\n",
              "      <td id=\"T_c9b29_row5_col1\" class=\"data row5 col1\" >0</td>\n",
              "      <td id=\"T_c9b29_row5_col2\" class=\"data row5 col2\" >0</td>\n",
              "      <td id=\"T_c9b29_row5_col3\" class=\"data row5 col3\" >0</td>\n",
              "      <td id=\"T_c9b29_row5_col4\" class=\"data row5 col4\" >0</td>\n",
              "      <td id=\"T_c9b29_row5_col5\" class=\"data row5 col5\" >0</td>\n",
              "      <td id=\"T_c9b29_row5_col6\" class=\"data row5 col6\" >0</td>\n",
              "      <td id=\"T_c9b29_row5_col7\" class=\"data row5 col7\" >0</td>\n",
              "      <td id=\"T_c9b29_row5_col8\" class=\"data row5 col8\" >136</td>\n",
              "      <td id=\"T_c9b29_row5_col9\" class=\"data row5 col9\" >251</td>\n",
              "      <td id=\"T_c9b29_row5_col10\" class=\"data row5 col10\" >244</td>\n",
              "      <td id=\"T_c9b29_row5_col11\" class=\"data row5 col11\" >50</td>\n",
              "      <td id=\"T_c9b29_row5_col12\" class=\"data row5 col12\" >0</td>\n",
              "      <td id=\"T_c9b29_row5_col13\" class=\"data row5 col13\" >0</td>\n",
              "      <td id=\"T_c9b29_row5_col14\" class=\"data row5 col14\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_c9b29_level0_row6\" class=\"row_heading level0 row6\" >6</th>\n",
              "      <td id=\"T_c9b29_row6_col0\" class=\"data row6 col0\" >0</td>\n",
              "      <td id=\"T_c9b29_row6_col1\" class=\"data row6 col1\" >0</td>\n",
              "      <td id=\"T_c9b29_row6_col2\" class=\"data row6 col2\" >0</td>\n",
              "      <td id=\"T_c9b29_row6_col3\" class=\"data row6 col3\" >0</td>\n",
              "      <td id=\"T_c9b29_row6_col4\" class=\"data row6 col4\" >0</td>\n",
              "      <td id=\"T_c9b29_row6_col5\" class=\"data row6 col5\" >0</td>\n",
              "      <td id=\"T_c9b29_row6_col6\" class=\"data row6 col6\" >0</td>\n",
              "      <td id=\"T_c9b29_row6_col7\" class=\"data row6 col7\" >0</td>\n",
              "      <td id=\"T_c9b29_row6_col8\" class=\"data row6 col8\" >0</td>\n",
              "      <td id=\"T_c9b29_row6_col9\" class=\"data row6 col9\" >84</td>\n",
              "      <td id=\"T_c9b29_row6_col10\" class=\"data row6 col10\" >253</td>\n",
              "      <td id=\"T_c9b29_row6_col11\" class=\"data row6 col11\" >173</td>\n",
              "      <td id=\"T_c9b29_row6_col12\" class=\"data row6 col12\" >13</td>\n",
              "      <td id=\"T_c9b29_row6_col13\" class=\"data row6 col13\" >0</td>\n",
              "      <td id=\"T_c9b29_row6_col14\" class=\"data row6 col14\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_c9b29_level0_row7\" class=\"row_heading level0 row7\" >7</th>\n",
              "      <td id=\"T_c9b29_row7_col0\" class=\"data row7 col0\" >185</td>\n",
              "      <td id=\"T_c9b29_row7_col1\" class=\"data row7 col1\" >163</td>\n",
              "      <td id=\"T_c9b29_row7_col2\" class=\"data row7 col2\" >70</td>\n",
              "      <td id=\"T_c9b29_row7_col3\" class=\"data row7 col3\" >13</td>\n",
              "      <td id=\"T_c9b29_row7_col4\" class=\"data row7 col4\" >0</td>\n",
              "      <td id=\"T_c9b29_row7_col5\" class=\"data row7 col5\" >0</td>\n",
              "      <td id=\"T_c9b29_row7_col6\" class=\"data row7 col6\" >0</td>\n",
              "      <td id=\"T_c9b29_row7_col7\" class=\"data row7 col7\" >0</td>\n",
              "      <td id=\"T_c9b29_row7_col8\" class=\"data row7 col8\" >0</td>\n",
              "      <td id=\"T_c9b29_row7_col9\" class=\"data row7 col9\" >0</td>\n",
              "      <td id=\"T_c9b29_row7_col10\" class=\"data row7 col10\" >137</td>\n",
              "      <td id=\"T_c9b29_row7_col11\" class=\"data row7 col11\" >252</td>\n",
              "      <td id=\"T_c9b29_row7_col12\" class=\"data row7 col12\" >100</td>\n",
              "      <td id=\"T_c9b29_row7_col13\" class=\"data row7 col13\" >0</td>\n",
              "      <td id=\"T_c9b29_row7_col14\" class=\"data row7 col14\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_c9b29_level0_row8\" class=\"row_heading level0 row8\" >8</th>\n",
              "      <td id=\"T_c9b29_row8_col0\" class=\"data row8 col0\" >159</td>\n",
              "      <td id=\"T_c9b29_row8_col1\" class=\"data row8 col1\" >252</td>\n",
              "      <td id=\"T_c9b29_row8_col2\" class=\"data row8 col2\" >252</td>\n",
              "      <td id=\"T_c9b29_row8_col3\" class=\"data row8 col3\" >215</td>\n",
              "      <td id=\"T_c9b29_row8_col4\" class=\"data row8 col4\" >103</td>\n",
              "      <td id=\"T_c9b29_row8_col5\" class=\"data row8 col5\" >0</td>\n",
              "      <td id=\"T_c9b29_row8_col6\" class=\"data row8 col6\" >0</td>\n",
              "      <td id=\"T_c9b29_row8_col7\" class=\"data row8 col7\" >0</td>\n",
              "      <td id=\"T_c9b29_row8_col8\" class=\"data row8 col8\" >0</td>\n",
              "      <td id=\"T_c9b29_row8_col9\" class=\"data row8 col9\" >0</td>\n",
              "      <td id=\"T_c9b29_row8_col10\" class=\"data row8 col10\" >24</td>\n",
              "      <td id=\"T_c9b29_row8_col11\" class=\"data row8 col11\" >252</td>\n",
              "      <td id=\"T_c9b29_row8_col12\" class=\"data row8 col12\" >183</td>\n",
              "      <td id=\"T_c9b29_row8_col13\" class=\"data row8 col13\" >0</td>\n",
              "      <td id=\"T_c9b29_row8_col14\" class=\"data row8 col14\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_c9b29_level0_row9\" class=\"row_heading level0 row9\" >9</th>\n",
              "      <td id=\"T_c9b29_row9_col0\" class=\"data row9 col0\" >23</td>\n",
              "      <td id=\"T_c9b29_row9_col1\" class=\"data row9 col1\" >211</td>\n",
              "      <td id=\"T_c9b29_row9_col2\" class=\"data row9 col2\" >252</td>\n",
              "      <td id=\"T_c9b29_row9_col3\" class=\"data row9 col3\" >252</td>\n",
              "      <td id=\"T_c9b29_row9_col4\" class=\"data row9 col4\" >252</td>\n",
              "      <td id=\"T_c9b29_row9_col5\" class=\"data row9 col5\" >221</td>\n",
              "      <td id=\"T_c9b29_row9_col6\" class=\"data row9 col6\" >116</td>\n",
              "      <td id=\"T_c9b29_row9_col7\" class=\"data row9 col7\" >32</td>\n",
              "      <td id=\"T_c9b29_row9_col8\" class=\"data row9 col8\" >0</td>\n",
              "      <td id=\"T_c9b29_row9_col9\" class=\"data row9 col9\" >0</td>\n",
              "      <td id=\"T_c9b29_row9_col10\" class=\"data row9 col10\" >128</td>\n",
              "      <td id=\"T_c9b29_row9_col11\" class=\"data row9 col11\" >252</td>\n",
              "      <td id=\"T_c9b29_row9_col12\" class=\"data row9 col12\" >183</td>\n",
              "      <td id=\"T_c9b29_row9_col13\" class=\"data row9 col13\" >0</td>\n",
              "      <td id=\"T_c9b29_row9_col14\" class=\"data row9 col14\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_c9b29_level0_row10\" class=\"row_heading level0 row10\" >10</th>\n",
              "      <td id=\"T_c9b29_row10_col0\" class=\"data row10 col0\" >0</td>\n",
              "      <td id=\"T_c9b29_row10_col1\" class=\"data row10 col1\" >0</td>\n",
              "      <td id=\"T_c9b29_row10_col2\" class=\"data row10 col2\" >32</td>\n",
              "      <td id=\"T_c9b29_row10_col3\" class=\"data row10 col3\" >157</td>\n",
              "      <td id=\"T_c9b29_row10_col4\" class=\"data row10 col4\" >241</td>\n",
              "      <td id=\"T_c9b29_row10_col5\" class=\"data row10 col5\" >255</td>\n",
              "      <td id=\"T_c9b29_row10_col6\" class=\"data row10 col6\" >253</td>\n",
              "      <td id=\"T_c9b29_row10_col7\" class=\"data row10 col7\" >253</td>\n",
              "      <td id=\"T_c9b29_row10_col8\" class=\"data row10 col8\" >253</td>\n",
              "      <td id=\"T_c9b29_row10_col9\" class=\"data row10 col9\" >253</td>\n",
              "      <td id=\"T_c9b29_row10_col10\" class=\"data row10 col10\" >255</td>\n",
              "      <td id=\"T_c9b29_row10_col11\" class=\"data row10 col11\" >253</td>\n",
              "      <td id=\"T_c9b29_row10_col12\" class=\"data row10 col12\" >173</td>\n",
              "      <td id=\"T_c9b29_row10_col13\" class=\"data row10 col13\" >0</td>\n",
              "      <td id=\"T_c9b29_row10_col14\" class=\"data row10 col14\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_c9b29_level0_row11\" class=\"row_heading level0 row11\" >11</th>\n",
              "      <td id=\"T_c9b29_row11_col0\" class=\"data row11 col0\" >0</td>\n",
              "      <td id=\"T_c9b29_row11_col1\" class=\"data row11 col1\" >0</td>\n",
              "      <td id=\"T_c9b29_row11_col2\" class=\"data row11 col2\" >0</td>\n",
              "      <td id=\"T_c9b29_row11_col3\" class=\"data row11 col3\" >0</td>\n",
              "      <td id=\"T_c9b29_row11_col4\" class=\"data row11 col4\" >42</td>\n",
              "      <td id=\"T_c9b29_row11_col5\" class=\"data row11 col5\" >103</td>\n",
              "      <td id=\"T_c9b29_row11_col6\" class=\"data row11 col6\" >214</td>\n",
              "      <td id=\"T_c9b29_row11_col7\" class=\"data row11 col7\" >252</td>\n",
              "      <td id=\"T_c9b29_row11_col8\" class=\"data row11 col8\" >252</td>\n",
              "      <td id=\"T_c9b29_row11_col9\" class=\"data row11 col9\" >252</td>\n",
              "      <td id=\"T_c9b29_row11_col10\" class=\"data row11 col10\" >253</td>\n",
              "      <td id=\"T_c9b29_row11_col11\" class=\"data row11 col11\" >202</td>\n",
              "      <td id=\"T_c9b29_row11_col12\" class=\"data row11 col12\" >25</td>\n",
              "      <td id=\"T_c9b29_row11_col13\" class=\"data row11 col13\" >0</td>\n",
              "      <td id=\"T_c9b29_row11_col14\" class=\"data row11 col14\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_c9b29_level0_row12\" class=\"row_heading level0 row12\" >12</th>\n",
              "      <td id=\"T_c9b29_row12_col0\" class=\"data row12 col0\" >0</td>\n",
              "      <td id=\"T_c9b29_row12_col1\" class=\"data row12 col1\" >0</td>\n",
              "      <td id=\"T_c9b29_row12_col2\" class=\"data row12 col2\" >0</td>\n",
              "      <td id=\"T_c9b29_row12_col3\" class=\"data row12 col3\" >0</td>\n",
              "      <td id=\"T_c9b29_row12_col4\" class=\"data row12 col4\" >0</td>\n",
              "      <td id=\"T_c9b29_row12_col5\" class=\"data row12 col5\" >0</td>\n",
              "      <td id=\"T_c9b29_row12_col6\" class=\"data row12 col6\" >13</td>\n",
              "      <td id=\"T_c9b29_row12_col7\" class=\"data row12 col7\" >69</td>\n",
              "      <td id=\"T_c9b29_row12_col8\" class=\"data row12 col8\" >69</td>\n",
              "      <td id=\"T_c9b29_row12_col9\" class=\"data row12 col9\" >69</td>\n",
              "      <td id=\"T_c9b29_row12_col10\" class=\"data row12 col10\" >69</td>\n",
              "      <td id=\"T_c9b29_row12_col11\" class=\"data row12 col11\" >25</td>\n",
              "      <td id=\"T_c9b29_row12_col12\" class=\"data row12 col12\" >0</td>\n",
              "      <td id=\"T_c9b29_row12_col13\" class=\"data row12 col13\" >0</td>\n",
              "      <td id=\"T_c9b29_row12_col14\" class=\"data row12 col14\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_c9b29_level0_row13\" class=\"row_heading level0 row13\" >13</th>\n",
              "      <td id=\"T_c9b29_row13_col0\" class=\"data row13 col0\" >0</td>\n",
              "      <td id=\"T_c9b29_row13_col1\" class=\"data row13 col1\" >0</td>\n",
              "      <td id=\"T_c9b29_row13_col2\" class=\"data row13 col2\" >0</td>\n",
              "      <td id=\"T_c9b29_row13_col3\" class=\"data row13 col3\" >0</td>\n",
              "      <td id=\"T_c9b29_row13_col4\" class=\"data row13 col4\" >0</td>\n",
              "      <td id=\"T_c9b29_row13_col5\" class=\"data row13 col5\" >0</td>\n",
              "      <td id=\"T_c9b29_row13_col6\" class=\"data row13 col6\" >0</td>\n",
              "      <td id=\"T_c9b29_row13_col7\" class=\"data row13 col7\" >0</td>\n",
              "      <td id=\"T_c9b29_row13_col8\" class=\"data row13 col8\" >0</td>\n",
              "      <td id=\"T_c9b29_row13_col9\" class=\"data row13 col9\" >0</td>\n",
              "      <td id=\"T_c9b29_row13_col10\" class=\"data row13 col10\" >0</td>\n",
              "      <td id=\"T_c9b29_row13_col11\" class=\"data row13 col11\" >0</td>\n",
              "      <td id=\"T_c9b29_row13_col12\" class=\"data row13 col12\" >0</td>\n",
              "      <td id=\"T_c9b29_row13_col13\" class=\"data row13 col13\" >0</td>\n",
              "      <td id=\"T_c9b29_row13_col14\" class=\"data row13 col14\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_c9b29_level0_row14\" class=\"row_heading level0 row14\" >14</th>\n",
              "      <td id=\"T_c9b29_row14_col0\" class=\"data row14 col0\" >0</td>\n",
              "      <td id=\"T_c9b29_row14_col1\" class=\"data row14 col1\" >0</td>\n",
              "      <td id=\"T_c9b29_row14_col2\" class=\"data row14 col2\" >0</td>\n",
              "      <td id=\"T_c9b29_row14_col3\" class=\"data row14 col3\" >0</td>\n",
              "      <td id=\"T_c9b29_row14_col4\" class=\"data row14 col4\" >0</td>\n",
              "      <td id=\"T_c9b29_row14_col5\" class=\"data row14 col5\" >0</td>\n",
              "      <td id=\"T_c9b29_row14_col6\" class=\"data row14 col6\" >0</td>\n",
              "      <td id=\"T_c9b29_row14_col7\" class=\"data row14 col7\" >0</td>\n",
              "      <td id=\"T_c9b29_row14_col8\" class=\"data row14 col8\" >0</td>\n",
              "      <td id=\"T_c9b29_row14_col9\" class=\"data row14 col9\" >0</td>\n",
              "      <td id=\"T_c9b29_row14_col10\" class=\"data row14 col10\" >0</td>\n",
              "      <td id=\"T_c9b29_row14_col11\" class=\"data row14 col11\" >0</td>\n",
              "      <td id=\"T_c9b29_row14_col12\" class=\"data row14 col12\" >0</td>\n",
              "      <td id=\"T_c9b29_row14_col13\" class=\"data row14 col13\" >0</td>\n",
              "      <td id=\"T_c9b29_row14_col14\" class=\"data row14 col14\" >0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "This data frame really helps us to understand how computer see images. As you can see white pixels are stored as 0 while black is the number 255 and it shades over 0 to 255. Since we understand how computers sees a image, we can move on the our next part. In following parts, we try different ideas to make computer understand which image represent which number."
      ],
      "metadata": {
        "id": "WbsNn27eGPT4"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Pixel Similarity"
      ],
      "metadata": {
        "id": "YJJ8RohfGqz7"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Idea behind pixe similarity is finding the average pixel value for every pixel of digits.First step is getting the average of pixel values for each group."
      ],
      "metadata": {
        "id": "DuTc8sPlGsL-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "zero_tensors = [tensor(Image.open(o)) for o in zeros]\n",
        "one_tensors = [tensor(Image.open(o)) for o in ones]\n",
        "two_tensors = [tensor(Image.open(o)) for o in twos]\n",
        "three_tensors = [tensor(Image.open(o)) for o in threes]\n",
        "four_tensors =[tensor(Image.open(o)) for o in fours]\n",
        "five_tensors = [tensor(Image.open(o)) for o in fives]\n",
        "six_tensors = [tensor(Image.open(o)) for o in sixes]\n",
        "seven_tensors = [tensor(Image.open(o)) for o in sevens]\n",
        "eight_tensors = [tensor(Image.open(o)) for o in eights]\n",
        "nine_tensors = [tensor(Image.open(o)) for o in nines]\n",
        "\n",
        "len(nine_tensors),len(zero_tensors)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cLFPjVOYFFK5",
        "outputId": "3cc56716-e8e2-4dbe-97d1-97fb146c1fa7"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(5949, 5923)"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can also check that images look good. Since we have now tensors, we need to use fastai's show_image function to display it."
      ],
      "metadata": {
        "id": "6SYCl9sYGGN-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "show_image(five_tensors[1]);"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "6WVeeA6cGU7t",
        "outputId": "1f885ed8-fd7a-4750-f2b6-57a85307633d"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 72x72 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAGv0lEQVR4nO2by28bxwGHv5nd5WP5FCnrZVnyQ3DsOG4cx6n7sIuiRlEkQA859BKgx/bUQ0/9Ewr01AK99d4ibdMHisJGkRatUTRG4LR2AFu2JdmSbckiKYmiSGrJJXdnelAsxxtXOVhcEe5+AC+zS+6PH2Z2HjsrtNZEPEHudYB+IxISIBISIBISIBISwNzp4Dfld17YLuh99VvxrPKohgSIhASIhASIhASIhASIhASIhASIhASIhASIhATYcegeKtJAWCYiFkPELCjkUTl7+3B7KEknY6AlIED4IH1N6oGDUXPQSyXU5uZzx+gPIUIgUzYyZaMLObx8ksrrKepT6pPjmgtnb/C9oX9QlC5xAWU/RsnP8oO/fZfsdIb9FzXM3nvuKOELEQJhWsh8Dgo5VCZBNxunNWTRKkq8FHgJaB9yGRtd3/oK8EZ2nhHDfaqN+1oiNw3iNY3odHclXuhCZDyOSKfovjzO6skkzQmNeajJ21NX+eHgle0/bAmJwZMJaVV5NJTBgjfAmpem7OVYcgfIzQqK12vo9Y1dyRe+kOF9tF4aZu1EjMZrbYYG65wsLvPF9D0GZGL7PFd36WpFVSlqKsavql/j+vo4i2t5Opsx6EhERzCx4CFrTVSnsyv5QhfSemmY+2+aHH9tnnen/oD8pE4YQvDpTq+hPKrK4F+to9xxRvjzxbOMfOgzNV3BX1oGtbVUo30fT/m7li80IebkAZzjI1ROW4weL3G+OEdcWCgUvtb8xclxuX6MajfFumtTbdtsdiyqlSxGzWTopsZ+2ECvb6Bdt3c5e/bLATZfGWHxHY+vHL7Dj/dfJCMNIIavNa72+PmDCyxePkByRZNcVcQaisKmx8hCCW9pGbRChfAMKTQhXVsyNljjiL1KRhpYGAA4usuG0syXi4zc9InXPKwNF9nqQqeLbjRgF5vE5xGakE5a8tbIDGdS97BFbLvc0ZolP405lyT1uysAaCA8BU8T3j2krfm4Ns6oVYPkkxHlB60DvFd5ncTKM9d8Qye0uUysqbhVGmbaGXuq/P31E1z/cIrsAy+sKDsSmhCr4eE/tJmpD22XKRQLzQKph5L42u6MI56X8IRUHbJ3BffXCsCWjK72ebSeY2Cmi1VphBVlR/Z0+u+jaTdj2As1qNb2Mso2ezbb9bWmqxWibuHfmt2rGJ8hWiAKEAkJEAkJEAkJEJoQuVancNvFXbaZ99pUVQdLSHTCx9w/hsxkwoqyI6EJUbUN4rNlEisGj7wMDbV1aZHwUYM5ZDoFQjz92QNC63a166LWaxjuJG1tAZAQJu+8epV3f3QGvzGBsXlw+/zUosQuKfLTG3BvEd1qob3eD+/DE+J5aM9DutDWFgqBRPL9whXe/PLHlLw8JS+3ff4vZs6xMpsn1syQLtso33+xhDwm1tD8fvU03yrc5KhVoSBNLNNhzHBwYuXt8zLHWswcHOW9yVM8On+YwWua3EwDubCMv1btWb7QhViO5tbaCEfsVbqpZSSSjDTJfWptVSI5apUhU+ZQfIW/jx/jo9bLWK0UmdUUvEhC8h+Vabb28euTX+ePb3yBRjOJX7cwcx2K+SZfGl7gG9lpTsQqTJhJztt3ORKr8J8T46yoLPZSHu4/7Fm+0IX4c/Mk7y4w2jjNklUgvQp2RbE5ZrM2nOTycZP0pEvecJgwfQ6ZCSZNl2NDZa7tT+KlHy8+9oa9mdxpTWJ6kYPrg0i3i3Da5FNJlB1jczzHpX3nuPTt4/z0xG84bDYZNpKcK9ylc9SkVpwgJQT0aMF5z2a7XqkMpfJn1k4zcwNk81luvTLC9JH9FO05hg2Yipeo5lP8NTGJMAy07/dESt8N3VWjgSqvULwu+ck/3+KX62dpKpdT8Qpv5/5NJyNA9C523wnRnodyHNJLXdJzFrcbw7S1oiBNpiwfZfV2BNt3Qh6TvP6AA3+qcG12kg/aY6z44SxC960Qv1zBn51HbpiseBkc3cu+5Qn9sWHmGRj5HCKdRqV99pkNbOFDTzvcLfq2hohUClXMIpMeeWPrwZar1dZjvR7StzWkeXqclVMmX526wauxOjc6WW67o1hNDVr17Lp9W0O6KYlbUIwm6uRkggfdAteaE5htjVb6xRuYfR52uUN2LsHdM4OoIcXPZi7QvlpkYraO7uFugL6tIWajQ3JVcWd1iEvOALWHebLzCrnh9PS6YqcXmffyFTNp2wg7CcODeLkEZqUOG01Uvb4rO4j+1ytmfdtklOOA48DqGoLw9ov0bZPZKyIhAXa8h/w/EtWQAJGQAJGQAJGQAJGQAJGQAP8F6EqnNUxgsAsAAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "For each pixel location, we want to calculate the average of that pixel's intensity over all images. To do this, we combine all images in the list into single three-dimensional tensor. These type of tensors called rank-3 tensor. At this part, we will use stack function that comes with PyTorch. In addition, some operations in PyTorch, such as taking a mean, requires to use float types.\n",
        "Generally, when images are floats, the pixel values are expected to be between 0 and 1, so we will also divide by 255 here:\n"
      ],
      "metadata": {
        "id": "59Hfb5ryGd-T"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "stacked_zeros = torch.stack(zero_tensors).float()/255\n",
        "stacked_ones = torch.stack(one_tensors).float()/255 \n",
        "stacked_twos = torch.stack(two_tensors).float()/255\n",
        "stacked_threes = torch.stack(three_tensors).float()/255\n",
        "stacked_fours = torch.stack(four_tensors).float()/255\n",
        "stacked_fives = torch.stack(five_tensors).float()/255\n",
        "stacked_sixes = torch.stack(six_tensors).float()/255\n",
        "stacked_sevens = torch.stack(seven_tensors).float()/255\n",
        "stacked_eights = torch.stack(eight_tensors).float()/255\n",
        "stacked_nines = torch.stack(nine_tensors).float()/255\n",
        "\n",
        "\n",
        "stacked_threes.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "12zG9aYlIEj0",
        "outputId": "3ec38fef-a45e-4560-995c-2d3b1a0c0232"
      },
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([6131, 28, 28])"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Shape is one of the most important attributes of tensors. It tells us length of each axis. In this example, we have 6131 images each of size 28x28 pixels. We can also find out tensor rank with these methods:\n"
      ],
      "metadata": {
        "id": "NDkE1Q0iIrdp"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "len(stacked_threes.shape),stacked_threes.ndim"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2FZ34u4QJay7",
        "outputId": "66996da5-df3b-4fa1-ed3b-e47ef9bb538d"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(3, 3)"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Our next step is calculating the averages. We calculate the mean of the all image tensors by taking the mean along dimension 0 of our stacked, rank-3 tensor."
      ],
      "metadata": {
        "id": "IjrDKqiwJqv-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "mean1 = stacked_ones.mean(0)\n",
        "show_image(mean1);\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "RBlqwmZkJqXu",
        "outputId": "acb52f78-6c93-4de5-9902-53f04ea5624c"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 72x72 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAIbElEQVR4nO2cSZIjxxFFn8eQA6bqJs3YLdK00bDgQsfQEXRKHUE34UpmIiWqmxS7BiCHGFyLSAwFlmgkC0CRMnyzssRUCeTHj+8e7l4lqsoVe5iX/gC/NFwJOcKVkCNcCTnClZAjuB968s/mL/+3Iehv+a/y1ONXhRzhSsgRroQc4UrIEa6EHOEHo8zFIE8a/vdxgX3XyxAiAmIQI2AtYm05ioApzwGgefcrmqbbOYMqmhJkBc3odCwvfB5plyNkUsHu4p1DmhppGrStoa7IjUOtQa1BtheWFVGQkCBmJETIGdOPEAI6jGiM6DhCSoUo+NnEXFYhYooyRBDvEO/RtkZnNXlWkVpH9obsBKbrkaxIUmzISFTMEAs5gFgDKSNQyMgKoo+U9VNxGUIOlohYg1RVUcZyTr6ZEW5qxpVjXBhSDanae4qJiolgB8WOil97bJ/xlUO6gFGFoagDmJaQAfLPUsnlPcSY4hPWgjGoM6TKkCohNRAbIdX7l9tRyJGdYuwomCiolWI1P9aQfyQuv2S2R2PAWbK3pMYQZkJYCLGF2O6/WdsLJgJGUDsREgQVeUSGqhbD3T/wsz7iy4ddA2pAHWQPuSrH3dOpXJsaQIpFoCApQ1JIuUSgnNmVQ3/RHnIo6V1oPAinRshOypKpizrSfPs6AIMVwW2m02UwISMhI2OAECDG4iFZi7E+A5dXSH6cLxTpg1pIlZJrResMGdCyTErkECSBCYqJioSEhIhOZKhO0eUZ6oBLEKL6fePLOq0DBYFsIVWQGsizhF8OpGjRJGhnUJFJGWCHjN0EpB+g62EM+xxkq45nJGfn38s8EQUOWx8q07LxkOuMNIm2CfgqYlwhTLJggmKDYsaMGffq0Ekdu2z1mZnqxTd3OzKmFBwjZAe5Bp0l5suet8t75u2Ar2L5kBHsAK7LuHVA1j266UuWOgY0xJOQAS+12z0Ij3rgH24WWbU9HzdrGh8R0ck3wA2K6zIyhGKmWyM9gW8c4uW2/1tTtVOEmSmvVhs+W9zyu/k3LKuhEBIE24HfZPzdiHno0fUGHQZ0HMveZetHJ8D5CXnqgx56iJWSe7SJN4t73jZ3fOTWGFFSMtgR/EZxm4zpQknTt97xzBD7FF4g7CrYg7u+ZKduGfh89TWf1d/xqf8OZzIxWupOqO4nddxt0E2H9sNJfeMQFyVEsyKWvYdIWS5xriwXHb9v3rG0PZUkhuhIncNtoLpPmM0I/TBFlnwWMuAlU/dpLxIbISyVT1d3/Kn5B0EdvXq66JGNpbpXqg8Bc98V7xhHNIazVc8uZ6rHkcBacuUIc4HXI5/NPvDWbvAS6dVz39e4e0P1oNj7fqeO3Tb/THi5KOMsuXGEBbx+/cAfZu/4ja1oJNJnz7qrqG6F6jYW7+i6x1HlTLg8IVMtROuKsPKElfLb1S1v3C1eLL06vo43hPua+jvFP0S060sCdoaocowX8RCxFm0rwtISlpk/Lt/xdiJknT3vxhXmwdJ8mJZL158tqhzjcgrZlhCrCpm1xJua7iODLiOfVHfMzUDQxD/ja754+AR/Z6juEtKVIvIlyIAXqJiJc9A2hLljfCXU85E37pZGIkHh3+GGrx5u8A/g70akH8lnSsKewmV2u2IQa5HKI/MZ6dWC4ZVlvFHm7UAlibVWfJkCX6zf8P4/K/xDqbATY0nmLoSLKWRbbde2Ji0qwlwIC2VeBbwk+ux5T8u/uhXp1uM2IENC8+k2bj8G5yVkqw4/NaUWc9LNjOFjz7gSdB6oXWSdK75NC4Javry9wd9aXK9IzkUdRiBN5zuzj5xdIaUXY8FXaFORWs+4MMQWTJPwJtFrxTdhybdhzmbdUD0IdryMiR7jfISITL5RIW2DzFvSzYzxxjMuhThTrM2EbPlqfM2X3euyXO49ri+1U6Ze77bnu2tAwa8sdZ+WCts+rvdoXdQRWyG2pWQoRonZ8O244P2w4P16gekMJpQWZjmVlB6OmCfe47RNKjgHIYe+sW1ZLmbkZUNYOcLckBpQp6jCw1Dz9/VHfHW34sPdDNuV+imAuj2pGCnTAk8Rc0KclpDDHq5I6fBXHuqK1DhiW3q3uZqaU9kwRMuHvmXT18TOYUJpN6DsVCDTz/59zpctnM5DDsmoqpJzzFp03pKWNXHhCTNDbITsigJSMGw2NTFa+ocK6SxmnLpzMinEWfBuqp8qwrbLb/Y76BNGn5Oa6nYABpmO08Xk2pJqIVWlZammXHSOhqzCkAVGg4yCSRTfFKbGuJnMtYxRqBjE5LNlrqch5MBEd5GlqtCmJree2DpSbch+IiODBEE6ixpFjcV0BjsIUjoPqJQOv1qzmy76nxScMOKciJC9bxR1GHAO9Y7sLdkJ2ZaCMsLUrJ5IEUGNFu+IE1nbBpzZe4ia00eUp/B8QrZmN404yDQuhbOot+VbdhQiMkgEO5SlkeP0OGXEQQLYcSJr+60/EV5POVN2jNMnZtO3WtY9jy5GVJFU5j00A/lwUmgiIikm6dTs5uLZ6ukJyYrmjOSMhIQJCds7IKPGkB2kbgq722UgWyLAr7U0pTYR25UunYbSrtSUpxLi+bLV0xGi+6EVyWWQRWIhxQ4JFYuzSnZFDWUMYjsSMXlHVlyn2F4xQ0LGCDFBmvY1h4XqMynn+YSoAhndFsNTLqMJfY9sNljvsd9U1NaitQcR1Jh9SrhdXlM7UkJChtKh074nb3sxFyohnkYh0wyIprQPESkVYsaADCPYqVrGwf6k3Hl8rjgNwYxhR8RumfyqSoi7+a6E5lQIgv3E8nT7ezgOp2eYTv4pON/2/xFBZ3uXk+P61xBHuBJyhCshR7gScgS5/jOEx7gq5AhXQo5wJeQIV0KOcCXkCFdCjvBfyA24wDgJH38AAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Then we apply same procedure to all of the digits.\n"
      ],
      "metadata": {
        "id": "H0EUBxxXJTB2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "mean0 = stacked_zeros.mean(0)\n",
        "mean2 = stacked_twos.mean(0)\n",
        "mean3 = stacked_threes.mean(0)\n",
        "mean4 = stacked_fours.mean(0)\n",
        "mean5 = stacked_fives.mean(0)\n",
        "mean6 = stacked_sixes.mean(0)\n",
        "mean7 = stacked_sevens.mean(0)\n",
        "mean8 = stacked_eights.mean(0)\n",
        "mean9 = stacked_nines.mean(0)"
      ],
      "metadata": {
        "id": "3fLy9OaIKfX6"
      },
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Then, we pick arbitrary 4 and measure its distance from our \"ideal digits\"."
      ],
      "metadata": {
        "id": "WF9k1INuKx5x"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "a_4 = stacked_fours[4]\n",
        "show_image(a_4);"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "QHpw0cR5LE55",
        "outputId": "1f8a21e9-66d4-4e59-9164-ceafce456ea4"
      },
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 72x72 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAHDUlEQVR4nO2bS28b1xWAvztvvkSRFG3JslzJ70edOo8GFdJVgXrRZFOgRhYBugi6b9Efkx/QRdMiiyIoUBQtgqKJY8B2bdhNKjuBociJaYkWqQdfw+HMvbcL2mg8tuQm8TB0O99OImfO4Yf7OPcMRmitSfkPxredwLiRComRComRComRColh7fbhj41z/7Nb0F/VO+Jx/09HSIxUSIxUSIxUSIxUSIxUSIxUSIxUSIxdC7NxwDx5lGCmQJQxUY4g91kHs7GNam6ger2nHm+8hQjB8utTvHz2Y16rXOeUs8Zrf/4ley5MMHXeQi2vPPWQYytEWBbCcYjyitOFGqecNQ7bFmYhJChaaNdOJO7YriFGPodRrSCLEae9OxQNCUBxoou/R6OyTjJxE7nr08B10VkPMyOZNlt4YngWC0IbyxeISCUSdnynTDZDWMpSKnY57diATaAjei2P8rpG+INE4iYrRAjMPVXkwjTKNQmzFplaG710Cy0lPKnBLR5zQm9bZNclwg8SSTk5IUIgHIfw+CwrP/GIShGl6RbNDyrMrWRRQYAOvtqPUigyqyaF6zXUxmYiaScmxJrdR/e5fTRP2ez53hpFt8+U1+HiZBlcFyHlrkJUIUNQdsg5W8O/0YRagQaUevLo+rp5J3JXIDiyl9vnFC8e/pTfLPyJhhpwc1Digz3HELkMRBF0uzteH5aztOYsjmTbw/vpkLZWGBKQaiglARITEmVN9s00OTWxii1M+lpwNyoh+ga66z9xuoQFi6ACJWdYjS6FJkvBAeyWRnd7aPmMCQkmTM7tv8YLmRUMBFvK4Ya/D7tlIBuNJw55v2zi7w+Z84Zrxd+7x/lL/SS5ukJuJrN+QIJ1iBVoLm3PczOYefgDzX8//7+0ySg9mpIpsSh2R3KtNsuV9jyKZ6d5n5gQ05cEzQy1XhGAnIjY72wSZTVGoYBw3aRCfyOSE9Ib4K1a1LaLKBQFQ3LQraNyEqM4gZHxkgr9jUhOyEaH8k1Fa60AQNEwOWhtYOYjdDGP8HYXIj2wCwOKpg/AvUGBejuPOUhmd3lAYkLUvQaTl1fJ3LGQWpMXLkdtj0LeJyp64O0+ZaKMoDLZoWx1UGjqwQSd7QxG8IwKQUrw+9gduBR4fB4N64mFUpP1MzmC+alHrxECs1LGOjhPd05xdt9NjjhrANzamsL+3MVuJXOoe0BiQnQUoXs+dldz2V9gORourscLdbZPSTqzsX6GEAjThPIkwYEy5v4eb0xe4ogVAtDYLFD4HMx2P6mUgQQLMy0lyu9jdzUftWepWm3w1nCNCG0ptHk/tGEiTBNzbxU1VeSLs5PIl1u8eewiVUOQNZLpjO1EcqddrdHhALunWGlVuJMrA2vYQoKjUObwa8I0EZ6LrE7SXchjvbLBtZd+e/8mw4V3lHVM4g2ibM3n9ocz/OH7Nr8u/5Mf5j9h+0yGd8SLGNEivWlBb0ahyyGT5S3KWZ/Xl8/y8+kLvJrtJJ3eIyQuxFzbZO8Vj9vTk4TPS15w+pyeuoD/XZs/9p9n/lCdXxw4z7y9zpzV41crP+Xq9UMs5Jq8mr2SdHqPkLgQtbVNbskkf2KWt1uHOenVeMkZ8LPyZSqLXQ57dY47q7xV/xHvLx9G3/XI1Q1unyknndpjSV5Iu41qt8mtzvD+5lHssmTRrbHoShbdj+iogLZWXK3vJ38+ixlozFDT7OeSTu2xjKzJXPpXixtvn+DioWP87rka99p52tsZ6NhYLYPSEpSvb+DPFvCrD6elUGgFKEAlu8COTIhYqTEThHjNCstiFnfdYKqm8TYlmXoX606T6E4N1zuNX80/dK3UGq0EItkiFRihENX1MWp1SmFE/osiph9itPuIYIDu9VE7tBN9PaCtIpwVj+qFe7B6L9E8RyZEhwNkOIBWC+PWsE8kd/rul+pniaavwd0A+cmtxPMcuyd3QdWjPS84kE+uTbgbYyckzJkEJUXZHk6hUCv62kCMqFgdu0eZ/UlBdq7FvNcE4N3OId7bOEGmMYIVlXEbIUIQZQUzxRZVqwXAp/1pPl6fxu6NRsjYjBDhuhgZj853FG8tvMtBqwdk+dvdI0T/KJFZG825ZnyEmCbYDiov+YELkAWg47u4W2D4IaMYI+MzZZQCJUEKmson0BEAr8x9RnvRx5/NP+EGT4fxEQKgNCIwuBtZdPSwU7bXbVEqdpHeM/6g6quiggC53aK0JHjj6pv8vnUSgIvNeTZvVPCayfZSHzA2QtAalCS7rgiWJ3ivcZwPA4Pb6yUy6wKzG44kDbHba6rfxgtE5lQFSkVkKUeUd3AaXYztLqqxseN55+uw0wtEY7PLPEA2mtAYFmUW90/8I4w/PlNmTEiFxEiFxEiFxNh1l/l/JB0hMVIhMVIhMVIhMVIhMVIhMf4NU7/HhymdLp8AAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Data scientists use two main ways to measure distance in this context: mean absolute difference and root mean squared error.\n",
        "\n",
        "\n",
        "*   For mean absolute difference, just take the mean of the absolute value of differences.\n",
        "*   For RMSE, take the mean of the square of differences (which makes everything positive) and then take the square root (which undoes the squaring). \n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "X16F_QYaLRmu"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's try both of them:"
      ],
      "metadata": {
        "id": "htZOv7yHNHLC"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "dist_4_abs = (a_4 - mean4).abs().mean()\n",
        "dist_4_sqr = ((a_4 - mean4)**2).mean().sqrt()\n",
        "dist_4_abs,dist_4_sqr"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iolyr98FM3-6",
        "outputId": "71531f43-e30c-44b8-ec11-0cc2c6fd5223"
      },
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor(0.1028), tensor(0.1933))"
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "dist_7_abs = (a_4 - mean7).abs().mean()\n",
        "dist_7_sqr = ((a_4 - mean7)**2).mean().sqrt()\n",
        "dist_7_abs,dist_7_sqr"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FYJS-8DeNM13",
        "outputId": "690765eb-5035-47a8-cfc1-9aee6f9a53ca"
      },
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor(0.1180), tensor(0.2358))"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "As you can see, there is less difference with 4 according to 7. So our model will give right prediction in this case.\n",
        "\n",
        "PyTorch already provides both of these as loss functions. You’ll find these inside\n",
        "torch.nn.functional, which the PyTorch team recommends importing as F (and is\n",
        "available by default under that name in fastai):"
      ],
      "metadata": {
        "id": "h4wZdYpcNWce"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "F.l1_loss(a_4.float(),mean7), F.mse_loss(a_4,mean7).sqrt()\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GjU5kg4pNugZ",
        "outputId": "117f17fe-3fda-426f-809a-b91d273f8a65"
      },
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor(0.1180), tensor(0.2358))"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Here, MSE stands for mean squared error, and l1 refers to the standard mathematical jargon for mean absolute value."
      ],
      "metadata": {
        "id": "d376dV3qOyPf"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let’s create tensors for our digts from validation directory. These are the tensors we will use to calculate a metric measuring the quality of our first-try model, which measures distance from an ideal image:"
      ],
      "metadata": {
        "id": "RyAYeJ7gTTjs"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "valid_0_tens = torch.stack([tensor(Image.open(o))\n",
        " for o in (path/'testing'/'0').ls()])\n",
        "valid_0_tens = valid_0_tens.float()/255\n",
        "\n",
        "valid_1_tens = torch.stack([tensor(Image.open(o))\n",
        " for o in (path/'testing'/'1').ls()])\n",
        "valid_1_tens = valid_1_tens.float()/255\n",
        "\n",
        "valid_2_tens = torch.stack([tensor(Image.open(o))\n",
        " for o in (path/'testing'/'2').ls()])\n",
        "valid_2_tens = valid_2_tens.float()/255\n",
        "\n",
        "valid_3_tens = torch.stack([tensor(Image.open(o))\n",
        " for o in (path/'testing'/'3').ls()])\n",
        "valid_3_tens = valid_3_tens.float()/255\n",
        "\n",
        "valid_4_tens = torch.stack([tensor(Image.open(o))\n",
        " for o in (path/'testing'/'4').ls()])\n",
        "valid_4_tens = valid_4_tens.float()/255\n",
        "\n",
        "valid_5_tens = torch.stack([tensor(Image.open(o))\n",
        " for o in (path/'testing'/'5').ls()])\n",
        "valid_5_tens = valid_5_tens.float()/255\n",
        "\n",
        "valid_6_tens = torch.stack([tensor(Image.open(o))\n",
        " for o in (path/'testing'/'6').ls()])\n",
        "valid_6_tens = valid_6_tens.float()/255\n",
        "\n",
        "valid_7_tens = torch.stack([tensor(Image.open(o))\n",
        " for o in (path/'testing'/'7').ls()])\n",
        "valid_7_tens = valid_7_tens.float()/255\n",
        "\n",
        "valid_8_tens = torch.stack([tensor(Image.open(o))\n",
        " for o in (path/'testing'/'8').ls()])\n",
        "valid_8_tens = valid_8_tens.float()/255\n",
        "\n",
        "valid_9_tens = torch.stack([tensor(Image.open(o))\n",
        " for o in (path/'testing'/'9').ls()])\n",
        "valid_9_tens = valid_9_tens.float()/255\n",
        "\n",
        "\n",
        "valid_3_tens.shape,valid_7_tens.shape ##It’s good to get in the habit of checking shapes as you go. "
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cmwKIQ9dTc3d",
        "outputId": "79d5925d-afa4-445e-ffd0-b9f5793bcf81"
      },
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(torch.Size([1010, 28, 28]), torch.Size([1028, 28, 28]))"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Next,We can write a simple function that calculates the mean absolute error using an expression very similar to the one we wrote in the last section:"
      ],
      "metadata": {
        "id": "_jxFGZQJUjrI"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def mnist_distance(a,b): return (a-b).abs().mean((-1,-2))\n",
        "mnist_distance(a_4, mean4)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QgTO7AV2U3cD",
        "outputId": "b11bbbc3-598a-445b-f8a7-9cde2b614671"
      },
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(0.1028)"
            ]
          },
          "metadata": {},
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "In addition, we can also call this function with tensors. Let's try it and see what happens?"
      ],
      "metadata": {
        "id": "42yZnScdVIjK"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "valid_3_dist = mnist_distance(valid_3_tens, mean3)\n",
        "valid_3_dist, valid_3_dist.shape\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qbZpbbJWVP77",
        "outputId": "fde50667-bf80-4669-f7e7-729d4c387b59"
      },
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor([0.1366, 0.1234, 0.1268,  ..., 0.1224, 0.1778, 0.1430]),\n",
              " torch.Size([1010]))"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "As you can see, , it returned the distance for every single image as a vector (i.e., a rank-1 tensor) of length 1,010 (the number of 3s in our validation set). Then, we can define another function to understand which digit is the given picture."
      ],
      "metadata": {
        "id": "1z4TnqQFVUsd"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def is_what(x):\n",
        "  find_array = [mnist_distance(x,mean0),mnist_distance(x,mean1),mnist_distance(x,mean2),mnist_distance(x,mean3),mnist_distance(x,mean4),\n",
        "                mnist_distance(x,mean5),mnist_distance(x,mean6),mnist_distance(x,mean7),mnist_distance(x,mean8),mnist_distance(x,mean9)]\n",
        "  index, elem = min(enumerate(find_array), key=itemgetter(1))\n",
        "  return(index,elem)\n",
        "\n",
        "is_what(a_4)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "W6cHUShHW462",
        "outputId": "72fb4ba2-da45-41d0-ebb7-c4e77c690970"
      },
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(4, tensor(0.1028))"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "With this function, we can see that our model works or not and you can give couple of try to understand it better. Differs from the book example, we compare all digits. Therefore, accuracy will be way lower than we aim with this method. We need something better."
      ],
      "metadata": {
        "id": "PeIuaMwBcDMM"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Stochastic Gradient Descent"
      ],
      "metadata": {
        "id": "MU7K6XTmcbS4"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "In this part, I am not going to explain what is SGD since there is great part in fast.ai book which explains SGD very well. However, I will start with same example in the book to illustrate with basic concepts."
      ],
      "metadata": {
        "id": "zFQeQc8IcjdC"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def f(x): return x**2\n",
        "\n",
        "## With graph\n",
        "\n",
        "plot_function(f, 'x', 'x**2')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 279
        },
        "id": "fQdWvsmqd5_B",
        "outputId": "ccd7b04c-50d7-423a-fba8-1cfcf6b73afd"
      },
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU1f3/8dcnO0kgEJIAWSBAWGRfwr6IUC1uoCgqLkhBEZeq3b7VLrale21tXdpaRBRREUW0orhQ0YLsYQ07gZCNQEJCQvb1/P6YwV8ak5CE3NzJzOf5eMzDO3MPc98ZMJ+595x7jhhjUEop5bm87A6glFLKXloIlFLKw2khUEopD6eFQCmlPJwWAqWU8nA+dgdoqrCwMBMbG2t3DKWUalN27dp1zhgTXte+NlcIYmNjSUhIsDuGUkq1KSKSUt8+vTSklFIeTguBUkp5OC0ESinl4bQQKKWUh9NCoJRSHs7yQiAi3iKyR0Q+rGOfv4isEpEkEdkuIrFW51FKKfW/WuOM4DHgcD37FgDnjTFxwF+BP7ZCHqWUUjVYWghEJBq4HlhaT5OZwHLn9mpgmoiIFVmSsgpZvPYQ5ZXVVry9UkpZ6tn/HGf7yRxL3tvqM4K/Af8H1PfbNwpIAzDGVAL5QOfajURkoYgkiEhCdnZ2s4Kk5RazbHMyG46cbdafV0opu6TmFPPX/xxje3KuJe9vWSEQkRuALGPMrst9L2PMEmNMvDEmPjy8zjukL2ly33C6dgjgrZ1plxtHKaVa1dsJaXgJ3Doy2pL3t/KMYAIwQ0ROAW8BU0Xk9VptMoAYABHxAUIAS859vL2E2fHRbDyWzem8EisOoZRSLa6yqprVu9KZ3DecyI7tLDmGZYXAGPOkMSbaGBML3AFsMMbcXavZB8C9zu1bnW0sWztz9sgYqg2s3pVu1SGUUqpFbTyezZkLpdwxKsayY7T6fQQislhEZjifvgx0FpEk4PvAE1Yeu3vnQCbEdebthDSqq3WtZqWU61u1M43OQX5M7d/FsmO0SiEwxnxpjLnBuf2UMeYD53apMWa2MSbOGDPaGHPS6iy3xceQfr6ELSes6X1XSqmWklVQyueHs7hlZDR+Ptb9uva4O4u/PbArIe18eWtnqt1RlFKqQWt2Z1BZbbgt3rrLQuCBhSDA15ubh0fx2cGznC8qtzuOUkrVyRjD2zvTiO/RibiIYEuP5XGFAOD2UTGUV1WzZk+G3VGUUqpOO5JzOXmuiNss7CS+yCMLwRXdOjAspiMrd6Ri4SAlpZRqtpU7Umkf4MONQyItP5ZHFgKAOaNjSMoqZFfKebujKKXU/8grLmfdgTPcNCyKdn7elh/PYwvBDUMiCfb34c0d2mmslHIta3ZnUF5ZzZzR3VvleB5bCIL8fZg5LJKP9meSX1xhdxyllAIcncRv7UxlaExHBkR2aJVjemwhAJgzujtlldW8v1c7jZVSrmF36nmOnS1kTit0El/k0YVgUFQIg6NCtNNYKeUyVu5II8jPmxuHWt9JfJFHFwJwnBUcOVPAnrQ8u6MopTxcfkkFH+4/zczhUQT5+7TacT2+EMwYFkmQnzcrt2unsVLKXu/vyaC0opo5o1qnk/gijy8Ewf4+zBwexdr9p7XTWCllG2MMb2xPYWh0CIOjQ1r12B5fCADuHN2d0opq1uzR6amVUvZISHF0Et85pnXPBkALAeDoNB4W05E3tmunsVLKHm9sS6G9v0+rdhJfpIXA6c4x3UnKKmSHRWuCKqVUfXKLylmXeIZZI6II9Gu9TuKLtBA43TgkkvYBPryhncZKqVa2elca5VXV3Dmmhy3Ht3Lx+gAR2SEi+0TkoIj8qo4280QkW0T2Oh/3WZXnUtr5eXPLiGg+OXCGnMIyu2IopTxMdbVh5Q7HdNP9ura3JYOVZwRlwFRjzFBgGDBdRMbW0W6VMWaY87HUwjyXdNeY7pRXVfN2gnYaK6Vax5YTOSSfK7Klk/giKxevN8aYQudTX+fDpXti+3Rpz5ieoby5I4UqXdNYKdUKVmw7RadAX64b3M22DJb2EYiIt4jsBbKA9caY7XU0u0VE9ovIahGpc3INEVkoIgkikpCdnW1lZO4Z14O03BI2HrP2OEoplZlfwvpDZ7ltVAwBvtZPN10fSwuBMabKGDMMiAZGi8igWk3WArHGmCHAemB5Pe+zxBgTb4yJDw8PtzIy1wzoSnh7f1ZsS7H0OEoptXJ7Kga426ZO4otaZdSQMSYP+AKYXuv1HGPMxZ7ZpcDI1sjTED8fL+aMiuGLo1mk5RbbHUcp5abKK6tZuTONq/pFEBMaaGsWK0cNhYtIR+d2O+Bq4EitNjUvis0ADluVpynmjOmOl4gOJVVKWeazQ2fILijjnrH2ng2AtWcE3YAvRGQ/sBNHH8GHIrJYRGY42zzqHFq6D3gUmGdhnkbrFtKOb10RwdsJaZRWVNkdRynlhlZsTSEmtB2T+1p7ubsxLLuFzRizHxhex+tP1dh+EnjSqgyX456xsXx68CwfH8jk5uHRdsdRSrmRY2cL2J6cyxPX9sfbS+yOo3cW12d87870Cg9i+RbtNFZKtazXtp7Cz8eL2+JbbxWyhmghqIeXlzB3bA/2puWxTxetUUq1kAulFazZncGMoZGEBvnZHQfQQtCgW0ZGE+TnzfKtp+yOopRyE6sT0ikur2Le+Fi7o3xNC0ED2gf4csvIaD7cl6nzDymlLlt1tWHFthRGdO/IoKjWXXymIVoILmHuuB6UV1Xz1s40u6Mopdq4jcezST5XxL0udDYAWgguKS6iPRPjwnh9WwqVVdV2x1FKtWGvbU0hLNifawfZN69QXbQQNMLccT3IzC9l/aGzdkdRSrVRKTlFfHE0izvHdMfPx7V+9bpWGhc17YouRHdqxytbTtkdRSnVRi3fkoK3CHfZON10fbQQNIK3lzB3XA92JOdy8HS+3XGUUm1MYVkl7ySkcd3gbnTpEGB3nG/QQtBIt8d3p52vN69uPmV3FKVUG/PurnQKyir5zoRYu6PUSQtBI4UE+nLLyCj+ve+0DiVVSjVadbXh1S2nGBbTkeHdO9kdp05aCJpg3vhYyiureVNnJVVKNdJ/jzmGjLrq2QBoIWiSuIj2TOoTxoptKZRX6lBSpdSlLducTER71xsyWpMWgiaaP6EnWQVlfHwg0+4oSikXl5RVwKbj57hnbA+XGzJak+smc1FX9g2nV1gQy75Kxhhd4F4pVb9XNjtmGZ3jgkNGa9JC0EReXsJ3JsSyLz2fXSnn7Y6jlHJR54vKeXd3OjcPiyIs2N/uOA2ycqnKABHZISL7nKuQ/aqONv4iskpEkkRku4jEWpWnJd0yMpqQdr4s3ZRsdxSllIt6c0cqpRXVLJjU0+4ol2TlGUEZMNUYMxQYBkwXkbG12iwAzhtj4oC/An+0ME+LCfTz4c4x3fns0BlSc3SBe6XU/yqvrGb5llNM6hNG3y7t7Y5zSZYVAuNQ6Hzq63zUvqg+E1ju3F4NTBMR+9dta4R7x8XiJcIrW/SsQCn1vz7cf5qsgjLum9TL7iiNYmkfgYh4i8heIAvH4vXbazWJAtIAjDGVQD7QuY73WSgiCSKSkJ2dbWXkRusaEsANQ7rx9s40LpRW2B1HKeUijDEs3ZRMn4hgJvcJsztOo1haCIwxVcaYYUA0MFpEBjXzfZYYY+KNMfHh4eEtG/IyLJjYi6LyKlbt0LUKlFIO207mcijzAvMn9qSNXOBonVFDxpg84Atgeq1dGUAMgIj4ACFATmtkagmDo0MY3TOUV7ecokLXKlBKAUs3nSQ0yI+bh0fZHaXRrBw1FC4iHZ3b7YCrgSO1mn0A3OvcvhXYYNrY4PyFk3qRkVfCukS9wUwpT5eUVcDnR7KYO64HAb7edsdpNCvPCLoBX4jIfmAnjj6CD0VksYjMcLZ5GegsIknA94EnLMxjian9I+gdHsSSjSf1BjOlPNzSTcn4+3hxz9gedkdpEh+r3tgYsx8YXsfrT9XYLgVmW5WhNXh5CfdP6sUTaxLZeiKH8XFto3NIKdWysgpKWbM7g9tGRdPZxW8gq03vLG4BNw2PIizYjyWbTtodRSllk9e2pFBRXc2CiW1jyGhNWghaQICvN/eOi+XLo9kcPVNgdxylVCsrLq9kxbYUrhnQhZ5hQXbHaTItBC3k7rE9CPD14iU9K1DK47yTkE5+SQULJ7e9swHQQtBiOgX5cXt8DP/em0FmfondcZRSraSyqpqXNp1kZI9OjOwRanecZtFC0ILum9SLagPLvtJpJ5TyFB8lZpJ+voRFV/a2O0qzaSFoQTGhgdwwpBtvbk8lv1innVDK3RljePG/J4mLCGZa/wi74zSbFoIW9sDk3hSVV/H69hS7oyilLPbfY9kczrzAwsm98PJqG9NJ1EULQQsbENmBK/uG88rmZEorquyOo5Sy0Iv/PUHXDgHcNKztTCdRFy0EFlh0ZW/OFZazele63VGUUhbZm5bHtpO5LJjY06XXI26Mtp3eRY3tFcrQmI4s2XiSSp2MTim39OKXJ2gf4OPy6xE3hhYCC4gID03pTWpuMR/pZHRKuZ2krAI+OXiGeeNjCfa3bKaeVqOFwCJXX9GFPhHB/PPLEzoZnVJu5p9fnqSdrzffmeD66xE3hhYCi3h5CQ9d1ZsjZwrYcCTL7jhKqRaSllvM+3szmDO6O6FBfnbHaRFaCCx045BIoju144UvkvSsQCk38dKmk3gJ3D/ZPc4GQAuBpXy8vXjgyt7sSXWMLlBKtW1ZBaW8tTONWcOj6RbSzu44LcbKFcpiROQLETkkIgdF5LE62kwRkXwR2et8PFXXe7Vls0dGExbsz9+/SLI7ilLqMi376hSVVdUsmtJ2p5Ooi5VnBJXAD4wxA4CxwMMiMqCOdpuMMcOcj8UW5rFFgK83Cyf35Kukc+xJPW93HKVUM50vKmfF1lNcN7hbm5xquiGWFQJjTKYxZrdzuwA4DLTt2++a6a4xPegU6MvzG/SsQKm26pXNyRSVV/HI1Di7o7S4VukjEJFYHMtWbq9j9zgR2SciH4vIwHr+/EIRSRCRhOzsbAuTWiPI34cFE3uy4UgWBzLy7Y6jlGqiC6UVvLLlFNMHdqV/1w52x2lxlhcCEQkG3gUeN8ZcqLV7N9DDGDMUeB54v673MMYsMcbEG2Piw8PDrQ1skbnjY+kQ4MPzG47bHUUp1UTLN5+ioLTSLc8GwOJCICK+OIrAG8aYNbX3G2MuGGMKndvrAF8RccvV3zsE+DJvQk8+PXiWw5m166FSylUVllXy8uZkpvWPYFBUiN1xLGHlqCEBXgYOG2OeqadNV2c7RGS0M0+OVZnsNn9CLEF+3rygI4iUajNe35ZCXnEF353Wx+4olrFykowJwD1Aoojsdb72E6A7gDHmReBW4EERqQRKgDuMG9951THQj7njY3nxvyc4fraAPl3a2x1JKdWA4vJKXtp4kkl9whgW09HuOJaxrBAYY74CGlypwRjzAvCCVRlc0f2TerF8yyme/fw4L9w5wu44SqkGrNiaQk5ROY9/y33PBkDvLG51oUF+3Ds+lo8SMzl2tsDuOEqpehSXV/Iv59lAW12UvrG0ENjg/km9CPT15rnPdQSRUq7qta0p5BaV8/i3+todxXJaCGygZwVKubaiskqWbDzJ5L7hjOzRye44ltNCYJOLZwXP6lmBUi7n/58NuHffwEVaCGzSKciPeRNiWZeYyZEzel+BUq6isKySJRtPMLlvOCO6u//ZAGghsNX9k3oR7OfDX9cfszuKUsrpla+SOV9cwQ+udv++gYu0ENioY6AfCyY57jZOTNc5iJSyW35xBUs2neRbV3RhqBvfN1CbFgKbzZ/Yk46Bvvxl/VG7oyjl8V7adJKC0kq+70FnA6CFwHYdAnx5YHJvvjyaza4UXcVMKbvkFJaxbHMy1w/pxoBI95thtCFaCFzAveN7EBbsx18+074Cpezyr40nKa2o4nseMlKoJi0ELiDQz4eHpsSx5UQOm5PO2R1HKY9zJr+U5VtOcdOwKOIiPG8OsAYLgYh0EJFvLM4pIkOsi+SZ7hzTnciQAP70yRHceN49pVzScxuOU20M3/OwvoGL6i0EInIbcAR417n4/Kgau1+1OpinCfD15vGr+7IvPZ9PD56xO45SHiP5XBGrdqZx5+juxIQG2h3HFg2dEfwEGGmMGQZ8B1ghIjc79zU4q6hqnlnDo+gdHsSfPztGZVW13XGU8gjPrD+Gn7cXj0z1vL6BixoqBN7GmEwAY8wO4CrgZyLyKKDXLizg4+3FD6/pR1JWIWv2ZNgdRym3dyAjn7X7TjN/Yizh7f3tjmObhgpBQc3+AWdRmALMBOpcZF5dvumDujIkOoS/rT9GaUWV3XGUcmt//uwoIe18WTj5G12hHqWhQvAgtS4BGWMKgOnA/Eu9sYjEiMgXInLI2cfwWB1tRESeE5EkEdkvIh6/UouI8OPp/TmdX8rr21LsjqOU29p6Iocvj2bz4JTehLTztTuOreotBMaYfUAsgIhMq/F6hTHmjUa8dyXwA2PMAGAs8LCIDKjV5lqgj/OxEPhnk9K7qQlxYUzqE8bzG5LIL66wO45Sbqe62vD7jw/TLSSAeeNj7Y5ju0vdR3CliEzAcUmoSYwxmcaY3c7tAuAwEFWr2UzgNeOwDegoIt2aeix39MS1/blQWsE//qsL3SvV0j5KzGR/ej4/uKYfAb7edsexXUPDR38B+AP/AfxE5KnmHkREYoHhwPZau6KAtBrP0/lmsUBEFopIgogkZGdnNzdGmzIwMoSbh0XxyuZTnM4rsTuOUm6jvLKapz89Sv+u7bl5+Dd+3Xikhi4N/QrHfQS/BI4YYxY35wAiEgy8CzxujGnWxPvGmCXGmHhjTHx4eHhz3qJN+v41jptbntFpqpVqMW9uTyE1t5gfX9sfby8dCQ+XvjTU3hjzR6BZ91yLiC+OIvCGMWZNHU0ygJgaz6OdrykgulMg88bH8u7udA5n6uI1Sl2uC6UVPLchiXG9OjOlr+d8qbyUSxWCg7X+22giIsDLwGFjzDP1NPsAmOscPTQWyL9474JyeHhKHCHtfPntR4d16gmlLtM/vjhBblE5P7nuChy/ohRY2FkMTADuAaaKyF7n4zoRWSQii5xt1gEngSTgJeChZhzHrYUE+vLo1D58lXSOL496Rv+IUlZIyy1m2eZkZg2PYnB0iN1xXIpPfTtqdRY/JyJPNaWfwBjzFZeYisI4vuI+3Nj39FR3j+3Bim0p/HbdYSb1CcPHWyeNVaqp/vTpUbwEfvjtfnZHcTmWdxary+fn48UT1/YnKauQlTvTLv0HlFL/Y0/qedbuO83CSb2I7NjO7jgu51JfLTsAa4Hgmi+KyBSrAqm6XTOgC6N7hvK39ce4UKo3mSnVWMYYfvPRYcLb+/PAlZ49lUR9GiwExpi/AW8Dgc4O3XYi8jzw+1ZJp74mIvz8+gHkFJXz9w16k5lSjfXh/kx2pZznB1f3Jci/3qvhHq0xF5vH4BjiuQXYCZzG0RGsWtng6BBmj4xm2eZkks8V2R1HKZdXUl7FHz4+wsDIDsyOj7n0H/BQjSkEFUAJ0A4IAJKNMTpZvk1+NL0fft5e/Pajw3ZHUcrlLdl4koy8En5x40C9eawBjSkEO3EUglHAJGCOiLxjaSpVr4j2ATwytQ//OXyWTcd1OKlS9TmdV8I//5vE9UO6MbpnqN1xXFpjCsECY8xTzllHM40xM3HcCKZsMn9iLD06B7J47SFdyUypevzh4yMYA09e29/uKC7vkoXAGJNQx2srrImjGsPfx5ufXncFx7MKWaFrFij1DTtP5fLBvtM8cGVvojt55jrETaF3JrVRVw/owqQ+YTyz/hjnCsvsjqOUy6isqubn7x8gMiSARVf2sjtOm6CFoI0SEX45YyClFVX88eMjdsdRymW8sT2VI2cK+PkNAwj00+GijaGFoA3rHR7M/Ik9eWdXOrtTz9sdRynbnSss4y+fHWViXBjTB3W1O06boYWgjXt0ah+6dPDnF/8+SFW1zk6qPNvTnxyluLyKX84YoLOLNoEWgjYuyN+Hn14/gMSMfFbuSLU7jlK22ZN6nlUJaSyY2JO4iGYtoeKxtBC4gRuHdGN878786ZMj2nGsPFJlVTU/fe8AXTsE8N1pfeyO0+ZoIXADIsLimYMoqajid3rHsfJAy7emcCjzAr+4cQDBOp9Qk2khcBNxEcE8MLk3a/ZksOXEObvjKNVqzuSX8sxnR5nSL1w7iJvJskIgIstEJEtEDtSzf4qI5NdYvewpq7J4ikemxtE9NJCfv3+A8kq941h5hl9/eIjKasPiGYO0g7iZrDwjeBWYfok2m4wxw5wPXfjmMgX4evOrmQM5kV3Eko0n7I6jlOW+PJrFR4mZfHdqHN076x3EzWVZITDGbARyrXp/Vber+kVw/eBuPLchiZPZhXbHUcoyxeWV/Oz9A8RFBHP/ZL2D+HLY3UcwTkT2icjHIjKwvkYislBEEkQkITtbZ9y8lF/cOAB/Hy9+8l4ijmWhlXI/z3x2jPTzJfx+1mD8fbztjtOm2VkIdgM9jDFDgeeB9+traIxZYoyJN8bEh4eHt1rAtiqiQwA/ue4Ktp3M5Z2EdLvjKNXiEtPzWbY5mTvHdGdUrE4xfblsKwTGmAvGmELn9jrAV0TC7Mrjbm6Pj2F0bCi/XXeY7AK9t0C5j8qqap5Ys5+wYH9+PF2nmG4JthUCEekqzi5+ERntzJJjVx534+Ul/G7WYErKq/jl2oN2x1GqxSz9KpmDpy/wqxkDCWnna3cct2Dl8NGVwFagn4iki8gCEVkkIoucTW4FDojIPuA54A6jF7RbVFxEMI9Oi+Oj/Zl8cuCM3XGUumwnsgt5Zv0xrhnQRe8ZaEGW3YJnjJlzif0vAC9YdXzl8MCVvVmXeIaf//sAY3uF0jHQz+5ISjVLVbXh/1bvp52vN7+5Se8ZaEl2jxpSFvP19uLp2UM4X1TO4g8P2R1HqWZ7bespdqWc56kbBhDRIcDuOG5FC4EHGBgZwoNTerNmdwZfHMmyO45STZaaU8yfPnFMIzFrRJTdcdyOFgIP8cjUOPpEBPPkmkTyiyvsjqNUo1VXG360eh/eXsLvbh6sl4QsoIXAQ/j7ePOX24aSXVimo4hUm/LKllNsT87lqRsHENmxnd1x3JIWAg8yJLojD18Vx3t7MvjkQKbdcZS6pKSsQv70yRGm9Y9g9shou+O4LS0EHuaRq+IYGNmBn753QBexUS6tsqqaH7yzj3Z+3vx+ll4SspIWAg/j5+PFM7cNo6C0kp/qXETKhf3zyxPsS8vjNzcN0lFCFtNC4IH6dW3PD67py6cHz+pcRMol7UvL49nPj3Pj0EhuGBJpdxy3p4XAQ903qRdje4Xyy7UHSckpsjuOUl8rLq/k8VV7iWjvz29mDrI7jkfQQuChvL2EZ24bho+X8PiqvVRW6YpmyjX8+sPDnMop4i+3DSMkUOcSag1aCDxYZMd2/PbmwexJzeP5DUl2x1GK9YfOsnJHKgsn92Jc7852x/EYWgg83I1DI5k1PIrnNxxnR7IuKKfscya/lP9bvY8B3Trw/av72h3Ho2ghUCy+aRDdQwN57K095BWX2x1HeaCqasNjb+2hrLKa5+8criuOtTItBIpgfx+enzOCc4Vl/Gj1fh1SqlrdCxuS2J6cy+KZg+gdHmx3HI+jhUABMDg6hB9P78/6Q2dZsS3F7jjKg+xIzuXZz49x8/AobtEJ5Wxh5cI0y0QkS0QO1LNfROQ5EUkSkf0iMsKqLKpxFkzsydT+Efzmw8MkpufbHUd5gJzCMh5duYfuoYH8WtcYsI2VZwSvAtMb2H8t0Mf5WAj808IsqhFEhD/PHkrnYD8eenOXzlKqLFVVbXh81V5yi8v5+10jCPa3bJ0sdQmWFQJjzEagoWEoM4HXjMM2oKOIdLMqj2qc0CA/XrhzBJl5pfxw9T7tL1CWeX7DcTYdP8evZgxkYGSI3XE8mp19BFFAWo3n6c7XlM1G9ujEk9ddwfpDZ3lp00m74yg39NXxczz7+XFmDY/ijlExdsfxeG2is1hEFopIgogkZGdn2x3HI8yfEMu1g7ryx0+OsvVEjt1xlBvJyCvh0bf2EBcezG9u1n4BV2BnIcgAan4ViHa+9g3GmCXGmHhjTHx4eHirhPN0IsKfbh1CbOdAHnlzN6fzSuyOpNxAaUUVi1bsoqKymhfvGUmgn/YLuAI7C8EHwFzn6KGxQL4xRldLcSHtA3z51z3xlFVW8+DruyitqLI7kmrDjDH87P0DJGbk88ztw/R+ARdi5fDRlcBWoJ+IpIvIAhFZJCKLnE3WASeBJOAl4CGrsqjmi4sI5i+3DWVfej5P/fuAdh6rZnt9Wwqrd6Xz6LQ+XD2gi91xVA2WnZcZY+ZcYr8BHrbq+KrlfHtgV747NY7nNyQxoFsH5k3oaXck1cZsPZHDr9YeYmr/CB6f1sfuOKqWNtFZrOz3vW/15eoBXVj84SE2HtMOe9V4qTnFPPjGLmLDgvjbHcPw8tLOYVejhUA1ipeX8Nfbh9G3S3sefnM3J7IL7Y6k2oCC0goWLN8JwNK58XQI0PUFXJEWAtVowf4+vDQ3Hj9vL+5fnqAzlaoGOWYU3UvyuSL+cdcIYsOC7I6k6qGFQDVJTGggL94zkvTzJTywYhdllTqSSH2TMYZfrT3IhiNZ/HLGQMb3DrM7kmqAFgLVZKNiQ3l69hC2J+fyxLuJOpJIfcPLXyXz2tYUFk7uxd1je9gdR12C3s2hmmXmsCjScov582fHiAkN1BWl1Nc+OXCG3647zLWDuvLE9P52x1GNoIVANdvDV8WRmlvMc58fJ6pjALeP6m53JGWzXSm5PPbWHobFdOSvt+sIobZCC4FqNhHhtzcPJqugjCfXJNIp0I9rBna1O5ayybGzBcx/NYHIju1YOjeeAF9dbrKt0D4CdVl8vb34x10jGBzdke+u3MOO5IZmHlfuKiOvhLkv78Dfx4vX5o+mc7C/3ZFUE2ghUJct0M+HV+aNIqpTOxYs38nhzAt2R1KtKKewjLkvb6eorJLl80cTExpodyTVRFoIVJSIbzwAAA+wSURBVIsIDfLjtfmjCfb34Z6Xt+sNZx4iv6SCuct2kH6+hKX3xnNFtw52R1LNoIVAtZjoToG8ft8YAO5eup203GKbEykrFZVVMv/VnRw7W8CL94xkTK/OdkdSzaSFQLWo3uHBrFgwhuLyKu5aup0z+aV2R1IWKK2oYuGKBPam5fH8nOFc1S/C7kjqMmghUC3uim4dWD5/NLlF5cx5aZsWAzdTWlHF/a8lsOVEDk/fOoTpg3Sp8bZOC4GyxLCYjiyfP5rsgjItBm7kYhH4Kukcf7plCLNGRNsdSbUALQTKMiN7dPq6GNyxZCuZ+brcZVtWUl7FfcsdReDpW4cyO14XnXcXlhYCEZkuIkdFJElEnqhj/zwRyRaRvc7HfVbmUa1vZI9OvLZgNDmF5cx+cSspOUV2R1LNUFBawb3LdrD5hKMI3DpSzwTciZVLVXoDfweuBQYAc0RkQB1NVxljhjkfS63Ko+wzonsn3rx/LEVllcx+cSvHzxbYHUk1wfmicu5aup3dqed57o7hWgTckJVnBKOBJGPMSWNMOfAWMNPC4ykXNjg6hFUPjAPgtn9tZX96ns2JVGOcvVDK7Uu2cuRMAf+6ZyQ3Do20O5KygJWFIApIq/E83flabbeIyH4RWS0idV50FJGFIpIgIgnZ2bpMYlvVt0t73lk0jiB/H+5Yso0vj2bZHUk1ICmrgFn/2EL6+RJenTeKaVfogvPuyu7O4rVArDFmCLAeWF5XI2PMEmNMvDEmPjw8vFUDqpbVo3MQax4cT2znIO5bnsDqXel2R1J1SDiVyy3/3EpZZTWrFo5jfJwuLOPOrCwEGUDNb/jRzte+ZozJMcaUOZ8uBUZamEe5iIgOAax6YCxje3Xmh+/s47nPj+viNi7k48RM7lq6ndAgP9Y8OJ7B0SF2R1IWs7IQ7AT6iEhPEfED7gA+qNlARGreiTIDOGxhHuVC2gf4smzeKGYNj+KZ9cd4fNVeSit02Us7GWN4YcNxHnxjNwMjO/Dug+Pp3lknkPMElq1HYIypFJFHgE8Bb2CZMeagiCwGEowxHwCPisgMoBLIBeZZlUe5Hj8fL/5y21B6RwTz9KdHSc0tZsk98YS31ymMW1tpRRVPrknkvT0Z3DQskj/cMkTXE/Ag0tZOyePj401CQoLdMVQL+zgxk++9vZdOgX78464RDO/eye5IHuN0XgkPvr6Lfen5/PCavjx8VRwiurKYuxGRXcaY+Lr22d1ZrBQA1w7uxupF4/H2Em7/1zZW7ki1O5JH2HLiHDc+/xUnsot48e6RPDK1jxYBD6SFQLmMQVEhrH1kImN6hfLkmkR+9M4+issr7Y7llqqrDf/88gR3L91Ox0Bf3n94AtMH6TKjnkrXLFYupVOQH69+ZzR/XX+Mv3+ZxJ60PF64czj9u+qCJy0lu6CM77+9l03Hz3H94G788dYhBPvrrwJPpmcEyuV4ewk//HY/VswfQ15xBTNf2MyKbSk6xLQFbDyWzbXPbmJHci6/u3kwL9w5XIuA0kKgXNfEPmF8/NgkxvTqzM/fP8C9r+zU6aybqbi8kp+9n8jcZTvoFOjLvx+ZwJ1jumt/gAK0ECgXF97en1fnjeLXMweyMzmXa/76X97bk65nB02w81Qu1z67iTe2p3LfxJ6s/e5EvdSm/ocWAuXyvLyEe8bFsu6xSfTp0p7vrdrHva/sJDVH10RuSH5xBU+uSWT2i1upqjasvH8sP7thgN4foL5B7yNQbUpVtWHF1lM8/elRqozhsWl9WTCxJ34++p3mImMMa/dnsnjtIXKLylgwsSffu7ovgX7aF+DJGrqPQAuBapNO55Xwiw8Osv7QWXqGBfHT665g2hURHn/NOzE9n8UfHmTnqfMMjgrh97MGMyhK5wpSWgiUG/viaBa//vAQJ7OLmNQnjB9P7++Rv/gy8kr42/pjrN6dTmigHz/8dj9ui4/B28uzC6P6/7QQKLdWUVXNa1tTeO7z4+SXVHD94G58/5q+9A4Ptjua5c4VlvGPL07w+rYUAOaO68Gj3+pDhwBfm5MpV6OFQHmEC6UVLN14kqVfJVNaUcX1QyJZdGUvBka63xnC6bwSXtp0krd2pFFWWcWtI6N57Ft9ierYzu5oykVpIVAe5VxhGS9tOskb21IpLKtkSr9w5k/oycS4MLza+KWSAxn5vLrlFP/em0G1gZnDInloShxxEe5/9qMujxYC5ZHyiytYse0Ur245xbnCcnqGBXH32B7MGh5FpyA/u+M1Wkl5FZ8ePMNrW0+xOzWPdr7e3BYfzf2TexHdSdcLUI2jhUB5tLLKKj5OPMPyrafYk5qHr7dwVb8IZo2IYkq/CJccV19VbdiRnMt7e9JZl3iGwrJKYjsHcs+4WG4dGU1IO+0DUE3TUCHQgcXK7fn7eHPT8ChuGh7FodMXWLM7nff3nuazQ2cJ9PNmSr9wrhnQlcl9wwm18UyhuLySbSdz+PTAWf5z+Cw5ReUE+Xlz7eBuzBoexdhendv8pS3lmiw9IxCR6cCzOFYoW2qM+UOt/f7AazjWKs4BbjfGnGroPfWMQLWEyqpqtpzI4dODZ/js0FmyCxxLZw/o1oGJfcKI79GJoTEd6dIhwLIM+cUV7M/IY3dKHptPnGNP6nkqqgzB/j5c1T+CawZ0YdoVEXojmGoRtlwaEhFv4BhwNZCOYw3jOcaYQzXaPAQMMcYsEpE7gJuNMbc39L5aCFRLq6427EvPY3PSOTYn5bAr5TzlVdUAdO0QQP9u7YkLD6Z3RDAxnQLpGuJPlw4BtG/EEM3SiirOXijlTH4pGXklnMgu5ERWEUfPFpB8rggAERgUGcL4uM5M6B3GmF6h+Pu43uUq1bbZVQjGAb80xnzb+fxJAGPM72u0+dTZZquI+ABngHDTQCgtBMpqpRVVHMq8wN7UPPal53HsbCEnswspq6z+n3Z+3l4EB/gQ5O+Nv483Fy/aVFRVU1hWRWFZBaUV//tnfLyEHp0DiYsIZkh0R4bFdGRwdIiO+1eWs6uPIApIq/E8HRhTXxvnYvf5QGfgXM1GIrIQWAjQvXt3q/IqBUCArzcjundiRI11k6urDRl5JWTklXz9Df98cQWFZRUUlVVRVln1dVsfLy+C/H1oH+BDhwAfunQIoGtIAN1C2tGjcyC+3jovknItbeLiozFmCbAEHGcENsdRHsjLS4gJDSQmVIdrKvdj5VeTDCCmxvNo52t1tnFeGgrB0WmslFKqlVhZCHYCfUSkp4j4AXcAH9Rq8wFwr3P7VmBDQ/0DSimlWp5ll4ac1/wfAT7FMXx0mTHmoIgsBhKMMR8ALwMrRCQJyMVRLJRSSrUiS/sIjDHrgHW1XnuqxnYpMNvKDEoppRqmwxeUUsrDaSFQSikPp4VAKaU8nBYCpZTycG1uGmoRyQZSmvnHw6h117KLcNVc4LrZNFfTaK6mccdcPYwx4XXtaHOF4HKISEJ9c23YyVVzgetm01xNo7maxtNy6aUhpZTycFoIlFLKw3laIVhid4B6uGoucN1smqtpNFfTeFQuj+ojUEop9U2edkaglFKqFi0ESinl4dy6EIjI0yJyRET2i8h7ItKxnnbTReSoiCSJyBOtkGu2iBwUkWoRqXcomIicEpFEEdkrIpavz9mEXK36eTmPGSoi60XkuPO/neppV+X8vPaKSO1pz1sqS4M/v4j4i8gq5/7tIhJrRY5m5JonItk1Pp/7WinXMhHJEpED9ewXEXnOmXu/iIxwkVxTRCS/xuf1VF3tLMgVIyJfiMgh5/+Pj9XRpmU/M2OM2z6AawAf5/YfgT/W0cYbOAH0AvyAfcAAi3NdAfQDvgTiG2h3Cghrxc/rkrns+Lycx/0T8IRz+4m6/i6d+wotznHJnx94CHjRuX0HsKoVPp/G5JoHvNBa/55qHHcyMAI4UM/+64CPAQHGAttdJNcU4EMbPq9uwAjndnvgWB1/ly36mbn1GYEx5jNjTKXz6TYcq6TVNhpIMsacNMaUA28BMy3OddgYc9TKYzRHI3O1+uflNBNY7txeDtzUCsesS2N+/ppZVwPTRESwll1/L5dkjNmIY72R+swEXjMO24COItLNBXLZwhiTaYzZ7dwuAA7jWN+9phb9zNy6ENQyH0cFrS0KSKvxPJ1vfuh2McBnIrJLRBbaHcbJrs+rizEm07l9BuhST7sAEUkQkW0iYkWxaMzP/3Ub5xeRfKCzBVmamgvgFuelhNUiElPHfju48v+D40Rkn4h8LCIDW/vgzsuKw4HttXa16GfWJhavb4iI/AfoWseunxpj/u1s81OgEnjDlXI1wkRjTIaIRADrReSI81uM3bks0VC2mk+MMUZE6hv33MP5mfUCNohIojHmREtnbaPWAiuNMWUi8gCOs5apNmdyZbtx/HsqFJHrgPeBPq11cBEJBt4FHjfGXLDyWG2+EBhjvtXQfhGZB9wATDPOi2u1ZAA1vxlFO1+zNFcj3yPD+d8sEXkPx+n/ZRWCFshlyecFDWcTkbMi0s0Yk+k8Bc6q5z0ufmYnReRLHN+mWrIQNObnv9gmXUR8gBAgpwUzNCuXMaZmhqU4+l1cgWX/pi5HzV++xph1IvIPEQkzxlg+GZ2I+OIoAm8YY9bU0aRFPzO3vjQkItOB/wNmGGOK62m2E+gjIj1FxA9H554lo02aQkSCRKT9xW0cHd91jm5oZXZ9Xh8A9zq37wW+cfYiIp1ExN+5HQZMAA61cI7G/Pw1s94KbKjnS0ir5qp1DXkGjmvPruADYK5zJMxYIL/GZUDbiEjXi307IjIax+9Lqws6zmO+DBw2xjxTT7OW/cxau0e8NR9AEo7raHudj4sjOSKBdTXaXYejZ/4EjkskVue6Gcc1vTLgLPBp7Vw4Rn/scz4OukouOz4v5zE7A58Dx4H/AKHO1+OBpc7t8UCi8zNLBBZYlOUbPz+wGMcXDoAA4B3nv78dQK9W+owulev3zn9L+4AvgP6tlGslkAlUOP99LQAWAYuc+wX4uzN3Ig2MpGvlXI/U+Ly2AeNbKddEHP2D+2v87rrOys9Mp5hQSikP59aXhpRSSl2aFgKllPJwWgiUUsrDaSFQSikPp4VAKaU8nBYCpZTycFoIlFLKw2khUOoyicgo50RuAc47wg+KyCC7cynVWHpDmVItQER+g+OO4nZAujHm9zZHUqrRtBAo1QKc8/vsBEpxTEVQZXMkpRpNLw0p1TI6A8E4VpQKsDmLUk2iZwRKtQBxrI/8FtAT6GaMecTmSEo1Wptfj0Apu4nIXKDCGPOmiHgDW0RkqjFmg93ZlGoMPSNQSikPp30ESinl4bQQKKWUh9NCoJRSHk4LgVJKeTgtBEop5eG0ECillIfTQqCUUh7u/wE5I5wmAJTguQAAAABJRU5ErkJggg==\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "PyTorch is able to automatically compute the derivative of nearly any function Most of the time, it will be at least as fast as any derivative function that you can create by hand. \n",
        "\n",
        "Let’s see an\n",
        "example.\n",
        "First, let’s pick a tensor value at which we want gradients:"
      ],
      "metadata": {
        "id": "Xkw9zZttesXk"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "xt = tensor(3.).requires_grad_()"
      ],
      "metadata": {
        "id": "VOgoxdwKe74Q"
      },
      "execution_count": 24,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "requires_grad is the function that we use to tell PyTorch that we want to calculate gradients with respect to that variable at that value.\n",
        "\n",
        "Now let's calculate our function with that value and check whether we see something interesting."
      ],
      "metadata": {
        "id": "JmGd8ACrfBwt"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "yt = f(xt)\n",
        "yt"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "t07iN8u7fb5f",
        "outputId": "56ed2759-0077-4ade-c0f9-a90373fd9e77"
      },
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(9., grad_fn=<PowBackward0>)"
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "As you can see, PyTorch prints not just the value calculated, but also note that we mark that with gradient operation.\n",
        "\n",
        "Finally, we tell PyTorch to calculate the gradients for us:"
      ],
      "metadata": {
        "id": "LFcbe91cfc98"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "yt.backward()"
      ],
      "metadata": {
        "id": "9OroPQvsfqQ3"
      },
      "execution_count": 26,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can now view the gradients by checking the grad attribute of our tensor:"
      ],
      "metadata": {
        "id": "68yXdj2cfwn-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "xt.grad"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0gjrWPMhf11C",
        "outputId": "9e845690-2f4a-45c9-9d3c-f95a5c1d400a"
      },
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(6.)"
            ]
          },
          "metadata": {},
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "As you all know, derrivative of x**2 is 2*x, and we pick x=3, so the gradients must be 2*3 = 6 which is the same result PyTorch gives us.\n",
        "\n",
        "\n",
        "Now, we'll repeat same steps with a vector arguments:\n"
      ],
      "metadata": {
        "id": "Svqxely8f3ol"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "xt = tensor([3.,4.,10.]).requires_grad_()\n",
        "xt"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "E2IF1PLJgWwR",
        "outputId": "ff62af8e-b401-48c2-abab-d7b7be0ad840"
      },
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([ 3.,  4., 10.], requires_grad=True)"
            ]
          },
          "metadata": {},
          "execution_count": 28
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "And we’ll add sum to our function so it can take a vector (i.e., a rank-1 tensor) and\n",
        "return a scalar (i.e., a rank-0 tensor):\n"
      ],
      "metadata": {
        "id": "nPTDACBijFJ1"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def f(x): return (x**2).sum()\n",
        "yt = f(xt)\n",
        "yt\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9q7DzHgTgbhO",
        "outputId": "ed13c4d9-aa24-45c6-d287-15f11650a87f"
      },
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(125., grad_fn=<SumBackward0>)"
            ]
          },
          "metadata": {},
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Our gradients are 2*xt, as we’d expect!"
      ],
      "metadata": {
        "id": "1KpnAIaYgYHP"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "yt.backward()\n",
        "xt.grad"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "h_octRe1jKdj",
        "outputId": "89740f0b-81c4-44b5-aba8-da04dc568e2d"
      },
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([ 6.,  8., 20.])"
            ]
          },
          "metadata": {},
          "execution_count": 30
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now, it is time to understand concept of learning rate. Most of the approaches start with multiplying gradient with some small number, learning rate. The learning rate is often a number between 0.001 and 0.1, although it could be any‐\n",
        "thing. Once you’ve picked a learning rate, you can adjust your parameters using this simple function:"
      ],
      "metadata": {
        "id": "VIR-25TpoQOj"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "w -= w.grad * lr"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 171
        },
        "id": "D3_-8mo3oztQ",
        "outputId": "41c42f93-892a-4082-d904-8b790df20227"
      },
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "error",
          "ename": "NameError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-31-7c4551e25a66>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mw\u001b[0m \u001b[0;34m-=\u001b[0m \u001b[0mw\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgrad\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0mlr\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m: name 'w' is not defined"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "This is known as stepping your parameters, using an optimization step. If you pick learning rate that's too low, it means you have to do a lot of steps.\n",
        "If you pick it too high, that's worse. It may ends up loss getting worse.If the learning rate is too high, it may also “bounce” around, rather than diverging.\n",
        "\n",
        "\n",
        "Now let's move on to end-to-end SGD example. \n",
        "\n",
        "\n",
        "Let's start with a very basic, synthetic model. Assume you're measuring the speed of a roller coaster as it passes over a bump. It would start quickly and then slow as it climbed the hill; it would be the slowest at the top and then speed up as it descended. You'd like to create a model that shows how speed changes over time. If you were to manually measure the speed every second for 20 seconds, it would look like this:"
      ],
      "metadata": {
        "id": "NRpgQOFIoyio"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "time = torch.arange(0,20).float(); time"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XbT6SL9Np_md",
        "outputId": "72b5060d-746a-41f4-c1bc-a99a6e664a4a"
      },
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([ 0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19.])"
            ]
          },
          "metadata": {},
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "speed = torch.randn(20)*3 + 0.75*(time-9.5)**2 + 1\n",
        "plt.scatter(time,speed);"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        },
        "id": "2i09P9lOqD6m",
        "outputId": "f18445c7-ab48-44d4-fe54-87a1b44ccd60"
      },
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAD4CAYAAAD1jb0+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAUiUlEQVR4nO3de4xcZ3nH8e9TxymrQNmEbF17Q+rQRka0EXFYRVBTRAmJQ4piN0JRKGpdiGShQgVqa3CERGlVKaZWobRFVC5JMVVKEoLjWNyMa4NQ/yBkEwdyMcZOFBdvHHu5mHBZlcQ8/WPOJuv17O7szpyZc2a/H2k05zaeR8ezvznznve8JzITSVL9/EqvC5AkLYwBLkk1ZYBLUk0Z4JJUUwa4JNXUWd18s/PPPz9XrlzZzbeUpNq7//77v5+ZQ9OXdzXAV65cyejoaDffUpJqLyKONFtuE4ok1ZQBLkk1ZYBLUk0Z4JJUUwa4JNVUV3uhLMTO/WNs3X2QJ09OsGJwgE1rV7F+9XCvy5Kknqt0gO/cP8ZNOx5i4plTAIydnOCmHQ8BGOKSFr1KN6Fs3X3wufCeNPHMKbbuPtijiiSpOiod4E+enJjXcklaTCod4CsGB+a1XJIWk0oH+Ka1qxhYuuS0ZQNLl7Bp7aoeVSRJ1VHpk5iTJyrthSJJZ6p0gEMjxA1sSTpTpZtQJEkzM8AlqaYMcEmqKQNckmpqzgCPiFUR8eCUx9MR8d6IOC8i9kTEoeL53G4ULElqmDPAM/NgZl6amZcCrwJ+DtwNbAb2ZubFwN5iXpLUJfNtQrkCeCwzjwDrgO3F8u3A+k4WJkma3XwD/AbgM8X0ssw8Vkw/BSxr9oKI2BgRoxExOj4+vsAyJUnTtRzgEXE2cC3w2enrMjOBbPa6zNyWmSOZOTI0NLTgQiVJp5vPEfibgAcy83gxfzwilgMUzyc6XZwkaWbzCfC38nzzCcAuYEMxvQG4p1NFSZLm1lKAR8Q5wJXAjimLtwBXRsQh4I3FvCSpS1oazCozfwa8ZNqyH9DolSJJaqLse/pWfjRCSaqjbtzT10vpJakE3binrwEuSSXoxj19DXBJKkE37ulrgEtSCbpxT19PYkpSCbpxT18DXJJKUvY9fW1CkaSaMsAlqaYMcEmqKQNckmrKAJekmjLAJammDHBJqikDXJJqygCXpJoywCWpplq9pdpgRNwVEd+JiAMR8ZqIOC8i9kTEoeL53LKLlSQ9r9Uj8I8BX87MlwOvBA4Am4G9mXkxsLeYr5yd+8dYs2UfF23+Amu27GPn/rFelyRJHTFngEfEi4HXAbcAZOYvMvMksA7YXmy2HVhfVpELNXlLo7GTEyTP39LIEJfUD1o5Ar8IGAf+IyL2R8Qni7vUL8vMY8U2TwHLmr04IjZGxGhEjI6Pj3em6hZ145ZGktQrrQT4WcBlwCcyczXwM6Y1l2RmAtnsxZm5LTNHMnNkaGio3XrnpRu3NJKkXmklwI8CRzPz3mL+LhqBfjwilgMUzyfKKXHhunFLI0nqlTkDPDOfAr4XEZP3AboCeBTYBWwolm0A7imlwjZ045ZGktQrrd6R5y+A2yLibOBx4O00wv/OiLgROAJcX06JC9eNWxpJ6l87949VOj+i0XzdHSMjIzk6Otq195OkhZrsxTa1I8TA0iXcfN0lXQ/xiLg/M0emL/dKTElqog692AxwSWqiDr3YDHBJaqIOvdgMcElqog692FrthSJJi0oderEZ4JI0g/WrhysV2NPZhCJJNWWAS1JNGeCSVFMGuCTVlAEuSTVlgEtSTRngklRTBrgk1ZQBLkk1ZYBLUk0Z4JJUUy2NhRIRTwA/AU4Bz2bmSEScB9wBrASeAK7PzB+VU6Ykabr5HIH/QWZeOuW2PpuBvZl5MbC3mJckdUk7TSjrgO3F9HZgffvlSJJa1WqAJ/CViLg/IjYWy5Zl5rFi+ilgWbMXRsTGiBiNiNHx8fE2y5UkTWp1PPDXZuZYRPw6sCcivjN1ZWZmRDS9vX1mbgO2QeOu9G1VK0l6TktH4Jk5VjyfAO4GLgeOR8RygOL5RFlFSpLONGeAR8Q5EfGiyWngKuBhYBewodhsA3BPWUVKks7UShPKMuDuiJjc/r8y88sRcR9wZ0TcCBwBri+vTEnSdHMGeGY+DryyyfIfAFeUUZQkaW5eiSlJNWWAS1JNtdqNcNHauX+MrbsP8uTJCVYMDrBp7SrWrx7udVmSZIDPZuf+MW7a8RATz5wCYOzkBDfteAjAEJfUczahzGLr7oPPhfekiWdOsXX3wR5VJEnPM8Bn8eTJiXktl6RuMsBnsWJwYF7LJambDPBZbFq7ioGlS05bNrB0CZvWrupRRZL0PE9izmLyRKW9UCRVkQE+h/Wrhw1sSZVkE4ok1ZQBLkk1ZROKpL7V71dSG+CS+tJiuJLaJhRJfWkxXEltgEvqS4vhSmoDXFJfWgxXUhvgkvrSYriSuuUAj4glEbE/Ij5fzF8UEfdGxOGIuCMizi6vTEman/Wrh7n5uksYHhwggOHBAW6+7pK+OYEJ8+uF8h7gAPBrxfyHgY9m5u0R8W/AjcAnOlyfJC1Yv19J3dIReERcAPwh8MliPoA3AHcVm2wH1pdRoCSpuVabUP4JeB/wy2L+JcDJzHy2mD8KNP2ai4iNETEaEaPj4+NtFStJet6cAR4RbwZOZOb9C3mDzNyWmSOZOTI0NLSQf0KS1EQrbeBrgGsj4hrgBTTawD8GDEbEWcVR+AXAWHllSpKmm/MIPDNvyswLMnMlcAOwLzPfBnwVeEux2QbgntKqlCSdoZ1+4O8H/jIiDtNoE7+lMyVJkloxr8GsMvNrwNeK6ceByztfkiSpFV6JKUk1ZYBLUk0Z4JJUUwa4JNWUAS5JNWWAS1JNGeCSVFMGuCTVlAEuSTVlgEtSTRngklRTBrgk1dS8BrOSpG7auX+MrbsP8uTJCVYMDrBp7aq+vsflfBngkipp5/4xbtrxEBPPnAJg7OQEN+14CMAQL9iEIqmStu4++Fx4T5p45hRbdx/sUUXVY4BLqqQnT07Ma/liZIBLqqQVgwPzWr4YtXJX+hdExDcj4lsR8UhE/G2x/KKIuDciDkfEHRFxdvnlSlosNq1dxcDSJactG1i6hE1rV/Wooupp5Qj8/4A3ZOYrgUuBqyPi1cCHgY9m5m8DPwJuLK9MSYvN+tXD3HzdJQwPDhDA8OAAN193iScwp5izF0pmJvDTYnZp8UjgDcAfF8u3Ax8CPtH5EiUtVutXDxvYs2ipDTwilkTEg8AJYA/wGHAyM58tNjkKNN3LEbExIkYjYnR8fLwTNUuSaDHAM/NUZl4KXEDjTvQvb/UNMnNbZo5k5sjQ0NACy5QkTTevXiiZeRL4KvAaYDAiJptgLgDGOlybJGkWrfRCGYqIwWJ6ALgSOEAjyN9SbLYBuKesIiVJZ2rlUvrlwPaIWEIj8O/MzM9HxKPA7RHx98B+4JYS65QkTdNKL5RvA6ubLH+cRnu4JKkHHMyqZI6mJqksBniJHE1Ni50HMOVyLJQSOZqaFrPJA5ixkxMkzx/A7Nxvh7VOMcBL5GhqWsw8gCmfAV4iR1PTYuYBTPkM8BI5mpoWMw9gymeAl8jR1LSYeQBTPnuhlMzR1LRYTX7u7YVSHgNcUmk8gCmXTSiSVFMGuCTVlAEuSTVlgEtSTXkSs+IcS0LSTAzwCnMwLEmzsQmlwhxLQtJsDPAKcywJSbNp5Z6YL42Ir0bEoxHxSES8p1h+XkTsiYhDxfO55Ze7uDiWhKTZtHIE/izwV5n5CuDVwLsi4hXAZmBvZl4M7C3m1UGOJSFpNq3cE/MYcKyY/klEHACGgXXA64vNtgNfA95fSpWLVCfGkrAXi9S/IjNb3zhiJfB14HeB/83MwWJ5AD+anJ/2mo3ARoALL7zwVUeOHGm/arVkei8WaBzBOyKiVC8RcX9mjkxf3vJJzIh4IfA54L2Z+fTUddn4Fmj6TZCZ2zJzJDNHhoaG5lm22mEvFqm/tRTgEbGURnjflpk7isXHI2J5sX45cKKcErVQ9mKR+lsrvVACuAU4kJkfmbJqF7ChmN4A3NP58tQOe7FI/a2VI/A1wJ8Ab4iIB4vHNcAW4MqIOAS8sZhXhdiLRepvrfRC+R8gZlh9RWfLUSd5RxSpvzkWSp/zjihS//JSekmqKQNckmrKJhRJM/JK3mozwCU15Xj01WcTiqSmvJK3+gxwSU15JW/1GeCSmvJK3uozwCU15ZW81edJTElNeSVv9RngkmbklbzVZoBrVvYDlqrLANeM7AcsVZsnMTUj+wFL1WaAa0b2A5aqzSYUzWjF4ABjTcJ6Pv2AbUOXyuMRuGbUbj/gyTb0sZMTJM+3oe/cP1ZCtdLi08o9MW+NiBMR8fCUZedFxJ6IOFQ8n1tumeqF9auHufm6SxgeHCCA4cEBbr7ukpaPoG1D772d+8dYs2UfF23+Amu27PPLs8+00oTyKeBfgU9PWbYZ2JuZWyJiczH//s6Xp15rpx+wbei9ZS+i/jfnEXhmfh344bTF64DtxfR2YH2H61IfcCyN3vIXUP9baBv4ssw8Vkw/BSybacOI2BgRoxExOj4+vsC3Ux05lkZv+Quo/7V9EjMzE8hZ1m/LzJHMHBkaGmr37VQj7bahqz3+Aup/C+1GeDwilmfmsYhYDpzoZFHqH46l0Tub1q46rQ0c/AXUbxZ6BL4L2FBMbwDu6Uw5kjrFX0D9b84j8Ij4DPB64PyIOAr8DbAFuDMibgSOANeXWaSkhfEXUH+bM8Az860zrLqiw7VIkubBKzElqaYMcEmqKQNckmrKAJekmjLAJammHA9cqjDHU9dsDHCpohxNUHOxCUWqKEcT1Fw8AlelLeYmBEcT1FwMcFVWPzQhtPMF1Il7kqq/2YSiyqp7E0K79wR1PHXNxQBXZdW9CaHdLyBHE9RcbEJRZdW9CaETX0COJqjZeASuyqp7E4J3xFHZDHBVVt2bEOr+BaTqswlFlVbnJoTJuhdrN0iVzwCXSlTnLyBVX1tNKBFxdUQcjIjDEbG5U0VJkua24CPwiFgCfBy4EjgK3BcRuzLz0U4VJ/XaYr4SVNXXzhH45cDhzHw8M38B3A6s60xZUu+1eyGOVLZ2AnwY+N6U+aPFMqkydu4fY82WfVy0+Qus2bJvXuFb9ytB1f9KP4kZERuBjQAXXnhh2W8nPafdsVTqfiWo+l87R+BjwEunzF9QLDtNZm7LzJHMHBkaGmrj7aT5afcI2gtxVHXtBPh9wMURcVFEnA3cAOzqTFlS+9o9gvZCHFXdgptQMvPZiHg3sBtYAtyamY90rDKpTe2OpeKFOKq6ttrAM/OLwBc7VIvUUZvWrjqtDRzmfwTthTiqMq/EVN/yCFr9zgBXX/MIWv3M0QglqaYMcEmqKQNckmrKAJekmjLAJammIjO792YR48CRBb78fOD7HSyn06yvPdbXHutrT9Xr+83MPGMskq4GeDsiYjQzR3pdx0ysrz3W1x7ra0/V65uJTSiSVFMGuCTVVJ0CfFuvC5iD9bXH+tpjfe2pen1N1aYNXJJ0ujodgUuSpjDAJammKhfgEXF1RByMiMMRsbnJ+l+NiDuK9fdGxMou1vbSiPhqRDwaEY9ExHuabPP6iPhxRDxYPD7YrfqK938iIh4q3nu0yfqIiH8u9t+3I+KyLta2asp+eTAino6I907bpqv7LyJujYgTEfHwlGXnRcSeiDhUPJ87w2s3FNsciogNXaxva0R8p/j/uzsiBmd47ayfhRLr+1BEjE35P7xmhtfO+rdeYn13TKntiYh4cIbXlr7/2paZlXnQuLPPY8DLgLOBbwGvmLbNnwP/VkzfANzRxfqWA5cV0y8CvtukvtcDn+/hPnwCOH+W9dcAXwICeDVwbw//r5+icYFCz/Yf8DrgMuDhKcv+AdhcTG8GPtzkdecBjxfP5xbT53apvquAs4rpDzerr5XPQon1fQj46xb+/2f9Wy+rvmnr/xH4YK/2X7uPqh2BXw4czszHM/MXwO3AumnbrAO2F9N3AVdERHSjuMw8lpkPFNM/AQ4AdRtseh3w6Wz4BjAYEct7UMcVwGOZudArczsiM78O/HDa4qmfse3A+iYvXQvsycwfZuaPgD3A1d2oLzO/kpnPFrPfoHFD8Z6YYf+1opW/9bbNVl+RG9cDn+n0+3ZL1QJ8GPjelPmjnBmQz21TfIh/DLykK9VNUTTdrAbubbL6NRHxrYj4UkT8TlcLgwS+EhH3R8TGJutb2cfdcAMz/+H0cv8BLMvMY8X0U8CyJttUZT++g8Yvqmbm+iyU6d1FE8+tMzRBVWH//T5wPDMPzbC+l/uvJVUL8FqIiBcCnwPem5lPT1v9AI1mgVcC/wLs7HJ5r83My4A3Ae+KiNd1+f3nFBFnA9cCn22yutf77zTZ+C1dyb62EfEB4Fngthk26dVn4RPAbwGXAsdoNFNU0VuZ/ei78n9LVQvwMeClU+YvKJY13SYizgJeDPygK9U13nMpjfC+LTN3TF+fmU9n5k+L6S8CSyPi/G7Vl5ljxfMJ4G4aP1WnamUfl+1NwAOZeXz6il7vv8LxyWal4vlEk216uh8j4s+ANwNvK75kztDCZ6EUmXk8M09l5i+Bf5/hfXu9/84CrgPumGmbXu2/+ahagN8HXBwRFxVHaTcAu6ZtswuYPOP/FmDfTB/gTivazG4BDmTmR2bY5jcm2+Qj4nIa+7grXzARcU5EvGhymsbJroenbbYL+NOiN8qrgR9PaS7olhmPfHq5/6aY+hnbANzTZJvdwFURcW7RRHBVsax0EXE18D7g2sz8+QzbtPJZKKu+qedU/miG923lb71MbwS+k5lHm63s5f6bl16fRZ3+oNFL4rs0zlB/oFj2dzQ+rAAvoPHT+zDwTeBlXazttTR+Tn8beLB4XAO8E3hnsc27gUdonFX/BvB7XazvZcX7fquoYXL/Ta0vgI8X+/chYKTL/7/n0AjkF09Z1rP9R+OL5BjwDI122BtpnFPZCxwC/hs4r9h2BPjklNe+o/gcHgbe3sX6DtNoP578DE72yloBfHG2z0KX6vvP4rP1bRqhvHx6fcX8GX/r3aivWP6pyc/clG27vv/afXgpvSTVVNWaUCRJLTLAJammDHBJqikDXJJqygCXpJoywCWppgxwSaqp/wc4jck+WNMuPwAAAABJRU5ErkJggg==\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can use SGD to look for a function that matches our observations. We won't be able to consider every potential function, so we'll assume it'll be quadratic, i.e., a function of the form a*(time**2)+(b*time)+c."
      ],
      "metadata": {
        "id": "Cy-8TJn9qGTf"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def f(t, params):\n",
        " a,b,c = params\n",
        " return a*(t**2) + (b*t) + c"
      ],
      "metadata": {
        "id": "g42f_AVxqTLB"
      },
      "execution_count": 34,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "In other words, the task of finding the best conceivable function that matches the data has been narrowed down to finding the best quadratic function. Because every quadratic function is fully specified by the three parameters a, b, and c, this substantially simplifies the problem. As a result, we just need to discover the optimum values for a, b, and c to find the best quadratic function. In addition, if we can solve this for three parameters, we can apply same approach for other, more complex functions."
      ],
      "metadata": {
        "id": "52clAANhqZke"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def mse(preds, targets): return ((preds-targets)**2).mean()"
      ],
      "metadata": {
        "id": "f4hZzYwJqyX_"
      },
      "execution_count": 35,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now, let's work through our seven-step process.\n",
        "\n",
        "**Step 1: Initialize the parameters**\n",
        "\n",
        "First, we initialize the parameters to random values and tell PyTorch that we want to track their gradients using requires_grad_:\n"
      ],
      "metadata": {
        "id": "_MSdMhmTq1pv"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "params = torch.randn(3).requires_grad_()"
      ],
      "metadata": {
        "id": "NGqY1nosuvMo"
      },
      "execution_count": 36,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Step 2: Calculate the predictions**\n"
      ],
      "metadata": {
        "id": "i6vnOsZVuXmk"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "preds = f(time, params)"
      ],
      "metadata": {
        "id": "uaboSJKUuZI2"
      },
      "execution_count": 37,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "In this part, we can create a function to compare our predictions with our target."
      ],
      "metadata": {
        "id": "WD7Zly7Xub-o"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def show_preds(preds, ax=None):\n",
        " if ax is None: ax=plt.subplots()[1]\n",
        " ax.scatter(time, speed)\n",
        " ax.scatter(time, to_np(preds), color='red')\n",
        " ax.set_ylim(-300,100)\n",
        "show_preds(preds)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 269
        },
        "id": "oWPgocnRuz9l",
        "outputId": "79434a17-3b69-4ed8-c91a-bf2d90a78e81"
      },
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD8CAYAAACfF6SlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAYe0lEQVR4nO3df6xcZZ3H8feHoiyyamG9i9DSH7gVA/5AmCCuP4KCtrDGIllN3WbBH7Ehwu66yeqWNFGjaURZd1fURatLhA0RXVegURBaXDVrUuFWammRwgUp9FqhwgprSlDod/+Y58r0dmbu3D5zZs6c83klk3vmOWfufHs69zvPeX4dRQRmZlYvhww7ADMzGzwnfzOzGnLyNzOrISd/M7MacvI3M6shJ38zsxrqS/KXdKWkRyRtayk7StIGSfemn0emckm6XNKEpK2STulHDGZm1rt+1fy/BiybVrYauDUilgC3pucAZwNL0mMVcEWfYjAzsx71JflHxI+Ax6YVLweuSttXAee2lF8dTZuAuZKO6UccZmbWm0ML/N1HR8TutP0r4Oi0PQ94qOW4XalsN9NIWkXz6oAjjjji1Je97GXFRWtmVjGbN2/+dUSMtdtXZPL/g4gISbNeRyIi1gHrABqNRoyPj/c9NjOzqpK0s9O+Ikf7PDzVnJN+PpLKJ4HjWo6bn8rMzGxAikz+64EL0vYFwA0t5eenUT+nA4+3NA+ZmdkA9KXZR9LXgTOAF0naBXwMuBT4pqT3AzuBd6XDbwTOASaAvcB7+xGDmZn1ri/JPyLe3WHXmW2ODeCifryvmZkdHM/wNTOrISd/M7MacvI3M6shJ38zsxpy8jczq6GBzPAdluvvmOSym3fwy988ybFzD+fDS0/g3FfPG3ZYZmZDV9nkf/0dk1zy7Tt58vfPADD5mye55Nt3AvgLwMxqr7LNPpfdvOMPiX/Kk79/hstu3jGkiMzMyqOyyf+Xv3lyVuVmZnVS2WafY+cezmSbRH/s3MOHEI2Z2ewU3WdZ2Zr/h5eewOHPmbNf2eHPmcOHl54wpIjMzHoz1Wc5+ZsnCZ7ts7z+jv4tgFzZ5H/uq+fxqfNewby5hyNg3tzD+dR5r3Bnr5mV3iD6LCvb7APNLwAnezMbNYPos6x08s/leQJmNgyD6LOsbLNPrkG0uZlZdV1/xySvu/T7LF79XV536fdnlTsG0Wfp5N+B5wmY2cHKrTwOos/SzT4deJ6AmR2sbpXHXhN40X2Wrvl30KltzfMEzGwmo1B5LDz5S3pA0p2StkgaT2VHSdog6d7088ii45gtzxMws4M1CpXHQdX83xQRJ0dEIz1fDdwaEUuAW9PzUvE8ATM7WKNQeRxWm/9y4Iy0fRXwA+AfhxRLR54nYGYHYypvlHmo+CCSfwC3SArgyxGxDjg6Inan/b8Cjh5AHAPneQJm9VX2yuMgkv/rI2JS0p8CGyTd3bozIiJ9MRxA0ipgFcCCBQuKj7SPfD8BMyuzwtv8I2Iy/XwEuA44DXhY0jEA6ecjHV67LiIaEdEYGxsrOtS+8jwBs9GWM0lrFBSa/CUdIen5U9vAW4FtwHrggnTYBcANRcYxDKMw1MvM2qvDDP+ia/5HA/8j6WfAbcB3I+J7wKXAWyTdC5yVnlfKKAz1MrP26nDlXmibf0TcD7yqTfmjwJlFvvewfXjpCfu1+UP5hnqZWXt1uHL3DN+CeJ6A2eiqw5W71/YpUNmHeplVWc5Q6zpcuTv5m1nl5A61HoVJWrmc/M2sckZhVc1hc/IvMc8QtjrL+fzXocM2l5N/SXmGsNVZ7ud/ELdBHHUe7VNSdRhnbNZJ7ud/FFbVHDbX/EuqH5etbjayUZX7+a9Dh20uJ/+Syr1s7Uezkb88bFj60WxT9Q7bXG72Kancy9bcy+Y6rG1i5eVmm+K55l9SuZetuZfN/RgqZ/WWc+XoZpviOfmXWM5la+5ls4fKWY5+NDu62aZYbvapqNzL5n6sbVL19dCtM49WKz/X/Csq97I5d20TdziPPk+yqjYn/wrLuWzO/fLI7TPwJLfh8iSr6nPyt45yvjzc4Tzacs9/HVbFHHVO/lYIdzgP3zCbbTxap/yc/K0QuTW/fjQbjHqfQU78ZWi28Widcqv2aJ9rroFFi+CQQ5o/r7lm2BHVRu6dzHJHK436JLXc+L02js1kaDV/ScuAzwFzgK9GRH9v4n7NNbBqFezd23y+c2fzOcDKlb3/jjVr4MEHYcECWLu299faSHc4Q/6VQ87rc+N3s43NZCjJX9Ic4IvAW4BdwO2S1kfEXX17kzVrnk38U/bubZb3ksD95TF0w+xwzm02yX19bvxutrGZDKvZ5zRgIiLuj4jfAdcCy/v6Dg8+OLvy6bp9efRi6stj506IePbLw01PA5E7SS232ST39bnxu9nGZjKs5D8PeKjl+a5Uth9JqySNSxrfs2fP7N5hwYLZlU837C8Py5Kb/HJr3rmvz40/t8/Fqq/UHb4RsS4iGhHRGBsbm92L166F5z1v/7LnPa9Z3othf3mAO6wz5Ca/3Jp37uv7kbzPffU8frz6zfzi0r/gx6vf7MRv+xlWh+8kcFzL8/mprH+m2tYPts197dr92/xh9l8eO3e2L+9FP/ocai6nzTp3qGo/Jjm5zd2KpIgY/JtKhwL3AGfSTPq3A38VEds7vabRaMT4+PiAIkxyOmynJ29ofnmsW9fb71i0qP2Xx8KF8MADvcVgWYY52sesHyRtjohG233DSP4Aks4B/pXmUM8rI6JrlXooyT9XzpfHIYc0O4qnk2DfvuLf38xGXimT/2yNZPLPkVvzz73yMLOR1y35l7rDt9ZyO6w92sjMunDyL6uVK5u19IULm009CxfOrtbej9FGZlZZTv5ltnJls4ln377mz9k01+QOVQUPNTWrMCf/qsptNvIMZbNKc/KvqtxmI/cZmFWaR/tYe/0YampmQ+XRPjZ7/egzMLPScvK39nL7DMAdxmYl5uRv7eX2GbjD2KzU3OZvxfDaRGZD5zZ/GzxPMjMrNSd/K4Y7jM1KzcnfitGPDmMzK4yTvxUjt8MYPFrIrEDDupOX1cHKlQe/fLTvZGZWKNf8rZy8vIRZoZz8rZw8WsisUE7+Vk4eLWRWqMKSv6SPS5qUtCU9zmnZd4mkCUk7JC0tKgYbYR4tZFaoojt8/yUi/qm1QNKJwArgJOBYYKOkl0bEMwXHYqNkqlPXN6A3K8Qwmn2WA9dGxFMR8QtgAjhtCHFY2eXcyQw8VNSsi6KT/8WStkq6UtKRqWwe8FDLMbtS2QEkrZI0Lml8z549BYdqleKF5cy6ykr+kjZK2tbmsRy4AngJcDKwG/jsbH9/RKyLiEZENMbGxnJCtbrxUFGzrrLa/CPirF6Ok/QV4Dvp6SRwXMvu+anMrH88VNSsqyJH+xzT8vQdwLa0vR5YIekwSYuBJcBtRcVhNeWhomZdFdnm/xlJd0raCrwJ+HuAiNgOfBO4C/gecJFH+ljfeaioWVeFDfWMiL/usm8t4L9CK46Hipp15Rm+Vl0eKmrWkVf1NGvHq4paxbnmb9aOh4paxTn5m7XjoaJWcU7+Zu14qKhVnJO/WTseKmoV5+Rv1k4/7kFsVmIe7WPWSc49iM1KzjV/M7MacvI3K4oniVmJudnHrAieJGYl55q/WRE8ScxKzsnfrAieJGYl5+RvVgRPErOSc/I3K4IniVnJOfmbFcGTxKzkPNrHrCieJGYl5pq/mVkNZSV/Se+UtF3SPkmNafsukTQhaYekpS3ly1LZhKTVOe9vVmmeJGYFym322QacB3y5tVDSicAK4CTgWGCjpJem3V8E3gLsAm6XtD4i7sqMw6xaPEnMCpZV84+In0fEjja7lgPXRsRTEfELYAI4LT0mIuL+iPgdcG061sxaeZKYFayoNv95wEMtz3elsk7lbUlaJWlc0viePXsKCdSslDxJzAo2Y/KXtFHStjaPwmvsEbEuIhoR0RgbGyv67czKw5PErGAztvlHxFkH8XsngeNans9PZXQpN7Mpa9fu3+YPniRmfVVUs896YIWkwyQtBpYAtwG3A0skLZb0XJqdwusLisFsdHmSmBUsa7SPpHcAnwfGgO9K2hIRSyNiu6RvAncBTwMXRcQz6TUXAzcDc4ArI2J71r/ArKo8ScwKpIgYdgw9aTQaMT4+PuwwzMxGhqTNEdFot88zfM3MasjJ36yqPEPYuvDCbmZV5BnCNgPX/M2qyDOEbQZO/mZV5BnCNgMnf7Mq8gxhm4GTv1kV+TaSNgMnf7Mq8gxhm4FH+5hVlWcIWxeu+ZuZ1ZCTv5lZDTn5m5nVkJO/mbXn5SEqzR2+ZnYgLw9Rea75m9mBvDxE5Tn5m9mBvDxE5Tn5m9mBvDxE5Tn5m9mBvDxE5WUlf0nvlLRd0j5JjZbyRZKelLQlPb7Usu9USXdKmpB0uSTlxGBmBfDyEJWXO9pnG3Ae8OU2++6LiJPblF8BfAD4CXAjsAy4KTMOM+s3Lw9RaVk1/4j4eUTs6PV4SccAL4iITdG8c/zVwLk5MZiZ2ewV2ea/WNIdkn4o6Q2pbB6wq+WYXamsLUmrJI1LGt+zZ0+BoZqZ1cuMzT6SNgIvbrNrTUTc0OFlu4EFEfGopFOB6yWdNNvgImIdsA6g0WjEbF9vZmbtzVjzj4izIuLlbR6dEj8R8VREPJq2NwP3AS8FJoH5LYfOT2VmVjVeHqLUCmn2kTQmaU7aPh5YAtwfEbuBJySdnkb5nA90/BIxsxE1tTzEzp0Q8ezyEP4CKI3coZ7vkLQLeC3wXUk3p11vBLZK2gJ8C7gwIh5L+z4IfBWYoHlF4JE+ZlXj5SFKT81BN+XXaDRifHx82GGYWS8OOaRZ459Ogn37Bh9PTUnaHBGNdvs8w9fM+s/LQ5Sek7+Z9Z+Xhyg9J38z6z8vD1F6vpmLmRXDy0OUmmv+ZmY15ORvZlZDTv5mZjXk5G9mVkNO/mZmNeTkb2bl5IXhCuWhnmZWPlMLw02tDzS1MBx4+GifuOZvZuXjheEK5+RvZuXz4IOzK7dZc/I3s/LxwnCFc/I3s/LxwnCFc/I3s/LxwnCF82gfMysnLwxXKNf8zcxqKPcevpdJulvSVknXSZrbsu8SSROSdkha2lK+LJVNSFqd8/5mZnZwcmv+G4CXR8QrgXuASwAknQisAE4ClgH/JmmOpDnAF4GzgROBd6djzcxsgLKSf0TcEhFPp6ebgPlpezlwbUQ8FRG/ACaA09JjIiLuj4jfAdemY83MbID62eb/PuCmtD0PeKhl365U1qm8LUmrJI1LGt+zZ08fQzUzq7cZk7+kjZK2tXksbzlmDfA00NeVlyJiXUQ0IqIxNjbWz19tZlXnheG6mnGoZ0Sc1W2/pPcAbwPOjIhIxZPAcS2HzU9ldCk3M+sPLww3o9zRPsuAjwBvj4jWVZjWAyskHSZpMbAEuA24HVgiabGk59LsFF6fE4OZ2QG8MNyMcid5fQE4DNggCWBTRFwYEdslfRO4i2Zz0EUR8QyApIuBm4E5wJURsT0zBjOz/XlhuBllJf+I+LMu+9YCByzEERE3AjfmvK+ZWVcLFjSbetqVG+AZvmZWRV4YbkZO/mZWPV4YbkZe2M3MqskLw3Xlmr+ZWQ05+ZuZ1ZCTv5lZDTn5m5nVkJO/mVkNOfmbmdWQk7+ZWTsVXxXU4/zNzKarwaqgrvmbmU1Xg1VBnfzNzKarwaqgTv5mZtN1Wv2zQquCOvmbmU1Xg1VBnfzNzKarwaqgHu1jZtZOxVcFdc3fzKyGcm/gfpmkuyVtlXSdpLmpfJGkJyVtSY8vtbzmVEl3SpqQdLnSzX/NzGxwcmv+G4CXR8QrgXuAS1r23RcRJ6fHhS3lVwAfAJakx7LMGMzMbJaykn9E3BIRT6enm4D53Y6XdAzwgojYFBEBXA2cmxODmZnNXj/b/N8H3NTyfLGkOyT9UNIbUtk8YFfLMbtSmZmZDdCMo30kbQRe3GbXmoi4IR2zBngamFr5aDewICIelXQqcL2kk2YbnKRVwCqABRWaXGFmNmwzJv+IOKvbfknvAd4GnJmacoiIp4Cn0vZmSfcBLwUm2b9paH4q6/Te64B1AI1GI2aK1czMepM72mcZ8BHg7RGxt6V8TNKctH08zY7d+yNiN/CEpNPTKJ/zgRtyYjAzK6WSLwmdO8nrC8BhwIY0YnNTGtnzRuATkn4P7AMujIjH0ms+CHwNOJxmH8FN03+pmdlIG4EloZVaakqv0WjE+Pj4sMMwM5vZokXNhD/dwoXwwAMDC0PS5ohotNvnGb5mZv02AktCO/mbmfXbCCwJ7eRvZtZvI7AktJO/mVm/jcCS0F7S2cysCCVfEto1fzOzGnLyNzOrISd/M7MacvI3M6shJ38zsxpy8jczqyEnfzOzGnLyNzOrISd/M7MyKvh+AJ7ha2ZWNgO4H4Br/mZmZbNmzbOJf8revc3yPnHyNzMrmwHcD8DJ38ysbAZwPwAnfzOzshnA/QCyk7+kT0raKmmLpFskHZvKJelySRNp/yktr7lA0r3pcUFuDGZmlTKA+wFk38Bd0gsi4om0/bfAiRFxoaRzgL8BzgFeA3wuIl4j6ShgHGgAAWwGTo2I/+32Pr6Bu5nZ7BR6A/epxJ8cQTOhAywHro6mTcBcSccAS4ENEfFYSvgbgGW5cZiZWe/6Ms5f0lrgfOBx4E2peB7wUMthu1JZp/J2v3cVkAa38ltJOw4yxBcBvz7I1w6C48vj+PI4vjxljm9hpx09JX9JG4EXt9m1JiJuiIg1wBpJlwAXAx87qDCniYh1wLrc3yNpvNOlTxk4vjyOL4/jy1P2+DrpKflHxFk9/r5rgBtpJv9J4LiWffNT2SRwxrTyH/T4+83MrA/6MdpnScvT5cDdaXs9cH4a9XM68HhE7AZuBt4q6UhJRwJvTWVmZjYg/Wjzv1TSCcA+YCdwYSq/keZInwlgL/BegIh4TNIngdvTcZ+IiMf6EEc32U1HBXN8eRxfHseXp+zxtZU91NPMzEaPZ/iamdWQk7+ZWQ1VKvlLWiZpR1pSYnWb/YdJ+kba/xNJiwYY23GS/lvSXZK2S/q7NsecIenxtFTGFkkfHVR86f0fkHRneu8DplN3W7JjALGd0HJetkh6QtKHph0z0PMn6UpJj0ja1lJ2lKQNaemSDWlQQ7vXFr7ESYf4LpN0d/r/u07S3A6v7fpZKDC+j0uabPk/PKfDa7v+rRcY3zdaYntA0pYOry38/GWLiEo8gDnAfcDxwHOBn9FcaqL1mA8CX0rbK4BvDDC+Y4BT0vbzgXvaxHcG8J0hnsMHgBd12X8OcBMg4HTgJ0P8v/4VsHCY5w94I3AKsK2l7DPA6rS9Gvh0m9cdBdyffh6Zto8cUHxvBQ5N259uF18vn4UC4/s48A89/P93/VsvKr5p+z8LfHRY5y/3UaWa/2nARETcHxG/A66lOfS01XLgqrT9LeBMSRpEcBGxOyJ+mrb/D/g5HWY2l1inJTsG7UzgvojYOYT3/oOI+BEwfaRa62fsKuDcNi8dyBIn7eKLiFsi4un0dBPNeTZD0eH89aKXv/Vs3eJLeeNdwNf7/b6DUqXk38uyEX84Jv0BPA78yUCia5Gam14N/KTN7tdK+pmkmySdNNDAmusy3SJpc1paY7qel+Yo2Ao6/9EN8/wBHB3N+SzQvDo5us0xZTmP76N5JdfOTJ+FIl2cmqWu7NBsVobz9wbg4Yi4t8P+YZ6/nlQp+Y8ESX8M/Bfwodh/UTyAn9JsyngV8Hng+gGH9/qIOAU4G7hI0hsH/P4zkvRc4O3Af7bZPezzt59oXv+Xciy1pDXA0zRn5bczrM/CFcBLgJOB3TSbVsro3XSv9Zf+b6lKyb/TchJtj5F0KPBC4NGBRNd8z+fQTPzXRMS3p++PiCci4rdp+0bgOZJeNKj4ImIy/XwEuI7m5XWrXs5x0c4GfhoRD0/fMezzlzw81RSWfj7S5pihnkdJ7wHeBqxMX1AH6OGzUIiIeDginomIfcBXOrzvsM/focB5wDc6HTOs8zcbVUr+twNLJC1OtcMVNJeYaLUemBpZ8ZfA9zt9+PsttRH+O/DziPjnDse8eKoPQtJpNP9/BvLlJOkISc+f2qbZMbht2mGdluwYpI41rmGevxatn7ELgBvaHDO0JU4kLQM+Arw9IvZ2OKaXz0JR8bX2Ib2jw/v28rdepLOAuyNiV7udwzx/szLsHud+PmiORrmH5kiANansEzQ/6AB/RLO5YAK4DTh+gLG9nmYTwFZgS3qcQ3M5jAvTMRcD22mOXtgE/PkA4zs+ve/PUgxT5681PgFfTOf3TqAx4P/fI2gm8xe2lA3t/NH8EtoN/J5mu/P7afYh3QrcC2wEjkrHNoCvtrz2felzOAG8d4DxTdBsL5/6DE6NfjsWuLHbZ2FA8f1H+mxtpZnQj5keX3p+wN/6IOJL5V+b+sy1HDvw85f78PIOZmY1VKVmHzMz65GTv5lZDTn5m5nVkJO/mVkNOfmbmdWQk7+ZWQ05+ZuZ1dD/AxNuGzdioUXBAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Step 3: Calculate the loss**"
      ],
      "metadata": {
        "id": "4fHQjEQavpLz"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "loss = mse(preds, speed)\n",
        "loss"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rJQX7RN0vs6q",
        "outputId": "baba4ad6-ab44-471b-c22e-695236d464f1"
      },
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(25823.8086, grad_fn=<MeanBackward0>)"
            ]
          },
          "metadata": {},
          "execution_count": 39
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Our goal is now to improve this. To do that, we’ll need to know the gradients.\n",
        "\n",
        "\n",
        "**Step 4: Calculate the gradients**"
      ],
      "metadata": {
        "id": "jDQZJyYEvsEe"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "loss.backward()\n",
        "params.grad"
      ],
      "metadata": {
        "id": "12NTA0t6wKQp"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "params.grad * 1e-5"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "X0KyuuyrwOiF",
        "outputId": "3399481e-8165-4f5c-d7fb-3780e642ead8"
      },
      "execution_count": 41,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([-0.5320, -0.0342, -0.0025])"
            ]
          },
          "metadata": {},
          "execution_count": 41
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can use these gradients to improve our parameters. We’ll need to pick a learning rate for now, we’ll just use 1e-5."
      ],
      "metadata": {
        "id": "KnNI_S5twJtE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "params"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8k-FUW-zyAND",
        "outputId": "a1e8cb76-4c0b-422a-e7d3-f143eea23f60"
      },
      "execution_count": 42,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([-0.7658, -0.7506,  1.3525], requires_grad=True)"
            ]
          },
          "metadata": {},
          "execution_count": 42
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Step 5: Step the weights**\n",
        "\n",
        "Now we need to update the parameters based on the gradients just calculated."
      ],
      "metadata": {
        "id": "USPKEzZ_yI27"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "lr = 1e-5\n",
        "params.data -= lr * params.grad.data\n",
        "params.grad = None"
      ],
      "metadata": {
        "id": "sTiMV848yNyM"
      },
      "execution_count": 43,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's check that loss improved:"
      ],
      "metadata": {
        "id": "wp5vAMYJyZKS"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "preds = f(time,params)\n",
        "mse(preds, speed)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "w27MyfzuycKv",
        "outputId": "d39ac253-0fe6-4228-b32a-51ab198d1025"
      },
      "execution_count": 44,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(5435.5356, grad_fn=<MeanBackward0>)"
            ]
          },
          "metadata": {},
          "execution_count": 44
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We need to repeat this a few times, so it is useful to create function:\n"
      ],
      "metadata": {
        "id": "_TruEsalyhFG"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def apply_step(params, prn=True):\n",
        " preds = f(time, params)\n",
        " loss = mse(preds, speed)\n",
        " loss.backward()\n",
        " params.data -= lr * params.grad.data\n",
        " params.grad = None\n",
        " if prn: print(loss.item())\n",
        " return preds\n"
      ],
      "metadata": {
        "id": "n9LZX2k4yn2n"
      },
      "execution_count": 45,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Step 6: Repeat the process**"
      ],
      "metadata": {
        "id": "_VNZRB3ByvW3"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "for i in range(7): apply_step(params)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hab30Ss1yxYY",
        "outputId": "162033eb-a9af-4843-ff63-d7bdcb873183"
      },
      "execution_count": 46,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "5435.53564453125\n",
            "1577.44921875\n",
            "847.3778076171875\n",
            "709.2225341796875\n",
            "683.0758056640625\n",
            "678.1243896484375\n",
            "677.1838989257812\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Step 7: Stop**\n",
        "\n",
        "We just decided to stop after 10 epochs. In reality, we'd keep an eye on the training and validation losses, as well as our metrics, to determine when to stop."
      ],
      "metadata": {
        "id": "REdduB5AzBpB"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# The MNIST Loss Function"
      ],
      "metadata": {
        "id": "4zpZdBzvzWoG"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We already have our xs—the photographs themselves—as independent variables.\n",
        "We'll combine them all into a single tensor and convert them from a list of matrices(a rank-3 tensor)  to a list of vectors (a rank-2 tensor). This may be accomplished with view, a PyTorch technique that alters the form of a tensor without altering its contents. - 1 is a specific view parameter that says \"make this axis as big as it has to be to fit all of the data\":"
      ],
      "metadata": {
        "id": "Y9VToJLjza_3"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "train_x = torch.cat([stacked_threes, stacked_eights]).view(-1, 28*28)\n",
        "                    "
      ],
      "metadata": {
        "id": "Jb2V0Wawz7Cu"
      },
      "execution_count": 47,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We need a label for each image. We’ll use 1 for 3s and 0 for 8s:"
      ],
      "metadata": {
        "id": "xY86xZKvzrOy"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "train_y = tensor([1]*len(threes) + [0]*len(eights)).unsqueeze(1)\n",
        "train_x.shape,train_y.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-m5pTt5b06fr",
        "outputId": "8838fd44-8de4-468c-f5db-cb57e2db69b3"
      },
      "execution_count": 48,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(torch.Size([11982, 784]), torch.Size([11982, 1]))"
            ]
          },
          "metadata": {},
          "execution_count": 48
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "When indexed in PyTorch, a Dataset must return a tuple of (x,y). Python has a zip function that, when used with list, gives a straightforward method to achieve this:"
      ],
      "metadata": {
        "id": "6cs4oL9UKTOC"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "dset = list(zip(train_x,train_y))\n",
        "x,y = dset[0]\n",
        "x.shape,y"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TdRSfJmJKgzI",
        "outputId": "1bb4d933-9825-462c-c417-51c2acd53e00"
      },
      "execution_count": 49,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(torch.Size([784]), tensor([1]))"
            ]
          },
          "metadata": {},
          "execution_count": 49
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "valid_x = torch.cat([valid_3_tens, valid_8_tens]).view(-1, 28*28)\n",
        "valid_y = tensor([1]*len(valid_3_tens) + [0]*len(valid_8_tens)).unsqueeze(1)\n",
        "valid_dset = list(zip(valid_x,valid_y))"
      ],
      "metadata": {
        "id": "znPbviOQKoje"
      },
      "execution_count": 51,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now we need a random weight for every pixel."
      ],
      "metadata": {
        "id": "rvnbcQJxKu3l"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def init_params(size, std=1.0): return (torch.randn(size)*std).requires_grad_()\n",
        "weights = init_params((28*28,1))"
      ],
      "metadata": {
        "id": "ROHyAaTkKuHd"
      },
      "execution_count": 52,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "The function weights*pixels won’t be flexible enough—it is always equal to 0 when\n",
        "the pixels are equal to 0 (i.e., its intercept is 0). You might remember from high school\n",
        "math that the formula for a line is y=w*x+b; we still need the b. We’ll initialize it to a\n",
        "random number too:\n"
      ],
      "metadata": {
        "id": "QRIuDHxgK9KJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "bias = init_params(1)"
      ],
      "metadata": {
        "id": "jnKUsmWHLJ6e"
      },
      "execution_count": 53,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can now calculate a prediction for one image:\n"
      ],
      "metadata": {
        "id": "oJj4_CZvLNgk"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "(train_x[0]*weights.T).sum() + bias"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BwX4RvlDLN5D",
        "outputId": "53324c61-576e-460c-ecfc-83769b903ad9"
      },
      "execution_count": 54,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([20.2336], grad_fn=<AddBackward0>)"
            ]
          },
          "metadata": {},
          "execution_count": 54
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "While we could calculate the prediction for each image using a Python for loop, this would be extremely slow.In this case, there’s an extremely convenient mathematical operation that calculates w*x for every row of a matrix—it’s called matrix multiplication. In Python, matrix multiplication is represented with the @ operator. Let’s try it:"
      ],
      "metadata": {
        "id": "3rjtObtlLaQt"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def linear1(xb): return xb@weights + bias\n",
        "preds = linear1(train_x)\n",
        "preds"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "I1hfPBiVLqoX",
        "outputId": "cc3244b2-046d-4cae-a774-54094ba18df7"
      },
      "execution_count": 55,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[20.2336],\n",
              "        [17.0644],\n",
              "        [15.2384],\n",
              "        ...,\n",
              "        [15.4866],\n",
              "        [19.8041],\n",
              "        [20.5976]], grad_fn=<AddBackward0>)"
            ]
          },
          "metadata": {},
          "execution_count": 55
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's check our accuracy. To determine if each output represents a 3 or a 8, we just check whether it is bigger than 0, allowing us to measure our accuracy for each item (using broadcasting, so no loops!) the following:"
      ],
      "metadata": {
        "id": "VztD0ac6L06d"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "corrects = (preds>0.0).float() == train_y\n",
        "corrects"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qeQ0x4QcNPXQ",
        "outputId": "bbf007a4-b4fc-41ef-86a3-48cd8c8b896f"
      },
      "execution_count": 56,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[ True],\n",
              "        [ True],\n",
              "        [ True],\n",
              "        ...,\n",
              "        [False],\n",
              "        [False],\n",
              "        [False]])"
            ]
          },
          "metadata": {},
          "execution_count": 56
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "corrects.float().mean().item()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5O8k-zpVNVGh",
        "outputId": "e0a44a3b-57a6-419b-8cf3-2d54443f8974"
      },
      "execution_count": 57,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.5104323029518127"
            ]
          },
          "metadata": {},
          "execution_count": 57
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now let’s see what the change in accuracy is for a small change in one of the weights:\n",
        "\n"
      ],
      "metadata": {
        "id": "w1OlHDddNUt-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "weights.data[0] *=1.0001"
      ],
      "metadata": {
        "id": "Lv3E5ttVNcd8"
      },
      "execution_count": 62,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "preds = linear1(train_x)\n",
        "((preds>0.0).float() == train_y).float().mean().item()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Rv2ItRdnONGh",
        "outputId": "8f5fd784-d5ea-4703-9db8-893416c486f0"
      },
      "execution_count": 63,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.5104323029518127"
            ]
          },
          "metadata": {},
          "execution_count": 63
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "As we’ve seen, we need gradients in order to improve our model using SGD, and in\n",
        "order to calculate gradients we need a loss function that represents how good our\n",
        "model is. That is because the gradients are a measure of how that loss function\n",
        "changes with small tweaks to the weights. Then, let's write a loss function that helps us."
      ],
      "metadata": {
        "id": "0RLtPTyTOV-t"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def mnist_loss(predictions, targets):\n",
        " predictions = predictions.sigmoid()\n",
        " return torch.where(targets==1, 1-predictions, predictions).mean()"
      ],
      "metadata": {
        "id": "zkmrRLnROWt2"
      },
      "execution_count": 64,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now that we have a loss function suitable for driving SGD, we can consider some of\n",
        "the details involved in the next phase of the learning process, which is to change or\n",
        "update the weights based on the gradients. \"This is called an optimization step.\n",
        "To take an optimization step, we need to calculate the loss over one or more data\n",
        "items. How many should we use?... Calculating it for the whole dataset would take a long time. Calculating it for a single\n",
        "item would not use much information, so it would result in an imprecise and unsta‐\n",
        "ble gradient. So instead we compromise: we calculate the average loss for a few data items at a\n",
        "time. This is called a mini-batch...One simple and effective thing we can\n",
        "vary is what data items we put in each mini-batch. Rather than simply enumerating\n",
        "our dataset in order for every epoch, instead what we normally do is randomly shuffle\n",
        "it on every epoch, before we create mini-batches. PyTorch and fastai provide a class\n",
        "that will do the shuffling and mini-batch collation for you, called DataLoader.\n",
        "A DataLoader can take any Python collection and turn it into an iterator over many\n",
        "batches, like so:\" (This part directly taken from fast.ai book page.170 highly recommended to read)"
      ],
      "metadata": {
        "id": "xFkdCXWFPsI3"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "coll = range(15)\n",
        "dl = DataLoader(coll, batch_size=5, shuffle=True)\n",
        "list(dl)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3DPrV5wiPtIw",
        "outputId": "739f3156-5e30-4359-da9a-46ae82c4d89d"
      },
      "execution_count": 65,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[tensor([ 3, 12,  8, 10,  2]),\n",
              " tensor([ 9,  4,  7, 14,  5]),\n",
              " tensor([ 1, 13,  0,  6, 11])]"
            ]
          },
          "metadata": {},
          "execution_count": 65
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We don't simply want any Python collection for training a model; we want a collection with independent and dependent variables (the inputs and targets of the model). A Dataset is a PyTorch collection that comprises tuples of independent and dependent variables. Here is an example of a very basic Dataset:"
      ],
      "metadata": {
        "id": "8TqAiNpcQ9Yg"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "ds = L(enumerate(string.ascii_lowercase))\n",
        "ds"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qNWcd1vZRCys",
        "outputId": "481bd068-5aa6-4f95-d4b2-78101e5448b5"
      },
      "execution_count": 66,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(#26) [(0, 'a'),(1, 'b'),(2, 'c'),(3, 'd'),(4, 'e'),(5, 'f'),(6, 'g'),(7, 'h'),(8, 'i'),(9, 'j')...]"
            ]
          },
          "metadata": {},
          "execution_count": 66
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Then lets see what happen if we pass Dataset to a DataLoader:"
      ],
      "metadata": {
        "id": "dvk5GQDHRESL"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "dl = DataLoader(ds, batch_size=6, shuffle=True)\n",
        "list(dl)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IGiMXfRYRNzl",
        "outputId": "66230533-09aa-4217-bc52-7526725a1f55"
      },
      "execution_count": 67,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[(tensor([17, 18, 10, 22,  8, 14]), ('r', 's', 'k', 'w', 'i', 'o')),\n",
              " (tensor([20, 15,  9, 13, 21, 12]), ('u', 'p', 'j', 'n', 'v', 'm')),\n",
              " (tensor([ 7, 25,  6,  5, 11, 23]), ('h', 'z', 'g', 'f', 'l', 'x')),\n",
              " (tensor([ 1,  3,  0, 24, 19, 16]), ('b', 'd', 'a', 'y', 't', 'q')),\n",
              " (tensor([2, 4]), ('c', 'e'))]"
            ]
          },
          "metadata": {},
          "execution_count": 67
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We get back many batches that are\n",
        "themselves tuples of tensors representing batches of independent and dependent\n",
        "variables.\n",
        "\n",
        "We are now ready to write our first training loop for a model using SGD!"
      ],
      "metadata": {
        "id": "jEiGFoCXRSWe"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "for x,y in dl:\n",
        " \n",
        " loss = loss_func(pred, y)\n",
        " loss.backward()\n",
        " parameters -= parameters.grad * lr"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 243
        },
        "id": "XTHggApcRWLh",
        "outputId": "eff530f9-f5e5-4fbf-c019-4f52be0d4057"
      },
      "execution_count": 69,
      "outputs": [
        {
          "output_type": "error",
          "ename": "NameError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-69-a5522c718cc3>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0my\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdl\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m  \u001b[0mloss\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mloss_func\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpred\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      4\u001b[0m  \u001b[0mloss\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbackward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m  \u001b[0mparameters\u001b[0m \u001b[0;34m-=\u001b[0m \u001b[0mparameters\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgrad\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0mlr\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mNameError\u001b[0m: name 'loss_func' is not defined"
          ]
        }
      ]
    }
  ]
}