norse-from-scratch.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ed9d5547",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Learning Norse from scratch\n",
    "\n",
    "\n",
    "This notebook is for you if \n",
    "\n",
    "* Have not worked with Norse before\n",
    "* You are not familiar with [PyTorch](https://pytorch.org)\n",
    "* You prefer visual explanations"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15080bc2",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Agenda\n",
    "\n",
    "1. Learning PyTorch \n",
    "  * Familiarize yourself with the deep learning library PyTorch\n",
    "2. Learning about spiking neurons\n",
    "  * Quickly understand what spiking neurons are and why they are relevant\n",
    "3. Learning Norse\n",
    "  * Simulate and visualise neurons in minutes!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "866ed3a4",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## 1. Crawling before you can walk: PyTorch\n",
    "\n",
    "Before we get to Norse, we need to cover some basics of PyTorch."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "838931c3",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "First, we need to install torch:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1e0941bb",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# This is the CPU installation\n",
    "!pip install torch --quiet\n",
    "# Uncomment and use this line if you have a GPU!\n",
    "#!pip3 install torch==1.10.0+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0df39588",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We can now import torch!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "8598dad9",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "import torch"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1c0e62d",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Working with Tensors\n",
    "\n",
    "Now that we have PyTorch installed, we can create a vector:\n",
    "$$\n",
    "  v =\n",
    "  \\left[ {\\begin{array}{cc}\n",
    "    1 \\\\\n",
    "    2  \\\\\n",
    "  \\end{array} } \\right]\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "145ca8af",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([1, 2])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "v = torch.tensor([1, 2])\n",
    "v"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c0a3f44",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Working with matrices\n",
    "\n",
    "It turns out we can use the same notation to create matrices\n",
    "\n",
    "$$\n",
    "  m =\n",
    "  \\left[ {\\begin{array}{cc}\n",
    "    1 & 2\\\\\n",
    "    3 & 4  \\\\\n",
    "  \\end{array} } \\right]\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "8ab3ac2c",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1, 2],\n",
       "        [3, 4]])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m = torch.tensor([[1, 2], [3, 4]])\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e591d314",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "You may wonder why it's called `torch.tensor`. That's because PyTorch calles vectors, matrices, and higher-order objects **tensors**. \n",
    "\n",
    "Below you can see the relationship between a **scalar** value, a **vector**, a **matrix**, and a **tensor** in 3D.\n",
    "\n",
    "![](https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2Fhadrienj.github.io%2Fassets%2Fimages%2F2.1%2Fscalar-vector-matrix-tensor.png&f=1&nofb=1)\n",
    "&copy; [Hadrienj](https://hadrienj.github.io/posts/Deep-Learning-Book-Series-2.1-Scalars-Vectors-Matrices-and-Tensors/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "43b8fa7e",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Exercise 1: Create a tensor\n",
    "#\n",
    "# You now know how to create a vector and a matrix. Create a tensor with 2x2x2 numbers\n",
    "# Can you make it look like the image above?\n",
    "\n",
    "t = ..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee197bba",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Inspecting a tensor\n",
    "\n",
    "Understanding the dimensionality, called the **shape**, of a tensor is hugely important. **You should at all times know the shape of the tensor you are working with**.\n",
    "\n",
    "Here is how you can see the shape of a tensor:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "1205ddf1",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([2, 3])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.tensor([[1, 2, 3], [4, 5, 6]]).shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4c45585",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "This tells us there are two dimensions containing two elements in the row and three elements in the column.\n",
    "\n",
    "Note, you can find much more information about tensors here: https://pytorch.org/tutorials/beginner/basics/tensorqs_tutorial.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ac108fd4",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Exercise 2: Analyzing a tensor\n",
    "#\n",
    "# What is the shape of the tensor you created before?\n",
    "\n",
    "..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "89355034",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## PyTorch modules\n",
    "\n",
    "PyTorch consist of a number of **modules** that contain useful functionalities. Examples include:\n",
    "* `torch.nn.Linear` - Linear mapping between modules\n",
    "* `torch.nn.ReLU`   - Rectified Linear Unit\n",
    "* `torch.nn.Sequential` - Put modules together in sequence\n",
    "\n",
    "If you are unfamiliar with the concepts above, please read more in the PyTorch documentation here: https://pytorch.org/tutorials/beginner/basics/buildmodel_tutorial.html"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0296b2ba",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Assume that we would like to apply the `ReLU` to one of our tensors. That works like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "10de784c",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0, 1])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "module = torch.nn.ReLU()\n",
    "module(torch.tensor([-1, 1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0aa7dc8",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Or the `Linear` module, that maps a number of inputs to a number of outputs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "2d162e18",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([-0.2071], grad_fn=<AddBackward0>)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "module = torch.nn.Linear(in_features=2, out_features=1)\n",
    "module(torch.tensor([-1.0, 0.4]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc118b90",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "What just happened? Where did that number come from?\n",
    "A linear module simply applies `ax + b`. We specified the `x` in our tensor, but where did the `a` and the `b` come from?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac2c4153",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Torch creates there **parameters** automatically. As a user, we *can* specify them, but we don't have to. Here are the current weights and biases in the layer:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "02685a01",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "module.weight"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "006438bc",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "module.bias"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "996a3e3f",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Creating a PyTorch network\n",
    "\n",
    "We are now ready to create a PyTorch network! You haven't seen the `Sequential` module yet, but perhaps you can figure out how to apply it anyway?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a4bf89f1",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Exercise 3: Using PyTorch modules\n",
    "#\n",
    "# Use the Sequential module to create a network consisting of two modules:\n",
    "#  - Linear(2, 3)\n",
    "#  - ReLU()\n",
    "# Documentation is available here: https://pytorch.org/docs/stable/generated/torch.nn.Sequential.html#torch.nn.Sequential\n",
    "\n",
    "module = ...\n",
    "\n",
    "module(torch.tensor([-2, 3]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4650615",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Spiking neurons\n",
    "\n",
    "The `ReLU` unit you saw before was a neuron activation unit. However, it only applies a function *once*. Real neurons live for years and exist over a large period of time. That is exactly what makes them so computationally interesting, and that is what we will be modelling now."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0cdb1372",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "\n",
    "![](images/spiking_neuron.png)\n",
    "\n",
    "Our neurons receives a number of inputs and can choose to do two things: **not spike (0)** or **spike (1)**.\n",
    "\n",
    "But, instead of doing it *once* like with the `ReLU` unit, we are doing it **over multiple timesteps**."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75cd61bb",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Here is a visualization of **how the neuron behaves internally in time** if we give two types of inputs: a constant input of 0.1 or a sawtooth input oscillating between -0.1 and 0.1:\n",
    "\n",
    "![](https://ncskth.github.io/norse-rl/_images/li.gif)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16d6f3aa",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Here is a visualization of **how the neuron behaves internally in time** if we give it constant inputs: 0, 0.1, and 0.3:\n",
    "\n",
    "\n",
    "![](https://ncskth.github.io/norse-rl/_images/spikes.gif)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d4184d1",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Remembering state\n",
    "\n",
    "You may have noticed that the next value of the neuron, say $v_t$, depends on the *previous* value $v_{t-1}$. We model this using **state**: a way to **remember what the neuron value was before**. Visually, it looks like this:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ededd236",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "\n",
    "\n",
    "<table>\n",
    "<thead>\n",
    "<tr>\n",
    "    <td><img src=\"images/neuron_state.png\"/></td>\n",
    "    <td><pre>\n",
    "output, state = module(input, state)\n",
    "</pre>\n",
    "</td>\n",
    "</tr>\n",
    "    </thead>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d928efa",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We can now model this in Norse!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92724bdf",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<h2 style=\"text-align: center; font-size: 400%\"> Norse = PyTorch + ⚡️Spikes</h2>\n",
    "\n",
    "Norse uses the exact same coding style as PyTorch, except, we use spiking neurons.\n",
    "\n",
    "First, we need to install and import Norse:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "32e991cb",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "!pip install norse --quiet\n",
    "import norse"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "id": "2401662e",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "db918a14d79746eeaa43abd297b0d951",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "interactive(children=(FloatSlider(value=0.3, description='x', max=0.5, step=0.02), Output()), _dom_classes=('w…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import ipywidgets as widgets\n",
    "from ipywidgets import interact\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "module = norse.torch.LIFCell()\n",
    "# use interact decorator to decorate the function, so the function can receive the slide bar's value with parameter x.\n",
    "@interact(x=widgets.FloatSlider(min=0.0, max=0.5, step=0.02, value=0.3))\n",
    "def simulate(x=0.15):\n",
    "    with torch.no_grad():\n",
    "        s = None\n",
    "        out, ss = [], []\n",
    "        for i in range(100):\n",
    "            z, s = module(torch.tensor([x]), s)\n",
    "            out.append(out)\n",
    "            ss.append(s)\n",
    "        vs = torch.stack([s.v for s in ss]).squeeze()\n",
    "        plt.figure(figsize=(14, 4))\n",
    "        plt.ylim(0,1.05)\n",
    "        plt.plot(vs)\n",
    "        plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab15c68b",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "One simple neuron model we can apply is called the **Leaky integrate-and-fire** model, or `LIF` for short:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "9ece6dbc",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "module = norse.torch.LIF()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34f365c9",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Before we provide it any data, we need to recall the *temporal* properties of our spiking neurons. We cannot just provide it with single values, we need to **provide a matrix of values**: one dimension being input, another being time. We saw above that an input of 0.3 should make it spike, let's try that:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a00dd3b2",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Exercise 4: Generate data\n",
    "#\n",
    "# What is this code doing? Examine its shape and print out its contents\n",
    "data = torch.ones((100, 1)) * 0.3"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75bb9cb3",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We are now ready to apply this to our neuron:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "84be92c0",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "output = module(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5b21e283",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Exercise 5: Analyse module output\n",
    "#\n",
    "# Look at the `output` variable. What does it contain? Why?"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}