01_materials/labs/lab_2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Backpropagation in Multilayer Neural Networks\n",
    "\n",
    "While we will primarily be working with high-level, abstract toolkits like Keras in this course, understanding how backpropagation works is absolutely essential to using neural networks. \n",
    "\n",
    "In this exercise, we will build our own backpropagation algorithm - working through each step, to ensure that we can follow it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Just like in Lab 1, we'll be working with the MNIST dataset. We will load it and plot an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from sklearn.datasets import load_digits\n",
    "\n",
    "digits = load_digits()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sample_index = 45\n",
    "plt.figure(figsize=(3, 3))\n",
    "plt.imshow(digits.images[sample_index], cmap=plt.cm.gray_r,\n",
    "           interpolation='nearest')\n",
    "plt.title(\"image label: %d\" % digits.target[sample_index]);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preprocessing\n",
    "\n",
    "Of course, we need to split our data into training and testing sets before we use it, just the same as in Lab 1:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "data = np.asarray(digits.data, dtype='float32')\n",
    "target = np.asarray(digits.target, dtype='int32')\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    data, target, test_size=0.15, random_state=37)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Numpy Implementation\n",
    "\n",
    "## a) Logistic Regression\n",
    "\n",
    "In this section we will implement a logistic regression model trainable with SGD using numpy. Here are the objectives:\n",
    "\n",
    "- Implement the softmax function $\\sigma(\\mathbf{x})_i = \\frac{e^{x_i}}{\\sum_{j=1}^n e^{x_j}}$;\n",
    "- Implement the negative log likelihood function $NLL(Y_{true}, Y_{pred}) = - \\sum_{i=1}^{n}{y_{true, i} \\cdot \\log(y_{pred, i})}$;\n",
    "- Train a logistic regression model on the MNIST dataset;\n",
    "- Evaluate the model on the training and testing sets.\n",
    "\n",
    "Before we get there, let's write a function that one-hot encodes the class labels:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def one_hot(n_classes, y):\n",
    "    return np.eye(n_classes)[y]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "one_hot(n_classes=10, y=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "one_hot(n_classes=10, y=[0, 4, 9, 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The softmax function\n",
    "\n",
    "Now we will implement the softmax function. Recall that the softmax function is defined as follows:\n",
    "\n",
    "$$\n",
    "softmax(\\mathbf{x}) = \\frac{1}{\\sum_{i=1}^{n}{e^{x_i}}}\n",
    "\\cdot\n",
    "\\begin{bmatrix}\n",
    "  e^{x_1}\\\\\\\\\n",
    "  e^{x_2}\\\\\\\\\n",
    "  \\vdots\\\\\\\\\n",
    "  e^{x_n}\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "This is implemented for you using numpy - we want to be able to apply the softmax function to a batch of samples at once, so we will use numpy's vectorized operations to do so.\n",
    "\n",
    "Our method also handles _stability issues_ that can occur when the values in `X` are very large. We will subtract the maximum value from each row of `X` to avoid overflow in the exponentiation. This isn't part of the softmax function itself, but it's a useful trick to know about."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def softmax(X):\n",
    "    X_max = np.max(X, axis=-1, keepdims=True)\n",
    "    exp = np.exp(X - X_max) # Subtract the max to avoid overflow in the exponentiation\n",
    "    return exp / np.sum(exp, axis=-1, keepdims=True)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's make sure that this works one vector at a time (and check that the components sum to one):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(softmax([10, 2, -3]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we are using our model to make predictions, we will want to be able to make predictions for multiple samples at once.\n",
    "Let's make sure that our implementation of softmax works for a batch of samples:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = np.array([[10, 2, -3],\n",
    "              [-1, 5, -20]])\n",
    "print(softmax(X))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Probabilities should sum to 1:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(np.sum(softmax([10, 2, -3])))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"softmax of 2 vectors:\")\n",
    "X = np.array([[10, 2, -3],\n",
    "              [-1, 5, -20]])\n",
    "print(softmax(X))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The sum of probabilities for each input vector of logits should some to 1:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(np.sum(softmax(X), axis=1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we will implement a function that, given the true one-hot encoded class `Y_true` and some predicted probabilities `Y_pred`, returns the negative log likelihood.\n",
    "\n",
    "Recall that the negative log likelihood is defined as follows:\n",
    "\n",
    "$$\n",
    "NLL(Y_{true}, Y_{pred}) = - \\sum_{i=1}^{n}{y_{true, i} \\cdot \\log(y_{pred, i})}\n",
    "$$\n",
    "\n",
    "For example, if we have $y_{true} = [1, 0, 0]$ and $y_{pred} = [0.99, 0.01, 0]$, then the negative log likelihood is $- \\log(0.99) \\approx 0.01$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def nll(Y_true, Y_pred):\n",
    "    Y_true = np.asarray(Y_true)\n",
    "    Y_pred = np.asarray(Y_pred)\n",
    "\n",
    "    # Ensure Y_pred doesn't have zero probabilities to avoid log(0)\n",
    "    Y_pred = np.clip(Y_pred, 1e-15, 1 - 1e-15)\n",
    "\n",
    "    # Calculate negative log likelihood\n",
    "    loss = -np.sum(Y_true * np.log(Y_pred))\n",
    "    return loss\n",
    "\n",
    "# Make sure that it works for a simple sample at a time\n",
    "print(nll([1, 0, 0], [.99, 0.01, 0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We should see a very high value for this negative log likelihood, since the model is very confident that the third class is the correct one, but the true class is the first one:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(nll([1, 0, 0], [0.01, 0.01, .98]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make sure that your implementation can compute the average negative log likelihood of a group of predictions: `Y_pred` and `Y_true` can therefore be past as 2D arrays:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Check that the average NLL of the following 3 almost perfect\n",
    "# predictions is close to 0\n",
    "Y_true = np.array([[0, 1, 0],\n",
    "                   [1, 0, 0],\n",
    "                   [0, 0, 1]])\n",
    "\n",
    "Y_pred = np.array([[0,   1,    0],\n",
    "                   [.99, 0.01, 0],\n",
    "                   [0,   0,    1]])\n",
    "\n",
    "print(nll(Y_true, Y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Now that we have our softmax and negative log likelihood functions, we can implement a logistic regression model. \n",
    "In this section, we have built the model for you, but you will need to complete a few key parts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class LogisticRegression:\n",
    "\n",
    "    def __init__(self, input_size, output_size):\n",
    "        # Initialize the weights and biases with random numbers\n",
    "        self.W = np.random.uniform(size=(input_size, output_size),\n",
    "                                   high=0.1, low=-0.1)\n",
    "        self.b = np.random.uniform(size=output_size,\n",
    "                                   high=0.1, low=-0.1)\n",
    "        \n",
    "        # Store the input size and output size\n",
    "        self.output_size = output_size\n",
    "        self.input_size = input_size\n",
    "        \n",
    "    def forward(self, X):\n",
    "        # Compute the linear combination of the input and weights\n",
    "        Z = None # TODO\n",
    "        \n",
    "        # Return the softmax of the linear combination\n",
    "        return None # TODO\n",
    "    \n",
    "    def predict(self, X):\n",
    "        # Return the most probable class for each sample in X\n",
    "        if len(X.shape) == 1:\n",
    "            return np.argmax(self.forward(X))\n",
    "        else:\n",
    "            return np.argmax(self.forward(X), axis=1)\n",
    "            \n",
    "    def loss(self, X, y):\n",
    "        # Compute the negative log likelihood over the data provided\n",
    "        y_onehot = one_hot(self.output_size, y)\n",
    "        return None # TODO\n",
    "\n",
    "    def grad_loss(self, X, y_true, y_pred):\n",
    "        # Compute the gradient of the loss with respect to W and b for a single sample (X, y_true)\n",
    "        # y_pred is the output of the forward pass\n",
    "    \n",
    "        # Gradient with respect to weights\n",
    "        grad_W = np.dot(X.T, (y_pred - y_true))\n",
    "    \n",
    "        # Gradient with respect to biases\n",
    "        grad_b = np.sum(y_pred - y_true, axis=0)\n",
    "    \n",
    "        return grad_W, grad_b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Build a model and test its forward inference\n",
    "n_features = X_train.shape[1]\n",
    "n_classes = len(np.unique(y_train))\n",
    "lr = LogisticRegression(n_features, n_classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "We can evaluate the model on an example, visualizing the prediction probabilities:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def plot_prediction(model, sample_idx=0, classes=range(10)):\n",
    "    fig, (ax0, ax1) = plt.subplots(nrows=1, ncols=2, figsize=(10, 4))\n",
    "\n",
    "    ax0.imshow(X_test[sample_idx:sample_idx+1].reshape(8, 8),\n",
    "               cmap=plt.cm.gray_r, interpolation='nearest')\n",
    "    ax0.set_title(\"True image label: %d\" % y_test[sample_idx]);\n",
    "\n",
    "\n",
    "    ax1.bar(classes, one_hot(len(classes), y_test[sample_idx]), label='true')\n",
    "    ax1.bar(classes, model.forward(X_test[sample_idx]), label='prediction', color=\"red\")\n",
    "    ax1.set_xticks(classes)\n",
    "    prediction = model.predict(X_test[sample_idx])\n",
    "    ax1.set_title('Output probabilities (prediction: %d)'\n",
    "                  % prediction)\n",
    "    ax1.set_xlabel('Digit class')\n",
    "    ax1.legend()\n",
    "    \n",
    "plot_prediction(lr, sample_idx=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Now it's time to start training! We will train for a single epoch, and then evaluate the model on the training and testing sets. Read through the following and make sure that you understand what we are doing here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "lr = LogisticRegression(input_size=X_train.shape[1], output_size=10)\n",
    "\n",
    "learning_rate = 0.01\n",
    "\n",
    "for i in range(len(X_train)):\n",
    "    # Get the current sample and corresponding label\n",
    "    x = X_train[i:i+1]  # Reshape to keep the batch dimension\n",
    "    y = y_train[i:i+1]  # Reshape to keep the batch dimension\n",
    "\n",
    "    # Compute the forward pass and the gradient of the loss with respect to W and b\n",
    "    y_pred = lr.forward(x)\n",
    "    grad_W, grad_b = lr.grad_loss(x, one_hot(lr.output_size, y), y_pred)\n",
    "\n",
    "    # Update the weights and biases\n",
    "    lr.W -= learning_rate * grad_W\n",
    "    lr.b -= learning_rate * grad_b\n",
    "\n",
    "    # Print the average negative log likelihood every 100 steps\n",
    "    if i % 100 == 0:\n",
    "        avg_nll = lr.loss(X_train[max(0, i-100):i], y_train[max(0, i-100):i])\n",
    "        print(\"Average NLL over the last 100 samples at step %d: %0.f\" % (i, avg_nll))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Evaluate the trained model on the first example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "plot_prediction(lr, sample_idx=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## b) Feedforward Multilayer\n",
    "\n",
    "The objective of this section is to implement the backpropagation algorithm (SGD with the chain rule) on a single layer neural network using the sigmoid activation function.\n",
    "\n",
    "Now it's your turn to\n",
    "\n",
    "- Implement the `sigmoid` and its element-wise derivative `dsigmoid` functions:\n",
    "\n",
    "$$\n",
    "sigmoid(x) = \\frac{1}{1 + e^{-x}}\n",
    "$$\n",
    "\n",
    "$$\n",
    "dsigmoid(x) = sigmoid(x) \\cdot (1 - sigmoid(x))\n",
    "$$\n",
    "\n",
    "Remember that you can use your `sigmoid` function inside your `dsigmoid` function.\n",
    "\n",
    "Just like with our softmax function, we also want to make sure that we don't run into stability issues with our sigmoid function. We will use `np.clip` to ensure that the input to the sigmoid function is not too large or too small."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sigmoid(X):\n",
    "    # Clip X to prevent overflow or underflow\n",
    "    X = np.clip(X, -500, 500) # This ensures that np.exp(X) doesn't overflow\n",
    "    return None # TODO\n",
    "\n",
    "\n",
    "def dsigmoid(X):\n",
    "    return None # TODO\n",
    "\n",
    "\n",
    "x = np.linspace(-5, 5, 100)\n",
    "plt.plot(x, sigmoid(x), label='sigmoid')\n",
    "plt.plot(x, dsigmoid(x), label='dsigmoid')\n",
    "plt.legend(loc='best');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now it's your turn to complete the neural network code, so that we can train it on the MNIST dataset.\n",
    "\n",
    "Some parts have been completed for you already. Often, you'll be able to refer back to the code from the previous section to help you complete the code in this section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class NeuralNet():\n",
    "    \"\"\"MLP with 1 hidden layer with a sigmoid activation\"\"\"\n",
    "\n",
    "    def __init__(self, input_size, hidden_size, output_size):\n",
    "        # Initializes the weights with random numbers\n",
    "        self.W_h = np.random.uniform(size=(input_size, hidden_size),\n",
    "                                     high=0.1, low=-0.1)\n",
    "        self.b_h = np.random.uniform(size=hidden_size,\n",
    "                                     high=0.1, low=-0.1)\n",
    "        self.W_o = np.random.uniform(size=(hidden_size, output_size),\n",
    "                                     high=0.1, low=-0.1)\n",
    "        self.b_o = np.random.uniform(size=output_size,\n",
    "                                     high=0.1, low=-0.1)\n",
    "\n",
    "        # Store the input size, hidden size and output size\n",
    "        self.input_size = input_size\n",
    "        self.hidden_size = hidden_size\n",
    "        self.output_size = output_size\n",
    "\n",
    "    def forward_hidden(self, X):\n",
    "        # Compute the linear combination of the input and weights\n",
    "        self.Z_h = None # TODO\n",
    "\n",
    "        # Apply the sigmoid activation function\n",
    "        return None # TODO\n",
    "\n",
    "    def forward_output(self, H):\n",
    "        # Compute the linear combination of the hidden layer activation and weights\n",
    "        self.Z_o = None # TODO\n",
    "\n",
    "        # Apply the sigmoid activation function\n",
    "        return None # TODO\n",
    "\n",
    "    def forward(self, X):\n",
    "        # Compute the forward activations of the hidden and output layers\n",
    "        H = self.forward_hidden(X)\n",
    "        Y = self.forward_output(H)\n",
    "\n",
    "        return Y\n",
    "\n",
    "    def loss(self, X, y):\n",
    "        y_onehot = one_hot(self.output_size, y)\n",
    "        return None # TODO\n",
    "\n",
    "    def grad_loss(self, X, y_true):\n",
    "        y_true = one_hot(self.output_size, y_true)\n",
    "        y_pred = self.forward(X)\n",
    "\n",
    "        # Compute the error at the output layer\n",
    "        error_o = y_pred - y_true\n",
    "\n",
    "        # Compute the gradient of the loss with respect to W_o and b_o\n",
    "        grad_W_o = np.dot(self.Z_h.T, error_o)\n",
    "        grad_b_o = np.sum(error_o, axis=0)\n",
    "\n",
    "        # Compute the error at the hidden layer\n",
    "        error_h = np.dot(error_o, self.W_o.T) * dsigmoid(self.Z_h)\n",
    "\n",
    "        # Compute the gradient of the loss with respect to W_h and b_h\n",
    "        grad_W_h = np.dot(X.T, error_h)\n",
    "        grad_b_h = np.sum(error_h, axis=0)\n",
    "\n",
    "        return {\"W_h\": grad_W_h, \"b_h\": grad_b_h, \"W_o\": grad_W_o, \"b_o\": grad_b_o}\n",
    "\n",
    "    def train(self, x, y, learning_rate):\n",
    "        # Ensure x is 2D\n",
    "        x = x[np.newaxis, :]\n",
    "        # Compute the gradient for the sample and update the weights\n",
    "        grads = self.grad_loss(x, y)\n",
    "    \n",
    "        self.W_h -= learning_rate * grads[\"W_h\"]\n",
    "        self.b_h -= learning_rate * grads[\"b_h\"]\n",
    "        self.W_o -= learning_rate * grads[\"W_o\"]\n",
    "        self.b_o -= learning_rate * grads[\"b_o\"]\n",
    "        \n",
    "    def predict(self, X):\n",
    "        if len(X.shape) == 1:\n",
    "            return np.argmax(self.forward(X))\n",
    "        else:\n",
    "            return np.argmax(self.forward(X), axis=1)\n",
    "\n",
    "    def accuracy(self, X, y):\n",
    "        y_preds = np.argmax(self.forward(X), axis=1)\n",
    "        return np.mean(y_preds == y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Once the code is written, we can test our model on a single sample:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "n_hidden = 10\n",
    "model = NeuralNet(n_features, n_hidden, n_classes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.loss(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.accuracy(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_prediction(model, sample_idx=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "And now it's time to train!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "losses, accuracies, accuracies_test = [], [], []\n",
    "losses.append(model.loss(X_train, y_train))\n",
    "accuracies.append(model.accuracy(X_train, y_train))\n",
    "accuracies_test.append(model.accuracy(X_test, y_test))\n",
    "\n",
    "print(\"Random init: train loss: %0.5f, train acc: %0.3f, test acc: %0.3f\"\n",
    "      % (losses[-1], accuracies[-1], accuracies_test[-1]))\n",
    "\n",
    "for epoch in range(15):\n",
    "    for i, (x, y) in enumerate(zip(X_train, y_train)):\n",
    "        model.train(x, y, 0.001)\n",
    "\n",
    "    losses.append(model.loss(X_train, y_train))\n",
    "    accuracies.append(model.accuracy(X_train, y_train))\n",
    "    accuracies_test.append(model.accuracy(X_test, y_test))\n",
    "    print(\"Epoch #%d, train loss: %0.5f, train acc: %0.3f, test acc: %0.3f\"\n",
    "          % (epoch + 1, losses[-1], accuracies[-1], accuracies_test[-1]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.plot(losses)\n",
    "plt.title(\"Training loss\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.plot(accuracies, label='train')\n",
    "plt.plot(accuracies_test, label='test')\n",
    "plt.ylim(0, 1.1)\n",
    "plt.ylabel(\"accuracy\")\n",
    "plt.legend(loc='best');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_prediction(model, sample_idx=4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## c) Exercises\n",
    "\n",
    "### Look at worst prediction errors\n",
    "\n",
    "- Use numpy to find test samples for which the model made the worst predictions,\n",
    "- Use the `plot_prediction` to look at the model predictions on those,\n",
    "- Would you have done any better?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hyper parameters settings\n",
    "\n",
    "- Experiment with different hyperparameters:\n",
    "  - learning rate,\n",
    "  - size of hidden layer,\n",
    "  - implement the support for a second hidden layer.\n",
    "  - What is the best test accuracy you can get?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Your code here"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}