ppcic_ml_ann_keras.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "ppcic_ml_ann_keras.ipynb",
      "provenance": [],
      "toc_visible": true,
      "authorship_tag": "ABX9TyM11Ly58b/aquZVp6ERd83l",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/MLRG-CEFET-RJ/ml-class/blob/master/ppcic_ml_ann_keras.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bodQcdklWBKs"
      },
      "source": [
        "$$\n",
        "\\text{ReLU(x)} = max(x,0)\n",
        "$$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ac4Fr6oNcwtf"
      },
      "source": [
        "# Example 1: ANN to learn to XOR function\n",
        "\n",
        "This example trains a very simple neural network to solve the XOR problem.\n",
        "\n",
        "![link text](https://www.tech-quantum.com/wp-content/uploads/2019/03/XOR-Problem-768x433.png)\n",
        "\n",
        "Notice that this a binary classification problem. The training set has four examples."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YgOGjx5Xb7HI"
      },
      "source": [
        "import numpy as np\n",
        "from keras.models import Sequential\n",
        "from keras.layers.core import Dense\n",
        "\n",
        "model = Sequential()\n",
        "\n",
        "l1 = Dense(16, input_dim=2, activation='relu')\n",
        "model.add(l1)\n",
        "\n",
        "l2 = Dense(1, activation='sigmoid')\n",
        "model.add(l2)\n",
        "\n",
        "model.compile(loss='mean_squared_error', optimizer='adam', metrics=['binary_accuracy'])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gfuTeErBfnqV",
        "outputId": "cfecf678-fe74-49fc-a94d-b3d143941d1d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 242
        }
      },
      "source": [
        "print(model.summary())"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_12\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "dense_25 (Dense)             (None, 16)                48        \n",
            "_________________________________________________________________\n",
            "dense_26 (Dense)             (None, 1)                 17        \n",
            "=================================================================\n",
            "Total params: 65\n",
            "Trainable params: 65\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n",
            "None\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "di5RgRw9_jdj"
      },
      "source": [
        "The image below (generated with this [tool](http://alexlenail.me/NN-SVG/index.html)) provides a visual perspective of the architecture."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aRh5ZJIcjLAs",
        "outputId": "8a4ea8fa-d67d-463a-aa3b-f86cc253aeeb",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 842
        }
      },
      "source": [
        "%%html\n",
        "<svg xmlns=\"http://www.w3.org/2000/svg\" width=\"1400\" height=\"800\" style=\"cursor: move;\"><g transform=\"translate(31.029124448207995,22.95774483714024) scale(0.9619274546647754)\"><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,90\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,130\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,170\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,210\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,250\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,290\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,330\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,370\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,410\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,450\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,490\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,530\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,90\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,130\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,170\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,210\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,250\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,290\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,330\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,370\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,410\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,450\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,490\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,530\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,90, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,130, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,170, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,210, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,250, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,290, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,330, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,370, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,410, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,450, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,490, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,530, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,570\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,610\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,650\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,690\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,570\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,610\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,650\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,690\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,570, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,610, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,650, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,690, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><circle r=\"10\" class=\"node\" id=\"0_0\" cx=\"540\" cy=\"370\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"0_1\" cx=\"540\" cy=\"410\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_0\" cx=\"720\" cy=\"90\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_1\" cx=\"720\" cy=\"130\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_2\" cx=\"720\" cy=\"170\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_3\" cx=\"720\" cy=\"210\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_4\" cx=\"720\" cy=\"250\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_5\" cx=\"720\" cy=\"290\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_6\" cx=\"720\" cy=\"330\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_7\" cx=\"720\" cy=\"370\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_8\" cx=\"720\" cy=\"410\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_9\" cx=\"720\" cy=\"450\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_10\" cx=\"720\" cy=\"490\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_11\" cx=\"720\" cy=\"530\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_12\" cx=\"720\" cy=\"570\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_13\" cx=\"720\" cy=\"610\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_14\" cx=\"720\" cy=\"650\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_15\" cx=\"720\" cy=\"690\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"2_0\" cx=\"900\" cy=\"390\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><text class=\"text\" dy=\".35em\" x=\"505\" y=\"730\" style=\"font-size: 12px;\">Input Layer ∈ ℝ²</text><text class=\"text\" dy=\".35em\" x=\"685\" y=\"730\" style=\"font-size: 12px;\">Hidden Layer ∈ ℝ¹⁶</text><text class=\"text\" dy=\".35em\" x=\"865\" y=\"730\" style=\"font-size: 12px;\">Output Layer ∈ ℝ¹</text></g><defs><marker id=\"arrow\" viewBox=\"0 -5 10 10\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto\" refX=\"40\"><path d=\"M0,-5L10,0L0,5\" style=\"stroke: rgb(80, 80, 80); fill: none;\"></path></marker></defs></svg>"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<svg xmlns=\"http://www.w3.org/2000/svg\" width=\"1400\" height=\"800\" style=\"cursor: move;\"><g transform=\"translate(31.029124448207995,22.95774483714024) scale(0.9619274546647754)\"><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,90\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,130\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,170\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,210\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,250\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,290\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,330\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,370\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,410\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,450\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,490\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,530\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,90\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,130\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,170\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,210\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,250\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,290\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,330\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,370\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,410\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,450\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,490\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,530\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,90, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,130, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,170, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,210, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,250, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,290, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,330, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,370, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,410, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,450, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,490, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,530, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,570\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,610\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,650\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,690\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,570\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,610\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,650\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,690\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,570, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,610, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,650, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,690, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><circle r=\"10\" class=\"node\" id=\"0_0\" cx=\"540\" cy=\"370\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"0_1\" cx=\"540\" cy=\"410\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_0\" cx=\"720\" cy=\"90\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_1\" cx=\"720\" cy=\"130\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_2\" cx=\"720\" cy=\"170\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_3\" cx=\"720\" cy=\"210\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_4\" cx=\"720\" cy=\"250\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_5\" cx=\"720\" cy=\"290\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_6\" cx=\"720\" cy=\"330\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_7\" cx=\"720\" cy=\"370\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_8\" cx=\"720\" cy=\"410\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_9\" cx=\"720\" cy=\"450\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_10\" cx=\"720\" cy=\"490\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_11\" cx=\"720\" cy=\"530\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_12\" cx=\"720\" cy=\"570\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_13\" cx=\"720\" cy=\"610\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_14\" cx=\"720\" cy=\"650\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_15\" cx=\"720\" cy=\"690\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"2_0\" cx=\"900\" cy=\"390\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><text class=\"text\" dy=\".35em\" x=\"505\" y=\"730\" style=\"font-size: 12px;\">Input Layer ∈ ℝ²</text><text class=\"text\" dy=\".35em\" x=\"685\" y=\"730\" style=\"font-size: 12px;\">Hidden Layer ∈ ℝ¹⁶</text><text class=\"text\" dy=\".35em\" x=\"865\" y=\"730\" style=\"font-size: 12px;\">Output Layer ∈ ℝ¹</text></g><defs><marker id=\"arrow\" viewBox=\"0 -5 10 10\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto\" refX=\"40\"><path d=\"M0,-5L10,0L0,5\" style=\"stroke: rgb(80, 80, 80); fill: none;\"></path></marker></defs></svg>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RzIIgYx3fpqA",
        "outputId": "da0e6848-58be-4bf0-d89f-17413be0495d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "X = np.array([[0,0],[0,1],[1,0],[1,1]], \"float32\")\n",
        "y = np.array([[0],[1],[1],[0]], \"float32\")\n",
        "\n",
        "model.fit(X, y, epochs=500, verbose=2)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/500\n",
            "1/1 - 0s - loss: 0.2599 - binary_accuracy: 0.5000\n",
            "Epoch 2/500\n",
            "1/1 - 0s - loss: 0.2594 - binary_accuracy: 0.2500\n",
            "Epoch 3/500\n",
            "1/1 - 0s - loss: 0.2589 - binary_accuracy: 0.2500\n",
            "Epoch 4/500\n",
            "1/1 - 0s - loss: 0.2583 - binary_accuracy: 0.2500\n",
            "Epoch 5/500\n",
            "1/1 - 0s - loss: 0.2579 - binary_accuracy: 0.2500\n",
            "Epoch 6/500\n",
            "1/1 - 0s - loss: 0.2574 - binary_accuracy: 0.2500\n",
            "Epoch 7/500\n",
            "1/1 - 0s - loss: 0.2569 - binary_accuracy: 0.2500\n",
            "Epoch 8/500\n",
            "1/1 - 0s - loss: 0.2564 - binary_accuracy: 0.2500\n",
            "Epoch 9/500\n",
            "1/1 - 0s - loss: 0.2560 - binary_accuracy: 0.2500\n",
            "Epoch 10/500\n",
            "1/1 - 0s - loss: 0.2555 - binary_accuracy: 0.2500\n",
            "Epoch 11/500\n",
            "1/1 - 0s - loss: 0.2551 - binary_accuracy: 0.2500\n",
            "Epoch 12/500\n",
            "1/1 - 0s - loss: 0.2547 - binary_accuracy: 0.2500\n",
            "Epoch 13/500\n",
            "1/1 - 0s - loss: 0.2542 - binary_accuracy: 0.2500\n",
            "Epoch 14/500\n",
            "1/1 - 0s - loss: 0.2538 - binary_accuracy: 0.2500\n",
            "Epoch 15/500\n",
            "1/1 - 0s - loss: 0.2534 - binary_accuracy: 0.5000\n",
            "Epoch 16/500\n",
            "1/1 - 0s - loss: 0.2530 - binary_accuracy: 0.5000\n",
            "Epoch 17/500\n",
            "1/1 - 0s - loss: 0.2526 - binary_accuracy: 0.5000\n",
            "Epoch 18/500\n",
            "1/1 - 0s - loss: 0.2523 - binary_accuracy: 0.5000\n",
            "Epoch 19/500\n",
            "1/1 - 0s - loss: 0.2519 - binary_accuracy: 0.5000\n",
            "Epoch 20/500\n",
            "1/1 - 0s - loss: 0.2515 - binary_accuracy: 0.5000\n",
            "Epoch 21/500\n",
            "1/1 - 0s - loss: 0.2512 - binary_accuracy: 0.5000\n",
            "Epoch 22/500\n",
            "1/1 - 0s - loss: 0.2508 - binary_accuracy: 0.5000\n",
            "Epoch 23/500\n",
            "1/1 - 0s - loss: 0.2505 - binary_accuracy: 0.5000\n",
            "Epoch 24/500\n",
            "1/1 - 0s - loss: 0.2501 - binary_accuracy: 0.5000\n",
            "Epoch 25/500\n",
            "1/1 - 0s - loss: 0.2498 - binary_accuracy: 0.5000\n",
            "Epoch 26/500\n",
            "1/1 - 0s - loss: 0.2495 - binary_accuracy: 0.5000\n",
            "Epoch 27/500\n",
            "1/1 - 0s - loss: 0.2492 - binary_accuracy: 0.5000\n",
            "Epoch 28/500\n",
            "1/1 - 0s - loss: 0.2488 - binary_accuracy: 0.5000\n",
            "Epoch 29/500\n",
            "1/1 - 0s - loss: 0.2485 - binary_accuracy: 0.5000\n",
            "Epoch 30/500\n",
            "1/1 - 0s - loss: 0.2482 - binary_accuracy: 0.5000\n",
            "Epoch 31/500\n",
            "1/1 - 0s - loss: 0.2479 - binary_accuracy: 0.5000\n",
            "Epoch 32/500\n",
            "1/1 - 0s - loss: 0.2476 - binary_accuracy: 0.5000\n",
            "Epoch 33/500\n",
            "1/1 - 0s - loss: 0.2473 - binary_accuracy: 0.5000\n",
            "Epoch 34/500\n",
            "1/1 - 0s - loss: 0.2471 - binary_accuracy: 0.5000\n",
            "Epoch 35/500\n",
            "1/1 - 0s - loss: 0.2468 - binary_accuracy: 0.5000\n",
            "Epoch 36/500\n",
            "1/1 - 0s - loss: 0.2465 - binary_accuracy: 0.5000\n",
            "Epoch 37/500\n",
            "1/1 - 0s - loss: 0.2462 - binary_accuracy: 0.5000\n",
            "Epoch 38/500\n",
            "1/1 - 0s - loss: 0.2459 - binary_accuracy: 0.5000\n",
            "Epoch 39/500\n",
            "1/1 - 0s - loss: 0.2456 - binary_accuracy: 0.5000\n",
            "Epoch 40/500\n",
            "1/1 - 0s - loss: 0.2453 - binary_accuracy: 0.5000\n",
            "Epoch 41/500\n",
            "1/1 - 0s - loss: 0.2451 - binary_accuracy: 0.5000\n",
            "Epoch 42/500\n",
            "1/1 - 0s - loss: 0.2448 - binary_accuracy: 0.5000\n",
            "Epoch 43/500\n",
            "1/1 - 0s - loss: 0.2445 - binary_accuracy: 0.5000\n",
            "Epoch 44/500\n",
            "1/1 - 0s - loss: 0.2442 - binary_accuracy: 0.5000\n",
            "Epoch 45/500\n",
            "1/1 - 0s - loss: 0.2439 - binary_accuracy: 0.5000\n",
            "Epoch 46/500\n",
            "1/1 - 0s - loss: 0.2437 - binary_accuracy: 0.5000\n",
            "Epoch 47/500\n",
            "1/1 - 0s - loss: 0.2434 - binary_accuracy: 0.5000\n",
            "Epoch 48/500\n",
            "1/1 - 0s - loss: 0.2431 - binary_accuracy: 0.5000\n",
            "Epoch 49/500\n",
            "1/1 - 0s - loss: 0.2428 - binary_accuracy: 0.5000\n",
            "Epoch 50/500\n",
            "1/1 - 0s - loss: 0.2426 - binary_accuracy: 0.5000\n",
            "Epoch 51/500\n",
            "1/1 - 0s - loss: 0.2423 - binary_accuracy: 0.5000\n",
            "Epoch 52/500\n",
            "1/1 - 0s - loss: 0.2420 - binary_accuracy: 0.5000\n",
            "Epoch 53/500\n",
            "1/1 - 0s - loss: 0.2417 - binary_accuracy: 0.5000\n",
            "Epoch 54/500\n",
            "1/1 - 0s - loss: 0.2414 - binary_accuracy: 0.5000\n",
            "Epoch 55/500\n",
            "1/1 - 0s - loss: 0.2411 - binary_accuracy: 0.5000\n",
            "Epoch 56/500\n",
            "1/1 - 0s - loss: 0.2409 - binary_accuracy: 0.5000\n",
            "Epoch 57/500\n",
            "1/1 - 0s - loss: 0.2406 - binary_accuracy: 0.5000\n",
            "Epoch 58/500\n",
            "1/1 - 0s - loss: 0.2403 - binary_accuracy: 0.5000\n",
            "Epoch 59/500\n",
            "1/1 - 0s - loss: 0.2400 - binary_accuracy: 0.5000\n",
            "Epoch 60/500\n",
            "1/1 - 0s - loss: 0.2397 - binary_accuracy: 0.5000\n",
            "Epoch 61/500\n",
            "1/1 - 0s - loss: 0.2394 - binary_accuracy: 0.5000\n",
            "Epoch 62/500\n",
            "1/1 - 0s - loss: 0.2391 - binary_accuracy: 0.5000\n",
            "Epoch 63/500\n",
            "1/1 - 0s - loss: 0.2389 - binary_accuracy: 0.5000\n",
            "Epoch 64/500\n",
            "1/1 - 0s - loss: 0.2386 - binary_accuracy: 0.5000\n",
            "Epoch 65/500\n",
            "1/1 - 0s - loss: 0.2383 - binary_accuracy: 0.5000\n",
            "Epoch 66/500\n",
            "1/1 - 0s - loss: 0.2380 - binary_accuracy: 0.5000\n",
            "Epoch 67/500\n",
            "1/1 - 0s - loss: 0.2377 - binary_accuracy: 0.5000\n",
            "Epoch 68/500\n",
            "1/1 - 0s - loss: 0.2374 - binary_accuracy: 0.5000\n",
            "Epoch 69/500\n",
            "1/1 - 0s - loss: 0.2371 - binary_accuracy: 0.5000\n",
            "Epoch 70/500\n",
            "1/1 - 0s - loss: 0.2369 - binary_accuracy: 0.5000\n",
            "Epoch 71/500\n",
            "1/1 - 0s - loss: 0.2366 - binary_accuracy: 0.5000\n",
            "Epoch 72/500\n",
            "1/1 - 0s - loss: 0.2363 - binary_accuracy: 0.5000\n",
            "Epoch 73/500\n",
            "1/1 - 0s - loss: 0.2360 - binary_accuracy: 0.5000\n",
            "Epoch 74/500\n",
            "1/1 - 0s - loss: 0.2357 - binary_accuracy: 0.5000\n",
            "Epoch 75/500\n",
            "1/1 - 0s - loss: 0.2354 - binary_accuracy: 0.5000\n",
            "Epoch 76/500\n",
            "1/1 - 0s - loss: 0.2351 - binary_accuracy: 0.5000\n",
            "Epoch 77/500\n",
            "1/1 - 0s - loss: 0.2348 - binary_accuracy: 0.5000\n",
            "Epoch 78/500\n",
            "1/1 - 0s - loss: 0.2345 - binary_accuracy: 0.5000\n",
            "Epoch 79/500\n",
            "1/1 - 0s - loss: 0.2343 - binary_accuracy: 0.5000\n",
            "Epoch 80/500\n",
            "1/1 - 0s - loss: 0.2340 - binary_accuracy: 0.5000\n",
            "Epoch 81/500\n",
            "1/1 - 0s - loss: 0.2337 - binary_accuracy: 0.5000\n",
            "Epoch 82/500\n",
            "1/1 - 0s - loss: 0.2335 - binary_accuracy: 0.5000\n",
            "Epoch 83/500\n",
            "1/1 - 0s - loss: 0.2332 - binary_accuracy: 0.5000\n",
            "Epoch 84/500\n",
            "1/1 - 0s - loss: 0.2329 - binary_accuracy: 0.5000\n",
            "Epoch 85/500\n",
            "1/1 - 0s - loss: 0.2326 - binary_accuracy: 0.5000\n",
            "Epoch 86/500\n",
            "1/1 - 0s - loss: 0.2324 - binary_accuracy: 0.5000\n",
            "Epoch 87/500\n",
            "1/1 - 0s - loss: 0.2321 - binary_accuracy: 0.5000\n",
            "Epoch 88/500\n",
            "1/1 - 0s - loss: 0.2318 - binary_accuracy: 0.5000\n",
            "Epoch 89/500\n",
            "1/1 - 0s - loss: 0.2315 - binary_accuracy: 0.5000\n",
            "Epoch 90/500\n",
            "1/1 - 0s - loss: 0.2313 - binary_accuracy: 0.5000\n",
            "Epoch 91/500\n",
            "1/1 - 0s - loss: 0.2310 - binary_accuracy: 0.5000\n",
            "Epoch 92/500\n",
            "1/1 - 0s - loss: 0.2307 - binary_accuracy: 0.7500\n",
            "Epoch 93/500\n",
            "1/1 - 0s - loss: 0.2304 - binary_accuracy: 0.7500\n",
            "Epoch 94/500\n",
            "1/1 - 0s - loss: 0.2302 - binary_accuracy: 0.7500\n",
            "Epoch 95/500\n",
            "1/1 - 0s - loss: 0.2299 - binary_accuracy: 0.7500\n",
            "Epoch 96/500\n",
            "1/1 - 0s - loss: 0.2296 - binary_accuracy: 0.7500\n",
            "Epoch 97/500\n",
            "1/1 - 0s - loss: 0.2293 - binary_accuracy: 0.7500\n",
            "Epoch 98/500\n",
            "1/1 - 0s - loss: 0.2290 - binary_accuracy: 0.7500\n",
            "Epoch 99/500\n",
            "1/1 - 0s - loss: 0.2287 - binary_accuracy: 0.7500\n",
            "Epoch 100/500\n",
            "1/1 - 0s - loss: 0.2285 - binary_accuracy: 0.7500\n",
            "Epoch 101/500\n",
            "1/1 - 0s - loss: 0.2282 - binary_accuracy: 0.7500\n",
            "Epoch 102/500\n",
            "1/1 - 0s - loss: 0.2279 - binary_accuracy: 0.7500\n",
            "Epoch 103/500\n",
            "1/1 - 0s - loss: 0.2276 - binary_accuracy: 0.7500\n",
            "Epoch 104/500\n",
            "1/1 - 0s - loss: 0.2273 - binary_accuracy: 0.7500\n",
            "Epoch 105/500\n",
            "1/1 - 0s - loss: 0.2270 - binary_accuracy: 0.7500\n",
            "Epoch 106/500\n",
            "1/1 - 0s - loss: 0.2267 - binary_accuracy: 1.0000\n",
            "Epoch 107/500\n",
            "1/1 - 0s - loss: 0.2264 - binary_accuracy: 1.0000\n",
            "Epoch 108/500\n",
            "1/1 - 0s - loss: 0.2262 - binary_accuracy: 1.0000\n",
            "Epoch 109/500\n",
            "1/1 - 0s - loss: 0.2259 - binary_accuracy: 1.0000\n",
            "Epoch 110/500\n",
            "1/1 - 0s - loss: 0.2256 - binary_accuracy: 1.0000\n",
            "Epoch 111/500\n",
            "1/1 - 0s - loss: 0.2253 - binary_accuracy: 1.0000\n",
            "Epoch 112/500\n",
            "1/1 - 0s - loss: 0.2250 - binary_accuracy: 1.0000\n",
            "Epoch 113/500\n",
            "1/1 - 0s - loss: 0.2247 - binary_accuracy: 1.0000\n",
            "Epoch 114/500\n",
            "1/1 - 0s - loss: 0.2244 - binary_accuracy: 1.0000\n",
            "Epoch 115/500\n",
            "1/1 - 0s - loss: 0.2241 - binary_accuracy: 1.0000\n",
            "Epoch 116/500\n",
            "1/1 - 0s - loss: 0.2238 - binary_accuracy: 1.0000\n",
            "Epoch 117/500\n",
            "1/1 - 0s - loss: 0.2235 - binary_accuracy: 1.0000\n",
            "Epoch 118/500\n",
            "1/1 - 0s - loss: 0.2232 - binary_accuracy: 1.0000\n",
            "Epoch 119/500\n",
            "1/1 - 0s - loss: 0.2229 - binary_accuracy: 1.0000\n",
            "Epoch 120/500\n",
            "1/1 - 0s - loss: 0.2226 - binary_accuracy: 1.0000\n",
            "Epoch 121/500\n",
            "1/1 - 0s - loss: 0.2223 - binary_accuracy: 1.0000\n",
            "Epoch 122/500\n",
            "1/1 - 0s - loss: 0.2220 - binary_accuracy: 1.0000\n",
            "Epoch 123/500\n",
            "1/1 - 0s - loss: 0.2217 - binary_accuracy: 1.0000\n",
            "Epoch 124/500\n",
            "1/1 - 0s - loss: 0.2214 - binary_accuracy: 1.0000\n",
            "Epoch 125/500\n",
            "1/1 - 0s - loss: 0.2211 - binary_accuracy: 1.0000\n",
            "Epoch 126/500\n",
            "1/1 - 0s - loss: 0.2208 - binary_accuracy: 1.0000\n",
            "Epoch 127/500\n",
            "1/1 - 0s - loss: 0.2205 - binary_accuracy: 1.0000\n",
            "Epoch 128/500\n",
            "1/1 - 0s - loss: 0.2202 - binary_accuracy: 1.0000\n",
            "Epoch 129/500\n",
            "1/1 - 0s - loss: 0.2199 - binary_accuracy: 1.0000\n",
            "Epoch 130/500\n",
            "1/1 - 0s - loss: 0.2196 - binary_accuracy: 1.0000\n",
            "Epoch 131/500\n",
            "1/1 - 0s - loss: 0.2192 - binary_accuracy: 1.0000\n",
            "Epoch 132/500\n",
            "1/1 - 0s - loss: 0.2189 - binary_accuracy: 1.0000\n",
            "Epoch 133/500\n",
            "1/1 - 0s - loss: 0.2186 - binary_accuracy: 1.0000\n",
            "Epoch 134/500\n",
            "1/1 - 0s - loss: 0.2183 - binary_accuracy: 1.0000\n",
            "Epoch 135/500\n",
            "1/1 - 0s - loss: 0.2180 - binary_accuracy: 1.0000\n",
            "Epoch 136/500\n",
            "1/1 - 0s - loss: 0.2177 - binary_accuracy: 1.0000\n",
            "Epoch 137/500\n",
            "1/1 - 0s - loss: 0.2174 - binary_accuracy: 1.0000\n",
            "Epoch 138/500\n",
            "1/1 - 0s - loss: 0.2171 - binary_accuracy: 1.0000\n",
            "Epoch 139/500\n",
            "1/1 - 0s - loss: 0.2168 - binary_accuracy: 1.0000\n",
            "Epoch 140/500\n",
            "1/1 - 0s - loss: 0.2165 - binary_accuracy: 1.0000\n",
            "Epoch 141/500\n",
            "1/1 - 0s - loss: 0.2162 - binary_accuracy: 1.0000\n",
            "Epoch 142/500\n",
            "1/1 - 0s - loss: 0.2159 - binary_accuracy: 1.0000\n",
            "Epoch 143/500\n",
            "1/1 - 0s - loss: 0.2155 - binary_accuracy: 1.0000\n",
            "Epoch 144/500\n",
            "1/1 - 0s - loss: 0.2152 - binary_accuracy: 1.0000\n",
            "Epoch 145/500\n",
            "1/1 - 0s - loss: 0.2149 - binary_accuracy: 1.0000\n",
            "Epoch 146/500\n",
            "1/1 - 0s - loss: 0.2146 - binary_accuracy: 1.0000\n",
            "Epoch 147/500\n",
            "1/1 - 0s - loss: 0.2143 - binary_accuracy: 1.0000\n",
            "Epoch 148/500\n",
            "1/1 - 0s - loss: 0.2140 - binary_accuracy: 1.0000\n",
            "Epoch 149/500\n",
            "1/1 - 0s - loss: 0.2137 - binary_accuracy: 1.0000\n",
            "Epoch 150/500\n",
            "1/1 - 0s - loss: 0.2133 - binary_accuracy: 1.0000\n",
            "Epoch 151/500\n",
            "1/1 - 0s - loss: 0.2130 - binary_accuracy: 1.0000\n",
            "Epoch 152/500\n",
            "1/1 - 0s - loss: 0.2127 - binary_accuracy: 1.0000\n",
            "Epoch 153/500\n",
            "1/1 - 0s - loss: 0.2124 - binary_accuracy: 1.0000\n",
            "Epoch 154/500\n",
            "1/1 - 0s - loss: 0.2121 - binary_accuracy: 1.0000\n",
            "Epoch 155/500\n",
            "1/1 - 0s - loss: 0.2118 - binary_accuracy: 1.0000\n",
            "Epoch 156/500\n",
            "1/1 - 0s - loss: 0.2114 - binary_accuracy: 1.0000\n",
            "Epoch 157/500\n",
            "1/1 - 0s - loss: 0.2111 - binary_accuracy: 1.0000\n",
            "Epoch 158/500\n",
            "1/1 - 0s - loss: 0.2108 - binary_accuracy: 1.0000\n",
            "Epoch 159/500\n",
            "1/1 - 0s - loss: 0.2105 - binary_accuracy: 1.0000\n",
            "Epoch 160/500\n",
            "1/1 - 0s - loss: 0.2101 - binary_accuracy: 1.0000\n",
            "Epoch 161/500\n",
            "1/1 - 0s - loss: 0.2098 - binary_accuracy: 1.0000\n",
            "Epoch 162/500\n",
            "1/1 - 0s - loss: 0.2095 - binary_accuracy: 1.0000\n",
            "Epoch 163/500\n",
            "1/1 - 0s - loss: 0.2092 - binary_accuracy: 1.0000\n",
            "Epoch 164/500\n",
            "1/1 - 0s - loss: 0.2088 - binary_accuracy: 1.0000\n",
            "Epoch 165/500\n",
            "1/1 - 0s - loss: 0.2085 - binary_accuracy: 1.0000\n",
            "Epoch 166/500\n",
            "1/1 - 0s - loss: 0.2082 - binary_accuracy: 1.0000\n",
            "Epoch 167/500\n",
            "1/1 - 0s - loss: 0.2078 - binary_accuracy: 1.0000\n",
            "Epoch 168/500\n",
            "1/1 - 0s - loss: 0.2075 - binary_accuracy: 1.0000\n",
            "Epoch 169/500\n",
            "1/1 - 0s - loss: 0.2072 - binary_accuracy: 1.0000\n",
            "Epoch 170/500\n",
            "1/1 - 0s - loss: 0.2068 - binary_accuracy: 1.0000\n",
            "Epoch 171/500\n",
            "1/1 - 0s - loss: 0.2065 - binary_accuracy: 1.0000\n",
            "Epoch 172/500\n",
            "1/1 - 0s - loss: 0.2062 - binary_accuracy: 1.0000\n",
            "Epoch 173/500\n",
            "1/1 - 0s - loss: 0.2058 - binary_accuracy: 1.0000\n",
            "Epoch 174/500\n",
            "1/1 - 0s - loss: 0.2055 - binary_accuracy: 1.0000\n",
            "Epoch 175/500\n",
            "1/1 - 0s - loss: 0.2052 - binary_accuracy: 1.0000\n",
            "Epoch 176/500\n",
            "1/1 - 0s - loss: 0.2049 - binary_accuracy: 1.0000\n",
            "Epoch 177/500\n",
            "1/1 - 0s - loss: 0.2045 - binary_accuracy: 1.0000\n",
            "Epoch 178/500\n",
            "1/1 - 0s - loss: 0.2042 - binary_accuracy: 1.0000\n",
            "Epoch 179/500\n",
            "1/1 - 0s - loss: 0.2039 - binary_accuracy: 1.0000\n",
            "Epoch 180/500\n",
            "1/1 - 0s - loss: 0.2035 - binary_accuracy: 1.0000\n",
            "Epoch 181/500\n",
            "1/1 - 0s - loss: 0.2032 - binary_accuracy: 1.0000\n",
            "Epoch 182/500\n",
            "1/1 - 0s - loss: 0.2028 - binary_accuracy: 1.0000\n",
            "Epoch 183/500\n",
            "1/1 - 0s - loss: 0.2025 - binary_accuracy: 1.0000\n",
            "Epoch 184/500\n",
            "1/1 - 0s - loss: 0.2022 - binary_accuracy: 1.0000\n",
            "Epoch 185/500\n",
            "1/1 - 0s - loss: 0.2018 - binary_accuracy: 1.0000\n",
            "Epoch 186/500\n",
            "1/1 - 0s - loss: 0.2015 - binary_accuracy: 1.0000\n",
            "Epoch 187/500\n",
            "1/1 - 0s - loss: 0.2011 - binary_accuracy: 1.0000\n",
            "Epoch 188/500\n",
            "1/1 - 0s - loss: 0.2008 - binary_accuracy: 1.0000\n",
            "Epoch 189/500\n",
            "1/1 - 0s - loss: 0.2004 - binary_accuracy: 1.0000\n",
            "Epoch 190/500\n",
            "1/1 - 0s - loss: 0.2001 - binary_accuracy: 1.0000\n",
            "Epoch 191/500\n",
            "1/1 - 0s - loss: 0.1997 - binary_accuracy: 1.0000\n",
            "Epoch 192/500\n",
            "1/1 - 0s - loss: 0.1994 - binary_accuracy: 1.0000\n",
            "Epoch 193/500\n",
            "1/1 - 0s - loss: 0.1990 - binary_accuracy: 1.0000\n",
            "Epoch 194/500\n",
            "1/1 - 0s - loss: 0.1987 - binary_accuracy: 1.0000\n",
            "Epoch 195/500\n",
            "1/1 - 0s - loss: 0.1984 - binary_accuracy: 1.0000\n",
            "Epoch 196/500\n",
            "1/1 - 0s - loss: 0.1980 - binary_accuracy: 1.0000\n",
            "Epoch 197/500\n",
            "1/1 - 0s - loss: 0.1977 - binary_accuracy: 1.0000\n",
            "Epoch 198/500\n",
            "1/1 - 0s - loss: 0.1973 - binary_accuracy: 1.0000\n",
            "Epoch 199/500\n",
            "1/1 - 0s - loss: 0.1970 - binary_accuracy: 1.0000\n",
            "Epoch 200/500\n",
            "1/1 - 0s - loss: 0.1966 - binary_accuracy: 1.0000\n",
            "Epoch 201/500\n",
            "1/1 - 0s - loss: 0.1963 - binary_accuracy: 1.0000\n",
            "Epoch 202/500\n",
            "1/1 - 0s - loss: 0.1959 - binary_accuracy: 1.0000\n",
            "Epoch 203/500\n",
            "1/1 - 0s - loss: 0.1955 - binary_accuracy: 1.0000\n",
            "Epoch 204/500\n",
            "1/1 - 0s - loss: 0.1952 - binary_accuracy: 1.0000\n",
            "Epoch 205/500\n",
            "1/1 - 0s - loss: 0.1948 - binary_accuracy: 1.0000\n",
            "Epoch 206/500\n",
            "1/1 - 0s - loss: 0.1945 - binary_accuracy: 1.0000\n",
            "Epoch 207/500\n",
            "1/1 - 0s - loss: 0.1941 - binary_accuracy: 1.0000\n",
            "Epoch 208/500\n",
            "1/1 - 0s - loss: 0.1938 - binary_accuracy: 1.0000\n",
            "Epoch 209/500\n",
            "1/1 - 0s - loss: 0.1934 - binary_accuracy: 1.0000\n",
            "Epoch 210/500\n",
            "1/1 - 0s - loss: 0.1931 - binary_accuracy: 1.0000\n",
            "Epoch 211/500\n",
            "1/1 - 0s - loss: 0.1927 - binary_accuracy: 1.0000\n",
            "Epoch 212/500\n",
            "1/1 - 0s - loss: 0.1924 - binary_accuracy: 1.0000\n",
            "Epoch 213/500\n",
            "1/1 - 0s - loss: 0.1920 - binary_accuracy: 1.0000\n",
            "Epoch 214/500\n",
            "1/1 - 0s - loss: 0.1917 - binary_accuracy: 1.0000\n",
            "Epoch 215/500\n",
            "1/1 - 0s - loss: 0.1913 - binary_accuracy: 1.0000\n",
            "Epoch 216/500\n",
            "1/1 - 0s - loss: 0.1910 - binary_accuracy: 1.0000\n",
            "Epoch 217/500\n",
            "1/1 - 0s - loss: 0.1906 - binary_accuracy: 1.0000\n",
            "Epoch 218/500\n",
            "1/1 - 0s - loss: 0.1903 - binary_accuracy: 1.0000\n",
            "Epoch 219/500\n",
            "1/1 - 0s - loss: 0.1899 - binary_accuracy: 1.0000\n",
            "Epoch 220/500\n",
            "1/1 - 0s - loss: 0.1896 - binary_accuracy: 1.0000\n",
            "Epoch 221/500\n",
            "1/1 - 0s - loss: 0.1892 - binary_accuracy: 1.0000\n",
            "Epoch 222/500\n",
            "1/1 - 0s - loss: 0.1888 - binary_accuracy: 1.0000\n",
            "Epoch 223/500\n",
            "1/1 - 0s - loss: 0.1885 - binary_accuracy: 1.0000\n",
            "Epoch 224/500\n",
            "1/1 - 0s - loss: 0.1881 - binary_accuracy: 1.0000\n",
            "Epoch 225/500\n",
            "1/1 - 0s - loss: 0.1878 - binary_accuracy: 1.0000\n",
            "Epoch 226/500\n",
            "1/1 - 0s - loss: 0.1874 - binary_accuracy: 1.0000\n",
            "Epoch 227/500\n",
            "1/1 - 0s - loss: 0.1871 - binary_accuracy: 1.0000\n",
            "Epoch 228/500\n",
            "1/1 - 0s - loss: 0.1867 - binary_accuracy: 1.0000\n",
            "Epoch 229/500\n",
            "1/1 - 0s - loss: 0.1863 - binary_accuracy: 1.0000\n",
            "Epoch 230/500\n",
            "1/1 - 0s - loss: 0.1860 - binary_accuracy: 1.0000\n",
            "Epoch 231/500\n",
            "1/1 - 0s - loss: 0.1856 - binary_accuracy: 1.0000\n",
            "Epoch 232/500\n",
            "1/1 - 0s - loss: 0.1853 - binary_accuracy: 1.0000\n",
            "Epoch 233/500\n",
            "1/1 - 0s - loss: 0.1849 - binary_accuracy: 1.0000\n",
            "Epoch 234/500\n",
            "1/1 - 0s - loss: 0.1845 - binary_accuracy: 1.0000\n",
            "Epoch 235/500\n",
            "1/1 - 0s - loss: 0.1842 - binary_accuracy: 1.0000\n",
            "Epoch 236/500\n",
            "1/1 - 0s - loss: 0.1838 - binary_accuracy: 1.0000\n",
            "Epoch 237/500\n",
            "1/1 - 0s - loss: 0.1835 - binary_accuracy: 1.0000\n",
            "Epoch 238/500\n",
            "1/1 - 0s - loss: 0.1831 - binary_accuracy: 1.0000\n",
            "Epoch 239/500\n",
            "1/1 - 0s - loss: 0.1827 - binary_accuracy: 1.0000\n",
            "Epoch 240/500\n",
            "1/1 - 0s - loss: 0.1824 - binary_accuracy: 1.0000\n",
            "Epoch 241/500\n",
            "1/1 - 0s - loss: 0.1820 - binary_accuracy: 1.0000\n",
            "Epoch 242/500\n",
            "1/1 - 0s - loss: 0.1816 - binary_accuracy: 1.0000\n",
            "Epoch 243/500\n",
            "1/1 - 0s - loss: 0.1813 - binary_accuracy: 1.0000\n",
            "Epoch 244/500\n",
            "1/1 - 0s - loss: 0.1809 - binary_accuracy: 1.0000\n",
            "Epoch 245/500\n",
            "1/1 - 0s - loss: 0.1805 - binary_accuracy: 1.0000\n",
            "Epoch 246/500\n",
            "1/1 - 0s - loss: 0.1802 - binary_accuracy: 1.0000\n",
            "Epoch 247/500\n",
            "1/1 - 0s - loss: 0.1799 - binary_accuracy: 1.0000\n",
            "Epoch 248/500\n",
            "1/1 - 0s - loss: 0.1795 - binary_accuracy: 1.0000\n",
            "Epoch 249/500\n",
            "1/1 - 0s - loss: 0.1792 - binary_accuracy: 1.0000\n",
            "Epoch 250/500\n",
            "1/1 - 0s - loss: 0.1788 - binary_accuracy: 1.0000\n",
            "Epoch 251/500\n",
            "1/1 - 0s - loss: 0.1785 - binary_accuracy: 1.0000\n",
            "Epoch 252/500\n",
            "1/1 - 0s - loss: 0.1781 - binary_accuracy: 1.0000\n",
            "Epoch 253/500\n",
            "1/1 - 0s - loss: 0.1777 - binary_accuracy: 1.0000\n",
            "Epoch 254/500\n",
            "1/1 - 0s - loss: 0.1773 - binary_accuracy: 1.0000\n",
            "Epoch 255/500\n",
            "1/1 - 0s - loss: 0.1769 - binary_accuracy: 1.0000\n",
            "Epoch 256/500\n",
            "1/1 - 0s - loss: 0.1765 - binary_accuracy: 1.0000\n",
            "Epoch 257/500\n",
            "1/1 - 0s - loss: 0.1761 - binary_accuracy: 1.0000\n",
            "Epoch 258/500\n",
            "1/1 - 0s - loss: 0.1756 - binary_accuracy: 1.0000\n",
            "Epoch 259/500\n",
            "1/1 - 0s - loss: 0.1752 - binary_accuracy: 1.0000\n",
            "Epoch 260/500\n",
            "1/1 - 0s - loss: 0.1748 - binary_accuracy: 1.0000\n",
            "Epoch 261/500\n",
            "1/1 - 0s - loss: 0.1743 - binary_accuracy: 1.0000\n",
            "Epoch 262/500\n",
            "1/1 - 0s - loss: 0.1739 - binary_accuracy: 1.0000\n",
            "Epoch 263/500\n",
            "1/1 - 0s - loss: 0.1734 - binary_accuracy: 1.0000\n",
            "Epoch 264/500\n",
            "1/1 - 0s - loss: 0.1730 - binary_accuracy: 1.0000\n",
            "Epoch 265/500\n",
            "1/1 - 0s - loss: 0.1725 - binary_accuracy: 1.0000\n",
            "Epoch 266/500\n",
            "1/1 - 0s - loss: 0.1721 - binary_accuracy: 1.0000\n",
            "Epoch 267/500\n",
            "1/1 - 0s - loss: 0.1716 - binary_accuracy: 1.0000\n",
            "Epoch 268/500\n",
            "1/1 - 0s - loss: 0.1712 - binary_accuracy: 1.0000\n",
            "Epoch 269/500\n",
            "1/1 - 0s - loss: 0.1707 - binary_accuracy: 1.0000\n",
            "Epoch 270/500\n",
            "1/1 - 0s - loss: 0.1703 - binary_accuracy: 1.0000\n",
            "Epoch 271/500\n",
            "1/1 - 0s - loss: 0.1698 - binary_accuracy: 1.0000\n",
            "Epoch 272/500\n",
            "1/1 - 0s - loss: 0.1694 - binary_accuracy: 1.0000\n",
            "Epoch 273/500\n",
            "1/1 - 0s - loss: 0.1689 - binary_accuracy: 1.0000\n",
            "Epoch 274/500\n",
            "1/1 - 0s - loss: 0.1685 - binary_accuracy: 1.0000\n",
            "Epoch 275/500\n",
            "1/1 - 0s - loss: 0.1680 - binary_accuracy: 1.0000\n",
            "Epoch 276/500\n",
            "1/1 - 0s - loss: 0.1676 - binary_accuracy: 1.0000\n",
            "Epoch 277/500\n",
            "1/1 - 0s - loss: 0.1671 - binary_accuracy: 1.0000\n",
            "Epoch 278/500\n",
            "1/1 - 0s - loss: 0.1667 - binary_accuracy: 1.0000\n",
            "Epoch 279/500\n",
            "1/1 - 0s - loss: 0.1662 - binary_accuracy: 1.0000\n",
            "Epoch 280/500\n",
            "1/1 - 0s - loss: 0.1658 - binary_accuracy: 1.0000\n",
            "Epoch 281/500\n",
            "1/1 - 0s - loss: 0.1653 - binary_accuracy: 1.0000\n",
            "Epoch 282/500\n",
            "1/1 - 0s - loss: 0.1649 - binary_accuracy: 1.0000\n",
            "Epoch 283/500\n",
            "1/1 - 0s - loss: 0.1644 - binary_accuracy: 1.0000\n",
            "Epoch 284/500\n",
            "1/1 - 0s - loss: 0.1640 - binary_accuracy: 1.0000\n",
            "Epoch 285/500\n",
            "1/1 - 0s - loss: 0.1635 - binary_accuracy: 1.0000\n",
            "Epoch 286/500\n",
            "1/1 - 0s - loss: 0.1631 - binary_accuracy: 1.0000\n",
            "Epoch 287/500\n",
            "1/1 - 0s - loss: 0.1627 - binary_accuracy: 1.0000\n",
            "Epoch 288/500\n",
            "1/1 - 0s - loss: 0.1622 - binary_accuracy: 1.0000\n",
            "Epoch 289/500\n",
            "1/1 - 0s - loss: 0.1618 - binary_accuracy: 1.0000\n",
            "Epoch 290/500\n",
            "1/1 - 0s - loss: 0.1613 - binary_accuracy: 1.0000\n",
            "Epoch 291/500\n",
            "1/1 - 0s - loss: 0.1609 - binary_accuracy: 1.0000\n",
            "Epoch 292/500\n",
            "1/1 - 0s - loss: 0.1604 - binary_accuracy: 1.0000\n",
            "Epoch 293/500\n",
            "1/1 - 0s - loss: 0.1600 - binary_accuracy: 1.0000\n",
            "Epoch 294/500\n",
            "1/1 - 0s - loss: 0.1596 - binary_accuracy: 1.0000\n",
            "Epoch 295/500\n",
            "1/1 - 0s - loss: 0.1591 - binary_accuracy: 1.0000\n",
            "Epoch 296/500\n",
            "1/1 - 0s - loss: 0.1587 - binary_accuracy: 1.0000\n",
            "Epoch 297/500\n",
            "1/1 - 0s - loss: 0.1582 - binary_accuracy: 1.0000\n",
            "Epoch 298/500\n",
            "1/1 - 0s - loss: 0.1578 - binary_accuracy: 1.0000\n",
            "Epoch 299/500\n",
            "1/1 - 0s - loss: 0.1574 - binary_accuracy: 1.0000\n",
            "Epoch 300/500\n",
            "1/1 - 0s - loss: 0.1569 - binary_accuracy: 1.0000\n",
            "Epoch 301/500\n",
            "1/1 - 0s - loss: 0.1565 - binary_accuracy: 1.0000\n",
            "Epoch 302/500\n",
            "1/1 - 0s - loss: 0.1560 - binary_accuracy: 1.0000\n",
            "Epoch 303/500\n",
            "1/1 - 0s - loss: 0.1556 - binary_accuracy: 1.0000\n",
            "Epoch 304/500\n",
            "1/1 - 0s - loss: 0.1552 - binary_accuracy: 1.0000\n",
            "Epoch 305/500\n",
            "1/1 - 0s - loss: 0.1547 - binary_accuracy: 1.0000\n",
            "Epoch 306/500\n",
            "1/1 - 0s - loss: 0.1543 - binary_accuracy: 1.0000\n",
            "Epoch 307/500\n",
            "1/1 - 0s - loss: 0.1539 - binary_accuracy: 1.0000\n",
            "Epoch 308/500\n",
            "1/1 - 0s - loss: 0.1534 - binary_accuracy: 1.0000\n",
            "Epoch 309/500\n",
            "1/1 - 0s - loss: 0.1530 - binary_accuracy: 1.0000\n",
            "Epoch 310/500\n",
            "1/1 - 0s - loss: 0.1526 - binary_accuracy: 1.0000\n",
            "Epoch 311/500\n",
            "1/1 - 0s - loss: 0.1521 - binary_accuracy: 1.0000\n",
            "Epoch 312/500\n",
            "1/1 - 0s - loss: 0.1517 - binary_accuracy: 1.0000\n",
            "Epoch 313/500\n",
            "1/1 - 0s - loss: 0.1512 - binary_accuracy: 1.0000\n",
            "Epoch 314/500\n",
            "1/1 - 0s - loss: 0.1508 - binary_accuracy: 1.0000\n",
            "Epoch 315/500\n",
            "1/1 - 0s - loss: 0.1504 - binary_accuracy: 1.0000\n",
            "Epoch 316/500\n",
            "1/1 - 0s - loss: 0.1500 - binary_accuracy: 1.0000\n",
            "Epoch 317/500\n",
            "1/1 - 0s - loss: 0.1495 - binary_accuracy: 1.0000\n",
            "Epoch 318/500\n",
            "1/1 - 0s - loss: 0.1491 - binary_accuracy: 1.0000\n",
            "Epoch 319/500\n",
            "1/1 - 0s - loss: 0.1487 - binary_accuracy: 1.0000\n",
            "Epoch 320/500\n",
            "1/1 - 0s - loss: 0.1482 - binary_accuracy: 1.0000\n",
            "Epoch 321/500\n",
            "1/1 - 0s - loss: 0.1478 - binary_accuracy: 1.0000\n",
            "Epoch 322/500\n",
            "1/1 - 0s - loss: 0.1474 - binary_accuracy: 1.0000\n",
            "Epoch 323/500\n",
            "1/1 - 0s - loss: 0.1469 - binary_accuracy: 1.0000\n",
            "Epoch 324/500\n",
            "1/1 - 0s - loss: 0.1465 - binary_accuracy: 1.0000\n",
            "Epoch 325/500\n",
            "1/1 - 0s - loss: 0.1461 - binary_accuracy: 1.0000\n",
            "Epoch 326/500\n",
            "1/1 - 0s - loss: 0.1457 - binary_accuracy: 1.0000\n",
            "Epoch 327/500\n",
            "1/1 - 0s - loss: 0.1452 - binary_accuracy: 1.0000\n",
            "Epoch 328/500\n",
            "1/1 - 0s - loss: 0.1448 - binary_accuracy: 1.0000\n",
            "Epoch 329/500\n",
            "1/1 - 0s - loss: 0.1444 - binary_accuracy: 1.0000\n",
            "Epoch 330/500\n",
            "1/1 - 0s - loss: 0.1440 - binary_accuracy: 1.0000\n",
            "Epoch 331/500\n",
            "1/1 - 0s - loss: 0.1435 - binary_accuracy: 1.0000\n",
            "Epoch 332/500\n",
            "1/1 - 0s - loss: 0.1431 - binary_accuracy: 1.0000\n",
            "Epoch 333/500\n",
            "1/1 - 0s - loss: 0.1427 - binary_accuracy: 1.0000\n",
            "Epoch 334/500\n",
            "1/1 - 0s - loss: 0.1423 - binary_accuracy: 1.0000\n",
            "Epoch 335/500\n",
            "1/1 - 0s - loss: 0.1418 - binary_accuracy: 1.0000\n",
            "Epoch 336/500\n",
            "1/1 - 0s - loss: 0.1414 - binary_accuracy: 1.0000\n",
            "Epoch 337/500\n",
            "1/1 - 0s - loss: 0.1410 - binary_accuracy: 1.0000\n",
            "Epoch 338/500\n",
            "1/1 - 0s - loss: 0.1406 - binary_accuracy: 1.0000\n",
            "Epoch 339/500\n",
            "1/1 - 0s - loss: 0.1402 - binary_accuracy: 1.0000\n",
            "Epoch 340/500\n",
            "1/1 - 0s - loss: 0.1398 - binary_accuracy: 1.0000\n",
            "Epoch 341/500\n",
            "1/1 - 0s - loss: 0.1393 - binary_accuracy: 1.0000\n",
            "Epoch 342/500\n",
            "1/1 - 0s - loss: 0.1389 - binary_accuracy: 1.0000\n",
            "Epoch 343/500\n",
            "1/1 - 0s - loss: 0.1385 - binary_accuracy: 1.0000\n",
            "Epoch 344/500\n",
            "1/1 - 0s - loss: 0.1381 - binary_accuracy: 1.0000\n",
            "Epoch 345/500\n",
            "1/1 - 0s - loss: 0.1377 - binary_accuracy: 1.0000\n",
            "Epoch 346/500\n",
            "1/1 - 0s - loss: 0.1372 - binary_accuracy: 1.0000\n",
            "Epoch 347/500\n",
            "1/1 - 0s - loss: 0.1368 - binary_accuracy: 1.0000\n",
            "Epoch 348/500\n",
            "1/1 - 0s - loss: 0.1364 - binary_accuracy: 1.0000\n",
            "Epoch 349/500\n",
            "1/1 - 0s - loss: 0.1360 - binary_accuracy: 1.0000\n",
            "Epoch 350/500\n",
            "1/1 - 0s - loss: 0.1356 - binary_accuracy: 1.0000\n",
            "Epoch 351/500\n",
            "1/1 - 0s - loss: 0.1352 - binary_accuracy: 1.0000\n",
            "Epoch 352/500\n",
            "1/1 - 0s - loss: 0.1348 - binary_accuracy: 1.0000\n",
            "Epoch 353/500\n",
            "1/1 - 0s - loss: 0.1344 - binary_accuracy: 1.0000\n",
            "Epoch 354/500\n",
            "1/1 - 0s - loss: 0.1340 - binary_accuracy: 1.0000\n",
            "Epoch 355/500\n",
            "1/1 - 0s - loss: 0.1335 - binary_accuracy: 1.0000\n",
            "Epoch 356/500\n",
            "1/1 - 0s - loss: 0.1331 - binary_accuracy: 1.0000\n",
            "Epoch 357/500\n",
            "1/1 - 0s - loss: 0.1327 - binary_accuracy: 1.0000\n",
            "Epoch 358/500\n",
            "1/1 - 0s - loss: 0.1323 - binary_accuracy: 1.0000\n",
            "Epoch 359/500\n",
            "1/1 - 0s - loss: 0.1319 - binary_accuracy: 1.0000\n",
            "Epoch 360/500\n",
            "1/1 - 0s - loss: 0.1315 - binary_accuracy: 1.0000\n",
            "Epoch 361/500\n",
            "1/1 - 0s - loss: 0.1311 - binary_accuracy: 1.0000\n",
            "Epoch 362/500\n",
            "1/1 - 0s - loss: 0.1307 - binary_accuracy: 1.0000\n",
            "Epoch 363/500\n",
            "1/1 - 0s - loss: 0.1303 - binary_accuracy: 1.0000\n",
            "Epoch 364/500\n",
            "1/1 - 0s - loss: 0.1299 - binary_accuracy: 1.0000\n",
            "Epoch 365/500\n",
            "1/1 - 0s - loss: 0.1295 - binary_accuracy: 1.0000\n",
            "Epoch 366/500\n",
            "1/1 - 0s - loss: 0.1291 - binary_accuracy: 1.0000\n",
            "Epoch 367/500\n",
            "1/1 - 0s - loss: 0.1287 - binary_accuracy: 1.0000\n",
            "Epoch 368/500\n",
            "1/1 - 0s - loss: 0.1283 - binary_accuracy: 1.0000\n",
            "Epoch 369/500\n",
            "1/1 - 0s - loss: 0.1279 - binary_accuracy: 1.0000\n",
            "Epoch 370/500\n",
            "1/1 - 0s - loss: 0.1275 - binary_accuracy: 1.0000\n",
            "Epoch 371/500\n",
            "1/1 - 0s - loss: 0.1271 - binary_accuracy: 1.0000\n",
            "Epoch 372/500\n",
            "1/1 - 0s - loss: 0.1267 - binary_accuracy: 1.0000\n",
            "Epoch 373/500\n",
            "1/1 - 0s - loss: 0.1263 - binary_accuracy: 1.0000\n",
            "Epoch 374/500\n",
            "1/1 - 0s - loss: 0.1259 - binary_accuracy: 1.0000\n",
            "Epoch 375/500\n",
            "1/1 - 0s - loss: 0.1255 - binary_accuracy: 1.0000\n",
            "Epoch 376/500\n",
            "1/1 - 0s - loss: 0.1251 - binary_accuracy: 1.0000\n",
            "Epoch 377/500\n",
            "1/1 - 0s - loss: 0.1247 - binary_accuracy: 1.0000\n",
            "Epoch 378/500\n",
            "1/1 - 0s - loss: 0.1244 - binary_accuracy: 1.0000\n",
            "Epoch 379/500\n",
            "1/1 - 0s - loss: 0.1240 - binary_accuracy: 1.0000\n",
            "Epoch 380/500\n",
            "1/1 - 0s - loss: 0.1236 - binary_accuracy: 1.0000\n",
            "Epoch 381/500\n",
            "1/1 - 0s - loss: 0.1232 - binary_accuracy: 1.0000\n",
            "Epoch 382/500\n",
            "1/1 - 0s - loss: 0.1228 - binary_accuracy: 1.0000\n",
            "Epoch 383/500\n",
            "1/1 - 0s - loss: 0.1224 - binary_accuracy: 1.0000\n",
            "Epoch 384/500\n",
            "1/1 - 0s - loss: 0.1220 - binary_accuracy: 1.0000\n",
            "Epoch 385/500\n",
            "1/1 - 0s - loss: 0.1216 - binary_accuracy: 1.0000\n",
            "Epoch 386/500\n",
            "1/1 - 0s - loss: 0.1213 - binary_accuracy: 1.0000\n",
            "Epoch 387/500\n",
            "1/1 - 0s - loss: 0.1209 - binary_accuracy: 1.0000\n",
            "Epoch 388/500\n",
            "1/1 - 0s - loss: 0.1205 - binary_accuracy: 1.0000\n",
            "Epoch 389/500\n",
            "1/1 - 0s - loss: 0.1201 - binary_accuracy: 1.0000\n",
            "Epoch 390/500\n",
            "1/1 - 0s - loss: 0.1197 - binary_accuracy: 1.0000\n",
            "Epoch 391/500\n",
            "1/1 - 0s - loss: 0.1193 - binary_accuracy: 1.0000\n",
            "Epoch 392/500\n",
            "1/1 - 0s - loss: 0.1190 - binary_accuracy: 1.0000\n",
            "Epoch 393/500\n",
            "1/1 - 0s - loss: 0.1186 - binary_accuracy: 1.0000\n",
            "Epoch 394/500\n",
            "1/1 - 0s - loss: 0.1182 - binary_accuracy: 1.0000\n",
            "Epoch 395/500\n",
            "1/1 - 0s - loss: 0.1178 - binary_accuracy: 1.0000\n",
            "Epoch 396/500\n",
            "1/1 - 0s - loss: 0.1175 - binary_accuracy: 1.0000\n",
            "Epoch 397/500\n",
            "1/1 - 0s - loss: 0.1171 - binary_accuracy: 1.0000\n",
            "Epoch 398/500\n",
            "1/1 - 0s - loss: 0.1167 - binary_accuracy: 1.0000\n",
            "Epoch 399/500\n",
            "1/1 - 0s - loss: 0.1163 - binary_accuracy: 1.0000\n",
            "Epoch 400/500\n",
            "1/1 - 0s - loss: 0.1160 - binary_accuracy: 1.0000\n",
            "Epoch 401/500\n",
            "1/1 - 0s - loss: 0.1156 - binary_accuracy: 1.0000\n",
            "Epoch 402/500\n",
            "1/1 - 0s - loss: 0.1152 - binary_accuracy: 1.0000\n",
            "Epoch 403/500\n",
            "1/1 - 0s - loss: 0.1149 - binary_accuracy: 1.0000\n",
            "Epoch 404/500\n",
            "1/1 - 0s - loss: 0.1145 - binary_accuracy: 1.0000\n",
            "Epoch 405/500\n",
            "1/1 - 0s - loss: 0.1141 - binary_accuracy: 1.0000\n",
            "Epoch 406/500\n",
            "1/1 - 0s - loss: 0.1138 - binary_accuracy: 1.0000\n",
            "Epoch 407/500\n",
            "1/1 - 0s - loss: 0.1134 - binary_accuracy: 1.0000\n",
            "Epoch 408/500\n",
            "1/1 - 0s - loss: 0.1130 - binary_accuracy: 1.0000\n",
            "Epoch 409/500\n",
            "1/1 - 0s - loss: 0.1127 - binary_accuracy: 1.0000\n",
            "Epoch 410/500\n",
            "1/1 - 0s - loss: 0.1123 - binary_accuracy: 1.0000\n",
            "Epoch 411/500\n",
            "1/1 - 0s - loss: 0.1120 - binary_accuracy: 1.0000\n",
            "Epoch 412/500\n",
            "1/1 - 0s - loss: 0.1116 - binary_accuracy: 1.0000\n",
            "Epoch 413/500\n",
            "1/1 - 0s - loss: 0.1112 - binary_accuracy: 1.0000\n",
            "Epoch 414/500\n",
            "1/1 - 0s - loss: 0.1109 - binary_accuracy: 1.0000\n",
            "Epoch 415/500\n",
            "1/1 - 0s - loss: 0.1105 - binary_accuracy: 1.0000\n",
            "Epoch 416/500\n",
            "1/1 - 0s - loss: 0.1102 - binary_accuracy: 1.0000\n",
            "Epoch 417/500\n",
            "1/1 - 0s - loss: 0.1098 - binary_accuracy: 1.0000\n",
            "Epoch 418/500\n",
            "1/1 - 0s - loss: 0.1094 - binary_accuracy: 1.0000\n",
            "Epoch 419/500\n",
            "1/1 - 0s - loss: 0.1091 - binary_accuracy: 1.0000\n",
            "Epoch 420/500\n",
            "1/1 - 0s - loss: 0.1088 - binary_accuracy: 1.0000\n",
            "Epoch 421/500\n",
            "1/1 - 0s - loss: 0.1084 - binary_accuracy: 1.0000\n",
            "Epoch 422/500\n",
            "1/1 - 0s - loss: 0.1080 - binary_accuracy: 1.0000\n",
            "Epoch 423/500\n",
            "1/1 - 0s - loss: 0.1077 - binary_accuracy: 1.0000\n",
            "Epoch 424/500\n",
            "1/1 - 0s - loss: 0.1073 - binary_accuracy: 1.0000\n",
            "Epoch 425/500\n",
            "1/1 - 0s - loss: 0.1070 - binary_accuracy: 1.0000\n",
            "Epoch 426/500\n",
            "1/1 - 0s - loss: 0.1066 - binary_accuracy: 1.0000\n",
            "Epoch 427/500\n",
            "1/1 - 0s - loss: 0.1063 - binary_accuracy: 1.0000\n",
            "Epoch 428/500\n",
            "1/1 - 0s - loss: 0.1060 - binary_accuracy: 1.0000\n",
            "Epoch 429/500\n",
            "1/1 - 0s - loss: 0.1056 - binary_accuracy: 1.0000\n",
            "Epoch 430/500\n",
            "1/1 - 0s - loss: 0.1053 - binary_accuracy: 1.0000\n",
            "Epoch 431/500\n",
            "1/1 - 0s - loss: 0.1049 - binary_accuracy: 1.0000\n",
            "Epoch 432/500\n",
            "1/1 - 0s - loss: 0.1046 - binary_accuracy: 1.0000\n",
            "Epoch 433/500\n",
            "1/1 - 0s - loss: 0.1042 - binary_accuracy: 1.0000\n",
            "Epoch 434/500\n",
            "1/1 - 0s - loss: 0.1039 - binary_accuracy: 1.0000\n",
            "Epoch 435/500\n",
            "1/1 - 0s - loss: 0.1036 - binary_accuracy: 1.0000\n",
            "Epoch 436/500\n",
            "1/1 - 0s - loss: 0.1032 - binary_accuracy: 1.0000\n",
            "Epoch 437/500\n",
            "1/1 - 0s - loss: 0.1029 - binary_accuracy: 1.0000\n",
            "Epoch 438/500\n",
            "1/1 - 0s - loss: 0.1026 - binary_accuracy: 1.0000\n",
            "Epoch 439/500\n",
            "1/1 - 0s - loss: 0.1022 - binary_accuracy: 1.0000\n",
            "Epoch 440/500\n",
            "1/1 - 0s - loss: 0.1019 - binary_accuracy: 1.0000\n",
            "Epoch 441/500\n",
            "1/1 - 0s - loss: 0.1016 - binary_accuracy: 1.0000\n",
            "Epoch 442/500\n",
            "1/1 - 0s - loss: 0.1012 - binary_accuracy: 1.0000\n",
            "Epoch 443/500\n",
            "1/1 - 0s - loss: 0.1009 - binary_accuracy: 1.0000\n",
            "Epoch 444/500\n",
            "1/1 - 0s - loss: 0.1006 - binary_accuracy: 1.0000\n",
            "Epoch 445/500\n",
            "1/1 - 0s - loss: 0.1003 - binary_accuracy: 1.0000\n",
            "Epoch 446/500\n",
            "1/1 - 0s - loss: 0.0999 - binary_accuracy: 1.0000\n",
            "Epoch 447/500\n",
            "1/1 - 0s - loss: 0.0996 - binary_accuracy: 1.0000\n",
            "Epoch 448/500\n",
            "1/1 - 0s - loss: 0.0993 - binary_accuracy: 1.0000\n",
            "Epoch 449/500\n",
            "1/1 - 0s - loss: 0.0990 - binary_accuracy: 1.0000\n",
            "Epoch 450/500\n",
            "1/1 - 0s - loss: 0.0986 - binary_accuracy: 1.0000\n",
            "Epoch 451/500\n",
            "1/1 - 0s - loss: 0.0983 - binary_accuracy: 1.0000\n",
            "Epoch 452/500\n",
            "1/1 - 0s - loss: 0.0980 - binary_accuracy: 1.0000\n",
            "Epoch 453/500\n",
            "1/1 - 0s - loss: 0.0977 - binary_accuracy: 1.0000\n",
            "Epoch 454/500\n",
            "1/1 - 0s - loss: 0.0974 - binary_accuracy: 1.0000\n",
            "Epoch 455/500\n",
            "1/1 - 0s - loss: 0.0970 - binary_accuracy: 1.0000\n",
            "Epoch 456/500\n",
            "1/1 - 0s - loss: 0.0967 - binary_accuracy: 1.0000\n",
            "Epoch 457/500\n",
            "1/1 - 0s - loss: 0.0964 - binary_accuracy: 1.0000\n",
            "Epoch 458/500\n",
            "1/1 - 0s - loss: 0.0961 - binary_accuracy: 1.0000\n",
            "Epoch 459/500\n",
            "1/1 - 0s - loss: 0.0958 - binary_accuracy: 1.0000\n",
            "Epoch 460/500\n",
            "1/1 - 0s - loss: 0.0955 - binary_accuracy: 1.0000\n",
            "Epoch 461/500\n",
            "1/1 - 0s - loss: 0.0952 - binary_accuracy: 1.0000\n",
            "Epoch 462/500\n",
            "1/1 - 0s - loss: 0.0948 - binary_accuracy: 1.0000\n",
            "Epoch 463/500\n",
            "1/1 - 0s - loss: 0.0945 - binary_accuracy: 1.0000\n",
            "Epoch 464/500\n",
            "1/1 - 0s - loss: 0.0942 - binary_accuracy: 1.0000\n",
            "Epoch 465/500\n",
            "1/1 - 0s - loss: 0.0939 - binary_accuracy: 1.0000\n",
            "Epoch 466/500\n",
            "1/1 - 0s - loss: 0.0936 - binary_accuracy: 1.0000\n",
            "Epoch 467/500\n",
            "1/1 - 0s - loss: 0.0933 - binary_accuracy: 1.0000\n",
            "Epoch 468/500\n",
            "1/1 - 0s - loss: 0.0930 - binary_accuracy: 1.0000\n",
            "Epoch 469/500\n",
            "1/1 - 0s - loss: 0.0927 - binary_accuracy: 1.0000\n",
            "Epoch 470/500\n",
            "1/1 - 0s - loss: 0.0924 - binary_accuracy: 1.0000\n",
            "Epoch 471/500\n",
            "1/1 - 0s - loss: 0.0921 - binary_accuracy: 1.0000\n",
            "Epoch 472/500\n",
            "1/1 - 0s - loss: 0.0918 - binary_accuracy: 1.0000\n",
            "Epoch 473/500\n",
            "1/1 - 0s - loss: 0.0915 - binary_accuracy: 1.0000\n",
            "Epoch 474/500\n",
            "1/1 - 0s - loss: 0.0912 - binary_accuracy: 1.0000\n",
            "Epoch 475/500\n",
            "1/1 - 0s - loss: 0.0909 - binary_accuracy: 1.0000\n",
            "Epoch 476/500\n",
            "1/1 - 0s - loss: 0.0906 - binary_accuracy: 1.0000\n",
            "Epoch 477/500\n",
            "1/1 - 0s - loss: 0.0903 - binary_accuracy: 1.0000\n",
            "Epoch 478/500\n",
            "1/1 - 0s - loss: 0.0900 - binary_accuracy: 1.0000\n",
            "Epoch 479/500\n",
            "1/1 - 0s - loss: 0.0897 - binary_accuracy: 1.0000\n",
            "Epoch 480/500\n",
            "1/1 - 0s - loss: 0.0894 - binary_accuracy: 1.0000\n",
            "Epoch 481/500\n",
            "1/1 - 0s - loss: 0.0891 - binary_accuracy: 1.0000\n",
            "Epoch 482/500\n",
            "1/1 - 0s - loss: 0.0888 - binary_accuracy: 1.0000\n",
            "Epoch 483/500\n",
            "1/1 - 0s - loss: 0.0885 - binary_accuracy: 1.0000\n",
            "Epoch 484/500\n",
            "1/1 - 0s - loss: 0.0882 - binary_accuracy: 1.0000\n",
            "Epoch 485/500\n",
            "1/1 - 0s - loss: 0.0880 - binary_accuracy: 1.0000\n",
            "Epoch 486/500\n",
            "1/1 - 0s - loss: 0.0877 - binary_accuracy: 1.0000\n",
            "Epoch 487/500\n",
            "1/1 - 0s - loss: 0.0874 - binary_accuracy: 1.0000\n",
            "Epoch 488/500\n",
            "1/1 - 0s - loss: 0.0871 - binary_accuracy: 1.0000\n",
            "Epoch 489/500\n",
            "1/1 - 0s - loss: 0.0868 - binary_accuracy: 1.0000\n",
            "Epoch 490/500\n",
            "1/1 - 0s - loss: 0.0865 - binary_accuracy: 1.0000\n",
            "Epoch 491/500\n",
            "1/1 - 0s - loss: 0.0863 - binary_accuracy: 1.0000\n",
            "Epoch 492/500\n",
            "1/1 - 0s - loss: 0.0860 - binary_accuracy: 1.0000\n",
            "Epoch 493/500\n",
            "1/1 - 0s - loss: 0.0857 - binary_accuracy: 1.0000\n",
            "Epoch 494/500\n",
            "1/1 - 0s - loss: 0.0854 - binary_accuracy: 1.0000\n",
            "Epoch 495/500\n",
            "1/1 - 0s - loss: 0.0851 - binary_accuracy: 1.0000\n",
            "Epoch 496/500\n",
            "1/1 - 0s - loss: 0.0849 - binary_accuracy: 1.0000\n",
            "Epoch 497/500\n",
            "1/1 - 0s - loss: 0.0846 - binary_accuracy: 1.0000\n",
            "Epoch 498/500\n",
            "1/1 - 0s - loss: 0.0843 - binary_accuracy: 1.0000\n",
            "Epoch 499/500\n",
            "1/1 - 0s - loss: 0.0840 - binary_accuracy: 1.0000\n",
            "Epoch 500/500\n",
            "1/1 - 0s - loss: 0.0838 - binary_accuracy: 1.0000\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tensorflow.python.keras.callbacks.History at 0x7f2c1ec966d8>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 64
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bflJ7aN6qpT3",
        "outputId": "1ec7a6b1-35ad-480b-907c-c3abc605b1fa",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 86
        }
      },
      "source": [
        "print(model.predict(X))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[0.3763654 ]\n",
            " [0.7490106 ]\n",
            " [0.74157757]\n",
            " [0.25008917]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mZA48q48gZpZ",
        "outputId": "1a135126-0a31-4e9b-b0df-6db91c49977e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 86
        }
      },
      "source": [
        "print(model.predict(X).round())"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[0.]\n",
            " [1.]\n",
            " [1.]\n",
            " [0.]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1xEiL-1LdK5-"
      },
      "source": [
        "# Example 2: ANN to learn to XOR function\n",
        "\n",
        "This second exxample solves the same XOR problem, but with a somewhat different architecture. \n",
        "\n",
        "Note that using Dense(activation='relu') is equivalent to first add a Dense layer and then add Activation('relu')."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JrVo8gwwcFo7"
      },
      "source": [
        "from keras.models import Sequential\n",
        "from keras.layers.core import Dense, Dropout, Activation\n",
        "from keras.optimizers import SGD\n",
        "import numpy as np\n",
        "\n",
        "model = Sequential()\n",
        "\n",
        "model.add(Dense(8, input_dim=2))\n",
        "model.add(Activation('relu'))\n",
        "\n",
        "model.add(Dense(1))\n",
        "model.add(Activation('sigmoid'))\n",
        "\n",
        "sgd = SGD(lr=0.1)\n",
        "model.compile(loss='binary_crossentropy', optimizer=sgd)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "D8fhAmSlhJea",
        "outputId": "0906487d-ced1-4563-f17c-acfdf1ce6512",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 311
        }
      },
      "source": [
        "print(model.summary())"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_34\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "dense_65 (Dense)             (None, 8)                 24        \n",
            "_________________________________________________________________\n",
            "activation_36 (Activation)   (None, 8)                 0         \n",
            "_________________________________________________________________\n",
            "dense_66 (Dense)             (None, 1)                 9         \n",
            "_________________________________________________________________\n",
            "activation_37 (Activation)   (None, 1)                 0         \n",
            "=================================================================\n",
            "Total params: 33\n",
            "Trainable params: 33\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n",
            "None\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wheNml6liX-i"
      },
      "source": [
        "from IPython.display import SVG, display\n",
        "def show_svg():\n",
        "    display(SVG(url='http://upload.wikimedia.org/wikipedia/en/a/a4/Flag_of_the_United_States.svg'))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_S8v14T_i9p_",
        "outputId": "280e2ea0-cc90-4e29-bd22-48badb377026",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 842
        }
      },
      "source": [
        "%%html\n",
        "<svg xmlns=\"http://www.w3.org/2000/svg\" width=\"1400\" height=\"800\" style=\"cursor: move;\"><g transform=\"translate(31.029124448207995,22.95774483714024) scale(0.9619274546647754)\"><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,250\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,290\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,330\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,370\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,410\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,450\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,250\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,290\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,330\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,370\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,410\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,450\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,250, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,290, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,330, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,370, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,410, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,450, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,490\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,530\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,490\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,530\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,490, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,530, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><circle r=\"10\" class=\"node\" id=\"0_0\" cx=\"540\" cy=\"370\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"0_1\" cx=\"540\" cy=\"410\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_0\" cx=\"720\" cy=\"250\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_1\" cx=\"720\" cy=\"290\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_2\" cx=\"720\" cy=\"330\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_3\" cx=\"720\" cy=\"370\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_4\" cx=\"720\" cy=\"410\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_5\" cx=\"720\" cy=\"450\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_6\" cx=\"720\" cy=\"490\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_7\" cx=\"720\" cy=\"530\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"2_0\" cx=\"900\" cy=\"390\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><text class=\"text\" dy=\".35em\" x=\"505\" y=\"570\" style=\"font-size: 12px;\">Input Layer ∈ ℝ²</text><text class=\"text\" dy=\".35em\" x=\"685\" y=\"570\" style=\"font-size: 12px;\">Hidden Layer ∈ ℝ⁸</text><text class=\"text\" dy=\".35em\" x=\"865\" y=\"570\" style=\"font-size: 12px;\">Output Layer ∈ ℝ¹</text></g><defs><marker id=\"arrow\" viewBox=\"0 -5 10 10\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto\" refX=\"40\"><path d=\"M0,-5L10,0L0,5\" style=\"stroke: rgb(80, 80, 80); fill: none;\"></path></marker></defs></svg>"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<svg xmlns=\"http://www.w3.org/2000/svg\" width=\"1400\" height=\"800\" style=\"cursor: move;\"><g transform=\"translate(31.029124448207995,22.95774483714024) scale(0.9619274546647754)\"><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,250\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,290\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,330\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,370\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,410\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,450\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,250\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,290\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,330\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,370\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,410\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,450\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,250, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,290, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,330, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,370, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,410, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,450, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,490\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,370, 720,530\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,490\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M540,410, 720,530\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,490, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><path class=\"link\" marker-end=\"\" d=\"M720,530, 900,390\" style=\"stroke-width: 0.5; stroke-opacity: 1; stroke: rgb(80, 80, 80);\"></path><circle r=\"10\" class=\"node\" id=\"0_0\" cx=\"540\" cy=\"370\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"0_1\" cx=\"540\" cy=\"410\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_0\" cx=\"720\" cy=\"250\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_1\" cx=\"720\" cy=\"290\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_2\" cx=\"720\" cy=\"330\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_3\" cx=\"720\" cy=\"370\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_4\" cx=\"720\" cy=\"410\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_5\" cx=\"720\" cy=\"450\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_6\" cx=\"720\" cy=\"490\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"1_7\" cx=\"720\" cy=\"530\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><circle r=\"10\" class=\"node\" id=\"2_0\" cx=\"900\" cy=\"390\" style=\"fill: rgb(255, 255, 255); stroke: rgb(51, 51, 51);\"></circle><text class=\"text\" dy=\".35em\" x=\"505\" y=\"570\" style=\"font-size: 12px;\">Input Layer ∈ ℝ²</text><text class=\"text\" dy=\".35em\" x=\"685\" y=\"570\" style=\"font-size: 12px;\">Hidden Layer ∈ ℝ⁸</text><text class=\"text\" dy=\".35em\" x=\"865\" y=\"570\" style=\"font-size: 12px;\">Output Layer ∈ ℝ¹</text></g><defs><marker id=\"arrow\" viewBox=\"0 -5 10 10\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto\" refX=\"40\"><path d=\"M0,-5L10,0L0,5\" style=\"stroke: rgb(80, 80, 80); fill: none;\"></path></marker></defs></svg>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5Zy9UZVMhL6Z",
        "outputId": "70af05d3-c2db-4246-d5ef-afdd8a662d76",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "X = np.array([[0,0],[0,1],[1,0],[1,1]])\n",
        "y = np.array([[0],[1],[1],[0]])\n",
        "\n",
        "model.fit(X, y, batch_size=1, epochs=100)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.7455\n",
            "Epoch 2/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.7110\n",
            "Epoch 3/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.6909\n",
            "Epoch 4/100\n",
            "4/4 [==============================] - 0s 1ms/step - loss: 0.6757\n",
            "Epoch 5/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.6631\n",
            "Epoch 6/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.6530\n",
            "Epoch 7/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.6446\n",
            "Epoch 8/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.6356\n",
            "Epoch 9/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.6263\n",
            "Epoch 10/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.6213\n",
            "Epoch 11/100\n",
            "4/4 [==============================] - 0s 1ms/step - loss: 0.6112\n",
            "Epoch 12/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.6065\n",
            "Epoch 13/100\n",
            "4/4 [==============================] - 0s 1ms/step - loss: 0.5978\n",
            "Epoch 14/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.6030\n",
            "Epoch 15/100\n",
            "4/4 [==============================] - 0s 1ms/step - loss: 0.5955\n",
            "Epoch 16/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.5823\n",
            "Epoch 17/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.5796\n",
            "Epoch 18/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.5731\n",
            "Epoch 19/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.5619\n",
            "Epoch 20/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.5606\n",
            "Epoch 21/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.5567\n",
            "Epoch 22/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.5490\n",
            "Epoch 23/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.5413\n",
            "Epoch 24/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.5327\n",
            "Epoch 25/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.5314\n",
            "Epoch 26/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.5246\n",
            "Epoch 27/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.5221\n",
            "Epoch 28/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.5188\n",
            "Epoch 29/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.5100\n",
            "Epoch 30/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.5068\n",
            "Epoch 31/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4943\n",
            "Epoch 32/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.5038\n",
            "Epoch 33/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.4931\n",
            "Epoch 34/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4842\n",
            "Epoch 35/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4910\n",
            "Epoch 36/100\n",
            "4/4 [==============================] - 0s 4ms/step - loss: 0.4784\n",
            "Epoch 37/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.4733\n",
            "Epoch 38/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.4755\n",
            "Epoch 39/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.4683\n",
            "Epoch 40/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4656\n",
            "Epoch 41/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.4571\n",
            "Epoch 42/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4634\n",
            "Epoch 43/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4533\n",
            "Epoch 44/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4510\n",
            "Epoch 45/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4499\n",
            "Epoch 46/100\n",
            "4/4 [==============================] - 0s 1ms/step - loss: 0.4481\n",
            "Epoch 47/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4417\n",
            "Epoch 48/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4406\n",
            "Epoch 49/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4379\n",
            "Epoch 50/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4320\n",
            "Epoch 51/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4338\n",
            "Epoch 52/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4301\n",
            "Epoch 53/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4316\n",
            "Epoch 54/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4261\n",
            "Epoch 55/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4216\n",
            "Epoch 56/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4231\n",
            "Epoch 57/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.4166\n",
            "Epoch 58/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4244\n",
            "Epoch 59/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4161\n",
            "Epoch 60/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4122\n",
            "Epoch 61/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4126\n",
            "Epoch 62/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4084\n",
            "Epoch 63/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4156\n",
            "Epoch 64/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4110\n",
            "Epoch 65/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4073\n",
            "Epoch 66/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4054\n",
            "Epoch 67/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4038\n",
            "Epoch 68/100\n",
            "4/4 [==============================] - 0s 4ms/step - loss: 0.4014\n",
            "Epoch 69/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4000\n",
            "Epoch 70/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4051\n",
            "Epoch 71/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.4002\n",
            "Epoch 72/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.3981\n",
            "Epoch 73/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.3956\n",
            "Epoch 74/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.3952\n",
            "Epoch 75/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.3956\n",
            "Epoch 76/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.3948\n",
            "Epoch 77/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.3957\n",
            "Epoch 78/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.3946\n",
            "Epoch 79/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.3922\n",
            "Epoch 80/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.3905\n",
            "Epoch 81/100\n",
            "4/4 [==============================] - 0s 1ms/step - loss: 0.3895\n",
            "Epoch 82/100\n",
            "4/4 [==============================] - 0s 1ms/step - loss: 0.3880\n",
            "Epoch 83/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.3868\n",
            "Epoch 84/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.3868\n",
            "Epoch 85/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.3889\n",
            "Epoch 86/100\n",
            "4/4 [==============================] - 0s 1ms/step - loss: 0.3879\n",
            "Epoch 87/100\n",
            "4/4 [==============================] - 0s 1ms/step - loss: 0.3861\n",
            "Epoch 88/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.3845\n",
            "Epoch 89/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.3839\n",
            "Epoch 90/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.3824\n",
            "Epoch 91/100\n",
            "4/4 [==============================] - 0s 1ms/step - loss: 0.3820\n",
            "Epoch 92/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.3810\n",
            "Epoch 93/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.3805\n",
            "Epoch 94/100\n",
            "4/4 [==============================] - 0s 1ms/step - loss: 0.3800\n",
            "Epoch 95/100\n",
            "4/4 [==============================] - 0s 6ms/step - loss: 0.3855\n",
            "Epoch 96/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.3828\n",
            "Epoch 97/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.3814\n",
            "Epoch 98/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.3801\n",
            "Epoch 99/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.3789\n",
            "Epoch 100/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.3778\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tensorflow.python.keras.callbacks.History at 0x7f2c0e44eac8>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 80
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MamdOxY0orsd",
        "outputId": "47872736-1f56-4825-9c26-966c11021dca",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 86
        }
      },
      "source": [
        "print(model.predict(X))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[0.48097742]\n",
            " [0.48097742]\n",
            " [0.9649625 ]\n",
            " [0.05655888]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "832BAOFphUjL",
        "outputId": "c8474809-cad7-40a6-d172-84df2d4bf882",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 86
        }
      },
      "source": [
        "print(model.predict_proba(X))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[0.48097742]\n",
            " [0.48097742]\n",
            " [0.9649625 ]\n",
            " [0.05655888]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GeGFSaXKptNR"
      },
      "source": [
        "# Example: simpler model for the XOR problem"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "O63XkPaepxiY"
      },
      "source": [
        " from keras.models import Sequential\n",
        " from keras.layers.core import Dense, Dropout, Activation\n",
        " from keras.optimizers import SGD\n",
        " import numpy as np\n",
        " \n",
        " X = np.array([[0,0],[0,1],[1,0],[1,1]])\n",
        " y = np.array([[0],[1],[1],[0]])\n",
        " \n",
        " model = Sequential()\n",
        " model.add(Dense(2, input_dim=2))\n",
        " model.add(Activation('tanh'))\n",
        " model.add(Dense(1))\n",
        " model.add(Activation('sigmoid'))\n",
        " \n",
        " sgd = SGD(lr=0.1)\n",
        " model.compile(loss='mean_squared_error', optimizer=sgd)\n",
        " \n",
        " model.fit(X, y, batch_size=1, epochs=1000)\n",
        " print(model.predict_proba(X))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uSx9hQBareKA"
      },
      "source": [
        "# Ploting learning curves"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "W3oF8V38rkiL",
        "outputId": "1b93d8f8-912c-47b1-c200-651f9034efc9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        " from keras.models import Sequential\n",
        " from keras.layers.core import Dense, Dropout, Activation\n",
        " from keras.optimizers import SGD\n",
        " import numpy as np\n",
        " \n",
        " X = np.array([[0,0],[0,1],[1,0],[1,1]])\n",
        " y = np.array([[0],[1],[1],[0]])\n",
        " \n",
        " model = Sequential()\n",
        " model.add(Dense(2, input_dim=2))\n",
        " model.add(Activation('tanh'))\n",
        " model.add(Dense(1))\n",
        " model.add(Activation('sigmoid'))\n",
        " \n",
        " sgd = SGD(lr=0.1)\n",
        " model.compile(loss='mean_squared_error', optimizer=sgd, metrics=['acc'])\n",
        " \n",
        " history = model.fit(X, y, batch_size=1, epochs=100)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2553 - acc: 0.2500\n",
            "Epoch 2/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2550 - acc: 0.5000\n",
            "Epoch 3/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2545 - acc: 0.5000\n",
            "Epoch 4/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.2542 - acc: 0.5000\n",
            "Epoch 5/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2539 - acc: 0.5000\n",
            "Epoch 6/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2536 - acc: 0.7500\n",
            "Epoch 7/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2533 - acc: 0.7500\n",
            "Epoch 8/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2529 - acc: 0.5000\n",
            "Epoch 9/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2526 - acc: 0.5000\n",
            "Epoch 10/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2523 - acc: 0.5000\n",
            "Epoch 11/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2520 - acc: 0.5000\n",
            "Epoch 12/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2515 - acc: 0.7500\n",
            "Epoch 13/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2512 - acc: 0.7500\n",
            "Epoch 14/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.2509 - acc: 0.5000\n",
            "Epoch 15/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2507 - acc: 0.5000\n",
            "Epoch 16/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.2504 - acc: 0.5000\n",
            "Epoch 17/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2501 - acc: 0.7500\n",
            "Epoch 18/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2497 - acc: 0.7500\n",
            "Epoch 19/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2494 - acc: 0.5000\n",
            "Epoch 20/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2491 - acc: 0.7500\n",
            "Epoch 21/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2488 - acc: 0.7500\n",
            "Epoch 22/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2485 - acc: 0.7500\n",
            "Epoch 23/100\n",
            "4/4 [==============================] - 0s 1ms/step - loss: 0.2481 - acc: 0.5000\n",
            "Epoch 24/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2478 - acc: 0.5000\n",
            "Epoch 25/100\n",
            "4/4 [==============================] - 0s 1ms/step - loss: 0.2476 - acc: 0.7500\n",
            "Epoch 26/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2471 - acc: 0.7500\n",
            "Epoch 27/100\n",
            "4/4 [==============================] - 0s 1ms/step - loss: 0.2468 - acc: 0.7500\n",
            "Epoch 28/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2466 - acc: 0.7500\n",
            "Epoch 29/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2463 - acc: 0.7500\n",
            "Epoch 30/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2460 - acc: 0.7500\n",
            "Epoch 31/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2456 - acc: 0.7500\n",
            "Epoch 32/100\n",
            "4/4 [==============================] - 0s 1ms/step - loss: 0.2453 - acc: 0.7500\n",
            "Epoch 33/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2450 - acc: 0.7500\n",
            "Epoch 34/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2446 - acc: 0.7500\n",
            "Epoch 35/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2442 - acc: 0.7500\n",
            "Epoch 36/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2440 - acc: 0.7500\n",
            "Epoch 37/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2437 - acc: 0.7500\n",
            "Epoch 38/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2432 - acc: 0.7500\n",
            "Epoch 39/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2429 - acc: 0.7500\n",
            "Epoch 40/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2427 - acc: 0.7500\n",
            "Epoch 41/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2423 - acc: 0.7500\n",
            "Epoch 42/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.2420 - acc: 0.7500\n",
            "Epoch 43/100\n",
            "4/4 [==============================] - 0s 1ms/step - loss: 0.2416 - acc: 0.7500\n",
            "Epoch 44/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2413 - acc: 0.7500\n",
            "Epoch 45/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2409 - acc: 0.7500\n",
            "Epoch 46/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.2406 - acc: 0.7500\n",
            "Epoch 47/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2402 - acc: 0.7500\n",
            "Epoch 48/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2400 - acc: 0.7500\n",
            "Epoch 49/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2395 - acc: 0.7500\n",
            "Epoch 50/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.2392 - acc: 0.7500\n",
            "Epoch 51/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2388 - acc: 0.7500\n",
            "Epoch 52/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2384 - acc: 0.7500\n",
            "Epoch 53/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.2381 - acc: 0.7500\n",
            "Epoch 54/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2378 - acc: 0.7500\n",
            "Epoch 55/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2374 - acc: 0.7500\n",
            "Epoch 56/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2371 - acc: 0.7500\n",
            "Epoch 57/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2366 - acc: 0.7500\n",
            "Epoch 58/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2363 - acc: 0.7500\n",
            "Epoch 59/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2359 - acc: 0.7500\n",
            "Epoch 60/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2356 - acc: 0.7500\n",
            "Epoch 61/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2352 - acc: 0.7500\n",
            "Epoch 62/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.2348 - acc: 0.7500\n",
            "Epoch 63/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.2344 - acc: 0.7500\n",
            "Epoch 64/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2341 - acc: 0.7500\n",
            "Epoch 65/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2337 - acc: 0.7500\n",
            "Epoch 66/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.2333 - acc: 0.7500\n",
            "Epoch 67/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.2329 - acc: 0.7500\n",
            "Epoch 68/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.2325 - acc: 0.7500\n",
            "Epoch 69/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2321 - acc: 0.7500\n",
            "Epoch 70/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.2317 - acc: 0.7500\n",
            "Epoch 71/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2314 - acc: 0.7500\n",
            "Epoch 72/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.2310 - acc: 0.7500\n",
            "Epoch 73/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2306 - acc: 0.7500\n",
            "Epoch 74/100\n",
            "4/4 [==============================] - 0s 1ms/step - loss: 0.2302 - acc: 0.7500\n",
            "Epoch 75/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2298 - acc: 0.7500\n",
            "Epoch 76/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2294 - acc: 0.7500\n",
            "Epoch 77/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2290 - acc: 0.7500\n",
            "Epoch 78/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2286 - acc: 0.7500\n",
            "Epoch 79/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2282 - acc: 0.7500\n",
            "Epoch 80/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2278 - acc: 0.7500\n",
            "Epoch 81/100\n",
            "4/4 [==============================] - 0s 5ms/step - loss: 0.2275 - acc: 0.7500\n",
            "Epoch 82/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2270 - acc: 0.7500\n",
            "Epoch 83/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2266 - acc: 0.7500\n",
            "Epoch 84/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2262 - acc: 0.7500\n",
            "Epoch 85/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2258 - acc: 0.7500\n",
            "Epoch 86/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2255 - acc: 0.7500\n",
            "Epoch 87/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2250 - acc: 0.7500\n",
            "Epoch 88/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2247 - acc: 0.7500\n",
            "Epoch 89/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2243 - acc: 0.7500\n",
            "Epoch 90/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2238 - acc: 0.7500\n",
            "Epoch 91/100\n",
            "4/4 [==============================] - 0s 3ms/step - loss: 0.2235 - acc: 0.7500\n",
            "Epoch 92/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2230 - acc: 0.7500\n",
            "Epoch 93/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2227 - acc: 0.7500\n",
            "Epoch 94/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2222 - acc: 0.7500\n",
            "Epoch 95/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2218 - acc: 0.7500\n",
            "Epoch 96/100\n",
            "4/4 [==============================] - 0s 1ms/step - loss: 0.2215 - acc: 0.7500\n",
            "Epoch 97/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2211 - acc: 0.7500\n",
            "Epoch 98/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2207 - acc: 0.7500\n",
            "Epoch 99/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2203 - acc: 0.7500\n",
            "Epoch 100/100\n",
            "4/4 [==============================] - 0s 2ms/step - loss: 0.2199 - acc: 0.7500\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "umxEq6l6rTaO",
        "outputId": "e20f575a-2948-4060-8c8e-05b9e49cbb79",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        }
      },
      "source": [
        "from matplotlib import pyplot as plt\n",
        "plt.plot(history.history['acc'])\n",
        "plt.title('model accuracy')\n",
        "plt.ylabel('accuracy')\n",
        "plt.xlabel('epoch')\n",
        "plt.legend(['train'], loc='upper left')\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de7xcdXnv8c+zZ1+SQCQhBNAkkKgBuVRuacRCe6yirwgKtCpFRaWtUi9UtNoW2x5U2nOOthVbW7xSLBa5iajRoggItgoiQaKGe0BoduQSArkA2ZfZ85w/1lqz18ys2+xkZSezvu/XKy/3zLrMbxzW71nr+d3M3RERkerqm+4CiIjI9FIgEBGpOAUCEZGKUyAQEak4BQIRkYpTIBARqTgFAqkUM/t3M/u7gvs+bGYnlF0mkemmQCAiUnEKBCK7ITPrn+4ySO9QIJBdTpiS+XMz+4WZPWtm/2Zm+5nZd81sq5ndYGZzY/ufbGZ3mdkmM7vZzA6JbTvKzH4WHnclMKPts15nZqvDY28xs5cWLONJZnanmW0xs3Vm9rG27ceH59sUbj8zfH+mmX3KzB4xs81m9qPwvVeY2XDC/w8nhH9/zMyuNrNLzWwLcKaZLTezW8PPeNTM/tXMBmPHH2Zm15vZU2b2uJn9lZntb2bPmdm82H5Hm9kGMxso8t2l9ygQyK7qDcCrgYOA1wPfBf4KmE/w3+37AczsIOBy4APhtmuBb5vZYFgpfhP4D2Bv4GvheQmPPQq4GPgTYB7wBWClmQ0VKN+zwNuBOcBJwHvM7NTwvAeG5f2XsExHAqvD4/4ROAb4rbBMfwE0Cv5/cgpwdfiZXwUmgA8C+wAvB14FvDcsw2zgBuB7wAuAFwM3uvtjwM3AabHzvg24wt3HC5ZDeowCgeyq/sXdH3f39cB/A7e5+53uPgJ8Azgq3O8PgP909+vDiuwfgZkEFe2xwADwT+4+7u5XA7fHPuMs4Avufpu7T7j7JcBoeFwmd7/Z3X/p7g13/wVBMPpf4ea3ADe4++Xh525099Vm1gf8EXCOu68PP/MWdx8t+P/Jre7+zfAzt7n7He7+E3evu/vDBIEsKsPrgMfc/VPuPuLuW939tnDbJcAZAGZWA95MECylohQIZFf1eOzvbQmv9wz/fgHwSLTB3RvAOmBBuG29t86s+Ejs7wOBD4WplU1mtglYFB6XycxeZmY3hSmVzcC7Ce7MCc/xYMJh+xCkppK2FbGurQwHmdl3zOyxMF30fwuUAeBbwKFmtoTgqWuzu/90imWSHqBAILu7XxNU6ACYmRFUguuBR4EF4XuRA2J/rwP+j7vPif2b5e6XF/jcy4CVwCJ33wv4PBB9zjrgRQnHPAmMpGx7FpgV+x41grRSXPtUwZ8D7gWWuvvzCFJn8TK8MKng4VPVVQRPBW9DTwOVp0Agu7urgJPM7FVhY+eHCNI7twC3AnXg/WY2YGa/DyyPHfsl4N3h3b2Z2R5hI/DsAp87G3jK3UfMbDlBOijyVeAEMzvNzPrNbJ6ZHRk+rVwMXGBmLzCzmpm9PGyTuB+YEX7+APA3QF5bxWxgC/CMmb0EeE9s23eA55vZB8xsyMxmm9nLYtu/ApwJnIwCQeUpEMhuzd3vI7iz/ReCO+7XA6939zF3HwN+n6DCe4qgPeGa2LGrgHcB/wo8DawN9y3ivcD5ZrYVOI8gIEXn/R/gRIKg9BRBQ/ER4eYPA78kaKt4Cvgk0Ofum8NzXkTwNPMs0NKLKMGHCQLQVoKgdmWsDFsJ0j6vBx4DHgB+N7b9xwSN1D9z93i6TCrItDCNSDWZ2Q+Ay9z9oukui0wvBQKRCjKz3wSuJ2jj2Drd5ZHppdSQSMWY2SUEYww+oCAgoCcCEZHK0xOBiEjF7XYTV+2zzz6+ePHi6S6GiMhu5Y477njS3dvHpgC7YSBYvHgxq1atmu5iiIjsVswstZuwUkMiIhWnQCAiUnEKBCIiFbfbtREkGR8fZ3h4mJGRkekuSqlmzJjBwoULGRjQ+iEisuP0RCAYHh5m9uzZLF68mNaJJnuHu7Nx40aGh4dZsmTJdBdHRHpIT6SGRkZGmDdvXs8GAQAzY968eT3/1CMiO19PBAKgp4NApArfUUR2vp5IDe0MG58ZZXwimI7DDObOGmSwv/s4um18gs3PTS4NO3Ogj71mDWYcManRcLaOjHPB9+8DYMZgjXe8fDF7DLX+jFetWsfwU891nsCMk494Pi/eN326/TXrN/P9ux4rVB4R2bledch+HLFozg4/rwJBAeMTDdZv2tbx/n7PmwHApk2buOyyy3jve9+be64NW0fZ9NwYAO97+5v4+wsv4uWHHJhzVODZsTqbt9X5zA8mVyx88fw9ec1h+0/uM1rnL67+BRAErDh3eGLLCJ94w0tTP+Nff7CW7931WMexIjL99n3eDAWC6RJNzLdw7iz23mOQX67fTCM2Wd+mTZv47Gc/2xEI6vU6/f39HeeaMVDjoP1mc9U3VvLkM2OFy9EIP/I7f3o8MwdrvOpTP2Tb+ETLPtHrvz3lMN728sUt217xDzd17N9u2/gERyyaw7fed1zhconI7k2BoICoAu4L75L7CO6uI+eeey4PPvggRx55JAMDA8yYMYO5c+dy7733cv/993Pqqaeybt06RkZGeOsfv5vTzjgTgOUvfQmXfvsH/GrsKU488USOP/54brnlFhYsWMC3vvUtZs6c2VKOKCAN9fcxWAvSUqP1Rss+0euktNVgfx+j442O91uPn2Co1jNNRyJSQM8Fgo9/+y7u/vWWHXrOg/bbkzctW9RsrDUz4tN3f+ITn2DNmjWsXr2am2++mZNOOok1a9Y0u3lefPHF7L333mzbto0jjj6G15x4Muw7u7nKuDs88MADXH755XzpS1/itNNO4+tf/zpnnHFGSzmigDTUX2NoICUQhHf8Q/21ju8x1F9jtJ79RDBab7DnUM/9ZyEiGXTrV0BU5UdPBGaTlXKS5cuXt/T1/8xnPsMRRxzBsccey6Pr1/PIrx5s2b/hzpIlSzjyyCMBOOaYY3j44Yc7yxE9EQz0NSv60bZUTxQYhhKeCIb6+zoCR7vR8UbisSLSu3ru1u+jrz9sh5/zmZFxHnryWSy8h+8zI2s9nz322KP5980338wNN9zArbfeyqxZs3jZb/0242OjLfs7MDQ01Hxdq9XYtq2zcdqbTwR9zco6LTUUPTHEDQ30MVIkNZTwNCEivUu3fgVEVae1PBFMRoLZs2ezdWvyin+bN29m7ty5zJo1i3vvvZfVP7s9tjU4YdFV4hpEbQS19ECwA1JDeiIQqZaeeyIog7c3Fttkughg3rx5HHfccRx++OHMnDmT/fbbr7ltxYoVfP7zn+eQQw7h4IMP5oijf5O+tr6ZRVcLjfYb7O/DzBjs72OsLRCMTWSnhtr3bzdWbyQ+TYhI71IgKCC6Y282FmMdd/GXXXZZ4rFDQ0N897vfbb6+59EtzJ4R/N/+y3sf4JGNz3LAvnuyZs2a5j4f/vCHU8thBrUwIgU5/7Y2gvEoECQ9ERRoI6g3lBoSqRjd+hXQ3n00r7E4+1zeDCjR+Yqeq+EQf5YIUj1dtBH014p1H1VqSKRSdMUX0P5E0GedTwTFzxULKM02guLlaA0EneMCoieExNTQQOcTRPv51UYgUj09c8VPtWIuIrpjt+18InD3sDKPxiOE71PsZBON1j2TKvbJ7qPdp4bGJxx3GBpQakikSnoiEMyYMYONGzeWFgyi6rcv3n20YOXdep7gX7zRGYoFFXfnuS2beOzZyYo8MTU0nvFEkLB/y7EZTxMi0rt6orF44cKFDA8Ps2HDhlLOv2XbOFtG6gxsDaZ8ePq5MUbGG/jTM7o6T8OdxzeNMDJzgI0z+hmfaPD4llHGNw4yazD/Lvz+J0e45r7neOsJweukO/zsNoI+JhpOfaJBf8I0ElmD0USkd/VEIBgYGCh11a5PfPdeLv7xeu7/u9cC8LGVd/GNO9fz84++pqvzbHxmlJO+cgPnn3IYbz9qMeueeo6TL72Jf3jjS3nTEYtyj//kl39K3Scr6aCNIDk1NJhQ0cenpcgOBEoNiVSJbv0KaO9Jk9Rts9h5Wu+40+YLSj1+vLVr59BAUq+hCfr7LLGib05LkfJ5zbSSxhGIVIqu+ALa+9ZHKZlu2yTa77jzKubO4ydaKunE1FDGXEGTo5GTg5hSQyLVVOoVb2YrzOw+M1trZucmbP+0ma0O/91vZpvKLM9UtVeuQwM13GmuWFb4PG2NsXkVc+fxjdwnk9F6I7XXT/MJJGUsgVJDItVUWhuBmdWAC4FXA8PA7Wa20t3vjvZx9w/G9v9T4KiyyrM9ku7Eo/e7Wa6yOep3oC0Q5AzymixH+5NJ5wCxrAFhhVNDeiIQqZQyr/jlwFp3f8jdx4ArgFMy9n8zcHmJ5ZmypNRQ9H635wmOD84VzRfUVWqo5ckkudfQdqeG1EYgUillXvELgHWx18Phex3M7EBgCfCDlO1nmdkqM1tVVhfRLJ0pme5y+5Pn6bzj7qbheXS8kdBG0DnXUFpqJ/eJQKkhkUraVW79TgeudvfEGtHdv+juy9x92fz583dy0YKUSfudePR+d+fprGjzBnm1HJ+UGkroNZR2R5/fRqDUkEgVlXnFrwfineMXhu8lOZ1dNC0EnQ2w250aar+rL9xG0NmNdayt99J2pYYyZi4Vkd5VZiC4HVhqZkvMbJCgsl/ZvpOZvQSYC9xaYlm2S6mpoZyJ4CJJE8IljUPImka6cGpIbQQilVLaFe/udeBs4DrgHuAqd7/LzM43s5Nju54OXOFlzhq3nZLuxGEKqaGEHHzR1FDShHBJFXt2r6G8xmKlhkSqqNQpJtz9WuDatvfOa3v9sTLLsCN0juidYmoooXtmkcVigs9KPnZy28BkWafcRqDUkEgV6davgNF6e2+dqaaG0toI8p8skkb9Jo1D2K7UUHiebsZGiMjuT1d8AaP1iZZJ3AZzUizp5+mcEC5pvqDMY/uTytGaGkqacK5IuaN5iqKlMEWkGhQICuh8IuhuRPDkeTonhCucGmqmlZLaCCYr9vayxuWVW6uTiVSTrvoc7s5YQv99mEobQWdFW3RAWWJqKKnXUMakc/19Rp9l9Rqa0OpkIhWkQJAjMzc/hdRQe0VbZEH5lnJkPJkEXUwnUtsIzCzspZQ+jkBPBCLVo6s+R9E78WLn6uzamTRfUOKxBVJD9YbT8Ozun1mfp9SQSDXpqs8x1rwTn6yAo8bYsSn0GppqamhsIuvJpNHyv1kDwqLRyImfkdHjSER6lwJBjqT++/21Pvr7rPvUUMKEcEUHlCVN/zBjoDUgNYNWRmWe9XlZ8xSJSO/SVZ8jbdWubuYImjxXZ0WbNF9QZjkyxjMUGRmc9QSi1JBINemqz5E2EVvR/v8t50pKDRVsb8gfWdy58E2SoYH0AJY1GE1EepcCQY5mBZxwJz+lXkMJqaFoW96x8f1bjh1vpO7TLjc1pCcCkcrRVZ8jMzU0hV5D7dM3FB2lnDhP0UB7Y/F2poYy5ikSkd6lqz5H2l120f7/LedKGVAWbStUjlhFHfVeaqaGCj0R5HUfVWpIpGoUCHKkLehedB2BlnOldB+NtuUdC63zFPX1GYO1yYq9UBtBRgBTakikmnTV54gq2RmJbQRTGVCW1kaQkxpKmKeoWY7xLlJDGQFMvYZEqklXfY7M1NBUeg21B5SivYZSpn+IV+zbnRoa75wCQ0R6nwJBjrS77Cn1GtrONoKkSjoekIo1FicHsMl5ivSfhEjV6KrPkT6OoPsBZWMT6d1HoykkUsuRUknH7/CLtREkL4RTZJ4iEelNuupzpM3f021qqD7RYKLhGU8EeW0EyamhwVjFXig1lDLpnJapFKkuBYIcUbqlfdWvblNDaQFlRldtBAmpoYHuU0P1hlNvewJp9o7SOAKRytFVn2O03mCw1kdf2/KN3fYaymp0jm9PPz55Qrh4QJpMY2WnhqAzFZU2cE5Eep+u+hzpvXW6G1CW1egc355+fEo54m0E9Qa1hC6miZ83nhYIlBoSqRoFghx5d+J5s4Y2z5PSkNs+X1B6OVJSQ7EBYkV6/UQ9j9qfQIqklUSkN+mqz5FeAffR8KC3TdHzBMd19j6Kb08/PqXXUNs4gtxAkPIEUqTHkYj0Jl31OdJTMt0tYJ92x90+X1Dq8SkTwrV3H81L7aSVW6khkepSIMgxOt45YyjE7uRzun02z5NS0bbPF5R1fGpqKNZrKO+OPr2NQKkhkarSVZ8jfURvdwvYZ6Veiqx2ljmgbLyL1NBATmpITwQilaNAkGO0PsFQQi+cwW4DQcYdd5GZTEfrjeQnk7ZeQ0n7tJS7llzu5uymeiIQqRxd9TmSJoqD4rOGxs8TP679XFOedC4MBJNzBeW0EQwkl1upIZHq0lWfI6sCjrYXOk/WE0HO4LSsSj6q2McmGqllLVLutJHPItL7Sr3qzWyFmd1nZmvN7NyUfU4zs7vN7C4zu6zM8kxFagXcba+hjDaC+HxBSbImhIu3VXTXfTRligm1EYhUTn9ZJzazGnAh8GpgGLjdzFa6+92xfZYCHwGOc/enzWzfssozVandR1MaXbPOAympoYHs1FDW3Xr8Dn/7UkOaYkKkqsq86pcDa939IXcfA64ATmnb513Ahe7+NIC7P1FieaZkLLWNoK+5vYj81FB6QBnLaV+Izp/WnlGk3GMKBCKVVeZVvwBYF3s9HL4XdxBwkJn92Mx+YmYrSizPlGT134+2FzpPxoRweW0EeT2OIKjIx7YnNVRgniIR6U2lpYa6+PylwCuAhcB/mdlvuPum+E5mdhZwFsABBxywUwuY1X8/2l7sPOkV7VB/jY3PjKUfmzMGITp/WtBq/6xo/9byaXUykaoq88pfDyyKvV4Yvhc3DKx093F3/xVwP0FgaOHuX3T3Ze6+bP78+aUVOOFz89sIuug1lFbR5o0jyOt6Gu0zOp5fmQ/UDLPOEdFauF6kusq88m8HlprZEjMbBE4HVrbt802CpwHMbB+CVNFDJZapK+MTjjspI4u7nWsovaLdrtRQbIWzIm0EZpb4eUXmKRKR3lRaIHD3OnA2cB1wD3CVu99lZueb2cnhbtcBG83sbuAm4M/dfWNZZepWoQq4aGooo6LNG1CW3eMoKMdzYxPUG16oMk/6vCLzFIlIbyq1jcDdrwWubXvvvNjfDvxZ+G+Xk9WlcioDytIq2rQF5ZvHZrYRBBX/lpHx1LImfl5C91GlhkSqSVd+hqw78f5aH7U+2zGpoZQF5SePzX8y2TJST90n8fMSRhYrNSRSTQoEGfIWdO9mAfusijZK1aStdlaksXjLtvCJIKE9I+3zWj9DvYZEqkpXfoa80bbdLGCf2WsoZUH5+LFp5YiC1HanhlIWvhGR3qcrP0Peql3x9YJzz5VR0eatbVBkHMGWbVFqqMgTQUKvIaWGRCpLgSDD5ERsU+v/33KurNRQNP9PSlAplBrq6omgM4ApNSRSXbryM+RNzbyjU0NpQSUrNTTYfCKI2ggKNhar15CIhHTlZyiUGuqi11Da6l+FU0MJx9f6jIGaxXoNTTE1pAFlIpWlQJAhb9WurnoNZSwakzcmIW9CuKH+Glu3dZka0oAyEQnpys+Qt6B7Un/81HNlrBWQt+xlXv5+qL9vso2gSGooYQCbUkMi1aUrP0N+G0F3qaH8NoL0J4LcQNBNr6GEAWzqNSRSXQoEGXZoaihjQrjJ1c7S2wiyKumhgVpzDMJUUkP1iQYTDdcTgUhF6crPkN9YXKzX0GRFm5MaSplvKC9/H6/ApzKgTAvXi1SbrvwMUf4/vbdPsQFleXfrOyI11Py74BQT4xPORMNbPlepIZFqUiDIMFqfoL/PqPVZ4vbBgqmhrO6fwftBBZy2/nFW19OoHM2/Cyw1Ge0ffV70HbI+Q0R6V6Er38yuMbOTzKxSNUWRO/EiqaHJ1Et676P4fp3Hp/c4CsoRbDMLViDL0z6ALS9QiUhvK3rlfxZ4C/CAmX3CzA4usUy7jCA3n9VIWzQQ5Dc6x/frOD5jDEL8+KH+PswKBIK2wKPUkEi1FQoE7n6Du78VOBp4GLjBzG4xsz80s4EyCzid8ivgGhMNp54ya2jzPAVGKMf3Szo+sxwDUSAoVpFPNk63pob0RCBSTYWvfDObB5wJvBO4E/hngsBwfSkl2wUUbaTNeyrIS71Eufn0kcXFUkNFK/KO1JB6DYlUWqGlKs3sG8DBwH8Ar3f3R8NNV5rZqrIKN93GcgZZDcUaXfcYSj9P8447paKN5gtKSw2N5SxK30wNFazI2wPYmFJDIpVWdM3iz7j7TUkb3H3ZDizPLiW3//5Adkpn8jz5FW3WKOWiTyaFU0Nt5VZqSKTail75h5rZnOiFmc01s/eWVKZdRvHUUHYX0iIVbdYo5bzpH6KKfcqpoYyFb0Sk9xW98t/l7puiF+7+NPCucoq068itgHMaeZvnKVDRBhPBpU0xkT/pXPx/87SnhtRrSKTaigaCmsX6JZpZDRgsp0i7jiKzfkJ6I+/keQqkhgZyUkNF2gjUa0hEpqBoG8H3CBqGvxC+/pPwvZ6Wt6D7ZH/88lJD9YkG9Yx5ioJjay3lydNe7slApUAgUkVFA8FfElT+7wlfXw9cVEqJdiE7LDVUoKJNG6VcZFbRyXEEU0wNNVNXSg2JVFGhQODuDeBz4b/KKJwaynsiKFDRpk1gV2T6hymnhtRrSEQoPo5gKfD/gEOBGdH77v7Cksq1Syg6oje/jaBAamigj2dH64llCLbvwAFlzXJPpob6DPpTJtcTkd5W9BbwywRPA3Xgd4GvAJeWVahdRdBGsGNSQ3kVbVpqqGj7Akx9QFmUAisyT5GI9J6igWCmu98ImLs/4u4fA04qr1jTz913XGqoQEWbNqCsWI+j7lJD0VTVk20EWrhepMqKNhaPhlNQP2BmZwPrgT3LK9b0qzechhe7E88fR5Bf0ab1GirWRtBdasjMWj5PC9eLVFvRq/8cYBbwfuAY4AzgHWUValdQtO8/FBtHkFfRDg0kDyjLm6coKGN3TwTRMZPjCLRwvUiV5QaCcPDYH7j7M+4+7O5/6O5vcPefFDh2hZndZ2ZrzezchO1nmtkGM1sd/nvnFL/HDhc1pBapgIumhrJsV2qoy3EEwb61ll5DeiIQqa7c1JC7T5jZ8d2eOAwgFwKvBoaB281spbvf3bbrle5+drfnL1uRvv/9fUafFWkszq9oU1NDBXsc5e2T9Xl5A+dEpLcVbSO408xWAl8Dno3edPdrMo5ZDqx194cAzOwK4BSgPRDsdOueeo5/uO4+xjMWlHl2LKqA0+/Eg1x7jf/8xaOsfeKZ1P3ueORp9n1exjzVBBXzyHiD91x6R8v7j20ZCbaXkBq69cGNvOfSO/j58CYOnLdH4WNFpLcUDQQzgI3AK2PvOZAVCBYA62Kvh4GXJez3BjP7HeB+4IPuvq59BzM7CzgL4IADDihY5HT//cCTrPz5r3nh/D0yu3T+xoK9OHzBXpnnet1Ln8/Phzfx4Ib0QDBn1gCvPmT/zPMc+8J5fP/uxxPPs3zJ3iyaOyv12OfvNZMVh+3P8iVzMz8jbsXh+3N9+Hl77zHICYfsV/hYEekt5u7lnNjsjcAKd39n+PptwMviaaBw1bNn3H3UzP6EoC3ilclnDCxbtsxXrdq+tXC+/ONf8fFv383q817NnFk9P3eeiAhmdkfa+jFFRxZ/meAJoIW7/1HGYeuBRbHXC8P34sdvjL28CPj7IuXZXpp2WURkUtHU0Hdif88Afg/4dc4xtwNLzWwJQQA4HXhLfAcze35s2cuTgXsKlme7RN0mB9VTRkSk8KRzX4+/NrPLgR/lHFMPB59dB9SAi939LjM7H1jl7iuB95vZyQRTVzwFnNn9V+jeaH2CgZpR09w6IiKFnwjaLQX2zdvJ3a8Frm1777zY3x8BPjLFMkyZBlCJiEwq2kawldY2gscI1ijYLWkAlYjIpKKpodllF2RnGh3X3DoiIpFCtaGZ/Z6Z7RV7PcfMTi2vWOUK1gBWakhEBIpPOvdRd98cvXD3TcBHyylS+ZQaEhGZVLQ2TNpvqg3N02603lDXURGRUNHacJWZXWBmLwr/XQDckXvULkptBCIik4rWhn8KjAFXAlcAI8D7yipU2YLUkNoIRESgeK+hZ4GO9QR2V6P1BnNn6YlARASK9xq63szmxF7PNbPryitWuYJeQwoEIiJQPDW0T9hTCAB3f5oCI4t3VWMaWSwi0lQ0EDTMrLkQgJktJmE20t2Fuo+KiEwq2gX0r4EfmdkPAQN+m3ChmN1RkcXkRUSqomhj8ffMbBlB5X8n8E1gW5kFK1OwRq9SQyIiUHzSuXcC5xAsLrMaOBa4ldalK3cL7q7UkIhITNHa8BzgN4FH3P13gaOATdmH7JrqDafhKBCIiISK1oYj7j4CYGZD7n4vcHB5xSqPlqkUEWlVtLF4OBxH8E3gejN7GnikvGKVZ3R8AkDjCEREQkUbi38v/PNjZnYTsBfwvdJKVaLJJwIFAhERmMIMou7+wzIKsrMoNSQi0qpyt8Wj9TA1pCcCERGgioFgPHwiUBuBiAhQxUCg1JCISIsKBgKlhkRE4ipXGzZTQ3oiEBEBqhgI6mojEBGJq1xtqNSQiEirytWGaiwWEWlVvUAwricCEZG4ytWGaiMQEWlVudowCgSDtcp9dRGRRKXWhma2wszuM7O1ZnZuxn5vMDMPV0Er1Wh9glqf0a9AICIClBgIzKwGXAi8FjgUeLOZHZqw32yChW9uK6sscaPjWq9YRCSuzBpxObDW3R9y9zHgCuCUhP3+FvgkMFJiWZq0cL2ISKsya8QFwLrY6+HwvSYzOxpY5O7/WWI5WgTrFavrqIhIZNpujc2sD7gA+FCBfc8ys1VmtmrDhg3b9bmj9YZ6DImIxJRZI64HFsVeLwzfi8wGDgduNrOHgWOBlUkNxu7+RXdf5u7L5s+fv12FUhuBiEirMmvE24GlZrbEzAaB04GV0UZ33+zu+7j7YndfDPwEONndV5VYJsYmGkoNiYjElBYI3L0OnA1cB9wDXOXud5nZ+WZ2chges8EAAAk7SURBVFmfmydoI9ATgYhIpOs1i7vh7tcC17a9d17Kvq8osyyR0XG1EYiIxFWuRgy6jyo1JCISqWAgUGpIRCSucjWiBpSJiLSqXI0YdB9VakhEJFK9QFCfUGOxiEhM5WpEpYZERFpVrkZUryERkVaVCgT1iQYTDdcTgYhITKVqRC1TKSLSqVI1YjMQKDUkItJUsUAwAaDUkIhITKVqxNFxpYZERNpVqkZUakhEpFPFAoFSQyIi7SpVI+qJQESkU7UCgdoIREQ6VKpGjFJDg7VKfW0RkUyVqhE1oExEpFOlasTJxmK1EYiIRKoVCKI2AvUaEhFpqlSNONlrqFJfW0QkU6VqxGZqaECpIRGRSLUCgVJDIiIdKlUjjtYb9Bn099l0F0VEZJdRqUAwNhGsTmamQCAiEqlUIBgd18L1IiLtKlUrauF6EZFOlaoVtXC9iEinigWCCT0RiIi0qVStODreUBuBiEibStWKSg2JiHQqNRCY2Qozu8/M1prZuQnb321mvzSz1Wb2IzM7tMzyKDUkItKptFrRzGrAhcBrgUOBNydU9Je5+2+4+5HA3wMXlFUeUK8hEZEkZdaKy4G17v6Qu48BVwCnxHdw9y2xl3sAXmJ5gjYCpYZERFr0l3juBcC62Oth4GXtO5nZ+4A/AwaBVyadyMzOAs4COOCAA6ZcoNG6BpSJiLSb9lrR3S909xcBfwn8Tco+X3T3Ze6+bP78+VP+LKWGREQ6lVkrrgcWxV4vDN9LcwVwaonlUa8hEZEEZQaC24GlZrbEzAaB04GV8R3MbGns5UnAAyWWJ5hrSE8EIiItSmsjcPe6mZ0NXAfUgIvd/S4zOx9Y5e4rgbPN7ARgHHgaeEdZ5YHwiUBtBCIiLcpsLMbdrwWubXvvvNjf55T5+XH1iQb1his1JCLSpjK3x2MTWp1MRCRJZWrFaJnKQQUCEZEWlakVR+vRE4FSQyIicRUKBBOAUkMiIu0qUys2nwjUa0hEpEVlasWojUCpIRGRVtUJBEoNiYgkqkytONlYXJmvLCJSSGVqxeYTwYBSQyIicdUJBON6IhARSVKZWlGpIRGRZJWpFcea3UeVGhIRiatMIFCvIRGRZJWpFZUaEhFJVpla8YC9Z/Haw/fXgDIRkTalrkewK3nNYfvzmsP2n+5iiIjscirzRCAiIskUCEREKk6BQESk4hQIREQqToFARKTiFAhERCpOgUBEpOIUCEREKs7cfbrL0BUz2wA8MsXD9wGe3IHF2V1U8XtX8TtDNb93Fb8zdP+9D3T3+UkbdrtAsD3MbJW7L5vucuxsVfzeVfzOUM3vXcXvDDv2eys1JCJScQoEIiIVV7VA8MXpLsA0qeL3ruJ3hmp+7yp+Z9iB37tSbQQiItKpak8EIiLSRoFARKTiKhMIzGyFmd1nZmvN7NzpLk8ZzGyRmd1kZneb2V1mdk74/t5mdr2ZPRD+79zpLuuOZmY1M7vTzL4Tvl5iZreFv/eVZjY43WXc0cxsjpldbWb3mtk9ZvbyivzWHwz/+15jZpeb2Yxe+73N7GIze8LM1sTeS/xtLfCZ8Lv/wsyO7vbzKhEIzKwGXAi8FjgUeLOZHTq9pSpFHfiQux8KHAu8L/ye5wI3uvtS4Mbwda85B7gn9vqTwKfd/cXA08AfT0upyvXPwPfc/SXAEQTfv6d/azNbALwfWObuhwM14HR67/f+d2BF23tpv+1rgaXhv7OAz3X7YZUIBMByYK27P+TuY8AVwCnTXKYdzt0fdfefhX9vJagYFhB810vC3S4BTp2eEpbDzBYCJwEXha8NeCVwdbhLL37nvYDfAf4NwN3H3H0TPf5bh/qBmWbWD8wCHqXHfm93/y/gqba3037bU4CveOAnwBwze343n1eVQLAAWBd7PRy+17PMbDFwFHAbsJ+7PxpuegzYb5qKVZZ/Av4CaISv5wGb3L0evu7F33sJsAH4cpgSu8jM9qDHf2t3Xw/8I/A/BAFgM3AHvf97Q/pvu931W1UCQaWY2Z7A14EPuPuW+DYP+gv3TJ9hM3sd8IS73zHdZdnJ+oGjgc+5+1HAs7SlgXrttwYI8+KnEATCFwB70JlC6Xk7+retSiBYDyyKvV4YvtdzzGyAIAh81d2vCd9+PHpUDP/3iekqXwmOA042s4cJUn6vJMidzwlTB9Cbv/cwMOzut4WvryYIDL38WwOcAPzK3Te4+zhwDcF/A73+e0P6b7vd9VtVAsHtwNKwZ8EgQePSymku0w4X5sb/DbjH3S+IbVoJvCP8+x3At3Z22cri7h9x94Xuvpjgd/2Bu78VuAl4Y7hbT31nAHd/DFhnZgeHb70KuJse/q1D/wMca2azwv/eo+/d0793KO23XQm8Pew9dCywOZZCKsbdK/EPOBG4H3gQ+OvpLk9J3/F4gsfFXwCrw38nEuTMbwQeAG4A9p7uspb0/V8BfCf8+4XAT4G1wNeAoekuXwnf90hgVfh7fxOYW4XfGvg4cC+wBvgPYKjXfm/gcoI2kHGCp78/TvttASPoFfkg8EuCHlVdfZ6mmBARqbiqpIZERCSFAoGISMUpEIiIVJwCgYhIxSkQiIhUnAKByE5kZq+IZkgV2VUoEIiIVJwCgUgCMzvDzH5qZqvN7AvhegfPmNmnw7nwbzSz+eG+R5rZT8K54L8Rmyf+xWZ2g5n93Mx+ZmYvCk+/Z2wdga+GI2RFpo0CgUgbMzsE+APgOHc/EpgA3kowwdkqdz8M+CHw0fCQrwB/6e4vJRjZGb3/VeBCdz8C+C2CkaIQzAr7AYK1MV5IMFeOyLTpz99FpHJeBRwD3B7erM8kmOCrAVwZ7nMpcE24LsAcd/9h+P4lwNfMbDawwN2/AeDuIwDh+X7q7sPh69XAYuBH5X8tkWQKBCKdDLjE3T/S8qbZ/27bb6rzs4zG/p5A16FMM6WGRDrdCLzRzPaF5lqxBxJcL9EMl28BfuTum4Gnzey3w/ffBvzQgxXihs3s1PAcQ2Y2a6d+C5GCdCci0sbd7zazvwG+b2Z9BDNAvo9g8Zfl4bYnCNoRIJgS+PNhRf8Q8Ifh+28DvmBm54fneNNO/BoihWn2UZGCzOwZd99zusshsqMpNSQiUnF6IhARqTg9EYiIVJwCgYhIxSkQiIhUnAKBiEjFKRCIiFTc/wcELGoLjblWoAAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "d5aFlE4osNu7",
        "outputId": "febf9e9d-2315-4c76-ce2b-107cac65f55b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        }
      },
      "source": [
        "plt.plot(history.history['loss'])\n",
        "plt.title('model loss')\n",
        "plt.ylabel('loss')\n",
        "plt.xlabel('epoch')\n",
        "plt.legend(['train'], loc='upper left')\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3hVZfb28e9KI5TQQw1NRaWIlNCLZRQRFRh7wTbWsaCDOoOv488ZHcfeUFSw9wI2LFhAugQJRZogXUIvUgMkgfX+cXacIxJJICcn5f5c17nIfnY5a7sxN7s9j7k7IiIi+RUT7QJERKRkUXCIiEiBKDhERKRAFBwiIlIgCg4RESkQBYeIiBSIgkMkgszsVTP7Tz6XXW5mpxzudkQiTcEhIiIFouAQEZECUXBImRdcIrrDzGab2U4ze8nMapvZKDPbbmajzaxa2PJ9zGyemW0xs3Fm1ixsXhszmxGs9x6QuN93nWlms4J1vzOzVodY8zVmttjMNpvZSDOrF7SbmT1hZuvNbJuZzTGzlsG83mY2P6htlZndfkj/waTMU3CIhJwDnAocDZwFjAL+H5BM6P+TAQBmdjTwDnBrMO8L4FMzSzCzBOBj4A2gOjA82C7Bum2Al4HrgBrAUGCkmZUrSKFmdjLwAHA+UBdYAbwbzO4J9Aj2o0qwzKZg3kvAde6eBLQEvi3I94rkUnCIhDzt7uvcfRUwEZjq7jPdfTfwEdAmWO4C4HN3/8bds4FHgfJAF6ATEA886e7Z7j4CmBb2HdcCQ919qrvvdffXgD3BegVxCfCyu89w9z3AnUBnM2sMZANJwLGAufuP7r4mWC8baG5mld39F3efUcDvFQEUHCK51oX9vOsA05WCn+sR+hc+AO6+D1gJ1A/mrfLf9hy6IuznRsBtwWWqLWa2BWgQrFcQ+9ewg9BZRX13/xZ4BhgCrDezYWZWOVj0HKA3sMLMxptZ5wJ+rwig4BApqNWEAgAI3VMg9Mt/FbAGqB+05WoY9vNK4H53rxr2qeDu7xxmDRUJXfpaBeDug929HdCc0CWrO4L2ae7eF6hF6JLa+wX8XhFAwSFSUO8DZ5jZn8wsHriN0OWm74ApQA4wwMzizexsoEPYui8A15tZx+AmdkUzO8PMkgpYwzvAlWbWOrg/8l9Cl9aWm1n7YPvxwE5gN7AvuAdziZlVCS6xbQP2HcZ/BynDFBwiBeDuC4H+wNPARkI30s9y9yx3zwLOBq4ANhO6H/Jh2LrpwDWELiX9AiwOli1oDaOBu4EPCJ3lHAlcGMyuTCigfiF0OWsT8Egw71JguZltA64ndK9EpMBMAzmJiEhB6IxDREQKRMEhIiIFouAQEZECUXCIiEiBxEW7gKJQs2ZNb9y4cbTLEBEpUaZPn77R3ZP3by8TwdG4cWPS09OjXYaISIliZisO1K5LVSIiUiAKDhERKRAFh4iIFEiZuMdxINnZ2WRkZLB79+5olxJRiYmJpKSkEB8fH+1SRKSUKLPBkZGRQVJSEo0bN+a3nZmWHu7Opk2byMjIoEmTJtEuR0RKiTJ7qWr37t3UqFGj1IYGgJlRo0aNUn9WJSJFq8wGB1CqQyNXWdhHESlaEQ0OM+tlZgvNbLGZDTrA/IFmNt/MZpvZGDMLH5xmr5nNCj4jw9pfNbNlYfNaR6r+LZlZ/JKZhXoQFhH5n4gFh5nFEhq+8nRCI5FdZGbN91tsJpDq7q2AEcDDYfN2uXvr4NNnv/XuCJs3K1L78EtmNis3Z7Js4072ZO8t1G1v2bKFZ599tsDr9e7dmy1bthRqLSIiBRHJM44OwGJ3XxoMcPMu0Dd8AXcf6+6ZwWQakBLBegqscY0K1K9anl3Ze/lp/Q7Wb9tdaGcfeQVHTk7OH673xRdfULVq1UKpQUTkUEQyOOoTGmM5V0bQlpergFFh04lmlm5maWbWb79l7w8ubz0RDJ35O2Z2bbB++oYNGw5pB8yMGpXKcXTtJConxrF2225Wbt7Fvn2HHx6DBg1iyZIltG7dmvbt29O9e3f69OlD8+ahk7J+/frRrl07WrRowbBhw35dr3HjxmzcuJHly5fTrFkzrrnmGlq0aEHPnj3ZtWvXYdclInIwxeJxXDPrD6QCJ4Q1N3L3VWZ2BPCtmc1x9yXAncBaIAEYBvwDuHf/bbr7sGA+qampf/ib/t+fzmP+6m0HrTN77z6ycvYRG2OUi4/lj247N69XmXvOapHn/AcffJC5c+cya9Ysxo0bxxlnnMHcuXN/fWz25Zdfpnr16uzatYv27dtzzjnnUKNGjd9sY9GiRbzzzju88MILnH/++XzwwQf079//oPshInI4InnGsQpoEDadErT9hpmdAtwF9HH3Pbnt7r4q+HMpMA5oE0yv8ZA9wCuELokVifjYGMrFx7LXnV1Ze8neu4/Cum3eoUOH37xrMXjwYI4//ng6derEypUrWbRo0e/WadKkCa1bh54NaNeuHcuXLy+kakRE8hbJM45pQFMza0IoMC4ELg5fwMzaAEOBXu6+Pqy9GpDp7nvMrCbQleDGuZnVdfc1FnrOtB8w93AL/aMzgwPJzMph7dbd7NiTQ4wZVSvEU6dyInGxh57DFStW/PXncePGMXr0aKZMmUKFChU48cQTD/guRrly/7tKFxsbq0tVIlIkIhYc7p5jZjcBXwGxwMvuPs/M7gXS3X0k8AhQCRgevG/wc/AEVTNgqJntI3RW9KC7zw82/ZaZJQMGzAKuj9Q+5KVCQhxHJFdid/ZeNu7Ywy87s9m+O4eUauVJSsxf1x5JSUls3779gPO2bt1KtWrVqFChAgsWLCAtLa0wyxcROSwRvcfh7l8AX+zX9n9hP5+Sx3rfAcflMe/kwqzxcCTGx5JSrQLVK+awcvMulm3cSY1K5aiVVI74g5x91KhRg65du9KyZUvKly9P7dq1f53Xq1cvnn/+eZo1a8YxxxxDp06dIr0rIiL5ZmXh5bbU1FTffyCnH3/8kWbNmhXad+zb56zdtpuNO/YQY0a1igkkV0ogIS620L7jUBX2vopI2WBm0909df/2YvFUVWkQE2PUq1qe6hUT2LB9D5t3ZLF5RxbVKyXk6wxERKSkUHAUssT4WBpUr0DtyvtYv303m3dk8cvOLJKTypGcVI4Y9R0lIiVcmf5ncCQv0yXExZBSrQJNa1ciKTGOddt2s3TDDrJyCrfrkoMpC5ciRaRoldngSExMZNOmTRH/xZoYH0ujGhVpVKMie7L3sWj9Drbtyo7od+bKHY8jMTGxSL5PRMqGMnupKiUlhYyMDA61O5JDsW/vPjbvzGL1cqd8fAxJifEkxEU2u3NHABQRKSxlNjji4+OjMire7uy9PD9+CS9PWsa23Tn0ODqZf/Q6hhb1qhR5LSIih6LMPo4bbdt3Z/NG2gpemLCUrbuyuaRjI27reTRVKyREuzQRESDvx3HL7D2OaEtKjOeGE49i3O0ncWmnRrw1dQUnPzae175bTlbOvmiXJyKSJwVHlFWpEM+/+7bks5u707RWJe4ZOY+THxvHiOkZ7C2E7ttFRAqbgqOYaF6vMu9e24nX/9KBahUSuH34D/R/cSq/7MyKdmkiIr+h4ChGzIweRycz8qauPHxOK6b//At9hkxi4doDd4YoIhINCo5iyMw4v30D3ru2E3uy93H2s5MZMT2jUEYeFBE5XAqOYqxNw2qMvKkbx9RJ4vbhP9Dv2cl8v2xztMsSkTJOwVHM1amSyIjru/D4+cezYfsezh86hWtfT2fROl2+EpHo0HscJciurL28OHEpQycsJTMrh7PbpjDg5KY0rFEh2qWJSCmU13scCo4SaPPOLJ4bt5jXpqwgK2cfnY+owbntUjj9uDpUSCiznQGISCFTcJSi4Mi1dutuhqevZMSMDFZsyqRmpQQGnd6Ms9vUJyZG3beLyOFRcJTC4Mjl7kxdtpmHvlzAzJ+3kNqoGvf2bUnzepWjXZqIlGBR6XLEzHqZ2UIzW2xmgw4wf6CZzTez2WY2xswahc3ba2azgs/IsPYmZjY12OZ7ZlbmO3cyMzodUYMPru/Cw+e2YunGnZz1zCT++8WPZGblRLs8ESllIhYcZhYLDAFOB5oDF5lZ8/0WmwmkunsrYATwcNi8Xe7eOvj0CWt/CHjC3Y8CfgGuitQ+lDQxMcb5qQ0Ye9uJnJ+awrAJSzn18QmMXbA+2qWJSCkSyTOODsBid1/q7lnAu0Df8AXcfay7ZwaTacAfDhxhZgacTChkAF4D+hVq1aVAlQrxPHB2K4Zf35kKCbFc+eo07vxwts4+RKRQRDI46gMrw6Yzgra8XAWMCptONLN0M0szs9xwqAFscffc34B5btPMrg3WTy/KwZqKk/aNq/P5gO5cd8IRvDttJWcMnsQPK7dEuywRKeGKxQuAZtYfSAUeCWtuFNyUuRh40syOLMg23X2Yu6e6e2pycnIhVluyJMTFcOfpzXjr6o7szt7Ln5+dzB3Df2DVll3RLk1ESqhIBscqoEHYdErQ9htmdgpwF9DH3ffktrv7quDPpcA4oA2wCahqZrkvKxxwm/J7XY6syZe39ODKrk34ZNZqTnp0HPd9Np+1W3dHuzQRKWEiGRzTgKbBU1AJwIXAyPAFzKwNMJRQaKwPa69mZuWCn2sCXYH5Hnp2eCxwbrDo5cAnEdyHUqVKhXjuPrM5Y+84kb7H1+OVycvo9tC3DHxvFvNXb4t2eSJSQkT0PQ4z6w08CcQCL7v7/WZ2L5Du7iPNbDRwHLAmWOVnd+9jZl0IBco+QuH2pLu/FGzzCEI32qsTeiqrf/iZyoGU9vc4DtXKzZm8PHkZ701bSWbWXnofV4fbex7DEcmVol2aiBQDegFQwZGnrZnZvDx5GS9MXMqenH1c2L4Bt5zSlFpJidEuTUSiSMGh4DioDdv38PS3i3h76s+Ui4vhhpOO4qpuTUiMj412aSISBVF5c1xKluSkctzbtyVf/60HXY+qySNfLeTkR8cxbqFeIBSR/1FwyO8ckVyJYZel8u61naiUGMcVr0zjP5/NZ0/O3miXJiLFgIJD8tTpiBqMvKkbl3VuxIuTlnH2s9+xYK2evhIp6xQc8ocS42O5t29Lhl3ajtVbdtH7qYnc/fFcftmZFe3SRCRKNOqP5EvPFnVo37g6T4z+iTfTVvDp7NVc0/0ILurQkOoVy3wHxSJlip6qkgJbsHYbD3yxgPE/baBcXAx/blOfG086igbVNYStSGmix3EVHIXup3XbeWXycj6ckUFCXAyPnnc8p7WoE+2yRKSQ6HFcKXRH107igbOPY/TAE2hSsyLXvTGd+z+fT/befdEuTUQiSMEhh61B9QoMv74zl3VuxAsTl3HW05MYu2A9ZeFsVqQsUnBIoSgXF3r6auil7diVvZcrX53GBcPSmLp0kwJEpJTRPQ4pdNl79/Hu9z/z1JjFbNyxh2Z1K3N550b0bV2f8gnqvkSkpNDNcQVHkduVtZePZ63ite+Ws2DtdmolleOuM5rR5/h6hEYBFpHiTMGh4Igadydt6Wb++8WPzFm1lQ5NqnNv3xYcW6dytEsTkT+gp6okasyMzkfW4OMbu/LfPx/HT+u20/upifzfJ3oDXaQkUnBIkYmNMS7u2JCxt51I/06NeDNtBSc+Oo430lawb1/pP/MVKS0UHFLkqlVM4N6+Lfnilu60qFeZuz+eyxWvTmP9do1/LlISKDgkao6tU5m3ru7Iff1aMnXpJk5/ciJfz1urx3dFijkFh0SVmXFpp0Z8dnM3alVO5No3ptNvyGS+mLOGvbp8JVIsRTQ4zKyXmS00s8VmNugA8wea2Xwzm21mY8ys0X7zK5tZhpk9E9Y2LtjmrOBTK5L7IEWjae0kPr6xC/f/uSVbdmVzw1szOPXx8UxctCHapYnIfiIWHGYWCwwBTgeaAxeZWfP9FpsJpLp7K2AE8PB+8+8DJhxg85e4e+vgo3FNS4lycbFc0rER3952IkMubosDl770PX97bxabduyJdnkiEojkGUcHYLG7L3X3LOBdoG/4Au4+1t0zg8k0ICV3npm1A2oDX0ewRimGYmOMM1rVZdQt3bn55KP4bPZqTnl8PN/MXxft0kSEyAZHfWBl2HRG0JaXq4BRAGYWAzwG3J7Hsq8El6nutjxeQTaza80s3czSN2zQ5Y6SKDE+ltt6HsMXA7pTv1p5rnk9nfs+m09WjnrfFYmmYnFz3Mz6A6nAI0HTDcAX7p5xgMUvcffjgO7B59IDbdPdh7l7qrunJicnR6JsKSJNayfxwV+7cEWXxrw0aRnnPa+xz0WiKZLBsQpoEDadErT9hpmdAtwF9HH33AvZnYGbzGw58ChwmZk9CODuq4I/twNvE7okJqVcubhY/tWnBc/3b8uKzZmcMXgS/xo5j62Z2dEuTaTMieSY49OApmbWhFBgXAhcHL6AmbUBhgK9wm9yu/slYctcQegG+iAziwOquvtGM4sHzgRGR3AfpJjp1bIuHZvU4LFvFvL6lOWM/GE1l3duzEUdGlCrcmK0yxMpEyJ2xuHuOcBNwFfAj8D77j7PzO41sz7BYo8AlYDhwT2LkQfZbDngKzObDcwiFEgvRGYPpLiqVjGB//Q7jpE3deO4+lV4YvRPdHnwW258ewY/rtElLJFIU++4UuIt37iTN9NW8F76SnbuyeGC9g25refR1KxULtqliZRo6lZdwVHqbcnM4qkxi3hjygrKx8fy774tOLttysFXFJEDUrfqUupVrZDAPWe14Mtbe9C8XmUGvv8D938+X12XiBQyBYeUOkfVqsSbV3fk8s6NeGHiMq58dRpbMjXuh0hhUXBIqRQfG8O/+7bkgbOPY8qSjZz82HjeSFtBzl69PChyuBQcUqpd1KEhH9/Ylaa1KnH3x3PpPXgiX85dowAROQy6OS5lgrvz1by1/PeLBfy8OZN6VRK5uGNDLu7YiOoVE6JdnkixpKeqFBwC5Ozdx5gF63ljygomLd5IlfLx3H1mc85pW588uj0TKbP0VJUIEBcbw2kt6vDm1R356tYeNK1ViduH/8BlL3/Pys2ZB9+AiCg4pOw6pk4S71/Xmfv6tmDGil/o+cQEXpq0TI/vihyEgkPKtJgY49LOjflm4Al0PKI69302n/Oe/45F67ZHuzSRYkvBIQLUq1qeV65oz5MXtGbZxp30HjyRh75cQGZWTrRLEyl2FBwiATOjX5v6fDPwBPq2rs9z45Zw6uMT+HLuWsrCQyQi+aXgENlPzUrlePS84xl+fWeSEuO4/s3pXP1aum6eiwQUHCJ5aN+4Op/d3I1/ntGMKUs3ceoT43l+/BLdPJcyT8Eh8gfiYmO4uvsRjB54Aj2aJvPgqAVc/vL3bNyx5+Ari5RSCg6RfKhXtTxDL23Hg2cfx7Tlm+n91ESmLt0U7bJEokLBIZJPZsaFQd9XFcvFccGwNAa8M5PlG3dGuzSRIqXgECmgZnUr8+nN3bjxpCP5Zv46/vT4eP7fR3PYuis72qWJFImIBoeZ9TKzhWa22MwGHWD+QDObb2azzWyMmTXab35lM8sws2fC2tqZ2Zxgm4NNHQxJFFQqF8cdpx3L+L+fSP+ODXlv2kpOf3IC3y3ZGO3SRCIuYsFhZrHAEOB0oDlwkZk132+xmUCqu7cCRgAP7zf/PmDCfm3PAdcATYNPr0IuXSTfaiUl8u++Lfngr10oFx/LJS9O5f7P57Ntt84+pPSK5BlHB2Cxuy919yzgXaBv+ALuPtbdcx+OTwN+HSDazNoBtYGvw9rqApXdPc1Db2S9DvSL4D6I5EvrBlX5fEA3Lu7QkBcmLqPrA9/y0JcLWL99d7RLEyl0kQyO+sDKsOmMoC0vVwGjAMwsBngMuP0A28wowDZFikyFhDju//NxfHpTN3ocnczz45fQ/aGxvJ++8uAri5QgxeLmuJn1B1KBR4KmG4Av3D0j77UOus1rzSzdzNI3bNhQGGWK5MtxKVUYcklbxgw8gfaNq/P3EbP57xc/6sVBKTUiGRyrgAZh0ylB22+Y2SnAXUAfd899q6ozcJOZLQceBS4zsweD9VPCVj/gNgHcfZi7p7p7anJy8uHui0iBHZFciVevbM+lnRoxbMJSrnsjnU16cVBKgUgGxzSgqZk1MbME4EJgZPgCZtYGGEooNNbntrv7Je7e0N0bE7pc9bq7D3L3NcA2M+sUPE11GfBJBPdB5LDExcZwX7+W3Nu3BWMXbqDLg99y98dz+XmT+r2SkitiweHuOcBNwFfAj8D77j7PzO41sz7BYo8AlYDhZjbLzEbmsblwNwAvAouBJQT3RUSKs8s6N+arW3vQt3U93p32Myc+Opb/fvEjWTn7ol2aSIFpzHGRIrZ2626eGvMT73y/khb1KvPUhW04qlalaJcl8jsac1ykmKhTJZEHzm7FsEvbsXrLLs58eiLD9eSVlCAKDpEo6dmiDl/e2oO2Datxx4jZ3PnhHPbk7I12WSIHla/gMLNbgu4/zMxeMrMZZtYz0sWJlHa1Kyfy+l868NcTj+Sd73/m/OensGKTOk2U4i2/Zxx/cfdtQE+gGnAp8GDEqhIpQ+JiY/hHr2MZemk7lm7YyZ8eG8/fR/ygJ6+k2MpvcOR2JNgbeMPd54W1iUghOK1FHUbfdgL9OzXik1mrOemxcfzfJ3PZuScn2qWJ/EZ+g2O6mX1NKDi+MrMkQM8RihSy2pUT+VefFkz8+0lc0rEhb6StoNdTE5iyRINGSfGR3+C4ChgEtA86JYwHroxYVSJlXK3KidzbtyXvX9eZWDMueiGN//tkLtvV664UA/kNjs7AQnffEvQr9U9ga+TKEhGA9o2rM+qWHlzZtTFvpK2g5xMTGD1/XbTLkjIuv8HxHJBpZscDtxF6Y/v1iFUlIr8qnxDLPWe14MO/dqFK+Xiufj2dG9+eoX6vJGryGxw5wfgXfYFn3H0IkBS5skRkf20aVuPTm7txe8+j+WbeOno+MYEv566JdllSBuU3OLab2Z2EHsP9PBgvIz5yZYnIgcTHxnDTyU359OZu1K2ayPVvzuBv780iM0tPXknRyW9wXADsIfQ+x1pC3Zk/8seriEikHFMniY9u6MqtpzTl41mrOPe5KWT8ovc+pGjkKziCsHgLqGJmZwK73V33OESiKD42hltPOZqXL2/Pyl8y6fvMZNKW6rFdibz8djlyPvA9cB5wPjDVzM6NZGEikj8nHVuLj2/sSpXy8Vw4LI1b3p3Jys06+5DIyVe36mb2A3Bq7mBLZpYMjHb34yNcX6FQt+pSFmzfnc1z45bw8uRl7NsHl3VuxMCeR1MhIS7apUkJdbjdqseEj9AHbCrAuiJSBJIS4/l7r2MZd/tJnN22Pi9NXsYZgycx8+dfol2alDL5/eX/pZl9ZWZXmNkVwOfAF5ErS0QOVZ0qiTx4TivevroTWTn7OPf5KTz+zU9k71UvQVI48j0CoJmdA3QNJie6+0cRq6qQ6VKVlFXbdmfzr0/m8eHMVbSoV5lHzzueZnUrR7ssKSHyulSloWNFyoAv567lnx/PYeuubAac3JTrTzyS+FhdbZY/dkj3OMxsu5ltO8Bnu5lty8eX9jKzhWa22MwGHWD+QDObb2azzWyMmTUK2hsFg0XNMrN5ZnZ92Drjgm3OCj618vefQKTs6tWyDl//7QROa1GHx775iXOe+45F67ZHuywpoSJ2xmFmscBPwKlABjANuMjd54ctcxIw1d0zzeyvwInufoGZJQS17TGzSsBcoIu7rzazccDt7p7vUwidcYj8z+ez13D3J3PZsSeH2049mqu7H0FsjIbXkd873KeqDkUHYLG7L3X3LOBdQn1d/crdxwbdtAOkEXojHXfPcvfcHtzKRbhOkTLljFZ1+erWHpx0TDIPjFrAec9/x9INO6JdlpQgkfyFXB9YGTadEbTl5SpgVO6EmTUws9nBNh5y99Vhy74SXKa628wO+E8lM7vWzNLNLH3Dhg2HvhcipVByUjme79+OJy9ozZINO+k9eCIvT1rGvn2l/56nHL5i8S/5YIyPVML6v3L3le7eCjgKuNzMagezLnH344DuwefSA23T3Ye5e6q7pyYnJ0d2B0RKIDOjX5v6fP23HnQ5sib3fjaf696czg4NVSsHEcngWAU0CJtOCdp+w8xOAe4C+oRdnvpVcKYxl1BI4O6rgj+3A28TuiQmIoeoduVEXro8lf87sznfLljPOc9+py5L5A9FMjimAU3NrElws/tCYGT4AmbWBhhKKDTWh7WnmFn54OdqQDdgoZnFmVnNoD0eOJNQqIjIYTAz/tKtCa9d2YG123bT55lJjJieQY5eGpQDiFhwuHsOcBPwFfAj8L67zzOze82sT7DYI0AlYHhwzyI3WJoR6kjxB2A88Ki7zyF0o/yr4N7HLEJnMC9Eah9EyppuTWvyyY1dSalWgduH/8CpT0zgwxkZ7NW9DwmjFwBF5Hfcna/nr+PJ0Yv4cc02UhtV44kLWtOgeoVolyZFKBqP44pICWVmnNaiDp/f3I1HzzuehWu3c/pTE/loZka0S5NiQMEhInmKiTHObZfCF7d059g6SfztvR+4+Z2ZbM3MjnZpEkUKDhE5qAbVK/DutZ24vefRjJqzhtOenMCkRRujXZZEiYJDRPIlLjaGm05uykc3dKViuVj6vzSVez6ZS2aW3vsoaxQcIlIgx6VU4fMB3bmiS2Nem7KC3k9NZNryzdEuS4qQgkNECiwxPpZ/9WnBO9d0Imefc/7QKTw4aoHe+ygjFBwicsg6H1mDr27twYXtG/D8+CVc8uJU1m/fHe2yJMIUHCJyWCqWi+OBs1vx+PnH80PGFs4YPIm0pZuiXZZEkIJDRArF2W1T+PjGrlQqF8dFL6Rx76fz2ZW1N9plSQQoOESk0BxbpzKf3dyN/h0b8fLkZZz+1ATdOC+FFBwiUqgqlovjvn4tefuajuTscy4YOoUnvvlJN85LEQWHiERElyNr8uWtPejXuj5PjVnExS9MZfWWXdEuSwqBgkNEIqZSuTgev6A1j59/PPNWb+W0Jybw2nfL1dtuCafgEJGIO7ttCp8P6M7xDapyz8h59B0yiVkrt0S7LDlECg4RKRKNa1bkjas68PRFbVi/bQ/nPPcdz41bolLh420AABJsSURBVHHOSyAFh4gUGTPjrOPrMfq2E+jVog4PfbmAa15PV2+7JYyCQ0SKXOXEeJ65uA33nNWcCYs2cMbTE5mTsTXaZUk+KThEJCrMjCu7NuH96zqzb59zzvPf8e73P1MWRiUt6RQcIhJVbRpW47MB3enYpDqDPpzDHSNms2OPumovziIaHGbWy8wWmtliMxt0gPkDzWy+mc02szFm1ihob2RmM8xslpnNM7Prw9ZpZ2Zzgm0ONjOL5D6ISORVr5jAq1d24OaTj+KDGRmc+vh4xvy4LtplSR4iFhxmFgsMAU4HmgMXmVnz/RabCaS6eytgBPBw0L4G6OzurYGOwCAzqxfMew64BmgafHpFah9EpOjExhi39TyGD/7ahaTEOK56LZ2b3p7Bph17ol2a7CeSZxwdgMXuvtTds4B3gb7hC7j7WHfPDCbTgJSgPcvdc/+2lMut08zqApXdPc1DF0JfB/pFcB9EpIi1bViNz27uzsBTj+brees47cmJfLtAZx/FSSSDoz6wMmw6I2jLy1XAqNwJM2tgZrODbTzk7quD9TPys00zu9bM0s0sfcOGDYe4CyISDQlxMQz4U1M+uakrNSsl8JdX07nzwzkapraYKBY3x82sP5AKPJLb5u4rg0tYRwGXm1ntgmzT3Ye5e6q7pyYnJxduwSJSJJrVrcwnN3Xluh5H8O60nzlz8CQ9tlsMRDI4VgENwqZTgrbfMLNTgLuAPmGXp34VnGnMBboH66ccbJsiUnqUi4vlzt7NePvqTmRm7eXs5ybz/Hi9cR5NkQyOaUBTM2tiZgnAhcDI8AXMrA0wlFBorA9rTzGz8sHP1YBuwEJ3XwNsM7NOwdNUlwGfRHAfRKSY6HxkDb68tTunNKvNg6MWcNVr0/TGeZRELDjcPQe4CfgK+BF4393nmdm9ZtYnWOwRoBIwPHj0NjdYmgFTzewHYDzwqLvPCebdALwILAaWEHZfRERKt6oVEnj2krbc17cFkxZvpM+QSSxYuy3aZZU5Vhbe0kxNTfX09PRolyEihWj6is389c0ZbN+dwz/PbMZF7RsSE6PXugqTmU1399T924vFzXERkYJq16g6nw3oRttGVbnro7lc+EIaSzbsiHZZZYKCQ0RKrFpJibx5VUcePqcVC9Zs4/SnJvLGlOXRLqvUU3CISIlmZpzfvgGjbzuBbkfV5O5P5nHfZ/M1ymAEKThEpFSolZTIC5elckWXxrw0aRl/fXM6u7L2RrusUknBISKlRmyM8a8+LbjnrOaM/nEdfZ6ZxLzVemGwsCk4RKTUubJrE177Swe27sqm35DQC4O6dFV4FBwiUip1b5rMV7f24NTmoRcGz352MunLN0e7rFJBwSEipVa1igkMubgtT13YmnXb9nDu81P465vTWbk58+ArS54UHCJSqpkZfVvXZ+ztJzLw1KMZt3ADpz81kc9nr4l2aSWWgkNEyoTyCbEM+FNTRt92Ak1rV+LGt2dwzydz2ZOjJ68KSsEhImVK/arlee/azlzdrQmvTVnBuc9NYdnGndEuq0RRcIhImZMQF8M/z2zO0Evb8fPmTM4cPJEPZ2QcfEUBFBwiUoad1qIOo27pTov6VRj4/g8MfH8Wu7N16epgFBwiUqbVq1qed67pxK2nNOXDGavo/+JUNu343ZhyEkbBISJlXmyMcespRzPk4rbMWbWVPz/7HYvXq6fdvCg4REQCZ7SqyzvXdiIzK4c/PzuZL+eujXZJxZKCQ0QkTNuG1fjohq40qVmR69+czr2fzicrZ1+0yypWFBwiIvtpUL0Cw6/vzBVdGvPy5GWc+/x3TF+h7kpyKThERA6gXFws/+rTgmcvacvqLbs557kpXP1aOj+t2x7t0qIuosFhZr3MbKGZLTazQQeYP9DM5pvZbDMbY2aNgvbWZjbFzOYF8y4IW+dVM1tmZrOCT+tI7oOIlG29j6vLhL+fyB2nHcPUpZvo/dREPp65KtplRVXEgsPMYoEhwOlAc+AiM2u+32IzgVR3bwWMAB4O2jOBy9y9BdALeNLMqoatd4e7tw4+syK1DyIiABUS4rjxpKOY8PeTaN+4Ore+N4tXJy+LdllRE8kzjg7AYndf6u5ZwLtA3/AF3H2su+d2U5kGpATtP7n7ouDn1cB6IDmCtYqIHFS1igm8cmV7ejavzb8+nc/j3/zEvjI4zkckg6M+sDJsOiNoy8tVwKj9G82sA5AALAlrvj+4hPWEmZU70MbM7FozSzez9A0bNhS8ehGRA0iMj+XZS9pyXrsUBo9ZRO/BE/l2wTrcy06AFIub42bWH0gFHtmvvS7wBnClu+c+D3cncCzQHqgO/ONA23T3Ye6e6u6pyck6WRGRwhMXG8PD57Zi8EVt2JW9l7+8ms4FQ9NYsqFsvDQYyeBYBTQIm04J2n7DzE4B7gL6uPuesPbKwOfAXe6eltvu7ms8ZA/wCqFLYiIiRcrM6HN8PUYPPIH/9GvJovXbOevpSWWis8RIBsc0oKmZNTGzBOBCYGT4AmbWBhhKKDTWh7UnAB8Br7v7iP3WqRv8aUA/YG4E90FE5A/Fx8bQv1MjRt3Sg5ZBZ4m3D/+BXVmlt7PEiAWHu+cANwFfAT8C77v7PDO718z6BIs9AlQChgeP1uYGy/lAD+CKAzx2+5aZzQHmADWB/0RqH0RE8qtOlUTevrojA/7UlA9mZHDOc9+V2iFqrSzc0ElNTfX09PRolyEiZcTYhesZ8M5M4mNjGHJxWzofWSPaJR0SM5vu7qn7txeLm+MiIqXJScfU4pMbu1KtQjz9X5rK4DGLSlV/VwoOEZEIOCK5Eh/f2JXex9Xl8W9+os8zk5i1cku0yyoUCg4RkQhJSozn6Yva8MJlqWzJzObsZyczZOziEv/Oh4JDRCTCTm1em68H9uDMVvV45KuF3PnhHLL3ltxLV3HRLkBEpCyonBjPUxe2plGNCjz97WJWb93NkIvbkJQYH+3SCkxnHCIiRcTMuK3nMTx0znFMXryRkx8bz3vTfmZvCevvSsEhIlLELmjfkBHXdyalWnn+8cEczhg8sUQNFKXgEBGJgjYNq/HhX7sw5OK2bN+dw0XDpjLyh9XRLitfFBwiIlFiZpzRqi6fD+jG8Q2qMOCdmQwdv6TYP3Wl4BARibKqFRJ446qOnNGqLg+MWsA/PpjNjj050S4rTwoOEZFiIDE+lqcvbMONJx3J8OkZ9HpyAmlLN0W7rANScIiIFBMxMcYdpx3L8Os6ExtjXPRCGvd8MpetmdnRLu03FBwiIsVMauPqjLqlO5d1asTraSs46bFxvPN98XlsV8EhIlIMVUiI4999W/LZzd04KrkSd344h7OfncyyjTujXZqCQ0SkOGtRrwrvXdeJpy5szfJNmZw5eGLURxlUcIiIFHNmRt/W9Rl1S3daBKMM3vLuTLbuis69DwWHiEgJUa9qed65phMDTz2az2av4bQnJjDhpw1FXoeCQ0SkBImNMQb8qSkf39CVpMQ4Lnv5e/758Rx2ZxfdGOcKDhGREui4lCp8enM3runehDfTfqbvM5NZtG57kXx3RIPDzHqZ2UIzW2xmgw4wf6CZzTez2WY2xswaBe2tzWyKmc0L5l0Qtk4TM5sabPM9M0uI5D6IiBRXifGx3HVGc177Swc27thDn2cmMzx9ZcS/N2LBYWaxwBDgdKA5cJGZNd9vsZlAqru3AkYADwftmcBl7t4C6AU8aWZVg3kPAU+4+1HAL8BVkdoHEZGS4ISjkxl1S3daN6jKHSNmc9dHcyI6xnkkzzg6AIvdfam7ZwHvAn3DF3D3se6eGUymASlB+0/uvij4eTWwHkg2MwNOJhQyAK8B/SK4DyIiJUKtyom8eXVHrj/hSN6a+jOXvJjGhu17IvJdkQyO+kD4OVNG0JaXq4BR+zeaWQcgAVgC1AC2uHtu7195btPMrjWzdDNL37Ch6J86EBEparExxqDTj+WpC1szZ9VW+jwzKSL3PYrFzXEz6w+kAo/s114XeAO40t0LdN7l7sPcPdXdU5OTkwuvWBGRYq5v6/qMuL4LTWsnUbtKYqFvP5Jjjq8CGoRNpwRtv2FmpwB3ASe4+56w9srA58Bd7p4WNG8CqppZXHDWccBtioiUdS3rV+H1v3SIyLYjecYxDWgaPAWVAFwIjAxfwMzaAEOBPu6+Pqw9AfgIeN3dc+9n4KHRTcYC5wZNlwOfRHAfRERkPxELjuCM4CbgK+BH4H13n2dm95pZn2CxR4BKwHAzm2VmucFyPtADuCJon2VmrYN5/wAGmtliQvc8XorUPoiIyO9ZcR+isDCkpqZ6enp6tMsQESlRzGy6u6fu314sbo6LiEjJoeAQEZECUXCIiEiBKDhERKRAFBwiIlIgZeKpKjPbAKw4xNVrAhsLsZySoizud1ncZyib+619zp9G7v67rjfKRHAcDjNLP9DjaKVdWdzvsrjPUDb3W/t8eHSpSkRECkTBISIiBaLgOLhh0S4gSsrifpfFfYayud/a58OgexwiIlIgOuMQEZECUXCIiEiBKDj+gJn1MrOFZrbYzAZFu55IMLMGZjbWzOab2TwzuyVor25m35jZouDPatGutbCZWayZzTSzz4LpJmY2NTje7wXjwpQqZlbVzEaY2QIz+9HMOpf2Y21mfwv+bs81s3fMLLE0Hmsze9nM1pvZ3LC2Ax5bCxkc7P9sM2tbkO9ScOTBzGKBIcDpQHPgIjNrHt2qIiIHuM3dmwOdgBuD/RwEjHH3psCYYLq0uYXQWDG5HgKecPejgF+Aq6JSVWQ9BXzp7scCxxPa/1J7rM2sPjAASHX3lkAsoUHlSuOxfhXotV9bXsf2dKBp8LkWeK4gX6TgyFsHYLG7L3X3LOBdoG+Uayp07r7G3WcEP28n9IukPqF9fS1Y7DWgX3QqjAwzSwHOAF4Mpg04GcgdcbI07nMVQgOkvQTg7lnuvoVSfqwJDZFd3szigArAGkrhsXb3CcDm/ZrzOrZ9CY2w6sHQ3FXNrG5+v0vBkbf6wMqw6YygrdQys8ZAG2AqUNvd1wSz1gK1o1RWpDwJ/B3YF0zXALYEI1dC6TzeTYANwCvBJboXzawipfhYu/sq4FHgZ0KBsRWYTuk/1rnyOraH9ftNwSEAmFkl4APgVnffFj4vGOu91Dy3bWZnAuvdfXq0aylicUBb4Dl3bwPsZL/LUqXwWFcj9K/rJkA9oCK/v5xTJhTmsVVw5G0V0CBsOiVoK3XMLJ5QaLzl7h8GzetyT12DP9dHq74I6Ar0MbPlhC5Bnkzo2n/V4HIGlM7jnQFkuPvUYHoEoSApzcf6FGCZu29w92zgQ0LHv7Qf61x5HdvD+v2m4MjbNKBp8PRFAqEbaiOjXFOhC67tvwT86O6Ph80aCVwe/Hw58ElR1xYp7n6nu6e4e2NCx/Vbd78EGAucGyxWqvYZwN3XAivN7Jig6U/AfErxsSZ0iaqTmVUI/q7n7nOpPtZh8jq2I4HLgqerOgFbwy5pHZTeHP8DZtab0LXwWOBld78/yiUVOjPrBkwE5vC/6/3/j9B9jveBhoS6pD/f3fe/8VbimdmJwO3ufqaZHUHoDKQ6MBPo7+57ollfYTOz1oQeCEgAlgJXEvoHZKk91mb2b+ACQk8QzgSuJnQ9v1QdazN7BziRUPfp64B7gI85wLENQvQZQpftMoEr3T0939+l4BARkYLQpSoRESkQBYeIiBSIgkNERApEwSEiIgWi4BARkQJRcIgUc2Z2Ym4PviLFgYJDREQKRMEhUkjMrL+ZfW9ms8xsaDDexw4zeyIYD2KMmSUHy7Y2s7RgLISPwsZJOMrMRpvZD2Y2w8yODDZfKWwcjbeCF7hEokLBIVIIzKwZobeTu7p7a2AvcAmhTvXS3b0FMJ7Q27wArwP/cPdWhN7az21/Cxji7scDXQj16AqhXotvJTQ2zBGE+lsSiYq4gy8iIvnwJ6AdMC04GShPqEO5fcB7wTJvAh8G42JUdffxQftrwHAzSwLqu/tHAO6+GyDY3vfunhFMzwIaA5Miv1siv6fgECkcBrzm7nf+ptHs7v2WO9Q+fsL7UdqL/t+VKNKlKpHCMQY418xqwa9jPTci9P9Ybi+sFwOT3H0r8IuZdQ/aLwXGByMwZphZv2Ab5cysQpHuhUg+6F8tIoXA3eeb2T+Br80sBsgGbiQ0WFKHYN56QvdBINTF9fNBMOT2UguhEBlqZvcG2zivCHdDJF/UO65IBJnZDnevFO06RAqTLlWJiEiB6IxDREQKRGccIiJSIAoOEREpEAWHiIgUiIJDREQKRMEhIiIF8v8BIf3TOJtBQTEAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tgq6G33ipobE"
      },
      "source": [
        "# Example: ANN trained on the Digits dataset\n",
        "\n",
        "In this example, we train a neural network on the [Digits dataset](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_digits.html)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4dnGceA6teox",
        "outputId": "57b74fad-a0a6-4051-d937-dcc712ef0248",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 309
        }
      },
      "source": [
        "from sklearn.datasets import load_digits\n",
        "dig = load_digits()\n",
        "print(dig.data.shape)\n",
        "\n",
        "import matplotlib.pyplot as plt \n",
        "plt.gray() \n",
        "plt.matshow(dig.images[13]) \n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(1797, 64)\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 0 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPoAAAECCAYAAADXWsr9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAL+0lEQVR4nO3d34sd9RnH8c+nmwStkay/GtSIa6EERGgSJFQUTQ0RrWJz0QsDCpGW9KKVLC2I9qb6D0h6UYQQNYIa0WikSGsVTBChtU3iWmMSi8qKCWqMIcSINCQ+vTgTSdc0O7uZ73dP9nm/4JCzZ8+Z59kNnzMzZ2fmcUQIwPT2naluAEB5BB1IgKADCRB0IAGCDiRA0IEE+iLotm+2/a7t92zfV7jWo7b32d5Rss4J9S6zvdn2Ttvv2F5duN5Ztv9h+62m3oMl6zU1B2y/afvF0rWaeqO237Y9Yntr4VqDtjfa3m17l+1rCtaa3/xMx2+HbA93svCImNKbpAFJ70v6vqRZkt6SdGXBetdLWiRpR6Wf72JJi5r750r6d+Gfz5JmN/dnSnpD0o8K/4y/kfSUpBcr/U5HJV1Yqdbjkn7R3J8labBS3QFJn0i6vIvl9cMafbGk9yLig4g4IulpST8tVSwiXpN0oNTyT1Lv44jY3tz/QtIuSZcWrBcRcbj5cmZzK3ZUlO15km6VtK5Ujalie456K4ZHJCkijkTEwUrll0p6PyI+7GJh/RD0SyV9dMLXe1QwCFPJ9pCkheqtZUvWGbA9ImmfpFciomS9NZLulfR1wRpjhaSXbW+zvapgnSskfSbpsWbXZJ3tcwrWO9EdkjZ0tbB+CHoKtmdLek7ScEQcKlkrIo5FxAJJ8yQttn1ViTq2b5O0LyK2lVj+KVwXEYsk3SLpV7avL1Rnhnq7eQ9HxEJJX0oq+hmSJNmeJel2Sc92tcx+CPpeSZed8PW85rFpw/ZM9UL+ZEQ8X6tus5m5WdLNhUpcK+l226Pq7XLdaPuJQrW+ERF7m3/3Sdqk3u5fCXsk7Tlhi2ijesEv7RZJ2yPi064W2A9B/6ekH9i+onknu0PSn6a4p87Ytnr7eLsi4qEK9S6yPdjcP1vSMkm7S9SKiPsjYl5EDKn3//ZqRNxZotZxts+xfe7x+5JuklTkLygR8Ymkj2zPbx5aKmlniVpjrFCHm+1Sb9NkSkXEUdu/lvRX9T5pfDQi3ilVz/YGSUskXWh7j6TfR8Qjpeqpt9a7S9LbzX6zJP0uIv5cqN7Fkh63PaDeG/kzEVHlz16VzJW0qff+qRmSnoqIlwrWu0fSk81K6ANJdxesdfzNa5mkX3a63OajfADTWD9sugMojKADCRB0IAGCDiRA0IEE+irohQ9nnLJa1KPeVNfrq6BLqvnLrPofRz3qTWW9fgs6gAKKHDBju+pROLNmzZrwa44dO6aBgYFJ1Zs7d+6EX3P48GHNnj17UvUuuOCCCb/m888/n9TrpN7vZqIOHDig888/f1L1Dh6c+Jmfp/P73L9//4Rfc/ToUc2YMbkDSb/66qtJvW6yIsJjH5vyQ2C7cMkll1StNzzczUU/2lq5cmXVepMJ3ul44YUXqtZbv3591XojIyPjP6kwNt2BBAg6kABBBxIg6EACBB1IgKADCRB0IAGCDiTQKug1RyYB6N64QW8uMvhH9S5Be6WkFbavLN0YgO60WaNXHZkEoHttgp5mZBIwXXV2Uktzonztc3YBtNAm6K1GJkXEWklrpfqnqQI4tTab7tN6ZBKQwbhr9NojkwB0r9U+ejMnrNSsMACFcWQckABBBxIg6EACBB1IgKADCRB0IAGCDiRA0IEEpsWklqGhoar1lixZUrXemjVrqtYbHBysWm/16tVV69WeRMOkFgBVEHQgAYIOJEDQgQQIOpAAQQcSIOhAAgQdSICgAwkQdCCBNiOZHrW9z/aOGg0B6F6bNfp6STcX7gNAQeMGPSJek3SgQi8ACmEfHUiA2WtAAp0FndlrQP9i0x1IoM2f1zZI+puk+bb32P55+bYAdKnNkMUVNRoBUA6b7kACBB1IgKADCRB0IAGCDiRA0IEECDqQAEEHEnBE94elc6x7t1auXFm13gMPPFC1Xu1Zb7Vn59WevRYRHvsYa3QgAYIOJEDQgQQIOpAAQQcSIOhAAgQdSICgAwkQdCABgg4k0ObikJfZ3mx7p+13bK+u0RiA7rS5rvtRSb+NiO22z5W0zfYrEbGzcG8AOtJm9trHEbG9uf+FpF2SLi3dGIDuTGgf3faQpIWS3ijRDIAyWo9ksj1b0nOShiPi0Em+z+w1oE+1CrrtmeqF/MmIeP5kz2H2GtC/2nzqbkmPSNoVEQ+VbwlA19rso18r6S5JN9oeaW4/KdwXgA61mb32uqRvXZoGwJmDI+OABAg6kABBBxIg6EACBB1IgKADCRB0IAGCDiTQ+qQWTJ3ly5dPdQtFLViwoGq90dHRqvX6AWt0IAGCDiRA0IEECDqQAEEHEiDoQAIEHUiAoAMJEHQgAYIOJNDmKrBn2f6H7bea2WsP1mgMQHfaHOv+H0k3RsTh5vrur9v+S0T8vXBvADrS5iqwIelw8+XM5saABuAM0mof3faA7RFJ+yS9EhHMXgPOIK2CHhHHImKBpHmSFtu+auxzbK+yvdX21q6bBHB6JvSpe0QclLRZ0s0n+d7aiLg6Iq7uqjkA3WjzqftFtgeb+2dLWiZpd+nGAHSnzafuF0t63PaAem8Mz0TEi2XbAtClNp+6/0vSwgq9ACiEI+OABAg6kABBBxIg6EACBB1IgKADCRB0IAGCDiTg3lmoHS/U5jTWDg0NDVWtNzIyUrXeli1bqtab7rPsIsJjH2ONDiRA0IEECDqQAEEHEiDoQAIEHUiAoAMJEHQgAYIOJEDQgQRaB70Z4vCmbS4MCZxhJrJGXy1pV6lGAJTTdiTTPEm3SlpXth0AJbRdo6+RdK+krwv2AqCQNpNabpO0LyK2jfM8Zq8BfarNGv1aSbfbHpX0tKQbbT8x9knMXgP617hBj4j7I2JeRAxJukPSqxFxZ/HOAHSGv6MDCbQZsviNiNgiaUuRTgAUwxodSICgAwkQdCABgg4kQNCBBAg6kABBBxIg6EACzF7Dt0z3WW+1Z6/Vni3H7DUgKYIOJEDQgQQIOpAAQQcSIOhAAgQdSICgAwkQdCABgg4k0Oqacc2lnr+QdEzSUS7pDJxZJnJxyB9HxP5inQAohk13IIG2QQ9JL9veZntVyYYAdK/tpvt1EbHX9vckvWJ7d0S8duITmjcA3gSAPtRqjR4Re5t/90naJGnxSZ7D7DWgT7WZpnqO7XOP35d0k6QdpRsD0J02m+5zJW2yffz5T0XES0W7AtCpcYMeER9I+mGFXgAUwp/XgAQIOpAAQQcSIOhAAgQdSICgAwkQdCABgg4kMJHz0fvW4OBg1Xo33HBD1XrnnXde1XrDw8NV682ZM6dqvdqz5foBa3QgAYIOJEDQgQQIOpAAQQcSIOhAAgQdSICgAwkQdCABgg4k0Crotgdtb7S92/Yu29eUbgxAd9oe6/4HSS9FxM9sz5L03YI9AejYuEG3PUfS9ZJWSlJEHJF0pGxbALrUZtP9CkmfSXrM9pu21zWDHP6H7VW2t9re2nmXAE5Lm6DPkLRI0sMRsVDSl5LuG/skRjIB/atN0PdI2hMRbzRfb1Qv+ADOEOMGPSI+kfSR7fnNQ0sl7SzaFYBOtf3U/R5JTzafuH8g6e5yLQHoWqugR8SIJPa9gTMUR8YBCRB0IAGCDiRA0IEECDqQAEEHEiDoQAIEHUjAEdH9Qu3uF3oKtWdprV+/vmq92mrPshsdHa1ab/ny5VXr1RYRHvsYa3QgAYIOJEDQgQQIOpAAQQcSIOhAAgQdSICgAwkQdCCBcYNue77tkRNuh2wP12gOQDfGvWZcRLwraYEk2R6QtFfSpsJ9AejQRDfdl0p6PyI+LNEMgDImGvQ7JG0o0QiAcloHvbmm++2Snv0/32f2GtCn2g5wkKRbJG2PiE9P9s2IWCtprVT/NFUApzaRTfcVYrMdOCO1CnozJnmZpOfLtgOghLYjmb6UdEHhXgAUwpFxQAIEHUiAoAMJEHQgAYIOJEDQgQQIOpAAQQcSIOhAAqVmr30maTLnrF8oaX/H7fRDLepRr1a9yyPiorEPFgn6ZNneGhFXT7da1KPeVNdj0x1IgKADCfRb0NdO01rUo96U1uurfXQAZfTbGh1AAQQdSICgAwkQdCABgg4k8F/pGZ+sZXlzvAAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 288x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i1sW43QZmQ_i"
      },
      "source": [
        "$$\n",
        "[0,0,0,1,0,0,0,0,0,0]\n",
        "$$"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sBqEI_qXbftv",
        "outputId": "b1347aa1-0ac4-46a2-be3f-6038acd9a52c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "import pandas as pd\n",
        "from keras.layers import Dense\n",
        "from keras.models import Sequential\n",
        "from keras.optimizers import RMSprop, Adadelta, Adam\n",
        "from sklearn.datasets import load_digits\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "dig = load_digits()\n",
        "\n",
        "print(dig.target)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[0 1 2 ... 8 9 8]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "k_azFjYJmqkk",
        "outputId": "55b60b2a-93a9-46bf-cba1-eb27e5e46404",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "onehot_target = pd.get_dummies(dig.target)\n",
        "x_train, x_val, y_train, y_val = train_test_split(dig.data, onehot_target, test_size=0.1, random_state=20)\n",
        "\n",
        "model = Sequential()\n",
        "model.add(Dense(128, input_dim=x_train.shape[1], activation='relu'))\n",
        "model.add(Dense(128, activation='relu'))\n",
        "model.add(Dense(10, activation='softmax'))\n",
        "\n",
        "model.summary()\n",
        "\n",
        "model.compile(optimizer=Adadelta(), loss='categorical_crossentropy', metrics=['categorical_accuracy'])\n",
        "model.fit(x_train, y_train, epochs=50, batch_size=64)\n",
        "\n",
        "scores = model.evaluate(x_val, y_val)\n",
        "print(\"\\n%s: %.2f%%\" % (model.metrics_names[1], scores[1]*100))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_35\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "dense_67 (Dense)             (None, 128)               8320      \n",
            "_________________________________________________________________\n",
            "dense_68 (Dense)             (None, 128)               16512     \n",
            "_________________________________________________________________\n",
            "dense_69 (Dense)             (None, 10)                1290      \n",
            "=================================================================\n",
            "Total params: 26,122\n",
            "Trainable params: 26,122\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n",
            "Epoch 1/50\n",
            "26/26 [==============================] - 0s 3ms/step - loss: 5.8203 - categorical_accuracy: 0.1045\n",
            "Epoch 2/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 5.7574 - categorical_accuracy: 0.1045\n",
            "Epoch 3/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 5.6959 - categorical_accuracy: 0.1039\n",
            "Epoch 4/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 5.6355 - categorical_accuracy: 0.1039\n",
            "Epoch 5/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 5.5754 - categorical_accuracy: 0.1051\n",
            "Epoch 6/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 5.5167 - categorical_accuracy: 0.1045\n",
            "Epoch 7/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 5.4586 - categorical_accuracy: 0.1051\n",
            "Epoch 8/50\n",
            "26/26 [==============================] - 0s 1ms/step - loss: 5.4012 - categorical_accuracy: 0.1051\n",
            "Epoch 9/50\n",
            "26/26 [==============================] - 0s 1ms/step - loss: 5.3442 - categorical_accuracy: 0.1033\n",
            "Epoch 10/50\n",
            "26/26 [==============================] - 0s 1ms/step - loss: 5.2882 - categorical_accuracy: 0.1033\n",
            "Epoch 11/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 5.2330 - categorical_accuracy: 0.1045\n",
            "Epoch 12/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 5.1782 - categorical_accuracy: 0.1039\n",
            "Epoch 13/50\n",
            "26/26 [==============================] - 0s 1ms/step - loss: 5.1243 - categorical_accuracy: 0.1020\n",
            "Epoch 14/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 5.0708 - categorical_accuracy: 0.1008\n",
            "Epoch 15/50\n",
            "26/26 [==============================] - 0s 1ms/step - loss: 5.0185 - categorical_accuracy: 0.1027\n",
            "Epoch 16/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 4.9665 - categorical_accuracy: 0.1039\n",
            "Epoch 17/50\n",
            "26/26 [==============================] - 0s 1ms/step - loss: 4.9152 - categorical_accuracy: 0.1039\n",
            "Epoch 18/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 4.8648 - categorical_accuracy: 0.1039\n",
            "Epoch 19/50\n",
            "26/26 [==============================] - 0s 1ms/step - loss: 4.8146 - categorical_accuracy: 0.1082\n",
            "Epoch 20/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 4.7650 - categorical_accuracy: 0.1095\n",
            "Epoch 21/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 4.7163 - categorical_accuracy: 0.1076\n",
            "Epoch 22/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 4.6680 - categorical_accuracy: 0.1088\n",
            "Epoch 23/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 4.6205 - categorical_accuracy: 0.1107\n",
            "Epoch 24/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 4.5735 - categorical_accuracy: 0.1126\n",
            "Epoch 25/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 4.5269 - categorical_accuracy: 0.1156\n",
            "Epoch 26/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 4.4809 - categorical_accuracy: 0.1150\n",
            "Epoch 27/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 4.4356 - categorical_accuracy: 0.1194\n",
            "Epoch 28/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 4.3909 - categorical_accuracy: 0.1200\n",
            "Epoch 29/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 4.3470 - categorical_accuracy: 0.1224\n",
            "Epoch 30/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 4.3036 - categorical_accuracy: 0.1218\n",
            "Epoch 31/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 4.2612 - categorical_accuracy: 0.1237\n",
            "Epoch 32/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 4.2190 - categorical_accuracy: 0.1274\n",
            "Epoch 33/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 4.1774 - categorical_accuracy: 0.1286\n",
            "Epoch 34/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 4.1370 - categorical_accuracy: 0.1299\n",
            "Epoch 35/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 4.0967 - categorical_accuracy: 0.1317\n",
            "Epoch 36/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 4.0573 - categorical_accuracy: 0.1323\n",
            "Epoch 37/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 4.0184 - categorical_accuracy: 0.1323\n",
            "Epoch 38/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 3.9804 - categorical_accuracy: 0.1317\n",
            "Epoch 39/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 3.9428 - categorical_accuracy: 0.1330\n",
            "Epoch 40/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 3.9058 - categorical_accuracy: 0.1330\n",
            "Epoch 41/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 3.8690 - categorical_accuracy: 0.1342\n",
            "Epoch 42/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 3.8332 - categorical_accuracy: 0.1336\n",
            "Epoch 43/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 3.7980 - categorical_accuracy: 0.1354\n",
            "Epoch 44/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 3.7633 - categorical_accuracy: 0.1373\n",
            "Epoch 45/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 3.7295 - categorical_accuracy: 0.1373\n",
            "Epoch 46/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 3.6962 - categorical_accuracy: 0.1385\n",
            "Epoch 47/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 3.6635 - categorical_accuracy: 0.1416\n",
            "Epoch 48/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 3.6315 - categorical_accuracy: 0.1422\n",
            "Epoch 49/50\n",
            "26/26 [==============================] - 0s 1ms/step - loss: 3.6001 - categorical_accuracy: 0.1429\n",
            "Epoch 50/50\n",
            "26/26 [==============================] - 0s 2ms/step - loss: 3.5690 - categorical_accuracy: 0.1435\n",
            "6/6 [==============================] - 0s 2ms/step - loss: 3.5674 - categorical_accuracy: 0.1444\n",
            "\n",
            "categorical_accuracy: 14.44%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-pAv-bbD8Y6Z"
      },
      "source": [
        "# Example: Comparing ANN and SVC on the Digits dataset\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ddZzSPHKvC9_",
        "outputId": "df7b8d10-58d7-4d40-f453-8fd6690a5ed8",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "import numpy\n",
        "import pandas\n",
        "from sklearn.datasets import load_digits\n",
        "from sklearn import preprocessing\n",
        "from sklearn.model_selection import KFold\n",
        "from sklearn.svm import SVC\n",
        "from sklearn.metrics import zero_one_loss\n",
        "from keras.models import Sequential\n",
        "from keras.layers.core import Dense, Activation\n",
        "\n",
        "dataset = load_digits()\n",
        "\n",
        "X = dataset[\"data\"]\n",
        "y = dataset[\"target\"]\n",
        "y_indicators = pandas.get_dummies(y).values\n",
        "\n",
        "# ablation study\n",
        "\n",
        "# Center each feature and scale the variance to be unitary\n",
        "X = preprocessing.scale(X)\n",
        "\n",
        "svc = SVC(gamma=0.001)\n",
        "\n",
        "# Set up variables\n",
        "svc_error = 0\n",
        "ann_error = 0\n",
        "n_folds = 10\n",
        "\n",
        "kf = KFold(n_splits=n_folds)\n",
        "\n",
        "for train_inds, test_inds in kf.split(X):\n",
        "    X_train, X_test = X[train_inds], X[test_inds]\n",
        "    y_train, y_test = y[train_inds], y[test_inds]\n",
        "    y_train_indicators, y_test_indicators = y_indicators[train_inds, :], y_indicators[test_inds, :]\n",
        "\n",
        "    # Use SVM\n",
        "    svc.fit(X_train, y_train)\n",
        "    y_pred = svc.predict(X_test)\n",
        "    svc_error += zero_one_loss(y_test, y_pred)\n",
        "\n",
        "    # Use deep learner\n",
        "    ann = Sequential()\n",
        "    ann.add(Dense(256, input_dim=X.shape[1], kernel_initializer=\"glorot_uniform\"))\n",
        "    ann.add(Activation(\"relu\"))\n",
        "    ann.add(Dense(10, kernel_initializer=\"glorot_uniform\"))\n",
        "    ann.add(Activation(\"sigmoid\"))\n",
        "    ann.compile(loss='categorical_crossentropy', optimizer='sgd')\n",
        "\n",
        "    ann.fit(X_train, y_train_indicators, epochs=50, batch_size=32)\n",
        "    y_pred = ann.predict(X_test)\n",
        "    # The predicted class is the output response with the largest value\n",
        "    y_pred = numpy.argmax(y_pred, 1)\n",
        "    ann_error += zero_one_loss(y_test, y_pred)\n",
        "\n",
        "print(svc_error/n_folds)\n",
        "print(ann_error/n_folds)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 2.1895\n",
            "Epoch 2/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 2.0350\n",
            "Epoch 3/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.9101\n",
            "Epoch 4/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.7926\n",
            "Epoch 5/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.6736\n",
            "Epoch 6/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.5505\n",
            "Epoch 7/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.4237\n",
            "Epoch 8/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 1.2962\n",
            "Epoch 9/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.1716\n",
            "Epoch 10/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.0531\n",
            "Epoch 11/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.9434\n",
            "Epoch 12/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.8446\n",
            "Epoch 13/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.7568\n",
            "Epoch 14/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.6801\n",
            "Epoch 15/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.6135\n",
            "Epoch 16/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.5562\n",
            "Epoch 17/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.5069\n",
            "Epoch 18/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.4643\n",
            "Epoch 19/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.4277\n",
            "Epoch 20/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.3959\n",
            "Epoch 21/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.3682\n",
            "Epoch 22/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.3439\n",
            "Epoch 23/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.3225\n",
            "Epoch 24/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.3035\n",
            "Epoch 25/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.2867\n",
            "Epoch 26/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.2716\n",
            "Epoch 27/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.2579\n",
            "Epoch 28/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.2455\n",
            "Epoch 29/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.2343\n",
            "Epoch 30/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.2241\n",
            "Epoch 31/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.2147\n",
            "Epoch 32/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.2060\n",
            "Epoch 33/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1981\n",
            "Epoch 34/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1907\n",
            "Epoch 35/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1838\n",
            "Epoch 36/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1774\n",
            "Epoch 37/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1714\n",
            "Epoch 38/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1658\n",
            "Epoch 39/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1606\n",
            "Epoch 40/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1556\n",
            "Epoch 41/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1510\n",
            "Epoch 42/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1467\n",
            "Epoch 43/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1425\n",
            "Epoch 44/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1386\n",
            "Epoch 45/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1349\n",
            "Epoch 46/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1313\n",
            "Epoch 47/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1280\n",
            "Epoch 48/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1248\n",
            "Epoch 49/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1217\n",
            "Epoch 50/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1188\n",
            "WARNING:tensorflow:7 out of the last 10 calls to <function Model.make_predict_function.<locals>.predict_function at 0x7f2c25932048> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/tutorials/customization/performance#python_or_tensor_args and https://www.tensorflow.org/api_docs/python/tf/function for  more details.\n",
            "Epoch 1/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 2.1820\n",
            "Epoch 2/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 2.0163\n",
            "Epoch 3/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 1.8910\n",
            "Epoch 4/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 1.7776\n",
            "Epoch 5/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.6644\n",
            "Epoch 6/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.5466\n",
            "Epoch 7/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.4225\n",
            "Epoch 8/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.2943\n",
            "Epoch 9/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.1669\n",
            "Epoch 10/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 1.0451\n",
            "Epoch 11/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.9327\n",
            "Epoch 12/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.8317\n",
            "Epoch 13/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.7428\n",
            "Epoch 14/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.6659\n",
            "Epoch 15/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.5996\n",
            "Epoch 16/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.5429\n",
            "Epoch 17/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.4943\n",
            "Epoch 18/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.4525\n",
            "Epoch 19/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.4165\n",
            "Epoch 20/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.3853\n",
            "Epoch 21/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.3581\n",
            "Epoch 22/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.3342\n",
            "Epoch 23/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.3132\n",
            "Epoch 24/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.2946\n",
            "Epoch 25/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.2778\n",
            "Epoch 26/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.2629\n",
            "Epoch 27/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.2496\n",
            "Epoch 28/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.2373\n",
            "Epoch 29/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.2263\n",
            "Epoch 30/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.2162\n",
            "Epoch 31/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.2070\n",
            "Epoch 32/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1984\n",
            "Epoch 33/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1906\n",
            "Epoch 34/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1834\n",
            "Epoch 35/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1767\n",
            "Epoch 36/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1704\n",
            "Epoch 37/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1646\n",
            "Epoch 38/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1591\n",
            "Epoch 39/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1540\n",
            "Epoch 40/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1492\n",
            "Epoch 41/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1447\n",
            "Epoch 42/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1404\n",
            "Epoch 43/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1364\n",
            "Epoch 44/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1325\n",
            "Epoch 45/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1290\n",
            "Epoch 46/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1256\n",
            "Epoch 47/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1223\n",
            "Epoch 48/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1192\n",
            "Epoch 49/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1163\n",
            "Epoch 50/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1134\n",
            "Epoch 1/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 2.2589\n",
            "Epoch 2/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 2.1273\n",
            "Epoch 3/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 2.0289\n",
            "Epoch 4/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.9411\n",
            "Epoch 5/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.8542\n",
            "Epoch 6/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.7633\n",
            "Epoch 7/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 1.6663\n",
            "Epoch 8/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 1.5625\n",
            "Epoch 9/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.4524\n",
            "Epoch 10/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 1.3375\n",
            "Epoch 11/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 1.2212\n",
            "Epoch 12/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.1065\n",
            "Epoch 13/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.9964\n",
            "Epoch 14/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.8946\n",
            "Epoch 15/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.8025\n",
            "Epoch 16/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.7206\n",
            "Epoch 17/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.6493\n",
            "Epoch 18/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.5873\n",
            "Epoch 19/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.5340\n",
            "Epoch 20/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.4880\n",
            "Epoch 21/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.4481\n",
            "Epoch 22/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.4136\n",
            "Epoch 23/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.3834\n",
            "Epoch 24/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.3569\n",
            "Epoch 25/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.3336\n",
            "Epoch 26/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.3130\n",
            "Epoch 27/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.2947\n",
            "Epoch 28/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.2784\n",
            "Epoch 29/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.2636\n",
            "Epoch 30/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.2504\n",
            "Epoch 31/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.2383\n",
            "Epoch 32/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.2274\n",
            "Epoch 33/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.2175\n",
            "Epoch 34/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.2083\n",
            "Epoch 35/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1998\n",
            "Epoch 36/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1920\n",
            "Epoch 37/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1849\n",
            "Epoch 38/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1782\n",
            "Epoch 39/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1719\n",
            "Epoch 40/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1661\n",
            "Epoch 41/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1608\n",
            "Epoch 42/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1557\n",
            "Epoch 43/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1509\n",
            "Epoch 44/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1463\n",
            "Epoch 45/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1422\n",
            "Epoch 46/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1381\n",
            "Epoch 47/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1343\n",
            "Epoch 48/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1307\n",
            "Epoch 49/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1274\n",
            "Epoch 50/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1241\n",
            "Epoch 1/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 2.2395\n",
            "Epoch 2/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 2.0828\n",
            "Epoch 3/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 1.9678\n",
            "Epoch 4/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 1.8640\n",
            "Epoch 5/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.7596\n",
            "Epoch 6/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.6497\n",
            "Epoch 7/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.5333\n",
            "Epoch 8/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 1.4120\n",
            "Epoch 9/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 1.2890\n",
            "Epoch 10/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.1681\n",
            "Epoch 11/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.0533\n",
            "Epoch 12/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.9475\n",
            "Epoch 13/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.8522\n",
            "Epoch 14/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.7678\n",
            "Epoch 15/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.6938\n",
            "Epoch 16/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.6295\n",
            "Epoch 17/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.5737\n",
            "Epoch 18/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.5253\n",
            "Epoch 19/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.4833\n",
            "Epoch 20/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.4466\n",
            "Epoch 21/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.4144\n",
            "Epoch 22/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.3863\n",
            "Epoch 23/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.3613\n",
            "Epoch 24/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.3393\n",
            "Epoch 25/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.3195\n",
            "Epoch 26/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.3017\n",
            "Epoch 27/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.2858\n",
            "Epoch 28/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.2715\n",
            "Epoch 29/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.2584\n",
            "Epoch 30/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.2466\n",
            "Epoch 31/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.2357\n",
            "Epoch 32/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.2257\n",
            "Epoch 33/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.2165\n",
            "Epoch 34/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.2081\n",
            "Epoch 35/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.2002\n",
            "Epoch 36/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1929\n",
            "Epoch 37/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1861\n",
            "Epoch 38/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1798\n",
            "Epoch 39/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1738\n",
            "Epoch 40/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1683\n",
            "Epoch 41/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1631\n",
            "Epoch 42/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1582\n",
            "Epoch 43/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1535\n",
            "Epoch 44/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1492\n",
            "Epoch 45/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1450\n",
            "Epoch 46/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1410\n",
            "Epoch 47/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1374\n",
            "Epoch 48/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1337\n",
            "Epoch 49/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1304\n",
            "Epoch 50/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1272\n",
            "Epoch 1/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 2.2816\n",
            "Epoch 2/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 2.1121\n",
            "Epoch 3/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 1.9956\n",
            "Epoch 4/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 1.8927\n",
            "Epoch 5/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 1.7894\n",
            "Epoch 6/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.6804\n",
            "Epoch 7/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.5641\n",
            "Epoch 8/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 1.4418\n",
            "Epoch 9/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 1.3157\n",
            "Epoch 10/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 1.1900\n",
            "Epoch 11/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 1.0691\n",
            "Epoch 12/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.9558\n",
            "Epoch 13/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.8528\n",
            "Epoch 14/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.7609\n",
            "Epoch 15/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.6802\n",
            "Epoch 16/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.6100\n",
            "Epoch 17/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.5494\n",
            "Epoch 18/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.4972\n",
            "Epoch 19/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.4523\n",
            "Epoch 20/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.4135\n",
            "Epoch 21/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.3800\n",
            "Epoch 22/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.3508\n",
            "Epoch 23/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.3255\n",
            "Epoch 24/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.3032\n",
            "Epoch 25/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.2836\n",
            "Epoch 26/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.2662\n",
            "Epoch 27/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.2508\n",
            "Epoch 28/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.2371\n",
            "Epoch 29/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.2246\n",
            "Epoch 30/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.2134\n",
            "Epoch 31/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.2032\n",
            "Epoch 32/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1940\n",
            "Epoch 33/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1856\n",
            "Epoch 34/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1779\n",
            "Epoch 35/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1708\n",
            "Epoch 36/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1643\n",
            "Epoch 37/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1582\n",
            "Epoch 38/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1526\n",
            "Epoch 39/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1473\n",
            "Epoch 40/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1424\n",
            "Epoch 41/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1378\n",
            "Epoch 42/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1335\n",
            "Epoch 43/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1295\n",
            "Epoch 44/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1257\n",
            "Epoch 45/50\n",
            "45/45 [==============================] - 0s 1ms/step - loss: 0.1221\n",
            "Epoch 46/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1187\n",
            "Epoch 47/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1154\n",
            "Epoch 48/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1124\n",
            "Epoch 49/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1095\n",
            "Epoch 50/50\n",
            "45/45 [==============================] - 0s 2ms/step - loss: 0.1068\n",
            "0.039780253791395846\n",
            "0.032832714329928826\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Je1TZV8S79ZM"
      },
      "source": [
        "Some things to notice in the example above:\n",
        "\n",
        "*   The function [zero_one_loss](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.zero_one_loss.html) was used to evaluate the models.\n",
        "*   10-fold cross-validation was used to evaluate both models. For this, the [KFold](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.KFold.html) class of Scikit-Learn was used.\n",
        "* A batch size of 32 was used. This way, in each training step, the network is provided a batch of 32 training examples.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jN60OrC3-E7u"
      },
      "source": [
        "# Example: ANN trained on the MNIST dataset\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qu4IbIR7cWVm"
      },
      "source": [
        "The following example was taken from https://github.com/fchollet/keras/blob/master/examples/mnist_mlp.py. The [MNIST dataset](https://en.wikipedia.org/wiki/MNIST_database) is used."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MweZkvhB-VvW",
        "outputId": "f602f24d-8f8a-4cda-8af6-b6d81a18093a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "'''Trains a simple deep NN on the MNIST dataset.\n",
        "Gets to 98.40% test accuracy after 20 epochs\n",
        "(there is *a lot* of margin for parameter tuning).\n",
        "2 seconds per epoch on a K520 GPU.\n",
        "'''\n",
        "\n",
        "from __future__ import print_function\n",
        "\n",
        "import keras\n",
        "from keras.datasets import mnist\n",
        "from keras.models import Sequential\n",
        "from keras.layers import Dense, Dropout\n",
        "from keras.optimizers import RMSprop\n",
        "\n",
        "batch_size = 128\n",
        "num_classes = 10\n",
        "epochs = 20\n",
        "\n",
        "# the data, split between train and test sets\n",
        "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
        "\n",
        "x_train = x_train.reshape(60000, 784)\n",
        "x_test = x_test.reshape(10000, 784)\n",
        "x_train = x_train.astype('float32')\n",
        "x_test = x_test.astype('float32')\n",
        "x_train /= 255\n",
        "x_test /= 255\n",
        "print(x_train.shape[0], 'train samples')\n",
        "print(x_test.shape[0], 'test samples')\n",
        "\n",
        "# convert class vectors to binary class matrices\n",
        "y_train = keras.utils.to_categorical(y_train, num_classes)\n",
        "y_test = keras.utils.to_categorical(y_test, num_classes)\n",
        "\n",
        "model = Sequential()\n",
        "\n",
        "model.add(Dense(512, activation='relu', input_shape=(784,)))\n",
        "model.add(Dropout(0.2))\n",
        "\n",
        "model.add(Dense(512, activation='relu'))\n",
        "model.add(Dropout(0.2))\n",
        "\n",
        "model.add(Dense(num_classes, activation='softmax'))\n",
        "\n",
        "model.summary()\n",
        "\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=RMSprop(),\n",
        "              metrics=['accuracy'])\n",
        "\n",
        "history = model.fit(x_train, y_train,\n",
        "                    batch_size=batch_size,\n",
        "                    epochs=epochs,\n",
        "                    verbose=1,\n",
        "                    validation_data=(x_test, y_test))\n",
        "\n",
        "score = model.evaluate(x_test, y_test, verbose=0)\n",
        "print('Test loss:', score[0])\n",
        "print('Test accuracy:', score[1])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "60000 train samples\n",
            "10000 test samples\n",
            "Model: \"sequential_36\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "dense_70 (Dense)             (None, 512)               401920    \n",
            "_________________________________________________________________\n",
            "dropout_2 (Dropout)          (None, 512)               0         \n",
            "_________________________________________________________________\n",
            "dense_71 (Dense)             (None, 512)               262656    \n",
            "_________________________________________________________________\n",
            "dropout_3 (Dropout)          (None, 512)               0         \n",
            "_________________________________________________________________\n",
            "dense_72 (Dense)             (None, 10)                5130      \n",
            "=================================================================\n",
            "Total params: 669,706\n",
            "Trainable params: 669,706\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n",
            "Epoch 1/20\n",
            "469/469 [==============================] - 10s 21ms/step - loss: 0.2479 - accuracy: 0.9234 - val_loss: 0.1148 - val_accuracy: 0.9643\n",
            "Epoch 2/20\n",
            "469/469 [==============================] - 10s 20ms/step - loss: 0.1023 - accuracy: 0.9688 - val_loss: 0.0760 - val_accuracy: 0.9764\n",
            "Epoch 3/20\n",
            "469/469 [==============================] - 10s 20ms/step - loss: 0.0758 - accuracy: 0.9772 - val_loss: 0.0711 - val_accuracy: 0.9778\n",
            "Epoch 4/20\n",
            "469/469 [==============================] - 10s 21ms/step - loss: 0.0584 - accuracy: 0.9820 - val_loss: 0.0689 - val_accuracy: 0.9803\n",
            "Epoch 5/20\n",
            "469/469 [==============================] - 9s 20ms/step - loss: 0.0510 - accuracy: 0.9840 - val_loss: 0.0919 - val_accuracy: 0.9760\n",
            "Epoch 6/20\n",
            "469/469 [==============================] - 10s 20ms/step - loss: 0.0418 - accuracy: 0.9876 - val_loss: 0.0831 - val_accuracy: 0.9797\n",
            "Epoch 7/20\n",
            "469/469 [==============================] - 10s 20ms/step - loss: 0.0386 - accuracy: 0.9887 - val_loss: 0.0714 - val_accuracy: 0.9827\n",
            "Epoch 8/20\n",
            "469/469 [==============================] - 10s 21ms/step - loss: 0.0345 - accuracy: 0.9898 - val_loss: 0.0855 - val_accuracy: 0.9814\n",
            "Epoch 9/20\n",
            "469/469 [==============================] - 10s 20ms/step - loss: 0.0297 - accuracy: 0.9910 - val_loss: 0.0810 - val_accuracy: 0.9828\n",
            "Epoch 10/20\n",
            "469/469 [==============================] - 10s 21ms/step - loss: 0.0291 - accuracy: 0.9913 - val_loss: 0.0885 - val_accuracy: 0.9824\n",
            "Epoch 11/20\n",
            "469/469 [==============================] - 11s 24ms/step - loss: 0.0273 - accuracy: 0.9920 - val_loss: 0.0838 - val_accuracy: 0.9831\n",
            "Epoch 12/20\n",
            "469/469 [==============================] - 12s 25ms/step - loss: 0.0250 - accuracy: 0.9927 - val_loss: 0.1094 - val_accuracy: 0.9806\n",
            "Epoch 13/20\n",
            "469/469 [==============================] - 10s 22ms/step - loss: 0.0231 - accuracy: 0.9936 - val_loss: 0.1157 - val_accuracy: 0.9832\n",
            "Epoch 14/20\n",
            "469/469 [==============================] - 9s 20ms/step - loss: 0.0231 - accuracy: 0.9937 - val_loss: 0.0998 - val_accuracy: 0.9826\n",
            "Epoch 15/20\n",
            "469/469 [==============================] - 9s 20ms/step - loss: 0.0197 - accuracy: 0.9942 - val_loss: 0.1039 - val_accuracy: 0.9838\n",
            "Epoch 16/20\n",
            "469/469 [==============================] - 10s 20ms/step - loss: 0.0217 - accuracy: 0.9943 - val_loss: 0.1073 - val_accuracy: 0.9823\n",
            "Epoch 17/20\n",
            "469/469 [==============================] - 10s 20ms/step - loss: 0.0195 - accuracy: 0.9950 - val_loss: 0.1171 - val_accuracy: 0.9840\n",
            "Epoch 18/20\n",
            "469/469 [==============================] - 9s 20ms/step - loss: 0.0175 - accuracy: 0.9950 - val_loss: 0.1080 - val_accuracy: 0.9841\n",
            "Epoch 19/20\n",
            "469/469 [==============================] - 10s 20ms/step - loss: 0.0192 - accuracy: 0.9952 - val_loss: 0.1123 - val_accuracy: 0.9825\n",
            "Epoch 20/20\n",
            "469/469 [==============================] - 10s 20ms/step - loss: 0.0167 - accuracy: 0.9957 - val_loss: 0.1208 - val_accuracy: 0.9829\n",
            "Test loss: 0.12075675278902054\n",
            "Test accuracy: 0.9829000234603882\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a0G8yt1rB-K6",
        "outputId": "944de588-54cb-42c9-a1b4-8200bdd9f2f4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        }
      },
      "source": [
        "import keras\n",
        "from matplotlib import pyplot as plt\n",
        "\n",
        "plt.plot(history.history['accuracy'])\n",
        "plt.plot(history.history['val_accuracy'])\n",
        "plt.title('model accuracy')\n",
        "plt.ylabel('accuracy')\n",
        "plt.xlabel('epoch')\n",
        "plt.legend(['train', 'val'], loc='upper left')\n",
        "plt.show()"
      ],
      "execution_count": 88,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU9bn48c+TPYGEBIJAFhYVFVAEjbhvtXrdUWzdbbW32la91ba2V2971Z+tV3trvV1cWmtpta6UVqUWF1QQraiArMoqApOEJZAEsm/z/P74noRhGMJAcjLJzPN+vYac5XtmnhmS88z5bkdUFWOMMSZcUqwDMMYY0ztZgjDGGBORJQhjjDERWYIwxhgTkSUIY4wxEVmCMMYYE5ElCGMAEfmziPwsyrLrReTLfsdkTKxZgjDGGBORJQhj4oiIpMQ6BhM/LEGYPsOr2vmhiCwVkToR+aOIDBGR10SkRkTeEpG8kPIXi8inIlItInNEZEzIvoki8ol33ItARthrXSgii71jPxCR8VHGeIGILBKRnSISEJF7w/af4j1ftbf/em97poj8UkQ2iMgOEXnf23aGiJRG+By+7C3fKyLTReQZEdkJXC8ik0Rknvcam0TkERFJCzl+nIjMEpFKEdkiIv8lIkNFpF5EBoWUO0ZEKkQkNZr3buKPJQjT11wGnA0cBlwEvAb8FzAY9/v8XQAROQx4Hrjd2zcT+IeIpHkny5eBvwADgb96z4t37ERgKvAtYBDwe2CGiKRHEV8d8DUgF7gA+I6IXOI97wgv3t96MU0AFnvHPQQcC5zkxfQjIBjlZzIZmO695rNAG/A9IB84ETgLuNmLIRt4C3gdKAAOBd5W1c3AHODykOe9DnhBVVuijMPEGUsQpq/5rapuUdUy4D3gI1VdpKqNwEvARK/cFcA/VXWWd4J7CMjEnYBPAFKBX6lqi6pOB+aHvMZNwO9V9SNVbVPVp4Am77hOqeocVV2mqkFVXYpLUqd7u68G3lLV573X3a6qi0UkCfgGcJuqlnmv+YGqNkX5mcxT1Ze912xQ1YWq+qGqtqrqelyCa4/hQmCzqv5SVRtVtUZVP/L2PQVcCyAiycBVuCRqEpQlCNPXbAlZboiw3t9bLgA2tO9Q1SAQAAq9fWW6+0yVG0KWRwA/8KpoqkWkGij2juuUiBwvIrO9qpkdwLdx3+TxnuPzCIfl46q4Iu2LRiAshsNE5FUR2exVO/1PFDEAvAKMFZFRuKu0Har68QHGZOKAJQgTr8pxJ3oARERwJ8cyYBNQ6G1rNzxkOQDcr6q5IY8sVX0+itd9DpgBFKvqAOB3QPvrBIBDIhyzDWjcy746ICvkfSTjqqdChU/J/DiwEhitqjm4KrjQGA6OFLh3FTYNdxVxHXb1kPAsQZh4NQ24QETO8hpZf4CrJvoAmAe0At8VkVQRmQJMCjn2D8C3vasBEZF+XuNzdhSvmw1UqmqjiEzCVSu1exb4sohcLiIpIjJIRCZ4VzdTgYdFpEBEkkXkRK/NYzWQ4b1+KvATYF9tIdnATqBWRI4AvhOy71VgmIjcLiLpIpItIseH7H8auB64GEsQCc8ShIlLqroK9034t7hv6BcBF6lqs6o2A1NwJ8JKXHvF30OOXQDcCDwCVAFrvbLRuBm4T0RqgLtxiar9eTcC5+OSVSWugfpob/cdwDJcW0gl8HMgSVV3eM/5JO7qpw7YrVdTBHfgElMNLtm9GBJDDa766CJgM7AGODNk/79wjeOfqGpotZtJQGI3DDLGhBKRd4DnVPXJWMdiYssShDGmg4gcB8zCtaHUxDoeE1tWxWSMAUBEnsKNkbjdkoMBu4IwxhizF3YFYYwxJqK4mdgrPz9fR44cGeswjDGmT1m4cOE2VQ0fWwPEUYIYOXIkCxYsiHUYxhjTp4jIXrszWxWTMcaYiCxBGGOMicgShDHGmIjipg0ikpaWFkpLS2lsbIx1KL7LyMigqKiI1FS7t4sxpnvEdYIoLS0lOzubkSNHsvvEnfFFVdm+fTulpaWMGjUq1uEYY+JEXFcxNTY2MmjQoLhODgAiwqBBgxLiSskY03PiOkEAcZ8c2iXK+zTG9Jy4rmIyxph40dTaRnV9C1X1zVTVtVBd30yVt56XlcbVxw/f95PsJ0sQPquurua5557j5ptv3q/jzj//fJ577jlyc3N9iswY0xPagkpjSxsNLW00djyCIduC1DS2UFXfftJv3rVc18KOBpcE6pvb9voaxwzPtQTRF1VXV/PYY4/tkSBaW1tJSdn7xz9z5ky/QzMmoagqOxpaCFQ2sLGynkBVvftZWU9dUyvgqmrbK2tFoGNNiLhdxD1a2pSmkBN++8m/qSVIc1sw6hiTBAZkppKXlUZuVirDBmQwZlgOeVmp5PVL69iXl5VKblYaef3cekZqcjd9SruzBOGzO++8k88//5wJEyaQmppKRkYGeXl5rFy5ktWrV3PJJZcQCARobGzktttu46abbgJ2TR1SW1vLeeedxymnnMIHH3xAYWEhr7zyCpmZmTF+Z8b0Po0tbZRVuwRQWtmeAHYlhJrG1t3K52alMnxgFgMyU1EF9W7vreoe4LbtWgYNdmzt2J6UJORmpTEsNZmM1CQy05JJT0kmMy2ZjJRkMtOSyEhN7nhktpfz1vulp5CXlUpORipJSb2nPTFhEsT/+8enfFa+s1ufc2xBDvdcNK7TMg8++CDLly9n8eLFzJkzhwsuuIDly5d3dEedOnUqAwcOpKGhgeOOO47LLruMQYMG7fYca9as4fnnn+cPf/gDl19+OX/729+49tpru/W9GOMnVaW2qZVttc1sq22ioqaJbbVNbKtpoqK2mcaWNlTd6Xm3k7G2n7Ldht1O1iHrVfXNBCob2Lxz9558aSlJFOdlUjwwi5KReRTnZVE8MIvigW5bToaNG+pMwiSI3mLSpEm7jVX4zW9+w0svvQRAIBBgzZo1eySIUaNGMWHCBACOPfZY1q9f32PxmsSlqrQGlbag+9naFvR+Kq3BIG1BpaXN7d/Z2LLHSb9j3UsITa17VrWIwKB+aWSlpXSsC7t65Yn3z67qHfH2e+sIIpCTkcrJh+ZTPDCT4QNdEhg+MIvB/dN71TfyviZhEsS+vun3lH79+nUsz5kzh7feeot58+aRlZXFGWecEXEsQ3p6esdycnIyDQ0NPRKriS+qyva6ZtZV1PF5RS3rKmpZV1HHum117GxooaXNO+l7SaEteGA3E2s/6ef3T2dwdjqj8vuR33/Xen7/9I7lgf3SSLYTeK+VMAkiVrKzs6mpiXz3xh07dpCXl0dWVhYrV67kww8/7OHoTDxqbg2yYXsdn3ckgjrWbavl86217Aypg09LSeLg/H6MGZbNwH5ppCQlkZwkpCQLKUlCSlISKUlCcrKQuts+tz0lWdy2pCSyM1LciT87jYFZaaQkx/0Qq4RgCcJngwYN4uSTT+bII48kMzOTIUOGdOw799xz+d3vfseYMWM4/PDDOeGEE2IYqelLmlrbKK9uJOA1vq7f5hLCuopaNlbWE/rlf0hOOgfn9+eiows4ZHB/Dh7cj0MG96cgN9O+vZtOxc09qUtKSjT8hkErVqxgzJgxMYqo5yXa+41nbUFl044GApUNBKrqKa1qoNRLBoHKBrbUNBL6p5ueksSo/H67JYCDB/djVH4/sq0h1nRCRBaqakmkfXYFYUwMtLYF2VLTRHl1A2VVDZR6J/72ZFBe3UBryGWACAzLyaBoYFZHY2xxXhZFXg+doTkZ1hhrup0lCGN8UN/c6k7+1Y2UVTVQVl1Pecey644Z3gic3z+d4oGZTCjO5cLxw1x3zDzXJXPYgEzSUqxe3/QsSxDGRKCqNLYEqW9upb7ZjYqtb26jobmNhha3rX29vrmNrTWNXkJwVwRV9S27PV9ykjA0J4PC3EwmjRpIYW4mBbmZFOZlUpibQWFuFplp/oyGNeZAWYIwCUdV2bC9nmVlO1hWtoPlZTvYVttEfbObJ6c9IexP81xWWjKF3gl/fFGuW+5IAJkclJ1uPXtMn+NrghCRc4FfA8nAk6r6YNj+EcBUYDBQCVyrqqXevp8DF3hFf6qqL/oZq4lPqkppVQNLS3ewtKya5WU7WFa6o6O7Z1pKEmOG5TAqvx9ZaSlkpiWTlZpMVloymWkp3k9vPbV9OaVjPctbz0hNsinXTdzxLUGISDLwKHA2UArMF5EZqvpZSLGHgKdV9SkR+RLwAHCdiFwAHANMANKBOSLymqp271wZJq6oKmXVDSwv28HS0h0dVwjVXnVParIwZlgOFx5dwPjCARxVNIDDhmSTat/sjYnIzyuIScBaVV0HICIvAJOB0AQxFvi+tzwbeDlk+1xVbQVaRWQpcC4wzcd4e4X+/ftTW1sb6zB6PVVlY2U9n5bv5LPynR3JoLKuGYCUJOHwodmcd+RQjiwcwPjCXA4b2p/0FKvnNyZafiaIQiAQsl4KHB9WZgkwBVcNdSmQLSKDvO33iMgvgSzgTHZPLACIyE3ATQDDh3f/XOimd2hpC7JmSy2flu/gs007+bR8JyvKd1LjTdGcnCSMPqg/Xx5zEEcV5XJU4QCOGJrt2xTIxiSKWDdS3wE8IiLXA3OBMqBNVd8UkeOAD4AKYB6wx90yVPUJ4AlwA+V6Kuj9ceedd1JcXMwtt9wCwL333ktKSgqzZ8+mqqqKlpYWfvaznzF58uQYR9o71DW1ssJLAu0JYfXm2o459TNTkxkzLJvJEwsYVzCAcQU5HDbEkoExfvAzQZQBxSHrRd62DqpajruCQET6A5eparW3737gfm/fc8DqLkXz2p2weVmXnmIPQ4+C8x7stMgVV1zB7bff3pEgpk2bxhtvvMF3v/tdcnJy2LZtGyeccAIXX3xxwjVybqtt6kgE7VcFX2yv6+g9NLBfGuMKcrjhlJGMHZbDuIIBjMrvZ9NDGNND/EwQ84HRIjIKlxiuBK4OLSAi+UClqgaBu3A9mtobuHNVdbuIjAfGA2/6GKtvJk6cyNatWykvL6eiooK8vDyGDh3K9773PebOnUtSUhJlZWVs2bKFoUOHxjpcX6gqgcqGjkTQfmWwZWdTR5mivEzGFeRwycRCxhXkMLYgh6E5GQmXNI3pTXxLEKraKiK3Am/gurlOVdVPReQ+YIGqzgDOAB4QEcVVMd3iHZ4KvOedHHbiur+2hr/GftnHN30/ffWrX2X69Ols3ryZK664gmeffZaKigoWLlxIamoqI0eOjDjNd1/U0hZk7dbaPa4MQtsLDh3cn5MPyWdsgbsqGFuQw4BMmy/ImN7G1zYIVZ0JzAzbdnfI8nRgeoTjGnE9meLCFVdcwY033si2bdt49913mTZtGgcddBCpqanMnj2bDRs2xDrEA9baFmTGknI+WlfJZ5t2smpLDc3ejWEyUt0Yg/b2grHDcjjcGo+N6TNi3UidEMaNG0dNTQ2FhYUMGzaMa665hosuuoijjjqKkpISjjjiiFiHeEA+2VjFT15azmebdpKXlcq4ggHccNJI78ogh1H5/a29wJg+zBJED1m2bFcDeX5+PvPmzYtYri+Mgaisa+Z/X1/JC/MDDMlJ55GrJ3LBUcOsvcCYOGMJwkQtGFReXBDg56+vpKaxlRtPHcVtXz6M/un2a2RMPLK/bBOV5WU7+MnLy1kcqGbSyIH89JIjOXxodqzDMsb4KO4ThKomRNWHX3cG3NHQwsNvruIvH25gYL80Hr78aC6dWJgQn6kxiS6uE0RGRgbbt29n0KBBcX1CU1W2b99ORkZGtz7nS4vK+J+ZK6isa+a6E0bw/XMOt+6oxiSQuE4QRUVFlJaWUlFREetQfJeRkUFRUVG3PNeqzTX89yvL+fiLSiYU5/LnGyZxZOGAbnluY0zfEdcJIjU1lVGjRsU6jD6jtqmVX7+1mqn/Wk92RgoPTjmKy0uK7V7HxiSouE4QJjqqyj+XbeJnr65g885GrjyumB+dewQD+6XFOjRjTAxZgkhgdU2tvLViC89/vJEP11UyriCHx649hmOG58U6NJNI2lpBBJJ6eIR9sA3KF8GaWbAjAMFWaGtxP0MfbS2ubLAVgu3723Yvmz0MDvmSexQeC8nxcWqNj3dhotbSFmTu6gpeWVzOrM+20NDSRsGADP7fxeO49oQRNvLZ+CvYBhWroPwTd3IuXwSbl0NqBow4BUadCqNOg8FjIMmHO/3VV8Ln77iksHYW1G8HSXIn+KQUSE51P5NSXcIK3ZaS5m1Pcfs6yqbAttXw7s/h3QchPce9h0POdAlj4MHd/z56iCWIBBAMKgs2VPHK4jJmLttEVX0LuVmpTDmmkMkTCikZkWftDLHS1gq1W6BfPqSk9/zrB9tg7VvuxJk9FHIK3M/0HPetvkvPHYTKdV4i8BLCpiXQUu/2p2VDwQSYdCM07YQv5sKqf7p9Wfkw8hR3oh11Ggw69MDiUYUty2HNmy4pBD4CDULmQBh9Now+x53EswZ27b2C+wy/mOsS0OfvwMpX3fa8kbuuLkadBhl9p8OH+NV/vqeVlJToggULYh1Gr7Ji005eWVzOP5aUU1bdQGZqMmePHcLkCQWcOnowaSm95F7MwaD7449FV+SmGihd4KoLMvMgM9f9zBjgviF2VXMd7Ch1VRjVAfdzR6m3XAo7y0Db3Anx2Ouh5BswoLDrr7svTTWw6Fn46HGoWr/n/tQslyiyvYSRPdR9yw5NItnDIDXTlVeF6o27J4PyJdC0w+1PyYRh46FgIhQc434OOnTPq4TqjfDFe7D+PXey3endQiZ7GIz0ri5GnepOup29t3Xvwpo3YM1bUFPutg+b4BLC6HOg8Bh/q7RUYfvnu5LF+veguRYkGYpKdiWMgmP2XR3V2gQN1dBYDQ1Vbrmhavf1/gfBaXccUKgislBVSyLuswQRXwKV9cxYUs6MxeWs2lJDcpJw2uh8Jk8o5OyxQ+jXm6bFaNwBHz4O8x5zJ+PiSe6Pp2iS+wNO69f9r1lbARvnuceGf7mbSGkwctm0bC9h5EJGblgCCVnPyHXfgDtO/IFdCaGhcvfnlGR3gh1QDAOKILfYnfw+fwdWveaqO8ZcBMd/C4af2P1JszoAH/8eFj7tTt7FJ8CJN8OQI6FmM9Rs8h7e8s5Nu7a1RpiSPmOAi7+uwlXXgKuGGXrk7slg8BH7Xy+v6q5Avpi7K2HUeV3Wc4fDyNN2JYzmOlj9hrtS2PCBaytIz3HVPKPPgUO/7JJarLQ2Q+l8WDfb/V+XfQIopA/YlfAaq72Tf1gCaL/i2puMAe7/8ZppBxSaJYg4V1nXzD+XlvPy4nIWbqgCoGREHpMnFnL+kUMZ1D8GVRedaap1J6l//cb9ERx+gTvRBj6G7WtcGUmGIeOg6DgvcRzn6nL354TZ/q22PRlsmLfr+VMyoLAERpwEw0+A9OywP8wI39JC19uaI79maj930h9QFJIEhu9azx629xNl1XqY/yR88rRLnkOPgknfgqO+suub+oEqXQjzHoHPXnHr4y6BE26BomOjO17VxRSeQGo2w85ylyQLvYQwZJw/1WWqrv3ii7nwxbuw/n33/xFq8JhdVUfDT+ieq0A/1Fe69/D5O/D5bKjbtuvLR/gXkM6+oGQM6PKVkCWIODZz2Sbu+OsS6pvbOHyIu1fzReMLKB6YFevQ9tTS4E6A7/8K6rfB6H+DM//L1UO3q690VT6l86H0Y3dia65x+7IGuUTRnjQKjoH0/ruODQZh26pdyWDjvF1VFOkD3AljxIkw/CT3mgd6ElN17yU0YaRnuwSQmdf1b/3NdbB0Gnz8BGz9zNWXH/M1OO6bLvlEq63V1YN/+Jire08fAMd+zSWd/Xme3ioYhC3LXKJIyXCJIXd4rKPqcyxBxKFgUPm/t1bz23fWMnF4Lv9z6VGMGZYT67Aia22ChU/Bew+5BtmDz4QzfwzFx+372GAbVKx0CSPgJY1t3u3JJcl9Wy08Fmq3uoTQ4K6g6D90VzIYcSIcNLbnu1F2lao7+X38e1jpNd4ecYE7wY88Ze+JqHEnLPoLfPQ7dwWVNxKO/w5MvMYlMmNCWIKIMzWNLXzvxcW8tWIrl5cU8dNLjiQ9pRee/NpaYNEzMPch2FkKI052iWHkyV173vpKKFvoJY2PXX1uv0FeMvASQt6o2DR6+6V6I8z/I3zylEuCB41zvX/GXwFp3tVi1QZ31bHwKXfVNfwk175w+Pl9LzmaHhOzBCEi5wK/xt2T+klVfTBs/whgKjAYqMTde7rU2/e/wAVAEjALuE07CTZREsQX2+q48ekFfLGtjrsvHMvXThzR+yYibGuFpS+6fuHVG1yV0Jk/hoPPiK+Tdiy0NMCy6e6qYvMyVxc98VrXQL5iBiAw7lKXGAqjbF8wCa2zBOFblxYRSQYeBc4GSoH5IjJDVT8LKfYQ8LSqPiUiXwIeAK4TkZOAk4HxXrn3gdOBOX7F2xfMWbWV7z6/iOQk4S//PomTDsmHuu2w+jXX42XQIbENMNgGy//uBgttXwvDjobzH3J1w5YYukdqJhxznUsKGz901UgfPu7aYk76D5h0k2sQN6Yb+NnncRKwVlXXAYjIC8BkIDRBjAW+7y3PBl72lhXIANIAAVKBLT7G2qupKk/MXcfPX1/JYUOy+cPXSihuWguv3OO+TbY2ut4zFzwER1/V8yfjYBBW/gNmPwAVK1z1xxXPuvpySwz+EHFVaSNOdFVuKRm7qpqM6SZ+JohCIBCyXgocH1ZmCTAFVw11KZAtIoNUdZ6IzAY24RLEI6q6IvwFROQm4CaA4cPjs/dCQ3Mb//m3pcxYUs5FRw7moaMCpL88xTXIpma5hHDkZa465+XvwLo5cMEve64xsmwhvPo9N0I2/zD4ylQYe6k/0ySYyLpjFLAxEcR61NQdwCMicj0wFygD2kTkUGAM0H6tPEtETlXV90IPVtUngCfAtUH0WNQ9pKy6gW/9ZQGbykt54YilHL/lZWRtueuVcs79rldKpjex3oiT4L1fwpwHXOPtV/60e/fR7tZUA+/8DD76vRuAdMnvYPzl1hhqTBzxM0GUAaGdrYu8bR1UtRx3BYGI9AcuU9VqEbkR+FBVa719rwEnArsliHj28ReV/OaZaXyzdSYXZ35A0voW1z30wofdIKDwE3FSMpz+I9f98W/fhCe/DOf8FI7/dvdX86x6Df75AzdA6rh/h7Pu7lPzyxhjouNngpgPjBaRUbjEcCVwdWgBEckHKlU1CNyF69EEsBG4UUQewFUxnQ78ysdYe4+2Ft6b8Uf6LXqSZ5LWEEzNImni9a7xcfBh+z5+xEnw7ffhlVvg9TtdldPkx1w30K7auQle+5HrLXPQWPjqn92ANWNMXPItQahqq4jcCryB6+Y6VVU/FZH7gAWqOgM4A3hARBRXxXSLd/h04EvAMlyD9euq+g+/Yu0VarfS+vFU6j94glNbt7M5rYCG0+8n87jr9v/bedZAuPI51yf+zZ/A706Gy550VxcHIhiEhX+Ct+51g96+9N9w0nfd9MfGmLhlA+ViRRW2rYGNH8C6OeiKV5FgC3PajqZi7PVMufzrJCd3Q33+piUw/Rtu0rPTfgSn/XD/Jk3bugL+cZubqmHUaXDhr2LfndYY021iMg7ChGlrdfPGbJjnksKGeW4+IqAlI5+/yzn8qe0svnPZuXx1QjdO9zzsaLjpXZj5Qzc+Yf17MOUP+55SuqXRTY3x/q9cj6hLHo9NF1pjTMxYgvBLS6PrArrxAzf9cGD+rknncke4wWMjTmJj9tGc+5dy8rLS+f03juXIQh8ae9P7w6WPw8Gnw6vfd1VOkx+DI86PXP6LufCP26Hycxh/Jfzb/e6GNsaYhGIJors07oCNH+26Oij/ZNeU0AeNg6OvcKOdR5zk7geAu/3nfzz+AWkpyfztOycxdECGvzEefaWb4nr6DfDCVa6H09n37ZrVtL4S3vxvWPyM60p73UvupibGmIRkCaKrmuvh+Svdt27U3Z+2YKI7+Y44CYqP3+tApkfeWcuS0h08ds0x/ieHdvmHwjffgln3uLuJbfjAjZko/wRev8tNBHfK91x7hY3MNSahWYLoqk+edjf+OPk2OOQsd0e0KO6EtmhjFY/MXsuUiYWcf9SwHgg0REo6nPega3R+5WZ4dJK77WVhCXztFXc3MGNMwrME0RWtTfCvX7tprM++L+rD6ptb+f60JQzNyeDeyeN8DHAfjjgfhv0L3rrH3ebzuH+3kdDGmA6WILpi8XPuhuiXPLpfh/3PzBWs317Hc988gZyMGN8ScUChGyNhjDFhbEa1A9XWAu8/7ObcP/jMqA+bvWorz3y4kW+eMooTD+mG0c3GGOMTSxAHatl0d5ev034Y9diAqrpmfjR9KYcPyeYH5xzuc4DGGNM1VsV0IIJtbubUIUfBYedGdYiq8l8vLaO6vpmnbphERqrV9Rtjeje7gjgQn70C29fAaT+I+urhpUVlvLZ8Mz8453DGFuT4HKAxxnSdJYj9pequHvIPgzEXR3VIaVU997zyKZNGDuTGUw/2OUBjjOkeliD21+rXYctyOPUHUXUJDQaVH0xbQlCVX15+NMlJNpeRMaZvsASxP1Rh7i/cXEpHfiWqQ/74/hd89EUl91w8juKBNjLZGNN3WILYH+tmuwn4Tv1+VFNmr9y8k1+8sYpzxg7hq8cW7bO8Mcb0JpYg9sfchyCn0E17vQ9NrW3c/sJicjJTeGDKUYhNk22M6WMsQURr/b9gw7/cnEvts5924uFZq1m5uYafXzaeQf33Xd4YY3obSxDReu8h6DcYjvnaPot+/EUlT8xdx1WThnPWmCE9EJwxxnQ/XxOEiJwrIqtEZK2I3Blh/wgReVtElorIHBEp8rafKSKLQx6NInKJn7F2qnQhfP4OnPQfkJrZadGaxha+P20xwwdm8ZMLxvRQgMYY0/18SxAikgw8CpwHjAWuEpGxYcUeAp5W1fHAfcADAKo6W1UnqOoE4EtAPfCmX7Hu03sPQWYelHxjn0Xv+8dnlFc38PDlR9Mv3QaqG2P6Lj+vICYBa1V1nao2Ay8Ak8PKjAXe8ZZnR9gP8BXgNVWt9y3SzmxeBqtmwgk3u3szd+L15Zv568JSbj7jUGQlug4AABipSURBVI4dEfkmQcYY01f4mSAKgUDIeqm3LdQSYIq3fCmQLSLhU5xeCTwf6QVE5CYRWSAiCyoqKroh5Aje+yWk58CkmzottrWmkf96aRlHFubw3bNG+xOLMcb0oFg3Ut8BnC4ii4DTgTKgrX2niAwDjgLeiHSwqj6hqiWqWjJ48ODuj65iNXz6Mky6ETJz91pMVbnzb8uoa2rl/y6fQFpKrD9WY4zpOj8rycuA4pD1Im9bB1Utx7uCEJH+wGWqWh1S5HLgJVVt8THOvXv/YdcofcLNnRZ7/uMA76zcyj0XjWX0kM6roYwxpq/w86vufGC0iIwSkTRcVdGM0AIiki8i7THcBUwNe46r2Ev1ku8qv4Cl01zDdL/8vRZraG7jZ//8jJMPHcTXTxzZc/EZY4zPfEsQqtoK3IqrHloBTFPVT0XkPhFpnwb1DGCViKwGhgD3tx8vIiNxVyDv+hVjp/71KzcZ34m3dlpsQ2Ud9c1tXHnccJJsIj5jTBzxtR+mqs4EZoZtuztkeTowfS/HrmfPRu2esaMMFj3rBsXlDOu0aKCyAcAm4jPGxB1rTY3kg98A6qbV2IdApet9W5zX+QA6Y4zpayxBhKvdCgufgvFXQt6IfRYPVNWTlZbMwH5pPRCcMcb0HEsQ4eY9Cm1NcMr3oioeqGygOC/LZms1xsSdqBKEiPxdRC4I6XEUn+orYf6TMG4K5B8a1SGlVfUUD7TqJWNM/In2hP8YcDWwRkQeFJHDfYwpdj76PTTXutuJRkFVCVTWU5RnDdTGmPgTVYJQ1bdU9RrgGGA98JaIfCAiN4hIqp8B9pjGnfDR43DEhTAkfE7ByKrqW6hrbrMeTMaYuBR1lZE3R9L1wDeBRcCvcQljli+R9bT5T0LjDjjtjqgPsR5Mxph4FtU4CBF5CTgc+Atwkapu8na9KCIL/AquxzTXwbxH4NCzoWBi1IcFqrwEYVcQxpg4FO1Aud+o6uxIO1S1pBvjiY2FT0H9djjth/t1mA2SM8bEs2irmMaKSMd0piKSJyKdz2DXV7Q0uoFxI0+F4cfv16GBqnryslLpbzcGMsbEoWgTxI2hs6yqahVwoz8h9bC6Chh48H5fPYBrg7CrB2NMvIr2q2+yiIiqKnTcTjQ+hg7nFsMNM8G9tf1SWtXA2GE5PgRljDGxF+0VxOu4BumzROQs3BTcr/sXVgzs50joYFApq2qgyAbJGWPiVLRXEP8JfAv4jrc+C3jSl4j6iC01jTS3BSm2QXLGmDgVVYJQ1SDwuPcwWA8mY0z8i3YcxGjgAWAskNG+XVUP9imuXs8GyRlj4l20bRB/wl09tAJnAk8Dz/gVVF8QqKpHBAotQRhj4lS0CSJTVd8GRFU3qOq9wAX+hdX7BSobGJKdQXpKcqxDMcYYX0TbSN3kTfW9RkRuBcqA/v6F1fsFbJpvY0yci/YK4jYgC/gucCxwLfD1fR0kIueKyCoRWSsid0bYP0JE3haRpSIyR0SKQvYNF5E3RWSFiHwmIiOjjLVHlFbWWw8mY0xc22eC8AbFXaGqtapaqqo3qOplqvphFMc9CpyHa9y+SkTC59F+CHhaVccD9+Eawts9DfxCVccAk4CtUb8rnzW3Btm0s5Ei68FkjIlj+0wQqtoGnHIAzz0JWKuq61S1GXgBmBxWZizwjrc8u32/l0hSVHWWF0OtqtYfQAy+KK9uQBWGW4IwxsSxaKuYFonIDBG5TkSmtD/2cUwhEAhZL/W2hVoCtD/PpUC2d9+Jw4Bq71ani0TkF94VyW5E5CYRWSAiCyoqKqJ8K13XMc239WAyxsSxaBNEBrAd+BJwkfe4sBte/w7gdBFZBJyOa/xuwzWen+rtPw44GHezot2o6hOqWqKqJYMHD+6GcKJjg+SMMYkg2pHUNxzAc5cBxSHrRd620Octx7uCEJH+wGWqWi0ipcBiVV3n7XsZOAH44wHE0e0CVfWkJgtDcjL2XdgYY/qoaEdS/wnYY7pTVf1GJ4fNB0aLyChcYrgSuDrsefOBSm8qj7uAqSHH5orIYFWtwF259Jo71wUq6ynMzSQ5af8m+DPGmL4k2nEQr4YsZ+DaC8o7O0BVW70xE28AycBUVf1URO4DFqjqDOAM4AERUWAucIt3bJuI3AG8LSICLAT+EP3b8legqsGql4wxcS/aKqa/ha6LyPPA+1EcNxOYGbbt7pDl6cD0vRw7CxgfTXw9rbSynnPGDY11GMYY46toG6nDjQYO6s5A+oq6pla21zXbKGpjTNyLtg2iht3bIDbj7hGRcEqrvB5MNoraGBPnoq1iyvY7kL6iY5pva4MwxsS5qKqYRORSERkQsp4rIpf4F1bvZYPkjDGJIto2iHtUdUf7iqpWA/f4E1LvFqhsICstmYH90mIdijHG+CraBBGpXLRdZONKoMrN4up63xpjTPyKNkEsEJGHReQQ7/EwbmxCwglU2n0gjDGJIdoE8R9AM/AiblbWRrxBbYlEVSmtaqDIejAZYxJAtL2Y6oA9bviTaKrrW6htarUeTMaYhBBtL6ZZIpIbsp4nIm/4F1bvZD2YjDGJJNoqpnyv5xIAqlpFAo6ktmm+jTGJJNoEERSR4e0r3v2h95jdNd51XEFYgjDGJIBou6r+GHhfRN4FBHczn5t8i6qXClTWk5eVSv/0hOzha4xJMNE2Ur8uIiW4pLAIeBlo8DOw3sim+TbGJJJoJ+v7JnAb7q5wi3F3d5uHu5FPwiitrGfMsJxYh2GMMT0i2jaI23D3ht6gqmcCE4Hqzg+JL8GgNwbCBskZYxJEtAmiUVUbAUQkXVVXAof7F1bvs7Wmiea2oE3zbYxJGNG2tpZ64yBeBmaJSBWwwb+wep+NNs23MSbBRNtIfam3eK+IzAYGAK/v6zgRORf4Ne6e1E+q6oNh+0cAU4HBQCVwraqWevvagGVe0Y2qenE0sfql4z4QNkjOGJMg9ru/pqq+G005EUkGHgXOBkqB+SIyQ1U/Cyn2EPC0qj4lIl8CHgCu8/Y1qOqE/Y3PL4GqekSg0BKEMSZBHOg9qaMxCVirqutUtRk3yd/ksDJjgXe85dkR9vcagcoGhmRnkJ6SHOtQjDGmR/iZIAqBQMh6qbct1BJgird8KZAtIoO89QwRWSAiH+7t7nUicpNXZkFFRUV3xr6HQJVN822MSSx+Joho3AGcLiKLgNOBMqDN2zdCVUuAq4Fficgh4Qer6hOqWqKqJYMHD/Y10NLKeuvBZIxJKH7OGVEGFIesF3nbOqhqOd4VhIj0By5rnxRQVcu8n+tEZA5u7MXnPsa7V82tQTbtbKTIejAZYxKIn1cQ84HRIjJKRNKAK4EZoQVEJF9E2mO4C9ejqX068fT2MsDJQGjjdo8qr25A1XowGWMSi28JQlVbgVuBN4AVwDRV/VRE7hOR9i6rZwCrRGQ1MAS439s+Bneb0yW4xusHw3o/9SibxdUYk4h8nZZUVWcCM8O23R2yPB2YHuG4D4Cj/Ixtf9h9IIwxiSjWjdR9QqCqntRkYWhORqxDMcaYHmMJIgqBynoKcjNJTpJYh2KMMT3GEkQUAlUN1sXVGJNwLEFEobTSBskZYxKPJYh9qGtqZXtdM0V2BWGMSTCWIPahtMp6MBljEpMliH2wab6NMYnKEsQ+2CA5Y0yisgSxD4HKBjJTkxnULy3WoRhjTI+yBLEP7dN8i9gYCGNMYrEEsQ8Bm+bbGJOgLEF0QlUprWqw9gdjTEKyBNGJ6voWaptaKbIeTMaYBGQJohPWg8kYk8gsQXSiY5pva4MwxiQgSxCd2HUFYVVMxpjEYwmiE4HKenKzUsnOSI11KMYY0+MsQXTCpvk2xiQyXxOEiJwrIqtEZK2I3Blh/wgReVtElorIHBEpCtufIyKlIvKIn3HujU3zbYxJZL4lCBFJBh4FzgPGAleJyNiwYg8BT6vqeOA+4IGw/T8F5voVY2eCQW8MhF1BGGMSlJ9XEJOAtaq6TlWbgReAyWFlxgLveMuzQ/eLyLHAEOBNH2Pcq601TTS3BSmyLq7GmATlZ4IoBAIh66XetlBLgCne8qVAtogMEpEk4JfAHZ29gIjcJCILRGRBRUVFN4XtdPRgskFyxpgEFetG6juA00VkEXA6UAa0ATcDM1W1tLODVfUJVS1R1ZLBgwd3a2Ad94GwKwhjTIJK8fG5y4DikPUib1sHVS3Hu4IQkf7AZapaLSInAqeKyM1AfyBNRGpVdY+Gbr+0D5IrzLUrCGNMYvIzQcwHRovIKFxiuBK4OrSAiOQDlaoaBO4CpgKo6jUhZa4HSnoyOYCrYhqSk05GanJPvqwxxvQavlUxqWorcCvwBrACmKaqn4rIfSJysVfsDGCViKzGNUjf71c8+8um+TbGJDo/ryBQ1ZnAzLBtd4csTwem7+M5/gz82YfwOlVa1cCkUQN7+mWNMabXiHUjda/U0hZk044G68FkjEloliAiKK9uIKjYGAhjTEKzBBGBTfNtjDGWICKyab6NMcYSRESBynpSkoRhAyxBGGMSlyWICAJVDRTkZpKcJLEOxRhjYsYSRAQBm+bbGGMsQURSWmWD5IwxxhJEmPrmVrbVNtskfcaYhGcJIkxpleviWmSD5IwxCc4SRBib5tsYYxxLEGE6EoS1QRhjEpwliDCBqgYyU5PJ758W61CMMSamLEGECVTWU5SXiYiNgTDGJDZLEGECVQ3W/mCMMViC2I2qUlpZb9N8G2MMliB2s6OhhZqmVruCMMYYLEHsZqPXg6nIejAZY4wliFAd94GweZiMMcbfBCEi54rIKhFZKyJ3Rtg/QkTeFpGlIjJHRIpCtn8iIotF5FMR+bafcbbbdR8Iu4IwxhjfEoSIJAOPAucBY4GrRGRsWLGHgKdVdTxwH/CAt30TcKKqTgCOB+4UkQK/Ym0XqKxnQGYqORmpfr+UMcb0en5eQUwC1qrqOlVtBl4AJoeVGQu84y3Pbt+vqs2q2uRtT/c5zg6ui6tVLxljDPh74i0EAiHrpd62UEuAKd7ypUC2iAwCEJFiEVnqPcfPVbU8/AVE5CYRWSAiCyoqKrocsOviatVLxhgDsW+kvgM4XUQWAacDZUAbgKoGvKqnQ4Gvi8iQ8INV9QlVLVHVksGDB3cpkGBQKbVBcsYY08HPBFEGFIesF3nbOqhquapOUdWJwI+9bdXhZYDlwKk+xsrWmiaa24I2SM4YYzx+Joj5wGgRGSUiacCVwIzQAiKSLyLtMdwFTPW2F4lIprecB5wCrPIx1o4eTEV2BWGMMYCPCUJVW4FbgTeAFcA0Vf1URO4TkYu9YmcAq0RkNTAEuN/bPgb4SESWAO8CD6nqMr9iBZvm2xhjwqX4+eSqOhOYGbbt7pDl6cD0CMfNAsb7GVu49kFydic5Y4xxYt1I3WsEquo5KDudjNTkWIdijDG9giUIT6Cy3nowGWNMCEsQntKqBuvBZIwxISxBAC1tQTbtsDEQxhgTyhIEUF7dQFCtB5MxxoSyBEFIDyabh8kYYzpYgiBkmm+7gjDGmA6WIHA9mJKThGEDMmIdijHG9BqWIHDTfBfkZpCSbB+HMca0szMi3hgIq14yxpjdWIIASqssQRhjTLiETxD1za1sq222O8kZY0yYhE8QDc1tXHx0AeOLcmMdijHG9Cq+zubaFwzqn85vrpoY6zCMMabXSfgrCGOMMZFZgjDGGBORJQhjjDERWYIwxhgTka8JQkTOFZFVIrJWRO6MsH+EiLwtIktFZI6IFHnbJ4jIPBH51Nt3hZ9xGmOM2ZNvCUJEkoFHgfOAscBVIjI2rNhDwNOqOh64D3jA214PfE1VxwHnAr8SEeuHaowxPcjPK4hJwFpVXaeqzcALwOSwMmOBd7zl2e37VXW1qq7xlsuBrcBgH2M1xhgTxs8EUQgEQtZLvW2hlgBTvOVLgWwRGRRaQEQmAWnA5+EvICI3icgCEVlQUVHRbYEbY4yJ/UC5O4BHROR6YC5QBrS17xSRYcBfgK+rajD8YFV9AnjCK1shIhu6EEs+sK0Lx/vN4usai69rLL6u6c3xjdjbDj8TRBlQHLJe5G3r4FUfTQEQkf7AZapa7a3nAP8EfqyqH+7rxVS1S1VQIrJAVUu68hx+svi6xuLrGouva3p7fHvjZxXTfGC0iIwSkTTgSmBGaAERyReR9hjuAqZ629OAl3AN2NN9jNEYY8xe+JYgVLUVuBV4A1gBTFPVT0XkPhG52Ct2BrBKRFYDQ4D7ve2XA6cB14vIYu8xwa9YjTHG7MnXNghVnQnMDNt2d8jydGCPKwRVfQZ4xs/YIniih19vf1l8XWPxdY3F1zW9Pb6IRFVjHYMxxpheyKbaMMYYE5ElCGOMMRElVIKIYm6odBF50dv/kYiM7MHYikVktoh85s1BdVuEMmeIyI6Qhvu7Iz2Xz3GuF5Fl3usviLBfROQ33me4VESO6cHYDg/5bBaLyE4RuT2sTI9+hiIyVUS2isjykG0DRWSWiKzxfubt5dive2XWiMjXezC+X4jISu//76W9TXOzr98FH+O7V0TKQv4Pz9/LsZ3+vfsY34shsa0XkcV7Odb3z6/LVDUhHkAybjT2wbiR2UuAsWFlbgZ+5y1fCbzYg/ENA47xlrOB1RHiOwN4Ncaf43ogv5P95wOvAQKcAHwUw//vzcCIWH6GuN54xwDLQ7b9L3Cnt3wn8PMIxw0E1nk/87zlvB6K7xwgxVv+eaT4ovld8DG+e4E7ovj/7/Tv3a/4wvb/Erg7Vp9fVx+JdAURzdxQk4GnvOXpwFkiIj0RnKpuUtVPvOUaXNfg8KlJ+oLJuPErqm6AY643Ir6nnQV8rqpdGV3fZao6F6gM2xz6e/YUcEmEQ/8NmKWqlapaBczCTVzpe3yq+qa6buoAH+IGucbEXj6/aETz995lncXnnTsuB57v7tftKYmUIKKZG6qjjPcHsgMYRA/zqrYmAh9F2H2iiCwRkddEZFyPBuYo8KaILBSRmyLsj+Zz7glXsvc/zFh/hkNUdZO3vBk3Bihcb/kcv4G7IoxkX78LfrrVqwKbupcqut7w+Z0KbFFv4tEIYvn5RSWREkSfIG7Kkb8Bt6vqzrDdn+CqTI4Gfgu83NPxAaeo6jG4adxvEZHTYhBDp7yR+BcDf42wuzd8hh3U1TX0yr7mIvJjoBV4di9FYvW78DhwCDAB2ISrxumNrqLzq4de/7eUSAlin3NDhZYRkRRgALC9R6Jzr5mKSw7Pqurfw/er6k5VrfWWZwKpIpLfU/F5r1vm/dyKmw5lUliRaD5nv50HfKKqW8J39IbPENjSXu3m/dwaoUxMP0dxE2heCFzjJbE9RPG74AtV3aKqbeom8PzDXl431p9fCm6euRf3ViZWn9/+SKQEsc+5obz19t4iXwHe2dsfR3fz6iv/CKxQ1Yf3UmZoe5uIuGnQk+jZBNZPRLLbl3GNmcvDis0Avub1ZjoB2BFSndJT9vrNLdafoSf09+zrwCsRyrwBnCMieV4VyjneNt+JyLnAj4CLVbV+L2Wi+V3wK77QNq1L9/K60fy9++nLwEpVLY20M5af336JdSt5Tz5wPWxW43o3/Njbdh/uDwEgA1ctsRb4GDi4B2M7BVfVsBRY7D3OB74NfNsrcyvwKa5HxofAST38+R3svfYSL472zzA0RsHdSfBzYBlQ0sMx9sOd8AeEbIvZZ4hLVJuAFlw9+L/j2rXeBtYAbwEDvbIlwJMhx37D+11cC9zQg/GtxdXft/8etvfsKwBmdva70EPx/cX73VqKO+kPC4/PW9/j770n4vO2/7n9dy6kbI9/fl192FQbxhhjIkqkKiZjjDH7wRKEMcaYiCxBGGOMicgShDHGmIgsQRhjjInIEoQxvYA3y+yrsY7DmFCWIIwxxkRkCcKY/SAi14rIx94c/r8XkWQRqRWR/xN3H4+3RWSwV3aCiHwYcl+FPG/7oSLyljdh4Ccicoj39P1FZLp3L4Zne2omYWP2xhKEMVESkTHAFcDJqjoBaAOuwY3eXqCq44B3gXu8Q54G/lNVx+NG/rZvfxZ4VN2EgSfhRuKCm8H3dmAsbqTtyb6/KWM6kRLrAIzpQ84CjgXme1/uM3ET7QXZNSnbM8DfRWQAkKuq73rbnwL+6s2/U6iqLwGoaiOA93wfqzd3j3cXspHA+/6/LWMiswRhTPQEeEpV79pto8h/h5U70PlrmkKW27C/TxNjVsVkTPTeBr4iIgdBx72lR+D+jr7ilbkaeF9VdwBVInKqt/064F11dwssFZFLvOdIF5GsHn0XxkTJvqEYEyVV/UxEfoK7C1gSbgbPW4A6YJK3byuunQLcVN6/8xLAOuAGb/t1wO9F5D7vOb7ag2/DmKjZbK7GdJGI1Kpq/1jHYUx3syomY4wxEdkVhDHGmIjsCsIYY0xEliCMMcZEZAnCGGNMRJYgjDHGRGQJwhhjTET/H9A7wLOlXQmiAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Skb5Mr-H9uxG"
      },
      "source": [
        "# Miscelaneous examples"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XLdsTpt-x1tZ",
        "outputId": "4b7581e1-944d-4b3c-b30b-fe6ee7b85820",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 104
        }
      },
      "source": [
        "import numpy as np \n",
        "\n",
        "a = np.arange(6).reshape(2,3) + 10\n",
        "\n",
        "print(a)\n",
        "\n",
        "print(np.argmax(a))\n",
        "\n",
        "print(np.argmax(a, axis=0))\n",
        "\n",
        "print(np.argmax(a, axis=1))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[10 11 12]\n",
            " [13 14 15]]\n",
            "5\n",
            "[1 1 1]\n",
            "[2 2]\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}