02_neural_networks.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Creative machine learning - Neural networks\n",
    "\n",
    "### Author: Philippe Esling (esling@ircam.fr)\n",
    "\n",
    "In this course we will cover\n",
    "1. A [quick introduction](#intro) on the principles of neural networks\n",
    "2. An implementation for a [single neuron](#neuron) in Numpy and JAX.\n",
    "3. An exercise on [multi-layer perceptron (MLP)](#mlp) through manual derivation.\n",
    "4. An introduction on [using Pytorch](#pytorch) for defining networks\n",
    "5. An exercise on [audio classification](#audio) using an MLP with Pytorch."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"intro\"></a>\n",
    "# Introducing neural networks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this tutorial, we will cover more advanced models known as *neural networks*. The tutorial starts by performing a simple **single neuron** discrimination of two random distributions. We will exhibit the manual implementation using Numpy, and then simplify it with JAX. Then, we will study the typical **XOR problem** by using a more advanced 2-layer **perceptron**. Finally, we generalize the use of neural networks in order to perform classification on a given set of audio files, using the PyTorch library, which will provide simplified implementation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are going to use relatively _low-level_ libraries to perform the first exercises (implementing your own neurons). To observe this idea in simple setups, we are going to use the `numpy` library and also initialize the homemade course library `cml` and style for future plotting and exercise. We also set the random generator to a fixed point with `rng = np.random.RandomState(1)`, to ensure reproducibility"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>\n",
       "        .bk-notebook-logo {\n",
       "            display: block;\n",
       "            width: 20px;\n",
       "            height: 20px;\n",
       "            background-image: url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAUCAYAAACNiR0NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAABx0RVh0U29mdHdhcmUAQWRvYmUgRmlyZXdvcmtzIENTNui8sowAAAOkSURBVDiNjZRtaJVlGMd/1/08zzln5zjP1LWcU9N0NkN8m2CYjpgQYQXqSs0I84OLIC0hkEKoPtiH3gmKoiJDU7QpLgoLjLIQCpEsNJ1vqUOdO7ppbuec5+V+rj4ctwzd8IIbbi6u+8f1539dt3A78eXC7QizUF7gyV1fD1Yqg4JWz84yffhm0qkFqBogB9rM8tZdtwVsPUhWhGcFJngGeWrPzHm5oaMmkfEg1usvLFyc8jLRqDOMru7AyC8saQr7GG7f5fvDeH7Ej8CM66nIF+8yngt6HWaKh7k49Soy9nXurCi1o3qUbS3zWfrYeQDTB/Qj6kX6Ybhw4B+bOYoLKCC9H3Nu/leUTZ1JdRWkkn2ldcCamzrcf47KKXdAJllSlxAOkRgyHsGC/zRday5Qld9DyoM4/q/rUoy/CXh3jzOu3bHUVZeU+DEn8FInkPBFlu3+nW3Nw0mk6vCDiWg8CeJaxEwuHS3+z5RgY+YBR6V1Z1nxSOfoaPa4LASWxxdNp+VWTk7+4vzaou8v8PN+xo+KY2xsw6une2frhw05CTYOmQvsEhjhWjn0bmXPjpE1+kplmmkP3suftwTubK9Vq22qKmrBhpY4jvd5afdRA3wGjFAgcnTK2s4hY0/GPNIb0nErGMCRxWOOX64Z8RAC4oCXdklmEvcL8o0BfkNK4lUg9HTl+oPlQxdNo3Mg4Nv175e/1LDGzZen30MEjRUtmXSfiTVu1kK8W4txyV6BMKlbgk3lMwYCiusNy9fVfvvwMxv8Ynl6vxoByANLTWplvuj/nF9m2+PDtt1eiHPBr1oIfhCChQMBw6Aw0UulqTKZdfVvfG7VcfIqLG9bcldL/+pdWTLxLUy8Qq38heUIjh4XlzZxzQm19lLFlr8vdQ97rjZVOLf8nclzckbcD4wxXMidpX30sFd37Fv/GtwwhzhxGVAprjbg0gCAEeIgwCZyTV2Z1REEW8O4py0wsjeloKoMr6iCY6dP92H6Vw/oTyICIthibxjm/DfN9lVz8IqtqKYLUXfoKVMVQVVJOElGjrnnUt9T9wbgp8AyYKaGlqingHZU/uG2NTZSVqwHQTWkx9hxjkpWDaCg6Ckj5qebgBVbT3V3NNXMSiWSDdGV3hrtzla7J+duwPOToIg42ChPQOQjspnSlp1V+Gjdged7+8UN5CRAV7a5EdFNwCjEaBR27b3W890TE7g24NAP/mMDXRWrGoFPQI9ls/MWO2dWFAar/xcOIImbbpA3zgAAAABJRU5ErkJggg==);\n",
       "        }\n",
       "    </style>\n",
       "    <div>\n",
       "        <a href=\"https://bokeh.org\" target=\"_blank\" class=\"bk-notebook-logo\"></a>\n",
       "        <span id=\"adff00e8-c97e-44db-826e-40f8d52c667d\">Loading BokehJS ...</span>\n",
       "    </div>\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/javascript": "(function(root) {\n  function now() {\n    return new Date();\n  }\n\n  const force = true;\n\n  if (typeof root._bokeh_onload_callbacks === \"undefined\" || force === true) {\n    root._bokeh_onload_callbacks = [];\n    root._bokeh_is_loading = undefined;\n  }\n\nconst JS_MIME_TYPE = 'application/javascript';\n  const HTML_MIME_TYPE = 'text/html';\n  const EXEC_MIME_TYPE = 'application/vnd.bokehjs_exec.v0+json';\n  const CLASS_NAME = 'output_bokeh rendered_html';\n\n  /**\n   * Render data to the DOM node\n   */\n  function render(props, node) {\n    const script = document.createElement(\"script\");\n    node.appendChild(script);\n  }\n\n  /**\n   * Handle when an output is cleared or removed\n   */\n  function handleClearOutput(event, handle) {\n    const cell = handle.cell;\n\n    const id = cell.output_area._bokeh_element_id;\n    const server_id = cell.output_area._bokeh_server_id;\n    // Clean up Bokeh references\n    if (id != null && id in Bokeh.index) {\n      Bokeh.index[id].model.document.clear();\n      delete Bokeh.index[id];\n    }\n\n    if (server_id !== undefined) {\n      // Clean up Bokeh references\n      const cmd_clean = \"from bokeh.io.state import curstate; print(curstate().uuid_to_server['\" + server_id + \"'].get_sessions()[0].document.roots[0]._id)\";\n      cell.notebook.kernel.execute(cmd_clean, {\n        iopub: {\n          output: function(msg) {\n            const id = msg.content.text.trim();\n            if (id in Bokeh.index) {\n              Bokeh.index[id].model.document.clear();\n              delete Bokeh.index[id];\n            }\n          }\n        }\n      });\n      // Destroy server and session\n      const cmd_destroy = \"import bokeh.io.notebook as ion; ion.destroy_server('\" + server_id + \"')\";\n      cell.notebook.kernel.execute(cmd_destroy);\n    }\n  }\n\n  /**\n   * Handle when a new output is added\n   */\n  function handleAddOutput(event, handle) {\n    const output_area = handle.output_area;\n    const output = handle.output;\n\n    // limit handleAddOutput to display_data with EXEC_MIME_TYPE content only\n    if ((output.output_type != \"display_data\") || (!Object.prototype.hasOwnProperty.call(output.data, EXEC_MIME_TYPE))) {\n      return\n    }\n\n    const toinsert = output_area.element.find(\".\" + CLASS_NAME.split(' ')[0]);\n\n    if (output.metadata[EXEC_MIME_TYPE][\"id\"] !== undefined) {\n      toinsert[toinsert.length - 1].firstChild.textContent = output.data[JS_MIME_TYPE];\n      // store reference to embed id on output_area\n      output_area._bokeh_element_id = output.metadata[EXEC_MIME_TYPE][\"id\"];\n    }\n    if (output.metadata[EXEC_MIME_TYPE][\"server_id\"] !== undefined) {\n      const bk_div = document.createElement(\"div\");\n      bk_div.innerHTML = output.data[HTML_MIME_TYPE];\n      const script_attrs = bk_div.children[0].attributes;\n      for (let i = 0; i < script_attrs.length; i++) {\n        toinsert[toinsert.length - 1].firstChild.setAttribute(script_attrs[i].name, script_attrs[i].value);\n        toinsert[toinsert.length - 1].firstChild.textContent = bk_div.children[0].textContent\n      }\n      // store reference to server id on output_area\n      output_area._bokeh_server_id = output.metadata[EXEC_MIME_TYPE][\"server_id\"];\n    }\n  }\n\n  function register_renderer(events, OutputArea) {\n\n    function append_mime(data, metadata, element) {\n      // create a DOM node to render to\n      const toinsert = this.create_output_subarea(\n        metadata,\n        CLASS_NAME,\n        EXEC_MIME_TYPE\n      );\n      this.keyboard_manager.register_events(toinsert);\n      // Render to node\n      const props = {data: data, metadata: metadata[EXEC_MIME_TYPE]};\n      render(props, toinsert[toinsert.length - 1]);\n      element.append(toinsert);\n      return toinsert\n    }\n\n    /* Handle when an output is cleared or removed */\n    events.on('clear_output.CodeCell', handleClearOutput);\n    events.on('delete.Cell', handleClearOutput);\n\n    /* Handle when a new output is added */\n    events.on('output_added.OutputArea', handleAddOutput);\n\n    /**\n     * Register the mime type and append_mime function with output_area\n     */\n    OutputArea.prototype.register_mime_type(EXEC_MIME_TYPE, append_mime, {\n      /* Is output safe? */\n      safe: true,\n      /* Index of renderer in `output_area.display_order` */\n      index: 0\n    });\n  }\n\n  // register the mime type if in Jupyter Notebook environment and previously unregistered\n  if (root.Jupyter !== undefined) {\n    const events = require('base/js/events');\n    const OutputArea = require('notebook/js/outputarea').OutputArea;\n\n    if (OutputArea.prototype.mime_types().indexOf(EXEC_MIME_TYPE) == -1) {\n      register_renderer(events, OutputArea);\n    }\n  }\n  if (typeof (root._bokeh_timeout) === \"undefined\" || force === true) {\n    root._bokeh_timeout = Date.now() + 5000;\n    root._bokeh_failed_load = false;\n  }\n\n  const NB_LOAD_WARNING = {'data': {'text/html':\n     \"<div style='background-color: #fdd'>\\n\"+\n     \"<p>\\n\"+\n     \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n     \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n     \"</p>\\n\"+\n     \"<ul>\\n\"+\n     \"<li>re-rerun `output_notebook()` to attempt to load from CDN again, or</li>\\n\"+\n     \"<li>use INLINE resources instead, as so:</li>\\n\"+\n     \"</ul>\\n\"+\n     \"<code>\\n\"+\n     \"from bokeh.resources import INLINE\\n\"+\n     \"output_notebook(resources=INLINE)\\n\"+\n     \"</code>\\n\"+\n     \"</div>\"}};\n\n  function display_loaded() {\n    const el = document.getElementById(\"adff00e8-c97e-44db-826e-40f8d52c667d\");\n    if (el != null) {\n      el.textContent = \"BokehJS is loading...\";\n    }\n    if (root.Bokeh !== undefined) {\n      if (el != null) {\n        el.textContent = \"BokehJS \" + root.Bokeh.version + \" successfully loaded.\";\n      }\n    } else if (Date.now() < root._bokeh_timeout) {\n      setTimeout(display_loaded, 100)\n    }\n  }\n\n  function run_callbacks() {\n    try {\n      root._bokeh_onload_callbacks.forEach(function(callback) {\n        if (callback != null)\n          callback();\n      });\n    } finally {\n      delete root._bokeh_onload_callbacks\n    }\n    console.debug(\"Bokeh: all callbacks have finished\");\n  }\n\n  function load_libs(css_urls, js_urls, callback) {\n    if (css_urls == null) css_urls = [];\n    if (js_urls == null) js_urls = [];\n\n    root._bokeh_onload_callbacks.push(callback);\n    if (root._bokeh_is_loading > 0) {\n      console.debug(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n      return null;\n    }\n    if (js_urls == null || js_urls.length === 0) {\n      run_callbacks();\n      return null;\n    }\n    console.debug(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n    root._bokeh_is_loading = css_urls.length + js_urls.length;\n\n    function on_load() {\n      root._bokeh_is_loading--;\n      if (root._bokeh_is_loading === 0) {\n        console.debug(\"Bokeh: all BokehJS libraries/stylesheets loaded\");\n        run_callbacks()\n      }\n    }\n\n    function on_error(url) {\n      console.error(\"failed to load \" + url);\n    }\n\n    for (let i = 0; i < css_urls.length; i++) {\n      const url = css_urls[i];\n      const element = document.createElement(\"link\");\n      element.onload = on_load;\n      element.onerror = on_error.bind(null, url);\n      element.rel = \"stylesheet\";\n      element.type = \"text/css\";\n      element.href = url;\n      console.debug(\"Bokeh: injecting link tag for BokehJS stylesheet: \", url);\n      document.body.appendChild(element);\n    }\n\n    for (let i = 0; i < js_urls.length; i++) {\n      const url = js_urls[i];\n      const element = document.createElement('script');\n      element.onload = on_load;\n      element.onerror = on_error.bind(null, url);\n      element.async = false;\n      element.src = url;\n      console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n      document.head.appendChild(element);\n    }\n  };\n\n  function inject_raw_css(css) {\n    const element = document.createElement(\"style\");\n    element.appendChild(document.createTextNode(css));\n    document.body.appendChild(element);\n  }\n\n  const js_urls = [\"https://cdn.bokeh.org/bokeh/release/bokeh-3.2.0.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-gl-3.2.0.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-widgets-3.2.0.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-tables-3.2.0.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-mathjax-3.2.0.min.js\", \"https://unpkg.com/@holoviz/panel@1.1.0/dist/panel.min.js\"];\n  const css_urls = [];\n\n  const inline_js = [    function(Bokeh) {\n      Bokeh.set_log_level(\"info\");\n    },\nfunction(Bokeh) {\n    }\n  ];\n\n  function run_inline_js() {\n    if (root.Bokeh !== undefined || force === true) {\n          for (let i = 0; i < inline_js.length; i++) {\n      inline_js[i].call(root, root.Bokeh);\n    }\nif (force === true) {\n        display_loaded();\n      }} else if (Date.now() < root._bokeh_timeout) {\n      setTimeout(run_inline_js, 100);\n    } else if (!root._bokeh_failed_load) {\n      console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n      root._bokeh_failed_load = true;\n    } else if (force !== true) {\n      const cell = $(document.getElementById(\"adff00e8-c97e-44db-826e-40f8d52c667d\")).parents('.cell').data().cell;\n      cell.output_area.append_execute_result(NB_LOAD_WARNING)\n    }\n  }\n\n  if (root._bokeh_is_loading === 0) {\n    console.debug(\"Bokeh: BokehJS loaded, going straight to plotting\");\n    run_inline_js();\n  } else {\n    load_libs(css_urls, js_urls, function() {\n      console.debug(\"Bokeh: BokehJS plotting callback run at\", now());\n      run_inline_js();\n    });\n  }\n}(window));",
      "application/vnd.bokehjs_load.v0+json": ""
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/javascript": "(function(root) {\n  function now() {\n    return new Date();\n  }\n\n  var force = true;\n  var py_version = '3.2.0'.replace('rc', '-rc.');\n  var is_dev = py_version.indexOf(\"+\") !== -1 || py_version.indexOf(\"-\") !== -1;\n  var reloading = false;\n  var Bokeh = root.Bokeh;\n  var bokeh_loaded = Bokeh != null && (Bokeh.version === py_version || (Bokeh.versions !== undefined && Bokeh.versions.has(py_version)));\n\n  if (typeof (root._bokeh_timeout) === \"undefined\" || force) {\n    root._bokeh_timeout = Date.now() + 5000;\n    root._bokeh_failed_load = false;\n  }\n\n  function run_callbacks() {\n    try {\n      root._bokeh_onload_callbacks.forEach(function(callback) {\n        if (callback != null)\n          callback();\n      });\n    } finally {\n      delete root._bokeh_onload_callbacks;\n    }\n    console.debug(\"Bokeh: all callbacks have finished\");\n  }\n\n  function load_libs(css_urls, js_urls, js_modules, js_exports, callback) {\n    if (css_urls == null) css_urls = [];\n    if (js_urls == null) js_urls = [];\n    if (js_modules == null) js_modules = [];\n    if (js_exports == null) js_exports = {};\n\n    root._bokeh_onload_callbacks.push(callback);\n\n    if (root._bokeh_is_loading > 0) {\n      console.debug(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n      return null;\n    }\n    if (js_urls.length === 0 && js_modules.length === 0 && Object.keys(js_exports).length === 0) {\n      run_callbacks();\n      return null;\n    }\n    if (!reloading) {\n      console.debug(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n    }\n\n    function on_load() {\n      root._bokeh_is_loading--;\n      if (root._bokeh_is_loading === 0) {\n        console.debug(\"Bokeh: all BokehJS libraries/stylesheets loaded\");\n        run_callbacks()\n      }\n    }\n    window._bokeh_on_load = on_load\n\n    function on_error() {\n      console.error(\"failed to load \" + url);\n    }\n\n    var skip = [];\n    if (window.requirejs) {\n      window.requirejs.config({'packages': {}, 'paths': {'ace': '//cdnjs.cloudflare.com/ajax/libs/ace/1.4.7', 'jspanel': 'https://cdn.jsdelivr.net/npm/jspanel4@4.12.0/dist/jspanel', 'jspanel-modal': 'https://cdn.jsdelivr.net/npm/jspanel4@4.12.0/dist/extensions/modal/jspanel.modal', 'jspanel-tooltip': 'https://cdn.jsdelivr.net/npm/jspanel4@4.12.0/dist/extensions/tooltip/jspanel.tooltip', 'jspanel-hint': 'https://cdn.jsdelivr.net/npm/jspanel4@4.12.0/dist/extensions/hint/jspanel.hint', 'jspanel-layout': 'https://cdn.jsdelivr.net/npm/jspanel4@4.12.0/dist/extensions/layout/jspanel.layout', 'jspanel-contextmenu': 'https://cdn.jsdelivr.net/npm/jspanel4@4.12.0/dist/extensions/contextmenu/jspanel.contextmenu', 'jspanel-dock': 'https://cdn.jsdelivr.net/npm/jspanel4@4.12.0/dist/extensions/dock/jspanel.dock', 'gridstack': 'https://cdn.jsdelivr.net/npm/gridstack@7.2.3/dist/gridstack-all', 'notyf': 'https://cdn.jsdelivr.net/npm/notyf@3/notyf.min'}, 'shim': {'ace/ext-language_tools': {'deps': ['ace/ace']}, 'ace/ext-modelist': {'deps': ['ace/ace']}, 'jspanel': {'exports': 'jsPanel'}, 'gridstack': {'exports': 'GridStack'}}});\n      require([\"ace/ace\"], function(ace) {\n\twindow.ace = ace\n\ton_load()\n      })\n      require([\"ace/ext-language_tools\"], function() {\n\ton_load()\n      })\n      require([\"ace/ext-modelist\"], function() {\n\ton_load()\n      })\n      require([\"jspanel\"], function(jsPanel) {\n\twindow.jsPanel = jsPanel\n\ton_load()\n      })\n      require([\"jspanel-modal\"], function() {\n\ton_load()\n      })\n      require([\"jspanel-tooltip\"], function() {\n\ton_load()\n      })\n      require([\"jspanel-hint\"], function() {\n\ton_load()\n      })\n      require([\"jspanel-layout\"], function() {\n\ton_load()\n      })\n      require([\"jspanel-contextmenu\"], function() {\n\ton_load()\n      })\n      require([\"jspanel-dock\"], function() {\n\ton_load()\n      })\n      require([\"gridstack\"], function(GridStack) {\n\twindow.GridStack = GridStack\n\ton_load()\n      })\n      require([\"notyf\"], function() {\n\ton_load()\n      })\n      root._bokeh_is_loading = css_urls.length + 12;\n    } else {\n      root._bokeh_is_loading = css_urls.length + js_urls.length + js_modules.length + Object.keys(js_exports).length;\n    }\n\n    var existing_stylesheets = []\n    var links = document.getElementsByTagName('link')\n    for (var i = 0; i < links.length; i++) {\n      var link = links[i]\n      if (link.href != null) {\n\texisting_stylesheets.push(link.href)\n      }\n    }\n    for (var i = 0; i < css_urls.length; i++) {\n      var url = css_urls[i];\n      if (existing_stylesheets.indexOf(url) !== -1) {\n\ton_load()\n\tcontinue;\n      }\n      const element = document.createElement(\"link\");\n      element.onload = on_load;\n      element.onerror = on_error;\n      element.rel = \"stylesheet\";\n      element.type = \"text/css\";\n      element.href = url;\n      console.debug(\"Bokeh: injecting link tag for BokehJS stylesheet: \", url);\n      document.body.appendChild(element);\n    }    if (((window['ace'] !== undefined) && (!(window['ace'] instanceof HTMLElement))) || window.requirejs) {\n      var urls = ['https://cdnjs.cloudflare.com/ajax/libs/ace/1.4.11/ace.js', 'https://cdnjs.cloudflare.com/ajax/libs/ace/1.4.11/ext-language_tools.js', 'https://cdnjs.cloudflare.com/ajax/libs/ace/1.4.11/ext-modelist.js'];\n      for (var i = 0; i < urls.length; i++) {\n        skip.push(urls[i])\n      }\n    }    if (((window['jsPanel'] !== undefined) && (!(window['jsPanel'] instanceof HTMLElement))) || window.requirejs) {\n      var urls = ['https://cdn.holoviz.org/panel/1.1.0/dist/bundled/floatpanel/jspanel4@4.12.0/dist/jspanel.js', 'https://cdn.holoviz.org/panel/1.1.0/dist/bundled/floatpanel/jspanel4@4.12.0/dist/extensions/modal/jspanel.modal.js', 'https://cdn.holoviz.org/panel/1.1.0/dist/bundled/floatpanel/jspanel4@4.12.0/dist/extensions/tooltip/jspanel.tooltip.js', 'https://cdn.holoviz.org/panel/1.1.0/dist/bundled/floatpanel/jspanel4@4.12.0/dist/extensions/hint/jspanel.hint.js', 'https://cdn.holoviz.org/panel/1.1.0/dist/bundled/floatpanel/jspanel4@4.12.0/dist/extensions/layout/jspanel.layout.js', 'https://cdn.holoviz.org/panel/1.1.0/dist/bundled/floatpanel/jspanel4@4.12.0/dist/extensions/contextmenu/jspanel.contextmenu.js', 'https://cdn.holoviz.org/panel/1.1.0/dist/bundled/floatpanel/jspanel4@4.12.0/dist/extensions/dock/jspanel.dock.js'];\n      for (var i = 0; i < urls.length; i++) {\n        skip.push(urls[i])\n      }\n    }    if (((window['GridStack'] !== undefined) && (!(window['GridStack'] instanceof HTMLElement))) || window.requirejs) {\n      var urls = ['https://cdn.holoviz.org/panel/1.1.0/dist/bundled/gridstack/gridstack@7.2.3/dist/gridstack-all.js'];\n      for (var i = 0; i < urls.length; i++) {\n        skip.push(urls[i])\n      }\n    }    if (((window['Notyf'] !== undefined) && (!(window['Notyf'] instanceof HTMLElement))) || window.requirejs) {\n      var urls = ['https://cdn.holoviz.org/panel/1.1.0/dist/bundled/notificationarea/notyf@3/notyf.min.js'];\n      for (var i = 0; i < urls.length; i++) {\n        skip.push(urls[i])\n      }\n    }    var existing_scripts = []\n    var scripts = document.getElementsByTagName('script')\n    for (var i = 0; i < scripts.length; i++) {\n      var script = scripts[i]\n      if (script.src != null) {\n\texisting_scripts.push(script.src)\n      }\n    }\n    for (var i = 0; i < js_urls.length; i++) {\n      var url = js_urls[i];\n      if (skip.indexOf(url) !== -1 || existing_scripts.indexOf(url) !== -1) {\n\tif (!window.requirejs) {\n\t  on_load();\n\t}\n\tcontinue;\n      }\n      var element = document.createElement('script');\n      element.onload = on_load;\n      element.onerror = on_error;\n      element.async = false;\n      element.src = url;\n      console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n      document.head.appendChild(element);\n    }\n    for (var i = 0; i < js_modules.length; i++) {\n      var url = js_modules[i];\n      if (skip.indexOf(url) !== -1 || existing_scripts.indexOf(url) !== -1) {\n\tif (!window.requirejs) {\n\t  on_load();\n\t}\n\tcontinue;\n      }\n      var element = document.createElement('script');\n      element.onload = on_load;\n      element.onerror = on_error;\n      element.async = false;\n      element.src = url;\n      element.type = \"module\";\n      console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n      document.head.appendChild(element);\n    }\n    for (const name in js_exports) {\n      var url = js_exports[name];\n      if (skip.indexOf(url) >= 0 || root[name] != null) {\n\tif (!window.requirejs) {\n\t  on_load();\n\t}\n\tcontinue;\n      }\n      var element = document.createElement('script');\n      element.onerror = on_error;\n      element.async = false;\n      element.type = \"module\";\n      console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n      element.textContent = `\n      import ${name} from \"${url}\"\n      window.${name} = ${name}\n      window._bokeh_on_load()\n      `\n      document.head.appendChild(element);\n    }\n    if (!js_urls.length && !js_modules.length) {\n      on_load()\n    }\n  };\n\n  function inject_raw_css(css) {\n    const element = document.createElement(\"style\");\n    element.appendChild(document.createTextNode(css));\n    document.body.appendChild(element);\n  }\n\n  var js_urls = [\"https://cdnjs.cloudflare.com/ajax/libs/ace/1.4.11/ace.js\", \"https://cdnjs.cloudflare.com/ajax/libs/ace/1.4.11/ext-language_tools.js\", \"https://cdnjs.cloudflare.com/ajax/libs/ace/1.4.11/ext-modelist.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-3.2.0.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-gl-3.2.0.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-widgets-3.2.0.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-tables-3.2.0.min.js\", \"https://cdn.holoviz.org/panel/1.1.0/dist/panel.min.js\"];\n  var js_modules = [];\n  var js_exports = {};\n  var css_urls = [];\n  var inline_js = [    function(Bokeh) {\n      inject_raw_css(\"/*\\n ~ CML // Creative Machine Learning ~\\n mml.css : CSS styling information for Panel and Bokeh\\n \\n This file defines the main CSS styling information for the CML course\\n \\n Author               :  Philippe Esling\\n                        <esling@ircam.fr>\\n*/\\n\\nbody {\\n  display: flex;\\n  height: 100vh;\\n  margin: 0px;\\n  overflow-x: hidden;\\n  overflow-y: hidden;\\n}\\n\\n.bk-root .bk, .bk-root .bk:before, .bk-root .bk:after {\\n  font-family: \\\"Josefin Sans\\\";\\n}\\n\\nimg {\\n  max-width: 100%;\\n}\\n\\n#container {\\n  padding:0px;\\n  height:100vh;\\n  width: 100vw;\\n  max-width: 100vw;\\n}\\n\\n#sidebar .mdc-list {\\n  padding-left: 5px;\\n  padding-right: 5px;\\n}\\n\\n.mdc-drawer-app-content {\\n  flex: auto;\\n  position: relative;\\n  overflow: hidden;\\n}\\n\\n.mdc-drawer {\\n  background: #FAFAFA; /* GRAY 50 */\\n}\\n\\n.mdc-drawer-app-content {\\n  margin-left: 0 !important;\\n}\\n\\n.title-bar {\\n  display: contents;\\n  justify-content: center;\\n  align-content: center;\\n  width: 100%;\\n}\\n\\n.mdc-top-app-bar .bk-menu {\\n  color: black\\n}\\n\\n.app-header {\\n  display: contents;\\n  padding-left: 10px;\\n  font-size: 1.25em;\\n}\\n\\nimg.app-logo {\\n  padding-right: 10px;\\n  font-size: 28px;\\n  height: 30px;\\n  max-width: inherit;\\n  padding-top: 12px;\\n  padding-bottom: 6px;\\n}\\n\\n#app-title {\\n  padding-right: 12px;\\n  padding-left: 12px;\\n}\\n\\n.title {\\n  font-family: \\\"Josefin Sans\\\";\\n  color: #fff;\\n  text-decoration: none;\\n  text-decoration-line: none;\\n  text-decoration-style: initial;\\n  text-decoration-color: initial;\\n  font-weight: 400;\\n  font-size: 2em;\\n  line-height: 2em;\\n  white-space: nowrap;\\n}\\n\\n.main-content {\\n  overflow-y: scroll;\\n  overflow-x: auto;\\n}\\n\\n#header {\\n  position: absolute;\\n  z-index: 7;\\n}\\n\\n#header-items {\\n  width: 100%;\\n  margin-left:15px;\\n}\\n\\n.pn-busy-container {\\n  align-items: center;\\n  justify-content: center;\\n  display: flex;\\n}\\n\\n.mdc-drawer__content {\\n  overflow-x: hidden;\\n}\\n.mdc-drawer__content, .main-content {\\n  padding: 12px;\\n}\\n\\n.main-content {\\n  height: calc(100vh - 88px);\\n  max-height: calc(100vh - 88px);\\n  padding-right: 32px;\\n}\\n\\nbutton.mdc-button.mdc-card-button {\\n  color: transparent;\\n  height: 50px;\\n}\\n\\np.mdc-button {\\n  display: none;\\n}\\n\\ndiv.mdc-card {\\n  border-radius: 0px\\n}\\n\\n.mdc-card .card-header {\\n  display: flex;\\n}\\n\\n.mdc-card-title {\\n  font-family: \\\"Josefin Sans\\\";\\n  font-weight: bold;\\n  align-items: center;\\n  display: flex !important;\\n  position: relative !important;\\n}\\n\\n.mdc-card-title:nth-child(2) {\\n  margin-left: -1.4em;\\n}\\n\\n.pn-modal {\\n  overflow-y: scroll;\\n  width: 100%;\\n  display: none;\\n  position: absolute;\\n  top: 0;\\n  left: 0;\\n}\\n\\n.pn-modal-content {\\n  font-family: \\\"Josefin Sans\\\";\\n  background-color: #0e0e0e;\\n  margin: auto;\\n  margin-top: 25px;\\n  margin-bottom: 25px;\\n  padding: 15px 20px 20px 20px;\\n  border: 1px solid #888;\\n  width: 80% !important;\\n}\\n\\n.pn-modal-close {\\n  position: absolute;\\n  right: 25px;\\n  z-index: 100;\\n}\\n\\n.pn-modal-close:hover,\\n.pn-modal-close:focus {\\n  color: #000;\\n  text-decoration: none;\\n  cursor: pointer;\\n}\\n\\n.custom_button_bokeh button.bk-btn.bk-btn-default {\\n    font-size:48pt;\\n    background-color: #05b7ff;\\n    border-color: #05b7ff;\\n}\");\n    },    function(Bokeh) {\n      Bokeh.set_log_level(\"info\");\n    },\nfunction(Bokeh) {} // ensure no trailing comma for IE\n  ];\n\n  function run_inline_js() {\n    if ((root.Bokeh !== undefined) || (force === true)) {\n      for (var i = 0; i < inline_js.length; i++) {\n        inline_js[i].call(root, root.Bokeh);\n      }\n      // Cache old bokeh versions\n      if (Bokeh != undefined && !reloading) {\n\tvar NewBokeh = root.Bokeh;\n\tif (Bokeh.versions === undefined) {\n\t  Bokeh.versions = new Map();\n\t}\n\tif (NewBokeh.version !== Bokeh.version) {\n\t  Bokeh.versions.set(NewBokeh.version, NewBokeh)\n\t}\n\troot.Bokeh = Bokeh;\n      }} else if (Date.now() < root._bokeh_timeout) {\n      setTimeout(run_inline_js, 100);\n    } else if (!root._bokeh_failed_load) {\n      console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n      root._bokeh_failed_load = true;\n    }\n    root._bokeh_is_initializing = false\n  }\n\n  function load_or_wait() {\n    // Implement a backoff loop that tries to ensure we do not load multiple\n    // versions of Bokeh and its dependencies at the same time.\n    // In recent versions we use the root._bokeh_is_initializing flag\n    // to determine whether there is an ongoing attempt to initialize\n    // bokeh, however for backward compatibility we also try to ensure\n    // that we do not start loading a newer (Panel>=1.0 and Bokeh>3) version\n    // before older versions are fully initialized.\n    if (root._bokeh_is_initializing && Date.now() > root._bokeh_timeout) {\n      root._bokeh_is_initializing = false;\n      root._bokeh_onload_callbacks = undefined;\n      console.log(\"Bokeh: BokehJS was loaded multiple times but one version failed to initialize.\");\n      load_or_wait();\n    } else if (root._bokeh_is_initializing || (typeof root._bokeh_is_initializing === \"undefined\" && root._bokeh_onload_callbacks !== undefined)) {\n      setTimeout(load_or_wait, 100);\n    } else {\n      Bokeh = root.Bokeh;\n      bokeh_loaded = Bokeh != null && (Bokeh.version === py_version || (Bokeh.versions !== undefined && Bokeh.versions.has(py_version)));\n      root._bokeh_is_initializing = true\n      root._bokeh_onload_callbacks = []\n      if (!reloading && (!bokeh_loaded || is_dev)) {\n\troot.Bokeh = undefined;\n      }\n      load_libs(css_urls, js_urls, js_modules, js_exports, function() {\n\tconsole.debug(\"Bokeh: BokehJS plotting callback run at\", now());\n\trun_inline_js();\n      });\n    }\n  }\n  // Give older versions of the autoload script a head-start to ensure\n  // they initialize before we start loading newer version.\n  setTimeout(load_or_wait, 100)\n}(window));",
      "application/vnd.holoviews_load.v0+json": ""
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/javascript": "\nif ((window.PyViz === undefined) || (window.PyViz instanceof HTMLElement)) {\n  window.PyViz = {comms: {}, comm_status:{}, kernels:{}, receivers: {}, plot_index: []}\n}\n\n\n    function JupyterCommManager() {\n    }\n\n    JupyterCommManager.prototype.register_target = function(plot_id, comm_id, msg_handler) {\n      if (window.comm_manager || ((window.Jupyter !== undefined) && (Jupyter.notebook.kernel != null))) {\n        var comm_manager = window.comm_manager || Jupyter.notebook.kernel.comm_manager;\n        comm_manager.register_target(comm_id, function(comm) {\n          comm.on_msg(msg_handler);\n        });\n      } else if ((plot_id in window.PyViz.kernels) && (window.PyViz.kernels[plot_id])) {\n        window.PyViz.kernels[plot_id].registerCommTarget(comm_id, function(comm) {\n          comm.onMsg = msg_handler;\n        });\n      } else if (typeof google != 'undefined' && google.colab.kernel != null) {\n        google.colab.kernel.comms.registerTarget(comm_id, (comm) => {\n          var messages = comm.messages[Symbol.asyncIterator]();\n          function processIteratorResult(result) {\n            var message = result.value;\n            console.log(message)\n            var content = {data: message.data, comm_id};\n            var buffers = []\n            for (var buffer of message.buffers || []) {\n              buffers.push(new DataView(buffer))\n            }\n            var metadata = message.metadata || {};\n            var msg = {content, buffers, metadata}\n            msg_handler(msg);\n            return messages.next().then(processIteratorResult);\n          }\n          return messages.next().then(processIteratorResult);\n        })\n      }\n    }\n\n    JupyterCommManager.prototype.get_client_comm = function(plot_id, comm_id, msg_handler) {\n      if (comm_id in window.PyViz.comms) {\n        return window.PyViz.comms[comm_id];\n      } else if (window.comm_manager || ((window.Jupyter !== undefined) && (Jupyter.notebook.kernel != null))) {\n        var comm_manager = window.comm_manager || Jupyter.notebook.kernel.comm_manager;\n        var comm = comm_manager.new_comm(comm_id, {}, {}, {}, comm_id);\n        if (msg_handler) {\n          comm.on_msg(msg_handler);\n        }\n      } else if ((plot_id in window.PyViz.kernels) && (window.PyViz.kernels[plot_id])) {\n        var comm = window.PyViz.kernels[plot_id].connectToComm(comm_id);\n        comm.open();\n        if (msg_handler) {\n          comm.onMsg = msg_handler;\n        }\n      } else if (typeof google != 'undefined' && google.colab.kernel != null) {\n        var comm_promise = google.colab.kernel.comms.open(comm_id)\n        comm_promise.then((comm) => {\n          window.PyViz.comms[comm_id] = comm;\n          if (msg_handler) {\n            var messages = comm.messages[Symbol.asyncIterator]();\n            function processIteratorResult(result) {\n              var message = result.value;\n              var content = {data: message.data};\n              var metadata = message.metadata || {comm_id};\n              var msg = {content, metadata}\n              msg_handler(msg);\n              return messages.next().then(processIteratorResult);\n            }\n            return messages.next().then(processIteratorResult);\n          }\n        }) \n        var sendClosure = (data, metadata, buffers, disposeOnDone) => {\n          return comm_promise.then((comm) => {\n            comm.send(data, metadata, buffers, disposeOnDone);\n          });\n        };\n        var comm = {\n          send: sendClosure\n        };\n      }\n      window.PyViz.comms[comm_id] = comm;\n      return comm;\n    }\n    window.PyViz.comm_manager = new JupyterCommManager();\n    \n\n\nvar JS_MIME_TYPE = 'application/javascript';\nvar HTML_MIME_TYPE = 'text/html';\nvar EXEC_MIME_TYPE = 'application/vnd.holoviews_exec.v0+json';\nvar CLASS_NAME = 'output';\n\n/**\n * Render data to the DOM node\n */\nfunction render(props, node) {\n  var div = document.createElement(\"div\");\n  var script = document.createElement(\"script\");\n  node.appendChild(div);\n  node.appendChild(script);\n}\n\n/**\n * Handle when a new output is added\n */\nfunction handle_add_output(event, handle) {\n  var output_area = handle.output_area;\n  var output = handle.output;\n  if ((output.data == undefined) || (!output.data.hasOwnProperty(EXEC_MIME_TYPE))) {\n    return\n  }\n  var id = output.metadata[EXEC_MIME_TYPE][\"id\"];\n  var toinsert = output_area.element.find(\".\" + CLASS_NAME.split(' ')[0]);\n  if (id !== undefined) {\n    var nchildren = toinsert.length;\n    var html_node = toinsert[nchildren-1].children[0];\n    html_node.innerHTML = output.data[HTML_MIME_TYPE];\n    var scripts = [];\n    var nodelist = html_node.querySelectorAll(\"script\");\n    for (var i in nodelist) {\n      if (nodelist.hasOwnProperty(i)) {\n        scripts.push(nodelist[i])\n      }\n    }\n\n    scripts.forEach( function (oldScript) {\n      var newScript = document.createElement(\"script\");\n      var attrs = [];\n      var nodemap = oldScript.attributes;\n      for (var j in nodemap) {\n        if (nodemap.hasOwnProperty(j)) {\n          attrs.push(nodemap[j])\n        }\n      }\n      attrs.forEach(function(attr) { newScript.setAttribute(attr.name, attr.value) });\n      newScript.appendChild(document.createTextNode(oldScript.innerHTML));\n      oldScript.parentNode.replaceChild(newScript, oldScript);\n    });\n    if (JS_MIME_TYPE in output.data) {\n      toinsert[nchildren-1].children[1].textContent = output.data[JS_MIME_TYPE];\n    }\n    output_area._hv_plot_id = id;\n    if ((window.Bokeh !== undefined) && (id in Bokeh.index)) {\n      window.PyViz.plot_index[id] = Bokeh.index[id];\n    } else {\n      window.PyViz.plot_index[id] = null;\n    }\n  } else if (output.metadata[EXEC_MIME_TYPE][\"server_id\"] !== undefined) {\n    var bk_div = document.createElement(\"div\");\n    bk_div.innerHTML = output.data[HTML_MIME_TYPE];\n    var script_attrs = bk_div.children[0].attributes;\n    for (var i = 0; i < script_attrs.length; i++) {\n      toinsert[toinsert.length - 1].childNodes[1].setAttribute(script_attrs[i].name, script_attrs[i].value);\n    }\n    // store reference to server id on output_area\n    output_area._bokeh_server_id = output.metadata[EXEC_MIME_TYPE][\"server_id\"];\n  }\n}\n\n/**\n * Handle when an output is cleared or removed\n */\nfunction handle_clear_output(event, handle) {\n  var id = handle.cell.output_area._hv_plot_id;\n  var server_id = handle.cell.output_area._bokeh_server_id;\n  if (((id === undefined) || !(id in PyViz.plot_index)) && (server_id !== undefined)) { return; }\n  var comm = window.PyViz.comm_manager.get_client_comm(\"hv-extension-comm\", \"hv-extension-comm\", function () {});\n  if (server_id !== null) {\n    comm.send({event_type: 'server_delete', 'id': server_id});\n    return;\n  } else if (comm !== null) {\n    comm.send({event_type: 'delete', 'id': id});\n  }\n  delete PyViz.plot_index[id];\n  if ((window.Bokeh !== undefined) & (id in window.Bokeh.index)) {\n    var doc = window.Bokeh.index[id].model.document\n    doc.clear();\n    const i = window.Bokeh.documents.indexOf(doc);\n    if (i > -1) {\n      window.Bokeh.documents.splice(i, 1);\n    }\n  }\n}\n\n/**\n * Handle kernel restart event\n */\nfunction handle_kernel_cleanup(event, handle) {\n  delete PyViz.comms[\"hv-extension-comm\"];\n  window.PyViz.plot_index = {}\n}\n\n/**\n * Handle update_display_data messages\n */\nfunction handle_update_output(event, handle) {\n  handle_clear_output(event, {cell: {output_area: handle.output_area}})\n  handle_add_output(event, handle)\n}\n\nfunction register_renderer(events, OutputArea) {\n  function append_mime(data, metadata, element) {\n    // create a DOM node to render to\n    var toinsert = this.create_output_subarea(\n    metadata,\n    CLASS_NAME,\n    EXEC_MIME_TYPE\n    );\n    this.keyboard_manager.register_events(toinsert);\n    // Render to node\n    var props = {data: data, metadata: metadata[EXEC_MIME_TYPE]};\n    render(props, toinsert[0]);\n    element.append(toinsert);\n    return toinsert\n  }\n\n  events.on('output_added.OutputArea', handle_add_output);\n  events.on('output_updated.OutputArea', handle_update_output);\n  events.on('clear_output.CodeCell', handle_clear_output);\n  events.on('delete.Cell', handle_clear_output);\n  events.on('kernel_ready.Kernel', handle_kernel_cleanup);\n\n  OutputArea.prototype.register_mime_type(EXEC_MIME_TYPE, append_mime, {\n    safe: true,\n    index: 0\n  });\n}\n\nif (window.Jupyter !== undefined) {\n  try {\n    var events = require('base/js/events');\n    var OutputArea = require('notebook/js/outputarea').OutputArea;\n    if (OutputArea.prototype.mime_types().indexOf(EXEC_MIME_TYPE) == -1) {\n      register_renderer(events, OutputArea);\n    }\n  } catch(err) {\n  }\n}\n",
      "application/vnd.holoviews_load.v0+json": ""
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<style>*[data-root-id],\n",
       "*[data-root-id] > * {\n",
       "  box-sizing: border-box;\n",
       "  font-family: var(--jp-ui-font-family);\n",
       "  font-size: var(--jp-ui-font-size1);\n",
       "  color: var(--vscode-editor-foreground, var(--jp-ui-font-color1));\n",
       "}\n",
       "\n",
       "/* Override VSCode background color */\n",
       ".cell-output-ipywidget-background:has(> .cell-output-ipywidget-background\n",
       "    > .lm-Widget\n",
       "    > *[data-root-id]),\n",
       ".cell-output-ipywidget-background:has(> .lm-Widget > *[data-root-id]) {\n",
       "  background-color: transparent !important;\n",
       "}\n",
       "</style>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.holoviews_exec.v0+json": "",
      "text/html": [
       "<div id='e583b20a-24d7-495a-a630-22151fac871f'>\n",
       "  <div id=\"b9d25770-d7da-44a3-b6aa-96e6bc39030e\" data-root-id=\"e583b20a-24d7-495a-a630-22151fac871f\" style=\"display: contents;\"></div>\n",
       "</div>\n",
       "<script type=\"application/javascript\">(function(root) {\n",
       "  var docs_json = {\"b800f6f9-4ac9-4a33-8f2c-eae14a778909\":{\"version\":\"3.2.0\",\"title\":\"Bokeh Application\",\"roots\":[{\"type\":\"object\",\"name\":\"panel.models.browser.BrowserInfo\",\"id\":\"e583b20a-24d7-495a-a630-22151fac871f\"},{\"type\":\"object\",\"name\":\"panel.models.comm_manager.CommManager\",\"id\":\"ce379d3e-3c43-4834-bef8-148f8d40bfe3\",\"attributes\":{\"plot_id\":\"e583b20a-24d7-495a-a630-22151fac871f\",\"comm_id\":\"00118831ab4f47c5979eb65e5587f95f\",\"client_comm_id\":\"9337a532c6ba49fea893109558871639\"}}],\"defs\":[{\"type\":\"model\",\"name\":\"ReactiveHTML1\"},{\"type\":\"model\",\"name\":\"FlexBox1\",\"properties\":[{\"name\":\"align_content\",\"kind\":\"Any\",\"default\":\"flex-start\"},{\"name\":\"align_items\",\"kind\":\"Any\",\"default\":\"flex-start\"},{\"name\":\"flex_direction\",\"kind\":\"Any\",\"default\":\"row\"},{\"name\":\"flex_wrap\",\"kind\":\"Any\",\"default\":\"wrap\"},{\"name\":\"justify_content\",\"kind\":\"Any\",\"default\":\"flex-start\"}]},{\"type\":\"model\",\"name\":\"FloatPanel1\",\"properties\":[{\"name\":\"config\",\"kind\":\"Any\",\"default\":{\"type\":\"map\"}},{\"name\":\"contained\",\"kind\":\"Any\",\"default\":true},{\"name\":\"position\",\"kind\":\"Any\",\"default\":\"right-top\"},{\"name\":\"offsetx\",\"kind\":\"Any\",\"default\":null},{\"name\":\"offsety\",\"kind\":\"Any\",\"default\":null},{\"name\":\"theme\",\"kind\":\"Any\",\"default\":\"primary\"},{\"name\":\"status\",\"kind\":\"Any\",\"default\":\"normalized\"}]},{\"type\":\"model\",\"name\":\"GridStack1\",\"properties\":[{\"name\":\"mode\",\"kind\":\"Any\",\"default\":\"warn\"},{\"name\":\"ncols\",\"kind\":\"Any\",\"default\":null},{\"name\":\"nrows\",\"kind\":\"Any\",\"default\":null},{\"name\":\"allow_resize\",\"kind\":\"Any\",\"default\":true},{\"name\":\"allow_drag\",\"kind\":\"Any\",\"default\":true},{\"name\":\"state\",\"kind\":\"Any\",\"default\":[]}]},{\"type\":\"model\",\"name\":\"drag1\",\"properties\":[{\"name\":\"slider_width\",\"kind\":\"Any\",\"default\":5},{\"name\":\"slider_color\",\"kind\":\"Any\",\"default\":\"black\"},{\"name\":\"value\",\"kind\":\"Any\",\"default\":50}]},{\"type\":\"model\",\"name\":\"click1\",\"properties\":[{\"name\":\"terminal_output\",\"kind\":\"Any\",\"default\":\"\"},{\"name\":\"debug_name\",\"kind\":\"Any\",\"default\":\"\"},{\"name\":\"clears\",\"kind\":\"Any\",\"default\":0}]},{\"type\":\"model\",\"name\":\"FastWrapper1\",\"properties\":[{\"name\":\"object\",\"kind\":\"Any\",\"default\":null},{\"name\":\"style\",\"kind\":\"Any\",\"default\":null}]},{\"type\":\"model\",\"name\":\"NotificationAreaBase1\",\"properties\":[{\"name\":\"position\",\"kind\":\"Any\",\"default\":\"bottom-right\"},{\"name\":\"_clear\",\"kind\":\"Any\",\"default\":0}]},{\"type\":\"model\",\"name\":\"NotificationArea1\",\"properties\":[{\"name\":\"notifications\",\"kind\":\"Any\",\"default\":[]},{\"name\":\"position\",\"kind\":\"Any\",\"default\":\"bottom-right\"},{\"name\":\"_clear\",\"kind\":\"Any\",\"default\":0},{\"name\":\"types\",\"kind\":\"Any\",\"default\":[{\"type\":\"map\",\"entries\":[[\"type\",\"warning\"],[\"background\",\"#ffc107\"],[\"icon\",{\"type\":\"map\",\"entries\":[[\"className\",\"fas fa-exclamation-triangle\"],[\"tagName\",\"i\"],[\"color\",\"white\"]]}]]},{\"type\":\"map\",\"entries\":[[\"type\",\"info\"],[\"background\",\"#007bff\"],[\"icon\",{\"type\":\"map\",\"entries\":[[\"className\",\"fas fa-info-circle\"],[\"tagName\",\"i\"],[\"color\",\"white\"]]}]]}]}]},{\"type\":\"model\",\"name\":\"Notification\",\"properties\":[{\"name\":\"background\",\"kind\":\"Any\",\"default\":null},{\"name\":\"duration\",\"kind\":\"Any\",\"default\":3000},{\"name\":\"icon\",\"kind\":\"Any\",\"default\":null},{\"name\":\"message\",\"kind\":\"Any\",\"default\":\"\"},{\"name\":\"notification_type\",\"kind\":\"Any\",\"default\":null},{\"name\":\"_destroyed\",\"kind\":\"Any\",\"default\":false}]},{\"type\":\"model\",\"name\":\"TemplateActions1\",\"properties\":[{\"name\":\"open_modal\",\"kind\":\"Any\",\"default\":0},{\"name\":\"close_modal\",\"kind\":\"Any\",\"default\":0}]},{\"type\":\"model\",\"name\":\"BootstrapTemplateActions1\",\"properties\":[{\"name\":\"open_modal\",\"kind\":\"Any\",\"default\":0},{\"name\":\"close_modal\",\"kind\":\"Any\",\"default\":0}]},{\"type\":\"model\",\"name\":\"MaterialTemplateActions1\",\"properties\":[{\"name\":\"open_modal\",\"kind\":\"Any\",\"default\":0},{\"name\":\"close_modal\",\"kind\":\"Any\",\"default\":0}]}]}};\n",
       "  var render_items = [{\"docid\":\"b800f6f9-4ac9-4a33-8f2c-eae14a778909\",\"roots\":{\"e583b20a-24d7-495a-a630-22151fac871f\":\"b9d25770-d7da-44a3-b6aa-96e6bc39030e\"},\"root_ids\":[\"e583b20a-24d7-495a-a630-22151fac871f\"]}];\n",
       "  var docs = Object.values(docs_json)\n",
       "  if (!docs) {\n",
       "    return\n",
       "  }\n",
       "  const py_version = docs[0].version.replace('rc', '-rc.')\n",
       "  const is_dev = py_version.indexOf(\"+\") !== -1 || py_version.indexOf(\"-\") !== -1\n",
       "  function embed_document(root) {\n",
       "    var Bokeh = get_bokeh(root)\n",
       "    Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "    for (const render_item of render_items) {\n",
       "      for (const root_id of render_item.root_ids) {\n",
       "\tconst id_el = document.getElementById(root_id)\n",
       "\tif (id_el.children.length && (id_el.children[0].className === 'bk-root')) {\n",
       "\t  const root_el = id_el.children[0]\n",
       "\t  root_el.id = root_el.id + '-rendered'\n",
       "\t}\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "  function get_bokeh(root) {\n",
       "    if (root.Bokeh === undefined) {\n",
       "      return null\n",
       "    } else if (root.Bokeh.version !== py_version && !is_dev) {\n",
       "      if (root.Bokeh.versions === undefined || !root.Bokeh.versions.has(py_version)) {\n",
       "\treturn null\n",
       "      }\n",
       "      return root.Bokeh.versions.get(py_version);\n",
       "    } else if (root.Bokeh.version === py_version) {\n",
       "      return root.Bokeh\n",
       "    }\n",
       "    return null\n",
       "  }\n",
       "  function is_loaded(root) {\n",
       "    var Bokeh = get_bokeh(root)\n",
       "    return (Bokeh != null && Bokeh.Panel !== undefined)\n",
       "  }\n",
       "  if (is_loaded(root)) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    var attempts = 0;\n",
       "    var timer = setInterval(function(root) {\n",
       "      if (is_loaded(root)) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else if (document.readyState == \"complete\") {\n",
       "        attempts++;\n",
       "        if (attempts > 200) {\n",
       "          clearInterval(timer);\n",
       "\t  var Bokeh = get_bokeh(root)\n",
       "\t  if (Bokeh == null || Bokeh.Panel == null) {\n",
       "            console.warn(\"Panel: ERROR: Unable to run Panel code because Bokeh or Panel library is missing\");\n",
       "\t  } else {\n",
       "\t    console.warn(\"Panel: WARNING: Attempting to render but not all required libraries could be resolved.\")\n",
       "\t    embed_document(root)\n",
       "\t  }\n",
       "        }\n",
       "      }\n",
       "    }, 25, root)\n",
       "  }\n",
       "})(window);</script>"
      ]
     },
     "metadata": {
      "application/vnd.holoviews_exec.v0+json": {
       "id": "e583b20a-24d7-495a-a630-22151fac871f"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Base imports\n",
    "import time\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from cml.plot import initialize_bokeh\n",
    "from cml.panel import initialize_panel\n",
    "from jupyterthemes.stylefx import set_nb_theme\n",
    "from bokeh.io import show\n",
    "initialize_bokeh()\n",
    "initialize_panel()\n",
    "set_nb_theme(\"onedork\")\n",
    "rng = np.random.RandomState(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As usual, to simplify your work, we provide sets of functions that allows fast problem definition and plotting functionnalities (from our `cml` library).\n",
    "\n",
    "  |**File**|*Explanation*|\n",
    "  |-------:|:---------|\n",
    "  |`scatter_boundary`|Plots the decision boundary of a single neuron with 2-dimensional inputs|\n",
    "  |`scatter_classes`|Plots (bi-dimensionnal) input patterns|"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "from cml.plot import scatter_classes, scatter_boundary, center_plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Remember that a single neuron is only able to learn _linearly separable_ problems. To produce such classes of problems, we provide a script that draw a set of random 2-dimensional points, then choose a random line in this space that will act as the linear frontier between 2 classes (hence defining a linear 2-class problem). The variables that will be used by your code are the following.  \n",
    "\n",
    "```Python\n",
    "y_class       # classes of the observqtions \n",
    "x_inputs      # 2 x n final matrix of random input observations\n",
    "weights       # 2 x 1 vector of neuron weights\n",
    "bias          # 1 x 1 vector of bias\n",
    "```\n",
    "\n",
    "You can execute the code below to see our simple classification problem. (Note that running the same cell multiple times produces a different starting dataset). In order to have a well-defined classification problem, we can rely on the `make_blobs` function provided by `scikit-learn` in the `sklearn.datasets` package."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "db96ef1057cd45da817ced34897af51c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "BokehModel(combine_events=True, render_bundle={'docs_json': {'6def2045-006f-446e-a328-4567bc4f338d': {'version…"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "from sklearn.datasets import make_blobs\n",
    "# Properties of the problem\n",
    "n_observations = 200\n",
    "noise = 0.2\n",
    "c1_center = [-2, -1]\n",
    "c2_center = [2, 1]\n",
    "# Create points\n",
    "x_coords, y_class = make_blobs(n_samples=n_observations, centers=[c1_center, c2_center], n_features=2, cluster_std=0.45)\n",
    "x_inputs = x_coords + (noise * np.random.randn(n_observations, 2))\n",
    "# Plot the corresponding pattern\n",
    "plot = center_plot(scatter_classes(x_inputs, y_class))\n",
    "plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The previous example does not allow us to know the exact values (slope and bias) of our ground truth separation line. Hence, we can define a more complex separation problem (with points laying almost right on the separation fronteer), but with known values for our ground truth."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.9487778751104576\n",
      "-0.6070693334649737\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3f1582665e9247cfa9dbf27ce6ecf2f6",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "BokehModel(combine_events=True, render_bundle={'docs_json': {'b6fdbe14-b644-482a-aabd-ffc04f3f1147': {'version…"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Number of points to generate\n",
    "n_observations = 100;\n",
    "# Generate 2-dimensional random points\n",
    "x_inputs = np.random.rand(int(n_observations), 2) * 2 - 1;\n",
    "# Slope of separating line\n",
    "sep_slope = np.log(np.random.rand() * 10);\n",
    "sep_bias = np.random.rand() * 2 - 1;\n",
    "# Create the indexes for a two-class problem\n",
    "y_class = (x_inputs[:, 1] - x_inputs[:, 0] * sep_slope - sep_bias > 0) * 1;\n",
    "# Plot the corresponding pattern\n",
    "plot = center_plot(scatter_classes(x_inputs, y_class))\n",
    "print(sep_slope)\n",
    "print(sep_bias)\n",
    "plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"neuron\"></a>\n",
    "## Single neuron\n",
    "\n",
    "For the first parts of the tutorial, we will perform the simplest classification model possible in a neural network setting, a single neuron. We briefly recall here that; given an input vector $ \\mathbf{x} \\in \\mathbb{R}^{n} $, a single neuron computes the function  \n",
    "\n",
    "$$\n",
    "\\begin{equation}\n",
    "\\bar{y}=\\phi\\left(\\sum_{i = 1}^{n}w_{i}.x_{i} + b\\right)\n",
    "\\end{equation}\n",
    "$$\n",
    "\n",
    "with $ \\mathbf{w} \\in \\mathbb{R}^{n} $ a weight vector, $ b $ a bias and $ \\phi\\left( \\cdot \\right) $ an *activation function*. Therefore, if we consider the *threshold* activation function ($ \\mathbb{I}_{0}\\left(x\\right)=1 $ if $ x \\geq 0$), a single neuron simply performs an *affine transform* and then a *linear* discrimination of the space. \n",
    "\n",
    "***\n",
    "\n",
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #410819; border-color: #cb2e47\">\n",
    "\n",
    "> **Important note**\n",
    "> The following implementation tries to remain as close as possible to the original neuron model by McCulloch & Pitts, notably by using the _threshold activation_ function. Although this leads to a very simple implementation (akin to using an _identity_ activation, note that it is not to be used afterwards.\n",
    "\n",
    "</div>\n",
    "\n",
    "***\n",
    "\n",
    "As we will see later on, a **neural network** will simply be composed of _layers_ of these neurons, which produce successive computations\n",
    "\n",
    "<img src=\"images/02_feedforward_nn.png\" align=\"center\"/>\n",
    "\n",
    "Geometrically, a single neuron computes an hyperplane that separates the space. In order to learn, we have to adjust the weights and know \"how much wrong we are\". To do so, we consider that we know the desired output $ y_{j} $ of a system for a given example $ \\mathbf{x}_{j} $ (eg. a predicted value for a regression system, a class value for a classification system). Therefore, we define the MSE loss function $ \\mathcal{L}_{\\mathcal{D}} $ over a whole dataset as\n",
    "\n",
    "$$\n",
    "\\begin{equation}\n",
    "\\mathcal{L}_{\\mathcal{D}}=\\sum_{j=1}^{|\\mathcal{D}|}\\left\\Vert \\bar{y}_{j}-y_{j}\\right\\Vert ^{2}\n",
    "\\end{equation}\n",
    "$$\n",
    "\n",
    "with $|\\mathcal{D}|$ being the size of our dataset. In order to know how to change the weights based on the value of the errors, we need to now \"how to change it to make it better\". Therefore, we should compute the sets of derivatives of the error given each parameter\n",
    "\n",
    "$$\n",
    "\\begin{equation}\n",
    "\\Delta\\bar{\\mathbf{w}}=\\left(\\frac{\\delta\\mathcal{L}_{\\mathcal{D}}}{\\delta w_{1}},\\ldots,\\frac{\\delta\\mathcal{L}_{\\mathcal{D}}}{\\delta w_{n}}\\right)\n",
    "\\end{equation}\n",
    "$$ \n",
    "\n",
    "***\n",
    "\n",
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #192841; border-color: #779ecb\">\n",
    "\n",
    "> ### Exercise (course)\n",
    ">   1. Perform the derivatives of the output given a single neuron\n",
    ">   2. Perform the derivatives for the bias as well\n",
    "\n",
    "</div>\n",
    "\n",
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training your own neuron\n",
    "\n",
    "We will start by training a single neuron to learn how to perform this discrimination with a linear problem (so that a single neuron is enough to solve it). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0 0 0 0 1 1 1 1 1 0 1 1 1 0 1 0 0 0 0 0 1 1 1 0 1 1 1 0 1 0 1 1 0 0 1 1 0\n",
      " 0 0 0 1 0 0 1 0 1 1 1 0 0 0 1 1 1 1 0 0 1 1 1 0 0 1 0 0 0 1 0 1 0 1 0 0 1\n",
      " 0 1 1 1 1 0 0 1 0 1 0 0 1 1 1 1 1 0 1 1 1 0 0 1 1 0]\n"
     ]
    }
   ],
   "source": [
    "# Learning rate\n",
    "eta = 1e-2;\n",
    "# Weight decay\n",
    "lambda_r = 0.1\n",
    "# Number of epochs\n",
    "n_epochs = 50\n",
    "# Initialize the weights\n",
    "weights = np.random.randn(1, 2);\n",
    "bias = np.random.randn(1, 1);\n",
    "# Save the weight history for plotting\n",
    "weights_history = np.zeros((n_epochs + 1, 2));\n",
    "bias_history = np.zeros((n_epochs + 1, 1));\n",
    "print(y_class)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you need to update the following code loop to ensure that your neuron learns to separate between the classes \n",
    "***\n",
    "\n",
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #192841; border-color: #779ecb\">\n",
    "\n",
    "> ### Exercise (course)\n",
    ">   1. Update the loop so that it computes the forward propagation error\n",
    ">   2. Update the loop to perform learning (based on back-propagation)\n",
    ">   3. Run the learning procedure, which should produce a result similar to that displayed on the website\n",
    ">   4. Perform multiple re-runs by **tweaking the hyperparameters** (learning rate, weight decay)\n",
    ">   5. What observations can you make on the learning process?\n",
    ">   6. (Optional) Change the input patterns, and confirm your observations.\n",
    ">   6. (Optional) Incorporate the bias in the weights to obtain a **vectorized** code.\n",
    "\n",
    "</div>\n",
    "\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[-0.06234079 -0.1264729  -0.08307224 -0.23273931  0.62242883  0.01680905\n",
      "   0.34183238  0.00361634  0.5588409  -0.00347255  0.5569873   0.07539968\n",
      "   0.17680282 -0.04301378  0.2655158  -0.06454653 -0.36461444 -0.25888115\n",
      "  -0.21210544  0.00542308  0.20342839  0.16024209  0.0867162  -0.26687421\n",
      "   0.5418322   0.32283504  0.34741941 -0.02796747  0.0867811  -0.44096831\n",
      "   0.18974836  0.10179449 -0.2507066  -0.4100464   0.45216347  0.28901233\n",
      "  -0.11228673 -0.24567279 -0.12659244 -0.04866655  0.60773486 -0.26624914\n",
      "  -0.06163186  0.25871984 -0.01613827  0.6732571   0.49974233  0.26867387\n",
      "  -0.52610051 -0.19101167 -0.42611392  0.52916024  0.28488139  0.60147241\n",
      "   0.13371078 -0.05575655 -0.23047055  0.41619441  0.19453296  0.56460957\n",
      "  -0.29114517 -0.02780339  0.42350527 -0.30833713 -0.43830811 -0.18722451\n",
      "   0.29817116 -0.19604521  0.35943784 -0.37982663  0.22821    -0.04887366\n",
      "  -0.01529686  0.74997036 -0.3249201   0.01384669  0.3740634   0.09343428\n",
      "   0.63807555 -0.17238918 -0.27196451  0.05168274 -0.04227491  0.34618305\n",
      "  -0.25369434 -0.22102946  0.14210999  0.0600366   0.5078504   0.47253564\n",
      "   0.37752995 -0.09430559  0.19397431  0.07081444  0.23859718 -0.13793553\n",
      "  -0.16439851  0.11673746  0.24177879 -0.37729193]]\n",
      "[[False False False False  True  True  True  True  True False  True  True\n",
      "   True False  True False False False False  True  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.07751987 -0.14318023 -0.10148664 -0.24283371  0.61206214  0.00102615\n",
      "   0.34298232 -0.00916387  0.55545645 -0.00853712  0.54529218  0.0566219\n",
      "   0.15751983 -0.06126506  0.26679813 -0.08625866 -0.37628786 -0.27288849\n",
      "  -0.21953115 -0.01500465  0.18327268  0.14863411  0.07496488 -0.27389328\n",
      "   0.53313739  0.32467917  0.34434131 -0.05038962  0.074904   -0.45404621\n",
      "   0.17276465  0.08533319 -0.26780447 -0.42183499  0.45066783  0.28785357\n",
      "  -0.12311765 -0.26215977 -0.14190441 -0.06350803  0.59800798 -0.27311799\n",
      "  -0.07005313  0.24581239 -0.02506535  0.66728815  0.49315319  0.25244488\n",
      "  -0.53609773 -0.20276406 -0.4395259   0.52307151  0.27123389  0.59009889\n",
      "   0.13005129 -0.07575204 -0.24835049  0.41570204  0.1944786   0.56131422\n",
      "  -0.30361744 -0.04499823  0.42218757 -0.32089018 -0.44872858 -0.1950293\n",
      "   0.28684831 -0.20989329  0.34504339 -0.3944279   0.2152049  -0.06904433\n",
      "  -0.02871868  0.74107814 -0.33205668  0.00356441  0.36138959  0.09007633\n",
      "   0.62657503 -0.18394777 -0.28365878  0.0362597  -0.05647161  0.32828055\n",
      "  -0.26202239 -0.23897901  0.12566088  0.0395881   0.49611338  0.46523347\n",
      "   0.37565485 -0.10027755  0.19294874  0.05238363  0.21949654 -0.15164204\n",
      "  -0.17127383  0.10476681  0.23120833 -0.38808591]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.06557069 -0.13064684 -0.08877014 -0.23126212  0.62020128  0.01277974\n",
      "   0.3498363   0.00200992  0.56238168  0.00103743  0.55397871  0.06876837\n",
      "   0.16935908 -0.04874645  0.27393562 -0.07292186 -0.36384489 -0.2603845\n",
      "  -0.20860693 -0.00222444  0.19518744  0.15892135  0.08558243 -0.26283135\n",
      "   0.54125263  0.33146419  0.3520647  -0.03705227  0.08554783 -0.44099526\n",
      "   0.18406588  0.09688311 -0.25468168 -0.40918231  0.45762983  0.29541029\n",
      "  -0.1118822  -0.24918643 -0.12966498 -0.0516859   0.60607207 -0.26209031\n",
      "  -0.05953289  0.25597197 -0.01462403  0.67429184  0.50099586  0.26326475\n",
      "  -0.52334946 -0.19101289 -0.42646508  0.53068853  0.28144287  0.59853601\n",
      "   0.1387695  -0.06281337 -0.23514524  0.42259911  0.20218784  0.5681971\n",
      "  -0.29130572 -0.03276463  0.42922897 -0.30849125 -0.43624927 -0.18412662\n",
      "   0.2965125  -0.19767934  0.35510573 -0.38130501  0.22550959 -0.05609677\n",
      "  -0.01733231  0.74838564 -0.32073307  0.01416933  0.37102915  0.09889527\n",
      "   0.63488958 -0.1723135  -0.27158951  0.04779503 -0.04481163  0.33913726\n",
      "  -0.25073802 -0.2257976   0.13704371  0.05214982  0.50500928  0.47333764\n",
      "   0.3830015  -0.09014083  0.2008652   0.06447559  0.23104513 -0.13969511\n",
      "  -0.16066044  0.11530812  0.24094389 -0.37577678]]\n",
      "[[False False False False  True  True  True  True  True  True  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.07428943 -0.13900613 -0.09576149 -0.24440089  0.61440685  0.00508084\n",
      "   0.3349157  -0.00756463  0.55194925 -0.01313152  0.54841528  0.06332192\n",
      "   0.16506146 -0.05549714  0.25829607 -0.07781935 -0.37716348 -0.27144307\n",
      "  -0.22314086 -0.007289    0.19162644  0.14997364  0.07610127 -0.2780644\n",
      "   0.53379847  0.31597599  0.33967605 -0.04122514  0.07614114 -0.45412983\n",
      "   0.17852562  0.0902969  -0.26385511 -0.42281489  0.44519106  0.28140289\n",
      "  -0.12357615 -0.25867663 -0.13884539 -0.06048818  0.59977815 -0.27740641\n",
      "  -0.07221756  0.24861533 -0.02662921  0.66633966  0.49195049  0.25794396\n",
      "  -0.53900987 -0.20282662 -0.43927849  0.52159619  0.27474098  0.59315729\n",
      "   0.12492719 -0.06864596 -0.24368902  0.4092685   0.1867376   0.55776077\n",
      "  -0.30353757 -0.04000846  0.41644486 -0.32082005 -0.45092338 -0.19822872\n",
      "   0.28855595 -0.20830362  0.34946921 -0.39303056  0.21795404 -0.06176858\n",
      "  -0.02668865  0.74279607 -0.3363847   0.00321257  0.3645042   0.08453737\n",
      "   0.629893   -0.18408469 -0.28411749  0.0401776  -0.05393918  0.33545123\n",
      "  -0.265091   -0.23422115  0.13078903  0.04755627  0.49905768  0.46448279\n",
      "   0.37015876 -0.10453956  0.18598109  0.05878669  0.22715948 -0.14991441\n",
      "  -0.17511707  0.10620809  0.23207136 -0.38971877]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.06234025 -0.12647274 -0.08304499 -0.23282931  0.62254599  0.01683443\n",
      "   0.34176968  0.00360916  0.55887449 -0.00355697  0.55710181  0.07546839\n",
      "   0.17690072 -0.04297854  0.26543356 -0.06448256 -0.3647205  -0.25893908\n",
      "  -0.21221663  0.00549121  0.2035412   0.16026089  0.08671881 -0.26700248\n",
      "   0.54191371  0.32276101  0.34739944 -0.02788779  0.08678497 -0.44107888\n",
      "   0.18982684  0.10184683 -0.25073232 -0.41016221  0.45215306  0.28895961\n",
      "  -0.11234071 -0.24570329 -0.12660597 -0.04866605  0.60784224 -0.26637873\n",
      "  -0.06169732  0.25877491 -0.0161879   0.67334335  0.49979317  0.26876383\n",
      "  -0.5262616  -0.19107545 -0.42621767  0.52921321  0.28494996  0.60159442\n",
      "   0.13364541 -0.05570729 -0.23048377  0.41616557  0.19444684  0.56464366\n",
      "  -0.29122585 -0.02777486  0.42348626 -0.30842112 -0.43844406 -0.18732604\n",
      "   0.29822014 -0.19608968  0.35953155 -0.37990767  0.22825873 -0.04882103\n",
      "  -0.01530228  0.75010357 -0.32506109  0.0138175   0.37414376  0.09335632\n",
      "   0.63820755 -0.17245041 -0.27204822  0.05171293 -0.04227919  0.34630794\n",
      "  -0.25380664 -0.22103974  0.14217185  0.06011799  0.50795357  0.47258695\n",
      "   0.37750541 -0.09440284  0.19389756  0.07087865  0.23870807 -0.13796748\n",
      "  -0.16450368  0.11674941  0.24180692 -0.37740964]]\n",
      "[[False False False False  True  True  True  True  True False  True  True\n",
      "   True False  True False False False False  True  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.07751933 -0.14318007 -0.10145939 -0.2429237   0.6121793   0.00105153\n",
      "   0.34291963 -0.00917105  0.55549004 -0.00862154  0.54540669  0.05669061\n",
      "   0.15761773 -0.06122982  0.26671589 -0.08619468 -0.37639393 -0.27294642\n",
      "  -0.21964234 -0.01493652  0.1833855   0.1486529   0.0749675  -0.27402154\n",
      "   0.53321891  0.32460513  0.34432134 -0.05030994  0.07490787 -0.45415678\n",
      "   0.17284314  0.08538553 -0.26783019 -0.42195079  0.45065742  0.28780084\n",
      "  -0.12317163 -0.26219027 -0.14191794 -0.06350753  0.59811536 -0.27324758\n",
      "  -0.07011859  0.24586746 -0.02511497  0.66737441  0.49320402  0.25253485\n",
      "  -0.53625883 -0.20282784 -0.43962965  0.52312448  0.27130246  0.5902209\n",
      "   0.12998591 -0.07570277 -0.24836371  0.4156732   0.19439247  0.56134831\n",
      "  -0.30369811 -0.0449697   0.42216856 -0.32097417 -0.44886454 -0.19513083\n",
      "   0.2868973  -0.20993775  0.3451371  -0.39450893  0.21525363 -0.0689917\n",
      "  -0.0287241   0.74121136 -0.33219768  0.00353522  0.36146995  0.08999837\n",
      "   0.62670703 -0.184009   -0.28374249  0.03628989 -0.05647589  0.32840543\n",
      "  -0.26213468 -0.2389893   0.12572274  0.03966949  0.49621656  0.46528478\n",
      "   0.37563031 -0.1003748   0.19287199  0.05244784  0.21960743 -0.151674\n",
      "  -0.171379    0.10477875  0.23123646 -0.38820361]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.06557016 -0.13064668 -0.08874288 -0.23135212  0.62031844  0.01280512\n",
      "   0.34977361  0.00200274  0.56241527  0.00095301  0.55409322  0.06883707\n",
      "   0.16945698 -0.04871122  0.27385338 -0.07285789 -0.36395096 -0.26044243\n",
      "  -0.20871811 -0.00215631  0.19530025  0.15894015  0.08558505 -0.26295962\n",
      "   0.54133415  0.33139015  0.35204473 -0.0369726   0.08555169 -0.44110583\n",
      "   0.18414436  0.09693545 -0.2547074  -0.40929811  0.45761942  0.29535757\n",
      "  -0.11193618 -0.24921693 -0.12967851 -0.0516854   0.60617945 -0.2622199\n",
      "  -0.05959835  0.25602703 -0.01467366  0.6743781   0.50104669  0.26335472\n",
      "  -0.52351055 -0.19107667 -0.42656883  0.5307415   0.28151144  0.59865802\n",
      "   0.13870413 -0.0627641  -0.23515846  0.42257028  0.20210171  0.5682312\n",
      "  -0.29138639 -0.0327361   0.42920996 -0.30857525 -0.43638523 -0.18422814\n",
      "   0.29656149 -0.19772381  0.35519944 -0.38138604  0.22555832 -0.05604415\n",
      "  -0.01733773  0.74851886 -0.32087407  0.01414014  0.37110951  0.09881732\n",
      "   0.63502158 -0.17237472 -0.27167322  0.04782522 -0.04481591  0.33926214\n",
      "  -0.25085032 -0.22580789  0.13710556  0.05223121  0.50511246  0.47338895\n",
      "   0.38297696 -0.09023808  0.20078845  0.0645398   0.23115603 -0.13972707\n",
      "  -0.16076561  0.11532007  0.24097202 -0.37589449]]\n",
      "[[False False False False  True  True  True  True  True  True  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.07428889 -0.13900596 -0.09573424 -0.24449088  0.61452401  0.00510621\n",
      "   0.33485301 -0.00757181  0.55198284 -0.01321594  0.5485298   0.06339063\n",
      "   0.16515936 -0.05546191  0.25821383 -0.07775538 -0.37726954 -0.271501\n",
      "  -0.22325205 -0.00722087  0.19173926  0.14999244  0.07610388 -0.27819267\n",
      "   0.53387998  0.31590195  0.33965609 -0.04114547  0.07614501 -0.4542404\n",
      "   0.1786041   0.09034924 -0.26388083 -0.42293069  0.44518065  0.28135017\n",
      "  -0.12363013 -0.25870713 -0.13885892 -0.06048768  0.59988553 -0.27753599\n",
      "  -0.07228302  0.2486704  -0.02667884  0.66642592  0.49200133  0.25803393\n",
      "  -0.53917097 -0.2028904  -0.43938224  0.52164916  0.27480955  0.5932793\n",
      "   0.12486182 -0.06859669 -0.24370224  0.40923966  0.18665147  0.55779487\n",
      "  -0.30361824 -0.03997993  0.41642585 -0.32090405 -0.45105933 -0.19833025\n",
      "   0.28860494 -0.20834809  0.34956292 -0.3931116   0.21800276 -0.06171595\n",
      "  -0.02669407  0.74292929 -0.33652569  0.00318338  0.36458456  0.08445942\n",
      "   0.630025   -0.18414591 -0.2842012   0.04020779 -0.05394346  0.33557612\n",
      "  -0.2652033  -0.23423143  0.13085088  0.04763766  0.49916086  0.4645341\n",
      "   0.37013421 -0.10463681  0.18590434  0.0588509   0.22727037 -0.14994637\n",
      "  -0.17522224  0.10622003  0.2320995  -0.38983648]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.06233972 -0.12647257 -0.08301773 -0.2329193   0.62266315  0.0168598\n",
      "   0.34170699  0.00360198  0.55890808 -0.00364139  0.55721633  0.07553709\n",
      "   0.17699862 -0.0429433   0.26535132 -0.06441859 -0.36482657 -0.25899701\n",
      "  -0.21232782  0.00555934  0.20365401  0.16027968  0.08672143 -0.26713074\n",
      "   0.54199523  0.32268697  0.34737948 -0.02780812  0.08678883 -0.44118946\n",
      "   0.18990533  0.10189916 -0.25075803 -0.41027801  0.45214265  0.28890689\n",
      "  -0.11239469 -0.24573378 -0.12661949 -0.04866555  0.60794962 -0.26650832\n",
      "  -0.06176278  0.25882997 -0.01623753  0.67342961  0.499844    0.2688538\n",
      "  -0.52642269 -0.19113923 -0.42632142  0.52926618  0.28501853  0.60171642\n",
      "   0.13358003 -0.05565802 -0.23049699  0.41613674  0.19436071  0.56467776\n",
      "  -0.29130652 -0.02774633  0.42346726 -0.30850512 -0.43858002 -0.18742757\n",
      "   0.29826913 -0.19613414  0.35962526 -0.37998871  0.22830745 -0.0487684\n",
      "  -0.0153077   0.75023679 -0.32520208  0.01378831  0.37422412  0.09327837\n",
      "   0.63833955 -0.17251164 -0.27213193  0.05174312 -0.04228348  0.34643283\n",
      "  -0.25391894 -0.22105002  0.14223371  0.06019938  0.50805675  0.47263826\n",
      "   0.37748086 -0.09450009  0.19382081  0.07094287  0.23881896 -0.13799944\n",
      "  -0.16460885  0.11676135  0.24183505 -0.37752735]]\n",
      "[[False False False False  True  True  True  True  True False  True  True\n",
      "   True False  True False False False False  True  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.0775188  -0.1431799  -0.10143213 -0.2430137   0.61229645  0.0010769\n",
      "   0.34285693 -0.00917823  0.55552363 -0.00870596  0.54552121  0.05675931\n",
      "   0.15771563 -0.06119458  0.26663365 -0.08613071 -0.3765     -0.27300435\n",
      "  -0.21975353 -0.01486839  0.18349831  0.1486717   0.07497012 -0.27414981\n",
      "   0.53330042  0.3245311   0.34430138 -0.05023027  0.07491173 -0.45426735\n",
      "   0.17292162  0.08543786 -0.26785591 -0.42206659  0.450647    0.28774812\n",
      "  -0.12322561 -0.26222077 -0.14193146 -0.06350703  0.59822274 -0.27337717\n",
      "  -0.07018405  0.24592252 -0.0251646   0.66746066  0.49325486  0.25262481\n",
      "  -0.53641992 -0.20289162 -0.4397334   0.52317745  0.27137103  0.5903429\n",
      "   0.12992054 -0.0756535  -0.24837693  0.41564437  0.19430635  0.56138241\n",
      "  -0.30377879 -0.04494117  0.42214955 -0.32105817 -0.44900049 -0.19523235\n",
      "   0.28694628 -0.20998222  0.34523081 -0.39458997  0.21530236 -0.06893907\n",
      "  -0.02872952  0.74134458 -0.33233867  0.00350603  0.36155031  0.08992041\n",
      "   0.62683903 -0.18407022 -0.2838262   0.03632008 -0.05648018  0.32853032\n",
      "  -0.26224698 -0.23899958  0.1257846   0.03975088  0.49631974  0.4653361\n",
      "   0.37560577 -0.10047205  0.19279524  0.05251205  0.21971832 -0.15170595\n",
      "  -0.17148417  0.10479069  0.2312646  -0.38832132]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.06556962 -0.13064651 -0.08871563 -0.23144211  0.62043559  0.01283049\n",
      "   0.34971091  0.00199556  0.56244886  0.0008686   0.55420774  0.06890578\n",
      "   0.16955488 -0.04867598  0.27377114 -0.07279392 -0.36405702 -0.26050037\n",
      "  -0.2088293  -0.00208818  0.19541307  0.15895894  0.08558766 -0.26308788\n",
      "   0.54141567  0.33131612  0.35202477 -0.03689293  0.08555556 -0.44121641\n",
      "   0.18422284  0.09698779 -0.25473311 -0.40941391  0.457609    0.29530485\n",
      "  -0.11199016 -0.24924742 -0.12969204 -0.0516849   0.60628683 -0.26234949\n",
      "  -0.05966381  0.2560821  -0.01472329  0.67446435  0.50109753  0.26344468\n",
      "  -0.52367164 -0.19114045 -0.42667258  0.53079447  0.28158     0.59878002\n",
      "   0.13863875 -0.06271483 -0.23517169  0.42254144  0.20201558  0.5682653\n",
      "  -0.29146707 -0.03270757  0.42919096 -0.30865924 -0.43652118 -0.18432967\n",
      "   0.29661047 -0.19776827  0.35529315 -0.38146708  0.22560705 -0.05599152\n",
      "  -0.01734315  0.74865208 -0.32101506  0.01411095  0.37118987  0.09873936\n",
      "   0.63515358 -0.17243595 -0.27175693  0.04785541 -0.04482019  0.33938703\n",
      "  -0.25096262 -0.22581817  0.13716742  0.0523126   0.50521563  0.47344026\n",
      "   0.38295242 -0.09033533  0.2007117   0.06460401  0.23126692 -0.13975902\n",
      "  -0.16087078  0.11533201  0.24100015 -0.3760122 ]]\n",
      "[[False False False False  True  True  True  True  True  True  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.07428836 -0.1390058  -0.09570698 -0.24458088  0.61464117  0.00513159\n",
      "   0.33479031 -0.00757899  0.55201644 -0.01330036  0.54864432  0.06345933\n",
      "   0.16525726 -0.05542667  0.25813159 -0.07769141 -0.37737561 -0.27155893\n",
      "  -0.22336324 -0.00715275  0.19185207  0.15001123  0.0761065  -0.27832093\n",
      "   0.5339615   0.31582792  0.33963613 -0.0410658   0.07614887 -0.45435097\n",
      "   0.17868259  0.09040158 -0.26390655 -0.42304649  0.44517024  0.28129744\n",
      "  -0.12368411 -0.25873762 -0.13887245 -0.06048718  0.59999291 -0.27766558\n",
      "  -0.07234848  0.24872546 -0.02672847  0.66651217  0.49205217  0.2581239\n",
      "  -0.53933206 -0.20295419 -0.43948599  0.52170213  0.27487811  0.5934013\n",
      "   0.12479645 -0.06854742 -0.24371546  0.40921083  0.18656534  0.55782897\n",
      "  -0.30369892 -0.03995139  0.41640685 -0.32098804 -0.45119529 -0.19843178\n",
      "   0.28865392 -0.20839256  0.34965662 -0.39319264  0.21805149 -0.06166332\n",
      "  -0.02669949  0.74306251 -0.33666669  0.00315419  0.36466492  0.08438146\n",
      "   0.630157   -0.18420714 -0.28428491  0.04023798 -0.05394774  0.335701\n",
      "  -0.2653156  -0.23424172  0.13091274  0.04771905  0.49926403  0.46458541\n",
      "   0.37010967 -0.10473407  0.18582759  0.05891512  0.22738126 -0.14997833\n",
      "  -0.17532741  0.10623198  0.23212763 -0.38995418]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.06233918 -0.12647241 -0.08299048 -0.23300929  0.62278031  0.01688517\n",
      "   0.34164429  0.0035948   0.55894167 -0.00372581  0.55733085  0.0756058\n",
      "   0.17709652 -0.04290806  0.26526908 -0.06435461 -0.36493264 -0.25905495\n",
      "  -0.21243901  0.00562747  0.20376683  0.16029848  0.08672405 -0.26725901\n",
      "   0.54207675  0.32261294  0.34735951 -0.02772845  0.0867927  -0.44130003\n",
      "   0.18998381  0.1019515  -0.25078375 -0.41039381  0.45213224  0.28885417\n",
      "  -0.11244867 -0.24576428 -0.12663302 -0.04866505  0.608057   -0.26663791\n",
      "  -0.06182824  0.25888504 -0.01628716  0.67351586  0.49989484  0.26894377\n",
      "  -0.52658378 -0.19120302 -0.42642516  0.52931915  0.28508709  0.60183843\n",
      "   0.13351466 -0.05560875 -0.23051021  0.4161079   0.19427458  0.56471186\n",
      "  -0.2913872  -0.02771779  0.42344825 -0.30858912 -0.43871597 -0.18752909\n",
      "   0.29831811 -0.19617861  0.35971896 -0.38006974  0.22835618 -0.04871577\n",
      "  -0.01531312  0.75037001 -0.32534308  0.01375912  0.37430447  0.09320041\n",
      "   0.63847155 -0.17257286 -0.27221564  0.05177331 -0.04228776  0.34655771\n",
      "  -0.25403124 -0.22106031  0.14229557  0.06028077  0.50815993  0.47268957\n",
      "   0.37745632 -0.09459735  0.19374406  0.07100708  0.23892985 -0.1380314\n",
      "  -0.16471401  0.11677329  0.24186319 -0.37764506]]\n",
      "[[False False False False  True  True  True  True  True False  True  True\n",
      "   True False  True False False False False  True  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.07751826 -0.14317974 -0.10140488 -0.24310369  0.61241361  0.00110228\n",
      "   0.34279423 -0.00918541  0.55555722 -0.00879037  0.54563573  0.05682802\n",
      "   0.15781353 -0.06115934  0.26655141 -0.08606674 -0.37660606 -0.27306228\n",
      "  -0.21986472 -0.01480026  0.18361112  0.14869049  0.07497273 -0.27427807\n",
      "   0.53338194  0.32445706  0.34428142 -0.0501506   0.07491559 -0.45437792\n",
      "   0.1730001   0.0854902  -0.26788163 -0.42218239  0.45063659  0.2876954\n",
      "  -0.12327959 -0.26225126 -0.14194499 -0.06350653  0.59833012 -0.27350675\n",
      "  -0.07024951  0.24597759 -0.02521423  0.66754691  0.49330569  0.25271478\n",
      "  -0.53658101 -0.2029554  -0.43983715  0.52323042  0.27143959  0.59046491\n",
      "   0.12985517 -0.07560423 -0.24839016  0.41561553  0.19422022  0.56141651\n",
      "  -0.30385947 -0.04491264  0.42213055 -0.32114217 -0.44913645 -0.19533388\n",
      "   0.28699526 -0.21002669  0.34532451 -0.39467101  0.21535108 -0.06888644\n",
      "  -0.02873494  0.74147779 -0.33247967  0.00347683  0.36163067  0.08984246\n",
      "   0.62697103 -0.18413145 -0.28390991  0.03635027 -0.05648446  0.32865521\n",
      "  -0.26235928 -0.23900986  0.12584646  0.03983228  0.49642291  0.46538741\n",
      "   0.37558122 -0.10056931  0.19271849  0.05257627  0.21982922 -0.15173791\n",
      "  -0.17158934  0.10480264  0.23129273 -0.38843903]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.06556909 -0.13064635 -0.08868837 -0.2315321   0.62055275  0.01285586\n",
      "   0.34964821  0.00198838  0.56248245  0.00078418  0.55432226  0.06897449\n",
      "   0.16965278 -0.04864074  0.2736889  -0.07272994 -0.36416309 -0.2605583\n",
      "  -0.20894049 -0.00202005  0.19552588  0.15897774  0.08559028 -0.26321615\n",
      "   0.54149719  0.33124208  0.3520048  -0.03681325  0.08555942 -0.44132698\n",
      "   0.18430133  0.09704012 -0.25475883 -0.40952971  0.45759859  0.29525213\n",
      "  -0.11204414 -0.24927792 -0.12970556 -0.0516844   0.60639421 -0.26247908\n",
      "  -0.05972927  0.25613716 -0.01477292  0.6745506   0.50114837  0.26353465\n",
      "  -0.52383273 -0.19120424 -0.42677633  0.53084744  0.28164857  0.59890203\n",
      "   0.13857338 -0.06266556 -0.23518491  0.42251261  0.20192946  0.5682994\n",
      "  -0.29154775 -0.03267904  0.42917195 -0.30874324 -0.43665714 -0.1844312\n",
      "   0.29665946 -0.19781274  0.35538685 -0.38154812  0.22565577 -0.05593889\n",
      "  -0.01734857  0.74878529 -0.32115606  0.01408176  0.37127023  0.09866141\n",
      "   0.63528558 -0.17249717 -0.27184064  0.0478856  -0.04482448  0.33951191\n",
      "  -0.25107492 -0.22582845  0.13722928  0.052394    0.50531881  0.47349157\n",
      "   0.38292787 -0.09043259  0.20063495  0.06466823  0.23137781 -0.13979098\n",
      "  -0.16097595  0.11534395  0.24102829 -0.37612991]]\n",
      "[[False False False False  True  True  True  True  True  True  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.07428782 -0.13900563 -0.09567972 -0.24467087  0.61475833  0.00515696\n",
      "   0.33472761 -0.00758617  0.55205003 -0.01338478  0.54875883  0.06352804\n",
      "   0.16535516 -0.05539143  0.25804935 -0.07762743 -0.37748168 -0.27161686\n",
      "  -0.22347442 -0.00708462  0.19196488  0.15003003  0.07610912 -0.2784492\n",
      "   0.53404302  0.31575388  0.33961616 -0.04098612  0.07615273 -0.45446154\n",
      "   0.17876107  0.09045392 -0.26393227 -0.42316229  0.44515983  0.28124472\n",
      "  -0.12373809 -0.25876812 -0.13888597 -0.06048668  0.60010029 -0.27779517\n",
      "  -0.07241394  0.24878053 -0.0267781   0.66659842  0.492103    0.25821386\n",
      "  -0.53949315 -0.20301797 -0.43958974  0.5217551   0.27494668  0.59352331\n",
      "   0.12473108 -0.06849816 -0.24372868  0.40918199  0.18647922  0.55786307\n",
      "  -0.3037796  -0.03992286  0.41638784 -0.32107204 -0.45133124 -0.1985333\n",
      "   0.28870291 -0.20843702  0.34975033 -0.39327367  0.21810022 -0.06161069\n",
      "  -0.02670491  0.74319573 -0.33680768  0.003125    0.36474527  0.08430351\n",
      "   0.63028901 -0.18426837 -0.28436862  0.04026817 -0.05395203  0.33582589\n",
      "  -0.2654279  -0.234252    0.1309746   0.04780044  0.49936721  0.46463672\n",
      "   0.37008513 -0.10483132  0.18575084  0.05897933  0.22749215 -0.15001029\n",
      "  -0.17543258  0.10624392  0.23215577 -0.39007189]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.06233864 -0.12647224 -0.08296322 -0.23309928  0.62289746  0.01691055\n",
      "   0.34158159  0.00358762  0.55897526 -0.00381023  0.55744536  0.07567451\n",
      "   0.17719442 -0.04287283  0.26518684 -0.06429064 -0.3650387  -0.25911288\n",
      "  -0.21255019  0.0056956   0.20387964  0.16031727  0.08672666 -0.26738727\n",
      "   0.54215826  0.3225389   0.34733955 -0.02764878  0.08679656 -0.4414106\n",
      "   0.1900623   0.10200384 -0.25080947 -0.41050961  0.45212183  0.28880145\n",
      "  -0.11250265 -0.24579478 -0.12664655 -0.04866455  0.60816438 -0.26676749\n",
      "  -0.0618937   0.2589401  -0.01633678  0.67360211  0.49994567  0.26903373\n",
      "  -0.52674487 -0.1912668  -0.42652891  0.52937212  0.28515566  0.60196043\n",
      "   0.13344929 -0.05555948 -0.23052344  0.41607907  0.19418846  0.56474596\n",
      "  -0.29146788 -0.02768926  0.42342925 -0.30867311 -0.43885193 -0.18763062\n",
      "   0.2983671  -0.19622308  0.35981267 -0.38015078  0.22840491 -0.04866314\n",
      "  -0.01531854  0.75050323 -0.32548407  0.01372993  0.37438483  0.09312245\n",
      "   0.63860355 -0.17263409 -0.27229935  0.0518035  -0.04229204  0.3466826\n",
      "  -0.25414353 -0.22107059  0.14235742  0.06036216  0.50826311  0.47274089\n",
      "   0.37743178 -0.0946946   0.1936673   0.07107129  0.23904074 -0.13806336\n",
      "  -0.16481918  0.11678523  0.24189132 -0.37776277]]\n",
      "[[False False False False  True  True  True  True  True False  True  True\n",
      "   True False  True False False False False  True  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.07751772 -0.14317957 -0.10137762 -0.24319368  0.61253077  0.00112765\n",
      "   0.34273154 -0.00919259  0.55559081 -0.00887479  0.54575024  0.05689673\n",
      "   0.15791143 -0.06112411  0.26646917 -0.08600276 -0.37671213 -0.27312022\n",
      "  -0.21997591 -0.01473213  0.18372393  0.14870929  0.07497535 -0.27440634\n",
      "   0.53346346  0.32438303  0.34426145 -0.05007093  0.07491946 -0.4544885\n",
      "   0.17307859  0.08554254 -0.26790735 -0.42229819  0.45062618  0.28764268\n",
      "  -0.12333357 -0.26228176 -0.14195851 -0.06350603  0.5984375  -0.27363634\n",
      "  -0.07031497  0.24603265 -0.02526386  0.66763317  0.49335653  0.25280475\n",
      "  -0.5367421  -0.20301919 -0.4399409   0.52328339  0.27150816  0.59058691\n",
      "   0.1297898  -0.07555497 -0.24840338  0.4155867   0.19413409  0.56145061\n",
      "  -0.30394014 -0.04488411  0.42211154 -0.32122616 -0.44927241 -0.19543541\n",
      "   0.28704425 -0.21007115  0.34541822 -0.39475205  0.21539981 -0.06883381\n",
      "  -0.02874036  0.74161101 -0.33262066  0.00344764  0.36171103  0.0897645\n",
      "   0.62710303 -0.18419268 -0.28399361  0.03638046 -0.05648875  0.32878009\n",
      "  -0.26247158 -0.23902015  0.12590832  0.03991367  0.49652609  0.46543872\n",
      "   0.37555668 -0.10066656  0.19264174  0.05264048  0.21994011 -0.15176987\n",
      "  -0.17169451  0.10481458  0.23132086 -0.38855674]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-6.55685493e-02 -1.30646183e-01 -8.86611170e-02 -2.31622095e-01\n",
      "   6.20669909e-01  1.28812377e-02  3.49585517e-01  1.98120410e-03\n",
      "   5.62516040e-01  6.99760211e-04  5.54436774e-01  6.90431940e-02\n",
      "   1.69750682e-01 -4.86055035e-02  2.73606657e-01 -7.26659709e-02\n",
      "  -3.64269155e-01 -2.60616231e-01 -2.09051677e-01 -1.95192107e-03\n",
      "   1.95638690e-01  1.58996531e-01  8.55928973e-02 -2.63344413e-01\n",
      "   5.41578702e-01  3.31168047e-01  3.51984841e-01 -3.67335827e-02\n",
      "   8.55632834e-02 -4.41437552e-01  1.84379813e-01  9.70924613e-02\n",
      "  -2.54784551e-01 -4.09645515e-01  4.57588182e-01  2.95199404e-01\n",
      "  -1.12098120e-01 -2.49308417e-01 -1.29719088e-01 -5.16838964e-02\n",
      "   6.06501588e-01 -2.62608662e-01 -5.97947314e-02  2.56192230e-01\n",
      "  -1.48225461e-02  6.74636858e-01  5.01199202e-01  2.63624618e-01\n",
      "  -5.23993824e-01 -1.91268017e-01 -4.26880079e-01  5.30900411e-01\n",
      "   2.81717139e-01  5.99024034e-01  1.38508008e-01 -6.26162951e-02\n",
      "  -2.35198132e-01  4.22483774e-01  2.01843330e-01  5.68333494e-01\n",
      "  -2.91628424e-01 -3.26505088e-02  4.29152946e-01 -3.08827237e-01\n",
      "  -4.36793090e-01 -1.84532724e-01  2.96708440e-01 -1.97857205e-01\n",
      "   3.55480561e-01 -3.81629155e-01  2.25704500e-01 -5.58862592e-02\n",
      "  -1.73539902e-02  7.48918513e-01 -3.21297052e-01  1.40525707e-02\n",
      "   3.71350586e-01  9.85834501e-02  6.35417579e-01 -1.72558401e-01\n",
      "  -2.71924343e-01  4.79157913e-02 -4.48287632e-02  3.39636799e-01\n",
      "  -2.51187213e-01 -2.25838738e-01  1.37291140e-01  5.24753867e-02\n",
      "   5.05421988e-01  4.73542883e-01  3.82903331e-01 -9.05298416e-02\n",
      "   2.00558200e-01  6.47324395e-02  2.31488700e-01 -1.39822939e-01\n",
      "  -1.61081115e-01  1.15355896e-01  2.41056419e-01 -3.76247614e-01]]\n",
      "[[False False False False  True  True  True  True  True  True  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.07428728 -0.13900547 -0.09565247 -0.24476086  0.61487548  0.00518234\n",
      "   0.33466492 -0.00759335  0.55208362 -0.0134692   0.54887335  0.06359675\n",
      "   0.16545306 -0.05535619  0.25796711 -0.07756346 -0.37758774 -0.2716748\n",
      "  -0.22358561 -0.00701649  0.19207769  0.15004882  0.07611174 -0.27857747\n",
      "   0.53412453  0.31567985  0.3395962  -0.04090645  0.0761566  -0.45457212\n",
      "   0.17883955  0.09050625 -0.26395799 -0.42327809  0.44514942  0.281192\n",
      "  -0.12379207 -0.25879861 -0.1388995  -0.06048618  0.60020767 -0.27792476\n",
      "  -0.0724794   0.24883559 -0.02682773  0.66668468  0.49215384  0.25830383\n",
      "  -0.53965424 -0.20308175 -0.43969349  0.52180807  0.27501525  0.59364531\n",
      "   0.1246657  -0.06844889 -0.24374191  0.40915316  0.18639309  0.55789716\n",
      "  -0.30386028 -0.03989433  0.41636884 -0.32115604 -0.4514672  -0.19863483\n",
      "   0.28875189 -0.20848149  0.34984404 -0.39335471  0.21814894 -0.06155806\n",
      "  -0.02671033  0.74332895 -0.33694868  0.00309581  0.36482563  0.08422555\n",
      "   0.63042101 -0.18432959 -0.28445232  0.04029836 -0.05395631  0.33595078\n",
      "  -0.2655402  -0.23426228  0.13103646  0.04788183  0.49947039  0.46468803\n",
      "   0.37006059 -0.10492857  0.18567409  0.05904354  0.22760304 -0.15004225\n",
      "  -0.17553775  0.10625586  0.2321839  -0.3901896 ]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.06233811 -0.12647208 -0.08293597 -0.23318928  0.62301462  0.01693592\n",
      "   0.3415189   0.00358044  0.55900885 -0.00389464  0.55755988  0.07574322\n",
      "   0.17729232 -0.04283759  0.2651046  -0.06422667 -0.36514477 -0.25917081\n",
      "  -0.21266138  0.00576373  0.20399245  0.16033607  0.08672928 -0.26751554\n",
      "   0.54223978  0.32246487  0.34731959 -0.02756911  0.08680042 -0.44152117\n",
      "   0.19014078  0.10205618 -0.25083519 -0.41062542  0.45211142  0.28874873\n",
      "  -0.11255663 -0.24582527 -0.12666007 -0.04866405  0.60827176 -0.26689708\n",
      "  -0.06195916  0.25899517 -0.01638641  0.67368837  0.49999651  0.2691237\n",
      "  -0.52690596 -0.19133058 -0.42663266  0.52942509  0.28522423  0.60208244\n",
      "   0.13338392 -0.05551022 -0.23053666  0.41605023  0.19410233  0.56478005\n",
      "  -0.29154855 -0.02766073  0.42341024 -0.30875711 -0.43898788 -0.18773215\n",
      "   0.29841608 -0.19626754  0.35990638 -0.38023182  0.22845363 -0.04861051\n",
      "  -0.01532396  0.75063645 -0.32562507  0.01370074  0.37446519  0.0930445\n",
      "   0.63873555 -0.17269532 -0.27238306  0.05183369 -0.04229633  0.34680748\n",
      "  -0.25425583 -0.22108087  0.14241928  0.06044355  0.50836628  0.4727922\n",
      "   0.37740724 -0.09479185  0.19359055  0.07113551  0.23915163 -0.13809531\n",
      "  -0.16492435  0.11679718  0.24191945 -0.37788048]]\n",
      "[[False False False False  True  True  True  True  True False  True  True\n",
      "   True False  True False False False False  True  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.07751719 -0.14317941 -0.10135036 -0.24328367  0.61264793  0.00115303\n",
      "   0.34266884 -0.00919977  0.5556244  -0.00895921  0.54586476  0.05696543\n",
      "   0.15800933 -0.06108887  0.26638693 -0.08593879 -0.37681819 -0.27317815\n",
      "  -0.22008709 -0.01466401  0.18383674  0.14872808  0.07497797 -0.27453461\n",
      "   0.53354497  0.32430899  0.34424149 -0.04999126  0.07492332 -0.45459907\n",
      "   0.17315707  0.08559488 -0.26793307 -0.422414    0.45061577  0.28758996\n",
      "  -0.12338755 -0.26231225 -0.14197204 -0.06350553  0.59854488 -0.27376593\n",
      "  -0.07038043  0.24608772 -0.02531349  0.66771942  0.49340737  0.25289471\n",
      "  -0.53690319 -0.20308297 -0.44004465  0.52333636  0.27157673  0.59070892\n",
      "   0.12972442 -0.0755057  -0.2484166   0.41555786  0.19404797  0.5614847\n",
      "  -0.30402082 -0.04485558  0.42209254 -0.32131016 -0.44940836 -0.19553694\n",
      "   0.28709323 -0.21011562  0.34551193 -0.39483308  0.21544853 -0.06878118\n",
      "  -0.02874578  0.74174423 -0.33276166  0.00341845  0.36179139  0.08968655\n",
      "   0.62723503 -0.1842539  -0.28407732  0.03641065 -0.05649303  0.32890498\n",
      "  -0.26258387 -0.23903043  0.12597017  0.03999506  0.49662927  0.46549003\n",
      "   0.37553214 -0.10076381  0.19256499  0.05270469  0.220051   -0.15180183\n",
      "  -0.17179968  0.10482652  0.231349   -0.38867445]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-6.55680132e-02 -1.30646019e-01 -8.86338612e-02 -2.31712088e-01\n",
      "   6.20787067e-01  1.29066118e-02  3.49522821e-01  1.97402451e-03\n",
      "   5.62549630e-01  6.15342022e-04  5.54551291e-01  6.91119008e-02\n",
      "   1.69848583e-01 -4.85702661e-02  2.73524416e-01 -7.26019975e-02\n",
      "  -3.64375221e-01 -2.60674164e-01 -2.09162865e-01 -1.88379216e-03\n",
      "   1.95751502e-01  1.59015326e-01  8.55955143e-02 -2.63472678e-01\n",
      "   5.41660219e-01  3.31094012e-01  3.51964878e-01 -3.66539107e-02\n",
      "   8.55671475e-02 -4.41548125e-01  1.84458296e-01  9.71447990e-02\n",
      "  -2.54810269e-01 -4.09761317e-01  4.57577770e-01  2.95146682e-01\n",
      "  -1.12152100e-01 -2.49338913e-01 -1.29732614e-01 -5.16833960e-02\n",
      "   6.06608968e-01 -2.62738249e-01 -5.98601912e-02  2.56247295e-01\n",
      "  -1.48721752e-02  6.74723112e-01  5.01250038e-01  2.63714585e-01\n",
      "  -5.24154916e-01 -1.91331799e-01 -4.26983828e-01  5.30953381e-01\n",
      "   2.81785706e-01  5.99146039e-01  1.38442636e-01 -6.25670273e-02\n",
      "  -2.35211355e-01  4.22454939e-01  2.01757204e-01  5.68367591e-01\n",
      "  -2.91709101e-01 -3.26219779e-02  4.29133939e-01 -3.08911234e-01\n",
      "  -4.36929045e-01 -1.84634251e-01  2.96757424e-01 -1.97901672e-01\n",
      "   3.55574269e-01 -3.81710193e-01  2.25753226e-01 -5.58336304e-02\n",
      "  -1.73594104e-02  7.49051732e-01 -3.21438047e-01  1.40233802e-02\n",
      "   3.71430945e-01  9.85054946e-02  6.35549579e-01 -1.72619627e-01\n",
      "  -2.72008051e-01  4.79459810e-02 -4.48330473e-02  3.39761685e-01\n",
      "  -2.51299511e-01 -2.25849021e-01  1.37352998e-01  5.25567779e-02\n",
      "   5.05525165e-01  4.73594194e-01  3.82878789e-01 -9.06270953e-02\n",
      "   2.00481449e-01  6.47966529e-02  2.31599591e-01 -1.39854897e-01\n",
      "  -1.61186284e-01  1.15367840e-01  2.41084552e-01 -3.76365323e-01]]\n",
      "[[False False False False  True  True  True  True  True  True  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.07428675 -0.13900531 -0.09562521 -0.24485086  0.61499264  0.00520771\n",
      "   0.33460222 -0.00760053  0.55211721 -0.01355362  0.54898787  0.06366546\n",
      "   0.16555097 -0.05532096  0.25788487 -0.07749948 -0.37769381 -0.27173273\n",
      "  -0.2236968  -0.00694836  0.1921905   0.15006762  0.07611435 -0.27870573\n",
      "   0.53420605  0.31560581  0.33957624 -0.04082678  0.07616046 -0.45468269\n",
      "   0.17891804  0.09055859 -0.2639837  -0.4233939   0.445139    0.28113928\n",
      "  -0.12384605 -0.25882911 -0.13891302 -0.06048568  0.60031505 -0.27805434\n",
      "  -0.07254486  0.24889066 -0.02687736  0.66677093  0.49220467  0.2583938\n",
      "  -0.53981533 -0.20314553 -0.43979724  0.52186104  0.27508382  0.59376732\n",
      "   0.12460033 -0.06839962 -0.24375513  0.40912432  0.18630697  0.55793126\n",
      "  -0.30394095 -0.0398658   0.41634983 -0.32124003 -0.45160315 -0.19873636\n",
      "   0.28880087 -0.20852596  0.34993775 -0.39343575  0.21819767 -0.06150543\n",
      "  -0.02671575  0.74346217 -0.33708967  0.00306662  0.36490599  0.0841476\n",
      "   0.63055301 -0.18439082 -0.28453603  0.04032855 -0.0539606   0.33607566\n",
      "  -0.26565249 -0.23427257  0.13109832  0.04796322  0.49957356  0.46473934\n",
      "   0.37003605 -0.10502583  0.18559734  0.05910776  0.22771393 -0.1500742\n",
      "  -0.17564292  0.10626781  0.23221203 -0.39030731]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.06233757 -0.12647192 -0.08290871 -0.23327927  0.62313178  0.0169613\n",
      "   0.3414562   0.00357327  0.55904244 -0.00397906  0.5576744   0.07581192\n",
      "   0.17739022 -0.04280235  0.26502236 -0.06416269 -0.36525083 -0.25922874\n",
      "  -0.21277257  0.00583185  0.20410526  0.16035486  0.0867319  -0.2676438\n",
      "   0.5423213   0.32239083  0.34729962 -0.02748943  0.08680429 -0.44163175\n",
      "   0.19021926  0.10210851 -0.25086091 -0.41074122  0.452101    0.288696\n",
      "  -0.11261061 -0.24585577 -0.1266736  -0.04866355  0.60837914 -0.26702667\n",
      "  -0.06202462  0.25905024 -0.01643604  0.67377462  0.50004735  0.26921367\n",
      "  -0.52706706 -0.19139436 -0.42673641  0.52947806  0.28529279  0.60220444\n",
      "   0.13331854 -0.05546095 -0.23054988  0.4160214   0.1940162   0.56481415\n",
      "  -0.29162923 -0.0276322   0.42339123 -0.30884111 -0.43912384 -0.18783367\n",
      "   0.29846507 -0.19631201  0.36000009 -0.38031286  0.22850236 -0.04855788\n",
      "  -0.01532938  0.75076967 -0.32576606  0.01367155  0.37454555  0.09296654\n",
      "   0.63886755 -0.17275654 -0.27246676  0.05186388 -0.04230061  0.34693237\n",
      "  -0.25436813 -0.22109116  0.14248114  0.06052494  0.50846946  0.47284351\n",
      "   0.3773827  -0.09488911  0.1935138   0.07119972  0.23926253 -0.13812727\n",
      "  -0.16502952  0.11680912  0.24194759 -0.37799818]]\n",
      "[[False False False False  True  True  True  True  True False  True  True\n",
      "   True False  True False False False False  True  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.07751665 -0.14317924 -0.10132311 -0.24337367  0.61276509  0.0011784\n",
      "   0.34260615 -0.00920695  0.55565799 -0.00904363  0.54597928  0.05703414\n",
      "   0.15810723 -0.06105363  0.26630469 -0.08587482 -0.37692426 -0.27323608\n",
      "  -0.22019828 -0.01459588  0.18394955  0.14874688  0.07498059 -0.27466287\n",
      "   0.53362649  0.32423496  0.34422153 -0.04991158  0.07492719 -0.45470964\n",
      "   0.17323555  0.08564721 -0.26795878 -0.4225298   0.45060536  0.28753724\n",
      "  -0.12344153 -0.26234275 -0.14198557 -0.06350503  0.59865226 -0.27389551\n",
      "  -0.07044589  0.24614278 -0.02536312  0.66780568  0.4934582   0.25298468\n",
      "  -0.53706428 -0.20314675 -0.4401484   0.52338933  0.27164529  0.59083092\n",
      "   0.12965905 -0.07545643 -0.24842982  0.41552903  0.19396184  0.5615188\n",
      "  -0.3041015  -0.04482705  0.42207353 -0.32139416 -0.44954432 -0.19563846\n",
      "   0.28714222 -0.21016009  0.34560564 -0.39491412  0.21549726 -0.06872855\n",
      "  -0.0287512   0.74187745 -0.33290265  0.00338926  0.36187174  0.08960859\n",
      "   0.62736703 -0.18431513 -0.28416103  0.03644084 -0.05649732  0.32902986\n",
      "  -0.26269617 -0.23904071  0.12603203  0.04007645  0.49673245  0.46554134\n",
      "   0.3755076  -0.10086107  0.19248824  0.05276891  0.22016189 -0.15183379\n",
      "  -0.17190485  0.10483847  0.23137713 -0.38879216]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-6.55674770e-02 -1.30645855e-01 -8.86066055e-02 -2.31802081e-01\n",
      "   6.20904226e-01  1.29319860e-02  3.49460125e-01  1.96684493e-03\n",
      "   5.62583220e-01  5.30923833e-04  5.54665807e-01  6.91806077e-02\n",
      "   1.69946483e-01 -4.85350288e-02  2.73442175e-01 -7.25380240e-02\n",
      "  -3.64481288e-01 -2.60732096e-01 -2.09274053e-01 -1.81566325e-03\n",
      "   1.95864313e-01  1.59034121e-01  8.55981313e-02 -2.63600944e-01\n",
      "   5.41741735e-01  3.31019976e-01  3.51944914e-01 -3.65742388e-02\n",
      "   8.55710117e-02 -4.41658697e-01  1.84536780e-01  9.71971368e-02\n",
      "  -2.54835988e-01 -4.09877120e-01  4.57567359e-01  2.95093961e-01\n",
      "  -1.12206079e-01 -2.49369409e-01 -1.29746141e-01 -5.16828956e-02\n",
      "   6.06716348e-01 -2.62867836e-01 -5.99256509e-02  2.56302361e-01\n",
      "  -1.49218044e-02  6.74809365e-01  5.01300874e-01  2.63804552e-01\n",
      "  -5.24316007e-01 -1.91395581e-01 -4.27087578e-01  5.31006352e-01\n",
      "   2.81854273e-01  5.99268044e-01  1.38377263e-01 -6.25177596e-02\n",
      "  -2.35224577e-01  4.22426104e-01  2.01671077e-01  5.68401688e-01\n",
      "  -2.91789778e-01 -3.25934470e-02  4.29114933e-01 -3.08995231e-01\n",
      "  -4.37065001e-01 -1.84735778e-01  2.96806408e-01 -1.97946139e-01\n",
      "   3.55667978e-01 -3.81791230e-01  2.25801952e-01 -5.57810017e-02\n",
      "  -1.73648305e-02  7.49184950e-01 -3.21579041e-01  1.39941898e-02\n",
      "   3.71511303e-01  9.84275390e-02  6.35681580e-01 -1.72680853e-01\n",
      "  -2.72091759e-01  4.79761707e-02 -4.48373315e-02  3.39886571e-01\n",
      "  -2.51411809e-01 -2.25859304e-01  1.37414857e-01  5.26381690e-02\n",
      "   5.05628342e-01  4.73645506e-01  3.82854247e-01 -9.07243489e-02\n",
      "   2.00404698e-01  6.48608662e-02  2.31710482e-01 -1.39886854e-01\n",
      "  -1.61291453e-01  1.15379783e-01  2.41112686e-01 -3.76483031e-01]]\n",
      "[[False False False False  True  True  True  True  True  True  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.07428621 -0.13900514 -0.09559796 -0.24494085  0.6151098   0.00523308\n",
      "   0.33453953 -0.00760771  0.5521508  -0.01363803  0.54910238  0.06373416\n",
      "   0.16564887 -0.05528572  0.25780263 -0.07743551 -0.37779987 -0.27179066\n",
      "  -0.22380799 -0.00688023  0.19230332  0.15008641  0.07611697 -0.278834\n",
      "   0.53428757  0.31553178  0.33955627 -0.04074711  0.07616433 -0.45479326\n",
      "   0.17899652  0.09061093 -0.26400942 -0.4235097   0.44512859  0.28108656\n",
      "  -0.12390003 -0.25885961 -0.13892655 -0.06048518  0.60042243 -0.27818393\n",
      "  -0.07261032  0.24894573 -0.02692699  0.66685719  0.49225551  0.25848376\n",
      "  -0.53997642 -0.20320931 -0.43990099  0.52191401  0.27515238  0.59388932\n",
      "   0.12453496 -0.06835035 -0.24376835  0.40909549  0.18622084  0.55796536\n",
      "  -0.30402163 -0.03983727  0.41633082 -0.32132403 -0.45173911 -0.19883789\n",
      "   0.28884986 -0.20857042  0.35003146 -0.39351679  0.2182464  -0.06145281\n",
      "  -0.02672117  0.74359539 -0.33723067  0.00303743  0.36498635  0.08406964\n",
      "   0.63068501 -0.18445204 -0.28461974  0.04035874 -0.05396488  0.33620055\n",
      "  -0.26576479 -0.23428285  0.13116018  0.04804462  0.49967674  0.46479066\n",
      "   0.3700115  -0.10512308  0.18552059  0.05917197  0.22782482 -0.15010616\n",
      "  -0.17574809  0.10627975  0.23224017 -0.39042502]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.06233704 -0.12647175 -0.08288145 -0.23336926  0.62324894  0.01698667\n",
      "   0.34139351  0.00356609  0.55907603 -0.00406348  0.55778891  0.07588063\n",
      "   0.17748812 -0.04276711  0.26494012 -0.06409872 -0.3653569  -0.25928668\n",
      "  -0.21288376  0.00589998  0.20421807  0.16037366  0.08673452 -0.26777207\n",
      "   0.54240281  0.32231679  0.34727966 -0.02740976  0.08680815 -0.44174232\n",
      "   0.19029775  0.10216085 -0.25088663 -0.41085702  0.45209059  0.28864328\n",
      "  -0.11266459 -0.24588626 -0.12668712 -0.04866305  0.60848652 -0.26715625\n",
      "  -0.06209008  0.2591053  -0.01648567  0.67386087  0.50009818  0.26930363\n",
      "  -0.52722815 -0.19145814 -0.42684016  0.52953104  0.28536136  0.60232645\n",
      "   0.13325317 -0.05541168 -0.2305631   0.41599256  0.19393008  0.56484825\n",
      "  -0.29170991 -0.02760367  0.42337223 -0.3089251  -0.43925979 -0.1879352\n",
      "   0.29851405 -0.19635648  0.3600938  -0.38039389  0.22855109 -0.04850526\n",
      "  -0.0153348   0.75090289 -0.32590706  0.01364236  0.37462591  0.09288859\n",
      "   0.63899955 -0.17281777 -0.27255047  0.05189407 -0.0423049   0.34705726\n",
      "  -0.25448043 -0.22110144  0.142543    0.06060633  0.50857264  0.47289482\n",
      "   0.37735815 -0.09498636  0.19343705  0.07126393  0.23937342 -0.13815923\n",
      "  -0.16513469  0.11682106  0.24197572 -0.37811589]]\n",
      "[[False False False False  True  True  True  True  True False  True  True\n",
      "   True False  True False False False False  True  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.07751612 -0.14317908 -0.10129585 -0.24346366  0.61288225  0.00120377\n",
      "   0.34254345 -0.00921413  0.55569158 -0.00912805  0.54609379  0.05710285\n",
      "   0.15820513 -0.0610184   0.26622245 -0.08581084 -0.37703033 -0.27329401\n",
      "  -0.22030947 -0.01452775  0.18406237  0.14876567  0.0749832  -0.27479114\n",
      "   0.53370801  0.32416092  0.34420156 -0.04983191  0.07493105 -0.45482021\n",
      "   0.17331404  0.08569955 -0.2679845  -0.4226456   0.45059495  0.28748451\n",
      "  -0.1234955  -0.26237325 -0.14199909 -0.06350453  0.59875964 -0.2740251\n",
      "  -0.07051135  0.24619785 -0.02541275  0.66789193  0.49350904  0.25307465\n",
      "  -0.53722538 -0.20321053 -0.44025215  0.5234423   0.27171386  0.59095292\n",
      "   0.12959368 -0.07540716 -0.24844305  0.41550019  0.19387571  0.5615529\n",
      "  -0.30418218 -0.04479852  0.42205452 -0.32147815 -0.44968027 -0.19573999\n",
      "   0.2871912  -0.21020455  0.34569935 -0.39499516  0.21554599 -0.06867592\n",
      "  -0.02875662  0.74201067 -0.33304365  0.00336007  0.3619521   0.08953064\n",
      "   0.62749903 -0.18437635 -0.28424474  0.03647103 -0.0565016   0.32915475\n",
      "  -0.26280847 -0.239051    0.12609389  0.04015784  0.49683562  0.46559265\n",
      "   0.37548306 -0.10095832  0.19241148  0.05283312  0.22027278 -0.15186574\n",
      "  -0.17201002  0.10485041  0.23140526 -0.38890986]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-6.55669409e-02 -1.30645691e-01 -8.85793497e-02 -2.31892074e-01\n",
      "   6.21021384e-01  1.29573601e-02  3.49397429e-01  1.95966534e-03\n",
      "   5.62616811e-01  4.46505644e-04  5.54780324e-01  6.92493146e-02\n",
      "   1.70044384e-01 -4.84997915e-02  2.73359935e-01 -7.24740506e-02\n",
      "  -3.64587354e-01 -2.60790028e-01 -2.09385241e-01 -1.74753433e-03\n",
      "   1.95977125e-01  1.59052916e-01  8.56007484e-02 -2.63729209e-01\n",
      "   5.41823252e-01  3.30945941e-01  3.51924951e-01 -3.64945669e-02\n",
      "   8.55748758e-02 -4.41769270e-01  1.84615264e-01  9.72494746e-02\n",
      "  -2.54861706e-01 -4.09992922e-01  4.57556948e-01  2.95041239e-01\n",
      "  -1.12260058e-01 -2.49399905e-01 -1.29759667e-01 -5.16823951e-02\n",
      "   6.06823728e-01 -2.62997423e-01 -5.99911107e-02  2.56357427e-01\n",
      "  -1.49714335e-02  6.74895619e-01  5.01351710e-01  2.63894519e-01\n",
      "  -5.24477099e-01 -1.91459362e-01 -4.27191327e-01  5.31059323e-01\n",
      "   2.81922840e-01  5.99390048e-01  1.38311891e-01 -6.24684919e-02\n",
      "  -2.35237799e-01  4.22397269e-01  2.01584951e-01  5.68435786e-01\n",
      "  -2.91870455e-01 -3.25649162e-02  4.29095927e-01 -3.09079228e-01\n",
      "  -4.37200956e-01 -1.84837305e-01  2.96855393e-01 -1.97990605e-01\n",
      "   3.55761686e-01 -3.81872267e-01  2.25850679e-01 -5.57283729e-02\n",
      "  -1.73702507e-02  7.49318169e-01 -3.21720036e-01  1.39649994e-02\n",
      "   3.71591662e-01  9.83495834e-02  6.35813581e-01 -1.72742080e-01\n",
      "  -2.72175467e-01  4.80063604e-02 -4.48416157e-02  3.40011457e-01\n",
      "  -2.51524107e-01 -2.25869588e-01  1.37476715e-01  5.27195602e-02\n",
      "   5.05731520e-01  4.73696817e-01  3.82829705e-01 -9.08216025e-02\n",
      "   2.00327947e-01  6.49250796e-02  2.31821373e-01 -1.39918812e-01\n",
      "  -1.61396622e-01  1.15391726e-01  2.41140819e-01 -3.76600739e-01]]\n",
      "[[False False False False  True  True  True  True  True  True  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.07428568 -0.13900498 -0.0955707  -0.24503084  0.61522696  0.00525846\n",
      "   0.33447683 -0.00761489  0.55218439 -0.01372245  0.5492169   0.06380287\n",
      "   0.16574677 -0.05525048  0.25772039 -0.07737154 -0.37790594 -0.27184859\n",
      "  -0.22391918 -0.0068121   0.19241613  0.15010521  0.07611959 -0.27896226\n",
      "   0.53436909  0.31545774  0.33953631 -0.04066744  0.07616819 -0.45490384\n",
      "   0.17907501  0.09066327 -0.26403514 -0.4236255   0.44511818  0.28103384\n",
      "  -0.12395401 -0.2588901  -0.13894008 -0.06048468  0.60052981 -0.27831352\n",
      "  -0.07267578  0.24900079 -0.02697662  0.66694344  0.49230635  0.25857373\n",
      "  -0.54013752 -0.20327309 -0.44000474  0.52196698  0.27522095  0.59401133\n",
      "   0.12446959 -0.06830108 -0.24378157  0.40906665  0.18613471  0.55799946\n",
      "  -0.30410231 -0.03980874  0.41631182 -0.32140803 -0.45187506 -0.19893941\n",
      "   0.28889884 -0.20861489  0.35012517 -0.39359782  0.21829512 -0.06140018\n",
      "  -0.02672659  0.74372861 -0.33737166  0.00300824  0.36506671  0.08399169\n",
      "   0.63081701 -0.18451327 -0.28470345  0.04038893 -0.05396917  0.33632543\n",
      "  -0.26587709 -0.23429313  0.13122204  0.04812601  0.49977992  0.46484197\n",
      "   0.36998696 -0.10522033  0.18544384  0.05923618  0.22793571 -0.15013812\n",
      "  -0.17585326  0.10629169  0.2322683  -0.39054273]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.0623365  -0.12647159 -0.0828542  -0.23345926  0.6233661   0.01701204\n",
      "   0.34133081  0.00355891  0.55910962 -0.0041479   0.55790343  0.07594934\n",
      "   0.17758602 -0.04273188  0.26485788 -0.06403475 -0.36546297 -0.25934461\n",
      "  -0.21299495  0.00596811  0.20433089  0.16039245  0.08673713 -0.26790034\n",
      "   0.54248433  0.32224276  0.3472597  -0.02733009  0.08681202 -0.44185289\n",
      "   0.19037623  0.10221319 -0.25091234 -0.41097282  0.45208018  0.28859056\n",
      "  -0.11271856 -0.24591676 -0.12670065 -0.04866255  0.6085939  -0.26728584\n",
      "  -0.06215554  0.25916037 -0.0165353   0.67394713  0.50014902  0.2693936\n",
      "  -0.52738924 -0.19152193 -0.42694391  0.52958401  0.28542993  0.60244845\n",
      "   0.1331878  -0.05536241 -0.23057633  0.41596373  0.19384395  0.56488234\n",
      "  -0.29179059 -0.02757514  0.42335322 -0.3090091  -0.43939575 -0.18803673\n",
      "   0.29856303 -0.19640094  0.3601875  -0.38047493  0.22859981 -0.04845263\n",
      "  -0.01534022  0.75103611 -0.32604805  0.01361317  0.37470627  0.09281063\n",
      "   0.63913155 -0.172879   -0.27263418  0.05192426 -0.04230918  0.34718214\n",
      "  -0.25459273 -0.22111173  0.14260486  0.06068773  0.50867581  0.47294613\n",
      "   0.37733361 -0.09508361  0.1933603   0.07132815  0.23948431 -0.13819119\n",
      "  -0.16523986  0.11683301  0.24200385 -0.3782336 ]]\n",
      "[[False False False False  True  True  True  True  True False  True  True\n",
      "   True False  True False False False False  True  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.07751558 -0.14317892 -0.1012686  -0.24355365  0.6129994   0.00122915\n",
      "   0.34248075 -0.00922131  0.55572517 -0.00921247  0.54620831  0.05717156\n",
      "   0.15830303 -0.06098316  0.26614021 -0.08574687 -0.37713639 -0.27335195\n",
      "  -0.22042066 -0.01445962  0.18417518  0.14878447  0.07498582 -0.2749194\n",
      "   0.53378952  0.32408689  0.3441816  -0.04975224  0.07493491 -0.45493079\n",
      "   0.17339252  0.08575189 -0.26801022 -0.4227614   0.45058454  0.28743179\n",
      "  -0.12354948 -0.26240374 -0.14201262 -0.06350402  0.59886702 -0.27415469\n",
      "  -0.07057681  0.24625292 -0.02546238  0.66797818  0.49355987  0.25316462\n",
      "  -0.53738647 -0.20327431 -0.4403559   0.52349527  0.27178243  0.59107493\n",
      "   0.12952831 -0.0753579  -0.24845627  0.41547136  0.19378959  0.56158699\n",
      "  -0.30426285 -0.04476999  0.42203552 -0.32156215 -0.44981623 -0.19584152\n",
      "   0.28724019 -0.21024902  0.34579305 -0.3950762   0.21559471 -0.06862329\n",
      "  -0.02876204  0.74214389 -0.33318464  0.00333088  0.36203246  0.08945268\n",
      "   0.62763103 -0.18443758 -0.28432845  0.03650122 -0.05650588  0.32927964\n",
      "  -0.26292077 -0.23906128  0.12615575  0.04023923  0.4969388   0.46564396\n",
      "   0.37545851 -0.10105558  0.19233473  0.05289733  0.22038367 -0.1518977\n",
      "  -0.17211519  0.10486235  0.2314334  -0.38902757]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-6.55664047e-02 -1.30645526e-01 -8.85520940e-02 -2.31982066e-01\n",
      "   6.21138542e-01  1.29827342e-02  3.49334733e-01  1.95248575e-03\n",
      "   5.62650401e-01  3.62087455e-04  5.54894840e-01  6.93180215e-02\n",
      "   1.70142284e-01 -4.84645542e-02  2.73277694e-01 -7.24100772e-02\n",
      "  -3.64693420e-01 -2.60847961e-01 -2.09496429e-01 -1.67940542e-03\n",
      "   1.96089937e-01  1.59071711e-01  8.56033654e-02 -2.63857475e-01\n",
      "   5.41904769e-01  3.30871906e-01  3.51904987e-01 -3.64148950e-02\n",
      "   8.55787400e-02 -4.41879843e-01  1.84693748e-01  9.73018123e-02\n",
      "  -2.54887424e-01 -4.10108724e-01  4.57546537e-01  2.94988517e-01\n",
      "  -1.12314038e-01 -2.49430401e-01 -1.29773193e-01 -5.16818947e-02\n",
      "   6.06931108e-01 -2.63127009e-01 -6.00565705e-02  2.56412492e-01\n",
      "  -1.50210626e-02  6.74981873e-01  5.01402546e-01  2.63984486e-01\n",
      "  -5.24638191e-01 -1.91523144e-01 -4.27295077e-01  5.31112294e-01\n",
      "   2.81991407e-01  5.99512053e-01  1.38246518e-01 -6.24192242e-02\n",
      "  -2.35251022e-01  4.22368434e-01  2.01498825e-01  5.68469883e-01\n",
      "  -2.91951132e-01 -3.25363853e-02  4.29076921e-01 -3.09163225e-01\n",
      "  -4.37336911e-01 -1.84938832e-01  2.96904377e-01 -1.98035072e-01\n",
      "   3.55855394e-01 -3.81953305e-01  2.25899405e-01 -5.56757441e-02\n",
      "  -1.73756708e-02  7.49451388e-01 -3.21861030e-01  1.39358089e-02\n",
      "   3.71672020e-01  9.82716279e-02  6.35945582e-01 -1.72803306e-01\n",
      "  -2.72259175e-01  4.80365501e-02 -4.48458999e-02  3.40136343e-01\n",
      "  -2.51636405e-01 -2.25879871e-01  1.37538574e-01  5.28009514e-02\n",
      "   5.05834697e-01  4.73748129e-01  3.82805163e-01 -9.09188562e-02\n",
      "   2.00251196e-01  6.49892929e-02  2.31932264e-01 -1.39950770e-01\n",
      "  -1.61501791e-01  1.15403669e-01  2.41168952e-01 -3.76718448e-01]]\n",
      "[[False False False False  True  True  True  True  True  True  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.07428514 -0.13900481 -0.09554344 -0.24512083  0.61534412  0.00528383\n",
      "   0.33441413 -0.00762207  0.55221798 -0.01380687  0.54933142  0.06387158\n",
      "   0.16584467 -0.05521524  0.25763815 -0.07730756 -0.37801201 -0.27190653\n",
      "  -0.22403036 -0.00674397  0.19252894  0.150124    0.0761222  -0.27909053\n",
      "   0.5344506   0.3153837   0.33951635 -0.04058776  0.07617206 -0.45501441\n",
      "   0.17915349  0.09071561 -0.26406086 -0.4237413   0.44510777  0.28098111\n",
      "  -0.12400799 -0.2589206  -0.1389536  -0.06048418  0.60063719 -0.2784431\n",
      "  -0.07274124  0.24905586 -0.02702625  0.66702969  0.49235718  0.2586637\n",
      "  -0.54029861 -0.20333688 -0.44010849  0.52201996  0.27528952  0.59413333\n",
      "   0.12440421 -0.06825182 -0.2437948   0.40903782  0.18604859  0.55803355\n",
      "  -0.30418298 -0.03978021  0.41629281 -0.32149203 -0.45201102 -0.19904094\n",
      "   0.28894783 -0.20865936  0.35021887 -0.39367886  0.21834385 -0.06134755\n",
      "  -0.02673201  0.74386183 -0.33751266  0.00297905  0.36514707  0.08391373\n",
      "   0.63094901 -0.1845745  -0.28478716  0.04041912 -0.05397345  0.33645032\n",
      "  -0.26598939 -0.23430342  0.13128389  0.0482074   0.4998831   0.46489328\n",
      "   0.36996242 -0.10531759  0.18536709  0.0593004   0.22804661 -0.15017008\n",
      "  -0.17595843  0.10630363  0.23229643 -0.39066043]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.06233596 -0.12647142 -0.08282694 -0.23354925  0.62348325  0.01703742\n",
      "   0.34126811  0.00355173  0.55914321 -0.00423232  0.55801795  0.07601804\n",
      "   0.17768392 -0.04269664  0.26477564 -0.06397077 -0.36556903 -0.25940254\n",
      "  -0.21310613  0.00603624  0.2044437   0.16041125  0.08673975 -0.2680286\n",
      "   0.54256585  0.32216872  0.34723973 -0.02725042  0.08681588 -0.44196346\n",
      "   0.19045472  0.10226553 -0.25093806 -0.41108862  0.45206977  0.28853784\n",
      "  -0.11277254 -0.24594726 -0.12671418 -0.04866205  0.60870128 -0.26741543\n",
      "  -0.062221    0.25921543 -0.01658493  0.67403338  0.50019985  0.26948357\n",
      "  -0.52755033 -0.19158571 -0.42704766  0.52963698  0.2854985   0.60257046\n",
      "   0.13312243 -0.05531315 -0.23058955  0.41593489  0.19375782  0.56491644\n",
      "  -0.29187126 -0.02754661  0.42333422 -0.3090931  -0.4395317  -0.18813826\n",
      "   0.29861202 -0.19644541  0.36028121 -0.38055597  0.22864854 -0.0484\n",
      "  -0.01534564  0.75116933 -0.32618905  0.01358398  0.37478663  0.09273268\n",
      "   0.63926356 -0.17294022 -0.27271789  0.05195445 -0.04231347  0.34730703\n",
      "  -0.25470502 -0.22112201  0.14266672  0.06076912  0.50877899  0.47299744\n",
      "   0.37730907 -0.09518087  0.19328355  0.07139236  0.2395952  -0.13822315\n",
      "  -0.16534503  0.11684495  0.24203199 -0.37835131]]\n",
      "[[False False False False  True  True  True  True  True False  True  True\n",
      "   True False  True False False False False  True  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.07751504 -0.14317875 -0.10124134 -0.24364365  0.61311656  0.00125452\n",
      "   0.34241806 -0.00922849  0.55575876 -0.00929688  0.54632283  0.05724026\n",
      "   0.15840093 -0.06094792  0.26605796 -0.0856829  -0.37724246 -0.27340988\n",
      "  -0.22053185 -0.01439149  0.18428799  0.14880326  0.07498844 -0.27504767\n",
      "   0.53387104  0.32401285  0.34416164 -0.04967257  0.07493878 -0.45504136\n",
      "   0.17347101  0.08580423 -0.26803594 -0.42287721  0.45057413  0.28737907\n",
      "  -0.12360346 -0.26243424 -0.14202615 -0.06350352  0.5989744  -0.27428427\n",
      "  -0.07064227  0.24630798 -0.02551201  0.66806444  0.49361071  0.25325458\n",
      "  -0.53754756 -0.2033381  -0.44045965  0.52354824  0.271851    0.59119693\n",
      "   0.12946293 -0.07530863 -0.24846949  0.41544252  0.19370346  0.56162109\n",
      "  -0.30434353 -0.04474145  0.42201651 -0.32164615 -0.44995218 -0.19594304\n",
      "   0.28728917 -0.21029349  0.34588676 -0.39515723  0.21564344 -0.06857067\n",
      "  -0.02876746  0.74227711 -0.33332563  0.00330169  0.36211282  0.08937473\n",
      "   0.62776304 -0.18449881 -0.28441215  0.03653141 -0.05651017  0.32940452\n",
      "  -0.26303307 -0.23907156  0.12621761  0.04032062  0.49704198  0.46569528\n",
      "   0.37543397 -0.10115283  0.19225798  0.05296155  0.22049456 -0.15192966\n",
      "  -0.17222036  0.1048743   0.23146153 -0.38914528]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-6.55658686e-02 -1.30645362e-01 -8.85248382e-02 -2.32072059e-01\n",
      "   6.21255700e-01  1.30081083e-02  3.49272037e-01  1.94530616e-03\n",
      "   5.62683991e-01  2.77669266e-04  5.55009357e-01  6.93867284e-02\n",
      "   1.70240185e-01 -4.84293168e-02  2.73195453e-01 -7.23461038e-02\n",
      "  -3.64799486e-01 -2.60905893e-01 -2.09607617e-01 -1.61127651e-03\n",
      "   1.96202749e-01  1.59090506e-01  8.56059825e-02 -2.63985740e-01\n",
      "   5.41986286e-01  3.30797870e-01  3.51885023e-01 -3.63352230e-02\n",
      "   8.55826041e-02 -4.41990415e-01  1.84772232e-01  9.73541501e-02\n",
      "  -2.54913143e-01 -4.10224526e-01  4.57536126e-01  2.94935795e-01\n",
      "  -1.12368017e-01 -2.49460897e-01 -1.29786720e-01 -5.16813942e-02\n",
      "   6.07038489e-01 -2.63256596e-01 -6.01220302e-02  2.56467558e-01\n",
      "  -1.50706917e-02  6.75068127e-01  5.01453382e-01  2.64074452e-01\n",
      "  -5.24799283e-01 -1.91586926e-01 -4.27398826e-01  5.31165264e-01\n",
      "   2.82059974e-01  5.99634058e-01  1.38181146e-01 -6.23699565e-02\n",
      "  -2.35264244e-01  4.22339599e-01  2.01412698e-01  5.68503981e-01\n",
      "  -2.92031809e-01 -3.25078544e-02  4.29057915e-01 -3.09247222e-01\n",
      "  -4.37472866e-01 -1.85040359e-01  2.96953361e-01 -1.98079538e-01\n",
      "   3.55949102e-01 -3.82034342e-01  2.25948131e-01 -5.56231154e-02\n",
      "  -1.73810910e-02  7.49584607e-01 -3.22002025e-01  1.39066185e-02\n",
      "   3.71752379e-01  9.81936723e-02  6.36077582e-01 -1.72864532e-01\n",
      "  -2.72342883e-01  4.80667398e-02 -4.48501840e-02  3.40261229e-01\n",
      "  -2.51748703e-01 -2.25890154e-01  1.37600432e-01  5.28823426e-02\n",
      "   5.05937874e-01  4.73799440e-01  3.82780620e-01 -9.10161098e-02\n",
      "   2.00174445e-01  6.50535063e-02  2.32043155e-01 -1.39982727e-01\n",
      "  -1.61606961e-01  1.15415612e-01  2.41197086e-01 -3.76836156e-01]]\n",
      "[[False False False False  True  True  True  True  True  True  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.0742846  -0.13900465 -0.09551619 -0.24521083  0.61546127  0.00530921\n",
      "   0.33435144 -0.00762925  0.55225157 -0.01389129  0.54944593  0.06394028\n",
      "   0.16594257 -0.05518001  0.25755591 -0.07724359 -0.37811807 -0.27196446\n",
      "  -0.22414155 -0.00667584  0.19264175  0.1501428   0.07612482 -0.27921879\n",
      "   0.53453212  0.31530967  0.33949638 -0.04050809  0.07617592 -0.45512498\n",
      "   0.17923197  0.09076794 -0.26408658 -0.42385711  0.44509736  0.28092839\n",
      "  -0.12406197 -0.25895109 -0.13896713 -0.06048368  0.60074457 -0.27857269\n",
      "  -0.07280669  0.24911092 -0.02707587  0.66711595  0.49240802  0.25875366\n",
      "  -0.5404597  -0.20340066 -0.44021224  0.52207293  0.27535808  0.59425534\n",
      "   0.12433884 -0.06820255 -0.24380802  0.40900898  0.18596246  0.55806765\n",
      "  -0.30426366 -0.03975168  0.41627381 -0.32157602 -0.45214697 -0.19914247\n",
      "   0.28899681 -0.20870382  0.35031258 -0.3937599   0.21839257 -0.06129492\n",
      "  -0.02673743  0.74399504 -0.33765365  0.00294986  0.36522743  0.08383577\n",
      "   0.63108101 -0.18463572 -0.28487086  0.04044931 -0.05397773  0.33657521\n",
      "  -0.26610168 -0.2343137   0.13134575  0.04828879  0.49998627  0.46494459\n",
      "   0.36993788 -0.10541484  0.18529034  0.05936461  0.2281575  -0.15020203\n",
      "  -0.17606359  0.10631558  0.23232457 -0.39077814]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.06233543 -0.12647126 -0.08279969 -0.23363924  0.62360041  0.01706279\n",
      "   0.34120542  0.00354455  0.5591768  -0.00431673  0.55813246  0.07608675\n",
      "   0.17778182 -0.0426614   0.2646934  -0.0639068  -0.3656751  -0.25946047\n",
      "  -0.21321732  0.00610437  0.20455651  0.16043004  0.08674237 -0.26815687\n",
      "   0.54264736  0.32209469  0.34721977 -0.02717075  0.08681974 -0.44207404\n",
      "   0.1905332   0.10231787 -0.25096378 -0.41120443  0.45205936  0.28848512\n",
      "  -0.11282652 -0.24597775 -0.1267277  -0.04866155  0.60880866 -0.26754501\n",
      "  -0.06228646  0.2592705  -0.01663456  0.67411964  0.50025069  0.26957353\n",
      "  -0.52771142 -0.19164949 -0.42715141  0.52968995  0.28556706  0.60269246\n",
      "   0.13305705 -0.05526388 -0.23060277  0.41590606  0.1936717   0.56495054\n",
      "  -0.29195194 -0.02751808  0.42331521 -0.3091771  -0.43966766 -0.18823978\n",
      "   0.298661   -0.19648988  0.36037492 -0.38063701  0.22869727 -0.04834737\n",
      "  -0.01535106  0.75130254 -0.32633004  0.01355479  0.37486698  0.09265472\n",
      "   0.63939556 -0.17300145 -0.2728016   0.05198464 -0.04231775  0.34743191\n",
      "  -0.25481732 -0.22113229  0.14272858  0.06085051  0.50888217  0.47304875\n",
      "   0.37728453 -0.09527812  0.1932068   0.07145657  0.23970609 -0.1382551\n",
      "  -0.1654502   0.11685689  0.24206012 -0.37846902]]\n",
      "[[False False False False  True  True  True  True  True False  True  True\n",
      "   True False  True False False False False  True  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.07751451 -0.14317859 -0.10121408 -0.24373364  0.61323372  0.0012799\n",
      "   0.34235536 -0.00923567  0.55579235 -0.0093813   0.54643734  0.05730897\n",
      "   0.15849883 -0.06091268  0.26597572 -0.08561892 -0.37734853 -0.27346781\n",
      "  -0.22064303 -0.01432336  0.1844008   0.14882206  0.07499105 -0.27517593\n",
      "   0.53395256  0.32393882  0.34414167 -0.0495929   0.07494264 -0.45515193\n",
      "   0.17354949  0.08585657 -0.26806166 -0.42299301  0.45056371  0.28732635\n",
      "  -0.12365744 -0.26246473 -0.14203967 -0.06350302  0.59908178 -0.27441386\n",
      "  -0.07070773  0.24636305 -0.02556164  0.66815069  0.49366155  0.25334455\n",
      "  -0.53770865 -0.20340188 -0.4405634   0.52360121  0.27191956  0.59131894\n",
      "   0.12939756 -0.07525936 -0.24848271  0.41541369  0.19361733  0.56165519\n",
      "  -0.30442421 -0.04471292  0.42199751 -0.32173015 -0.45008814 -0.19604457\n",
      "   0.28733815 -0.21033795  0.34598047 -0.39523827  0.21569217 -0.06851804\n",
      "  -0.02877288  0.74241033 -0.33346663  0.0032725   0.36219318  0.08929677\n",
      "   0.62789504 -0.18456003 -0.28449586  0.0365616  -0.05651445  0.32952941\n",
      "  -0.26314536 -0.23908185  0.12627947  0.04040201  0.49714516  0.46574659\n",
      "   0.37540943 -0.10125008  0.19218123  0.05302576  0.22060545 -0.15196162\n",
      "  -0.17232553  0.10488624  0.23148966 -0.38926299]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-6.55653324e-02 -1.30645198e-01 -8.84975824e-02 -2.32162052e-01\n",
      "   6.21372858e-01  1.30334825e-02  3.49209341e-01  1.93812657e-03\n",
      "   5.62717582e-01  1.93251077e-04  5.55123873e-01  6.94554353e-02\n",
      "   1.70338085e-01 -4.83940795e-02  2.73113212e-01 -7.22821303e-02\n",
      "  -3.64905552e-01 -2.60963825e-01 -2.09718805e-01 -1.54314760e-03\n",
      "   1.96315561e-01  1.59109301e-01  8.56085995e-02 -2.64114005e-01\n",
      "   5.42067802e-01  3.30723835e-01  3.51865060e-01 -3.62555511e-02\n",
      "   8.55864683e-02 -4.42100988e-01  1.84850716e-01  9.74064879e-02\n",
      "  -2.54938861e-01 -4.10340329e-01  4.57525714e-01  2.94883074e-01\n",
      "  -1.12421997e-01 -2.49491393e-01 -1.29800246e-01 -5.16808938e-02\n",
      "   6.07145869e-01 -2.63386183e-01 -6.01874900e-02  2.56522624e-01\n",
      "  -1.51203208e-02  6.75154381e-01  5.01504217e-01  2.64164419e-01\n",
      "  -5.24960374e-01 -1.91650708e-01 -4.27502576e-01  5.31218235e-01\n",
      "   2.82128541e-01  5.99756063e-01  1.38115774e-01 -6.23206888e-02\n",
      "  -2.35277467e-01  4.22310764e-01  2.01326572e-01  5.68538078e-01\n",
      "  -2.92112486e-01 -3.24793236e-02  4.29038909e-01 -3.09331219e-01\n",
      "  -4.37608821e-01 -1.85141886e-01  2.97002346e-01 -1.98124005e-01\n",
      "   3.56042810e-01 -3.82115379e-01  2.25996858e-01 -5.55704866e-02\n",
      "  -1.73865112e-02  7.49717826e-01 -3.22143019e-01  1.38774280e-02\n",
      "   3.71832738e-01  9.81157168e-02  6.36209583e-01 -1.72925759e-01\n",
      "  -2.72426591e-01  4.80969295e-02 -4.48544682e-02  3.40386115e-01\n",
      "  -2.51861000e-01 -2.25900438e-01  1.37662291e-01  5.29637338e-02\n",
      "   5.06041051e-01  4.73850752e-01  3.82756078e-01 -9.11133634e-02\n",
      "   2.00097694e-01  6.51177196e-02  2.32154047e-01 -1.40014685e-01\n",
      "  -1.61712130e-01  1.15427556e-01  2.41225219e-01 -3.76953864e-01]]\n",
      "[[False False False False  True  True  True  True  True  True  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.07428407 -0.13900448 -0.09548893 -0.24530082  0.61557843  0.00533458\n",
      "   0.33428874 -0.00763643  0.55228516 -0.01397571  0.54956045  0.06400899\n",
      "   0.16604047 -0.05514477  0.25747367 -0.07717962 -0.37822414 -0.27202239\n",
      "  -0.22425274 -0.00660771  0.19275456  0.15016159  0.07612744 -0.27934706\n",
      "   0.53461364  0.31523563  0.33947642 -0.04042842  0.07617978 -0.45523555\n",
      "   0.17931046  0.09082028 -0.2641123  -0.42397291  0.44508695  0.28087567\n",
      "  -0.12411595 -0.25898159 -0.13898066 -0.06048318  0.60085195 -0.27870228\n",
      "  -0.07287215  0.24916599 -0.0271255   0.6672022   0.49245885  0.25884363\n",
      "  -0.54062079 -0.20346444 -0.44031599  0.5221259   0.27542665  0.59437734\n",
      "   0.12427347 -0.06815328 -0.24382124  0.40898015  0.18587633  0.55810175\n",
      "  -0.30434434 -0.03972315  0.4162548  -0.32166002 -0.45228293 -0.19924399\n",
      "   0.2890458  -0.20874829  0.35040629 -0.39384093  0.2184413  -0.06124229\n",
      "  -0.02674285  0.74412826 -0.33779465  0.00292067  0.36530778  0.08375782\n",
      "   0.63121301 -0.18469695 -0.28495457  0.0404795  -0.05398202  0.33670009\n",
      "  -0.26621398 -0.23432398  0.13140761  0.04837018  0.50008945  0.4649959\n",
      "   0.36991334 -0.10551209  0.18521358  0.05942882  0.22826839 -0.15023399\n",
      "  -0.17616876  0.10632752  0.2323527  -0.39089585]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.06233489 -0.12647109 -0.08277243 -0.23372923  0.62371757  0.01708817\n",
      "   0.34114272  0.00353737  0.55921039 -0.00440115  0.55824698  0.07615546\n",
      "   0.17787972 -0.04262617  0.26461116 -0.06384283 -0.36578117 -0.25951841\n",
      "  -0.21332851  0.0061725   0.20466932  0.16044884  0.08674498 -0.26828513\n",
      "   0.54272888  0.32202065  0.34719981 -0.02709107  0.08682361 -0.44218461\n",
      "   0.19061168  0.1023702  -0.2509895  -0.41132023  0.45204895  0.2884324\n",
      "  -0.1128805  -0.24600825 -0.12674123 -0.04866105  0.60891604 -0.2676746\n",
      "  -0.06235192  0.25932556 -0.01668419  0.67420589  0.50030152  0.2696635\n",
      "  -0.52787251 -0.19171327 -0.42725516  0.52974292  0.28563563  0.60281447\n",
      "   0.13299168 -0.05521461 -0.23061599  0.41587722  0.19358557  0.56498464\n",
      "  -0.29203262 -0.02748955  0.4232962  -0.30926109 -0.43980361 -0.18834131\n",
      "   0.29870999 -0.19653434  0.36046863 -0.38071804  0.22874599 -0.04829474\n",
      "  -0.01535648  0.75143576 -0.32647104  0.0135256   0.37494734  0.09257677\n",
      "   0.63952756 -0.17306267 -0.2728853   0.05201483 -0.04232203  0.3475568\n",
      "  -0.25492962 -0.22114258  0.14279043  0.0609319   0.50898535  0.47310007\n",
      "   0.37725998 -0.09537538  0.19313005  0.07152079  0.23981698 -0.13828706\n",
      "  -0.16555537  0.11686884  0.24208825 -0.37858673]]\n",
      "[[False False False False  True  True  True  True  True False  True  True\n",
      "   True False  True False False False False  True  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.07751397 -0.14317842 -0.10118683 -0.24382363  0.61335088  0.00130527\n",
      "   0.34229267 -0.00924285  0.55582594 -0.00946572  0.54655186  0.05737768\n",
      "   0.15859673 -0.06087745  0.26589348 -0.08555495 -0.37745459 -0.27352574\n",
      "  -0.22075422 -0.01425523  0.18451361  0.14884085  0.07499367 -0.2753042\n",
      "   0.53403407  0.32386478  0.34412171 -0.04951322  0.07494651 -0.4552625\n",
      "   0.17362797  0.0859089  -0.26808738 -0.42310881  0.4505533   0.28727363\n",
      "  -0.12371142 -0.26249523 -0.1420532  -0.06350252  0.59918916 -0.27454345\n",
      "  -0.07077319  0.24641811 -0.02561127  0.66823695  0.49371238  0.25343452\n",
      "  -0.53786974 -0.20346566 -0.44066715  0.52365418  0.27198813  0.59144094\n",
      "   0.12933219 -0.07521009 -0.24849594  0.41538485  0.19353121  0.56168929\n",
      "  -0.30450488 -0.04468439  0.4219785  -0.32181414 -0.45022409 -0.1961461\n",
      "   0.28738714 -0.21038242  0.34607418 -0.39531931  0.21574089 -0.06846541\n",
      "  -0.0287783   0.74254354 -0.33360762  0.00324331  0.36227354  0.08921881\n",
      "   0.62802704 -0.18462126 -0.28457957  0.03659179 -0.05651874  0.32965429\n",
      "  -0.26325766 -0.23909213  0.12634133  0.04048341  0.49724833  0.4657979\n",
      "   0.37538489 -0.10134734  0.19210448  0.05308997  0.22071634 -0.15199357\n",
      "  -0.17243069  0.10489818  0.2315178  -0.3893807 ]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-6.55647963e-02 -1.30645034e-01 -8.84703267e-02 -2.32252045e-01\n",
      "   6.21490016e-01  1.30588566e-02  3.49146645e-01  1.93094698e-03\n",
      "   5.62751172e-01  1.08832888e-04  5.55238390e-01  6.95241422e-02\n",
      "   1.70435986e-01 -4.83588422e-02  2.73030972e-01 -7.22181569e-02\n",
      "  -3.65011619e-01 -2.61021758e-01 -2.09829992e-01 -1.47501868e-03\n",
      "   1.96428373e-01  1.59128096e-01  8.56112165e-02 -2.64242271e-01\n",
      "   5.42149319e-01  3.30649800e-01  3.51845096e-01 -3.61758792e-02\n",
      "   8.55903324e-02 -4.42211560e-01  1.84929200e-01  9.74588257e-02\n",
      "  -2.54964580e-01 -4.10456131e-01  4.57515303e-01  2.94830352e-01\n",
      "  -1.12475976e-01 -2.49521889e-01 -1.29813773e-01 -5.16803933e-02\n",
      "   6.07253249e-01 -2.63515770e-01 -6.02529497e-02  2.56577689e-01\n",
      "  -1.51699500e-02  6.75240634e-01  5.01555053e-01  2.64254386e-01\n",
      "  -5.25121466e-01 -1.91714489e-01 -4.27606325e-01  5.31271206e-01\n",
      "   2.82197108e-01  5.99878068e-01  1.38050401e-01 -6.22714210e-02\n",
      "  -2.35290689e-01  4.22281928e-01  2.01240445e-01  5.68572175e-01\n",
      "  -2.92193163e-01 -3.24507927e-02  4.29019903e-01 -3.09415216e-01\n",
      "  -4.37744776e-01 -1.85243413e-01  2.97051330e-01 -1.98168472e-01\n",
      "   3.56136518e-01 -3.82196417e-01  2.26045584e-01 -5.55178578e-02\n",
      "  -1.73919313e-02  7.49851045e-01 -3.22284014e-01  1.38482376e-02\n",
      "   3.71913096e-01  9.80377612e-02  6.36341584e-01 -1.72986985e-01\n",
      "  -2.72510299e-01  4.81271191e-02 -4.48587524e-02  3.40511001e-01\n",
      "  -2.51973298e-01 -2.25910721e-01  1.37724149e-01  5.30451250e-02\n",
      "   5.06144229e-01  4.73902063e-01  3.82731536e-01 -9.12106170e-02\n",
      "   2.00020943e-01  6.51819330e-02  2.32264938e-01 -1.40046643e-01\n",
      "  -1.61817299e-01  1.15439499e-01  2.41253353e-01 -3.77071572e-01]]\n",
      "[[False False False False  True  True  True  True  True  True  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.07428353 -0.13900432 -0.09546168 -0.24539081  0.61569559  0.00535995\n",
      "   0.33422605 -0.00764361  0.55231875 -0.01406012  0.54967497  0.0640777\n",
      "   0.16613837 -0.05510953  0.25739143 -0.07711564 -0.3783302  -0.27208032\n",
      "  -0.22436393 -0.00653959  0.19286737  0.15018039  0.07613005 -0.27947532\n",
      "   0.53469515  0.3151616   0.33945645 -0.04034875  0.07618365 -0.45534613\n",
      "   0.17938894  0.09087262 -0.26413801 -0.42408871  0.44507654  0.28082295\n",
      "  -0.12416993 -0.25901209 -0.13899418 -0.06048267  0.60095933 -0.27883186\n",
      "  -0.07293761  0.24922105 -0.02717513  0.66728845  0.49250969  0.2589336\n",
      "  -0.54078188 -0.20352822 -0.44041974  0.52217887  0.27549522  0.59449935\n",
      "   0.1242081  -0.06810401 -0.24383446  0.40895131  0.18579021  0.55813585\n",
      "  -0.30442501 -0.03969462  0.41623579 -0.32174402 -0.45241888 -0.19934552\n",
      "   0.28909478 -0.20879276  0.3505     -0.39392197  0.21849003 -0.06118966\n",
      "  -0.02674827  0.74426148 -0.33793564  0.00289148  0.36538814  0.08367986\n",
      "   0.63134501 -0.18475818 -0.28503828  0.04050969 -0.0539863   0.33682498\n",
      "  -0.26632628 -0.23433427  0.13146947  0.04845157  0.50019263  0.46504721\n",
      "   0.36988879 -0.10560935  0.18513683  0.05949304  0.22837928 -0.15026595\n",
      "  -0.17627393  0.10633946  0.23238083 -0.39101356]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.06233436 -0.12647093 -0.08274518 -0.23381923  0.62383473  0.01711354\n",
      "   0.34108003  0.00353019  0.55924398 -0.00448557  0.5583615   0.07622416\n",
      "   0.17797762 -0.04259093  0.26452892 -0.06377885 -0.36588723 -0.25957634\n",
      "  -0.2134397   0.00624063  0.20478213  0.16046763  0.0867476  -0.2684134\n",
      "   0.5428104   0.32194662  0.34717984 -0.0270114   0.08682747 -0.44229518\n",
      "   0.19069017  0.10242254 -0.25101522 -0.41143603  0.45203854  0.28837967\n",
      "  -0.11293448 -0.24603874 -0.12675476 -0.04866054  0.60902342 -0.26780419\n",
      "  -0.06241738  0.25938063 -0.01673382  0.67429214  0.50035236  0.26975347\n",
      "  -0.52803361 -0.19177705 -0.42735891  0.52979589  0.2857042   0.60293647\n",
      "   0.13292631 -0.05516534 -0.23062922  0.41584839  0.19349944  0.56501873\n",
      "  -0.29211329 -0.02746102  0.4232772  -0.30934509 -0.43993957 -0.18844284\n",
      "   0.29875897 -0.19657881  0.36056234 -0.38079908  0.22879472 -0.04824211\n",
      "  -0.0153619   0.75156898 -0.32661203  0.01349641  0.3750277   0.09249881\n",
      "   0.63965956 -0.1731239  -0.27296901  0.05204502 -0.04232632  0.34768168\n",
      "  -0.25504192 -0.22115286  0.14285229  0.06101329  0.50908852  0.47315138\n",
      "   0.37723544 -0.09547263  0.1930533   0.071585    0.23992787 -0.13831902\n",
      "  -0.16566054  0.11688078  0.24211639 -0.37870443]]\n",
      "[[False False False False  True  True  True  True  True False  True  True\n",
      "   True False  True False False False False  True  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n",
      "[[-0.07751344 -0.14317826 -0.10115957 -0.24391362  0.61346804  0.00133064\n",
      "   0.34222997 -0.00925003  0.55585953 -0.00955014  0.54666638  0.05744638\n",
      "   0.15869463 -0.06084221  0.26581124 -0.08549098 -0.37756066 -0.27358368\n",
      "  -0.22086541 -0.0141871   0.18462643  0.14885965  0.07499629 -0.27543246\n",
      "   0.53411559  0.32379075  0.34410174 -0.04943355  0.07495037 -0.45537308\n",
      "   0.17370646  0.08596124 -0.2681131  -0.42322461  0.45054289  0.28722091\n",
      "  -0.1237654  -0.26252573 -0.14206673 -0.06350202  0.59929654 -0.27467303\n",
      "  -0.07083865  0.24647318 -0.02566089  0.6683232   0.49376322  0.25352448\n",
      "  -0.53803083 -0.20352944 -0.4407709   0.52370716  0.2720567   0.59156295\n",
      "   0.12926682 -0.07516082 -0.24850916  0.41535602  0.19344508  0.56172338\n",
      "  -0.30458556 -0.04465586  0.42195949 -0.32189814 -0.45036005 -0.19624762\n",
      "   0.28743612 -0.21042689  0.34616789 -0.39540035  0.21578962 -0.06841278\n",
      "  -0.02878372  0.74267676 -0.33374862  0.00321412  0.3623539   0.08914086\n",
      "   0.62815904 -0.18468249 -0.28466328  0.03662198 -0.05652302  0.32977918\n",
      "  -0.26336996 -0.23910241  0.12640318  0.0405648   0.49735151  0.46584921\n",
      "   0.37536035 -0.10144459  0.19202773  0.05315419  0.22082724 -0.15202553\n",
      "  -0.17253586  0.10491013  0.23154593 -0.38949841]]\n",
      "[[False False False False  True  True  True False  True False  True  True\n",
      "   True False  True False False False False False  True  True  True False\n",
      "   True  True  True False  True False  True  True False False  True  True\n",
      "  False False False False  True False False  True False  True  True  True\n",
      "  False False False  True  True  True  True False False  True  True  True\n",
      "  False False  True False False False  True False  True False  True False\n",
      "  False  True False  True  True  True  True False False  True False  True\n",
      "  False False  True  True  True  True  True False  True  True  True False\n",
      "  False  True  True False]]\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "85340aa812544b08ba35a681c1e69322",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "BokehModel(combine_events=True, render_bundle={'docs_json': {'eb00ef43-eca1-4834-a1a7-4c86da1ca86e': {'version…"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "verbose = False\n",
    "weights_history[0] = weights\n",
    "bias_history[0] = bias\n",
    "# Update loop\n",
    "for i in range(n_epochs):\n",
    "    \n",
    "    ######################\n",
    "    # YOUR CODE GOES HERE\n",
    "    ######################\n",
    "    \n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "    \n",
    "    \n",
    "    \n",
    "    ######################\n",
    "    # Solution  \n",
    "    y_real = np.dot(weights, x_inputs.T) + bias \n",
    "    print(y_real)\n",
    "    y_bar = y_real > 0\n",
    "    print(y_bar)\n",
    "    if np.sum(np.abs(y_bar - y_class)) == 0:\n",
    "        break\n",
    "    error = y_class - y_bar\n",
    "    weights = weights + eta * (np.dot(error, x_inputs))\n",
    "    bias = bias + eta * (np.sum(error))\n",
    "    ######################\n",
    "    \n",
    "    weights_history[i + 1] = weights\n",
    "    bias_history[i + 1] = bias\n",
    "    \n",
    "    if (verbose):\n",
    "        print('%2d. error = %f, weights = %f, %f, %f' % (i, np.sum(np.abs(error)) / n_observations, bias[0, 0], weights[0, 0], weights[0, 1]))\n",
    "plot = center_plot(scatter_boundary(x_inputs, y_class, weights_history, bias_history, i + 1))\n",
    "plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Neuron implementation in `JAX`\n",
    "\n",
    "As seen in the previous course, the `NumPy` implementation requires us to perform manual differentiation of our loss function to understand how to update the parameters. However, the recent [`JAX`](https://github.com/google/jax) library provides **automatic differentiation** on top of the `NumPy` library. If you have not yet completed the set of [tutorials](https://jax.readthedocs.io/en/latest/), we strongly encourage you to do so. We will introduce a few more functions compared to last time, but we try to keep the implementation as simple as possible on purpose."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.6669880563534684\n",
      "[[0.09395266 2.6316104 ]]\n",
      "0.02581308106627382\n",
      "[[-0.5478341  -0.41040868]]\n",
      "-0.7776194131918178\n",
      "[[-0.98370177  0.63491774]]\n",
      "0.9486338224949431\n",
      "[[-0.25498217  0.50253564]]\n",
      "0.701671794647513\n",
      "[[1.9510722 1.2204119]]\n",
      "-1.0510815639071178\n",
      "[[-1.6119251  -0.38320577]]\n",
      "-0.3675481161171661\n",
      "[[-0.6364542 -0.2260025]]\n",
      "-1.1374596907250272\n",
      "[[-0.5978162  -0.64802283]]\n",
      "-1.3221475225908594\n",
      "[[ 0.35871094 -1.5076838 ]]\n",
      "1.7722582783074305\n",
      "[[0.21505953 0.22573107]]\n",
      "-0.34745899102186334\n",
      "[[2.4453242  0.34449932]]\n",
      "0.6701401628971514\n",
      "[[-2.6203442  1.5492581]]\n",
      "0.3222715203493051\n",
      "[[ 0.12408989 -1.4613106 ]]\n",
      "0.060342934170488124\n",
      "[[-1.5378464   0.13951701]]\n",
      "-1.0434500017467254\n",
      "[[-1.9313912  0.7459076]]\n",
      "-1.0099418765878465\n",
      "[[0.79624736 2.258274  ]]\n",
      "0.44173637238220625\n",
      "[[-0.47181112  1.4670489 ]]\n",
      "1.1288768534468012\n",
      "[[ 0.64250743 -0.08448732]]\n",
      "-1.8380677677579502\n",
      "[[-0.42198017  0.09233776]]\n",
      "-0.9387686311201282\n",
      "[[-0.2058157 -1.3440574]]\n",
      "-0.20184052243138237\n",
      "[[1.5818256 0.8235546]]\n",
      "1.0453712773541248\n",
      "[[0.11820083 0.35360345]]\n",
      "0.5381619658145043\n",
      "[[ 0.98560697 -0.2221148 ]]\n",
      "0.8121186697722536\n",
      "[[-1.5206225   0.86475956]]\n",
      "0.2411063006295245\n",
      "[[0.57941645 0.40935126]]\n",
      "-0.9525095310029906\n",
      "[[-1.0085236   0.14594752]]\n",
      "-0.13626675631863647\n",
      "[[ 0.32705155 -1.7108105 ]]\n",
      "1.267248208556173\n",
      "[[-0.6495515  0.8658945]]\n",
      "0.1736336445901797\n",
      "[[ 0.4883449 -1.2877216]]\n",
      "-1.223254774207143\n",
      "[[-1.1421554   0.18813887]]\n",
      "1.4153199811177524\n",
      "[[ 0.9680279  -0.28532556]]\n",
      "0.4577109797826604\n",
      "[[-0.41740415  0.9939139 ]]\n",
      "0.7288758428914587\n",
      "[[0.6719696  0.05453992]]\n",
      "1.9684347309098176\n",
      "[[-1.2911141  -0.00316227]]\n",
      "-0.547788007825014\n",
      "[[-0.6296603   0.39877775]]\n",
      "-0.6794182716515045\n",
      "[[-0.28633317 -1.0243068 ]]\n",
      "-2.506230317020084\n",
      "[[ 0.32621318 -1.2958742 ]]\n",
      "0.14696049490701452\n",
      "[[1.6205432 0.5088258]]\n",
      "0.6061954904535464\n",
      "[[ 0.97230756 -0.0724481 ]]\n",
      "-0.022538890053004787\n",
      "[[ 1.9373144  -0.54883045]]\n",
      "0.013422257380147637\n",
      "[[-0.2912193 -0.7430835]]\n",
      "0.9359448937456367\n",
      "[[ 0.49296162 -1.768273  ]]\n",
      "0.4206226601707677\n",
      "[[-1.6338937  0.4401209]]\n",
      "0.4116196409053295\n",
      "[[1.4446731  0.02883328]]\n",
      "-0.07132392476214137\n",
      "[[-1.5405455  2.094114 ]]\n",
      "-0.045437575551217295\n",
      "[[1.0958145 1.4713451]]\n",
      "1.0408859729898774\n",
      "[[-0.4166812   0.42955747]]\n",
      "-0.09403473465553544\n",
      "[[-0.90503496 -0.99111557]]\n",
      "-0.4208439532804717\n",
      "[[0.11492246 0.3468238 ]]\n",
      "-0.5519885648743732\n",
      "[[-0.48638132 -0.51319647]]\n",
      "-0.12109754794565243\n",
      "[[-0.21988106  0.12021656]]\n",
      "0.19014135902999077\n",
      "[[ 0.10936075 -0.09835161]]\n",
      "0.5121373947463833\n",
      "[[-1.0534155 -1.1813745]]\n",
      "0.1315384666668556\n",
      "[[0.7478694 1.9315408]]\n",
      "-0.33161712421648365\n",
      "[[-0.12736602  0.19425823]]\n",
      "-1.6323862806162175\n",
      "[[1.37221   0.7553252]]\n",
      "0.6191140727503102\n",
      "[[ 1.8662019  -0.45410785]]\n",
      "-0.9925737805679127\n",
      "[[-0.46992424  0.90677863]]\n",
      "-0.16134638680500063\n",
      "[[-0.7226741 -1.9938856]]\n",
      "1.192404330680825\n",
      "[[ 1.4802018 -0.3925187]]\n",
      "0.25073655197789024\n",
      "[[ 0.2555096  -0.76492536]]\n",
      "-0.813616247470488\n",
      "[[ 0.7119774  -0.05678674]]\n",
      "0.7036236301227704\n",
      "[[-0.49632192 -1.196573  ]]\n",
      "-0.2681421391355984\n",
      "[[ 0.15639232 -1.5185548 ]]\n",
      "-0.482559478653049\n",
      "[[-1.0932051  -0.90279543]]\n",
      "1.24461048024104\n",
      "[[0.656158   0.99799144]]\n",
      "0.6768601170532068\n",
      "[[1.1245258 1.5117818]]\n",
      "3.1875026865663405\n",
      "[[-0.44078156 -1.4308499 ]]\n",
      "-1.0805647546615882\n",
      "[[0.44342387 1.3113836 ]]\n",
      "0.010229301452766577\n",
      "[[0.23189715 0.3589866 ]]\n",
      "0.43782968043634884\n",
      "[[0.62063396 0.00861138]]\n",
      "1.3277876229781747\n",
      "[[-1.1421033 -1.1581447]]\n",
      "-0.2511450308456539\n",
      "[[-0.28254586 -0.81651163]]\n",
      "1.5931106271669735\n",
      "[[0.14690225 0.83077425]]\n",
      "0.1708177071000119\n",
      "[[-1.0614742  -0.20414957]]\n",
      "-0.7092539021598433\n",
      "[[0.05897428 0.24914013]]\n",
      "-0.133024591061901\n",
      "[[-0.54424417  0.073318  ]]\n",
      "-0.01735693949031517\n",
      "[[ 1.4643172  -0.69291085]]\n",
      "-0.1010929133811006\n",
      "[[-0.7717426  0.3632196]]\n",
      "-0.5640396275510929\n",
      "[[-1.1842579   0.29991984]]\n",
      "-0.17903559513839135\n",
      "[[-0.19880274  1.1357255 ]]\n",
      "1.0110588281146295\n",
      "[[0.32473707 2.424747  ]]\n",
      "0.9209958357540212\n",
      "[[ 0.02659999 -0.5348896 ]]\n",
      "1.9330898341447647\n",
      "[[1.5162413 0.7735894]]\n",
      "-0.7953632188887011\n",
      "[[-0.590551    0.29641792]]\n",
      "-1.0115351967331279\n",
      "[[-1.2027413   0.31876743]]\n",
      "2.150779945582542\n",
      "[[ 2.8664668 -0.6817525]]\n",
      "0.4251400838063583\n",
      "[[0.30090362 0.5724228 ]]\n",
      "0.44115150707339956\n",
      "[[ 0.6957258 -1.6233048]]\n",
      "-0.8174393268657748\n",
      "[[-1.4113259  1.4858023]]\n",
      "0.4378924797989946\n",
      "[[-1.2033409  -0.38708815]]\n",
      "0.09972327789420866\n",
      "[[-1.3675247  -0.39473203]]\n",
      "0.06540606852407127\n",
      "[[ 0.0181483 -0.9816103]]\n",
      "1.2244306714266617\n",
      "[[-1.0284454  -0.51397115]]\n",
      "-0.7695990489637248\n",
      "[[ 0.88815546 -0.18553007]]\n",
      "0.19211967912663602\n",
      "[[ 0.1693827 -1.6110046]]\n",
      "-1.7232530617473043\n",
      "[[0.66278595 2.1035798 ]]\n",
      "0.46125920260304093\n",
      "[[1.1778432 0.5313306]]\n",
      "-1.085366778877025\n",
      "[[-0.1335118   0.55583626]]\n",
      "1.8233782346552136\n",
      "[[ 0.42328727 -1.317684  ]]\n"
     ]
    }
   ],
   "source": [
    "import jax.numpy as jnp\n",
    "from jax import grad, jit, random\n",
    "\n",
    "seed = np.random.randint(0, 1e9)\n",
    "np.random.seed(23)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "We keep our previousmy generated dataset but we will rely on `jnp.ndarray` instead of `np.ndarray`, by simply casting the arrays to JAX-compliant ones."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_jax = jnp.asarray(x_inputs)\n",
    "y_jax = jnp.asarray(y_class)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will also keep the same hyperparameters as previously (see `eta`, `lambda_r` and `n_epochs` above) and randomly initialize our parameters "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "eta = 0.01\n",
    "# Perform key splitting\n",
    "key_w, key_b, key = random.split(key, 3)\n",
    "# Initialize the weights\n",
    "weights = random.normal(key_w, (1, 2));\n",
    "bias = random.normal(key_b, (1, 1));\n",
    "# Save the weight history for plotting\n",
    "weights_history = jnp.zeros((n_epochs + 1, 2));\n",
    "bias_history = jnp.zeros((n_epochs + 1, 1));\n",
    "# Record the first \n",
    "weights_history = weights_history.at[0].set(weights[0])\n",
    "bias_history = bias_history.at[0].set(bias[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we discussed earlier, the original model by McCulloch & Pitts has several issues regarding its use for numerical optimization (as it was originally observed from a neuroscience point of view)\n",
    "\n",
    "***\n",
    "\n",
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #410819; border-color: #cb2e47\">\n",
    "\n",
    "> **Changes from the `NumPy` implementation**\n",
    ">   1. (Smoothness) Our _activation function_ will change from _identity_ to sigmoid, i.e. $ \\phi(\\mathbf{x}) = \\frac{1}{1 + e^{-\\mathbf{x}}} $\n",
    ">   2. (Outliers) Our _loss function_ will change from $ L_{1} $ to $ L_{2} $ (MSE - Mean Squared Error)\n",
    "\n",
    "</div>\n",
    "\n",
    "***\n",
    "\n",
    "As discussed in the previous course, for using the very handy `grad` function provided by `JAX`, we need to define the whole forward function that we want to derive given our parameters. Hence this function also needs to **contain the forward pass**. This is summarized in the following function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "import jax\n",
    "# Single neuron full function\n",
    "def loss_function(x, y, w, b):\n",
    "    # Forward pass\n",
    "    y_bar = jax.nn.sigmoid(jnp.dot(w, x.T) + b)\n",
    "    # Error computation\n",
    "    error = jnp.sum((y_bar - y) ** 2)\n",
    "    print(error)\n",
    "    return error\n",
    "\n",
    "grad_FUNCTION = grad(loss_function, argnums=[2, 3])\n",
    "dW, dB = grad_FUNCTION(x, y, w, b)\n",
    "# Gradient of the loss\n",
    "grad_loss_function = grad(loss_function, argnums=[2, 3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we can now define the main training loop, which is almost exactly similar to the previously defined one minus the gradient computation operations. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {},
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.holoviews_exec.v0+json": "",
      "text/html": [
       "<div id='10943'>\n",
       "  <div class=\"bk-root\" id=\"cb1f00b2-35bb-42aa-985b-45ccf221d183\" data-root-id=\"10943\"></div>\n",
       "</div>\n",
       "<script type=\"application/javascript\">(function(root) {\n",
       "  function embed_document(root) {\n",
       "    var docs_json = {\"01f6016f-7c38-4dc6-927d-8fe974e68ceb\":{\"defs\":[{\"extends\":null,\"module\":null,\"name\":\"ReactiveHTML1\",\"overrides\":[],\"properties\":[]},{\"extends\":null,\"module\":null,\"name\":\"FlexBox1\",\"overrides\":[],\"properties\":[{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_content\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_items\"},{\"default\":\"row\",\"kind\":null,\"name\":\"flex_direction\"},{\"default\":\"wrap\",\"kind\":null,\"name\":\"flex_wrap\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"justify_content\"}]},{\"extends\":null,\"module\":null,\"name\":\"GridStack1\",\"overrides\":[],\"properties\":[{\"default\":\"warn\",\"kind\":null,\"name\":\"mode\"},{\"default\":null,\"kind\":null,\"name\":\"ncols\"},{\"default\":null,\"kind\":null,\"name\":\"nrows\"},{\"default\":true,\"kind\":null,\"name\":\"allow_resize\"},{\"default\":true,\"kind\":null,\"name\":\"allow_drag\"},{\"default\":[],\"kind\":null,\"name\":\"state\"}]},{\"extends\":null,\"module\":null,\"name\":\"click1\",\"overrides\":[],\"properties\":[{\"default\":\"\",\"kind\":null,\"name\":\"terminal_output\"},{\"default\":\"\",\"kind\":null,\"name\":\"debug_name\"},{\"default\":0,\"kind\":null,\"name\":\"clears\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationAreaBase1\",\"overrides\":[],\"properties\":[{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationArea1\",\"overrides\":[],\"properties\":[{\"default\":[],\"kind\":null,\"name\":\"notifications\"},{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"},{\"default\":[{\"background\":\"#ffc107\",\"icon\":{\"className\":\"fas fa-exclamation-triangle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"warning\"},{\"background\":\"#007bff\",\"icon\":{\"className\":\"fas fa-info-circle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"info\"}],\"kind\":null,\"name\":\"types\"}]},{\"extends\":null,\"module\":null,\"name\":\"Notification\",\"overrides\":[],\"properties\":[{\"default\":null,\"kind\":null,\"name\":\"background\"},{\"default\":3000,\"kind\":null,\"name\":\"duration\"},{\"default\":null,\"kind\":null,\"name\":\"icon\"},{\"default\":\"\",\"kind\":null,\"name\":\"message\"},{\"default\":null,\"kind\":null,\"name\":\"notification_type\"},{\"default\":false,\"kind\":null,\"name\":\"_destroyed\"}]},{\"extends\":null,\"module\":null,\"name\":\"TemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]},{\"extends\":null,\"module\":null,\"name\":\"MaterialTemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]}],\"roots\":{\"references\":[{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"f1Nf/dcgM0AFQi97TC8wwA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"10985\"},\"selection_policy\":{\"id\":\"10984\"}},\"id\":\"10733\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"10613\",\"type\":\"HelpTool\"},{\"attributes\":{},\"id\":\"10962\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"axis\":{\"id\":\"10604\"},\"coordinates\":null,\"dimension\":1,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"10607\",\"type\":\"Grid\"},{\"attributes\":{\"line_color\":\"#99149F\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10740\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10605\",\"type\":\"BasicTicker\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.6},\"fill_color\":{\"field\":\"color\"},\"hatch_alpha\":{\"value\":0.6},\"hatch_color\":{\"field\":\"color\"},\"line_alpha\":{\"value\":0.6},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10624\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"10963\",\"type\":\"Selection\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10733\"},\"glyph\":{\"id\":\"10734\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10736\"},\"nonselection_glyph\":{\"id\":\"10735\"},\"view\":{\"id\":\"10738\"}},\"id\":\"10737\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"10609\",\"type\":\"WheelZoomTool\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#920FA2\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10736\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10608\",\"type\":\"PanTool\"},{\"attributes\":{\"source\":{\"id\":\"10733\"}},\"id\":\"10738\",\"type\":\"CDSView\"},{\"attributes\":{\"overlay\":{\"id\":\"10614\"}},\"id\":\"10610\",\"type\":\"BoxZoomTool\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#920FA2\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10735\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10611\",\"type\":\"SaveTool\"},{\"attributes\":{\"line_color\":\"#9F1A9B\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10746\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10612\",\"type\":\"ResetTool\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"6BQ9cA+lQECdaB8IuRs8wA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"10987\"},\"selection_policy\":{\"id\":\"10986\"}},\"id\":\"10739\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10739\"},\"glyph\":{\"id\":\"10740\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10742\"},\"nonselection_glyph\":{\"id\":\"10741\"},\"view\":{\"id\":\"10744\"}},\"id\":\"10743\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"10614\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#99149F\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10742\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10739\"}},\"id\":\"10744\",\"type\":\"CDSView\"},{\"attributes\":{\"source\":{\"id\":\"10661\"}},\"id\":\"10666\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#99149F\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10741\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#A51F97\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10752\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"Tvr5qiXWWkA8Y+a0GZxWwA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"10989\"},\"selection_policy\":{\"id\":\"10988\"}},\"id\":\"10745\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10622\"},\"glyph\":{\"id\":\"10624\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10626\"},\"nonselection_glyph\":{\"id\":\"10625\"},\"view\":{\"id\":\"10628\"}},\"id\":\"10627\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10745\"},\"glyph\":{\"id\":\"10746\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10748\"},\"nonselection_glyph\":{\"id\":\"10747\"},\"view\":{\"id\":\"10750\"}},\"id\":\"10749\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"10964\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#9F1A9B\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10748\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10745\"}},\"id\":\"10750\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#9F1A9B\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10747\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10965\",\"type\":\"Selection\"},{\"attributes\":{\"line_color\":\"#AC2593\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10758\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"exgJw2ntWMC0Ip1xDPRUQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"10991\"},\"selection_policy\":{\"id\":\"10990\"}},\"id\":\"10751\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10667\"},\"glyph\":{\"id\":\"10668\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10670\"},\"nonselection_glyph\":{\"id\":\"10669\"},\"view\":{\"id\":\"10672\"}},\"id\":\"10671\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10751\"},\"glyph\":{\"id\":\"10752\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10754\"},\"nonselection_glyph\":{\"id\":\"10753\"},\"view\":{\"id\":\"10756\"}},\"id\":\"10755\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"tools\":[{\"id\":\"10608\"},{\"id\":\"10609\"},{\"id\":\"10610\"},{\"id\":\"10611\"},{\"id\":\"10612\"},{\"id\":\"10613\"}]},\"id\":\"10615\",\"type\":\"Toolbar\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#A51F97\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10754\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10751\"}},\"id\":\"10756\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#A51F97\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10753\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#B22C8E\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10764\",\"type\":\"Line\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"field\":\"color\"},\"hatch_alpha\":{\"value\":0.1},\"hatch_color\":{\"field\":\"color\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10625\",\"type\":\"Circle\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"My6/b6PIQcB3iHTgu9M9QA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"10993\"},\"selection_policy\":{\"id\":\"10992\"}},\"id\":\"10757\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10757\"},\"glyph\":{\"id\":\"10758\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10760\"},\"nonselection_glyph\":{\"id\":\"10759\"},\"view\":{\"id\":\"10762\"}},\"id\":\"10761\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#AC2593\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10760\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10757\"}},\"id\":\"10762\",\"type\":\"CDSView\"},{\"attributes\":{\"background_fill_alpha\":0.2,\"background_fill_color\":\"darkgrey\",\"border_line_alpha\":0.8,\"border_line_color\":\"grey\",\"border_line_width\":3,\"coordinates\":null,\"group\":null,\"items\":[{\"id\":\"10640\"}],\"label_text_color\":\"white\"},\"id\":\"10639\",\"type\":\"Legend\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#AC2593\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10759\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#B83289\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10770\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10966\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"10632\",\"type\":\"AllLabels\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"UdDZUXcyNsCnGLHaC5IyQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"10995\"},\"selection_policy\":{\"id\":\"10994\"}},\"id\":\"10763\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.2},\"fill_color\":{\"field\":\"color\"},\"hatch_alpha\":{\"value\":0.2},\"hatch_color\":{\"field\":\"color\"},\"line_alpha\":{\"value\":0.2},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10626\",\"type\":\"Circle\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10763\"},\"glyph\":{\"id\":\"10764\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10766\"},\"nonselection_glyph\":{\"id\":\"10765\"},\"view\":{\"id\":\"10768\"}},\"id\":\"10767\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"10967\",\"type\":\"Selection\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#B22C8E\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10766\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10763\"}},\"id\":\"10768\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#B22C8E\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10765\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#BD3784\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10776\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"NWc4KIBpMMALacy4XGUrQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"10997\"},\"selection_policy\":{\"id\":\"10996\"}},\"id\":\"10769\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10769\"},\"glyph\":{\"id\":\"10770\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10772\"},\"nonselection_glyph\":{\"id\":\"10771\"},\"view\":{\"id\":\"10774\"}},\"id\":\"10773\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#B83289\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10772\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10631\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"source\":{\"id\":\"10769\"}},\"id\":\"10774\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#B83289\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10771\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#C23D80\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10782\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"s2kWOwdfKsD7H7y2OfUlQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"10999\"},\"selection_policy\":{\"id\":\"10998\"}},\"id\":\"10775\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"line_color\":\"#380499\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10668\",\"type\":\"Line\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10775\"},\"glyph\":{\"id\":\"10776\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10778\"},\"nonselection_glyph\":{\"id\":\"10777\"},\"view\":{\"id\":\"10780\"}},\"id\":\"10779\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#BD3784\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10778\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10968\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"source\":{\"id\":\"10775\"}},\"id\":\"10780\",\"type\":\"CDSView\"},{\"attributes\":{\"source\":{\"id\":\"10622\"}},\"id\":\"10628\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#BD3784\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10777\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#C7427B\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10788\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10969\",\"type\":\"Selection\"},{\"attributes\":{\"label\":{\"field\":\"label\"},\"renderers\":[{\"id\":\"10627\"}]},\"id\":\"10640\",\"type\":\"LegendItem\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"e8iaafdBJsA60CN/Qn0iQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11001\"},\"selection_policy\":{\"id\":\"11000\"}},\"id\":\"10781\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10781\"},\"glyph\":{\"id\":\"10782\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10784\"},\"nonselection_glyph\":{\"id\":\"10783\"},\"view\":{\"id\":\"10786\"}},\"id\":\"10785\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#C23D80\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10784\",\"type\":\"Line\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#380499\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10669\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10781\"}},\"id\":\"10786\",\"type\":\"CDSView\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"TP1bNQxHAUBvgSxT+Bb9vw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"10963\"},\"selection_policy\":{\"id\":\"10962\"}},\"id\":\"10667\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#C23D80\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10783\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#CD4975\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10794\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"QGS5xYdoI8CLDpMJxxUgQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11003\"},\"selection_policy\":{\"id\":\"11002\"}},\"id\":\"10787\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10787\"},\"glyph\":{\"id\":\"10788\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10790\"},\"nonselection_glyph\":{\"id\":\"10789\"},\"view\":{\"id\":\"10792\"}},\"id\":\"10791\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#C7427B\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10790\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10787\"}},\"id\":\"10792\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#C7427B\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10789\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#D14F71\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10800\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"7fqXNbpRIcCQPdIFbKQcQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11005\"},\"selection_policy\":{\"id\":\"11004\"}},\"id\":\"10793\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"10970\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#D14F71\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10801\",\"type\":\"Line\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10793\"},\"glyph\":{\"id\":\"10794\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10796\"},\"nonselection_glyph\":{\"id\":\"10795\"},\"view\":{\"id\":\"10798\"}},\"id\":\"10797\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#CD4975\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10796\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10971\",\"type\":\"Selection\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#380499\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10670\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10793\"}},\"id\":\"10798\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#CD4975\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10795\",\"type\":\"Line\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10799\"},\"glyph\":{\"id\":\"10800\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10802\"},\"nonselection_glyph\":{\"id\":\"10801\"},\"view\":{\"id\":\"10804\"}},\"id\":\"10803\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"Oun7YiZyH8AsXTX21fEZQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11007\"},\"selection_policy\":{\"id\":\"11006\"}},\"id\":\"10799\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"line_color\":\"#D6556D\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10806\",\"type\":\"Line\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#D14F71\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10802\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10799\"}},\"id\":\"10804\",\"type\":\"CDSView\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"+TsAVgnZ+j9RMe0aAEz2vw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"10959\"},\"selection_policy\":{\"id\":\"10958\"}},\"id\":\"10655\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"11018\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"11046\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"line_color\":\"#2F0495\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10662\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"11019\",\"type\":\"Selection\"},{\"attributes\":{\"source\":{\"id\":\"10667\"}},\"id\":\"10672\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"11047\",\"type\":\"Selection\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10655\"},\"glyph\":{\"id\":\"10656\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10658\"},\"nonselection_glyph\":{\"id\":\"10657\"},\"view\":{\"id\":\"10660\"}},\"id\":\"10659\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"background_fill_alpha\":0.9,\"background_fill_color\":\"#0f0f0f\",\"below\":[{\"id\":\"10600\"}],\"border_fill_alpha\":0.8,\"border_fill_color\":\"#111111\",\"center\":[{\"id\":\"10603\"},{\"id\":\"10607\"},{\"id\":\"10639\"}],\"height\":450,\"left\":[{\"id\":\"10604\"}],\"outline_line_alpha\":0.25,\"outline_line_color\":\"#E0E0E0\",\"outline_line_width\":3,\"renderers\":[{\"id\":\"10627\"},{\"id\":\"10647\"},{\"id\":\"10653\"},{\"id\":\"10659\"},{\"id\":\"10665\"},{\"id\":\"10671\"},{\"id\":\"10677\"},{\"id\":\"10683\"},{\"id\":\"10689\"},{\"id\":\"10695\"},{\"id\":\"10701\"},{\"id\":\"10707\"},{\"id\":\"10713\"},{\"id\":\"10719\"},{\"id\":\"10725\"},{\"id\":\"10731\"},{\"id\":\"10737\"},{\"id\":\"10743\"},{\"id\":\"10749\"},{\"id\":\"10755\"},{\"id\":\"10761\"},{\"id\":\"10767\"},{\"id\":\"10773\"},{\"id\":\"10779\"},{\"id\":\"10785\"},{\"id\":\"10791\"},{\"id\":\"10797\"},{\"id\":\"10803\"},{\"id\":\"10809\"},{\"id\":\"10815\"},{\"id\":\"10821\"},{\"id\":\"10827\"},{\"id\":\"10833\"},{\"id\":\"10839\"},{\"id\":\"10845\"},{\"id\":\"10851\"},{\"id\":\"10857\"},{\"id\":\"10863\"},{\"id\":\"10869\"},{\"id\":\"10875\"},{\"id\":\"10881\"},{\"id\":\"10887\"},{\"id\":\"10893\"},{\"id\":\"10899\"},{\"id\":\"10905\"},{\"id\":\"10911\"},{\"id\":\"10917\"},{\"id\":\"10923\"},{\"id\":\"10929\"},{\"id\":\"10935\"},{\"id\":\"10941\"}],\"right\":[{\"id\":\"10639\"}],\"title\":{\"id\":\"10590\"},\"toolbar\":{\"id\":\"10615\"},\"width\":700,\"x_range\":{\"id\":\"10641\"},\"x_scale\":{\"id\":\"10596\"},\"y_range\":{\"id\":\"10642\"},\"y_scale\":{\"id\":\"10598\"}},\"id\":\"10589\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#250591\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10658\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10655\"}},\"id\":\"10660\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#250591\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10657\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#42039D\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10674\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"lyb5c2Jt/j9dM4qhZnb5vw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"10961\"},\"selection_policy\":{\"id\":\"10960\"}},\"id\":\"10661\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10661\"},\"glyph\":{\"id\":\"10662\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10664\"},\"nonselection_glyph\":{\"id\":\"10663\"},\"view\":{\"id\":\"10666\"}},\"id\":\"10665\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#2F0495\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10664\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"wWpSiHKuA0ByRTf7zaQAwA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"10965\"},\"selection_policy\":{\"id\":\"10964\"}},\"id\":\"10673\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#2F0495\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10663\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#4A02A0\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10680\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"11020\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10673\"},\"glyph\":{\"id\":\"10674\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10676\"},\"nonselection_glyph\":{\"id\":\"10675\"},\"view\":{\"id\":\"10678\"}},\"id\":\"10677\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"11048\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#42039D\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10676\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10673\"}},\"id\":\"10678\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"11021\",\"type\":\"Selection\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#42039D\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10675\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"11049\",\"type\":\"Selection\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"iJcZs56DBkBk/0SxFxoDwA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"10967\"},\"selection_policy\":{\"id\":\"10966\"}},\"id\":\"10679\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"line_color\":\"#5201A3\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10686\",\"type\":\"Line\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10679\"},\"glyph\":{\"id\":\"10680\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10682\"},\"nonselection_glyph\":{\"id\":\"10681\"},\"view\":{\"id\":\"10684\"}},\"id\":\"10683\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#4A02A0\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10682\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10679\"}},\"id\":\"10684\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#4A02A0\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10681\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#5A00A5\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10692\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"rEyU3fflCUA5335BoAUGwA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"10969\"},\"selection_policy\":{\"id\":\"10968\"}},\"id\":\"10685\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10685\"},\"glyph\":{\"id\":\"10686\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10688\"},\"nonselection_glyph\":{\"id\":\"10687\"},\"view\":{\"id\":\"10690\"}},\"id\":\"10689\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#5201A3\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10688\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10685\"}},\"id\":\"10690\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#5201A3\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10687\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"11022\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"line_color\":\"#6200A6\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10698\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"11050\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"q9k2HCcCDkAGUCrw4IwJwA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"10971\"},\"selection_policy\":{\"id\":\"10970\"}},\"id\":\"10691\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"11023\",\"type\":\"Selection\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10691\"},\"glyph\":{\"id\":\"10692\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10694\"},\"nonselection_glyph\":{\"id\":\"10693\"},\"view\":{\"id\":\"10696\"}},\"id\":\"10695\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"11051\",\"type\":\"Selection\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#5A00A5\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10694\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10691\"}},\"id\":\"10696\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#5A00A5\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10693\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#6C00A8\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10704\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"IUKd9/WMEUB9p2z6GucNwA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"10973\"},\"selection_policy\":{\"id\":\"10972\"}},\"id\":\"10697\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10697\"},\"glyph\":{\"id\":\"10698\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10700\"},\"nonselection_glyph\":{\"id\":\"10699\"},\"view\":{\"id\":\"10702\"}},\"id\":\"10701\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#6200A6\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10700\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10697\"}},\"id\":\"10702\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#6200A6\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10699\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#7300A8\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10710\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"dHhL1BvJFEBuzVSumrQRwA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"10975\"},\"selection_policy\":{\"id\":\"10974\"}},\"id\":\"10703\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10703\"},\"glyph\":{\"id\":\"10704\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10706\"},\"nonselection_glyph\":{\"id\":\"10705\"},\"view\":{\"id\":\"10708\"}},\"id\":\"10707\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"11024\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#6C00A8\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10706\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"11052\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"source\":{\"id\":\"10703\"}},\"id\":\"10708\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#6C00A8\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10705\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"11025\",\"type\":\"Selection\"},{\"attributes\":{\"line_color\":\"#7B02A8\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10716\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"11053\",\"type\":\"Selection\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"pGtd8y4HGUC62KvITk4VwA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"10977\"},\"selection_policy\":{\"id\":\"10976\"}},\"id\":\"10709\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10709\"},\"glyph\":{\"id\":\"10710\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10712\"},\"nonselection_glyph\":{\"id\":\"10711\"},\"view\":{\"id\":\"10714\"}},\"id\":\"10713\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#7300A8\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10712\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10709\"}},\"id\":\"10714\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#7300A8\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10711\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#8204A7\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10722\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"y92nFYrUHkBAH48/nzcawA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"10979\"},\"selection_policy\":{\"id\":\"10978\"}},\"id\":\"10715\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10715\"},\"glyph\":{\"id\":\"10716\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10718\"},\"nonselection_glyph\":{\"id\":\"10717\"},\"view\":{\"id\":\"10720\"}},\"id\":\"10719\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#7B02A8\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10718\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10715\"}},\"id\":\"10720\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#7B02A8\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10717\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#8908A5\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10728\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"11026\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"BW51InieI0DOuCrL8qggwA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"10981\"},\"selection_policy\":{\"id\":\"10980\"}},\"id\":\"10721\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#8908A5\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10729\",\"type\":\"Line\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10721\"},\"glyph\":{\"id\":\"10722\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10724\"},\"nonselection_glyph\":{\"id\":\"10723\"},\"view\":{\"id\":\"10726\"}},\"id\":\"10725\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"11027\",\"type\":\"Selection\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#8204A7\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10724\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10721\"}},\"id\":\"10726\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#8204A7\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10723\",\"type\":\"Line\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10727\"},\"glyph\":{\"id\":\"10728\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10730\"},\"nonselection_glyph\":{\"id\":\"10729\"},\"view\":{\"id\":\"10732\"}},\"id\":\"10731\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"8CgahURAKkBXns25A0EmwA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"10983\"},\"selection_policy\":{\"id\":\"10982\"}},\"id\":\"10727\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"line_color\":\"#920FA2\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10734\",\"type\":\"Line\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#8908A5\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10730\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10727\"}},\"id\":\"10732\",\"type\":\"CDSView\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"DlizCwkgEcC0DPks3ZwLQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11033\"},\"selection_policy\":{\"id\":\"11032\"}},\"id\":\"10877\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"10634\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"line_color\":\"#FCAC32\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10884\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10635\",\"type\":\"AllLabels\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10877\"},\"glyph\":{\"id\":\"10878\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10880\"},\"nonselection_glyph\":{\"id\":\"10879\"},\"view\":{\"id\":\"10882\"}},\"id\":\"10881\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#FBA337\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10880\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"11000\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"source\":{\"id\":\"10877\"}},\"id\":\"10882\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"10972\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"11028\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#FBA337\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10879\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#FDB32E\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10890\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"11001\",\"type\":\"Selection\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"Lok82Km4EMDfCU5x7uwKQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11035\"},\"selection_policy\":{\"id\":\"11034\"}},\"id\":\"10883\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"10954\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"10973\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"11029\",\"type\":\"Selection\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10883\"},\"glyph\":{\"id\":\"10884\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10886\"},\"nonselection_glyph\":{\"id\":\"10885\"},\"view\":{\"id\":\"10888\"}},\"id\":\"10887\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"data\":{\"color\":[\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"midnightblue\",\"midnightblue\",\"firebrick\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"firebrick\",\"midnightblue\",\"firebrick\",\"midnightblue\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"firebrick\",\"firebrick\",\"midnightblue\",\"midnightblue\",\"firebrick\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"firebrick\",\"midnightblue\",\"firebrick\",\"firebrick\",\"midnightblue\",\"midnightblue\",\"firebrick\",\"firebrick\",\"midnightblue\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"midnightblue\",\"firebrick\",\"firebrick\",\"firebrick\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"firebrick\",\"midnightblue\",\"midnightblue\",\"firebrick\",\"midnightblue\",\"midnightblue\",\"firebrick\",\"midnightblue\",\"midnightblue\",\"firebrick\",\"midnightblue\",\"firebrick\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"firebrick\",\"firebrick\",\"firebrick\",\"midnightblue\",\"midnightblue\",\"firebrick\",\"firebrick\",\"midnightblue\",\"firebrick\",\"firebrick\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"firebrick\",\"midnightblue\",\"midnightblue\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"midnightblue\",\"midnightblue\",\"firebrick\",\"midnightblue\",\"firebrick\",\"firebrick\",\"midnightblue\"],\"label\":[\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 1\",\"class 1\",\"class 2\",\"class 1\",\"class 1\",\"class 1\",\"class 2\",\"class 1\",\"class 2\",\"class 1\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 1\",\"class 1\",\"class 1\",\"class 2\",\"class 2\",\"class 1\",\"class 1\",\"class 2\",\"class 1\",\"class 1\",\"class 1\",\"class 2\",\"class 1\",\"class 2\",\"class 2\",\"class 1\",\"class 1\",\"class 2\",\"class 2\",\"class 1\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 1\",\"class 2\",\"class 2\",\"class 2\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 2\",\"class 1\",\"class 1\",\"class 2\",\"class 1\",\"class 1\",\"class 2\",\"class 1\",\"class 1\",\"class 2\",\"class 1\",\"class 2\",\"class 1\",\"class 1\",\"class 1\",\"class 2\",\"class 2\",\"class 2\",\"class 1\",\"class 1\",\"class 2\",\"class 2\",\"class 1\",\"class 2\",\"class 2\",\"class 1\",\"class 1\",\"class 1\",\"class 2\",\"class 1\",\"class 1\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 1\",\"class 1\",\"class 2\",\"class 1\",\"class 2\",\"class 2\",\"class 1\"],\"x\":{\"__ndarray__\":\"as0OiTNR479gxq5drgDuv/gGXpsI0+m/FJz9UCZ31j+0X9/0aVPuPwwlCwVVJeI/CP38KYDiy78Aysf3ZRrYPzoN7kcsaO4/EMu1wyQo1j+efFByoDPmv7Jyfoh+cuU/2FYx5e0V0D+wdW1q51SyP4qT+/ASR+O/EGWJ+d6Nyz8wi+dPQdTGv+T4fKCWjd2/1Evy9vOS2r8sjerC3nXcPxiwdX6foNa/1pcf2aed57/kX4K0bH7gvwTZOcYWVNq//kl/oEpL7L8QJmKlwxezP5BtUkbT/cs/wrb1qRIo4T/mQwzH3OHov+pozTATFea/0vvUYRh/6D82vc++QXLiP1QHQe18l+C/tCHNzRmJ6T+c+RdaZljkPxjEQZZa69o/eNKOEGVMxb+GtIiOL1ziP7gqmFwi7O+/Sng7f0Wj4b/Qig++p27pP5hlbVDJC+U/XKGo/E6j5L9MjyKGVGDTv9i94c4qL+8/2CSIPO+W6r+ImIAA+Hbev9CkHY3Sgrc/QIVmbpKr6L8y14zmBjDkv8CyY6yVSsw/8iIXtZGC67+KLWFCdlDpvwrEeCM25Oy/KhVTHycJ5z+g+Ox2RCS5v5ygzyMWF9M/QFOTOlnVlT+UZHGRNrvTv5r0mA0Jl+g/kmrPGv6Q4D/g8FfMpYHLv4A3goQdANo/0BG/uG3nyD84rWmw2/DFv9DikQnmsLg/AL0QSP+1yz/gtTEF9HO/Py5qQbZYlOE/AOjza5l9hr/gnqT4bACiP/ba4g+fn+c/FBRGs2nB5j8g8ViqDFW7vzyPLkc7POm/9Gt6hY4d4L8Ck/ublm7iP9h8Z9zNGcw/bNWqj1ks7r/wCbQ+2VTTv/CX1b2sae0/+LEINTDj1z84pCepozLVv7aRezjcUO8/mL+T8+Ul7z+evunD1JHtP9TyDB/HuN+/0J2vQTic7j/40J74dbTpP9i4S7053M2/6NS1XDZ/7r8IvfMxbHfOP2CYbusAZ9I/ZE9JbeYt3z9I422hxODEP2BKBTW2Fu2/2CX93mKQ0L/IYHSbZ0bqvyrYgQzS8OS/SqAaGqYJ7D8=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]},\"y\":{\"__ndarray__\":\"gjXMT4md4T/cuXD6cXXiv8DtSFZviaa/Hv7tyixQ4T8Eng4+djDfPyiWY4vU/d6/CudCXw145j/wEh4WXnfMP4TRmWp0f9w/AHturCi3tz8ylRiVgJvnv6DUGXIJELO/JNo3EpX17j9gy2btvuTIvyy2aNZzp+U/Yv93b/to5z/Y+Jr4X3XSP0iWwXuUteu/OA8gXvzt77/E/EeNOmXvPxyvtmo+ce6/nAf6qUQM1D/IjjHkxhXeP7x1NaAoFtM/QA1e1W1lzT/05doLUhfYv6qs0PLuOeK/QFQFKZWRrL8kb2xZcYbfv04Q4s85sOu/xNbWT6wq6T/yVT91vefrv3BkDLVqpNG/lr+hTTjo4j/eOpp39NDmP16FY0XWjuq/sL5spHnP6T/ekSw7nbPgvwCv8AnQ3b+/UPPoM5ojxL/4/x1bz9zYP9SiyU+mWtW/nKYe0aMp2T80MsNbhTfUv9zoIK6XJNw/IJO3VVXS5z8sPs0xe4TQP3CMIW1raLQ/MGpSQ7A80L++96G8ZdLnvybMWwmij+O/JDZv1IOz7D+wI0YbZVPtv1CMpd+0wMw/mMTB8dvn6b/8j1AdvaLvvw77LuR29Oy/TGa1i/lE2r9U5X2rSQnlv97NuYGHguA/8ouElDO657/cuu1YdDDSP8oz7l0JjeC/+EX6RiGn3b/0eZLLMPvcP9ASCiQt4LG/BC91sGAL1L/gmLVUNgPlP3L5XplL1uk/QEC7QzriqD+QY12NWkruv7hrcl0ZJ+i/FGK140fL0j9gDBHQo4CrP4gyiVBMoMM/yiHdV4Fs5D8kjEb9MY7nP/xaHU3o+9K/4hLfxXNm5b9gSwmGtb+uvyA3ToQMcuu/aovAOJl96j+8GwTiQerqPzg/Das21s0/rL6Fx8O72r/EE0ecrO/Qv9b9cNxk3+i/dhHNKKJW4r92zwWge9fqv3BTPL7h/dc/ZPDajlSS379Az+iq0VHbPzQu+zqjpuY/bLzErHFo7b9wsgaWo2Gyv8qVn1HidOE/gEEhzVxc7b+Art+ZkIm5v1KA4Q15guy/8G+i0BNiwT8=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]}},\"selected\":{\"id\":\"10637\"},\"selection_policy\":{\"id\":\"10636\"}},\"id\":\"10622\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"11038\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#FCAC32\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10886\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"11010\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"source\":{\"id\":\"10883\"}},\"id\":\"10888\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"10636\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#FCAC32\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10885\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"11039\",\"type\":\"Selection\"},{\"attributes\":{\"line_color\":\"#FDBB2B\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10896\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10955\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"11011\",\"type\":\"Selection\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"GrxEmYFaEMAc5i5zpUwKQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11037\"},\"selection_policy\":{\"id\":\"11036\"}},\"id\":\"10889\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"10637\",\"type\":\"Selection\"},{\"attributes\":{\"end\":0.9811070212744188,\"start\":-0.9978010023482744},\"id\":\"10642\",\"type\":\"Range1d\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10889\"},\"glyph\":{\"id\":\"10890\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10892\"},\"nonselection_glyph\":{\"id\":\"10891\"},\"view\":{\"id\":\"10894\"}},\"id\":\"10893\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#FDB32E\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10892\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10889\"}},\"id\":\"10894\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#FDB32E\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10891\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#FDC328\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10902\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"LeTb0W0EEMAGZSWnFroJQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11039\"},\"selection_policy\":{\"id\":\"11038\"}},\"id\":\"10895\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"11002\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"10974\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"11030\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10895\"},\"glyph\":{\"id\":\"10896\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10898\"},\"nonselection_glyph\":{\"id\":\"10897\"},\"view\":{\"id\":\"10900\"}},\"id\":\"10899\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#FDBB2B\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10898\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"11003\",\"type\":\"Selection\"},{\"attributes\":{\"source\":{\"id\":\"10895\"}},\"id\":\"10900\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"10975\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"11031\",\"type\":\"Selection\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#FDBB2B\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10897\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#FCCB25\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10908\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"11040\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"W8irVvNqD8C4cluUozMJQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11041\"},\"selection_policy\":{\"id\":\"11040\"}},\"id\":\"10901\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"10956\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"11012\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10901\"},\"glyph\":{\"id\":\"10902\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10904\"},\"nonselection_glyph\":{\"id\":\"10903\"},\"view\":{\"id\":\"10906\"}},\"id\":\"10905\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"11041\",\"type\":\"Selection\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#FDC328\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10904\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10957\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"11013\",\"type\":\"Selection\"},{\"attributes\":{\"source\":{\"id\":\"10901\"}},\"id\":\"10906\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#FDC328\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10903\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10601\",\"type\":\"BasicTicker\"},{\"attributes\":{\"line_color\":\"#FAD524\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10914\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"g0aiVqrZDsCiV0ft67cIQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11043\"},\"selection_policy\":{\"id\":\"11042\"}},\"id\":\"10907\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10907\"},\"glyph\":{\"id\":\"10908\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10910\"},\"nonselection_glyph\":{\"id\":\"10909\"},\"view\":{\"id\":\"10912\"}},\"id\":\"10911\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#FCCB25\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10910\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10907\"}},\"id\":\"10912\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"11004\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#FCCB25\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10909\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10976\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"11032\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"line_color\":\"#F8DD24\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10920\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"pR8vEp1TDsDUB2RtwkUIQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11045\"},\"selection_policy\":{\"id\":\"11044\"}},\"id\":\"10913\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"11005\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"10977\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"11033\",\"type\":\"Selection\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10913\"},\"glyph\":{\"id\":\"10914\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10916\"},\"nonselection_glyph\":{\"id\":\"10915\"},\"view\":{\"id\":\"10918\"}},\"id\":\"10917\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#FAD524\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10916\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"11042\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"source\":{\"id\":\"10913\"}},\"id\":\"10918\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"10958\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"11014\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#FAD524\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10915\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#F6E525\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10926\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"11043\",\"type\":\"Selection\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"5lnNCJrXDcDuDdoWJNwHQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11047\"},\"selection_policy\":{\"id\":\"11046\"}},\"id\":\"10919\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"10959\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"11015\",\"type\":\"Selection\"},{\"attributes\":{\"end\":0.9786206343820172,\"start\":-0.9975749786838923},\"id\":\"10641\",\"type\":\"Range1d\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10919\"},\"glyph\":{\"id\":\"10920\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10922\"},\"nonselection_glyph\":{\"id\":\"10921\"},\"view\":{\"id\":\"10924\"}},\"id\":\"10923\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#F8DD24\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10922\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10919\"}},\"id\":\"10924\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#F8DD24\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10921\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#F3EE26\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10932\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"Wx+fp5hkDcCM8/J0MHoHQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11049\"},\"selection_policy\":{\"id\":\"11048\"}},\"id\":\"10925\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"11006\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10925\"},\"glyph\":{\"id\":\"10926\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10928\"},\"nonselection_glyph\":{\"id\":\"10927\"},\"view\":{\"id\":\"10930\"}},\"id\":\"10929\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"10978\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"11034\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"10596\",\"type\":\"LinearScale\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#F6E525\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10928\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10925\"}},\"id\":\"10930\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"11007\",\"type\":\"Selection\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#F6E525\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10927\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10979\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"11035\",\"type\":\"Selection\"},{\"attributes\":{\"line_color\":\"#EFF821\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10938\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"Ixaih7P5DMBSEDHXJB8HQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11051\"},\"selection_policy\":{\"id\":\"11050\"}},\"id\":\"10931\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#1B068C\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10651\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"11044\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"10960\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"11016\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10931\"},\"glyph\":{\"id\":\"10932\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10934\"},\"nonselection_glyph\":{\"id\":\"10933\"},\"view\":{\"id\":\"10936\"}},\"id\":\"10935\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_color\":\"#0C0786\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10644\",\"type\":\"Line\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#F3EE26\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10934\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#1B068C\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10650\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"11045\",\"type\":\"Selection\"},{\"attributes\":{\"source\":{\"id\":\"10931\"}},\"id\":\"10936\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"10961\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"11017\",\"type\":\"Selection\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#F3EE26\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10933\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"lKJP3fz09D+XAkC/6grxvw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"10955\"},\"selection_policy\":{\"id\":\"10954\"}},\"id\":\"10643\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"axis\":{\"id\":\"10600\"},\"coordinates\":null,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"10603\",\"type\":\"Grid\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"rnX8CCKWDMD0rUL1VsoGQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11053\"},\"selection_policy\":{\"id\":\"11052\"}},\"id\":\"10937\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10643\"},\"glyph\":{\"id\":\"10644\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10646\"},\"nonselection_glyph\":{\"id\":\"10645\"},\"view\":{\"id\":\"10648\"}},\"id\":\"10647\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10937\"},\"glyph\":{\"id\":\"10938\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10940\"},\"nonselection_glyph\":{\"id\":\"10939\"},\"view\":{\"id\":\"10942\"}},\"id\":\"10941\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"coordinates\":null,\"group\":null,\"text\":\"Observations\",\"text_color\":\"#E0E0E0\",\"text_font\":\"Josefin Sans\",\"text_font_size\":\"2.1em\"},\"id\":\"10590\",\"type\":\"Title\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#0C0786\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10646\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10598\",\"type\":\"LinearScale\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#EFF821\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10940\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10643\"}},\"id\":\"10648\",\"type\":\"CDSView\"},{\"attributes\":{\"source\":{\"id\":\"10937\"}},\"id\":\"10942\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#0C0786\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10645\",\"type\":\"Line\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#EFF821\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10939\",\"type\":\"Line\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10649\"},\"glyph\":{\"id\":\"10650\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10652\"},\"nonselection_glyph\":{\"id\":\"10651\"},\"view\":{\"id\":\"10654\"}},\"id\":\"10653\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"11008\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"10980\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"XUfl6Xy39z9Rnm4E1ILzvw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"10957\"},\"selection_policy\":{\"id\":\"10956\"}},\"id\":\"10649\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"11036\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"axis_label\":\"y\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"10631\"},\"group\":null,\"major_label_policy\":{\"id\":\"10632\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"10605\"}},\"id\":\"10604\",\"type\":\"LinearAxis\"},{\"attributes\":{\"line_color\":\"#250591\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10656\",\"type\":\"Line\"},{\"attributes\":{\"axis_label\":\"x\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"10634\"},\"group\":null,\"major_label_policy\":{\"id\":\"10635\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"10601\"}},\"id\":\"10600\",\"type\":\"LinearAxis\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#1B068C\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10652\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"11009\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"10981\",\"type\":\"Selection\"},{\"attributes\":{\"source\":{\"id\":\"10649\"}},\"id\":\"10654\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"11037\",\"type\":\"Selection\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"PHbNX7DtHMCwlxM+6dAXQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11009\"},\"selection_policy\":{\"id\":\"11008\"}},\"id\":\"10805\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"line_color\":\"#DA5A68\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10812\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10990\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"10991\",\"type\":\"Selection\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10805\"},\"glyph\":{\"id\":\"10806\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10808\"},\"nonselection_glyph\":{\"id\":\"10807\"},\"view\":{\"id\":\"10810\"}},\"id\":\"10809\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#D6556D\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10808\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10805\"}},\"id\":\"10810\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#D6556D\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10807\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#DE6064\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10818\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"p03R883kGsA7zpLBKxgWQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11011\"},\"selection_policy\":{\"id\":\"11010\"}},\"id\":\"10811\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10811\"},\"glyph\":{\"id\":\"10812\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10814\"},\"nonselection_glyph\":{\"id\":\"10813\"},\"view\":{\"id\":\"10816\"}},\"id\":\"10815\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#DA5A68\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10814\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10982\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"children\":[{\"id\":\"10944\"},{\"id\":\"10589\"},{\"id\":\"11165\"}],\"margin\":[0,0,0,0],\"name\":\"Row00291\"},\"id\":\"10943\",\"type\":\"Row\"},{\"attributes\":{\"source\":{\"id\":\"10811\"}},\"id\":\"10816\",\"type\":\"CDSView\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00288\",\"sizing_mode\":\"stretch_width\"},\"id\":\"11165\",\"type\":\"Spacer\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#DA5A68\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10813\",\"type\":\"Line\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00287\",\"sizing_mode\":\"stretch_width\"},\"id\":\"10944\",\"type\":\"Spacer\"},{\"attributes\":{\"line_color\":\"#E3675F\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10824\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10983\",\"type\":\"Selection\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"2S1qFFM3GcDp3BzHhqwUQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11013\"},\"selection_policy\":{\"id\":\"11012\"}},\"id\":\"10817\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10817\"},\"glyph\":{\"id\":\"10818\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10820\"},\"nonselection_glyph\":{\"id\":\"10819\"},\"view\":{\"id\":\"10822\"}},\"id\":\"10821\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"10992\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#DE6064\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10820\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10817\"}},\"id\":\"10822\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#DE6064\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10819\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10993\",\"type\":\"Selection\"},{\"attributes\":{\"line_color\":\"#E66D5A\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10830\",\"type\":\"Line\"},{\"attributes\":{\"client_comm_id\":\"e6ca93619f0f4a2e92c5006ee6d78017\",\"comm_id\":\"227cc0bab5df490cb913b8e9ce8a0cec\",\"plot_id\":\"10943\"},\"id\":\"11496\",\"type\":\"panel.models.comm_manager.CommManager\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"uyyLyFnPF8AI/TZGiXsTQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11015\"},\"selection_policy\":{\"id\":\"11014\"}},\"id\":\"10823\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10823\"},\"glyph\":{\"id\":\"10824\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10826\"},\"nonselection_glyph\":{\"id\":\"10825\"},\"view\":{\"id\":\"10828\"}},\"id\":\"10827\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#E3675F\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10826\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10823\"}},\"id\":\"10828\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#E3675F\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10825\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#EA7356\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10836\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"RX0SRXWdFsDMXF0CNXgSQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11017\"},\"selection_policy\":{\"id\":\"11016\"}},\"id\":\"10829\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"10984\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10829\"},\"glyph\":{\"id\":\"10830\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10832\"},\"nonselection_glyph\":{\"id\":\"10831\"},\"view\":{\"id\":\"10834\"}},\"id\":\"10833\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#E66D5A\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10832\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10985\",\"type\":\"Selection\"},{\"attributes\":{\"source\":{\"id\":\"10829\"}},\"id\":\"10834\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#E66D5A\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10831\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#ED7952\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10842\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10994\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"NZEiH3iWFcDEgDYqH5kRQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11019\"},\"selection_policy\":{\"id\":\"11018\"}},\"id\":\"10835\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10835\"},\"glyph\":{\"id\":\"10836\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10838\"},\"nonselection_glyph\":{\"id\":\"10837\"},\"view\":{\"id\":\"10840\"}},\"id\":\"10839\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"10995\",\"type\":\"Selection\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#EA7356\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10838\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10835\"}},\"id\":\"10840\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#EA7356\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10837\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#F0804D\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10848\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"bUb+6BWyFMAhrotESdcQQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11021\"},\"selection_policy\":{\"id\":\"11020\"}},\"id\":\"10841\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10841\"},\"glyph\":{\"id\":\"10842\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10844\"},\"nonselection_glyph\":{\"id\":\"10843\"},\"view\":{\"id\":\"10846\"}},\"id\":\"10845\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#ED7952\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10844\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10841\"}},\"id\":\"10846\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"10986\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#ED7952\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10843\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#F38748\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10854\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"jK9GXgTqE8AXwlzIZS0QQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11023\"},\"selection_policy\":{\"id\":\"11022\"}},\"id\":\"10847\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"10987\",\"type\":\"Selection\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10847\"},\"glyph\":{\"id\":\"10848\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10850\"},\"nonselection_glyph\":{\"id\":\"10849\"},\"view\":{\"id\":\"10852\"}},\"id\":\"10851\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#F0804D\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10850\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10996\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"source\":{\"id\":\"10847\"}},\"id\":\"10852\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#F0804D\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10849\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#F68E44\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10860\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10997\",\"type\":\"Selection\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"QdJEcGg5E8C6PkXFuC4PQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11025\"},\"selection_policy\":{\"id\":\"11024\"}},\"id\":\"10853\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10853\"},\"glyph\":{\"id\":\"10854\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10856\"},\"nonselection_glyph\":{\"id\":\"10855\"},\"view\":{\"id\":\"10858\"}},\"id\":\"10857\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#F38748\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10856\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10853\"}},\"id\":\"10858\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#F38748\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10855\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#F89540\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10866\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"r9P/oXOcEsDghnPP7SMOQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11027\"},\"selection_policy\":{\"id\":\"11026\"}},\"id\":\"10859\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"10988\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10859\"},\"glyph\":{\"id\":\"10860\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10862\"},\"nonselection_glyph\":{\"id\":\"10861\"},\"view\":{\"id\":\"10864\"}},\"id\":\"10863\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#F68E44\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10862\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10859\"}},\"id\":\"10864\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"10989\",\"type\":\"Selection\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#F68E44\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10861\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"#FA9C3B\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10872\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"NNfa4B4QEsD5n7hSTjUNQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11029\"},\"selection_policy\":{\"id\":\"11028\"}},\"id\":\"10865\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"10998\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#FA9C3B\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10873\",\"type\":\"Line\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10865\"},\"glyph\":{\"id\":\"10866\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10868\"},\"nonselection_glyph\":{\"id\":\"10867\"},\"view\":{\"id\":\"10870\"}},\"id\":\"10869\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#F89540\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10868\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"10999\",\"type\":\"Selection\"},{\"attributes\":{\"source\":{\"id\":\"10865\"}},\"id\":\"10870\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#F89540\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10867\",\"type\":\"Line\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"10871\"},\"glyph\":{\"id\":\"10872\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"10874\"},\"nonselection_glyph\":{\"id\":\"10873\"},\"view\":{\"id\":\"10876\"}},\"id\":\"10875\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"uCqYXCLs77+2kXs43FDvPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]},\"y\":{\"__ndarray__\":\"Ly1SEPqREcDmsVhPvF4MQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[2]}},\"selected\":{\"id\":\"11031\"},\"selection_policy\":{\"id\":\"11030\"}},\"id\":\"10871\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"line_color\":\"#FBA337\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10878\",\"type\":\"Line\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"#FA9C3B\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"10874\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"10871\"}},\"id\":\"10876\",\"type\":\"CDSView\"}],\"root_ids\":[\"10943\",\"11496\"]},\"title\":\"Bokeh Application\",\"version\":\"2.4.3\"}};\n",
       "    var render_items = [{\"docid\":\"01f6016f-7c38-4dc6-927d-8fe974e68ceb\",\"root_ids\":[\"10943\"],\"roots\":{\"10943\":\"cb1f00b2-35bb-42aa-985b-45ccf221d183\"}}];\n",
       "    root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "    for (const render_item of render_items) {\n",
       "      for (const root_id of render_item.root_ids) {\n",
       "\tconst id_el = document.getElementById(root_id)\n",
       "\tif (id_el.children.length && (id_el.children[0].className === 'bk-root')) {\n",
       "\t  const root_el = id_el.children[0]\n",
       "\t  root_el.id = root_el.id + '-rendered'\n",
       "\t}\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "  if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    var attempts = 0;\n",
       "    var timer = setInterval(function(root) {\n",
       "      if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else if (document.readyState == \"complete\") {\n",
       "        attempts++;\n",
       "        if (attempts > 200) {\n",
       "          clearInterval(timer);\n",
       "          console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\");\n",
       "        }\n",
       "      }\n",
       "    }, 25, root)\n",
       "  }\n",
       "})(window);</script>"
      ],
      "text/plain": [
       "Row\n",
       "    [0] HSpacer()\n",
       "    [1] Bokeh(Figure)\n",
       "    [2] HSpacer()"
      ]
     },
     "execution_count": 50,
     "metadata": {
      "application/vnd.holoviews_exec.v0+json": {
       "id": "10943"
      }
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Update loop\n",
    "for i in range(n_epochs):\n",
    "    \n",
    "    ######################\n",
    "    # YOUR CODE GOES HERE\n",
    "    ######################\n",
    "    \n",
    "    ######################\n",
    "    # Solution \n",
    "    gradients = grad_loss_function(x_jax, y_jax, weights, bias)\n",
    "    weights = weights - eta * gradients[0]\n",
    "    bias = bias - eta * gradients[1]\n",
    "    ######################\n",
    "    weights_history = weights_history.at[i + 1].set(weights[0])\n",
    "    bias_history = bias_history.at[i + 1].set(bias[0])\n",
    "    \n",
    "    if (verbose):\n",
    "        print('%2d. error = %f, weights = %f, %f, %f' % (i, np.sum(np.abs(error)) / n_observations, bias[0, 0], weights[0, 0], weights[0, 1]))\n",
    "plot = center_plot(\n",
    "    scatter_boundary(\n",
    "        np.array(x_inputs), \n",
    "        np.array(y_class), \n",
    "        np.array(weights_history), \n",
    "        np.array(bias_history), \n",
    "        i + 1)\n",
    "    )\n",
    "plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Multi-layer networks\n",
    "\n",
    "In the following, we define the overall architecture of the exercise to fill. The goal is to define a more advanced problem \n",
    "\n",
    "### 2-layer XOR problem\n",
    "\n",
    "In most cases, classification problems are far from being linear. Therefore, we need more advanced methods to be able to compute non-linear class boundaries. The advantage of neural networks is that the same principle can be applied in a *layer-wise* fashion. This allows to further discriminate the space in sub-regions (as seen in the course). We will try to implement the 2-layer *perceptron* that can provide a solution to the infamous XOR problem. The idea is now to have the output of the first neurons to be connected to a set of other neurons. Therefore, if we take back our previous formulation, we have the same output for the first neuron(s) $y$, that we will now term as $y^{(1)}$. Then, we feed these outputs to a second layer of neurons, which gives\n",
    "\n",
    "\n",
    "$$\n",
    "\\begin{equation}\n",
    "y^{(2)}=\\sigma\\left(\\sum_{i = 1}^{n}w_{i}.y^{(1)}_{i} + b\\right)\n",
    "\\end{equation}\n",
    "$$  \n",
    "\n",
    "Finally, we will rely on the same loss $\\mathcal{L_{D}}$ as in the previous exercise, but the outputs used are $y^{(2)}$ instead of $y$. As in the previous case, we now need to compute the derivatives of the weights and biases for several layers . However, you should see that some form of generalization might be possible for any number of layer.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***\n",
    "\n",
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #192841; border-color: #779ecb\">\n",
    "\n",
    "> ### Exercise (course)\n",
    ">   1. Perform the derivatives for the last layer specifically\n",
    ">   2. Define a generalized derivative for any previous layer\n",
    "\n",
    "</div>\n",
    "\n",
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can construct the prototypical set of XOR values by using the following code (note that this is the most simple case, but still this is typically a problem that cannot be solved by a _linear classifier_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9078103cecac492aac7b4c32cd0fa5e7",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "BokehModel(combine_events=True, render_bundle={'docs_json': {'de2b6b77-74be-4248-9acd-e7940c151767': {'version…"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "from cml.plot import center_plot, scatter_classes\n",
    "# Input patterns\n",
    "n_augment = 1000\n",
    "x_inputs = np.array([[-1, -1],[-1,  1],[1, -1],[1,  1]])\n",
    "y_class = np.array([0, 1, 1, 0])\n",
    "augmented_x = []\n",
    "augmented_y = []\n",
    "for x, y in zip(x_inputs, y_class):\n",
    "    augmented_x.append(x + (np.random.randn(n_augment, 2) * 0.2))\n",
    "    augmented_y.append(np.repeat(y, n_augment))\n",
    "augmented_x = np.concatenate(augmented_x)\n",
    "augmented_y = np.concatenate(augmented_y)\n",
    "# Corresponding classes\n",
    "# Initialize based on their sizes\n",
    "n_inputs = x_inputs.shape[0]\n",
    "n_outputs = 1\n",
    "# Plot the XOR problem\n",
    "plot = center_plot(scatter_classes(augmented_x, augmented_y, title = \"XOR classification\"))\n",
    "plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Observing the problem interactively"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating the object\n"
     ]
    },
    {
     "data": {},
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.holoviews_exec.v0+json": "",
      "text/html": [
       "<div id='b06edcc4-a1f7-41b4-8757-223a14ca4d44'>\n",
       "  <div id=\"b2784f13-bd52-421d-a26d-d047186809d8\" data-root-id=\"b06edcc4-a1f7-41b4-8757-223a14ca4d44\" style=\"display: contents;\"></div>\n",
       "</div>\n",
       "<script type=\"application/javascript\">(function(root) {\n",
       "  var docs_json = {\"4f6e6553-a494-4971-933b-b068f8dedd6a\":{\"version\":\"3.2.0\",\"title\":\"Bokeh Application\",\"roots\":[{\"type\":\"object\",\"name\":\"Column\",\"id\":\"b06edcc4-a1f7-41b4-8757-223a14ca4d44\",\"attributes\":{\"name\":\"Column00492\",\"stylesheets\":[\"\\n:host(.pn-loading.pn-arc):before, .pn-loading.pn-arc:before {\\n  background-image: url(\\\"data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHN0eWxlPSJtYXJnaW46IGF1dG87IGJhY2tncm91bmQ6IG5vbmU7IGRpc3BsYXk6IGJsb2NrOyBzaGFwZS1yZW5kZXJpbmc6IGF1dG87IiB2aWV3Qm94PSIwIDAgMTAwIDEwMCIgcHJlc2VydmVBc3BlY3RSYXRpbz0ieE1pZFlNaWQiPiAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgZmlsbD0ibm9uZSIgc3Ryb2tlPSIjYzNjM2MzIiBzdHJva2Utd2lkdGg9IjEwIiByPSIzNSIgc3Ryb2tlLWRhc2hhcnJheT0iMTY0LjkzMzYxNDMxMzQ2NDE1IDU2Ljk3Nzg3MTQzNzgyMTM4Ij4gICAgPGFuaW1hdGVUcmFuc2Zvcm0gYXR0cmlidXRlTmFtZT0idHJhbnNmb3JtIiB0eXBlPSJyb3RhdGUiIHJlcGVhdENvdW50PSJpbmRlZmluaXRlIiBkdXI9IjFzIiB2YWx1ZXM9IjAgNTAgNTA7MzYwIDUwIDUwIiBrZXlUaW1lcz0iMDsxIj48L2FuaW1hdGVUcmFuc2Zvcm0+ICA8L2NpcmNsZT48L3N2Zz4=\\\");\\n  background-size: auto calc(min(50%, 400px));\\n}\",{\"type\":\"object\",\"name\":\"ImportedStyleSheet\",\"id\":\"fcdd54a8-c48b-4b51-89e8-ada0b4aff044\",\"attributes\":{\"url\":\"https://cdn.holoviz.org/panel/1.1.0/dist/css/loading.css\"}},{\"type\":\"object\",\"name\":\"ImportedStyleSheet\",\"id\":\"1a828e8b-4990-4e28-b528-2ba8080973ce\",\"attributes\":{\"url\":\"cml/cml.css\"}},{\"type\":\"object\",\"name\":\"ImportedStyleSheet\",\"id\":\"7a3bc9f4-c246-4436-b7af-1e5773db43f1\",\"attributes\":{\"url\":\"https://cdn.holoviz.org/panel/1.1.0/dist/css/listpanel.css\"}},{\"type\":\"object\",\"name\":\"ImportedStyleSheet\",\"id\":\"8adb45d0-3f9f-492d-b1df-7471f07cfdd7\",\"attributes\":{\"url\":\"https://cdn.holoviz.org/panel/1.1.0/dist/bundled/theme/default.css\"}},{\"type\":\"object\",\"name\":\"ImportedStyleSheet\",\"id\":\"ab77e90b-a838-4b1c-8c37-133586dc5cac\",\"attributes\":{\"url\":\"https://cdn.holoviz.org/panel/1.1.0/dist/bundled/theme/native.css\"}}],\"margin\":0,\"align\":\"start\",\"children\":[{\"type\":\"object\",\"name\":\"Row\",\"id\":\"f28aeaad-6a1e-4704-a290-5254cac512fe\",\"attributes\":{\"name\":\"Row00485\",\"stylesheets\":[\"\\n:host(.pn-loading.pn-arc):before, .pn-loading.pn-arc:before {\\n  background-image: url(\\\"data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHN0eWxlPSJtYXJnaW46IGF1dG87IGJhY2tncm91bmQ6IG5vbmU7IGRpc3BsYXk6IGJsb2NrOyBzaGFwZS1yZW5kZXJpbmc6IGF1dG87IiB2aWV3Qm94PSIwIDAgMTAwIDEwMCIgcHJlc2VydmVBc3BlY3RSYXRpbz0ieE1pZFlNaWQiPiAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgZmlsbD0ibm9uZSIgc3Ryb2tlPSIjYzNjM2MzIiBzdHJva2Utd2lkdGg9IjEwIiByPSIzNSIgc3Ryb2tlLWRhc2hhcnJheT0iMTY0LjkzMzYxNDMxMzQ2NDE1IDU2Ljk3Nzg3MTQzNzgyMTM4Ij4gICAgPGFuaW1hdGVUcmFuc2Zvcm0gYXR0cmlidXRlTmFtZT0idHJhbnNmb3JtIiB0eXBlPSJyb3RhdGUiIHJlcGVhdENvdW50PSJpbmRlZmluaXRlIiBkdXI9IjFzIiB2YWx1ZXM9IjAgNTAgNTA7MzYwIDUwIDUwIiBrZXlUaW1lcz0iMDsxIj48L2FuaW1hdGVUcmFuc2Zvcm0+ICA8L2NpcmNsZT48L3N2Zz4=\\\");\\n  background-size: auto calc(min(50%, 400px));\\n}\",{\"id\":\"fcdd54a8-c48b-4b51-89e8-ada0b4aff044\"},{\"id\":\"1a828e8b-4990-4e28-b528-2ba8080973ce\"},{\"id\":\"7a3bc9f4-c246-4436-b7af-1e5773db43f1\"},{\"id\":\"8adb45d0-3f9f-492d-b1df-7471f07cfdd7\"},{\"id\":\"ab77e90b-a838-4b1c-8c37-133586dc5cac\"}],\"margin\":0,\"align\":\"start\",\"children\":[{\"type\":\"object\",\"name\":\"Column\",\"id\":\"6d7c392c-c26d-477a-a2c3-1351c0469f7e\",\"attributes\":{\"name\":\"Problem parameters\",\"stylesheets\":[\"\\n:host(.pn-loading.pn-arc):before, .pn-loading.pn-arc:before {\\n  background-image: url(\\\"data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHN0eWxlPSJtYXJnaW46IGF1dG87IGJhY2tncm91bmQ6IG5vbmU7IGRpc3BsYXk6IGJsb2NrOyBzaGFwZS1yZW5kZXJpbmc6IGF1dG87IiB2aWV3Qm94PSIwIDAgMTAwIDEwMCIgcHJlc2VydmVBc3BlY3RSYXRpbz0ieE1pZFlNaWQiPiAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgZmlsbD0ibm9uZSIgc3Ryb2tlPSIjYzNjM2MzIiBzdHJva2Utd2lkdGg9IjEwIiByPSIzNSIgc3Ryb2tlLWRhc2hhcnJheT0iMTY0LjkzMzYxNDMxMzQ2NDE1IDU2Ljk3Nzg3MTQzNzgyMTM4Ij4gICAgPGFuaW1hdGVUcmFuc2Zvcm0gYXR0cmlidXRlTmFtZT0idHJhbnNmb3JtIiB0eXBlPSJyb3RhdGUiIHJlcGVhdENvdW50PSJpbmRlZmluaXRlIiBkdXI9IjFzIiB2YWx1ZXM9IjAgNTAgNTA7MzYwIDUwIDUwIiBrZXlUaW1lcz0iMDsxIj48L2FuaW1hdGVUcmFuc2Zvcm0+ICA8L2NpcmNsZT48L3N2Zz4=\\\");\\n  background-size: auto calc(min(50%, 400px));\\n}\",{\"id\":\"fcdd54a8-c48b-4b51-89e8-ada0b4aff044\"},{\"id\":\"1a828e8b-4990-4e28-b528-2ba8080973ce\"},{\"id\":\"7a3bc9f4-c246-4436-b7af-1e5773db43f1\"},{\"id\":\"8adb45d0-3f9f-492d-b1df-7471f07cfdd7\"},{\"id\":\"ab77e90b-a838-4b1c-8c37-133586dc5cac\"}],\"margin\":[5,10],\"align\":\"start\",\"children\":[{\"type\":\"object\",\"name\":\"Div\",\"id\":\"2ddad829-e435-41e5-b06f-7b40df98625a\",\"attributes\":{\"stylesheets\":[\"\\n:host(.pn-loading.pn-arc):before, .pn-loading.pn-arc:before {\\n  background-image: url(\\\"data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHN0eWxlPSJtYXJnaW46IGF1dG87IGJhY2tncm91bmQ6IG5vbmU7IGRpc3BsYXk6IGJsb2NrOyBzaGFwZS1yZW5kZXJpbmc6IGF1dG87IiB2aWV3Qm94PSIwIDAgMTAwIDEwMCIgcHJlc2VydmVBc3BlY3RSYXRpbz0ieE1pZFlNaWQiPiAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgZmlsbD0ibm9uZSIgc3Ryb2tlPSIjYzNjM2MzIiBzdHJva2Utd2lkdGg9IjEwIiByPSIzNSIgc3Ryb2tlLWRhc2hhcnJheT0iMTY0LjkzMzYxNDMxMzQ2NDE1IDU2Ljk3Nzg3MTQzNzgyMTM4Ij4gICAgPGFuaW1hdGVUcmFuc2Zvcm0gYXR0cmlidXRlTmFtZT0idHJhbnNmb3JtIiB0eXBlPSJyb3RhdGUiIHJlcGVhdENvdW50PSJpbmRlZmluaXRlIiBkdXI9IjFzIiB2YWx1ZXM9IjAgNTAgNTA7MzYwIDUwIDUwIiBrZXlUaW1lcz0iMDsxIj48L2FuaW1hdGVUcmFuc2Zvcm0+ICA8L2NpcmNsZT48L3N2Zz4=\\\");\\n  background-size: auto calc(min(50%, 400px));\\n}\",{\"id\":\"fcdd54a8-c48b-4b51-89e8-ada0b4aff044\"},{\"id\":\"1a828e8b-4990-4e28-b528-2ba8080973ce\"},{\"id\":\"8adb45d0-3f9f-492d-b1df-7471f07cfdd7\"},{\"id\":\"ab77e90b-a838-4b1c-8c37-133586dc5cac\"}],\"margin\":[5,10],\"align\":\"start\",\"text\":\"<b>Problem parameters</b>\"}},{\"type\":\"object\",\"name\":\"Slider\",\"id\":\"822fcd2c-2f14-46bc-b8f2-6e3fe61f9433\",\"attributes\":{\"stylesheets\":[\"\\n:host(.pn-loading.pn-arc):before, .pn-loading.pn-arc:before {\\n  background-image: url(\\\"data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHN0eWxlPSJtYXJnaW46IGF1dG87IGJhY2tncm91bmQ6IG5vbmU7IGRpc3BsYXk6IGJsb2NrOyBzaGFwZS1yZW5kZXJpbmc6IGF1dG87IiB2aWV3Qm94PSIwIDAgMTAwIDEwMCIgcHJlc2VydmVBc3BlY3RSYXRpbz0ieE1pZFlNaWQiPiAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgZmlsbD0ibm9uZSIgc3Ryb2tlPSIjYzNjM2MzIiBzdHJva2Utd2lkdGg9IjEwIiByPSIzNSIgc3Ryb2tlLWRhc2hhcnJheT0iMTY0LjkzMzYxNDMxMzQ2NDE1IDU2Ljk3Nzg3MTQzNzgyMTM4Ij4gICAgPGFuaW1hdGVUcmFuc2Zvcm0gYXR0cmlidXRlTmFtZT0idHJhbnNmb3JtIiB0eXBlPSJyb3RhdGUiIHJlcGVhdENvdW50PSJpbmRlZmluaXRlIiBkdXI9IjFzIiB2YWx1ZXM9IjAgNTAgNTA7MzYwIDUwIDUwIiBrZXlUaW1lcz0iMDsxIj48L2FuaW1hdGVUcmFuc2Zvcm0+ICA8L2NpcmNsZT48L3N2Zz4=\\\");\\n  background-size: auto calc(min(50%, 400px));\\n}\",{\"id\":\"fcdd54a8-c48b-4b51-89e8-ada0b4aff044\"},{\"id\":\"1a828e8b-4990-4e28-b528-2ba8080973ce\"},{\"id\":\"8adb45d0-3f9f-492d-b1df-7471f07cfdd7\"},{\"id\":\"ab77e90b-a838-4b1c-8c37-133586dc5cac\"}],\"margin\":[5,10],\"align\":\"start\",\"title\":\"Number observations\",\"start\":10,\"end\":1000,\"value\":100}},{\"type\":\"object\",\"name\":\"Slider\",\"id\":\"9d47cec0-fecc-4924-8375-73b7ed849e5c\",\"attributes\":{\"stylesheets\":[\"\\n:host(.pn-loading.pn-arc):before, .pn-loading.pn-arc:before {\\n  background-image: url(\\\"data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHN0eWxlPSJtYXJnaW46IGF1dG87IGJhY2tncm91bmQ6IG5vbmU7IGRpc3BsYXk6IGJsb2NrOyBzaGFwZS1yZW5kZXJpbmc6IGF1dG87IiB2aWV3Qm94PSIwIDAgMTAwIDEwMCIgcHJlc2VydmVBc3BlY3RSYXRpbz0ieE1pZFlNaWQiPiAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgZmlsbD0ibm9uZSIgc3Ryb2tlPSIjYzNjM2MzIiBzdHJva2Utd2lkdGg9IjEwIiByPSIzNSIgc3Ryb2tlLWRhc2hhcnJheT0iMTY0LjkzMzYxNDMxMzQ2NDE1IDU2Ljk3Nzg3MTQzNzgyMTM4Ij4gICAgPGFuaW1hdGVUcmFuc2Zvcm0gYXR0cmlidXRlTmFtZT0idHJhbnNmb3JtIiB0eXBlPSJyb3RhdGUiIHJlcGVhdENvdW50PSJpbmRlZmluaXRlIiBkdXI9IjFzIiB2YWx1ZXM9IjAgNTAgNTA7MzYwIDUwIDUwIiBrZXlUaW1lcz0iMDsxIj48L2FuaW1hdGVUcmFuc2Zvcm0+ICA8L2NpcmNsZT48L3N2Zz4=\\\");\\n  background-size: auto calc(min(50%, 400px));\\n}\",{\"id\":\"fcdd54a8-c48b-4b51-89e8-ada0b4aff044\"},{\"id\":\"1a828e8b-4990-4e28-b528-2ba8080973ce\"},{\"id\":\"8adb45d0-3f9f-492d-b1df-7471f07cfdd7\"},{\"id\":\"ab77e90b-a838-4b1c-8c37-133586dc5cac\"}],\"margin\":[5,10],\"align\":\"start\",\"title\":\"Noise level\",\"start\":0.0,\"end\":5.0,\"value\":0.1,\"step\":0.1}},{\"type\":\"object\",\"name\":\"Checkbox\",\"id\":\"cb8e17ac-0430-4852-b598-f09f0b3f6967\",\"attributes\":{\"stylesheets\":[\"\\n:host(.pn-loading.pn-arc):before, .pn-loading.pn-arc:before {\\n  background-image: url(\\\"data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHN0eWxlPSJtYXJnaW46IGF1dG87IGJhY2tncm91bmQ6IG5vbmU7IGRpc3BsYXk6IGJsb2NrOyBzaGFwZS1yZW5kZXJpbmc6IGF1dG87IiB2aWV3Qm94PSIwIDAgMTAwIDEwMCIgcHJlc2VydmVBc3BlY3RSYXRpbz0ieE1pZFlNaWQiPiAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgZmlsbD0ibm9uZSIgc3Ryb2tlPSIjYzNjM2MzIiBzdHJva2Utd2lkdGg9IjEwIiByPSIzNSIgc3Ryb2tlLWRhc2hhcnJheT0iMTY0LjkzMzYxNDMxMzQ2NDE1IDU2Ljk3Nzg3MTQzNzgyMTM4Ij4gICAgPGFuaW1hdGVUcmFuc2Zvcm0gYXR0cmlidXRlTmFtZT0idHJhbnNmb3JtIiB0eXBlPSJyb3RhdGUiIHJlcGVhdENvdW50PSJpbmRlZmluaXRlIiBkdXI9IjFzIiB2YWx1ZXM9IjAgNTAgNTA7MzYwIDUwIDUwIiBrZXlUaW1lcz0iMDsxIj48L2FuaW1hdGVUcmFuc2Zvcm0+ICA8L2NpcmNsZT48L3N2Zz4=\\\");\\n  background-size: auto calc(min(50%, 400px));\\n}\",{\"id\":\"fcdd54a8-c48b-4b51-89e8-ada0b4aff044\"},{\"id\":\"1a828e8b-4990-4e28-b528-2ba8080973ce\"},{\"id\":\"8adb45d0-3f9f-492d-b1df-7471f07cfdd7\"},{\"id\":\"ab77e90b-a838-4b1c-8c37-133586dc5cac\"}],\"margin\":[5,10],\"align\":\"start\",\"label\":\"Ground truth\"}},{\"type\":\"object\",\"name\":\"Checkbox\",\"id\":\"b8752df4-2f6c-4834-872a-c9d48b5e56ed\",\"attributes\":{\"stylesheets\":[\"\\n:host(.pn-loading.pn-arc):before, .pn-loading.pn-arc:before {\\n  background-image: url(\\\"data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHN0eWxlPSJtYXJnaW46IGF1dG87IGJhY2tncm91bmQ6IG5vbmU7IGRpc3BsYXk6IGJsb2NrOyBzaGFwZS1yZW5kZXJpbmc6IGF1dG87IiB2aWV3Qm94PSIwIDAgMTAwIDEwMCIgcHJlc2VydmVBc3BlY3RSYXRpbz0ieE1pZFlNaWQiPiAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgZmlsbD0ibm9uZSIgc3Ryb2tlPSIjYzNjM2MzIiBzdHJva2Utd2lkdGg9IjEwIiByPSIzNSIgc3Ryb2tlLWRhc2hhcnJheT0iMTY0LjkzMzYxNDMxMzQ2NDE1IDU2Ljk3Nzg3MTQzNzgyMTM4Ij4gICAgPGFuaW1hdGVUcmFuc2Zvcm0gYXR0cmlidXRlTmFtZT0idHJhbnNmb3JtIiB0eXBlPSJyb3RhdGUiIHJlcGVhdENvdW50PSJpbmRlZmluaXRlIiBkdXI9IjFzIiB2YWx1ZXM9IjAgNTAgNTA7MzYwIDUwIDUwIiBrZXlUaW1lcz0iMDsxIj48L2FuaW1hdGVUcmFuc2Zvcm0+ICA8L2NpcmNsZT48L3N2Zz4=\\\");\\n  background-size: auto calc(min(50%, 400px));\\n}\",{\"id\":\"fcdd54a8-c48b-4b51-89e8-ada0b4aff044\"},{\"id\":\"1a828e8b-4990-4e28-b528-2ba8080973ce\"},{\"id\":\"8adb45d0-3f9f-492d-b1df-7471f07cfdd7\"},{\"id\":\"ab77e90b-a838-4b1c-8c37-133586dc5cac\"}],\"margin\":[5,10],\"align\":\"start\",\"label\":\"Error\"}},{\"type\":\"object\",\"name\":\"Slider\",\"id\":\"683eecf8-9c72-4bf5-8870-6ae28ff64cc4\",\"attributes\":{\"stylesheets\":[\"\\n:host(.pn-loading.pn-arc):before, .pn-loading.pn-arc:before {\\n  background-image: url(\\\"data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHN0eWxlPSJtYXJnaW46IGF1dG87IGJhY2tncm91bmQ6IG5vbmU7IGRpc3BsYXk6IGJsb2NrOyBzaGFwZS1yZW5kZXJpbmc6IGF1dG87IiB2aWV3Qm94PSIwIDAgMTAwIDEwMCIgcHJlc2VydmVBc3BlY3RSYXRpbz0ieE1pZFlNaWQiPiAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgZmlsbD0ibm9uZSIgc3Ryb2tlPSIjYzNjM2MzIiBzdHJva2Utd2lkdGg9IjEwIiByPSIzNSIgc3Ryb2tlLWRhc2hhcnJheT0iMTY0LjkzMzYxNDMxMzQ2NDE1IDU2Ljk3Nzg3MTQzNzgyMTM4Ij4gICAgPGFuaW1hdGVUcmFuc2Zvcm0gYXR0cmlidXRlTmFtZT0idHJhbnNmb3JtIiB0eXBlPSJyb3RhdGUiIHJlcGVhdENvdW50PSJpbmRlZmluaXRlIiBkdXI9IjFzIiB2YWx1ZXM9IjAgNTAgNTA7MzYwIDUwIDUwIiBrZXlUaW1lcz0iMDsxIj48L2FuaW1hdGVUcmFuc2Zvcm0+ICA8L2NpcmNsZT48L3N2Zz4=\\\");\\n  background-size: auto calc(min(50%, 400px));\\n}\",{\"id\":\"fcdd54a8-c48b-4b51-89e8-ada0b4aff044\"},{\"id\":\"1a828e8b-4990-4e28-b528-2ba8080973ce\"},{\"id\":\"8adb45d0-3f9f-492d-b1df-7471f07cfdd7\"},{\"id\":\"ab77e90b-a838-4b1c-8c37-133586dc5cac\"}],\"margin\":[5,10],\"align\":\"start\",\"title\":\"Dense factor\",\"start\":0.01,\"end\":0.9,\"value\":0.1,\"step\":0.1}}]}},{\"type\":\"object\",\"name\":\"Row\",\"id\":\"1c3249ea-b07b-4604-8102-b4c9b08d0ccc\",\"attributes\":{\"name\":\"Row00476\",\"stylesheets\":[\"\\n:host(.pn-loading.pn-arc):before, .pn-loading.pn-arc:before {\\n  background-image: url(\\\"data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHN0eWxlPSJtYXJnaW46IGF1dG87IGJhY2tncm91bmQ6IG5vbmU7IGRpc3BsYXk6IGJsb2NrOyBzaGFwZS1yZW5kZXJpbmc6IGF1dG87IiB2aWV3Qm94PSIwIDAgMTAwIDEwMCIgcHJlc2VydmVBc3BlY3RSYXRpbz0ieE1pZFlNaWQiPiAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgZmlsbD0ibm9uZSIgc3Ryb2tlPSIjYzNjM2MzIiBzdHJva2Utd2lkdGg9IjEwIiByPSIzNSIgc3Ryb2tlLWRhc2hhcnJheT0iMTY0LjkzMzYxNDMxMzQ2NDE1IDU2Ljk3Nzg3MTQzNzgyMTM4Ij4gICAgPGFuaW1hdGVUcmFuc2Zvcm0gYXR0cmlidXRlTmFtZT0idHJhbnNmb3JtIiB0eXBlPSJyb3RhdGUiIHJlcGVhdENvdW50PSJpbmRlZmluaXRlIiBkdXI9IjFzIiB2YWx1ZXM9IjAgNTAgNTA7MzYwIDUwIDUwIiBrZXlUaW1lcz0iMDsxIj48L2FuaW1hdGVUcmFuc2Zvcm0+ICA8L2NpcmNsZT48L3N2Zz4=\\\");\\n  background-size: auto calc(min(50%, 400px));\\n}\",{\"id\":\"fcdd54a8-c48b-4b51-89e8-ada0b4aff044\"},{\"id\":\"1a828e8b-4990-4e28-b528-2ba8080973ce\"},{\"id\":\"7a3bc9f4-c246-4436-b7af-1e5773db43f1\"},{\"id\":\"8adb45d0-3f9f-492d-b1df-7471f07cfdd7\"},{\"id\":\"ab77e90b-a838-4b1c-8c37-133586dc5cac\"}],\"margin\":0,\"align\":\"start\",\"children\":[{\"type\":\"object\",\"name\":\"Figure\",\"id\":\"10ce6270-a7b5-48cd-9c2f-06692051ea22\",\"attributes\":{\"width\":700,\"height\":450,\"x_range\":{\"type\":\"object\",\"name\":\"DataRange1d\",\"id\":\"24e573ee-e299-476d-878f-4504fb71a714\"},\"y_range\":{\"type\":\"object\",\"name\":\"DataRange1d\",\"id\":\"8fc8f5b3-e327-4a86-9924-eeb63fce1e1f\"},\"x_scale\":{\"type\":\"object\",\"name\":\"LinearScale\",\"id\":\"845220f9-4ba7-4629-ab0f-c30e2782c619\"},\"y_scale\":{\"type\":\"object\",\"name\":\"LinearScale\",\"id\":\"e9c408ec-0f36-44b9-83ce-c6824afdd087\"},\"title\":{\"type\":\"object\",\"name\":\"Title\",\"id\":\"d23351fd-8cc5-404a-99ff-d8e9eb24205d\",\"attributes\":{\"text\":\"XOR classification\",\"text_color\":\"#E0E0E0\",\"text_font\":\"Josefin Sans\",\"text_font_size\":\"2.1em\"}},\"outline_line_color\":\"#E0E0E0\",\"outline_line_alpha\":0.25,\"outline_line_width\":3,\"renderers\":[{\"type\":\"object\",\"name\":\"GlyphRenderer\",\"id\":\"74354860-f399-4588-a423-27ada52075fc\",\"attributes\":{\"data_source\":{\"type\":\"object\",\"name\":\"ColumnDataSource\",\"id\":\"e4d13395-8852-4160-ad63-de25e37c50c5\",\"attributes\":{\"selected\":{\"type\":\"object\",\"name\":\"Selection\",\"id\":\"7799efa4-26e7-4b35-8d76-a50d5041abbd\",\"attributes\":{\"indices\":[],\"line_indices\":[]}},\"selection_policy\":{\"type\":\"object\",\"name\":\"UnionRenderers\",\"id\":\"dac9b878-6717-4468-8324-ab68da2fd234\"},\"data\":{\"type\":\"map\",\"entries\":[[\"x\",{\"type\":\"ndarray\",\"array\":{\"type\":\"bytes\",\"data\":\"QcSHv/5Dbr/z6YW/yihYv7jDeb82vnO/AMF+v/51bb9fLUm/qmNrv60icr+3vIG/Y8Riv4oZZb/VrHW/Z/aYv2PCi7+6XYC/ks5Yv3YPjr+BPIG/15Rpvx/mf781S3K/daVhvwWxY79Et2W/kG59v5s5Vb97spC/FOlMv0R7mb+m832/k7eEv6zvgL9AZVu/qJOEv3MCgr+ANH2/7Dp0v7pzg7/0Jou/cy2Dv4PqZb90gWa//aN8vyL4ab/TX5K/WPF7Px/XjT/pYXs/pxg+P10JYD+gr5M/O0l/P1dVQT9wOVU/RJGGP7s9Wz+xnGY/3cuEPyyofD8LF1E/7MZnPxoMgD9VXmw/SRF6P8mHcj8YqT8/a9tsPx5+cz+b04A/kNNMPyerlT+m3YU/FP6APzVBjj+3wF8/GjRqP+RVjT9rpZQ/5u2QP38OaT8mKTc/ipyXP9NxZD8up5I/hJeRPyJ/iT/i2m4/ewx3P45Xkz8MRVI/nTKGP2AXeT8mVIU/\"},\"shape\":[96],\"dtype\":\"float32\",\"order\":\"little\"}],[\"y\",{\"type\":\"ndarray\",\"array\":{\"type\":\"bytes\",\"data\":\"aJ5Sv8OIV7+fDYO/isWUv8+8lL/wPYe/CHOGv542jr+tCnK//5CTv55Zj788G3e/JZ6TvwZya7/eP1a/LkuQvyWKj79HHGS/vw5wvwFEhL9H/Yu/kJiDv587R78TP4y/OvFaP/cZPT9EjH0/ocGJPwg9fD9lDXs//yw7P+XniT/fa3M/1I+RPyLoeD8Yp48/JPuGP2YVbz92l28/JtyGP4WUkD9hnpQ/g8CDP1puYj9b44M/aO+IPwYdXT+kfpU/sBBwv1gdgr9Gn2a/Gsebv3/OSr8Im5W/c85lv3w5br90TZW/TkyMv4sBgb/IQHe/88CNv9Cwgb+bfoK/xNOdv7TaaL8VEZi/xmk2vxoIkL/mR3K/gB2Nv/TzfL8Oaoi/D7uCP+12UD/edYg/rnyHP3dDcj+Ygl4/WwqRP5Nvjj/7vYs/0aFtP0K4ez8acHk/+WltPz/jjj+zQZ8/m7eOP3cHiD+6FnU/b32LPy72Tz/Gem0/yhl3P1rdij8lXXs/\"},\"shape\":[96],\"dtype\":\"float32\",\"order\":\"little\"}],[\"color\",[\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"firebrick\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\",\"midnightblue\"]],[\"label\",[\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 2\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\",\"class 1\"]]]}}},\"view\":{\"type\":\"object\",\"name\":\"CDSView\",\"id\":\"a388787a-3878-43ef-b750-b23b33f67a39\",\"attributes\":{\"filter\":{\"type\":\"object\",\"name\":\"AllIndices\",\"id\":\"60f16997-e40f-4dee-8526-87eb5bd80fe4\"}}},\"glyph\":{\"type\":\"object\",\"name\":\"Circle\",\"id\":\"b0faec2e-f5ee-475e-9aa7-b01ad6a2c73d\",\"attributes\":{\"x\":{\"type\":\"field\",\"field\":\"x\"},\"y\":{\"type\":\"field\",\"field\":\"y\"},\"size\":{\"type\":\"value\",\"value\":10},\"line_color\":{\"type\":\"value\",\"value\":\"white\"},\"line_alpha\":{\"type\":\"value\",\"value\":0.6},\"fill_color\":{\"type\":\"field\",\"field\":\"color\"},\"fill_alpha\":{\"type\":\"value\",\"value\":0.6},\"hatch_color\":{\"type\":\"field\",\"field\":\"color\"},\"hatch_alpha\":{\"type\":\"value\",\"value\":0.6}}},\"nonselection_glyph\":{\"type\":\"object\",\"name\":\"Circle\",\"id\":\"f6aa2f82-bf92-4ac8-a228-36c23357d657\",\"attributes\":{\"x\":{\"type\":\"field\",\"field\":\"x\"},\"y\":{\"type\":\"field\",\"field\":\"y\"},\"size\":{\"type\":\"value\",\"value\":10},\"line_color\":{\"type\":\"value\",\"value\":\"white\"},\"line_alpha\":{\"type\":\"value\",\"value\":0.1},\"fill_color\":{\"type\":\"field\",\"field\":\"color\"},\"fill_alpha\":{\"type\":\"value\",\"value\":0.1},\"hatch_color\":{\"type\":\"field\",\"field\":\"color\"},\"hatch_alpha\":{\"type\":\"value\",\"value\":0.1}}},\"muted_glyph\":{\"type\":\"object\",\"name\":\"Circle\",\"id\":\"9c85cede-c558-4f9c-a62f-0cba7df17208\",\"attributes\":{\"x\":{\"type\":\"field\",\"field\":\"x\"},\"y\":{\"type\":\"field\",\"field\":\"y\"},\"size\":{\"type\":\"value\",\"value\":10},\"line_color\":{\"type\":\"value\",\"value\":\"white\"},\"line_alpha\":{\"type\":\"value\",\"value\":0.2},\"fill_color\":{\"type\":\"field\",\"field\":\"color\"},\"fill_alpha\":{\"type\":\"value\",\"value\":0.2},\"hatch_color\":{\"type\":\"field\",\"field\":\"color\"},\"hatch_alpha\":{\"type\":\"value\",\"value\":0.2}}}}}],\"toolbar\":{\"type\":\"object\",\"name\":\"Toolbar\",\"id\":\"cbfe0e2d-f3f9-4ccf-a5aa-16881cce1334\",\"attributes\":{\"tools\":[{\"type\":\"object\",\"name\":\"PanTool\",\"id\":\"d4f1351b-fb71-4da0-897c-ce7bc26e4e83\"},{\"type\":\"object\",\"name\":\"WheelZoomTool\",\"id\":\"3781654e-58b9-4819-96af-68adda64147b\"},{\"type\":\"object\",\"name\":\"BoxZoomTool\",\"id\":\"297dc34c-14eb-468e-b1a2-c9f874451f37\",\"attributes\":{\"overlay\":{\"type\":\"object\",\"name\":\"BoxAnnotation\",\"id\":\"1dc64517-bffd-44cb-adc7-7ed566cc3a26\",\"attributes\":{\"syncable\":false,\"level\":\"overlay\",\"visible\":false,\"left_units\":\"canvas\",\"right_units\":\"canvas\",\"bottom_units\":\"canvas\",\"top_units\":\"canvas\",\"line_color\":\"black\",\"line_alpha\":1.0,\"line_width\":2,\"line_dash\":[4,4],\"fill_color\":\"lightgrey\",\"fill_alpha\":0.5}}}},{\"type\":\"object\",\"name\":\"SaveTool\",\"id\":\"065301b2-c15b-4412-96c5-612bd3842169\"},{\"type\":\"object\",\"name\":\"ResetTool\",\"id\":\"dd3285d9-0303-4ac8-8047-8c4d9fd2d4bf\"},{\"type\":\"object\",\"name\":\"HelpTool\",\"id\":\"91af627b-74a5-4292-9052-ce56ce54d4e0\"}]}},\"left\":[{\"type\":\"object\",\"name\":\"LinearAxis\",\"id\":\"07f7987b-3ef1-4d30-bcec-43d72773c504\",\"attributes\":{\"ticker\":{\"type\":\"object\",\"name\":\"BasicTicker\",\"id\":\"191d3bad-6d90-41c0-b7f2-c631094f9e7e\",\"attributes\":{\"mantissas\":[1,2,5]}},\"formatter\":{\"type\":\"object\",\"name\":\"BasicTickFormatter\",\"id\":\"6d898d9e-0aaf-40a7-89cf-c335728d1954\"},\"axis_label\":\"$$\\\\color{white} \\\\mathbf{y}$$\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"major_label_policy\":{\"type\":\"object\",\"name\":\"AllLabels\",\"id\":\"193cd836-26a6-4c26-84e4-0239bd39f194\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"axis_line_color\":\"#E0E0E0\",\"axis_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"major_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4}}],\"right\":[{\"type\":\"object\",\"name\":\"Legend\",\"id\":\"407e594d-4b91-4081-82e0-a4b079961be1\",\"attributes\":{\"border_line_color\":\"grey\",\"border_line_alpha\":0.8,\"border_line_width\":3,\"background_fill_color\":\"darkgrey\",\"background_fill_alpha\":0.2,\"label_text_color\":\"white\",\"items\":[{\"type\":\"object\",\"name\":\"LegendItem\",\"id\":\"df990613-18a4-4b70-acf8-ae106f6e5143\",\"attributes\":{\"label\":{\"type\":\"field\",\"field\":\"label\"},\"renderers\":[{\"id\":\"74354860-f399-4588-a423-27ada52075fc\"}]}}]}}],\"below\":[{\"type\":\"object\",\"name\":\"LinearAxis\",\"id\":\"1b69dc99-398b-411e-a68c-fdd3c66f4c1d\",\"attributes\":{\"ticker\":{\"type\":\"object\",\"name\":\"BasicTicker\",\"id\":\"84eaa61f-f44a-4ea2-a2d5-130b75f0e516\",\"attributes\":{\"mantissas\":[1,2,5]}},\"formatter\":{\"type\":\"object\",\"name\":\"BasicTickFormatter\",\"id\":\"fd84a138-5cd0-4eb9-a054-fd531836277d\"},\"axis_label\":\"$$\\\\color{white} x$$\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"major_label_policy\":{\"type\":\"object\",\"name\":\"AllLabels\",\"id\":\"6f06d06d-2fd7-4cdf-8c10-9d106fe6d71f\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"axis_line_color\":\"#E0E0E0\",\"axis_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"major_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4}}],\"center\":[{\"type\":\"object\",\"name\":\"Grid\",\"id\":\"2f18ca04-f6ad-4038-931a-9d990fcc0897\",\"attributes\":{\"axis\":{\"id\":\"1b69dc99-398b-411e-a68c-fdd3c66f4c1d\"},\"grid_line_color\":\"#E0E0E0\",\"grid_line_alpha\":0.15}},{\"type\":\"object\",\"name\":\"Grid\",\"id\":\"5e7aee2a-ac0c-448f-b7f0-675a504b8aee\",\"attributes\":{\"dimension\":1,\"axis\":{\"id\":\"07f7987b-3ef1-4d30-bcec-43d72773c504\"},\"grid_line_color\":\"#E0E0E0\",\"grid_line_alpha\":0.15}},{\"id\":\"407e594d-4b91-4081-82e0-a4b079961be1\"}],\"background_fill_color\":\"#0f0f0f\",\"background_fill_alpha\":0.9,\"border_fill_color\":\"#111111\",\"border_fill_alpha\":0.8}}]}}]}},{\"type\":\"object\",\"name\":\"panel.models.ace.AcePlot\",\"id\":\"ebd4a220-ba62-4055-8e4f-066258876c80\",\"attributes\":{\"js_property_callbacks\":{\"type\":\"map\",\"entries\":[[\"change:readonly\",[{\"type\":\"object\",\"name\":\"CustomJS\",\"id\":\"0efda64b-f935-4f27-8353-983366fa6e85\",\"attributes\":{\"tags\":[[12536767680,[null,\"readonly\"],[null,\"disabled\"]]],\"args\":{\"type\":\"map\",\"entries\":[[\"bidirectional\",true],[\"properties\",{\"type\":\"map\",\"entries\":[[\"readonly\",\"disabled\"]]}],[\"source\",{\"id\":\"ebd4a220-ba62-4055-8e4f-066258876c80\"}],[\"target\",{\"id\":\"ebd4a220-ba62-4055-8e4f-066258876c80\"}]]},\"code\":\"\\n    var value = source['readonly'];\\n    value = value;\\n    value = value;\\n    try {\\n      var property = target.properties['disabled'];\\n      if (property !== undefined) { property.validate(value); }\\n    } catch(err) {\\n      console.log('WARNING: Could not set disabled on target, raised error: ' + err);\\n      return;\\n    }\\n    try {\\n      target['disabled'] = value;\\n    } catch(err) {\\n      console.log(err)\\n    }\\n    \"}}]],[\"change:disabled\",[{\"type\":\"object\",\"name\":\"CustomJS\",\"id\":\"eb9e0b7c-21c9-41aa-8b67-9a380f5e3cdc\",\"attributes\":{\"tags\":[[12536767680,[null,\"readonly\"],[null,\"disabled\"]]],\"args\":{\"type\":\"map\",\"entries\":[[\"bidirectional\",true],[\"properties\",{\"type\":\"map\",\"entries\":[[\"readonly\",\"disabled\"]]}],[\"source\",{\"id\":\"ebd4a220-ba62-4055-8e4f-066258876c80\"}],[\"target\",{\"id\":\"ebd4a220-ba62-4055-8e4f-066258876c80\"}]]},\"code\":\"\\n    var value = source['disabled'];\\n    value = value;\\n    value = value;\\n    try {\\n      var property = target.properties['readonly'];\\n      if (property !== undefined) { property.validate(value); }\\n    } catch(err) {\\n      console.log('WARNING: Could not set readonly on target, raised error: ' + err);\\n      return;\\n    }\\n    try {\\n      target['readonly'] = value;\\n    } catch(err) {\\n      console.log(err)\\n    }\\n    \"}}]]]},\"stylesheets\":[\"\\n:host(.pn-loading.pn-arc):before, .pn-loading.pn-arc:before {\\n  background-image: url(\\\"data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHN0eWxlPSJtYXJnaW46IGF1dG87IGJhY2tncm91bmQ6IG5vbmU7IGRpc3BsYXk6IGJsb2NrOyBzaGFwZS1yZW5kZXJpbmc6IGF1dG87IiB2aWV3Qm94PSIwIDAgMTAwIDEwMCIgcHJlc2VydmVBc3BlY3RSYXRpbz0ieE1pZFlNaWQiPiAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgZmlsbD0ibm9uZSIgc3Ryb2tlPSIjYzNjM2MzIiBzdHJva2Utd2lkdGg9IjEwIiByPSIzNSIgc3Ryb2tlLWRhc2hhcnJheT0iMTY0LjkzMzYxNDMxMzQ2NDE1IDU2Ljk3Nzg3MTQzNzgyMTM4Ij4gICAgPGFuaW1hdGVUcmFuc2Zvcm0gYXR0cmlidXRlTmFtZT0idHJhbnNmb3JtIiB0eXBlPSJyb3RhdGUiIHJlcGVhdENvdW50PSJpbmRlZmluaXRlIiBkdXI9IjFzIiB2YWx1ZXM9IjAgNTAgNTA7MzYwIDUwIDUwIiBrZXlUaW1lcz0iMDsxIj48L2FuaW1hdGVUcmFuc2Zvcm0+ICA8L2NpcmNsZT48L3N2Zz4=\\\");\\n  background-size: auto calc(min(50%, 400px));\\n}\",{\"id\":\"fcdd54a8-c48b-4b51-89e8-ada0b4aff044\"},{\"id\":\"1a828e8b-4990-4e28-b528-2ba8080973ce\"},{\"id\":\"8adb45d0-3f9f-492d-b1df-7471f07cfdd7\"},{\"id\":\"ab77e90b-a838-4b1c-8c37-133586dc5cac\"}],\"height\":60,\"min_height\":60,\"margin\":[5,10],\"align\":\"start\",\"code\":\"    def generate_data(self):\\n        self.x, self.y = xor_separation(self.number_observations, self.dense_factor)\\n        super().generate_data()\",\"theme\":\"chaos\",\"filename\":\"\",\"language\":\"python\"}}]}},{\"type\":\"object\",\"name\":\"panel.models.comm_manager.CommManager\",\"id\":\"763c3fd3-2c9d-4ee4-b5da-e75cb9427e1c\",\"attributes\":{\"plot_id\":\"b06edcc4-a1f7-41b4-8757-223a14ca4d44\",\"comm_id\":\"25d90df602df4719b0a094a836d778f0\",\"client_comm_id\":\"dda114156b6a4215b0cd92bfd72ba7c5\"}}],\"defs\":[{\"type\":\"model\",\"name\":\"ReactiveHTML1\"},{\"type\":\"model\",\"name\":\"FlexBox1\",\"properties\":[{\"name\":\"align_content\",\"kind\":\"Any\",\"default\":\"flex-start\"},{\"name\":\"align_items\",\"kind\":\"Any\",\"default\":\"flex-start\"},{\"name\":\"flex_direction\",\"kind\":\"Any\",\"default\":\"row\"},{\"name\":\"flex_wrap\",\"kind\":\"Any\",\"default\":\"wrap\"},{\"name\":\"justify_content\",\"kind\":\"Any\",\"default\":\"flex-start\"}]},{\"type\":\"model\",\"name\":\"FloatPanel1\",\"properties\":[{\"name\":\"config\",\"kind\":\"Any\",\"default\":{\"type\":\"map\"}},{\"name\":\"contained\",\"kind\":\"Any\",\"default\":true},{\"name\":\"position\",\"kind\":\"Any\",\"default\":\"right-top\"},{\"name\":\"offsetx\",\"kind\":\"Any\",\"default\":null},{\"name\":\"offsety\",\"kind\":\"Any\",\"default\":null},{\"name\":\"theme\",\"kind\":\"Any\",\"default\":\"primary\"},{\"name\":\"status\",\"kind\":\"Any\",\"default\":\"normalized\"}]},{\"type\":\"model\",\"name\":\"GridStack1\",\"properties\":[{\"name\":\"mode\",\"kind\":\"Any\",\"default\":\"warn\"},{\"name\":\"ncols\",\"kind\":\"Any\",\"default\":null},{\"name\":\"nrows\",\"kind\":\"Any\",\"default\":null},{\"name\":\"allow_resize\",\"kind\":\"Any\",\"default\":true},{\"name\":\"allow_drag\",\"kind\":\"Any\",\"default\":true},{\"name\":\"state\",\"kind\":\"Any\",\"default\":[]}]},{\"type\":\"model\",\"name\":\"drag1\",\"properties\":[{\"name\":\"slider_width\",\"kind\":\"Any\",\"default\":5},{\"name\":\"slider_color\",\"kind\":\"Any\",\"default\":\"black\"},{\"name\":\"value\",\"kind\":\"Any\",\"default\":50}]},{\"type\":\"model\",\"name\":\"click1\",\"properties\":[{\"name\":\"terminal_output\",\"kind\":\"Any\",\"default\":\"\"},{\"name\":\"debug_name\",\"kind\":\"Any\",\"default\":\"\"},{\"name\":\"clears\",\"kind\":\"Any\",\"default\":0}]},{\"type\":\"model\",\"name\":\"FastWrapper1\",\"properties\":[{\"name\":\"object\",\"kind\":\"Any\",\"default\":null},{\"name\":\"style\",\"kind\":\"Any\",\"default\":null}]},{\"type\":\"model\",\"name\":\"NotificationAreaBase1\",\"properties\":[{\"name\":\"position\",\"kind\":\"Any\",\"default\":\"bottom-right\"},{\"name\":\"_clear\",\"kind\":\"Any\",\"default\":0}]},{\"type\":\"model\",\"name\":\"NotificationArea1\",\"properties\":[{\"name\":\"notifications\",\"kind\":\"Any\",\"default\":[]},{\"name\":\"position\",\"kind\":\"Any\",\"default\":\"bottom-right\"},{\"name\":\"_clear\",\"kind\":\"Any\",\"default\":0},{\"name\":\"types\",\"kind\":\"Any\",\"default\":[{\"type\":\"map\",\"entries\":[[\"type\",\"warning\"],[\"background\",\"#ffc107\"],[\"icon\",{\"type\":\"map\",\"entries\":[[\"className\",\"fas fa-exclamation-triangle\"],[\"tagName\",\"i\"],[\"color\",\"white\"]]}]]},{\"type\":\"map\",\"entries\":[[\"type\",\"info\"],[\"background\",\"#007bff\"],[\"icon\",{\"type\":\"map\",\"entries\":[[\"className\",\"fas fa-info-circle\"],[\"tagName\",\"i\"],[\"color\",\"white\"]]}]]}]}]},{\"type\":\"model\",\"name\":\"Notification\",\"properties\":[{\"name\":\"background\",\"kind\":\"Any\",\"default\":null},{\"name\":\"duration\",\"kind\":\"Any\",\"default\":3000},{\"name\":\"icon\",\"kind\":\"Any\",\"default\":null},{\"name\":\"message\",\"kind\":\"Any\",\"default\":\"\"},{\"name\":\"notification_type\",\"kind\":\"Any\",\"default\":null},{\"name\":\"_destroyed\",\"kind\":\"Any\",\"default\":false}]},{\"type\":\"model\",\"name\":\"TemplateActions1\",\"properties\":[{\"name\":\"open_modal\",\"kind\":\"Any\",\"default\":0},{\"name\":\"close_modal\",\"kind\":\"Any\",\"default\":0}]},{\"type\":\"model\",\"name\":\"BootstrapTemplateActions1\",\"properties\":[{\"name\":\"open_modal\",\"kind\":\"Any\",\"default\":0},{\"name\":\"close_modal\",\"kind\":\"Any\",\"default\":0}]},{\"type\":\"model\",\"name\":\"MaterialTemplateActions1\",\"properties\":[{\"name\":\"open_modal\",\"kind\":\"Any\",\"default\":0},{\"name\":\"close_modal\",\"kind\":\"Any\",\"default\":0}]}]}};\n",
       "  var render_items = [{\"docid\":\"4f6e6553-a494-4971-933b-b068f8dedd6a\",\"roots\":{\"b06edcc4-a1f7-41b4-8757-223a14ca4d44\":\"b2784f13-bd52-421d-a26d-d047186809d8\"},\"root_ids\":[\"b06edcc4-a1f7-41b4-8757-223a14ca4d44\"]}];\n",
       "  var docs = Object.values(docs_json)\n",
       "  if (!docs) {\n",
       "    return\n",
       "  }\n",
       "  const py_version = docs[0].version.replace('rc', '-rc.')\n",
       "  const is_dev = py_version.indexOf(\"+\") !== -1 || py_version.indexOf(\"-\") !== -1\n",
       "  function embed_document(root) {\n",
       "    var Bokeh = get_bokeh(root)\n",
       "    Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "    for (const render_item of render_items) {\n",
       "      for (const root_id of render_item.root_ids) {\n",
       "\tconst id_el = document.getElementById(root_id)\n",
       "\tif (id_el.children.length && (id_el.children[0].className === 'bk-root')) {\n",
       "\t  const root_el = id_el.children[0]\n",
       "\t  root_el.id = root_el.id + '-rendered'\n",
       "\t}\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "  function get_bokeh(root) {\n",
       "    if (root.Bokeh === undefined) {\n",
       "      return null\n",
       "    } else if (root.Bokeh.version !== py_version && !is_dev) {\n",
       "      if (root.Bokeh.versions === undefined || !root.Bokeh.versions.has(py_version)) {\n",
       "\treturn null\n",
       "      }\n",
       "      return root.Bokeh.versions.get(py_version);\n",
       "    } else if (root.Bokeh.version === py_version) {\n",
       "      return root.Bokeh\n",
       "    }\n",
       "    return null\n",
       "  }\n",
       "  function is_loaded(root) {\n",
       "    var Bokeh = get_bokeh(root)\n",
       "    return (Bokeh != null && Bokeh.Panel !== undefined)\n",
       "  }\n",
       "  if (is_loaded(root)) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    var attempts = 0;\n",
       "    var timer = setInterval(function(root) {\n",
       "      if (is_loaded(root)) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else if (document.readyState == \"complete\") {\n",
       "        attempts++;\n",
       "        if (attempts > 200) {\n",
       "          clearInterval(timer);\n",
       "\t  var Bokeh = get_bokeh(root)\n",
       "\t  if (Bokeh == null || Bokeh.Panel == null) {\n",
       "            console.warn(\"Panel: ERROR: Unable to run Panel code because Bokeh or Panel library is missing\");\n",
       "\t  } else {\n",
       "\t    console.warn(\"Panel: WARNING: Attempting to render but not all required libraries could be resolved.\")\n",
       "\t    embed_document(root)\n",
       "\t  }\n",
       "        }\n",
       "      }\n",
       "    }, 25, root)\n",
       "  }\n",
       "})(window);</script>"
      ],
      "text/plain": [
       "Column\n",
       "    [0] Row\n",
       "        [0] Param(ClassificationXOR, name='Problem parameters')\n",
       "        [1] ParamMethod(method, _pane=Bokeh)\n",
       "    [1] CodeEditor(height=60, language='python', theme='chaos', value='    def generate_data(sel...)"
      ]
     },
     "execution_count": 55,
     "metadata": {
      "application/vnd.holoviews_exec.v0+json": {
       "id": "b06edcc4-a1f7-41b4-8757-223a14ca4d44"
      }
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from cml.tasks import ClassificationLinear, ClassificationXOR\n",
    "explorer = ClassificationXOR()\n",
    "explorer.render()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Solving the problem with the `scikit-learn` library\n",
    "\n",
    "As a side note, and as discussed in the previous course, there is obviously a lot of libraries that can perform the training of such networks for you directly, without understanding what is exactly going on behind the scene. A quite extensive library is `scikit-learn`, which contains already coded models and learning procedure, that will allow us to _learn_ the parameters of this unknown function.\n",
    "\n",
    "Here we will use a `MLPClassifier` model to perfom classification on standardized features that we rescale through the `LinearRegression` and that this polynomial should be of degree 2. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.neural_network import MLPClassifier\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.pipeline import make_pipeline\n",
    "# Create the MLP classifier\n",
    "classifier = MLPClassifier(alpha=1, max_iter=1000)\n",
    "# Standardize the input features\n",
    "clf = make_pipeline(StandardScaler(), classifier)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now based on this model definition, training it can be performed with a single line of code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "from cml.data import xor_separation\n",
    "# Generate a XOR dataset\n",
    "x_inputs, y_classes = xor_separation(500, 0.1)\n",
    "# Train the MLP model\n",
    "clf.fit(x_inputs, y_classes);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Multi-layer implementation\n",
    "\n",
    "In this exercise, we will implement the XOR classification resolution using NumPy and JAX. This classification problem cannot be solved with a simple linear classifier. Hence, we will need to use at least a 2-layer network. The minimal set of variables that will be used by your code should be the following.\n",
    "\n",
    "```Python\n",
    "x_inputs          # 2 x n matrix of random points\n",
    "y_class           # classes of the patterns \n",
    "n_hidden          # Number of hidden units\n",
    "eta               # Learning rate parameter\n",
    "momentum          # Momentum parameter (bonus)\n",
    "weights1          # 1st layer weights\n",
    "weights2          # 2nd layer weights\n",
    "mse_limit         # Sum-squared error limit\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Number of hidden units\n",
    "n_hidden = 2\n",
    "# Learning rate parameter\n",
    "eta = 0.005\n",
    "# Momentum parameter\n",
    "momentum = 0.1\n",
    "# Sum-squared error limit\n",
    "mse_limit = 0.002"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Exercise 1 - 2-layers XOR classification\n",
    "\n",
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #013220; border-color: #03C03C\">\n",
    "\n",
    "> To complete this exercise, you will need to have a basic understanding of NumPy and JAX. You can use the resources provided in the previous exercises to learn more about these libraries.\n",
    "\n",
    "We help you out by first defining the XOR classification problem to solve (and outlining some basic properties)\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating the object\n"
     ]
    }
   ],
   "source": [
    "from cml.data import xor_separation\n",
    "# Definition of our data properties\n",
    "n_observations = 500\n",
    "noise_factor = 0.1\n",
    "# Properties of our problem\n",
    "n_input = 2\n",
    "n_output = 1\n",
    "# Generate a XOR dataset\n",
    "x_inputs, y_classes = xor_separation(n_observations, noise_factor)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also show some potential implementation of the **initialization scheme**. Note that you can later try to replace this naive implementation by some more refined functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 1st layer weights\n",
    "weights1 = (np.random.randn(n_hidden, n_input) * 0.01)\n",
    "# 2nd layer weights\n",
    "weights2 = (np.random.randn(n_output, n_hidden) * 0.01)\n",
    "# 1st layer biases\n",
    "bias1 = (np.random.randn(n_hidden, 1) * 0.01)\n",
    "# 2nd layer biases\n",
    "bias2 = (np.random.randn(n_output, 1) * 0.01)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***\n",
    "\n",
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #013220; border-color: #03C03C\">\n",
    "\n",
    "> ### Question 1.1 - Implement 2-layer XOR classification with NumPy\n",
    "\n",
    "> 1. Update the forward propagation and error computation\n",
    "> 2. Update the back-propagation part to learn the weights of both layers.\n",
    "> 3. Run the learning and check your network results\n",
    "  \n",
    "*For optional questions, please look at the end of this exercise's code boxes for more information*\n",
    "\n",
    "</div>\n",
    "\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "######################\n",
    "# YOUR CODE GOES HERE\n",
    "######################\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***\n",
    "\n",
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #013220; border-color: #03C03C\">\n",
    "\n",
    "> ### Question 1.2 - Implement XOR classification with JAX\n",
    "\n",
    "> 1. Perform the same implementation as previously with JAX\n",
    "> 2. Implement gradient descent to optimize the weights and bias.\n",
    "\n",
    "</div>\n",
    "\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "######################\n",
    "# YOUR CODE GOES HERE\n",
    "######################\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***\n",
    "\n",
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #013220; border-color: #03C03C\">\n",
    "\n",
    "> ### Question 1.3 - Understanding properties of our implementation\n",
    "\n",
    "> 4. Perform multiple re-runs of the learning procedure (re-launching with different initializations)\n",
    ">     1. What observations can you make on the learning process?\n",
    ">     2. What happens if you initialize all weights to zeros?\n",
    ">     3. Change your initialization and regularization scheme\n",
    "\n",
    "</div>\n",
    "\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "######################\n",
    "# YOUR CODE GOES HERE\n",
    "######################\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***\n",
    "\n",
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #192841; border-color: #779ecb\">\n",
    "\n",
    "> ### Going further\n",
    "\n",
    "> The following set of questions are optionnal addendum to your previous code that allow to understand more in-depth properties about _regularization_ both for neural networks and optimization in general.\n",
    "\n",
    "> 1. (Optional) Implement the *weight decay* constraint in your network.\n",
    "> 2. (Optional) Add the *momentum* to the learning procedure.\n",
    "\n",
    "> *Weight decay* constraint\n",
    "> As nothing constrains the weights in the network, we can note that usually all weights vector given a multiplicative factor might be equivalent, which can stall the learning (and lead to exploding weights). The *weight decay* allows to regularize the learning by penalizing weights with a too wide amplitude. The idea is to add this constraint as a term to the final loss (which leads to an indirect \"pressure\" on the learning process. Therefore, the final loss will be defined as\n",
    "> $$\n",
    "\\begin{equation}\n",
    "\\mathcal{L}_{final}=\\mathcal{L_D} + \\lambda \\sum_{l} \\sum_{i} \\sum_{j} \\left( W_{ij}^{l} \\right)^{2}\n",
    "\\end{equation}\n",
    "$$\n",
    "> where the parameter $\\lambda$ controls the relative importance of the two terms.\n",
    "\n",
    "> *Momentum* in learning\n",
    "> Usually, in complex problems, the gradient can be very noisy and, therefore, the learning might oscillate widely. In order to reduce this problem, we can *smooth* the different gradient updates by retaining the values of the gradient at each iteration and then performing an update based on the latest gradient $\\delta_{i}^{t}$ and the gradient at the previous iteration $\\delta_{i}^{t-1}$. Therefore, a gradient update is applied as\n",
    "> $$\n",
    "\\begin{equation}\n",
    "\\delta_{final}^{t} = \\delta_{i}^{t} + m.\\delta_{i}^{t-1}\n",
    "\\end{equation}\n",
    "$$\n",
    "> with $m$ the momentum parameter, which control the amount of gradient smoothing.\n",
    "\n",
    "</div>\n",
    "\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "######################\n",
    "# YOUR CODE GOES HERE\n",
    "######################\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (Optional) 3-layer audio classification\n",
    "\n",
    "Once again, note that the following paragraph and subsequent questions are _optional_ but should be a quite simple extension of our previous work on 2-layer networks. If you struggle with the data import mechanisms, you can look down at the PyTorch section (with a **mandatory exercise**) that will provide base code for this aspect. \n",
    "\n",
    "Finally, we will attack a complete audio classification problem and try to perform neural network learning on a set of audio files. The data structure will be the same as the one used for parts 1 and 2. As discussed during the courses, even though a 2-layer neural network can provide non-linear boundaries, it can not perform \"holes\" inside those regions. In order to obtain an improved classification, we will now rely on a 3-layer neural network. The modification to the code of section 3.2 should be minimal, as the back-propagation will be similar for the new layer as one of the two others. We do not develop the math here as it is simply a re-application of the previous rules with an additional layer (which derivatives you should have generalized in the previous exercise).  \n",
    "\n",
    "However, up until now, we only performed *binary classification* problems, but this time we need to obtain a decision rule for multiple classes. Therefore, we cannot rely on simply computing the distance between desired patterns and the obtained binary value. The idea here is to rely on the *softmax regression*, by considering classes as a vector of probabilities. The desired answers will therefore be considered as a set of *probabilities*, where the desired class is $1$ and the others are $0$ (called *one-hot* representation). Then, the cost function will rely on the softmax formulation\n",
    "\n",
    "$$\n",
    "\\begin{equation}\n",
    "\\mathcal{L_D}(\\theta) = - \\frac{1}{m} \\left[ \\sum_{i=1}^{m} \\sum_{j=1}^{k} 1\\left\\{y^{(i)} = j\\right\\} \\log \\frac{e^{\\theta_{j}^{T} x^{(i)}}}{\\sum_{l=1}^{k} e^{ \\theta_{l}^{T} x^{(i)} }}  \\right]\n",
    "\\end{equation}\n",
    "$$\n",
    "\n",
    "Therefore, we compute the output of the softmax by taking \n",
    "\n",
    "$$\n",
    "\\begin{equation}\n",
    "p(y^{(i)} = j | x^{(i)}; \\theta) = \\frac{e^{\\theta_{j}^{T} x^{(i)}}}{\\sum_{l=1}^{k} e^{ \\theta_{l}^{T} x^{(i)}} }\n",
    "\\end{equation}\n",
    "$$\n",
    "\n",
    "By taking derivatives, we can show that the gradient of the softmax layer is\n",
    "\n",
    "$$\n",
    "\\begin{equation}\n",
    "\\nabla_{\\theta_{j}} \\mathcal{L_D}(\\theta) = - \\frac{1}{m} \\sum_{i=1}^{m}{ \\left[ x^{(i)} \\left( 1\\{ y^{(i)} = j\\}  - p(y^{(i)} = j \\mid x^{(i)}, \\theta) \\right) \\right]}\n",
    "\\end{equation}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sweet activation functions\n",
    "\n",
    "As discussed in the course, the interest of stacking layers is that there is an _activation function_, which allows non-linear interactions between the dimensions (and avoids to only compute a single huge affine transform). Although the `sigmoid` function has been historically the most used, there has been some large developments since. Notably the `ReLU` (Rectified Linear Unit) is one of the major difference in modern networks (we will see more about that in a later course)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Function for computing the Sigmoid activation\n",
    "def sigmoid(x):\n",
    "    return 1 / (1 + np.exp(-x))\n",
    "# Derivative\n",
    "def dsigmoid(a):\n",
    "    return a * (1.0 - a)\n",
    "# Function for computing the ReLU activation\n",
    "def relu(x):\n",
    "    return np.maximum(0, x)\n",
    "# Derivative\n",
    "def drelu(x):\n",
    "    if (x < 0):\n",
    "        return 0\n",
    "    return 1\n",
    "# Function for computing the Tanh activation\n",
    "def tanh(x):\n",
    "    return np.tanh(x);\n",
    "# Derivative\n",
    "def dtanh(x): \n",
    "    return np.cosh(x) ^ -2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, we plot some simple examples of what these activation functions look like. You can try to rely on these functions in your previous training code and witness the differences in training.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {},
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.holoviews_exec.v0+json": "",
      "text/html": [
       "<div id='4210'>\n",
       "  <div class=\"bk-root\" id=\"9d0d059d-3d72-4cc5-aadf-97f393bc9b0b\" data-root-id=\"4210\"></div>\n",
       "</div>\n",
       "<script type=\"application/javascript\">(function(root) {\n",
       "  function embed_document(root) {\n",
       "    var docs_json = {\"c6b166bf-d91b-4d39-bc41-f9de9c5a081a\":{\"defs\":[{\"extends\":null,\"module\":null,\"name\":\"ReactiveHTML1\",\"overrides\":[],\"properties\":[]},{\"extends\":null,\"module\":null,\"name\":\"FlexBox1\",\"overrides\":[],\"properties\":[{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_content\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_items\"},{\"default\":\"row\",\"kind\":null,\"name\":\"flex_direction\"},{\"default\":\"wrap\",\"kind\":null,\"name\":\"flex_wrap\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"justify_content\"}]},{\"extends\":null,\"module\":null,\"name\":\"GridStack1\",\"overrides\":[],\"properties\":[{\"default\":\"warn\",\"kind\":null,\"name\":\"mode\"},{\"default\":null,\"kind\":null,\"name\":\"ncols\"},{\"default\":null,\"kind\":null,\"name\":\"nrows\"},{\"default\":true,\"kind\":null,\"name\":\"allow_resize\"},{\"default\":true,\"kind\":null,\"name\":\"allow_drag\"},{\"default\":[],\"kind\":null,\"name\":\"state\"}]},{\"extends\":null,\"module\":null,\"name\":\"click1\",\"overrides\":[],\"properties\":[{\"default\":\"\",\"kind\":null,\"name\":\"terminal_output\"},{\"default\":\"\",\"kind\":null,\"name\":\"debug_name\"},{\"default\":0,\"kind\":null,\"name\":\"clears\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationAreaBase1\",\"overrides\":[],\"properties\":[{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationArea1\",\"overrides\":[],\"properties\":[{\"default\":[],\"kind\":null,\"name\":\"notifications\"},{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"},{\"default\":[{\"background\":\"#ffc107\",\"icon\":{\"className\":\"fas fa-exclamation-triangle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"warning\"},{\"background\":\"#007bff\",\"icon\":{\"className\":\"fas fa-info-circle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"info\"}],\"kind\":null,\"name\":\"types\"}]},{\"extends\":null,\"module\":null,\"name\":\"Notification\",\"overrides\":[],\"properties\":[{\"default\":null,\"kind\":null,\"name\":\"background\"},{\"default\":3000,\"kind\":null,\"name\":\"duration\"},{\"default\":null,\"kind\":null,\"name\":\"icon\"},{\"default\":\"\",\"kind\":null,\"name\":\"message\"},{\"default\":null,\"kind\":null,\"name\":\"notification_type\"},{\"default\":false,\"kind\":null,\"name\":\"_destroyed\"}]},{\"extends\":null,\"module\":null,\"name\":\"TemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]},{\"extends\":null,\"module\":null,\"name\":\"MaterialTemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]}],\"roots\":{\"references\":[{\"attributes\":{},\"id\":\"4153\",\"type\":\"HelpTool\"},{\"attributes\":{},\"id\":\"4095\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"4145\",\"type\":\"BasicTicker\"},{\"attributes\":{\"children\":[{\"id\":\"4211\"},{\"id\":\"4209\"},{\"id\":\"4272\"}],\"margin\":[0,0,0,0],\"name\":\"Row00190\"},\"id\":\"4210\",\"type\":\"Row\"},{\"attributes\":{},\"id\":\"4228\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"4093\",\"type\":\"DataRange1d\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"AAAAAAAAFMBDYt7OkJgTwIbEvJ0hMRPAyiabbLLJEsANiXk7Q2ISwFDrVwrU+hHAk0022WSTEcDXrxSo9SsRwBoS83aGxBDAXXTRRRddEMBArV8pUOsPwMdxHMdxHA/ATjbZZJNNDsDU+pUCtX4NwFq/UqDWrwzA4YMPPvjgC8BoSMzbGRILwO4MiXk7QwrAdNFFF110CcD7lQK1fqUIwIFav1Kg1gfACB988MEHB8CO4ziO4zgGwBWo9SsFagXAm2yyySabBMAiMW9nSMwDwKj1KwVq/QLAL7rooosuAsC1fqVArV8BwDxDYt7OkADAhA8++OCD/7+SmLczJOb9v54hMW9nSPy/qqqqqqqq+r+4MyTm7Qz5v8S8nSExb/e/0kUXXXTR9b/ezpCYtzP0v+xXCtT6lfK/+OCDDz748L8I1PqVArXuvyjm7QyJeeu/QPjggw8+6L9YCtT6lQLlv3Acx3Ecx+G/IF100UUX3b9QgVq/UqDWv4ClQK1fKdC/YJNNNtlkw78Ab2dIzNupvwBvZ0jM26k/YJNNNtlkwz+ApUCtXynQP1CBWr9SoNY/EF100UUX3T9wHMdxHMfhP1gK1PqVAuU/QPjggw8+6D8g5u0MiXnrPwjU+pUCte4/+OCDDz748D/sVwrU+pXyP9zOkJi3M/Q/0EUXXXTR9T/EvJ0hMW/3P7gzJObtDPk/rKqqqqqq+j+cITFvZ0j8P5CYtzMk5v0/hA8++OCD/z88Q2LezpAAQLR+pUCtXwFALrrooosuAkCo9SsFav0CQCIxb2dIzANAmmyyySabBEAUqPUrBWoFQI7jOI7jOAZACB988MEHB0CAWr9SoNYHQPyVArV+pQhAdNFFF110CUDsDIl5O0MKQGhIzNsZEgtA4IMPPvjgC0Bcv1Kg1q8MQNT6lQK1fg1ATDbZZJNNDkDIcRzHcRwPQECtXylQ6w9AXHTRRRddEEAaEvN2hsQQQNavFKj1KxFAlE022WSTEUBQ61cK1PoRQAyJeTtDYhJAyiabbLLJEkCGxLydITETQERi3s6QmBNAAAAAAAAAFEA=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]},\"y\":{\"__ndarray__\":\"jo9jffZpez8zICEkXk5+P3YuqyAfwIA/p+JVusaDgj8N8g/shXaEP/byi48qnYY/0BjB9/n8iD/WZNY4u5uLP6vI/AXCf44/EJSHDf3XkD/6lO+Q+ZmSP06i3332iZQ/VzdyJ3Kslj8nJDSWTwaZPwVPZcvcnJs/Nz7K0Nh1nj/SLj+0vMugP2FCABM4hKI/cEoQV/dnpD/TN0jn1HqmPztu44ftwKg/IjxyKqA+qz9GueiGjPitP6x4HxzIebA/Bir+B2Easj+U/wqnteCzPwKDQzN/z7U/d+J5foXptz/t5tFglzG6PykCu3GBqrw/UGnG7AJXvz+DDaxX4BzBP1E7QSabqsI/LnfnpNJVxD9YIjhZgR/GPxsQCKdxCMg/578cdjMRyj/8P9qtETrMP6bqzeUHg84/wifXXdx10D+UG9WmsrnRP5JJRzBzDNM/B5ki3VNt1D+dUU4YU9vVP7IneGw4Vdc/QWJr05bZ2D+hWanP0GbaPzYfqEoe+9s/yqt2F5SU3T+jHszeLDHfP6/wmZBpZ+A/G6pE9LU14T9l8KvacALiPy9TK5iXzOI/3U5KljST4z8n7MPJY1XkPzLX2HNWEuU/fbNuEVbJ5T81W9xnxnnmPzZylawmI+c/H2wU0RHF5z9XhYwGPl/oPwBwiZR78eg/BtB4IrN76T/6+z2W4/3pP2n3sakfeOo/NSLGVovq6j8ssW82WVXrP5/8FOrHuOs/1jJnoh8V7D+6n8jRr2rsPyHD5RPNuew/scMwUM8C7T+gj5cZEEbtPw2gHkvpg+0/vjoA37O87T/qEHz8xvDtP2t0kTd3IO4/PtxY/RVM7j8byYEn8XPuP4R8i7FSmO4/WPuOioC57j/b+89+vNfuPxMNvDRE8+4/Dq55OVEM7z+I1aQZGSPvP95eToPNN+8/Rm7EbpxK7z/tAhFMsFvvP1iDeDMwa+8/YMOTF0B57z/eDOj3AIbvP22mHBORke8/nfsgGAyc7z800MFVi6XvPzjAT+glru8/dqgW5fC17z9GU32D/7zvP8C9t0Njw+8/4jgFEyzJ7z8=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]}},\"selected\":{\"id\":\"4234\"},\"selection_policy\":{\"id\":\"4233\"}},\"id\":\"4123\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"children\":[{\"id\":\"4090\"},{\"id\":\"4129\"},{\"id\":\"4168\"}]},\"id\":\"4208\",\"type\":\"Row\"},{\"attributes\":{},\"id\":\"4216\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"source\":{\"id\":\"4201\"}},\"id\":\"4206\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"4229\",\"type\":\"AllLabels\"},{\"attributes\":{\"coordinates\":null,\"group\":null,\"text\":\"Sigmoid\",\"text_color\":\"#E0E0E0\",\"text_font\":\"Josefin Sans\",\"text_font_size\":\"2.1em\"},\"id\":\"4091\",\"type\":\"Title\"},{\"attributes\":{\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"4216\"},\"group\":null,\"major_label_policy\":{\"id\":\"4217\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"4106\"}},\"id\":\"4105\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"4217\",\"type\":\"AllLabels\"},{\"attributes\":{\"style\":{\"font-size\":\"250%\"},\"text\":\"Different activation functions\"},\"id\":\"4207\",\"type\":\"Div\"},{\"attributes\":{\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"4231\"},\"group\":null,\"major_label_policy\":{\"id\":\"4232\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"4180\"}},\"id\":\"4179\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"4231\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"4097\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"4219\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"yellow\",\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"4204\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"4239\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"yellow\",\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"4203\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"4232\",\"type\":\"AllLabels\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"orange\",\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"4164\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"4220\",\"type\":\"AllLabels\"},{\"attributes\":{},\"id\":\"4240\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"4190\",\"type\":\"SaveTool\"},{\"attributes\":{},\"id\":\"4192\",\"type\":\"HelpTool\"},{\"attributes\":{\"axis\":{\"id\":\"4144\"},\"coordinates\":null,\"dimension\":1,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"4147\",\"type\":\"Grid\"},{\"attributes\":{\"children\":[{\"id\":\"4207\"},{\"id\":\"4208\"}]},\"id\":\"4209\",\"type\":\"Column\"},{\"attributes\":{},\"id\":\"4191\",\"type\":\"ResetTool\"},{\"attributes\":{\"overlay\":{\"id\":\"4193\"}},\"id\":\"4189\",\"type\":\"BoxZoomTool\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00187\",\"sizing_mode\":\"stretch_width\"},\"id\":\"4272\",\"type\":\"Spacer\"},{\"attributes\":{\"overlay\":{\"id\":\"4154\"}},\"id\":\"4150\",\"type\":\"BoxZoomTool\"},{\"attributes\":{},\"id\":\"4233\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"4193\",\"type\":\"BoxAnnotation\"},{\"attributes\":{},\"id\":\"4187\",\"type\":\"PanTool\"},{\"attributes\":{},\"id\":\"4234\",\"type\":\"Selection\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"red\",\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"4125\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"4180\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"4171\",\"type\":\"DataRange1d\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"AAAAAAAAFMBDYt7OkJgTwIbEvJ0hMRPAyiabbLLJEsANiXk7Q2ISwFDrVwrU+hHAk0022WSTEcDXrxSo9SsRwBoS83aGxBDAXXTRRRddEMBArV8pUOsPwMdxHMdxHA/ATjbZZJNNDsDU+pUCtX4NwFq/UqDWrwzA4YMPPvjgC8BoSMzbGRILwO4MiXk7QwrAdNFFF110CcD7lQK1fqUIwIFav1Kg1gfACB988MEHB8CO4ziO4zgGwBWo9SsFagXAm2yyySabBMAiMW9nSMwDwKj1KwVq/QLAL7rooosuAsC1fqVArV8BwDxDYt7OkADAhA8++OCD/7+SmLczJOb9v54hMW9nSPy/qqqqqqqq+r+4MyTm7Qz5v8S8nSExb/e/0kUXXXTR9b/ezpCYtzP0v+xXCtT6lfK/+OCDDz748L8I1PqVArXuvyjm7QyJeeu/QPjggw8+6L9YCtT6lQLlv3Acx3Ecx+G/IF100UUX3b9QgVq/UqDWv4ClQK1fKdC/YJNNNtlkw78Ab2dIzNupvwBvZ0jM26k/YJNNNtlkwz+ApUCtXynQP1CBWr9SoNY/EF100UUX3T9wHMdxHMfhP1gK1PqVAuU/QPjggw8+6D8g5u0MiXnrPwjU+pUCte4/+OCDDz748D/sVwrU+pXyP9zOkJi3M/Q/0EUXXXTR9T/EvJ0hMW/3P7gzJObtDPk/rKqqqqqq+j+cITFvZ0j8P5CYtzMk5v0/hA8++OCD/z88Q2LezpAAQLR+pUCtXwFALrrooosuAkCo9SsFav0CQCIxb2dIzANAmmyyySabBEAUqPUrBWoFQI7jOI7jOAZACB988MEHB0CAWr9SoNYHQPyVArV+pQhAdNFFF110CUDsDIl5O0MKQGhIzNsZEgtA4IMPPvjgC0Bcv1Kg1q8MQNT6lQK1fg1ATDbZZJNNDkDIcRzHcRwPQECtXylQ6w9AXHTRRRddEEAaEvN2hsQQQNavFKj1KxFAlE022WSTEUBQ61cK1PoRQAyJeTtDYhJAyiabbLLJEkCGxLydITETQERi3s6QmBNAAAAAAAAAFEA=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]},\"y\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABvZ0jM26k/YJNNNtlkwz+ApUCtXynQP1CBWr9SoNY/EF100UUX3T9wHMdxHMfhP1gK1PqVAuU/QPjggw8+6D8g5u0MiXnrPwjU+pUCte4/+OCDDz748D/sVwrU+pXyP9zOkJi3M/Q/0EUXXXTR9T/EvJ0hMW/3P7gzJObtDPk/rKqqqqqq+j+cITFvZ0j8P5CYtzMk5v0/hA8++OCD/z88Q2LezpAAQLR+pUCtXwFALrrooosuAkCo9SsFav0CQCIxb2dIzANAmmyyySabBEAUqPUrBWoFQI7jOI7jOAZACB988MEHB0CAWr9SoNYHQPyVArV+pQhAdNFFF110CUDsDIl5O0MKQGhIzNsZEgtA4IMPPvjgC0Bcv1Kg1q8MQNT6lQK1fg1ATDbZZJNNDkDIcRzHcRwPQECtXylQ6w9AXHTRRRddEEAaEvN2hsQQQNavFKj1KxFAlE022WSTEUBQ61cK1PoRQAyJeTtDYhJAyiabbLLJEkCGxLydITETQERi3s6QmBNAAAAAAAAAFEA=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]}},\"selected\":{\"id\":\"4240\"},\"selection_policy\":{\"id\":\"4239\"}},\"id\":\"4201\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"4154\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"coordinates\":null,\"group\":null,\"text\":\"Tanh\",\"text_color\":\"#E0E0E0\",\"text_font\":\"Josefin Sans\",\"text_font_size\":\"2.1em\"},\"id\":\"4130\",\"type\":\"Title\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"orange\",\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"4165\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"4132\",\"type\":\"DataRange1d\"},{\"attributes\":{\"background_fill_alpha\":0.9,\"background_fill_color\":\"#0f0f0f\",\"below\":[{\"id\":\"4101\"}],\"border_fill_alpha\":0.8,\"border_fill_color\":\"#111111\",\"center\":[{\"id\":\"4104\"},{\"id\":\"4108\"}],\"height\":250,\"left\":[{\"id\":\"4105\"}],\"outline_line_alpha\":0.25,\"outline_line_color\":\"#E0E0E0\",\"outline_line_width\":3,\"renderers\":[{\"id\":\"4127\"}],\"title\":{\"id\":\"4091\"},\"toolbar\":{\"id\":\"4116\"},\"width\":400,\"x_range\":{\"id\":\"4093\"},\"x_scale\":{\"id\":\"4097\"},\"y_range\":{\"id\":\"4095\"},\"y_scale\":{\"id\":\"4099\"}},\"id\":\"4090\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"tools\":[{\"id\":\"4109\"},{\"id\":\"4110\"},{\"id\":\"4111\"},{\"id\":\"4112\"},{\"id\":\"4113\"},{\"id\":\"4114\"}]},\"id\":\"4116\",\"type\":\"Toolbar\"},{\"attributes\":{},\"id\":\"4110\",\"type\":\"WheelZoomTool\"},{\"attributes\":{},\"id\":\"4109\",\"type\":\"PanTool\"},{\"attributes\":{\"axis\":{\"id\":\"4179\"},\"coordinates\":null,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"4182\",\"type\":\"Grid\"},{\"attributes\":{\"overlay\":{\"id\":\"4115\"}},\"id\":\"4111\",\"type\":\"BoxZoomTool\"},{\"attributes\":{},\"id\":\"4112\",\"type\":\"SaveTool\"},{\"attributes\":{\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"4228\"},\"group\":null,\"major_label_policy\":{\"id\":\"4229\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"4184\"}},\"id\":\"4183\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"4113\",\"type\":\"ResetTool\"},{\"attributes\":{},\"id\":\"4222\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"4175\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"4141\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"4102\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"4138\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"4223\",\"type\":\"AllLabels\"},{\"attributes\":{},\"id\":\"4149\",\"type\":\"WheelZoomTool\"},{\"attributes\":{},\"id\":\"4152\",\"type\":\"ResetTool\"},{\"attributes\":{\"line_color\":\"orange\",\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"4163\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"4225\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"4136\",\"type\":\"LinearScale\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"4162\"},\"glyph\":{\"id\":\"4163\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"4165\"},\"nonselection_glyph\":{\"id\":\"4164\"},\"view\":{\"id\":\"4167\"}},\"id\":\"4166\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"4226\",\"type\":\"AllLabels\"},{\"attributes\":{\"axis\":{\"id\":\"4140\"},\"coordinates\":null,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"4143\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"4134\",\"type\":\"DataRange1d\"},{\"attributes\":{\"line_color\":\"red\",\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"4124\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"4177\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"4236\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"background_fill_alpha\":0.9,\"background_fill_color\":\"#0f0f0f\",\"below\":[{\"id\":\"4140\"}],\"border_fill_alpha\":0.8,\"border_fill_color\":\"#111111\",\"center\":[{\"id\":\"4143\"},{\"id\":\"4147\"}],\"height\":250,\"left\":[{\"id\":\"4144\"}],\"outline_line_alpha\":0.25,\"outline_line_color\":\"#E0E0E0\",\"outline_line_width\":3,\"renderers\":[{\"id\":\"4166\"}],\"title\":{\"id\":\"4130\"},\"toolbar\":{\"id\":\"4155\"},\"width\":400,\"x_range\":{\"id\":\"4132\"},\"x_scale\":{\"id\":\"4136\"},\"y_range\":{\"id\":\"4134\"},\"y_scale\":{\"id\":\"4138\"}},\"id\":\"4129\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"source\":{\"id\":\"4123\"}},\"id\":\"4128\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"4237\",\"type\":\"Selection\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"red\",\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"4126\",\"type\":\"Line\"},{\"attributes\":{\"line_color\":\"yellow\",\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"4202\",\"type\":\"Line\"},{\"attributes\":{\"coordinates\":null,\"group\":null,\"text\":\"ReLU\",\"text_color\":\"#E0E0E0\",\"text_font\":\"Josefin Sans\",\"text_font_size\":\"2.1em\"},\"id\":\"4169\",\"type\":\"Title\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"AAAAAAAAFMBDYt7OkJgTwIbEvJ0hMRPAyiabbLLJEsANiXk7Q2ISwFDrVwrU+hHAk0022WSTEcDXrxSo9SsRwBoS83aGxBDAXXTRRRddEMBArV8pUOsPwMdxHMdxHA/ATjbZZJNNDsDU+pUCtX4NwFq/UqDWrwzA4YMPPvjgC8BoSMzbGRILwO4MiXk7QwrAdNFFF110CcD7lQK1fqUIwIFav1Kg1gfACB988MEHB8CO4ziO4zgGwBWo9SsFagXAm2yyySabBMAiMW9nSMwDwKj1KwVq/QLAL7rooosuAsC1fqVArV8BwDxDYt7OkADAhA8++OCD/7+SmLczJOb9v54hMW9nSPy/qqqqqqqq+r+4MyTm7Qz5v8S8nSExb/e/0kUXXXTR9b/ezpCYtzP0v+xXCtT6lfK/+OCDDz748L8I1PqVArXuvyjm7QyJeeu/QPjggw8+6L9YCtT6lQLlv3Acx3Ecx+G/IF100UUX3b9QgVq/UqDWv4ClQK1fKdC/YJNNNtlkw78Ab2dIzNupvwBvZ0jM26k/YJNNNtlkwz+ApUCtXynQP1CBWr9SoNY/EF100UUX3T9wHMdxHMfhP1gK1PqVAuU/QPjggw8+6D8g5u0MiXnrPwjU+pUCte4/+OCDDz748D/sVwrU+pXyP9zOkJi3M/Q/0EUXXXTR9T/EvJ0hMW/3P7gzJObtDPk/rKqqqqqq+j+cITFvZ0j8P5CYtzMk5v0/hA8++OCD/z88Q2LezpAAQLR+pUCtXwFALrrooosuAkCo9SsFav0CQCIxb2dIzANAmmyyySabBEAUqPUrBWoFQI7jOI7jOAZACB988MEHB0CAWr9SoNYHQPyVArV+pQhAdNFFF110CUDsDIl5O0MKQGhIzNsZEgtA4IMPPvjgC0Bcv1Kg1q8MQNT6lQK1fg1ATDbZZJNNDkDIcRzHcRwPQECtXylQ6w9AXHTRRRddEEAaEvN2hsQQQNavFKj1KxFAlE022WSTEUBQ61cK1PoRQAyJeTtDYhJAyiabbLLJEkCGxLydITETQERi3s6QmBNAAAAAAAAAFEA=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]},\"y\":{\"__ndarray__\":\"Ed9olkH/779xHDX2Fv/vv2PTXMvi/u+/tykv86L+779nyqrQVP7vv5RqJTH1/e+/ANXYKoD9778+DPrz8Pzvv+R3srBB/O+/seP1NWv7778B7b6+ZPrvvzRIspAj+e+/GFCDjJr377/6GqylufXvv5SfGjxt8++/g1tUUZ3w7782/jqRLO3vvwpKByX36O+/kLQ0RdHj778FKfN8hd3vv2dPQ4/S1e+/GGkq62jM77+jbX6Z58Dvv1i8zYvYsu+/1CEZMayh779DqukytIzvv2zAbjwdc++/qVLVrudT779zlYUr3y3vv8N/uOiQ/+6/fDIb2kDH7r+tmFja3YLuv0cK+Tj1L+6/zv7GYKbL7b/XD/S8l1LtvzWhkZzuwOy/ljVLlkwS7L8lhcTR1UHrvy50/oZESuq/sMhCwQ4m6b89c/imo8/nv3hPlK3EQea/JYsBmPx35L/eJkVXMW/iv9AYbbpGJuC/sE3YDH49279KKI4ofLrVvzdYpvc6p8+/IIThpDM/w7+rNaLaLNapv6s1otos1qk/IIThpDM/wz83WKb3OqfPP0oojih8utU/o03YDH492z/QGG26RibgP94mRVcxb+I/JYsBmPx35D9zT5StxEHmPz1z+Kajz+c/sMhCwQ4m6T8udP6GRErqPySFxNHVQes/lTVLlkwS7D81oZGc7sDsP9cP9LyXUu0/z/7GYKbL7T9GCvk49S/uP62YWNrdgu4/fDIb2kDH7j/Df7jokP/uP3OVhSvfLe8/qVLVrudT7z9swG48HXPvP0Oq6TK0jO8/1CEZMayh7z9YvM2L2LLvP6NtfpnnwO8/GGkq62jM7z9nT0OP0tXvPwUp83yF3e8/kLQ0RdHj7z8KSgcl9+jvPzb+OpEs7e8/g1tUUZ3w7z+Unxo8bfPvP/oarKW59e8/GFCDjJr37z80SLKQI/nvPwHtvr5k+u8/seP1NWv77z/kd7KwQfzvPz4M+vPw/O8/ANXYKoD97z+UaiUx9f3vP2fKqtBU/u8/tykv86L+7z9j01zL4v7vP3EcNfYW/+8/Ed9olkH/7z8=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]}},\"selected\":{\"id\":\"4237\"},\"selection_policy\":{\"id\":\"4236\"}},\"id\":\"4162\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"4151\",\"type\":\"SaveTool\"},{\"attributes\":{},\"id\":\"4184\",\"type\":\"BasicTicker\"},{\"attributes\":{\"axis\":{\"id\":\"4183\"},\"coordinates\":null,\"dimension\":1,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"4186\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"4106\",\"type\":\"BasicTicker\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"4115\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"tools\":[{\"id\":\"4148\"},{\"id\":\"4149\"},{\"id\":\"4150\"},{\"id\":\"4151\"},{\"id\":\"4152\"},{\"id\":\"4153\"}]},\"id\":\"4155\",\"type\":\"Toolbar\"},{\"attributes\":{\"background_fill_alpha\":0.9,\"background_fill_color\":\"#0f0f0f\",\"below\":[{\"id\":\"4179\"}],\"border_fill_alpha\":0.8,\"border_fill_color\":\"#111111\",\"center\":[{\"id\":\"4182\"},{\"id\":\"4186\"}],\"height\":250,\"left\":[{\"id\":\"4183\"}],\"outline_line_alpha\":0.25,\"outline_line_color\":\"#E0E0E0\",\"outline_line_width\":3,\"renderers\":[{\"id\":\"4205\"}],\"title\":{\"id\":\"4169\"},\"toolbar\":{\"id\":\"4194\"},\"width\":400,\"x_range\":{\"id\":\"4171\"},\"x_scale\":{\"id\":\"4175\"},\"y_range\":{\"id\":\"4173\"},\"y_scale\":{\"id\":\"4177\"}},\"id\":\"4168\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"axis\":{\"id\":\"4105\"},\"coordinates\":null,\"dimension\":1,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"4108\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"4188\",\"type\":\"WheelZoomTool\"},{\"attributes\":{\"source\":{\"id\":\"4162\"}},\"id\":\"4167\",\"type\":\"CDSView\"},{\"attributes\":{\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"4225\"},\"group\":null,\"major_label_policy\":{\"id\":\"4226\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"4141\"}},\"id\":\"4140\",\"type\":\"LinearAxis\"},{\"attributes\":{\"tools\":[{\"id\":\"4187\"},{\"id\":\"4188\"},{\"id\":\"4189\"},{\"id\":\"4190\"},{\"id\":\"4191\"},{\"id\":\"4192\"}]},\"id\":\"4194\",\"type\":\"Toolbar\"},{\"attributes\":{},\"id\":\"4099\",\"type\":\"LinearScale\"},{\"attributes\":{\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"4219\"},\"group\":null,\"major_label_policy\":{\"id\":\"4220\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"4102\"}},\"id\":\"4101\",\"type\":\"LinearAxis\"},{\"attributes\":{\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"4222\"},\"group\":null,\"major_label_policy\":{\"id\":\"4223\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"4145\"}},\"id\":\"4144\",\"type\":\"LinearAxis\"},{\"attributes\":{\"client_comm_id\":\"400ac8e23f2e4acfb1875b46ea760e99\",\"comm_id\":\"b3a2ab33f97f433e8825b36c625b8d7b\",\"plot_id\":\"4210\"},\"id\":\"4363\",\"type\":\"panel.models.comm_manager.CommManager\"},{\"attributes\":{},\"id\":\"4114\",\"type\":\"HelpTool\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"4201\"},\"glyph\":{\"id\":\"4202\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"4204\"},\"nonselection_glyph\":{\"id\":\"4203\"},\"view\":{\"id\":\"4206\"}},\"id\":\"4205\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"4123\"},\"glyph\":{\"id\":\"4124\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"4126\"},\"nonselection_glyph\":{\"id\":\"4125\"},\"view\":{\"id\":\"4128\"}},\"id\":\"4127\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"4148\",\"type\":\"PanTool\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00186\",\"sizing_mode\":\"stretch_width\"},\"id\":\"4211\",\"type\":\"Spacer\"},{\"attributes\":{\"axis\":{\"id\":\"4101\"},\"coordinates\":null,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"4104\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"4173\",\"type\":\"DataRange1d\"}],\"root_ids\":[\"4210\",\"4363\"]},\"title\":\"Bokeh Application\",\"version\":\"2.4.3\"}};\n",
       "    var render_items = [{\"docid\":\"c6b166bf-d91b-4d39-bc41-f9de9c5a081a\",\"root_ids\":[\"4210\"],\"roots\":{\"4210\":\"9d0d059d-3d72-4cc5-aadf-97f393bc9b0b\"}}];\n",
       "    root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "    for (const render_item of render_items) {\n",
       "      for (const root_id of render_item.root_ids) {\n",
       "\tconst id_el = document.getElementById(root_id)\n",
       "\tif (id_el.children.length && (id_el.children[0].className === 'bk-root')) {\n",
       "\t  const root_el = id_el.children[0]\n",
       "\t  root_el.id = root_el.id + '-rendered'\n",
       "\t}\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "  if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    var attempts = 0;\n",
       "    var timer = setInterval(function(root) {\n",
       "      if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else if (document.readyState == \"complete\") {\n",
       "        attempts++;\n",
       "        if (attempts > 200) {\n",
       "          clearInterval(timer);\n",
       "          console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\");\n",
       "        }\n",
       "      }\n",
       "    }, 25, root)\n",
       "  }\n",
       "})(window);</script>"
      ],
      "text/plain": [
       "Row\n",
       "    [0] HSpacer()\n",
       "    [1] Bokeh(Column)\n",
       "    [2] HSpacer()"
      ]
     },
     "execution_count": 23,
     "metadata": {
      "application/vnd.holoviews_exec.v0+json": {
       "id": "4210"
      }
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from bokeh.models import Div\n",
    "from bokeh.layouts import column, row\n",
    "from cml.plot import cml_figure\n",
    "# Functions to\n",
    "funcs = [('Sigmoid',sigmoid,'red'), ('Tanh',tanh,'orange'), ('ReLU',relu,'yellow')]\n",
    "# Generating the x axis\n",
    "x = np.linspace(-5, 5, 100)\n",
    "plots = []\n",
    "for (name, func, color) in funcs:\n",
    "    cur_plot = cml_figure(plot_width=400, plot_height=250, title=name)\n",
    "    cur_plot.line(x, func(x), color=color, line_width=4)\n",
    "    plots.append(cur_plot)\n",
    "plot = center_plot(column(Div(text = \"Different activation functions\", style={'font-size': '250%'}), row(*plots)))\n",
    "plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Coding the whole network from scratch\n",
    "\n",
    "You should now have all the tools necessary to apply neural networks from scratch to a more complex problem. In the following exercise, we simply removed any guideline code, and you need to code all the procedure for training a NN and **apply it to audio data**. You will use the spectral features discussed in the previous exercise as an input.\n",
    "\n",
    "***\n",
    "\n",
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #192841; border-color: #779ecb\">\n",
    "\n",
    "> ### Going further **(optional exercise) **\n",
    "\n",
    ">  1. Based on the previous neural network, upgrade the code to a 3-layer neural network\n",
    ">  2. Implement the *softmax regression* on top of your 3-layer network\n",
    ">  3. Use the provided code to perform classification on a pre-defined set of features\n",
    ">  4. As previously, change the set of features to assess their different accuracies\n",
    ">  5. Evaluate the neural network accuracy for all features combinations\n",
    ">  6. What happens if the learning rate is too large ? What is this phenomenon ?\n",
    ">  7. Perform a more advanced visualization of the learning process.\n",
    "  \n",
    "\n",
    "</div>\n",
    "\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "######################\n",
    "# YOUR CODE GOES HERE\n",
    "######################\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using `Pytorch` to enjoy life\n",
    "\n",
    "Up to now, we have been writing every operations by ourselves (in order to better understand the mathematics behind NN). However, there exists of course some simplifying libraries that provide large simplifications to this question.\n",
    "\n",
    "One of the most powerful and complete library of this sort is `Pytorch`, which has been developed for several years (even prior to the recent boom of deep learning). `Pytorch` provides a large set of pre-coded layers, but also **computational graphs** and **autograd**, which are very powerful paradigms allowing to define complex operators and automatically taking derivatives.\n",
    "\n",
    "## An (extremely) fast and dirty introduction to `Pytorch`\n",
    "\n",
    "PyTorch is a popular open-source deep learning framework based on the Torch library. It is primarily developed by Facebook AI research team, and it provides a simple and efficient way to build deep learning models. In this extremely short crash course, we will cover the basics of PyTorch, including:\n",
    "\n",
    "* Tensors\n",
    "* Automatic differentiation\n",
    "* Neural networks\n",
    "* Training a neural network on a dataset\n",
    "\n",
    "For a complete tutorial on PyTorch, please check out the [official PyTorch documentation](https://pytorch.org/tutorials/). This tutorial covers a wide range of topics, including the basics of PyTorch, building and training neural networks, working with data and dataloaders, and deploying models to production."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0.8657, 1.6808],\n",
      "        [1.3434, 0.8406],\n",
      "        [1.2232, 1.6256],\n",
      "        [0.8954, 0.7533],\n",
      "        [1.0638, 0.1683],\n",
      "        [0.5485, 1.4991],\n",
      "        [1.1644, 0.6346],\n",
      "        [0.2061, 0.6225],\n",
      "        [1.7834, 1.4292],\n",
      "        [0.9946, 1.0269]], device='mps:0')\n",
      "tensor([[0.7322, 0.9143],\n",
      "        [0.9577, 0.2017],\n",
      "        [0.6909, 0.8956],\n",
      "        [0.2449, 0.2012],\n",
      "        [0.2686, 0.0524],\n",
      "        [0.2230, 0.5372],\n",
      "        [0.7144, 0.0667],\n",
      "        [0.0177, 0.5494],\n",
      "        [0.8551, 0.8944],\n",
      "        [0.3271, 0.7141]], device='mps:0')\n",
      "tensor([[0.4127],\n",
      "        [0.4358],\n",
      "        [0.4109],\n",
      "        [0.4068],\n",
      "        [0.4083],\n",
      "        [0.4056],\n",
      "        [0.4248],\n",
      "        [0.4037],\n",
      "        [0.4189],\n",
      "        [0.4065]], device='mps:0', grad_fn=<SigmoidBackward0>)\n"
     ]
    }
   ],
   "source": [
    "import torch \n",
    "a = torch.rand(10, 2)\n",
    "b = torch.rand(10, 2)\n",
    "import torch.nn as nn\n",
    "a = a.to(\"mps\")\n",
    "b = b.to(\"mps\")\n",
    "model = nn.Sequential(\n",
    "    nn.Linear(2, 5),\n",
    "    nn.ReLU(),\n",
    "    nn.Linear(5, 1),\n",
    "    nn.Sigmoid()\n",
    ")\n",
    "model = model.to(\"mps\")\n",
    "out = model(a)\n",
    "out =out.to(\"cpu\")\n",
    "print(out)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'nn' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "Cell \u001b[0;32mIn[47], line 4\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mtorch\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mnn\u001b[39;00m\n\u001b[1;32m      3\u001b[0m x \u001b[38;5;241m=\u001b[39m torch\u001b[38;5;241m.\u001b[39mones((\u001b[38;5;241m100\u001b[39m, \u001b[38;5;241m3\u001b[39m))\n\u001b[0;32m----> 4\u001b[0m net \u001b[38;5;241m=\u001b[39m \u001b[43mnn\u001b[49m\u001b[38;5;241m.\u001b[39mSequential(\n\u001b[1;32m      5\u001b[0m     [\n\u001b[1;32m      6\u001b[0m       nn\u001b[38;5;241m.\u001b[39mLinear(\u001b[38;5;241m3\u001b[39m, \u001b[38;5;241m20\u001b[39m),\n\u001b[1;32m      7\u001b[0m       nn\u001b[38;5;241m.\u001b[39mReLU(),\n\u001b[1;32m      8\u001b[0m       nn\u001b[38;5;241m.\u001b[39mLinear(\u001b[38;5;241m20\u001b[39m, \u001b[38;5;241m1\u001b[39m),\n\u001b[1;32m      9\u001b[0m       nn\u001b[38;5;241m.\u001b[39mSigmoid()\n\u001b[1;32m     10\u001b[0m     ]);\n\u001b[1;32m     11\u001b[0m l1 \u001b[38;5;241m=\u001b[39m nn\u001b[38;5;241m.\u001b[39mLinear(\u001b[38;5;241m3\u001b[39m, \u001b[38;5;241m20\u001b[39m)\n\u001b[1;32m     12\u001b[0m o1 \u001b[38;5;241m=\u001b[39m l1(x)\n",
      "\u001b[0;31mNameError\u001b[0m: name 'nn' is not defined"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "import torch.nn\n",
    "x = torch.ones((100, 3))\n",
    "net = nn.Sequential(\n",
    "    [\n",
    "      nn.Linear(3, 20),\n",
    "      nn.ReLU(),\n",
    "      nn.Linear(20, 1),\n",
    "      nn.Sigmoid()\n",
    "    ]);\n",
    "l1 = nn.Linear(3, 20)\n",
    "o1 = l1(x)\n",
    "r1 = nn.ReLU()\n",
    "o2 = r1(o1)\n",
    "l2 = nn.Linear(20, 1)\n",
    "s2 = nn.Sigmoid()\n",
    "y = s2(l2(o2))\n",
    "print(y.shape)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tensors\n",
    "At its core, PyTorch is all about tensors. A tensor is a generalization of vectors and matrices to an arbitrary number of dimensions. A scalar is a 0-dimensional tensor, a vector is a 1-dimensional tensor, and a matrix is a 2-dimensional tensor.\n",
    "\n",
    "#### Creating Tensors\n",
    "We can create a PyTorch tensor from a Python list or a NumPy array using the `torch.tensor()` function, but also tensors of a specific size and/or with all zeros or all ones using the `torch.zeros()` and `torch.ones()` functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([1, 2, 3, 4])\n",
      "tensor([[1, 2],\n",
      "        [3, 4]])\n",
      "tensor([[0., 0., 0.],\n",
      "        [0., 0., 0.]])\n",
      "tensor([[1., 1.],\n",
      "        [1., 1.],\n",
      "        [1., 1.]])\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "import numpy as np\n",
    "# create a tensor from a Python list\n",
    "x = torch.tensor([1, 2, 3, 4])\n",
    "print(x)\n",
    "# create a tensor from a NumPy array\n",
    "y = torch.tensor(np.array([[1, 2], [3, 4]]))\n",
    "print(y)\n",
    "# create a 2x3 tensor of zeros\n",
    "z = torch.zeros((2, 3))\n",
    "print(z)\n",
    "# create a 3x2 tensor of ones\n",
    "w = torch.ones((3, 2))\n",
    "print(w)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Tensor operations\n",
    "\n",
    "PyTorch tensors support a wide range of operations, including arithmetic operations like addition, subtraction, multiplication, and division, as well as more advanced operations like matrix multiplication, element-wise multiplication, and concatenation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([ 6,  8, 10, 12])\n",
      "tensor([ 5, 12, 21, 32])\n",
      "tensor([[19, 22],\n",
      "        [43, 50]])\n",
      "tensor([[1, 2, 5, 6],\n",
      "        [3, 4, 7, 8]])\n"
     ]
    }
   ],
   "source": [
    "# create two tensors\n",
    "x = torch.tensor([1, 2, 3, 4])\n",
    "y = torch.tensor([5, 6, 7, 8])\n",
    "# add the two tensors\n",
    "z = x + y\n",
    "print(z)\n",
    "# element-wise multiplication\n",
    "w = x * y\n",
    "print(w)\n",
    "# matrix multiplication\n",
    "a = torch.tensor([[1, 2], [3, 4]])\n",
    "b = torch.tensor([[5, 6], [7, 8]])\n",
    "c = torch.matmul(a, b)\n",
    "print(c)\n",
    "# concatenation\n",
    "d = torch.cat((a, b), dim=1)\n",
    "print(d)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Automatic differentiation\n",
    "\n",
    "The core of PyTorch is the autograd package. It provides automatic differentiation for all operations on Tensors.\n",
    "\n",
    "PyTorch's autograd package provides automatic differentiation for all operations on Tensors. It is a define-by-run framework, which means that your backpropagation is defined by how your code is run, and that every single iteration can be different.\n",
    "\n",
    "To compute gradients, the `requires_grad` property of Tensors needs to `True` (which is the case by default). This tells PyTorch to track all operations on the Tensor. You can then call `.backward()` on a Tensor to compute the gradients with respect to that Tensor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(11.)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "tensor(11.)"
      ]
     },
     "execution_count": 98,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch\n",
    "\n",
    "x = torch.tensor(2.0, requires_grad=True)\n",
    "y = 2*x**2 + 3*x - 1\n",
    "y.backward()\n",
    "\n",
    "print(x.grad)\n",
    "x.grad"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Defining our network\n",
    "\n",
    "When building neural networks we frequently think of arranging the computation into layers, some of which have learnable parameters which will be optimized during learning. In `PyTorch`, the `nn` package provides higher-level abstractions over raw computational graphs that are useful for building neural networks. The `nn` package defines a set of `Modules`, which are roughly equivalent to neural network layers. A `Module` receives input `Tensors` and computes output `Tensors`, but may also hold internal state such as `Tensors` containing learnable parameters. The nn package also defines a set of useful loss functions that are commonly used when training neural networks.\n",
    "\n",
    "In the following example, we use the `nn` package to show how easy it is to instantiate our previous three-layers network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "# Define the input dimensions\n",
    "in_size = 1000\n",
    "# Number of neurons in a layer\n",
    "hidden_size = 100\n",
    "# Output (target) dimension\n",
    "output_size = 10\n",
    "# Use the nn package to define our model and loss function.\n",
    "model = torch.nn.Sequential(\n",
    "    torch.nn.Linear(in_size, hidden_size),\n",
    "    torch.nn.Sigmoid(),\n",
    "    torch.nn.Linear(hidden_size, hidden_size),\n",
    "    torch.nn.Tanh(),\n",
    "    torch.nn.Linear(hidden_size, output_size),\n",
    "    torch.nn.Softmax(dim=1)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Optimizing the network\n",
    "\n",
    "Up to this point we have updated the weights of our models by manually performing the gradient descent algorithm (changing the parameters vectors). Although this is not a huge burden for simple optimization algorithms like stochastic gradient descent, in practice we often train neural networks using more sophisticated optimizers like AdaGrad, RMSProp or Adam (that we will see later in this course)\n",
    "\n",
    "The `optim` package in PyTorch abstracts the idea of an optimization algorithm and provides implementations of commonly used optimization algorithms, and greatly simplfies the training loop associated with training a neural network.\n",
    "\n",
    "For the sake of presentation we will use random inputs $\\mathbf{x}$ that should be matched with random outputs $\\mathbf{y}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "batch_size = 64\n",
    "# Create random Tensors to hold inputs and outputs\n",
    "x = torch.randn(batch_size, in_size)\n",
    "y = torch.randn(batch_size, output_size)\n",
    "\n",
    "class MyLayer(nn.Module):\n",
    "\n",
    "    def forward()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the following example we optimize the model using the Adam algorithm provided by the `optim` package, based on a `MSE` loss."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(641.4479, grad_fn=<MseLossBackward0>)\n"
     ]
    }
   ],
   "source": [
    "# Learning rate\n",
    "learning_rate = 1e-4\n",
    "# Loss function that we will use\n",
    "loss_fn = torch.nn.MSELoss(reduction='sum')\n",
    "# Optimizer to fit the weights of the network\n",
    "optimizer = torch.optim.RMSProp(model.parameters(), lr=learning_rate)\n",
    "for t in range(500):\n",
    "    # Forward pass: compute predicted y by passing x to the model.\n",
    "    y_pred = model(x)\n",
    "    # Compute the loss.\n",
    "    loss = loss_fn(y_pred, y)\n",
    "    # Before the backward pass, zero all of the network gradients\n",
    "    optimizer.zero_grad()\n",
    "    # Backward pass: compute gradient of the loss with respect to parameters\n",
    "    loss.backward()\n",
    "    # Calling the step function to update the parameters\n",
    "    optimizer.step()\n",
    "print(loss)\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2 - Pytorch to classify audio\n",
    "\n",
    "Now that we know the main components of `Pytorch` to define and optimize networks, your assignement is to define a complete classification problem from audio data, by relying on this toolbox\n",
    "\n",
    "***\n",
    "\n",
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #013220; border-color: #03C03C\">\n",
    "\n",
    "> ### Question 1.1 - Model and definition\n",
    "\n",
    ">   1. Use `Pytorch` to define a model for audio classification\n",
    ">   2. Import the audio features dataset and check that your model produces an output\n",
    "\n",
    "</div>\n",
    "\n",
    "***\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "######################\n",
    "# YOUR CODE GOES HERE\n",
    "######################\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "***\n",
    "\n",
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #013220; border-color: #03C03C\">\n",
    "\n",
    "> ### Question 1.2 - Training the model\n",
    "\n",
    ">   3. Write the optimization loop (think carefully about the _loss function_\n",
    ">   4. As previously, change the set of features to assess their different accuracies\n",
    ">   5. (Optional) Think of how you could use more complex features (time series, audio, STFT) to classify your data\n",
    "\n",
    "</div>\n",
    "\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "######################\n",
    "# YOUR CODE GOES HERE\n",
    "######################\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}