Regression Uncertainties.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "1bba0128",
   "metadata": {},
   "source": [
    "# Regression uncertainties\n",
    "\n",
    "In the previous notebook we discussed how to fit data to a neural network. Now, let's try to add uncertainties to it.\n",
    "\n",
    "We are going to use the `torchbnn` module for that, so that we do not have to go into the details of how everything is implemented.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "23feddde",
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Tuple\n",
    "\n",
    "# import standard PyTorch modules\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "\n",
    "import torchbnn as bnn\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from mpl_toolkits.mplot3d import axes3d\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb1286f0",
   "metadata": {},
   "source": [
    "We start as before: generating the same dataset, which is simple enough that we can visualize the results easily.\n",
    "\n",
    "$f(x) = (3 + \\kappa) x^2 + \\epsilon$, where $\\epsilon \\sim \\mathcal{N}(\\mu=0, \\sigma=10)$ and $\\kappa \\sim \\mathcal{N}(\\mu=0, \\sigma=0.03)$.\n",
    "\n",
    "In this case we do know the true model, so it is interesting to take some time to pinpoint the role of $\\kappa$ and $\\epsilon$. These variables add fluctuation to the results. $\\epsilon$ adds Gaussian noise in a way that is completely independent from $x$ and cannot be traced down to a particular functional dependence. $\\kappa$ changes a specific parameter of the model, in this case the coefficient 3, by around 1%.\n",
    "\n",
    "When fitting a model, the nomenclature *epistemic uncertainty* is often used to refer to uncertainties coming to effects related to different functional models. That is, one can imagine that there are different functions that may fit the data due to the effect of $\\kappa$, such as: $g(x) = 3x^2$ or $h(x) = 2.95x^2$.\n",
    "\n",
    "The nomenclature *aleatoric uncertainty* is used to refer to whichever uncertainty cannot be tracked down to a given model dependence. In this example, different constant factors could be added to the model $g$ to account for the fluctuations in $\\epsilon$.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "5d457cd8",
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_data(N: int) -> np.ndarray:\n",
    "    x = 2*np.random.randn(N, 1)\n",
    "    epsilon = 10*np.random.randn(N, 1)\n",
    "    kappa = 0.03*np.random.randn(N, 1)\n",
    "    z = (3 + kappa)*x**2 + epsilon\n",
    "    return np.concatenate((x, z), axis=1).astype(np.float32)\n",
    "\n",
    "train_data = generate_data(N=1000)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48433f6f",
   "metadata": {},
   "source": [
    "PyTorch allows you to create a class that outputs a single data entry and use that to feed input to your neural network. We will use the following class to feed the data to the neural network. This looks useless if all your data fits in a Numpy array, but notice that if you have a lot of data and cannot load it all in memory, this allows you to read data on demand, as you need it and only the needed samples are stored at a single time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "30205402",
   "metadata": {},
   "outputs": [],
   "source": [
    "class MyDataset(torch.utils.data.Dataset):\n",
    "    def __init__(self, data: np.ndarray):\n",
    "        self.data = data\n",
    "    def __len__(self) -> int:\n",
    "        \"\"\"How many samples do I have?\"\"\"\n",
    "        return len(self.data)\n",
    "    def __getitem__(self, idx):\n",
    "        # give me item with index idx\n",
    "        return {\"data\": self.data[idx, 0:1], \"target\": self.data[idx, 1:]}\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "cc0b0774",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1000\n"
     ]
    }
   ],
   "source": [
    "my_dataset = MyDataset(train_data)\n",
    "print(len(my_dataset))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "6dccfac6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'data': array([-3.1838348], dtype=float32), 'target': array([27.882494], dtype=float32)}\n"
     ]
    }
   ],
   "source": [
    "print(my_dataset[1])"
   ]
  },
  {
   "attachments": {
    "elbo.png": {
     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgsAAACdCAYAAAA6/zu5AAAABHNCSVQICAgIfAhkiAAAABl0RVh0U29mdHdhcmUAZ25vbWUtc2NyZWVuc2hvdO8Dvz4AACAASURBVHic7N0/UONYn/D7795667EyayP7jawboTeyN7JvhDbCG9lPhDtCT4Q3Qk+EbtXWljM8VVuFJsIToYnQRGgiNBGaqDU3QR2hjlBHqCPUESI6NzBNG2MbTAPdzJxPFVPTsv7//enod875FyGEQJIkSZIkaY7/61uvgCRJkiRJ3zcZLEiSJEmStJAMFiRJkiRJWkgGC5IkSZIkLSSDBUmSJEmSFpLBgiRJkiRJC8lgQZIkSZKkhWSwIEmSJEnSQjJYkCRJkiRpIRksSJIkSZK0kAwWJEmSJElaSAYLkiRJkiQtJIMFSZIkSZIWksGCJEmSJEkLyWBBkiRJkqSFZLAgSZIkSdJCMliQJEmSJGmhZwwWCori+eYuSZIkSdLLeL5gIbaxfBktSIvkuG2VqqahaW1G6dfNrUhc7FH8JGs2LXN7WOFjJw4YOiH5PaMF/R7ejJHmDX9aOeFwSJA9x7wj7O6I9InnWoQOQ3/OChc+pqahaVVU3eZ5zgpJ+uv4X4+ZKI9G2G6KpkFGm4FtoE6PlGaEaQroX7mK0ixFllFUq3f3+2PnN6cYSFGUBb8rKMr90y5ers4gjuh/7YZkHr1+wiAwv3JGsxVZQqIUwLxtKkhcGyeuoikZecti2NXGP1XbmFqPnq3iDxtz5lCQJQn5nV05b/jnn2PcgUcyb8XVKrqu0zLa6HP3cUFkdRm1PLzqvHG+Rk4aJwuCpYIsiQjDhKzIKbIc1CqKqqCqVbRGi5ZevbPfFKNPq9+lr3qMjKmNU7q4aRcI6GnBk2+RJP3liCWdH22IWn1bnFwKIcSlOOhUxPrR5fi3412xtbUj9g6OxOFWXZTWdsXx4b7Y3d4S2wenyy5KmuftlqiAKHUOxeWTzPBMHO/tiI16WQACEJXVTbGzdyzOhRBCnIu3B3tifWX8G+W6WN/eFQdvz4UQ5+J4b0esr5RuT7v/edpFLsTealPsXXzt+p+KndVVsXf2tfOZ72ynLtYO5u3tS/F2qy5WNo6uj8ex2Kitiv3z2+McbzbFxtH8eeyv1sXunZ02b/iX389PT8XJXkeUQJTWdsXJ6ak4OzsTp6en4uTtkdjfWhWVUk109mdfgxdH66K5cfxE59IsR2K9tiVO7qz6mTja6YhmvS6aq01RA0FpRax11sTaal1UStycU6Xamtg+PJsx71Ox3VwTB3P3z5xlS5K0lOWChbNd0SytiO2JK+/tVkWU1g9v/n15diR2N5qiAgJKora2Jfbf3v/YkJZwuiea5bJY2Xr7tPM92RrfsJn1cDoXe01EaWVTHM06nG83r495U+w9+HA/TbBwvr8qVjafeF9MWRQsnB+siXJlQxzf/HwmduqI1f2pDTvfE82VTfF25mweGyx8nvWqAMT64ex1PFovC6jdunbH3orN2qqYXtWnNeOBfXYg1uurYvv4/DpIORQdEKzuiS+rcinOT4/E3mZTlEFARXT2z+7M/fKwI2rr8wJnGSxI0lNYImchZdSziY0BduPLUEVRuIqjm2JQRWtj2V3UUo2VikKjP8BsPUvZ5l+X3ifKcxKn9bTzVbXrzxoq6q1S3YLIbmMX24TRiPaswzk57Yse7oiBndO3n3hfPFTu0+//hm4PMG7KycefZ+Jo6kt51WTQ8LGfIQEhCiJglbYx+yNHVVOBD4Th7W/82cjGa9n0nup71oNEWMYbkt6IoXH9eSFLSYGSpk98WlOo6m36o4j4cJ0KH/n1H12cqTQFpTugG9k46cttgST91Tw8WIgcBn8omHbv1nfyLMsgL/jySTXHtX3aQUwSORTDgUwuei0mH3Y3A3NCu0U3bBNEQ1rzHiozp31+ReDgaiY97QUXOiEdDfi16GD3J1cgI82guJNooNDuGUSO+8TJfhFBeAUrbaY/3Y8VJPEHANTq5Agp7iim22+/6DEjzdGHJ4T2RD5TmpIBjdbsHCetO8LplIB3+P50sNWg184ZOfJOI0nP5YEJjgWB4/Kx0sM0JodnJMkVqBOJdmlI0fdxDBUw8UcjvAga3+jF77acJPAJkxyq46Svxq234HEyWZplZJlCo2eg5QlhmKK2DBrV27fUPI2I4pRCbWAYOmoe43shKTrdfhttatlpnJDlOVlepdVtUC1SojAizVV0Y2L+eUoUxWRU0Vut24lpRUaSpGRZRq7odA39ZnicpORZRqY26LU0yGKCKKFQdRqtBtrST4QM3zQwky5hOKTxok+U6zWIA6JUQTeM8X7IUrKqxufDFnoB1ZbNdGFGEft4cQ6KgnJdjVdt9GgrIV6UoyhAUVCg0up10RXIIo8g4TppU7kZPl+MM3pHqTugOzlekZB8ALU948lttGm9GxFkFv2nKoFJQsJPUOkZU+fctdzHDYHaJoPJFc18vHct7hbKZIRuQHorgVWjbeqkbkCijAPCoihAa2MaVSgSAi8iUxSUoqDQ2/NLFLU2t2IrIE9iPlKm25i3U1Ta7Rb8+jtZnsFUam+rbZDZHgkNmVItSc/gYSULuY/jf6Lc7nL7vhIRxoD25eaN1qU/UU6tNPqY30OgUEQMezaR2sa0LHp6xsDQ6XmTeeQ5se/iWD3+/mZIGHtYo5g8tPk3rTtRfS3D7zcwrAg0HTUdYugtuoOUhtmlGPUZTaenFymh62Kb/8HfzRFJ6jN0QgqtRaMaY+k6ZpBTxC62G4PWQMejp+kMJ1+Y8oTAHdLv/p2/Dyaq46UR3mg8/M0wIg8dbD+j2mhRTQa0tC7eMtXiioRRu4GZmN8mUChihu0W/VBBb6jEAws3HGG0BhOZ/wlh+ImG0bg7eZ4ShyOsN2948+YfDLyINC9gcvg/BnhRerMPiywicPq8MQf44fhNd+EqBg7uBzC67ds/xCERoOna3YnUFq3aHwTRMjtjsSwMeE8Joz3rQkvxTIvf1A4H0YjWxHEswpB3K40ZxzYniyO84RvevHnDPyyHIM7Ir2st+AOTN2/eYDkBSZ7fTJOELgPzDdYoIE6X+9QShzHQYsahvKFWx/cVTdPu/tgyaLwPCZ+9iqkk/UU9JLHhcn9NAKJcXxOdTufL3+qKKIGo75w9TQbF2b7o1OuivuRfc+v43llf7K8KqIiNL1lo4vJwXZRKMxLyjjdEibpY3TkQF0KIi6Mtsbq6fZPAdr6/Kkqsicl8t6ONsqC+I+6r83G+2xSUmmJ973b2+UEHwcqa2NybTMU6F7tNRGnt4E7y1kFnOhnsevgagnpH7B5ObtSx2Cgjalv3pHmd74o6CKiJ+nXNiPLqrjh9SJr82Y5YAQEdcXj/2NfmJDheHIvNlbJYnazecHkk1ksIOpP74kislypi/madiu3rGhzlWwlwn4eXb2ryfHa8URH13TMxbVaC49F6SUBJrKx2bl0Xa/WygIqYnX96Ifaas66Zxyc4HnZKAlbE5sGxOD4e/x0dHoi9nU2xVq+LzvbhzGN4tlsXrO3PrQVxeTiuYQErYnJ1Lw8741oK9R1xayvO90SzvC7uVvi4L8lwnBBKfXdhDZqLvdVxkubMi+xQdGbuc5ngKElP4UGfIaIwApoMw+BWffjE1vn19xrtz/XJv5Zm4kW95ad7QH1+VW+zWoeG+mVcpdVCv/qFIIJ+9/b8FN6hat1xYWfbIZx4eYz837mqbN0qotYbGvzsE+b2gvrs120PXCVUW8at78S6VoNfM/Tu5KtVFV0rcRXHpPRuFa/O22RFARIFvTtZnKvT0ODnNAEWvLrdKNDaJiQ/8u73f2J0VaLAnF3E/eQKQtvkp8LkZLKsWhnnxTTbE/utyMiuVBpz97eOZTX54T//4NMvDt6oi6kCeUT0HuATvuORt83xcS58HF/HcrR5M5yQEEZXsLKNHw4njk3OqPWv/Fbu0p1ZojZOAE3TFJ5kj17nK1R0GupEq6mqRqs3pG/PPxmzNANVnZuvoHQteuVf+fnTe0ZOhH2dUDsuBQDejXBiG+f6lEoch8x0aS9bCpWHBO9gZbt953PSFwVBEEHTwZr5nUFFLX8kfZZGpSRJesBniJwk+QS11lRyW0YQvoemNefifRxFUZb/e8iMWzZhHGI1CrI4wBs5OE5AzqxENIAa+pwP1lWtAkV+q5GZIiugpKE9KKt8/PCe3m7Q0aszhi/bEKY2/d1Wuf5G/9AZNOgNHKJoh2YJPv72D4yef2+x/JNIR1g/fWCl178d1kQh0VUNY/I7eJaTo6IuOAGqpsVaCeB3HDcdL8J1iDsbrJXg6jeH68FkrkPYfmjNgHFeQrll3N7XRYD/B9T6FsacKVW1NLchq6Vd5yuUjD79dpv25z+jReOekzHPcsrKonEMrH4NgA+uQ1AARYDjqmxsrAAfcJ3PDR5FOC5Y1vLfHHPf43cqGN0FN5LUxQmqbDnmnIBiXIPnyfarJEm3PCBYyEhzoNG6ffNOPdw/SqxZ8y7e701G6PRo6AZ2BHrbpG93F7zbKVPVB79o2UPWCHCD6xtTETPyU5rDAe3Zk9yZ98ySgdIT1SR4aAB132waNuF1wPDhl79j9O9vsvhr5aHPOyq0px4caRjysWzQnjwJFYBicQyk9LB6ZQDeOQ4xMY6T0x+42GYFeIczioEEx0kwrQfWDMgzMkBv3S6pyT2X36hj9eeX4BTFFerCh/TD5WHAe5iTr3AP5d69R8OyqAN88nH8nNx3CFo2ztCiCXzyHLwcCn+Ip9uY2rIrUeB7v0GluyC3KWVkDmAY4LTmHZ3rJNYHlDJKkrS8B3yGUFAVKFdvhwTxyOFduYfTfcIK2qlP33KXfoNVDBtv4RtNQdhv8e+uxl4SfcnEzm/fWIpisnh/8QO30R9SDfr0gypKnlMdJoRP9TnmO6I0bMIwp2X8wLuf2hhqSDRsPT4YSVxGuUl/zuEat01gTCW65QTBO2jZtxNsVRWVnCyHRRFr2+pT+/kHPnxwGVoZgWoRN0CzTFZ++oH3I4ewreBiET30mXsdkKm3rosMd/QbpbWDBQ/NgjyH6tys/+WEQQTUac+uM7mQqqpc3ZeIWDWx12ze/HbFb46NTUR34KNWwepYvPn1NxwvJvdC2pa3fPPjhY/3G9S2+szc9UWCa/bwuz7hwiLMnDyvUH0dby6S9Oo8IFioolWn2vnPfQajnI7rTDRE8wQ0A3uoL13qjqot/r3wGf70gfLG6HaVrSwh5br3isJlMDIYWvfMCyAKSBouw5657Jp+34ov/zN5DJTWkMBN0N/8yrsf2rSrEYGl3w4YHnTQMty+S+Gac8dQFAUq2u1qnkVIEEFzeDvPA0VHL2dkGYu7IGn0seo/8M93n/jlx19Y2x+NS5R0C6v5A//5h0evp6APkodnESgaWhmKydoF0YBh3MRJegsemjl5BtUneap9zlcwMLTlp9Z1HaJxCcn8tVHpWj3Kv/3Mpz9+4qfKFifXSQk9y6T/60/8MWiT0sPvLn8zKHyP36ixfackJid2B1hOQmsYEMxsCWxyRjn5VRXZ/pskPY8HlSy0e6vkN93R5fj9PknPJ3rKUgUAVLRF2YGPpVSplu9+z0yinGoF8iKHPOfmu0NRUHx+Y51F00jabazcpKEqKCgoqorWaN1pi2Ha53UopvokKooCru4WChdFcb0+08OBfJw3oU4PvzN+cWv43DXMPlcXHD/QJp+c1Z6L62n8/ddP/P7PFl0lwu9PBAw306bczd0ryJOAgdnnx8zk9NZvtzXabcruZD5ITtC3+PWqxrYx/SRo0Gpd4UQpi5+WGqa1xj//8RuU17FukhKq9KwO1ptf+Zi3GZnLPGkMuu0STnZ9BIoIqxdgeNGdNgRuiwiTGo25xelLiDz8T1BqGw9KW52mthpUrIgYFn4+U9oWZu1nfvwAdcv6sizDol/7iR8+fETdtmaXDCxU4LsB1EwMEpI4J8tSkjAgiDKUdh83ch7WPkgcEpUb2NrSKyFJ0kM8rNLEqdjtdMTO4ZHY3+qI9Z2HdBL0fTk/2hKrlZpY2zkQx8dHYn93RxycXIqzvVVRLtfF+sa2OLo4E/ubHbG2UhGlUklU6muis74pDs7uzE3srZZEqVITtVpFVMpfOlEq1zfEzP5uLo/FznpHNGslUSqVRa3ZEeu7b4U4PxRbnTVRr4yHr6x2xMb+qRCn+2KjsyZWyl+Gbx6eC3GyJ9Y7q9fDa6LZ2RD7p2LG8HWxfXQhLo93xHqnKWqlkiiVKqK+1hG7d6qXnYmj3S3RmegMqtzcENs3HUldiLcHu2Kz+aWjKSiJlc6W2Dv+/+5MS6kiaisrYmVlRazUKtfV7647mbrVh8OsqpMX4mizKVY3D8TR0YHY3d4RO+srgtKGOJ6xW892m4LOwf0nwOWRWC8janfq1h2LzQqisjlr7hPLmdU3xPmBWF/bEodHB2J7vSO2Zh74KSfbolbeFHdrVT686uTZ4ZZY73REs1YWpVJJlCp1sdZZFzvHy3bwcCw2yvOqIt52tlsXlNbEwZ3uLlYFpelOs6bNqr54KvbWKl86iipXRK2+KjqbO2Lv8OROleD7nO8253SsJqtOStJTWKojqYuzU3H+fF3TvYBLcX52Ik5OzsTFxHZcXlws0ePeidhprorNw7OpaS7FxdmR2Fkti9La/tI3u7+m+R1JXZ6fitOzCyHEhdhfZX57AOf7YrXcEXP6T3oy8zuSuhRnp2cPPt6n2yszApbxfL6mI6nHertZESs7z90j7HM/sM/FXvNumxkvs2xJ+mtYoiMpUDWde0rZv3MKVa1Bo6Hdqm6nLKhrfkfgMFB6DLva1DQKqtbGdkzUKJT9YXwlpaqjayoQEURQbxuzj1G1h92OcJZqnvIpKWi69sDEvpChq2I/onrhc2lZNsrI4QkblHx5iYOT97GXbuBBkqSHWipYkAC9RSPxCeY8m9IwojDaj/h+K80U+YRXFYy5OQkKbWcIzvC7D9Ayd0jcG92T0/DCdItRN2LwzYKtr1UQDHxazuBReRuSJD2MDBaWpfUJ/Da+2aXv+IRxQpomxKHHsN+mH5tEXu+ZevErSEMP1w1I/uxtz2QBQ6tP23T5SEHkWAyDdPa4VRPPTun3w+Vr0ryUdITpNnCH398jrTV0aTjmTeNUr0nmmwzUESNZqiBJz+qBvU5Kk9SWhRdY5ElEnCZECahVnd4wYEHrul8pJ+h38QwXpx1gtkc4Yf+FmmB+TimebZOqVdq29aWL5aqBaTfoD4bAda0QdX5thWrPY5SaWL7OqPud1Z8rYoZ2jOWNvknPnfdSGgz9PqY1pOHaz7OOzzHP1MXy23jurE9UMa7tkZAQ/wmuEkn61v5FCCG+9UpI94uHLczMIXZaQIbTMCi8BPuV98ebRj5xBqCit417uoS+X5ZmVLVnCBaylFTVHtHNN5CnpNzfFHiepqDdzX+YN/zJPXA9l1eQpjnaEx+XIkspqvP2S0bkR+PqvIqO0daff/9J0p+YDBZeg2xES/OxsuC634KUof5/EzsC72HtS0uSJEnSo8mchVcgGg75wzD50gZWQpxWqMpXJUmSJOkFyJyF717IyP1AzUhxR6PxoNQnvNIZvvJPEJIkSdLrIIOF712WkHxaoWebfO6nKnUcPta6U12GS5IkSdLzkMHC9y6JSUothq3qdWc/OX74nlXbWth3kiRJkiQ9FRksfPcKCq36pVfAZISTrOMs1emRJEmSJD2eTHD83jUMWje9SGaMLJ924CLboJEkSZJeiqw6+d0riOweI61LIwpIukNGn5MXJEmSJOkFyGDhlSjyjEKtyoZlJEmSpBcngwVJkiRJkhaSOQuSJEmSJC0kgwVJkiRJkhaSwYIkSZIkSQvJYEGSJEmSpIVksCBJkiRJ0kIyWJAkSZIkaSEZLEiSJEmStJAMFiRJkiRJWkgGC5IkvRpF4mKP4meZd+b2sMLHThwwdELyp1whSfqOyGBBkqTXIfPo9RN6ZuNZZl9kCUlWLBghxbf7WMMhtmXhJRO/VduY2oieHbNgDpL0askuqqWvVkQOppOgNVTQegx6DWSnmNLTShj2RrTdkMa3OLmKBMdoE1oRfq8KiY3eG9KIbfTrUapdFzsw6AchruwWVvqTkSUL0tfJfHpdl9ZwRDf3cd70cNJvvVLSn03m9nH1IX3tWyy9IOgbDKpD3F51PEjTqb5z8dPJ8RSMgUliWUSyeEH6k5HBgvRV4qHFrw0LU7seUNbQZNeY0pOKGNg5fbv1TZZehBbmzwr9Qe9Lr6+KgsJ7oun0iarJoOFjezJ7QfpzkcGC9HiFz2D0gdVeFxVoDROKPKAngwXpCRWBg6uZ9LRvsfQcb/ATH5s21mSqRJaRAUU+XYSg0O4ZRI5L+nIrKUnPTuYsSI9WBC7B1QoDQ0YH0uNlcUCUKuiGga4CWUpW1bgu8Cf0Aqot++bfnxWxjxfn12/5BUUBaqNHWwnxohxFAYqCApVWr4uuQBZ5BAkoChSFcjN8rtTF+R1W93u3l5/EJICuzpjYaNN6NyLILPrTKy1Jr5QsWZAeLfACrsoGhvat10R6lYqYYbtFP1TQGyrxwMINRxitAV8qGiSE4Scaxt0aEEWeEocjrDdvePPmHwy8iDQvYHL4PwZ4UXpTpbHIIgKnzxtzgB+mZPesYuw4vKNOt307IE6iiCtqaNqMidQWrdofBNFSe0OSvmuyZEF6pBA/uALD4Nt8SZZetTyk3+qSWDHhddaiPmzTU/+D39sHE+dUSppVZj6UVcPCMdoo0f/hh/eQq236hopCH6dlXA/PUbt9WtcFAFrXoe97JGaAb82Y6S0xrvcBAK/fJZz4JYveQ2mdGTEMUEWrQpCkwH3LkKTXQZYsSI8T+QSfoG7IUEFaVkFom/xUmDiT1RuUggJoto0vVW+LjOxKpTr3S5eOZTUB+PSLw01eYR4RvQf4hO94XxpLKnwcX8e6ychdIIuIPkJ50yXyffzPf55JNYdSu4sxc0IVtQppmt6/DEl6JWSwID1KGoZ8pEyroX3rVZFem3SE9dMHVnp9br2YRyHRVQ2jNfGhP8vJUZmVGvBZ1bRYKwH8juOm40W4DnFng7USXP3mcD2YzHUI2/bDknCv8xJa0wFx5BNelelZvbntiahqiaKQ9SelPw8ZLEiPkBMG74AGLVmwIC0pD33eUaHd1W8NT8OQj2WD9mQEoQAUi1tFVHpYvTIA7xyHmBjHyekPXGyzArzDGcVAguMkmFb7YY2G5RmfqNGYagUqGPl8qplYxvxJi+IKVZGJv9KfhwwWpEeIxslbtca3aU1PetXiKAZaU9/7c4LgHbTat3NgVBWVnOyeZgvaVp8awAeXoTXEVS36DTAskxXgw8ghDB1cLKyHBriKwuf8gy+r6eH4n2haFvMbnS7Ic6hqsiqE9OchgwVpeVFAeAU0WgtumJI0m6IoUNHQJgPNIiSIpvIVABQdvZyR3VdtodHHqgN84pcff6FlmePUQt3CagKfPHo9D/3z8IeoatRQxjHDtWQ0JNS2cRcmR+bkGVSrMliQ/jxksCAtLY1CPgIrLRkqSMtrtNuUi3yih8acoG/x61UNw5h+wDZota6Io/SeuWqY1tr4f8vrWDdJCVV6VocSV3zMDSxziQd4o0u7MhGoJEN6jsooGKIvnDAiTGo0WrLYTfrzkMGCtKSCKHgHQKOx+Jb5lDLfxgmfJ2Es6Pd4bOu88cjGS590df70lLaD10sY9D2CwMOxR8SFAqWpfIXx2BjtJu+i+xstUHsW62WomRaT/TipPQuzAhXTorvU87vF0DUIhy6B72DaKXYYcG9FijgiVtp0X+7ykKRn9y9CCPGtV0JaXpFlFNUqL59CFdFX/x9++lRn9yzm3qrqTyAPTLq+STAynqE3ywLXaJF7Mda8l84sZDjwyDWNIlMwh9ZErkbMsD1Edz269720Jh6WE5DlOXm1j+88MNHuT6rIEtKiiq6Ba/wr/1D2uQzMu/skczF0Hyvzl3zYLycdNuhrEUFvaiFFRpIp6A/s9CSxddqFS+q8tuzfnDgIyBs97hTwSH95smThNYostP/9v6l2/cVZ4s8hDQk/ASUdXXuB5WUuvUEVx3mOQOEBUo92y0a1RwxtG1v36NqTb7kN7JHBqOfc3xeAZtDvd1GiX/ktTl/+2H1nlKp+/QAeJ8zWp/MVPqv2sNsRjndf4sIzUaoPDhQgZOiq2A/OovweZITDHrr6r/zbf/RlSZk0kwwWXiNVRyuX0bTqiz9AiyjkPbxQcmOBbw1Q7cE3qnURY7VNctu76Rq5qmlknsutQnGtj607mO49DzOlit7o0pWpHrdFPuFVBWNuu+EKbWcIzpDpTh6/N5k7JO6NvlFX2o9VxbA9Rt3yt14R6Tsmg4XXSO8T5TnJNyjmjMLxY7LcaNzp2OfJJQ521MV+zrLnBWK7x49Zj+GtVgYV+BgTT+U4GLZFbg+4/8s6/KW/PUzKAoZWn7bp8pGCyLEYBunscasmnp3S74ffb4lMOsJ0G7jD1xkNqvObyZQk2TeEtIyUKPoEQOMFakJEI4eiF3yb6plFwHD0npW+d6tJ3yLLuKKgyOFWwojWpac2GAUOrbaMBh6kamDaDfqDIcC4xUN1fgha7XmMUhPL1xndmyDywoqYoR1jeSPZ9oj0p/QXCxZyksAnTHKo6rSMNo1b95yCLElIs4wsU2j0DLQ8IQxT1JZBo3r7LpCnEVGcUqgNDENHzWN8LyRFp9tvT9XnzknjhCzPyfIqrW6DapEShRFprqIbE/PPU6IoJqOK3mqNu+29WcWMJEnJsoxc0eka+s3wOEnJs4xMbdBraZDFBFFCoeo0Wo3b9dofo4iJ3gFUaDS+9i0kJwlDkkKjZTSoKpClKaqmXb94x3h+jjG6GyokgUuUXdfXLwoKFPRuj2rsEqbKdffDBSg63V4LlZzY94mL666MadDtNRYmh+aewy+fVtjp315+EieAhnpnYg3DUBj6EbSNx++VJCKMItKiSqNl0GrM+dRUZMRRwhvwHwAAIABJREFURJIpaIZBq6qQhS5elFNtm/S++vi8BGXJtggUGrbHIH2e3AXN9Bktald6kUKlNxrx4NSG70Aajhh5EVkBasOkPVVk8xLXmfSKiL+Ky7diZ31T7L89F5dCiIuTfdFZWRHrB6cTI52Lo50tsV4vC1gT+ycHYmvnQBxu1QWlNXFw8WW8w826qHd2xdHJiTje3xD1labobB2Ks8szsVOvie3T6eWfiP2tTbFaQ1DeFMdnh2JnZ18cn56J0+MdsVquiY2jC3F5si+2dw/F29MzcXq0JerlFbFzMrmKx2J3qyNWSghW98TNKp0eiu3N6+GdA3FxvCu2947EydmZeLvXEZVKRxycf+U+fLslKiBgTRxcPn42l6f7otNcEztHp+Ls9Ehsb+6J44N1UesciJvZnu+JJk2xd3F3+rf7W2JztSYAARXR3NgRR+dCnOxvic216+GVptjYORLjTb4QRzvrollBlGprYmP783AhhLgU+6t1sXtr35yJ3TqC2pY4ubXkC7G3iqC0IY5nbddBR7CyLaYP/bTD9aljJ4QQl2fiYKMpVrcOxen5pbi8OBXHOx3R7OyJk6l9cHmyK9ZWVsXWwVtxcnIkttdWRH1tQ+y+vRCXR+uiPLkfJemOC/F2uylqqzs359b50aaolxBQFptvx8Oe9jqTXrvvK1g42xedel3Ul/xrbh3fO+uL/VUBFbFx/OU2enm4LkqlptibPqOPN0SJuljdORAXQoiLoy2xurotPk96vr8qSlMPzKONsqC+c++D4ny3KSg1xfre8a0b+kEHwcqa2Ny7FRmI3SaitHb35n/QmfHAEUIcrCGod8Tu4eRGHYuNMqK2dSK+xvlec3yDeMADcZ7Lkx1RL6+I7ZOJLboOQpqTB+JkS1RK6+Jo/sqIJghArO5P7IWLz8NXxf6tnXMqtldq4u4umBEsnO2K+vWNsNPpTPw1Ra2EYHX/zn4XQghxvCnKLFjna3eDhUtxtF4RlY2jO8f57VZNlNb2J266b8VmBVHbntiQ811RpyzWj2SIIN3v8nhDVLh73ztaL90KFoQQT3idSa/d9xUsCCEuLy+X/3vIjN/uiNX6qti99SwePxQ6h9PjbooyiM6c1+fDDoLK7bfOs926YM6b8KSLvXHQMn0xnWzVBEy/4V5fwDMezjPfTj+v252H7Pn4TblzsHjl7jG+mSBK69M77KFOxfYKojz9UHy7KcpMlcYcrQsqm+KtmOdSHHTG60N9R5x9Hnq0LsrXN7f67tmX0Y83RWVivMn5TAcLlwcdAYjO4dTxPx6fF8070eXnzdsWtQecA3eO3cmWqFAWG8czRj7ZFrXJQOB0R6yAWNufXLcjsV5ClDdnzUCSJl3fC+q7d976T7Zrd4OFJ7vOpNfuu8tZUJRnyg5q2YSxDRRkcUgYJWRpQA4UecHdFPUauj57XapaBcLJ5mqhyAoo6Q/8Zqkz3bPzeLt19OqM4cumf2uNqeZor9u3/6o08oQwvgJAf2RyY+5aOO/LmKPbjRGlYcSnskF7YqWLLAdFW/C9U6Fn9ej/+jOf3o1wYhunkeM5Aa2NdeKffxn3QGg5NCjwHY+GlT6oX4A0ToA6xlRzvXEQ8Km0htWb851draJSsGzPxGkQ8hHtzjkBgK6j8YkwiKHdAq1KFciLHG7qoxQUV6Bps2Zw23/913/x448/LreC0qu2ubnJ//zP/1z/KyJ8B3SqD6zN9HzXmfS6fHfBwvPJCB0La5TSsAZYbZOuquD/8Nuc8ZUZSWxjLXvImmvjBgVGW4EiZuSnNIc+7Qety+3OaW6UlKepVac80XwmFTHxe4ASjZlPtfuFQchVqUvXmBx63d11y77d2+BDuiY2LMzaz/z44QOuE+AMEpyoi5PZBNEv/PDexQkdXN1lGLYZeA9LtcryDMrdqeTXGNf7QLk3ojdvNkVBgcqyOXJpkgL67HPi+lh+TFKgBUqP4dYQw/VI+xYakHkuYWUd7wGV+//7v/8b27aXW0HpVfvb3/725R9pQgKQj6+tB52qz3SdSa/L9xUspD59y2XZXGfFsPEWtphWEPZb/LursZdEXxpMyW9fKkXBxA178QO30R9SDfr0gypKnlMdJoRdbcEUr1wcXTeI06D1qDbvE6L4ChrG7aCAED+C5vB2632KqlLKb5fe3NXAspr8+M8/+OQ5WCTkpo+h6OhWkx/+8w88x8NqOaQ9l4fWaFQUBapTb16hg/uhRt9aEA7mGTlVlq2urjd0+DknnywsuJnneB/U9M87XYFqm2E/Y2BaVNWCjDZB0qf1gOX+7W9/u/3wkP5ars/r92lMAg+slvw815n0unxfwYJmYA/15UvLVW3x74XP8KcPlDemWlbLElIYF9kXLoORwfAhnR1EAUnDZdgzl13TVyuPYz4CVFq0HlXFXUFVoKRNfVoIA8KrGv3pxui1BtqnkOye1x/NtFiz3/Db1W/8+HOd3bPx7a9q2nSsv/PrrxbtSMUKH96AlaZpoKgTi80YDTzUzYDBortrlpNXHlq8+0W13WXln/8vQZRj61NP/CgkpoLZ+7zgBD+EXjDEMpdckCQpDRo1+P2Dxyh0GBkTvxXj/8z6jPYc15n0unxnLTiqaLqOvuxf9Z5QVqlSLV/XCZ6QRDnVyvX33zzn5rtDUVCQk817rdU0EquNNXJxPQ/P8/GDkDi7P8wZr8PdC7IoCri6W+xeFMX1+kwP5+at887wO+MXc4Y/XBxdN7TbMh7ZSJJG26hxledf1qGIsPo/8WlWb4N6g0Yp5vNi51K7WJ+bqV21vvQIqHSxemXgIx81C3OJ0pBqt8tKlt6UcGWuyaAY4N/TP0USR1y1Wvd0XzzjGOk23k6daDggmjxARcJw4FPd9PjSWKdGQ/EwzSGu6+J5Hr7vE0TJPaUwz6yIcWzv/v4xlp1t6DD0X7pPiJxwOCR48sVG2N3Rk+8jWGY/NRgMO5T4yE+9LqPrpkiL1GPofwCuiFyPKJ06m57hOpNel++rZOHZGAy9LVLToj0ssFsqaRyjGA7eIKFht+lFbUwnx+2beGEEpRzXbJPoGr3hiJ42MbuGhaVrWIOUqlJQ5DkfP42T/8r1DVzf5c4XiSJkaDr4UUyppDBodwl6Nl4vw+qPCKOMUinAMrq0zCFuK8K0PaIQSoVLz0gxrBEjzac38IjDEqViRK+b0B+6mMVoPDz6PDzG6I8YKCNMxydKSpSwabV9zIHPcv3cZMTJuOXGpvH4N4fG0GOnbdF3VXpKQhhDoQJGm7tzNegaH3HijMVFGQpt26T2i4th926VWhi2zcrPQxq2udzbvmbhml2cUYCJzyhqEwTWPS3zFcThe1a7CwKKeDRx7By63Yiu5WIbCg07Iq5a9Ns9Wr0uDWICP6LohcRWa2KeCl27i20MsUMVpcjJPn7iCqBUY23o4d8a/x5FjDvwxt+xZ1Gr6Pq4AbPpQo8vEpyujTIKnjyxTTH6tPpd+qrHyLjnO8uTbEtBZHUZtTy8OSdNkcWEYUSSFRRFRo5KVVFRVZWq3qDV0uck5o4bZnuOoG6Z/aT2PKLcpGf9wn/+279ilcpoXYdBt4bvVWlUc/I7Z9AzXGfS6/Ktq2O8rEtxfnYiTk7OxMVEzbPLi4slGrE5ETvNVbF5eDY1zaW4ODsSO6tlUVqbUw//1ToWG6Vxwyxb8+syPtjF2ak4uxDic5W/W1WvJlwerovy6t4zN+wyq1Gm61/OT8Xp+QPPjMtD0SlP1zl/nIuzU3F6NntG54frot7ZFSfT63t5Lk4ONsTKjGq5i12K89NTcbLXESUQpbVdcXJ6Ks7OzsTp6ak4eXsk9rdWRaVUE539Wa1rXIq3W03R+eoWvxY5FdvNtQc0Kva12yLExdG6aG4cz7gfXIrTgy2x1qyLZnNV1CvjBozqax2xttYUK5Xr6oUgKK+I9Z3jGeftkVi/09DXU3rofvrsUpyfnYqz641d7j4o/dX8xYKFJ3C0IUoz2je4cbIlKuXZLfy9Wp8bKaIjHtvCwkxvt0Rl4cPtrdiqrYidx7YA9SDzg4VlnO81RWXmQ+YpnYnd+nQ9+EmXYn9tQTsQC5zvrQpArE+3LXHtaL0soCa2p4/V6Y6ov0C9+svDjqitHz5o/z56W8RbsVmbEfBdnoq9tbro7L29vu6vW/Kcuh4uz0/E4e66WCmPg4baxuFUwPDcwcJy+0mSlvGd5Sy8AnqLRuLP/Z6ZhhHFzGL1VyyJxkW7zemaDF852zDg46x8hRstBqMG3jD4fnsaBCBmOFJx7slp+HoajYZC6MWz90ceEsQ1jEdkoEZBBKzSNua1LaICHwjDyRO/wLeHVK3+s9erV7oDupGNk94/7uO2BbKRjdeyp6rGFgSmgVVYuP3WdfF7SpICNf1WfopSbdC1POJoj9UyfPi5h+m9bCbJMvtJkpYhg4VlaX0Cv41vduk7PmGckKYJcegx7LfpxyaR13umh0ZBGnq4bkDygk/PNE64Aiqt1pN8k8zDEVa/S2/wHogZWTbenA/NanvEkAGm/03T9xYoCPt9soE7v/2FJ2S4EYPCpmvauEFEkqYkcUQwsuj1RmhuxPI9JEcE4RWstJn9ubsgiT8AU90Y5x5O0MJ8iQ2nQa+dM3Luy3h95LaQ4o5iuv321LWbkrdGxIE58Z0+I/sANBozgyRF7+MOm8AVv3nhfRv2xB66nyRpOX+RBMenpbYsvMAiTyLiNCFKQK3q9IYB9rPdN3OCfhfPcHHaAWZ7hBM+/xsdQDzuahLjK5IbJ6mtHrbe47pnYoqiQJkbhai0XY+s18dteF+ysL8TmW8x0ke4L9ZlskZvFNArMuI4JolSCqWKZti4/Tk9VN4nCQk/QaVnzD6fch83BGqbDLpfllAEHr/rbbzpheYRnp9QXDcoVRTFuDvqVo7nx3xuTnQ82KStAWmIG6YoikJRFGiGiTG1Mq22QWZ7JEy3UPr120Lm471rYd85xXV61tTS0pgYWGk15u5vzeiywh+8zzOmezP/si7fcD9J0rK+9XcQ6WFOdpqifpNdeC5268/9Lf+zcX8OzOh45mVdiLOHJhsuO+ezs0cnpJ6fvf5+9cbf+EtidpcfZ+KgUxHM6LX0eKM8uz+K8yOxvbV+3YshgpWO2Nw/EeL8SOxsrYv69Tf9lc6m2P/8Af9kX2x0VkSJsqivTwy/vaJzeyL92m15aI+hN+NSEhuLegy72BerIEq3RprKWfiG+0mSliVLFl6DbER/oGJnn197CoriPXEKz//qkJC85ysaY3oqKtozLV99QJ8K81Sfa6Ve0Pgbv4ZahITheFiRZ6RJiO9FKO0Rp16X212lFKTZJ7S2dneG1TZDp027CPn3nz5CXsU0x99GbKdNqwj4958+UWgm5udPJg2TUc/HL2xCz5z9Jl6tUiUlSWBe8szjtgWyLAWt+6CSujiKAIO2sWCkLCMDGov6UfmG+0mSliVzFl6BaDjkD8Oke3NnSIjTytLNCj/KdTPPJeOxjTFJ37frb/wVnYZaUBTjP1SNVm9IEMf4w7sPV8jIMlDndaACGJZJDeCjhxN8TrJJiaJxmx0fXIebwWS4Tkjb6i3oPExFLX8kndv20GO3BbI0A1V9wGeclDD4CM0uc/Inx/MLfd6XOtjm/cHky+8nSVqeLFn47oWM3A/UjBR3NBoPSn3CK53hC3yQzOOID8Dawtco6dW6/sZfavfptx/WDdpYPg4WlAURq25hNX/gn398wnc88raJGjuM8nU2Vn/h5999HD+n3VMhcXFSE29hxwIqqgpZMacN8EdvC+RZTnnRtnyW+njvYWXHWJDsG+M4MfVBTPf+6OPl95MkPYIsWfjeZQnJpxV6tkm326Xb7dIg5WOt9aCOg75WHMZAnfZ9redJr1IeBrwHjPYjyquV+/oFrWJaa5SAq98c3BQCx6VqDRlZHUpc8Zsz7jgudBwUq39P6dU42U+d0439827LWOq7vGMFszcvUi+IbRNXGxHYD43mX3Y/SdJjyGDhe5fEJKUW7VaVarVKtaqMmxW2rRfIdL4uCq0YdzKupT+HMIh4XDCoUlUhzxdXaVV7Nr0ywDucoYXjt7BMDaVr0asAfzg4kY/j6Vj3diyQk+cVqnNe6R+/LePPKVf3bAtk+O47qPfn1MrJiZ0uZmIR+ss1ffyS+0mSHkMGC9+9gkKb6MkwGeEk6w/6Fvr1i44I30O53ZX5Cn9Kn7/xPyYY1NC162/9CxlYZg2ADz/9SNS2rnNvDGxzBfjAqNsnNCx6953SRU5+VZ2T6Po12wK6rt8kJc6V+XjvoNnv3QkEsmiE2WozYEDomzPzIhZ7qf0kSY8jg4XvXcOgddNDYcbI8mkHL9RnfBwSUaLdM15gYdKLizz8T49PXm0YdT5Ec1qTnBzPsqgDUMG0vjR6pFsWTeDTx4Ke1b3/63ocEpUbNLQZv33ltqitBpXrZN55Ms/lD+oYjZwkjonCAM+xMNsG5qjA9CN8q7Ug8XCxF9lPkvRY37rupnSfS/F2uyM29vbF7sa62Dw8e7Eln26vCErr4kg2NP+ncna4JdY7HdGslUWpVBKlSl2sddbFzvGSFfNPtkXtQf2gXIj9tZKgvjvVh8SFOOiUBSs7D2rf4Hy3KUqd2/0ePNm2iGOxUa6J7Zkrcine7jRF+bqjqFKpLCq1ulhd3xI7+0fi9MGLuq9viOfbT5L0tf5FCCG+dcAi3a/IMwq1+ui3luUl2Pr/YdQ6JneNF1uq9JokDHWD1M0YPXt9/oxRSyccZPfUBHi8qF/F1EKSBycmLiugpwXYqfOMn/Wefz9Jf03yM8QroTx3oBCP6LZ7uJ/7aIhdvPd1BgPjOZcqvWo6pt3Ad/zn7+grcXDyPvYzPgBblo0ycoiebQkv4AX2k/TXJIMFCYA0GPHrb7/gxTnj3AgHZWtEX/vWayZ9z6rmCCsdMkqfcykFwcCn5QyeN9FWtxh1Iwbea23N6IX2k/SXJIMFCQDNHLDZ2aCrRLj9Hq7mEjgt2aSLdA8N2zXxzSFzOg79aplvMlBHjF7gbbk1dGk4Jm767It6ci+5n6S/HpmzIE0oyJKEXNXRq/KGIz1cHlqYfhfPMZ42wExdegMYuuaL9LAKQOZjWgmWa9N40o0J6OkBdvIMOQvfYj9JfykyWJAk6UkUaUqhaU+aW1NkKUX1aef5IHlKiob2pAsuSNMc7RkaQPhm+0n6y5DBgiRJkiRJC8mcBUmSJEmSFpLBgiRJkiRJC8lgQZIkSZKkhWSwIEmSJEnSQjJYkCRJkiRpIRksSJIkSZK0kAwWJEmSJElaSAYLkiRJkiQtJIMFSZIkSZIWksGC9P2ILLRqFU3T0PrBV84sJxwOCZ6lA8EIuzsifeTU8cjGe+zEiUPXjh8+/IVkvo0TPnVH1SmePSKeM9t42BqfK1WVrvfsnWRLD5Q4LdSqhqZp9Lz8W6/Os3ie8x2Cfo/n2WVffy39r+dYLen5FVlGUa0+WVvwRTH7BFEUZcHvCopy/7RLrAVK1yMZGUtOd3c+kdVl1PLwnr4ZfiAnjRMWXtNZyHDgkWsaRaZgDq2bToka/R5Bu4fvenSXXb88Jc5mREDzht8jDYaMwnlbolLVNLRGi3ZLm9tBVB6Y9AKTYPTUnY9p9PrQ7Y1w/P6dDpIadkRqQzps0H/iJT9E6g9xwofucwW9a9M3/gK9NxQ5xijF717/O/GwnIAsz8mrfXyn/c16s32R871IiYKQOM0pioIMqCoqiqqgVnVardacPkeuO/J7lrj3Ca4lIb0+b7dEBUSpcygun2SGZ+J4b0ds1MsCEICorG6Knb1jcS6EEOJcvD3YE+sr498o18X69q44eHsuhDgXx3s7Yn2ldHva/c/TLrNdm2Jl8/irt+biaF00N46faN/MciTWa1viZN7PZwdirdYUe2fjf57vNUVt6+3UOHtibXVXnC276LdborZx9PDh97i8OBOnp8diq46AFbF5dCpOT8/E2dmpOD15K44PdsX6SkmUm1vi+GLGDM73xVpzW5w8384WF4frojm9/yac7dTF2sEzrsAclxen4uR4R6zWN8XR6ak4PdoS9ea2OD45EScnb8XR4b7Y3VoTtUpdbO1uiOYTnNuvwenOiugcTgy4PBenJ4dio4ZgdU/MOo1eynOe7xcn+2JzrSnqzaZYXRnfSyvNjuisrYrmSlmUru+PUBHNjT3x9s78L8X+al3sLn3jfLivuZZksPAane6JZrksVhYc9Ec52RI1EDDrhD0Xe01EaWVTHM06md9uigoIaIq9x57sTxIsvBWbtVWx/6x3pEXBwonYWimJ5udIQQghjtZFqbIppo/W8WZNrO4vubOeOFi4nlhslhGs7MwOXs52RR1EuXMwdaO/FIfrNbF++NwP6lOxXa+LndPZv36rYGHsVGyvXh/bt5uiuT1jJc8PxXqtJCp/1WDh2mHn2wcLY099vl+Ktzuror6+L06vJzjdWRFQFpuTF/3FmXh7sC3WauMXq1J9aypgeP5g4WuuJZmz8BrpfaI8J3FaTztf9XMXtyrqrWKygshuYxfbhNGI9qyi88lpn6Xo/2GykY3Xsul9o9Le2O7xY9Zj2Ne+DFQU+BgTT5V+GrZFbg+IXnQNZ0hCwk9QMYw7xZMAaFWqwKcg4FZWROJgR13s7nMXKuvYVpXhwOd7z0xQbp13OWlWQLWLO2x/q1X6fnyrbw/Tnvh8z9wuxrDK0DXRr49/mqSATmNyAapGqzckSCK26yWu3v1I2/ra3KxlPf5aksGC9IXy5X++XA45od2iG7YJoiGteQ/hmdO+tBR3FNPtf6NvokXAcPSelb6FMTk4y7iioJj+VKp16akeo+DbPgKzMOA9JYx2Y/YIcUwCUFVv5chEI4eiZzJnqieldns0/BH+a8qXi4e0rRAApWvjdPVvuz4S8NTne0FKFz92aStfhmXpFZR19FkvTkqDoWtRAz75/ou/LDz2WvqLJTjmJIFPmORQ1WkZbRq3DuY4wSTNMrJModEz0PKEMExRWwaN6u1HUJ5GRHFKoTYwDB01j/G9kBSdbr89FbWOk+KyPCfLq7S6DapFShRGpLmKbkzMP0+JopiMKnqrdROtjlcxI0lSsiwjV3S6hn4zPE5S8iwjUxv0WhpkMUGUUKg6jVYDbeknaIZvGphJlzAc3iTofTN5ShjGFNUGRktDISdNQfucLZT5eO9a2HcKXDJCNyC9lZCp0TZ1UjcgUcYBTlEUoLUxjSoUCYEXkSkKSlFQ6G3M1uIik9xz+OXTCjv927eTJE4Abaq0BkDDMBSGfgRt45E75etFQQS0aBuzD3Ds+XygzOrAmrhRxnh+jjG6e8NNApcou05wLQoKFPRuj2rsEqYTx0DR6fZaqOTEvk9cKCgUFDTo9hq3k3dVg7b2D7ywoPfsJRlPIw0C0s9ho9KiJwsX5iqyhCiKiFPQGq3x/XT2iMRRRJIpaIZBq6qQhS5elFNtm/Qa9xcpPu35rtAyp1MCY6IEaLTmB9KNLu3yD/z0KWNRiuz3dC39dUoWiohhzyZS25iWRU/PGBg6PS+ZGCkn9l0cq8ff3wwJYw9rFJOHNv+mdSeqtGT4/QaGFYGmo6ZDDL1Fd5DSMLsUoz6jZHr5KaHrYpv/wd/NEUnqM3RCCq1Foxpj6TpmkFPELrYbg9ZAx6On6Qwny8LyhMAd0u/+nb8Pwi8Z+WmENxoPfzOMyEMH28+oNlpUkwEtrYu3TLJ8kTBqNzAT87sIFFLfpNV1yLQGWuFjDQP+//bOHzxRrVvj762gk046uZV00smt9Kukk1NpKvmqeKvwVaG0i6eSU8WpwqnCqcJUYaowVZjbhKniVCGVpJJUkmrdQk38H01iJnMOv+fheRKULcLauPbe71rLbcrQpkbliefhe15acq4xosCH3d7D3t4e/q2bcIMIMRJEPR9OS8Pe3h5000Uvjh+P6XkWWtoe9K6LIHzODQ9hmV+AnAJFmP1s378FGGGpAloqy7j3XMyby/vhw/UegLyCZUL92DegmSEKhy5cTXh6IfLh30qQl6yExVEA32phb28Pe/820PVCxAkQhwF82xjdA6MLL4wfp0KjnouuvgfNsOEF0ZIpUgGynIHn/vRFm+dJIgS2hrqdvFm00t+XBEG3jrLmAKIKvalAjC2osga7N2sFSWBCkeqwIh6iCDiaBEnRYLMqmpKPZsvdYGr97e19gShAcAfkJHnN/ech8ABywvKlkMn5fKS+tEMlxfbcnFC1UKDCllvx4OLZpgcnJQKy1Lh4Em8Mz2rEMEsEeRcNYlCg0tFI4DI4P6BS6ZAmh/ZPSsSgQtM6kPNGhlA4ohW6kUf6nSKBKVLteFatf1oFIV+h/eNp2VyfOkUQUzldUPafrhALnVZAKFSpczb9pS6okQHlDlbq9ycnRwWAgBwVxpERmVKHrjfRjt0cUR4goEpLtE2bsULgeH1coUyuQVO3jm6OCgTkaFpPdtMpEConK6MghmfVsSI5T0c3s/sBEApzgqf+MRUzNTpfaHCJwHEsikK2SNVqdWorUo4BoXSyXNh1sU8Z1GhjaeJbCxyvR/eNqRzRxcXFeDun05MOHdZKVCg26PhyieLq6oCyzJrz7h9Tcaz+Lk2rTQeT/fMi1Gs6zOdonYleH+YJpcX7+5EEjtlsjnIZhsCU6PiiQ6XaS0Wncwzf6Pu9VTtrWClwrC0+s/onFWLyi2Lh4XmNMtkanT+++ZL2s6Dc4dQ7+x0qIEO1xQ667uR2Y+/TnNeIAUPru+NIZMlUp5/tKwSOH6QvfayZBUGD7fvwt9w8s/xs05yooFQQIXFPw05WliE+fMOCg8WyYPEdnKCCA8ApJjyvjcmsle98xUNWhDg1ghUlAfjuYGUI72PTLPAQgpfLM+vqopADfkQQ1emJKx6iwOAhDBYSAK1KYcCyAHosxJkA/pHQ5jbcdPyaQFA0FBjg/uu6PNL5AAARaUlEQVR/UFatFycgejVRF5ruQW6bmJ41TJAAmTKUqWXgKIwAjlupV2BVHfUMAPxA13y66YE3nrr53oU5NYvTM01Emj61FrmaxPfwHUC168FxnKdNlxA/AMW6snyUwXPgEOLZiYsdEXsufgCQRBFJkow3gJcU6JaHwLfQXLb8EkW447jVIydeg15lAABfzacEVonvjWdRvsK0wqf3eyYstgl9jQCCEzgg6v08W9wAUXcRxjH6Xh1uszUrkHspsQfTdNdOV2+Kp6swVzUUB3AsC44/9YYkQNew0Ht+yL49iQtd/wJeXdS9sEoTavIXtNa4n/Y8eHeAKE51eF6EyNzDdTYfIe/M3qfwHQ8PUKCW17yp58C9z0LT68/rqz5IX/pYzgJGP6Zbb5s0LBvwAg+6lCAKXNhdE6bpIgaQLM2CkYMoLm+ZF7JAEs8k5UmiZOVU8yJzKllMEhgtCmJGa1WbtDmFIGFWSjVa29q8HQn1lgnfP0KRAe6+/BvluvMmD6vtSODqBr5BQVOdvrAxfPcHICuYnhWMoxgZdt0NKENv5gAAt5YJNwGQuDAtDo1GHsAtLHOiTvZhWoCubxZxEgY9AAWU5VmbCVwX90wFen2F3oHjwWH0wPoZjKYiC6jrKhRFedzKkgh+TcdKohhg1z08WdT1OjLAlBMWwzZdyI0asgC+m+b4xzSBY9qQ9MVkMdNwHDdet30ZiWdAKZdR3nhTYLwoSx8LXm7CfovkQ4kPXfNRNlTsNMioZ0Ktu+BVFZxVR3PyvVkJTYODqb08Y+lKAhfe/ZwD8IgAUQDuPG/0ueMIhXhGJZwgeQAEQdj4I3dn749fCrZzB5QUrJBEAEjgtS3EpTZa5U3O+mP0pQ/nLOyOCJ5ZhySWYfiAqGhoGuqaC8ouEaSNkI02KnBhTdbLkwBdJ0Sx3cJmGiZ2+cwA80aRBJs6UM81Ixnwxg7D7V+/odz01mctfHMCON49IKtzHc+H6wNFZXZ2Bux4xmENkq6jAAD3DkwnRuyYcGUDZltHEcC9bcKOgcRpwxYNTC9briOKIyAjzQlmA1j2LTJ1fXUoZ5IgAQfuTW78tozXb7NllIUtD2WBZ692WYeWAx6dsNCC6aswui1oeQC3FkwPQGSh7SnQn4l3HQm5Xp61lC23YNk27I03C63VT/wNPk+bE8VGCLf0uIOWgcTQdxtxEruoqy7qtgGZ41BWBdjm1Po/p6KluNCtNx4uhCHusCrT6/gZ2RuPftk62gd5BJb96LREtgUvW5sNU17Lju0dAAIL9h1Q0uor7TT2dGieAtvRNncAP0Bf+ljREKGDpm5tPYJlywbstSPABF5Txr8sAcc9H4+2Fc8aaZJMT++v/8GVmm3wbhNNlwcbx+DbPXiqsOaIXxNWMuB5MeTy7/j+SUGZ8+C35Zc7Iz0L3VhDc5MBezwSCuW1OaGQ78B7yI2iFqbgOA4Pz83n8xqMioG9Lw/4Yhow4ENtOeB4QK/q2Pv8BaYdILY9KLq9cWdiWRbg+dnO75mwbnNo6mtcyDhCDB78z1DCjePNGUXZ+seI5TgwcfyM8yhB14v44z/fcG+b0NFDrDkosyJEvYjf//cbbNOGLpsI69azyz1xHC9e4+3OGjz/jklAWAVt/enf2NKg9kwE7Q1DKGMLuiOj296tJ+npGnzFhT2xwSTBfdhDBDwOpvi6Bogt+FoXm821bYAoIofPo/u60NNiRDEAcTJLygK8gnYzQkvTwXMJIihwe83V4dzz7NzegcBycIcKTHX5SUWegbqRwPQsKFv1+Z/flz6WsyCUYbTF7acZOWH964mD9qdbZBpdzDih4zUbEQASC61uGW39mbYAwHfRkyy069q2Z/qxSZ7+mL4HrNyGa/Ug7n3G998VKLwPVxdnHYaNbloEq2khsbTNzmc8QyKIwszunuPhjiljPkxaFEXAH4Uire4EHFS9jsyXP3H/7RM+ZQ9wNe5ZdV1D8/MnfGspCFGHs0VYkSAIADutl4jQbdng9l201j2Zohhx9jU/gC9nsn5bUV7w+BckCPceogRrk+0Imo6KsYcvD1/wx58FdG5GF4PXDFT13/D5sw7F56B7z59DHMZgeP7D5PZZxkIujcn+0ELd+AKhO+soJJEP1w0QcxIUVUTo9iAqI+c4cW2ESntmSTFy22hPqqMlIupmE6zdhBWMrgpb1tFWhXE0VAgWCSStC22VDcYWWjaLuv/0hrDXAzhl1iZZBXVJh+1js4iATZBUKNnfYXke0FRnX4t9+D+A/NHkPHpwPKDutqFrL/u43dt7D7ZzC6baxryvkIQeTN2Aw2rovjC67Gf3pY/lLICDIO5giMXy4DOLBY96fgw+O14Hi2M8rjskCZKJZ7sMQUBPUaDHGiSOBQsWLMdBkOSFXAzzTM4hmTO6JEmAh8VpriRJlq4tJQmAsafLze9feH8ys3/lGUbheFYnRjw9rADA1y1YtoDfPt/j639kqKwPpznlMDweGyIMZ48FEsQ9Fy2tiT8iDdczr62BLUMtAHYcY/Lzn/RMaOYPoNxaGOFwsoSs7iMA1i4HsYoOLfcn/rgFCvrUFG9ZRzP3Cb/f3oE71LcaQfGqinw7fByNRZaGVtKCa5bXdshe4ONBbs1pTN6DGI7tA8ij/JLiRqIEiTHhB0B93YXiVOhqBl/+ugdK+tOyDqtCr2fw+c873AktaM9egBh+cAup/h4poLZhvIwEjJ4f8dxDIw5gd9voejwkqQh+6nuGjgbNktC1dYiJg6Zchy2YiMfG67kehLn5ck5SUHbq+O3TD+Qa5zBYgJXrYLv/wu/fijhqjvoJJ0qAayBQHaybpY/sLr4+8BD9LrpjnaBvf0dWnhsMgIUoAm0vBOQ1Da5h8dkkw7T34SptmD0F+qNGLIHfasPPH8I3JhdMgMTa0DQeepl/0qvxImR5RU6GGd7B3gML9i0g6yLCXoA4jhH1AniuiyAWUG+58DeeBlnCz+5L64I7/k70zw+olM1R5eiULi7O6aRzRKdXQ7o5LlEmU6Ba45DOBzd0sl+lSj5LDMNQtlCham2fTm8WWqPjEkNMNke5XJaymaciSplCg84W3k9Ewws6qlWpmGOIYTKUK1ap1rkk6p/RQbVChexof75UpcbJNdH1CTWqFcpnnvbvn/WJro6pVi2N9+eoWG3QyTUt2V+jw/MBDS+OqFYtUo5hiGGyVKhUqbNQUuKGzjsHVJ0qBpUpNujwsZDUgC5PO7RffCo0BTCUrx7Q8cX/LRwLJku5fJ7y+Tzlc9mpAiqg7P6aehZLQieH18dUKVapc3ZOZ8dHdHh8QCUGVFyaQP2CGpnZcMpV3HQKBKZCp3PxjP3jEoEp0fqSDctqQwzp8rBCteNzOj/ep2qjQ1fPJsEf0mkFVNqmmMZrQyeHl9SpValaKVCWGdtiqUrVxsmzYb9zDdFpBVTc5NyvDiiHDDXmQ9yujyiPzIa1JS6oweSX5rT/WaGT1yc1KhWyxGRyVCyVqFTKUwZZyhdLVCoVqfBo+xnKl0qUz02F3t10qJiZtr9RmPRTaNyQTkorwu8moXQTGxyeUy0z6l+VyfE3HSosrUdSmQnNu2hkiKmeUL/fH2+nVGUYqi65nv3jImU2qG+xEDq55Nl0NBUHPbg8plqpQo2jYzo96dBBrUSlxslCyPbwcp9yTIayuRzlslOFmZgcVTqXy0Om38neB+cHlGcm55OhbDZPxUqDDjqndHGzqW1uUBviJ/alf4yzMGJI/Zsrurq6ocHU9RgOBltUKLyio2KJ9s9u5o4Z0uDmnI5KGWIqK2LqU9azspDUgG6ur6k/pHHFzezKGOLL/SzlV1VJeTNWF5Ia9q/pur+hNQ3PqJrZsujVTgpJvYzhWY0ypePtq4u+hIt9ys7nwRjzc/MsvIyL/QxhOsZ+eEoVTD/A+9QpzhUiemRIZ7UMAQzVzoY0OClRtnFMh4Wn4kiX+zkqdG6WfO60szCg4+Kc432xT5klTgYR0fCkQswGNrYqz8JzjCpC3ix9dvbPalSoduhq3tiGfbo6bVB+zTPhrdi9vb9HISl6cV/6B0VDAAALXpAgScKM+pxdE5u/gGuixdbRVufrnbPgBAWGqYHzvbeJr04Zw0EQR6FNobdcrzBB1g2wXfOnFWdieRHiM0tREyKrDV9t/bSiV6+FVXVooQlr5+knE9imA9lYHw726xAh8O+Rl6WnZ4jvws+UUX6cQubBc/cLqxojWKiGhiwe8Fdbh94OoelNGEYFzA8TbddG2xZhPBvKM1oS4AX+8X/HtCEYxtIluCiOIPDPtflyWE6AKApLlhRC2C0XsqHPRRsBYHlI9S6Myh386fwQuzi/d7P3XfLyvvQPcxbeAFGG1HPgrrDL0PORlJW3UwynTBHDdb8DZWWmUNMMoo6u6qO1VW7rn0GAdpeD+Yym4WMjo9WVYLc3SbP7CsIuzMiA+at6VQtw4AWAe8wJEsMxnYXnhijlEK2K7pF0GAUA3z7hT86ALgFc3UA9cwdL1eCrm1Re5SFL2cc8M4mvo52YcFaIvKNetCA2fh8ESBILzw6W21nswQ1yKD9Tu+X1vJO975JX9KXUWdgWoQnXUeBoKpqmAy/oIQx7CDwb7aaCZqDBtzfIyvUiEoSeDctyd5NR7cOSILAM6JoC4yuAng1DtxCsuAZy24JkaphOavaxSOA1m4ha1i87qzCBU7poowVtZ+UgAxiaC83W/yazCgDAot4+BucYsGwbpqGj7d9DnismJikKeq63og0BWquKDDKoGZN4/TIMo4CHBwG6UV5x3CyyYYJzW7C7BjRThOloK65zCM8XoG6SznQHlC0frcSAqhmwXB+9MEQv8OF2ddTrXQiWj/Y7aF93b++75JV9aadrI39zBteXdHF+RqenZ3R+cUU3OxUqDOh8v0SN0xsa9I+pWjpeuub0S3O5T9lCjQ4PD+nwdFZ3MBz0qd8f0GAwoMH477Wr1P0zatSO6GonS9nnVFuSz35T+mf7VOtcbaGTmeLygPIrNAtL978LN3RSq9HJzVu3O6SLgxodXizvWP3zDh0eHlKjmPnlNAsjhtQfDJfoFSZc02GxSus0a8PB/ItDGq6p/zAvcJwcM1hoZ46bDhUrpxtpsa6P8pSrHNDh4SGdvHUHHPbp6vKczk5P6fTsgi6v+y/rR69id/a+O83C6/vSfxERvbH7krIDgrYMLTIRmDKACKZURmL3YLx/zN3uiAO4Xjia4hNkqAsLlNu2FyLEpim4tyFBGMYQhJedXxRGU+vEWx+NMOQgLNQbX7X/vYgRRiyEDfUaG7cZsiu/U9xz4Y2n2HhZxc5noXeFq4Grs3DjxYRHiatBCQx4b9TRvaaCoOVC3+paJXA1BYHhbfS8SXoe3N5o5M1L6ksjLT84u7B3IA5DQFim23h1y6/uS6mz8CsQdSELDvTIHU9bh2iL/43AJNib5ZdOSUn5gAS2AdNy4UcCynoLpibNLWEmCEwdrmzCkF//w/QSZyGyNeixAbv5dxqZpGxL6iz8Avi6gP/ptTF0J1oIF3VWA+9FMFMlZUrK35wEgeOBU5VX6zZCu4tQba4pcjT/0T5sh4Nan0/SlPJPI3UWPjweNO5f8MpHMCbJxEMHrd8TtAcetF9cIJeSkpKS8vH5YOmeUxaIeujd51E3NEzqVIWmibucunkBlZSUlJSUlFeQOgsfnV6AHiOjLU+KDcVwvB8oGfpPqCeQkpKSkvJPJHUWPjwJEmGqKmGvC7NXg6n9qtLvlJSUlJRfjTQp00dHKkN+rNQWoas7UNzn65WnpKSkpKS8FanA8cOTwDfq6AoqJN9FT22jOxEvpKSkpKSkvAOps/CLkMQREo7fQbKOlJSUlJSU9aTOQkpKSkpKSspaUs1CSkpKSkpKylpSZyElJSUlJSVlLamzkJKSkpKSkrKW1FlISUlJSUlJWUvqLKSkpKSkpKSs5f8BpLzYp7Bd9NQAAAAASUVORK5CYII="
    }
   },
   "cell_type": "markdown",
   "id": "6bd7c62d",
   "metadata": {},
   "source": [
    "## Let's add uncertainties\n",
    "\n",
    "The method shown before finds the neural network parameters which maximize the log-likelihood of the data. But not all parameters are equally likely and we can estimate an uncertainty for them.\n",
    "\n",
    "With an uncertainty for the parameters, we can propagate the uncertainty through the neural network and obtain an uncertainty on the prediction of the regression output.\n",
    "\n",
    "This can be done assuming each weight in the network function has a given probability distribution and instead of fitting a single value for the weight, we fit the parameters of this probability distribution. For the example shown here, we assume that the probability distribution of the weights is Gaussian and we aim to obtain the mean and variance of the Gaussian.\n",
    "\n",
    "We are going to include the epistemic uncertainty through the variation of the weights. That is, the fact that the weights vary and lead to different effective functions allow us to model different $f(x)$ dependence relationships and this is attributed to the epistemic uncertainty.\n",
    "\n",
    "We additionally assume that the data collected has some aleatoric uncertainty, which means that every point is uncertain by some fixed unknown amount. To model this effect, we assume that the likelihood function $p(\\text{data}|\\theta)$ can be modelled by a Gaussian distribution with a certain standard deviation $\\sigma_a$. This standard deviation will be used to model the aleatoric uncertainty.\n",
    "\n",
    "There are several ways to predict uncertainties in a model in general. The method we use here is called Variational Inference. In this method, we assume that we know to a good approximation some probability distribution for the function $f(x)$, but we don't know its exact probabilities. For example, we assume that the weights of the function are distributed according to a Gaussian and this already restricts how some functions $f(x)$ are more likely than others. The task then becomes simply how to discover which weights are more likely than others, transforming a function- and distribution-finding problem into a weights-finding problem. How do we find then the weights under this approximation?\n",
    "\n",
    "Since we assumed we know that all weights are distributed with a Gaussian distribution, we already know that if we fix the mean value for each weight and the standard deviation of each weight, we can in principle calculate the probability distribution modelled for our network. Let us call this probability distribution $q$. We know what the true probability distribution for the weights should really be, since this comes from the Bayes theorem: \n",
    "\n",
    "$p(\\text{weights}|\\text{data}) \\propto p(\\text{data}|\\text{weights}) p(\\text{weights})$.\n",
    "\n",
    "In this equation, $p(\\text{weights})$ is Gaussian for each weight and $p(\\text{data}|\\text{weights})$ is the likelihood that our prediction matches data with a fixed set of weights, which we shall also assume to be Gaussian and its standard deviation shall model the aleatoric uncertainty. Now, since we know the true form of the weights distribution, given the data, we can minimize some sort of *distance* between $p(\\text{weights}|\\text{data})$ and $q$, which is our approximation. How can we measure distances between probability distributions? There isn't exactly an easy-to-use concept for distances here (but see this if you want to go into more advanced things: http://www.scholarpedia.org/article/Fisher-Rao_metric ), but there is a concept of *divergence* that we can easily take advantage of. Divergences do not necessarily respect the triangular inequality, so they are not interchangeable with distances, but they provide the key element that we need here: smaller divergences between $q$ and $p(\\text{weights}|\\text{data})$ lead to them being more similar to one another.\n",
    "\n",
    "The divergence we shall use here is called the Kullback-Leibler divergence and it is closely related to the concept of Mutual Information and Entropy in Information Theory (which in reality \"borrowed\" these ideas from Statistical Mechanics). The Kullback-Leibler divergence between two distributions is defined as:\n",
    "\n",
    "$KL(q||p) = \\int q(x) \\log \\left(\\frac{p(x)}{q(x)}\\right) dx$.\n",
    "\n",
    "If we plug in this definition of the Kullback-Leibler divergence in the analytical form of our $q$, we obtain a final number that we may choose to minimize, so that our final $p(\\text{weights}|\\text{data})$ is maximized. The details of this derivation can be seen here and in cited sources: https://arxiv.org/pdf/1505.05424.pdf\n",
    "\n",
    "The final loss function obtained is:\n",
    "\n",
    "$\\mathcal{L} = -\\mathbb{E}_{\\text{data}}\\left[\\log p(\\text{data}|\\text{weights})\\right] + \\frac{1}{M} KL(\\text{weights}|\\text{prior})$\n",
    "\n",
    "As mentioned, the first term is assumed to come from a Gaussian with the standard deviation given by the aleatoric uncertainty (assumed to be the same for every data point, but this could be changed to be data-point specific as well!). The second term corresponds to a penalty for varying the weights away from the prior assumption that the weights are Gaussian with a mean zero and standard deviation 0.1. In this equation, $M$ is the number of batches used.\n",
    "\n",
    "A quick sketch from the mentioned paper showing the justification behind this equation is given below:\n",
    "\n",
    "![elbo.png](attachment:elbo.png)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "f8d501ff",
   "metadata": {},
   "outputs": [],
   "source": [
    "class BayesianNetwork(nn.Module):\n",
    "    \"\"\"\n",
    "        A model Bayesian Neural network.\n",
    "        Each weight is represented by a Gaussian with a mean and a standard deviation.\n",
    "        Each evaluation of forward leads to a different choice of the weights, so running\n",
    "        forward several times we can check the effect of the weights variation on the same input.\n",
    "        The nll function implements the negative log likelihood to be used as the first part of the loss\n",
    "        function (the second shall be the Kullback-Leibler divergence).\n",
    "        The negative log-likelihood is simply the negative log likelihood of a Gaussian\n",
    "        between the prediction and the true value. The standard deviation of the Gaussian is left as a\n",
    "        parameter to be fit: sigma.\n",
    "    \"\"\"\n",
    "    def __init__(self, input_dimension: int=1, output_dimension: int=1):\n",
    "        super(BayesianNetwork, self).__init__()\n",
    "        hidden_dimension = 100\n",
    "        self.model = nn.Sequential(\n",
    "                                   bnn.BayesLinear(prior_mu=0,\n",
    "                                                   prior_sigma=1,\n",
    "                                                   in_features=input_dimension,\n",
    "                                                   out_features=hidden_dimension),\n",
    "                                   nn.ReLU(),\n",
    "                                   bnn.BayesLinear(prior_mu=0,\n",
    "                                                   prior_sigma=1,\n",
    "                                                   in_features=hidden_dimension,\n",
    "                                                   out_features=hidden_dimension),\n",
    "                                   nn.ReLU(),\n",
    "                                   bnn.BayesLinear(prior_mu=0,\n",
    "                                                   prior_sigma=1,\n",
    "                                                   in_features=hidden_dimension,\n",
    "                                                   out_features=output_dimension)\n",
    "                                    )\n",
    "        self.log_sigma2 = nn.Parameter(torch.ones(1), requires_grad=True)\n",
    "\n",
    "    def forward(self, x: torch.Tensor) -> torch.Tensor:\n",
    "        \"\"\"\n",
    "        Calculate the result f(x) applied on the input x.\n",
    "        \"\"\"\n",
    "        return self.model(x)\n",
    "\n",
    "    def nll(self, prediction: torch.Tensor, target: torch.Tensor) -> torch.Tensor:\n",
    "        \"\"\"\n",
    "        Calculate the negative log-likelihood (divided by the batch size, since we take the mean).\n",
    "        \"\"\"\n",
    "        error = prediction - target\n",
    "        squared_error = error**2\n",
    "        sigma2 = torch.exp(self.log_sigma2)[0]\n",
    "        norm_error = 0.5*squared_error/sigma2\n",
    "        norm_term = 0.5*(np.log(2*np.pi) + self.log_sigma2[0])*target.shape[1]\n",
    "        return norm_error.sum(1).mean(0) + norm_term\n",
    "\n",
    "    def aleatoric_uncertainty(self) -> torch.Tensor:\n",
    "        \"\"\"\n",
    "            Get the aleatoric component of the uncertainty.\n",
    "        \"\"\"\n",
    "        return torch.exp(0.5*self.log_sigma2[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "7b9beb21",
   "metadata": {},
   "outputs": [],
   "source": [
    "# create the neural network:\n",
    "b_network = BayesianNetwork()\n",
    "\n",
    "# create the object to load the data:\n",
    "B = 10\n",
    "loader = torch.utils.data.DataLoader(my_dataset, batch_size=B)\n",
    "\n",
    "# create the optimizer to be used \n",
    "optimizer = torch.optim.Adam(b_network.parameters(), lr=0.001)\n",
    "\n",
    "# the Kullback-Leibler divergence should be scaled by 1/number_of_batches\n",
    "# see https://arxiv.org/abs/1505.05424 for more information on this\n",
    "number_of_batches = len(my_dataset)/float(B)\n",
    "weight_kl = 1.0/float(number_of_batches)\n",
    "# divide by B, because we implicitly divide by the number of sample taking the mean in the `nll` function above\n",
    "weight_kl /= float(B)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c68ba2e2",
   "metadata": {},
   "source": [
    "The criteria for finding the optimal weights are based on the Bayes' theorem, on which the posterior probability of the weights is proportional to the likelihood of the data given the weights and to the prior probability of the weights. We assume the prior probability of the weights is Gaussian corresponding to a unit Gaussian centred at zero and with standard deviation 0.1. This prior has a regularizing effect, preventing overtraining.\n",
    "\n",
    "We can translate the Bayes theorem and the assumption that the final posterior distribution is also Gaussian into an optimization procedure to find the posterior mean and variance of the posterior distribution. The function optimized to obtain the mean and variances of the Gaussians for the weights is the sum between the mean-squared-error (corresponding to a Gaussian log-likelihood of the data) and the Kullback-Leibler divergence between the weights distribution and the prior Gaussian."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "fbea6b0c",
   "metadata": {},
   "outputs": [],
   "source": [
    "kl_loss = bnn.BKLLoss(reduction='sum', last_layer_only=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "b92ed4b0",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 0/500  total: 2294.17400, -LL: 703.32788, prior: 0.05234, aleatoric unc.: 1.70432\n",
      "Epoch 1/500  total: 2164.52190, -LL: 4314.84668, prior: 0.05399, aleatoric unc.: 1.75715\n",
      "Epoch 2/500  total: 1671.93444, -LL: 330.02435, prior: 0.05557, aleatoric unc.: 1.80130\n",
      "Epoch 3/500  total: 1633.34416, -LL: 1548.86133, prior: 0.05711, aleatoric unc.: 1.84716\n",
      "Epoch 4/500  total: 1527.28808, -LL: 247.82253, prior: 0.05899, aleatoric unc.: 1.89552\n",
      "Epoch 5/500  total: 1440.56923, -LL: 314.48566, prior: 0.06116, aleatoric unc.: 1.94140\n",
      "Epoch 6/500  total: 1575.21707, -LL: 193.92581, prior: 0.06411, aleatoric unc.: 1.99621\n",
      "Epoch 7/500  total: 1020.99699, -LL: 311.39798, prior: 0.06605, aleatoric unc.: 2.03106\n",
      "Epoch 8/500  total: 889.46376, -LL: 228.17020, prior: 0.06773, aleatoric unc.: 2.06522\n",
      "Epoch 9/500  total: 1075.89125, -LL: 2626.64404, prior: 0.07027, aleatoric unc.: 2.10846\n",
      "Epoch 10/500  total: 783.86553, -LL: 1672.57385, prior: 0.07258, aleatoric unc.: 2.14209\n",
      "Epoch 11/500  total: 836.25104, -LL: 971.67743, prior: 0.07488, aleatoric unc.: 2.17698\n",
      "Epoch 12/500  total: 572.92991, -LL: 691.55060, prior: 0.07698, aleatoric unc.: 2.20801\n",
      "Epoch 13/500  total: 516.95223, -LL: 516.95093, prior: 0.07892, aleatoric unc.: 2.23500\n",
      "Epoch 14/500  total: 785.52413, -LL: 916.61438, prior: 0.08203, aleatoric unc.: 2.27769\n",
      "Epoch 15/500  total: 591.27327, -LL: 184.35178, prior: 0.08498, aleatoric unc.: 2.31321\n",
      "Epoch 16/500  total: 427.04376, -LL: 561.74152, prior: 0.08683, aleatoric unc.: 2.33933\n",
      "Epoch 17/500  total: 524.27719, -LL: 15.85236, prior: 0.08955, aleatoric unc.: 2.37532\n",
      "Epoch 18/500  total: 430.90639, -LL: 273.38846, prior: 0.09209, aleatoric unc.: 2.40437\n",
      "Epoch 19/500  total: 383.77823, -LL: 6.95667, prior: 0.09448, aleatoric unc.: 2.43451\n",
      "Epoch 20/500  total: 378.80883, -LL: 264.90891, prior: 0.09667, aleatoric unc.: 2.46322\n",
      "Epoch 21/500  total: 315.44016, -LL: 72.41981, prior: 0.09888, aleatoric unc.: 2.49201\n",
      "Epoch 22/500  total: 311.58309, -LL: 175.78893, prior: 0.10110, aleatoric unc.: 2.51807\n",
      "Epoch 23/500  total: 329.50581, -LL: 572.85419, prior: 0.10369, aleatoric unc.: 2.55035\n",
      "Epoch 24/500  total: 275.12305, -LL: 307.36102, prior: 0.10603, aleatoric unc.: 2.57908\n",
      "Epoch 25/500  total: 238.38241, -LL: 58.51337, prior: 0.10824, aleatoric unc.: 2.60498\n",
      "Epoch 26/500  total: 254.61200, -LL: 339.08179, prior: 0.11041, aleatoric unc.: 2.63303\n",
      "Epoch 27/500  total: 268.43455, -LL: 1827.35986, prior: 0.11269, aleatoric unc.: 2.66386\n",
      "Epoch 28/500  total: 200.06057, -LL: 172.36951, prior: 0.11525, aleatoric unc.: 2.69245\n",
      "Epoch 29/500  total: 185.10391, -LL: 676.92700, prior: 0.11748, aleatoric unc.: 2.71670\n",
      "Epoch 30/500  total: 185.27358, -LL: 353.72168, prior: 0.11965, aleatoric unc.: 2.74223\n",
      "Epoch 31/500  total: 202.86429, -LL: 56.65427, prior: 0.12221, aleatoric unc.: 2.77346\n",
      "Epoch 32/500  total: 210.36308, -LL: 88.56605, prior: 0.12501, aleatoric unc.: 2.80590\n",
      "Epoch 33/500  total: 190.61076, -LL: 109.12581, prior: 0.12819, aleatoric unc.: 2.83927\n",
      "Epoch 34/500  total: 156.23295, -LL: 87.33363, prior: 0.13093, aleatoric unc.: 2.86657\n",
      "Epoch 35/500  total: 158.04877, -LL: 245.68523, prior: 0.13412, aleatoric unc.: 2.89595\n",
      "Epoch 36/500  total: 119.76969, -LL: 119.74051, prior: 0.13642, aleatoric unc.: 2.92126\n",
      "Epoch 37/500  total: 135.00420, -LL: 33.82713, prior: 0.13874, aleatoric unc.: 2.94849\n",
      "Epoch 38/500  total: 117.64032, -LL: 39.42981, prior: 0.14125, aleatoric unc.: 2.97495\n",
      "Epoch 39/500  total: 131.96143, -LL: 98.90752, prior: 0.14446, aleatoric unc.: 3.00607\n",
      "Epoch 40/500  total: 129.90317, -LL: 234.60854, prior: 0.14738, aleatoric unc.: 3.03759\n",
      "Epoch 41/500  total: 149.99737, -LL: 20.71658, prior: 0.15074, aleatoric unc.: 3.07639\n",
      "Epoch 42/500  total: 103.29358, -LL: 118.18483, prior: 0.15347, aleatoric unc.: 3.10597\n",
      "Epoch 43/500  total: 109.35400, -LL: 66.08629, prior: 0.15591, aleatoric unc.: 3.13605\n",
      "Epoch 44/500  total: 93.15535, -LL: 7.04618, prior: 0.15849, aleatoric unc.: 3.16537\n",
      "Epoch 45/500  total: 83.13112, -LL: 409.44940, prior: 0.16091, aleatoric unc.: 3.19226\n",
      "Epoch 46/500  total: 91.04438, -LL: 73.30114, prior: 0.16382, aleatoric unc.: 3.22404\n",
      "Epoch 47/500  total: 83.55522, -LL: 244.28879, prior: 0.16646, aleatoric unc.: 3.25397\n",
      "Epoch 48/500  total: 73.99347, -LL: 57.34751, prior: 0.16900, aleatoric unc.: 3.28170\n",
      "Epoch 49/500  total: 86.06144, -LL: 97.08743, prior: 0.17213, aleatoric unc.: 3.31545\n",
      "Epoch 50/500  total: 70.17201, -LL: 472.36411, prior: 0.17489, aleatoric unc.: 3.34469\n",
      "Epoch 51/500  total: 61.18556, -LL: 65.52729, prior: 0.17738, aleatoric unc.: 3.37370\n",
      "Epoch 52/500  total: 62.92316, -LL: 35.62979, prior: 0.18012, aleatoric unc.: 3.40398\n",
      "Epoch 53/500  total: 71.60829, -LL: 121.34455, prior: 0.18321, aleatoric unc.: 3.43726\n",
      "Epoch 54/500  total: 53.54997, -LL: 25.85874, prior: 0.18630, aleatoric unc.: 3.46713\n",
      "Epoch 55/500  total: 72.00782, -LL: 46.47833, prior: 0.18960, aleatoric unc.: 3.50607\n",
      "Epoch 56/500  total: 53.76796, -LL: 30.84623, prior: 0.19247, aleatoric unc.: 3.53761\n",
      "Epoch 57/500  total: 48.81183, -LL: 114.88335, prior: 0.19508, aleatoric unc.: 3.56618\n",
      "Epoch 58/500  total: 44.38314, -LL: 53.28208, prior: 0.19776, aleatoric unc.: 3.59475\n",
      "Epoch 59/500  total: 55.60695, -LL: 47.80334, prior: 0.20111, aleatoric unc.: 3.63241\n",
      "Epoch 60/500  total: 48.88861, -LL: 143.25693, prior: 0.20443, aleatoric unc.: 3.66788\n",
      "Epoch 61/500  total: 53.50728, -LL: 4.94818, prior: 0.20783, aleatoric unc.: 3.70812\n",
      "Epoch 62/500  total: 47.01873, -LL: 77.59438, prior: 0.21115, aleatoric unc.: 3.74518\n",
      "Epoch 63/500  total: 46.15298, -LL: 187.30032, prior: 0.21436, aleatoric unc.: 3.78296\n",
      "Epoch 64/500  total: 38.49360, -LL: 192.40350, prior: 0.21705, aleatoric unc.: 3.81615\n",
      "Epoch 65/500  total: 38.48498, -LL: 89.96780, prior: 0.21999, aleatoric unc.: 3.85294\n",
      "Epoch 66/500  total: 33.08836, -LL: 19.20866, prior: 0.22247, aleatoric unc.: 3.88479\n",
      "Epoch 67/500  total: 32.53665, -LL: 10.17999, prior: 0.22546, aleatoric unc.: 3.91701\n",
      "Epoch 68/500  total: 31.42953, -LL: 30.17547, prior: 0.22803, aleatoric unc.: 3.95171\n",
      "Epoch 69/500  total: 29.76948, -LL: 60.30394, prior: 0.23062, aleatoric unc.: 3.98382\n",
      "Epoch 70/500  total: 36.32508, -LL: 22.38429, prior: 0.23436, aleatoric unc.: 4.02776\n",
      "Epoch 71/500  total: 29.30958, -LL: 7.98826, prior: 0.23705, aleatoric unc.: 4.06465\n",
      "Epoch 72/500  total: 28.79093, -LL: 6.86609, prior: 0.24080, aleatoric unc.: 4.10310\n",
      "Epoch 73/500  total: 24.90846, -LL: 39.40199, prior: 0.24306, aleatoric unc.: 4.13660\n",
      "Epoch 74/500  total: 27.69084, -LL: 14.49685, prior: 0.24588, aleatoric unc.: 4.17586\n",
      "Epoch 75/500  total: 30.61155, -LL: 66.61439, prior: 0.24992, aleatoric unc.: 4.22242\n",
      "Epoch 76/500  total: 25.08618, -LL: 42.09279, prior: 0.25323, aleatoric unc.: 4.26264\n",
      "Epoch 77/500  total: 29.45682, -LL: 6.08524, prior: 0.25746, aleatoric unc.: 4.31628\n",
      "Epoch 78/500  total: 21.04458, -LL: 15.61365, prior: 0.26008, aleatoric unc.: 4.35182\n",
      "Epoch 79/500  total: 23.43780, -LL: 14.76187, prior: 0.26322, aleatoric unc.: 4.39626\n",
      "Epoch 80/500  total: 20.39375, -LL: 52.22060, prior: 0.26632, aleatoric unc.: 4.43429\n",
      "Epoch 81/500  total: 20.52036, -LL: 20.75696, prior: 0.26912, aleatoric unc.: 4.47560\n",
      "Epoch 82/500  total: 20.58786, -LL: 23.43789, prior: 0.27223, aleatoric unc.: 4.51860\n",
      "Epoch 83/500  total: 22.81071, -LL: 4.76024, prior: 0.27617, aleatoric unc.: 4.57214\n",
      "Epoch 84/500  total: 21.30295, -LL: 11.03628, prior: 0.27926, aleatoric unc.: 4.62280\n",
      "Epoch 85/500  total: 19.57655, -LL: 25.29999, prior: 0.28254, aleatoric unc.: 4.67018\n",
      "Epoch 86/500  total: 16.27523, -LL: 39.32928, prior: 0.28471, aleatoric unc.: 4.71189\n",
      "Epoch 87/500  total: 17.87385, -LL: 5.49629, prior: 0.28736, aleatoric unc.: 4.76154\n",
      "Epoch 88/500  total: 17.28383, -LL: 18.67742, prior: 0.29044, aleatoric unc.: 4.81090\n",
      "Epoch 89/500  total: 18.66865, -LL: 3.55819, prior: 0.29501, aleatoric unc.: 4.86776\n",
      "Epoch 90/500  total: 15.61203, -LL: 38.41304, prior: 0.29754, aleatoric unc.: 4.91515\n",
      "Epoch 91/500  total: 15.95930, -LL: 49.24847, prior: 0.30078, aleatoric unc.: 4.96588\n",
      "Epoch 92/500  total: 16.55630, -LL: 20.53808, prior: 0.30533, aleatoric unc.: 5.02590\n",
      "Epoch 93/500  total: 12.66083, -LL: 4.94757, prior: 0.30831, aleatoric unc.: 5.07009\n",
      "Epoch 94/500  total: 14.76228, -LL: 71.20815, prior: 0.31009, aleatoric unc.: 5.12316\n",
      "Epoch 95/500  total: 14.62404, -LL: 14.50025, prior: 0.31300, aleatoric unc.: 5.18295\n",
      "Epoch 96/500  total: 14.90159, -LL: 12.78674, prior: 0.31677, aleatoric unc.: 5.24626\n",
      "Epoch 97/500  total: 11.74080, -LL: 7.30341, prior: 0.31915, aleatoric unc.: 5.29628\n",
      "Epoch 98/500  total: 11.48438, -LL: 5.00458, prior: 0.32149, aleatoric unc.: 5.34559\n",
      "Epoch 99/500  total: 11.39528, -LL: 9.96454, prior: 0.32386, aleatoric unc.: 5.39713\n",
      "Epoch 100/500  total: 12.73329, -LL: 7.94399, prior: 0.32652, aleatoric unc.: 5.45732\n",
      "Epoch 101/500  total: 12.82492, -LL: 5.70899, prior: 0.33021, aleatoric unc.: 5.52423\n",
      "Epoch 102/500  total: 10.29261, -LL: 6.06767, prior: 0.33286, aleatoric unc.: 5.57872\n",
      "Epoch 103/500  total: 11.17191, -LL: 14.21713, prior: 0.33638, aleatoric unc.: 5.63869\n",
      "Epoch 104/500  total: 11.81327, -LL: 7.13694, prior: 0.34007, aleatoric unc.: 5.70998\n",
      "Epoch 105/500  total: 9.54222, -LL: 24.48591, prior: 0.34191, aleatoric unc.: 5.76379\n",
      "Epoch 106/500  total: 10.36091, -LL: 6.09785, prior: 0.34401, aleatoric unc.: 5.82894\n",
      "Epoch 107/500  total: 9.04908, -LL: 11.80495, prior: 0.34631, aleatoric unc.: 5.88464\n",
      "Epoch 108/500  total: 11.12780, -LL: 6.29300, prior: 0.35127, aleatoric unc.: 5.96363\n",
      "Epoch 109/500  total: 9.48051, -LL: 9.15429, prior: 0.35349, aleatoric unc.: 6.03229\n",
      "Epoch 110/500  total: 8.67597, -LL: 6.08984, prior: 0.35566, aleatoric unc.: 6.09170\n",
      "Epoch 111/500  total: 9.08319, -LL: 5.30393, prior: 0.35811, aleatoric unc.: 6.16096\n",
      "Epoch 112/500  total: 9.66198, -LL: 25.57287, prior: 0.36034, aleatoric unc.: 6.23762\n",
      "Epoch 113/500  total: 9.07391, -LL: 6.92154, prior: 0.36347, aleatoric unc.: 6.31789\n",
      "Epoch 114/500  total: 9.18264, -LL: 9.77642, prior: 0.36589, aleatoric unc.: 6.39721\n",
      "Epoch 115/500  total: 8.56463, -LL: 4.63342, prior: 0.36976, aleatoric unc.: 6.47501\n",
      "Epoch 116/500  total: 8.41445, -LL: 16.63805, prior: 0.37295, aleatoric unc.: 6.54858\n",
      "Epoch 117/500  total: 8.15860, -LL: 4.07057, prior: 0.37583, aleatoric unc.: 6.62903\n",
      "Epoch 118/500  total: 7.66945, -LL: 12.75606, prior: 0.37780, aleatoric unc.: 6.70255\n",
      "Epoch 119/500  total: 7.49265, -LL: 8.45567, prior: 0.38185, aleatoric unc.: 6.77566\n",
      "Epoch 120/500  total: 7.88782, -LL: 8.02437, prior: 0.38530, aleatoric unc.: 6.86073\n",
      "Epoch 121/500  total: 7.91334, -LL: 9.30178, prior: 0.38813, aleatoric unc.: 6.95149\n",
      "Epoch 122/500  total: 8.09812, -LL: 4.83049, prior: 0.39001, aleatoric unc.: 7.04736\n",
      "Epoch 123/500  total: 7.35547, -LL: 5.80612, prior: 0.39266, aleatoric unc.: 7.13706\n",
      "Epoch 124/500  total: 7.19076, -LL: 7.61347, prior: 0.39384, aleatoric unc.: 7.22395\n",
      "Epoch 125/500  total: 7.22080, -LL: 7.48228, prior: 0.39673, aleatoric unc.: 7.31573\n",
      "Epoch 126/500  total: 6.51365, -LL: 10.89216, prior: 0.39865, aleatoric unc.: 7.39362\n",
      "Epoch 127/500  total: 6.40662, -LL: 6.31307, prior: 0.39938, aleatoric unc.: 7.47254\n",
      "Epoch 128/500  total: 6.93306, -LL: 7.01754, prior: 0.40160, aleatoric unc.: 7.56822\n",
      "Epoch 129/500  total: 6.74271, -LL: 6.63430, prior: 0.40347, aleatoric unc.: 7.66625\n",
      "Epoch 130/500  total: 6.44686, -LL: 11.64886, prior: 0.40589, aleatoric unc.: 7.75812\n",
      "Epoch 131/500  total: 6.72675, -LL: 5.24210, prior: 0.40782, aleatoric unc.: 7.86481\n",
      "Epoch 132/500  total: 6.80228, -LL: 13.50842, prior: 0.41071, aleatoric unc.: 7.97619\n",
      "Epoch 133/500  total: 5.97443, -LL: 4.43141, prior: 0.41181, aleatoric unc.: 8.06552\n",
      "Epoch 134/500  total: 5.92402, -LL: 5.32649, prior: 0.41321, aleatoric unc.: 8.14983\n",
      "Epoch 135/500  total: 6.17365, -LL: 5.18150, prior: 0.41440, aleatoric unc.: 8.25499\n",
      "Epoch 136/500  total: 5.96319, -LL: 5.07157, prior: 0.41622, aleatoric unc.: 8.35583\n",
      "Epoch 137/500  total: 6.23591, -LL: 5.90405, prior: 0.41917, aleatoric unc.: 8.47259\n",
      "Epoch 138/500  total: 6.16629, -LL: 7.00212, prior: 0.42168, aleatoric unc.: 8.59072\n",
      "Epoch 139/500  total: 5.70816, -LL: 4.46014, prior: 0.42198, aleatoric unc.: 8.69135\n",
      "Epoch 140/500  total: 5.89141, -LL: 4.32210, prior: 0.42329, aleatoric unc.: 8.80822\n",
      "Epoch 141/500  total: 5.86564, -LL: 4.79264, prior: 0.42409, aleatoric unc.: 8.92402\n",
      "Epoch 142/500  total: 5.82845, -LL: 7.37854, prior: 0.42666, aleatoric unc.: 9.04643\n",
      "Epoch 143/500  total: 5.52245, -LL: 6.09641, prior: 0.42617, aleatoric unc.: 9.15368\n",
      "Epoch 144/500  total: 5.55177, -LL: 3.60275, prior: 0.42706, aleatoric unc.: 9.26851\n",
      "Epoch 145/500  total: 5.50052, -LL: 7.01150, prior: 0.42671, aleatoric unc.: 9.38153\n",
      "Epoch 146/500  total: 5.57487, -LL: 4.41672, prior: 0.42908, aleatoric unc.: 9.51134\n",
      "Epoch 147/500  total: 5.30800, -LL: 4.13941, prior: 0.43043, aleatoric unc.: 9.61923\n",
      "Epoch 148/500  total: 5.39514, -LL: 3.87754, prior: 0.43000, aleatoric unc.: 9.74195\n",
      "Epoch 149/500  total: 5.25161, -LL: 4.52955, prior: 0.43019, aleatoric unc.: 9.85236\n",
      "Epoch 150/500  total: 5.23710, -LL: 5.29330, prior: 0.43140, aleatoric unc.: 9.96564\n",
      "Epoch 151/500  total: 5.55625, -LL: 12.59121, prior: 0.43435, aleatoric unc.: 10.11595\n",
      "Epoch 152/500  total: 5.40288, -LL: 5.01670, prior: 0.43468, aleatoric unc.: 10.26004\n",
      "Epoch 153/500  total: 5.29786, -LL: 3.69959, prior: 0.43491, aleatoric unc.: 10.40679\n",
      "Epoch 154/500  total: 5.17616, -LL: 5.03672, prior: 0.43485, aleatoric unc.: 10.53531\n",
      "Epoch 155/500  total: 5.16887, -LL: 3.54868, prior: 0.43612, aleatoric unc.: 10.66768\n",
      "Epoch 156/500  total: 5.13514, -LL: 5.77639, prior: 0.43507, aleatoric unc.: 10.79695\n",
      "Epoch 157/500  total: 5.15844, -LL: 5.82699, prior: 0.43707, aleatoric unc.: 10.93950\n",
      "Epoch 158/500  total: 4.89757, -LL: 4.33982, prior: 0.43826, aleatoric unc.: 11.05050\n",
      "Epoch 159/500  total: 5.09657, -LL: 9.34318, prior: 0.43866, aleatoric unc.: 11.18306\n",
      "Epoch 160/500  total: 5.03004, -LL: 4.23301, prior: 0.43979, aleatoric unc.: 11.32898\n",
      "Epoch 161/500  total: 5.19724, -LL: 6.94482, prior: 0.44151, aleatoric unc.: 11.49262\n",
      "Epoch 162/500  total: 4.93899, -LL: 3.74737, prior: 0.44072, aleatoric unc.: 11.62339\n",
      "Epoch 163/500  total: 5.00967, -LL: 4.06411, prior: 0.44184, aleatoric unc.: 11.76584\n",
      "Epoch 164/500  total: 4.92523, -LL: 3.70284, prior: 0.44245, aleatoric unc.: 11.90633\n",
      "Epoch 165/500  total: 4.90263, -LL: 5.19476, prior: 0.44214, aleatoric unc.: 12.03179\n",
      "Epoch 166/500  total: 4.90347, -LL: 5.04848, prior: 0.44011, aleatoric unc.: 12.17206\n",
      "Epoch 167/500  total: 4.85970, -LL: 4.04275, prior: 0.43931, aleatoric unc.: 12.30186\n",
      "Epoch 168/500  total: 4.90966, -LL: 6.62824, prior: 0.43810, aleatoric unc.: 12.45021\n",
      "Epoch 169/500  total: 4.91850, -LL: 4.91183, prior: 0.43710, aleatoric unc.: 12.60517\n",
      "Epoch 170/500  total: 5.03885, -LL: 4.35656, prior: 0.43715, aleatoric unc.: 12.80363\n",
      "Epoch 171/500  total: 4.86051, -LL: 4.03450, prior: 0.43588, aleatoric unc.: 12.94091\n",
      "Epoch 172/500  total: 4.86936, -LL: 4.70055, prior: 0.43449, aleatoric unc.: 13.09483\n",
      "Epoch 173/500  total: 4.83001, -LL: 3.85964, prior: 0.43462, aleatoric unc.: 13.22493\n",
      "Epoch 174/500  total: 4.89690, -LL: 3.95164, prior: 0.43540, aleatoric unc.: 13.39555\n",
      "Epoch 175/500  total: 4.92513, -LL: 5.99314, prior: 0.43664, aleatoric unc.: 13.56865\n",
      "Epoch 176/500  total: 4.89771, -LL: 4.14453, prior: 0.44024, aleatoric unc.: 13.74470\n",
      "Epoch 177/500  total: 4.69510, -LL: 3.80845, prior: 0.43905, aleatoric unc.: 13.83333\n",
      "Epoch 178/500  total: 4.77653, -LL: 3.71582, prior: 0.44094, aleatoric unc.: 13.95735\n",
      "Epoch 179/500  total: 4.76925, -LL: 5.64237, prior: 0.43987, aleatoric unc.: 14.08263\n",
      "Epoch 180/500  total: 4.71789, -LL: 3.96352, prior: 0.43841, aleatoric unc.: 14.19105\n",
      "Epoch 181/500  total: 4.77807, -LL: 3.81962, prior: 0.43777, aleatoric unc.: 14.32319\n",
      "Epoch 182/500  total: 4.72213, -LL: 6.11314, prior: 0.43537, aleatoric unc.: 14.42246\n",
      "Epoch 183/500  total: 4.69993, -LL: 4.08305, prior: 0.43216, aleatoric unc.: 14.51312\n",
      "Epoch 184/500  total: 4.73802, -LL: 4.12818, prior: 0.43229, aleatoric unc.: 14.65313\n",
      "Epoch 185/500  total: 4.67419, -LL: 4.12100, prior: 0.42907, aleatoric unc.: 14.72696\n",
      "Epoch 186/500  total: 4.83301, -LL: 3.82574, prior: 0.43024, aleatoric unc.: 14.91049\n",
      "Epoch 187/500  total: 4.76206, -LL: 3.85872, prior: 0.42845, aleatoric unc.: 15.05879\n",
      "Epoch 188/500  total: 4.72714, -LL: 5.29572, prior: 0.42632, aleatoric unc.: 15.15222\n",
      "Epoch 189/500  total: 4.61022, -LL: 4.08400, prior: 0.42602, aleatoric unc.: 15.21289\n",
      "Epoch 190/500  total: 4.65364, -LL: 4.05610, prior: 0.42413, aleatoric unc.: 15.27519\n",
      "Epoch 191/500  total: 4.67309, -LL: 3.88015, prior: 0.42243, aleatoric unc.: 15.35476\n",
      "Epoch 192/500  total: 4.70079, -LL: 6.96650, prior: 0.41984, aleatoric unc.: 15.43811\n",
      "Epoch 193/500  total: 4.72687, -LL: 3.88274, prior: 0.41902, aleatoric unc.: 15.57714\n",
      "Epoch 194/500  total: 4.76679, -LL: 4.13592, prior: 0.41894, aleatoric unc.: 15.73339\n",
      "Epoch 195/500  total: 4.60989, -LL: 4.12250, prior: 0.41892, aleatoric unc.: 15.76806\n",
      "Epoch 196/500  total: 4.69393, -LL: 3.89446, prior: 0.41817, aleatoric unc.: 15.85876\n",
      "Epoch 197/500  total: 4.60156, -LL: 4.46961, prior: 0.41520, aleatoric unc.: 15.85203\n",
      "Epoch 198/500  total: 4.61889, -LL: 3.85427, prior: 0.41157, aleatoric unc.: 15.88660\n",
      "Epoch 199/500  total: 4.62590, -LL: 3.92039, prior: 0.40973, aleatoric unc.: 15.91067\n",
      "Epoch 200/500  total: 4.60572, -LL: 3.78996, prior: 0.40960, aleatoric unc.: 15.92806\n",
      "Epoch 201/500  total: 4.66548, -LL: 4.11024, prior: 0.40654, aleatoric unc.: 15.99439\n",
      "Epoch 202/500  total: 4.63417, -LL: 3.85801, prior: 0.40403, aleatoric unc.: 16.04211\n",
      "Epoch 203/500  total: 4.71727, -LL: 4.08271, prior: 0.40567, aleatoric unc.: 16.17369\n",
      "Epoch 204/500  total: 4.57863, -LL: 3.82427, prior: 0.40441, aleatoric unc.: 16.14296\n",
      "Epoch 205/500  total: 4.70977, -LL: 4.62687, prior: 0.40592, aleatoric unc.: 16.25426\n",
      "Epoch 206/500  total: 4.59091, -LL: 4.70749, prior: 0.39964, aleatoric unc.: 16.22959\n",
      "Epoch 207/500  total: 4.60897, -LL: 3.91997, prior: 0.39924, aleatoric unc.: 16.23401\n",
      "Epoch 208/500  total: 4.62371, -LL: 4.81561, prior: 0.39723, aleatoric unc.: 16.26201\n",
      "Epoch 209/500  total: 4.59145, -LL: 4.11868, prior: 0.39567, aleatoric unc.: 16.24049\n",
      "Epoch 210/500  total: 4.63973, -LL: 4.25721, prior: 0.39458, aleatoric unc.: 16.29369\n",
      "Epoch 211/500  total: 4.63437, -LL: 3.94333, prior: 0.39565, aleatoric unc.: 16.31607\n",
      "Epoch 212/500  total: 4.62322, -LL: 5.20113, prior: 0.39551, aleatoric unc.: 16.35107\n",
      "Epoch 213/500  total: 4.58316, -LL: 4.30533, prior: 0.39006, aleatoric unc.: 16.33248\n",
      "Epoch 214/500  total: 4.60708, -LL: 4.27092, prior: 0.38732, aleatoric unc.: 16.31506\n",
      "Epoch 215/500  total: 4.55818, -LL: 4.60177, prior: 0.38425, aleatoric unc.: 16.28519\n",
      "Epoch 216/500  total: 4.58447, -LL: 4.49458, prior: 0.38426, aleatoric unc.: 16.26051\n",
      "Epoch 217/500  total: 4.61161, -LL: 4.06498, prior: 0.38396, aleatoric unc.: 16.31507\n",
      "Epoch 218/500  total: 4.54134, -LL: 4.49735, prior: 0.38046, aleatoric unc.: 16.22637\n",
      "Epoch 219/500  total: 4.65291, -LL: 3.99961, prior: 0.37965, aleatoric unc.: 16.31837\n",
      "Epoch 220/500  total: 4.58140, -LL: 4.96251, prior: 0.37568, aleatoric unc.: 16.30339\n",
      "Epoch 221/500  total: 4.59126, -LL: 3.90479, prior: 0.37506, aleatoric unc.: 16.33533\n",
      "Epoch 222/500  total: 4.60168, -LL: 4.12840, prior: 0.37475, aleatoric unc.: 16.35800\n",
      "Epoch 223/500  total: 4.61224, -LL: 4.14133, prior: 0.37542, aleatoric unc.: 16.39504\n",
      "Epoch 224/500  total: 4.56135, -LL: 4.94861, prior: 0.37697, aleatoric unc.: 16.34018\n",
      "Epoch 225/500  total: 4.56561, -LL: 3.90656, prior: 0.37348, aleatoric unc.: 16.31787\n",
      "Epoch 226/500  total: 4.54507, -LL: 3.87136, prior: 0.37132, aleatoric unc.: 16.24625\n",
      "Epoch 227/500  total: 4.57453, -LL: 3.82144, prior: 0.37349, aleatoric unc.: 16.23209\n",
      "Epoch 228/500  total: 4.56564, -LL: 4.09081, prior: 0.37190, aleatoric unc.: 16.20979\n",
      "Epoch 229/500  total: 4.55820, -LL: 4.32381, prior: 0.37007, aleatoric unc.: 16.16996\n",
      "Epoch 230/500  total: 4.53165, -LL: 4.17904, prior: 0.36693, aleatoric unc.: 16.10915\n",
      "Epoch 231/500  total: 4.55167, -LL: 3.88500, prior: 0.36874, aleatoric unc.: 16.08617\n",
      "Epoch 232/500  total: 4.55554, -LL: 4.20630, prior: 0.37019, aleatoric unc.: 16.07415\n",
      "Epoch 233/500  total: 4.64898, -LL: 3.87531, prior: 0.36958, aleatoric unc.: 16.19588\n",
      "Epoch 234/500  total: 4.58516, -LL: 3.82347, prior: 0.37062, aleatoric unc.: 16.23983\n",
      "Epoch 235/500  total: 4.52828, -LL: 3.99777, prior: 0.36784, aleatoric unc.: 16.13410\n",
      "Epoch 236/500  total: 4.56419, -LL: 3.88711, prior: 0.36415, aleatoric unc.: 16.12881\n",
      "Epoch 237/500  total: 4.51364, -LL: 3.85443, prior: 0.36235, aleatoric unc.: 16.06821\n",
      "Epoch 238/500  total: 4.53692, -LL: 4.16873, prior: 0.36272, aleatoric unc.: 16.01422\n",
      "Epoch 239/500  total: 4.52414, -LL: 4.43059, prior: 0.36026, aleatoric unc.: 15.98025\n",
      "Epoch 240/500  total: 4.52974, -LL: 3.82850, prior: 0.35667, aleatoric unc.: 15.96161\n",
      "Epoch 241/500  total: 4.54721, -LL: 3.81967, prior: 0.35717, aleatoric unc.: 15.95436\n",
      "Epoch 242/500  total: 4.52174, -LL: 3.85391, prior: 0.35371, aleatoric unc.: 15.91074\n",
      "Epoch 243/500  total: 4.52171, -LL: 3.84012, prior: 0.35449, aleatoric unc.: 15.87800\n",
      "Epoch 244/500  total: 4.48056, -LL: 4.70426, prior: 0.35078, aleatoric unc.: 15.78308\n",
      "Epoch 245/500  total: 4.53157, -LL: 3.92313, prior: 0.35472, aleatoric unc.: 15.79237\n",
      "Epoch 246/500  total: 4.53108, -LL: 3.85934, prior: 0.35546, aleatoric unc.: 15.77559\n",
      "Epoch 247/500  total: 4.52557, -LL: 4.12003, prior: 0.35167, aleatoric unc.: 15.76002\n",
      "Epoch 248/500  total: 4.50074, -LL: 4.28565, prior: 0.34953, aleatoric unc.: 15.73245\n",
      "Epoch 249/500  total: 4.55166, -LL: 4.17833, prior: 0.34960, aleatoric unc.: 15.77043\n",
      "Epoch 250/500  total: 4.56264, -LL: 4.14792, prior: 0.34951, aleatoric unc.: 15.82986\n",
      "Epoch 251/500  total: 4.54331, -LL: 4.13038, prior: 0.35015, aleatoric unc.: 15.84215\n",
      "Epoch 252/500  total: 4.58222, -LL: 3.93371, prior: 0.34975, aleatoric unc.: 15.92820\n",
      "Epoch 253/500  total: 4.54323, -LL: 4.07649, prior: 0.35027, aleatoric unc.: 15.92687\n",
      "Epoch 254/500  total: 4.55835, -LL: 4.00149, prior: 0.35056, aleatoric unc.: 15.97446\n",
      "Epoch 255/500  total: 4.48282, -LL: 3.77635, prior: 0.35023, aleatoric unc.: 15.89263\n",
      "Epoch 256/500  total: 4.52993, -LL: 4.34475, prior: 0.34669, aleatoric unc.: 15.86973\n",
      "Epoch 257/500  total: 4.48462, -LL: 5.00446, prior: 0.34372, aleatoric unc.: 15.79595\n",
      "Epoch 258/500  total: 4.48687, -LL: 4.10884, prior: 0.34404, aleatoric unc.: 15.74077\n",
      "Epoch 259/500  total: 4.47548, -LL: 4.94737, prior: 0.34176, aleatoric unc.: 15.66432\n",
      "Epoch 260/500  total: 4.45308, -LL: 4.01868, prior: 0.34024, aleatoric unc.: 15.58120\n",
      "Epoch 261/500  total: 4.48031, -LL: 3.95064, prior: 0.33870, aleatoric unc.: 15.53961\n",
      "Epoch 262/500  total: 4.50446, -LL: 3.82153, prior: 0.33506, aleatoric unc.: 15.56419\n",
      "Epoch 263/500  total: 4.46654, -LL: 4.43109, prior: 0.33716, aleatoric unc.: 15.50787\n",
      "Epoch 264/500  total: 4.50047, -LL: 4.56283, prior: 0.33743, aleatoric unc.: 15.50358\n",
      "Epoch 265/500  total: 4.53483, -LL: 4.92259, prior: 0.33880, aleatoric unc.: 15.54195\n",
      "Epoch 266/500  total: 4.48853, -LL: 4.06707, prior: 0.33707, aleatoric unc.: 15.52835\n",
      "Epoch 267/500  total: 4.48643, -LL: 4.31537, prior: 0.33551, aleatoric unc.: 15.52833\n",
      "Epoch 268/500  total: 4.43746, -LL: 3.74320, prior: 0.33580, aleatoric unc.: 15.44481\n",
      "Epoch 269/500  total: 4.45904, -LL: 4.12249, prior: 0.33215, aleatoric unc.: 15.37424\n",
      "Epoch 270/500  total: 4.43447, -LL: 4.48685, prior: 0.32961, aleatoric unc.: 15.30517\n",
      "Epoch 271/500  total: 4.44404, -LL: 4.04688, prior: 0.32919, aleatoric unc.: 15.24123\n",
      "Epoch 272/500  total: 4.47093, -LL: 4.14307, prior: 0.33486, aleatoric unc.: 15.25717\n",
      "Epoch 273/500  total: 4.38182, -LL: 4.15838, prior: 0.33034, aleatoric unc.: 15.08905\n",
      "Epoch 274/500  total: 4.47173, -LL: 3.82219, prior: 0.32999, aleatoric unc.: 15.08135\n",
      "Epoch 275/500  total: 4.36634, -LL: 4.26105, prior: 0.32649, aleatoric unc.: 14.94182\n",
      "Epoch 276/500  total: 4.52364, -LL: 3.95488, prior: 0.32549, aleatoric unc.: 15.03032\n",
      "Epoch 277/500  total: 4.40352, -LL: 4.13583, prior: 0.32405, aleatoric unc.: 14.97387\n",
      "Epoch 278/500  total: 4.40713, -LL: 4.51679, prior: 0.32256, aleatoric unc.: 14.89801\n",
      "Epoch 279/500  total: 4.48028, -LL: 4.06760, prior: 0.32299, aleatoric unc.: 14.96830\n",
      "Epoch 280/500  total: 4.52136, -LL: 5.16841, prior: 0.32424, aleatoric unc.: 15.07113\n",
      "Epoch 281/500  total: 4.37791, -LL: 4.02710, prior: 0.32618, aleatoric unc.: 14.97818\n",
      "Epoch 282/500  total: 4.47312, -LL: 3.96956, prior: 0.32588, aleatoric unc.: 14.99776\n",
      "Epoch 283/500  total: 4.37466, -LL: 3.90545, prior: 0.32387, aleatoric unc.: 14.89272\n",
      "Epoch 284/500  total: 4.45828, -LL: 3.83861, prior: 0.32713, aleatoric unc.: 14.90926\n",
      "Epoch 285/500  total: 4.47821, -LL: 3.77745, prior: 0.32877, aleatoric unc.: 14.95312\n",
      "Epoch 286/500  total: 4.38676, -LL: 3.77643, prior: 0.32696, aleatoric unc.: 14.86059\n",
      "Epoch 287/500  total: 4.41101, -LL: 3.87555, prior: 0.32184, aleatoric unc.: 14.80666\n",
      "Epoch 288/500  total: 4.38039, -LL: 3.79602, prior: 0.31874, aleatoric unc.: 14.73124\n",
      "Epoch 289/500  total: 4.40861, -LL: 3.94453, prior: 0.31796, aleatoric unc.: 14.68562\n",
      "Epoch 290/500  total: 4.42048, -LL: 4.85468, prior: 0.31717, aleatoric unc.: 14.66400\n",
      "Epoch 291/500  total: 4.47149, -LL: 4.16605, prior: 0.32087, aleatoric unc.: 14.74144\n",
      "Epoch 292/500  total: 4.38311, -LL: 4.23254, prior: 0.32335, aleatoric unc.: 14.68072\n",
      "Epoch 293/500  total: 4.43105, -LL: 6.23792, prior: 0.32213, aleatoric unc.: 14.64300\n",
      "Epoch 294/500  total: 4.39989, -LL: 3.77106, prior: 0.32154, aleatoric unc.: 14.64436\n",
      "Epoch 295/500  total: 4.45488, -LL: 4.01945, prior: 0.31872, aleatoric unc.: 14.69023\n",
      "Epoch 296/500  total: 4.38851, -LL: 3.99385, prior: 0.31663, aleatoric unc.: 14.62358\n",
      "Epoch 297/500  total: 4.42410, -LL: 4.41566, prior: 0.31540, aleatoric unc.: 14.64059\n",
      "Epoch 298/500  total: 4.44718, -LL: 4.26388, prior: 0.31464, aleatoric unc.: 14.68472\n",
      "Epoch 299/500  total: 4.38908, -LL: 3.72025, prior: 0.31473, aleatoric unc.: 14.65539\n",
      "Epoch 300/500  total: 4.46657, -LL: 4.89952, prior: 0.31657, aleatoric unc.: 14.68762\n",
      "Epoch 301/500  total: 4.42263, -LL: 4.18494, prior: 0.31660, aleatoric unc.: 14.71025\n",
      "Epoch 302/500  total: 4.42304, -LL: 4.25995, prior: 0.31648, aleatoric unc.: 14.71077\n",
      "Epoch 303/500  total: 4.37865, -LL: 3.74415, prior: 0.31368, aleatoric unc.: 14.63672\n",
      "Epoch 304/500  total: 4.38759, -LL: 4.06919, prior: 0.31460, aleatoric unc.: 14.57741\n",
      "Epoch 305/500  total: 4.38715, -LL: 3.76870, prior: 0.31255, aleatoric unc.: 14.57457\n",
      "Epoch 306/500  total: 4.39209, -LL: 3.94298, prior: 0.31312, aleatoric unc.: 14.54549\n",
      "Epoch 307/500  total: 4.31401, -LL: 3.90953, prior: 0.30980, aleatoric unc.: 14.37811\n",
      "Epoch 308/500  total: 4.36241, -LL: 4.02659, prior: 0.31063, aleatoric unc.: 14.32905\n",
      "Epoch 309/500  total: 4.36258, -LL: 4.23499, prior: 0.31151, aleatoric unc.: 14.27872\n",
      "Epoch 310/500  total: 4.32726, -LL: 3.76714, prior: 0.30940, aleatoric unc.: 14.19101\n",
      "Epoch 311/500  total: 4.38596, -LL: 3.81679, prior: 0.30674, aleatoric unc.: 14.17865\n",
      "Epoch 312/500  total: 4.42068, -LL: 3.75325, prior: 0.31111, aleatoric unc.: 14.24413\n",
      "Epoch 313/500  total: 4.37524, -LL: 3.84334, prior: 0.30775, aleatoric unc.: 14.23629\n",
      "Epoch 314/500  total: 4.27371, -LL: 3.87354, prior: 0.30729, aleatoric unc.: 14.06987\n",
      "Epoch 315/500  total: 4.39870, -LL: 3.89449, prior: 0.30772, aleatoric unc.: 14.08975\n",
      "Epoch 316/500  total: 4.30602, -LL: 3.67755, prior: 0.30566, aleatoric unc.: 14.00856\n",
      "Epoch 317/500  total: 4.33313, -LL: 3.97410, prior: 0.30577, aleatoric unc.: 13.94800\n",
      "Epoch 318/500  total: 4.43432, -LL: 3.71468, prior: 0.30888, aleatoric unc.: 14.04741\n",
      "Epoch 319/500  total: 4.37897, -LL: 3.75499, prior: 0.31079, aleatoric unc.: 14.08446\n",
      "Epoch 320/500  total: 4.30563, -LL: 3.65509, prior: 0.30797, aleatoric unc.: 13.97607\n",
      "Epoch 321/500  total: 4.37533, -LL: 4.24918, prior: 0.30791, aleatoric unc.: 13.98534\n",
      "Epoch 322/500  total: 4.36393, -LL: 3.74137, prior: 0.30698, aleatoric unc.: 13.98914\n",
      "Epoch 323/500  total: 4.35164, -LL: 4.10119, prior: 0.30806, aleatoric unc.: 13.96811\n",
      "Epoch 324/500  total: 4.35739, -LL: 3.90398, prior: 0.30846, aleatoric unc.: 13.93596\n",
      "Epoch 325/500  total: 4.38250, -LL: 4.44499, prior: 0.30673, aleatoric unc.: 13.97261\n",
      "Epoch 326/500  total: 4.35315, -LL: 4.42376, prior: 0.30701, aleatoric unc.: 13.95997\n",
      "Epoch 327/500  total: 4.36795, -LL: 4.17653, prior: 0.30855, aleatoric unc.: 13.97699\n",
      "Epoch 328/500  total: 4.30396, -LL: 3.91943, prior: 0.30560, aleatoric unc.: 13.88542\n",
      "Epoch 329/500  total: 4.36057, -LL: 4.99554, prior: 0.30607, aleatoric unc.: 13.87517\n",
      "Epoch 330/500  total: 4.25497, -LL: 3.72196, prior: 0.30206, aleatoric unc.: 13.75286\n",
      "Epoch 331/500  total: 4.32408, -LL: 3.83675, prior: 0.30096, aleatoric unc.: 13.70025\n",
      "Epoch 332/500  total: 4.41900, -LL: 4.16212, prior: 0.30270, aleatoric unc.: 13.81388\n",
      "Epoch 333/500  total: 4.27495, -LL: 3.77374, prior: 0.30219, aleatoric unc.: 13.72647\n",
      "Epoch 334/500  total: 4.34971, -LL: 3.86083, prior: 0.30085, aleatoric unc.: 13.73270\n",
      "Epoch 335/500  total: 4.37895, -LL: 4.20601, prior: 0.30247, aleatoric unc.: 13.77878\n",
      "Epoch 336/500  total: 4.36633, -LL: 3.74334, prior: 0.30218, aleatoric unc.: 13.81359\n",
      "Epoch 337/500  total: 4.31169, -LL: 3.69489, prior: 0.30070, aleatoric unc.: 13.75855\n",
      "Epoch 338/500  total: 4.29754, -LL: 3.98524, prior: 0.30092, aleatoric unc.: 13.69109\n",
      "Epoch 339/500  total: 4.35188, -LL: 3.76089, prior: 0.29924, aleatoric unc.: 13.71738\n",
      "Epoch 340/500  total: 4.36790, -LL: 6.01456, prior: 0.29973, aleatoric unc.: 13.73557\n",
      "Epoch 341/500  total: 4.32057, -LL: 3.93740, prior: 0.30091, aleatoric unc.: 13.73691\n",
      "Epoch 342/500  total: 4.31353, -LL: 4.73162, prior: 0.29983, aleatoric unc.: 13.69596\n",
      "Epoch 343/500  total: 4.29736, -LL: 3.70710, prior: 0.30202, aleatoric unc.: 13.64569\n",
      "Epoch 344/500  total: 4.30277, -LL: 4.39192, prior: 0.29873, aleatoric unc.: 13.58624\n",
      "Epoch 345/500  total: 4.28730, -LL: 3.70689, prior: 0.30125, aleatoric unc.: 13.53884\n",
      "Epoch 346/500  total: 4.28112, -LL: 3.88174, prior: 0.29967, aleatoric unc.: 13.46685\n",
      "Epoch 347/500  total: 4.29716, -LL: 3.81197, prior: 0.29814, aleatoric unc.: 13.42146\n",
      "Epoch 348/500  total: 4.33209, -LL: 3.68184, prior: 0.29644, aleatoric unc.: 13.45658\n",
      "Epoch 349/500  total: 4.25540, -LL: 4.01308, prior: 0.29434, aleatoric unc.: 13.34073\n",
      "Epoch 350/500  total: 4.33118, -LL: 3.82876, prior: 0.29366, aleatoric unc.: 13.41339\n",
      "Epoch 351/500  total: 4.35809, -LL: 3.98745, prior: 0.29452, aleatoric unc.: 13.47417\n",
      "Epoch 352/500  total: 4.36339, -LL: 4.11955, prior: 0.29371, aleatoric unc.: 13.56014\n",
      "Epoch 353/500  total: 4.27057, -LL: 4.07453, prior: 0.29478, aleatoric unc.: 13.48736\n",
      "Epoch 354/500  total: 4.35561, -LL: 4.00709, prior: 0.29816, aleatoric unc.: 13.54832\n",
      "Epoch 355/500  total: 4.30775, -LL: 3.76553, prior: 0.29959, aleatoric unc.: 13.53639\n",
      "Epoch 356/500  total: 4.30931, -LL: 3.85962, prior: 0.29742, aleatoric unc.: 13.51869\n",
      "Epoch 357/500  total: 4.31901, -LL: 4.00900, prior: 0.29731, aleatoric unc.: 13.50655\n",
      "Epoch 358/500  total: 4.30852, -LL: 3.72225, prior: 0.29563, aleatoric unc.: 13.47658\n",
      "Epoch 359/500  total: 4.28606, -LL: 3.78456, prior: 0.29504, aleatoric unc.: 13.45554\n",
      "Epoch 360/500  total: 4.23826, -LL: 3.60652, prior: 0.29418, aleatoric unc.: 13.32370\n",
      "Epoch 361/500  total: 4.33118, -LL: 4.33109, prior: 0.29590, aleatoric unc.: 13.36398\n",
      "Epoch 362/500  total: 4.30136, -LL: 3.88332, prior: 0.29581, aleatoric unc.: 13.35263\n",
      "Epoch 363/500  total: 4.26135, -LL: 3.99345, prior: 0.29653, aleatoric unc.: 13.29258\n",
      "Epoch 364/500  total: 4.28480, -LL: 4.00098, prior: 0.29697, aleatoric unc.: 13.25982\n",
      "Epoch 365/500  total: 4.29093, -LL: 3.72887, prior: 0.29551, aleatoric unc.: 13.25455\n",
      "Epoch 366/500  total: 4.31470, -LL: 3.67486, prior: 0.29167, aleatoric unc.: 13.27328\n",
      "Epoch 367/500  total: 4.28997, -LL: 3.87706, prior: 0.29307, aleatoric unc.: 13.25669\n",
      "Epoch 368/500  total: 4.25890, -LL: 3.61237, prior: 0.29420, aleatoric unc.: 13.21329\n",
      "Epoch 369/500  total: 4.28513, -LL: 3.66183, prior: 0.29501, aleatoric unc.: 13.19332\n",
      "Epoch 370/500  total: 4.24566, -LL: 3.59994, prior: 0.29461, aleatoric unc.: 13.11074\n",
      "Epoch 371/500  total: 4.28892, -LL: 3.62588, prior: 0.29155, aleatoric unc.: 13.12838\n",
      "Epoch 372/500  total: 4.25191, -LL: 3.96459, prior: 0.29308, aleatoric unc.: 13.06497\n",
      "Epoch 373/500  total: 4.27148, -LL: 3.65914, prior: 0.29409, aleatoric unc.: 13.05644\n",
      "Epoch 374/500  total: 4.22671, -LL: 3.59360, prior: 0.29335, aleatoric unc.: 12.96356\n",
      "Epoch 375/500  total: 4.25709, -LL: 3.75483, prior: 0.29473, aleatoric unc.: 12.92731\n",
      "Epoch 376/500  total: 4.30109, -LL: 3.96422, prior: 0.29433, aleatoric unc.: 12.96784\n",
      "Epoch 377/500  total: 4.27804, -LL: 4.62750, prior: 0.29492, aleatoric unc.: 12.96339\n",
      "Epoch 378/500  total: 4.30291, -LL: 3.66387, prior: 0.29426, aleatoric unc.: 13.02547\n",
      "Epoch 379/500  total: 4.28989, -LL: 3.85442, prior: 0.29308, aleatoric unc.: 13.02152\n",
      "Epoch 380/500  total: 4.28133, -LL: 3.64806, prior: 0.29297, aleatoric unc.: 13.04834\n",
      "Epoch 381/500  total: 4.27531, -LL: 3.91687, prior: 0.29161, aleatoric unc.: 13.04516\n",
      "Epoch 382/500  total: 4.22670, -LL: 3.65751, prior: 0.29002, aleatoric unc.: 12.96334\n",
      "Epoch 383/500  total: 4.25438, -LL: 3.83289, prior: 0.29040, aleatoric unc.: 12.92650\n",
      "Epoch 384/500  total: 4.28284, -LL: 4.12477, prior: 0.29173, aleatoric unc.: 12.93924\n",
      "Epoch 385/500  total: 4.28154, -LL: 5.06327, prior: 0.29164, aleatoric unc.: 12.94179\n",
      "Epoch 386/500  total: 4.29184, -LL: 3.71668, prior: 0.28974, aleatoric unc.: 12.99222\n",
      "Epoch 387/500  total: 4.23449, -LL: 3.61562, prior: 0.29012, aleatoric unc.: 12.93269\n",
      "Epoch 388/500  total: 4.20235, -LL: 3.64429, prior: 0.28787, aleatoric unc.: 12.83775\n",
      "Epoch 389/500  total: 4.23469, -LL: 3.65001, prior: 0.28924, aleatoric unc.: 12.78895\n",
      "Epoch 390/500  total: 4.25743, -LL: 3.70225, prior: 0.28938, aleatoric unc.: 12.79457\n",
      "Epoch 391/500  total: 4.28773, -LL: 3.82099, prior: 0.29032, aleatoric unc.: 12.83519\n",
      "Epoch 392/500  total: 4.24977, -LL: 3.72148, prior: 0.28917, aleatoric unc.: 12.82497\n",
      "Epoch 393/500  total: 4.23988, -LL: 3.75097, prior: 0.29060, aleatoric unc.: 12.79168\n",
      "Epoch 394/500  total: 4.23515, -LL: 4.51935, prior: 0.29009, aleatoric unc.: 12.74808\n",
      "Epoch 395/500  total: 4.25556, -LL: 3.92254, prior: 0.28824, aleatoric unc.: 12.74187\n",
      "Epoch 396/500  total: 4.24138, -LL: 4.08155, prior: 0.28770, aleatoric unc.: 12.73601\n",
      "Epoch 397/500  total: 4.32805, -LL: 3.77373, prior: 0.28914, aleatoric unc.: 12.85976\n",
      "Epoch 398/500  total: 4.29116, -LL: 4.03029, prior: 0.28854, aleatoric unc.: 12.89174\n",
      "Epoch 399/500  total: 4.29906, -LL: 4.08009, prior: 0.29080, aleatoric unc.: 12.94973\n",
      "Epoch 400/500  total: 4.24325, -LL: 4.37855, prior: 0.29110, aleatoric unc.: 12.89844\n",
      "Epoch 401/500  total: 4.21631, -LL: 4.37101, prior: 0.29081, aleatoric unc.: 12.83269\n",
      "Epoch 402/500  total: 4.21490, -LL: 3.70825, prior: 0.28925, aleatoric unc.: 12.77170\n",
      "Epoch 403/500  total: 4.24548, -LL: 3.97789, prior: 0.28979, aleatoric unc.: 12.73532\n",
      "Epoch 404/500  total: 4.25498, -LL: 3.91230, prior: 0.28735, aleatoric unc.: 12.75526\n",
      "Epoch 405/500  total: 4.23092, -LL: 3.64123, prior: 0.28916, aleatoric unc.: 12.73843\n",
      "Epoch 406/500  total: 4.30377, -LL: 3.91702, prior: 0.29196, aleatoric unc.: 12.76713\n",
      "Epoch 407/500  total: 4.25391, -LL: 3.67363, prior: 0.29132, aleatoric unc.: 12.80679\n",
      "Epoch 408/500  total: 4.22299, -LL: 3.62789, prior: 0.28925, aleatoric unc.: 12.74367\n",
      "Epoch 409/500  total: 4.26734, -LL: 3.80965, prior: 0.29048, aleatoric unc.: 12.76100\n",
      "Epoch 410/500  total: 4.25349, -LL: 3.69565, prior: 0.29013, aleatoric unc.: 12.76318\n",
      "Epoch 411/500  total: 4.28336, -LL: 3.69440, prior: 0.29020, aleatoric unc.: 12.79956\n",
      "Epoch 412/500  total: 4.29031, -LL: 4.51302, prior: 0.29134, aleatoric unc.: 12.83943\n",
      "Epoch 413/500  total: 4.19793, -LL: 3.90017, prior: 0.29209, aleatoric unc.: 12.74592\n",
      "Epoch 414/500  total: 4.19218, -LL: 3.57106, prior: 0.29010, aleatoric unc.: 12.65540\n",
      "Epoch 415/500  total: 4.25412, -LL: 3.74360, prior: 0.29186, aleatoric unc.: 12.65665\n",
      "Epoch 416/500  total: 4.26500, -LL: 3.91594, prior: 0.29214, aleatoric unc.: 12.68659\n",
      "Epoch 417/500  total: 4.21341, -LL: 3.72687, prior: 0.29015, aleatoric unc.: 12.63162\n",
      "Epoch 418/500  total: 4.21933, -LL: 3.62715, prior: 0.29108, aleatoric unc.: 12.57077\n",
      "Epoch 419/500  total: 4.25920, -LL: 3.62662, prior: 0.28931, aleatoric unc.: 12.61524\n",
      "Epoch 420/500  total: 4.24498, -LL: 3.89066, prior: 0.29000, aleatoric unc.: 12.60225\n",
      "Epoch 421/500  total: 4.26176, -LL: 4.10444, prior: 0.28994, aleatoric unc.: 12.64584\n",
      "Epoch 422/500  total: 4.19053, -LL: 3.76841, prior: 0.29019, aleatoric unc.: 12.56762\n",
      "Epoch 423/500  total: 4.20613, -LL: 4.07726, prior: 0.28902, aleatoric unc.: 12.51632\n",
      "Epoch 424/500  total: 4.20716, -LL: 3.57636, prior: 0.28839, aleatoric unc.: 12.48092\n",
      "Epoch 425/500  total: 4.18982, -LL: 3.90812, prior: 0.28768, aleatoric unc.: 12.40683\n",
      "Epoch 426/500  total: 4.20946, -LL: 4.42304, prior: 0.28793, aleatoric unc.: 12.37388\n",
      "Epoch 427/500  total: 4.22372, -LL: 4.08125, prior: 0.28901, aleatoric unc.: 12.37556\n",
      "Epoch 428/500  total: 4.23337, -LL: 4.16943, prior: 0.28858, aleatoric unc.: 12.40270\n",
      "Epoch 429/500  total: 4.28320, -LL: 3.92145, prior: 0.29058, aleatoric unc.: 12.47761\n",
      "Epoch 430/500  total: 4.23783, -LL: 3.58313, prior: 0.28923, aleatoric unc.: 12.49446\n",
      "Epoch 431/500  total: 4.20762, -LL: 3.79757, prior: 0.28842, aleatoric unc.: 12.43921\n",
      "Epoch 432/500  total: 4.26053, -LL: 3.61355, prior: 0.29010, aleatoric unc.: 12.51055\n",
      "Epoch 433/500  total: 4.20546, -LL: 3.60644, prior: 0.28897, aleatoric unc.: 12.47210\n",
      "Epoch 434/500  total: 4.17361, -LL: 3.62394, prior: 0.28800, aleatoric unc.: 12.38231\n",
      "Epoch 435/500  total: 4.18867, -LL: 3.65719, prior: 0.28752, aleatoric unc.: 12.32489\n",
      "Epoch 436/500  total: 4.22270, -LL: 4.32778, prior: 0.28681, aleatoric unc.: 12.32984\n",
      "Epoch 437/500  total: 4.16955, -LL: 3.78783, prior: 0.28606, aleatoric unc.: 12.25807\n",
      "Epoch 438/500  total: 4.22591, -LL: 4.07864, prior: 0.28614, aleatoric unc.: 12.28022\n",
      "Epoch 439/500  total: 4.19177, -LL: 4.09317, prior: 0.28660, aleatoric unc.: 12.25075\n",
      "Epoch 440/500  total: 4.27602, -LL: 3.54679, prior: 0.28683, aleatoric unc.: 12.33664\n",
      "Epoch 441/500  total: 4.18954, -LL: 3.72110, prior: 0.28622, aleatoric unc.: 12.31076\n",
      "Epoch 442/500  total: 4.25897, -LL: 3.84474, prior: 0.28746, aleatoric unc.: 12.37183\n",
      "Epoch 443/500  total: 4.19797, -LL: 3.61508, prior: 0.28697, aleatoric unc.: 12.33025\n",
      "Epoch 444/500  total: 4.19945, -LL: 4.23817, prior: 0.28682, aleatoric unc.: 12.30664\n",
      "Epoch 445/500  total: 4.25961, -LL: 3.78511, prior: 0.28722, aleatoric unc.: 12.37745\n",
      "Epoch 446/500  total: 4.25115, -LL: 3.63889, prior: 0.28812, aleatoric unc.: 12.41472\n",
      "Epoch 447/500  total: 4.18577, -LL: 3.74992, prior: 0.28674, aleatoric unc.: 12.35474\n",
      "Epoch 448/500  total: 4.22803, -LL: 3.74013, prior: 0.28695, aleatoric unc.: 12.35672\n",
      "Epoch 449/500  total: 4.24271, -LL: 4.02800, prior: 0.28769, aleatoric unc.: 12.39347\n",
      "Epoch 450/500  total: 4.22130, -LL: 3.77517, prior: 0.28920, aleatoric unc.: 12.40454\n",
      "Epoch 451/500  total: 4.19642, -LL: 4.85910, prior: 0.28584, aleatoric unc.: 12.35776\n",
      "Epoch 452/500  total: 4.17571, -LL: 3.55679, prior: 0.28438, aleatoric unc.: 12.30047\n",
      "Epoch 453/500  total: 4.20105, -LL: 4.47895, prior: 0.28587, aleatoric unc.: 12.26487\n",
      "Epoch 454/500  total: 4.18347, -LL: 3.77700, prior: 0.28563, aleatoric unc.: 12.22977\n",
      "Epoch 455/500  total: 4.23386, -LL: 3.76747, prior: 0.28721, aleatoric unc.: 12.26009\n",
      "Epoch 456/500  total: 4.24766, -LL: 3.71218, prior: 0.28666, aleatoric unc.: 12.31672\n",
      "Epoch 457/500  total: 4.23320, -LL: 3.63747, prior: 0.28566, aleatoric unc.: 12.36460\n",
      "Epoch 458/500  total: 4.21914, -LL: 4.22862, prior: 0.28573, aleatoric unc.: 12.33850\n",
      "Epoch 459/500  total: 4.23816, -LL: 3.77558, prior: 0.28672, aleatoric unc.: 12.39121\n",
      "Epoch 460/500  total: 4.18095, -LL: 3.59544, prior: 0.28580, aleatoric unc.: 12.33295\n",
      "Epoch 461/500  total: 4.19751, -LL: 3.65706, prior: 0.28622, aleatoric unc.: 12.29170\n",
      "Epoch 462/500  total: 4.17444, -LL: 3.58506, prior: 0.28652, aleatoric unc.: 12.24185\n",
      "Epoch 463/500  total: 4.21997, -LL: 3.79871, prior: 0.28532, aleatoric unc.: 12.25640\n",
      "Epoch 464/500  total: 4.17849, -LL: 3.67556, prior: 0.28590, aleatoric unc.: 12.20991\n",
      "Epoch 465/500  total: 4.19469, -LL: 3.65216, prior: 0.28387, aleatoric unc.: 12.18799\n",
      "Epoch 466/500  total: 4.18374, -LL: 3.77503, prior: 0.28464, aleatoric unc.: 12.13527\n",
      "Epoch 467/500  total: 4.15980, -LL: 3.54591, prior: 0.28497, aleatoric unc.: 12.09160\n",
      "Epoch 468/500  total: 4.15528, -LL: 3.64045, prior: 0.28466, aleatoric unc.: 12.03012\n",
      "Epoch 469/500  total: 4.20050, -LL: 3.53782, prior: 0.28442, aleatoric unc.: 12.04823\n",
      "Epoch 470/500  total: 4.18814, -LL: 4.13648, prior: 0.28479, aleatoric unc.: 12.02533\n",
      "Epoch 471/500  total: 4.23799, -LL: 4.06228, prior: 0.28585, aleatoric unc.: 12.09650\n",
      "Epoch 472/500  total: 4.21309, -LL: 3.93179, prior: 0.28526, aleatoric unc.: 12.13297\n",
      "Epoch 473/500  total: 4.22759, -LL: 4.34791, prior: 0.28510, aleatoric unc.: 12.16390\n",
      "Epoch 474/500  total: 4.19927, -LL: 3.58355, prior: 0.28564, aleatoric unc.: 12.18269\n",
      "Epoch 475/500  total: 4.20806, -LL: 3.66223, prior: 0.28625, aleatoric unc.: 12.19320\n",
      "Epoch 476/500  total: 4.18165, -LL: 3.72681, prior: 0.28742, aleatoric unc.: 12.15077\n",
      "Epoch 477/500  total: 4.18182, -LL: 3.96265, prior: 0.28656, aleatoric unc.: 12.11517\n",
      "Epoch 478/500  total: 4.20171, -LL: 3.59803, prior: 0.28639, aleatoric unc.: 12.10981\n",
      "Epoch 479/500  total: 4.20908, -LL: 3.62256, prior: 0.28814, aleatoric unc.: 12.13264\n",
      "Epoch 480/500  total: 4.20871, -LL: 3.70293, prior: 0.28707, aleatoric unc.: 12.14405\n",
      "Epoch 481/500  total: 4.16861, -LL: 4.15197, prior: 0.28590, aleatoric unc.: 12.09377\n",
      "Epoch 482/500  total: 4.19767, -LL: 5.47841, prior: 0.28523, aleatoric unc.: 12.05883\n",
      "Epoch 483/500  total: 4.25352, -LL: 4.24806, prior: 0.28549, aleatoric unc.: 12.17083\n",
      "Epoch 484/500  total: 4.19220, -LL: 3.62151, prior: 0.28546, aleatoric unc.: 12.16249\n",
      "Epoch 485/500  total: 4.20604, -LL: 4.07630, prior: 0.28566, aleatoric unc.: 12.16024\n",
      "Epoch 486/500  total: 4.21192, -LL: 3.81535, prior: 0.28562, aleatoric unc.: 12.18320\n",
      "Epoch 487/500  total: 4.22036, -LL: 3.71840, prior: 0.28568, aleatoric unc.: 12.21167\n",
      "Epoch 488/500  total: 4.17873, -LL: 3.67013, prior: 0.28418, aleatoric unc.: 12.17093\n",
      "Epoch 489/500  total: 4.17590, -LL: 3.84937, prior: 0.28326, aleatoric unc.: 12.13036\n",
      "Epoch 490/500  total: 4.19966, -LL: 3.53921, prior: 0.28432, aleatoric unc.: 12.13181\n",
      "Epoch 491/500  total: 4.15192, -LL: 3.69806, prior: 0.28322, aleatoric unc.: 12.05840\n",
      "Epoch 492/500  total: 4.20871, -LL: 3.99247, prior: 0.28498, aleatoric unc.: 12.09097\n",
      "Epoch 493/500  total: 4.24564, -LL: 3.53812, prior: 0.28562, aleatoric unc.: 12.15556\n",
      "Epoch 494/500  total: 4.21448, -LL: 3.74054, prior: 0.28474, aleatoric unc.: 12.16671\n",
      "Epoch 495/500  total: 4.16589, -LL: 3.60851, prior: 0.28430, aleatoric unc.: 12.13879\n",
      "Epoch 496/500  total: 4.19407, -LL: 3.67642, prior: 0.28451, aleatoric unc.: 12.12040\n",
      "Epoch 497/500  total: 4.20313, -LL: 4.11066, prior: 0.28443, aleatoric unc.: 12.11105\n",
      "Epoch 498/500  total: 4.17024, -LL: 3.79871, prior: 0.28372, aleatoric unc.: 12.08894\n",
      "Epoch 499/500  total: 4.20190, -LL: 3.76114, prior: 0.28452, aleatoric unc.: 12.09669\n"
     ]
    }
   ],
   "source": [
    "epochs = 500\n",
    "# for each epoch\n",
    "for epoch in range(epochs):\n",
    "    losses = list()\n",
    "    # for each mini-batch given by the loader:\n",
    "    for batch in loader:\n",
    "        # get the input in the mini-batch\n",
    "        # this has size (B, C)\n",
    "        # where B is the mini-batch size\n",
    "        # C is the number of features (1 in this case)\n",
    "        features = batch[\"data\"]\n",
    "        # get the targets in the mini-batch (there shall be B of them)\n",
    "        target = batch[\"target\"]\n",
    "        # get the output of the neural network:\n",
    "        prediction = b_network(features)\n",
    "        \n",
    "        # calculate the loss function being minimized\n",
    "        # in this case, it is the mean-squared error between the prediction and the target values added\n",
    "        # to the Kullback-Leibler divergence between the current weight Gaussian and\n",
    "        # the prior Gaussian, set to the unit Normal distribution\n",
    "        nll = b_network.nll(prediction, target)\n",
    "        prior = weight_kl *kl_loss(b_network)\n",
    "        loss = nll + prior\n",
    "\n",
    "        # clean the optimizer temporary gradient storage\n",
    "        optimizer.zero_grad()\n",
    "        # calculate the gradient of the loss function as a function of the gradients\n",
    "        loss.backward()\n",
    "        # ask the Adam optimizer to change the parameters in the direction of - gradient\n",
    "        # Adam scales the gradient by a constant which is adaptively tuned\n",
    "        # take a look at the Adam paper for more details: https://arxiv.org/abs/1412.6980\n",
    "        optimizer.step()\n",
    "        \n",
    "        ale = b_network.aleatoric_uncertainty().detach().numpy()\n",
    "\n",
    "        losses.append(loss.detach().cpu().item())\n",
    "    avg_loss = np.mean(np.array(losses))\n",
    "    print(f\"Epoch {epoch}/{epochs}  total: {avg_loss:.5f}, -LL: {nll.item():.5f}, prior: {prior.item():.5f}, aleatoric unc.: {ale:.5f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15f3f633",
   "metadata": {},
   "source": [
    "After the fit, each time we perform a prediction with the neural network, the weights of the neural network are different: they are randomly chosen from a Gaussian distribution with the fit mean and standard deviation. The logic here is that all these weights match the expected posterior probability optimized, so they should all provide trustworthy results given the data available (but not equally probable!).\n",
    "\n",
    "In order to obtain an uncertainty band for each prediction, we perform each prediction with the neural network fit several times to obtain the results with different set of weights following the posterior probability mentioned. After taking several results for the same input data, we can use the mean prediction as the \"nominal\" prediction and the root-mean-squared-error of the predictions as the uncertainty of the prediction, given the available data.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "a3d244e2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's generate new data!\n",
    "test_data = generate_data(N=1000)\n",
    "\n",
    "b_predicted = list()\n",
    "for k in range(10):\n",
    "    p = b_network(torch.from_numpy(test_data[:,0:1])).detach().numpy()\n",
    "    b_predicted.append(p[:,0])\n",
    "b_predicted = np.stack(b_predicted, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85fed034",
   "metadata": {},
   "source": [
    "We can now take the average result for each sample, and their root-mean-squared-error as an estimate of the mean and epistemic uncertainty for the results.\n",
    "\n",
    "The aleatoric uncertainty is fitted as an independent parameter. Since we assume the aleatoric uncertainty is independent, we can calculate the total uncertainty as the sum of squares of the epistemic and aleatoric uncertainty."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "4a56960d",
   "metadata": {},
   "outputs": [],
   "source": [
    "b_mean = np.mean(b_predicted, axis=1)\n",
    "b_sigma = np.std(b_predicted, axis=1)\n",
    "aleatoric_uncertainty = b_network.aleatoric_uncertainty().detach().numpy()\n",
    "\n",
    "total_uncertainty = (b_sigma**2 + aleatoric_uncertainty**2)**0.5"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ef3d430",
   "metadata": {},
   "source": [
    "Let's check how big are those uncertainties found:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "4d01c41f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Average epistemic uncertainty:  4.342738\n"
     ]
    }
   ],
   "source": [
    "print(\"Average epistemic uncertainty: \", np.mean(b_sigma))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "67d456a1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Aleatoric uncertainty:  12.096693\n"
     ]
    }
   ],
   "source": [
    "print(\"Aleatoric uncertainty: \", aleatoric_uncertainty)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "364bdcd7",
   "metadata": {},
   "source": [
    "Note that the aleatoric uncertainty is very close to the standard deviation of the $\\epsilon$ component of the model we created in the beginning! Clearly the model could fit the uncertainty coming from that component of the noise.\n",
    "\n",
    "It is not easy to estimate the effect of the epistemic uncertainty, as it is different for every data point (as it is scaled by $x^2$), but we can plot it to take a look at its effect.\n",
    "\n",
    "Note that the uncertainties are the standard deviations of Gaussian models and therefore they correspond to a $1\\sigma$ quantile band, which is a 67% confidence band. The quantile corresponding to $2\\sigma$ corresponds to a 95% confidence band in a Gaussian model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "8b9142e8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAj0AAAG0CAYAAADZxpaMAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAACyGUlEQVR4nOzdeXxU5b348c85Z9bsCwmEfV9FxaWIouIGSF1r63L9WbXWtopStbbeumLttVdrb6vV2tb2Ymuttl6X9notaFUQFRBUVBYpQXZIAmSdycycOec8vz/OzCSTTEICSQjk++5rajJz5pxnhsmZ73me7/N9NKWUQgghhBDiCKcf6gYIIYQQQvQECXqEEEII0SdI0COEEEKIPkGCHiGEEEL0CRL0CCGEEKJPkKBHCCGEEH2CBD1CCCGE6BMk6BFCCCFEnyBBjxBCCCH6BAl6hBBCCNEn9Iqg5yc/+Qknnngiubm5lJaWctFFF7Fhw4a0baLRKHPnzqW4uJicnBwuueQSKisr07bZtm0bX/7yl8nKyqK0tJTvf//7WJbVky9FCCGEEL2U51A3AGDJkiXMnTuXE088EcuyuPPOO5k5cybr1q0jOzsbgFtvvZX/+7//44UXXiA/P5+bbrqJr3zlK7z33nsA2LbNl7/8ZQYMGMD777/P7t27+frXv47X6+XBBx/sUDscx2HXrl3k5uaiaVq3vV4hhBBCdB2lFA0NDQwcOBBdb6c/R/VCVVVVClBLlixRSilVW1urvF6veuGFF1LbrF+/XgFq2bJlSimlXnvtNaXruqqoqEht8+STT6q8vDwVi8U6dNzt27crQG5yk5vc5CY3uR2Gt+3bt7f7Pd8renpaqqurA6CoqAiADz/8kHg8ztlnn53aZvz48QwdOpRly5Zx0kknsWzZMiZPnkz//v1T28yaNYsbbriBtWvXMmXKlFbHicVixGKx1O8qseD89u3bycvL65bXJoQQQoiuVV9fz5AhQ8jNzW13u14X9DiOwy233MIpp5zCUUcdBUBFRQU+n4+CgoK0bfv3709FRUVqm+YBT/Lx5GOZ/OQnP+H+++9vdX9eXp4EPUIIIcRhZn+pKb0ikbm5uXPnsmbNGp5//vluP9YPf/hD6urqUrft27d3+zGFEEIIcWj0qp6em266iVdffZV33nmHwYMHp+4fMGAApmlSW1ub1ttTWVnJgAEDUtt88MEHaftLzu5KbtOS3+/H7/d38asQQgghRG/UK3p6lFLcdNNNvPzyy7z11luMGDEi7fHjjz8er9fLm2++mbpvw4YNbNu2jWnTpgEwbdo0PvvsM6qqqlLbvPHGG+Tl5TFx4sSeeSFCCCGE6LV6RU/P3Llz+fOf/8zf/vY3cnNzUzk4+fn5BINB8vPzue6667jtttsoKioiLy+Pm2++mWnTpnHSSScBMHPmTCZOnMhVV13Fww8/TEVFBXfffTdz586V3hwhhBBCoKnklKVD2Yg2Eo8WLFjANddcA7jFCb/3ve/x3HPPEYvFmDVrFr/61a/Shq62bt3KDTfcwOLFi8nOzubqq6/mP//zP/F4Ohbb1dfXk5+fT11dnSQyCyGEEIeJjn5/94qgp7eQoEcIIYQ4/HT0+7tX5PQIIYQQQnQ3CXqEEEII0SdI0COEEEKIPqFXzN4SQgghxJHLcRQ7ayOETYtsn4dBBUF0vecX9pagR/SoLVu2MGLECD7++GOOPfbYQ90cIYQQ3ay8qoFFayrZtCdE1LIJeAxGleQw66j+jC5tf62sribDW0cwTdPavc2fP/+g9v3KK690WVvbc80113DRRRf1yLGEEEJ0nfKqBha8t4U1u+ooyPIysl8OBVle1uyqY8F7WyivaujR9khPTw/ryS6+3bt3p37+y1/+wr333suGDRtS9+Xk5HTLcYUQQgjHUSxaU0l12GRMaU6qJl9uwEuO38PGqhCvr61kZL+cHhvqkp6eHlRe1cCTizfx8zf+xWNvbuTnb/yLJxdv6rZId8CAAalbfn4+mqal3ff8888zYcIEAoEA48eP51e/+lXquaZpctNNN1FWVkYgEGDYsGH85Cc/AWD48OEAXHzxxWialvo9kw8++IApU6YQCAQ44YQT+Pjjj9Met22b6667jhEjRhAMBhk3bhyPPvpo6vH58+fzhz/8gb/97W+pHqrFixcDcMcddzB27FiysrIYOXIk99xzD/F4vGvePCGEEAdlZ22ETXtClOUHWhUh1jSNsvwA5VUhdtZGeqxN0tPTQ5JdfNVhk7L8AFm+II2mxZpddeyqi3DtKcN7dGzz2Wef5d577+Xxxx9nypQpfPzxx1x//fWpStaPPfYYf//73/nrX//K0KFD2b59e2oV+pUrV1JaWsqCBQuYPXs2hmFkPEYoFOK8887jnHPO4U9/+hObN2/mu9/9bto2juMwePBgXnjhBYqLi3n//ff51re+RVlZGZdeeim3334769evp76+ngULFgBQVFQEQG5uLk8//TQDBw7ks88+4/rrryc3N5cf/OAH3fjOCSGE6IiwaRG1bLJ8wYyPB30GlfVRwqbVY22SoKcH9MYuvvvuu4+f/exnfOUrXwFgxIgRrFu3jt/85jdcffXVbNu2jTFjxjB9+nQ0TWPYsGGp55aUlABQUFDQ5gr2AH/+859xHIff//73BAIBJk2axI4dO7jhhhtS23i9Xu6///7U7yNGjGDZsmX89a9/5dJLLyUnJ4dgMEgsFmt1rLvvvjv18/Dhw7n99tt5/vnnJegRQoheINvnIeAxaDQtcgPeVo9HTBu/xyDb13OhiAQ9PaAzXXxDirK6vT3hcJhNmzZx3XXXcf3116futyyL/Px8wE0ePueccxg3bhyzZ8/mvPPOY+bMmZ06zvr16zn66KMJBAKp+6ZNm9ZquyeeeIL//u//Ztu2bUQiEUzT7NDMrr/85S889thjbNq0iVAohGVZsnyIEEL0EoMKgowqyWHNrjpy/J607z+lFLvrokwelM+ggsw9Qd1Bcnp6QFMXX+YYM+gziFl2j3XxhUIhAJ566ilWr16duq1Zs4bly5cDcNxxx7F582YeeOABIpEIl156KV/96le7vC3PP/88t99+O9dddx2vv/46q1ev5tprr8U0zXaft2zZMq688krmzJnDq6++yscff8xdd9213+cJIYToGbquMeuo/hRl+9hYFaIhGsdyHBqicTZWhSjK9jFzUv8erdcjPT09oLd18fXv35+BAwfyxRdfcOWVV7a5XV5eHpdddhmXXXYZX/3qV5k9ezbV1dUUFRXh9Xqxbbvd40yYMIFnnnmGaDSa6u1JBlVJ7733HieffDI33nhj6r5NmzalbePz+Vod6/3332fYsGHcddddqfu2bt3a/gsXQgjRo0aX5nLtKcNTdXoq66P4PQaTB+Uzc1LP1+mRoKcH9MYuvvvvv5958+aRn5/P7NmzicVirFq1ipqaGm677Tb+67/+i7KyMqZMmYKu67zwwgsMGDCAgoICwM2hefPNNznllFPw+/0UFha2Osa//du/cdddd3H99dfzwx/+kC1btvDII4+kbTNmzBj++Mc/smjRIkaMGMEzzzzDypUrGTFiRGqb4cOHs2jRIjZs2EBxcTH5+fmMGTOGbdu28fzzz3PiiSfyf//3f7z88svd+p4JIYTovNGluYyckdMrKjLL8FYP6I1dfN/85jf53e9+x4IFC5g8eTKnn346Tz/9dCrYyM3N5eGHH+aEE07gxBNPZMuWLbz22mvouvuR+dnPfsYbb7zBkCFDmDJlSsZj5OTk8L//+7989tlnTJkyhbvuuouHHnoobZtvf/vbfOUrX+Gyyy5j6tSp7Nu3L63XB+D6669n3LhxnHDCCZSUlPDee+9xwQUXcOutt3LTTTdx7LHH8v7773PPPfd0wzslhBDiYOm6xpCiLMYPyGNIUdYhCXgANKWUOiRH7oXq6+vJz8+nrq6uWxJim5fijlnukNbo0pxD0sUnhBBCHCk6+v0tw1s9qDd18QkhhBB9jQQ9PSzZxSeEEEKIniU5PUIIIYToEyToEUIIIUSfIEGPEEIIIfoECXqEEEII0SdI0COEEEKIPkGCHiGEEEL0CTJlvaeZJjz4oPvznXeCz3do2yOEEEL0EdLTI3rMNddcw0UXXXRIjv3000+n1g3rLTRN45VXXjnUzRBCiD5Dgp6e5jhQWwuVlbBli/t7N2poaOCWW25h2LBhBINBTj75ZFauXJm2zTXXXIOmaWm32bNnpx6PxWJcddVV5OXlMXbsWP75z3+mPf+nP/0pN998c7e+ju62ZcsWNE1j9erVPfI8IYQQPU+Gt3rS+vXwP/8Db78NlgUNDTBxIlx8MUyY0C2H/OY3v8maNWt45plnGDhwIH/60584++yzWbduHYMGDUptN3v2bBYsWJD63e/3p37+7W9/y4cffsiyZcv4xz/+wb/9279RWVmJpmls3ryZp556ilWrVnVL+4UQQoiuIj09PWX9enjsMVi9GrKyoLjYvX38sXv/+vVdfshIJMKLL77Iww8/zGmnncbo0aOZP38+o0eP5sknn0zb1u/3M2DAgNStsLCwWdPXc8EFFzBp0iTmzp3Lnj172Lt3LwA33HADDz30UKcWaL3//vspKSkhLy+P73znO5immXps4cKFTJ8+nYKCAoqLiznvvPPYtGlT6vFkz8pLL73EGWecQVZWFscccwzLli1LO8bTTz/N0KFDycrK4uKLL2bfvn3ttim5uvyUKVPQNI0ZM2YA4DgOP/rRjxg8eDB+v59jjz2WhQsX7vd5K1eu5JxzzqFfv37k5+dz+umn89FHH3X4PQIYPnw4v/jFL9LuO/bYY5k/f37qd03T+N3vfsfFF19MVlYWY8aM4e9//3vac9auXct5551HXl4eubm5nHrqqWnvqRBC9BUS9PQEx4GXX4a9e90eHb8fdB3y8tyenr174ZVXunyoy7IsbNsmEAik3R8MBnn33XfT7lu8eDGlpaWMGzeOG264IS1IOOaYY3j33XeJRCIsWrSIsrIy+vXrx7PPPksgEODiiy/ucJvefPNN1q9fz+LFi3nuued46aWXuP/++1OPh8NhbrvtNlatWsWbb76JrutcfPHFOC3em7vuuovbb7+d1atXM3bsWK644gosywJgxYoVXHfdddx0002sXr2aM844gx//+MfttuuDDz4A4J///Ce7d+/mpZdeAuDRRx/lZz/7GY888giffvops2bN4oILLmDjxo3tPq+hoYGrr76ad999l+XLlzNmzBjmzJlDQ0NDh9+rjrr//vu59NJL+fTTT5kzZw5XXnkl1dXVAOzcuZPTTjsNv9/PW2+9xYcffsg3vvGN1HslhBB9ihIpdXV1ClB1dXVdu+PNm5W66iqlbr1VqbvvVur0093b3Xcrdd99St1yi/v45s1de1yl1LRp09Tpp5+udu7cqSzLUs8884zSdV2NHTs2tc1zzz2n/va3v6lPP/1Uvfzyy2rChAnqxBNPVJZlKaWUMk1T3XjjjWr48OHqhBNOUEuXLlX79u1TI0eOVNu2bVN33XWXGjVqlJo5c6basWNHm225+uqrVVFRkQqHw6n7nnzySZWTk6Ns2874nD179ihAffbZZ0oppTZv3qwA9bvf/S61zdq1axWg1q9fr5RS6oorrlBz5sxJ289ll12m8vPz22xbcr8ff/xx2v0DBw5U//Ef/5F234knnqhuvPHGdp/Xkm3bKjc3V/3v//5v6j5Avfzyy20+Z9iwYernP/952n3HHHOMuu+++9L2cffdd6d+D4VCClD/+Mc/lFJK/fCHP1QjRoxQpmm22z4hhDicdfT7W3p6ekJDA0SjkJ2d+fHsbPfxbugFeOaZZ1BKMWjQIPx+P4899hhXXHEFut70T3/55ZdzwQUXMHnyZC666CJeffVVVq5cyeLFiwHwer088cQTbN68mZUrVzJ9+nS+973vMW/ePD7++GNeeeUVPvnkE0466STmzZvXbnuOOeYYsrKaVpmfNm0aoVCI7du3A7Bx40auuOIKRo4cSV5eHsOHDwdg27Ztafs5+uijUz+XlZUBUFVVBbjDcVOnTk3bftq0aZ1411z19fXs2rWLU045Je3+U045hfX7GY6srKzk+uuvZ8yYMeTn55OXl0coFGr1OrpC8/ciOzubvLy81HuxevVqTj31VLxeb5cfVwghDjcS9PSE3FwIBCAczvx4OOw+npvb5YceNWoUS5YsSQUWH3zwAfF4nJEjR7b5nJEjR9KvXz/Ky8szPv7222+zdu1abrrpJhYvXsycOXPIzs7m0ksvTQVKB+r888+nurqap556ihUrVrBixQqAtLwfIO1LXNM0gFZDYIfS1VdfzerVq3n00Ud5//33Wb16NcXFxa1eR3t0XUcplXZfPB5vtV3LgEbTtNR7EQwGD6D1QghxZJKgpycMHQrjx8P27dDiSwylYMcON9dn6NBua0J2djZlZWXU1NSwaNEiLrzwwja33bFjB/v27Uv1oDQXjUaZO3cuv/nNbzAMA9u2U1/E8Xgc27bbbccnn3xCJBJJ/b58+XJycnIYMmQI+/btY8OGDdx9992cddZZTJgwgZqamk6/1gkTJqSCpebHaY8vUSSyefvz8vIYOHAg7733Xtq27733HhMnTmzzeclt5s2bx5w5c5g0aRJ+vz+V/N1RJSUl7N69O/V7fX09mzdv7tQ+jj76aJYuXZoxWBJCiL5Ggp6eoOvutPR+/dxZWtGom7RcVwfr1rn3X3SRu10XW7RoEQsXLmTz5s288cYbnHHGGYwfP55rr70WgFAoxPe//32WL1/Oli1bePPNN7nwwgsZPXo0s2bNarW/Bx54gDlz5jBlyhTAHep56aWX+PTTT3n88cdbDQW1ZJom1113HevWreO1117jvvvu46abbkLXdQoLCykuLua3v/0t5eXlvPXWW9x2222dfs3z5s1j4cKFPPLII2zcuJHHH388bcZVJqWlpQSDQRYuXEhlZSV1dXUAfP/73+ehhx7iL3/5Cxs2bODf//3fWb16Nd/97nfbfd6YMWN45plnWL9+PStWrODKK6/sdK/LmWeeyTPPPMPSpUv57LPPuPrqqzEMo1P7uOmmm6ivr+fyyy9n1apVbNy4kWeeeYYNGzYA8PLLLzN+/PhO7VMIIQ5bPZJhdJjotkTmpHXrlPrRj5Q6+milJk5U6sorlXrwQff+bvKXv/xFjRw5Uvl8PjVgwAA1d+5cVVtbm3q8sbFRzZw5U5WUlCiv16uGDRumrr/+elVRUdFqX5999pkaPXq0CoVCqfts21Y33HCDysvLUyeeeKLauHFjm225+uqr1YUXXqjuvfdeVVxcrHJyctT111+votFoaps33nhDTZgwQfn9fnX00UerxYsXpyX8ZkocrqmpUYB6++23U/f9/ve/V4MHD1bBYFCdf/756pFHHmk3kVkppZ566ik1ZMgQpeu6Ov3001Ovb/78+WrQoEHK6/WqY445JpUk3N7zPvroI3XCCSeoQCCgxowZo1544YVWicnsJ5G5rq5OXXbZZSovL08NGTJEPf300xkTmVvuIz8/Xy1YsCD1+yeffKJmzpypsrKyVG5urjr11FPVpk2blFJKLViwQMlpQAhxuOvo97emVMvxlr6rvr6e/Px86urqOlV3plOiUfj3f4dYDG69FUaP7pYeHiGEEKKv6Oj3t1Rk7mm6Dsk1oIYPl4BHCCGE6CES9PQ0nw+aVdQVQgghRM+QbgYhhBBC9AnS0yOEEEKI7mWa8OCD7s933umOehwC0tMjhBBCiD5Bgh4hhBBC9AkS9AghhBCiT5CgRwghhBB9giQy9zDTNnlwqZvMdeepd+IzDk0ylxBCCNHXSE+P6DHXXHMNF1100SE59tNPP01BsihkN5oxYwa33HJLtx+nsxYvXoymadTW1h7qpgghxCEjQU8Pc5RDbbSWylAlW2q34CinW4/X0NDALbfcwrBhwwgGg5x88smsXLkybZtrrrkGTdPSbrNnz049HovFuOqqq8jLy2Ps2LH885//THv+T3/6U26++eZufR2Hi5deeokHHnigQ9v2ZIB08skns3v3bvLz83vkeEII0Zxpm8xnMfNZjGmbh6wdMrzVg9bvWc//rPsf3t78NpZj0WA2MLFkIhePv5gJJRO65Zjf/OY3WbNmDc888wwDBw7kT3/6E2effTbr1q1j0KBBqe1mz57NggULUr/7/f7Uz7/97W/58MMPWbZsGf/4xz/4t3/7NyorK9E0jc2bN/PUU0+xatWqbmn/4aaoqOhQNyEjn8/HgAEDDnUzhBDikJKenh6yfs96HlvxGKsrVpPlzaI4q5jiYDEf7/6Yx1Y8xvo967v8mJFIhBdffJGHH36Y0047jdGjRzN//nxGjx7Nk08+mbat3+9nwIABqVthYWFT29ev54ILLmDSpEnMnTuXPXv2sHfvXgBuuOEGHnrooU4t0Hr//fdTUlJCXl4e3/nOdzDNpqh/4cKFTJ8+nYKCAoqLiznvvPPYtGlT6vEtW7agaRovvfQSZ5xxBllZWRxzzDEsW7Ys7RhPP/00Q4cOJSsri4svvph9+/btt13bt2/n0ksvpaCggKKiIi688EK2bNmSejw5PNde+1v23vzqV79izJgxBAIB+vfvz1e/+tXUvpYsWcKjjz6a6l1LHmvNmjWce+655OTk0L9/f6666qrU+508xs0338wtt9xCYWEh/fv356mnniIcDnPttdeSm5vL6NGj+cc//pF6Tqbhrffee48ZM2aQlZVFYWEhs2bNoqamJuN7M3/+fI499ti0+37xi18wfPjwVu/PI488QllZGcXFxcydO5d4PJ7aJhaLcccddzBkyBD8fj+jR4/m97///f7+aYQQoktI0NMDHOXw8ucvs7dxLxP6TcDv8aNrOnn+PCaWTGRv415e+fyVLh/qsiwL27YJBAJp9weDQd599920+xYvXkxpaSnjxo3jhhtuSAsSjjnmGN59910ikQiLFi2irKyMfv368eyzzxIIBLj44os73KY333yT9evXs3jxYp577jleeukl7r///tTj4XCY2267jVWrVvHmm2+i6zoXX3wxjpP+3tx1113cfvvtrF69mrFjx3LFFVdgWRYAK1as4LrrruOmm25i9erVnHHGGfz4xz9ut13xeJxZs2aRm5vL0qVLee+998jJyWH27NlpQc3+2t/cqlWrmDdvHj/60Y/YsGEDCxcu5LTTTgPg0UcfZdq0aVx//fXs3r2b3bt3M2TIEGpraznzzDOZMmUKq1atYuHChVRWVnLppZem7fsPf/gD/fr144MPPuDmm2/mhhtu4Gtf+xonn3wyH330ETNnzuSqq66isbExY9tWr17NWWedxcSJE1m2bBnvvvsu559/PrZtt/s+7c/bb7/Npk2bePvtt/nDH/7A008/zdNPP516/Otf/zrPPfccjz32GOvXr+c3v/kNOTk5B3VMIYToMCVS6urqFKDq6uq6dL+bazarq166St268FZ195t3q9MXnK5OX3C6uvvNu9V9b9+nbvnHLeqql65Sm2s2d+lxlVJq2rRp6vTTT1c7d+5UlmWpZ555Rum6rsaOHZva5rnnnlN/+9vf1KeffqpefvllNWHCBHXiiScqy7KUUkqZpqluvPFGNXz4cHXCCSeopUuXqn379qmRI0eqbdu2qbvuukuNGjVKzZw5U+3YsaPNtlx99dWqqKhIhcPh1H1PPvmkysnJUbZtZ3zOnj17FKA+++wzpZRSmzdvVoD63e9+l9pm7dq1ClDr169XSil1xRVXqDlz5qTt57LLLlP5+flttu2ZZ55R48aNU47jpO6LxWIqGAyqRYsWdbj9p59+uvrud7+rlFLqxRdfVHl5eaq+vj7jMZtvm/TAAw+omTNnpt23fft2BagNGzaknjd9+vTU45ZlqezsbHXVVVel7tu9e7cC1LJly5RSSr399tsKUDU1Nan36JRTTmnz/WjpvvvuU8ccc0zafT//+c/VsGHDUr9fffXVatiwYanPjVJKfe1rX1OXXXaZUkqpDRs2KEC98cYbHT6uEOLIEGtsUPfdd7q6777TVayxocv339Hvb+np6QENsQaiVpRsb3bGx7N92UStKA2xhi4/9jPPPINSikGDBuH3+3nssce44oor0PWmf/rLL7+cCy64gMmTJ3PRRRfx6quvsnLlShYvXgyA1+vliSeeYPPmzaxcuZLp06fzve99j3nz5vHxxx/zyiuv8Mknn3DSSScxb968dttzzDHHkJWVlfp92rRphEIhtm/fDsDGjRu54oorGDlyJHl5eanhk23btqXt5+ijj079XFZWBkBVVRXgDsdNnTo1bftp06a1265PPvmE8vJycnNzycnJIScnh6KiIqLRaNrw2v7a39w555zDsGHDGDlyJFdddRXPPvtsmz0vzdvx9ttvp9qQk5PD+PHjAdLa0fz1G4ZBcXExkydPTt3Xv3//tPekpWRPT1ebNGkShmGkfi8rK0u1YfXq1RiGwemnn97lxxVCiI6QROYekOvPJeAJEI6HMwY+YTNMwBMg15/b5cceNWoUS5YsIRwOU19fT1lZGZdddhkjR45s8zkjR46kX79+lJeXZ/xifPvtt1m7di2/+93v+P73v8+cOXPIzs7m0ksv5fHHHz+o9p5//vkMGzaMp556ioEDB+I4DkcddVTaEBO4gViSpmkArYbAOiMUCnH88cfz7LPPtnqspKTkgPaZm5vLRx99xOLFi3n99de59957mT9/PitXrmxz+nwoFOL888/noYceavVYMriD9NcP7nvQmfckGAx26rXouo5SKu2+5rk67bUr2YbOHlMIIbqa9PT0gKH5Qxnfbzzb67a3+uJQSrGjfgcT+k1gaP7QbmtDdnY2ZWVl1NTUsGjRIi688MI2t92xYwf79u1L+5JNikajzJ07l9/85jcYhoFt26kvv3g8vt+ckE8++YRIJJL6ffny5eTk5DBkyBD27dvHhg0buPvuuznrrLOYMGFCm4m17ZkwYQIrVqxIu2/58uXtPue4445j48aNlJaWMnr06LRb82ne7bU/E4/Hw9lnn83DDz/Mp59+ypYtW3jrrbcAd0ZVy/fruOOOY+3atQwfPrxVO7KzM/cUHoijjz6aN998s8Pbl5SUUFFRkfb5Xb16daeOOXnyZBzHYcmSJZ16nhBCdBUJenqArulcPP5i+mX1Y/3e9UStKI5yqIvVsW7POvpl9eOi8Reha13/z7Fo0SIWLlzI5s2beeONNzjjjDMYP3481157LeD2LHz/+99n+fLlbNmyhTfffJMLL7yQ0aNHM2vWrFb7e+CBB5gzZw5TpkwB4JRTTuGll17i008/5fHHH+eUU05ptz2maXLdddexbt06XnvtNe677z5uuukmdF2nsLCQ4uJifvvb31JeXs5bb73Fbbfd1unXPG/ePBYuXMgjjzzCxo0befzxx1m4cGG7z7nyyivp168fF154IUuXLmXz5s0sXryYefPmsWPHjg61v6VXX32Vxx57jNWrV7N161b++Mc/4jgO48aNA2D48OGsWLGCLVu2sHfvXhzHYe7cuVRXV3PFFVewcuVKNm3axKJFi7j22msPOsm4uR/+8IesXLmSG2+8kU8//ZTPP/+cJ598Mm2WWHMzZsxgz549PPzww2zatIknnngibXZYRwwfPpyrr76ab3zjG7zyyiup9/ivf/1rapvx48fz8ssvH9RrE0KItkjQ00MmlExg3tR5HDvgWCLxCPsa91Edqea4suOYN3Vet9XpqaurY+7cuYwfP56vf/3rTJ8+nUWLFqWGIQzD4NNPP+WCCy5g7NixXHfddRx//PEsXbo0rVYPuFOp//rXv6bNVvrqV7/Kl7/8ZU499VQ+/fRTHn300Xbbc9ZZZzFmzBhOO+00LrvsMi644ALmz58PuEMozz//PB9++CFHHXUUt956Kz/96U87/ZpPOukknnrqKR599FGOOeYYXn/9de6+++52n5OVlcU777zD0KFD+cpXvsKECRO47rrriEajadPx22t/SwUFBbz00kuceeaZTJgwgV//+tc899xzTJo0CYDbb78dwzCYOHEiJSUlbNu2jYEDB/Lee+9h2zYzZ85k8uTJ3HLLLRQUFGQMrA7U2LFjef311/nkk0/40pe+xLRp0/jb3/6Gx5N5xHvChAn86le/4oknnuCYY47hgw8+4Pbbb+/0cZ988km++tWvcuONNzJ+/Hiuv/56wuFw6vENGzZQV1d3wK9LCCHao6mW4y19WH19Pfn5+dTV1XWq7kxnRK0o//7Pfydmxbh12q2MLhrdLT08outdc8011NbW8sorrxzqpgghxGHFjIR48KHzALjzjlfxBbu2VEVHv78lkbmH6ZpOQaAAgOEFwyXgEUIIIXqIBD09zGf4mD9j/qFuhhBCCNHnSNAjRAc1rywshBDi8CNjK0IIIYToEyToEUIIIUSfIEGPEEIIIfoECXqEEEII0SdI0COEEEKIPqFXBD3vvPMO559/PgMHDkTTtFbF36655ho0TUu7zZ49O22b6upqrrzySvLy8igoKOC6664jFAr14KsQQgghRG/WK4KecDjMMcccwxNPPNHmNrNnz2b37t2p23PPPZf2+JVXXsnatWt54403ePXVV3nnnXf41re+1d1NF0IIIcRholfU6Tn33HM599xz293G7/czYMCAjI+tX7+ehQsXsnLlSk444QQAfvnLXzJnzhweeeQRBg4c2OVtFkIIIcThpVf09HTE4sWLKS0tZdy4cdxwww3s27cv9diyZcsoKChIBTwAZ599Nrqus2LFijb3GYvFqK+vT7sJIYQQ4sh0WAQ9s2fP5o9//CNvvvkmDz30EEuWLOHcc8/Ftm0AKioqKC0tTXuOx+OhqKiIioqKNvf7k5/8hPz8/NRtyJAh3fo6hBBCCHHo9Irhrf25/PLLUz9PnjyZo48+mlGjRrF48WLOOuusA97vD3/4Q2677bbU7/X19RL4CCGEEEeow6Knp6WRI0fSr18/ysvLARgwYABVVVVp21iWRXV1dZt5QODmCeXl5aXdhBBCCHFkOiyDnh07drBv3z7KysoAmDZtGrW1tXz44Yepbd566y0cx2Hq1KmHqplCCCGE6EV6xfBWKBRK9doAbN68mdWrV1NUVERRURH3338/l1xyCQMGDGDTpk384Ac/YPTo0cyaNQuACRMmMHv2bK6//np+/etfE4/Huemmm7j88stl5pYQQgghgF7S07Nq1SqmTJnClClTALjtttuYMmUK9957L4Zh8Omnn3LBBRcwduxYrrvuOo4//niWLl2K3+9P7ePZZ59l/PjxnHXWWcyZM4fp06fz29/+9lC9JCGEEEL0Mr2ip2fGjBkopdp8fNGiRfvdR1FREX/+85+7sllCCCGEOIL0ip4eIYQQQojuJkGPEEIIIfoECXqEEEII0SdI0COEEEKIPkGCHiGEEEL0CRL0CCGEEKJPkKBHCCGEEH2CBD1CCCGE6BMk6BFCCCFEnyBBjxBCCCH6BAl6hBBCCNEnSNAjhBBCiD5Bgh4hhBBC9AkS9AghhBCiT5CgRwghhBB9ggQ9QgghhOgTJOgRQgghRJ8gQY8QQggh+gQJeoQQQgjRJ0jQI4QQQog+QYIeIYQQQvQJEvQIIYQQok+QoEcIIYQQfYIEPUIIIYToEyToEUIIIUSfIEGPEEIIIfoECXqEEEII0SdI0COEEEKIPkGCHiGEEEL0CRL0CCGEEKJPkKBHCCGEEH2CBD1CCCGE6BMk6BFCCCFEnyBBjxBCCCH6BAl6hBBCCNEnSNAjhBBCiD5Bgh4hhBBC9AkS9AghhBCiT5CgRwghhBB9ggQ9QgghhOgTJOgRQgghRJ8gQY8QQggh+gQJeoQQQgjRJ0jQI4QQQog+QYIeIYQQQvQJEvQIIYQQok/wHOoGCCGEEIcd04QHH3R/vvNO8PkObXtEh0hPjxBCCCH6BAl6hBBCCNEnSNAjhBBCiD5Bgh4hhBBC9AkS9AghhBCiT5CgRwghhBB9ggQ9QgghhOgTJOgRQgghRJ8gQY8QQggh+gQJeoQQQgjRJ0jQI4QQQnSSaZvMZzHzWYxpm4e6OaKDJOgRQgghRJ8gQY8QQgghupVjWzj1dcT37WHLuvdxbOuQtENWWRdCCCFEt1m/4lVeXPhffFS7nriu2Pbst5nw2hgunjmPCVPP69G2SNAjhBBCiG6xfsWrPPbiHVSZteQ4Hry2TrEvn49r1rH9xTuYBz0a+MjwVnczTbjnHpgxw/2vKQlvQghxuHMcRdSyaTQttlc34jjqUDep13Fsi5dff4y9Vh0Tc4cTwIOh6eT5cpmYN5K9Vh2vvP7LHh3qkqBHCCFE32SaMH++e+vEBWl5VQNPvbOJHdURdtREeOLtcp5cvInyqoZua+rhaNv65XxeV84Qfymalh5uaOgM9pewvm4j29Yv77E2SdAjhBBCdFB5VQML3tvCuooGPIZG0GuQF/CyZlcdC97bIoFPMw21lUSdONm+rIyPZ3uziTpxGmore6xNEvQIIYQQHeA4ikVrKqkOm4zql42ha6BBbsDDmNIcqsMmr6+tlKGuhNyC/gR0L2GzMePj4XiYgO4lt6B/j7VJgh4hhBCiA3bWRti0J0RZfgBN09Ie0zSNsvwA5VUhdtZGDlELe5ehE05ifP5otseqUMpJe0zhsCO2hwn5Yxg64aQea1OvCHreeecdzj//fAYOHIimabzyyitpjyuluPfeeykrKyMYDHL22WezcePGtG2qq6u58sorycvLo6CggOuuu45QKNSDr0IIIcThxHEUdZE4e0OxDiUjh02LqGWT5cs88TnoM4hZNmHz0NSg6W10w8PFM+fRz5PPuoYtRLCwlUOdWc+6+i/o58nnopk3oxs9N5G8VwQ94XCYY445hieeeCLj4w8//DCPPfYYv/71r1mxYgXZ2dnMmjWLaDSa2ubKK69k7dq1vPHGG7z66qu88847fOtb3+qplyCEEOIwUl7VwG+WbGL5pn2s+GIfj725cb/JyNk+DwGPQWMbQU3EtPF7DLLbCIr6oglTz2PeJQ8xpXAcYd2iyhOj2qznuMJJzLvkob5Zp+fcc8/l3HPPzfiYUopf/OIX3H333Vx44YUA/PGPf6R///688sorXH755axfv56FCxeycuVKTjjhBAB++ctfMmfOHB555BEGDhyYcd+xWIxYLJb6vb6+votfmRBCiN4mmYxcVxdmolfHa3jID7rJyLvqIlx7ynBGl+a2et6ggiCjSnJYs6uOUfnpw1tKKXbXRZk8KJ9BBcGeeimHhQlTz+P2SdN58N4zsOMmV1/5c0Yfe2aP9vAk9YqenvZs3ryZiooKzj777NR9+fn5TJ06lWXLlgGwbNkyCgoKUgEPwNlnn42u66xYsaLNff/kJz8hPz8/dRsyZEj3vRAhhBCHXPNk5NEl2fg8BpqmdSgZWdc1Zh3Vn6JsH5v2hrEdhVLQELXYWBWiKNvHzEn90XUtw5H7Nt3woOfl4y0uYfjEkw9JwAOHQdBTUVEBQP/+6dnd/fv3Tz1WUVFBaWlp2uMej4eioqLUNpn88Ic/pK6uLnXbvn17F7deCCFEb3KwycijS3O59pThTByQi2UronGb+micyYPy2+whEr1HrxjeOlT8fj9+v/9QN0MIIUQPaUpGDoIdb/V40GdQWR9tNxl5dGku1582ip3vBXEcxbfPGM2IshLp4TkM9PqengEDBgBQWZlevKiysjL12IABA6iqqkp73LIsqqurU9scKqZtMl9fwvzhWzCxD2lbhBCir+uqZGRd1wh4DLJ8HoYUZUnAc5jo9UHPiBEjGDBgAG+++Wbqvvr6elasWMG0adMAmDZtGrW1tXz44Yepbd566y0cx2Hq1Kk93uaWbBwWF9TygL4U05a1t4QQolu1s7xEMhl5d12UuG3yu6IV/K5oBZYTTyUjjy7NkWTkI1SvGN4KhUKUl5enft+8eTOrV6+mqKiIoUOHcsstt/DjH/+YMWPGMGLECO655x4GDhzIRRddBMCECROYPXs2119/Pb/+9a+Jx+PcdNNNXH755W3O3BJCCNH3JJORd9VF2LS3FttR6JpGQ9SislaSkY90vSLoWbVqFWeccUbq99tuuw2Aq6++mqeffpof/OAHhMNhvvWtb1FbW8v06dNZuHAhgUAg9Zxnn32Wm266ibPOOgtd17nkkkt47LHHevy1CCGE6N2Sycj/+HATK2yFo5xEMnIpMyf1l2TkI1ivCHpmzJiBUm1XwtQ0jR/96Ef86Ec/anOboqIi/vznP3dH84QQQvR2pgkPPuj+fPvt+91ckpH7pl4R9AghhBA9LZmMDEgych/R6xOZhRBCCCG6ggQ9QgghhOgTJOgRQgghRJ8gOT1CCCEOL82Tlu+8E3y+tIcdR9EQiRO3HSLVjQwq9Uq+jgAk6BFCCHEEqQ7HeOGdTQzftA/LcVj85kaGlxUy6yiZii5keEsIIcQRojoc4+NttayraCDg1SnM8pEf9LJmVx0L3ttCeVXDoW6iOMQk6BFCCNH7tbO0BLhDWuVVYSJxm2FFPv5YuorfF39A0KcYU5pDddjk9bWVOE7bNeHEkU+CHiGEEIe9nbURasImuX4Pmpaev6NpGmX5AcqrQuysjRyiForeQIIeIYQQh72waWE5Dl4j89da0GcQs2zCbayuLvoGCXoOZ6EQzJjh3kKhQ90aIYQ4ZLJ9Hjy6Ttx2Mj4eMW38HoNsn8zf6csO6l8/Ho9TUVFBY2MjJSUlFBUVdVW7hBBCiA4ryTX4n8EfE45ZHGv/W9pjSil210WZPCifQQXBQ9RC0Rt0uqenoaGBJ598ktNPP528vDyGDx/OhAkTKCkpYdiwYVx//fWsXLmyO9oqhBCiL9hP0nImuq5RlO3FY2hsrg5jOwqloCFqsbEqRFG2j5mT+ku9nj6uU0HPf/3XfzF8+HAWLFjA2WefzSuvvMLq1av517/+xbJly7jvvvuwLIuZM2cye/ZsNm7c2F3tFkIIcaQKheDpp91bJ4bug14PZfkBxpfmYtmKaNymPhpn8qB8rj1luNTpEZ0b3lq5ciXvvPMOkyZNyvj4l770Jb7xjW/w61//mgULFrB06VLGjBnTJQ0V+2GacM89sHQpnHqq+3OLKqVCCNHrmSY89BDU1kJ+fqefHvR6uHb6CHavCuI4im+fMZoRZSXSwyOATgY9zz33XOrnhoYGcnMzR81+v5/vfOc7B9cyIYQQfZJSCuUobNuhorqRQQWFnRqW0HWNgMcAYEhRlgQ8IuWAZ2+deuqpVFRUdGVbRFdwHFiyBB54oMNj4UII0VuUVzXw0ZYaYpZDfdTi10s28eTiTVJNWXSJAw56pkyZwtSpU/n888/T7l+9ejVz5sw56IYJIYQ48jiOYnt1I59X1LO9ujGtQnJ5VQN/fH8rVeEoALoGHl3js511/PH9rVSHYwCYtsl8FjOfxZi2XNyJjjvgKesLFizgvvvuY/r06bzyyiuUlpZy99138+KLL0rQ0xvYttvbYxgZVyEWQoieVl7VwKI1lWzaEyJq2QQ8BqNKcph1VH9G9sth0ZpKttU0MiVuYyuFaTms3VmHt8RLtNFmU1WYwuFyLhMH7qDq9Nx///34/X7OOeccbNvmrLPOYtmyZXzpS1/qqvYJIYQ4ApRXNbDgvS1Uh03K8gNk+YI0mhZrdtWxqy7CuUcN4OPtNextiBIzbTRA1zT8Xp3KUAy/Y7G7LkJ9NBeptCMO1AEPb1VWVvLd736XH//4x0ycOBGv18s111wjAY8QQog0jqNYtKaS6rDJmNIccgNevHacc/76a76+aAF1dWH+ua6KbfsaiVsOPp+Req7XY1CU7cO0Yizot5IfG0tkSEscsAMOekaMGME777zDCy+8wIcffsiLL77It771LX760592ZfuOCEqBoxSNptVqDLvbJYe5OlHkSwghutLO2gib9oQoyw+0WgwUNAbkBfhib4iGaBy/12j1fE3T8HsNHEdhyyrp4iAc8PDWf//3f3P55Zenfp89ezZvv/025513Hlu2bOGJJ57okgYe7sqrGthVGyFmO+ysjfDE2+WMGljPrKP6S6EsIUSfEDYtopZNli+IHjf50nO/QXNsUKAMg6BPx1EKn1cnFrdbPV8pRSzukKNrGDL9XByEA+7paR7wJB133HG8//77vPXWWwfVqCNFeVUDzy7fRti00ICgxyAv4GXNrjoWvLfloKdghswQM45dzYxjVxMy91O1VHp8hBCHSLbPQ8Bj0NjGCucR0yHb72FwQRYeQ8M0mwKfuGVTHTbxeHQCHl2CHnFQunyV9eHDh/P+++939W4PO6kx7EaTLK8BGqBBbsDDmNIcqsMmr6+t7PmhrsWLpYaPEKJHDSoIMqokh911UZRqec5TVNRHOXpQAdNGFlOSG8DvN1C4aQGxuENJrp9+OX5yAl78nsxfW46jiFp2Ko1AIcNgorVOBT3btm3r0HaFhYUA7Ny5s/MtOkIkx7D75wZwI54mmqZRlh+gvCrEztpIzzZMihcKIXqYrmvMOqo/Rdk+yveEMS0bRylMy2Zf2KQoy8eso/oze/IAhhZlEfAYGJqGz6MzcVAe+UEvQ4uyKMr2otG6p6e8qoGn3tnEjuoIO2oiPLX0C3bVRonEM/csib6rU0HPiSeeyLe//e12V1Gvq6vjqaee4qijjuLFF1886AYerprGsFsn5QEEfQYxyybcRnevEEIc9pJrAs6YwehfPsy1Jw5kUlke0bhDXWOcqOXQPzfA108exujSXEaX5vL1k4fhz1GM+06Io79eRShex9GDC7jypKEEva3TUJNT4ddVNOAxNIJeg5yAl3DMYnddVCo5izSdSmT+8pe/TE5ODueccw6BQIDjjz+egQMHEggEqKmpYd26daxdu5bjjjuOhx9+uE8XKWwaw45nfDxi2vg9Btm+gyqVlGLaJvP1JTBiK3duGcJ+y3eFQlBSApoGn38OAwZ0STuEEKIto0tzGdkvh4a3iombcbyGTl62H63ZpI6hxX5+O3oDdQ7kx+Hqk4czeeIorFi41f4UijfXVVIdthnVL5v1iXyfXJ+HoM8gYtq8tb6S8UMGyPpbAuhkT8+f/vQnfvCDH7Br1y4aGhooKytj7969bNy4EYArr7ySDz/8kGXLlvXpgAeaxrArG6LQYmxZKcXuuiijS3MYVNB1ZbZsFIsLa3lg+DZMWs+AyCgahUcekaEuIUSP0HWN/KCXfjl+8oPeDFPY3WsxLfHfIYXugqGZcnZilsPmfY1tTIUHn0fni72NPZ9GIHqtTnUzDBw4kNWrVzNr1iwikQgPPvggpaWl3dW2w1pyDHvXnn2Ux20w3Ho9DVGLytoQRdk+Zk7qf+ivPpSC55+HDz6A116DnJxD2x4hhGihvKqBf3zo5uw4SvHU0i9oqI0S9BlYlk2WzwPx1hduuqZJGoFI06menu9973ucf/75nHrqqWiaxrPPPsvKlSuJRCSKzmR0aS5XfGkoAY+RKk5YFzGZPCifa08ZftB1ehxH4SiFrRTbaxpRSuEASwprecB4t+O9PY4Dn3wC557rDnsJIcSBMk23NEYXlcfYtDfUZs7OvlAM03LanArvKNWlaQTi8NepoOfmm29m1apVzJ49G6UUTzzxBNOmTSMvL48JEyZw+eWX85//+Z/84x//6K72HlbKqxp4+/NKTNsB3Flbxdl+zp5YetABT3lVAwve3UzMcohZNn94fwu7a6M4raaDCiHE4evdDXupDpuM6pft1ujRmnJ2lIJY3GFXbSTDVHgwLYeR/bK6NI1AHN46Xafn6KOP5q677mLUqFEsX76choYG3n33XW655RYKCwv529/+xqWXXtodbT2sNJ9R4NU1dF3D79HZWRfhD+9vPagZBcl9f17VkFqULzvgJRy3MC0H21HYODygv8N8fUnHe3yEEOJAhEIwY4Z7O8jeYsdRROMOCnf0fVtNuN2cHb9Xx+812LQ3jO0oN43AtIiYNh5D48wJvSCNQPQaB9znl0xeBpg6dSpTp05N/Z4p4u5Lmi+uN6pfNmt1HRQYusaoftlsqnMLE47sl9PpP8bm+x5RlM3Hiafn+DwEPQbKAcs5wLJcjuPW71mxAk491Z1q6tvvPDAhhOg0E5sH9XcBnTvt2/HhS+Xu7KqNQJE7PLWxooGjS2yyM5yKdM29mPzy5DI2bKtgha1wlEMoGifb76Eo2yvL/Yg0XV6RGcgYkfcl7S2ud7CFCdtduE9zZzs4yp3VIIQQ3cnZuw9ryFDi48YTDWceYuqotHo7iW8mDY1Q3OajbTXUNrbOD0rm7Ewoy+P600YxuCjI4MIg1586koEFgYx1fUTf1i1BT1/XVJgw8x/cwRQm3N++NRKruieWt3CHuZZ2fphL1uoSQrSj/Ivd7D7tbFRFBQ2RODvqI+yqPbBigC17xw2t6aupLD9ATWOc8j2th82a5+zoukbAY5Dl8zCkKCtj5WYhJOjpBvtfXO/ACxPub98Kt7dHxrCFEN2lfMc+dnz/HgIVO9EAj67h0TTCpsVzH2ynOhxr87mmbTKfxTygL8XG7ZFurwd7REkuBUEv2/c1YlqO5OyIgyJBTzdob3G9gy1M2O7Cfcrt5UmOczdn4/CA8S7zR2zF1GToSwhxYBxH8cbaSiJxG4+eGofC0Nzp5DWNJpuqwp0a6mqvBzs/4OW4YYXkBjzEbYdo3E7l7JTlBw5Zzo5jWzj1dcT37WHLuvdxbKkFdDiQAc9ukCpMWBdh095abMcBDWxHsWlvmKKCggMuTNh835v37E107UDItIhYNpruXnVl2rONw9LCWmxNcQ/H7X+piiTThAcfdH++805JbhairzJNGu65n1Ebq8jyZD5/leYGqA6b1Ect8ju42+Y92NkZdhv0Gowvy0OrDuLRNa49dSTPbw0csiGs9Ste5cWF/8VHteuJ64ptz36bCa+N4eKZ85gw9bxD0ibRMdLT001Gl+Zy7SnDmTggFytRRNByFJPKcg+6MGFy3+NLc1G4yXzhaJxsnwefR8fQNRQQjTs0mjbRuE0fn1AnhDhYpgkPPIB36RJsy8LbxkVb0KtjOQ5xu+M9yu32juP2jo8qySY34DnkOTvrV7zKYy/ewcc1G8hxPJTG/RT78vm4Zh2PvXgH61e8ekjaJTpGgp5uNLo0l+tPG8Wgwiz8HoNBBUG+eeqoLumOHV2ay7XTR+D36Pg9BlefPJyy/AC6pmE7it21EXbWNLq32gi7aiNE4u4I+pKCOh4ILMfUOxkJSXKzEH1TIuBhyRJ03N7kuJP5/BGJO3h0Ha/hfr2Y2MwfvsWdTGFnPm8ke7CLsn1uvR3VFDBt3RemKNvHmRP6H/LkZMe2ePn1x9hr1TExdzgBPBiaTp4vl4l5I9lr1fHK67+Uoa5eTIKebqbrGgGvjqFpBLxGlybc6bqGrmkYmsag/CAxy8FyHGK2Q9i08OjuMT26m2BYWd8FFZttGxYvdk+AEvgIcWRLLinxwAPu3z7g9+gUBn2E2phMUdUQpSjbR17Ag2mbPKAvZXFBbSppuS3pveNN948rOfje8a6ybf1yPq8rZ4i/FE1L//rU0BnsL2F93Ua2rV9+iFoo9keCniOArRTPLNvKztoIsbiDbSt3XS7cOheGrpHlNbAchZXocm5QMUq+00D+bSY7fPEDK2YohOhzNGBkaTZBr4HlJKIT5Z6HInGbwiwfo0qzD6heW7J3fGBBwD2WBjMmlDCyX+9YCLmhtpKoEyfbl5Xx8WxvNlEnTkNtZQ+3THSUBD2HOVspTMvhX3saUh2/mgaWrWiMWcSTJyU0fIaOrcByHKoazFSgUxeJY1oOkY7WDXIct7fn7LPdqs3S4yPEkaW6GoYPhzFjIMOC0kXZfiYPyser6ygFcdvtYc72ebjiS0MoyvYf8KG/2BuiOuSeUxwFf1y2lScXbzqopXu6Sm5BfwK6l7DZmPHxcDxMQPeSW9C/h1smOkqCnsOYk+i5USiGFWejJ66sNA0MQ0usYWOTjG50zZ0yb1oOdWE3wIl64Oz/V8eZl4XYEA61W1+jFaVgyRIZ6hLiSGKa8NBDUFtLR2dAqAw/HYjyqgaeXb6Nxrg7lKYB2QEva3bV8ezybUTihzZXZuiEkxifP5rtsSqUSh+uUzjsiO1hQv4Yhk446RC1UOyPBD2HsZ21ERwFOpr7P41ESWb3cUPXsG2FnThx2UrhKPfqqeVp6rP+8I3TN7OqYm+fXztNiD4rmbD87rvtBjzV4Rif7awj7jhoGvgMHb+hEzZtnlnxBXfE/8l8FreZuJxJqipzo0mwWZ2xHJ+HMaU5VDeaVIfjHODKgl1CNzxcPHMe/Tz5rGvYQgQLWznUmfWsq/+Cfp58Lpp5M7oh1WB6Kwl6upljW9h1NRTvi9C4cyubP3unyzL7G03LPQEkxrUMTWse86BpGgqVOHcpYnEn9ahuZB5vr6yNyrpdQog2KeCLqnCbxQlrDzA4SVZl7p8bgBaztDRNo39ugIhpH/Lz04Sp5zHvkoeYUjiOsG5R5YlRbdZzXOEk5l3ykNTp6eUkHO1G61e8ypMv3sVb1npqSxQfsJ23/vwVTn9pAt+++IGD/uPI8nncKZypKMetxqyUwrYVSnMfcJSiMd50otA0Mk/91CBquRVPAwfVMiHEYSVZgNS2U7O02hKzHGoiJjnNqiebuuK/jqoA3aAwx2DfAQQnyarMA4JGxseDPh1HqdS6gofShKnncfuk6Tx47xnYcZOrr/w5o489U3p4DgPS09NN1q94lfufv5FX7XU0ehRFESiOQki3+Xv8M+7/y40HXcRqUEEQXQM3q8c9ESTX3fIYWmpmhcLtIi7M8ra7PwfF34ev5z/HbO/c4qRCiMNTKAQzZriTEjqYl+c4bqHV5sUJTV2xrDTEspJ6vB77gIKTpqrMmc89EdNB17QuLftxMEtJ6IYHPS8fb3EJwyeeLAHPYUKCnm7g2BYvLvwFa6gkoDT6RTV8DvgdnYFOEL/SWaOqeHnRowc11OUGN25Gz9Z9YezESUYDdF0nJ+BlUKG7AnFZQZDcgNcd8lJk7npWoOvwfr96HjDePbDAJ3kSnTHD/VkI0XuZJmzZAlu37reHRylF1HISi34qTMehwbA586o4l5xfg5WowxONH1hwkqzKXNkQpWVCtFKKyoYoQZ/Ral3BA7V+xas88rOv8FHtelZFynng2W/znw/OkYrKRzgJerrBtvXL+ah2HY6myHO8LYaSNPIdHzYOq2rWHnQRK0PT8Hl0xpYkl7tI9uy4FaCLsn0EvAaaBn6vgaG5WT+2lfkqLDvgbntQbBtWr4bZsyXwEaI3Mk233MT550NNzX5naVWHY3y4pYadNY3sCcWoCZus2VPLRefvY8Ug2JprE3fcFdD3HmBwkqrKnOUj0mxoLGRabKwKUZTloyi75fn0wMhSEn2XBD3doKG2krByp357Veu32IcBmiKsYl1SxMrQNK6aNoxBBcHEshQ6AwqCBH3pY+PRuJ3ID9SwnPTZW+D28vTL8aOhuauy60s7PQNDCNHLJQOexx6DTz916261ozoc45PtdVSFo3h0zV2CApu/D1/Pzrxm5xGlsBwHQ9cOODgZXZrLlScNJcvrnrsUEI7GmTwonytPGkrQe/BDSLKURN8mQU83yC3oT7bmFueKa61PKCY2KI1szd9lRaz0xJITuuYuTdHydBMxbSrr3cX8DF0j4GtZQt2ddtoyUDoopgkTJkBBAdxxh9TyEeJQC4Xc/J0//7lDNXgUsDkxU6s46MPQNaJxG12DgLfFDCvcCRKF2X4C3gM/j4wuzaWsIAi4tcWuPnk43zk9fc1Cx1FELZtG02J7dWOn8odkKYm+TYKebjB0wkkcVzARXWnU6y2nbirqdBMDnRMKJx1UESsnsXq7rRTbaxrbrK+jlKI6bGI5qllAlH7C6tb5ENEoLF0qQY8QhxnbUdRETXL8HpI9xJaj8HrcdQRTZxHNrQvm0TXC0fhBTyt3Z5i6/x1SmJWWHxSJWyx4dzM7qiPsqInwxNvlnarYLEtJ9G2Sbt4NdMPDJbNvYc3z/+IDbTfRgCInBjoO1XoEW1N8iQFcPOu7B5zxX17VwP+9v5mY5c7d+sP7W6irjeJoipb5gzHLIRK3MTTNLU7oKGy72RT2xH9N2yGSYeaEaZs8yGLQHe60p+HrbGOVchMlJ050S9u/9hrk9I61dIToE0IhOO88sKwOV1mG5JCVItvQiNsWj0zayfLiBoZaVuaLLA1ilo2nm6aVR+IWu+ui1Fc14DE0d1gqUbF51559WHFrv0NgzZeSyPFmt3pclpI4sklPTzeZMPU87rv8V5xnTCTL0qgOwr4g5CiDC31Hc99lvzrgOj3lVQ0seG8Ln1e5623pmkZ2wEs4bmFaTmoWV1KjaRMxLRpjFpbjuFPcM5yTHAW1jWb3VTxNBj8PPii9PkL0hFAITjvNveD44otOBTzgFgX06Bpxu6kkBri7cZxmPcSKxILGitpIPLUkTldyHEV1OI5lK0YUZWPobndQbqBzFZtlKYm+TYKebjRh6nk8fO8SLvBMYOoeH5fWD+H5K17iZz/+4IADnlSp9rDJiKLsVDdNjs9D0GOgcE8+yT/7iGlT02imBUIapJamgPQTV12j1Spo6lJKuUNdHVyvy3EU26sb+byivtNj90L0acklJT75BFVbi5XIgYlaTocvawxdozDgo8aM8fuSD/mgJASJBY3jdsuAwS2EWr4nTEVdNLVOllLuMHyjaR/U3/DO2ggR08bn0Vut4N6yYnN79XdkKYm+Tf5Vu5lueDDyC9lHkIn5wxgx+bSD+mNKlmovyw+gmdH0BzX3SsxRipjlYHgNqsMmjqPwew1iqcX6tDYrMluOcoe54nZiHL+JjcMDxrsYI3Zw5+bB+DLMTHMchRm38TqKuOXgVyr9SMneniVL3ORmX9uDZeVVDSxaU8mmPSGilk3AYzCqJIdZR/VPS2oUQrTNVgrLcghHLfbVRTA0jZKoRZajaGM1mpS4rvjr8M/YFTaJxuMor9uz7Dgq7eIoVRwVCHp06qJxHBRL/lXJ7tooMdthZ00jT7xdzojiLCJxC5+nc71BYdPCUQpDy3ytnqzYbFfv4xe/vJKPatcT1xXbnv02E14bw8Uz56UuNidMPY95wIsL/4vl9e9TqylILCVx0cybZSmJI5gEPYeZZKn2LF8QlaGjRINU3k4ssaSEz6PjURpxy3IvxzT3yisTQ3fjktqwSbbP1zLfGRvF0sJabBzu2TzUnX6fUF7VwJsfbOKc2ggDLJvdDY2UxiyylWrdpaiUO8zl88Gdd7YKfpJDeNVhk7L8AFm+II2m5Y7d10W49pThEvgIkUmyh2fJEiKNUUgUE9Q1CHoMnERhQa/tgKElane1LegzKDX8VNa5QQfKvbhKrbtFcuaW2+NSnOVD4fYy//6dzfhMC82AgNcgL+BlXUUD4boopbneTs0WzfZ50DUtrZe6ucaISaCugk8Ce9BrK8l3dHy2J1V/Z/uLdzAP0gIfWUqi75F/3cNMU6l2i2CGxxVNS1E4iW5lQ3NPbNkBD5rSUt3NSVqzJxuGjqFpRCy3m9jTwazl8qoGFqzcRWhfA7N1d9q8mwvg5hjZjsLbMvKxbVi82P3vPfekAp/mQ3hjSnNSXdm5AS85fg8bq0K8vraSkf1yurQkvRBHEgXUNJoUJM4Byb8jQ9Px6u6kBstW6J7W/b6m5vDgtCh2QBEx49RGHeKOk0qI0DUNn7/58zR3xpXm1vsylcKxFVUNUUYnihRqaOQGPOT4s/nIVtQ0xt2p7R38Ex6UqD0WjrVOoo6Uv0X5J79jdeEuagLQzwlR77cZaGrk+XLJD+Sxrv4LXnn9l4w7YXYqsEkuJaGDLCXRR0hOz2EmWap9d1209ewJ5Xag6JqG36OnavYkr4y8uo5HT9bxyTy8FfC6yYGOos3eIAfF4sI6zj7uM+4ZsYWYsnhjrRukjOqXSDDEPbl6dB2l3B4qy1Gt2+w47lBXsxyftCG8DGP3ZfkByqtC7KyNHMA7KMQRKMPyLzHLIWrZbSQVa+66fSrzpIaQbvH0RJM/jA6zoyFMOG7h0XWMxPnDURCLOc13B7jnn8qGGJbtDpM3xi1CMSvtGFqiinwk3rlFSfVE0UOPobG52l12Rymo2fA2K1Y/zHatHNOA0ghkOQZ7/Q7/yoqyJ7pP6u+IFAl6DjOpUu3ZPjZXh1NZyCHTImLZaIAnUT/D79EJeA1M2yG5YbIXyMjQQ2JomttlrdyiYPubgaGAJYW13O0sYeOe2lZBStxxEseGWNyhLhKnIWq1SoDEstzqsGPGQHV1syG8zFddQZ9BzLIJm1IxVQhCITjnHHj3Xfj441TQk6zjlZlK/b8b+Li/m9jcYyzh/CnrqQm4j1mOIugxUucMd0kb92Kq5d4VipDpFi/UcM8nVqK3N96s8rMbOCmi8c4VGAx6PZTlBxhfmotlK6JmnO3lzxMzwowKlKADPqXjx0OhCVFD8a/YLhSO1N8RgAQ9h6XRpblce8pwxpfmpk5a4WicbJ8Hn0dPOzkVZfvw6jqNcTvV46Npbu5OS8l4xcE9yXV07RxHqVZBigIiMSd1Ikt2fcdth1DMIpIpYFEKHnqIsp8/RDYOjW0ENRHTxu8xyG4jKBKiTzFN2L49MYe8KXDQE70yLcUdh5jluGv1OQrTcmiIWlSHY+kbJnblNbRWQ1ABr4Gn+YVTopdZKcjy6ngSJ5iA18CbOI9E43ZTcGXZxOI2VQ3RThcYDHo9XD1tCIM8JqXmPmq8WxmfV0auL4ihNKxEszR0suOwR49SF2uQ+jsCkKDnsDW6NJdrp49IrLVlcPXJwynLD7Q6yQW8BoVZXnyGTsyyU3k+zXtkkqdJy1GEYzYaGgXZvlZDS23RNS2VZ5TkOO4ZMJlzk6wn5DF0HAU1jfEMw3MKliwh79ePc8tPvsO+yupW2yil2F0XZXRpDoMKMmU1CdFHmCbMnw8PPZSx/o7foxNIJC4nWY6iMWal7tMSQ+Bxx+GznfWtAh8FGROdPbpOMKCnxUIqkeBc5Pe6idO6hq/ZBZhtKxqiNjXhGA1Rd8jLZ+gEEwnOa3bVseC9LfsNfKzqvTz2xP9jbehfrPLs4hPfXj6JbiHuWBSYOiGvk5pN5lVgaw5ROyb1dwQgicyHNceKUVITQ7cddn/8EppKr6YcMW2qwyZRy8FOdS1roFTGrmQ3sdHBY2gEO7F2jt+j0784i3X7ohTkaO5YO9C8NLR7cnVPjMn1e+qjFvkt9qUcB6JRBmz4jJsfnstj33+CfgOKCPoMIqbN7rooRdk+Zk7qL0nMou8KhWDOHNi2DS6+OOMmGlCY5YNEXp9SikjcSi02rOH2+GqA49F4quRD/JaH0mY115M1vbwZ9q9rzXqAtKalI0DD5zHwGhqegA/bViiP2yNcG42xYVcDftyJCZ5mBQbz8rLYWBXirfWVmYun2haNO7fyibMLjd3kOzoB08c+22Kn0UhDfAulcZ2wx2GPP0ZQV9goHGCbuYdhvlKpvyMk6Dlcvfqne3l05S/5V3GMuA4frv4xZQ1QFtGpKg4QidvsqY8Tdxz8HgOfYRBPdGe3LE3W7LyVqLTauZrMmqZx1sT+VHy0l017a92TamJqa7KWR7PZrYlp9a2Lm8Vth331UYptBcph5MbPeOQ7Z/LDh1+kcsAg/B6DyYPymTlJ6vQI0RFBr4FtuHk1VmIoS0/M5jR0jbDH4eKvRYkbJiOMEhqbJxcnTgxxWxFodQ2kiNtNNbj0ZK6gpnH08CJ2V7tfLSMGF/D5JjdP0FGKmnAcj6GTG/DgMUhb9C85SeGLvXXYlpM2DrF+xav8zz9+xmv+rVT7VbPZWT7KnCx2a4006ha1XofRYT81Hh+7jXrqfVBq65xSeCxfmf1dqb8jDo/hrfnz56Mlplwmb+PHj089Ho1GmTt3LsXFxeTk5HDJJZdQWXnkJqu9+qd7uePDh1jvqyM7Bv1DkG952Jxr80lBnJJ9UWrCJnHHIcvrJiBqiUoaCtzx92b7a/5z3HZPZvWReKfalMwzmjggNxU0xS336lKhsG23bpCTSH7UNQ1vs8QiOzG0Fo5bTYsNAoZj8+U3n+eiSSXces7YVqstC9FnVFe7a9cNH+7+3EGG7s6WCnoNvIY7ucEwdBTQiMOWfMX2XCuVa9d8mNqtx+OWsEhewCggYjnuhIlmna16YtZn826a/ICXXL8HTYOCLB9XnTSUIcVB/J7MPcnJSQrNh+Q2rn6Lx168g/drP8PUVYvZWTH667lkKw9xFHv97vD8Ub4hlEQ9HFXr48Ev3ckP7/6HBDwCOEyCHoBJkyaxe/fu1O3dd99NPXbrrbfyv//7v7zwwgssWbKEXbt28ZWvfOUQtrb7WGaUx1Y9Tp3HYoSZRZYNBpCjPJREoDoI63PjWDV78OnQPAMxlR+jtZiOrpKPuz8qpdgTimVONm7H6NJcrjipP9fO2sScf4sS8TpNAZfm7te0HCzbSRQq86TaZdkOtnITqJNtdP+jOHb5P5n09UsY5LFlSEv0TRUVMGyYW828urpTa9fZiUKlYdMiajo0RCyipk2jsvnZtDh1AVCJ4SnbUe6yNcmcHzRKc/1kez1u4nNiinuW16Ag2DTopVRTmYt3y/dQF4kTtx0aErNKdU2jf16AkSW5BL2eNgsMJicp6MpmXBWMrXD4zT8foipew1BvMRqtZ2dVOQ18yTeCQXYWEQ9U+kyqrQbGxHM4MXsMZ118mwxpiZTD5pPg8XgYMGBAq/vr6ur4/e9/z5///GfOPPNMABYsWMCECRNYvnw5J510ZCWtLV/0e8qNekrjPvREzBo1YLevkdocMA2oC0DEu4uj9+6jn2cgTn4h4HYfo5qGsVKaT8JI9MRYtkNNY5xsf6fXVMdUig39HW6YvY0XX8hO5O24PXSW7aDpUJDlTSVKxywnleuTiYGicON6GDQQjjkGFi6UVdrFYc9xFDtrI4RNi2yfh0EFwcxBfSgEF12ECodTz9tV08hg9l/Xr6bRxJeYNWXZiaFtBfu8FudeF6M2QFPvr3KXq3EcJ5X3A+55Iz/LS9APO2JuYnTccQiFLJxEp6tKnFc8uk6/bB+m5RC3HXbVNrqzStEJ+gwGFQQZUZxFheXQYpWb1CSF4o3/zcKGz9hWBKYO73u+oDiqc4xpZZid5bAnEGWy7uPowHC8+9ZzrDaQa7/2KH9Z9DO0NpasEH3XYRP0bNy4kYEDBxIIBJg2bRo/+clPGDp0KB9++CHxeJyzzz47te348eMZOnQoy5YtazfoicVixGJNsxXq6+u79TV0hco9W4jriizL/aeLGlCRDSG/QlPgsyFmQL0PVvWPMW7fdkbWadh5BXi0pi7otup3JAMRx1GEYhaxeMeLh4FbWDC5b12n6SSuFAq3SrOhp9cAcrfPsFRFqk1AcmX4Tz+F2bMl8BGHtQ6tKxcKwXnngW0TaYziTwQWMcvm6fe38I3aCKW2kzHJGKDRtNldH2FIs/XvHAN+c+wePiltaAp4SFZy19wKyWip5SYcpdhZE8Fj2Fh2nKjHdpepicTRtfRxcicR+ZwwopjPd/qIxG2GFmVh5gfYVO+2QNfd/L+V/6cRsSx8uoamGTRELSprQzgf3s2vfAtpKGlql6NBRZZDKL6HASEIeRU5zWZnmYnZWXvMOoZa2WQNG8rQ8VPRXpeAR7R2WHwqpk6dytNPP83ChQt58skn2bx5M6eeeioNDQ1UVFTg8/koKChIe07//v2pqKhod78/+clPyM/PT92GDBnSja+ia/QvGY7X0WjULBSKej80+kBTYCiIG6B09791fvi4v8V2exs4bo5OR6ehxyxFOGZRUR9Nm0KqcLu47URSdMvQKW3auqa44uI6zvm6RSwAPo+Ov/lJNSFZIbqt8MrdNDFMZtuwciVMnNipvAYheovkunJrdtVRkOVlZL8cCrJaTNkOheDcc2H1aiKRGLsjsdQ0bF3TyA54CZsWoUz1dXB7TfaGYliWW57CDWrA9ig+LmtkV57d6jkBn45XdxftdJzEUDeKWNzBtBzMuJP4m1d4jcRszLSDNl1MGYZG0GdQ2xhvVXV5dGkuZfmB1JCZO5MzzrC693nFs5B6P+gKAjb4bbfdlgb1XtgXAL8Fe4wYpq6I6U2zs0q8eZTlDZLeHdGuw+LTce655/K1r32No48+mlmzZvHaa69RW1vLX//614Pa7w9/+EPq6upSt+3bt3dRi7vPSbOuY7SdR5XXJEKcsM898ei4PTyWDkbiHGPp0BCAd4daVNR/jlVTk5q6nhb6pM2gaCpSqGsQidv8ddUOInGLiGlTURslZjnEbPdmWk5a7k/zAoUOim15NssHwyUX1tDodRc+1FpUe3aXzCBt1ebmLEeltUuIw1XLdeVyA14MXSM34GVMaQ7VYZPF73+OmjgRPvgA5TjUNJpYDmlLx+T6PGR53aGmbfvCrS4+6qMWkXjTEhQhj8NXLq3jxplbsbXWf2fJOlpxx12NvWmGpzvLKxa3idukThzuxEut1U6UglDMDah0TWuVlJwU9HooKwgwqDCLwYVBbjhtOBu/+CU1AfA4bsCjJy7kvInAR2nu0H1pzEOZHSRiOOwNQsBxZ2fdcP6P8BT16/S/iehbDougp6WCggLGjh1LeXk5AwYMwDRNamtr07aprKzMmAPUnN/vJy8vL+3W23l8AeadcBP5lodt/himOxuUuA627p4odAWW4Z48vLZ7/6f94lQYuygOxzF0HV/zkszJMXItvRCZ19DJ8XmoaTSprI9RURd1Z1cBRrPEx8r6WKo3aFBBsN3lK9yFR/W0as9aomhhcqFTIDXDzAEc2+1dMi2nzcBIiMPB/taVG5RlMOK/n0DV1LpDwgqilpMq8pfODUhqIvFWfxdx20kVC+woDbdqsuWotnOFEkNs7vEy/y0mL6wcpdyk5DYaoaHwRRrwNtSwfdVzfGbtwNHc81bzNnkd0B036InrYKIyz8468dyOv1jRZx2WQU8oFGLTpk2UlZVx/PHH4/V6efPNN1OPb9iwgW3btjFt2rRD2Mruc97/+xEPHX8HY6I52ImhLEt3Axxf4grJmzhJWIYbONQF4NP+NlVZFmU1UYw2ag+qxIlW1xK9Nhr0y/FTH4kTs+3E7Kr0Kstxx+Gt9ZU4jluF1WMk8oKSs8GAbbkOJ/7bXs77ajVWsPUwm6FrZPsNsr2exCrwpI7h9eqphVNNy0k9dqBTeIU4VNpaV06Pm5z8u59x83fmMOXd11IzLRVu8GC0EYVoWqIYaIveFK+h4zU0Gn0Os66McOK3Y2zNS/TgZI6fsJUiFrdbVWBOTfps9bzmfU9N+T1GoiiXaTkMK8wC3H1H403L0ljVe9mxdS2f1G1kVaScn7/zELs80dRrTntvEkNdyQ6qBp8ts7PEATssgp7bb7+dJUuWsGXLFt5//30uvvhiDMPgiiuuID8/n+uuu47bbruNt99+mw8//JBrr72WadOmHXEzt5o77//9iL/f8THTt0NxozvOXRhN1LZR7hh4zOMGPB4bbA2iHthQDB+U2cSsrQypSd+nSiwuCG69jGTwopQ7xOTRM39cfIbOF3sbU6ueJ0+aKrkYTzM78hx+elwDptY6g8dr6PTPCyRqMTXV6kkUbcXQtFROUWqvjgO7d7uzuu68s1NTeYXoadk+T6slWzyNYS65/SqOf2kB2dV70JolHicvLOw2OjiVUm5ZiBYRSV7AQ8BrYHtgW76b+5faRfPh7GbPsRx3X36fnhbOuLtuuidTEJS8uElOOoiYNpoG+yJxdtZGiFk2O2saeeqdTfxz0Qt8FNrIv/whdKUoifsoMfJwdDdwMhMXammvEzfoyY/CmeZg7vzaYwweNlmGs0SnHRbh8Y4dO7jiiivYt28fJSUlTJ8+neXLl1NS4qb4//znP0fXdS655BJisRizZs3iV7/61SFudffzBXMIerMYU9PIJ/0h7HWDHa/jBjwKt/dH6W7QY9juTIjaALw3yGJgA0zZCZsL3P1pmpYqOGbbDpat8HkgFHOToN0FBluffZNj92HTotBHs4VNk4k4Tc9xUCzuH+W+nK3MVyfhb7GvmOVgJKfVZ3jNuqahEonUh8WHV4hmBhUEGVWSw5pddeTpDtN//wjHvfInNOVgazpK09OK/mkaBDw6oVimvwb376Aw6G1V7kHTNHJyPCwYspe6xB9ZKmjJ8JelATleD/WW29uj/E3zutxyEhC3aPMPs3mQ1Biz8Hl04rbDnoYoHl1DT8zc/PyD13i+4iG2FZgELajx2xSYNsfoPsbF86j11RMz3PxEr+1elTua+7uhYNoeH7lDR8jsLHHADovvjeeff77dxwOBAE888QRPPPFED7Wo96jL9dKvPsCkBp11OY04GpgeQINA3D1pRD3uVVLccIe8FO5JZHMh7MqB43fCxwPc85k/MZTkKDeJ2We4OT2edooCJsfus30eHMetBwJNPTRAKslRKdiabfNPo4Y5W/dyQtwm2GzXjaZNTuJEq+O21bQUPprtS9GqO18pReyfb+G8tYT49FPJ/fH96IGWIZUQh5aua8w6qj/15V8w9+vn4lFNfRqJSYoEvAZaosNSw10/Kxp3Z28l/wYaTIvGuE22pjG0ODutx8a0TR7Q32bJl9YRs5sVGNXcYMjtQW2KXpI/VTeaaHgSuUDJUhPuMJXP60n9rmkafo+G5TRb0EaDgMfAbxh8fdownlinYVowoiib9boGDgQa69m6+3G2ZTeQF4Esx625s9fvsDK+hTF6CUMijWwNum22EucrR3N7q08M5TAld5jMzhIHRT49R4C6PD+/mPlTTqnwMiAEeRHIjkEw7gY34J7YLN39QQGWxz2phAPwzigwvZDfEMWOu706uu4mQ4ZMi0H5WeQFvZh25knlpu0wsl8WgwqCaXV60iQqoGnNfn0sfyX/PvgL6uLuGT5uO9RGY60WG1SKxCKmTYk+zbvzHSAUs9hVGya6cRP203/go2u/S/mOfZ1/M4XoZqOjNdwx98utAh5Nc3PbvC0uMIJeg7KgP9Wb4ihFOBonx+chJ+ChKLvt4F7XE52tmnsRYrRYOiJ5bBRYjtub60lebUCierv7t2kk6m7pOhiGnhp+TmrewxsxHXweHeXYeMJ1DKiOss3eQaVRS8DSCNpu749PaRSa0KhbVDn1zPCMYkjEw6AGyItCbhSGhwy+bR/N45f8twxniYMmQc8RYtxxM5kUGMlRezSCNsQ9bo9PcjaXncyRcdzHWor53ODn/ZJNDKhuxHbcQMOja5w5oR+luX68up6YXaVSY/iOcmdjnTnBXfW8sd2lK9LzBPwejWUlDdw3YjsxzSEad1BO64TJ5O+24x5Po6l6c9x2Z3SZtpM4YWvotsWUv/yWYSPKKP9w7YG/qUJ0JdN0885OPhndSv87SeastXVCDvo8qb8Dv8fgmpOHU1YQTFu/riOSy0i0HKNSkBo29vuMtC5ar+Guk1dWGHSrKxu6O2HB02wYDlITDTbtDeEoRSBUy+q3ruf9wCZWlFksHWJSnhuh2u8Q9jYdX0Mn1/GyV4sQt22mMogLIoMYsw+O2QP/M+dpfv4fH8rsLNElJOg5QmzaG6JKz2JvXoBjK3TG7XOTmw3ldg9rgM9yh7japMG+HHh9pMIT3pyamvraZ5XUNMYpyPLi9xipis7JpSP65/lTVWRbzkppn4auaViOe7K1HPfqsK0prskhLT3RRe84bsIkgFfXMDTdvaKlKcfg3X/tTc0YEeKQqa6GMWPg8cfdAps0Jeqnbu3ML28+nKsBg/KDab0s1T6b4d9VDP9mA9WRmlbPT+0n1bnU+ljJshCOcnt8ku3KCRoEfQbZPvfkkZwen+VLP5noiYkGn+2oJ9hYS7nayrKsL2j0Ql7MPf+g3GKqO3IhajT1dPmUjqU57LCrmZA/Cl9pf9YNgE0lOkMnnCSzs0SXkaDnCKBQvLthL3HHQdd1KouCHB8ZxrF7PRRG3boXhuPW69nvYj24Q13/NzpKdmgrWV6dnICX+kicXbURovGmSq7JFZiDzQKd5nV6kguYZpIa5tIUK0pD/HhajJimUjO/mvfwN58t4jX01GM76yLNaoq0fmEacOo938EZNkxmdolDwzRh/nx46KFWMxk7KhK32RWKpoaNY5bDM8u3Ekn8LVYT4ehLKtmWD9V+MBPV11tRrZOYm/+WzO2JmnZaU5PFRHXdraTuKNLrfDWjo1FbU0eNtZvygjiaUvSLaPhtd4JFwHYvxuI67MwGU1c4miKkxQnpNiV6Lued8R2QvJ0jjs/wMZ8ZzGcGPqPzazp2FflkHQGUgu21jWknonhuASO0oZxQ5cNvuz088Q4EPABobuDz97Eh9tWuIb71PSzHwbQdbMdxT4CJZGfTTq/I/MXeUOrEajstp6w3JU42v8fSFY8fZ3Hi1XvZ442n2pC80vR6tNQxm9cQCScWUmz3vXEUWk01/OxnMHq01PMRPcc04YEHYPHiVO9OZ9kKKhtihC0n1XupafCvqgYq6qLE28izy0hr6gFVbfwtQiKHrtnfVdxWBL2GW1A0cXereQ1Kkd9oM6AmSvknt7Gif4R9Qaj3OVRmKRwNAome5qDlBj/grqNV63XY57EYbxVw51d/IcNYoltJn+ERQAFRK1FUrFkMEM8poFDlceGWzfxjcB01QWgv46alqAHvDohRvekhhoYKUNl5OI5DMiHZ0NwyOTWNcRxHUV7VwF9X7WizjoeG5k4jazYNyy2EmLxDI56hIImeqP2haxpasxL62V5jv2uJJdf1QinYudOt5/PJJ1BU1Il3QohOqqiAceMgGoXjj8+4SatPulJpn2cFWLaD5TgEW1QnHFacTRiLe4+pwfavRFlN+3PaiYM0ve0OJw0t0avT1DiF25tbmOV1/2wTzUiNGCs4usJNOK7OivN5EcSNXcQT1eF9NjR63do7+TGIezTihrs4cpYJQ0MeYoZimGcQd176GJNOvhAzEmr7BQhxkKSn5wig4U4XtTOdzTQd1W8kc3b0Y1Q1bY83ZaA0d/z90+J6NmXtIDtSR7M5VKkZJ+GYxbZ9YRatqaSm0UxdBbYumpY+eytiOWiAJ5GjozTFt8/cxpn/L06o2Xi/u/aWW+m5+f4G5Qfx6FqiPa1fmFKKgKfFbBXLcis4FxS4X0xCdCXThFtvhUGDoL6+zR6elp9Whdsz6rS401Eq41BSzInwjTM38adxjWypC9M8be2Fj7elct1aHszQtIy9OwBoiQKgzYbBNKA0158awnZ7XN0eXn9DHSP3wqYCeH8YrCmBPUEwdYh43Jujuz07cQOiXhgUDxC0NWzdnT2qaRrjzFx+cNEjTDr5wozvlRBdSYKeI4CmwZCCrDamlLszm4zSoVygjWFyBR0LfBKzvrLi7n+/KHLY7N9JbmgvKCeVfGw7irBp8dulm/l4ew0luYHMbUz8f3LiiFJuT43P484EaV7vY8VgmHyjYl+iiqxXd2ePtCyPr+vuSs4AcUdhK3e9Ibcvyg2UCn3e1tk+tg11dVBWBvPmSa6POHimCffcA2efDe++2+6m8TYS6x2lsGwnlauT3KplYr9SUB+NJ5aMUURi6YHVF1WNVNRFU0PSyb+55LIwbeX1JWdvuTMkm4bStERB0CSvoZNVV8HrhTt4bzjUZLvFUG2PmzcYNRLr/+lQl0jd8FluEKQDOY6HkkY4eW+AL+WMYfCwyTKkJXqMBD1HAA2N6eP64dX1xBWf2x9jOw6NcRuvrlOY7cNTWExBIIcZW0BvL8UgcX7zOm4SdMzjVnveUOzwUfEe+tdEKQ5biWO7J+Wt+0L8q6IBy8q84+bLSrjT1XUGFAQBUsGaUop44lq3MgeOvQHuPhN8AaPNtYe8ho6ha/gMPRWIJU/aupaeZJ3RL38Jp54KIelSFwfANOGOO6CkBB5+GL74ot2EZZVY3yrzFm4vS8i0sZuNAjtKYTXrfbEch6Xle1JFQB0nPT15UHGQqGUTs5y03l+N5lPWWzN0txfHVskziBso7aqLsLs2SiTu/s333xfhlaF72VHgVntv8RKIexJT8B03Ybne584gNTWoMNwJC2MbfIzNG4GRXyjFBkWPkpyeI8Sofjn0z/Pzrwb3RBW1HDyaRo7PQ2G2j6DPwHZs0HTswjxurMzmf/272VpIxtDXo9zihiG/e8WWbUJx2B3u2lwAuWacgTU2u/K9eA2dCQPy2LkpxL+qMgcPqdogiatN03bY2xBzk6NpfmJOJlrCznz42cnwnS0WQ+rdLwwtMVureQykAzl+D978LIywB83W0BoyLZDYhm3bYOJEd9jrtdcgJwfHUeysjRA2LbJ9HndWWjtVqUUfVF0NRx/t5ooBePZ/Ok2uY5eR5n5mHUcRtx38iVlTkbiNHbep9sMJN4AiymCrrs0O23DUQjlucKO1+MwmS0Q0r8jcdHgtVXurRbMImxaNZpziUIh3SxzqMnfopqaAxQy3uGCj3+31afS6Q105podzfOPJzbHRCgtbtUGI7iZBzxEk6PMk6ug4DMwJ4jU8+D16Ig8m/eTiKS6lRu1m+mZYPwBqgk1d314bck33Ki2uu709AQtqg+5NDybH6h0mVMbIMyvJ8WmU5Qeoqtyb8TSWSmROXEFatmJXXYx4joPSmrJyWp5wLR0eOqqBn/zTg9dWaLaDF4WyHbzNtnPzmnR3kSDViRkt4H4T1dTAjh0wcCDVX/8Gfzn/W2ysNYlaNgGPwaiSHGYd1T9Vj6glCZL6kFAIZs6EVasg3sb08DYoyJx710yyRwYFHkPHdhSVXoupt7oXITkxGBx2UoF/81FtBXy2oxZlOeDNcBzVtCho8lhNOXqtLyhQikjMocCsoUrfzYpBiYCnvY92Yj6Fg3veOGavRp1HMdj08fC3/ocRk0/lPx+5CFsd2Iw2cZjy+dzyDYeYBD1HIF3TyPIZGG2sip6iwQfDYfJuGFIH2/Ih4nOXsNA0iHjdyVYB212/q9GbKHSouUmItR5YMRQ+i1cy9R9zOG7M3dToA9MCl2QuQFP/TRPbaer5SW3fookKeHmMyaf94vztr36yHTemCcds9u4NMTrDy2o+9daM2/hp/xzdrLE4jY0UPvEo33niUX75i//BHH8UjabFml117KqLcO0pw1sFPuVVDSxaU8mmPaEOB0niMLVtm1v6oJPBTlKroCLTNokNVGJb06+YN2M74WalTbI8OpFEE+wMPUfJIa9WJR0SUU7LZ2i4tXfiiVlgqccVZDfs5L1BIXbnJJa16cgfU2KYK8cCR9foH9GYnD+KUcee0YEnC9F9ZDBVsDPfnVJ6wm63cmrY7+bw6AqKIu7VWtTjBjzgLm+RCiw0CPvgnZJ9/H3bDxhV/XJqhpXtpM/0yvTflj9nOp8q4LNSxfmXRgl5HHcVeKV4f+PeVif1uO3QEI3jKLAdh13hWKrK7P607Gm6+Zav8r3Z4xlUsZUxpTlUh01eX1uZVuG5vKqBBe9tYc2uOgqyvIzsl0NBlpc1u+pY8N4Wyqsa9n9g0ftVV8PAgTBs2AEHPNBU0HN/GyUvEuxEsnLLCVxFeYFmMyHTP9y1Ucv9vKvWAVGbQVdyKro7CwAch0mVMKIaFo4OsS3P7fVNHLBDHKBf3MsoMwe/N4AhZSJELyBBj0ABOwpgX76fL+8u4ZgKGFmrMSAE+VFoTMzK0FR6sANusqKWuCTdkR1nRfwlihrcL4UDLEDbZhvXlMIFl8cJeRwMXWNrdWPaST0StwnHbOK2SpX1N3Q3QLKVahVoJW9W8+AsQ5u/8a0vM2/OUYyK1VBeFWJnbQRwr6YXramkOmwypjSH3IAXQ9fIDXjbDJLEYSYUgmnT3ETl3bsPeneaphH0ttHBrtJ/rNNtzrwszHVnbsVsUXxHS9afIvGZbfbc+oiVWmQ04+6bFSlsLp6Y6ZUTracwHKe8ED4Y4s7AcnDz/NJqTrRFgS8OJ+7R+OmxP2DwsKOoy/W28wQheo4EPaKJpmMW9qfQDjCpNovsOFRnuXk1erPgBs0NegzlfoB03McMBXuCJoYV3//VoKLDicbJQEQBn/WHL19hE/Y4mJad1nNU02hiK5W2SrWRquKsUstiNM+pUEBD3G6a2tsGw7b43jfP4YZ/v4xwdS0AO2sjbNoToiw/0KpIoqa5OU7NgyRxmNm2DYqLYfny9iv+dZLXyBRyNKMUIY/D+VcqPi1V2Jpq1aOpVMv+nQ5Q7mffLfLZ6pCARkFjBR8M2s27I2FvTmJxYt2dpRXX3Py+5vvLdAyPA2ft8jIjOJEZ598kS0qIXkU+jaKV6hwP/sKRTKsrY1y1QTDuJjc3T1DUlDv8lTzvJX/XFezKgeN3wPAai9J6OlcQcT+P28CqQTD1yj1YWrhZMqdyq1K3MXSQnJkSs5y00v3Nn9+RZo7c8i/GjRkEeXmEt+0gatltLrIa9BnELJtwuyvPi14luRL60KEwaVK31XDK9CnVEiWPawIOx38rxgeD3L8HQ299gRAKmU35clrr/bTs/UnSNQ2frifq7zS1RQNyQztZOLqGHflNQ9nNObr7t2+01eOj3BlbP7SnceLQk/AU9WvvLRDikJBEZpFh8iqJqe1ljK71Q2gvqwtC2Lrbw6PhXs1pJE6Oyq3pY+mwL+jW2DEcNwDKNuGEnW6tn915GY7d6oqzgxGSgsGFbp0fB3c2mK3UfqP4qG3TfKk7BViWwtesN6k9qeY2NDDuuPH8R2Epv3nsJYzS1if4iGnj9xhkd2rleXHIVFe7y5Ts3OlOP/d176KIzf/uNNxeyYiheOJLUer8LbZtEcy313nSfNO0HkjNLfSZVuxQwXE73QkMr44J0ehr2rb5cHaysbbuLhgKiSKgGmBDjgnHVkKh5uOWH/+Rx5765v7fACEOAenpEanAw0lUZAW3Gzxi2dj5xXz/jIc5bo9BlukOcyUvIu1Ez4/HcYOfRq+b5JzsR3E0dyr8B4Ohxg9lNa2Pq1RTINHZCd77GuPURy23KnTcIh539jMd2H19rY7TfPX5/UQ9LVMl8quruOP/ncJ3Z43n1F/9B3rc7RlQSrG7Lsro0hwGJYowil4oWUl5+nS35k5VVdcmo+1H8vMUN+CuM2wmzLV5YVw04+yq5s6dXJa6s/2hspb7adr6qN0wqAE+GOr2njb60nemWu440VgH8Njgt92LmqOr4LgKsPMC1OfLZ130bhL0CKCpsrKVqGhsOQ7ZPg9l+QHOPPtipgbGMm2XlxzT3dbSAQX+uHsCjBnuSVIBRmKfccPtErd12FwM6/rDpIqmAyYXOGx+ot3fAqLNVdXXc8FXqjn7qjhhj4Pj0KoKLbgBiDsry933weQVay1+Tg0RKMXxr/yReV+ezDcuPYUdX+ykKNvHzEn9pV5Pb5RcI8vvhx//GN5/v0MroasMt4MV8sIZX4f/mgYVOdCR7KHB+cHUZ7Hl58shc75c6vOqHMbtVHxcBhuK3ZmZqvnf4X4+rnHD/ZvOisPxu9w1tb4o1SV3RxwWpN9dpPgMnbLsILujBgOzssjyevHoiY9IXhGF8XF8dUcVnwSr+LzEPfnpJGZw4X4B6MrtDTKbf7ISJ9GoD94aDV/arjD9lpurgJbKw9ESixm21FYcFAxqbMmL0+CDh08Mc91aN7hpvgZZfdQm31G4owUKx+aAv6k6Grrk1e7lZzechTIM9C++AKRWT6/RsopyJ7T82CTD6OYpLp21OwjjvuuWfdCb/R2lHUe1zqNeXxlKXSDoaGkNMBLPce9r2psGFNTH2By0+HxMxxutqWa9PgoKG+HYaoOj9f682G8XVVlQGu3gCxbiEJPQXKRoGgS87sKegVQlZ5eua+i6QbRgAJovyIytBidvh3F73S5yXbknR7/tBkOZD+DOAlkxFHYF4+SEdgEOXve0jaaB0Ymvjw2V9SjlXqm+PD7Kr46vwkpMUVeJoSzbUuha0xRfRft5Qy1SINJuLbn7art9um27dV0Mw+1VkIVNDw3ThG98w/3HLS7ukoCnvceTM6tStzY+JA1euOhyaPC7Q8FtHcNyWs/e2ranMfWZtFr2bCaSlNMCKAfGVcLiMovPS+lQwKO1zHNTUBqCGRUGJw08Hv+AQQce7QlxiEjQIzrE79EJeA23F0XTqSzyE8oJUGj5+VJNIbmmO6vD0TLkArSkQWUevDQhhD/WtHgp0Gol9bYooCqUIfchkSeUmuau0gMTLdOXywGMUygSizd2ZGPHgV/8wh1KOeEEWdy0GzmOYnt1I59X1LO9fAdq0CD3fV+w4ID32dGPhsKtdVMRiqaKXFqOoqIuSnU4lrbt2nzIu9PNpzmQBpQVB1MroOstuodKs/3k+N0rD03ZHL8dskx4a6RbZb2jx1PN9qsp6B+CUys97O2XLUNZ4rAln1zRIRoaRdk+vIae+rJXmk5dlkEkfyAjwlmAW8ujgzvEMmDFIKj0mgxOJDlnCnra+tIJePS0bVaWhvnaV2p5eHojpuE+S9fcHB4ncbXsJk6nH0M5qlNXrM3b097TMrb7ww8hNxd0XXp/ulh5VQN/eP4dBpTkMa4sn8FjhqB27eqSvJuO9PS4K6RbhONO2ueiPhbn0521ROM2tqP4ImAy5eZOHLyND1myJzY74ElLap4yrADLUQysiVHrt1k8CirzSU/Y7wSvAxP3wBW1A6nql9X5HQjRi0jQIzos6DMozfVj6JqbZ5CY0podCPCd8d8kz3SHrzpMA8eAT/u7Cc/9q1XGFajb2mXzIoQKsFFsz7f5n6NMfnQ6xPUW3f64Qw2t65o03dHRmWQHk8fh7kA19f5oGqxefTB767tME+bORWkao/rncfWVM/A4TQnJqsV/u5OjFI4Cr968p1Fhxm02V4Wpipn8+8mNnPKV6raHgDsoHLVSn7+8YPqgcE1tBKOhnLdGOmwvoPMf1EQPT2HYnc5+3bYiLi4+ieDAjnRLCdG7SdAjOiXo9eDz6Pg9On7D/W9ZfoCrvvkf/Lj0Mvo1dn6ftg4bSmBLEfTbF3UrEHZAYzw9uzP5ReMAzx4Ns6+yaTBabEPrHAsn1QPk5i4dkjSFKVNIVYy77DLpAWqPacLtt7sBYzCI+tWvUg9lmmWd1JEZV8nHnWZBfUclP0PhWNOzFGBaCtNysB2VtrZbx3fcutHrK+pSsxB1mmrzDKmFyq1388q4GJEWU9A7c7wR1TB2L+zJh7who0CXOS/iyCBBj9gvJ1HtuNG0iVpuRKJr7qwrt6S9e2Y956qnuCb3Xo7eSee+LRJ5QPuy4PWxkB2Hk7fufx9t5f84wK48WDkI6rzpEVSm75zmxdp6xR/EX//a1ANkGDBkiDvrqI9Iy8upqEXdcov7PiSDQr8ffvazg6qnk+mZcdvBdhwc3GT4mNX02elAmhqQWOAzw8rmUW+c6d+o5rcnqI7G9O2qqI9iJWYp1kYsnDic+oVbK+sf/fbiJEpKdEoiuJpUBaPqDNYPQHJ3xBFHwnfRrkjcYsG7m9lRHcFy4miawjTcBT+bL/mQXHyzYcDJ+CsDTNsSZfmwdoa72hof0qAmC94ZAeOq3NXfI4GMm1Gc46e82bJWnSjx03pn+2lWpqc4Hdz2oDgO7NjhzjpKHVyDqVPhjTcgJ6eDu1HsrI0QNi2yfR4GFQR7V/2gigoYPRoVdpcWGdzsIRt3GvYBdlq0+bzmjykgHLfJVk2pL5qmdWoYU+G+z82rJ5uG4rfH72Vrnt2lHxYzrlLxiKrbTH0xLC3oRGNbSgQ8J+yCibGhrM3fA5qsGSeOPBL0iDZFTIuqhjj1VQ14DA2PYWA5bjKmuzynjidxgk8uvlmSGwBNZ+1AmL3BDV7Cftofd2h5X+K/G/q7hQ+P2w7rS2mqepjYZkxpLh9sa/813HbOdkJZNpd/CvOXJHfdOpE5U72Vti6Um0+Wae9LsdtCCqXcRTBz26j/U1YGa9ZAURHgJvguWlPJpj0hopZNwGMwqiSHWUf1Z3RpN9YQKi+HsWM73CPT1lYdea8PVGrfiXyc5vtvq1RBW/tRuEuh2FbrxWvT3oIueBEewyYnHCeGwyvjnYPbp4LcKJyz2Y9eMJp4ro5H33vwjRSiF5KgR2SkgJpwHMtWjCjKZoOu4TjuiuU6GrZyk46TvT1h0yJq2ZT6miKTHcVexu+NE/PA2tJOJjknWAZ8MAyCMZi6HVYMaWpgXYPZNKWW9GGq1OvQoDYAzx3tXsHf8IEip8VXkmrxc4sZwGkcmobAdE1r9/u81XT6/TyeyQF9l+3eneoZUsAo4MY2Nu2OQOJAtPdeNA9Iu6u9Cndhz/Ye7+hxLUdREVRM/K675tzYvSHK6gsOvHEtoq+htVAa+g0r+1vs7VhHX2YK+oVhYgVsLAYrdwTFAT/5WQZbI73hUyFE15MBW5FRzHKIWDa+FkUKk9yp4G4BwEbTIhSJ4zd0GpvnQWiwu0BjQz84cxMMqAf9QBIaNIj4YfEIOGE7kEgw/aSitmnBRo3MK6wrN1CpzIGfnwxfvayOkCc9ubn5MI/aT/Jqq0DmAKe6dyTg2V/SbWeO1xXbHUrt9cB1pHeuI1oGsB2Zpt6yHRoau7PijPqeW3Qw+XFPG0o80EY6MG2rm/j/zuAK9mYf4H4UeONw2hcQNeDDIaB5oKwgwMCCIEHvQU4t6yN8ho/5zGA+M/AZ3bs4reg60tPTzXyGjwecM2C1B049FQ6TPw4nMdOkrWRhhZvsaVoOO2sjvPDhDvbGPOyNZCi8p8HaATC4DkbvhY8yLG64XxpgwNJRMGk3BB1FQYGH7U5Te6B1EKIZTYexdNiZpzAzhPrJYEd1YKX25PGUUhl7l/b3vAPZ9nC97s70ejv7WpK9a817ehyaKgZrdE0vUNxqGuJyA3qnU/s0dcVPp4dZNLTOXZuOpvY5B7PgGzB0n0PICx8MAjsZkxxgklNJCCZWwYfNEqc0ID/oS6zNdVBNFaJXk6BHZKQnZmZlWrVcodyTuHIndwQNg/ygj70xhz110cwrnSd6fTSlOGU7fDwA90r1AGqIrC0DrwVn7HgYClONar5Jiq9F74+lwaTvWujA549C/0j6F1JnJwQ1XyW+O6WCuha/d/X+k7riNbWXo3Ow+1eA7ThojkNX9UvYjrt8SWfakBTywpwrFVuKYuQ16q22O5ik8cJQjNcncuD98soNEAM2HLsTspSH9f2bFqFLfaZ6cHV5IQ4VGd4SGfk9OkGPgWk5bde10RLVjTXIDXiYMrSAkrxAs+2avhh0SK2o/nmZzqkVBgMaOLBvbw3iXnh9RDUhP5yUSGbOtKuWQ14a7pBDvR8GfR/Kc5MzdQ6gHW0cs6tk2ndHhry6YkjsYF9Xd7ax+T46siJ5R/fV6r4ONrLaD0fNhZWD3QKZLWkkprI3v6MjHBi5B5aO7MRzWlIwqA6+tANO3A6NAR81ud6M+WUxq6veTSF6Lwl6REYaUJjtxWNobK4Op66CTcfBcTLn0GiaxoiSnNQyD35db1o1vXlKg4IdBQHOqRrI8TtBszhg9QFYMRgmVAJO6x21m2iswVHfhR05B9GAXuhgelG6Ij/mQHKJ2mtzm8Ffhvyrrgqm9pfbBW7vzsnXQskdsD0/EYBl6uRsOeS6vwbYMGU7BE34ZDCdm0aWlIiOT/0CsixYVwob+wG63uauDqhwohCHGQl6RJuCPg9l+QHGl+Zi2Ypo3May3WUc2qpcHGy2HlZym5ZfpO70YEU8Nx/DH2R2OZTUccC9PjGPu5hivW4xfpdCNbtgNfeTSxE3YM6VDdx3uokp+Zut9NTXYG/4uk2brr6f3JaQF2ZdBcuH0mqB3Zaxg6PSZxa291pP2urmuy0dBfUHmqic8KVt0JAfoCKX1IsztLYjqM7mpx0JJBm575GgR7Qr6PVw7fQRDC4KMqgwiwF5AYx2lmqI7KeLPPmFoGtgaDpoOpX9gpREYXwFBxz4oMMX/eDt0VAWcsvoQ8euXh3gt8fbjL8Zqv2d6+Lve18TXa83BDyQaEcHcrRCXph1JXw8sOP7bv45bLV/BybtgrFVsHQ4hLIybbQfzcYl8xvha2uDxANZrYbV2oprNNwhbSGOdPIpF/ul6xoBj0GWzyAn4EHXNDJ1oCil2NMQbfuKMTlkkOHh7QVQlw1n/osOr72Vaf+OBp+UuYWMj94Fege+UpN5MtvzYOytcMdZdKjXp7d8WbfUW9t1OGkrVt6WDcV3wLJhEO/g2bPln4Nq9sO0L9zCgGvLYH0ZB3xG9sWhXwPM+AK+O+y3ZJeNz3hx4k4yyJB31Gw5GSGOZBL0iE7R0PDoGroGduLSWCloiFpsrApRmOXDY2TuCdrvl7EGFYVevrQD+tfS+UI1yZ57HTb3g4/KQDkOQ/Z1fBeNHvjpdAjcDZ8Utr+tOwLS+0KMg/nqai95uuWto88/3GR6DaYB82bC8Ns5oBXS0/5NFAyvhKwoLBsBDQfSs9NsX8GYu1ZdrulOQ/cFc4g7CqVU2iQEjURBTbpntl5vIsNWoi0yZf0wluPLYfHqYwEwfQdTmrVzDF3Di46VOLHGLJv6aJzJg0o5uSyPpasyn0LbOrG2DBxWD3Z7ak7dDB8MhtiBnLM0cDywbgAYNpy8Dd4b3rkv5SnzwGvDxp/D0PABtOEg9fYvopZTz1Ub9+9vH71Khlyeaj8ccwPszD+wXab1GjmQ0wibS+mSeftZJlz8RS5vDmqgIg9yY1AftWiMWThe0rqZkgnaXl1PzaZMDukdCZKBDnDY1EMTPU+CniOEWwRxBubHigeHb8fO7d5OvOQK66AYmBXkxjNGM6KshMbayrafpCUKynXkJKu7U3VH7AWvA/8q4YC/JGwdlg6DaVugvKhzz40b7tX9Dcvh5/90A6ie0hX1bLpbso0dnYnV26X6KBO9htV+OPYgAp4kR8HYKsW7Q2Fz/4NuJgD9QjC5ChpyB4DWkLp/+54IdmI9ubTkbBI1tnqquNSRzueD+fMPdStEJ8nw1hHGp3Tmbx7GPfZ0jG7+59U0t7s8y+dhSFFWhwqwJS88HQV2YppV2hdmi7yCzcXu2lmnfQEl9RxYYZbEkNey4W4exonbO7+fJ09qPeTVWy+Qe7pdvfV9OBDJXkcN2J4VZ9D3Dz7g0TQoiOzgvWFgHcxlZuKCIS8MV+8cy8iIn42leqtqz/WxWCp3ruXfpK5ptF4O9fDQfMgqx5cjw1figEhPT3fz+eCee+DBB3v2sBjMd05P/W5i8wBLerQNmWgAmttLZDngJNcXoJ1yJBp8nBjymrINqnNga3HzHXb84LW58E6uuw7Y4HpYNXj/T2vuS3NhUAg+ehIKYp17bk851Bfxh0MPVXtCXvjy/7P5tH/HEtrbpGBgHeQ3xHhxMgd9iWk4bpHBz0ph1An/zvrPbkYjjmGkv9tR26HA0DES+TutmtXZXDkhjiDS09MTkt2g8+e7Px+KJmDwgH06//x4MmfWFHR7L1B7NA18Hp2BBUH8Hh2/3mKhorbo8HkZ+IHxe0A/iBN3Ra6b6DxlK2B2/HmO7haiG/R9+MHZB/ml2AW0dm6i8+9Dtc9m8lxYNegAqz1rQNxm6hfu8NPuXHh3JAd3plVQWg9TdrqzxtAhFGsaZ9U1LW2G2KiiHII+o+3p6b38A+IzfDwUP5PaJ/PZ+tfBFAU7OSYtRDsk6OljksNfj0RPZ8+vc/neCu2QdPdpQMCroyd6fTTcoa2268U22VIIOf+/vTsPk6uqEz7+PffW2ns6ne7O0h2yQZQlEGIiBCEqiCPjPOi4vL74vIA+OGjAQHyGbQbRCYs4DKARCegDOgKCzvs6yOhEMY6IkRAgrCEhJJCFTtKdtfeu5d7z/nFq6a6u7lR3V9XtdP0+z9NJurrr1snt6rq/Oud3fr8YnPNuJcHY6Afg2vDyTDg4CeaNsEZQ1IZ7z4YTrzZ5H14Zbsjj+Lo27kRtWHVuNyddA+9VjXIiRMOcA/DWdHhuDhyshLifMe3M8sXh/G0wudt0Q0/aur89USU92Q4m/TBzG8uJOtlDNlfrnH7HhJioJOgpYQFXcfszfnY/NpX2H1bxPy+eSiBv7RtHTqncn5B7qxTtoWtZvBvmHARrLJ0kLHi5GYJRU8V2JG/x91ZB4z/COZebZREvlOJKRT53h0Vt+Jfz4Ffvi9I3mncADpzzDlREYP0s0Pl4VdXQdBQ+8B4crA2R+Zvhs61Ebzs9KG9n8ew6/JY1qJaWBqxEyYnxwEZxs3uu5OWIopKcHpES0BarYufAK0GiZy/hduu54g5AZ2/YOJRJFSHeqjT1Sc7eAxuaxpAoqiASMsnOUzph3iH46wkc8+qqgbgNf22Gmhthage8ej/UjtN8n+PRUD8CpY7dFPRYz6bd5aYgZcwyy1EjEoMz9pn+WH+ZNcxAR0KbhP3mdnhtKhwtgzkdIzvErEllNFaHeLsT4hl1evy28rzdRGpruVvErZBDkR1YJUeCHpFVKhHatiGxUyLqRkecDJ3Kmcwhu3WkMxYByxQa2TkJOoNwwQ54tdHMvoz6AqTgQJX5qO6GSZ2wszG3u7rK7PRp+EdYsQFu/x8IjIPX9eNdtu3wFoClGGIV55iiNlz/EfjeWaRLKeTKgdNa4LUZZnk0bxw4513Y1gBvNWD+k4n/n8qYnYk7LpYyW+wzd28BhAM2QZ9NarpHJZoEFzngSZbS4BUbPvQhqZ8jPCfLWyJn6WTo01h2pPqYnbEjjpN6zXU1ien4oR3r5Tjz6x1RZ8AXW2qDTOuCpTsTy11jXPdpL4edDeCLwlnbgRyX0Bwb7l4KFTeamYRiGEnV5ONZ///TaJuC7y6Hyhvhe2czonWyaQdh3j7z6WvN5O/V04VT95gaUM/PyvhaooBgZqwyv7E61WZiuI7wlkrfd3wsagnhLZnpESMW0Bar3jnBfKIUnHfegHdwTuJq5PavgZaoYOcyzIuvMuv8Q8l8YZ9SEWR3NH14BeyrVsRdzVnvwVuT4WDFcA+YAwXxADw3F3Bg/l7YWg/kkL8TTxQ2nNYOrxVxyet43zI+nOT/yyW3ZrL97Q/DiSugaySJ5xoWvAe7amDvZPK2hAWYRqP7YXM9vN40dOHLbA9ZEUzn3iUTmccLqYwsxjMJekReua4mbppy0X9GXiX/SCRfDnW5Gsn0+0kN1WzaY/49YAYAs7XXF4dzd5h8Czcf+dk2bJ1uHuDkvfDmFHL6DdpbDVOuhyV74PePmJ1nYuxyjXl2l8Oca80MXO4Hh5NbYHu9ydnJZ1ShNNR3w9yDJvE5k5WxpdxSpkbVEMP0rDu6jcWcrgCXvRrj5m1TCUxvNjkyRWyJMyTJ1RFDkKBH5FV7j82//3EOj8zYwqFJlTzVdGTA1xWYwGcUaxOZ152W9t4BHasH/A1gQVu5zUmtDn0BU90564FGMZDN0wAHFu6GTTPgWJvetIINzXDScnhdEp2LJpmonFPAo2HaEZiRSCJ+IV/Jyf2P3wHva4N1s6E1S8BDlof0WRYqI+jpjjqpma6qUPF3XNpY3Bw7i8BvWqCtDerHwVyTBDoiBxL0HM8CAVi2LP3vcaA7GsfnKP55fZCDTdNY9T8+7jm1jT/O8bGn0skam6RyUDRE9NCZqZlhUmdPZMCNmS0sknZPNomrH3rX9ODK2658GzbNBGKw8D1orYKWGoa9UO6rhrobzPbmbd+Dxt48jaVEZOv1lU2qZ1aVCTiPddAFLdDth+2TYVddfsbanxWDc3eZCuCvTWVEz0FLQcCygPT6V19fHFspLAVhvw8KPHtoo7jZOZfApsS67jnnDBiPEMcLCXqOZ/3f2URHUFa4gMoDPmL9go+Aq/j2Mz6uf7UWlKLDcvj837Wjysz3uFqnCqm5WmfdiTKUcNiffrFX4MsWbaj03+tnwZSjUB01yxZ5exfvTwQ/Gir74P374fnpwDBxaFcQpl1n3vm/9kOZ+cmX3eUw71qTTwUMHR1pmHbYnP83G+GV6eQ/McaFql74wC6A6Wxp2jvCbWJpmcu+n1o0nRffM8tafbH8Bx82FgsOWJx3oJybTzqXgAozroIcmdURoyRBz0QRCMCqVSb4uf12cLx5gZpeE2a/305MvWfZShtVrH2yjooTZ9N6+hl0t+3hr1PaR/VY8fjA4/tsZWZ7hls680FLGZxwAHwKdkzOUzE5AAWd4cQOHA0VPbBgL6yfyZCJz3urzMzPy9+HBUeyf48YXpcfLroEdkxKlCsYjgNL34WXp8Pe2kRycr5pKO+D01ph4wxonRykoT2U+nI+YqsNOw4Ribu4PmjriuJW5Ge3no3Fzc5SAi9a8MYbUF0NJ3ncayVJAh2RBxL0TDTJFwaPgh/LUkwqM7X3e+IulYmXYq01cW36BIX8JspoaetjxWvT+IdALRedvwWHkW1D7uhNT4+kJnQUWDkEMTvrMcsa+6DbB9unkN93+gq6ymD9XMCF+qMw8yi80ETWpY0zvp7+twRAx2baRjj8bha83DTMN2qY2gZBB3YmEs/Xzy3gwByzjLp+lil0OdSOrLHa2daT2rWVWZF5JCwNV75qs+rNqQQ+vwxUgHE1oyNEnknQM1F5GPyEAz4cn0W5z07V5nG1JmDbhPwWftuiL+5ypC9KnW0RiPh5874yOoJhWstqaT6yg0980WXjcBczoCfmpp7BKhGxaD2CwEnBq9NMBdzFu0xhw0iQ/C9zWNBWYz6IQ91RmHcYnptB1iWws66EM/fBfz0KVbLTKyVZUPD7Z5nPfe4QpZM0TDsKsw7D802wL59LmUNxYdEueK8aDlTYYA39+zaaGZnM/Y6NtSHUXvNvOyPoGWqnF5iZnN9ueh8VO/dCe3v6l2XSOEhE7i8chmuugZtuGjf5imJikKBnousf/KxaVbSHtS3FtKowTocPy1JUBv3YvnQVHldr4q6pKtudyEmwMN2iK+IWzz4MPu3S4YcL/4/FlimDX8lrgj52JW9ObOvVWo+8aJ2CN6dCxA9Lt8P6ORSubKcPDtaYDzSE+hLds5tIBUB9AbMkNulGmNEBr5Twbq8uP3zki/Bic8YXMn/GGqYdgoZu87PcWwN7JxVhgK5Zwnx1KrzSDHXdQJb0urEUQnY1gxqIvtXaNeTzfF9XX+rfNsrk5uyxuXlLPYEZ+SwhXSCyjCUKSIKeUpHM+Unavx8eeqigD6kwjRFRatAUvKUUPssi5ri4rptYlkr3BUq+nlfE4MnHqymP9YHWHPHF+NA/OLgWxDOSnuPD7PzKdcAbZpsHX/wObGymsL8hCvpCpiM3GoiZthUnHIZt1UCZmTmou8F8u6Vh6z0wt7OAY/LQ9kqzvXzQrEy2SZM4fHAPRLRp/dFSY/Jz9hZg51VWDpz+HrwyLVHHZ6hxJgyYjRlhUB51XZyM+3T2RHB09npXccdilfth2GRBLAZvvmlmdKrH2WyOEB6QoKdUNTbCgQPmHdUjjxT94YM+i0mhABHHTb1jVco0RMxGJWaBaqM2b69WHC6fxC3fqGdLshmjHqJCbw4XmEHfYsPG2YADi3eYWQa30B3UFeCHqB+2TUsMKg6n7oOqOBwNweZGOHFl+i5XPQd3rRuf/b2emwJLv5ang7lwcqvZCXU0BO/VmF5rz+a7js6xaPDHYfFuWH+CmdnJ9fEHxPwjGXNiuTYzbScc9kPiuR9yLV7/9zomHzxIX6iMg5/PsVmc12xblq9E0UnQU8oCAZPv861vpfN+ipT7o5RiVn05h22LHlejtcZxNbHEW9r+r/GWUuk+Q65pRhTwW3zj/DN44H81QWsr3XWT+cbZUX7TfISRvpUe8hpkw8Y5mOBnNxwJwNv1FKdjXSIIej25rKMBF3y9EHZgci9sqofQdQy7Nf6rG+CePwwOjA4H4bSv5bDbqZiisGQXbGmE7mCitYgDMR9snop3vRZcmNoBzYfh+RMSy5+Q/WmWZYyWAv8YqiZrzeBSDjE///70XOoOt7L9nIUE3N+hMb8r02vCo36sopDlK+EhCXqEZ3k/teVBQtVhdvVFTEPSAS/s6atHcmnM1drsVlGKyqCP2vp0ufugVlz36lS+8yc/sUiU2s4jHA0ozr/Mx5h3o9iJpS6AGCzaAy815XGrey6UGUc8DJ1AZ0ViN1JiRgjXLH8F41AVMW0OqqPw5Elw/xI8DRhO3g9VPbBrErQHIOIzs3o+B3wxM4vWHQD88Pw8j8aZSUNNj5lherfWtBLZV5PbXTPzdyxLYdsq9SMY6Y/C0vDTp+ew8J23sYBOf4jPXBnGSbxZiMQdYonfHZ89eClZCJEmQY9I65/3k9z1VWDhgE15wJfK6fH7zDJW/3fRflsR9FlonWiu6GqUPTDisC1FyGfjuBqfZfbyVsQtfvJkM7MO7abbjvDBr0J0rNcDH7w4E3Bg0U54aQZoG++CCkXqt9gFeoPQWw6ttYmvayARXKTOmDabiyxlivhpy4SFPhdCveZ4fQFzW1nM1NOL+MB2zRbsqM/cz078jLSGgIZQPHHcxJLM4QqI+xMtO7IYlxvTXJif2OL+di2snz2KY2REPVpD2J+oIZXDLKSrzTlv7IJnfxygvWYWPpUO3C2l8ClFzDFJ+3HHJWBZqTcE40pmLqEQHpOgR2RXpGrPkbhLXJtAJXPrLZhrenICyFYq0X1x8HEUUBMO0Ec6wbn/0WqjNjvudTlUXsNhu51Tv+6m+jGNqqibDS/NBOJw9ruwszqxW8ib3o9DSwRF8Rx+06NANGNlpDOU9VuBgfNnMaB75KMbPzSccBBaKmFrA/hdk78zGqONO0JxWNAKP//1JKZ09WFHo0T9Noe1RvWbBVVAY2UApSDot1lywmQq3vAN2Yal6GT5SoxjEvSIYyvg8pfrarNspcxsTkXIvHgrpVMV+4fryp4UczVHe6MEE8dMNmkym8fSs0caaOr2E7s1kjg2bK2p4oLLOswSy0j54K/J2YA4nPUu7KiDtkrGXwAkBtIQjMCZLaZmUneIIStn58rknQ28zVLQG9VZm+wme7A19EIcCyyLvpp0AKOU2dGYmWrXF3OxlCLstwmXSSKwELmSoEfkrn+ri1Wr4JlnxnzI/tvUwczmKBgwTaNQw64euUB3JE53LE6ZpfD5bUgEOoGghd9SKFelKthmqovZbL0vhO32seIC+PGSUf5nfInt52ACoHdMteC+4CiPJ/LPBSsOZ+w2eVmREPx1TqIlVh4mNLPnNiuCPvPMq43Axl/UUd9ycPD3KUy9qdR2RvOX31a4rhrQrLci5CMcsPHHPZ7dsSyzE/SSS8wbI9mJJcY5CXrEyGX2+RqDoM8i7LOzbzfHvMD7LAbl+aS+nmhS6mhN2GfWq5IJz8kZHpXIgdCJekHZVgEUZofTD9YqHlyr2R+GeSvMLqJR8cFzc82Ypx+EGV3w5hToLMO7/J9S5UJjO8w5AuubwQ3AS4VsRZH4+QYc2HIPBJtOZO/7zuD+Z48S7OzAX529l5VOlBOPxV0Crk5MFGqUBZYvHfj7LEVdRZDKoA+7z6Mnk22bIOfOO715fCFGacIFPffddx//+q//yv79+1mwYAGrV69m8eLFXg9rYsrD2r0CahK9umKOCV6A1FIUQMhno4bIenVcndiqC12ROIG4i467ZklMa2ylTECV6lPEoKDDBEcDb2zshc7vmNYHX78AHhzt7I+CljrzAUAsUcG3CVn+KhQNdhyaj5gu6utnwv5J5qNQKiLwxvcgYE/mSE0dJ723DQtNXFm0Jb7HxO0m+Tj5bNMD/k7cqpRZ0E180XU0qt+bAldrqsuDTCoPoDqKGPTYNnzkI1JbRxzXJlTQ88QTT7By5UrWrFnDkiVLuPfee7nwwgt56623qK+v93p4YgjJXl1x26LDTbeRSMz2Y9tDRwc68Ufc0ThOYobHSld17ou6qX5c5njK7O7qL0sglBRwYM1a8wHQEob3rzDF8UbFD68mdn+d+Q4cLjcVhaP+4cchhqDBH4NwDAIuzG4znc2dILxbD+/mu5Es4Hfg7Xtgene6d6wpqG1xqAJ0v5YR/R/apJmlZxozCzQo1f+29AZ321bE+7UhsZXi7xdOJ/x4EbufSyFBMUFMqKDn7rvv5oorruDyyy8HYM2aNfzmN7/hoYce4oYbbvB4dCWiogLWrTPLXr298NhjOd3NthQVIR/+qrApRKjNLq1jXa8U4CbePVt2lqQdN73cBYkLy4ibc6U19kL7d1K1AtlTDiddC7GRXn9seKn/duhEAUQ7DntqoD2YCKx8SCCUQTlQ1wVzD8LuamiZDLHEtfhg5SiPCUzvgC+/Ajc/A7ZrE7V8lDsRkiurmT+GbNWfLAVOlt2FYIpvWmpwg9D+YzBBfDrq1ySS/MOWmY3UmoDPYm5dRdZj5I1SUFMDM2fCzTeb32shJoAJE/REo1FeeuklbrzxxtRtlmVx/vnn89xzz2W9TyQSIRJJv4Xq6OjI+n1ihJLLXocPw89/nvPdFBDy2/1e/XOX67f7LDV4x/sYWnY1d0Pk1sG3jzgnyIaNmb0gHTh5L5RHoKUKjoYTRfxKIRBK1Bcq64PaLqiLQjgOjg82TocDVXC0ItFlvd+5yKxA7ZCejRmOCxwsm0RNbwe2dnByOL+DviVRUSHzuehozaG+KPOUwmdbw/7oBiy1avMR8tvUB3zp1dBCbk23bRPkyIyOmKAmTNBz8OBBHMehoaFhwO0NDQ1s3bo1633uuOMOvv3tbxdjeKWpthZ27zYJz9/6Fjz6qGmFnkepJQMgHk/kS/S/6limYJvWZvUo5LMHvENP7hTL2DAzZsmcoLFwGLzitb0STvwaLN4L26ZAZ4DUBdrS4FqM76DIBRyzRNS/IrOLGX84DjM64ZU6IAw9FdAThvf6RS6VEXjzezClzwbtDOjCkTkjM6LTkFhPVckimKM4hxrw+1QqcNeYulENZSHKgzb2MapjKiDgs8xspzK7GxsqgoTGEpkLIVImTNAzGjfeeCMrV6Y7OHZ0dNDU1OThiCaoZI+v22+Hri7427/NW/DjuBorUcfHcTVx10Uljq2AUMAacAHzWwqrX46QxpTu91kKC3BHM81URHM7wT3Ghpnnw3DNp+GdOugKmuApWc0aKFpF5vZys+yngPoOmHkENswC/BDzZ6/I3A389hFYdCh9W9TcZcAykwM5zcbkanAuzcgle8RNKrdNGQatsS3FtOows2ZOQm3MPXM9OQpbKQJ+G2IFCnosC6680uzGlNkdUQImTNBTV1eHbdu0trYOuL21tZXGxuxdh4PBIMGgFFEpqooK+NOf8lLocPvBLiZHHcoTnwf8Cp9l4aYKu5mlhczS/LZK1zxRQKXfxo5bqfuM45gnJ4t74blHc/ve7ZUmJ0mrwlZk3ldnPioj8NbdZiasEEYbsiST5ulXCHPEMz3a5OuUBwMD+myFEnWjsj3mwN1biX9rU7DT4tjLYaNiWTBnDvzXf0mujig5EyboCQQCnHnmmaxbt46LL74YANd1WbduHVdddZW3gxOD9a/1c/PN8OabI7q7BtZvO8jfJqo5W8pUwlUk2lVo077Cbyl8tkJlZJ32v5D4+u32KjVzO8H5l/Tn2RJ2Ryofx8hVPh9HqcwymCM7ulJQEfDhD+T+stp/9iq5E9HV6VmjvPUOVQqamsyWc8nZESVswgQ9ACtXruTSSy9l0aJFLF68mHvvvZfu7u7Ubi4xDgUCpsDZnXeapa8LL8zpbo6r2X2kx+Q+AH7bwrLNDI7uVwhlUsBngqAcTYSZnlKS+aMaS8DluumGJ6bMwcieCJYauHSaq+RTzmcptKVM0cG4NShQHxXbhgUL4LzzJNgRggkW9Hz+85/nwIEDfPOb32T//v2cfvrprF27dlBysxinKirgqafg9NMhY5kyk9aaSNxJ7XSxlKIq6CPgs9DapP8mg6FcKcZhl2pRFImNUgOiqJFWNhjLU0clDqAwyctjJoUEhchqQgU9AFdddZUsZx3Pkju+9u+Hk04ytX6yMA1K7Yx344nlAJVepBhpEGMhEz3Hs7GECyO97+gXwgpMupwLMSQphC/Gp8ZGaG83S16XXWYKpfVvTGopmieV4bhD9exKd24vlHF1oRNjolJ/9LtthD9gz54P4TC8+67ZEdnRYX53hBBZTbiZHjHBZG53/9jHYNMmFLD0xDpTg4dETyM0riYRCJnu7Zk9tcTENuYk6kSCTebT5ljHLPqzrKIC/vznYj+qEMc9CXrE8aOiAv7619Sncw8fJhbyoTpMq4m+uEO51gRss8w12nhnZLnMkvk8EQzK58nhPirjbyHE+CfLW+K45rdN9drKoJ9pVWVUhvxUhPxjvhDlfn8JeCaCRCrYgM+P+f05fF9eWJaZ4fzmNyESkSUsIcZAZnrE8au2FrZvR61ahe+ZZ/A5iTLDeSpem20Op4h9rcUwChFsDBe+9l82K9rMjlJw6qnwu99JEUEh8kRmesTxLVnk8A9/MLVI8pzDoxj+Xb2SxY2C8+oM96+G3H8MBZ/bs224+mro7IT16yXgESKPJOgRE0My4XnXLtiwAfz+gj5c0ZY2RNGEAxY+y8LGLJv23/mXrQhi3gWDsGQJbNlinstSX0eIvJOgR0w8zc3Q1weHDpntvGLCKFSgaSmTF5as8K0wyfH9e7Rlylvg4/fDN74Bhw/Ds8/KzI4QBSQ5PWLiqqiAv/zFXEwWLIB9+7wekcijXPbNjXaruQbijma4+cLhAqKcBALw9tsmSBdCFIUEPWLiq62FPXvMv/fvh3nzxnxI2bNVPANrbvdLKs4h6hlTLy6t8zuzpJR5Lm7aZP4WQhSdBD2itDQ2wssvjzrw6X/RdZG8nmLIPMfJz32WwsnTTr1MOodoaUQBVThsZnUkT0cIT0nQI0rP3LnmqtbVBZ/4hCnhPwIDZhtEwWVLPDR1dRIVKEfaGTQHuWwCzOnnv3Ah/P73kqcjxDghicyidCVL+W/ZYuqh+P05b3lPXnSztSuQnV3FU8jznGxxMiJKmUKCvb0mGNu4UQIeIcYRCXqESLa36OqCL38557tl1vDJ9wVYAidjuFm1sczxDHd+FeCzh/8JDPiZ+/2wYgXs2CHbzYUYxyToESIpEIA1a8w79M2bvR7NcauQwZpmcGLzaORyP1ulS09mC65URQVcd51pDRGNwl13SbAjxDgnOT1CZPP+95vgZ/dumDMH4nGvR1SykoGOyvL3aHJ6RhIoZc3f8vtR27fLVnMhjkMy0yPEcJqbIRYz7+a/8Q3TIiBHpbo8le+04mwzOy7g6uKcYwWoadNQhw6htEZFoxLwCHGckqBHiFwEAmb5Ih43lZ5nzMgp6blUA598GuocJismHys3J5djDb6jMgUtOzvNTFJLi9TWEWICkKBHiJFKFjt0HNi5s2B9viwmdtA0lgTwVF3CxNJWLh3Qh30cpczsTTJHJxKRZp9CTECS0yPEWDQ3myTWri6zVXnDhgE5JvnYXXS8VX/ONYjJ9f+lM/6dOn7GTE+285VLMMTmzSaHSwgx4clMjxD5kNz27rom6Nm3D8rL83LoQm6LP67121014OYsHwPU15slSq3NhwQ8QpQMCXqEKITGRujq4q197Vz/g7VEw/kJgGB8Bz7FCswG7N4ajt8PZ52Vzs3RGlpbJT9HiBIly1tCFFB5wEe8roE7n3ieypCfjt4Y2154g4dv/QJ+DxeuculQPppjFvL4mSylUJkP8rWvwT33SL0cIURWEvQIUUDTa8LMmVLBG3vbqQj6qAz50E0z+cRdf2RSmZ8jPTFmBFzuuOsrNO7eUdSx5TMwGWqZKSnfAZCFQoWCZsbm5Zdl5kYIkRNZ3hKigCxLccHJ9QR8Fpt2H2Ffey8zJ4exLcXuI73YtsXU6XX86AdP8rWfvcg//7/X2NHagUrmBVVWFmxsx0uCdDJ4ilo+lt/+f9lzsAulXbNktWuXBDxCiJzJTI8QBbS9rZOnN7fRHYnT1hFh16EeygI2teUB6itDBP0WHb0xIjGXU6dX87GTG5hbnwh0Ghuho2PwQaNR1LXXon/4w1GNqRhLT2PR4wtw9apf0j1pSuq2vpjDgc4In1wwjRmTyjwcnRDieCZBjxAFsr2tk4fX7+Rwd5Tm2jJOaqjkQGeEve29TC4P8tXz5lAe8tEdjVMe8DG9Joxl5ZAGHAjAffeh7ruP7W2d/O6NVnYc6CISdwj6bGrK/FS07GbFVz8xaCq3UAHPsYoEAnRNmsIttz/OjNnTUf0SkLXWbGvtpDvqgIYjPRH6Yi4VcQe/bRGNO+zviNBYFeLvz5ye2zkSQogsJOgRogBcV/O7N1o53B1lXn1F6iI/tSZMY3WIt9u6+ONbbVx53pwxXcTn1lcye1kFLUd7BwRP8H5aPtedNaB673AP9zy9jZoyP5UhU1jxcHeEV/YcpTfqEPBZOK7mtBnVtHVG2N7WRXXIjws4rottWdSWB5gzpRy/bXG0J8a1F5xIU+3QMzAKaG3rJLx+J2+3dTG1OkQ4YNMbddjX3sfkiiCfnV/PH7e2gYKeSJzOSJxoPEo0rplaFeLqj87jxIaqUZ8rIYSQoEeIAmg52suOA+birjK2VSulmFodYntbFy1He4cNFnJhWSrrMYY6bnc0Tl/coSwQBsxMy462bnqjDrXlATRwpCdKwGczvSbM1v2dVIX9zJ9aSczRBGyLypAPpRRx16W1o4/u6LEbss6tr+TypSekZqZaO/oI+uwBy3ozJ5fxuzda2d7WydHeGJaCOfUVfGZhEyc2Fi6/SQhRGiToEaIAMgOLTOGAnXOwkG/lAR8hn01PNE5lyE9nX5zDPVEqEoFMNO7gsywCtlkcqwj6ONAV4X1UUVcxcCt4b9QsqZUHhn8pcV1Ny9Fe4q7mkwumooHemDNoFmqomStZ0hJC5IMEPUIUQGZgkSnXYKEQMrfRRx2XuOvit31orenqi1NfFaIyZMZWVxFg16EeInEHSP9ftNbsa+/j1OnViSW17La3dbL2jf283tJOTzROWcDHqdOr+fgpjVlno4aauRJCiLGSoEeIAsgMLDITd3MJFgrFshQXntLA3vZe3m7roiLow1KK7kicaNwlHPAxZ0o6D2l6TZi2jggtR3sJ+e0BuTi15QE+dnLDkDMx29s6ufcPb7NtfyeO1iRTnt890M3W/Z1cc/689G41IYQoMKnTI0QBJAOL2vIAb7d10dkXI+66dPbFeLut65jBQqEl82tOmVZN3HEBONoTY0plkNObaqgtN8tYWmt6Yy4fmV/PB06o5WhPjJ0HuznaE+PU6dVcvvSEIYMW19U89vxuXt1zFMd1qQz5qC0PUhny4bgur+45ymPP78Z1x/MGeiHERCIzPUIUSC6Ju16PL5k/s2VfB795fR+RmIPfNgnK/WdzvrCkmdl1I8u1ee9IDxveOYStYHJFMDVzFPTZBCosWjv6eP6dQ7x3pIfmyfnrTSaEEEORoEeIAhrvibnJ/Jmm2jJmTyk/ZoA2klybdw52094TY3JlIOsOtuoyP4e6orxzsFuCHiFEUUjQI0SBHS+JuYUI0LQCNWTZwvER+AkhSocEPUKIlHwGaLPqyqkJBzjaE6OhyhqUzN3eE6M6HGBWnczyCCGKQxKZhRAF0TSpjA/OqsXVmkPdUSJxB1drInGHQ91RXK05a3YtTdJLSwhRJDLTI4QoCMtS/O8PNtPWFWFbayedfelCjLalWNBUwxeWNI+b/CYhxMQnQY8QomDm1ldyzfnzWPt6ojhhLE6Z38dpM6q58JRGz3ewCSFKiwQ9QoiCmltfydc+PH53sAkhSocEPUKIgjtedrAJISY2SWQWQgghREmQoEcIIYQQJUGCHiGEEEKUBAl6hBBCCFESJOgRQgghREmQoEcIIYQQJUGCHiGEEEKUBAl6hBBCCFESJOgRQgghREmQisz9aK0B6Ojo8HgkQgghhMhV8rqdvI4PRYKefjo7OwFoamryeCRCCCGEGKnOzk6qq6uH/LrSxwqLSojruuzdu5fKykqUKp1miB0dHTQ1NbFnzx6qqqq8Ho6n5FykyblIk3ORJuciTc5FmtfnQmtNZ2cn06ZNw7KGztyRmZ5+LMtixowZXg/DM1VVVSX/i5sk5yJNzkWanIs0ORdpci7SvDwXw83wJEkisxBCCCFKggQ9QgghhCgJEvQIgsEgt9xyC8Fg0OuheE7ORZqcizQ5F2lyLtLkXKQdL+dCEpmFEEIIURJkpkcIIYQQJUGCHiGEEEKUBAl6hBBCCFESJOgRQgghREmQoEcMKRKJcPrpp6OU4pVXXvF6OEW3c+dOvvzlLzNr1izC4TBz5szhlltuIRqNej20orjvvvs44YQTCIVCLFmyhI0bN3o9pKK74447+MAHPkBlZSX19fVcfPHFvPXWW14Py3Pf+c53UEpxzTXXeD0Uz7S0tPDFL36RyZMnEw6HOfXUU3nxxRe9HlbROY7DzTffPOB1ctWqVcfsgeUVCXrEkK677jqmTZvm9TA8s3XrVlzX5YEHHmDz5s3cc889rFmzhptuusnroRXcE088wcqVK7nlllvYtGkTCxYs4MILL6Strc3roRXVM888w/Lly9mwYQNPP/00sViMj33sY3R3d3s9NM+88MILPPDAA5x22mleD8UzR44cYenSpfj9fv77v/+bN998k3/7t39j0qRJXg+t6O68807uv/9+fvCDH7BlyxbuvPNOvvvd77J69Wqvh5adFiKL3/72t3r+/Pl68+bNGtAvv/yy10MaF7773e/qWbNmeT2Mglu8eLFevnx56nPHcfS0adP0HXfc4eGovNfW1qYB/cwzz3g9FE90dnbqefPm6aefflqfd955esWKFV4PyRPXX3+9Puecc7wexrhw0UUX6S996UsDbvv0pz+tL7nkEo9GNDyZ6RGDtLa2csUVV/Czn/2MsrIyr4czrrS3t1NbW+v1MAoqGo3y0ksvcf7556dusyyL888/n+eee87DkXmvvb0dYMI/B4ayfPlyLrroogHPjVL061//mkWLFvHZz36W+vp6zjjjDH70ox95PSxPnH322axbt45t27YB8Oqrr/KXv/yFv/mbv/F4ZNlJw1ExgNaayy67jCuvvJJFixaxc+dOr4c0bmzfvp3Vq1dz1113eT2Ugjp48CCO49DQ0DDg9oaGBrZu3erRqLznui7XXHMNS5cu5ZRTTvF6OEX3+OOPs2nTJl544QWvh+K5d955h/vvv5+VK1dy00038cILL/D1r3+dQCDApZde6vXwiuqGG26go6OD+fPnY9s2juNw2223cckll3g9tKxkpqdE3HDDDSilhv3YunUrq1evprOzkxtvvNHrIRdMrueiv5aWFj7+8Y/z2c9+liuuuMKjkQsvLV++nDfeeIPHH3/c66EU3Z49e1ixYgWPPvoooVDI6+F4znVdFi5cyO23384ZZ5zBV77yFa644grWrFnj9dCK7he/+AWPPvoojz32GJs2beKnP/0pd911Fz/96U+9HlpW0oaiRBw4cIBDhw4N+z2zZ8/mc5/7HE899RRKqdTtjuNg2zaXXHLJuH0ij0Su5yIQCACwd+9eli1bxgc/+EF+8pOfYFkT+71CNBqlrKyM//iP/+Diiy9O3X7ppZdy9OhRnnzySe8G55GrrrqKJ598kj//+c/MmjXL6+EU3X/+53/yqU99Ctu2U7c5joNSCsuyiEQiA7420c2cOZMLLriAH//4x6nb7r//fm699VZaWlo8HFnxNTU1ccMNN7B8+fLUbbfeeiuPPPLIuJwZluWtEjFlyhSmTJlyzO/7/ve/z6233pr6fO/evVx44YU88cQTLFmypJBDLJpczwWYGZ4Pf/jDnHnmmTz88MMTPuABCAQCnHnmmaxbty4V9Liuy7p167jqqqu8HVyRaa25+uqr+dWvfsWf/vSnkgx4AD760Y/y+uuvD7jt8ssvZ/78+Vx//fUlFfAALF26dFDpgm3btjFz5kyPRuSdnp6eQa+Ltm3juq5HIxqeBD1igObm5gGfV1RUADBnzhxmzJjhxZA809LSwrJly5g5cyZ33XUXBw4cSH2tsbHRw5EV3sqVK7n00ktZtGgRixcv5t5776W7u5vLL7/c66EV1fLly3nsscd48sknqaysZP/+/QBUV1cTDoc9Hl3xVFZWDspjKi8vZ/LkySWZ33Tttddy9tlnc/vtt/O5z32OjRs38uCDD/Lggw96PbSi++QnP8ltt91Gc3MzJ598Mi+//DJ33303X/rSl7weWnbebh4T4927775bslvWH374YQ1k/SgFq1ev1s3NzToQCOjFixfrDRs2eD2kohvq5//www97PTTPlfKWda21fuqpp/Qpp5yig8Ggnj9/vn7wwQe9HpInOjo69IoVK3Rzc7MOhUJ69uzZ+p/+6Z90JBLxemhZSU6PEEIIIUrCxE9QEEIIIYRAgh4hhBBClAgJeoQQQghREiToEUIIIURJkKBHCCGEECVBgh4hhBBClAQJeoQQQghREiToEUIIIURJkKBHCCGEECVBgh4hhBBClAQJeoQQQghREiToEUJMWD//+c8Jh8Ps27cvddvll1/OaaedRnt7u4cjE0J4QRqOCiEmLK01p59+Oueeey6rV6/mlltu4aGHHmLDhg1Mnz7d6+EJIYrM5/UAhBCiUJRS3HbbbXzmM5+hsbGR1atX8+yzz0rAI0SJkpkeIcSEt3DhQjZv3szvf/97zjvvPK+HI4TwiOT0CCEmtLVr17J161Ycx6GhocHr4QghPCQzPUKICWvTpk0sW7aMBx54gJ/85CdUVVXxy1/+0uthCSE8Ijk9QogJaefOnVx00UXcdNNNfOELX2D27NmcddZZbNq0iYULF3o9PCGEB2SmRwgx4Rw+fJizzz6bZcuWsWbNmtTtF110EY7jsHbtWg9HJ4TwigQ9QgghhCgJksgshBBCiJIgQY8QQgghSoIEPUIIIYQoCRL0CCGEEKIkSNAjhBBCiJIgQY8QQgghSoIEPUIIIYQoCRL0CCGEEKIkSNAjhBBCiJIgQY8QQgghSoIEPUIIIYQoCf8fYxj3GlZgYsoAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "ax.scatter(test_data[:, 0], test_data[:, 1], alpha=0.5, label=\"Test data\")\n",
    "ax.errorbar(test_data[:, 0], b_mean, yerr=2*total_uncertainty, alpha=0.5, fmt='or', label=\"95% band total unc.\")\n",
    "ax.errorbar(test_data[:, 0], b_mean, yerr=2*b_sigma, alpha=0.5, fmt='og', label=\"95% band epistemic unc.\")\n",
    "ax.set(xlabel=\"$x$\", ylabel=\"$f(x)$\")\n",
    "#ax.set_yscale('log')\n",
    "plt.legend(frameon=False)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "55635d1f",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}