diff --git a/CHANGES.rst b/CHANGES.rst index 7091b2b81..38794760c 100644 --- a/CHANGES.rst +++ b/CHANGES.rst @@ -21,7 +21,8 @@ moments. The breaking changes are: - structure of the logging database has changed - there is an additional boolean flag named ``scaling`` in minimize and maximize - +- :gh:`251` Allows the loading, running and visualization of benchmarks + (:ghuser:`janosg`, :ghuser:`mpetrosian` and :ghuser:`roecla`) - :gh:`196` Adds support for multistart optimizations (:ghuser:`asouther4` and :ghuser:`janosg`) - :gh:`248` Adds the fides optimizer (:ghuser:`roecla`) diff --git a/docs/source/_static/images/history_trust-ncg.gif b/docs/source/_static/images/history_trust-ncg.gif index 5119ccfc9..e69de29bb 100644 Binary files a/docs/source/_static/images/history_trust-ncg.gif and b/docs/source/_static/images/history_trust-ncg.gif differ diff --git a/docs/source/how_to_guides/optimization/how_to_benchmark_optimization_algorithms.ipynb b/docs/source/how_to_guides/optimization/how_to_benchmark_optimization_algorithms.ipynb new file mode 100644 index 000000000..d99e77049 --- /dev/null +++ b/docs/source/how_to_guides/optimization/how_to_benchmark_optimization_algorithms.ipynb @@ -0,0 +1,297 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "6be356db", + "metadata": {}, + "source": [ + "# How to Benchmark Optimization Algorithms\n", + "\n", + "Benchmarking optimization algorithms is an important step when developing a new algorithm or when searching for a algorithm that is good at solving a particular problem. \n", + "\n", + "In general, benchmarking constists of the following steps:\n", + "\n", + "- Defining the test problems (or getting pre-implemented ones)\n", + "- Defining the optimization algorithms and the tuning parameters you want to try\n", + "- Running the benchmark\n", + "- Plotting the results\n", + "\n", + "Estimagic helps you with all of these steps!" + ] + }, + { + "cell_type": "markdown", + "id": "8671802f", + "metadata": {}, + "source": [ + "## Get Test Problems\n", + "\n", + "Estimagic includes the problems of [Moré and Wild (2009)](https://epubs.siam.org/doi/pdf/10.1137/080724083) as well as [Cartis and Roberts](https://arxiv.org/abs/1710.11005).\n", + "\n", + "Each problem consist of the `inputs` (the criterion function and the start parameters) and the `solution` (the optimal parameters and criterion value) and optionally provides more information.\n", + "\n", + "Below we load a subset of the Moré and Wild problems and look at one Rosenbrock problem with difficult start parameters." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "c3640855", + "metadata": {}, + "outputs": [], + "source": [ + "from estimagic import get_benchmark_problems\n", + "\n", + "problems = get_benchmark_problems(\"example\")" + ] + }, + { + "cell_type": "markdown", + "id": "c628b987", + "metadata": {}, + "source": [ + "## Specifying the Optimizers\n", + "\n", + "To select optimizers you want to benchmark on the set of problems, you can simply specify them as a list. Advanced examples that do not only compare algorithms but also vary the `algo_options` can be found below. " + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "2340cd3a", + "metadata": {}, + "outputs": [], + "source": [ + "optimizers = [\n", + " \"scipy_lbfgsb\",\n", + " \"scipy_neldermead\",\n", + " \"scipy_truncated_newton\",\n", + "]" + ] + }, + { + "cell_type": "markdown", + "id": "9703f954", + "metadata": {}, + "source": [ + "## Running the Benchmark\n", + "\n", + "Once you have your problems and your optimizers, you can simply use `run_benchmark`. The results are a dictionary with one entry for each (problem, algorithm) combination. Each entry not only saves the solution but also the history of the algorithm's criterion and parameter history. \n" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "f0ef15da", + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "from estimagic import run_benchmark\n", + "\n", + "results = run_benchmark(\n", + " problems,\n", + " optimizers,\n", + " logging_directory=\"benchmark_logs\",\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "61c365ab", + "metadata": {}, + "source": [ + "## Profile plots\n", + "\n", + "**Profile Plots** compare optimizers over a whole problem set. \n", + "\n", + "The literature distinguishes **data profiles** and **performance profiles**. Data profiles use a normalized runtime measure whereas performance profile use an absolute one. The profile plot does not normalize runtime by default. To do this, simply set `normalize_runtime` to True. For background information check [Moré and Wild (2009)](https://epubs.siam.org/doi/pdf/10.1137/080724083). " + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "07918672", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjgAAAGoCAYAAABL+58oAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABb5ElEQVR4nO3deZxcZZX/8c+p6jV7ICGGhJCABJIQCCEJCftmABNJEBQREFRElHUcFhlEwjiOjisyKIgOi4pEZdh+yAgGibIEshGWsIZAdrOStbdazu+PWrq60+muXm7f6urv+/XqV27dqrp1bgOpw3nO8zzm7oiIiIgUk0jYAYiIiIh0NCU4IiIiUnSU4IiIiEjRUYIjIiIiRUcJjoiIiBSdkrADaK3TTz/d//KXv4QdhoiIiBQGa+pkl6vgbNq0KewQREREpMB1uQRHREREpCVKcERERKToKMERERGRoqMER0RERIqOEhwREREpOkpwREREpOgowREREZGiowRHREREio4SHBERESk6SnBERESk6CjBERERkaKjBEdERESKTmAJjpndY2YbzOyNPTxvZna7mS0zs9fMbHxQsYiIiEj3EmQF5z7g9GaePwM4KP1zKXBngLGIiIhIN1IS1IXd/R9mNryZl8wAfuPuDrxkZv3MbLC7rwsqprC8sXwD/3htBcmkhx2KBKBy1zYOeG8JJbFY2KGIiLRaeXIHfWIbMBKBfUbZaTMYcdpZgV2/KYElOHkYAqzKebw6fa6oEpxk0vn+759nR1Vd2KFIQL7yz/kcWPXPsMMQESlYVSve7/TPDDPBsSbONVniMLNLSQ1j8fGPf5xFixYFGVeHiieSfOqw3tnHJVH1dRcTA/aqPpo1Dm6QbPJfaxGRwmGA4VjTX7kdLlpTTd/Etk75rFxhJjirgf1yHg8F1jb1Qne/G7gbYMKECX7kkUcGH10HeWvFJn7/6BwAvvWF45h4yJCQI5KOVPP222yY9T8A7H3F5fQ89piQIxIRaVpyxzrqXr6b2NJHIBlPnzVKRk2nfPLXiPQbFmp8HS3MBOdx4Aozmw0cBWwrxv6b9R/tzB4P6t8rxEgkCDWv108SrDh0TIiRiIg0LblrE3ULfk3stT9Aor5XsOSg0yib8nWiex8YYnTBCSzBMbMHgROBAWa2GrgFKAVw97uAJ4FPAsuAKuCLQcUSpg0f7coe79O/Z4iRSBBq31gKQOl++xHt1y/cYEREcnj1VuoW3kvdkt9DvDp7PjriBMqPvoLoPqNCjC54Qc6iOq+F5x24PKjPLxTr0wlO357lVJSFWTCTjpasqaF22TJA1RsRKRxeu5O6xb+hbvFvoK5+FCE6bDLlR19JdPDhIUbXefSNG7BMBUfVm+JT+9ZbkEhNq6w49NCQoxGR7s5jVdQteZC6hfdATX1Tb2TwOMqPuYqS/SaFGF3nU4ITsA3pHhwlOMWnJj08RSRC+ahDwg1GRLotj9cRe/2P1M3/FV61OXs+ss/oVMVm+LGYdb8ZnkpwApRIJtm4tQpQg3ExyjQYl33840R69Ag5GhHpbjwRI/bmY9S9fBe+o34trsjeH6fs6CsoOfCUbpnYZCjBCdCW7dUk0qsXq4JTXBLbthFbuRJQ/42IdC5PJoi/8yS1836Bb6tfL9f6DaN8yuWUjDwdi0RDjLAwKMEJ0Pot9TOoBinBKSo1S5dmjyvGqv9GRILnniT+3hzq5t1Bcsvy7HnrPZiyyZdROupMLFoaYoSFRQlOgDZs1RTxYpXpv7HycsoPOijkaESkmLk7iQ/+Qe28O0hueCt73noMoOyor1J66NlYSVmIERYmJTgBWr+lfnrePv2U4BSTTP9N+SEHYyX6z0hEghFf+TK1L95Oct2r2XNW0Y+yiV+m9PDPYaWVIUZX2PQ3c4AyFZy9+lRSVqrx0GIRX7+exMaNgKaHi0gwEmtfofbF/yaxan79ybJelB15MWXjL8TK9D/NLVGCE6BMD46qN8UlOz0cJTgi0rES69+kdt5/k/jgufqTJZWUHXE+ZRO+iFX0DS+4LkYJToAyFZxBeynBKSaZ4alI716U7l9cm9OJSDgSm5ZRN+/nxJf9tf5ktIzSwz9H2cQvE+mxd3jBdVFKcAISTyTZvC2194cqOMXDk8nsDKqKMWOwSCTkiESkK0tuXUntvJ8Tf/tJILWsCJESSg/9NGVHfZVIr0GhxteVKcEJyKZtVSQ99S+rKjjFI7ZyJckdOwANT4lI2yW3r6Pu5buILX0UPLXlCxahZNSnKJ/8NSJ9h4YaXzFQghMQzaAqTrn9N+VKcESklZI7N1K34FfEXv8TJGLZ8yUjT6dsyteJ7nVAiNEVFyU4AcnsIg4waC9t01AsMv030QEDKBm0T8jRiEhX4dVbqV3wP8RefRDiNdnzJQecRNnRlxMdqP3sOpoSnIBkdhGPmDGgr/YpKgYej1P79ttAavXi7rzHi4jkx2t3ULfofupe+S3U1f+Pb3T/o1MbYX5sbIjRFTclOAHJVHD27ltJSVSNqMWg9r1leG0toP4bEWmex6qoe+X31C28B2q3Z89H9x1P2TFXUTJ0QojRdQ9KcAKSqeBoD6riUfPGG9njijHaYFNEdufxWmKv/ZG6Bb/Cq7Zkz0cGjaH86KuI7n+0qr+dRAlOQNZ/lGoy1h5UxaM2neCUDtuPaD8ttiUi9TwRI7b0Eepe/iW+c332fGTAQZQdfSUlB5ykxKaTKcEJQF0swUc7Uk1kg/qrwbgYJKurqV32PqDhKRGp58kE8befoHbenfj21dnz1n845VO+TsnI0zFTm0IYlOAEQLuIF5/at96GRGqtCiU4IuKeJP7u09S99HOSWz7Inrc++1I++WuUjPoUFtFXbJj02w/Ahtwp4kpwikJmejjRKOWjNJ1TpLtydxLL/07tvP8mufGd7HnruQ9lR11K6aFnY9HSECOUDCU4AchdA0cVnOJQszSV4JR//EAilZUhRyMinc3dSax8idoX/5vkP1/LnrfK/pRNvITSw8/FSipCjFAaU4ITgEwFpyQaYa8++jLs6hJbtxFbuQrQ6sUi3VF8zWLqXrydxOqF9SfLe1N25MWUHXEBVqb/kS1ESnACkJlBNaBvD6LajLHLy2yuCeq/EelOEv98g9p5d5D48Pn6k6WVlB1xIWVHXoRVaDZlIVOCE4BMBUfDU8Uhs/6NlZdTftDHQ45GRIKW2PQedfPuIL7smfqT0XJKx32OsglfJtJjr/CCk7wpwQmAFvkrHu6ebTAuH3UIVqL/ZESKVfKjFdTO+znxd/4P8NTJSAmlY8+hbNKlRHpp/7muRH9bd7Dq2hjbdqWW81cFp+uLr99AYtMmQMNTIsUquX0tdS/dSezNx8FTy0FgEUpHz6DsqMuI9B0SboDSJkpw2sHd2bKjhmQymT23bvPO7LEqOOFydxIffQRJb/M1qhfMzx5XjFWCI12fexLfuYFshaIb81g1sSUPEnv9T5CMp88aJQefQfmUrxPpPzzM8KSdlOC0w/ceeJ6X31yzx+f30SrGoXF3Nv7n9+rXr2mnSO/elO63X4dcSyQs8Q9foOZv38G3rW75xd1QyYEnU3b0FUQHjAw7FOkASnDaKBZPNJvc9CgvZdigPp0YkeSKr1vXYckNQOXECZhmxEkXlazaTO3ff0D87T+HHUpBig4/lvIplxP92NiwQ5EOpASnjeriiezx0IF9mHncwQ2eH7X/QHpWlHV2WJJWs/TN7HHf8z5HtHfvNl/LKiqoHH9ER4Ql0qncnfibj1Hzjx9CzbbUybKelE34EtZzQLjBFYjogJFKbIqUEpw2qovVJzjTpxzEJyYcGGI00lhtOsGJ9O5Nn09NV/VFup3kRyuoeeZWEqvq+8hKDjyZ8pP+jUjvj4UYmUjnUILTRrkJTllpNMRIpDF3p+atVIJTPnqUkhvpVjwRo27RvdS9dBck6oDUPknlJ/8bpR8/NeToRDqPEpw2qm2Q4OjXWEhiq9eQ3LYdgIoxY0KORqTzJNa9Ss1fZ5Hc/F76jFF6+LmUH3M1Vt72YVqRrkjfzG2Um+CUq4JTUGpzt1YYMzrESEQ6h9fupPaFnxF7dTaZ6d+RvT9Oxam3EN1X/WPSPSnBaaO6WDx7rCGqwlLzZrr/pl8/SvbdN+RoRIIVW/YMtc/+J75zfepEtIyyo76aaiSOloYbnEiIlOC0kSo4hcmTSWrffAuAitGjMLOQIxIJRnLnemqf/c8G+yVFh06k4tRbtECdCEpw2ix3mni5enAKRmzVKpI7U6tJV4zW8JQUH/cksdf+SO3zP4W61L53lPeh4oTrKBk9U0m9SJq+mduowSyqElVwCkXu+jfl6r+RIpPY9B41c2aRXPdq9lzJIdMoP+F6Ij32DjEykcKjBKeNNE28MGXWv4nutRclH9NaH1IcPF5L3cu/pG7hPdk9k6zPECpOuZmS4ceGHJ1IYVKC00a1ajIuOJ5MUvNWqv+mfPRoleqlKMRXzadmzq341hWpExahdPwXKJ/yday0R7jBiRQwJThtpCbjwhP78EO8qgqAikM1PCVdm1dvpea5HxFf+mj2XGSf0VScOovoIP37LdISJThtpCGqwlOTu/7NaC3wJ12TuxN/50lq5/4XXr0ldbKkkvJjrqR03OexiP7aFsmH/ktpo0wFpyQaIaqtAApCpsE4OnAgJfsMDDkakdZLbltNzTPfIbHihey56PDjqDj5W0T6DgkxMpGuRwlOG2UqOBqeKgwej1P79juAVi+WrseTcWKLf0vtvJ9DvAYA67EX5SfeSMnI09VPJtIGSnDaqC6eajLW8FRhqPvgA7wm9cWg9W+kK0n88w1qnrmV5Ia3sudKDz2b8uO+gVX0DTEyka5NCU4bZSo4WgOnMGj9G+lqvK6K2nl3EHvld+BJAKz/cCpOvYWSoRNDjk6k61OC00a12SEq/QoLQWb9m5KPDaJkby14JoUt/sE/qHnmO/iOdakTkRLKJl5C2aSvYCXl4QYnUiT07dxG2QqOhqhC5/E4te++C6TWvxEpVMldm6id+33i7/4ley667xGUnzqL6N4HhhiZSPFRgtNGtUpwCkbtsmV4bS0AFWM0PVwKj3uS2BsPU/vcT6B2e+pkWS/Kj/sGpWPPwUwzMUU6mhKcNspstqlZVOGrzem/qRg9KsRIRHaX2LKc2jn/TmLNwuy5koOmUn7iN4n02ifEyESKmxKcNsps1aAKTvhq3kz33wwZQrR//5CjEUnxeB11C/+Huvl3QyIGgPUaRMXJ36LkwJNCjk6k+CnBaaPsOjiaRRUqr6uj9t33AK1/I4UjvmYxtXNmkdyyPH3GKB33ecqPuQor6xlqbCLdhRKcNlKTcWGofW8ZxFL/d6z1byRsXrOd2ud/Suz1P2XPRQaMTO0fNfiwECMT6X6U4LSRpokXhszwFEC5+m8kJO5O/L2/Uvvsf+JVm1Ino+WUTfk6ZeO/gEVLww1QpBvSt3MbqYJTGDINxqXD9iPap0/I0Uh3lNyxjpq/fZfE8rnZc9Fhk6k45dtE+g0LLzCRbk4JThu4e3YWlRKc8CRra6l9L9V/o/VvpLN5MkHs1QepfeF2iFUBYBX9KD/hekpGfUr7R4mETAlOG2SSG9AQVZhq33kXEql/Fmowls6U2Pg2NX+dRXL9G9lzJaPOpPyE64hUaiafSCHQt3MbZPpvQBWcMNUuXZo6MKNilPpvJHgeq6bupbuoW3QfeOrvAes7lIpTbqFk/ynhBiciDSjBaYO6WG4FRwlOWDINxqX770+kV6+Qo5FiF1/xIjXP/Du+bXXqhEUpm3AxZUddhpVWhhuciOwm0ATHzE4HfgZEgV+7+/cbPd8X+B0wLB3Lj9z93iBj6gh1quCELlldTd37qTVGtD2DBClZtYXaf/yQ+Fv/L3suMuhQKj4xi+jAQ0KMTESaE1iCY2ZR4OfAJ4DVwAIze9zd38x52eXAm+7+KTMbCLxjZg+4e11QcXWEzCrGoIX+wlL79juQTALqv5FguDvxtx6n9u8/xGu2pk6W9qD8mKsoPfw8LKL/9kUKWZAVnEnAMndfDmBms4EZQG6C40BvS0036AVsAeKNL1RIqmpi3Pd/S7KPVcHpeImdO9n+yGMkd+7Y42tiq1alDiIRyg85uJMik8bi788l/v4zuHvYoXQ437qCxNpXso+jB5xIxck3Eek9OMSoRCRfQSY4Q4BVOY9XA0c1es0dwOPAWqA3cK67JwOMqd2eWbScJcvWZx/3qCgLMZritO2Pf2Ln03/N67VlI0YQ6dEj4IikKXWvP0TtnFlhhxE46zGA8pP+jZKDPqGp3yJdSJAJTlN/EzT+37zTgCXAycCBwF/N7Dl3397gQmaXApcCDBsW7sJZH+2syR4fO3YYI/fbK8Roio/H41TNmweAVVQ02zwcqSin72fP6azQJEfs/b9R+8y/px5Ey7EeRfjfgUUpOeB4yqdcgVVoEUmRribIBGc1sF/O46GkKjW5vgh831P17WVm9gFwCDA/90XufjdwN8CECRNCrYUnEqkCU3lplOvOOzrMUIpSzWuvk9yxE4C9vvwleh53bMgRSWPx1Yuo+fN14EkoqaDH2b8muu+4sMMSEWkgEuC1FwAHmdkIMysDPkdqOCrXSuAUADMbBBwMLKeAJZKp/CoaCfJX133tev4FAKy8nMqJE0KORhpLbHqX6sevgEQtWJTKaT9WciMiBSmwCo67x83sCuApUtPE73H3pWZ2Wfr5u4DvAPeZ2eukhrRucPdNQcXUEeLpCk40qrH4jpasqaF60SIAKo88kkhFRcgRSa7ktjVUP/xVqE01f1dM/XdKDjgh5KhERJoW6Do47v4k8GSjc3flHK8FpgYZQ0dLpKcml0RVwelo1QsX4rW1APQ8RsN/hSRZtYWqR76K79oIQPlx36B09IyQoxIR2TN9S7dSZogqElEFp6NlhqcivXtRcfhhIUcjGV5XRfWjX8c/+hCA0iMvomzCl8INSkSkBUpwWinTZFyiHpwOldi2jZrXXgegx+TJWIl2ESkEnohR/cQ12U0lSw6ZTvlx/xpyVCIiLdO3dCvFE+kmYw1RdaiqeS9lVybuoeGpguCepOapm0iseBGA6PDjqJj6Hcz0776IFD79TdVKmR6cqIaoOtSuF9JfogMGUD5yZMjRiLtT+/cfEH8n1UIX+dhhVE7/MRYtDTkyEZH8KMFppUwPjpqMO058/Xrq3nsPSDUXm4b/Qle34H+IvfI7ACJ7jaDHzF9gpVoxWkS6Dn2TtFKmB0fr4HScTPUGoMexx4QYiQDElj5C3Qu3AWC9BlF51t1YZb9QYxIRaS19S7eShqg6lrtnZ0+VDhtG2X77tfAOCVL8/bnU/HVW6kF5HyrP+iWRPtpcUkS6HiU4raQm444V+3AF8bWpHTx6qnoTqviaxVT/+V/BExAtp3LGHUQHfDzssERE2kTf0q2UXehPFZwOkaneAPQ4ekqIkXRviU3LqH6s4RYMJUPGhx2WiEibKcFppexCf6rgtJsnk1S9mOq/KR91CCUDBoQcUfeU3L6O6ke+CrXbAaj4xCxKDjwx3KBERNpJ39KtpIX+Ok7tW2+R+OgjAHoeo+GpMCSrP6L64UvxnesBKDv2Xygdc1bIUYmItJ++pVspntRmmx0lOzwVjVI5+ahwg+mGPFZF9aOXk/zoAwBKx39BWzCISNFQgtNKiUyTsSo47eKxGFUvzwegctw4or16hRxR95LaguEbJP/5GgAlh0yj/PhrMVPiLiLFQd/SraQm445R/coSvKoK0No3nc09Sc3TN5P48HkAovsfQ8XU/9AWDCJSVPQ3WitlF/pTk3G7ZIanrKKCyvFHhBxN91L73I+Jv/0EAJGPjaVy+k+1BYOIFB19S7dSZhaVFvpru2RVFdWvvAJAj0kTiZSXhxxR91G38B5ii+4HINJ/BJUzf4GVaQsGESk+SnBaKa4KTrtVzZ8PsRgAPTR7qtPElj5G7XM/AcB67kPlp39JpLJ/yFGJiARD39KtlN1sU03GbZYZnor07UPFoWNCjqZ7iC//OzV//XbqQXmfVHLTZ99wgxIRCZC+pVsp02Qc0TTxNkl89BG1S98EoMeUKVg0GnJExS+xdkkTWzAcFHZYIiKBUoLTSplp4qrgtM2uF+eBp36HWtwveInN71P16NchXgMWoXLaj7QFg4h0C/qWbqX6hf70q2uLqhdSw1MlgwZR9vEDQ46muCV3rKP64UuzWzCUn3oLJQeeFHJUIiKdQ9/SreDuJLM9OBqiaq3Y2rXULU+tmtvj2GO0qFyAvHor1Q9/tX4LhmOupuzQs0OOSkSk8yjBaYVMgzFoJeO22PXCi9njnsccHWIkxc1jVVQ9djnJLcsBKB13PmUTLwk5KhGRzqVv6VbILPIHGqJqLXenKj17quyAEZTuqxk8QfBEjOo//yvJda8CUHLwJyk/8QZVy0Sk29G3dCvkVnBKNIuqVeref5/4+tRwida+CYa7U/PXW0h88BwA0WFTqDjtu9qCQUS6Jf3N1wqZBmOAiIaoWiW7c7gZPaZMCTeYIlX3/E+Jv/U4AJFBY6j81G3agkFEui19S7dC7hCVmozz54kEVfPmAVA+ZjQle2n13I5Wt+h+6hbeA4D125/KmXdiZT1DjkpEJDwlYQfQleyqiWWPC6UHJ1ldTXL79rDDaFbte8tIbkvF2LPAdg5P7vgnJGItv7CAxVe9TO0/fgiA9RxIj0/fTaTHXiFHJSISLiU4eVq3eQeX//TJ7ONC2GyzbsVK1t/8bbyuLuxQ8lNaSo9Jk8KOIqvmb/9B7NXZYYfRccp7U3nWXUT6Dgk7EhGR0BVGGaILeH35hgaPhw3qG1Ik9aoXLeo6yQ3Qc8pkIj0KZ+fq2NtPtvyiriJaRuWZ/0104MFhRyIiUhBUwclTbv/NTRcex0FD9w4xmpT4pk0AWM8e9L/oCyFH07xIWTkV4w4PO4wsr9uVXeG3ZOTplIw4LuSI2sOI7jeJSO+PhR2IiEjBUIKTp6TXTxE/ZNiAECOpl9i8GYDSQYPodfzxIUfTtSR3/DN7XHLAiZSOmh5iNCIi0tE0RJWn3DVwIgXQfwMQ35RKcKJ7F0bC1ZX4jnXZY+s9OMRIREQkCEpw8pTMTXAKYFVYdyeRHqIqGRD+cFlXk9xen+BE+ijBEREpNkpw8tRwH6rwE5zkzp14bS0AUSU4reY71qYOLIL1HBhuMCIi0uGU4OQp6bmrGIef4CTSw1MAJQP0Bd1amQqO9dpHq/2KiBShPTYZm9n45t7o7os7PpzClUgUVg9OfPOm7LEqOK3n6SbjiPpvRESKUnOzqH6c/rMCmAC8ChhwGPAycGywoRWW3FlUhdCD06CCoybjVkumm4zVYCwiUpz2OETl7ie5+0nACmC8u09w9yOBI4BlnRVgocj04EQihhVAghPfmK7glJYS6dM73GC6GE8m8J2pnc1VwRERKU759OAc4u6vZx64+xvAuMAiKlCZWVSFUL0BSKSHqEr23gvTzuat4rs2QjIOgGkGlYhIUcpnob+3zOzXwO8ABy4A3go0qgKUTKaajAthBhXkrIEzQMNTrZW7Bo4qOCIixSmfBOeLwNeAq9OP/wHcGVhEBSrh9UNUhSBbwVGC02rJ3EX+VMERESlKLSY47l5jZncBT7r7O50QU0FKpmdRRQtgOMhjMRIfbQUgurdmULVWcnv9Ng2q4IiIFKcWv63N7ExgCfCX9ONxZvZ4wHEVnMwsqkLowYlv3pI9VgWn9bKL/JX1wsrVoC0iUozyKUfcAkwCtgK4+xJgeGARFahEpgcnGn6Ck8hZA0cJTutlFvnTFg0iIsUrnwQn7u7bAo+kwBXSLKp4zho4WuSv9Vxr4IiIFL18mozfMLPPA1EzOwi4Cngx2LAKT2YdnEKYRZXZZBPUg9MWSa1iLCJS9PKp4FwJjAFqgd8D24BrAoypICULaBZVPJ3gRPr2IVJWFnI0XYvX7oTa7YAqOCIixSyfCs7B7n4TcFPQwRSyzF5UkQKYRZXZpkFbNLRe7hRx9eCIiBSvfL6tf2Jmb5vZd8xsTOARFajCmkWVquBokb/Wy13kTxUcEZHi1WKCk96P6kRgI3C3mb1uZt8KOrBCUyizqNy9voKjBuNWy8ygAlVwRESKWV7jLe7+T3e/HbiM1Jo43w4yqEJUKLOokjt24HV1gCo4bZGt4FgU6zkw3GBERCQw+Sz0N8rMZpnZG8AdpGZQDQ08sgKTTBbGSsa5M6hUwWm9TA+O9doHi+TTgiYiIl1RPn/D3ws8CEx197UBx1OwCmUvqgZr4KjJuNU8s8if+m9ERIpaPntRTe6MQApdskDWwYnnVnAGKsFprWwFR/03IiJFbY8Jjpm9DnhTTwHu7ocFFlUBSqabjMPuwUlsTlVwrLSUSG/to9QanozjOzcAquCIiBS75io40zstii6gYCo4G+uniFsBTFnvSnznRvAEoCniIiLFbo8JjruvyByb2SBgYvrhfHffEHRghSazVUPYPTiZjTbVYNx6WuRPRKT7yGcW1WeB+cBngM8CL5vZOUEHVmgyC/2FPYsq02SsKeKt13CRv31DjERERIKWzyyqm4CJmaqNmQ0E5gAPBRlYoSmECo7X1ZHcltrYvUQJTqs1WOSv98dCjERERIKWTzki0mhIanOe78PMTjezd8xsmZl9cw+vOdHMlpjZUjP7ez7XDUOyABKc+JYt2WPtIt562QpOeR+svFe4wYiISKDyqeD8xcyeIrUWDsC5wJMtvcnMosDPgU8Aq4EFZva4u7+Z85p+wC+A0919pZnt08r4O02iAGZRJTbmLvKnCk5rJXeklnFS9UZEpPjlsw7OdWb2aeBYUlPE73b3R/K49iRgmbsvBzCz2cAM4M2c13weeNjdV6Y/q2CblztzJeNkbS3bH36E+ObNDc4nNm7MHkdDbDKOr3iB2Ft/zs5I6ioS614DNINKRKQ7yGutend/2Mz+ARwHrMzz2kOAVTmPVwNHNXrNSKDUzOYCvYGfuftv8rx+p+rMHpxdc//O9sce3/MLIhFKQhqicneqn7wearaF8vkdIdJ3SNghiIhIwJpb6O8J4Jvu/oaZDQYWAwuBA8zsV+5+WwvXbioTaLxwYAlwJHAKUAnMM7OX3P3dRrFcClwKMGzYsBY+Nhj1s6iCT3DqPvgwe1wyaFDDJyMRep18IlZaGngcTarblU1urKIflHetxQatx96UHv65sMMQEZGANVfBGeHub6SPvwj81d2/YGa9gReA21q49mpgv5zHQ4HGe1mtBja5+y5gV7pKdDjQIMFx97uBuwEmTJjQ1OrKgcv24HRCghNbsxqA8lGHMOiWwtq43Wu2Zo/Lj/9XSsecFV4wIiIie9BcQ0ks5/gU0o3F7r4DSOZx7QXAQWY2wszKgM8BjcddHgOOM7MSM+tBagjrrXyD70ydtZKxuxNbvQaA0iGFN5TiOUNTVtEvvEBERESa0VwFZ5WZXUmqyjIe+AuAmVUCLY6PuHvczK4AngKiwD3uvtTMLks/f5e7v2VmfwFeI5U0/TqnalRQstPELdgm48RHH+HV1QCUDh0a6Ge1RW4Fh4q+ocUhIiLSnOYSnC8D/w6cCpzr7lvT5ycD9+ZzcXd/kkZTyt39rkaPfwj8MM94Q9NZFZxM9Qa6QgVHCY6IiBSm5vai2gBc1sT5Z4FngwyqEHXWLKrY6tXZ45JCTHCqcxKcyn7hBSIiItKMcDdW6kI6axZVfE2qgmM9ehDt3y/Qz2qL3CEqK+8TXiAiIiLNUIKTp86aRRVbk5poVjpkCBbiqsl7kh2iKuuFRUOaqi4iItICJTh5qt+LKrhfWWoGVWqIqnRo4Q1PAXj1VkD9NyIiUtha/LY2sx+YWR8zKzWzZ8xsk5ld0BnBFZJsD06AVZXk9u0kd+4ECjjBqc0s8qcER0REClc+5Yip7r4dmE5qyvhI4LpAoyowmeoNBNuD03AGVeFNEYf6JmM1GIuISCHLJ8HJNFp8EnjQ3bcEGE9ByvTfQLA9OLE1uQnOvoF9TntkmoxVwRERkUKWz2ab/8/M3gaqga+b2UCgJtiwCktmBhUEu5t4pv/GysuJhrSZZks8dx8qERGRAtXit7W7fxOYAkxw9xiwC5gRdGCFJJEzRNUZFZzSIUOwABOptvJkAmq2A6rgiIhIYWuxgmNmUeA4YLiZ5b7+J4FFVWA6rQcnM0W8QBuMqdtJZkN4JTgiIlLI8hqiIjUk9Tr5bbJZdHITnKBmUSV27iS5dStQmFs0QP0UcVCTsYiIFLZ8Epyh7n5Y4JEUsM6o4MRzGoxLCrSC02AVY1VwRESkgOXT6PF/ZjY18EgKWGfMoir0TTah8Uab/cILREREpAX5VHBeAh4xswgQAwxwd+82GxF1xiyqbIJTWkrJPvsE8hnt1WCIShUcEREpYPkkOD8mNYvqdfecb/pupDNmUcXWpLdo2HffgpxBBY0qOOrBERGRApbPN+l7wBvdNbmBRk3GAQ9RFerwFOQmOAblvUONRUREpDn5VHDWAXPN7P+A2sxJd+8208Qb9OAEMIsqWVVFYktqgeiCnSJOTpNxRR9SI5YiIiKFKZ8E54P0T1n6p9tpOIuq47/YY2vXZo8LO8HRKsYiItI1tJjguPutAGbW0913BR9S4Qm6B6crzKCC+iZjNRiLiEiha7EcYWZTzOxN4K3048PN7BeBR1ZAGs6iCiDByayBE41SMmhQh1+/o2QrOJVKcEREpLDlM95yG3AasBnA3V8Fjg8wpoLTWRWc0sEfw0ryGTUMh4aoRESkq8irocTdVzU6lQggloIV9ErGmSniJQU8PAX1TcYaohIRkUKXT7lglZkdDbiZlQFXkR6u6i6SAc6iStbWkti4CYDSoUM79NodyRMxqEu1YKmCIyIihS6fCs5lwOXAEGA1MC79uNtIBDiLKr52HaR7fAq6wbjBNg2q4IiISGHLZxbVJuD8ToilYAW50F9meAq6xhRxUJOxiIgUvj0mOGb238AeVy9296sCiagABTmLKjtF3IzSwYM79NodSRttiohIV9JcBWdhp0VR4ILcTTwzRbxk0CCstLRDr92RsqsYoyEqEREpfHtMcNz9/tzHZtYnddp3BB5Vgampq5801p4enPiWj/CamgbnYitTE9QKdXjK3fFtq0huXp49pwqOiIgUuhZ7cMxsAnAv0Dv10LYCX3L3RQHHVhB2Vtfxo9kvZh+3dRbV9j8/ydbf/m6Pzxdqg3H1I5eRWPFCg3Oq4IiISKHLZ5r4PcDX3f05ADM7llTCc1iQgRWK91ZvbvB4YP8ebbpO9SuvNPt8xaFj2nTdIHm8dvfkpv9wKOsZTkAiIiJ5yifB2ZFJbgDc/Xkz6zbDVIlEfYPx9ecdQ8+KNu43mkgNc5UMGULfs2Y2eKpkn4GUjxzZ1hADk9t3U3ro2USHTSa63yQsgB3VRUREOlJzs6jGpw/nm9kvgQdJzao6F5gbfGiFIZ7TYDx4715tvo7H0wnOXnvR89hj2h1XZ8idORXd/xhKR04NMRoREZH8NVfB+XGjx7fkHO9x+nixSSTqE5ySaDsW+UtXcGjPNTpZZvdwUN+NiIh0Lc3NojqpMwMpVB210aYnUwmORQt3M83GtLifiIh0VS2WE8ysr5n9xMwWpn9+bGbd5tsu3lEVnHimghNtZ0SdR9sziIhIV5XPN/Y9wA7gs+mf7aRmUXULuYv8tWcVY09kKjhddYiqX2hxiIiItFY+4yUHuvvZOY9vNbMlAcVTcHJnUbVro81sD04XHKKKlkFJRbjBiIiItEI+39jV6bVvADCzY4Dq4EIqLLkVnPYMUXXFCg7paeJW0U9Tw0VEpEvJp5xwGfCbnL6bj4CLggupsMRzKzjR9g9RdcUeHDUYi4hIV9NsgmNmUeACdz88vRcV7r69UyIrEA17cNo/RGUlXW+ISg3GIiLS1TT7bevuCTM7Mn3crRKbjI5uMqY9SVInyzQZq8FYRES6mnzKCa+Y2ePAn4BdmZPu/nBgURWQjm4y7orr4KiCIyIiXU0+37Z7AZuBk3POOdA9Epx0BSdi1r6F/rJDVF2jB8fds3tRqYIjIiJdTUs9OAOBnwPL3H1rp0RUYDJNxu1pMAbqp4l3lSGqWDUk46ljVXBERKSL2eO3rZldAiwF/ht428zO7LSoCkimgtOe4SlPJsFTiVJXaTLO3Ulcs6hERKSrae7b9hpgjLtvNLMDgAeAxzslqgKS6cFpT4NxtnoDXWaauFYxFhGRrqy5skSdu28EcPflQHnnhFRYMhWcjljkD7rOQn/ah0pERLqy5io4Q83s9j09dverggurcGQ222xPg3HDCk4XHKJSBUdERLqY5r5tr2v0eFGQgRSqjq/gdJEhKlVwRESkC9tjguPu93dmIIUqkcz04LQjwYl3wR4cJTgiItKFdY2GkBBlhqja1WSc7IIVnEyTcWklVlIWaiwiIiKtpQSnBZlZVO0aoop3wQQnu4pxv3ADERERaYPm1sH5r/Sfn+m8cApPdh2c9sx+SnbFIaqtAFhlv1DjEBERaYvmvrU/aWalwI2dFUwhqu/B6YCNNumCFZxy9d+IiEjX09wsqr8Am4CeZrYdMFJ7UBng7t6nE+ILXX0PTjsqOLlNxl1lL6pMgqNVjEVEpAva47e2u1/n7n2BP7t7H3fvnftnJ8YYqmR2mng7Kji5TcaRrpHgkG4y1gwqERHpilpcdc7dZ5jZIGBi+tTLmRWOu4PMZpuRjqrgdIEhKvckXrsdUJOxiIh0TS1+a6ebjOcDnwE+C8w3s3OCDqxQdPhCf11hiKp2B3jqvlXBERGRriiffQO+BUx09w0AZjYQmAM8FGRghaIjmowbbNXQBYaoGi7y1y+8QERERNoon7JEJJPcpG3O831FIdEBTcZdrYLTIMFRk7GIiHRB+Xxr/8XMnjKzi83sYuDPwJP5XNzMTjezd8xsmZl9s5nXTTSzRCEOfWVnUbWnyTgRzx53hWni2VWM0RCViIh0Tfk0GV9nZp8GjiU1Rfxud3+kpfeZWRT4OfAJYDWwwMwed/c3m3jdfwFPtSH+wHXEXlSkk6TUhbpAgqMhKhER6eLy6cHB3R8GHm7ltScBy9x9OYCZzQZmAG82et2VwP9SP0urYOyoquWfW3YC7ZsmXr2ofiP2Qq7gxN9/ltg7T5L8aEX9SVVwRESkC8orwWmjIcCqnMergaNyX2BmQ4CzgJMpwATnrwuXZ48rykrbdA1PJtn1939kH1tFRbvjCoLHa6h+8jqI19SfjJRgFd1mySMRESkiQSY4TZU8vNHj24Ab3D1htucKiZldClwKMGzYsI6Kr0Wbt1Vnj0+bdGCbrpHYujV7bD16UDJgQHvDCkRy4zvZ5MZ6fwwr60XpoWdjkSD/FREREQlGXt9eZlYJDHP3d1px7dXAfjmPhwJrG71mAjA7ndwMILX/VdzdH819kbvfDdwNMGHChMZJUmBq6mIA7N2nkuEf69ema8Q31E9AG3Dl5R0RViAS6+tHDnuccy+Rfvs182oREZHCls9Cf58ClpDamwozG2dmj+dx7QXAQWY2wszKgM8BDd7n7iPcfbi7Dye1rs7XGyc3YaqpS03vLi9rexUjsaF+0eeSffZpd0xBSWxYmjoo74P1HRpuMCIiIu2Uz9SgWaQahrcCuPsSYHhLb3L3OHAFqdlRbwF/dPelZnaZmV3WtnA7V01danp3ZTsSnPjG+gQnWqDDUwDJdAUnus9omhsuFBER6Qry+eaOu/u2tnzpufuTNFozx93v2sNrL271BwQsk+BUdECCE+nbl0h5eYfE1dE8XkNy8/sARAeNDjkaERGR9svnm/sNM/s8EDWzg4CrgBeDDaswdGSCUzJwYIfEFITkxnfAU8NxkX2U4IiISNeXzxDVlcAYoBb4PbANuCbAmApGhyQ46R6ckn0KN8HJbTBWBUdERIpBs9/c6VWGH3f3U4GbOiekwpFJcNraZOyJBInNm4HCruA0bDDW7CkREen6mq3guHsCqDKzbrmcbXubjBObt0AytU1DISc4ajAWEZFik883dw3wupn9FdiVOenuVwUWVYHIDlGVty3BaTCDqkCniKvBWEREilE+39x/Tv90K7F4IruTeEVpWxOc+kX+SgYW5hTx5MZ31WAsIiJFJ5/dxO/vjEAKTW0skT1uewVnU+rArGC3aEisX5o9VgVHRESKRYvf3Omp4d8DRgPZnSLd/YAA4wpdZngK2j6LKrNNQ7R/f6y0bZt1Bk0NxiIiUozymSZ+L3AnEAdOAn4D/DbIoApBTW37E5zMNg1do8F4lBqMRUSkaOST4FS6+zOAufsKd58FnBxsWOGr7ogKTrrJOFqga+DkNhir/0ZERIpJXrOozCwCvGdmVwBrgMKcEtSBatuZ4HgsRuKjj4DCreDkNhhHB40JORoREZGOk08F5xqgB6ktGo4ELgQuCjCmgtDeHpz45s3gDhTuKsaJDVrBWEREilM+s6gWpA93Al8MNpzC0d4hqswWDVC4FZzE+jdSB2owFhGRIpPPLKqRwHXA/rmvd/ei7sNp7xBVYmNuglOYI3pqMBYRkWKVzzf3n4C7gF8BiRZeWzQaVHDasA5OZoo4kQjRvffqqLA6jBqMRUSkmOXzzR139zsDj6TANKjgtGEl48wif9G998ai0Q6Lq6M0bDBWgiMiIsVlj9/cZpYpO/w/M/s68AhQm3ne3bcEHFuoMhWcSMQoLcmnF7uhzDYNBdt/06DBWDOoRESkuDRXmlgEOJBpzrgu5zkHinsl4/RCfxWlJW3qT8k0GRfqDKpkZouG8t5qMBYRkaKzxwTH3Ud0ZiCFpibW9p3Ek3V1JLdtAwq4gpNOcKL7jFaDsYiIFJ09jr2Y2UQz+1jO4y+Y2WNmdnvO8FXRylZw2j2DqvASHDUYi4hIsWuuueSXQB2AmR0PfJ/UPlTbgLuDDy1cmYX+2rsGTiFu06AGYxERKXbNfXtHcxqJzwXudvf/Bf7XzJYEHlnI2pXgFHgFRw3GIiJS7Jqr4ETNLPPtfgrwt5zn2rb7ZBfSIQlONEq0f/+ODKtDqMFYRESKXXPf3g8CfzezTUA18ByAmX2c1DBVUWvfEFVmivgALNL6KeZBy1RwtIKxiIgUq+ZmUX3XzJ4BBgNPu6d3jkxVfa7sjODC1J4EJ5Fe5K8Qt2hQg7GIiHQHzX57u/tLTZx7N7hwCkf7hqhSFZzowAEdGlNHSG58F5Kpe1ODsYiIFKvCGz8pEG1NcJLV1SR37ASgZJ/Cq+A0aDDeRw3GIiJSnIq+Wbgt1n+0k3giCUBFWcv7SMU3biRZk9rFIr5+ffZ8mKsYJ7etxmM1u51PrFqQOijvjfVTg7GIiBQnJTiNzFm4nP9+eH72cUVZabOv3/bIo2z7wx+bfC6sKeK1z99G3YJfN/uaVIOxCngiIlKc9A3XyOJ31zV4fOC+zU/zrlnyapPnIz17Ujp0aIfF1Rrx959t8TUlw4/rhEhERETCoQpOI/FkMnv80ytO44AWEhxPpFYELt1/f/qeNSN7vuzjBxGprAwmyBZ49UcARIdNoXTsObs9bxX9iA6d0NlhiYiIdBolOI0kk6nZ8Afs27/F5Cb1hvSWB3vtRY/Jk4MMLS/uSbxmKwDRjx1K6cjTwg1IREQkBBqiaiTTXByN5LcAnsdTCY5FC+NX6TXbwFP3YJVFvyeqiIhIkwrjW7mAJNIVnJJ8E5Z0BYdoy7OtOoNXbckeW2XhbRMhIiLSGZTgNJJocwWnQBKcdP8NgPVQgiMiIt2TEpxGMk3G0TwrOJkm44Kp4OQmOBqiEhGRbkoJTiOZJuNovptkJjIVnMLo126Y4KiCIyIi3ZMSnEZa3WScKLAmY1VwRERElOA0lki0ssm40IaoMk3GZT2xkrJwgxEREQmJEpxGEukenEirKziFNUSl4SkREenOlOA0kklwWl/BKYxfpVenKjganhIRke6sML6VC0g80bom4/oKToEMUamCIyIiogSnsUwFJ58mY3evr+CUFMgQVVU6wdEaOCIi0o0pwWmkVU3GORtzWr7TygPk7tkhqoiGqEREpBsL/1u5wNRXcPL41WSqN1AYFZy6nZCMAxqiEhGR7k0JTiOZvaii0TyGqHISnIKo4GgNHBEREUAJzm7qF/pr+VeTm+AUwjo4DTfa7BdeICIiIiFTgtNIMrubeB7r4ORWcApgiKrhRpuq4IiISPelBCdHMukkPf9p4oU2RJWszq3gqAdHRES6r/C/lQtIImdWVD49OMRzm4wLYYhKPTgiIiKgBKeBTIMx5FnBSeZUcApgqwavSSc40XIorQw3GBERkRApwcmRaTCGNkwTL4CtGnIX+TPLby8tERGRYhT+t3IBSeQkOPk0GXu8wCo42W0aNDwlIiLdmxKcHLlDVJF8KjjJAqvgZDfaVIOxiIh0b+F/KxeQ3Cbj1ldwCqDJWBttioiIAEpwGmhtk3HDCk4BDFFle3A0RCUiIt2bEpwcDZuMW1vBCfdX6bFqiFenYlEFR0REujklODkaNhl3ra0aGu5DpQRHRES6NyU4ORoMUeVTkUkUziyqBvtQaYhKRES6OSU4ORqsZJzPEFWigIaocio4EVVwRESkmwv0W9nMTjezd8xsmZl9s4nnzzez19I/L5rZ4UHG05J4opVNxonCaTL2BvtQqYIjIiLdW2AJjplFgZ8DZwCjgfPMbHSjl30AnODuhwHfAe4OKp58JIukgmOV/cILREREpAAEWXaYBCxz9+UAZjYbmAG8mXmBu7+Y8/qXgKEBxtOiJcv+mT1uqsm4bsVKdvz5zyRragConr+g/skAmoyTOzdS9/JdeNXmll+75f3UQaQEyvt0eCwiIiJdSZAJzhBgVc7j1cBRzbz+y8D/BRhPix57/p3scWV56W7Pb/3DH6hZ/EqT741UVHR4PLFXHyT22h9a9R7rOUD7UImISLcXZILT1LesN3EOMzuJVIJz7B6evxS4FGDYsGEdFd9uyktLqI2lhp1GDO632/PJHTuzx6X77Zc9rpxwJNH+Hd/Ym50ZZVEiex3Q8huipZQdeXGHxyEiItLVBJngrAb2y3k8FFjb+EVmdhjwa+AMd29yLMbd7ybdnzNhwoQmk6SO4J669LTJBzVdBUk/X3H4Yexz42490x0fT6IOAOs9iJ5feCTwzxMRESkWQXbGLgAOMrMRZlYGfA54PPcFZjYMeBi40N3fDTCWvCTTCcweh3g83YTcWUNA6QSHaFnnfJ6IiEiRCKyC4+5xM7sCeAqIAve4+1Izuyz9/F3At4G9gV+kk4q4u08IKqaWZBOcPaR9nplltacXdLRMBSe6ez+QiIiI7Fmgi7e4+5PAk43O3ZVzfAlwSZAxtEY6vyG6xwpOJgHqnATHE7F0QKrgiIiItIZWMs7R4hBVZiuHPNbI6RDZCo4SHBERkdZQgpMjmWwpwVEPjoiISFegBCdHZhZVZA8VGu/0ISolOCIiIm2hBCeHtzQCFdIsKjUZi4iItI4SnDR3z6MHp3NnUanJWEREpG2U4KR5zvKBkRabjDt3mjiq4IiIiLSKEpw0z8lw9pTg1PfgdNIQVVyzqERERNpCCU5aMifB2eMIVCfPolKTsYiISNsowUlLNqjg7Gkp484eolIPjoiISFsowUnL7cHZY4FGs6hERES6BCU4aZlF/qCZHpz0EFVnrIPjyQR4IvVAFRwREZFWUYKT1mCIak9NxJ05iyrTfwNKcERERFpJCU5a7iyqPa6Dk3lNZwxRZfpvACtRgiMiItIaSnDSGg5RNf0a78RZVK4KjoiISJspwUnLa6G/ztyLKqeCo4X+REREWkcJTloynyGqztyqIaeCo4X+REREWkcJTlp+TcbpBKcTVjLWEJWIiEjbKcFJa9hk3Pxr9ljh6Uiq4IiIiLSZEpy0ZB57UXXqSsaq4IiIiLSZEpy03FlULffgdMYQlZqMRURE2koJTlpLs6jcPbQKjoaoREREWkcJTlqLTcYNdhvvhK0aNEQlIiLSZkpw0lpcyTiv3Tg7UFwJjoiISFspwUlrcSXjTP8NdNJWDTlDVNqqQUREpFWU4KTl1YOT1vlDVGoyFhERaQ0lOGkNVjJuqoSTW8HphIX+Gm7VoAqOiIhIayjBSWtxq4YGQ1SaRSUiIlLIlOCkbd5WlT2ONkpwkjU11H3wYf2JThiiSu74Z/0DJTgiIiKtUhJ2AIXiiXnvZY9z85vEjh2sveZf8F1VOc8HO0QVX7OY2Cu/rT+hHhwREZFWUQUn7eND+gOpNXCGDeqbPV+3fHmD5AagdP/9A40l/v6z2ePIwIOxzhgSExERKSKq4KSddMQIDh42gBEf68fAfj3rn4gnsod9zppJjymTKRs2LNBYklvezx73+Mz9gX6WiIhIMVJpIG3YoL5MHj2UQXv1anDek/UJTo+jJgWe3AAkNy8HoOSg07DyXi28WkREmrNw4UKuuuqqDrve3LlzmT59OgCzZs3iRz/6UZOvu+666xgzZgzXXXddqz+jVy/93d9equC0JFGf4BAN/tflsSp8+xoAInsfEPjniYgUuwkTJjBhwoRO/9xf/vKXbNy4kfLy8k7/bFGC0yLPGaKyaDTwz0tu+RBITVmP7HVg4J8nItJev35iMR+s+6hDrzlicH8umT6+2dfs2rWLz372s6xevZpEIsHNN9/MAQccwNVXX82uXbsoLy/nmWeeYdGiRfzoRz/iiSeeYNasWbz//vusWbOGVatWcf311/OVr3yFCy+8kHPOOYcZM2YAcP7553Puuedy5plnthjrq6++ysknn9zgemeeeSa7du3iqKOO4sYbb2TChAmcf/75JBIJzjjjDH7yk5+wc+dO1q1bx7nnnsv27duJx+PceeedHHfccQD867/+K88++yz9+/dn9uzZDBw4sP2/2G5ECU4LcoeoLNoJ08Nz+m9UwRER2bO//OUv7Lvvvvz5z38GYNu2bRxxxBH84Q9/YOLEiWzfvp3Kysrd3vfaa6/x0ksvsWvXLo444gimTZvGJZdcwk9/+lNmzJjBtm3bePHFF7n//vx6IJu63uOPP06vXr1YsmQJANOnT+fqq6/mvPPO46677sq+9/e//z2nnXYaN910E4lEgqqq1KSWXbt2MX78eH784x/z7//+79x6663ccccd7fyNdS9KcFoS79whquSWVP8NFiHSb3jgnyci0l4tVVqCMnbsWK699lpuuOEGpk+fTr9+/Rg8eDATJ04EoE+fPk2+b8aMGVRWVlJZWclJJ53E/PnzmTlzJpdffjkbNmzg4Ycf5uyzz6akJL+/8/d0vVzz5s3j0UcfBeDzn/881157LQATJ07kS1/6ErFYjJkzZzJu3DgAIpEI5557LgAXXHABn/70p1v52xE1GbfAc3pwrKQThqg2pyo41m+YNtkUEWnGyJEjWbRoEWPHjuXGG2/kkUceyWudssavyTy+8MILeeCBB7j33nv54he/mHcce7pePo4//nj+8Y9/MGTIEC688EJ+85vf5PUZ0jIlOC3JbTLuhBWME+kKTnQvDU+JiDRn7dq19OjRgwsuuIBrr72Wl156ibVr17JgwQIAduzYQTwe3+19jz32GDU1NWzevJm5c+dmKz4XX3wxt912GwBjxozJO449XS/X5MmT+d///V8AZs+enT2/YsUK9tlnH77yla/w5S9/mcWLFwOQTCZ56KGHgNQw1rHHHpt3PJKiIaoWNKzgBPvr8ngdvnUlAJG91WAsItKc119/neuuu45IJEJpaSl33nkn7s6VV15JdXU1lZWVzJkzZ7f3TZo0iWnTprFy5Upuvvlm9t13XwAGDRrEqFGjdhteasmerpfrtttu44ILLuDHP/4x06ZNo2/f1IKyc+fO5Yc//CGlpaX06tUrW8Hp2bMnS5cu5cgjj6Rv37784Q9/aOVvR8xzNpnsCiZMmOALFy7stM/b/vj/Y+vvHwRg6H33EKmoCOyzEpvepeq3qXHWitO/T+mo6YF9lohIdzRr1ix69eqV7YHJVVVVxdixY1m8eHE2AekoVVVVVFZWYmbMnj2bBx98kMcee6xDP6Mba3L8ThWcFnRmBSfTfwMQ0RCViEinmTNnDl/60pf4xje+0eHJDcCiRYu44oorcHf69evHPffc0+GfIQ0pwWlJJ/bg1Cc4RmSvEYF+lohIdzRr1qwmz5966qmsXLmywbmnnnqKG264ocG5ESNG8Mgjj7T6c4877jheffXVVr9P2k4JTguyFRwzLOgEZ8sHqY/qMwQr3X3tBhER6TynnXYap512WthhSBtpFlVLMglOwMNTUL/Inxb4ExERaR8lOC3IVHCC3qbBEzGSH30IaIsGERGR9lKC04LsEFXA2zQkt62CZGq9hqimiIuIiLSLEpyWZCs4mkElIiLSVSjBaUFnDVFl96BCCY6ISEdauHAhV111VagxnHjiiTS1htt9993HFVdcEUJELevVq1fYIbSLZlG1JLPZZtAJTmYPql6DsPKu/S+ViHQvNXO/T3Lj2x16zcjAQ6g48Zsdcq0JEyYwYcKEDrlW2OLxeN6bgHZ3quC0wJOdVcHJzKBS/42ISD527drFtGnTOPzwwzn00EP5wx/+wIIFCzj66KM5/PDDmTRpEjt27GDu3LlMn55aGX7WrFlceOGFnHzyyRx00EH86le/AlIbbeauLHz++efz+OOPN/m59913H5/+9Kc5/fTTOeigg7j++uuzzz399NNMmTKF8ePH85nPfIadO3fu9v57772XkSNHcsIJJ/DCCy9kz2/cuJGzzz6biRMnMnHixOxzs2bN4tJLL2Xq1Kl84QtfYNasWVx00UVMnTqV4cOH8/DDD3P99dczduxYTj/9dGKxGJBaXPCEE07gyCOP5LTTTmPdunUA/OpXv2LixIkcfvjhnH322VRVVQHwwQcfMGXKFCZOnMjNN9/c5n8uBcPdu9TPkUce6Z1p409/5ivOPc/X/Mu/BvYZyUTct/9svG//yRivfvb7gX2OiEgxeeihh/ySSy7JPt66dauPGDHC58+f7+7u27Zt81gs5s8++6xPmzbN3d1vueUWP+yww7yqqso3btzoQ4cO9TVr1vjcuXN9xowZ2esMHz7cY7FYk5977733+ogRI3zr1q1eXV3tw4YN85UrV/rGjRv9uOOO8507d7q7+/e//32/9dZb3d39hBNO8AULFvjatWt9v/328w0bNnhtba0fffTRfvnll7u7+3nnnefPPfecu7uvWLHCDznkkGzM48eP96qqquzjY445xuvq6nzJkiVeWVnpTz75pLu7z5w50x955BGvq6vzKVOm+IYNG9zdffbs2f7FL37R3d03bdqUvZebbrrJb7/9dnd3/9SnPuX333+/u7vfcccd3rNnz7b9g+l8TeYLqnO1oDN6cHz7GkjUAqrgiIjka+zYsVx77bXccMMNTJ8+nX79+jF48ODsbt59+vRp8n0zZsygsrKSyspKTjrpJObPn8/MmTO5/PLL2bBhAw8//DBnn312s0NBp5xySnZLh9GjR7NixQq2bt3Km2++yTHHHANAXV0dU6ZMafC+l19+mRNPPJGBAwcCcO655/Luu+8Cqe0i3nzzzexrt2/fzo4dOwA488wzqaysXwD2jDPOoLS0lLFjx5JIJDj99NOzv5MPP/yQd955hzfeeINPfOITACQSCQYPHgzAG2+8wbe+9S22bt3Kzp07s4sZvvDCC9kdzy+88MLdVnHuapTgtCQZfA9OcrMajEVEWmvkyJEsWrSIJ598khtvvJGpU6di1uS+iw00fk3m8YUXXsgDDzzA7NmzW9wrqry8PHscjUaJx+O4O5/4xCd48MEHW/X5Gclkknnz5jVIZDJ69uzZ5OdndlLPXDMSiWRjGTNmDPPmzdvtWhdffDGPPvoohx9+OPfddx9z585tMbauSD04LfB4poIT3K8qsaV+inhUCY6ISF7Wrl1Ljx49uOCCC7j22mt56aWXWLt2LQsWLABgx44dxOPx3d732GOPUVNTw+bNm5k7d2624nPxxRdz2223ATBmzJhWxzN58mReeOEFli1bBqR2EM9UZzKOOuoo5s6dy+bNm4nFYvzpT3/KPjd16lTuuOOO7OMlS5a0OoaMgw8+mI0bN2YTnFgsxtKlS4HU72Xw4MHEYjEeeOCB7HuOOeYYZs+eDdDgfFelCk4LPJH+jyPAdXCyM6h67I1V9gvsc0REisnrr7/Oddddl61i3Hnnnbg7V155JdXV1VRWVjJnzpzd3jdp0iSmTZvGypUrufnmm9l3330BGDRoEKNGjWLmzJltimfgwIHcd999nHfeedTWptoO/uM//oORI0dmXzN48GBmzZrFlClTGDx4MOPHjyeRboW4/fbbufzyyznssMOIx+Mcf/zx3HXXXW2KpaysjIceeoirrrqKbdu2EY/HueaaaxgzZgzf+c53OOqoo9h///0ZO3ZsdhjsZz/7GZ///Of52c9+xtlnn92mzy0k5u5hx9AqEyZM8KbWEgjK+lu/Q+1bb1E+ahSDbgmmq3zX788luX4p0f0m0eOc5suiIiLSdrNmzaJXr15ce+21uz1XVVXF2LFjWbx4cba/RrqEJsfVNETVgqC3anBPZhf50x5UIiLhmDNnDocccghXXnmlkpsioSGqlqSHqILaqsF3rIdYNaAZVCIiQZs1a1aT50899VRWrlzZ4NxTTz2120yiESNG8MgjjwQVnnQgJTgt8EQSACsJZhZVcov2oBIRKUSnnXZadgq1dD0aompJZogqElCCo002RUREOlygCY6ZnW5m75jZMjPbbVMRS7k9/fxrZjY+yHjawrNDVAFXcCr6Yj32DuQzREREupvAEhwziwI/B84ARgPnmdnoRi87Azgo/XMpcGdQ8bRZeoiKgIaoEukKTnSvA4tqgSUREZEwBdmDMwlY5u7LAcxsNjADeDPnNTOA33hqrvpLZtbPzAa7+7oA42rStt/fRvWiV3Y7H1+f2rQssWYBVY91/Jb2yQ2pHXjVYCwiItJxgkxwhgCrch6vBo7K4zVDgE5PcGKrV1K3JrbnF1StJbF8bWCfH9n744FdW0SkO1u4cCG/+c1vuP322zvkekuWLGHt2rV88pOf7JDrtcV//ud/8m//9m+tes99993HwoULG6yWHLS2xNlRgkxwmhpvabyqYD6vwcwuJTWExbBhw9ofWROi/fci2qfpvCpS7lSO6kukdzBrI0T6DKF09JmBXFtEJGgf3f8b6j5c0aHXLBu+P/0v+kKHXGvChAlMmDChQ64FqQRn4cKFTSY48Xi82U06O0qYiUNrFGuCsxrYL+fxUKBxCSSf1+DudwN3Q2ol444NM6X/V75F/68EcWUREQnCrl27+OxnP8vq1atJJBLcfPPNHHDAAVx99dXs2rWL8vJynnnmGRYtWsSPfvQjnnjiCWbNmsX777/PmjVrWLVqFddffz1f+cpXuPDCCznnnHOYMWMGAOeffz7nnnsuZ57Z8H8+6+rq+Pa3v011dTXPP/88N954I2+99RZr167lww8/ZMCAAUydOrVBpWT69Olce+21nHjiifTq1Yurr76aJ554gsrKSh577DEGDRrE+vXrueyyy1i+PLXw65133snRRx/NzJkzWbVqFTU1NVx99dVceumlfPOb36S6uppx48YxZswYHnjgAX73u99x++23U1dXx1FHHcUvfvELotEo9957L9/73vcYPHgwI0eObLBJaGMXX3wxffr0YeHChfzzn//kBz/4Aeeccw4AP/zhD/njH/9IbW0tZ511Frfeeis/+MEPqKio4KqrruJf/uVfePXVV/nb3/7GM888w7333svQoUN3i/MnP/lJdiPTSy65hGuuuYYPP/yQM844g2OPPZYXX3yRIUOG8NhjjzW56WiruHsgP6SSp+XACKAMeBUY0+g104D/I1XJmQzMb+m6Rx55pIuIiDz00EN+ySWXZB9v3brVR4wY4fPnz3d3923btnksFvNnn33Wp02b5u7ut9xyix922GFeVVXlGzdu9KFDh/qaNWt87ty5PmPGjOx1hg8f7rFYrMnPvffee/3yyy/PPr7lllt8/PjxXlVV1eTz06ZN82effdbd3QF//PHH3d39uuuu8+985zvu7v7Zz37Wf/rTn7q7ezwe961bt7q7++bNm93dvaqqyseMGeObNm1yd/eePXtmr//mm2/69OnTva6uzt3dv/a1r/n999/va9eu9f322883bNjgtbW1fvTRRzeIq7GLLrrIzznnHE8kEr506VI/8MAD3d39qaee8q985SueTCY9kUj4tGnT/O9//7vPmzfPzznnHHd3P/bYY33ixIleV1fns2bN8rvuumu3OBcuXOiHHnqo79y503fs2OGjR4/2xYsX+wcffODRaNRfeeUVd3f/zGc+47/97W/3GGcTmswXAptF5e5x4ArgKeAt4I/uvtTMLjOzy9IvezKdBC0DfgV8Pah4RESkuIwdO5Y5c+Zwww038Nxzz7Fy5UoGDx6c3R28T58+TQ4XzZgxg8rKSgYMGMBJJ53E/PnzOeGEE1i2bBkbNmzgwQcf5Oyzz27VUNOZZ56ZV8WhrKyM6dOnA3DkkUfy4YcfAvC3v/2Nr33tawBEo9HsdhG33347hx9+OJMnT2bVqlW89957u10zU6WaOHEi48aN45lnnmH58uW8/PLLnHjiiQwcOJCysjLOPffcFuObOXMmkUiE0aNHs379egCefvppnn76aY444gjGjx/P22+/zXvvvceRRx7JokWL2LFjB+Xl5UyZMoWFCxfy3HPPcdxxx+127eeff56zzjqLnj170qtXLz796U/z3HPPAakVoseNG7fb76U9Ah0odPcnSSUxuefuyjl24PIgYxARkeI0cuRIFi1axJNPPsmNN97I1KlT81puo/FrMo8vvPBCHnjgAWbPnp0dRslXz549s8clJSUkk8ns45qamuxxaWlp9vOi0SjxeHyP15w7dy5z5sxh3rx59OjRgxNPPLHBtTLcnYsuuojvfe97Dc4/+uijrV5+JHcIy9Obcbs7N954I1/96ld3e/3w4cO59957OfrooznssMN49tlnef/99xk1alSTcebzudFolOrq6lbF3RStZCwiIl3S2rVr6dGjBxdccAHXXnstL730EmvXrmXBggUA7Nixo8kE4rHHHqOmpobNmzczd+7cbMXn4osv5rbbbgNgzJgxe/zc3r17s2PHjj0+P3z4cJYsWUIymWTVqlXMnz+/xXs55ZRTuPPO1FJwiUSC7du3s23bNvr370+PHj14++23eemll7KvLy0tJRaLZd/70EMPsWHDBgC2bNnCihUrOOqoo5g7dy6bN28mFovxpz/9qcU4mnLaaadxzz33sHPnTgDWrFmT/azjjz+eH/3oRxx//PEcd9xx3HXXXYwbNy6bWOXGefzxx/Poo49SVVXFrl27eOSRR5qs9HQU7UUlIiJd0uuvv851111HJBKhtLSUO++8E3fnyiuvpLq6msrKSubMmbPb+yZNmsS0adNYuXIlN998M/vuuy8AgwYNYtSoUcycObPZzz3ppJP4/ve/z7hx47jxxht3e/6YY45hxIgRjB07lkMPPZTx41tepP9nP/sZl156Kf/zP/9DNBrlzjvv5PTTT+euu+7isMMO4+CDD2by5MnZ11966aUcdthhjB8/ngceeID/+I//YOrUqSSTSUpLS/n5z3/O5MmTmTVrFlOmTGHw4MGMHz+eRGb7oVaYOnUqb731FlOmTAGgV69e/O53v2OfffbhuOOO47vf/S5TpkyhZ8+eVFRUNEhaGsd58cUXM2nSJCDVZHzEEUd0yHBUU6y5klEhmjBhgi9cuDDsMEREpAuaNWsWvXr14tprr93tuaqqKsaOHcvixYuzPTDSJTQ5DqchKhER6fbmzJnDIYccwpVXXqnkpkiogiMiItKEp556ihtuuKHBuREjRvDII4+EFFHH+O53v7tbP85nPvMZbrrpppAiarcmKzhKcERERKQr0xCViIiIdA9KcERERKToKMERERGRoqMER0RERIqOEhwREREpOkpwREREpOh0uWniZrYRWBHApQcAmwK4blege+9+uut9g+69O957d71v6B73vsndT298ssslOEExs4XuPiHsOMKge+9+995d7xt0793x3rvrfUP3vncNUYmIiEjRUYIjIiIiRUcJTr27ww4gRLr37qe73jfo3ruj7nrf0I3vXT04IiIiUnRUwREREZGiowRHREREio4SHMDMTjezd8xsmZl9M+x4OpqZ3WNmG8zsjZxze5nZX83svfSf/XOeuzH9u3jHzE4LJ+r2M7P9zOxZM3vLzJaa2dXp893h3ivMbL6ZvZq+91vT54v+3gHMLGpmr5jZE+nH3eW+PzSz181siZktTJ/rLvfez8weMrO30//NTyn2ezezg9P/rDM/283smmK/77y5e7f+AaLA+8ABQBnwKjA67Lg6+B6PB8YDb+Sc+wHwzfTxN4H/Sh+PTv8OyoER6d9NNOx7aON9DwbGp497A++m76873LsBvdLHpcDLwOTucO/p+/kG8HvgifTj7nLfHwIDGp3rLvd+P3BJ+rgM6Ndd7j19T1Hgn8D+3em+m/tRBQcmAcvcfbm71wGzgRkhx9Sh3P0fwJZGp2eQ+guB9J8zc87Pdvdad/8AWEbqd9TluPs6d1+cPt4BvAUMoXvcu7v7zvTD0vSP0w3u3cyGAtOAX+ecLvr7bkbR37uZ9SH1P3L/A+Dude6+lW5w7zlOAd539xV0r/veIyU4qS+8VTmPV6fPFbtB7r4OUokAsE/6fFH+PsxsOHAEqUpGt7j39DDNEmAD8Fd37y73fhtwPZDMOdcd7htSSezTZrbIzC5Nn+sO934AsBG4Nz00+Wsz60n3uPeMzwEPpo+7033vkRKcVCm/se48d77ofh9m1gv4X+Aad9/e3EubONdl793dE+4+DhgKTDKzQ5t5eVHcu5lNBza4+6J839LEuS533zmOcffxwBnA5WZ2fDOvLaZ7LyE1DH+nux8B7CI1NLMnxXTvmFkZcCbwp5Ze2sS5LnvfLVGCk8pg98t5PBRYG1IsnWm9mQ0GSP+5IX2+qH4fZlZKKrl5wN0fTp/uFveekS7VzwVOp/jv/RjgTDP7kNRw88lm9juK/74BcPe16T83AI+QGn7oDve+GlidrlICPEQq4ekO9w6phHaxu69PP+4u990sJTiwADjIzEaks+DPAY+HHFNneBy4KH18EfBYzvnPmVm5mY0ADgLmhxBfu5mZkRqTf8vdf5LzVHe494Fm1i99XAmcCrxNkd+7u9/o7kPdfTip/5b/5u4XUOT3DWBmPc2sd+YYmAq8QTe4d3f/J7DKzA5OnzoFeJNucO9p51E/PAXd576bF3aXcyH8AJ8kNcPmfeCmsOMJ4P4eBNYBMVIZ/JeBvYFngPfSf+6V8/qb0r+Ld4Azwo6/Hfd9LKny62vAkvTPJ7vJvR8GvJK+9zeAb6fPF/2959zPidTPoir6+ybVh/Jq+mdp5u+y7nDv6XsZByxM/zv/KNC/O9w70APYDPTNOVf0953Pj7ZqEBERkaKjISoREREpOkpwREREpOgowREREZGiowRHREREio4SHBERESk6SnBEQmRmbmY/znl8rZnN6qBr32dm53TEtVr4nM+kd29+ttH54WZW3Wi347IO/NyLzWzfnMe/NrPRHXTdjY3ibvV10/f/RjviuMbMeuQ8fjKztpGItEwJjki4aoFPm9mAsAPJZWbRVrz8y8DX3f2kJp57393H5fzUdVCIABcD2QTH3S9x9zc76Np/aBR3R123Na4htcYJAO7+SU+tSi0ieVCCIxKuOHA38C+Nn2hcgTGznek/TzSzv5vZH83sXTP7vpmdb2bzzex1Mzsw5zKnmtlz6ddNT78/amY/NLMFZvaamX0157rPmtnvgdebiOe89PXfMLP/Sp/7NqkFFe8ysx/mc8OZ+0gfn2Nm9+Xc7+1m9qKZLW9079enP/vV9P2eA0wAHkhXWCrNbK6ZTdhTrJnPNrPvpq/zkpkNyifm9Hv/YGafzHl8n5mdna7UPGdmi9M/Rzfx3ovN7I6cx0+Y2Ynp4zvNbKGZLTWzW9PnriKVvD2bqYyZ2YeZRNjMvpG+tzfM7Jr0ueHpStqv0td62lKrWGNmV5nZm+l/3rPzvWeRLi3slQb1o5/u/APsBPoAHwJ9gWuBWenn7gPOyX1t+s8Tga3AYKAcWAPcmn7uauC2nPf/hdT/yBxEahXrCuBS4Fvp15STWv11RPq6u4ARTcS5L7ASGEhqY8O/ATPTz80FJjTxnuFANfWrSP889z7Sx+cA9+XE+6d0vKOBZenzZwAvAj3Sj/dq6nMzj1uI1YFPpY9/kPk9NIr7YlI7Uy/J+akEzgLuT7+mjNSuzJWkqiwV6fMHAQtz7v+NnGvekfMZTwAnNrqfaPoeDks//hAYkPOeD4EBwJGkEtCeQC9SqxYfkf68ODAu/fo/Ahekj9cC5enjfmH/e68f/XTGjyo4IiHz1A7nvwGuasXbFrj7OnevJbXs+tPp86+T+qLL+KO7J939PWA5cAipPYq+YGZLgJdJLet+UPr18939gyY+byIw1903unsceABobqfqjNwhqsvzeP2j6XjfBDLVlVOBe929CsDdt7RwjeZirSOVXAAsouHvKlfjIapq4P9Ibd5ZTirp+kf6fCnwKzN7nVSC1tp+nc+a2WJSW2uMyeP9xwKPuPsud98JPAwcl37uA3df0sT9vUaq2nUBqSRIpOgpwREpDLeR6mXpmXMuTvq/UTMzUlWDjNqc42TO4ySpqkVG471YHDDgypwv7xHunkmQdu0hPsvzPvKRG1NFo+dy78ty/mzNnjLNxRpz98y1EjT8XTXL3WtIVVhOA84ltVs5pIYX1wOHk6ogNdVInf1nmVYBYKkND68FTnH3w4A/s/vvpLHm7i/395d7f9OAn5Oq/iwys7zvW6SrUoIjUgDSVYk/kkpyMj4k9YUEMINUpaC1PmNmkXRfzgGkNth7CviamZUCmNlIS+0+3ZyXgRPMbEC6Afk84O9tiAdgvZmNMrMIqWGfljwNfCkzo8jM9kqf3wH0DjjWxmYDXyRVMXkqfa4vsM7dk8CFpIaaGvsQGJf+Z7EfMCl9vg+ppHJbuh/ojJz37On+/gHMNLMe6X9uZwHP7Sng9O95P3d/Frge6EdqaEukqCmLFykcPwauyHn8K+AxM5tPakfgPVVXmvMOqS/3QcBl7l5jZr8mNXSxOF0Z2gjMbO4i7r7OzG4EniVVQXjS3R9rQzwA3yQ1TLSK1E7nzX7ZuvtfzGwcsNDM6oAngX8j1bNzl5lVA1M6ONZzzezYnMdfd/cXSSVbvwEe9/oZYb8A/tfMPpP+zKb+Ob0AfEBqCPENYHE61lfN7BVSfTTL06/LuBv4PzNb5zkz1Nx9cboxe3761K/d/RUzG76He4kCvzOzvqR+Hz91zcaSbkC7iYuIiEjR0RCViIiIFB0lOCIiIlJ0lOCIiIhI0VGCIyIiIkVHCY6IiIgUHSU4IiIiUnSU4IiIiEjR+f8d8UkYJaOX/wAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "from estimagic import profile_plot\n", + "\n", + "fig, ax = profile_plot(\n", + " problems=problems,\n", + " results=results,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "4ada4410", + "metadata": {}, + "source": [ + "The x axis shows runtime per problem. The y axis shows the share of problems each algorithm solved within that runtime. Thus, higher and further to the left values are desirable. Higher means more problems were solved and further to the left means, the algorithm found the solutions earlier. \n", + "\n", + "You can choose:\n", + "\n", + "- whether to use `n_evaluations` or `walltime` as **`runtime_measure`**\n", + "- whether to normalize runtime such that you the runtime of each problem is shown as a multiple of the fastest algorithm on that problem\n", + "- how to determine when an evaluation is close enough to the optimum to be counted as converged. Convergence is always based on some measure of distance between the true solution and the solution found by an optimizer. Whether distiance is measured in parameter space, function space or a combination of both can be specified. \n", + "\n", + "Below, we consider a problem to be solved if the distance between the parameters found by the optimizer and the true solution parameters are at most 0.1% of the distance between the start parameters and true solution parameters. " + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "efc15318", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjgAAAGoCAYAAABL+58oAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABVLUlEQVR4nO3deZxcZZX/8c+p6u6kk5AFErKHRGRLSAhJhyTsiIZAkERBEAFBRUZFlpkfi4iMcdTRcUUGBwYdWUYkKrKJCAxIlC0rBAibBsgespKt96o6vz/qVnV1p7q7uru2rv6+X69+perWrVunb4A6nOc8z2PujoiIiEgpCRU6ABEREZFsU4IjIiIiJUcJjoiIiJQcJTgiIiJScpTgiIiISMkpK3QAHTV79mx//PHHCx2GiIiIFAdLd7DbVXC2bdtW6BBERESkyHW7BEdERESkPUpwREREpOQowREREZGSowRHRERESo4SHBERESk5SnBERESk5CjBERERkZKjBEdERERKjhIcERERKTlKcERERKTkKMERERGRkqMER0REREpOzhIcM/uVmW0xs5WtvG5mdouZrTKzV81sSq5iERERkZ4llxWcu4DZbbx+OnBI8HMZcFsOYxEREZEeJGcJjrv/DdjRxilzgXs8bhEw0MyG5yoeERER6TkK2YMzEliX8nx9cExERESkS8oK+NmW5pinPdHsMuLDWHz4wx9m+fLluYxLREREsqS8vJxJkybl/XMLmeCsB0anPB8FbEx3orvfAdwBUFVV5VOnTs19dCIiItJtFXKI6hHgs8FsqhnALnffVMB4REREpETkrIJjZvcBJwODzWw98E2gHMDdbwceA84AVgE1wOdyFYuIiIj0LDlLcNz9/HZed+DyXH2+iIiI9FxayVhERERKjhIcERERKTlKcERERKTkKMERERGRkqMER0REREqOEhwREREpOUpwREREpOQowREREZGSowRHRERESo4SHBERESk5SnBERESk5CjBERERkZKjBEdERERKjhIcERERKTlKcERERKTkKMERERGRkqMER0REREqOEhwREREpOUpwREREpOQowREREZGSowRHRERESo4SHBERESk5SnBERESk5CjBERERkZKjBEdERERKjhIcERERKTlKcERERKTklBU6gFIQjcbYsae20GFIlnh1NV5fX+gwRERKxoAh+9N7v755/UwlOF1U3xDhip/9mc0fVBc6FMmC43e/x3nbXit0GCIiJeWNT36GKeeemdfP1BBVF727aaeSmxJh7nxs56pChyEiIlmgCk4XuXvy8VnHHcaYof0LGI10RZ+177L/e/Ghxp2TqqgbPqrAEYmIlIYxk4/M+2cqwemi1ASn6rDhHPXhYQWMRrpi2+InqAEoK2PClZcR7tev0CGJiEgnaYgqi8ys0CFIJ8Vqa6ldshSAPlVTldyIiHRzSnC6KJZSwZHuq2bxkuTMqb4nnljgaEREpKuU4HRRan4TUgWn26r+698ACA0YQO+jJhU4GhER6SolOF2U2oOj/KZ7imzZQv2bbwLQ9/jjsHC4wBGJiEhXKcHpIg1QdX/Vzz6XfKzhKRGR0qAEp6tSh6hCKuF0N+6eHJ4qHzuWioPGFDgiERHJBiU4XdS8yVgJTndT/9bbRLZsAaDviScUOBoREckWJThZpAJO91P9t3j1hnCYvscfV9hgREQka5TgdFFqk7G6jLuXWH09NYsWA1B59GTC/bUKtYhIqVCC00XN8pvChSGdULt0KV4b35pBzcUiIqVFCU4XOanTxJXidCfVf30WgNB+/aiccnSBoxERkWxSgtNFGqHqniLbtlO3ciUAfY47DivTtmwiIqVECU4XNVvor4BxSMdUP/dcMjvtp9lTIiIlRwlOFzWbJK4STrfg7snZU+WjRlE+blyBIxIRkWxTgtNFGqLqfhpWvUNk4yYA+p50ohJTEZESpASni5oPUemLsjuo/utf4w/MtPaNiEiJUoLTRargdC/e0ED1iy8C0PuoSYQHDSpwRCIikgtKcLqo+W7iynCKXe1LL+HVNYDWvhERKWVKcLpIu4l3L3uDjTWtTx/6VE0tcDQiIpIrSnC6KLWCE1IFp6hFd+6k7pVXAeh77EysoqLAEYmISK4oweki114N3Ub1c89DLAZo53ARkVKnBKeLmq+DU7AwpB3uTnUwPFU2fDgVhxxS4IhERCSXlOB0VUqGoyGq4tW4ejWN69YB8eqNGsJFREqbEpwuirnajLuDRHMxZvQ9QcNTIiKlTglOFqmCU5w8EqHm+ecB6H3kBMoGH1DgiEREJNeU4HSRmoyLX+3LLxPbsxfQ2jciIj2FEpwuap7fKMMpRtV/exYA692bymlVBY5GRETyQQlOFzmpKxkXMBBJK7p7N7UvvQxAnxnTCfXuXeCIREQkH3Ka4JjZbDN728xWmdnX0rw+wMz+aGavmNnrZva5XMaTC833olKGU2xqnn8BolEgvnO4iIj0DDlLcMwsDPwcOB0YD5xvZuNbnHY58Ia7HwWcDPzYzLrV8rLNdxOXYrP3b/HZU+EDh9DrsMMKHI2IiORLWQ6vfQywyt3fBTCzBcBc4I2UcxzYz+Klj37ADiCSw5iypqExyoPPvslvnlqZPKYKTu54LMaeR/9E44YNmb8nEqHxvdUA9D3hBCykEVkRkVSxvZuJrl1EdNMreKQhZ59TcdR5hIdNzNn108llgjMSWJfyfD0wvcU5twKPABuB/YDz3D2Ww5iy5oXX1zVLbgDKy/QFmit1r7zCzt/c1+n399PWDCIieN0uIuuWEF23mOjaxcQ+eC8vn1s27oSSSnDSlTNarop3GrAC+AhwMPB/Zvasu+9udiGzy4DLAMaMGZP9SDthx+7a5OP+fXrx0apx7NenVwEjKm2Nm95PPg7vvz9kWo0xo++JJ1A2dGiOIhMRKV7eWEt048tE1y4msm4Rsc1vsO9XMVDeB+s9IGdxWFn+J3jkMsFZD4xOeT6KeKUm1eeA73u8kWWVmb0HHA4sST3J3e8A7gCoqqoqiqWD6xuaRtLu/vo8QiENT+VS9IOd8QfhMCNuvUXDTSIiaXgsQuz9lUTWLQ6GnlZAtHHfE8PlhIdPJjxmBmWjpxMadiQWymVKkH+5/G2WAoeY2ThgA/Bp4DMtzlkLnAo8a2ZDgcOAd3MYU9Y0ROIzc8rLQkpu8iC2aycA4QEDlNyIiATcndj2VUTXLoonNeuXQcPeNGcaoaHjKRs9g/CY6YRHHI2VV+Y93nzKWYLj7hEz+yrwBBAGfuXur5vZl4LXbwe+DdxlZq8RH9K63t235SqmbKpviCc4vcpLK+MtVokKTnjQwILGISJSaLFdG4isW0R07WKi6xbjNdvTnhcaNC6ezIyeTtnoY3I6BFWMcvrt7O6PAY+1OHZ7yuONwKxcxpAr9Y3xIaqK8nCBI+kZookKzsCBBY1DRCTfYjU7iAaNwZG1i/Bd69KeZ30PJDxmOmVjZhAePZ3QfsPyHGlxUfmhkxoaExUcJTj5EN25C1CCIyKlzxtqiG5YRmTtYqLrFhHb+nb6E3v1p2z0NMLBsFNo0DgtV5JCCU4n1QcJjio4ueeRCLHd8Yl1ISU4IlJiPNpIdNOrRBPDTu+/CrE0S8KFexEeOaWpMfjAI7CQvoNaowSnk+oj6sHJl+juplUDVMERke7OPUZs69+Jrn0x3hi8YTk01u57ooUJDTuSstHTCY+ZQXj4UViZliPJlL6dO6khmCZeUabsOdeSU8SB8MCe1SQnIt2fu+O71hFZG1Ro1i/Baz9Ie27ogA8HFZoZhEdNxXrtl+doS4cSnE5KVnAqlODkWmznzuTj8MBBhQtERCRDseptwdTteFLjezalPc/6j2iq0IyeTqjv4DxHWrqU4HSSponnT7RZgqMKjogUH6/fQ3T9sniVZt1iYttXpT3Peg8kHCQ0ZWOmYwNGqzE4R/Tt3EmJhf7UZJx7zROcgQWLQ0QkwSP1RDe9Eu+jWbuY2ObXwaP7nlheSXhkFWVjphMePYPQkEMx02Kl+aAEp5MSWzVomnjuJRKcUN++WHl5YYMRkR7JY1FiW95ITt2ObngZovX7nhgqIzxsUnyBvTEzCQ+biIX1361CUILTSarg5E9yDRytYiwieeLuxHa8G+y6vYjI+mVQvzvtuaEhh8cX2Bs9g/DIqVhFnzxHK+kowemk+kb14ORLdGd8tkFowMDCBiIiJS225/3krtvRtYvx6i1pz7MBo+OrBY+ZQXj0MYQqNfmhGOnbuRMi0RixWHxT816aJp5zquCISC543S4i65YkN6r0D1anPc/6HJBcXC88Zgah/iPyG6h0ihKcTkjsQwWaJp5r7p7swQmrgiMiXeCNtUQ3vhSv0qxdRGzLm4Dve2JFX8KjpgUJzUxCBxysmU7dkBKcTkjsQwVa6C/XvKYGGhsBVXBEpGM8FiH2/sr4asFrXyS66RWINu57Yric8Iij47tuj5lBaOgELKSvx+5Of4OdUJ+S4PSq0C3MJa2BIyKZcndi2/+RrNBENyyDhuo0ZxqhoePjTcFjphMecTRWXpn3eCW39O3cCarg5E9UqxiLSBtiuzYETcHxBfa8Zkfa80L7j0vuul02ahrWW//DVOqU4HSCenDyR/tQiUiqWM2OYOp2fLaT71qf9jzrNzQYcgoag/sNzXOkUmhKcDpBFZz8ie7amXysVYxFeh5vqCG6IdgCYe0iYtv+nv7EXv0pG31McraTDRqrxuAeTglOJ6gHJ3+SFZzycqxv34LGIiK559HGYAuE+JBT9P3XIBbZ98Sy3vHG4CChCR14BBbS/3BKE307d0KzBEcrGedUcg2cAQP0f2MiJcg9Rmzr28m1aKLrl0Okdt8TLUxo2JHxBfZGTyc8fDJWVpH/gKXbUILTCak9ONqqIbcSqxhreEqkNLg7vnNtctft6LoleN3OtOeGDjikaQuEUVVYr375DVa6NSU4ndDQrIKjW5hLMa1iLNLtxfZuJboumLq9bhG+5/2051n/EUGFJtgCoe/gPEcqpUTfzp2QOkSlCk5uNa1irBlUIt2F1+8hsn5pvI9m7WJiO95Je55VDiI8+hjCo2dQNmY6NmC0hqIla5TgdEKDenDywhsbie3dC0B4kNbAESlWHqknunEF0XWL4lsgbH4dPLbvieWVhEdVJRfYCw0+FLNQ/gOWHkEJTic068HRNPGcie7alXysHhyR4uGxKLEtbwRTtxcT3fgyROv3PTFURnj4UfGm4DEzCA+biIXL8x+w9EhKcDohUcGpKA+rnJpDqasYh7TIn0jBuDuxHe8GC+wtIrJ+KdTvSXtuaMjh8anbY2YQHjkFK++T52hF4pTgdEKiB6eXqjc5pW0aRAontmdT055O6xbj1VvTnmcDxwS7bgeNwZX6d1WKgxKcTmgIhqgqtE1DTiXWwAFt0yCSa163i8i6JfEKzdpF+M41ac+zPoPjU7eD2U6h/sPzHKlIZpTgdJC7s27rbgB6len2ZZu7E92+HaJRIhs2Jo9rFlV2xKq3QWOaRdSkB3JiO9clKzSxLW8Cvu9pFf0oGzUtvuv26BmEDjhYQ/PSLegbuoO+9+vneHvtdkAbbebC9p/dQs2ixc2OhfbbD1My2WX1S35Jw/M3FzoMKXbhCsIjJgdbIMwgNHQ8FtK/f9L96J/aDnB3Fr+5Ifn8oGEDCxdMCfKGBmoWL9nneMWHDy5ANKUl8s5CJTeSnoUIHTg+vuv26BmERx6NlfUudFQiXaYEpwMi0aZ1HQ4eMYgvz60qYDSlp3HzFvB4ibzfrI9R8eGDsbJyKicfVeDIurfYjveoffxr8Sflfeh14rXaw0cAsF4D4glNbw0BS+lRgtMBqQnO8ZPG0Fs7iWdVZNOm5OO+J51Ir4NVuekqr99L7R+vhIb4gom9Z/875R/+aIGjEhHJPS0h2QHRWFMDXnlYty7bUhOc8mHDChhJaXCPUffE14nteA+Aiun/pORGRHoMfUt3QCTSVMEJK8HJusYgwQkN6E+ob98CR9P9NSz5BZF3/gJAeNwJVMz4SoEjEhHJH31Ld0Ak1pTglCnBybpEBad8mNbV6KrIu3+l4YVbgfhCbJWz/wMLadafiPQc+pbugNQeHCU42de46X0AyoYrwemK2AdrgqZih/JKKj9+C9a7f6HDEhHJq1a7ZM1sSltvdPeXsh9OcUtNcMIhLXSVTbG9e4ntji+gWD5c/Ted5Q3V1D5yZXKfoN6n/TvhwR8ucFQiIvnX1jSgHwd/9gaqgFcAAyYBi4Hjcxta8WlewVG5P5sa338/+VgVnM5xd+qe+AaxHe8AUDHtUsoP+ViBoxIRKYxWx1nc/RR3PwVYA0xx9yp3nwocDazKV4DFJNoswVEFJ5sim5TgdFXD0l8SWfV/AIQPOo6KY68ocEQiIoWTSSPJ4e7+WuKJu68EJucsoiKmHpzcScygwozyYUMLG0w3FFn9HA3P3wKADRhF5Rk/UFOxiPRomaxU96aZ/RL4NfGd2C4E3sxpVEWqWQ+OEpysSsygCg8ZjJWXFzia7iW2cy21j10HOJRVUnnWLVqZVkR6vEwSnM8BXwauCp7/DbgtZxEVMS30lzuJCk65hqc6xBtqqH3kKqiPN2j3nvVtwoMPLXBUIiKF126C4+51ZnY78Ji7v52HmIqWKji54e7JHhz132TO3an7v5uIbf8HABVVn6f8sNkFjkpEpDi0+y1tZmcBK4DHg+eTzeyRHMdVlJpPE1eCky3RDz7A6+sBbdHQEQ3L7iTy9ycACI+ZScVxV7XzDhGRniOTb+lvAscAOwHcfQUwNmcRFbHUBEdDVNmTugeVKjiZiax5nobnbwbA+o+i8owfqqlYRCRFJt/SEXfflfNIuoHmQ1SaJp4tjSlTxNWD077YznXU/ula8BiU9abyrJuxyoGFDktEpKhk0mS80sw+A4TN7BDgSuCF3IZVnDRNPDeSFZzycsKDDyhsMEXOG2uo/WNKU/HH/o3wkMMLHJWISPHJ5Fv6CmACUA/8BtgFXJ3DmIpWVJtt5kRyBtXQoZh6m1oVbyr+JrFtfwegfOrFlB9+RoGjEhEpTplUcA5z9xuBG3MdTLGLRJTg5EJyBtUIDU+1pfGlu4m8/WcAwqOn0+v4fy5wRCIixSuTb+mfmNlbZvZtM5uQ84iKWCRlHRxNE88Oj0SIbNkCqP+mLZE1L1L/7E8AsP4j6D3nR1gok/8/ERHpmdr9lg72ozoZ2ArcYWavmdk3ch1YMYqqByfrIlu2QjQKQJl2EU8rtmsDdY8FTcXhXlR+/GeEKgcVOiwRkaKW0be0u7/v7rcAXyK+Js6/5jKoYtWsyVi9IlmROkVcFZx9eWMttX+8Cq/bCUDvj80nfOARhQ1KRKQbyGShvyPMbL6ZrQRuJT6DalTOIytCmiaefY3vaw2c1rg7dU/NJ7b1LQDKj76I8iM+XuCoRES6h0wG8e8E7gNmufvGHMdT1BIJTshMKxlnSaLB2Pr2IbTffgWOprg0vvxrIm/9CYDwqGn0OuFfChyRiEj3kcleVDPyEUh3kJgmrupN9iSniA8bjpnua0Jk3RLq//YjAGy/YfGm4rB2WRcRyVSrCY6ZvQZ4upcAd/dJOYuqSDUGFRw1GGePNtncV2z3Rur+9P/AoxCuiDcV99ECiCIiHdFWBefMvEXRTUSj8XxPDcbZEaurI7pjBwDlmkEFgEfqqP3j1XjtBwD0/ug3CQ/t0asziIh0SqsJjruvSTw2s6HAtODpEnffkuvAilEkMZ25TAlONkTeb9qDqmzEiAJGUhziTcX/RmzLGwCUT/4M5ePnFjgqEZHuKZNZVOcCS4BPAecCi83snFwHVowiQQVHDcbZ0XyTTVVwGlf8hsibjwAQHjmVXideW+CIRES6r0xmUd0ITEtUbcxsCPAUcH8uAytGEfXgZFVkY9OkvLJhPTvBiaxfSv1ffwCA9RtK7zk/VlOxiEgXZPJNHWoxJLU9w/dhZrPN7G0zW2VmX2vlnJPNbIWZvW5mf83kuoUSVYKTVYkKTnj//Qn17l3gaAontmcTdY+mNhXfTKjv4EKHJSLSrWVSwXnczJ4gvhYOwHnAY+29yczCwM+BjwHrgaVm9oi7v5FyzkDgv4DZ7r7WzA7sYPx5FYkpwcmmxCrGPXmLBo/UB03F8Wbr3qfeRHjYxAJHJSLS/WWyDs61ZvZJ4HjiU8TvcPcHM7j2McAqd38XwMwWAHOBN1LO+QzwgLuvDT6raJuXn1z6Dovf2ABoHZxM7V34V+pWrgQg9sGa5Jd4QsOaWgCscQ21f74+7/EVg9iu9cQ2vw5A+VGfpnzCJwockYhIachoO2J3f8DM/gacAKzN8NojgXUpz9cD01uccyhQbmYLgf2An7n7PRleP2/e37GXnz+4NPm8skK7OLcnsmULO27/74zOtbp/EHlrW44jKm7hEVPodVLPTPJERHKhrYX+HgW+5u4rzWw48BKwDPiQmf3C3W9u59rpyhwtFw4sA6YCpwKVwItmtsjd/94ilsuAywDGjBnTzsdm3869dc2ef+JEbXbYnoa1zfPgUK/64EGY1H80QpUxen2oD1YxOo/RFZfQgJH0nv19NRWLiGRRW6WIce6+Mnj8OeD/3P2zZrYf8DxwczvXXg+kfmuNAlruZbUe2Obu1UB1UCU6CmiW4Lj7HcAdAFVVVelWV86pxkg0+fjbXziFSQcPzXcI3U5kQ9Nf9bAbL6L+qfiU5z6f+a0WrhMRkZxrq1u2MeXxqQSNxe6+B4ilfUdzS4FDzGycmVUAnwYeaXHOw8AJZlZmZn2ID2G9mWnw+dIYafp1y8vCBYyk+0jsMRXq3x8rT8lJyysLFJGIiPQkbVVw1pnZFcSrLFOAxwHMrBJot5bu7hEz+yrwBBAGfuXur5vZl4LXb3f3N83sceBV4knTL1OqRkUjsQcVaAZVphqDCk758OF4Y03yuCnBERGRPGgrwfkC8G/AR4Hz3H1ncHwGcGcmF3f3x2gxpdzdb2/x/IfADzOMtyAizSo4SnDa4+7JRfzKRo6Axtrka1bep1BhiYhID9LWXlRbgC+lOf4M8Ewugyo2jdGmHhwlOO2L7dlDrLoaCCo4kZTWqzJVcEREJPf0bZ2BZhWcsHpw2tOYsgVD+YiUCo6FQDOFREQkD5TgZCB1FpUqOO2LbNyUfFw2YjieSHDKKzHTIokiIpJ7+rbOgJqMOyZZwQmHKTvwQGiMryNkGp4SEZE8affb2sx+YGb9zazczJ42s21mdmE+gisWmibeMYkEp2zYMCwcxiNNFRwREZF8yKQcMcvddwNnEp8yfihwbU6jKjKR1CZjVXDalRiiKh8xIn4gGKLSFHEREcmXTL6tE12hZwD3ufuOtk4uRYkKTsiMsBKcNnkkQmRLfM/U8hHD48cS6+AowRERkTzJZNfIP5rZW0At8BUzGwLUtfOekpLowSlTg3G7Ips3Qyy4X0EFx1XBERGRPGv3G9vdvwbMBKrcvRGoBubmOrBikphFpeGp9jVuSJ0iPjw4mEhwtMifiIjkR7sVHDMLAycAY80s9fyf5CyqIpNYB0czqNqX2IMK4ov8QVMFh7LehQhJRER6oIyGqIgPSb1GZptslpzEEJVmULUvsYt4aEB/Qv36xQ9qiEpERPIskwRnlLtPynkkRSySGKJSD067GjcFm2wmZlCBpomLiEjeZfKN/Wczm5XzSIpYsoKjbRra5O5Na+AkhqfcVcEREZG8y6SCswh40MxCQCNggLt7/5xGVkQS08RVwWlbbPduvDo+Jbx8ZFDBiTaCByObSnBERCRPMklwfkx8FtVr7u45jqcoJXYTV5Nx25ptsjk8SHASw1NoqwYREcmfTL6x/wGs7KnJDTTNolIFp23NN9lMrIFT03SCKjgiIpInmVRwNgELzezPQH3ioLv3mGniTQv9qQenLckKTlkZZQcOAVKmiKMeHBERyZ9MEpz3gp+K4KfH0UJ/mWlM7EE1bBgWCu5VSoKjCo6IiORLuwmOu38LwMz6unt17kMqPk1NxqrgtCWSmEGVWMGYFhUc9eCIiEietFuSMLOZZvYG8Gbw/Cgz+6+cR1ZEImoybpc3NqZsstm0Bg4aohIRkQLI5Bv7ZuA0YDuAu78CnJjDmIqOpom3r/H9zRD0obdWwdEQlYiI5EtG39juvq7FoWgOYilaES30167IptRNNlXBERGRwsqkyXidmR0LuJlVAFcSDFf1FKrgtK8xZYp42m0aALSbuIiI5Ekm39hfAi4HRgLrgcnB8x4jsdCfEpzWJRqMQwMHEuqTksiogiMiIgWQySyqbcAFeYilKLl7soJTpiGqViWniA8f3ux4sx6cst75DElERHqwVhMcM/tPoNXVi939ypxEVGQS/TegdXBak7rJZnIPqsRriQQnXIGFlCCKiEh+tFXBWZa3KIpYaoJTpiGqtGK7duM18S0ZylpUcJJ7UWl4SkRE8qjVBMfd7059bmb944d9T86jKiKJ4SkorYX+PBYjsnlzGzW61sWqt0K0Ifm8ce2G5ONw/zCxD1Ynn0e3xPvR1X8jIiL51G4PjplVAXcC+8Wf2k7g8+6+PMexFYUb7ng6+bhUhqg8FuP9r99I4+o1Wb9243M3EFvWsM9xJTgiIpJPmUwT/xXwFXd/FsDMjiee8EzKZWDFoL4xwvqtu5PPDxo2sHDBZFF02/acJDehXg2Eeu2b3ACEhhye9c8TERFpTSYJzp5EcgPg7s+ZWY8YporFmsZvjp84hiMOGlzAaLInVp/cFJ5+s0+j14c/nNH7vHYn9Qu/B0BoVBXhA1LeZ0bF2KGE90tTqSnrTdlBx3YpZhERkY5oaxbVlODhEjP7b+A+4h0b5wELcx9a4XlKf8qho/cvXCBZ5vV1yceVEydSOXVKG2c3afzH/+GDd8bfd9ZnKRuZ2ftERETyra0Kzo9bPP9myuNOtKZ2P7GUDMfMChhJdnl90zCS9eqV8fti768M3hQifKCGnEREpHi1NYvqlHwGUoxSh6hCJZXgNA1RdSTBiW6OJzihAw7GtO2CiIgUsXanBZnZADP7iZktC35+bGYD8hFcoTVLcEKlk+Ck9uBY78wSHPcY0c2vAxAeemRO4hIREcmWTOY9/wrYA5wb/OwmPouq5KUOUZVWBaepByeUYQXHP1gDDXvj71GCIyIiRS6TWVQHu/vZKc+/ZWYrchRPUWmW4JRQBcfrOj5ElRieAggPU4IjIiLFLZMKTm2w9g0AZnYcUNvG+SVDPThNookG43AFocGH5iIsERGRrMmkgvMl4J6UvpsPgItzF1LxKNUKTrMenIqKjN6TbDAecjgWLs9JXCIiItnSZoJjZmHgQnc/KtiLCnff3dZ7Sok3myZewECyLFHBsV69sFD7RTyPNhLb8hag4SkREeke2kxw3D1qZlODxz0msUmIlvgQVabDU7HtqyAaf0946IScxSUiIpItmQxRvWxmjwC/B6oTB939gZxFVSSaTxMvjY02oWmIqsP9N2gGlYiIdA+ZJDj7A9uBj6Qcc6DkExwv2Wni8QQn0yniscQMqoq+hPYfl6uwREREsqa9HpwhwM+BVe6+My8RFZHm6+AUMJAsaxqi6liDcfjA8ZiVTiVLRERKV6vfVmZ2KfA68J/AW2Z2Vt6iKhKlupJxYh0c69W7/XMba4ltWwVoeEpERLqPtio4VwMT3H2rmX0IuBd4JC9RFYlSnyaeSQ9ObOtb4FFAM6hERKT7aGu8ocHdtwK4+7tA5rsyloiUAk6P7cFJbTDWHlQiItJdtFXBGWVmt7T23N2vzF1YxSF1iMpKMMHJZKPNRP+NVQ7C+o/IaVwiIiLZ0laCc22L58tzGUgxisViycelNETVkXVwEjuIh4YeWVJJnoiIlLZWExx3vzufgRQjL9EhqliGQ1Retxv/YDWg/hsREeleNOe3DbESXAfHo1GIRID2KzjRLW8kH6v/RkREuhMlOG0oxWnizXYSb6cHJ/r+a8nHIVVwRESkG2lrHZz/CP78VP7CKS6lOE08VpeS4LSzDk5iBWPbbzihPgfkNC4REZFsaquCc4aZlQM35CuYYlOSQ1T1dcnHoYq2VzJOTBHX8JSIiHQ3bc2iehzYBvQ1s92AEd+DygB39/55iK+gSnGaeKZDVLHqbfjezYCGp0REpPtptYLj7te6+wDgT+7e3933S/0zjzEWTGoFJ1wiQ1TNEpw2moxjWuBPRES6sXZ3E3f3uWY2FJgWHFqcWOG41DVrMi6RCk6sviH5uK0EJ7HAH0B46PicxiQiIpJt7c6iCpqMlwCfAs4FlpjZObkOrBh4CTYZN+vBaaPJOJHghAaNw3rtl/O4REREsqndCg7wDWCau28BMLMhwFPA/bkMrBhEm/XgFDCQLMpkiMrdk0NUoaET8hKXiIhINmWyDk4okdwEtmf4vm6vFKeJe137Tca+ewNetxPQCsYiItI9ZZKoPG5mT5jZJWZ2CfAn4LFMLm5ms83sbTNbZWZfa+O8aWYWLbahLy/BaeKxlApOa1s1NNtBfNjEnMckIiKSbZk0GV9rZp8Ejic+RfwOd3+wvfeZWRj4OfAxYD2w1Mwecfc30pz3H8ATnYg/p0qxyTiTIapkg3GojNCQw/MRloiISFZl0oODuz8APNDBax8DrHL3dwHMbAEwF3ijxXlXAH+gaZZW0eiOQ1SRbdvY/dDDxPZW7/NabO9m6la+F38Sgronrk97jeiGl+OnDD4EK2t/x3EREZFik1GC00kjgXUpz9cD01NPMLORwCeAj1CMCU43rODsfvAh9j79l3bPs1Ajkb+3XTTT+jciItJd5TLBSZcReIvnNwPXu3u0rZWCzewy4DKAMWPGZCu+dqXkN92mgtO4cSMQH34KH5Cyf5THiH2wOv6aOb0PhtD+41q9jlUeQMXUi3MZqoiISM5klOCYWSUwxt3f7sC11wOjU56PAja2OKcKWBAkN4OJ738VcfeHUk9y9zuAOwCqqqpaJkk50x23aohu3wFA5ZSjGXzVlcnjsb1bqP7FRwDodeq/UjHp3ILEJyIikg+ZLPT3cWAF8b2pMLPJZvZIBtdeChxiZuPMrAL4NNDsfe4+zt3HuvtY4uvqfKVlclNI3a0Hx2MxIjviCU6z6g1AJKW5OKy+GhERKW2ZTBOfT7xheCeAu68Axrb3JnePAF8lPjvqTeB37v66mX3JzL7UuXDzq7v14MT27IFIBIDw/vs3e82jTVs0UNb2LuIiIiLdXSZDVBF339WZIRp3f4wWa+a4++2tnHtJhz8gx7rbOjjR7duTj8sGt17BQTOjRESkxGWS4Kw0s88AYTM7BLgSeCG3YRWH7jZEFUlJcPYZoopqiEpERHqOTIaorgAmAPXAb4BdwNU5jKlodLchqkSDMUBZyyGqSOoQlRIcEREpbW1WcIJVhh9x948CN+YnpOLR/So4QYITDhMaMKDFi027iGvxPhERKXVtVnDcPQrUmNmAts4rVbFutpt4dEd8iCo8aBAWav5X26zJOKwmYxERKW2Z9ODUAa+Z2f8ByfX/3f3K1t9SGmLdrsk4XsEpa9l/A82niauCIyIiJS6TBOdPwU+Pk6jghMy6xUJ/iSbjfRqMAU9pMkZNxiIiUuIy2U387nwEUowS08S7Q/+Nx2JEk4v87b/vCRGtgyMiIj1HuwlOMDX8e8B4oHfiuLt/KIdxFYXEEFU3KN4Q270bolGglSGq1GniZb33fV1ERKSEZDJN/E7gNiACnALcA/xvLoMqFqlDVMUukjJFvOUqxgCeutCfmoxFRKTEZZLgVLr704C5+xp3nw98JLdhFYdoNxqiira1yB80NRlbCEK53EReRESk8DKaRWVmIeAfZvZVYANwYG7DKg4ei//ZHSo4if4bSD9ElWwyDvfqFg3TIiIiXZFJBedqoA/xLRqmAhcBF+cwpqKR6MHpDglOcpuGcJhQ//3SnBA0GavBWEREeoBMZlEtDR7uBT6X23CKS7IHpxsNUYUP2H+fRf7iJ8QrOGowFhGRniCTWVSHAtcCB6We7+4l34fTnaaJJxf52z9N/w0pTcZqMBYRkR4gkx6c3wO3A78AorkNp7hEk9PEiz/BiaRUcNKfkKjgaJE/EREpfZkkOBF3vy3nkRSh7jJN3GMxoh98ALSyBg6pTcaq4IiISOlrNcExs0Qp4I9m9hXgQSC5mIq770j7xhIS6yZDVLFdu5KL/KVbAwdINhmrgiMiIj1BWxWc5YADiW/3a1Nec6DkVzL2bjKLKpIyRTw8uL0eHCU4IiJS+lpNcNx9XD4DKUbdZYgqdZG/1oaoiCamiSvBERGR0tfqOjhmNs3MhqU8/6yZPWxmt6QMX5W07jJNPLItZRXjVoeo6gANUYmISM/Q1kJ//w00AJjZicD3ie9DtQu4I/ehFV53WegvuYpxWRmh/v3TnuOJhf7UZCwiIj1AWz044ZRG4vOAO9z9D8AfzGxFziMrAsndxDNZ77mAEkNUZfvv3/qU9qimiYuISM/R1ld32MwSCdCpwF9SXusRuzV2mx6coIKTdpPNQLLJWAmOiIj0AG0lKvcBfzWzbUAt8CyAmX2Y+DBVyesu08Qj29tPcJJNxhqiEhGRHqCtWVTfNbOngeHAk56YMx2v+lyRj+AKLfEbF3MFx2OxZAWnrJVVjN09ZSVj7UUlIiKlr82hJndflObY33MXTnGJxWJAcSc40Z27IIiz1QpOtJH40kVoN3EREekRirx9trC6wzTx1DVwWp0iHk0uQI1poT8REekBekSzcGftqo4nBu1VcKI7dxLdszcfIe2jYdWq5ONW96Gq2930RE3GIiLSAyjBacXKd7ewZnO8l7qtCk7NokVs+9l/NjXsFFC6ncS9fg/Vvzot+dzUZCwiIj2Ahqha8dq7m5OPDxo6sNXzal99rSiSm7IRwwntt98+x6Pvv9bseWjI4fkKSUREpGBUwWlFLCVn+dwZk1s9zxsbk48HX31lDiNqg4XodcThaRf58/o9yce9T/8B4aHj8xmZiIhIQSjBaVWwirFBWbiNQldjBICy4cPpM2NGPgLrkNQEJzxySgEjERERyR8NUbUiMerU6tYHifOi8QTHyoszV0xtMLZe6fepEhERKTVKcFqRWNewvQniiSEqKyvPcUSdVB8kOBaG8srCxiIiIpInSnBakWjBabeC01jkFZxgiMp692/3dxERESkVSnBakazgtJcTRIIm4yKt4HiigtNr3xlWIiIipUoJTiuSPTjtDFIlKzhlRV7BUYIjIiI9iBKcVjQNUbVzXqSbDFGpwVhERHoQJTitaBqiaqeCEynyJuNgFpUSHBER6UmU4LQi08WJE0NUFG0FJ5HgaIhKRER6DiU4rfBgkKq9jTYTTcZWXpwVnNRZVCIiIj2FEpxWeCz+Z7s9OEXcZOyROog2xJ+ogiMiIj2IEpxWOJmNUTUt9FeECU5d0zYNGqISEZGeRAlOa4L8pr0hqqZZVMU3RJVcAwcNUYmISM+iBKcVsaaFcFrlsRhEo/EnRVjBoV77UImISM+kBKcdbVZwguoNFGsFJ3WISgmOiIj0HEpwWhHLYJ64N0twiq+C0zzBUQ+OiIj0HEpwWpMYoWqjgpNoMAawcBEmOHVNQ1SaRSUiIj2JEpxWJGZRtTVC5UU/RKUeHBER6ZmU4LQillEFpynBKcaVjJNDVGW9sbKKwgYjIiKSR0pwWpPYi6qtcyIpQ1TFuBeVtmkQEZEeSglOK5p2E8+sglOUTcZ12klcRER6JiU4rWjaTbyNc5pVcIowwUkMUamCIyIiPYwSnFZksM5f8wpOEQ5RJXcS760ER0REehYlOK3IaIgqUuxNxokeHA1RiYhIz6IEpxWZDFGRug5OUU4TVw+OiIj0TEpwWuGZTBOPFO8QlXsMkgmOhqhERKRnUYLTimQFp40unGYrGZeFcx5ThzTUgMcA7SQuIiI9jxKcViS3osp0FlWRDVGl7kOlWVQiItLTKMFpRWKrhrZ2E/dItOlJsQ1R1e9KPlYPjoiI9DQ5TXDMbLaZvW1mq8zsa2lev8DMXg1+XjCzo3IZT0dksJl4iybj4ppF1XwncSU4IiLSs+QswTGzMPBz4HRgPHC+mY1vcdp7wEnuPgn4NnBHruLpqA4v9FfEQ1RqMhYRkZ4ml2WHY4BV7v4ugJktAOYCbyROcPcXUs5fBIzKYTztemfDDh55/m3qG6O8+Pp6oGmIKlZTw64/PEBk69bk+bVLliYf52slY49FaHjhP4l9sKbN82J7NiUfK8EREZGeJpffyiOBdSnP1wPT2zj/C8CfcxhPu+56/BVefWdzs2MV5fHZUTUvvMiePz2W9n1WXg7h/Myiirz9OA1L/yfzN1gIqxyUu4BERESKUC4TnHSDO2k7W8zsFOIJzvGtvH4ZcBnAmDFjshXfPvbU1CcfHzR0ABXlYT79kSMBiO7dm3ytbNiwpiGpUIh+p36kzfVysimy9sXgc8sIDRrX9skGZYfNwXr1y31gIiIiRSSXCc56YHTK81HAxpYnmdkk4JfA6e6+Pd2F3P0Ogv6cqqqqTNp/OyUW9N1MHz+Sr194QosXY8mHw3/0g4JsrunuRNctASB80Ez6zLst7zGIiIh0B7mcRbUUOMTMxplZBfBp4JHUE8xsDPAAcJG7/z2HsWQkFmtjanhKgkOoMLPrfdd6POitKRvd1mifiIhIz5azMoS7R8zsq8ATQBj4lbu/bmZfCl6/HfhX4ADgv4Ihnoi7V+UqpvYkE5zQvgmOpyQ4VqAEJ7JucfJxeNQxBYlBRESkO8jpOIu7PwY81uLY7SmPLwUuzWUMHZFY+yacroKTeLFAyQ1AdH0wa6tXf0JDDitYHCIiIsVOKxmniCX2bkpTwUkOUeWpmbil1P6bslFVWKjI9r4SEREpIkpwUrTVg5MYoirU8FTsg/fw6vgaPOHRGp4SERFpixKcFFFvvQcnWcEpUIKTqN4AhNVgLCIi0iYlOCkSbTZp17SJFbYHJ7ou3n9jlfsTOuDDBYlBRESku1CCkyIxRJWuydg9UcHJfw+Oe4zo+mD9m1HT8raooIiISHelBCdFW9PEKWAPTmz7Krz2A0D9NyIiIplQgpMisYN4mwv9FSDBSe2/KVOCIyIi0i4lOCkSWzWkGwJKLvRnhUtwrO8QbNDYvH++iIhId6MEJ0W0zSGqIPnJcwXHY1Ei65cB8eEp9d+IiIi0TwlOilgRThOPbX0b6ncD6r8RERHJlBKcFG314BRqFlU0Zf8pbbApIiKSGSU4KZpWMk77IgCW5x6cSDA93PqPIDRgVF4/W0REpLtSgpOi2IaoPNpIdP1yQLuHi4iIdIQSnIC7N20YXiS7ice2vAGNNYCmh4uIiHSEEpxAonoDbU8Tz+cspkiz/aeU4IiIiGRKCU4g0X8DxTNElVz/ZuAYQvsNy9vnioiIdHdKcAKpFZx0CY7nOcHxSAPRjS8Dmj0lIiLSUUpwAin5TZs9OPla6C/6/msQqQMgPHpaXj5TRESkVCjBCTQbomprL6o89eAkdg8HzaASERHpKCU4gfaGqPLdg5NY4C+0/8GE+g7Oy2eKiIiUCiU4gfYqOMlZVHlIcDxSR3TTK4BmT4mIiHSGEpxAe9PEE5tt5mOrhujGVyDaCCjBERER6QwlOIFimibetP+UUTZKDcYiIiIdpQQnkFrBCbcxTTwfQ1SJBf5CQw7FKgfm/PNERERKjRKcgHs7s6jytFWDN9QQ27wSgLDWvxEREekUJTiB1CGqtDPB8zRNPLrxJYhFACjT9HAREZFOUYITKJaVjBPbM2AhwqOm5vSzRERESlVZoQMoFtt21SQfh8xwdyIbN+KN8dlMjatXA13rwfFYlNiOdyEWbfWcyJoX4jEcOB7rtV+nP0tERKQnU4IT+OPzf08+DoVC7LjjF1Q/s3DfE7uQ4NQ+fDnR1c9ldG6ZpoeLiIh0moaoAiOHNFVLxg4bQN0rr6Y9r+Kggzp1fW+szTi5AaPskI916nNEREREFZykU44ex2FjBjP6wP6MHNyf9fX1AIT3359Bn7sYgFDffvQ6/LBOXT+2c23ycUXV5wkNP6rVc0MDxxAefEinPkdERESU4CSNGTqAMUMHJJ/HggSnz3HH0Wda1xfbi32wOvm47NDZhIeO7/I1RUSkfcuWLeOee+7hlltuycr1Fi5cyI9+9CMeffRR5s+fT79+/bjmmmv2Oe/aa6/lscce44wzzuCHP/xhhz6jX79+7N27Nyvx9lRKcNLwaBQi8anaod69snLN1AQnNKhzw1wiItJxVVVVVFVV5f1z//u//5utW7fSq1d2vkekY5TgpOFB9QbAsvQPZuyDNfHr9R2CVfTNyjVFRIrBLx99ifc2fZDVa44bPohLz5zS5jnV1dWce+65rF+/nmg0yk033cSHPvQhrrrqKqqrq+nVqxdPP/00y5cvb1Zxeeedd9iwYQPr1q3juuuu44tf/CIXXXQR55xzDnPnzgXgggsu4LzzzuOss85qN9ZXXnmFj3zkI82ud9ZZZ1FdXc306dO54YYbqKqq4oILLiAajXL66afzk5/8hL1797Jp0ybOO+88du/eTSQS4bbbbuOEE04A4P/9v//HM888w6BBg1iwYAFDhgzp+o3tQZTgpBGry0GCszOe4Kh6IyKSHY8//jgjRozgT3/6EwC7du3i6KOP5re//S3Tpk1j9+7dVFZW7vO+V199lUWLFlFdXc3RRx/NnDlzuPTSS/npT3/K3Llz2bVrFy+88AJ33313RnGku94jjzxCv379WLFiBQBnnnkmV111Feeffz6333578r2/+c1vOO2007jxxhuJRqPU1MSXLKmurmbKlCn8+Mc/5t/+7d/41re+xa233trFO9azKMFJwxuaEpxQ1io4q+PXGzg2K9cTESkW7VVacmXixIlcc801XH/99Zx55pkMHDiQ4cOHMy3om+zfv3/a982dO5fKykoqKys55ZRTWLJkCfPmzePyyy9ny5YtPPDAA5x99tmUlWX2Fdna9VK9+OKLPPTQQwB85jOfSfbsTJs2jc9//vM0NjYyb948Jk+eDMSXKznvvPMAuPDCC/nkJz/ZwbsjmiaeRrMhqiz04HjtTqjbBaiCIyKSLYceeijLly9n4sSJ3HDDDTz44INYBtvptDwn8fyiiy7i3nvv5c477+Rzn/tcxnG0dr1MnHjiifztb39j5MiRXHTRRdxzzz0ZfYa0TwlOGp46RFXR9QSneYPx2C5fT0REYOPGjfTp04cLL7yQa665hkWLFrFx40aWLl0KwJ49e4gEE0ZSPfzww9TV1bF9+3YWLlyYrPhccskl3HzzzQBMmDAh4zhau16qGTNm8Ic//AGABQsWJI+vWbOGAw88kC9+8Yt84Qtf4KWXXgIgFotx//33A/FhrOOPPz7jeCROQ1RpxLLcZKwER0Qk+1577TWuvfZaQqEQ5eXl3Hbbbbg7V1xxBbW1tVRWVvLUU0/t875jjjmGOXPmsHbtWm666SZGjBgBwNChQzniiCP2GV5qT2vXS3XzzTdz4YUX8uMf/5g5c+YwYEB8WZKFCxfywx/+kPLycvr165es4PTt25fXX3+dqVOnMmDAAH7729928O6Iecomk91BVVWVL1u2LKefUbN0Kdt+/FMAhn3vu1SMG9el69U/dzMNS38JFqbfFcuwcHk2whQRkQ5qa92ampoaJk6cyEsvvZRMQLKlpqaGyspKzIwFCxZw33338fDDD2f1M3qwtON3quCkke1p4skp4gNGKrkRESlCTz31FJ///Of5l3/5l6wnNwDLly/nq1/9Ku7OwIED+dWvfpX1z5DmlOCk4VmeJh7buRpQg7GISKHNnz8/7fGPfvSjrF27ttmxJ554guuvv77ZsXHjxvHggw92+HNPOOEEXnnllQ6/TzpPCU4aqT04XZ0m7h4j9kH8XxpNERcR6T5OO+00TjvttEKHIZ2kWVRpeEND8rH17t21a+3ZDNF4wqQKjoiISH4owUnD6+riD8JhLMOFnlqjGVQiIiL5pwQnjcQQVfaniKuCIyIikg9KcNJIzKLKxjYNyQSnrDfWb2iXryciIiLtU4KThicrOBVdvlbqJptmut0iIvm2bNkyrrzyyoLGcPLJJ5NuDbe77rqLr371qwWIqH39+vUrdAhdollUaTQlOF1rMAZtsikipa9u4feJbX0rq9cMDTmc3id/LSvXqqqqoqqqKivXKrRIJJLxJqA9nUoKacTqstOD45EGfPdGQP03IiLZVl1dzZw5czjqqKM48sgj+e1vf8vSpUs59thjOeqoozjmmGPYs2cPCxcu5MwzzwTi6+BcdNFFfOQjH+GQQw7hF7/4BRDfaDN1ZeELLriARx55JO3n3nXXXXzyk59k9uzZHHLIIVx33XXJ15588klmzpzJlClT+NSnPsXevXv3ef+dd97JoYceykknncTzzz+fPL5161bOPvtspk2bxrRp05KvzZ8/n8suu4xZs2bx2c9+lvnz53PxxRcza9Ysxo4dywMPPMB1113HxIkTmT17No2NjUB8ccGTTjqJqVOnctppp7Fp0yYAfvGLXzBt2jSOOuoozj77bGpqagB47733mDlzJtOmTeOmm27q9N9L0XD3bvUzdepUz7VNN97ka8473zd/59+7dJ3ItlW++ycTfPdPJnjD6w9lKToREXF3v//++/3SSy9NPt+5c6ePGzfOlyxZ4u7uu3bt8sbGRn/mmWd8zpw57u7+zW9+0ydNmuQ1NTW+detWHzVqlG/YsMEXLlzoc+fOTV5n7Nix3tjYmPZz77zzTh83bpzv3LnTa2trfcyYMb527VrfunWrn3DCCb537153d//+97/v3/rWt9zd/aSTTvKlS5f6xo0bffTo0b5lyxavr6/3Y4891i+//HJ3dz///PP92WefdXf3NWvW+OGHH56MecqUKV5TU5N8ftxxx3lDQ4OvWLHCKysr/bHHHnN393nz5vmDDz7oDQ0NPnPmTN+yZYu7uy9YsMA/97nPubv7tm3bkr/LjTfe6Lfccou7u3/84x/3u+++293db731Vu/bt2/n/mLyL22+oDpXGt4QVHB6d62Ck+i/AU0RFxHJtokTJ3LNNddw/fXXc+aZZzJw4ECGDx+e3M27f//+ad83d+5cKisrqays5JRTTmHJkiXMmzePyy+/nC1btvDAAw9w9tlntzkUdOqppya3dBg/fjxr1qxh586dvPHGGxx33HEANDQ0MHPmzGbvW7x4MSeffDJDhgwB4LzzzuPvf/87EN8u4o033kieu3v3bvbs2QPAWWedRWVlZfK1008/nfLyciZOnEg0GmX27NnJe7J69WrefvttVq5cycc+9jEAotEow4cPB2DlypV84xvfYOfOnezduze5mOHzzz+f3PH8oosu2mcV5+5GCU4anqUhqtiO1cnHSnBERLLr0EMPZfny5Tz22GPccMMNzJo1C7O0+y420/KcxPOLLrqIe++9lwULFrS7V1SvlO+HcDhMJBLB3fnYxz7Gfffd16HPT4jFYrz44ovNEpmEvn37pv38xE7qiWuGQqFkLBMmTODFF1/c51qXXHIJDz30EEcddRR33XUXCxcubDe27kg9OGnEsjRN3IM9qKz3QKx39jdvExHpyTZu3EifPn248MILueaaa1i0aBEbN25k6dKlAOzZs4dIJLLP+x5++GHq6urYvn07CxcuTFZ8LrnkEm6++WYAJkyY0OF4ZsyYwfPPP8+qVauA+A7iiepMwvTp01m4cCHbt2+nsbGR3//+98nXZs2axa233pp8vmLFig7HkHDYYYexdevWZILT2NjI66+/DsTvy/Dhw2lsbOTee+9Nvue4445jwYIFAM2Od1eq4KThWVroL7mLuBqMRUSy7rXXXuPaa69NVjFuu+023J0rrriC2tpaKisreeqpp/Z53zHHHMOcOXNYu3YtN910EyNGjABg6NChHHHEEcybN69T8QwZMoS77rqL888/n/rge+Q73/kOhx56aPKc4cOHM3/+fGbOnMnw4cOZMmUK0WgUgFtuuYXLL7+cSZMmEYlEOPHEE7n99ts7FUtFRQX3338/V155Jbt27SISiXD11VczYcIEvv3tbzN9+nQOOuggJk6cmBwG+9nPfsZnPvMZfvazn3H22Wd36nOLibl7oWPokKqqKk+3lkC2uDvrPnMhuNP/E/MYeN65nb7W3v8+Ca/ZTtn4uVSe9t0sRikiIp0xf/58+vXrxzXXXLPPazU1NUycOJGXXnop2V8j3ULacTUNUbXgjY0QJH1d2WjT6/fgNdsB9d+IiBS7p556isMPP5wrrrhCyU2J0BBVC8mNNoFQRedXMk4MT4ESHBGRYjF//vy0xz/60Y+ydu3aZseeeOKJfWYSjRs3jgcffDBX4UkWKcFpIdF/A13rwWk+RVw9OCIi3c1pp52WnEIt3Y+GqFrwhobk466sgxPb8V7ycWjgmC7FJCIiIh2T0wTHzGab2dtmtsrM9tlUxOJuCV5/1cym5DKeTCS2aYDsVHBsv+FYWdf3tBIREZHM5SzBMbMw8HPgdGA8cL6ZjW9x2unAIcHPZcBtuYonU6lDVKEubLaZ6MHR8JSIiEj+5bIH5xhglbu/C2BmC4C5wBsp58wF7vH4XPVFZjbQzIa7+6YcxpXWrt/cTO3yl2nY2Jg8Vr/4P4m907lbFNseX+hJDcYiIiL5l8sEZySwLuX5emB6BueMBPKe4DSuX0vDhsZmx3zLEqK7G1p5R2ZC+3+oS+8XEZGuWbZsGffccw+33HJLVq63YsUKNm7cyBlnnJGV63XGv//7v/P1r3+9Q++56667WLZsWbPVknOtM3FmSy4TnHQL77RcVTCTczCzy4gPYTFmTG4adsOD9ifcvymvKj8wRvmYg7t0zVD/kZQf8fGuhiYiUtQ+uPseGlavaf/EDqgYexCDLv5sVq5VVVVFVVVVVq4F8QRn2bJlaROcSCTS5iad2VLIxKEjSjXBWQ+MTnk+CtjYiXNw9zuAOyC+knF2w4wb9MVvMOiLubiyiIjkQnV1Neeeey7r168nGo1y00038aEPfYirrrqK6upqevXqxdNPP83y5cv50Y9+xKOPPsr8+fN555132LBhA+vWreO6667ji1/8IhdddBHnnHMOc+fOBeCCCy7gvPPO46yzzmr2mQ0NDfzrv/4rtbW1PPfcc9xwww28+eabbNy4kdWrVzN48GBmzZrVrFJy5plncs0113DyySfTr18/rrrqKh599FEqKyt5+OGHGTp0KJs3b+ZLX/oS7777LgC33XYbxx57LPPmzWPdunXU1dVx1VVXcdlll/G1r32N2tpaJk+ezIQJE7j33nv59a9/zS233EJDQwPTp0/nv/7rvwiHw9x5551873vfY/jw4Rx66KHNNglt6ZJLLqF///4sW7aM999/nx/84Aecc845APzwhz/kd7/7HfX19XziE5/gW9/6Fj/4wQ/o3bs3V155Jf/8z//MK6+8wl/+8heefvpp7rzzTkaNGrVPnD/5yU+SG5leeumlXH311axevZrTTz+d448/nhdeeIGRI0fy8MMPp910tEPcPSc/xJOnd4FxQAXwCjChxTlzgD8Tr+TMAJa0d92pU6e6iIjI/fff75deemny+c6dO33cuHG+ZMkSd3fftWuXNzY2+jPPPONz5sxxd/dvfvObPmnSJK+pqfGtW7f6qFGjfMOGDb5w4UKfO3du8jpjx471xsbGtJ975513+uWXX558/s1vftOnTJniNTU1aV+fM2eOP/PMM+7uDvgjjzzi7u7XXnutf/vb33Z393PPPdd/+tOfurt7JBLxnTt3urv79u3b3d29pqbGJ0yY4Nu2bXN39759+yav/8Ybb/iZZ57pDQ0N7u7+5S9/2e+++27fuHGjjx492rds2eL19fV+7LHHNourpYsvvtjPOeccj0aj/vrrr/vBBx/s7u5PPPGEf/GLX/RYLObRaNTnzJnjf/3rX/3FF1/0c845x93djz/+eJ82bZo3NDT4/Pnz/fbbb98nzmXLlvmRRx7pe/fu9T179vj48eP9pZde8vfee8/D4bC//PLL7u7+qU99yv/3f/+31TjTSJsv5GwWlbtHgK8CTwBvAr9z99fN7Etm9qXgtMeCJGgV8AvgK7mKR0RESsvEiRN56qmnuP7663n22WdZu3Ytw4cPT+4O3r9//7TDRXPnzqWyspLBgwdzyimnsGTJEk466SRWrVrFli1buO+++zj77LM7NNR01llnZVRxqKio4MwzzwRg6tSprF69GoC//OUvfPnLXwYgHA4nt4u45ZZbOOqoo5gxYwbr1q3jH//4xz7XTFSppk2bxuTJk3n66ad59913Wbx4MSeffDJDhgyhoqKC8847r9345s2bRygUYvz48WzevBmAJ598kieffJKjjz6aKVOm8NZbb/GPf/yDqVOnsnz5cvbs2UOvXr2YOXMmy5Yt49lnn+WEE07Y59rPPfccn/jEJ+jbty/9+vXjk5/8JM8++ywQXyF68uTJ+9yXrsjpQKG7P0Y8iUk9dnvKYwcuz2UMIiJSmg499FCWL1/OY489xg033MCsWbMwS7vvYjMtz0k8v+iii7j33ntZsGBBchglU3379k0+LisrIxaLJZ/XpWwBVF5envy8cDhMJBJp9ZoLFy7kqaee4sUXX6RPnz6cfPLJza6V4O5cfPHFfO9732t2/KGHHsrofqRKHcLyYF9Gd+eGG27gn/7pn/Y5f+zYsdx5550ce+yxTJo0iWeeeYZ33nmHI444Im2cmXxuOBymtra2Q3Gno5WMRUSkW9q4cSN9+vThwgsv5JprrmHRokVs3LiRpUuXArBnz560CcTDDz9MXV0d27dvZ+HChcmKzyWXXMLNN98MwIQJE1r93P322489e/a0+vrYsWNZsWIFsViMdevWsWTJknZ/l1NPPZXbbosvBReNRtm9eze7du1i0KBB9OnTh7feeotFixYlzy8vL6exsTH53vvvv58tW7YAsGPHDtasWcP06dNZuHAh27dvp7Gxkd///vftxpHOaaedxq9+9Sv27t0LwIYNG5KfdeKJJ/KjH/2IE088kRNOOIHbb7+dyZMnJxOr1DhPPPFEHnroIWpqaqiurubBBx9MW+nJFu1FJSIi3dJrr73GtddeSygUory8nNtuuw1354orrqC2tpbKykqeeuqpfd53zDHHMGfOHNauXctNN93EiBEjABg6dChHHHEE8+bNa/NzTznlFL7//e8zefJkbrjhhn1eP+644xg3bhwTJ07kyCOPZMqU9hfp/9nPfsZll13G//zP/xAOh7ntttuYPXs2t99+O5MmTeKwww5jxowZyfMvu+wyJk2axJQpU7j33nv5zne+w6xZs4jFYpSXl/Pzn/+cGTNmMH/+fGbOnMnw4cOZMmUK0Wi03VhamjVrFm+++SYzZ84EoF+/fvz617/mwAMP5IQTTuC73/0uM2fOpG/fvvTu3btZ0tIyzksuuYRjjjkGiDcZH3300VkZjkrH2ioZFaOqqipftmxZocMQEZFuaP78+fTr149rrrlmn9dqamqYOHEiL730UrIHRrqFtONwGqISEZEe76mnnuLwww/niiuuUHJTIlTBERERSeOJJ57g+uuvb3Zs3LhxPPjggwWKKDu++93v7tOP86lPfYobb7yxQBF1WdoKjhIcERER6c40RCUiIiI9gxIcERERKTlKcERERKTkKMERERGRkqMER0REREqOEhwREREpOd1umriZbQXW5Ojyg4FtObp2qdA9yozuU/t0j9qne5QZ3af2lfI92ubus1se7HYJTi6Z2TJ3ryp0HMVM9ygzuk/t0z1qn+5RZnSf2tcT75GGqERERKTkKMERERGRkqMEp7k7Ch1AN6B7lBndp/bpHrVP9ygzuk/t63H3SD04IiIiUnJUwREREZGSowRHRERESo4SHMDMZpvZ22a2ysy+Vuh4CsnMRpvZM2b2ppm9bmZXBcf3N7P/M7N/BH8OSnnPDcG9e9vMTitc9PllZmEze9nMHg2e6x6lMLOBZna/mb0V/PM0U/doX2b2z8G/ayvN7D4z693T75OZ/crMtpjZypRjHb4nZjbVzF4LXrvFzCzfv0sutXKffhj8O/eqmT1oZgNTXutZ98nde/QPEAbeAT4EVACvAOMLHVcB78dwYErweD/g78B44AfA14LjXwP+I3g8PrhnvYBxwb0MF/r3yNO9+hfgN8CjwXPdo+b3527g0uBxBTBQ92ifezQSeA+oDJ7/Drikp98n4ERgCrAy5ViH7wmwBJgJGPBn4PRC/255uE+zgLLg8X/05PukCg4cA6xy93fdvQFYAMwtcEwF4+6b3P2l4PEe4E3i/xGeS/wLi+DPecHjucACd6939/eAVcTvaUkzs1HAHOCXKYd1jwJm1p/4f3z/B8DdG9x9J7pH6ZQBlWZWBvQBNtLD75O7/w3Y0eJwh+6JmQ0H+rv7ix7/Fr8n5T0lId19cvcn3T0SPF0EjAoe97j7pAQn/uW9LuX5+uBYj2dmY4GjgcXAUHffBPEkCDgwOK2n3r+bgeuAWMox3aMmHwK2AncGw3i/NLO+6B414+4bgB8Ba4FNwC53fxLdp3Q6ek9GBo9bHu9JPk+8IgM98D4pwYmX5Frq8XPnzawf8Afganff3dapaY6V9P0zszOBLe6+PNO3pDlW0veIeFViCnCbux8NVBMfVmhNT7xHBH0kc4kPGYwA+prZhW29Jc2xkr9P7WjtnvToe2VmNwIR4N7EoTSnlfR9UoITz1ZHpzwfRbxE3GOZWTnx5OZed38gOLw5KGUS/LklON4T799xwFlmtpr4kOZHzOzX6B6lWg+sd/fFwfP7iSc8ukfNfRR4z923unsj8ABwLLpP6XT0nqynaXgm9XjJM7OLgTOBC4JhJ+iB90kJDiwFDjGzcWZWAXwaeKTAMRVM0D3/P8Cb7v6TlJceAS4OHl8MPJxy/NNm1svMxgGHEG9YK1nufoO7j3L3scT/efmLu1+I7lGSu78PrDOzw4JDpwJvoHvU0lpghpn1Cf7dO5V435vu0746dE+CYaw9ZjYjuLefTXlPyTKz2cD1wFnuXpPyUs+7T4Xuci6GH+AM4rOF3gFuLHQ8Bb4XxxMvT74KrAh+zgAOAJ4G/hH8uX/Ke24M7t3blEj3fQfu18k0zaLSPWp+byYDy4J/lh4CBukepb1P3wLeAlYC/0t8lkuPvk/AfcR7khqJVxi+0Jl7AlQF9/Ud4FaC1ftL5aeV+7SKeK9N4r/ft/fU+6StGkRERKTkaIhKRERESo4SHBERESk5SnBERESk5CjBERERkZKjBEdERERKjhIckQIyMzezH6c8v8bM5mfp2neZ2TnZuFY7n/OpYLfwZ1ocH2tmtWa2IuWnIoufe4mZjUh5/kszG5+l625tEXeHrxv8/ivbP7PV919tZn1Snj+WujO0iLRNCY5IYdUDnzSzwYUOJJWZhTtw+heAr7j7KWlee8fdJ6f8NGQpRIjvup1McNz9Und/I0vX/m2LuLN13Y64mvjmmwC4+xke37BURDKgBEeksCLAHcA/t3yhZQXGzPYGf55sZn81s9+Z2d/N7PtmdoGZLTGz18zs4JTLfNTMng3OOzN4f9jMfmhmS83sVTP7p5TrPmNmvwFeSxPP+cH1V5rZfwTH/pX44pC3m9kPM/mFE79H8PgcM7sr5fe9xcxeMLN3W/zu1wWf/Urw+55DfHGye4MKS6WZLTSzqtZiTXy2mX03uM4iMxuaSczBe39rZmekPL/LzM4OKjXPmtlLwc+xad57iZndmvL8UTM7OXh8m5ktM7PXzexbwbEriSdvzyQqY2a2OpEIm9m/BL/bSjO7Ojg2Nqik/SK41pNmVpm4npm9Efx9L8j0dxbp1gq90qB+9NOTf4C9QH9gNTAAuAaYH7x2F3BO6rnBnycDO4HhxFe93QB8K3jtKuDmlPc/Tvx/ZA4hvtJpb+Ay4BvBOb2IrzY8LrhuNTAuTZwjiG8rMIT4Rpp/AeYFry0EqtK8ZyxQS9OKqj9P/T2Cx+cAd6XE+/sg3vHAquD46cALQJ/g+f7pPjfxvJ1YHfh48PgHifvQIu5LiO+EviLlpxL4BHB3cE4F8dViK4lXWXoHxw8BlqX8/itTrnlrymc8Cpzc4vcJB7/DpOD5amBwyntWA4OBqcQT0L5AP+B14Ojg8yLA5OD83wEXBo83Ar2CxwML/c+9fvSTjx9VcEQKzOO7td8DXNmBty11903uXk98efUng+OvEf+iS/idu8fc/R/Au8DhwCzgs2a2AlhMfAn8Q4Lzl7j7e2k+bxqw0OObQiZ2KD4xgzhTh6guz+D8h4J43wAS1ZWPAnd6sK+Ou+9o5xptxdpAPLkAWE7ze5Wq5RBVLfBn4hur9iKedP0tOF4O/MLMXiOeoHW0X+dcM3sJeBmYkMH7jwcedPdqd99LfIPOE4LX3nP3FWl+v1eJV7suJJ4EiZQ8JTgixeFm4r0sfVOORQj+HTUzI141SKhPeRxLeR4jXrVIaLkXiwMGXJHy5T3O3RMJUnUr8VmGv0cmUmPq3eK11N/LUv7syJ4ybcXa6O6Ja0Vpfq/a5O51xCsspwHnEd9JHuLDi5uBo4hXkNI1Uif/LgO9ASy+6eE1wKnuPgn4E/vek5ba+v1S71/q7zcH+Dnx6s9yM8v49xbprpTgiBSBoCrxO+JJTsJq4l9IAHOJVwo66lNmFgr6cj5EfJO9J4Avm1k5gJkdamZ927oI8UrPSWY2OGhAPh/4ayfiAdhsZkeYWYj4sE97ngQ+n5hRZGb7B8f3APvlONaWFgCfI14xeSI4NgDY5O4x4CLiQ00trQYmB38Xo4FjguP9iSeVu4J+oNNT3tPa7/c3YJ7FdyDvS/wePttawMF9Hu3uzwDXAQOJD22JlDRl8SLF48fAV1Oe/wJ42MyWEN89ubXqSlveJv7lPhT4krvXmdkviQ9dvBRUhrYC89q6iLtvMrMbgGeIVxAec/eHOxEPwNeIDxOtI76DcZtftu7+uJlNBpaZWQPwGPB14j07t5tZLTAzy7GeZ2bHpzz/iru/QDzZugd4xJtmhP0X8Acz+1Twmen+np4H3iM+hLgSeCmI9RUze5l4H827wXkJdwB/NrNNnjJDzd1fChqzlwSHfunuL5vZ2FZ+lzDwazMbQPx+/NQ1G0t6AO0mLiIiIiVHQ1QiIiJScpTgiIiISMlRgiMiIiIlRwmOiIiIlBwlOCIiIlJylOCIiIhIyVGCIyIiIiXn/wMd3uavWxi5wgAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "fig, ax = profile_plot(\n", + " problems=problems,\n", + " results=results,\n", + " runtime_measure=\"n_evaluations\",\n", + " stopping_criterion=\"x\",\n", + " x_precision=0.001,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "70744c8b", + "metadata": {}, + "source": [ + "## Convergence plots\n", + "\n", + "**Convergence Plots** look at particular problems and show the convergence of each optimizer on each problem. They are similar to what you know from the dashboard." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "df3dc55b", + "metadata": { + "scrolled": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1gAAAEYCAYAAABBWFftAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAByD0lEQVR4nO3dd3wc5bX/8c/ZVXU32AbbFBuCqSYU00tooYNpwUmAhCSEm1xqciFAuASn3eQmhAuE/CCQACEhQOg1FBNMN2AbG4wJ3eAGLuAmWWV3z++PmZVG0q60kla7K+n7fr32pd2Z2ZkzWnuPnnme54y5OyIiIiIiItJ9sWIHICIiIiIi0leogSUiIiIiIpInamCJiIiIiIjkiRpYIiIiIiIieaIGloiIiIiISJ6ogSUiIiIiIpInamCJiIiI9FFmNtXM/laA45xuZs/39HG6olC/A5E0NbBECsTMFpjZejNbZ2afm9kjZrZpDxznN2a20MzWmNlHZnZpZN04M/MwhnVm9qmZPWxmX853HCIi/VGr7/pPzOwWMxtU7LhKjZltZWZ3mNnyMF+9a2a/N7NNihDLj83sw/AzW2Rmd0bWTTezM7q5fzXw+hk1sEQK6xh3HwSMBj4Fft8Dx/gzsI27DwH2Br5uZie02mZYGMcXgSeB+8zs9B6IRUSkP0p/1+8E7AxcUtxwcmNmZQU6zheAl4ElwM5hvtoHeB/YtxAxRGL5JnAacEj4mU0Cnsrj/gvyO5XSogaWSBG4ex1wN7BdepmZDTWzW8OreR+Z2X+bWczMNgivqB0TbjfIzN4zs29k2ffb7l4TWZQCvpBl20/c/WpgKvC/ZqbvBBGRPHH3T4DHCRpaAJjZsWb2ppmtCntHto2su8jMFpvZWjN728wODpfHzOxiM3vfzFaa2T/MbINwXXpkwjfN7GMzWxEduRCqMrM7w/3ONrMvRo65IDzu60CNmZV1EOOmZnZvmKtWmtm1mc7dzH5rZs+b2dAMq6cCL7j7D919Ufi7WubuV7n7HZF9fDfMd5+Z2YNmNiaybm8ze9XMVoc/946sG29mz4Tn+yQwop2PaTfgcXd/P4zjE3e/IdzPL4H9gGvD3q1rw+VXR0aKzDKz/SLHnmpmd5vZ38xsDfA94MfAlHAfc9uJRfoI/TElUgRmNgCYAsyILP49MBTYAvgS8A3gW+7+GfBt4EYzGwX8HzDH3W9tZ/8Xm9k6YBEwEPh7ByHdC4wCtu7aGYmISGvhcLcjgPfC1xOA24HzgZHAo8BDZlZhZlsDZwO7uftg4DBgQbirc4HjCHLDGOBz4A+tDrcvwXf4wcBPoo0iYDJwF7ABQT6438zKI+u/BhwFDCPIQdlijAMPAx8B44CxwB2R/aQbgzcCOwKHuvvqDL+aQ4B7Mv3OIvs5CPgVcDLBqI+P0scKG5ePANcAGwJXAo+Y2Ybh2/8OzCJoWP0c+GY7h5oBfMPMLjSzSeE5AuDulwLPAWe7+yB3Pztc9SpBozn9+7zLzKoi+5xMcBF1GMGokv8B7gz38UWkz1MDS6Sw7jezVcAa4MvAbwHCL/QpwCXuvtbdFwC/Ixi2gLs/QZAcnyJIgv/R3kHc/dfAYGAX4K9ApgQXtST8uUGnz0hERFq738zWAguBZcDl4fIpwCPu/qS7NwJXANUEw7mTQCWwnZmVu/uCdK8KwXf+pe6+yN3rCXqATmo1/Oyn7r7e3ecCcwmGgKfNcve7w2NeCVQBe0bWX+PuC919fQcx7k7QwLvQ3Wvcvc7do4UtygkaZxsQDJOszfL7GQF8kn5hZmeHvWXrwsYZwCnATe4+OzznS4C9zGwcQR58193/6u4Jd78d+DdwjJltRtArdZm717v7s8BDWeLA3f8GnEPQoH0GWGZmF2fbPv0ed18ZHvt3BJ9b9ALlS+5+v7unwt+p9DNqYIkU1nHuPozgy/hs4Bkz25gg2VQQXKFL+4jg6mDaDcAOwM3uvrKjA3ngNWA98NMONk8f57NcTkJERNp1XNgLdQCwDc1D1MYQ+Z539xRBI2ysu79H0Gs0leCP/DsiQ+I2J5gruyq8SPcWQYNso8gxP4k8rwWihTUWtjrmojCWNuvbixHYFPjI3RNZzvsLBL03P3X3hizbAKwk6JVKH+PaMDdeRdBIyxTHuvB9Y1uvC30UWfd5q6Hyrbdtwd1vc/dDCHqcvgf8zMwOy7a9mf2Xmb0VDk9cRTD6JDoMcWHmd0p/oQaWSBG4e9Ld7yVIkPsCK4BGgiSathmwGJp6uP4I3Ap834IJwrkqA7bsYJvjCa6yvt2J/YqISDvc/RngFoJeIAhGCzR9z5uZETRaFofb/93d9w23ceB/w00XAke4+7DIo8rdF+cYSlPF2nCu7SY0j1wgPFZaezEuBDaz7IUb3gK+BfwzHPKYzVNA6+JLrbWOYyDBcMDFrdeF0jlzKTA83D66rkPu3ujudwGvE1zQhJa/G8L5VhcRDF0cHjYMVwMW3VXrXedyfOk71MASKQILTAaGA2+5exL4B/BLMxtsZpsDPwTSZV1/HP78NkGivjU6Tjyy35iZ/YeZDQ+PsTtwFlkqIpnZRmZ2NsHwlUvCK5UiIpI/VwFfNrOdCL7njzKzg8M5UP8F1AMvmtnWZnaQmVUCdQSjD5LhPq4nyA+bA5jZyDCH5GpXMzshbBidHx5zRpZts8YIvELQgPm1mQ00syoz2yf65nC43o+BaWaW7eLeVGA/M7vSzMaG5zQCiM4b+zvwLTPbKfyd/A/wcjiE/lFggpl93YKiHFMIikY97O4fATOBn4bzxvYFjsn2i7Hg/l1Hhbk3ZmZHANsTVDmEoOLvFpG3DAYSwHKgzMx+AgzJtv/IPsaZCkn1G/qgRQrrobD4xBrgl8A33f3NcN05QA3wAfA8QXK5ycx2JWhsfSNsiP0vwdWwbGPEjycodbuWoIH2e9qWg19lZjXAG8CRwFfc/ab8nKKIiKS5+3KC0QeXufvbwKkE38krCP7wPyYcTlcJ/Dpc/glB4aH0xbWrgQeBJ8K5XTOAPToRxgMEc6s+J5jbe0I4vypTvFljDHPQMQRDAT8mGGo4JcM+/gL8DPhXOGeq9fp3COaAbQLMDc/pBYKeqcvCbZ4Kn99D0KjbEvhquG4lcDRB428l8CPgaHdfER7i6wS/n88ILiBmLQpFkI9/HJ7PKuA3wPcjc8uuJpjv9rmZXUNQFfKfwDsEQw/r6HhI4F3hz5VmNruDbaUPMHf1WoqIiIiIiOSDerBERERERETyRA0sERERERGRPFEDS0REREREJE/UwBIREREREcmTbPcx6DMOP/xwf+yxx4odhoiIdI51vEnfoDwlItJrZcxVfb4Ha8WKFR1vJCIiUiTKUyIifUufb2CJiIiIiIgUihpYIiIiIiIieaIGloiIiIiISJ6ogSUiIiIiIpInamCJiIiIiIjkSck0sMzsJjNbZmbzsqw3M7vGzN4zs9fNbJdCxygiIv2bcpWIiHSkZBpYwC3A4e2sPwLYKnycCVxXgJhERESibkG5SkRE2lEyNxp292fNbFw7m0wGbnV3B2aY2TAzG+3uS3sqpnfPPoWKNY0YECsrx8rjDD35RAYdPLmnDikiIiWs1HLVkleep/Haa5rjI0Z84AbE4kF6jw8ZwgZnnkHFuPZCFhGRfCqlHqyOjAUWRl4vCpe1YWZnmtlMM5u5fPnyLh8wXp/AGsqgoYxUrZNcnWDN/Xd1eX8iItLn5ZSr8pWnUo31WENZ0yPWEMM/X0VyxQqSK1bQ8MEHrPvX013ev4iIdF5vamBZhmWeaUN3v8HdJ7n7pJEjR3b5gCs23JDPhkNtdBeNiS7vT0TyZ+bMmZx77rl529/06dM5+uijAZg6dSpXXHFFxu0uvPBCtt9+ey688MJOH2PQoEHdilF6hZxyVb7yVOWGG1G3cRnrNy6jYsRnVIz4jPrNhjLwS/tjlZXBsRoaurx/Eeke5ar+qWSGCOZgEbBp5PUmwJKePOCT23+NV95azPjRw7io4h80Lq7DU6mePKSI5GjSpElMmjSp4Mf94x//yPLly6kM/3gVaaWguWrkNjsw8qpbqa1vZN21e1Ada2DlqC+y4Snfo+7N+STr60F5S6RolKv6p97UwHoQONvM7gD2AFb35PwrgPKyoIOvoTGJxeLBQk/25CFFSsqfHp7Nh0s/z+s+x48ezhlHZy+sVlNTw8knn8yiRYtIJpNcdtllbLHFFpx33nnU1NRQWVnJU089xaxZs7jiiit4+OGHmTp1Ku+//z6LFy9m4cKF/OhHP+K73/0up512GieddBKTJwfzJk855RSmTJnCscce22Gcc+fO5aCDDmqxv2OPPZaamhr22GMPLrnkEiZNmsQpp5xCMpnkiCOO4Morr2TdunUsXbqUKVOmsGbNGhKJBNdddx377bcfAP/1X//F008/zfDhw7njjjvoTu+FlKSC5yqAuBm1Xkk1DcQTNQBYLMhhujAofVkx8hQoV0n7OmxgmdnRwM+BzcPtDXB3H5LPQMzsduAAYISZLQIuB8oJDnY98ChwJPAeUAt8K5/Hz6SyPGhUNSSSEAt/VUpUIj3qscceY8yYMTzyyCMArF69mp133pk777yT3XbbjTVr1lBdXd3mfa+//jozZsygpqaGnXfemaOOOoozzjiD//u//2Py5MmsXr2aF198kb/85S85xZFpfw8++CCDBg1izpw5ABx99NGcd955fO1rX+P6669veu/f//53DjvsMC699FKSySS1tbVAkJB32WUXfve73/Gzn/2Mn/70p1x77bXd/I0J9O9cBRCLGbVexYasJZ4MGljEwwuDSV0YFMk35SppTy49WFcBJwBvhFWReoS7f62D9Q6c1VPHz6S8LGxgNSahqQcLPNmIxcsLGYpIUXR0Ba8nTJw4kQsuuICLLrqIo48+mmHDhjF69Gh22203AIYMyfz38uTJk6murqa6upoDDzyQV155heOOO46zzjqLZcuWce+993LiiSdSVpZbx322/UW99NJL3H///QB8/etf54ILLgBgt91249vf/jaNjY0cd9xx7LTTTgDEYjGmTJkCwKmnnsoJJ5zQyd+OtOMq+mmugnQDKxgKVJaoTS8MYtKFQenDipGnQLlK2pdLkYuFwLyeTFilqiLSg2WxyD/0xtoiRSTS902YMIFZs2YxceJELrnkEu677z7MMtUNaKn1NunXp512Grfddhs333wz3/pW7p0J2faXi/33359nn32WsWPHctppp3HrrbfmdAzpln6bqwBiZtSmqgAoSwY5Kj1EkFS//JWI9CjlKmlPLg2sHwGPmtklZvbD9KOnAysFFWEPVmMi1TzUAvAGNbBEesqSJUsYMGAAp556KhdccAEzZsxgyZIlvPrqqwCsXbuWRKJtNc8HHniAuro6Vq5cyfTp05uuIp5++ulcddVVAGy//fY5x5Ftf1F77rkn99xzDwB33HFH0/KPPvqIUaNG8d3vfpfvfOc7zJ49G4BUKsXdd98NBEMz9t1335zjkQ7121wFwR9A6z1oYJU3DRFMN7A0RFAk35SrpD259D/+ElgHVAEVPRtOaUk3sBLJFFQ0/6q8oaZYIYn0eW+88QYXXnghsViM8vJyrrvuOtydc845h/Xr11NdXc20adPavG/33XfnqKOO4uOPP+ayyy5jzJgxAGy00UZsu+22bYZMdCTb/qKuuuoqTj31VH73u99x1FFHMXToUCAoo/vb3/6W8vJyBg0a1HRVcODAgbz55pvsuuuuDB06lDvvvLOTvx1pR7/NVWnrCRtYqZY9WBoiKJJ/ylXSHutoNIWZzXT3wteXzJNJkyb5zJkzu/Tee599i788NheAP1a+TsNbC4hVr2f0//yU+Ogd8xmmiHTD1KlTGTRoUNO48qja2lomTpzI7Nmzm5JKvtTW1lJdXY2Zcccdd3D77bfzwAMP5PUY/VinxqT05lzVnTwVdfuvvsfRVc+TIsaQ8+fy6X//hIb336fqizsy6pKL8xCpiHSHclWflDFX5dKDNc3MDnX3J/IcUMlL92ABuKWfm3qwRHqJadOm8e1vf5sf/vCHeU9YALNmzeLss8/G3Rk2bBg33XRT3o8hOeu3uSqtLhwiGCMFjeubh7arB0ukpClX9T25NLDOAn5kZvVAIz1U+rYUpYtcAKRikcaWilyIlJSpU6dmXH7IIYfw8ccft1j2+OOPc9FFF7VYNn78eO67775OH3e//fZj7ty5nX6f9Ih+m6vS6qy5JLQ3rG0eIphUA0ukFChX9R8dNrDcfXAhAilF0R6sFM1l2lEPlkivddhhh3HYYYcVOwzJs/6cq9LqwjlYAF63tqlMu3qwRHof5areLZcbDe+fabm7P5v/cEpLebSBZfGmQZbqwRIRKS39OVel1RHpwapfg8VV5EJEpBhyGSJ4YeR5FbA7MAs4qEciKiGV5S0bWMErU5l2EZHS029zVVp0iCAN65p7sJIq0y4iUki5DBE8JvrazDYFftNjEZWQ8rLm24SlCBtYDjRqiKCISCnpz7kqrT46B6s+MkTQ1YMlIlJIudxouLVFwA75DqQUVZQ3tz9TNJezVw+WiEjJ6ze5Kq2hVQPLwuJMGiIoIlJYHTawzOz3ZnZN+LgWeA7oF6VIKqI9WNHbhakHS6ToZs6cybnnnlvUGA444AAy3b/olltu4eyzzy5CRB0bNGhQsUPoEf05V6XVxwY0PW/Rg6UqgiJFo1zVNb09V+UyByv6iSSA2939hR6Kp6REy7QnIw0s3QdLpPgmTZrEpEm98r6ybSQSCcrKcvk6lnb021yVlopV0Ohxyi0JkQaWerBEike5qn/KZQ7WXwoRSClqUabdwxaWm6oISr9RN/3XpJb/O6/7jI3chqoDLs66vqamhpNPPplFixaRTCa57LLL2GKLLTjvvPOoqamhsrKSp556ilmzZnHFFVfw8MMPM3XqVN5//30WL17MwoUL+dGPfsR3v/tdTjvtNE466SQmT54MwCmnnMKUKVM49thj2xz3lltu4cEHH6S2tpb333+f448/nt/8JpjC88QTT3D55ZdTX1/Plltuyc0339zm6trNN9/Mr371K0aPHs2ECROorKwEYPny5Xzve99rusfJVVddxT777MPUqVNZsmQJCxYsYMSIEUyYMIEPP/yQpUuX8s4773DllVcyY8YM/vnPfzJ27FgeeughysvLmTVrFj/84Q9Zt24dI0aM4JZbbmH06NHceOON3HDDDTQ0NPCFL3yBv/71rwwYMIAPP/yQr3/96yQSCQ4//PC8fIalqD/nqrSYGbWpKoZaTThEcHiwQg0s6cOKkadAuUq5qn25DBE82sxeM7PPzGyNma01szWFCK7YsvVgoTlYIj3mscceY8yYMcydO5d58+Zx+OGHM2XKFK6++mrmzp3LtGnTqK6ubvO+119/nUceeYSXXnqJn/3sZyxZsoQzzjiDm2++GYDVq1fz4osvcuSRR2Y99pw5c7jzzjt54403uPPOO1m4cCErVqzgF7/4BdOmTWP27NlMmjSJK6+8ssX7li5dyuWXX84LL7zAk08+yfz585vWnXfeefzgBz/g1Vdf5Z577uGMM85oWjdr1iweeOAB/v73vwPw/vvv88gjj/DAAw9w6qmncuCBB/LGG29QXV3NI488QmNjI+eccw533303s2bN4tvf/jaXXnopACeccAKvvvoqc+fOZdttt+XPf/5z0/G///3v8+qrr7Lxxht38VMpff05V6XFY0ZtKvhjyevXQjzMYaoiKJJ3ylXKVe3JpZ/vKuAE4A139w627VNa3Acr3YGFhghK/9HRFbyeMHHiRC644AIuuugijj76aIYNG8bo0aPZbbfdABgyZEjG902ePJnq6mqqq6s58MADeeWVVzjuuOM466yzWLZsGffeey8nnnhiu8MbDj74YIYOHQrAdtttx0cffcSqVauYP38+++yzDwANDQ3stddeLd738ssvc8ABBzBy5EgApkyZwjvvvAPAtGnTWiSxNWvWsHbtWgCOPfbYFgn4iCOOoLy8nIkTJ5JMJpuu4k2cOJEFCxbw9ttvM2/ePL785S8DkEwmGT16NADz5s3jv//7v1m1ahXr1q1rukHlCy+8wD333APAaaedxkUXXdT+B9B7XUU/zVVpsZhR68HNhoMeLA0RlL6vGHkKlKuUq9qXSwNrITCvPyasFvfB8uYWloYIivScCRMmMGvWLB599FEuueQSDj30UMysw/e13ib9+rTTTuO2227jjjvu4Kabbmp3H+mhEgDxeJxEIoG78+Uvf5nbb7+9U8dPS6VSvPTSSxmvZA4cODDj8WOxGOXl5U37jMViTbFsv/32vPTSS232dfrpp3P//ffzxS9+kVtuuYXp06d3GFsf029zVVrQwAp7sKL3wVIDSyTvlKuUq9qTS5n2HwGPmtklZvbD9KOnAysFZfEY6c+6uQiTgXqwRHrMkiVLGDBgAKeeeioXXHABM2bMYMmSJbz66qsArF27lkQi0eZ9DzzwAHV1daxcuZLp06c3XUU8/fTTueqqqwDYfvvtOx3PnnvuyQsvvMB7770HQG1tbdMVv7Q99tiD6dOns3LlShobG7nrrrua1h166KFce+21Ta/nzJnT6RjStt56a5YvX96UtBobG3nzzTeB4PcyevRoGhsbue2225res88++3DHHXcAtFjeB/XbXJUWs+YeLOrXqMiFSA9SrspOuSq3HqxfAuuAKqCiZ8MpLWZGeVmchsZkcw8W6sES6UlvvPEGF154YdOVseuuuw5355xzzmH9+vVUV1czbdq0Nu/bfffdOeqoo/j444+57LLLGDNmDAAbbbQR2267Lccdd1yX4hk5ciS33HILX/va16ivrwfgF7/4BRMmTGjaZvTo0UydOpW99tqL0aNHs8suu5AM571cc801nHXWWey4444kEgn2339/rr/++i7FUlFRwd133825557L6tWrSSQSnH/++Wy//fb8/Oc/Z4899mDzzTdn4sSJTUM7rr76ar7+9a9z9dVXc+KJJ3bpuL1Ev81VabEWc7DWNQ0RVA+WSP4pV2WnXAXW0WgKM5vp7r22vuSkSZM8U+3/XJ3y83tZt76BH8UWsOl7r2PljQzb5S0Gnf86Zl25T7OI5NvUqVMZNGgQF1xwQZt1tbW1TJw4kdmzZzeNWZdeoVNjRXpzrupunkq75Ian+OLy2zlqwAwoq6Jx4Fmse+JJYoMGscmfbshDpCLSHcpVfVLGXJVLC2GamR2a52B6jXQlwWS0ygWokqBILzBt2jS22WYbzjnnHCWsvq9f5ypIDxEM52Yk6iAW5H1XFUGRkqZc1ffkMkTwLOBHZlYPNBK01NzdM5dH6WMqyoI2aLKppy9MWI21WGXvvsu0SF8xderUjMsPOeSQpnt6pD3++ONtqhONHz+e++67r6fCk8Lo17kK0kMEq5oXpML5HxoiKFISlKv6j1xuNDy4EIGUqoqwTGay9UhKFboQ6ZUOO+ywprKw0nf091wFLasIAuBqYIn0VspVvVvWBpaZbePu/zazXTKtd/fZPRdW6SgvD3qwPvlsHRMiy1XoQkSk+JSrmsUj98ECmhpYqiIoIlJY7fVg/RA4E/hdhnUOHNQjEZWYyvLgV+TpOWzpqVjqwRIRKQXKVaGYGeuiDaxUY/hTDSwRkULK2sBy9zPDp0e4e110nZlVZXhLn7TvxE2Zv2A5bWotqsiFiEjRKVc1i5ZpByDVEPx0x9371E08RURKWS5VBF/McVmfdNReE9hmsxFNPVgeKXIhIiIlo1/nKmh1o2EAb2x+rl4sEZGCydrAMrONzWxXoNrMdjazXcLHAcCAQgVYCgYPiNyzUkMERUrCzJkzOffcc/O2vzlz5vDoo4/mbX9d8T//8z+dfs8tt9zC2Wef3QPRZNeVOHuKclWzeCzWsoGV7sECUKl2kaJQrgr0t1zVXg/WYcAVwCbAlQTj239HMN79xz0fWumIxQxvPbJCPVgiRTVp0iSuueaavO2vvaSVSCTydpz2lFLDpT0lFqdyVSgWM+q8glQ6YaWae7BU6EKkOJSriqeYcbY3B+svwF/M7ER3v6eAMZWcmFlzkYuQerCkP/j8L7fSsOCjvO6zYtzmDP/mN7Kur6mp4eSTT2bRokUkk0kuu+wytthiC8477zxqamqorKzkqaeeYtasWVxxxRU8/PDDTJ06lffff5/FixezcOFCfvSjH/Hd736X0047jZNOOonJkycDcMoppzBlyhSOPfbYFsdsaGjgJz/5CevXr+f555/nkksu4a233mLJkiUsWLCAESNGcOihhzJz5kyuvfZaAI4++mguuOACDjjgAAYNGsR5553Hww8/THV1NQ888AAbbbQRn376Kd/73vf44IMPALjuuuvYe++9Oe6441i4cCF1dXWcd955nHnmmVx88cWsX7+enXbaie23357bbruNv/3tb1xzzTU0NDSwxx578P/+3/8jHo9z880386tf/YrRo0czYcIEKisryeb0009nyJAhzJw5k08++YTf/OY3nHTSSQD89re/5R//+Af19fUcf/zx/PSnP+U3v/kNVVVVnHvuufzgBz9g7ty5/Otf/+Kpp57i5ptvZpNNNmkT55VXXslNN90EwBlnnMH555/PggULOOKII9h333158cUXGTt2LA888ADV1dVd/8eTgXJVs1gsyFVJ4sRIgEd6rdTAkj6qGHkKlKuUq9qXyxys6WZ2jZnNNrNZZna1mW3YraP2MvFYLFLkIj0HSw0skZ7w2GOPMWbMGObOncu8efM4/PDDmTJlCldffTVz585l2rRpGb/4Xn/9dR555BFeeuklfvazn7FkyRLOOOMMbr75ZgBWr17Niy++yJFHHtnmvRUVFfzsZz9jypQpzJkzhylTpgAwa9YsHnjgAf7+97+3G3NNTQ177rknc+fOZf/99+fGG28E4Nxzz+VLX/oSc+fOZfbs2Wy//fYA3HTTTcyaNYuZM2dyzTXXsHLlSn79619TXV3NnDlzuO2223jrrbe48847eeGFF5gzZw7xeJzbbruNpUuXcvnll/PCCy/w5JNPMn/+/A5/p0uXLuX555/n4Ycf5uKLLwbgiSee4N133+WVV15hzpw5zJo1i2effZb999+f5557DgiGtqxbt47Gxkaef/559ttvvzZxzpo1i5tvvpmXX36ZGTNmcOONN/Laa68B8O6773LWWWfx5ptvMmzYMO65p0fbP/0+V8XCIhaJpmunzQ0s9WCJ5JdylXJVezq80TBwB/AscGL4+hTgTuCQbh89wswOB64G4sCf3P3XrdYPBf4GbEYQ9xXufnM+Y8gmFov2XoXPVUVQ+oGOruD1hIkTJ3LBBRdw0UUXcfTRRzNs2DBGjx7NbrvtBsCQIUMyvm/y5MlUV1dTXV3NgQceyCuvvMJxxx3HWWedxbJly7j33ns58cQTKSvL5WsvcOyxx+Z0FauiooKjjz4agF133ZUnn3wSgH/961/ceuutAMTjcYYOHQrANddcw3333QfAwoULeffdd9lww5ZtgfSVz/R5r1+/nlGjRvHyyy9zwAEHMHLkSACmTJnCO++80258xx13HLFYjO22245PP/0UCJLWE088wc477wzAunXrePfdd/nGN77BrFmzWLt2LZWVleyyyy7MnDmT5557LuMwl+eff57jjz+egQMHAnDCCSfw3HPPceyxxzJ+/Hh22mmnpt/LggULOvxddkO/z1XxMFc1Eqca1IMl/UIx8hQoV6UpV2WWy6e3gbv/PPL6F2Z2XLePHGFmceAPwJeBRcCrZvagu0ebu2cB8939GDMbCbxtZre5e0OGXeZVPJZhiKDmYIn0iAkTJjBr1iweffRRLrnkEg499NCcyku33ib9+rTTTuO2227jjjvuaBoakKv0FzFAWVkZqcgfqXV1zRXBy8vLm44Xj8fbHQc/ffp0pk2bxksvvcSAAQM44IADWuwrzd355je/ya9+9asWy++///5Ol9uODstw96afl1xyCf/xH//RZvtx48Zx8803s/fee7Pjjjvy9NNP8/7777PttttmjDOX48bjcdavX9+puDup3+eq9MXAhJeF1wIjjSo1sETySrkqoFyVWS5DBJ82s6+aWSx8nAw8kmlDC2zahTh2B95z9w/CJHQHMLnVNg4MtuDTGgR8BhRkNl88WuQi/fmoB0ukRyxZsoQBAwZw6qmncsEFFzBjxgyWLFnCq6++CsDatWszJoUHHniAuro6Vq5cyfTp05uupp1++ulcddVVAE3DHjIZPHgwa9euzbp+3LhxzJkzh1QqxcKFC3nllVc6PJeDDz6Y6667DoBkMsmaNWtYvXo1w4cPZ8CAAfz73/9mxowZTduXl5fT2NjY9N67776bZcuWAfDZZ5/x0UcfscceezB9+nRWrlxJY2Mjd911V4dxZHLYYYdx0003sW7dOgAWL17cdKz999+fK664gv3335/99tuP66+/np122qkpWUbj3H///bn//vupra2lpqaG++67j/32269LMXVTTrmqG3kKSjxXNQ8RjIeRNB/WVUVQJK+Uq5Sr2pNLD9Z/EFRj+lv4OgbUmNkPAXf3pj5Qd3czux/YtZNxjAUWRl4vAvZotc21wIPAEmAwMMXdM16SM7MzgTMBNttss06G0lZwVTDdwgoT2AdPk1z5PvENt+z2/kWk2RtvvMGFF15ILBajvLyc6667DnfnnHPOYf369VRXVzNt2rQ279t999056qij+Pjjj7nssssYM2YMABtttBHbbrstxx13XLvHPfDAA/n1r3/NTjvtxCWXXNJm/T777MP48eOZOHEiO+ywA7vsskuH53L11Vdz5pln8uc//5l4PM51113H4YcfzvXXX8+OO+7I1ltvzZ577tm0/ZlnnsmOO+7ILrvswm233cYvfvELDj30UFKpFOXl5fzhD39gzz33ZOrUqey1116MHj2aXXbZhWQX/ng+9NBDeeutt9hrr70AGDRoEH/7298YNWoU++23H7/85S/Za6+9GDhwIFVVVS0SUes4Tz/9dHbffXcgmDi888479/RwwExyylXdyFOQx1yV7zwFzT1Yjd52DpZ6sETyS7lKuao91l6XWZd2aPYH4BZ3f7UT7/kKcJi7nxG+Pg3Y3d3PiWxzErAPQQLdEngS+KK7r2lv35MmTfKZM2d2/kQirn9gJrF/PsThq94Fg+G7vw5AbOMdGfi19icUikjPmzp1KoMGDeKCCy5os662tpaJEycye/bspnHl0it0bmxJZ3bchTwVvq9HclU+8hTAnx95jQdfeJtfbvBnNo8vpZE9WfdyMNpizNX/R9lGG3X7GCLSdcpVfVLGXNXuEEEzqzCzb5nZFWb22/B5RXvvAQ4EXjKz983sdTN7w8xe7+A9i4DokI1NCK7+RX0LuNcD7wEfAtt0sN+8aNmD1Sz16TyVaxcpYdOmTWObbbbhnHPOUcLqw7qQq7qSp6BX5Cpo9ExDBNWDJVKqlKv6nqxDBM1sO4JhDi8AswhaGAcAl5rZZHd/M8tbj+hCHK8CW5nZeGAx8FXg6622+Rg4GHjOzDYCtgY+6MKxOi0Ws+aBFg7Vx/6e9Q+eA54i+ck8yjZrPUJERApp6tSpGZcfcsghfPzxxy2WPf7441x00UUtlo0fP76pUlJv9ctf/rLNGPevfOUrXHrppUWKqDC6mKu6kqeg1HOVtWxgWbSKYOYR9SJSQMpV/SdXtTcH6/fA9939yehCMzuEYIz5gZne5O4fhduNAqpyCcLdE2Z2NvA4Qenbm9z9TTP7Xrj+euDnwC1m9gZBAr3I3Vfksv/uisdi0ZHsxEZ/sel5cukcNbBEepHDDjuMww47rNhh5N2ll17a5xJUjjqdq7qSp8L3lXSuaj0Hyz1B070bNQdLpFdRrurd2mtgjW2dsADcfZqZ/T7bm8zsWOB3wBhgGbA58BaQvSRKsN9HgUdbLbs+8nwJcGh7++gpQRXB5iGCVj0cG7Y5vuojkkteK0ZIIiIS6HSu6mqeCvdb0rkKWg8RLA+eq4qgiEjBtDcHK2Zmla0XmlkV7TfMfg7sCbzj7uMJhkq80K0oiyxmRotSIO7ER0wInq5eXJSYREQE6Fqu6nN5CiJl2puGCDY2r1QPlohIwbTXwLoVuMfMxqUXhM//Afy1nfc1uvtKgqQXc/engZ26H2rxBDcajnDHKgcFT1XkQkSkmLqSq/pcnoLIEMHwPlieau610hBBEZHCydoT5e6/CMeaP2tmA8LFNcAV7p51iCCwyswGAc8Ct5nZMgp0k8WeEosZHq0i6A4VwV2zvVENLBGRYulirupzeQqaG1iJ9H2woj1YqiIoIlIw7d5o2N2vBa41s8Hh66y3jjazzdz9Y4K72q8HfgCcAgwFfpa3iIsgHmvV0eeOhQ0sGmpx96a7RouISGHlmqv6cp6C5lyV7sGKDhFUD5aISOG028BKa69hFXE/sIu715jZPe5+IvCX7gRXKjL2YJUPTL+AxvVQMSDje0VEpDByyFX300fzFETnYIVVBFORTjk1sERECqbdGw13UrQLZ4s87rfogiqCza8dsEiDSvOwRER6hT6bp6DtHKwWQwRTqiIoIlIo+WxgeZbnvV4sFmt5QqkUVjGo+bXmYYmI9AZ9Nk9BpiqCzT1YGiIoIlI4HQ4RNLNy4PvA/uGiZ4Dr3aOXxgD4opmtIbhCWB0+J3zt7j4kTzEXXHBvkZZzrJrmYAFev67AEYmISFSOuarP5imI3gcrTO0WaUOqgSUiUjC5zMG6juBOhf8vfH1auOyM6Ebu6Tsb9j2Z7oNFtIHVWFvwmEREpIUOc1VfzlMQqSKYHiIYHdquBpaISMHk0sDazd2/GHn9LzOb21MBlaJ4vG2RCyuPFLXQHCwRkWLr97kq1roHK3ppUGXaRUQKJpc5WEkz2zL9wsy2APrVbNlY6xLs0TLtqMiFiEgJUK6ylj1YLVKXerBERAomlx6sC4GnzewDggEHmwPf7tGoSkw8FmtRRbDNEEE1sEREik25qqkHKz1EsLkHS0MERUQKJ5cG1vPAVsDWBEnr3x29wcw2AnYLX77i7su6HGEJaH0fLAdikQaWqgiKiBRdp3JVX8tTEFS8BUiQYYigyrSLiBRMLkMEX3L3end/3d3nuns98FK2jc3sZOAV4CvAycDLZnZSfsItjnjM2pRpp7y66aU3qMiFiEiR5Zyr+mKeguYhgs09WM3r1IMlIlI4WXuwzGxjYCxBKdudaf6qHgIMyPY+4FKCycbLwv2MBKYBd+cl4iIIrgq2KtNuMSgfAI21GiIoIlIkXcxVfS5PQaSKoMq0i4gUVXtDBA8DTgc2AX5Hc9JaA/y4nffFWg21WEl+b2hccG16sDx4ZRWDghLtGiIoIlIsXclVfS5PQWQOVrrIRXSlqgiKiBRM1gaWu/8F+IuZneju93Rin4+Z2ePA7eHrKcCj3Yix6GIxa1vkguBmw16jIhciIsXSxVzV5/IURHuwVORCRKSYOixy0cnGFe5+oZmdAOxLcAHtBne/r4vxlYR4rO19sACoCEafaA6WiEhxdSZX9cU8BZE5WGiIoIhIMeVSRbDT3P1eM3sW2A/4uCeOUUjxWMuRI+n2VdO9sBrWFTgiERHpjr6WpyBTD1bzOk+qiqCISKHkbcy5mT1sZjuEz0cDbxDcg+RWMzs/X8cphpi16sEKZ2RZedDA0hBBEZHS15fzFETnYAXXTlvMHnb1YImIFEqHDSwzG2Bml5nZjeHrrczs6Aybjnf3eeHzbwFPuvsxwJ708ps9xuOty7RriKCISCnJMVf12TwFzUMEM/VgaYigiEjh5NKDdTNQD+wVvl4E/CLDdo2R5wcTThh297VAr/5mD3qwopqLXACqIigiUny55Ko+m6eg+UbDjZmKXKiKoIhIweTSwNrS3X9DmJjcfT2tbwoVWGhm55jZ8cAuwGMAZlYNlOcp3qKIxWK4ZShyoSGCIiKlIpdc1WfzFDTPwWoqcoGKXIiIFEMuDayGMPk4gJltSXCVsLXvANsT3I9kiruvCpfvSXBlsddKj2tv4q16sJINeLIREREpmlxyVZ/NU9Ccq5pvNBxZqQaWiEjB5FJF8HKCq3ybmtltwD4EyamF8KaN38uw/Gng6e6FWVyxrDcaHti8rLEW4kMLGpeIiDTpMFf15TwFkTlY6RsNq4qgiEhR5HIfrCfNbDbBFT4DznP3FT0eWQlpfR+sNmXaCYYJWpUaWCIixaBc1TxE0DFSVkbMExCzoDCTerBERAomlyqCxwMJd3/E3R8GEmZ2XI9HVkLisVirHqwwUbVqYImISHEoVzX3YAF4LJxSFi7ylGd4h4iI9IRc5mBd7u6r0y/CMeuXZ9vYzPbJZVlvElwVjI61CH60GCKoBpaISDHlnKv6Yp6C4JYiaekGlqUbXSkNERQRKZRcGliZtmlvaOHvc1zWa8Rjhreoc5GuIjigeYlKtYuIFFNnclWfy1PQsgcrZa0KXWiIoIhIweRS5GKmmV0J/IGgZXEOMKv1Rma2F7A3MNLMfhhZNQTCGbe9VCwWazEHK1ORC69XA0tEpIg6zFV9OU9B8xwsgFSbIYJqYImIFEouPVjnAA3AncBdQB1wVobtKoBBBI22wZHHGuCkfARbLHHroEw76GbDIiLFlUuu6rN5Clr3YLVsYKkHS0SkcHKpIlgDXJzDds8Az5jZLe7+EYCZxYBB7r6mo/eb2eHA1QRXEf/k7r/OsM0BwFUEN4Rc4e5f6mi/+RCLtyzTnq4iqCIXIiKlIZdc1d08FW5fsrkqHmu+ZpoeImjmOCrTLiJSSB02sMxsAnABMC66vbsflOUtvzKz7wFJguEZQ83sSnf/bTvHiBMM6/gysAh41cwedPf5kW2GAf8PONzdPzazUR3Fni8xa1mmPT0Hy8rVwBIRKQWdzFWdzlPhMUo7V8Xa68FSFUERkULJZQ7WXcD1wJ8IklFHtnP3NWZ2CvAocBFBAmsvce0OvOfuHwCY2R3AZGB+ZJuvA/e6+8fQdMPIgoi3vtFwmKisrALi5ZBshIbaQoUjIiJtdSZXdSVPQYnnqsxzsMLspSGCIiIFk0sDK+Hu13Vin+VmVg4cB1zr7o1m1tGls7HAwsjrRcAerbaZEO57OsGY+avd/dZMOzOzM4EzATbbbLNOhJ5ZPBZrUaWdSHPLygfiyVWqIigiUlydyVVdyVOQx1yV7zwFWaoIhlxl2kVECiaXIhcPmdl/mtloM9sg/Whn+z8CC4CBwLNmtjnBBOL2WIZlrZNdGbArcBRwGHBZOCSk7Rvdb3D3Se4+aeTIkR0cumOxmGWsIgg0zcPSEEERkaLqTK7qSp6CPOaqfOcpCEZbpCWbyrSrB0tEpNBy6cH6ZvjzwsgyB7bItLG7XwNcE1n0kZkd2MExFgGbRl5vAizJsM2KcCJzjZk9C3wReKeDfXdbMAcrItLAsooBwTo1sEREiinnXNXFPAWlnqtibXuwguxlamCJiBRQhz1Y7j4+wyNj4wrAzDYysz+b2T/D19vRnPiyeRXYyszGm1kF8FXgwVbbPADsZ2ZlZjaAYFjGWx3Fnw+xmGGWpQerPN2DpTlYIiLF0plc1cU8BaWeqyJ5KknLOVieVANLRKRQcunBwsx2ALYDqtLLss1/Am4BbgYuDV+/Q3Bfkj9n27+7J8zsbOBxgtK3N7n7m2GVJ9z9end/y8weA14HUgTlceflEn8+RBtY0fZV+l5YmoMlIlJcnchVt9DJPBXuq6RzVSzTEEE0RFBEpNByKdN+OXAAQdJ6FDgCeB7I1sAa4e7/MLNLoCkhdTi71t0fDfcfXXZ9q9e/peMqTz0jFu3siw4RDEu1N6wrbDwiItKkk7mqS3kq3LZkc1WLHqymBlbQsHI1sERECiaXIhcnAQcDn7j7twjGkle2s32NmW1I2Aoxsz2B1d0NtNhaDBGM3k+kLPhVeKKhwBGJiEhEZ3JVn81T6V6sJK2LXKiKoIhIoeQyRHC9u6fMLGFmQ4BlZClwEfohwZj0Lc3sBWAkQeLr1Vo0sKI9WPEwfyfVwBIRKaLO5Ko+macg6MVK4c0NrLAHS0MERUQKJ5cG1szwzvQ3EtyIcR3wSraN3X22mX0J2JqgpO3b7t6Yh1iLyqJDBKOTsMoqgkWJ+gJHJCIiETnnqr6apyCch5VsHiJoGiIoIlJwHTaw3P0/w6fXhxN3h7j765m2De8lUuPuK8xsMLAv8AXg/jzFWzQWy1JFMB40sNSDJSJSPLnmqr6cp6D5XlgJWhW5UBVBEZGC6XAOlpk9lX7u7gvc/fXossh2lwH/AmaY2S+Aq4ARwHlmdlXeIi6SaHUmj94Hq6x5iGB0uYiIFE4uuaqv5yloLnSRaDUHSz1YIiKFk7UHy8yqgAHACDMbTvMd7IcAYzK85WvAtuF7PgY2dvdaMysD5uQz6GJoeR+syIp0D5anIJWAeHlB4xIR6c86mav6dJ4CMhS5CFeogSUiUjDtDRH8D+B8ggQ1i+av6TXAHzJsX+fuDUCDmb3v7rXQVP6214+fyzZE0NINLAiGCaqBJSJSSJ3JVX06T0G0ByseLFAVQRGRgsvawHL3q4Grzewcd/99DvsaZmYnECS3IeFzwtdDux9qcWUvctFcBdiTDRgDCxiViEj/1slc1afzFEA8HuSq9BDBdGtTQwRFRAonlyqCn5jZYHdfa2b/DewC/MLdZ7fa7hngmPD5s5Hn6de9WixLmXaiPViqJCgiUiy55Ko+nacg0oPlre+DpQaWiEih5NLAuszd7zKzfYHDgCuA64A9ohuFN3bss1r2YEWWtx4iKCIixdBhrurreQqa52AlrOUQQVcVQRGRgumwiiCQHrh9FHCduz8AVLSzfZ8Ua9HAiiSq6BDBhBpYIiJFolxFcw9Wo7e6fqoeLBGRgsmlgbXYzP4InAw8amaVOb6vT4lWEfQWPVjNDSySGiIoIlIkylVE7oPlQQ+WaYigiEjB5ZJ8TgYeBw5391XABsCFPRlUKcp6o+Gy5qqBrjlYIiLFolxF8xDBRm81RFANLBGRgmnvPlhD3H0NUAVMD5dtANQDM9t5X5xgiMa46P7d/cq8RFwksWxVBFv0YDUWLiAREelSruqreQoic7Ba3wcrqTLtIiKF0l6Ri78DRxPcV8Rp/pomfL1Flvc9BNQBbwB95pJZtAfr87W1rP5sHeVlcYaVaYigiEgRdSVX9ck8BdE5WGEPVroqk/ep0xQRKWnt3Qfr6PDn+E7ucxN337FbUZWgaBXB39/zCvP/+REAp+9WzSHhcg0RFBEprC7mqj6ZpwDiYa5qbLrRcPBDQwRFRAqn3TLtZlYGHAFsEy6aDzzu7ol23vZPMzvU3Z/IU4wlYaMNBjU9t8gQwZffWc4h6TpVGiIoIlJwXchVfTJPQWQOVqpVkQuVaRcRKZisRS7MbAzwJvBfwBhgLPAj4M1wXTYzgPvMbL2ZrTGztWa2Jp9BF8PRe01oen7sPhPYOGxwLVsbyd8aIigiUlBdzFV9Mk9B9jLt6sESESmc9nqw/ofgXiJXRRea2bnAr4BvZnnf74C9gDfco9Ugerfy8njT8x232IhTth7B7+58iUQkiWmIoIhIwXUlV/XJPAXNPVgNraoIqky7iEjhtNfA2tPdT2+90N2vMbO323nfu8C8vpa0sGhnnzNmxGAAGqO/Qg0RFBEptK7kqr6Zp2i+D1brMu2qIigiUjjtNbDWt7Outp11S4HpZvZPgjK5QB8ofxutS5VyxmwYNrDUgyUiUkxdyVV9M08R6cFKBRcF06lLQwRFRAqnvQbWUDM7IcNyA4a0874Pw0dF+OgbrGXl3wFV5QwfXMWqtZHcrjlYIiKF1pVc1TfzFM1zsBIpA0xDBEVEiqC9BtYzwDFZ1j2b7U3u/lMAMxscvPR1XQ+vhEQbWGG+Gr3hYD5fW0eCOGUkIdFQnNhERPqvTueqPpunaO7BSgHEKyJdWI67Yy0uFoqISE9o7z5Y3+rKDs1sB+CvwAbh6xXAN9z9zS5FWCpaNLCCK4EbbzCQ+QuW0+hllFkST6qBJSJSSF3JVX02T9F8H6xUyqGsorkHK1gI8XiWd4qISL5kLdPeDTcAP3T3zd19c4LSuTf2wHEKrLmBlZ4WvcGQAUBkMrGGCIqI9AZ9NE81DxFMpRyLl9M05CJYWJygRET6mZ5oYA1096fTL9x9OjCwB45TUBaL9mAFCWvDIdUANISFLlxDBEVEeoM+maegeYhgMuVQVt1y8IUqCYqIFER7c7C66gMzu4xg+AXAqQSTifuOVg2sRFMPlhpYIiK9QJ/NU009WO5Y+QCwyMgK9WCJiBRETg0sM9sbGBfd3t1vzbL5t4GfAveGr58FujSfq6RY2x6spiGC6V+LhgiKiBRNJ3JV38xTQDzePESQ8mrg8+aVamCJiBREhw0sM/srsCUwB0iPL3CgTdIyszhwl7sfkscYS0TLMu0AG4Q9WI0aIigiUlS55qq+naeic7BSYQ9W8xws3QtLRKQwcunBmgRsl8sd7909aWa1ZjbU3Vd3P7wSEmtbpn3YoEpiMaMRDREUESmynHJVn85TROZguWPl1VjrKoIiItLjcmlgzQM2JrjzfS7qgDfM7EmgJr3Q3c/tfHilJFpFMEhS8ViM4YOqIj1YGiIoIlIknclVfTRPtawiSMWAFoMv1IMlIlIYuTSwRgDzzewVoKkF4e7HZtn+kfDRKWZ2OHA1EAf+5O6/zrLdbsAMYIq7393Z43SVtR0hCATDBBvXpOdgqQdLRKRIOpOrupSnoPRzVfQ+WFZWjcq0i4gUXi4NrKm57MjMnnD3Q939L2Z2ibv/KtcgwjHxfwC+DCwCXjWzB919fobt/hd4PNd9541FKtpHRqBEG1ipZfNpfOshyrc9ptDRiYj0d1M72qA7eSp8f8nnqvQQwUw9WKhMu4hIQXR4Hyx3fwb4NzA4fLwVLmttZOT5VzoZx+7Ae+7+gbs3AHcAkzNsdw5wD7Csk/vvvhY9WM0NrMryMj5PDm56Xf/yHwsYlIiIQM65qjt5CnpBrmpZpr1aRS5ERIqgwwaWmZ0MvEKQjE4GXjazkzJs2mERjHaMBRZGXi8Kl0XjGAscD1zf0c7M7Ewzm2lmM5cvX96NsFrstPl5pIEVjxkPrd+7eV39uvwcT0REcpZjrupOnoI85qoeyVO0utFw+QAVuRARKYJchgheCuzm7ssAzGwkMA1oPaZ8CzN7kKCvJ/28STtztqBl/1DTW1q9vgq4KKwA1W7A7n4DcAPApEmTuptQA1mOGYsZq1KDeTa5F/vHX8I1D0tEpBhyyVXdyVOQx1zVI3mK4KIfRHqwosdUA0tEpCByaWDF0gkrtJLMPV/RYRJXdDKORcCmkdebAEtabTMJuCNMWCOAI80s4e73d/JYXRNNlJEklU5mDR6WalclQRGRYsglV3UnT0EvyFXROVit74OlHiwRkcLIpYH1mJk9Dtwevp4CPNp6oyzzsnL1KrCVmY0HFgNfBb7eav/j08/N7Bbg4YI1roKjNscSWRoLi180eHMlQXeno142ERHJqw5zVTfzFPSCXNWiTHt5dcs+NzWwREQKosMGlrtfaGYnAvsQfFXf4O735TMId0+Y2dkEFZfiwE3u/qaZfS9c3+G8q55mWYpcpHuwGlPx9EpIJSBeXrjgRET6OeWqQCwyRJCyaixySdBVRVBEpCBy6cHC3e8hqIjUY9z9UdpebcyYrNz99J6MJaMsZdpj8fQQwcivMlGvBpaISIEpVzVf9APw8ioNERQRKYKsVQTN7Pnw51ozWxN5rDWzNYULsURk68EKu7bq03OwAE9qHpaISCEoV7UUizaw4i3vg6UiFyIihZG1B8vd9w1/Ds62TSZm9hBtqyqtBmYCf3T3us4GWRKylGmPxcI5WKnmBhYJVRIUESmEruSqPpunaJ4XDODxKlqcZlINLBGRQsjlPlh/zWVZxAfAOuDG8LEG+BSYEL7undop0w5QH21gqQdLRKSgOpmr+maeomUPVrKsSkUuRESKIJc5WNtHX5hZGbBrO9vv7O77R14/ZGbPuvv+ZvZmV4IsCZEGlmco016fav5Vukq1i4gUWmdyVd/MU7Scg5WKV7e40bCGCIqIFEZ7c7AuMbO1wI7RMe0EV/keaGefI81ss8h+NiO4FwhArx07l63oelMVwWhbVTcbFhEpiC7mqj6Zp6C5TDuAxytbFblQFUERkUJobw7Wr4Bfmdmv3P2STuzzv4Dnzex9gnbJeOA/zWwg8JduRVtMsSxVBMMGViJa5EI9WCIiBdHFXNU38xQthwimKIN4ZHSFerBERAoilyGCr5jZUHdfDWBmw4ADst040d0fNbOtgG0IEte/IxOGr+p2xKWgxX2wWt1oGDQHS0Sk8HLOVX05T0V7sJLuxMoqm1eqgSUiUhAdFrkALk8nLAB3XwVc3sF7diUYD78jcLKZfaPLEZaKbFUELcMQQVURFBEptM7mqr6Xp4B4PNKDlUphFZEGlqoIiogURC49WJkaYVnfF1Zt2hKYA6QHfDtwa2eDKyktqghGe7DCBparyIWISBHlnKv6bJ6iZQ9WKuVYWVXTaw0RFBEpjFwaWDPN7ErgDwQJ6BxgVjvbTwK2c/fW9xjp3Vr0YDU/Td8Hq9FVpl1EpIg6k6v6Zp6i1Rwsd6yiuYGlIYIiIoWRyxDBcwiqKt0J3AXUAWe1s/08YOPuh1ZiopWZMhS5iA4RdA0RFBEptM7kqr6Zp2g1ByvlWHm0B0tVBEVECqHDHix3rwEu7sQ+RwDzzewVoKkrx92P7Xx4paNFmXZvf4igerBERAqrk7mqT+YpaC68BMEQQcqrgMb0guIEJSLSz3TYwDKzCcAFwLjo9u5+UJa3TM1HYCWngzLtLRtY6sESESmkTuaqqYWJqvBaDBFMOVZRDawNF/S5EZEiIiUplzlYdwHXA3+ieTJwVu7+THeDKnmZerA0RFBEpJhyzlV9OU+1nYNV3fRaRS5ERAojlwZWwt2v62gjM3ve3fc1s7W0KAOBAe7uQ7oaZEnooEx7ihhuccyTGiIoIlJ4HeaqPp+nyDAHK9LAIqk5WCIihZBLA+shM/tP4D5ajlX/LLqRu+8b/hyc1whLRYsy7c2i492JV0BivXqwREQKr8Nc1efzFM2jKiDswSqPNLDUgyUiUhC5NLC+Gf68MLLMgS0ybWxmWwKL3L3ezA4guInjreFNH3uvaBXBSJKKJjOPV2KJ9erBEhEpvJxzVZ/NU7Seg5WCygFNrz3ZWIyQRET6nVyqCI7v5D7vASaZ2ReAPwMPAn8Hjux8eKXDsvRgRZOZxyqCJ7rRsIhIQXUyV/XJPAUZbjRcEWlgNdQVIyQRkX4nlyqC38i03N2z3fE+5e4JMzseuMrdf29mr3UnyJLToshF8xBBj5cHP1VFUESkoDqZq/psnope9Eu2bmDp4p+ISEHkMkRwt8jzKuBgYDaQrYHVaGZfIxiucUy4rLzLEZYSs6BxlaFMO6gHS0SkiDqTq/psnmpxHyx3rHJg88pGXfwTESmEXIYInhN9bWZDgb+285ZvAd8DfunuH5rZeOBv3YqyVOTYwHLNwRIRKahO5qo+m6fa3geruYHljcpNIiKFkEsPVmu1wFbZVrr7fOBcADMbDgx29193LbwSkx7bnuE+WACpWHgBVFUERUSKLWuu6st5qs0crAHRIYLKTSIihZDLHKyHaL5fSAzYDvhHO9tPB44N9z0HWG5mz7j7D7sbbNFlKHTRIpmle7A0RFBEpKA6k6v6cp5qMQfL03OwHDBd/BMRKZBcerCuiDxPAB+5+6J2th/q7mvM7AzgZne/3Mxe71aUpSJsTHkq0oMVj4x3T8/BUpELEZFC60yu6rN5Kt5qiCBl1WAObrjmYImIFEQs2wozuwXA3Z8Bxrn7M+7+QgeNK4AyMxsNnAw8nLdIS0BktlXTs2gPljcNEVQPlohIIXQxV/XZPNVyiGAq6MFKL9LFPxGRgsjawAK+GHl+Xif2+TPgceA9d3/VzLYA3u1KcCWngzlYSRW5EBEptK7kqj6bp1qXaae8mvRFQU8kihSViEj/0t4QQW9nXfY3ud8F3BV5/QFwYlf2VXIyNrAiQwStMnjSUFvIqERE+rNO56q+nKdaVBF0x8oHYOY4KnIhIlIo7TWwNjGzawgGF6SfN3H3c6OvzexH7v4bM/s9GRJe6+17pQwNrGgyaywLyuF6/RrcHctQFENERPIq51zVH/JUi4t+KYd4RWSIoHqwREQKob0G1oWR5zNz2Ndbndi2d2pqYDUvijawEmWDgiepBDSuh4oBiIhIj+pMrurzeapNmXazpgaWJxqLFJWISP+StYHl7n/pzI7c/aGuvK9XaUpcmedgNcYjN3SsXx2WxxURkZ7SmZzTH/JU6yGCwcJwmXqwREQKois3Gs7IzB5sb727H5uvYxVN0wjBLEMEow2sujUweHTBQhMRkfb1hzwV7cFKhrcUMbNgDpYaWCIiBZG3BhawF7AQuB14mWhV8xyY2eHA1UAc+JO7/7rV+lOAi8KX64Dvu/vc7gbdqRgJklS2IhcNLXqw1hQwMhERyUG38hSUfq5qcx8sgDBPee0qzQ8WESmA9u6D9b/hz6/kuK+NgR8DOxAkny8DK8J7kjzT3hvNLA78ATgC2A74mplt12qzD4EvufuOwM+BG3KMK39iGYpcRBJVQ6xVD5aIiPSoTuaqLuep8Bgln6taDBFMpQCwymC4ujfUklzyWiHDERHpl9q7D9aRZlYOXJLLjtw96e6Pufs3gT2B94DpZnZODm/fneB+JB+4ewNwBzC51f5fdPfPw5czgE1yiSu/2r8PVmOkgYV6sERECiHnXNXNPAW9IFdlmoNlVYPD4KDxzXsLGY6ISL/U3hDBx4AVwEAzW0PQuvD0T3cf0voNZlYJHAV8DRgHXAPk8m0+lmDYRtoiYI92tv8O8M9sK83sTOBMgM022yyHw+eogzLtDbHmohbqwRIRKYhO5apu5CnIY67qqTyVcQ5WvByAxs+HsOJv72D3fr95mKDFGHTwQQw98YS8xSAi0t+1V0XwQuBCM3vA3Sdn2y7NzP5CMOzin8BP3X1eJ+LINCA8480jzexAgqS1b7adufsNhMMyJk2a1KUbJmeUIcoWc7BiVaRzu+ZgiYj0vM7kqm7mKchjruqpPNXiPljhxcD48GE0LloEHsMbYnjD6hbvWX3X3ZRVfUDZhsEojNiICZRtcQAWi+crLBGRfqXDIhfuPtnMNgJ2Cxe97O7LM2x6GlADTADOjUyizdrjFbEI2DTyehNgSeuNzGxH4E/AEe6+sqPY887CicLhuHZo2YOVdIPKwVC/Rj1YIiIFlGOu6k6egl6Qq1oMEUwGDazhp3+TtU8+SePc+/BEXTRSGpYPB4y1jz7FwC80d87Z0E0p+8JBmJVhQzehfLtjsLKqAp2FiEjv1mEDK5w4fAUwnSAJ/d7MLnT3u6PbuXt787k68iqwlZmNBxYDXwW+3iqOzQiGcZzm7u9041hdlqnwUnQOVjKVwqqG4PVr1IMlIlJAueSqbuYp6AW5qsWNhsMerPKxY9ng9NNpmDeY+n/9DyTrm7bxZIzGz4bRsHIYVZt8SryqIVi+eiGNs5pvF9bw8vVU7Ho6sQ23wKo3aJsQY+VYeTU2eLSqFIpIv5dLmfb/BnZz92UAZjYSmAbc3e67OsHdE2Z2NvA4Qenbm9z9TTP7Xrj+euAnwIbA/wu/vBPuPilfMeSm/TlYqZRjlUOC+42oB0tEpJCUq2iZk56c+QEpd4YNqqK8LEY8tgvlX7qX4YMqGTFsACOHDaRi4UI+vfjHgFFXewwVIzch9dkHJFe8Cw014I7FEsSq62lY9Ieshe0tngwe1UOIb7Q98Y1GE6scBLGWf2bEho4lPnZXbOimaoiJSJ+VSwMrlk5YoZW0X32wS9z9UeDRVsuujzw/Azgj38ftlKYiF82LWk8otqpghIl6sERECkq5KrTl2OG8v/hzVqyu5c5/vZl1uwGV5Wy16QacvMXWDPngbRrefZeGd98N11aEj674HCtfRsXw1VhFY8YtykdWUDVxR+KjtsUGbURsyFhiIydg5QMybi8i0pvk0sB6zMweJ7gxI8AUWiWXfqOpgZWKLDJiMSOV8rAHa2iwQj1YIiKFpFwV+vl3DuRfsz7kyVkf8NEnq7NuV1vfyNz3PmVpw1jOrFzKBjRSUR5vOfrPIVVbC5G5x7nwxnLql43Iur5ukWPJfxGvjhRZtBixUdtSud9/Ubbp7p06nohIKcmlyMWFZnYCQSUkA25w9/t6PLJSlKEHC4JerBQe9GBVBvcb8fq1BQ5ORKT/Uq5qNrCqgmP22Zpj9tmamroGGhqTJJIpEskUDY1JPlu7nk8+q+G9RSuZ+/6nLFsFvxh7AAAbDKnmO0fuzL47NpeO90SCxLJlpNZmzmueTJFaX4vXrgd3UjU1rH/tNerenA/JZJYojfrlYxmw2QeRHaVIffom6+/+NuU7nkzlfv+FVQzM8n4RkdKVSw8W7n4vud8npO9quqzXsoUVjxmJJKRSKaisDLaITCIWEZGep1zV1sCqCga2Kv63+cbDgid7fIFkMsWM+Yu5a/qbfLh0FZ+tWc9v73iRWMzYe4egYKKVlVE+Zkynjjv4iMPxZDJjz9eKq65h/axZ1C8fxoYX/RMrbwjmfS2dS+PcOyBRR+Pr/yCx4HmqDrqM2IgvYGXVWPWwLvwGREQKL6cGloSaRgi2bmDFgCRJd6w8zGSN6wsbm4iISCfF4zH2mbgpe22/CdPnLODGh2ZTW9/I/901g42GD2TLsRt0ed8Wj0O87b20Bh9zFOtnzYJEgnXPvMSwKScT33BLyrf6MhU7nkzdk5eTXPQqvmYJ6+//fnOs4/ejcp/ziY/cussxiYgUghpYnWBk7sFKV21KJR3S9wlJJfBUAovpVywiIqUtFjMO2mU8wwZV8fO/PEtDY5Jf/u05rj3vSAZUlef1WJVbb03FF75Aw3vvseahh6l57vlWWwwC35t4bAnx6nVgYc79ZD41L50JGBY3qnbamQFHXkRs0EZ5jU9EpLty+uvfzKqBzdz97R6Op7RZ2zLt0HwvrGTKW96IMVEPFWpgiYgUgnJV9+0yYTTfOnIn/vzIa6xcvZ7HXnmPE/bfNq/HMDOGHHM0K/7vKkgkSK5YkXG7JEOA7Pd+rv1wCTUvn86A3fcmPm4/YoNGtVhfPnYM8WHD8he4iEiOcrnR8DEEN2+sAMab2U7Az9z92B6OrfRkK3KR7sHyVHMPFuCJOk3QFREpAOWq/Dlm7wlMm/kBH326mgdfeJuj95pARXnboX7dUb37bgz7xmk0fvRRxvXJNWtoeO89UmvXZd+Jx6hfPJz6+94C3mqz2irLGf3b31I2alTb94qI9KBculemArsD0wHcfY6Zjeu5kEpYhjLt0NzASqYic7AAGusKFZmISH83FeWqvDAzjt9/G66662U+X1vH9DkLOHS3LfN+jCFHHtHuNu5Oat26NqNGABJLl/LZTTfQ+NHS7O+vb+Sz637HqMv/t9vxioh0Ri4NrIS7r9Yd1yHLFKywyAWkUg5llU3LPaEGlohIgShX5dF+O27ObU++wfJVtdz77FscvOv4plxXKGZGfPDgjOviQ4aw8a+voHHBRySWvk3y4xkkP/uA1KqPoXE99Z9sQOPnw6h7ayE1zz/KwH2PLGjsItK/5dLAmmdmXwfiZrYVcC7wYs+GVaIsnVxaFbmw5h4sWs/BEhGRQlCuyqOyeIzJ+2zNnx55jaUr1/Hs3I84cOfxxQ6rBTOjYvw4KsaPg70Pa7Gu/o0n+PRXf4ZUnM9v/As1z76MVQ9vsU188GCGnvwV4kOyz/MSEemKXC5HnQNsD9QDfwdWA+f3YEwly7KWaU83sFItilyoB0tEpGCUq/Lsy7ttyfDBQU677ck3aGjMdtPg0lM58VAG7T0BgFR9nLrX32X9y6+0eKyb9hSf//VvRY5URPqidhtYZhYHHnT3S919t/Dx3+7eT1sOHZRpb9OD1U9/TSIiBaRc1TOqKsr42sE7ALB8VS0Pv/ROkSPqnGFnXkbVthsTG1BPrLquxcOqgqIdtS+8SGLZsiJHKiJ9TbsNLHdPArVmNrRA8ZS2WOYqgtE5WOrBEhEpLOWqnnPIrluwychgCN3d0+ezprb3DH2PVVQw6vIrGXPN9Yz4xiSGHzaAYXuvZuiO7zBoq7CxmEqx5qGHixuoiPQ5uczBqgPeMLMngZr0Qnc/t8eiKlntVxFMecsiF+rBEhEpGOWqHhCPx/jm4V/kl399jpq6Ru6ZPp9vHblzscPqlNigkVQd+GMAksv+Te1tJ1E2oIaKsVU0LK5j3fRnqNp+eyhv+SeRlZVRuc02xCorM+1WRCSrXBpYj4QPyXIfrBY3Gi6vbl6hMu0iIoWiXNVDdttmDNuNG8n8Bct5ZMa7HL33BEYO6533eIyP2oayCYeReOdxKoe+ScPiLaGxkRVXXZ15+xEjGPmjC6jYbLMCRyoivVmHDSx3/0shAukVmqr/Zp6DlUylWpVp7z1DKUREejPlqp5jZnzjsC9y8R+n0ZhIccdT8zjnxD2KHVaXVe51Nol3n6RscA3lG9TS+NmArNsmV6zg059MZeB++0K87c2WY1VVlI0cSWzwYDCIVVVTvvlmqkwo0s912MAKy93+CtgOaJpg5O5b9GBcpSnswfJWNz2MW3ORCyuL9GAl1hcsNBGR/ky5qmdtu/kIdt92LK+8tZh/zV7AMXtvzbjRw4odVpfENhhP5b7nU//clQz8wnsk11dD+aA22zV+VkXdgmF4XR3rnpzWqWPYwAFYrG2DzMrLGfb1rzJw3327HL+IlL5cyrTfDFwHJIADgVuBv/ZkUKWq6QaW3roHK/g1JtvcaFg9WCIiBaJc1cNOO3RHYmak3PnFX59l5eraYofUZRWTvk3VYb/E4mWUDVhPWfnyNo/qjRYycMICYlV1WDzR/CgHqyrHqsqbpw604jW1pNaubfNIfvYZn/3pJpKff17gMxaRQsplDla1uz9lZubuHwFTzew54PIejq0EZWtgRXqwYnGIlUEqoTlYIiKFo1zVwzbbaChfO2QHbnvyDZavquXym6ez+7Zjm9ZXlsc5ZNKWbDikup29lI7y7SYT22BLGt/+J6QaM28DDEw2kvr8Q5KfvgmNLUemeApSNpz4mH2o2PWbpGobaFzwEYmVK9vsy+vqqXn2WbyujlW338mG//m9njgtESkBOVURNLMY8K6ZnQ0sBkb1bFglqsMiF2F1wfJqqF+rMu0iIoWjXFUAXzlgOz5bs55/vvweC5etYeGyNS3Wz1+wnJ9++8AiRdd58Y13IL7xDjlt63VraHj1zzS89jdIBiNULAZxPoelD5OYtZABx11H9Re/mHUfqdoa1s+cRc2zz1KxxXhiQwa32aZi3DjKx4zp2gmJSEnIpYF1PjAAOBf4OXAQ8M0ejKl0WeYy7dH7YAFYvBJnrcq0i4gUzvkoV/U4M+O7x+xCLGY8NetDEskgH6bcSaWcOe99yoJPVjFu42HFDbQHWNUQKvf7ARV7fg+vWwXuJJfOpXHunSQXzyS1dC6193yH6mOuITZkdMZ9DDv1VNbPmQuJBJ/fkr0uS/XuuzNw7z0hwzyu+LChVE6YkK/TEpEekEsVwVfDp+uAb/VsOCUuPdS6VQ9Wi/tgAZQH86vVgyUiUhjKVYUTj8U485hdOfOYXZuWffrZOr73u0dIufPQi+9wzgm7FzHCnmXl1U23ZIkNGUPZFw6h7p8XkXj3CVLL3qLmr8dTue/5xIZu2vym8mrio3ekfOONGHrSiay+4852j7H+lVdY/8orWdcPP+M7DD7k4Lycj4jkXy5VBCcAFwKbR7d394N6MK7SZOmaIK2qCKaHCCbDHqyyqmALFbkQESkI5ari2miDQey+7VhmzF/EM3MW8I1Dd2TooKqO39gHWLycqiN/Q/3Tw2l8/U5oWEf9v37RZrvYyG2oOuRyhh43mYH77ovXt70Im6qrY90TT1Lz3POQSrVZn7bq1r9Ste02lI8dm3UbESmeXIYI3gVcD9wIJHs2nNLWPEKwVZELazUHq0w9WCIiBaZcVWTH7jOBGfMX0ZhI8eMb/8WwQZVsMGQAY0cMZtNRQ9h842GM2XBw06iPvsRiZVQdfBll4/ej7snL8dq2RS5Sy/9N7e1fCwphNb3RiI3ajvKtvkzZNkdTPnAsld/fkmFf/xrJVava7KNxyVJWXvN7vKGBFb+/lsFHHEGsuprqXXbGynL5k05ECiGX/40Jd7+uxyPpFTKPESwvC3q21q1vwN2xsIGlOVgiIgWjXFVk240byRZjhvPBks9ZtHwNi5a33WbsiMGceeyu7PSFjQsfYAGUbXEAA7/9GKkV77aYr51Y8DwNM/8MycagynBEaulc6pfOpf7lG6g+/FeUbfEl4kOHEh86tM3+KzbfnMYPP2TNgw/RuOAjPrvuegAGHXYoG3zr9B49NxHJXdYGlpltED59yMz+E7gPaBrz5u6f9XBspSdLFcGtNtmQ6XM+YnVNPQs+WcVG6TlYKtMuItKjlKtKh5lx9vG7cftT81hfnyCVSrFsVS0rIvfLWrxiLZffNJ39dtyMM47ahWGD+94wQgvnW0XFx+xE+bZH0/jO4y0uvnpDLYkPn8VXL4T6Nax/4Czim+0J8Yo2+40NHEX5Dicw9OSvUP/229S//U7TunVPPMmgA75ExfjxPXdiIpKz9nqwZhE0JdLdNhdG1jmwRU8FVbKy3Gh40jZjuPHh2QC8+u8lHJO+2bDmYImI9DTlqhKy5dgN+O9v7N9iWV1Dgo8/Xc3r73/KXdPnU9eQ4LnXP+a1dz/hO0ftzIE7j8Oy3LC3L4kNH0flHv/RZrkfcDGJdx6n7smfQGMtyY9nZHx/EmicdzexjXZg+FemYCO+S3LVWpb9/JeQTPLZTbew0U8vx2KxjO8XkcLJ2sByd10GaS1LA2vjDQax6aghLFy2hpn/XsIx49JzsNa33oOIiOSRclXpq6ooY8KmGzJh0w05cOdx3PjwbF56cxHr1jdw9d0v88pbi/nP43djyIDKYodaFGZG+daHEx+1DfXPX0VqzZIMW3kw7DCVIPXpPOqfnBf0cpVVUjl6OPWLhtHw7rss/tZXwcAqqqFyKBaLgxmV223L0MnHUrbRRgU/P5H+qL0hgrsBC939k/D1N4ATgY+Aqf1y2EXTFCxvs2q3bcawcNka3lm0kobNy4kBXr82mJPVD67MiYgUg3JV77Lh0AFcfMq+vPTmIv744Ew+X1vHS28u4t8fr+CMo3Zhn4mb9tucGRs+jupjrsq6PrVuGY2v/4PGN+4KimgkGyDZQPXGNTR8OhBvLCdVH/5ZV9cIrGh6b+KTT6h5+l/EB6aCv2UsjpVVQlkVVjEAqx5IfNgwYgMGhBeTw88g/VFEPpOmzyf6OZm1fJ1hWdnw4QzYZx/KRmzYqd+LSG/U3hDBPwKHAJjZ/sCvgXOAnYAbgJN6OriSY5mLXADssd0m3Pvsv3GHBfUbBGNS6lbjaz/BstxwUEREuk25qhfaa/tN2H7cSP5w36vMmL+Iz9fW8ds7XuS+5zZg6MBKqirK2G7cSHbcciM2HTWk3za6omKDRlG599lU7PEfJN6bRvKTN5oKaQzbrJb1b64MqhzXr8NrmiuMpBrLSKwZDG4k10VvXFwfPlYX7BxW3XEnFV/4AlbRdo5ZGwbxIUOIDx+OlZc3/w0WNtws2oALn8eHDWPAbpOIDRrUcychkoP2GljxyJW/KcAN7n4PcI+ZzenxyEpQ+gu+dZl2gAmbbMjwwVV8vraOFz4d3DToP/npvKx3dBcRkW5Truqlhgys5OJT9uHp1xZw86NzWFNbz3uLmzscX5i3EIDhg6uYOH4U++64GbtuPYayeP+eY2Txcsq3PoLyrY9oWlYFDI5sk1g8m8T8+/HGYKpC48oG6t6uIVWfAhxvrIeGdeDBHQ08GSPVWI4nI7/bpj912mncOhArC3rDYmWR+4VG3u+Ou+O1teBOw7vvdu3Ec/T5TTdTOXEHYpUZhpzG45SNGkX5mNFYReYhqWUjNqR8s81U9l66pd0GlpmVuXsCOBg4M8f3dYmZHQ5cDcSBP7n7r1utt3D9kUAtcLq7z853HB1EGf5s28CKxYy9t9+UR2a8y9MfV3HKyBgxUqQ+eQO2+nJhwxQR6T+Uq3oxM+OgXcaz2zZjuPPpN/n3R8GwthWra/l8bVBt7/O1dTz7+sc8+/rHDB1YyTabj2CzUUPZZNQQNhk5hDEbDmZAVXkxT6PklI3dhbKxuzS9rgaGtNrGU0lSy+aT+OhFfN2nLdcl6vGa5Xhd0LvldWuCSoc5ydwgSzRsSOOazUnWlGfdpuntsTKwMlI19STXrodkuuS9Z/oTrGXsjY3UzX4tx1izhFBRQXzYsLbDHrPF282NrKKC2JDBTUM0m3rnmh6xYG6dxSAWDuGMRbaLxSjfZBOqdtyR8o01z64UtJd8bgeeMbMVwHrgOQAz+wJ57k82szjwB+DLwCLgVTN70N3nRzY7AtgqfOwBXBf+LJzw5ogN777Hqr/f3mb15ESS8vULWVuf4NP6zRkeW0fNI/+k8clniY9NEDMYnMMkXqveAKseBhYPrgzFgp/E4uGyGGZlLbvGiY51bjUuuqi6cfxuxV6s43ZDsc63qL+rrr+/e4fub/8uC/9vOjZqW8o2mVSIQylX9QGDB1RyxlHNDQJ3Z9HyNbz+/jLe+OBTXnv3E+oaEqyuqefl+Yt5ef7iFu8fOrCSoQOrqKyIUx6PEY/HqKooY+jASgZWV1BRFqcsHgv+VsWIx2OUxYPcGTPYdNRQdtqqb96bKxuLxYlvPJH4xhNz2j5Vu5LUsrfwdctI1a4ET+H1a0i8Ow1fE/08MreAyipWUDZiBYzIQ/BkmA7vRmLtQOqXb0CyJkv5/1SMVEMHDTzAGxpILFuWlzhLQqb5cR0sN8AGVBMbOBCLl4W/suiwzPTr9Agva7lN5HnzYdqua70fq6wMjlkeXDSx8gpiAwdglRU0fW4t5v+1OIHs50xk/mBk/YB99yHeA0NK26si+EszewoYDTzh3vRPOUYwvj2fdgfec/cPAMzsDmAyEE1ak4FbwzhmmNkwMxvt7kvzHEt28eaxy2sefCjjJgc3PRvM+rDD3uJJBg9/D4BUTQ/GJyJSIsp3Pq0gDSzlqr7JzNh01FA2HTWUo/bairqGBC/OW8iM+Yv4+NPVfPLZuhZ/YK+uqWd1TddvjXLQLuP7XQOrs2IDNiQ2bt82y32//yL16XxSqz4itWYxJBoyvNtJrf2U1Cevk1r7SfsHSiUh2fFn2ebvZ3PKh66jfOi6dt/nSSNVX4F7pkaWkVxfSXLdADwRz7C+c9rrbCvb8mCsrApwvL6e5Jo1+Pq64HXKg3dHfrp70KqMPsJtvaEBX99O5erWrdEMxdoyxe6rG0mtXtPhtr1Z1cQdCtvAAnD3NjdjcPd3Mm3bTWOBaN/zItpe8cu0zVigTdIyszMJh4lsttlmeQty0IEH0PD++6Rqs/wjTqUgmWyzuNFizKyfQFk8xqStx7R7DPcUvm4Z3lgLyQR4IviySSXwVDKY0JpKBsvT/0HC/2RtnouI9AO9MVf1VJ7qq6oqyjhol/EctEtQlb+hMcmSlWtZvHwNS1au5ZOV66ipa6SuIUEimSKZTFFT38iamnpq6xppSCTb/Zuy6IM9ejGzGPGNdyC+8Q5526c31AaVEsMiHngq+Bso1QiJOjzTh+kOqUa8oQYaa/FEfVBpMSqVxBvW4fVrwyqMjbT4m8kdTzaE+6gLbrcTbueJumBZm302QmPnb8sz8MzTiQ3MT3eeu9P48cfUzXuTVG1tdEXm59HXWZZ7KkWqthavrcWTwdy9oGHXtGHY6KPVupbPvdV7gpabt3ke/EjhdfWkamrwZCLYpKEBr2u+OXdvUSoz+DJ9tbX+35PLNsFC9xsIqkcxadKkvLU2Buy2GwN2263dbbyxEU+lvxAcTyRorKtn1MDBJJIpqgf1vbvWZ5PxCzD3d3fnwL3wuN1QrPPtbkO+aJ9TNw7b7/5dduO98RyqhPU+ectVPZWn+ouK8jjjNh7GuI2H5bS9u5NK/+HokEymSKRSTf894jG1sEqJVQzAKgYUO4ycuacgkUMjwL3pgrlVD8vb8c2Mis03p2LzzfO2z1LiySTe2Bi+aKfR2Hp5ZL1nWg/BvLceUCoNrEXAppHXmwCt77SXyzZFZ+XlbbJrT3Q99gbdK6vbnbkq3TisiEh2fSZX9TdmRjx6T6Z4jP55W2PpCWYxKM/9D3X9mdI5Fo9j8e4P2SykUql1+iqwlZmNN7MK4KvAg622eRD4hgX2BFZrTLuIiBSQcpWIiHSoJHqw3D1hZmcDjxOUvr3J3d80s++F668HHiUoe/seQenbbxUrXhER6X+Uq0REJBfWvXkypW/SpEk+c+bMYochIiKd029G0ShPiYj0WhlzVakMERQREREREen11MASERERERHJEzWwRERERERE8kQNLBERERERkTzp80UuzGw58FE3djECWJGncIpN51J6+sp5QN85l75yHtC7z2WFux9e7CAKIQ95CnrPZ60486c3xAiKM596Q4zQv+LMmKv6fAOru8xsprtPKnYc+aBzKT195Tyg75xLXzkP6FvnIu3rLZ+14syf3hAjKM586g0xguIEDREUERERERHJGzWwRERERERE8kQNrI7dUOwA8kjnUnr6ynlA3zmXvnIe0LfORdrXWz5rxZk/vSFGUJz51BtiBMWpOVgiIiIiIiL5oh4sERERERGRPFEDS0REREREJE/UwGqHmR1uZm+b2XtmdnGx4+ksM1tgZm+Y2Rwzmxku28DMnjSzd8Ofw4sdZ2tmdpOZLTOzeZFlWeM2s0vCz+htMzusOFFnluVcpprZ4vBzmWNmR0bWleS5mNmmZva0mb1lZm+a2Xnh8l73ubRzLr3qczGzKjN7xczmhufx03B5r/tMpHtKMVd15TujmMwsbmavmdnD4euSi9PMhpnZ3Wb27/D3ulepxWlmPwg/73lmdnv4PVX0GHvL3xVZ4vxt+Jm/bmb3mdmwUowzsu4CM3MzG1HMOLPFaGbnhHG8aWa/6bEY3V2PDA8gDrwPbAFUAHOB7YodVyfPYQEwotWy3wAXh88vBv632HFmiHt/YBdgXkdxA9uFn00lMD78zOLFPocOzmUqcEGGbUv2XIDRwC7h88HAO2G8ve5zaedcetXnAhgwKHxeDrwM7NkbPxM9uvXvoCRzVWe/M4r9AH4I/B14OHxdcnECfwHOCJ9XAMNKKU5gLPAhUB2+/gdweinEmCUXl9x3ZZY4DwXKwuf/W6pxhss3BR4nuHH6iGLGmeV3eSAwDagMX4/qqRjVg5Xd7sB77v6BuzcAdwCTixxTPkwm+JIm/Hlc8ULJzN2fBT5rtThb3JOBO9y93t0/BN4j+OxKQpZzyaZkz8Xdl7r77PD5WuAtgmTa6z6Xds4lm5I8Fw+sC1+Whw+nF34m0i0lmau68J1RNGa2CXAU8KfI4pKK08yGEPzB+GcAd29w91WUWJxAGVBtZmXAAGAJJRBjb/m7IlOc7v6EuyfClzOATUoxztD/AT8iyEVpRYkzS4zfB37t7vXhNst6KkY1sLIbCyyMvF5E+3+ElSIHnjCzWWZ2ZrhsI3dfCkECBEYVLbrOyRZ3b/2czg67+2+KDEvoFediZuOAnQl6THr159LqXKCXfS7hsKY5wDLgSXfv9Z+JdFrJf645fmcU01UEfxSmIstKLc4tgOXAzeFQxj+Z2UBKKE53XwxcAXwMLAVWu/sTpRRjK73xu/LbwD/D5yUVp5kdCyx297mtVpVSnBOA/czsZTN7xsx2C5fnPUY1sLKzDMt6W037fdx9F+AI4Cwz27/YAfWA3vg5XQdsCexEkIR+Fy4v+XMxs0HAPcD57r6mvU0zLCv1c+l1n4u7J919J4Irmrub2Q7tbF6y5yHdUtKfaye+M4rCzI4Glrn7rGLH0oEyguFO17n7zkANwbC2khFelJpMMMRqDDDQzE4tblRdUpL/p8zsUiAB3JZelGGzosRpZgOAS4GfZFqdYVmxfp9lwHCC4fQXAv8wM6MHYlQDK7tFBGNJ0zYh6OruNdx9SfhzGXAfQXfnp2Y2GiD8uSz7HkpKtrh73efk7p+GfxingBtp7oYu6XMxs3KCP5Ruc/d7w8W98nPJdC699XMBCIcKTQcOp5d+JtJlJfu5dvI7o1j2AY41swUEwysPMrO/UXpxLgIWhb3UAHcTNLhKKc5DgA/dfbm7NwL3AnuXWIxRvea70sy+CRwNnOLhpCFKK84tCRrWc8P/S5sAs81sY0orzkXAveEQ+1cIeq1H0AMxqoGV3avAVmY23swqgK8CDxY5ppyZ2UAzG5x+TjBJch7BOXwz3OybwAPFibDTssX9IPBVM6s0s/HAVsArRYgvZ+kv9NDxBJ8LlPC5hFd4/gy85e5XRlb1us8l27n0ts/FzEamq0mZWTXBHzf/phd+JtItJZmruvCdURTufom7b+Lu4wh+d/9y91MpvTg/ARaa2dbhooOB+ZRWnB8De5rZgPDzP5hg7l0pxRjVK74rzexw4CLgWHevjawqmTjd/Q13H+Xu48L/S4sIitx8UkpxAvcDBwGY2QSCYjEreiTGrlTG6C8P4EiCykfvA5cWO55Oxr4FQUWUucCb6fiBDYGngHfDnxsUO9YMsd9OMESrkeA/6Xfai5ugW/p94G3giGLHn8O5/BV4A3g9/E89utTPBdiXoLv8dWBO+DiyN34u7ZxLr/pcgB2B18J45wE/CZf3us9Ej27/Wyi5XNWV74xiP4ADaK4iWHJxEgxfnhn+Tu8nGOpUUnECPyW40DMv/E6tLIUYs+TikvuuzBLnewTzg9L/j64vxThbrV9ApIp1MeLM8rusAP4W/vucDRzUUzFauFMRERERERHpJg0RFBERERERyRM1sERERERERPJEDSwREREREZE8UQNLREREREQkT9TAEhERERERyRM1sKTozMzN7HeR1xeY2dQ87fsWMzspH/vq4DhfMbO3zOzpVsvHmdl6M5sTeVTk8binm9mYyOs/mdl2edrv8lZxd3q/4fnP63jLrO8/P7xDfPr1o+l7P4mIFIryVLeOqzwl/Y4aWFIK6oETzGxEsQOJMrN4Jzb/DvCf7n5ghnXvu/tOkUdDnkIEOB1oSlzufoa7z8/Tvu9sFXe+9tsZ5wNNicvdj3T3VUWIQ0T6N+Wprjsd5SnpZ9TAklKQAG4AftB6Resre2a2Lvx5gJk9Y2b/MLN3zOzXZnaKmb1iZm+Y2ZaR3RxiZs+F2x0dvj9uZr81s1fN7HUz+4/Ifp82s78T3HS2dTxfC/c/z8z+N1z2E4Kbal5vZr/N5YTT5xE+P8nMbomc7zVm9qKZfdDq3H8UHntueL4nAZOA28Ird9VmNt3MJmWLNX1sM/tluJ8ZZrZRLjGH773TzI6MvL7FzE4MrwA+Z2azw8feGd57upldG3n9sJkdED6/zsxmmtmbZvbTcNm5BEn56fQVVzNbkP4Dx8x+GJ7bPDM7P1w2LrxCe2O4ryfMrDq9PzObH37ed+R6ziIiKE8pTylPSWcU6q7PeuiR7QGsA4YQ3Pl7KHABMDVcdwtwUnTb8OcBwCpgNMGd4hcDPw3XnQdcFXn/YwQXE7YiuJt3FXAm8N/hNpXATGB8uN8aYHyGOMcAHwMjgTLgX8Bx4brpwKQM7xkHrKf5Dux/iJ5H+Pwk4JZIvHeF8W4HvBcuPwJ4ERgQvt4g03HTrzuI1YFjwue/Sf8eWsV9OrA8EvccoBo4HvhLuE0Fwd3lqwmu3lWFy7cCZkbOf15kn9dGjvEwcECr84mH57Bj+HoBLe8GvwAYAexK8IfFQGAQ8Cawc3i8BLBTuP0/gFPD50uAyvD5sGL/u9dDDz16zwPlKeUp5Sk9OvFQD5aUBHdfA9wKnNuJt73q7kvdvR54H3giXP4GwRdY2j/cPeXu7wIfANsAhwLfMLM5wMvAhgRfuACvuPuHGY63GzDd3Ze7ewK4Ddg/hzijQy/OymH7+8N45wPpq3aHADe7ey2Au3/WwT7ai7WBIGkAzKLl7yqq9dCL9cA/gYPMrJIgmT4bLi8HbjSzNwgSb2fHwZ9sZrOB14Dtc3j/vsB97l7j7uuAe4H9wnUfuvucDOf3OsFV1FMJkpuISM6Up1pQnlKeknaogSWl5CqCMeIDI8sShP9OzcwIrkal1UeepyKvUwRXw9K81XEcMOCcyJfyeHdPJ76aLPFZjueRi2hMVa3WRc/LIj9bn0d72ou10d3T+0rS8nfVLnevI7hydxgwBUgPYfgB8CnwRYIrk5kmSDd9lqEqADMbT3A1+GB33xF4hLa/k9baO7/o7y96fkcBfyC4qjjLzHI+bxGR0FUoT4HylPKUtEsNLCkZ4dWufxAkr7QFBF80AJMJrkB11lfMLBaOd98CeBt4HPi+mZUDmNkEMxvY3k4IriB+ycxGWDCx+GvAM12IB+BTM9vWzGIEwxk68gTwbQsrFZnZBuHytcDgHo61tTuAbxFciXs8XDYUWOruKeA0giEUrS0Adgo/i02B3cPlQwj+WFgdjrM/IvKebOf3LHCcmQ0IP7fjgeeyBRz+njd196eBHwHDCIZsiIjkTHmqXcpTLSlP9WNqGUup+R1wduT1jcADZvYK8BTZr9q1522CL+2NgO+5e52Z/YmgS352eMVxOXBceztx96VmdgnwNMGVqUfd/YEuxANwMcHwh4XAPDr4EnX3x8xsJ2CmmTUAjwI/JhgLf72ZrQf2ynOsU8xs38jr/3T3FwmS6K3Ag95caer/AfeY2VfCY2b6nF4APiQYGjMPmB3GOtfMXiMYn/5BuF3aDcA/zWypRypfuftsCyZcvxIu+pO7v2Zm47KcSxz4m5kNJfh9/J+rypOIdI3yVOZjK08pT0nImntgRUREREREpDs0RFBERERERCRP1MASERERERHJEzWwRERERERE8kQNLBERERERkTxRA0tERERERCRP1MASERERERHJEzWwRERERERE8uT/A+wU1yR24vbZAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "from estimagic import convergence_plot\n", + "\n", + "fig, axes = convergence_plot(\n", + " problems=problems,\n", + " results=results,\n", + " n_cols=2,\n", + " problem_subset=[\"rosenbrock_good_start\", \"box_3d\"],\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "11d035a2", + "metadata": {}, + "source": [ + "The further to the left and the lower the curve of an algorithm the better that algorithm performed.\n", + "\n", + "Often we are more interested in how close each algorithm got to the true solution in parameter space, not in criterion space as above. For this we simply set the **`distance_measure`** to `parameter_space`. " + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "db2c868c", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1gAAAEYCAYAAABBWFftAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAACCKUlEQVR4nO3dd5xU5fX48c+Zso2lN6kCKhZEBbF3Y2+oUYldEzXmZ42xEWNEE5PYa762RIyxYBdbLKjYEQFBBFREOkhnl+07M+f3x72zO7s7OzuzO7vTzvv1Gpnbz51d5+xznyaqijHGGGOMMcaYtvOkOgBjjDHGGGOMyRZWwDLGGGOMMcaYJLECljHGGGOMMcYkiRWwjDHGGGOMMSZJrIBljDHGGGOMMUliBSxjjDHGGGOMSRIrYBljjDHGZCkRmSAiT3XAdc4TkU/b+zqt0VGfgTFhVsAypoOIyBIRqRSRMhHZJCJvisigdrjO7SKyXERKRWSpiNwQsW2IiKgbQ5mIrBGRN0Tk8GTHYYwxuajRd/3PIvKEiBSnOq50IyLbicgkEVnn5quFIvKAiAxMQSx/FJHF7s9shYg8F7Ftqohc0MbzWwEvx1gBy5iOdbyqFgP9gDXAA+1wjX8DO6hqF2Bf4AwRObnRPt3cOHYF3gNeEZHz2iEWY4zJReHv+t2AUcD41IYTHxHxddB1tgW+BFYBo9x8tR+wCNi/I2KIiOVc4GzgMPdnNgZ4P4nn75DP1KQXK2AZkwKqWgW8COwUXiciXUXkSfdp3lIR+ZOIeESkh/tE7Xh3v2IR+VFEzmnm3N+rannEqhCwbTP7/qyq9wETgNtExL4TjDEmSVT1Z+AdnIIWACJygojME5HNbu3IjhHbrhORlSKyRUS+F5FfuOs9InK9iCwSkQ0i8ryI9HC3hVsmnCsiy0RkfWTLBVeBiDznnneWiOwacc0l7nW/AcpFxNdCjINE5GU3V20QkQej3buI3CEin4pI1yibJwCfqepVqrrC/azWquq9qjop4hwXuvluo4i8JiL9I7btKyJfiUiJ++++EduGishH7v2+B/SK8WPaA3hHVRe5cfysqo+657kVOAB40K3detBdf19ES5GZInJAxLUniMiLIvKUiJQCFwN/BMa555gTIxaTJeyPKWNSQESKgHHAtIjVDwBdgWHAQcA5wPmquhH4NfCYiPQB7gFmq+qTMc5/vYiUASuATsAzLYT0MtAH2L51d2SMMaYxt7nb0cCP7vJw4FngSqA38Bbwuojkicj2wKXAHqraGTgSWOKe6nLgRJzc0B/YBPyz0eX2x/kO/wXw58hCETAWeAHogZMPXhURf8T204FjgW44Oai5GL3AG8BSYAgwAJgUcZ5wYfAxYBfgCFUtifLRHAa8FO0zizjPocDfgdNwWn0sDV/LLVy+CdwP9ATuBt4UkZ7u4c8AM3EKVn8Bzo1xqWnAOSJyjYiMce8RAFW9AfgEuFRVi1X1UnfTVziF5vDn+YKIFESccyzOQ9RuOK1K/gY8555jV0zWswKWMR3rVRHZDJQChwN3ALhf6OOA8aq6RVWXAHfhNFtAVd/FSY7v4yTB38a6iKr+A+gMjAb+C0RLcJFWuf/2SPiOjDHGNPaqiGwBlgNrgZvc9eOAN1X1PVWtBe4ECnGacweBfGAnEfGr6pJwrQrOd/4NqrpCVatxaoBOadT87GZVrVTVOcAcnCbgYTNV9UX3mncDBcDeEdvvV9XlqlrZQox74hTwrlHVclWtUtXIgS38OIWzHjjNJCua+Xx6AT+HF0TkUre2rMwtnAGcCTyuqrPcex4P7CMiQ3Dy4EJV/a+qBlT1WeA74HgRGYxTK3Wjqlar6sfA683Egao+BVyGU6D9CFgrItc3t3/4GFXd4F77LpyfW+QDyi9U9VVVDbmfqckxVsAypmOdqKrdcL6MLwU+EpGtcJJNHs4TurClOE8Hwx4FdgYmquqGli6kjq+BSuDmFnYPX2djPDdhjDEmphPdWqiDgR2ob6LWn4jveVUN4RTCBqjqjzi1RhNw/sifFNEkbmucvrKb3Yd0C3AKZH0jrvlzxPsKIHJgjeWNrrnCjaXJ9lgxAoOApaoaaOa+t8WpvblZVWua2QdgA06tVPgaD7q58V6cQlq0OMrc4wY03uZaGrFtU6Om8o33bUBVn1bVw3BqnC4GbhGRI5vbX0T+ICIL3OaJm3Fan0Q2Q1we/UiTK6yAZUwKqGpQVV/GSZD7A+uBWpwkGjYYWAl1NVyPAE8CvxOng3C8fMA2LexzEs5T1u8TOK8xxpgYVPUj4AmcWiBwWgvUfc+LiOAUWla6+z+jqvu7+yhwm7vrcuBoVe0W8SpQ1ZVxhlI3Yq3b13Yg9S0XcK8VFivG5cBgaX7ghgXA+cD/3CaPzXkfaDz4UmON4+iE0xxwZeNtrnDOXA10d/eP3NYiVa1V1ReAb3AeaELDzwa3v9V1OE0Xu7sFwxJAIk/V+NTxXN9kDytgGZMC4hgLdAcWqGoQeB64VUQ6i8jWwFVAeFjXP7r//honUT8Z2U484rweEfmtiHR3r7EncAnNjIgkIn1F5FKc5ivj3SeVxhhjkude4HAR2Q3ne/5YEfmF2wfqD0A18LmIbC8ih4pIPlCF0/og6J7jYZz8sDWAiPR2c0i8dheRk92C0ZXuNac1s2+zMQLTcQow/xCRTiJSICL7RR7sNtf7IzBFRJp7uDcBOEBE7haRAe499QIi+409A5wvIru5n8nfgC/dJvRvAcNF5AxxBuUYhzNo1BuquhSYAdzs9hvbHzi+uQ9GnPm7jnVzr0dEjgZG4IxyCM6Iv8MiDukMBIB1gE9E/gx0ae78EecYIjaQVM6wH7QxHet1d/CJUuBW4FxVneduuwwoB34CPsVJLo+LyO44ha1z3ILYbThPw5prI34SzlC3W3AKaA/QdDj4zSJSDswFjgFOVdXHk3OLxhhjwlR1HU7rgxtV9XvgLJzv5PU4f/gf7zanywf+4a7/GWfgofDDtfuA14B33b5d04C9EghjMk7fqk04fXtPdvtXRYu32RjdHHQ8TlPAZThNDcdFOcd/gFuAD9w+U423/4DTB2wgMMe9p89waqZudPd5333/Ek6hbhvgV+62DcBxOIW/DcC1wHGqut69xBk4n89GnAeIzQ4KhZOP/+jez2bgduB3EX3L7sPp77ZJRO7HGRXyf8APOE0Pq2i5SeAL7r8bRGRWC/uaLCCqVmtpjDHGGGOMMclgNVjGGGOMMcYYkyRWwDLGGGOMMcaYJLECljHGGGOMMcYkiRWwjDHGGGOMMSZJmpvHoI6IdAPOAYZE7q+ql7dbVEl01FFH6dtvv53qMIwxxiRGWt4lYucMzlWWp4wxJmNFzVUtFrBw5hqYhjOcc8bNkbN+/fqWdzLGGJPpMjZXWZ4yxpjsEk8Bq0BVr2r3SIwxxpjWs1xljDEmLcTTB+u/InKhiPQTkR7hV7tHZowxxsTPcpUxxpi0EE8NVg1wB3ADEJ6VWIFh7RWUMcYYkyDLVcYYY9JCPAWsq4BtVdUaiRtjjElXlquMMcakhXiaCM4DKto7EGOMMaYNLFcZY4xJC/HUYAWB2SLyIVAdXpnsoW9F5HHgOGCtqu4cZbsA9wHH4CTR81R1VjJjMMYYk7EsVxljjEkL8RSwXnVfLRIRL/COqh7WilieAB4Enmxm+9HAdu5rL+Ah919jjDHmVeLIVW3MU2C5yhhjTAtaLGCp6n/iPZmqBkWkQkS6qmpJIoGo6sciMiTGLmOBJ1VVgWki0k1E+qnq6kSuk4iFl55JXmktAviKffS84irytx/TXpczxhjTSvHmqrbkKff4tMpVK6d9QuD/HgDAI4KEp7wUL1LYHV/vPvS84nJ8Pbq3x+WNMcZE0WIBS0QWUz8iUx1VbW5kpipgroi8B5RH7N/WZhoDgOURyyvcdU2SlohcBFwEMHjw4FZf0FsdQGqcjyiwEcremGQFLGOMSUMJ5qr2ylMQZ65KVp7SYE1dnlIafQCVmwhu3ET5Rx/R9aQTW30NY4wxiYlnkIsxwB7u6wDgfuCpGPu/CdwIfAzMjHi1lURZ1ySZAqjqo6o6RlXH9O7du9UX3Ni7Fxu7119Gq6tafS5jTHLNmDGDyy9PXveaqVOnctxxxwEwYcIE7rzzzqj7XXPNNYwYMYJrrrkm4WsUFxe3KUYTUyK5qr3yFMSZq5KVp3zdnDy1sTuEBuWTv20xef2ryeu1EfGGAKhduarV5zfGtI3lqtwUTxPBDY1W3SsinwJ/bmb/uJsUJmgFMChieSDQrllj+p7n8c70Rdy/eTKioIHqlg8yxnSIMWPGMGZMx9coP/LII6xbt478/PwOv7ZpXiK5qh3zFHRwrvIN3I6bup8AwCUn7cERe2xD7bxXqXr3T2xZ4CdQ2pnAKitgGZMqlqtyUzxNBEdHLHpwnhJ2jrH/dsDfgZ2AgvD6GE0K4/UacKmITMLpMFzSnv2vALp2avRLGaptz8sZk3b+9cYsFq/elNRzDu3XnQuOG93s9vLyck477TRWrFhBMBjkxhtvZNiwYVxxxRWUl5eTn5/P+++/z8yZM7nzzjt54403mDBhAosWLWLlypUsX76ca6+9lgsvvJCzzz6bU045hbFjxwJw5plnMm7cOE444YQW45wzZw6HHnpog/OdcMIJlJeXs9deezF+/HjGjBnDmWeeSTAY5Oijj+buu++mrKyM1atXM27cOEpLSwkEAjz00EMccMABAPzhD3/gww8/pHv37kyaNIm21F6YeonkqnbMU9DBucrvrW+IUhsIAuDb7gj48Fa8hdUESjtTu2oVqopItMo1YzJbKvIUWK4yscUziuBdEe8DwBLgtBj7TwRuAu4BDgHOJ3qTiQZE5FngYKCXiKxwz+EHUNWHgbdwhr39EWfo2/PjiL1NuhQ5BSzFuQEN1LT3JY3JeW+//Tb9+/fnzTffBKCkpIRRo0bx3HPPsccee1BaWkphYWGT47755humTZtGeXk5o0aN4thjj+WCCy7gnnvuYezYsZSUlPD555/zn//EV3kR7XyvvfYaxcXFzJ49G4DjjjuOK664gtNPP52HH3647thnnnmGI488khtuuIFgMEhFhTM9U3l5OaNHj+auu+7illtu4eabb+bBBx9s4ydmXInkqlblKUi/XOXz1RewAkGnSaDkFeEbfiSe5Z8AoFVVBDdtwtejR3uGYkxOsVxlYomnieAhCZ6zUFXfFxFR1aXABBH5BCcJxbrO6S1sV+CSBGNpky7hGizBKWUFrQbL5JaWnuC1h5EjR3L11Vdz3XXXcdxxx9GtWzf69evHHnvsAUCXLl2iHjd27FgKCwspLCzkkEMOYfr06Zx44olccsklrF27lpdffplf/vKX+HzxPFdq/nyRvvjiC1599VUAzjjjDK6++moA9thjD379619TW1vLiSeeyG677QaAx+Nh3LhxAJx11lmcfPLJCX46pjkJ5qpW5Sn3OmmVq/xeb9372kCofv1OJ+H9fErdcmDlKitgmayUijwFlqtMbC0OcuEOMXu5iNwtIveHXzEOqRIRD7BQRC4VkZOAPkmLuAN17eS0HNHwg82g1WAZ096GDx/OzJkzGTlyJOPHj+eVV16Jq2lT433Cy2effTZPP/00EydO5Pzz469MaO588TjwwAP5+OOPGTBgAGeffTZPPhl9yiRrspU8CeaqrMlTvihNBAG8A0bj69uzbrlm1coOjcuYbGe5ysQSzyiCbwFDgLnEN9rSlUARcDmwO3AWcG5bgkyVLo36YKnVYBnT7latWkVRURFnnXUWV199NdOmTWPVqlV89dVXAGzZsoVAINDkuMmTJ1NVVcWGDRuYOnVq3VPE8847j3vvvReAESNGxB1Hc+eLtPfee/PSSy8BMGnSpLr1S5cupU+fPlx44YX85je/YdasWQCEQiFefPFFwGmasf/++8cdj2lRIrnqSrIkT3k8gtfj/PETbiIIzh9EeaOOB49T6Kr9cV5K4jMmW1muMrHEU/9YoKpXtbSTiLyrqkeo6lciMl5V/04H9JNqT/l5TtOLuvF1gwFUQzgPPo0x7WHu3Llcc801eDwe/H4/Dz30EKrKZZddRmVlJYWFhUyZMqXJcXvuuSfHHnssy5Yt48Ybb6R///4A9O3blx133LFJk4mWNHe+SPfeey9nnXUWd911F8ceeyxdu3YFnGF077jjDvx+P8XFxXVPBTt16sS8efPYfffd6dq1K88991yCn46JocVclY15CsDv8xKsCVAbUcACyBsxFm/h2wTLi6j5aUGKojMmO1muMrGI01w8xg4ivwfKgDeAunHKVXVjo/2+VtVR7vtZqpqaRrGNjBkzRmfMmNGqY9dsKuOiO97g7iWv4w8p+Vuto89tLyP5zQ6iaIxJgQkTJlBcXFzXrjxSRUUFI0eOZNasWXVJJVkqKiooLCxERJg0aRLPPvsskydPTuo1clhCbVLiyVXZmKcAzvzLy5RV1nDs3ttx0Qm7N9i25tqzqV4WRPICDLj/TqSwu+UwY1LEclVWipqr4qnBqgHuAG6gvjJHgcbD2cYuqWUgX0Tn4TCtKbPkZEyGmDJlCr/+9a+56qqrkp6wAGbOnMmll16KqtKtWzcef/zxpF/DxC2eXJV1eQrA744kWBsMNt22zc5UL5uD1vgo+9dxiE8oHPsgvqEHdHSYxphmWK7KPvEUsK4CtlXV9S3sN0xEXsMpyYXf11HVlgfzTzPh+UU0PIygClpdFmMWMGNMKkyYMCHq+sMOO4xly5Y1WPfOO+9w3XXXNVg3dOhQXnnllYSve8ABBzBnzpyEjzPtIp5clXV5CupzVeQogmH5I/ej7EPndzRYlY+vUyXV0/7PCljGpIDlqtwRTwFrHs5cHi0ZG/H+ztaFk158dQWselq9JTXBGGOS4sgjj+TII49MdRgm+eLJVVmXp6C+tUUg2LSA5R+0dd37sh+3R7QSqGHzp/8PvH68XbrS7fRfUTBip44K1xgTB8tVmS2eAlYQmC0iH9KwXfvlkTup6kdJji3lfA1qsFw15SmKxhhjTAwt5qpszFMQ0UQwSg2Wv29fJD8fra5Gq0Io7ui41ZsBCK5dx7rbbqf3tddQsHP8I5cZY4xpXjwFrFfdV87xep2CVWQBS2usBssYY9LQq+Rorgo/DIxWgyV5efS68goqvpiGaojgyplo6SpA8A47lMoZs9GaGtbddju+KKOPic9LwciRdDrwALzNTFTsKShI6v0YY0yma7GApar/6YhA0pHX48Ej0qiJYFnK4jHGGBNdLucqX4xBLgAKR+1G4ajdAAisnEXl8+cAkLdvZ4r2voQNDzyI1tZSu3Rp1ONrFv1E6avNjzjm6dqVvMGDkaJCxOulaN99KRqze7P7G2NMtmu2gCUiz6vqaSIylygjL6nqLrFOLCKdnd00o0skPq+nvgZLgZqMvh1jjMkqbclV2ZKn/G4frGhNBBvz9h+Fp9dwQut/ILDgDYrOvQhPcTHlH3yIBmqb7B/cuIman36Kec5QSQlVc+fWLVd8/gXdzjidLiccn+CdGGNMdohVg3WF++9xiZxQREYCTwI9nEVZB5yrqt+2LsTU8nk9hCKbCFoNljFpYcaMGTz55JPcf//9KYvh4IMP5s4772TMmDEN1j/xxBPMmDGDBx98MEWRNa+4uJiysqz6Hks4V2VdnvI130SwMRHBv+NxVH9yN6FNiwmtnU/hLiMp3GVks8fUrlxJ1Tdz0UCgyTYNBKhduYraFSsgUEtg3Xq0uprNzzxL+cefIHl+Oh14IJ2Pss76JjdZrmqdTM9VzRawVHW1+/b/qWqDcSJF5DbguqZHAfAIcJWqfujuezDwKLBvW4NNhXDbdnD6YqnVYBmTFsaMGdMkWWSqQCCAzxdPl1jTWCtzVVblqfph2qM3EWzMt/0xVH9yD6DUfvcm3r6xB7fwDxiAf8CAuM5ds3w56/72D4KbNjmFLqBm6TI6HXoInry8uM5hTDaxXJWb4vmUDqdpgjo6yrqwTuGkBaCqU0WkUyvjSzmfz4NGDCJow7SbXFI19R+E1n2X1HN6eu9AwcHXN7u9vLyc0047jRUrVhAMBrnxxhsZNmwYV1xxBeXl5eTn5/P+++8zc+ZM7rzzTt544w0mTJjAokWLWLlyJcuXL+faa6/lwgsv5Oyzz+aUU05h7FhndO4zzzyTcePGccIJTac7euKJJ3jttdeoqKhg0aJFnHTSSdx+++0AvPvuu9x0001UV1ezzTbbMHHiRIqLixscP3HiRP7+97/Tr18/hg8fTn6+M1rbunXruPjii+vmOLn33nvZb7/9mDBhAqtWrWLJkiX06tWL4cOHs3jxYlavXs0PP/zA3XffzbRp0/jf//7HgAEDeP311/H7/cycOZOrrrqKsrIyevXqxRNPPEG/fv147LHHePTRR6mpqWHbbbflv//9L0VFRSxevJgzzjiDQCDAUUcdlZSfYZpKJFdlV56KMchFNJ7OW+EdOIbgiq8IfP8/9IA/IB5vUmLJGzSIvrdMoHTya9T89BM1Py2GYJDghg14+vVLyjWMiZSKPAWWqyxXxRarD9bvgP+HMxnjNxGbOgOfxTjnTyJyI/Bfd/ksYHFbA00VJ3HVl7ACC14ndMAf8HTqlbqgjMlib7/9Nv379+fNN98EoKSkhFGjRvHcc8+xxx57UFpaSmFhYZPjvvnmG6ZNm0Z5eTmjRo3i2GOP5YILLuCee+5h7NixlJSU8Pnnn/Of/zQ/FsLs2bP5+uuvyc/PZ/vtt+eyyy6jsLCQv/71r0yZMoVOnTpx2223cffdd/PnP/+57rjVq1dz0003MXPmTLp27cohhxzCqFGjALjiiiv4/e9/z/7778+yZcs48sgjWbBgAQAzZ87k008/pbCwsC7xfvjhh8yfP5999tmHl156idtvv52TTjqJN998k2OPPZbLLruMyZMn07t3b5577jluuOEGHn/8cU4++WQuvPBCAP70pz/x73//m8suu4wrrriC3/3ud5xzzjn885//TNrPKV20MldlVZ7y++LvgxXm2/E4giu+QsvXUfn6FUheccsHJaBwAHg9VdS43bcq/ncbef1ijzbo7b8b/l1OQ8QTcz9j0oHlKstVscSqwXoG+B/wdyCyGL9FVTfGOO7XwM3Ayzglk4+B89sYZ8pENhEMCyx4jbwxv05BNMZ0rJae4LWHkSNHcvXVV3Pddddx3HHH0a1bN/r168cee+wBQJcuXaIeN3bsWAoLCyksLOSQQw5h+vTpnHjiiVxyySWsXbuWl19+mV/+8pcxmzf84he/oGvXrgDstNNOLF26lM2bNzN//nz2228/AGpqathnn30aHPfll19y8MEH07t3bwDGjRvHDz/8AMCUKVOYP39+3b6lpaVs2eLUhJ9wwgkNEvDRRx+N3+9n5MiRBIPBuqd4I0eOZMmSJXz//fd8++23HH744QAEg0H6ubUC3377LX/605/YvHkzZWVldRNUfvbZZ7z00ksAnH322Vx3XXONDzJWa3JVVuUpfwujCEY9ZtvDqf7grxCsJfjT1HaJS6vygB2c2H6ciadkU8z9A9+9QXDlLAqOvBXx+tslJpN9UpGnwHKV5arYYvXBKgFKgNNFZDSwP84ITZ8BzRawVHUTcDmAiHQHNqtqk5GdMoXP46kflsp9E/jpYytgGdNOhg8fzsyZM3nrrbcYP348RxxxBCLS4nGN9wkvn3322Tz99NNMmjSJxx9/POY5wk0lALxeL4FAAFXl8MMP59lnn03o+mGhUIgvvvgi6pPMTp0atkoLX9/j8eD3++vO6fF46mIZMWIEX3zxRZNznXfeebz66qvsuuuuPPHEE0ydOrXF2LJBa3JV1uWpcBPBBGqwpKAL+QdeS83spyHUdPCKZPBGVIqp9ES6Nt8KU6tKobqUwPdvUVm9hcIT7rdClklrlqssV8XSYj2824ziP0BPoBcwUUT+FGW/P4vIDu77fBH5APgRWCMihyU37I4THp0JwNNjGADB9T+kKhxjst6qVasoKirirLPO4uqrr2batGmsWrWKr776CoAtW7YQiDKa2eTJk6mqqmLDhg1MnTq17inieeedx7333gvAiBGxO/NHs/fee/PZZ5/x448/AlBRUVH3xC9sr732YurUqWzYsIHa2lpeeOGFum1HHHFEgxGaZs+enXAMYdtvvz3r1q2rS1q1tbXMmzcPcD6Xfv36UVtby9NPP113zH777cekSZMAGqzPNvHkqmzNU+Fh2uPtgxWWt9vpFJ/3BsW/frtdXp0vfBtPV+cpvmfwMbH3P/8tPFs5I+oHl3xC7ZxJyf2QjEkyy1XNs1wV3yAXZwCjVLUKQET+AcwC/tpov3HAX9z35+IU3voAw3GS3pRkBNzRGsyD5XVHQKopR1WzqqRtTLqYO3cu11xzTd2TsYceeghV5bLLLqOyspLCwkKmTGn6dbLnnnty7LHHsmzZMm688Ub69+8PQN++fdlxxx058cQTWxVP7969eeKJJzj99NOprq4G4K9//SvDhw+v26dfv35MmDCBffbZh379+jF69GiCbnOt+++/n0suuYRddtmFQCDAgQceyMMPP9yqWPLy8njxxRe5/PLLKSkpIRAIcOWVVzJixAj+8pe/sNdee7H11lszcuTIuqYd9913H2eccQb33Xcfv/zlL1t13QwRT67KzjxV10QwsQJWR/D17EVNSSnB9Rti7ieF3Sj65b8of/pUdPNSqr/4J77tj7b+ziZtWa5qnuUqkJZaRYjI/4DTVXWzu9wNeEpVj2u039eqOsp9/xLwrqo+4i7PUtXRyQ+/ZWPGjNEZM2a0+vjxj77Prz56mu7BKgpG9KOw+B0Aii/9CvE3rUY1xnS8CRMmUFxczNVXX91kW0VFBSNHjmTWrFl1bdZNRkjoCVY8uSpb89RT737DC1Pn4xHhlVvHJTGytlt39z1UTv8KX//+9L/7zhb3Dyz5lMpXLgbAt+MJFB71t/YO0ZgOY7kqK0XNVfEM1VMNzBORJ0RkIvAtUCYi94tI5Kxp1SKys4j0Bg4B3o3YVtTaqFMtcpCLyPbgWlOeinCMMQmYMmUKO+ywA5dddpklrOwXT67KyjwVHuQipEowzWqxfD17AhDcsIF4urn5huyPb5tDAWdAqcp3/oRWlbRrjMakmuWq7BNPE8FX3FfY1Gb2uwJ4EegN3KOqiwFE5Bjg6zbEmFL+yFEEI+cJqa3o+GCMMVFNmDAh6vrDDjusbk6PsHfeeafJ6ERDhw7llVdewWS0eHJVVuYpn7c+NwWCIbxRRr9NFW8vp4Cl1dWEysrwdu7c4jH5B11LYMUMZ9CL+a9SvuwLik6fhKe4d3uHa0y7slyVO+IpYD0HbIszKtOicPv2xlT1S8LjsTZc/xbwVluCTKUGfbCk/uPS6rIURWSMaYsjjzyyblhYk1VazFXZmqf8EYMx1QZD5MfYt6P5etb3oQpu2BBXAcvTdSCdznqJqvdvIbjkE7RsDbVf/5f8A65qz1CNSSuWqzJbs4+5RMQnIrcDK3A6/z4FLBeR20UkZ8ZO9Xo99a0rPREFrFprImiMMalmuaphU/ZEhmrvCOEaLIBACwNdRPJ06Ufhif+Hd8DuANTMfRGtsZYjxpjMEKsdwR1AD2Coqu7udgzeBugGtNxTNUs4NViuyCaC1gfLGGPSQc7nqoY1WPFPNtwRwn2wAILr1yd0rIjgH322s1BdSu38yckMzRhj2k2sAtZxwIWquiW8QlVLgd8Bx7R3YOnCeTLoVmFF1mBZAcsYY9JBzucqf0QfrNo0q8HydO0K4Xm6NsRfgxXmG3YI0mUgADVfP4Vqet2fMcZEE6uApdFmtlfVINDsUEAicqqIdHbf/0lEXhaRlAx9mwyRg1xsKqute28FLGOMSQsJ56psy1MNmgim2SiC4vE0GEkw8eO95I06EwDdvJTAD++2cIQxxqRerALWfBE5p/FKETkL+C7GcTeq6hYR2R84EqdN/ENtCzN1IpsIzlu6sX6DFbCMSakZM2Zw+eWXJ+18s2fP5q23UjvOwd/+lvicP0888QSXXnppO0TTvNbE2Y5ak6uyKk81aCIYSK8mggBet4CVSB+sSP4RJyGF3QGo/uRO1EbxNRnEcpUj13JVrFEELwFeFpFfAzNxngTuARQCJ8U4LvztfizwkKpOFpEJSYg1JUYN36rufQgPIRU8olaDZUyKjRkzhjFjxiTtfLNnz2bGjBkcc0zTVmWBQACfL55BV9vmb3/7G3/84x/b/TptlWZxtiZXZVWeSucaLKgf6CKwZg1Vc79t1Tm0/+nUznoSSsrQyfdQeOCFeDp3Bmk6x6f4fEgH/P9qTDwsV6VOKuNs9qegqiuBvUTkUGAETkek/6nq+y2cc6WIPAIcBtwmIvnEN6FxWtpjhwGs6FFMaK3zxKxa/RRKDevWrWNgimMzpr1t+s+T1CxZmtRz5g3Zmu7nNqlwqFNeXs5pp53GihUrCAaD3HjjjQwbNowrrriC8vJy8vPzef/995k5cyZ33nknb7zxBhMmTGDRokWsXLmS5cuXc+2113LhhRdy9tlnc8oppzB27FgAzjzzTMaNG8cJJ5zQ4Jo1NTX8+c9/prKykk8//ZTx48ezYMECVq1axZIlS+jVqxdHHHEEM2bM4MEHHwTguOOO4+qrr+bggw+muLiYK664gjfeeIPCwkImT55M3759WbNmDRdffDE//fQTAA899BD77rsvJ554IsuXL6eqqoorrriCiy66iOuvv57Kykp22203RowYwdNPP81TTz3F/fffT01NDXvttRf/93//h9frZeLEifz973+nX79+DB8+nPz85gfmPu+88+jSpQszZszg559/5vbbb+eUU04B4I477uD555+nurqak046iZtvvpnbb7+dgoICLr/8cn7/+98zZ84cPvjgA95//30mTpzIwIEDm8R599138/jjjwNwwQUXcOWVV7JkyRKOPvpo9t9/fz7//HMGDBjA5MmTKSwsbP0vTxStzFVZlaf8vvTtgwXg6+UM1R4qKWHtrW15ojzM+ee7eWx68cpm95L8fHpfdy0FO+3YhmuZTJGKPAWWqyxXxdZiQlHVD1T1AVW9P47CFcBpwDvAUaq6GWd0p2vaFGWKeTzOEzJRqFTnl2POgp+oqK6NdZgxphXefvtt+vfvz5w5c/j222856qijGDduHPfddx9z5sxhypQpUb/4vvnmG958802++OILbrnlFlatWsUFF1zAxIkTASgpKeHzzz+P+tQvLy+PW265hXHjxjF79mzGjRsHwMyZM5k8eTLPPPNMzJjLy8vZe++9mTNnDgceeCCPPfYYAJdffjkHHXQQc+bMYdasWYwYMQKAxx9/nJkzZzJjxgzuv/9+NmzYwD/+8Q8KCwuZPXs2Tz/9NAsWLOC5557js88+Y/bs2Xi9Xp5++mlWr17NTTfdxGeffcZ7773H/PnzW/xMV69ezaeffsobb7zB9ddfD8C7777LwoULmT59OrNnz2bmzJl8/PHHHHjggXzyySeA07SlrKyM2tpaPv30Uw444IAmcc6cOZOJEyfy5ZdfMm3aNB577DG+/tqZs3fhwoVccsklzJs3j27duvHSSy+1GGtrJZirsipPRdZgpdsoggAFI0d26PW0uprNTz9DlK55xiSN5SrLVbEktR5RRDzAdFXdObxOVVcDq+M49ijgPsAL/EtV/9Foe1ec+U0Gu3HfqaoTkxh+rOgA6NGlkErNAyBfqyirqKEoPyemWTE5qqUneO1h5MiRXH311Vx33XUcd9xxdOvWjX79+rHHHnsA0KVLl6jHjR07lsLCQgoLCznkkEOYPn06J554Ipdccglr167l5Zdf5pe//GVCzSdOOOGEuJ5i5eXlcdxxxwGw++6789577wHwwQcf8OSTTwLg9Xrp2rUrAPfffz+vvPIKAMuXL2fhwoX0jBjOGqh78hm+78rKSvr06cOXX37JwQcfTO/evQEYN24cP/zwQ8z4TjzxRDweDzvttBNr1qwBnKT17rvvMmrUKADKyspYuHAh55xzDjNnzmTLli3k5+czevRoZsyYwSeffML999/f5NyffvopJ510Ep06dQLg5JNP5pNPPuGEE05g6NCh7LbbbnWfy5IlS1r8LNtbW/KUe3za5arIPljp2ESwYKcd6X//fa0aRbCx4KrZBFbNJ1Rei1YHQIMEf54LIeeBZ8i7A9WLS6lZtIjqefMp2HlEm69p0lsq8hRYrgqzXBVdUgtYqhoSkTkiMlhVl8V7nIh4gX8Ch+NMFvmViLymqpHF3UuA+ap6vIj0Br4XkadVtSaZ9xDLzkN7U1vUA7ZsoMBTgzY/mKIxppWGDx/OzJkzeeuttxg/fjxHHHEEEqWfRWON9wkvn3322Tz99NNMmjSprmlAvMJfxAA+n49QqP6P16qqqrr3fr+/7nper5dAINDsOadOncqUKVP44osvKCoq4uCDD25wrjBV5dxzz+Xvf/97g/WvvvpqXJ9HpMhmGeGn+qrK+PHj+e1vf9tk/yFDhjBx4kT23XdfdtllFz788EMWLVrEjjs2bXIVq5Yg8rper5fKysqE4m4Prc1TkL65Kp2HaQ/z9emNr0/vtp9oxx2arAqu+57K165AS1cQqv2C6uUjIRCi5KUXrIBl2o3lKoflqujianMuIn1F5Dj31aeF3fsB80TkfRF5Lfxq4Zg9gR9V9Sc3CU0CxjbaR4HO4vy0ioGNQPO/GckU/gVRJeR1nhAUSDXW+sCY5Fu1ahVFRUWcddZZXH311UybNo1Vq1bx1VdfAbBly5aoSWHy5MlUVVWxYcMGpk6dWvc07bzzzuPee+8FqGv2EE3nzp3ZsmVLs9uHDBnC7NmzCYVCLF++nOnTp7d4L7/4xS946CFncLpgMEhpaSklJSV0796doqIivvvuO6ZNm1a3v9/vp7a2tu7YF198kbVr1wKwceNGli5dyl577cXUqVPZsGEDtbW1vPDCCy3GEc2RRx7J448/TllZGQArV66su9aBBx7InXfeyYEHHsgBBxzAww8/zG677VaXLCPjPPDAA3n11VepqKigvLycV155hQMOOKBVMbVVArmqNXkK0jRXpfsgF+3N23t7ik6diHTeCo8/SH5P5/e4esEPrL7mMtbfez+BtetSHKXJNparLFfF0mINloicBtwBTMVpK/eAiFyjqi82c8jNrYhjALA8YnkFsFejfR4EXgNWAZ2BcdrMjIMichFwEcDgwYNbEU7jE7r/KgS9RQAUSo217zamHcydO5drrrkGj8eD3+/noYceQlW57LLLqKyspLCwkClTpjQ5bs899+TYY49l2bJl3HjjjfTv3x+Avn37suOOO3LiiSfGvO4hhxzCP/7xD3bbbTfGjx/fZPt+++3H0KFDGTlyJDvvvDOjR7c8bdJ9993HRRddxL///W+8Xi8PPfQQRx11FA8//DC77LIL22+/PXvvvXfd/hdddBG77LILo0eP5umnn+avf/0rRxxxBKFQCL/fzz//+U/23ntvJkyYwD777EO/fv0YPXo0wVb0uzniiCNYsGAB++yzDwDFxcU89dRT9OnThwMOOIBbb72VffbZh06dOlFQUNAgETWO87zzzmPPPfcEnI7Do0aN6vDmgAnmqtbkKUhirkpmnkr3Ydo7gqdLP4p++S8qnj+Hgn7rqF7bE1SoXb6B2uUb8BR3oscFv0l1mCaLWK6yXBWLtFRIEJE5wOGqutZd7g1MUdVdYxyzNbCdqk4RkSLAq6rNFrdF5FTgSFW9wF0+G9hTVS+L2OcUYD/gKmAb4D1gV1UtjRX/mDFjdMaMGTHvsSWrrrqawKpVFO29F1v6/EivNR+yNtiVovPfpl/Pzm06tzGm7SZMmEBxcTFXX311k20VFRWMHDmSWbNm1bUrNxkhobYlieaqRPOUe0y75Kq25qkNJRX8+janAu53Y8dw1F7btvpcmU6rywgs+4KK6d9Q/tEnBCvy0IAPX79e9Lv7XpwueMakhuWqrBQ1V8XzTeMJJyzXhljHiciFwIvAI+6qAcCrLVxjBTAoYnkgztO/SOcDL6vjR2Ax0LQxdnuIaEMa8tXXYIWsBsuYtDZlyhR22GEHLrvsMktY2S/uXNXKPAVpmqsih2nPxSaCkSS/GP92h9P1zD/Q+8rzyeu9CYDAz+soe+wYguuam3vamNSxXJV94hnk4m0ReQd41l0eB/wvxv6X4LRT/xJAVRfG0W/rK2A7ERkKrAR+BZzRaJ9lwC+AT0SkL7A98FMc8SeNRvTBKpRqyq18ZUxamDBhQtT1hx12GMuWNRzH4J133uG6665rsG7o0KF1IyVlqltvvbVJG/dTTz2VG264IUURdbhEclVr8hSkaa7yRTYRzPECViT/TmMpGDWX6tXzQYXAmg1UTbmZol89k3Dne2OSwXJV7uSqFgtYqnqNiJwM7I9TDfaoqsb66Varak34y0tEfBB7uD1VDYjIpTjzkniBx1V1nohc7G5/GPgL8ISIzHXjuE5V17d4h0kgEX2wwjVYPglBoMMGMDTGJMmRRx7JkUcemeowku6GG27IugSViARzVcJ5yr1GWuYqf4NBLnKzD1Zzio64gJK3rgIgUF6E7+e5BFfOwjdw9xRHZkxslqsyWzyDXNymqtcBL0dZF81HIvJHoFBEDgf+H/B6S9dR1beAtxqtezji/SrgiJbO0z7qS1jhAhYAteVAr1QEZIwxJkKCuapVeQrSM1c1mGg4TYdpTxVf375IURFaUUGwohOwgZqZE62AZYxpV/H0wTo8yrqjY+x/PbAOmAv8FnhLVTO7qFo3TDuEvAX162tTP6eLMcYYILFclVV5SkTqClm53gerMREhb9hQAEKBvgAEf5pKcMOiVIZljMlysQar+J3bxGF7Efkm4rUY+CbGOS9T1cdU9VRVPUVVHxORK5IeeUeqa6qt4PXXrdagNRE0xphUamWuyro8FS5gWQ1WU/nDhgEQ2FyLhpzPqebLR2IdYowxbRKrBusZ4Hic+TyOj3jtrqpnxTju3CjrzmttgOmhvjOseuoLWIRqUxCLMcaYCK3JVVmXp8JzYeXqPFixhGuwCIWgx0EABL5/i8CKtk3hYowxzWm2D5aqlgAlwOnxnEhETscZTWmoiLwWsakzznC5mU8VIgtYVoNljDEplUiuyuY8ZU0Em5fn1mABVPzQCS0dChqk/K7b8Q7et+1zYwkU7rknnfbZu+V9jTE5IZ5h2uP1ObAaZ9SHuyLWbyF2k8L0F9EHSz15davVRhE0xphMkrV5KjwXlg3T3pS3d288nTsT2rKFmsXLcMrTrtXTk3KNimlf4u/bp0FhzhiTu5JWwFLVpcBSYB8R2RrYTlWniEghUIiTwDJTM32wJBRISTjGGGMSl815qq4Gy/pgNSEidD/7LErfeBMNBAAlVLICgm4zf48P8eXFOgOS3xn8RVG3BtasgWCQDY/+i61u/Qvi9UbdzxiTO+IqYEVJRD5VjZqIRORC4CKgB7ANzkz3D+NMvJihnBKWqqLeiBosayJojDFpI95clY15qq4Pls2DFVWnAw+g04EH1C2HNi+j8u3xhFbPie8EvkI6nf0Snm6Dm2za/PwLlL78CrVLlrDh4UfIGziwyT552w+nYIcdWh2/MSazxDMPVqKJ6BJgT+BLAFVdKCJ9khJtqkRMNIxEfGRWwDLGmLSQYK7Kujzld2tNrAYrPp5ugyka919qv3me2gWvN5/PVQmt+w4ClVS9eyOFp05s0mer64ljqZg2jcCq1VR88ikV0c7j9dLv9n/gHzAg6fdijEk/8dRgJZqIqlW1Rtx+SyLiwymaZIeIGiwJWQHLGGPSRCK5KuvyVN0w7dYHK24iHvJ2/RV5u/4q5n5VH9xK7ZxnCa6cSc30x8jb8yLCvzsAkpdHz99dzNp/3IaWRy1eQTBI6etv0vPii5J5C8aYNBVPASvRRPSRiPwRKBSRw4H/B7ze5khTqP6LVNEG82BZHyxjjEkTieSqrMtT4SaCARumPenyD/g9gSWfoiXLqfn8AUJr5+MfeSqIB2/fnZGCLuRvtx0DH3vU7ePV0IYH/knlV19R/skndD31l/h69kzBXRhjOlI8Y5M2TkQvEDsRXQ+sA+YCvwXeAv7U1kBTKzyKYMNh2sWaCBpjTLpIJFdlXZ6yGqz2I/4iCo+9EynqBUDgx/epfOViKl++iPJnT0drnFor8Xjw5OU1eXUZe7xzomCQ0ldeJbB2bZNXqMb+njAmm8RTg3U98BsaJqJ/NbezqoaAx9xXdojsgxXRRNAmGjbGmLQRd67KxjwVHqbd5sFqH96+Iyg65xWqp9xC4Mf36tbr5qVUf3wHBYfd1Oyx+dtuS/6IEVTPm0fZlPcpm/J+k308nTrR99a/4N9qq3aJ3xjTseKpwSoEHlfVU1X1FOBxd11UInKciHwtIhtFpFREtohIabICTo2IElZkActqsIwxJl3EnauyMU/5wzVYNshFu/EUdqfguLvpdO7rFI17Cu/AMQDUzn2BwJJPYx7b9eSTYm4PlZdTMum5pMVqjEmteGqw3gcOA8rc5ULgXWDfZva/FzgZmKuqGd1puE5EZ9bIJoJ1c2gYY4xJtURy1b1kWZ7yhftg2TDt7UpEkB5DASg44lbKnzoZasqpeu8mOp3/JuIriHpcwYid6HvLBGpX/9xkW8UXX1A1ew4V076ketFP5G9jkxUbk+niKWAVqGo4YaGqZSISfbY9x3Lg22xJWg1ow4mGrYmgMcakjURyVdblKavB6niergPIP+APVL9/C1q2htpvnidv9DnN7p8/fDj5w4c3WV+w4w6s+v0fIBhk85NP0vnoo5rs4+3RI+qxxpj0FE8Bq1xERqvqLAAR2R2ojLH/tcBbIvIRUB1eqap3tynSVIpoISgeLwH14JMQYjVYxhiTLhLJVVmXp3xe64OVCv6dT6Zm1pPopiXUfPUv/CNPQfyxnkE35evTh+LDD6Ps7Xeo/v4Hqr//Iep+va75A0W7756MsI0x7SyePlhXAi+IyCci8gnwHHBpjP1vBSqAAqBzxCuDRY5yAQGcRIbNg2WMMeniSuLPVVmXp8LDtFsNVscSj4/8vf8fAFqxkepP7qb2xymEytYmdJ6uJ5+Et3u3mPtUfBK7n5cxJn20WIOlql+JyA7A9jglje9UNVbVTQ9VPSJZAaYFtw+WquIRoVZ9FEitNRE0xpg0kWCuyro8VVfAsj5YHc63/VF4pj9KaMOP1M6ZRO2cSUhRTzqd/z8kL77aLG+XLvS79x4Ca9c12Vby3HNUzphJ5Zxv0EAA8cXT+MgYk0rx1GAB7AHsAowCTheR5hsZwxQRyarEJZFNBAWC6tRg2TxYxhiTVuLNVVmXp8LzYAWCIbKoa1lGEPGQf+DV1Ld2Aa3YQGDx1ITO48nPJ2/QwCavon32cc5ZWUn1ggVJjNwY015afAwiIv8FtgFmA+FHYwo82cwhlwDXikg1UIvzjaOq2qXN0aaMNFiqrWsi2HTGdmOMMR0vwVyVdXnK7/bBUoVgSPF5pYUjTDL5huxPpws/QCs2UPHi+VC9hcD3b+Pf/pg2n7tw113A44FQiIqZsygYOTIJERtj2lM89cxjgJ3iHW1JVTO6HXtsiscjBMI1WNZE0Bhj0kXcuSob81R4mHZwarHCNVqm43iKe0Nxb3zb/ILA/FcJLPkUrS5D8ovbeN5i8nfcgep586mcOQs99xxErABtTDqL5xv4WyChqcVFpLuI7CkiB4ZfrQsvTYS/yBRAqHXLpZ1XfWhNMYwxJj0klKuyLU/5IwpUtQHrh5VK/u3dYdaDNQQWfZCUcxaOHu2cct06Nj7yKCWvvEqoqiop5zbGJF88NVi9gPkiMp2Gw9meEG1nEbkAuAIYiNNUY2/gC+DQtgabMnV9sBQR0Igmg6Gf5+Ltt0tq4jLGGBMWd67Kxjzl83nr3ttIgqnlHbQXUtANrdpM7Q9v498p6p9LCSncfTSb//sUAOVTPwLAU1hI56OObPO5jTHJF08Ba0KC57wCp6PxNFU9xB3V6eZEA0sv9aNciAjf1GzD1r41AITK1uBt/kBjjDEdY0IC+2ZdnoqswbK5sFJLvH582x5G7bcvElz6OYGln+Hber82ndO/1VZ0Pvpoyj//jFBJKQCBNWuSEa4xph202ERQVT+K9opxSJWqVgGISL6qfoczbG7mimjrLMDHVbvWb7ORBI0xJuUSzFVZl6ci+2DZUO2p59/lVBAvhAJUTr6UwKKpbT5n93PPZuAjD+Pt2ROAUHlFm89pjGkfLRawRGRvEflKRMpEpEZEgiJSGuOQFSLSDXgVeE9EJgOrkhNuirnDtAc04mML2kAXxhiTagnmqqzLUw1qsKyJYMp5+46g4Ng7weODYC2Vb/6B0JbVSTm3p5Mzt1aovDwp5zPGJF88g1w8CJwOLAQKgQvcdVGp6kmqullVJwA3Av8GTmxzpKnUoA+WEIhoFKhWwDLGmHQQd67KxjzVoA+WNRFMC/7tDqfguLudhWA1NV8/k5Tzejp1AiBUYQUsY9JVXOO4quqPgFdVg6o6ETg42n4i4hGRbyOO+0hVX1PVDG9HF9EHC+qGaQfAhmo3xpi0EE+uytY81XAUQStgpQv/NofiHTAGgNpvX0Rr2t6sr66AZTVYxqSteApYFSKSB8wWkdtF5PdAp2g7qmoImCMig5MZZMqFR2lXmtRgWRNBY4xJC3HlqmzNU74Gg1xYH6x04h99tvOmegu1819t8/k8ReEClvXBMiZdxTOK4Nk4BbFLgd8Dg4CTY+zfD5jnDpVb93iluWHdM0HdhH7uMO2RNVjWRNAYY9JCIrkq6/KU34ZpT1u+YQcjXQeiJSuo/vxBAt+/3WQfT69tyT94POL1t3g+q8EyJv3FU8A6UVXvA6pwh7EVkSuA+5rZP6OHuo0uchTBRjVY1kTQGGPSQSK5KuvylM+GaU9b4vGSN+osqqf+A6pLCa6a1WSf4KpZePrsRN7IU1o8X3iQC62qQgMBxBfPn3LGmI4Uz/+V59I0QZ0XZR3gtGdvTSAicpR7Ti/wL1X9R5R9DgbuBfzAelU9qDXXaj0FAcVDSAWPqDURNMaY9BB3rmptnoL0zVV+n/XBSmf+kacR2rSE0MbFTbYF18yDmjJq570SZwGrvuVrqKICb5cuSY3VGNN2zRawROR04AxgqIi8FrGpC7AhxnF7Aw8AOwJ5OEmoXFWb/QYQES/wT+BwYAXwlYi8pqrzI/bpBvwfcJSqLhORPi3fXpLUj3GBx20uGMBLHgFrImiMMSnUmlzVmjzlHpe2uSqyBuuBl7/kkddmdMRlW2XUdltx+Sl74fXENc5WVhBfHgWH/inqturP7qNm+mOEVs8huP5HvL22jX0utwYLnGaCVsAyJv3EqsH6HFgN9ALuili/BfgmxnEPAr8CXgDGAOcA27UQx57Aj6r6E4CITALGAvMj9jkDeFlVlwGo6toWzpk8EX2wwgLqJU8C1kTQGGNSqzW5qjV5CtI4V3UrLsDn9RAIhqisDlBZHeiIy7bK1NlLGdSnK6ccvFOqQ0kL/hEnUTP9MQBq572M96BrY+7fuAbLGJN+mi1gqepSYKmIHAZUqmpIRIYDOwBzY51UVX8UEa+qBoGJIvJ5C3EMAJZHLK8A9mq0z3DALyJTgc7Afar6ZLSTichFwEUAgwcnY6CoiGHa3bd1/bCsBssYY1KmtbmqFXkKkpirkp2nigvzuO6M/Zj1Q3Ims20vs35YzZpN5TwzZS6jh/djWP/uqQ4p5TzdBuMdtBfB5V9SO+9VxNN0oAvp1Av/rr9CvP6GBawyG+jCmHQUTx+sj4EDRKQ78D4wAxgHnNnM/g2GysV5shh1WPcIEmWdNlr2AbsDv8CZRPILEZmmqj80OVD1UeBRgDFjxjQ+T+IimgiKu1A3kqDVYBljTDpIJFe1Jk9BEnNV0vMUsOeOA9hzxwHJOFW7WbhiI9c9/B7BkHLP89O49/Ijc6qpYHP8O59McPmXUF1KzYx/R91HKzeSv98VVoNlTAaI51tNVLUCZ7jbB1T1JCBWvf5Z1A+VW44zVO4vW7jGCne/sIHAqij7vK2q5aq6HieZ7hpH/Ekl7icWcD8664NljDFpIZFc1Zo8BRmUq9LVdgN71DUNXLa2hOVrSlMcUXrwbXs43q33BW9e05c4D3RrZkwkuH5hwwKWDdVuTFqKpwZLRGQfnKeAv2nuOBHZDrgT2AanWcbVqhrvULhfAduJyFBgJU7b+DMa7TMZeFBEfDidkvcC7onz/G0jEU0E3QeYQbUmgsYYk0ZazFVtzFOQ7rkqQ+y900Ce+2Ae4BSyhvTrltqA0oD48ig6+dGo24JrF1DxzDgIBaiacjMFY+v3UytgGZOW4qnBuhIYD7yiqvNEZBjwYZT9HgfewHkKOAtnhKa4qGoA50niO8AC4Hn3WheLyMXuPguAt3E6LU/HGR7323iv0TYSjrNpHyxrImiMMengSlrOVa3OU5AJuSozDOjVuS6XLl9rNVgt8fbZEf+oswEIrZ5N4Ic3wOv8DWI1WMakpxZrsNz5Qj6KWP4JuDzKrp1V9TH3/R0i0nQmvdjXeQt4q9G6hxst3wHckch5kyKyD5Y07INlTQSNMSb14sxVbcpT7nnTN1dliPw8H327d+LnjeUsX1uS6nAyQv4+lxBY+C66ZTU1n92Dp2gkoS1l1gfLmDQVax6se1X1ShF5naadeFHVExqtKhCRUdQXRwojl1U14USWLqRBE0GHjSJojDGpl2Cuyto8lWkG9enqFrCsBisekldEwaE3UDn5UqjegkglAMEtpWigCjw+xBNPrw9jTEeI9X/jf91/74zzXKuBuyOWf45YVuDQxEJLJ9LkrY0iaIwxaSGRXJXFeSqzDOrTla++W8WqDVuoDQTx+7ypDint+YYdjG/bwwn8+B4EtwBF1H73AWUP/Bv8hRQeeze+oQekOkxjDLHnwZrp/vuRiPR236+Lsf8hyQ8vzSh4wk0EsSaCxhiTaonkqpzIUxlicN8uAIRCyqr1W9h6q26pDShD5B8ynsCyLxBfEAANuAXT2kqqP3/ACljGpIlmB7kQxwQRWQ98B/wgIutE5M8dF16aqGshWN/6xEYRNMaY1LNclZkG9ela996aCcbPU9yHonFP4uu3rbMirye+bZyK19Da+QR/bnZubWNMB4o1iuCVwH7AHqraU1W74ww3u5+I/L4jgksbkX2wGtVgWRNBY4xJqSuxXJVxBvXuUpdal9lAFwnx9hqOb+DOAGjAR/6hN9TPlfXNc6kMzRjjilXAOgc4XVUXh1e4ozKd5W7LIW4WiDKKIMFAimIyxhiD5aqMlJ/no083Z8Jcq8FKXHiy4VB5OdKpD75hBwEQ+P5ttMoKrMakWqwClt+dhb4Bt227v7mDROT9eNZllIgmgvWjCDofnfXBMsaYlEo4V2VlnspA4WaCy9ZYgSBR4QIWoRBaXY1/13HOcqCK2gWvpy4wYwwQu4BVk8g2ESkQkR5ALxHpLiI93NcQoH8b40yx+lEExUYRNMaYdBJ3rsruPJV5wgNdrHZHEjTxqytgAaGycryD90G6DAAgsKjx/NrGmI4Wa5j2XUUkWr29AAVR1v8Wpy18f2Am9aWSUuCfbYgxjdT3wap1PzoNVKUyIGOMyXWJ5KocyFOZI1yDFQwpy9eWMqx/9xRHlDk8nYrq3ocqyvH16olvyH7UfvM8wVVfo4EqxBftTzVjTEeINUx7QpNSqOp9wH0icpmqPtDmyNJJuAuW1mfjKs1z3tTaLOrGGJMqieSqrM5TGWiHwb3q3n+7eK0VsBLQoAarvBwA76C9qf3meQjWEFw1G9/gvVMVnjE5L1YTwdb6WUQ6A4jIn0TkZREZ3Q7X6TDhWiu0vgarroAVrLV+WMYYk1myLk9lon49i+nRpRCAb39am+JoMkvDApbzoNc3aE/Cj4GDy6alIixjjKs9Clg3quoWEdkfOBL4D/BQO1ynA0UO0+68q9L8+s015R0ekTHGmFbLwjyVeUSEkcP6AE4NVjAUSnFEmUOKmtZgSWE3PH12ACCw/MuUxGWMcbRHASvcU/VY4CFVnQzktcN1Oo5EvHVLWJVaf0tqBSxjjMkk2ZenMtTOQ50CVnlVLUt/ttEE4+Upri9gBdevJ7BxI4GNG6H7KEI1PgLLv6d29dK69S29gqU2VL4xyRRrkAtExAu8o6qHJXDOlSLyCHAYcJuI5NM+BbmOp/VvqyMLWNYPyxhjUqYVuSp781SGCddgAcz9aY31w4qTp6h+kIuSF16k5IUXI7bu5KyfNT6hcxYfcTg9fn1+MsIzJufFTCiqGgQqRKRrAuc8DXgHOEpVNwM9gGtaHWE6iOiD5YlSg2VNBI0xJnVakauyL09lqK16FNOzq9MPa671w4qbeDz4Bw9O6jnL3v+AUFlZUs9pTK6KWYPlqgLmish7QF1JQlUvj7azqlaIyFpgf2AhEHD/zWD1fbDCb6tCkTVYVsAyxpgUiztXZWeeykxOP6y+TP16CfOXrCMYCuH1WGViPPqMv47KOXMg2LDvWs3cFwmt+TaOM3jIO+AqQluqKH3lVQgGqZg1i+IDD2yXeI3JJfEUsN50X3ERkZuAMcD2wETADzwF7NeaANNC0/JVg0EutMaaCBpjTIrFnauyMk9lsJFD+zD16yWUV9WyfG0pQ7bqluqQMoK3e3eKDz64yXo95CBC63+AZkY4Dm1eRtXb1wOQ32c9/iPOoWzKFEJbyqj8croVsIxJghYLWKr6HxEpBAar6vdxnPMkYBQwyz1+VXg43IwVpYlglfrrt1sByxhjUirBXJV9eSqDDehd/9FvKKmwAlYbiceLt8+OzW73bDUSz/RHCG1cTO28V/Dvfj6FY8ZQ/uFUKr+ZS6iiokEfL2NM4lqshxeR44HZwNvu8m4i8lqMQ2pUVXGHhBCRTjH2zRDS5G2DGixrImiMMSmVYK7KwjyVubp3Lqx7v2lLVQojyQ0ign/ESQCENi4m9PM3FO21l7OxtpbKr2enLjhjskQ8TQQnAHsCUwFUdbaIDI2x//Pu6EzdRORC4NfAY22MMy0oimDDtBtjTBqaQPy5KmvzVCbq3rmg7v3GLZUpjCR3+HY8gepP7wMNUvHib8BXhORvjVYH2fjYv9j89DNJv2be0KH0+v0ViC+ePz2NyWzx/JYHVLUkPP+TS5vbWVXvFJHDgVKc9u1/VtX32hZmikX2wXLf1+Jzi1sKtZYQjDEmxeLOVVmZpzJYvt9HpwI/5VW1bCy1fNoRPJ164Rt2EIFFH0CgCgJV+LsUUbOuB1pVRbAq+TWJlRs3UvXtPAp32zXp5zYm3cRTwPpWRM4AvCKyHXA58HmsA9xE9Z6I9AI2tD3M1JKIPlj1yVsIiQ+v1kIokLLYjDHGAAnmqmzLU5mue+dCyqtqrYlgB8o/9E94egxDa8qp/e4NCgf9jOR78Q7+BZJXnLTrhCorqPxyuvPehoE3OSKeAtZlwA1ANfAMztwhf2m8k4jsDfwD2Ohu/y/QC/CIyDmq+naygu54EuUddQUstQKWMcakWou5KrvzVGbr0bmAFetK2WRNBDuMp7gP+ftfCYB38F5UvX4lRQOW4u0/j6JT/p206wQ2bqovYFXaz9fkhngmmzhWVW9Q1T3c15+AE6Ls9yDwN+BZ4APgAlXdCjgQ+HvSIk6FyDEuIpqfqHidN80MhWqMMabDxJOrsjdPZbjuXZyBLqyAlRr+bQ/Dv/MvAQgun45Wbk7auT1F9YOYqBWwTI6Ip4A1Ps51PlV9V1VfAH5W1WkAqvpdWwJMK6pENu8PiVsBaAUsY4xJtXhyVfbnqQwVHklw45YqnAEeTUfzbX+0+04JLJ+etPNKfn5dB/ZQhU1rY3JDs00EReRo4BhggIjcH7GpC86s941FTiXe+BFFZn9bRu2DVV/A0pAVsIwxJhUSzFXZm6cyXA+3gBUIhthSWUOXovwWjjDJ5u0/Crz5EKwmuHwa/uFHJOW8IoIUFaLlFdZE0OSMWH2wVgEzcJpYzIxYvwX4fZT9dxWRUpwGdYXue9zlgij7ZxBpuCSgajVYxhiTBhLJVVmcpzJb5FDtm0orrYCVAuLLx9t/FMHl0wgs+zKp5/YUFhEsr7AmgiZnNFvAUtU5wBwRecbdb7Cqfh9jf287xJce6oZpV3dRUJRQuA+WDXJhjDEpkUiuyuo8leF6NJpseOutUhhMDvMO3ovg8mno5qWESlfj6dIvKef1FBUSxJoImtwRTx+so4DZwNsAIrKbiLzWnkGlHYmYCAvqClxqNVjGGJMuLFdlsO5dbLLhdOAbvHfd++DyaUk7r6fQKUBbE0GTK+IZpn0CsCcwFUBVZ4vIkPYLKf15RAihhHAehlofLGOMSbkJWK7KWN0b1WCZ1PD02Qnyu0B1KTXfPE9oyxpngwi+Ifvj7TuiVecVt4BlTQRNroingBVQ1ZLIwR1yVeOBjYJWg2WMMenCclUGK8r3U5Dno6omwMZS+yM8VcTjxTdwDIFFHxD6eS41P8+t21Yz/V8UnTEJb89tEj6vp6gIgFCF/WxNboinieC3InIG4BWR7UTkAeDzWAeIyNYicpj7vlBEOrd0ERE5SkS+F5EfReT6GPvtISJBETkljtiTQiJGEXSWncW6QS6sD5YxxqRaQrmqNXnK3Tdtc1WmC/fDsrmwUss/6iynFquxQCVVb/4BrU3851PfRND6YJncEE8B6zJgBFCNMzljKXBlczuLyIXAi8Aj7qqBwKuxLiAiXuCfwNHATsDpIrJTM/vdBrwTR9zJ06gPlridsMKDXKjVYBljTKrFnatak6fc49I7V2W48EiCVsBKLd+gPSn+3acUXzGn7pW3128BCG34kaoPbk14rrK6JoJWg2VyRIsFLFWtUNUbVHUPVR3jvo/VQPoSYD+c5IaqLgT6tHCZPYEfVfUnVa0BJgFjo+x3GfASsLaluNtFeIyLcA0W4VEErYBljDGplGCuak2egkzJVRmqR5f6yYZNaol4EI+37pW39+/wDhgDQGD+q9R8+UgLZ2go3ERQa2vRgLX6MdmvxQKWiIwRkZdFZJaIfBN+xTik2k084eN9tDyB4wBgecTyCnddZBwDgJOAh+OI+SIRmSEiM9atW9fS7i1r0kQwXINlfbCMMSYdJJirWpOnIIm5Kul5Kgt0j2gimGgNiWlf4vFRcMwdSGdn2PaaLx6k5psX4j4+3EQQbKh2kxviaSL4NPAE8Evg+IhXcz4SkT/iTOJ4OPAC8HoL14jWK7nxt+u9wHWqGmwpYFV91H2COaZ3794t7d5q4QKWNRE0xpiUSyRXtSZPQRJzVUflqUwSbiJYXRukotryarrxFPem8ORHkIJuAFRP/Ruhyk1xHSsRBSwbSdDkgnhGEVynqonMJXI98BtgLvBb4C3gXy0cswIYFLE8EFjVaJ8xwCS39qgXcIyIBFT11QRia51GfbA87nIQm2jYGGPSRCK5qjV5CtI9V2W4yMmGz7n11brUO7hPV275zSEUF+alKDIT5u0xjIKj/k7lq7+DYC3Bn6biGXFSi8d5iiJqsKyAZXJAPAWsm0TkX8D7OJ2HAVDVl6PtrKoh4DH3Fa+vgO1EZCiwEvgVcEaj8w4NvxeRJ4A3OjxhNe6DZU0EjTEmXcSdq1qZpyBTclWGGty3a937QDBU937Rqk18OGsxx++3fSrCMo14t94XKeyBVm6kduEU/PEUsAqL6t5bAcvkgngKWOcDOwB+IPyNp0DUApaI7Icz4ePW7vkFUFUd1twFVDUgIpfijLjkBR5X1XkicrG7vcV+V+2qUR+sMBvkwhhj0kbcuao1eQoyIFdluGH9u3PNr/Zl0ar6Zmcffr2YTVuq+HzeCitgpQnxePFtcwi1375EcNnnaE05ktcp9jFF1gfL5JZ4Cli7qurIBM75b+D3wEygxf5SYar6Fk4zjch1UZOVqp6XQDxt5xawtHETQeuDZYwx6SKRXNWqPAVpnquywP67DGb/XQbXLQdDISZ/+j0Llq5j05bKuoEwTGr5tjuc2m9fgmAtgcUf4d/+mJj7e6wPlskx8QxyMS3aPB8xlKjq/1R1rapuCL9aG2BaajxMuxWwjDEm1RLJVdmfp7LEPiOcLm+qMG3+yhRHY8K8g/aCvGIAahe8SXD9wrqXBpoOsx8eph0gZHNhmRwQTw3W/sC5IrIYp117uCnFLpE7icho9+2HInIHTrOMyHbws5ITcgqF+2DRaJh2G+TCGGNSrcVclRN5KstsP6gnPboUsrG0ki/mLefovbZNdUgGEK8f37CDCHz3JsHFH1Gx+KP6bcV96XTOZCS/uH5doQ1yYXJLPAWso+I8112NlsdEvFfg0DjPk3akyTxYzmIwXAGoQTQURDzeFERnjDGG+HJV1uapbOXxCHvvNJC3pi1k7k9rKa2opktRfqrDMoB/pxMJfPdmk/VatobA8mn4tz2sbp3k5YHHA6EQan2wTA5osYClqksBRKQPUBBjv0Pc/Yap6k+R20QkZsfh9Ndw6pO6iYYjP75QAKyAZYwxKRFPrsruPJW99hnhFLBCIWXuorXsN3JQyweZdufbeh+KTn+WUOlqZ4WGqHr7eggFCK6Y0bCAJYKnqIhQWZnVYJmc0GIfLBE5QUQWAouBj4AlwP9iHPJilHXxT/edjuqmwWpUgyURBSrrh2WMMSmTYK7KvjyVxYb17173fs2mshRGYhrzbjUS//AjnNf2R+HpOwKA4MqZTfYNNxO0ApbJBfE0EfwLsDcwRVVHicghwOmNdxKRHYARQFcROTliUxdi1HxlhCZNBMMTDdd/fBqqbVTPZYwxpgO1mKuyOk9lseLCPDoV+CmvqmXtpvJUh2Ni8A0cQ83qOYTWfodWb0HyO9dt8xQWEgTUBrkwOSCeUQRr3dGVPCLiUdUPgd2i7Lc9cBzQDTg+4jUauDAp0aaMRF0KYjVYxhiTJuLJVVmcp7Jbn+7OPEtWwEpv3gHhbo1KcGXDMWPCIwlaDZbJBfHUYG0WkWLgY+BpEVkLNBk2T1UnA5NFZB9V/SLJcaYXt4QV8OTVr4syLKkxxpgO02Kuyqk8lWX6di9m8erNrLECVlrz9h8F4gENEVw5E9+wg+q21TcRtEEuTPaLpwZrLFCBMynj28AinCd+UWVl0mrUBys80XAt9SMZaa19YRhjTArFnauyMk9lucgaLHVzsUk/kl+Mp/cOAAQa9cMKTzZsTQRNLohZgyUiXmCyqh4GhID/dEhU6aZxHyx3da1E1GDV2heGMcakguWq7BcuYNUEgpSUVdOts3WZS1fegWMIrZ1PaM08tLYC8TtNAz1FNsiFyR0xa7BUNQhUiEjXDoonTTlFqrpnZuEaLImswbIvDGOMSQXLVdkvXMACWLPZmgmmM+9Atx9WKEBw1Zy69VJofbBM7oinD1YVMFdE3gPqvtVU9fJoO4tIN+AcYEjk+ZvbPxNIo+EBPeE+WNTXYFkByxhjUiruXJWNeSrb9e1WX8Bau6mc7Qf1TGE0JhZf/9F174MrZ+Lbeh+gvgaL2lq0thbx+1MRnjEdIp4C1pvuK15vAdOAuThNNbJHXbtvp4RVY00EjTEmXSSSq7I3T2WpBjVYNhdWWpPCbnh6bkdow0KCK2bUrQ/3wQKnFstrBSyTxVosYKlqom3ZC1T1qlbGk56azIPlLNZaDZYxxqSFBHNV9uWpLFdU4KdzYR5bKmtsqPYM4B04xilg/fwNGqhGfPlIUUQBq6ISb5cuKYzQmPbV4iiCIrKdiLwoIvNF5KfwK8Yh/xWRC0Wkn4j0CL+SGHPKSZQ+WASsgGWMMamSYK7K+jyVjWwurMzhHbC78yZYQ/DnuUDDGiy1flgmy8XTRHAicBNwD3AIcD6NZ95tqAa4A7iB+nEhFBjW+jBTrJlRBGusBssYY9JFIrkq+/JUDujTvROLVm2yAlYGqBvoArcf1sAxeNxBLgDKPvoI/3ffdWxQfj9FY8bg7WZj4Zj2F08Bq1BV3xcRUdWlwAQR+QQnkUVzFbCtqq5PWpSp1qSJoLMcwgseH4QCYPNgGWNMKiWSq7IvT+WAvuEarM3lhEKKxxPrWa9JJU+nXkj3IeimJU4/rL1+Wz/IBVD29jspiavkhRfoM/568oYMScn1Te6IaxRBEfEAC0XkUmAl0CfG/vNwJnvMWnXlLQB/EVSXWg2WMcakViK5KuvzVDYKNxGsDYTYXFZFjy6FLRxhUsk3YHdqNy0huHo2GqzFP3gw/iFDqF2yJGUxhUpKWXPzXyg+9JCmQ0QnQd6woXTad9+kn9dknngKWFcCRcDlwF9wml6cG2P/IDBbRD4EqsMrs2n4WwnPi6WK+AvR6lIbRdAYY1LrSuLPVVmfp7JR5EiCq9ZvsQJWmvMOHEPtty9BbSWhdd/h3WokW/39VkLlqWniWf7xJ2x+8r9oZSVb3nyr3a7jHzCAvK23brfzm8zQbAFLRPoAfwS2xRnK9u+qen4c53zVfWWPZkYRVAX8zhd8YNGHlE86MwXBGWNM+vEPP4q80We3+3VamateJdvyVA7oEzEX1oQnpnLo6KH85phR5OfF86zYdDRv353r3ofWL8S71UhEBG9xcUri6XLM0Xi7dGbzpOcJbdmS3JOHQmhtLQCBNWutgGVi1mA9CcwEHgCOA+4HzmvphKr6HxHJA4a7q75X1do2xplajaqRw32wFMVT1JPgpiVo1WZ09eYUBGeMMekntNUuHXWphHNVVuapHDCoT1d2GdaHb35aS20gxDvTF7HdwB4cPmabVIdmopBug+r6qQc3/kQ6zHrVaf/96bT//kk/b+3PP7P6SmfmB62ubmFvkwtiFbC2UtUb3PfviMiseE4oIgcD/wGW4IzgNEhEzlXVj9sQZ9pQ1QY1WPn7X0nNjIlooCq1gRljTBrxdB/SUZdKOFdle57KVh6PcMtvDuHbxWv5y38+pro2yKKVmzh8TMvHmo4nHh+e7kMIbfiR0MZFqQ6nXXny66ftCVXZ34MmdgFLRKQ79cPceiOXVXVjM8fdBRyhqt+7JxkOPAvsnpyQO55E1mCpRrxVvP1HUXjCqBREZYwxhtblqqzLU7lCRBg5rC9bb9WNH5ZvYOmaklSHZGLw9NjGKWBtiDV9auaTgoK692oFLEPsAlZXnGYXke3jwk8GY80X4g8nLQBV/UFE0qFmOGk87TDyjDHGmFZpTa7K+jyV7bbu29UpYP282W1ZYnk5HXl6DoOFoKUr0doKxF/U8kEZSCJqsKyJoIEYBSxVHdLKc84UkX8D/3WXz8RJfpmr8Re3uxiKqM0yxhjT8VqZq7IvT+WYrbfqBkB5VS0bSivp1TU7/3DPdJ4e9f3jQhsX4+07IoXRtB/xeJC8PLSmxpoIGgA87XDOi3HmGLkcuAKY767LDqp1NVjBkBWwjDEmA2V3nsoBW/ftWvd+6c+bUxeIicnTM6KAtSG7+2GFmwlaE0ED8c2DFTd3kseZqrozcHcyz51SjfpgdSpwWpJUVNmgU8YYk0myNk/lmCFuDRbA0jUl7L59/9QFY5rl6bY1iAc0RGhjtvfDyodSCFkTQUOSa7BUNQTMEZHByTxvyjVqItipMA+AssqaVERjjDGmlbI2T+WYLp3y6d7ZqTGwGqz0Jb48pJvzv1q2F7A8+eEaLCtgmThrsERkf2A7VZ0oIr2BYlVd3Mzu/YB5IjIdqJuuW1VPaHO0aaLYLWCVWwHLGGPSRgK5KuvzVC7Yum9XNm2pspEE05y3xzACm5YQzPomgs5AF9ZE0EAcBSwRuQkYA2wPTAT8wFPAfo3286lqALi5HeJMH6p0KqivwbLRi4wxJvXiyVU5k6dyxNZbdWP2j2tYvraUQDCEz9se3cpNW3l6bgOLPkBLljtzhnrzWz6ogyTz7zeP2wfLmggaiK8G6yRgFO6wt6q6SkQ6R9lvOjBaVT8SkQdU9bIkxplajfpghWuwgiGlujZIQV5Su7IZY4xJXDy5KnvzVA4KD3QRCIZYtX4LgyMGvjDpo24kQQ1R9kAazQotHvy7jKPg0Bta3jee0+VbDZapF8/jnhpVVZz5RBCRTs3sF/kYYL9m9mmWiBwlIt+LyI8icn2U7WeKyDfu63MR2TXRa7RaRAFLgeLC+ulSrB+WMcakhXhyVZvylHve9M1VOWbriIEullg/rLTl3WokDf/XSxMaonbOswRWzmp53zjYKIImUjxVL8+LyCNANxG5EPg18FiU/Vo9ZrmIeIF/AocDK4CvROQ1VZ0fsdti4CBV3SQiRwOPAnu19poJxRe5EFGDBU4By+bfMMaYlIsnV7Vpbo10z1W5ZlDvLnXvV2/YksJITCye7ltTeNLDBFfPSXUo9TREzYyJEKym+pO78I57qs3NBa2JoInUYgFLVe8UkcOBUpy27X9W1fei7LqDiHyDUx7Zxn2Pu6yqukuMy+wJ/KiqPwGIyCRgLM7cJOE4Po/YfxowsKXYk6bJMO31BSwb6MIYY1IvzlzVljwF6Z6rckx+no9uxQVsLqti7abylg8wKeMbsh++Ia2qNG4/wVpqZvyb0Oo5BBa9j3/bw9p0OmsiaCLF1XnITVLRClWRdmxDHAOA5RHLK4j9xO83wP+a2ygiFwEXAQwenISReBs91OjUqAbLGGNM6sWRq9qSpyCJuSrpeSpH9eneyQpYplXy9vgNNXNfgOpSaj57AN82v2hTLVZdE8GaGjQUQjw26Eoua/GnLyIni8hCESkRkVIR2SIipY33U9WlsV4tXSbKuqhNOUTkEJykdV1zJ1PVR1V1jKqO6d27dwuXTlzXTvUj4JRWWFWwMcakWjy5qo15CpKYq9o7T+WKPt2crnZrN1sByyRGCrqQN+bXAIQ2LiK05ts2nc/jDtOOKlpjD99zXTzF69uBE1S1q6p2UdXOqtqlxaMSswIYFLE8EFjVeCcR2QX4FzBWVTckOYYYGjYRjCxglZRZAcsYY9KA5aoc1Ke7U8Bat7mCYCiU4mhMpvHvdALhv/FqF7zRpnOJO9Ew2GTDJr4C1hpVXdDOcXwFbCciQ0UkD/gV8FrkDiIyGHgZOFtVf2jneBpq1AcrP89XNzT75nJra2uMMWnAclUOChewgiFlY2lliqMxmcZT3AfvoD0BCPzwNhoKtPpc4YmGAbTa/jbMdc32wRKRk923M0TkOeBVoK5IrqovJysIVQ2IyKXAO4AXeFxV54nIxe72h4E/Az2B/3PbyAZUtWMmVIjSKKRrp3yqagJWg2WMMSlkuSq39e1ePxr/2k3l9O7W3EwyxkTn3+FYgsu/RCs2EFz+Jb6tWzcYR3gUQYCQ1WDlvFiDXBwf8b4COCJiWXGe0DUhInNp2ia9BJgB/LW55hKq+hbwVqN1D0e8vwC4IEa87ahpCatrcQFrNpVTajVYxhiTSgnnqtbmKUj3XJV7+kQUsNZsKmfE0BQGYzKSb7vD4YO/QrCG2gVvtrqAFR5FEGwkQROjgKWq5wOIyH6q+lnkNhGJ9dv3PyAIPOMu/8r9txR4gobJMPOok5PD/bBKyu0phTHGpEorc1V256kc0rtb/TyUNpKgaQ3J74xv6EEEfnyPwPdvUbvtoa0asr1BDZbNhZXz4umD9UCc68L2U9XxqjrXfd0AHKyqtwFDWhNkyjXqgwX1BazNZfaUwhhj0kAiuSr78lSOyvf76N7Z+cPWRhI0rZU35jzw+iEUoOqNP1D7wzsJn0MKIge5sL8Nc12sPlj7APsCvUXkqohNXXDanjenWET2UtUv3fPsCRS721rfezBNhNuU9OzqPDXbXFZFbSCI3xfrIzHGGNMeWpmrsjpP5Zo+3TqxaYvNhWVaz9tvVwpPeIDK166AYDVV7/4J35ADkLyilg92WRNBEylWDVYeTsLxAZ0jXqXAKTGOuwD4l4gsFpElOEPVXiginYC/JyPojiZRarDCzRJUsZGLjDEmdVqTq7IuT+WycD8sK2CZtvAN2Z+Cw292FmorCa6YntDx1kTQRIrVB+sj4CMReSLOCRjDx30FjBSRroCo6uaIzc+3OtJUilbA6lrfsXbd5gr69ihufJQxxph21ppclZV5KoeFC1jrSyoIBkN4vfH0fjCmKd82h4LHB6EAgSWf4Rt2cNzHWhNBEynWKIIAJFK4AhCRfOCXOO3YfeHaH1W9pRXxpa1eER1r15XYUzNjjEmlRHJVruSpXNGnW8RcWFsqbah202qSV4R3wO4El39JYMmnCR6bV/c+ZAWsnNcej3kmA2Nx2rGXR7yySu+uEQWsTRUpjMQYY0yCciJP5Yo+jebCMqYtfEOcwUe1ZDmhzcviPk48nrp+WGpNBHNerEEublPV60TkVFV9IYFzDlTVo5IQW/qI0kQwP89H58I8tlTWsL7ECljGGJMKrcxV2ZenclhkAeueF6bRt3snfjd2Dwb26ZLCqEym8g7ZHz65G4DAkk/J2+2MuI+Vgny0utqaCJqYNVjHiIgfGJ/gOT8XkZFtiCn9NChg1b8NNxNcX2oFLGOMSZHW5Krsy1M5rE+3TuT5nQEj122u4NvF6/jLkx+zpcJqEUziPD23Q4r7AiTeTNCtwQpV2e9erotVwHobWA/sIiKlIrIl8t8Yx+0PzBSR70XkGxGZKyLfJDXqlKovYfVymwlaDZYxxqRMa3JVluep3JLn93LlKXuzz4iB7Dy0DwA/byzjzue+IBgMpTg6k2lEpK6ZYHD5dDRQE/ex4ZEErYmgiTWK4DXANSIyWVXHJnDOo9seVpqJrMGKUFfA2mwFLGOMSYVW5qrsy1M5br+Rg9hv5CBUldue+Ywv5q1g9sKfeXPaQk7Yb/tUh2cyjHfgntR++zIEqght+BFv353iOk7y3QKWNRHMeS0OcqGqY0Wkr4gc5756R9tPRMKNnbc088oO2rQGq7yqlorq2lRFZIwxOS+eXJUzeSqHiQhXnLIXW/Vw+mVNmflTiiMymcgTUaAKrp0f/3EF4SaCVsDKdS0WsETkVGA6cCpwGjBdRKJN3viM++9MYIb778yI5cwVUYMVUb6ykYuMMSZNxJmrsjdPmTqF+X5+sfswAJb+XMKSnzenNiCTcTzdtgZ/IQChtQviPk6siaBxtTgPFvAnYA9VXQvgPhWcArwYuZOqHuf+OzTZQaZawwaC9SWsrSImF169oYwhW3XrqJCMMcY01GKuyuY8ZRo6aNetefq9uQB8NHspQ47qltqATEYRjxdv7x0IrvqaYCIFrPAw7VaDlfPimQfLE05Yrg2xjhOR/USkk/v+LBG5W0QGtzHO1IoyTDtAv571BayfN5Z1ZETGGGMaijtXZWWeMg307VHMDoN7AfDxnKWEQtrCEcY05OmzIwChdd+joUB8x7g1WKFqK2DlungKWG+LyDsicp6InAe8CbwVY/+HgAoR2RW4FlgK/LfNkaZSM8O0dy7Kp1OBH4DVG6z5vjHGpFAiuSr78pRp4qDdtgackX4XLF2X4mhMpvG6BSyC1YQ2Lo7rmPpBLqyJYK6LZ5CLa4BHgF2AXYFHVfW6GIcEVFWBscB9qnof0DkZwaajfj2dW1u5zgpYxhiTKgnmqpzKU7lq/5GD8XqcB6TT5q9McTQm04RrsABCcQ50Ie4gF1pdjYZsioBcFk8fLFT1ZeDlOM+5RUTGA2cBB4qIF/C3Mr401LCZwaA+Xfhx5UaWry1JUTzGGGMgoVyV5XnKAHTplM/APl1Y+nMJq9bbQ1CTGE+PbcDrh2AtwbUL8O/U8iwQ4VEEAbSmpm7QC5N74mkimKhxQDXwG1X9GRgA3NEO1+k4zfTBAhjctysAJeXVlJRZm1tjjMkA2ZenTFRbdXf6Sls/aZMo8frx9BoOQGjtd/Edk19foLJmgrktrhqsBG3BaXIRFJHhwA7As+1wnY4TUb4qe28Knk71w7PvsK6UwzY7bXN/fuFlJGJkwfrjhfwRI8jfZlh7R2qMMaZl2ZenTFR93Zy8dlM5oZDi8UgLRxhTz9tnJ0Jr5hFc9TVljx+Nd+DuFBx0HZIfvUWxRNRghaqq8NK1o0I1aSauApaIFAKDVfX7OHb/GDhARLoD7+PMLTIOOLPVUaaap76ir/S11xtsKsZpxA/AOwvY3AHh5O88gj7jr0e83g64mjHGZIYEclX25SkTVXg6lZpAkE1bKunZtSjFEZlM4u23K7VzXwANoiXLCZQsp2L1NxSOfRBPt6YDj3oimgSqjSSY0+KZaPh4YDbwtru8m4i8FusQVa0ATgYeUNWTgBFJiDVlCkeOxNu7d6rDqFP97TyqFy5MdRjGGJM2EsxVWZenTHRb9ahvcWLNBE2ifDscS95ev8W3w7F4+u0GQGjjT1Q8fx5aU9Fkf2siaMLiqcGaAOwJTAVQ1dkiMiTG/iIi++A8CfyNuy6jq1q83bvT/7570ED0eRDumvQ5X85fSbfO+Tzyh+ORRk0QAmvWUDljZrPHx0urqtny5psA1C5bTsEOO7TpfMYYk0UmEH+uyro8ZaLrG9Fsf82mckbYFNMmAeL1k7/vZQCohqj+6A5qv/4vWr6W4Oqv8W29X4P9Iwe5WHf33Yg/r0Pj7Qh5Ww+m5+WX4cnPb3nnHBZPASugqiUicbdbvhIYD7yiqvNEZBjwYSvjSxvi8SB50f9HGTF8AJ9+9zPrygP8tL6c7Qb2aLA9b9Ag8gYNanMMqkrZBx+glZWUvvEGlbNmNdheuNtudD7qyDZfxxhjMlAiuepKsjBPmab6du+EiDM+1eoNVoNlWk/EQ96eF1D7tTNlXnDtd00KWN7u9X//hUpKOzS+jlK5fj3lH31M5yMOT3UoaS2eAta3InIG4BWR7YDLgc+b21lVPwI+ilj+yT0ma+2xQ38eeW0mANc+/B63X3x4k0JWMogIeUO2pnrBdwTXriO4tuHEiVWz51C4xxh8PXsm/drGGJPm4s5VuZincpXf56VnlyLWl1SwZpMVsEzbeIp6IsV90bI1UefG8vXpTY/fXUz1vPjmzco0lXNmEyoppeyDDyg+/DASqHzJOfEUsC4DbsAZ0vYZ4B3gr413EpF7VfVKEXmdxpNFAap6QhtjTVu9u3VinxED+WLeCkIh5er/e5eeXQpBwO/1svVWXfnVoTszrH/3Nl+r26/Gsfn5F9Cq+s6ToYpKAqtXAxDcuMkKWMaYXNRirsrlPJXLturRifUlFdYHyySFp/cOBMvWEGxm6Pbigw6k+KADOziqjlHy0suUvPAitUuWUvPTYhsdO4aYBSx38sXXVPUwnMQVy3/df+9MRmCZ5ndjx7C+pIKFKzYCsKG0sm7bzxvLmPPjGu665AgG9u7Spuvkb789fW/8U4N11QsXsubGmwAIldlkisaY3JJArsrpPJWr+vYo5tvF61izsTzVoZgs4O2zI8HFH6Gbl6I15Uhep5YPyhKdDj6IkhdfAlXKP/jAClgxxCxguXOEVIhIV1UtaWHfme6/H8XaL1t1LS7gjt8dzrT5K/n6h9UEQyEAtlTU8OWClVTVBHhmylyuPX2/Fs6UOE9xfSfe0BZ7QmeMyS3x5qpcz1O5KjxU++ayKiqraynM96c4IpPJPH12rHsfXPc9vgGjUxhNx/L17EnBqN2omvU15Z98SmD9+jafU/Lz6XbaqfgHDkxChOkjniaCVcBcEXkPqHv8o6oN2quLyDexTqKqu7QqwgwiIuwzYiD7jGj4S3LHs5/z6dxlzPhuFdU1AfLzkju/s6dz/YR3wS1Wg2WMyUkt5irLU7lpq0YjCQ7ZqlvqgjEZzxtRwAqtnQ85VMACKD70UKpmfY3W1FA1J+ZXatxqliyh39//hqdT9tQGxvOX/pvuqyUhnDbtzwCvA5Wxd88d++8ymE/nLqO6Nsi8JesYPbxfUs/vKSoiPExSqMxqsIwxOSmeXGV5Kgc1GKp9Y5kVsEybSOd+kN8Fqkub7YeVzQpHj6LzMUdT/UPb52PVmhpqly0juHYdGx5+hJ7/73fO37NtIZIWQ8i3WMBS1f/EcyJV3U1EdgBOx0le891/31XVtk0AleF2Hlo/SfEPyzckvYAlHg+e4k6EtpQRshosY0wOiidXWZ7KTf0iClifzl3OloqaBtsL8nyM3r4fRdZ00MRBRJx+WMu/JLRuQarD6XDi8dD9nLOTci4NhVh/191UzpxF5VczWHH+b1o+KA552wyj62mnUrDLLikb6VBUmwyk1HAHZ7jbvwM7AXVTVKtqzJ5tIjIO+Cdwm6re0WIgIkcB9+FM9vgvVf1Ho+3ibj8GqADOU9VZTU7UyJgxY3TGjBkt7dbufnfXm6za4BR+nr7xZIoLkzv53Kqr/kBg1Wo83bqRN7jpnFu+/v3pfsbpzc7lZYwxaSahrNiaXJVonnKPSXquSpc8la1UldNveYnK6ubL0Fv1KOa6M/ZLymi/JvtVfXwntTOfAI8P/y6nJXy8FHTFP+IkPF36Jz+4DBMqK2P1+BsIrlvX8s6J8vmi1ogV7LA9PS68EF+f3lEOSljUXBVPE8GJwE3APcAhwPnNnUxEBgC/Ak4CNgG/B15pMTJnBKh/AocDK4CvROQ1VY2cSOBoYDv3tRfwkPtvRthpSK+6AtYt//mII8Zsw27bbUWvrkVJOb+3e3cCq1YT2ryZqs2bm+7wzVzK3n6HvGHu3xri/if8i9doWVrYXr/sLoo0WCZyWQTx5+Ht1rUuHF/PnuTvtCP5223X5ns3xhjizFWtzVPusVmfq7KRiHDwbkP435c/NrvPzxvLuO7hKYwavhX9enamuNBPQZ6fbsUFHLDL4A6M1mQCb5+dqAUIBaid/UyrzlHz5aP4hh3kNDeMpplHTNLshmbWe/zg9SPePPD6QTwNLxD+ey3aOSK3NdlHInaPfi7vgN3x9h0RPa5weMXF9L3pz1RMnw7BYMx94xEsKaHs/Q/QykoIRH+oUjX3W1aPH0/P3/2OojG7t/ma0cRTgzVTVXcXkbmqOtJd94mqHtBov4+AzsDzwIvAxsjtqtpgudGx+wATVPVId3m8e8zfI/Z5BJiqqs+6y98DB6vq6ljxp8uTwU1bKrnlPx/z06pNdesmnH8Qo7ZLTnPBqgXfUfLSS2h1dZNtNQubTyqp5OvXj/733JXqMIwx6SnRGqwWc1Vb8pR7fLvkqnTJU9lu05ZKAsFQk/UfzVnK0+/OJRTl76H+vTrz0FXHdkR4JoNobQWVr15CcP0PrThYobo0+UGlofwDryZv9/M6/LrB0lLKP/oo6sjagQ0bqPisfg76Hhf/luKDD2rL5Vpdg1UlIh5goYhcCqwE+kTZb2uczsO/BS5qdGEFYjUpHAAsj1heQdMnftH2GQA0SVoiclE4hsGD0+PJU/fOhdx8/sHc88I0Zv3ghNy9uDBp5y/YcQcK/hR9+pfAmjVsfu4FQpWVgDr/c9flkabL2mR7+K1GXdbG293DIpdDZeWEKpyBvTQUQssryN9xh7bdtDHG1IsnV7UlT0ESc1U65qls171z9Jx7ykE7scOgXrz22fcsX1fK2k3ldQWxwiSP+muyg/iLKDp1YquPD66ZR83MJwiungNRKzqaqfxooVKk6f4hCAXRYA2EXx0qNf2fvF260OX445vdXrHHHmx45FE8hYUUjh7VLjHE881xJVAEXA78BTgUOLfxTqo6pA1xRPsJNP4timefcCyPAo+C82SwDXElVZdO+dx03kFU1wTYVFZFzy7JK2DF4uvbl16XX9oh14pXYP0GCLW9KtgYY1xX0kKuamOegiTmqnTNU7lq52F92HmYUx5XVQLBEJU1AUJB+9GY5PP2HUHhMXF1+2wX9a3XNKLQFn5ArpE7NtzWYB9tuCnaubypH80vmqK998I/ZGtC5RV4uzTTRLON4hlF8Cv3bRlOm/b2sAKIHJlhILCqFftkhPw8X4N5OXKRr1fPVIdgjMkilqtMsogIfp8Xv8+b6lCMaRcS0Y8qRZVMKeffaqt2PX+LBSwRGQ5cg9O0om5/VT00iXF8BWwnIkNxmnX8Cjij0T6vAZeKyCScJhklLfW/MsYYkxssVxljjEkX8TQRfAF4GHgMaJc2XaoacNvMv4Mz9O3jqjpPRC52tz8MvIUz7O2POEPfttcTSmOMMZnHcpUxxpi0EPcogh0UT9LZ6EzGGJORWjWKYHsF054sTxljTMZKbBRBEenhvn1dRP4fzjwhdWOAtzScrTHGGNPeLFcZY4xJN7GaCM7EGQYkXDK7JmJbPMPZGmOMMe3NcpUxxpi00mwBS1WHdmQgxhhjTKIsVxljjEk3nuY2iMgeIrJVxPI5IjJZRO6PaJJhjDHGpIzlKmOMMemm2UEuRGQWcJiqbhSRA4FJwGXAbsCOqnpKh0XZBiKyDljahlP0AtYnKZxMYPeb3XLpfnPpXiH77ne9qh7V0k7ZkKuSkKcgu37+di/pye4lPWXLvWTqfUTNVbH6YHkjOgePAx5V1ZeAl0RkdjsE2C5UtXdbjheRGao6JlnxpDu73+yWS/ebS/cKuXe/ETI+V7U1T0F2/fztXtKT3Ut6ypZ7yZb7CGu2iSDgFZFwAewXwAcR2+KZP8sYY4xpb5arjDHGpJVYyedZ4CMRWQ9UAp8AiMi2QEkHxGaMMca0xHKVMcaYtBJrFMFbReR9oB/wrtZ31vLgtG/PFY+mOoAOZveb3XLpfnPpXiH37hewXBUhm37+di/pye4lPWXLvWTLfQAxBrkwxhhjjDHGGJOYWH2wjDHGGGOMMcYkwApYxhhjjDHGGJMkVsCKQUSOEpHvReRHEbk+1fEkg4g8LiJrReTbiHU9ROQ9EVno/ts9Ytt49/6/F5EjUxN164jIIBH5UEQWiMg8EbnCXZ+t91sgItNFZI57vze767PyfgFExCsiX4vIG+5yNt/rEhGZKyKzRWSGuy5r79fEL1NzVYzv6AkistL9XZ8tIsekOtZ4JPr/aLoSke0jPvvZIlIqIldmys8lm/7OaeZe7hCR70TkGxF5RUS6ueuHiEhlxM/n4ZQFHkUz99Ls71Q6/1zioqr2ivICvMAiYBiQB8wBdkp1XEm4rwOB0cC3EetuB653318P3Oa+38m973xgqPt5eFN9Dwncaz9gtPu+M/CDe0/Zer8CFLvv/cCXwN7Zer/uPVwFPAO84S5n870uAXo1Wpe192uvuH8vMjZXxfiOngBcner4WnE/cf8/mikv9/frZ2DrTPm5ZNPfOc3cyxGAz31/W8S9DIncL91ezdxL1N+pdP+5xPOyGqzm7Qn8qKo/qWoNMAkYm+KY2kxVPwY2Nlo9FviP+/4/wIkR6yeparWqLgZ+xPlcMoKqrlbVWe77LcACYADZe7+qqmXuot99KVl6vyIyEDgW+FfE6qy81xhy7X5NUxmbq2J8R2eT5v4fzRS/ABap6tJUBxKvbPo7J9q9qOq7qhpwF6cBAzs8sFZo5ufSnLT+ucTDCljNGwAsj1heQfZ98Yf1VdXV4CQ8oI+7Pms+AxEZAozCqdXJ2vt1m8zNBtYC76lqNt/vvcC1QChiXbbeKziF5XdFZKaIXOSuy+b7NfHJip91o+9ogEvdJlCPZ0KzOlci/49mil/hzDUXlok/F8je78pfA/+LWB7qNpv/SEQOSFVQCYr2O5XpPxcrYMUgUdbl2pj2WfEZiEgx8BJwpaqWxto1yrqMul9VDarqbjhPtPYUkZ1j7J6x9ysixwFrVXVmvIdEWZcR9xphP1UdDRwNXCIiB8bYNxvu18Qn43/WUb6jHwK2AXYDVgN3pS66hCTy/2jaE5E84ATgBXdVpv5cYsnY/39E5AYgADztrloNDFbVUbjN50WkS6rii1Nzv1MZ+3MJswJW81YAgyKWBwKrUhRLe1sjIv0A3H/Xuusz/jMQET9O4n5aVV92V2ft/Yap6mZgKnAU2Xm/+wEniMgSnCZRh4rIU2TnvQKgqqvcf9cCr+A0l8ja+zVxy+ifdbTvaFVd4z4sCgGPkSFNgxL8fzQTHA3MUtU1kLk/F1dWfVeKyLnAccCZ6nZacpvTbXDfz8TptzQ8dVG2LMbvVEb+XCJZAat5XwHbichQ9ynOr4DXUhxTe3kNONd9fy4wOWL9r0QkX0SGAtsB01MQX6uIiAD/Bhao6t0Rm7L1fntHjCZUCBwGfEcW3q+qjlfVgao6BOf/zQ9U9Syy8F4BRKSTiHQOv8fp5PwtWXq/JiEZm6ua+44O/yHsOgnndz2tteL/0UxwOhHNAzPx5xIha74rReQo4DrgBFWtiFjfW0S87vthOPfyU2qijE+M36mM+7k0kepRNtL5BRyDM6rRIuCGVMeTpHt6FqcathbnCcFvgJ7A+8BC998eEfvf4N7/98DRqY4/wXvdH6dK+Rtgtvs6Jovvdxfga/d+vwX+7K7PyvuNuIeDqR9FMCvvFWeEuDnua174+yhb79deCf9+ZGSuivEd/V9grrv+NaBfqmON414S/n80nV9AEbAB6BqxLiN+Ltn0d04z9/IjTv+k8P8zD7v7/tL93ZsDzAKOT3X8cdxLs79T6fxziecl7k0YY4wxxhhjjGkjayJojDHGGGOMMUliBSxjjDHGGGOMSRIrYBljjDHGGGNMklgByxhjjDHGGGOSxApYxhhjjDHGGJMkVsAyKSciKiJ3RSxfLSITknTuJ0TklGScq4XrnCoiC0Tkw0brh4hIpYjMjnjlJfG654lI/4jlf4nITkk677pGcSd8Xvf+Wz1XiohcKSJFEctvhef6MsaYjmJ5qk3XtTxlco4VsEw6qAZOFpFeqQ4kUnjCvjj9Bvh/qnpIlG2LVHW3iFdNkkIEOA+oS1yqeoGqzk/SuZ9rFHeyzpuIK3HmYwFAVY9R1c0piMMYk9ssT7XeeVieMjnGClgmHQSAR4HfN97Q+MmeiJS5/x4sIh+JyPMi8oOI/ENEzhSR6SIyV0S2iTjNYSLyibvfce7xXhG5Q0S+EpFvROS3Eef9UESewZn8rnE8p7vn/1ZEbnPX/RlnwsyHReSOeG44fB/u+1NE5ImI+71fRD4XkZ8a3fu17rXnuPd7CjAGeNp9clcoIlNFZExzsYavLSK3uueZJiJ944nZPfY5ETkmYvkJEfml+wTwExGZ5b72jXLseSLyYMTyGyJysPv+IRGZISLzRORmd93lOEn5w/ATVxFZEv4DR0Sucu/tWxG50l03xH1C+5h7rndFpDB8PhGZ7/68J8V7z8YYg+Upy1OWp0wiUj3Tsb3sBZQBXYAlQFfgamCCu+0J4JTIfd1/DwY2A/2AfGAlcLO77Qrg3ojj38Z5mLAdzuzhBcBFwJ/cffKBGcBQ97zlwNAocfYHlgG9AR/wAXCiu20qMCbKMUOASupnXP9n5H24708BnoiI9wU33p2AH931RwOfA0Xuco9o1w0vtxCr4s7wDtwe/hwaxX0esC4i7tlAIXAS8B93nzyc2eQLcZ7eFbjrtwNmRNz/txHnfDDiGm8ABze6H697D7u4y0uAXhHHLAF6Abvj/GHRCSjGmb1+lHu9ALCbu//zwFnu+1VAvvu+W6p/7+1lL3tlzgvLU5anLE/ZK4GX1WCZtKCqpcCTwOUJHPaVqq5W1WpgEfCuu34uzhdY2POqGlLVhcBPwA7AEcA5IjIb+BLoifOFCzBdVRdHud4ewFRVXaeqAeBp4MA44oxsenFJHPu/6sY7Hwg/tTsMmKiqFQCqurGFc8SKtQYnaQDMpOFnFalx04tK4H/AoSKSj5NMP3bX+4HHRGQuTuJNtB38aSIyC/gaGBHH8fsDr6hquaqWAS8DB7jbFqvq7Cj39w3OU9SzcJKbMcbEzfJUA5anLE+ZGKyAZdLJvThtxDtFrAvg/p6KiOA8jQqrjngfilgO4TwNC9NG11FAgMsivpSHqmo48ZU3E5/EeR/xiIypoNG2yPuSiH8b30cssWKtVdXwuYI0/KxiUtUqnCd3RwLjgHATht8Da4BdcZ5MRusgXfezdBUAiMhQnKfBv1DVXYA3afqZNBbr/iI/v8j7Oxb4J85TxZkiEvd9G2OM614sT4HlKctTJiYrYJm04T7teh4neYUtwfmiARiL8wQqUaeKiMdt7z4M+B54B/idiPgBRGS4iHSKdRKcJ4gHiUgvcToWnw581Ip4ANaIyI4i4sFpztCSd4FfiztSkYj0cNdvATq3c6yNTQLOx3kS9467riuwWlVDwNk4TSgaWwLs5v4sBgF7uuu74PyxUOK2sz864pjm7u9j4EQRKXJ/bicBnzQXsPs5D1LVD4FrgW44TTaMMSZulqdisjzVkOWpHGYlY5Nu7gIujVh+DJgsItOB92n+qV0s3+N8afcFLlbVKhH5F06V/Cz3ieM64MRYJ1HV1SIyHvgQ58nUW6o6uRXxAFyP0/xhOfAtLXyJqurbIrIbMENEaoC3gD/itIV/WEQqgX2SHOs4Edk/Yvn/qernOEn0SeA1rR9p6v+Al0TkVPea0X5OnwGLcZrGfAvMcmOdIyJf47RP/8ndL+xR4H8islojRr5S1VnidLie7q76l6p+LSJDmrkXL/CUiHTF+TzuURvlyRjTOpanol/b8pTlKeOS+hpYY4wxxhhjjDFtYU0EjTHGGGOMMSZJrIBljDHGGGOMMUliBSxjjDHGGGOMSRIrYBljjDHGGGNMklgByxhjjDHGGGOSxApYxhhjjDHGGJMkVsAyxhhjjDHGmCT5/1ntj/jhtHpmAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "fig, axes = convergence_plot(\n", + " problems=problems,\n", + " results=results,\n", + " n_cols=2,\n", + " problem_subset=[\"rosenbrock_good_start\", \"box_3d\"],\n", + " distance_measure=\"parameter_distance\",\n", + " stopping_criterion=\"x\",\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3ad594cb", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/source/how_to_guides/optimization/index.rst b/docs/source/how_to_guides/optimization/index.rst index 414b6b067..755c30571 100644 --- a/docs/source/how_to_guides/optimization/index.rst +++ b/docs/source/how_to_guides/optimization/index.rst @@ -16,3 +16,4 @@ Optimization how_to_choose_optimizer how_to_scale_optimization_problems how_to_do_multistart_optimizations.ipynb + how_to_benchmark_optimization_algorithms.ipynb diff --git a/environment.yml b/environment.yml index 5b5622bac..277788709 100644 --- a/environment.yml +++ b/environment.yml @@ -25,7 +25,7 @@ dependencies: - numpy - pandas - pdbpp - - petsc4py>=3.11 + - petsc4py>=3.16.1 - pygmo - pytest - pytest-cov @@ -52,6 +52,6 @@ dependencies: - pre-commit - Py-BOBYQA - DFO-LS - - fides>=0.6.3 + - fides==0.6.3 - gif - gif[matplotlib] diff --git a/src/estimagic/__init__.py b/src/estimagic/__init__.py index 678f991a2..97cbfa969 100644 --- a/src/estimagic/__init__.py +++ b/src/estimagic/__init__.py @@ -1,4 +1,6 @@ from estimagic import utilities +from estimagic.benchmarking.get_benchmark_problems import get_benchmark_problems +from estimagic.benchmarking.run_benchmark import run_benchmark from estimagic.differentiation.derivatives import first_derivative from estimagic.estimation.estimate_ml import estimate_ml from estimagic.estimation.estimate_msm import estimate_msm @@ -6,6 +8,8 @@ from estimagic.inference.bootstrap import bootstrap from estimagic.optimization.optimize import maximize from estimagic.optimization.optimize import minimize +from estimagic.visualization.convergence_plot import convergence_plot +from estimagic.visualization.profile_plot import profile_plot try: from ._version import version as __version__ @@ -24,5 +28,9 @@ "estimate_msm", "estimate_ml", "get_moments_cov", + "run_benchmark", + "get_benchmark_problems", + "profile_plot", + "convergence_plot", "__version__", ] diff --git a/src/estimagic/benchmarking/__init__.py b/src/estimagic/benchmarking/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/src/estimagic/benchmarking/cartis_roberts.py b/src/estimagic/benchmarking/cartis_roberts.py new file mode 100644 index 000000000..7a8d00588 --- /dev/null +++ b/src/estimagic/benchmarking/cartis_roberts.py @@ -0,0 +1,721 @@ +"""Define the medium scale CUTEst Benchmark Set. + +This benchmark set is contains 60 test cases for nonlinear least squares +solvers. It was used to benchmark all modern model based non-linear +derivative free least squares solvers (e.g. POUNDERS, DFOGN, DFOLS). + +The parameter dimensions are of medium scale, varying between 25 and 100. + +The benchmark set is based on Table 3 in Cartis and Roberts (2019). +Implementation is based either on sources cited in the SIF files or +where available, on AMPL implementaions available here: +- https://vanderbei.princeton.edu/ampl/nlmodels/cute/index.html + + +""" +from functools import partial + +import numpy as np +from estimagic.benchmarking.more_wild import brown_almost_linear +from estimagic.benchmarking.more_wild import watson + + +def argtrig(x): + dim_in = len(x) + fvec = ( + dim_in + - np.sum(np.cos(x)) + + np.arange(1, dim_in + 1) * (1 - np.cos(x) - np.sin(x)) + ) + return fvec + + +def artif(x): + dim_in = len(x) + xvec = np.concatenate([[0], x, [0]]) + fvec = np.zeros(dim_in) + for i in range(dim_in): + fvec[i] = -0.05 * (xvec[i + 1] + xvec[i + 2] + xvec[i]) + np.arctan( + np.sin(np.mod(i + 1, 100) * xvec[i + 1]) + ) + return fvec + + +def arwhdne(x): + dim_in = len(x) + fvec = np.zeros(2 * (dim_in - 1)) + fvec[: dim_in - 1] = x[:-1] ** 2 + x[-1] ** 2 + fvec[dim_in - 1 :] = 4 * x[:-1] - 3 + return fvec + + +def bdvalues(x): + dim_in = len(x) + h = 1 / (dim_in + 1) + xvec = np.concatenate([[0], x, [0]]) + fvec = np.zeros(dim_in) + for i in range(2, dim_in + 2): + fvec[i - 2] = ( + -xvec[i - 2] + + 2 * xvec[i - 1] + - xvec[i] + + 0.5 * h ** 2 * (xvec[i - 1] + i * h + 1) ** 3 + ) + return fvec + + +def bratu_2d(x, alpha): + x = x.reshape((int(np.sqrt(len(x))), int(np.sqrt(len(x))))) + p = x.shape[0] + 2 + h = 1 / (p - 1) + c = h ** 2 * alpha + xvec = np.zeros((x.shape[0] + 2, x.shape[1] + 2)) + xvec[1 : x.shape[0] + 1, 1 : x.shape[1] + 1] = x + fvec = np.zeros_like(x) + for i in range(2, p): + for j in range(2, p): + fvec[i - 2, j - 2] = ( + 4 * xvec[i - 1, j - 1] + - xvec[i, j - 1] + - xvec[i - 2, j - 1] + - xvec[i - 1, j] + - xvec[i - 1, j - 2] + - c * np.exp(xvec[i - 1, j - 1]) + ) + return fvec.flatten() + + +def bratu_3d(x, alpha): + n = int(np.cbrt(len(x))) + x = x.reshape((n, n, n)) + p = x.shape[0] + 2 + h = 1 / (p - 1) + c = h ** 2 * alpha + xvec = np.zeros((x.shape[0] + 2, x.shape[1] + 2, x.shape[2] + 2)) + xvec[1 : x.shape[0] + 1, 1 : x.shape[1] + 1, 1 : x.shape[2] + 1] = x + fvec = np.zeros_like(x) + for i in range(2, p): + for j in range(2, p): + for k in range(2, p): + fvec[i - 2, j - 2, k - 2] = ( + 6 * xvec[i - 1, j - 1, k - 1] + - xvec[i, j - 1, k - 1] + - xvec[i - 2, j - 1, k - 1] + - xvec[i - 1, j, k - 1] + - xvec[i - 1, j - 2, k - 1] + - xvec[i - 1, j - 1, k] + - xvec[i - 1, j - 1, k - 2] + - c * np.exp(xvec[i, j, k]) + ) + return fvec.flatten() + + +def broydn_3d(x): + kappa_1 = 2 + kappa_2 = 1 + fvec = np.zeros_like(x) + fvec[0] = -2 * x[1] + kappa_2 + (3 - kappa_1 * x[0]) * x[0] + fvec[1 : len(x) - 1] = ( + -x[:-2] - 2 * x[2:] + kappa_2 + (3 - kappa_1 * x[1:-1]) * x[1:-1] + ) + fvec[-1] = -x[-2] + kappa_2 + (3 - kappa_1 * x[-1]) * x[-1] + return fvec + + +def broydn_bd(x): + dim_in = len(x) + fvec = np.zeros(dim_in) + for i in range(1, 1 + dim_in): + ji = [] + lb = np.max([1, i - 5]) + ub = np.min([dim_in, i + 1]) + for j in range(lb, ub + 1): + if j != i: + ji.append(j) + fvec[i - 1] = x[i - 1] * (2 + 5 * x[i - 1] ** 2) - np.sum( + x[np.array(ji) - 1] * (1 + x[np.array(ji) - 1]) + ) + return fvec + + +def cbratu_2d(x): + n = int(np.sqrt(len(x) / 2)) + x = x.reshape((2, n, n)) + xvec = np.zeros((x.shape[0], x.shape[1] + 2, x.shape[2] + 2)) + xvec[0, 1 : x.shape[1] + 1, 1 : x.shape[2] + 1] = x[0, :, :] + xvec[1, 1 : x.shape[1] + 1, 1 : x.shape[2] + 1] = x[1, :, :] + p = x.shape[1] + 2 + h = 1 / (p - 1) + alpha = 5 + c = h ** 2 * alpha + fvec = np.zeros_like(x) + for i in range(2, p): + for j in range(2, p): + fvec[0, i - 2, j - 2] = ( + 4 * xvec[0, i - 1, j - 1] + - xvec[0, i, j - 1] + - xvec[0, i - 2, j - 1] + - xvec[0, i - 1, j] + - xvec[0, i - 1, j - 2] + - c * np.exp(xvec[0, i - 1, j - 1]) * np.cos(xvec[0, i - 1, j - 1]) + ) + fvec[1, i - 2, j - 2] = ( + 4 * xvec[1, i - 1, j - 1] + - xvec[1, i, j - 1] + - xvec[1, i - 2, j - 1] + - xvec[1, i - 1, j] + - xvec[1, i - 1, j - 2] + - c * np.exp(xvec[1, i - 1, j - 1]) * np.sin(xvec[1, i - 1, j - 1]) + ) + return fvec.flatten() + + +def chandheq(x): + dim_in = len(x) + constant = 1 + w = np.ones(dim_in) / dim_in + h = np.ones(dim_in) + fvec = np.zeros(dim_in) + for i in range(dim_in): + fvec[i] = (-0.5 * constant * w * x[i] / (x[i] + x) * h[i] * h + h[i] - 1).sum() + return fvec + + +def chemrcta(x): + dim_in = int(len(x) / 2) + x = x.reshape((2, dim_in)) + # define the out vector + fvec = np.zeros(2 * dim_in) + # define some auxuliary params + pem = 1 + peh = 5.0 + d = 0.135 + b = 0.5 + beta = 2.0 + gamma = 25.0 + h = 1 / (dim_in - 1) + cu1 = -h * pem + cui1 = 1 / (h ** 2 * pem) + 1 / h + cui = -1 / h - 2 / (h ** 2 * pem) + ct1 = -h * peh + cti1 = 1 / (h ** 2 * peh) + 1 / h + cti = -beta - 1 / h - 2 / (h ** 2 * peh) + fvec[0] = cu1 * x[0, 1] - x[0, 0] + h * pem + fvec[1] = ct1 * x[1, 1] - x[1, 0] + h * peh + for i in range(2, dim_in): + fvec[i] = ( + -d * x[0, i - 1] * np.exp(gamma - gamma / x[1, i - 1]) + + (cui1) * x[0, i - 2] + + cui * x[0, i - 1] + + x[0, i] / (h ** 2 * pem) + ) + fvec[dim_in - 2 + i] = ( + b * d * x[0, i - 1] * np.exp(gamma - gamma / x[1, i - 1]) + + beta * x[1, i - 1] + + cti1 * x[1, i - 2] + + cti * x[1, i - 1] + + x[1, i] / (h ** 2 * peh) + ) + fvec[-2] = x[0, -1] - x[0, -2] + fvec[-1] = x[1, -1] - x[1, -2] + return fvec + + +def chemrctb(x): + dim_in = int(len(x)) + # define the out vector + fvec = np.zeros(dim_in) + # define some auxuliary params + pe = 5.0 + d = 0.135 + b = 0.5 + gamma = 25.0 + h = 1 / (dim_in - 1) + ct1 = -h * pe + cti1 = 1 / (h ** 2 * pe) + 1 / h + cti = -1 / h - 2 / (h ** 2 * pe) + fvec[0] = ct1 * x[1] - x[0] + h * pe + for i in range(2, dim_in): + fvec[i - 1] = ( + d * (b + 1 - x[i - 1]) * np.exp(gamma - gamma / x[i - 1]) + + cti1 * x[i - 2] + + cti * x[i - 1] + + x[i] / (h ** 2 * pe) + ) + fvec[-1] = x[-1] - x[-2] + return fvec + + +def chnrsbne(x): + alfa = np.array( + [ + 1.25, + 1.40, + 2.40, + 1.40, + 1.75, + 1.20, + 2.25, + 1.20, + 1.00, + 1.10, + 1.50, + 1.60, + 1.25, + 1.25, + 1.20, + 1.20, + 1.40, + 0.50, + 0.50, + 1.25, + 1.80, + 0.75, + 1.25, + 1.40, + 1.60, + 2.00, + 1.00, + 1.60, + 1.25, + 2.75, + 1.25, + 1.25, + 1.25, + 3.00, + 1.50, + 2.00, + 1.25, + 1.40, + 1.80, + 1.50, + 2.20, + 1.40, + 1.50, + 1.25, + 2.00, + 1.50, + 1.25, + 1.40, + 0.60, + 1.50, + ] + ) + dim_in = len(x) + fvec = np.zeros(2 * (dim_in - 1)) + fvec[: dim_in - 1] = 4 * alfa[1:] * (x[:-1] - x[1:] ** 2) + fvec[dim_in - 1 :] = x[1:] - 1 + return fvec + + +def drcavty(x, r): + m = int(np.sqrt(len(x))) + x = x.reshape((m, m)) + h = 1 / (m + 2) + xvec = np.zeros((m + 4, m + 4)) + xvec[2 : m + 2, 2 : m + 2] = x + xvec[-2, :] = -h / 2 + xvec[-1, :] = h / 2 + fvec = np.zeros_like(x) + for i in range(m): + for j in range(m): + fvec[i, j] = ( + 20 * xvec[i + 2, j + 2] + - 8 * xvec[i + 1, j + 2] + - 8 * xvec[i + 3, j + 2] + - 8 * xvec[i + 2, j + 1] + - 8 * xvec[i + 2, j + 3] + + 2 * xvec[i + 1, j + 3] + + 2 * xvec[i + 3, j + 2] + + 2 * xvec[i + 1, j + 1] + + 2 * xvec[i + 3, j + 3] + + xvec[i, j + 2] + + xvec[i + 4, j + 2] + + xvec[i + 2, j] + + xvec[i + 2, j + 4] + + (r / 4) + * (xvec[i + 2, j + 3] - xvec[i + 2, j + 1]) + * ( + xvec[i, j + 2] + + xvec[i + 1, j + 1] + + xvec[i + 1, j + 3] + - 4 * xvec[i + 1, j + 2] + - 4 * xvec[i + 3, j + 2] + - xvec[i + 3, j + 2] + - xvec[i + 3, j + 3] + - xvec[i + 4, j + 2] + ) + - (r / 4) + * (xvec[i + 3, j + 2] - xvec[i + 1, j + 2]) + * ( + xvec[i + 2, j] + + xvec[i + 1, j + 1] + + xvec[i + 3, j + 1] + - 4 * xvec[i + 2, j + 1] + - 4 * xvec[i + 2, j + 3] + - xvec[i + 1, j + 3] + - xvec[i + 3, j + 3] + - xvec[i + 2, j + 4] + ) + ) + + return fvec.flatten() + + +def freurone(x): + dim_in = len(x) + fvec = np.zeros((2, dim_in - 1)) + for i in range(dim_in - 1): + fvec[0, i] = (5.0 - x[i + 1]) * x[i + 1] ** 2 + x[i] - 2 * x[i + 1] - 13.0 + fvec[1, i] = (1.0 + x[i + 1]) * x[i + 1] ** 2 + x[i] - 14 * x[i + 1] - 29.0 + return fvec.flatten() + + +def hatfldg(x): + dim_in = len(x) + fvec = np.zeros(dim_in) + for i in range(1, dim_in - 1): + fvec[i - 1] = x[i] * (x[i - 1] - x[i + 1]) + x[i] - x[12] + 1 + fvec[-2] = x[0] - x[12] + 1 - x[0] * x[1] + fvec[-1] = x[-1] - x[12] + 1 + x[-2] * x[-1] + return fvec + + +def integreq(x): + dim_in = len(x) + h = 1 / (dim_in + 1) + t = np.arange(1, dim_in + 1) * h + xvec = np.concatenate([[0], x, [0]]) + fvec = np.zeros_like(x) + for i in range(1, dim_in): + fvec[i - 1] = ( + xvec[i] + + h + * ( + (1 - t[i - 1]) * (t[:i] * (xvec[1 : i + 1] + t[:i] + 1) ** 3).sum() + + t[i - 1] * ((1 - t[i:]) * (xvec[i + 1 : -1] + t[i:] + 1) ** 3).sum() + ) + / 2 + ) + fvec[-1] = ( + xvec[-2] + + h + * ( + (1 - t[-1]) * (t * (xvec[1:-1] + t + 1) ** 3).sum() + + t[-1] * ((1 - t[-1]) * (xvec[-2] + t[-1] + 1) ** 3) + ) + / 2 + ) + return fvec + + +def msqrta(x): + dim_in = int(np.sqrt(len(x))) + xmat = x.reshape((dim_in, dim_in)) + bmat = 5 * xmat + amat = np.zeros((dim_in, dim_in)) + for i in range(1, dim_in + 1): + for j in range(1, dim_in + 1): + amat[i - 1, j - 1] = (bmat[i - 1, :] * bmat[:, j - 1]).sum() + fmat = np.zeros((dim_in, dim_in)) + for i in range(1, dim_in + 1): + for j in range(1, dim_in + 1): + fmat[i - 1, j - 1] = (xmat[i - 1, :] * xmat[:, j - 1]).sum() - amat[ + i - 1, j - 1 + ] + return fmat.flatten() + + +def penalty_1(x, a=1e-5): + fvec = np.sqrt(a) * (x - 2) + fvec = np.concatenate([fvec, [x @ x - 1 / 4]]) + return fvec + + +def penalty_2(x, a=1e-10): + dim_in = len(x) + y = np.exp(np.arange(1, 2 * dim_in + 1) / 10) + np.exp(np.arange(2 * dim_in) / 10) + fvec = np.zeros(2 * dim_in) + fvec[0] = x[0] - 0.2 + fvec[1:dim_in] = np.sqrt(a) * ( + np.exp(x[1:] / 10) + np.exp(x[:-1] / 10) - y[1:dim_in] + ) + fvec[dim_in:-1] = np.sqrt(a) * (np.exp(x[1:] / 10) - np.exp(-1 / 10)) + fvec[-1] = (np.arange(1, dim_in + 1)[::-1] * x ** 2).sum() - 1 + return fvec + + +def vardimne(x): + dim_in = len(x) + fvec = np.zeros(dim_in + 2) + fvec[:-2] = x - 1 + fvec[-2] = (np.arange(1, dim_in + 1) * (x - 1)).sum() + fvec[-1] = ((np.arange(1, dim_in + 1) * (x - 1)).sum()) ** 2 + return fvec + + +def yatpsq_1(x, dim_in): + xvec = x[: dim_in ** 2] + xvec = xvec.reshape((dim_in, dim_in)) + yvec = x[dim_in ** 2 : dim_in ** 2 + dim_in] + zvec = x[dim_in ** 2 + dim_in : dim_in ** 2 + 2 * dim_in] + fvec = np.zeros((dim_in, dim_in)) + for i in range(dim_in): + for j in range(dim_in): + fvec[i, j] = ( + xvec[i, j] ** 3 + - 10 * xvec[i, j] ** 2 + - (yvec[i] + zvec[j]) + * (xvec[i, j] * np.cos(xvec[i, j]) - np.sin(xvec[i, j])) + ) + fvec = fvec.flatten() + temp = (np.sin(xvec) / xvec).sum(axis=0) - 1 + fvec = np.concatenate([fvec, temp]) + temp = (np.sin(xvec) / xvec).sum(axis=1) - 1 + fvec = np.concatenate([fvec, temp]) + return fvec + + +def yatpsq_2(x, dim_in): + xvec = x[: dim_in ** 2] + xvec = xvec.reshape((dim_in, dim_in)) + yvec = x[dim_in ** 2 : dim_in ** 2 + dim_in] + zvec = x[dim_in ** 2 + dim_in : dim_in ** 2 + 2 * dim_in] + fvec = np.zeros((dim_in, dim_in)) + for i in range(dim_in): + for j in range(dim_in): + fvec[i, j] = xvec[i, j] - (yvec[i] + zvec[j]) * (1 + np.cos(xvec[i, j])) - 1 + fvec = fvec.flatten() + temp = (np.sin(xvec) + xvec).sum(axis=0) - 1 + fvec = np.concatenate([fvec, temp]) + temp = (np.sin(xvec) + xvec).sum(axis=1) - 1 + fvec = np.concatenate([fvec, temp]) + return fvec + + +def get_start_points_msqrta(dim_in, flag=1): + bmat = np.zeros((dim_in, dim_in)) + for i in range(1, dim_in + 1): + for j in range(1, dim_in + 1): + bmat[i - 1, j - 1] = np.sin(((i - 1) * dim_in + j) ** 2) + if flag == 2: + bmat[2, 0] = 0 + xmat = 0.2 * bmat + return xmat.flatten() + + +def get_start_points_bdvalues(n): + h = 1 / (n + 1) + x = np.zeros(n) + for i in range(n): + x[i] = (i + 1) * h * ((i + 1) * h - 1) + return x + + +CARTIS_ROBERTS_PROBLEMS = { + "argtrig": { + "criterion": argtrig, + "start_x": np.ones(100) / 100, + "solution_x": None, + "start_criterion": 32.99641, + "solution_criterion": 0, + }, + "artif": { + "criterion": artif, + "start_x": np.ones(100), + "solution_x": None, + "start_criterion": 36.59115, + "solution_criterion": 0, + }, + "arwhdne": { + "criterion": arwhdne, + "start_x": np.ones(100), + "solution_x": None, + "start_criterion": 495, + "solution_criterion": 27.66203, + }, + "bdvalues": { + "criterion": bdvalues, + "start_x": 1000 * get_start_points_bdvalues(100), + "solution_x": None, + "start_criterion": 1.943417e7, + "solution_criterion": 0, + }, + "bratu_2d": { + "criterion": partial(bratu_2d, alpha=4), + "start_x": np.zeros(64), + "solution_x": None, + "start_criterion": 0.1560738, + "solution_criterion": 0, + }, + "bratu_2d_t": { + "criterion": partial(bratu_2d, alpha=6.80812), + "start_x": np.zeros(64), + "solution_x": None, + "start_criterion": 0.4521311, + "solution_criterion": 1.853474e-5, + }, + "bratu_3d": { + "criterion": partial(bratu_3d, alpha=6.80812), + "start_x": np.zeros(27), + "solution_x": None, + "start_criterion": 4.888529, + "solution_criterion": 0, + }, + "brownale": { + "criterion": brown_almost_linear, + "start_x": 0.5 * np.ones(100), + "solution_x": None, + "start_criterion": 2.524757e5, + "solution_criterion": 0, + }, + "broydn_3d": { + "criterion": broydn_3d, + "start_x": -np.ones(100), + "solution_x": None, + "start_criterion": 111, + "solution_criterion": 0, + }, + "cbratu_2d": { + "criterion": cbratu_2d, + "start_x": np.zeros(2 * 5 * 5), + "solution_x": None, + "start_criterion": 0.4822531, + "solution_criterion": 0, + }, + "broydn_bd": { + "criterion": broydn_bd, + "start_x": np.ones(100), + "solution_x": None, + "start_criterion": 2404, + "solution_criterion": 0, + }, + "chandheq": { + "criterion": chandheq, + "start_x": np.arange(1, 101) / 100, + "solution_x": None, + "start_criterion": 6.923365, + "solution_criterion": 0, + }, + "chemrcta": { + "criterion": chemrcta, + "start_x": np.ones(100), + "solution_x": None, + "start_criterion": 3.0935, + "solution_criterion": 0, + "lower_bounds": np.concatenate([np.zeros(50), 1e-6 * np.ones(50)]), + }, + "chemrctb": { + "criterion": chemrctb, + "start_x": np.ones(100), + "solution_x": None, + "start_criterion": 1.446513, + "solution_criterion": 1.404424e-3, + "lower_bounds": 1e-6 * np.ones(100), + }, + "chnrsbne": { + "criterion": chnrsbne, + "start_x": -np.ones(50), + "solution_x": None, + "start_criterion": 7635.84, + "solution_criterion": 0, + }, + "drcavty1": { + "criterion": partial(drcavty, r=500), + "start_x": np.zeros(100), + "solution_x": None, + "start_criterion": 0.4513889, + "solution_criterion": 0, + }, + "drcavty2": { + "criterion": partial(drcavty, r=1000), + "start_x": np.zeros(100), + "solution_x": None, + "start_criterion": 0.4513889, + "solution_criterion": 5.449602e-3, + }, + "drcavty3": { + "criterion": partial(drcavty, r=4500), + "start_x": np.zeros(100), + "solution_x": None, + "start_criterion": 0.4513889, + "solution_criterion": 0, + }, + "freurone": { + "criterion": freurone, + "start_x": np.concatenate([np.array([0.5, -2]), np.zeros(98)]), + "solution_x": None, + "start_criterion": 9.95565e4, + "solution_criterion": 1.196458e4, + }, + "hatfldg": { + "criterion": hatfldg, + "start_x": np.ones(25), + "solution_x": None, + "start_criterion": 27, + "solution_criterion": 10, + }, + "integreq": { + "criterion": integreq, + "start_x": np.arange(1, 101) / 101 * (np.arange(1, 101) / 101 - 1), + "solution_x": None, + "start_criterion": 0.5730503, + "solution_criterion": 0, + }, + "msqrta": { + "criterion": msqrta, + "start_x": get_start_points_msqrta(10), + "solution_x": None, + "start_criterion": 212.7162, + "solution_criterion": 0, + }, + "msqrtb": { + "criterion": msqrta, + "start_x": get_start_points_msqrta(10, flag=2), + "solution_x": None, + "start_criterion": 205.0753, + "solution_criterion": 0, + }, + "penalty_1": { + "criterion": penalty_1, + "start_x": np.arange(1, 101), + "solution_x": None, + "start_criterion": 1.144806e11, + "solution_criterion": 9.025000e-9, + }, + "penalty_2": { + "criterion": penalty_2, + "start_x": np.ones(100) * 0.5, + "solution_x": None, + "start_criterion": 1.591383e6, + "solution_criterion": 0.9809377, + }, + "vardimne": { + "criterion": vardimne, + "start_x": 1 - np.arange(1, 101) / 100, + "solution_x": None, + "start_criterion": 1.310584e14, + "solution_criterion": 0, + }, + "watsonne": { + "criterion": watson, + "start_x": np.zeros(31), + "solution_x": None, + "start_criterion": 30, + "solution_criterion": 0, + }, + "yatpsq_1": { + "criterion": partial(yatpsq_1, dim_in=10), + "start_x": np.concatenate([np.ones(100) * 6, np.zeros(20)]), + "solution_x": None, + "start_criterion": 2.073643e6, + "solution_criterion": 0, + }, + "yatpsq_2": { + "criterion": partial(yatpsq_2, dim_in=10), + "start_x": np.concatenate([np.ones(100) * 10, np.zeros(20)]), + "solution_x": None, + "start_criterion": 1.831687e5, + "solution_criterion": 0, + }, +} diff --git a/src/estimagic/benchmarking/get_benchmark_problems.py b/src/estimagic/benchmarking/get_benchmark_problems.py new file mode 100644 index 000000000..9ef7b12dd --- /dev/null +++ b/src/estimagic/benchmarking/get_benchmark_problems.py @@ -0,0 +1,255 @@ +import warnings +from functools import partial + +import numpy as np +import pandas as pd +from estimagic.benchmarking.cartis_roberts import CARTIS_ROBERTS_PROBLEMS +from estimagic.benchmarking.more_wild import MORE_WILD_PROBLEMS +from estimagic.benchmarking.noise_distributions import NOISE_DISTRIBUTIONS + + +def get_benchmark_problems( + name, + additive_noise=False, + additive_noise_options=None, + multiplicative_noise=False, + multiplicative_noise_options=None, +): + """Get a dictionary of test problems for a benchmark. + + Args: + name (str): The name of the set of test problems. Currently "more_wild" + is the only supported one. + additive_noise (bool): Whether to add additive noise to the problem. + Default False. + additive_noise_options (dict or None): Specifies the amount and distribution + of the addititve noise added to the problem. Has the entries: + - distribition (str): One of "normal", "gumbel", "uniform", "logistic", + "laplace". Default "normal". + - std (float): The standard deviation of the noise. This works for all + distributions, even if those distributions are normally not specified + via a standard deviation (e.g. uniform). + - correlation (float): Number between 0 and 1 that specifies the auto + correlation of the noise. + multiplicative_noise (bool): Whether to add multiplicative noise to the problem. + Default False. + multiplicative_noise_options (dict or None): Specifies the amount and + distribition of the multiplicative noise added to the problem. Has entries: + - distribition (str): One of "normal", "gumbel", "uniform", "logistic", + "laplace". Default "normal". + - std (float): The standard deviation of the noise. This works for all + distributions, even if those distributions are normally not specified + via a standard deviation (e.g. uniform). + - correlation (float): Number between 0 and 1 that specifies the auto + correlation of the noise. + - clipping_value (float): A non-negative float. Multiplicative noise + becomes zero if the function value is zero. To avoid this, we do not + implement multiplicative noise as `f_noisy = f * epsilon` but by + `f_noisy` = f + (epsilon - 1) * f_clipped` where f_clipped is bounded + away from zero from both sides by the clipping value. + + Returns: + dict: Nested dictionary with benchmark problems of the structure: + {"name": {"inputs": {...}, "solution": {...}, "info": {...}}} + where "inputs" are keyword arguments for ``minimize`` such as the criterion + function and start parameters. "solution" contains the entries "params" and + "value" and "info" might contain information about the test problem. + + """ + raw_problems = _get_raw_problems(name) + + if additive_noise: + additive_options = _process_noise_options(additive_noise_options, False) + else: + additive_options = None + + if multiplicative_noise: + multiplicative_options = _process_noise_options( + multiplicative_noise_options, True + ) + else: + multiplicative_options = None + + problems = {} + for name, specification in raw_problems.items(): + inputs = _create_problem_inputs( + specification, + additive_options=additive_options, + multiplicative_options=multiplicative_options, + ) + + problems[name] = { + "inputs": inputs, + "solution": _create_problem_solution(specification), + "info": specification.get("info", {}), + } + + return problems + + +def _get_raw_problems(name): + if name == "more_wild": + raw_problems = MORE_WILD_PROBLEMS + elif name == "cartis_roberts": + warnings.warn( + "Only a subset of the cartis_roberts benchmark suite is currently " + "implemented. Do not use this for any published work." + ) + raw_problems = CARTIS_ROBERTS_PROBLEMS + elif name == "example": + subset = { + "linear_full_rank_good_start", + "rosenbrock_good_start", + "helical_valley_good_start", + "powell_singular_good_start", + "freudenstein_roth_good_start", + "bard_good_start", + "box_3d", + "jenrich_sampson", + "brown_dennis_good_start", + "chebyquad_6", + "bdqrtc_8", + "mancino_5_good_start", + } + raw_problems = {k: v for k, v in MORE_WILD_PROBLEMS.items() if k in subset} + else: + raise NotImplementedError() + return raw_problems + + +def _create_problem_inputs(specification, additive_options, multiplicative_options): + _criterion = partial( + _internal_criterion_template, + criterion=specification["criterion"], + additive_options=additive_options, + multiplicative_options=multiplicative_options, + ) + _x = specification["start_x"] + + _params = pd.DataFrame(_x.reshape(-1, 1), columns=["value"]) + + inputs = {"criterion": _criterion, "params": _params} + return inputs + + +def _create_problem_solution(specification): + _solution_x = specification.get("solution_x") + if _solution_x is None: + _solution_x = specification["start_x"] * np.nan + + _params = pd.DataFrame(_solution_x.reshape(-1, 1), columns=["value"]) + _value = specification["solution_criterion"] + + solution = { + "params": _params, + "value": _value, + } + return solution + + +def _internal_criterion_template( + params, criterion, additive_options, multiplicative_options +): + x = params["value"].to_numpy() + critval = criterion(x) + + noise = _get_combined_noise( + critval, + additive_options=additive_options, + multiplicative_options=multiplicative_options, + ) + + noisy_critval = critval + noise + + if isinstance(noisy_critval, np.ndarray): + out = { + "root_contributions": noisy_critval, + "value": noisy_critval @ noisy_critval, + } + else: + out = noisy_critval + return out + + +def _get_combined_noise(fval, additive_options, multiplicative_options): + size = len(np.atleast_1d(fval)) + if multiplicative_options is not None: + options = multiplicative_options.copy() + std = options.pop("std") + clipval = options.pop("clipping_value") + scaled_std = std * _clip_away_from_zero(fval, clipval) + multiplicative_noise = _sample_from_distribution( + **options, std=scaled_std, size=size + ) + else: + multiplicative_noise = 0 + + if additive_options is not None: + additive_noise = _sample_from_distribution(**additive_options, size=size) + else: + additive_noise = 0 + + return multiplicative_noise + additive_noise + + +def _sample_from_distribution(distribution, mean, std, size, correlation=0): + sample = NOISE_DISTRIBUTIONS[distribution](size=size) + dim = size if isinstance(size, int) else size[1] + if correlation != 0 and dim > 1: + chol = np.linalg.cholesky(np.diag(np.ones(dim) - correlation) + correlation) + sample = (chol @ sample.T).T + sample = sample / sample.std() + sample *= std + sample += mean + return sample + + +def _process_noise_options(options, is_multiplicative): + options = {} if options is None else options + + defaults = {"std": 0.01, "distribution": "normal", "correlation": 0, "mean": 0} + if is_multiplicative: + defaults["clipping_value"] = 1 + + processed = { + **defaults, + **options, + } + + distribution = processed["distribution"] + if distribution not in NOISE_DISTRIBUTIONS: + raise ValueError( + f"Invalid distribution: {distribution}. " + "Allowed are {list(NOISE_DISTRIBUTIONS)}" + ) + + std = processed["std"] + if std < 0: + raise ValueError(f"std must be non-negative. Not: {std}") + + corr = processed["correlation"] + if corr < 0: + raise ValueError(f"corr must be non-negative. Not: {corr}") + + if is_multiplicative: + clipping_value = processed["clipping_value"] + if clipping_value < 0: + raise ValueError( + f"clipping_value must be non-negative. Not: {clipping_value}" + ) + + return processed + + +def _clip_away_from_zero(a, clipval): + is_scalar = np.isscalar(a) + a = np.atleast_1d(a) + + is_positive = a >= 0 + + clipped = np.where(is_positive, np.clip(a, clipval, np.inf), a) + clipped = np.where(~is_positive, np.clip(clipped, -np.inf, -clipval), clipped) + + if is_scalar: + clipped = clipped[0] + return clipped diff --git a/src/estimagic/benchmarking/more_wild.py b/src/estimagic/benchmarking/more_wild.py new file mode 100644 index 000000000..a0534dc79 --- /dev/null +++ b/src/estimagic/benchmarking/more_wild.py @@ -0,0 +1,1186 @@ +"""Define the More-Wild Benchmark Set. + +This benchmark set is contains 53 test cases for nonlinear least squares solvers. +The test cases are built out of 22 functions, originally derived from the CUTEr +Problems. It was used to benchmark all modern model based non-linear derivative +free least squares solvers (e.g. POUNDERS, DFOGN, DFOLS). + +The parameter dimensions are quite small, varying between 2 and 12. + +The benchmark set was first described In More and Wild, 2009. Fortran and Matlab Code +is available here. We use the following sources of information to construct the +benchmark set: + +- https://www.mcs.anl.gov/~more/dfo/fortran/dfovec.f for the function implementation +- https://www.mcs.anl.gov/~more/dfo/fortran/dfoxs.f for the base starting points +- https://www.mcs.anl.gov/~more/dfo/fortran/dfo.dat for: + - The mapping test cases to criterion functions (column 1) + - The dimensionalities of parameter vectors (column 2) + - The dimensionalities of the output (column 3) + - Whether the base start vector is multiplied by a factor of ten or not (column 4). + +""" +from functools import partial + +import numpy as np + + +def linear_full_rank(x, dim_out): + temp = 2 * x.sum() / dim_out + 1 + out = np.full(dim_out, -temp) + out[: len(x)] += x + return out + + +def linear_rank_one(x, dim_out): + dim_in = len(x) + sm = np.arange(1, dim_in + 1) @ x + fvec = np.arange(1, dim_out + 1) * sm - 1.0 + return fvec + + +def linear_rank_one_zero_columns_rows(x, dim_out): + dim_in = len(x) + sm = (np.arange(2, dim_in) * x[1:-1]).sum() + fvec = np.arange(dim_out) * sm - 1.0 + fvec[-1] = -1.0 + return fvec + + +def rosenbrock(x): + fvec = np.zeros(2) + fvec[0] = 10 * (x[1] - x[0] ** 2) + fvec[1] = 1.0 - x[0] + return fvec + + +def helical_valley(x): + temp = 8 * np.arctan(1.0) + temp1 = np.sign(x[1]) * 0.25 + if x[0] > 0: + temp1 = np.arctan(x[1] / x[0]) / temp + elif x[0] < 0: + temp1 = np.arctan(x[1] / x[0]) / temp + 0.5 + temp2 = np.sqrt(x[0] ** 2 + x[1] ** 2) + fvec = np.zeros(3) + fvec[0] = 10 * (x[2] - 10 * temp1) + fvec[1] = 10 * (temp2 - 1.0) + fvec[2] = x[2] + return fvec + + +def powell_singular(x): + fvec = np.zeros(4) + fvec[0] = x[0] + 10 * x[1] + fvec[1] = np.sqrt(5.0) * (x[2] - x[3]) + fvec[2] = (x[1] - 2 * x[2]) ** 2 + fvec[3] = np.sqrt(10.0) * (x[0] - x[3]) ** 2 + return fvec + + +def freudenstein_roth(x): + fvec = np.zeros(2) + fvec[0] = -13 + x[0] + ((5 - x[1]) * x[1] - 2) * x[1] + fvec[1] = -29 + x[0] + ((1.0 + x[1]) * x[1] - 14) * x[1] + return fvec + + +def bard(x, y): + fvec = np.zeros(len(y)) + for i in range(1, round(len(y) / 2) + 1): + temp = len(y) + 1 - i + fvec[i - 1] = y[i - 1] - (x[0] + i / (x[1] * temp + x[2] * i)) + for i in range(round(len(y) / 2) + 1, len(y) + 1): + temp = len(y) + 1 - i + fvec[i - 1] = y[i - 1] - (x[0] + i / (x[1] * temp + x[2] * temp)) + return fvec + + +def kowalik_osborne(x, y1, y2): + temp1 = y1 * (y1 + x[1]) + temp2 = y1 * (y1 + x[2]) + x[3] + fvec = y2 - x[0] * temp1 / temp2 + return fvec + + +def meyer(x, y): + temp = 5 * np.arange(1, len(y) + 1) + 45 + x[2] + temp1 = x[1] / temp + temp2 = np.exp(temp1) + fvec = x[0] * temp2 - y + return fvec + + +def watson(x): + dim_in = len(x) + fvec = np.zeros(31) + for i in range(1, 30): + temp = i / 29 + sum_1 = (np.arange(1, dim_in) * temp ** np.arange(dim_in - 1) * x[1:]).sum() + sum_2 = (temp ** np.arange(dim_in) * x).sum() + fvec[i - 1] = sum_1 - sum_2 ** 2 - 1.0 + fvec[29] = x[0] + fvec[30] = x[1] - x[0] ** 2 - 1.0 + return fvec + + +def box_3d(x, dim_out): + fvec = np.zeros(dim_out) + for i in range(1, dim_out + 1): + fvec[i - 1] = ( + np.exp(-i / 10 * x[0]) + - np.exp(-i / 10 * x[1]) + + (np.exp(-i) - np.exp(-i / 10)) * x[2] + ) + return fvec + + +def jennrich_sampson(x, dim_out): + fvec = ( + 2 * (1.0 + np.arange(1, dim_out + 1)) + - np.exp(np.arange(1, dim_out + 1) * x[0]) + - np.exp(np.arange(1, dim_out + 1) * x[1]) + ) + return fvec + + +def brown_dennis(x, dim_out): + fvec = np.zeros(dim_out) + for i in range(1, dim_out + 1): + temp = i / 5 + temp_1 = x[0] + temp * x[1] - np.exp(temp) + temp_2 = x[2] + np.sin(temp) * x[3] - np.cos(temp) + fvec[i - 1] = temp_1 ** 2 + temp_2 ** 2 + return fvec + + +def chebyquad(x, dim_out): + fvec = np.zeros(dim_out) + dim_in = len(x) + for i in range(1, dim_in + 1): + temp_1 = 1.0 + temp_2 = 2 * x[i - 1] - 1.0 + temp_3 = 2 * temp_2 + for j in range(dim_out): + fvec[j] = fvec[j] + temp_2 + temp_4 = temp_3 * temp_2 - temp_1 + temp_1 = temp_2 + temp_2 = temp_4 + for i in range(1, dim_out + 1): + fvec[i - 1] = fvec[i - 1] / dim_in + if i % 2 == 0: + fvec[i - 1] = fvec[i - 1] + 1 / (i ** 2 - 1.0) + return fvec + + +def brown_almost_linear(x): + dim_in = len(x) + sm = -(dim_in + 1) + x.sum() + product = x.prod() + fvec = x + sm + fvec[dim_in - 1] = product - 1.0 + return fvec + + +def osborne_one(x, y): + temp = 10 * np.arange(len(y)) + temp_1 = np.exp(-x[3] * temp) + temp_2 = np.exp(-x[4] * temp) + fvec = y - (x[0] + x[1] * temp_1 + x[2] * temp_2) + return fvec + + +def osborne_two(x, y): + temp_array = np.zeros((4, len(y))) + temp = np.arange(len(y)) / 10 + temp_array[0] = np.exp(-x[4] * temp) + temp_array[1] = np.exp(-x[5] * (temp - x[8]) ** 2) + temp_array[2] = np.exp(-x[6] * (temp - x[9]) ** 2) + temp_array[3] = np.exp(-x[7] * (temp - x[10]) ** 2) + fvec = y - (temp_array.T * x[:4]).T.sum(axis=0) + return fvec + + +def bdqrtic(x): + # the length of array x should be more then 5. + dim_in = len(x) + fvec = np.zeros(2 * (dim_in - 4)) + for i in range(dim_in - 4): + fvec[i] = -4 * x[i] + 3 + fvec[dim_in - 4 + i] = ( + x[i] ** 2 + + 2 * x[i + 1] ** 2 + + 3 * x[i + 2] ** 2 + + 4 * x[i + 3] ** 2 + + 5 * x[dim_in - 1] ** 2 + ) + return fvec + + +def cube(x): + fvec = 10 * (x - np.roll(x, 1) ** 3) + fvec[0] = x[0] - 1.0 + return fvec + + +def mancino(x): + dim_in = len(x) + fvec = np.zeros(dim_in) + for i in range(dim_in): + sm = 0 + for j in range(dim_in): + temp = np.sqrt(x[i] ** 2 + (i + 1) / (j + 1)) + sm += temp * ((np.sin(np.log(temp))) ** 5 + (np.cos(np.log(temp))) ** 5) + fvec[i] = 1400 * x[i] + (i + 1 - 50) ** 3 + sm + return fvec + + +def heart_eight(x, y): + dim_y = len(y) + fvec = np.zeros(dim_y) + fvec[0] = x[0] + x[1] - y[0] + fvec[1] = x[2] + x[3] - y[1] + fvec[2] = x[4] * x[0] + x[5] * x[1] - x[6] * x[2] - x[7] * x[3] - y[2] + fvec[3] = x[6] * x[0] + x[7] * x[1] + x[4] * x[2] + x[5] * x[3] - y[3] + fvec[4] = ( + x[0] * (x[4] ** 2 - x[6] ** 2) + - 2 * x[2] * x[4] * x[6] + + x[1] * (x[5] ** 2 - x[7] ** 2) + - 2 * x[3] * x[5] * x[7] + - y[4] + ) + fvec[5] = ( + x[2] * (x[4] ** 2 - x[6] ** 2) + + 2 * x[0] * x[4] * x[6] + + x[3] * (x[5] ** 2 - x[7] ** 2) + + 2 * x[1] * x[5] * x[7] + - y[5] + ) + fvec[6] = ( + x[0] * x[4] * (x[4] ** 2 - 3 * x[6] ** 2) + + x[2] * x[6] * (x[6] ** 2 - 3 * x[4] ** 2) + + x[1] * x[5] * (x[5] ** 2 - 3 * x[7] ** 2) + + x[3] * x[7] * (x[7] ** 2 - 3 * x[5] ** 2) + - y[6] + ) + fvec[7] = ( + x[2] * x[4] * (x[4] ** 2 - 3 * x[6] ** 2) + - x[0] * x[6] * (x[6] ** 2 - 3 * x[4] ** 2) + + x[3] * x[5] * (x[5] ** 2 - 3 * x[7] ** 2) + - x[1] * x[7] * (x[7] ** 2 - 3 * x[5] ** 2) + - y[7] + ) + return fvec + + +def get_start_points_mancino(n): + x = np.zeros(n) + for i in range(1, n + 1): + sm = 0 + for j in range(1, n + 1): + sm += np.sqrt(i / j) * ( + (np.sin(np.log(np.sqrt(i / j)))) ** 5 + + (np.cos(np.log(np.sqrt(i / j)))) ** 5 + ) + x[i - 1] = -8.7110e-04 * ((i - 50) ** 3 + sm) + return x + + +y_vec = np.array( + [ + 0.1400, + 0.1800, + 0.2200, + 0.2500, + 0.2900, + 0.3200, + 0.3500, + 0.3900, + 0.3700, + 0.5800, + 0.7300, + 0.9600, + 1.3400, + 2.1000, + 4.3900, + ] +) + +v_vec = np.array( + [ + 4.0000, + 2.0000, + 1.0000, + 0.5000, + 0.2500, + 0.1670, + 0.1250, + 0.1000, + 0.0833, + 0.0714, + 0.0625, + ] +) + +y2_vec = np.array( + [ + 0.1957, + 0.1947, + 0.1735, + 0.1600, + 0.0844, + 0.0627, + 0.0456, + 0.0342, + 0.0323, + 0.0235, + 0.0246, + ] +) + +y3_vec = np.array( + [ + 34780, + 28610, + 23650, + 19630, + 16370, + 13720, + 11540, + 9744, + 8261, + 7030, + 6005, + 5147, + 4427, + 3820, + 3307, + 2872, + ] +) +y4_vec = np.array( + [ + 8.44e-1, + 9.08e-1, + 9.32e-1, + 9.36e-1, + 9.25e-1, + 9.08e-1, + 8.81e-1, + 8.5e-1, + 8.18e-1, + 7.84e-1, + 7.51e-1, + 7.18e-1, + 6.85e-1, + 6.58e-1, + 6.28e-1, + 6.03e-1, + 5.8e-1, + 5.58e-1, + 5.38e-1, + 5.22e-1, + 5.06e-1, + 4.9e-1, + 4.78e-1, + 4.67e-1, + 4.57e-1, + 4.48e-1, + 4.38e-1, + 4.31e-1, + 4.24e-1, + 4.2e-1, + 4.14e-1, + 4.11e-1, + 4.06e-1, + ] +) +y5_vec = np.array( + [ + 1.366e0, + 1.191e0, + 1.112e0, + 1.013e0, + 9.91e-1, + 8.85e-1, + 8.31e-1, + 8.47e-1, + 7.86e-1, + 7.25e-1, + 7.46e-1, + 6.79e-1, + 6.08e-1, + 6.55e-1, + 6.16e-1, + 6.06e-1, + 6.02e-1, + 6.26e-1, + 6.51e-1, + 7.24e-1, + 6.49e-1, + 6.49e-1, + 6.94e-1, + 6.44e-1, + 6.24e-1, + 6.61e-1, + 6.12e-1, + 5.58e-1, + 5.33e-1, + 4.95e-1, + 5.0e-1, + 4.23e-1, + 3.95e-1, + 3.75e-1, + 3.72e-1, + 3.91e-1, + 3.96e-1, + 4.05e-1, + 4.28e-1, + 4.29e-1, + 5.23e-1, + 5.62e-1, + 6.07e-1, + 6.53e-1, + 6.72e-1, + 7.08e-1, + 6.33e-1, + 6.68e-1, + 6.45e-1, + 6.32e-1, + 5.91e-1, + 5.59e-1, + 5.97e-1, + 6.25e-1, + 7.39e-1, + 7.1e-1, + 7.29e-1, + 7.2e-1, + 6.36e-1, + 5.81e-1, + 4.28e-1, + 2.92e-1, + 1.62e-1, + 9.8e-2, + 5.4e-2, + ] +) + + +linear_full_rank_solution_x = np.array( + [ + -0.9999999988839997, + -1.0000000177422066, + -1.0000000115935452, + -1.0000000228208163, + -1.0000000488884697, + -0.9999999970458138, + -0.999999957053959, + -1.0000000040514776, + -0.9999999708374043, + ] +) + +freudenstein_roth_solution_x = np.array([11.4127789219781, -0.8968052599835741]) + + +bard_solution_x = np.array( + [0.08241056005476516, 1.1330360796060677, 2.3436951913379658] +) + +kowalik_osborne_solution_x = np.array( + [0.19280693401647758, 0.19128233030789646, 0.12305650338704374, 0.1360623315234073] +) + + +meyer_solution_x = np.array( + [0.005609636453940975, 6181.3463491557495, 345.22363473367955] +) + +watson_6_solution_x = np.array( + [ + -0.01572508595814696, + 1.0124348692251488, + -0.23299161822960684, + 1.2604300607312298, + -1.5137288869025518, + 0.9929964192277573, + ] +) + + +# Note: only nlopt_neldermead got close to the correct optimal criterion value. +# Parameter values might be less precise than others but should be precise enough +# for all practical purposes. +watson_9_solution_x = np.array( + [ + -1.5307729818292037e-05, + 0.9997897038761921, + 0.014763956456196943, + 0.14634240306061744, + 1.000820801996808, + -2.617730533377693, + 4.104402503186126, + -3.1436119083184844, + 1.0526263240326197, + ] +) + +# Note: only nlopt_nobyqa got close to the correct optimal criterion value. +# Parameter values might be less precise than others but should be precise enough +# for all practical purposes. +watson_12_solution_x = np.array( + [ + -1.257374334661004e-07, + 1.000009574359581, + -0.0005801330054146337, + 0.339181153679104, + -0.01717885040751319, + 0.1133023927390161, + 0.19016852711009063, + -0.21697797575421524, + -0.20528305553311146, + 0.9344814896242725, + -0.8979508634897754, + 0.3182351206188577, + ] +) + +brown_dennis_solution_x = np.array( + [-11.594439969349615, 13.203630099554186, -0.40343943943781074, 0.2367787758603151] +) + +chebyquad_6_solution_x = np.array( + [ + 0.06687659094608964, + 0.2887406731194441, + 0.36668229924164747, + 0.6333177007583523, + 0.7112593268805555, + 0.9331234090539102, + ] +) + +chebyquad_7_solution_x = np.array( + [ + 0.0580691496209753, + 0.23517161235742137, + 0.3380440947400461, + 0.49999999999999983, + 0.6619559052599537, + 0.7648283876425783, + 0.9419308503790245, + ] +) + +chebyquad_8_solution_x = np.array( + [ + 0.043152760689960816, + 0.19309084165259105, + 0.2663287079773684, + 0.5000000016286815, + 0.5000000007226908, + 0.8069091602434582, + 0.7336712939109635, + 0.9568472402172841, + ] +) + +chebyquad_9_solution_x = np.array( + [ + 0.04420534613578318, + 0.19949067230988682, + 0.23561910847105574, + 0.4160469078926057, + 0.5839530921074088, + 0.4999999999999922, + 0.800509327690123, + 0.7643808915289372, + 0.9557946538642177, + ] +) + +chebyquad_10_solution_x = np.array( + [ + 0.07474816709152399, + 0.17151817795786592, + 0.28643415454482585, + 0.35964645053932914, + 0.4707505262783716, + 0.6167383355304029, + 0.6167383367837294, + 0.7998108031241883, + 0.844854641539109, + 0.9670066274628275, + ] +) + +chebyquad_11_solution_x = np.array( + [ + 0.02995874447661457, + 0.1373112070822553, + 0.18836638791417698, + 0.3588431173822416, + 0.3588431160884765, + 0.5000000000242054, + 0.6411568833224512, + 0.6411568815391566, + 0.8116336110470005, + 0.8626887929155374, + 0.9700412549151204, + ] +) + +osborne_one_solution_x = np.array( + [ + 0.37541005253870485, + 1.9358469347077125, + -1.4646871598379403, + 0.012867534697214533, + 0.02212269960299629, + ] +) + +osborne_two_solution_x = np.array( + [ + 1.3099771555174913, + 0.4315537955622272, + 0.6336616986693765, + 0.5994305344293098, + 0.7541832304802704, + 0.9042885759622441, + 1.365811821857166, + 4.823698851312894, + 2.398684862961737, + 4.568874597996633, + 5.675341470445994, + ] +) + +bdqrtic_8_solution_x = np.array( + [ + 0.616075443630495, + 0.4861767187980861, + 0.39190293828200784, + 0.32635052133139375, + 5.7665311977077046e-09, + 9.348707442258251e-09, + 7.066347917413364e-09, + -2.030598138768078e-09, + ] +) + +bdqrtic_10_solution_x = np.array( + [ + 0.6255364749479968, + 0.4851009828850974, + 0.3671943518989714, + 0.28518847760113386, + 0.33016716122418716, + 0.37757199483645576, + -3.24040819296658e-09, + -1.8973118921921425e-08, + -2.2244236071548075e-08, + 1.9263207246002433e-09, + ] +) + + +bdqrtic_11_solution_x = np.array( + [ + 0.6251418193253757, + 0.4858196102070445, + 0.3712502347939938, + 0.28350403794642487, + 0.31694697562905494, + 0.33873300184720523, + 0.3759208995980027, + -1.8942209640948616e-08, + 3.418631657404969e-08, + -4.003185000628104e-09, + 3.166166094063382e-09, + ] +) + + +bdqrtic_12_solution_x = np.array( + [ + 0.6248003622228653, + 0.48537650602979937, + 0.37165912289534886, + 0.2859718523039759, + 0.31552001728813406, + 0.3253724392486982, + 0.33781861543778574, + 0.37402021737899876, + -4.429208872117422e-09, + -1.008941638491605e-08, + -2.5608732325955336e-08, + 4.485976896804288e-09, + ] +) + + +mancino_5_solution_x = np.array( + [ + 84.28291101102532, + 79.20603967293438, + 74.3364141135311, + 69.6711474112178, + 65.20718113814442, + ] +) + +mancino_8_solution_x = np.array( + [ + 84.43334222593528, + 79.33454939399172, + 74.44387011026309, + 69.7592945870252, + 65.27853533617875, + 60.9988580578957, + 56.9169379354432, + 53.028761291567236, + ] +) + +mancino_10_solution_x = np.array( + [ + 84.53434289477315, + 79.42084435375007, + 74.51601545241338, + 69.81844699647671, + 65.32637991893166, + 61.03748806452533, + 56.94869518846038, + 53.056052319528746, + 49.35469508461959, + 45.83889035077595, + ] +) + + +mancino_12_solution_x = np.array( + [ + 84.63591921594158, + 79.5076423105225, + 74.5885724920863, + 69.87791406833868, + 65.37444824684921, + 61.07626530788906, + 56.98054088428213, + 53.08338921660163, + 49.379816810523714, + 45.86378591833196, + 42.52838225939789, + 39.36606891417026, + ] +) + +heart_eight_solution_x = np.array( + [ + -0.311626605565399, + -0.37837339443458845, + 0.3282442301180765, + -0.3722442301180588, + -1.282227094270286, + 2.4943003120854743, + 1.5548658787873983, + -1.384637842863253, + ] +) + + +MORE_WILD_PROBLEMS = { + "linear_full_rank_good_start": { + "criterion": partial(linear_full_rank, dim_out=45), + "start_x": np.ones(9), + "solution_x": linear_full_rank_solution_x, + "start_criterion": 72, + "solution_criterion": 36, + }, + "linear_full_rank_bad_start": { + "criterion": partial(linear_full_rank, dim_out=45), + "start_x": np.ones(9) * 10, + "solution_x": linear_full_rank_solution_x, + "start_criterion": 1125, + "solution_criterion": 36, + }, + "linear_rank_one_good_start": { + "criterion": partial(linear_rank_one, dim_out=35), + "start_x": np.ones(7), + # no unique solution + "solution_x": None, + "start_criterion": 1.165420e7, + "solution_criterion": 8.380281690143324, + }, + "linear_rank_one_bad_start": { + "criterion": partial(linear_rank_one, dim_out=35), + "start_x": np.ones(7) * 10, + # no unique solution + "solution_x": None, + "start_criterion": 1.168591e9, + "solution_criterion": 8.380282, + }, + "linear_rank_one_zero_columns_rows_good_start": { + "criterion": partial(linear_rank_one_zero_columns_rows, dim_out=35), + "start_x": np.ones(7), + # no unique solution + "solution_x": None, + "start_criterion": 4.989195e6, + "solution_criterion": 9.880597014926506, + }, + "linear_rank_one_zero_columns_rows_bad_start": { + "criterion": partial(linear_rank_one_zero_columns_rows, dim_out=35), + "start_x": np.ones(7) * 10, + # no unique solution + "solution_x": None, + "start_criterion": 5.009356e8, + "solution_criterion": 9.880597014926506, + }, + "rosenbrock_good_start": { + "criterion": rosenbrock, + "start_x": np.array([-1.2, 1]), + "solution_x": np.ones(2), + "start_criterion": 24.2, + "solution_criterion": 0, + }, + "rosenbrock_bad_start": { + "criterion": rosenbrock, + "start_x": np.array([-1.2, 1]) * 10, + "solution_x": np.ones(2), + "start_criterion": 1.795769e6, + "solution_criterion": 0, + }, + "helical_valley_good_start": { + "criterion": helical_valley, + "start_x": np.array([-1, 0, 0]), + "solution_x": np.array([1, 0, 0]), + "start_criterion": 2500, + "solution_criterion": 0, + }, + "helical_valley_bad_start": { + "criterion": helical_valley, + "start_x": np.array([-10, 0, 0]), + "solution_x": np.array([1, 0, 0]), + "start_criterion": 10600, + "solution_criterion": 0, + }, + "powell_singular_good_start": { + "criterion": powell_singular, + "start_x": np.array([3, -1, 0, 1]), + "solution_x": np.zeros(4), + "start_criterion": 215, + "solution_criterion": 0, + }, + "powell_singular_bad_start": { + "criterion": powell_singular, + "start_x": np.array([3, -1, 0, 1]) * 10, + "solution_x": np.zeros(4), + "start_criterion": 1.615400e6, + "solution_criterion": 0, + }, + "freudenstein_roth_good_start": { + "criterion": freudenstein_roth, + "start_x": np.array([0.5, -2]), + "solution_x": freudenstein_roth_solution_x, + "start_criterion": 400.5, + "solution_criterion": 48.984253679240013, + }, + "freudenstein_roth_bad_start": { + "criterion": freudenstein_roth, + "start_x": np.array([0.5, -2]) * 10, + "solution_x": freudenstein_roth_solution_x, + "start_criterion": 1.545754e8, + "solution_criterion": 48.984253679240013, + }, + "bard_good_start": { + "criterion": partial(bard, y=y_vec), + "start_x": np.ones(3), + "solution_x": bard_solution_x, + "start_criterion": 41.68170, + "solution_criterion": 0.00821487730657897, + }, + "bard_bad_start": { + "criterion": partial(bard, y=y_vec), + "start_x": np.ones(3) * 10, + "solution_x": bard_solution_x, + "start_criterion": 1306.234, + "solution_criterion": 0.00821487730657897, + }, + "kowalik_osborne": { + "criterion": partial( + kowalik_osborne, + y1=v_vec, + y2=y2_vec, + ), + "start_x": np.array([0.25, 0.39, 0.415, 0.39]), + "solution_x": kowalik_osborne_solution_x, + "start_criterion": 5.313172e-3, + "solution_criterion": 0.00030750560384924, + }, + "meyer": { + "criterion": partial(meyer, y=y3_vec), + "start_x": np.array([0.02, 4000, 250]), + "solution_x": meyer_solution_x, + "start_criterion": 1.693608e9, + "solution_criterion": 87.945855170395831, + }, + "watson_6_good_start": { + "criterion": watson, + "start_x": 0.5 * np.ones(6), + "solution_x": watson_6_solution_x, + "start_criterion": 16.43083, + "solution_criterion": 0.00228767005355236, + }, + "watson_6_bad_start": { + "criterion": watson, + "start_x": 5 * np.ones(6), + "solution_x": watson_6_solution_x, + "start_criterion": 2.323367e6, + "solution_criterion": 0.00228767005355236, + }, + "watson_9_good_start": { + "criterion": watson, + "start_x": 0.5 * np.ones(9), + "solution_x": watson_9_solution_x, + "start_criterion": 26.90417, + "solution_criterion": 1.399760e-6, + }, + "watson_9_bad_start": { + "criterion": watson, + "start_x": 5 * np.ones(9), + "solution_x": watson_9_solution_x, + "start_criterion": 8.158877e6, + "solution_criterion": 1.399760e-6, + }, + "watson_12_good_start": { + "criterion": watson, + "start_x": 0.5 * np.ones(12), + "solution_x": watson_12_solution_x, + "start_criterion": 73.67821, + "solution_criterion": 4.722381e-10, + }, + "watson_12_bad_start": { + "criterion": watson, + "start_x": 5 * np.ones(12), + "solution_x": watson_12_solution_x, + "start_criterion": 2.059384e7, + "solution_criterion": 4.722381e-10, + }, + "box_3d": { + "criterion": partial(box_3d, dim_out=10), + "start_x": np.array([0, 10, 20]), + "solution_x": np.array([1, 10, 1]), + "start_criterion": 1031.154, + "solution_criterion": 0, + }, + "jennrich_sampson": { + "criterion": partial(jennrich_sampson, dim_out=10), + "start_x": np.array([0.3, 0.4]), + "solution_x": np.array([0.2578252135686162] * 2), + "start_criterion": 4171.306, + "solution_criterion": 124.3621823556148, + }, + "brown_dennis_good_start": { + "criterion": partial(brown_dennis, dim_out=20), + "start_x": np.array([25, 5, -5, -1]), + "solution_x": brown_dennis_solution_x, + "start_criterion": 7.926693e6, + "solution_criterion": 85822.20162635, + }, + "brown_dennis_bad_start": { + "criterion": partial(brown_dennis, dim_out=20), + "start_x": np.array([25, 5, -5, -1]) * 10, + "solution_x": brown_dennis_solution_x, + "start_criterion": 3.081064e11, + "solution_criterion": 85822.20162635, + }, + "chebyquad_6": { + "criterion": partial(chebyquad, dim_out=6), + "start_x": np.arange(1, 7) / 7, + "solution_x": chebyquad_6_solution_x, + "start_criterion": 4.642817e-2, + "solution_criterion": 0, + }, + "chebyquad_7": { + "criterion": partial(chebyquad, dim_out=7), + "start_x": np.arange(1, 8) / 8, + "solution_x": chebyquad_7_solution_x, + "start_criterion": 3.377064e-2, + "solution_criterion": 0, + }, + "chebyquad_8": { + "criterion": partial(chebyquad, dim_out=8), + "start_x": np.arange(1, 9) / 9, + "solution_x": chebyquad_8_solution_x, + "start_criterion": 3.861770e-2, + "solution_criterion": 0.003516873725677, + }, + "chebyquad_9": { + "criterion": partial(chebyquad, dim_out=9), + "start_x": np.arange(1, 10) / 10, + "solution_x": chebyquad_9_solution_x, + "start_criterion": 2.888298e-2, + "solution_criterion": 0, + }, + "chebyquad_10": { + "criterion": partial(chebyquad, dim_out=10), + "start_x": np.arange(1, 11) / 11, + "solution_x": chebyquad_10_solution_x, + "start_criterion": 3.376327e-2, + "solution_criterion": 0.00477271369637536, + }, + "chebyquad_11": { + "criterion": partial(chebyquad, dim_out=11), + "start_x": np.arange(1, 12) / 12, + "solution_x": chebyquad_11_solution_x, + "start_criterion": 2.674060e-2, + "solution_criterion": 0.00279976155186576, + }, + "brown_almost_linear": { + "criterion": brown_almost_linear, + "start_x": 0.5 * np.ones(10), + "solution_x": np.ones(10), + "start_criterion": 273.2480, + "solution_criterion": 0, + }, + "osborne_one": { + "criterion": partial(osborne_one, y=y4_vec), + "start_x": np.array([0.5, 1.5, 1, 0.01, 0.02]), + "solution_x": osborne_one_solution_x, + "start_criterion": 16.17411, + "solution_criterion": 0.00005464894697483, + }, + "osborne_two_good_start": { + "criterion": partial(osborne_two, y=y5_vec), + "start_x": np.array([1.3, 0.65, 0.65, 0.7, 0.6, 3, 5, 7, 2, 4.5, 5.5]), + "solution_x": osborne_two_solution_x, + "start_criterion": 2.093420, + "solution_criterion": 0.0401377362935477, + }, + "osborne_two_bad_start": { + "criterion": partial(osborne_two, y=y5_vec), + "start_x": 10 * np.array([1.3, 0.65, 0.65, 0.7, 0.6, 3, 5, 7, 2, 4.5, 5.5]), + "solution_x": osborne_two_solution_x, + "start_criterion": 199.6847, + "solution_criterion": 0.0401377362935477, + }, + "bdqrtic_8": { + "criterion": bdqrtic, + "start_x": np.ones(8), + "solution_x": bdqrtic_8_solution_x, + "start_criterion": 904, + "solution_criterion": 10.2389734213174, + }, + "bdqrtic_10": { + "criterion": bdqrtic, + "start_x": np.ones(10), + "solution_x": bdqrtic_10_solution_x, + "start_criterion": 1356, + "solution_criterion": 18.28116175359353, + }, + "bdqrtic_11": { + "criterion": bdqrtic, + "start_x": np.ones(11), + "solution_x": bdqrtic_11_solution_x, + "start_criterion": 1582, + "solution_criterion": 22.260591734883817, + }, + "bdqrtic_12": { + "criterion": bdqrtic, + "start_x": np.ones(12), + "solution_x": bdqrtic_12_solution_x, + "start_criterion": 1808, + "solution_criterion": 26.2727663967939, + }, + "cube_5": { + "criterion": cube, + "start_x": 0.5 * np.ones(5), + "solution_x": np.ones(5), + "start_criterion": 56.5, + "solution_criterion": 0, + }, + "cube_6": { + "criterion": cube, + "start_x": 0.5 * np.ones(6), + "solution_x": np.ones(6), + "start_criterion": 70.5625, + "solution_criterion": 0, + }, + "cube_8": { + "criterion": cube, + "start_x": 0.5 * np.ones(8), + "solution_x": np.ones(8), + "start_criterion": 98.6875, + "solution_criterion": 0, + }, + "mancino_5_good_start": { + "criterion": mancino, + "start_x": get_start_points_mancino(5), + "solution_x": mancino_5_solution_x, + "start_criterion": 2.539084e9, + "solution_criterion": 0, + }, + "mancino_5_bad_start": { + "criterion": mancino, + "start_x": 10 * get_start_points_mancino(5), + "solution_x": mancino_5_solution_x, + "start_criterion": 6.873795e12, + "solution_criterion": 0, + }, + "mancino_8": { + "criterion": mancino, + "start_x": get_start_points_mancino(8), + "solution_x": mancino_8_solution_x, + "start_criterion": 3.367961e9, + "solution_criterion": 0, + }, + "mancino_10": { + "criterion": mancino, + "start_x": get_start_points_mancino(10), + "solution_x": mancino_10_solution_x, + "start_criterion": 3.735127e9, + "solution_criterion": 0, + }, + "mancino_12_good_start": { + "criterion": mancino, + "start_x": get_start_points_mancino(12), + "solution_x": mancino_12_solution_x, + "start_criterion": 3.991072e9, + "solution_criterion": 0, + }, + "mancino_12_bad_start": { + "criterion": mancino, + "start_x": 10 * get_start_points_mancino(12), + "solution_x": mancino_12_solution_x, + "start_criterion": 1.130015e13, + "solution_criterion": 0, + }, + "heart_eight_good_start": { + "criterion": partial( + heart_eight, + y=np.array([-0.69, -0.044, -1.57, -1.31, -2.65, 2, -12.6, 9.48]), + ), + "start_x": np.array([-0.3, -0.39, 0.3, -0.344, -1.2, 2.69, 1.59, -1.5]), + "solution_x": heart_eight_solution_x, + "start_criterion": 9.385672, + "solution_criterion": 0, + }, + "heart_eight_bad_start": { + "criterion": partial( + heart_eight, + y=np.array([-0.69, -0.044, -1.57, -1.31, -2.65, 2, -12.6, 9.48]), + ), + "start_x": 10 * np.array([-0.3, -0.39, 0.3, -0.344, -1.2, 2.69, 1.59, -1.5]), + "solution_x": heart_eight_solution_x, + "start_criterion": 3.365815e10, + "solution_criterion": 0, + }, + "brown_almost_linear_medium": { + "criterion": brown_almost_linear, + "start_x": 0.5 * np.ones(100), + "solution_x": np.ones(100), + "start_criterion": 2.524757e5, + "solution_criterion": 0, + }, +} diff --git a/src/estimagic/benchmarking/noise_distributions.py b/src/estimagic/benchmarking/noise_distributions.py new file mode 100644 index 000000000..529a1c607 --- /dev/null +++ b/src/estimagic/benchmarking/noise_distributions.py @@ -0,0 +1,36 @@ +import numpy as np + + +def _standard_logistic(size): + scale = np.sqrt(3) / np.pi + return np.random.logistic(loc=0, scale=scale, size=size) + + +def _standard_uniform(size): + ub = np.sqrt(3) + lb = -ub + return np.random.uniform(lb, ub, size=size) + + +def _standard_normal(size): + return np.random.normal(size=size) + + +def _standard_gumbel(size): + gamma = 0.577215664901532 + scale = np.sqrt(6) / np.pi + loc = -scale * gamma + return np.random.gumbel(loc=loc, scale=scale, size=size) + + +def _standard_laplace(size): + return np.random.laplace(scale=np.sqrt(0.5), size=size) + + +NOISE_DISTRIBUTIONS = { + "normal": _standard_normal, + "gumbel": _standard_gumbel, + "logistic": _standard_logistic, + "uniform": _standard_uniform, + "laplace": _standard_laplace, +} diff --git a/src/estimagic/benchmarking/process_benchmark_results.py b/src/estimagic/benchmarking/process_benchmark_results.py new file mode 100644 index 000000000..48695273d --- /dev/null +++ b/src/estimagic/benchmarking/process_benchmark_results.py @@ -0,0 +1,303 @@ +import numpy as np +import pandas as pd + + +def create_convergence_histories( + problems, results, stopping_criterion, x_precision, y_precision +): + """Create tidy DataFrame with all information needed for the benchmarking plots. + + Args: + problems (dict): estimagic benchmarking problems dictionary. Keys are the + problem names. Values contain information on the problem, including the + solution value. + results (dict): estimagic benchmarking results dictionary. Keys are + tuples of the form (problem, algorithm), values are dictionaries of the + collected information on the benchmark run, including 'criterion_history' + and 'time_history'. + stopping_criterion (str): one of "x_and_y", "x_or_y", "x", "y". Determines + how convergence is determined from the two precisions. + x_precision (float or None): how close an algorithm must have gotten to the + true parameter values (as percent of the Euclidean distance between start + and solution parameters) before the criterion for clipping and convergence + is fulfilled. + y_precision (float or None): how close an algorithm must have gotten to the + true criterion values (as percent of the distance between start + and solution criterion value) before the criterion for clipping and + convergence is fulfilled. + + Returns: + pandas.DataFrame: tidy DataFrame with the following columns: + - problem + - algorithm + - n_evaluations + - walltime + - criterion + - criterion_normalized + - monotone_criterion + - monotone_criterion_normalized + - parameter_distance + - parameter_distance_normalized + - monotone_parameter_distance + - monotone_parameter_distance_normalized + + """ + # get solution values for each problem + f_opt = pd.Series( + {name: prob["solution"]["value"] for name, prob in problems.items()} + ) + x_opt = { + name: prob["solution"]["params"]["value"] for name, prob in problems.items() + } + + # build df from results + time_sr = _get_history_as_stacked_sr_from_results(results, "time_history") + time_sr.name = "walltime" + criterion_sr = _get_history_as_stacked_sr_from_results(results, "criterion_history") + x_dist_sr = _get_history_of_the_parameter_distance(results, x_opt) + df = pd.concat([time_sr, criterion_sr, x_dist_sr], axis=1) + + df.index = df.index.rename({"evaluation": "n_evaluations"}) + df = df.sort_index().reset_index() + + first_evaluations = df.query("n_evaluations == 0").groupby("problem") + f_0 = first_evaluations["criterion"].mean() + x_0_dist = first_evaluations["parameter_distance"].mean() + x_opt_dist = {name: 0 for name in problems} + + # normalizations + df["criterion_normalized"] = _normalize( + df=df, col="criterion", start_values=f_0, target_values=f_opt + ) + df["parameter_distance_normalized"] = _normalize( + df=df, + col="parameter_distance", + start_values=x_0_dist, + target_values=x_opt_dist, + ) + # create monotone versions of columns + df["monotone_criterion"] = _make_history_monotone(df, "criterion") + df["monotone_parameter_distance"] = _make_history_monotone(df, "parameter_distance") + df["monotone_criterion_normalized"] = _make_history_monotone( + df, "criterion_normalized" + ) + df["monotone_parameter_distance_normalized"] = _make_history_monotone( + df, "parameter_distance_normalized" + ) + + if stopping_criterion is not None: + df, converged_info = _clip_histories( + df=df, + stopping_criterion=stopping_criterion, + x_precision=x_precision, + y_precision=y_precision, + ) + else: + converged_info = None + + return df, converged_info + + +def _get_history_as_stacked_sr_from_results(results, key): + """Get history as stacked Series from results. + + Args: + results (dict): estimagic benchmarking results dictionary. + key (str): name of the history for which to build the Series, e.g. + criterion_history. + + Returns: + pandas.Series: index levels are 'problem', 'algorithm' and 'evaluation'. + the name is the key with '_history' stripped off. + + """ + histories = {tup: res[key] for tup, res in results.items()} + sr = pd.concat(histories) + sr.index.names = ["problem", "algorithm", "evaluation"] + sr.name = key.replace("_history", "") + return sr + + +def _get_history_of_the_parameter_distance(results, x_opt): + """Calculate the history of the distances to the optimal parameters. + + Args: + results (dict): estimagic benchmarking results dictionary. Keys are + tuples of the form (problem, algorithm), values are dictionaries of the + collected information on the benchmark run, including 'params_history'. + x_opt (dict): the keys are the problems, the values are pandas.Series with the + optimal parameters for the respective problem. + + Returns: + pandas.Series: index levels are "problem", "algorithm", "evaluation". The name + is "parameter_distance". + + """ + x_dist_history = {} + for (prob, algo), res in results.items(): + param_history = res["params_history"] + x_dist_history[(prob, algo)] = pd.Series( + np.linalg.norm(param_history - x_opt[prob], axis=1) + ) + + sr = pd.concat(x_dist_history) + sr.index.names = ["problem", "algorithm", "evaluation"] + sr.name = "parameter_distance" + return sr + + +def _make_history_monotone(df, target_col, direction="minimize"): + """Create a monotone Series, i.e. the best so far instead of the current evaluation. + + Args: + df (pandas.Dataframe): must contain ["problem", "algorithm", "n_evaluations"] + and the target_col as columns. + target_col (str): column of which to create the monotone version. + direction (str): "minimize" or "maximize". "minimize" makes the history + monotonically decreasing, "maximize" means the history will be monotonically + increasing. + + Retruns: + pd.Series: target column where all values that are not weak improvements are + replaced with the best value so far. Index is the same as that of df. + + """ + sorted_df = df.sort_values(["problem", "algorithm", "n_evaluations"]) + grouped = sorted_df.groupby(["problem", "algorithm"])[target_col] + + if direction == "minimize": + out = grouped.apply(np.minimum.accumulate) + elif direction == "maximize": + out = grouped.apply(np.maximum.accumulate) + else: + raise ValueError("Only maximize and minimize are allowed as directions.") + + return out + + +def _normalize(df, col, start_values, target_values): + """Normalize the values in **col** relative to the total possible improvement. + + We normalize the values of **col** by calculating the share of the distance between + the start and target values that is still missing. + + Note: This is correct whether we have a minimization or a maximization problem + because in the case of a maximization both the sign of the numerator and denominator + in the line where normalized is defined would be switched, i.e. that cancels out. + (In the case of a maximization the total improvement would be target - start values + and the currently still missing improvement would be target - current values) + + Args: + df (pandas.DataFrame): contains the columns **col** and "problem". + col (str): name of the column to normalize + start_values (pandas.Series): index are problems, values are start values + target_values (pandas.Series): index are problems, values are target values. + + Returns: + pandas.Series: index is the same as that of sr. The lower the value the closer + the current value is to the target value. 0 means the target value has been + reached. 1 means the current value is as far from the target value as the + start value. + + """ + # expand start and target values to the length of the full DataFrame + start_values = df["problem"].map(start_values) + target_values = df["problem"].map(target_values) + + normalized = (df[col] - target_values) / (start_values - target_values) + return normalized + + +def _clip_histories(df, stopping_criterion, x_precision, y_precision): + """Shorten the DataFrame to just the evaluations until each algorithm converged. + + Args: + df (pandas.DataFrame): index levels are ['problem', 'algorithm', 'evaluation']. + Columns must include "monotone_criterion_normalized" if stopping_criterion + includes y and "monotone_parameter_distance_normalized" if x is in + the stopping_criterion. + stopping_criterion (str): one of "x_and_y", "x_or_y", "x", "y". + x_precision (float): when an algorithm's parameters are closer than this to the + true solution's parameters, the algorithm is counted as having converged. + y_precision (float): when an algorithm's criterion value is closer than this to + the solution value, the algorithm is counted as having converged. + + Returns: + shortened (pandas.DataFrame): the entered DataFrame with all histories + shortened to stop once conversion according to the given criteria is + reached. + converged_info (pandas.DataFrame): columns are the algorithms, index are the + problems. The values are boolean and True when the algorithm arrived at + the solution with the desired precision. + + """ + # drop problems with no known solution + if "x" in stopping_criterion: + df = df[df["monotone_parameter_distance_normalized"].notnull()] + if "y" in stopping_criterion: + df = df[df["monotone_criterion_normalized"].notnull()] + + # determine convergence in the known problems + if "x" in stopping_criterion: + x_converged = df["monotone_parameter_distance_normalized"] < x_precision + if "y" in stopping_criterion: + y_converged = df["monotone_criterion_normalized"] < y_precision + + # determine converged function evaluations + if stopping_criterion == "y": + converged = y_converged + elif stopping_criterion == "x": + converged = x_converged + elif stopping_criterion == "x_and_y": + converged = y_converged & x_converged + elif stopping_criterion == "x_or_y": + converged = y_converged | x_converged + else: + raise NotImplementedError( + f"You specified {stopping_criterion} as stopping_criterion but only the " + "following are allowed: 'x_and_y', 'x_or_y', 'x', or 'y'." + ) + + first_converged = _find_first_converged(converged, df) + + # keep first converged and non-converged + shortened = df[~converged | first_converged] + + # create converged_info + converged.index = pd.MultiIndex.from_frame(df[["problem", "algorithm"]]) + grouped = converged.groupby(["problem", "algorithm"]) + converged_info = grouped.any().unstack("algorithm") + + return shortened, converged_info + + +def _find_first_converged(converged, df): + """Identify the first converged entry for each problem run. + + Args: + converged (pandas.Series): same index as df, True where an algorithm has gotten + sufficiently close to the solution. + df (pandas.DataFrame): contains the "problem", "algorithm" and "n_evaluations" + columns. + + Returns: + pandas.Series: same index as converged. Only True for the first converged entry + for each problem run, i.e. problem and algorithm combination. + + """ + # this function can probably be implemented much quicker and easier by shifting + # the converged Series to identify the first converged entries + + converged_with_multi_index = converged.copy(deep=True) + multi_index = pd.MultiIndex.from_frame( + df[["problem", "algorithm", "n_evaluations"]] + ) + converged_with_multi_index.index = multi_index + + only_converged = converged_with_multi_index[converged_with_multi_index] + first_true_indices = only_converged.groupby(["problem", "algorithm"]).idxmin() + first_trues = pd.Series( + converged_with_multi_index.index.isin(first_true_indices.values), + index=converged.index, + ) + return first_trues diff --git a/src/estimagic/benchmarking/run_benchmark.py b/src/estimagic/benchmarking/run_benchmark.py new file mode 100644 index 000000000..aee3ebfe2 --- /dev/null +++ b/src/estimagic/benchmarking/run_benchmark.py @@ -0,0 +1,136 @@ +"""Functions to create, run and visualize optimization benchmarks. + +TO-DO: +- Add other benchmark sets: + - finish medium scale problems from https://arxiv.org/pdf/1710.11005.pdf, Page 34. + - add scalar problems from https://github.com/AxelThevenot +- Add option for deterministic noise or wiggle. + +""" +from pathlib import Path + +import numpy as np +from estimagic import batch_evaluators +from estimagic.logging.read_log import read_optimization_histories +from estimagic.optimization.optimize import minimize + + +def run_benchmark( + problems, + optimize_options, + logging_directory, + batch_evaluator="joblib", + n_cores=1, + error_handling="continue", + fast_logging=True, + seed=None, +): + """Run problems with different optimize options. + + Args: + problems (dict): Nested dictionary with benchmark problems of the structure: + {"name": {"inputs": {...}, "solution": {...}, "info": {...}}} + where "inputs" are keyword arguments for ``minimize`` such as the criterion + function and start parameters. "solution" contains the entries "params" and + "value" and "info" might contain information about the test problem. + optimize_options (list or dict): Either a list of algorithms or a Nested + dictionary that maps a name for optimizer settings + (e.g. ``"lbfgsb_strict_criterion"``) to a dictionary of keyword arguments + for arguments for ``minimize`` (e.g. ``{"algorithm": "scipy_lbfgsb", + "algo_options": {"convergence.relative_criterion_tolerance": 1e-12}}``). + Alternatively, the values can just be an algorithm which is then benchmarked + at default settings. + batch_evaluator (str or callable): See :ref:`batch_evaluators`. + logging_directory (pathlib.Path): Directory in which the log databases are + saved. + n_cores (int): Number of optimizations that is run in parallel. Note that in + addition to that an optimizer might parallelize. + error_handling (str): One of "raise", "continue". + fast_logging (bool): Whether the slightly unsafe but much faster database + configuration is chosen. + + Returns: + dict: Nested Dictionary with information on the benchmark run. The outer keys + are tuples where the first entry is the name of the problem and the second + the name of the optimize options. The values are dicts with the entries: + "runtime", "params_history", "criterion_history", "solution" + + """ + np.random.seed(seed) + logging_directory = Path(logging_directory) + logging_directory.mkdir(parents=True, exist_ok=True) + + if isinstance(batch_evaluator, str): + batch_evaluator = getattr( + batch_evaluators, f"{batch_evaluator}_batch_evaluator" + ) + + opt_options = _process_optimize_options(optimize_options) + + log_options = {"fast_logging": fast_logging, "if_table_exists": "replace"} + + kwargs_list = [] + names = [] + for prob_name, problem in problems.items(): + for option_name, options in opt_options.items(): + kwargs = { + **options, + **problem["inputs"], + "logging": logging_directory / f"{prob_name}_{option_name}.db", + "log_options": log_options, + } + kwargs_list.append(kwargs) + names.append((prob_name, option_name)) + + log_paths = [kwargs["logging"] for kwargs in kwargs_list] + + raw_results = batch_evaluator( + func=minimize, + arguments=kwargs_list, + n_cores=n_cores, + error_handling=error_handling, + unpack_symbol="**", + ) + + results = {} + for name, result, log_path in zip(names, raw_results, log_paths): + histories = read_optimization_histories(log_path) + stop = histories["metadata"]["timestamps"].max() + start = histories["metadata"]["timestamps"].min() + runtime = (stop - start).total_seconds() + + results[name] = { + "params_history": histories["params"], + "criterion_history": histories["values"], + "time_history": histories["metadata"]["timestamps"] - start, + "solution": result, + "runtime": runtime, + } + + return results + + +def _process_optimize_options(raw_options): + if not isinstance(raw_options, dict): + dict_options = {} + for option in raw_options: + if isinstance(option, str): + dict_options[option] = option + else: + dict_options[option.__name__] = option + else: + dict_options = raw_options + + out_options = {} + for name, option in dict_options.items(): + if not isinstance(option, dict): + option = {"algorithm": option} + + if "log_options" in option: + raise ValueError( + "Log options cannot be specified as part of optimize_options. Logging " + "behavior is configured by the run_benchmark function." + ) + out_options[name] = option + + return out_options diff --git a/src/estimagic/logging/database_utilities.py b/src/estimagic/logging/database_utilities.py index e3ad8df6a..c8d47a038 100644 --- a/src/estimagic/logging/database_utilities.py +++ b/src/estimagic/logging/database_utilities.py @@ -299,7 +299,7 @@ def read_last_rows( Args: database (sqlalchemy.MetaData) table_name (str): name of the table to retrieve. - n_int (int): number of rows to retrieve. + n_rows (int): number of rows to retrieve. return_type (str): either "list_of_dicts" or "dict_of_lists". path (str or pathlib.Path): location of the database file. If the file does not exist, it will be created. Using a path is much slower than a diff --git a/src/estimagic/logging/read_log.py b/src/estimagic/logging/read_log.py index 944641561..91a729131 100644 --- a/src/estimagic/logging/read_log.py +++ b/src/estimagic/logging/read_log.py @@ -99,6 +99,48 @@ def read_start_params(path_or_database): return start_params +def read_optimization_histories(path_or_database): + """Read a histories out values, parameters and other information.""" + database = load_database(**_process_path_or_database(path_or_database)) + + start_params = read_start_params(path_or_database) + + raw_res, _ = read_new_rows( + database=database, + table_name="optimization_iterations", + last_retrieved=0, + return_type="dict_of_lists", + ) + + params_history = pd.DataFrame(raw_res["params"], columns=start_params.index) + value_history = pd.Series(raw_res["value"]) + + metadata = pd.DataFrame() + metadata["timestamps"] = raw_res["timestamp"] + metadata["valid"] = raw_res["valid"] + metadata["has_value"] = value_history.notnull() + metadata["has_derivative"] = [d is not None for d in raw_res["internal_derivative"]] + + histories = { + "values": value_history.dropna(), + "params": params_history, + "metadata": metadata, + } + + if "contributions" in raw_res: + first_contrib = raw_res["contributions"][0] + if isinstance(first_contrib, pd.Series): + columns = first_contrib.index + else: + columns = None + contributions_history = pd.DataFrame( + raw_res["contributions"], columns=columns + ).dropna() + histories["contributions"] = contributions_history + + return histories + + def _process_path_or_database(path_or_database): """Make inputs for load_database out of path_or_database. diff --git a/src/estimagic/visualization/convergence_plot.py b/src/estimagic/visualization/convergence_plot.py new file mode 100644 index 000000000..09029f7ae --- /dev/null +++ b/src/estimagic/visualization/convergence_plot.py @@ -0,0 +1,160 @@ +import matplotlib.pyplot as plt +import numpy as np +import seaborn as sns +from estimagic.benchmarking.process_benchmark_results import ( + create_convergence_histories, +) +from estimagic.visualization.colors import get_colors + +plt.rcParams.update( + { + "axes.spines.right": False, + "axes.spines.top": False, + "legend.frameon": False, + } +) + + +def convergence_plot( + problems=None, + results=None, + problem_subset=None, + algorithm_subset=None, + n_cols=2, + distance_measure="criterion", + monotone=True, + normalize_distance=True, + runtime_measure="n_evaluations", + stopping_criterion="y", + x_precision=1e-4, + y_precision=1e-4, +): + """Plot convergence of optimizers for a set of problems. + + This creates a grid of plots, showing the convergence of the different + algorithms on each problem. The faster a line falls, the faster the algorithm + improved on the problem. The algorithm converged where its line reaches 0 + (if normalize_distance is True) or the horizontal blue line labeled "true solution". + + Each plot shows on the x axis the runtime_measure, which can be walltime or number + of evaluations. Each algorithm's convergence is a line in the plot. Convergence can + be measured by the criterion value of the particular time/evaluation. The + convergence can be made monotone (i.e. always taking the bast value so far) or + normalized such that the distance from the start to the true solution is one. + + Args: + problems (dict): estimagic benchmarking problems dictionary. Keys are the + problem names. Values contain information on the problem, including the + solution value. + results (dict): estimagic benchmarking results dictionary. Keys are + tuples of the form (problem, algorithm), values are dictionaries of the + collected information on the benchmark run, including 'criterion_history' + and 'time_history'. + problem_subset (list, optional): List of problem names. These must be a subset + of the keys of the problems dictionary. If provided the convergence plot is + only created for the problems specified in this list. + algorithm_subset (list, optional): List of algorithm names. These must be a + subset of the keys of the optimizer_options passed to run_benchmark. If + provided only the convergence of the given algorithms are shown. + n_cols (int): number of columns in the plot of grids. The number + of rows is determined automatically. + distance_measure (str): One of "criterion", "parameter_distance". + monotone (bool): If True the best found criterion value so far is plotted. + If False the particular criterion evaluation of that time is used. + normalize_distance (bool): If True the progress is scaled by the total distance + between the start value and the optimal value, i.e. 1 means the algorithm + is as far from the solution as the start value and 0 means the algorithm + has reached the solution value. + runtime_measure (str): "n_evaluations" or "walltime". + stopping_criterion (str): "x_and_y", "x_or_y", "x", "y" or None. If None, no + clipping is done. + x_precision (float or None): how close an algorithm must have gotten to the + true parameter values (as percent of the Euclidean distance between start + and solution parameters) before the criterion for clipping and convergence + is fulfilled. + y_precision (float or None): how close an algorithm must have gotten to the + true criterion values (as percent of the distance between start + and solution criterion value) before the criterion for clipping and + convergence is fulfilled. + + Returns: + fig, axes + + """ + df, _ = create_convergence_histories( + problems=problems, + results=results, + stopping_criterion=stopping_criterion, + x_precision=x_precision, + y_precision=y_precision, + ) + + if problem_subset is not None: + df = df[df["problem"].isin(problem_subset)] + if algorithm_subset is not None: + df = df[df["algorithm"].isin(algorithm_subset)] + + # plot configuration + outcome = ( + f"{'monotone_' if monotone else ''}" + + distance_measure + + f"{'_normalized' if normalize_distance else ''}" + ) + + # create plots + remaining_problems = df["problem"].unique() + n_rows = int(np.ceil(len(remaining_problems) / n_cols)) + figsize = (n_cols * 6, n_rows * 4) + fig, axes = plt.subplots(nrows=n_rows, ncols=n_cols, figsize=figsize) + algorithms = {tup[1] for tup in results.keys()} + palette = get_colors("categorical", number=len(algorithms)) + + for ax, prob_name in zip(axes.flatten(), remaining_problems): + to_plot = df[df["problem"] == prob_name] + sns.lineplot( + data=to_plot, + x=runtime_measure, + y=outcome, + hue="algorithm", + lw=2.5, + alpha=1.0, + ax=ax, + palette=palette, + ) + ax.set_title(prob_name.replace("_", " ").title()) + if distance_measure == "criterion" and not normalize_distance: + f_opt = problems[prob_name]["solution"]["value"] + ax.axhline(f_opt, label="true solution", lw=2.5) + + # style plots + y_labels = { + "criterion": "Current Function Value", + "monotone_criterion": "Best Function Value Found So Far", + "criterion_normalized": "Share of Function Distance to Optimum\n" + + "Missing From Current Criterion Value", + "monotone_criterion_normalized": "Share of Function Distance to Optimum\n" + + "Missing From Best So Far", + "parameter_distance": "Distance Between Current and Optimal Parameters", + "parameter_distance_normalized": "Share of the Parameter Distance to Optimum\n" + + "Missing From Current Parameters", + "monotone_parameter_distance_normalized": "Share of the Parameter Distance " + + "to Optimum\n Missing From the Best Parameters So Far", + "monotone_parameter_distance": "Distance Between the Best Parameters So Far\n" + "and the Optimal Parameters", + } + x_labels = { + "n_evaluations": "Number of Function Evaluations", + "walltime": "Elapsed Time", + } + for ax in axes.flatten(): + ax.set_ylabel(y_labels[outcome]) + ax.set_xlabel(x_labels[runtime_measure]) + ax.legend(title=None) + + # make empty plots invisible + n_empty_plots = len(axes.flatten()) - len(remaining_problems) + if n_empty_plots > 0: + for ax in axes.flatten()[-n_empty_plots:]: + ax.set_visible(False) + fig.tight_layout() + return fig, axes diff --git a/src/estimagic/visualization/profile_plot.py b/src/estimagic/visualization/profile_plot.py new file mode 100644 index 000000000..ae8a5c98a --- /dev/null +++ b/src/estimagic/visualization/profile_plot.py @@ -0,0 +1,222 @@ +import warnings + +import matplotlib.pyplot as plt +import numpy as np +import pandas as pd +import seaborn as sns +from estimagic.benchmarking.process_benchmark_results import ( + create_convergence_histories, +) +from estimagic.visualization.colors import get_colors + + +plt.rcParams.update( + { + "axes.spines.right": False, + "axes.spines.top": False, + "legend.frameon": False, + } +) + + +def profile_plot( + problems=None, + results=None, + runtime_measure="n_evaluations", + normalize_runtime=False, + stopping_criterion="y", + x_precision=1e-4, + y_precision=1e-4, +): + """Compare optimizers over a problem set. + + This plot answers the question: What percentage of problems can each algorithm + solve within a certain runtime budget? + + The runtime budget is plotted on the x axis and the share of problems each + algorithm solved on the y axis. + + Thus, algorithms that are very specialized and perform well on some share of + problems but are not able to solve more problems with a larger computational budget + will have steep increases and then flat lines. Algorithms that are robust but slow, + will have low shares in the beginning but reach very high. + + Note that failing to converge according to the given stopping_criterion and + precisions is scored as needing an infinite computational budget. + + For details, see the description of performance and data profiles by + Moré and Wild (2009). + + Args: + problems (dict): estimagic benchmarking problems dictionary. Keys are the + problem names. Values contain information on the problem, including the + solution value. + results (dict): estimagic benchmarking results dictionary. Keys are + tuples of the form (problem, algorithm), values are dictionaries of the + collected information on the benchmark run, including 'criterion_history' + and 'time_history'. + runtime_measure (str): "n_evaluations" or "walltime". + This is the runtime until the desired convergence was reached by an + algorithm. This is called performance measure by Moré and Wild (2009). + normalize_runtime (bool): If True the runtime each algorithm needed for each + problem is scaled by the time the fastest algorithm needed. If True, the + resulting plot is what Moré and Wild (2009) called data profiles. + stopping_criterion (str): one of "x_and_y", "x_or_y", "x", "y". Determines + how convergence is determined from the two precisions. + x_precision (float or None): how close an algorithm must have gotten to the + true parameter values (as percent of the Euclidean distance between start + and solution parameters) before the criterion for clipping and convergence + is fulfilled. + y_precision (float or None): how close an algorithm must have gotten to the + true criterion values (as percent of the distance between start + and solution criterion value) before the criterion for clipping and + convergence is fulfilled. + + Returns: + fig, ax + + """ + if stopping_criterion is None: + raise ValueError( + "You must specify a stopping criterion for the performance plot. " + ) + df, converged_info = create_convergence_histories( + problems=problems, + results=results, + stopping_criterion=stopping_criterion, + x_precision=x_precision, + y_precision=y_precision, + ) + + solution_times = _create_solution_times( + df, + runtime_measure=runtime_measure, + converged_info=converged_info, + ) + + if normalize_runtime: + solution_times = solution_times.divide(solution_times.min(axis=1), axis=0) + # set again to inf because no inf Timedeltas were allowed. + solution_times[~converged_info] = np.inf + else: + if ( + runtime_measure == "walltime" + and (solution_times == pd.Timedelta(weeks=1000)).any().any() + ): + warnings.warn( + "Some algorithms did not converge. Their walltime has been " + "set to a very high value instead of infinity because Timedeltas do not" + "support infinite values." + ) + + # create performance profiles + alphas = _determine_alpha_grid(solution_times) + for_each_alpha = pd.concat( + {alpha: solution_times <= alpha for alpha in alphas}, + names=["alpha"], + ) + performance_profiles = for_each_alpha.groupby("alpha").mean().stack().reset_index() + + # Build plot + fig, ax = plt.subplots(figsize=(8, 6)) + n_algos = len(solution_times.columns) + sns.lineplot( + data=performance_profiles, + x="alpha", + y=0, + hue="algorithm", + ax=ax, + lw=2.5, + alpha=1.0, + palette=get_colors("categorical", n_algos), + ) + + # Plot Styling + xlabels = { + ("n_evaluations", True): "Multiple of Minimal Number of Function Evaluations\n" + "Needed to Solve the Problem", + ( + "walltime", + True, + ): "Multiple of Minimal Wall Time\nNeeded to Solve the Problem", + ("n_evaluations", False): "Number of Function Evaluations", + ("walltime", False): "Wall Time Needed to Solve the Problem", + } + + ax.set_xlabel(xlabels[(runtime_measure, normalize_runtime)]) + ax.set_ylabel("Share of Problems Solved") + spine_lw = ax.spines["bottom"].get_linewidth() + ax.axhline(1.0, color="silver", xmax=0.955, lw=spine_lw) + ax.legend(title=None) + fig.tight_layout() + + return fig, ax + + +def _create_solution_times(df, runtime_measure, converged_info): + """Find the solution time for each algorithm and problem. + + Args: + df (pandas.DataFrame): contains 'problem', 'algorithm' and *runtime_measure* + as columns. + runtime_measure (str): 'walltime' or 'n_evaluations'. + converged_info (pandas.DataFrame): columns are the algorithms, index are the + problems. The values are boolean and True when the algorithm arrived at + the solution with the desired precision. + + Returns: + solution_times (pandas.DataFrame): columns are algorithms, index are problems. + The values are either the number of evaluations or the walltime each + algorithm needed to achieve the desired precision. If the desired precision + was not achieved the value is set to np.inf (for n_evaluations) or 7000 days + (for walltime since there no infinite value is allowed). + + """ + solution_times = df.groupby(["problem", "algorithm"])[runtime_measure].max() + solution_times = solution_times.unstack() + + # inf not allowed for timedeltas so put something very large + if runtime_measure == "walltime": + inf_value = pd.Timedelta(weeks=1000) + elif runtime_measure == "n_evaluations": + inf_value = np.inf + else: + raise ValueError( + "Only 'walltime' or 'n_evaluations' are allowed as " + f"runtime_measure. You specified {runtime_measure}." + ) + + solution_times[~converged_info] = inf_value + return solution_times + + +def _determine_alpha_grid(solution_times): + switch_points = _find_switch_points(solution_times=solution_times) + + # add point to the right + point_to_right = switch_points[-1] * 1.05 + extended_switch_points = np.append(switch_points, point_to_right) + mid_points = (extended_switch_points[:-1] + extended_switch_points[1:]) / 2 + alphas = sorted(np.append(extended_switch_points, mid_points)) + return alphas + + +def _find_switch_points(solution_times): + """Determine the switch points of the performance profiles. + + Args: + solution_times (pandas.DataFrame): columns are the names of the algorithms, + the index are the problems. Values are performance measures. + They can be either float, when normalize_runtime was True or int when the + runtime_measure are not normalized function evaluations or datetime when + the not normalized walltime is used. + + Returns: + list: sorted switching points + + """ + switch_points = np.unique(solution_times.values) + if pd.api.types.is_float_dtype(switch_points): + switch_points += 1e-10 + switch_points = switch_points[np.isfinite(switch_points)] + return switch_points diff --git a/tests/__init__.py b/tests/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/tests/benchmarking/__init__.py b/tests/benchmarking/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/tests/benchmarking/test_cartis_roberts.py b/tests/benchmarking/test_cartis_roberts.py new file mode 100644 index 000000000..5d65fe95c --- /dev/null +++ b/tests/benchmarking/test_cartis_roberts.py @@ -0,0 +1,37 @@ +import numpy as np +import pytest +from estimagic.benchmarking.cartis_roberts import CARTIS_ROBERTS_PROBLEMS +from estimagic.benchmarking.cartis_roberts import get_start_points_bdvalues +from estimagic.benchmarking.cartis_roberts import get_start_points_msqrta +from numpy.testing import assert_array_almost_equal + + +@pytest.mark.parametrize("name, specification", list(CARTIS_ROBERTS_PROBLEMS.items())) +def test_cratis_roberts_function_at_start_x(name, specification): + _criterion = specification["criterion"] + _x = specification["start_x"] + _contributions = _criterion(_x) + calculated = _contributions @ _contributions + expected = specification["start_criterion"] + assert np.allclose(calculated, expected) + + +def test_get_start_points_bdvalues(): + expected = np.array([-0.1389, -0.2222, -0.2500, -0.2222, -0.1389]) + result = get_start_points_bdvalues(5) + assert_array_almost_equal(expected, result, decimal=4) + + +def test_get_start_points_msqrta(): + matlab_mat = np.array( + [ + [0.8415, -0.7568, 0.4121, -0.2879, -0.1324], + [-0.9918, -0.9538, 0.9200, -0.6299, -0.5064], + [0.9988, -0.4910, -0.6020, 0.9395, -0.9301], + [-0.9992, -0.0265, -0.4041, 0.2794, -0.8509], + [0.9235, 0.1935, 0.9365, -0.8860, 0.1760], + ] + ) + expected = 0.2 * matlab_mat.flatten() + result = get_start_points_msqrta(5) + assert_array_almost_equal(result, expected, decimal=4) diff --git a/tests/benchmarking/test_get_benchmark_problems.py b/tests/benchmarking/test_get_benchmark_problems.py new file mode 100644 index 000000000..327ab4938 --- /dev/null +++ b/tests/benchmarking/test_get_benchmark_problems.py @@ -0,0 +1,33 @@ +from itertools import product + +import numpy as np +import pytest +from estimagic import get_benchmark_problems + +PARMETRIZATION = [] +for name in ["more_wild", "cartis_roberts"]: + for additive, multiplicative in product([False, True], repeat=2): + PARMETRIZATION.append((name, additive, multiplicative)) + + +@pytest.mark.parametrize("name, additive_noise, multiplicative_noise", PARMETRIZATION) +def test_get_problems(name, additive_noise, multiplicative_noise): + is_noisy = any((additive_noise, multiplicative_noise)) + problems = get_benchmark_problems( + name=name, + additive_noise=additive_noise, + multiplicative_noise=multiplicative_noise, + ) + first_name = list(problems)[0] + first = problems[first_name] + func = first["inputs"]["criterion"] + params = first["inputs"]["params"] + + np.random.seed() + first_eval = func(params)["value"] + second_eval = func(params)["value"] + + if is_noisy: + assert first_eval != second_eval + else: + assert first_eval == second_eval diff --git a/tests/benchmarking/test_more_wild.py b/tests/benchmarking/test_more_wild.py new file mode 100644 index 000000000..bb5ab21cd --- /dev/null +++ b/tests/benchmarking/test_more_wild.py @@ -0,0 +1,27 @@ +import numpy as np +import pytest +from estimagic.benchmarking.more_wild import get_start_points_mancino +from estimagic.benchmarking.more_wild import MORE_WILD_PROBLEMS + + +@pytest.mark.parametrize("name, specification", list(MORE_WILD_PROBLEMS.items())) +def test_more_wild_function_at_start_x(name, specification): + _criterion = specification["criterion"] + _x = specification["start_x"] + _contributions = _criterion(_x) + calculated = _contributions @ _contributions + expected = specification["start_criterion"] + assert np.allclose(calculated, expected) + + if specification.get("solution_x") is not None: + _x = specification["solution_x"] + _contributions = _criterion(_x) + calculated = _contributions @ _contributions + expected = specification["solution_criterion"] + assert np.allclose(calculated, expected, rtol=1e-8, atol=1e-8) + + +def test_get_start_points_mancino(): + expected = (np.array([102.4824, 96.3335, 90.4363, 84.7852, 79.3747]),) + result = get_start_points_mancino(5) + assert np.allclose(expected, result) diff --git a/tests/benchmarking/test_noise_distributions.py b/tests/benchmarking/test_noise_distributions.py new file mode 100644 index 000000000..b32dedaa1 --- /dev/null +++ b/tests/benchmarking/test_noise_distributions.py @@ -0,0 +1,28 @@ +import numpy as np +import pandas as pd +import pytest +from estimagic.benchmarking.get_benchmark_problems import _sample_from_distribution +from estimagic.benchmarking.noise_distributions import NOISE_DISTRIBUTIONS + + +@pytest.mark.parametrize("distribution", NOISE_DISTRIBUTIONS) +def test_sample_from_distribution(distribution): + np.random.seed(1234) + mean = 0.33 + std = 0.55 + correlation = 0.44 + sample = _sample_from_distribution( + distribution=distribution, + mean=mean, + std=std, + size=(100_000, 5), + correlation=correlation, + ) + calculated_mean = sample.mean() + calculated_std = sample.std() + corrmat = pd.DataFrame(sample).corr().to_numpy().round(2) + calculated_avgcorr = corrmat[~np.eye(len(corrmat)).astype(bool)].mean() + + assert np.allclose(calculated_mean, mean, atol=0.001) + assert np.allclose(calculated_std, std, atol=0.001) + assert np.allclose(calculated_avgcorr, correlation, atol=0.001) diff --git a/tests/benchmarking/test_process_benchmark_results.py b/tests/benchmarking/test_process_benchmark_results.py new file mode 100644 index 000000000..5264e7657 --- /dev/null +++ b/tests/benchmarking/test_process_benchmark_results.py @@ -0,0 +1,440 @@ +import numpy as np +import pandas as pd +import pytest +from estimagic.benchmarking.process_benchmark_results import _clip_histories +from estimagic.benchmarking.process_benchmark_results import _find_first_converged +from estimagic.benchmarking.process_benchmark_results import ( + _get_history_as_stacked_sr_from_results, +) +from estimagic.benchmarking.process_benchmark_results import ( + _get_history_of_the_parameter_distance, +) +from estimagic.benchmarking.process_benchmark_results import _make_history_monotone +from estimagic.benchmarking.process_benchmark_results import _normalize +from estimagic.benchmarking.process_benchmark_results import ( + create_convergence_histories, +) + +PROBLEMS = ["prob1", "prob2", "prob3"] + + +@pytest.fixture +def problem_algo_eval_df(): + df = pd.DataFrame() + df["problem"] = ["prob1"] * 8 + ["prob2"] * 6 + df["algorithm"] = ["algo1"] * 4 + ["algo2"] * 4 + ["algo1"] * 3 + ["algo2"] * 3 + df["n_evaluations"] = [0, 1, 2, 3] * 2 + [0, 1, 2] * 2 + return df + + +def test_find_first_converged(problem_algo_eval_df): + # we can assume monotonicity, i.e. no switch back from True to False + converged = pd.Series( + [ # in the middle + False, + False, + True, + True, + ] + + [ # last entry + False, + False, + False, + True, + ] + + [ # first entry + True, + True, + True, + ] + + [ # not converged + False, + False, + False, + ] + ) + res = _find_first_converged(converged, problem_algo_eval_df) + expected = pd.Series( + [ # in the middle + False, + False, + True, + False, + ] + + [ # last entry + False, + False, + False, + True, + ] + + [ # first entry + True, + False, + False, + ] + + [ # not converged + False, + False, + False, + ] + ) + pd.testing.assert_series_equal(res, expected) + + +def test_normalize_minimize(): + start_values = pd.Series([5, 4, 10], index=PROBLEMS) + target_values = pd.Series([1, 0, 0], index=PROBLEMS) + + df = pd.DataFrame() + df["problem"] = PROBLEMS * 3 + df["criterion"] = start_values.tolist() + [2, 3, 9] + target_values.tolist() + + res = _normalize( + df=df, col="criterion", start_values=start_values, target_values=target_values + ) + + # total improvements are [4, 4, 10] + # missing improvements are [1, 3, 9] for the [2, 3, 9] part + + expected = pd.Series([1] * 3 + [0.25, 0.75, 0.9] + [0] * 3) + pd.testing.assert_series_equal(res, expected) + + +def test_normalize_maximize(): + start_values = pd.Series([1, 2, 3], index=PROBLEMS) + target_values = pd.Series([5, 7, 10], index=PROBLEMS) + + df = pd.DataFrame() + df["problem"] = PROBLEMS * 3 + df["criterion"] = start_values.tolist() + [2, 4, 9] + target_values.tolist() + + res = _normalize( + df=df, col="criterion", start_values=start_values, target_values=target_values + ) + + # total improvements are [4, 5, 7] + # missing improvements are [3, 3, 1] for the [2, 4, 9] part + + expected = pd.Series([1] * 3 + [3 / 4, 3 / 5, 1 / 7] + [0] * 3) + pd.testing.assert_series_equal(res, expected) + + +@pytest.fixture +def df_for_clip_histories(problem_algo_eval_df): + df = problem_algo_eval_df + df["monotone_criterion_normalized"] = [ + # prob1, algo1: converged on 2nd + 1.8, # keep, not converged + 0.05, # keep, 1st converged + 0.03, # clip + 0.0, # clip + # prob1, algo2: converged on last + 5.4, # keep, not converged + 3.3, # keep, not converged + 2.2, # keep, not converged + 0.08, # keep, 1st converged + # prob2, algo1: converged on first + 0.08, # keep, 1st converged + 0.04, # drop + 0.01, # drop + # prob2, algo2: not converged + 3.3, # keep, not converged + 2.2, # keep, not converged + 1.1, # keep, not converged + ] + df["monotone_parameter_distance_normalized"] = [ + # prob1, algo1: converged on 3rd -> 1 after y criterion + 1.8, # keep, not converged + 0.5, # keep, not converged + 0.05, # keep, 1st converged + 0.01, # clip + # prob1, algo2: converged on 3rd -> 1 before y criterion + 2.2, # keep, not converged + 1.1, # keep, not converged + 0.04, # keep, 1st converged + 0.03, # clip + # prob2, algo1: not converged (converged in y) + 3.0, # keep, not converged + 3.0, # keep, not converged + 3.0, # keep, not converged + # prob2, algo2: converged on 3rd (not converged in y) + 2.2, # keep, not converged + 1.1, # keep, not converged + 0.04, # keep, 1st converged + ] + return df + + +def test_clip_histories_y(df_for_clip_histories): + expected_shortened = df_for_clip_histories.loc[[0, 1, 4, 5, 6, 7, 8, 11, 12, 13]] + expected_info = pd.DataFrame( + {"algo1": [True, True], "algo2": [True, False]}, + index=["prob1", "prob2"], + ) + res_shortened, res_info = _clip_histories( + df=df_for_clip_histories, + stopping_criterion="y", + x_precision=0.1, + y_precision=0.1, + ) + pd.testing.assert_frame_equal(res_shortened, expected_shortened) + pd.testing.assert_frame_equal(res_info, expected_info, check_names=False) + + +def test_clip_histories_x(df_for_clip_histories): + expected_shortened = df_for_clip_histories.loc[ + [0, 1, 2, 4, 5, 6, 8, 9, 10, 11, 12, 13] + ] + expected_info = pd.DataFrame( + {"algo1": [True, False], "algo2": [True, True]}, + index=["prob1", "prob2"], + ) + res_shortened, res_info = _clip_histories( + df=df_for_clip_histories, + stopping_criterion="x", + x_precision=0.1, + y_precision=0.1, + ) + pd.testing.assert_frame_equal(res_shortened, expected_shortened) + pd.testing.assert_frame_equal(res_info, expected_info, check_names=False) + + +def test_clip_histories_x_and_y_with_nan(df_for_clip_histories): + df = df_for_clip_histories + df.loc[df["problem"] == "prob2", "monotone_parameter_distance_normalized"] = np.nan + + expected_shortened = df.loc[[0, 1, 2, 4, 5, 6, 7]] + expected_info = pd.DataFrame( + {"algo1": [True], "algo2": [True]}, + index=["prob1"], + ) + + res_shortened, res_info = _clip_histories( + df=df, + stopping_criterion="x_and_y", + x_precision=0.1, + y_precision=0.1, + ) + pd.testing.assert_frame_equal(res_shortened, expected_shortened) + pd.testing.assert_frame_equal(res_info, expected_info, check_names=False) + + +def test_clip_histories_x_or_y_no_nan(df_for_clip_histories): + df = df_for_clip_histories + + expected_shortened = df.loc[[0, 1, 4, 5, 6, 8, 11, 12, 13]] + expected_info = pd.DataFrame( + {"algo1": [True, True], "algo2": [True, True]}, + index=["prob1", "prob2"], + ) + + res_shortened, res_info = _clip_histories( + df=df, + stopping_criterion="x_or_y", + x_precision=0.1, + y_precision=0.1, + ) + pd.testing.assert_frame_equal(res_shortened, expected_shortened) + pd.testing.assert_frame_equal(res_info, expected_info, check_names=False) + + +def test_make_history_monotone_minimize(): + sorted_df = pd.DataFrame( + columns=["problem", "algorithm", "n_evaluations", "to_make_monotone"], + data=[ + # already monotone + ["prob1", "algo1", 0, 3.3], + ["prob1", "algo1", 1, 2.2], + ["prob1", "algo1", 2, 1.1], + # 3rd & 4th entry must be changed + ["prob1", "algo2", 0, 3.3], + ["prob1", "algo2", 1, 1.1], + ["prob1", "algo2", 2, 2.2], # 1.1 + ["prob1", "algo2", 2, 5.0], # 1.1 + # up, down, up, down + ["prob2", "algo1", 0, 2.2], # 2.2 + ["prob2", "algo1", 1, 3.3], # 2.2 + ["prob2", "algo1", 2, 1.1], # 1.1 + ["prob2", "algo1", 3, 2.5], # 1.1 + ["prob2", "algo1", 4, 2.0], # 1.1 + ], + ) + np.random.seed(40954) + shuffled = sorted_df.sample(frac=1) + + res_shuffled = _make_history_monotone( + df=shuffled, target_col="to_make_monotone", direction="minimize" + ) + res_sorted = _make_history_monotone( + df=sorted_df, target_col="to_make_monotone", direction="minimize" + ) + + expected = pd.Series( + [ # prob1, algo1 + 3.3, + 2.2, + 1.1, + # prob1, algo2 + 3.3, + 1.1, + 1.1, + 1.1, + # prob2, algo1 + 2.2, + 2.2, + 1.1, + 1.1, + 1.1, + ], + name="to_make_monotone", + ) + pd.testing.assert_series_equal(res_sorted, expected) + pd.testing.assert_series_equal(res_shuffled, expected) + + +@pytest.fixture +def benchmark_results(): + sec = pd.Timedelta(seconds=1) + results = { + ("prob1", "algo1"): { + "criterion_history": pd.Series([1, 2, 3]), + "time_history": pd.Series([sec, 2 * sec, 3 * sec]), + "params_history": pd.DataFrame( + [ + [1, 2], + [1, 1], + [0.5, 0.5], + ] + ), + }, + ("prob1", "algo2"): { + "criterion_history": pd.Series([1, 2.5]), + "time_history": pd.Series([0.5 * sec, 1.5 * sec]), + "params_history": pd.DataFrame([[2, 3], [2, 2]]), + }, + ("prob2", "algo1"): { + "criterion_history": pd.Series([50, 40]), + "time_history": pd.Series([3 * sec, 3.5 * sec]), + "params_history": pd.DataFrame([[2], [4]]), + }, + ("prob2", "algo2"): { + "criterion_history": pd.Series([35]), + "time_history": pd.Series([3.2 * sec]), + "params_history": pd.DataFrame([[4.2]]), + }, + } + return results + + +def test_get_history_of_the_parameter_distance(benchmark_results): + x_opt = {"prob1": np.array([1, 1]), "prob2": np.array([3])} + res = _get_history_of_the_parameter_distance(results=benchmark_results, x_opt=x_opt) + expected_df = pd.DataFrame( + columns=["problem", "algorithm", "evaluation", "parameter_distance"], + data=[ + ["prob1", "algo1", 0, 1], + ["prob1", "algo1", 1, 0], + ["prob1", "algo1", 2, np.sqrt(2 * 0.5 ** 2)], + ["prob1", "algo2", 0, np.sqrt(5)], + ["prob1", "algo2", 1, np.sqrt(2)], + ["prob2", "algo1", 0, 1], + ["prob2", "algo1", 1, 1], + ["prob2", "algo2", 0, 1.2], + ], + ) + expected = expected_df.set_index(["problem", "algorithm", "evaluation"])[ + "parameter_distance" + ] + pd.testing.assert_series_equal(res, expected) + + +def test_get_history_as_stacked_sr_from_results(benchmark_results): + res = _get_history_as_stacked_sr_from_results( + benchmark_results, key="criterion_history" + ) + expected_df = pd.DataFrame( + columns=["problem", "algorithm", "evaluation", "criterion"], + data=[ + ["prob1", "algo1", 0, 1], + ["prob1", "algo1", 1, 2], + ["prob1", "algo1", 2, 3], + ["prob1", "algo2", 0, 1], + ["prob1", "algo2", 1, 2.5], + ["prob2", "algo1", 0, 50], + ["prob2", "algo1", 1, 40], + ["prob2", "algo2", 0, 35], + ], + ) + expected = expected_df.set_index(["problem", "algorithm", "evaluation"])[ + "criterion" + ] + pd.testing.assert_series_equal(res, expected) + + +def test_create_convergence_histories(benchmark_results): + problems = { + "prob1": { + "solution": {"value": 5, "params": pd.DataFrame(data={"value": [1, 1]})} + }, + "prob2": { + "solution": {"value": 1, "params": pd.DataFrame(data={"value": [3]})} + }, + } + res, _ = create_convergence_histories( + problems=problems, + results=benchmark_results, + stopping_criterion=None, + x_precision=None, + y_precision=None, + ) + + expected_criterion = pd.DataFrame( + columns=["problem", "algorithm", "evaluation", "criterion"], + data=[ + ["prob1", "algo1", 0, 1], + ["prob1", "algo1", 1, 2], + ["prob1", "algo1", 2, 3], + ["prob1", "algo2", 0, 1], + ["prob1", "algo2", 1, 2.5], + ["prob2", "algo1", 0, 50], + ["prob2", "algo1", 1, 40], + ["prob2", "algo2", 0, 35], + ], + ) + + expected_x_distance = pd.DataFrame( + columns=["problem", "algorithm", "evaluation", "parameter_distance"], + data=[ + ["prob1", "algo1", 0, 1], + ["prob1", "algo1", 1, 0], + ["prob1", "algo1", 2, np.sqrt(2 * 0.5 ** 2)], + ["prob1", "algo2", 0, np.sqrt(5)], + ["prob1", "algo2", 1, np.sqrt(2)], + ["prob2", "algo1", 0, 1], + ["prob2", "algo1", 1, 1], + ["prob2", "algo2", 0, 1.2], + ], + ) + + expected = pd.merge( + expected_criterion, + expected_x_distance, + on=["problem", "algorithm", "evaluation"], + ).rename(columns={"evaluation": "n_evaluations"}) + + to_compare = [ + "problem", + "algorithm", + "n_evaluations", + "criterion", + "parameter_distance", + ] + # missing: + # - criterion_normalized + # - monotone_criterion + # - monotone_criterion_normalized + # - parameter_distance_normalized + # - monotone_parameter_distance + # - monotone_parameter_distance_normalized + + pd.testing.assert_frame_equal(res[to_compare], expected) diff --git a/tests/benchmarking/test_run_benchmark.py b/tests/benchmarking/test_run_benchmark.py new file mode 100644 index 000000000..fa4c5c5dc --- /dev/null +++ b/tests/benchmarking/test_run_benchmark.py @@ -0,0 +1,58 @@ +from estimagic import get_benchmark_problems +from estimagic.benchmarking.run_benchmark import run_benchmark + + +def test_run_benchmark_dict_options(tmpdir): + all_problems = get_benchmark_problems("more_wild") + first_two_names = list(all_problems)[:2] + first_two = {name: all_problems[name] for name in first_two_names} + + optimize_options = { + "default_lbfgsb": "scipy_lbfgsb", + "tuned_lbfgsb": { + "algorithm": "scipy_lbfgsb", + "algo_options": {"convergence.relative_criterion_tolerance": 1e-10}, + }, + } + + logging_directory = tmpdir / "benchmark_logs" + + res = run_benchmark( + problems=first_two, + optimize_options=optimize_options, + logging_directory=logging_directory, + ) + + expected_keys = { + ("linear_full_rank_good_start", "default_lbfgsb"), + ("linear_full_rank_bad_start", "default_lbfgsb"), + ("linear_full_rank_good_start", "tuned_lbfgsb"), + ("linear_full_rank_bad_start", "tuned_lbfgsb"), + } + + assert set(res) == expected_keys + + +def test_run_benchmark_list_options(tmpdir): + all_problems = get_benchmark_problems("example") + first_two_names = list(all_problems)[:2] + first_two = {name: all_problems[name] for name in first_two_names} + + optimize_options = ["scipy_lbfgsb", "scipy_neldermead"] + + logging_directory = tmpdir / "benchmark_logs" + + res = run_benchmark( + problems=first_two, + optimize_options=optimize_options, + logging_directory=logging_directory, + ) + + expected_keys = { + ("linear_full_rank_good_start", "scipy_lbfgsb"), + ("rosenbrock_good_start", "scipy_lbfgsb"), + ("linear_full_rank_good_start", "scipy_neldermead"), + ("rosenbrock_good_start", "scipy_neldermead"), + } + + assert set(res) == expected_keys diff --git a/tests/visualization/test_convergence_plot.py b/tests/visualization/test_convergence_plot.py new file mode 100644 index 000000000..ee1451c18 --- /dev/null +++ b/tests/visualization/test_convergence_plot.py @@ -0,0 +1,47 @@ +import matplotlib.pyplot as plt +import pytest +from estimagic import get_benchmark_problems +from estimagic.benchmarking.run_benchmark import run_benchmark +from estimagic.visualization.convergence_plot import convergence_plot + +# integration test to make sure non default argument do not throw Errors +profile_options = [ + {"n_cols": 3}, + {"distance_measure": "parameter_distance"}, + {"monotone": False}, + {"normalize_distance": False}, + {"runtime_measure": "walltime"}, + {"stopping_criterion": None}, + {"stopping_criterion": "x"}, + {"stopping_criterion": "x_and_y"}, + {"stopping_criterion": "x_or_y"}, + {"x_precision": 1e-5}, + {"y_precision": 1e-5}, +] + + +@pytest.mark.parametrize("options", profile_options) +def test_convergence_plot_options(options): + problems = get_benchmark_problems("example") + stop_after_10 = { + "stopping_max_criterion_evaluations": 10, + "stopping_max_iterations": 10, + } + optimizers = { + "lbfgsb": {"algorithm": "scipy_lbfgsb", "algo_options": stop_after_10}, + "nm": {"algorithm": "scipy_neldermead", "algo_options": stop_after_10}, + } + results = run_benchmark( + problems, + optimizers, + n_cores=1, # must be 1 for the test to work + logging_directory="logging", + ) + + convergence_plot( + problems=problems, + results=results, + problem_subset=["bard_good_start"], + **options + ) + plt.close() diff --git a/tests/visualization/test_profile_plot.py b/tests/visualization/test_profile_plot.py new file mode 100644 index 000000000..63b5100da --- /dev/null +++ b/tests/visualization/test_profile_plot.py @@ -0,0 +1,137 @@ +import matplotlib.pyplot as plt +import numpy as np +import pandas as pd +import pytest +from estimagic import get_benchmark_problems +from estimagic.benchmarking.run_benchmark import run_benchmark +from estimagic.visualization.profile_plot import _create_solution_times +from estimagic.visualization.profile_plot import _determine_alpha_grid +from estimagic.visualization.profile_plot import _find_switch_points +from estimagic.visualization.profile_plot import profile_plot + + +@pytest.fixture +def performance_ratios(): + df = pd.DataFrame( + data={"algo1": [1.0, 1.0, 4.0], "algo2": [1.5, np.inf, 1.0]}, + index=["prob1", "prob2", "prob3"], + ) + return df + + +def test_find_switch_points(performance_ratios): + res = _find_switch_points(performance_ratios) + expected = np.array([1.0, 1.5, 4.0]) + np.testing.assert_array_almost_equal(res, expected) + + +def test_determine_alpha_grid(performance_ratios): + res = _determine_alpha_grid(performance_ratios) + expected = np.array([1.0 + 1e-10, 1.25, 1.5, 2.75, 4.0, 4.0 * 1.025, 4.0 * 1.05]) + np.testing.assert_array_almost_equal(res, expected) + + +def test_create_solution_times_n_evaluations(): + df = pd.DataFrame( + columns=["problem", "algorithm", "n_evaluations"], + data=[ + ["prob1", "algo1", 0], + ["prob1", "algo1", 1], + # + ["prob1", "algo2", 2], + ["prob1", "algo2", 3], + # + ["prob2", "algo1", 5], + # + ["prob2", "algo2", 0], + ["prob2", "algo2", 1], + ], + ) + info = pd.DataFrame( + { + "algo1": [True, True], + "algo2": [True, False], + }, + index=["prob1", "prob2"], + ) + expected = pd.DataFrame( + { + "algo1": [1, 5], + "algo2": [3, np.inf], + }, + index=pd.Index(["prob1", "prob2"], name="problem"), + ) + expected.columns.name = "algorithm" + + res = _create_solution_times( + df=df, runtime_measure="n_evaluations", converged_info=info + ) + pd.testing.assert_frame_equal(res, expected) + + +def test_create_solution_times_walltime(): + df = pd.DataFrame( + columns=["problem", "algorithm", "n_evaluations", "walltime"], + data=[ + ["prob1", "algo1", 0, pd.Timedelta(seconds=0)], + ["prob1", "algo1", 1, pd.Timedelta(seconds=1)], + # + ["prob1", "algo2", 2, pd.Timedelta(seconds=2)], + ["prob1", "algo2", 3, pd.Timedelta(seconds=3)], + # + ["prob2", "algo1", 5, pd.Timedelta(seconds=5)], + # + ["prob2", "algo2", 0, pd.Timedelta(seconds=0)], + ["prob2", "algo2", 1, pd.Timedelta(seconds=1)], + ], + ) + info = pd.DataFrame( + { + "algo1": [True, True], + "algo2": [True, False], + }, + index=["prob1", "prob2"], + ) + expected = pd.DataFrame( + { + "algo1": [pd.Timedelta(seconds=1), pd.Timedelta(seconds=5)], + "algo2": [pd.Timedelta(seconds=3), pd.Timedelta(weeks=1000)], + }, + index=pd.Index(["prob1", "prob2"], name="problem"), + ) + expected.columns.name = "algorithm" + + res = _create_solution_times(df=df, runtime_measure="walltime", converged_info=info) + pd.testing.assert_frame_equal(res, expected) + + +# integration test to make sure non default argument do not throw Errors +profile_options = [ + {"runtime_measure": "walltime"}, + {"stopping_criterion": "x_or_y"}, +] + + +@pytest.mark.parametrize("options", profile_options) +def test_profile_plot_options(options): + problems = get_benchmark_problems("example") + stop_after_10 = { + "stopping_max_criterion_evaluations": 10, + "stopping_max_iterations": 10, + } + optimizers = { + "lbfgsb": {"algorithm": "scipy_lbfgsb", "algo_options": stop_after_10}, + "neldermead": { + "algorithm": "scipy_neldermead", + "algo_options": stop_after_10, + }, + } + results = run_benchmark( + problems, + optimizers, + n_cores=1, # must be 1 for the test to work + logging_directory="logging", + ) + + profile_plot(problems=problems, results=results, **options) + plt.close() diff --git a/tox.ini b/tox.ini index 4b3322da1..0a21ca97d 100644 --- a/tox.ini +++ b/tox.ini @@ -100,6 +100,7 @@ filterwarnings = ignore: indexing past lexsort depth may impact performance. ignore: Method .ptp is deprecated and will be removed in a future version. Use numpy.ptp instead. ignore: In a future version of pandas all arguments of concat except for the argument 'objs' will be keyword-only + ignore: Only a subset of the cartis_roberts markers = wip: Tests that are work-in-progress. slow: Tests that take a long time to run and are skipped in continuous integration.