3.5 新闻分类：多分类问题.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1 路透社数据集\n",
    "\n",
    "本节使用路透社数据集，它包含许多短新闻及其对应的主题，由路透社在 1986 年发布。它\n",
    "是一个简单的、广泛使用的文本分类数据集。它包括 46 个不同的主题：某些主题的样本更多，\n",
    "但训练集中每个主题都有至少 10 个样本。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 防止报错 OMP: Error #15: Initializing libiomp5.dylib, but found libiomp5.dylib already initialized.\n",
    "import os\n",
    "os.environ[\"KMP_DUPLICATE_LIB_OK\"]=\"TRUE\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from keras.datasets import reuters\n",
    "\n",
    "(train_data, train_labels), (test_data, test_labels) = reuters.load_data(num_words=10000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2 准备数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "def vectorize_sequences(sequences, dimension=10000):\n",
    "    results = np.zeros((len(sequences), dimension))\n",
    "    for i, sequence in enumerate(sequences):\n",
    "        results[i, sequence] = 1.\n",
    "    return results\n",
    "# 数据向量化\n",
    "x_train = vectorize_sequences(train_data) \n",
    "x_test = vectorize_sequences(test_data) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def to_one_hot(labels, dimension=46):\n",
    "    results = np.zeros((len(labels), dimension))\n",
    "    for i, label in enumerate(labels):\n",
    "        results[i, label] = 1.\n",
    "    return results\n",
    "#标签向量化\n",
    "one_hot_train_labels = to_one_hot(train_labels)\n",
    "one_hot_test_labels = to_one_hot(test_labels) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3 构建网络"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 模型定义\n",
    "from keras import models\n",
    "from keras import layers\n",
    "model = models.Sequential()\n",
    "model.add(layers.Dense(64, activation='relu', input_shape=(10000,)))\n",
    "model.add(layers.Dense(64, activation='relu'))\n",
    "model.add(layers.Dense(46, activation='softmax'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 编译模型\n",
    "model.compile(optimizer='rmsprop',\n",
    "    loss='categorical_crossentropy',\n",
    "    metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4 验证方法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 留出验证集\n",
    "x_val = x_train[:1000]\n",
    "partial_x_train = x_train[1000:]\n",
    "y_val = one_hot_train_labels[:1000]\n",
    "partial_y_train = one_hot_train_labels[1000:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 7982 samples, validate on 1000 samples\n",
      "Epoch 1/20\n",
      "7982/7982 [==============================] - 2s 207us/step - loss: 2.5248 - acc: 0.4981 - val_loss: 1.7192 - val_acc: 0.6140\n",
      "Epoch 2/20\n",
      "7982/7982 [==============================] - 1s 135us/step - loss: 1.4444 - acc: 0.6888 - val_loss: 1.3462 - val_acc: 0.7090\n",
      "Epoch 3/20\n",
      "7982/7982 [==============================] - 1s 140us/step - loss: 1.0953 - acc: 0.7643 - val_loss: 1.1706 - val_acc: 0.7410\n",
      "Epoch 4/20\n",
      "7982/7982 [==============================] - 1s 142us/step - loss: 0.8697 - acc: 0.8166 - val_loss: 1.0850 - val_acc: 0.7570\n",
      "Epoch 5/20\n",
      "7982/7982 [==============================] - 1s 139us/step - loss: 0.7037 - acc: 0.8484 - val_loss: 0.9839 - val_acc: 0.7830\n",
      "Epoch 6/20\n",
      "7982/7982 [==============================] - 1s 137us/step - loss: 0.5676 - acc: 0.8795 - val_loss: 0.9396 - val_acc: 0.8050\n",
      "Epoch 7/20\n",
      "7982/7982 [==============================] - 1s 138us/step - loss: 0.4610 - acc: 0.9039 - val_loss: 0.9089 - val_acc: 0.8020\n",
      "Epoch 8/20\n",
      "7982/7982 [==============================] - 1s 133us/step - loss: 0.3714 - acc: 0.9230 - val_loss: 0.9347 - val_acc: 0.7910\n",
      "Epoch 9/20\n",
      "7982/7982 [==============================] - 1s 138us/step - loss: 0.3042 - acc: 0.9312 - val_loss: 0.8907 - val_acc: 0.8060\n",
      "Epoch 10/20\n",
      "7982/7982 [==============================] - 1s 142us/step - loss: 0.2542 - acc: 0.9412 - val_loss: 0.9072 - val_acc: 0.8110\n",
      "Epoch 11/20\n",
      "7982/7982 [==============================] - 1s 143us/step - loss: 0.2190 - acc: 0.9474 - val_loss: 0.9165 - val_acc: 0.8100\n",
      "Epoch 12/20\n",
      "7982/7982 [==============================] - 1s 148us/step - loss: 0.1881 - acc: 0.9509 - val_loss: 0.9069 - val_acc: 0.8160\n",
      "Epoch 13/20\n",
      "7982/7982 [==============================] - 1s 136us/step - loss: 0.1707 - acc: 0.9526 - val_loss: 0.9344 - val_acc: 0.8090\n",
      "Epoch 14/20\n",
      "7982/7982 [==============================] - 1s 135us/step - loss: 0.1539 - acc: 0.9555 - val_loss: 0.9625 - val_acc: 0.8060\n",
      "Epoch 15/20\n",
      "7982/7982 [==============================] - 1s 141us/step - loss: 0.1394 - acc: 0.9562 - val_loss: 0.9676 - val_acc: 0.8130\n",
      "Epoch 16/20\n",
      "7982/7982 [==============================] - 1s 146us/step - loss: 0.1320 - acc: 0.9559 - val_loss: 1.0304 - val_acc: 0.8010\n",
      "Epoch 17/20\n",
      "7982/7982 [==============================] - 1s 152us/step - loss: 0.1222 - acc: 0.9578 - val_loss: 1.0302 - val_acc: 0.7960\n",
      "Epoch 18/20\n",
      "7982/7982 [==============================] - 1s 141us/step - loss: 0.1203 - acc: 0.9577 - val_loss: 1.0422 - val_acc: 0.8070\n",
      "Epoch 19/20\n",
      "7982/7982 [==============================] - 1s 157us/step - loss: 0.1146 - acc: 0.9592 - val_loss: 1.0972 - val_acc: 0.7980\n",
      "Epoch 20/20\n",
      "7982/7982 [==============================] - 1s 143us/step - loss: 0.1118 - acc: 0.9598 - val_loss: 1.0728 - val_acc: 0.8010\n"
     ]
    }
   ],
   "source": [
    "# 训练模型，训练20轮\n",
    "history = model.fit(partial_x_train,\n",
    "    partial_y_train,\n",
    "    epochs=20,\n",
    "    batch_size=512,\n",
    "    validation_data=(x_val, y_val))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XmYFNXZ9/HvzS47ggqKzoAY2WQZ\nJ4iCgstjEOO+Iu76IC5xS6K8alwwPGrcCGpMTCIxMhGNRkWDkhgxbgEFRBSRgAiCIA4om6AycL9/\nnJqmGXpmepip6Z6Z3+e66urqqtPVd9f01N3nnKpT5u6IiIgA1Mt0ACIikj2UFEREJEFJQUREEpQU\nREQkQUlBREQSlBRERCRBSUGqlJnVN7MNZrZPVZbNJDPrYmZVfu62mR1lZouTns83s0PTKbsT7/UH\nM7thZ19fxnZ/aWZ/qurtSuY0yHQAkllmtiHpaVPgO2BL9PwSdy+oyPbcfQvQvKrL1gXuvn9VbMfM\nLgbOdvfBSdu+uCq2LbWfkkId5+6Jg3L0S/Rid3+ltPJm1sDdi6ojNhGpfmo+kjJFzQNPmtkTZrYe\nONvMDjazaWa2xsxWmNk4M2sYlW9gZm5mudHzCdH6l8xsvZn9x8w6VbRstP4YM/uvma01swfM7C0z\nO7+UuNOJ8RIzW2hmX5vZuKTX1jez+81stZl9AgwpY//cZGYTSyx7yMzui+YvNrN50ef5JPoVX9q2\nlpnZ4Gi+qZk9HsU2FzgwxfsuirY718yOj5YfADwIHBo1za1K2re3Jr1+ZPTZV5vZc2bWIZ19Ux4z\nOzGKZ42ZvWpm+yetu8HMlpvZOjP7OOmz9jezWdHylWZ2d7rvJzFwd02acHeAxcBRJZb9EvgeOI7w\nI2IX4IfAQYSaZmfgv8AVUfkGgAO50fMJwCogH2gIPAlM2ImyuwPrgROiddcCm4HzS/ks6cT4PNAK\nyAW+Kv7swBXAXKAj0BZ4PfyrpHyfzsAGoFnStr8E8qPnx0VlDDgC2AT0itYdBSxO2tYyYHA0fw/w\nGtAGyAE+KlH2dKBD9Dc5K4phj2jdxcBrJeKcANwazR8dxdgHaAL8Bng1nX2T4vP/EvhTNN8tiuOI\n6G90Q7TfGwI9gCVA+6hsJ6BzNP8uMCyabwEclOn/hbo8qaYg6XjT3V9w963uvsnd33X36e5e5O6L\ngEeAQWW8/ml3n+Hum4ECwsGoomV/DMx29+ejdfcTEkhKacZ4h7uvdffFhANw8XudDtzv7svcfTVw\nZxnvswj4kJCsAP4HWOPuM6L1L7j7Ig9eBf4FpOxMLuF04Jfu/rW7LyH8+k9+36fcfUX0N/kLIaHn\np7FdgOHAH9x9trt/C4wCBplZx6Qype2bspwJTHL3V6O/0Z1AS0JyLiIkoB5RE+Sn0b6DkNz3M7O2\n7r7e3aen+TkkBkoKko6lyU/MrKuZ/d3MvjCzdcBooF0Zr/8iaX4jZXcul1Z2z+Q43N0Jv6xTSjPG\ntN6L8Au3LH8BhkXzZxGSWXEcPzaz6Wb2lZmtIfxKL2tfFetQVgxmdr6ZvR8106wBuqa5XQifL7E9\nd18HfA3slVSmIn+z0ra7lfA32svd5wM/JfwdvoyaI9tHRS8AugPzzewdMxua5ueQGCgpSDpKno75\nO8Kv4y7u3hK4mdA8EqcVhOYcAMzM2P4gVlJlYlwB7J30vLxTZp8Ejop+aZ9ASBKY2S7A08AdhKad\n1sA/0ozji9JiMLPOwMPApUDbaLsfJ223vNNnlxOapIq314LQTPV5GnFVZLv1CH+zzwHcfYK7DyA0\nHdUn7Bfcfb67n0loIrwXeMbMmlQyFtlJSgqyM1oAa4FvzKwbcEk1vOeLQJ6ZHWdmDYCrgN1iivEp\n4Goz28vM2gLXl1XY3VcCbwLjgfnuviBa1RhoBBQCW8zsx8CRFYjhBjNrbeE6jiuS1jUnHPgLCfnx\nYkJNodhKoGNxx3oKTwAXmVkvM2tMODi/4e6l1rwqEPPxZjY4eu+fE/qBpptZNzM7PHq/TdG0hfAB\nzjGzdlHNYm302bZWMhbZSUoKsjN+CpxH+If/HeGXcqyiA+8ZwH3AamBf4D3CdRVVHePDhLb/Dwid\noE+n8Zq/EDqO/5IU8xrgGuBZQmftqYTklo5bCDWWxcBLwJ+TtjsHGAe8E5XpCiS3w/8TWACsNLPk\nZqDi179MaMZ5Nnr9PoR+hkpx97mEff4wIWENAY6P+hcaA78i9AN9QaiZ3BS9dCgwz8LZbfcAZ7j7\n95WNR3aOhaZZkZrFzOoTmitOdfc3Mh2PSG2hmoLUGGY2xMxaRU0QvyCc0fJOhsMSqVWUFKQmGQgs\nIjRBDAFOdPfSmo9EZCeo+UhERBJUUxARkYQaNyBeu3btPDc3N9NhiIjUKDNnzlzl7mWdxg3UwKSQ\nm5vLjBkzMh2GiEiNYmblXZkPqPlIRESSKCmIiEiCkoKIiCTUuD4FEalemzdvZtmyZXz77beZDkXS\n0KRJEzp27EjDhqUNfVU2JQURKdOyZcto0aIFubm5hMFpJVu5O6tXr2bZsmV06tSp/BekUCeajwoK\nIDcX6tULjwUVuhW9SN327bff0rZtWyWEGsDMaNu2baVqdbW+plBQACNGwMaN4fmSJeE5wPBKjwsp\nUjcoIdQclf1bxVZTMLO9zWxqdNPyuWZ2VYoygy3chH12NN1c1XHceOO2hFBs48awXEREthdn81ER\n8FN37wb0By43s+4pyr3h7n2iaXRVB/HZZxVbLiLZZfXq1fTp04c+ffrQvn179tprr8Tz779P77YL\nF1xwAfPnzy+zzEMPPURBFbUtDxw4kNmzZ1fJtqpbbM1H7r6CcAMP3H29mc0j3D7xo7jeM5V99glN\nRqmWi0jVKygINfHPPgv/Z2PGVK6ptm3btokD7K233krz5s352c9+tl0Zd8fdqVcv9e/c8ePHl/s+\nl19++c4HWYtUS0ezmeUCfdn+7lDFDo5uQP6SmfUo5fUjzGyGmc0oLCys0HuPGQNNm26/rGnTsFxE\nqlZxH96SJeC+rQ8vjpM7Fi5cSM+ePRk5ciR5eXmsWLGCESNGkJ+fT48ePRg9elvDQ/Ev96KiIlq3\nbs2oUaPo3bs3Bx98MF9++SUAN910E2PHjk2UHzVqFP369WP//ffn7bffBuCbb77hlFNOoXfv3gwb\nNoz8/PxyawQTJkzggAMOoGfPntxwww0AFBUVcc455ySWjxs3DoD777+f7t2707t3b84+++wq32dp\nKc6wcU2E+8nOBE5Osa4l0DyaHwosKG97Bx54oFfUhAnuOTnuZuFxwoQKb0Kkzvroo4/SLpuT4x7S\nwfZTTk7VxHLLLbf43Xff7e7uCxYscDPzd955J7F+9erV7u6+efNmHzhwoM+dO9fd3QcMGODvvfee\nb9682QGfPHmyu7tfc801fscdd7i7+4033uj3339/ovx1113n7u7PP/+8/+hHP3J39zvuuMMvu+wy\nd3efPXu216tXz997770d4ix+v6VLl3pOTo4XFhb6999/74cddpi/8MILPm3aNB8yZEii/Ndff+3u\n7u3bt/fvvvtuu2U7I9XfDJjhaRyzY60pRDfvfgYocPe/pUhI69x9QzQ/GWhoZu2qOo7hw2HxYti6\nNTzqrCOReFR3H96+++7LD3/4w8TzJ554gry8PPLy8pg3bx4ffbRja/Uuu+zCMcccA8CBBx7I4sWL\nU2775JNP3qHMm2++yZlnnglA79696dEjZeNGwvTp0zniiCNo164dDRs25KyzzuL111+nS5cuzJ8/\nn6uuuoopU6bQqlUrAHr06MHZZ59NQUHBTl98Vllxnn1kwB+Bee5+Xyll2kflMLN+UTyr44pJROJV\nWl9dXH14zZo1S8wvWLCAX//617z66qvMmTOHIUOGpDxfv1GjRon5+vXrU1RUlHLbjRs33qGMV/Cm\nZKWVb9u2LXPmzGHgwIGMGzeOSy65BIApU6YwcuRI3nnnHfLz89myZUuF3q8qxFlTGACcAxyRdMrp\nUDMbaWYjozKnAh+a2fvAOOBMr+heF5Gskck+vHXr1tGiRQtatmzJihUrmDJlSpW/x8CBA3nqqacA\n+OCDD1LWRJL179+fqVOnsnr1aoqKipg4cSKDBg2isLAQd+e0007jtttuY9asWWzZsoVly5ZxxBFH\ncPfdd1NYWMjGkufTV4M4zz56EyjzKgp3fxB4MK4YRKR6FTfNVuXZR+nKy8uje/fu9OzZk86dOzNg\nwIAqf4+f/OQnnHvuufTq1Yu8vDx69uyZaPpJpWPHjowePZrBgwfj7hx33HEce+yxzJo1i4suugh3\nx8y46667KCoq4qyzzmL9+vVs3bqV66+/nhYtWlT5ZyhPjbtHc35+vusmOyLVZ968eXTr1i3TYWSF\noqIiioqKaNKkCQsWLODoo49mwYIFNGiQXYNDpPqbmdlMd88v77XZ9UlERLLYhg0bOPLIIykqKsLd\n+d3vfpd1CaGyatenERGJUevWrZk5c2amw4hVnRglVURE0qOkICIiCUoKIiKSoKQgIiIJSgoiktUG\nDx68w4VoY8eO5bLLLivzdc2bNwdg+fLlnHrqqaVuu7xT3MeOHbvdRWRDhw5lzZo16YRepltvvZV7\n7rmn0tupakoKIpLVhg0bxsSJE7dbNnHiRIYNG5bW6/fcc0+efvrpnX7/kklh8uTJtG7deqe3l+2U\nFEQkq5166qm8+OKLfPfddwAsXryY5cuXM3DgwMR1A3l5eRxwwAE8//zzO7x+8eLF9OzZE4BNmzZx\n5pln0qtXL8444ww2bdqUKHfppZcmht2+5ZZbABg3bhzLly/n8MMP5/DDDwcgNzeXVatWAXDffffR\ns2dPevbsmRh2e/HixXTr1o3//d//pUePHhx99NHbvU8qs2fPpn///vTq1YuTTjqJr7/+OvH+3bt3\np1evXomB+P79738nbjLUt29f1q9fv9P7NhVdpyAiabv6aqjqG4r16QPR8TSltm3b0q9fP15++WVO\nOOEEJk6cyBlnnIGZ0aRJE5599llatmzJqlWr6N+/P8cff3yp9yl++OGHadq0KXPmzGHOnDnk5eUl\n1o0ZM4Zdd92VLVu2cOSRRzJnzhyuvPJK7rvvPqZOnUq7dtsP4Dxz5kzGjx/P9OnTcXcOOuggBg0a\nRJs2bViwYAFPPPEEv//97zn99NN55plnyrw/wrnnnssDDzzAoEGDuPnmm7ntttsYO3Ysd955J59+\n+imNGzdONFndc889PPTQQwwYMIANGzbQpEmTCuzt8qmmICJZL7kJKbnpyN254YYb6NWrF0cddRSf\nf/45K1euLHU7r7/+euLg3KtXL3r16pVY99RTT5GXl0ffvn2ZO3duuYPdvfnmm5x00kk0a9aM5s2b\nc/LJJ/PGG28A0KlTJ/r06QOUPTw3wNq1a1mzZg2DBg0C4LzzzuP1119PxDh8+HAmTJiQuHJ6wIAB\nXHvttYwbN441a9ZU+RXVqimISNrK+kUfpxNPPJFrr72WWbNmsWnTpsQv/IKCAgoLC5k5cyYNGzYk\nNzc35XDZyVLVIj799FPuuece3n33Xdq0acP5559f7nbKGjeueNhtCENvl9d8VJq///3vvP7660ya\nNInbb7+duXPnMmrUKI499lgmT55M//79eeWVV+jatetObT8V1RREJOs1b96cwYMHc+GFF27Xwbx2\n7Vp23313GjZsyNSpU1mS6obsSQ477DAKonuDfvjhh8yZMwcIw243a9aMVq1asXLlSl566aXEa1q0\naJGy3f6www7jueeeY+PGjXzzzTc8++yzHHrooRX+bK1ataJNmzaJWsbjjz/OoEGD2Lp1K0uXLuXw\nww/nV7/6FWvWrGHDhg188sknHHDAAVx//fXk5+fz8ccfV/g9y6KagojUCMOGDePkk0/e7kyk4cOH\nc9xxx5Gfn0+fPn3K/cV86aWXcsEFF9CrVy/69OlDv379gHAXtb59+9KjR48dht0eMWIExxxzDB06\ndGDq1KmJ5Xl5eZx//vmJbVx88cX07du3zKai0jz22GOMHDmSjRs30rlzZ8aPH8+WLVs4++yzWbt2\nLe7ONddcQ+vWrfnFL37B1KlTqV+/Pt27d0/cRa6qaOhsESmThs6ueSozdLaaj0REJEFJQUREEpQU\nRKRcNa2ZuS6r7N9KSUFEytSkSRNWr16txFADuDurV6+u1AVtOvtIRMrUsWNHli1bRmFhYaZDkTQ0\nadKEjh077vTrlRREpEwNGzakU6dOmQ5Dqomaj0REJEFJQUREEpQUREQkQUlBREQSlBRERCRBSUFE\nRBKUFEREJEFJQUREEpQUREQkQUlBREQSYksKZra3mU01s3lmNtfMrkpRxsxsnJktNLM5ZpYXVzwi\nIlK+OMc+KgJ+6u6zzKwFMNPM/unuHyWVOQbYL5oOAh6OHkVEJANiqym4+wp3nxXNrwfmAXuVKHYC\n8GcPpgGtzaxDXDGJiEjZqqVPwcxygb7A9BKr9gKWJj1fxo6JAzMbYWYzzGyGhu8VEYlP7EnBzJoD\nzwBXu/u6kqtTvGSHO3m4+yPunu/u+bvttlscYYqICDEnBTNrSEgIBe7+txRFlgF7Jz3vCCyPMyYR\nESldnGcfGfBHYJ6731dKsUnAudFZSP2Bte6+Iq6YRESkbHGefTQAOAf4wMxmR8tuAPYBcPffApOB\nocBCYCNwQYzxiIhIOWJLCu7+Jqn7DJLLOHB5XDGIiEjF6IpmERFJUFIQEZEEJQUREUlQUhARkQQl\nBRERSVBSEBGRBCUFERFJUFIQEZEEJQUREUlQUhARkQQlBRERSVBSEBGRBCUFERFJUFIQEZEEJQUR\nEUlQUhARkYQ6kxQ++QSuvho2b850JCIi2avOJIV58+DXv4bx4zMdiYhI9qozSeHYY+GQQ2D0aNi0\nKdPRiIhkpzqTFMzg//4PPv8cfvObTEcjIpKd6kxSABg0CI4+Gu64A9aty3Q0IiLZp04lBYAxY2D1\narj//kxHIiKSfepcUsjPh1NOgXvvhVWrMh2NiEh2qXNJAUJn8zffwF13ZToSEZHsUieTQvfucM45\n8OCDoeNZRESCOpkUAG69FbZsgdtvz3QkIiLZo84mhdxcuOQS+OMfYeHCTEcjIpId6mxSALjxRmjY\nMNQaRESkjieF9u3hqqvgL3+BDz7IdDQiIplXp5MCwM9/Di1bwk03ZToSEZHMq/NJYdddQ2KYNAmm\nTct0NCIimVXnkwKEJqTddw99DCIidVlsScHMHjWzL83sw1LWDzaztWY2O5pujiuW8jRvHhLCq6/C\nK69kKgoRkcyLs6bwJ2BIOWXecPc+0TQ6xljKdcklsPfecMMN4J7JSEREMie2pODurwNfxbX9qta4\ncTg19d134fnnMx2NiEhmZLpP4WAze9/MXjKzHqUVMrMRZjbDzGYUFhbGFsy558L++4czkbZsie1t\nRESyViaTwiwgx917Aw8Az5VW0N0fcfd8d8/fbbfdYguoQYMw7MXcueHaBRGRuiZjScHd17n7hmh+\nMtDQzNplKp5ip5wCffvCLbfA99+HZQUFYViMevXCY0FBJiMUEYlPg0y9sZm1B1a6u5tZP0KCWp2p\neIrVqxduxDN0aBgXqWVLGDECNm4M65csCc8Bhg/PXJwiInEwj+lUGzN7AhgMtANWArcADQHc/bdm\ndgVwKVAEbAKudfe3y9tufn6+z5gxI5aYi7mHW3cuXBialJYu3bFMTg4sXhxrGCIiVcbMZrp7frnl\n4koKcamOpADwxhtw2GGlrzeDrVtjD0NEpEqkmxQyffZR1jr0UDjmmNCclMo++1RvPCIi1SGtpGBm\n+5pZ42h+sJldaWat4w0t8375y1AbaFCi56Vp09DvICJS26RbU3gG2GJmXYA/Ap2AWn/SZl4enHYa\n1K8PHTuGJqOcHHjkEXUyi0jtlG5S2OruRcBJwFh3vwboEF9Y2eP222Hz5pActm4NnctKCCJSW6Wb\nFDab2TDgPODFaFnDeELKLvvvD+efD7/5TeqzkEREapN0k8IFwMHAGHf/1Mw6ARPiCyu73HxzOE11\ndEaH7BMRiV9aScHdP3L3K939CTNrA7Rw9ztjji1r5OTAyJEwfjy89FKmoxERiU+6Zx+9ZmYtzWxX\n4H1gvJndF29o2eWWW6BXLzj+ePjznzMdjYhIPNJtPmrl7uuAk4Hx7n4gcFR8YWWfXXeF114LVzqf\ndx7cdZfuuyAitU+6SaGBmXUATmdbR3Od07IlTJ4Mw4bBqFFw9dW6qllEapd0B8QbDUwB3nL3d82s\nM7AgvrCyV6NGMGECdOgA990HX3wRmpMaN850ZCIilZdWUnD3vwJ/TXq+CDglrqCyXb16cO+9sOee\n8LOfwZdfwnPPQatWmY5MRKRy0u1o7mhmz5rZl2a20syeMbOOcQeX7X7601BreOutMHje8uWZjkhE\npHLS7VMYD0wC9gT2Al6IltV5w4fD3/8OixbBIYfA/PmZjkhEZOelmxR2c/fx7l4UTX8C4rsvZg3z\nP/8D//43bNoEAwbAtGmZjkhEZOekmxRWmdnZZlY/ms4mC+6Slk3y8uDtt6FNGzjiCHixzp6jJSI1\nWbpJ4ULC6ahfACuAUwlDX0iSffcN/Qs9esCJJ8Kjj2Y6IhGRikl3mIvP3P14d9/N3Xd39xMJF7JJ\nCbvvDlOnwlFHwUUXhXsy6CI3EakpKnPntWurLIpapnlzeOEFOOcc+MUv4PLLYcuWTEclIlK+dC9e\nS8WqLIpaqGFDeOyxcC3DXXfBypVQUABNmmQ6MhGR0lWmpqBGkXKYwZ13wtix8Oyz4Syl99/PdFQi\nIqUrMymY2XozW5diWk+4ZkHScNVV8MQT8N570KdPODtp0iQ1KYlI9ikzKbh7C3dvmWJq4e6VaXqq\nc844I9y57e674ZNP4IQTwl3dxo2D9eszHZ2ISFCZ5iOpoDZtwlhJn3wCTz0Fe+wRahEdO8K118Kn\nn2Y6QhGp65QUqkFBAeTmhoH0cnPhySfhtNPCNQ3Tp8OPfwwPPABdusApp8Abb+g0VhHJDCWFmBUU\nwIgRsGRJONAvWRKeFxSE9f36hfnFi8M9Gl57LQyul58Pjz8O33+fyehFpK4xr2E/SfPz833GjBmZ\nDiNtubkhEZSUkxMSQUkbN4YkMXYsfPQRtG8Pl10Gl1wSLowTkZqlqCiMh/bqq+FU9dzcbdMee4QW\nhOpgZjPdPb/cckoK8apXL3VTkFnZd21zh1degfvvh5deCjfxGT4c/t//C81MIpK9li6FKVPg5ZfD\n//HateF/vuSxoHHj8AOxOEkkz+fmhh+FVZU0lBSyREVrCql8/HE4S+lPfwrNSRdeCDfdBPvsU4WB\nishO+/ZbePPNkARefhnmzg3LO3aEIUPCdOSRoaawZEn43081FRZuv91GjbZPFMcfH/ogd4aSQpYo\n7lPYuHHbsqZN4ZFHwi//ivjiC7jjDvjtb8PzSy6BG24IvyZEpPq4w8KF25LA1Klh6PxGjUKfYHEi\n6N491BDS9c038NlnpSeNyy6DW27ZuZiVFLJIQQHceGP4Y++zD4wZU/GEkGzpUrj99jAKa6NG8JOf\nwHXXQdu2VReziGzjHpqA3nhjWyJYtCis69JlWxIYPBiaNYs3jookmWRKCnXAwoUwenS4JWjz5nDN\nNeF6B90rWmqjDRu2r3GXPDgmP0817x5+ia9Zs/20du2Oy1ItLx6BoGnTMCrBkCHwox/VnD6+jCcF\nM3sU+DHwpbv3TLHegF8DQ4GNwPnuPqu87Sop7Oijj+DWW+Gvfw0XyF13Xag9xPmLRaQ6LF8Ozz0H\nzzwT7m4Y59AwzZpB69bbT61abf88Lw8GDgwdxDVNNiSFw4ANwJ9LSQpDgZ8QksJBwK/d/aDytquk\nULr33oObbw53fdt993Cm0siRGplVapZFi+BvfwvTf/4TlnXtCiedFDpuiyUfutKZh9QH/tatoWXL\n0Alcm2U8KURB5AIvlpIUfge85u5PRM/nA4PdfUVZ21RSKN+0aeE+Dq+8AnvtFeYvuCD0P4hkG/dQ\n2y1OBLNnh+V5eXDyyWHq1i2zMdYG6SaFTF7RvBewNOn5smjZDsxshJnNMLMZhSXP2ZId9O8P//xn\nOCMiJyfUFrp2hfHjYbXurC1ZwB1mzAhnz3XrBj17hrNqmjWDe+8NtYWZM8MJGkoI1SuTI52m6kNP\nWW1x90eARyDUFOIMqjYZPDicOz1lSriu4cILw/L994dDDtk2de1afVdVSt21ZQu8/XboH3j22XA2\nXv36cPjhYWDIE0+EDh0yHaVkMiksA/ZOet4RWJ6hWGots21nSbz1VkgSb78d7ucwfnwo07o1HHzw\ntiTRr184m0nqNndYtQoWLAjTokXhDKDvvgsXUSY/plpW8vGbb8LUuDEcfTTcdlu4GGvXXTP9SSVZ\nJpPCJOAKM5tI6GheW15/guw8s3DWxMCB4bl7+Ed/++1t00svhXX16kHv3tvXJnJydv78aMle7vDV\nV9sO/MnTwoXhtMxiZuF0zMaNQ/9U48bbzxc/tmqVel2TJuG7dMwx0KJF5j6zlC3Os4+eAAYD7YCV\nwC1AQwB3/210SuqDwBDCKakXuHu5PcjqaI7P11+HobzfeiskienTwy87CNX6gw8Obb/duoUmpx/8\nIBwk4rZlC3z+eTg9sWvXULORivnqq3CQT3XwX7NmW7l69cIFlvvtt+OUm6uTFWqyrDj7KA5KCtWn\nqAg++GBbTWL69NCEkPyVyckJB+quXbcli65dwymxFalZrFkTtr1oUbjZUPH8okVhrJjNm7eV7d49\n/OIsbvL6wQ/UJ+Iexs1ZuDD19PXX28qawd57pz7wd+pUM8/Bl/IpKUgsvv02/Lr8+OMdp+SrTVu3\n3pYgihNGbi6sXJn64J980IIwZEfnzuEg1blzmPbYY1uS+s9/tr2mTZuQIIqTRFx9Iu7h869fH9rW\nN2zYfr7k85LrvvkmNKG0aBHiS35MZ1njxrBiRekH/uTbutarFxJ2ly6w777hsUuXcODv3FnXrtRF\nSgpSrbZuDU08xQli3rxt8ytS9BQ1ahSSRPEBPzkBdOpU/lAdW7fCf/+7LUG8/XY41x3CAbFXr+1r\nE5067VhzcQ81lMLCMH35ZdnPdg22AAAMp0lEQVSPhYXpX1Fbr962g3nxgb1p0+2Tyvr1YUp3myWH\nXm7QIHyu4gN+8qSmHilJSaEWqeoB9arb2rUwf34Y5bF9+3Dg33PPqm/yKe4TKU4U06aFgy+EWkZ+\nfjgLpvhAv2rV9s1SyVq1gt12C81gxY/t2oXlyQf6VPPNm4df4uk0n7mHmIoTRHINo+SyjRvDfis+\n8O+zT0gMIulQUqglqnLo7bpmy5Ywrn1xn8js2eHiqJIH+1QHf7WrS22jpFBLVMVNekREasIwF5KG\nzz6r2HIRkcpQUshypd1yU7fiFJE4KClkuTFjdrxArGnTsFxEpKopKWS54cNDp3LxMBM5OepkFpH4\n6IS2GmD4cCUBEakeqimIiEiCkoKIiCQoKYiISIKSgoiIJCgpiIhIgpJCHVBQEIbLqFcvPBYUZDoi\nEclWOiW1lis5oN6SJeE56DRXEdmRagq13I03bj/CKoTnN96YmXhEJLspKdRyGlBPRCpCSaGW04B6\nIlIRSgq1nAbUE5GKUFKo5TSgnohUhM4+qgM0oJ6IpEs1BRERSVBSEBGRBCUFERFJUFIQEZEEJQVJ\ni8ZPEqkbdPaRlEvjJ4nUHaopSLk0fpJI3aGkIOXS+EkidYeSgpRL4yeJ1B2xJgUzG2Jm881soZmN\nSrH+fDMrNLPZ0XRxnPHIztH4SSJ1R2xJwczqAw8BxwDdgWFm1j1F0SfdvU80/SGueGTnafwkkboj\nzrOP+gEL3X0RgJlNBE4APorxPSUmGj9JpG6Is/loL2Bp0vNl0bKSTjGzOWb2tJntnWpDZjbCzGaY\n2YzCwsI4YpWY6ToHkZohzqRgKZZ5iecvALnu3gt4BXgs1Ybc/RF3z3f3/N12262Kw5S4FV/nsGQJ\nuG+7zkGJQST7xJkUlgHJv/w7AsuTC7j7anf/Lnr6e+DAGOORDNF1DiI1R5xJ4V1gPzPrZGaNgDOB\nSckFzKxD0tPjgXkxxiMZouscRGqO2Dqa3b3IzK4ApgD1gUfdfa6ZjQZmuPsk4EozOx4oAr4Czo8r\nHsmcffYJTUaplotIdjH3ks382S0/P99nzJiR6TCkAkqOnQThOged1ipSfcxsprvnl1dOVzRL7HSd\ng0jNoVFSpVroOgeRmkE1BakRdJ2DSPVQTUGynu7nIFJ9VFOQrKfrHESqj5KCZD1d5yBSfZQUJOvp\nfg4i1UdJQbJeVdzPQR3VIulRUpCsV9nrHDQgn0j6dEWz1Hq5uamH2cjJgcWLqzsakczQFc0iEXVU\ni6RPSUFqvaroqFafhNQVSgpS61W2o1p9ElKXKClIrVfZjmpdPCd1iZKC1AnDh4dO5a1bw2NFhseo\nij4JNT9JTaGkIFKOyvZJqPlJahIlBZFyVLZPQs1PUpMoKYiUo7J9Emp+kppESUEkDZXpk8iG5icl\nFUmXkoJIzDLd/KSkIhWhpCASs0w3P2VDUpGaQ0lBpBpksvkp00kFKl/TUE2l+igpiGS5yjY/ZTqp\nVLamkQ3NX3UqKbl7jZoOPPBAF6lrJkxwz8lxNwuPEyZU7LVNm7qHQ2qYmjZNfxs5Odu/tnjKyakZ\nr6/s56/s64u3sbN/v6p4vbs7MMPTOMZm/CBf0UlJQaTiMplUzFIf1M2q5/VKSkG6SUH3UxCRchUU\nhD6Ezz4LzU5jxqTfL1LZ+1lU9vX16oVDaUlmoY8n7tdn+vMX0/0URKTKVKajvLJ9IpnuU8l0n0x1\n3w9ESUFEYlXZU3Ir+/q6npQqLJ02pmya1KcgIhWVyY5e9SnETH0KIlLTVKZPpipeD+n3KSgpiIjU\nAepoFhGRCos1KZjZEDObb2YLzWxUivWNzezJaP10M8uNMx4RESlbbEnBzOoDDwHHAN2BYWbWvUSx\ni4Cv3b0LcD9wV1zxiIhI+eKsKfQDFrr7Inf/HpgInFCizAnAY9H808CRZmYxxiQiImWIMynsBSxN\ner4sWpayjLsXAWuBtiU3ZGYjzGyGmc0oLCyMKVwREWkQ47ZT/eIveapTOmVw90eARwDMrNDMUlz0\nnRXaAasyHUQZsj0+yP4YFV/lKL7KqUx8OekUijMpLAP2TnreEVheSpllZtYAaAV8VdZG3X23qgyy\nKpnZjHRO+cqUbI8Psj9GxVc5iq9yqiO+OJuP3gX2M7NOZtYIOBOYVKLMJOC8aP5U4FWvaRdOiIjU\nIrHVFNy9yMyuAKYA9YFH3X2umY0mXG49Cfgj8LiZLSTUEM6MKx4RESlfnM1HuPtkYHKJZTcnzX8L\nnBZnDNXskUwHUI5sjw+yP0bFVzmKr3Jij6/GDXMhIiLx0TAXIiKSoKQgIiIJSgoVZGZ7m9lUM5tn\nZnPN7KoUZQab2Vozmx1NN6faVowxLjazD6L33mFIWQvGRWNOzTGzvGqMbf+k/TLbzNaZ2dUlylT7\n/jOzR83sSzP7MGnZrmb2TzNbED22KeW150VlFpjZeanKxBTf3Wb2cfQ3fNbMWpfy2jK/DzHGd6uZ\nfZ70dxxaymvLHCMtxvieTIptsZnNLuW1se6/0o4pGfv+pXPTBU3bJqADkBfNtwD+C3QvUWYw8GIG\nY1wMtCtj/VDgJcLFg/2B6RmKsz7wBZCT6f0HHAbkAR8mLfsVMCqaHwXcleJ1uwKLosc20Xybaorv\naKBBNH9XqvjS+T7EGN+twM/S+A58AnQGGgHvl/x/iiu+EuvvBW7OxP4r7ZiSqe+fagoV5O4r3H1W\nNL8emMeOw3dkuxOAP3swDWhtZh0yEMeRwCfunvEr1N39dXa8cDJ5bK7HgBNTvPRHwD/d/St3/xr4\nJzCkOuJz9394GB4GYBrhAtGMKGX/pSOdMdIqraz4ovHWTgeeqOr3TUcZx5SMfP+UFCohGuq7LzA9\nxeqDzex9M3vJzHpUa2BhqJB/mNlMMxuRYn0641JVhzMp/R8xk/uv2B7uvgLCPy6we4oy2bIvLyTU\n/lIp7/sQpyui5q1HS2n+yIb9dyiw0t0XlLK+2vZfiWNKRr5/Sgo7ycyaA88AV7v7uhKrZxGaRHoD\nDwDPVXN4A9w9jzBs+eVmdliJ9WmNORWn6Cr344G/plid6f1XEdmwL28EioCCUoqU932Iy8PAvkAf\nYAWhiaakjO8/YBhl1xKqZf+Vc0wp9WUpllVq/ykp7AQza0j44xW4+99Krnf3de6+IZqfDDQ0s3bV\nFZ+7L48evwSeJVTRk6UzLlXcjgFmufvKkisyvf+SrCxuVosev0xRJqP7MupY/DEw3KNG5pLS+D7E\nwt1XuvsWd98K/L6U9830/msAnAw8WVqZ6th/pRxTMvL9U1KooKj98Y/APHe/r5Qy7aNymFk/wn5e\nXU3xNTOzFsXzhM7ID0sUmwScG52F1B9YW1xNrUal/jrL5P4rIXlsrvOA51OUmQIcbWZtouaRo6Nl\nsTOzIcD1wPHuvrGUMul8H+KKL7mf6qRS3jedMdLidBTwsbsvS7WyOvZfGceUzHz/4upRr60TMJBQ\nPZsDzI6mocBIYGRU5gpgLuFMimnAIdUYX+fofd+PYrgxWp4cnxHuivcJ8AGQX837sCnhIN8qaVlG\n9x8hQa0ANhN+fV1EuLfHv4AF0eOuUdl84A9Jr70QWBhNF1RjfAsJ7cnF38PfRmX3BCaX9X2opvge\nj75fcwgHuA4l44ueDyWccfNJdcYXLf9T8fcuqWy17r8yjikZ+f5pmAsREUlQ85GIiCQoKYiISIKS\ngoiIJCgpiIhIgpKCiIgkKCmIRMxsi20/gmuVjdhpZrnJI3SKZKtYb8cpUsNscvc+mQ5CJJNUUxAp\nRzSe/l1m9k40dYmW55jZv6IB3/5lZvtEy/ewcH+D96PpkGhT9c3s99GY+f8ws12i8lea2UfRdiZm\n6GOKAEoKIsl2KdF8dEbSunXu3g94EBgbLXuQMAR5L8JgdOOi5eOAf3sY0C+PcCUswH7AQ+7eA1gD\nnBItHwX0jbYzMq4PJ5IOXdEsEjGzDe7ePMXyxcAR7r4oGrjsC3dva2arCEM3bI6Wr3D3dmZWCHR0\n9++StpFLGPd+v+j59UBDd/+lmb0MbCCMBvucR4MBimSCagoi6fFS5ksrk8p3SfNb2NandyxhLKoD\ngZnRyJ0iGaGkIJKeM5Ie/xPNv00Y1RNgOPBmNP8v4FIAM6tvZi1L26iZ1QP2dvepwHVAa2CH2opI\nddEvEpFtdrHtb97+srsXn5ba2MymE35IDYuWXQk8amY/BwqBC6LlVwGPmNlFhBrBpYQROlOpD0ww\ns1aE0Wvvd/c1VfaJRCpIfQoi5Yj6FPLdfVWmYxGJm5qPREQkQTUFERFJUE1BREQSlBRERCRBSUFE\nRBKUFEREJEFJQUREEv4/BrNN2C47XlAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff75a088a20>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XmcFNW5//HPww6CgIwryKKSqBDA\ncURIUHEJgiIYNCqSRRFREzT6i0mIeqMx6s01xmuMXiOgxsRRJCrGJeKCRNxlUBaFsMiiI4gDsgoI\nA8/vj1PdNEPPTA8zvczM9/169au7q05VP13TU0+dc6pOmbsjIiIC0CDbAYiISO5QUhARkTglBRER\niVNSEBGROCUFERGJU1IQEZE4JQXZg5k1NLNNZtaxJstmk5kdYWY1fv61mZ1mZssS3i8wsxNSKbsX\nnzXBzK7b2+VFUtEo2wFI9ZnZpoS3LYCvgR3R+8vcvbAq63P3HUDLmi5bH7j7N2tiPWY2CviBu/dP\nWPeomli3SEWUFOoAd4/vlKMj0VHu/kp55c2skbuXZiI2kcro95hb1HxUD5jZLWb2uJk9ZmYbgR+Y\nWV8ze8fM1pnZSjO728waR+UbmZmbWefo/SPR/BfMbKOZvW1mXapaNpo/yMwWmtl6M/uzmb1pZheV\nE3cqMV5mZovNbK2Z3Z2wbEMz+18zW2NmHwMDK9g+N5jZxDLT7jWzO6PXo8xsfvR9Po6O4stbV7GZ\n9Y9etzCzv0exfQQcm+Rzl0Tr/cjMhkTTvwXcA5wQNc2tTti2NyUsf3n03deY2dNmdnAq26Yq2zkW\nj5m9YmZfmtnnZvbLhM/5r2ibbDCzIjM7JFlTnZm9Efs7R9tzevQ5XwI3mFlXM5sWfZfV0XZrnbB8\np+g7lkTz/2RmzaKYj0ood7CZbTazduV9X6mEu+tRhx7AMuC0MtNuAbYBZxEOBJoDxwHHE2qLhwEL\ngTFR+UaAA52j948Aq4ECoDHwOPDIXpQ9ANgIDI3m/T9gO3BROd8llRj/CbQGOgNfxr47MAb4COgA\ntAOmh5970s85DNgE7JOw7i+Aguj9WVEZA04BtgA9onmnAcsS1lUM9I9e3wH8G2gLdALmlSl7HnBw\n9De5MIrhwGjeKODfZeJ8BLgpej0girEX0Az4P+DVVLZNFbdza2AV8DOgKbAv0Dua92tgNtA1+g69\ngP2AI8pua+CN2N85+m6lwBVAQ8Lv8RvAqUCT6HfyJnBHwvf5MNqe+0TlvxPNGwfcmvA5PwcmZ/v/\nsDY/sh6AHjX8By0/KbxayXLXAv+IXifb0f8loewQ4MO9KDsSeD1hngErKScppBhjn4T5TwHXRq+n\nE5rRYvPOKLujKrPud4ALo9eDgIUVlH0O+Gn0uqKk8Eni3wL4SWLZJOv9EDgzel1ZUngYuC1h3r6E\nfqQOlW2bKm7nHwJF5ZT7OBZvmempJIUllcRwLjAjen0C8DnQMEm57wBLAYvezwKG1fT/VX16qPmo\n/vg08Y2ZHWlmz0fNARuAm4G8Cpb/POH1ZiruXC6v7CGJcXj4Ly4ubyUpxpjSZwHLK4gX4FFgePT6\nQiDeOW9mg83s3aj5ZB3hKL2ibRVzcEUxmNlFZjY7agJZBxyZ4nohfL/4+tx9A7AWaJ9QJqW/WSXb\n+VBgcTkxHEpIDHuj7O/xIDObZGafRTH8tUwMyzyc1LAbd3+TUOvoZ2bdgY7A83sZk6A+hfqk7OmY\n9xOOTI9w932B3xCO3NNpJeFIFgAzM3bfiZVVnRhXEnYmMZWdMvs4cJqZdSA0bz0axdgceAL4b0LT\nThvgpRTj+Ly8GMzsMOA+QhNKu2i9/0lYb2Wnz64gNEnF1teK0Ez1WQpxlVXRdv4UOLyc5cqb91UU\nU4uEaQeVKVP2+/0P4ay5b0UxXFQmhk5m1rCcOP4G/IBQq5nk7l+XU05SoKRQf7UC1gNfRR11l2Xg\nM58D8s3sLDNrRGin3j9NMU4Crjaz9lGn468qKuzuqwhNHA8BC9x9UTSrKaGduwTYYWaDCW3fqcZw\nnZm1sXAdx5iEeS0JO8YSQn4cRagpxKwCOiR2+JbxGHCJmfUws6aEpPW6u5db86pARdv5GaCjmY0x\nsyZmtq+Z9Y7mTQBuMbPDLehlZvsRkuHnhBMaGprZaBISWAUxfAWsN7NDCU1YMW8Da4DbLHTeNzez\n7yTM/zuhuelCQoKQalBSqL9+DvyY0PF7P+FIOa2iHe/5wJ2Ef/LDgQ8IR4g1HeN9wFRgLjCDcLRf\nmUcJfQSPJsS8DrgGmEzorD2XkNxScSOhxrIMeIGEHZa7zwHuBt6LyhwJvJuw7MvAImCVmSU2A8WW\nn0Jo5pkcLd8RGJFiXGWVu53dfT3wXeAcQsf2QuCkaPYfgKcJ23kDodO3WdQseClwHeGkgyPKfLdk\nbgR6E5LTM8CTCTGUAoOBowi1hk8If4fY/GWEv/M2d3+rit9dyoh1zohkXNQcsAI4191fz3Y8UnuZ\n2d8Indc3ZTuW2k4Xr0lGmdlAQnPAVsIpjaWEo2WRvRL1zwwFvpXtWOoCNR9JpvUDlhCaFQYCZ6tj\nUPaWmf034VqJ29z9k2zHUxeo+UhEROJUUxARkbha16eQl5fnnTt3znYYIiK1ysyZM1e7e0WngAO1\nMCl07tyZoqKibIchIlKrmFllV/UDaj4SEZEESgoiIhKnpCAiInFKCiIiEqekICIicUoKIiI5rrAQ\nOneGBg3Cc2FhZUvsPSUFEanzqrtTzebyhYUwejQsXw7u4Xn06DQmhmzf+q2qj2OPPdZFJLMeecS9\nUyd3s/D8yCO1Z/lHHnFv0cI97FLDo0WL1NeR7eU7ddp92dijU6fUlo+hnNuqln1kfSdf1YeSgkjV\naae69zvVbC9vlnx5s9SWj1FSEKlDsrlTz/ZOMds71Wwvn+magvoURHJcdduUr78eNm/efdrmzWF6\nKj4pZ0Dq8qbn2vIdy7k7d3nTc235W2+FFi12n9aiRZieDkoKImlW3U7KbO/Us71TzPZONdvLjxgB\n48ZBp05gFp7HjQvT0yKV6kQuPdR8JLVJdZtu3LPf/JDtPoGa2Ia1uaO8pqA+BZGaUZ1/6JpoD872\nTj22jvq+U63tlBREakB1d6g1ceZILuzUpfZLNSnUuttxFhQUuO6nIJnSuXPo2C2rUydYtiz9y8cU\nFoY+hE8+CW3pt96axjZlqZPMbKa7F1RWTh3NIhWobidtTZ05MmJESCI7d4ZnJQRJFyUFqfOqc/ZP\ndc98yfiZIyLVpKQgdVp1z/GviSN9HeVLbaKkIHVadc/x15G+1DfqaJY6rUGDUEMoyywcuYvUF+po\nljojm30CIvWNkoLktFzoExCpT5QUJKepT0Aks9SnIDlNfQIiNUN9ClInqE9AJLOUFCSnqU9AJLOU\nFCSnqU9AJLMaZTsAkcqMGKEkIJIpqilI2lX3zmMikjmqKUhaxa4ziJ1WGrvOAHT0L5KL0lpTMLOB\nZrbAzBab2dgk8zuZ2VQzm2Nm/zazDumMRzKvutcZiEhmpS0pmFlD4F5gEHA0MNzMji5T7A7gb+7e\nA7gZ+O90xSPZUd37EYhIZqWzptAbWOzuS9x9GzARGFqmzNHA1Oj1tCTzpZbTdQYitUs6k0J74NOE\n98XRtESzgXOi198DWplZu7IrMrPRZlZkZkUlJSVpCVbSQ9cZiNQu6UwKlmRa2QELrgVOMrMPgJOA\nz4DSPRZyH+fuBe5esP/++9d8pJI2us5ApHZJ59lHxcChCe87ACsSC7j7CmAYgJm1BM5x9/VpjEmy\nQNcZiNQe6awpzAC6mlkXM2sCXAA8k1jAzPLMLBbDr4EH0xiPiIhUIm1Jwd1LgTHAi8B8YJK7f2Rm\nN5vZkKhYf2CBmS0EDgTU0iwikkUaOltEpB7Q0NlSYzRMhUj9oWEupEIapkKkflFNQSqkYSpE6hcl\nBamQhqkQqV+UFKRCGqZCpH5RUpAKaZgKkfpFSUEqpGEqROoXnX0kldIwFSL1h2oKIiISp6QgIiJx\nSgoiIhKnpCAiInFKCiIiEqekUA9oQDsRSZVOSa3jNKCdiFSFagp1nAa0E5GqUFKo4zSgnYhUhZJC\nHacB7USkKpQU6jgNaCciVaGkUMdpQDsRqQqdfVQPaEA7EUmVagoiIhKnpCAiInFKCiIiEqekICIi\ncUoKIiISp6QgIiJxSgq1gEY5FZFM0XUKOU6jnIpIJpm7ZzuGKikoKPCioqJsh5ExnTuHRFBWp06w\nbFmmo5FMKC2Fjz+G+fNh3rzwPH8+fPopHHEE9OoVHj17Qvfuew5jIpKMmc1094LKyqmmkOM0ymnd\ntXkzLFiwa6cfeyxaBNu37yrXvj0cdRT06AGLF8Pf/w7/939hXoMG8M1v7koUsccBB2TnOyVyh5Ur\nYdYsmD07xN6nDwwdmhvxSXKqKeQ41RQyb9u2sM2XLAlH7EuWhEdxMTRsCE2b7no0abL7+4qmu4cd\nfmznv3x5mAZh53744WHnn/g48kjYd9/d49u5M/ztYzvbWbPCI/FA4eCD90wURxwRPicdSkth4cJd\nscQeJSW7yuy3H3z5ZYihXz8YNgy+9z2N2JspqdYUlBRyXNk+BQjNBRrUbu+5h51T4g4/MQEUF4cd\nb0yzZnDYYdChQ1j266/DY9u2Xa+TPZJp1iwc2Zfd+XftGhJHdXz55e5JYtas0PxUWhrmN28OBx4I\n++8PeXnJnxNft26dPIls2gRz5uz+OXPnwtatYX6TJqFZKzEh9egRktvcufDUU+Exd24oX1AQEsQ5\n58A3vlG9bZBOmzeHbWiW7Uj2Tk4kBTMbCPwJaAhMcPffl5nfEXgYaBOVGevu/6ponfUtKUBIDNdf\nH44EO3YMw15nOiGUloYdTqtWYafRpk36jjpr2tdfw9NPw5NPhiP1JUtgw4bdyxx0UDhSP+yw3R+H\nHx7mVXVH4B6agBKTx86d4Qi+YcOa+26V+frrUCuZNQs+/BBWrYLVq8MRfOy57J35Yho2DH/rWJJo\n2TI0dy1evKuGs99+cMwxoX8jlgCOPBIaN648tkWLdiWI994L07p1Cwli2LCwzmzugHfuhJkz4fnn\n4bnnwuuDD4YTToATTwyPbt1qz/9B1pOCmTUEFgLfBYqBGcBwd5+XUGYc8IG732dmRwP/cvfOFa23\nPiaFbPv3v+Gqq3Yd2UHYYbRrt+fRZUXP1T0SrqpFi2D8eHjoobAD7NAh7GgSd/iHHRaa6PbZJ7Ox\n5ZLNm8P2KZssyj6vXx9qNIk1gPbta2bH/emnIXE/9RRMnx52yF267EoQffpkZue7YQO8/HJIAi+8\nEJKoGfTtC6ecEg4opk8PtUmAtm1DU1gsUeTnp5YQsyEXOpp7A4vdfUkU0ERgKDAvoYwDsRbT1sCK\nNMYjVfTpp3DttTBpUqihTJgQduzJdhoffhie16zZdRRZVrt24R9r8GAYNCgki5q2bVvYudx/P7z6\nakheQ4fCZZfBaafVnqO6TGrRIvx9s9m2f+ihcOWV4VFSAs88ExLE3XfDH/8YjtCHDAmJKJbQO3as\nmR3wwoW7agOvvx5qeG3awMCBcOaZ4Tkvb1d599AfNH16KD99Ojz7bJjXokVIICeeGBLF8cdX/eyw\nrVvLT85DhsBxx1X/O1cknTWFc4GB7j4qev9D4Hh3H5NQ5mDgJaAtsA9wmrvPrGi9qimk39at4R/x\nttvCEduvfgW//GVqP+4dO2Dt2uQ/6sWLYcoU+PzzcPTVu3dIEGeeGf7Zq3PEuXjxrlpBSUnoiL/0\nUhg5MuxQpHZavz7ssJ96Kvx2vvpq17wGDUJiSKz1Jb5u2zb5OrdtCzvyWCJYvDhM79Yt/BbPPBO+\n/W1oVIVD5s8/hzfeCOudPj30ubiHpFVQEJJE376hbLL/jcTXid8xUYMG4ayzyy5LPa5EudB89H3g\n9DJJobe7X5lQ5v9FMfzRzPoCDwDd3X1nmXWNBkYDdOzY8djlyU7HkWpzD0do11wDS5eGjr877gjN\nKzVl50744INd/5AzZoTphxwCZ5wRksSpp4b268ps2wb//GeoFUydGmoFQ4aEjvnvfjezbfeSfjt3\nwooVyU8OWLIEvvhi9/Jt2uyeJA44AN58E156KXSWN20aaq6xRFCTv/N168JnxWoSM2bs6vCPadEi\n9abX/fcPSa46Nd1cSAp9gZvc/fTo/a8B3P2/E8p8RKhNfBq9XwL0cfcvkqwSUE0hXRYsgJ/9DF58\nMZwNc/fdobkl3VatCm23zz0X/lk3bgxnr/Tvv+uf9fDDd1/m44931Qq++CIcLcZqBYcckv6YJTdt\n3BgOZpIljKVLQ7NQhw7hNzV4cEgImbrwb/PmcKJG06a7Ou8zfdFhLiSFRoSO5lOBzwgdzRe6+0cJ\nZV4AHnf3v5rZUcBUoL1XEJSSQs3asAF+9zu4667wI/3tb+GnP81OZ9m2baEK/txzoSaxcGGYfuSR\n4R/56KPhscfglVdCLWDw4FCVHjBAtQKp2I4dob9r//1r7yml1ZX1pBAFcQZwF+F00wfd/VYzuxko\ncvdnojOOxgMtCZ3Ov3T3lypap5JCzdi5Ex55JPQXfP55OMq+7bZwHnuuWLQoJIfnn4fXXgtHeoce\nuqtW0L59tiMUqT1yIimkg5JC9c2cGc7yePvt0Nn75z+H51y2cWOoOfTqpVqByN5INSnoBL16pKQk\ndMIed1xob33wwV2JIde1agXHHquEIJJuGhCvHnCHhx8OZxVt2gRXXw033hiGMRARSaSkUMeVlITO\n2MmTw7nS990XOmxFRJJR81Ed9vzz8K1vhec//CFc4auEICIVUVKogzZtCrWDwYPDBTszZoThKtQe\nLyKVqTQpmNkYMyvngnHJNW+/Hc7QGT8efvGLkBB69Mh2VCJSW6RSUzgImGFmk8xsoFl9vfQjt23f\nDv/1X2HExtLSMLLp7bdnfmRSEandKk0K7n4D0JUwLtFFwCIzu83MDq9wQYkrLAzjqjRoEJ4LC2t2\n/fPnh8G2brkFfvSjMBjXiSfW7GeISP2QUp9CNOzE59GjlDCq6RNmdnsaY6sTYndOi916cfny8L4m\nEsPOneHCs/z8cHvGJ58M4wGVvX2jiEiqKr2i2cyuAn4MrAYmAE+7+3YzawAscveM1hhq2xXN6brH\n8mefwcUXhxuCnHEGPPBAuEOYiEgyNXmTnTxgmLvvtmtz951mNnhvA6wvEm+mnsr0VDz+OFx+eRhA\n7i9/CTUP9fSISE1IpfnoX8CXsTdm1srMjgdw9/npCqyuKO9uVntzl6u1a8O9mS+4INz8fdascOqp\nEoKI1JRUksJ9wKaE919F0yQFt96657jpLVqE6anasiU0D/XoEWoJN98chpju2rVmYxURSaX5yBLv\nbxA1G2l4jBSNGBGer78+NBl17BgSQmx6RT75JNx+b/x4+PLLcNP5p55K/z1aRaT+SmXnviTqbI7V\nDn4CLElfSHXPiBGpJQEIZyi99lo4q+jpp8O0s88OQ12fdJKaikQkvVJpProc+Dbh7mnFwPFE90uW\nmrN5c6gR9OwJJ58cLj77xS/CrQSffDLcnlIJQUTSrdKaQnS/5AsyEEu9tGxZaCKaMCF0JPfoEV5f\neCE0b57t6ESkvqk0KZhZM+ASoBvQLDbd3UemMa46zT3UBO6+G555JtQAzj4brroKTjhBNQIRyZ5U\nmo/+Thj/6HTgNaADsDGdQdVVX30F998fagOnnAKvvx7ukbx0KTzxRBiaQglBRLIplY7mI9z9+2Y2\n1N0fNrNHgRfTHVhd88gjobN43bowiumDD4brDdREJCK5JJWawvboeZ2ZdQdaA53TFlEd4x5uffnD\nH4Yb3rz+Orz/fhiiQglBRHJNKjWFcdH9FG4AngFaAv+V1qjqiK1b4ZJL4NFHQxL4y1+gSZNsRyUi\nUr4Kk0I06N0Gd18LTAcOy0hUdcDq1aHz+M034bbbYOxY9ReISO6rsPnI3XcCYzIUS52xYAH06QNF\nRWFYil//WglBRGqHVJqPXjaza4HHCeMeAeDuX5a/SP3173/DsGHQqFF43adPtiMSEUldKkkhdj3C\nTxOmOWpK2sPDD8Oll8IRR8Dzz0OXLtmOSESkalK5olm7tkrs3Am/+U0Y6O7UU8M1B23aZDsqEZGq\nS+WK5h8lm+7uf6v5cGqfrVvhootC38GoUWHIisaNsx2ViMjeSaX5KHGg5mbAqcD7QL1PCiUlMHQo\nvP023H47XHutOpRFpHZLpfnoysT3ZtaaMPRFvTZ/Ppx5JqxcGZqLzjkn2xGJiFTf3twsZzNQr+/5\nNXVqSALNmoV7H/Tune2IRERqRip9Cs8SzjaCcF3D0cCkdAaVyx54AC6/PNwj+fnnoVOnbEckIlJz\nUqkp3JHwuhRY7u7FaYonZ+3cGW6p+fvfw4ABMGkStG6d7ahERGpWKknhE2Clu28FMLPmZtbZ3Zel\nNbIcc889ISFcdll43Uh3qRaROiiVUVL/AexMeL8jmlYpMxtoZgvMbLGZjU0y/3/NbFb0WGhm61IL\nO7Pcw6mmffvCffcpIYhI3ZXK7q2Ru2+LvXH3bWZW6VifZtYQuBf4LuHezjPM7Bl3n5ewrmsSyl8J\nHFOV4DPljTfCeEYPPqhTTkWkbkulplBiZkNib8xsKLA6heV6A4vdfUmUVCYCQysoPxx4LIX1Ztz4\n8bDvvnDeedmOREQkvVKpKVwOFJrZPdH7YiDpVc5ltAc+TXhfDByfrKCZdQK6AK+WM380MBqgY8eO\nKXx0zVm7Fv7xj3DV8j77ZPSjRUQyLpWL1z4G+phZS8DcPdX7MydraPEk0wAuAJ5w9x3lxDAOGAdQ\nUFBQ3jrS4tFHw1AWl16ayU8VEcmOSpuPzOw2M2vj7pvcfaOZtTWzW1JYdzFwaML7DsCKcspeQA42\nHbmHpqNOncJw2A0aQOfOUFiY7chERNIjlT6FQe4ePysougvbGSksNwPoamZdoo7pCwi389yNmX0T\naAu8nVrImVNUBLNnw4oVsHx5SBLLl8Po0UoMIlI3pZIUGppZ09gbM2sONK2gPADuXkq4a9uLwHxg\nkrt/ZGY3J3ZcEzqYJ7p7RpuFUjF+fDjbaPv23adv3hwuZBMRqWussn2xmf0SGAI8FE26GHjG3W9P\nc2xJFRQUeFFRUdo/Z9MmOPjg8JyMWbjKWUSkNjCzme5eUFm5VDqabzezOcBphM7jKUCdH/Hn8cdD\nQjjwQFi1as/5GT4JSkQkI1JpPgL4nHBV8zmE+ynMT1tEOWL8eDj6aLjjDmjRYvd5LVqEu6yJiNQ1\n5dYUzOwbhM7h4cAa4HFCc9PJGYota+bOhXffhTvvhB/8IDQVXX89fPJJqCHceiuMGJHtKEVEal5F\nzUf/AV4HznL3xQBmdk0F5euM8eOhSRP44Q/D+xEjlAREpH6oqPnoHEKz0TQzG29mp5L8grQ6ZcsW\n+Pvfw3UJeXnZjkZEJLPKTQruPtndzweOBP4NXAMcaGb3mdmADMWXcU89BevW6QpmEamfKu1odvev\n3L3Q3QcTrkqeBewxDHZdMX48HH449O+f7UhERDIv1bOPAHD3L939fnc/JV0BZdPCheGey5dcEoa0\nEBGpb7TrSzBhAjRsGEZEFRGpj5QUItu2wV//CmedFa5kFhGpj5QUIs8+CyUl6mAWkfpNSSEyfjwc\neiicfnq2IxERyR4lBWDZMnjpJRg5MvQpiIjUV0oKwIMPhueLL85uHCIi2Vbvk0JpaUgKp58e7rAm\nIlKf1fukMGUKfPaZOphFREBJgQkTwj0Tzjor25GIiGRfvU4KK1fCc8+Fi9UaN852NCIi2Vevk8JD\nD8GOHWFYCxERqcdJYefO0HTUvz907ZrtaEREckO9TQqvvgpLl6qDWUQkUb1NChMmwH77hZvpiIhI\nUC+TwurVMHlyuN1ms2bZjkZEJHfUy6Twt7+FUVFHjcp2JCIiuaXeJQX3MPhdnz7QvXu2oxERyS2N\nsh1Apr35JvznP/DAA9mOREQk99S7msKECdCqFZx/frYjERHJPfUqKaxbB5MmwYUXwj77ZDsaEZHc\nU6+SwqOPwpYtujZBRKQ89SYpxDqYe/WC/PxsRyMikpvqTVKYORNmzQq1BLNsRyMikpvqTVKYMgWa\nN4cRI7IdiYhI7qo3SeGGG2DBAmjdOtuRiIjkrnqTFAAOPTTbEYiI5La0JgUzG2hmC8xssZmNLafM\neWY2z8w+MrNH0xmPiIhULG1XNJtZQ+Be4LtAMTDDzJ5x93kJZboCvwa+4+5rzeyAdMUjIiKVS2dN\noTew2N2XuPs2YCIwtEyZS4F73X0tgLt/kcZ4RESkEulMCu2BTxPeF0fTEn0D+IaZvWlm75jZwGQr\nMrPRZlZkZkUlJSVpCldERNKZFJJdDeBl3jcCugL9geHABDNrs8dC7uPcvcDdC/bff/8aD1RERIJ0\nJoViIPF8nw7AiiRl/unu2919KbCAkCRERCQL0pkUZgBdzayLmTUBLgCeKVPmaeBkADPLIzQnLUlj\nTCIiUoG0JQV3LwXGAC8C84FJ7v6Rmd1sZkOiYi8Ca8xsHjAN+IW7r0lXTCIiUjFzL9vMn9sKCgq8\nqKgo22GIiNQqZjbT3QsqK1evrmgWEZGKKSmIiEickoKIiMQpKYiISJySgoiIxCkpiIhInJKCiIjE\nKSmIiEickoKIiMQpKYiISFza7rwmInXL9u3bKS4uZuvWrdkORSrQrFkzOnToQOPGjfdqeSUFEUlJ\ncXExrVq1onPnzpglu12KZJu7s2bNGoqLi+nSpcterUPNRyKSkq1bt9KuXTslhBxmZrRr165atTkl\nBRFJmRJC7qvu30hJQURE4pQURCQtCguhc2do0CA8FxZWb31r1qyhV69e9OrVi4MOOoj27dvH32/b\nti2ldVx88cUsWLCgwjL33nsvhdUNthZTR7OI1LjCQhg9GjZvDu+XLw/vAUaM2Lt1tmvXjlmzZgFw\n00030bJlS6699trdyrg77k6DBsmPdx966KFKP+enP/3p3gVYR6imICI17vrrdyWEmM2bw/Satnjx\nYrp3787ll19Ofn4+K1euZPTo0RQUFNCtWzduvvnmeNl+/foxa9YsSktLadOmDWPHjqVnz5707duX\nL774AoAbbriBu+66K15+7NjcTtjKAAAQC0lEQVSx9O7dm29+85u89dZbAHz11Vecc8459OzZk+HD\nh1NQUBBPWIluvPFGjjvuuHh8sTtdLly4kFNOOYWePXuSn5/PsmXLALjtttv41re+Rc+ePbk+HRsr\nBUoKIlLjPvmkatOra968eVxyySV88MEHtG/fnt///vcUFRUxe/ZsXn75ZebNm7fHMuvXr+ekk05i\n9uzZ9O3blwcffDDput2d9957jz/84Q/xBPPnP/+Zgw46iNmzZzN27Fg++OCDpMv+7Gc/Y8aMGcyd\nO5f169czZcoUAIYPH84111zD7NmzeeuttzjggAN49tlneeGFF3jvvfeYPXs2P//5z2to61SNkoKI\n1LiOHas2vboOP/xwjjvuuPj7xx57jPz8fPLz85k/f37SpNC8eXMGDRoEwLHHHhs/Wi9r2LBhe5R5\n4403uOCCCwDo2bMn3bp1S7rs1KlT6d27Nz179uS1117jo48+Yu3ataxevZqzzjoLCBebtWjRglde\neYWRI0fSvHlzAPbbb7+qb4gaoKQgIjXu1luhRYvdp7VoEaanwz777BN/vWjRIv70pz/x6quvMmfO\nHAYOHJj0vP0mTZrEXzds2JDS0tKk627atOkeZWLNQBXZvHkzY8aMYfLkycyZM4eRI0fG40h22qi7\n58Qpv0oKIlLjRoyAceOgUycwC8/jxu19J3NVbNiwgVatWrHvvvuycuVKXnzxxRr/jH79+jFp0iQA\n5s6dm7QmsmXLFho0aEBeXh4bN27kySefBKBt27bk5eXx7LPPAuGiwM2bNzNgwAAeeOABtmzZAsCX\nX35Z43GnQmcfiUhajBiRmSRQVn5+PkcffTTdu3fnsMMO4zvf+U6Nf8aVV17Jj370I3r06EF+fj7d\nu3endevWu5Vp164dP/7xj+nevTudOnXi+OOPj88rLCzksssu4/rrr6dJkyY8+eSTDB48mNmzZ1NQ\nUEDjxo0566yz+N3vflfjsVfGUqkG5ZKCggIvKirKdhgi9c78+fM56qijsh1GTigtLaW0tJRmzZqx\naNEiBgwYwKJFi2jUKDeOs5P9rcxsprsXVLZsbnwDEZFaZNOmTZx66qmUlpbi7tx///05kxCqq258\nCxGRDGrTpg0zZ87MdhhpoY5mERGJU1IQEZE4JQUREYlTUhARkTglBRGpFfr377/HhWh33XUXP/nJ\nTypcrmXLlgCsWLGCc889t9x1V3aq+1133cXmhFH+zjjjDNatW5dK6LWKkoKI1ArDhw9n4sSJu02b\nOHEiw4cPT2n5Qw45hCeeeGKvP79sUvjXv/5FmzZt9np9uUqnpIpIlV19NSQZKbpaevWCaMTqpM49\n91xuuOEGvv76a5o2bcqyZctYsWIF/fr1Y9OmTQwdOpS1a9eyfft2brnlFoYOHbrb8suWLWPw4MF8\n+OGHbNmyhYsvvph58+Zx1FFHxYeWALjiiiuYMWMGW7Zs4dxzz+W3v/0td999NytWrODkk08mLy+P\nadOm0blzZ4qKisjLy+POO++Mj7I6atQorr76apYtW8agQYPo168fb731Fu3bt+ef//xnfMC7mGef\nfZZbbrmFbdu20a5dOwoLCznwwAPZtGkTV155JUVFRZgZN954I+eccw5TpkzhuuuuY8eOHeTl5TF1\n6tSa+yOgpCAitUS7du3o3bs3U6ZMYejQoUycOJHzzz8fM6NZs2ZMnjyZfffdl9WrV9OnTx+GDBlS\n7gBz9913Hy1atGDOnDnMmTOH/Pz8+Lxbb72V/fbbjx07dnDqqacyZ84crrrqKu68806mTZtGXl7e\nbuuaOXMmDz30EO+++y7uzvHHH89JJ51E27ZtWbRoEY899hjjx4/nvPPO48knn+QHP/jBbsv369eP\nd955BzNjwoQJ3H777fzxj3/kd7/7Ha1bt2bu3LkArF27lpKSEi699FKmT59Oly5d0jI+UlqTgpkN\nBP4ENAQmuPvvy8y/CPgD8Fk06R53n5DOmESk+io6ok+nWBNSLCnEjs7dneuuu47p06fToEEDPvvs\nM1atWsVBBx2UdD3Tp0/nqquuAqBHjx706NEjPm/SpEmMGzeO0tJSVq5cybx583abX9Ybb7zB9773\nvfhIrcOGDeP1119nyJAhdOnShV69egHlD89dXFzM+eefz8qVK9m2bRtdunQB4JVXXtmtuaxt27Y8\n++yznHjiifEy6RheO219CmbWELgXGAQcDQw3s6OTFH3c3XtFj7QkhJq+V6yIZMfZZ5/N1KlTef/9\n99myZUv8CL+wsJCSkhJmzpzJrFmzOPDAA5MOl50oWS1i6dKl3HHHHUydOpU5c+Zw5plnVrqeisaP\niw27DeUPz33llVcyZswY5s6dy/333x//vGRDaWdieO10djT3Bha7+xJ33wZMBIZWskyNi90rdvly\ncN91r1glBpHap2XLlvTv35+RI0fu1sG8fv16DjjgABo3bsy0adNYvnx5hes58cQTKYx2Ah9++CFz\n5swBwrDb++yzD61bt2bVqlW88MIL8WVatWrFxo0bk67r6aefZvPmzXz11VdMnjyZE044IeXvtH79\netq3bw/Aww8/HJ8+YMAA7rnnnvj7tWvX0rdvX1577TWWLl0KpGd47XQmhfbApwnvi6NpZZ1jZnPM\n7AkzOzTZisxstJkVmVlRSUlJlYLI5L1iRST9hg8fzuzZs+N3PgMYMWIERUVFFBQUUFhYyJFHHlnh\nOq644go2bdpEjx49uP322+nduzcQ7qJ2zDHH0K1bN0aOHLnbsNujR49m0KBBnHzyybutKz8/n4su\nuojevXtz/PHHM2rUKI455piUv89NN93E97//fU444YTd+ituuOEG1q5dS/fu3enZsyfTpk1j//33\nZ9y4cQwbNoyePXty/vnnp/w5qUrb0Nlm9n3gdHcfFb3/IdDb3a9MKNMO2OTuX5vZ5cB57n5KReut\n6tDZDRqEGsKe8cHOnSmvRqTe09DZtUd1hs5OZ02hGEg88u8ArEgs4O5r3P3r6O144NiaDiLT94oV\nEanN0pkUZgBdzayLmTUBLgCeSSxgZgcnvB0CzK/pIDJ9r1gRkdosbUnB3UuBMcCLhJ39JHf/yMxu\nNrMhUbGrzOwjM5sNXAVcVNNxZPNesSJ1TW27U2N9VN2/kW7HKSIpWbp0Ka1ataJdu3ZpPy1S9o67\ns2bNGjZu3Bi/liFGt+MUkRrVoUMHiouLqeoZgJJZzZo1o0OHDnu9vJKCiKSkcePGexx9St2jUVJF\nRCROSUFEROKUFEREJK7WnX1kZiVAxQObZE8esDrbQVRA8VVPrscHuR+j4que6sTXyd33r6xQrUsK\nuczMilI55StbFF/15Hp8kPsxKr7qyUR8aj4SEZE4JQUREYlTUqhZ47IdQCUUX/XkenyQ+zEqvupJ\ne3zqUxARkTjVFEREJE5JQURE4pQUqsjMDjWzaWY2Pxr2+2dJyvQ3s/VmNit6/CbDMS4zs7nRZ+8x\npKwFd5vZ4uhWqPkZjO2bCdtllpltMLOry5TJ+PYzswfN7Asz+zBh2n5m9rKZLYqe25az7I+jMovM\n7McZiu0PZvaf6O832czalLNshb+FNMd4k5l9lvB3PKOcZQea2YLo9zg2g/E9nhDbMjObVc6yad2G\n5e1Tsvb7c3c9qvAADgbyo9etgIXA0WXK9Aeey2KMy4C8CuafAbwAGNAHeDdLcTYEPidcVJPV7Qec\nCOQDHyZMux0YG70eC/xPkuX2A5ZEz22j120zENsAoFH0+n+SxZbKbyHNMd4EXJvCb+Bj4DCgCTC7\n7P9TuuIrM/+PwG+ysQ3L26dk6/enmkIVuftKd38/er2RcAOh9tmNqsqGAn/z4B2gTZm74GXKqcDH\n7p71K9TdfTrwZZnJQ4GHo9cPA2cnWfR04GV3/9Ld1wIvAwPTHZu7v+ThRlYA7xBud5s15Wy/VPQG\nFrv7EnffBkwkbPcaVVF8Fm4OcR7wWE1/bioq2Kdk5fenpFANZtYZOAZ4N8nsvmY228xeMLNuGQ0M\nHHjJzGaa2egk89sDnya8LyY7ie0Cyv9HzOb2iznQ3VdC+McFDkhSJhe25UhCzS+Zyn4L6TYmauJ6\nsJzmj1zYficAq9x9UTnzM7YNy+xTsvL7U1LYS2bWEngSuNrdN5SZ/T6hSaQn8Gfg6QyH9x13zwcG\nAT81sxPLzE9226yMnpts4b7dQ4B/JJmd7e1XFVndlmZ2PVAKFJZTpLLfQjrdBxwO9AJWEppoysr6\nbxEYTsW1hIxsw0r2KeUulmRatbafksJeMLPGhD9eobs/VXa+u29w903R638Bjc0sL1PxufuK6PkL\nYDKhip6oGDg04X0HYEVmoosbBLzv7qvKzsj29kuwKtasFj1/kaRM1rZl1Kk4GBjhUQNzWSn8FtLG\n3Ve5+w533wmML+ezs/pbNLNGwDDg8fLKZGIblrNPycrvT0mhiqL2xweA+e5+ZzllDorKYWa9Cdt5\nTYbi28fMWsVeEzokPyxT7BngR9FZSH2A9bFqagaVe3SWze1XxjNA7GyOHwP/TFLmRWCAmbWNmkcG\nRNPSyswGAr8Chrj75nLKpPJbSGeMif1U3yvns2cAXc2sS1R7vICw3TPlNOA/7l6cbGYmtmEF+5Ts\n/P7S1aNeVx9AP0L1bA4wK3qcAVwOXB6VGQN8RDiT4h3g2xmM77Doc2dHMVwfTU+Mz4B7CWd9zAUK\nMrwNWxB28q0TpmV1+xES1EpgO+Ho6xKgHTAVWBQ97xeVLQAmJCw7ElgcPS7OUGyLCW3Jsd/gX6Ky\nhwD/qui3kMHt9/fo9zWHsIM7uGyM0fszCGfcfJyuGJPFF03/a+x3l1A2o9uwgn1KVn5/GuZCRETi\n1HwkIiJxSgoiIhKnpCAiInFKCiIiEqekICIicUoKIhEz22G7j+BaYyN2mlnnxBE6RXJVo2wHIJJD\ntrh7r2wHIZJNqimIVCIaT/9/zOy96HFENL2TmU2NBnybamYdo+kHWrjHwezo8e1oVQ3NbHw0Zv5L\nZtY8Kn+Vmc2L1jMxS19TBFBSEEnUvEzz0fkJ8za4e2/gHuCuaNo9hCHIexAGpLs7mn438JqHAf3y\nCVfCAnQF7nX3bsA64Jxo+ljgmGg9l6fry4mkQlc0i0TMbJO7t0wyfRlwirsviQYu+9zd25nZasLQ\nDduj6SvdPc/MSoAO7v51wjo6E8a97xq9/xXQ2N1vMbMpwCbCaLBPezQYoEg2qKYgkhov53V5ZZL5\nOuH1Dnb16Z1JGIvqWGBmNHKnSFYoKYik5vyE57ej128RRvUEGAG8Eb2eClwBYGYNzWzf8lZqZg2A\nQ919GvBLoA2wR21FJFN0RCKyS3Pb/ebtU9w9dlpqUzN7l3AgNTyadhXwoJn9AigBLo6m/wwYZ2aX\nEGoEVxBG6EymIfCImbUmjF77v+6+rsa+kUgVqU9BpBJRn0KBu6/Odiwi6abmIxERiVNNQURE4lRT\nEBGROCUFERGJU1IQEZE4JQUREYlTUhARkbj/D4Llhx5IphyrAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff759be8438>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 绘制训练损失和验证损失\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline # jupyter notebook显示图像 \n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "epochs = range(1, len(loss) + 1)\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Loss')\n",
    "plt.legend()\n",
    "plt.show()\n",
    "\n",
    "plt.clf() # 清空图形\n",
    "acc = history.history['acc']\n",
    "val_acc = history.history['val_acc']\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 分析模型：网络在第9轮的时候开始过拟合。\n",
    "\n",
    "### 调整为训练9轮"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 7982 samples, validate on 1000 samples\n",
      "Epoch 1/9\n",
      "7982/7982 [==============================] - 1s 181us/step - loss: 2.6117 - acc: 0.4644 - val_loss: 1.7295 - val_acc: 0.6350\n",
      "Epoch 2/9\n",
      "7982/7982 [==============================] - 1s 136us/step - loss: 1.4070 - acc: 0.7095 - val_loss: 1.2745 - val_acc: 0.7310\n",
      "Epoch 3/9\n",
      "7982/7982 [==============================] - 1s 135us/step - loss: 1.0388 - acc: 0.7813 - val_loss: 1.1017 - val_acc: 0.7640\n",
      "Epoch 4/9\n",
      "7982/7982 [==============================] - 1s 130us/step - loss: 0.8120 - acc: 0.8319 - val_loss: 1.0280 - val_acc: 0.7830\n",
      "Epoch 5/9\n",
      "7982/7982 [==============================] - 1s 129us/step - loss: 0.6463 - acc: 0.8634 - val_loss: 0.9524 - val_acc: 0.8060\n",
      "Epoch 6/9\n",
      "7982/7982 [==============================] - 1s 136us/step - loss: 0.5268 - acc: 0.8916 - val_loss: 0.9208 - val_acc: 0.8100\n",
      "Epoch 7/9\n",
      "7982/7982 [==============================] - 1s 134us/step - loss: 0.4197 - acc: 0.9113 - val_loss: 0.9013 - val_acc: 0.8150\n",
      "Epoch 8/9\n",
      "7982/7982 [==============================] - 1s 132us/step - loss: 0.3482 - acc: 0.9257 - val_loss: 0.8928 - val_acc: 0.8160\n",
      "Epoch 9/9\n",
      "7982/7982 [==============================] - 1s 130us/step - loss: 0.2863 - acc: 0.9349 - val_loss: 0.8857 - val_acc: 0.8190\n",
      "2246/2246 [==============================] - 0s 178us/step\n"
     ]
    }
   ],
   "source": [
    "model = models.Sequential()\n",
    "model.add(layers.Dense(64, activation='relu', input_shape=(10000,)))\n",
    "model.add(layers.Dense(64, activation='relu'))\n",
    "model.add(layers.Dense(46, activation='softmax'))\n",
    "model.compile(optimizer='rmsprop',\n",
    "    loss='categorical_crossentropy',\n",
    "    metrics=['accuracy'])\n",
    "model.fit(partial_x_train,\n",
    "    partial_y_train,\n",
    "    epochs=9,\n",
    "    batch_size=512,\n",
    "    validation_data=(x_val, y_val))\n",
    "results = model.evaluate(x_test, one_hot_test_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.98543361564459597, 0.78628673202101929]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5 在新的数据上生成预测结果\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "predictions = model.predict(x_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(46,)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predictions[0].shape # 每个元素长度为46"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.99999982"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sum(predictions[1]) # 元素总和接近1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.argmax(predictions[23]) #argmax返回的是最大数的索引.即预测类别"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}