Chapter3.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Chapter3.ipynb",
      "provenance": [],
      "toc_visible": true,
      "authorship_tag": "ABX9TyM0P1lz+ZLJRkXideJuyxNF",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/hank199599/deep_learning_keras_log/blob/main/Chapter3.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ruo9MV9r4NGL"
      },
      "source": [
        "# 3-1 核心元件\r\n",
        "![3-1](https://github.com/hank199599/deep_learning_keras_log/blob/main/pictures/3-1.png?raw=true)  \r\n",
        "* 層(layers)：組成一個神經網路模型\r\n",
        "* 輸入資料(input data) 與 目標(target)：用來訓練及檢測一個神經網路\r\n",
        "* 損失函數(loss functions)：取得學習的回饋信號\r\n",
        "* 優化器(optimizer)：決定學習進行的方式"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N9Gz7uWYmAGd"
      },
      "source": [
        "## 層(layers)\r\n",
        "|向量資料|儲存的張量|使用的層|\r\n",
        "|---|---|---|\r\n",
        "|1D|2D|densely-connected layer 密集連接層\r\n",
        "|2D|3D|recurrent layer 循環層\r\n",
        "|3D|4D|Conv2D 2D卷積層  \r\n",
        "  \r\n",
        "須包含一層batch 維度，因此在儲存時的維度較訓練資料多一維"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w1YCOrvPrlT9"
      },
      "source": [
        "## 模型\r\n",
        "神經網路拓樸定義了一個**假設空間(hypothesis space，只在該神經網路拓樸下權重參數所有可能的組態)**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f5olH7E8vq69"
      },
      "source": [
        "## 損失函數 (loss function)\r\n",
        "* 二元分類問題：二元交叉熵(binary crossentropy)\r\n",
        "* 多類別分類問題：分類交叉熵(categorical crossentropy)\r\n",
        "* 迴歸問題：均方差(meansquared error)\r\n",
        "* 序列學習問題：連結時序問題(connectionist temporal classfication)\r\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G9q3tyDG4VhI"
      },
      "source": [
        "# 3-4 二元分類：電影評論為正評或負評\r\n",
        "[IMDb,Internet Movie Database](https://www.imdb.com)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EKvah1THxKCl"
      },
      "source": [
        "載入IMBD資料集"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LqBXkSRpxPTE",
        "outputId": "4f183c3a-b3b4-476a-c457-d2741401b3be"
      },
      "source": [
        "from keras.datasets import imdb\r\n",
        "\r\n",
        "(train_data,train_labels),(test_data,test_labels) = imdb.load_data(num_words=10000)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<string>:6: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray\n",
            "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/datasets/imdb.py:159: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray\n",
            "  x_train, y_train = np.array(xs[:idx]), np.array(labels[:idx])\n",
            "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/datasets/imdb.py:160: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray\n",
            "  x_test, y_test = np.array(xs[idx:]), np.array(labels[idx:])\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "98KrY9xEx7P8"
      },
      "source": [
        "### num_words\r\n",
        "表示讀取資料時，只允許「單字對應數字的字典」中編號0~9999的單字載入"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ErqTkxRjyJqL",
        "outputId": "cb651c47-4e57-47b1-efd5-ee88ff268352"
      },
      "source": [
        "max([max(sequence) for sequence in train_data])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "9999"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "li9eb6nD0izW"
      },
      "source": [
        "將數字還原成文字"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5bvfUJdc0mxd"
      },
      "source": [
        "word_index = imdb.get_word_index() #取得單字對應數字的字典\r\n",
        "reverse_word_index = dict(\r\n",
        "    [(value,key) for (key,value) in word_index.items()] #反轉為數字對應單字的字典\r\n",
        ")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 153
        },
        "id": "iBjZYAKf2Lon",
        "outputId": "2c1bdc78-2099-4382-e7f5-ea9ab44d3729"
      },
      "source": [
        "decord_review = ' '.join([reverse_word_index.get(i-3,'?') for i in train_data[0]])\r\n",
        "decord_review"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "\"? this film was just brilliant casting location scenery story direction everyone's really suited the part they played and you could just imagine being there robert ? is an amazing actor and now the same being director ? father came from the same scottish island as myself so i loved the fact there was a real connection with this film the witty remarks throughout the film were great it was just brilliant so much that i bought the film as soon as it was released for ? and would recommend it to everyone to watch and the fly fishing was amazing really cried at the end it was so sad and you know what they say if you cry at a film it must have been good and this definitely was also ? to the two little boy's that played the ? of norman and paul they were just brilliant children are often left out of the ? list i think because the stars that play them all grown up are such a big profile for the whole film but these children are amazing and should be praised for what they have done don't you think the whole story was so lovely because it was true and was someone's life after all that was shared with us all\""
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uEdtOxAY2XUe"
      },
      "source": [
        "## 準備資料\r\n",
        "方法一：\r\n",
        "1. 填補資料中每個子list的內容，使它們具有相同的長度\r\n",
        "2. 在將整筆資料轉換成shape (樣本數、填補後的樣本長度)\r\n",
        "3. 輸入到崁入層(Embedding layer)\r\n",
        "  \r\n",
        "方法二： \r\n",
        "1. 對資料中每個子list做[One-shot編碼](https://colab.research.google.com/github/hank199599/data_science_from_scratch_reading_log/blob/main/Chapter19.ipynb#scrollTo=QzUQbu-DtfQa)，將其轉換成由0與1組成的向量\r\n",
        "\r\n",
        "```python\r\n",
        "def one_hot_encode(i:int,num_labels:int=10)->List[float]:\r\n",
        "  return [1.0 if j==i else 0.0 for j in range(num_labels)]\r\n",
        "```\r\n",
        "2. 輸入到可處理浮點數的密集層(Dense layer)\r\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ufOW1gsK48Cg"
      },
      "source": [
        "import numpy as np\r\n",
        "\r\n",
        "def vectorize_sequences(sequences,dimension=10000):\r\n",
        "  results = np.zeros((len(sequences),dimension))\r\n",
        "  for i,sequences in enumerate(sequences):\r\n",
        "    results[i,sequences] = 1.\r\n",
        "  return results"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WYSWI3_E7L9H"
      },
      "source": [
        "### 將訓練資料轉換為維度為10000的向量"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tGgO-vbb5jyv"
      },
      "source": [
        "x_train = vectorize_sequences(train_data) # 將訓練資料向量化"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ESnI3SLH5yhO"
      },
      "source": [
        "x_test = vectorize_sequences(test_data)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-6X_steF55zg"
      },
      "source": [
        ""
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CqokQILv55rP",
        "outputId": "c050b7e2-d001-49c1-9cfa-32c75ceaf4c1"
      },
      "source": [
        "x_train[0]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([0., 1., 1., ..., 0., 0., 0.])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QJ0DmbzT7YTW"
      },
      "source": [
        "### 將標籤資料向量化"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Hmyo2C9s7cwe"
      },
      "source": [
        "y_train = np.array(train_labels).astype('float32')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hsLJM5E77mWL"
      },
      "source": [
        "y_test = np.array(test_labels).astype('float32')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kYzzxHZt7tMB"
      },
      "source": [
        "## 建立神經網路"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NLipmDmd77Qw"
      },
      "source": [
        "### 建立密集層Dense堆疊結構，需確認的關鍵：\r\n",
        "* 需要使用多少層?\r\n",
        "* 每一層要有多少個神經元?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "19hK5I7O8V9O"
      },
      "source": [
        "from keras import models\r\n",
        "from keras import layers\r\n",
        "\r\n",
        "model = models.Sequential()\r\n",
        "model.add(layers.Dense(16,activation='relu',input_shape=(10000,)))\r\n",
        "model.add(layers.Dense(16,activation='relu'))\r\n",
        "model.add(layers.Dense(1,activation='sigmoid'))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KbyFY3zmBFZe"
      },
      "source": [
        "編譯模型"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pcz1lJsuA3dS"
      },
      "source": [
        "model.compile(optimizer='rmsprop',\r\n",
        "       loss='binary_crossentropy', # 指定損失函數，並進行compile\r\n",
        "       metrics=['accuracy'])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WMHebFurBMGX"
      },
      "source": [
        "調整優化器"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qA49Af0MBOMW"
      },
      "source": [
        "from keras import optimizers\r\n",
        "\r\n",
        "model.compile(optimizer=optimizers.RMSprop(lr=0.001), #調整優化器參數lr的值\r\n",
        "       loss='binary_crossentropy',\r\n",
        "       metrics=['accuracy'])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8s6jwHPZC3KI"
      },
      "source": [
        "使用自行定義的損失函數與metrics函數"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wFmROI7NC98A"
      },
      "source": [
        "from keras import losses\r\n",
        "from keras import metrics\r\n",
        "\r\n",
        "model.compile(optimizer=optimizers.RMSprop(lr=0.001), #調整優化器參數lr的值\r\n",
        "       loss=losses.binary_crossentropy, # 自行指定其他的損失函數與metrics函數\r\n",
        "       metrics=[metrics.binary_accuracy])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ooTWmOGsDS4f"
      },
      "source": [
        "## 設定驗證資料集  \r\n",
        "從原始訓練資料中分離出10000樣本來建立驗證資料集"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YXHIyPuGDgjW"
      },
      "source": [
        "x_val = x_train[:10000]\r\n",
        "partial_x_train = x_train[10000:]\r\n",
        "\r\n",
        "y_val = y_train[:10000]\r\n",
        "partial_y_train = y_train[10000:]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OXhXOnuXFqZR"
      },
      "source": [
        "## 訓練模型"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Yc9-iCOQFxT4",
        "outputId": "587ecb1e-c8e0-4b4b-d116-b1885b1e44ac"
      },
      "source": [
        "model.compile(optimizer='rmsprop', #調整優化器參數lr的值\r\n",
        "       loss='binary_crossentropy', # 自行指定其他的損失函數與metrics函數\r\n",
        "       metrics=['acc'])\r\n",
        "\r\n",
        "history = model.fit(partial_x_train,\r\n",
        "           partial_y_train,\r\n",
        "           epochs=20,\r\n",
        "           batch_size=512,\r\n",
        "           validation_data=(x_val,y_val))\r\n",
        "\r\n",
        "# model.fit()會回傳一個history的物件\r\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/20\n",
            "30/30 [==============================] - 2s 36ms/step - loss: 0.5721 - acc: 0.7353 - val_loss: 0.3689 - val_acc: 0.8708\n",
            "Epoch 2/20\n",
            "30/30 [==============================] - 1s 26ms/step - loss: 0.3038 - acc: 0.9080 - val_loss: 0.3537 - val_acc: 0.8496\n",
            "Epoch 3/20\n",
            "30/30 [==============================] - 1s 26ms/step - loss: 0.2209 - acc: 0.9314 - val_loss: 0.2770 - val_acc: 0.8917\n",
            "Epoch 4/20\n",
            "30/30 [==============================] - 1s 27ms/step - loss: 0.1635 - acc: 0.9506 - val_loss: 0.2742 - val_acc: 0.8898\n",
            "Epoch 5/20\n",
            "30/30 [==============================] - 1s 27ms/step - loss: 0.1311 - acc: 0.9619 - val_loss: 0.2935 - val_acc: 0.8855\n",
            "Epoch 6/20\n",
            "30/30 [==============================] - 1s 27ms/step - loss: 0.1085 - acc: 0.9679 - val_loss: 0.3010 - val_acc: 0.8867\n",
            "Epoch 7/20\n",
            "30/30 [==============================] - 1s 27ms/step - loss: 0.0881 - acc: 0.9744 - val_loss: 0.3153 - val_acc: 0.8838\n",
            "Epoch 8/20\n",
            "30/30 [==============================] - 1s 28ms/step - loss: 0.0717 - acc: 0.9816 - val_loss: 0.3409 - val_acc: 0.8801\n",
            "Epoch 9/20\n",
            "30/30 [==============================] - 1s 27ms/step - loss: 0.0538 - acc: 0.9882 - val_loss: 0.3704 - val_acc: 0.8808\n",
            "Epoch 10/20\n",
            "30/30 [==============================] - 1s 27ms/step - loss: 0.0478 - acc: 0.9897 - val_loss: 0.4292 - val_acc: 0.8712\n",
            "Epoch 11/20\n",
            "30/30 [==============================] - 1s 27ms/step - loss: 0.0407 - acc: 0.9907 - val_loss: 0.4374 - val_acc: 0.8680\n",
            "Epoch 12/20\n",
            "30/30 [==============================] - 1s 26ms/step - loss: 0.0277 - acc: 0.9953 - val_loss: 0.4479 - val_acc: 0.8740\n",
            "Epoch 13/20\n",
            "30/30 [==============================] - 1s 27ms/step - loss: 0.0230 - acc: 0.9960 - val_loss: 0.5001 - val_acc: 0.8717\n",
            "Epoch 14/20\n",
            "30/30 [==============================] - 1s 27ms/step - loss: 0.0162 - acc: 0.9980 - val_loss: 0.5101 - val_acc: 0.8731\n",
            "Epoch 15/20\n",
            "30/30 [==============================] - 1s 26ms/step - loss: 0.0137 - acc: 0.9982 - val_loss: 0.5399 - val_acc: 0.8715\n",
            "Epoch 16/20\n",
            "30/30 [==============================] - 1s 26ms/step - loss: 0.0103 - acc: 0.9993 - val_loss: 0.5705 - val_acc: 0.8680\n",
            "Epoch 17/20\n",
            "30/30 [==============================] - 1s 27ms/step - loss: 0.0069 - acc: 0.9998 - val_loss: 0.6125 - val_acc: 0.8690\n",
            "Epoch 18/20\n",
            "30/30 [==============================] - 1s 27ms/step - loss: 0.0084 - acc: 0.9987 - val_loss: 0.6377 - val_acc: 0.8691\n",
            "Epoch 19/20\n",
            "30/30 [==============================] - 1s 27ms/step - loss: 0.0043 - acc: 0.9997 - val_loss: 0.7171 - val_acc: 0.8641\n",
            "Epoch 20/20\n",
            "30/30 [==============================] - 1s 26ms/step - loss: 0.0037 - acc: 0.9999 - val_loss: 0.6990 - val_acc: 0.8647\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qZ1tFLEKLlkD",
        "outputId": "0c750958-bef4-4199-8591-39b5de3527aa"
      },
      "source": [
        "history_dict = history.history\r\n",
        "history_dict.keys()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "dict_keys(['loss', 'acc', 'val_loss', 'val_acc'])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 57
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3mHAZ8r3L2pp"
      },
      "source": [
        "繪製訓練與驗證的損失函數"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "id": "LBTezclZL1lp",
        "outputId": "cb84e446-c863-4a80-c4a7-650706a0fe04"
      },
      "source": [
        "import matplotlib.pyplot as plt\r\n",
        "\r\n",
        "history_dict = history.history\r\n",
        "loss_values = history_dict['loss']\r\n",
        "val_loss_values = history_dict['val_loss']\r\n",
        "\r\n",
        "epochs = range(1,len(loss_values)+1)\r\n",
        "\r\n",
        "plt.plot(epochs,loss_values,'bo',label='Training loss')\r\n",
        "plt.plot(epochs,val_loss_values,'b',label='Validation loss')\r\n",
        "plt.title('Training and validation loss')\r\n",
        "plt.xlabel('Epochs')\r\n",
        "plt.ylabel('Loss')\r\n",
        "plt.legend()\r\n",
        "\r\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZgU1dn38e8NgsiqLEYedhPUKDsDiLihJoIQcCOCvCJxIRhxj4Y8JEKImEXiY4hoxN0Eg8QkBCOuKC4xGgaCKIiKCApBBJRN1oH7/ePUQDPM0sNMTfX0/D7X1Vd3V1dX31M05+6z1Dnm7oiISNVVLekAREQkWUoEIiJVnBKBiEgVp0QgIlLFKRGIiFRxSgQiIlWcEoGUKzN7xswuLe99k2Rmy83srBiO62b2jejx783sp+nsexCfM9TMnj/YOIs57ulmtrK8jysV75CkA5DkmdmWlKe1gR3A7uj59919arrHcve+ceyb7dx9ZHkcx8xaAx8DNdw9Lzr2VCDtf0OpepQIBHevm//YzJYDV7j7iwX3M7ND8gsXEckeahqSIuVX/c3sR2b2GfCwmR1hZv8ws7Vm9mX0uHnKe+aY2RXR4+Fm9rqZTYz2/djM+h7kvm3M7FUz22xmL5rZZDP7YxFxpxPjz83sn9HxnjezximvX2JmK8xsvZmNKeb89DCzz8ysesq288xsYfS4u5n9y8w2mNlqM7vbzGoWcaxHzOy2lOc3R+/5r5ldVmDffmb2HzPbZGafmtm4lJdfje43mNkWM+uZf25T3n+Smc01s43R/UnpnpvimNk3o/dvMLNFZjYg5bVzzGxxdMxVZvbDaHvj6N9ng5l9YWavmZnKpQqmEy4lOQpoCLQCRhC+Mw9Hz1sC24C7i3l/D+B9oDHwa+BBM7OD2Pdx4N9AI2AccEkxn5lOjBcD3wOOBGoC+QXT8cC90fH/J/q85hTC3d8CvgLOKHDcx6PHu4Ebor+nJ3Am8INi4iaKoU8Uz7eAtkDB/omvgGHA4UA/4CozOzd67dTo/nB3r+vu/ypw7IbA08Ck6G+7E3jazBoV+BsOODclxFwDeAp4PnrfNcBUMzs22uVBQjNjPaAd8FK0/SZgJdAE+Brwv4DmvalgSgRSkj3AWHff4e7b3H29u//F3be6+2ZgAnBaMe9f4e73u/tu4FGgKeE/fNr7mllLoBtwq7vvdPfXgZlFfWCaMT7s7h+4+zZgOtAp2n4h8A93f9XddwA/jc5BUf4EDAEws3rAOdE23H2eu7/p7nnuvhy4r5A4CvPdKL533f0rQuJL/fvmuPs77r7H3RdGn5fOcSEkjg/d/Q9RXH8ClgDfSdmnqHNTnBOBusAvo3+jl4B/EJ0bYBdwvJnVd/cv3X1+yvamQCt33+Xur7kmQKtwSgRSkrXuvj3/iZnVNrP7oqaTTYSmiMNTm0cK+Cz/gbtvjR7WLeW+/wN8kbIN4NOiAk4zxs9SHm9Niel/Uo8dFcTri/oswq//883sUOB8YL67r4jiOCZq9vgsiuN2Qu2gJPvFAKwo8Pf1MLOXo6avjcDINI+bf+wVBbatAJqlPC/q3JQYs7unJs3U415ASJIrzOwVM+sZbb8DWAo8b2bLzGx0en+GlCclAilJwV9nNwHHAj3cvT77miKKau4pD6uBhmZWO2Vbi2L2L0uMq1OPHX1mo6J2dvfFhAKvL/s3C0FoYloCtI3i+N+DiYHQvJXqcUKNqIW7NwB+n3Lckn5N/5fQZJaqJbAqjbhKOm6LAu37e4/r7nPdfSCh2WgGoaaBu29295vc/WhgAHCjmZ1ZxliklJQIpLTqEdrcN0TtzWPj/sDoF3YuMM7Maka/Jr9TzFvKEuOTQH8zOznq2B1Pyf9PHgeuIyScPxeIYxOwxcyOA65KM4bpwHAzOz5KRAXjr0eoIW03s+6EBJRvLaEp6+gijj0LOMbMLjazQ8zsIuB4QjNOWbxFqD3cYmY1zOx0wr/RtOjfbKiZNXD3XYRzsgfAzPqb2TeivqCNhH6V4priJAZKBFJadwGHAeuAN4FnK+hzhxI6XNcDtwFPEK53KMxBx+jui4CrCYX7auBLQmdmcfLb6F9y93Up239IKKQ3A/dHMacTwzPR3/ASodnkpQK7/AAYb2abgVuJfl1H791K6BP5ZzQS58QCx14P9CfUmtYDtwD9C8Rdau6+k1Dw9yWc93uAYe6+JNrlEmB51EQ2kvDvCaEz/EVgC/Av4B53f7kssUjpmfplpDIysyeAJe4ee41EJNupRiCVgpl1M7Ovm1m1aHjlQEJbs4iUka4slsriKOCvhI7blcBV7v6fZEMSyQ5qGhIRqeLUNCQiUsVVuqahxo0be+vWrZMOQ0SkUpk3b946d29S2GuVLhG0bt2a3NzcpMMQEalUzKzgFeV7qWlIRKSKUyIQEanilAhERKq4StdHUJhdu3axcuVKtm/fXvLOkqhatWrRvHlzatSokXQoIhLJikSwcuVK6tWrR+vWrSl6zRNJmruzfv16Vq5cSZs2bZIOR0QiWdE0tH37dho1aqQkkOHMjEaNGqnmJpJhsiIRAEoClYT+nUQyT9YkAhGRTLV7N9x7Lzz/POzcmXQ0B1IiKAfr16+nU6dOdOrUiaOOOopmzZrtfb6zhH/13Nxcrr322hI/46STTiqXWOfMmUP//v3L5Vgikp5f/hJ+8AM4+2xo0gQuvhimT4fNm5OOLKiSiWDqVGjdGqpVC/dTp5bteI0aNWLBggUsWLCAkSNHcsMNN+x9XrNmTfLy8op8b05ODpMmTSrxM954442yBSkiiXjjDRg7Fi66CGbOhAsvhBdeCM8bN4Z+/eD++2HNmuRijDURmFkfM3vfzJYWtii1mf2fmS2Ibh+Y2YY444FQ6I8YAStWgHu4HzGi7MmgoOHDhzNy5Eh69OjBLbfcwr///W969uxJ586dOemkk3j//feB/X+hjxs3jssuu4zTTz+do48+er8EUbdu3b37n3766Vx44YUcd9xxDB06lPwZZGfNmsVxxx1H165dufbaa0v85f/FF19w7rnn0qFDB0488UQWLlwIwCuvvLK3RtO5c2c2b97M6tWrOfXUU+nUqRPt2rXjtddeK98TJpKFNmwIv/5btoT77oPvfAcefBA++wxeeQWuvhoWLw5lUNOmcPLJMHEiLF1awYG6eyw3oDrwEWHt1JrA28Dxxex/DfBQScft2rWrF7R48eIDthWlVSv3kAL2v7VqlfYhijV27Fi/4447/NJLL/V+/fp5Xl6eu7tv3LjRd+3a5e7uL7zwgp9//vnu7v7yyy97v3799r63Z8+evn37dl+7dq03bNjQd+7c6e7uderU2bt//fr1/dNPP/Xdu3f7iSee6K+99ppv27bNmzdv7suWLXN398GDB+89bqrUzxs1apSPGzfO3d1nz57tHTt2dHf3/v37++uvv+7u7ps3b/Zdu3b5xIkT/bbbbnN397y8PN+0adNBn6PS/HuJVFZ79rgPGuR+yCHub75Z/H5vv+0+bpx7p077yqR27dx/8hP33NywT1kBuV5EuRpnjaA7sNTdl3lYz3QaYVWpogwhrP0aq08+Kd32shg0aBDVq1cHYOPGjQwaNIh27dpxww03sGjRokLf069fPw499FAaN27MkUceyZpC6ovdu3enefPmVKtWjU6dOrF8+XKWLFnC0UcfvXd8/pAhQ0qM7/XXX+eSSy4B4IwzzmD9+vVs2rSJXr16ceONNzJp0iQ2bNjAIYccQrdu3Xj44YcZN24c77zzDvXq1TvY0yJSJTzwAPz5z3DbbdCjR9H7mUGHDqH56D//gY8/hrvugkaN4PbbIScHWrWCa66Bt9+OJ9Y4E0Ez4NOU5yujbQcws1ZAGw5cpDv/9RFmlmtmuWvXri1TUC1blm57WdSpU2fv45/+9Kf07t2bd999l6eeeqrIsfSHHnro3sfVq1cvtH8hnX3KYvTo0TzwwANs27aNXr16sWTJEk499VReffVVmjVrxvDhw3nsscfK9TNFssnixXDddXDWWXDzzaV7b+vW4b1z5oR+g4cfhs6dQ2KpjImgNAYDT7r77sJedPcp7p7j7jlNmhQ6nXbaJkyA2rX331a7dtgep40bN9KsWciDjzzySLkf/9hjj2XZsmUsX74cgCeeeKLE95xyyilMjTpH5syZQ+PGjalfvz4fffQR7du350c/+hHdunVjyZIlrFixgq997WtceeWVXHHFFcyfP7/c/waRbLB9OwweDHXrwmOPhUEpB6txYxg+HP7+d1i3DgYNKrcw9xNnIlgFtEh53jzaVpjBVECzEMDQoTBlSqhqmYX7KVPC9jjdcsst/PjHP6Zz587l/gse4LDDDuOee+6hT58+dO3alXr16tGgQYNi3zNu3DjmzZtHhw4dGD16NI8++igAd911F+3ataNDhw7UqFGDvn37MmfOHDp27Ejnzp154oknuO6668r9bxDJBj/8IbzzDjz6aOgALi916sBhh5Xf8VLFtmaxmR0CfACcSUgAc4GL3X1Rgf2OA54F2ngaweTk5HjBhWnee+89vvnNb5ZX6JXWli1bqFu3Lu7O1VdfTdu2bbnhhhuSDusA+veSbPX3v8O558KNN8JvfpN0NPszs3nunlPYa7HVCNw9DxgFPAe8B0x390VmNt7MBqTsOhiYlk4SkOLdf//9dOrUiRNOOIGNGzfy/e9/P+mQRKqMlSvhssugS5fQyVuZxFYjiItqBJWf/r0k2+zeDWecAfPmwfz5cMwxSUd0oOJqBFkxDbWISJJuvx1efTX0C2RiEihJpowaEhGplF5/HcaNCwNOostyKh0lAhGRg/Tll2EKiTZt4J57wkjEykhNQyIiB8EdrrwSVq8OE8vVr590RAdPNYJy0Lt3b5577rn9tt11111cddVVRb7n9NNPJ7/T+5xzzmHDhgPn2xs3bhwTJ04s9rNnzJjB4sWL9z6/9dZbefHFF0sTfqE0XbVI8aZMgb/8JfQPdOuWdDRlo0RQDoYMGcK0adP22zZt2rS05vuBMGvo4YcfflCfXTARjB8/nrPOOuugjiUi6Vm0CK6/Hr79bbjppqSjKTslgnJw4YUX8vTTT+9dhGb58uX897//5ZRTTuGqq64iJyeHE044gbFjxxb6/tatW7Nu3ToAJkyYwDHHHMPJJ5+8d6pqCNcIdOvWjY4dO3LBBRewdetW3njjDWbOnMnNN99Mp06d+Oijjxg+fDhPPvkkALNnz6Zz5860b9+eyy67jB07duz9vLFjx9KlSxfat2/PkiVLiv37NF21yD7btoW1BOrXD6OEyjKFRKbIuj6C66+HBQvK95idOoXZAIvSsGFDunfvzjPPPMPAgQOZNm0a3/3udzEzJkyYQMOGDdm9ezdnnnkmCxcupEOHDoUeZ968eUybNo0FCxaQl5dHly5d6Nq1KwDnn38+V155JQA/+clPePDBB7nmmmsYMGAA/fv358ILL9zvWNu3b2f48OHMnj2bY445hmHDhnHvvfdy/fXXA9C4cWPmz5/PPffcw8SJE3nggQeK/PvGjh1L586dmTFjBi+99BLDhg1jwYIFTJw4kcmTJ9OrVy+2bNlCrVq1mDJlCmeffTZjxoxh9+7dbN26tTSnWiTj3XRTqBE8+ywcdVTS0ZSPLMhlmSG1eSi1WWj69Ol06dKFzp07s2jRov2acQp67bXXOO+886hduzb169dnwIB9F2C/++67nHLKKbRv356pU6cWOY11vvfff582bdpwTDSo+dJLL+XVV1/d+/r5558PQNeuXfdOVFcUTVctEvztb2Ht4R/+MCw7mS2yrkZQ3C/3OA0cOJAbbriB+fPns3XrVrp27crHH3/MxIkTmTt3LkcccQTDhw8vcvrpkgwfPpwZM2bQsWNHHnnkEebMmVOmePOnsi7LNNajR4+mX79+zJo1i169evHcc8/tna766aefZvjw4dx4440MGzasTLGKZIJPPoHLLw/rA8Q9W3FFU42gnNStW5fevXtz2WWX7a0NbNq0iTp16tCgQQPWrFnDM888U+wxTj31VGbMmMG2bdvYvHkzTz311N7XNm/eTNOmTdm1a9feqaMB6tWrx+ZCVsA+9thjWb58OUujNe/+8Ic/cNpppx3U36bpqqWqW7kyXC+waxf86U9Qs2bSEZWvrKsRJGnIkCGcd955e5uI8qdtPu6442jRogW9evUq9v1dunThoosuomPHjhx55JF0SxmT9vOf/5wePXrQpEkTevTosbfwHzx4MFdeeSWTJk3a20kMUKtWLR5++GEGDRpEXl4e3bp1Y+TIkQf1d+WvpdyhQwdq166933TVL7/8MtWqVeOEE06gb9++TJs2jTvuuIMaNWpQt25dLWAjldqbb4ZWhiefDNcN/PGP8I1vJB1V+dOkc1Lh9O8lmWzXrlDw//a38NZb0KABXHEFjBoVVg+rrDTpnIhICdavDxeJTZ4Mq1ZB27Zw991w6aVhtbFspkQgIlXau+/CpEnwhz+EZSa/9a2QEPr0yY5rBNKRNYnA3bHKOuNTFVLZmiIlO+3ZA7NmheafF1+EWrVg2DC49lo44YSko6t4WZEIatWqxfr162nUqJGSQQZzd9avX0+tWrWSDkWqqM2bw9XAkybBhx9Cs2bwi1+EyeMaNUo6uuRkRSJo3rw5K1euZO3atUmHIiWoVasWzZs3TzoMqWI2b4bx40OTz6ZNcOKJ4fkFF0CNGklHl7ysSAQ1atSgTZs2SYchIhkoNxeGDIFly+C734XrrguJQPaJtSvEzPqY2ftmttTMRhexz3fNbLGZLTKzx+OMR0Sqjj17YOJEOOkk2LED5swJF4MpCRwothqBmVUHJgPfAlYCc81sprsvTtmnLfBjoJe7f2lmR8YVj4hUHZ99FoZ9Pv88nH8+3H8/NGyYdFSZK84aQXdgqbsvc/edwDRgYIF9rgQmu/uXAO7+eYzxiEgV8Oyz0LFjWEz+978PF4cpCRQvzkTQDPg05fnKaFuqY4BjzOyfZvammfUp7EBmNsLMcs0sVx3CIlKYHTvCFNF9+8LXvhb6Br7//cq7jnBFSvpyiUOAtsDpwBDgfjM7YKkud5/i7jnuntOkSZMKDlFEMt0HH4S+gDvvhKuvDlNDVMXrAQ5WnIlgFdAi5XnzaFuqlcBMd9/l7h8DHxASg4hIidzhkUegSxdYvhxmzAjTQhx2WNKRVS5xJoK5QFsza2NmNYHBwMwC+8wg1AYws8aEpqJlMcYkIlli0yYYOhS+972wePzbb8PAgr2QkpbYEoG75wGjgOeA94Dp7r7IzMabWf7SW88B681sMfAycLO7r48rJhHJDm+9FZaQnT4dbrstTBOh6xQPXlZMQy0iVcOePfDrX8NPfxqmh3j88dA3ICXTNNQiUun9979wySXw0kvhCuH77oPDDxhaIgcj6VFDIiLF2rEjTBLXoUNYMezBB2HaNCWB8qREICIZaffuMFPosceG+YE6dIB58+Cyy3RtQHlTIhCRjOIOf/tbKPiHD4fGjcNUEbNnw3HHJR1ddlIiEJGM8dJLYVK4888PHcNPPglz54ZVw1QLiI8SgYgkLr+wP/NMWL069AO8805YL0AJIH5KBCKSmPfeC4V99+6wYAH83/+F6SIuuwwO0ZjGCqNTLSIV7pNPYNy40Blcp054fOONUK9e0pFVTUoEIlJhPv8cbr8d7r03NPlcfz38+MehQ1iSo0QgIrHbuDHMDHrnnbB1a5gfaOxYaNGi5PdK/JQIRCQ2X30Fv/tdmBbiyy9h0CD4+c/DtQGSOZQIRKTcbd8eVgf7xS9Cc1C/fjB+fJguWjKPRg2JSLnZuTMkgG98A264Adq3hzfegH/8Q0kgkykRiEiZ5eWFBWKOPRauugpatQoXh734IvTsmXR0UhIlAhE5aHv2wBNPQLt2oQO4YUOYNQtefx169046OkmXEoGIlJo7/P3vYXGYwYPDxV9//WtYML5vX10NXNkoEYhI2tzDBHA9esC554ZO4ccfD8tEnneeEkBlpUQgIiXauTO0+Z92Gpx9NqxZE+YDWrwYhgyB6tWTjlDKItbho2bWB/gtUB14wN1/WeD14cAdwKpo093u/kCcMYlI0Xbvho8/hnff3f/2/vuhQ7hpU5g8GS6/HA49NOlopbzElgjMrDowGfgWsBKYa2Yz3X1xgV2fcPdRccUhIgdyh1WrDizwFy+Gbdv27demTegIHjAgrA8wcCAcdlhycUs84qwRdAeWuvsyADObBgwECiYCEYnZzp1hgrf58/cV+hs27Hu9adNQ4I8cGe7btYPjj4e6dZOLWSpOnImgGfBpyvOVQI9C9rvAzE4FPgBucPdPC+5gZiOAEQAtW7YsdSBTp8KYMWHGw5YtYcIEGDq01IcRqZQ2bw5TPb/wQljnt3370K6fX+CfcAI0apR0lJKkpKeYeAr4k7vvMLPvA48CZxTcyd2nAFMAcnJyvDQfMHUqjBgRJroCWLEiPAclA8l+a9aE6R0WLICHHgpLP2pkjxQU56ihVUDq3ILN2dcpDIC7r3f3HdHTB4Cu5R3EmDH7kkC+rVvDdpFs9tFH0KtXWPxl5sxwwZeSgBQmzkQwF2hrZm3MrCYwGJiZuoOZNU15OgB4r7yD+OST0m0XyQbz58NJJ4UZP2fPhnPOSToiyWSxJQJ3zwNGAc8RCvjp7r7IzMab2YBot2vNbJGZvQ1cCwwv7ziK6lI4iK4GkUph9uww3r9WLfjnP8Ni8CLFMfdSNbknLicnx3Nzc9Pev2AfAUDt2jBlivoIJPtMmwbDhoXJ3559Fpo1SzoiyRRmNs/dcwp7LeuvLB46NBT6rVqF9tFWrZQEJDv99rdhNFDPnvDaa0oCkr6kRw1ViKFDVfBL9nKH//1f+OUv4fzzQy24Vq2ko5LKpEokApFstWsXXHlluFhs5Ei4+27N+yOll/VNQyLZ6quvwgygjz4KP/sZ3HOPkoAcHNUIRCqhdeugf3+YOxfuu2/fRZIiB0OJQKSSWbEiTAW9fDn85S+hViBSFkoEIpXIO+9Anz5hOPQLL8AppyQdkWQD9RGIVBKvvhoKfrMwPFRJQMqLagQiGWjTJli4MNzefjvc5s+Hr38dnntOV8ZL+VIiEEnQnj1hRbD8wj6/4P/44337HHEEdOwI110Ho0drymgpf0oEIhVky5bQxp9a6C9cGLYDVKsGbdtCt25wxRWh8O/QAZo316yhEi8lApFytHlzmP556dL9bx9+CP/97779GjQIhfzw4aHA79gxLBBTu3ZioUsVpkQgUkobNhxY0Off1qzZf9+jjoJvfAO+/e1w3759KPRbttSvfMkcSgQiaViwAK69Nizuvn79/q81axYK+f79w33+7etfh3r1kolXpDSUCERKsHAhnHUW1KwJF164f2F/9NFqzpHKT4lApBiLFsGZZ4bZPF95JfzKF8k2uqBMpAjvvQdnnAE1asDLLysJSPZSIhApxPvvhyRgBi+9FIZ1imQrNQ2JFLB0aUgCu3fDnDlw3HFJRyQSr1hrBGbWx8zeN7OlZja6mP0uMDM3s0LX0xSpKMuWQe/esGNHqAkcf3zSEYnEL7ZEYGbVgclAX+B4YIiZHfDfyszqAdcBb8UVi0g6VqwISWDrVpg9G9q1SzoikYoRZ42gO7DU3Ze5+05gGjCwkP1+DvwK2B5jLCLF+vTTkAQ2bQrTO3fsmHREIhUnzkTQDPg05fnKaNteZtYFaOHuTxd3IDMbYWa5Zpa7du3a8o9UqrRVq0ISWL8enn8eunRJOiKRipXYqCEzqwbcCdxU0r7uPsXdc9w9p0mTJvEHJ1XG6tUhCXz+eZjeuVu3pCMSqXhxJoJVQIuU582jbfnqAe2AOWa2HDgRmKkOY6koa9aE0UGrV8Ozz8KJJyYdkUgy0koEZlYn+gWPmR1jZgPMrEYJb5sLtDWzNmZWExgMzMx/0d03untjd2/t7q2BN4EB7p57UH+JSCl8/nlIAp98ArNmwUknJR2RSHLSrRG8CtQys2bA88AlwCPFvcHd84BRwHPAe8B0d19kZuPNbMDBhyxSNuvWhbmDPv4Ynn5aSz6KpHtBmbn7VjO7HLjH3X9tZgtKepO7zwJmFdh2axH7np5mLCIH7Ysv4FvfCusD/OMfcPrpSUckkrx0awRmZj2BoUD+CJ/q8YQkEo8NG8K6AIsXw4wZYTI5EUk/EVwP/Bj4W9S8czTwcnxhlb+NG8MYcamaPv0Uzj47TCn9t7+FxyISpJUI3P0Vdx/g7r+KOo3Xufu1McdWrh5+GBo2hJ494Sc/CbNJ7tiRdFQSt9Wr4ZprwtoBCxbAX/4C55yTdFQimSXdUUOPm1l9M6sDvAssNrOb4w2tfPXuDaOj2Y5+8YswYuTww0NTwa9+Bbm5YZIxyQ6ffw433hgWjvn97+HSS+GDD+A730k6MpHMY+5e8k5mC9y9k5kNBboAo4F57t4h7gALysnJ8dzcso0w3bgxLDIye3a4LVoUth9xROg8PPPMcDv2WK0rW9msWwd33AF33w3bt8OwYfDTn4aEIFKVmdk8dy/0Oq10Rw3ViK4bOBe42913mVnJGSRDNWgAAwaEG8Bnn4WZJmfPhhdfDG3IENaizU8KZ54Znktm+vJL+M1v4Le/ha++gosvhltvhWOOSToykcyXbiK4D1gOvA28amatgKzpej3qqFBwXHwxuMNHH+2rLTz9NDz2WNivW7dw8VHjxsnGK/ts3Ah33QV33hkGAwwaBOPGafpokdJIq2mo0DeaHRJdNFahyqNpqDT27AkjTZ5/PjQxnHlmGH9eTWu7JWrLFpg0CSZODLWBc8+Fn/0MOlR4Y6VI5VDmpiEzawCMBU6NNr0CjAc2lkuEGaxaNejUKdzq1oWrr4Zf/3pfx7NUrK1bYfLk8G+wbh307x9qAF27Jh2ZSOWV7u/ah4DNwHej2ybg4biCylRXXQUXXRSGn772WtLRVC1ffRXa/48+Gm65JRT8b74JTz2lJCBSVun2EXzd3S9Ief6zdKaYyDZmMGUKzJ8PgwfDf/4DRx6ZdFTZbdWqMALovvtCE9AZZ8CTT8LJJycdmUj2SLdGsM3M9v7XM7NewLZ4Qsps9evD9OlhEYmArZQAABIeSURBVJNLLgl9CFL+cnNh6FBo3To0A/XuHWphs2crCYiUt3QTwUhgspktj9YOuBv4fmxRZbhOnUJH5fPPw+23Jx1N9ti9OwzdPeWUMELrqadg1ChYujRcEawEIBKPtJqG3P1toKOZ1Y+ebzKz64GFcQaXya68MlyUNnYs9OoVfrHKwdm8GR56KCTXZctCLeDOO+Hyy0MNTETiVapBkO6+yd3zrx+4MYZ4Kg2z0G7dtm24/mDNmqQjqnyWL4ebboLmzeH666Fp09D+/+GHcMMNSgIiFaUso+Gr/OQLdevCn/8cLmq6+GLNVZQOd3jjjXDh19e/HkYC9esHb70Fr78OF1wAh6Q7hEFEykVZEkGlnWKiPLVvH0a1vPQS/PznSUeTubZsgWnTwrrAvXqFqTxuvjnUCh5/HLp3TzpCkaqr2N9eZraZwgt8Aw6LJaJK6HvfC/0F48eHDs2zzko6ouRt2hR+4b/yCsyZA/PmhRpT27bhgrBLL4U6dZKOUkSgDFNMJKWip5hI11dfhV+169aFee+bNk06ooq1YUMY3vnKK+E2f34YWlujRjgvp50WrgHo3VvTc4gkoTxmHz3YD+4D/JawrOUD7v7LAq+PBK4GdgNbgBHuvjjOmOJSp07oL+jWDYYMCU0f2dzW/cUXoeCfMycU/AsWhPb/mjWhRw8YMyYU/j17Qu3aSUcrIsWJrUZgZtWBD4BvASuBucCQ1ILezOrnj0IyswHAD9y9T3HHzdQaQb7HHgvNHmPGwG23JR1N+di2LbTlv/fevl/8CxeGgv/QQ0Nhf9ppYS2HHj3gMDUaimScpGoE3YGl7r4sCmIaMBDYmwhShqIC1CFDO6CnTg0F+yefQMuWMGFCuOq1MMOGhYLy9tvDhVGVYW3c3bth5Ur4+OMwjv/jj/fdli0L6zXkO+wwOOmkMNPnaaeFZp9atZKLXUTKLs5E0Az4NOX5SqBHwZ3M7GrCNQk1gTMKO5CZjQBGALRs2bLcAy3O1KkwYkSY9RJgxYrwHIpOBr/7Hfz73/D//l+Yj6h584qJtTi7doXmm48+OrCg/+QTyEuZULxaNWjRAtq0gb59w32bNqGjt3Pn0PwjItkjzqahC4E+7n5F9PwSoIe7jypi/4uBs9390uKOW9FNQ61bh8K/oFatQnNJUZYsgZycUHC+/HIy/QW7doXP/vOf4a9/De36+Zo0CTN55hfyqbeWLUMnr4hkj6SahlYBLVKeN4+2FWUacG+M8RyUTz4p3fZ8xx0XZiodOjRMW/3LXxa/f3nJy9u/8F+/Plz4NmBAWLzlm98Mya1u3YqJR0QyX5yJYC7Q1szaEBLAYODi1B3MrK27fxg97Qd8SIZp2bLwGkE6LVQXXxz6C371q9Bf0K9f+ccHofCfM2df4b9u3b7Cf9Cg0E+hDlwRKUpsicDd88xsFPAcYfjoQ+6+yMzGA7nuPhMYZWZnAbuAL4Fim4WSMGHC/n0EEIZDTpiQ3vvvuitMnzBsWOgvKK8ujry8kGSmT99X+Neps6/w79NHhb+IpEcXlKWhNKOGCvPhh2EVrVatwprH9eqFX+yF3Rfcltoxm5cHr766r/BfuzYU/t/5Tij8+/ZV4S8ihSuuj0CJoIL8/e9w3XWhw3bLljAGPx01a+5LDFu2hDb/2rX3L/x1wZaIlCSxK4tln4EDww1CEti6NczDv2VLevebN0P16qGf4ZxzVPiLSPlRIkiAWWjS0aRrIpIJNP2XiEgVp0QgIlLFKRGIiFRxSgQiIlWcEoGISBWnRCAiUsUpEYiIVHFKBCIiVZwSgYhIFadEICJSxSkRiIhUcUoEFWDq1LAqWLVq4X7q1KQjEhHZR5POxWzq1P0XtlmxIjyH0q1pICISF9UIYjZmzP6rm0F4PmZMMvGIiBSkRBCzoha5L2q7iEhFizURmFkfM3vfzJaa2ehCXr/RzBab2UIzm21mreKMJwlFrVFcXmsXi4iUVWyJwMyqA5OBvsDxwBAzO77Abv8Bcty9A/Ak8Ou44knKhAkHriZWu3bYLiKSCeKsEXQHlrr7MnffCUwDBqbu4O4vu3t+C/qbQPMY40nE0KEwZUpYuN4s3E+Zoo5iEckccY4aagZ8mvJ8JdCjmP0vB54p7AUzGwGMAGhZCdtUhg5VwS8imSsjOovN7P8BOcAdhb3u7lPcPcfdc5o0aVKxwYmIZLk4awSrgBYpz5tH2/ZjZmcBY4DT3H1HjPGIiEgh4qwRzAXamlkbM6sJDAZmpu5gZp2B+4AB7v55jLGIiEgRYksE7p4HjAKeA94Dprv7IjMbb2YDot3uAOoCfzazBWY2s4jDiYhITGKdYsLdZwGzCmy7NeXxWXF+voiIlCwjOouleJq0TkTipEnnMpwmrRORuKlGkOE0aZ2IxE2JIMNp0joRiZsSQYbTpHUiEjclggynSetEJG5KBBlOk9aJSNw0aqgS0KR1IhIn1QhERKo4JYIqQBekiUhx1DSU5XRBmoiURDWCLKcL0kSkJEoEWU4XpIlISZQIspwuSBORkigRZDldkCYiJVEiyHK6IE1ESqJEUAUMHQrLl8OePeG+tElAw09FspuGj0qxNPxUJPupRiDF0vBTkewXayIwsz5m9r6ZLTWz0YW8fqqZzTezPDO7MM5Y5OBo+KlI9ostEZhZdWAy0Bc4HhhiZscX2O0TYDjweFxxSNlo+KlI9ouzRtAdWOruy9x9JzANGJi6g7svd/eFwJ4Y45Ay0PBTkewXZyJoBnya8nxltK3UzGyEmeWaWe7atWvLJThJT3kMP9WoI5HMVilGDbn7FGAKQE5OjiccTpVTlvUQNOpIJPPFWSNYBbRIed482iZViEYdiWS+OBPBXKCtmbUxs5rAYGBmjJ8nGag8Rh2paUkkXrElAnfPA0YBzwHvAdPdfZGZjTezAQBm1s3MVgKDgPvMbFFc8UgyyjrqKL9pacUKcN/XtKRkIFJ+zL1yNbnn5OR4bm5u0mFImgr2EUAYdZRuh3Pr1qHwL6hVqzBdhoikx8zmuXtOYa/pymKJVVlHHemCNpH4VYpRQ1K5lWXUUcuWhdcIdEGbSPlRjUAymi5oE4mfEoFkNF3QJhI/NQ1JxtMFbSLxUo1AspouaBMpmRKBZDVd0CZSMiUCyWqZcEGbEolkOiUCyWplHXVU1qYlXRktlYESgWS1pC9oUx+FVAYaNSRZL8kL2nRltFQGqhGIFKOsTUvltdSn+hkkTkoEIsUoa9NSeVwZrQ5riZ27V6pb165dXaQy+eMf3Vu1cjcL93/8Y+ne36qVe0gB+99atUr/82vX3v+9tWuXLo6y/g2SPCDXiyhXNQ21SIarVi0U3wWZwZ49Jb+/rFN5l3Uq8fxjjBkT+kZatgw1Il3ZXbE0DbVIJVbWfoakRz6paSvzKRGIZLikO6yzIZGUVbYnIiUCkQyXdId1ZU8kULaCvErUaIrqPMjUmzqLRUqvLJ29Ze1sLmtnt1nh7zerHPFnSmc9xXQWx1poA32A94GlwOhCXj8UeCJ6/S2gdUnHVCIQqXiVOZEknYgyIZG4J5QIgOrAR8DRQE3gbeD4Avv8APh99Hgw8ERJx1UiEKl8kkwkSRfkSX9+vuISQZx9BN2Bpe6+zN13AtOAgQX2GQg8Gj1+EjjTzCzGmEQkAUOHhqGqe/aE+9IMHS1rH0lZ+zgqex9LOuJMBM2AT1Oer4y2FbqPu+cBG4FGBQ9kZiPMLNfMcteuXRtTuCKSqcqSSMpakFf2zvp0VIpRQ+4+xd1z3D2nSZMmSYcjIpVIeax7nWSNpjymKSlJnLOPrgJapDxvHm0rbJ+VZnYI0ABYH2NMIlIFlWUG2qQ/P/99cV6ZHWcimAu0NbM2hAJ/MHBxgX1mApcC/wIuBF6KOjVERCQSdyKLLRG4e56ZjQKeI4wgesjdF5nZeELv9UzgQeAPZrYU+IKQLEREpALFujCNu88CZhXYdmvK4+3AoDhjEBGR4lWKzmIREYmPEoGISBWnRCAiUsVVuoVpzGwtUMgyGxmhMbAu6SCKofjKJtPjg8yPUfGVTVnia+XuhV6IVekSQSYzs1wvYgWgTKD4yibT44PMj1HxlU1c8alpSESkilMiEBGp4pQIyteUpAMogeIrm0yPDzI/RsVXNrHEpz4CEZEqTjUCEZEqTolARKSKUyIoJTNrYWYvm9liM1tkZtcVss/pZrbRzBZEt1sLO1aMMS43s3eiz84t5HUzs0lmttTMFppZlwqM7diU87LAzDaZ2fUF9qnw82dmD5nZ52b2bsq2hmb2gpl9GN0fUcR7L432+dDMLq2g2O4wsyXRv9/fzOzwIt5b7Hch5hjHmdmqlH/Hc4p4bx8zez/6Po6uwPieSIltuZktKOK9sZ7DosqUCv3+FbWGpW5FrsXcFOgSPa4HfMCBazGfDvwjwRiXA42Lef0c4BnAgBOBtxKKszrwGeFCl0TPH3Aq0AV4N2Xbr4HR0ePRwK8KeV9DYFl0f0T0+IgKiO3bwCHR418VFls634WYYxwH/DCN70Cxa5vHFV+B138D3JrEOSyqTKnI759qBKXk7qvdfX70eDPwHgcuwZnpBgKPefAmcLiZNU0gjjOBj9w98SvF3f1VwlToqVLX1H4UOLeQt54NvODuX7j7l8ALQJ+4Y3P35z0s7wrwJmHhp8QUcf7Skc7a5mVWXHzROunfBf5U3p+bjmLKlAr7/ikRlIGZtQY6A28V8nJPM3vbzJ4xsxMqNDBw4Hkzm2dmIwp5PZ31pCvCYIr+z5fk+cv3NXdfHT3+DPhaIftkwrm8jFDDK0xJ34W4jYqarx4qomkjE87fKcAad/+wiNcr7BwWKFMq7PunRHCQzKwu8BfgenffVODl+YTmjo7A74AZFRzeye7eBegLXG1mp1bw55fIzGoCA4A/F/Jy0ufvAB7q4Rk31trMxgB5wNQidknyu3Av8HWgE7Ca0PySiYZQfG2gQs5hcWVK3N8/JYKDYGY1CP9gU939rwVfd/dN7r4lejwLqGFmjSsqPndfFd1/DvyNUP1Olc560nHrC8x39zUFX0j6/KVYk99kFt1/Xsg+iZ1LMxsO9AeGRgXFAdL4LsTG3de4+2533wPcX8RnJ/pdtLBW+vnAE0XtUxHnsIgypcK+f0oEpRS1Jz4IvOfudxaxz1HRfphZd8J5Xl9B8dUxs3r5jwmdiu8W2G0mMCwaPXQisDGlClpRivwVluT5KyB/TW2i+78Xss9zwLfN7Iio6ePb0bZYmVkf4BZggLtvLWKfdL4LccaY2u90XhGfvXdt86iWOJhw3ivKWcASd19Z2IsVcQ6LKVMq7vsXV094tt6AkwlVtIXAguh2DjASGBntMwpYRBgB8SZwUgXGd3T0uW9HMYyJtqfGZ8BkwmiNd4CcCj6HdQgFe4OUbYmeP0JSWg3sIrSzXg40AmYDHwIvAg2jfXOAB1LeexmwNLp9r4JiW0poG87/Dv4+2vd/gFnFfRcq8Pz9Ifp+LSQUak0Lxhg9P4cwUuajuGIsLL5o+yP537uUfSv0HBZTplTY909TTIiIVHFqGhIRqeKUCEREqjglAhGRKk6JQESkilMiEBGp4pQIRCJmttv2nxm13GbCNLPWqTNfimSSQ5IOQCSDbHP3TkkHIVLRVCMQKUE0H/2voznp/21m34i2tzazl6JJ1WabWcto+9csrBHwdnQ7KTpUdTO7P5pz/nkzOyza/9poLvqFZjYtoT9TqjAlApF9DivQNHRRymsb3b09cDdwV7Ttd8Cj7t6BMOnbpGj7JOAVD5PmdSFckQrQFpjs7icAG4ALou2jgc7RcUbG9ceJFEVXFotEzGyLu9ctZPty4Ax3XxZNDvaZuzcys3WEaRN2RdtXu3tjM1sLNHf3HSnHaE2YN75t9PxHQA13v83MngW2EGZZneHRhHsiFUU1ApH0eBGPS2NHyuPd7Ouj60eY+6kLMDeaEVOkwigRiKTnopT7f0WP3yDMlgkwFHgtejwbuArAzKqbWYOiDmpm1YAW7v4y8COgAXBArUQkTvrlIbLPYbb/AubPunv+ENIjzGwh4Vf9kGjbNcDDZnYzsBb4XrT9OmCKmV1O+OV/FWHmy8JUB/4YJQsDJrn7hnL7i0TSoD4CkRJEfQQ57r4u6VhE4qCmIRGRKk41AhGRKk41AhGRKk6JQESkilMiEBGp4pQIRESqOCUCEZEq7v8D2t0kPTRGWxcAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JYqkA-NtNJgC"
      },
      "source": [
        "繪製訓練與驗證的準確度"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "id": "SVN-XK16NJgb",
        "outputId": "f30ccf82-92ac-4242-b14f-1b7e373ca1bd"
      },
      "source": [
        "import matplotlib.pyplot as plt\r\n",
        "\r\n",
        "plt.clf()\r\n",
        "acc = history_dict['acc']\r\n",
        "val_acc = history_dict['val_acc']\r\n",
        "\r\n",
        "plt.plot(epochs,acc,'bo',label='Training acc')\r\n",
        "plt.plot(epochs,val_acc,'b',label='Validation acc')\r\n",
        "plt.title('Training and validation accuracy')\r\n",
        "plt.xlabel('Epochs')\r\n",
        "plt.ylabel('Accuracy')\r\n",
        "plt.legend()\r\n",
        "\r\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZgU1dn38e+PYV+VRUV2FUWMso2oGBV3XCJiNEqIgpoQXGMSYzSahKA8idFEX+OSB6OASsQtGkw0LojLEzdGZHFDEFFBxBGVRUS2+/3jVENP0z3TPdPVPTPcn+uqq6tOLX26pqfuPudUnSMzwznnnMtWg2JnwDnnXN3igcM551xOPHA455zLiQcO55xzOfHA4ZxzLiceOJxzzuXEA4erMUmPSxqZ722LSdJiSUfFcFyTtEc0/1dJv85m22q8zwhJT1Y3n85VRv4cx/ZJ0pqkxebAN8CmaPnHZjal8LmqPSQtBn5oZk/n+bgG9DSzhfnaVlJ34H2gkZltzEc+natMw2JnwBWHmbVMzFd2kZTU0C9Grrbw72Pt4FVVrgJJgyUtkfRLSZ8AEyXtKOlfksolfRHNd07a51lJP4zmR0n6P0nXR9u+L+m4am7bQ9LzklZLelrSLZLuyZDvbPJ4taT/Rsd7UlL7pPVnSvpA0gpJV1Zyfg6Q9ImkkqS0YZLmRvMDJb0k6UtJyyTdLKlxhmNNknRN0vIvon0+lnROyrYnSHpd0ipJH0kam7T6+ej1S0lrJB2UOLdJ+w+SNFPSyuh1ULbnJsfz3FbSxOgzfCHpkaR1QyXNjj7De5KGROkVqgUljU38nSV1j6rszpX0IfBMlP5A9HdYGX1H9knav5mkP0V/z5XRd6yZpH9Luijl88yVNCzdZ3WZeeBw6ewCtAW6AaMJ35OJ0XJX4Gvg5kr2PwCYD7QH/gjcIUnV2PbvwKtAO2AscGYl75lNHr8PnA3sBDQGLgWQ1Bu4LTr+rtH7dSYNM3sF+Ao4IuW4f4/mNwE/jT7PQcCRwPmV5JsoD0Oi/BwN9ARS21e+As4CdgBOAM6TdHK07tDodQcza2lmL6Ucuy3wb+Cm6LP9Gfi3pHYpn2Gbc5NGVef5bkLV5z7RsW6I8jAQuAv4RfQZDgUWZzofaRwG7A0cGy0/TjhPOwGzgOSq1euBAcAgwvf4MmAzMBn4QWIjSX2AToRz43JhZj5t5xPhH/ioaH4wsB5oWsn2fYEvkpafJVR1AYwCFiataw4YsEsu2xIuShuB5knr7wHuyfIzpcvjVUnL5wP/ieZ/A0xNWtciOgdHZTj2NcCd0XwrwkW9W4ZtLwEeTlo2YI9ofhJwTTR/J/CHpO32TN42zXFvBG6I5rtH2zZMWj8K+L9o/kzg1ZT9XwJGVXVucjnPQEfCBXrHNNv9byK/lX3/ouWxib9z0mfbrZI87BBt04YQ2L4G+qTZrinwBaHdCEKAubXQ/2/1YfISh0un3MzWJRYkNZf0v1HRfxWhamSH5OqaFJ8kZsxsbTTbMsdtdwU+T0oD+ChThrPM4ydJ82uT8rRr8rHN7CtgRab3IpQuTpHUBDgFmGVmH0T52DOqvvkkysf/EEofVamQB+CDlM93gKQZURXRSmBMlsdNHPuDlLQPCL+2EzKdmwqqOM9dCH+zL9Ls2gV4L8v8prPl3EgqkfSHqLprFVtLLu2jqWm694q+0/cBP5DUABhOKCG5HHngcOmk3mr3c2Av4AAza83WqpFM1U/5sAxoK6l5UlqXSravSR6XJR87es92mTY2s7cIF97jqFhNBaHK6x3Cr9rWwK+qkwdCiSvZ34FpQBczawP8Nem4Vd0a+TGhailZV2BpFvlKVdl5/ojwN9shzX4fAbtnOOZXhNJmwi5ptkn+jN8HhhKq89oQSiWJPHwGrKvkvSYDIwhViGstpVrPZccDh8tGK0Lx/8uovvy3cb9h9Au+DBgrqbGkg4DvxJTHB4ETJX07asgeR9X/G38HfkK4cD6Qko9VwBpJvYDzsszD/cAoSb2jwJWa/1aEX/ProvaC7yetKydUEe2W4diPAXtK+r6khpJOB3oD/8oyb6n5SHuezWwZoe3h1qgRvZGkRGC5Azhb0pGSGkjqFJ0fgNnAGdH2pcCpWeThG0KpsDmhVJfIw2ZCtd+fJe0alU4OikqHRIFiM/AnvLRRbR44XDZuBJoRfs29DPynQO87gtDAvILQrnAf4YKRTrXzaGZvAhcQgsEyQj34kip2u5fQYPuMmX2WlH4p4aK+Grg9ynM2eXg8+gzPAAuj12TnA+MkrSa0ydyftO9aYDzwX4W7uQ5MOfYK4ERCaWEFobH4xJR8Z6uq83wmsIFQ6vqU0MaDmb1KaHy/AVgJPMfWUtCvCSWEL4DfUbEEl85dhBLfUuCtKB/JLgXmATOBz4FrqXituwvYl9Bm5qrBHwB0dYak+4B3zCz2Eo+rvySdBYw2s28XOy91lZc4XK0laX9Ju0dVG0MI9dqPVLWfc5lE1YDnAxOKnZe6zAOHq812IdwquobwDMJ5ZvZ6UXPk6ixJxxLag5ZTdXWYq4RXVTnnnMuJlzicc87lZLvo5LB9+/bWvXv3YmfDOefqlNdee+0zM+uQmr5dBI7u3btTVlZW7Gw451ydIim1xwHAq6qcc87lyAOHc865nHjgcM45lxMPHM4553LigcM551xOYg0cku6U9KmkNzKsl6SbJC2MhnDsn7RupKQF0TQyKX2ApHnRPjdVMrKcc85Vy5Qp0L07NGgQXqdMqWqP2iXu/Mdd4pgEDKlk/XGE4R97EoYovQ22DHX5W8KwogOB30raMdrnNuBHSftVdnznnMvJlCkwejR88AGYhdfRowsbPGpy4S9E/mMNHGb2PKFb40yGAndZ8DJhJLGOhHGFnzKzxGhiTwFDonWtzexlC32l3AWcnPHozjmXoyuvhLVrK6atXRvSs1XMC38+8l+VYrdxdKLicJlLorTK0pekSd+GpNGSyiSVlZeX5zXTzrl41bSqpSb7f/hhbunp3ruYF/6a5j8bxQ4csTGzCWZWamalHTps88S8c66WqumFt6b7d00dtLeK9FTFvvDXNP/ZKHbgWErFcZY7R2mVpXdOk+6cy6Ni/uKv6YW3pvuPHw/Nm1dMa948pGej2Bf+muY/K2YW60QYSP6NDOtOIIxRLOBA4NUovS3wPrBjNL0PtI3WvRptq2jf46vKw4ABA8y57ck995h162Ymhdd77slt3+bNzcLv9TA1b579MWq6v1Rx38QkFWb/xGeo7vnr1i39+3frlv171+T81TT/yYAyS3ftTpeYr4kwLvMywhjES4BzgTHAmGi9gFuA9whjBJcm7XsOYezlhcDZSemlwBvRPjcTjSlS2eSBw21PanrhqemFr67vX1O16cJfU0UJHLVl8sDhtic1vXAW+xd/sUs8+VBbLvw1lSlwFLuNwzmXZ8WuY6/p/iNGwIQJ0K0bSOF1woSQXoj982HECFi8GDZvDq+FfO9C8MDhXC1Uk8blYjeu5qNxtqYX3vp+4S42DxzOxaCYD4DV9MJdH37xu3gpVGPVb6WlpeYjALpCSVz4k28Jbd48+4tn9+4hWKTq1i38es42D1deGaqnunYNQcMv3C5Xkl4zs9Jt0j1wOJdfNb3wN2gQShqppFD14lyhZAocXlXlXJ4Vu3Haubh54HAujbrcOO1c3DxwOJeirjdOOxc3b+NwLoU3TjsXeOO4Bw6XJW+cdi7wxnHnsuSN085VzgOHcym8cdq5ynngcC6FN047V7mGxc6Ac7XRiBEeKJzLxEscrl6q6Qh2zrnMvMTh6p3UvqISz2GAlyKcy4dYSxyShkiaL2mhpMvTrO8mabqkuZKeldQ5Sj9c0uykaZ2kk6N1kyS9n7Sub5yfwdU9NR1z2jlXudhKHJJKCMPCHk0YNnampGlm9lbSZtcDd5nZZElHAL8HzjSzGUDf6DhtCcPHPpm03y/M7MG48u7qtpr2FeWcq1ycJY6BwEIzW2Rm64GpwNCUbXoDz0TzM9KsBzgVeNzM1qZZ59w2/DkM5+IVZ+DoBHyUtLwkSks2Bzglmh8GtJLULmWbM4B7U9LGR9VbN0hqku7NJY2WVCaprLy8vHqfwBVNTRq3/TkM5+JV7LuqLgUOk/Q6cBiwFNiUWCmpI7Av8ETSPlcAvYD9gbbAL9Md2MwmmFmpmZV26NAhpuy7ONS0k0F/DsO5eMXWV5Wkg4CxZnZstHwFgJn9PsP2LYF3zKxzUtpPgH3MbHSGfQYDl5rZiZXlxfuqqlvy0cmgc67mitFX1Uygp6QekhoTqpympWSqvaREHq4A7kw5xnBSqqmiUgiSBJwMvBFD3l0ReeO2c7VbbIHDzDYCFxKqmd4G7jezNyWNk3RStNlgYL6kd4GdgS210JK6A12A51IOPUXSPGAe0B64Jq7P4IrDG7edq928W3VX66Q+wAehcdvbKZwrLO9W3dUZ3rjtXO3mgcPFoqZ9RY0YERrCN28Orx40nKs9vK8ql3feV5Rz9ZuXOFzeeV9RztVvHjhc3vnttM7Vbx44XN757bTO1W8eOFzeeV9RztVvHjhc3vnttM7Vb35XlYuFj9ntXP3lJQ7nnHM58cDh0qrpA3zOufrLq6rcNvwBPudcZbzE4bbhD/A55yrjgcNtwx/gc85VxgOH24Y/wOecq4wHDrcNf4DPOVcZDxxuG/4An3OuMrEGDklDJM2XtFDS5WnWd5M0XdJcSc9K6py0bpOk2dE0LSm9h6RXomPeF41n7vLMx8NwzmUSW+CQVALcAhwH9AaGS+qdstn1wF1mth8wDvh90rqvzaxvNJ2UlH4tcIOZ7QF8AZwb12dwzjm3rThLHAOBhWa2yMzWA1OBoSnb9AaeieZnpFlfgSQBRwAPRkmTgZPzlmPnnHNVijNwdAI+SlpeEqUlmwOcEs0PA1pJahctN5VUJullSYng0A740sw2VnJMACSNjvYvKy8vr+lncc45Fyl24/ilwGGSXgcOA5YCm6J13cysFPg+cKOk3XM5sJlNMLNSMyvt0KFDXjPtnHPbszgDx1KgS9Jy5yhtCzP72MxOMbN+wJVR2pfR69LodRHwLNAPWAHsIKlhpmO6wPuacs7FJc7AMRPoGd0F1Rg4A5iWvIGk9pISebgCuDNK31FSk8Q2wMHAW2ZmhLaQU6N9RgL/jPEz1EmJvqY++ADMtvY15cHDOZcPsQWOqB3iQuAJ4G3gfjN7U9I4SYm7pAYD8yW9C+wMJB4x2xsokzSHECj+YGZvRet+CfxM0kJCm8cdcX2Gusr7mnLOxUnhR3z9VlpaamVlZcXORsE0aBBKGqmk8FyGc85lQ9JrUVtzBcVuHHcx8L6mnHNx8sBRD3lfU865OHngqIe8rynnXJx8BMB6asQIDxTOuXh4icM551xOPHA455zLiQcO55xzOfHA4ZxzLiceOJxzzuXEA4dzzrmceOBwzjmXEw8czjnncuKBo5by8TScc7WVPzleCyXG00h0jZ4YTwP8aXDnXPF5iaMW8vE0nHO1mQeOWujDD3NLd865Qoo1cEgaImm+pIWSLk+zvpuk6ZLmSnpWUucova+klyS9Ga07PWmfSZLelzQ7mvrG+RmKwcfTcM7VZrEFDkklwC3AcUBvYLik3imbXQ/cZWb7AeOA30fpa4GzzGwfYAhwo6Qdkvb7hZn1jabZcX2GYvHxNJxztVmcJY6BwEIzW2Rm64GpwNCUbXoDz0TzMxLrzexdM1sQzX8MfAp0iDGvtYqPp+Gcq83iDBydgI+SlpdEacnmAKdE88OAVpLaJW8gaSDQGHgvKXl8VIV1g6Qm6d5c0mhJZZLKysvLa/I5imLECFi8OIwRvnixBw3nXO1R7MbxS4HDJL0OHAYsBTYlVkrqCNwNnG1mm6PkK4BewP5AW+CX6Q5sZhPMrNTMSjt02G4KK845F7s4n+NYCnRJWu4cpW0RVUOdAiCpJfBdM/syWm4N/Bu40sxeTtpnWTT7jaSJhODjnHOuQKoscUj6jqTqlExmAj0l9ZDUGDgDmJZy7PZJx74CuDNKbww8TGg4fzBln47Rq4CTgTeqkTfnnHPVlE1AOB1YIOmPknple2Az2whcCDwBvA3cb2ZvShon6aRos8HAfEnvAjsDifuGvgccCoxKc9vtFEnzgHlAe+CabPPknHOu5mRmVW8Uqo2GA2cDBkwE7jWz1fFmLz9KS0utrKys2Nlwzrk6RdJrZlaamp5VFZSZrQIeJNxS25FwB9QsSRflNZfOOedqvWzaOE6S9DDwLNAIGGhmxwF9gJ/Hmz3nnHO1TTZ3VX0XuMHMnk9ONLO1ks6NJ1vOOedqq2wCx1ggcQsskpoBO5vZYjObHlfGnHPO1U7ZtHE8AGxOWt4UpTnnnNsOZRM4GkZ9TQEQzTeOL0vOOedqs2wCR3nScxdIGgp8Fl+WnHPO1WbZtHGMITx0dzMgQseFZ8WaK+ecc7VWlYHDzN4DDoz6ksLM1sSeK+ecc7VWVp0cSjoB2AdoGrqIAjMbF2O+nHPO1VLZPAD4V0J/VRcRqqpOA7rFnK86b8oU6N4dGjQIr1OmFDtHzjmXH9k0jg8ys7OAL8zsd8BBwJ7xZqtumzIFRo+GDz4As/A6erQHD+dc/ZBN4FgXva6VtCuwgdBflcvgyith7dqKaWvXhnTnnKvrsmnjeFTSDsB1wCxC77i3x5qrOu7DD3NLd865uqTSwBENsjQ9GpXvIUn/Apqa2cqC5K6O6to1VE+lS3fOubqu0qqqaJzvW5KWv/GgUbXx46F584ppzZuHdOecq+uyaeOYLum7StyHmwNJQyTNl7RQ0uVp1neTNF3SXEnPSuqctG6kpAXRNDIpfYCkedExb6pOvuI2YgRMmADduoEUXidMCOnOOVfXVTkCoKTVQAtgI6GhXICZWesq9isB3gWOBpYQxiAfbmZvJW3zAPAvM5ss6QjgbDM7U1JboAwoJbSpvAYMMLMvJL0KXAy8AjwG3GRmj1eWFx8B0DnnclftEQDNrJWZNTCzxmbWOlquNGhEBgILzWxR1DHiVGBoyja9gWei+RlJ648FnjKzz83sC+ApYIikjkBrM3vZQsS7Czg5i7w455zLkyrvqpJ0aLr01IGd0uhE6NcqYQlwQMo2c4BTgP9HGI62laR2GfbtFE1L0qSny/doYDRAV2+Vds65vMnmdtxfJM03JZQkXgOOyMP7XwrcLGkU8DywlDDeR42Z2QRgAoSqqnwc0znnXHadHH4neVlSF+DGLI69FOiStNw5Sks+9seEEgdRJ4rfNbMvJS0FBqfs+2y0f+eU9ArHdM45F69s7qpKtQTYO4vtZgI9JfWQ1Bg4A5iWvIGk9tGzIgBXAHdG808Ax0jaUdKOwDHAE2a2DFgl6cDobqqzgH9W4zM455yrpmzaOP5CuLMJQqDpS3iCvFJmtlHShYQgUALcaWZvShoHlJnZNEKp4veSjFBVdUG07+eSriYEH4BxZvZ5NH8+MAloBjweTc455wokm9txRyYtbgQWm9l/Y81VnvntuM45l7tMt+Nm0zj+ILDOzDZFByqR1NzM1laxn3POuXooqyfHCdVCCc2Ap+PJjnPOudoumxJH0+ThYs1sjaTmle3g8uPyy+Fvf4MuXUK3Jd26hY4SE/PdukGHDqFbE+ecK5RsAsdXkvqb2SwIfUUBX8ebLbdmDdx8M/TsCZ06wXvvwTPPwOrVFbdr2nTbYJK83LkzNMxqgGDnnMtONpeUS4AHJH1M6KdqF8JQsi5G//gHfPUV/OUv8O1vhzQz+PLLMK7HBx9snRLLjz4Ky5dXPE7LlnD44XDMMXD00bDnnl5Ccc7VTJV3VQFIagTsFS3ON7MNseYqz+riXVWHHw4ffQQLFuR2oV+3LuyXCCqvvQZPPhlKLBBKI0cfHQLJkUdCu3bx5N85V/dV+64qSRcAU8zsjWh5R0nDzezWGPLpgPffh2efhauvzr100LRpqN7q2TMsn3tueF20CJ56KgSRBx+EO+4Ixx4wYGtpZNAgaNw4rx/FOVcPZfMcx2wz65uS9rqZ9Ys1Z3lU10ocv/tdmBYvjmfUwI0boawsBJGnnoKXXoJNm6BFCxg8eGuJpFevbQPXunWwYsXW6bPPKr6mpjVsCPvuC336bJ26dvXqMufqgkwljmwCxzxgv6gb88Q4G3PNbJ9YchqDuhQ4Nm+GPfaA3XaDpwt00/OqVTBjxtYSyYIFIb1TJ9h7b/j8863B4KuvMh+nVatQ9dW+/dbXdetgzpxQVZb4qrVpA/vtVzGYfOtb0KxZ5mM75wqvJg8A/ge4T9L/Rss/xrv5iM0LL4SqqquvLtx7tm4NQ4eGCUJJJxFEPvoIdtklXNjbtds2MCRe27aFJk0yv8eaNTBvXggiiWnixK2BqEGD0HCfHEz69IFdd/XSiXO1TTYljgaEcS2OjJLmAruY2QUx5y1v6lKJ4+yz4aGH4JNPth23vL7ZvDm0vSQHkzlzQqN+wk47hbvKDj44vPbrB40aFS/Pzm1Pql3iMLPNkl4Bdge+B7QHHsp/Ft2aNfDAAzB8eP0PGhBKGXvsEabvfndr+sqVMHduCCKvvgr//W+4PRnCeTnggBBEvv1tOPDAUGJyzhVOxsAhaU9geDR9BtwHYGaHFyZr258HHwxVN6NGFTsnxdWmDRxySJgSPv44BJD/+78wjR8fSiwNGoQqreRSSae0Y0I65/IlY1WVpM3AC8C5ZrYwSltkZrsVMH95UVeqqgYPDhfI+fO9Xr8qq1fDK69sDSQvvQRro243u3ffWiLZay/4+usQkNesCa+p85Wt23ln+OMfw11mzm1vqlNVdQph8KUZkv4DTCU8Oe5isGgRPPdc+CXtQaNqrVrBUUeFCWDDhlC1lQgkTz0F99xT+TFatNg6tWy5db5Dh63zM2bAscfCaafBn/8cunBxbnuXTeN4C2AoocrqCOAu4GEzezL+7OVHXShxjB0L48aFhuEuXarc3FXBLATjxYvTB4hmzUI1V1XWrYPrroP/+R8oKQnP11x8sTfQu+1DtZ/jSDnIjsBpwOlmdmRV29cWtT1wbN4cntvYc89wC6yrfRYtCgHj3/8OtybfemvFNhjn6qNMgSOnMcfN7Aszm5Bt0JA0RNJ8SQslXZ5mfVdJMyS9LmmupOOj9BGSZidNmyX1jdY9Gx0zsW6nXD5DbfTcc6Gksb03itdmu+0WOpF85JHQvnLooTBy5LadSjq3PcgpcOQiesL8FuA4oDcwXFLvlM2uAu6Pui85A7gVwMymmFnfqKuTM4H3zWx20n4jEuvN7NO4PkOhTJoUbik9+eRi58RVRgoPSb71FvzqV3DvvaHx/dZbQ5ctzm0vYgscwEBgoZktMrP1hMb1oSnbGJC4C78N8HGa4wyP9q2XVq8Ot+Gefvr28exGfdC8ebiJYe5cKC2FCy4Iz5a8+mqxc7atjRuLnQNXH8UZODoBHyUtL4nSko0FfiBpCfAYcFGa45wO3JuSNjGqpvq1lP4eJEmjJZVJKisvL6/WByiEBx8Mt5GefXaxc+Jy1atXuHtr6tRwG/WBB8KYMaFvr2IqL4cbbwzPtzRuHG5PPvpoOP98uOEG+Ne/4N13w51ocTEL52HJEli/Pr73ccWRU+N4TgeWTgWGmNkPo+UzgQPM7MKkbX4W5eFPkg4C7gC+ZWabo/UHAH8zs32T9ulkZksltSI8wX6Pmd1VWV5qc+P4YYeF7kXeecdvw63LVq0Kd8bddBPsuGN49mPkyOzu3MqHjRvhP/+BO+8MgWHDhlAaOvLIreO6vPtueCo/oaQkBJWePcONGYnu+Hv2DKNHlpRUfI8NG0Jnl8uXw6efhtdM859+WrG006EDdOwY+h5LN3XsGJ6Z8bvVapeadHJYXUuB5BtLO0dpyc4FhgCY2UuSmhK6NEm0W5xBSmnDzJZGr6sl/Z1QJVZp4KiJjRvjG3r1vffg+efDrZ4eNOq21q3Dcx6jRoVf9uecE8aL/9OfYP/9t70I58vbb4fOIu++O/wA6dABLroo5GPffStuaxYu/AsWbJ3efTe8vvBCxZ6PGzWC3XcPfYUlgsWKFenz0LhxuOjvvHMIAH37hvmddgrVesuXhxLZxx/DsmWhiu+TT8LdhMmksE9yQOnaFQYODFWBbdrk9dS5GogzcMwEekrqQQgYZwDfT9nmQ0LniZMk7Q00BcphS+eK3wO23PQoqSGwg5l9Fo1KeCIQS+fjZuEf8PPPYcqUeC7skyeH4555Zv6P7Ypjv/3Cj4G77oLLLoODDgoXz379QgkgMe25Z/VLIytXhuqxiRPD0/MNG8IJJ4RgccIJmX+1SyGwdOgQBu1KZhYu5olAkpjKy8MNAIceujUYJIJEYr5169z/PzZtCqWSREBJBJXk5bKyrXetSdC7dzifBx4YXnv1KlyJzlUUW1UVQHR77Y1ACXCnmY2XNA4oM7Np0V1WtwMtCQ3llyUeLJQ0GPiDmR2YdLwWwPNAo+iYTwM/M7NK72mpblXVNdfAr38NEybAj36U8+6V2rwZevQIX/4nnsjvsV3t8MUXodrotdfCRfD117d2i9KyJfTvXzGY7L575gvh5s3hKfaJE0PvyevWwT77hJLNiBHhAl4frVoVbjp46SV4+eUwJdqQ2rQJJZGDDgrTwIGhmjAuZmHavDlM6eZT05o0Cb0c1FV5eQCwrqpu4Ni0CYYMCV1YvPJK+DWZL888E+qf770Xzjgjf8d1tdemTaEtq6xs6zR7dggCEC6EqcEEQsl08uTwrM8OO4Tek88+O6zf3qo4zUKp6OWXtwaTefO2VnvtvXfFUsnee4e+yr78suL0xRfbpqVLX706/N2qe5mUQuebw4aF2+13q2M9/XngqGbj+PLloc62TZvwj96yZX7ydNZZMG1aKJ77yHfbrw0bwnMhiVJJWVnocyv5TiQp3BV19tnh4tO0afHyWxutXg0zZ24NJC+9lLk9Jp2WLUNATje1ahXapxo0CH+HBg2ym0+8fvpp+D+fMye81377hSAybFiYr+2B3wNHDe6qeuaZ0JneD36wtV2iJlatCqPqnXUW/PWvNTuWq3/Wr4c33gjBZPXq0MGi91+WPbNw48lLL4V2ml9yL0kAABdoSURBVNatMweGNm0KcyfXokWh14GHHw7DA5iF0sfJJ4cgctBB8d1AURMeOGp4O+7YsaGDuzvvrPkzF3fcAT/8YfhiH3hg1ds75+qP5ctD9zUPPwxPPx1+KOy0U+iVYNgwOOKIyodhLiQPHDUMHJs2heqCl18OxeJ99qn+sQ45JNyt8vbbtb+o6pyLz6pV8PjjIYg89lgoYbZqBccfH4LI0UeHKq+1a7dOX3+d2/J114UajurwwJGHBwCXLQvtHe3bhzs9WrTI/RgLF4YHrP7wB/jlL2ucJedcPfHNNzB9eggi//xn+HFZHSUl4RbwxPTEE+GaUx3FeACw3unYMQwOdOyx4RmPO+/M/RiTJ4dfED/4Qf7z55yru5o0CSWN448PbZ8vvhiqsxs3DgGgWbOKASF5OXm+EG02HjhydPTRoWfU8ePh8MNze3hv8+YQOI45xsfFds5lVlISqrRr65gv/txlNYwdG/6g550X7svP1jPPhH6DfNwN51xd5oGjGho2DA/uNWsG3/teaIzKxqRJ4RbAoamdyzvnXB3igaOaOnUKHcvNmwc/+UnV269cCf/4R3jq1x/gcs7VZR44amDIkHBn1O23hxJIZR54IJRMvJrKOVfXeeCooauvDj2Njh4d+tDJZNKk0G/O/vsXLGvOORcLDxw11KhR6OK6cePQ3pHosC7ZggWhm4FRo/yBP+dc3eeBIw+6dAm32c6ZAz/72bbrJ03yZzecc/WHB448OfFE+PnP4bbb4P77t6Zv2hQG9RkyJIxo5pxzdZ0Hjjz6/e9Dp4U//GHonRPCsxtLlnijuHOu/og1cEgaImm+pIWSLk+zvqukGZJelzQ3GjEQSd0lfS1pdjT9NWmfAZLmRce8Sao9rQaJ9o6GDUN7xzffhGqqHXeE73yn2Llzzrn8iC1wSCoBbgGOA3oDw6OhYpNdBdxvZv0IY5LfmrTuPTPrG01jktJvA34E9IymIXF9huro1i0M7zlrVniy3J/dcM7VN3GWOAYCC81skZmtB6YCqc9MG9A6mm8DfFzZASV1BFqb2csWuvW9Czg5v9muuaFD4ZJLQgBZt86rqZxz9UucnRx2Aj5KWl4CHJCyzVjgSUkXAS2Ao5LW9ZD0OrAKuMrMXoiOuSTlmGm7C5Q0GhgN0LVr1+p/imq69towTvmmTVvHjnbOufqg2I3jw4FJZtYZOB64W1IDYBnQNarC+hnwd0mtKznONsxsgpmVmllphw4d8p7xqjRuDC+8ADNm+LMbzrn6Jc4Sx1IgeaTkzlFasnOJ2ijM7CVJTYH2ZvYp8E2U/pqk94A9o/07V3HMWiMxoIpzztUncZY4ZgI9JfWQ1JjQ+D0tZZsPgSMBJO0NNAXKJXWIGteRtBuhEXyRmS0DVkk6MLqb6izgnzF+BueccyliK3GY2UZJFwJPACXAnWb2pqRxQJmZTQN+Dtwu6aeEhvJRZmaSDgXGSdoAbAbGmNnn0aHPByYBzYDHo8k551yB+Jjjzjnn0so05nixG8edc87VMR44nHPO5cQDh3POuZx44HDOOZcTDxzOOedy4oHDOedcTjxwOOecy4kHDueccznxwOGccy4nHjicc87lxAOHc865nHjgcM45l5M4x+Nwzjk2bNjAkiVLWLduXbGz4jJo2rQpnTt3plGjRllt74HDORerJUuW0KpVK7p37458OMxax8xYsWIFS5YsoUePHlnt41VVzrlYrVu3jnbt2nnQqKUk0a5du5xKhB44nHOx86BRu+X694k1cEgaImm+pIWSLk+zvqukGZJelzRX0vFR+tGSXpM0L3o9ImmfZ6Njzo6mneL8DM455yqKLXBEY4bfAhwH9AaGS+qdstlVwP1m1o8wJvmtUfpnwHfMbF9gJHB3yn4jzKxvNH0a12dwzhXelCnQvTs0aBBep0yp2fFWrFhB37596du3L7vssgudOnXasrx+/fpK9y0rK+Piiy+u8j0GDRpUs0zWMXE2jg8EFprZIgBJU4GhwFtJ2xjQOppvA3wMYGavJ23zJtBMUhMz+ybG/DrnimzKFBg9GtauDcsffBCWAUaMqN4x27Vrx+zZswEYO3YsLVu25NJLL92yfuPGjTRsmP5SWFpaSmnpNiOnbuPFF1+sXubqqDirqjoBHyUtL4nSko0FfiBpCfAYcFGa43wXmJUSNCZG1VS/VobKOUmjJZVJKisvL6/2h3DOFc6VV24NGglr14b0fBo1ahRjxozhgAMO4LLLLuPVV1/loIMOol+/fgwaNIj58+cD8Oyzz3LiiScCIeicc845DB48mN12242bbrppy/Fatmy5ZfvBgwdz6qmn0qtXL0aMGIGZAfDYY4/Rq1cvBgwYwMUXX7zluMkWL17MIYccQv/+/enfv3+FgHTttdey77770qdPHy6/PNT8L1y4kKOOOoo+ffrQv39/3nvvvfyeqAyKfTvucGCSmf1J0kHA3ZK+ZWabASTtA1wLHJO0zwgzWyqpFfAQcCZwV+qBzWwCMAGgtLTUYv4czrk8+PDD3NJrYsmSJbz44ouUlJSwatUqXnjhBRo2bMjTTz/Nr371Kx566KFt9nnnnXeYMWMGq1evZq+99uK8887b5tmH119/nTfffJNdd92Vgw8+mP/+97+Ulpby4x//mOeff54ePXowfPjwtHnaaaedeOqpp2jatCkLFixg+PDhlJWV8fjjj/PPf/6TV155hebNm/P5558DMGLECC6//HKGDRvGunXr2Lx5c/5PVBpxBo6lQJek5c5RWrJzgSEAZvaSpKZAe+BTSZ2Bh4GzzGxLGDWzpdHrakl/J1SJbRM4nHN1T9euoXoqXXq+nXbaaZSUlACwcuVKRo4cyYIFC5DEhg0b0u5zwgkn0KRJE5o0acJOO+3E8uXL6dy5c4VtBg4cuCWtb9++LF68mJYtW7LbbrtteU5i+PDhTJgwYZvjb9iwgQsvvJDZs2dTUlLCu+++C8DTTz/N2WefTfPmzQFo27Ytq1evZunSpQwbNgwID/EVSpxVVTOBnpJ6SGpMaPyelrLNh8CRAJL2BpoC5ZJ2AP4NXG5m/01sLKmhpPbRfCPgROCNGD+Dc66Axo+H6Nq4RfPmIT3fWrRosWX+17/+NYcffjhvvPEGjz76aMZnGpo0abJlvqSkhI0bN1Zrm0xuuOEGdt55Z+bMmUNZWVmVjffFElvgMLONwIXAE8DbhLun3pQ0TtJJ0WY/B34kaQ5wLzDKQoXghcAewG9SbrttAjwhaS4wm1CCuT2uz+CcK6wRI2DCBOjWDaTwOmFC9RvGs7Vy5Uo6dQpNsJMmTcr78ffaay8WLVrE4sWLAbjvvvsy5qNjx440aNCAu+++m02bNgFw9NFHM3HiRNZGDUCff/45rVq1onPnzjzyyCMAfPPNN1vWxy3W5zjM7DEz29PMdjez8VHab8xsWjT/lpkdbGZ9oltrn4zSrzGzFkm33PY1s0/N7CszG2Bm+5nZPmb2EzPbFOdncM4V1ogRsHgxbN4cXuMOGgCXXXYZV1xxBf369cuphJCtZs2aceuttzJkyBAGDBhAq1ataNOmzTbbnX/++UyePJk+ffrwzjvvbCkVDRkyhJNOOonS0lL69u3L9ddfD8Ddd9/NTTfdxH777cegQYP45JNP8p73dJRo8a/PSktLraysrNjZcG679Pbbb7P33nsXOxtFt2bNGlq2bImZccEFF9CzZ09++tOfFjtbW6T7O0l6zcy2uR/ZuxxxzrkCuP322+nbty/77LMPK1eu5Mc//nGxs1Rtxb4d1znntgs//elPa1UJoya8xOGccy4nHjicc87lxAOHc865nHjgcM45lxMPHM65eu3www/niSeeqJB24403ct5552XcZ/DgwSRu4T/++OP58ssvt9lm7NixW56nyOSRRx7hrbe2dgj+m9/8hqeffjqX7NdKHjicc/Xa8OHDmTp1aoW0qVOnZuxoMNVjjz3GDjvsUK33Tg0c48aN46ijjqrWsWoTvx3XOVcwl1wC0dAYedO3L9x4Y+b1p556KldddRXr16+ncePGLF68mI8//phDDjmE8847j5kzZ/L1119z6qmn8rvf/W6b/bt3705ZWRnt27dn/PjxTJ48mZ122okuXbowYMAAIDyjMWHCBNavX88ee+zB3XffzezZs5k2bRrPPfcc11xzDQ899BBXX301J554IqeeeirTp0/n0ksvZePGjey///7cdtttNGnShO7duzNy5EgeffRRNmzYwAMPPECvXr0q5Gnx4sWceeaZfPXVVwDcfPPNWwaTuvbaa7nnnnto0KABxx13HH/4wx9YuHAhY8aMoby8nJKSEh544AF23333ap9zL3E45+q1tm3bMnDgQB5//HEglDa+973vIYnx48dTVlbG3Llzee6555g7d27G47z22mtMnTqV2bNn89hjjzFz5swt60455RRmzpzJnDlz2HvvvbnjjjsYNGgQJ510Etdddx2zZ8+ucKFet24do0aN4r777mPevHls3LiR2267bcv69u3bM2vWLM4777y01WGJ7tdnzZrFfffdt2WUwuTu1+fMmcNll10GhO7XL7jgAubMmcOLL75Ix44da3ROvcThnCuYykoGcUpUVw0dOpSpU6dyxx13AHD//fczYcIENm7cyLJly3jrrbfYb7/90h7jhRdeYNiwYVu6Nj/ppJO2rHvjjTe46qqr+PLLL1mzZg3HHntspfmZP38+PXr0YM899wRg5MiR3HLLLVxyySVACEQAAwYM4B//+Mc2+xe7+3UvcWSQ73GPnXPFM3ToUKZPn86sWbNYu3YtAwYM4P333+f6669n+vTpzJ07lxNOOCFjd+pVGTVqFDfffDPz5s3jt7/9bbWPk5Domj1Tt+zF7n7dA0caiXGPP/gAzLaOe+zBw7m6qWXLlhx++OGcc845WxrFV61aRYsWLWjTpg3Lly/fUpWVyaGHHsojjzzC119/zerVq3n00Ue3rFu9ejUdO3Zkw4YNTEm6ULRq1YrVq1dvc6y99tqLxYsXs3DhQiD0cnvYYYdl/XmK3f26B440CjXusXOucIYPH86cOXO2BI4+ffrQr18/evXqxfe//30OPvjgSvfv378/p59+On369OG4445j//3337Lu6quv5oADDuDggw+u0JB9xhlncN1119GvX78K44E3bdqUiRMnctppp7HvvvvSoEEDxowZk/VnKXb3696tehoNGoSSRiopjBHgnMued6teN9SabtUlDZE0X9JCSZenWd9V0gxJr0uaK+n4pHVXRPvNl3RstsfMh0zjG8cx7rFzztU1sQUOSSXALcBxQG9guKTeKZtdRRhSth9hTPJbo317R8v7AEOAWyWVZHnMGivkuMfOOVfXxFniGAgsNLNFZrYemAoMTdnGgNbRfBvg42h+KDDVzL4xs/eBhdHxsjlmjRVr3GPn6qvtoUq8Lsv17xPncxydgI+SlpcAB6RsMxZ4UtJFQAsg8Sx+J+DllH07RfNVHTMvRozwQOFcPjRt2pQVK1bQrl07JBU7Oy6FmbFixYqcnu8o9gOAw4FJZvYnSQcBd0v6Vj4OLGk0MBqgqzdOOFc0nTt3ZsmSJZSXlxc7Ky6Dpk2b0rlz56y3jzNwLAW6JC13jtKSnUtow8DMXpLUFGhfxb5VHZPoeBOACRDuqqreR3DO1VSjRo3o0aNHsbPh8ijONo6ZQE9JPSQ1JjR2T0vZ5kPgSABJewNNgfJouzMkNZHUA+gJvJrlMZ1zzsUothKHmW2UdCHwBFAC3Glmb0oaB5SZ2TTg58Dtkn5KaCgfZaGV5k1J9wNvARuBC8xsE0C6Y8b1GZxzzm3LHwB0zjmXVqYHALeLwCGpHPig2PnIoD3wWbEzUQnPX814/mrG81czNc1fNzPrkJq4XQSO2kxSWbqIXlt4/mrG81cznr+aiSt/3smhc865nHjgcM45lxMPHMU3odgZqILnr2Y8fzXj+auZWPLnbRzOOedy4iUO55xzOfHA4ZxzLiceOApAUpdowKq3JL0p6SdpthksaaWk2dH0mwLncbGkedF7b/O0pIKbogG05krqX8C87ZV0XmZLWiXpkpRtCnr+JN0p6VNJbySltZX0lKQF0euOGfYdGW2zQNLIAubvOknvRH+/hyXtkGHfSr8LMeZvrKSlSX/D4zPsG/tgbhnyd19S3hZLmp1h30Kcv7TXlIJ9B83Mp5gnoCPQP5pvBbwL9E7ZZjDwryLmcTHQvpL1xwOPAwIOBF4pUj5LgE8IDyYV7fwBhwL9gTeS0v4IXB7NXw5cm2a/tsCi6HXHaH7HAuXvGKBhNH9tuvxl812IMX9jgUuz+Pu/B+wGNAbmpP4vxZW/lPV/An5TxPOX9ppSqO+glzgKwMyWmdmsaH418DZbxxepK4YCd1nwMrCDpI5FyMeRwHtmVtSeAMzseeDzlOShwORofjJwcppdjwWeMrPPzewL4CmiHqLjzp+ZPWlmG6PFlwm9SxdFhvOXjYIM5lZZ/hQGFfkecG++3zdblVxTCvId9MBRYJK6A/2AV9KsPkjSHEmPS9qnoBkLnUw+Kek1hbFMUqUbmKsYwe8MMv/DFvP8AexsZsui+U+AndNsU1vO4zmEEmQ6VX0X4nRhVJV2Z4Zqltpw/g4BlpvZggzrC3r+Uq4pBfkOeuAoIEktgYeAS8xsVcrqWYTqlz7AX4BHCpy9b5tZf8J47hdIOrTA718lha70TwIeSLO62OevAgt1ArXyXndJVxJ6nZ6SYZNifRduA3YH+gLLCNVBtdFwKi9tFOz8VXZNifM76IGjQCQ1IvyBp5jZP1LXm9kqM1sTzT8GNJLUvlD5M7Ol0eunwMOEKoFk2QzMFbfjgFlmtjx1RbHPX2R5ovouev00zTZFPY+SRgEnAiOiC8s2svguxMLMlpvZJjPbDNye4X2Lff4aAqcA92XaplDnL8M1pSDfQQ8cBRDVid4BvG1mf86wzS7RdkgaSPjbrChQ/lpIapWYJzSivpGy2TTgrOjuqgOBlUlF4kLJ+EuvmOcvyTQgcYfKSOCfabZ5AjhG0o5RVcwxUVrsJA0BLgNOMrO1GbbJ5rsQV/6S28yGZXjfYg/mdhTwjpktSbeyUOevkmtKYb6Dcbb8+7TlLoZvE4qMc4HZ0XQ8MAYYE21zIfAm4S6Rl4FBBczfbtH7zonycGWUnpw/AbcQ7miZB5QW+By2IASCNklpRTt/hAC2DNhAqCM+F2gHTAcWAE8DbaNtS4G/Je17DrAwms4uYP4WEuq2E9/Bv0bb7go8Vtl3oUD5uzv6bs0lXAA7puYvWj6ecBfRe4XMX5Q+KfGdS9q2GOcv0zWlIN9B73LEOedcTryqyjnnXE48cDjnnMuJBw7nnHM58cDhnHMuJx44nHPO5cQDh3PVJGmTKvbam7eeWiV1T+6Z1bnapGGxM+BcHfa1mfUtdiacKzQvcTiXZ9F4DH+MxmR4VdIeUXp3Sc9EnfhNl9Q1St9ZYXyMOdE0KDpUiaTbo/EWnpTULNr+4mgchrmSphbpY7rtmAcO56qvWUpV1elJ61aa2b7AzcCNUdpfgMlmth+hg8GbovSbgOcsdNDYn/DEMUBP4BYz2wf4EvhulH450C86zpi4PpxzmfiT485Vk6Q1ZtYyTfpi4AgzWxR1RPeJmbWT9BmhG40NUfoyM2svqRzobGbfJB2jO2HMhJ7R8i+BRmZ2jaT/AGsIPQA/YlHnjs4Vipc4nIuHZZjPxTdJ85vY2iZ5AqHfsP7AzKjHVucKxgOHc/E4Pen1pWj+RUJvrgAjgBei+enAeQCSSiS1yXRQSQ2ALmY2A/gl0AbYptTjXJz8l4pz1ddM0uyk5f+YWeKW3B0lzSWUGoZHaRcBEyX9AigHzo7SfwJMkHQuoWRxHqFn1nRKgHui4CLgJjP7Mm+fyLkseBuHc3kWtXGUmtlnxc6Lc3HwqirnnHM58RKHc865nHiJwznnXE48cDjnnMuJBw7nnHM58cDhnHMuJx44nHPO5eT/A+qbTjQ1oQLcAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eR6mMCfuOjbs"
      },
      "source": [
        "驗證準確度在第四個周期後，準確度就下降  \r\n",
        "發生**過度適配(Overfitting)**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lrqFWkGkO2g_"
      },
      "source": [
        "## 重新建立訓練模型\r\n",
        "為了防止過度適配，將訓練週期限制在第四個周期後即結束訓練"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kyBrn5wPO0_p",
        "outputId": "70a6435a-5465-4dc3-f93a-eb3947091a6f"
      },
      "source": [
        "model = models.Sequential()\r\n",
        "model.add(layers.Dense(16,activation='relu',input_shape=(10000,)))\r\n",
        "model.add(layers.Dense(16,activation='relu'))\r\n",
        "model.add(layers.Dense(1,activation='sigmoid'))\r\n",
        "\r\n",
        "model.compile(optimizer='rmsprop', #調整優化器參數lr的值\r\n",
        "       loss='binary_crossentropy', # 自行指定其他的損失函數與metrics函數\r\n",
        "       metrics=['accuracy'])\r\n",
        "\r\n",
        "model.fit(x_train,y_train,epochs=4,batch_size=512)\r\n",
        "results=model.evaluate(x_test,y_test)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/4\n",
            "49/49 [==============================] - 2s 21ms/step - loss: 0.5700 - accuracy: 0.7042\n",
            "Epoch 2/4\n",
            "49/49 [==============================] - 1s 21ms/step - loss: 0.2872 - accuracy: 0.9070\n",
            "Epoch 3/4\n",
            "49/49 [==============================] - 1s 21ms/step - loss: 0.2055 - accuracy: 0.9314\n",
            "Epoch 4/4\n",
            "49/49 [==============================] - 1s 21ms/step - loss: 0.1667 - accuracy: 0.9437\n",
            "782/782 [==============================] - 1s 2ms/step - loss: 0.3039 - accuracy: 0.8801\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ysutO-b1Psz8",
        "outputId": "fc4af346-a151-48c8-b789-2b07a022ff84"
      },
      "source": [
        "results"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[0.3038855493068695, 0.880079984664917]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 61
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "clF3sYgBQalR"
      },
      "source": [
        "將準確度提升到88%準確度"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gihTGhT7QooD"
      },
      "source": [
        "## 使用神經網路對新資料進行預測"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Al2fb27xQ0Zj",
        "outputId": "6b4b1a73-edc1-4d09-96da-f8cee0128aa9"
      },
      "source": [
        "model.predict(x_test)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[0.17708778],\n",
              "       [0.9999931 ],\n",
              "       [0.7763829 ],\n",
              "       ...,\n",
              "       [0.09998342],\n",
              "       [0.0706093 ],\n",
              "       [0.6768282 ]], dtype=float32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 62
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b36pPqiBRbhB"
      },
      "source": [
        "## 延伸實作"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G7MfCUgoRdqq"
      },
      "source": [
        "### 1. 使用一個隱藏層"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8yqHAKCoR_yb",
        "outputId": "bd5e5d3e-6f70-4558-ab7e-858b2f80e5fa"
      },
      "source": [
        "model = models.Sequential()\r\n",
        "model.add(layers.Dense(16,activation='relu',input_shape=(10000,)))\r\n",
        "model.add(layers.Dense(1,activation='sigmoid'))\r\n",
        "\r\n",
        "model.compile(optimizer='rmsprop', #調整優化器參數lr的值\r\n",
        "       loss='binary_crossentropy', # 自行指定其他的損失函數與metrics函數\r\n",
        "       metrics=['accuracy'])\r\n",
        "\r\n",
        "model.fit(x_train,y_train,epochs=4,batch_size=512)\r\n",
        "results=model.evaluate(x_test,y_test)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/4\n",
            "49/49 [==============================] - 1s 21ms/step - loss: 0.5348 - accuracy: 0.7770\n",
            "Epoch 2/4\n",
            "49/49 [==============================] - 1s 21ms/step - loss: 0.3008 - accuracy: 0.9041\n",
            "Epoch 3/4\n",
            "49/49 [==============================] - 1s 21ms/step - loss: 0.2262 - accuracy: 0.9263\n",
            "Epoch 4/4\n",
            "49/49 [==============================] - 1s 20ms/step - loss: 0.1889 - accuracy: 0.9379\n",
            "782/782 [==============================] - 2s 2ms/step - loss: 0.2853 - accuracy: 0.8850\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UR0TMnutRiE1"
      },
      "source": [
        "### 2. 使用三個隱藏層"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Omw_eUuGSAQi",
        "outputId": "81c70b2b-fb7b-4353-9a73-727af481ba60"
      },
      "source": [
        "model = models.Sequential()\r\n",
        "model.add(layers.Dense(16,activation='relu',input_shape=(10000,)))\r\n",
        "model.add(layers.Dense(16,activation='relu'))\r\n",
        "model.add(layers.Dense(16,activation='relu'))\r\n",
        "model.add(layers.Dense(1,activation='sigmoid'))\r\n",
        "\r\n",
        "model.compile(optimizer='rmsprop', #調整優化器參數lr的值\r\n",
        "       loss='binary_crossentropy', # 自行指定其他的損失函數與metrics函數\r\n",
        "       metrics=['accuracy'])\r\n",
        "\r\n",
        "model.fit(x_train,y_train,epochs=4,batch_size=512)\r\n",
        "results=model.evaluate(x_test,y_test)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/4\n",
            "49/49 [==============================] - 2s 24ms/step - loss: 0.5886 - accuracy: 0.7137\n",
            "Epoch 2/4\n",
            "49/49 [==============================] - 1s 24ms/step - loss: 0.3003 - accuracy: 0.9034\n",
            "Epoch 3/4\n",
            "49/49 [==============================] - 1s 24ms/step - loss: 0.1953 - accuracy: 0.9370\n",
            "Epoch 4/4\n",
            "49/49 [==============================] - 1s 25ms/step - loss: 0.1578 - accuracy: 0.9461\n",
            "782/782 [==============================] - 1s 2ms/step - loss: 0.3214 - accuracy: 0.8792\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V6cn-I2WRi7n"
      },
      "source": [
        "### 3. 使用32個單元"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5t6ZsY8tSBw5",
        "outputId": "dc53a833-812c-44c1-8784-c21502633e11"
      },
      "source": [
        "model = models.Sequential()\r\n",
        "model.add(layers.Dense(32,activation='relu',input_shape=(10000,)))\r\n",
        "model.add(layers.Dense(32,activation='relu'))\r\n",
        "model.add(layers.Dense(1,activation='sigmoid'))\r\n",
        "\r\n",
        "model.compile(optimizer='rmsprop', #調整優化器參數lr的值\r\n",
        "       loss='binary_crossentropy', # 自行指定其他的損失函數與metrics函數\r\n",
        "       metrics=['accuracy'])\r\n",
        "\r\n",
        "model.fit(x_train,y_train,epochs=4,batch_size=512)\r\n",
        "results=model.evaluate(x_test,y_test)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/4\n",
            "49/49 [==============================] - 2s 27ms/step - loss: 0.5341 - accuracy: 0.7427\n",
            "Epoch 2/4\n",
            "49/49 [==============================] - 1s 27ms/step - loss: 0.2482 - accuracy: 0.9128\n",
            "Epoch 3/4\n",
            "49/49 [==============================] - 1s 27ms/step - loss: 0.1810 - accuracy: 0.9355\n",
            "Epoch 4/4\n",
            "49/49 [==============================] - 1s 27ms/step - loss: 0.1488 - accuracy: 0.9478\n",
            "782/782 [==============================] - 1s 2ms/step - loss: 0.3076 - accuracy: 0.8810\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NRh23NwzRjDJ"
      },
      "source": [
        "### 4. 使用64個單元"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nHmNlUMvSBee",
        "outputId": "a4612115-3fc5-4174-90ec-decda63bdf9c"
      },
      "source": [
        "model = models.Sequential()\r\n",
        "model.add(layers.Dense(64,activation='relu',input_shape=(10000,)))\r\n",
        "model.add(layers.Dense(64,activation='relu'))\r\n",
        "model.add(layers.Dense(1,activation='sigmoid'))\r\n",
        "\r\n",
        "model.compile(optimizer='rmsprop', #調整優化器參數lr的值\r\n",
        "       loss='binary_crossentropy', # 自行指定其他的損失函數與metrics函數\r\n",
        "       metrics=['accuracy'])\r\n",
        "\r\n",
        "model.fit(x_train,y_train,epochs=4,batch_size=512)\r\n",
        "results=model.evaluate(x_test,y_test)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/4\n",
            "49/49 [==============================] - 2s 34ms/step - loss: 0.5190 - accuracy: 0.7370\n",
            "Epoch 2/4\n",
            "49/49 [==============================] - 2s 35ms/step - loss: 0.2287 - accuracy: 0.9156\n",
            "Epoch 3/4\n",
            "49/49 [==============================] - 2s 35ms/step - loss: 0.1688 - accuracy: 0.9405\n",
            "Epoch 4/4\n",
            "49/49 [==============================] - 2s 34ms/step - loss: 0.1356 - accuracy: 0.9523\n",
            "782/782 [==============================] - 2s 2ms/step - loss: 0.3246 - accuracy: 0.8787\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1snTRfbKRw4a"
      },
      "source": [
        "### 5. 使用mse損失函數"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3mntUv7hSBOF",
        "outputId": "8fb4dec1-9b75-400d-a8d8-7ab5c5269936"
      },
      "source": [
        "model = models.Sequential()\r\n",
        "model.add(layers.Dense(16,activation='relu',input_shape=(10000,)))\r\n",
        "model.add(layers.Dense(16,activation='relu'))\r\n",
        "model.add(layers.Dense(1,activation='sigmoid'))\r\n",
        "\r\n",
        "model.compile(optimizer='rmsprop', #調整優化器參數lr的值\r\n",
        "       loss='mse', # 自行指定其他的損失函數與metrics函數\r\n",
        "       metrics=['accuracy'])\r\n",
        "\r\n",
        "model.fit(x_train,y_train,epochs=4,batch_size=512)\r\n",
        "results=model.evaluate(x_test,y_test)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/4\n",
            "49/49 [==============================] - 2s 24ms/step - loss: 0.1895 - accuracy: 0.7375\n",
            "Epoch 2/4\n",
            "49/49 [==============================] - 1s 24ms/step - loss: 0.0837 - accuracy: 0.9076\n",
            "Epoch 3/4\n",
            "49/49 [==============================] - 1s 24ms/step - loss: 0.0590 - accuracy: 0.9335\n",
            "Epoch 4/4\n",
            "49/49 [==============================] - 1s 24ms/step - loss: 0.0476 - accuracy: 0.9467\n",
            "782/782 [==============================] - 1s 1ms/step - loss: 0.0861 - accuracy: 0.8849\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VMAbeBqmR2JK"
      },
      "source": [
        "### 6. 使用tanh啟動函數"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Y_DQPFGiSA7e",
        "outputId": "9d3851da-b10d-4665-ce81-2036bd60d925"
      },
      "source": [
        "model = models.Sequential()\r\n",
        "model.add(layers.Dense(16,activation='tanh',input_shape=(10000,)))\r\n",
        "model.add(layers.Dense(16,activation='tanh'))\r\n",
        "model.add(layers.Dense(1,activation='sigmoid'))\r\n",
        "\r\n",
        "model.compile(optimizer='rmsprop', #調整優化器參數lr的值\r\n",
        "       loss='binary_crossentropy', # 自行指定其他的損失函數與metrics函數\r\n",
        "       metrics=['accuracy'])\r\n",
        "\r\n",
        "model.fit(x_train,y_train,epochs=4,batch_size=512)\r\n",
        "results=model.evaluate(x_test,y_test)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/4\n",
            "49/49 [==============================] - 2s 23ms/step - loss: 0.5105 - accuracy: 0.7614\n",
            "Epoch 2/4\n",
            "49/49 [==============================] - 1s 23ms/step - loss: 0.2372 - accuracy: 0.9156\n",
            "Epoch 3/4\n",
            "49/49 [==============================] - 1s 23ms/step - loss: 0.1689 - accuracy: 0.9391\n",
            "Epoch 4/4\n",
            "49/49 [==============================] - 1s 23ms/step - loss: 0.1393 - accuracy: 0.9513\n",
            "782/782 [==============================] - 1s 2ms/step - loss: 0.3216 - accuracy: 0.8760\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EFV_yHJM4g-P"
      },
      "source": [
        "# 3-5 多元分類：數位新聞專欄\r\n",
        "建構神經網路，將路透社的新聞專欄分成46個完全不同的主題  \r\n",
        "**單標籤多類別分類(signal-label,multiclass classfication)**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zvmpDhYjVJYK"
      },
      "source": [
        "載入路透社資料集"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "24VEloE2VN3Q",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "8bc95f99-3350-4837-c540-24ea5a746168"
      },
      "source": [
        "from keras.datasets import reuters\r\n",
        "\r\n",
        "(train_data,train_labels),(test_data,test_labels) = reuters.load_data(num_words=10000)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/datasets/reuters.py:148: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray\n",
            "  x_train, y_train = np.array(xs[:idx]), np.array(labels[:idx])\n",
            "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/datasets/reuters.py:149: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray\n",
            "  x_test, y_test = np.array(xs[idx:]), np.array(labels[idx:])\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uYkfd96cVsnL",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ed17e115-5b8c-46a0-b3c0-eecfedda74f9"
      },
      "source": [
        "len(train_data)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "8982"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DNLHEL31Vu8r",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "27845c8f-5d81-4fd6-b2b5-8b13f4e86ddc"
      },
      "source": [
        "len(test_data)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2246"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0bI3yK4NVzvr",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4300805d-cfb0-4f5f-8990-808be96c329a"
      },
      "source": [
        "train_data[10]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[1,\n",
              " 245,\n",
              " 273,\n",
              " 207,\n",
              " 156,\n",
              " 53,\n",
              " 74,\n",
              " 160,\n",
              " 26,\n",
              " 14,\n",
              " 46,\n",
              " 296,\n",
              " 26,\n",
              " 39,\n",
              " 74,\n",
              " 2979,\n",
              " 3554,\n",
              " 14,\n",
              " 46,\n",
              " 4689,\n",
              " 4329,\n",
              " 86,\n",
              " 61,\n",
              " 3499,\n",
              " 4795,\n",
              " 14,\n",
              " 61,\n",
              " 451,\n",
              " 4329,\n",
              " 17,\n",
              " 12]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9zoZi86rXH3p"
      },
      "source": [
        "將數字還原成文字"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7BxDRAbUXH3w"
      },
      "source": [
        "word_index = reuters.get_word_index() #取得單字對應數字的字典\r\n",
        "reverse_word_index = dict(\r\n",
        "    [(value,key) for (key,value) in word_index.items()] #反轉為數字對應單字的字典\r\n",
        ")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        },
        "id": "ZvHy6ZT9XH3y",
        "outputId": "eed84967-26c7-49af-db51-d08acb1444d3"
      },
      "source": [
        "decord_newseire = ' '.join([reverse_word_index.get(i-3,'?') for i in train_data[0]])\r\n",
        "decord_newseire"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'? ? ? said as a result of its december acquisition of space co it expects earnings per share in 1987 of 1 15 to 1 30 dlrs per share up from 70 cts in 1986 the company said pretax net should rise to nine to 10 mln dlrs from six mln dlrs in 1986 and rental operation revenues to 19 to 22 mln dlrs from 12 5 mln dlrs it said cash flow per share this year should be 2 50 to three dlrs reuter 3'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AA6hGL3-XUzV",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "6fd4fa27-ddd2-42af-b7b0-6726a19c8698"
      },
      "source": [
        "train_labels[10] #第10筆訓練資料的標籤為第3類"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WdwNfYXRXfps"
      },
      "source": [
        "## 準備資料"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ccqP2VrDXizz"
      },
      "source": [
        "import numpy as np\r\n",
        "\r\n",
        "def vectorize_sequences(sequences,dimension=10000):\r\n",
        "  results=np.zeros((len(sequences),dimension))\r\n",
        "  for i,sequence in enumerate(sequences):\r\n",
        "    results[i,sequence]=1.\r\n",
        "  return results"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T68HIFbuYFHI"
      },
      "source": [
        "### 將訓練資料轉換為維度為10000的向量"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wUwQxgNiYFHg"
      },
      "source": [
        "x_train = vectorize_sequences(train_data) # 將訓練資料向量化"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Z-viCDgMYFHh"
      },
      "source": [
        "x_test = vectorize_sequences(test_data)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SPyR6fQ_Xh5G"
      },
      "source": [
        "## 對向量進行one-hot編碼"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ti_cuud1YhjV"
      },
      "source": [
        "from keras.utils.np_utils import to_categorical\r\n",
        "\r\n",
        "one_hot_train_labels = to_categorical(train_labels)\r\n",
        "one_hot_test_labels = to_categorical(test_labels)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ibrTo5v9bejN"
      },
      "source": [
        "## 建立神經網路\r\n",
        "輸出空間的維度會大很多"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MDxAjRs3bsrj"
      },
      "source": [
        "from keras import models\r\n",
        "from keras import layers\r\n",
        "\r\n",
        "model = models.Sequential()\r\n",
        "model.add(layers.Dense(64,activation='relu',input_shape=(10000,)))\r\n",
        "model.add(layers.Dense(64,activation='relu'))\r\n",
        "model.add(layers.Dense(46,activation='softmax'))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5yUGQPS-c3hO"
      },
      "source": [
        "### 編譯模型"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Een9CKjec8ws"
      },
      "source": [
        "model.compile(\r\n",
        "  optimizer='rmsprop',\r\n",
        "  loss='categorical_crossentropy',\r\n",
        "  metrics=['accuracy']\r\n",
        ")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CQnGeQqAdMLc"
      },
      "source": [
        "## 驗證"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Jx_uMyFfdOZA"
      },
      "source": [
        "x_val = x_train[:1000]\r\n",
        "partial_x_train = x_train[1000:]\r\n",
        "\r\n",
        "y_val = one_hot_train_labels[:1000]\r\n",
        "partial_y_train = one_hot_train_labels[1000:]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZVYnkB5ydsPu"
      },
      "source": [
        "進行訓練神經網路20個週期"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "maXoB6MFdwKN",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d2465356-33f6-4b9d-91b3-b82949bf7f17"
      },
      "source": [
        "history = model.fit(\r\n",
        "    partial_x_train,\r\n",
        "    partial_y_train,\r\n",
        "    epochs=20,\r\n",
        "    batch_size=512,\r\n",
        "    validation_data=(x_val,y_val))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/20\n",
            "16/16 [==============================] - 1s 55ms/step - loss: 3.0763 - accuracy: 0.4188 - val_loss: 1.6952 - val_accuracy: 0.6680\n",
            "Epoch 2/20\n",
            "16/16 [==============================] - 1s 35ms/step - loss: 1.4678 - accuracy: 0.7169 - val_loss: 1.3053 - val_accuracy: 0.7320\n",
            "Epoch 3/20\n",
            "16/16 [==============================] - 1s 34ms/step - loss: 1.0584 - accuracy: 0.7813 - val_loss: 1.1493 - val_accuracy: 0.7600\n",
            "Epoch 4/20\n",
            "16/16 [==============================] - 1s 35ms/step - loss: 0.8649 - accuracy: 0.8196 - val_loss: 1.0671 - val_accuracy: 0.7750\n",
            "Epoch 5/20\n",
            "16/16 [==============================] - 1s 35ms/step - loss: 0.6518 - accuracy: 0.8654 - val_loss: 1.0048 - val_accuracy: 0.7970\n",
            "Epoch 6/20\n",
            "16/16 [==============================] - 1s 34ms/step - loss: 0.5285 - accuracy: 0.8929 - val_loss: 0.9510 - val_accuracy: 0.8170\n",
            "Epoch 7/20\n",
            "16/16 [==============================] - 1s 35ms/step - loss: 0.4179 - accuracy: 0.9176 - val_loss: 0.9417 - val_accuracy: 0.8160\n",
            "Epoch 8/20\n",
            "16/16 [==============================] - 1s 35ms/step - loss: 0.3437 - accuracy: 0.9303 - val_loss: 0.9231 - val_accuracy: 0.8120\n",
            "Epoch 9/20\n",
            "16/16 [==============================] - 1s 34ms/step - loss: 0.2777 - accuracy: 0.9408 - val_loss: 0.9310 - val_accuracy: 0.8150\n",
            "Epoch 10/20\n",
            "16/16 [==============================] - 1s 35ms/step - loss: 0.2343 - accuracy: 0.9513 - val_loss: 0.9448 - val_accuracy: 0.8100\n",
            "Epoch 11/20\n",
            "16/16 [==============================] - 1s 35ms/step - loss: 0.1970 - accuracy: 0.9541 - val_loss: 0.9469 - val_accuracy: 0.8130\n",
            "Epoch 12/20\n",
            "16/16 [==============================] - 1s 36ms/step - loss: 0.1699 - accuracy: 0.9557 - val_loss: 0.9671 - val_accuracy: 0.8210\n",
            "Epoch 13/20\n",
            "16/16 [==============================] - 1s 35ms/step - loss: 0.1570 - accuracy: 0.9597 - val_loss: 0.9671 - val_accuracy: 0.8190\n",
            "Epoch 14/20\n",
            "16/16 [==============================] - 1s 36ms/step - loss: 0.1516 - accuracy: 0.9556 - val_loss: 0.9864 - val_accuracy: 0.8130\n",
            "Epoch 15/20\n",
            "16/16 [==============================] - 1s 36ms/step - loss: 0.1348 - accuracy: 0.9581 - val_loss: 1.0062 - val_accuracy: 0.8190\n",
            "Epoch 16/20\n",
            "16/16 [==============================] - 1s 36ms/step - loss: 0.1234 - accuracy: 0.9620 - val_loss: 1.0674 - val_accuracy: 0.7950\n",
            "Epoch 17/20\n",
            "16/16 [==============================] - 1s 35ms/step - loss: 0.1161 - accuracy: 0.9596 - val_loss: 1.0457 - val_accuracy: 0.8150\n",
            "Epoch 18/20\n",
            "16/16 [==============================] - 1s 36ms/step - loss: 0.1121 - accuracy: 0.9602 - val_loss: 1.0494 - val_accuracy: 0.8180\n",
            "Epoch 19/20\n",
            "16/16 [==============================] - 1s 35ms/step - loss: 0.1039 - accuracy: 0.9622 - val_loss: 1.0614 - val_accuracy: 0.8080\n",
            "Epoch 20/20\n",
            "16/16 [==============================] - 1s 37ms/step - loss: 0.1063 - accuracy: 0.9605 - val_loss: 1.1288 - val_accuracy: 0.8010\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ABHYH46vhagC"
      },
      "source": [
        "繪製訓練與驗證的損失函數"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "id": "5ZUH1tsxhagF",
        "outputId": "e6bc4b77-2246-4006-c941-7feb95ecb18c"
      },
      "source": [
        "import matplotlib.pyplot as plt\r\n",
        "\r\n",
        "loss = history.history['loss']\r\n",
        "val_loss = history.history['val_loss']\r\n",
        "\r\n",
        "epochs = range(1,len(loss)+1)\r\n",
        "\r\n",
        "plt.plot(epochs,loss,'bo',label='Training loss')\r\n",
        "plt.plot(epochs,val_loss,'b',label='Validation loss')\r\n",
        "plt.title('Training and validation loss')\r\n",
        "plt.xlabel('Epochs')\r\n",
        "plt.ylabel('Loss')\r\n",
        "plt.legend()\r\n",
        "\r\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZgU5dX38e8BBggOsruBMBDFBWUdQMUFNSaIBNyi8s6lEowKcYlooiQ8UWLC+zxJTF7D4xbcjSgkxuBK3BHRuAAigmJcAoqiIsomqAyc94+7Bpqhe6aHmerqmf59rquu7q6tT9f01Ol7qbvM3RERkcLVKOkAREQkWUoEIiIFTolARKTAKRGIiBQ4JQIRkQKnRCAiUuCUCKROmdlMMzu7rtdNkpktNbPvxLBfN7N9ouc3mdkvs1l3J96nzMwe39k4q9jvYDNbXtf7ldxrknQAkjwzW5/ysgXwNbA5en2+u0/Ndl/ufnwc6zZ07j6mLvZjZiXAf4Aidy+P9j0VyPpvKIVHiUBw9+KK52a2FPiRuz9ZeT0za1JxchGRhkNVQ5JRRdHfzK4ws4+B282sjZk9bGYrzeyL6HmnlG1mmdmPouejzGyOmV0TrfsfMzt+J9ftamazzWydmT1pZteb2d0Z4s4mxl+b2fPR/h43s/Ypy880s2VmtsrMJlRxfAaa2cdm1jhl3klmtjB6PsDM/mVmq81shZldZ2ZNM+zrDjP7Tcrrn0XbfGRmoyute4KZvWpma83sAzObmLJ4dvS42szWm9mhFcc2ZfvDzOwVM1sTPR6W7bGpipkdEG2/2swWm9nwlGVDzeyNaJ8fmtlPo/nto7/PajP73MyeMzOdl3JMB1yqswfQFugCnEf4ztweve4MbASuq2L7gcBbQHvgd8CtZmY7se49wMtAO2AicGYV75lNjP8H+CGwG9AUqDgxHQjcGO1/r+j9OpGGu78EfAkcU2m/90TPNwPjos9zKHAs8OMq4iaKYUgUz3HAvkDl9okvgbOA1sAJwFgzOzFadmT02Nrdi939X5X23RZ4BJgcfbY/Ao+YWbtKn2GHY1NNzEXAQ8Dj0XYXAVPNbL9olVsJ1YwtgYOAp6P5lwHLgQ7A7sAvAI17k2NKBFKdLcBV7v61u29091Xu/nd33+Du64BJwFFVbL/M3W92983AncCehH/4rNc1s85Af+BKd//G3ecAD2Z6wyxjvN3d/+3uG4G/Ar2j+acCD7v7bHf/GvhldAwyuRcYCWBmLYGh0TzcfZ67v+ju5e6+FPhzmjjSOS2Kb5G7f0lIfKmfb5a7v+7uW9x9YfR+2ewXQuJ4293/EsV1L7AE+H7KOpmOTVUOAYqB/4n+Rk8DDxMdG2ATcKCZ7eruX7j7/JT5ewJd3H2Tuz/nGgAt55QIpDor3f2rihdm1sLM/hxVnawlVEW0Tq0eqeTjiifuviF6WlzDdfcCPk+ZB/BBpoCzjPHjlOcbUmLaK3Xf0Yl4Vab3Ivz6P9nMmgEnA/PdfVkUR/eo2uPjKI7/SygdVGe7GIBllT7fQDN7Jqr6WgOMyXK/FfteVmneMqBjyutMx6bamN09NWmm7vcUQpJcZmbPmtmh0fzfA+8Aj5vZe2Y2PruPIXVJiUCqU/nX2WXAfsBAd9+VbVURmap76sIKoK2ZtUiZt3cV69cmxhWp+47es12mld39DcIJ73i2rxaCUMW0BNg3iuMXOxMDoXor1T2EEtHe7t4KuCllv9X9mv6IUGWWqjPwYRZxVbffvSvV72/dr7u/4u4jCNVGMwglDdx9nbtf5u7dgOHApWZ2bC1jkRpSIpCaakmoc18d1TdfFfcbRr+w5wITzaxp9Gvy+1VsUpsY7wOGmdnhUcPu1VT/f3IP8BNCwvlbpTjWAuvNbH9gbJYx/BUYZWYHRomocvwtCSWkr8xsACEBVVhJqMrqlmHfjwLdzez/mFkTMzsdOJBQjVMbLxFKD5ebWZGZDSb8jaZFf7MyM2vl7psIx2QLgJkNM7N9oragNYR2laqq4iQGSgRSU9cC3wI+A14E/pmj9y0jNLiuAn4DTCdc75DOTsfo7ouBCwgn9xXAF4TGzKpU1NE/7e6fpcz/KeEkvQ64OYo5mxhmRp/haUK1ydOVVvkxcLWZrQOuJPp1HW27gdAm8nzUE+eQSvteBQwjlJpWAZcDwyrFXWPu/g3hxH884bjfAJzl7kuiVc4ElkZVZGMIf08IjeFPAuuBfwE3uPsztYlFas7ULiP1kZlNB5a4e+wlEpGGTiUCqRfMrL+ZfdvMGkXdK0cQ6ppFpJZ0ZbHUF3sA9xMabpcDY9391WRDEmkYVDUkIlLgVDUkIlLg6l3VUPv27b2kpCTpMERE6pV58+Z95u4d0i2rd4mgpKSEuXPnJh2GiEi9YmaVryjfSlVDIiIFTolARKTAKRGIiBS4etdGICK5t2nTJpYvX85XX31V/cqSqObNm9OpUyeKioqy3kaJQESqtXz5clq2bElJSQmZ7yskSXN3Vq1axfLly+natWvW2xVE1dDUqVBSAo0ahcepuo23SI189dVXtGvXTkkgz5kZ7dq1q3HJrcGXCKZOhfPOgw3RLU2WLQuvAcrKMm8nIttTEqgfdubv1OBLBBMmbEsCFTZsCPNFRKQAEsH779dsvojkn1WrVtG7d2969+7NHnvsQceOHbe+/uabb6rcdu7cuVx88cXVvsdhhx1WJ7HOmjWLYcOG1cm+cqXBJ4LOlW/yV818Eam9um6Xa9euHQsWLGDBggWMGTOGcePGbX3dtGlTysvLM25bWlrK5MmTq32PF154oXZB1mMNPhFMmgQtWmw/r0WLMF9E6l5Fu9yyZeC+rV2urjtpjBo1ijFjxjBw4EAuv/xyXn75ZQ499FD69OnDYYcdxltvvQVs/wt94sSJjB49msGDB9OtW7ftEkRxcfHW9QcPHsypp57K/vvvT1lZGRWjND/66KPsv//+9OvXj4svvrjaX/6ff/45J554Ij179uSQQw5h4cKFADz77LNbSzR9+vRh3bp1rFixgiOPPJLevXtz0EEH8dxzz9XtAatCbI3FZrY3cBewO+GG2lPc/U+V1hkMPAD8J5p1v7tfXZdxVDQIT5gQqoM6dw5JQA3FIvGoql2urv/vli9fzgsvvEDjxo1Zu3Ytzz33HE2aNOHJJ5/kF7/4BX//+9932GbJkiU888wzrFu3jv3224+xY8fu0Of+1VdfZfHixey1114MGjSI559/ntLSUs4//3xmz55N165dGTlyZLXxXXXVVfTp04cZM2bw9NNPc9ZZZ7FgwQKuueYarr/+egYNGsT69etp3rw5U6ZM4Xvf+x4TJkxg8+bNbKh8EGMUZ6+hcuAyd59vZi2BeWb2hLu/UWm959w91gq1sjKd+EVyJZftcj/4wQ9o3LgxAGvWrOHss8/m7bffxszYtGlT2m1OOOEEmjVrRrNmzdhtt9345JNP6NSp03brDBgwYOu83r17s3TpUoqLi+nWrdvW/vkjR45kypQpVcY3Z86crcnomGOOYdWqVaxdu5ZBgwZx6aWXUlZWxsknn0ynTp3o378/o0ePZtOmTZx44on07t27VsemJmKrGnL3Fe4+P3q+DngT6BjX+4lIfshlu9wuu+yy9fkvf/lLjj76aBYtWsRDDz2UsS99s2bNtj5v3Lhx2vaFbNapjfHjx3PLLbewceNGBg0axJIlSzjyyCOZPXs2HTt2ZNSoUdx11111+p5VyUkbgZmVAH2Al9IsPtTMXjOzmWbWI8P255nZXDObu3LlyhgjFZHaSqpdbs2aNXTsGH5r3nHHHXW+//3224/33nuPpUuXAjB9+vRqtzniiCOYGjWOzJo1i/bt27Prrrvy7rvvcvDBB3PFFVfQv39/lixZwrJly9h9990599xz+dGPfsT8+fPr/DNkEnsiMLNi4O/AJe6+ttLi+UAXd+8F/C8Zbkbu7lPcvdTdSzt0SHtfBRHJE2VlMGUKdOkCZuFxypT4q2cvv/xyfv7zn9OnT586/wUP8K1vfYsbbriBIUOG0K9fP1q2bEmrVq2q3GbixInMmzePnj17Mn78eO68804Arr32Wg466CB69uxJUVERxx9/PLNmzaJXr1706dOH6dOn85Of/KTOP0Mmsd6z2MyKgIeBx9z9j1msvxQodffPMq1TWlrqujGNSG69+eabHHDAAUmHkbj169dTXFyMu3PBBRew7777Mm7cuKTD2kG6v5eZzXP30nTrx1YisHCd863Am5mSgJntEa2HmQ2I4lkVV0wiIrVx880307t3b3r06MGaNWs4//zzkw6pTsTZa2gQcCbwupktiOb9AugM4O43AacCY82sHNgInOFxFlFERGph3LhxeVkCqK3YEoG7zwGqHP3I3a8DrosrBhERqV6Dv7JYRESqpkQgIlLglAhERAqcEoGI5L2jjz6axx57bLt51157LWPHjs24zeDBg6noaj506FBWr169wzoTJ07kmmuuqfK9Z8yYwRtvbBsZ58orr+TJJ5+sSfhp5dNw1UoEIpL3Ro4cybRp07abN23atKwGfoMwamjr1q136r0rJ4Krr76a73znOzu1r3ylRCAiee/UU0/lkUce2XoTmqVLl/LRRx9xxBFHMHbsWEpLS+nRowdXXXVV2u1LSkr47LNwneqkSZPo3r07hx9++NahqiFcI9C/f3969erFKaecwoYNG3jhhRd48MEH+dnPfkbv3r159913GTVqFPfddx8ATz31FH369OHggw9m9OjRfP3111vf76qrrqJv374cfPDBLFmypMrPl/Rw1Q3+nsUiUrcuuQQWLKh+vZro3RuuvTbz8rZt2zJgwABmzpzJiBEjmDZtGqeddhpmxqRJk2jbti2bN2/m2GOPZeHChfTs2TPtfubNm8e0adNYsGAB5eXl9O3bl379+gFw8sknc+655wLwX//1X9x6661cdNFFDB8+nGHDhnHqqadut6+vvvqKUaNG8dRTT9G9e3fOOussbrzxRi655BIA2rdvz/z587nhhhu45ppruOWWWzJ+vqSHq1aJQETqhdTqodRqob/+9a/07duXPn36sHjx4u2qcSp77rnnOOmkk2jRogW77rorw4cP37ps0aJFHHHEERx88MFMnTqVxYsXVxnPW2+9RdeuXenevTsAZ599NrNnz966/OSTTwagX79+Wweqy2TOnDmceeaZQPrhqidPnszq1atp0qQJ/fv35/bbb2fixIm8/vrrtGzZssp9Z0MlAhGpkap+ucdpxIgRjBs3jvnz57Nhwwb69evHf/7zH6655hpeeeUV2rRpw6hRozIOP12dUaNGMWPGDHr16sUdd9zBrFmzahVvxVDWtRnGevz48Zxwwgk8+uijDBo0iMcee2zrcNWPPPIIo0aN4tJLL+Wss86qVawqEYhIvVBcXMzRRx/N6NGjt5YG1q5dyy677EKrVq345JNPmDlzZpX7OPLII5kxYwYbN25k3bp1PPTQQ1uXrVu3jj333JNNmzZtHToaoGXLlqxbt26Hfe23334sXbqUd955B4C//OUvHHXUUTv12ZIerlolAhGpN0aOHMlJJ520tYqoYtjm/fffn7333ptBgwZVuX3fvn05/fTT6dWrF7vtthv9+/ffuuzXv/41AwcOpEOHDgwcOHDryf+MM87g3HPPZfLkyVsbiQGaN2/O7bffzg9+8APKy8vp378/Y8aM2anPVXEv5Z49e9KiRYvthqt+5plnaNSoET169OD4449n2rRp/P73v6eoqIji4uI6uYFNrMNQx0HDUIvknoahrl/yZhhqERGpH5QIREQKnBKBiGSlvlUjF6qd+TspEYhItZo3b86qVauUDPKcu7Nq1SqaN29eo+3Ua0hEqtWpUyeWL1/OypUrkw5FqtG8eXM6depUo22UCESkWkVFRXTt2jXpMCQmqhoSESlwSgQiIgVOiUBEpMApEYiIFDglAhGRAqdEICJS4JQIREQKnBKBiEiBUyIQESlwSgQiIgVOiUBEpMApEYiIFDglAhGRAhdbIjCzvc3sGTN7w8wWm9lP0qxjZjbZzN4xs4Vm1jeueEREJL04h6EuBy5z9/lm1hKYZ2ZPuPsbKescD+wbTQOBG6NHERHJkdhKBO6+wt3nR8/XAW8CHSutNgK4y4MXgdZmtmdcMYmIyI5y0kZgZiVAH+ClSos6Ah+kvF7OjskCMzvPzOaa2VzdIUlEpG7FngjMrBj4O3CJu6/dmX24+xR3L3X30g4dOtRtgCIiBS7WRGBmRYQkMNXd70+zyofA3imvO0XzREQkR+LsNWTArcCb7v7HDKs9CJwV9R46BFjj7iviiklERHYUZ6+hQcCZwOtmtiCa9wugM4C73wQ8CgwF3gE2AD+MMR4REUkjtkTg7nMAq2YdBy6IKwYREameriwWESlwSgQiIgVOiUBEpMApEYiIFDglAhGRAqdEICJS4JQIREQKnBKBiEiBUyIQESlwSgQiIgVOiUBEpMApEYiIFDglAhGRAqdEICJS4JQIREQKnBKBiEiBUyIQESlwBZUI3nsv6QhERPJPwSSCu+6CffaB119POhIRkfxSMIlg2DBo2RImTkw6EhGR/FIwiaBtWxg3Du6/H159NeloRETyR8EkAgiJoE0buPLKpCMREckfBZUIWrWCn/4UHn4YXnop6WhERPJDQSUCgIsugnbtVCoQEalQcImgZUu44gp4/HGYMyfpaEREkldwiQDgggtg991VKhARgQJNBC1awM9/Ds88EyYRkUJWkIkA4PzzYa+94Je/BPekoxERSU7BJoLmzWHCBHj+eXjiiaSjERFJTsEmAoBzzoHOnVUqEJHCVtCJoFmzkARefhkeeSTpaEREkhFbIjCz28zsUzNblGH5YDNbY2YLoimRPjxnnw3duoUeRCoViEghirNEcAcwpJp1nnP33tF0dYyxZFRUFJLAq6/CP/6RRAQiIsmKLRG4+2zg87j2X5fKyqB7d7jqKtiyJeloRERyK+k2gkPN7DUzm2lmPTKtZGbnmdlcM5u7cuXKOg+iSZMwPPWiRfC3v9X57kVE8pp5jBXjZlYCPOzuB6VZtiuwxd3Xm9lQ4E/uvm91+ywtLfW5c+fWeaybN0PPnqFEsGgRNG5c528hIpIYM5vn7qXpliVWInD3te6+Pnr+KFBkZu2TiqdxY/jVr2DJErjnnqSiEBHJvcQSgZntYWYWPR8QxbIqqXgATj4ZevUKCaG8PMlIRERyJ87uo/cC/wL2M7PlZnaOmY0xszHRKqcCi8zsNWAycIbHWU+VhUaN4Oqr4d13wz2ORUQKQVZtBGa2C7DR3beYWXdgf2Cmu2+KO8DK4mojqOAOAwbAypXw739D06axvZWISM7URRvBbKC5mXUEHgfOJFwn0OCYhVLBsmVw221h3tSpUFISSgwlJeG1iEhD0STL9czdN5jZOcAN7v47M1sQZ2BJGjIEDj0UfvObMAzFhRfChg1h2bJlcN554XlZWXIxiojUlWxLBGZmhwJlQMWoPA22g6UZ/PrX8OGHcNll25JAhQ0bwsilIiINQbaJ4BLg58A/3H2xmXUDGvQtXY45Bo46Cr74Iv3y99/PbTwiInHJqmrI3Z8FngUws0bAZ+5+cZyBJa2ireCoo9Iv79w5t/GIiMQlqxKBmd1jZrtGvYcWAW+Y2c/iDS15Rx4JB+1wTXS41eWkSbmPR0QkDtlWDR3o7muBE4GZQFdCz6EG7+abw2Pr1qGU0KULTJmihmIRaTiy7TVUZGZFhERwnbtvMrOCGL3/kENg6FB48UVYvRp23TXpiERE6la2JYI/A0uBXYDZZtYFWBtXUPnmV7+Czz+Ha69NOhIRkbqXVSJw98nu3tHdh3qwDDg65tjyRmkpjBgBf/gDvP120tGIiNStbBuLW5nZHyvuCWBmfyCUDgrGf/93uG9B//7w0ENJRyMiUneyrRq6DVgHnBZNa4Hb4woqHx1wAMybB/vsA8OHh5veb96cdFQiIrWXbSL4trtf5e7vRdOvgG5xBpaPSkpgzhwYPToMPzF0KKxKdOBsEZHayzYRbDSzwytemNkgYGM8IeW35s3h1ltDF9JZs6BfP5g/P+moRER2XraJYAxwvZktNbOlwHXA+bFFVQ+ce24oHWzZAocdBrcXVEWZiDQk2fYaes3dewE9gZ7u3gc4JtbI6oH+/UO7weGHh+qi88+Hr79OOioRkZqp0R3KovsMV1w/cGkM8dQ7HTrAP/8J48eH6qIjjoAPPkg6KhGR7NXmVpVWZ1HUc02ahO6l998PS5ZA377w1FNJRyUikp3aJIKCGGKiJk46CV55BXbbDb77Xfjtb8OtL0VE8lmVicDM1pnZ2jTTOmCvHMVYr+y3H7z0EpxySqguOuUUWFswg3GISH1UZSJw95buvmuaqaW7ZztgXcEpLobp08OQFA8+CAMGwBtvJB2ViEh6takakiqYwaWXhraCL74IyeCvf006KhGRHSkRxOyoo8IFZz17wumnhy6nt922432QRUSSokSQAx07hquQr78eNm6Ec84J88aNg3//O+noRCTfLV0Kv/sdPBPTneKVCHKkaVP48Y/h9dfh2Wfhe9+D664LjcvHHRe6npaXJx2liOSLDz6AP/4RBg6Erl3hiivg8cfjeS8lghwzC/dCnjYt/KF/8xt4663Qu6ikBK6+Gj76KOkoRSQJH30Ef/oTDBoEnTvDZZeFH4i//S289164XikO5vWso3tpaanPnTs36TDqVHk5PPII3HgjPPZYuEDtxBNDCWLw4JA8RKRh+uQTuO++0NNwzpxw7VGvXnDaaWHaZ5+6eR8zm+fupWmXKRHkl3fegZtuCoPYff457L8/jB0LZ50FrVsnHZ2I1IWVK0N18PTpoap4yxY48MDQoeS008L/fV1TIqiHNm4M3U1vvDFcoNaiBYwcGb4ogwdDUVHSEYrkRnk5vPBCqB9v2hQ6ddp+2nXXpCOs3jffwGefwcyZ4eT/9NPhxlbdu4f/6dNPhx494o1BiSBhU6fChAnw/vuh3m/SJCgry377efNCQrj33tDttFUrGDYsVB8NGRIuYBNpSNavDyf+Bx4I1aarVkHjxunvClhcvGNyqDy1bZt9FevmzbBpU5jKy7c937QJ1q2DNWtg9eqaPW5MuXvLt7+97Zd/z565q/pVIkjQ1Klw3nnbXzfQokUYqbQmyQDCPp58Ev7xj3Df5FWroFkz+M53QlIYPjyMcyRSH61YEb7XDzwQLsT8+mto0wZOOAFGjAg97Zo1Cw2qy5dnnlasCFUtqZo3D122i4q2P7GnO+HvzCmxWbNQdduqVdWPhx4aBqVMot0vkURgZrcBw4BP3f2gNMsN+BMwFNgAjHL3au/1Vd8SQUkJLFu24/wuXULf4J1VXg7PPw8zZoTEsGxZ+HINGhSSwoknhl8eIvnKHRYvDsOwPPAAvPxymN+1azjxjxgRvs81rQYtL4ePP94+OXz4YZg2bw77a9IkPKZO2cwrLt7xBN+qVUg0+S6pRHAksB64K0MiGApcREgEA4E/ufvA6vZb3xJBo0bpf2GY7firZWe5w8KFISHMmAGvvRbmH3RQGBH1xBOhTx/1PpLkVfyAeeCBML33Xpjfv/+2k3+PHvquxiGxqiEzKwEezpAI/gzMcvd7o9dvAYPdfUVV+6xviSCuEkFV/vOf8E82YwY891xIOHvvHf7JhgwJw16oXUHi8uWXoXrmo4+2f1y6NFT5fP55aPQ99tjwnfz+92EvjWUcu6oSQZIjiHYEUu/ltTyaV2UiqG8mTUrfRjBpUnzv2bUrXHJJmD77LNS7zpgBt9wSrmYuKgp1lccdF+6b0K9faIgTqcqXX+54cv/oox3npRt2vVmzcLKvqO//7nehZcvcfwZJL8kSwcPA/7j7nOj1U8AV7r7Dz30zOw84D6Bz5879lqX7iZ3HattrqK589VW4YOWJJ8L06qthfps2cMwxITEcdxx065b72CQZ7qHTwYoVoV59xYptU+XX69fvuH3z5rDnnuEkX/kx9Xnr1qruSZqqhiStlStDUb0iMVTca7lbt21J4ZhjQqKQ3NqyJfx9Kn5xf/RRODHXZjyqin2mnuQ//jj0lKmsuDicxCumPfZIf8LXCb7+yNeqoQeBC81sGqGxeE11SUDqVocOcMYZYXIPI6FWJIV77oE//zk0dpeWhqRw9NHhhFBcvG1q2lQngppwD3XkqSf4dFNtT/rpmEG7dttO7gccsP1JPvWkrzakwhJnr6F7gcFAe+AT4CqgCMDdb4q6j14HDCF0H/1humqhylQiyI1Nm0J3vorE8NJL6S/madJk+8SQadpll1AnvNtu4URTMXXoEPaRpE2bws2DvvginKQrpsqvV68O627eHKYtW7Z/zPS84nHTJvj003CVaWVt226rTsk07b57SLwiO0MXlEmtrVkTEsPq1aGueGemdN1lzUIyqEgMu+++faJIndq0CfvYuDE0vleevvwy/fzU5RUn99ST/Lp1mT+3Waj+aNMmPDZtGhrWGzcOpaV0j5mWNWkSEmHlE/yee9aPfuhSv+Vr1ZDUI61aheqhneUeTsaffhqqPdJNn3wShuT++ONwVWlljRrt3LUXjRuHEkmLFuGE3rZtGHagZ89tr1On1HmtWqlHlTR8SgSSE2bhZNy1a5iq4h5KIJUTxcqVoRtiixaZp4oTfuqkAfpEqqZEIHmnojqmdet4huMVke3pDmUiIgVOiUBEpMApEYiIFDglAhGRAqdEUA9MnRpGMW3UKDxOnZp0RCLSkKjXUJ6rfIezZcvCa0hm4DoRaXhUIshzEyZsP4Q1hNcTJiQTj4g0PEoEee7992s2X0SkppQI8lznzjWbLyJSU0oEeW7SpDBMQqq473AmIoVFiSDPlZXBlCnhHsdm4XHKFDUUi0jdUa+heqCsTCd+EYmPSgQiIgVOiUBEpMApEYiIFDglAhGRAqdEUAA0VpGIVEW9hho4jVUkItVRiaCB01hFIlIdJYIGTmMViUh1lAgaOI1VJCLVUSJo4DRWkYhUR4mggdNYRSJSHfUaKgAaq0hEqqISgYhIgVMiEBEpcEoEIiIFTolAsqJhKkQaLjUWS7U0TIVIw+IxivwAAArASURBVBZricDMhpjZW2b2jpmNT7N8lJmtNLMF0fSjOOORnaNhKkQatthKBGbWGLgeOA5YDrxiZg+6+xuVVp3u7hfGFYfUnoapEGnY4iwRDADecff33P0bYBowIsb3k5homAqRhi3ORNAR+CDl9fJoXmWnmNlCM7vPzPZOtyMzO8/M5prZ3JUrV8YRq1RBw1SINGxJ9xp6CChx957AE8Cd6VZy9ynuXurupR06dMhpgKJhKkQaujh7DX0IpP7C7xTN28rdV6W8vAX4XYzxSC1omAqRhivOEsErwL5m1tXMmgJnAA+mrmBme6a8HA68GWM8kiBdhyCSv2IrEbh7uZldCDwGNAZuc/fFZnY1MNfdHwQuNrPhQDnwOTAqrngkOboOQSS/mbsnHUONlJaW+ty5c5MOQ2qgpCSc/Cvr0gWWLs11NCKFyczmuXtpumVJNxZLAdB1CCL5TYlAYqfrEETymxKBxE7XIYjkNyUCiV1dXIegXkci8dHoo5ITtbkOQb2OROKlEoHkPY1+KhIvJQLJe+p1JBIvJQLJe+p1JBIvJQLJe3XR60iNzSKZKRFI3qttr6OKxuZly8B9W2OzkoFIoCEmpMHTEBciGmJCCpwam0WqpkQgDV5dNDarjUEaMiUCafBq29isNgZp6JQIpMGrbWNzXVzQphKF5DM1FotUo1GjUBKozAy2bKl++8pDZEAokei+z5JLaiwWqYXatjGoRCH5TolApBq1bWOoba8ltVFI3JQIRKpR2zYGlSgk3ykRiGShrCxcfLZlS3isSd1+QyhR1DaRKBHlOXevV1O/fv1cpL65+273Ll3czcLj3Xdnv22XLu7hFL791KVLbra/+273Fi2237ZFi+w/Q223r9jHzh4/CYC5nuG8mviJvaaTEoEUmtqeSM3SJwKz7Lav74moYh+1SSRJb18XlAhE6rkkSxS1TSRJJ6KkSzT5kMjclQhEClptT0RJlwiSTiRJb18XicS96kSgxmKRBq62vZ5q29hd2+1r2+uqto3tSW+fi1u1KhGIFIDa9HqqbSJJOhHVNpEkvX1ORs/NVFTI10lVQyKFpzZ15EnX8SddNVcBtRGISCFLutdPkomsQlWJQIPOiYjkualTQ5vA+++HKqVJk2o+YGFVg841qYsgRUQkPmVl8Y5Uq8ZiEZECF2siMLMhZvaWmb1jZuPTLG9mZtOj5S+ZWUmc8YiIyI5iSwRm1hi4HjgeOBAYaWYHVlrtHOALd98H+H/Ab+OKR0RE0ouzRDAAeMfd33P3b4BpwIhK64wA7oye3wcca2YWY0wiIlJJnImgI/BByuvl0by067h7ObAGaFd5R2Z2npnNNbO5K1eujClcEZHCVC96Dbn7FGAKgJmtNLNlCYeUSXvgs6SDqEK+xwf5H6Piqx3FVzu1ia9LpgVxJoIPgb1TXneK5qVbZ7mZNQFaAauq2qm7d6jLIOuSmc3N1E83H+R7fJD/MSq+2lF8tRNXfHFWDb0C7GtmXc2sKXAG8GCldR4Ezo6enwo87fXtCjcRkXouthKBu5eb2YXAY0Bj4DZ3X2xmVxMudX4QuBX4i5m9A3xOSBYiIpJDsbYRuPujwKOV5l2Z8vwr4AdxxpBjU5IOoBr5Hh/kf4yKr3YUX+3EEl+9G2tIRETqloaYEBEpcEoEIiIFTomghsxsbzN7xszeMLPFZvaTNOsMNrM1ZrYgmq5Mt68YY1xqZq9H773DmN0WTI7GeFpoZn1zGNt+KcdlgZmtNbNLKq2T8+NnZreZ2admtihlXlsze8LM3o4e22TY9uxonbfN7Ox068QU3+/NbEn0N/yHmbXOsG2V34cY45toZh+m/B2HZti2yjHJYoxvekpsS81sQYZtYz1+mc4pOf3+ZbpRgaYMd/KBPYG+0fOWwL+BAyutMxh4OMEYlwLtq1g+FJgJGHAI8FJCcTYGPga6JH38gCOBvsCilHm/A8ZHz8cDv02zXVvgveixTfS8TY7i+y7QJHr+23TxZfN9iDG+icBPs/gOvAt0A5oCr1X+f4orvkrL/wBcmcTxy3ROyeX3TyWCGnL3Fe4+P3q+DniTHYfOyHcjgLs8eBFobWZ7JhDHscC77p74leLuPpvQhTlV6lhYdwInptn0e8AT7v65u38BPAEMyUV87v64h6FZAF4kXLSZiAzHLxvZjElWa1XFF41vdhpwb12/bzaqOKfk7PunRFAL0bDZfYCX0iw+1MxeM7OZZtYjp4GBA4+b2TwzOy/N8mzGgcqFM8j8z5fk8auwu7uviJ5/DOyeZp18OZajCaW8dKr7PsTpwqjq6rYMVRv5cPyOAD5x97czLM/Z8at0TsnZ90+JYCeZWTHwd+ASd19bafF8QnVHL+B/gRk5Du9wd+9LGAL8AjM7MsfvX63oavPhwN/SLE76+O3AQzk8L/tam9kEoByYmmGVpL4PNwLfBnoDKwjVL/loJFWXBnJy/Ko6p8T9/VMi2AlmVkT4g0119/srL3f3te6+Pnr+KFBkZu1zFZ+7fxg9fgr8g1D8TpXNOFBxOx6Y7+6fVF6Q9PFL8UlFlVn0+GmadRI9lmY2ChgGlEUnix1k8X2Ihbt/4u6b3X0LcHOG9036+DUBTgamZ1onF8cvwzklZ98/JYIaiuoTbwXedPc/Zlhnj2g9zGwA4ThXOZheHca3i5m1rHhOaFBcVGm1B4Gzot5DhwBrUoqguZLxV1iSx6+S1LGwzgYeSLPOY8B3zaxNVPXx3Whe7MxsCHA5MNzdN2RYJ5vvQ1zxpbY7nZThfbMZkyxO3wGWuPvydAtzcfyqOKfk7vsXV0t4Q52AwwlFtIXAgmgaCowBxkTrXAgsJvSAeBE4LIfxdYve97UohgnR/NT4jHD3uHeB14HSHB/DXQgn9lYp8xI9foSktALYRKhnPYdwb4yngLeBJ4G20bqlwC0p244G3ommH+YwvncI9cMV38ObonX3Ah6t6vuQo/j+En2/FhJOantWji96PZTQU+bdXMYXzb+j4nuXsm5Oj18V55Scff80xISISIFT1ZCISIFTIhARKXBKBCIiBU6JQESkwCkRiIgUOCUCkYiZbbbtR0ats5EwzawkdeRLkXwS660qReqZje7eO+kgRHJNJQKRakTj0f8uGpP+ZTPbJ5pfYmZPR4OqPWVmnaP5u1u4P8Br0XRYtKvGZnZzNOb842b2rWj9i6Ox6Bea2bSEPqYUMCUCkW2+Valq6PSUZWvc/WDgOuDaaN7/Ane6e0/CgG+To/mTgWc9DJrXl3BFKsC+wPXu3gNYDZwSzR8P9In2MyauDyeSia4sFomY2Xp3L04zfylwjLu/Fw0O9rG7tzOzzwjDJmyK5q9w9/ZmthLo5O5fp+yjhDBu/L7R6yuAInf/jZn9E1hPGGV1hkcD7onkikoEItnxDM9r4uuU55vZ1kZ3AmHsp77AK9GImCI5o0Qgkp3TUx7/FT1/gTBaJkAZ8Fz0/ClgLICZNTazVpl2amaNgL3d/RngCqAVsEOpRCRO+uUhss23bPsbmP/T3Su6kLYxs4WEX/Ujo3kXAbeb2c+AlcAPo/k/AaaY2TmEX/5jCSNfptMYuDtKFgZMdvfVdfaJRLKgNgKRakRtBKXu/lnSsYjEQVVDIiIFTiUCEZECpxKBiEiBUyIQESlwSgQiIgVOiUBEpMApEYiIFLj/D1uLhpzOpTwjAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-fm8Fa0nhagI"
      },
      "source": [
        "繪製訓練與驗證的準確度"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "id": "Hu-h_h8rhagJ",
        "outputId": "f58a014e-e6f3-45ab-f5ce-8aa46e682487"
      },
      "source": [
        "import matplotlib.pyplot as plt\r\n",
        "\r\n",
        "plt.clf()\r\n",
        "acc = history.history['accuracy']\r\n",
        "val_acc = history.history['val_accuracy']\r\n",
        "\r\n",
        "plt.plot(epochs,acc,'bo',label='Training acc')\r\n",
        "plt.plot(epochs,val_acc,'b',label='Validation acc')\r\n",
        "plt.title('Training and validation accuracy')\r\n",
        "plt.xlabel('Epochs')\r\n",
        "plt.ylabel('Accuracy')\r\n",
        "plt.legend()\r\n",
        "\r\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZgU1fX/8fdhWAcIyKbIjqKoP2UbMWJUNJqgInw1JopEQZMgqDGaqMHggusTt2iMRsUYY4AIGg0ucYmiUePKiICKG+ogKCD7IuvA+f1xa6Bpumd6ll5m+vN6nn66uupW9emannv63qq6Ze6OiIjkr3rZDkBERLJLiUBEJM8pEYiI5DklAhGRPKdEICKS55QIRETynBKB7MLMnjGzETVdNpvMrMTMjknDdt3M9o6m7zGzK1IpW4X3GW5m/6lqnCLlMV1HUDeY2bqYl4XAJmBr9Pocd5+c+ahyh5mVAD939xdqeLsO9HD3eTVV1sy6Al8ADdy9tCbiFClP/WwHIDXD3ZuVTZdX6ZlZfVUukiv0fcwN6hqq48xsoJktNLPfmtli4AEz283MnjKzpWa2MpruGLPOf83s59H0SDP7n5ndEpX9wsyOq2LZbmb2ipmtNbMXzOwuM5uUJO5UYrzWzF6LtvcfM2sTs/wMM5tvZsvNbFw5++cQM1tsZgUx804ysznRdH8ze8PMVpnZIjO708waJtnW38zsupjXl0TrfG1mZ8eVPcHM3jWzNWa2wMzGxyx+JXpeZWbrzOzQsn0bs/4AM5thZquj5wGp7ptK7udWZvZA9BlWmtm0mGVDzWxW9Bk+M7NB0fyduuHMbHzZ39nMukZdZD8zsy+BF6P5j0R/h9XRd+SAmPWbmNmt0d9zdfQda2Jm/zazX8Z9njlmdlKizyrJKRHkhz2AVkAXYBTh7/5A9LozsAG4s5z1DwE+BtoANwH3m5lVoew/gLeB1sB44Ixy3jOVGE8HzgLaAQ2BiwHMbH/g7mj7e0bv15EE3P0t4Fvg6Ljt/iOa3gpcFH2eQ4HvA+eWEzdRDIOieI4FegDxxye+Bc4EWgInAGPM7P+iZUdEzy3dvZm7vxG37VbAv4E7os/2B+DfZtY67jPssm8SqGg/TyR0NR4Qbeu2KIb+wN+BS6LPcARQkmx/JHAksB/ww+j1M4T91A6YCcR2Zd4C9AMGEL7HlwLbgAeBn5YVMrNeQAfCvpHKcHc96tiD8A95TDQ9ENgMNC6nfG9gZczr/xK6lgBGAvNilhUCDuxRmbKESqYUKIxZPgmYlOJnShTj5TGvzwWejaavBKbELGsa7YNjkmz7OuCv0XRzQiXdJUnZC4F/xbx2YO9o+m/AddH0X4Hfx5TbJ7Zsgu3eDtwWTXeNytaPWT4S+F80fQbwdtz6bwAjK9o3ldnPQHtChbtbgnL3lsVb3vcvej2+7O8c89m6lxNDy6hMC0Ki2gD0SlCuMbCScNwFQsL4c6b/3+rCQy2C/LDU3TeWvTCzQjO7N2pqryF0RbSM7R6Js7hswt3XR5PNKll2T2BFzDyABckCTjHGxTHT62Ni2jN22+7+LbA82XsRfv2fbGaNgJOBme4+P4pjn6i7ZHEUxw2E1kFFdooBmB/3+Q4xs5eiLpnVwOgUt1u27flx8+YTfg2XSbZvdlLBfu5E+JutTLBqJ+CzFONNZPu+MbMCM/t91L20hh0tizbRo3Gi94q+01OBn5pZPWAYoQUjlaREkB/iTw37DbAvcIi7f4cdXRHJuntqwiKglZkVxszrVE756sS4KHbb0Xu2TlbY3ecSKtLj2LlbCEIX00eEX53fAX5XlRgILaJY/wCeADq5ewvgnpjtVnQq39eErpxYnYGvUogrXnn7eQHhb9YywXoLgL2SbPNbQmuwzB4JysR+xtOBoYTusxaEVkNZDMuAjeW814PAcEKX3XqP60aT1CgR5KfmhOb2qqi/+ap0v2H0C7sYGG9mDc3sUODENMX4T2CwmX0vOrB7DRV/1/8B/IpQET4SF8caYJ2Z9QTGpBjDw8BIM9s/SkTx8Tcn/NreGPW3nx6zbCmhS6Z7km0/DexjZqebWX0zOxXYH3gqxdji40i4n919EaHv/s/RQeUGZlaWKO4HzjKz75tZPTPrEO0fgFnAaVH5IuCUFGLYRGi1FRJaXWUxbCN0s/3BzPaMWg+HRq03oop/G3Arag1UmRJBfrodaEL4tfUm8GyG3nc44YDrckK//FRCBZBIlWN09w+A8wiV+yJCP/LCClZ7iHAA80V3XxYz/2JCJb0WuC+KOZUYnok+w4vAvOg51rnANWa2lnBM4+GYddcD1wOvWThb6btx214ODCb8ml9OOHg6OC7uVFW0n88AthBaRd8QjpHg7m8TDkbfBqwGXmZHK+UKwi/4lcDV7NzCSuTvhBbZV8DcKI5YFwPvATOAFcCN7Fx3/R04kHDMSapAF5RJ1pjZVOAjd097i0TqLjM7Exjl7t/Ldiy1lVoEkjFmdrCZ7RV1JQwi9AtPq2g9kWSibrdzgQnZjqU2UyKQTNqDcGrjOsI58GPc/d2sRiS1lpn9kHA8ZQkVdz9JOdQ1JCKS59QiEBHJc7Vu0Lk2bdp4165dsx2GiEit8s477yxz97aJltW6RNC1a1eKi4uzHYaISK1iZvFXo2+nriERkTynRCAikueUCERE8pwSgYhInlMiEBHJc0oEIiI5bvJk6NoV6tULz5MnV7RG5SgRiEjOS3dFmG7ViX/yZBg1CubPB/fwPGpUze4DJQIRqVB1K+JsV4S1Of5x42D9+p3nrV8f5teYbN8rs7KPfv36uYhUzqRJ7l26uJuF50mTKrduYaF7qMbCo7Aw9W1Ud/0uXXZet+zRpUt+xG+WeH2z1NYvAxR7kno16xV7ZR9KBJKPslmRV7ciy3ZFmO/xlykvEahrSCTNaqJbIptdC19+Wbn5Nb1+5/i7PVcwv6bfP9vxX389FBbuPK+wMMyvMckyRK4+1CKQTMvmr3H32v+LNNtdO7U9/rJtVPU7WAZ1DYlUTbYrIffsV+TZ7mMv20ZtPcZR3fhrihKBSBVl+9d4TcSQCxVZtivC2h5/TSgvEdS6O5QVFRW5hqGWTKlXL1Sd8cxg27aK1+/aNfTpx+vSBUpKUouh7BhBbD9/YSFMmADDh6e+jXHjQr92586hfznVdaVuMLN33L0o0TIdLJY6rzoHa3PhQN/w4aHS79IlJKAuXSqXBMq2UVISkldJiZKA7EyJQOq06p5xU92KvCYq8bLtqCKXdFHXkNRpNdU1o24Vqe3K6xpSIpA6rbp9/CJ1hY4RSN6qbh+/SD5QIpCcV52DvRm5KlOkllMikJxW3YO9NXWwVqQu0zECyWk1cbBXRHSMQGqx6g74JSIVUyKQnKaDvSLpp0QgOU0He0XST4lAcpoO9oqknxKBpF11b8yi4RVE0qt+tgOQui1+5Myy0z9BFbpIrlCLQNKqurdJFJH0UyKQtNLpnyK5T4lA0kqnf4rkPiUCSSud/imS+9KaCMxskJl9bGbzzGxsguVdzGy6mc0xs/+aWcd0xiOZp9M/RXJf2sYaMrMC4BPgWGAhMAMY5u5zY8o8Ajzl7g+a2dHAWe5+Rnnb1VhDIiKVl62xhvoD89z9c3ffDEwBhsaV2R94MZp+KcFyERFJs3Qmgg7AgpjXC6N5sWYDJ0fTJwHNzax1/IbMbJSZFZtZ8dKlS9MSrIhIvsr2weKLgSPN7F3gSOArYGt8IXef4O5F7l7Utm3bTMcoIlKnpTMRfAV0inndMZq3nbt/7e4nu3sfYFw0b1UaY5IqqO4QESKS29KZCGYAPcysm5k1BE4DnogtYGZtzKwshsuAv6YxHqmC6t4hTERyX9oSgbuXAucDzwEfAg+7+wdmdo2ZDYmKDQQ+NrNPgN0BnV2eYzREhEjdp1tVSrnq1QstgXhmYTRQEakddKtKqTINESFS9ykRSLk0RIRI3adEIOXSEBEidZ9uTCMVGj5cFb9IXaYWgYhInlMiEBHJc0oEIiJ5TolARCTPKRGIiOQ5JYI8oEHjRKQ8On20jisbNK5svKCyQeNAp4SKSKAWQR2nQeNEpCJKBHXcl19Wbr6I5B8lgjpOg8aJSEWUCOo4DRonIhVRIqjjNGiciFREZw3lAQ0aJyLlUYtARCTPKRGIiOQ5JQIRkTynRCAikueUCERE8pwSgYhInlMiEBHJc0oEIiJ5TomgFtD9BEQknXRlcY7T/QREJN3UIshxup+AiKSbEkGO0/0ERCTdlAhynO4nICLppkSQ43Q/ARFJNx0sznFlB4THjQvdQZ07hySgA8V138aNMGMGvPoq/O9/8O670LgxtGix66Nly4rnFxaGe1KIxDN3z3YMlVJUVOTFxcXZDiMvlZZCQYEqk3RZtQpefz1U/K++GpLA5s1h2f77Q//+sHUrrF4dHqtW7ZhevRq2bSt/+w0bQs+ecOCBcNBBO5733LPm/6ZLlsCcOfDeezueFyyAX/0Kxo4N36Nc5Q5r14bP8M034XnJEli5Evr1gyOPDAm5tjGzd9y9KNEytQikQosXh3/eiRNDZdOoUfhHKHuOnS5vXtOm0LFjaNV06RKeW7XKTGLZuhW+/RaaN8+dRPbVVzt+7b/6aqgs3aF+fSgqggsugMMPh8MOg9aty9+We/h88ckhNmEsWwZz58LLL+98Lcpuu+2cGA48EP7f/4NmzSr+DOvXh23GVvhz5sDSpTvKtG8fttmuHVx+OTz3XPgudelStf1WVWvWwMKFOyr22Eo+ft7Gjcm307QpHHssDB4Mxx8fPl9tpxaBJLVlC9x5J4wfDxs2wM9/Dm3awKZN4R8l2XOyZWvXhudYTZuGhFD2KEsQZdMdOkCDBonj27x513/mZK+XLQtJrGVL2GefXR89eqRW8VXVunWha++113ZU/l98sWMfDBgA3/teqPgPOWTX40I1beXKUGnHVuDvvRfiLNO9+84JYu+9oaRk5wp/3rwdLZEmTUICiU8qbdqE5e4hAZ17brg48p574LTT0vs5ISSAq6+GO+4IrdpYBQUhQe2++47nskfs63btwvfjf/+Dp54KjwULwjaKikJSOOEE6Ns3fLZcVF6LQIlAEnrxRfjlL8OvvUGD4I9/DBVmdbiHCvnLL8Nj/vxdp7/5Zud1zELXRZcu4Z9xxYodlfyqVYnfp2nTxP/MzZuHiuyTT8Ij/hTcPffcOTGUTXfvHrpV4j/LypUVJ6Cy6dhrQdq2DRX+4YeHyr9379AKyLZt28LfIb5L55NPdu52MoO99tq1wu/ePbUun88/h5/+FN54A844I/zY+M530vN5Jk2CSy8Nf4Ozz4Zjjtn5u9GqVdUqbvewb8qSwptvhnl77BESwuDB4b3S+eOispQIJGULFsBvfgOPPALdusHtt8OJJ2auO2XDhhBDomTxzTfhHzdRJR87r2nT1N5r/Xr47LMdiSH2sWzZjnL16oV90blzSD5llXv8r8uysm3bJv5l2b596OffZ5/c6Z5KxYYN8OGH4dd/165wwAGp7+NkSkvDSQ/XXBOS/KRJoVVUU959F84/PxxzOeSQkGyKElaBNWPpUnjmGfj3v+HZZ0MrpGFDOOqoHa2Fbt3S9/6pUCKQCm3aBLfeGv45t22Dyy6DSy4Jzf18tGIFfPrpzslhwYLQn56s+2D33UNffq52DeSi118PrYP58+GKK8IxhOq0jpYvD9u4997QJXXjjTBiRGb/Jlu27NyF9MknYf5++4VW4MEHh8cBB2S2JVheIsDda9WjX79+LjXrqafc997bHdxPPtn9iy+yHZHkk9Wr3c88M3z/Dj3U/bPPKr+N0lL3e+5xb9XKvaDA/Ve/cl+5suZjrYpPPnG/7Tb3H/zAvUWL8DnBvUkT98MOc7/wQvfJk0O5bdvSFwdQ7Enq1bRW2sAg4GNgHjA2wfLOwEvAu8Ac4PiKtqlEUHPmzXMfPDh8C3r2dP/Pf7IdkeSzhx4KFWXz5u4PPph6pfj66+59+4bv8ZFHus+Zk9Ywq2XrVvePP3afNCkkqwEDQkIoSw4tW7ofc4z7ZZe5P/aY+8KFNffeWUkEQAHwGdAdaAjMBvaPKzMBGBNN7w+UVLRdJYLqW7fOfdw494YN3Zs1c7/5ZvdNm7IdlYj7/PnuRxwRaqZTT3VfsSJ52cWL3UeODGU7dHCfMiW9v6jTZcsW91mz3O+7z33UKPc+fdzr19+RHNq3dx8yxP3aa93ff7/q71NeIkhnD1V/YJ67fw5gZlOAocDcmDIOlJ0v0AL4Oo3x5D13+Oc/w8HgBQtC3+yNN4azZURyQefO4Yy1m26CK68MxxAmTgwXcZXZsgXuuguuuiocyB47Nlx5n0tn6FRG/frQq1d4/PznYd6GDTBrVriosOzxxBPhf/WAA9IQRLIMUd0HcArwl5jXZwB3xpVpD7wHLARWAv2SbGsUUAwUd+7cueopMY999JH70UeHXxi9erm/8kq2IxIp39tvu/fo4W7m/rvfuW/e7P7ii+4HHBC+x4MGhW6WfLFqlfvatVVfn3JaBNk+v2EY8Dd37wgcD0w0s11icvcJ7l7k7kVt27bNeJC1WWlp+NXfqxfMnBl+SRUXh7MXRHLZwQeH7+zPfgY33BCuUzj66HDa7+OPw9NPV//altqkRYv0tXrSmQi+AjrFvO4YzYv1M+BhAHd/A2gMtEljTHll9uxwDvXYseE85g8/DFd15sLFSyKpaNYM7rsPHn00XBB49dXwwQcwZEjtuhYj11WYCMzsxES/0lMwA+hhZt3MrCFwGvBEXJkvge9H77MfIREsRapl06ZwTnZRURhb5ZFHwj/SHntkOzKRqjn55HCV+5VX5u+1LemUSgV/KvCpmd1kZj1T3bC7lwLnA88BHwIPu/sHZnaNmQ2Jiv0G+IWZzQYeAkZGfVl1SiZvPv/WW2G8k+uug2HDwj/PKaek7/1EpPZL6cpiM/sOoT//LMKZPg8AD7n72vSGt6vadmVx/M3nIQwoNmFCzd5TYP36cEXl7beHgdruvTeMjCgiAuVfWZxSl4+7rwH+CUwhnOlzEjDTzH5ZY1HWUZm4+fxLL4VBv267Dc45J/ShKgmISKpSOUYwxMz+BfwXaAD0d/fjgF6Erh0pRzpvPr96daj4jz46HDh76SW4++70jOQoInVXKueP/Ai4zd1fiZ3p7uvN7GfpCavu6Nw5DKiVaH51/PvfIQksWhQuELvmmvSPYS8idVMqXUPjgbfLXphZEzPrCuDu09MSVR1S0zefX7YsXBE8eHC4ycobb8AttygJiEjVpZIIHgFi74a6NZonKRg+PBwY7tIldN906VK1A8Xu8PDD4d61U6eGy+tnzgzj24uIVEcqXUP13X1z2Qt33xxdFyApGj686mcIlZaGawD+8Ad4++1wbcD06eHgsIhITUilRbA05rx/zGwosKyc8lIDVq0KXT577RXu67pyZTgQ/MYbSgIiUrNSaRGMBiab2Z2AAQuAM9MaVR77/PNwk+377w83Eh84MNxm74QTdOcrEUmPChOBu38GfNfMmkWv16U9qjzjHobb/cMfYNq0UOEPGwYXXQR9+mQ7OhGp61IafszMTgAOABpbNNKTu1+TxrjyQnz//267wW9/C+edF64OFhHJhAoTgZndAxQCRwF/Idxn4O1yV5JyrVoFf/kL/OlP4cKyHj3gz3+GM8+Epk2zHZ2I5JtUWgQD3P0gM5vj7leb2a3AM+kOrC5S/7+I5KJUEsHG6Hm9me0JLCeMNyQp2rYNxo8PF5Gp/19Eck0qieBJM2sJ3AzMJIw+el9ao6pDvv02dPk89lh4vuEG9f+LSG4pNxFEN6SZ7u6rgEfN7Cmgsbuvzkh0tdyXX8LQoTBnTjggfOGFuquSiOSechOBu28zs7uAPtHrTcCmTARW273xBpx0EmzYAE89Bccdl+2IREQSS+UQ5XQz+5GZfsumauLEcCC4WTN4800lARHJbakkgnMIg8xtMrM1ZrbWzNakOa5aaevWcKP4M8+Eww4Lt43cb79sRyUiUr5UrixunolAaru1a8PAck8+CaNHh9NEGzTIdlQiIhVL5YKyIxLNj79RTT774gsYMgQ+/DBcF3DeedmOSEQkdamcPnpJzHRjoD/wDnB0WiKqZV59FU4+OQwX8eyzcMwx2Y5IRKRyUukaOjH2tZl1Am5PW0S1yP33w5gx0K1b6BLaZ59sRyQiUnlVGdhgIZDXh0C3boVf/xp+/nM46qhwZpCSgIjUVqkcI/gT4WpiCImjN+EK47y0enW4Ucyzz8IFF8Ctt0L9lMZwFRHJTalUYcUx06XAQ+7+WpriyWnz5sGJJ4bne++FUaOyHZGISPWlkgj+CWx0960AZlZgZoXuvj69oeWWd9+F738/DBHx/PPhgjERkbogpSuLgSYxr5sAL6QnnNzkDr/8JTRqBDNmKAmISN2SSiJoHHt7ymi6MH0h5Z5LLoHXXoPFi+Hoo2Hy5GxHJCJSc1JJBN+aWd+yF2bWD9iQvpByy8SJYeTQMvPnh2MDSgYiUleYu5dfwOxgYArwNWDAHsCp7v5O+sPbVVFRkRcXF1dcsIa0bQvLlu06v0sXKCnJWBgiItViZu+4e1GiZalcUDbDzHoC+0azPnb3LTUZYK7asiVxEoBwrwERkbqgwq4hMzsPaOru77v7+0AzMzs3/aFl31//mnxZ586Zi0NEJJ1SOUbwi+gOZQC4+0rgF+kLKTds2ADXXAM9ekCTJjsvKywM9x8WEakLUkkEBbE3pTGzAqBh+kLKDXfeCV9/DX/5C9x3XzgmYBaeJ0wIQ06LiNQFqVxQ9iww1czujV6fAzyTvpCyb/Vq+P3vYdAgOCIahFsVv4jUVakkgt8Co4DR0es5hDOH6qxbboEVK+CGG7IdiYhI+lXYNeTu24C3gBLCvQiOBj5Mb1jZs2QJ3HYb/OQn0KdPtqMREUm/pC0CM9sHGBY9lgFTAdz9qMyElh033AAbN8K112Y7EhGRzCivRfAR4df/YHf/nrv/CdhamY2b2SAz+9jM5pnZ2ATLbzOzWdHjEzNblWg7mTJ/PtxzD5x1lu4vICL5o7xjBCcDpwEvmdmzhKuLrZzyO4nOLroLOJZwM5sZZvaEu88tK+PuF8WU/yWQ1c6Y8ePDmUFXXZXNKEREMitpi8Ddp7n7aUBP4CXgQqCdmd1tZj9IYdv9gXnu/rm7byYkkqHllB8GPJR66DVr7lz4+9/Djec7dsxWFCIimZfKweJv3f0f0b2LOwLvEs4kqkgHYEHM64XRvF2YWRegG/BikuWjzKzYzIqXLl2awltX3hVXQNOmcNlladm8iEjOqtQ9i919pbtPcPfv13AcpwH/LLv5TYL3neDuRe5e1LZt2xp+63CPgcceg9/8Btq0qfHNi4jktKrcvD5VXwGdYl53jOYlchpZ7Bb63e9CAvj1r7MVgYhI9qQzEcwAephZNzNrSKjsn4gvFI1suhvwRhpjSerFF+GFF0IyaN48GxGIiGRX2hKBu5cC5wPPES5Ae9jdPzCza8xsSEzR04ApXtGNEdISYzgm0KkTjBmT6XcXEckNqQwxUWXu/jTwdNy8K+Nej09nDOV5/HF4++0wsFzjxtmKQkQku9LZNZTTtm6FceNg331hxIhsRyMikj1pbRHkssmTw7UDDz8M9fN2L4iI5GmLYPPmcPVw377wox9lOxoRkezKy9/CEyaEG8/fcw/Uy8tUKCKyQ95Vg99+C9ddB0ceCT9IZaAMEZE6Lu9aBH/8Y7jnwL/+FQaYExHJd3nVIlixAm66CU48EQ49NNvRiIjkhrxKBDfdBGvWwPXXZzsSEZHckTeJYNEiuOMOOP10OPDAbEcjIpI78iYR3HUXbNkCV1+d7UhERHJL3iSCq64KA8zttVe2IxERyS15kwgaNIDDD892FCIiuSdvEoGIiCSmRCAikueUCERE8pwSgYhInlMiEBHJc0oEIiJ5TolARCTPKRGIiOQ5JQIRkTynRCAikueUCERE8pwSgYhInlMiEBHJc0oEIiJ5TolARCTPKRGIiOQ5JQIRkTynRCAikueUCERE8pwSgYhInlMiEBHJc0oEIiJ5TolARCTPKRGIiOQ5JQIRkTynRCAikueUCERE8lxaE4GZDTKzj81snpmNTVLmJ2Y218w+MLN/pDMeERHZVf10bdjMCoC7gGOBhcAMM3vC3efGlOkBXAYc5u4rzaxduuIREZHE0pYIgP7APHf/HMDMpgBDgbkxZX4B3OXuKwHc/Zs0xiMi1bRlyxYWLlzIxo0bsx2KJNG4cWM6duxIgwYNUl4nnYmgA7Ag5vVC4JC4MvsAmNlrQAEw3t2fjd+QmY0CRgF07tw5LcGKSMUWLlxI8+bN6dq1K2aW7XAkjruzfPlyFi5cSLdu3VJeL9sHi+sDPYCBwDDgPjNrGV/I3Se4e5G7F7Vt2zbDIYpImY0bN9K6dWslgRxlZrRu3brSLbZ0JoKvgE4xrztG82ItBJ5w9y3u/gXwCSExiEiOUhLIbVX5+6QzEcwAephZNzNrCJwGPBFXZhqhNYCZtSF0FX2exphERCRO2hKBu5cC5wPPAR8CD7v7B2Z2jZkNiYo9Byw3s7nAS8Al7r48XTGJSGZNngxdu0K9euF58uTqbW/58uX07t2b3r17s8cee9ChQ4ftrzdv3lzuusXFxVxwwQUVvseAAQOqF2QtZO6e7RgqpaioyIuLi7Mdhkhe+vDDD9lvv/1SKjt5MowaBevX75hXWAgTJsDw4dWPZfz48TRr1oyLL754+7zS0lLq10/nOTC1Q6K/k5m94+5Ficpn+2CxiNRR48btnAQgvB43rmbfZ+TIkYwePZpDDjmESy+9lLfffptDDz2UPn36MGDAAD7++GMA/vvf/zJ48GAgJJGzzz6bgQMH0r17d+64447t22vWrNn28gMHDuSUU06hZ8+eDB8+nLIfzk8//TQ9e/akX79+XHDBBdu3G6ukpITDDz+cvn370rdvX15//fXty2688UYOPPBAevXqxdix4VrbefPmccwxx9CrVy/69u3LZ599VrM7qhxKnSKSFl9+Wbn51bFw4UJef/11CgoKWLNmDa+++ir169fnhRde4He/+0s7/44AAA3PSURBVB2PPvroLut89NFHvPTSS6xdu5Z9992XMWPG7HLu/bvvvssHH3zAnnvuyWGHHcZrr71GUVER55xzDq+88grdunVj2LBhCWNq164dzz//PI0bN+bTTz9l2LBhFBcX88wzz/D444/z1ltvUVhYyIoVKwAYPnw4Y8eO5aSTTmLjxo1s27at5ndUEkoEIpIWnTvD/PmJ59e0H//4xxQUFACwevVqRowYwaeffoqZsWXLloTrnHDCCTRq1IhGjRrRrl07lixZQseOHXcq079//+3zevfuTUlJCc2aNaN79+7bz9MfNmwYEyZM2GX7W7Zs4fzzz2fWrFkUFBTwySefAPDCCy9w1llnUVhYCECrVq1Yu3YtX331FSeddBIQLgrLJHUNiUhaXH99OCYQq7AwzK9pTZs23T59xRVXcNRRR/H+++/z5JNPJj2nvlGjRtunCwoKKC0trVKZZG677TZ23313Zs+eTXFxcYUHs7NJiUBE0mL48HBguEsXMAvPNXWguDyrV6+mQ4cOAPztb3+r8e3vu+++fP7555SUlAAwderUpHG0b9+eevXqMXHiRLZu3QrAscceywMPPMD66ADKihUraN68OR07dmTatGkAbNq0afvyTFAiEJG0GT4cSkpg27bwnO4kAHDppZdy2WWX0adPn0r9gk9VkyZN+POf/8ygQYPo168fzZs3p0WLFruUO/fcc3nwwQfp1asXH3300fZWy6BBgxgyZAhFRUX07t2bW265BYCJEydyxx13cNBBBzFgwAAWL15c47Eno9NHRSRllTl9tC5bt24dzZo1w90577zz6NGjBxdddFG2w9pOp4+KiKTZfffdR+/evTnggANYvXo155xzTrZDqhadNSQiUkkXXXRRTrUAqkstAhGRPKdEICKS55QIRETynBKBiEieUyIQkVrjqKOO4rnnnttp3u23386YMWOSrjNw4EDKTjk//vjjWbVq1S5lxo8fv/18/mSmTZvG3Lk7brl+5ZVX8sILL1Qm/JylRCAitcawYcOYMmXKTvOmTJmSdOC3eE8//TQtW+5yN9yUxCeCa665hmOOOaZK28o1On1URKrkwgth1qya3Wbv3nD77cmXn3LKKVx++eVs3ryZhg0bUlJSwtdff83hhx/OmDFjmDFjBhs2bOCUU07h6quv3mX9rl27UlxcTJs2bbj++ut58MEHadeuHZ06daJfv35AuEZgwoQJbN68mb333puJEycya9YsnnjiCV5++WWuu+46Hn30Ua699loGDx7MKaecwvTp07n44ospLS3l4IMP5u6776ZRo0Z07dqVESNG8OSTT7JlyxYeeeQRevbsuVNMJSUlnHHGGXz77bcA3HnnndtvjnPjjTcyadIk6tWrx3HHHcfvf/975s2bx+jRo1m6dCkFBQU88sgj7LXXXtXa72oRiEit0apVK/r3788zzzwDhNbAT37yE8yM66+/nuLiYubMmcPLL7/MnDlzkm7nnXfeYcqUKcyaNYunn36aGTNmbF928sknM2PGDGbPns1+++3H/fffz4ABAxgyZAg333wzs2bN2qni3bhxIyNHjmTq1Km89957lJaWcvfdd29f3qZNG2bOnMmYMWMSdj+VDVc9c+ZMpk6duv0uarHDVc+ePZtLL70UCMNVn3feecyePZvXX3+d9u3bV2+nohaBiFRReb/c06mse2jo0KFMmTKF+++/H4CHH36YCRMmUFpayqJFi5g7dy4HHXRQwm28+uqrnHTSSduHgh4yZMj2Ze+//z6XX345q1atYt26dfzwhz8sN56PP/6Ybt26sc8++wAwYsQI7rrrLi688EIgJBaAfv368dhjj+2yfi4MV50XLYKavm+qiGTP0KFDmT59OjNnzmT9+vX069ePL774gltuuYXp06czZ84cTjjhhKTDT1dk5MiR3Hnnnbz33ntcddVVVd5OmbKhrJMNY50Lw1XX+URQdt/U+fPBPTyPGqVkIFJbNWvWjKOOOoqzzz57+0HiNWvW0LRpU1q0aMGSJUu2dx0lc8QRRzBt2jQ2bNjA2rVrefLJJ7cvW7t2Le3bt2fLli1Mjqkomjdvztq1a3fZ1r777ktJSQnz5s0DwiiiRx55ZMqfJxeGq67ziSBT900VkcwZNmwYs2fP3p4IevXqRZ8+fejZsyenn346hx12WLnr9+3bl1NPPZVevXpx3HHHcfDBB29fdu2113LIIYdw2GGH7XRg97TTTuPmm2+mT58+O91PuHHjxjzwwAP8+Mc/5sADD6RevXqMHj065c+SC8NV1/lhqOvVCy2BeGZhjHQRSZ2Goa4dNAx1nGT3R03HfVNFRGqjOp8IMnnfVBGR2qjOJ4Js3TdVpK6qbd3J+aYqf5+8uI5g+HBV/CI1oXHjxixfvpzWrVtjZtkOR+K4O8uXL6/09QV5kQhEpGZ07NiRhQsXsnTp0myHIkk0btyYjh07VmodJQIRSVmDBg3o1q1btsOQGlbnjxGIiEj5lAhERPKcEoGISJ6rdVcWm9lSYH6240iiDbAs20GUQ/FVT67HB7kfo+KrnurE18Xd2yZaUOsSQS4zs+Jkl3DnAsVXPbkeH+R+jIqvetIVn7qGRETynBKBiEieUyKoWROyHUAFFF/15Hp8kPsxKr7qSUt8OkYgIpLn1CIQEclzSgQiInlOiaCSzKyTmb1kZnPN7AMz+1WCMgPNbLWZzYoeV2Y4xhIzey96711u52bBHWY2z8zmmFnfDMa2b8x+mWVma8zswrgyGd9/ZvZXM/vGzN6PmdfKzJ43s0+j592SrDsiKvOpmY3IUGw3m9lH0d/vX2bWMsm65X4X0hzjeDP7KubveHySdQeZ2cfR93FsBuObGhNbiZnNSrJuWvdhsjolo98/d9ejEg+gPdA3mm4OfALsH1dmIPBUFmMsAdqUs/x44BnAgO8Cb2UpzgJgMeFCl6zuP+AIoC/wfsy8m4Cx0fRY4MYE67UCPo+ed4umd8tAbD8A6kfTNyaKLZXvQppjHA9cnMJ34DOgO9AQmB3//5Su+OKW3wpcmY19mKxOyeT3Ty2CSnL3Re4+M5peC3wIdMhuVJU2FPi7B28CLc2sfRbi+D7wmbtn/Upxd38FWBE3eyjwYDT9IPB/CVb9IfC8u69w95XA88CgdMfm7v9x99Lo5ZtA5cYdrmFJ9l8q+gPz3P1zd98MTCHs9xpVXnwWbqzwE+Chmn7fVJRTp2Ts+6dEUA1m1hXoA7yVYPGhZjbbzJ4xswMyGhg48B8ze8fMRiVY3gFYEPN6IdlJZqeR/J8vm/uvzO7uviiaXgzsnqBMLuzLswktvEQq+i6k2/lR99Vfk3Rt5ML+OxxY4u6fJlmesX0YV6dk7PunRFBFZtYMeBS40N3XxC2eSeju6AX8CZiW4fC+5+59geOA88zsiAy/f4XMrCEwBHgkweJs779deGiH59y51mY2DigFJicpks3vwt3AXkBvYBGh+yUXDaP81kBG9mF5dUq6v39KBFVgZg0If7DJ7v5Y/HJ3X+Pu66Lpp4EGZtYmU/G5+1fR8zfAvwjN71hfAZ1iXneM5mXSccBMd18SvyDb+y/GkrIus+j5mwRlsrYvzWwkMBgYHlUUu0jhu5A27r7E3be6+zbgviTvndXvopnVB04GpiYrk4l9mKROydj3T4mgkqL+xPuBD939D0nK7BGVw8z6E/bz8gzF19TMmpdNEw4qvh9X7AngzOjsoe8Cq2OaoJmS9FdYNvdfnCeAsrMwRgCPJyjzHPADM9st6vr4QTQvrcxsEHApMMTd1ycpk8p3IZ0xxh53OinJe88AephZt6iVeBphv2fKMcBH7r4w0cJM7MNy6pTMff/SdSS8rj6A7xGaaHOAWdHjeGA0MDoqcz7wAeEMiDeBARmMr3v0vrOjGMZF82PjM+Auwtka7wFFGd6HTQkVe4uYeVndf4SktAjYQuhn/RnQGpgOfAq8ALSKyhYBf4lZ92xgXvQ4K0OxzSP0DZd9B++Jyu4JPF3edyGD+29i9P2aQ6jU2sfHGL0+nnCmzGfpijFRfNH8v5V972LKZnQfllOnZOz7pyEmRETynLqGRETynBKBiEieUyIQEclzSgQiInlOiUBEJM8pEYhEzGyr7Twyao2NhGlmXWNHvhTJJfWzHYBIDtng7r2zHYRIpqlFIFKBaDz6m6Ix6d82s72j+V3N7MVoULXpZtY5mr+7hXsEzI4eA6JNFZjZfdGY8/8xsyZR+QuisejnmNmULH1MyWNKBCI7NInrGjo1Ztlqdz8QuBO4PZr3J+BBdz+IMOjbHdH8O4CXPQya15dwRSpAD+Audz8AWAX8KJo/FugTbWd0uj6cSDK6slgkYmbr3L1ZgvklwNHu/nk0ONhid29tZssIwyZsieYvcvc2ZrYU6Ojum2K20ZUwbnyP6PVvgQbufp2ZPQusI4yyOs2jAfdEMkUtApHUeJLpytgUM72VHcfoTiCM/dQXmBGNiCmSMUoEIqk5Neb5jWj6dcJomQDDgVej6enAGAAzKzCzFsk2amb1gE7u/hLwW6AFsEurRCSd9MtDZIcmtvMNzJ9197JTSHczszmEX/XDonm/BB4ws0uApcBZ0fxfARPM7GeEX/5jCCNfJlIATIqShQF3uPuqGvtEIinQMQKRCkTHCIrcfVm2YxFJB3UNiYjkObUIRETynFoEIiJ5TolARCTPKRGIiOQ5JQIRkTynRCAikuf+Px3kg9XMdYElAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wNiMGrsKUtEG"
      },
      "source": [
        "驗證準確度在第九個周期後，準確度就下降  \r\n",
        "發生**過度適配(Overfitting)**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JBaaF39kUtEQ"
      },
      "source": [
        "## 重新建立訓練模型\r\n",
        "為了防止過度適配，將訓練週期限制在第四個周期後即結束訓練"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XRbKgB83UtEQ",
        "outputId": "68898321-bb6f-40f4-fe11-a8c03ae388e5"
      },
      "source": [
        "model = models.Sequential()\r\n",
        "model.add(layers.Dense(64,activation='relu',input_shape=(10000,)))\r\n",
        "model.add(layers.Dense(64,activation='relu'))\r\n",
        "model.add(layers.Dense(46,activation='softmax'))\r\n",
        "\r\n",
        "model.compile(\r\n",
        "  optimizer='rmsprop',\r\n",
        "  loss='categorical_crossentropy',\r\n",
        "  metrics=['accuracy']\r\n",
        ")\r\n",
        "\r\n",
        "model.fit(\r\n",
        "    partial_x_train,\r\n",
        "    partial_y_train,\r\n",
        "    epochs=9,\r\n",
        "    batch_size=512,\r\n",
        "    validation_data=(x_val,y_val))\r\n",
        "results=model.evaluate(x_test,one_hot_test_labels)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/9\n",
            "16/16 [==============================] - 1s 48ms/step - loss: 3.1010 - accuracy: 0.3698 - val_loss: 1.7667 - val_accuracy: 0.6360\n",
            "Epoch 2/9\n",
            "16/16 [==============================] - 1s 37ms/step - loss: 1.5206 - accuracy: 0.6930 - val_loss: 1.2999 - val_accuracy: 0.7330\n",
            "Epoch 3/9\n",
            "16/16 [==============================] - 1s 45ms/step - loss: 1.0736 - accuracy: 0.7777 - val_loss: 1.0981 - val_accuracy: 0.7650\n",
            "Epoch 4/9\n",
            "16/16 [==============================] - 1s 38ms/step - loss: 0.8162 - accuracy: 0.8248 - val_loss: 1.0119 - val_accuracy: 0.8000\n",
            "Epoch 5/9\n",
            "16/16 [==============================] - 1s 36ms/step - loss: 0.6469 - accuracy: 0.8665 - val_loss: 0.9416 - val_accuracy: 0.8070\n",
            "Epoch 6/9\n",
            "16/16 [==============================] - 1s 37ms/step - loss: 0.5121 - accuracy: 0.8987 - val_loss: 0.9278 - val_accuracy: 0.8020\n",
            "Epoch 7/9\n",
            "16/16 [==============================] - 1s 36ms/step - loss: 0.4121 - accuracy: 0.9197 - val_loss: 0.8979 - val_accuracy: 0.8160\n",
            "Epoch 8/9\n",
            "16/16 [==============================] - 1s 36ms/step - loss: 0.3332 - accuracy: 0.9314 - val_loss: 0.9002 - val_accuracy: 0.8190\n",
            "Epoch 9/9\n",
            "16/16 [==============================] - 1s 35ms/step - loss: 0.2628 - accuracy: 0.9446 - val_loss: 0.9280 - val_accuracy: 0.8080\n",
            "71/71 [==============================] - 0s 2ms/step - loss: 1.0085 - accuracy: 0.7823\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "I9tElpiZUtER",
        "outputId": "1fde4b89-a5c1-45c6-bfd8-193da2b8b7d0"
      },
      "source": [
        "results"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[1.0085341930389404, 0.7822796106338501]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sEmi56NtUtER"
      },
      "source": [
        "將準確度提升到88%準確度"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lNDrCYvIhSdn"
      },
      "source": [
        "## 資料瓶頸\r\n",
        "假定最終輸出有46個神經元，中間層應避免使用少於46神經元來傳遞資訊。 \r\n",
        "避免資訊被壓縮。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o_0uQTApTu2S"
      },
      "source": [
        "## 延伸實作"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BOMsFdcVTu2a"
      },
      "source": [
        "### 1. 使用一個隱藏層"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lUgHZbMhTu2b",
        "outputId": "226cf06e-caa1-4dd7-997c-83f50a990d61"
      },
      "source": [
        "model = models.Sequential()\r\n",
        "model.add(layers.Dense(64,activation='relu',input_shape=(10000,)))\r\n",
        "model.add(layers.Dense(46,activation='softmax'))\r\n",
        "\r\n",
        "model.compile(\r\n",
        "  optimizer='rmsprop',\r\n",
        "  loss='categorical_crossentropy',\r\n",
        "  metrics=['accuracy']\r\n",
        ")\r\n",
        "\r\n",
        "model.fit(\r\n",
        "    partial_x_train,\r\n",
        "    partial_y_train,\r\n",
        "    epochs=9,\r\n",
        "    batch_size=512,\r\n",
        "    validation_data=(x_val,y_val))\r\n",
        "results=model.evaluate(x_test,one_hot_test_labels)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/9\n",
            "16/16 [==============================] - 1s 48ms/step - loss: 3.0221 - accuracy: 0.4674 - val_loss: 1.7696 - val_accuracy: 0.6620\n",
            "Epoch 2/9\n",
            "16/16 [==============================] - 1s 38ms/step - loss: 1.5300 - accuracy: 0.7113 - val_loss: 1.3219 - val_accuracy: 0.7240\n",
            "Epoch 3/9\n",
            "16/16 [==============================] - 1s 37ms/step - loss: 1.0790 - accuracy: 0.7836 - val_loss: 1.1241 - val_accuracy: 0.7710\n",
            "Epoch 4/9\n",
            "16/16 [==============================] - 1s 38ms/step - loss: 0.8304 - accuracy: 0.8342 - val_loss: 1.0060 - val_accuracy: 0.7940\n",
            "Epoch 5/9\n",
            "16/16 [==============================] - 1s 36ms/step - loss: 0.6705 - accuracy: 0.8690 - val_loss: 0.9382 - val_accuracy: 0.8050\n",
            "Epoch 6/9\n",
            "16/16 [==============================] - 1s 38ms/step - loss: 0.5373 - accuracy: 0.8974 - val_loss: 0.8934 - val_accuracy: 0.8150\n",
            "Epoch 7/9\n",
            "16/16 [==============================] - 1s 37ms/step - loss: 0.4464 - accuracy: 0.9157 - val_loss: 0.8647 - val_accuracy: 0.8160\n",
            "Epoch 8/9\n",
            "16/16 [==============================] - 1s 37ms/step - loss: 0.3839 - accuracy: 0.9270 - val_loss: 0.8373 - val_accuracy: 0.8220\n",
            "Epoch 9/9\n",
            "16/16 [==============================] - 1s 36ms/step - loss: 0.3112 - accuracy: 0.9371 - val_loss: 0.8218 - val_accuracy: 0.8200\n",
            "71/71 [==============================] - 0s 2ms/step - loss: 0.8978 - accuracy: 0.7988\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EYkhsyu1Tu2c"
      },
      "source": [
        "### 2. 使用三個隱藏層"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ejQ2ckNOTu2c",
        "outputId": "487fa419-06fc-4f05-83d0-f7ff4aa74567"
      },
      "source": [
        "model = models.Sequential()\r\n",
        "model.add(layers.Dense(64,activation='relu',input_shape=(10000,)))\r\n",
        "model.add(layers.Dense(64,activation='relu'))\r\n",
        "model.add(layers.Dense(46,activation='softmax'))\r\n",
        "\r\n",
        "model.compile(\r\n",
        "  optimizer='rmsprop',\r\n",
        "  loss='categorical_crossentropy',\r\n",
        "  metrics=['accuracy']\r\n",
        ")\r\n",
        "\r\n",
        "model.fit(\r\n",
        "    partial_x_train,\r\n",
        "    partial_y_train,\r\n",
        "    epochs=9,\r\n",
        "    batch_size=512,\r\n",
        "    validation_data=(x_val,y_val))\r\n",
        "results=model.evaluate(x_test,one_hot_test_labels)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/9\n",
            "16/16 [==============================] - 1s 48ms/step - loss: 3.1435 - accuracy: 0.4162 - val_loss: 1.7653 - val_accuracy: 0.6310\n",
            "Epoch 2/9\n",
            "16/16 [==============================] - 1s 38ms/step - loss: 1.4881 - accuracy: 0.6965 - val_loss: 1.3421 - val_accuracy: 0.7140\n",
            "Epoch 3/9\n",
            "16/16 [==============================] - 1s 39ms/step - loss: 1.0851 - accuracy: 0.7710 - val_loss: 1.1444 - val_accuracy: 0.7680\n",
            "Epoch 4/9\n",
            "16/16 [==============================] - 1s 38ms/step - loss: 0.8491 - accuracy: 0.8236 - val_loss: 1.0467 - val_accuracy: 0.7890\n",
            "Epoch 5/9\n",
            "16/16 [==============================] - 1s 41ms/step - loss: 0.6695 - accuracy: 0.8584 - val_loss: 0.9854 - val_accuracy: 0.7890\n",
            "Epoch 6/9\n",
            "16/16 [==============================] - 1s 38ms/step - loss: 0.5403 - accuracy: 0.8869 - val_loss: 0.9531 - val_accuracy: 0.8160\n",
            "Epoch 7/9\n",
            "16/16 [==============================] - 1s 38ms/step - loss: 0.4276 - accuracy: 0.9124 - val_loss: 0.9139 - val_accuracy: 0.8150\n",
            "Epoch 8/9\n",
            "16/16 [==============================] - 1s 38ms/step - loss: 0.3387 - accuracy: 0.9289 - val_loss: 0.9247 - val_accuracy: 0.8140\n",
            "Epoch 9/9\n",
            "16/16 [==============================] - 1s 37ms/step - loss: 0.2869 - accuracy: 0.9386 - val_loss: 0.9037 - val_accuracy: 0.8220\n",
            "71/71 [==============================] - 0s 2ms/step - loss: 0.9731 - accuracy: 0.7841\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C1rnBWiZTu2c"
      },
      "source": [
        "### 3. 使用32個單元"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FBPn3p6TTu2c",
        "outputId": "23472ff0-7f81-484d-b16a-e7ef070bd208"
      },
      "source": [
        "model = models.Sequential()\r\n",
        "model.add(layers.Dense(32,activation='relu',input_shape=(10000,)))\r\n",
        "model.add(layers.Dense(32,activation='relu'))\r\n",
        "model.add(layers.Dense(46,activation='softmax'))\r\n",
        "\r\n",
        "model.compile(\r\n",
        "  optimizer='rmsprop',\r\n",
        "  loss='categorical_crossentropy',\r\n",
        "  metrics=['accuracy']\r\n",
        ")\r\n",
        "\r\n",
        "model.fit(\r\n",
        "    partial_x_train,\r\n",
        "    partial_y_train,\r\n",
        "    epochs=9,\r\n",
        "    batch_size=512,\r\n",
        "    validation_data=(x_val,y_val))\r\n",
        "results=model.evaluate(x_test,one_hot_test_labels)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/9\n",
            "16/16 [==============================] - 1s 40ms/step - loss: 3.3485 - accuracy: 0.3252 - val_loss: 2.2853 - val_accuracy: 0.5800\n",
            "Epoch 2/9\n",
            "16/16 [==============================] - 0s 30ms/step - loss: 2.0021 - accuracy: 0.6265 - val_loss: 1.6580 - val_accuracy: 0.6510\n",
            "Epoch 3/9\n",
            "16/16 [==============================] - 1s 42ms/step - loss: 1.4701 - accuracy: 0.6967 - val_loss: 1.3805 - val_accuracy: 0.7050\n",
            "Epoch 4/9\n",
            "16/16 [==============================] - 1s 32ms/step - loss: 1.2098 - accuracy: 0.7474 - val_loss: 1.2428 - val_accuracy: 0.7250\n",
            "Epoch 5/9\n",
            "16/16 [==============================] - 0s 31ms/step - loss: 1.0231 - accuracy: 0.7813 - val_loss: 1.1491 - val_accuracy: 0.7520\n",
            "Epoch 6/9\n",
            "16/16 [==============================] - 1s 32ms/step - loss: 0.8722 - accuracy: 0.8200 - val_loss: 1.0913 - val_accuracy: 0.7620\n",
            "Epoch 7/9\n",
            "16/16 [==============================] - 0s 31ms/step - loss: 0.7546 - accuracy: 0.8428 - val_loss: 1.0513 - val_accuracy: 0.7810\n",
            "Epoch 8/9\n",
            "16/16 [==============================] - 1s 32ms/step - loss: 0.6392 - accuracy: 0.8647 - val_loss: 1.0107 - val_accuracy: 0.7850\n",
            "Epoch 9/9\n",
            "16/16 [==============================] - 0s 30ms/step - loss: 0.5663 - accuracy: 0.8788 - val_loss: 0.9803 - val_accuracy: 0.7900\n",
            "71/71 [==============================] - 0s 2ms/step - loss: 1.0588 - accuracy: 0.7707\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "StiLrkuyTu2d"
      },
      "source": [
        "### 4. 使用128個單元"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Bl9j_3PETu2d",
        "outputId": "63aab294-0e0a-493d-dcdd-b0707d750ea0"
      },
      "source": [
        "model = models.Sequential()\r\n",
        "model.add(layers.Dense(128,activation='relu',input_shape=(10000,)))\r\n",
        "model.add(layers.Dense(128,activation='relu'))\r\n",
        "model.add(layers.Dense(46,activation='softmax'))\r\n",
        "\r\n",
        "model.compile(\r\n",
        "  optimizer='rmsprop',\r\n",
        "  loss='categorical_crossentropy',\r\n",
        "  metrics=['accuracy']\r\n",
        ")\r\n",
        "\r\n",
        "model.fit(\r\n",
        "    partial_x_train,\r\n",
        "    partial_y_train,\r\n",
        "    epochs=9,\r\n",
        "    batch_size=512,\r\n",
        "    validation_data=(x_val,y_val))\r\n",
        "results=model.evaluate(x_test,one_hot_test_labels)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/9\n",
            "16/16 [==============================] - 1s 65ms/step - loss: 2.7893 - accuracy: 0.4535 - val_loss: 1.3531 - val_accuracy: 0.6980\n",
            "Epoch 2/9\n",
            "16/16 [==============================] - 1s 57ms/step - loss: 1.1454 - accuracy: 0.7578 - val_loss: 1.0936 - val_accuracy: 0.7630\n",
            "Epoch 3/9\n",
            "16/16 [==============================] - 1s 54ms/step - loss: 0.7825 - accuracy: 0.8333 - val_loss: 0.9593 - val_accuracy: 0.8050\n",
            "Epoch 4/9\n",
            "16/16 [==============================] - 1s 51ms/step - loss: 0.5486 - accuracy: 0.8816 - val_loss: 0.8798 - val_accuracy: 0.8200\n",
            "Epoch 5/9\n",
            "16/16 [==============================] - 1s 54ms/step - loss: 0.4040 - accuracy: 0.9146 - val_loss: 0.8472 - val_accuracy: 0.8170\n",
            "Epoch 6/9\n",
            "16/16 [==============================] - 1s 54ms/step - loss: 0.3024 - accuracy: 0.9317 - val_loss: 0.8754 - val_accuracy: 0.8130\n",
            "Epoch 7/9\n",
            "16/16 [==============================] - 1s 53ms/step - loss: 0.2360 - accuracy: 0.9469 - val_loss: 0.9095 - val_accuracy: 0.8020\n",
            "Epoch 8/9\n",
            "16/16 [==============================] - 1s 53ms/step - loss: 0.1888 - accuracy: 0.9536 - val_loss: 0.8981 - val_accuracy: 0.8230\n",
            "Epoch 9/9\n",
            "16/16 [==============================] - 1s 53ms/step - loss: 0.1794 - accuracy: 0.9516 - val_loss: 1.0168 - val_accuracy: 0.7930\n",
            "71/71 [==============================] - 0s 2ms/step - loss: 1.1019 - accuracy: 0.7698\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jUO5Tlvk4nCA"
      },
      "source": [
        "# 3-6 迴歸：預測房價\r\n",
        "預測連續值而不是離散標籤"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pS_3oNaGa6EH"
      },
      "source": [
        "## 準備資料\r\n",
        "載入1970年代中期波士頓的郊區資料  \r\n",
        "本例有506筆資料，分為404個訓練樣本和102個測試資料"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MrlyhwJbbu4R",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3801fcba-1456-4737-cf3d-88790f307c09"
      },
      "source": [
        "from keras.datasets import boston_housing\r\n",
        "\r\n",
        "(train_data,train_targets),(test_data,test_targets) = boston_housing.load_data()"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/boston_housing.npz\n",
            "57344/57026 [==============================] - 0s 0us/step\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "otLDBVidcKhD",
        "outputId": "67627714-829a-4bb5-f226-7a0e0e68da14"
      },
      "source": [
        "train_data.shape"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(404, 13)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GiwclzYycO28",
        "outputId": "bafd5997-c701-4181-b1a3-9597ac65ce4f"
      },
      "source": [
        "test_data.shape"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(102, 13)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CSipspQOcU2L",
        "outputId": "b82680ed-ce69-422f-f879-e672884f6c90"
      },
      "source": [
        "train_targets"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([15.2, 42.3, 50. , 21.1, 17.7, 18.5, 11.3, 15.6, 15.6, 14.4, 12.1,\n",
              "       17.9, 23.1, 19.9, 15.7,  8.8, 50. , 22.5, 24.1, 27.5, 10.9, 30.8,\n",
              "       32.9, 24. , 18.5, 13.3, 22.9, 34.7, 16.6, 17.5, 22.3, 16.1, 14.9,\n",
              "       23.1, 34.9, 25. , 13.9, 13.1, 20.4, 20. , 15.2, 24.7, 22.2, 16.7,\n",
              "       12.7, 15.6, 18.4, 21. , 30.1, 15.1, 18.7,  9.6, 31.5, 24.8, 19.1,\n",
              "       22. , 14.5, 11. , 32. , 29.4, 20.3, 24.4, 14.6, 19.5, 14.1, 14.3,\n",
              "       15.6, 10.5,  6.3, 19.3, 19.3, 13.4, 36.4, 17.8, 13.5, 16.5,  8.3,\n",
              "       14.3, 16. , 13.4, 28.6, 43.5, 20.2, 22. , 23. , 20.7, 12.5, 48.5,\n",
              "       14.6, 13.4, 23.7, 50. , 21.7, 39.8, 38.7, 22.2, 34.9, 22.5, 31.1,\n",
              "       28.7, 46. , 41.7, 21. , 26.6, 15. , 24.4, 13.3, 21.2, 11.7, 21.7,\n",
              "       19.4, 50. , 22.8, 19.7, 24.7, 36.2, 14.2, 18.9, 18.3, 20.6, 24.6,\n",
              "       18.2,  8.7, 44. , 10.4, 13.2, 21.2, 37. , 30.7, 22.9, 20. , 19.3,\n",
              "       31.7, 32. , 23.1, 18.8, 10.9, 50. , 19.6,  5. , 14.4, 19.8, 13.8,\n",
              "       19.6, 23.9, 24.5, 25. , 19.9, 17.2, 24.6, 13.5, 26.6, 21.4, 11.9,\n",
              "       22.6, 19.6,  8.5, 23.7, 23.1, 22.4, 20.5, 23.6, 18.4, 35.2, 23.1,\n",
              "       27.9, 20.6, 23.7, 28. , 13.6, 27.1, 23.6, 20.6, 18.2, 21.7, 17.1,\n",
              "        8.4, 25.3, 13.8, 22.2, 18.4, 20.7, 31.6, 30.5, 20.3,  8.8, 19.2,\n",
              "       19.4, 23.1, 23. , 14.8, 48.8, 22.6, 33.4, 21.1, 13.6, 32.2, 13.1,\n",
              "       23.4, 18.9, 23.9, 11.8, 23.3, 22.8, 19.6, 16.7, 13.4, 22.2, 20.4,\n",
              "       21.8, 26.4, 14.9, 24.1, 23.8, 12.3, 29.1, 21. , 19.5, 23.3, 23.8,\n",
              "       17.8, 11.5, 21.7, 19.9, 25. , 33.4, 28.5, 21.4, 24.3, 27.5, 33.1,\n",
              "       16.2, 23.3, 48.3, 22.9, 22.8, 13.1, 12.7, 22.6, 15. , 15.3, 10.5,\n",
              "       24. , 18.5, 21.7, 19.5, 33.2, 23.2,  5. , 19.1, 12.7, 22.3, 10.2,\n",
              "       13.9, 16.3, 17. , 20.1, 29.9, 17.2, 37.3, 45.4, 17.8, 23.2, 29. ,\n",
              "       22. , 18. , 17.4, 34.6, 20.1, 25. , 15.6, 24.8, 28.2, 21.2, 21.4,\n",
              "       23.8, 31. , 26.2, 17.4, 37.9, 17.5, 20. ,  8.3, 23.9,  8.4, 13.8,\n",
              "        7.2, 11.7, 17.1, 21.6, 50. , 16.1, 20.4, 20.6, 21.4, 20.6, 36.5,\n",
              "        8.5, 24.8, 10.8, 21.9, 17.3, 18.9, 36.2, 14.9, 18.2, 33.3, 21.8,\n",
              "       19.7, 31.6, 24.8, 19.4, 22.8,  7.5, 44.8, 16.8, 18.7, 50. , 50. ,\n",
              "       19.5, 20.1, 50. , 17.2, 20.8, 19.3, 41.3, 20.4, 20.5, 13.8, 16.5,\n",
              "       23.9, 20.6, 31.5, 23.3, 16.8, 14. , 33.8, 36.1, 12.8, 18.3, 18.7,\n",
              "       19.1, 29. , 30.1, 50. , 50. , 22. , 11.9, 37.6, 50. , 22.7, 20.8,\n",
              "       23.5, 27.9, 50. , 19.3, 23.9, 22.6, 15.2, 21.7, 19.2, 43.8, 20.3,\n",
              "       33.2, 19.9, 22.5, 32.7, 22. , 17.1, 19. , 15. , 16.1, 25.1, 23.7,\n",
              "       28.7, 37.2, 22.6, 16.4, 25. , 29.8, 22.1, 17.4, 18.1, 30.3, 17.5,\n",
              "       24.7, 12.6, 26.5, 28.7, 13.3, 10.4, 24.4, 23. , 20. , 17.8,  7. ,\n",
              "       11.8, 24.4, 13.8, 19.4, 25.2, 19.4, 19.4, 29.1])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K_iWURfZcuOm"
      },
      "source": [
        "## 準備資料\r\n",
        "對資料進行**正規化(normalization)**  \r\n",
        "對每個特徵值減去平均值並減去標準差，簡化問題"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AoCrAcKJcwai"
      },
      "source": [
        "mean = train_data.mean(axis=0)\r\n",
        "train_data -= mean\r\n",
        "std = train_data.std(axis=0)\r\n",
        "train_data /= std\r\n",
        "\r\n",
        "test_data -= mean\r\n",
        "test_data /= std "
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oIZ8eheSd6aE"
      },
      "source": [
        "## 建立神經網路\r\n",
        "以單一unit結束且沒有啟動函數(代表將是線性層)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gu1vaNKRgVx9"
      },
      "source": [
        "from keras import models\r\n",
        "from keras import layers\r\n",
        "\r\n",
        "def build_model():\r\n",
        "  model = models.Sequential()\r\n",
        "  model.add(layers.Dense(64,activation='relu',input_shape=(train_data.shape[1],)))\r\n",
        "  model.add(layers.Dense(64,activation='relu'))\r\n",
        "  model.add(layers.Dense(1))\r\n",
        "  model.compile(optimizer='rmsprop',loss='mse',metrics=['mae'])\r\n",
        "  return model"
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A7vCiNDRb3If"
      },
      "source": [
        "## K-fold cross validation(K折交叉驗證)：驗證模型的成效\r\n",
        "將可用資料拆分成K個區塊，保持模型不變的情況下：  \r\n",
        "選一個區塊作為驗證集，而其餘區做當作訓練集。  \r\n",
        "![pic3-2](https://github.com/hank199599/deep_learning_keras_log/blob/main/pictures/3-2.png?raw=true)  \r\n",
        "\r\n",
        "**使資料集太小，使變異性增大的缺點平均掉。**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9Ik7nxUfeA0D",
        "outputId": "87d592a0-abef-445c-f182-581e1152f784"
      },
      "source": [
        "import numpy as np\r\n",
        "import os\r\n",
        "os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'  #關掉TensorFlow的警告\r\n",
        "\r\n",
        "k=4 # 進行4折交叉驗證\r\n",
        "num_val_samples = len(train_data)//k\r\n",
        "num_epochs=100\r\n",
        "all_scores=[]\r\n",
        "\r\n",
        "for i in range(k):\r\n",
        "  print(\"process fold #\",i)\r\n",
        "  # 準備驗證資料，來自#k區塊\r\n",
        "  val_data = train_data[i*num_val_samples:(i+1)*num_val_samples]\r\n",
        "  val_targets = train_targets[i*num_val_samples:(i+1)*num_val_samples]\r\n",
        "\r\n",
        "  # 準備訓練資料，來自#k以外的區塊\r\n",
        "  partical_train_data = np.concatenate([train_data[:i*num_val_samples],train_data[(i+1)*num_val_samples:]],axis=0)\r\n",
        "  partical_train_targets = np.concatenate([train_targets[:i*num_val_samples],train_targets[(i+1)*num_val_samples:]],axis=0)  \r\n",
        "  \r\n",
        "  model = build_model()\r\n",
        "  model.fit(partical_train_data,partical_train_targets,epochs=num_epochs,batch_size=1,verbose=0)\r\n",
        "\r\n",
        "  val_mse,val_mae = model.evaluate(val_data,val_targets,verbose=0)\r\n",
        "  all_scores.append(val_mae)"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "process fold # 0\n",
            "process fold # 1\n",
            "process fold # 2\n",
            "process fold # 3\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4wwyj7A2fnrl",
        "outputId": "61937b30-84ad-4460-bd83-4bdc27da1b3d"
      },
      "source": [
        "all_scores"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[2.5371220111846924, 2.5387237071990967, 2.766490936279297, 2.494704484939575]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WfPBUiJHfqJS",
        "outputId": "89d33ec0-685a-40b3-b0a6-3cb6cb518499"
      },
      "source": [
        "np.mean(all_scores)"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2.5842602849006653"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g_4InBXDyq1f"
      },
      "source": [
        "## 嘗試500 epochs來訓練"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LabIAiY-yvSv",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c5e1c319-e28d-4455-be6f-4106ead71a5b"
      },
      "source": [
        "import numpy as np\r\n",
        "import os\r\n",
        "os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'  #關掉TensorFlow的警告\r\n",
        "\r\n",
        "k=4 # 進行4折交叉驗證\r\n",
        "num_val_samples = len(train_data)//k\r\n",
        "num_epochs=500\r\n",
        "all_mae_history=[]\r\n",
        "\r\n",
        "for i in range(k):\r\n",
        "  print(\"process fold #\",i)\r\n",
        "  # 準備驗證資料，來自#k區塊\r\n",
        "  val_data = train_data[i*num_val_samples:(i+1)*num_val_samples]\r\n",
        "  val_targets = train_targets[i*num_val_samples:(i+1)*num_val_samples]\r\n",
        "\r\n",
        "  # 準備訓練資料，來自#k以外的區塊\r\n",
        "  partical_train_data = np.concatenate([train_data[:i*num_val_samples],train_data[(i+1)*num_val_samples:]],axis=0)\r\n",
        "  partical_train_targets = np.concatenate([train_targets[:i*num_val_samples],train_targets[(i+1)*num_val_samples:]],axis=0)  \r\n",
        "  \r\n",
        "  model = build_model()\r\n",
        "  history = model.fit(partical_train_data,partical_train_targets,validation_data=(val_data,val_targets),epochs=num_epochs,batch_size=1,verbose=0)\r\n",
        "  mae_history = history.history['val_mae']\r\n",
        "  all_mae_history.append(mae_history)"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "process fold # 0\n",
            "process fold # 1\n",
            "process fold # 2\n",
            "process fold # 3\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Y9_F0gKVzkR-"
      },
      "source": [
        "建立平均K折驗證分數的歷史"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TpABSmaNzjt_"
      },
      "source": [
        "average_mae_history = [np.mean([x[i] for x in all_mae_history]) for i in range(num_epochs)]"
      ],
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "q_nxzE6Nz_i3"
      },
      "source": [
        "## 繪製驗證分數"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "I8m__2eX0BYt",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 279
        },
        "outputId": "a7e31af2-69b3-4c85-87bc-a74d7422166c"
      },
      "source": [
        "import matplotlib.pyplot as plt\r\n",
        "\r\n",
        "plt.plot(range(1,len(average_mae_history)+1),average_mae_history)\r\n",
        "plt.xlabel('Epochs')\r\n",
        "plt.ylabel('Validation MAE')\r\n",
        "plt.show()"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO2dd5gV5fX4P2fvdpaydKStIBYEQVxRxC72msREbDFG4y/GJCbxa6JJrNFoTKKmmESiJhqNxi52FHtBitKLFEFAYJfO9nZ+f0zZuffOLcDeXVjO53nus3fembnzzu7d98zpoqoYhmEYRixZbT0BwzAMY9fEBIRhGIYRigkIwzAMIxQTEIZhGEYoJiAMwzCMULLbegItSffu3bWkpKStp2EYhrHbMGPGjPWq2iNsX7sSECUlJUyfPr2tp2EYhrHbICIrEu0zE5NhGIYRigkIwzAMIxQTEIZhGEYoJiAMwzCMUDIuIEQkIiKfichLIft+JiLzRWS2iEwWkYGBfY0iMtN9Tcz0PA3DMIxoWiOK6WpgAdApZN9nQKmqVonIlcBdwHnuvmpVHdkK8zMMwzBCyKgGISL9gNOBB8L2q+rbqlrlbk4B+mVyPoZhGEb6ZNrEdC/wc6ApjWMvA14NbOeLyHQRmSIi5yQ6SUSucI+bXl5evkOT/Mvkxbz7+Y6daxiG0V7JmIAQkTOAMlWdkcaxFwGlwO8DwwNVtRS4ALhXRAaHnauqE1S1VFVLe/QITQZMyd/eWcoHi01AGIZhBMmkBjEWOEtElgNPAMeLyKOxB4nIOOBXwFmqWuuNq+pq9+cy4B3g4ExNNEvA+iYZhmFEkzEBoarXq2o/VS0BxgNvqepFwWNE5GDgfhzhUBYYLxaRPPd9dxxhMz9Tc80SockEhGEYRhStXotJRG4FpqvqRByTUhHwlIgAfKmqZwEHAPeLSBOOELtTVTMmIESgyVQIwzCMKFpFQKjqOzhmIlT1xsD4uATHfwQMb425AWRlCdab2zAMIxrLpMZMTIZhGGGYgMBxUpuJyTAMIxoTEICYBmEYhhGHCQi8MFeTEIZhGEFMQOD5IExAGIZhBDEBgTmpDcMwwjABgeVBGIZhhGECAkeDMPlgGIYRjQkILMzVMAwjDBMQmA/CMAwjDBMQmA/CMAwjDBMQeD4IExCGYRhBTEDgmpjS6XlnGIaxB2ECAjMxGYZhhGECAnNSG4ZhhGECAsjKslpMhmEYsWRcQIhIREQ+E5GXQvblicj/RGSJiHwiIiWBfde744tE5ORMztFqMRmGYcTTGhrE1cCCBPsuAzap6j7APcDvAERkKE4f6wOBU4C/iUgkUxO0ct+GYRjxZFRAiEg/4HTggQSHnA087L5/GjhBnObUZwNPqGqtqn4BLAFGZ2qelkltGIYRT6Y1iHuBnwOJgkj7AisBVLUB2AJ0C467rHLH4hCRK0RkuohMLy8v36FJWi0mwzCMeDImIETkDKBMVWdk6hoAqjpBVUtVtbRHjx479BmmQRiGYcSTSQ1iLHCWiCwHngCOF5FHY45ZDfQHEJFsoDOwITju0s8dywhiTmrDMIw4MiYgVPV6Ve2nqiU4Due3VPWimMMmApe47891j1F3fLwb5bQ3MASYmqm5OhpEpj7dMAxj9yS7tS8oIrcC01V1IvAg8B8RWQJsxBEkqOo8EXkSmA80AFepamOm5pQlQqPV2jAMw4iiVQSEqr4DvOO+vzEwXgN8M8E5twO3t8L0LJPaMAwjBMukxmoxGYZhhGECAtMgDMMwwjABgeOktlpMhmEY0ZiAwGoxGYZhhGECAjcPwoKYDMMwojABgWVSG4ZhhGECAqvFZBiGEYYJCJyGQaZBGIZhRGMCAqvFZBiGEYYJCMzEZBiGEYYJCMxJbRiGEYYJCCyT2jAMIwwTEFgtJsMwjDBMQGA+CMMwjDBMQGA+CMMwjDAy1g9CRPKB94A89zpPq+pNMcfcAxznbhYCPVW1i7uvEZjj7vtSVc/K1FytFpNhGEY8mWwYVAscr6oVIpIDfCAir6rqFO8AVf2p915EfgQcHDi/WlVHZnB+PmJOasMwjDgy2ZNaVbXC3cxxX8mW4fOBxzM1n2RYuW/DMIx4MuqDEJGIiMwEyoA3VPWTBMcNBPYG3goM54vIdBGZIiLnZHKeFuZqGIYRT0YFhKo2umaifsBoERmW4NDxOD6KxsDYQFUtBS4A7hWRwWEnisgVriCZXl5evkPzNCe1YRhGPK0SxaSqm4G3gVMSHDKeGPOSqq52fy4D3iHaPxE8boKqlqpqaY8ePXZofk4/CBMQhmEYQTImIESkh4h4EUkFwInAwpDj9geKgY8DY8Uikue+7w6MBeZnaq6WB2EYhhFPJqOY+gAPi0gERxA9qaovicitwHRVnegeNx54QqO9xAcA94tIk3vunaqaQQFhJibDMIxYMiYgVHU2IWYhVb0xZvvmkGM+AoZnam6xZGWZk9owDCMWy6TGajEZhmGEYQIC80EYhmGEYQIC80EYhmGEYQICq8VkGIYRhgkIQMCc1IZhGDGYgMBJlAOrx2QYhhHEBASOiQkwR7VhGEYAExA4TmowR7VhGEYQExA4iXJgfgjDMIwgJiBwEuXANAjDMIwgCQWEiDwZeP+7mH2TMjmp1sZ8EIZhGPEk0yCGBN6fGLNvx+pq76KYD8IwDCOeZAIi2WrZrlZST4MwAWEYhtFMsmquhSJyMI4QKXDfi/sqaI3JtRYi5qQ2DMOIJZmAWAPc7b5fG3jvbbcbPBOTJcoZhmE0k1BAqOpxifaJSE5mptM2ZJkGYRiGEUfaYa7icIKIPAisSuP4fBGZKiKzRGSeiNwScsx3RKRcRGa6r8sD+y4RkcXu65K072gHMCe1YRhGPCk7yonI4cAFwDlAV+Aq4P/S+Oxa4HhVrXA1jg9E5FVVnRJz3P9U9Ycx1+wK3ASU4jjEZ4jIRFXdlMZ1txsxJ7VhGEYcyfIgfisii4HbAa99aLmqPpzOQq0OFe5mjvtKdwU+GXhDVTe613oDOCXNc7cby4MwDMOIJ5mJ6XJgHfB34D+quoHtDG8VkYiIzATKcBb8T0IO+4aIzBaRp0WkvzvWF1gZOGaVOxZ2jStEZLqITC8vL9+e6fmYickwDCOeZAKiD3AbcCawVET+gxPumtIs5aGqjao6EugHjBaRYTGHvAiUqOpBOFrCw9s1e+caE1S1VFVLe/TYsfw9c1IbhmHEk1BAuIv7a6p6CTAYeB74EFgtIv/dnouo6mbgbWLMRKq6QVVr3c0HgEPc96uB/oFD+7ljGcGvxWQSwjAMwyetKCZVrVXVZ1T1XGAf4LVU54hIDxHp4r4vwCnXsTDmmD6BzbOABe7714GTRKRYRIqBk9yxjGA+CMMwjHgSmotE5Gc7+dl9gIdFJIIjiJ5U1ZdE5FZguqpOBH4sImcBDcBG4DsAqrpRRH4DTHM/61ZV3biT80lIlismzQdhGIbRTDJ/wh+AmcCrOCGrEtiXciVVVS/yKXb8xsD764HrE5z/EPBQquu0BJ4G0WgCwjAMwyeZgDgYOB84HZgBPA5M1nZYj8J3UpsPwjAMwyeZk3qWql7nRiE9CJwNzHdNQu2KSJZpEIZhGLGkdFKLSA8cbWI4Tj5CWaYn1dr4JibTIAzDMHySOam/C3wLyAeeBr6lqu1OOECzBtHU1MYTMQzD2IVI5oN4AJgLrMApfXGSV7MIQFXbjakp20xMhmEYcSQTEAnLfbc3sjwBYSqEYRiGT7J+EO+25kTakojvg2jjiRiGYexCpN0Poj3jJcqZk9owDKMZExA0axCWSW0YhtGMCQggO2JhroZhGLGk01FuX+BaYGDweFU9PoPzalUsD8IwDCOedHo7PAX8A/gn0JjZ6bQNfia1CQjDMAyfdAREg6r+PeMzaUOsWJ9hGEY86fggXhSRH4hIHxHp6r0yPrNWpDmT2gSEYRiGRzoaxCXuz2sDYwoMavnptA1eJnWDCQjDMAyflAJCVfdujYm0JV4mtYW5GoZhNJNOFFMOcCVwtDv0DnC/qtanOC8feA/Ic6/ztKreFHPMz4DLcTrKlQPfVdUV7r5GYI576JeZrP0UsSgmwzCMONIxMf0dyAH+5m5f7I5dnuK8WuB4Va1whcwHIvKqqk4JHPMZUKqqVSJyJXAXcJ67r9rtRZFxLIrJMAwjnnQExKGqOiKw/ZaIzEp1ktt5rsLdzHFfGnPM24HNKcBFacynxYmYickwDCOOdKKYGkVksLchIoNIMx9CRCIiMhOnydAbqvpJksMvw+l/7ZEvItNFZIqInJPkGle4x00vLy9PZ1pxRMxJbRiGEUc6GsS1wNsisgwQnIzqS9P5cFVtBEaKSBfgOREZpqpzY48TkYuAUuCYwPBAVV3tCqS3RGSOqi4NucYEYAJAaWnpDq3w1pPaMAwjnnSimCaLyBBgP3dokarWbs9FVHWziLwNnILThMhHRMYBvwKOCX6uqq52fy4TkXdw2p7GCYiWwHwQhmEY8SQ0MYnI8e7PrwOnA/u4r9PdsaSISA9Xc0BECoATgYUxxxwM3A+cFWxnKiLFIpLnvu8OjAXmb9+tpY8fxWTywTAMwyeZBnEM8BZwZsg+BZ5N8dl9gIdFJIIjiJ5U1ZdE5FZguqpOBH4PFAFPue1MvXDWA4D7RaTJPfdOVc2cgIiYickwDCOWZB3lvJyFW1X1i+A+EUmZPKeqs3HMQrHjNwbej0tw7kfA8FTXaCk8DcKc1IZhGM2kE8X0TMjY0y09kbbE6yhnYa6GYRjNJNQgRGR/4ECgc4zPoROQn+mJtSaWSW0YhhFPMh/EfsAZQBei/RDbgO9lclKtjUUxGYZhxJPMB/EC8IKIjFHVj1txTq2OiJAlJiAMwzCCpJMo95mIXIVjbvJNS6r63YzNqg2IZIk1DDIMwwiQjpP6P0Bv4GTgXaAfjpmpXZElYmGuhmEYAdIREPuo6g1Apao+jJM0d1hmp9X6RLLETEyGYRgB0hEQXt+HzSIyDOgM9MzclNqGiJiJyTAMI0g6PogJIlIM3ABMxMl8vjH5KbsfkYhpEIZhGEHSKdb3gPv2XdpRH+pYImICwjAMI0iyRLmfJTtRVe9u+em0HVlZYpnUhmEYAZJpEB3dn/sBh+KYl8BJmpuayUm1BaZBGIZhRJMsUe4WABF5Dxilqtvc7ZuBl1tldq2IE8XU1rMwDMPYdUgniqkXUBfYrnPH2hWOgDAJYRiG4ZFOFNMjwFQRec7dPgf4d8Zm1EY4mdRtPQvDMIxdh5QahKrejtODepP7ulRV70h1nojki8hUEZklIvNE5JaQY/JE5H8iskREPhGRksC+693xRSJy8vbc1I6QJdYwyDAMI0iyKKZOqrpVRLoCy92Xt6+rqm5M8dm1wPGqWiEiOcAHIvKqqk4JHHMZsElV9xGR8cDvgPNEZCgwHqf+017AmyKyr6o27sA9poVlUhuGYUSTzMT0X5xy3zNwWox6iLudNCdCVRWocDdz3FfsCnw2cLP7/mngr+L0Hj0beEJVa4EvRGQJMBrIWFXZSFaWZVIbhmEESBbFdIb7M2V70US4/ahnAPsA96nqJzGH9AVWutdpEJEtQDd3PKhprHLHwq5xBXAFwIABA3Z0qkSyrNy3YRhGkGQmplHJTlTVT1N9uGsSGikiXYDnRGSYqs7d/mkmvcYEYAJAaWnpDq/wlgdhGIYRTTIT0x+T7FPg+HQvoqqbReRt4BQgKCBWA/2BVSKSjVMIcENg3KOfO5YxLJPaMAwjmmQmpuN25oNFpAdQ7wqHAuBEHCd0kInAJTi+hXOBt1RVRWQi8F8RuRvHST2EDGdvmwZhGIYRTTp5ELhlvocS3VHukRSn9QEedv0QWcCTqvqSiNwKTFfVicCDwH9cJ/RGnMglVHWeiDwJzAcagKsyGcEEThRTgwkIwzAMn5QCQkRuAo7FERCvAKcCH+Ak0CVEVWcDB4eM3xh4XwN8M8H5twO3p5pfS5ETyaKyrqG1LmcYhrHLk06pjXOBE4C1qnopMALHV9CuKMyNUF2XUSXFMAxjtyIdAVGtqk1Ag4h0AsqIdiC3C4rysqmoNQ3CMAzDIx0BMd0NU/0nTk7Dp2QwYa2tKMyLUFXXyMqNVZz91w/YWFmX+iTDMIx2TEIBISL3ichYVf2Bqm5W1X/gRCJd4pqa2hUdXA1iwnvLmLVqCxNnZjSq1jAMY5cnmZP6c+APItIHeBJ4XFU/a51ptT5FudnUNTSRE3FkZnW9lf42DGPPJqEGoap/UtUxwDE4yWsPichCEblJRPZttRm2EoV5jqz0kuWq681hbRjGnk065b5XqOrvVPVg4HycfhALMj6zVqYoLwIEBISFvBqGsYeTUkCISLaInCkijwGvAouAr2d8Zq1MYa6jQVTWOpqDaRCGYcSyeN02Sq57mbcXloXuV1U+WroebSdle5I5qU8UkYdwKql+D6cP9WBVHa+qL7TWBFuLItfEtKnKiV6qrjMfhGEY0UxfsQmASfPXhu5/cvpKLvjnJ7w0e01rTitjJNMgrgc+Ag5Q1bNU9b+qWtlK82p1CnMdE9MGN7x1a019W07HMIxdEK8cT5ZI6P6l5c4SuXpzdcbnsnx9JXe+ujCj2kqyYn1pV2ttD3RwNYgNFbUAbK6yPAjDMKJpbHQsC9lZ4QKiwW1sH0kgQFqSyx6extLySi48bAD9uxZm5BrpJMrtETQLCEcwbKsxJ7VhtCeq6hqoa9g507G7/hPJCl86G5ua3P2ZFxCbqjJv5TAB4dIp3xEQnnN6Z79IhmHsWgy98XW+ef/OFYFoFgAJ9rvmnlgBUdfQxA3Pz6VsW81OXT9IlRtpOXnBOpoyVInaBIRLp4KcqO1aExCGsdPUNTQx+vY3eWEXqUwwa+XmnTrf80Ek1iCcn/WN0evHWwvL+M+UFdwycf5OXT9IjZvMe/OL8/n7u0tb7HODmIBwyYlk0cF1VIMJCGP3RlV5Zc6auIWqtdlUVUfZtlqWr69KeWxVXQMl173MQx980Qoz2zEaXRtTIh+Ep2HEVob2HMkNTZn5e0z9YmNGPtcERIDOAS2itsHyIIxdk3c/L+erFFEy7y9ezw8e+5R73vg8o3Opqmtg3dbEZpPNrp08nf+n9dsc/99DH+66AqLej2IK3+891VftZB7Vyo1VfLhkfdrHl22r3anrJSJjAkJE+ovI2yIyX0TmicjVIcdcKyIz3ddcEWkUka7uvuUiMsfdNz1T8wwSNDPVNTShqnz25aZ2k/Ri7P6oKpc8NJVz7vsw6XHek+qsVTtnUknF+f/8hMN+Oznhfi8aMB2N3BMiOZEsPl+3jQ0VtTS0kAbUUv/Dte7CX5/A5u+1DIjVILygpnSncfqf3+fCBz5Je15lSYT0zpBJDaIBuEZVhwKHA1eJyNDgAar6e1UdqaojcfIu3lXVoK50nLu/NIPz9OkUpUE0MXHWV3ztbx8xcdZXrXF5wwilvrGJSnfh2eb+TPXE6K2r5Rl6svTwbPqJFuDN1elrEN69fbG+kpPueY9DbnuTv73TMrb1lmon7AWx1CcQeBU14QJie9nqfk5lTI+asm01vDT7K99B7bEhQ+0JMiYgVHWNqn7qvt+GU7+pb5JTzgcez9R80qFzjKN6/pqtAGnZTw0jU3z7wakceNPrAGx0w7BThVF6C0imBYRHotI0WzwTUxrVkStCQsvnfbUl5XkzV27mlTlO5vLn67aFCqvtjUpcu6WGsm01/vw9vIW/LoFm4wm52N+Hd7gC//3kS+6PcSovK69g0ry1NMYIsvUV0X+/s/7yIT/872e8u6g8avxP40dmxNLRKj4IESnB6U8dqjOJSCFwCvBMYFiBSSIyQ0SuSPLZV4jIdBGZXl5enuiwtIgVEN6XIVFIm2G0Bh8v2wA4T+nek2IwoCIMr6ZYbKz824vKeGl2y2vEYYs7wObq9E1MYR0dO+Qm60jgcM59H/KDxz7lk2UbOOme93jsky/jjtleAXH4HZMZfftkRtw6KWrc1yASCQi3AkOsgKhrbN7+5XNzuOPVhdQ1NFFR24CqcvK973HFf2YwbbljQMnNdhadF2Y6f6vqukYeeH8Za11T0osxf8NThvVGMpCcl/GlT0SKcBb+n6jq1gSHnQl8GGNeOlJVRwGn4pinjg47UVUnqGqpqpb26NFjp+ba0c2F8B7OPAExeWFZnCQ3jExy56sLuezf06LGtlTX+50OvcTORMSaJjwu/dc0fvjf9Nq6fLB4PY9OWcHidds4/o/vJNVGtiYSENvhpN4WKG+zb68ihvXt5JuoVNW/91krN1Ny3cssWBO9nExZ5iwfn365yR+rc03FQQFV19DElxuarQJrt9RwwA2vcfnD05i2fGPc767kupepqW/kxLvf9WssxQq8WSs3M235Rv93FGtiCtOgrn92DsNuep3X562l3o2OmrFiE9c/O5t8V0Dc/cbnzFixkQfeX8ZtLzcX0f7sy8307pTvb+dm6Ck2owJCRHJwhMNjqvpskkPHE2NeUtXV7s8y4DlgdKbm6XHOyL5cdPgArj7BaXfhRSJ89uVmnpga/1RiGJniH+8uZXJMxdD1FbVsrHQWoJQCImCjTpZEtay8gpoE5qFfPT+HXz8/lxPveY9l5ZVMXrAu4edsq6nng8XrowTB5qo6/4k3lQaxalMVv3hmjr9d0q0DXQpyfSf3o598yajfvMEX6yuZ4mpUp/7pfT5ft80/Z2l5BYC/2IKzwP748c94MzD3m1+cx9G/f5stVfWoKk9NX0l1fSNvLijjm//4mFWb4iPEvtxYxeKyCn87eI0VGyo5+74P+eY/PvbvM1aDCN6/l5T7zKerAHhy+ip/3+9fX8TjU1dGCdxpyzcRqxys3VpD9465/nYmtAfIbBSTAA8CC1T17iTHdcZpSvRCYKyDiHT03gMnAXMzNVePEf27cNs5w+nftQCAlwMVGddXWG0mo20p21bbbGLaDg2iJsHTe0VtA8f/8V2ue2Z23D5VZX2MxtClMCfuOI+3FpZx0YOfsN+vX+PthWWoKiNvfYNnP3US5BIJIY9HPl4Rtd2tKI/OhTl8+uVmXpz1ld8CeNoXG6Pu/UcBbcgTFkEH8hJ3UQ/WVvME3VF3vcVdry/ijzGhwDNWbCKWVZui/ZDBa8wMSb6L0yDcv4Fqs/nIY9HabSRj6hcb6Zgf/btXjTeJZ4JMahBjgYuB4wOhrKeJyPdF5PuB474GTIqpFNsL+EBEZgFTgZdV9bUMzjWKvOx4+64V7zPaioIc5/tYvq2WT93FK9XzYmVggapKEFHjObLfXxwfb79mSw2VdY0cuFcnf+z7j37KvxPkKHy0dIP//tJ/T2PZ+ujCz94TdE19I+8vLmfR2m08OW2lvz8nEn1HXQpz6OIugD96/DO/NtrPn5nNU9Od8yJZwqKABrF8g3PNoH/AE0wVtc2/A0+h2lrTwN9DoqR++Zyjyfx0XHPjzFitIuik3lId7efp0zmfLdX1VNc1cvPEeayvqA34QDROm1q9uZruRbmEMah7B95aWMZNE+fF7StMwz+zs2TsCqr6Aam/x6jqv4F/x4wtA0ZkZGJpECvhoXUKYxmZobFJ+deHX3DBYQNa5Z+qpSnKz6a6vpF3FpXz5gLH7JTKZBPUIMJCLq9/djb5OeGO7iVlFX656q+P6se8r5rLQ9z84ny+M3ZvILokfuxTd1BgQLMN/t8fLefOVxf6418b1ZecSJafJOfRpSDHj9gCWBh4yp69egsiMOHiQ7js4eYUKS9JLbh4e6aeYA2kdIN9BvXo4L9fudHRIO6/+BDuf3dplBDaHLM2jN2nO0/PWMVLs7/i3x8t54v1lRzUrzPghNuGCey+XQrYVFUfFcX09VF9ycvOihK2l44t4clpK6msa0wZqNASWHxOCGECwjSI3ZdX5qzhtpcXcPekxFnF/5v2Jb99ZdfrpLulut5f0N6c75hGRvTrnNDpu3x9JTNWbPSjmCBcg3h86kr+9eFywAkX9PhoyXrG3f0uf5m8GIAT9u8Zd64nfA66eVLcPo8vyqM1iEXrtrFmS3VctJOXEb5qc7QJp0thTsKeCqpOg699ehb5Y8HFPGjO8u69bGtt4PxoCfGLU/aPu8Z/LhtNcWHzU72nQXTIzSYnkkVdQxOrN1dz4QNTeH7magpzIxS7Jrhj93OCZSa7wvzjZRt8gb6tpiEulBWgR8f8uAU/PydCtw55UWM3nXkgXV1tozCFmbElMAERQp5pELsNtQ2NvD4vvLuXh2eeSFbC/RfPzGHCe8tadG6JePij5WknX464ZZIfQbettoGOedkM7lmUMK/g2D+8wzf+/nFUIpX3PlGcvDc+ecE6LnCzd73Oab065fOHb0Yr87e9PJ93P08eUh5bLqOxSRlzx1t+2KuHt/Cu2lTNEYO70b3IWRA7F+Ty0xP3JRGd8nPoV9zcA2FgoB/CorXbWL25mjVbqv2oomA5kNiksjMO6hP3+UV52VE+F2+eBbkRcrOzqGts4raX5vPhkg0sK6+kU34OL/7oSO469yBGl3QF4DX3e6mqvtDalCChrUfHPL+rpUd+doRuIaYnT2jEHp8JTECEEGpiCvnDlm2r4dqnZqV0wBmZ4+5Jn/P//jMjad0arwRzVgvW6J//1VZufGFu2mWWP1+3jXVba5jw3lJumjiPHz8eHmo6d/WWpN+nA/p0oiAnkjJstLK2wX/QWbyuguP+8I6/6MeiOE/dv4hxVnfIjVCQG+HcQ/r5Y8WFOTw+dSWXPDQ16fUTERsq60UelW2tZWifTr4tvkthDocMLOa3XxsOwCEDi6POK8rLJpIlfO3gvvz8lP3oGnjS3lrTwNg732LMHW/5wjVRvajvHzM4tNlOx/zsKGe456TukBchN5JFfWNTlF+itqGRfsWFfKu0P92L8qKijuobm/NXNiawRPTomBenEeTnZNG1Q5iAcDWI3AiXH7k3vzrtgNDPbAlMQIQQpkFsq22IS4657aUFPDVjVconWCNzrHLNEN5C8PbCMp6cvjLqGPVr9Lfcda98bAaPfLwi7daSJ93zHof9djK/fWVhwmNWbarijL98EBpV5HFISTF52RFfgzj8t5O57aX4ElN34jQAACAASURBVNKLyyrYu7tjdvn5M7P5Yn0lb8xPHKY6eUEZ6yvqeOS7oxk1oAuAb8oIcuSQxLlGYQ9Wsbw+L3oON74wjyVlFVTXN9KtKM/3H3gROp7ZpjDG/OLlLN1z3kh+cOw+/pN2r07RJhmPRHka150ab15yPj+Hnh2bP8uzIBTmOCamuau3Mmf1Fl/LCFoYsrKEohhf17otjoCK9Vd4lHQrjItMy83OijJzeXj32iE3m1+fMZTvHT0o9DNbAhMQIYQJCIj/43pPcYmONzJPjqsVeK0eH/tkBffGhC16Nt+wNpAbK+tSmkvC8P7mntlqzB2T+frfogvojbhlEne8uiDtLF7Pb+DZrsMY0a8z+TlZfujq2q01PPDBF1FJZuDY3g/cq3PUWGITE7y/uJwuhTmM3ac7A9wn6lj7N8B+vRy7/3fH7s33jnKc1Z79/42fHs1RQ7qnvM9Yxt39LgDdi3J9wedlUHvx/T075kedU5QfvZh6T9Ud8rIp6RatEezXq2PKOfzrO4dGf36eo0Esv/N0vjGqWYMqzItEObyH9ulEGJ4A8wTW2hTF9I7fvydFedFCsL6xiexI/HfW05YK88xJ3SbkRsJ/8bGOau8fP50np9aiqq4hYRZteyTbVQu86qXV9Y2s3VoTtSh7AiLMxHTpv6ZGmUvS7Z/gPe1tqqrjf9O+ZM2WGj79sjkevqa+kS3V9dz/7jLWbglfHGKb6KjrLt4W8vc7a8ReHD6oK0cN6UFedoT6Ro2qdDo8xGEcDFGF5Lk8W2vq6V6URyRL2K+3c16YM/Wiwwdy6dgSrj5hiB8FddaIvVhy+6kM7NbB11rSoTTGbNS9Yx7fP8Z5Gu7hPr0fv39Prjh6EL86/QCm/Wocr/z4KCDe/l7q2v2XlVfSOfDUPaRnEeOGRjvaj963BzedOZS/XTjKHztu/578ZNwQfzuosRzQp2PUeDCqaN8EwsfLWyjp5vw+Yk1cvzhlf75zRAl3feMgfnDsYLoU5pLtNiHyek3UNTRxaElXzh65V9S5nhnOfBBthBL+pBXrqPbU4ZxdqFjTqN+84Rd22x1obFL+/eEXO+zH8f6ZvGqd1XWNNClRi7IXQRKmQQTj6CF1QpeH98+5enN1VAbwtx+ayoI1W6NMT7FJVh5XPzGTk+55l8raBu54dQGPh9QQ8hjZvwtPXDGGDnnZ5OU437fY7+P/pkWfP6xvtAaRqFyMqrKtpsG/p3EHOAtq7O8GoEthLjedeSCdC3N8AVFd3+gLai9n4/j9e/LpDSdyyZiB7N+7eRE995B+3HCGU9R5ry4FUZ/doyiPi8eUsPzO0ylwF+jc7Cx+edoBdO2QS4+OeRzQpyPFhTm+xuBxyMBijtynOz8ZN8QvU3Htyfvxxs+OYcygaK2mU342l47dm9OGRzunPa0lS6Izk/fv3Sxo87MjUZaETgU5XHvyfjz+vcOjPst7GPEEZjDzGmDM4G7cfNaBfOvQ/vzcjaLyHnKO3tcx49U2NJETyeLub42MOtczMe3WeRC7M70757NvryJuOGMoFz/Y/HS5KYEGkaF2sDtETRpVM3clXpr9FTe/OJ/1FXX838n7bff5ngpe5ydiOT9XbapigGtq8PIA0vkz1dQ3sbR8MyP6dQ4tXzB39Ra6F+X5i0mwpg/Ae5+Xs1+vIo4Y7CxKhbmR0NINHp+vq2DBmq3c/27yCKpg3oO3AMaaxoKCCqKffDvmZYcmxHlU1Db4ZpF9ehZxzsi9OGVYfHRPEE9ABCOqvN/xoSVd6dohl1vOHsZ5bh/oN356NEN6daSuoYlVm6q48tjBUdFcXgRTMkSERy8/zNcwgjx6+WEAfLzUuZ7nxwhzcIfhmWyyYv7uw/s1C9qsLOHOrw/numfnuJ8V4YqjB8d9lhe8UBKjUY3euyt9OudHCU2PW84axspNVazdUsNbC8v832ts5V7P3NYp3wREm5CXHWHST4+JG09kYmqppiaZYkt1Pdc+NYvffn14Wv+ErYln9gir5JkOnlruxbt7GkBwUfbqEqVTMG7c3e+ypbqe35x9IBePKYnbf8ZfPiAvO4tTh/UGnBo9sVTXN/rj3Ypy+WpLckd2OrX8g5pNrpvp/39PzUp6TnAhLMiNhJquwFnUK2oa/OJvIsK94w9OOad8V5MJ/l7rfa26eVG757yRvDx7jZ+3kJudxU1nHuheqzlxLSxiJ4xY30oiPAdyQYyDO1GZEs8sFNs7IrakxfjRA3hi2kpmrtyc8LOa3JsaEIiQ+s4RJdx81oEJ57tPzyL26Vnka5/fOrS/v+++C0YxuKcjbA4f1I0/jR/Joa5ZLZOYgNgONlbGOqmdf4ZY9XFX47+ffMmk+evYu3sHrs9gSNyO4Dv6c3bMTOc5Xiti6vAHzTrVvvBILci9sgkLQ+rjeDb52oYm37y4IkRA1NQ3+Qlg+dkR1m1NXgl4WUxSWRhBAZHK4ZkbyeKZK49ARLjnvBGM7F/M+AkfJzy+sUmjNIgw3v/5cVEFAAFfSzrzoGYbeUNIz+a9uhQkjLSZ+stxbK2pp6a+scV9eZ0C9YtOObC3n5eQaFE/zk1wGx2y8H7t4L7MWd3cm8J7qk+kjXih1cWFufzrO4fy9IxVUaU7ktG3SwHL7zw9auz0QK5GJEs4e2Sy1jothwmINMnNzorSIF6bu9ZfRNq6MXy67IpizFOjw+pfpYO36HuO+TANwtMuNlfV8drcNZx8YHPt/ERlF4LDTU3KonXborQv77rLAhU+m/c1+ubIqrrGlO0g56yOL/aWJc2myyE9i7jULW8BcM7Ivfizm+kcxo1nDvXNIl872InA8R5iTh/eh5fnrCEnIv5YVV0jVXWNFOUlLv4Wliuwd/cOcQuZZ0fPTtMv16NjXqi5qCUImmb+esHBvDp3LT96/DM/0zmWjvk5zLvl5ND/k3vOi/YDeB+dqF+FZ2LqXJDDmMHdOC4kI313YNfxru6iPHPlEfz5/IMpLszx/+k3V9Xx/Udn+Md4/xS7Kt6XeVfrrf3u5+Ws8cwv7tymLNvAHdtR8sLTQGI1iJUBDcLLJH5zQRnff/RTno+JHgojmAD39KerOPVP70flV6xxneBhZpvahiZf26ysa2DdtuQC4pU58Xk0+wUco5N+ejS9OzeHeQ7qUcQfv5m4VFmYuc4zh3qLcVj/gNjQ0R3hoH5ODsWQQBmM1qa0xPE5BPMYsiNZnOlGW40aUJzoVDrkZacVHeT5KRLVs/K+PrHmrd0NExApOGRgMWeN2IuuHfJYX1HHf6asYOStb0QdU1PfFNea8M3567j3zcS1f1qT7W2YnornPlvFo1NWpD4wCdV1jVzy0FS/Fr73lD9+whTuD5S8UNW4xjBBghqEU9LAc1LHaxAeTwXq7yeiKfDL8j7rH4HKn8nn1Ohrm1W1jZRtraVfcXTEzolDeyW9/mmuj+O80v6hzvI+XfLjxjxi8wCgOeLOExA5Ieacji0QNjn+0P68+bNjOGxQt53+rB3lZyfux8s/PpIhISGo6Wo2qfC0k0QRjzeeMZTenfLp0znx32l3wAREmuzTs4hFa7dxw/PxbSmuf3YOI26dFOWsvvyR6dz7ZmIzQGsieF/mxDQ0NiV14jY2Kfe++Tmbq+p4YupKHp/6JR8tXc8X61Pbz8OIbbpeGbOI1zc2sb6ilmuenMWpf3rfL1QX5O2FZb5dvLK20fcJ5edksXZrjW9uiv3sj5Zu4Gf/m8lfJi9OWBVV1dEi/jftS1a7AiKRkxfgzBHNdvja+ia/pEJdYxNl22oZFuNYPXKf7ky5/gR/+8FLSv33L1w1lh+dMITPbjiRO78xPPR6/YvjhQDAbecMC40+8sygPYqaNYhfn34A54/uz9cPduzZLeEDEJGoInptQSRL0nZk7yheDkdsAp/HuKG9mPLLExJqGLsL5oNIk2F7deLFFAXWtlTX060No4R2JprqlD+9z5Kyijibsse7n5dx75uLWb6+kq01DdQ1NHLBP53CbonOSUbsU32swNhUWcf4f07xHbiL1m1jXOCpe+oXG7k00JJzY2Wd74we2b8LU5ZtZN5XWzhkYFeq6+IX9mc/S25malKn2Fps6CjAuAN68eaCdQzsVsgKN8x19N5d/e/H1OUb4845qH9n30kKTu5MQWDxCC40B7jZucVJonpiNRKPw/YOj2zxFKIebmZvbnYWlx/lOI5//7pT/iM2jNtIzNXj9uWUYX3YLyRctT2RyY5y/UXkbRGZLyLzROTqkGOOFZEtgYZCNwb2nSIii0RkiYhcl6l5pks6TyRhFV+/2lyd0knZUtQkeBpOx8S0JMTZGvXZ9c15Blur63c63yK2JWOwPDXA6N9OjoruifWfxDZpWba+wm8Re+Q+TnTN9OVOcbpEDXOS0aTKrFXhzuO9uztP78FY9vyQp++gU3uEa5v3yIlIlH06WPcotnlOGCLiJ2HNvPFEX9ik6jLm7Q9qC14iWTolKQyHSJYwdK/wMhvtiUyamBqAa1R1KHA4cJWIDA057n1VHem+bgUQkQhwH3AqMBQ4P8G5rcbAELvur0+PDhkN6xlxxJ1vMfq3k9O6xlPTV/La3OSF/z5asp6nZ4Tb0IOhkGFVRhPZS9PBr2eUJa6A2LkKtqk0iPjrOz9VlaPueotX5qyJ2l9T38RSV8j171rIwG6FfOaWvtgRAVFd18hHS5qb3nj2+aK8bF9LzMuO0KdzPnnZWQzsFl9iInhPsSUocrOzogRBMMQ03f7CL/xwLM9fNZYuhbmc58bMd0ogILycBa9L26DuzWagI4d054NfHMepw5Mnxhl7HhkTEKq6RlU/dd9vAxYA6QbvjgaWqOoyVa0DngDOzsxM06N3iLPp5AN7R21vTJLw9PHSDZRc97IfH//5um1xXbiufXp2VHRUGBc88EnCBKngov3YJ81OZC/xJx0ndaLy1Z7TVlG21TbExcQHUVUmzvoqqRBJ5YOIpaLW0RjKttWycmM1zwVMRF60ihennpcdYWifTn6piFTCJ4wPlqyPinv3whQ75uf4CV1VdQ0M7lFEUV42o/fuyvNXjeW80ubkpu8cUeK/95LQPHIiWVGCILb6Zzp0ys9hZH9HM7nhjKHMvPHEhDbvF394JHd+fTiDehTxtwtHcfd50VFQ/RL4NIw9m1ZxUotICXAw8EnI7jEiMktEXhURL82wLxCs2byK9IVLRgirt9QpppF4olK+AE/NcG7n/cVOeYST7nmPb/z9o52aU1VdQ1R9naDZ54YX5vk+iXSriQIJF35Pg6hKI+nsxdlr+PHjn/kdy8KIbYMZ5icIsrGyHlXlvZDKqyP7d6FjXjZ/nLQIcEIL9+vdkeUbKqmsbUjbHHb1Cc3F2ipqG6LCJL3Y+W5FuX4doIraBk4/qA8nuQ8KI/t38c1Gl4wZyLUn70eWOFpXVpb4pash/vu0s70qIllCl5DS0B5DenVk/OgBAJw2vE/cd9cwwsi4gBCRIuAZ4CeqGhsb+CkwUFVHAH8Bnt+Bz79CRKaLyPTy8u0v27wzxMY4J3Pyefbo9RV1oVUyd8TB/LX7PqL0tjf97dgn9pqGpuhuVlXh1w6SqG6+d146lWKnfeE4aZOtebFmn1gfRCybquq49aX5XPt0fK+EbkV5XHPSvs2x5zkR9u/dEVV8P0J2GgtwUEAA/GTcvox3TTenH9SH80r7c+95I33ncVVdI+ePHsAdX4+PNOrVOR8R4dMbTuTj648H4J1rj/NNPen4GQyjrcmogBCRHBzh8JiqPhu7X1W3qmqF+/4VIEdEugOrgf6BQ/u5Y3Go6gRVLVXV0h49EjczyQSxYYHJ2pJ6jT8mzVvLxQ82K1Ke8zV4bqpF3CNVJdKa+kbue3sJf3Pj91+Y+RU3TYwP0w0Sm8/h4ZmYErXtfPij5bzq+gUWuRnmifpkPPzR8qjmNYW5Ebf3cuL73lhZx4uz1oTuy8vO4oxAmGm/4gL2dm3sC9c4c4mtHBpG7FP8UUO6c/vXhjP/1pPJy47wu3MPYlCPIt8fESYsvXwD75guhbl+hFLnghzf1BSWqGYYuxoZC3MVx8D6ILBAVe9OcExvYJ2qqoiMxhFYG4DNwBAR2RtHMIwHLsjUXNPlmhP3ZdL8dVG26SBhTmoPz9w8a1X0uVtrGuhckBPlv/hifSUfLlnPt8cMTOiwDFtMY00p1XWNPDolugT0s5+u5rZzwmPrnfmECwivLEOYCUpVuWniPGfud5zGio2V7rHxWsGMFRv9Yz327dWRmSs3J61ZtHJjFRsq6+jTOd/PYg6e370oj99+bTj9uxawV5cCP5t4cZkjIPp0zo8qrNe9KJe9u3fgm6X9WVZeyYh+0VFqRXnZfnmJ2LLKnrC58LCBcfOsb0ivdEhYotqgHh0oT1G3yTBak0zmQYwFLgbmiMhMd+yXwAAAVf0HcC5wpYg0ANXAeHVWvgYR+SHwOhABHlLVebEXaG1+dMIQfnTCEEquezl0fzITU6zN3WNzVR2dC3LYUNm8MHgdtob365ywLEBYglesBlHb0BjXkSqVqWVrdbiA8JPOQkxBQXPRvK+2Uub2HQ5zDr+9MN4MuH9vR0AcfkfiaC+v4uk1J+3HxFlfRfkivC5mFxw2wB/zms4vXudENsVqEEN6duTxK6Jr+AeZe8vJCfd5ncbC8DSIVElnng/i4e+O9vtUvPnTY3bJelnGnkvGBISqfgAkXY1U9a/AXxPsewV4JQNTywgisKkysYkpGLffo2Me3z9mML95aT6bquoZ2C08Aqpsay2vzlnDCQf0iltwgsKgsUmJZElcDZ7quqY4Z2iqhSuRD8ITSBUh+4MNiibNW+tHS9339lJG9i+OKisR1jw+UVcugL9fOIrsSBbfe2Q64JiPOgR8PxcfPjC0kBw4VTE/X+eZmFqn5IEXEJCqiZTngzhm32az6M46qg2jpbFM6haiX3FBUg0iKABKBxb74YmT5q1lZP8uzFoZn5R175ufs3DtNn4ybgg/GbdvVDRS0JxU19BEQW4kqosZQE1DY5zG4PVPSERsb2OPWlcg1aVwpn+8bEPU9vcemc6vTz+Af7y7lO8fM5inQnI4vJ7Gsfxk3BBOHd6HFRuaE+b6FRf45Zo7F+Twm3OGJZxL3y4FzPvKiYvo0zlag0iUavDhdcfvVI9xr9tXMGIp+rrOhXelLoSGkQj7lrYQ/boUsm5rjV80LpZX566hICfCuAN6cfW4If4C8rd3lrK5yikCeM7IvXjhqrH+OV4msbfIBU02QQ3Cc1bHdi6rrmuM60aVk538KTVRSGiiLO1Ypi3fRG5MGYnbXl7A+oo6bns5vEprohh8z47fv7iQiw4fwLfHDGSvzs0aREGKOjd9A+Uo+nZJT0D07VKwU02VfnnaAdx17kGMGZy8WF1s1zLD2BUxAbED/ObsA7kt5sm1X3EBW2saOPCm10P9A/WNSnFhDg9cUsr+vTvRM5A4Ne+rrdTUN3HE4O5RGdve07qXXBc0IdUECuudc9+HLC2viOt97PQKjhEQkSxWbqzig0D7yWA/i9hs7Pc+L6ehsWm7MqcP7NspqhpqMm47Zxj79Cziw+uOj9vnhYRmZQm3nTOcW88eRlaW+BpEfoomQ0HBk04UU0tQmJvNtxJUYA1i8sHYHTABsQNcPKaEiw6PjmAJLkb73/Ba6HmRwGJdlJftC5m5blRU7875dCnM5cCYGi+rN1ezsbIuqgxH7JP+F+WVrN5cHdPUpjHOpJQlwlF3vc1FgVDbIb96tfmcgOD5w6RFfPuhqUxeWLZdAmJk/y4Jq6SCU/PnxKG9ePGHR/q/x9gnfEgcCeQJiFQ2++BnFndwNDavEJ5hGKkxAdFCeAtQMhpiWpMO6+uEVt7xqlNN06sd/3zAzAROhvZ5938cZaL5z8fR/Rguf2Q6y8or/UJy4AiI2ISsDRXRYZRxkU8BweNlQlfXNUYt+KmapR+7X/LuWfdffAj//HZpVDP4MBJpCJ6JKfb3GUuw4mmPojyW33k61526P9BcAr21udCNtMpUFzXDaElMQLQQngP5pCSNYGJbk3aLKefs1XvKiWT5zuVBbpG3xTHVVp/5NLxgX7AoXE19E7E5d7EJeWF+Cw+vuF9lXUOUIAkWpgv6ATzn7pgkzWKuPHYwJd3DndIj+ndh1IDmqqeJNIgDXcGaqoSIp0EU5WX7Jh9PLMSa3lqLy48axPI7T7dSF8ZugUUxtRAXHDaAzVX1XHT4QCbFNLcZ4UYpxZpdusYIiI6BRcMrsFdaUsyy7WjKMyAQ8lld3xi1sP+/owdFdWurbWiMas0J8L/pK8nPyeLbR5T48fnPfro6qrDggK6FfrLgh9cdz6G3v0ljk/LMlUeQn5OVNJQ22cLoOegPuvl1ttY0JNQgDi3pyoSLD0lYudSjuEMuvzrtgKh+wGMGd+PCwwZw1XH7JD3XMAwTEC1GYW42/3fyfqH7ThvWm1krN8dpEIWBeP5g5c8gpwzr7bflTIfegXDOmhgBUb4t2rxUXdcYp0EAPPzxCh7+eIXfmze26myngmxe/OGRTJq/lq4dcnnhqrE88vFyhvbp5PsFSgcWMz3mPO/cVHQrymNrTUPSbOSTYirpJuJ7Rw+K2s6JZHH71xJnkhuG0YyZmHaSXp3ibcnD+kY7Qj3HcX2MzTwY6XLzWQcSxlFDesSVik7GXoGy5I4G0USH3AjPXDnGL5PhtZicvXoLT09fGfo5EB01FUxO61SQw/B+nbnmJEcgDuvbmbvOHRHlNH7kstGcP7o5s9k/Nw3Timd6S5GyYRhGhjENYid57+fHxfVZePr7R/DmgnW8Pm8dL876yndIhhXh+/uFo6JCXj0uGTOQDZV15ESyGNKriLUputKdOqw3r85dy+GDuvmmpNp6Jzz1nIP7csjArtx0ZgH79e7EoO4dePaz1Vz6r2lJPzPIL08/gNfmruX9xevTWuQLc7MZEtKbOJVZCKCnK3S3Vm9/HwfDMFoOExA7SZgZJD8nwhkH7cVJQ3vz7TEDo/wCsSTq4nXL2c15FsP6dub9QN5CGH+/6BD//fWnHcCLs76iotZxLntNZPbqUsDPTtyXSfOiu9Z165Dr1zoCuOjwAXFF/ooLc/3s33QWeYCikGinLmmce/NZB9IhN9vvwWAYRttgSnwGyc3O4tCSrr4tf0cZGhO7/6fxI1Oe071jHuXbaqlpaIpz9ga7jl142IC43rrDQvpvOwLCMSGl6nscPMfjoe+Uctc3DmJ439S9vXt2zOf33xyRsDuaYRitg2kQrUBh7s4tdKMGRld0PWpI6ifrnh3zWbRuK41NSn6MlhNceIsLc+MK6MVGV4GT5+FrECnyIJrnEOjItm9PK0ZnGLsZpkG0AiLCIQOLuTOk81g69O1SwGc3nOhvd8hLLXB6dcpj5UYnQim2810wd6G4Q25c4biOIT6GrjtgYuoZcOCbcDCM3Q/TIFqJZ648YqfOLw481afTjaxXwPF9VqDbGkRnKBcX5oQIiBDfwQ6YmHam6J1hGG2PCYjdiJOG9uLgAcUpC8FBcz/oS8eWxEVJ5cdoELEaSaf8HESIis7Kzc4KmJjSExBW0towdm8y2XK0P/AI0AtQYIKq/inmmAuBX+BUQNgGXKmqs9x9y92xRqBBVUszNdfdhQnfbv4VZAlxZTSCDOrhhJgeHeKvCAqIroW5/Pzk/cnOyuLJ6SupbWiiKD+bnEgWdQ1NXHPivr6JqtnEZM8VhrEnkMn/9AbgGlX9VEQ6AjNE5A1VnR845gvgGFXdJCKnAhOAwwL7j1PV5PGdeyi52Vl+RdeHvzs6bv+pw3rz1jXH+IIiSNDE1KtTPsUdcvnNOcMY3KMDN784n4752eS5AuLY/Xr6RfWKC3Pp1iE3Zb/lIKcf1IfGFEX1DMPYNclky9E1wBr3/TYRWQD0BeYHjvkocMoUoF+m5tPe6NulgKXllfxp/MiotpUeIhIqHCBagwhmgn9n7N58Z+zegNuatBYKcpuFyeVH7c3XR/Xdrnned8Go7TreMIxdh1YxEotICXAw8EmSwy4DXg1sKzBJRGaIyBVJPvsKEZkuItPLy8sTHdbuGNLT6eOcquR1GEHfQCJ/hldwLzfSLEw65GUn7P9sGEb7I+MCQkSKgGeAn6jq1gTHHIcjIH4RGD5SVUcBpwJXicjRYeeq6gRVLVXV0h499pzMW6+pUHlMf4d0uebEfXnmyjEJ9+/jlsloq7LYhmG0PRn1NopIDo5weExVn01wzEHAA8Cpqup3vFfV1e7PMhF5DhgNvJfJ+e5OXH7UILbVNnDBYfEF8dLhRycMSbr/L+cfzAdL1rdaq07DMHY9MqZBiGO7eBBYoKp3JzhmAPAscLGqfh4Y7+A6thGRDsBJwNxMzXV3pCA3wi9POyBjjWe6FOZyxkF7pT7QMIx2SyY1iLHAxcAcEZnpjv0SGACgqv8AbgS6AX9zbeFeOGsv4Dl3LBv4r6qGN3o2DMMwMkImo5g+gOSNf1X1cuDykPFlwIgMTc0wDMNIA0t1NQzDMEIxAWEYhmGEYgLCMAzDCMUEhGEYhhGKCQjDMAwjFBMQhmEYRiii2n4qbYpIObBiB07tDuxpVWPtnvcM7J73DHbmngeqamidonYlIHYUEZm+p/WbsHveM7B73jPI1D2bickwDMMIxQSEYRiGEYoJCIcJbT2BNsDuec/A7nnPICP3bD4IwzAMIxTTIAzDMIxQTEAYhmEYoezxAkJEThGRRSKyRESua+v5tBQi8pCIlInI3MBYVxF5Q0QWuz+L3XERkT+7v4PZIjKq7Wa+Y4hIfxF5W0Tmi8g8EbnaHW+39wwgIvkiMlVEZrn3fYs7vreIfOLe3/9EJNcdz3O3l7j7S9py/juKiERE5DMRecndbtf3CyAiy0VkjojMFJHp7lhGv997tIAQkQhwH07f66HA+SIytG1n1WL8GzglZuw6YLKqDgEmmK79cAAABMFJREFUu9vg3P8Q93UF8PdWmmNL0gBco6pDgcNx+pgPpX3fM0AtcLyqjgBGAqeIyOHA74B7VHUfYBNOz3fcn5vc8Xvc43ZHrgYWBLbb+/16HKeqIwM5D5n9fqvqHvsCxgCvB7avB65v63m14P2VAHMD24uAPu77PsAi9/39wPlhx+2uL+AF4MQ97J4LgU+Bw3CyarPdcf97DrwOjHHfZ7vHSVvPfTvvs5+7GB4PvITTmKzd3m/gvpcD3WPGMvr93qM1CKAvsDKwvcoda6/0UtU17vu1OK1doZ39HlwzwsHAJ+wB9+yaW2YCZcAbwFJgs6o2uIcE782/b3f/Fpy2v7sT9wI/B5rc7W607/v1UGCSiMwQkSvcsYx+vzPZk9rYhVFVFZF2F+MsIkXAM8BPVHWr29ccaL/3rKqNwEgR6QI8B+zfxlPKGCJyBlCmqjNE5Ni2nk8rc6SqrhaRnsAbIrIwuDMT3+89XYNYDfQPbPdzx9or60SkD4D7s8wdbxe/BxHJwREOj6nqs+5wu77nIKq6GXgbx8TSRUS8B8Dgvfn37e7vDGxo5anuDGOBs0RkOfAEjpnpT7Tf+/VR1dXuzzKcB4HRZPj7vacLiGnAEDcCIhcYD0xs4zllkonAJe77S3Ds9N74t93Ih8OBLQG1dbdAHFXhQWCBqt4d2NVu7xlARHq4mgMiUoDjd1mAIyjOdQ+LvW/v93Eu8Ja6RurdAVW9XlX7qWoJzv/rW6p6Ie30fj1EpIOIdPTeAycBc8n097utHS9t/QJOAz7Hsdv+qq3n04L39TiwBqjHsT9ehmN7nQwsBt4EurrHCk4011JgDlDa1vPfgfs9EsdGOxuY6b5Oa8/37N7HQcBn7n3PBW50xwcBU4ElwFNAnjue724vcfcPaut72Il7PxZ4aU+4X/f+Zrmved5alenvt5XaMAzDMELZ001MhmEYRgJMQBiGYRihmIAwDMMwQjEBYRiGYYRiAsIwDMMIxQSEYaRARBrdCpreq8Wq/opIiQQq7hrGroSV2jCM1FSr6si2noRhtDamQRjGDuLW57/LrdE/VUT2ccdLROQttw7/ZBEZ4I73EpHn3N4Ns0TkCPejIiLyT7efwyQ3IxoR+bE4/S1mi8gTbXSbxh6MCQjDSE1BjInpvMC+Lao6HPgrTpVRgL8AD6vqQcBjwJ/d8T8D76rTu2EUTkYsODX771PVA4HNwDfc8euAg93P+X6mbs4wEmGZ1IaRAhGpUNWikPHlOM16lrmFAteqajcRWY9Te7/eHV+jqt1FpBzop6q1gc8oAd5Qp+ELIvILIEdVbxOR14AK4HngeVWtyPCtGkYUpkEYxs6hCd5vD7WB9400+wZPx6mnMwqYFqhWahitggkIw9g5zgv8/Nh9/xFOpVGAC4H33feTgSvBb/LTOdGHikgW0F9V3wZ+gVOmOk6LMYxMYk8khpGaArdjm8drquqFuhaLyGwcLeB8d+xHwL9E5FqgHLjUHb8amCAil+FoClfiVNwNIwI86goRAf6sTr8Hw2g1zAdhGDuI64MoVdX1bT0Xw8gEZmIyDMMwQjENwjAMwwjFNAjDMAwjFBMQhmEYRigmIAzDMIxQTEAYhmEYoZiAMAzDMEL5/9nmktaT8b4KAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gJPj8vGdARXx"
      },
      "source": [
        "## 指數平均移動值\r\n",
        "使現在的資料會被過去的資料所影響  \r\n",
        "時間點越近的資料影響越大，越遠則越小\r\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IV0MehTPA81F"
      },
      "source": [
        "def smooth_curve(points,factor=0.9):\r\n",
        "  smoothed_points=[]\r\n",
        "  for point in points:\r\n",
        "    if smoothed_points:\r\n",
        "      previous=smoothed_points[-1]\r\n",
        "      smoothed_points.append(previous*factor+point*(1-factor))\r\n",
        "    else:\r\n",
        "      smoothed_points.append(point)\r\n",
        "  return smoothed_points"
      ],
      "execution_count": 16,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-PJVSy5tBpog"
      },
      "source": [
        "省略前10個資料點，並套用指數移動平均值"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YFNsX-v8Bi8F"
      },
      "source": [
        "smooth_mae_history = smooth_curve(average_mae_history[10:])"
      ],
      "execution_count": 17,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5kzV3NEXB7H_"
      },
      "source": [
        "### 圖3.13 每個訓練週期的驗證分數MAE，不包括前10個資料點"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YcFE8gBZCGnX",
        "outputId": "cc2b839c-4594-4626-bee8-2da1808ac5a3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 279
        }
      },
      "source": [
        "plt.plot(range(1,len(smooth_mae_history)+1),smooth_mae_history)\r\n",
        "plt.xlabel('Epochs')\r\n",
        "plt.ylabel('Validation MAE')\r\n",
        "plt.show()"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYkAAAEGCAYAAACQO2mwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3hc1bXw4d9S7726yg1ccANhDDa9d5KQBEiAEAKpBC5pJLlfCiGVhJBOaCEJBEJo4VJNB2Nwxca9V9lW712a9f1xzoxG0ow02BppZK33efQws8+ZOfsYadbstraoKsYYY0wgUUNdAWOMMZHLgoQxxpigLEgYY4wJyoKEMcaYoCxIGGOMCSpmqCswkHJycrSoqGioq2GMMcPGypUrK1Q1N9jxIypIFBUVsWLFiqGuhjHGDBsisruv49bdZIwxJigLEsYYY4KyIGGMMSYoCxLGGGOCsiBhjDEmKAsSxhhjgrIgYYwxJigLEsYYM4y9uqGU+9/ZQacnPNs+WJAwxphh7LHle/nHe7uJjpKwvL8FCWOMGaY6Oj28v6OSBZNzwnYNCxLGGDNMbTxQT0NrBydNyg7bNSxIGGPMMLW7qhGAyXkpYbuGBQljjBmG9lY18bPnNwIwOjMxbNexIGGMMcPQt55Yw/7aFgDSEmLDdh0LEsYYMwxFSXhmM/W6zqBcxRhjzIDKSApf68GfBQljjBmGqhrbiBJ49dZTw3odCxLGGDNA1u+v5fVNpXjCtPrZX2VDG+dMLwjrzCYIY5AQkbEi8oaIbBCR9SJyc4BzviUiq92fdSLSKSJZ7rHzRGSziGwTkdvCVU9jjAlVTVMbrR2dQY9/6eGVfP6hFby0/mDY61LV2EZ2SlzYrxPOlkQH8A1VnQ7MB74qItP9T1DVO1V1jqrOAb4LvKWqVSISDfwJOB+YDlzZ87XGGDPY5tz+Cl/4+4qgxyvq2wB4dvX+sNaj06NUNbWRnTyMg4SqHlDVVe7jemAjMLqPl1wJPOo+ngdsU9UdqtoGPAZcGq66GmNMf7wJ9N7ZWhHwuMejtLitjGW7qsJSh7e2lPOzFzZS2diKKmSnxIflOv5iwn4FQESKgLnA0iDHk4DzgK+5RaOBvX6n7ANOCPLaG4EbAcaNGzcg9TXGDJ1XNpTynxV7+dJpk5g7NgPpZ6pnRUMrOYPwYVnf0t7P8Q5UITU+huqmNjo9OuBJ9779xBpK61pRdQLWxNzkAX3/QMI+cC0iKcCTwC2qWhfktIuBd1X1I4dfVb1XVYtVtTg3N/dwqmqMiQA/f3EjizaU8vE/L2HVnpo+z73rlS0U3/EqOysaw16v2uauIOH9kPZX0+x0NU3MS0EVqpvaDvuab24uY29Vk+95tBsw73tnJwBH56ce9jX6E9YgISKxOAHiEVV9qo9Tr6CrqwmgBBjr93yMW2aMOcLFx0T7Hq/ZGzxIdHR6+P1rWwH4YE912OvlHyTKG1p7Ha9pco5Pcr/dVzYcXpB4b3sln/vbcr7z5IcA1LW0+1ZYA6QnxpKbGv4WVDhnNwnwALBRVe/q47x04FTgv37Fy4EpIjJBROJwgsiz4aqrMSYydHqUHeUN3HDyBPJS41m3v5aH3t3J0x/s63bejvIGlmyv9D3feCBYJ8XAqWvu8D1ev7/39bxBZFKuMyW1MkAgCWbJtgp+8tyGbmUPv78bgL3VTkti3b5aANISnFGCqQWp/XbFDYRwjkksAK4G1orIarfse8A4AFW9xy37GLBIVX3tRVXtEJGvAS8D0cCDqro+jHU1xkSAvVVNtHZ4mJKXyqwxGTy1qoSnVjmdCB+bOwaAbWX1nHXX277XJMVF88Lag6TEx3LTGZOJGoBxgJb2TraU1vPXt3fw08uOISMprltLYsm2Ck4/Oq/ba2rc4xNz3JZEY+gtiavud4Zrv37mFNITY/F4lPd3OEFwb1UzP/6/9aQnxiICZ08v4MlV+5g7LvOw7jFUYQsSqroY6Pf/lqo+BDwUoPwF4IUBr5gxJmJtKa0HYEp+CsdPyOLVjaW+Y9c/tJz7rinmL2/u8JXFRUdxwcxCnli5j9++uoW54zI45ajDH5v87lNrefoDJzhNK0jla2dM8QWJouwklu3sPXxa645BTMr76C0Jr+3lDRw7LpMVu6upbGzjU8VjeHzFPv727i4m5SYzrSCNTDcdR7gX0XnZimtjTMTYWtYAOB+AE3KSefe2M/j4sc7M+dc2lfHsmv28sPYAnzh2DP/voun892sLOGZUmu/1f1+ya0Dq4Q0QAO9uc77R17mzm2aMSqesvncAKK1rJTpKGJ+dREyUsLe6mfZOT7/Xamrr6sba5t7/X9/aTk5KHD+6ZAZfP2MyANvLG7nhlAl87YzJ3HTGZC6ZPerQb/AjsCBhjIkYW0vrGZWeQKqb+np0RiK/vnw2P/vYTGKjhVv+vZrm9k6uXziB6xdOYFphGlMLu4LE6z1mAx0Kj0dJiusaPP9wXw0ej1Lb3E5MlDAmM5HKxrZeM5w2HKhjUm4y8THRzJ+YzQOLdzLjhy/7uo2C2XSw3vf4pXUHmf+z13htUxmnHJVLUlwM0/zu72Nzx5CRFMc3zjmauJjB+fi2IGGMCYvP3r+UHz370YYSt5c3+rprvKKihKtOGMcJE5wtOudNyGK6X+vB+/iGkyegCqv7mBEVSElNMxf94R3+9MY2AHZWNtLU1sn/XjiNn1w6g8a2TvZWN7Fo/UGm5KeSnRJHW4eHhtauFkBlQyuvbypjxqh0AK5bUARAW4cnYNcUOAFxyfYK3yB4RlIsr28q42CdM4NpTIazkdC47CQAEmOjA75PuFmQMMaExeJtFTz0Ebt/9lQ1Md79UOxpbJZTXjy++4BtWkIsu35xIV893emWCdQVFMz28gYW/OJ11pXUcefLmwF4dYMzDnLujAKmux/6D7+/m+3ljXzp1IlkJTvTTqvcgen2Tg+f+ut7AMwc7Zx/5rR8tv30fEalJ7CrMvAajrN/+zZX3beUdftqSU+M5axp+d2Oj3KDxAR3IPwHFw9NZqJBWXFtjBlZOv2yoKpqSFM1a5vbqW1uZ1xW4CDhzVM0MTfwgG16YixxMVGU1bUEPB7Iqt2911e8tqmMGaPSGJuVRKZ7zX+8txsROO3oPN9rKhvbGJ+dzMvrD7K9vJGrThjHFfO6lnfFREcxPjuZ3ZW9u7/8u6oeX7mXEydmk57YfX8Ib5BIioth1y8uDPmeBpq1JIwxA67Kb/pnVYhTQb1jCWMzAweJL582ie9dMJVL5wQesBUR8tPiKf0IQaJn11RHp4eNB+o41p1emhIfQ05KHK0dHqYVpJGeGEuWGzi8i+Xe31FJSnwMt18yg6S47t+7i3KS2F3ZyLqSWl5cewBwptf6d0GpwgkTsrnh5ImcN6OA1HjnPfLTEkK+j3CyIGGMOWwHa1uo9gsGlY1dXT7f/M+akN7DFySCtCSS42O48ZRJxEYH/9jKT02gtC607qaG1g5e2VDKyVNy+O75UwEnaNS3dDAlv6u1MtoNWseOzwDwpee+/bn13LVoM0u2VzJ3XAYxAeo1PjuZioY2LvrDYr78yCo8HuX7T6/j0/e+3+28EydlU5CewD1XH8c/v3ACZ03L93UzDTULEsaYw/LWlnLm//w1Lvvzu74yb8psgDe3lPebHO/+d3bw5UdWIQJFh/HhmJ+WQElNMx6P0ulRat1UGZUNrbR1dJ+O+uTKfZTVt3LLWUcxa4wTAF7dWAYEXoPgHW8oSEtg4eQc9lY18/vXt7GjvJF5RVkB61PUY3xld1VTt9Xjr/zPKXzupCLmjsvwlc0Zm8H91xYP2uyl/kRGLYwxw9KeyiaufXAZQLe+d29L4ocXT0cV1uyt7fN97nh+IwCZSXGkxB/6UOm8CVnsqWrij29s43evbWX27YuobmzjuDte5dbHV3c7983NZUzISea48ZkU5Tgf5v9evgcROMovcV6Hu9bh6AJnFlVMdBQPf+EEdvzsAt85xUGCxPjs7gFvXUmtLzdVYXoCU/JT+dElM/psHQ21yK2ZMWZIeTwatH+/rK6Fl9Yd4K0tZb4y79h0bVM7m925/2dNy0fEmekEXR+4PcVGOy8+nAABcM2J4zl3Rj73vr2Dh951MqU+unwPAM996IwJqCr/WbGXd7ZWcMqUHMDpphKB6qZ2zpqW3y31+K8un8Unjh3DDL9pt0C39B9zxmYQSM+ZWv+3Zj/N7Z1cPX88//3qgsO618Fis5uMMQH9/vWt3P3qVpbcdgYlNc1MzEn2bXLz5ze3+6a3ZifHcc6MAh5dtofWjk7OvOstKhpayU6OY0xmIudOL+Cf7+1ib1UTS3dW8s63zyAxrvuc/5yUeA7UtvDnzxx7WHUWEb546iReXt+VzuM/K7q6d7aXN/CT5zbw5uZyAK5fOBFwPvC9E46umtd9X5oZo9L5zadmB7zeizefzJbS+l7345UUF0NearxvWu4id3rthbMKyYuQgen+WEvCGBPQs2ucLTgX/vJ1PnnPe3z/6XW+Y/trmn2PLz9ujK9PfV91MxVuzqJRGYmICDeeOpHGtk6eX3uAioY2pv3gJd7Y1NUCUVUqG9r44qkTOcbt9z8cc8ZkdNvW03+viVsfX+MLEK/eeopvoRrgGwM4aXJ2yNeaVpjGpXP62nATRmc6U1lHu1NagaDTfCORtSSMMYG536y9Sx5eWn+Qu1/dwmPL9nKwroWpBamcclQuXz9zCh+4mwOd+Zu3fC/37qswZ0zvrpibH/uA1795Gjkp8dS3dtDW6SF3gHaXi4oSfv7xmazeW0NmUhw/fWGj79iH+5x6RglMzuu+Yc8LXz+Zqsa2bvtZDIQCt8Vw9vR8X+srUqa3hsJaEsaYXjwe7Zbq+u1vnc7UglTufnWrL23E9FFpfO+CaSTHxzB7bDoJsV0fJz+57Bh+fOkxgPOhfcdlx/CDi6Yj4mRubWzr5KF3dwFQ4XbFeKeWDoRzZhTw7fOm9soIqwqpCTE8+7WFvV4zOS+FeRMCD0AfDu+iOP+WxEBvaxpO1pIwxvTyxKp91Da3Ex0lXDpnFOOyk3jx5pPZXt7Ap//6PpWNbd26dFITYln6vbOY/eNFzBydztXzx3d7v8+6zz9ZPIbY6Cgu+9O7rNvvzHg64O62Fo59qr2zlgB+8fGZ3PbUWn5w0fQB6dYK1VdPn0xpXQuXHzeGdftrST7MwfnBNrxqa4wZFE+t2sfR+am8ePPJvlk8IsLkvFS+fNok7nh+Iy3t3WcqpSfG8vzXFzImI3h/uze76/RRaSze6sx4evj93aTGx/jWKgwk/66jTxWP5bSj88gbhC0//WUlx/HHq5wB+d9dMXdQrz0QrLvJGNNNp0dZu6+W+ROzAu7y5s10elR+7wVnM0alk54U26u8p+mFaZTVt7KzopE3NpfxsWNH98pdNNCiooSC9IQB2bluJLGWhDGmm61l9TS2dTI7yNz/Eydl89xNC5lemBbweCgumFnIrxdt5pv/WUNLu4djRoWv++fRG+azrbwhbO9/pAtbS0JExorIGyKyQUTWi8jNQc47TURWu+e85Ve+S0TWusdWhKuexpjuvAvhpo8KHgSOGZ1+WN/IR2UkcsExhax0M6oeVZDazysO3YmTsnuNkZjQhbO7qQP4hqpOB+YDXxWRbgnRRSQD+DNwiarOAD7Z4z1OV9U5qlocxnoaE/FeXn+QBxfvHJRredNrjM8Kb4K5OX75iqYM0n7N5qMLW3eTqh4ADriP60VkIzAa2OB32lXAU6q6xz2vrNcbGWP44j9XAs6OZ6HszXA4dlU2UpieEHQV8UCZ7Q5UTy1IHXYzfkaSQRm4FpEiYC6wtMeho4BMEXlTRFaKyDV+xxRY5Jbf2Md73ygiK0RkRXl5+UBX3ZiIsr829L0SDtXuyuC7ww2kWWPS+cllx/CvG+aH/Vrm0IU9SIhICvAkcIuq1vU4HAMcB1wInAv8PxE5yj22UFWPBc7H6ao6JdD7q+q9qlqsqsW5ubmBTglZz43NjYk0mw/2/BMaeLsrGynKDv9eBiLC1fPH+zbxMZEprEFCRGJxAsQjqvpUgFP2AS+raqOqVgBvA7MBVLXE/W8Z8DQwL5x1rW9pp/iOV/niP1dYsDARa11JeINEfUs7FQ1tvVJcm5ErnLObBHgA2KiqdwU57b/AQhGJEZEk4ARgo4gki0iq+z7JwDnAuiDvcdjK6lv43N+WU9nYxsvrSyn/CBupGxNuTW0dvsdvb+m7S7W1o5NP3rOEd7YeWterd9B6Qs7wSUBnwiucLYkFwNXAGe401tUicoGIfElEvgSgqhuBl4APgWXA/aq6DsgHFovIGrf8eVV9KVwV/d2rW31T8QB2Bdi43Jih4t1LOTc1nlV7qmlo7Qh67t6qJpbvqubzDy1nybYK7n9nx0e61q5KJ2OqtSSMVzhnNy0G+p2Goap3Anf2KNuB2+00GBJiu8/i2F3ZGJZEX8Yciio30d7Jk3N46oMS9lQ2BV3DUObu79zeqVx1vzNP5PqFE0KeEbWrwhskrCVhHJaWAydtsFd0lLCnyloSJnJ4d4fzroAu8dvLode59b1nPzW2dYZ8rfX76xifnURSnE1JNQ4LEnQ15wFGZSR026vXmKG2vdz5dn+yu9VmSXXw38+DtU5L4q5PzeZKd4e10roW7n51Cwdqm1myrYJP/fU9im57vtvGQe9tr2TmD1/mxXUHw5Jozwxf9nUBqGhsoyg7iae+soAvP7ySg4MwF92YUG0tq6cgLYEJOcnExUT1uVaitK6F1PgYPn7sGHJT43l02R7e3lLO3a9uZdnOKpZsr/Sd++G+Wt9eB69uLKXeHevwbhZkDFiQAKCyoZWJuSlkJcdRkJ7Aqj3V/b/ImEGyrayBKfkpiAijMxIpqQ7e3VRW30JempMKO9dNib3DbYns79FN5d9t9cGeauaOy+CyOaO5dM6ogb4FM4xZdxNOd5N3A5WC9ARKa1ttrYSJCB6Psq2sgclubqPRGYns62NMoqqxjWx38x7vdqDr3c19es7a21bmZEZtae9k3f46isdncu1JRWQk2eI202XEBwlVpbKx1feHVZiWQFunxzejxJihtL+2maa2Tqa4+zH315KoaWr37cuQmRRHXHQUq9z9p3vaUupke121p5q2Dg/zJ2YPcO3NkWDEBwmA9757Jl88ZSIABelOH+0BG5cwEWCr+21/irvBz+jMRCoaWjlQGzhQ1DV3BYmoKOHy4jHdjvtvEbqupJb2Tg/vbqsgOko4wYKECWDEBwkRISclnky3uynf7c8tCzCV0JjBttX9tj85t6u7CeDEn78esEu01i9IANxy5pRux6cVprLz5xfwhyvn0trhYeOBOhZvq2Tu2AxSLBOrCWDEB4mevMnGqhvbh7gmxsDyXdWMyUz0fYkpSE/wHStv6J4+pr3TQ2NbJxl+QSIvLYFzZ+QT4y4GuuHkiYgIc9w1F0u2V7J2Xw0LJueE+1bMMGVfHXrw/jFWN9mYhBla7Z0e3tteySV+s41mj80gOkro9Ci7K5vIS+0KGrXNzhebnntM//XqYl+rw7vyelRGIrHRwuMr9uJRWDjFgoQJzFoSPaTGxxATJRYkzJB7+P3dNLR2cPb0fF9ZSnwMr916KgDffWotLe1dq6l9QSKxe5AAJzj4p+aIjhJGZSSyo7yRpLho3wZAxvRkQaIHESEjKY4q624yQ+zfy/dy3PhMTjuq+z4pozOdcYltZQ08/P5uX3lNk/M7mxYgSATiHd84YUIWcTH2UWACs9+MADKTYqm2KbBmCHk8ys6KRuaOzeiVnC82OoqbzpgMwBMr9/nK69yWREaIQcKbWeDCWbZ4zgRnQSKAzOQ4624yQ6qkppnWDg+T3EV0PX3jnKO5bkERe6qafOMN3sVx3pZGf65bUATABTMLDr/C5ohlA9cBZCbFstNNmWzMUNjh/v5NzAmeR2lMZhJNbZ1UN7XT2NrBsl1VjM9O6jaY3ZerTyzis/PHh5xG3IxMFiQCyEmJ5/0dVaiq/QGZIbHXTVff1+Y/Y9wWw5uby7j18TUAfOLYMUHPD8R+v01/rLspgKPyU6ltbqe0zrYxNUPDOybmXbcTiDdIeANEfEzXWIUxA8VaEgF4d/3acKC22+IlYwZLTXM7yXHRfc46GpfVffe4tT8612YpmQEX9DdKRB73e/zLHscW9ffGIjJWRN4QkQ0isl5Ebg5y3mnu/tfrReQtv/LzRGSziGwTkdtCu52BMbXASaa2vqRuMC9rjE91U1u/2VhTE2J56isn+Z5bgDDh0NdvlX/Sl7N7HMulfx3AN1R1OjAf+KqITPc/QUQygD8Dl6jqDOCTbnk08CfgfGA6cGXP14ZTakIsR+ensnRn1WBd0phuaprayUjqfyrrseMyEYHZY9IHoVZmJOqru6mvDRX63WxBVQ8AB9zH9SKyERgNbPA77SrgKVXd455X5pbPA7ap6g4AEXkMuLTHa8NqweQcHlm6m5b2ThJiowfrssYAUNPURmaI+zqs//G5REfZALQJj75aEkkiMldEjgMS3cfHep9/lIuISBEwF1ja49BRQKaIvCkiK0XkGrd8NLDX77x9blmg975RRFaIyIry8vKPUq0+zRmXQWuHhz1Vtt+1GXw1Te29cjAFkxQXQ3yMfZEx4dFXS+IAcJf7+KDfY+/zkIhICvAkcIuq9uzkjwGOA87ECTzvicj7ob43gKreC9wLUFxcPGDbyWW6f6DefDjGDJay+hZ2VDRy0mTb38EMvaBBQlVPD3ZMREL6iuOe9yTwiKo+FeCUfUClqjYCjSLyNjDbLR/rd94YoCSUaw4Ub5K02iYLEmZw/eCZ9QAkx9nkQzP0Qp4OIY4zReQBnA/xfs8HHgA2qupdQU77L7BQRGJEJAk4AdgILAemiMgEEYkDrgCeDbWuA8EXJKwlYQbZ1jJno6HPzh8/xDUxJoR1EiIyH2eA+TIgC/gq8M0Q3nsBcDWwVkRWu2XfA8YBqOo9qrpRRF4CPgQ8wP2qus697teAl4Fo4EFVXf9RbuxwpSVYkDCDa9nOKp7+oISyulaunj+esT3WQRgzFIIGCRH5Gc6U1D3Ao8CPgRWq+vdQ3lhVFwP9TrlQ1TuBOwOUvwC8EMq1wiHNWhJmkP30hY2s2VsDwPhsCxAmMvTV3fQFoBT4C/BPVa0khKmvR4roKCE1PsaChBk04/1aDtaKMJGiryBRCNwBXAxsF5F/4kyFHTGjaWmJsb4c/caEW2NrBwBXzhvLSZNsZpOJDEGDhKp2qupLqnotMAl4BngXKBGRfw1WBYdSemIstc3tbDxQR9Ftz7OltH6oq2SOYNVNbSyYnM3PPz6L1ITQ1kgYE24hzW5S1VZVfVJVLwcmAy+Ft1qRIS8tnjX7avne02sBJyWzMeHipOIIbZW1MYOlr4HrWwezIpHom+cczSV/XExFg5My3P6ATThVNbX5FnEaEyn6akn8GvgskA2kAKl+P4H3VDzCHDM6nXuvLvY9b+/0DGFtzJGs06PUNreHnK/JmMHS1yD0XOBK4EJgJc402NfUu6HuCHHCxCzfY+/AojGHqq3Dw3s7Kjn1qO6JlOua21G11qqJPH0NXK9R1dtUdQ7OyulLgQ0icsmg1S4CpCbEcsbUPAAaWjuHuDZmuPvJcxu49sFlrN9fy9p9tRTd9jyPr9jLand9RKFtcmUiTCgrrnNxWhUzcdJxjLjR2wc/dzwzfvCStSTMYWnv9PDch/sBqGho46lVTnabbz/xoe8cm/pqIk1fA9efBz4FJABPAJ/y2+9hxEmOj6GhxYKE+eh2VTQyOjOR3yzaQrWbMLK0toWtpQ3dzjv96FzrbjIRp6+WxP3AOmA3cC5wjpOzz6GqI6rbKSU+hoY2CxKmu/01zVz4+3d46Lp5zB6bEfD4ab9+ky+eOpG3t1T4yndXNbKtrCtIPPKFEzhufOag1NmYj6KvIBE0VfhIlJIQY91NppelOyupbmpnw4G6XkHilsc+YLPbWnhveyU7Kxq4fuEEnvmghNc2ltHmN1tuYm6y7YBoIlJf+0m8NZgViXTJcRYkTG8f7qsFoKK+tVt5c1snz6ze3+u86YVpvLz+IJsOdl+9n59qA9YmMo2YPEyHKzk+hpKa5qGuhokgtc3tvqytFQ2ttHV4+OwDS7npjMkUpvfe4TcvNZ6zpudTWt/C8p1VnD29gNK6Ft7dVkGU7VFtIpQFiRClJsRQ32LJ/oyjqa2D2T9e5Hte0djGrspGlu2s4gt/X8F91xT3es3H5o4mPTGWr5w2GU7rKv+fs48ahBobc2hC3plupBuVkcDB2hY6bNW1AfZUNXV7XlHfyl63rLXDE7DVmZdmXUpm+Ok3SIjIUSJyn4gsEpHXvT+DUblIUpSdTIdH2VdtXU4G9lZ1/R4cnZ9KRUMruyu7AsfqPTVERwkv3nyyryw/LX5Q62jMQAilu+k/wD3AfUDIS45FZCzwDyAfZ7Oie1X1dz3OOQ1nn+udbtFTqnq7e2wXUO9es0NVe7ffB9HE3GQAdlY0UpSTPJRVMRHA25L4znlTOVjbzH9W7mPd/lrf8SdX7WNcVhLTCtMYnZFISU0z+daSMMNQKEGiQ1X/cgjv3QF8Q1VXiUgqsFJEXlHVDT3Oe0dVLwryHqerakWQY4OqKNsJDD97YSMnT8khJtp66kay7eUNJMdF86VTJ7K2pJaHl+7hqVUlTMhJZndlIx0eZY47Jba1w+mizEu1loQZfkL5pPs/EfmKiBSKSJb3p78XqeoBVV3lPq4HNgKjD7O+QyYrOY6Zo9PZWtbAlh4rZc3Isq2sgceX72XmmHREhFljMji+yFkId+pRuYzJdLYe9QaJ848pACDXgoQZhkIJEtcC3wKW4GSDXQms+CgXEZEinPxPSwMcPlFE1ojIiyIyw69cgUUislJEbuzjvW8UkRUisqK8vPyjVOsjERF+/vGZAOypagzbdUxka+/0sHxXFR0e5UeXdP26Hp2fCjiB4Z/Xz+OaE8dz0axCAH5w8XSWff9MkuJsMqEZfvr9rVXVCYdzARFJAZ4EblHVuh6HVwHjVbVBRC7A2SJ1intsoaqWiEge8IqIbFLVtwPU717gXoDi4u6HCv8AACAASURBVOKwpjH3bk7vP0BpRo69VU2cdddbtHZ4iIuOYkpequ/YrWcfTVZyPOfPLCA+JprbLz3Gdyw2Ooo8WyxnhqlQZjfFisjXReQJ9+drIhLS9lnueU8Cj6jqUz2Pq2qdqja4j18AYkUkx31e4v63DHgamBfyXYVJemIsGUmx7K6yIDESvbqx1De+0KlKtN8CuPSkWG4+awrxMZZawxxZQulu+gtwHPBn9+c4t6xP4mQDfADYqKp3BTmnwD0PEZnn1qdSRJLdwW5EJBk4ByfZ4JAbn5Xkmw9vRpa3t5TjjQudnhG195YZwULpJD1eVWf7PX9dRNaE8LoFwNXAWhFZ7ZZ9DxgHoKr3AJcDXxaRDqAZuEJVVUTygafd+BED/EtVXwrpjsIsJyWeA7UtQ10NMwR2VzZx/jGFbDpYxyeOGzPU1TFmUIQSJDpFZJKqbgcQkYmEsF5CVRcDfSakUdU/An8MUL4DmN37FUMvIymOjQd6Dq2YkaC8oZVTUuP502dOG+qqGDNoQgkS3wLeEJEdOB/644HrwlqrCJaRFEtNs+VwGkma2zr5w+tbqW/psGmsZsQJZXbTayIyBTjaLdqsqq19veZIlpkUS1NbJ60dnTZIOUL87zPreNLdajQnxXaOMyNLX9uXnqGqr4vIx3scmiwiBJqtNBJ4t5esaWonP82CxHBXVtdCelJs0IBfVtfCf1eX+J5bS8KMNH21JE4FXgcuDnBMgREZJDLdIFHd1Ga5eIa59k4P8372GgDXLSjihxfP6HXO0p3OwjmvnBQLEmZk6Wtnuh+6D29X1Z3+x0TksBbYDWcZSc4SkZomG5cY7nZXdq2c/9u7uwIGiR3lzjkXzizk+bUHyLYgYUaYUNZJPBmg7ImBrshw0RUk2oa4JuZwtHV4+M6Ta7uVldV1TW1uaO3gK4+sZNGGg4zOSOQ3n5rNv2+cz+iM3jvOGXMk62tMYiowA0jvMS6RBozYfpbsZOebZHmDBYnhbNnOKlburu5Wtm5/LWe4XYhPf1DCC2sPArBwcg4JsdGcMDF70OtpzFDra0ziaOAiIIPu4xL1wA3hrFQky0mJI0qg1BbUDWuBtqJdtL6UhZNzaev08Nya/b7yEyb0m/TYmCNWX2MS/wX+KyInqup7g1iniBbjJms7WGdBYjirdde6PHbjfP69fC+dHuX/1uxn+a4qSutaaevwcNGsQs6ens/Fs0YNcW2NGTqhLKb7QES+itP15OtmUtXPh61WEa4g3dnv2gxf3gWRs8akM39iNjsrGnnuw/1sL+8azL5gZiEXzCwcqioaExFCGbj+J1AAnAu8BYzB6XIasQrSrCURSTwe5cmV+3hrS+j7idQ0tRMXHUVirLM+YkJOMo9/8US+fsZk4mKiiI+J4vgi62YyJpSWxGRV/aSIXKqqfxeRfwHvhLtikawgPYHF2yJiV1WDs5/0t574kCiBDbefR0Js34sc2zo81Da3kZ4Ui5tEEoDioiyKi7L4yumT6fQoyfG2SZAxobQkvCN8NSJyDJAO5IWvSpGvID2BhtaOgIOfZvA9466I9ii8samsz3Nrm9o56n9f5NFle0lPDLwtSkJstAUIY1yhBIl7RSQT+H/As8AG4FdhrVWEK0x3hmZKrctpyHk8yopd1XzupCLioqNYva8GgM0H66ltbmdbWT2/WbQZVWfV9P7aZt9rM4IECWNMl1AS/N3vPnwLmBje6gwP3nQcB2tbmey3haUZfOUNrbR2eJiUl8L47CTfCulz73Z2uk2Ki6aprZNrTiwiNzWeSr/1LYlxlnvLmP70tZju1r5eGGy3uZHA25I44Pet1AyNPe4ugWMzE5mQk8yOikY6Oj2+401tztYnlY2t5KbGU9HgJDDOSYnj7On5g19hY4aZvloS3q/IRwPH43Q1gbOwblk4KxXpuloS1t001LxbyY7NSmJibgpvbC4LuN9HeX0rUwvwBYnXbj2N9CTrbjKmP30tpvsxgIi8DRyrqvXu8x8Bzw9K7SJUQmw0mUmxNg02AuytakYERmckMi4rifZOZXtZQ6/z1pbUMmt0Bne+vBkRSEu0gWljQhHKwHU+4J+oqM0t65OIjBWRN0Rkg4isF5GbA5xzmojUishq9+cHfsfOE5HNIrJNRG4L5WYGU36aLaiLBGX1LWQmxZEQG+3b62FHRWOv83710mZm376I1g4PqnSb+mqMCS6Ur1P/AJaJyNPu88uAh0J4XQfwDVVdJSKpwEoReUVVN/Q47x1Vvci/QESigT8BZwP7gOUi8myA1w6ZwnRbUBcJaprafZl589wgEaglYYw5NP22JFT1pzh7Wle7P9ep6s9DeN0BVV3lPq4HNgKjQ6zXPGCbqu5Q1TbgMeDSEF87KCw1R2SobmrzbQTlbUlsLw8eJFLjY3jh6ycPSt2MORL0NbspTVXrRCQL2OX+eI9lqWpVqBcRkSJgLrA0wOETRWQNsB/4pqquxwkme/3O2QecEOS9bwRuBBg3blyoVTpsBWmJVDa22V7XQ6y6qZ3RGc5Egmx3/2lvd9PTXzmJwvREnl97gLzUeLKT45icl0Ke7ShoTMj66m76F06q8JU425V6ifs8pDUTIpKCs3HRLapa1+PwKmC8qjaIyAXAM8CUEOsOgKreC9wLUFxcrP2cPmC802DL6loZm5U0WJc1PdQ0tTFjVBoA8THRZCTFsrvSmfE0ISeZjKQ4rl84YjdSNOaw9TW76SL3v4f8FyYisTgB4hFV7bUntn/QUNUXROTPIpIDlABj/U4d45ZFjHw3SBysa7EgMYSc7qauqay5KfG+rWVTLLWGMYetr+6mY/t6oXe8oY/XC/AAsDHYwjsRKQBKVVVFZB7OGEklUANMcffSLgGuAK7q63qDLcft2qiobx3imow8Le2d/PT5jeyqbKSl3UOGOyYBMCYzka1lDSTFRRMTHcrkPWNMX/r6qvWbPo4pcEY/770AuBpYKyKr3bLvAeMAVPUe4HLgyyLSATQDV6iTZKdDRL4GvAxEAw+6YxURIzfFGST1Ls4yg+cXL27in+/v9j3P9AsS49xW3Thr3RkzIPrqbjr9cN5YVRfjjF/0dc4fgT8GOfYC8MLh1CGcspLjELG9rofC4m0VnDE1j5MmZfPTFzYy3R2TAHxdfzY4bczACKnT1k0RPp3uO9P9I1yVGg5ioqPITIqzlsQga2jtYHt5AxfPGsUXTp7I9QsndFsYl5XstCoSY62ryZiB0G+QEJEfAqfhBIkXgPOBxTiL7Ea0nJQ4G5MYZBv216EKM8c4rYeeK6dnjUkH4PLjxvZ6rTHmowulJXE5MBv4QFWvE5F84OHwVmt4yEmJt5bEIPP+exemJwY8Pjkvla0/PZ9YG7Q2ZkCE8pfUrKoenMHkNKCM7tNTR6yclHgqG21MYjB5dwNMTQj+/cYChDEDJ5SWxAoRyQDuw1lY1wC8F9ZaDRM5KfHW3TTI6ls6AEhNsDTfxgyGoF+5RORPIrJAVb+iqjXulNWzgWtV9brBq2LkykmNo7Gtkzm3L6KxtWOoqzMi1LlBwhbKGTM4+mqXbwF+LSK7RORXIjJXVXep6oeDVblIl+OulahpamdLaf0Q12ZkqG9pJyU+hugoS/VtzGAIGiRU9XeqeiJwKs4q6AdFZJOI/FBEjhq0GkYw74I6cHY+M+Hx5uYyKt0B6/qWjj7HI4wxAyuUVOG7VfWXqjoXuBJnP4mNYa/ZMODNOgqwv8b2uw6H8vpWPve35Xz9sQ8ApyVhQcKYwdNvkBCRGBG5WEQeAV4ENgMfD3vNhoEcv5ZEiQWJAXGgtplbH1/N4yucTPErdjkZ6XeUO+m/nZaEDVobM1j6SvB3Nk7L4QJgGc7GPzeqau+9IUcoCxID7+V1B3lqVQnPrt7PudML+POb24Guger6lo5uLThjTHj11ZL4LrAEmKaql6jqvyxAdBcXE8WK/z2LeROyKK2zMYmBsLfaCbYdHuX6vy9nbUkt4AThxVsrKK9vtZaEMYOorwR//WV5NTiticL0BFbtqR7qqhwR9lY1MSEnmarGNlbsrua8GQWcNDmbH/x3PZ99wNnYcExm4NXWxpiBZ0tTB0Beajx7q5r561vbh7oqw97e6mYm5CRzwcwCAG4+awpF2cm+41MLUrnpjMlDVT1jRhwLEgMgN9UZm/j5i5tsUd1HtHxXFUu2VQDw+qZSNh6oY1xWEt86dyr/uuEEphWmMSGnK0g8esN8kuJsdpMxg8WCxADIS+3au2BnhQ3b9KfTo3g8znbkn7znPa663+lGemyZM6Pp4tmjyEqO46RJOUDXfuIAGUk2HmHMYLIgMQC8exgAbC9v6HW8ypIA+qgqF/zuHb75xJpu5S3tneyrbuaMqXkcNz6z2zH/bUh7pgY3xoSXtdsHgLe7CWB7Wfcg8fL6g3zxnyt58ssn9frwG4ne2VrB5tJ6NpfWMyk3xVe+tbSBvdVNFBcF/jf63RVzLLurMUMgbH91IjJWRN4QkQ0isl5Ebu7j3ONFpENELvcr6xSR1e7Ps+Gq50CYVpjGs19bwNSCVJZsr+x27M3NZQC8uPbAUFQt4vjPArvz5c2+x0t3VlLf0sHYzMB7U186ZzQXzCwMe/2MMd2F86tZB/ANVZ0OzAe+KiLTe54kItHAL4FFPQ41q+oc9+eSMNZzQMwak8HFs0exYnc1B2q7FtZtL3PGKF5cd5C2Ds9QVS9i1DY7Cfru/vScbuV3PO9kerHprcZElrAFCVU9oKqr3Mf1OPmeRgc49SbgSZzNjIa1Yrc7yRsYDta28MHeaqYVplFS08xzH+4fyupFhLrmDtITY5lamBrw+ImTsge5RsaYvgxKJ6+IFAFzgaU9ykcDHwP+EuBlCSKyQkTeF5HL+njvG93zVpSXlw9grT86b7qIysZW2js9fPepD+n0KH/97HHERUex+aClE69tdhL0TcxJ6XXs/100nYwkS7lhTCQJ+8C1iKTgtBRuUdW6HofvBr6jqp4As1bGq2qJiEwEXheRtaraa7Waqt4L3AtQXFysA38HoctKdgawV+6u5qElu/hgTw0/uGg647KTGJWRwD7L70RdczvpibHExUTx0i0nowpX3fc+1U3tjMsKPB5hjBk6YQ0SIhKLEyAeUdWnApxSDDzmBogc4AIR6VDVZ1S1BEBVd4jImzgtkYhe0pyR6Mzh/8d7u31l155UBMDozERKqi1I1Da3Mz7bCQZTC9IAfGM1FiSMiTzhnN0kwAPARlW9K9A5qjpBVYtUtQh4AviKqj4jIpkiEu++Tw6wANgQrroOlCi/3dLOm1HAozfM9+2gNjojsVem2L1VTXR6hrTxM+jqWpyWhD/vOMTYLBu0NibShLMlsQC4GlgrIqvdsu8B4wDcPbODmQb8VUQ8OIHsF6oa8UHC3y1nT/F9UwYYnZFEeX0rLe2dJMRG8972Sq68730Abj5zCv9z9sjY7K+2uZ20HkHid1fMZWdFo6XbMCYChe2vUlUXAyEvj1XVz/k9XgLMDEO1Bo1/viGAY0Y7AeODPTXExQifuf9937HfvbZ1RASJ9k4PTW2dvVoSyfExHDM6fYhqZYzpiy1hHWBT8pxZO/Ex0d3K503IIjpKeG97BR/sqcGj8IOLnGUj/mk9jmQ1Te0AvYKEMSZyWft+gP33awsCLppLTYhlemEaK3ZXMyk3hdSEGK5bUER9Swe/fXWLrxvqSFZa1wJAflp8P2caYyKFtSQGWFJcTNC5/tML09h4oI7dVU2Mz05CRHyDtSNh+9OuIJHQz5nGmEhhQWIQTStMpbqpnWU7Kxmf5YxZeKd97q488lOMH6h1gkRhus1iMma4sCAxiKYVOoPXLe0eX0bYowuc9BRLtlXyyoZSVIfflNifvbCRqx9Y2u95B2tbiI6SbllzjTGRzcYkBlFxURa3XzqDxNhoLj9uDOCMVeSnxXP/4p3cv3gn//nSiRxflDXENf1o7n17B+DMXgqWzrulvZO/vbuTnJQ439oRY0zks5bEIIqOEq45sYhPFo/ttnmO/3TZd7YMbf6pw3HanW8GzXT7+qYyGts6bTzCmGHGgkQE+PUnZ/OvL5zAnLEZvLejsv8XRJBad1orOIPve6qaAp5XXt8KwH3XFA9KvYwxA8OCRAQYk5nESZNzmFaYxray3tufRrKdPQbc91UHDhJVjW2IQPYIWRNizJHCgkQEmZSbTHVTO9XDaE/sLW7683/dcAIA+4IkMaxpaiMtIbbbftXGmMhnA9cRxDs2sbOykcwI/8ZdUtNMtAhrS2pJiY9hXlEWcdFRQYNEVVP7iFlZbsyRxIJEBPEGiR3ljRw7LnOIa9O3Bb94HYC54zKYPiqNmOgoRmUksDdId1N1YxuZSZaOw5jhxtr+EWS0u79zpO87sd9vdfgHe2qY507ZzUtL8A1Q91Td1Eam7TpnzLBjQSKCxMdEk5ca3+1DuD+r9lTz0roDvcrrWtrxhGmviiXbu2ZgZSTF8vmFEwDISooLOp5S0dAa8V1oxpjeLEhEmFEZieyvDT1IfPzPS/jSw6u6lTW0djDrR4u465UtA1099lY18Z8Ve0mIjWLLHefz9rdP9401ZKXEUd3UPUg0tnZwwz9WUFrXajvPGTMMWZCIMKMzDm2b0xq/D+cd5c402vve2TFg9QIoq2/h1DvfYOnOKibkpBAXE0VaQtc4Q1ZSHNVN3Vswy3dV8cqGUi6cVciNp0wc0PoYY8LPgkSEGZWRwI6KRnZVNLJkewWVDYH7+AHO+M2bvsfbyxt6PW4Nsvr5UJTWtTDvp6/h/fyvberdrZSZHEenR6lr6Vpgt7XUqcsdlx5zxKdCN+ZIFM49rseKyBsiskFE1ovIzX2ce7yIdIjI5X5l14rIVvfn2nDVM9JcPHsUAL96eRNX3beUK+59P+i5O8q7FrJtL3Mel9Q08z//XtNVXj4wi/MeXban2/Mvnjqp1znehXKLNpT60oJvLq0nNzXexiOMGabCOQW2A/iGqq4SkVRgpYi80nOvahGJBn4JLPIrywJ+CBQD6r72WVWtDmN9I8KsMRmcODGbD/fVArA1yArsntlivVNPP9jT9U+UnhjLt/6zhl9dPpvJ7o55h6qsvpXs5Dgeu3E+k/NSuuWe8vIGgm8/8SFTC1J56ZZT2Fpaz1H5h3dtY8zQCVtLQlUPqOoq93E9sBEYHeDUm4AngTK/snOBV1S1yg0MrwDnhauukSYvLT7oojSvlvbuXUkVbrdUU2snAA9+rpgbTp7Aqj01nHXXW4ddp+rGNjKT45iSnxowQED3lBubDtbj8ShbyxqYkpd62Nc3xgyNQRmTEJEiYC6wtEf5aOBjwF96vGQ0sNfv+T4CB5gjUs9MqZsO1vU6p7GtA4ALZxUytSCV8npnjKC+1Sk/blwWV59Y5Du/vfPwxieqm9rI6medw/TCNF8K9LiYKPZWN9HU1slR+RYkjBmuwh4kRCQFp6Vwi6r2/LS7G/iOqh7yJ5iI3CgiK0RkRXn58E2z7S+vx6Y8P31+Y69zvC2G04/OIzc13teSaGhxgkRyfDTpibHcfukMAGr8srUeiurGdjL6WTEdFSX8+pOzufPyWbR1eHhp3UEA624yZhgLa5AQkVicAPGIqj4V4JRi4DER2QVcDvxZRC4DSoCxfueNcct6UdV7VbVYVYtzc3MHtP5DJc+vJXHWtPyA+183uC2GlPhoclL8gkRrO4mx0b5Eet79tmubDy9pYHVTW8i5l7w78P38xU3Ex0RxVIG1JIwZrsI5u0mAB4CNqnpXoHNUdYKqFqlqEfAE8BVVfQZ4GThHRDJFJBM4xy0bEcZmdu0BXZAeT1WAVcxNbndTUlwMOSlxVDS0oqo0tHaQktA1HyEj0fn2X32ILYmOTg9r9tZQVt/qCzj98R8k//ElM7qtpTDGDC/hnN20ALgaWCsiq92y7wHjAFT1nmAvVNUqEfkJsNwtul1Vq8JY14gyZ2wG919TTGxMFCt3V1Pb3E5Hp6dbmm1vSyI5Pobc1Hha2j00tHZQ39JBanzX/1ZvvqRD7W56cd1Bbnr0A8CZLRUK//UQF8wqPKTrGmMiQ9iChKouBkLezFhVP9fj+YPAgwNcrWFBRDhrej4AuyoaUYWa5nZyUrrGKpranDGJ5PhoCtPdxIA1zb1bEknelsShdTd51zsApCaE/utyxfFj2Vxab60IY4Y5SxUe4bzjAFWNbd2ChK8lERdDUbaTYnxXRRMNLR2kxPcOErWH2JKobW5HBP7x+Xkc72Z7DcUvPjHrkK5njIkslpYjwnmDRGVD95ZAk1930/gcJ3HeO1vLWbG7uluQSImPISZKurUkVJWi257ntyEkAKxrbictIZaTp+RaWg1jRiALEhHOvyXhr9GvuyktIZbs5DgeWeqkztjml4pDRMhMjusWZLwL9X732tZ+r1/b3B7yWIQx5shjQSLCeVcxL9le0W18oLa5nbjoKOLcwezx2V1puM+dUdDtPfLT4imt73rt+v3OcpVQPvzrWjpIS7ReSWNGKgsSEc6bD+mRpXs47c43feU7yhspyknypcjwjkssmJzNt889utt75KUmUFbXlU12wwEnSISynai1JIwZ2SxIRLjY6CjflNbm9k5f+Y7yBibldq1HGO8GiYK0xF65lfLT4inz21bUu/Ndnbs6O5DG1g5aOzp9YxLGmJHJ+hGGAf98r+2dHraXN7C7qokLZnatQShMd1Zpd3h6ZzjJTU2gsrHVt9aittmZ6VTV2EZbh4e4mN7fFWb88GVmjUmnrsWChDEjmbUkhgH/D/4p33+R8+5+h06Pdkt3kZfmTI/tmRzQKYtHFSrcwWtvkAD45Uubgl73w321TndTCN1SxpgjkwWJYSBA4wCAs6bl+R6felQuv/nkbG49+6he501wu6Le31Hp7BzX3M7UglRyUuL427s7eW97ZY/rdbVdWto9IedsMsYceSxIDAP+LYlR6Qn89GPHcM9njyUprqu3UET4xHFjAq5lmD8xmyl5Kdz79g6K73iFTQfrOWZ0Ov9300Jio6O47qFlVPtNsfWmIfcq8ps5ZYwZWSxIDAPjsro+pP/y2eP4zAnjOe+Y0HMiRUUJ15xUxIYDdb5Ef2kJsRSmJ3L/tcW0tHtYvK3Cd37PAe2inOTDvANjzHBlQWIYeOSG+fz+yrns+sWFzB6bcUjvcemcUd2ee6e1njQph7SEGJZs9wsSzd1TePgHKWPMyGJBYhgYnZHIJbNH9X9iH9ISYnnzm6f5NjSKiXamyUZHCRNyUyip6Vps5x8kUuJjunVrGWNGFgsSI0hRTjJXzx8PQF1LVyDISY6jrK6Fi/+wmCdW7vN1N/3vhdN45qsnDUldjTGRwYLECDN/UjYAc/26rbJT4thcWs/aklqWbKvwtSTOmV7A5DzbVc6Ykcz6EUaY44uyWP79s8j120c7O8VZRwGwq7KRibnOQLXlbDLG2KfACOQfIKAriSDAqj01rNpTQ3xMVLeU48aYkcm6m4wvaET5pXx67Mb53bZLNcaMTGH7FBCRsSLyhohsEJH1InJzgHMuFZEPRWS1iKwQkYV+xzrd8tUi8my46mkgO9kJEtecWMTccRk8+Lli5o7LHOJaGWMiQTj7EzqAb6jqKhFJBVaKyCuqusHvnNeAZ1VVRWQW8Dgw1T3WrKpzwlg/4zpufCZfWDiBm86cYmnBjTHdhC1IqOoB4ID7uF5ENgKjgQ1+5zT4vSSZ7glPzSBJjIvmfy+aPtTVMMZEoEHpdBaRImAusDTAsY+JyCbgeeDzfocS3C6o90Xksj7e+0b3vBXl5eUDXHNjjBnZwh4kRCQFeBK4RVXreh5X1adVdSpwGfATv0PjVbUYuAq4W0QmBXp/Vb1XVYtVtTg3NzcMd2CMMSNXWIOEiMTiBIhHVPWpvs5V1beBiSKS4z4vcf+7A3gTpyVijDFmEIVzdpMADwAbVfWuIOdMds9DRI4F4oFKEckUkXi3PAdYgN9YhjHGmMERztlNC4CrgbUistot+x4wDkBV7wE+AVwjIu1AM/Bpd6bTNOCvIuLBCWS/6DEryhhjzCAI5+ymxYD0c84vgV8GKF8CzAxT1YwxxoTIltQaY4wJyoKEMcaYoET1yFm/JiLlwO5DeGkOUNHvWUeekXrfMHLv3e57ZAnlvseratD1A0dUkDhUIrLCXZMxoozU+4aRe+923yPLQNy3dTcZY4wJyoKEMcaYoCxIOO4d6goMkZF63zBy793ue2Q57Pu2MQljjDFBWUvCGGNMUBYkjDHGBDXig4SInCcim0Vkm4jcNtT1GUgi8qCIlInIOr+yLBF5RUS2uv/NdMtFRH7v/jt86CZcHJaCbZ17pN+7iCSIyDIRWePe94/d8gkistS9v3+LSJxbHu8+3+YeLxrK+h8uEYkWkQ9E5Dn3+Ui5710ista7DbRbNmC/6yM6SIhINPAn4HxgOnCliBxJW7Q9BJzXo+w24DVVnYKzfaw3MJ4PTHF/bgT+Mkh1DAfv1rnTgfnAV93/r0f6vbcCZ6jqbGAOcJ6IzMfJj/ZbVZ0MVAPXu+dfD1S75b8lQB61YeZmYKPf85Fy3wCnq+ocvzURA/e7rqoj9gc4EXjZ7/l3ge8Odb0G+B6LgHV+zzcDhe7jQmCz+/ivwJWBzhvuP8B/gbNH0r0DScAq4AScFbcxbrnvdx54GTjRfRzjnidDXfdDvN8x7ofhGcBzOMlFj/j7du9hF5DTo2zAftdHdEsCZ8/tvX7P97llR7J8dfYfBzgI5LuPj8h/ix5b5x7x9+52uawGyoBXgO1Ajap2uKf435vvvt3jtUD24NZ4wNwNfBvwuM+zGRn3DaDAIhFZKSI3umUD9rsezv0kTIRTVRWRI3YOdM+tc939rYAj995VtROYIyIZwNPA1CGuUtiJyEVAmaquFJHThro+Q2ChqpaISB7wiohs8j94uL/rI70lUQKM9Xs+sI35DAAAA0FJREFUxi07kpWKSCGA+98yt/yI+rcIsnXuiLh3AFWtAd7A6WbJEBHvF0L/e/Pdt3s8Hagc5KoOhAXAJSKyC3gMp8vpdxz59w102+q5DOeLwTwG8Hd9pAeJ5cAUdxZEHHAF8OwQ1yncngWudR9fi9Nf7y2/xp39MB+o9WuuDisiQbfOPaLvXURy3RYEIpKIMw6zESdYXO6e1vO+vf8elwOvq9tRPZyo6ndVdYyqFuH8Db+uqp/hCL9vABFJFpFU72PgHGAdA/m7PtSDLkP9A1wAbMHpu/3+UNdngO/tUeAA0I7T93g9Tt/ra8BW4FUgyz1XcGZ6bQfWAsVDXf/DuO+FOP20HwKr3Z8LjvR7B2YBH7j3vQ74gVs+EVgGbAP+A8S75Qnu823u8YlDfQ8D8G9wGvDcSLlv9x7XuD/rvZ9hA/m7bmk5jDHGBDXSu5uMMcb0wYKEMcaYoCxIGGOMCcqChDHGmKAsSBhjjAnKgoQx/RCRTjfDpvdnwLIFi0iR+GXpNSbSWFoOY/rXrKpzhroSxgwFa0kYc4jcPP6/cnP5LxORyW55kYi87ubrf01Exrnl+SLytLvfwxoROcl9q2gRuc/dA2KRu1oaEfm6OHtifCgijw3RbZoRzoKEMf1L7NHd9Gm/Y7WqOhP4I04mUoA/AH9X1VnAI8Dv3fLfA2+ps9/DsTgrZMHJ7f8nVZ0B1ACfcMtvA+a67/OlcN2cMX2xFdfG9ENEGlQ1JUD5LpxNfna4CQUPqmq2iFTg5Ohvd8sPqGqOiJQDY1S11e89ioBX1NkcBhH5DhCrqneIyEtAA/AM8IyqNoT5Vo3pxVoSxhweDfL4o2j1e9xJ11jhhTh5do4FlvtlNDVm0FiQMObwfNrvv++5j5fgZCMF+Azwjvv4NeDL4NscKD3Ym4pIFDBWVd8AvoOTzrpXa8aYcLNvJsb0L9Hd7c3rJVX1ToPNFJEPcVoDV7plNwF/E5FvAeXAdW75zcC9InI9TovhyzhZegOJBh52A4kAv1dnjwhjBpWNSRhziNwxiWJVrRjquhgTLtbdZIwxJihrSRhjjAnKWhLGGGOCsiBhjDEmKAsSxhhjgrIgYYwxJigLEsYYY4L6/yTYOJmcxN2HAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s1f2Nz0_Cllw"
      },
      "source": [
        "MAE在第80個週期後停止改善並開始往上升\r\n",
        "發生過度適配的情形"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GaBz6fsgCxwy"
      },
      "source": [
        "## 訓練最終模型"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "n16FMYs_CzgM",
        "outputId": "afd4e3d4-af1d-42a1-897d-3997a7cf00ed",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "model = build_model()\r\n",
        "model.fit(train_data,train_targets,epochs=80,batch_size=16,verbose=0)\r\n",
        "\r\n",
        "test_mse_score,test_mae_score = model.evaluate(test_data,test_targets)"
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "4/4 [==============================] - 0s 2ms/step - loss: 17.7433 - mae: 2.6968\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pTGtej0YDM--",
        "outputId": "795c3afc-07f2-4dcc-ff2b-60b9b65eaef8",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "test_mae_score"
      ],
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2.696845054626465"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pNxJwEB-DghJ"
      },
      "source": [
        "仍有$2,550美元左右的誤差"
      ]
    }
  ]
}