01_tensorflow_nlp_seq2seq_desaster_tweets.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "764fa481",
   "metadata": {},
   "source": [
    "# tf-NLP\n",
    "\n",
    "## (Re) Introduction to Tensorflow Natural Language Processing\n",
    "\n",
    "### Abstract\n",
    "\n",
    "Twitter has become an important communication channel in times of emergency.\n",
    "The ubiquitousness of smartphones enables people to announce an emergency they’re observing in real-time. Because of this, more agencies are interested in programatically monitoring Twitter (i.e. disaster relief organizations and news agencies).\n",
    "\n",
    "But, it’s not always clear whether a person’s words are actually announcing a disaster. Here Machine Learning can help us segmenting large quantities of incoming messages. But what type Machine Learning is the best suited?\n",
    "\n",
    "I am running a couple of experiments to show the performace of different solutions and their trade-off - how long does it take to get a prediction from a given algorithm? Is a higher accuracy worth the wait?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15701fbf",
   "metadata": {},
   "source": [
    "![(Re) Introduction to Tensorflow Natural Language Processing](./assets/tf_nlp_desaster_tweets_06.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "daaf2c02",
   "metadata": {},
   "source": [
    "## Dataset\n",
    "\n",
    "[https://www.kaggle.com/competitions/nlp-getting-started/data](https://storage.googleapis.com/ztm_tf_course/nlp_getting_started.zip)\n",
    "\n",
    "```\n",
    ".\n",
    "├── data\n",
    "│   ├── test.csv\n",
    "│   └── train.csv\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "9adcc9e1",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 12:32:46.054605: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n",
      "To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n"
     ]
    }
   ],
   "source": [
    "import datetime\n",
    "import io\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import random\n",
    "import time\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "from sklearn.naive_bayes import MultinomialNB\n",
    "from sklearn.pipeline import Pipeline\n",
    "import tensorflow as tf\n",
    "# tf-hub bug https://stackoverflow.com/questions/69339917/importerror-cannot-import-name-dnn-logit-fn-builder-from-partially-initialize\n",
    "from tensorflow_estimator.python.estimator.canned.dnn import dnn_logit_fn_builder\n",
    "import tensorflow_hub as hub\n",
    "from tensorflow.keras.layers import (\n",
    "    TextVectorization,\n",
    "    Embedding,\n",
    "    Input,\n",
    "    Dense,\n",
    "    GlobalAveragePooling1D,\n",
    "    GlobalMaxPool1D,\n",
    "    LSTM,\n",
    "    GRU,\n",
    "    Bidirectional,\n",
    "    Conv1D\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "919cd9a5",
   "metadata": {},
   "outputs": [],
   "source": [
    "from helper_functions import (\n",
    "    create_tensorboard_callback,\n",
    "    plot_loss_curves,\n",
    "    compare_histories,\n",
    "    calculate_metrics,\n",
    "    time_to_prediction\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "1a7fb487",
   "metadata": {},
   "outputs": [],
   "source": [
    "SEED = 42\n",
    "LOG_DIR = 'tensorboad'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d3c15fe",
   "metadata": {},
   "source": [
    "### Exploration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "61a48d12",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>keyword</th>\n",
       "      <th>location</th>\n",
       "      <th>text</th>\n",
       "      <th>target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Our Deeds are the Reason of this #earthquake M...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Forest fire near La Ronge Sask. Canada</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>All residents asked to 'shelter in place' are ...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>13,000 people receive #wildfires evacuation or...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Just got sent this photo from Ruby #Alaska as ...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id keyword location                                               text  \\\n",
       "0   1     NaN      NaN  Our Deeds are the Reason of this #earthquake M...   \n",
       "1   4     NaN      NaN             Forest fire near La Ronge Sask. Canada   \n",
       "2   5     NaN      NaN  All residents asked to 'shelter in place' are ...   \n",
       "3   6     NaN      NaN  13,000 people receive #wildfires evacuation or...   \n",
       "4   7     NaN      NaN  Just got sent this photo from Ruby #Alaska as ...   \n",
       "\n",
       "   target  \n",
       "0       1  \n",
       "1       1  \n",
       "2       1  \n",
       "3       1  \n",
       "4       1  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df = pd.read_csv('data/train.csv')\n",
    "test_df = pd.read_csv('data/test.csv')\n",
    "\n",
    "train_df.head(5)\n",
    "# target 1 = disaster / 0 = not a disaster"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4735bfaf",
   "metadata": {},
   "source": [
    "|   | id | keyword | location | text | target |\n",
    "|  -- | -- | -- | -- | -- | -- |\n",
    "| 0 | 1 | NaN | NaN | Our Deeds are the Reason of this #earthquake M... | 1 |\n",
    "| 1 | 4 | NaN | NaN | Forest fire near La Ronge Sask. Canada | 1 |\n",
    "| 2 | 5 | NaN | NaN | All residents asked to 'shelter in place' are ... | 1 |\n",
    "| 3 | 6 | NaN | NaN | 13,000 people receive #wildfires evacuation or... | 1 |\n",
    "| 4 | 7 | NaN | NaN | Just got sent this photo from Ruby #Alaska as ... | 1 |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "1c0dd3cd",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0    4342\n",
      "1    3271\n",
      "Name: target, dtype: int64\n",
      "7613 3263\n"
     ]
    }
   ],
   "source": [
    "# checking if the dataset is balanced\n",
    "print(train_df.target.value_counts())\n",
    "# 0    4342\n",
    "# 1    3271\n",
    "\n",
    "print(len(train_df), len(test_df))\n",
    "# 7613 3263"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "dd53101c",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>keyword</th>\n",
       "      <th>location</th>\n",
       "      <th>text</th>\n",
       "      <th>target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2644</th>\n",
       "      <td>3796</td>\n",
       "      <td>destruction</td>\n",
       "      <td>NaN</td>\n",
       "      <td>So you have a new weapon that can cause un-ima...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2227</th>\n",
       "      <td>3185</td>\n",
       "      <td>deluge</td>\n",
       "      <td>NaN</td>\n",
       "      <td>The f$&amp;amp;@ing things I do for #GISHWHES Just...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5448</th>\n",
       "      <td>7769</td>\n",
       "      <td>police</td>\n",
       "      <td>UK</td>\n",
       "      <td>DT @georgegalloway: RT @Galloway4Mayor: ÛÏThe...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>132</th>\n",
       "      <td>191</td>\n",
       "      <td>aftershock</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Aftershock back to school kick off was great. ...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6845</th>\n",
       "      <td>9810</td>\n",
       "      <td>trauma</td>\n",
       "      <td>Montgomery County, MD</td>\n",
       "      <td>in response to trauma Children of Addicts deve...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        id      keyword               location  \\\n",
       "2644  3796  destruction                    NaN   \n",
       "2227  3185       deluge                    NaN   \n",
       "5448  7769       police                     UK   \n",
       "132    191   aftershock                    NaN   \n",
       "6845  9810       trauma  Montgomery County, MD   \n",
       "\n",
       "                                                   text  target  \n",
       "2644  So you have a new weapon that can cause un-ima...       1  \n",
       "2227  The f$&amp;@ing things I do for #GISHWHES Just...       0  \n",
       "5448  DT @georgegalloway: RT @Galloway4Mayor: ÛÏThe...       1  \n",
       "132   Aftershock back to school kick off was great. ...       0  \n",
       "6845  in response to trauma Children of Addicts deve...       0  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df_shuffle = train_df.sample(frac=1, random_state=SEED)\n",
    "\n",
    "train_df_shuffle.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "590719b1",
   "metadata": {},
   "source": [
    "|   | id | keyword | location | text | target |\n",
    "|  -- | -- | -- | -- | -- | -- |\n",
    "| 2644 | 3796 | destruction | NaN | So you have a new weapon that can cause un-ima... | 1 |\n",
    "| 2227 | 3185 | deluge | NaN | The f$&amp;@ing things I do for #GISHWHES Just... | 0 |\n",
    "| 5448 | 7769 | police | UK | DT @georgegalloway: RT @Galloway4Mayor: ÛÏThe... | 1 |\n",
    "| 132 | 191 | aftershock | NaN | Aftershock back to school kick off was great. ... | 0 |\n",
    "| 6845 | 9810 | trauma | Montgomery County, MD | in response to trauma Children of Addicts deve... | 0 |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "aa92271b",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Target: 0 (not a disaster)\n",
      "Text: Do you have a plan? Emergency Preparedness for #Families of\n",
      "Children with Special Needs  http://t.co/RdOVqaUAx5  #autism #specialneeds\n",
      "\n",
      "---\n",
      "\n",
      "Target: 0 (not a disaster)\n",
      "Text: @dopeitsval ahh you're bomb baby ??\n",
      "\n",
      "---\n",
      "\n",
      "Target: 0 (not a disaster)\n",
      "Text: @SirTitan45  Mega mood swing on a 24 hr schedule. Isn't that how structural failure occurs?\n",
      "\n",
      "---\n",
      "\n",
      "Target: 0 (not a disaster)\n",
      "Text: @FoxNewsInsider All Obama is doing is giving a false time schedule on Iran testing there first bomb      Bomb = Nuclear Suicide Vest\n",
      "\n",
      "---\n",
      "\n",
      "Target: 1 (disaster)\n",
      "Text: SSP East says a car AEG 061 driven by a young man collided with Akram's carhot words were exchanged n he did firing in air1bullet hit tyre\n",
      "\n",
      "---\n",
      "\n"
     ]
    }
   ],
   "source": [
    "random_index = random.randint(0, len(train_df)-5)\n",
    "\n",
    "for row in train_df_shuffle[['text', 'target']][random_index:random_index+5].itertuples():\n",
    "    _, text, target = row\n",
    "    print(f\"Target: {target}\",\"(disaster)\" if target>0 else \"(not a disaster)\")\n",
    "    print(f\"Text: {text}\\n\")\n",
    "    print(\"---\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b39dc76",
   "metadata": {},
   "source": [
    "```\n",
    "Target: 1 (disaster)\n",
    "Text: @WesleyLowery ?????? how are you going to survive this devastation?\n",
    "\n",
    "---\n",
    "\n",
    "Target: 0 (not a disaster)\n",
    "Text: Hollywood movie about trapped miners released in Chile http://t.co/xe0EE1Fzfh\n",
    "\n",
    "---\n",
    "\n",
    "Target: 0 (not a disaster)\n",
    "Text: #sing #tsunami Beginners #computer tutorial.: http://t.co/ukQYbhxMQI Everyone Wants To Learn To Build A Pc. Re http://t.co/iDWS2ZgYsa\n",
    "\n",
    "---\n",
    "\n",
    "Target: 1 (disaster)\n",
    "Text: #Reddit updates #content #policy promises to quarantine Û÷extremely offensiveÛª communities http://t.co/EHGtZhKAn4\n",
    "\n",
    "---\n",
    "\n",
    "Target: 1 (disaster)\n",
    "Text: Japan Marks 70th Anniversary of Hiroshima Atomic Bombing http://t.co/cQLM9jOJOP\n",
    "\n",
    "---\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3917a4d2",
   "metadata": {},
   "source": [
    "### Train Test Split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "7ec679f1",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "6851 762\n"
     ]
    }
   ],
   "source": [
    "train_tweets, val_tweets, train_labels, val_labels = train_test_split(\n",
    "    train_df_shuffle['text'].to_numpy(),\n",
    "    train_df_shuffle['target'].to_numpy(),\n",
    "    test_size=0.1,\n",
    "    random_state=SEED)\n",
    "\n",
    "print(len(train_tweets), len(val_tweets))\n",
    "# 6851 762"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb3ca9f6",
   "metadata": {},
   "source": [
    "### Tokenization and Embedding\n",
    "\n",
    "Machine learning models take vectors (arrays of numbers) as input. When working with text, the first thing you must do is come up with a strategy to convert strings to numbers (or to [\"vectorize\" the text](https://www.tensorflow.org/text/guide/word_embeddings)) before feeding it to the model.\n",
    "\n",
    "\n",
    "#### Tokenization\n",
    "\n",
    "[TextVectorization layer](https://www.tensorflow.org/tutorials/keras/text_classification) is a Keras layer to standardize, tokenize, and vectorize the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "61cec259",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "15\n"
     ]
    }
   ],
   "source": [
    "# find average number of words in tweets\n",
    "average_tokens_per_tweet=round(sum([len(i.split()) for i in train_tweets])/len(train_tweets))\n",
    "print(average_tokens_per_tweet)\n",
    "# 15"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "e8471748",
   "metadata": {},
   "outputs": [],
   "source": [
    "# create text vectorizer\n",
    "max_features = 10000 # limit to most common words\n",
    "sequence_length = 15 # limit to average number of words in tweets\n",
    "\n",
    "text_vectorizer = TextVectorization(\n",
    "    max_tokens=max_features, # set a value to only include most common words\n",
    "    standardize='lower_and_strip_punctuation',\n",
    "    split='whitespace',\n",
    "    ngrams=None, # set value to form common word groups\n",
    "    output_mode='int',\n",
    "    output_sequence_length=sequence_length, # set value to limit tweet size\n",
    "    pad_to_max_tokens=True # fluff tweets that are shorter than set max length\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "edbe4972",
   "metadata": {},
   "outputs": [],
   "source": [
    "# fit text vectorizer to training data\n",
    "text_vectorizer.adapt(train_tweets)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "0fc1b897",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(3, 15), dtype=int64, numpy=\n",
       "array([[ 274,   32, 4046,    1,    5,  370,    2, 5962,   88,    4,    0,\n",
       "           0,    0,    0,    0],\n",
       "       [ 706,  576,  473,  214,  206,   32,  354,  172, 1569,    0,    0,\n",
       "           0,    0,    0,    0],\n",
       "       [   1, 4013,    1,    1,    1,    1,    1,    1, 3878,    1,    1,\n",
       "           4,    1,    1,    1]])>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# test fitted vectorizer\n",
    "sample_sentence1 = \"Next I'm buying Coca-Cola to put the cocaine back in\"\n",
    "sample_sentence2 = \"Hey guys, wanna feel old? I'm 40. You're welcome.\"\n",
    "sample_sentence3 = \"Beef chicken pork bacon chuck shortloin sirloin shank eu, bresaola voluptate in enim ea kielbasa laboris brisket laborum, jowl labore id porkchop elit ad commodo.\"\n",
    "text_vectorizer([sample_sentence1,sample_sentence2,sample_sentence3])\n",
    "\n",
    "# <tf.Tensor: shape=(3, 15), dtype=int64, numpy=\n",
    "# array([[ 274,   32, 4046,    1,    5,  370,    2, 5962,   88,    4,    0,\n",
    "#            0,    0,    0,    0],\n",
    "#        [ 706,  576,  473,  214,  206,   32,  354,  172, 1569,    0,    0,\n",
    "#            0,    0,    0,    0],\n",
    "#        [   1, 4013,    1,    1,    1,    1,    1,    1, 3878,    1,    1,\n",
    "#            4,    1,    1,    1]])>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "c11297ea",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tweet: Rory McIlroy to Test Ankle Injury in Weekend Practice #chinadotcom #sports http://t.co/UDTGWfSc3P http://t.co/V5wSx0LQN2    \n",
      "\n",
      "Vector: [[4718 3646    5 1246 3182  345    4 1238 1614    1 1588    1    1    0\n",
      "     0]]\n"
     ]
    }
   ],
   "source": [
    "random_tweet = random.choice(train_tweets)\n",
    "vector = text_vectorizer([random_tweet])\n",
    "\n",
    "print(\n",
    "    f'Tweet: {random_tweet}\\\n",
    "    \\n\\nVector: {vector}'\n",
    ")\n",
    "\n",
    "# Tweet: Ignition Knock (Detonation) Sensor ACDelco GM Original Equipment 213-4678\n",
    "# Vector: [[ 888  885  580 1767    1 1671 1623 1863    1    1    1    0    0    0\n",
    "#      0]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "427fd692",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10000 ['', '[UNK]', 'the', 'a', 'in', 'to', 'of', 'and', 'i', 'is'] ['painthey', 'painful', 'paine', 'paging', 'pageshi', 'pages', 'paeds', 'pads', 'padres', 'paddytomlinson1']\n"
     ]
    }
   ],
   "source": [
    "# get unique vocabulary\n",
    "words_in_vocab = text_vectorizer.get_vocabulary()\n",
    "top_10_words = words_in_vocab[:10]\n",
    "bottom_10_words = words_in_vocab[-10:]\n",
    "\n",
    "print(\n",
    "    len(words_in_vocab),\n",
    "    top_10_words,\n",
    "    bottom_10_words\n",
    ")\n",
    "# 10000\n",
    "# ['', '[UNK]', 'the', 'a', 'in', 'to', 'of', 'and', 'i', 'is']\n",
    "# ['painthey', 'painful', 'paine', 'paging', 'pageshi', 'pages', 'paeds', 'pads', 'padres', 'paddytomlinson1']\n",
    "\n",
    "# The [UNK] stands for unknown - meaning outside of the 10.000 tokens limit"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "88aa3adc",
   "metadata": {},
   "source": [
    "#### Embedding\n",
    "\n",
    "The now vectorize data can be used as the first layer of the classification model, feeding transformed strings into an [Embedding layer](https://www.tensorflow.org/text/guide/word_embeddings). The Embedding layer takes the integer-encoded vocabulary and looks up the embedding vector for each word-index. These vectors are learned as the model trains. The vectors add a dimension to the output array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "1eef5af6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# create embedding layer\n",
    "embedding = Embedding(\n",
    "    input_dim = max_features,\n",
    "    output_dim = 128,\n",
    "    input_length = sequence_length\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "a508d154",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tweet: Californian Bush Fires 2015 http://t.co/rjdX29wosp    \n",
      "\n",
      "Embedding: [[[-0.02276199  0.02443738  0.02346965 ... -0.02255568  0.0031268\n",
      "   -0.03683709]\n",
      "  [ 0.03843458  0.02319275  0.02064592 ...  0.03542275  0.04694393\n",
      "    0.00230541]\n",
      "  [ 0.00060918  0.04467062 -0.01757476 ... -0.02506196  0.04068417\n",
      "    0.01316834]\n",
      "  ...\n",
      "  [ 0.01372515  0.01294402  0.03904391 ... -0.0131917   0.01678599\n",
      "    0.02401787]\n",
      "  [ 0.01372515  0.01294402  0.03904391 ... -0.0131917   0.01678599\n",
      "    0.02401787]\n",
      "  [ 0.01372515  0.01294402  0.03904391 ... -0.0131917   0.01678599\n",
      "    0.02401787]]], (1, 15, 128)\n"
     ]
    }
   ],
   "source": [
    "random_tweet = random.choice(train_tweets)\n",
    "sample_embedd = embedding(text_vectorizer([random_tweet]))\n",
    "\n",
    "print(\n",
    "    f'Tweet: {random_tweet}\\\n",
    "    \\n\\nEmbedding: {sample_embedd}, {sample_embedd.shape}'\n",
    ")\n",
    "\n",
    "# Tweet: @biggangVH1 looks like George was having a panic attack. LOL.    \n",
    "\n",
    "# Embedding: [[[-0.02811491 -0.02710991 -0.04273632 ...  0.01480064 -0.02413664\n",
    "#     0.02612327]\n",
    "#   [-0.013403   -0.04941868  0.03431542 ...  0.00432001 -0.03614474\n",
    "#     0.04559914]\n",
    "#   [ 0.00161045 -0.02501463  0.02461291 ... -0.02123032  0.02596099\n",
    "#    -0.02626952]\n",
    "#   ...\n",
    "#   [ 0.03742747  0.03854593 -0.02052871 ...  0.01287705 -0.04228047\n",
    "#    -0.02316147]\n",
    "#   [ 0.03742747  0.03854593 -0.02052871 ...  0.01287705 -0.04228047\n",
    "#    -0.02316147]\n",
    "#   [ 0.03742747  0.03854593 -0.02052871 ...  0.01287705 -0.04228047\n",
    "#    -0.02316147]]], (1, 15, 128)\n",
    "\n",
    "# the shape tells us the the layer received 1 input with the length of\n",
    "# 15 words (as set before and filled up with zeros if tweet is shorter)\n",
    "# and each of those words is now represented by a 128dim vector"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b47fbb1",
   "metadata": {},
   "source": [
    "## Experiments\n",
    "\n",
    "* Model 0: Naive Bayes (baseline)\n",
    "* Model 1: Feed-forward neural network (dense model)\n",
    "* Model 2: LSTM model (RNN)\n",
    "* Model 3: GRU model (RNN)\n",
    "* Model 4: Bidirectional-LSTM model (RNN)\n",
    "* Model 5: 1D-Convolutional Neural Network (CNN)\n",
    "* Model 6: TensorFlow Hub pre-trained NLP feature extractor\n",
    "* Model 6a: Model 6 with added complexity (add dense layer)\n",
    "* Model 6b: Model 6a with 10% training data\n",
    "* Model 6c: Model 6a with 10% training data (fixed sampling)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1321631",
   "metadata": {},
   "source": [
    "### Model 0: Naive Bayes tf-ids"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e30a8d08",
   "metadata": {},
   "source": [
    "#### Tokenization and Modelling Pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "fa1d0d69",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-1 {color: black;background-color: white;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>Pipeline(steps=[(&#x27;tfidf&#x27;, TfidfVectorizer()), (&#x27;clf&#x27;, MultinomialNB())])</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" ><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">Pipeline</label><div class=\"sk-toggleable__content\"><pre>Pipeline(steps=[(&#x27;tfidf&#x27;, TfidfVectorizer()), (&#x27;clf&#x27;, MultinomialNB())])</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" ><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">TfidfVectorizer</label><div class=\"sk-toggleable__content\"><pre>TfidfVectorizer()</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-3\" type=\"checkbox\" ><label for=\"sk-estimator-id-3\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">MultinomialNB</label><div class=\"sk-toggleable__content\"><pre>MultinomialNB()</pre></div></div></div></div></div></div></div>"
      ],
      "text/plain": [
       "Pipeline(steps=[('tfidf', TfidfVectorizer()), ('clf', MultinomialNB())])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_0 = Pipeline([\n",
    "    (\"tfidf\", TfidfVectorizer()),\n",
    "    (\"clf\", MultinomialNB())\n",
    "])\n",
    "\n",
    "# fit to training data\n",
    "model_0.fit(train_tweets, train_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0fe07cf0",
   "metadata": {},
   "source": [
    "#### Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "5114f333",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Baseline accuracy: 79.27%\n"
     ]
    }
   ],
   "source": [
    "baseline_score = model_0.score(val_tweets, val_labels)\n",
    "print(f\"Baseline accuracy: {baseline_score*100:.2f}%\")\n",
    "# Baseline accuracy: 79.27%"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d8c5e71",
   "metadata": {},
   "source": [
    "#### Predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "4172f8ef",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['DFR EP016 Monthly Meltdown - On Dnbheaven 2015.08.06 http://t.co/EjKRf8N8A8 #Drum and Bass #heavy #nasty http://t.co/SPHWE6wFI5'\n",
      " 'FedEx no longer to transport bioterror germs in wake of anthrax lab mishaps http://t.co/qZQc8WWwcN via @usatoday'\n",
      " 'Gunmen kill four in El Salvador bus attack: Suspected Salvadoran gang members killed four people and wounded s... http://t.co/CNtwB6ScZj'\n",
      " '@camilacabello97 Internally and externally screaming'\n",
      " 'Radiation emergency #preparedness starts with knowing to: get inside stay inside and stay tuned http://t.co/RFFPqBAz2F via @CDCgov'\n",
      " 'Investigators rule catastrophic structural failure resulted in 2014 Virg.. Related Articles: http://t.co/Cy1LFeNyV8'\n",
      " 'How the West was burned: Thousands of wildfires ablaze in #California alone http://t.co/iCSjGZ9tE1 #climate #energy http://t.co/9FxmN0l0Bd'\n",
      " \"Map: Typhoon Soudelor's predicted path as it approaches Taiwan; expected to make landfall over southern China by S\\x89Û_ http://t.co/JDVSGVhlIs\"\n",
      " '\\x89Ûª93 blasts accused Yeda Yakub dies in Karachi of heart attack http://t.co/mfKqyxd8XG #Mumbai'\n",
      " 'My ears are bleeding  https://t.co/k5KnNwugwT']\n",
      "[1 1 1 0 0 1 1 1 1 0]\n"
     ]
    }
   ],
   "source": [
    "baseline_preds = model_0.predict(val_tweets)\n",
    "\n",
    "print(val_tweets[:10])\n",
    "print(baseline_preds[:10])\n",
    "# array([1, 1, 1, 0, 0, 1, 1, 1, 1, 0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ad776be",
   "metadata": {},
   "source": [
    "> `[1 1 1 0 0 1 1 1 1 0]`\n",
    "\n",
    "1. 'DFR EP016 Monthly Meltdown - On Dnbheaven 2015.08.06 http://t.co/EjKRf8N8A8 #Drum and Bass #heavy #nasty http://t.co/SPHWE6wFI5'\n",
    "2. 'FedEx no longer to transport bioterror germs in wake of anthrax lab mishaps http://t.co/qZQc8WWwcN via @usatoday'\n",
    "3. 'Gunmen kill four in El Salvador bus attack: Suspected Salvadoran gang members killed four people and wounded s... http://t.co/CNtwB6ScZj'\n",
    "4. '@camilacabello97 Internally and externally screaming'\n",
    "5. 'Radiation emergency #preparedness starts with knowing to: get inside stay inside and stay tuned http://t.co/RFFPqBAz2F via @CDCgov'\n",
    "6. 'Investigators rule catastrophic structural failure resulted in 2014 Virg.. Related Articles: http://t.co/Cy1LFeNyV8'\n",
    "7. 'How the West was burned: Thousands of wildfires ablaze in #California alone http://t.co/iCSjGZ9tE1 #climate #energy http://t.co/9FxmN0l0Bd'\n",
    "8. \"Map: Typhoon Soudelor's predicted path as it approaches Taiwan; expected to make landfall over southern China by S\\x89Û_ http://t.co/JDVSGVhlIs\"\n",
    "9. '\\x89Ûª93 blasts accused Yeda Yakub dies in Karachi of heart attack http://t.co/mfKqyxd8XG #Mumbai'\n",
    "10. 'My ears are bleeding  https://t.co/k5KnNwugwT']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "43506e34",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 79.26509186351706, Precision: 0.8111390004213173, Recall: 0.7926509186351706, F1-Score: 0.7862189758049549\n"
     ]
    }
   ],
   "source": [
    "baseline_metrics = calculate_metrics(\n",
    "    y_true=val_labels,\n",
    "    y_pred=baseline_preds\n",
    ")\n",
    "\n",
    "print(f\"Accuracy: {baseline_metrics['accuracy']}, Precision: {baseline_metrics['precision']}, Recall: {baseline_metrics['recall']}, F1-Score: {baseline_metrics['f1']}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07d4de5d",
   "metadata": {},
   "source": [
    "#### Model 0 Metrics\n",
    "\n",
    "* Accuracy: 79.26509186351706,\n",
    "* Precision: 0.8111390004213173,\n",
    "* Recall: 0.7926509186351706,\n",
    "* F1-Score: 0.7862189758049549"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8f9b07e",
   "metadata": {},
   "source": [
    "### Model 1: Simple Dense\n",
    "\n",
    "#### Model Building and Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "323901a0",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model_1_dense\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " input_1 (InputLayer)        [(None, 1)]               0         \n",
      "                                                                 \n",
      " text_vectorization (TextVec  (None, 15)               0         \n",
      " torization)                                                     \n",
      "                                                                 \n",
      " embedding (Embedding)       (None, 15, 128)           1280000   \n",
      "                                                                 \n",
      " global_average_pooling1d (G  (None, 128)              0         \n",
      " lobalAveragePooling1D)                                          \n",
      "                                                                 \n",
      " dense (Dense)               (None, 1)                 129       \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 1,280,129\n",
      "Trainable params: 1,280,129\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# build model\n",
    "## inputs are single 1-dimensional strings\n",
    "inputs = Input(shape=(1,), dtype=tf.string)\n",
    "## turn strings into numbers\n",
    "x = text_vectorizer(inputs)\n",
    "## create embedding from vectorized input\n",
    "x = embedding(x)\n",
    "## instead of returning a prediction for every token/word\n",
    "## condense all to a single prediction for entire input string\n",
    "x = GlobalAveragePooling1D()(x)\n",
    "## sigmoid activated output for binary classification\n",
    "outputs = Dense(1, activation='sigmoid')(x)\n",
    "\n",
    "model_1 = tf.keras.Model(inputs, outputs, name='model_1_dense')\n",
    "model_1.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "946c829d",
   "metadata": {},
   "source": [
    "```\n",
    "Model: \"model_1_dense\"\n",
    "_________________________________________________________________\n",
    " Layer (type)                Output Shape              Param #   \n",
    "=================================================================\n",
    " input_3 (InputLayer)        [(None, 1)]               0         \n",
    "                                                                 \n",
    " text_vectorization (TextVec  (None, 15)               0         \n",
    " torization)                                                     \n",
    "                                                                 \n",
    " embedding (Embedding)       (None, 15, 128)           1280000   \n",
    "                                                                 \n",
    " global_average_pooling1d (G  (None, 128)              0         \n",
    " lobalAveragePooling1D)                                          \n",
    "                                                                 \n",
    " dense_2 (Dense)             (None, 1)                 129       \n",
    "                                                                 \n",
    "=================================================================\n",
    "Total params: 1,280,129\n",
    "Trainable params: 1,280,129\n",
    "Non-trainable params: 0\n",
    "_________________________________________________________________\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "381177db",
   "metadata": {},
   "outputs": [],
   "source": [
    "# compile model\n",
    "model_1.compile(\n",
    "    loss='binary_crossentropy',\n",
    "    optimizer=tf.keras.optimizers.Adam(),\n",
    "    metrics=['accuracy']\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "f32f2ddf",
   "metadata": {},
   "outputs": [],
   "source": [
    "# there seems to be an issue with the tb callback\n",
    "# https://github.com/keras-team/keras/issues/15163\n",
    "# changed histogram_freq=0\n",
    "## create a callback to track experiments in TensorBoard\n",
    "def create_tensorboard_callback_bugged(dir_name, experiment_name):\n",
    "    # log progress to log directory\n",
    "    log_dir = dir_name + \"/\" + experiment_name + \"/\" + datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\")\n",
    "    tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=0)\n",
    "    print(f\"INFO :: Saving TensorBoard Log to: {log_dir}\")\n",
    "    return tensorboard_callback"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "33a6c814",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO :: Saving TensorBoard Log to: tensorboad/model_1_dense/20230424-123502\n",
      "Epoch 1/5\n",
      "215/215 [==============================] - 3s 11ms/step - loss: 0.6107 - accuracy: 0.6933 - val_loss: 0.5342 - val_accuracy: 0.7612\n",
      "Epoch 2/5\n",
      "215/215 [==============================] - 2s 11ms/step - loss: 0.4420 - accuracy: 0.8168 - val_loss: 0.4712 - val_accuracy: 0.7835\n",
      "Epoch 3/5\n",
      "215/215 [==============================] - 2s 10ms/step - loss: 0.3477 - accuracy: 0.8596 - val_loss: 0.4560 - val_accuracy: 0.7913\n",
      "Epoch 4/5\n",
      "215/215 [==============================] - 2s 11ms/step - loss: 0.2846 - accuracy: 0.8904 - val_loss: 0.4663 - val_accuracy: 0.7887\n",
      "Epoch 5/5\n",
      "215/215 [==============================] - 2s 11ms/step - loss: 0.2372 - accuracy: 0.9123 - val_loss: 0.4825 - val_accuracy: 0.7861\n"
     ]
    }
   ],
   "source": [
    "# model training\n",
    "model_1_history = model_1.fit(\n",
    "    x=train_tweets,\n",
    "    y=train_labels,\n",
    "    epochs=5,\n",
    "    validation_data=(val_tweets, val_labels),\n",
    "    callbacks=[create_tensorboard_callback_bugged(\n",
    "        dir_name=LOG_DIR,\n",
    "        experiment_name='model_1_dense'\n",
    "    )]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e7d2797",
   "metadata": {},
   "source": [
    "#### Model Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "6ae7b987",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24/24 [==============================] - 0s 1ms/step - loss: 0.4825 - accuracy: 0.7861\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.4824865162372589, 0.7860892415046692]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_1.evaluate(val_tweets, val_labels)\n",
    "# loss: 0.4830 - accuracy: 0.7887"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "c427b619",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24/24 [==============================] - 0s 996us/step\n",
      "Prediction: [0.3651343] (not a disaster)\n"
     ]
    }
   ],
   "source": [
    "model_1_preds = model_1.predict(val_tweets)\n",
    "sample_prediction=model_1_preds[0]\n",
    "print(f\"Prediction: {sample_prediction}\",\"(disaster)\" if sample_prediction>0.5 else \"(not a disaster)\")\n",
    "# Prediction: [0.32197773] (not a disaster)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "563f42ef",
   "metadata": {},
   "outputs": [],
   "source": [
    "# convert model prediction probabilities to binary label format\n",
    "model_1_preds = tf.squeeze(tf.round(model_1_preds))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "51c9167d",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'accuracy': 78.60892388451444, 'precision': 0.7911704681403405, 'recall': 0.7860892388451444, 'f1': 0.7830068921982195}\n"
     ]
    }
   ],
   "source": [
    "model_1_metrics = calculate_metrics(\n",
    "    y_true=val_labels,\n",
    "    y_pred=model_1_preds\n",
    ")\n",
    "\n",
    "print(model_1_metrics)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1429eb01",
   "metadata": {},
   "source": [
    "#### Model 1 Metrics\n",
    "\n",
    "* Accuracy: 78.87139107611549\n",
    "* Precision: 0.7969619064252174\n",
    "* Recall: 0.7887139107611548\n",
    "* F1: 0.7847294282013199"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "59d16afa",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([False, False, False, False])"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# the model performs worse than the baseline model\n",
    "np.array(list(model_1_metrics.values())) > np.array(list(baseline_metrics.values()))\n",
    "# array([False, False, False, False])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cfec6135",
   "metadata": {},
   "source": [
    "#### Visualize the Embedding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "973ccb9a",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['', '[UNK]', 'the', 'a', 'in', 'to', 'of', 'and', 'i', 'is']\n"
     ]
    }
   ],
   "source": [
    "# get vocab from text vectorizer layer\n",
    "words_in_vocab = text_vectorizer.get_vocabulary()\n",
    "top_10_words = words_in_vocab[:10]\n",
    "print(top_10_words)\n",
    "# ['', '[UNK]', 'the', 'a', 'in', 'to', 'of', 'and', 'i', 'is']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "6063681c",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(10000, 128)\n"
     ]
    }
   ],
   "source": [
    "# get embedding weight matrix\n",
    "embed_weights = model_1.get_layer('embedding').get_weights()[0]\n",
    "print(embed_weights.shape)\n",
    "# (10000, 128) same size as max_features - one weight for every word in vocabulary"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc1c819d",
   "metadata": {},
   "source": [
    "The [embeddeding weights](https://www.tensorflow.org/tutorials/text/word2vec) started as random numbers assigned to each token/word in our dataset. By fitting this embedding space to our dataset these weights can now be used to group the words in our dataset. Words that belong to the same class should also have similar vectors representing them.\n",
    "\n",
    "We can use the [Tensorflow Projector](https://projector.tensorflow.org/) to display our embedding space:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "0afc7197",
   "metadata": {},
   "outputs": [],
   "source": [
    "out_v = io.open('embedding_weights/vectors_model1.tsv', 'w', encoding='utf-8')\n",
    "out_m = io.open('embedding_weights/metadata_model1.tsv', 'w', encoding='utf-8')\n",
    "\n",
    "for index, word in enumerate(words_in_vocab):\n",
    "    if index == 0:\n",
    "        continue # skip padding\n",
    "    vec = embed_weights[index]\n",
    "    out_v.write('\\t'.join([str(x) for x in vec]) + '\\n')\n",
    "    out_m.write(word + '\\n')\n",
    "    \n",
    "out_v.close()\n",
    "out_m.close()\n",
    "\n",
    "# Upload both files to the [Tensorflow Projector](https://projector.tensorflow.org/)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fade3d6f",
   "metadata": {},
   "source": [
    "![(Re) Introduction to Tensorflow Natural Language Processing](./assets/tf_nlp_desaster_tweets_01.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e19f1f6a",
   "metadata": {},
   "source": [
    "The projection shows a clear separation between our two classes showing each word in our vocabulary as a member of one of two clusters."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f1a633b",
   "metadata": {},
   "source": [
    "### Model 2: LSTM Long-term Short-term Memory RNN\n",
    "\n",
    "#### Model Building and Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "70e5e9ef",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model_2_LSTM\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " input_2 (InputLayer)        [(None, 1)]               0         \n",
      "                                                                 \n",
      " text_vectorization (TextVec  (None, 15)               0         \n",
      " torization)                                                     \n",
      "                                                                 \n",
      " embedding (Embedding)       (None, 15, 128)           1280000   \n",
      "                                                                 \n",
      " lstm (LSTM)                 (None, 64)                49408     \n",
      "                                                                 \n",
      " dense_1 (Dense)             (None, 64)                4160      \n",
      "                                                                 \n",
      " dense_2 (Dense)             (None, 1)                 65        \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 1,333,633\n",
      "Trainable params: 1,333,633\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 12:36:18.974737: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:36:18.975932: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:36:18.976773: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n"
     ]
    }
   ],
   "source": [
    "inputs = Input(shape=(1,), dtype='string')\n",
    "x = text_vectorizer(inputs)\n",
    "x = embedding(x)\n",
    "# x = LSTM(64, return_sequences=True)(x)\n",
    "x = LSTM(64)(x)\n",
    "x = Dense(64, activation='relu')(x)\n",
    "outputs = Dense(1, activation='sigmoid')(x)\n",
    "\n",
    "model_2 = tf.keras.Model(inputs, outputs, name=\"model_2_LSTM\")\n",
    "model_2.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ad9cd3b",
   "metadata": {},
   "source": [
    "```\n",
    "Model: \"model_2_LSTM\"\n",
    "_________________________________________________________________\n",
    " Layer (type)                Output Shape              Param #   \n",
    "=================================================================\n",
    " input_3 (InputLayer)        [(None, 1)]               0         \n",
    "                                                                 \n",
    " text_vectorization (TextVec  (None, 15)               0         \n",
    " torization)                                                     \n",
    "                                                                 \n",
    " embedding (Embedding)       (None, 15, 128)           1280000   \n",
    "                                                                 \n",
    " lstm_2 (LSTM)               (None, 64)                49408     \n",
    "                                                                 \n",
    " dense_3 (Dense)             (None, 64)                4160      \n",
    "                                                                 \n",
    " dense_4 (Dense)             (None, 1)                 65        \n",
    "                                                                 \n",
    "=================================================================\n",
    "Total params: 1,333,633\n",
    "Trainable params: 1,333,633\n",
    "Non-trainable params: 0\n",
    "_________________________________________________________________\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "f433734f",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_2.compile(\n",
    "    loss='binary_crossentropy',\n",
    "    optimizer=tf.keras.optimizers.Adam(),\n",
    "    metrics=['accuracy']\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "6a3bf8b1",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO :: Saving TensorBoard Log to: tensorboad/model_2_lstm/20230424-123624\n",
      "Epoch 1/5\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 12:36:24.703596: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:36:24.705086: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:36:24.706383: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n",
      "2023-04-24 12:36:25.306131: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:36:25.307521: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:36:25.308696: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "214/215 [============================>.] - ETA: 0s - loss: 0.2236 - accuracy: 0.9182"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 12:36:29.358635: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:36:29.360184: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:36:29.361054: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "215/215 [==============================] - 5s 15ms/step - loss: 0.2235 - accuracy: 0.9183 - val_loss: 0.5882 - val_accuracy: 0.7808\n",
      "Epoch 2/5\n",
      "215/215 [==============================] - 3s 14ms/step - loss: 0.1558 - accuracy: 0.9425 - val_loss: 0.5705 - val_accuracy: 0.7835\n",
      "Epoch 3/5\n",
      "215/215 [==============================] - 3s 14ms/step - loss: 0.1300 - accuracy: 0.9501 - val_loss: 0.6402 - val_accuracy: 0.7795\n",
      "Epoch 4/5\n",
      "215/215 [==============================] - 3s 14ms/step - loss: 0.1056 - accuracy: 0.9590 - val_loss: 0.8448 - val_accuracy: 0.7808\n",
      "Epoch 5/5\n",
      "215/215 [==============================] - 3s 14ms/step - loss: 0.0846 - accuracy: 0.9641 - val_loss: 0.9677 - val_accuracy: 0.7730\n"
     ]
    }
   ],
   "source": [
    "model_2_history = model_2.fit(\n",
    "    train_tweets,\n",
    "    train_labels,\n",
    "    epochs=5,\n",
    "    validation_data=(val_tweets, val_labels),\n",
    "    callbacks=[create_tensorboard_callback_bugged(\n",
    "        dir_name=LOG_DIR,\n",
    "        experiment_name='model_2_lstm'\n",
    "    )]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46a54a2a",
   "metadata": {},
   "source": [
    "#### Model Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "980dbdb7",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24/24 [==============================] - 0s 3ms/step - loss: 0.9677 - accuracy: 0.7730\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.9677377343177795, 0.7729659080505371]"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_2.evaluate(val_tweets, val_labels)\n",
    "# loss: 1.6905 - accuracy: 0.7743"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "3f803c8b",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24/24 [==============================] - 0s 3ms/step\n",
      "Prediction: [0.02846662] (not a disaster)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 12:36:49.735299: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:36:49.736520: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:36:49.737740: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n"
     ]
    }
   ],
   "source": [
    "# make predictions\n",
    "model_2_preds = model_2.predict(val_tweets)\n",
    "sample_prediction=model_2_preds[0]\n",
    "print(f\"Prediction: {sample_prediction}\",\"(disaster)\" if sample_prediction>0.5 else \"(not a disaster)\")\n",
    "# Prediction: [0.04310093] (not a disaster)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "e551d1fa",
   "metadata": {},
   "outputs": [],
   "source": [
    "# convert model prediction probabilities to binary label format\n",
    "model_2_preds = tf.squeeze(tf.round(model_2_preds))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "54d18580",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'accuracy': 77.29658792650919, 'precision': 0.7738445106757977, 'recall': 0.7729658792650919, 'f1': 0.7713337273803944}\n"
     ]
    }
   ],
   "source": [
    "model_2_metrics = calculate_metrics(\n",
    "    y_true=val_labels,\n",
    "    y_pred=model_2_preds\n",
    ")\n",
    "\n",
    "print(model_2_metrics)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6fc99273",
   "metadata": {},
   "source": [
    "#### Model 2 Metrics\n",
    "\n",
    "* Accuracy: 76.77165354330708\n",
    "* Precision: 0.7674723453090632\n",
    "* Recall: 0.7677165354330708\n",
    "* F1: 0.7668863186407149"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "fce55fd3",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([False, False, False, False])"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# the model performs worse than the previous model\n",
    "np.array(list(model_2_metrics.values())) > np.array(list(model_1_metrics.values()))\n",
    "# array([False, False, False, False])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb4a4864",
   "metadata": {},
   "source": [
    "### Model 3: GRU Gated Recurrent Unit RNN\n",
    "\n",
    "#### Model Building and Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "02b3f4c1",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model_3_GRU\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " input_3 (InputLayer)        [(None, 1)]               0         \n",
      "                                                                 \n",
      " text_vectorization (TextVec  (None, 15)               0         \n",
      " torization)                                                     \n",
      "                                                                 \n",
      " embedding (Embedding)       (None, 15, 128)           1280000   \n",
      "                                                                 \n",
      " gru (GRU)                   (None, 64)                37248     \n",
      "                                                                 \n",
      " dense_3 (Dense)             (None, 64)                4160      \n",
      "                                                                 \n",
      " dense_4 (Dense)             (None, 1)                 65        \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 1,321,473\n",
      "Trainable params: 1,321,473\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 12:37:09.316422: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:37:09.317993: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:37:09.319004: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n"
     ]
    }
   ],
   "source": [
    "inputs = Input(shape=(1,), dtype=tf.string)\n",
    "x = text_vectorizer(inputs)\n",
    "x = embedding(x)\n",
    "# x = GRU(64, return_sequences=True)(x)\n",
    "x = GRU(64)(x)\n",
    "x = Dense(64, activation='relu')(x)\n",
    "outputs = Dense(1, activation='sigmoid')(x)\n",
    "\n",
    "model_3 = tf.keras.Model(inputs, outputs, name=\"model_3_GRU\")\n",
    "model_3.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7904b6a4",
   "metadata": {},
   "source": [
    "```\n",
    "Model: \"model_3_GRU\"\n",
    "_________________________________________________________________\n",
    " Layer (type)                Output Shape              Param #   \n",
    "=================================================================\n",
    " input_4 (InputLayer)        [(None, 1)]               0         \n",
    "                                                                 \n",
    " text_vectorization (TextVec  (None, 15)               0         \n",
    " torization)                                                     \n",
    "                                                                 \n",
    " embedding (Embedding)       (None, 15, 128)           1280000   \n",
    "                                                                 \n",
    " gru (GRU)                   (None, 64)                37248     \n",
    "                                                                 \n",
    " dense_5 (Dense)             (None, 64)                4160      \n",
    "                                                                 \n",
    " dense_6 (Dense)             (None, 1)                 65        \n",
    "                                                                 \n",
    "=================================================================\n",
    "Total params: 1,321,473\n",
    "Trainable params: 1,321,473\n",
    "Non-trainable params: 0\n",
    "_________________________________________________________________\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "d1bd4357",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_3.compile(\n",
    "    loss='binary_crossentropy',\n",
    "    optimizer=tf.keras.optimizers.Adam(),\n",
    "    metrics=['accuracy']\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "87b9e60b",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO :: Saving TensorBoard Log to: tensorboad/model_3_gru/20230424-123715\n",
      "Epoch 1/5\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 12:37:15.976726: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:37:15.978524: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:37:15.979462: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n",
      "2023-04-24 12:37:16.621129: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:37:16.622329: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:37:16.623360: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "215/215 [==============================] - ETA: 0s - loss: 0.1472 - accuracy: 0.9477"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 12:37:20.577407: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:37:20.578712: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:37:20.579970: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "215/215 [==============================] - 5s 15ms/step - loss: 0.1472 - accuracy: 0.9477 - val_loss: 0.8435 - val_accuracy: 0.7743\n",
      "Epoch 2/5\n",
      "215/215 [==============================] - 3s 14ms/step - loss: 0.0797 - accuracy: 0.9689 - val_loss: 0.8957 - val_accuracy: 0.7769\n",
      "Epoch 3/5\n",
      "215/215 [==============================] - 3s 13ms/step - loss: 0.0634 - accuracy: 0.9742 - val_loss: 1.3662 - val_accuracy: 0.7703\n",
      "Epoch 4/5\n",
      "215/215 [==============================] - 3s 13ms/step - loss: 0.0550 - accuracy: 0.9769 - val_loss: 1.0799 - val_accuracy: 0.7743\n",
      "Epoch 5/5\n",
      "215/215 [==============================] - 3s 13ms/step - loss: 0.0504 - accuracy: 0.9772 - val_loss: 1.2978 - val_accuracy: 0.7717\n"
     ]
    }
   ],
   "source": [
    "model_3_history = model_3.fit(\n",
    "    train_tweets,\n",
    "    train_labels,\n",
    "    epochs=5,\n",
    "    validation_data=(val_tweets, val_labels),\n",
    "    callbacks=[create_tensorboard_callback_bugged(\n",
    "        dir_name=LOG_DIR,\n",
    "        experiment_name='model_3_gru'\n",
    "    )]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5525a6d4",
   "metadata": {},
   "source": [
    "#### Model Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "b3fa68ab",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24/24 [==============================] - 0s 2ms/step - loss: 1.2978 - accuracy: 0.7717\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[1.2977620363235474, 0.7716535329818726]"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_3.evaluate(val_tweets, val_labels)\n",
    "# loss: 1.2467 - accuracy: 0.7703"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "a3f7a77d",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24/24 [==============================] - 0s 2ms/step\n",
      "Prediction: [0.00038226] (not a disaster)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 12:45:56.597718: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:45:56.598814: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:45:56.599643: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n"
     ]
    }
   ],
   "source": [
    "# make predictions\n",
    "model_3_preds = model_3.predict(val_tweets)\n",
    "sample_prediction=model_3_preds[0]\n",
    "print(f\"Prediction: {sample_prediction}\",\"(disaster)\" if sample_prediction>0.5 else \"(not a disaster)\")\n",
    "# Prediction: [0.00013416] (not a disaster)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "df5aa843",
   "metadata": {},
   "outputs": [],
   "source": [
    "# convert model prediction probabilities to binary label format\n",
    "model_3_preds = tf.squeeze(tf.round(model_3_preds))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "01bf658c",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'accuracy': 77.16535433070865, 'precision': 0.780814749599424, 'recall': 0.7716535433070866, 'f1': 0.7666937899362429}\n"
     ]
    }
   ],
   "source": [
    "model_3_metrics = calculate_metrics(\n",
    "    y_true=val_labels,\n",
    "    y_pred=model_3_preds\n",
    ")\n",
    "\n",
    "print(model_3_metrics)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ccf8209d",
   "metadata": {},
   "source": [
    "#### Model 3 Metrics\n",
    "\n",
    "* Accuracy: 76.9028871391076\n",
    "* Precision: 0.7768747910576218\n",
    "* Recall: 0.7690288713910761\n",
    "* F1: 0.7643954892702002"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "fb35f14d",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[False  True False False]\n",
      "[False False False False]\n"
     ]
    }
   ],
   "source": [
    "# the model performs (sometimes) better than the lstm but still worse than the baseline model\n",
    "print(np.array(list(model_3_metrics.values())) > np.array(list(model_2_metrics.values())))\n",
    "# array([ True,  True,  True, False])\n",
    "print(np.array(list(model_3_metrics.values())) > np.array(list(baseline_metrics.values())))\n",
    "# array([False, False, False, False])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aced1f5b",
   "metadata": {},
   "source": [
    "### Model 4: Bi-Directional RNN\n",
    "\n",
    "#### Model Building and Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "036e14f1",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 12:46:19.963503: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:46:19.964747: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:46:19.966069: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n",
      "2023-04-24 12:46:20.082928: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis' with dtype int32 and shape [1]\n",
      "\t [[{{node gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis}}]]\n",
      "2023-04-24 12:46:20.121402: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:46:20.122657: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:46:20.123493: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model_4_bidirectional\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " input_4 (InputLayer)        [(None, 1)]               0         \n",
      "                                                                 \n",
      " text_vectorization (TextVec  (None, 15)               0         \n",
      " torization)                                                     \n",
      "                                                                 \n",
      " embedding (Embedding)       (None, 15, 128)           1280000   \n",
      "                                                                 \n",
      " bidirectional (Bidirectiona  (None, 15, 128)          98816     \n",
      " l)                                                              \n",
      "                                                                 \n",
      " bidirectional_1 (Bidirectio  (None, 128)              74496     \n",
      " nal)                                                            \n",
      "                                                                 \n",
      " dense_5 (Dense)             (None, 1)                 129       \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 1,453,441\n",
      "Trainable params: 1,453,441\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 12:46:20.277522: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:46:20.278685: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:46:20.279586: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n",
      "2023-04-24 12:46:20.389457: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis' with dtype int32 and shape [1]\n",
      "\t [[{{node gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis}}]]\n",
      "2023-04-24 12:46:20.417844: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:46:20.418745: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:46:20.419599: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n"
     ]
    }
   ],
   "source": [
    "inputs = Input(shape=(1,), dtype=tf.string)\n",
    "x = text_vectorizer(inputs)\n",
    "x = embedding(x)\n",
    "x = Bidirectional(LSTM(64, return_sequences=True))(x)\n",
    "x = Bidirectional(GRU(64))(x)\n",
    "outputs = Dense(1, activation='sigmoid')(x)\n",
    "\n",
    "model_4 = tf.keras.Model(inputs, outputs, name='model_4_bidirectional')\n",
    "model_4.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67ad000c",
   "metadata": {},
   "source": [
    "```\n",
    "Model: \"model_4_bidirectional\"\n",
    "_________________________________________________________________\n",
    " Layer (type)                Output Shape              Param #   \n",
    "=================================================================\n",
    " input_6 (InputLayer)        [(None, 1)]               0         \n",
    "                                                                 \n",
    " text_vectorization (TextVec  (None, 15)               0         \n",
    " torization)                                                     \n",
    "                                                                 \n",
    " embedding (Embedding)       (None, 15, 128)           1280000   \n",
    "                                                                 \n",
    " bidirectional_2 (Bidirectio  (None, 15, 128)          98816     \n",
    " nal)                                                            \n",
    "                                                                 \n",
    " bidirectional_3 (Bidirectio  (None, 128)              74496     \n",
    " nal)                                                            \n",
    "                                                                 \n",
    " dense_8 (Dense)             (None, 1)                 129       \n",
    "                                                                 \n",
    "=================================================================\n",
    "Total params: 1,453,441\n",
    "Trainable params: 1,453,441\n",
    "Non-trainable params: 0\n",
    "_________________________________________________________________\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "df157e9b",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_4.compile(\n",
    "    loss='binary_crossentropy',\n",
    "    optimizer=tf.keras.optimizers.Adam(),\n",
    "    metrics=['accuracy']\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "56692589",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO :: Saving TensorBoard Log to: tensorboad/model_4_bidirectional/20230424-124628\n",
      "Epoch 1/5\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 12:46:28.330220: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:46:28.331507: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:46:28.332457: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n",
      "2023-04-24 12:46:28.453143: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis' with dtype int32 and shape [1]\n",
      "\t [[{{node gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis}}]]\n",
      "2023-04-24 12:46:28.493967: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:46:28.494999: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:46:28.495898: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n",
      "2023-04-24 12:46:28.641634: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:46:28.642864: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:46:28.643723: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n",
      "2023-04-24 12:46:28.753421: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis' with dtype int32 and shape [1]\n",
      "\t [[{{node gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis}}]]\n",
      "2023-04-24 12:46:28.785238: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:46:28.786445: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:46:28.787671: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n",
      "2023-04-24 12:46:29.256945: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis' with dtype int32 and shape [1]\n",
      "\t [[{{node gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis}}]]\n",
      "2023-04-24 12:46:29.679133: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis' with dtype int32 and shape [1]\n",
      "\t [[{{node gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis}}]]\n",
      "2023-04-24 12:46:30.309847: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:46:30.311163: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:46:30.312066: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n",
      "2023-04-24 12:46:30.430795: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis' with dtype int32 and shape [1]\n",
      "\t [[{{node gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis}}]]\n",
      "2023-04-24 12:46:30.471712: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:46:30.473025: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:46:30.473962: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 12:46:30.613911: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:46:30.615007: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:46:30.616041: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n",
      "2023-04-24 12:46:30.721258: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis' with dtype int32 and shape [1]\n",
      "\t [[{{node gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis}}]]\n",
      "2023-04-24 12:46:30.751747: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:46:30.752866: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:46:30.754165: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n",
      "2023-04-24 12:46:31.246348: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis' with dtype int32 and shape [1]\n",
      "\t [[{{node gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis}}]]\n",
      "2023-04-24 12:46:31.612678: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis' with dtype int32 and shape [1]\n",
      "\t [[{{node gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis}}]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "215/215 [==============================] - ETA: 0s - loss: 0.0968 - accuracy: 0.9692"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 12:46:37.640795: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:46:37.642189: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:46:37.643057: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n",
      "2023-04-24 12:46:37.761285: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis' with dtype int32 and shape [1]\n",
      "\t [[{{node gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis}}]]\n",
      "2023-04-24 12:46:37.802923: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:46:37.804253: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:46:37.805433: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n",
      "2023-04-24 12:46:37.963315: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:46:37.964442: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:46:37.965576: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n",
      "2023-04-24 12:46:38.075448: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis' with dtype int32 and shape [1]\n",
      "\t [[{{node gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis}}]]\n",
      "2023-04-24 12:46:38.105916: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:46:38.107003: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:46:38.107903: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "215/215 [==============================] - 10s 25ms/step - loss: 0.0968 - accuracy: 0.9692 - val_loss: 1.2948 - val_accuracy: 0.7756\n",
      "Epoch 2/5\n",
      "215/215 [==============================] - 4s 20ms/step - loss: 0.0536 - accuracy: 0.9765 - val_loss: 1.3635 - val_accuracy: 0.7717\n",
      "Epoch 3/5\n",
      "215/215 [==============================] - 4s 20ms/step - loss: 0.0463 - accuracy: 0.9793 - val_loss: 1.5739 - val_accuracy: 0.7651\n",
      "Epoch 4/5\n",
      "215/215 [==============================] - 4s 20ms/step - loss: 0.0405 - accuracy: 0.9813 - val_loss: 1.4010 - val_accuracy: 0.7612\n",
      "Epoch 5/5\n",
      "215/215 [==============================] - 4s 20ms/step - loss: 0.0431 - accuracy: 0.9784 - val_loss: 1.3421 - val_accuracy: 0.7717\n"
     ]
    }
   ],
   "source": [
    "model_4_history = model_4.fit(\n",
    "    train_tweets,\n",
    "    train_labels,\n",
    "    epochs=5,\n",
    "    validation_data=(val_tweets, val_labels),\n",
    "    callbacks=[create_tensorboard_callback_bugged(\n",
    "        dir_name=LOG_DIR,\n",
    "        experiment_name='model_4_bidirectional'\n",
    "    )]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d03d0db",
   "metadata": {},
   "source": [
    "#### Model Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "8ebcc8dd",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24/24 [==============================] - 0s 5ms/step - loss: 1.3421 - accuracy: 0.7717\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[1.342091679573059, 0.7716535329818726]"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_4.evaluate(val_tweets, val_labels)\n",
    "# loss: 1.7367 - accuracy: 0.7756"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "55284fa7",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 12:47:24.216666: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:47:24.217836: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:47:24.219140: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n",
      "2023-04-24 12:47:24.327543: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis' with dtype int32 and shape [1]\n",
      "\t [[{{node gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis}}]]\n",
      "2023-04-24 12:47:24.365285: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:47:24.366353: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:47:24.367199: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n",
      "2023-04-24 12:47:24.531892: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:47:24.533971: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:47:24.535534: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n",
      "2023-04-24 12:47:24.643857: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis' with dtype int32 and shape [1]\n",
      "\t [[{{node gradients/ReverseV2_grad/ReverseV2/ReverseV2/axis}}]]\n",
      "2023-04-24 12:47:24.672723: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_2_grad/concat/split_2/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_2_grad/concat/split_2/split_dim}}]]\n",
      "2023-04-24 12:47:24.673609: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_grad/concat/split/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_grad/concat/split/split_dim}}]]\n",
      "2023-04-24 12:47:24.674381: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/split_1_grad/concat/split_1/split_dim' with dtype int32\n",
      "\t [[{{node gradients/split_1_grad/concat/split_1/split_dim}}]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24/24 [==============================] - 1s 4ms/step\n",
      "Prediction: [0.1428202] (not a disaster)\n"
     ]
    }
   ],
   "source": [
    "# make predictions\n",
    "model_4_preds = model_4.predict(val_tweets)\n",
    "sample_prediction=model_4_preds[0]\n",
    "print(f\"Prediction: {sample_prediction}\",\"(disaster)\" if sample_prediction>0.5 else \"(not a disaster)\")\n",
    "# Prediction: [0.00025736] (not a disaster)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "1f6b23a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# convert model prediction probabilities to binary label format\n",
    "model_4_preds = tf.squeeze(tf.round(model_4_preds))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "96bb9434",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'accuracy': 77.16535433070865, 'precision': 0.773559668677779, 'recall': 0.7716535433070866, 'f1': 0.7694523344399302}\n"
     ]
    }
   ],
   "source": [
    "model_4_metrics = calculate_metrics(\n",
    "    y_true=val_labels,\n",
    "    y_pred=model_4_preds\n",
    ")\n",
    "\n",
    "print(model_4_metrics)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6879a12",
   "metadata": {},
   "source": [
    "#### Model 4 Metrics\n",
    "\n",
    "* Accuracy: 77.55905511811024\n",
    "* Precision: 0.7777490986405654\n",
    "* Recall: 0.7755905511811023\n",
    "* F1: 0.7733619560087615"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "40d1c1a1",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[False False False  True]\n",
      "[False False False False]\n"
     ]
    }
   ],
   "source": [
    "# the model performs better than the gru but worse than the baseline model\n",
    "print(np.array(list(model_4_metrics.values())) > np.array(list(model_3_metrics.values())))\n",
    "# [ True  True  True  True]\n",
    "print(np.array(list(model_4_metrics.values())) > np.array(list(baseline_metrics.values())))\n",
    "# [False False False False]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f0a3057",
   "metadata": {},
   "source": [
    "### Model 5: Conv1D\n",
    "\n",
    "#### Model Building and Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "12e1f2c8",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model_5_conv1d\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " input_5 (InputLayer)        [(None, 1)]               0         \n",
      "                                                                 \n",
      " text_vectorization (TextVec  (None, 15)               0         \n",
      " torization)                                                     \n",
      "                                                                 \n",
      " embedding (Embedding)       (None, 15, 128)           1280000   \n",
      "                                                                 \n",
      " conv1d (Conv1D)             (None, 11, 64)            41024     \n",
      "                                                                 \n",
      " global_max_pooling1d (Globa  (None, 64)               0         \n",
      " lMaxPooling1D)                                                  \n",
      "                                                                 \n",
      " dense_6 (Dense)             (None, 1)                 65        \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 1,321,089\n",
      "Trainable params: 1,321,089\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "inputs = Input(shape=(1,), dtype=tf.string)\n",
    "x = text_vectorizer(inputs)\n",
    "x = embedding(x)\n",
    "x = Conv1D(\n",
    "    filters=64,\n",
    "    kernel_size=5, # check 5 words at a time\n",
    "    activation='relu',\n",
    "    padding='valid',\n",
    "    strides=1\n",
    ")(x)\n",
    "x = GlobalMaxPool1D()(x)\n",
    "outputs = Dense(1, activation='sigmoid')(x)\n",
    "\n",
    "model_5 = tf.keras.Model(inputs, outputs, name='model_5_conv1d')\n",
    "model_5.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c8a623a",
   "metadata": {},
   "source": [
    "```\n",
    "Model: \"model_5_conv1d\"\n",
    "_________________________________________________________________\n",
    " Layer (type)                Output Shape              Param #   \n",
    "=================================================================\n",
    " input_14 (InputLayer)       [(None, 1)]               0         \n",
    "                                                                 \n",
    " text_vectorization (TextVec  (None, 15)               0         \n",
    " torization)                                                     \n",
    "                                                                 \n",
    " embedding (Embedding)       (None, 15, 128)           1280000   \n",
    "                                                                 \n",
    " conv1d_6 (Conv1D)           (None, 11, 64)            41024     \n",
    "                                                                 \n",
    " global_max_pooling1d_5 (Glo  (None, 64)               0         \n",
    " balMaxPooling1D)                                                \n",
    "                                                                 \n",
    " dense_14 (Dense)            (None, 1)                 65        \n",
    "                                                                 \n",
    "=================================================================\n",
    "Total params: 1,321,089\n",
    "Trainable params: 1,321,089\n",
    "Non-trainable params: 0\n",
    "_________________________________________________________________\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "d08d1436",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_5.compile(\n",
    "    loss='binary_crossentropy',\n",
    "    optimizer=tf.keras.optimizers.Adam(),\n",
    "    metrics=['accuracy']\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "dc5f4cd4",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO :: Saving TensorBoard Log to: tensorboad/model_5_conv1d/20230424-124800\n",
      "Epoch 1/5\n",
      "215/215 [==============================] - 3s 12ms/step - loss: 0.1265 - accuracy: 0.9556 - val_loss: 0.8945 - val_accuracy: 0.7651\n",
      "Epoch 2/5\n",
      "215/215 [==============================] - 2s 11ms/step - loss: 0.0757 - accuracy: 0.9742 - val_loss: 1.0369 - val_accuracy: 0.7625\n",
      "Epoch 3/5\n",
      "215/215 [==============================] - 2s 11ms/step - loss: 0.0621 - accuracy: 0.9750 - val_loss: 1.1544 - val_accuracy: 0.7585\n",
      "Epoch 4/5\n",
      "215/215 [==============================] - 2s 11ms/step - loss: 0.0551 - accuracy: 0.9775 - val_loss: 1.1593 - val_accuracy: 0.7625\n",
      "Epoch 5/5\n",
      "215/215 [==============================] - 2s 11ms/step - loss: 0.0515 - accuracy: 0.9778 - val_loss: 1.2429 - val_accuracy: 0.7546\n"
     ]
    }
   ],
   "source": [
    "model_5_history = model_5.fit(\n",
    "    train_tweets,\n",
    "    train_labels,\n",
    "    epochs=5,\n",
    "    validation_data=(val_tweets, val_labels),\n",
    "    callbacks=[create_tensorboard_callback_bugged(\n",
    "        dir_name=LOG_DIR,\n",
    "        experiment_name='model_5_conv1d'\n",
    "    )]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e2901a1",
   "metadata": {},
   "source": [
    "#### Model Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "a0d506e5",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24/24 [==============================] - 0s 2ms/step - loss: 1.2429 - accuracy: 0.7546\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[1.2428834438323975, 0.7545931935310364]"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_5.evaluate(val_tweets, val_labels)\n",
    "# loss: 1.3018 - accuracy: 0.7454"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "d6f6ec8b",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24/24 [==============================] - 0s 1ms/step\n",
      "Prediction: [0.02216522] (not a disaster)\n"
     ]
    }
   ],
   "source": [
    "# make predictions\n",
    "model_5_preds = model_5.predict(val_tweets)\n",
    "sample_prediction=model_5_preds[0]\n",
    "print(f\"Prediction: {sample_prediction}\",\"(disaster)\" if sample_prediction>0.5 else \"(not a disaster)\")\n",
    "# Prediction: [0.05387583] (not a disaster)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "538b2a99",
   "metadata": {},
   "outputs": [],
   "source": [
    "# convert model prediction probabilities to binary label format\n",
    "model_5_preds = tf.squeeze(tf.round(model_5_preds))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "id": "e3d5248e",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'accuracy': 75.45931758530183, 'precision': 0.7552799761217521, 'recall': 0.7545931758530183, 'f1': 0.7527017331356143}\n"
     ]
    }
   ],
   "source": [
    "model_5_metrics = calculate_metrics(\n",
    "    y_true=val_labels,\n",
    "    y_pred=model_5_preds\n",
    ")\n",
    "\n",
    "print(model_5_metrics)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "739a2e56",
   "metadata": {},
   "source": [
    "#### Model 5 Metrics\n",
    "\n",
    "* Accuracy: 74.67191601049869\n",
    "* Precision: 0.7465631385096111\n",
    "* Recall: 0.7467191601049868\n",
    "* F1: 0.7453858813570734"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "df8bfce3",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[False False False False]\n",
      "[False False False False]\n"
     ]
    }
   ],
   "source": [
    "# the model performs worse than the gru and worse than the baseline model\n",
    "print(np.array(list(model_5_metrics.values())) > np.array(list(model_4_metrics.values())))\n",
    "# [False False False False]\n",
    "print(np.array(list(model_5_metrics.values())) > np.array(list(baseline_metrics.values())))\n",
    "# [False False False False]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be663544",
   "metadata": {},
   "source": [
    "### Model 6: Transfer Learning Feature Extractor\n",
    "\n",
    "__USE__ Feature Extractor - [Universal Sentence Encoder](https://arxiv.org/abs/1803.11175)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "id": "ead8f0a7",
   "metadata": {},
   "outputs": [],
   "source": [
    "# https://tfhub.dev/google/collections/universal-sentence-encoder/1\n",
    "embed = hub.load(\"https://tfhub.dev/google/universal-sentence-encoder/4\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "id": "cf6995ed",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tf.Tensor(\n",
      "[[ 0.06844153 -0.0325974  -0.01901028 ... -0.03307429 -0.04625704\n",
      "  -0.08149158]\n",
      " [ 0.02810995 -0.06714624  0.02414106 ... -0.02519046  0.03197665\n",
      "   0.02462349]\n",
      " [ 0.03188843 -0.0167392  -0.03194157 ... -0.04541751 -0.05822486\n",
      "  -0.07237621]], shape=(3, 512), dtype=float32)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 12:53:22.132291: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype string\n",
      "\t [[{{node inputs}}]]\n"
     ]
    }
   ],
   "source": [
    "# test the encoder\n",
    "sample_sentence1 = \"Next I'm buying Coca-Cola to put the cocaine back in\"\n",
    "sample_sentence2 = \"Hey guys, wanna feel old? I'm 40. You're welcome.\"\n",
    "sample_sentence3 = \"Beef chicken pork bacon chuck shortloin sirloin shank eu, bresaola voluptate in enim ea kielbasa laboris brisket laborum, jowl labore id porkchop elit ad commodo.\"\n",
    "\n",
    "embed_sample = embed([\n",
    "    sample_sentence1,\n",
    "    sample_sentence2,\n",
    "    sample_sentence3\n",
    "])\n",
    "\n",
    "print(embed_sample)\n",
    "\n",
    "# the encoder turns each input into size-512 feature vectors\n",
    "# [[ 0.06844153 -0.0325974  -0.01901028 ... -0.03307429 -0.04625704\n",
    "#  -0.08149158]\n",
    "# [ 0.02810995 -0.06714624  0.02414106 ... -0.02519046  0.03197665\n",
    "#   0.02462349]\n",
    "# [ 0.03188843 -0.0167392  -0.03194157 ... -0.04541751 -0.05822486\n",
    "#  -0.07237621]], shape=(3, 512), dtype=float32)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8cc76bc9",
   "metadata": {},
   "source": [
    "#### Model Building and Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "id": "be52accc",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model_6_use\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " sentence-encoder (KerasLaye  (None, 512)              256797824 \n",
      " r)                                                              \n",
      "                                                                 \n",
      " dense_7 (Dense)             (None, 1)                 513       \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 256,798,337\n",
      "Trainable params: 513\n",
      "Non-trainable params: 256,797,824\n",
      "_________________________________________________________________\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 12:53:32.368917: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype string\n",
      "\t [[{{node inputs}}]]\n",
      "2023-04-24 12:53:32.429803: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder' with dtype string and shape [?]\n",
      "\t [[{{node Placeholder}}]]\n"
     ]
    }
   ],
   "source": [
    "model_6 = tf.keras.models.Sequential(name='model_6_use')\n",
    "\n",
    "model_6.add(hub.KerasLayer(\n",
    "    'https://tfhub.dev/google/universal-sentence-encoder/4', \n",
    "    input_shape=[], # layer excepts string of variable length and returns a 512 feature vector\n",
    "    dtype=tf.string, \n",
    "    trainable=False,\n",
    "    name='sentence-encoder'\n",
    "))\n",
    "\n",
    "model_6.add(Dense(1, activation='sigmoid'))\n",
    "\n",
    "model_6.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37036030",
   "metadata": {},
   "source": [
    "```\n",
    "Model: \"model_6_use\"\n",
    "_________________________________________________________________\n",
    " Layer (type)                Output Shape              Param #   \n",
    "=================================================================\n",
    " sentence-encoder (KerasLaye  (None, 512)              256797824 \n",
    " r)                                                              \n",
    "                                                                 \n",
    " dense_5 (Dense)             (None, 1)                 513       \n",
    "                                                                 \n",
    "=================================================================\n",
    "Total params: 256,798,337\n",
    "Trainable params: 513\n",
    "Non-trainable params: 256,797,824\n",
    "_________________________________________________________________\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "id": "6e26e3cb",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_6.compile(\n",
    "    loss='binary_crossentropy',\n",
    "    optimizer=tf.keras.optimizers.Adam(),\n",
    "    metrics=['accuracy']\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "id": "271a7010",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO :: Saving TensorBoard Log to: tensorboad/model_6_use/20230424-125338\n",
      "Epoch 1/5\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 12:53:39.474331: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall}}]]\n",
      "2023-04-24 12:53:39.474424: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_1' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_1}}]]\n",
      "2023-04-24 12:53:39.474525: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_2' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_2}}]]\n",
      "2023-04-24 12:53:39.474606: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_4' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_4}}]]\n",
      "2023-04-24 12:53:39.474681: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_5' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_5}}]]\n",
      "2023-04-24 12:53:39.474726: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_6' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_6}}]]\n",
      "2023-04-24 12:53:39.474826: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_7' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_7}}]]\n",
      "2023-04-24 12:53:39.474912: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_9' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_9}}]]\n",
      "2023-04-24 12:53:39.474978: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_10' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_10}}]]\n",
      "2023-04-24 12:53:39.475025: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_11' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_11}}]]\n",
      "2023-04-24 12:53:39.475073: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_17' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_17}}]]\n",
      "2023-04-24 12:53:39.475121: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_18' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_18}}]]\n",
      "2023-04-24 12:53:39.475168: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_19' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_19}}]]\n",
      "2023-04-24 12:53:39.475214: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_20' with dtype float\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_20}}]]\n",
      "2023-04-24 12:53:39.475261: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_21' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_21}}]]\n",
      "2023-04-24 12:53:39.475307: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_25' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_25}}]]\n",
      "2023-04-24 12:53:39.475354: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_26' with dtype float\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_26}}]]\n",
      "2023-04-24 12:53:39.475401: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_27' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_27}}]]\n",
      "2023-04-24 12:53:39.475447: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_29' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_29}}]]\n",
      "2023-04-24 12:53:39.475494: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_30' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_30}}]]\n",
      "2023-04-24 12:53:39.475540: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_31' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_31}}]]\n",
      "2023-04-24 12:53:39.475598: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_32' with dtype int32 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_32}}]]\n",
      "2023-04-24 12:53:39.475683: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_33' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_33}}]]\n",
      "2023-04-24 12:53:39.475773: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_64' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_64}}]]\n",
      "2023-04-24 12:53:39.475862: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_65' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_65}}]]\n",
      "2023-04-24 12:53:39.475977: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_66' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_66}}]]\n",
      "2023-04-24 12:53:39.476091: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_67' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_67}}]]\n",
      "2023-04-24 12:53:39.476206: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_68' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_68}}]]\n",
      "2023-04-24 12:53:39.476320: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_69' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_69}}]]\n",
      "2023-04-24 12:53:39.476416: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_70' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_70}}]]\n",
      "2023-04-24 12:53:39.476503: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_71' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_71}}]]\n",
      "2023-04-24 12:53:39.476589: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_72' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_72}}]]\n",
      "2023-04-24 12:53:39.476673: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_73' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_73}}]]\n",
      "2023-04-24 12:53:39.476759: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_74' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_74}}]]\n",
      "2023-04-24 12:53:39.476844: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_75' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_75}}]]\n",
      "2023-04-24 12:53:39.476929: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_76' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_76}}]]\n",
      "2023-04-24 12:53:39.477014: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_77' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_77}}]]\n",
      "2023-04-24 12:53:39.477099: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_78' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_78}}]]\n",
      "2023-04-24 12:53:39.477184: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_79' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_79}}]]\n",
      "2023-04-24 12:53:39.477269: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_80' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_80}}]]\n",
      "2023-04-24 12:53:39.477354: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_81' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_81}}]]\n",
      "2023-04-24 12:53:39.477439: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_82' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_82}}]]\n",
      "2023-04-24 12:53:39.477524: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_83' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_83}}]]\n",
      "2023-04-24 12:53:39.477600: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_84' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_84}}]]\n",
      "2023-04-24 12:53:39.477675: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_85' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_85}}]]\n",
      "2023-04-24 12:53:39.477766: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_86' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_86}}]]\n",
      "2023-04-24 12:53:39.477827: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_87' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_87}}]]\n",
      "2023-04-24 12:53:39.477876: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_88' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_88}}]]\n",
      "2023-04-24 12:53:39.477923: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_89' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_89}}]]\n",
      "2023-04-24 12:53:39.477970: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_90' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_90}}]]\n",
      "2023-04-24 12:53:39.478017: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_91' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_91}}]]\n",
      "2023-04-24 12:53:39.478063: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_92' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_92}}]]\n",
      "2023-04-24 12:53:39.478110: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_93' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_93}}]]\n",
      "2023-04-24 12:53:39.645421: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall}}]]\n",
      "2023-04-24 12:53:39.645499: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_1' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_1}}]]\n",
      "2023-04-24 12:53:39.645562: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_2' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_2}}]]\n",
      "2023-04-24 12:53:39.645615: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_4' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_4}}]]\n",
      "2023-04-24 12:53:39.645672: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_5' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_5}}]]\n",
      "2023-04-24 12:53:39.645720: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_6' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_6}}]]\n",
      "2023-04-24 12:53:39.645803: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_7' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_7}}]]\n",
      "2023-04-24 12:53:39.645882: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_9' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_9}}]]\n",
      "2023-04-24 12:53:39.645932: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_10' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_10}}]]\n",
      "2023-04-24 12:53:39.646022: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_11' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_11}}]]\n",
      "2023-04-24 12:53:39.646110: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_17' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_17}}]]\n",
      "2023-04-24 12:53:39.646199: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_18' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_18}}]]\n",
      "2023-04-24 12:53:39.646287: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_19' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_19}}]]\n",
      "2023-04-24 12:53:39.646375: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_20' with dtype float\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_20}}]]\n",
      "2023-04-24 12:53:39.646463: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_21' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_21}}]]\n",
      "2023-04-24 12:53:39.646550: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_25' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_25}}]]\n",
      "2023-04-24 12:53:39.646638: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_26' with dtype float\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_26}}]]\n",
      "2023-04-24 12:53:39.646733: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_27' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_27}}]]\n",
      "2023-04-24 12:53:39.646868: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_29' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_29}}]]\n",
      "2023-04-24 12:53:39.646965: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_30' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_30}}]]\n",
      "2023-04-24 12:53:39.647059: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_31' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_31}}]]\n",
      "2023-04-24 12:53:39.647151: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_32' with dtype int32 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_32}}]]\n",
      "2023-04-24 12:53:39.647269: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_33' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_33}}]]\n",
      "2023-04-24 12:53:39.647386: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_64' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_64}}]]\n",
      "2023-04-24 12:53:39.647507: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_65' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_65}}]]\n",
      "2023-04-24 12:53:39.647598: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_66' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_66}}]]\n",
      "2023-04-24 12:53:39.647689: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_67' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_67}}]]\n",
      "2023-04-24 12:53:39.647779: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_68' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_68}}]]\n",
      "2023-04-24 12:53:39.647869: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_69' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_69}}]]\n",
      "2023-04-24 12:53:39.647960: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_70' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_70}}]]\n",
      "2023-04-24 12:53:39.648050: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_71' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_71}}]]\n",
      "2023-04-24 12:53:39.648140: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_72' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_72}}]]\n",
      "2023-04-24 12:53:39.648243: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_73' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_73}}]]\n",
      "2023-04-24 12:53:39.648335: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_74' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_74}}]]\n",
      "2023-04-24 12:53:39.648439: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_75' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_75}}]]\n",
      "2023-04-24 12:53:39.648544: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_76' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_76}}]]\n",
      "2023-04-24 12:53:39.648637: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_77' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_77}}]]\n",
      "2023-04-24 12:53:39.648728: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_78' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_78}}]]\n",
      "2023-04-24 12:53:39.648819: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_79' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_79}}]]\n",
      "2023-04-24 12:53:39.648910: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_80' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_80}}]]\n",
      "2023-04-24 12:53:39.649000: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_81' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_81}}]]\n",
      "2023-04-24 12:53:39.649091: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_82' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_82}}]]\n",
      "2023-04-24 12:53:39.649183: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_83' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_83}}]]\n",
      "2023-04-24 12:53:39.649275: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_84' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_84}}]]\n",
      "2023-04-24 12:53:39.649407: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_85' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_85}}]]\n",
      "2023-04-24 12:53:39.649535: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_86' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_86}}]]\n",
      "2023-04-24 12:53:39.649642: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_87' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_87}}]]\n",
      "2023-04-24 12:53:39.649743: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_88' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_88}}]]\n",
      "2023-04-24 12:53:39.649889: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_89' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_89}}]]\n",
      "2023-04-24 12:53:39.650023: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_90' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_90}}]]\n",
      "2023-04-24 12:53:39.650112: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_91' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_91}}]]\n",
      "2023-04-24 12:53:39.650196: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_92' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_92}}]]\n",
      "2023-04-24 12:53:39.650280: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_93' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_93}}]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "215/215 [==============================] - 3s 7ms/step - loss: 0.6505 - accuracy: 0.7211 - val_loss: 0.6142 - val_accuracy: 0.7782\n",
      "Epoch 2/5\n",
      "215/215 [==============================] - 1s 5ms/step - loss: 0.5826 - accuracy: 0.7897 - val_loss: 0.5642 - val_accuracy: 0.7795\n",
      "Epoch 3/5\n",
      "215/215 [==============================] - 1s 5ms/step - loss: 0.5397 - accuracy: 0.7933 - val_loss: 0.5323 - val_accuracy: 0.7887\n",
      "Epoch 4/5\n",
      "215/215 [==============================] - 1s 5ms/step - loss: 0.5106 - accuracy: 0.7971 - val_loss: 0.5115 - val_accuracy: 0.7913\n",
      "Epoch 5/5\n",
      "215/215 [==============================] - 1s 5ms/step - loss: 0.4902 - accuracy: 0.8000 - val_loss: 0.4965 - val_accuracy: 0.7900\n"
     ]
    }
   ],
   "source": [
    "model_6_history = model_6.fit(\n",
    "    train_tweets,\n",
    "    train_labels,\n",
    "    epochs=5,\n",
    "    validation_data=(val_tweets, val_labels),\n",
    "    callbacks=[create_tensorboard_callback_bugged(\n",
    "        dir_name=LOG_DIR,\n",
    "        experiment_name='model_6_use'\n",
    "    )]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c8d5d34",
   "metadata": {},
   "source": [
    "#### Model Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "id": "09dd1006",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24/24 [==============================] - 0s 4ms/step - loss: 0.4965 - accuracy: 0.7900\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.49651169776916504, 0.7900262475013733]"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_6.evaluate(val_tweets, val_labels)\n",
    "# loss: 0.4982 - accuracy: 0.7887"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "51c8c93e",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24/24 [==============================] - 0s 5ms/step\n",
      "Prediction: [0.36729172] (not a disaster)\n"
     ]
    }
   ],
   "source": [
    "# make predictions\n",
    "model_6_preds = model_6.predict(val_tweets)\n",
    "sample_prediction=model_6_preds[0]\n",
    "print(f\"Prediction: {sample_prediction}\",\"(disaster)\" if sample_prediction>0.5 else \"(not a disaster)\")\n",
    "# Prediction: [0.36132362] (not a disaster)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "5ef4ea2f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# convert model prediction probabilities to binary label format\n",
    "model_6_preds = tf.squeeze(tf.round(model_6_preds))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "id": "1cd3128b",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'accuracy': 79.00262467191601, 'precision': 0.7907348439526268, 'recall': 0.7900262467191601, 'f1': 0.7887751851796453}\n"
     ]
    }
   ],
   "source": [
    "model_6_metrics = calculate_metrics(\n",
    "    y_true=val_labels,\n",
    "    y_pred=model_6_preds\n",
    ")\n",
    "\n",
    "print(model_6_metrics)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97bb6dcd",
   "metadata": {},
   "source": [
    "#### Model 6 Metrics\n",
    "\n",
    "* Accuracy: 78.87139107611549\n",
    "* Precision: 0.7891485217486439\n",
    "* Recall: 0.7887139107611548\n",
    "* F1: 0.7876016937745534"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "id": "de659f7f",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ True  True  True  True]\n",
      "[False False False  True]\n"
     ]
    }
   ],
   "source": [
    "# the model performs better than the conv1d and gettin close to the baseline model\n",
    "print(np.array(list(model_6_metrics.values())) > np.array(list(model_5_metrics.values())))\n",
    "# [ True  True  True  True]\n",
    "print(np.array(list(model_6_metrics.values())) > np.array(list(baseline_metrics.values())))\n",
    "# [False False False  True]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "83a4c724",
   "metadata": {},
   "source": [
    "### Model 6a\n",
    "\n",
    "The model get's close to the baseline model - I will try to add another Dense layer and see if this improves the performance.\n",
    "\n",
    "Otherwise the model is identical - only written for functional API (no reason, just for practise :) )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "id": "c47d4109",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model_6_use\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " input_6 (InputLayer)        [(None,)]                 0         \n",
      "                                                                 \n",
      " sentence-encoder (KerasLaye  (None, 512)              256797824 \n",
      " r)                                                              \n",
      "                                                                 \n",
      " dense_8 (Dense)             (None, 64)                32832     \n",
      "                                                                 \n",
      " dense_9 (Dense)             (None, 1)                 65        \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 256,830,721\n",
      "Trainable params: 32,897\n",
      "Non-trainable params: 256,797,824\n",
      "_________________________________________________________________\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 12:54:31.752641: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype string\n",
      "\t [[{{node inputs}}]]\n",
      "2023-04-24 12:54:31.812902: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder' with dtype string and shape [?]\n",
      "\t [[{{node Placeholder}}]]\n"
     ]
    }
   ],
   "source": [
    "inputs = Input(shape=[], dtype=tf.string)\n",
    "\n",
    "x = hub.KerasLayer(\n",
    "    'https://tfhub.dev/google/universal-sentence-encoder/4', \n",
    "    trainable=False,\n",
    "    dtype=tf.string,\n",
    "    name='sentence-encoder'\n",
    ")(inputs)\n",
    "\n",
    "x = Dense(64, activation='relu')(x)\n",
    "output = Dense(1, activation='sigmoid')(x)\n",
    "\n",
    "model_6a = tf.keras.models.Model(inputs, output, name='model_6_use')\n",
    "model_6a.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "921ddc7a",
   "metadata": {},
   "source": [
    "```\n",
    "Model: \"model_6_use\"\n",
    "_________________________________________________________________\n",
    " Layer (type)                Output Shape              Param #   \n",
    "=================================================================\n",
    " input_6 (InputLayer)        [(None,)]                 0         \n",
    "                                                                 \n",
    " sentence-encoder (KerasLaye  (None, 512)              256797824 \n",
    " r)                                                              \n",
    "                                                                 \n",
    " dense_13 (Dense)            (None, 64)                32832     \n",
    "                                                                 \n",
    " dense_14 (Dense)            (None, 1)                 65        \n",
    "                                                                 \n",
    "=================================================================\n",
    "Total params: 256,830,721\n",
    "Trainable params: 32,897\n",
    "Non-trainable params: 256,797,824\n",
    "_________________________________________________________________\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "id": "fa2068b4",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_6a.compile(\n",
    "    loss='binary_crossentropy',\n",
    "    optimizer=tf.keras.optimizers.Adam(),\n",
    "    metrics=['accuracy']\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "id": "06fd1590",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO :: Saving TensorBoard Log to: tensorboad/model_6a_use/20230424-125442\n",
      "Epoch 1/5\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 12:54:42.791013: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall}}]]\n",
      "2023-04-24 12:54:42.791155: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_1' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_1}}]]\n",
      "2023-04-24 12:54:42.791279: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_2' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_2}}]]\n",
      "2023-04-24 12:54:42.791366: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_4' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_4}}]]\n",
      "2023-04-24 12:54:42.791471: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_5' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_5}}]]\n",
      "2023-04-24 12:54:42.791561: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_6' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_6}}]]\n",
      "2023-04-24 12:54:42.791611: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_7' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_7}}]]\n",
      "2023-04-24 12:54:42.791668: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_9' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_9}}]]\n",
      "2023-04-24 12:54:42.791723: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_10' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_10}}]]\n",
      "2023-04-24 12:54:42.791802: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_11' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_11}}]]\n",
      "2023-04-24 12:54:42.791889: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_17' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_17}}]]\n",
      "2023-04-24 12:54:42.791948: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_18' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_18}}]]\n",
      "2023-04-24 12:54:42.792028: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_19' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_19}}]]\n",
      "2023-04-24 12:54:42.792115: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_20' with dtype float\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_20}}]]\n",
      "2023-04-24 12:54:42.792178: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_21' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_21}}]]\n",
      "2023-04-24 12:54:42.792233: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_25' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_25}}]]\n",
      "2023-04-24 12:54:42.792286: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_26' with dtype float\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_26}}]]\n",
      "2023-04-24 12:54:42.792366: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_27' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_27}}]]\n",
      "2023-04-24 12:54:42.792429: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_29' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_29}}]]\n",
      "2023-04-24 12:54:42.792509: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_30' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_30}}]]\n",
      "2023-04-24 12:54:42.792597: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_31' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_31}}]]\n",
      "2023-04-24 12:54:42.792683: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_32' with dtype int32 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_32}}]]\n",
      "2023-04-24 12:54:42.792745: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_33' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_33}}]]\n",
      "2023-04-24 12:54:42.792800: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_64' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_64}}]]\n",
      "2023-04-24 12:54:42.792854: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_65' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_65}}]]\n",
      "2023-04-24 12:54:42.792934: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_66' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_66}}]]\n",
      "2023-04-24 12:54:42.792996: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_67' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_67}}]]\n",
      "2023-04-24 12:54:42.793076: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_68' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_68}}]]\n",
      "2023-04-24 12:54:42.793165: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_69' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_69}}]]\n",
      "2023-04-24 12:54:42.793251: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_70' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_70}}]]\n",
      "2023-04-24 12:54:42.793311: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_71' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_71}}]]\n",
      "2023-04-24 12:54:42.793390: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_72' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_72}}]]\n",
      "2023-04-24 12:54:42.793473: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_73' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_73}}]]\n",
      "2023-04-24 12:54:42.793556: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_74' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_74}}]]\n",
      "2023-04-24 12:54:42.793639: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_75' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_75}}]]\n",
      "2023-04-24 12:54:42.793722: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_76' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_76}}]]\n",
      "2023-04-24 12:54:42.793805: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_77' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_77}}]]\n",
      "2023-04-24 12:54:42.793888: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_78' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_78}}]]\n",
      "2023-04-24 12:54:42.793972: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_79' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_79}}]]\n",
      "2023-04-24 12:54:42.794055: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_80' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_80}}]]\n",
      "2023-04-24 12:54:42.794139: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_81' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_81}}]]\n",
      "2023-04-24 12:54:42.794282: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_82' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_82}}]]\n",
      "2023-04-24 12:54:42.794389: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_83' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_83}}]]\n",
      "2023-04-24 12:54:42.794446: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_84' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_84}}]]\n",
      "2023-04-24 12:54:42.794521: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_85' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_85}}]]\n",
      "2023-04-24 12:54:42.794616: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_86' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_86}}]]\n",
      "2023-04-24 12:54:42.794723: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_87' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_87}}]]\n",
      "2023-04-24 12:54:42.794805: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_88' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_88}}]]\n",
      "2023-04-24 12:54:42.794896: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_89' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_89}}]]\n",
      "2023-04-24 12:54:42.794978: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_90' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_90}}]]\n",
      "2023-04-24 12:54:42.795060: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_91' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_91}}]]\n",
      "2023-04-24 12:54:42.795142: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_92' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_92}}]]\n",
      "2023-04-24 12:54:42.795223: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_93' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_93}}]]\n",
      "2023-04-24 12:54:42.954875: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall}}]]\n",
      "2023-04-24 12:54:42.954988: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_1' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_1}}]]\n",
      "2023-04-24 12:54:42.955111: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_2' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_2}}]]\n",
      "2023-04-24 12:54:42.955219: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_4' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_4}}]]\n",
      "2023-04-24 12:54:42.955319: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_5' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_5}}]]\n",
      "2023-04-24 12:54:42.955417: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_6' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_6}}]]\n",
      "2023-04-24 12:54:42.955514: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_7' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_7}}]]\n",
      "2023-04-24 12:54:42.955601: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_9' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_9}}]]\n",
      "2023-04-24 12:54:42.955680: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_10' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_10}}]]\n",
      "2023-04-24 12:54:42.955773: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_11' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_11}}]]\n",
      "2023-04-24 12:54:42.955879: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_17' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_17}}]]\n",
      "2023-04-24 12:54:42.955986: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_18' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_18}}]]\n",
      "2023-04-24 12:54:42.956093: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_19' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_19}}]]\n",
      "2023-04-24 12:54:42.956199: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_20' with dtype float\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_20}}]]\n",
      "2023-04-24 12:54:42.956288: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_21' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_21}}]]\n",
      "2023-04-24 12:54:42.956374: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_25' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_25}}]]\n",
      "2023-04-24 12:54:42.956459: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_26' with dtype float\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_26}}]]\n",
      "2023-04-24 12:54:42.956546: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_27' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_27}}]]\n",
      "2023-04-24 12:54:42.956632: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_29' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_29}}]]\n",
      "2023-04-24 12:54:42.956718: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_30' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_30}}]]\n",
      "2023-04-24 12:54:42.956804: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_31' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_31}}]]\n",
      "2023-04-24 12:54:42.956891: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_32' with dtype int32 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_32}}]]\n",
      "2023-04-24 12:54:42.956955: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_33' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_33}}]]\n",
      "2023-04-24 12:54:42.957011: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_64' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_64}}]]\n",
      "2023-04-24 12:54:42.957090: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_65' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_65}}]]\n",
      "2023-04-24 12:54:42.957177: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_66' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_66}}]]\n",
      "2023-04-24 12:54:42.957264: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_67' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_67}}]]\n",
      "2023-04-24 12:54:42.957323: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_68' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_68}}]]\n",
      "2023-04-24 12:54:42.957376: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_69' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_69}}]]\n",
      "2023-04-24 12:54:42.957429: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_70' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_70}}]]\n",
      "2023-04-24 12:54:42.957511: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_71' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_71}}]]\n",
      "2023-04-24 12:54:42.957573: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_72' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_72}}]]\n",
      "2023-04-24 12:54:42.957619: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_73' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_73}}]]\n",
      "2023-04-24 12:54:42.957664: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_74' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_74}}]]\n",
      "2023-04-24 12:54:42.957710: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_75' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_75}}]]\n",
      "2023-04-24 12:54:42.957756: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_76' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_76}}]]\n",
      "2023-04-24 12:54:42.957801: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_77' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_77}}]]\n",
      "2023-04-24 12:54:42.957847: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_78' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_78}}]]\n",
      "2023-04-24 12:54:42.957892: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_79' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_79}}]]\n",
      "2023-04-24 12:54:42.957937: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_80' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_80}}]]\n",
      "2023-04-24 12:54:42.957983: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_81' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_81}}]]\n",
      "2023-04-24 12:54:42.958028: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_82' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_82}}]]\n",
      "2023-04-24 12:54:42.958073: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_83' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_83}}]]\n",
      "2023-04-24 12:54:42.958118: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_84' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_84}}]]\n",
      "2023-04-24 12:54:42.958163: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_85' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_85}}]]\n",
      "2023-04-24 12:54:42.958219: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_86' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_86}}]]\n",
      "2023-04-24 12:54:42.958265: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_87' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_87}}]]\n",
      "2023-04-24 12:54:42.958310: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_88' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_88}}]]\n",
      "2023-04-24 12:54:42.958355: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_89' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_89}}]]\n",
      "2023-04-24 12:54:42.958400: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_90' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_90}}]]\n",
      "2023-04-24 12:54:42.958445: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_91' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_91}}]]\n",
      "2023-04-24 12:54:42.958491: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_92' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_92}}]]\n",
      "2023-04-24 12:54:42.958536: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_93' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_93}}]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "215/215 [==============================] - 3s 7ms/step - loss: 0.5023 - accuracy: 0.7901 - val_loss: 0.4501 - val_accuracy: 0.7966\n",
      "Epoch 2/5\n",
      "215/215 [==============================] - 1s 5ms/step - loss: 0.4134 - accuracy: 0.8148 - val_loss: 0.4418 - val_accuracy: 0.8071\n",
      "Epoch 3/5\n",
      "215/215 [==============================] - 1s 5ms/step - loss: 0.4005 - accuracy: 0.8235 - val_loss: 0.4354 - val_accuracy: 0.8136\n",
      "Epoch 4/5\n",
      "215/215 [==============================] - 1s 5ms/step - loss: 0.3925 - accuracy: 0.8270 - val_loss: 0.4274 - val_accuracy: 0.8084\n",
      "Epoch 5/5\n",
      "215/215 [==============================] - 1s 5ms/step - loss: 0.3853 - accuracy: 0.8288 - val_loss: 0.4236 - val_accuracy: 0.8163\n"
     ]
    }
   ],
   "source": [
    "model_6a_history = model_6a.fit(\n",
    "    train_tweets,\n",
    "    train_labels,\n",
    "    epochs=5,\n",
    "    validation_data=(val_tweets, val_labels),\n",
    "    callbacks=[create_tensorboard_callback_bugged(\n",
    "        dir_name=LOG_DIR,\n",
    "        experiment_name='model_6a_use'\n",
    "    )]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9e6bd84",
   "metadata": {},
   "source": [
    "#### Model Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "id": "dfb643f5",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24/24 [==============================] - 0s 5ms/step - loss: 0.4236 - accuracy: 0.8163\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.4235517978668213, 0.8162729740142822]"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_6a.evaluate(val_tweets, val_labels)\n",
    "# loss: 0.4305 - accuracy: 0.8123"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "id": "bea0d0a2",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24/24 [==============================] - 0s 4ms/step\n",
      "Prediction: [0.2115508] (not a disaster)\n"
     ]
    }
   ],
   "source": [
    "# make predictions\n",
    "model_6a_preds = model_6a.predict(val_tweets)\n",
    "sample_prediction=model_6a_preds[0]\n",
    "print(f\"Prediction: {sample_prediction}\",\"(disaster)\" if sample_prediction>0.5 else \"(not a disaster)\")\n",
    "# Prediction: [0.15243901] (not a disaster)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "id": "24cb03dd",
   "metadata": {},
   "outputs": [],
   "source": [
    "# convert model prediction probabilities to binary label format\n",
    "model_6a_preds = tf.squeeze(tf.round(model_6a_preds))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "id": "8077ca1b",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'accuracy': 81.62729658792651, 'precision': 0.8163098416789168, 'recall': 0.8162729658792651, 'f1': 0.8157217027284339}\n"
     ]
    }
   ],
   "source": [
    "model_6a_metrics = calculate_metrics(\n",
    "    y_true=val_labels,\n",
    "    y_pred=model_6a_preds\n",
    ")\n",
    "\n",
    "print(model_6a_metrics)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1aba0147",
   "metadata": {},
   "source": [
    "#### Model 6a Metrics\n",
    "\n",
    "* Accuracy: 81.23359580052494\n",
    "* Precision: 0.8148798668657973\n",
    "* Recall: 0.8123359580052494\n",
    "* F1: 0.810686575717776"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "id": "fb7384c3",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ True  True  True  True]\n",
      "[ True  True  True  True]\n"
     ]
    }
   ],
   "source": [
    "# the model performs better than the conv1d AND better than the baseline model~!\n",
    "print(np.array(list(model_6a_metrics.values())) > np.array(list(model_5_metrics.values())))\n",
    "# [ True  True  True  True]\n",
    "print(np.array(list(model_6a_metrics.values())) > np.array(list(baseline_metrics.values())))\n",
    "# [ True  True  True  True]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36663f4e",
   "metadata": {},
   "source": [
    "### Model 6b: Transfer Learning Feature Extractor (10% Dataset)\n",
    "\n",
    "#### Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "id": "bfe9769d",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7613 761\n"
     ]
    }
   ],
   "source": [
    "# create data subset with 10% of the training data\n",
    "train_10_percent = train_df_shuffle[['text', 'target']].sample(frac=0.1, random_state=SEED)\n",
    "print(len(train_df_shuffle), len(train_10_percent))\n",
    "# 7613 761"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "id": "f7b44b88",
   "metadata": {},
   "outputs": [],
   "source": [
    "train_tweets_10_percent = train_10_percent['text'].to_list()\n",
    "train_labels_10_percent = train_10_percent['target'].to_list()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "id": "251f1ffd",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.3274228064811984 1.1867816091954022\n"
     ]
    }
   ],
   "source": [
    "# check label distribution in randomized subset\n",
    "dist_full_dataset = train_df_shuffle['target'].value_counts()\n",
    "dist_10_percent_subset = train_10_percent['target'].value_counts()\n",
    "\n",
    "print(\n",
    "    (dist_full_dataset[0]/dist_full_dataset[1]),\n",
    "    (dist_10_percent_subset[0]/dist_10_percent_subset[1])\n",
    ")\n",
    "\n",
    "# the full dataset has 33% more \"no-desaster\" tweets\n",
    "# in the subset the overhang is lower with 19%\n",
    "# 1.3274228064811984 1.1867816091954022"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a78b31b",
   "metadata": {},
   "source": [
    "#### Model Building and Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "id": "1965d53f",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model_6b_use_10_percent\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " sentence-encoder (KerasLaye  (None, 512)              256797824 \n",
      " r)                                                              \n",
      "                                                                 \n",
      " dense_10 (Dense)            (None, 64)                32832     \n",
      "                                                                 \n",
      " dense_11 (Dense)            (None, 1)                 65        \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 256,830,721\n",
      "Trainable params: 32,897\n",
      "Non-trainable params: 256,797,824\n",
      "_________________________________________________________________\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 12:56:35.551668: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype string\n",
      "\t [[{{node inputs}}]]\n",
      "2023-04-24 12:56:35.610254: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder' with dtype string and shape [?]\n",
      "\t [[{{node Placeholder}}]]\n"
     ]
    }
   ],
   "source": [
    "model_6b = tf.keras.models.Sequential(name='model_6b_use_10_percent')\n",
    "\n",
    "model_6b.add(hub.KerasLayer(\n",
    "    'https://tfhub.dev/google/universal-sentence-encoder/4', \n",
    "    input_shape=[], # layer excepts string of variable length and returns a 512 feature vector\n",
    "    dtype=tf.string, \n",
    "    trainable=False,\n",
    "    name='sentence-encoder'\n",
    "))\n",
    "\n",
    "model_6b.add(Dense(64, activation='relu'))\n",
    "model_6b.add(Dense(1, activation='sigmoid'))\n",
    "\n",
    "model_6b.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4446f362",
   "metadata": {},
   "source": [
    "```\n",
    "Model: \"model_6b_use_10_percent\"\n",
    "_________________________________________________________________\n",
    " Layer (type)                Output Shape              Param #   \n",
    "=================================================================\n",
    " sentence-encoder (KerasLaye  (None, 512)              256797824 \n",
    " r)                                                              \n",
    "                                                                 \n",
    " dense_10 (Dense)            (None, 64)                32832     \n",
    "                                                                 \n",
    " dense_11 (Dense)            (None, 1)                 65        \n",
    "                                                                 \n",
    "=================================================================\n",
    "Total params: 256,830,721\n",
    "Trainable params: 32,897\n",
    "Non-trainable params: 256,797,824\n",
    "_________________________________________________________________\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "id": "d9ccd16e",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_6b.compile(\n",
    "    loss='binary_crossentropy',\n",
    "    optimizer=tf.keras.optimizers.Adam(),\n",
    "    metrics=['accuracy']\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "id": "f3cdd58b",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO :: Saving TensorBoard Log to: tensorboad/model_6b_use/20230424-125641\n",
      "Epoch 1/5\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 12:56:42.365682: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall}}]]\n",
      "2023-04-24 12:56:42.365863: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_1' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_1}}]]\n",
      "2023-04-24 12:56:42.365946: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_2' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_2}}]]\n",
      "2023-04-24 12:56:42.366024: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_4' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_4}}]]\n",
      "2023-04-24 12:56:42.366087: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_5' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_5}}]]\n",
      "2023-04-24 12:56:42.366185: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_6' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_6}}]]\n",
      "2023-04-24 12:56:42.366268: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_7' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_7}}]]\n",
      "2023-04-24 12:56:42.366381: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_9' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_9}}]]\n",
      "2023-04-24 12:56:42.366484: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_10' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_10}}]]\n",
      "2023-04-24 12:56:42.366600: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_11' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_11}}]]\n",
      "2023-04-24 12:56:42.366674: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_17' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_17}}]]\n",
      "2023-04-24 12:56:42.366757: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_18' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_18}}]]\n",
      "2023-04-24 12:56:42.366852: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_19' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_19}}]]\n",
      "2023-04-24 12:56:42.366926: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_20' with dtype float\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_20}}]]\n",
      "2023-04-24 12:56:42.366981: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_21' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_21}}]]\n",
      "2023-04-24 12:56:42.367029: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_25' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_25}}]]\n",
      "2023-04-24 12:56:42.367075: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_26' with dtype float\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_26}}]]\n",
      "2023-04-24 12:56:42.367121: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_27' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_27}}]]\n",
      "2023-04-24 12:56:42.367168: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_29' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_29}}]]\n",
      "2023-04-24 12:56:42.367215: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_30' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_30}}]]\n",
      "2023-04-24 12:56:42.367261: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_31' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_31}}]]\n",
      "2023-04-24 12:56:42.367308: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_32' with dtype int32 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_32}}]]\n",
      "2023-04-24 12:56:42.367353: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_33' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_33}}]]\n",
      "2023-04-24 12:56:42.367400: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_64' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_64}}]]\n",
      "2023-04-24 12:56:42.367448: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_65' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_65}}]]\n",
      "2023-04-24 12:56:42.367494: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_66' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_66}}]]\n",
      "2023-04-24 12:56:42.367540: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_67' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_67}}]]\n",
      "2023-04-24 12:56:42.367586: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_68' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_68}}]]\n",
      "2023-04-24 12:56:42.367632: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_69' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_69}}]]\n",
      "2023-04-24 12:56:42.367678: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_70' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_70}}]]\n",
      "2023-04-24 12:56:42.367724: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_71' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_71}}]]\n",
      "2023-04-24 12:56:42.367769: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_72' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_72}}]]\n",
      "2023-04-24 12:56:42.367815: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_73' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_73}}]]\n",
      "2023-04-24 12:56:42.367861: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_74' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_74}}]]\n",
      "2023-04-24 12:56:42.367907: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_75' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_75}}]]\n",
      "2023-04-24 12:56:42.367952: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_76' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_76}}]]\n",
      "2023-04-24 12:56:42.367998: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_77' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_77}}]]\n",
      "2023-04-24 12:56:42.368044: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_78' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_78}}]]\n",
      "2023-04-24 12:56:42.368090: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_79' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_79}}]]\n",
      "2023-04-24 12:56:42.368135: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_80' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_80}}]]\n",
      "2023-04-24 12:56:42.368220: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_81' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_81}}]]\n",
      "2023-04-24 12:56:42.368306: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_82' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_82}}]]\n",
      "2023-04-24 12:56:42.368403: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_83' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_83}}]]\n",
      "2023-04-24 12:56:42.368514: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_84' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_84}}]]\n",
      "2023-04-24 12:56:42.368625: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_85' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_85}}]]\n",
      "2023-04-24 12:56:42.368735: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_86' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_86}}]]\n",
      "2023-04-24 12:56:42.368845: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_87' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_87}}]]\n",
      "2023-04-24 12:56:42.368955: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_88' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_88}}]]\n",
      "2023-04-24 12:56:42.369054: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_89' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_89}}]]\n",
      "2023-04-24 12:56:42.369151: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_90' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_90}}]]\n",
      "2023-04-24 12:56:42.369261: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_91' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_91}}]]\n",
      "2023-04-24 12:56:42.369372: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_92' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_92}}]]\n",
      "2023-04-24 12:56:42.369482: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_93' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_93}}]]\n",
      "2023-04-24 12:56:42.532077: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall}}]]\n",
      "2023-04-24 12:56:42.532169: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_1' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_1}}]]\n",
      "2023-04-24 12:56:42.532263: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_2' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_2}}]]\n",
      "2023-04-24 12:56:42.532364: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_4' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_4}}]]\n",
      "2023-04-24 12:56:42.532458: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_5' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_5}}]]\n",
      "2023-04-24 12:56:42.532557: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_6' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_6}}]]\n",
      "2023-04-24 12:56:42.532657: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_7' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_7}}]]\n",
      "2023-04-24 12:56:42.532799: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_9' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_9}}]]\n",
      "2023-04-24 12:56:42.532895: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_10' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_10}}]]\n",
      "2023-04-24 12:56:42.532979: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_11' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_11}}]]\n",
      "2023-04-24 12:56:42.533056: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_17' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_17}}]]\n",
      "2023-04-24 12:56:42.533105: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_18' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_18}}]]\n",
      "2023-04-24 12:56:42.533153: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_19' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_19}}]]\n",
      "2023-04-24 12:56:42.533199: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_20' with dtype float\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_20}}]]\n",
      "2023-04-24 12:56:42.533246: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_21' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_21}}]]\n",
      "2023-04-24 12:56:42.533293: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_25' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_25}}]]\n",
      "2023-04-24 12:56:42.533340: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_26' with dtype float\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_26}}]]\n",
      "2023-04-24 12:56:42.533386: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_27' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_27}}]]\n",
      "2023-04-24 12:56:42.533433: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_29' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_29}}]]\n",
      "2023-04-24 12:56:42.533479: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_30' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_30}}]]\n",
      "2023-04-24 12:56:42.533551: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_31' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_31}}]]\n",
      "2023-04-24 12:56:42.533628: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_32' with dtype int32 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_32}}]]\n",
      "2023-04-24 12:56:42.533711: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_33' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_33}}]]\n",
      "2023-04-24 12:56:42.533795: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_64' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_64}}]]\n",
      "2023-04-24 12:56:42.533889: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_65' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_65}}]]\n",
      "2023-04-24 12:56:42.533998: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_66' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_66}}]]\n",
      "2023-04-24 12:56:42.534105: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_67' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_67}}]]\n",
      "2023-04-24 12:56:42.534212: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_68' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_68}}]]\n",
      "2023-04-24 12:56:42.534320: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_69' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_69}}]]\n",
      "2023-04-24 12:56:42.534415: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_70' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_70}}]]\n",
      "2023-04-24 12:56:42.534500: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_71' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_71}}]]\n",
      "2023-04-24 12:56:42.534596: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_72' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_72}}]]\n",
      "2023-04-24 12:56:42.534701: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_73' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_73}}]]\n",
      "2023-04-24 12:56:42.534794: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_74' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_74}}]]\n",
      "2023-04-24 12:56:42.534887: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_75' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_75}}]]\n",
      "2023-04-24 12:56:42.534971: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_76' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_76}}]]\n",
      "2023-04-24 12:56:42.535055: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_77' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_77}}]]\n",
      "2023-04-24 12:56:42.535139: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_78' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_78}}]]\n",
      "2023-04-24 12:56:42.535223: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_79' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_79}}]]\n",
      "2023-04-24 12:56:42.535307: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_80' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_80}}]]\n",
      "2023-04-24 12:56:42.535389: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_81' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_81}}]]\n",
      "2023-04-24 12:56:42.535473: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_82' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_82}}]]\n",
      "2023-04-24 12:56:42.535556: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_83' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_83}}]]\n",
      "2023-04-24 12:56:42.535638: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_84' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_84}}]]\n",
      "2023-04-24 12:56:42.535721: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_85' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_85}}]]\n",
      "2023-04-24 12:56:42.535806: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_86' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_86}}]]\n",
      "2023-04-24 12:56:42.535888: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_87' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_87}}]]\n",
      "2023-04-24 12:56:42.535971: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_88' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_88}}]]\n",
      "2023-04-24 12:56:42.536056: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_89' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_89}}]]\n",
      "2023-04-24 12:56:42.536139: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_90' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_90}}]]\n",
      "2023-04-24 12:56:42.536224: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_91' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_91}}]]\n",
      "2023-04-24 12:56:42.536308: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_92' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_92}}]]\n",
      "2023-04-24 12:56:42.536391: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_93' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_93}}]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24/24 [==============================] - 2s 24ms/step - loss: 0.6725 - accuracy: 0.6767 - val_loss: 0.6367 - val_accuracy: 0.7913\n",
      "Epoch 2/5\n",
      "24/24 [==============================] - 0s 11ms/step - loss: 0.6059 - accuracy: 0.7858 - val_loss: 0.5616 - val_accuracy: 0.7953\n",
      "Epoch 3/5\n",
      "24/24 [==============================] - 0s 11ms/step - loss: 0.5323 - accuracy: 0.7976 - val_loss: 0.4931 - val_accuracy: 0.8045\n",
      "Epoch 4/5\n",
      "24/24 [==============================] - 0s 11ms/step - loss: 0.4737 - accuracy: 0.8121 - val_loss: 0.4471 - val_accuracy: 0.8163\n",
      "Epoch 5/5\n",
      "24/24 [==============================] - 0s 10ms/step - loss: 0.4355 - accuracy: 0.8121 - val_loss: 0.4146 - val_accuracy: 0.8281\n"
     ]
    }
   ],
   "source": [
    "model_6b_history = model_6b.fit(\n",
    "    train_tweets_10_percent,\n",
    "    train_labels_10_percent,\n",
    "    epochs=5,\n",
    "    validation_data=(val_tweets, val_labels),\n",
    "    callbacks=[create_tensorboard_callback_bugged(\n",
    "        dir_name=LOG_DIR,\n",
    "        experiment_name='model_6b_use'\n",
    "    )]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af76b12c",
   "metadata": {},
   "source": [
    "#### Model Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "id": "04688cad",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24/24 [==============================] - 0s 5ms/step - loss: 0.4146 - accuracy: 0.8281\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.41460439562797546, 0.8280839920043945]"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_6b.evaluate(val_tweets, val_labels)\n",
    "# loss: 0.3375 - accuracy: 0.8675"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "id": "f720c4ef",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24/24 [==============================] - 0s 4ms/step\n",
      "Prediction: [0.27583903] (not a disaster)\n"
     ]
    }
   ],
   "source": [
    "# make predictions\n",
    "model_6b_preds = model_6b.predict(val_tweets)\n",
    "sample_prediction=model_6b_preds[0]\n",
    "print(f\"Prediction: {sample_prediction}\",\"(disaster)\" if sample_prediction>0.5 else \"(not a disaster)\")\n",
    "# Prediction: [0.16887127] (not a disaster)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "id": "e5f0b5dc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# convert model prediction probabilities to binary label format\n",
    "model_6b_preds = tf.squeeze(tf.round(model_6b_preds))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "id": "97a60d8b",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'accuracy': 82.80839895013123, 'precision': 0.8279254463064694, 'recall': 0.8280839895013123, 'f1': 0.8278306241621576}\n"
     ]
    }
   ],
   "source": [
    "model_6b_metrics = calculate_metrics(\n",
    "    y_true=val_labels,\n",
    "    y_pred=model_6b_preds\n",
    ")\n",
    "\n",
    "print(model_6b_metrics)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "335b3218",
   "metadata": {},
   "source": [
    "#### Model 6b Metrics\n",
    "\n",
    "* Accuracy: 86.74540682414698\n",
    "* Precision: 0.8695676801900293\n",
    "* Recall: 0.8674540682414699\n",
    "* F1: 0.8666326892977956"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "id": "f3d7571b",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ True  True  True  True]\n",
      "[ True  True  True  True]\n"
     ]
    }
   ],
   "source": [
    "# the model performs even better with only 10% AND better than the baseline model ?\n",
    "print(np.array(list(model_6b_metrics.values())) > np.array(list(model_6_metrics.values())))\n",
    "# [ True  True  True  True]\n",
    "print(np.array(list(model_6b_metrics.values())) > np.array(list(baseline_metrics.values())))\n",
    "# [ True  True  True  True]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c12d827",
   "metadata": {},
   "source": [
    "### Model 6c: USE Data Leakage Issue (10% Dataset)\n",
    "#### Data Leakage Issue\n",
    "\n",
    "Both the random 10% subset and the validation data was taken from the shuffled data `train_df_shuffle`. This means that the new training dataset might contain a small amount of data entries that are also part of the validation set. Since we are now only working with 10% of the original dataset this data leakage can affect our training massively.\n",
    "\n",
    "The model needs to be retrained with a clean training dataset..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5696aee5",
   "metadata": {},
   "source": [
    "##### Training 10% Split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "id": "c6d9773d",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "6851 685\n"
     ]
    }
   ],
   "source": [
    "# create data subset with 10% of the training data\n",
    "train_10_percent_split = int(0.1 * len(train_tweets))\n",
    "train_tweets_10_percent_clean = train_tweets[:train_10_percent_split]\n",
    "train_labels_10_percent_clean = train_labels[:train_10_percent_split]\n",
    "print(len(train_tweets), len(train_tweets_10_percent_clean))\n",
    "# 6851 685"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "id": "ee9d94c0",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0    3928\n",
      "1    2923\n",
      "dtype: int64 0    406\n",
      "1    279\n",
      "dtype: int64\n",
      "1.3438248374957236 1.4551971326164874\n"
     ]
    }
   ],
   "source": [
    "# check label distribution in randomized subset\n",
    "dist_train_dataset = pd.Series(train_labels).value_counts()\n",
    "dist_10_percent_train_subset = pd.Series(train_labels_10_percent_clean).value_counts()\n",
    "\n",
    "\n",
    "print(dist_train_dataset, dist_10_percent_train_subset)\n",
    "# 0    3928\n",
    "# 1    2923\n",
    "# ---------\n",
    "# 406\n",
    "# 279\n",
    "\n",
    "print(\n",
    "    (dist_train_dataset[0]/dist_train_dataset[1]),\n",
    "    (dist_10_percent_train_subset[0]/dist_10_percent_train_subset[1])\n",
    ")\n",
    "\n",
    "# the full dataset has 34% more \"no-desaster\" tweets\n",
    "# in the subset the overhang is lower with 46% - not ideal\n",
    "# 1.3438248374957236 1.4551971326164874"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "id": "53b9b0b0",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 12:58:56.355764: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype string\n",
      "\t [[{{node inputs}}]]\n",
      "2023-04-24 12:58:56.441005: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder' with dtype string and shape [?]\n",
      "\t [[{{node Placeholder}}]]\n"
     ]
    }
   ],
   "source": [
    "# use the same model as before but with cleaned weights\n",
    "model_6c = tf.keras.models.clone_model(model_6b)\n",
    "\n",
    "model_6c.compile(\n",
    "    loss='binary_crossentropy',\n",
    "    optimizer=tf.keras.optimizers.Adam(),\n",
    "    metrics=['accuracy']\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "id": "47e574fc",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO :: Saving TensorBoard Log to: tensorboad/model_6c_use/20230424-125900\n",
      "Epoch 1/5\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 12:59:01.654044: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall}}]]\n",
      "2023-04-24 12:59:01.654120: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_1' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_1}}]]\n",
      "2023-04-24 12:59:01.654171: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_2' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_2}}]]\n",
      "2023-04-24 12:59:01.654257: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_4' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_4}}]]\n",
      "2023-04-24 12:59:01.654318: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_5' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_5}}]]\n",
      "2023-04-24 12:59:01.654376: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_6' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_6}}]]\n",
      "2023-04-24 12:59:01.654421: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_7' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_7}}]]\n",
      "2023-04-24 12:59:01.654478: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_9' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_9}}]]\n",
      "2023-04-24 12:59:01.654530: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_10' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_10}}]]\n",
      "2023-04-24 12:59:01.654584: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_11' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_11}}]]\n",
      "2023-04-24 12:59:01.654639: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_17' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_17}}]]\n",
      "2023-04-24 12:59:01.654727: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_18' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_18}}]]\n",
      "2023-04-24 12:59:01.654838: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_19' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_19}}]]\n",
      "2023-04-24 12:59:01.654939: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_20' with dtype float\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_20}}]]\n",
      "2023-04-24 12:59:01.655027: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_21' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_21}}]]\n",
      "2023-04-24 12:59:01.655114: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_25' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_25}}]]\n",
      "2023-04-24 12:59:01.655200: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_26' with dtype float\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_26}}]]\n",
      "2023-04-24 12:59:01.655287: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_27' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_27}}]]\n",
      "2023-04-24 12:59:01.655362: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_29' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_29}}]]\n",
      "2023-04-24 12:59:01.655411: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_30' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_30}}]]\n",
      "2023-04-24 12:59:01.655472: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_31' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_31}}]]\n",
      "2023-04-24 12:59:01.655519: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_32' with dtype int32 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_32}}]]\n",
      "2023-04-24 12:59:01.655609: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_33' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_33}}]]\n",
      "2023-04-24 12:59:01.655696: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_64' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_64}}]]\n",
      "2023-04-24 12:59:01.655783: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_65' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_65}}]]\n",
      "2023-04-24 12:59:01.655870: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_66' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_66}}]]\n",
      "2023-04-24 12:59:01.655957: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_67' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_67}}]]\n",
      "2023-04-24 12:59:01.656076: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_68' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_68}}]]\n",
      "2023-04-24 12:59:01.656134: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_69' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_69}}]]\n",
      "2023-04-24 12:59:01.656182: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_70' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_70}}]]\n",
      "2023-04-24 12:59:01.656228: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_71' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_71}}]]\n",
      "2023-04-24 12:59:01.656273: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_72' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_72}}]]\n",
      "2023-04-24 12:59:01.656318: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_73' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_73}}]]\n",
      "2023-04-24 12:59:01.656363: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_74' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_74}}]]\n",
      "2023-04-24 12:59:01.656408: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_75' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_75}}]]\n",
      "2023-04-24 12:59:01.656454: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_76' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_76}}]]\n",
      "2023-04-24 12:59:01.656514: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_77' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_77}}]]\n",
      "2023-04-24 12:59:01.656559: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_78' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_78}}]]\n",
      "2023-04-24 12:59:01.656605: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_79' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_79}}]]\n",
      "2023-04-24 12:59:01.656663: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_80' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_80}}]]\n",
      "2023-04-24 12:59:01.656708: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_81' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_81}}]]\n",
      "2023-04-24 12:59:01.656752: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_82' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_82}}]]\n",
      "2023-04-24 12:59:01.656797: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_83' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_83}}]]\n",
      "2023-04-24 12:59:01.656841: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_84' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_84}}]]\n",
      "2023-04-24 12:59:01.656886: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_85' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_85}}]]\n",
      "2023-04-24 12:59:01.656931: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_86' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_86}}]]\n",
      "2023-04-24 12:59:01.656976: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_87' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_87}}]]\n",
      "2023-04-24 12:59:01.657020: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_88' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_88}}]]\n",
      "2023-04-24 12:59:01.657065: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_89' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_89}}]]\n",
      "2023-04-24 12:59:01.657155: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_90' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_90}}]]\n",
      "2023-04-24 12:59:01.657202: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_91' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_91}}]]\n",
      "2023-04-24 12:59:01.657247: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_92' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_92}}]]\n",
      "2023-04-24 12:59:01.657292: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_93' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_93}}]]\n",
      "2023-04-24 12:59:01.811197: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall}}]]\n",
      "2023-04-24 12:59:01.811311: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_1' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_1}}]]\n",
      "2023-04-24 12:59:01.811362: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_2' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_2}}]]\n",
      "2023-04-24 12:59:01.811426: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_4' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_4}}]]\n",
      "2023-04-24 12:59:01.811474: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_5' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_5}}]]\n",
      "2023-04-24 12:59:01.811612: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_6' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_6}}]]\n",
      "2023-04-24 12:59:01.811712: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_7' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_7}}]]\n",
      "2023-04-24 12:59:01.811804: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_9' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_9}}]]\n",
      "2023-04-24 12:59:01.811879: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_10' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_10}}]]\n",
      "2023-04-24 12:59:01.811926: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_11' with dtype float and shape [?,512]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_11}}]]\n",
      "2023-04-24 12:59:01.811973: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_17' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_17}}]]\n",
      "2023-04-24 12:59:01.812021: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_18' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_18}}]]\n",
      "2023-04-24 12:59:01.812068: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_19' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_19}}]]\n",
      "2023-04-24 12:59:01.812114: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_20' with dtype float\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_20}}]]\n",
      "2023-04-24 12:59:01.812159: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_21' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_21}}]]\n",
      "2023-04-24 12:59:01.812205: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_25' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_25}}]]\n",
      "2023-04-24 12:59:01.812251: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_26' with dtype float\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_26}}]]\n",
      "2023-04-24 12:59:01.812296: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_27' with dtype float and shape [?,1]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_27}}]]\n",
      "2023-04-24 12:59:01.812343: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_29' with dtype int32\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_29}}]]\n",
      "2023-04-24 12:59:01.812389: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_30' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_30}}]]\n",
      "2023-04-24 12:59:01.812434: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_31' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_31}}]]\n",
      "2023-04-24 12:59:01.812481: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_32' with dtype int32 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_32}}]]\n",
      "2023-04-24 12:59:01.812527: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_33' with dtype float and shape [?,320]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_33}}]]\n",
      "2023-04-24 12:59:01.812626: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_64' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_64}}]]\n",
      "2023-04-24 12:59:01.812714: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_65' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_65}}]]\n",
      "2023-04-24 12:59:01.812794: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_66' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_66}}]]\n",
      "2023-04-24 12:59:01.812847: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_67' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_67}}]]\n",
      "2023-04-24 12:59:01.812894: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_68' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_68}}]]\n",
      "2023-04-24 12:59:01.812941: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_69' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_69}}]]\n",
      "2023-04-24 12:59:01.813003: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_70' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_70}}]]\n",
      "2023-04-24 12:59:01.813064: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_71' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_71}}]]\n",
      "2023-04-24 12:59:01.813112: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_72' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_72}}]]\n",
      "2023-04-24 12:59:01.813159: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_73' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_73}}]]\n",
      "2023-04-24 12:59:01.813206: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_74' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_74}}]]\n",
      "2023-04-24 12:59:01.813251: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_75' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_75}}]]\n",
      "2023-04-24 12:59:01.813297: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_76' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_76}}]]\n",
      "2023-04-24 12:59:01.813343: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_77' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_77}}]]\n",
      "2023-04-24 12:59:01.813388: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_78' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_78}}]]\n",
      "2023-04-24 12:59:01.813434: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_79' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_79}}]]\n",
      "2023-04-24 12:59:01.813480: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_80' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_80}}]]\n",
      "2023-04-24 12:59:01.813525: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_81' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_81}}]]\n",
      "2023-04-24 12:59:01.813570: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_82' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_82}}]]\n",
      "2023-04-24 12:59:01.813615: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_83' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_83}}]]\n",
      "2023-04-24 12:59:01.813661: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_84' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_84}}]]\n",
      "2023-04-24 12:59:01.813706: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_85' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_85}}]]\n",
      "2023-04-24 12:59:01.813751: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_86' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_86}}]]\n",
      "2023-04-24 12:59:01.813797: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_87' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_87}}]]\n",
      "2023-04-24 12:59:01.813853: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_88' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_88}}]]\n",
      "2023-04-24 12:59:01.813902: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_89' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_89}}]]\n",
      "2023-04-24 12:59:01.813948: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_90' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_90}}]]\n",
      "2023-04-24 12:59:01.813993: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_91' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_91}}]]\n",
      "2023-04-24 12:59:01.814039: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_92' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_92}}]]\n",
      "2023-04-24 12:59:01.814085: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_93' with dtype int64 and shape [?]\n",
      "\t [[{{node gradients/StatefulPartitionedCall_grad/StatefulPartitionedCall_93}}]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "22/22 [==============================] - 2s 25ms/step - loss: 0.6740 - accuracy: 0.6730 - val_loss: 0.6514 - val_accuracy: 0.7664\n",
      "Epoch 2/5\n",
      "22/22 [==============================] - 0s 11ms/step - loss: 0.6032 - accuracy: 0.8204 - val_loss: 0.5941 - val_accuracy: 0.7756\n",
      "Epoch 3/5\n",
      "22/22 [==============================] - 0s 11ms/step - loss: 0.5262 - accuracy: 0.8248 - val_loss: 0.5376 - val_accuracy: 0.7795\n",
      "Epoch 4/5\n",
      "22/22 [==============================] - 0s 11ms/step - loss: 0.4640 - accuracy: 0.8263 - val_loss: 0.5018 - val_accuracy: 0.7848\n",
      "Epoch 5/5\n",
      "22/22 [==============================] - 0s 11ms/step - loss: 0.4233 - accuracy: 0.8365 - val_loss: 0.4877 - val_accuracy: 0.7913\n"
     ]
    }
   ],
   "source": [
    "model_6c_history = model_6c.fit(\n",
    "    train_tweets_10_percent_clean,\n",
    "    train_labels_10_percent_clean,\n",
    "    epochs=5,\n",
    "    validation_data=(val_tweets, val_labels),\n",
    "    callbacks=[create_tensorboard_callback_bugged(\n",
    "        dir_name=LOG_DIR,\n",
    "        experiment_name='model_6c_use'\n",
    "    )]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0007f969",
   "metadata": {},
   "source": [
    "#### Model Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "id": "4a2b5dd5",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24/24 [==============================] - 0s 4ms/step - loss: 0.4146 - accuracy: 0.8281\n",
      "24/24 [==============================] - 0s 4ms/step - loss: 0.4877 - accuracy: 0.7913\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.48767024278640747, 0.7913385629653931]"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_6b.evaluate(val_tweets, val_labels)\n",
    "# evaluation with data leakage - performance suspiciously high\n",
    "# loss: 0.3375 - accuracy: 0.8675\n",
    "model_6c.evaluate(val_tweets, val_labels)\n",
    "# the model trained on the cleaned-up 10% dataset - as expected -\n",
    "# performs worse than the model trained on the full dataset\n",
    "# loss: 0.4904 - accuracy: 0.7822"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "id": "2fc4b1a8",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24/24 [==============================] - 0s 4ms/step\n",
      "Prediction: [0.22952469] (not a disaster)\n"
     ]
    }
   ],
   "source": [
    "# make predictions\n",
    "model_6c_preds = model_6c.predict(val_tweets)\n",
    "sample_prediction=model_6c_preds[0]\n",
    "print(f\"Prediction: {sample_prediction}\",\"(disaster)\" if sample_prediction>0.5 else \"(not a disaster)\")\n",
    "# Prediction: [0.22155239] (not a disaster)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "id": "cfbfbdbe",
   "metadata": {},
   "outputs": [],
   "source": [
    "# convert model prediction probabilities to binary label format\n",
    "model_6c_preds = tf.squeeze(tf.round(model_6c_preds))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "id": "5c8e9f6d",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'accuracy': 79.13385826771653, 'precision': 0.791659328034437, 'recall': 0.7913385826771654, 'f1': 0.790332908959664}\n"
     ]
    }
   ],
   "source": [
    "model_6c_metrics = calculate_metrics(\n",
    "    y_true=val_labels,\n",
    "    y_pred=model_6c_preds\n",
    ")\n",
    "\n",
    "print(model_6c_metrics)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48207996",
   "metadata": {},
   "source": [
    "#### Model 6c Metrics\n",
    "\n",
    "* Accuracy: 78.21522309711287\n",
    "* Precision: 0.7827333187650908\n",
    "* Recall: 0.7821522309711286\n",
    "* F1: 0.7808542546238821"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "id": "cd6770de",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[False False False False]\n",
      "[False False False  True]\n"
     ]
    }
   ],
   "source": [
    "# the model performs worse than the baseline but gets close\n",
    "# considering that we only used 10% of the data\n",
    "print(np.array(list(model_6c_metrics.values())) > np.array(list(model_6b_metrics.values())))\n",
    "# [False False False False]\n",
    "print(np.array(list(model_6c_metrics.values())) > np.array(list(baseline_metrics.values())))\n",
    "# [False False False  True]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c13fd47f",
   "metadata": {},
   "source": [
    "## Compare Experiments"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "id": "4cb538a3",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>accuracy</th>\n",
       "      <th>precision</th>\n",
       "      <th>recall</th>\n",
       "      <th>f1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>6b_use_10_percent</th>\n",
       "      <td>0.828084</td>\n",
       "      <td>0.827925</td>\n",
       "      <td>0.828084</td>\n",
       "      <td>0.827831</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6a_use_added_dense</th>\n",
       "      <td>0.816273</td>\n",
       "      <td>0.816310</td>\n",
       "      <td>0.816273</td>\n",
       "      <td>0.815722</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6c_use_10_percent_cleaned</th>\n",
       "      <td>0.791339</td>\n",
       "      <td>0.791659</td>\n",
       "      <td>0.791339</td>\n",
       "      <td>0.790333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6_use</th>\n",
       "      <td>0.790026</td>\n",
       "      <td>0.790735</td>\n",
       "      <td>0.790026</td>\n",
       "      <td>0.788775</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0_baseline</th>\n",
       "      <td>0.792651</td>\n",
       "      <td>0.811139</td>\n",
       "      <td>0.792651</td>\n",
       "      <td>0.786219</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1_simple_dense</th>\n",
       "      <td>0.786089</td>\n",
       "      <td>0.791170</td>\n",
       "      <td>0.786089</td>\n",
       "      <td>0.783007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2_lstm</th>\n",
       "      <td>0.772966</td>\n",
       "      <td>0.773845</td>\n",
       "      <td>0.772966</td>\n",
       "      <td>0.771334</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4_bidirectional</th>\n",
       "      <td>0.771654</td>\n",
       "      <td>0.773560</td>\n",
       "      <td>0.771654</td>\n",
       "      <td>0.769452</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3_gru</th>\n",
       "      <td>0.771654</td>\n",
       "      <td>0.780815</td>\n",
       "      <td>0.771654</td>\n",
       "      <td>0.766694</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5_conv1d</th>\n",
       "      <td>0.754593</td>\n",
       "      <td>0.755280</td>\n",
       "      <td>0.754593</td>\n",
       "      <td>0.752702</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           accuracy  precision    recall        f1\n",
       "6b_use_10_percent          0.828084   0.827925  0.828084  0.827831\n",
       "6a_use_added_dense         0.816273   0.816310  0.816273  0.815722\n",
       "6c_use_10_percent_cleaned  0.791339   0.791659  0.791339  0.790333\n",
       "6_use                      0.790026   0.790735  0.790026  0.788775\n",
       "0_baseline                 0.792651   0.811139  0.792651  0.786219\n",
       "1_simple_dense             0.786089   0.791170  0.786089  0.783007\n",
       "2_lstm                     0.772966   0.773845  0.772966  0.771334\n",
       "4_bidirectional            0.771654   0.773560  0.771654  0.769452\n",
       "3_gru                      0.771654   0.780815  0.771654  0.766694\n",
       "5_conv1d                   0.754593   0.755280  0.754593  0.752702"
      ]
     },
     "execution_count": 106,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# combine training results in dataframe\n",
    "model_metrics_all = pd.DataFrame({\n",
    "    \"0_baseline\": baseline_metrics,\n",
    "    \"1_simple_dense\": model_1_metrics,\n",
    "    \"2_lstm\": model_2_metrics,\n",
    "    \"3_gru\": model_3_metrics,\n",
    "    \"4_bidirectional\": model_4_metrics,\n",
    "    \"5_conv1d\": model_5_metrics,\n",
    "    \"6_use\": model_6_metrics,\n",
    "    \"6a_use_added_dense\": model_6a_metrics,\n",
    "    \"6b_use_10_percent\": model_6b_metrics,\n",
    "    \"6c_use_10_percent_cleaned\": model_6c_metrics\n",
    "})\n",
    "\n",
    "# transpose\n",
    "model_metrics_all_transposed = model_metrics_all.transpose()\n",
    "\n",
    "# rescale accuracy\n",
    "model_metrics_all_transposed['accuracy'] = model_metrics_all_transposed['accuracy']/100\n",
    "\n",
    "# sort by F1-score\n",
    "model_metrics_all_transposed_sorted = model_metrics_all_transposed.sort_values(by=['f1'], ascending=False)\n",
    "model_metrics_all_transposed_sorted\n",
    "# The best performing 6b_use_10_percent is faulty\n",
    "# The next best is the USE with 100% of training data 6a_use_added_dense\n",
    "# The USE with 10% training data performs closely to the baseline and simple dense model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "595cb27d",
   "metadata": {},
   "source": [
    "|    | accuracy | precision | recall | f1 |\n",
    "| -- | -- | -- | -- | -- |\n",
    "| 6b_use_10_percent | 0.867454 | 0.869568 | 0.867454 | 0.866633 |\n",
    "| 6a_use_added_dense | 0.813648 | 0.814169 | 0.813648 | 0.812791 |\n",
    "| 0_baseline | 0.792651 | 0.811139 | 0.792651 | 0.786219 |\n",
    "| 1_simple_dense | 0.788714 | 0.796962 | 0.788714 | 0.784729 |\n",
    "| 6_use | 0.784777 | 0.785083 | 0.784777 | 0.783692 |\n",
    "| 6c_use_10_percent_cleaned | 0.782152 | 0.782733 | 0.782152 | 0.780854 |\n",
    "| 2_lstm | 0.775591 | 0.776962 | 0.775591 | 0.773741 |\n",
    "| 3_gru | 0.770341 | 0.773598 | 0.770341 | 0.767494 |\n",
    "| 4_bidirectional | 0.765092 | 0.764730 | 0.765092 | 0.764430 |\n",
    "| 5_conv1d | 0.759843 | 0.759995 | 0.759843 | 0.758468 |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "id": "e5deeae3",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f7ee47996c0>"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABE0AAALzCAYAAADkj/keAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAACvk0lEQVR4nOzdeZyNdf/H8feZYWbsQsbSMPZ9ZzSWuBGphFJS2UoUUxgiZSmypCxlSUR0t9B9Ky2kZaK7LNkSIrssmbFklxnmfH5/+M3VHHMph1ksr+fjcR7Mdb7Xdb7nOudc5zrv67t4zMwEAAAAAAAAHwEZXQEAAAAAAICrEaEJAAAAAACAC0ITAAAAAAAAF4QmAAAAAAAALghNAAAAAAAAXBCaAAAAAAAAuCA0AQAAAAAAcEFoAgAAAAAA4CJTRlfgUni9Xv3+++/KkSOHPB5PRlcHAAAAAHCdMzOdOHFChQoVUkAA7Q1uVNdEaPL7778rLCwso6sBAAAAALjB7NmzR7fccktGVwMZ5JoITXLkyCHp/Js1Z86cGVwbAAAAAMD17vjx4woLC3N+j+LGdE2EJkldcnLmzEloAgAAAABINwwRcWOjYxYAAAAAAIALQhMAAAAAAAAXhCYAAAAAAAAurokxTQAAAAAAuNp4vV4lJCRkdDXgp8yZMyswMPCSyhKaAAAAAADgp4SEBO3cuVNerzejq4LLkDt3bhUoUOAfB/olNAEAAAAAwA9mpv379yswMFBhYWEKCGDki2uFmen06dM6cOCAJKlgwYJ/W57QBAAAAAAAP5w7d06nT59WoUKFlDVr1oyuDvyUJUsWSdKBAweUP3/+v+2qQxwGAAAAAIAfEhMTJUlBQUEZXBNcrqSw6+zZs39bjtAEAAAAAIDL8E/jYeDqdamvHaEJAAAAAACAC0ITAAAAAAAAFwwECwAAAABAKgh/dn66Pt6uUXel6+OlhbNnzypz5swZXY2LoqUJAAAAAAA3iIULF6pevXrKnTu38ubNq7vvvlvbt2937t+7d6/atWunPHnyKFu2bKpZs6Z+/PFH5/7PPvtMtWrVUkhIiPLly6fWrVs793k8Hs2bN8/n8XLnzq2ZM2dKknbt2iWPx6M5c+aoQYMGCgkJ0XvvvafDhw+rXbt2Kly4sLJmzapKlSrpgw8+8NmO1+vV6NGjVbJkSQUHB6tIkSIaPny4JKlRo0aKioryKX/w4EEFBQUpJibmivYXoQkAAAAAADeIU6dOKTo6WqtWrVJMTIwCAgLUunVreb1enTx5Ug0aNNC+ffv06aef6ueff1a/fv3k9XolSfPnz1fr1q1155136qefflJMTIwiIiL8rsOzzz6rnj17atOmTWrWrJnOnDmjGjVqaP78+dqwYYO6du2q9u3ba8WKFc46AwYM0KhRozRo0CBt3LhR77//vkJDQyVJXbp00fvvv6/4+Hin/LvvvqvChQurUaNGV7S/6J4DAAAAAMAN4r777vP5e8aMGbr55pu1ceNGLV26VAcPHtTKlSuVJ08eSVLJkiWdssOHD9eDDz6oF1980VlWpUoVv+vQq1cv3XvvvT7L+vbt6/z/qaee0pdffqkPP/xQEREROnHihF577TVNnDhRHTt2lCSVKFFC9erVkyTde++9ioqK0ieffKIHHnhAkjRz5kx16tTpimc4oqUJAAAAAAA3iK1bt6pdu3YqXry4cubMqfDwcEnS7t27tXbtWlWrVs0JTC60du1aNW7c+IrrULNmTZ+/ExMTNWzYMFWqVEl58uRR9uzZ9eWXX2r37t2SpE2bNik+Pv6ijx0SEqL27dtrxowZkqQ1a9Zow4YN6tSp0xXXlZYmAAAAAADcIFq0aKGiRYtq2rRpKlSokLxerypWrKiEhARlyZLlb9f9p/s9Ho/MzGfZ2bNnU5TLli2bz9+vvPKKXnvtNY0fP16VKlVStmzZ1KtXLyUkJFzS40rnu+hUrVpVe/fu1dtvv61GjRqpaNGi/7jeP6GlCQAAAAAAN4DDhw9r8+bNGjhwoBo3bqxy5crpyJEjzv2VK1fW2rVr9ccff7iuX7ly5b8dWPXmm2/W/v37nb+3bt2q06dP/2O9lixZopYtW+qRRx5RlSpVVLx4cW3ZssW5v1SpUsqSJcvfPnalSpVUs2ZNTZs2Te+//74effTRf3zcS0FoAgAAAADADeCmm25S3rx5NXXqVG3btk3ffvutoqOjnfvbtWunAgUKqFWrVlqyZIl27NihuXPnatmyZZKkIUOG6IMPPtCQIUO0adMmrV+/Xi+//LKzfqNGjTRx4kT99NNPWrVqlZ544olLmk64VKlS+vrrr7V06VJt2rRJ3bp1U1xcnHN/SEiI+vfvr379+umdd97R9u3btXz5ck2fPt1nO126dNGoUaNkZj6z+lwJQhMAAAAAAG4AAQEBmj17tlavXq2KFSuqd+/eeuWVV5z7g4KC9NVXXyl//vy68847ValSJY0aNUqBgYGSpIYNG+o///mPPv30U1WtWlWNGjXymeFmzJgxCgsLU/369fXQQw+pb9++ypo16z/Wa+DAgapevbqaNWumhg0bOsFNcoMGDVKfPn00ePBglStXTm3bttWBAwd8yrRr106ZMmVSu3btFBIScgV76i8eu7DD0VXo+PHjypUrl44dO6acOXNmdHUAAAAAANe5v/sdeubMGe3cuVPFihVLtR/nuHK7du1SiRIltHLlSlWvXv1vy17qa8hAsAAAAAAA4Jp19uxZHT58WAMHDtStt976j4GJP+ieAwAAAAAArllLlixRwYIFtXLlSk2ZMiVVt01LEwAAAAAAcM1q2LBhiqmOUwstTQAAAAAAAFwQmgAAAAAAALige46k8Gfn+1V+V8hDl1y2UrEifm37w5HnLrlsuV83+bVtAAAAAABw6WhpAgAAAAAA4ILQBAAAAAAAwAWhCQAAAAAAgAtCEwAAAAAAkCYWL14sj8ejo0ePpmrZ9EJoAgAAAAAA0kSdOnW0f/9+5cqVK1XLphdmzwEAAAAAIDW8kM4/9l84lqabT0hIUFBQ0BVtIygoSAUKFEj1sumFliYAAAAAANwAGjZsqKioKEVFRSlXrlzKly+fBg0aJDOTJIWHh2vYsGHq0KGDcubMqa5du0qSfvjhB9WvX19ZsmRRWFiYnn76aZ06dcrZbnx8vPr376+wsDAFBwerZMmSmj59uqSUXW5+++03tWjRQjfddJOyZcumChUqaMGCBa5lJWnu3LmqUKGCgoODFR4erjFjxvg8p/DwcI0YMUKPPvqocuTIoSJFimjq1Kmpts9oaYIMEf7sfL/K7wp56JLLVipWxK9tr++43q/yAAAAAHCtmjVrlh577DGtWLFCq1atUteuXVWkSBE9/vjjkqRXX31VgwcP1pAhQyRJ27dv1x133KGXXnpJM2bM0MGDB53g5e2335YkdejQQcuWLdPrr7+uKlWqaOfOnTp06JDr4/fo0UMJCQn63//+p2zZsmnjxo3Knj27a9nVq1frgQce0AsvvKC2bdtq6dKl6t69u/LmzatOnTo55caMGaNhw4bpueee03//+189+eSTatCggcqUKXPF+4vQBDe8TWXLXXLZcr9uSsOaAAAAAEDaCgsL07hx4+TxeFSmTBmtX79e48aNc0KTRo0aqU+fPk75Ll266OGHH1avXr0kSaVKldLrr7+uBg0a6I033tDu3bv14Ycf6uuvv1aTJk0kScWLF7/o4+/evVv33XefKlWq9I9lx44dq8aNG2vQoEGSpNKlS2vjxo165ZVXfEKTO++8U927d5ck9e/fX+PGjdOiRYtSJTShew4AAAAAADeIW2+9VR6Px/k7MjJSW7duVWJioiSpZs2aPuV//vlnzZw5U9mzZ3duzZo1k9fr1c6dO7V27VoFBgaqQYMGl/T4Tz/9tF566SXVrVtXQ4YM0bp16y5adtOmTapbt67Psrp16/rUV5IqV67s/N/j8ahAgQI6cODAJdXnnxCaAAAAAAAASVK2bNl8/j558qS6deumtWvXOreff/5ZW7duVYkSJZQlSxa/tt+lSxft2LFD7du31/r161WzZk1NmDDhiuqcOXNmn789Ho+8Xu8VbTMJ3XMAADcuf0a4T+PR6QEAANLDjz/+6PP38uXLVapUKQUGBrqWr169ujZu3KiSJUu63l+pUiV5vV599913TvecfxIWFqYnnnhCTzzxhAYMGKBp06bpqaeeSlGuXLlyWrJkic+yJUuWqHTp0hetb2qjpQkAAAAAADeI3bt3Kzo6Wps3b9YHH3ygCRMmqGfPnhct379/fy1dulRRUVFau3attm7dqk8++URRUVGSzs9e07FjRz366KOaN2+edu7cqcWLF+vDDz903V6vXr305ZdfaufOnVqzZo0WLVqkcuXcx5ns06ePYmJiNGzYMG3ZskWzZs3SxIkT1bdv3yvfEZeIliYAAAAAANwgOnTooD///FMREREKDAxUz549namF3VSuXFnfffednn/+edWvX19mphIlSqht27ZOmTfeeEPPPfecunfvrsOHD6tIkSJ67rnnXLeXmJioHj16aO/evcqZM6fuuOMOjRs3zrVs9erV9eGHH2rw4MEaNmyYChYsqKFDh/oMApvWPJY0IfNV7Pjx48qVK5eOHTumnDlzpvr2r6bpbz8cee6Sy17LM7mwzwFcFeieAwAALuLvfoeeOXNGO3fuVLFixRQSEpJBNfRfw4YNVbVqVY0fPz6jq5LhLvU1vKzuOZMmTVJ4eLhCQkJUu3ZtrVix4m/Ljx8/XmXKlFGWLFkUFham3r1768yZM5fz0AAAAAAAAOnC79Bkzpw5io6O1pAhQ7RmzRpVqVJFzZo1u+h0Pu+//76effZZDRkyRJs2bdL06dM1Z86cizbVAQAAAAAAuBr4PabJ2LFj9fjjj6tz586SpClTpmj+/PmaMWOGnn322RTlly5dqrp16+qhh853rwgPD1e7du1SjNgLAAAAAADSzuLFizO6Ctccv1qaJCQkaPXq1T7TCAUEBKhJkyZatmyZ6zp16tTR6tWrnS48O3bs0IIFC3TnnXde9HHi4+N1/PhxnxsAAAAAAEB68qulyaFDh5SYmKjQ0FCf5aGhofr1119d13nooYd06NAh1atXT2amc+fO6Yknnvjb7jkjR47Uiy++6E/VAAAAAAAAUtVlDQTrj8WLF2vEiBGaPHmy1qxZo48++kjz58/XsGHDLrrOgAEDdOzYMee2Z8+etK4mAAAAAACAD79amuTLl0+BgYGKi4vzWR4XF6cCBQq4rjNo0CC1b99eXbp0kSRVqlRJp06dUteuXfX8888rICBlbhMcHKzg4GB/qgYAAAAAAJCq/GppEhQUpBo1aigmJsZZ5vV6FRMTo8jISNd1Tp8+nSIYCQwMlCSZmb/1BQAAAAAASBd+z54THR2tjh07qmbNmoqIiND48eN16tQpZzadDh06qHDhwho5cqQkqUWLFho7dqyqVaum2rVra9u2bRo0aJBatGjhhCcAAAAAAABXG79Dk7Zt2+rgwYMaPHiwYmNjVbVqVS1cuNAZHHb37t0+LUsGDhwoj8ejgQMHat++fbr55pvVokULDR8+PPWeBQAAksKfne9X+V0hl1620qxKfm17fcf1fpUHAAC4Hr3wwguaN2+e1q5dK0nq1KmTjh49qnnz5mVovS6V36GJJEVFRSkqKsr1vgvnfc6UKZOGDBmiIUOGXM5DAUgl/v+YfOiSy1YqVsSvbfNjEriBvZDLj7LH0q4eAACkAX8vslwpzqvT3mWFJgBwJTaVLXfJZcv9uikNa5K2CKoAAABwNUtISFBQUFBGV+OqluZTDgMAMt6msuUu+QYAAIDrU8OGDRUVFaVevXopX758atasmTZs2KDmzZsre/bsCg0NVfv27XXo0CFnHa/Xq9GjR6tkyZIKDg5WkSJFfIbb6N+/v0qXLq2sWbOqePHiGjRokM6ePZsRTy9NEJoAAAAAAHCDmDVrloKCgrRkyRKNGjVKjRo1UrVq1bRq1SotXLhQcXFxeuCBB5zyAwYM0KhRozRo0CBt3LhR77//vjOmqSTlyJFDM2fO1MaNG/Xaa69p2rRpGjduXEY8tTRB9xwAANIA3dDcMfguAAAZq1SpUho9erQk6aWXXlK1atU0YsQI5/4ZM2YoLCxMW7ZsUcGCBfXaa69p4sSJ6tixoySpRIkSqlevnlN+4MCBzv/Dw8PVt29fzZ49W/369UunZ5S2CE0AAAAAALhB1KhRw/n/zz//rEWLFil79uwpym3fvl1Hjx5VfHy8GjdufNHtzZkzR6+//rq2b9+ukydP6ty5c8qZM2ea1D0jEJoAAIDrwo3SugcAgCuRLVs25/8nT55UixYt9PLLL6coV7BgQe3YseNvt7Vs2TI9/PDDevHFF9WsWTPlypVLs2fP1pgxY1K93hmF0AQAAAAAgBtQ9erVNXfuXIWHhytTppTxQKlSpZQlSxbFxMSoS5cuKe5funSpihYtqueff95Z9ttvv6VpndMboQkAAACAa9sLufwoeyzt6gFcY3r06KFp06apXbt26tevn/LkyaNt27Zp9uzZeuuttxQSEqL+/furX79+CgoKUt26dXXw4EH98ssveuyxx1SqVCnt3r1bs2fPVq1atTR//nx9/PHHGf20UhWz5wAAAAAAcAMqVKiQlixZosTERDVt2lSVKlVSr169lDt3bgUEnI8LBg0apD59+mjw4MEqV66c2rZtqwMHDkiS7rnnHvXu3VtRUVGqWrWqli5dqkGDBmXkU0p1tDQBAAAAACAVXO0zuS1evDjFslKlSumjjz666DoBAQF6/vnnfbrgJDd69GhnNp4kvXr1cv7/wgsv6IUXXnD+njlzpj9VznC0NAEAAAAAAHBBaAIAAAAAAOCC7jkAAADXkPBn5/tVflfIQ5dctlKxIn5t+2pvhp5hGJQUAK4bhCYAAAC4LJvKlrvksuV+3ZSGNUlb/gdVl1620qxKfm37Rgmq2OcArhZ0zwEAAAAAAHBBSxMAAADgGnGjtO4BgKsFLU0AAAAAAABc0NIEAAAAAC6C1j3AjY2WJgAAAAAAAC4ITQAAAAAAuAGYmbp27ao8efLI4/Fo7dq1GV2lqx7dcwAAAAAASAX+dOdKDf52CVu4cKFmzpypxYsXq3jx4tqyZYtatGih1atXa//+/fr444/VqlWrtKnsNYqWJgAAAAAA3AC2b9+uggULqk6dOipQoIBOnTqlKlWqaNKkSRldtasWLU0AAAAAALjOderUSbNmzZIkeTweFS1aVLt27VLz5s0zuGZXN0ITAAAAAACuc6+99ppKlCihqVOnauXKlQoMDMzoKl0TCE0AAAAAALjO5cqVSzly5FBgYKAKFCiQ0dW5ZhCaAAAAAMANLvzZ+X6V3xXy0CWXrVSsiF/bXt9xvV/lgbTEQLAAAAAAAAAuaGkCAAAAALhq+DNtr79T7gL+IjQBAAAAAOAGdPLkSW3bts35e+fOnVq7dq3y5MmjIkX861Z1vSI0AQAAAADgBrRq1Sr961//cv6Ojo6WJHXs2FEzZ87MoFpdXQhNAAAAAABIBVd7d6FevXqpV69ezt8NGzaUmWVcha4BDAQLAAAAAADggtAEAAAAAADABaEJAAAAAACAC0ITAAAAAAAAF4QmAAAAAAAALghNAAAAAAC4DMw8c+261NeO0AQAAAAAAD8EBgZKkhISEjK4Jrhcp0+fliRlzpz5b8tlSo/KAAAAAABwvciUKZOyZs2qgwcPKnPmzAoIoD3CtcLMdPr0aR04cEC5c+d2ArCLITQBAAAAAMAPHo9HBQsW1M6dO/Xbb79ldHVwGXLnzq0CBQr8YzlCEwAAAAAA/BQUFKRSpUrRRecalDlz5n9sYZKE0AQAAAAAgMsQEBCgkJCQjK4G0hAdrwAAAAAAAFwQmgAAAAAAALggNAEAAAAAAHBBaAIAAAAAAOCC0AQAAAAAAMAFoQkAAAAAAIALQhMAAAAAAAAXhCYAAAAAAAAuCE0AAAAAAABcEJoAAAAAAAC4IDQBAAAAAABwQWgCAAAAAADggtAEAAAAAADABaEJAAAAAACAC0ITAAAAAAAAF4QmAAAAAAAALghNAAAAAAAAXBCaAAAAAAAAuLis0GTSpEkKDw9XSEiIateurRUrVly0bMOGDeXxeFLc7rrrrsuuNAAAAAAAQFrzOzSZM2eOoqOjNWTIEK1Zs0ZVqlRRs2bNdODAAdfyH330kfbv3+/cNmzYoMDAQN1///1XXHkAAAAAAIC04ndoMnbsWD3++OPq3LmzypcvrylTpihr1qyaMWOGa/k8efKoQIECzu3rr79W1qxZCU0AAAAAAMBVza/QJCEhQatXr1aTJk3+2kBAgJo0aaJly5Zd0jamT5+uBx98UNmyZbtomfj4eB0/ftznBgAAAAAAkJ78Ck0OHTqkxMREhYaG+iwPDQ1VbGzsP66/YsUKbdiwQV26dPnbciNHjlSuXLmcW1hYmD/VBAAAAAAAuGLpOnvO9OnTValSJUVERPxtuQEDBujYsWPObc+ePelUQwAAAAAAgPMy+VM4X758CgwMVFxcnM/yuLg4FShQ4G/XPXXqlGbPnq2hQ4f+4+MEBwcrODjYn6oBAAAAAACkKr9amgQFBalGjRqKiYlxlnm9XsXExCgyMvJv1/3Pf/6j+Ph4PfLII5dXUwAAAAAAgHTkV0sTSYqOjlbHjh1Vs2ZNRUREaPz48Tp16pQ6d+4sSerQoYMKFy6skSNH+qw3ffp0tWrVSnnz5k2dmgMAAAAAAKQhv0OTtm3b6uDBgxo8eLBiY2NVtWpVLVy40Bkcdvfu3QoI8G3AsnnzZv3www/66quvUqfWAAAAAAAAaczv0ESSoqKiFBUV5Xrf4sWLUywrU6aMzOxyHgoAAAAAACBDpOvsOQAAAAAAANcKQhMAAAAAAAAXhCYAAAAAAAAuCE0AAAAAAABcEJoAAAAAAAC4IDQBAAAAAABwQWgCAAAAAADggtAEAAAAAADABaEJAAAAAACAC0ITAAAAAAAAF4QmAAAAAAAALghNAAAAAAAAXBCaAAAAAAAAuCA0AQAAAAAAcEFoAgAAAAAA4ILQBAAAAAAAwAWhCQAAAAAAgAtCEwAAAAAAABeEJgAAAAAAAC4ITQAAAAAAAFwQmgAAAAAAALggNAEAAAAAAHBBaAIAAAAAAOCC0AQAAAAAAMAFoQkAAAAAAIALQhMAAAAAAAAXhCYAAAAAAAAuCE0AAAAAAABcEJoAAAAAAAC4IDQBAAAAAABwQWgCAAAAAADggtAEAAAAAADABaEJAAAAAACAC0ITAAAAAAAAF4QmAAAAAAAALghNAAAAAAAAXBCaAAAAAAAAuCA0AQAAAAAAcEFoAgAAAAAA4ILQBAAAAAAAwAWhCQAAAAAAgAtCEwAAAAAAABeEJgAAAAAAAC4ITQAAAAAAAFwQmgAAAAAAALggNAEAAAAAAHBBaAIAAAAAAOCC0AQAAAAAAMAFoQkAAAAAAIALQhMAAAAAAAAXhCYAAAAAAAAuCE0AAAAAAABcEJoAAAAAAAC4IDQBAAAAAABwQWgCAAAAAADggtAEAAAAAADABaEJAAAAAACAC0ITAAAAAAAAF4QmAAAAAAAALi4rNJk0aZLCw8MVEhKi2rVra8WKFX9b/ujRo+rRo4cKFiyo4OBglS5dWgsWLLisCgMAAAAAAKSHTP6uMGfOHEVHR2vKlCmqXbu2xo8fr2bNmmnz5s3Knz9/ivIJCQm6/fbblT9/fv33v/9V4cKF9dtvvyl37typUX8AAAAAAIA04XdoMnbsWD3++OPq3LmzJGnKlCmaP3++ZsyYoWeffTZF+RkzZuiPP/7Q0qVLlTlzZklSeHj4ldUaAAAAAAAgjfnVPSchIUGrV69WkyZN/tpAQICaNGmiZcuWua7z6aefKjIyUj169FBoaKgqVqyoESNGKDEx8cpqDgAAAAAAkIb8amly6NAhJSYmKjQ01Gd5aGiofv31V9d1duzYoW+//VYPP/ywFixYoG3btql79+46e/ashgwZ4rpOfHy84uPjnb+PHz/uTzUBAAAAAACuWJrPnuP1epU/f35NnTpVNWrUUNu2bfX8889rypQpF11n5MiRypUrl3MLCwtL62oCAAAAAAD48Cs0yZcvnwIDAxUXF+ezPC4uTgUKFHBdp2DBgipdurQCAwOdZeXKlVNsbKwSEhJc1xkwYICOHTvm3Pbs2eNPNQEAAAAAAK6YX6FJUFCQatSooZiYGGeZ1+tVTEyMIiMjXdepW7eutm3bJq/X6yzbsmWLChYsqKCgINd1goODlTNnTp8bAAAAAABAevK7e050dLSmTZumWbNmadOmTXryySd16tQpZzadDh06aMCAAU75J598Un/88Yd69uypLVu2aP78+RoxYoR69OiRes8CAAAAAAAglfk95XDbtm118OBBDR48WLGxsapataoWLlzoDA67e/duBQT8lcWEhYXpyy+/VO/evVW5cmUVLlxYPXv2VP/+/VPvWQAAAAAAAKQyv0MTSYqKilJUVJTrfYsXL06xLDIyUsuXL7+chwIAAAAAAMgQaT57DgAAAAAAwLWI0AQAAAAAAMAFoQkAAAAAAIALQhMAAAAAAAAXhCYAAAAAAAAuCE0AAAAAAABcEJoAAAAAAAC4IDQBAAAAAABwQWgCAAAAAADggtAEAAAAAADABaEJAAAAAACAC0ITAAAAAAAAF4QmAAAAAAAALghNAAAAAAAAXBCaAAAAAAAAuCA0AQAAAAAAcEFoAgAAAAAA4ILQBAAAAAAAwAWhCQAAAAAAgAtCEwAAAAAAABeEJgAAAAAAAC4ITQAAAAAAAFwQmgAAAAAAALggNAEAAAAAAHBBaAIAAAAAAOCC0AQAAAAAAMAFoQkAAAAAAIALQhMAAAAAAAAXhCYAAAAAAAAuCE0AAAAAAABcEJoAAAAAAAC4IDQBAAAAAABwQWgCAAAAAADggtAEAAAAAADABaEJAAAAAACAC0ITAAAAAAAAF4QmAAAAAAAALghNAAAAAAAAXBCaAAAAAAAAuCA0AQAAAAAAcEFoAgAAAAAA4ILQBAAAAAAAwAWhCQAAAAAAgAtCEwAAAAAAABeEJgAAAAAAAC4ITQAAAAAAAFwQmgAAAAAAALggNAEAAAAAAHBBaAIAAAAAAOCC0AQAAAAAAMAFoQkAAAAAAIALQhMAAAAAAAAXhCYAAAAAAAAuCE0AAAAAAABcEJoAAAAAAAC4IDQBAAAAAABwQWgCAAAAAADggtAEAAAAAADABaEJAAAAAACAC0ITAAAAAAAAF5cVmkyaNEnh4eEKCQlR7dq1tWLFiouWnTlzpjwej88tJCTksisMAAAAAACQHvwOTebMmaPo6GgNGTJEa9asUZUqVdSsWTMdOHDgouvkzJlT+/fvd26//fbbFVUaAAAAAAAgrfkdmowdO1aPP/64OnfurPLly2vKlCnKmjWrZsyYcdF1PB6PChQo4NxCQ0OvqNIAAAAAAABpza/QJCEhQatXr1aTJk3+2kBAgJo0aaJly5ZddL2TJ0+qaNGiCgsLU8uWLfXLL79cfo0BAAAAAADSgV+hyaFDh5SYmJiipUhoaKhiY2Nd1ylTpoxmzJihTz75RO+++668Xq/q1KmjvXv3XvRx4uPjdfz4cZ8bAAAAAABAekrz2XMiIyPVoUMHVa1aVQ0aNNBHH32km2++WW+++eZF1xk5cqRy5crl3MLCwtK6mgAAAAAAAD78Ck3y5cunwMBAxcXF+SyPi4tTgQIFLmkbmTNnVrVq1bRt27aLlhkwYICOHTvm3Pbs2eNPNQEAAAAAAK6YX6FJUFCQatSooZiYGGeZ1+tVTEyMIiMjL2kbiYmJWr9+vQoWLHjRMsHBwcqZM6fPDQAAAAAAID1l8neF6OhodezYUTVr1lRERITGjx+vU6dOqXPnzpKkDh06qHDhwho5cqQkaejQobr11ltVsmRJHT16VK+88op+++03denSJXWfCQAAAAAAQCryOzRp27atDh48qMGDBys2NlZVq1bVwoULncFhd+/erYCAvxqwHDlyRI8//rhiY2N10003qUaNGlq6dKnKly+fes8CAAAAAAAglfkdmkhSVFSUoqKiXO9bvHixz9/jxo3TuHHjLudhAAAAAAAAMkyaz54DAAAAAABwLSI0AQAAAAAAcEFoAgAAAAAA4ILQBAAAAAAAwAWhCQAAAAAAgAtCEwAAAAAAABeEJgAAAAAAAC4ITQAAAAAAAFwQmgAAAAAAALggNAEAAAAAAHBBaAIAAAAAAOCC0AQAAAAAAMAFoQkAAAAAAIALQhMAAAAAAAAXhCYAAAAAAAAuCE0AAAAAAABcEJoAAAAAAAC4IDQBAAAAAABwQWgCAAAAAADggtAEAAAAAADABaEJAAAAAACAC0ITAAAAAAAAF4QmAAAAAAAALghNAAAAAAAAXBCaAAAAAAAAuCA0AQAAAAAAcEFoAgAAAAAA4ILQBAAAAAAAwAWhCQAAAAAAgAtCEwAAAAAAABeEJgAAAAAAAC4ITQAAAAAAAFwQmgAAAAAAALggNAEAAAAAAHBBaAIAAAAAAOCC0AQAAAAAAMAFoQkAAAAAAIALQhMAAAAAAAAXhCYAAAAAAAAuCE0AAAAAAABcEJoAAAAAAAC4IDQBAAAAAABwQWgCAAAAAADggtAEAAAAAADABaEJAAAAAACAC0ITAAAAAAAAF4QmAAAAAAAALghNAAAAAAAAXBCaAAAAAAAAuCA0AQAAAAAAcEFoAgAAAAAA4ILQBAAAAAAAwAWhCQAAAAAAgAtCEwAAAAAAABeEJgAAAAAAAC4ITQAAAAAAAFwQmgAAAAAAALggNAEAAAAAAHBBaAIAAAAAAOCC0AQAAAAAAMDFZYUmkyZNUnh4uEJCQlS7dm2tWLHiktabPXu2PB6PWrVqdTkPCwAAAAAAkG78Dk3mzJmj6OhoDRkyRGvWrFGVKlXUrFkzHThw4G/X27Vrl/r27av69etfdmUBAAAAAADSi9+hydixY/X444+rc+fOKl++vKZMmaKsWbNqxowZF10nMTFRDz/8sF588UUVL178iioMAAAAAACQHvwKTRISErR69Wo1adLkrw0EBKhJkyZatmzZRdcbOnSo8ufPr8cee+ySHic+Pl7Hjx/3uQEAAAAAAKQnv0KTQ4cOKTExUaGhoT7LQ0NDFRsb67rODz/8oOnTp2vatGmX/DgjR45Urly5nFtYWJg/1QQAAAAAALhiaTp7zokTJ9S+fXtNmzZN+fLlu+T1BgwYoGPHjjm3PXv2pGEtAQAAAAAAUsrkT+F8+fIpMDBQcXFxPsvj4uJUoECBFOW3b9+uXbt2qUWLFs4yr9d7/oEzZdLmzZtVokSJFOsFBwcrODjYn6oBAAAAAACkKr9amgQFBalGjRqKiYlxlnm9XsXExCgyMjJF+bJly2r9+vVau3atc7vnnnv0r3/9S2vXrqXbDQAAAAAAuGr51dJEkqKjo9WxY0fVrFlTERERGj9+vE6dOqXOnTtLkjp06KDChQtr5MiRCgkJUcWKFX3Wz507tySlWA4AAAAAAHA18Ts0adu2rQ4ePKjBgwcrNjZWVatW1cKFC53BYXfv3q2AgDQdKgUAAAAAACDN+R2aSFJUVJSioqJc71u8ePHfrjtz5szLeUgAAAAAAIB0RZMQAAAAAAAAF4QmAAAAAAAALghNAAAAAAAAXBCaAAAAAAAAuCA0AQAAAAAAcEFoAgAAAAAA4ILQBAAAAAAAwAWhCQAAAAAAgAtCEwAAAAAAABeEJgAAAAAAAC4ITQAAAAAAAFwQmgAAAAAAALggNAEAAAAAAHBBaAIAAAAAAOCC0AQAAAAAAMAFoQkAAAAAAIALQhMAAAAAAAAXhCYAAAAAAAAuCE0AAAAAAABcEJoAAAAAAAC4IDQBAAAAAABwQWgCAAAAAADggtAEAAAAAADABaEJAAAAAACAC0ITAAAAAAAAF4QmAAAAAAAALghNAAAAAAAAXBCaAAAAAAAAuCA0AQAAAAAAcEFoAgAAAAAA4ILQBAAAAAAAwAWhCQAAAAAAgAtCEwAAAAAAABeEJgAAAAAAAC4ITQAAAAAAAFwQmgAAAAAAALggNAEAAAAAAHBBaAIAAAAAAOCC0AQAAAAAAMAFoQkAAAAAAIALQhMAAAAAAAAXhCYAAAAAAAAuCE0AAAAAAABcEJoAAAAAAAC4IDQBAAAAAABwQWgCAAAAAADggtAEAAAAAADABaEJAAAAAACAC0ITAAAAAAAAF4QmAAAAAAAALghNAAAAAAAAXBCaAAAAAAAAuCA0AQAAAAAAcEFoAgAAAAAA4ILQBAAAAAAAwAWhCQAAAAAAgAtCEwAAAAAAABeEJgAAAAAAAC4ITQAAAAAAAFwQmgAAAAAAALi4rNBk0qRJCg8PV0hIiGrXrq0VK1ZctOxHH32kmjVrKnfu3MqWLZuqVq2qf//735ddYQAAAAAAgPTgd2gyZ84cRUdHa8iQIVqzZo2qVKmiZs2a6cCBA67l8+TJo+eff17Lli3TunXr1LlzZ3Xu3FlffvnlFVceAAAAAAAgrfgdmowdO1aPP/64OnfurPLly2vKlCnKmjWrZsyY4Vq+YcOGat26tcqVK6cSJUqoZ8+eqly5sn744YcrrjwAAAAAAEBa8Ss0SUhI0OrVq9WkSZO/NhAQoCZNmmjZsmX/uL6ZKSYmRps3b9Ztt9120XLx8fE6fvy4zw0AAAAAACA9+RWaHDp0SImJiQoNDfVZHhoaqtjY2Iuud+zYMWXPnl1BQUG66667NGHCBN1+++0XLT9y5EjlypXLuYWFhflTTQAAAAAAgCuWLrPn5MiRQ2vXrtXKlSs1fPhwRUdHa/HixRctP2DAAB07dsy57dmzJz2qCQAAAAAA4MjkT+F8+fIpMDBQcXFxPsvj4uJUoECBi64XEBCgkiVLSpKqVq2qTZs2aeTIkWrYsKFr+eDgYAUHB/tTNQAAAAAAgFTlV0uToKAg1ahRQzExMc4yr9ermJgYRUZGXvJ2vF6v4uPj/XloAAAAAACAdOVXSxNJio6OVseOHVWzZk1FRERo/PjxOnXqlDp37ixJ6tChgwoXLqyRI0dKOj8+Sc2aNVWiRAnFx8drwYIF+ve//6033ngjdZ8JAAAAAABAKvI7NGnbtq0OHjyowYMHKzY2VlWrVtXChQudwWF3796tgIC/GrCcOnVK3bt31969e5UlSxaVLVtW7777rtq2bZt6zwIAAAAAACCV+R2aSFJUVJSioqJc77twgNeXXnpJL7300uU8DAAAAAAAQIZJl9lzAAAAAAAArjWEJgAAAAAAAC4ITQAAAAAAAFwQmgAAAAAAALggNAEAAAAAAHBBaAIAAAAAAOCC0AQAAAAAAMAFoQkAAAAAAIALQhMAAAAAAAAXhCYAAAAAAAAuCE0AAAAAAABcEJoAAAAAAAC4IDQBAAAAAABwQWgCAAAAAADggtAEAAAAAADABaEJAAAAAACAC0ITAAAAAAAAF4QmAAAAAAAALghNAAAAAAAAXBCaAAAAAAAAuCA0AQAAAAAAcEFoAgAAAAAA4ILQBAAAAAAAwAWhCQAAAAAAgAtCEwAAAAAAABeEJgAAAAAAAC4ITQAAAAAAAFwQmgAAAAAAALggNAEAAAAAAHBBaAIAAAAAAOCC0AQAAAAAAMAFoQkAAAAAAIALQhMAAAAAAAAXhCYAAAAAAAAuCE0AAAAAAABcEJoAAAAAAAC4IDQBAAAAAABwQWgCAAAAAADggtAEAAAAAADABaEJAAAAAACAC0ITAAAAAAAAF4QmAAAAAAAALghNAAAAAAAAXBCaAAAAAAAAuCA0AQAAAAAAcEFoAgAAAAAA4ILQBAAAAAAAwAWhCQAAAAAAgAtCEwAAAAAAABeEJgAAAAAAAC4ITQAAAAAAAFwQmgAAAAAAALggNAEAAAAAAHBBaAIAAAAAAOCC0AQAAAAAAMAFoQkAAAAAAIALQhMAAAAAAAAXhCYAAAAAAAAuCE0AAAAAAABcEJoAAAAAAAC4uKzQZNKkSQoPD1dISIhq166tFStWXLTstGnTVL9+fd1000266aab1KRJk78tDwAAAAAAcDXwOzSZM2eOoqOjNWTIEK1Zs0ZVqlRRs2bNdODAAdfyixcvVrt27bRo0SItW7ZMYWFhatq0qfbt23fFlQcAAAAAAEgrfocmY8eO1eOPP67OnTurfPnymjJlirJmzaoZM2a4ln/vvffUvXt3Va1aVWXLltVbb70lr9ermJiYK648AAAAAABAWvErNElISNDq1avVpEmTvzYQEKAmTZpo2bJll7SN06dP6+zZs8qTJ89Fy8THx+v48eM+NwAAAAAAgPTkV2hy6NAhJSYmKjQ01Gd5aGioYmNjL2kb/fv3V6FChXyClwuNHDlSuXLlcm5hYWH+VBMAAAAAAOCKpevsOaNGjdLs2bP18ccfKyQk5KLlBgwYoGPHjjm3PXv2pGMtAQAAAAAApEz+FM6XL58CAwMVFxfnszwuLk4FChT423VfffVVjRo1St98840qV678t2WDg4MVHBzsT9UAAAAAAABSlV8tTYKCglSjRg2fQVyTBnWNjIy86HqjR4/WsGHDtHDhQtWsWfPyawsAAAAAAJBO/GppIknR0dHq2LGjatasqYiICI0fP16nTp1S586dJUkdOnRQ4cKFNXLkSEnSyy+/rMGDB+v9999XeHi4M/ZJ9uzZlT179lR8KgAAAAAAAKnH79Ckbdu2OnjwoAYPHqzY2FhVrVpVCxcudAaH3b17twIC/mrA8sYbbyghIUFt2rTx2c6QIUP0wgsvXFntAQAAAAAA0ojfoYkkRUVFKSoqyvW+xYsX+/y9a9euy3kIAAAAAACADJWus+cAAAAAAABcKwhNAAAAAAAAXBCaAAAAAAAAuCA0AQAAAAAAcEFoAgAAAAAA4ILQBAAAAAAAwAWhCQAAAAAAgAtCEwAAAAAAABeEJgAAAAAAAC4ITQAAAAAAAFwQmgAAAAAAALggNAEAAAAAAHBBaAIAAAAAAOCC0AQAAAAAAMAFoQkAAAAAAIALQhMAAAAAAAAXhCYAAAAAAAAuCE0AAAAAAABcEJoAAAAAAAC4IDQBAAAAAABwQWgCAAAAAADggtAEAAAAAADABaEJAAAAAACAC0ITAAAAAAAAF4QmAAAAAAAALghNAAAAAAAAXBCaAAAAAAAAuCA0AQAAAAAAcEFoAgAAAAAA4ILQBAAAAAAAwAWhCQAAAAAAgAtCEwAAAAAAABeEJgAAAAAAAC4ITQAAAAAAAFwQmgAAAAAAALggNAEAAAAAAHBBaAIAAAAAAOCC0AQAAAAAAMAFoQkAAAAAAIALQhMAAAAAAAAXhCYAAAAAAAAuCE0AAAAAAABcEJoAAAAAAAC4IDQBAAAAAABwQWgCAAAAAADggtAEAAAAAADABaEJAAAAAACAC0ITAAAAAAAAF4QmAAAAAAAALghNAAAAAAAAXBCaAAAAAAAAuCA0AQAAAAAAcEFoAgAAAAAA4ILQBAAAAAAAwAWhCQAAAAAAgAtCEwAAAAAAABeEJgAAAAAAAC4ITQAAAAAAAFwQmgAAAAAAALggNAEAAAAAAHBxWaHJpEmTFB4erpCQENWuXVsrVqy4aNlffvlF9913n8LDw+XxeDR+/PjLrSsAAAAAAEC68Ts0mTNnjqKjozVkyBCtWbNGVapUUbNmzXTgwAHX8qdPn1bx4sU1atQoFShQ4IorDAAAAAAAkB78Dk3Gjh2rxx9/XJ07d1b58uU1ZcoUZc2aVTNmzHAtX6tWLb3yyit68MEHFRwcfMUVBgAAAAAASA9+hSYJCQlavXq1mjRp8tcGAgLUpEkTLVu2LNUqFR8fr+PHj/vcAAAAAAAA0pNfocmhQ4eUmJio0NBQn+WhoaGKjY1NtUqNHDlSuXLlcm5hYWGptm0AAAAAAIBLcVXOnjNgwAAdO3bMue3ZsyejqwQAAAAAAG4wmfwpnC9fPgUGBiouLs5neVxcXKoO8hocHMz4JwAAAAAAIEP51dIkKChINWrUUExMjLPM6/UqJiZGkZGRqV45AAAAAACAjOJXSxNJio6OVseOHVWzZk1FRERo/PjxOnXqlDp37ixJ6tChgwoXLqyRI0dKOj947MaNG53/79u3T2vXrlX27NlVsmTJVHwqAAAAAAAAqcfv0KRt27Y6ePCgBg8erNjYWFWtWlULFy50BofdvXu3AgL+asDy+++/q1q1as7fr776ql599VU1aNBAixcvvvJnAAAAAAAAkAb8Dk0kKSoqSlFRUa73XRiEhIeHy8wu52EAAAAAAAAyzFU5ew4AAAAAAEBGIzQBAAAAAABwQWgCAAAAAADggtAEAAAAAADABaEJAAAAAACAC0ITAAAAAAAAF4QmAAAAAAAALghNAAAAAAAAXBCaAAAAAAAAuCA0AQAAAAAAcEFoAgAAAAAA4ILQBAAAAAAAwAWhCQAAAAAAgAtCEwAAAAAAABeEJgAAAAAAAC4ITQAAAAAAAFwQmgAAAAAAALggNAEAAAAAAHBBaAIAAAAAAOCC0AQAAAAAAMAFoQkAAAAAAIALQhMAAAAAAAAXhCYAAAAAAAAuCE0AAAAAAABcEJoAAAAAAAC4IDQBAAAAAABwQWgCAAAAAADggtAEAAAAAADABaEJAAAAAACAC0ITAAAAAAAAF4QmAAAAAAAALghNAAAAAAAAXBCaAAAAAAAAuCA0AQAAAAAAcEFoAgAAAAAA4ILQBAAAAAAAwAWhCQAAAAAAgAtCEwAAAAAAABeEJgAAAAAAAC4ITQAAAAAAAFwQmgAAAAAAALggNAEAAAAAAHBBaAIAAAAAAOCC0AQAAAAAAMAFoQkAAAAAAIALQhMAAAAAAAAXhCYAAAAAAAAuCE0AAAAAAABcEJoAAAAAAAC4IDQBAAAAAABwQWgCAAAAAADggtAEAAAAAADABaEJAAAAAACAC0ITAAAAAAAAF4QmAAAAAAAALghNAAAAAAAAXBCaAAAAAAAAuCA0AQAAAAAAcEFoAgAAAAAA4ILQBAAAAAAAwMVlhSaTJk1SeHi4QkJCVLt2ba1YseJvy//nP/9R2bJlFRISokqVKmnBggWXVVkAAAAAAID04ndoMmfOHEVHR2vIkCFas2aNqlSpombNmunAgQOu5ZcuXap27drpscce008//aRWrVqpVatW2rBhwxVXHgAAAAAAIK34HZqMHTtWjz/+uDp37qzy5ctrypQpypo1q2bMmOFa/rXXXtMdd9yhZ555RuXKldOwYcNUvXp1TZw48YorDwAAAAAAkFYy+VM4ISFBq1ev1oABA5xlAQEBatKkiZYtW+a6zrJlyxQdHe2zrFmzZpo3b95FHyc+Pl7x8fHO38eOHZMkHT9+3J/qXjJv/Gm/yh/32CWXTfwz0a9tn0y89PJptT/SA/s8/bHP0x/7PP2xz9Mf+zz9sc/TH/s8/bHP0x/7/OLbNrv054rrj1+hyaFDh5SYmKjQ0FCf5aGhofr1119d14mNjXUtHxsbe9HHGTlypF588cUUy8PCwvypbprJ5VfpTX6VjvCrIv7V5FrGPk9/7PP0xz5Pf+zz9Mc+T3/s8/THPk9/7PP0dyPt8xMnTijXDfTawpdfoUl6GTBggE/rFK/Xqz/++EN58+aVx+PJwJr55/jx4woLC9OePXuUM2fOjK7ODYF9nv7Y5+mPfZ7+2Ofpj32e/tjn6Y99nv7Y5+nvWt7nZqYTJ06oUKFCGV0VZCC/QpN8+fIpMDBQcXFxPsvj4uJUoEAB13UKFCjgV3lJCg4OVnBwsM+y3Llz+1PVq0rOnDmvuQPEtY59nv7Y5+mPfZ7+2Ofpj32e/tjn6Y99nv7Y5+nvWt3ntDCBXwPBBgUFqUaNGoqJiXGWeb1excTEKDIy0nWdyMhIn/KS9PXXX1+0PAAAAAAAwNXA7+450dHR6tixo2rWrKmIiAiNHz9ep06dUufOnSVJHTp0UOHChTVy5EhJUs+ePdWgQQONGTNGd911l2bPnq1Vq1Zp6tSpqftMAAAAAAAAUpHfoUnbtm118OBBDR48WLGxsapataoWLlzoDPa6e/duBQT81YClTp06ev/99zVw4EA999xzKlWqlObNm6eKFSum3rO4SgUHB2vIkCEpuhoh7bDP0x/7PP2xz9Mf+zz9sc/TH/s8/bHP0x/7PP2xz3Gt8xjzJwEAAAAAAKTg15gmAAAAAAAANwpCEwAAAAAAABeEJgAAAAAAAC4ITQAAAAAAAFwQmuCSeb1e5/+MHwxcuaNHj6pKlSr65JNPMroqAAA4Tp48qRdeeEExMTGSOO/D1Wv9+vX6/PPPde7cuYyuCq5jhCa4ZAEBATp58qQkyePxZHBtIPkGWbj25M6dW4UKFdKIESOcz1Za4GQXAOCPLVu26PPPP9e///1veb3eG/q8LzExURLfpVer1157TQ8//LB27NiR0VXBdYzQBJds8eLFKliwoPP36NGj9eOPP2ZgjW5cSV/gAQG+H2G+0K8dSVdE3njjDa1fv16zZs1Kk8dJTEy8oU92rzVJn21cOxITE2/4Yy9XeK8/1atXV7t27bR582Z98MEHkm7cc4zAwECdPn1aR48elXTj7oerTdKFwylTpihTpkx6++239eeff2ZwrXC9IjTBJatRo4YKFSqkf/3rX8qaNas+++wzFShQIKOrdUMKDAyUJH322Wfq06ePXn/9df3+++/8OL5GmJkyZcok6fxr2aNHD40ZM0a7d+9O9ccKDAzUiRMnNGzYML399ttatGhRqj8GUoeZOZ/tmJgYffbZZ9q1a1fGVgp/y+v1KjAwUB6PRzt27NCRI0cyukoZIul49tVXX2njxo0ZXBtcqaQfo23btlXhwoU1Z84cHTx4UB6P54YIDC5sxXvw4EG1bNlSHTp0kERr66uBmTnvxUyZMmnYsGGaNGmSfvrppwyuGa5XhCa4qOQHJEmKj49XXFycvvvuOz3//PP6/vvvVbRo0Qys4Y3rjz/+UKtWrdSpUyedOHFCU6dOVadOnfTOO+9kdNVwCTwej1asWKHy5curXbt2Wr16tXbt2qUpU6ak2mMkfXZnzJihW265RV999ZXmzp2r++67T5MnT9axY8dS7bGQOjwejzZu3KiaNWuqa9euev7559WiRQuNGTMmo6uGiwgICNDvv/+uu+++W02aNNFtt92mRx991AkOrtcfmF6v1+e5LVmyRIUKFdITTzyhyMhITZ8+XSdOnMjAGsJfyVsLJbViveWWW9SyZUsdPHhQb7/9tqQbIzC4sBVv7ty51aJFC61fv17/+9//JNE9OqN5PB4FBgZqw4YN6tixo7Jly6aTJ0/qnXfe0aFDhzK6ergOEZrAVVKT/uRXFfLly6eXX35ZjRo10jfffCOJL4304NZcf8GCBTpw4IA2bdqkqVOn6rvvvtO6dev07rvv8mVxDfjzzz/Vr18/1ahRQ/Pnz9ewYcP09NNPa9y4cVqzZs1lbfPCkNPj8ejQoUOaNm2axowZo++//16ff/65GjVqpOHDh2vFihWp9XSQSs6dO6ehQ4eqUqVK+uWXX7Ru3Tp169ZNzzzzjFauXJnR1YNShiA7d+5UmzZtlC1bNs2bN0/Tp0/Xrl279PTTT2vv3r3X5Q9Mr9ergIAAeTweZyymr776Sn379tWaNWvUsWNHTZgwQXPnzs3gmuJSJJ3HJbUW+v7777V582YnRLn33ntVqVIlffHFF9qwYYOk6zMMTB4a/fHHH+rVq5cOHDggScqcObPuuusuVa9eXYMHD5aUMlhB6vun3xgff/yxIiMjlSVLFp08eVL169fXjBkz9P3331+X71FkMAOSOXfunM/fw4YNs169etnMmTOdZT/88INlypTJ5syZY2ZmiYmJ6VrHG0ViYqJ5vV7n7wMHDpiZWXx8vHXs2NEmTJhgZmavvPKKFShQwBo2bGirV6/OkLrC3YWfpyRr1qyxbNmy2fLly51lJ06csNq1a1uLFi0sISHBr8c5e/as8//jx487///3v/9tdevWNTOztWvXWrNmzSxPnjw2fvx4vx8Dqedi74uVK1da7ty57fTp02Z2/vh78803W/PmzW337t3pWUVcIDEx0fW77rPPPrMaNWo4f3/++eeWNWtWu/fee6+71yz58z9x4oQ9/PDDVrt2bevWrZvdf//9tnfvXuf+Fi1aWOvWre2XX37JiKriMsydO9eKFClilSpVsjx58ljfvn1ty5YtZmb2xRdf2G233WY9e/bM2EqmgeTH4/j4eJs3b579+9//trJly1rfvn19yv7nP/+xYsWK2VtvvWVmnP+mpeTnvydOnHC9r0ePHtaiRQuf+26//XaLiIiwPXv2pH0lcUMhJoWPpP70cXFxatasmd577z399ttv6ty5s4YPH64///xTdevWVZcuXdS3b19JpO1pJelK3tdff63GjRtr6NChOnPmjIKCgrR371598cUXioiI0OTJk/Xqq69q0aJFql69uuLi4nTw4MGMrv4Nz5KNT7Fy5Upt2LBBZ86ckSQVLFhQmTJl0unTpyWdv8KVPXt2Pfvss/r888/11Vdf+fVYmTJlUkJCgnr27KmuXbtq6dKlkqTs2bNr+/bt6tOnj+rXr69ChQrp559/Vs+ePZUpUybeJxkg+fti27ZtPl0YMmXKpDp16mjGjBkqU6aMPvjgA02bNk0LFixQWFiY8/4xrqClq6SWFQEBAfr55581ffp05wro999/r4iICP3222+qUaOGnnjiCb366qv6z3/+o7CwsGtyYN8L319JzzXpu37Dhg0aMWKEDh06pBYtWujDDz/U//73P4WEhDjrPPnkk9q1a5c+/vjj9Ks4LovX69Wrr76q5557TtHR0frxxx81Y8YMrVy5Uq+88ook6Y477lCdOnW0YsUK5/vpejkOJR2PN2/erLCwMP33v/9VRESEHnroIc2fP99njIx69eqpTp06Gjt2rM6cOaOAgIDrZj9cbTwej/73v/+pePHievbZZ1O0pJWkpUuXqly5cpKks2fPSpImTZqkVatW6eOPP1ZCQkL6VxzXrwwMbHAVio+Pt6ZNm1p0dLT17t3buRr95ptv2k033WQLFiwwM7OtW7daaGioPfPMM2Zm9r///c9ef/31DKv39ejUqVPWqVMny5Ejhw0cONAWLFhg+/fvNzOzDz74wAICAqx3794+6+zYscP69+9vy5Yty4gq4wI//fSTVatWzUqXLm1Fixa1+++/377++mv7888/rVmzZtahQwef8gsXLjSPx2O1atWyQ4cO/eP2k662fPTRR5YvXz6rV6+effjhh06Lo3Xr1lnVqlWtcOHCzhXDpPXGjh1rs2bN4kpZGjp27JjzGiW/mrl9+3arX7++hYaGWs2aNW3GjBlmZvbjjz9ayZIlLVeuXDZixAifdT799FN76aWX0vcJwHHkyBG777777KabbrKOHTs6rcQ++ugj83g8li1bNnvmmWcsNjbWzM5/xr755htbtGhRBtb6yuzZs8fOnDnj/O31eq1v374WGhpqLVq0cI5RH374oVWrVs2GDh3qs363bt3s9ttvt2+++SZd642LS94qMbkZM2bYG2+8YWbnzwOfeOIJCw4OtltuucX+85//mNn5FpItWrSwBx980Od9ca3xer0+rRj27dtnDRo0sCFDhtiwYcOc+1atWmVNmza1tm3b+qzft29f83g8NmDAAGd7SF3nzp2z999/3ypUqGCRkZGWNWtW+/HHH33uNzPr2bOnVa5cOcXy+vXrW61ateynn35K13rj+kYTgRuY2xSBQUFByps3r8aNG6fg4GBlzpxZktS1a1eVKVNGU6ZM0e+//66SJUtq5MiRGj9+vGrWrKmGDRv6XGWCf9yuRi5btkw//fSTVq9erWHDhql58+bObEW33nqrqlevrp9//ln79+/XiRMntGvXLg0YMEDff/+9smfPnt5PARfYvXu3nnrqKdWtW1crVqzQl19+qRw5cujBBx9UYGCg7r77bq1fv15vvfWWs86ePXvUrVs37d+/X/v27XPdbvI+vh6PR2fOnNHkyZPVs2dPff/997r//vtVvXp1SVKxYsXUuHFjnT17VomJiTp58qS8Xq+mT5+u6dOnO+MRIPWNHz9eDRo0cGZTCQwMVHx8vNatW6c333xT1apV07///W/VqFFDTz75pJYvX66IiAjVq1dPpUuXVv369Z0roOvWrdPUqVN15MgRnTp1KiOf1g3p8OHDeuSRR3T69GktW7ZMb775pmrVqiVJqlKlim699Vb961//0ujRoxUaGipJ2rJli1577TXt2bPnmhz767XXXlP16tWVkJCgX3/9Va1bt9bp06dVs2ZN3XTTTfJ4PMqbN68k6Z577lFkZKS+/vprrV692tlG9+7dtXXrVi1fvvya3AfXo6RxS9atW+eM1yFJjzzyiLp27aqFCxeqTJky2rFjh95//32VL19ekyZNUmJioqpVq6batWtr48aNPq/ztcTMnPH6klomhIaG6sCBAxo6dKiKFi3qtGKoUaOG7rnnHv3000+aN2+es434+Hg1adJE33zzjU6cOHFdjluU0cxMx48f1wMPPKCPP/5YERERev75552WI0nfjY0bN9a5c+c0fvx4Z/mBAwcUHx+vVatW6Z133nFeZ+CKZXRqg4z31Vdf2cqVK+3gwYNmdr6FQ7Zs2eypp55y+tabmS1ZssRy5MhhM2fOdFqgLFiwwKZNm+YzjgIu3YVXPP7880/n/7Nnz7bcuXPbvn377Mcff7RZs2bZCy+8YCNHjrSEhAT7+eefrUSJEla4cGFr1KiR5c6d25o3b26///57RjyVG9aF41MkvZ5z5861UqVKOctfeeUVy5kzp7Vo0cJOnjxp+/btswEDBpjH47HWrVvbvffea7lz57bvvvvuH1t/xMbG2tdff21nzpyx3bt3W8WKFW38+PF27tw5mzt3rs2cOdOGDx9uO3futIMHD1qrVq0sV65cFhkZabVq1bI8efLYO++8k/o7A05LgyNHjti6det87uvfv795PB5r2LChz+e0UaNGVr9+fTMz27ZtmzVr1szy589v7dq1s/bt21vWrFmtU6dOKfp1I3VdbNySX3/91QoWLGgbN240M7Pff//djh075ozh8f7775vH47HHHnvMJk+ebMOHD7d8+fJZ27ZtnbGorhXJn3+2bNmsevXqlilTJuvYsaMlJiba4cOHLSoqygoVKuQzZsB3331n//rXvywqKspne0uXLk23usNd8tf0f//7n4WHh1uRIkWsRIkS9vnnnzv3HTt2zJo1a2aDBw+2kydPmplZ27ZtLTg42F5++WUzMzt48OA1OVbEhd/T06ZNs3vuucf5DH/zzTfm8XjsvffeM7O/9tlvv/1m3bp1s8yZM9tLL71kbdq0sSZNmtjatWvT9wncIJK/V3///Xfnfbhq1SoLDAx0xlJMcujQIRs4cKBlyZLFpkyZYqtWrbJBgwbZoEGD7JtvvrG4uLh0rT+ub4QmN7CFCxdaeHi4Va5c2YoWLWp33HGHff3112ZmNmLECMudO7etXLnSZ51HHnnEypYta2vWrEmxvYs1+4S75F8OP/zwg91999324IMP2vz5883r9drPP/9szZs3t5CQECtTpozde++91qBBA8udO7dFR0eb2flm/jExMTZlyhSfk1O6XKSP5IHXhWHVmDFj7Omnn7aPPvrIihUrZhUqVLBPPvnEuT/pNZo5c6b16tXL2rRpY+vXr3fuTzrJcxs0tEuXLlagQAHbuHGjnThxwrp162a33HKL5c+f3+68806rXr26VahQwUqWLOms8+mnn9qUKVNs8uTJPtvivZI6EhIS7K677rLu3bv7dK1asmSJbd682czO/+AoV66c1atXzyeQ3rBhgwUEBNj06dPN7HzgMmHCBBswYIB17tzZVqxY4ZTl9UobyT/LW7dutS+++MKOHDliZmYbN260ypUr2/33329dunSxdu3aWdmyZe2mm26ySZMmmZnZu+++ay1btrSGDRvarbfeanPnzs2Ip3HZzp075+yDs2fP2rZt28zj8VhQUJDzQzLJokWLrG7duikCkhdeeMHKlCnj+tx532as3bt324oVK6xDhw42evRo+/HHH+2RRx6x/Pnz2//+9z8zM9u8ebPlyZPHZs+ebWZmf/zxh7Vp08bq169vd9xxh50+fdp5j1yrXVLefPNNGzhwoHXq1MkCAgJszpw5zrlry5YtLSIiwvmhnuT48ePWu3dva9q0qU/QYnbxQb1x+dasWWPbtm1z/k46dnTt2tVKly5thw8f9il/+vRp69Onj5UtW9by589vpUuX9hlkn2MPUguhyQ3iwi+4HTt2WO3ate2FF14ws/Np+n333We5c+d2yoSHh1uHDh3s6NGjzrJ9+/ZZtWrVnB8BF9s+Ls2ZM2fss88+s3LlytkTTzxhTZs2tSJFitjUqVPN7PyX9aeffmrbtm2zHTt2mJlZv379rHz58q773Ov18iWeDpLv402bNlnDhg2tTJky9uijj9rixYvN7PzMNR6Px/Lnz28vv/yy0wf83Llz9sknn6T4IZLkYl/wyWe7OXbsmBUuXNieffZZO3v2rP3555/28ccf29KlS23dunV27NgxW7t2rWXOnNk5Ib4QIWfqSfosDhkyxMqVK+eM/WRmFhwcbA8//LAdO3bMzMymTJliISEhTiuUpHW7d+9upUuXtl27drk+xsVaQSD1nDt3zqKjoy04ONiKFStmVapUcT4/X331lUVGRlr79u1t4sSJ9sknn9iIESMsODjY57N04VhE18LxOPn7asWKFfbVV1+Z1+u1ZcuWWcmSJa19+/Y+P1Ti4+Nt+PDhVqZMGZ+wfsOGDda/f3/buXNnelYfF3A7Ttx1113m8XisRYsWPi1aK1SoYA8//LAdPnzYdu3aZXfccYfddttttmDBAmvTpo098MAD9vPPP6dn9dNEQkKCdezY0UJDQ23y5MnWt29fK1mypFWrVs22b99uZue/y4OCgmzatGnOesnPs5K3qOZYnHby5cvnjI+U/Ni6f/9+y507tw0fPtxZlvx1OHr0KDNIIk0RmlznLtZ1YPLkyVapUiUzO39Qevrppy1XrlzWoUMH5+To888/t+DgYJs/f77Purg8F+6/7777zpo3b2533HGHMxCkmVmnTp2scePG9v3336fYRmxsrN1+++02evTof9w+0lZ8fLxt27bNunbtaj169LCJEyda/fr1rXTp0k6ZSpUqWcuWLX1aFSxfvtzuuusumzZtWorgwu3zevLkSatXr5699957Pt25Xn31VStYsKB9++23rvUbPny4NWvWzLXrHO+V1JV8f1avXt06dOjgnIh/8MEHljVrVp/BMKtUqWIPPPCAz+t94sQJ83g8NmTIkBSvDyfoae/rr7+2cePGWadOneynn36yX375xerXr2933XWXM5hgfHy8mf31er/33nsWGRnp2gT8ag1LYmNj7f3337dTp075LN+7d6/dcccdFhoaan369HG6IiUNTv3JJ5/4vA/Xrl1rLVu2tDvvvDNd64+LS0xMvOixPS4uzrJly2atW7d23sdmZvPnz7ecOXM63R4++eQTa9y4sRUrVszuuusun+5lV+t7+p94vV7bu3evlStXzmbOnOks37JliwUFBdnw4cOdIKlPnz4WHh7+t92cr9X9cLVLOh/q0qWLtW/f3ue+pPf1mDFjLFeuXLZjxw7buHGjTZgwwX777beLbgtITYQmN4ipU6dav3797NdffzUzs9dff9169eplEyZMsPz581u9evVsyZIlTvmkA07VqlWtRo0aPq1NzPjS8MfFrhDv27fPIiIiLHv27D6z3Sxbtszq169vTz31lJ07d84OHz5s7733nvXu3dvy5ctnzZo1c8ZNQPq48ET0+PHj1qZNG8uWLZs9/PDDTiuQn3/+2YoUKeI0W1+wYIHlzp3b6tWrZy+++KJ169bNsmXLZk8++aRPkPJPGjRoYBERESn6kpcrV87atWvnzKq0YMECmz59utWvX9/y5cvnNLNG2kj+uU46Zs6dO9duueUWmzlzpnN/RESENW3a1PncfvXVVxYQEGBffvmlz3a+/PJL+mCnMa/Xm+J4vGfPHitVqpTlzZvXRowY4Sz/9ttvrW7dus4scWbnj9vbtm2zN9980woVKuQacl3N3njjDfN4PPbDDz84yzZt2mTVqlWzBx980LZt25YiaL399tutTp06znEmycSJE6106dI+3QrNCPky2uLFi61Dhw7Wv39/+/77751Wbv3797ebb745RcuRFi1aWMOGDZ2gNz4+3ue75np4PdetW2cBAQHOD+yk4/Vzzz1n+fPnd1ooHD161Dwej/Xt2zfD6nojSDpmLl682BYuXOhz31NPPWVt2rSxs2fPul5AKFu2rJUoUcI8Ho+1atXKeX8DaY3Q5DpzYX/T9evXW+XKla1YsWL2wgsvOGOUTJw40QIDAy08PNxmzZrlrH/27FmbPn26vf/++2Z2/mRy1apV6fwsrh/JD/gbN260yZMn248//ugMujtr1iwrWrSojRs3zme9wYMHW926dW3+/Pl2+vRpGz58uN1zzz0+g7ZdSyfq16qLdXfyer02e/ZsK1iwoPXq1ctZnpCQYBMnTjSPx+OcgH7zzTf29NNP20MPPWT33HOPz7R5//QaJj32/v37LTAw0MaOHWsJCQnOevPmzbNMmTLZhx9+aGZmb7/9trVo0cKio6N9riYi9SV/X1x40nbnnXdas2bNnBYKP/30k3k8Hps5c6Zzsv7AAw9YuXLlXFsCXQ8/Uq5Gbq9Z0mdp0qRJljdvXhs2bJjPOn369LH69evbl19+aQkJCTZixAi79dZbLTw8/JodTDlpfJakrkQLFy604sWLO8eMQ4cOWXx8vHO1fceOHc5AmL///ruNHDnSpk2bZomJiSkuqCDtxcfH2/33329TpkzxWZ6QkGC9evWyrFmz2qOPPmoRERFWqlQpe+KJJ5wy+fLlsx49evgMKr1582bzeDw2ZcoUn8/I9dTdNzY21kqVKmUvvviimf3V3XXLli2WOXNmi46Otj/++MPMzofadDFLfRe2/jhz5ozddtttVqJECZ8LPHPnzrXs2bOnOIeJj4+3GTNmWM6cOa1y5cpOK3ggvRCaXEfcTrQ7duxobdq08enDmqRo0aL2yCOP+PRVXrBggTVu3NhmzZrlk/LyA/2fnT592j766KMUy8+cOWOPPfaY5cyZ0+rWrWtly5a1yMhIO3XqlHm9XmvdurW1atXKNmzY4Kyze/duu/322+2OO+6wo0ePpmhKfb2cyFzNkn+e9uzZY++8846tW7fO+bzs27fPOnToYEWKFPFZb+/evVanTh274447zOyvz07SmCZJ205MTHRO0v5O0mv97LPPWsGCBX1mZFm0aJFlzpzZ6tevbzt27LCEhASfH/A0UU1bsbGx9sgjj9jtt99uTz75pNPEffXq1Va4cGF7+eWXnR8n7du3t7JlyzoD3P366692yy23pBgfCmkrMTHRnn76aatVq5bde++9NnDgQDM7f/xu3ry5tW7d2rZs2eKU37Rpk91+++3WuXNnO336tG3atMk+/fTTFNu8FiQdD7777jvzeDzOgK0ffvihVahQwfr06WNPPvmkdezY0YoXL27lypVzBod//vnnrXz58pY/f34LDw93xm4yu3ae//XiyJEjNmPGjBQtTtevX2/ly5d3WrB5vV57/fXXLTw83OmW8vbbb1tQUJDz+iV9Py1YsOCaO8/bvHmzffLJJ5c0jsWJEyfsueeeszJlyvgM5Prf//7XChcubDlz5kwx09O1tj+uVsn348mTJ23ZsmXOa7B//34bMWKEZc+e3d544w07ffq07dy50ypUqOAzNpjZ+TCrYMGCzliMSdvmfBjphdDkOnDhCUtUVJS99957Fhsbazly5EjxQz7px9ucOXOsXLlyVqZMGRs4cKDde++9ljVrVhsyZEh6Vf26kjTwZ9LV5SSzZs2yW2+91fmxe/LkSfN4PPbUU0+Z2fmrfNWqVUtxhXPixIk2evRonx++fDmkv169elnu3LmtWrVqVqJECWvdurVz3+LFi+2WW25xmvQnjTny6aefmsfjsa+++irF9s6dO2dr1661f/3rXxYdHf2P3TGSn3CEhoZa9+7dbevWrWZmNm7cOIuOjraIiAiffr0XTmWN1Pftt99aaGiotW7d2qZOnWq9e/f2GXj36aefturVq9uiRYvM7PxJe5YsWaxv375O1yxeo7SxadMmK168uP3yyy8+y3/77TerXr261alTx+bMmWPTpk2zm266yQYMGGBmZh9//LFVq1bNXnnlFZ/1XnrpJStfvrxPlxazazOUTDpfaNGihVWrVs0OHz5sR44cscmTJ1uZMmWsU6dO9uqrr9q7775rDzzwgM8YTWvWrHF+kCP9JD/Hu/B8L/n5xhdffGF58uTxmXnk999/t27dulnt2rWddWvVqmX169d3/e65FgKwkydPWocOHeymm26yOnXqmMfjsREjRvxjN41Vq1ZZvXr1rHbt2vbZZ5/ZqlWrrEWLFvb1119btWrV7MknnzSza2MfXIveeOMNCwkJsdKlS1uxYsVs+fLlzr4eOHCg1axZ06KiomzPnj1Wvnx5pyVJUpk//vjD5+LTtXj8xbWN0OQ6snnzZhs0aJBVr17dvv32W9uzZ4/lzJnTuVLkdoDZsGGD9erVy7p162adOnXymbmBE/pLlzQAW7169axVq1bOvk5MTLSaNWvaxIkTzex8s8PKlStbyZIlffpxPvnkk9akSRPXH9nIGMeOHbPevXtbgwYNnB9Ly5Yts6xZs9qrr75qZuf7Pw8ePNjy5s3rdLkyO//lPnnyZJ+ZNJKCjJEjR1qOHDmsS5cutmTJkkvqj5v0fvr444+tSJEiVqlSJWvQoIHlz5//orOtIHW4jUnk9Xqte/fu9uyzzzrL5s6d69MX/siRI1ayZEnr06ePMxbE4MGD7ZFHHvGZCYkgNPXFx8dbeHi4derUyed77J133rHmzZs7r+emTZssR44cdvvttzstyNq3b2933HGHz5SVx48f92l9ci27sMvfmDFjnGUXniP079/f7rvvPtfxl/jBkr727NmTYkyZOXPmmMfjccLBd99918qVK5eixcRbb71ltWrVcsL2ZcuWWeXKla/J8ZNWrVpltWrVsqZNm9ovv/xiJ06csFGjRlm+fPlSzFzlZtu2bXbrrbdaqVKlLHv27PbAAw/Y2bNnrUOHDlanTh3Oe9PAihUrbO7cufbAAw/Yxx9/bL/88ovVrVvXGjZs6FxkOH36tMXExFi2bNns+eefd/41SxliJZ8eHUhPhCbXgYSEBBsyZIhVr17dGjdu7Hwxbt++3Zo0aWIdO3Z0yib9AFi+fLkzha2Z74k7ByT/JD95/P777y0gIMA+++wzZx8++OCD1rt3b2vVqpXlzZvXhg8f7nS3SfrB/PPPP1uxYsVs6NChF53xCGnj75p3jh492mkhtHz5cqtZs6ZlzZrVQkJCnJYdP//8s0VERNj9999/0e0nOXLkiDVq1MhnumF/f3x89913NnToUOvXr59P4MKP79SXfJ/u3LnTpzVPrVq17KOPPrKdO3da7dq1rWDBgjZhwgSfdV5//XUrWLDgRaeXRupLCqS++OILCwgIcFr6mJ2f1rlbt26WmJho99xzj+XKlcueffZZO3LkiFPmxx9/tMqVK9sTTzyRok/91XgsdhvY9p9c2OUv+UCu+/fvt71799rYsWOtUKFCNn369FStL/y3bNkyy5cvn82bN88OHz5sTZo0sR9//NH27t1rjRo1cmYwOn36tBUsWNCee+45n7Fm3nzzTcubN+8ldQe92m3cuNHGjh3r0zVpw4YNFhER8Y/PL+lzcurUKdu7d69zweHMmTNWo0YNGzt2bNpV/AaRtI+THyvDw8MtZ86czgD5Zma7du2yihUrWr9+/Xzeq59++qm1bNnSPB6PderUiVY/uKoQmlxjLvYD680337SSJUvarbfe6rP85ZdftipVqtgbb7zhLFu6dKndcccdFhMTk2I7HKAuz9mzZ61fv3721ltvWYECBaxJkyZ29OhRS0hIsO7du1u2bNnsnnvu8flS37Vrl3Xq1Mm5wpl89iKkj+Rf7IcOHbLPP//ctm/f7oQRSVeuhg4daoULF7ZnnnnG1q1bZ2XKlLGHH37YzM6fcI0fP96KFSuW4krXhT+y3n77bStRooSZnf9Rd+edd9p9991ngwcPdq4WXiz8uNgPNsKStHX8+HFr3769FSpUyCZMmGAHDx60I0eO2D333GPVqlWzbNmyWVRUlDNo5rlz53wGbG7UqJHTgizp+MprlvaSxoVq1qyZc1L+0ksvWalSpSxr1qx2//3326ZNm5zyCxYscMLsvn372tSpUzOk3v5I/n29c+dOW7Nmzd9OlZok+bHk5ptvtqeeesoZSyA6OtqqVq1qJUqUsE8++SRN6o1Lk/z1bdasmZUpU8ayZs1qt912m+3bt88SExNtzpw5litXLqcb9vjx461w4cL20ksv2Z49e2zfvn12zz33WLdu3XxauJldm62FvF6vzzh827Ztsxo1alhkZKR1797d1qxZ4xxfL/ad6fV67cyZM7Z161abPXu2ValSxWrXru1zPIB/LhxAOLnPPvvMsmbNan369PFZPnToUKtWrZrNmzfPZ/mWLVu40ICrEqHJNWratGn2zjvv2Lfffmtm568wdO3a1YoUKeLTx3X//v02cOBAy5Qpk0VERFjz5s0tS5YsznS2uHKbNm2yIkWKWP369W3SpEl22223mcfjsTFjxpiZ2eeff26VK1d2vjC8Xq8dO3bM+vbta3Xr1k0xBgrBVdpJuup84Xv/+eeft1y5cllERISVKFHCOnTo4Nz366+/Wq1atZxZpk6dOmW1a9c2j8dj33zzjZmd745zKVeh58yZYw0aNLCpU6da2bJlbdCgQdanTx+rWrWq1apVy5lJ5WLvgeTdvi7nCjP8s2bNGitevLjdeeed9v3339uOHTucfT5gwADLly+fvfXWWz7rLF261O6++25bu3atmfF5Tm+zZ8+20NBQu/fee61UqVLOrEVm50/eS5UqZV27dvVZZ+/evda0aVP74IMPzOzaes1OnTplHTp0sMKFC1vFihUtb9689tFHH/3j8SjpWPLvf//bsmXL5gwM+uWXX6b4EXMt7Y/rwYVdAs+dO2cVKlSwTJky+cyEY/bXYNTlypVzlg0aNMjCw8OtfPnyli9fPqtTp851ORvMihUrLDAw0O677z6bPHmyNWjQwCIiIlIck5Mk/94/fvy4jRkzxipVquSMaYQr9/rrr9t9991nTz/9tH333XfO8rvuussaN27sM+HBqVOnrG7duvboo49e9P3J7xRcTQhNrjExMTEWFhZmFStWtMaNG1uuXLls5MiR5vV6bcmSJVa3bl3r2bNnivU+//xzGzdunPXp04c0PZVNmTLFqlWrZgcOHDCz8wM+9unTx/LmzWt79uwxs/NXOIsUKWIVKlSwDh06WFhYmFWrVi1FYIK0kZiYaG+++abPVMBJX8bvvPOOVa9e3b744gszO9/iJ3PmzNa7d28zOz9iu8fjca5Wb9myxR588EGLjIy0p59+2udx/ukLft26dVahQgVr3LixjR492ln+ww8/WLVq1Sw6OtrMUl6pubD7HFLXxQbOnTRpkjVt2tS5L/kPmb1791rFihWtefPmNm/ePNu4caO9/vrrVqRIEYuKivLpOsWPzvSxe/duK1mypA0dOtSOHz9uX3zxhbVs2dLCwsKcVmCdOnWycuXK2ejRo23JkiU2Z84cK1u2rDVv3tw5NiS5GrvjJLd27VqrX7++NWnSxNatW2dr1qyx7t27W6FChXyuxv+T8PBwa9SoUYpZ9q7FlgjXuuTHis2bN9vw4cPt3LlztnHjRuvatavdeuutKVqlfvvtt85sXWbnx/TZtWuXffrppz7d067297O/Tp06Zb/++qvz97lz56xx48bWq1evFAPoJ3/uU6ZMsY0bN9rp06ddp3yH/w4ePGgNGza0okWL2uDBg61SpUpWtmxZZ4rnFStWWOHChe2VV17xOc7MmDHDbr75Zvvss88yqurAJSM0uYacO3fO7r77bnvmmWecZUOHDrWAgACnOfiAAQMsMjLSGeH+wuaYybfFiXzqePLJJy0iIsLM/jopOXTokN1yyy3OaOxer9dWr15tr7zyivXs2dPeffddZ/3r7UTmarVx40Zr0KCB3Xvvvc4yr9drt912m40aNcrMzs9EUK9ePQsNDXX68m/fvt1KlChh9evXt9GjR1uFChWsR48ezlSy/mrYsKF5PB6fq7kJCQnWp0+fFIMuJrUoSTJ58mTzeDw0m09FyUOoCwe87NatmzP14ciRI61Pnz5Wq1Yt69u3rx08eNBWr15tbdq0sZtvvtmqVKlixYoVs9mzZ6f3U8D/e++991IMyvzbb79Z/vz5ne/NnTt32qhRoyw0NNRq1KhhYWFhPgHmtWTRokXWtGlTJ5w3O/+DOUeOHJc0qHjSD8stW7b4XAFGxkpISLCuXbtajhw5rGnTpk7Xm7i4OCtWrJj169fPJxQ7duyYDRw48G/HLbnew/ak89lKlSrZo48+6ixL/rwXLVpklSpVssKFCzuttOE/tzD1o48+sipVqjjB85EjR2zUqFEWFBTkjJ/4xBNPWK1atVKEft9//33aVxpIBYQmV6GLXd1ZtGiRFS9e3MzOf0k++uijzuBKSSeJ69evt7vvvtsefPBBn6m5kiMs+Wdvv/22M4/8xST9mJ00aZKVKlXKmV0haf92797dbrrpJluxYsVFt3G9n8hcDZJ3aXnvvfcsZ86czg+Ko0ePWtu2bW369On22GOPWY4cOaxHjx4pZhX47rvvrEWLFla9enUbOXKkz33+voZ79+41j8eTon/vvffe6wQ6Fw5Ou2jRIqtYsaKVKlXKpk+fTtCWyuLj4613797WsmVL6927tzPeU1xcnDVu3Nhy585trVq1sqefftp69Ohht956q9PKyOv12u+//57ic85xNv0kfR6+/fZby5Url/P5TfrsDx061HLmzOkTDBw9etQ2b97sM9jrtXY8TkxMTDFTyvbt261YsWK2evXqS9pG8vMN3rNXh9GjR1uNGjVs5cqVdvbsWZ+LX6+88oqVKlXKFixY4LPOhg0bLDw83N555530ru5VY968eRYZGZniM5E0rkvu3Llt4MCBtC65TBceH5YsWeL89hgxYoSFh4f73B8bG2t16tSxNm3amNn5i4klSpSwxx9/3LUlHOc1uNoFCFedTJkySZLWrl2r/fv3O8tvvvlmBQQE6IUXXlDJkiX122+/KSYmRhMmTFC+fPl04sQJVaxYUXXq1NHGjRu1evVq1+0HBPCy/519+/bp0Ucf1dy5c3Xu3LmLlvN4PJKk8uXLKywsTJMnT5b01/49cOCAjh49qj59+qTYjtfrlSQFBgamxVOApMTEREl/fZ4CAgJ0++23q1mzZnrmmWckSbly5dLhw4fVpUsX7d69W8uXL9fEiROVP39+xcXFadCgQdq1a5duu+02ffDBB/rxxx/17LPPSrr817Bw4cIaMmSIFixYoFGjRunQoUNasmSJfv/9d7Vp00bS+fdWYGCg9uzZo9atW6t169Zq1aqVVq9erUcffdR57+HKffrppypRooRWrVqlunXravPmzXrmmWe0fPly5c+fX++++662bt2qDz/8UGPGjNHEiROVJUsW5ciRw9lGwYIFVatWLUlyPuscZ9NP0uchJCREFStW1KRJkyT99dnPmTOnTpw4oZdeekkJCQmSzn/2S5curaCgIOdYca0djwMCAhQZGSnpr+PR77//rj///FMFCxb823WTHx+T1uU9m37MzNnvyR07dkxTpkxRy5YtVbNmTXk8HmXOnNk5rvTt21d58+bVW2+9pRUrVuidd95Rp06dVKFCBa1atUrt27dP76eSoX744QctXrxYjzzyiDp27KgWLVo4nwlJeuaZZ1SqVCllzZpVq1at0rBhw3yO3bh0SceH6dOnq2DBgho9erRWrlwp6fwx+JZbbtH69eud8qGhoWrevLkOHz6sw4cPK2/evHr88ceVLVs2ZcuWLcX2Oa/BVS+jUxuclzzB/e6776xo0aJWtGhRy58/v40fP95iY2Nt9+7dFhkZably5UoxUNuMGTPs1VdfNTOzw4cP+zTXxaVLuurYp08fCw8Pt40bN/7jOl6v10aMGOGMWL9hwwabM2eO3XvvvbZkyRL74Ycf0rrauEDyKxbz5s2zDh062MKFC83M7JtvvrHQ0FAbN26cmf01Nenbb7/trJOQkGAvvvii3XXXXc6Uw0lS42p0QkKCDR061EJCQqxWrVquI8tv27bNbrnlFmvVqpVt27btih/zRndhVyczs5MnT1qnTp18ppp87733LCQkxBo3buxT9ujRo5aYmGgzZsywMmXKOE3mkbbcprC8mPj4eOvbt69VqVLF57g7aNAge+yxx6x06dLO2FPXm6Tj0gsvvGANGza8aLkLPwcTJ060/Pnzu86mh7SR/Dvk8OHDdvz4cafFz44dO6xChQrO4MVuFi5caPXr17ewsDALDQ21119/3bnvYuMzXa++++47q1q1qrVq1crpBpLkl19+sR49ejjjleHKjRw50ooWLWozZsyw3377zekOtmTJEitbtqyNHz/ep3zHjh2tcePGN9R7EtcvQpOryO7du2358uV2//3320svvWTr16+3gQMHWsWKFZ3R/p977jkrW7asT3/Mr7/+2iIjI61///4+XXI4SPkn+YlMYmKi3XTTTdavX7+/bcqZtI8PHz5skyZNsptvvtmKFi1qOXPmtEmTJqUoh/Rz8OBBa9GiheXJk8f69etnX331lZ09e9aOHDlizzzzjIWGhtrJkyfNzKxdu3ZWsWJFq1u3rg0dOtQqVKhgxYsXd8YGSitbt261RYsW+XQFS/qReOrUKdu6dWuaPv71LDY21t5//33nNU5y4WcxJibG/vjjD9u/f7+1adPGcuXKZe3atbNcuXI5Qdovv/xiDz30kNWpU8fy5cv3tz9okDr+bgpLt+Np0rL169fbI488YkFBQda+fXtr3LixFSlSJMUPqutV06ZN7ZVXXnH+/vLLL23NmjVmZq7jOxQvXtxmzpzJd1Q683q99tRTT1l4eLjVr1/f7rrrLouNjbWEhAQrXry4RUdHO4OPJ71uybtc7d2712eQ1xuV1+v1+WwnH6+P93Tq8Xq9dvr0aatdu7YNGTIkxX1mZj169LAaNWrY8OHDLTY21pYvX27Vq1d3LugmR1dAXIsITa4id999t3k8HmvUqJHPIJPjx4+3KlWq2MKFC+3AgQP26KOPWubMma1hw4Z2++23W0hIiA0ePDgDa359GThwoE2YMMHq1atnRYsW9Zk27Z/s37/ffvjhB59+8sgYkyZNslq1atnu3btT3Ld69WorX768devWzczOtyL44osvrEuXLvbQQw/Z8OHD07u6KUb4x5V54403zOPx+LQ4GDBggPXs2dPefPNNn3ECTp8+bS1btrSWLVvazp077dixY3brrbda2bJlnR8sI0aMcFonJeH1Sl1uJ9Kvv/66tW/f3vr06ePzWv7Tvh83bpw9/vjj9thjj/n0n7+eZ4TZt2+flS5d2tasWWMrVqywWrVqWVBQkC1fvtwps2fPHmvVqhXjO6Sj5K17vF6v/fHHH9aiRQuLjIy0hQsX2urVq61OnTrWqFEjO3jwoL3xxhtWsGBB++9//+ts4+TJk9amTRtnWuzkruf3tD/4Ie6/5O/Lf7J582YLCgpyWt8mtWpK+o48cOCAjRo1ykJCQqxatWqWJUsW69at2zU3VhRwMYQmV4GkA/2BAwcsW7Zsdtttt9mpU6ec+3fs2GF33nmnPfnkk07ZOXPm2MSJE23IkCEWGxubYlvw3x9//GG33XablStXzmbOnGkdO3a03LlzW5s2bZzpKv/OhV86nMhknD///NNq1qxpUVFRZvbXa5P0mpw5c8Zef/11y5MnT4pZI5J/wfMaXtsqV65sbdq0sXXr1lnt2rWtUqVK9tBDD1lgYKC1a9fOmfJ74cKFdtNNNznvhT/++MNq1qxpHo/HmQEr+bGV90Xq2bZtmz344IPO91jS52/Pnj0WGRlp4eHhNnLkSGvatKk1bNjQXnvtNTO7+Hdd8uXJ/38jnLh/+eWXljNnTrv77rstICDAunfv7nP/ypUrrUCBAnbvvffS5S+NJD8PuLCrzMqVK83MbM2aNVa9enUn0N+2bZsVLVrUatasab/88ouZmbVo0cLKli1rTZs2teeff96KFi1qkZGRPlPsApfL6/X6HB8vJTw5cOCAFS9e3Glpkvx7MC4uzo4dO2ZmZr/++qvFxMTYzp07nfv5bYLrAaN+pSG3Qb7cBAQEKDExUTfffLOio6O1bt06/fLLL879xYoVU5YsWRQbG+sMxPTAAw+oR48eeuGFFxQaGqrExESZGQO5XaKk18bMnGXbtm3Tnj17NH36dHXs2FEzZ87UrFmzNHfuXC1atOhvX89z5875DGKVmJjoDEKI9BcSEqJz584pJCRE0l+Dcya9JsHBwWrVqpVKliypzp07+6wbGBgor9crM+M1vEYlvd4TJkzQxx9/rHfffVc1a9bUqlWr9N577+mLL75QXFycXn75ZUlS7ty5FRQUpG3btkmSPvroI1WoUEGzZ89Wp06dJP01CB7vi9S1efNmrV27Vq+88oqkvwZjXbBggXLlyqWtW7fq2Wef1ejRo/Xrr79qxowZio+Pv+h3XdL3qXR+YEH7/wE3r7VBXi/HuXPndOLECYWEhGjr1q3OgLhnz56VdH4Q6piYGM2dO1clSpTIyKpel+Li4vTTTz9JOn+O4fF45PF4dO7cOfXu3VsRERHatm2b1qxZoxw5cigsLEz33nuvatSoofvvv19fffWVypcvL0l65513NHz4cIWGhmrt2rXq37+/li5dqjJlymTkU8R1IDExUR6PRwEBAVq7dq2qV6/uTGTwd4OxhoSEqHXr1nr//fe1Z88e53vwzz//1OzZs7VmzRpJUpkyZdSoUSOFh4crMTFRXq+X3ya4PmRoZHOdSp6oJm8C/ncJbvL78ufPb+3bt7f9+/c7y1q2bOl0Jfi7x8Pfu3DckuTee+89y507d4r7mzdvbrVq1fJJzZMkT9pPnTplU6ZMSeUa48LPTfLP1MUkJCTYgAEDrGTJkk4roaTt/PLLL04z/88//5wBPa9TSZ/fBx980Dwej/Xv39/n/smTJ1u1atXsm2++scOHD1uHDh0sW7ZsVqFCBcudOzfvizSW9Hn8888/bdiwYVa2bFlbtWqVmZ0/rnbr1s1GjBhhZmZPP/205ciRwzp16mTbt2+/6DZvhNYkf8fr9dr333/v/E2Xv/Rz9OhRe/DBB+2pp57yWT5r1izr16+fde/e3enWsHTpUsuWLZtlzpzZHnjgAVu/fr1TftmyZbZkyZKLjstxo7/HkTpOnDhhbdu2tYCAAHvyySd9Wrf/nTVr1thtt91mVapUsQkTJtjChQvtjjvusBIlSjgtqYDrFdFfGkhKVJ977jndeeed6tatm3788UenVYNbi4WkqxGS9Nprr+ndd9/VAw88oKlTp6pDhw76+uuvnelIL/Z4cJd01VH66yrmmDFj1Lp1a3Xt2lUffPCBJKlKlSqSpHnz5kn66+pcly5dtHbtWs2fP19nzpyRdP6KniW74jxmzBgVKlRI33zzjY4cOZIuz+tGkXTl47///a8kKXPmzJKk+Pj4i66TOXNm3X777cqTJ4+6d++us2fPKjExUbt27dKgQYO0fPlyJSYm6q677lLr1q3T/kkg1Viy1mGXUm7ChAkKDAzUqVOn9Oeffzr3N2/eXGfPnlVsbKzy5MmjiRMn6u2331b37t0VFxfnvC8u9fFw6cxMHo9HXq9XISEhuvPOOxUeHq5Ro0ZJOt8ibMeOHfr2229VpEgR/fjjj/r000/19ttvq3jx4tq4caMOHjzobM/r9fq0Jpk6daqqVq2qH374IUOeX0bxeDyqV6+ezEyJiYkKDAxkGs809vXXX0s6P4X1pEmT9Prrrzv37du3T1988YUmTpyohIQEVapUSZJUoEABNWvWTJUrV9acOXNUsWJFSdLRo0c1YcIELV261DnuJL1+lzvFPXChH374QYUKFdLOnTu1a9cuTZ48WVmzZr2kdatV+7/27jygpvz9A/j7dku0UUJNSUpISpYslRBpjLIMjcrIWMZEJdlr7IVk+9r3bez7MpEiSySjSEOhGKU0IaJSqHuf3x/97tEly8yo0PP6Z6Zzzznuufee7TnP53laYvv27TAyMsLmzZvh7e2N2rVrIz4+Hm3atCnnd85YJau8eM3X6+HDh9SlSxeytLSkwMBAsrW1pW+++abMCtLv0q1bN1JSUqJp06bRsGHDePzxP5Samkq2traUmZlJRK+fzmRlZZGtrS0ZGRnR0qVLaejQoWRhYUFz5swhiURC7u7u1KFDB7l1yVrDqqmp0Z07d+Se/Bw7dowaN25MTZs2pT179lTcBlYhUqmU1q9fT9ra2hQZGUkbN26k9u3by3USeHN+opKn1WFhYaStrU0NGzak7t27C+P9Hz16VJGbwD6Rd9WoeNfTdNk8AQEBVK9ePbp06ZLwWlFREWlpacm16yyN65aUj3c9KV+xYgU1bdqUdu3aRUREu3btIpFIRDNnzpSb7/bt2+Tr6yt0tiq9vvPnz1PLli2pfv36tHHjRs7CZOUqLS2NRCKRXCH+v//+m/r16ycU8w8NDSUjIyMaPHiw3LI7duwgLS0t+umnn2jjxo20bds2aty4MVlbW8tlnjD2qcXHx1OLFi1o1qxZRFTSgXPy5Mm0cuVKuXNkWedV2TFVIpFQXl6eXEY8Z0Gxrx0HTcpBREQEGRoaUmpqqjDNx8eHrKyshMrn7zq4yKZfuXKFRCKRcAFJVHIRz6m2HychIYEaN25MP/74IxG9Pvjv2rWLHBwcqLCwkIhKClbp6elRw4YN6cWLFxQeHk4NGzakAQMGUHh4OEVGRtK3335LV65coWPHjgnrz8zMJDs7O6pTpw4FBQV9dGoj+3eysrLI3t6eatasSfXq1aOtW7d+9LJJSUm0c+dOmjp1KkVFRQnTeV/6Ml29epV69uxJP/74I/n7+79333tz2GP37t3p1KlTlJ+fT5s2bSJjY+Myg2/82/j0SgcwsrKyaO7cuXT06FFhuM2dO3fI3d2dbGxshCF45ubm5OjoSGFhYfTo0SOKjo6mjh07kr29vVw77tLtoidNmiS0amWsPL169YrmzZtHNWvWFALxly5donr16gkFpJ89e0YBAQFkaGgoFwyRSCR0/PhxsrGxofbt25OZmZkwHI2x8iSVSikkJIQaNmxI7du3J2NjY3JyciIjIyPS0NCg9evXf/ChQekixxKJhAPUrErgoMm/9L4DxG+//UZNmjSRi8Devn2bBg4cSDY2NpSfn09EH3466ubmRpaWlu8dw83KVlxcTJs2bSJ1dXW5NpWTJ08WOgp4e3uTmpoaDRkyRLgAl0gkdO7cObKwsKBGjRqRuro6TZgwQW7dEomElixZQp6ennKBMfbpvHnCvnTpEhkaGlKtWrWEjK1/+1SjdIs89mU5fPgwaWpq0pAhQ2jSpEmkoaFBPXv2FIJhZR2XZb+lPXv2kEgkooYNG1Lfvn1JS0uL5s+fX6HvnxEdOXKExGIxWVhYkIGBARkYGAi1TA4cOEDNmzen4OBgIiK6fv06ffvtt6Sqqkq2trakrq5Onp6ecvvv6dOnSUdHh5ydnbmzCKsQpX9/WVlZ1KpVKxo4cCARlbQvX7FiBWloaAg1TM6dO0ddu3YVHuIQyR+rsrOz6eXLl2Wun7HykJmZSc7OztSvXz+6deuWcF8ybNgwsrGxodOnT5e5HP82WVXGQZP/KDk5Wfh/WRBk8+bNZGhoSAkJCXLz7t69m6ysrGjVqlXvXF9xcbFwkf/s2TMSiUQ0ffp0PlB9pNKBqIsXL5KrqyvZ2toK04YNG0YdO3YkHR0dsra2prNnzwqvXb58mTIyMoiIKDc3l5KSkujx48dy65dd6BQUFJTnZrD/l56eTkQln/utW7do/PjxZGlpSffv3yeiD5/AZa+XTillXy43NzcaMmSI8PfVq1epe/fu1KNHDyHj5H1ZIs2aNaNOnTpRTEyMcJH4oWXYp3HhwgX64YcfaOrUqbR3714iIsrIyCAHBwdq164dJScnU35+Pvn6+pK5ublQePvJkyeUkJBAYWFhlJaWJqxPdp68efOmEHRhrDyUDmiUFhoaSlOnTqVffvmFRCKR0MI8OTmZunXrRl27diWikuPL//73P2rWrBkdPXqUiOQfDMiOP1y4l1WkqKgoSkpKIqLX10aZmZmkra1N27dvJ6J3/zbXrFlDV65cqeB3zFjl4gqi/9KJEyfQsGFDjBo1CsnJyQBeF+oaPHgwnj59isOHD8st06VLF2hpaeH69etCkVEZqVQqFG5TVFTEnj17kJ6ejuPHj+PHH3/k4l9liIuLw+PHjwFArmja+fPn0bp1a8yaNQs3btxAdHQ0tm/fDgDo06cPzp8/D3d3d0RHR8POzg4AkJmZiXXr1gmFA9XV1WFqagotLS2hnTPwuuhujRo1KnRbq4LSBZJ3794NIyMj9OjRA66ursjOzkbjxo3RtWtXVK9eXWgV+64ih28WhszLyyv/DWDl6vnz58jOzoaWlpYwrUWLFvjpp5+QlZWFxYsXv3NZWZHtgwcPYuXKlWjfvj1UVVWFfZuLZX469P9FSN90584dHDt2DOvWrUOrVq0AlLTA3bJlC+7evYuDBw9CVVUVffr0gZaWFgIDAwEAmpqasLCwwLfffgsDAwOhhaWsCHeTJk3QunXrittAVmVIpVI4ODhgwYIFctOLioowbNgwuLu7QyQSITc3FwoKCvD19QUAGBsb45dffkFcXBz2798PkUgEBwcHNG7cGP7+/gAg17Zcdvzhwr2sItna2sLU1BRAybWtVCqFmpoaXr16hQcPHgjzlS4qfebMGVhYWCAwMBBPnz6tpHfOWCWp3JjNl0cqldKhQ4eoRYsW1LNnT2rQoAGtXLlSeGogG4sdHBxMmpqadP36dbno7NChQ6lTp05y6yz9xCEpKYk6d+5MtWrVot9++638N+gL9ezZMzI2NqbvvvuOiF5Hw+/cuUPNmjUjHx8fSk5Opt9++42srKyoQYMGQtaBra0t2dra0qZNmygtLY3Onj1LNjY21LFjR0pMTKy0bWJEKSkplJaWRvb29rRq1Sr63//+R02aNKFu3boRUUl70lmzZpGpqalce7vc3FwiKvkdlN6frl69SlZWVjR8+PCK3RBWLuzs7GjEiBFymV5Pnz4lHx8fsrW1FTKTynpaW/p3wRlH5a+goIDu379PL168ICKie/fukaenJ2lqagqfv6y2lK+vL1lYWAjT/P39qVWrVnJDXBmrDBEREfTs2TO5abdu3aKmTZvSoUOHiKjk2LJz505SVFQU6m1lZmbS4MGDyczMTFhu+/bttGnTJiLi7Db2eVq3bh1ZWVkJTRRk7t+/T7169aJatWrRlClThGsuxqoSDpr8Q1KplE6fPk1TpkyhgoIC8vDwIBsbm7f6k0skEjI1NaVevXrJdb5xdXWlX3755a0TZmFhIQ0fPpzU1NRoxIgRbx2wmLzi4mI6cOAAVatWjS5cuCBM37t3L6mrq9ODBw+EaSdOnCAjIyMaN24cEZVc8IwaNYqqV69O1tbWpKmpSV5eXnwRU8HevHF99OgR1alTh3R1dSkgIECYHhsbS9WqVaMtW7YQUUmRZCcnJ2rXrh1duXKFunfvToGBgXLDLfLy8mjw4MGkrq5O3t7ecr8H9uVav349qamp0Y0bN+Sm7927l8zMzISOKqWVHsLFwZLy8ebnumDBAtLU1KSWLVtS//79helnz56l2rVrU2BgIBG9/m6WL19ODRs2FIppZmZmvnNIBGOV4fTp08LvPD4+nkQiEd27d094PS8vj4YNG0b6+vrC7/rMmTNUo0YNue46jH1uoqKi6NSpUzRw4ECqXbs2LVu2TO718ePHk4qKCrm6unInT1alcdDkX8jPzxeedN67d48aNGhAU6ZMEZ5GyJ5oxsXFUdOmTcnS0pIWLlxIvr6+pKmpKdeFhajkglFNTY26detGf/zxR8VuzBdGIpEIFyQvX76kH374gVq1aiW8fujQIWrevLncgf3Zs2c0evRo0tDQEC5yXr58SSkpKRQTEyPUxyDiIlcV4V2fcVFREa1du5bEYrEwnlbGx8eH6tevL+x3ERER1KFDBzIwMKBevXoJ7R2JSrK8tLS0yNHRkcfcfmUkEgmZmJjQkCFD5LJNJBIJKSsrU2hoqNy00oHQ5cuXU926dSkyMrJC3/PX7s2ASUZGBvXo0YMOHDhAK1eupDp16pCXlxcRlZw7f/31V1JWVqa4uDjhnOnk5CRXq0aGj8esor153CAiio6OJpFIRBEREURUUi/NwsKC1q9fLzff/v37SSQSCcXKnzx5QuvXrxcKwjL2OYqMjKTGjRuTs7OzUEtKJjExkby8vCgsLKxy3hxjnxEOmvwHsqE4M2bMIFNTUzp+/LjwmuxC8vLlyzRmzBjq0aMH2dnZvRUUycjIoPHjx/NQnH/o8uXLNGHCBAoKCiKRSES7d+8mIqKjR49Ss2bNaNOmTXIXPvPnzyeRSCT31LO04uJifgpdwUJDQ+nnn3+mqVOn0uXLl6m4uJjy8vLIysqKnJyciOh1CnN6ejrp6urSlClThOWfPn0qFO4lKrlAdXd3J319fdq/f3/FbgyrMEePHiWRSEQbN24UjsEJCQlkYmJC586dIyJ6q7uKubk5GRkZ0ebNmzmj7BN4s8XkjRs3aPDgwTR79mzy9PSkJUuWCK8dPHiQxGKx0OHo+vXrZGlpSdWrVyd3d3dq06YN6evry3U5Y6yiZGVl0Y4dO+QyFYneHj7Tp08fsra2JolEQk+fPqXevXvTwIED5YoTb9q0iapVq0YikeitbGE+7rDPlVQqlWvhXvp6mH+3jL3GQZP/oHRVaQsLCxo2bJhwopRdzMuUHv/35gUn36z/M8HBwaSkpESjR48mb29v0tDQID09PSHDx9nZmb777juKiYkRlpkyZQo5OTlRgwYNhLoHrHLk5uaSq6sr1apVi/z8/KhTp05kb28vpOz//vvvJBKJ6NSpU0T0ej9bvHgxiUSit74/2f7z8uVLunr1Knc2qgK8vb3J3NycunfvTr/99hu1bt2aunTpIld7ID09nfr06cNjsD+B4uJimj17ttyDAaKSffPYsWPUqFEj6tSpE9na2pJIJKIVK1bIzefo6EgdO3ak4uJievXqFa1bt440NTVpzZo1QjCFscqwatUqEolEckE7f39/8vX1pTVr1gjXFTdu3KBq1arR6tWriaikG2Lbtm2pf//+dOvWLbp16xYNGDCANm7cKDwEk527+MaTfSk4u4+xd+OgyX8kO6Hu2LGDDAwMaOfOnXTlyhXy9PQss6goH5A+XlkXGk+ePCFra2sKCQkRpp06dYrq1q1LY8eOJaKSGhjdunUjPT09mjZtGnl4eJCZmRlFRUXx51/Byvq8w8LCyMrKilJTU4moZCx48+bNydjYmO7fv0+FhYU0YMAAMjc3l1suLy9PKKLHqraXL1/S4cOHqU+fPmRra0ujR4+Wez02NpZ0dHTo+++/5zHYn8Bff/1Fw4cPl8vsunTpEnXr1o1cXV1p48aNRFSyj1paWpKTkxNlZ2cL8/75559UvXp1oS7R3bt3ycnJSa4d/JsPGhirKBYWFtS/f3/6888/qV27dmRubk7u7u4kFovJ1dVVGOY5ZswY+uabb4TaO0ePHiUjIyNq1KgRKSsrU69evSgnJ6cSt4Qxxlh54aDJ/ytrHOs/ZW9vT7Vr1yaRSEQ9e/bkk+e/9GYmDtHrAMqzZ8+oRo0acrULXr58SfPnzyclJSX666+/iKik1szkyZOpZ8+eZG9vT9evXxfm58BJ+Xtzf7pz547w/zNmzCBfX18iIlqyZAnp6OiQnZ2dXEHf+Ph40tDQoIULF1bYe2ZfHolEQs+fPxf+lgWxMzMzuRPWJ/LmeVFWVPnFixdkYmJCioqKQq0HIqLz58+TSCSiPXv2yB3Hhw8fTqqqqkLGz5EjR6hu3bpCwIUzLllFkx0vzp49S2KxmCZOnEheXl5CEeKIiAjq0qULubq6ElFJlmTdunVp4sSJwjoePHhAFy9epLi4OGEaZ5YwxtjXR6GyWx5/DiQSCRQUFCASieR6k3+sgoICrF69GpcvX4aRkRHOnDmD0NBQ1KpV69O/2a+cVCqFgoICFBQUcPv2baxYsQIXL17E48ePAQDZ2dlo3bo1Lly4ICxTrVo1dO7cGWKxGLNnzwYA1K9fH3PnzsXu3bsRGRkJMzMzSKVSAIBYLK74DatiZPtTZGQk2rZtC09PT9y9excAcPfuXcTExKBt27ZYvHgx5s2bhzNnzqBDhw7IzMxEeno6LC0t4eHhgRMnToCIKnlr2OdKQUEBKioqkEqlICIoKioCAHR1ddGsWbNKfndfPolEApFIJOyDc+fOxdChQ5GYmAhlZWUEBgZCQ0MDaWlpwjI2Njbo06cP5s2bh4yMDGH6vHnzsG3bNqirqwMAOnToAHt7e6xduxZAyXfJWEVSVFSEVCqFnZ0dXFxcMH/+fKipqaFatWoAAAcHB7i4uODWrVsIDQ2Furo65syZg4ULF+LKlSsAgLp166Jdu3Zo3bo1iEjYZxhjjH1d+CoFJTfRDx8+hIuLCxwcHODg4IDAwMCPXv7JkyeYPXs2pkyZgkuXLsHOzk44ebJ/RkFBAa9evYKnpyesrKywefNmDBo0CH369AEAGBkZwdTUFAkJCbh48aKwXGFhIdTU1LBx40acPn1amK6qqgrgdWCMlR/Z7112gxUSEoL+/fvDyckJkydPhrKyMgBg1KhRuHz5MszMzHDr1i14eHhAJBIhKysLq1atEi5GQ0JCEBYWxheg7INkQTr2acj2ZVmAOTs7G0BJMCozMxPHjx8HAAwYMACWlpYIDQ1FcnKysPyaNWuQkJCAzZs349WrVwAALS0t4TgOANra2pg/fz5iYmIqYpNYFfOxwXbZfMuWLYNYLMbz589RWFgovN6jRw8UFxfj2bNnAIAhQ4bAwsIC169ff2tdIpGIH8owxthXiu8iAVy7dg12dnYAgPnz52PYsGGYN28eZsyYIZwo30dfXx/37t3D+PHjAQDFxcV88vyX8vLyMH36dKSkpCAyMhKxsbGIjIxETEwMgoKCAABDhw7F8+fPMXnyZMTFxSE9PR179+6Fn58ftm7dijZt2ry1Xv4uyo/solMsFkMikeDJkyd49eoVDh06hFmzZmHatGmwt7fHN998AwAwNTWFk5MT4uLicPXqVWRnZyM1NRXjxo1DaGgo6tWrBwCoUaMGgJL9iTFWcWTHyw0bNsDS0lLI4Pvpp5/QvHlzRERE4Pz58wCA6dOnIzY2FuHh4SgqKgIA1KlTBz4+Pvj777/fe+zV19cv5y1hVZUsiHrt2jXk5+cDKDuQIjtvaWtrY+LEidi7d69cQERfXx/3799HTk4OgJIAbUxMDDw8PCpgKxhjjH0uqlTQRCqVCkM0Sjt9+jRMTU2xd+9eODo6wsDAAAUFBcjIyHhvtkjp9YlEordSxNk/p6qqCkNDQyxfvhytWrXCyZMn0atXL6irqyM4OBjx8fFo3749Zs2aBalUij59+qB58+aIiYmBi4sLBg4cCHV1dR7SUUGISLg4PXLkCBo1aoR9+/YhJSUFsbGx6NixIwD5fUVdXR2bN2+GhoYGnJ2d0bt3b7Rp0waPHj3C77//jvbt28v9G7w/MVaxHj16hO7du2PatGkYOXIkHB0dhQcI3t7eePz4MUJDQ1FYWAg7Ozs4Ojpiz549iIuLE9axaNEirFq1igPWrFK8evUKP//8M2xtbbF3714AeGc2miwLdfbs2SAiTJkyBadPn8bz58+xbds2aGpqwtraWphfWVkZVFITsPw3hDHG2GehytyNSCQS4eItOTkZf//9Nzp27AgFBQWcOXMGrVu3xuPHj9GjRw9kZGRg8eLF8Pb2LvOCj4gglUqF1+7du4datWpBQ0OjQrfpa0NEUFBQgJubGzQ0NDBz5kxs2rQJgwYNwujRo9G9e3fMmTMHO3bsQMeOHREeHo5bt27h6dOn6Ny5s9x6OFW/YohEIty8eRO3b9/G9u3bMXHiRAwdOhTR0dEwNjbGzZs3YWlpKTc0Kj8/H7Vq1cKRI0eQmpqKv/76CwYGBmjXrh2A13VtGGOV4/Tp08jPz8f169ehqakp91rbtm1hb2+PqKgohIeHo0+fPpg1axZMTEwQGRmJNm3aQElJSTgOlz73MlZRCgoKEBUVBWNjY5w9exadOnWCkZFRmdcHIpEIxcXFUFRUxPLlyzFgwACkpKTA0tISZ8+ehb+/P1q1avXWMowxxqqOKnNnIhaLkZ+fj4EDB8La2hphYWFISkoCAFhaWmLRokUwMDBAhw4dEBcXB19fX4hEIhw7dgxPnjwR1lN66E1eXh4GDhyIFi1aCEUu2b8nuwjR0NBAamoqjh49imnTpiEwMBBqamrQ0NDA/v37cfjwYQAlwzcsLS2FgIksK4gvZsrPm3VLgJIaJS4uLsjLy8Pw4cOhpKSEDh06QEVFBVFRUUhPTxfmzc/Px/r165GdnY3atWujdevWcHFxQbt27YQ6QBwwYaxyyPbrJ0+eID4+HtWqVcORI0ewcuVK+Pj4YObMmQCAcePGoVq1ajh69CgyMjKgr6+P0NBQTJ48GUpKSgBeH4c5YMIqmizwbmVlBQ8PDyQkJAjXDaULG5cmy2h0cXGBqakpDAwMMHHiRLmh15xZwhhjVddXe3fy5rCau3fvwt7eHjk5OQgPD8eECRNgamoKADAzM4Oenh5+/vlnLFmyRKi9cPPmTSxatAjx8fHCemQn1rlz58LAwADPnj1DZGQkWrRoUUFb9uUpa0jUh6SlpSEuLg4ODg4AgAcPHsDY2BidOnXCpUuXylyGL87Lz5udh16+fCm8tmTJEmhoaAiZQkBJ+vL48eNx7tw5+Pv7IzY2FomJiZg4cSKWL1/+VpBR9vSPv0PGPi1ZZuTHkAU6bGxs0Lp1a9SpUwfTpk3DqVOn8PfffyM4OBiBgYGoV68enJ2dcfr0aeHhQ9euXYVuJIxVJgUFBfz999+Ij4+Hj48PmjdvjuPHjyMlJQXAuwMnsvpZBw8exMqVK9G+fXuoqqpCIpFwBitjjFVxX+3wHNnNlyzlMjU1FQUFBdi0aRPq1auH3NxcEBGKiorQo0cPnD59Gjt27ICBgQHMzMxw8+ZNhISEwN7eXi4gsm/fPkyZMgVisRgbN25E3759K2sTvwilU7Nzc3PlhjC97yKkZcuWaNSoEfr27Qt7e3vs378fPXv2xPHjx4UuLKziyIIhe/bswdq1a6Gurg4LCwt4e3vD3Nwcbm5uQnHITp06AQDc3d2FNtBubm54/vw5jI2NcezYMTRu3Fhu/XwxytinJzvGikQiPH78GDVq1ICKisoHlzM3N8eRI0dw48YNNGzYEFKpFPr6+nBzc0N0dDQAwNfXF127doWlpaXcspwpxj4HmZmZMDExgVgsho+PD8aMGYMFCxYgPj4eGzZsgLm5+VvLKCoqori4WDg/yTJWOJjPGGPsqw2aFBUVoX///ujduzeGDh2K+/fvQyQSYe3atXj+/DlycnJw/vx5aGlpITg4GMuXL4empibWrl0LFRUVvHjxAgsXLoSrq6uwzosXL+KHH37AvHnz4O3tLXT3YO8mFovx4MEDjB8/Hg8ePECjRo3g4OCAvn37vjdooqGhgX379mHx4sWIiYmBn58fvL29hde57kXFKi4uxujRo7Fnzx5MnjwZRUVFOHnyJMLDw3Hx4kVMmzYNoaGhOHLkCMzNzaGlpQWgpCXpt99+i6ysLBQUFKBly5YA+PtjrCKIRCIUFhbC19cXhw4dwrZt29C9e/ePWlZLSws2NjbCvnrt2jXcu3cPQ4cOBVBygykLmPBTePa5yc/Px8OHDwGU1OEpKCjAli1b0KRJE9SvX/+t+WUPeGTZUgoKCnyOYowxJvgqzghlpVkqKSlBWVkZ8+bNQ05ODvr27QtXV1ds3LgRt2/fRv369TFs2DDhvwAQGBiIxMRE7Nu3D0lJSULARDbUp3379nj06BEmTJjAAZOPdPr0abRo0QLPnz+Hi4sLqlevjgEDBuDChQvvvSAhIlhYWGDt2rWIiooSAiay74IvZspPWen1aWlpiI+PR3h4OMaPHw8/Pz/Url0bly5dwh9//AEtLS2MHj0aR44cEZ5Ey9SsWRNNmjQRAiZct4SxipGRkYGffvoJKSkpUFRUxNq1a/H48eOPXv7evXtYv349fvzxR1hbW6Np06b48ccf35qPAybsc5Oamgo7Ozts2LBBGD5qYmKCDh06COc42dA1IhKySVasWAFdXV2cOnWqMt8+Y4yxz8xXceciu2CLioqSC6CsXbsWWVlZWLJkCVRVVfHrr7/i2rVr2LdvHwICAjB27FgYGBjA3NwcL168EJ40GBkZAXh9g146NbN27doVuGVfjrLaORMR9u3bhyFDhuDAgQNC+7/i4mIcOXLko9o5KykpCZXtAa5bUp5kF49lBTTu37+P/Px8tGzZUqj78/TpU1y8eFHoejN69Ghoa2tjzZo17y2MzN8hYxVDSUkJJiYmCA4Oxr59+3DgwAGcOHHio+uOEBFyc3MhEolw7tw5bNiwQWi3ytjnTFVVFSEhIZg6dSpCQkKQkJCAUaNG4ezZs9i5cyeA1xmPIpEIZ86cgYWFBRYtWoSQkBB06dKlkreAMcbY5+SrGZ4TExODzp07Izw8HA4ODiAi1KpVS+i+0q9fP5ibm0NNTQ3Z2dlQVFTE4cOHsWfPHowZMwbVq1d/a518c/dxStctSU1NhYKCAgwMDCASiRAbGwt/f3+kpqbC1dUV9+7dw9KlSzFy5EhhmdKp3W+2c3748CHq1q3LTzLL0ZupyBEREYiKikKbNm3g5OQERUVFZGdno3r16jA0NET16tWxatUqDBgwAABw+/ZtPH78GO3atcO4ceOwd+/et9qUMsYqXr169TBmzBhoa2sDAPr374+5c+eiffv2MDQ0/ODyDRo0wMiRI6GqqgrgdRYaZ4qxz13Xrl2xdetWdOnSRSjuP3ToUFy4cEGoUycWi5GRkQEfHx+cOXMG3t7emDhxItTV1SvzrTPGGPsMiegremT0/fff4+HDh4iIiJArdteoUSPY2dlh5cqVyMnJwezZs3H16lXcvn0bc+bMEcZos38vLy8PXl5eiIyMhL+/P1xdXaGoqIjBgwcjPT0dycnJGDJkCAICAqCrqwuJRIKIiAj06NFDWIesaK9sfSNGjEB4eDjS09OFi3ZWfiQSCby9vbF9+3ZYWFggMTERFhYWWL9+PbS1tdGxY0doa2tj+/bt0NPTA1DynQUFBSE/Px/z58/n4BZjnyFZYPvJkyfQ0dHBrFmz4Ofn996i2qWPx2X9zdiXpKw6WnFxcXB2doa1tTVCQkJgbGxcSe+OMcbY5+6relw0d+5cxMbGYseOHXLThw4dioMHD+L8+fPQ1dWFg4MDfv75Z2RlZQkBE26T+PHeHFYTHx8PS0tLPH78GLt370bPnj2hpaWFWrVqwczMDOnp6ViyZAmWLVsGXV1dAMClS5ewcuVKoV0l8HY75/z8fERERHDApAIEBQVh3LhxEIvFSEhIwJkzZ5CQkICsrCzMnj0bNWrUEPaZBQsW4OrVq7hy5QpcXV2xbds2dOrUSa6N4/uGXjHGKpZYLEZxcTG0tLQQEBCARYsW4ebNm8LrsvOfVCoV2qvKjscLFy7EhQsXOGDCvmilAyay37uenh4iIyOxf/9+Dpgwxhh7r68qaNKkSROMHj0aM2fOxN9//y1MT01NxbNnzzBp0iS8fPkSvXv3xuDBgwFAqJXB6cYfr/SwJalUipiYGDRq1AihoaGwtbVFgwYNhM/Ty8sLOjo62L9/Pw4fPowbN25g2bJlcHV1haGhIfT19YV1HThwAE2aNMG2bduwceNG/P7772jTpk2Fb19VI5FIUFxcjKVLl+LmzZvQ1dWFWCyGgYEBpk+fjtjYWBw/fhw+Pj4YO3Ys9u/fj6FDh6JXr14oKipCdHQ0nJ2dAbyuL8RD2xj7vMj2yRkzZkBJSQlr1qxBZmYmNm7ciPXr1wOAUBBTVuOhZcuWmD179j8qHsvY5052faKrq4tmzZpV8rthjDH2Jfjshuc8fvwYmpqaUFBQQFFREZSUlP7R8gUFBWjcuDFat26NkSNHorCwEPv27cOECRNQWFiIDh06lNM7rzpyc3Ph4eGBpUuXwsDAACNGjMCFCxcwf/58JCQkIDs7G1FRUejUqRMmTZqEe/fuYe7cuTh79iy++eYb5ObmYu7cuUJNDCLC1q1bMWLECAQGBnI750qQnp4OV1dXqKmpITw8XC4Vv1WrVujcuTMWLVoEAHj06BHy8vJQVFSEJk2aAJCva8MY+zzJ9tM9e/bA1dUV9erVQ15eHrZv347evXsDADIzMzFy5EihaxnXeGCMMcZYVfdZBU3WrFmDjRs3YtmyZWjbti0AICcnB0lJSTAxMUHdunUByBcOLU02ZjUqKgrjxo3DgwcPUFBQgKCgIHh6er53WVa2ssYBZ2RkoHPnzjAxMUFYWBgePHiAgQMH4vLly+jcuTMMDAwgkUhw+fJltG3bFkuWLAERISsrCxkZGbCysnpr/Q8ePIBIJBK+Y/bvvRnAKOs7LMvu3bvh5uaG8+fPw9raWthXZDdThw8ffmsZInpnxx3G2Ofn2rVrmDFjBg4fPozJkycjKChIeG3ChAlYsWIFevfujaCgIB6ywBhjjDGGSu6e82YAo1OnTggKCkJ4eDjMzc2xevVqTJo0CUZGRnjx4gUmT56MX375Raid8GbwQ3bjZmdnh1OnTiE5ORktWrSQG4vNAZN/RvaZXrt2Debm5iAi6OnpYdGiRejTpw8iIiLQvXt3bNu2DYqKiqhZsyZEIhEUFRVhb28v94RSV1dXqGkiy2SQrb9evXoVv3FfKVnA5I8//oCZmRnU1NQ+arkePXqgR48eGDVqFDZt2oTmzZvj3r17SEpKwtSpU8tcRiQS8T7F2BeiqKgIK1asQGpqKu7cuYMGDRoAKAm0PnnyBNWqVcP+/fvlCnQzxhhjjFV1lZZpUvrpd05ODtTV1aGoqIiAgABERETA09MTBw4cgJeXF0xMTLBs2TKcOnUKAwcOREBAwHufnr8ZUOGq//+eVCrFmjVr4OXlJQydAkq+M09PT1y9ehW3bt0S5n/27BnU1dWxZcsWzJs3D3PnzkXfvn0r6+1XSbt27cKYMWOgqqoKRUVFfPfddwgJCfmooW7R0dFwcnLCq1evMHToUGzbtg02NjbYunUrtxFm7CuQm5sLDQ0NACXBEpFIxJlijDHGGGPvUanDcxISEuDn54eWLVti/Pjx0NXVxYsXL9C6dWvk5OSgb9++WLFiBQAgPz8fixYtwvr16xETEwM9Pb0ys03eHJrAtRb+uytXruDnn39Gq1atsG7dOmF6TEwMunXrhgULFmDkyJFITEzEnDlzkJqaiuTkZCxYsEAouMsqxuXLl+Hu7g4/Pz/Y2tri4sWL8Pb2hp+fH3x9faGjo/PeIWqFhYUICgrC0qVLER0djcLCQrRr1w4AD21j7GvC50bGGGOMsY9T4Y+XZDUQgoOD0bFjRxgbG6Nfv35QVVWFRCJB9erVMWPGDGRlZcndoKmpqeHbb7+Fjo4Odu3aBQBvZZMAJUMT8vPzMWfOHOFv9t+Ym5tjxIgR2L17N+Lj44XpFhYW6NSpEwIDAyGRSGBmZobmzZvDxcUFjx49EgImn1HZnK+W7DO+cuUKJBIJ+vXrh+bNm2P48OH43//+h0OHDmH//v0A3j9ErUaNGhg8eDBUVFSwf/9+tGvXDsXFxcITacbY14HPjYwxxhhjH6fCgyYikQjPnj3DiRMnsHr1aqxbtw7W1tZQUVERLuJcXFzQtWtX3Lx5E4mJicKyFhYWyM3NhbKysjBNIpEAgDD8JiQkBA0aNMCZM2e4TeInoqSkBEdHR9ja2sLPz0+YLhaL8erVK2RlZWHIkCEAgEmTJmHMmDEAXgey+Gb703lXAEr2Gd+9exdaWlqoU6eOsG94enrC3NwcR48exbVr1z74bzRq1AgBAQEICQlBWloaFBUV+QaLMcYYY4wxViVVykDmQ4cOIS0tDe7u7jh+/Dh69uwJV1dXTJ8+HQkJCQCAgIAAJCUlYefOnZBKpQBK6mUQEWrVqgWg5AZSdjN37NgxNG7cGJs2bcKaNWsQERGB2rVrV8bmffaISPhMPzYLxNDQEF5eXrh8+TICAwNx//59HD58GPXr18fhw4eF7kSysfFExHVkyoEsOBIWFoawsDBhuixA0qtXL8TFxeHWrVsQi8V4+fIlAMDHxwdJSUlyQcjSpFKpsA4FBQU4OTnB2NgYW7duLc/NYYwxxhhjjLHPWqXUNNmzZw9WrlyJgQMHYtGiRXBxcUFBQQEiIyOhpKSEEydOoGbNmhg2bBh27NiBbt26oWvXrtiwYQNEIhFCQ0NhYGAAAMjKysL333+PO3fuYMyYMfD19YWKikpFb9IXo/Q49sLCQigpKQnBjQ+1pi0qKsLatWsREBAALS0t5OTkYNWqVXBzc6uQ986Ahw8fYty4cdi+fTv09PRw+fJluTbNjx49Qp8+faClpYXff/8dUqlU6HBjY2MDU1NTrF+/Xu67Ll0o+fr165gyZQo6d+6MQYMGceCRMcYYY4wxVqVVSqaJqakpsrOzsXv3bgwdOhSzZs3CggULsHz5chQXF2P69OkAgKCgINSpUwcZGRlIT0+Hh4cH/vzzTyFgApQUI7WyskJsbCz8/f05YPL/3hULkwVMZsyYAVtbW/Tt2xdTpkwBgA92UFBSUoKXlxcuXryIlStX4tGjR0LAhOuWlL+XL19i3759ePz4MXbv3g2JRIKlS5fKzVO7dm14e3sjIiICYWFhUFBQELJTDA0NkZmZCaDkuy49tO3FixcYMWIErK2tUbt2bQ6YMMYYY4wxxhiAShk/YW5ujjp16uDUqVPw8fERprdt2xb29vZITU1FXl4edHV14eHhgerVq2PSpElCy9Ti4mKIxWKIRCI4OTlxS9s3vNnOuXSr2OvXr2PIkCEoKirCmDFjkJ2djZCQEKioqCAgIOCj1m9qagpTU1MAr7MUuG5J+VNWVoapqSmMjY3h6OiInJwc+Pn5wd3dHc2aNQNQEgxxdHSEh4cHPDw8cPToUTRv3hwvXrxAUlISvL29hfXJAmiLFy/GnDlzYGFhgZMnT6Jt27aVsn2MMcYYY4wx9rmptJbD9+/fR/369TF27FgsWLBAmN6vXz8AEDp9lCZ7q3yD/mGl2zlPmjRJGMKxZcsW3Lx5EzNnzkS1atWQkpKC7t27Iy8vDxEREWjVqtU711l6GAfALSsrQ+m2v8XFxbCyskLjxo2xc+dOuUwhqVSKHj16IDExEQ0bNsTt27dhYmKCnTt3Qk9PDwCQmZmJQYMG4c6dO5g3bx4GDBhQKdvEGGOMMcYYY5+rShmeAwB6enqYPn06jh07huDgYGRnZyM6OhqZmZno37//W/OXrs3Ayvauds7Vq1cX5unQoQNGjBgBAPD19UWbNm3w3XffoX79+pg+fTpevXr11nplXXAUFRXx/Plz7Nu3DwC3rKwMst+/rNDuggULsHfvXpw4cUKYRyKRQEFBAbt27cLu3bvRq1cvLF++HFFRUULABADU1dXh6+uLxMREDpgwxhhjjDHGWBkqLdMEKCksGhwcjDlz5sDc3ByJiYkYOXKkXOYJ+2eePn2Kfv36YdiwYXB3dwfwdoYIUFIvJjQ0FIsWLYK1tTVWrlyJMWPG4NChQ/juu+/KXC4kJASBgYEYNGgQQkJCoKamVnEbxt6pf//+SEtLw5kzZ6CqqoqsrCxoamrKteaW4ewgxhhjjDHGGPt4lRo0kbl9+zYyMjJgYmIiPAkvPQyBfbzNmzcjKCgIt2/fxvHjx7Fs2TLUqFEDZmZmGDBgAJo1a4asrCx07NgRI0eOxNixYwGUtKRdsWIF9PX1kZSUBFVVVeHzP3bsGMaMGQNFRUXMnDkTLi4ulbmJX6X/8nu/ffs2WrZsiSlTpiAtLQ1xcXFYtWoVWrdu/UnWzxhjjDHGGGNVVaUUgn1To0aN0KhRIwCvhxbwDd6/o6KiAn19faxbt06unfORI0cQFhaGEydOQEdHB0pKSvjzzz/x8OFDXL9+Henp6fjjjz+QnZ0tZJDcu3cPHh4eSEpKgp+fH7dzLgdEBKlU+p+yP4yMjGBlZQV/f38YGxtj9erVcgETgOsAMcYYY4wxxti/8VlkmrBP59q1a3Bzc4OOjg4cHR0xYcIEAEB0dDR8fHzQpUsXLFy4EKGhofjhhx+gp6eH9PR0/Prrr5g6daqwnuLiYkycOBHPnz/HlClTUL9+/crapK9W6eyP7OxshIeHo3nz5vjmm29Qp06djxpK89dff6Fv375ISUnB6tWr4eHh8da6GWOMMcYYY4z9Oxw0+Qp16dIFZ8+excGDB9G7d28AJfVj/P39kZqaii1btkBVVRVJSUm4c+cOOnToAG1tbWF5Wcvi/Px8rltSAWbMmIHFixejWbNmePLkCerVq4eTJ0+iWrVqH1z26tWrOHXqFEaPHi3Unymrhg1jjDHGGGOMsX+u0rrnsPKzbds2AMC5c+eEaUpKSrh79y6ICKqqqgCAZs2awdnZGdra2pBIJEJLZ1nrWg6YfFpSqfStaWFhYfj999+xY8cOxMTE4OzZs7h06RKGDx+OoqKiD67T0tISY8eOhaKiolyXI8YYY4wxxhhj/x0HTb5C/7SdM1DSPpiHc5QPIhJq9QAldXtkli9fji5duqBnz56Ii4uDq6sratWqhS5dukBJSem96yutqKiIgyWMMcYYY4wx9olx0OQrFRAQADc3N8ycORPfffcdunfvDhsbG7i5uVX2W6tSZLVFxGIx8vLyMGHCBPz22294+vQpAMDQ0BAqKirw9PSEvb09zMzM8Oeff2LIkCFC5khpxcXFwvpyc3Mxbtw4AHhngIUxxhhjjDHG2L/HQZOvlJKSEqZOnYpr164hJCQEycnJWLBgAQCAy9hUHFn2zpIlS6Cvr4+4uDgoKiri5cuXAEqCIPPnz0diYiJiY2OxYsUK1K1bFzk5OZgwYQJSUlIAQBiqI8smmTt3LoyNjXHu3Dk8fPiwEraMMcYYY4wxxr5+nM//leN2zhVP1vVGlmUSFxeHNWvWYPXq1XBzcxMK7QLAsGHDcPToUXTq1AlNmjQRll+3bh2uXLmCR48ewcTERMgkOXLkCMaOHQtlZWWsW7cOffr0qazNZIwxxhhjjLGvHgdNqpAPta9ln4ZYLEZaWhoaNGgAADhx4gSUlZXh7OyMp0+f4tatWygoKICKigratWsHLy8vbN26FQcOHICjoyNOnjyJvLw8rFq1CtbW1gCABw8eoGfPnrh//z78/Pzg4+ODGjVqVOZmMsYYY4wxxthXj1sOM/Yflc4cAUoCHC1atICzszPWrVuHc+fOoWfPnmjRogUKCwtRp04dREdHo379+vjpp58wYcIEXL16FTt27EB+fj4MDQ0xceJEYX1EhNjYWOzatQt+fn6oX79+ZWwmY4wxxhhjjFU5HDRh7BMrKirCihUrMHnyZCQnJ8PAwACHDx9GXFwcWrduDQ0NDbRo0QLe3t54/vw5tm7dipo1awJ4XTgWKKl3IutqVHo6Y4wxxhhjjLGKwUETxv6jlJQU+Pv7Y9++fcK0jIwM9OvXD1paWggLC3srGwUA+vfvj8aNG2POnDly02W7JAdJGGOMMcYYY6xycfccxv6jtLQ0HDt2TK47kZ6eHvz9/REeHo4zZ85AQUEBeXl5OHjwIEJDQ2FlZYU///yzzEKuIpGIAyaMMcYYY4wx9hngTBPG/iXZkJm8vDwsXLgQK1asQHJyMjQ1NQEA2dnZGDRoEDIzM5GQkIDi4mIMGjQI8fHxcHZ2xvz58yt5CxhjjDHGGGOMvQ9nmjD2L8myQdTV1fH9999DX18f48ePF17X1taGnp4erl27hqVLl0JRURFBQUG4cOGCEDApLi6ulPfOGGOMMcYYY+zDOGjCWBmICBKJ5KPnNzU1haenJ3bt2oX4+HgAwKtXryAWi9G9e3fs3bsXAGBsbAwtLS1IpVIQERQVues3Y4wxxhhjjH2u+I6NsTfIiraKxWLk5OQgNjYW7dq1EzrclEVJSQm9e/fG6dOnYWNjg1GjRiEuLg4qKirYtGkTdHV15eZ/sygsY4wxxhhjjLHPD9+5sSrrXeV8ZAGNqVOnwtDQENu2bcMff/zxwfXp6Ohg69at8PHxQXJyMpo1a4bDhw8LARMeisMYY4wxxhhjXxYuBMuqpNItgHNycoTirUQEIsL48eNx8uRJhISEwMrKCioqKqhRo8ZHrZOI8OrVKygrKwMAJBIJxGJx+W4QY4wxxhhjjLFPjjNNWJWkoKCAhIQE2NvbIygoCA8fPgRQUtz18ePHOHbsGEaNGoVvv/0WtWvX/mDARLasVCqFSCSCsrIypFIppFIpB0wYY4wxxhhj7AvFQRNWpcgySYKDg9GxY0cYGxujX79+qF69ujDP9evX8eDBAzg6On70eouLiyESiaCgoCAMw1FQUODaJYwxxhhjjDH2BeM7OlaliEQiPHv2DCdOnMDq1auxbt06WFtbQ0VFRZjHwsICL168wNmzZwHI1yJJTU1FYWGh8Lesw46sC05wcDC+//57pKamVsDWMMYYY4wxxhgrTxw0YVXOoUOHkJaWBnd3dxw/fhw9e/aEq6srpk+fjhs3bqB27dpwd3fHr7/+ipcvXwoBkZycHGzduhWXL18GIF+r5OjRo2jatCnWr1+PYcOGwdDQsLI2jzHGGGOMMcbYJ8KFYFmVs2fPHqxcuRIDBw7EokWL4OLigoKCApw8eRLKyso4deoUcnNz0bFjR+jo6KBbt25o2rQpgoODUa1aNezYsQONGjUCAKSkpMDLywvx8fEYO3YsfH195bJWGGOMMcYYY4x9uThowqqca9euwc3NDTo6OnB0dMSECRMAANHR0fDy8oKDgwPmz5+PlJQUzJs3D0lJScjLy4OzszPmzJkjrGfDhg0YO3Ys+vbti8DAQNSvX7+yNokxxhhjjDHGWDngoAmrkrp06YKzZ8/i4MGD6N27NwCgqKgI/v7+SE1NxebNm6GmpgYAyMvLg4KCAlRVVQGU1DhRVFREVFQU1NXV0bJly0rbDsYYY4wxxhhj5YdrmrAqadu2bQCAc+fOCdOUlJRw9+5dAICamhpk8UQ1NTWoqqpCKpWCiIQ6JnZ2dhwwYYwxxhhjjLGvmGJlvwHGKoOenh6mT5+O3bt3Q1tbG8OHD8etW7eQmZmJ0aNHAyjptFP6v9w+mDHGGGOMMcaqFh6ew6qsoqIiBAcHY86cOTA3N0diYiJGjhyJBQsWVPZbY4wxxhhjjDH2GeCgCavybt++jYyMDJiYmEBPTw8AQERChgljjDHGGGOMsaqJgyaMlSKRSKCgoMABE8YYY4wxxhhjHDRhjDHGGGOMMcYYKwtXtmSMMcYYY4wxxhgrAwdNGGOMMcYYY4wxxsrAQRPGGGOMMcYYY4yxMnDQhDHGGGOMMcYYY6wMHDRhjDHGGGOMMcYYKwMHTRhjjDHGGGOMMcbKwEETxhhjjDHGGGOMsTJw0IQxxhhjjDHGGGOsDBw0YYwxxhhjjDHGGCsDB00YY4wxxhhjjDHGysBBE8YYY4wxxhhjjLEy/B/bJB7Sh7hZ0QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1200x800 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot model results\n",
    "model_metrics_all_transposed_sorted.plot(\n",
    "    kind='bar',\n",
    "    rot=30,\n",
    "    figsize=(12,8)).legend(bbox_to_anchor=(1.0, 1.0)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "acc50914",
   "metadata": {},
   "source": [
    "![(Re) Introduction to Tensorflow Natural Language Processing](./assets/tf_nlp_desaster_tweets_02.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 198,
   "id": "51b0dad2",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "      <iframe id=\"tensorboard-frame-5c9c913d1382f3b3\" width=\"100%\" height=\"800\" frameborder=\"0\">\n",
       "      </iframe>\n",
       "      <script>\n",
       "        (function() {\n",
       "          const frame = document.getElementById(\"tensorboard-frame-5c9c913d1382f3b3\");\n",
       "          const url = new URL(\"/\", window.location);\n",
       "          const port = 6006;\n",
       "          if (port) {\n",
       "            url.port = port;\n",
       "          }\n",
       "          frame.src = url;\n",
       "        })();\n",
       "      </script>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Load TensorBoard\n",
    "%load_ext tensorboard\n",
    "%tensorboard --logdir './tensorboad/'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d932131",
   "metadata": {},
   "source": [
    "![(Re) Introduction to Tensorflow Natural Language Processing](./assets/tf_nlp_desaster_tweets_03.png)\n",
    "\n",
    "![(Re) Introduction to Tensorflow Natural Language Processing](./assets/tf_nlp_desaster_tweets_04.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6746b1c",
   "metadata": {},
   "source": [
    "## Saving & Loading Trained Model\n",
    "\n",
    "### HDF5 Format (Higher Compatibility to 3rd Parties)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 199,
   "id": "67c7fb6f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# save best performing model to HDF5 format\n",
    "model_6a.save('./models/6a_use_added_dense.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 200,
   "id": "d67cbf8b",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-23 20:15:34.310727: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype string\n",
      "\t [[{{node inputs}}]]\n",
      "2023-04-23 20:15:34.368474: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder' with dtype string and shape [?]\n",
      "\t [[{{node Placeholder}}]]\n"
     ]
    }
   ],
   "source": [
    "# restore model with custom TF Hub layer (only HDF5 format)\n",
    "# restoring requires import tensorflow_hub as hub\n",
    "loaded_model_6a = tf.keras.models.load_model(\n",
    "    './models/6a_use_added_dense.h5',\n",
    "    custom_objects={'KerasLayer': hub.KerasLayer}\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 201,
   "id": "2097ac4e",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24/24 [==============================] - 0s 4ms/step - loss: 0.4252 - accuracy: 0.8136\n",
      "24/24 [==============================] - 0s 4ms/step - loss: 0.4252 - accuracy: 0.8136\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 201,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# verify loaded model\n",
    "loaded_model_6a.evaluate(val_tweets, val_labels) == model_6a.evaluate(val_tweets, val_labels)\n",
    "# True"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff4cb27c",
   "metadata": {},
   "source": [
    "### Saved Model Format (Tensorflow Default)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "id": "d6371b2a",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 13:01:23.842540: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype string\n",
      "\t [[{{node inputs}}]]\n",
      "2023-04-24 13:01:23.900380: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'input_6' with dtype string and shape [?]\n",
      "\t [[{{node input_6}}]]\n",
      "2023-04-24 13:01:23.975478: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype string and shape [?]\n",
      "\t [[{{node inputs}}]]\n",
      "2023-04-24 13:01:24.035951: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'input_6' with dtype string and shape [?]\n",
      "\t [[{{node input_6}}]]\n",
      "2023-04-24 13:01:24.116094: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype string and shape [?]\n",
      "\t [[{{node inputs}}]]\n",
      "2023-04-24 13:01:24.173919: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'input_6' with dtype string and shape [?]\n",
      "\t [[{{node input_6}}]]\n",
      "2023-04-24 13:01:24.238763: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype string and shape [?]\n",
      "\t [[{{node inputs}}]]\n",
      "2023-04-24 13:01:24.302643: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'input_6' with dtype string and shape [?]\n",
      "\t [[{{node input_6}}]]\n",
      "2023-04-24 13:01:24.365712: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype string and shape [?]\n",
      "\t [[{{node inputs}}]]\n",
      "2023-04-24 13:01:24.430964: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'input_6' with dtype string and shape [?]\n",
      "\t [[{{node input_6}}]]\n",
      "2023-04-24 13:01:24.503968: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype string and shape [?]\n",
      "\t [[{{node inputs}}]]\n",
      "2023-04-24 13:01:24.559578: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype string and shape [?]\n",
      "\t [[{{node inputs}}]]\n",
      "2023-04-24 13:01:24.628859: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype string and shape [?]\n",
      "\t [[{{node inputs}}]]\n",
      "2023-04-24 13:01:24.688769: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype string and shape [?]\n",
      "\t [[{{node inputs}}]]\n",
      "2023-04-24 13:01:24.789767: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'input_6' with dtype string and shape [?]\n",
      "\t [[{{node input_6}}]]\n",
      "2023-04-24 13:01:24.861017: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype string and shape [?]\n",
      "\t [[{{node inputs}}]]\n",
      "2023-04-24 13:01:24.924212: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype string and shape [?]\n",
      "\t [[{{node inputs}}]]\n",
      "2023-04-24 13:01:24.986337: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype string and shape [?]\n",
      "\t [[{{node inputs}}]]\n",
      "2023-04-24 13:01:25.055508: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype string and shape [?]\n",
      "\t [[{{node inputs}}]]\n",
      "2023-04-24 13:01:25.142956: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype string and shape [?]\n",
      "\t [[{{node inputs}}]]\n",
      "2023-04-24 13:01:25.200113: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype string and shape [?]\n",
      "\t [[{{node inputs}}]]\n",
      "2023-04-24 13:01:25.257083: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype string and shape [?]\n",
      "\t [[{{node inputs}}]]\n",
      "2023-04-24 13:01:25.312180: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype string and shape [?]\n",
      "\t [[{{node inputs}}]]\n",
      "WARNING:absl:Found untraced functions such as _update_step_xla while saving (showing 1 of 1). These functions will not be directly callable after loading.\n",
      "2023-04-24 13:01:26.096560: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'serving_default_input_6' with dtype string and shape [?]\n",
      "\t [[{{node serving_default_input_6}}]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: ./models/6a_use_added_dense/assets\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: ./models/6a_use_added_dense/assets\n"
     ]
    }
   ],
   "source": [
    "# save best performing model to saved_model format\n",
    "model_6a.save('./models/6a_use_added_dense')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "id": "975efb5b",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-24 13:01:43.789634: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype string and shape [?]\n",
      "\t [[{{node inputs}}]]\n",
      "2023-04-24 13:01:43.847042: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'Placeholder' with dtype string and shape [?]\n",
      "\t [[{{node Placeholder}}]]\n"
     ]
    }
   ],
   "source": [
    "loaded_model_6a_saved_model = tf.keras.models.load_model(\n",
    "    './models/6a_use_added_dense',\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "id": "d6fb3d7c",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24/24 [==============================] - 0s 5ms/step - loss: 0.4236 - accuracy: 0.8163\n",
      "24/24 [==============================] - 0s 5ms/step - loss: 0.4236 - accuracy: 0.8163\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 110,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# verify loaded model\n",
    "loaded_model_6a_saved_model.evaluate(val_tweets, val_labels) == model_6a.evaluate(val_tweets, val_labels)\n",
    "# True"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e58c3fc",
   "metadata": {},
   "source": [
    "## Best Model Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "id": "0051fdc1",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24/24 [==============================] - 0s 4ms/step\n",
      "tf.Tensor([0. 1. 1. 0. 1.], shape=(5,), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "# find most wrong predictions\n",
    "## create dataframe with validation tweets, labels and model predictions\n",
    "loaded_model_pred_probs = tf.squeeze(loaded_model_6a_saved_model.predict(val_tweets))\n",
    "loaded_model_preds = tf.round(loaded_model_pred_probs)\n",
    "print(loaded_model_preds[:5])\n",
    "# [0. 1. 1. 0. 1.]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "id": "b1bde614",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>text</th>\n",
       "      <th>target</th>\n",
       "      <th>pred</th>\n",
       "      <th>pred_prob</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>DFR EP016 Monthly Meltdown - On Dnbheaven 2015...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.211551</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>FedEx no longer to transport bioterror germs i...</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.788260</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Gunmen kill four in El Salvador bus attack: Su...</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.989896</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>@camilacabello97 Internally and externally scr...</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.231131</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Radiation emergency #preparedness starts with ...</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.704475</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>757</th>\n",
       "      <td>That's the ultimate road to destruction</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.142349</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>758</th>\n",
       "      <td>@SetZorah dad why dont you claim me that mean ...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.115426</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>759</th>\n",
       "      <td>FedEx will no longer transport bioterror patho...</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.895312</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>760</th>\n",
       "      <td>Crack in the path where I wiped out this morni...</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.718347</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>761</th>\n",
       "      <td>I liked a @YouTube video from @dannyonpc http:...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.146166</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>762 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                  text  target  pred  \\\n",
       "0    DFR EP016 Monthly Meltdown - On Dnbheaven 2015...       0   0.0   \n",
       "1    FedEx no longer to transport bioterror germs i...       0   1.0   \n",
       "2    Gunmen kill four in El Salvador bus attack: Su...       1   1.0   \n",
       "3    @camilacabello97 Internally and externally scr...       1   0.0   \n",
       "4    Radiation emergency #preparedness starts with ...       1   1.0   \n",
       "..                                                 ...     ...   ...   \n",
       "757            That's the ultimate road to destruction       0   0.0   \n",
       "758  @SetZorah dad why dont you claim me that mean ...       0   0.0   \n",
       "759  FedEx will no longer transport bioterror patho...       0   1.0   \n",
       "760  Crack in the path where I wiped out this morni...       0   1.0   \n",
       "761  I liked a @YouTube video from @dannyonpc http:...       0   0.0   \n",
       "\n",
       "     pred_prob  \n",
       "0     0.211551  \n",
       "1     0.788260  \n",
       "2     0.989896  \n",
       "3     0.231131  \n",
       "4     0.704475  \n",
       "..         ...  \n",
       "757   0.142349  \n",
       "758   0.115426  \n",
       "759   0.895312  \n",
       "760   0.718347  \n",
       "761   0.146166  \n",
       "\n",
       "[762 rows x 4 columns]"
      ]
     },
     "execution_count": 113,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pred_df = pd.DataFrame({\n",
    "    \"text\": val_tweets,\n",
    "    \"target\": val_labels,\n",
    "    \"pred\": loaded_model_preds,\n",
    "    \"pred_prob\": loaded_model_pred_probs\n",
    "})\n",
    "\n",
    "pred_df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "731d2ba7",
   "metadata": {},
   "source": [
    "|\t| text | target | pred | pred_prob |\n",
    "| -- | -- | -- | -- | -- | \n",
    "| 0 | DFR EP016 Monthly Meltdown - On Dnbheaven 2015... | 0 | 0.0 | 0.191866 |\n",
    "| 1 | FedEx no longer to transport bioterror germs i... | 0 | 1.0 | 0.793415 |\n",
    "| 2 | Gunmen kill four in El Salvador bus attack: Su... | 1 | 1.0 | 0.992559 |\n",
    "| 3 | @camilacabello97 Internally and externally scr... | 1 | 0.0 | 0.255227 |\n",
    "| 4 | Radiation emergency #preparedness starts with ... | 1 | 1.0 | 0.722030 |\n",
    "...\n",
    "| 757 | That's the ultimate road to destruction | 0 | 0.0 | 0.141685 |\n",
    "| 758 | @SetZorah dad why dont you claim me that mean ... | 0 | 0.0 | 0.111817 |\n",
    "| 759 | FedEx will no longer transport bioterror patho... | 0 | 1.0 | 0.898317 |\n",
    "| 760 | Crack in the path where I wiped out this morni... | 0 | 1.0 | 0.671607 |\n",
    "| 761 | I liked a @YouTube video from @dannyonpc http:... | 0 | 0.0 | 0.127992 |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "id": "76d40e12",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>text</th>\n",
       "      <th>target</th>\n",
       "      <th>pred</th>\n",
       "      <th>pred_prob</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>? High Skies - Burning Buildings ? http://t.co...</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.917166</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>759</th>\n",
       "      <td>FedEx will no longer transport bioterror patho...</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.895312</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>@madonnamking RSPCA site multiple 7 story high...</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.879588</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>628</th>\n",
       "      <td>@noah_anyname That's where the concentration c...</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.862645</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>109</th>\n",
       "      <td>[55436] 1950 LIONEL TRAINS SMOKE LOCOMOTIVES W...</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.843334</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>411</th>\n",
       "      <td>@SoonerMagic_ I mean I'm a fan but I don't nee...</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.057629</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>233</th>\n",
       "      <td>I get to smoke my shit in peace</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.056021</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>244</th>\n",
       "      <td>Reddit Will Now QuarantineÛ_ http://t.co/pkUA...</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.048836</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Ron &amp;amp; Fez - Dave's High School Crush https...</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.042986</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>Why are you deluged with low self-image? Take ...</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.036385</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>140 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                  text  target  pred  \\\n",
       "31   ? High Skies - Burning Buildings ? http://t.co...       0   1.0   \n",
       "759  FedEx will no longer transport bioterror patho...       0   1.0   \n",
       "49   @madonnamking RSPCA site multiple 7 story high...       0   1.0   \n",
       "628  @noah_anyname That's where the concentration c...       0   1.0   \n",
       "109  [55436] 1950 LIONEL TRAINS SMOKE LOCOMOTIVES W...       0   1.0   \n",
       "..                                                 ...     ...   ...   \n",
       "411  @SoonerMagic_ I mean I'm a fan but I don't nee...       1   0.0   \n",
       "233                    I get to smoke my shit in peace       1   0.0   \n",
       "244  Reddit Will Now QuarantineÛ_ http://t.co/pkUA...       1   0.0   \n",
       "23   Ron &amp; Fez - Dave's High School Crush https...       1   0.0   \n",
       "38   Why are you deluged with low self-image? Take ...       1   0.0   \n",
       "\n",
       "     pred_prob  \n",
       "31    0.917166  \n",
       "759   0.895312  \n",
       "49    0.879588  \n",
       "628   0.862645  \n",
       "109   0.843334  \n",
       "..         ...  \n",
       "411   0.057629  \n",
       "233   0.056021  \n",
       "244   0.048836  \n",
       "23    0.042986  \n",
       "38    0.036385  \n",
       "\n",
       "[140 rows x 4 columns]"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create another datframe that only contains wrong predictions\n",
    "most_wrong = pred_df[pred_df['target'] != pred_df['pred']].sort_values(\n",
    "    'pred_prob',\n",
    "    ascending=False\n",
    ")\n",
    "\n",
    "most_wrong"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78ef4b58",
   "metadata": {},
   "source": [
    "|\t| text | target | pred | pred_prob |\n",
    "| -- | -- | -- | -- | -- | \n",
    "| 31 | ? High Skies - Burning Buildings ? http://t.co... | 0 | 1.0 | 0.936771 |\n",
    "| 628 | @noah_anyname That's where the concentration c... | 0 | 1.0 | 0.907564 |\n",
    "| 759 | FedEx will no longer transport bioterror patho... | 0 | 1.0 | 0.898317 |\n",
    "| 251 | @AshGhebranious civil rights continued in the ... | 0 | 1.0 | 0.854326 |\n",
    "| 49 | @madonnamking RSPCA site multiple 7 story high... | 0 | 1.0 | 0.853781 |\n",
    "...\n",
    "| 233 | I get to smoke my shit in peace | 1 | 0.0 | 0.052748 |\n",
    "| 411 | @SoonerMagic_ I mean I'm a fan but I don't nee... | 1 | 0.0 | 0.046553 |\n",
    "| 244 | Reddit Will Now QuarantineÛ_ http://t.co/pkUA... | 1 | 0.0 | 0.044474 |\n",
    "| 23 | Ron &amp; Fez - Dave's High School Crush https... | 1 | 0.0 | 0.040853 |\n",
    "| 38 | Why are you deluged with low self-image? Take ... | 1 | 0.0 | 0.040229 |"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2cc73fb",
   "metadata": {},
   "source": [
    "#### False Positives"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "id": "a955d9f8",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Target: 0, Pred: 1.0, Probability: 0.9171655774116516\n",
      "Tweet: ? High Skies - Burning Buildings ? http://t.co/uVq41i3Kx2 #nowplaying\n",
      "----\n",
      "\n",
      "Target: 0, Pred: 1.0, Probability: 0.8953118324279785\n",
      "Tweet: FedEx will no longer transport bioterror pathogens in wake of anthrax lab mishaps http://t.co/lHpgxc4b8J\n",
      "----\n",
      "\n",
      "Target: 0, Pred: 1.0, Probability: 0.8795878887176514\n",
      "Tweet: @madonnamking RSPCA site multiple 7 story high rise buildings next to low density character residential in an area that floods\n",
      "----\n",
      "\n",
      "Target: 0, Pred: 1.0, Probability: 0.8626453876495361\n",
      "Tweet: @noah_anyname That's where the concentration camps and mass murder come in. \n",
      " \n",
      "EVERY. FUCKING. TIME.\n",
      "----\n",
      "\n",
      "Target: 0, Pred: 1.0, Probability: 0.8433336615562439\n",
      "Tweet: [55436] 1950 LIONEL TRAINS SMOKE LOCOMOTIVES WITH MAGNE-TRACTION INSTRUCTIONS http://t.co/xEZBs3sq0y http://t.co/C2x0QoKGlY\n",
      "----\n",
      "\n",
      "Target: 0, Pred: 1.0, Probability: 0.842764139175415\n",
      "Tweet: Ashes 2015: AustraliaÛªs collapse at Trent Bridge among worst in history: England bundled out Australia for 60 ... http://t.co/t5TrhjUAU0\n",
      "----\n",
      "\n",
      "Target: 0, Pred: 1.0, Probability: 0.8387770056724548\n",
      "Tweet: @freefromwolves GodsLove &amp; #thankU brother Danny for RT of NEW VIDEO http://t.co/cybKsXHF7d The Coming Apocalyptic US Earthquake &amp; Tsunami\n",
      "----\n",
      "\n",
      "Target: 0, Pred: 1.0, Probability: 0.8276335597038269\n",
      "Tweet: åÈMGN-AFRICAå¨ pin:263789F4 åÈ Correction: Tent Collapse Story: Correction: Tent Collapse story åÈ http://t.co/fDJUYvZMrv @wizkidayo\n",
      "----\n",
      "\n",
      "Target: 0, Pred: 1.0, Probability: 0.8238321542739868\n",
      "Tweet: @AshGhebranious civil rights continued in the 60s. And what about trans-generational trauma? if anything we should listen to the Americans.\n",
      "----\n",
      "\n",
      "Target: 0, Pred: 1.0, Probability: 0.8230549097061157\n",
      "Tweet: @SonofLiberty357 all illuminated by the brightly burning buildings all around the town!\n",
      "----\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for row in most_wrong[:10].itertuples():\n",
    "    _, text, target, pred, pred_prob = row\n",
    "    print(f\"Target: {target}, Pred: {pred}, Probability: {pred_prob}\")\n",
    "    print(f\"Tweet: {text}\")\n",
    "    print(\"----\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a4db8ca",
   "metadata": {},
   "source": [
    "```\n",
    "Target: 0, Pred: 1.0, Probability: 0.936771035194397\n",
    "Tweet: ? High Skies - Burning Buildings ? http://t.co/uVq41i3Kx2 #nowplaying\n",
    "----\n",
    "\n",
    "Target: 0, Pred: 1.0, Probability: 0.9075638055801392\n",
    "Tweet: @noah_anyname That's where the concentration camps and mass murder come in. \n",
    " \n",
    "EVERY. FUCKING. TIME.\n",
    "----\n",
    "\n",
    "Target: 0, Pred: 1.0, Probability: 0.8983168601989746\n",
    "Tweet: FedEx will no longer transport bioterror pathogens in wake of anthrax lab mishaps http://t.co/lHpgxc4b8J\n",
    "----\n",
    "\n",
    "Target: 0, Pred: 1.0, Probability: 0.8543258309364319\n",
    "Tweet: @AshGhebranious civil rights continued in the 60s. And what about trans-generational trauma? if anything we should listen to the Americans.\n",
    "----\n",
    "\n",
    "Target: 0, Pred: 1.0, Probability: 0.8537805676460266\n",
    "Tweet: @madonnamking RSPCA site multiple 7 story high rise buildings next to low density character residential in an area that floods\n",
    "----\n",
    "\n",
    "Target: 0, Pred: 1.0, Probability: 0.8336963057518005\n",
    "Tweet: [55436] 1950 LIONEL TRAINS SMOKE LOCOMOTIVES WITH MAGNE-TRACTION INSTRUCTIONS http://t.co/xEZBs3sq0y http://t.co/C2x0QoKGlY\n",
    "----\n",
    "\n",
    "Target: 0, Pred: 1.0, Probability: 0.833143413066864\n",
    "Tweet: Ashes 2015: AustraliaÛªs collapse at Trent Bridge among worst in history: England bundled out Australia for 60 ... http://t.co/t5TrhjUAU0\n",
    "----\n",
    "\n",
    "Target: 0, Pred: 1.0, Probability: 0.8305423855781555\n",
    "Tweet: @SonofLiberty357 all illuminated by the brightly burning buildings all around the town!\n",
    "----\n",
    "\n",
    "Target: 0, Pred: 1.0, Probability: 0.8277301788330078\n",
    "Tweet: Deaths 3 http://t.co/nApviyGKYK\n",
    "----\n",
    "\n",
    "Target: 0, Pred: 1.0, Probability: 0.8111591935157776\n",
    "Tweet: The Sound of Arson\n",
    "----\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41bdc6c1",
   "metadata": {},
   "source": [
    "#### False Negatives"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "id": "a8c9c32d",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Target: 1, Pred: 0.0, Probability: 0.08482133597135544\n",
      "Tweet: 'The way you move is like a full on rainstorm and I'm a house of cards'\n",
      "----\n",
      "\n",
      "Target: 1, Pred: 0.0, Probability: 0.08254487812519073\n",
      "Tweet: @willienelson We need help! Horses will die!Please RT &amp; sign petition!Take a stand &amp; be a voice for them! #gilbert23 https://t.co/e8dl1lNCVu\n",
      "----\n",
      "\n",
      "Target: 1, Pred: 0.0, Probability: 0.08116379380226135\n",
      "Tweet: VICTORINOX SWISS ARMY DATE WOMEN'S RUBBER MOP WATCH 241487 http://t.co/yFy3nkkcoH http://t.co/KNEhVvOHVK\n",
      "----\n",
      "\n",
      "Target: 1, Pred: 0.0, Probability: 0.07389821112155914\n",
      "Tweet: @DavidVonderhaar At least you were sincere ??\n",
      "----\n",
      "\n",
      "Target: 1, Pred: 0.0, Probability: 0.06220388039946556\n",
      "Tweet: You can never escape me. Bullets don't harm me. Nothing harms me. But I know pain. I know pain. Sometimes I share it. With someone like you.\n",
      "----\n",
      "\n",
      "Target: 1, Pred: 0.0, Probability: 0.05762890353798866\n",
      "Tweet: @SoonerMagic_ I mean I'm a fan but I don't need a girl sounding off like a damn siren\n",
      "----\n",
      "\n",
      "Target: 1, Pred: 0.0, Probability: 0.0560205839574337\n",
      "Tweet: I get to smoke my shit in peace\n",
      "----\n",
      "\n",
      "Target: 1, Pred: 0.0, Probability: 0.048835765570402145\n",
      "Tweet: Reddit Will Now QuarantineÛ_ http://t.co/pkUAMXw6pm #onlinecommunities #reddit #amageddon #freespeech #Business http://t.co/PAWvNJ4sAP\n",
      "----\n",
      "\n",
      "Target: 1, Pred: 0.0, Probability: 0.04298585653305054\n",
      "Tweet: Ron &amp; Fez - Dave's High School Crush https://t.co/aN3W16c8F6 via @YouTube\n",
      "----\n",
      "\n",
      "Target: 1, Pred: 0.0, Probability: 0.03638499975204468\n",
      "Tweet: Why are you deluged with low self-image? Take the quiz: http://t.co/XsPqdOrIqj http://t.co/CQYvFR4UCy\n",
      "----\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for row in most_wrong[-10:].itertuples():\n",
    "    _, text, target, pred, pred_prob = row\n",
    "    print(f\"Target: {target}, Pred: {pred}, Probability: {pred_prob}\")\n",
    "    print(f\"Tweet: {text}\")\n",
    "    print(\"----\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "198540a7",
   "metadata": {},
   "source": [
    "```\n",
    "Target: 1, Pred: 0.0, Probability: 0.07530134916305542\n",
    "Tweet: going to redo my nails and watch behind the scenes of desolation of smaug ayyy\n",
    "----\n",
    "\n",
    "Target: 1, Pred: 0.0, Probability: 0.07096730172634125\n",
    "Tweet: @DavidVonderhaar At least you were sincere ??\n",
    "----\n",
    "\n",
    "Target: 1, Pred: 0.0, Probability: 0.06341199576854706\n",
    "Tweet: Lucas Duda is Ghost Rider. Not the Nic Cage version but an actual 'engulfed in flames' badass. #Mets\n",
    "----\n",
    "\n",
    "Target: 1, Pred: 0.0, Probability: 0.06147787347435951\n",
    "Tweet: You can never escape me. Bullets don't harm me. Nothing harms me. But I know pain. I know pain. Sometimes I share it. With someone like you.\n",
    "----\n",
    "\n",
    "Target: 1, Pred: 0.0, Probability: 0.05444430932402611\n",
    "Tweet: @willienelson We need help! Horses will die!Please RT &amp; sign petition!Take a stand &amp; be a voice for them! #gilbert23 https://t.co/e8dl1lNCVu\n",
    "----\n",
    "\n",
    "Target: 1, Pred: 0.0, Probability: 0.0527476891875267\n",
    "Tweet: I get to smoke my shit in peace\n",
    "----\n",
    "\n",
    "Target: 1, Pred: 0.0, Probability: 0.04655275493860245\n",
    "Tweet: @SoonerMagic_ I mean I'm a fan but I don't need a girl sounding off like a damn siren\n",
    "----\n",
    "\n",
    "Target: 1, Pred: 0.0, Probability: 0.04447368532419205\n",
    "Tweet: Reddit Will Now QuarantineÛ_ http://t.co/pkUAMXw6pm #onlinecommunities #reddit #amageddon #freespeech #Business http://t.co/PAWvNJ4sAP\n",
    "----\n",
    "\n",
    "Target: 1, Pred: 0.0, Probability: 0.04085317254066467\n",
    "Tweet: Ron &amp; Fez - Dave's High School Crush https://t.co/aN3W16c8F6 via @YouTube\n",
    "----\n",
    "\n",
    "Target: 1, Pred: 0.0, Probability: 0.0402291864156723\n",
    "Tweet: Why are you deluged with low self-image? Take the quiz: http://t.co/XsPqdOrIqj http://t.co/CQYvFR4UCy\n",
    "----\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba1c6e1f",
   "metadata": {},
   "source": [
    "## Test Dataset Predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "id": "bcd033e1",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>keyword</th>\n",
       "      <th>location</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2406</th>\n",
       "      <td>8051</td>\n",
       "      <td>refugees</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Refugees as citizens - The Hindu http://t.co/G...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>134</th>\n",
       "      <td>425</td>\n",
       "      <td>apocalypse</td>\n",
       "      <td>Currently Somewhere On Earth</td>\n",
       "      <td>@5SOStag honestly he could say an apocalypse i...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>411</th>\n",
       "      <td>1330</td>\n",
       "      <td>blown%20up</td>\n",
       "      <td>Scout Team</td>\n",
       "      <td>If you bored as shit don't nobody fuck wit you...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>203</th>\n",
       "      <td>663</td>\n",
       "      <td>attack</td>\n",
       "      <td>NaN</td>\n",
       "      <td>@RealTwanBrown Yesterday I Had A Heat Attack ?...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>889</th>\n",
       "      <td>2930</td>\n",
       "      <td>danger</td>\n",
       "      <td>Leeds</td>\n",
       "      <td>The Devil Wears Prada is still one of my favou...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        id     keyword                      location  \\\n",
       "2406  8051    refugees                           NaN   \n",
       "134    425  apocalypse  Currently Somewhere On Earth   \n",
       "411   1330  blown%20up                   Scout Team    \n",
       "203    663      attack                           NaN   \n",
       "889   2930      danger                        Leeds    \n",
       "\n",
       "                                                   text  \n",
       "2406  Refugees as citizens - The Hindu http://t.co/G...  \n",
       "134   @5SOStag honestly he could say an apocalypse i...  \n",
       "411   If you bored as shit don't nobody fuck wit you...  \n",
       "203   @RealTwanBrown Yesterday I Had A Heat Attack ?...  \n",
       "889   The Devil Wears Prada is still one of my favou...  "
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# get the test dataset and randomize\n",
    "test_df_shuffle = test_df.sample(frac=1, random_state=SEED)\n",
    "\n",
    "test_df_shuffle.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3dfd5cb",
   "metadata": {},
   "source": [
    "|\t| id \t | keyword | location | text |\n",
    "| -- | -- | -- | -- | -- | \n",
    "| 2406 | 8051 | refugees | NaN | Refugees as citizens - The Hindu http://t.co/G... |\n",
    "| 134 | 425 | apocalypse | Currently Somewhere On Earth | @5SOStag honestly he could say an apocalypse i... |\n",
    "| 411 | 1330 | blown%20up | Scout Team | If you bored as shit don't nobody fuck wit you... |\n",
    "| 203 | 663 | attack | NaN | @RealTwanBrown Yesterday I Had A Heat Attack ?... |\n",
    "| 889 | 2930 | danger | Leeds | The Devil Wears Prada is still one of my favou... |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "id": "8a8aaa7b",
   "metadata": {},
   "outputs": [],
   "source": [
    "test_tweets = test_df_shuffle['text']\n",
    "test_tweets = np.array(test_tweets.values.tolist())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "id": "4c09e46f",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "102/102 [==============================] - 0s 4ms/step\n"
     ]
    }
   ],
   "source": [
    "loaded_model_pred_probs_test = tf.squeeze(loaded_model_6a_saved_model.predict(test_tweets))\n",
    "loaded_model_preds_test = tf.round(loaded_model_pred_probs_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "id": "72917080",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>text</th>\n",
       "      <th>pred</th>\n",
       "      <th>pred_prob</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Refugees as citizens - The Hindu http://t.co/G...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.691977</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>@5SOStag honestly he could say an apocalypse i...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.123520</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>If you bored as shit don't nobody fuck wit you...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.094058</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>@RealTwanBrown Yesterday I Had A Heat Attack ?...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.124991</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>The Devil Wears Prada is still one of my favou...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.032189</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3258</th>\n",
       "      <td>Free Kindle Book - Aug 3-7 - Thriller - Desola...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.082993</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3259</th>\n",
       "      <td>HitchBot travels Europe and greeted with open ...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.651606</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3260</th>\n",
       "      <td>If you told me you was drowning. I would not l...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.258282</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3261</th>\n",
       "      <td>First time for everything! @ Coney Island Cycl...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.685601</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3262</th>\n",
       "      <td>Rocky Fire #cali #SCFD #wildfire #LakeCounty h...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.802510</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3263 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                   text  pred  pred_prob\n",
       "0     Refugees as citizens - The Hindu http://t.co/G...   1.0   0.691977\n",
       "1     @5SOStag honestly he could say an apocalypse i...   0.0   0.123520\n",
       "2     If you bored as shit don't nobody fuck wit you...   0.0   0.094058\n",
       "3     @RealTwanBrown Yesterday I Had A Heat Attack ?...   0.0   0.124991\n",
       "4     The Devil Wears Prada is still one of my favou...   0.0   0.032189\n",
       "...                                                 ...   ...        ...\n",
       "3258  Free Kindle Book - Aug 3-7 - Thriller - Desola...   0.0   0.082993\n",
       "3259  HitchBot travels Europe and greeted with open ...   1.0   0.651606\n",
       "3260  If you told me you was drowning. I would not l...   0.0   0.258282\n",
       "3261  First time for everything! @ Coney Island Cycl...   1.0   0.685601\n",
       "3262  Rocky Fire #cali #SCFD #wildfire #LakeCounty h...   1.0   0.802510\n",
       "\n",
       "[3263 rows x 3 columns]"
      ]
     },
     "execution_count": 120,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pred_test_df = pd.DataFrame({\n",
    "    \"text\": test_tweets,\n",
    "    \"pred\": loaded_model_preds_test,\n",
    "    \"pred_prob\": loaded_model_pred_probs_test\n",
    "})\n",
    "\n",
    "pred_test_df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0b6a8cb",
   "metadata": {},
   "source": [
    "|\t| text | pred | pred_prob |\n",
    "| -- | -- | -- | -- | \n",
    "| 0 | Refugees as citizens - The Hindu http://t.co/G... | 1.0 | 0.717548 |\n",
    "| 1 | @5SOStag honestly he could say an apocalypse i... | 0.0 | 0.093809 |\n",
    "| 2 | If you bored as shit don't nobody fuck wit you... | 0.0 | 0.082749 |\n",
    "| 3 | @RealTwanBrown Yesterday I Had A Heat Attack ?... | 0.0 | 0.097784 |\n",
    "| 4 | The Devil Wears Prada is still one of my favou... | 0.0 | 0.028455 |\n",
    "...\n",
    "| 3258 | Free Kindle Book - Aug 3-7 - Thriller - Desola... | 0.0 | 0.080420 |\n",
    "| 3259 | HitchBot travels Europe and greeted with open ... | 1.0 | 0.603396 |\n",
    "| 3260 | If you told me you was drowning. I would not l... | 0.0 | 0.208108 |\n",
    "| 3261 | First time for everything! @ Coney Island Cycl... | 1.0 | 0.617582 |\n",
    "| 3262 | Rocky Fire #cali #SCFD #wildfire #LakeCounty h... | 1.0 | 0.794876 |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "id": "c079997d",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1/1 [==============================] - 0s 27ms/step\n",
      "Pred: 1, Probability: 0.9131472706794739\n",
      "Text: Watch This Airport Get Swallowed Up By A Sandstorm In Under A Minute http://t.co/OFhjcn36o1\n",
      "-------\n",
      "\n",
      "1/1 [==============================] - 0s 26ms/step\n",
      "Pred: 1, Probability: 0.9558534622192383\n",
      "Text: RT @LivingSafely: NWS posts Severe #Thunderstorm Warnings for parts of #AR #NC #OK. Seek strong shelter if at risk: http://t.co/kImr0l24Fb\n",
      "-------\n",
      "\n",
      "1/1 [==============================] - 0s 25ms/step\n",
      "Pred: 1, Probability: 0.5604234933853149\n",
      "Text: But is feast.dll still vulnerable to DLL hijacking?  https://t.co/3vaig1PtYk https://t.co/BZDNKpOry1\n",
      "-------\n",
      "\n",
      "1/1 [==============================] - 0s 30ms/step\n",
      "Pred: 0, Probability: 0.06958078593015671\n",
      "Text: I'm incredibly sad &amp; heartbroken so I'm just going to drown my sorrows in food &amp; watch tv all night ?? http://t.co/yQiZ2p1rSm\n",
      "-------\n",
      "\n",
      "1/1 [==============================] - 0s 38ms/step\n",
      "Pred: 1, Probability: 0.9855814576148987\n",
      "Text: Suicide bomber kills 15 in Saudi security site mosque - Reuters http://t.co/KCObrZBVDs http://t.co/y62HSFVIAQ\n",
      "-------\n",
      "\n",
      "1/1 [==============================] - 0s 28ms/step\n",
      "Pred: 0, Probability: 0.21613247692584991\n",
      "Text: ÛÏ@5SOSGlobalSquad: me: will give all of my money to hear social casualty live \n",
      "\n",
      "#MTVHottest 5sosÛ I have about 5 buck here I'd give it all??\n",
      "-------\n",
      "\n",
      "1/1 [==============================] - 0s 30ms/step\n",
      "Pred: 1, Probability: 0.6258124113082886\n",
      "Text: Obama warns there will be another war without Iran deal #military http://t.co/gM7QlTRLuW http://t.co/wTutMlHDMH\n",
      "-------\n",
      "\n",
      "1/1 [==============================] - 0s 28ms/step\n",
      "Pred: 0, Probability: 0.33986303210258484\n",
      "Text: @tcbinaflash77 honestly I can't understand why @chanelwestcoast is blaming the police when she wasn't cuffed buy them. #1oak\n",
      "-------\n",
      "\n",
      "1/1 [==============================] - 0s 26ms/step\n",
      "Pred: 0, Probability: 0.13408930599689484\n",
      "Text: @dudeandpal it's so bad my eyes and ears are bleeding\n",
      "-------\n",
      "\n",
      "1/1 [==============================] - 0s 28ms/step\n",
      "Pred: 0, Probability: 0.04976771026849747\n",
      "Text: The Best Drought-Tolerant Perennials http://t.co/AdZONC7Olv http://t.co/ih9iTMhFjZ\n",
      "-------\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# make random prediction on sub-samples of the test dataset\n",
    "test_samples = random.sample(test_tweets.tolist(), 10)\n",
    "\n",
    "for test_sample in test_samples:\n",
    "    pred_prob = tf.squeeze(loaded_model_6a_saved_model.predict([test_sample]))\n",
    "    pred = tf.round(pred_prob)\n",
    "    \n",
    "    print(f\"Pred: {int(pred)}, Probability: {pred_prob}\")\n",
    "    print(f\"Text: {test_sample}\")\n",
    "    print(\"-------\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5777e12f",
   "metadata": {},
   "source": [
    "```\n",
    "1/1 [==============================] - 0s 26ms/step\n",
    "Pred: 0, Probability: 0.15592874586582184\n",
    "Text: @EllaEMusic_ You should have just simply let on that you had electrocuted yourself while plugging in your phone charger. It works for me...\n",
    "-------\n",
    "\n",
    "1/1 [==============================] - 0s 25ms/step\n",
    "Pred: 1, Probability: 0.5131102800369263\n",
    "Text: Evacuation drill at work. The fire doors wouldn't open so i got to smash the emergency release glass #feelingmanly\n",
    "-------\n",
    "\n",
    "1/1 [==============================] - 0s 33ms/step\n",
    "Pred: 1, Probability: 0.9751215577125549\n",
    "Text: Rare photographs show the nightmare aftermath of #Hiroshima | #NoNukes #Amerikkka #WhiteTerrorism #Nuclear #Disaster  http://t.co/8tWLAKdaBf\n",
    "-------\n",
    "\n",
    "1/1 [==============================] - 0s 28ms/step\n",
    "Pred: 1, Probability: 0.676400899887085\n",
    "Text: kayaking about killed us so mom and grandma came to the rescue.\n",
    "-------\n",
    "\n",
    "1/1 [==============================] - 0s 26ms/step\n",
    "Pred: 1, Probability: 0.9690964818000793\n",
    "Text: Pasco officials impressed by drone video showing floods: Drone videos have given an up-close view of flooded areasÛ_ http://t.co/PrUunEDids\n",
    "-------\n",
    "\n",
    "1/1 [==============================] - 0s 29ms/step\n",
    "Pred: 0, Probability: 0.07793092727661133\n",
    "Text: just trying to smoke and get Taco Bell\n",
    "-------\n",
    "\n",
    "1/1 [==============================] - 0s 26ms/step\n",
    "Pred: 1, Probability: 0.9799614548683167\n",
    "Text: RT AbbsWinston: #Zionist #Terrorist kidnapped 15 #Palestinians in overnight terror on Palestinian Villages Û_ http://t.co/J5mKcbKcov\n",
    "-------\n",
    "\n",
    "1/1 [==============================] - 0s 24ms/step\n",
    "Pred: 1, Probability: 0.8005449771881104\n",
    "Text: ME: gun shot wounds 3 4 6 7 'rapidly lethal' would have killed in 30-60 seconds or few minutes max. #kerricktrial\n",
    "-------\n",
    "\n",
    "1/1 [==============================] - 0s 24ms/step\n",
    "Pred: 1, Probability: 0.9902410507202148\n",
    "Text: Three Israeli soldiers wounded in West Bank terrorist attack - Haaretz http://t.co/Mwd1iPMoWT #world\n",
    "-------\n",
    "\n",
    "1/1 [==============================] - 0s 29ms/step\n",
    "Pred: 0, Probability: 0.03447313234210014\n",
    "Text: @_Souuul * gains super powers im now lava girl throws you ina chest wrapped in chains &amp; sinks you down the the bottom of the ocean*\n",
    "-------\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48be0515",
   "metadata": {},
   "source": [
    "## Speed/Score Tradeoff"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "id": "bada5a90",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "102/102 [==============================] - 0s 4ms/step\n",
      "0.5129240550013492 0.00015719401011380607\n"
     ]
    }
   ],
   "source": [
    "model_6_time_to_prediction, model_6_prediction_time_weighted = time_to_prediction(\n",
    "    model = loaded_model_6a_saved_model,\n",
    "    samples = test_tweets\n",
    ")\n",
    "\n",
    "print(model_6_time_to_prediction, model_6_prediction_time_weighted)\n",
    "# 0.5129240550013492 0.00015719401011380607"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "id": "c971dfd8",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.05225817499922414 1.6015376953485792e-05\n"
     ]
    }
   ],
   "source": [
    "model_0_time_to_prediction, model_0_prediction_time_weighted = time_to_prediction(\n",
    "    model = model_0,\n",
    "    samples = test_tweets\n",
    ")\n",
    "\n",
    "print(model_0_time_to_prediction, model_0_prediction_time_weighted)\n",
    "# 0.05225817499922414 1.6015376953485792e-05"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "id": "631ca8fd",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([<AxesSubplot: title={'center': 'TimeToPrediction'}>,\n",
       "       <AxesSubplot: title={'center': 'TimeToPrediction (weighted)'}>],\n",
       "      dtype=object)"
      ]
     },
     "execution_count": 124,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA/gAAAKqCAYAAACO6vXfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAACGJklEQVR4nOzde1hVdd7//9cG5SAKZCiIkliZh1RIUMQ0LVFMp4amKTXvRGT025Sn0ErLALOJ8lBmmpRT6jR1azVFjToYoaYpNxqG5jEzzVOgRoCigrLX749+rGnHQVAUWT0f17Uv25/1Xmu919rbcV57nWyGYRgCAAAAAAD1mlNdNwAAAAAAAC4fAR8AAAAAAAsg4AMAAAAAYAEEfAAAAAAALICADwAAAACABRDwAQAAAACwAAI+AAAAAAAWQMAHAAAAAMACCPgAAAAAAFgAAR8AgDo0cuRIBQYG1nUb9VpiYqJsNpvDWGBgoEaOHFlr6+BzAgDUBwR8AABqmc1mq9Zr3bp1ddLfwYMHq93jwYMHL7q8X9c7OTnJ399fAwYMqLPtu1THjh1TYmKisrOz67oVAAAuSYO6bgAAAKt55513HN7/4x//UFpaWrnxDh06aNGiRbLb7VezPTVr1qxcL3PmzNGRI0f0yiuvlKutjv79+2vEiBEyDEMHDhzQ66+/rrvuuksrV67U3XffXWu9V9fevXvl5FSz4xjHjh3T9OnTFRgYqODgYIdpdfE5AQBQUwR8AABq2f/8z/84vP+///s/paWllRuvKx4eHuV6WbZsmX7++edL7vGWW25xmPe+++5Tly5dNHfu3EoD/rlz5+Ti4lLjIF4drq6utbq8hg0b1uryAAC4EjhFHwCAOvTba7vLTp+fPXu2FixYoBtvvFGNGjXSgAEDdPjwYRmGoRkzZqhVq1Zyd3fXH//4R+Xl5ZVb7n/+8x/17t1bHh4eatKkiQYPHqydO3fWqLfjx48rNjZWvr6+cnNzU1BQkJYuXVqteTt37iwfHx8dOHBAkrRu3TrZbDYtW7ZM06ZNU8uWLdWoUSMVFhZKkjIzMzVw4EB5eXmpUaNG6tOnjzZu3FhuuV9++aW6desmNzc33XTTTXrjjTcqXH9F1+Dn5+fr8ccfV2BgoFxdXdWqVSuNGDFCJ0+e1Lp169StWzdJUkxMjHnJwZIlSyRVfA1+UVGRJk2apICAALm6uqpdu3aaPXu2DMNwqLPZbBo7dqxSUlLUqVMnubq66tZbb1Vqamq19iUAANXFEXwAAK5B7777rkpKSjRu3Djl5eVp5syZevDBB3XXXXdp3bp1euqpp/Tdd9/ptdde0+TJk/X222+b877zzjuKjo5WZGSkXnrpJZ05c0YLFy5Ur1699PXXX1frZnFnz55V37599d1332ns2LFq06aNPvjgA40cOVL5+fmaMGFClfP//PPP+vnnn3XzzTc7jM+YMUMuLi6aPHmyiouL5eLiojVr1ujuu+9WSEiIEhIS5OTkpMWLF+uuu+7Shg0b1L17d0nSN998owEDBqhZs2ZKTEzUhQsXlJCQIF9f34tuz+nTp9W7d2/t3r1bo0aNUteuXXXy5El9+umnOnLkiDp06KDnnntO8fHxGjNmjHr37i1J6tmzZ4XLMwxD9957r9auXavY2FgFBwdr9erVeuKJJ3T06NFylzp8+eWX+uijj/Too4+qSZMmmjdvnu6//34dOnRI119//UX7BwCgWgwAAHBFPfbYY0Zl/+RGR0cbrVu3Nt8fOHDAkGQ0a9bMyM/PN8enTp1qSDKCgoKM8+fPm+PDhg0zXFxcjHPnzhmGYRinTp0yvL29jdGjRzusJycnx/Dy8io3Xmbw4MEOfcydO9eQZPzzn/80x0pKSozw8HCjcePGRmFhoTkuyYiNjTVOnDhhHD9+3MjMzDT69etnSDLmzJljGIZhrF271pBk3HjjjcaZM2fMee12u9G2bVsjMjLSsNvt5viZM2eMNm3aGP379zfHoqKiDDc3N+OHH34wx3bt2mU4OzuX27+tW7c2oqOjzffx8fGGJOOjjz4qt+1l692yZYshyVi8eHG5mt9+TikpKYYk4/nnn3eo+/Of/2zYbDbju+++c9g/Li4uDmPbtm0zJBmvvfZauXUBAHCpOEUfAIBr0AMPPCAvLy/zfVhYmKRfru9v0KCBw3hJSYmOHj0qSUpLS1N+fr6GDRumkydPmi9nZ2eFhYVp7dq11Vr/qlWr5Ofnp2HDhpljDRs21Pjx43X69Gl98cUXDvVvvfWWmjVrpubNmyssLEwbN25UXFycJk6c6FAXHR0td3d38312drb27dunhx56SD/99JPZb1FRkfr166f169fLbrertLRUq1evVlRUlG644QZz/g4dOigyMvKi2/Ovf/1LQUFBuu+++8pN++0j9qpj1apVcnZ21vjx4x3GJ02aJMMw9J///MdhPCIiQjfddJP5vkuXLvL09NT3339f43UDAFAZTtEHAOAa9OsQK8kM+wEBARWO//zzz5Kkffv2SZLuuuuuCpfr6elZrfX/8MMPatu2bbkb4HXo0MGc/mt//OMfNXbsWNlsNjVp0kS33nqrPDw8yi23TZs2Du/L+o2Ojq60l4KCAhUXF+vs2bNq27Ztuent2rXTqlWrqtye/fv36/7776+ypiZ++OEH+fv7q0mTJg7jle2f336eknTdddeZnxsAALWBgA8AwDXI2dm5RuPG/39jt7JHub3zzjvy8/MrV/fro/+1qVWrVoqIiLho3a+P3kv/7XfWrFnlHk1XpnHjxiouLr7sHuvSxT43AABqAwEfAAALKTsNvHnz5tUK3JVp3bq1tm/fLrvd7nAUf8+ePeb02lDWr6enZ5X9NmvWTO7u7uYR/1/bu3dvtdazY8eOKmtqcqp+69at9fnnn+vUqVMOR/Fre/8AAFATXIMPAICFREZGytPTUy+88ILOnz9fbvqJEyeqtZxBgwYpJydHy5cvN8cuXLig1157TY0bN1afPn1qpd+QkBDddNNNmj17tk6fPl1pv87OzoqMjFRKSooOHTpkTt+9e7dWr1590fXcf//92rZtmz7++ONy08qOopddUpCfn3/R5Q0aNEilpaWaP3++w/grr7wim82mu++++6LLAACgtnEEHwAAC/H09NTChQv18MMPq2vXrho6dKiaNWumQ4cOaeXKlbr99tvLhdKKjBkzRm+88YZGjhyprKwsBQYG6sMPP9TGjRs1d+7ccteeXyonJyf9/e9/1913361bb71VMTExatmypY4ePaq1a9fK09NT//73vyVJ06dPV2pqqnr37q1HH33U/MHh1ltv1fbt26tczxNPPKEPP/xQDzzwgEaNGqWQkBDl5eXp008/VXJysoKCgnTTTTfJ29tbycnJatKkiTw8PBQWFlbuvgGSdM899+jOO+/UM888o4MHDyooKEifffaZPvnkE02cONHhhnoAAFwtBHwAACzmoYcekr+/v1588UXNmjVLxcXFatmypXr37q2YmJhqLcPd3V3r1q3TlClTtHTpUhUWFqpdu3ZavHixRo4cWav99u3bVxkZGZoxY4bmz5+v06dPy8/PT2FhYfp//+//mXVdunTR6tWrFRcXp/j4eLVq1UrTp0/Xjz/+eNGA37hxY23YsEEJCQn6+OOPtXTpUjVv3lz9+vVTq1atJP3ylIClS5dq6tSpeuSRR3ThwgUtXry4woDv5OSkTz/9VPHx8Vq+fLkWL16swMBAzZo1S5MmTarV/QMAQHXZDO7uAgAAAABAvcc1+AAAAAAAWAABHwAAAAAACyDgAwAAAABgAQR8AAAAAAAsgIAPAAAAAIAFEPABAAAAALCABnXdQH1jt9t17NgxNWnSRDabra7bAQAAAABYnGEYOnXqlPz9/eXkVPlxegJ+DR07dkwBAQF13QYAAAAA4Hfm8OHDatWqVaXTCfg11KRJE0m/7FhPT8867gYAAAAAYHWFhYUKCAgw82hlCPg1VHZavqenJwEfAAAAAHDVXOwycW6yBwAAAACABRDwAQAAAACwAAI+AAAAAAAWwDX4AAAAAH53SktLdf78+bpuA5AkNWzYUM7Ozpe9HAI+AAAAgN8NwzCUk5Oj/Pz8um4FcODt7S0/P7+L3kivKgR8AAAAAL8bZeG+efPmatSo0WWFKaA2GIahM2fO6Pjx45KkFi1aXPKyCPgAAAAAfhdKS0vNcH/99dfXdTuAyd3dXZJ0/PhxNW/e/JJP1+cmewAAAAB+F8quuW/UqFEddwKUV/a9vJx7QxDwAQAAAPyucFo+rkW18b0k4AMAAAAAYAEEfAAAAACo50aOHKmoqKi6buOaYbPZlJKSIkk6ePCgbDabsrOzL3l5tbGMq4Gb7OF3L3DKyrpuAUAVDr44uK5bAAD8DlzN/09Y03/bLnbqdkJCgl599VUZhnE5bdVKH4mJidWa39PTU506ddKMGTN011131VaLFQoICNCPP/4oHx+fatWPHDlS+fn55g8El7KMukLABwAAAIBr2I8//mj+9/LlyxUfH6+9e/eaY40bN1bjxo2viT4uZvHixRo4cKBOnjypZ555Rn/4wx+0Y8cO3XjjjeVqz58/r4YNG152387OzvLz86vzZVwN9f4U/QULFigwMFBubm4KCwvT5s2bK61dsmSJbDabw8vNze0qdgsAAAAANePn52e+vLy8ZLPZHMYaN25c7hT9vn37aty4cZo4caKuu+46+fr6atGiRSoqKlJMTIyaNGmim2++Wf/5z38c1rVjxw7dfffdaty4sXx9ffXwww/r5MmTF+2jefPmevnll9WqVSu5uroqODhYqamp5bbF29tbfn5+6tSpkxYuXKizZ88qLS1N0i9H+BcuXKh7771XHh4e+tvf/iZJ+uSTT9S1a1e5ubnpxhtv1PTp03XhwgVzmfv27dMdd9whNzc3dezY0VxemYpOr9+5c6f+8Ic/yNPTU02aNFHv3r21f/9+JSYmaunSpfrkk0/MzLhu3boKl/HFF1+oe/fucnV1VYsWLTRlyhSHvvr27avx48frySefVNOmTeXn51flGQ61oV4H/OXLlysuLk4JCQnaunWrgoKCFBkZqePHj1c6j6enp3788Ufz9cMPP1zFjgEAAADg6li6dKl8fHy0efNmjRs3Tn/961/1wAMPqGfPntq6dasGDBighx9+WGfOnJEk5efn66677tJtt92mr776SqmpqcrNzdWDDz540XW9+uqrmjNnjmbPnq3t27crMjJS9957r/bt21fpPGXPfi8pKTHHEhMTdd999+mbb77RqFGjtGHDBo0YMUITJkzQrl279MYbb2jJkiVm+Lfb7frTn/4kFxcXZWZmKjk5WU899VSVvR49elR33HGHXF1dtWbNGmVlZWnUqFG6cOGCJk+erAcffFADBw40M2PPnj0rXMagQYPUrVs3bdu2TQsXLtRbb72l559/vtxn4OHhoczMTM2cOVPPPfdcuR8galO9PkX/5Zdf1ujRoxUTEyNJSk5O1sqVK/X2229rypQpFc5T9isTAAAAAFhZUFCQpk2bJkmaOnWqXnzxRfn4+Gj06NGSpPj4eC1cuFDbt29Xjx49NH/+fN1222164YUXzGW8/fbbCggI0Lfffqtbbrml0nXNnj1bTz31lIYOHSpJeumll7R27VrNnTtXCxYsKFd/5swZTZs2Tc7OzurTp485/tBDD5n5TpJGjRqlKVOmKDo6WpJ04403asaMGXryySeVkJCgzz//XHv27NHq1avl7+8vSXrhhRd09913V9rrggUL5OXlpWXLlpmXAPx629zd3VVcXFxlbnz99dcVEBCg+fPny2azqX379jp27JieeuopxcfHy8npl2PpXbp0UUJCgiSpbdu2mj9/vtLT09W/f/9Kl3056m3ALykpUVZWlqZOnWqOOTk5KSIiQhkZGZXOd/r0abVu3Vp2u11du3bVCy+8oFtvvbXS+uLiYhUXF5vvCwsLa2cDAAAAAOAK6tKli/nfzs7Ouv7669W5c2dzzNfXV5LMM6C3bdumtWvXVngt/f79+ysN+IWFhTp27Jhuv/12h/Hbb79d27ZtcxgbNmyYnJ2ddfbsWTVr1kxvvfWWQ5+hoaEO9du2bdPGjRvNI/aSVFpaqnPnzunMmTPavXu3AgICzHAvSeHh4RXvkP9fdna2evfufVnX9+/evVvh4eEONw68/fbbdfr0aR05ckQ33HCDJMfPQJJatGhR5Rnnl6veBvyTJ0+qtLTU/FKW8fX11Z49eyqcp127dnr77bfVpUsXFRQUaPbs2erZs6d27typVq1aVThPUlKSpk+fXuv9AwAAAMCV9NsAa7PZHMbKwqndbpf0y8HQe+65Ry+99FK5ZbVo0aJWenrllVcUEREhLy8vNWvWrNx0Dw8Ph/enT5/W9OnT9ac//alc7aXeT63s0oCroaLPoGx/Xwn1NuBfivDwcIdfc3r27KkOHTrojTfe0IwZMyqcZ+rUqYqLizPfFxYWKiAg4Ir3CgAAAABXU9euXfWvf/1LgYGBatCg+lHR09NT/v7+2rhxo8Pp9hs3blT37t0dav38/HTzzTfXqKe9e/dWOk+HDh10+PBh/fjjj+aPEP/3f/9X5TK7dOmipUuXVnqXfhcXF5WWlla5jA4dOuhf//qXDMMwfyjZuHGjmjRpUunB46uh3t5kz8fHR87OzsrNzXUYz83NrfY19g0bNtRtt92m7777rtIaV1dXeXp6OrwAAAAAwGoee+wx5eXladiwYdqyZYv279+v1atXKyYm5qKB94knntBLL72k5cuXa+/evZoyZYqys7M1YcKEy+opPj5e//jHPzR9+nTt3LlTu3fv1rJly8x7C0REROiWW25RdHS0tm3bpg0bNuiZZ56pcpljx45VYWGhhg4dqq+++kr79u3TO++8Yz7yLzAwUNu3b9fevXt18uRJnT9/vtwyHn30UR0+fFjjxo3Tnj179MknnyghIUFxcXHm9fd1od4GfBcXF4WEhCg9Pd0cs9vtSk9Pv+g1F2VKS0v1zTff1NrpJgAAAABQX5UdhS8tLdWAAQPUuXNnTZw4Ud7e3hcNrePHj1dcXJwmTZqkzp07KzU1VZ9++qnatm17WT1FRkZqxYoV+uyzz9StWzf16NFDr7zyilq3bi3pl/uwffzxxzp79qy6d++uv/zlLw7X61fk+uuv15o1a3T69Gn16dNHISEhWrRokXk0f/To0WrXrp1CQ0PVrFkzbdy4sdwyWrZsqVWrVmnz5s0KCgrSI488otjYWPOHh7piMwzDqNMOLsPy5csVHR2tN954Q927d9fcuXP1/vvva8+ePfL19dWIESPUsmVLJSUlSZKee+459ejRQzfffLPy8/M1a9YspaSkKCsrSx07dqzWOgsLC+Xl5aWCggKO5ltE4JSVdd0CgCocfHFwXbcAALCIc+fO6cCBA2rTps0lX78NXClVfT+rm0Pr9TX4Q4YM0YkTJxQfH6+cnBwFBwcrNTXVvPHeoUOHHH5p+vnnnzV69Gjl5OTouuuuU0hIiDZt2lTtcA8AAAAAwLWqXh/BrwscwbcejuAD1zaO4AMAagtH8HEtq40j+PX2GnwAAAAAAPBfBHwAAAAAACyAgA8AAAAAgAUQ8AEAAAD8rtjt9rpuASinNr6X9fou+gAAAABQXS4uLnJyctKxY8fUrFkzubi4yGaz1XVb+J0zDEMlJSU6ceKEnJyc5OLicsnLIuADAAAA+F1wcnJSmzZt9OOPP+rYsWN13Q7goFGjRrrhhhscHvVeUwR8AAAAAL8bLi4uuuGGG3ThwgWVlpbWdTuAJMnZ2VkNGjS47DNKCPgAAAAAfldsNpsaNmyohg0b1nUrQK3iJnsAAAAAAFgAAR8AAAAAAAsg4AMAAAAAYAEEfAAAAAAALICADwAAAACABRDwAQAAAACwAAI+AAAAAAAWQMAHAAAAAMACCPgAAAAAAFgAAR8AAAAAAAsg4AMAAAAAYAEEfAAAAAAALICADwAAAACABRDwAQAAAACwAAI+AAAAAAAWQMAHAAAAAMACCPgAAAAAAFgAAR8AAAAAAAsg4AMAAAAAYAEEfAAAAAAALICADwAAAACABRDwAQAAAACwAAI+AAAAAAAWQMAHAAAAAMACCPgAAAAAAFgAAR8AAAAAAAsg4AMAAAAAYAEEfAAAAAAALICADwAAAACABRDwAQAAAACwAAI+AAAAAAAWQMAHAAAAAMACCPgAAAAAAFhAvQ/4CxYsUGBgoNzc3BQWFqbNmzdXa75ly5bJZrMpKirqyjYIAAAAAMBVUK8D/vLlyxUXF6eEhARt3bpVQUFBioyM1PHjx6uc7+DBg5o8ebJ69+59lToFAAAAAODKqtcB/+WXX9bo0aMVExOjjh07Kjk5WY0aNdLbb79d6TylpaUaPny4pk+frhtvvPEqdgsAAAAAwJVTbwN+SUmJsrKyFBERYY45OTkpIiJCGRkZlc733HPPqXnz5oqNja3WeoqLi1VYWOjwAgAAAADgWlNvA/7JkydVWloqX19fh3FfX1/l5ORUOM+XX36pt956S4sWLar2epKSkuTl5WW+AgICLqtvAAAAAACuhHob8Gvq1KlTevjhh7Vo0SL5+PhUe76pU6eqoKDAfB0+fPgKdgkAAAAAwKVpUNcNXCofHx85OzsrNzfXYTw3N1d+fn7l6vfv36+DBw/qnnvuMcfsdrskqUGDBtq7d69uuummcvO5urrK1dW1lrsHAAAAAKB21dsj+C4uLgoJCVF6ero5ZrfblZ6ervDw8HL17du31zfffKPs7Gzzde+99+rOO+9UdnY2p94DAAAAAOq1ensEX5Li4uIUHR2t0NBQde/eXXPnzlVRUZFiYmIkSSNGjFDLli2VlJQkNzc3derUyWF+b29vSSo3DgAAAABAfVOvA/6QIUN04sQJxcfHKycnR8HBwUpNTTVvvHfo0CE5OdXbkxQAAAAAAKg2m2EYRl03UZ8UFhbKy8tLBQUF8vT0rOt2UAsCp6ys6xYAVOHgi4PrugUAAIA6Vd0cyuFtAAAAAAAsgIAPAAAAAIAFEPABAAAAALAAAj4AAAAAABZAwAcAAAAAwAII+AAAAAAAWAABHwAAAAAACyDgAwAAAABgAQR8AAAAAAAsgIAPAAAAAIAFEPABAAAAALAAAj4AAAAAABZAwAcAAAAAwAII+AAAAAAAWAABHwAAAAAACyDgAwAAAABgAQR8AAAAAAAsgIAPAAAAAIAFEPABAAAAALAAAj4AAAAAABZAwAcAAAAAwAII+AAAAAAAWAABHwAAAAAACyDgAwAAAABgAQR8AAAAAAAsgIAPAAAAAIAFEPABAAAAALAAAj4AAAAAABZAwAcAAAAAwAII+AAAAAAAWAABHwAAAAAACyDgAwAAAABgAQR8AAAAAAAsgIAPAAAAAIAFEPABAAAAALAAAj4AAAAAABZAwAcAAAAAwAII+AAAAAAAWAABHwAAAAAACyDgAwAAAABgAQR8AAAAAAAsoN4H/AULFigwMFBubm4KCwvT5s2bK6396KOPFBoaKm9vb3l4eCg4OFjvvPPOVewWAAAAAIAro14H/OXLlysuLk4JCQnaunWrgoKCFBkZqePHj1dY37RpUz3zzDPKyMjQ9u3bFRMTo5iYGK1evfoqdw4AAAAAQO2yGYZh1HUTlyosLEzdunXT/PnzJUl2u10BAQEaN26cpkyZUq1ldO3aVYMHD9aMGTOqVV9YWCgvLy8VFBTI09PzknvHtSNwysq6bgFAFQ6+OLiuWwAAAKhT1c2h9fYIfklJibKyshQREWGOOTk5KSIiQhkZGRed3zAMpaena+/evbrjjjsqrSsuLlZhYaHDCwAAAACAa029DfgnT55UaWmpfH19HcZ9fX2Vk5NT6XwFBQVq3LixXFxcNHjwYL322mvq379/pfVJSUny8vIyXwEBAbW2DQAAAAAA1JZ6G/AvVZMmTZSdna0tW7bob3/7m+Li4rRu3bpK66dOnaqCggLzdfjw4avXLAAAAAAA1dSgrhu4VD4+PnJ2dlZubq7DeG5urvz8/Cqdz8nJSTfffLMkKTg4WLt371ZSUpL69u1bYb2rq6tcXV1rrW8AAAAAAK6EensE38XFRSEhIUpPTzfH7Ha70tPTFR4eXu3l2O12FRcXX4kWAQAAAAC4aurtEXxJiouLU3R0tEJDQ9W9e3fNnTtXRUVFiomJkSSNGDFCLVu2VFJSkqRfrqcPDQ3VTTfdpOLiYq1atUrvvPOOFi5cWJebAQAAAADAZavXAX/IkCE6ceKE4uPjlZOTo+DgYKWmppo33jt06JCcnP57kkJRUZEeffRRHTlyRO7u7mrfvr3++c9/asiQIXW1CQAAAAAA1AqbYRhGXTdRn1T3+YOoPwKnrKzrFgBU4eCLg+u6BQAAgDpV3Rxab6/BBwAAAAAA/0XABwAAAADAAgj4AAAAAABYAAEfAAAAAAALIOADAAAAAGABBHwAAAAAACyAgA8AAAAAgAUQ8AEAAAAAsAACPgAAAAAAFkDABwAAAADAAgj4AAAAAABYAAEfAAAAAAALIOADAAAAAGABBHwAAAAAACyAgA8AAAAAgAUQ8AEAAAAAsAACPgAAAAAAFkDABwAAAADAAgj4AAAAAABYAAEfAAAAAAALIOADAAAAAGABBHwAAAAAACyAgA8AAAAAgAUQ8AEAAAAAsAACPgAAAAAAFkDABwAAAADAAgj4AAAAAABYAAEfAAAAAAALIOADAAAAAGABBHwAAAAAACyAgA8AAAAAgAUQ8AEAAAAAsAACPgAAAAAAFkDABwAAAADAAgj4AAAAAABYAAEfAAAAAAALIOADAAAAAGABBHwAAAAAACyAgA8AAAAAgAUQ8AEAAAAAsAACPgAAAAAAFlDvA/6CBQsUGBgoNzc3hYWFafPmzZXWLlq0SL1799Z1112n6667ThEREVXWAwAAAABQX9TrgL98+XLFxcUpISFBW7duVVBQkCIjI3X8+PEK69etW6dhw4Zp7dq1ysjIUEBAgAYMGKCjR49e5c4BAAAAAKhdNsMwjLpu4lKFhYWpW7dumj9/viTJbrcrICBA48aN05QpUy46f2lpqa677jrNnz9fI0aMqNY6CwsL5eXlpYKCAnl6el5W/7g2BE5ZWdctAKjCwRcH13ULAAAAdaq6ObTeHsEvKSlRVlaWIiIizDEnJydFREQoIyOjWss4c+aMzp8/r6ZNm1ZaU1xcrMLCQocXAAAAAADXmnob8E+ePKnS0lL5+vo6jPv6+ionJ6day3jqqafk7+/v8CPBbyUlJcnLy8t8BQQEXFbfAAAAAABcCfU24F+uF198UcuWLdPHH38sNze3SuumTp2qgoIC83X48OGr2CUAAAAAANXToK4buFQ+Pj5ydnZWbm6uw3hubq78/PyqnHf27Nl68cUX9fnnn6tLly5V1rq6usrV1fWy+wUAAAAA4Eqqt0fwXVxcFBISovT0dHPMbrcrPT1d4eHhlc43c+ZMzZgxQ6mpqQoNDb0arQIAAAAAcMXV2yP4khQXF6fo6GiFhoaqe/fumjt3roqKihQTEyNJGjFihFq2bKmkpCRJ0ksvvaT4+Hi99957CgwMNK/Vb9y4sRo3blxn2wEAAAAAwOWq1wF/yJAhOnHihOLj45WTk6Pg4GClpqaaN947dOiQnJz+e5LCwoULVVJSoj//+c8Oy0lISFBiYuLVbB0AAAAAgFplMwzDqOsm6pPqPn8Q9UfglJV13QKAKhx8cXBdtwAAAFCnqptD6+01+AAAAAAA4L8I+AAAAAAAWAABHwAAAAAACyDgAwAAAABgAQR8AAAAAAAsgIAPAAAAAIAFEPABAAAAALAAAj4AAAAAABZAwAcAAAAAwAII+AAAAAAAWAABHwAAAAAACyDgAwAAAABgAQR8AAAAAAAsgIAPAAAAAIAFEPABAAAAALAAAj4AAAAAABZAwAcAAAAAwAII+AAAAAAAWAABHwAAAAAACyDgAwAAAABgAQR8AAAAAAAsgIAPAAAAAIAFEPABAAAAALAAAj4AAAAAABZAwAcAAAAAwAII+AAAAAAAWAABHwAAAAAACyDgAwAAAABgAQR8AAAAAAAsgIAPAAAAAIAFEPABAAAAALAAAj4AAAAAABZAwAcAAAAAwAII+AAAAAAAWAABHwAAAAAACyDgAwAAAABgAQR8AAAAAAAsgIAPAAAAAIAFEPABAAAAALAAAj4AAAAAABZAwAcAAAAAwALqfcBfsGCBAgMD5ebmprCwMG3evLnS2p07d+r+++9XYGCgbDab5s6de/UaBQAAAADgCqrXAX/58uWKi4tTQkKCtm7dqqCgIEVGRur48eMV1p85c0Y33nijXnzxRfn5+V3lbgEAAAAAuHLqdcB/+eWXNXr0aMXExKhjx45KTk5Wo0aN9Pbbb1dY361bN82aNUtDhw6Vq6vrVe4WAAAAAIArp94G/JKSEmVlZSkiIsIcc3JyUkREhDIyMmptPcXFxSosLHR4AQAAAABwram3Af/kyZMqLS2Vr6+vw7ivr69ycnJqbT1JSUny8vIyXwEBAbW2bAAAAAAAaku9DfhXy9SpU1VQUGC+Dh8+XNctAQAAAABQToO6buBS+fj4yNnZWbm5uQ7jubm5tXoDPVdXV67XBwAAAABc8+rtEXwXFxeFhIQoPT3dHLPb7UpPT1d4eHgddgYAAAAAwNVXb4/gS1JcXJyio6MVGhqq7t27a+7cuSoqKlJMTIwkacSIEWrZsqWSkpIk/XJjvl27dpn/ffToUWVnZ6tx48a6+eab62w7AAAAAAC4XPU64A8ZMkQnTpxQfHy8cnJyFBwcrNTUVPPGe4cOHZKT039PUjh27Jhuu+028/3s2bM1e/Zs9enTR+vWrbva7QMAAAAAUGtshmEYdd1EfVJYWCgvLy8VFBTI09OzrttBLQicsrKuWwBQhYMvDq7rFgAAAOpUdXNovb0GHwAAAAAA/BcBHwAAAAAACyDgAwAAAABgAQR8AAAAAAAsgIAPAAAAAIAFEPABAAAAALAAAj4AAAAAABZAwAcAAAAAwAII+AAAAAAAWAABHwAAAAAACyDgAwAAAABgAQR8AAAAAAAsgIAPAAAAAIAFEPABAAAAALAAAj4AAAAAABZAwAcAAAAAwAII+AAAAAAAWAABHwAAAAAAC2hQ1w0AAACgfgqcsrKuWwBQhYMvDq7rFnCVcQQfAAAAAAALIOADAAAAAGABBHwAAAAAACyAgA8AAAAAgAUQ8AEAAAAAsAACPgAAAAAAFkDABwAAAADAAgj4AAAAAABYAAEfAAAAAAALIOADAAAAAGABBHwAAAAAACyAgA8AAAAAgAUQ8AEAAAAAsAACPgAAAAAAFkDABwAAAADAAgj4AAAAAABYAAEfAAAAAAALIOADAAAAAGABBHwAAAAAACyAgA8AAAAAgAUQ8AEAAAAAsAACPgAAAAAAFlDvA/6CBQsUGBgoNzc3hYWFafPmzVXWf/DBB2rfvr3c3NzUuXNnrVq16ip1CgAAAADAlVOvA/7y5csVFxenhIQEbd26VUFBQYqMjNTx48crrN+0aZOGDRum2NhYff3114qKilJUVJR27NhxlTsHAAAAAKB21euA//LLL2v06NGKiYlRx44dlZycrEaNGuntt9+usP7VV1/VwIED9cQTT6hDhw6aMWOGunbtqvnz51/lzgEAAAAAqF0N6rqBS1VSUqKsrCxNnTrVHHNyclJERIQyMjIqnCcjI0NxcXEOY5GRkUpJSal0PcXFxSouLjbfFxQUSJIKCwsvo3tcS+zFZ+q6BQBV4H9vgWsX/4YC1zb+DbWOss/SMIwq6+ptwD958qRKS0vl6+vrMO7r66s9e/ZUOE9OTk6F9Tk5OZWuJykpSdOnTy83HhAQcAldAwBqymtuXXcAAED9xL+h1nPq1Cl5eXlVOr3eBvyrZerUqQ5H/e12u0JCQrR161bZbLY67Ay1pVu3btqyZUtdt4FaUFhYqICAAB0+fFienp513Q5qCX9HrYXP01r4PK2Df0Otib+j1mEYhkJCQuTv719lXb0N+D4+PnJ2dlZubq7DeG5urvz8/Cqcx8/Pr0b1kuTq6ipXV9dyY1X9aoL6xdnZmX/ILMbT05PP1EL4O2otfJ7WwudpPfwbai38HbUWFxcXOTlVfRu9enuTPRcXF4WEhCg9Pd0cs9vtSk9PV3h4eIXzhIeHO9RLUlpaWqX1lXnsscdq3jCuWXyewLWNv6PWwudpLXyewLWNv6PWUp3P02Zc7Cr9a9jy5csVHR2tN954Q927d9fcuXP1/vvva8+ePfL19dWIESPUsmVLJSUlSfrlMXl9+vTRiy++qMGDB2vZsmV64YUXtHXrVnXq1KmOtwbA5SosLJSXl5cKCgr4tRoAgBrg31DAGurtKfqSNGTIEJ04cULx8fHKyclRcHCwUlNTzRvpHTp0yOEUhp49e+q9997TtGnT9PTTT6tt27ZKSUkh3AMW4erqqoSEhHKX1QAAgKrxbyhgDfX6CD4AAAAAAPhFvb0GHwAAAAAA/BcBHwAAAAAACyDgAwAAAABgAQR8AAAAAAAsgIAPAAAAAIAFEPABAAAAALAAAj4AAAAAABZAwAcAAAAAwAII+AAAAAAAWAABHwAAAAAACyDgAwAAAABgAQR8AAAAAAAsgIAPAAAAAIAFEPABAAAAALAAAj4AAAAAABZAwAcAAAAAwAII+AAAAAAAWAABHwAAAAAACyDgAwAAAABgAQR8AAAAAAAsgIAPAAAAAIAFEPABAAAAALAAAj4AAAAAABZAwAcAAAAAwAII+AAAAAAAWAABHwAAAAAACyDgAwAAAABgAQR8AEC9N3LkSAUGBtZ1G/VaYmKibDabw1hgYKBGjhxZa+uo689p8+bNcnFx0Q8//HDV1nk5+zAwMFB/+MMfarehGvrtZ/bTTz/Jw8NDq1atqrumAACVIuADAK5JNputWq9169bVSX8HDx6sdo8HDx686PJ+Xe/k5CR/f38NGDCgzrbvUh07dkyJiYnKzs6u61bKeeaZZzRs2DC1bt26rlupVbt27VJiYmK1vmeX6/rrr9df/vIXPfvss1d8XQCAmmtQ1w0AAFCRd955x+H9P/7xD6WlpZUb79ChgxYtWiS73X4121OzZs3K9TJnzhwdOXJEr7zySrna6ujfv79GjBghwzB04MABvf7667rrrru0cuVK3X333bXWe3Xt3btXTk41OxZw7NgxTZ8+XYGBgQoODnaYVhefU5ns7Gx9/vnn2rRp01Vd76Xsw5ratWuXpk+frr59+16VMyQeeeQRzZs3T2vWrNFdd911xdcHAKg+Aj4A4Jr0P//zPw7v/+///k9paWnlxuuKh4dHuV6WLVumn3/++ZJ7vOWWWxzmve+++9SlSxfNnTu30oB/7tw5ubi4XJEQ6erqWqvLa9iwYa0uryYWL16sG264QT169Liq663tfXgt6NChgzp16qQlS5YQ8AHgGsMp+gCAeu+31wmXnT4/e/ZsLViwQDfeeKMaNWqkAQMG6PDhwzIMQzNmzFCrVq3k7u6uP/7xj8rLyyu33P/85z/q3bu3PDw81KRJEw0ePFg7d+6sUW/Hjx9XbGysfH195ebmpqCgIC1durRa83bu3Fk+Pj46cOCAJGndunWy2WxatmyZpk2bppYtW6pRo0YqLCyUJGVmZmrgwIHy8vJSo0aN1KdPH23cuLHccr/88kt169ZNbm5uuummm/TGG29UuP6Krh/Pz8/X448/rsDAQLm6uqpVq1YaMWKETp48qXXr1qlbt26SpJiYGPOSgyVLlkiq+Br8oqIiTZo0SQEBAXJ1dVW7du00e/ZsGYbhUGez2TR27FilpKSoU6dOcnV11a233qrU1NRq7cuUlBTdddddDvcZiIuL0/XXX++wrnHjxslms2nevHnmWG5urmw2mxYuXGiOFRcXKyEhQTfffLNcXV0VEBCgJ598UsXFxRfdh9u3b1efPn3k7u6uVq1a6fnnn9fixYsrvZzjyy+/VPfu3eXm5qYbb7xR//jHP8xpS5Ys0QMPPCBJuvPOOyu8dKW63+Oyfevm5qZOnTrp448/rnR/9u/fX//+97/LfU4AgLrFEXwAgGW9++67Kikp0bhx45SXl6eZM2fqwQcf1F133aV169bpqaee0nfffafXXntNkydP1ttvv23O+8477yg6OlqRkZF66aWXdObMGS1cuFC9evXS119/Xa1Toc+ePau+ffvqu+++09ixY9WmTRt98MEHGjlypPLz8zVhwoQq5//555/1888/6+abb3YYnzFjhlxcXDR58mQVFxfLxcVFa9as0d13362QkBAlJCTIyclJixcv1l133aUNGzaoe/fukqRvvvlGAwYMULNmzZSYmKgLFy4oISFBvr6+F92e06dPq3fv3tq9e7dGjRqlrl276uTJk/r000915MgRdejQQc8995zi4+M1ZswY9e7dW5LUs2fPCpdnGIbuvfderV27VrGxsQoODtbq1av1xBNP6OjRo+Uudfjyyy/10Ucf6dFHH1WTJk00b9483X///Tp06JCuv/76Svs+evSoDh06pK5duzqM9+7dW6+88op27typTp06SZI2bNggJycnbdiwQePHjzfHJOmOO+6QJNntdt1777368ssvNWbMGHXo0EHffPONXnnlFX377bdKSUmpspeyID516lR5eHjo73//e6VH+r/77jv9+c9/VmxsrKKjo/X2229r5MiRCgkJ0a233qo77rhD48eP17x58/T000+rQ4cOkmT+Wd3v8Weffab7779fHTt2VFJSkn766SfFxMSoVatWFfYVEhJSbt8BAK4BBgAA9cBjjz1mVPbPVnR0tNG6dWvz/YEDBwxJRrNmzYz8/HxzfOrUqYYkIygoyDh//rw5PmzYMMPFxcU4d+6cYRiGcerUKcPb29sYPXq0w3pycnIMLy+vcuNlBg8e7NDH3LlzDUnGP//5T3OspKTECA8PNxo3bmwUFhaa45KM2NhY48SJE8bx48eNzMxMo1+/foYkY86cOYZhGMbatWsNScaNN95onDlzxpzXbrcbbdu2NSIjIw273W6OnzlzxmjTpo3Rv39/cywqKspwc3MzfvjhB3Ns165dhrOzc7n927p1ayM6Otp8Hx8fb0gyPvroo3LbXrbeLVu2GJKMxYsXl6v57eeUkpJiSDKef/55h7o///nPhs1mM7777juH/ePi4uIwtm3bNkOS8dprr5Vb1699/vnnhiTj3//+t8P48ePHDUnG66+/bhiGYeTn5xtOTk7GAw88YPj6+pp148ePN5o2bWpu4zvvvGM4OTkZGzZscFhecnKyIcnYuHGjOfbbfThu3DjDZrMZX3/9tTn2008/GU2bNjUkGQcOHHCYV5Kxfv16h55dXV2NSZMmmWMffPCBIclYu3atQz81+R4HBwcbLVq0cPj78tlnnxmSHD6zMps2bTIkGcuXLy83DQBQdzhFHwBgWQ888IC8vLzM92FhYZJ+ub6/QYMGDuMlJSU6evSoJCktLU35+fkaNmyYTp48ab6cnZ0VFhamtWvXVmv9q1atkp+fn4YNG2aONWzYUOPHj9fp06f1xRdfONS/9dZbatasmZo3b66wsDBt3LhRcXFxmjhxokNddHS03N3dzffZ2dnat2+fHnroIf30009mv0VFRerXr5/Wr18vu92u0tJSrV69WlFRUbrhhhvM+Tt06KDIyMiLbs+//vUvBQUF6b777is37beP2KuOVatWydnZ2TxSXmbSpEkyDEP/+c9/HMYjIiJ00003me+7dOkiT09Pff/991Wu56effpIkXXfddQ7jzZo1U/v27bV+/XpJ0saNG+Xs7KwnnnhCubm52rdvn6RfjuD36tXL3MYPPvhAHTp0UPv27R2+H2XXo1f1/UhNTVV4eLjDDQibNm2q4cOHV1jfsWNH80yIsp7btWt30W2Wqv89/vHHH5Wdna3o6GiHvy/9+/dXx44dK1x22b48efLkRfsAAFw9nKIPALCsX4dYSWZ4CQgIqHD8559/liQz2FV2AzFPT89qrf+HH35Q27Zty90Ar+z06d8+j/2Pf/yjxo4dK5vNpiZNmujWW2+Vh4dHueW2adPG4X1Zv9HR0ZX2UlBQoOLiYp09e1Zt27YtN71du3YXfbb5/v37df/991dZUxM//PCD/P391aRJE4fxyvbPbz9P6ZegWfa5XYxRwfXivXv3Nrd7w4YNCg0NVWhoqJo2baoNGzbI19dX27Zt00MPPWTOs2/fPu3evbvSpyMcP3680h5++OEHhYeHlxv/7WUYZS5nm6v7PS7bz5V9L7Zu3VpuvGxfXsoPOwCAK4eADwCwLGdn5xqNl4WWske5vfPOO/Lz8ytX9+uj/7WpVatWioiIuGjdr4/eS//td9asWeUeTVemcePG5W4AV99c7HOrTNn1+RWF4l69emnRokX6/vvvtWHDBvXu3Vs2m029evXShg0b5O/vL7vd7nAU3W63q3Pnznr55ZcrXN9vf0C6HJe6zdKV/R6X7UsfH59LXgYAoPYR8AEA+I2y08CbN29ercBdmdatW2v79u2y2+0OR/H37NljTq8NZf16enpW2W+zZs3k7u5uHtn9tb1791ZrPTt27KiypiZHdFu3bq3PP/9cp06dcjiKX9v7p3379pJkPo3g18qCe1pamrZs2aIpU6ZI+uWGegsXLpS/v788PDwUEhJiznPTTTdp27Zt6tevX42PYLdu3VrfffddufGKxqqrsh6q+z0u2881+V6U7cuysy0AANcGrsEHAOA3IiMj5enpqRdeeEHnz58vN/3EiRPVWs6gQYOUk5Oj5cuXm2MXLlzQa6+9psaNG6tPnz610m9ISIhuuukmzZ49W6dPn660X2dnZ0VGRiolJUWHDh0yp+/evVurV6++6Hruv/9+bdu2rcLHp5UdUS67pCA/P/+iyxs0aJBKS0s1f/58h/FXXnlFNptNd99990WXUR0tW7ZUQECAvvrqq3LT2rRpo5YtW+qVV17R+fPndfvtt0v6Jfjv379fH374oXr06OFwtPvBBx/U0aNHtWjRonLLO3v2rIqKiirtJTIyUhkZGcrOzjbH8vLy9O67717y9lW2z6v7PW7RooWCg4O1dOlSFRQUmNPT0tK0a9euCteZlZUlLy8v3XrrrZfcNwCg9nEEHwCA3/D09NTChQv18MMPq2vXrho6dKiaNWumQ4cOaeXKlbr99tvLhdKKjBkzRm+88YZGjhyprKwsBQYG6sMPP9TGjRs1d+7ccteeXyonJyf9/e9/1913361bb71VMTExatmypY4ePaq1a9fK09NT//73vyVJ06dPV2pqqnr37q1HH33U/MHh1ltv1fbt26tczxNPPKEPP/xQDzzwgEaNGqWQkBDl5eXp008/VXJysoKCgnTTTTfJ29tbycnJatKkiTw8PBQWFlbuvgGSdM899+jOO+/UM888o4MHDyooKEifffaZPvnkE02cONHhhnqX649//KM+/vhjGYZR7oh37969tWzZMnXu3Nm8eVzXrl3l4eGhb7/91uH6e0l6+OGH9f777+uRRx7R2rVrdfvtt6u0tFR79uzR+++/r9WrVys0NLTCPp588kn985//VP/+/TVu3DjzMXk33HCD8vLyLuma9uDgYDk7O+ull15SQUGBXF1dddddd6l58+bV/h4nJSVp8ODB6tWrl0aNGqW8vDzze1HRj0ZpaWm65557uAYfAK4xHMEHAKACDz30kNLT09WyZUvNmjVLEyZM0LJlyxQcHKyYmJhqLcPd3V3r1q3T8OHDtXTpUk2aNEl5eXlavHixJkyYUKv99u3bVxkZGQoNDdX8+fM1btw4LVmyRH5+fnr88cfNui5dumj16tVq1qyZ4uPj9fbbb2v69OkV3hn/txo3bqwNGzbor3/9q1atWqXx48fr9ddfV7t27cznpTds2FBLly6Vs7OzHnnkEQ0bNqzc0wLKODk56dNPP9XEiRO1YsUKTZw4Ubt27dKsWbMqvb79Uo0aNUpHjx7Vxo0by00rO02/V69e5liDBg3Mm+H9+vr7sr5TUlL04osv6ptvvtHkyZM1ffp0bdmyRRMmTNAtt9xSaR8BAQFau3atOnTooBdeeEFz585VdHS0Ro0aJUlyc3Or8bb5+fkpOTlZx48fV2xsrIYNG2Yeea/u93jgwIH64IMPVFpaqqlTp+qjjz7S4sWLK/yhYs+ePdqxY4dGjhxZ414BAFeWzajOXVoAAADquX79+snf31/vvPNOXbdSzsSJE/XGG2/o9OnTld5Y71oxceJErV+/XllZWRzBB4BrDAEfAAD8LmRmZqp3797at29frd3A71KcPXvW4UkIP/30k2655RZ17dpVaWlpddZXdfz0009q3bq13n//fQ0aNKiu2wEA/AYBHwAA4CoKDg5W37591aFDB+Xm5uqtt97SsWPHlJ6erjvuuKOu2wMA1GPcZA8AAOAqGjRokD788EO9+eabstls6tq1q9566y3CPQDgsnEEHwAAAAAAC+Au+gAAAAAAWAABHwAAAAAAC+Aa/Bqy2+06duyYmjRpwqNhAAAAAABXnGEYOnXqlPz9/eXkVPlxegJ+DR07dkwBAQF13QYAAAAA4Hfm8OHDatWqVaXTCfg11KRJE0m/7FhPT8867gYAAAAAYHWFhYUKCAgw82hlCPg1VHZavqenJwEfAAAAAHDVXOwycW6yBwAAAACABVxSwF+wYIECAwPl5uamsLAwbd68ucr6Dz74QO3bt5ebm5s6d+6sVatWOUw3DEPx8fFq0aKF3N3dFRERoX379jnU5OXlafjw4fL09JS3t7diY2N1+vRpc/q5c+c0cuRIde7cWQ0aNFBUVFSFvRQXF+uZZ55R69at5erqqsDAQL399tuXshsAAAAAALhm1DjgL1++XHFxcUpISNDWrVsVFBSkyMhIHT9+vML6TZs2adiwYYqNjdXXX3+tqKgoRUVFaceOHWbNzJkzNW/ePCUnJyszM1MeHh6KjIzUuXPnzJrhw4dr586dSktL04oVK7R+/XqNGTPGnF5aWip3d3eNHz9eERERlfb/4IMPKj09XW+99Zb27t2r//3f/1W7du1quhsAAAAAALim2AzDMGoyQ1hYmLp166b58+dL+uWxcQEBARo3bpymTJlSrn7IkCEqKirSihUrzLEePXooODhYycnJMgxD/v7+mjRpkiZPnixJKigokK+vr5YsWaKhQ4dq9+7d6tixo7Zs2aLQ0FBJUmpqqgYNGqQjR47I39/fYZ0jR45Ufn6+UlJSHMZTU1M1dOhQff/992ratGlNNttUWFgoLy8vFRQUcA0+AAAA6ozdbldJSUldtwGgFjRs2FDOzs6VTq9uDq3RTfZKSkqUlZWlqVOnmmNOTk6KiIhQRkZGhfNkZGQoLi7OYSwyMtIM3wcOHFBOTo7DUXcvLy+FhYUpIyNDQ4cOVUZGhry9vc1wL0kRERFycnJSZmam7rvvvmr1/+mnnyo0NFQzZ87UO++8Iw8PD917772aMWOG3N3dq7sbAAAAgDpVUlKiAwcOyG6313UrAGqJt7e3/Pz8LnojvarUKOCfPHlSpaWl8vX1dRj39fXVnj17KpwnJyenwvqcnBxzetlYVTXNmzd3bLxBAzVt2tSsqY7vv/9eX375pdzc3PTxxx/r5MmTevTRR/XTTz9p8eLFFc5TXFys4uJi831hYWG11wcAAADUNsMw9OOPP8rZ2VkBAQFycuK+2UB9ZhiGzpw5Y1723qJFi0te1u/qMXl2u102m03vvvuuvLy8JEkvv/yy/vznP+v111+v8Ch+UlKSpk+ffrVbBQAAACp04cIFnTlzRv7+/mrUqFFdtwOgFpRl0ePHj6t58+ZVnq5flRr93Ofj4yNnZ2fl5uY6jOfm5srPz6/Cefz8/KqsL/vzYjW/vYnfhQsXlJeXV+l6K9KiRQu1bNnSDPeS1KFDBxmGoSNHjlQ4z9SpU1VQUGC+Dh8+XO31AQAAALWttLRUkuTi4lLHnQCoTWU/2J0/f/6Sl1GjgO/i4qKQkBClp6ebY3a7Xenp6QoPD69wnvDwcId6SUpLSzPr27RpIz8/P4eawsJCZWZmmjXh4eHKz89XVlaWWbNmzRrZ7XaFhYVVu//bb79dx44dc3i83rfffisnJye1atWqwnlcXV3l6enp8AIAAADq2uVcpwvg2lMbf6drfMFOXFycFi1apKVLl2r37t3661//qqKiIsXExEiSRowY4XATvgkTJig1NVVz5szRnj17lJiYqK+++kpjx441N2LixIl6/vnn9emnn+qbb77RiBEj5O/vbz7LvkOHDho4cKBGjx6tzZs3a+PGjRo7dqyGDh3qcAf9Xbt2KTs7W3l5eSooKFB2drays7PN6Q899JCuv/56xcTEaNeuXVq/fr2eeOIJjRo1ipvsAQAAAADqtRoH/CFDhmj27NmKj49XcHCwsrOzlZqaat4k79ChQ/rxxx/N+p49e+q9997Tm2++qaCgIH344YdKSUlRp06dzJonn3xS48aN05gxY9StWzedPn1aqampcnNzM2veffddtW/fXv369dOgQYPUq1cvvfnmmw69DRo0SLfddpv+/e9/a926dbrtttt02223mdMbN26stLQ05efnKzQ0VMOHD9c999yjefPm1XQ3AAAAAKhFI0eONA/w4ZcDoWVPHjt48KBsNpvDwcuaqo1lVFdJSYluvvlmbdq06Yosf926dbLZbMrPz6/2PImJiQoODr4i/VSkb9++mjhxovm+R48e+te//nXF12szDMO44muxkOo+fxD1SKLXxWsA1J3EgrruAACuKefOndOBAwfUpk0bhwNiV/3/09Tgf58vdupxQkKCHn/8cRmGIW9v78ts7PL6SExMrNb8np6e6tSpk2bMmKG77rqrtlp0WNfHH3+sqKgolZaW6sSJE/Lx8VGDBhe/T/rIkSOVn59v/kAgqcbLuBzz5s3Tv//9b6WlpV2R5ZeUlCgvL0++vr7VPq09MTFRKSkpVf7A0bdvXwUHB2vu3LmX3eNvl7VixQo9/vjj2rt3b6VPvqj077aqn0N5pgYAAACAK+rHH380X3PnzpWnp6fD2OTJk+Xl5XVFw311+7iYxYsX68cff9TGjRvl4+OjP/zhD/r+++8rrL2cm6X9mrOzs/z8/C4rmNfGMqrDMAzNnz9fsbGxV2wdLi4ul/28+Kvt7rvv1qlTp/Sf//zniq6HgA8AAADgivLz8zNfXl5estlsDmONGzcud4p+3759NW7cOE2cOFHXXXedfH19tWjRIvP+X02aNNHNN99cLjDt2LFDd999txo3bixfX189/PDDOnny5EX7aN68uV5++WW1atVKrq6uCg4OVmpqarlt8fb2lp+fnzp16qSFCxfq7Nmz5pFqm82mhQsX6t5775WHh4f+9re/SZI++eQTde3aVW5ubrrxxhs1ffp0XbhwwVzmvn37dMcdd8jNzU0dO3Ysd+S7otPrd+7cqT/84Q/y9PRUkyZN1Lt3b+3fv1+JiYlaunSpPvnkE9lsNtlsNq1bt67CZXzxxRfq3r27XF1d1aJFC02ZMsWhr759+2r8+PF68skn1bRpU/n5+VV5hoMkZWVlaf/+/Ro8eLA59uc//9m8B5skTZw4UTabTXv27JH0yxF5Dw8Pff7555J+uZF7UlKS2rRpI3d3d/NS7zIVnaK/aNEiBQQEqFGjRrrvvvv08ssvV/iD0TvvvKPAwEB5eXlp6NChOnXqlKRfznr44osv9Oqrr5r77eDBg5Kq/k5JUlFRkUaMGKHGjRurRYsWmjNnTrn1Ojs7a9CgQVq2bFmV++9yEfABAAAAXJOWLl0qHx8fbd68WePGjdNf//pXPfDAA+rZs6e2bt2qAQMG6OGHH9aZM2ckSfn5+brrrrt022236auvvlJqaqpyc3P14IMPXnRdr776qubMmaPZs2dr+/btioyM1L333qt9+/ZVOk/ZjbpLSkrMscTERN1333365ptvNGrUKG3YsEEjRozQhAkTtGvXLr3xxhtasmSJGf7tdrv+9Kc/ycXFRZmZmUpOTtZTTz1VZa9Hjx7VHXfcIVdXV61Zs0ZZWVkaNWqULly4oMmTJ+vBBx/UwIEDzTMTevbsWeEyBg0apG7dumnbtm1auHCh3nrrLT3//PPlPgMPDw9lZmZq5syZeu6556o89X7Dhg265ZZb1KRJE3OsT58+Wrdunfn+iy++kI+Pjzm2ZcsWnT9/3uwzKSlJ//jHP5ScnKydO3fq8ccf1//8z//oiy++qHCdGzdu1COPPKIJEyYoOztb/fv3N/fvr+3fv18pKSlasWKFVqxYoS+++EIvvviipF8+//DwcI0ePdrcbwEBAdX6Tj3xxBP64osv9Mknn+izzz7TunXrtHXr1nLr7969uzZs2FDpvqsNV/b8DAAAAAC4REFBQZo2bZokaerUqXrxxRfl4+Oj0aNHS5Li4+O1cOFCbd++XT169ND8+fN122236YUXXjCX8fbbbysgIEDffvutbrnllkrXNXv2bD311FMaOnSoJOmll17S2rVrNXfuXC1YsKBc/ZkzZzRt2jQ5OzurT58+5vhDDz1kPmFMkkaNGqUpU6YoOjpaknTjjTdqxowZevLJJ5WQkKDPP/9ce/bs0erVq80nhL3wwgu6++67K+11wYIF8vLy0rJly9SwYUNJctg2d3d3FRcXy8/Pr9JlvP766woICND8+fNls9nUvn17HTt2TE899ZTi4+PN68S7dOmihIQESVLbtm01f/58paenq3///hUu94cffnB40pn0y5kAEyZM0IkTJ9SgQQPt2rVLzz77rNatW6dHHnlE69atU7du3dSoUSMVFxfrhRde0Oeff24+Nv3GG2/Ul19+qTfeeMNhX5d57bXXdPfdd5uXWNxyyy3atGmTVqxY4VBnt9u1ZMkS88eHhx9+WOnp6frb3/4mLy8vubi4qFGjRg777WLfKX9/f7311lv65z//qX79+kn65UeRih7D7u/vr8OHD8tut1d6Hf7lIuADAAAAuCZ16dLF/G9nZ2ddf/316ty5szlW9iSv48ePS5K2bdumtWvXqnHjxuWWtX///koDfmFhoY4dO6bbb7/dYfz222/Xtm3bHMaGDRsmZ2dnnT17Vs2aNdNbb73l0GdoaKhD/bZt27Rx40aHI8qlpaU6d+6czpw5o927dysgIMAhFJcF28pkZ2erd+/eZri/FLt371Z4eLjDdey33367Tp8+rSNHjuiGG26Q5PgZSFKLFi3M/V2Rs2fPlrtBXKdOndS0aVN98cUXcnFx0W233aY//OEP5g8nX3zxhfr27StJ+u6773TmzJlyPyCUlJQ4PCHt1/bu3av77rvPYax79+7lAn5gYKDDmQUX2xbp4t+ps2fPqqSkRGFhYeZ406ZN1a5du3L17u7ustvtKi4uvmKPaSfgAwAAALgm/TbA2mw2h7GycGq32yVJp0+f1j333KOXXnqp3LJatGhRKz298sorioiIkJeXl5o1a1ZuuoeHh8P706dPa/r06frTn/5Urva3Qbi6rlQ4rEhFn0HZ/q6Ij4+Pvvnmm3Lz3HHHHVq3bp1cXV3Vt29fdenSRcXFxdqxY4c2bdpkHn0/ffq0JGnlypVq2bKlw3JcXV2v6raU9VPVd+q7776r9vrz8vLk4eFxRT8/Aj4AAAAAS+jatav+9a9/KTAwsEZ3i/f09JS/v782btzocAr4xo0b1b17d4daPz8/3XzzzTXqae/evZXO06FDBx0+fFg//vij+SPE//3f/1W5zC5dumjp0qU6f/58hUfxXVxcVFpaWuUyOnTooH/9618yDMP8oWTjxo1q0qRJhaeXV9dtt92mhQsXOixX+uU6/EWLFsnV1VV/+9vf5OTkpDvuuEOzZs1ScXGxefZEx44d5erqqkOHDlV4On5F2rVrpy1btjiM/fZ9dVS03y72nbrpppvUsGFDZWZmmmc9/Pzzz/r222/L9b9jx45Kz0KoLdxkDwAAAIAlPPbYY8rLy9OwYcO0ZcsW7d+/X6tXr1ZMTMxFA+8TTzyhl156ScuXL9fevXs1ZcoUZWdna8KECZfVU3x8vP7xj39o+vTp2rlzp3bv3q1ly5aZ9xaIiIjQLbfcoujoaG3btk0bNmzQM888U+Uyx44dq8LCQg0dOlRfffWV9u3bp3feeUd79+6V9Mup6Nu3b9fevXt18uTJCh/X9+ijj+rw4cMaN26c9uzZo08++UQJCQmKi4u7rOvD77zzTp0+fVo7d+50GO/bt6927dqlnTt3qlevXubYu+++q9DQUPPMhyZNmmjy5Ml6/PHHtXTpUu3fv19bt27Va6+9pqVLl1a4znHjxmnVqlV6+eWXtW/fPr3xxhv6z3/+U+PH6AUGBiozM1MHDx7UyZMnZbfbL/qdaty4sWJjY/XEE09ozZo12rFjh0aOHFnhPtywYYMGDBhQo55qioAPAAAAwBLKjsKXlpZqwIAB6ty5syZOnChvb++Lhtbx48crLi5OkyZNUufOnZWamqpPP/1Ubdu2vayeIiMjtWLFCn322Wfq1q2bevTooVdeeUWtW7eWJDk5Oenjjz/W2bNn1b17d/3lL3+p8A7wv3b99ddrzZo1On36tPr06aOQkBAtWrTIPJo/evRotWvXTqGhoWrWrJk2btxYbhktW7bUqlWrtHnzZgUFBemRRx5RbGys+cPDpbr++ut133336d1333UY79y5s7y9vRUcHGxez963b1+Vlpaa19+XmTFjhp599lklJSWpQ4cOGjhwoFauXKk2bdpUuM7bb79dycnJevnllxUUFKTU1FQ9/vjjNb4EYvLkyXJ2dlbHjh3VrFkzHTp0qFrfqVmzZql379665557FBERoV69eikkJMRh2UePHtWmTZscbsB4JdgMwzCu6BosprCwUF5eXiooKJCnp2ddt4PakOhV1x0AqEpiQV13AADXlHPnzunAgQNq06bNJV/DDVxJ27dvV//+/bV///4Kb053NYwePVp79uy54o+lq66nnnpKP//8s958881Ka6r6u13dHMoRfAAAAABArenSpYteeuklHThw4Kqtc/bs2dq2bZu+++4783T+skcTXguaN2+uGTNmXPH1cJM9AAAAAECtGjly5FVd3+bNmzVz5kydOnVKN954o+bNm6e//OUvV7WHqkyaNOmqrIeADwAAAACo195///26buGawCn6AAAAAABYAAEfAAAAAAALIOADAAAA9RAPwwKsxW63X/YyuAYfAAAAqEcaNmwom82mEydOqFmzZrLZbHXdEoDLYBiGSkpKdOLECTk5OcnFxeWSl0XABwAAAOoRZ2dntWrVSkeOHNHBgwfruh0AtaRRo0a64YYb5OR06SfaE/ABAACAeqZx48Zq27atzp8/X9etAKgFzs7OatCgwWWfkUPABwAAAOohZ2dnOTs713UbAK4h3GQPAAAAAAALIOADAAAAAGABBHwAAAAAACzgkgL+ggULFBgYKDc3N4WFhWnz5s1V1n/wwQdq37693Nzc1LlzZ61atcphumEYio+PV4sWLeTu7q6IiAjt27fPoSYvL0/Dhw+Xp6envL29FRsbq9OnT5vTz507p5EjR6pz585q0KCBoqKiquxp48aNatCggYKDg2u07QAAAAAAXItqHPCXL1+uuLg4JSQkaOvWrQoKClJkZKSOHz9eYf2mTZs0bNgwxcbG6uuvv1ZUVJSioqK0Y8cOs2bmzJmaN2+ekpOTlZmZKQ8PD0VGRurcuXNmzfDhw7Vz506lpaVpxYoVWr9+vcaMGWNOLy0tlbu7u8aPH6+IiIgqtyE/P18jRoxQv379arr5AAAAAABck2yGYRg1mSEsLEzdunXT/PnzJUl2u10BAQEaN26cpkyZUq5+yJAhKioq0ooVK8yxHj16KDg4WMnJyTIMQ/7+/po0aZImT54sSSooKJCvr6+WLFmioUOHavfu3erYsaO2bNmi0NBQSVJqaqoGDRqkI0eOyN/f32GdI0eOVH5+vlJSUirchqFDh6pt27ZydnZWSkqKsrOzq739hYWF8vLyUkFBgTw9Pas9H65hiV513QGAqiQW1HUHAAAAdaq6ObRGR/BLSkqUlZXlcITcyclJERERysjIqHCejIyMckfUIyMjzfoDBw4oJyfHocbLy0thYWFmTUZGhry9vc1wL0kRERFycnJSZmZmTTZBixcv1vfff6+EhIQazQcAAAAAwLWsQU2KT548qdLSUvn6+jqM+/r6as+ePRXOk5OTU2F9Tk6OOb1srKqa5s2bOzbeoIGaNm1q1lTHvn37NGXKFG3YsEENGlRv04uLi1VcXGy+LywsrPb6AAAAAAC4Wn43d9EvLS3VQw89pOnTp+uWW26p9nxJSUny8vIyXwEBAVewSwAAAAAALk2NAr6Pj4+cnZ2Vm5vrMJ6bmys/P78K5/Hz86uyvuzPi9X89iZ+Fy5cUF5eXqXr/a1Tp07pq6++0tixY9WgQQM1aNBAzz33nLZt26YGDRpozZo1Fc43depUFRQUmK/Dhw9Xa30AAAAAAFxNNQr4Li4uCgkJUXp6ujlmt9uVnp6u8PDwCucJDw93qJektLQ0s75Nmzby8/NzqCksLFRmZqZZEx4ervz8fGVlZZk1a9askd1uV1hYWLV69/T01DfffKPs7Gzz9cgjj6hdu3bKzs6udDmurq7y9PR0eAEAAAAAcK2p0TX4khQXF6fo6GiFhoaqe/fumjt3roqKihQTEyNJGjFihFq2bKmkpCRJ0oQJE9SnTx/NmTNHgwcP1rJly/TVV1/pzTfflCTZbDZNnDhRzz//vNq2bas2bdro2Weflb+/v/ks+w4dOmjgwIEaPXq0kpOTdf78eY0dO1ZDhw51uIP+rl27VFJSory8PJ06dcq8O35wcLCcnJzUqVMnh21p3ry53Nzcyo0DAAAAAFDf1DjgDxkyRCdOnFB8fLxycnIUHBys1NRU8yZ5hw4dkpPTf08M6Nmzp9577z1NmzZNTz/9tNq2bauUlBSHUP3kk0+qqKhIY8aMUX5+vnr16qXU1FS5ubmZNe+++67Gjh2rfv36ycnJSffff7/mzZvn0NugQYP0ww8/mO9vu+02SVINnwQIAAAAAEC9YzNIvzVS3ecPoh5J9KrrDgBUJbGgrjsAAACoU9XNob+bu+gDAAAAAGBlBHwAAAAAACyAgA8AAAAAgAUQ8AEAAAAAsAACPgAAAAAAFkDABwAAAADAAgj4AAAAAABYAAEfAAAAAAALIOADAAAAAGABBHwAAAAAACyAgA8AAAAAgAUQ8AEAAAAAsAACPgAAAAAAFkDABwAAAADAAgj4AAAAAABYAAEfAAAAAAALIOADAAAAAGABBHwAAAAAACyAgA8AAAAAgAUQ8AEAAAAAsAACPgAAAAAAFkDABwAAAADAAgj4AAAAAABYwCUF/AULFigwMFBubm4KCwvT5s2bq6z/4IMP1L59e7m5ualz585atWqVw3TDMBQfH68WLVrI3d1dERER2rdvn0NNXl6ehg8fLk9PT3l7eys2NlanT582p587d04jR45U586d1aBBA0VFRZXr46OPPlL//v3VrFkzeXp6Kjw8XKtXr76UXQAAAAAAwDWlxgF/+fLliouLU0JCgrZu3aqgoCBFRkbq+PHjFdZv2rRJw4YNU2xsrL7++mtFRUUpKipKO3bsMGtmzpypefPmKTk5WZmZmfLw8FBkZKTOnTtn1gwfPlw7d+5UWlqaVqxYofXr12vMmDHm9NLSUrm7u2v8+PGKiIiosJf169erf//+WrVqlbKysnTnnXfqnnvu0ddff13T3QAAAAAAwDXFZhiGUZMZwsLC1K1bN82fP1+SZLfbFRAQoHHjxmnKlCnl6ocMGaKioiKtWLHCHOvRo4eCg4OVnJwswzDk7++vSZMmafLkyZKkgoIC+fr6asmSJRo6dKh2796tjh07asuWLQoNDZUkpaamatCgQTpy5Ij8/f0d1jly5Ejl5+crJSXlottz6623asiQIYqPj6/W9hcWFsrLy0sFBQXy9PSs1jy4xiV61XUHAKqSWFDXHQAAANSp6ubQGh3BLykpUVZWlsMRcicnJ0VERCgjI6PCeTIyMsodUY+MjDTrDxw4oJycHIcaLy8vhYWFmTUZGRny9vY2w70kRUREyMnJSZmZmTXZBAd2u12nTp1S06ZNL3kZAAAAAABcCxrUpPjkyZMqLS2Vr6+vw7ivr6/27NlT4Tw5OTkV1ufk5JjTy8aqqmnevLlj4w0aqGnTpmbNpZg9e7ZOnz6tBx98sNKa4uJiFRcXm+8LCwsveX0AAAAAAFwpv9u76L/33nuaPn263n///XI/HvxaUlKSvLy8zFdAQMBV7BIAAAAAgOqpUcD38fGRs7OzcnNzHcZzc3Pl5+dX4Tx+fn5V1pf9ebGa397E78KFC8rLy6t0vVVZtmyZ/vKXv+j999+v9IZ8ZaZOnaqCggLzdfjw4RqvDwAAAACAK61GAd/FxUUhISFKT083x+x2u9LT0xUeHl7hPOHh4Q71kpSWlmbWt2nTRn5+fg41hYWFyszMNGvCw8OVn5+vrKwss2bNmjWy2+0KCwurySbof//3fxUTE6P//d//1eDBgy9a7+rqKk9PT4cXAAAAAADXmhpdgy9JcXFxio6OVmhoqLp37665c+eqqKhIMTExkqQRI0aoZcuWSkpKkiRNmDBBffr00Zw5czR48GAtW7ZMX331ld58801Jks1m08SJE/X888+rbdu2atOmjZ599ln5+/ubz7Lv0KGDBg4cqNGjRys5OVnnz5/X2LFjNXToUIc76O/atUslJSXKy8vTqVOnlJ2dLUkKDg6W9Mtp+dHR0Xr11VcVFhZmXr/v7u4uLy/upA4AAAAAqL9qHPCHDBmiEydOKD4+Xjk5OQoODlZqaqp5k7xDhw7Jyem/Jwb07NlT7733nqZNm6ann35abdu2VUpKijp16mTWPPnkkyoqKtKYMWOUn5+vXr16KTU1VW5ubmbNu+++q7Fjx6pfv35ycnLS/fffr3nz5jn0NmjQIP3www/m+9tuu02SVPYkwDfffFMXLlzQY489pscee8ysi46O1pIlS2q6KwAAAAAAuGbYjLL0i2qp7vMHUY8kcvYGcE1LLKjrDgAAAOpUdXPo7/Yu+gAAAAAAWAkBHwAAAAAACyDgAwAAAABgAQR8AAAAAAAsgIAPAAAAAIAFEPABAAAAALAAAj4AAAAAABZAwAcAAAAAwAII+AAAAAAAWAABHwAAAAAACyDgAwAAAABgAQR8AAAAAAAsgIAPAAAAAIAFEPABAAAAALAAAj4AAAAAABZAwAcAAAAAwAII+AAAAAAAWAABHwAAAAAACyDgAwAAAABgAQR8AAAAAAAsgIAPAAAAAIAFEPABAAAAALAAAj4AAAAAABZwSQF/wYIFCgwMlJubm8LCwrR58+Yq6z/44AO1b99ebm5u6ty5s1atWuUw3TAMxcfHq0WLFnJ3d1dERIT27dvnUJOXl6fhw4fL09NT3t7eio2N1enTp83p586d08iRI9W5c2c1aNBAUVFRFfaybt06de3aVa6urrr55pu1ZMmSS9kFAAAAAABcU2oc8JcvX664uDglJCRo69atCgoKUmRkpI4fP15h/aZNmzRs2DDFxsbq66+/VlRUlKKiorRjxw6zZubMmZo3b56Sk5OVmZkpDw8PRUZG6ty5c2bN8OHDtXPnTqWlpWnFihVav369xowZY04vLS2Vu7u7xo8fr4iIiAp7OXDggAYPHqw777xT2dnZmjhxov7yl79o9erVNd0NAAAAAABcU2yGYRg1mSEsLEzdunXT/PnzJUl2u10BAQEaN26cpkyZUq5+yJAhKioq0ooVK8yxHj16KDg4WMnJyTIMQ/7+/po0aZImT54sSSooKJCvr6+WLFmioUOHavfu3erYsaO2bNmi0NBQSVJqaqoGDRqkI0eOyN/f32GdI0eOVH5+vlJSUhzGn3rqKa1cudLhx4WhQ4cqPz9fqamp1dr+wsJCeXl5qaCgQJ6entWaB9e4RK+67gBAVRIL6roDAACAOlXdHFqjI/glJSXKyspyOELu5OSkiIgIZWRkVDhPRkZGuSPqkZGRZv2BAweUk5PjUOPl5aWwsDCzJiMjQ97e3ma4l6SIiAg5OTkpMzOz2v1frBcAAAAAAOqrBjUpPnnypEpLS+Xr6+sw7uvrqz179lQ4T05OToX1OTk55vSysapqmjdv7th4gwZq2rSpWVMdlfVSWFios2fPyt3dvdw8xcXFKi4uNt8XFhZWe30AAAAAAFwt3EX/IpKSkuTl5WW+AgIC6rolAAAAAADKqVHA9/HxkbOzs3Jzcx3Gc3Nz5efnV+E8fn5+VdaX/Xmxmt/exO/ChQvKy8urdL016cXT07PCo/eSNHXqVBUUFJivw4cPV3t9AAAAAABcLTUK+C4uLgoJCVF6ero5ZrfblZ6ervDw8ArnCQ8Pd6iXpLS0NLO+TZs28vPzc6gpLCxUZmamWRMeHq78/HxlZWWZNWvWrJHdbldYWFi1+79YLxVxdXWVp6enwwsAAAAAgGtNja7Bl6S4uDhFR0crNDRU3bt319y5c1VUVKSYmBhJ0ogRI9SyZUslJSVJkiZMmKA+ffpozpw5Gjx4sJYtW6avvvpKb775piTJZrNp4sSJev7559W2bVu1adNGzz77rPz9/c1n2Xfo0EEDBw7U6NGjlZycrPPnz2vs2LEaOnSowx30d+3apZKSEuXl5enUqVPKzs6WJAUHB0uSHnnkEc2fP19PPvmkRo0apTVr1uj999/XypUrL3X/AQAAAABwTahxwB8yZIhOnDih+Ph45eTkKDg4WKmpqebN6w4dOiQnp/+eGNCzZ0+99957mjZtmp5++mm1bdtWKSkp6tSpk1nz5JNPqqioSGPGjFF+fr569eql1NRUubm5mTXvvvuuxo4dq379+snJyUn333+/5s2b59DboEGD9MMPP5jvb7vtNklS2ZMA27Rpo5UrV+rxxx/Xq6++qlatWunvf/+7IiMja7obAAAAAAC4ptiMsvSLaqnu8wdRjyR61XUHAKqSWFDXHQAAANSp6uZQ7qIPAAAAAIAFEPABAAAAALAAAj4AAAAAABZAwAcAAAAAwAII+AAAAAAAWAABHwAAAAAACyDgAwAAAABgAQR8AAAAAAAsgIAPAAAAAIAFEPABAAAAALAAAj4AAAAAABZAwAcAAAAAwAII+AAAAAAAWAABHwAAAAAACyDgAwAAAABgAQR8AAAAAAAsgIAPAAAAAIAFEPABAAAAALAAAj4AAAAAABZAwAcAAAAAwAII+AAAAAAAWAABHwAAAAAACyDgAwAAAABgAZcU8BcsWKDAwEC5ubkpLCxMmzdvrrL+gw8+UPv27eXm5qbOnTtr1apVDtMNw1B8fLxatGghd3d3RUREaN++fQ41eXl5Gj58uDw9PeXt7a3Y2FidPn3aoWb79u3q3bu33NzcFBAQoJkzZ5brZe7cuWrXrp3c3d0VEBCgxx9/XOfOnbuU3QAAAAAAwDWjxgF/+fLliouLU0JCgrZu3aqgoCBFRkbq+PHjFdZv2rRJw4YNU2xsrL7++mtFRUUpKipKO3bsMGtmzpypefPmKTk5WZmZmfLw8FBkZKRD8B4+fLh27typtLQ0rVixQuvXr9eYMWPM6YWFhRowYIBat26trKwszZo1S4mJiXrzzTfNmvfee09TpkxRQkKCdu/erbfeekvLly/X008/XdPdAAAAAADANcVmGIZRkxnCwsLUrVs3zZ8/X5Jkt9sVEBCgcePGacqUKeXqhwwZoqKiIq1YscIc69Gjh4KDg5WcnCzDMOTv769JkyZp8uTJkqSCggL5+vpqyZIlGjp0qHbv3q2OHTtqy5YtCg0NlSSlpqZq0KBBOnLkiPz9/bVw4UI988wzysnJkYuLiyRpypQpSklJ0Z49eyRJY8eO1e7du5Wenm72MmnSJGVmZurLL7+s1vYXFhbKy8tLBQUF8vT0rMmuw7Uq0auuOwBQlcSCuu4AAACgTlU3h9boCH5JSYmysrIUERHx3wU4OSkiIkIZGRkVzpORkeFQL0mRkZFm/YEDB5STk+NQ4+XlpbCwMLMmIyND3t7eZriXpIiICDk5OSkzM9OsueOOO8xwX7aevXv36ueff5Yk9ezZU1lZWeYlBd9//71WrVqlQYMG1WQ3AAAAAABwzWlQk+KTJ0+qtLRUvr6+DuO+vr7mUfLfysnJqbA+JyfHnF42VlVN8+bNHRtv0EBNmzZ1qGnTpk25ZZRNu+666/TQQw/p5MmT6tWrlwzD0IULF/TII49UeYp+cXGxiouLzfeFhYWV1gIAAAAAUFd+V3fRX7dunV544QW9/vrr2rp1qz766COtXLlSM2bMqHSepKQkeXl5ma+AgICr2DEAAAAAANVTo4Dv4+MjZ2dn5ebmOozn5ubKz8+vwnn8/PyqrC/782I1v72J34ULF5SXl+dQU9Eyfr2OZ599Vg8//LD+8pe/qHPnzrrvvvv0wgsvKCkpSXa7vcL+p06dqoKCAvN1+PDhCusAAAAAAKhLNQr4Li4uCgkJcbhJnd1uV3p6usLDwyucJzw83KFektLS0sz6Nm3ayM/Pz6GmsLBQmZmZZk14eLjy8/OVlZVl1qxZs0Z2u11hYWFmzfr163X+/HmH9bRr107XXXedJOnMmTNycnLcZGdnZ0m/PKqvIq6urvL09HR4AQAAAABwranxKfpxcXFatGiRli5dqt27d+uvf/2rioqKFBMTI0kaMWKEpk6datZPmDBBqampmjNnjvbs2aPExER99dVXGjt2rCTJZrNp4sSJev755/Xpp5/qm2++0YgRI+Tv76+oqChJUocOHTRw4ECNHj1amzdv1saNGzV27FgNHTpU/v7+kqSHHnpILi4uio2N1c6dO7V8+XK9+uqriouLM3u55557tHDhQi1btkwHDhxQWlqann32Wd1zzz1m0AcAAAAAoD6q0U32pF8ee3fixAnFx8crJydHwcHBSk1NNW9od+jQIYej5D179tR7772nadOm6emnn1bbtm2VkpKiTp06mTVPPvmkioqKNGbMGOXn56tXr15KTU2Vm5ubWfPuu+9q7Nix6tevn5ycnHT//fdr3rx55nQvLy999tlneuyxxxQSEiIfHx/Fx8drzJgxZs20adNks9k0bdo0HT16VM2aNdM999yjv/3tbzXdDQAAAAAAXFNsRmXnpqNC1X3+IOqRRK+67gBAVRIL6roDAACAOlXdHPq7uos+AAAAAABWRcAHAAAAAMACCPgAAAAAAFgAAR8AAAAAAAsg4AMAAAAAYAEEfAAAAAAALICADwAAAACABRDwAQAAAACwAAI+AAAAAAAWQMAHAAAAAMACCPgAAAAAAFgAAR8AAAAAAAsg4AMAAAAAYAEEfAAAAAAALICADwAAAACABRDwAQAAAACwAAI+AAAAAAAWQMAHAAAAAMACCPgAAAAAAFgAAR8AAAAAAAsg4AMAAAAAYAEEfAAAAAAALICADwAAAACABVxSwF+wYIECAwPl5uamsLAwbd68ucr6Dz74QO3bt5ebm5s6d+6sVatWOUw3DEPx8fFq0aKF3N3dFRERoX379jnU5OXlafjw4fL09JS3t7diY2N1+vRph5rt27erd+/ecnNzU0BAgGbOnFmul/z8fD322GNq0aKFXF1ddcstt5TrBwAAAACA+qbGAX/58uWKi4tTQkKCtm7dqqCgIEVGRur48eMV1m/atEnDhg1TbGysvv76a0VFRSkqKko7duwwa2bOnKl58+YpOTlZmZmZ8vDwUGRkpM6dO2fWDB8+XDt37lRaWppWrFih9evXa8yYMeb0wsJCDRgwQK1bt1ZWVpZmzZqlxMREvfnmm2ZNSUmJ+vfvr4MHD+rDDz/U3r17tWjRIrVs2bKmuwEAAAAAgGuKzTAMoyYzhIWFqVu3bpo/f74kyW63KyAgQOPGjdOUKVPK1Q8ZMkRFRUVasWKFOdajRw8FBwcrOTlZhmHI399fkyZN0uTJkyVJBQUF8vX11ZIlSzR06FDt3r1bHTt21JYtWxQaGipJSk1N1aBBg3TkyBH5+/tr4cKFeuaZZ5STkyMXFxdJ0pQpU5SSkqI9e/ZIkpKTkzVr1izt2bNHDRs2vITd9csPCV5eXiooKJCnp+clLQPXmESvuu4AQFUSC+q6AwAAgDpV3RxaoyP4JSUlysrKUkRExH8X4OSkiIgIZWRkVDhPRkaGQ70kRUZGmvUHDhxQTk6OQ42Xl5fCwsLMmoyMDHl7e5vhXpIiIiLk5OSkzMxMs+aOO+4ww33Zevbu3auff/5ZkvTpp58qPDxcjz32mHx9fdWpUye98MILKi0trXSbi4uLVVhY6PACAAAAAOBaU6OAf/LkSZWWlsrX19dh3NfXVzk5ORXOk5OTU2V92Z8Xq2nevLnD9AYNGqhp06YONRUt49fr+P777/Xhhx+qtLRUq1at0rPPPqs5c+bo+eefr3Sbk5KS5OXlZb4CAgIqrQUAAAAAoK78ru6ib7fb1bx5c7355psKCQnRkCFD9Mwzzyg5ObnSeaZOnaqCggLzdfjw4avYMQAAAAAA1dOgJsU+Pj5ydnZWbm6uw3hubq78/PwqnMfPz6/K+rI/c3Nz1aJFC4ea4OBgs+a3N/G7cOGC8vLyHJZT0Xp+vY4WLVqoYcOGcnZ2Nms6dOignJwclZSUOJzeX8bV1VWurq4VbhsAAAAAANeKGh3Bd3FxUUhIiNLT080xu92u9PR0hYeHVzhPeHi4Q70kpaWlmfVt2rSRn5+fQ01hYaEyMzPNmvDwcOXn5ysrK8usWbNmjex2u8LCwsya9evX6/z58w7radeuna677jpJ0u23367vvvtOdrvdrPn222/VokWLCsM9AAAAAAD1RY1P0Y+Li9OiRYu0dOlS7d69W3/9619VVFSkmJgYSdKIESM0depUs37ChAlKTU3VnDlztGfPHiUmJuqrr77S2LFjJUk2m00TJ07U888/r08//VTffPONRowYIX9/f0VFRUn65Sj7wIEDNXr0aG3evFkbN27U2LFjNXToUPn7+0uSHnroIbm4uCg2NlY7d+7U8uXL9eqrryouLs7s5a9//avy8vI0YcIEffvtt1q5cqVeeOEFPfbYY5e8AwEAAAAAuBbU6BR96ZfH3p04cULx8fHKyclRcHCwUlNTzRvaHTp0SE5O//3doGfPnnrvvfc0bdo0Pf3002rbtq1SUlLUqVMns+bJJ59UUVGRxowZo/z8fPXq1Uupqalyc3Mza959912NHTtW/fr1k5OTk+6//37NmzfPnO7l5aXPPvtMjz32mEJCQuTj46P4+HiNGTPGrAkICNDq1av1+OOPq0uXLmrZsqUmTJigp556qqa7AQAAAACAa4rNMAyjrpuoT6r7/EHUI4ledd0BgKokFtR1BwAAAHWqujn0d3UXfQAAAAAArIqADwAAAACABRDwAQAAAACwAAI+AAAAAAAWQMAHAAAAAMACCPgAAAAAAFgAAR8AAAAAAAsg4AMAAAAAYAEEfAAAAAAALICADwAAAACABRDwAQAAAACwAAI+AAAAAAAWQMAHAAAAAMACCPgAAAAAAFgAAR8AAAAAAAsg4AMAAAAAYAEEfAAAAAAALICADwAAAACABRDwAQAAAACwAAI+AAAAAAAWQMAHAAAAAMACCPgAAAAAAFgAAR8AAAAAAAu4pIC/YMECBQYGys3NTWFhYdq8eXOV9R988IHat28vNzc3de7cWatWrXKYbhiG4uPj1aJFC7m7uysiIkL79u1zqMnLy9Pw4cPl6ekpb29vxcbG6vTp0w4127dvV+/eveXm5qaAgADNnDmz0p6WLVsmm82mqKiomm08AAAAAADXoBoH/OXLlysuLk4JCQnaunWrgoKCFBkZqePHj1dYv2nTJg0bNkyxsbH6+uuvFRUVpaioKO3YscOsmTlzpubNm6fk5GRlZmbKw8NDkZGROnfunFkzfPhw7dy5U2lpaVqxYoXWr1+vMWPGmNMLCws1YMAAtW7dWllZWZo1a5YSExP15ptvluvp4MGDmjx5snr37l3TzQcAAAAA4JpkMwzDqMkMYWFh6tatm+bPny9JstvtCggI0Lhx4zRlypRy9UOGDFFRUZFWrFhhjvXo0UPBwcFKTk6WYRjy9/fXpEmTNHnyZElSQUGBfH19tWTJEg0dOlS7d+9Wx44dtWXLFoWGhkqSUlNTNWjQIB05ckT+/v5auHChnnnmGeXk5MjFxUWSNGXKFKWkpGjPnj3muktLS3XHHXdo1KhR2rBhg/Lz85WSklLt7S8sLJSXl5cKCgrk6elZk12Ha1WiV113AKAqiQV13QEAAECdqm4OrdER/JKSEmVlZSkiIuK/C3ByUkREhDIyMiqcJyMjw6FekiIjI836AwcOKCcnx6HGy8tLYWFhZk1GRoa8vb3NcC9JERERcnJyUmZmpllzxx13mOG+bD179+7Vzz//bI4999xzat68uWJjY6u1zcXFxSosLHR4AQAAAABwralRwD958qRKS0vl6+vrMO7r66ucnJwK58nJyamyvuzPi9U0b97cYXqDBg3UtGlTh5qKlvHrdXz55Zd66623tGjRouptsKSkpCR5eXmZr4CAgGrPCwAAAADA1fK7uYv+qVOn9PDDD2vRokXy8fGp9nxTp05VQUGB+Tp8+PAV7BIAAAAAgEvToCbFPj4+cnZ2Vm5ursN4bm6u/Pz8KpzHz8+vyvqyP3Nzc9WiRQuHmuDgYLPmtzfxu3DhgvLy8hyWU9F6yqbt379fBw8e1D333GNOt9vtkn45G2Dv3r266aabyvXv6uoqV1fXCrcNAAAAAIBrRY2O4Lu4uCgkJETp6enmmN1uV3p6usLDwyucJzw83KFektLS0sz6Nm3ayM/Pz6GmsLBQmZmZZk14eLjy8/OVlZVl1qxZs0Z2u11hYWFmzfr163X+/HmH9bRr107XXXed2rdvr2+++UbZ2dnm695779Wdd96p7OxsTr0HAAAAANRrNTqCL0lxcXGKjo5WaGiounfvrrlz56qoqEgxMTGSpBEjRqhly5ZKSkqSJE2YMEF9+vTRnDlzNHjwYC1btkxfffWV+fg6m82miRMn6vnnn1fbtm3Vpk0bPfvss/L39zefUd+hQwcNHDhQo0ePVnJyss6fP6+xY8dq6NCh8vf3lyQ99NBDmj59umJjY/XUU09px44devXVV/XKK69Iktzc3NSpUyeHbfH29pakcuMAAAAAANQ3NQ74Q4YM0YkTJxQfH6+cnBwFBwcrNTXVvKHdoUOH5OT03xMDevbsqffee0/Tpk3T008/rbZt2yolJcUhVD/55JMqKirSmDFjlJ+fr169eik1NVVubm5mzbvvvquxY8eqX79+cnJy0v3336958+aZ0728vPTZZ5/pscceU0hIiHx8fBQfH68xY8Zc0o4BAAAAAKA+sRmGYdR1E/VJdZ8/iHok0auuOwBQlcSCuu4AAACgTlU3h/5u7qIPAAAAAICVEfABAAAAALAAAj4AAAAAABZAwAcAAAAAwAII+AAAAAAAWAABHwAAAAAACyDgAwAAAABgAQR8AAAAAAAsgIAPAAAAAIAFEPABAAAAALAAAj4AAAAAABZAwAcAAAAAwAII+AAAAAAAWAABHwAAAAAACyDgAwAAAABgAQR8AAAAAAAsgIAPAAAAAIAFEPABAAAAALAAAj4AAAAAABZAwAcAAAAAwAII+AAAAAAAWAABHwAAAAAACyDgAwAAAABgAZcU8BcsWKDAwEC5ubkpLCxMmzdvrrL+gw8+UPv27eXm5qbOnTtr1apVDtMNw1B8fLxatGghd3d3RUREaN++fQ41eXl5Gj58uDw9PeXt7a3Y2FidPn3aoWb79u3q3bu33NzcFBAQoJkzZzpMX7RokXr37q3rrrtO1113nSIiIi7aOwAAAAAA9UGNA/7y5csVFxenhIQEbd26VUFBQYqMjNTx48crrN+0aZOGDRum2NhYff3114qKilJUVJR27Nhh1sycOVPz5s1TcnKyMjMz5eHhocjISJ07d86sGT58uHbu3Km0tDStWLFC69ev15gxY8zphYWFGjBggFq3bq2srCzNmjVLiYmJevPNN82adevWadiwYVq7dq0yMjIUEBCgAQMG6OjRozXdDQAAAAAAXFNshmEYNZkhLCxM3bp10/z58yVJdrtdAQEBGjdunKZMmVKufsiQISoqKtKKFSvMsR49eig4OFjJyckyDEP+/v6aNGmSJk+eLEkqKCiQr6+vlixZoqFDh2r37t3q2LGjtmzZotDQUElSamqqBg0apCNHjsjf318LFy7UM888o5ycHLm4uEiSpkyZopSUFO3Zs6fCbSn9/9q7/5iq6j+O4y9AAefGFUdewF3TqZEOFUGl66/WJNFcG2WbmmUqaTpIneXPFGk5RM1lTifpMm2TaZmRM3fLaOXUGyJWpOLP1Kzt3nIIJCYg937/cJ5vNxC5Zvw4Ph8bY3zO+5zzPvfs8tnrnnvPra1VeHi41q9fr0mTJjXq+CsqKmSxWFReXq6wsLDGP3BouTItzd0BgIZkljd3BwAAAM2qsTnUryv41dXVKioqUlJS0v83EBiopKQkOZ3OetdxOp0+9ZKUnJxs1F+4cEEul8unxmKxKDEx0ahxOp3q0KGDEe4lKSkpSYGBgSooKDBqhg8fboT72/s5ffq0rl69Wm9v169fV01NjTp27HjHY66qqlJFRYXPDwAAAAAALY1fAf/KlSuqra2V1Wr1GbdarXK5XPWu43K5Gqy//ftuNZ06dfJZ3qZNG3Xs2NGnpr5t/H0f/7RgwQJFR0fXeQHi71asWCGLxWL82Gy2O9YCAAAAANBcHti76GdnZ2vHjh369NNPFRoaese6RYsWqby83Pi5fPlyE3YJAAAAAEDjtPGnOCIiQkFBQXK73T7jbrdbkZGR9a4TGRnZYP3t3263W1FRUT41cXFxRs0/b+J38+ZNlZaW+mynvv38fR+3vf3228rOztZXX32lvn37NnjMISEhCgkJabAGAAAAAIDm5tcV/ODgYCUkJCg/P98Y83g8ys/Pl91ur3cdu93uUy9J+/fvN+q7deumyMhIn5qKigoVFBQYNXa7XWVlZSoqKjJqvv76a3k8HiUmJho1Bw4cUE1Njc9+YmJiFB4eboytWrVKb731lhwOh89n+gEAAAAAaM38fov+3LlztXnzZm3btk0lJSWaOXOmKisrNWXKFEnSpEmTtGjRIqN+9uzZcjgcWrNmjU6dOqXMzEwdPXpU6enpkqSAgADNmTNHy5cv1549e/TTTz9p0qRJio6OVkpKiiSpV69eGjVqlKZNm6YjR47o0KFDSk9P1/jx4xUdHS1Jev755xUcHKzU1FSdOHFCO3fu1Lvvvqu5c+cavaxcuVJLly7Vli1b1LVrV7lcLrlcLl27du2eH0AAAAAAAFoCv96iL9362rs//vhDGRkZcrlciouLk8PhMG5o98svvygw8P+vGwwePFi5ublasmSJFi9erJ49eyovL0+xsbFGzfz581VZWanp06errKxMQ4cOlcPh8Pls/Pbt25Wenq4RI0YoMDBQY8eO1bp164zlFotFX375pdLS0pSQkKCIiAhlZGRo+vTpRs3GjRtVXV2t5557zueYli1bpszMTH8fCgAAAAAAWowAr9frbe4mWpPGfv8gWpFMS3N3AKAhmeXN3QEAAECzamwO9fsKPgAAACCJF8mBlo4XyR84D+zX5AEAAAAAYCYEfAAAAAAATICADwAAAACACRDwAQAAAAAwAQI+AAAAAAAmQMAHAAAAAMAECPgAAAAAAJgAAR8AAAAAABMg4AMAAAAAYAIEfAAAAAAATICADwAAAACACRDwAQAAAAAwAQI+AAAAAAAmQMAHAAAAAMAECPgAAAAAAJgAAR8AAAAAABMg4AMAAAAAYAIEfAAAAAAATICADwAAAACACRDwAQAAAAAwAQI+AAAAAAAmQMAHAAAAAMAE7ingb9iwQV27dlVoaKgSExN15MiRBus//vhjPfroowoNDVWfPn20b98+n+Ver1cZGRmKiopSu3btlJSUpLNnz/rUlJaWauLEiQoLC1OHDh2Umpqqa9eu+dQUFxdr2LBhCg0Nlc1m06pVq/zuBQAAAACA1sjvgL9z507NnTtXy5Yt07Fjx9SvXz8lJyfr999/r7f+8OHDmjBhglJTU/X9998rJSVFKSkpOn78uFGzatUqrVu3Tjk5OSooKFD79u2VnJysGzduGDUTJ07UiRMntH//fu3du1cHDhzQ9OnTjeUVFRUaOXKkHn74YRUVFWn16tXKzMzUpk2b/OoFAAAAAIDWKMDr9Xr9WSExMVEDBw7U+vXrJUkej0c2m02vvvqqFi5cWKd+3Lhxqqys1N69e42xxx57THFxccrJyZHX61V0dLRee+01vf7665Kk8vJyWa1Wbd26VePHj1dJSYl69+6twsJCDRgwQJLkcDj01FNP6ddff1V0dLQ2btyoN954Qy6XS8HBwZKkhQsXKi8vT6dOnWpUL41RUVEhi8Wi8vJyhYWF+fPQoaXKtDR3BwAaklne3B0AuBPmUKBlYw41jcbm0Db+bLS6ulpFRUVatGiRMRYYGKikpCQ5nc5613E6nZo7d67PWHJysvLy8iRJFy5ckMvlUlJSkrHcYrEoMTFRTqdT48ePl9PpVIcOHYxwL0lJSUkKDAxUQUGBnnnmGTmdTg0fPtwI97f3s3LlSl29elXh4eF37aU+VVVVqqqqMv4uL7/1JKmoqLjjOmhlqvx6jQtAU+P/LdByMYcCLRtzqGnczp93uz7vV8C/cuWKamtrZbVafcatVqtxlfyfXC5XvfUul8tYfnusoZpOnTr5Nt6mjTp27OhT061btzrbuL0sPDz8rr3UZ8WKFXrzzTfrjNtstjuuAwC4j7K5QggAwD1hDjWdP//8UxbLnc+rXwH/QbRo0SKfq/4ej0cJCQk6duyYAgICmrEz3C8DBw5UYWFhc7eB+6CiokI2m02XL1/mIzQmwnPUXDif5sL5NA/mUHPiOWoeXq9XCQkJio6ObrDOr4AfERGhoKAgud1un3G3263IyMh614mMjGyw/vZvt9utqKgon5q4uDij5p838bt586ZKS0t9tlPffv6+j7v1Up+QkBCFhITUGWvoVRO0LkFBQUxkJhMWFsY5NRGeo+bC+TQXzqf5MIeaC89RcwkODlZgYMP3yffrLvrBwcFKSEhQfn6+MebxeJSfny+73V7vOna73adekvbv32/Ud+vWTZGRkT41FRUVKigoMGrsdrvKyspUVFRk1Hz99dfyeDxKTEw0ag4cOKCamhqf/cTExCg8PLxRvTRWWlqaX/Vo2TifQMvGc9RcOJ/mwvkEWjaeo+bSmPPp9130d+7cqZdeeknvvfeeBg0apLVr1+qjjz7SqVOnZLVaNWnSJHXu3FkrVqyQdOur6R5//HFlZ2drzJgx2rFjh7KysnTs2DHFxsZKklauXKns7Gxt27ZN3bp109KlS1VcXKyTJ08qNDRUkjR69Gi53W7l5OSopqZGU6ZM0YABA5Sbmyvp1s3vYmJiNHLkSC1YsEDHjx/X1KlT9c477xhfp9eYXgC0XnzLBQAA94Y5FDAHvz+DP27cOP3xxx/KyMiQy+VSXFycHA6HcfO6X375xedtA4MHD1Zubq6WLFmixYsXq2fPnsrLy/MJ1PPnz1dlZaWmT5+usrIyDR06VA6Hwwj3krR9+3alp6drxIgRCgwM1NixY7Vu3TpjucVi0Zdffqm0tDQlJCQoIiJCGRkZRrhvbC8AWq+QkBAtW7aszsdqAABAw5hDAXPw+wo+AAAAAABoefz6DD4AAAAAAGiZCPgAAAAAAJgAAR8AAAAAABMg4AMwjW+++UYBAQEqKytr9Dpdu3bV2rVr/7OeAABoDZhDAXMg4ANoEpMnT1ZAQIBmzJhRZ1laWpoCAgI0efLkpm+sEcrKypSWlqaoqCiFhITokUce0b59+5q7LQDAA6I1z6Eff/yxHn30UYWGhqpPnz7Mn8B/jIAPoMnYbDbt2LFDf/31lzF248YN5ebmqkuXLs3Y2Z1VV1frySef1MWLF7Vr1y6dPn1amzdvVufOnZu7NQDAA6Q1zqGHDx/WhAkTlJqaqu+//14pKSlKSUnR8ePHm7s1wLQI+ACaTHx8vGw2m3bv3m2M7d69W126dFH//v19aquqqjRr1ix16tRJoaGhGjp0qAoLC31q9u3bp0ceeUTt2rXTE088oYsXL9bZ58GDBzVs2DC1a9dONptNs2bNUmVlZaN73rJli0pLS5WXl6chQ4aoa9euevzxx9WvXz//Dh4AgH+hNc6h7777rkaNGqV58+apV69eeuuttxQfH6/169f7d/AAGo2AD6BJTZ06VR988IHx95YtWzRlypQ6dfPnz9cnn3yibdu26dixY+rRo4eSk5NVWloqSbp8+bKeffZZPf300/rhhx/08ssva+HChT7bOH/+vEaNGqWxY8equLhYO3fu1MGDB5Went7ofvfs2SO73a60tDRZrVbFxsYqKytLtbW19/gIAABwb1rbHOp0OpWUlOQzlpycLKfT6c9hA/ADAR9Ak3rhhRd08OBBXbp0SZcuXdKhQ4f0wgsv+NRUVlZq48aNWr16tUaPHq3evXtr8+bNateund5//31J0saNG9W9e3etWbNGMTExmjhxYp3PH65YsUITJ07UnDlz1LNnTw0ePFjr1q3Thx9+qBs3bjSq359//lm7du1SbW2t9u3bp6VLl2rNmjVavnz5fXk8AABorNY2h7pcLlmtVp8xq9Uql8t17w8CgAa1ae4GADxYHnroIY0ZM0Zbt26V1+vVmDFjFBER4VNz/vx51dTUaMiQIcZY27ZtNWjQIJWUlEiSSkpKlJiY6LOe3W73+fvHH39UcXGxtm/fbox5vV55PB5duHBBvXr1umu/Ho9HnTp10qZNmxQUFKSEhAT99ttvWr16tZYtW+b38QMAcK9a2xwKoOkR8AE0ualTpxpv8duwYcN/tp9r167plVde0axZs+osa+wNiaKiotS2bVsFBQUZY7169ZLL5VJ1dbWCg4PvW78AANxNa5pDIyMj5Xa7fcbcbrciIyPvS48A6uIt+gCa3KhRo1RdXa2amholJyfXWd69e3cFBwfr0KFDxlhNTY0KCwvVu3dvSbdC9pEjR3zW++6773z+jo+P18mTJ9WjR486P40N5kOGDNG5c+fk8XiMsTNnzigqKopwDwBocq1pDrXb7crPz/cZ279/f513CwC4fwj4AJpcUFCQSkpKdPLkSZ8r47e1b99eM2fO1Lx58+RwOHTy5ElNmzZN169fV2pqqiRpxowZOnv2rObNm6fTp08rNzdXW7du9dnOggULdPjwYaWnp+uHH37Q2bNn9dlnn/l1g6CZM2eqtLRUs2fP1pkzZ/T5558rKytLaWlp/+oxAADgXrSmOXT27NlyOBxas2aNTp06pczMTB09etSvbQDwDwEfQLMICwtTWFjYHZdnZ2dr7NixevHFFxUfH69z587piy++UHh4uKRbbw/85JNPlJeXp379+iknJ0dZWVk+2+jbt6++/fZbnTlzRsOGDVP//v2VkZGh6OjoRvdps9n0xRdfqLCwUH379tWsWbM0e/bsOncbBgCgqbSWOXTw4MHKzc3Vpk2b1K9fP+3atUt5eXmKjY29twMHcFcBXq/X29xNAAAAAACAf4cr+AAAAAAAmAABHwAAAAAAEyDgAwAAAABgAgR8AAAAAABMgIAPAAAAAIAJEPABAAAAADABAj4AAAAAACZAwAcAAAAAwAQI+AAAAAAAmAABHwAAAAAAEyDgAwAAAABgAgR8AAAAAABM4H/j4li5gwTjTAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1200x800 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "TimeToPrediction = [model_6_time_to_prediction, model_0_time_to_prediction]\n",
    "TimeToPredictionWeighted = [model_6_prediction_time_weighted, model_0_prediction_time_weighted]\n",
    "index = ['Model 6', 'Model 0']\n",
    "\n",
    "prediction_times_df = pd.DataFrame({\n",
    "    'TimeToPrediction': TimeToPrediction,\n",
    "    'TimeToPrediction (weighted)': TimeToPredictionWeighted\n",
    "}, index=index)\n",
    "\n",
    "prediction_times_df.plot(\n",
    "    kind='bar',\n",
    "    rot=0,\n",
    "    subplots=True,\n",
    "    figsize=(12,8)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cdbd4a98",
   "metadata": {},
   "source": [
    "![(Re) Introduction to Tensorflow Natural Language Processing](./assets/tf_nlp_desaster_tweets_05.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "865e30ff",
   "metadata": {},
   "source": [
    "#### Comparing the Performance of all Models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "id": "d66730d6",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "102/102 [==============================] - 0s 994us/step\n",
      "102/102 [==============================] - 0s 2ms/step\n",
      "102/102 [==============================] - 0s 2ms/step\n",
      "102/102 [==============================] - 0s 4ms/step\n",
      "102/102 [==============================] - 0s 1ms/step\n",
      "102/102 [==============================] - 0s 4ms/step\n",
      "102/102 [==============================] - 0s 4ms/step\n",
      "102/102 [==============================] - 0s 4ms/step\n",
      "102/102 [==============================] - 0s 4ms/step\n"
     ]
    }
   ],
   "source": [
    "model_1_time_to_prediction, model_1_prediction_time_weighted = time_to_prediction(\n",
    "    model = model_1,\n",
    "    samples = test_tweets\n",
    ")\n",
    "\n",
    "model_2_time_to_prediction, model_2_prediction_time_weighted = time_to_prediction(\n",
    "    model = model_2,\n",
    "    samples = test_tweets\n",
    ")\n",
    "\n",
    "model_3_time_to_prediction, model_3_prediction_time_weighted = time_to_prediction(\n",
    "    model = model_3,\n",
    "    samples = test_tweets\n",
    ")\n",
    "\n",
    "model_4_time_to_prediction, model_4_prediction_time_weighted = time_to_prediction(\n",
    "    model = model_4,\n",
    "    samples = test_tweets\n",
    ")\n",
    "\n",
    "model_5_time_to_prediction, model_5_prediction_time_weighted = time_to_prediction(\n",
    "    model = model_5,\n",
    "    samples = test_tweets\n",
    ")\n",
    "\n",
    "model_6_time_to_prediction, model_6_prediction_time_weighted = time_to_prediction(\n",
    "    model = model_6,\n",
    "    samples = test_tweets\n",
    ")\n",
    "\n",
    "model_6a_time_to_prediction, model_6a_prediction_time_weighted = time_to_prediction(\n",
    "    model = model_6a,\n",
    "    samples = test_tweets\n",
    ")\n",
    "\n",
    "model_6b_time_to_prediction, model_6b_prediction_time_weighted = time_to_prediction(\n",
    "    model = model_6b,\n",
    "    samples = test_tweets\n",
    ")\n",
    "\n",
    "model_6c_time_to_prediction, model_6c_prediction_time_weighted = time_to_prediction(\n",
    "    model = model_6c,\n",
    "    samples = test_tweets\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "id": "33964b06",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'F1-Score')"
      ]
     },
     "execution_count": 126,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA/IAAAK9CAYAAACHG1c1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAADhoklEQVR4nOzdeVxO6f8/8NfdcrfdLdJOKm3apLJvhSjRkKEkSyY7gzGWGFSiGDtj34rhYx9jLIkoyV5CakIqy7SgRLu6z++Pfp2v477baNHM+/l4nMfHuc51ruXch8+8z3Wd6/AYhmFACCGEEEIIIYSQZkGiqRtACCGEEEIIIYSQ2qNAnhBCCCGEEEIIaUYokCeEEEIIIYQQQpoRCuQJIYQQQgghhJBmhAJ5QgghhBBCCCGkGaFAnhBCCCGEEEIIaUYokCeEEEIIIYQQQpoRCuQJIYQQQgghhJBmhAJ5QgghhBBCCCGkGaFAnhBCCCFV4vF48Pf3b+pm/OtFRkaCx+MhMjKSTfP29oa+vn691RESEgIej4e0tLR6K5MQQkjToECeEEJIg6oMHsRtvr6+bL7w8HD4+PjA0tISkpKSXxTAPHz4EMOHD4eenh5kZWXRqlUr9O/fH5s3b67HHjVv1f0en271GUDWp0OHDmHDhg31Xq6DgwOn/6qqqujUqRP27t0LoVBY7/U1pKCgIJw6daqpm0EIIaQBSTV1AwghhPw3LFu2DAYGBpw0S0tL9s+HDh3CkSNHYGtrCx0dnTqXf/36dfTp0wdt2rTBxIkToaWlhRcvXuDmzZvYuHEjfvzxx6/uw79B7969ceDAAU7ahAkT0LlzZ0yaNIlNEwgEAICioiJISX07/7lw6NAhJCQkYPbs2fVeduvWrREcHAwAeP36Nfbv3w8fHx88fvwYK1eurPf6arJr164veogQFBSE4cOHY+jQoZz0MWPGYOTIkZCRkamnFhJCCGkq387/MxNCCPlXGzhwIDp27Fjl8aCgIOzatQvS0tIYPHgwEhIS6lT+ihUroKysjDt37kBFRYVzLDs7+0ua/MUKCwshLy/fqHXWVtu2bdG2bVtO2pQpU9C2bVuMHj1aJL+srGxjNa3JKSsrc67B5MmTYWpqit9++w2BgYGQlpYWOUcoFKK0tLRBrpO4+r6GpKQkJCUl67VMQgghTYOm1hNCCPkm6OjofFXgkpKSAgsLC5EgHgA0NDRE0n7//Xd07twZ8vLyaNGiBXr37o3w8HBOnq1bt8LCwgIyMjLQ0dHB9OnT8e7dO04eBwcHWFpaIjY2Fr1794a8vDwWLVoEACgpKYGfnx+MjIwgIyMDXV1dzJ8/HyUlJdX2ZcaMGRAIBCgsLBQ55unpCS0tLZSXlwMA7t69CycnJ6ipqUFOTg4GBgb44Ycfqi2/Lj5/R97f3x88Hg+PHz/G6NGjoaysDHV1dSxZsgQMw+DFixcYMmQIlJSUoKWlhbVr14qU+aXXxcHBAWfPnkV6errYVwCys7Ph4+MDTU1NyMrKwtraGqGhoV/cd3l5eXTt2hUFBQV4/fo1ez1mzJiBgwcPsvdGWFgYAODVq1f44YcfoKmpCRkZGVhYWGDv3r0i5b58+RJDhw6FgoICNDQ08NNPP4ntu7h35IVCITZu3AgrKyvIyspCXV0dzs7OuHv3Ltu+goIChIaGstfI29sbQNXvyNflPk9MTESfPn0gLy+PVq1a4ddff/2CK0sIIeRr0Yg8IYSQRpGXl4c3b95w0tTU1OqtfD09Pdy4cQMJCQmcKfviBAQEwN/fH927d8eyZcvA5/Nx69YtXL58GQMGDABQEbAGBATA0dERU6dORXJyMrZt24Y7d+4gJiaG89Dh7du3GDhwIEaOHInRo0dDU1MTQqEQ3333Ha5du4ZJkybBzMwMDx8+xPr16/H48eNq32H28PDAli1bcPbsWYwYMYJNLywsxF9//QVvb29ISkoiOzsbAwYMgLq6Onx9faGiooK0tDScPHny6y5mLXh4eMDMzAwrV67E2bNnsXz5cqiqqmLHjh3o27cvVq1ahYMHD2Lu3Lno1KkTevfuDQBfdV1++eUX5OXl4eXLl1i/fj0A7isADg4OePr0KWbMmAEDAwMcO3YM3t7eePfuHWbNmvVF/Xz27BkkJSU5D4guX76Mo0ePYsaMGVBTU4O+vj6ysrLQtWtXNtBXV1fH+fPn4ePjg/fv37OvAhQVFaFfv354/vw5Zs6cCR0dHRw4cACXL1+uVXt8fHwQEhKCgQMHYsKECSgrK0N0dDRu3ryJjh074sCBAyKvShgaGlZZXl3u89zcXDg7O2PYsGFwd3fH8ePHsWDBAlhZWWHgwIF1v7iEEEK+HEMIIYQ0oH379jEAxG5VGTRoEKOnp1enesLDwxlJSUlGUlKS6datGzN//nzmwoULTGlpKSffkydPGAkJCcbNzY0pLy/nHBMKhQzDMEx2djbD5/OZAQMGcPL89ttvDABm7969bJq9vT0DgNm+fTunrAMHDjASEhJMdHQ0J3379u0MACYmJqbKvgiFQqZVq1bM999/z0k/evQoA4C5evUqwzAM88cffzAAmDt37tR0eaqloKDAjBs3TuwxAIyfnx+77+fnxwBgJk2axKaVlZUxrVu3Zng8HrNy5Uo2PTc3l5GTk+OU/TXXhWGqvjc2bNjAAGB+//13Nq20tJTp1q0bIxAImPfv31dbrr29PdOuXTvm9evXzOvXr5mkpCRm5syZDADG1dWVcz0kJCSYR48ecc738fFhtLW1mTdv3nDSR44cySgrKzOFhYWcdh49epTNU1BQwBgZGTEAmCtXrrDp48aN4/T18uXLDABm5syZIu2vvHcZpurfs/LvYmpqKsMwX3af79+/n00rKSlhtLS0RO5TQgghDY+m1hNCCGkUW7ZswcWLFzlbferfvz9u3LiB7777Dvfv38evv/4KJycntGrVCqdPn2bznTp1CkKhEEuXLoWEBPf/Bnk8HgDg0qVLKC0txezZszl5Jk6cCCUlJZw9e5ZznoyMDMaPH89JO3bsGMzMzNCuXTu8efOG3fr27QsAuHLlSpV94fF4GDFiBM6dO4f8/Hw2/ciRI2jVqhV69uwJAOwo8ZkzZ/Dx48faXqp6MWHCBPbPkpKS6NixIxiGgY+PD5uuoqICU1NTPHv2jE37mutSnXPnzkFLSwuenp5smrS0NGbOnIn8/HxERUXVWMbff/8NdXV1qKurw8zMDJs3b8agQYNEpsfb29vD3Nyc3WcYBidOnICrqysYhuH0y8nJCXl5eYiLi2Pbqa2tjeHDh7Pny8vLcxYarMqJEyfA4/Hg5+cncqzy3q2Lut7nAoGAs4YAn89H586dOb8vIYSQxkFT6wkhhDSKzp07V7vYXW2Ul5ez7ypXUlVVBZ/PBwB06tQJJ0+eRGlpKe7fv48//vgD69evx/DhwxEfHw9zc3OkpKRAQkKCE4h9Lj09HQBgamrKSefz+Wjbti17vFKrVq3YNlR68uQJkpKSoK6uLraOmhbg8/DwwIYNG3D69GmMGjUK+fn5OHfuHCZPnswGbfb29vj+++8REBCA9evXw8HBAUOHDsWoUaMafGXyNm3acPaVlZUhKysr8rqEsrIy3r59y+5/7XWpSnp6OoyNjUUezpiZmbHHa6Kvr49du3aBx+NBVlYWxsbGYtdX+PzrC69fv8a7d++wc+dO7Ny5U2zZlf1KT0+HkZGRSOD9+b0mTkpKCnR0dKCqqlpj3tqo633eunVrkXa3aNECDx48qJf2EEIIqT0K5AkhhDQbL168EAmirly5AgcHB04an89Hp06d0KlTJ5iYmGD8+PE4duyY2JHM+iAnJyeSJhQKYWVlhXXr1ok9R1dXt9oyu3btCn19fRw9ehSjRo3CX3/9haKiInh4eLB5eDwejh8/jps3b+Kvv/7ChQsX8MMPP2Dt2rW4efMm+/54QxC3+nlVK6IzDMP++WuvS0NSUFCAo6Njjfk+/70rPxE3evRojBs3Tuw57du3//oGNrHa/L6EEEIaBwXyhBBCmg0tLS2RKfnW1tbVnlM5CyAjIwNAxcJfQqEQiYmJ6NChg9hz9PT0AADJycmcT7WVlpYiNTW1VsGeoaEh7t+/j379+n3RtGcAcHd3x8aNG/H+/XscOXIE+vr66Nq1q0i+rl27omvXrlixYgUOHToELy8vHD58mDP9/VvxtdelqnP09PTw4MEDCIVCzqj833//zR5vKOrq6lBUVER5eXmN94aenh4SEhLAMAynL8nJyTXWY2hoiAsXLiAnJ6faUfnaXtf6uM8JIYQ0DXpHnhBCSLMhKysLR0dHztaiRQsAFSPz4kYGz507B+D/pg8PHToUEhISWLZsGTuSWqnyfEdHR/D5fGzatIlT5p49e5CXl4dBgwbV2FZ3d3e8evUKu3btEjlWVFSEgoKCGsvw8PBASUkJQkNDERYWBnd3d87x3NxckT5XPpyo6VNuTeVrr4uCggLy8vJE0l1cXJCZmYkjR46waWVlZdi8eTMEAgHs7e2/vvFVkJSUxPfff48TJ04gISFB5Pinr4O4uLjgn3/+wfHjx9m0wsLCKqfkf+r7778HwzAICAgQOfbpfaCgoCDy+Thx6uM+J4QQ0jRoRJ4QQsg34cGDB+yidE+fPkVeXh6WL18OoGLU3dXVtdrzf/zxRxQWFsLNzQ3t2rVDaWkprl+/zo5kVy5GZ2RkhF9++QWBgYHo1asXhg0bBhkZGdy5cwc6OjoIDg6Guro6Fi5ciICAADg7O+O7775DcnIytm7dik6dOnEW/KrKmDFjcPToUUyZMgVXrlxBjx49UF5ejr///htHjx7FhQsXalwzwNbWlm1vSUkJZ1o9AISGhmLr1q1wc3ODoaEhPnz4gF27dkFJSQkuLi41trEpfO11sbOzw5EjRzBnzhx06tQJAoEArq6umDRpEnbs2AFvb2/ExsZCX18fx48fR0xMDDZs2ABFRcUG7dfKlStx5coVdOnSBRMnToS5uTlycnIQFxeHS5cuIScnB0DFQnK//fYbxo4di9jYWGhra+PAgQOQl5evsY4+ffpgzJgx2LRpE548eQJnZ2cIhUJER0ejT58+mDFjBnuNLl26hHXr1kFHRwcGBgbo0qWLSHn1cZ8TQghpIk2yVj4hhJD/jMpPXtX0ibTqPlNX1afRPnX+/Hnmhx9+YNq1a8cIBAKGz+czRkZGzI8//shkZWWJ5N+7dy9jY2PDyMjIMC1atGDs7e2ZixcvcvL89ttvTLt27RhpaWlGU1OTmTp1KpObm8vJY29vz1hYWIhtU2lpKbNq1SrGwsKCrcfOzo4JCAhg8vLyauwTwzDML7/8wgBgjIyMRI7FxcUxnp6eTJs2bRgZGRlGQ0ODGTx4MHP37t1alV3pSz4/9/r1a06+cePGMQoKCiLni7s+X3Nd8vPzmVGjRjEqKioMAM7n2bKyspjx48czampqDJ/PZ6ysrJh9+/ZVW1517RQHADN9+nSxx7Kyspjp06czurq6jLS0NKOlpcX069eP2blzJydfeno689133zHy8vKMmpoaM2vWLCYsLKzGz88xTMWn/lavXs20a9eO4fP5jLq6OjNw4EAmNjaWzfP3338zvXv3ZuTk5Dh/fz7//Fylr7nPxbWREEJIw+MxDK1QQgghhBBCCCGENBf0jjwhhBBCCCGEENKMUCBPCCGEEEIIIYQ0IxTIE0IIIYQQQgghzQgF8oQQQgghhBBCSDNCgTwhhBBCCCGEENKMUCBPCCGEEEIIIYQ0I1JN3YBvkVAoxD///ANFRUXweLymbg4hhBBCCCGEkH85hmHw4cMH6OjoQEKi+jF3CuTF+Oeff6Crq9vUzSCEEEIIIYQQ8h/z4sULtG7duto8FMiLoaioCKDiAiopKTVxawghhBBCCCGE/Nu9f/8eurq6bDxaHQrkxaicTq+kpESBPCGEEEIIIYSQRlOb17tpsTtCCCGEEEIIIaQZoUCeEEIIIYQQQghpRiiQJ4QQQgghhBBCmhF6R/4LMQyDsrIylJeXN3VTCCHkP0dSUhJSUlL0iVBCCCGE/CdRIP8FSktLkZGRgcLCwqZuCiGE/GfJy8tDW1sbfD6/qZtCCCGEENKovolAfsuWLVi9ejUyMzNhbW2NzZs3o3PnzlXm37BhA7Zt24bnz59DTU0Nw4cPR3BwMGRlZQEA27Ztw7Zt25CWlgYAsLCwwNKlSzFw4MCvbqtQKERqaiokJSWho6MDPp9PI0KEENKIGIZBaWkpXr9+jdTUVBgbG0NCgt4UI4QQQsh/R5MH8keOHMGcOXOwfft2dOnSBRs2bICTkxOSk5OhoaEhkv/QoUPw9fXF3r170b17dzx+/Bje3t7g8XhYt24dAKB169ZYuXIljI2NwTAMQkNDMWTIENy7dw8WFhZf1d7S0lIIhULo6upCXl7+q8oihBDyZeTk5CAtLY309HSUlpayD3IJIYQQQv4LeAzDME3ZgC5duqBTp0747bffAIANkn/88Uf4+vqK5J8xYwaSkpIQERHBpv3888+4desWrl27VmU9qqqqWL16NXx8fGps0/v376GsrIy8vDyR78gXFxcjNTUVBgYG9B+OhBDShOjfY0IIIYT8m1QXh36uSecilpaWIjY2Fo6OjmyahIQEHB0dcePGDbHndO/eHbGxsbh9+zYA4NmzZzh37hxcXFzE5i8vL8fhw4dRUFCAbt26ic1TUlKC9+/fczZCCCGEEEIIIeRb1KRT69+8eYPy8nJoampy0jU1NfH333+LPWfUqFF48+YNevbsya4cP2XKFCxatIiT7+HDh+jWrRuKi4shEAjwxx9/wNzcXGyZwcHBCAgIqJ9OEUIIIYQQQgghDajZrQ4UGRmJoKAgbN26FXFxcTh58iTOnj2LwMBATj5TU1PEx8fj1q1bmDp1KsaNG4fExESxZS5cuBB5eXns9uLFi8boCqkDfX19bNiwocHrcXBwwOzZsxu8nq/Vu3dvHDp0qKmbgcTERLRu3RoFBQVN3RRCCCGEEEL+M5o0kFdTU4OkpCSysrI46VlZWdDS0hJ7zpIlSzBmzBhMmDABVlZWcHNzQ1BQEIKDgyEUCtl8fD4fRkZGsLOzQ3BwMKytrbFx40axZcrIyEBJSYmz/RtVLgrI4/HY67Ns2TKUlZXVe131HRDfuXMHkyZNqrfyvpS/vz86dOggkp6WlgYej4f4+Hg2bdeuXbC2toZAIICKigpsbGwQHBzMKavy9/h0a9euXbVtOH36NLKysjBy5Eg2befOnXBwcICSkhJ4PB7evXsncl5OTg68vLygpKQEFRUV+Pj4ID8/n9OH3r17Q0FBAb1792a/+lBp8ODBOHHiBCfN3NwcXbt2ZReaJIQQQgghhDS8Jg3k+Xw+7OzsOAvXCYVCREREVPk+e2FhochnhiQlJQFUfJKoKkKhECUlJfXQ6vpTLmRwI+Ut/ox/hRspb1EubPh1B52dnZGRkYEnT57g559/hr+/P1avXi02b2lpaYO2pfLViNpQV1dvVl8J2Lt3L2bPno2ZM2ciPj4eMTExmD9/PidwBio+jZiRkcHZqlu0EQA2bdqE8ePHc/4eFBYWwtnZWeQVk095eXnh0aNHuHjxIs6cOYOrV69yHo78/PPPaNWqFeLj46GtrY25c+eyx44cOQIJCQl8//33IuWOHz8e27Zta5AHQoQQQgghhBAxmCZ2+PBhRkZGhgkJCWESExOZSZMmMSoqKkxmZibDMAwzZswYxtfXl83v5+fHKCoqMv/73/+YZ8+eMeHh4YyhoSHj7u7O5vH19WWioqKY1NRU5sGDB4yvry/D4/GY8PDwWrUpLy+PAcDk5eWJHCsqKmISExOZoqKir+r3+Yf/MF2DLjF6C86wW9egS8z5h/98VbnVGTduHDNkyBBOWv/+/ZmuXbtyji9fvpzR1tZm9PX1GYZhmOfPnzMjRoxglJWVmRYtWjDfffcdk5qaWm09ADhbamoqc+XKFQYAc+7cOcbW1paRlpZmrly5wjx9+pT57rvvGA0NDUZBQYHp2LEjc/HiRU6Zenp6zPr169l9AMyuXbuYoUOHMnJycoyRkRHz559/cs55+PAh4+zszCgoKDAaGhrM6NGjmdevX7PH8/PzmTFjxjAKCgqMlpYWs2bNGsbe3p6ZNWtWlX3z8/NjrK2tRdJTU1MZAMy9e/cYhmGYIUOGMN7e3lWWU11Z1cnOzmZ4PB6TkJAg9njlNc7NzeWkJyYmMgCYO3fusGnnz59neDwe8+rVK4ZhGMbMzIw5f/48wzAMc+7cOcbc3JxhGIbJzc1ljIyMmOfPn4uts6SkhJGRkWEuXbpUp74Q8rXq699jQgghhJBvQXVx6Oea/B15Dw8PrFmzBkuXLkWHDh0QHx+PsLAwdgG858+fIyMjg82/ePFi/Pzzz1i8eDHMzc3h4+MDJycn7Nixg82TnZ2NsWPHwtTUFP369cOdO3dw4cIF9O/fv9H7J05YQgam/h6HjLxiTnpmXjGm/h6HsISMKs6sf3JycpyR94iICCQnJ7Ojth8/foSTkxMUFRURHR2NmJgYCAQCODs7Vzliv3HjRnTr1g0TJ05kR5l1dXXZ476+vli5ciWSkpLQvn175Ofnw8XFBREREbh37x6cnZ3h6uqK58+fV9v2gIAAuLu748GDB3BxcYGXlxdycnIAAO/evUPfvn1hY2ODu3fvIiwsDFlZWXB3d2fPnzdvHqKiovDnn38iPDwckZGRiIuL+5rLydLS0sLNmzeRnp5eL+VVunbtGuTl5WFmZlan827cuAEVFRV07NiRTXN0dISEhARu3boFALC2tsalS5cgFAoRHh6O9u3bA6i4TtOnT+f8hp/i8/no0KEDoqOjv7BXhBBCCCGEkLpo0lXrK82YMQMzZswQeywyMpKzLyUlBT8/P/j5+VVZ3p49e+qzefWqXMgg4K9EiJtEzwDgAQj4KxH9zbUgKcFrsHYwDIOIiAhcuHABP/74I5uuoKCA3bt3g8/nAwB+//13CIVC7N69GzxeRXv27dsHFRUVREZGYsCAASJlKysrg8/nQ15eXuxaB8uWLeM8VFFVVYW1tTW7HxgYiD/++AOnT5+u8r4AKt759/T0BAAEBQVh06ZNuH37NpydnfHbb7/BxsYGQUFBbP69e/dCV1cXjx8/ho6ODvbs2YPff/8d/fr1AwCEhoaidevWtbp+NfHz88OwYcOgr68PExMTdOvWDS4uLhg+fDhnSvzDhw8hEAg4544ePRrbt28XW256ejo0NTVFXi+pSWZmJjQ0NDhpUlJSUFVVRWZmJgBgzZo1mDx5MvT19dG+fXvs2LEDV69eRXx8PFatWgV3d3fcvXsXAwYMwKZNm9h7BAB0dHTq/aEFIYQQQgghRLxvIpD/L7mdmiMyEv8pBkBGXjFup+agm2HLeq//zJkzEAgE+PjxI4RCIUaNGgV/f3/2uJWVFSdAu3//Pp4+fQpFRUVOOcXFxUhJSUF0dDQGDhzIpu/YsQNeXl7VtuHTUWEAyM/Ph7+/P86ePYuMjAyUlZWhqKioxhH5yhFjoOIBhJKSErKzs9l2X7lyRSRIBoCUlBQUFRWhtLQUXbp0YdNVVVVhampabZ21pa2tjRs3biAhIQFXr17F9evXMW7cOOzevRthYWFsIG5qaorTp09zzq1uscWioiLIysrWSxs/16pVK5w5c4bdLykpgZOTE0JDQ7F8+XIoKioiOTkZzs7O2LFjB+cBkJycHAoLCxukXYQQQgghhBAuCuQbWfaHqoP4L8lXV3369MG2bdvA5/Oho6MDKSnuLaCgoMDZz8/Ph52dHQ4ePChSlrq6Ovh8Pmel9spXIqrzeR1z587FxYsXsWbNGhgZGUFOTg7Dhw+vcbE9aWlpzj6Px2O/XJCfnw9XV1esWrVK5DxtbW08ffq0xnaKo6SkhLy8PJH0ylXilZWVOemWlpawtLTEtGnTMGXKFPTq1QtRUVHo06cPgP/7ukJtqampITc3t87t1tLSYh9yVCorK0NOTk6VX4gICgrCgAEDYGdnh4kTJ2L58uWQlpbGsGHDcPnyZU4gn5OTA0NDwzq3ixBCCCGEEFJ3FMg3Mg3F2o2m1jZfXSkoKNQpcLS1tcWRI0egoaFR5UixuPL4fD7Ky8trVUdMTAy8vb3h5uYGoCII//zTZ3Vla2uLEydOQF9fX+RhBQAYGhpCWloat27dQps2bQAAubm5ePz4Mezt7ass19TUFC9fvkRWVhbnoUVcXBxkZWXZssQxNzcHgK/65rqNjQ0yMzORm5uLFi1a1Pq8bt264d27d4iNjYWdnR0A4PLlyxAKhZxZCZWSkpJw6NAh9iFNeXk5Pn78CAD4+PGjyG+bkJCA4cOHf2GvCCGEEEIIIXXR5Ivd/dd0NlCFtrIsqnr7nQdAW1kWnQ1UG7NZVfLy8oKamhqGDBmC6OhopKamIjIyEjNnzsTLly+rPE9fXx+3bt1CWloa3rx5w46Ui2NsbIyTJ08iPj4e9+/fx6hRo6rNXxvTp09HTk4OPD09cefOHaSkpODChQsYP348ysvLIRAI4OPjg3nz5uHy5ctISEiAt7d3je+eOzk5wdTUFJ6enrh+/TqePXuG48ePY/HixZg1axb7KcSpU6ciMDAQMTExSE9Px82bNzF27Fioq6tzPq1YVlaGzMxMzpaVlVVl/TY2NlBTU0NMTAwnPTMzE/Hx8exMg4cPHyI+Pp5d/M/MzAzOzs6YOHEibt++jZiYGMyYMQMjR46Ejo4OpyyGYTBp0iSsX7+enT3Ro0cP7Nq1C0lJSdi/fz969OjB5k9LS8OrV6/g6OhY089CCCGEEEIIqQcUyDcySQke/FwrRmY/D+Yr9/1czRt0obu6kJeXx9WrV9GmTRsMGzYMZmZm8PHxQXFxcbXvcs+dOxeSkpIwNzeHurp6te+7r1u3Di1atED37t3h6uoKJycn2NraflW7dXR0EBMTg/LycgwYMABWVlaYPXs2VFRU2GB99erV6NWrF1xdXeHo6IiePXuyo9VVkZKSQnh4ONq0aQNPT09YWlrCz88Ps2bNQmBgIJvP0dERN2/exIgRI2BiYoLvv/8esrKyiIiIQMuW/7f2waNHj6Ctrc3Z9PT0qqxfUlIS48ePF3nVYfv27bCxscHEiRMBAL1794aNjQ3n/fuDBw+iXbt26NevH1xcXNCzZ0/s3LlTpI6dO3dCU1MTgwcPZtP8/f1RXFyMLl26wMjICNOnT2eP/e9//8OAAQOqbTchhBBCCCENSlgOpEYDD49X/K+wdrODmysewzDiFlD/T3v//j2UlZWRl5cnEqwWFxcjNTUVBgYGX7XoWFhCBgL+SuQsfKetLAs/V3M4W2p/cbnk3y8zMxMWFhaIi4tr8uC5tLQUxsbGOHToEGeUnpDGUF//HhNCCCGkmUs8DYQtAN7/839pSjqA8yrA/Luma1cdVReHfo7ekW8izpba6G+uhdupOcj+UAwNxYrp9N/KSDz5dmlpaWHPnj14/vx5kwfyz58/x6JFiyiIJ4QQQgghTSPxNHB0LPD5B77fZ1Sku+9vVsF8bVEg34QkJXgN8ok58u83dOjQpm4CgIqFDuuyeCIhhBBCCCH1RlheMRL/eRAP/P80HhDmC7QbBEhINnLjGha9I08IIYQQQgghpPlJv86dTi+CAd6/qsj3L0OBPCGEEEIIIYSQ5ie/6q89fVG+ZoQCeUIIIYQQQgghzY9As37zNSMUyBNCCCGEEEIIaX70ulesTi/yYe9KPECpVUW+fxkK5AkhhBBCCCGEND8SkhWfmAMgGsz//33nlf+6he4ACuQJIYQQQgghhDRX5t9VfGJOSZubrqTzr/30HECfnyOEEEIIIYQQ0pyZf1fxibn06xUL2wk0K6bT/wtH4ivRiDxpFvT19bFhw4YGr8fBwQGzZ89u8Hq+Vu/evXHo0KGmbka9aqzfuLHqbsr+VCotLYW+vj7u3r3bpO0ghBBCCGlwEpKAQS/AanjF//6Lg3iAAvn/FG9vb/B4PPB4PPD5fBgZGWHZsmUoKyur97rqOyC+c+cOJk2aVG/lfSl/f3906NBBJD0tLQ08Hg/x8fFs2q5du2BtbQ2BQAAVFRXY2NggODiYU1bl7/Hp1q5du2rbcPr0aWRlZWHkyJFs2s6dO+Hg4AAlJSXweDy8e/dO5LycnBx4eXlBSUkJKioq8PHxQX5+PqcPvXv3hoKCAnr37o20tDTO+YMHD8aJEyeqv0Cf6NOnD3bv3l3r/HUVEhICFRWVBiv/34DP52Pu3LlYsGBBUzeFEEIIIYTUIwrkm5KwHEiNBh4er/hfYXmDV+ns7IyMjAw8efIEP//8M/z9/bF69WqxeUtLSxu0LQzD1Pohgrq6OuTl5Ru0PfVp7969mD17NmbOnIn4+HjExMRg/vz5nMAZACwsLJCRkcHZrl27Vm3ZmzZtwvjx4yEh8X9/fQsLC+Hs7IxFixZVeZ6XlxcePXqEixcv4syZM7h69Srn4cjPP/+MVq1aIT4+Htra2pg7dy577MiRI5CQkMD3339fq/7n5OQgJiYGrq6utcpPGo6XlxeuXbuGR48eNXVTCCGEEEJIPaFAvqkkngY2WAKhg4ETPhX/u8GyIr0BycjIQEtLC3p6epg6dSocHR1x+nRFnd7e3hg6dChWrFgBHR0dmJqaAgBevHgBd3d3qKioQFVVFUOGDBEZrf2Ut7c3oqKisHHjRnaUOS0tDZGRkeDxeDh//jzs7OwgIyODa9euISUlBUOGDIGmpiYEAgE6deqES5cuccr8fJoyj8fD7t274ebmBnl5eRgbG7P9qJSQkICBAwdCIBBAU1MTY8aMwZs3b9jjBQUFGDt2LAQCAbS1tbF27dqvvLr/5/Tp03B3d4ePjw+MjIxgYWEBT09PrFixgpNPSkoKWlpanE1NTa3Kcl+/fo3Lly+LBMizZ8+Gr68vunbtKva8pKQkhIWFYffu3ejSpQt69uyJzZs34/Dhw/jnn3/YPOPGjYOxsTG8vb2RlJQEAHj37h0WL16MLVu21Lr/Z8+eha2tLTQ1xX+zMzs7G66urpCTk4OBgQEOHjwokmfdunWwsrKCgoICdHV1MW3aNPZBSGRkJMaPH4+8vDz2HvP39wcAHDhwAB07doSioiK0tLQwatQoZGdn17rtlX2eMGEC1NXVoaSkhL59++L+/fvs8drcs5/bvXs3VFRUEBERUWP/Ku3atQu6urqQl5eHm5sb1q1bJzIL4c8//4StrS1kZWXRtm1bBAQEcB6QtWjRAj169MDhw4frdA0IIYQQQsi3iwL5ppB4Gjg6Fnj/Dzf9fUZFegMH85+Sk5PjjLxHREQgOTmZHbX9+PEjnJycoKioiOjoaMTExEAgEMDZ2bnKEfuNGzeiW7dumDhxIjvKrKuryx739fXFypUrkZSUhPbt2yM/Px8uLi6IiIjAvXv34OzsDFdXVzx//rzatgcEBMDd3R0PHjyAi4sLvLy8kJOTA6AiEOvbty9sbGxw9+5dhIWFISsrC+7u7uz58+bNQ1RUFP7880+Eh4cjMjIScXFxX3M5WVpaWrh58ybS09PrpbxK165dg7y8PMzMzOp03o0bN6CiooKOHTuyaY6OjpCQkMCtW7cAANbW1rh06RKEQiHCw8PRvn17ABXXafr06ZzfsCanT5/GkCFDqjzu7e2NFy9e4MqVKzh+/Di2bt0qEmxLSEhg06ZNePToEUJDQ3H58mXMnz8fANC9e3ds2LABSkpK7D1WOYPg48ePCAwMxP3793Hq1CmkpaXB29u71m0HgBEjRiA7Oxvnz59HbGwsbG1t0a9fP/b+qus9++uvv8LX1xfh4eHo169fjf0DgJiYGEyZMgWzZs1CfHw8+vfvL/IgKDo6GmPHjsWsWbOQmJiIHTt2ICQkRCRf586dER0dXadrQAghhBBCvmEMEZGXl8cAYPLy8kSOFRUVMYmJiUxRUdGXFV5exjBr2zGMn1IVmzLDrDWryFfPxo0bxwwZMoRhGIYRCoXMxYsXGRkZGWbu3LnscU1NTaakpIQ958CBA4ypqSkjFArZtJKSEkZOTo65cOFClXXZ29szs2bN4qRduXKFAcCcOnWqxrZaWFgwmzdvZvf19PSY9evXs/sAmMWLF7P7+fn5DADm/PnzDMMwTGBgIDNgwABOmS9evGAAMMnJycyHDx8YPp/PHD16lD3+9u1bRk5OTqTdn/Lz82Osra1F0lNTUxkAzL179xiGYZh//vmH6dq1KwOAMTExYcaNG8ccOXKEKS8v55QlISHBKCgocLbJkydXWf/69euZtm3bVnm88hrn5uZy0lesWMGYmJiI5FdXV2e2bt3KMAzDvHz5khk0aBCjq6vLDBo0iHn58iUTFRXFdOzYkXn79i0zYsQIxsDAgJk8eTLnHvlccXExIxAImISEBLHHk5OTGQDM7du32bSkpCQGAOc3/tyxY8eYli1bsvv79u1jlJWVq8xf6c6dOwwA5sOHD1Xm+fT+io6OZpSUlJji4mJOHkNDQ2bHjh1VllHVPTt//nxGW1u7yutR6fP+eXh4MIMGDeLk8fLy4vS5X79+TFBQECfPgQMHGG1tbU7axo0bGX19/Wrrb46++t9jQgghhJBvSHVx6Ofo83ONLf266Eg8BwO8f1WRz6BXvVd/5swZCAQCfPz4EUKhEKNGjWKnJAOAlZUV+Hw+u3///n08ffoUioqKnHKKi4uRkpKC6OhoDBw4kE3fsWMHvLy8qm3Dp6PCQMXopr+/P86ePYuMjAyUlZWhqKioxhH5yhFjAFBQUICSkhI7qnv//n1cuXIFAoFA5LyUlBQUFRWhtLQUXbp0YdNVVVXZ1wm+lra2Nm7cuIGEhARcvXoV169fx7hx47B7926EhYWx77ebmpqKvBKgpKRUZblFRUWQlZWtlzZ+rlWrVjhz5gy7X1JSAicnJ4SGhmL58uVQVFREcnIynJ2dsWPHDvz4449iy7l8+TI0NDRgYWEh9nhSUhKkpKRgZ2fHprVr105kyvilS5cQHByMv//+G+/fv0dZWRmKi4tRWFhY7XoJsbGx8Pf3x/3795GbmwuhUAgAeP78OczNzWu8Dvfv30d+fj5atmzJSS8qKkJKSgqA2t+za9euRUFBAe7evYu2bdvWqX/Jyclwc3PjnNO5c2fOb3T//n3ExMRwRuDLy8tFrpOcnBwKCwtr7DshhBBCCGkeKJBvbPlZ9Zuvjvr06YNt27aBz+dDR0cHUlLcW0BBQYHbjPx82NnZiX2HWV1dHXw+n7NSe1XvRFdXx9y5c3Hx4kWsWbMGRkZGkJOTw/Dhw2tcbE9aWpqzz+Px2KAtPz8frq6uWLVqlch52traePr0aY3tFEdJSQl5eXki6ZWrxCsrK3PSLS0tYWlpiWnTpmHKlCno1asXoqKi0KdPHwBgvx5QW2pqasjNza1zu7W0tESmrpeVlSEnJwdaWlpizwkKCsKAAQNgZ2eHiRMnYvny5ZCWlsawYcNw+fLlKgP506dP47vvvqtzGz+VlpaGwYMHY+rUqVixYgVUVVVx7do1+Pj4oLS0tMpAvqCgAE5OTnBycsLBgwehrq6O58+fw8nJqdaLN+bn50NbWxuRkZEixyofNtT2nu3VqxfOnj2Lo0ePwtfX96v7J66tAQEBGDZsmMixTx/45OTkQF1dvVZlEkIIIYSQbx8F8o1NUHOgW6d8daSgoFCnwNHW1hZHjhyBhoZGlSPF4srj8/koL6/dKvwxMTHw9vZmRx/z8/OrXUyvNmxtbXHixAno6+uLPKwAAENDQ0hLS+PWrVto06YNACA3NxePHz+Gvb19leWampri5cuXyMrK4jy0iIuLg6ysLFuWOJWjwQUFBV/aLdjY2CAzMxO5ublo0aJFrc/r1q0b3r17h9jYWHYk/PLlyxAKhZxZCZWSkpJw6NAh9iFNeXk5Pn78CKDiHfSqfluGYfDXX3/h999/r7It7dq1Q1lZGWJjY9GpUycAQHJyMueTebGxsRAKhVi7di07e+Ho0aOccsTdY3///Tfevn2LlStXsu/01/Ub6ra2tsjMzISUlBT09fXF5qntPdu5c2fMmDEDzs7OkJKSYt/jr03/TE1NcefOHU7a5/u2trZITk6u8e90QkICbGxsqs1DCCGEEEKaD1rsrrHpdQeUdADwqsjAA5RaVeT7Bnh5eUFNTQ1DhgxBdHQ0UlNTERkZiZkzZ+Lly5dVnqevr49bt24hLS0Nb968YUfKxTE2NsbJkycRHx+P+/fvY9SoUdXmr43p06cjJycHnp6euHPnDlJSUnDhwgWMHz8e5eXlEAgE8PHxwbx583D58mUkJCTA29ub80k3cZycnGBqagpPT09cv34dz549w/Hjx7F48WLMmjULkpKSAICpU6ciMDAQMTExSE9Px82bNzF27Fioq6ujW7dubHllZWXIzMzkbFlZVc/GsLGxgZqaGmJiYjjpmZmZiI+PZ2caPHz4EPHx8ezibGZmZnB2dsbEiRNx+/ZtxMTEYMaMGRg5ciR0dHQ4ZTEMg0mTJmH9+vXs7IkePXpg165dSEpKwv79+9GjRw+x7YuNjUVhYSF69uxZZR9MTU3h7OyMyZMn49atW4iNjcWECRMgJyfH5jEyMsLHjx+xefNmPHv2DAcOHMD27ds55ejr6yM/Px8RERF48+YNCgsL0aZNG/D5fPa806dPIzAwsMq2iOPo6Ihu3bph6NChCA8PR1paGq5fv45ffvmFfShQl3u2e/fuOHfuHAICAtgvL9Smfz/++CPOnTuHdevW4cmTJ9ixYwfOnz8PHu///u1YunQp9u/fj4CAADx69AhJSUk4fPgwFi9ezCkrOjoaAwYMqNN1IIQQQggh3y4K5BubhCTgXDnd+/Ng/v/vO6+syPcNkJeXx9WrV9GmTRsMGzYMZmZm8PHxQXFxcbXvcs+dOxeSkpIwNzdnpzdXZd26dWjRogW6d+8OV1dXODk5wdbW9qvaraOjg5iYGJSXl2PAgAGwsrLC7NmzoaKiwgbrq1evRq9eveDq6gpHR0f07NmT8962OFJSUggPD0ebNm3g6ekJS0tL+Pn5YdasWZyA0dHRETdv3sSIESNgYmKC77//HrKysoiIiOC8e/3o0SNoa2tzNj09vSrrl5SUxPjx40Veddi+fTtsbGwwceJEAEDv3r1hY2PDef/+4MGDaNeuHfr16wcXFxf07NkTO3fuFKlj586d0NTUxODBg9k0f39/FBcXo0uXLjAyMsL06dPFtu/PP/+Ei4uL2FkQn9q3bx90dHRgb2+PYcOGYdKkSdDQ0GCPW1tbY926dVi1ahUsLS1x8OBBBAcHc8ro3r07pkyZAg8PD6irq+PXX3+Furo6QkJCcOzYMZibm2PlypVYs2ZNtW35HI/Hw7lz59C7d2+MHz8eJiYmGDlyJNLT09lZGHW9Z3v27ImzZ89i8eLF2Lx5c63616NHD2zfvh3r1q2DtbU1wsLC8NNPP3GmzDs5OeHMmTMIDw9Hp06d0LVrV6xfv55zD924cQN5eXkYPnx4na4DIYQQQgj5dvEYhmGauhHfmvfv30NZWRl5eXkiwWpxcTFSU1NhYGDwdYuOJZ4GwhZwF75TalURxJt/3fvF5N8tMzMTFhYWiIuLqzbobwrt27fH4sWLOZ/5I/Vn4sSJ+Pvvv+v0KTkPDw9YW1tj0aJFDdiyplFv/x4TQgghhHwDqotDP0fvyDcV8++AdoMqVqfPz6p4J16v+zczEk++XVpaWtizZw+eP3/+TQXypaWl+P777zlfMSBfZ82aNejfvz8UFBRw/vx5hIaGYuvWrbU+v7S0FFZWVvjpp58asJWEEEIIIaSx0Yi8GI0yIk8IITVwd3dHZGQkPnz4gLZt2+LHH3/ElClTmrpZ3wz695gQQggh/yY0Ik8IIf8Cn69kTwghhBBCCECL3RFCCCGEEEIIIc0KBfKEEEIIIYQQQkgzQoE8IYQQQgghhBDSjFAgTwghhBBCCCGENCMUyBNCCCGEEEIIIc0IBfKEEEIIIYQQQkgzQoE8aRb09fWxYcOGBq/HwcEBs2fPbvB6vlbv3r1x6NChryojJCQEKioq1ebx9/dHhw4dvqqetLQ08Hg8xMfHf1U534rExES0bt0aBQUFTd0UQgghhBDyH0WBfBMqF5bjTuYdnHt2Dncy76BcWN6g9Xl7e4PH44HH44HP58PIyAjLli1DWVlZvddV3wHxnTt3MGnSpHor70tVFdiKC1Z37doFa2trCAQCqKiowMbGBsHBwZyyKn+PT7d27dpV24bTp08jKysLI0eOrK9uNTkHBwe2/zIyMmjVqhVcXV1x8uTJpm6aCHNzc3Tt2hXr1q1r6qYQQgghhJD/KKmmbsB/1aX0S1h5eyWyCrPYNE15Tfh29oWjnmOD1evs7Ix9+/ahpKQE586dw/Tp0yEtLY2FCxeK5C0tLQWfz2+wtjAMg/LyckhJ1XwbqqurN1g7GsLevXsxe/ZsbNq0Cfb29igpKcGDBw+QkJDAyWdhYYFLly5x0mq6Hps2bcL48eMhIfHveg43ceJE9sHSy5cv8ccff2DkyJHw9vbGzp07m7p5HOPHj8fEiROxcOHCWt2/hBBCCCGE1Kd/VyTQTFxKv4Q5kXM4QTwAZBdmY07kHFxKv1TFmV9PRkYGWlpa0NPTw9SpU+Ho6IjTp08DqBixHzp0KFasWAEdHR2YmpoCAF68eAF3d3eoqKhAVVUVQ4YMQVpaWpV1eHt7IyoqChs3bmRHWdPS0hAZGQkej4fz58/Dzs4OMjIyuHbtGlJSUjBkyBBoampCIBCgU6dOIsHt51PreTwedu/eDTc3N8jLy8PY2JjtR6WEhAQMHDgQAoEAmpqaGDNmDN68ecMeLygowNixYyEQCKCtrY21a9d+5dX9P6dPn4a7uzt8fHxgZGQECwsLeHp6YsWKFZx8UlJS0NLS4mxqampVlvv69WtcvnwZrq6unPR169bBysoKCgoK0NXVxbRp05Cfn8/JExISgjZt2kBeXh5ubm54+/atSPkrV66EpqYmFBUV4ePjg+LiYpE8u3fvhpmZGWRlZdGuXTts3bqVc/z27duwsbGBrKwsOnbsiHv37tV4vQBAXl4eWlpaaN26Nbp27YpVq1Zhx44d2LVrF+d+qOl+rLyP16xZA21tbbRs2RLTp0/Hx48f2Txbt26FsbExZGVloampieHDh7PHhEIhgoODYWBgADk5OVhbW+P48eOctvbv3x85OTmIioqqVd8IIYQQQgipTxTIN7JyYTlW3l4JBozIscq0VbdXNfg0+0pycnIoLS1l9yMiIpCcnIyLFy/izJkz+PjxI5ycnKCoqIjo6GjExMRAIBDA2dmZc96nNm7ciG7dumHixInIyMhARkYGdHV12eO+vr5YuXIlkpKS0L59e+Tn58PFxQURERG4d+8enJ2d4erqiufPn1fb9oCAALi7u+PBgwdwcXGBl5cXcnJyAADv3r1D3759YWNjg7t37yIsLAxZWVlwd3dnz583bx6ioqLw559/Ijw8HJGRkYiLi/uay8nS0tLCzZs3kZ6eXi/lVbp27Rrk5eVhZmbGSZeQkMCmTZvw6NEjhIaG4vLly5g/fz57/NatW/Dx8cGMGTMQHx+PPn36YPny5Zwyjh49Cn9/fwQFBeHu3bvQ1tYWCdIPHjyIpUuXYsWKFUhKSkJQUBCWLFmC0NBQAEB+fj4GDx4Mc3NzxMbGwt/fH3Pnzv3i/o4bNw4tWrRgp9jX9n68cuUKUlJScOXKFYSGhiIkJAQhISEAgLt372LmzJlYtmwZkpOTERYWht69e7PnBgcHY//+/di+fTsePXqEn376CaNHj+YE7Xw+Hx06dEB0dPQX940QQgghhJAvxhAReXl5DAAmLy9P5FhRURGTmJjIFBUVfVHZtzNuM5YhljVutzNuf203RIwbN44ZMmQIwzAMIxQKmYsXLzIyMjLM3Llz2eOamppMSUkJe86BAwcYU1NTRigUsmklJSWMnJwcc+HChSrrsre3Z2bNmsVJu3LlCgOAOXXqVI1ttbCwYDZv3szu6+npMevXr2f3ATCLFy9m9/Pz8xkAzPnz5xmGYZjAwEBmwIABnDJfvHjBAGCSk5OZDx8+MHw+nzl69Ch7/O3bt4ycnJxIuz/l5+fHWFtbi6SnpqYyAJh79+4xDMMw//zzD9O1a1cGAGNiYsKMGzeOOXLkCFNeXs4pS0JCglFQUOBskydPrrL+9evXM23btq3yeKVjx44xLVu2ZPc9PT0ZFxcXTh4PDw9GWVmZ3e/WrRszbdo0Tp4uXbpw+mtoaMgcOnSIkycwMJDp1q0bwzAMs2PHDqZly5acvx/btm3jXBtxxN0vn7Zh4MCBDMPU7n4cN24co6enx5SVlbF5RowYwXh4eDAMwzAnTpxglJSUmPfv34vUVVxczMjLyzPXr1/npPv4+DCenp6cNDc3N8bb27vKPpGG97X/HhNCCCGEfEuqi0M/Ry93NrLXha/rNV9dnTlzBgKBAB8/foRQKMSoUaPg7+/PHreysuK8F3///n08ffoUioqKnHKKi4uRkpKC6OhoDBw4kE3fsWMHvLy8qm1Dx44dOfv5+fnw9/fH2bNnkZGRgbKyMhQVFdU4It++fXv2zwoKClBSUkJ2djbb7itXrkAgEIicl5KSgqKiIpSWlqJLly5suqqqKvs6wdfS1tbGjRs3kJCQgKtXr+L69esYN24cdu/ejbCwMPb9dlNTU5FXApSUlKost6ioCLKysiLply5dQnBwMP7++2+8f/8eZWVlKC4uRmFhIeTl5ZGUlAQ3NzfOOd26dUNYWBi7n5SUhClTpojkuXLlCoCKVxFSUlLg4+ODiRMnsnnKysqgrKzMltG+fXtOG7t161bttaoJwzDg8XgAar4fK1lYWEBSUpLd19bWxsOHDwFUTIvX09ND27Zt4ezsDGdnZ/YVjadPn6KwsBD9+/fnlF9aWgobGxtOmpycHAoLC7+qb4QQQgghhHwJCuQbmbp87RZtq22+uurTpw+2bdsGPp8PHR0dkYW6FBQUOPv5+fmws7PDwYMHRduorg4+n89ZqV1TU7PGNnxex9y5c3Hx4kWsWbMGRkZGkJOTw/Dhw6ucul9JWlqas8/j8SAUCtl2u7q6YtWqVSLnaWtr4+nTpzW2UxwlJSXk5eWJpL979w4A2IC2kqWlJSwtLTFt2jRMmTIFvXr1QlRUFPr06QMA7NcDaktNTQ25ubmctLS0NAwePBhTp07FihUroKqqimvXrsHHxwelpaWQl5evYy/Fq3znfteuXZwHIAA4QXN9Ki8vx5MnT9CpUye2DdXdj5WquzcUFRURFxeHyMhIhIeHY+nSpfD398edO3fYPp49exatWrXilCEjI8PZz8nJgaGh4dd3khBCCCGEkDqiQL6R2WrYQlNeE9mF2WLfk+eBB015Tdhq2DZI/QoKCnUKHG1tbXHkyBFoaGhUOVIsrjw+n4/y8tq95x8TEwNvb292xDg/P7/axfRqw9bWFidOnIC+vr7YVcUNDQ0hLS2NW7duoU2bNgCA3NxcPH78GPb29lWWa2pqipcvXyIrK4vz0CIuLg6ysrJsWeKYm5sDwFd9f9zGxgaZmZnIzc1FixYtAACxsbEQCoVYu3YtO9J/9OhRznlmZma4desWJ+3mzZti84wdO1ZsHk1NTejo6ODZs2dVzrowMzPDgQMHUFxczI7Kf15PXYSGhiI3Nxfff/89gNrdj7UhJSUFR0dHODo6ws/PDyoqKrh8+TL69+8PGRkZPH/+vNr7AKhYTPHTRfIIIYQQQghpLLTYXSOTlJCEb2dfABVB+6cq9xd0XgBJiYYZ4awrLy8vqKmpYciQIYiOjkZqaioiIyMxc+ZMvHz5ssrz9PX1cevWLaSlpeHNmzfsaKg4xsbGOHnyJOLj43H//n2MGjWq2vy1MX36dOTk5MDT0xN37txBSkoKLly4gPHjx6O8vBwCgQA+Pj6YN28eLl++jISEBHh7e9f4STcnJyeYmprC09MT169fx7Nnz3D8+HEsXrwYs2bNYkemp06disDAQMTExCA9PR03b97E2LFjoa6uzplqXlZWhszMTM6WlZVVVfWwsbGBmpoaYmJi2DQjIyN8/PgRmzdvxrNnz3DgwAFs376dc97MmTMRFhaGNWvW4MmTJ/jtt9840+oBYNasWdi7dy/27duHx48fw8/PD48ePeLkCQgIQHBwMDZt2oTHjx/j4cOH2LdvH/tN9VGjRoHH42HixIlITEzEuXPnsGbNmmqvaaXCwkJkZmbi5cuXuHnzJhYsWIApU6Zg6tSp7AyGL70fP3XmzBls2rQJ8fHxSE9Px/79+yEUCmFqagpFRUXMnTsXP/30E0JDQ5GSkoK4uDhs3ryZXdAPqJgF8erVKzg6NtynIgkhhBBCCKkKBfJNwFHPEesc1kFDXoOTrimviXUO6xr0O/J1JS8vj6tXr6JNmzYYNmwYzMzM2M+SVTciOnfuXEhKSsLc3Bzq6urVvu++bt06tGjRAt27d4erqyucnJxga/t1MxJ0dHQQExOD8vJyDBgwAFZWVpg9ezZUVFTYYH316tXo1asXXF1d4ejoiJ49e8LOzq7acqWkpBAeHo42bdrA09MTlpaW8PPzw6xZsxAYGMjmc3R0xM2bNzFixAiYmJjg+++/h6ysLCIiItCyZUs236NHj6Ctrc3Z9PT0qqxfUlIS48eP50wtt7a2xrp167Bq1SpYWlri4MGDCA4O5pzXtWtX7Nq1Cxs3boS1tTXCw8OxePFiTh4PDw8sWbIE8+fPh52dHdLT0zF16lROngkTJmD37t3Yt28frKysYG9vj5CQEBgYGAAABAIB/vrrLzx8+BA2Njb45ZdfxL7eIM6uXbugra0NQ0NDDBs2DImJiThy5Ahn5fwvvR8/paKigpMnT6Jv374wMzPD9u3b8b///Q8WFhYAgMDAQCxZsgTBwcEwMzODs7Mzzp49y/YRAP73v/9hwIAB1f5WhBBCCCGENBQewzCi87v/496/fw9lZWXk5eWJBAfFxcVITU2FgYGB2EXH6qJcWI647Di8LnwNdXl12GrYfjMj8eTblZmZCQsLC8TFxVEg2QRKS0thbGyMQ4cOoUePHk3dnP+0+vz3mBBCCCGkqVUXh36O3pFvQpISkuik1ampm0GaGS0tLezZswfPnz+nQL4JPH/+HIsWLaIgnhBCCCGENBkK5AlphoYOHdrUTfjPMjIyqtOCkYQQQgghhNQ3ekeeEEIIIYQQQghpRiiQJ4QQQgghhBBCmhEK5AkhhBBCCCGEkGaEAnlCCCGEEEIIIaQZoUCeEEIIIYQQQghpRiiQJ4QQQgghhBBCmhEK5AkhhBBCCCGEkGaEAnnSLOjr62PDhg0NXo+DgwNmz57d4PUQQgghhBBCyJeiQL4JMeXlKLh1G3lnzqLg1m0w5eUNWp+3tzd4PB54PB74fD6MjIywbNkylJWV1Xtd9R0Q37lzB5MmTaq38r6Uv78/OnToIJKelpYGHo+H+Ph4Nm3Xrl2wtraGQCCAiooKbGxsEBwczCmr8vf4dGvXrl0j9IQQQgghhBDSXEk1dQP+q96HhyMrKBhlmZlsmpSWFjQXLYTSgAENVq+zszP27duHkpISnDt3DtOnT4e0tDQWLlwokre0tBR8Pr/B2sIwDMrLyyElVfNtqK6u3mDtaAh79+7F7NmzsWnTJtjb26OkpAQPHjxAQkICJ5+FhQUuXbrESavN9SCEEEIIIYT8d9GIfBN4Hx6OV7Nmc4J4ACjLysKrWbPxPjy8weqWkZGBlpYW9PT0MHXqVDg6OuL06dMAKkbshw4dihUrVkBHRwempqYAgBcvXsDd3R0qKipQVVXFkCFDkJaWVmUd3t7eiIqKwsaNG9lR5rS0NERGRoLH4+H8+fOws7ODjIwMrl27hpSUFAwZMgSampoQCATo1KmTSHD7+dR6Ho+H3bt3w83NDfLy8jA2Nmb7USkhIQEDBw6EQCCApqYmxowZgzdv3rDHCwoKMHbsWAgEAmhra2Pt2rVfeXX/z+nTp+Hu7g4fHx8YGRnBwsICnp6eWLFiBSeflJQUtLS0OJuamlq9tYMQQgghhBDy70OBfCNjysuRFRQMMIyYgxVpWUHBDT7NvpKcnBxKS0vZ/YiICCQnJ+PixYs4c+YMPn78CCcnJygqKiI6OhoxMTEQCARwdnbmnPepjRs3olu3bpg4cSIyMjKQkZEBXV1d9rivry9WrlyJpKQktG/fHvn5+XBxcUFERATu3bsHZ2dnuLq64vnz59W2PSAgAO7u7njw4AFcXFzg5eWFnJwcAMC7d+/Qt29f2NjY4O7duwgLC0NWVhbc3d3Z8+fNm4eoqCj8+eefCA8PR2RkJOLi4r7mcrK0tLRw8+ZNpKen10t5hBBCCCGEEFKJAvlGVng3VmQknoNhUJaZicK7sQ3aDoZhcOnSJVy4cAF9+/Zl0xUUFLB7925YWFjAwsICR44cgVAoxO7du2FlZQUzMzPs27cPz58/R2RkpNiylZWVwefzIS8vz44yS0pKsseXLVuG/v37w9DQEKqqqrC2tsbkyZNhaWkJY2NjBAYGwtDQUGSE/XPe3t7w9PSEkZERgoKCkJ+fj9u3bwMAfvvtN9jY2CAoKAjt2rWDjY0N9u7diytXruDx48fIz8/Hnj17sGbNGvTr1w9WVlYIDQ2tt/UC/Pz8oKKiAn19fZiamsLb2xtHjx6FUCjk5Hv48CEEAgFnmzJlSr20gRBCCCGEEPLv9E0E8lu2bIG+vj5kZWXRpUsXNhiryoYNG2Bqago5OTno6urip59+QnFxMXs8ODgYnTp1gqKiIjQ0NDB06FAkJyc3dDdqpez163rNV1dnzpyBQCCArKwsBg4cCA8PD/j7+7PHraysOO/F379/H0+fPoWioiIbaKqqqqK4uBgpKSmIjo7mBKEHDx6ssQ0dO3bk7Ofn52Pu3LkwMzODiooKBAIBkpKSahyRb9++PftnBQUFKCkpITs7m233lStXOG2rXEQuJSUFKSkpKC0tRZcuXdgyVFVV2dcJvpa2tjZu3LiBhw8fYtasWSgrK8O4cePg7OzMCeZNTU0RHx/P2ZYtW1YvbSCEEEIIIYT8OzX5qlpHjhzBnDlzsH37dnTp0gUbNmyAk5MTkpOToaGhIZL/0KFD8PX1xd69e9G9e3c8fvyYXY193bp1AICoqChMnz4dnTp1QllZGRYtWoQBAwYgMTERCgoKjd1FDqlaLtpW23x11adPH2zbtg18Ph86OjoiC6t9fn3y8/NhZ2cnNkBXV1cHn8/nrNSuqalZYxs+r2Pu3Lm4ePEi1qxZAyMjI8jJyWH48OFVTt2vJC0tzdnn8XhskJyfnw9XV1esWrVK5DxtbW08ffq0xnaKo6SkhLy8PJH0d+/eAaiYjfApS0tLWFpaYtq0aZgyZQp69eqFqKgo9OnTBwDYrwcQQgghhBBCSG01eSC/bt06TJw4EePHjwcAbN++HWfPnsXevXvh6+srkv/69evo0aMHRo0aBaBiETRPT0/cunWLzRMWFsY5JyQkBBoaGoiNjUXv3r0bsDc1k+9oByktLZRlZYl/T57Hg5SmJuQ72jVI/QoKCnUKHG1tbXHkyBFoaGhASUlJbB5x5fH5fJTX8j3/mJgYeHt7w83NDUBFEF7dYnq1YWtrixMnTkBfX1/sKvCGhoaQlpbGrVu30KZNGwBAbm4uHj9+DHt7+yrLNTU1xcuXL5GVlcV5aBEXFwdZWVm2LHHMzc0BVCyyRwghhBBCCCFfqkmn1peWliI2NhaOjo5smoSEBBwdHXHjxg2x53Tv3h2xsbHs9Ptnz57h3LlzcHFxqbKeyhFUVVVVscdLSkrw/v17ztZQeJKS0Fz0/z/1xuN9drBiX3PRQvA+eae8KXl5eUFNTQ1DhgxBdHQ0UlNTERkZiZkzZ+Lly5dVnqevr49bt24hLS0Nb968EXk3/FPGxsY4efIk4uPjcf/+fYwaNara/LUxffp05OTkwNPTE3fu3EFKSgouXLiA8ePHo7y8HAKBAD4+Ppg3bx4uX76MhIQEeHt7Q0Ki+r8STk5OMDU1haenJ65fv45nz57h+PHjWLx4MWbNmsWuBTB16lQEBgYiJiYG6enpuHnzJsaOHQt1dXV069aNLa+srAyZmZmcLSsr66v6TgghhBBCCPl3a9JA/s2bNygvLxeZjq2pqYnMKhaEGzVqFJYtW4aePXtCWloahoaGcHBwwKJFi8TmFwqFmD17Nnr06AFLS0uxeYKDg6GsrMxun66w3hCUBgxAq40bIPVZv6U0NdFq44YG/Y58XcnLy+Pq1ato06YNhg0bBjMzM/j4+KC4uLjKEXqgYrq8pKQkzM3Noa6uXu377uvWrUOLFi3QvXt3uLq6wsnJCba2tl/Vbh0dHcTExKC8vBwDBgyAlZUVZs+eDRUVFTZYX716NXr16gVXV1c4OjqiZ8+esLOrfiaElJQUwsPD0aZNG3h6esLS0hJ+fn6YNWsWAgMD2XyOjo64efMmRowYARMTE3z//feQlZVFREQEWrZsyeZ79OgRtLW1OZuent5X9Z0QQgghhBDy78ZjGHHzuxvHP//8g1atWuH69eucUcr58+cjKiqKM12+UmRkJEaOHInly5ejS5cuePr0KWbNmoWJEydiyZIlIvmnTp2K8+fP49q1a2jdurXYdpSUlKCkpITdf//+PXR1dZGXlycSrBYXFyM1NRUGBgaQlZX90q4DqPgUXeHdWJS9fg0pdXXId7T7ZkbiCSHkW1ef/x4TQgghhDS19+/fQ1lZWWwc+rkmfUdeTU0NkpKSIlOJs7KyoKWlJfacJUuWYMyYMZgwYQKAilXWCwoKMGnSJPzyyy+cqdEzZszAmTNncPXq1SqDeACQkZGBjIxMPfSobniSklDo0rnR6yWEEEIIIYQQ0nw16dR6Pp8POzs7REREsGlCoRARERGcEfpPFRYWirzHXPlecuXkAoZhMGPGDPzxxx+4fPkyDAwMGqgHhBBCCCGEEEJI42ryVevnzJmDcePGoWPHjujcuTM2bNiAgoICdhX7sWPHolWrVggODgYAuLq6Yt26dbCxsWGn1i9ZsgSurq5sQD99+nQcOnQIf/75JxQVFdn37ZWVlSEnJ9c0HSWEEEIIIYQQQupBkwfyHh4eeP36NZYuXYrMzEx06NABYWFh7AJ4z58/54zAL168GDweD4sXL8arV6+grq4OV1dXrFixgs2zbds2AICDgwOnrn379sHb27vB+0QIIYQQQgghhDSUJl3s7ltV3SIDtLgSIYR8G+jfY0IIIYT8m9RlsbsmfUeeEEIIIYQQQgghdUOBPCGEEEIIIYQQ0oxQIE8IIYQQQgghhDQjFMgTQgghhBBCCCHNCAXy5D/L398fHTp0+OpyQkJCoKKi8tXl1JW3tzeGDh3a6PXWt6a6fo0tLS0NPB4P8fHxdTqvtLQURkZGuH79esM0rA58fX3x448/NnUzCCGEEEL+8yiQb0JCIYNXybl4fCcTr5JzIRQ27AcErl69CldXV+jo6IDH4+HUqVMieRwcHMDj8cDj8SAjI4NWrVrB1dUVJ0+erHU9J06cgIODA5SVlSEQCNC+fXssW7YMOTk59dibpqGvr48NGzZw0jw8PPD48eOmaVA1IiMjwePxYGFhgfLycs4xFRUVhISENE3DvgCPx4OsrCzS09M56UOHDq3TJyUrr8m7d+/qt4ENaPv27TAwMED37t056VeuXIGLiwtatmwJeXl5mJub4+eff8arV68A1O3319fXB4/Hw82bNzn5Zs+ezfmM59y5cxEaGopnz57VbycJIYQQQkidUCDfRFLuZWP/ous4tf4eLu5JxKn197B/0XWk3MtusDoLCgpgbW2NLVu2VJtv4sSJyMjIQEpKCk6cOAFzc3OMHDkSkyZNqrGOX375BR4eHujUqRPOnz+PhIQErF27Fvfv38eBAwfqqyvfFDk5OWhoaDR1M6r07Nkz7N+/v9HrLS0trdfyeDweli5dWq9lNpYvvRYMw+C3336Dj48PJ33Hjh1wdHSElpYWTpw4gcTERGzfvh15eXlYu3YtJ29tf39ZWVksWLCg2jxqampwcnLCtm3b6t4ZQgghhBBSbyiQbwIp97IRtiMBBe9KOOkF70oQtiOhwYL5gQMHYvny5XBzc6s2n7y8PLS0tNC6dWt07doVq1atwo4dO7Br1y5cunSpyvNu376NoKAgrF27FqtXr0b37t2hr6+P/v3748SJExg3bhybd9u2bTA0NASfz4epqalIkM/j8bB79264ublBXl4exsbGOH36NABAKBSidevWIsHEvXv3ICEhwY7aPn/+HEOGDIFAIICSkhLc3d2RlZVVZfsdHBwwe/ZsTtqnI74ODg5IT0/HTz/9xM5aAMRPDf+a/gFAeXk5fHx8YGBgADk5OZiammLjxo1Vtr06P/74I/z8/FBSUlJlnnfv3mHChAlQV1eHkpIS+vbti/v377PHxU3j/3y01sHBATNmzMDs2bPZgA8A1q1bBysrKygoKEBXVxfTpk1Dfn5+nfsxY8YM/P7770hISKgyj1AoRHBwMHvdrK2tcfz4cQAVU9v79OkDAGjRogV4PB68vb1x5swZqKiosKPW8fHx4PF48PX1ZcudMGECRo8eze6fOHECFhYWkJGRgb6+vkjwrK+vj8DAQIwdOxZKSkpiH4KVl5fjhx9+QLt27fD8+XOx/YmNjUVKSgoGDRrEpr18+RIzZ87EzJkzsXfvXjg4OEBfXx+9e/fG7t27RR521Ob3B4BJkybh5s2bOHfuXLX5XF1dcfjw4WrzEEIIIYSQhkWBfCMTChlEH3lSbZ5rR580+DT7uho3bhxatGhR7RT7gwcPQiAQYNq0aWKPVwa7f/zxB2bNmoWff/4ZCQkJmDx5MsaPH48rV65w8gcEBMDd3R0PHjyAi4sLvLy8kJOTAwkJCXh6euLQoUMi9ffo0QN6enoQCoUYMmQIcnJyEBUVhYsXL+LZs2fw8PD44mtw8uRJtG7dGsuWLUNGRgYyMjLE5vva/gH/97Di2LFjSExMxNKlS7Fo0SIcPXq0zu2ePXs2ysrKsHnz5irzjBgxAtnZ2Th//jxiY2Nha2uLfv361fl1iNDQUPD5fMTExGD79u0AAAkJCWzatAmPHj1CaGgoLl++jPnz59e5Hz169MDgwYM5AfbngoODsX//fmzfvh2PHj3CTz/9hNGjRyMqKgq6uro4ceIEACA5ORkZGRnYuHEjevXqhQ8fPuDevXsAgKioKKipqSEyMpItNyoqin1oERsbC3d3d4wcORIPHz6Ev78/lixZIvKqwpo1a2BtbY179+5hyZIlnGMlJSUYMWIE4uPjER0djTZt2ojtT3R0NExMTKCoqMimHTt2DKWlpVVew88fKtXm9wcAAwMDTJkyBQsXLoRQKKwyX+fOnfHy5UukpaVVWx4hhBBCCGk4FMg3sown70RG4j+Xn1uCjCfvGqdBtSQhIQETE5Nq/+P9yZMnaNu2LaSlpasta82aNfD29sa0adNgYmKCOXPmYNiwYVizZg0nn7e3Nzw9PWFkZISgoCDk5+fj9u3bAAAvLy/ExMSwI5lCoRCHDx+Gl5cXACAiIgIPHz7EoUOHYGdnhy5dumD//v2IiorCnTt3vugaqKqqQlJSEoqKitDS0oKWllaD9U9aWhoBAQHo2LEjDAwM4OXlhfHjx39RIC8vLw8/Pz8EBwcjLy9P5Pi1a9dw+/ZtHDt2DB07doSxsTHWrFkDFRUVdjS7toyNjfHrr7/C1NQUpqamACoCyT59+kBfXx99+/bF8uXLv6gfQEWgHhYWhujoaJFjJSUlCAoKwt69e+Hk5IS2bdvC29sbo0ePxo4dOyApKQlVVVUAgIaGBrS0tKCsrAxlZWV06NCBDdwjIyPx008/4d69e8jPz8erV6/w9OlT2NvbA6iYYdCvXz8sWbIEJiYm8Pb2xowZM7B69WpOe/r27Yuff/4ZhoaGMDQ0ZNPz8/MxaNAgvH79GleuXIG6unqV/U1PT4eOjg4n7cmTJ1BSUoK2tnatrllNv/+nFi9ejNTUVBw8eLDKPJXt+Xy9AkIIIYQQ0ngokG9kBe+rD+Lrmq8xMQzDTicfOHAgBAIBBAIBLCws2OO1kZSUhB49enDSevTogaSkJE5a+/bt2T8rKChASUkJ2dkVrx106NABZmZm7Kh8VFQUsrOzMWLECLYOXV1d6OrqsmWYm5tDRUVFpJ76Vh/9A4AtW7bAzs4O6urqEAgE2LlzZ5VTsGvi4+ODli1bYtWqVSLH7t+/j/z8fLRs2ZL9TQUCAVJTU5GSklKneuzs7ETSLl26hH79+qFVq1ZQVFTEmDFj8PbtWxQWFta5H+bm5hg7dqzYUfmnT5+isLAQ/fv35/Rj//79NfbD3t4ekZGRYBgG0dHRGDZsGMzMzHDt2jVERUVBR0cHxsbGAKr+fZ88ecJZVK5jx45i6/L09ERBQQHCw8OhrKxcbbuKioogKyvLSfv072FtVff7f0pdXR1z587F0qVLq3yvX05ODgC+6PcjhBBCCCH1gwL5RqagJFOv+RpLeXk5njx5AgMDAwDA7t27ER8fj/j4ePadWhMTEzx79gwfP36slzo/H9nn8XicKb9eXl5sIH/o0CE4OzujZcuWX1yfhISEyMOI+uqLONX17/Dhw5g7dy58fHwQHh6O+Ph4jB8//osXTZOSksKKFSuwceNG/PPPP5xj+fn50NbWZn/Pyi05ORnz5s0DUPtro6CgwNlPS0vD4MGD0b59e5w4cQKxsbHsYotf2peAgADExcWJfHWh8r37s2fPcvqRmJhY48wCBwcHXLt2Dffv34e0tDTatWsHBwcHREZGIioqih2Nr4vPr0UlFxcXPHjwADdu3KixDDU1NeTm5nLSTExMkJeXV+WrHeJU9/t/bs6cOSgqKsLWrVvFHq983aK6mQSEEEIIIaRhUSDfyLSNVaCgUn2QLmghA21jlcZpUC2FhoYiNzcX33//PQCgVatWMDIygpGREfT09AAAo0aNQn5+fpUBQOUnv8zMzBATE8M5FhMTA3Nz8zq1adSoUUhISEBsbCyOHz/OTquvrOPFixd48eIFm5aYmIh3795VWY+6ujonOCovLxdZWI3P54t8yutz9dG/mJgYdO/eHdOmTYONjQ2MjIzqPDr+uREjRsDCwgIBAQGcdFtbW2RmZkJKSor9TSs3NTU1AKLXBkCtvoceGxsLoVCItWvXomvXrjAxMakxkKyJrq4uZsyYgUWLFnF+C3Nzc8jIyOD58+ci/aicmcHn8wFA5DesfE9+/fr1bNBeGchHRkZyFvWr6vc1MTGBpKRkje2fOnUqVq5cie+++w5RUVHV5rWxscHff//NeYgyfPhw8Pl8/Prrr2LPqerTelX9/p8TCARYsmQJVqxYgQ8fPogcT0hIgLS0NDsThxBCCCGEND4K5BuZhAQPvTyMq83T090YEhJ1mzpbG/n5+ewoJQCkpqYiPj5eZLp2YWEhMjMz8fLlS9y8eRMLFizAlClTMHXqVHbVb3G6dOmC+fPn4+eff8b8+fNx48YNpKenIyIiAiNGjEBoaCgAYN68eQgJCcG2bdvw5MkTrFu3DidPnsTcuXPr1B99fX10794dPj4+KC8vx3fffccec3R0hJWVFby8vBAXF4fbt29j7NixsLe3r3LKc9++fXH27FmcPXsWf//9N6ZOnSoSFOnr6+Pq1at49eoV3rx5I7ac+uifsbEx7t69iwsXLuDx48dYsmTJF7/b/6mVK1di7969KCgoYNMcHR3RrVs3DB06FOHh4UhLS8P169fxyy+/4O7duwAqrs3du3exf/9+PHnyBH5+ftWuHl/JyMgIHz9+xObNm/Hs2TMcOHCAXQTvayxcuBD//PMP5ysKioqKmDt3Ln766SeEhoYiJSUFcXFx2Lx5M3vv6enpgcfj4cyZM3j9+jU7it+iRQu0b98eBw8eZIP23r17Iy4uDo8fP+aMyP/888+IiIhAYGAgHj9+jNDQUPz22291+n1//PFHLF++HIMHD8a1a9eqzNenTx/k5+fj0aNHbJquri7Wr1+PjRs3wsfHB1FRUUhPT0dMTAwmT56MwMDAKssT9/uLM2nSJCgrK4ssKAlULMDXq1cvdoo9IYQQQghpfBTINwFDGw04T7YUGZkXtJCB82RLGNo0zDfJ7969CxsbG9jY2AComEJrY2Mj8rmqXbt2QVtbG4aGhhg2bBgSExNx5MiRKkfaP7Vq1SocOnQIt27dgpOTEywsLDBnzhy0b9+e/fzc0KFDsXHjRqxZswYWFhbYsWMH9u3bxxn1rC0vLy/cv38fbm5unMCCx+Phzz//RIsWLdC7d284Ojqibdu2OHLkSJVl/fDDDxg3bhwb8Ldt21bkwcWyZcuQlpYGQ0PDKqcW10f/Jk+ejGHDhsHDwwNdunTB27dvq/waQF307dsXffv2RVlZGZvG4/Fw7tw59O7dG+PHj4eJiQlGjhyJ9PR0aGpqAgCcnJywZMkSzJ8/H506dcKHDx8wduzYGuuztrbGunXrsGrVKlhaWuLgwYMIDg7+6n6oqqpiwYIFKC4u5qQHBgZiyZIlCA4OhpmZGZydnXH27Fn2lZBWrVohICAAvr6+0NTUxIwZM9hz7e3tUV5ezv5OqqqqMDc3h5aWFrtwH1Axg+Ho0aM4fPgwLC0tsXTpUixbtoz9TGFtzZ49GwEBAXBxccH169fF5mnZsiXc3NxEFp+bNm0awsPD8erVK7i5uaFdu3aYMGEClJSUqn2gIO73F0daWhqBgYEi1xeoeO1j4sSJteghIYQQQghpKDymtiuU/Ye8f/8eysrKyMvLg5KSEudYcXExUlNTYWBgILIIVV0JhUzFKvbvS6CgVDGdviFG4gkhzdeDBw/Qv39/pKSkQCAQNGlbzp8/j59//hkPHjyAlJRUk7YFqN9/jwkhhBBCmlp1cejnmv6/xP7DJCR4aGXaoqmbQQj5hrVv3x6rVq1CamoqrKysmrQtBQUF2Ldv3zcRxBNCCCGE/JfRf40RQsg3rq7T9hvK8OHDm7oJhBBCCCEE9I48IYQQQgghhBDSrFAgTwghhBBCCCGENCM0tZ4QQgghhBBCyDerXFiOuOw4vC58DXV5ddhq2EJSQrKpm9WkKJAnhBBCCCGEEPJNupR+CStvr0RWYRabpimvCd/OvnDUc2zCljUtmlpPCCGEEEIIIeSbcyn9EuZEzuEE8QCQXZiNOZFzcCn9UhO1rOlRIE8IIYQQQggh5JtSLizHytsrwYAROVaZtur2KpQLyxu7ad8ECuQJIYQQQgghhHxT4rLjREbiP8WAQWZhJuKy4xqxVd8OCuQJR1paGng8HuLj46vMExkZCR6Ph3fv3gEAQkJCoKKi0ijt+1xT1e3t7Y2hQ4c2er3Nmb+/Pzp06FDr/LW5FwEgOTkZWlpa+PDhw9c18CuVlpZCX18fd+/ebdJ2EEIIIYT8G7wufF2v+f5tKJBvQkJhOV48eoCkmCi8ePQAwgaeFuLt7Q0ej8duLVu2hLOzMx48eMDm0dXVRUZGBiwtLWtdroeHBx4/ftwQTebQ19fHhg0bmqTuL+Hg4IDZs2dXeTwqKgp9+/aFqqoq5OXlYWxsjHHjxqG0tFTkt/p809fXZ+vg8XhYuXKlSPmDBg0Cj8eDv79/w3TwG7Fw4UL8+OOPUFRUBPB/D5oqN3V1dbi4uODhw4ec8yqv8efX7tSpU+DxeOx+ZXkWFhYoL+f+HVVRUUFISAgAgM/nY+7cuViwYEED9JIQQggh5L9FXV69XvP921Ag30Se3LqOXdN9cHTZIpzbtBpHly3Cruk+eHLreoPW6+zsjIyMDGRkZCAiIgJSUlIYPHgwe1xSUhJaWlqQkqr9Bw3k5OSgoaFR5fHS0tKvavPX1P2tSkxMhLOzMzp27IirV6/i4cOH2Lx5M/h8PsrLy7Fx40b2d8rIyAAA7Nu3j92/c+cOW5auri4bTFZ69eoVIiIioK2t3ZjdanTPnz/HmTNn4O3tLXIsOTkZGRkZuHDhAkpKSjBo0CCRe1FWVharVq1Cbm5ujXU9e/YM+/fvrzaPl5cXrl27hkePHtWpH4QQQgghhMtWwxaa8prggSf2OA88aMlrwVbDtpFb9m2gQL4JPLl1HafXBSE/5w0nPT/nDU6vC2rQYF5GRgZaWlrQ0tJChw4d4OvrixcvXuD164opKeKmM587dw4mJiaQk5NDnz59kJaWxinz8+ntlVOod+/eDQMDA8jKygIA3r17hwkTJkBdXR1KSkro27cv7t+/zynrr7/+QqdOnSArKws1NTW4ubkBqBh5Tk9Px08//cSOtIqrGwC2bdsGQ0ND8Pl8mJqa4sCBA5zjPB4Pu3fvhpubGzsSfvr0afZ4eXk5fHx8YGBgADk5OZiammLjxo11vtbVCQ8Ph5aWFn799VdYWlrC0NAQzs7O2LVrF+Tk5KCsrMz+TlpaWgAqRn8r99XV/+/J4+DBg/HmzRvExMSwaaGhoRgwYECNDzkqf6u9e/eiTZs2EAgEmDZtGsrLy/Hrr79CS0sLGhoaWLFiBee858+fY8iQIRAIBFBSUoK7uzuysrjvMK1cuRKamppQVFSEj48PiouLRerfvXs3zMzMICsri3bt2mHr1q11uo5Hjx6FtbU1WrVqJXJMQ0MDWlpasLW1xezZs/HixQv8/fffnDyOjo7Q0tJCcHBwjXX9+OOP8PPzQ0lJSZV5WrRogR49euDw4cN16gchhBBCCOGSlJCEb2dfABAJ5iv3F3Re8J/9njwF8o1MKCzH5ZCd1ea5ErqzwafZA0B+fj5+//13GBkZoWXLlmLzvHjxAsOGDYOrqyvi4+MxYcIE+Pr61lj206dPceLECZw8eZJ9KDBixAhkZ2fj/PnziI2Nha2tLfr164ecnBwAwNmzZ+Hm5gYXFxfcu3cPERER6Ny5MwDg5MmTaN26NZYtW8YZpf7cH3/8gVmzZuHnn39GQkICJk+ejPHjx+PKlSucfAEBAXB3d8eDBw/g4uICLy8vth1CoRCtW7fGsWPHkJiYiKVLl2LRokU4evRora5rbWhpaSEjIwNXr1796rL4fD68vLywb98+Ni0kJAQ//PBDrc5PSUnB+fPnERYWhv/973/Ys2cPBg0ahJcvXyIqKgqrVq3C4sWLcevWLQAV12fIkCHIyclBVFQULl68iGfPnsHDw4Mt8+jRo/D390dQUBDu3r0LbW1tkSD94MGDWLp0KVasWIGkpCQEBQVhyZIlCA0NrXXfo6Oj0bFjx2rz5OXlsYE1n8/nHJOUlERQUBA2b96Mly9fVlvO7NmzUVZWhs2bN1ebr3PnzoiOjq5F6wkhhBBCSHUc9RyxzmEdNOS5g1Oa8ppY57DuP/0d+drPnyb14lXSI5GR+M99ePsGr5IeQdeifb3Xf+bMGQgEAgBAQUEBtLW1cebMGUhIiH+mUzm6vXbtWgCAqakpHj58iFWrVlVbT2lpKfbv38+OHF+7dg23b99GdnY2ZGRkAABr1qzBqVOncPz4cUyaNAkrVqzAyJEjERAQwJZjbW0NAFBVVYWkpCQUFRXZEWpx1qxZA29vb0ybNg0AMGfOHNy8eRNr1qxBnz592Hze3t7w9PQEAAQFBWHTpk24ffs2nJ2dIS0tzWmDgYEBbty4gaNHj8Ld3b3aftfWiBEjcOHCBdjb20NLSwtdu3ZFv379MHbsWCgpKdW5vB9++AG9evXCxo0bERsbi7y8PAwePLhW78cLhULs3bsXioqKMDc3R58+fZCcnIxz585BQkICpqamWLVqFa5cuYIuXbogIiICDx8+RGpqKnR1dQEA+/fvh4WFBe7cuYNOnTphw4YN8PHxgY+PDwBg+fLluHTpEmdU3s/PD2vXrsWwYcMAVFznxMRE7NixA+PGjatVv9PT06sM5Fu3bg2g4j4HgO+++w7t2rUTyefm5oYOHTrAz88Pe/bsqbIueXl5+Pn5YdGiRZg4cSKUlZXF5tPR0UF6enqt2k8IIYQQQqrnqOeIPrp9EJcdh9eFr6Eurw5bDdv/7Eh8JRqRb2T572p+F7cu+eqqT58+iI+PR3x8PG7fvg0nJycMHDiwysAjKSkJXbp04aR169atxnr09PQ407/v37+P/Px8tGzZEgKBgN1SU1ORkpICAIiPj0e/fv2+oncV7e3RowcnrUePHkhKSuKktW//fw9JFBQUoKSkhOzsbDZty5YtsLOzg7q6OgQCAXbu3Innz5+LrfPgwYOcPtVmNFZSUhL79u3Dy5cv8euvv6JVq1YICgqChYVFlbMNqmNtbQ1jY2McP34ce/fuxZgxY2q9zoG+vj67UBwAaGpqwtzcnPNwR1NTk70+SUlJ0NXVZYN4ADA3N4eKigp7nWu6bwoKCpCSkgIfHx/OtVu+fDl7P9RGUVER++rG56KjoxEbG4uQkBCYmJhg+/btVZazatUqhIaGitwnn/Px8UHLli2rfZAlJyeHwsLC2nWAEEIIIYTUSFJCEp20OsGlrQs6aXX6zwfxAI3INzqBSot6zVdXCgoKMDIyYvd3794NZWVl7Nq1C8uXL6/Xej6Vn58PbW1tREZGiuStfMddTk6u3uqvibS0NGefx+NBKBQCAA4fPoy5c+di7dq16NatGxQVFbF69Wp2avnnvvvuO07QKu597aq0atUKY8aMwZgxYxAYGMgGnJ/OCKitH374AVu2bEFiYiJu375d6/PEXYvqrk99yM/PBwDs2rVLJOCXlKz9P8xqampVLlRnYGAAFRUVmJqaIjs7Gx4eHlW+ytC7d284OTlh4cKFYhfOqyQlJYUVK1bA29sbM2bMEJsnJyeH8xCLEEIIIYSQ+kYj8o2slZkFBKpq1eZRbKmGVmYWjdIeHo8HCQkJFBUViT1uZmYmEhTevHmzzvXY2toiMzMTUlJSMDIy4mxqahXXo3379oiIiKiyjMoV3atjZmbGWfQNAGJiYmBubl7rtsbExKB79+6YNm0abGxsYGRkVO0osaKiIqc/X/pAokWLFtDW1mangtfVqFGj8PDhQ1haWtapv3VlZmaGFy9e4MWLF2xaYmIi3r17x9ZrZmYm8uDj0/tGU1MTOjo6ePbsmcj9YGBgUOu22NjYIDExscZ806dPR0JCAv74448q86xcuRJ//fUXbty4UW1ZI0aMgIWFRZUPWxISEmBjY1NjmwghhBBCCPlSNCLfyCQkJNHXexJOrwuqMk+fcZMg0UDTRUpKSpCZmQkAyM3NxW+//Yb8/Hy4urqKzT9lyhSsXbsW8+bNw4QJE9ipynXl6OiIbt26YejQofj1119hYmKCf/75h13grmPHjvDz80O/fv1gaGiIkSNHoqysDOfOnWO/y62vr4+rV69i5MiRkJGRYR8AfGrevHlwd3eHjY0NHB0d8ddff+HkyZO4dOlSrdtqbGyM/fv348KFCzAwMMCBAwdw586dOgWYlV6/fs35AgAAaGtr49SpU4iPj4ebmxsMDQ1RXFyM/fv349GjRzUuplaVFi1aICMjQ2Q0vb45OjrCysoKXl5e2LBhA8rKyjBt2jTY29uz76vPmjUL3t7e6NixI3r06IGDBw/i0aNHaNu2LVtOQEAAZs6cCWVlZTg7O6OkpAR3795Fbm4u5syZU6u2ODk5YcKECSgvL692JF9eXh4TJ06En58fhg4dyvlOfKXKPm3atKnGeleuXAknJyexx6KjoxEYGFir9hNCCCGEEPIlaES+CRh36Y7v5iwSGZlXbKmG7+YsgnGX7g1Wd1hYGLS1taGtrY0uXbrgzp07OHbsGBwcHMTmb9OmDU6cOIFTp07B2toa27dvR1BQ1Q8hqsLj8XDu3Dn07t0b48ePh4mJCUaOHIn09HRoamoCqPjE3LFjx3D69Gl06NABffv25cwGWLZsGdLS0mBoaFjl1OWhQ4di48aNWLNmDSwsLLBjxw7s27evyv6JM3nyZAwbNgweHh7o0qUL3r59yy6eV1eHDh2CjY0NZ9u1axc6d+6M/Px8TJkyBRYWFrC3t8fNmzdx6tQp2Nvbf1FdQMVrCp+/1lDfeDwe/vzzT7Ro0QK9e/eGo6Mj2rZtiyNHjrB5PDw8sGTJEsyfPx92dnZIT0/H1KlTOeVMmDABu3fvxr59+2BlZQV7e3uEhITU6YHJwIEDISUlVasHNTNmzEBSUhKOHTtWZZ5ly5bV6hWCvn37om/fvigrK+Ok37hxA3l5eRg+fHjNjSeEEEIIIeQL8RiGYZq6Ed+a9+/fQ1lZGXl5eSIriBcXFyM1NZXzffQvJRSWV6xi/y4XApUWaGVm0WAj8YT8W23ZsgWnT5/GhQsXmrop8PDwgLW1NRYtWtTUTflPqM9/jwkhhBBCmlp1cejnaGp9E5KQkGyQT8wR8l8yefJkvHv3Dh8+fOCsvt/YSktLYWVlhZ9++qnJ2kAIIYQQQv4bKJAnhDRrUlJS+OWXX5q6GeDz+Vi8eHFTN4MQQgghhPwH0DvyhBBCCCGEEEJIM0KBPCGEEEIIIYQQ0oxQIE8IIYQQQgghhDQjFMgTQgghhBBCCCHNCAXyhBBCCCGEEEJIM0KBPCGEEEIIIYQQ0oxQIE8IIYQQQgghhDQjFMiTfz19fX1s2LCh1vn9/f3RoUOHGvMtWbIEkyZN+vKG1ZOwsDB06NABQqGwqZtCCCGEEEIIaQQUyDchRsigOOUdCuOzUZzyDoyQafA6MzMzMWvWLBgZGUFWVhaampro0aMHtm3bhsLCQjafvr4+eDweeDwe5OXlYWVlhd27d3PKCgkJgYqKith6eDweTp061YA9aVqZmZnYuHEjfvnlFzbN29ubvWbS0tIwMDDA/PnzUVxczDmXx+NBVlYW6enpnPShQ4fC29tbpLyVK1dy8p06dQo8Ho/dd3Z2hrS0NA4ePFiPPSSEEEIIIYR8qyiQbyJFCW+Queo23ux6iJzDyXiz6yEyV91GUcKbBqvz2bNnsLGxQXh4OIKCgnDv3j3cuHED8+fPx5kzZ3Dp0iVO/mXLliEjIwMJCQkYPXo0Jk6ciPPnzzdY+5qT3bt3o3v37tDT0+OkOzs7IyMjA8+ePcP69euxY8cO+Pn5iZzP4/GwdOnSGuuRlZXFqlWrkJubW20+b29vbNq0qW6dIIQQQgghhDRLFMg3gaKEN3j7exLK80o56eV5pXj7e1KDBfPTpk2DlJQU7t69C3d3d5iZmaFt27YYMmQIzp49C1dXV05+RUVFaGlpoW3btliwYAFUVVVx8eLFemmLvr4+li9fjrFjx0IgEEBPTw+nT5/G69evMWTIEAgEArRv3x53797lnHfixAlYWFhARkYG+vr6WLt2Led4dnY2XF1dIScnBwMDA7Gj1O/evcOECROgrq4OJSUl9O3bF/fv369T+w8fPixyvQBARkYGWlpa0NXVxdChQ+Ho6Cj2ms2YMQO///47EhISqq3H0dERWlpaCA4Orjafq6sr7t69i5SUlDr1gxBCCCGEENL8UCDfyBghg3d/VR9svfvrWb1Ps3/79i3Cw8Mxffp0KCgoiM3z6XTtTwmFQpw4cQK5ubng8/n11qb169ejR48euHfvHgYNGoQxY8Zg7NixGD16NOLi4mBoaIixY8eCYSquRWxsLNzd3TFy5Eg8fPgQ/v7+WLJkCUJCQtgyvb298eLFC1y5cgXHjx/H1q1bkZ2dzal3xIgRyM7Oxvnz5xEbGwtbW1v069cPOTk5tWp3Tk4OEhMT0bFjx2rzJSQk4Pr162KvWY8ePTB48GD4+vpWW4akpCSCgoKwefNmvHz5ssp8bdq0gaamJqKjo2vVB0IIIYQQQkjzRYF8IytJzRMZif9ceV4JSlLz6rXep0+fgmEYmJqactLV1NQgEAggEAiwYMECzrEFCxZAIBBARkYGw4cPR4sWLTBhwoR6a5OLiwsmT54MY2NjLF26FO/fv0enTp0wYsQImJiYYMGCBUhKSkJWVhYAYN26dejXrx+WLFkCExMTeHt7Y8aMGVi9ejUA4PHjxzh//jx27dqFrl27ws7ODnv27EFRURFb57Vr13D79m0cO3YMHTt2hLGxMdasWQMVFRUcP368Vu1+/vw5GIaBjo6OyLEzZ85AIBBAVlYWVlZWyM7Oxrx588SWExwcjLCwsBqDbzc3N3To0EHsFP1P6ejoiLx3TwghhBBCCPn3oUC+kQk/VB/E1zXf17p9+zbi4+NhYWGBkpISzrF58+YhPj4ely9fRpcuXbB+/XoYGRnVW93t27dn/6ypqQkAsLKyEkmrHFFPSkpCjx49OGX06NEDT548QXl5OZKSkiAlJQU7Ozv2eLt27TgL8t2/fx/5+flo2bIl+wBDIBAgNTW11tPSKx8MyMrKihzr06cP4uPjcevWLYwbNw7jx4/H999/L7Ycc3NzjB07tsZReQBYtWoVQkNDkZSUVGUeOTk5zoKFhBBCCCGEkH8nqaZuwH+NhGLtpqbXNl9tGRkZgcfjITk5mZPetm1bABVB4OfU1NRgZGQEIyMjHDt2DFZWVujYsSPMzc0BAEpKSigoKIBQKISExP89E3r37h0AQFlZudo2SUtLs3+unNYvLq0+P6uWn58PbW1tREZGihyragX+z6mpqQEAcnNzoa6uzjmmoKDAPuzYu3cvrK2tsWfPHvj4+IgtKyAgACYmJjWu8N+7d284OTlh4cKFnJXtP5WTkyPSHkIIIYQQQsi/D43INzIZA2VIKlcfpEsqy0DGoPoguK5atmyJ/v3747fffkNBQUGdz9fV1YWHhwcWLlzIppmamqKsrAzx8fGcvHFxcQAAExOTr2rz58zMzBATE8NJi4mJgYmJCSQlJdGuXTuUlZUhNjaWPZ6cnMw+WAAAW1tbZGZmQkpKin1IUblVBug1MTQ0hJKSEhITE6vNJyEhgUWLFmHx4sWc6f2f0tXVxYwZM7Bo0SKUl5dXW97KlSvx119/4caNGyLHiouLkZKSAhsbm1r1gRBCCCGEENJ8USDfyHgSPKi4GlabR8W1LXgS4hee+xpbt25FWVkZOnbsiCNHjiApKQnJycn4/fff8ffff0NSUrLa82fNmoW//vqLXUnewsICAwYMwA8//ICIiAikpqYiLCwM06ZNg4eHB1q1alWv7f/5558RERGBwMBAPH78GKGhofjtt98wd+5cABUPFpydnTF58mTcunULsbGxmDBhAme2gaOjI7p164ahQ4ciPDwcaWlpuH79On755ReRFfKrIiEhAUdHR1y7dq3GvCNGjICkpCS2bNlSZZ6FCxfin3/+Efn83+esrKzg5eUl9jNzN2/ehIyMDLp161ZzBwghhBBCCCHNGgXyTUDOUg0tR5uJjMxLKsug5WgzyFnWbmS4rgwNDXHv3j04Ojpi4cKFsLa2RseOHbF582bMnTsXgYGB1Z5vbm6OAQMGcL5/fuTIEdjb22Py5MmwsLDAzJkzMWTIEOzevbve229ra4ujR4/i8OHDsLS0xNKlS7Fs2TLOVPN9+/ZBR0cH9vb2GDZsGCZNmgQNDQ32OI/Hw7lz59C7d2+MHz8eJiYmGDlyJNLT09l38mtjwoQJOHz4cI3T/qWkpDBjxgz8+uuvVc6EUFVVxYIFC1BcXFxjvcuWLRNb5//+9z94eXlBXl6+dh0ghBBCCCGENFs8pvLbXoT1/v17KCsrIy8vD0pKSpxjxcXFSE1NhYGBgdjFzuqCETIoSc2D8EMpJBT5kDFQbpCReFL/GIZBly5d8NNPP8HT07NJ2/LmzRuYmpri7t27MDAwaNK2ENKY6vPfY0IIIYSQplZdHPo5WuyuCfEkeJA1VGnqZpAvwOPxsHPnTjx8+LCpm4K0tDRs3bqVgnhCCCGEEEL+IyiQJ+QLdejQAR06dGjqZqBjx47o2LFjUzeDEEIIIYQQ0kjoHXlCCCGEEEIIIaQZoUCeEEIIIYQQQghpRiiQJ4QQQgghhBBCmhEK5AkhhBBCCCGEkGakyQP5LVu2QF9fH7KysujSpQtu375dbf4NGzbA1NQUcnJy0NXVxU8//cT5/vbVq1fh6uoKHR0d8Hg8nDp1qoF7QAghhBBCCCGENJ4mDeSPHDmCOXPmwM/PD3FxcbC2toaTkxOys7PF5j906BB8fX3h5+eHpKQk7NmzB0eOHMGiRYvYPAUFBbC2tsaWLVsaqxuEEEIIIYQQQkijadLPz61btw4TJ07E+PHjAQDbt2/H2bNnsXfvXvj6+orkv379Onr06IFRo0YBAPT19eHp6Ylbt26xeQYOHIiBAwc2TgcIIYQQQgghhJBG1mQj8qWlpYiNjYWjo+P/NUZCAo6Ojrhx44bYc7p3747Y2Fh2+v2zZ89w7tw5uLi4fFVbSkpK8P79e85G/v1CQkKgoqJSp3Nq87rG27dvoaGhgbS0tC9uW33p2rUrTpw40dTNIIQQQgghhNSjJgvk37x5g/LycmhqanLSNTU1kZmZKfacUaNGYdmyZejZsyekpaVhaGgIBwcHztT6LxEcHAxlZWV209XV/aryaksoFCI1NRUPHz5EamoqhEJhg9bn7e2NoUOHVnn8/v37+O6776ChoQFZWVno6+vDw8MD2dnZ8Pf3B4/Hq3arrIPH42HKlCki5U+fPh08Hg/e3t4N1MNvw4oVKzBkyBDo6+sDANLS0jjXSVVVFfb29oiOjuacV3mNP7928fHx4PF47IOByvI0NDTw4cMHTt4OHTrA39+f3V+8eDF8fX0b/N4ihBBCCCGENJ4mX+yuLiIjIxEUFIStW7ciLi4OJ0+exNmzZxEYGPhV5S5cuBB5eXns9uLFi3pqcdUSExOxYcMGhIaG4sSJEwgNDcWGDRuQmJjY4HWL8/r1a/Tr1w+qqqq4cOECkpKSsG/fPujo6KCgoABz585FRkYGu7Vu3RrLli3jpFXS1dXF4cOHUVRUxKYVFxfj0KFDaNOmTVN0r9EUFhZiz5498PHxETl26dIlZGRk4OrVq9DR0cHgwYORlZXFySMrK4s9e/bgyZMnNdb14cMHrFmzpto8AwcOxIcPH3D+/Pm6dYQQQgghhBDyzWqyQF5NTQ2SkpIigUxWVha0tLTEnrNkyRKMGTMGEyZMgJWVFdzc3BAUFITg4OCvGnGUkZGBkpISZ2tIiYmJOHr0qMgU/vfv3+Po0aNNEszHxMQgLy8Pu3fvho2NDQwMDNCnTx+sX78eBgYGEAgE0NLSYjdJSUkoKipy0irZ2tpCV1cXJ0+eZNNOnjyJNm3awMbGptp2VE53P3PmDExNTSEvL4/hw4ejsLAQoaGh0NfXR4sWLTBz5kyUl5ez5+Xm5mLs2LFo0aIF5OXlMXDgQJFgOCQkBG3atIG8vDzc3Nzw9u1bkfr//PNP2NraQlZWFm3btkVAQADKyspqfR3PnTsHGRkZdO3aVeRYy5YtoaWlBUtLSyxatAjv37/nrO8AAKampujTpw9++eWXGuv68ccfsW7duioXhwQASUlJuLi44PDhw7XuAyGEEEIIIeTb1mSBPJ/Ph52dHSIiItg0oVCIiIgIdOvWTew5hYWFkJDgNllSUhIAwDBMwzW2HgmFQoSFhVWbJywsrNGnQmtpaaGsrAx//PFHvVzLH374Afv27WP39+7dyy5qWJPCwkJs2rQJhw8fRlhYGCIjI+Hm5oZz587h3LlzOHDgAHbs2IHjx4+z53h7e+Pu3bs4ffo0bty4AYZh4OLigo8fPwIAbt26BR8fH8yYMQPx8fHo06cPli9fzqk3OjoaY8eOxaxZs5CYmIgdO3YgJCQEK1asqHW/o6OjYWdnV22eoqIi7N+/H0DF34PPrVy5EidOnMDdu3erLcfT0xNGRkZYtmxZtfk6d+4sMo2fEEIIIYQQ0nw16dT6OXPmYNeuXQgNDUVSUhKmTp2KgoICNuAbO3YsFi5cyOZ3dXXFtm3bcPjwYaSmpuLixYtYsmQJXF1d2YA+Pz8f8fHxiI+PBwCkpqYiPj4ez58/b/T+iZOenl7jYnrv379Henp6I7WoQteuXbFo0SKMGjUKampqGDhwIFavXi0yY6K2Ro8ejWvXriE9PR3p6emIiYnB6NGja3Xux48fsW3bNtjY2KB3794YPnw4rl27hj179sDc3ByDBw9Gnz59cOXKFQDAkydPcPr0aezevRu9evWCtbU1Dh48iFevXrEL023cuBHOzs6YP38+TExMMHPmTDg5OXHqDQgIgK+vL8aNG4e2bduif//+CAwMxI4dO2rd7/T0dOjo6Ig91r17dwgEAigoKGDNmjWws7NDv379RPLZ2trC3d0dCxYsqLYuHo+HlStXYufOnUhJSakyn46ODl68eEHvyRNCCCGEEPIv0aSfn/Pw8MDr16+xdOlSZGZmokOHDggLC2MXwHv+/DlnBH7x4sXg8XhYvHgxXr16BXV1dbi6unJGTO/evYs+ffqw+3PmzAEAjBs3DiEhIY3TsWrk5+fXa776tGLFCsyZMweXL1/GrVu3sH37dgQFBeHq1auwsrKqU1nq6uoYNGgQQkJCwDAMBg0aBDU1tVqdKy8vD0NDQ3ZfU1MT+vr6EAgEnLTKKeVJSUmQkpJCly5d2OMtW7aEqakpkpKS2Dxubm6cerp168aZHXH//n3ExMRw7qfy8nIUFxejsLAQ8vLyNba9qKgIsrKyYo8dOXIE7dq1Q0JCAubPn4+QkBBIS0uLzbt8+XKYmZkhPDwcGhoaVdbn5OSEnj17YsmSJTh06JDYPHJychAKhSgpKYGcnFyNfSCEEEIIIYR825o0kAeAGTNmYMaMGWKPRUZGcvalpKTg5+cHPz+/KstzcHD4pqfZfxqM1ke++tayZUuMGDECI0aMQFBQEGxsbLBmzRqEhobWuawffviB/W23bNlS6/M+D255PJ7YtPoeYc7Pz0dAQACGDRsmcqyq4PxzampqyM3NFXtMV1cXxsbGMDY2RllZGdzc3JCQkAAZGRmRvIaGhpg4cSJ8fX2xZ8+eautcuXIlunXrhnnz5ok9npOTAwUFBQriCSGEEEII+ZdoVqvW/xvo6enVuJiekpIS9PT0GqlFVePz+TA0NERBQcEXne/s7IzS0lJ8/PhRZBp7fTIzM0NZWRln4bi3b98iOTkZ5ubmbJ7PF5a7efMmZ9/W1hbJyckwMjIS2T5fm6EqNjY2tVqscPjw4ZCSksLWrVurzLN06VI8fvy4xoXqOnfujGHDhsHX11fs8YSEhBoXGSSEEEIIIYQ0H00+Iv9fIyEhAWdnZxw9erTKPM7OzrUOHOsqLy+PXT+gUsuWLf9fe3ceV1Wd/3H8fbko+yIqi2ZgbkkhKChquVSM0JjptJljbmm2uUWZWamlk9hkpaVtamor5mTLZKmNqbklLrmgpEZuOSyaCoKKyj2/PxzvzyuLgMD15Ov5eJzHo/M93/M9n3PvAXtzNm3ZskXJycm6//771bRpUxmGoX//+9/69ttvHR5aVx5Wq9V+afv5ZxhUhSZNmqh79+566KGH9O6778rHx0fPPPOM6tevr+7du0uShg0bpptuukmTJ09W9+7dtXjx4iIPHRw7dqzuuOMOXXvttbrnnnvk4uKiLVu2KDU1tciD8UoSHx+v0aNH6+jRo6pVq1aJ/SwWi4YNG6YXXnhBDz/8cLGX7QcFBSkxMVGvvPLKJbf70ksv6YYbbpCra9Ef6ZUrV6pLly5lqh8AAADAlY8z8k4QHh6u++67r8iZeV9fX9133332s8hVYfny5WrZsqXD9OKLLyo8PFyenp568sknFRUVpbZt2+qzzz7TzJkz1adPnwpvrzpe5ydJs2fPVnR0tO644w61a9dOhmHo22+/tV+S37ZtW82YMUNTp05VZGSklixZoueff95hjPj4eH3zzTdasmSJWrdurbZt2+r1118v19URERERatWqVal/qDmvX79+OnPmjKZNm1Zin6eeeqpMt1k0bdpUDz74oE6dOuXQfvDgQa1Zs6bMbwwAAAAAcOWzGFfyDeVOkpubKz8/P+Xk5BQJoadOndKePXvUsGHDMt83XRKbzaZ9+/YpLy9P3t7eCg0NrbIz8ag+Cxcu1MiRI5Wamur073PUqFE6evSo3nvvPafWAVSFyvx9DAAA4Gyl5dCLcWm9E7m4uKhhw4bOLgOVrGvXrtq9e7cOHjyoBg0aOLWWwMBA+5sbAAAAAPw5EOSBKjBixAhnlyBJevLJJ51dAgAAAIBKxnXcAAAAAACYCEEeAAAAAAATIcgDAAAAAGAiBHkAAAAAAEyEIA8AAAAAgIkQ5AEAAAAAMBGCPAAAAAAAJkKQR5V64YUXFBUVVWofwzA0ePBgBQQEyGKxaPPmzdVSW2WzWCz68ssvnV1Ghc2ZM0f+/v7OLqNKde7cWSNGjCj3emPGjNHgwYOrZOw/q/79+6tHjx5l7r98+XJZLBYdO3ZMkrRo0SJFRUXJZrNVTYEAAAAmRpB3IsMo1NGjPykz82sdPfqTDKOwSrfXv39/WSyWItOvv/5apdu9lEWLFmnOnDn65ptvlJGRoRtvvNGp9aBqhYWFacqUKc4uo8wyMzM1depUPffcc84u5aqSkJCgGjVq6OOPP3Z2KQAAAFccV2cXcLXKzl6sXbvHq6Ag097m5haspk3GKjAwvsq2m5CQoNmzZzu01a1bt8q2Vxbp6ekKCQlR+/btKzyGYRgqLCyUq2vVHtKnT59WzZo1q3Qblc2MNVe2y/kMZs6cqfbt2ys0NLSSq8Kl9O/fX2+88Yb69Onj7FIAAACuKJyRd4Ls7MXalvq4Q4iXpIKCLG1LfVzZ2YurbNtubm4KDg52mKxWqyTpq6++UqtWreTu7q7rrrtOL774os6ePWtf99ixYxo0aJDq1q0rX19f3XrrrdqyZYvD+JMmTVJQUJB8fHw0cOBAnTp1qtR6+vfvr6FDh2r//v2yWCwKCwuTJBUUFGjYsGEKDAyUu7u7br75Zq1fv96+3vnLcL/77jtFR0fLzc1NCxculNVq1YYNGyRJNptNAQEBatu2rX29jz76SA0aNLDPjxo1Sk2bNpWnp6euu+46jRkzRmfOnLEvP39rwMyZM9WwYUO5u7tLknbv3q2OHTvK3d1d4eHh+v777y/52Xfu3FnDhg3T008/rYCAAAUHB+uFF15w6HOpz7i4y5VHjBihzp07O2xnyJAhGjFihOrUqaP4+HN/GHrttdcUEREhLy8vNWjQQI899pjy8vIuWfd5e/fulcVi0YIFC3TLLbfI09NTkZGRWrt2rUO/VatWqUOHDvLw8FCDBg00bNgw5efn22vbt2+fnnjiCfsVIYZhqG7duvrXv/5lHyMqKkohISEOY7q5uenEiROSpP3796t79+7y9vaWr6+v7rvvPmVlZdn7l/S9XWzhwoXy8/Mr9axvcnKyunXr5tCWn5+vvn37ytvbWyEhIXr11VeLrFdQUKCnnnpK9evXl5eXl2JjY7V8+XL78vO3MixevFjNmzeXt7e3EhISlJGRYe+zfPlytWnTRl5eXvL399dNN92kffv22Zdf6mf2YuePn4kTJyooKEj+/v4aP368zp49q5EjRyogIEDXXHNNkT/2bdu2Tbfeeqs8PDxUu3ZtDR482OHYKSwsVGJiovz9/VW7dm09/fTTMgzDYQybzaakpCQ1bNhQHh4eioyMdPjOi9OtWzdt2LBB6enppfYDAAC42hDkq5lhFGrX7vGSjOKWSpJ27Z5Q5ZfZX2zlypXq27evhg8frh07dujdd9/VnDlz9NJLL9n73HvvvcrOztZ3332njRs3qlWrVrrtttt05MgRSdJnn32mF154QRMnTtSGDRsUEhKit956q9TtTp06VePHj9c111yjjIwMe1h/+umn9fnnn2vu3LnatGmTGjdurPj4ePu2znvmmWc0adIkpaWlqUOHDoqKirKHpW3btslisejnn3+2h44VK1aoU6dO9vV9fHw0Z84c7dixQ1OnTtWMGTP0+uuvO2zj119/1eeff64FCxZo8+bNstlsuuuuu1SzZk2tW7dO77zzjkaNGlWmz3nu3Lny8vLSunXr9M9//lPjx493+CPApT7jspo7d65q1qyp1atX65133pEkubi46I033tD27ds1d+5c/fDDD3r66afLNa4kPffcc3rqqae0efNmNW3aVL169bKHx/T0dCUkJOjuu+/W1q1bNW/ePK1atUpDhgyRJC1YsEDXXHONxo8fr4yMDGVkZMhisahjx4727+3o0aNKS0vTyZMn9csvv0g69721bt1anp6estls6t69u44cOaIVK1bo+++/12+//aaePXs61Hnx93axTz75RL169dLHH3+s3r17F7uvR44c0Y4dOxQTE+PQPnLkSK1YsUJfffWVlixZouXLl2vTpk0OfYYMGaK1a9cqOTlZW7du1b333quEhATt3r3b3ufEiROaPHmyPvzwQ/3444/av3+/nnrqKUnS2bNn1aNHD3Xq1Elbt27V2rVrNXjwYFksFkll+5ktzg8//KD//ve/+vHHH/Xaa69p3LhxuuOOO1SrVi2tW7dOjzzyiB5++GH9/vvvks790SI+Pl61atXS+vXrNX/+fP3nP/+xf6eS9Oqrr2rOnDl6//33tWrVKh05ckRffPGFw3aTkpL0wQcf6J133tH27dv1xBNP6IEHHtCKFStKrPXaa69VUFCQVq5cWeo+AQAAXHUMFJGTk2NIMnJycoosO3nypLFjxw7j5MmTFRr7yJG1xn+WXnfJ6ciRtZe7G0X069fPsFqthpeXl3265557DMMwjNtuu82YOHGiQ/8PP/zQCAkJMQzDMFauXGn4+voap06dcujTqFEj49133zUMwzDatWtnPPbYYw7LY2NjjcjIyFLrev31143Q0FD7fF5enlGjRg3j448/tredPn3aqFevnvHPf/7TMAzDWLZsmSHJ+PLLLx3GSkxMNLp27WoYhmFMmTLF6NmzpxEZGWl89913hmEYRuPGjY333nuvxFpeeeUVIzo62j4/btw4o0aNGkZ2dra9bfHixYarq6tx8OBBe9t3331nSDK++OKLEsfu1KmTcfPNNzu0tW7d2hg1apRhGGX7jPv162d0797dYfnw4cONTp06OWynZcuWJdZx3vz5843atWvb52fPnm34+fmV2H/Pnj2GJGPmzJn2tu3btxuSjLS0NMMwDGPgwIHG4MGDHdZbuXKl4eLiYv+ZCQ0NNV5//XWHPm+88YZxww03GIZhGF9++aURGxtrdO/e3Xj77bcNwzCMuLg449lnnzUMwzCWLFliWK1WY//+/UXqSElJMQyj+O/t/GczfPhwY9q0aYafn5+xfPnyUj+jn3/+2ZDksK3jx48bNWvWND777DN72x9//GF4eHgYw4cPNwzDMPbt22dYrVaHY8Qwzv2cjR492jCMc5+3JOPXX3+1L58+fboRFBRkH1NSiTVe6me2OP369TNCQ0ONwsJCe1uzZs2MDh062OfPnj1reHl5GZ9++qlhGIbx3nvvGbVq1TLy8vLsfRYuXGi4uLgYmZmZhmEYRkhIiP1n0zAM48yZM8Y111xjP1ZPnTpleHp6GmvWrHGoZ+DAgUavXr0Mw/j/n+mjR4869GnZsqXxwgsvFLs/l/v7GAAA4EpSWg69GPfIV7OCguxK7Vdet9xyi95++237vJeXlyRpy5YtWr16tcPZvMLCQp06dUonTpzQli1blJeXp9q1azuMd/LkSftlr2lpaXrkkUcclrdr107Lli2TdO4M4u23325f9u677xZ7JjQ9PV1nzpzRTTfdZG+rUaOG2rRpo7S0NIe+F58p7dSpk2bNmqXCwkKtWLFCXbp0UXBwsJYvX64WLVro119/dbgMfd68eXrjjTeUnp6uvLw8nT17Vr6+vg5jhoaGOjxHIC0tTQ0aNFC9evUc9rMsWrRo4TAfEhKi7Oxz33VZPuOyio6OLtL2n//8R0lJSfrll1+Um5urs2fP2r9fT0/PMo994T6cv/w9Oztb119/vbZs2aKtW7c6XKpuGIZsNpv27Nmj5s2bFztmp06dNHz4cB06dEgrVqxQ586d7d/bwIEDtWbNGvvVA+c//wtvkQgPD5e/v7/S0tLUunVrSUW/t/P+9a9/KTs7W6tXr7b3LcnJkyclyeHS/PT0dJ0+fVqxsbH2toCAADVr1sw+v23bNhUWFqpp06YO4xUUFDh8v56enmrUqJF9/sLjISAgQP3791d8fLz+8pe/KC4uTvfdd5/9M7/Uz2xJ3+kNN9wgF5f/vxgrKCjI4QGTVqtVtWvXtteRlpamyMhI++8KSbrppptks9m0c+dOubu7KyMjw+HzcHV1VUxMjP3y+l9//VUnTpzQX/7yF4daTp8+rZYtWxZb53keHh72WyoAAABwDkG+mrm5BVZqv/Ly8vJS48aNi7Tn5eXpxRdf1F133VVkmbu7u/Ly8hQSEuJwj+95ZX1lWUxMjMMlzkFBQWUtu0QXhgtJ6tixo44fP65Nmzbpxx9/1MSJExUcHKxJkyYpMjJS9erVU5MmTSRJa9euVe/evfXiiy8qPj5efn5+Sk5OLnK/88XbuBw1atRwmLdYLPbXa5XlM3ZxcSly7/GF9/Sfd3HNe/fu1R133KFHH31UL730kgICArRq1SoNHDhQp0+fLleQv3Afzl/mfeE+PPzwwxo2bFiR9a699toSx4yIiFBAQIBWrFihFStW6KWXXlJwcLBefvllrV+/XmfOnCn3wxBL+t5atmypTZs26f3331dMTIx9H4pTp04dSecu9y/PQyHz8vJktVq1ceNG+zMozvP29rb/d3HHw4Xf7+zZszVs2DAtWrRI8+bN0/PPP6/vv/9ebdu2veTPbEmK22Zpx2VlOH9ry8KFC1W/fn2HZW5ubqWue+TIEac/kBMAAOBKQ5CvZv7+reXmFqyCgiwVf5+8RW5uwfL3L/1MYWVr1aqVdu7cWWzIP788MzNTrq6u9gfSXax58+Zat26d+vbta2/76aef7P/t4eFR4vgXatSokf3+7vNPCj9z5ozWr19/yfd0+/v7q0WLFpo2bZpq1Kih66+/XoGBgerZs6e++eYbh/vj16xZo9DQUIfXil34ILGSNG/eXAcOHFBGRob97OiF+1lRZfmM69atq9TUVIe2zZs3FwliF9u4caNsNpteffVV+9nYzz777LJrvlirVq20Y8eOUr/nmjVrqrDQ8RkQFotFHTp00FdffaXt27fr5ptvlqenpwoKCvTuu+8qJibGHszPf/4HDhywn5XfsWOHjh07pvDw8EvW2KhRI7366qvq3LmzrFarpk2bVmpfX19f7dixw352vVGjRqpRo4bWrVtn/+PE0aNHtWvXLvvx1bJlSxUWFio7O1sdOnS4ZE2ladmypVq2bKnRo0erXbt2+uSTT9S2bdtL/sxWlubNm2vOnDnKz8+3fwerV6+Wi4uLmjVrJj8/P4WEhGjdunXq2LGjpHP3959/xoN07ooJNzc37d+/3+Fn8FJOnTql9PT0S561BwAAuNrwsLtqZrFY1bTJ2PNzFy+VJDVtMkYWi1XVaezYsfrggw/04osvavv27UpLS1NycrKef/55SVJcXJzatWunHj16aMmSJdq7d6/WrFmj5557zv6U+OHDh+v999/X7NmztWvXLo0bN07bt28vdy1eXl569NFHNXLkSC1atEg7duzQQw89pBMnTmjgwIGXXL9z5876+OOP7YEhICBAzZs317x58xxCRJMmTbR//34lJycrPT1db7zxRpEHdBUnLi5OTZs2Vb9+/bRlyxatXLmyUt4xXpbP+NZbb9WGDRv0wQcfaPfu3Ro3blyRYF+cxo0b68yZM3rzzTf122+/6cMPP7Q/BK8yjRo1SmvWrNGQIUO0efNm7d69W1999ZXDg9HCwsL0448/6uDBgzp8+LC9vXPnzvr0008VFRUlb29vubi4qGPHjg7fpXTuc4qIiFDv3r21adMmpaSkqG/fvurUqVORWy1K0rRpUy1btkyff/55qX8ccnFxUVxcnFatWmVv8/b21sCBAzVy5Ej98MMPSk1NVf/+/R0uV2/atKl69+6tvn37asGCBdqzZ49SUlKUlJSkhQsXlqnGPXv2aPTo0Vq7dq327dunJUuWaPfu3fbbEy71M1tZevfuLXd3d/Xr10+pqalatmyZhg4dqj59+tivqhk+fLgmTZqkL7/8Ur/88osee+wxHTt2zD6Gj4+PnnrqKT3xxBOaO3eu0tPTtWnTJr355puaO3duidv+6aef5ObmVuZbVwAAAK4WBHknCAyMV8SN0+Xm5nhpuZtbsCJunF6l75EvSXx8vL755hstWbJErVu3Vtu2bfX666/bz4hbLBZ9++236tixowYMGKCmTZvq/vvv1759++z/M9+zZ0+NGTNGTz/9tKKjo7Vv3z49+uijFapn0qRJuvvuu9WnTx+1atVKv/76qxYvXqxatWpdct1OnTqpsLCwyCvZLm6788479cQTT2jIkCGKiorSmjVrNGbMmEuO7+Lioi+++EInT55UmzZtNGjQoEs+KbwsyvIZx8fH2z/j1q1b6/jx4w5XQJQkMjJSr732ml5++WXdeOON+vjjj5WUlHTZNV+sRYsWWrFihXbt2qUOHTqoZcuWGjt2rMPzBMaPH6+9e/eqUaNGDpdMl/V7s1gs+uqrr1SrVi117NhRcXFxuu666zRv3rxy1dqsWTP98MMP+vTTT/Xkk0+W2G/QoEFKTk52uNT8lVdeUYcOHdStWzfFxcXp5ptvLvJcgtmzZ6tv37568skn1axZM/Xo0UPr168v9RaDC3l6euqXX37R3XffraZNm2rw4MF6/PHH9fDDD0u69M9sZfH09NTixYt15MgRtW7dWvfcc49uu+02hysZnnzySfXp00f9+vVTu3bt5OPjo7/97W8O40yYMEFjxoxRUlKSmjdvroSEBC1cuFANGzYscduffvqpevfuXa5bPwAAAK4GFuPiG26h3Nxc+fn5KScnp8iDz06dOqU9e/aU+m7qsjKMQh07tl4FBdlycwuUv3/raj8TD6B0hmEoNjZWTzzxhHr16uXscq4ahw8fVrNmzbRhw4YSw35l/j4GAABwttJy6MW4R96JLBaratVq6+wyAJTCYrHovffe07Zt25xdylVl7969euutt0o9Yw8AAHC1IsgDwCVERUUpKirK2WVcVWJiYsr8zAMAAICrDffIAwAAAABgIgR5AAAAAABMhCAPAAAAAICJEOQBAAAAADARgjwAAAAAACZCkAcAAAAAwEQI8qiQzp07a8SIEc4u44o2ZswYDR482D5/JXxm/fv3V48ePezzVVHTM888o6FDh1bqmAAAAAD+H0H+KtK/f39ZLBZNmjTJof3LL7+UxWIp11gLFizQhAkTKrO8Is7Xe36qXbu2EhIStHXr1irdbmXIzMzU1KlT9dxzzzm7lFJVxff41FNPae7cufrtt98qdVwAAAAA5xDknajQMLT66HF9kXVUq48eV6FhVPk23d3d9fLLL+vo0aOXNU5AQIB8fHwqqaqSJSQkKCMjQxkZGVq6dKlcXV11xx13VPl2L9fMmTPVvn17hYaGOruUUlXF91inTh3Fx8fr7bffrtRxAQAAAJxDkHeShYeOKWbtDt29OV2P7tinuzenK2btDi08dKxKtxsXF6fg4GAlJSWV2OePP/5Qr169VL9+fXl6eioiIkKffvqpQ58LL8l+9tlnFRsbW2ScyMhIjR8/3j4/c+ZMNW/eXO7u7rr++uv11ltvXbJeNzc3BQcHKzg4WFFRUXrmmWd04MABHTp0yN5n1KhRatq0qTw9PXXddddpzJgxOnPmjCRp7969cnFx0YYNGxzGnTJlikJDQ2Wz2SRJqampuv322+Xt7a2goCD16dNHhw8ftvf/17/+pYiICHl4eKh27dqKi4tTfn5+iXUnJyerW7duRdrPnj2rIUOGyM/PT3Xq1NGYMWNkXPAHnA8//FAxMTHy8fFRcHCw/v73vys7O9u+/OjRo+rdu7fq1q0rDw8PNWnSRLNnz7YvP3DggO677z75+/srICBA3bt31969e0us8+JL68PCwjRx4kQ9+OCD8vHx0bXXXqv33nvPYZ2ybKNbt25KTk4ucbsAAAAAKo4g7wQLDx3ToNS9yig449CeWXBGg1L3VmmYt1qtmjhxot588039/vvvxfY5deqUoqOjtXDhQqWmpmrw4MHq06ePUlJSiu3fu3dvpaSkKD093d62fft2bd26VX//+98lSR9//LHGjh2rl156SWlpaZo4caLGjBmjuXPnlrn2vLw8ffTRR2rcuLFq165tb/fx8dGcOXO0Y8cOTZ06VTNmzNDrr78u6VwwjYuLcwi7kjR79mz1799fLi4uOnbsmG699Va1bNlSGzZs0KJFi5SVlaX77rtPkpSRkaFevXrpwQcfVFpampYvX6677rrLIYBf6MiRI9qxY4diYmKKLJs7d65cXV2VkpKiqVOn6rXXXtPMmTPty8+cOaMJEyZoy5Yt+vLLL7V3717179/fvnzMmDHasWOHvvvuO6Wlpentt99WnTp17OvGx8fLx8dHK1eu1OrVq+Xt7a2EhASdPn26zJ/zq6++qpiYGP3888967LHH9Oijj2rnzp3l2kabNm30+++/l/pHBAAAAAAVZKCInJwcQ5KRk5NTZNnJkyeNHTt2GCdPnqzQ2GdtNiNqdaoR9MPPxU7BP/xstFydapy12S53N4ro16+f0b17d8MwDKNt27bGgw8+aBiGYXzxxRfGpQ6Frl27Gk8++aR9vlOnTsbw4cPt85GRkcb48ePt86NHjzZiY2Pt840aNTI++eQThzEnTJhgtGvXrtR6rVar4eXlZXh5eRmSjJCQEGPjxo2l1vrKK68Y0dHR9vl58+YZtWrVMk6dOmUYhmFs3LjRsFgsxp49e+x1dOnSxWGMAwcOGJKMnTt3Ghs3bjQkGXv37i11u+f9/PPPhiRj//79Du2dOnUymjdvbtgu+G5HjRplNG/evMSx1q9fb0gyjh8/bhiGYXTr1s0YMGBAsX0//PBDo1mzZg7jFxQUGB4eHsbixYsNw3A8Bs7XdOH3GBoaajzwwAP2eZvNZgQGBhpvv/12mbdhGP//M7R8+fIS9w24XJf7+xgAAOBKUloOvRhn5KvZT8fyipyJv5Ah6b8FZ/TTsbwqrePll1/W3LlzlZaWVmRZYWGhJkyYoIiICAUEBMjb21uLFy/W/v37Sxyvd+/e+uSTTyRJhmHo008/Ve/evSVJ+fn5Sk9P18CBA+Xt7W2f/vGPfzicxS/OLbfcos2bN2vz5s1KSUlRfHy8br/9du3bt8/eZ968ebrpppsUHBwsb29vPf/88w619ujRQ1arVV988YUkac6cObrlllsUFhYmSdqyZYuWLVvmUNv1118vSUpPT1dkZKRuu+02RURE6N5779WMGTNKfcbAyZMnJZ17HsHF2rZt6/BgwXbt2mn37t0qLCyUJG3cuFHdunXTtddeKx8fH3Xq1EmS7Pvz6KOPKjk5WVFRUXr66ae1Zs0a+1hbtmzRr7/+Kh8fH/t+BAQE6NSpU5f8nC/UokUL+39bLBYFBwfbL+8v6zY8PDwkSSdOnCjzdgEAAACUjauzC7jaZJ8+W6n9Kqpjx46Kj4/X6NGjHS7dlqRXXnlFU6dO1ZQpUxQRESEvLy+NGDGi1Muze/XqpVGjRmnTpk06efKkDhw4oJ49e0o6d0m8JM2YMaPIvfRWq7XUOr28vNS4cWP7/MyZM+Xn56cZM2boH//4h9auXavevXvrxRdfVHx8vPz8/JScnKxXX33Vvk7NmjXVt29fzZ49W3fddZc++eQTTZ061b48Ly9P3bp108svv1xk+yEhIbJarfr++++1Zs0aLVmyRG+++aaee+45rVu3Tg0bNiyyzvlL3Y8ePaq6deuWun8Xys/PV3x8vOLj4/Xxxx+rbt262r9/v+Lj4+2f/fk/Ynz77bf6/vvvddttt+nxxx/X5MmTlZeXp+joaH388cdFxi5PHTVq1HCYt1gs9mcJlHUbR44cKfd2AQAAAJQNQb6aBdYs20de1n6XY9KkSYqKilKzZs0c2levXq3u3bvrgQcekCTZbDbt2rVL4eHhJY51zTXXqFOnTvr444918uRJ/eUvf1FgYKAkKSgoSPXq1dNvv/1mP0tfURaLRS4uLvaz3mvWrFFoaKjDa94uPFt/3qBBg3TjjTfqrbfe0tmzZ3XXXXfZl7Vq1Uqff/65wsLC5Opa/OdusVh000036aabbtLYsWMVGhqqL774QomJiUX6NmrUSL6+vtqxY4eaNm3qsGzdunUO8z/99JOaNGkiq9WqX375RX/88YcmTZqkBg0aSFKRh/RJ58Jxv3791K9fP3Xo0EEjR47U5MmT1apVK82bN0+BgYHy9fUt6SO8LGXdRmpqqmrUqKEbbrihSuoAAAAArmZcWl/N2vp7K8Sthkp6a7tFUj23Gmrr713ltURERKh379564403HNqbNGliPwOdlpamhx9+WFlZWZccr3fv3kpOTtb8+fOLBPYXX3xRSUlJeuONN7Rr1y5t27ZNs2fP1muvvVbqmAUFBcrMzFRmZqbS0tI0dOhQ+xn087Xu379fycnJSk9P1xtvvGG/hP5CzZs3V9u2bTVq1Cj16tXLfum3JD3++OM6cuSIevXqpfXr1ys9PV2LFy/WgAEDVFhYqHXr1mnixInasGGD9u/frwULFujQoUNq3rx5sTW7uLgoLi5Oq1atKrJs//79SkxM1M6dO/Xpp5/qzTff1PDhwyVJ1157rWrWrKk333xTv/32m77++usi73gfO3asvvrqK/3666/avn27vvnmG3sdvXv3Vp06ddS9e3etXLlSe/bs0fLlyzVs2LASH2xYXmXdxsqVK9WhQweHzxkAAABA5SDIVzOrxaJ/NKkvSUXC/Pn5CU3qy2opKepXrvHjx9svmz7v+eefV6tWrRQfH6/OnTsrODhYPXr0uORY99xzj/744w+dOHGiSP9BgwZp5syZmj17tiIiItSpUyfNmTOn2EvTL7Ro0SKFhIQoJCREsbGxWr9+vebPn6/OnTtLku6880498cQTGjJkiKKiorRmzRqNGTOm2LEGDhyo06dP68EHH3Ror1evnlavXq3CwkJ16dJFERERGjFihPz9/eXi4iJfX1/9+OOP+utf/6qmTZvq+eef16uvvqrbb7+9xLoHDRqk5OTkIp9t3759dfLkSbVp00aPP/64hg8frsGDB0s6d6Z9zpw5mj9/vsLDwzVp0iRNnjzZYf2aNWtq9OjRatGihTp27Cir1Wp/zZunp6d+/PFHXXvttbrrrrvUvHlzDRw4UKdOnaq0M/Rl3UZycrIeeuihStkmAAAAAEcWwyjhHVpXsdzcXPn5+SknJ6dIADp16pT27Nmjhg0bFvsws7JaeOiYnt990OHBd/XcamhCk/rqWte/wuOiZBMmTND8+fO1devWKt+WYRiKjY3VE088oV69elX59q4k3333nZ588klt3bq1xFsVgMpQWb+PAQAArgSl5dCL8X/ZTtK1rr8S6vjpp2N5yj59VoE1XdXW37vazsRfTfLy8rR3715NmzZN//jHP6plmxaLRe+99562bdtWLdu7kuTn52v27NmEeAAAAKCK8H/aTmS1WHRTLR9nl/GnN2TIEH366afq0aNHkcvqq1JUVJSioqKqbXtXinvuucfZJQAAAAB/agR5/OnNmTNHc+bMcXYZAAAAAFApeNgdAAAAAAAmQpCvIJ4RCADOxe9hAABwtSLIl1ONGjUkSSdOnHByJQBwdTv/e/j872UAAICrBffIl5PVapW/v7+ys7MlnXuvtoUnzQNAtTEMQydOnFB2drb8/f1ltVqdXRIAAEC1IshXQHBwsCTZwzwAoPr5+/vbfx8DAABcTQjyFWCxWBQSEqLAwECdOXPG2eUAwFWnRo0anIkHAABXLYL8ZbBarfyPJAAAAACgWvGwOwAAAAAATIQgDwAAAACAiRDkAQAAAAAwEYI8AAAAAAAmQpAHAAAAAMBECPIAAAAAAJgIQR4AAAAAABMhyAMAAAAAYCJXRJCfPn26wsLC5O7urtjYWKWkpJTaf8qUKWrWrJk8PDzUoEEDPfHEEzp16tRljQkAAAAAgBk4PcjPmzdPiYmJGjdunDZt2qTIyEjFx8crOzu72P6ffPKJnnnmGY0bN05paWmaNWuW5s2bp2effbbCYwIAAAAAYBYWwzAMZxYQGxur1q1ba9q0aZIkm82mBg0aaOjQoXrmmWeK9B8yZIjS0tK0dOlSe9uTTz6pdevWadWqVRUa82K5ubny8/NTTk6OfH19K2M3AQAAAAAoUXlyqFPPyJ8+fVobN25UXFycvc3FxUVxcXFau3Ztseu0b99eGzdutF8q/9tvv+nbb7/VX//61wqPWVBQoNzcXIcJAAAAAIArkaszN3748GEVFhYqKCjIoT0oKEi//PJLsev8/e9/1+HDh3XzzTfLMAydPXtWjzzyiP3S+oqMmZSUpBdffLES9ggAAAAAgKrl9Hvky2v58uWaOHGi3nrrLW3atEkLFizQwoULNWHChAqPOXr0aOXk5NinAwcOVGLFAAAAAABUHqeeka9Tp46sVquysrIc2rOyshQcHFzsOmPGjFGfPn00aNAgSVJERITy8/M1ePBgPffccxUa083NTW5ubpWwRwAAAAAAVC2nnpGvWbOmoqOjHR5cZ7PZtHTpUrVr167YdU6cOCEXF8eyrVarJMkwjAqNCQAAAACAWTj1jLwkJSYmql+/foqJiVGbNm00ZcoU5efna8CAAZKkvn37qn79+kpKSpIkdevWTa+99ppatmyp2NhY/frrrxozZoy6detmD/SXGhMAAAAAALNyepDv2bOnDh06pLFjxyozM1NRUVFatGiR/WF1+/fvdzgD//zzz8tisej555/XwYMHVbduXXXr1k0vvfRSmccEAAAAAMCsnP4e+SsR75EHAAAAAFQn07xHHgAAAAAAlA9BHgAAAAAAEyHIAwAAAABgIgR5AAAAAABMhCAPAAAAAICJEOQBAAAAADARgjwAAAAAACZCkAcAAAAAwEQI8gAAAAAAmAhBHgAAAAAAEyHIAwAAAABgIgR5AAAAAABMhCAPAAAAAICJEOQBAAAAADARgjwAAAAAACZCkAcAAAAAwEQI8gAAAAAAmAhBHgAAAAAAEyHIAwAAAABgIgR5AAAAAABMhCAPAAAAAICJEOQBAAAAADARgjwAAAAAACZCkAcAAAAAwEQI8gAAAAAAmAhBHgAAAAAAEyHIAwAAAABgIgR5AAAAAABMhCAPAAAAAICJEOQBAAAAADARgjwAAAAAACZCkAcAAAAAwEQI8gAAAAAAmAhBHgAAAAAAEyHIAwAAAABgIgR5AAAAAABMhCAPAAAAAICJEOQBAAAAADARgjwAAAAAACZCkAcAAAAAwEQqHOQ//PBD3XTTTapXr5727dsnSZoyZYq++uqrSisOAAAAAAA4qlCQf/vtt5WYmKi//vWvOnbsmAoLCyVJ/v7+mjJlSmXWBwAAAAAALlChIP/mm29qxowZeu6552S1Wu3tMTEx2rZtW6UVBwAAAAAAHFUoyO/Zs0ctW7Ys0u7m5qb8/PzLLgoAAAAAABSvQkG+YcOG2rx5c5H2RYsWqXnz5pdbEwAAAAAAKIFrRVZKTEzU448/rlOnTskwDKWkpOjTTz9VUlKSZs6cWdk1AgAAAACA/6lQkB80aJA8PDz0/PPP68SJE/r73/+uevXqaerUqbr//vsru0YAAAAAAPA/5Q7yZ8+e1SeffKL4+Hj17t1bJ06cUF5engIDA6uiPgAAAAAAcIFy3yPv6uqqRx55RKdOnZIkeXp6EuIBAAAAAKgmFXrYXZs2bfTzzz9Xdi0AAAAAAOASKnSP/GOPPaYnn3xSv//+u6Kjo+Xl5eWwvEWLFpVSHAAAAAAAcGQxDMMo70ouLkVP5FssFhmGIYvFosLCwkopzllyc3Pl5+ennJwc+fr6OrscAAAAAMCfXHlyaIXOyO/Zs6dChQEAAAAAgMtToSAfGhpa2XUAAAAAAIAyqFCQl6T09HRNmTJFaWlpkqTw8HANHz5cjRo1qrTiAAAAAACAowo9tX7x4sUKDw9XSkqKWrRooRYtWmjdunW64YYb9P3331d2jQAAAAAA4H8q9LC7li1bKj4+XpMmTXJof+aZZ7RkyRJt2rSp0gp0Bh52BwAAAACoTuXJoRU6I5+WlqaBAwcWaX/wwQe1Y8eOigwJAAAAAADKoEJBvm7dutq8eXOR9s2bNyswMPByawIAAAAAACWo0MPuHnroIQ0ePFi//fab2rdvL0lavXq1Xn75ZSUmJlZqgQAAAAAA4P9V6B55wzA0ZcoUvfrqq/rvf/8rSapXr55GjhypYcOGyWKxVHqh1Yl75AEAAADA/ApthlL2HFH28VMK9HFXm4YBsrpcmXm1PDm0QkH+QsePH5ck+fj4XM4wVxSCPAAAAACY26LUDL347x3KyDllbwvxc9e4buFKuDHEiZUVr8ofdrdnzx7t3r1b0rkAfz7E7969W3v37q3IkAAAAAAAVIpFqRl69KNNDiFekjJzTunRjzZpUWqGkyqrHBUK8v3799eaNWuKtK9bt079+/e/3JoAAAAAAKiQQpuhF/+9Q8Vden6+7cV/71Ch7bIuTneqCgX5n3/+WTfddFOR9rZt2xb7NPtLmT59usLCwuTu7q7Y2FilpKSU2Ldz586yWCxFpq5du9r7ZGVlqX///qpXr548PT2VkJBgv4IAAAAAAPDnlbLnSJEz8RcyJGXknFLKniPVV1Qlq1CQt1gs9nvjL5STk6PCwsJyjTVv3jwlJiZq3Lhx2rRpkyIjIxUfH6/s7Oxi+y9YsEAZGRn2KTU1VVarVffee6+kcw/i69Gjh3777Td99dVX+vnnnxUaGqq4uDjl5+eXf2cBAAAAAKaRfbzkEF+RfleiCgX5jh07KikpySG0FxYWKikpSTfffHO5xnrttdf00EMPacCAAQoPD9c777wjT09Pvf/++8X2DwgIUHBwsH36/vvv5enpaQ/yu3fv1k8//aS3335brVu3VrNmzfT222/r5MmT+vTTTyuyuwAAAAAAkwj0ca/UfleiCr1H/uWXX1bHjh3VrFkzdejQQZK0cuVK5ebm6ocffijzOKdPn9bGjRs1evRoe5uLi4vi4uK0du3aMo0xa9Ys3X///fLy8pIkFRQUSJLc3f//S3FxcZGbm5tWrVqlQYMGFRmjoKDAvp507mmBAAAAAADzadMwQCF+7srMOVXsffIWScF+515FZ1YVOiMfHh6urVu36r777lN2draOHz+uvn376pdfftGNN95Y5nEOHz6swsJCBQUFObQHBQUpMzPzkuunpKQoNTXVIZxff/31uvbaazV69GgdPXpUp0+f1ssvv6zff/9dGRnFP5kwKSlJfn5+9qlBgwZl3gcAAAAAwJXD6mLRuG7hks6F9gudnx/XLfyKfZ98WVTojLwk1atXTxMnTqzMWspt1qxZioiIUJs2bextNWrU0IIFCzRw4EAFBATIarUqLi5Ot99+uwyj+KcSjh49WomJifb53NxcwjwAAAAAmFTCjSF6+4FWRd4jH3wFv0e+PMoV5A8fPqz8/HyFhoba27Zv367JkycrPz9fPXr00N///vcyj1enTh1ZrVZlZWU5tGdlZSk4OLjUdfPz85WcnKzx48cXWRYdHa3NmzcrJydHp0+fVt26dRUbG6uYmJhix3Jzc5Obm1uZ6wYAAAAAXNkSbgzRX8KDlbLniLKPn1Kgz7nL6c18Jv68cl1aP3ToUL3xxhv2+ezsbHXo0EHr169XQUGB+vfvrw8//LDM49WsWVPR0dFaunSpvc1ms2np0qVq165dqevOnz9fBQUFeuCBB0rs4+fnp7p162r37t3asGGDunfvXubaAAAAAADmZnWxqF2j2uoeVV/tGtX+U4R4qZxn5H/66SfNmTPHPv/BBx8oICBAmzdvlqurqyZPnqzp06erT58+ZR4zMTFR/fr1U0xMjNq0aaMpU6YoPz9fAwYMkCT17dtX9evXV1JSksN6s2bNUo8ePVS7du0iY86fP19169bVtddeq23btmn48OHq0aOHunTpUp7dBQAAAACYlFFYqBMbNursoUNyrVtXnjHRslitzi6rUpQryGdmZiosLMw+/8MPP+iuu+6Sq+u5Ye68884igftSevbsqUOHDmns2LHKzMxUVFSUFi1aZH8A3v79++Xi4njhwM6dO7Vq1SotWbKk2DEzMjKUmJiorKwshYSEqG/fvhozZky56gIAAAAAmFPukiXKmpiksxc8RN01OFhBz46W75/gBK/FKOkJcMUICgrSkiVLFBkZKencPe7vvvuu7r77bknn3uHesmVL5eXlVU211SQ3N1d+fn7KycmRr6+vs8sBAAAAAJRR7pIlOjh8hHRx1LWcu6y+/tQpV2SYL08OLdc98m3bttUbb7whm82mf/3rXzp+/LhuvfVW+/Jdu3bxtHcAAAAAgFMYhYXKmphUNMRL9rasiUkyCgurubLKVa4gP2HCBH399dfy8PBQz5499fTTT6tWrVr25cnJyerUqVOlFwkAAAAAwKWc2LDR4XL6IgxDZzMzdWLDxuorqgqU6x75Fi1aKC0tTatXr1ZwcLBiY2Mdlt9///0KDw+v1AIBAAAAACiLs4cOVWq/K1W5grx07r74C1/j9vvvv6tevXpycXFR165dK7U4AAAAAADKyrVu3Urtd6Uq16X1xQkPD9fevXsroRQAAAAAACrOMyZarsHB9gfbFWGxyDU4WJ4x0dVbWCW77CBfjofeAwAAAABQZSxWq4KeHf2/mYvC/P/mg54dbfr3yV92kAcAAAAA4Erh26WL6k+dItegIId216CgK/bVc+VV7nvkL/bss88qICCgMmoBAAAAAOCy+XbpIp/bbjv3FPtDh+Rat648Y6JNfyb+PIvBtfFF5Obmys/PTzk5OfL19XV2OQAAAACAP7ny5NBKvbT+wIEDevDBBytzSAAAAAAAcIFKDfJHjhzR3LlzK3NIAAAAAABwgXLdI//111+Xuvy33367rGIAAAAAAEDpyhXke/ToIYvFUuor5ywlva8PAAAAAABctnJdWh8SEqIFCxbIZrMVO23atKmq6gQAAAAAACpnkI+OjtbGjRtLXH6ps/UAAAAAAODylOvS+pEjRyo/P7/E5Y0bN9ayZcsuuygAAAAAAFC8cgX5+vXrq2HDhiUu9/LyUqdOnS67KFxaoWHop2N5yj59VoE1XdXW31tWnk8AAAAAAH965bq0vkmTJjp06JB9vmfPnsrKyqr0olC6hYeOKWbtDt29OV2P7tinuzenK2btDi08dMzZpQEAAAAAqli5gvzF979/++23pV5qj8q38NAxDUrdq4yCMw7tmQVnNCh1L2EeAAAAAP7kyhXk4VyFhqHndx9UcY8TPN82ZvdBFfLAQQAAAAD40ypXkLdYLEXeE89746vPT8fyipyJv5Ah6b8FZ/TTsbzqKwoAAAAAUK3K9bA7wzDUv39/ubm5SZJOnTqlRx55RF5eXg79FixYUHkVwi779NlK7QcAAAAAMJ9yBfl+/fo5zD/wwAOVWgxKF1izbF9XWfsBAAAAAMynXIlv9uzZVVUHyqCtv7dC3Goos+BMsffJWySFuNVQW3/v6i4NAAAAAFBNeNidiVgtFv2jSX1J50L7hc7PT2hSn/fJAwAAAMCfGEHeZLrW9dfMG8MU7FbDoT3ErYZm3himrnX9nVMYAAAAAKBacDO1CXWt66+EOn766Viesk+fVWBNV7X19+ZMPAAAAABcBQjyJmW1WHRTLR9nlwEAAAAAqGZcWg8AAAAAgIkQ5AEAAAAAMBGCPAAAAAAAJkKQBwAAAADARAjyAAAAAACYCEEeAAAAAAATIcgDAAAAAGAiBHkAAAAAAEyEIA8AAAAAgIkQ5AEAAAAAMBGCPAAAAAAAJkKQBwAAAADARFydXQAAmJVhFOrYsfUqKMiWm1ug/P1by2KxOrssAAAA/MkR5AGgArKzF2vX7vEqKMi0t7m5Batpk7EKDIx3YmUAAAD4s+PSegAop+zsxdqW+rhDiJekgoIsbUt9XNnZi51UGQAAAK4GBHkAKAfDKNSu3eMlGcUtlSTt2j1BhlFYrXUBAADg6kGQB4ByOHdPfGYpPQwVFGTo2LH11VYTAAAAri4EeQAoh4KC7ErtBwAAAJQXQR4AysHNLbBS+wEAAADlRZAHgHLw928tN7dgSZYSeljk5hYif//W1VkWAAAAriIEeQAoB4vFqqZNxp6fu3ipJKlpkzG8Tx4AAABVhiAPAOUUGBiviBuny80tyKHdzS1YETdO5z3yAAAAqFKuzi4AAMwoMDBedevG/e8p9tlycwuUv39rzsQDAACgyhHkAaCCLBaratVq6+wyAAAAcJXh0noAAAAAAEyEIA8AAAAAgIkQ5AEAAAAAMBGCPAAAAAAAJkKQBwAAAADARAjyAAAAAACYCEEeAAAAAAATIcgDAAAAAGAiBHkAAAAAAEyEIA8AAAAAgIkQ5AEAAAAAMJErIshPnz5dYWFhcnd3V2xsrFJSUkrs27lzZ1ksliJT165d7X3y8vI0ZMgQXXPNNfLw8FB4eLjeeeed6tgVAAAAAACqlNOD/Lx585SYmKhx48Zp06ZNioyMVHx8vLKzs4vtv2DBAmVkZNin1NRUWa1W3XvvvfY+iYmJWrRokT766COlpaVpxIgRGjJkiL7++uvq2i0AAAAAAKqE04P8a6+9poceekgDBgywnzn39PTU+++/X2z/gIAABQcH26fvv/9enp6eDkF+zZo16tevnzp37qywsDANHjxYkZGRpZ7pBwAAAADADJwa5E+fPq2NGzcqLi7O3ubi4qK4uDitXbu2TGPMmjVL999/v7y8vOxt7du319dff62DBw/KMAwtW7ZMu3btUpcuXYodo6CgQLm5uQ4TAAAAAABXIqcG+cOHD6uwsFBBQUEO7UFBQcrMzLzk+ikpKUpNTdWgQYMc2t98802Fh4frmmuuUc2aNZWQkKDp06erY8eOxY6TlJQkPz8/+9SgQYOK7xQAAAAAAFXI6ZfWX45Zs2YpIiJCbdq0cWh/88039dNPP+nrr7/Wxo0b9eqrr+rxxx/Xf/7zn2LHGT16tHJycuzTgQMHqqN8AAAAAADKzdWZG69Tp46sVquysrIc2rOyshQcHFzquvn5+UpOTtb48eMd2k+ePKlnn31WX3zxhf1J9i1atNDmzZs1efJkh8v4z3Nzc5Obm9tl7g0AAAAAAFXPqWfka9asqejoaC1dutTeZrPZtHTpUrVr167UdefPn6+CggI98MADDu1nzpzRmTNn5OLiuGtWq1U2m63yigcAAAAAwAmcekZeOvequH79+ikmJkZt2rTRlClTlJ+frwEDBkiS+vbtq/r16yspKclhvVmzZqlHjx6qXbu2Q7uvr686deqkkSNHysPDQ6GhoVqxYoU++OADvfbaa9W2XwAAAAAAVAWnB/mePXvq0KFDGjt2rDIzMxUVFaVFixbZH4C3f//+ImfXd+7cqVWrVmnJkiXFjpmcnKzRo0erd+/eOnLkiEJDQ/XSSy/pkUceqfL9AQAAAACgKlkMwzCcXcSVJjc3V35+fsrJyZGvr6+zywEAAAAA/MmVJ4ea+qn1AAAAAABcbQjyAAAAAACYCEEeAAAAAAATIcgDAAAAAGAiBHkAAAAAAEyEIA8AAAAAgIkQ5AEAAAAAMBGCPAAAAAAAJkKQBwAAAADARAjyAAAAAACYCEEeAAAAAAATIcgDAAAAAGAiBHkAAAAAAEyEIA8AAAAAgIkQ5AEAAAAAMBGCPAAAAAAAJkKQBwAAAADARAjyAAAAAACYCEEeAAAAAAATIcgDAAAAAGAiBHkAAAAAAEyEIA8AAAAAgIkQ5AEAAAAAMBGCPAAAAAAAJkKQBwAAAADARAjyAAAAAACYCEEeAAAAAAATIcgDAAAAAGAiBHkAAAAAAEyEIA8AAAAAgIkQ5AEAAAAAMBGCPAAAAAAAJkKQBwAAAADARAjyAAAAAACYCEEeAAAAAAATIcgDAAAAAGAiBHkAAAAAAEyEIA8AAAAAgIkQ5AEAAAAAMBGCPAAAAAAAJkKQBwAAAADARAjyAAAAAACYCEEeAAAAAAATIcgDAAAAAGAiBHkAAAAAAEyEIA8AAAAAgIkQ5AEAAAAAMBGCPAAAAAAAJkKQBwAAAADARAjyAAAAAACYCEEeAAAAAAATIcgDAAAAAGAiBHkAAAAAAEzE1dkFAAAAXK1sNpv27dunvLw8eXt7KzQ0VC4unGcBAJSOIA8AAOAEO3bs0KJFi5Sbm2tv8/X1VUJCgsLDw51YGQDgSseffAEAAKrZjh079NlnnzmEeEnKzc3VZ599ph07djipMgCAGRDkAQAAqpHNZtOiRYtK7bNo0SLZbLZqqggAYDYEeQAAgGq0b9++ImfiL5abm6t9+/ZVU0UAALMhyAMAAFSjvLy8Su0HALj6EOQBAACqkbe3d6X2AwBcfQjyAAAA1Sg0NFS+vr6l9vH19VVoaGg1VQQAMBuCPAAAQDVycXFRQkJCqX0SEhJ4nzwAoET8CwEAAFDNwsPDdd999xU5M+/r66v77ruP98gDAErl6uwCAAAArkbh4eG6/vrrtW/fPuXl5cnb21uhoaGciQcAXNIV8S/F9OnTFRYWJnd3d8XGxiolJaXEvp07d5bFYikyde3a1d6nuOUWi0WvvPJKdewOAABAmbi4uKhhw4aKiIhQw4YNCfEAgDJx+r8W8+bNU2JiosaNG6dNmzYpMjJS8fHxys7OLrb/ggULlJGRYZ9SU1NltVp177332vtcuDwjI0Pvv/++LBaL7r777uraLQAAAAAAqoTFMAzDmQXExsaqdevWmjZtmiTJZrOpQYMGGjp0qJ555plLrj9lyhSNHTtWGRkZ8vLyKrZPjx49dPz4cS1durRMNeXm5srPz085OTmXfKosAAAAAACXqzw51Kn3yJ8+fVobN27U6NGj7W0uLi6Ki4vT2rVryzTGrFmzdP/995cY4rOysrRw4ULNnTu3xDEKCgpUUFBgn8/NzS3jHgAAAAAAqoPNVqiDaduVd+yovP1rqX7zG+TiYnV2WU7h1CB/+PBhFRYWKigoyKE9KChIv/zyyyXXT0lJUWpqqmbNmlVin7lz58rHx0d33XVXiX2SkpL04osvlr1wAAAAAEC12b1ujX6Y857yjhy2t3kH1NGt/QerSWx7J1bmHE6/R/5yzJo1SxEREWrTpk2Jfd5//3317t1b7u7uJfYZPXq0cnJy7NOBAweqolwAAAAAQDntXrdGX7820SHES1LekcP6+rWJ2r1ujZMqcx6nBvk6derIarUqKyvLoT0rK0vBwcGlrpufn6/k5GQNHDiwxD4rV67Uzp07NWjQoFLHcnNzk6+vr8MEAAAAAHAum61QP8x5r9Q+y+a+J5utsJoqujI4NcjXrFlT0dHRDg+hs9lsWrp0qdq1a1fquvPnz1dBQYEeeOCBEvvMmjVL0dHRioyMrLSaAQAAAADV42Da9iJn4i92/I/DOpi2vZoqujI4/dL6xMREzZgxQ3PnzlVaWpoeffRR5efna8CAAZKkvn37OjwM77xZs2apR48eql27drHj5ubmav78+Zc8Gw8AAAAAuDLlHTtaqf3+LJz6sDtJ6tmzpw4dOqSxY8cqMzNTUVFRWrRokf0BePv375eLi+PfG3bu3KlVq1ZpyZIlJY6bnJwswzDUq1evKq0fAAAAAFA1vP1rVWq/Pwunv0f+SsR75AEAAADA+Wy2Qs14fGCpl9f71K6jQdNmmf5VdOXJoU6/tB4AAAAAgOK4uFh1a//Bpfa5pd9g04f48iLIAwAAAACuWE1i2+vOxGflHVDHod2ndh3dmfjsVfkeeaffIw8AAAAAQGmaxLZXo9ax555if+yovP1rqX7zG666M/HnEeQBAAAAAFc8FxerGtzQwtllXBG4tB4AAAAAABMhyAMAAAAAYCIEeQAAAAAATIQgDwAAAACAiRDkAQAAAAAwEYI8AAAAAAAmQpAHAAAAAMBECPIAAAAAAJiIq7MLAAAA5mPYDBXsyZHt+Gm5+NSUW0M/WVwszi4LAICrAkEeAACUy8nUwzr273QV5py2t1n9asq/WyN53FjHiZUBAHB14NJ6AABQZidTD+uPj9IcQrwkFeac1h8fpelk6mEnVQYAwNWDIA8AAMrEsBk69u/0Uvsc+/dvMmxGNVUEAMDViSAPAADKpGBPTpEz8RcrzClQwZ6caqoIAICrE0EeAACUie146SG+vP0AAEDFEOQBAECZuPjUrNR+AACgYgjyAACgTNwa+snqV3pIt/q5ya2hXzVVBADA1YkgDwAAysTiYpF/t0al9vHvdh3vkwcAoIoR5AEAQJl53FhHtR9oXuTMvNXPTbUfaM575AEAqAauzi4AAACYi8eNdeQeXlsFe3JkO35aLj415dbQjzPxAABUE4I8AAAoN4uLRe6N/J1dBgAAVyUurQcAAAAAwEQI8gAAAAAAmAhBHgAAAAAAEyHIAwAAAABgIgR5AAAAAABMhCAPAAAAAICJEOQBAAAAADARgjwAAAAAACZCkAcAAAAAwEQI8gAAAAAAmAhBHgAAAAAAEyHIAwAAAABgIgR5AAAAAABMhCAPAAAAAICJEOQBAAAAADARgjwAAAAAACZCkAcAAAAAwEQI8gAAAAAAmAhBHgAAAAAAEyHIAwAAAABgIgR5AAAAAABMhCAPAAAAAICJEOQBAAAAADARgjwAAAAAACZCkAcAAAAAwEQI8gAAAAAAmAhBHgAAAAAAEyHIAwAAAABgIgR5AAAAAABMhCAPAAAAAICJEOQBAAAAADARgjwAAAAAACZCkAcAAAAAwEQI8gAAAAAAmAhBHgAAAAAAEyHIAwAAAABgIgR5AAAAAABMhCAPAAAAAICJEOQBAAAAADCRKyLIT58+XWFhYXJ3d1dsbKxSUlJK7Nu5c2dZLJYiU9euXR36paWl6c4775Sfn5+8vLzUunVr7d+/v6p3BQAAAACAKuX0ID9v3jwlJiZq3Lhx2rRpkyIjIxUfH6/s7Oxi+y9YsEAZGRn2KTU1VVarVffee6+9T3p6um6++WZdf/31Wr58ubZu3aoxY8bI3d29unYLAAAAAIAqYTEMw3BmAbGxsWrdurWmTZsmSbLZbGrQoIGGDh2qZ5555pLrT5kyRWPHjlVGRoa8vLwkSffff79q1KihDz/8sEI15ebmys/PTzk5OfL19a3QGAAAAAAAlFV5cqhTz8ifPn1aGzduVFxcnL3NxcVFcXFxWrt2bZnGmDVrlu6//357iLfZbFq4cKGaNm2q+Ph4BQYGKjY2Vl9++WWJYxQUFCg3N9dhAgAAAADgSuTUIH/48GEVFhYqKCjIoT0oKEiZmZmXXD8lJUWpqakaNGiQvS07O1t5eXmaNGmSEhIStGTJEv3tb3/TXXfdpRUrVhQ7TlJSkvz8/OxTgwYNLm/HAAAAAACoIk6/R/5yzJo1SxEREWrTpo29zWazSZK6d++uJ554QlFRUXrmmWd0xx136J133il2nNGjRysnJ8c+HThwoFrqBwAAAACgvJwa5OvUqSOr1aqsrCyH9qysLAUHB5e6bn5+vpKTkzVw4MAiY7q6uio8PNyhvXnz5iU+td7NzU2+vr4OEwAAAAAAVyKnBvmaNWsqOjpaS5cutbfZbDYtXbpU7dq1K3Xd+fPnq6CgQA888ECRMVu3bq2dO3c6tO/atUuhoaGVVzwAAAAAAE7g6uwCEhMT1a9fP8XExKhNmzaaMmWK8vPzNWDAAElS3759Vb9+fSUlJTmsN2vWLPXo0UO1a9cuMubIkSPVs2dPdezYUbfccosWLVqkf//731q+fHl17BIAAAAAAFXG6UG+Z8+eOnTokMaOHavMzExFRUVp0aJF9gfg7d+/Xy4ujhcO7Ny5U6tWrdKSJUuKHfNvf/ub3nnnHSUlJWnYsGFq1qyZPv/8c918881Vvj8AAAAAAFQlp79H/krEe+QBAAAAANXJNO+RBwAAAAAA5UOQBwAAAADARAjyAAAAAACYCEEeAAAAAAATIcgDAAAAAGAiBHkAAAAAAEyEIA8AAAAAgIkQ5AEAAAAAMBGCPAAAAAAAJkKQBwAAAADARAjyAAAAAACYCEEeAAAAAAATIcgDAAAAAGAiBHkAAAAAAEyEIA8AAAAAgIkQ5AEAAAAAMBGCPAAAAAAAJkKQBwAAAADARAjyAAAAAACYCEEeAAAAAAATIcgDAAAAAGAiBHkAAAAAAEyEIA8AAAAAgIkQ5AEAAAAAMBGCPAAAAAAAJkKQBwAAAADARAjyAAAAAACYCEEeAAAAAAATIcgDAAAAAGAiBHkAAAAAAEyEIA8AAAAAgIkQ5AEAAAAAMBGCPAAAAAAAJkKQBwAAAADARAjyAAAAAACYCEEeAAAAAAATIcgDAAAAAGAiBHkAAAAAAEyEIA8AAAAAgIkQ5AEAAAAAMBGCPAAAAAAAJkKQBwAAAADARAjyAAAAAACYiKuzCwAAOJ/NZihj9zHl5xbIy9dNIU385eJicXZZAAAAKAZBHgCucuk/Z2vlvN3KP1Zgb/Pyd1OHnk3UqGWgEysDAABAcbi0HgCuYuk/Z2vRu6kOIV6S8o8VaNG7qUr/OdtJlQEAAKAkBHkAuErZbIZWzttdap9Vn+2WzWZUU0UAAAAoC4I8AFylMnYfK3Im/mJ5RwuUsftY9RQEAACAMiHIA8BVKj+39BBf3n4AAACoHgR5ALhKefm6VWo/AAAAVA+CPABcpUKa+MvLv/SQ7l3r3KvoAAAAcOUgyAPAVcrFxaIOPZuU2ufm+5rwPnkAAIArDEEeAK5ijVoGKuHhG4ucmfeu5aaEh2/kPfIAAABXIFdnFwAAcK5GLQPVMLLuuafY5xbIy/fc5fSciQcAALgyEeQBAHJxsah+s1rOLgMAAABlwKX1AAAAAACYCEEeAAAAAAATIcgDAAAAAGAiBHkAAAAAAEyEIA8AAAAAgIkQ5AEAAAAAMBGCPAAAAAAAJkKQBwAAAADARAjyAAAAAACYyBUR5KdPn66wsDC5u7srNjZWKSkpJfbt3LmzLBZLkalr1672Pv379y+yPCEhoTp2BQAAAACAKuXq7ALmzZunxMREvfPOO4qNjdWUKVMUHx+vnTt3KjAwsEj/BQsW6PTp0/b5P/74Q5GRkbr33nsd+iUkJGj27Nn2eTc3t6rbCQAAAAAAqonTz8i/9tpreuihhzRgwACFh4frnXfekaenp95///1i+wcEBCg4ONg+ff/99/L09CwS5N3c3Bz61apVqzp2BwAAAACAKuXUIH/69Glt3LhRcXFx9jYXFxfFxcVp7dq1ZRpj1qxZuv/+++Xl5eXQvnz5cgUGBqpZs2Z69NFH9ccff5Q4RkFBgXJzcx0mAAAAAACuRE4N8ocPH1ZhYaGCgoIc2oOCgpSZmXnJ9VNSUpSamqpBgwY5tCckJOiDDz7Q0qVL9fLLL2vFihW6/fbbVVhYWOw4SUlJ8vPzs08NGjSo+E4BAAAAAFCFnH6P/OWYNWuWIiIi1KZNG4f2+++/3/7fERERatGihRo1aqTly5frtttuKzLO6NGjlZiYaJ/Pzc0lzAMAAAAArkhOPSNfp04dWa1WZWVlObRnZWUpODi41HXz8/OVnJysgQMHXnI71113nerUqaNff/212OVubm7y9fV1mAAAAAAAuBI5NcjXrFlT0dHRWrp0qb3NZrNp6dKlateuXanrzp8/XwUFBXrggQcuuZ3ff/9df/zxh0JCQi67ZgAAAAAAnMnpT61PTEzUjBkzNHfuXKWlpenRRx9Vfn6+BgwYIEnq27evRo8eXWS9WbNmqUePHqpdu7ZDe15enkaOHKmffvpJe/fu1dKlS9W9e3c1btxY8fHx1bJPAAAAAABUFaffI9+zZ08dOnRIY8eOVWZmpqKiorRo0SL7A/D2798vFxfHvzfs3LlTq1at0pIlS4qMZ7VatXXrVs2dO1fHjh1TvXr11KVLF02YMIF3yQMAAAAATM9iGIbh7CKuNLm5ufLz81NOTg73ywMAAAAAqlx5cqjTz8hfic7/bYP3yQMAAAAAqsP5/FmWc+0E+WIcP35ckngFHQAAAACgWh0/flx+fn6l9uHS+mLYbDb997//lY+PjywWi7PLwRUiNzdXDRo00IEDB7jlAlcVjn1czTj+cbXi2MfVzFnHv2EYOn78uOrVq1fkOXEX44x8MVxcXHTNNdc4uwxcoXx9ffkHDVcljn1czTj+cbXi2MfVzBnH/6XOxJ/n9NfPAQAAAACAsiPIAwAAAABgIgR5oIzc3Nw0btw4ubm5ObsUoFpx7ONqxvGPqxXHPq5mZjj+edgdAAAAAAAmwhl5AAAAAABMhCAPAAAAAICJEOQBAAAAADARgjwAAAAAACZCkAcuMH36dIWFhcnd3V2xsbFKSUkpse/27dt19913KywsTBaLRVOmTKm+QoFKVp5jf8aMGerQoYNq1aqlWrVqKS4urtT+wJWuPMf/ggULFBMTI39/f3l5eSkqKkoffvhhNVYLVJ7yHPsXSk5OlsViUY8ePaq2QKAKlef4nzNnjiwWi8Pk7u5ejdUWRZAH/mfevHlKTEzUuHHjtGnTJkVGRio+Pl7Z2dnF9j9x4oSuu+46TZo0ScHBwdVcLVB5ynvsL1++XL169dKyZcu0du1aNWjQQF26dNHBgweruXLg8pX3+A8ICNBzzz2ntWvXauvWrRowYIAGDBigxYsXV3PlwOUp77F/3t69e/XUU0+pQ4cO1VQpUPkqcvz7+voqIyPDPu3bt68aKy6K188B/xMbG6vWrVtr2rRpkiSbzaYGDRpo6NCheuaZZ0pdNywsTCNGjNCIESOqoVKgcl3OsS9JhYWFqlWrlqZNm6a+fftWdblApbrc41+SWrVqpa5du2rChAlVWSpQqSpy7BcWFqpjx4568MEHtXLlSh07dkxffvllNVYNVI7yHv9z5szRiBEjdOzYsWqutGSckQcknT59Whs3blRcXJy9zcXFRXFxcVq7dq0TKwOqVmUc+ydOnNCZM2cUEBBQVWUCVeJyj3/DMLR06VLt3LlTHTt2rMpSgUpV0WN//PjxCgwM1MCBA6ujTKBKVPT4z8vLU2hoqBo0aKDu3btr+/bt1VFuiQjygKTDhw+rsLBQQUFBDu1BQUHKzMx0UlVA1auMY3/UqFGqV6+ewz+IgBlU9PjPycmRt7e3atasqa5du+rNN9/UX/7yl6ouF6g0FTn2V61apVmzZmnGjBnVUSJQZSpy/Ddr1kzvv/++vvrqK3300Uey2Wxq3769fv/99+oouViuTtsyAMD0Jk2apOTkZC1fvtzpD30BqouPj482b96svLw8LV26VImJibruuuvUuXNnZ5cGVInjx4+rT58+mjFjhurUqePscoBq165dO7Vr184+3759ezVv3lzvvvuu026rIsgDkurUqSOr1aqsrCyH9qysLB5khz+1yzn2J0+erEmTJuk///mPWrRoUZVlAlWiose/i4uLGjduLEmKiopSWlqakpKSCPIwjfIe++np6dq7d6+6detmb7PZbJIkV1dX7dy5U40aNaraooFKUhn/31+jRg21bNlSv/76a1WUWCZcWg9IqlmzpqKjo7V06VJ7m81m09KlSx3++gb82VT02P/nP/+pCRMmaNGiRYqJiamOUoFKV1m/+202mwoKCqqiRKBKlPfYv/7667Vt2zZt3rzZPt1555265ZZbtHnzZjVo0KA6ywcuS2X87i8sLNS2bdsUEhJSVWVeEmfkgf9JTExUv379FBMTozZt2mjKlCnKz8/XgAEDJEl9+/ZV/fr1lZSUJOncgzJ27Nhh/++DBw9q8+bN8vb2tp+pAcygvMf+yy+/rLFjx+qTTz5RWFiY/X4yb29veXt7O20/gIoo7/GflJSkmJgYNWrUSAUFBfr222/14Ycf6u2333bmbgDlVp5j393dXTfeeKPD+v7+/pJUpB0wg/L+7h8/frzatm2rxo0b69ixY3rllVe0b98+DRo0yGn7QJAH/qdnz546dOiQxo4dq8zMTEVFRWnRokX2B2Hs379fLi7/fxHLf//7X7Vs2dI+P3nyZE2ePFmdOnXS8uXLq7t8oMLKe+y//fbbOn36tO655x6HccaNG6cXXnihOksHLlt5j//8/Hw99thj+v333+Xh4aHrr79eH330kXr27OmsXQAqpLzHPvBnUt7j/+jRo3rooYeUmZmpWrVqKTo6WmvWrFF4eLizdoH3yAMAAAAAYCb8mQ0AAAAAABMhyAMAAAAAYCIEeQAAAAAATIQgDwAAAACAiRDkAQAAAAAwEYI8AAAAAAAmQpAHAAAAAMBECPIAAAAAAJgIQR4AACfp37+/evTo4ewyTCMsLExTpkyxz1ssFn355ZeXNWZljAEAQHUjyAMAUAUsFkup0wsvvKCpU6dqzpw51V7bnDlz5O/vXyljhYWF2ffJy8tLrVq10vz58ytl7EvJyMjQ7bffXqa+L7zwgqKioi5rDAAArhQEeQAAqkBGRoZ9mjJlinx9fR3annrqKfn5+VVaoHam8ePHKyMjQz///LNat26tnj17as2aNcX2PX36dKVtNzg4WG5ubk4fAwCA6kaQBwCgCgQHB9snPz8/WSwWhzZvb+8il9Z37txZQ4cO1YgRI1SrVi0FBQVpxowZys/P14ABA+Tj46PGjRvru+++c9hWamqqbr/9dnl7eysoKEh9+vTR4cOHi61r+fLlGjBggHJychyuDpCko0ePqm/fvqpVq5Y8PT11++23a/fu3ZfcVx8fHwUHB6tp06aaPn26PDw89O9//1vSuTP2EyZMUN++feXr66vBgwdLklatWqUOHTrIw8NDDRo00LBhw5Sfn28fMzs7W926dZOHh4caNmyojz/+uMh2L74s/vfff1evXr0UEBAgLy8vxcTEaN26dZozZ45efPFFbdmyxb7P56+EuHiMbdu26dZbb5WHh4dq166twYMHKy8vz778/Hc2efJkhYSEqHbt2nr88cd15syZS35OAABUFoI8AABXkLlz56pOnTpKSUnR0KFD9eijj+ree+9V+/bttWnTJnXp0kV9+vTRiRMnJEnHjh3TrbfeqpYtW2rDhg1atGiRsrKydN999xU7fvv27YtcIfDUU09JOhdSN2zYoK+//lpr166VYRj661//Wq6Q6urqqho1ajiceZ88ebIiIyP1888/a8yYMUpPT1dCQoLuvvtubd26VfPmzdOqVas0ZMgQ+zr9+/fXgQMHtGzZMv3rX//SW2+9pezs7BK3m5eXp06dOungwYP6+uuvtWXLFj399NOy2Wzq2bOnnnzySd1www32fe7Zs2eRMfLz8xUfH69atWpp/fr1mj9/vv7zn/841CVJy5YtU3p6upYtW6a5c+dqzpw5TrlFAgBwFTMAAECVmj17tuHn51ekvV+/fkb37t3t8506dTJuvvlm+/zZs2cNLy8vo0+fPva2jIwMQ5Kxdu1awzAMY8KECUaXLl0cxj1w4IAhydi5c2eZ69m1a5chyVi9erW97fDhw4aHh4fx2WeflbhvoaGhxuuvv24YhmEUFBQYEydONCQZ33zzjX15jx49HNYZOHCgMXjwYIe2lStXGi4uLsbJkyeNnTt3GpKMlJQU+/K0tDRDkn1bhmEYkowvvvjCMAzDePfddw0fHx/jjz/+KLbOcePGGZGRkUXaLxzjvffeM2rVqmXk5eXZly9cuNBwcXExMjMzDcM4952FhoYaZ8+etfe59957jZ49e5b4GQEAUNlcnfcnBAAAcLEWLVrY/9tqtap27dqKiIiwtwUFBUmS/ez0li1btGzZMnl7excZKz09XU2bNi3TdtPS0uTq6qrY2Fh7W+3atdWsWTOlpaWVuu6oUaP0/PPP69SpU/L29takSZPUtWtX+/KYmBiH/lu2bNHWrVsdLpc3DEM2m0179uzRrl275OrqqujoaPvy66+/vtTnCWzevFktW7ZUQEBAmfa3OGlpaYqMjJSXl5e97aabbpLNZtPOnTvtn/0NN9wgq9Vq7xMSEqJt27ZVeLsAAJQXQR4AgCtIjRo1HOYtFotDm8VikSTZbDZJ5y4p79atm15++eUiY4WEhFRhpf9v5MiR6t+/v/0e/fM1nndhMJbO1fzwww9r2LBhRca69tprtWvXrnLX4OHhUe51Kqq47+j89wEAQHUgyAMAYGKtWrXS559/rrCwMLm6lu2f9Zo1a6qwsNChrXnz5jp79qzWrVun9u3bS5L++OMP7dy5U+Hh4aWOV6dOHTVu3LhcNe/YsaPEda6//nqdPXtWGzduVOvWrSVJO3fu1LFjx0ocs0WLFpo5c6aOHDlS7Fn54vb5Ys2bN9ecOXOUn59v/+PD6tWr5eLiombNmpVx7wAAqHo87A4AABN7/PHHdeTIEfXq1Uvr169Xenq6Fi9erAEDBpQYXMPCwpSXl6elS5fq8OHDOnHihJo0aaLu3bvroYce0qpVq7RlyxY98MADql+/vrp3716pNY8aNUpr1qzRkCFDtHnzZu3evVtfffWV/aFyzZo1U0JCgh5++GGtW7dOGzdu1KBBg0o9696rVy8FBwerR48eWr16tX777Td9/vnnWrt2rX2f9+zZo82bN+vw4cMqKCgoMkbv3r3l7u6ufv36KTU1VcuWLdPQoUPVp08f+2X1AABcCQjyAACYWL169bR69WoVFhaqS5cuioiI0IgRI+Tv7y8Xl+L/mW/fvr0eeeQR9ezZU3Xr1tU///lPSdLs2bMVHR2tO+64Q+3atZNhGPr222+LXEp+uVq0aKEVK1Zo165d6tChg1q2bKmxY8eqXr169j6zZ89WvXr11KlTJ911110aPHiwAgMDSxyzZs2aWrJkiQIDA/XXv/5VERERmjRpkv1e9rvvvlsJCQm65ZZbVLduXX366adFxvD09NTixYt15MgRtW7dWvfcc49uu+02TZs2rVL3HwCAy2UxDMNwdhEAAAAAAKBsOCMPAAAAAICJEOQBAAAAADARgjwAAAAAACZCkAcAAAAAwEQI8gAAAAAAmAhBHgAAAAAAEyHIAwAAAABgIgR5AAAAAABMhCAPAAAAAICJEOQBAAAAADARgjwAAAAAACbyf51JBEZa3A6FAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1200x800 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(12, 8))\n",
    "plt.scatter(model_6c_time_to_prediction, model_6c_metrics['f1'], label='Pre-trained USE (10%)')\n",
    "plt.scatter(model_6b_time_to_prediction, model_6b_metrics['f1'], label='Pre-trained USE (10% / data leakage)')\n",
    "plt.scatter(model_6a_time_to_prediction, model_6a_metrics['f1'], label='Pre-trained USE (added Dense)')\n",
    "plt.scatter(model_6_time_to_prediction, model_6_metrics['f1'], label='Pre-trained USE')\n",
    "plt.scatter(model_5_time_to_prediction, model_5_metrics['f1'], label='1D-Convolutional Neural Network (CNN)')\n",
    "plt.scatter(model_4_time_to_prediction, model_4_metrics['f1'], label='Bidirectional-LSTM model (RNN)')\n",
    "plt.scatter(model_3_time_to_prediction, model_3_metrics['f1'], label='GRU model (RNN)')\n",
    "plt.scatter(model_2_time_to_prediction, model_2_metrics['f1'], label='LSTM model (RNN)')\n",
    "plt.scatter(model_1_time_to_prediction, model_1_metrics['f1'], label='Feed-forward neural network (dense model)')\n",
    "plt.scatter(model_0_time_to_prediction, baseline_metrics['f1'], label='Naive Bayes (baseline)')\n",
    "plt.legend()\n",
    "plt.title(\"F1-Score vs Time to Prediction\")\n",
    "plt.xlabel('Time to Prediction')\n",
    "plt.ylabel('F1-Score')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c434ad8a",
   "metadata": {},
   "source": [
    "![(Re) Introduction to Tensorflow Natural Language Processing](./assets/tf_nlp_desaster_tweets_06.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e866ee56",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "jupytext": {
   "formats": "ipynb,md"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}