DSB 2019 Maxwell Final Primary Objective.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [],
   "source": [
    "#imports\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import json\n",
    "import matplotlib\n",
    "%matplotlib inline\n",
    "\n",
    "import joblib"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problem Overview\n",
    "\n",
    "In this project, we were tasked with making a model to predict performance on assessements in the PBS Kids learning mobile application based on logged application data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Outline\n",
    "\n",
    "- Begin the process by taking a look at the *training* data basic features:\n",
    "    - Event code counts\n",
    "    - unique installations\n",
    "    - unique sessions\n",
    "- Add another layer by looking at grouped features:\n",
    "    - group by installation id: game time, event code counts,\n",
    "    - group by session within an installation\n",
    "- Compare train and test data as a reality check\n",
    "- Unpack inner data\n",
    "    - event data\n",
    "    - specs args\n",
    "- Join the train_labels to train data\n",
    "- Set up an initial naive baseline\n",
    "    - feature engineering\n",
    "    - K Nearest Neighbors\n",
    "- Improve the model by trying alternative methods\n",
    "    - CatBoost\n",
    "    - Light GBM Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Read in training data first\n",
    "train = pd.read_csv(\"../DSB2019/data-science-bowl-2019/train.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>event_id</th>\n",
       "      <th>game_session</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>event_data</th>\n",
       "      <th>installation_id</th>\n",
       "      <th>event_count</th>\n",
       "      <th>event_code</th>\n",
       "      <th>game_time</th>\n",
       "      <th>title</th>\n",
       "      <th>type</th>\n",
       "      <th>world</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>27253bdc</td>\n",
       "      <td>45bb1e1b6b50c07b</td>\n",
       "      <td>2019-09-06T17:53:46.937Z</td>\n",
       "      <td>{\"event_code\": 2000, \"event_count\": 1}</td>\n",
       "      <td>0001e90f</td>\n",
       "      <td>1</td>\n",
       "      <td>2000</td>\n",
       "      <td>0</td>\n",
       "      <td>Welcome to Lost Lagoon!</td>\n",
       "      <td>Clip</td>\n",
       "      <td>NONE</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>27253bdc</td>\n",
       "      <td>17eeb7f223665f53</td>\n",
       "      <td>2019-09-06T17:54:17.519Z</td>\n",
       "      <td>{\"event_code\": 2000, \"event_count\": 1}</td>\n",
       "      <td>0001e90f</td>\n",
       "      <td>1</td>\n",
       "      <td>2000</td>\n",
       "      <td>0</td>\n",
       "      <td>Magma Peak - Level 1</td>\n",
       "      <td>Clip</td>\n",
       "      <td>MAGMAPEAK</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>77261ab5</td>\n",
       "      <td>0848ef14a8dc6892</td>\n",
       "      <td>2019-09-06T17:54:56.302Z</td>\n",
       "      <td>{\"version\":\"1.0\",\"event_count\":1,\"game_time\":0...</td>\n",
       "      <td>0001e90f</td>\n",
       "      <td>1</td>\n",
       "      <td>2000</td>\n",
       "      <td>0</td>\n",
       "      <td>Sandcastle Builder (Activity)</td>\n",
       "      <td>Activity</td>\n",
       "      <td>MAGMAPEAK</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>b2dba42b</td>\n",
       "      <td>0848ef14a8dc6892</td>\n",
       "      <td>2019-09-06T17:54:56.387Z</td>\n",
       "      <td>{\"description\":\"Let's build a sandcastle! Firs...</td>\n",
       "      <td>0001e90f</td>\n",
       "      <td>2</td>\n",
       "      <td>3010</td>\n",
       "      <td>53</td>\n",
       "      <td>Sandcastle Builder (Activity)</td>\n",
       "      <td>Activity</td>\n",
       "      <td>MAGMAPEAK</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>1bb5fbdb</td>\n",
       "      <td>0848ef14a8dc6892</td>\n",
       "      <td>2019-09-06T17:55:03.253Z</td>\n",
       "      <td>{\"description\":\"Let's build a sandcastle! Firs...</td>\n",
       "      <td>0001e90f</td>\n",
       "      <td>3</td>\n",
       "      <td>3110</td>\n",
       "      <td>6972</td>\n",
       "      <td>Sandcastle Builder (Activity)</td>\n",
       "      <td>Activity</td>\n",
       "      <td>MAGMAPEAK</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   event_id      game_session                 timestamp  \\\n",
       "0  27253bdc  45bb1e1b6b50c07b  2019-09-06T17:53:46.937Z   \n",
       "1  27253bdc  17eeb7f223665f53  2019-09-06T17:54:17.519Z   \n",
       "2  77261ab5  0848ef14a8dc6892  2019-09-06T17:54:56.302Z   \n",
       "3  b2dba42b  0848ef14a8dc6892  2019-09-06T17:54:56.387Z   \n",
       "4  1bb5fbdb  0848ef14a8dc6892  2019-09-06T17:55:03.253Z   \n",
       "\n",
       "                                          event_data installation_id  \\\n",
       "0             {\"event_code\": 2000, \"event_count\": 1}        0001e90f   \n",
       "1             {\"event_code\": 2000, \"event_count\": 1}        0001e90f   \n",
       "2  {\"version\":\"1.0\",\"event_count\":1,\"game_time\":0...        0001e90f   \n",
       "3  {\"description\":\"Let's build a sandcastle! Firs...        0001e90f   \n",
       "4  {\"description\":\"Let's build a sandcastle! Firs...        0001e90f   \n",
       "\n",
       "   event_count  event_code  game_time                          title  \\\n",
       "0            1        2000          0        Welcome to Lost Lagoon!   \n",
       "1            1        2000          0           Magma Peak - Level 1   \n",
       "2            1        2000          0  Sandcastle Builder (Activity)   \n",
       "3            2        3010         53  Sandcastle Builder (Activity)   \n",
       "4            3        3110       6972  Sandcastle Builder (Activity)   \n",
       "\n",
       "       type      world  \n",
       "0      Clip       NONE  \n",
       "1      Clip  MAGMAPEAK  \n",
       "2  Activity  MAGMAPEAK  \n",
       "3  Activity  MAGMAPEAK  \n",
       "4  Activity  MAGMAPEAK  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Taking a brief look at some example observations we see that each represents information related to an \"event\". We get event data in an internal object as well as event meta data such as ID, related installation ID (normally one user), timestamp, and where they are in the application (title, world, type). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(11341042, 11)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to explore the features more easily let's sample a small piece of the total. Total shape has over 11 million observations.\n",
    "It's also worth noting that some of the installation ids in the training data don't contain assessments so we can eliminate those as they are not useful for training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "hasAssessmentIds = train[train[\"type\"] == \"Assessment\"].loc[:,'installation_id'].drop_duplicates()\n",
    "useful_train = pd.merge(train, hasAssessmentIds, on=\"installation_id\", how=\"inner\")\n",
    "del train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(8294138, 11)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "useful_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_sample = useful_train.sample(1000000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's list out the columns we're going to need to look at for each observation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['event_id', 'game_session', 'timestamp', 'event_data',\n",
       "       'installation_id', 'event_count', 'event_code', 'game_time', 'title',\n",
       "       'type', 'world'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_sample.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It seems like it could be useful to categorize different observations in the data. Event codes could represent some of these differences."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7ff86f900c10>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEICAYAAABfz4NwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de7hdVXnv8e+PBBQagUDCxYQ0EcP9QIQ0pEUqikDAlqAVIX0KQfEEEDzaamv0+Bw8Ujx4WvUUq3iipIRHBbGIRIHGGKnWI0gCibkQICHcNolJICCxXDThPX+MscnMylpzrr3nXnvvwO/zPPPZa413jjHHHHOu9e55WWspIjAzM9tloDtgZmaDgxOCmZkBTghmZpY5IZiZGeCEYGZmmROCmZkBMHSgO9BbI0aMiLFjxw50N8zMdir33nvvUxExsllsp00IY8eOZdGiRQPdDTOznYqkx1rFfMrIzMwAJwQzM8ucEMzMDHBCMDOzzAnBzMyANhKCpIMk3SlppaQVkj6Sy/eRNF/Sqvx3eC6XpKslrZa0VNKxhbam5/lXSZpeKD9O0rJc52pJ6sTKmplZa+0cIWwBPhYRhwOTgUslHQHMBBZExHhgQX4OcDowPk8zgGsgJRDgcuB4YBJweXcSyfPMKNSbUn/VzMysJyoTQkSsi4j78uPNwEpgFDAVmJNnmwOclR9PBa6P5G5gb0kHAqcB8yNiU0Q8A8wHpuTYnhFxV6QfZ7i+0JaZmfWTHn0wTdJY4C3AL4H9I2IdpKQhab882yjgiUK1rlxWVt7VpLzZ8meQjiQYM2YMAGNn3rbdPI9e9a6erJKZmWVtX1SWNAy4GfhoRDxXNmuTsuhF+Y6FEbMiYmJETBw5suknr83MrJfaSgiSdiUlg29FxPdy8fp8uof8d0Mu7wIOKlQfDaytKB/dpNzMzPpRO3cZCbgWWBkRXyyE5gLddwpNB24tlJ+f7zaaDPwmn1qaB5wqaXi+mHwqMC/HNkuanJd1fqEtMzPrJ+1cQzgBOA9YJmlJLvsUcBVwk6QLgceBs3PsduAMYDXwPPB+gIjYJOkKYGGe77MRsSk/vgS4DtgduCNPZmbWjyoTQkT8nObn+QFObjJ/AJe2aGs2MLtJ+SLgqKq+mJlZ5/iTymZmBjghmJlZ5oRgZmaAE4KZmWVOCGZmBjghmJlZ5oRgZmaAE4KZmWVOCGZmBjghmJlZ5oRgZmaAE4KZmWVOCGZmBjghmJlZ5oRgZmaAE4KZmWXt/ITmbEkbJC0vlH1H0pI8Pdr9S2qSxkp6oRD7WqHOcZKWSVot6er8c5lI2kfSfEmr8t/hnVhRMzMr184RwnXAlGJBRJwTERMiYgJwM/C9Qvjh7lhEXFwovwaYAYzPU3ebM4EFETEeWJCfm5lZP6tMCBHxM2BTs1j+L/99wA1lbUg6ENgzIu7KP7F5PXBWDk8F5uTHcwrlZmbWj+peQzgRWB8Rqwpl4yQtlvRTSSfmslFAV2GerlwGsH9ErAPIf/er2SczM+uFoTXrT2P7o4N1wJiIeFrSccD3JR0JqEnd6OnCJM0gnXZizJgxveiumZm10usjBElDgfcA3+kui4iXIuLp/Phe4GHgENIRwehC9dHA2vx4fT6l1H1qaUOrZUbErIiYGBETR44c2duum5lZE3VOGb0TeCAiXjkVJGmkpCH58ZtIF4/X5FNBmyVNztcdzgduzdXmAtPz4+mFcjMz60ft3HZ6A3AXcKikLkkX5tC57Hgx+U+BpZJ+BfwrcHFEdF+QvgT4BrCadORwRy6/CjhF0irglPzczMz6WeU1hIiY1qL8giZlN5NuQ202/yLgqCblTwMnV/XDzMw6y59UNjMzwAnBzMwyJwQzMwOcEMzMLHNCMDMzwAnBzMwyJwQzMwOcEMzMLHNCMDMzwAnBzMwyJwQzMwOcEMzMLHNCMDMzwAnBzMwyJwQzMwOcEMzMLHNCMDMzoL2f0JwtaYOk5YWyz0h6UtKSPJ1RiH1S0mpJD0o6rVA+JZetljSzUD5O0i8lrZL0HUm79eUKmplZe9o5QrgOmNKk/EsRMSFPtwNIOoL0W8tH5jpflTRE0hDgK8DpwBHAtDwvwOdzW+OBZ4ALGxdkZmadV5kQIuJnwKY225sK3BgRL0XEI8BqYFKeVkfEmoj4HXAjMFWSgHcA/5rrzwHO6uE6mJlZH6hzDeEySUvzKaXhuWwU8ERhnq5c1qp8X+DZiNjSUG5mZv2stwnhGuBgYAKwDvhCLleTeaMX5U1JmiFpkaRFGzdu7FmPzcysVK8SQkSsj4itEfEy8HXSKSFI/+EfVJh1NLC2pPwpYG9JQxvKWy13VkRMjIiJI0eO7E3XzcyshV4lBEkHFp6+G+i+A2kucK6k10kaB4wH7gEWAuPzHUW7kS48z42IAO4E3pvrTwdu7U2fzMysnqFVM0i6ATgJGCGpC7gcOEnSBNLpnUeBiwAiYoWkm4D7gS3ApRGxNbdzGTAPGALMjogVeRGfAG6U9PfAYuDaPls7MzNrW2VCiIhpTYpbvmlHxJXAlU3Kbwdub1K+hm2nnMzMbID4k8pmZgY4IZiZWeaEYGZmgBOCmZllTghmZgY4IZiZWeaEYGZmgBOCmZllTghmZgY4IZiZWVb51RU7u7Ezb9vu+aNXvWuAemJmNrj5CMHMzAAnBDMzy5wQzMwMcEIwM7PMCcHMzAAnBDMzyyoTgqTZkjZIWl4o+wdJD0haKukWSXvn8rGSXpC0JE9fK9Q5TtIySaslXS1JuXwfSfMlrcp/h3diRc3MrFw7RwjXAVMayuYDR0XE0cBDwCcLsYcjYkKeLi6UXwPMAMbnqbvNmcCCiBgPLMjPzcysn1UmhIj4GbCpoexHEbElP70bGF3WhqQDgT0j4q6ICOB64KwcngrMyY/nFMrNzKwf9cU1hA8AdxSej5O0WNJPJZ2Yy0YBXYV5unIZwP4RsQ4g/92vD/pkZmY9VOurKyT9d2AL8K1ctA4YExFPSzoO+L6kIwE1qR69WN4M0mknxowZ07tOm5lZU71OCJKmA38GnJxPAxERLwEv5cf3SnoYOIR0RFA8rTQaWJsfr5d0YESsy6eWNrRaZkTMAmYBTJw4sccJpRl/15GZWdKrU0aSpgCfAM6MiOcL5SMlDcmP30S6eLwmnwraLGlyvrvofODWXG0uMD0/nl4oNzOzflR5hCDpBuAkYISkLuBy0l1FrwPm57tH7853FP0p8FlJW4CtwMUR0X1B+hLSHUu7k645dF93uAq4SdKFwOPA2X2yZn3ERxBm9lpRmRAiYlqT4mtbzHszcHOL2CLgqCblTwMnV/XDzMw6y59UNjMzwAnBzMwyJwQzMwOcEMzMLHNCMDMzwAnBzMwyJwQzMwOcEMzMLHNCMDMzwAnBzMyyWl9/bf6uIzN79fARgpmZAU4IZmaWOSGYmRnghGBmZpkTgpmZAU4IZmaWtZUQJM2WtEHS8kLZPpLmS1qV/w7P5ZJ0taTVkpZKOrZQZ3qef5Wk6YXy4yQty3Wuzr+7bGZm/ajdI4TrgCkNZTOBBRExHliQnwOcDozP0wzgGkgJhPR7zMcDk4DLu5NInmdGoV7jsszMrMPaSggR8TNgU0PxVGBOfjwHOKtQfn0kdwN7SzoQOA2YHxGbIuIZYD4wJcf2jIi7IiKA6wttmZlZP6lzDWH/iFgHkP/ul8tHAU8U5uvKZWXlXU3KzcysH3XionKz8//Ri/IdG5ZmSFokadHGjRtrdNHMzBrVSQjr8+ke8t8NubwLOKgw32hgbUX56CblO4iIWRExMSImjhw5skbXzcysUZ2EMBfovlNoOnBrofz8fLfRZOA3+ZTSPOBUScPzxeRTgXk5tlnS5Hx30fmFtszMrJ+09W2nkm4ATgJGSOoi3S10FXCTpAuBx4Gz8+y3A2cAq4HngfcDRMQmSVcAC/N8n42I7gvVl5DuZNoduCNPZmbWj9pKCBExrUXo5CbzBnBpi3ZmA7OblC8CjmqnL2Zm1hn+pLKZmQFOCGZmljkhmJkZ4IRgZmaZE4KZmQFOCGZmljkhmJkZ4IRgZmaZE4KZmQFOCGZmljkhmJkZ4IRgZmaZE4KZmQFOCGZmljkhmJkZ4IRgZmaZE4KZmQE1EoKkQyUtKUzPSfqopM9IerJQfkahziclrZb0oKTTCuVTctlqSTPrrpSZmfVcWz+h2UxEPAhMAJA0BHgSuIX0G8pfioh/LM4v6QjgXOBI4I3AjyUdksNfAU4BuoCFkuZGxP297ZuZmfVcrxNCg5OBhyPiMUmt5pkK3BgRLwGPSFoNTMqx1RGxBkDSjXleJwQzs37UV9cQzgVuKDy/TNJSSbMlDc9lo4AnCvN05bJW5WZm1o9qJwRJuwFnAt/NRdcAB5NOJ60DvtA9a5PqUVLebFkzJC2StGjjxo21+m1mZtvriyOE04H7ImI9QESsj4itEfEy8HW2nRbqAg4q1BsNrC0p30FEzIqIiRExceTIkX3QdTMz69YXCWEahdNFkg4sxN4NLM+P5wLnSnqdpHHAeOAeYCEwXtK4fLRxbp7XzMz6Ua2LypL2IN0ddFGh+H9LmkA67fNodywiVki6iXSxeAtwaURsze1cBswDhgCzI2JFnX6ZmVnP1UoIEfE8sG9D2Xkl818JXNmk/Hbg9jp9MTOzevxJZTMzA5wQzMwsc0IwMzPACcHMzDInBDMzA5wQzMwsc0IwMzPACcHMzDInBDMzA5wQzMwsc0IwMzPACcHMzDInBDMzA5wQzMwsc0IwMzPACcHMzDInBDMzA/ogIUh6VNIySUskLcpl+0iaL2lV/js8l0vS1ZJWS1oq6dhCO9Pz/KskTa/bLzMz65m+OkJ4e0RMiIiJ+flMYEFEjAcW5OcApwPj8zQDuAZSAgEuB44HJgGXdycRMzPrH506ZTQVmJMfzwHOKpRfH8ndwN6SDgROA+ZHxKaIeAaYD0zpUN/MzKyJvkgIAfxI0r2SZuSy/SNiHUD+u18uHwU8UajblctalZuZWT8Z2gdtnBARayXtB8yX9EDJvGpSFiXl21dOCWcGwJgxY3rTVzMza6H2EUJErM1/NwC3kK4BrM+ngsh/N+TZu4CDCtVHA2tLyhuXNSsiJkbExJEjR9btupmZFdRKCJL+QNIbuh8DpwLLgblA951C04Fb8+O5wPn5bqPJwG/yKaV5wKmShueLyafmMjMz6yd1TxntD9wiqbutb0fEv0laCNwk6ULgceDsPP/twBnAauB54P0AEbFJ0hXAwjzfZyNiU82+mZlZD9RKCBGxBjimSfnTwMlNygO4tEVbs4HZdfpjZma9508qm5kZ4IRgZmaZE4KZmQFOCGZmljkhmJkZ4IRgZmaZE4KZmQFOCGZmljkhmJkZ4IRgZmaZE4KZmQFOCGZmljkhmJkZ0De/mGYlxs68bbvnj171rgHqiZlZOR8hmJkZ4IRgZmaZE4KZmQE1EoKkgyTdKWmlpBWSPpLLPyPpSUlL8nRGoc4nJa2W9KCk0wrlU3LZakkz662SmZn1Rp2LyluAj0XEfZLeANwraX6OfSki/rE4s6QjgHOBI4E3Aj+WdEgOfwU4BegCFkqaGxH31+ibmZn1UK8TQkSsA9blx5slrQRGlVSZCtwYES8Bj0haDUzKsdX595mRdGOe9zWREHwXkpkNFn1yDUHSWOAtwC9z0WWSlkqaLWl4LhsFPFGo1pXLWpWbmVk/qv05BEnDgJuBj0bEc5KuAa4AIv/9AvABQE2qB82TUrRY1gxgBsCYMWPqdn2nUHUE4SMMM+srtRKCpF1JyeBbEfE9gIhYX4h/HfhhftoFHFSoPhpYmx+3Kt9ORMwCZgFMnDixadKw7dVNKMW4k5HZq1uvE4IkAdcCKyPii4XyA/P1BYB3A8vz47nAtyV9kXRReTxwD+nIYbykccCTpAvPf9nbftng4YRhtnOpc4RwAnAesEzSklz2KWCapAmk0z6PAhcBRMQKSTeRLhZvAS6NiK0Aki4D5gFDgNkRsaJGv8zMrBfq3GX0c5pfF7i9pM6VwJVNym8vq2dmZp3nTyqbmRnghGBmZpkTgpmZAU4IZmaWOSGYmRngX0yzAeTPKZgNLj5CMDMzwAnBzMwyJwQzMwOcEMzMLHNCMDMzwHcZ2SDmu5DM+pcTgu20nDDM+pZPGZmZGeAjBHsV8xGEWc/4CMHMzAAfIdhrWN3fmzZ7tRk0CUHSFOCfSD+j+Y2IuGqAu2RWygnDXm0GRUKQNAT4CnAK0AUslDQ3Iu4f2J6Z9V7dI5CdPW47n0GREIBJwOqIWAMg6UZgKuCEYLaTGuiENNDxnZEiYqD7gKT3AlMi4oP5+XnA8RFxWcN8M4AZ+emhwIOF8AjgqZLFON77+GDum+OOO96z+B9GxMimc0bEgE/A2aTrBt3PzwO+3MM2Fjnemfhg7pvjjjteL16cBsttp13AQYXno4G1A9QXM7PXpMGSEBYC4yWNk7QbcC4wd4D7ZGb2mjIoLipHxBZJlwHzSLedzo6IFT1sZpbjHYsP5r457rjj9eKvGBQXlc3MbOANllNGZmY2wJwQzMwMcEIwM7PMCcHMzIBBcpdRT0naC5gCjAKC9JmFeRHxbI6L9HUYxfg9ka+gt1H/MNJXZxTjcyNiZX/Ub1jXYcAhwJoW8XHAW4D7I+KBNvvX9vIblnVKRMyvWr+S+u+PiH9po/26229PYGREPNzQ/tERsbTG+h/WPcaFsj4f/7r7V0P/3prHcnlE/CiXHQAQEb+WNBI4EXiw8c6+VvteWf/a2Pcr+163jXbHp9m2azJ+283T032ncfxrvHa6Xxu13vuq7HR3GUk6H7gc+BHwZC4eTfpivP8J/Br4KrCqIf5m4EPAARX1DwSmATeSPjDXHT83l63tcP3JEfGhvK5vBb4NPJz7fxEwIyLOyvGpwP8B/h34E+B/AS9XtE9ZPCKupwVJj5O+hLDl+kXJt9RKejwixlS0/0Hqbb8XSWOyAdgVuCAiFub278uxOut/XyfHv4141f71noiYlPv3X4FLgVuAU4EfAM8AMwEBnwcuAFYAJwBPRsQ7ct0d9r2IuL3i9beI9JUyrfoWZX2PiKskfaJi/UrbqIjvGhFH5vXbYdtFxHWSvl+yfX8KnNxq20TE9ZLuKRn/5/L269VrB/h0ydhXvvd1/0NQqt2PNA+WifT9RXs3KR8OPASsBMY2iY/Lsar6D5F2nMb4bnmgO13/vkLZncCx+fGbSC+4xYX4L4Bx+fEI4FdttF8Vn9ti+gHwn22s39IW0zLgpTbar7v9lgAH5rJJwAOkN0mAxW3Uv7rF9GXSC7rT4193/yr2byHpSAngD/I2WAbsAewL/BY4oND+82X7Xhuvv99V9K207/lx1frVib9Ytu2695GSeV4s2zZN6jeO/0sVfa96bdR672ssbzbtjKeMRPovoNHLOTaUbdm36EnSf4wvV9TfCrwReKwhfmBhnk7WL9ozIu4DiIg1+WvCi3WHRsQjOf6UpHbbL4ufCPwV6c2iqPtQ9OWK9dsfOI30n2hj/V+00X7d7TckItYBRMQ9kt4O/FDS6FyvanzeD3yM9OJtNI3Oj39VvGr/2kXScNL1QUXExty//5S0JT2M54HnJT0cEb/O8WfS2YZXNNv3qFg/KvoWFfHuduq0URYvarbtoHz7Nsa7Fbdd2fhX9b3qtVH3va/SzpgQrgTuk/Qj4IlcNoZ02HQFaXAX5q/Q7o4fRDosu5Z0yqas/q+BBZJWNcTfDFxGesPrZP2vSlpK2sBjJQ3PL9ZdSBv1cEnP5fjrJB0Q6VzwbqRPeVeNT1TEzyH9p/jTxoGX9CDwuYr1OxsYFhFLmtT/d7b9J9qq/TvYcfuNyf1qZ/vNkHRw5OsHEbFO0knA94EjSd+WW1b/fNL53l806d9ngGM6PP5V8ar962vAvbl/UejfsFy2VdKuEfF74JXva5b0euD1FfseFev35Yq+0Ub8o33QRqv4rhXbDsq378aKbQOwV8n4P1nR949Q/tq4tmL5zd77iq+dSjvdNQSAnIFPI104ESkrzouIZ3L8CODMhvgrP7jTRv1d2HZhpju+MCK2drq+pD9sWN11EfE7SSOAP42I77UYk72BwyPirjb6VxpvY/xL168uSYez7cJbj7afpGNIL6pVDW3uCrwvIr5VUX8f0qmF53vY5z4b/7r7V4v+7UH6Z2QraZ/6fUN8FPA24P8VipvuexXjV7XvV/a9bhs9HZ/itisZv72Bw0mnIHv82imM/2M96VuTdmq991W2vzMmhIEiaVhENB7O9UW7+0TEpr5ut2KZ+1O4EyEi1vfn8ntC0rHdpy8KZYOm/1Xbr434mRExt/B8aERsyY+HAYeR7vRp2kZP95+etN9s7HN5n45/O6+tZvNIejNwDOkceembXrF+fmPdEhGb2+jbDuPbqr6kvaP1HUctY4V5erTt2+j7iIgo+62E7bVzoWEwTcAm4Bukq/1qEj+MdNrhNuBg4DrgWeAe0n8BZW0vq4g/Tjr9dCPwH8CnKFwkIp2WOBq4m3TINgsYXojfQ7qbYyXpzo7jgfnAmjz/H1f1v43lV8Un5P6tzMv+Mem/nruBY/tg/ariVe0f22TqIt36d2zd/ldt/zbGv2r7VcXf02T6deHxBcDTpIuEp+e6C3L9acCnC/09Is/3CPBoXl7V+Je1/8mGcT+uOPa5jbLxP6ds2VWvrTZe+4+TLnaPyM/Py+vxjbztPlxR/0ngeuA3pCOlx/P0me79pGL7/Xkb9bfkMbmQhgvAZbEcr9r2HyjMOyrHniFdmzsk13kE+HneZitId4l1ASe39f460G/wPZ1IV9ovIx3aPgn8E+lWze74z/KGm0Y6PDuXdOj053kAm70g3wP8Bekc4d+0mD5GSkbzgYvzC+PLeWPsm5e9OG+MKcDewMfzRjm4EL8H+C+kN4engLfm2LF5nar6X7X8qvgS0q/RNY7rZNJdMnXXrype1f7LuezOwvRC/vuTPuh/1favGv+q7VcV3wL8EJgN/EueNue/s0lvbCNId4Y8Vxi7/Ul3axXvQrsNOD0/npTXtWr8y9qPsrHP85WN/28rll362srzVb3+lheWubCwbffI41NW//fASXn+9wBfIt398/fArO7EVbL9nm2j/jLgz4Bvkd7cbyXtQ7uXxQp12932N5FuQ98FeDdp31xC+qflj3P7k/O8hxfrvtoSQnFQxgB/B9xHyqafY/vbvlY31s07xXVsezEWp82kW8uuIN3v2zg9CyxpaPOvyDt+br8x/nbSLWWTc7zYv5VN+lfV/54uvzG+qmRsV/fB+vU03tj+e0n3e59RmOeRwuO6/a/a/lXj35Pt1yz+R6QX7yVsO2VbXL8lhcdrG+o3viksbogv7sn4N2n/sbKxb2P8X6pYdulrK9epev0tBkblee8EXp8fD8nbuaz+1ob+3Vt4/EDjmDbZfs+3Ub+4fXYH3gd8j/QGvakk9u0ebvvG7by4If5EQ3y7+Vtuw3ZmGkxT44ugUH5o3uhLC2UfaphnOekOgKNatPEE6T+k40riK7p3wkL5O0lvRutI/6Xu1RA/Or8wnibf75zLz2rSv6r+Vy2/Kn416T/Lc0gftvmT/Pg24J/7Yv0q4qXt5+fDSP99fZeU9NcU5q3b/6rtXzX+VduvNJ7/7kK6o+RO0n/2xfWbS/qA2z+Tjoi+QDqNcTnp90KeZdu96RuBPRqXXzH+Ve23HPs2xv+pimWXvrby36rX30l5G382r8MvgP9BOjL8eEX9F0n/ILwR+DBwcy4X2z5HULb9fttG/VbvT3sBj5XEprexbTaw7TMxT7L96dDluc5FwN+SEshfk04tTQd+3mzZO/SlnZkG0wR8sSJ+Eem2x8byN5M+dXgiMKZF3YmkxDKiRXz/PMhvaxJ7S94p/5LCKaxCfAzwddIdAHs0iR9MOtqp6n/V8kvj+fHppNsTf0A6ffE18n+FfbB+VfHK/jWU3QlsbCiv0/+q7V81/lXbrzTeUDaKdOhfTAh7ks7lzyS9Ob83r+NXSbcVvq1hekNh37y0jfEvbb9q7MvGv41ll7628t9DyR/mKplnL9IR1pdIb46fAA5ro/6xebyXA99k2wcY9wX+Ij8u236fa6P+x5stuyrW5raf3jANz/UOyH07CPi/eXscQHotLCcl69Lrp92T7zKyQU3p00BviIjnBrovrzUe+9egdrLGYJpIH6a7CPg30mHRr0h3hVxM+vBMrXjFsme10f5ewFWkOy+eztPKXLZ3G/GOrl9h+St7s/yq8emD8ev09qkbH5LjVwAnNCz/023Ee7J/bOrF/lOn/X2rxr5i/9mnbN3b2Xf6eP0a493rd0cb69eJ+mNqbvuhZcsviV9ExWtnpz1CkHQD6TzqHLb/gqjppB3y5ZrxS1otmjTA/1FRf2/Subw5kb8WQOnbJaeTzmW/XBF/qsPr16p/F5Bu5a1aftX4HF1z/Dq9feqO32bSHS33kG57/GlE/A3Q/eV591XEN9K7/eMC0vap2n96G7+A9CnhW1qte0ScI2leSf0Pky5Kt1r3dzbdcnnbRcTokvb7Ynz+usb69UX9vwX+oca61XpviIhzWoz/tg2xEyaEByPi0Baxh0jf1VInfjDpboviF7tEfj6KdNdFb9t/kDRDb+N9sX51+tfO+AypiNcZv05vn3biL0bE0fn5UNL53RGk21TvJn2XUll8jw7uH3Xjv4uI3VrEHoqIQypefy9GxOvz42brfjQl2y4idqtof6DXr5P1O/7eEBGHNIsV7Yw/kPOMpLPzx9OB9FF1SeeQPqRRN76GdK/xuML0pogYB6xvo/5jkv4uf5KzO76/0tf6PtFGvNPrV3f5VeNTd/w6vX3qxl95QUfEloiYQToy+QnpQmBVvO7+0cn4CxXrTkX9V75+ocW6V227qvYHev3q1t88wO8N1do5rzSYJmAs8B3SofdDpFvaNuSycT2Ib2Db1wkX45cCx7RY9ofbaH846XvmH8gbYRPpPODnSYd1VfFOr1+7y+/t+NQdv6p4p9uvin8TmNJk2R8kfcahKl53/+hk/Jiydc/rUVb/pop1L912bbQ/0OtXt/64mus2tmz5VfG23l8H+g2+ZmSXsfYAAALDSURBVHLYl3RI+s2exEkfSZ+U428l3b9c/DDOJOCP8uMjSJ90PKMH7R9GOic4rKF8Sjvxfli/qv7VGp+649duvDDf9X25f/TV8lvF6+4fnY73cN1PJH0K+NS+GJt22u/0+nWyfl/0vS/33cZpZ7yGMLdJ8TtIh6WtFOP3ku6jHkq6L30S6ULYO0kf/hjaED+e9ItJ3fHjK9r/Mek/oZWkr0/4SETcmvt+H+lTsmXxLnbUl+u3uWL5t9Ycn7rjV7V+jUT6RGxv63d6+Y3xuvtHJ+PPkr66o+m6R8SZ2v4XwT6Y2/o+6RfBhpM+ANirsWmj/R8Az3d4/f5bB+s/kfs/IO8NEXFmyXxJT7LHYJhId3F8k/SJxbflv+vY9kGdqvgy0oXPPUjfF7Jnbnd3tv2yV1m8nfaH5TpjSb9y9pH8fHEb8f5Yv6r+1RmfuuNXFV9cs36nl18Vr7t/dDL+fFnfu9sovBYbfxHshTpj00b73b/41sn162T9F2puu1r7blvvrwP9Bt+LhLAL6fav+cCEXLamB/HFzR7n50vaiFe1f39DnWGk+7q/mOtXxTu9flXLrzs+dcfv1R6vu390Ot6y7/n5r0hHAvuSf1azuL3rjE2b7Xd6/TpZ/8Wafa+177UzDfgbfG8n0v213yV978cOX53bKg78kvzRdGCXQvlepAxbGm+j/Z90b4xC2VDS1+ZurYr3w/pV9a/W+NQdv1d7vO7+0el41bqRvmZ7Delrltew7TeZh5G/QK3O2FW13w/r18n6UbfvfbFvlk0D/sZedyL9DODn2o0Dr2sx3wjS196Wxttof3T3Ttxk3hOq4v2wflX9qzU+dcfv1R6vu390Ot6TdWuYdw8a7mSpO3bN2u/0+nW4/lkD+d7QzrTTXVQ2M7PO2Bk/mGZmZh3ghGBmZoATgpmZZU4IZmYGOCGYmVn2/wG16kYbo+a0OAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "train_sample[\"event_code\"].value_counts().plot.bar()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>event_count</th>\n",
       "      <th>event_code</th>\n",
       "      <th>game_time</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>installation_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0006a69f</td>\n",
       "      <td>2418</td>\n",
       "      <td>155838</td>\n",
       "      <td>2994032</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>0006c192</td>\n",
       "      <td>2466</td>\n",
       "      <td>84224</td>\n",
       "      <td>2368500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>00129856</td>\n",
       "      <td>1363</td>\n",
       "      <td>49469</td>\n",
       "      <td>1805639</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>001d0ed0</td>\n",
       "      <td>734</td>\n",
       "      <td>35594</td>\n",
       "      <td>815923</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>00225f67</td>\n",
       "      <td>597</td>\n",
       "      <td>28540</td>\n",
       "      <td>571335</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>ff9715db</td>\n",
       "      <td>1872</td>\n",
       "      <td>96986</td>\n",
       "      <td>2024098</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>ffc90c32</td>\n",
       "      <td>557</td>\n",
       "      <td>55851</td>\n",
       "      <td>632449</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>ffd2871d</td>\n",
       "      <td>3024</td>\n",
       "      <td>114203</td>\n",
       "      <td>5537152</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>ffe3b25d</td>\n",
       "      <td>119</td>\n",
       "      <td>6090</td>\n",
       "      <td>104553</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>ffeb0b1b</td>\n",
       "      <td>1849</td>\n",
       "      <td>66958</td>\n",
       "      <td>2502118</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4070 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 event_count  event_code  game_time\n",
       "installation_id                                    \n",
       "0006a69f                2418      155838    2994032\n",
       "0006c192                2466       84224    2368500\n",
       "00129856                1363       49469    1805639\n",
       "001d0ed0                 734       35594     815923\n",
       "00225f67                 597       28540     571335\n",
       "...                      ...         ...        ...\n",
       "ff9715db                1872       96986    2024098\n",
       "ffc90c32                 557       55851     632449\n",
       "ffd2871d                3024      114203    5537152\n",
       "ffe3b25d                 119        6090     104553\n",
       "ffeb0b1b                1849       66958    2502118\n",
       "\n",
       "[4070 rows x 3 columns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_sample.groupby(\"installation_id\").sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tells us there are about 4000 installation ids in the sample and gives us information about total game time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>event_count</th>\n",
       "      <th>event_code</th>\n",
       "      <th>game_time</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>game_session</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>000050630c4b081b</td>\n",
       "      <td>3</td>\n",
       "      <td>2060</td>\n",
       "      <td>4804</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>00020f3bd042b3f0</td>\n",
       "      <td>2</td>\n",
       "      <td>4010</td>\n",
       "      <td>2230</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>00030865234e6b3d</td>\n",
       "      <td>27</td>\n",
       "      <td>4020</td>\n",
       "      <td>36526</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>000319989d08f8aa</td>\n",
       "      <td>494</td>\n",
       "      <td>9160</td>\n",
       "      <td>745700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>0007729b666af371</td>\n",
       "      <td>167</td>\n",
       "      <td>4070</td>\n",
       "      <td>194174</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>fffc036248965403</td>\n",
       "      <td>124</td>\n",
       "      <td>4070</td>\n",
       "      <td>93425</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>fffc152654849cbe</td>\n",
       "      <td>120</td>\n",
       "      <td>7210</td>\n",
       "      <td>125564</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>fffc54cfd9f4d7b2</td>\n",
       "      <td>37</td>\n",
       "      <td>3110</td>\n",
       "      <td>49684</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>fffcdc3a0ff75a77</td>\n",
       "      <td>1244</td>\n",
       "      <td>16142</td>\n",
       "      <td>1673053</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>fffda980b07029a3</td>\n",
       "      <td>23</td>\n",
       "      <td>4022</td>\n",
       "      <td>23902</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>49348 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                  event_count  event_code  game_time\n",
       "game_session                                        \n",
       "000050630c4b081b            3        2060       4804\n",
       "00020f3bd042b3f0            2        4010       2230\n",
       "00030865234e6b3d           27        4020      36526\n",
       "000319989d08f8aa          494        9160     745700\n",
       "0007729b666af371          167        4070     194174\n",
       "...                       ...         ...        ...\n",
       "fffc036248965403          124        4070      93425\n",
       "fffc152654849cbe          120        7210     125564\n",
       "fffc54cfd9f4d7b2           37        3110      49684\n",
       "fffcdc3a0ff75a77         1244       16142    1673053\n",
       "fffda980b07029a3           23        4022      23902\n",
       "\n",
       "[49348 rows x 3 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_sample.groupby(\"game_session\").sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Not sure game session ids are telling me what I thought they were here. They don't seem to overlap much, some sessions only contining 1 obseravtion.\n",
    "\n",
    "Let's take a deeper look at a set of events for a single ID."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>event_id</th>\n",
       "      <th>game_session</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>event_data</th>\n",
       "      <th>installation_id</th>\n",
       "      <th>event_count</th>\n",
       "      <th>event_code</th>\n",
       "      <th>game_time</th>\n",
       "      <th>title</th>\n",
       "      <th>type</th>\n",
       "      <th>world</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>819</td>\n",
       "      <td>28f975ea</td>\n",
       "      <td>80d34a30c2998653</td>\n",
       "      <td>2019-08-06T05:25:31.088Z</td>\n",
       "      <td>{\"distance\":4,\"dinosaur\":\"don\",\"target_distanc...</td>\n",
       "      <td>0006a69f</td>\n",
       "      <td>125</td>\n",
       "      <td>4020</td>\n",
       "      <td>155126</td>\n",
       "      <td>Air Show</td>\n",
       "      <td>Game</td>\n",
       "      <td>TREETOPCITY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>471</td>\n",
       "      <td>02a42007</td>\n",
       "      <td>05921a6eb858eeba</td>\n",
       "      <td>2019-08-06T05:16:12.546Z</td>\n",
       "      <td>{\"rocket\":3,\"coordinates\":{\"x\":330,\"y\":607,\"st...</td>\n",
       "      <td>0006a69f</td>\n",
       "      <td>78</td>\n",
       "      <td>4030</td>\n",
       "      <td>81224</td>\n",
       "      <td>Fireworks (Activity)</td>\n",
       "      <td>Activity</td>\n",
       "      <td>TREETOPCITY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3585</td>\n",
       "      <td>e9c52111</td>\n",
       "      <td>667e94bc234d1dbd</td>\n",
       "      <td>2019-08-29T16:26:04.802Z</td>\n",
       "      <td>{\"jar\":[{\"color\":\"blue\",\"amount\":1},{\"color\":\"...</td>\n",
       "      <td>0006a69f</td>\n",
       "      <td>31</td>\n",
       "      <td>2030</td>\n",
       "      <td>46808</td>\n",
       "      <td>Bottle Filler (Activity)</td>\n",
       "      <td>Activity</td>\n",
       "      <td>MAGMAPEAK</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1515</td>\n",
       "      <td>6c517a88</td>\n",
       "      <td>cfc0fcbfcace6a52</td>\n",
       "      <td>2019-08-06T05:46:33.542Z</td>\n",
       "      <td>{\"coordinates\":{\"x\":311,\"y\":55,\"stage_width\":1...</td>\n",
       "      <td>0006a69f</td>\n",
       "      <td>74</td>\n",
       "      <td>4070</td>\n",
       "      <td>93480</td>\n",
       "      <td>Dino Drink</td>\n",
       "      <td>Game</td>\n",
       "      <td>MAGMAPEAK</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1764</td>\n",
       "      <td>363d3849</td>\n",
       "      <td>faed1805baea37b0</td>\n",
       "      <td>2019-08-06T17:46:05.315Z</td>\n",
       "      <td>{\"media_type\":\"animation\",\"total_duration\":410...</td>\n",
       "      <td>0006a69f</td>\n",
       "      <td>4</td>\n",
       "      <td>3010</td>\n",
       "      <td>4293</td>\n",
       "      <td>All Star Sorting</td>\n",
       "      <td>Game</td>\n",
       "      <td>TREETOPCITY</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      event_id      game_session                 timestamp  \\\n",
       "819   28f975ea  80d34a30c2998653  2019-08-06T05:25:31.088Z   \n",
       "471   02a42007  05921a6eb858eeba  2019-08-06T05:16:12.546Z   \n",
       "3585  e9c52111  667e94bc234d1dbd  2019-08-29T16:26:04.802Z   \n",
       "1515  6c517a88  cfc0fcbfcace6a52  2019-08-06T05:46:33.542Z   \n",
       "1764  363d3849  faed1805baea37b0  2019-08-06T17:46:05.315Z   \n",
       "\n",
       "                                             event_data installation_id  \\\n",
       "819   {\"distance\":4,\"dinosaur\":\"don\",\"target_distanc...        0006a69f   \n",
       "471   {\"rocket\":3,\"coordinates\":{\"x\":330,\"y\":607,\"st...        0006a69f   \n",
       "3585  {\"jar\":[{\"color\":\"blue\",\"amount\":1},{\"color\":\"...        0006a69f   \n",
       "1515  {\"coordinates\":{\"x\":311,\"y\":55,\"stage_width\":1...        0006a69f   \n",
       "1764  {\"media_type\":\"animation\",\"total_duration\":410...        0006a69f   \n",
       "\n",
       "      event_count  event_code  game_time                     title      type  \\\n",
       "819           125        4020     155126                  Air Show      Game   \n",
       "471            78        4030      81224      Fireworks (Activity)  Activity   \n",
       "3585           31        2030      46808  Bottle Filler (Activity)  Activity   \n",
       "1515           74        4070      93480                Dino Drink      Game   \n",
       "1764            4        3010       4293          All Star Sorting      Game   \n",
       "\n",
       "            world  \n",
       "819   TREETOPCITY  \n",
       "471   TREETOPCITY  \n",
       "3585    MAGMAPEAK  \n",
       "1515    MAGMAPEAK  \n",
       "1764  TREETOPCITY  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "single_id = train_sample.query('installation_id==\"0006a69f\"')\n",
    "single_id.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7ff86f75a390>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAEICAYAAAB25L6yAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAUfUlEQVR4nO3de5AlZXnH8e/DLqiAIpcJXnAcRIV4RTLiBY1cvCCbYMoYlZRGrZjRJCpeUma1kkLLMkUqiaXGpKwN4qU0EKNgDBsFSgWjBhZ2WeSyIIiLKCirSIkxiuCTP7pnOXs4s9M9+54zL7vfT9UpznT3ec7TLz2/6e3TfToyE0lSvXZb7gYkSdtnUEtS5QxqSaqcQS1JlTOoJalyBrUkVW7lOIoecMABOTMzM47SkrRTWr9+/Y8yc2rUvLEE9czMDJdeeuk4SkvSTikiblxonoc+JKlyBrUkVc6glqTKGdSSVDmDWpIq1ymoI+ItEXFVRFwZEWdExP3H3ZgkqbFoUEfEw4E3AbOZ+QRgBfDycTcmSWp0PfSxEnhARKwE9gRuHl9LkqRBi17wkpnfj4i/B74L/B9wXmaeN7xcRMwBcwDT09PbzJtZvbZTM5tPXdVpOUnalXQ59LEv8CLgYOBhwF4R8Yrh5TJzTWbOZubs1NTIqyAlSUvQ5dDHc4HvZOaWzPwVcBbwzPG2JUma1yWovws8PSL2jIgAjgM2jbctSdK8RYM6My8GPgNsAK5oX7NmzH1Jklqdvj0vM08BThlzL5KkEbwyUZIqZ1BLUuUMakmqnEEtSZUzqCWpcga1JFXOoJakyhnUklQ5g1qSKmdQS1LlDGpJqpxBLUmVM6glqXIGtSRVzqCWpMoZ1JJUuS43tz00IjYOPH4aEW+eRHOSpA53eMnMa4HDASJiBfB94Owx9yVJavU99HEc8O3MvHEczUiS7q1vUL8cOGMcjUiSRut0c1uAiNgDOBF4xwLz54A5gOnp6SLNjTKzeu2iy2w+ddXY3l+SJq3PHvULgQ2Z+cNRMzNzTWbOZubs1NRUme4kSb2C+iQ87CFJE9cpqCNiT+B5wFnjbUeSNKzTMerM/Dmw/5h7kSSN4JWJklQ5g1qSKmdQS1LlDGpJqpxBLUmVM6glqXIGtSRVzqCWpMoZ1JJUOYNakipnUEtS5QxqSaqcQS1JlTOoJalyBrUkVc6glqTKGdSSVLmut+J6cER8JiKuiYhNEfGMcTcmSWp0uhUX8AHgi5n5kojYA9hzjD1JkgYsGtQR8SDgt4FXA2TmncCd421LkjSvy6GPRwFbgI9GxGURcVpE7DXmviRJrS6HPlYCRwBvzMyLI+IDwGrgrwcXiog5YA5genq6dJ/Fzaxe22m5zaeuGnMnkrR9Xfaovwd8LzMvbn/+DE1wbyMz12TmbGbOTk1NlexRknZpiwZ1Zv4AuCkiDm0nHQdcPdauJElbdT3r443Ap9ozPm4AXjO+liRJgzoFdWZuBGbH3IskaQSvTJSkyhnUklQ5g1qSKmdQS1LlDGpJqpxBLUmVM6glqXIGtSRVzqCWpMoZ1JJUOYNakipnUEtS5QxqSaqcQS1JlTOoJalyBrUkVc6glqTKdbrDS0RsBu4A7gbuykzv9iJJE9L1nokAx2Tmj8bWiSRpJA99SFLlugZ1AudFxPqImBtnQ5KkbXU99HFUZt4cEb8BnB8R12TmVwcXaAN8DmB6erpwm3WbWb120WU2n7pqAp1I2hl12qPOzJvb/94KnA0cOWKZNZk5m5mzU1NTZbuUpF3YokEdEXtFxAPnnwPPB64cd2OSpEaXQx8HAmdHxPzy/5qZXxxrV5KkrRYN6sy8AXjyBHqRJI3g6XmSVDmDWpIqZ1BLUuUMakmqnEEtSZUzqCWpcga1JFXOoJakyhnUklQ5g1qSKmdQS1LlDGpJqpxBLUmVM6glqXIGtSRVzqCWpMoZ1JJUuc5BHRErIuKyiDhnnA1JkrbVZ4/6ZGDTuBqRJI3WKagj4iBgFXDaeNuRJA3rukf9fuDtwK/H2IskaYRFgzoifge4NTPXL7LcXERcGhGXbtmypViDkrSr67JHfRRwYkRsBs4Ejo2ITw4vlJlrMnM2M2enpqYKtylJu65Fgzoz35GZB2XmDPBy4MuZ+YqxdyZJAjyPWpKqt7LPwpl5AXDBWDqRJI3kHrUkVc6glqTKGdSSVDmDWpIqZ1BLUuUMakmqnEEtSZUzqCWpcga1JFXOoJakyhnUklQ5g1qSKmdQS1LlDGpJqpxBLUmVM6glqXIGtSRVrstdyO8fEesi4vKIuCoi3j2JxiRJjS634volcGxm/iwidge+FhFfyMyLxtybJIkOQZ2ZCfys/XH39pHjbEqSdI9Ox6gjYkVEbARuBc7PzIvH25YkaV6nu5Bn5t3A4RHxYODsiHhCZl45uExEzAFzANPT08Ub3VXMrF676DKbT101gU4k1aLXWR+ZeTtwAXD8iHlrMnM2M2enpqYKtSdJ6nLWx1S7J01EPAB4LnDNuBuTJDW6HPp4KPDxiFhBE+yfzsxzxtuWJGlel7M+vgk8ZQK9SJJG8MpESaqcQS1JlTOoJalyBrUkVc6glqTKGdSSVDmDWpIqZ1BLUuUMakmqnEEtSZUzqCWpcga1JFXOoJakyhnUklQ5g1qSKmdQS1LlDGpJqlyXeyY+IiK+EhGbIuKqiDh5Eo1Jkhpd7pl4F/C2zNwQEQ8E1kfE+Zl59Zh7kyTRYY86M2/JzA3t8zuATcDDx92YJKnR6xh1RMzQ3Oj24nE0I0m6ty6HPgCIiL2BzwJvzsyfjpg/B8wBTE9PF2tQSzOzem2n5TafuqpIrVJ1utaSdiWd9qgjYneakP5UZp41apnMXJOZs5k5OzU1VbJHSdqldTnrI4CPAJsy833jb0mSNKjLHvVRwCuBYyNiY/s4Ycx9SZJaix6jzsyvATGBXiRJI3hloiRVzqCWpMoZ1JJUOYNakipnUEtS5QxqSaqcQS1JlTOoJalyBrUkVc6glqTKGdSSVDmDWpIqZ1BLUuUMakmqnEEtSZUzqCWpcga1JFWuyz0TT4+IWyPiykk0JEnaVpc96o8Bx4+5D0nSAhYN6sz8KnDbBHqRJI3gMWpJqtyidyHvKiLmgDmA6enpUmWlkWZWr110mc2nrpporS51Sta6r/ZUslaN61eyp3nF9qgzc01mzmbm7NTUVKmykrTL89CHJFWuy+l5ZwD/AxwaEd+LiD8ef1uSpHmLHqPOzJMm0YgkaTQPfUhS5QxqSaqcQS1JlTOoJalyBrUkVc6glqTKGdSSVDmDWpIqZ1BLUuUMakmqnEEtSZUzqCWpcga1JFXOoJakyhnUklQ5g1qSKmdQS1LlOgV1RBwfEddGxPURsXrcTUmS7tHlnokrgH8CXgg8DjgpIh437sYkSY0ue9RHAtdn5g2ZeSdwJvCi8bYlSZoXmbn9BSJeAhyfma9tf34l8LTMfMPQcnPAXPvjocC1i7z3AcCPltL0mOrUWsueJl/LniZfy57gkZk5NWrGonchB2LEtHule2auAdZ0qNcUjbg0M2e7Lj/uOrXWsqfJ17Knydeyp+3rcujje8AjBn4+CLh5qW8oSeqnS1BfAjwmIg6OiD2AlwOfH29bkqR5ix76yMy7IuINwLnACuD0zLyqwHt3PkwyoTq11rKnydeyp8nXsqftWPTDREnS8vLKREmqnEEtSZUzqCWpcga1JFWuywUvRUTEPsDxwMNpLpi5GTg3M29fjjqFewqaS+0H66zLJXxSGxGH0VyiP1jr85m5aTlrLVD/eZl5/nLXioi9gccCNyzndlC61gL1D8vMa5ajp3GvW/ser8nMj/ZY/kHAVGZ+e2j6kzLzmz3feyzrFxHPosmHKzPzvKXUmMgedUT8EbABOBrYE9gLOAZY386baJ3CPT0fuA54F3ACsAp4N3BdO69PT39J810qAayjOYc9gDP6fmthyVrb8ZFCdXrVioh/Hnj+LOBq4B+AKyLihD5vWuM2tYhev+g1/u4t4t09enopcA3w2Yi4KiKeOjD7Y33etPB2sG7g+Z8AHwIeCJyy1N+9iZyeFxHX0nw/yO1D0/cFLs7Mx06yTuGeNgEvzMzNQ9MPBv4rM3+zR0/fAh6fmb8amr4HcFVmPmbStSJioYubAjg2M/fq0VORWhGxITOPaJ9/BXhbZm6IiEcBn+5zqW6l29QHF5oFvCozH7QMPZUcp4X2dAN4bGber2OdjTS/e7dExJHAJ4B3ZuZZEXFZZj6lR08l12/re0fEJcAJmbklIvYCLsrMJ3atNW9Shz6CEd8PAvya0d8lMu46JWutpLnMftj3gd179vRr4GHAjUPTH9rOW45azwZeAfxsaPr84Z4+Staa96DM3ACQmTe0X8vbR43b1GuAtwG/HDHvpGXqqeQ4HQi8APjJiPf4Ro86KzLzFoDMXBcRxwDnRMRBC/S6PSXXb7c24Hej2Rne0vb4vxFxV89awOSC+r3Ahog4D7ipnTYNPA94zzLUKVnrdOCSiDhzoM4jaC6173to4M3AlyLiuqGeHg28YcFXjbfWRcDPM/PC4RntXkgfpWod1u6VBTATEftm5k8iYjf6/3GscZu6hOZ45r1CKyLetUw9lRync4C9M3Pj8IyIuKBHnTsi4pD549PtnvXRwOeAx/fsqeT67QOspw3/iHhIZv6g/Sylb+gDE7wysf0L8wKaA/VBsxd6bmYO/1WdSJ3CPT0OOHGozucz8+ol9LQb93wwOV/rksy8ezlr1SQiHjk06ZbMvDMiDgB+OzPP6lmvqm0qIvYDfpGZP+/7/uPqqWSdUiLiyTR/+K8bmr478NLM/FTPemNdv4jYEzgwM7/T+7VeQn7fEBF7Z+bwIQONQUTsl5m37cDrD2TgzIHM/GGx5iqyo+NUu4g4MTOr+AK6ZT+POiKu6LHsbRFxWkQcFxFL+ifEQK1HRMSZEfHfEfHO9q/w/LzP9ahzWER8ISLWRsQhEfGxiLg9ItZFROcPEjvotXceEU+KiIsi4qaIWNPuLczPW7e91w7VKTJOJWuVHPOIOCoiNkVz1sDTIuJ84NJ23J7Rs9bhEXERcAHwt8DfARe2/x+OWI71W+R9+vzulRynGrfNFw8/gDUDz/vUKrJ+28jMsT+AFy/w+H1gS48619IcX/06zYd1HwCevsSezgdeDxwO/CPNhxj7t/Mu61Hnq8Dv0nzIcyPNselop32pZ09vXeDxNuC2nrW+RnNO6IOBvwCuAg5ZwvoVGaeKx3wd8ETgGTR34HhWO/0I4Os9a22kOXNgePrTgcuXaf1K/e6VHKcat827aI6dnw58tH3c0f739OVYv21qLuVFvd8EfkVzXuNHRzzu6FFnw8DzaeDtNOc+3gD8Tc+eNg79/Ir5AR18nw51Lht4fv1C/Xas9QuaDy5OGfG4fQfX7xia872f3nP9ioxTxWM+WGvTDta6bjvzrl9iTzu6fqV+90qOU43b5lOBLwF/yj2HhL/Tp0bp9dumxlJetITG1wNPWGDeTUvZWIamHwqc0rOnq4D7D017LnA9zYdTXet8c+D5nw3Nu7JnT98AfmtHx6ld/nJgn6FpT2o3mB9PepwqHvPLB57/3g7W+iCwFngZ8Mz28bJ22oeWaf1K/e6VHKfqts32tbsBJwNfofkQ/oa+NUqu3zavX8qLltD4s4HpBebN9qjzvoI9vQV4zojpTwHO71HndTSnGg1PfzTw/p49HQocsMC8A3vW+kNGHBai+ZfIv0x6nCoe8xOBPUdMPwR4+xK2qxcCHwb+k+af0h+mueChT42S61fqd6/YONW4bQ69/uHAp3cgqIus3+DDsz4kqXKT+q6PlRHxuoj4YkR8MyIubz/Vfv3gJ7WTqlNrT4u8T69b+UTEPhFxakRcExE/bh+b2mkP7lFnpx7ziFjR1npPRBw1NO+vetaaH/NNO+GYF9meOrxP5+288DgVW79xjNWkvuvjDOB24OPcc7n1QcCrgP0y82WTrFNxT/stNIvmGOFBPWqdC3wZ+Hhm/qCd9pC2r+dm5vM61tnZx/w0mi/hWQe8ErgwM9/aztv6nSIday005q8GjruPj3mR7al9XZHtvPA4lVy/YrW2WupxnJ7HbK7dzrxvTbpOxT3dTXMGy3cGHvM/31lw/RactwuO+eAHdytpbkJ6FnA/+p/itauOeed1a5cvsp1PcJz6rl+xWvOPSV3w8pOI+INoLmkGmsubI+Jl3PuLWSZRp9aebgCOzsyDBx6PysyDgb5Xt90YEW+P5iq5+b4OjObrT2/azuuG7exjvsf8k8y8KzPnaD61/zKwd89aO/OYl1o3KLedlxynkutXslZjKene9wHMAP8G3Ap8q33c2k47eAl1trQ1rltKnZK1Cvf058CTF5j3xp619qW5Ou4amo32NmBTO22/ZR7zmraDTwLHj5j+WuBXO9GY7+h2XmTdSm7nhcep5PoVq7W15lJetKQ3gqfRnJu4P/Asmit2ep22NFRvf+AA4JMFeitSq0Sddoye2j5/HM2ViUsaJ+AwmvNK9x6afq9gmuD6VbsdDNT8xA68troxL1Wr5LqV3M5LjVPh9Su6HUzqw8RTaM4vXUlz2eeRwIXtipybme/tWGfUF6QcS/PPVDLzxB49FalVuKfhcXoazfdG9BqnttabaPZcNtFcYntyZv5HO6/zh2RjXr8at4OguZJsKbVqHPNS23mRdWuXL7KdFx6nkutXrNZWO/pXuuNflyuAFTSfrv+U5sveAR7AwIc5HepsoPmn6tHAc9r/3tI+f07PnorUKtxTkXEaqLV3+3wGuLTdYKDf9ylUt36Fe7qs8PrVNualtvMi61bxdlB6/YrU2lpzKS/q/Sbbfk/AZUPzNvaosxvN1UjnA4e305Z69VCRWoV7KjJO7fJXD/28N/BF4H3LOOY77XZQ8ZiX2s6LrFvF20HJ9StWa2uNpbxoCYNwMe3lp8BuA9P3YQlfUkJzruS/09w08rs72FuRWiXqlBwnmn/+HT40bSXNfeXuvq+vX43bQY1jXqpWyXWrcTsovH5Ft4PMyQX1/RaYfgDwxB2ou4qe35o37lo7UqfkOLUb70MWmHfUfX39atwOahzzUrVKrluN20Hh9Su+HfhdH5JUuWW/w4skafsMakmqnEEtSZUzqCWpcga1JFXu/wFtJOSukRTCxwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "single_id[\"event_code\"].value_counts().plot.bar()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "25"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#This user has 25 different game sessions\n",
    "single_id[\"game_session\"].value_counts().count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Game          28\n",
       "Activity      15\n",
       "Assessment     2\n",
       "Name: type, dtype: int64"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "single_id[\"type\"].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It might be interesting to compare some features in the train data to the test data, as a reality check that things are relatively similar."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "test = pd.read_csv(\"../DSB2019/data-science-bowl-2019/test.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1156414, 11)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7ff775d68910>"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEICAYAAABBBrPDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAdvklEQVR4nO3debRlVX3g8e8PSg2EQJVUiciQIopzOyABEk2L0WChHTFGo/aKlLY2RsE2iUZJlt2knRq7VzTRJKZRCdAOxMQBFA2WSLSNA5SAgCJWCQglCCWFSgcn5Nd/7P2sU9d79znv3TdV8f2sdda9d++zh7PP8HtnuPdFZiJJ0iS7LXUHJEnLm4FCktRkoJAkNRkoJElNBgpJUpOBQpLUtGKpOzDfVq9enWvXrl3qbkjSTuVLX/rSdzJzzbi8XS5QrF27lo0bNy51NyRppxIR35yU56UnSVKTgUKS1GSgkCQ1GSgkSU0GCklSk4FCktRkoJAkNRkoJElNu9wX7masPfm8HT5fd+pTl6gnkrRz84xCktRkoJAkNRkoJElNBgpJUpOBQpLUZKCQJDUZKCRJTQYKSVKTgUKS1GSgkCQ1GSgkSU0GCklSk4FCktRkoJAkNRkoJElNBgpJUpOBQpLUZKCQJDUZKCRJTQYKSVKTgUKS1GSgkCQ1GSgkSU0GCklSk4FCktRkoJAkNfUGiog4KCIujIirIuIrEfHymn7viNgQEZvq66qaHhHx1ojYHBGXR8RhnbrW1/k3RcT6TvpjIuKKWuatERGtNiRJi2fIGcWdwCsy8yHAUcCJEfFQ4GTggsw8FLigfgY4Fji0TicAb4dy0AdOAY4EjgBO6Rz4317nnSm3rqZPakOStEh6A0Vm3pSZl9T3twNXAQcAxwFn1tnOBJ5e3x8HnJXFF4CVEbE/8GRgQ2Zuy8zbgA3Aupq3d2Z+PjMTOGukrnFtSJIWyazuUUTEWuDRwBeB/TLzJijBBLhPne0A4IZOsS01rZW+ZUw6jTYkSYtkcKCIiL2ADwB/mJnfb806Ji3nkD5YRJwQERsjYuPWrVtnU1SS1GNQoIiIe1CCxHsy84M1+eZ62Yj6ektN3wIc1Cl+IHBjT/qBY9JbbewgM0/LzMMz8/A1a9YMWSRJ0kBDnnoK4F3AVZn55k7WucDMk0vrgXM66cfXp5+OAr5XLxudDxwTEavqTexjgPNr3u0RcVRt6/iRusa1IUlaJCsGzPNY4HnAFRFxWU37M+BU4P0R8ULgeuBZNe9jwFOAzcAdwAsAMnNbRLwOuLjO99rM3FbfvwQ4A9gD+HidaLQhSVokvYEiMz/L+PsIAE8cM38CJ06o63Tg9DHpG4GHj0m/dVwbkqTF4zezJUlNBgpJUpOBQpLUZKCQJDUZKCRJTQYKSVKTgUKS1GSgkCQ1GSgkSU0GCklSk4FCktRkoJAkNRkoJElNBgpJUpOBQpLUZKCQJDUZKCRJTQYKSVKTgUKS1GSgkCQ1GSgkSU0GCklSk4FCktRkoJAkNRkoJElNBgpJUpOBQpLUZKCQJDUZKCRJTQYKSVLTiqXuwFJZe/J5O3y+7tSnLlFPJGl584xCktRkoJAkNRkoJElNBgpJUlNvoIiI0yPiloi4spP25xHxrYi4rE5P6eT9aURsjoirI+LJnfR1NW1zRJzcST8kIr4YEZsi4h8i4p41/V718+aav3a+FlqSNNyQM4ozgHVj0t+SmY+q08cAIuKhwHOAh9UyfxsRu0fE7sDfAMcCDwWeW+cFeFOt61DgNuCFNf2FwG2Z+QDgLXU+SdIi6w0UmfkZYNvA+o4Dzs7MH2XmtcBm4Ig6bc7MazLzx8DZwHEREcBvAv9Uy58JPL1T15n1/T8BT6zzS5IW0TT3KE6KiMvrpalVNe0A4IbOPFtq2qT0fYHvZuadI+k71FXzv1fnlyQtorkGircD9wceBdwE/EVNH/cXf84hvVXXz4mIEyJiY0Rs3Lp1a6vfkqRZmlOgyMybM/OnmXkX8A7KpSUoZwQHdWY9ELixkf4dYGVErBhJ36Gumr8PEy6BZeZpmXl4Zh6+Zs2auSySJGmCOQWKiNi/8/F3gJknos4FnlOfWDoEOBS4CLgYOLQ+4XRPyg3vczMzgQuBZ9by64FzOnWtr++fCXyqzi9JWkS9v/UUEe8DjgZWR8QW4BTg6Ih4FOVS0HXAiwEy8ysR8X7gq8CdwImZ+dNaz0nA+cDuwOmZ+ZXaxKuBsyPi9cClwLtq+ruA/xMRmylnEs+ZemklSbPWGygy87ljkt81Jm1m/jcAbxiT/jHgY2PSr2H7patu+g+BZ/X1T5K0sPxmtiSp6W77M+N9/BlySSo8o5AkNXlGMUeecUi6u/CMQpLUZKCQJDUZKCRJTQYKSVKTgUKS1GSgkCQ1GSgkSU0GCklSk4FCktRkoJAkNRkoJElN/tbTAvG3oCTtKjyjkCQ1GSgkSU0GCklSk4FCktRkoJAkNRkoJElNBgpJUpOBQpLUZKCQJDUZKCRJTQYKSVKTgUKS1GSgkCQ1GSgkSU0GCklSk4FCktRkoJAkNRkoJElNBgpJUlNvoIiI0yPiloi4spN274jYEBGb6uuqmh4R8daI2BwRl0fEYZ0y6+v8myJifSf9MRFxRS3z1oiIVhuSpMU15IziDGDdSNrJwAWZeShwQf0McCxwaJ1OAN4O5aAPnAIcCRwBnNI58L+9zjtTbl1PG5KkRdQbKDLzM8C2keTjgDPr+zOBp3fSz8riC8DKiNgfeDKwITO3ZeZtwAZgXc3bOzM/n5kJnDVS17g2JEmLaK73KPbLzJsA6ut9avoBwA2d+bbUtFb6ljHprTYkSYtovm9mx5i0nEP67BqNOCEiNkbExq1bt862uCSpYa6B4uZ62Yj6ektN3wIc1JnvQODGnvQDx6S32vg5mXlaZh6emYevWbNmjoskSRpnroHiXGDmyaX1wDmd9OPr009HAd+rl43OB46JiFX1JvYxwPk17/aIOKo+7XT8SF3j2pAkLaIVfTNExPuAo4HVEbGF8vTSqcD7I+KFwPXAs+rsHwOeAmwG7gBeAJCZ2yLidcDFdb7XZubMDfKXUJ6s2gP4eJ1otCFJWkS9gSIznzsh64lj5k3gxAn1nA6cPiZ9I/DwMem3jmtDkrS4/Ga2JKnJQCFJajJQSJKaDBSSpCYDhSSpyUAhSWoyUEiSmgwUkqQmA4UkqclAIUlqMlBIkpoMFJKkJgOFJKnJQCFJajJQSJKaDBSSpCYDhSSpyUAhSWoyUEiSmgwUkqQmA4UkqclAIUlqMlBIkpoMFJKkJgOFJKnJQCFJajJQSJKaDBSSpCYDhSSpyUAhSWoyUEiSmgwUkqQmA4UkqclAIUlqMlBIkpqmChQRcV1EXBERl0XExpp274jYEBGb6uuqmh4R8daI2BwRl0fEYZ161tf5N0XE+k76Y2r9m2vZmKa/kqTZm48ziidk5qMy8/D6+WTggsw8FLigfgY4Fji0TicAb4cSWIBTgCOBI4BTZoJLneeETrl189BfSdIsLMSlp+OAM+v7M4Gnd9LPyuILwMqI2B94MrAhM7dl5m3ABmBdzds7Mz+fmQmc1alLkrRIpg0UCXwiIr4UESfUtP0y8yaA+nqfmn4AcEOn7Jaa1krfMiZdkrSIVkxZ/rGZeWNE3AfYEBFfa8w77v5CziH95ysuQeoEgIMPPrjdY0nSrEx1RpGZN9bXW4APUe4x3FwvG1Ffb6mzbwEO6hQ/ELixJ/3AMenj+nFaZh6emYevWbNmmkWSJI2Yc6CIiF+MiF+aeQ8cA1wJnAvMPLm0Hjinvj8XOL4+/XQU8L16aep84JiIWFVvYh8DnF/zbo+Io+rTTsd36pIkLZJpLj3tB3yoPrG6AnhvZv5zRFwMvD8iXghcDzyrzv8x4CnAZuAO4AUAmbktIl4HXFzne21mbqvvXwKcAewBfLxOkqRFNOdAkZnXAI8ck34r8MQx6QmcOKGu04HTx6RvBB4+1z5KkqbnN7MlSU0GCklSk4FCktRkoJAkNRkoJElNBgpJUpOBQpLUNO1vPWmO1p583g6frzv1qUvUE0lq84xCktRkoJAkNRkoJElN3qNYpryHIWm5MFDspAwkkhaLl54kSU2eUeyiPOOQNF88o5AkNXlGcTfVd8Yxbb6kXYeBQgvCQCPtOrz0JElqMlBIkpoMFJKkJgOFJKnJQCFJajJQSJKaDBSSpCYDhSSpyS/caVnyC3nS8uEZhSSpyUAhSWry0pN2Sl6akhaPZxSSpCYDhSSpyUtP2uV4WUqaXwYK3e0YSKTZ8dKTJKnJMwpphP+dT9rRsg8UEbEO+Ctgd+CdmXnqEndJavL/kWtXs6wDRUTsDvwN8FvAFuDiiDg3M7+6tD2Tls5CB6Llnq/Ft6wDBXAEsDkzrwGIiLOB4wADhXQ3tdSB6u4Y6CIzl7oPE0XEM4F1mfmi+vl5wJGZedLIfCcAJ9SPDwKu7mSvBr7TaMb8XTd/OffNfPOXW/4vZ+aasXNm5rKdgGdR7kvMfH4e8LZZ1rHR/Ltn/nLum/nmL/f87rTcH4/dAhzU+XwgcOMS9UWS7paWe6C4GDg0Ig6JiHsCzwHOXeI+SdLdyrK+mZ2Zd0bEScD5lMdjT8/Mr8yymtPMv9vmL+e+mW/+cs//mWV9M1uStPSW+6UnSdISM1BIkpoMFJKkJgOFJKlpWT/1NFsRsQ+wDjgASMp3Ls7PzO/W/KD8LEg3/6Ls3NEfUMeDKT8j0s0/NzOv6ivbV35I+5169gIeCFzT6Vtf3+e8bD3j/luZuaGR/4LM/PuRtEOARwNfzcyvDam/b/1FxN7Amsz8xkj5R2Tm5UPHdkz7D87Mr83D+A5uf9z4zOf6m7D99G2b9wXIzG9HxBrgN4CrZ55EnOf+PY6yrq/MzE/0jc3A/s+m/R3amK/9dpzO9j3n8ZmPY1+zj7vKU08RcTxwCvAJ4Fs1+UDKDwr+d+DbwN8Cm0byHwC8tA52Xx37A88FzqZ8GXAm/zmUnw05fFLZzDwrIl7dKH82ZeVNav/azHxSXdbHAe8FvlH7/2LK1/FbfWeKZTs7G7/aGxHXZ+bBrXzgksx8ev18HPCXwL8Avw78j8w8o6f8i2isP2BlrfMW4B7A8zPz4lr+kpo3cfkz86ye9l/TKl8/T5P/jNb4AHf1lG+uP+DgzHxprX/c9vPvesrfBpwMBPAm4PnAV4DHAv8T+MmU/XtGZh5R+/efgROBDwHHACsz82GTxiYzzxiwb2VP/lGN8f8yJajOZb9tblu1vespP3461/HZQuPYw4BjX6t/wPL+CY/ZTJQD9cox6auArwNXAWvH5B8CXDWwjq8D9xiTf0/gx62y9X2r/Kae9n/Y+XwhcFh9/yvAxgF9n2bZNlG+6Dhu+gjwb8DlE6YrgB8Bl3bq/BxwSH2/mrIj9tXfXH/AZcD+Ne0I4GuUnQvg0gHL/9YJ09uA78/D+Pbl943PtOvvkp7tp6/8FcCewL7A/wPu22n/snnoX3f5L6acGQL8IvCD1tgM3Ldm0/5oGz/qKdtc9vq+b/ueZnx+1DP2vce+vmlXuvQUlL8aRt1V81awPVJ3fYvyF+iQOn4K3A/45kj+/vW1VXbm/aTyM/NNqqNr78y8BCAzr6k/x97X977+tZbtLsplht+nHCS6Zk5p9wOeTPnLczT/cyNtr8jMa2v/vxMRQ+rvW393ZuZNtc6LIuIJwEcj4sDadt/4vAB4BWWnG/XcAeWZMr9vfKbZNodsP33b5k8y8w7gjoj4RmZ+u5a/LSKGjG9f/3aLiFWU+6aRmVtr/f9W62+NzUw7rfpzQP6kNvrKDtk2+rbvvv73jc+0x76mXSlQvAG4JCI+AdxQ0w6mnH69jjLgF9efKp/JP4hyaveugXV8G7ggIjaN5D+A8tdnqyzAHzbKn0Q52E6qIyLicsqKXxsRq+pOuhtlZff1PadYtpOAlwN3ZOanRwc+Iq6m/NWzV2ZeNib/X4BnR8T3a//vFRH3zXKt+56Ub91/oaf+j/Pz6+9g4NmU9ffbEXH/rPcnMvOmiDga+DDwMMqvC7eW/3jK9d7PjWn/z+dhfPvy39UzPtNsmycBH+zZfvq2zddHxD0y8yfAz343OyJ+gXLwmrZ/fwd8qfYvO8u/F7BHz9gwoP/05J/XaOPGnrKt/XZm3+/bvt84xfjc3NP+uGNfd9/ptcvcowCoEffJlBs2QYmi52fmbTX/ocDTRvJ3+EdIA+rYje03hWbyL87Mn/aV7Svfah/Ye2Rxb8rMH0fEauDfZ+YHB/R9zss2i9UwKxGxEnhIZn5+wLwPYfvNvh3WX0Q8krIjbhopcw/g9zLzPa3lj4h7Uy7v3dFof9rx7d0+WuMz5bb5yyNVj9t+WuUPrmV+MtK/A2r/PrkQ21dE7AnsN/MX/qSxGVL/HNtfCTwE+GJP3bNet2PamvP4AN9ttT/k2Nfs264UKJZSROyVmaOnlfNR770zc9ss5t+PzpMNmXnzfPdpuYiIw2YuocyizIKNT0SsyMw76/u9gAdTnioau/761u2A/Kdl5sQfyRxQflbjN2T55jK+s9l36gH5zsy8vWe+BwCPpFyDbx4MJ7U/ZN8bLdvqX2v8ImJlNp6Q6sufrYhYnZmt/1WxoyE3MnaGCdgGvBN4IjUAjuQ/mHL54jzg/sAZlCh8EeWvkr76r+jJ/xbl6YT/C/wZnRtTwIfr6yMop6A3UH6Qa1VnnosoT5BcRXma5EhgA3BNnf85rf4Dj6p1X1XLfZJyQ/cLwGGUy2wT+zegbwtdvi//sDHTFsojjEOWr2985rx9UG70Ph+4lXLz8Ni63i6o4/HcnnX7awPynzFm+nbn/Ws6/Xlo7ce1wHW1vtGxe8zI+PWtv77la45vz/hd39P+pcBZwPco9zqur9Ofz6xnyg361fX982o/31nXzcsGtN8c/56y9xvQv77xu7OO2QsZf2N6Yj7wnzrvD6j13ka5N/jA2t61wGfr+v4K5Ym3LcATBx1fl/oAP18T5cmDk4B/pRy0/4ryyNtM/meA364r5ZuUA2/UtAvqPON2xmcAvwtsBf54wvQKyuOBf1B3mLfVlbTvzIZeXz9LedZ5JfDKusLu39kZLqI8pvhrlP889biad1jdCCf2n/LkyZFjxuUoylMzG1r9G9C3hS7fl39XTbuwM/2gvn5qQPm+8WluHxO2i+62cQXlCZlDKE9JzSz7fpSnv1rr9l8H5N8JfBQ4Hfj7Ot1eX09nx6eazgOOre+PqGPRN359669v+frGt7XvbOtp/3bg6M4++hbK0z6vB06r6Vd22ry4s+73rP3ra781/tf2lP3UgP71jd8VwH8A3kMJKOdQtsE9OuXH5o+s+/dTHnfeDfgdth8bHlKX7VbqcbGmXTK6znb1QNEdrIOBVwGXUCL3G9nx8bLN48pSDvZnsH1H7E63Az+k3Bw6Zcz005E6f39mY+/Uf9nIPE+g3AQ+qva128erRua9o9V/YFNjbDaPaXuH/g3o22KXH81/JvBp4Cmdea7tvO8r3zc+ze1jwLZxWWf+G0fKX96zbvvW/SXAr1J2+pew/ZJxd/m72/+lI+UvncP4TVx/E5avb3xb+853e9q/YyTvS533X+ss4wH1/YXAL9T3u9ftoK/91vjf1VP2ywP61zd+3fW3B/B7wAcpB/b39uRva6zHS0fK3jCSv8P8E9fhkJl2hml05+ikP6iu0Ms7aS8dmefKmRUMPHxCPTdQ/iJ7zIT8n8xsnJ20J9Wd5Kb6+cvAPiPzPKLuELd2Nzjg6SPz/bDVf8pTV+dRnmT49To9u6b9dd1ZJvZvQN8Wunwzv37ei/LX2j9S/hi4pjNvX/1949PcPgZsG+dSvhj315S/MP+CcjnjFMrDCK11e2Vffn3djfL02YWUM4Xu8n+X7c/lbwX2HFO+NX59669v+frGt7Xv3NDT/o8pgf9+wMuAD9T8YPv3FI6u28BrO+39N8qZ5iuHtN8Y/zt6yn5yQP/6xm/S8WsfYH1P/vfZ/p2fb7HjZdcra3svBv6EEpT+iHKJaj3w2XH1/lw7Q2baGSbgzT35L6Y8vjma/gDgL+v736B8g3Vc+cMpQWf1hPz/Cjx+TPqjgQ31/X+kczmsM8/BwDsoTyXsOSb//sAHBvT/WMpjdB+hXKb4O+pfkHXjmNi/AX1b6PLN/DFpFwJbO2m95XvGp7l9DNg29gb+lPLt5b0of8F/lPKN2P171u2r+vJH0g6gXGLoHugfPzL9Uk3fDzhxwPj1rb/m8g0Y3wdRvyQ2po39etp/b13eK4F3d9rbF/jdzrz7UM643kI5aL4aePDA9lvj/7+YvN/vV/vY7N+A7eOV4+rvtDMxn3LA706ravp9KVdTDgL+d10f96XsK1dSgnjv/dnM9Kkn7ZwiIigHw+8vdV92Ro6fZmVINNkZJsqXB18M/DPl9OrLlKdY/oDyhaJm/pA6etp/x4D69wFOpTwNcmudrqppK3vy9+1ZvpmyV02oe5plO22hyy90/iL0r7vuto0Z/91r+dcBjx1p/zUD8qfZdnrX/yzbH7d8fdvffGz7H5+0budhfJr5A7edof1rjd+8j0+jfy+mZ9/Y5c4oIuJ9lOu0Z7Ljj2qtB+5NuSE1MT8znz2gjpdMap5yrfLMnvrPp1wvPDPrTyBE+UXO9ZTr6Xc18v+I8iNgk/q2ckLZ51MeGf7OFMv2ZcpjpwtZvrl+5iF/ofvXN/7fpDyBcxHl8c1PZ+Yf1/kuodwwbuVvnVD/kG3nSfSv/9vn2P7M8k1qvy9/6m2/Z9+aj/bXUZ5uGzVo21mE/vUdG3qPfWOWbccF3YUCxdWZ+aAJeV8HspWfmQ8cUMf9KTt8dLKyfl6bmTGp7ID6r6Z0clL+jzPznnNcvr66+5btAMoTMgtZvrl+5iF/Kft3NfCjzHxE/byCcm16NeVx3C8Au/fk7znF+h2y/n+4hO1Pte3Pw77Vl5+UR2TntO0sQv+mOTZ8PTMfOC6va1f6x0W3RcSz6tfggfKV+Ih4NuXLJ335Q+q4hvK89CGd6Vcy8xDgxwPq/2ZEvKp+e3Vmnv2i/ETyDT35P+ipv6/uaZbt5kUov9D5C92/vvH/2Y6cmXdm5gmUv0Y/Rbm52Zc/zbYzZP0vdPsLue2z0O0z3baz4P2bctvtN+T61M4wAWuBf6CcIn+d8ljdLTXtkL78kTpuYftPI3frOBF45IT2TxlQ/yrKb/l/ra6gbZTrjG+inCK28h/Zs3x9dU+zbC+bx/JzWj/zkL/Q/esb/3cD68a0/SLKo9V9+dNsO931P6n/C93+gm3787Bv9eX/CVNsO4vQv75jQ2//eo+vS32AX6CgsS/ltPnds82nfH3/iDrP4yjPYHe/pHQE8Kv1/UMp39B8yizqfzDlmuNeI+nrhuS36h9Q91TLNh9jM+36mY/8znxnzWf9Q9bdkPYn5c/HtjPN+Mxm+SiPE78COGZC3T+XP822Px/jM8vlm/W2sxj9m699Y3Tale5RnDsm+Tcpp86T/Cw/M58WEadQngVfQXm2/wjKt1mfRPlSzIqR/CMp/wXrScAaSqRu1f9fKH/ZXkX5qYmXZ+Y5tf+XUL75Oyn/u5SfmZi0fJ/sqfucKZZtXP5syx/ZGv8JFjI/KN/+na/6+8Z/y0jZvvZH8/vqP2OW7fct32zbvzO3/we2F9V5P0z5D2wfYcf/0DYu/45G/c1tfx72rdnue7Padpagfzu0P6R/jfmK2USV5TxRnhp5N+Ubmo+vrzex/QtIzfxaxxWUxwT3pHzbce+avgfbf49lUv4PBta/V32/lvKfxV5eP1/ak39Hz/INqXuuyzYf+VOtn3nIv3SB6+8b/772+/Kn2XYunYfx6V2+zr44+h/YrhiQP+dtfx72rWn3vaHHlmXbv97j61If4OcxUOxGeUxsA/ComnbN0PyZFTLuff182YD8vvq/OvJ5L8pz7W+u5fvyW8vXV3baZZs2f6r1sxPk943/Qte/1O1/mXIdfV9g4+h+NSB/ztv+PI3PNPvekGPLsu5f37TkB/j5nijPB/8j5TdVrp9NPuWfk+w5M7id9H0oUbmZP6D+T82sqE7aCspPFP+0L79V/4C6p1q2+RibadfPcs4fsu4Wsv5l0P51lCfLrq2vM/9Tey/Kgawvf87b/nyMz7Tt7+z965uW/MC+UBPl3zW+cTb5wL0mzLua8hPEzfwB9R84s4OMqeOxffmt+gfUPdWyzefYzHX9LOf82ay7hah/qdtv1LMnjSdrZvKn2fbnY3ymbX9n71/ftMvczJYkLYxd6Qt3kqQFYKCQJDUZKCRJTQYKSVKTgUKS1PT/AQBeVFiYqhZMAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "test[\"event_code\"].value_counts().plot.bar()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Event code, at least, looks fairly similar. That's good!\n",
    "Next, let's look at various other statistics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "42"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "42"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(train_sample[\"event_code\"].nunique())\n",
    "display(test[\"event_code\"].nunique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "358"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "365"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(train_sample[\"event_id\"].nunique())\n",
    "display(test[\"event_id\"].nunique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4070"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "1000"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(train_sample[\"installation_id\"].nunique())\n",
    "display(test[\"installation_id\"].nunique())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unpack the inner data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Code below based on code original written/published by Gabriel Preda\n",
    "# on Kaggle, accessed Mon, 18 Nov, 2019\n",
    "# https://www.kaggle.com/gpreda/2019-data-science-bowl-eda\n",
    "extracted_event_data = pd.io.json.json_normalize(train_sample.event_data.apply(json.loads))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(100000, 135)"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "extracted_event_data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>description</th>\n",
       "      <th>identifier</th>\n",
       "      <th>media_type</th>\n",
       "      <th>duration</th>\n",
       "      <th>level</th>\n",
       "      <th>round</th>\n",
       "      <th>event_count</th>\n",
       "      <th>game_time</th>\n",
       "      <th>event_code</th>\n",
       "      <th>coordinates.x</th>\n",
       "      <th>...</th>\n",
       "      <th>bowls</th>\n",
       "      <th>bird_height</th>\n",
       "      <th>hats_placed</th>\n",
       "      <th>stage_number</th>\n",
       "      <th>tutorial_step</th>\n",
       "      <th>toy_earned</th>\n",
       "      <th>molds</th>\n",
       "      <th>dwell_time</th>\n",
       "      <th>tape_length</th>\n",
       "      <th>bug_length</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>Which pail is right for the chicken?</td>\n",
       "      <td>chicken_whichwater</td>\n",
       "      <td>animation</td>\n",
       "      <td>902.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>126</td>\n",
       "      <td>169840.0</td>\n",
       "      <td>3110</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>26</td>\n",
       "      <td>28093.0</td>\n",
       "      <td>4070</td>\n",
       "      <td>661.0</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>Tap the bucket that holds the least amount of ...</td>\n",
       "      <td>Dot_TapBucketLeast</td>\n",
       "      <td>audio</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>24</td>\n",
       "      <td>26100.0</td>\n",
       "      <td>3010</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>Amazing!</td>\n",
       "      <td>Dot_Amazing</td>\n",
       "      <td>audio</td>\n",
       "      <td>1132.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>20</td>\n",
       "      <td>25383.0</td>\n",
       "      <td>3110</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>Three!</td>\n",
       "      <td>sid_3</td>\n",
       "      <td>animation</td>\n",
       "      <td>467.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>100</td>\n",
       "      <td>67773.0</td>\n",
       "      <td>3110</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 135 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                         description          identifier  \\\n",
       "0               Which pail is right for the chicken?  chicken_whichwater   \n",
       "1                                                NaN                 NaN   \n",
       "2  Tap the bucket that holds the least amount of ...  Dot_TapBucketLeast   \n",
       "3                                           Amazing!         Dot_Amazing   \n",
       "4                                             Three!               sid_3   \n",
       "\n",
       "  media_type  duration  level  round  event_count  game_time  event_code  \\\n",
       "0  animation     902.0    5.0   12.0          126   169840.0        3110   \n",
       "1        NaN       NaN    NaN    1.0           26    28093.0        4070   \n",
       "2      audio       NaN    NaN    NaN           24    26100.0        3010   \n",
       "3      audio    1132.0    NaN    NaN           20    25383.0        3110   \n",
       "4  animation     467.0    NaN    NaN          100    67773.0        3110   \n",
       "\n",
       "   coordinates.x  ...  bowls  bird_height  hats_placed  stage_number  \\\n",
       "0            NaN  ...    NaN          NaN          NaN           NaN   \n",
       "1          661.0  ...    NaN          NaN          NaN           NaN   \n",
       "2            NaN  ...    NaN          NaN          NaN           NaN   \n",
       "3            NaN  ...    NaN          NaN          NaN           NaN   \n",
       "4            NaN  ...    NaN          NaN          NaN           NaN   \n",
       "\n",
       "  tutorial_step toy_earned molds  dwell_time tape_length  bug_length  \n",
       "0           NaN        NaN   NaN         NaN         NaN         NaN  \n",
       "1           NaN        NaN   NaN         NaN         NaN         NaN  \n",
       "2           NaN        NaN   NaN         NaN         NaN         NaN  \n",
       "3           NaN        NaN   NaN         NaN         NaN         NaN  \n",
       "4           NaN        NaN   NaN         NaN         NaN         NaN  \n",
       "\n",
       "[5 rows x 135 columns]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "extracted_event_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7ff767aab7d0>"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEICAYAAABfz4NwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de7hdVXnv8e+PBBQagUDCxYQ0EcP9QIQ0pEUqikDAlqAVIX0KQfEEEDzaamv0+Bw8Ujx4WvUUq3iipIRHBbGIRIHGGKnWI0gCibkQICHcNolJICCxXDThPX+MscnMylpzrr3nXnvvwO/zPPPZa413jjHHHHOu9e55WWspIjAzM9tloDtgZmaDgxOCmZkBTghmZpY5IZiZGeCEYGZmmROCmZkBMHSgO9BbI0aMiLFjxw50N8zMdir33nvvUxExsllsp00IY8eOZdGiRQPdDTOznYqkx1rFfMrIzMwAJwQzM8ucEMzMDHBCMDOzzAnBzMyANhKCpIMk3SlppaQVkj6Sy/eRNF/Sqvx3eC6XpKslrZa0VNKxhbam5/lXSZpeKD9O0rJc52pJ6sTKmplZa+0cIWwBPhYRhwOTgUslHQHMBBZExHhgQX4OcDowPk8zgGsgJRDgcuB4YBJweXcSyfPMKNSbUn/VzMysJyoTQkSsi4j78uPNwEpgFDAVmJNnmwOclR9PBa6P5G5gb0kHAqcB8yNiU0Q8A8wHpuTYnhFxV6QfZ7i+0JaZmfWTHn0wTdJY4C3AL4H9I2IdpKQhab882yjgiUK1rlxWVt7VpLzZ8meQjiQYM2YMAGNn3rbdPI9e9a6erJKZmWVtX1SWNAy4GfhoRDxXNmuTsuhF+Y6FEbMiYmJETBw5suknr83MrJfaSgiSdiUlg29FxPdy8fp8uof8d0Mu7wIOKlQfDaytKB/dpNzMzPpRO3cZCbgWWBkRXyyE5gLddwpNB24tlJ+f7zaaDPwmn1qaB5wqaXi+mHwqMC/HNkuanJd1fqEtMzPrJ+1cQzgBOA9YJmlJLvsUcBVwk6QLgceBs3PsduAMYDXwPPB+gIjYJOkKYGGe77MRsSk/vgS4DtgduCNPZmbWjyoTQkT8nObn+QFObjJ/AJe2aGs2MLtJ+SLgqKq+mJlZ5/iTymZmBjghmJlZ5oRgZmaAE4KZmWVOCGZmBjghmJlZ5oRgZmaAE4KZmWVOCGZmBjghmJlZ5oRgZmaAE4KZmWVOCGZmBjghmJlZ5oRgZmaAE4KZmWXt/ITmbEkbJC0vlH1H0pI8Pdr9S2qSxkp6oRD7WqHOcZKWSVot6er8c5lI2kfSfEmr8t/hnVhRMzMr184RwnXAlGJBRJwTERMiYgJwM/C9Qvjh7lhEXFwovwaYAYzPU3ebM4EFETEeWJCfm5lZP6tMCBHxM2BTs1j+L/99wA1lbUg6ENgzIu7KP7F5PXBWDk8F5uTHcwrlZmbWj+peQzgRWB8Rqwpl4yQtlvRTSSfmslFAV2GerlwGsH9ErAPIf/er2SczM+uFoTXrT2P7o4N1wJiIeFrSccD3JR0JqEnd6OnCJM0gnXZizJgxveiumZm10usjBElDgfcA3+kui4iXIuLp/Phe4GHgENIRwehC9dHA2vx4fT6l1H1qaUOrZUbErIiYGBETR44c2duum5lZE3VOGb0TeCAiXjkVJGmkpCH58ZtIF4/X5FNBmyVNztcdzgduzdXmAtPz4+mFcjMz60ft3HZ6A3AXcKikLkkX5tC57Hgx+U+BpZJ+BfwrcHFEdF+QvgT4BrCadORwRy6/CjhF0irglPzczMz6WeU1hIiY1qL8giZlN5NuQ202/yLgqCblTwMnV/XDzMw6y59UNjMzwAnBzMwyJwQzMwOcEMzMLHNCMDMzwAnBzMwyJwQzMwOcEMzMLHNCMDMzwAnBzMwyJwQzMwOcEMzMLHNCMDMzwAnBzMwyJwQzMwOcEMzMLHNCMDMzoL2f0JwtaYOk5YWyz0h6UtKSPJ1RiH1S0mpJD0o6rVA+JZetljSzUD5O0i8lrZL0HUm79eUKmplZe9o5QrgOmNKk/EsRMSFPtwNIOoL0W8tH5jpflTRE0hDgK8DpwBHAtDwvwOdzW+OBZ4ALGxdkZmadV5kQIuJnwKY225sK3BgRL0XEI8BqYFKeVkfEmoj4HXAjMFWSgHcA/5rrzwHO6uE6mJlZH6hzDeEySUvzKaXhuWwU8ERhnq5c1qp8X+DZiNjSUG5mZv2stwnhGuBgYAKwDvhCLleTeaMX5U1JmiFpkaRFGzdu7FmPzcysVK8SQkSsj4itEfEy8HXSKSFI/+EfVJh1NLC2pPwpYG9JQxvKWy13VkRMjIiJI0eO7E3XzcyshV4lBEkHFp6+G+i+A2kucK6k10kaB4wH7gEWAuPzHUW7kS48z42IAO4E3pvrTwdu7U2fzMysnqFVM0i6ATgJGCGpC7gcOEnSBNLpnUeBiwAiYoWkm4D7gS3ApRGxNbdzGTAPGALMjogVeRGfAG6U9PfAYuDaPls7MzNrW2VCiIhpTYpbvmlHxJXAlU3Kbwdub1K+hm2nnMzMbID4k8pmZgY4IZiZWeaEYGZmgBOCmZllTghmZgY4IZiZWeaEYGZmgBOCmZllTghmZgY4IZiZWVb51RU7u7Ezb9vu+aNXvWuAemJmNrj5CMHMzAAnBDMzy5wQzMwMcEIwM7PMCcHMzAAnBDMzyyoTgqTZkjZIWl4o+wdJD0haKukWSXvn8rGSXpC0JE9fK9Q5TtIySaslXS1JuXwfSfMlrcp/h3diRc3MrFw7RwjXAVMayuYDR0XE0cBDwCcLsYcjYkKeLi6UXwPMAMbnqbvNmcCCiBgPLMjPzcysn1UmhIj4GbCpoexHEbElP70bGF3WhqQDgT0j4q6ICOB64KwcngrMyY/nFMrNzKwf9cU1hA8AdxSej5O0WNJPJZ2Yy0YBXYV5unIZwP4RsQ4g/92vD/pkZmY9VOurKyT9d2AL8K1ctA4YExFPSzoO+L6kIwE1qR69WN4M0mknxowZ07tOm5lZU71OCJKmA38GnJxPAxERLwEv5cf3SnoYOIR0RFA8rTQaWJsfr5d0YESsy6eWNrRaZkTMAmYBTJw4sccJpRl/15GZWdKrU0aSpgCfAM6MiOcL5SMlDcmP30S6eLwmnwraLGlyvrvofODWXG0uMD0/nl4oNzOzflR5hCDpBuAkYISkLuBy0l1FrwPm57tH7853FP0p8FlJW4CtwMUR0X1B+hLSHUu7k645dF93uAq4SdKFwOPA2X2yZn3ERxBm9lpRmRAiYlqT4mtbzHszcHOL2CLgqCblTwMnV/XDzMw6y59UNjMzwAnBzMwyJwQzMwOcEMzMLHNCMDMzwAnBzMwyJwQzMwOcEMzMLHNCMDMzwAnBzMyyWl9/bf6uIzN79fARgpmZAU4IZmaWOSGYmRnghGBmZpkTgpmZAU4IZmaWtZUQJM2WtEHS8kLZPpLmS1qV/w7P5ZJ0taTVkpZKOrZQZ3qef5Wk6YXy4yQty3Wuzr+7bGZm/ajdI4TrgCkNZTOBBRExHliQnwOcDozP0wzgGkgJhPR7zMcDk4DLu5NInmdGoV7jsszMrMPaSggR8TNgU0PxVGBOfjwHOKtQfn0kdwN7SzoQOA2YHxGbIuIZYD4wJcf2jIi7IiKA6wttmZlZP6lzDWH/iFgHkP/ul8tHAU8U5uvKZWXlXU3KzcysH3XionKz8//Ri/IdG5ZmSFokadHGjRtrdNHMzBrVSQjr8+ke8t8NubwLOKgw32hgbUX56CblO4iIWRExMSImjhw5skbXzcysUZ2EMBfovlNoOnBrofz8fLfRZOA3+ZTSPOBUScPzxeRTgXk5tlnS5Hx30fmFtszMrJ+09W2nkm4ATgJGSOoi3S10FXCTpAuBx4Gz8+y3A2cAq4HngfcDRMQmSVcAC/N8n42I7gvVl5DuZNoduCNPZmbWj9pKCBExrUXo5CbzBnBpi3ZmA7OblC8CjmqnL2Zm1hn+pLKZmQFOCGZmljkhmJkZ4IRgZmaZE4KZmQFOCGZmljkhmJkZ4IRgZmaZE4KZmQFOCGZmljkhmJkZ4IRgZmaZE4KZmQFOCGZmljkhmJkZ4IRgZmaZE4KZmQE1EoKkQyUtKUzPSfqopM9IerJQfkahziclrZb0oKTTCuVTctlqSTPrrpSZmfVcWz+h2UxEPAhMAJA0BHgSuIX0G8pfioh/LM4v6QjgXOBI4I3AjyUdksNfAU4BuoCFkuZGxP297ZuZmfVcrxNCg5OBhyPiMUmt5pkK3BgRLwGPSFoNTMqx1RGxBkDSjXleJwQzs37UV9cQzgVuKDy/TNJSSbMlDc9lo4AnCvN05bJW5WZm1o9qJwRJuwFnAt/NRdcAB5NOJ60DvtA9a5PqUVLebFkzJC2StGjjxo21+m1mZtvriyOE04H7ImI9QESsj4itEfEy8HW2nRbqAg4q1BsNrC0p30FEzIqIiRExceTIkX3QdTMz69YXCWEahdNFkg4sxN4NLM+P5wLnSnqdpHHAeOAeYCEwXtK4fLRxbp7XzMz6Ua2LypL2IN0ddFGh+H9LmkA67fNodywiVki6iXSxeAtwaURsze1cBswDhgCzI2JFnX6ZmVnP1UoIEfE8sG9D2Xkl818JXNmk/Hbg9jp9MTOzevxJZTMzA5wQzMwsc0IwMzPACcHMzDInBDMzA5wQzMwsc0IwMzPACcHMzDInBDMzA5wQzMwsc0IwMzPACcHMzDInBDMzA5wQzMwsc0IwMzPACcHMzDInBDMzA/ogIUh6VNIySUskLcpl+0iaL2lV/js8l0vS1ZJWS1oq6dhCO9Pz/KskTa/bLzMz65m+OkJ4e0RMiIiJ+flMYEFEjAcW5OcApwPj8zQDuAZSAgEuB44HJgGXdycRMzPrH506ZTQVmJMfzwHOKpRfH8ndwN6SDgROA+ZHxKaIeAaYD0zpUN/MzKyJvkgIAfxI0r2SZuSy/SNiHUD+u18uHwU8UajblctalZuZWT8Z2gdtnBARayXtB8yX9EDJvGpSFiXl21dOCWcGwJgxY3rTVzMza6H2EUJErM1/NwC3kK4BrM+ngsh/N+TZu4CDCtVHA2tLyhuXNSsiJkbExJEjR9btupmZFdRKCJL+QNIbuh8DpwLLgblA951C04Fb8+O5wPn5bqPJwG/yKaV5wKmShueLyafmMjMz6yd1TxntD9wiqbutb0fEv0laCNwk6ULgceDsPP/twBnAauB54P0AEbFJ0hXAwjzfZyNiU82+mZlZD9RKCBGxBjimSfnTwMlNygO4tEVbs4HZdfpjZma9508qm5kZ4IRgZmaZE4KZmQFOCGZmljkhmJkZ4IRgZmaZE4KZmQFOCGZmljkhmJkZ4IRgZmaZE4KZmQFOCGZmljkhmJkZ0De/mGYlxs68bbvnj171rgHqiZlZOR8hmJkZ4IRgZmaZE4KZmQE1EoKkgyTdKWmlpBWSPpLLPyPpSUlL8nRGoc4nJa2W9KCk0wrlU3LZakkz662SmZn1Rp2LyluAj0XEfZLeANwraX6OfSki/rE4s6QjgHOBI4E3Aj+WdEgOfwU4BegCFkqaGxH31+ibmZn1UK8TQkSsA9blx5slrQRGlVSZCtwYES8Bj0haDUzKsdX595mRdGOe9zWREHwXkpkNFn1yDUHSWOAtwC9z0WWSlkqaLWl4LhsFPFGo1pXLWpWbmVk/qv05BEnDgJuBj0bEc5KuAa4AIv/9AvABQE2qB82TUrRY1gxgBsCYMWPqdn2nUHUE4SMMM+srtRKCpF1JyeBbEfE9gIhYX4h/HfhhftoFHFSoPhpYmx+3Kt9ORMwCZgFMnDixadKw7dVNKMW4k5HZq1uvE4IkAdcCKyPii4XyA/P1BYB3A8vz47nAtyV9kXRReTxwD+nIYbykccCTpAvPf9nbftng4YRhtnOpc4RwAnAesEzSklz2KWCapAmk0z6PAhcBRMQKSTeRLhZvAS6NiK0Aki4D5gFDgNkRsaJGv8zMrBfq3GX0c5pfF7i9pM6VwJVNym8vq2dmZp3nTyqbmRnghGBmZpkTgpmZAU4IZmaWOSGYmRngX0yzAeTPKZgNLj5CMDMzwAnBzMwyJwQzMwOcEMzMLHNCMDMzwHcZ2SDmu5DM+pcTgu20nDDM+pZPGZmZGeAjBHsV8xGEWc/4CMHMzAAfIdhrWN3fmzZ7tRk0CUHSFOCfSD+j+Y2IuGqAu2RWygnDXm0GRUKQNAT4CnAK0AUslDQ3Iu4f2J6Z9V7dI5CdPW47n0GREIBJwOqIWAMg6UZgKuCEYLaTGuiENNDxnZEiYqD7gKT3AlMi4oP5+XnA8RFxWcN8M4AZ+emhwIOF8AjgqZLFON77+GDum+OOO96z+B9GxMimc0bEgE/A2aTrBt3PzwO+3MM2Fjnemfhg7pvjjjteL16cBsttp13AQYXno4G1A9QXM7PXpMGSEBYC4yWNk7QbcC4wd4D7ZGb2mjIoLipHxBZJlwHzSLedzo6IFT1sZpbjHYsP5r457rjj9eKvGBQXlc3MbOANllNGZmY2wJwQzMwMcEIwM7PMCcHMzIBBcpdRT0naC5gCjAKC9JmFeRHxbI6L9HUYxfg9ka+gt1H/MNJXZxTjcyNiZX/Ub1jXYcAhwJoW8XHAW4D7I+KBNvvX9vIblnVKRMyvWr+S+u+PiH9po/26229PYGREPNzQ/tERsbTG+h/WPcaFsj4f/7r7V0P/3prHcnlE/CiXHQAQEb+WNBI4EXiw8c6+VvteWf/a2Pcr+163jXbHp9m2azJ+283T032ncfxrvHa6Xxu13vuq7HR3GUk6H7gc+BHwZC4eTfpivP8J/Br4KrCqIf5m4EPAARX1DwSmATeSPjDXHT83l63tcP3JEfGhvK5vBb4NPJz7fxEwIyLOyvGpwP8B/h34E+B/AS9XtE9ZPCKupwVJj5O+hLDl+kXJt9RKejwixlS0/0Hqbb8XSWOyAdgVuCAiFub278uxOut/XyfHv4141f71noiYlPv3X4FLgVuAU4EfAM8AMwEBnwcuAFYAJwBPRsQ7ct0d9r2IuL3i9beI9JUyrfoWZX2PiKskfaJi/UrbqIjvGhFH5vXbYdtFxHWSvl+yfX8KnNxq20TE9ZLuKRn/5/L269VrB/h0ydhXvvd1/0NQqt2PNA+WifT9RXs3KR8OPASsBMY2iY/Lsar6D5F2nMb4bnmgO13/vkLZncCx+fGbSC+4xYX4L4Bx+fEI4FdttF8Vn9ti+gHwn22s39IW0zLgpTbar7v9lgAH5rJJwAOkN0mAxW3Uv7rF9GXSC7rT4193/yr2byHpSAngD/I2WAbsAewL/BY4oND+82X7Xhuvv99V9K207/lx1frVib9Ytu2695GSeV4s2zZN6jeO/0sVfa96bdR672ssbzbtjKeMRPovoNHLOTaUbdm36EnSf4wvV9TfCrwReKwhfmBhnk7WL9ozIu4DiIg1+WvCi3WHRsQjOf6UpHbbL4ufCPwV6c2iqPtQ9OWK9dsfOI30n2hj/V+00X7d7TckItYBRMQ9kt4O/FDS6FyvanzeD3yM9OJtNI3Oj39VvGr/2kXScNL1QUXExty//5S0JT2M54HnJT0cEb/O8WfS2YZXNNv3qFg/KvoWFfHuduq0URYvarbtoHz7Nsa7Fbdd2fhX9b3qtVH3va/SzpgQrgTuk/Qj4IlcNoZ02HQFaXAX5q/Q7o4fRDosu5Z0yqas/q+BBZJWNcTfDFxGesPrZP2vSlpK2sBjJQ3PL9ZdSBv1cEnP5fjrJB0Q6VzwbqRPeVeNT1TEzyH9p/jTxoGX9CDwuYr1OxsYFhFLmtT/d7b9J9qq/TvYcfuNyf1qZ/vNkHRw5OsHEbFO0knA94EjSd+WW1b/fNL53l806d9ngGM6PP5V8ar962vAvbl/UejfsFy2VdKuEfF74JXva5b0euD1FfseFev35Yq+0Ub8o33QRqv4rhXbDsq378aKbQOwV8n4P1nR949Q/tq4tmL5zd77iq+dSjvdNQSAnIFPI104ESkrzouIZ3L8CODMhvgrP7jTRv1d2HZhpju+MCK2drq+pD9sWN11EfE7SSOAP42I77UYk72BwyPirjb6VxpvY/xL168uSYez7cJbj7afpGNIL6pVDW3uCrwvIr5VUX8f0qmF53vY5z4b/7r7V4v+7UH6Z2QraZ/6fUN8FPA24P8VipvuexXjV7XvV/a9bhs9HZ/itisZv72Bw0mnIHv82imM/2M96VuTdmq991W2vzMmhIEiaVhENB7O9UW7+0TEpr5ut2KZ+1O4EyEi1vfn8ntC0rHdpy8KZYOm/1Xbr434mRExt/B8aERsyY+HAYeR7vRp2kZP95+etN9s7HN5n45/O6+tZvNIejNwDOkceembXrF+fmPdEhGb2+jbDuPbqr6kvaP1HUctY4V5erTt2+j7iIgo+62E7bVzoWEwTcAm4Bukq/1qEj+MdNrhNuBg4DrgWeAe0n8BZW0vq4g/Tjr9dCPwH8CnKFwkIp2WOBq4m3TINgsYXojfQ7qbYyXpzo7jgfnAmjz/H1f1v43lV8Un5P6tzMv+Mem/nruBY/tg/ariVe0f22TqIt36d2zd/ldt/zbGv2r7VcXf02T6deHxBcDTpIuEp+e6C3L9acCnC/09Is/3CPBoXl7V+Je1/8mGcT+uOPa5jbLxP6ds2VWvrTZe+4+TLnaPyM/Py+vxjbztPlxR/0ngeuA3pCOlx/P0me79pGL7/Xkb9bfkMbmQhgvAZbEcr9r2HyjMOyrHniFdmzsk13kE+HneZitId4l1ASe39f460G/wPZ1IV9ovIx3aPgn8E+lWze74z/KGm0Y6PDuXdOj053kAm70g3wP8Bekc4d+0mD5GSkbzgYvzC+PLeWPsm5e9OG+MKcDewMfzRjm4EL8H+C+kN4engLfm2LF5nar6X7X8qvgS0q/RNY7rZNJdMnXXrype1f7LuezOwvRC/vuTPuh/1favGv+q7VcV3wL8EJgN/EueNue/s0lvbCNId4Y8Vxi7/Ul3axXvQrsNOD0/npTXtWr8y9qPsrHP85WN/28rll362srzVb3+lheWubCwbffI41NW//fASXn+9wBfIt398/fArO7EVbL9nm2j/jLgz4Bvkd7cbyXtQ7uXxQp12932N5FuQ98FeDdp31xC+qflj3P7k/O8hxfrvtoSQnFQxgB/B9xHyqafY/vbvlY31s07xXVsezEWp82kW8uuIN3v2zg9CyxpaPOvyDt+br8x/nbSLWWTc7zYv5VN+lfV/54uvzG+qmRsV/fB+vU03tj+e0n3e59RmOeRwuO6/a/a/lXj35Pt1yz+R6QX7yVsO2VbXL8lhcdrG+o3viksbogv7sn4N2n/sbKxb2P8X6pYdulrK9epev0tBkblee8EXp8fD8nbuaz+1ob+3Vt4/EDjmDbZfs+3Ub+4fXYH3gd8j/QGvakk9u0ebvvG7by4If5EQ3y7+Vtuw3ZmGkxT44ugUH5o3uhLC2UfaphnOekOgKNatPEE6T+k40riK7p3wkL5O0lvRutI/6Xu1RA/Or8wnibf75zLz2rSv6r+Vy2/Kn416T/Lc0gftvmT/Pg24J/7Yv0q4qXt5+fDSP99fZeU9NcU5q3b/6rtXzX+VduvNJ7/7kK6o+RO0n/2xfWbS/qA2z+Tjoi+QDqNcTnp90KeZdu96RuBPRqXXzH+Ve23HPs2xv+pimWXvrby36rX30l5G382r8MvgP9BOjL8eEX9F0n/ILwR+DBwcy4X2z5HULb9fttG/VbvT3sBj5XEprexbTaw7TMxT7L96dDluc5FwN+SEshfk04tTQd+3mzZO/SlnZkG0wR8sSJ+Eem2x8byN5M+dXgiMKZF3YmkxDKiRXz/PMhvaxJ7S94p/5LCKaxCfAzwddIdAHs0iR9MOtqp6n/V8kvj+fHppNsTf0A6ffE18n+FfbB+VfHK/jWU3QlsbCiv0/+q7V81/lXbrzTeUDaKdOhfTAh7ks7lzyS9Ob83r+NXSbcVvq1hekNh37y0jfEvbb9q7MvGv41ll7628t9DyR/mKplnL9IR1pdIb46fAA5ro/6xebyXA99k2wcY9wX+Ij8u236fa6P+x5stuyrW5raf3jANz/UOyH07CPi/eXscQHotLCcl69Lrp92T7zKyQU3p00BviIjnBrovrzUe+9egdrLGYJpIH6a7CPg30mHRr0h3hVxM+vBMrXjFsme10f5ewFWkOy+eztPKXLZ3G/GOrl9h+St7s/yq8emD8ev09qkbH5LjVwAnNCz/023Ee7J/bOrF/lOn/X2rxr5i/9mnbN3b2Xf6eP0a493rd0cb69eJ+mNqbvuhZcsviV9ExWtnpz1CkHQD6TzqHLb/gqjppB3y5ZrxS1otmjTA/1FRf2/Subw5kb8WQOnbJaeTzmW/XBF/qsPr16p/F5Bu5a1aftX4HF1z/Dq9feqO32bSHS33kG57/GlE/A3Q/eV591XEN9K7/eMC0vap2n96G7+A9CnhW1qte0ScI2leSf0Pky5Kt1r3dzbdcnnbRcTokvb7Ynz+usb69UX9vwX+oca61XpviIhzWoz/tg2xEyaEByPi0Baxh0jf1VInfjDpboviF7tEfj6KdNdFb9t/kDRDb+N9sX51+tfO+AypiNcZv05vn3biL0bE0fn5UNL53RGk21TvJn2XUll8jw7uH3Xjv4uI3VrEHoqIQypefy9GxOvz42brfjQl2y4idqtof6DXr5P1O/7eEBGHNIsV7Yw/kPOMpLPzx9OB9FF1SeeQPqRRN76GdK/xuML0pogYB6xvo/5jkv4uf5KzO76/0tf6PtFGvNPrV3f5VeNTd/w6vX3qxl95QUfEloiYQToy+QnpQmBVvO7+0cn4CxXrTkX9V75+ocW6V227qvYHev3q1t88wO8N1do5rzSYJmAs8B3SofdDpFvaNuSycT2Ib2Db1wkX45cCx7RY9ofbaH846XvmH8gbYRPpPODnSYd1VfFOr1+7y+/t+NQdv6p4p9uvin8TmNJk2R8kfcahKl53/+hk/Jiydc/rUVb/pop1L912bbQ/0OtXt/64mus2tmz5VfG23l8H+g2+ZmSXsfYAAALDSURBVHLYl3RI+s2exEkfSZ+U428l3b9c/DDOJOCP8uMjSJ90PKMH7R9GOic4rKF8Sjvxfli/qv7VGp+649duvDDf9X25f/TV8lvF6+4fnY73cN1PJH0K+NS+GJt22u/0+nWyfl/0vS/33cZpZ7yGMLdJ8TtIh6WtFOP3ku6jHkq6L30S6ULYO0kf/hjaED+e9ItJ3fHjK9r/Mek/oZWkr0/4SETcmvt+H+lTsmXxLnbUl+u3uWL5t9Ycn7rjV7V+jUT6RGxv63d6+Y3xuvtHJ+PPkr66o+m6R8SZ2v4XwT6Y2/o+6RfBhpM+ANirsWmj/R8Az3d4/f5bB+s/kfs/IO8NEXFmyXxJT7LHYJhId3F8k/SJxbflv+vY9kGdqvgy0oXPPUjfF7Jnbnd3tv2yV1m8nfaH5TpjSb9y9pH8fHEb8f5Yv6r+1RmfuuNXFV9cs36nl18Vr7t/dDL+fFnfu9sovBYbfxHshTpj00b73b/41sn162T9F2puu1r7blvvrwP9Bt+LhLAL6fav+cCEXLamB/HFzR7n50vaiFe1f39DnWGk+7q/mOtXxTu9flXLrzs+dcfv1R6vu390Ot6y7/n5r0hHAvuSf1azuL3rjE2b7Xd6/TpZ/8Wafa+177UzDfgbfG8n0v213yV978cOX53bKg78kvzRdGCXQvlepAxbGm+j/Z90b4xC2VDS1+ZurYr3w/pV9a/W+NQdv1d7vO7+0el41bqRvmZ7Delrltew7TeZh5G/QK3O2FW13w/r18n6UbfvfbFvlk0D/sZedyL9DODn2o0Dr2sx3wjS196Wxttof3T3Ttxk3hOq4v2wflX9qzU+dcfv1R6vu390Ot6TdWuYdw8a7mSpO3bN2u/0+nW4/lkD+d7QzrTTXVQ2M7PO2Bk/mGZmZh3ghGBmZoATgpmZZU4IZmYGOCGYmVn2/wG16kYbo+a0OAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "extracted_event_data[\"event_code\"].value_counts().plot.bar()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Join the labels to the training set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_labels = pd.read_csv(\"../DSB2019/data-science-bowl-2019/train_labels.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Code below based on code original written/published by Gabriel Preda\n",
    "# on Kaggle, accessed Mon, 18 Nov, 2019\n",
    "# https://www.kaggle.com/gpreda/2019-data-science-bowl-eda\n",
    "\n",
    "numerical_columns = ['game_time']\n",
    "categorical_columns = ['type', 'world']\n",
    "\n",
    "comp_train_sample = pd.DataFrame({'installation_id': train_sample['installation_id'].unique()})\n",
    "comp_train_sample.set_index('installation_id', inplace = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_numeric_columns(df, column):\n",
    "    df = df.groupby('installation_id').agg({f'{column}': ['mean', 'sum', 'min', 'max', 'std', 'skew']})\n",
    "    df[column].fillna(df[column].mean(), inplace = True)\n",
    "    df.columns = [f'{column}_mean', f'{column}_sum', f'{column}_min', f'{column}_max', f'{column}_std', f'{column}_skew']\n",
    "    return df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/maxwell/anaconda3/lib/python3.7/site-packages/pandas/core/generic.py:6287: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  self._update_inplace(new_data)\n"
     ]
    }
   ],
   "source": [
    "for i in numerical_columns:\n",
    "    comp_train_sample = comp_train_sample.merge(get_numeric_columns(train_sample, i), left_index = True, right_index = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "We have 17690 training rows\n"
     ]
    }
   ],
   "source": [
    "# get the mode of the title\n",
    "labels_map = dict(train_labels.groupby('title')['accuracy_group'].agg(lambda x:x.value_counts().index[0]))\n",
    "# merge target\n",
    "labels = train_labels.loc[:, ['installation_id', 'title', 'accuracy_group']]\n",
    "# replace title with the mode\n",
    "labels['title'] = labels['title'].map(labels_map)\n",
    "# join train with labels\n",
    "comp_train_sample = labels.merge(comp_train_sample, on = 'installation_id', how = 'left')\n",
    "print('We have {} training rows'.format(comp_train_sample.shape[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This gives us the number of rows we can actually use in a future model.\n",
    "\n",
    "## Let's establish a naive baseline model to work from"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Code below based on code original written/published by Massoud Hosseinali\n",
    "# on Kaggle, accessed 20 Nov, 2019\n",
    "# https://www.kaggle.com/mhviraf/a-baseline-for-dsb-2019\n",
    "submission = pd.read_csv(\"../DSB2019/data-science-bowl-2019/sample_submission.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Bird Measurer (Assessment)': 0,\n",
       " 'Cart Balancer (Assessment)': 3,\n",
       " 'Cauldron Filler (Assessment)': 3,\n",
       " 'Chest Sorter (Assessment)': 0,\n",
       " 'Mushroom Sorter (Assessment)': 3}"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "labels_map = dict(train_labels.groupby('title')['accuracy_group'].agg(lambda x:x.value_counts().index[0])) # get the mode\n",
    "labels_map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1000, 2)"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>installation_id</th>\n",
       "      <th>accuracy_group</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>00abaee7</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>01242218</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>017c5718</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>01a44906</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>01bc6cb6</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  installation_id  accuracy_group\n",
       "0        00abaee7             NaN\n",
       "1        01242218             NaN\n",
       "2        017c5718             NaN\n",
       "3        01a44906             NaN\n",
       "4        01bc6cb6             NaN"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "submission['accuracy_group'] = test.groupby('installation_id').last()['title'].map(labels_map).reset_index(drop=True)\n",
    "submission.to_csv('submission.csv', index=None)\n",
    "display(submission.shape)\n",
    "submission.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7ff76855e9d0>"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAD4CAYAAADCb7BPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAR4UlEQVR4nO3df7DldV3H8edLVkHS+I0Sy7oYW7baD/EINf0yEURmYLWosJy2hqTRqFGrEbMCsWbULMrJ0vXHhDQKSpNuqbMDqNk0gdwNS5YkVvzBCiOrSxihEPjuj/NdulzPcr/7ufeccw/3+Zg5c7+fz/dz7vf92Xt3X/v9cb7fVBWSJO2vx0y7AEnSbDJAJElNDBBJUhMDRJLUxACRJDVZM+0CJunII4+s9evXT7sMSZop27dv/2pVHbWwf1UFyPr165mbm5t2GZI0U5J8cVS/h7AkSU0MEElSEwNEktTEAJEkNTFAJElNDBBJUhMDRJLUxACRJDUxQCRJTQwQSVITA0SS1MQAkSQ1MUAkSU0MEElSEwNEktTEAJEkNTFAJElNDBBJUhMDRJLUxACRJDUxQCRJTQwQSVITA0SS1MQAkSQ1MUAkSU2mGiBJTk9yc5KdSS4Ysf7AJFd0669Lsn7B+nVJ7kny25OqWZI0NLUASXIA8FbgBcBG4MVJNi4Ydi5wV1WdAFwCvHHB+kuAj467VknSt5vmHshJwM6qurWq7gcuBzYtGLMJuLRbvhI4JUkAkrwQuBXYMaF6JUnzTDNAjgVum9fe1fWNHFNVDwB3A0ck+Q7g1cDrFttIkvOSzCWZ271797IULkmaboBkRF/1HPM64JKqumexjVTVlqoaVNXgqKOOaihTkjTKmiluexdw3Lz2WuD2fYzZlWQNcAiwBzgZODvJm4BDgW8l+WZV/cX4y5YkwXQD5HpgQ5LjgS8D5wC/sGDMVmAz8C/A2cDHqqqAH987IMlFwD2GhyRN1tQCpKoeSHI+sA04AHh3Ve1IcjEwV1VbgXcBlyXZyXDP45xp1StJergM/0O/OgwGg5qbm5t2GZI0U5Jsr6rBwn4/iS5JamKASJKaGCCSpCYGiCSpiQEiSWpigEiSmhggkqQmBogkqYkBIklqYoBIkpoYIJKkJgaIJKmJASJJamKASJKaGCCSpCYGiCSpiQEiSWpigEiSmhggkqQmBogkqYkBIklqYoBIkpoYIJKkJgaIJKmJASJJamKASJKaGCCSpCYGiCSpiQEiSWpigEiSmkw1QJKcnuTmJDuTXDBi/YFJrujWX5dkfdd/apLtST7TfX3upGuXpNVuagGS5ADgrcALgI3Ai5NsXDDsXOCuqjoBuAR4Y9f/VeDMqvp+YDNw2WSqliTtNc09kJOAnVV1a1XdD1wObFowZhNwabd8JXBKklTVDVV1e9e/AzgoyYETqVqSBEw3QI4FbpvX3tX1jRxTVQ8AdwNHLBjzM8ANVXXfmOqUJI2wZorbzoi+2p8xSZ7O8LDWafvcSHIecB7AunXr9r9KSdJI09wD2QUcN6+9Frh9X2OSrAEOAfZ07bXA3wG/VFWf29dGqmpLVQ2qanDUUUctY/mStLpNM0CuBzYkOT7J44BzgK0LxmxleJIc4GzgY1VVSQ4FPgy8pqr+eWIVS5IeMrUA6c5pnA9sA/4DeH9V7UhycZKzumHvAo5IshN4FbD3Ut/zgROA30/y6e519ISnIEmrWqoWnnZ49BoMBjU3NzftMiRppiTZXlWDhf1+El2S1MQAkSQ1MUAkSU0MEElSEwNEktTEAJEkNTFAJElNDBBJUhMDRJLUxACRJDXpFSBJnjHuQiRJs6XvHsjbknwqycu7O+FKkla5XgFSVT8G/CLDZ3PMJXlvklPHWpkkaUXrfQ6kqm4Bfg94NfCTwFuSfDbJT4+rOEnSytX3HMgPJLmE4XM7ngucWVXf1y1fMsb6JEkrVN9nov8F8A7gd6vqG3s7q+r2JL83lsokSSta3wA5A/hGVT0IkOQxwEFVdW9VXTa26iRJK1bfcyBXA4+f1z6465MkrVJ9A+Sgqrpnb6NbPng8JUmSZkHfAPmfJCfubSR5FvCNRxgvSXqU63sO5BXAB5Lc3rWPAX5+PCVJkmZBrwCpquuTPA34XiDAZ6vqf8damSRpReu7BwLwbGB9955nJqGq3jOWqiRJK16vAElyGfDdwKeBB7vuAgwQSVql+u6BDICNVVXjLEaSNDv6XoV1I/DkcRYiSZotffdAjgRuSvIp4L69nVV11liqkiSteH0D5KJxFiFJmj19L+P9xyRPATZU1dVJDgYOGG9pkqSVrO/t3F8KXAm8ves6FvjguIqSJK18fU+i/zrwo8DX4aGHSx09rqIkSStf3wC5r6ru39tIsobh50CWJMnpSW5OsjPJBSPWH5jkim79dUnWz1v3mq7/5iTPX2otkqT90zdA/jHJ7wKP756F/gHg75ey4SQHAG8FXgBsBF6cZOOCYecCd1XVCQyffPjG7r0bgXOApwOnA3/ZfT9J0oT0DZALgN3AZ4BfAz7C8PnoS3ESsLOqbu32bi4HNi0Yswm4tFu+EjglSbr+y6vqvqr6PLCz+36SpAnpexXWtxg+0vYdy7jtY4Hb5rV3ASfva0xVPZDkbuCIrv/aBe89dtRGkpwHnAewbt26ZSlcktT/XlifZ8Q5j6p66hK2nRF9C7exrzF93jvsrNoCbAEYDAbeikWSlsn+3Atrr4OAnwUOX+K2dwHHzWuvBW7fx5hd3Yn7Q4A9Pd8rSRqjXudAqupr815frqo/A567xG1fD2xIcnySxzE8Kb51wZitwOZu+WzgY90NHbcC53RXaR0PbAA+tcR6JEn7oe8hrBPnNR/DcI/kiUvZcHdO43xgG8NPtb+7qnYkuRiYq6qtwLuAy5LsZLjncU733h1J3g/cBDwA/HpVPThyQ5KksUifO7Qn+fi85gPAF4A3V9XNY6prLAaDQc3NzU27DEmaKUm2V9VgYX/fq7B+avlLkiTNsr6HsF71SOur6k+XpxxJ0qzYn6uwns3/n+Q+E/gkD/8chyRpFdmfB0qdWFX/DZDkIuADVfWr4ypMkrSy9b2VyTrg/nnt+4H1y16NJGlm9N0DuQz4VJK/Y/iJ7xcB7xlbVZKkFa/vVVh/lOSjwI93Xb9SVTeMryxJ0krX9xAWwMHA16vqzxneWuT4MdUkSZoBfR9peyHwauA1Xddjgb8ZV1GSpJWv7x7Ii4CzgP8BqKrbWeKtTCRJs61vgNzf3cSwAJJ8x/hKkiTNgr4B8v4kbwcOTfJS4GqW9+FSkqQZ0/cqrDd3z0L/OvC9wB9U1VVjrUyStKItGiBJDgC2VdXzAENDkgT0OITVPWfj3iSHTKAeSdKM6PtJ9G8Cn0lyFd2VWABV9ZtjqUqStOL1DZAPdy9JkoBFAiTJuqr6UlVdOqmCJEmzYbFzIB/cu5Dkb8dciyRphiwWIJm3/NRxFiJJmi2LBUjtY1mStMotdhL9B5N8neGeyOO7Zbp2VdV3jrU6SdKK9YgBUlUHTKoQSdJs2Z/ngUiS9BADRJLUxACRJDUxQCRJTQwQSVITA0SS1MQAkSQ1mUqAJDk8yVVJbum+HraPcZu7Mbck2dz1HZzkw0k+m2RHkjdMtnpJEkxvD+QC4Jqq2gBc07UfJsnhwIXAycBJwIXzgubNVfU04JnAjyZ5wWTKliTtNa0A2QTsvUX8pcALR4x5PnBVVe2pqrsYPk739Kq6t6o+DlBV9wP/CqydQM2SpHmmFSBPqqo7ALqvR48Ycyxw27z2rq7vIUkOBc5kuBcjSZqgvk8k3G9JrgaePGLVa/t+ixF9D90ROMka4H3AW6rq1keo4zzgPIB169b13LQkaTFjC5Cqet6+1iX5SpJjquqOJMcAd44Ytgt4zrz2WuAT89pbgFuq6s8WqWNLN5bBYOAt6SVpmUzrENZWYHO3vBn40Igx24DTkhzWnTw/resjyR8ChwCvmECtkqQRphUgbwBOTXILcGrXJskgyTsBqmoP8Hrg+u51cVXtSbKW4WGwjcC/Jvl0kl+dxiQkaTVL1eo5qjMYDGpubm7aZUjSTEmyvaoGC/v9JLokqYkBIklqYoBIkpoYIJKkJgaIJKmJASJJamKASJKaGCCSpCYGiCSpiQEiSWpigEiSmhggkqQmBogkqYkBIklqYoBIkpoYIJKkJgaIJKmJASJJamKASJKaGCCSpCYGiCSpiQEiSWpigEiSmhggkqQmBogkqYkBIklqYoBIkpoYIJKkJgaIJKmJASJJamKASJKaTCVAkhye5Kokt3RfD9vHuM3dmFuSbB6xfmuSG8dfsSRpoWntgVwAXFNVG4BruvbDJDkcuBA4GTgJuHB+0CT5aeCeyZQrSVpoWgGyCbi0W74UeOGIMc8HrqqqPVV1F3AVcDpAkicArwL+cAK1SpJGmFaAPKmq7gDovh49YsyxwG3z2ru6PoDXA38C3LvYhpKcl2Quydzu3buXVrUk6SFrxvWNk1wNPHnEqtf2/RYj+irJDwEnVNUrk6xf7JtU1RZgC8BgMKie25YkLWJsAVJVz9vXuiRfSXJMVd2R5BjgzhHDdgHPmddeC3wC+BHgWUm+wLD+o5N8oqqegyRpYqZ1CGsrsPeqqs3Ah0aM2QacluSw7uT5acC2qvqrqvquqloP/Bjwn4aHJE3etALkDcCpSW4BTu3aJBkkeSdAVe1heK7j+u51cdcnSVoBUrV6TgsMBoOam5ubdhmSNFOSbK+qwcJ+P4kuSWpigEiSmhggkqQmBogkqYkBIklqYoBIkpoYIJKkJgaIJKmJASJJamKASJKaGCCSpCYGiCSpiQEiSWpigEiSmhggkqQmBogkqYkBIklqYoBIkpoYIJKkJgaIJKmJASJJamKASJKaGCCSpCYGiCSpSapq2jVMTJLdwBcb334k8NVlLGcWOOfVYbXNebXNF5Y+56dU1VELO1dVgCxFkrmqGky7jklyzqvDapvzapsvjG/OHsKSJDUxQCRJTQyQ/rZMu4ApcM6rw2qb82qbL4xpzp4DkSQ1cQ9EktTEAJEkNTFAFkhyepKbk+xMcsGI9QcmuaJbf12S9ZOvcvn0mO+rktyU5N+TXJPkKdOoczktNud5485OUklm/pLPPnNO8nPdz3pHkvdOusbl1uN3e12Sjye5ofv9PmMadS6XJO9OcmeSG/exPkne0v15/HuSE5e80ary1b2AA4DPAU8FHgf8G7BxwZiXA2/rls8Brph23WOe708BB3fLL5vl+fadczfuicAngWuBwbTrnsDPeQNwA3BY1z562nVPYM5bgJd1yxuBL0y77iXO+SeAE4Eb97H+DOCjQIAfBq5b6jbdA3m4k4CdVXVrVd0PXA5sWjBmE3Bpt3wlcEqSTLDG5bTofKvq41V1b9e8Flg74RqXW5+fMcDrgTcB35xkcWPSZ84vBd5aVXcBVNWdE65xufWZcwHf2S0fAtw+wfqWXVV9EtjzCEM2Ae+poWuBQ5Mcs5RtGiAPdyxw27z2rq5v5JiqegC4GzhiItUtvz7zne9chv+DmWWLzjnJM4HjquofJlnYGPX5OX8P8D1J/jnJtUlOn1h149FnzhcBL0myC/gI8BuTKW1q9vfv+6LWLKmcR59RexILr3PuM2ZW9J5LkpcAA+Anx1rR+D3inJM8BrgE+OVJFTQBfX7OaxgexnoOw73Mf0ryjKr6rzHXNi595vxi4K+r6k+S/AhwWTfnb42/vKlY9n+73AN5uF3AcfPaa/n23dqHxiRZw3DX95F2G1eyPvMlyfOA1wJnVdV9E6ptXBab8xOBZwCfSPIFhseKt874ifS+v9cfqqr/rarPAzczDJRZ1WfO5wLvB6iqfwEOYnjTwUerXn/f94cB8nDXAxuSHJ/kcQxPkm9dMGYrsLlbPhv4WHVnqGbQovPtDue8nWF4zPpxcVhkzlV1d1UdWVXrq2o9w/M+Z1XV3HTKXRZ9fq8/yPCCCZIcyfCQ1q0TrXJ59Znzl4BTAJJ8H8MA2T3RKidrK/BL3dVYPwzcXVV3LOUbeghrnqp6IMn5wDaGV3G8u6p2JLkYmKuqrcC7GO7q7mS453HO9Cpemp7z/WPgCcAHumsFvlRVZ02t6CXqOedHlZ5z3gacluQm4EHgd6rqa9Oreml6zvm3gHckeSXDQzm/PMP/GSTJ+xgegjyyO69zIfBYgKp6G8PzPGcAO4F7gV9Z8jZn+M9LkjRFHsKSJDUxQCRJTQwQSVITA0SS1MQAkSQ1MUAkSU0MEElSk/8DX7cHrUCRm7sAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "submission['accuracy_group'].plot(kind='hist')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7ff767b4d590>"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAD4CAYAAAAdIcpQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAASOElEQVR4nO3df7RndV3v8edLBgQ0HZSxaMAO1iwLXXmlCSlb1RVTxGLsXuzOXS0dCeOuG121WivR1YqyuMvWKikrf2DQGsgbInp1Eso1/qrVH4KDkoqjl7nqhQluTA4/TBQaevfH93PwMJyZzx44+3y/Z87zsdZZZ+/P/uzv9/1hH76v2T++e6eqkCTpYB437QIkSbPPsJAkdRkWkqQuw0KS1GVYSJK61ky7gDEcf/zxNTc3N+0yJGlFufHGG/+5qtYttuywDIu5uTl27Ngx7TIkaUVJ8v8OtMzDUJKkLsNCktRlWEiSugwLSVKXYSFJ6jIsJEldhoUkqcuwkCR1GRaSpK7D8hvckjRNcxdeO7X3/uqbXzrK67pnIUnqMiwkSV2GhSSpy7CQJHUZFpKkLsNCktRlWEiSugwLSVKXYSFJ6jIsJEldhoUkqcuwkCR1GRaSpC7DQpLUZVhIkrpGDYskv5Lk5iSfT/KXSY5OcnKS65PckuQ9SY5qfR/f5ne15XMLXucNrf1LSV48Zs2SpEcaLSySrAdeA2ysqmcDRwCbgd8DLqmqDcBdwHltlfOAu6rq+4BLWj+SnNLWexZwJvC2JEeMVbck6ZHGPgy1BjgmyRrgWOAO4AXANW35VuBlbXpTm6ctPyNJWvtVVXV/VX0F2AWcNnLdkqQFRguLqvpH4PeBW5mExD3AjcDdVbWvddsNrG/T64Hb2rr7Wv+nLmxfZJ2HJDk/yY4kO/bs2bP0A5KkVWzMw1DHMdkrOBn4buAJwEsW6Vrzqxxg2YHaH95QdWlVbayqjevWrXt0RUuSFjXmYagXAl+pqj1V9a/A+4EfBda2w1IAJwK3t+ndwEkAbfmTgb0L2xdZR5K0DMYMi1uB05Mc2849nAF8Afg4cE7rswX4YJve1uZpyz9WVdXaN7erpU4GNgA3jFi3JGk/a/pdHp2quj7JNcCngX3AZ4BLgWuBq5L8bmu7rK1yGXBlkl1M9ig2t9e5OcnVTIJmH3BBVT04Vt2SpEcaLSwAquoi4KL9mr/MIlczVdW3gJcf4HUuBi5e8gIlSYP4DW5JUpdhIUnqMiwkSV2GhSSpy7CQJHUZFpKkLsNCktRlWEiSugwLSVKXYSFJ6jIsJEldhoUkqcuwkCR1GRaSpC7DQpLUZVhIkroMC0lSl2EhSeoyLCRJXYaFJKnLsJAkdRkWkqQuw0KS1GVYSJK6DAtJUpdhIUnqMiwkSV2GhSSpy7CQJHUZFpKkLsNCktRlWEiSugwLSVKXYSFJ6jIsJEldhoUkqcuwkCR1GRaSpK5RwyLJ2iTXJPlikp1JfiTJU5JsT3JL+31c65skb02yK8lnk5y64HW2tP63JNkyZs2SpEcae8/ij4C/qarvB54D7AQuBD5aVRuAj7Z5gJcAG9rP+cDbAZI8BbgIeB5wGnDRfMBIkpbHaGGR5EnAjwOXAVTVA1V1N7AJ2Nq6bQVe1qY3AVfUxCeBtUlOAF4MbK+qvVV1F7AdOHOsuiVJjzTmnsUzgD3Anyf5TJI/S/IE4Dur6g6A9vtprf964LYF6+9ubQdqf5gk5yfZkWTHnj17ln40krSKjRkWa4BTgbdX1XOBb/DtQ06LySJtdZD2hzdUXVpVG6tq47p16x5NvZKkAxgzLHYDu6vq+jZ/DZPw+Kd2eIn2+84F/U9asP6JwO0HaZckLZPRwqKq/j9wW5JntqYzgC8A24D5K5q2AB9s09uAV7arok4H7mmHqT4MvCjJce3E9otamyRpmawZ+fX/B/DuJEcBXwbOZRJQVyc5D7gVeHnrex1wFrALuK/1par2Jvkd4FOt35uqau/IdUuSFhg1LKrqJmDjIovOWKRvARcc4HUuBy5f2uokSUP5DW5JUpdhIUnqGhQWSZ49diGSpNk1dM/iHUluSPJLSdaOWpEkaeYMCouq+jHg55l832FHkv+V5KdGrUySNDMGn7OoqluA3wBeD/wE8NZ2N9n/NFZxkqTZMPScxQ8muYTJXWNfAPxMVf1Am75kxPokSTNg6Pcs/gR4F/DGqvrmfGNV3Z7kN0apTJI0M4aGxVnAN6vqQYAkjwOOrqr7qurK0aqTJM2EoecsPgIcs2D+2NYmSVoFhobF0VX1L/MzbfrYcUqSJM2aoWHxjf2eif1DwDcP0l+SdBgZes7idcB7k8w/R+IE4L+MU5IkadYMCouq+lSS7weeyeTJdV+sqn8dtTJJ0sw4lFuU/zAw19Z5bhKq6opRqpIkzZRBYZHkSuB7gZuAB1tzAYaFJK0CQ/csNgKntAcUSZJWmaFXQ30e+K4xC5Ekza6hexbHA19IcgNw/3xjVZ09SlWSpJkyNCx+a8wiJEmzbeils3+b5HuADVX1kSTHAkeMW5okaVYMvUX5LwLXAO9sTeuBD4xVlCRptgw9wX0B8HzgXnjoQUhPG6soSdJsGRoW91fVA/MzSdYw+Z6FJGkVGBoWf5vkjcAx7dnb7wX+aryyJEmzZGhYXAjsAT4H/DfgOibP45YkrQJDr4b6NyaPVX3XuOVIkmbR0HtDfYVFzlFU1TOWvCJJ0sw5lHtDzTsaeDnwlKUvR5I0iwads6iqry34+ceq+kPgBSPXJkmaEUMPQ526YPZxTPY0vmOUiiRJM2foYag/WDC9D/gq8HNLXo0kaSYNvRrqP45diCRpdg09DPWrB1teVW9ZmnIkSbPoUK6G+mFgW5v/GeDvgNvGKEqSNFsO5eFHp1bV1wGS/Bbw3qp69ViFSZJmx9CweDrwwIL5B4C5Ja9mRsxdeO1U3verb37pVN5XknqGhsWVwA1J/jeTb3L/LHDFaFVJkmbK0C/lXQycC9wF3A2cW1X/c8i6SY5I8pkkH2rzJye5PsktSd6T5KjW/vg2v6stn1vwGm9o7V9K8uJDG6Ik6bEaetdZgGOBe6vqj4DdSU4euN5rgZ0L5n8PuKSqNjAJn/Na+3nAXVX1fcAlrR9JTgE2A88CzgTelsRHukrSMhr6WNWLgNcDb2hNRwJ/MWC9E4GXAn/W5sPkNiHXtC5bgZe16U1tnrb8jNZ/E3BVVd1fVV8BdgGnDalbkrQ0hu5Z/CxwNvANgKq6nWG3+/hD4NeBf2vzTwXurqp9bX43k+d5037f1l5/H3BP6/9Q+yLrPCTJ+Ul2JNmxZ8+egcOSJA0xNCweqKqi3aY8yRN6KyT5aeDOqrpxYfMiXauz7GDrfLuh6tKq2lhVG9etW9crT5J0CIZeDXV1kncCa5P8IvAL9B+E9Hzg7CRnMbmt+ZOY7GmsTbKm7T2cCNze+u8GTmJyPmQN8GRg74L2eQvXkSQtg6FXQ/0+k/MI7wOeCfxmVf1xZ503VNWJVTXH5AT1x6rq54GPA+e0bluAD7bpbW2etvxjbW9mG7C5XS11MrABuGHg+CRJS6C7Z9GuPPpwVb0Q2L4E7/l64Kokvwt8BristV8GXJlkF5M9is0AVXVzkquBLzC54+0FVfXgEtQhSRqoGxZV9WCS+5I8uarueTRvUlWfAD7Rpr/MIlczVdW3mDyBb7H1LwYufjTvLUl67Iaes/gW8Lkk22lXRAFU1WtGqUqSNFOGhsW17UeStAodNCySPL2qbq2qrQfrJ0k6vPWuhvrA/ESS941ciyRpRvXCYuEX4p4xZiGSpNnVC4s6wLQkaRXpneB+TpJ7mexhHNOmafNVVU8atTpJ0kw4aFhUlbcClyQd0vMsJEmrlGEhSeoyLCRJXYaFJKnLsJAkdRkWkqQuw0KS1GVYSJK6DAtJUpdhIUnqMiwkSV2GhSSpy7CQJHUZFpKkLsNCktRlWEiSugwLSVKXYSFJ6jIsJEldhoUkqcuwkCR1GRaSpC7DQpLUtWbaBWh1m7vw2qm871ff/NKpvK+0UrlnIUnqMiwkSV2GhSSpy7CQJHV5gltaJaZ1MQF4QcHhwD0LSVLXaGGR5KQkH0+yM8nNSV7b2p+SZHuSW9rv41p7krw1ya4kn01y6oLX2tL635Jky1g1S5IWN+aexT7g16rqB4DTgQuSnAJcCHy0qjYAH23zAC8BNrSf84G3wyRcgIuA5wGnARfNB4wkaXmMFhZVdUdVfbpNfx3YCawHNgFbW7etwMva9Cbgipr4JLA2yQnAi4HtVbW3qu4CtgNnjlW3JOmRluWcRZI54LnA9cB3VtUdMAkU4Gmt23rgtgWr7W5tB2rf/z3OT7IjyY49e/Ys9RAkaVUbPSySPBF4H/C6qrr3YF0XaauDtD+8oerSqtpYVRvXrVv36IqVJC1q1LBIciSToHh3Vb2/Nf9TO7xE+31na98NnLRg9ROB2w/SLklaJmNeDRXgMmBnVb1lwaJtwPwVTVuADy5of2W7Kup04J52mOrDwIuSHNdObL+otUmSlsmYX8p7PvAK4HNJbmptbwTeDFyd5DzgVuDlbdl1wFnALuA+4FyAqtqb5HeAT7V+b6qqvSPWLUnaz2hhUVV/z+LnGwDOWKR/ARcc4LUuBy5fuuokSYfCb3BLkroMC0lSl2EhSeoyLCRJXYaFJKnLsJAkdRkWkqQuw0KS1GVYSJK6DAtJUpdhIUnqMiwkSV2GhSSpy7CQJHUZFpKkLsNCktRlWEiSugwLSVKXYSFJ6jIsJEldhoUkqcuwkCR1GRaSpC7DQpLUZVhIkroMC0lSl2EhSeoyLCRJXYaFJKnLsJAkdRkWkqQuw0KS1GVYSJK6DAtJUpdhIUnqMiwkSV2GhSSpy7CQJHWtmLBIcmaSLyXZleTCadcjSavJigiLJEcAfwq8BDgF+K9JTpluVZK0eqyIsABOA3ZV1Zer6gHgKmDTlGuSpFUjVTXtGrqSnAOcWVWvbvOvAJ5XVb+8oM/5wPlt9pnAlx7DWx4P/PNjWH9WHC7jAMcyiw6XcYBjmfc9VbVusQVrHn09yyqLtD0s5arqUuDSJXmzZEdVbVyK15qmw2Uc4Fhm0eEyDnAsQ6yUw1C7gZMWzJ8I3D6lWiRp1VkpYfEpYEOSk5McBWwGtk25JklaNVbEYaiq2pfkl4EPA0cAl1fVzSO+5ZIczpoBh8s4wLHMosNlHOBYulbECW5J0nStlMNQkqQpMiwkSV2rNix6tw9J8vgk72nLr08yt/xVDjNgLK9KsifJTe3n1dOosyfJ5UnuTPL5AyxPkre2cX42yanLXeNQA8byk0nuWbBNfnO5axwiyUlJPp5kZ5Kbk7x2kT4rYrsMHMtK2S5HJ7khyT+0sfz2In2W9jOsqlbdD5OT5P8XeAZwFPAPwCn79fkl4B1tejPwnmnX/RjG8irgT6Zd64Cx/DhwKvD5Ayw/C/hrJt+7OR24fto1P4ax/CTwoWnXOWAcJwCntunvAP7PIn9fK2K7DBzLStkuAZ7Ypo8ErgdO36/Pkn6GrdY9iyG3D9kEbG3T1wBnJFnsy4HTdtjcCqWq/g7Ye5Aum4ArauKTwNokJyxPdYdmwFhWhKq6o6o+3aa/DuwE1u/XbUVsl4FjWRHaf+t/abNHtp/9r1Za0s+w1RoW64HbFszv5pF/NA/1qap9wD3AU5elukMzZCwA/7kdIrgmyUmLLF8Jho51pfiRdhjhr5M8a9rF9LTDGM9l8q/YhVbcdjnIWGCFbJckRyS5CbgT2F5VB9wuS/EZtlrDonv7kIF9ZsGQOv8KmKuqHwQ+wrf/tbHSrJRtMsSnmdyH5znAHwMfmHI9B5XkicD7gNdV1b37L15klZndLp2xrJjtUlUPVtV/YHJHi9OSPHu/Lku6XVZrWAy5fchDfZKsAZ7MbB5W6I6lqr5WVfe32XcBP7RMtS21w+a2L1V17/xhhKq6DjgyyfFTLmtRSY5k8uH67qp6/yJdVsx26Y1lJW2XeVV1N/AJ4Mz9Fi3pZ9hqDYshtw/ZBmxp0+cAH6t2pmjGdMey3/Hjs5kcq12JtgGvbFffnA7cU1V3TLuoRyPJd80fP05yGpP/F7823aoeqdV4GbCzqt5ygG4rYrsMGcsK2i7rkqxt08cALwS+uF+3Jf0MWxG3+1hqdYDbhyR5E7CjqrYx+aO6MskuJmm8eXoVH9jAsbwmydnAPiZjedXUCj6IJH/J5GqU45PsBi5icuKOqnoHcB2TK292AfcB506n0r4BYzkH+O9J9gHfBDbP6D9Gng+8AvhcOz4O8Ebg6bDitsuQsayU7XICsDWTB8M9Dri6qj405meYt/uQJHWt1sNQkqRDYFhIkroMC0lSl2EhSeoyLCRJXYaFJKnLsJAkdf07x/RjwtTCcEMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "train_labels['accuracy_group'].plot(kind='hist')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature Engineering"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to do any modeling, the data needs to be preprocessed and prepared. I have chosen to do a categorical aggregation of the data into summary statistics grouped by installion id (user) and accumulated for each session prior to an assessment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Code below based on code original written/published by Massoud Hosseinali\n",
    "# on Kaggle, accessed 13 Dec, 2019\n",
    "# https://www.kaggle.com/mhviraf/a-new-baseline-for-dsb-2019-catboost-model\n",
    "\n",
    "#first we need to make a mapping to the proper event code for a completion of each event\n",
    "\n",
    "#list all activity titles\n",
    "list_of_user_activities = list(set(train_sample['title'].value_counts().index).union(set(test['title'].value_counts().index)))\n",
    "#map each activity to its integer index (very simple hash)\n",
    "activities_map = dict(zip(list_of_user_activities, np.arange(len(list_of_user_activities))))\n",
    "\n",
    "# modify the df (essentially a rename)\n",
    "train_sample['title'] = train_sample['title'].map(activities_map)\n",
    "test['title'] = test['title'].map(activities_map)\n",
    "\n",
    "#assessment successful event, or win code, is 4100 for most\n",
    "#assesments but 4110 for the final\n",
    "#make a mapping for this \n",
    "win_code = dict(zip(activities_map.values(), (4100*np.ones(len(activities_map))).astype('int')))\n",
    "win_code[activities_map['Bird Measurer (Assessment)']] = 4110"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "#convert the time data\n",
    "train_sample['timestamp'] = pd.to_datetime(train_sample['timestamp'])\n",
    "test['timestamp'] = pd.to_datetime(test['timestamp'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Make some dictionaries of unique event values to track their aggregrates\n",
    "list_of_event_code = list(set(train_sample['event_code'].unique()).union(set(test['event_code'].unique())))\n",
    "list_of_event_id = list(set(train_sample['event_id'].unique()).union(set(test['event_id'].unique())))\n",
    "activities_labels = dict(zip(np.arange(len(list_of_user_activities)), list_of_user_activities))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [],
   "source": [
    "from collections import Counter\n",
    "\n",
    "# create a function to summarize the data in each session\n",
    "\n",
    "def agg_data(data, test_set=False):\n",
    "    \n",
    "    activityData = []\n",
    "    user_activities_count = {'Clip':0, 'Activity': 0, 'Assessment': 0, 'Game':0}\n",
    "    accuracy_groups = {0:0, 1:0, 2:0, 3:0}\n",
    "    accumulated_accuracy_group = 0\n",
    "    accumulated_accuracy=0\n",
    "    accumulated_correct_attempts = 0 \n",
    "    accumulated_incorrect_attempts = 0 \n",
    "    accumulated_actions = 0\n",
    "    counter = 0\n",
    "    durations = []\n",
    "    last_activity = 0\n",
    "    \n",
    "    event_code_count: Dict[str, int] = {ev: 0 for ev in list_of_event_code}\n",
    "    event_id_count: Dict[str, int] = {eve: 0 for eve in list_of_event_id}\n",
    "    title_count: Dict[str, int] = {eve: 0 for eve in activities_labels.values()} \n",
    "    \n",
    "    for i, session in data.groupby('game_session', sort=False):\n",
    "        session_type = session['type'].iloc[0]\n",
    "        session_title = session['title'].iloc[0]\n",
    "        \n",
    "        #if we find an assessment and a session\n",
    "        #look at assessment data\n",
    "        if (session_type == 'Assessment') & (test_set or len(session) > 1):\n",
    "            \n",
    "            all_attempts = session.query(f'event_code == {win_code[session_title]}')\n",
    "            #check true or false by searching json event data str\n",
    "            true_attempts = all_attempts['event_data'].str.contains('true').sum()\n",
    "            false_attempts = all_attempts['event_data'].str.contains('false').sum()\n",
    "            \n",
    "            features = user_activities_count.copy()\n",
    "            features.update(event_code_count.copy())\n",
    "            features.update(event_id_count.copy())\n",
    "            features.update(title_count.copy())\n",
    "            \n",
    "            #baseline the number of attempts\n",
    "            features['session_title'] = session['title'].iloc[0] \n",
    "            features['accumulated_correct_attempts'] = accumulated_correct_attempts\n",
    "            features['accumulated_incorrect_attempts'] = accumulated_incorrect_attempts\n",
    "            accumulated_correct_attempts += true_attempts \n",
    "            accumulated_incorrect_attempts += false_attempts\n",
    "            \n",
    "            #recalc average of the duration\n",
    "            if durations == []:\n",
    "                features['duration_mean'] = 0\n",
    "            else:\n",
    "                features['duration_mean'] = np.mean(durations)\n",
    "            durations.append((session.iloc[-1, 2] - session.iloc[0, 2]).seconds)\n",
    "            \n",
    "            #compute accuracy as described in the competition\n",
    "            features['accumulated_accuracy'] = accumulated_accuracy/counter if counter > 0 else 0\n",
    "            accuracy = true_attempts/(true_attempts+false_attempts) if (true_attempts+false_attempts) != 0 else 0\n",
    "            accumulated_accuracy += accuracy\n",
    "            if accuracy == 0:\n",
    "                features['accuracy_group'] = 0\n",
    "            elif accuracy == 1:\n",
    "                features['accuracy_group'] = 3\n",
    "            elif accuracy == 0.5:\n",
    "                features['accuracy_group'] = 2\n",
    "            else:\n",
    "                features['accuracy_group'] = 1\n",
    "                \n",
    "            \n",
    "            features.update(accuracy_groups)\n",
    "            features['accumulated_accuracy_group'] = accumulated_accuracy_group/counter if counter > 0 else 0\n",
    "            features['accumulated_actions'] = accumulated_actions\n",
    "            accumulated_accuracy_group += features['accuracy_group']\n",
    "            accuracy_groups[features['accuracy_group']] += 1\n",
    "                \n",
    "            if test_set or true_attempts+false_attempts > 0:\n",
    "                activityData.append(features)\n",
    "                \n",
    "            counter += 1\n",
    "            \n",
    "        # count how many actions in each event_code so far by type\n",
    "        def update_counters(counter: dict, col: str):\n",
    "                num_of_session_count = Counter(session[col])\n",
    "                for k in num_of_session_count.keys():\n",
    "                    x = k\n",
    "                    if col == 'title':\n",
    "                        x = activities_labels[k]\n",
    "                    counter[x] += num_of_session_count[k]\n",
    "                return counter\n",
    "            \n",
    "        event_code_count = update_counters(event_code_count, \"event_code\")\n",
    "        event_id_count = update_counters(event_id_count, \"event_id\")\n",
    "        title_count = update_counters(title_count, 'title')\n",
    "        \n",
    "        # sum total actions for a session\n",
    "        accumulated_actions += len(session)\n",
    "        if last_activity != session_type:\n",
    "            user_activities_count[session_type] += 1\n",
    "            last_activitiy = session_type\n",
    "                \n",
    "    return activityData[-1] if test_set else activityData"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Clip</th>\n",
       "      <th>Activity</th>\n",
       "      <th>Assessment</th>\n",
       "      <th>Game</th>\n",
       "      <th>2050</th>\n",
       "      <th>4100</th>\n",
       "      <th>2060</th>\n",
       "      <th>4110</th>\n",
       "      <th>2070</th>\n",
       "      <th>2075</th>\n",
       "      <th>...</th>\n",
       "      <th>Egg Dropper (Activity)</th>\n",
       "      <th>Tree Top City - Level 2</th>\n",
       "      <th>session_title</th>\n",
       "      <th>accumulated_correct_attempts</th>\n",
       "      <th>accumulated_incorrect_attempts</th>\n",
       "      <th>duration_mean</th>\n",
       "      <th>accumulated_accuracy</th>\n",
       "      <th>accuracy_group</th>\n",
       "      <th>accumulated_accuracy_group</th>\n",
       "      <th>accumulated_actions</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>81</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>81</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>3</td>\n",
       "      <td>0.00</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>64</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>86379.0</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0</td>\n",
       "      <td>3.00</td>\n",
       "      <td>65</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>2</td>\n",
       "      <td>13</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>81</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>43199.0</td>\n",
       "      <td>0.50</td>\n",
       "      <td>0</td>\n",
       "      <td>1.50</td>\n",
       "      <td>261</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>20</td>\n",
       "      <td>4</td>\n",
       "      <td>24</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>76</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>64794.5</td>\n",
       "      <td>0.25</td>\n",
       "      <td>0</td>\n",
       "      <td>0.75</td>\n",
       "      <td>603</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3974</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>82</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>3</td>\n",
       "      <td>0.00</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3975</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3976</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>7.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>3</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3977</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>76</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>3</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3978</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>3</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3979 rows × 513 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      Clip  Activity  Assessment  Game  2050  4100  2060  4110  2070  2075  \\\n",
       "0        0         1           2     1     0     0     0     0     0     0   \n",
       "1        0         1           0     0     0     0     0     0     0     0   \n",
       "2        0         2           1     2     0     0     0     1     0     0   \n",
       "3        1         5           2    13     2     1     1     1     2     0   \n",
       "4        1        20           4    24     5     4     2     3     2     0   \n",
       "...    ...       ...         ...   ...   ...   ...   ...   ...   ...   ...   \n",
       "3974     0         1           0     0     0     0     0     0     0     0   \n",
       "3975     0         0           0     0     0     0     0     0     0     0   \n",
       "3976     0         0           1     0     0     1     0     0     0     0   \n",
       "3977     0         0           0     0     0     0     0     0     0     0   \n",
       "3978     0         0           0     0     0     0     0     0     0     0   \n",
       "\n",
       "      ...  Egg Dropper (Activity)  Tree Top City - Level 2  session_title  \\\n",
       "0     ...                       0                        0             81   \n",
       "1     ...                       0                        0             81   \n",
       "2     ...                       0                        0             64   \n",
       "3     ...                       0                        0             81   \n",
       "4     ...                       0                        0             76   \n",
       "...   ...                     ...                      ...            ...   \n",
       "3974  ...                       0                        0             82   \n",
       "3975  ...                       0                        0             29   \n",
       "3976  ...                       0                        0             29   \n",
       "3977  ...                       0                        0             76   \n",
       "3978  ...                       0                        0             29   \n",
       "\n",
       "      accumulated_correct_attempts  accumulated_incorrect_attempts  \\\n",
       "0                                0                               0   \n",
       "1                                0                               0   \n",
       "2                                1                               0   \n",
       "3                                1                               1   \n",
       "4                                1                               3   \n",
       "...                            ...                             ...   \n",
       "3974                             0                               0   \n",
       "3975                             0                               0   \n",
       "3976                             0                               1   \n",
       "3977                             0                               0   \n",
       "3978                             0                               0   \n",
       "\n",
       "      duration_mean  accumulated_accuracy  accuracy_group  \\\n",
       "0               3.0                  0.00               0   \n",
       "1               0.0                  0.00               3   \n",
       "2           86379.0                  1.00               0   \n",
       "3           43199.0                  0.50               0   \n",
       "4           64794.5                  0.25               0   \n",
       "...             ...                   ...             ...   \n",
       "3974            0.0                  0.00               3   \n",
       "3975            0.0                  0.00               0   \n",
       "3976            7.0                  0.00               3   \n",
       "3977            0.0                  0.00               3   \n",
       "3978            0.0                  0.00               3   \n",
       "\n",
       "      accumulated_accuracy_group  accumulated_actions  \n",
       "0                           0.00                   35  \n",
       "1                           0.00                   16  \n",
       "2                           3.00                   65  \n",
       "3                           1.50                  261  \n",
       "4                           0.75                  603  \n",
       "...                          ...                  ...  \n",
       "3974                        0.00                   15  \n",
       "3975                        0.00                    0  \n",
       "3976                        0.00                    6  \n",
       "3977                        0.00                    0  \n",
       "3978                        0.00                    0  \n",
       "\n",
       "[3979 rows x 513 columns]"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#compile data for each installation id by running above function\n",
    "compiled_data = []\n",
    "for i, (ins_id, data) in enumerate(train_sample.groupby('installation_id', sort=False)):\n",
    "    compiled_data += agg_data(data)\n",
    "\n",
    "new_train_sample = pd.DataFrame(compiled_data)\n",
    "del compiled_data\n",
    "new_train_sample"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3979, 512)"
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_features = [x for x in new_train_sample.columns if x not in ['accuracy_group']]\n",
    "X, y = new_train_sample[all_features], new_train_sample['accuracy_group']\n",
    "X.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Fit and predict based on the model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0    3745\n",
      "3.0     229\n",
      "2.0       5\n",
      "dtype: int64\n",
      "0    2414\n",
      "3    1412\n",
      "2     127\n",
      "1      26\n",
      "Name: accuracy_group, dtype: int64\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.591605931138477"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "0.0007181355734393602"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import cohen_kappa_score\n",
    "from sklearn.model_selection import KFold\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.feature_extraction import DictVectorizer\n",
    "\n",
    "oof = np.zeros(len(X))\n",
    "\n",
    "NFOLDS = 2\n",
    "folds = KFold(n_splits=NFOLDS, shuffle=True, random_state=2019)\n",
    "\n",
    "for fold, (trn_idx, test_idx) in enumerate(folds.split(X, y)):\n",
    "    currX = X.loc[trn_idx, all_features]\n",
    "    scaler = StandardScaler()\n",
    "    scaler.fit(currX)\n",
    "    currX_sc = scaler.transform(currX)\n",
    "    \n",
    "    # K-Nearest Neighbors Model\n",
    "    model = KNeighborsClassifier(n_neighbors=4)\n",
    "    model.fit(currX_sc, y.loc[trn_idx])\n",
    "    oof[test_idx] = model.predict(X.loc[test_idx,all_features])\n",
    "\n",
    "print(pd.Series(oof).value_counts())\n",
    "print(pd.Series(y).value_counts())\n",
    "display(accuracy_score(y, oof))\n",
    "cohen_kappa_score(y, oof, weights=\"quadratic\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Looks like this model is pretty poor because the features are not really set up properly for a KNN classification. Let's move on to some better modeling."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model Improvements"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cat Boost\n",
    "Code below based on code original written/published by Massoud Hosseinali\n",
    "on Kaggle, accessed 13 Dec, 2019\n",
    "https://www.kaggle.com/mhviraf/a-new-baseline-for-dsb-2019-catboost-model\n",
    "\n",
    "Use our previously aggregated features to fit a category boosting model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [],
   "source": [
    "from catboost import CatBoostClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [],
   "source": [
    "cat_features = ['session_title']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [],
   "source": [
    "def make_classifier():\n",
    "    return CatBoostClassifier(loss_function='MultiClass',\n",
    "            task_type=\"CPU\",\n",
    "            learning_rate=0.01,\n",
    "            iterations=2000,\n",
    "            od_type=\"Iter\",\n",
    "            early_stopping_rounds=500,\n",
    "            random_seed=1001);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.40278917827350325"
      ]
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cb_oof = np.zeros(len(X))\n",
    "folds = KFold(n_splits=5, shuffle=True, random_state=1001)\n",
    "\n",
    "for fold, (trn_idx, test_idx) in enumerate(folds.split(X, y)):\n",
    "    clf = make_classifier()\n",
    "    clf.fit(X.loc[trn_idx, all_features], y.loc[trn_idx], eval_set=(X.loc[test_idx, all_features], y.loc[test_idx]),\n",
    "                          use_best_model=True, verbose = False, cat_features=cat_features)\n",
    "\n",
    "    cb_oof[test_idx] = clf.predict(X.loc[test_idx, all_features]).reshape(len(test_idx))\n",
    "    \n",
    "cohen_kappa_score(y, cb_oof, weights=\"quadratic\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see this model gives a much better result due to better use of the features produced earlier. It uses a gradient boosting on decision trees technique to achieve these results.\n",
    "I found this article particularly helpful to understanding how it performs compared to other methods. \n",
    "\n",
    "https://towardsdatascience.com/catboost-vs-light-gbm-vs-xgboost-5f93620723db"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Regression model\n",
    "\n",
    "Let's test one of the regression options to see how it performs. There is a fair amount of numerical data so it could be decent.\n",
    "\n",
    "Code below based on code original written/published by Andrew Lukyanenko\n",
    "on Kaggle, accessed 13 Dec, 2019\n",
    "https://www.kaggle.com/artgor/quick-and-dirty-regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [],
   "source": [
    "import lightgbm as lgb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [],
   "source": [
    "#specify params\n",
    "\n",
    "params = {  \n",
    "            'n_estimators':2000,\n",
    "            'boosting_type': 'gbdt',\n",
    "            'objective': 'regression',\n",
    "            'metric': 'rmse',\n",
    "            'subsample': 0.75,\n",
    "            'subsample_freq': 1,\n",
    "            'learning_rate': 0.04,\n",
    "            'feature_fraction': 0.9,\n",
    "            'max_depth': 15,\n",
    "            'lambda_l1': 1,  \n",
    "            'lambda_l2': 1,\n",
    "            'verbose': 100,\n",
    "            'early_stopping_rounds': 100,\n",
    "            'eval_metric': 'cappa'\n",
    "          }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [],
   "source": [
    "lgb_oof = np.zeros(len(X))\n",
    "folds = KFold(n_splits=5, shuffle=True, random_state=1001)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [],
   "source": [
    "for fold, (trn_idx, test_idx) in enumerate(folds.split(X, y)):\n",
    "    reg_model = lgb.LGBMRegressor()\n",
    "    reg_model.set_params(**params)\n",
    "    \n",
    "    reg_model.fit(X.loc[trn_idx, all_features], y.loc[trn_idx], eval_set=(X.loc[test_idx, all_features], y.loc[test_idx]),\n",
    "                    verbose = False, early_stopping_rounds=params['early_stopping_rounds'])\n",
    "\n",
    "    lgb_oof[test_idx] = reg_model.predict(X.loc[test_idx, all_features]).reshape(len(test_idx))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [],
   "source": [
    "#convert to kappa scoreable resuls\n",
    "lgb_oof[lgb_oof <= 1.12232214] = 0\n",
    "lgb_oof[np.where(np.logical_and(lgb_oof > 1.12232214, lgb_oof <= 1.73925866))] = 1\n",
    "lgb_oof[np.where(np.logical_and(lgb_oof > 1.73925866, lgb_oof <= 2.22506454))] = 2\n",
    "lgb_oof[lgb_oof > 2.22506454] = 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.33278018643269935"
      ]
     },
     "execution_count": 123,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cohen_kappa_score(y, lgb_oof, weights=\"quadratic\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see this model gives a somewhat worse result as compared to CatBoost due to worse use of the categorical features."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reflection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Data Science is an interative process full of trial and error exploration. This project gave me great insight into the process of tuning and retuning features and testing different types of modeling to predict an outcome. I'm looking forward to rerunning on the entire data set in order to make a submission to the competition."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}