week2_multiTask_Bandit_final.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "60ecd45d",
   "metadata": {
    "colab_type": "text",
    "id": "view-in-github"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/biarne-a/co-rise-recommendation/blob/main/week2_multiTask_Bandit_final.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "837f0383-ee06-41a4-b431-032c431dfcbb",
   "metadata": {
    "id": "837f0383-ee06-41a4-b431-032c431dfcbb"
   },
   "source": [
    "# **Welcome to week 2 project!**\n",
    "\n",
    "Congratulations on making it to week 2! 👏 In the first week of this course, we covered the basics of how to design personalized recommendation systems. We then provided some system design examples for large scale recommenders from corporations like Spotify and YouTube, as well as techniques for candidate generation, specifically the two-tower model being used at Twitter and Pinterest.\n",
    "\n",
    "This week, we covered details of ML approaches for recommendations: including multi-task recommenders and contextual bandits.\n",
    "\n",
    "In this week's project, we will touch upon both these methods at a high level. We first begin by developing a simple multi-task model, and then cover a contextual bandit model. For ease of use, we will re-use our H&M dataset from week 1 for Multi-task model, and will switch to using Yahoo! news dataset for contextual bandit model.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02a97fbf-e1ea-46f1-a355-fc1a4377a2ee",
   "metadata": {
    "id": "02a97fbf-e1ea-46f1-a355-fc1a4377a2ee"
   },
   "source": [
    "# **Multi-task Recommendations**\n",
    "\n",
    "Multi-task recommenders provide a way to predict multiple facets of user engagement in order to make comprehensive recommendations for content that users might like.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3aeb2bd8-37fc-42f5-bc25-9c1eeb305d23",
   "metadata": {
    "id": "3aeb2bd8-37fc-42f5-bc25-9c1eeb305d23",
    "outputId": "f21def9f-a67d-466a-b6b8-31bdf30e3cb5"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<img src=\"https://miro.medium.com/max/3688/1*rrIJOpJO8fkFECNHlwq-jQ.png\" width=\"500\" class=\"unconfined\"/>"
      ],
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.core.display import Image, display\n",
    "display(Image(url='https://miro.medium.com/max/3688/1*rrIJOpJO8fkFECNHlwq-jQ.png', width=500, unconfined=True))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c659f20-fe0d-4c07-a8bd-6f5b90be571d",
   "metadata": {
    "id": "5c659f20-fe0d-4c07-a8bd-6f5b90be571d"
   },
   "source": [
    "\n",
    "**Single task recommender:** we will start by deveoping a single task model that makes predictions about whether the user will purchase the article or not.\n",
    "\n",
    "**Multi-task recommender:** we then expand the scope to consider auxilliary prediction tasks, in order to improve the recommendation performance.\n",
    "\n",
    "Specifically, we make the folliwing three predictions using a multi-task setup:\n",
    "1. Predicting whether the user will make any purchase next week or not\n",
    "2. Predicting which category the user will make purchase in\n",
    "3. Predicting which specific item the user will buy\n",
    "\n",
    "We will use different loss functions to jointly train the multi-task network.\n",
    "\n",
    "**Negative Sampling**\n",
    "It is important to note that the dataset is only of positive cases -- cases wherein the user bought an article. To train our models, we will have to resort to negative sampling to develop a mixed dataset comprising of positive and negative examples.\n",
    "\n",
    "We will cover two categories of negative sampling:\n",
    "1. Random negative sampling\n",
    "2. Negative sampling with bias\n",
    "\n",
    "**Evaluation:**\n",
    "We will evaluate the ranker performance on various ranking metrics.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "add75169-0279-469c-98c2-36e6fc28ebdd",
   "metadata": {
    "id": "add75169-0279-469c-98c2-36e6fc28ebdd"
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "import tensorflow_recommenders as tfrs\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "from matplotlib import pyplot as plt\n",
    "from tqdm.notebook import tqdm\n",
    "from pathlib import Path\n",
    "from typing import Dict, Text\n",
    "from functools import reduce\n",
    "import os\n",
    "import pprint\n",
    "import tempfile\n",
    "from typing import Dict, Text\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "import tensorflow_recommenders as tfrs\n",
    "from tensorflow import feature_column\n",
    "import random"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "3081bc7b-ab90-41c0-9c21-35dfe59b4228",
   "metadata": {
    "id": "3081bc7b-ab90-41c0-9c21-35dfe59b4228"
   },
   "outputs": [],
   "source": [
    "article_df = pd.read_csv(\"hmdata/articles.csv.zip\")\n",
    "customer_df = pd.read_csv(\"hmdata/customers.csv.zip\")\n",
    "train0 = pd.read_csv('hmdata/transactions_train.csv.zip')\n",
    "transaction_df = train0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8e7c8031-0949-4c38-a650-4a4e33595556",
   "metadata": {
    "id": "8e7c8031-0949-4c38-a650-4a4e33595556"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b2a327e2-5d4a-483a-9de4-5298594d3878",
   "metadata": {
    "id": "b2a327e2-5d4a-483a-9de4-5298594d3878"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "340c115e-84f5-419d-9337-5e00a434ff2f",
   "metadata": {
    "id": "340c115e-84f5-419d-9337-5e00a434ff2f"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "2b073d71-0ffc-407c-9338-1a87cee00b07",
   "metadata": {
    "id": "2b073d71-0ffc-407c-9338-1a87cee00b07"
   },
   "source": [
    "# **Simple Feature Processing**\n",
    "\n",
    "In this section we focus on extracting features for trainnig the ranking models. We focus primarily on extracting user features, and article features, and leave sophisticated feature modelin for next week.\n",
    "\n",
    "For users specifically, we will focus on extracting the following features:\n",
    "1. age bucket\n",
    "2. no of past purchases\n",
    "3. min/max/avg price\n",
    "4. distrbution over product group\n",
    "5. distrbution over section\n",
    "\n",
    "For aticles, we will learn embedding based features for the article category, product group, and department.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4480de53-5cfd-44bf-864a-98a02d675dc6",
   "metadata": {
    "id": "4480de53-5cfd-44bf-864a-98a02d675dc6"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "a3b09610-07d8-4933-8bcd-1de5ec43193b",
   "metadata": {
    "id": "a3b09610-07d8-4933-8bcd-1de5ec43193b"
   },
   "outputs": [],
   "source": [
    "unique_customer_ids = customer_df.customer_id.unique()\n",
    "unique_article_ids = article_df.article_id.unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "ca8a544f-acd7-470d-8b51-1b6a9b051809",
   "metadata": {
    "id": "ca8a544f-acd7-470d-8b51-1b6a9b051809",
    "outputId": "b4c4a58b-80e2-4c4a-bdf2-b235495060e4"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "31788324 5108386\n"
     ]
    }
   ],
   "source": [
    "subset = train0[train0[\"t_dat\"]>\"2020-06-01\"]\n",
    "print(len(train0),len(subset))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "1e3a5838-bc19-43bf-b31a-48bcd2578933",
   "metadata": {
    "id": "1e3a5838-bc19-43bf-b31a-48bcd2578933",
    "outputId": "8f97fdff-e1b2-4a9a-c71e-fae7ffc506bb"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/15/f0f69s3n5235dwmr7mx8w97m0000gq/T/ipykernel_37687/2244244855.py:5: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  subset[\"train\"] = subset.apply (lambda row: split(row), axis=1)\n"
     ]
    }
   ],
   "source": [
    "def split(x):\n",
    "    if x[\"t_dat\"] < \"2020-08-22\":\n",
    "        return 1\n",
    "    return 0\n",
    "subset[\"train\"] = subset.apply(lambda row: split(row), axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "01ff690c-aef6-489f-8e13-037ed1c41084",
   "metadata": {
    "id": "01ff690c-aef6-489f-8e13-037ed1c41084"
   },
   "outputs": [],
   "source": [
    "joined = pd.merge(subset, article_df, on='article_id', how='outer')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "4d839a73-79fb-4364-897c-e8dd9607770b",
   "metadata": {
    "id": "4d839a73-79fb-4364-897c-e8dd9607770b"
   },
   "outputs": [],
   "source": [
    "f1 = joined.groupby(\"customer_id\", as_index=False).article_id.nunique() #unique articles\n",
    "f2 = joined.groupby(\"customer_id\", as_index=False).t_dat.nunique() #days active\n",
    "f3 = joined.groupby(\"customer_id\", as_index=False).size() #total purchases\n",
    "f4 = joined.groupby(\"customer_id\", as_index=False).price.min() #min price\n",
    "f5 = joined.groupby(\"customer_id\", as_index=False).price.max() #max price\n",
    "f6 = joined.groupby(\"customer_id\", as_index=False).price.mean() #mean price\n",
    "f7 = joined.groupby(\"customer_id\", as_index=False).product_group_name.nunique() #num unique product groups\n",
    "f8 = joined.groupby(\"customer_id\", as_index=False).department_name.nunique() #num unique department_name\n",
    "f9 = joined.groupby(\"customer_id\", as_index=False).section_name.nunique() #num unique section_name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "5c5eea45-0e60-48ef-8e48-90155fcdaf99",
   "metadata": {
    "id": "5c5eea45-0e60-48ef-8e48-90155fcdaf99"
   },
   "outputs": [],
   "source": [
    "data_frames = [f1,f2,f3,f4,f5,f6,f7,f8,f9]\n",
    "fAll = reduce(lambda left, right: pd.merge(left,right,on=['customer_id'], how='outer'), data_frames)\n",
    "fAll.columns = ['customer_id', 'nArticles', 'nDays', 'nPurchases', 'minPrice', 'maxPrice', 'meanPrice', 'nPGroups', 'nDept', 'nSect']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "e7ff03f5-65d4-4f91-98d8-68ccefbe48a2",
   "metadata": {
    "id": "e7ff03f5-65d4-4f91-98d8-68ccefbe48a2"
   },
   "outputs": [],
   "source": [
    "joinedF = pd.merge(joined, fAll, on='customer_id', how='outer')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "2f74ea24-619d-4f02-a64e-b72c277ba9b7",
   "metadata": {
    "id": "2f74ea24-619d-4f02-a64e-b72c277ba9b7",
    "outputId": "d848cc61-6a01-4240-f5fe-ff46bac59de0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['t_dat', 'customer_id', 'article_id', 'price', 'sales_channel_id', 'train', 'product_code', 'prod_name', 'product_type_no', 'product_type_name', 'product_group_name', 'graphical_appearance_no', 'graphical_appearance_name', 'colour_group_code', 'colour_group_name', 'perceived_colour_value_id', 'perceived_colour_value_name', 'perceived_colour_master_id', 'perceived_colour_master_name', 'department_no', 'department_name', 'index_code', 'index_name', 'index_group_no', 'index_group_name', 'section_no', 'section_name', 'garment_group_no', 'garment_group_name', 'detail_desc', 'nArticles', 'nDays', 'nPurchases', 'minPrice', 'maxPrice', 'meanPrice', 'nPGroups', 'nDept', 'nSect']\n"
     ]
    }
   ],
   "source": [
    "print(list(joinedF))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "b579529e-69b8-4770-8759-fe8f77bf4d66",
   "metadata": {
    "id": "b579529e-69b8-4770-8759-fe8f77bf4d66",
    "outputId": "649d2e39-2672-4c4a-f809-ce0941397b94"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>customer_id</th>\n",
       "      <th>article_id</th>\n",
       "      <th>price</th>\n",
       "      <th>product_group_name</th>\n",
       "      <th>department_name</th>\n",
       "      <th>section_name</th>\n",
       "      <th>nArticles</th>\n",
       "      <th>nDays</th>\n",
       "      <th>nPurchases</th>\n",
       "      <th>minPrice</th>\n",
       "      <th>maxPrice</th>\n",
       "      <th>meanPrice</th>\n",
       "      <th>nPGroups</th>\n",
       "      <th>nDept</th>\n",
       "      <th>nSect</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...</td>\n",
       "      <td>855834001</td>\n",
       "      <td>0.015831</td>\n",
       "      <td>Garment Full body</td>\n",
       "      <td>Dresses</td>\n",
       "      <td>Divided Collection</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.010153</td>\n",
       "      <td>0.027102</td>\n",
       "      <td>0.017130</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...</td>\n",
       "      <td>836130002</td>\n",
       "      <td>0.015831</td>\n",
       "      <td>Garment Full body</td>\n",
       "      <td>Dresses</td>\n",
       "      <td>Divided Collection</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.010153</td>\n",
       "      <td>0.027102</td>\n",
       "      <td>0.017130</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...</td>\n",
       "      <td>808841001</td>\n",
       "      <td>0.016932</td>\n",
       "      <td>Garment Lower body</td>\n",
       "      <td>Skirts</td>\n",
       "      <td>Divided Collection</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.010153</td>\n",
       "      <td>0.027102</td>\n",
       "      <td>0.017130</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...</td>\n",
       "      <td>817472002</td>\n",
       "      <td>0.010153</td>\n",
       "      <td>Garment Lower body</td>\n",
       "      <td>Shorts</td>\n",
       "      <td>Womens Everyday Collection</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.010153</td>\n",
       "      <td>0.027102</td>\n",
       "      <td>0.017130</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...</td>\n",
       "      <td>806529003</td>\n",
       "      <td>0.027102</td>\n",
       "      <td>Garment Lower body</td>\n",
       "      <td>Trouser</td>\n",
       "      <td>Womens Tailoring</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.010153</td>\n",
       "      <td>0.027102</td>\n",
       "      <td>0.017130</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5108381</th>\n",
       "      <td>b834a2cb440d2298493d254196abb73e101d8b35b155c6...</td>\n",
       "      <td>870982002</td>\n",
       "      <td>0.061000</td>\n",
       "      <td>Garment Upper body</td>\n",
       "      <td>Outdoor/Blazers</td>\n",
       "      <td>Divided Collection</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.061000</td>\n",
       "      <td>0.061000</td>\n",
       "      <td>0.061000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5108382</th>\n",
       "      <td>83b8a97fd3decf8e21d05a9acaea822eaa569c957ea232...</td>\n",
       "      <td>893427001</td>\n",
       "      <td>0.084729</td>\n",
       "      <td>Garment Full body</td>\n",
       "      <td>Trouser</td>\n",
       "      <td>Womens Everyday Collection</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.084729</td>\n",
       "      <td>0.084729</td>\n",
       "      <td>0.084729</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5108383</th>\n",
       "      <td>83b8a97fd3decf8e21d05a9acaea822eaa569c957ea232...</td>\n",
       "      <td>893427001</td>\n",
       "      <td>0.084729</td>\n",
       "      <td>Garment Full body</td>\n",
       "      <td>Trouser</td>\n",
       "      <td>Womens Everyday Collection</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.084729</td>\n",
       "      <td>0.084729</td>\n",
       "      <td>0.084729</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5108384</th>\n",
       "      <td>83b8a97fd3decf8e21d05a9acaea822eaa569c957ea232...</td>\n",
       "      <td>903910002</td>\n",
       "      <td>0.084729</td>\n",
       "      <td>Garment Full body</td>\n",
       "      <td>Dress</td>\n",
       "      <td>Womens Tailoring</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.084729</td>\n",
       "      <td>0.084729</td>\n",
       "      <td>0.084729</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5108385</th>\n",
       "      <td>b9036ca1353cd6d07511e309f8acab557559b724a36419...</td>\n",
       "      <td>910010002</td>\n",
       "      <td>0.050831</td>\n",
       "      <td>Garment Upper body</td>\n",
       "      <td>Tops Knitwear</td>\n",
       "      <td>Divided Collection</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.050831</td>\n",
       "      <td>0.050831</td>\n",
       "      <td>0.050831</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5108386 rows × 15 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               customer_id  article_id  \\\n",
       "0        0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...   855834001   \n",
       "1        0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...   836130002   \n",
       "2        0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...   808841001   \n",
       "3        0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...   817472002   \n",
       "4        0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...   806529003   \n",
       "...                                                    ...         ...   \n",
       "5108381  b834a2cb440d2298493d254196abb73e101d8b35b155c6...   870982002   \n",
       "5108382  83b8a97fd3decf8e21d05a9acaea822eaa569c957ea232...   893427001   \n",
       "5108383  83b8a97fd3decf8e21d05a9acaea822eaa569c957ea232...   893427001   \n",
       "5108384  83b8a97fd3decf8e21d05a9acaea822eaa569c957ea232...   903910002   \n",
       "5108385  b9036ca1353cd6d07511e309f8acab557559b724a36419...   910010002   \n",
       "\n",
       "            price  product_group_name  department_name  \\\n",
       "0        0.015831   Garment Full body          Dresses   \n",
       "1        0.015831   Garment Full body          Dresses   \n",
       "2        0.016932  Garment Lower body           Skirts   \n",
       "3        0.010153  Garment Lower body           Shorts   \n",
       "4        0.027102  Garment Lower body          Trouser   \n",
       "...           ...                 ...              ...   \n",
       "5108381  0.061000  Garment Upper body  Outdoor/Blazers   \n",
       "5108382  0.084729   Garment Full body          Trouser   \n",
       "5108383  0.084729   Garment Full body          Trouser   \n",
       "5108384  0.084729   Garment Full body            Dress   \n",
       "5108385  0.050831  Garment Upper body    Tops Knitwear   \n",
       "\n",
       "                       section_name  nArticles  nDays  nPurchases  minPrice  \\\n",
       "0                Divided Collection        6.0    4.0         6.0  0.010153   \n",
       "1                Divided Collection        6.0    4.0         6.0  0.010153   \n",
       "2                Divided Collection        6.0    4.0         6.0  0.010153   \n",
       "3        Womens Everyday Collection        6.0    4.0         6.0  0.010153   \n",
       "4                  Womens Tailoring        6.0    4.0         6.0  0.010153   \n",
       "...                             ...        ...    ...         ...       ...   \n",
       "5108381          Divided Collection        1.0    1.0         1.0  0.061000   \n",
       "5108382  Womens Everyday Collection        2.0    1.0         3.0  0.084729   \n",
       "5108383  Womens Everyday Collection        2.0    1.0         3.0  0.084729   \n",
       "5108384            Womens Tailoring        2.0    1.0         3.0  0.084729   \n",
       "5108385          Divided Collection        1.0    1.0         1.0  0.050831   \n",
       "\n",
       "         maxPrice  meanPrice  nPGroups  nDept  nSect  \n",
       "0        0.027102   0.017130       3.0    5.0    3.0  \n",
       "1        0.027102   0.017130       3.0    5.0    3.0  \n",
       "2        0.027102   0.017130       3.0    5.0    3.0  \n",
       "3        0.027102   0.017130       3.0    5.0    3.0  \n",
       "4        0.027102   0.017130       3.0    5.0    3.0  \n",
       "...           ...        ...       ...    ...    ...  \n",
       "5108381  0.061000   0.061000       1.0    1.0    1.0  \n",
       "5108382  0.084729   0.084729       1.0    2.0    2.0  \n",
       "5108383  0.084729   0.084729       1.0    2.0    2.0  \n",
       "5108384  0.084729   0.084729       1.0    2.0    2.0  \n",
       "5108385  0.050831   0.050831       1.0    1.0    1.0  \n",
       "\n",
       "[5108386 rows x 15 columns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "columns = ['customer_id','article_id',\n",
    "           'price',\n",
    "          'product_group_name', 'department_name',\n",
    "             'section_name',\n",
    "             'nArticles',\n",
    "             'nDays',\n",
    "             'nPurchases',\n",
    "             'minPrice',\n",
    "             'maxPrice',\n",
    "             'meanPrice',\n",
    "             'nPGroups',\n",
    "             'nDept',\n",
    "             'nSect']\n",
    "trainF = joinedF[columns][~joinedF[\"customer_id\"].isna()]\n",
    "trainF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "675e6078-f1ca-4993-ba1d-48a9dc94ea38",
   "metadata": {
    "id": "675e6078-f1ca-4993-ba1d-48a9dc94ea38"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "9ff60881-c2ab-4c1f-b69a-67c28becb6cf",
   "metadata": {
    "id": "9ff60881-c2ab-4c1f-b69a-67c28becb6cf"
   },
   "source": [
    "# Negative Sampling for training ranker\n",
    "\n",
    "If you notice the dataset, we only have purchase information. This is generallyt he case -- we often only observe positive interactions from users. However, to train a model, we would need access to positive and negative samples both.\n",
    "\n",
    "To generate negative sampled, here we focus on random sampling of negative examples per user-artice transaction. Other methods exist to sample negatives, namely:\n",
    "1. Random sampling\n",
    "2. Biased sampling from similar categories\n",
    "3. Cross-batch negative sampling\n",
    "\n",
    "In this project, we will go ahead with the random sampling strategy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "0ebf9e47-d778-4822-b37b-f0dd29323816",
   "metadata": {
    "id": "0ebf9e47-d778-4822-b37b-f0dd29323816"
   },
   "outputs": [],
   "source": [
    "tt = trainF.groupby(\"customer_id\", as_index=False).article_id.agg(['unique']).reset_index()\n",
    "tt.columns = ['customer_id', 'article_list']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "cfcb19e5-9e0d-4fa8-ab65-b46ced061c35",
   "metadata": {
    "id": "cfcb19e5-9e0d-4fa8-ab65-b46ced061c35",
    "outputId": "c699c8ca-a45f-4ef6-c2dd-0aee58ee0ca1"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>customer_id</th>\n",
       "      <th>article_list</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>00000dbacae5abe5e23885899a1fa44253a17956c6d1c3...</td>\n",
       "      <td>[568601043]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0000423b00ade91418cceaf3b26c6af3dd342b51fd051e...</td>\n",
       "      <td>[826211002]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>000058a12d5b43e67d225668fa1f8d618c13dc232df0ca...</td>\n",
       "      <td>[794321007]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>00006413d8573cd20ed7128e53b7b13819fe5cfc2d801f...</td>\n",
       "      <td>[827971001, 589440005, 818320001, 730683050, 9...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0000757967448a6cb83efb3ea7a3fb9d418ac7adf2379d...</td>\n",
       "      <td>[448509014, 719530003]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>596584</th>\n",
       "      <td>ffff8f9ecdce722b5bab97fff68a6d1866492209bfe524...</td>\n",
       "      <td>[889679001, 863001002, 859845001]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>596585</th>\n",
       "      <td>ffffbbf78b6eaac697a8a5dfbfd2bfa8113ee5b403e474...</td>\n",
       "      <td>[684209019, 599580046, 822311009, 791587007, 8...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>596586</th>\n",
       "      <td>ffffcd5046a6143d29a04fb8c424ce494a76e5cdf4fab5...</td>\n",
       "      <td>[853654002, 685816044, 805370005, 759871030, 6...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>596587</th>\n",
       "      <td>ffffcf35913a0bee60e8741cb2b4e78b8a98ee5ff2e6a1...</td>\n",
       "      <td>[636323002, 795975009, 884081001, 689365050, 5...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>596588</th>\n",
       "      <td>ffffd7744cebcf3aca44ae7049d2a94b87074c3d4ffe38...</td>\n",
       "      <td>[882810001]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>596589 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                              customer_id  \\\n",
       "0       00000dbacae5abe5e23885899a1fa44253a17956c6d1c3...   \n",
       "1       0000423b00ade91418cceaf3b26c6af3dd342b51fd051e...   \n",
       "2       000058a12d5b43e67d225668fa1f8d618c13dc232df0ca...   \n",
       "3       00006413d8573cd20ed7128e53b7b13819fe5cfc2d801f...   \n",
       "4       0000757967448a6cb83efb3ea7a3fb9d418ac7adf2379d...   \n",
       "...                                                   ...   \n",
       "596584  ffff8f9ecdce722b5bab97fff68a6d1866492209bfe524...   \n",
       "596585  ffffbbf78b6eaac697a8a5dfbfd2bfa8113ee5b403e474...   \n",
       "596586  ffffcd5046a6143d29a04fb8c424ce494a76e5cdf4fab5...   \n",
       "596587  ffffcf35913a0bee60e8741cb2b4e78b8a98ee5ff2e6a1...   \n",
       "596588  ffffd7744cebcf3aca44ae7049d2a94b87074c3d4ffe38...   \n",
       "\n",
       "                                             article_list  \n",
       "0                                             [568601043]  \n",
       "1                                             [826211002]  \n",
       "2                                             [794321007]  \n",
       "3       [827971001, 589440005, 818320001, 730683050, 9...  \n",
       "4                                  [448509014, 719530003]  \n",
       "...                                                   ...  \n",
       "596584                  [889679001, 863001002, 859845001]  \n",
       "596585  [684209019, 599580046, 822311009, 791587007, 8...  \n",
       "596586  [853654002, 685816044, 805370005, 759871030, 6...  \n",
       "596587  [636323002, 795975009, 884081001, 689365050, 5...  \n",
       "596588                                        [882810001]  \n",
       "\n",
       "[596589 rows x 2 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "42b22b40-e7c3-4b76-913f-9383119d114f",
   "metadata": {
    "id": "42b22b40-e7c3-4b76-913f-9383119d114f"
   },
   "outputs": [],
   "source": [
    "articlesSet = list(trainF.article_id.unique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "97f629c5-7b58-41ea-962d-ad559ee6197b",
   "metadata": {
    "id": "97f629c5-7b58-41ea-962d-ad559ee6197b",
    "outputId": "c4a891d9-7be3-4c82-b100-a673b13b649e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "45892\n",
      "['customer_id', 'article_list']\n"
     ]
    }
   ],
   "source": [
    "print(len(articlesSet))\n",
    "tt.columns = ['customer_id', 'article_list']\n",
    "print(list(tt))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "8d82cc39-6cdc-48d4-8a0b-1fba2d1171e5",
   "metadata": {
    "id": "8d82cc39-6cdc-48d4-8a0b-1fba2d1171e5"
   },
   "outputs": [],
   "source": [
    "def generate_negative_samples(train_df, article_list, num_neg=1):\n",
    "    negData = []\n",
    "    for index, row in train_df.iterrows():\n",
    "        allList = article_list\n",
    "        cust = row['customer_id']\n",
    "        art_list = row['article_list']\n",
    "        allList = list(set(allList) - set(art_list))\n",
    "        negs = random.sample(allList, num_neg)\n",
    "        for n in negs:\n",
    "            negData.append([cust,n])\n",
    "    negDf = pd.DataFrame(negData, columns=['customer_id','article_id'])\n",
    "    return negDf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "33d10277-f699-4e1f-9e1a-eb5a5b8d52d7",
   "metadata": {
    "id": "33d10277-f699-4e1f-9e1a-eb5a5b8d52d7"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "c9828161-6e0f-4ada-b435-c35e5d401cd3",
   "metadata": {
    "id": "c9828161-6e0f-4ada-b435-c35e5d401cd3",
    "outputId": "a8add935-19c1-4c8e-fcc7-e12f351b6da0"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>customer_id</th>\n",
       "      <th>article_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>00000dbacae5abe5e23885899a1fa44253a17956c6d1c3...</td>\n",
       "      <td>902130001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>00000dbacae5abe5e23885899a1fa44253a17956c6d1c3...</td>\n",
       "      <td>875950009</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0000423b00ade91418cceaf3b26c6af3dd342b51fd051e...</td>\n",
       "      <td>694848005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0000423b00ade91418cceaf3b26c6af3dd342b51fd051e...</td>\n",
       "      <td>848813002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>000058a12d5b43e67d225668fa1f8d618c13dc232df0ca...</td>\n",
       "      <td>859727001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79995</th>\n",
       "      <td>11297a0414218c18ab7109376c63c7d8f093b49bbd254d...</td>\n",
       "      <td>630675002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79996</th>\n",
       "      <td>1129a321da62de7224393a06c477b851d0581636f7b9e7...</td>\n",
       "      <td>911552002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79997</th>\n",
       "      <td>1129a321da62de7224393a06c477b851d0581636f7b9e7...</td>\n",
       "      <td>642910008</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79998</th>\n",
       "      <td>112a04b065c314ab86a3536a3d31c267f88200435cbe75...</td>\n",
       "      <td>750423022</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79999</th>\n",
       "      <td>112a04b065c314ab86a3536a3d31c267f88200435cbe75...</td>\n",
       "      <td>796783002</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>80000 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                             customer_id  article_id\n",
       "0      00000dbacae5abe5e23885899a1fa44253a17956c6d1c3...   902130001\n",
       "1      00000dbacae5abe5e23885899a1fa44253a17956c6d1c3...   875950009\n",
       "2      0000423b00ade91418cceaf3b26c6af3dd342b51fd051e...   694848005\n",
       "3      0000423b00ade91418cceaf3b26c6af3dd342b51fd051e...   848813002\n",
       "4      000058a12d5b43e67d225668fa1f8d618c13dc232df0ca...   859727001\n",
       "...                                                  ...         ...\n",
       "79995  11297a0414218c18ab7109376c63c7d8f093b49bbd254d...   630675002\n",
       "79996  1129a321da62de7224393a06c477b851d0581636f7b9e7...   911552002\n",
       "79997  1129a321da62de7224393a06c477b851d0581636f7b9e7...   642910008\n",
       "79998  112a04b065c314ab86a3536a3d31c267f88200435cbe75...   750423022\n",
       "79999  112a04b065c314ab86a3536a3d31c267f88200435cbe75...   796783002\n",
       "\n",
       "[80000 rows x 2 columns]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "negDf = generate_negative_samples(tt.head(40000), articlesSet, 2)\n",
    "negDf"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "affe3023-ad6e-4c30-b02d-df5626b479a5",
   "metadata": {
    "id": "affe3023-ad6e-4c30-b02d-df5626b479a5"
   },
   "source": [
    "### We next generate features for negatives"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "f652957e-c199-49db-a9f0-1268966ae57d",
   "metadata": {
    "id": "f652957e-c199-49db-a9f0-1268966ae57d",
    "outputId": "b276a76d-47b4-43ec-8b2e-01bd63bd1d0a"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>customer_id</th>\n",
       "      <th>article_id</th>\n",
       "      <th>product_group_name</th>\n",
       "      <th>department_name</th>\n",
       "      <th>section_name</th>\n",
       "      <th>nArticles</th>\n",
       "      <th>nDays</th>\n",
       "      <th>nPurchases</th>\n",
       "      <th>minPrice</th>\n",
       "      <th>maxPrice</th>\n",
       "      <th>meanPrice</th>\n",
       "      <th>nPGroups</th>\n",
       "      <th>nDept</th>\n",
       "      <th>nSect</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>00000dbacae5abe5e23885899a1fa44253a17956c6d1c3...</td>\n",
       "      <td>902130001</td>\n",
       "      <td>Garment Upper body</td>\n",
       "      <td>Woven Tops</td>\n",
       "      <td>Mama</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.050831</td>\n",
       "      <td>0.050831</td>\n",
       "      <td>0.050831</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>00000dbacae5abe5e23885899a1fa44253a17956c6d1c3...</td>\n",
       "      <td>875950009</td>\n",
       "      <td>Garment Full body</td>\n",
       "      <td>Baby Girl Jersey Fancy</td>\n",
       "      <td>Baby Girl</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.050831</td>\n",
       "      <td>0.050831</td>\n",
       "      <td>0.050831</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0000423b00ade91418cceaf3b26c6af3dd342b51fd051e...</td>\n",
       "      <td>694848005</td>\n",
       "      <td>Underwear</td>\n",
       "      <td>Casual Lingerie</td>\n",
       "      <td>Womens Lingerie</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.027102</td>\n",
       "      <td>0.027102</td>\n",
       "      <td>0.027102</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0000423b00ade91418cceaf3b26c6af3dd342b51fd051e...</td>\n",
       "      <td>848813002</td>\n",
       "      <td>Underwear</td>\n",
       "      <td>Functional Lingerie</td>\n",
       "      <td>Womens Lingerie</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.027102</td>\n",
       "      <td>0.027102</td>\n",
       "      <td>0.027102</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>000058a12d5b43e67d225668fa1f8d618c13dc232df0ca...</td>\n",
       "      <td>859727001</td>\n",
       "      <td>Garment Upper body</td>\n",
       "      <td>Knitwear</td>\n",
       "      <td>Womens Everyday Collection</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.061000</td>\n",
       "      <td>0.061000</td>\n",
       "      <td>0.061000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79995</th>\n",
       "      <td>11297a0414218c18ab7109376c63c7d8f093b49bbd254d...</td>\n",
       "      <td>630675002</td>\n",
       "      <td>Accessories</td>\n",
       "      <td>Hair Accessories</td>\n",
       "      <td>Womens Small accessories</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0.008458</td>\n",
       "      <td>0.033881</td>\n",
       "      <td>0.021169</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79996</th>\n",
       "      <td>1129a321da62de7224393a06c477b851d0581636f7b9e7...</td>\n",
       "      <td>911552002</td>\n",
       "      <td>Garment Upper body</td>\n",
       "      <td>Baby Exclusive</td>\n",
       "      <td>Baby Essentials &amp; Complements</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>0.005068</td>\n",
       "      <td>0.022017</td>\n",
       "      <td>0.016367</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79997</th>\n",
       "      <td>1129a321da62de7224393a06c477b851d0581636f7b9e7...</td>\n",
       "      <td>642910008</td>\n",
       "      <td>Garment Upper body</td>\n",
       "      <td>Jersey License</td>\n",
       "      <td>Contemporary Street</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>0.005068</td>\n",
       "      <td>0.022017</td>\n",
       "      <td>0.016367</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79998</th>\n",
       "      <td>112a04b065c314ab86a3536a3d31c267f88200435cbe75...</td>\n",
       "      <td>750423022</td>\n",
       "      <td>Garment Lower body</td>\n",
       "      <td>Trouser</td>\n",
       "      <td>Womens Casual</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>0.016932</td>\n",
       "      <td>0.016932</td>\n",
       "      <td>0.016932</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79999</th>\n",
       "      <td>112a04b065c314ab86a3536a3d31c267f88200435cbe75...</td>\n",
       "      <td>796783002</td>\n",
       "      <td>Garment Lower body</td>\n",
       "      <td>Young Girl Trouser</td>\n",
       "      <td>Young Girl</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>0.016932</td>\n",
       "      <td>0.016932</td>\n",
       "      <td>0.016932</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>80000 rows × 14 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                             customer_id  article_id  \\\n",
       "0      00000dbacae5abe5e23885899a1fa44253a17956c6d1c3...   902130001   \n",
       "1      00000dbacae5abe5e23885899a1fa44253a17956c6d1c3...   875950009   \n",
       "2      0000423b00ade91418cceaf3b26c6af3dd342b51fd051e...   694848005   \n",
       "3      0000423b00ade91418cceaf3b26c6af3dd342b51fd051e...   848813002   \n",
       "4      000058a12d5b43e67d225668fa1f8d618c13dc232df0ca...   859727001   \n",
       "...                                                  ...         ...   \n",
       "79995  11297a0414218c18ab7109376c63c7d8f093b49bbd254d...   630675002   \n",
       "79996  1129a321da62de7224393a06c477b851d0581636f7b9e7...   911552002   \n",
       "79997  1129a321da62de7224393a06c477b851d0581636f7b9e7...   642910008   \n",
       "79998  112a04b065c314ab86a3536a3d31c267f88200435cbe75...   750423022   \n",
       "79999  112a04b065c314ab86a3536a3d31c267f88200435cbe75...   796783002   \n",
       "\n",
       "       product_group_name         department_name  \\\n",
       "0      Garment Upper body              Woven Tops   \n",
       "1       Garment Full body  Baby Girl Jersey Fancy   \n",
       "2               Underwear         Casual Lingerie   \n",
       "3               Underwear     Functional Lingerie   \n",
       "4      Garment Upper body                Knitwear   \n",
       "...                   ...                     ...   \n",
       "79995         Accessories        Hair Accessories   \n",
       "79996  Garment Upper body          Baby Exclusive   \n",
       "79997  Garment Upper body          Jersey License   \n",
       "79998  Garment Lower body                 Trouser   \n",
       "79999  Garment Lower body      Young Girl Trouser   \n",
       "\n",
       "                        section_name  nArticles  nDays  nPurchases  minPrice  \\\n",
       "0                               Mama          1      1           1  0.050831   \n",
       "1                          Baby Girl          1      1           1  0.050831   \n",
       "2                    Womens Lingerie          1      1           1  0.027102   \n",
       "3                    Womens Lingerie          1      1           1  0.027102   \n",
       "4         Womens Everyday Collection          1      1           1  0.061000   \n",
       "...                              ...        ...    ...         ...       ...   \n",
       "79995       Womens Small accessories          2      1           2  0.008458   \n",
       "79996  Baby Essentials & Complements          3      2           3  0.005068   \n",
       "79997            Contemporary Street          3      2           3  0.005068   \n",
       "79998                  Womens Casual          2      1           4  0.016932   \n",
       "79999                     Young Girl          2      1           4  0.016932   \n",
       "\n",
       "       maxPrice  meanPrice  nPGroups  nDept  nSect  \n",
       "0      0.050831   0.050831         1      1      1  \n",
       "1      0.050831   0.050831         1      1      1  \n",
       "2      0.027102   0.027102         1      1      1  \n",
       "3      0.027102   0.027102         1      1      1  \n",
       "4      0.061000   0.061000         1      1      1  \n",
       "...         ...        ...       ...    ...    ...  \n",
       "79995  0.033881   0.021169         1      1      1  \n",
       "79996  0.022017   0.016367         3      3      3  \n",
       "79997  0.022017   0.016367         3      3      3  \n",
       "79998  0.016932   0.016932         1      1      1  \n",
       "79999  0.016932   0.016932         1      1      1  \n",
       "\n",
       "[80000 rows x 14 columns]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "joinedFN = pd.merge(negDf, article_df[['article_id','product_group_name', 'department_name','section_name']], on='article_id', how='left')\n",
    "joinedFN = pd.merge(joinedFN, fAll, on='customer_id', how='left')\n",
    "joinedFN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "40788433-0522-4e6a-a1ef-0a61b3670f50",
   "metadata": {
    "id": "40788433-0522-4e6a-a1ef-0a61b3670f50"
   },
   "outputs": [],
   "source": [
    "trainF['label1']=1\n",
    "joinedFN['label1']=0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "aaacc944-f2ed-4e94-afee-2599db25377a",
   "metadata": {
    "id": "aaacc944-f2ed-4e94-afee-2599db25377a"
   },
   "outputs": [],
   "source": [
    "data = [trainF.head(40000), joinedFN]\n",
    "trainDF = pd.concat(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "3294a2f5-4856-4317-ad4c-1bdf1e5066fb",
   "metadata": {
    "id": "3294a2f5-4856-4317-ad4c-1bdf1e5066fb",
    "outputId": "ccd9ab9b-2818-45f8-af8a-1fd4d71fe405"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>customer_id</th>\n",
       "      <th>article_id</th>\n",
       "      <th>price</th>\n",
       "      <th>product_group_name</th>\n",
       "      <th>department_name</th>\n",
       "      <th>section_name</th>\n",
       "      <th>nArticles</th>\n",
       "      <th>nDays</th>\n",
       "      <th>nPurchases</th>\n",
       "      <th>minPrice</th>\n",
       "      <th>maxPrice</th>\n",
       "      <th>meanPrice</th>\n",
       "      <th>nPGroups</th>\n",
       "      <th>nDept</th>\n",
       "      <th>nSect</th>\n",
       "      <th>label1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...</td>\n",
       "      <td>855834001</td>\n",
       "      <td>0.015831</td>\n",
       "      <td>Garment Full body</td>\n",
       "      <td>Dresses</td>\n",
       "      <td>Divided Collection</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.010153</td>\n",
       "      <td>0.027102</td>\n",
       "      <td>0.017130</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...</td>\n",
       "      <td>836130002</td>\n",
       "      <td>0.015831</td>\n",
       "      <td>Garment Full body</td>\n",
       "      <td>Dresses</td>\n",
       "      <td>Divided Collection</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.010153</td>\n",
       "      <td>0.027102</td>\n",
       "      <td>0.017130</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...</td>\n",
       "      <td>808841001</td>\n",
       "      <td>0.016932</td>\n",
       "      <td>Garment Lower body</td>\n",
       "      <td>Skirts</td>\n",
       "      <td>Divided Collection</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.010153</td>\n",
       "      <td>0.027102</td>\n",
       "      <td>0.017130</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...</td>\n",
       "      <td>817472002</td>\n",
       "      <td>0.010153</td>\n",
       "      <td>Garment Lower body</td>\n",
       "      <td>Shorts</td>\n",
       "      <td>Womens Everyday Collection</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.010153</td>\n",
       "      <td>0.027102</td>\n",
       "      <td>0.017130</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...</td>\n",
       "      <td>806529003</td>\n",
       "      <td>0.027102</td>\n",
       "      <td>Garment Lower body</td>\n",
       "      <td>Trouser</td>\n",
       "      <td>Womens Tailoring</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.010153</td>\n",
       "      <td>0.027102</td>\n",
       "      <td>0.017130</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79995</th>\n",
       "      <td>11297a0414218c18ab7109376c63c7d8f093b49bbd254d...</td>\n",
       "      <td>630675002</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Accessories</td>\n",
       "      <td>Hair Accessories</td>\n",
       "      <td>Womens Small accessories</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.008458</td>\n",
       "      <td>0.033881</td>\n",
       "      <td>0.021169</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79996</th>\n",
       "      <td>1129a321da62de7224393a06c477b851d0581636f7b9e7...</td>\n",
       "      <td>911552002</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Garment Upper body</td>\n",
       "      <td>Baby Exclusive</td>\n",
       "      <td>Baby Essentials &amp; Complements</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.005068</td>\n",
       "      <td>0.022017</td>\n",
       "      <td>0.016367</td>\n",
       "      <td>3.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79997</th>\n",
       "      <td>1129a321da62de7224393a06c477b851d0581636f7b9e7...</td>\n",
       "      <td>642910008</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Garment Upper body</td>\n",
       "      <td>Jersey License</td>\n",
       "      <td>Contemporary Street</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.005068</td>\n",
       "      <td>0.022017</td>\n",
       "      <td>0.016367</td>\n",
       "      <td>3.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79998</th>\n",
       "      <td>112a04b065c314ab86a3536a3d31c267f88200435cbe75...</td>\n",
       "      <td>750423022</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Garment Lower body</td>\n",
       "      <td>Trouser</td>\n",
       "      <td>Womens Casual</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.016932</td>\n",
       "      <td>0.016932</td>\n",
       "      <td>0.016932</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79999</th>\n",
       "      <td>112a04b065c314ab86a3536a3d31c267f88200435cbe75...</td>\n",
       "      <td>796783002</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Garment Lower body</td>\n",
       "      <td>Young Girl Trouser</td>\n",
       "      <td>Young Girl</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.016932</td>\n",
       "      <td>0.016932</td>\n",
       "      <td>0.016932</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>120000 rows × 16 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                             customer_id  article_id  \\\n",
       "0      0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...   855834001   \n",
       "1      0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...   836130002   \n",
       "2      0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...   808841001   \n",
       "3      0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...   817472002   \n",
       "4      0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...   806529003   \n",
       "...                                                  ...         ...   \n",
       "79995  11297a0414218c18ab7109376c63c7d8f093b49bbd254d...   630675002   \n",
       "79996  1129a321da62de7224393a06c477b851d0581636f7b9e7...   911552002   \n",
       "79997  1129a321da62de7224393a06c477b851d0581636f7b9e7...   642910008   \n",
       "79998  112a04b065c314ab86a3536a3d31c267f88200435cbe75...   750423022   \n",
       "79999  112a04b065c314ab86a3536a3d31c267f88200435cbe75...   796783002   \n",
       "\n",
       "          price  product_group_name     department_name  \\\n",
       "0      0.015831   Garment Full body             Dresses   \n",
       "1      0.015831   Garment Full body             Dresses   \n",
       "2      0.016932  Garment Lower body              Skirts   \n",
       "3      0.010153  Garment Lower body              Shorts   \n",
       "4      0.027102  Garment Lower body             Trouser   \n",
       "...         ...                 ...                 ...   \n",
       "79995       NaN         Accessories    Hair Accessories   \n",
       "79996       NaN  Garment Upper body      Baby Exclusive   \n",
       "79997       NaN  Garment Upper body      Jersey License   \n",
       "79998       NaN  Garment Lower body             Trouser   \n",
       "79999       NaN  Garment Lower body  Young Girl Trouser   \n",
       "\n",
       "                        section_name  nArticles  nDays  nPurchases  minPrice  \\\n",
       "0                 Divided Collection        6.0    4.0         6.0  0.010153   \n",
       "1                 Divided Collection        6.0    4.0         6.0  0.010153   \n",
       "2                 Divided Collection        6.0    4.0         6.0  0.010153   \n",
       "3         Womens Everyday Collection        6.0    4.0         6.0  0.010153   \n",
       "4                   Womens Tailoring        6.0    4.0         6.0  0.010153   \n",
       "...                              ...        ...    ...         ...       ...   \n",
       "79995       Womens Small accessories        2.0    1.0         2.0  0.008458   \n",
       "79996  Baby Essentials & Complements        3.0    2.0         3.0  0.005068   \n",
       "79997            Contemporary Street        3.0    2.0         3.0  0.005068   \n",
       "79998                  Womens Casual        2.0    1.0         4.0  0.016932   \n",
       "79999                     Young Girl        2.0    1.0         4.0  0.016932   \n",
       "\n",
       "       maxPrice  meanPrice  nPGroups  nDept  nSect  label1  \n",
       "0      0.027102   0.017130       3.0    5.0    3.0       1  \n",
       "1      0.027102   0.017130       3.0    5.0    3.0       1  \n",
       "2      0.027102   0.017130       3.0    5.0    3.0       1  \n",
       "3      0.027102   0.017130       3.0    5.0    3.0       1  \n",
       "4      0.027102   0.017130       3.0    5.0    3.0       1  \n",
       "...         ...        ...       ...    ...    ...     ...  \n",
       "79995  0.033881   0.021169       1.0    1.0    1.0       0  \n",
       "79996  0.022017   0.016367       3.0    3.0    3.0       0  \n",
       "79997  0.022017   0.016367       3.0    3.0    3.0       0  \n",
       "79998  0.016932   0.016932       1.0    1.0    1.0       0  \n",
       "79999  0.016932   0.016932       1.0    1.0    1.0       0  \n",
       "\n",
       "[120000 rows x 16 columns]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainDF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d7b30d68-d8d0-4d94-8a6b-1e989b9d211b",
   "metadata": {
    "id": "d7b30d68-d8d0-4d94-8a6b-1e989b9d211b"
   },
   "outputs": [],
   "source": [
    "trainDF.to_pickle(\"trainDF.pkl\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3fb5fcde-a4a8-416f-862f-1883f922fde4",
   "metadata": {
    "id": "3fb5fcde-a4a8-416f-862f-1883f922fde4"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b9965cc2-bb5c-4ccd-8095-dac367deb68b",
   "metadata": {
    "id": "b9965cc2-bb5c-4ccd-8095-dac367deb68b",
    "outputId": "e514a57a-fa5e-4cad-a40f-a2d2db57c164"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['customer_id', 'article_id', 'price', 'product_group_name', 'department_name', 'section_name', 'nArticles', 'nDays', 'nPurchases', 'minPrice', 'maxPrice', 'meanPrice', 'nPGroups', 'nDept', 'nSect', 'label1']\n"
     ]
    }
   ],
   "source": [
    "print(list(trainDF))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "857e7128-8eee-4b15-a6dd-14b8ba62301c",
   "metadata": {
    "id": "857e7128-8eee-4b15-a6dd-14b8ba62301c"
   },
   "source": [
    "## Single Task Ranker\n",
    "\n",
    "We begin by training a single task ranker for the task of predicting whether the user will purchase a given article or not."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "402f669a-3931-44b2-8239-9a43fd0a6573",
   "metadata": {},
   "outputs": [],
   "source": [
    "numFeatures = ['nArticles','nDays','nPurchases','minPrice','maxPrice','meanPrice','nPGroups','nDept','nSect']\n",
    "X1 = np.asarray(trainDF[numFeatures].values)\n",
    "y1 = np.asarray(trainDF['label1'].values)\n",
    "\n",
    "inp1 = tf.keras.Input((len(numFeatures)))\n",
    "\n",
    "x1 = tf.keras.layers.Dense(32, activation='relu')(inp1)\n",
    "out1 = tf.keras.layers.Dense(1, activation='sigmoid')(x1)\n",
    "bce = tf.keras.losses.BinaryCrossentropy(from_logits=False)\n",
    "metric = tf.keras.metrics.BinaryAccuracy()\n",
    "m = tf.keras.Model([inp1], out1)\n",
    "m.compile(loss=bce,\n",
    "          optimizer='adam',\n",
    "          metrics=[metric])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "0c6a7d3b-6c1a-4b2f-8e13-1d2464eed750",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model_11\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " input_23 (InputLayer)       [(None, 9)]               0         \n",
      "                                                                 \n",
      " dense_31 (Dense)            (None, 32)                320       \n",
      "                                                                 \n",
      " dense_32 (Dense)            (None, 1)                 33        \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 353\n",
      "Trainable params: 353\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "m.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "a19bad79-7ea8-4ef6-81e4-61e8e2ed3ce6",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.7683 - binary_accuracy: 0.6349\n",
      "Epoch 2/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.4939 - binary_accuracy: 0.8074\n",
      "Epoch 3/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.4573 - binary_accuracy: 0.8149\n",
      "Epoch 4/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.4255 - binary_accuracy: 0.8176\n",
      "Epoch 5/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.4076 - binary_accuracy: 0.8227\n",
      "Epoch 6/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3980 - binary_accuracy: 0.8242\n",
      "Epoch 7/50\n",
      "120/120 [==============================] - 0s 990us/step - loss: 0.3910 - binary_accuracy: 0.8255\n",
      "Epoch 8/50\n",
      "120/120 [==============================] - 0s 998us/step - loss: 0.3871 - binary_accuracy: 0.8260\n",
      "Epoch 9/50\n",
      "120/120 [==============================] - 0s 992us/step - loss: 0.3842 - binary_accuracy: 0.8270\n",
      "Epoch 10/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3825 - binary_accuracy: 0.8262\n",
      "Epoch 11/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3817 - binary_accuracy: 0.8265\n",
      "Epoch 12/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3809 - binary_accuracy: 0.8262\n",
      "Epoch 13/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3803 - binary_accuracy: 0.8265\n",
      "Epoch 14/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3799 - binary_accuracy: 0.8260\n",
      "Epoch 15/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3793 - binary_accuracy: 0.8270\n",
      "Epoch 16/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3793 - binary_accuracy: 0.8267\n",
      "Epoch 17/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3788 - binary_accuracy: 0.8260\n",
      "Epoch 18/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3794 - binary_accuracy: 0.8264\n",
      "Epoch 19/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3782 - binary_accuracy: 0.8264\n",
      "Epoch 20/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3781 - binary_accuracy: 0.8266\n",
      "Epoch 21/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3777 - binary_accuracy: 0.8265\n",
      "Epoch 22/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3772 - binary_accuracy: 0.8265\n",
      "Epoch 23/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3772 - binary_accuracy: 0.8267\n",
      "Epoch 24/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3775 - binary_accuracy: 0.8262\n",
      "Epoch 25/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3763 - binary_accuracy: 0.8271\n",
      "Epoch 26/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3763 - binary_accuracy: 0.8263\n",
      "Epoch 27/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3758 - binary_accuracy: 0.8272\n",
      "Epoch 28/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3762 - binary_accuracy: 0.8266\n",
      "Epoch 29/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3754 - binary_accuracy: 0.8269\n",
      "Epoch 30/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3751 - binary_accuracy: 0.8273\n",
      "Epoch 31/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3753 - binary_accuracy: 0.8273\n",
      "Epoch 32/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3752 - binary_accuracy: 0.8270\n",
      "Epoch 33/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3744 - binary_accuracy: 0.8280\n",
      "Epoch 34/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3746 - binary_accuracy: 0.8282\n",
      "Epoch 35/50\n",
      "120/120 [==============================] - 0s 957us/step - loss: 0.3740 - binary_accuracy: 0.8289\n",
      "Epoch 36/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3739 - binary_accuracy: 0.8289\n",
      "Epoch 37/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3740 - binary_accuracy: 0.8296\n",
      "Epoch 38/50\n",
      "120/120 [==============================] - 0s 960us/step - loss: 0.3732 - binary_accuracy: 0.8298\n",
      "Epoch 39/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3730 - binary_accuracy: 0.8294\n",
      "Epoch 40/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3726 - binary_accuracy: 0.8304\n",
      "Epoch 41/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3723 - binary_accuracy: 0.8305\n",
      "Epoch 42/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3723 - binary_accuracy: 0.8303\n",
      "Epoch 43/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3723 - binary_accuracy: 0.8301\n",
      "Epoch 44/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3717 - binary_accuracy: 0.8306\n",
      "Epoch 45/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3718 - binary_accuracy: 0.8305\n",
      "Epoch 46/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3718 - binary_accuracy: 0.8306\n",
      "Epoch 47/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3725 - binary_accuracy: 0.8304\n",
      "Epoch 48/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3716 - binary_accuracy: 0.8310\n",
      "Epoch 49/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3710 - binary_accuracy: 0.8315\n",
      "Epoch 50/50\n",
      "120/120 [==============================] - 0s 1ms/step - loss: 0.3718 - binary_accuracy: 0.8303\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x1bb33dfa0>"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.fit([X1], [y1], epochs=50, verbose=1, batch_size=1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "id": "1a069406-b11d-4e39-a902-a7e402c9c718",
   "metadata": {
    "id": "1a069406-b11d-4e39-a902-a7e402c9c718",
    "outputId": "adddcb44-2f84-407a-e16c-2ca4d4fcfc87",
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "numFeatures = ['nArticles','nDays','nPurchases','minPrice','maxPrice','meanPrice','nPGroups','nDept','nSect']\n",
    "catFeatures = ['product_group_name']\n",
    "X1 = np.asarray(trainDF[numFeatures].values)\n",
    "X2 = np.asarray(trainDF[catFeatures].values)\n",
    "y1 = np.asarray(trainDF['label1'].values)\n",
    "\n",
    "inp1 = tf.keras.Input((len(numFeatures)))\n",
    "inp2 = tf.keras.Input((len(catFeatures)), dtype=tf.string)\n",
    "\n",
    "unique_product_groups = trainDF['product_group_name'].unique().astype(str)\n",
    "\n",
    "x1 = tf.keras.layers.Dense(32, activation='relu')(inp1)\n",
    "embedding_dimension=50\n",
    "xx2 = tf.keras.layers.StringLookup(vocabulary=unique_product_groups, mask_token=None)(inp2)\n",
    "xx2 = tf.keras.layers.Embedding(len(unique_product_groups) + 1, embedding_dimension)(xx2)\n",
    "xx2 = tf.keras.layers.Flatten()(xx2)\n",
    "x2 = tf.keras.layers.Dense(32, activation='relu')(xx2)\n",
    "\n",
    "x12 = tf.keras.layers.concatenate([x1, x2])\n",
    "x12 = tf.keras.layers.Dense(32, activation='relu')(x12)\n",
    "out1 = tf.keras.layers.Dense(1, activation='sigmoid')(x12)\n",
    "bce = tf.keras.losses.BinaryCrossentropy(from_logits=False)\n",
    "metric = tf.keras.metrics.BinaryAccuracy()\n",
    "m = tf.keras.Model([inp1, inp2], out1)\n",
    "m.compile(loss=bce,\n",
    "          optimizer='adam',\n",
    "          metrics=[metric])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "id": "3d1f7480-1848-4ec5-8384-e18eec2541f7",
   "metadata": {
    "id": "3d1f7480-1848-4ec5-8384-e18eec2541f7",
    "outputId": "cb906a59-f529-4de1-8ced-6fe4beac799d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model_14\"\n",
      "__________________________________________________________________________________________________\n",
      " Layer (type)                   Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      " input_29 (InputLayer)          [(None, 1)]          0           []                               \n",
      "                                                                                                  \n",
      " string_lookup_10 (StringLookup  (None, 1)           0           ['input_29[0][0]']               \n",
      " )                                                                                                \n",
      "                                                                                                  \n",
      " embedding_10 (Embedding)       (None, 1, 50)        950         ['string_lookup_10[0][0]']       \n",
      "                                                                                                  \n",
      " input_28 (InputLayer)          [(None, 9)]          0           []                               \n",
      "                                                                                                  \n",
      " flatten_10 (Flatten)           (None, 50)           0           ['embedding_10[0][0]']           \n",
      "                                                                                                  \n",
      " dense_41 (Dense)               (None, 32)           320         ['input_28[0][0]']               \n",
      "                                                                                                  \n",
      " dense_42 (Dense)               (None, 32)           1632        ['flatten_10[0][0]']             \n",
      "                                                                                                  \n",
      " concatenate_9 (Concatenate)    (None, 64)           0           ['dense_41[0][0]',               \n",
      "                                                                  'dense_42[0][0]']               \n",
      "                                                                                                  \n",
      " dense_43 (Dense)               (None, 32)           2080        ['concatenate_9[0][0]']          \n",
      "                                                                                                  \n",
      " dense_44 (Dense)               (None, 1)            33          ['dense_43[0][0]']               \n",
      "                                                                                                  \n",
      "==================================================================================================\n",
      "Total params: 5,015\n",
      "Trainable params: 5,015\n",
      "Non-trainable params: 0\n",
      "__________________________________________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "m.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "id": "1c4171d3-a908-4d00-a39d-cf66b0382f04",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/50\n",
      "120/120 [==============================] - 1s 2ms/step - loss: 0.4683 - binary_accuracy: 0.7980\n",
      "Epoch 2/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3795 - binary_accuracy: 0.8288\n",
      "Epoch 3/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3719 - binary_accuracy: 0.8320\n",
      "Epoch 4/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3695 - binary_accuracy: 0.8335\n",
      "Epoch 5/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3694 - binary_accuracy: 0.8334\n",
      "Epoch 6/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3693 - binary_accuracy: 0.8327\n",
      "Epoch 7/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3657 - binary_accuracy: 0.8337\n",
      "Epoch 8/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3662 - binary_accuracy: 0.8340\n",
      "Epoch 9/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3648 - binary_accuracy: 0.8348\n",
      "Epoch 10/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3645 - binary_accuracy: 0.8344\n",
      "Epoch 11/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3625 - binary_accuracy: 0.8356\n",
      "Epoch 12/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3609 - binary_accuracy: 0.8361\n",
      "Epoch 13/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3602 - binary_accuracy: 0.8366\n",
      "Epoch 14/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3590 - binary_accuracy: 0.8368\n",
      "Epoch 15/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3583 - binary_accuracy: 0.8375\n",
      "Epoch 16/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3574 - binary_accuracy: 0.8376\n",
      "Epoch 17/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3569 - binary_accuracy: 0.8377\n",
      "Epoch 18/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3552 - binary_accuracy: 0.8390\n",
      "Epoch 19/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3544 - binary_accuracy: 0.8385\n",
      "Epoch 20/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3540 - binary_accuracy: 0.8394\n",
      "Epoch 21/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3551 - binary_accuracy: 0.8391\n",
      "Epoch 22/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3537 - binary_accuracy: 0.8393\n",
      "Epoch 23/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3529 - binary_accuracy: 0.8393\n",
      "Epoch 24/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3526 - binary_accuracy: 0.8398\n",
      "Epoch 25/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3528 - binary_accuracy: 0.8395\n",
      "Epoch 26/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3515 - binary_accuracy: 0.8405\n",
      "Epoch 27/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3512 - binary_accuracy: 0.8402\n",
      "Epoch 28/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3508 - binary_accuracy: 0.8410\n",
      "Epoch 29/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3503 - binary_accuracy: 0.8406\n",
      "Epoch 30/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3498 - binary_accuracy: 0.8413\n",
      "Epoch 31/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3493 - binary_accuracy: 0.8413\n",
      "Epoch 32/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3492 - binary_accuracy: 0.8415\n",
      "Epoch 33/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3495 - binary_accuracy: 0.8411\n",
      "Epoch 34/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3497 - binary_accuracy: 0.8412\n",
      "Epoch 35/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3486 - binary_accuracy: 0.8416\n",
      "Epoch 36/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3486 - binary_accuracy: 0.8417\n",
      "Epoch 37/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3478 - binary_accuracy: 0.8420\n",
      "Epoch 38/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3475 - binary_accuracy: 0.8420\n",
      "Epoch 39/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3494 - binary_accuracy: 0.8408\n",
      "Epoch 40/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3480 - binary_accuracy: 0.8419\n",
      "Epoch 41/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3470 - binary_accuracy: 0.8425\n",
      "Epoch 42/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3467 - binary_accuracy: 0.8427\n",
      "Epoch 43/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3467 - binary_accuracy: 0.8428\n",
      "Epoch 44/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3473 - binary_accuracy: 0.8421\n",
      "Epoch 45/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3475 - binary_accuracy: 0.8426\n",
      "Epoch 46/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3463 - binary_accuracy: 0.8424\n",
      "Epoch 47/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3462 - binary_accuracy: 0.8427\n",
      "Epoch 48/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3455 - binary_accuracy: 0.8426\n",
      "Epoch 49/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3468 - binary_accuracy: 0.8419\n",
      "Epoch 50/50\n",
      "120/120 [==============================] - 0s 2ms/step - loss: 0.3456 - binary_accuracy: 0.8428\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x1ba125400>"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.fit([X1, X2], [y1], epochs=50, verbose=1, batch_size=1000)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65ada5f3-8157-4088-95b8-8e17989b2d52",
   "metadata": {
    "id": "65ada5f3-8157-4088-95b8-8e17989b2d52"
   },
   "source": [
    "### **From single to multi-task recommendation model**\n",
    "We can now add another task to enrich the single task model to multi-task model. To do so, we first have to decide what the task 2 would be.\n",
    "\n",
    "By looking at the data we have, we hypothesize that being able to predict whether or not a customer would purchase an article from certain product category would be a good learnt model to have. Indeed, being able to predict which category would the user purchase in would help us narrow down the list of recommendations we want to surface infront of users.\n",
    "\n",
    "Based on this intuition, we want to develop a multi-task model with 2 tasks:\n",
    "1. Task 1: predict whether or not a user would purchase an article\n",
    "2. Task 2: predict whether or not a user would purchase any article from a given category\n",
    "\n",
    "The training data we have created thus far is solely based on prediction task 1: predicting whether or not a user would purchase a given article. Corredpondingly, in trainDF dataframe we have collected positive and nagetive examples for this task, and have added a label1 column as the final label to train the model.\n",
    "\n",
    "To develop a multi-task model for these two tasks, we will now need to add a label for the second task in the same dataset: trainDF. To do so, we will need to add a column: label2 to this dataframe -- for each row, we will need to identify whether the user has made any purchase in this category or not, and if the user has made a purchase in this category, then we will assign label2 = 1, else label2 = 0. Please note that for the same training example, it may happen that label1 = 0 but label2 = 1; this would happen in cases wherein the customer did not purchase this specific article but did purchase any other article from this category.\n",
    "\n",
    "The goal for this project is to write the function that adds this label2 column to the trainDF dataframe, to enable us to train a multi-task model."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f22e8d0-d2d0-4097-aed6-03aa817b3061",
   "metadata": {},
   "source": [
    "__First we want to prepare a dictionary that will give us in constant time the set of product groups any given user has purchased in the past.__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "id": "5efc902e-b3b4-4ae7-8ddb-4b1916823196",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>customer_id</th>\n",
       "      <th>article_id</th>\n",
       "      <th>price</th>\n",
       "      <th>product_group_name</th>\n",
       "      <th>department_name</th>\n",
       "      <th>section_name</th>\n",
       "      <th>nArticles</th>\n",
       "      <th>nDays</th>\n",
       "      <th>nPurchases</th>\n",
       "      <th>minPrice</th>\n",
       "      <th>maxPrice</th>\n",
       "      <th>meanPrice</th>\n",
       "      <th>nPGroups</th>\n",
       "      <th>nDept</th>\n",
       "      <th>nSect</th>\n",
       "      <th>label1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...</td>\n",
       "      <td>855834001</td>\n",
       "      <td>0.015831</td>\n",
       "      <td>Garment Full body</td>\n",
       "      <td>Dresses</td>\n",
       "      <td>Divided Collection</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.010153</td>\n",
       "      <td>0.027102</td>\n",
       "      <td>0.01713</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...</td>\n",
       "      <td>836130002</td>\n",
       "      <td>0.015831</td>\n",
       "      <td>Garment Full body</td>\n",
       "      <td>Dresses</td>\n",
       "      <td>Divided Collection</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.010153</td>\n",
       "      <td>0.027102</td>\n",
       "      <td>0.01713</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...</td>\n",
       "      <td>808841001</td>\n",
       "      <td>0.016932</td>\n",
       "      <td>Garment Lower body</td>\n",
       "      <td>Skirts</td>\n",
       "      <td>Divided Collection</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.010153</td>\n",
       "      <td>0.027102</td>\n",
       "      <td>0.01713</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...</td>\n",
       "      <td>817472002</td>\n",
       "      <td>0.010153</td>\n",
       "      <td>Garment Lower body</td>\n",
       "      <td>Shorts</td>\n",
       "      <td>Womens Everyday Collection</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.010153</td>\n",
       "      <td>0.027102</td>\n",
       "      <td>0.01713</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...</td>\n",
       "      <td>806529003</td>\n",
       "      <td>0.027102</td>\n",
       "      <td>Garment Lower body</td>\n",
       "      <td>Trouser</td>\n",
       "      <td>Womens Tailoring</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.010153</td>\n",
       "      <td>0.027102</td>\n",
       "      <td>0.01713</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                         customer_id  article_id     price  \\\n",
       "0  0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...   855834001  0.015831   \n",
       "1  0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...   836130002  0.015831   \n",
       "2  0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...   808841001  0.016932   \n",
       "3  0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...   817472002  0.010153   \n",
       "4  0001f8cef6b9702d54abf66fd89eb21014bf98567065a9...   806529003  0.027102   \n",
       "\n",
       "   product_group_name department_name                section_name  nArticles  \\\n",
       "0   Garment Full body         Dresses          Divided Collection        6.0   \n",
       "1   Garment Full body         Dresses          Divided Collection        6.0   \n",
       "2  Garment Lower body          Skirts          Divided Collection        6.0   \n",
       "3  Garment Lower body          Shorts  Womens Everyday Collection        6.0   \n",
       "4  Garment Lower body         Trouser            Womens Tailoring        6.0   \n",
       "\n",
       "   nDays  nPurchases  minPrice  maxPrice  meanPrice  nPGroups  nDept  nSect  \\\n",
       "0    4.0         6.0  0.010153  0.027102    0.01713       3.0    5.0    3.0   \n",
       "1    4.0         6.0  0.010153  0.027102    0.01713       3.0    5.0    3.0   \n",
       "2    4.0         6.0  0.010153  0.027102    0.01713       3.0    5.0    3.0   \n",
       "3    4.0         6.0  0.010153  0.027102    0.01713       3.0    5.0    3.0   \n",
       "4    4.0         6.0  0.010153  0.027102    0.01713       3.0    5.0    3.0   \n",
       "\n",
       "   label1  \n",
       "0       1  \n",
       "1       1  \n",
       "2       1  \n",
       "3       1  \n",
       "4       1  "
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainDF.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "id": "0b2951be-16df-487f-9bae-4dc9c68d1447",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Obviously, we need to consider only the positive examples (given that the negatives are the ones the user did not buy)\n",
    "positive_trainF = trainF[trainF['label1'] == 1]\n",
    "unique_product_groups_by_customer = positive_trainF.groupby(\"customer_id\", as_index=False).product_group_name.agg(['unique']).reset_index()\n",
    "unique_product_groups_by_customer.columns = ['customer_id', 'product_groups']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "id": "491d98a5-4f41-41c4-a51b-b9cb25f918fa",
   "metadata": {},
   "outputs": [],
   "source": [
    "unique_product_groups_by_customer['product_groups'] = unique_product_groups_by_customer['product_groups'].apply(set)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "id": "9bac801e-62cc-4867-adfe-9e93166bd72e",
   "metadata": {},
   "outputs": [],
   "source": [
    "product_groups_by_customer = unique_product_groups_by_customer.set_index('customer_id').to_dict('index')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "id": "39b6f339-ccae-4cb8-949d-dc7947333bf1",
   "metadata": {
    "id": "39b6f339-ccae-4cb8-949d-dc7947333bf1"
   },
   "outputs": [],
   "source": [
    "def add_label2_to_trainDF(trainDF, product_groups_by_customer):\n",
    "    \"\"\"\n",
    "    write code here to add a new column to trainDF\n",
    "    the new column would be named label2 and it represents the label for the task 2\n",
    "    \n",
    "    After this function has been run, it can be safely assumed that trainDF dataframe contains another column called \"label2\"\n",
    "    that describes the label for the second prediction task.\n",
    "    \"\"\"\n",
    "    def has_customer_bought_in_product_group(row):\n",
    "        product_groups_set = product_groups_by_customer[row['customer_id']]['product_groups']\n",
    "        return int(row['product_group_name'] in product_groups_set)\n",
    "\n",
    "    trainDF['label2'] = trainDF.apply(has_customer_bought_in_product_group, axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "id": "473b2d1b-c45a-4900-ac5c-90bcf4c32e48",
   "metadata": {},
   "outputs": [],
   "source": [
    "add_label2_to_trainDF(trainDF, product_groups_by_customer)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "779b2661-13de-4a7e-b32f-e091cf033e54",
   "metadata": {},
   "source": [
    "__The negative example for the above customer tells us that he did not bought the exact article (label1 == 0) but did bought an article from the group 'Garment Upper body' (label2 == 1)__"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2642a24-090e-4180-9be0-e9bb27a89695",
   "metadata": {
    "id": "f2642a24-090e-4180-9be0-e9bb27a89695"
   },
   "source": [
    "Now that we have both the label1 and label2 in our dataset, we can construct a simple multi-task model with shared bottom, which we call the common layers. Building on top of the common layers, we will have two separate modules: task1 layers and task2 layers. Layers of task 1 (\"task1a\" and \"task1b\") are dedicated to learning weights that help us perform better on task1. Correspondingly, layers of task 2 (\"task2a\" and \"task2b\") are dedicated to learning weights that help us perform better on task2.\n",
    "\n",
    "Below we implement such a multi-task model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "27a995da-7cc6-4666-a8cf-81242efccb2f",
   "metadata": {},
   "outputs": [],
   "source": [
    "numFeatures = ['nArticles','nDays','nPurchases','minPrice','maxPrice','meanPrice','nPGroups','nDept','nSect']\n",
    "catFeatures = ['product_group_name']\n",
    "X1 = np.asarray(trainDF[numFeatures].values)\n",
    "X2 = np.asarray(trainDF[catFeatures].values)\n",
    "y1 = np.asarray(trainDF['label1'].values)\n",
    "y2 = np.asarray(trainDF['label2'].values)\n",
    "\n",
    "inp1 = tf.keras.Input((len(numFeatures)))\n",
    "inp2 = tf.keras.Input((len(catFeatures)), dtype=tf.string)\n",
    "\n",
    "unique_product_groups = trainDF['product_group_name'].unique().astype(str)\n",
    "\n",
    "# Transform categorical variable (embedding) \n",
    "emb_inp2 = tf.keras.layers.StringLookup(vocabulary=unique_product_groups, mask_token=None)(inp2)\n",
    "emb_inp2 = tf.keras.layers.Embedding(len(unique_product_groups) + 1, embedding_dimension)(emb_inp2)\n",
    "emb_inp2 = tf.keras.layers.Flatten()(emb_inp2)\n",
    "# Concatenate with numerical features\n",
    "inp = tf.keras.layers.concatenate([inp1, emb_inp2])\n",
    "# Traverse common module\n",
    "common = tf.keras.layers.Dense(128, activation='relu',name=\"common1\")(inp)\n",
    "common = tf.keras.layers.Dense(64, activation='relu',name=\"common2\")(common)\n",
    "\n",
    "\n",
    "bce = tf.keras.losses.BinaryCrossentropy(from_logits=False)\n",
    "metric = tf.keras.metrics.BinaryAccuracy()\n",
    "m = tf.keras.Model([inp1, inp2], out1)\n",
    "m.compile(loss=bce,\n",
    "          optimizer='adam',\n",
    "          metrics=[metric])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "id": "0790138b-204b-400e-b04d-73c605c4ea43",
   "metadata": {
    "id": "0790138b-204b-400e-b04d-73c605c4ea43",
    "outputId": "7d2d36e6-3adb-420b-e404-fa74d411e82f",
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/50\n",
      "120/120 [==============================] - 2s 4ms/step - loss: 0.9000 - out1_loss: 0.4681 - out2_loss: 0.4319 - out1_binary_accuracy: 0.7977 - out2_binary_accuracy: 0.7771\n",
      "Epoch 2/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.7336 - out1_loss: 0.3711 - out2_loss: 0.3625 - out1_binary_accuracy: 0.8325 - out2_binary_accuracy: 0.8354\n",
      "Epoch 3/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.7185 - out1_loss: 0.3628 - out2_loss: 0.3557 - out1_binary_accuracy: 0.8364 - out2_binary_accuracy: 0.8378\n",
      "Epoch 4/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.7142 - out1_loss: 0.3602 - out2_loss: 0.3540 - out1_binary_accuracy: 0.8361 - out2_binary_accuracy: 0.8387\n",
      "Epoch 5/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.7095 - out1_loss: 0.3576 - out2_loss: 0.3519 - out1_binary_accuracy: 0.8377 - out2_binary_accuracy: 0.8392\n",
      "Epoch 6/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.7046 - out1_loss: 0.3554 - out2_loss: 0.3493 - out1_binary_accuracy: 0.8384 - out2_binary_accuracy: 0.8403\n",
      "Epoch 7/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.7051 - out1_loss: 0.3557 - out2_loss: 0.3494 - out1_binary_accuracy: 0.8382 - out2_binary_accuracy: 0.8402\n",
      "Epoch 8/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.7001 - out1_loss: 0.3528 - out2_loss: 0.3473 - out1_binary_accuracy: 0.8392 - out2_binary_accuracy: 0.8408\n",
      "Epoch 9/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6994 - out1_loss: 0.3526 - out2_loss: 0.3468 - out1_binary_accuracy: 0.8391 - out2_binary_accuracy: 0.8402\n",
      "Epoch 10/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6970 - out1_loss: 0.3514 - out2_loss: 0.3456 - out1_binary_accuracy: 0.8395 - out2_binary_accuracy: 0.8416\n",
      "Epoch 11/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6965 - out1_loss: 0.3511 - out2_loss: 0.3453 - out1_binary_accuracy: 0.8397 - out2_binary_accuracy: 0.8417\n",
      "Epoch 12/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6947 - out1_loss: 0.3499 - out2_loss: 0.3448 - out1_binary_accuracy: 0.8403 - out2_binary_accuracy: 0.8417\n",
      "Epoch 13/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6931 - out1_loss: 0.3487 - out2_loss: 0.3444 - out1_binary_accuracy: 0.8400 - out2_binary_accuracy: 0.8424\n",
      "Epoch 14/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6939 - out1_loss: 0.3495 - out2_loss: 0.3444 - out1_binary_accuracy: 0.8402 - out2_binary_accuracy: 0.8421\n",
      "Epoch 15/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6907 - out1_loss: 0.3474 - out2_loss: 0.3433 - out1_binary_accuracy: 0.8411 - out2_binary_accuracy: 0.8425\n",
      "Epoch 16/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6911 - out1_loss: 0.3481 - out2_loss: 0.3430 - out1_binary_accuracy: 0.8403 - out2_binary_accuracy: 0.8437\n",
      "Epoch 17/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6876 - out1_loss: 0.3459 - out2_loss: 0.3417 - out1_binary_accuracy: 0.8420 - out2_binary_accuracy: 0.8436\n",
      "Epoch 18/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6878 - out1_loss: 0.3460 - out2_loss: 0.3418 - out1_binary_accuracy: 0.8408 - out2_binary_accuracy: 0.8436\n",
      "Epoch 19/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6878 - out1_loss: 0.3460 - out2_loss: 0.3418 - out1_binary_accuracy: 0.8410 - out2_binary_accuracy: 0.8436\n",
      "Epoch 20/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6845 - out1_loss: 0.3440 - out2_loss: 0.3405 - out1_binary_accuracy: 0.8427 - out2_binary_accuracy: 0.8443\n",
      "Epoch 21/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6838 - out1_loss: 0.3436 - out2_loss: 0.3403 - out1_binary_accuracy: 0.8427 - out2_binary_accuracy: 0.8444\n",
      "Epoch 22/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6841 - out1_loss: 0.3438 - out2_loss: 0.3403 - out1_binary_accuracy: 0.8430 - out2_binary_accuracy: 0.8443\n",
      "Epoch 23/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6837 - out1_loss: 0.3434 - out2_loss: 0.3403 - out1_binary_accuracy: 0.8435 - out2_binary_accuracy: 0.8444\n",
      "Epoch 24/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6828 - out1_loss: 0.3438 - out2_loss: 0.3390 - out1_binary_accuracy: 0.8426 - out2_binary_accuracy: 0.8453\n",
      "Epoch 25/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6822 - out1_loss: 0.3429 - out2_loss: 0.3394 - out1_binary_accuracy: 0.8425 - out2_binary_accuracy: 0.8452\n",
      "Epoch 26/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6797 - out1_loss: 0.3416 - out2_loss: 0.3381 - out1_binary_accuracy: 0.8441 - out2_binary_accuracy: 0.8452\n",
      "Epoch 27/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6807 - out1_loss: 0.3423 - out2_loss: 0.3384 - out1_binary_accuracy: 0.8430 - out2_binary_accuracy: 0.8451\n",
      "Epoch 28/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6792 - out1_loss: 0.3414 - out2_loss: 0.3377 - out1_binary_accuracy: 0.8428 - out2_binary_accuracy: 0.8450\n",
      "Epoch 29/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6785 - out1_loss: 0.3411 - out2_loss: 0.3374 - out1_binary_accuracy: 0.8431 - out2_binary_accuracy: 0.8458\n",
      "Epoch 30/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6769 - out1_loss: 0.3403 - out2_loss: 0.3366 - out1_binary_accuracy: 0.8439 - out2_binary_accuracy: 0.8467\n",
      "Epoch 31/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6760 - out1_loss: 0.3396 - out2_loss: 0.3364 - out1_binary_accuracy: 0.8438 - out2_binary_accuracy: 0.8462\n",
      "Epoch 32/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6769 - out1_loss: 0.3404 - out2_loss: 0.3364 - out1_binary_accuracy: 0.8439 - out2_binary_accuracy: 0.8462\n",
      "Epoch 33/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6756 - out1_loss: 0.3397 - out2_loss: 0.3360 - out1_binary_accuracy: 0.8439 - out2_binary_accuracy: 0.8463\n",
      "Epoch 34/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6740 - out1_loss: 0.3388 - out2_loss: 0.3352 - out1_binary_accuracy: 0.8445 - out2_binary_accuracy: 0.8468\n",
      "Epoch 35/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6761 - out1_loss: 0.3401 - out2_loss: 0.3360 - out1_binary_accuracy: 0.8443 - out2_binary_accuracy: 0.8465\n",
      "Epoch 36/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6733 - out1_loss: 0.3381 - out2_loss: 0.3352 - out1_binary_accuracy: 0.8451 - out2_binary_accuracy: 0.8462\n",
      "Epoch 37/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6736 - out1_loss: 0.3384 - out2_loss: 0.3353 - out1_binary_accuracy: 0.8442 - out2_binary_accuracy: 0.8466\n",
      "Epoch 38/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6717 - out1_loss: 0.3371 - out2_loss: 0.3346 - out1_binary_accuracy: 0.8453 - out2_binary_accuracy: 0.8466\n",
      "Epoch 39/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6715 - out1_loss: 0.3370 - out2_loss: 0.3345 - out1_binary_accuracy: 0.8455 - out2_binary_accuracy: 0.8467\n",
      "Epoch 40/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6729 - out1_loss: 0.3380 - out2_loss: 0.3349 - out1_binary_accuracy: 0.8448 - out2_binary_accuracy: 0.8458\n",
      "Epoch 41/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6705 - out1_loss: 0.3365 - out2_loss: 0.3340 - out1_binary_accuracy: 0.8460 - out2_binary_accuracy: 0.8470\n",
      "Epoch 42/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6689 - out1_loss: 0.3353 - out2_loss: 0.3335 - out1_binary_accuracy: 0.8461 - out2_binary_accuracy: 0.8473\n",
      "Epoch 43/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6692 - out1_loss: 0.3356 - out2_loss: 0.3336 - out1_binary_accuracy: 0.8463 - out2_binary_accuracy: 0.8470\n",
      "Epoch 44/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6690 - out1_loss: 0.3351 - out2_loss: 0.3339 - out1_binary_accuracy: 0.8459 - out2_binary_accuracy: 0.8467\n",
      "Epoch 45/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6676 - out1_loss: 0.3341 - out2_loss: 0.3335 - out1_binary_accuracy: 0.8470 - out2_binary_accuracy: 0.8474\n",
      "Epoch 46/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6661 - out1_loss: 0.3336 - out2_loss: 0.3325 - out1_binary_accuracy: 0.8472 - out2_binary_accuracy: 0.8481\n",
      "Epoch 47/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6657 - out1_loss: 0.3330 - out2_loss: 0.3327 - out1_binary_accuracy: 0.8470 - out2_binary_accuracy: 0.8478\n",
      "Epoch 48/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6649 - out1_loss: 0.3327 - out2_loss: 0.3321 - out1_binary_accuracy: 0.8472 - out2_binary_accuracy: 0.8479\n",
      "Epoch 49/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6647 - out1_loss: 0.3324 - out2_loss: 0.3323 - out1_binary_accuracy: 0.8480 - out2_binary_accuracy: 0.8478\n",
      "Epoch 50/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6639 - out1_loss: 0.3321 - out2_loss: 0.3318 - out1_binary_accuracy: 0.8482 - out2_binary_accuracy: 0.8477\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x1dbce5e20>"
      ]
     },
     "execution_count": 143,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "embedding_dimension=50\n",
    "\n",
    "numFeatures = ['nArticles','nDays','nPurchases','minPrice','maxPrice','meanPrice','nPGroups','nDept','nSect']\n",
    "catFeatures = ['product_group_name']\n",
    "X1 = np.asarray(trainDF[numFeatures].values)\n",
    "X2 = np.asarray(trainDF[catFeatures].values)\n",
    "y1 = np.asarray(trainDF['label1'].values)\n",
    "y2 = np.asarray(trainDF['label2'].values)\n",
    "\n",
    "inp1 = tf.keras.Input((len(numFeatures)))\n",
    "inp2 = tf.keras.Input((len(catFeatures)), dtype=tf.string)\n",
    "\n",
    "unique_product_groups = trainDF['product_group_name'].unique().astype(str)\n",
    "\n",
    "\n",
    "# Transform categorical variable (embedding) \n",
    "emb_inp2 = tf.keras.layers.StringLookup(vocabulary=unique_product_groups, mask_token=None)(inp2)\n",
    "emb_inp2 = tf.keras.layers.Embedding(len(unique_product_groups) + 1, embedding_dimension)(emb_inp2)\n",
    "emb_inp2 = tf.keras.layers.Flatten()(emb_inp2)\n",
    "\n",
    "# Concatenate with numerical features\n",
    "inp = tf.keras.layers.concatenate([inp1, emb_inp2])\n",
    "\n",
    "# Traverse common module\n",
    "common = tf.keras.layers.Dense(128, activation='relu',name=\"common1\")(inp)\n",
    "common = tf.keras.layers.Dense(64, activation='relu',name=\"common2\")(common)\n",
    "\n",
    "# Tower for task 1\n",
    "x1 = tf.keras.layers.Dense(32, activation='relu',name=\"task1a\")(common)\n",
    "x1 = tf.keras.layers.Dense(16, activation='relu',name=\"task1b\")(x1)\n",
    "out1 = tf.keras.layers.Dense(1, activation='sigmoid',name=\"out1\")(x1)\n",
    "\n",
    "# Tower for task 2\n",
    "x2 = tf.keras.layers.Dense(32, activation='relu',name=\"task2a\")(common)\n",
    "x2 = tf.keras.layers.Dense(16, activation='relu',name=\"task2b\")(x2)\n",
    "out2 = tf.keras.layers.Dense(1, activation='sigmoid',name=\"out2\")(x2)\n",
    "\n",
    "bce = tf.keras.losses.BinaryCrossentropy(from_logits=False)\n",
    "metric = tf.keras.metrics.BinaryAccuracy()\n",
    "m2 = tf.keras.Model([inp1, inp2], [out1, out2])\n",
    "m2.compile(loss=bce,\n",
    "           optimizer='adam',\n",
    "           metrics=[metric])\n",
    "\n",
    "m2.fit([X1, X2], [y1, y2], epochs=50, verbose=1, batch_size=1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "id": "7a1dc2e8-e39d-4635-b5db-3fd9f36c0795",
   "metadata": {
    "id": "7a1dc2e8-e39d-4635-b5db-3fd9f36c0795",
    "outputId": "90d7675d-0d80-4c07-dd43-433fe7024efe"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model_17\"\n",
      "__________________________________________________________________________________________________\n",
      " Layer (type)                   Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      " input_35 (InputLayer)          [(None, 1)]          0           []                               \n",
      "                                                                                                  \n",
      " string_lookup_13 (StringLookup  (None, 1)           0           ['input_35[0][0]']               \n",
      " )                                                                                                \n",
      "                                                                                                  \n",
      " embedding_13 (Embedding)       (None, 1, 50)        950         ['string_lookup_13[0][0]']       \n",
      "                                                                                                  \n",
      " input_34 (InputLayer)          [(None, 9)]          0           []                               \n",
      "                                                                                                  \n",
      " flatten_13 (Flatten)           (None, 50)           0           ['embedding_13[0][0]']           \n",
      "                                                                                                  \n",
      " concatenate_12 (Concatenate)   (None, 59)           0           ['input_34[0][0]',               \n",
      "                                                                  'flatten_13[0][0]']             \n",
      "                                                                                                  \n",
      " common1 (Dense)                (None, 128)          7680        ['concatenate_12[0][0]']         \n",
      "                                                                                                  \n",
      " common2 (Dense)                (None, 64)           8256        ['common1[0][0]']                \n",
      "                                                                                                  \n",
      " task1a (Dense)                 (None, 32)           2080        ['common2[0][0]']                \n",
      "                                                                                                  \n",
      " task2a (Dense)                 (None, 32)           2080        ['common2[0][0]']                \n",
      "                                                                                                  \n",
      " task1b (Dense)                 (None, 16)           528         ['task1a[0][0]']                 \n",
      "                                                                                                  \n",
      " task2b (Dense)                 (None, 16)           528         ['task2a[0][0]']                 \n",
      "                                                                                                  \n",
      " out1 (Dense)                   (None, 1)            17          ['task1b[0][0]']                 \n",
      "                                                                                                  \n",
      " out2 (Dense)                   (None, 1)            17          ['task2b[0][0]']                 \n",
      "                                                                                                  \n",
      "==================================================================================================\n",
      "Total params: 22,136\n",
      "Trainable params: 22,136\n",
      "Non-trainable params: 0\n",
      "__________________________________________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "m2.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5866181-5319-472a-8b85-0004fa8c652e",
   "metadata": {
    "id": "f5866181-5319-472a-8b85-0004fa8c652e"
   },
   "source": [
    "### Checkpoint & goals\n",
    "By now you should have implemented the function **def add_label2_to_trainDF()** and replaced label1 in the above code by label2 and trained the multi-task model with these 2 tasks.\n",
    "\n",
    "Please record the performance of both the single task model and multi-task model and provide your interpretation of the results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ab845e8-0d97-4679-9992-14c0a80b8331",
   "metadata": {
    "id": "d030b3d7-9323-4ee1-8cf8-cc99c2048b44"
   },
   "source": [
    "So we came up with 2 models for the single task ranker:\n",
    "1. The first one only uses the numerical features. With 50 epochs of training, we come up with a binary accuracy of 0.8303._\n",
    "2. The second one also uses a categorical feature \"product_name_group\" that is embedded into a 50 dimensional space. Adding this feature improves binary accuracy up to 0.8428 at epoch 50. The categorical feature is an important one to have\n",
    "\n",
    "For the multi-task ranker, we also use the categorical feature. The new task which is to predict whether the user will buy an item of the same product group helps the model and improves the prediction accuracy for task 1. The binary accuracy reaches 0.8482 at epoch 50. This is because task 1 and 2 share a lot of commonalities. Being good at predicting whether a user will buy in a specific product group, helps to predict whether a user will buy a specific product. Task 2 guides the learning in the proper direction.\n",
    "\n",
    "NB: The above results is to be mitigated by the fact that the second multi-task model has a lot more parameters. Having more capacity to learn necessarily helps. More experiments should be conducated to assess this point and make a comparison on a fairer ground"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0478c09-91fa-404d-84f5-34ae7fe9f359",
   "metadata": {
    "id": "d0478c09-91fa-404d-84f5-34ae7fe9f359"
   },
   "source": [
    "## Freezing layers for secondary tasks\n",
    "\n",
    "Often when training a multi-task model, one needs to freeze parts of the nueral network and only train one specific task module, while keeping the weights of the other task module fixed. In Keras we can do this by freezing layers. The below example demonstrates how one could go about freezing layers in a neural network, and train only the trainable parts of a nueral network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "id": "3b40d34e-fa1c-4e09-b29a-7df832a3746b",
   "metadata": {
    "id": "3b40d34e-fa1c-4e09-b29a-7df832a3746b",
    "outputId": "2d75b525-0af3-46c8-b1de-e37c02bdb66a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<keras.engine.functional.Functional object at 0x1d4f84700> True\n",
      "<keras.engine.input_layer.InputLayer object at 0x1d5961640> True\n",
      "<keras.layers.preprocessing.string_lookup.StringLookup object at 0x1ba1b0220> True\n",
      "<keras.layers.core.embedding.Embedding object at 0x1ba119a90> True\n",
      "<keras.engine.input_layer.InputLayer object at 0x1d5961670> True\n",
      "<keras.layers.reshaping.flatten.Flatten object at 0x1d568a7f0> True\n",
      "<keras.layers.merging.concatenate.Concatenate object at 0x1d82679a0> True\n",
      "<keras.layers.core.dense.Dense object at 0x1d576e040> True\n",
      "<keras.layers.core.dense.Dense object at 0x1d56fc4c0> True\n",
      "<keras.layers.core.dense.Dense object at 0x1d0adfb20> True\n",
      "<keras.layers.core.dense.Dense object at 0x1d2c97c40> True\n",
      "<keras.layers.core.dense.Dense object at 0x1d56fc4f0> True\n",
      "<keras.layers.core.dense.Dense object at 0x1d2c1a550> True\n",
      "<keras.layers.core.dense.Dense object at 0x1ba9a00d0> True\n",
      "<keras.layers.core.dense.Dense object at 0x1d4fafd60> True\n"
     ]
    }
   ],
   "source": [
    "# lets print the training status of each layer of the models we have\n",
    "for k,v in m2._get_trainable_state().items():\n",
    "    print(k,v)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "id": "4842f73e-3367-4b47-890d-93936b18b212",
   "metadata": {
    "id": "4842f73e-3367-4b47-890d-93936b18b212"
   },
   "outputs": [],
   "source": [
    "# Lets try to freeze layers for task 2 so that no training example affects the module dedicated to task 2\n",
    "for layer in m2.layers:\n",
    "    if layer.name in [\"task2a\",\"task2b\"]:\n",
    "        layer.trainable = False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "id": "2b01277c-eb22-47b7-8bb7-7d1a106543c8",
   "metadata": {
    "id": "2b01277c-eb22-47b7-8bb7-7d1a106543c8",
    "outputId": "5ac9fff2-8294-4019-8a02-3197e0a39542"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<keras.engine.functional.Functional object at 0x1d4f84700> True\n",
      "<keras.engine.input_layer.InputLayer object at 0x1d5961640> True\n",
      "<keras.layers.preprocessing.string_lookup.StringLookup object at 0x1ba1b0220> True\n",
      "<keras.layers.core.embedding.Embedding object at 0x1ba119a90> True\n",
      "<keras.engine.input_layer.InputLayer object at 0x1d5961670> True\n",
      "<keras.layers.reshaping.flatten.Flatten object at 0x1d568a7f0> True\n",
      "<keras.layers.merging.concatenate.Concatenate object at 0x1d82679a0> True\n",
      "<keras.layers.core.dense.Dense object at 0x1d576e040> True\n",
      "<keras.layers.core.dense.Dense object at 0x1d56fc4c0> True\n",
      "<keras.layers.core.dense.Dense object at 0x1d0adfb20> True\n",
      "<keras.layers.core.dense.Dense object at 0x1d2c97c40> False\n",
      "<keras.layers.core.dense.Dense object at 0x1d56fc4f0> True\n",
      "<keras.layers.core.dense.Dense object at 0x1d2c1a550> False\n",
      "<keras.layers.core.dense.Dense object at 0x1ba9a00d0> True\n",
      "<keras.layers.core.dense.Dense object at 0x1d4fafd60> True\n"
     ]
    }
   ],
   "source": [
    "for k,v in m2._get_trainable_state().items():\n",
    "    print(k,v)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "id": "cf305533-a7bf-4dd5-aa5d-c02f27f33897",
   "metadata": {
    "id": "cf305533-a7bf-4dd5-aa5d-c02f27f33897",
    "outputId": "cfe0240a-08e7-4e2e-f6b0-4c92aad1eccc",
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/50\n",
      "120/120 [==============================] - 2s 5ms/step - loss: 0.6640 - out1_loss: 0.3321 - out2_loss: 0.3319 - out1_binary_accuracy: 0.8482 - out2_binary_accuracy: 0.8480\n",
      "Epoch 2/50\n",
      "120/120 [==============================] - 1s 4ms/step - loss: 0.6623 - out1_loss: 0.3315 - out2_loss: 0.3308 - out1_binary_accuracy: 0.8476 - out2_binary_accuracy: 0.8485\n",
      "Epoch 3/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6611 - out1_loss: 0.3302 - out2_loss: 0.3309 - out1_binary_accuracy: 0.8490 - out2_binary_accuracy: 0.8482\n",
      "Epoch 4/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6599 - out1_loss: 0.3297 - out2_loss: 0.3302 - out1_binary_accuracy: 0.8494 - out2_binary_accuracy: 0.8487\n",
      "Epoch 5/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6596 - out1_loss: 0.3293 - out2_loss: 0.3303 - out1_binary_accuracy: 0.8488 - out2_binary_accuracy: 0.8485\n",
      "Epoch 6/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6599 - out1_loss: 0.3295 - out2_loss: 0.3304 - out1_binary_accuracy: 0.8484 - out2_binary_accuracy: 0.8485\n",
      "Epoch 7/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6592 - out1_loss: 0.3287 - out2_loss: 0.3304 - out1_binary_accuracy: 0.8494 - out2_binary_accuracy: 0.8487\n",
      "Epoch 8/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6590 - out1_loss: 0.3281 - out2_loss: 0.3309 - out1_binary_accuracy: 0.8493 - out2_binary_accuracy: 0.8486\n",
      "Epoch 9/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6581 - out1_loss: 0.3282 - out2_loss: 0.3299 - out1_binary_accuracy: 0.8491 - out2_binary_accuracy: 0.8488\n",
      "Epoch 10/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6568 - out1_loss: 0.3271 - out2_loss: 0.3298 - out1_binary_accuracy: 0.8501 - out2_binary_accuracy: 0.8484\n",
      "Epoch 11/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6572 - out1_loss: 0.3276 - out2_loss: 0.3296 - out1_binary_accuracy: 0.8499 - out2_binary_accuracy: 0.8490\n",
      "Epoch 12/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6565 - out1_loss: 0.3268 - out2_loss: 0.3297 - out1_binary_accuracy: 0.8495 - out2_binary_accuracy: 0.8488\n",
      "Epoch 13/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6553 - out1_loss: 0.3263 - out2_loss: 0.3290 - out1_binary_accuracy: 0.8507 - out2_binary_accuracy: 0.8492\n",
      "Epoch 14/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6553 - out1_loss: 0.3256 - out2_loss: 0.3297 - out1_binary_accuracy: 0.8498 - out2_binary_accuracy: 0.8489\n",
      "Epoch 15/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6540 - out1_loss: 0.3249 - out2_loss: 0.3292 - out1_binary_accuracy: 0.8509 - out2_binary_accuracy: 0.8489\n",
      "Epoch 16/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6537 - out1_loss: 0.3247 - out2_loss: 0.3290 - out1_binary_accuracy: 0.8510 - out2_binary_accuracy: 0.8490\n",
      "Epoch 17/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6573 - out1_loss: 0.3273 - out2_loss: 0.3300 - out1_binary_accuracy: 0.8503 - out2_binary_accuracy: 0.8484\n",
      "Epoch 18/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6553 - out1_loss: 0.3257 - out2_loss: 0.3296 - out1_binary_accuracy: 0.8508 - out2_binary_accuracy: 0.8487\n",
      "Epoch 19/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6537 - out1_loss: 0.3246 - out2_loss: 0.3291 - out1_binary_accuracy: 0.8508 - out2_binary_accuracy: 0.8493\n",
      "Epoch 20/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6527 - out1_loss: 0.3237 - out2_loss: 0.3290 - out1_binary_accuracy: 0.8515 - out2_binary_accuracy: 0.8495\n",
      "Epoch 21/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6520 - out1_loss: 0.3231 - out2_loss: 0.3289 - out1_binary_accuracy: 0.8517 - out2_binary_accuracy: 0.8495\n",
      "Epoch 22/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6509 - out1_loss: 0.3222 - out2_loss: 0.3287 - out1_binary_accuracy: 0.8513 - out2_binary_accuracy: 0.8500\n",
      "Epoch 23/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6514 - out1_loss: 0.3228 - out2_loss: 0.3286 - out1_binary_accuracy: 0.8523 - out2_binary_accuracy: 0.8493\n",
      "Epoch 24/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6511 - out1_loss: 0.3224 - out2_loss: 0.3287 - out1_binary_accuracy: 0.8519 - out2_binary_accuracy: 0.8494\n",
      "Epoch 25/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6485 - out1_loss: 0.3204 - out2_loss: 0.3281 - out1_binary_accuracy: 0.8524 - out2_binary_accuracy: 0.8497\n",
      "Epoch 26/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6500 - out1_loss: 0.3217 - out2_loss: 0.3283 - out1_binary_accuracy: 0.8531 - out2_binary_accuracy: 0.8494\n",
      "Epoch 27/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6500 - out1_loss: 0.3214 - out2_loss: 0.3286 - out1_binary_accuracy: 0.8522 - out2_binary_accuracy: 0.8495\n",
      "Epoch 28/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6502 - out1_loss: 0.3216 - out2_loss: 0.3286 - out1_binary_accuracy: 0.8518 - out2_binary_accuracy: 0.8492\n",
      "Epoch 29/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6488 - out1_loss: 0.3204 - out2_loss: 0.3284 - out1_binary_accuracy: 0.8530 - out2_binary_accuracy: 0.8493\n",
      "Epoch 30/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6466 - out1_loss: 0.3189 - out2_loss: 0.3277 - out1_binary_accuracy: 0.8534 - out2_binary_accuracy: 0.8499\n",
      "Epoch 31/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6468 - out1_loss: 0.3187 - out2_loss: 0.3281 - out1_binary_accuracy: 0.8538 - out2_binary_accuracy: 0.8497\n",
      "Epoch 32/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6479 - out1_loss: 0.3198 - out2_loss: 0.3281 - out1_binary_accuracy: 0.8531 - out2_binary_accuracy: 0.8498\n",
      "Epoch 33/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6460 - out1_loss: 0.3182 - out2_loss: 0.3277 - out1_binary_accuracy: 0.8545 - out2_binary_accuracy: 0.8497\n",
      "Epoch 34/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6456 - out1_loss: 0.3181 - out2_loss: 0.3275 - out1_binary_accuracy: 0.8540 - out2_binary_accuracy: 0.8502\n",
      "Epoch 35/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6440 - out1_loss: 0.3169 - out2_loss: 0.3271 - out1_binary_accuracy: 0.8542 - out2_binary_accuracy: 0.8501\n",
      "Epoch 36/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6447 - out1_loss: 0.3171 - out2_loss: 0.3277 - out1_binary_accuracy: 0.8544 - out2_binary_accuracy: 0.8498\n",
      "Epoch 37/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6433 - out1_loss: 0.3162 - out2_loss: 0.3271 - out1_binary_accuracy: 0.8538 - out2_binary_accuracy: 0.8504\n",
      "Epoch 38/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6440 - out1_loss: 0.3163 - out2_loss: 0.3277 - out1_binary_accuracy: 0.8547 - out2_binary_accuracy: 0.8500\n",
      "Epoch 39/50\n",
      "120/120 [==============================] - 0s 4ms/step - loss: 0.6432 - out1_loss: 0.3162 - out2_loss: 0.3270 - out1_binary_accuracy: 0.8547 - out2_binary_accuracy: 0.8499\n",
      "Epoch 40/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6427 - out1_loss: 0.3149 - out2_loss: 0.3278 - out1_binary_accuracy: 0.8545 - out2_binary_accuracy: 0.8496\n",
      "Epoch 41/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6425 - out1_loss: 0.3152 - out2_loss: 0.3273 - out1_binary_accuracy: 0.8551 - out2_binary_accuracy: 0.8497\n",
      "Epoch 42/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6417 - out1_loss: 0.3147 - out2_loss: 0.3271 - out1_binary_accuracy: 0.8558 - out2_binary_accuracy: 0.8500\n",
      "Epoch 43/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6431 - out1_loss: 0.3156 - out2_loss: 0.3275 - out1_binary_accuracy: 0.8550 - out2_binary_accuracy: 0.8497\n",
      "Epoch 44/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6421 - out1_loss: 0.3151 - out2_loss: 0.3270 - out1_binary_accuracy: 0.8548 - out2_binary_accuracy: 0.8501\n",
      "Epoch 45/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6399 - out1_loss: 0.3131 - out2_loss: 0.3268 - out1_binary_accuracy: 0.8560 - out2_binary_accuracy: 0.8501\n",
      "Epoch 46/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6409 - out1_loss: 0.3140 - out2_loss: 0.3269 - out1_binary_accuracy: 0.8553 - out2_binary_accuracy: 0.8499\n",
      "Epoch 47/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6397 - out1_loss: 0.3129 - out2_loss: 0.3269 - out1_binary_accuracy: 0.8560 - out2_binary_accuracy: 0.8498\n",
      "Epoch 48/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6403 - out1_loss: 0.3134 - out2_loss: 0.3270 - out1_binary_accuracy: 0.8561 - out2_binary_accuracy: 0.8504\n",
      "Epoch 49/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6396 - out1_loss: 0.3129 - out2_loss: 0.3268 - out1_binary_accuracy: 0.8561 - out2_binary_accuracy: 0.8499\n",
      "Epoch 50/50\n",
      "120/120 [==============================] - 0s 3ms/step - loss: 0.6385 - out1_loss: 0.3119 - out2_loss: 0.3266 - out1_binary_accuracy: 0.8562 - out2_binary_accuracy: 0.8500\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x1dbccc6a0>"
      ]
     },
     "execution_count": 148,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m2.compile(loss=bce,\n",
    "           optimizer='adam',\n",
    "           metrics=[metric])\n",
    "\n",
    "m2.fit([X1, X2], [y1,y2], epochs=50, verbose=1, batch_size=1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9e9b0509-3d51-46a6-84be-7a2069d0ad5f",
   "metadata": {
    "id": "9e9b0509-3d51-46a6-84be-7a2069d0ad5f"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "48cccb53-490d-48fe-89f9-99895bdfc50d",
   "metadata": {
    "id": "48cccb53-490d-48fe-89f9-99895bdfc50d"
   },
   "source": [
    "## Tasks for Week 2 project:\n",
    "\n",
    "So glad that you have made it this far!! The above gives a high level view on how one could train single, and multi-task models and how one could go about freezing layers and training parts of a multi-task network.\n",
    "\n",
    "Lets re-iterate the main goal for this part of the project 2:\n",
    "\n",
    "- Goal: Implement the function that assigns label2 to the trainDF dataframe, and run the single task and multi-tsk models without layer freezing. Once done, compare the performance metrics for task 1 from both these models. Comment on whether the addition of second task is adding value to predictive power of first task.\n",
    "\n",
    "Optional: you can notice that we have used only numeric features in the model. Feel free to add other categorical features and re-train the model to see if the performance increases."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "649306f5-6cd8-4339-9921-41f6de1dc948",
   "metadata": {
    "id": "649306f5-6cd8-4339-9921-41f6de1dc948"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "06a6c993-12a8-4600-9881-cd1c326d3859",
   "metadata": {
    "id": "06a6c993-12a8-4600-9881-cd1c326d3859"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "d21b257c-d1e8-42c8-8926-86b8ee5ff88b",
   "metadata": {
    "id": "d21b257c-d1e8-42c8-8926-86b8ee5ff88b"
   },
   "source": [
    "# Contextual Bandits for Recommendations\n",
    "\n",
    "We can also use contextual bandits to produce personalized recommendations. Compared to multi-task recommenders, contextual bandits provide a balance between exploring new options and exploiting known information to find potential recommendations.\n",
    "\n",
    "## Background: multi-arm bandits\n",
    "The multi-armed bandit problem is an example of one-step reinforcement learning. To explore an example, lets assume we have a gambler who is provided a slot machine with multiple arms, each with its own unknown probability distribution of payouts. The objective is to pull the arms one-by-one in a sequence while gathering information in order to maximize the total payout over the long-run. The multi-armed bandit problem can be seen as a toy problem for reinforcement learning with one step rollout. Namely we have a game consisting of n rounds and in each round t:\n",
    "\n",
    "1. Player selects one of K actions (think of slot machines and pulling their arms, hence the name).\n",
    "2. Player gets reward of Rt . Each action i ∈ {1,2,…,K } has a fixed, but unknown to the player, reward distribution Pi with the expected reward μi .\n",
    "3. Given the history of actions and rewards Player updates their strategy.\n",
    "\n",
    "Given the history of actions and rewards, the player updates their strategy. The goal of the player is to maximize the reward, which naturally has to be done by exploring new options (and thus learning about other machines' distributions) and exploiting known actions that proved to give high rewards so far. Mathematically the player wishes to minimize the regret.\n",
    "\n",
    "## Contextual multi-arm bandits\n",
    "The contextual bandit problem is a generalization of the multi-armed bandit that extends the model by making actions conditional on the state of the environment. Unlike the classical multi-armed bandit, it addresses the problem of identifying the most appropriate content given all relevant contextual signals.\n",
    "\n",
    "1. Player gets context xt .\n",
    "2. Player selects at ∈ A .\n",
    "3. Player gets reward Rt(at) .\n",
    "4. Given the history of actions, rewards and contexts Player updates their strategy.\n",
    "\n",
    "A common real-world contextual bandit example is a recommendation system. Given a set of presented articles, a reward is determined by the click-through behavior of the user. If user clicks on the article, a payout of 1 is incurred and 0 otherwise. Click-through-rate (CRT) is used to determine the selection and placement of ads within the recommendation application.\n",
    "\n",
    "\n",
    "In explore phase the algorithm, given its internal estimates of goodness of each actions chooses the best possible action modulo exploration. For example, epsilon-greedy chooses the best action with probability (1−ϵ) and with probability ϵ it selects uniformly one of all possible actions.\n",
    "\n",
    "In the learning phase, the algorithm, after it selects an action and obtains a reward, updates its estimates of expected rewards of actions given the context. This is usually done via feeding a single example consisting of x,a,r (context, action, reward) to an estimator model doing a single update step (batch size 1). The learn phase is usually common for most algorithms and the main differences come from the exploration phase. Some algorithms include ϵ-greedy, ensemble of policies or approximations to Lin-UCB. \n",
    "ϵ-greedy tends to be a practical choice for initial experiments since it’s straightforward to understand and control its exploration. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc685243-6acc-460e-b6ba-15aa265daf97",
   "metadata": {
    "id": "dc685243-6acc-460e-b6ba-15aa265daf97"
   },
   "source": [
    "## Dataset: R6A - Yahoo! Front Page Today Module User Click Log Dataset, version 1.0 (1.1 GB)\n",
    "\n",
    "Our dataset contains a fraction of user click log for news articles displayed in the Featured Tab of the Today Module on Yahoo! Front Page (http://www.yahoo.com) during the first ten days in May 2009.  The articles were chosen uniformly at random from a hand-picked pool of high-quality articles, which allows one to use a recently developed method to obtain an unbiased evaluation of an arbitrary bandit algorithm.\n",
    "\n",
    "The dataset contains 45,811,883 user visits to the Today Module.  For each visit, both the user and each of the candidate articles are\n",
    "associated with a feature vector of dimension 6 (including a constant feature), constructed by conjoint analysis with a bilinear model.\n",
    "\n",
    "This dataset contains 10 files, corresponding to the first 10 days in May 2009:\n",
    "    ydata-fp-td-clicks-v1_0.20090501.gz\n",
    "    ydata-fp-td-clicks-v1_0.20090502.gz\n",
    "    ...\n",
    "    ydata-fp-td-clicks-v1_0.20090510.gz\n",
    "Each line in the files corresponds to a separate user visit.  An example line is as follows:\n",
    "\n",
    "1241160900 109513 0 |user 2:0.000012 3:0.000000 4:0.000006 5:0.000023 6:0.999958 1:1.000000 |109498 2:0.306008 3:0.000450 4:0.077048 5:0.230439 6:0.386055 1:1.000000 |109509 2:0.306008 3:0.000450 4:0.077048 5:0.230439 6:0.386055 1:1.000000 [[...more article features omitted...]] |109453 2:0.421669 3:0.000011 4:0.010902 5:0.309585 6:0.257833 1:1.000000\n",
    "\n",
    "which contains the following fields delimited with spaces:\n",
    "\n",
    "    * timestamp: e.g., 1241160900\n",
    "    * displayed_article_id: e.g., 109513\n",
    "    * user_click (0 for no-click and 1 for click): e.g., 0\n",
    "    * strings \"|user\" and \"|{article_id}\" indicate the start of user\n",
    "      and article features\n",
    "    * features are encoded as \"feature_id:feature_value\" pairs, and\n",
    "      feature_id starts from 1.\n",
    "\n",
    "The pool of available articles for recommendation for each user visit is the set of articles that appear in that line of data.  All user IDs (specifically, bcookies) are replaced by a common string 'user' so that no user information can be identified from this data.\n",
    "\n",
    "Each user or article is associated with six features.  Feature #1 is the constant (always 1) feature, and features #2-6 correspond to the 5 membership features constructed via conjoint analysis with a bilinear model.\n",
    "\n",
    "A unique property of this data set is that the displayed article is chosen uniformly at random from the candidate article pool.  Therefore, one can use an unbiased *offline* evaluation method to compare bandit algorithms in a reliable way.  Performance of some of the popular bandit algorithms can be found. We will cover the offline evaluation method in detail in week 4 of the course.\n",
    "\n",
    "The full dataset can be downloaded from:\n",
    "https://webscope.sandbox.yahoo.com/catalog.php?datatype=r&did=49\n",
    "We are going to work with a much smalled sample of this data, which can be downloaded from the Google drive link (data1.txt):\n",
    "https://drive.google.com/drive/folders/10LGZMgXRuz2qPr_QDbYdVVlKEcnQ25YL?usp=sharing\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "id": "54535a2f-7d82-489b-aa3f-79d51e3d441e",
   "metadata": {
    "id": "54535a2f-7d82-489b-aa3f-79d51e3d441e"
   },
   "outputs": [],
   "source": [
    "files = (\"yahoo/data1.txt\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d57d965-645f-4622-b721-7cd11500fc67",
   "metadata": {
    "id": "8d57d965-645f-4622-b721-7cd11500fc67"
   },
   "source": [
    "Now lets write a function to read this file, and store the data in articles / features / events -- which we will treat as global variables accessible to all functions later on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 153,
   "id": "a362ca4c-5f65-4f51-99c7-43b62fe6043f",
   "metadata": {
    "id": "a362ca4c-5f65-4f51-99c7-43b62fe6043f"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import fileinput\n",
    "\n",
    "def read_data(filenames):\n",
    "    \"\"\"\n",
    "    Reads a stream of events from the list of given files.\n",
    "    \n",
    "    Parameters\n",
    "    ----------\n",
    "    filenames : list\n",
    "        List of filenames\n",
    "    \n",
    "    Stores\n",
    "    -------    \n",
    "    articles : [article_ids]\n",
    "    features : [[article_1_features] .. [article_n_features]]\n",
    "    events : [\n",
    "                 0 : displayed_article_index (relative to the pool),\n",
    "                 1 : user_click,\n",
    "                 2 : [user_features],\n",
    "                 3 : [pool_indexes]\n",
    "             ]\n",
    "    \"\"\"\n",
    "\n",
    "    global articles, features, events, n_arms, n_events\n",
    "    articles = []\n",
    "    features = []\n",
    "    events = []\n",
    "\n",
    "    skipped = 0\n",
    "\n",
    "    with fileinput.input(files=filenames) as f:\n",
    "        for line in f:\n",
    "            cols = line.split()\n",
    "            if (len(cols) - 10) % 7 != 0:\n",
    "                skipped += 1\n",
    "            else:\n",
    "                pool_idx = []\n",
    "                pool_ids = []\n",
    "                for i in range(10, len(cols) - 6, 7):\n",
    "                    id = cols[i][1:]\n",
    "                    if id not in articles:\n",
    "                        articles.append(id)\n",
    "                        features.append([float(x[2:]) for x in cols[i + 1: i + 7]])\n",
    "                    pool_idx.append(articles.index(id))\n",
    "                    pool_ids.append(id)\n",
    "\n",
    "                events.append(\n",
    "                    [\n",
    "                        pool_ids.index(cols[1]),\n",
    "                        int(cols[2]),\n",
    "                        [float(x[2:]) for x in cols[4:10]],\n",
    "                        pool_idx,\n",
    "                    ]\n",
    "                )\n",
    "    features = np.array(features)\n",
    "    n_arms = len(articles)\n",
    "    n_events = len(events)\n",
    "    print(n_events, \"events with\", n_arms, \"articles\")\n",
    "    if skipped != 0:\n",
    "        print(\"Skipped events:\", skipped)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "888460da-4e01-4168-9573-761cf686786f",
   "metadata": {
    "id": "888460da-4e01-4168-9573-761cf686786f"
   },
   "source": [
    "Now lets read the file with the above defined function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "id": "07df3304-6701-47ff-9d0d-d5a59a5b8cb3",
   "metadata": {
    "id": "07df3304-6701-47ff-9d0d-d5a59a5b8cb3",
    "outputId": "5e318d9b-be7f-4377-c533-9bb80e98e16d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100000 events with 21 articles\n"
     ]
    }
   ],
   "source": [
    "read_data(files)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "63d941ee-efe5-40ab-83e5-f2152a3e86a1",
   "metadata": {
    "id": "63d941ee-efe5-40ab-83e5-f2152a3e86a1"
   },
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d943757e-660c-4648-bd50-1f5c17514e45",
   "metadata": {
    "id": "d943757e-660c-4648-bd50-1f5c17514e45"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "95e2039e-43ba-4a09-bbc9-b2b12687e722",
   "metadata": {
    "id": "95e2039e-43ba-4a09-bbc9-b2b12687e722"
   },
   "source": [
    "## ϵ-greedy policy implementation\n",
    "Having read the files, now lets implement a simple ϵ-greedy method. Here ϵ defines the amount of exploration we wish to perform, and for the remaining 1-ϵ times the model prefers exploitation. By convention, “epsilon” represents the percentage of time/trials dedicated for exploration, and it is also typical to do random exploration. This introduces some form of stochasticity.\n",
    "\n",
    "The choose_arm function returns the best arm's index based on the ϵ-greedy policy.\n",
    "The update functiom updates algorithm's parameters(matrices) for the selected arm. It looks at the selected arm, updates the number of times this arm was chosen, and then it updates the mean reward observed for the selected arm."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d74ce40f-1fda-4397-ae7d-21f6faeacad9",
   "metadata": {
    "id": "d74ce40f-1fda-4397-ae7d-21f6faeacad9"
   },
   "outputs": [],
   "source": [
    "class Egreedy:\n",
    "    \"\"\"\n",
    "    Epsilon greedy algorithm implementation\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, epsilon):\n",
    "        \"\"\"\n",
    "        Parameters\n",
    "        ----------\n",
    "        epsilon : number (Egreedy parameter, ideally between 0 and 1)\n",
    "        \"\"\"\n",
    "\n",
    "        self.e = round(epsilon, 1)  # epsilon parameter for Egreedy \n",
    "        self.algorithm = \"Egreedy (ε=\" + str(self.e) + \")\"\n",
    "        self.q = np.zeros(n_arms)  # average reward for each arm -- this represents the known mean reward for each arm\n",
    "        self.n = np.zeros(n_arms)  # number of times each arm was chosen\n",
    "\n",
    "    def choose_arm(self, t, user, pool_idx):\n",
    "        \"\"\"\n",
    "        Returns the best arm's index relative to the pool\n",
    "        Parameters\n",
    "        ----------\n",
    "        t : number (number of trial)\n",
    "        user : array (user features)\n",
    "        pool_idx : array of indexes (pool indexes for article identification)\n",
    "        \"\"\"\n",
    "\n",
    "        p = np.random.rand()\n",
    "        if p > self.e:\n",
    "            return np.argmax(self.q[pool_idx])\n",
    "        else:\n",
    "            return np.random.randint(low=0, high=len(pool_idx))\n",
    "\n",
    "    def update(self, displayed, reward, user, pool_idx):\n",
    "        \"\"\"\n",
    "        Updates algorithm's parameters(matrices)\n",
    "        Parameters\n",
    "        ----------\n",
    "        displayed : index (displayed article index relative to the pool)\n",
    "        reward : binary (user clicked or not)\n",
    "        user : array (user features)\n",
    "        pool_idx : array of indexes (pool indexes for article identification)\n",
    "        \"\"\"\n",
    "\n",
    "        a = pool_idx[displayed]\n",
    "        \n",
    "        # update counts pulled for chosen arm\n",
    "        self.n[a] += 1\n",
    "        \n",
    "        # update average/mean value/reward for chosen arm\n",
    "        self.q[a] += (reward - self.q[a]) / self.n[a]\n",
    "        \"\"\"\n",
    "        this can also be written as:\n",
    "        value = self.q[a]\n",
    "        new_value = ((self.n[a]-1)/float(self.n[a])) * value + (1 / float(self.n[a])) * reward\n",
    "        self.q[a] = new_value\n",
    "        \"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "13ca41bc-b6b7-49c3-867b-50bc836ccae3",
   "metadata": {
    "id": "13ca41bc-b6b7-49c3-867b-50bc836ccae3"
   },
   "source": [
    "# Policy evaluation to evaluate contextual bandits\n",
    "\n",
    "We base our evaluation on the seminal work on offline evaluation of bandits as presented in:\n",
    "- [WSDM 2011] Unbiased Offline Evaluation of Contextual-bandit-based News Article Recommendation Algorithms\n",
    "https://arxiv.org/pdf/1003.5956.pdf\n",
    "\n",
    "Compared to machine learning in the more standard supervised learning setting, evaluation of methods in a contextual bandit setting is frustratingly difficult. Our goal here is to measure the performance of a bandit algorithm A, that is, a rule for selecting an arm at each time step based on the preceding interactions and current context.\n",
    "\n",
    "We suppose that there is some unknown distribution D from which tuples are drawn i.i.d. of the form (x, r1, . . . , rK), each consisting of observed context and unobserved payoffs for all arms. We also posit access to a long sequence of logged events resulting from the interaction of the uniformly random logging policy with the world. Each such event consists of the context vector x, a selected arm a, and the resulting observed payoff ra. Crucially, this logged data is partially labeled in the sense that only the payoff ra is observed for the single arm a that was chosen uniformly at random.\n",
    "\n",
    "Our goal is to use this data to evaluate a bandit algorithm A, which is a (possibly randomized) mapping for selecting the arm at at time t based on the history ht−1 of t−1 preceding events together with the current context. Therefore, the data serves as a benchmark, with which people can evaluate and compare different bandit algorithms.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "854372ec-b76f-4c85-a30f-e303979cb9f2",
   "metadata": {
    "id": "854372ec-b76f-4c85-a30f-e303979cb9f2"
   },
   "source": [
    "The policy evaluator is shown in Algorithm 1 above. The method takes as input a bandit algorithm A and a desired\n",
    "number of “valid” events T on which to base the evaluation. We then step through the stream of logged events one by one. If, given the current history ht−1, it happens that the policy A chooses the same arm a as the one that was selected by the logging policy, then the event is retained (that is, added to the history), and the total payoff updated. Otherwise, if the policy A selects a different arm from the one that was taken by the logging policy, then the event is entirely ignored, and the algorithm proceeds to the next event without any change in its state.\n",
    "\n",
    "We next implement this evaluation function below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ad20a606-14d9-46b7-80d1-7e0f3bf70e76",
   "metadata": {
    "id": "ad20a606-14d9-46b7-80d1-7e0f3bf70e76"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b98f55d6-d75f-4229-b69a-4f8716d2cb1b",
   "metadata": {
    "id": "b98f55d6-d75f-4229-b69a-4f8716d2cb1b"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "id": "3eb3334e-253f-483c-9e39-4bdda2c3b00e",
   "metadata": {
    "id": "3eb3334e-253f-483c-9e39-4bdda2c3b00e"
   },
   "outputs": [],
   "source": [
    "import random\n",
    "import time\n",
    "\n",
    "\n",
    "def evaluate(A, size=100, learn_ratio = 0.9):\n",
    "    \"\"\"\n",
    "    Policy evaluator as described in the paper\n",
    "    Parameters\n",
    "    ----------\n",
    "    A : class (algorithm)\n",
    "    size : number (run the evaluation only on a portion of the dataset)\n",
    "    learn_ratio : number (perform learning(update parameters) only on a small portion of the traffic)\n",
    "    Returns\n",
    "    -------\n",
    "    learn : array (contains the ctr for each trial for the learning bucket)\n",
    "    deploy : array (contains the ctr for each trial for the deployment bucket)\n",
    "    \"\"\"\n",
    "    \n",
    "    start = time.time()\n",
    "    # we initialize the payoff and events parameters separately for learning phase of the events and deployment phase of events.\n",
    "    Payoff_deploy = 0 # total payoff for the deployment bucket\n",
    "    Payoff_learn = 0  # total payoff for the learning bucket\n",
    "    Events_deploy = 1 # counter of valid events for the deployment bucket\n",
    "    Events_learn = 0  # counter of valid events for the learning bucket\n",
    "\n",
    "    learn = []\n",
    "    deploy = []\n",
    "    global events\n",
    "    if size != 100:\n",
    "        k = int(n_events * size / 100)\n",
    "        events = random.sample(events, k)\n",
    "\n",
    "    \"\"\"\n",
    "    we run through the logged events, and treat each event either for learning & updating the parameters,\n",
    "    or for deployment purposes wherein we use the reward obtained as evaluation metric\n",
    "    \"\"\"\n",
    "    for t, event in enumerate(events):\n",
    "\n",
    "        displayed = event[0]\n",
    "        reward = event[1]\n",
    "        user = event[2]\n",
    "        pool_idx = event[3]\n",
    "\n",
    "        # select the arm based on the bandit policy\n",
    "        chosen = A.choose_arm(Payoff_learn + Payoff_deploy, user, pool_idx)\n",
    "        \n",
    "        \"\"\"\n",
    "        If, given the current history ht−1, it happens that the policy A chooses the same arm a\n",
    "        as the one that was selected by the logging policy, then the event is retained\n",
    "        (that is, added to the history), and the total payoff updated.\n",
    "        Otherwise, if the policy A selects a different arm from the one that was taken by the logging policy,\n",
    "        then the event is entirely ignored, and the algorithm proceeds to the next event without any change in its state.\n",
    "        \"\"\"\n",
    "        if chosen == displayed:\n",
    "            if random.random() < learn_ratio:\n",
    "                Payoff_learn += event[1]\n",
    "                Events_learn += 1\n",
    "                A.update(displayed, reward, user, pool_idx)\n",
    "                learn.append(Payoff_learn / Events_learn)\n",
    "            else:\n",
    "                Payoff_deploy += event[1]\n",
    "                Events_deploy += 1\n",
    "                deploy.append(Payoff_deploy / Events_deploy)\n",
    "\n",
    "    end = time.time()\n",
    "\n",
    "    execution_time = round(end - start, 1)\n",
    "    execution_time = (\n",
    "        str(round(execution_time / 60, 1)) + \"m\"\n",
    "        if execution_time > 60\n",
    "        else str(execution_time) + \"s\"\n",
    "    )\n",
    "    print(\n",
    "        \"{:<20}{:<10}{}\".format(\n",
    "            A.algorithm, round(Payoff_deploy / Events_deploy, 4), execution_time\n",
    "        )\n",
    "    )\n",
    "\n",
    "    return learn, deploy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c803ab77-45b6-468d-94b1-5d386fbbdef5",
   "metadata": {
    "id": "c803ab77-45b6-468d-94b1-5d386fbbdef5"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "45aa4ece-69ca-4538-a88c-92e9b18a7510",
   "metadata": {
    "id": "45aa4ece-69ca-4538-a88c-92e9b18a7510"
   },
   "source": [
    "Now lets run the evaluation method on the epsilon-greedy policy for different values of epsilon, and print the corresponding reward obtained during deployment phase of evaluation.\n",
    "## Reward based evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d226bd62-a6bd-4068-ab5d-4e2bfdf4bb9e",
   "metadata": {
    "id": "d226bd62-a6bd-4068-ab5d-4e2bfdf4bb9e",
    "outputId": "f2ea98c8-c432-4327-f4b7-a032422dca64"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Egreedy (ε=0.1)     0.0233    0.5s\n",
      "Egreedy (ε=0.1)     0.0292    0.4s\n",
      "Egreedy (ε=0.1)     0.023     0.4s\n",
      "Egreedy (ε=0.2)     0.0295    0.4s\n",
      "Egreedy (ε=0.5)     0.0306    0.3s\n",
      "Egreedy (ε=0.8)     0.0252    0.3s\n"
     ]
    }
   ],
   "source": [
    "_, deploy = evaluate(Egreedy(0.1),learn_ratio=0.25)\n",
    "rnd_ctr = deploy[-1]\n",
    "\n",
    "_, deploy = evaluate(Egreedy(0.1),learn_ratio=0.5)\n",
    "rnd_ctr = deploy[-1]\n",
    "\n",
    "_, deploy = evaluate(Egreedy(0.1),learn_ratio=0.9)\n",
    "rnd_ctr = deploy[-1]\n",
    "\n",
    "_, deploy = evaluate(Egreedy(0.25),learn_ratio=0.5)\n",
    "rnd_ctr = deploy[-1]\n",
    "\n",
    "_, deploy = evaluate(Egreedy(0.5),learn_ratio=0.5)\n",
    "rnd_ctr = deploy[-1]\n",
    "\n",
    "_, deploy = evaluate(Egreedy(0.75),learn_ratio=0.5)\n",
    "rnd_ctr = deploy[-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7fb3be40-ead6-4ce0-8208-9d003d1c31a4",
   "metadata": {
    "id": "7fb3be40-ead6-4ce0-8208-9d003d1c31a4"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9575185b-3a33-4b50-9b3d-83a4237e766f",
   "metadata": {
    "id": "9575185b-3a33-4b50-9b3d-83a4237e766f"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "00632099-405e-46f6-82d0-20e985ff08ab",
   "metadata": {
    "id": "00632099-405e-46f6-82d0-20e985ff08ab"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "bd0cfb8e-7c33-4f09-b917-ca26ea091142",
   "metadata": {
    "id": "bd0cfb8e-7c33-4f09-b917-ca26ea091142"
   },
   "source": [
    "# Goal 2: finish the implementation of UCB1 algorithm"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8bd37be-bf33-4613-b736-d674dc6d5889",
   "metadata": {
    "id": "b8bd37be-bf33-4613-b736-d674dc6d5889"
   },
   "source": [
    "## UCB-1 algorithm for arm selection\n",
    "\n",
    "Epsilon greedy performs pretty well, but it’s easy to see how selecting arms at random can be inefficient. If you have one movie that 50% of users have liked, and another at 5% have liked, epsilon greedy is equally likely to pick either of these movies when exploring random arms. Upper Confidence Bound algorithms were introduced as a class of bandit algorithm that explores more efficiently.\n",
    "\n",
    "Upper Confidence Bound algorithms construct a confidence interval of what each arm’s true performance might be, factoring in the uncertainty caused by variance in the data and the fact that we’re only able to observe a limited sample of pulls for any given arm. The algorithms then optimistically assume that each arm will perform as well as its upper confidence bound (UCB), selecting the arm with the highest UCB.\n",
    "\n",
    "### Motivation behind UCB\n",
    "Support an Artice A has been seen 100 times and has the best CTR. Article B has a slightly worse CTR than article A, but it hasn’t been seen by as many users, so there’s also more uncertainty about how well it’s going to perform in the long run. For this reason, it has a larger confidence bound, giving it a slightly higher UCB score than article A. Article C was published just moments ago, so almost no users have seen it. We’re extremely uncertain about how high its CTR will ultimately be, so its UCB is highest of all for now despite its initial CTR being low.\n",
    "\n",
    "Over time, more users will see articles B and C, and their confidence bounds will become more narrow and look more like that of article A. As we learn more about B and C, we’ll shift from exploration toward exploitation as the articles’ confidence intervals collapse toward their means. Unless the CTR of article B or C improves, the bandit will quickly start to favor article A again as the other articles’ confidence bounds shrink."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a209f8cb-1753-4f0c-b855-6ba8c1d99d9f",
   "metadata": {
    "id": "a209f8cb-1753-4f0c-b855-6ba8c1d99d9f"
   },
   "source": [
    "Let $n_t(a)$ be the number of times arm a is selected in rounds 1,2,…,t and $\\mu_t(a)$ be the average reward of arm a up to time t. The upper confidence bound is defined as:\n",
    "\n",
    "$UCB_t(a) = \\mu_t(a)+ \\sqrt{\\frac{2 \\log(t)}{ N_k(t)}}$\n",
    "\n",
    "\n",
    "where $\\mu_t(a)$ can be computed as `rewards[k] / pulls[k]` and the bound can be computed as `sqrt((2 * log(t)) / pulls[k]`.\n",
    "\n",
    "The UCB1 algorithm chooses the best arm based on an optimistic estimate.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "id": "f5881aa5-6396-469d-a582-95af04e2224a",
   "metadata": {
    "id": "f5881aa5-6396-469d-a582-95af04e2224a"
   },
   "outputs": [],
   "source": [
    "class Ucb1:\n",
    "    def __init__(self, alpha):\n",
    "        \"\"\"\n",
    "        Parameters\n",
    "        ----------\n",
    "        alpha : number (ucb parameter)\n",
    "        \"\"\"\n",
    "\n",
    "        self.alpha = round(alpha, 1)\n",
    "        self.algorithm = \"UCB1 (α=\" + str(self.alpha) + \")\"\n",
    "\n",
    "        self.q = np.zeros(n_arms)  # average reward for each arm\n",
    "        self.n = np.ones(n_arms)  # number of times each arm was chosen\n",
    "\n",
    "    def choose_arm(self, t, user, pool_idx):\n",
    "        \"\"\"\n",
    "        Returns the best arm's index relative to the pool\n",
    "        Parameters\n",
    "        ----------\n",
    "        t : number (number of trial)\n",
    "        user : array (user features)\n",
    "        pool_idx : array of indexes (pool indexes for article identification)\n",
    "        \"\"\"\n",
    "        \n",
    "        # write your code here\n",
    "        # return something here\n",
    "        avg_rewards = self.q[pool_idx]\n",
    "        bounds = np.sqrt((2 * np.log(t)) / self.n[pool_idx])\n",
    "        return np.argmax(avg_rewards + self.alpha * bounds)\n",
    "\n",
    "    def update(self, displayed, reward, user, pool_idx):\n",
    "        \"\"\"\n",
    "        Updates algorithm's parameters(matrices)\n",
    "        Parameters\n",
    "        ----------\n",
    "        displayed : index (displayed article index relative to the pool)\n",
    "        reward : binary (user clicked or not)\n",
    "        user : array (user features)\n",
    "        pool_idx : array of indexes (pool indexes for article identification)\n",
    "        \"\"\"\n",
    "\n",
    "        a = pool_idx[displayed]\n",
    "\n",
    "        # update self.n[a] here\n",
    "        # update self.q[a] here\n",
    "        self.n[a] += 1\n",
    "        self.q[a] += (reward - self.q[a]) / self.n[a]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d34852c-8ccf-4455-9863-5a5acc8e56f9",
   "metadata": {
    "id": "1d34852c-8ccf-4455-9863-5a5acc8e56f9"
   },
   "source": [
    "One of the most important features of the UCB is that it not only exponentially decays as the number of pulls on the given machine increases, but also increases as the timestep increases. In other words, arms that have been explored less are given a boost even if their estimated mean is low, especially if we’ve been playing for a while. In this way, the UCB1 algorithm is able to naturally define its own mix of exploration vs. exploitation without depending on a user supplied parameter like epsilon greedy."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "289e5976-b260-4df9-a00c-5ffe78ae46f0",
   "metadata": {
    "id": "289e5976-b260-4df9-a00c-5ffe78ae46f0"
   },
   "source": [
    "### Goal 2 for this week's project\n",
    "Finish the implementation of the UCB1 class above, and evaluate it for a set of parameters and compare its performance with respect to ϵ-greedy policy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "id": "44ac4c75-1e33-48f4-a3bb-fcd8cea76d77",
   "metadata": {
    "id": "44ac4c75-1e33-48f4-a3bb-fcd8cea76d77"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/15/f0f69s3n5235dwmr7mx8w97m0000gq/T/ipykernel_37687/1799494368.py:28: RuntimeWarning: divide by zero encountered in log\n",
      "  bounds = np.sqrt((2 * np.log(t)) / self.n[pool_idx])\n",
      "/var/folders/15/f0f69s3n5235dwmr7mx8w97m0000gq/T/ipykernel_37687/1799494368.py:28: RuntimeWarning: invalid value encountered in sqrt\n",
      "  bounds = np.sqrt((2 * np.log(t)) / self.n[pool_idx])\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "UCB1 (α=0.1)        0.0271    1.9s\n"
     ]
    }
   ],
   "source": [
    "_, deploy = evaluate(Ucb1(0.1))\n",
    "rnd_ctr = deploy[-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "id": "e68fc5bf-93cc-4f1e-ab66-cfe6249edc09",
   "metadata": {
    "id": "e68fc5bf-93cc-4f1e-ab66-cfe6249edc09"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/15/f0f69s3n5235dwmr7mx8w97m0000gq/T/ipykernel_37687/1799494368.py:28: RuntimeWarning: divide by zero encountered in log\n",
      "  bounds = np.sqrt((2 * np.log(t)) / self.n[pool_idx])\n",
      "/var/folders/15/f0f69s3n5235dwmr7mx8w97m0000gq/T/ipykernel_37687/1799494368.py:28: RuntimeWarning: invalid value encountered in sqrt\n",
      "  bounds = np.sqrt((2 * np.log(t)) / self.n[pool_idx])\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "UCB1 (α=0.1)        0.0266    1.5s\n",
      "UCB1 (α=0.1)        0.0344    1.3s\n",
      "UCB1 (α=0.1)        0.0199    1.5s\n",
      "UCB1 (α=0.2)        0.0242    1.4s\n",
      "UCB1 (α=0.5)        0.0294    1.5s\n",
      "UCB1 (α=0.8)        0.0248    1.5s\n"
     ]
    }
   ],
   "source": [
    "_, deploy = evaluate(Ucb1(0.1), learn_ratio=0.25)\n",
    "rnd_ctr = deploy[-1]\n",
    "\n",
    "_, deploy = evaluate(Ucb1(0.1), learn_ratio=0.5)\n",
    "rnd_ctr = deploy[-1]\n",
    "\n",
    "_, deploy = evaluate(Ucb1(0.1), learn_ratio=0.9)\n",
    "rnd_ctr = deploy[-1]\n",
    "\n",
    "_, deploy = evaluate(Ucb1(0.25), learn_ratio=0.5)\n",
    "rnd_ctr = deploy[-1]\n",
    "\n",
    "_, deploy = evaluate(Ucb1(0.5), learn_ratio=0.5)\n",
    "rnd_ctr = deploy[-1]\n",
    "\n",
    "_, deploy = evaluate(Ucb1(0.75), learn_ratio=0.5)\n",
    "rnd_ctr = deploy[-1]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0ae87cb7-2c41-4f59-ad08-994354b6d26b",
   "metadata": {
    "id": "b5711ffd-2cab-43d1-8972-9b3b26cb14a4"
   },
   "source": [
    "The UCB-1 algorithm gave us a better payoff than epsilon-greedy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "49d3378b-6868-4296-94ac-48756f32a3b1",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "include_colab_link": true,
   "name": "week2-multiTask-Bandit-final.ipynb",
   "provenance": []
  },
  "environment": {
   "kernel": "python3",
   "name": "tf2-gpu.2-6.m89",
   "type": "gcloud",
   "uri": "gcr.io/deeplearning-platform-release/tf2-gpu.2-6:m89"
  },
  "kernelspec": {
   "display_name": "co-rise-reco",
   "language": "python",
   "name": "co-rise-reco"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}