DSF-ASOS_Build_and_Deploy_a_Recommender_in_3_Hours.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Science Festival x ASOS\n",
    "## Build and Deploy a Recommender System in 3 Hours."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import tensorflow as tf\n",
    "import os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Import training data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train = pd.read_parquet(\"https://raw.githubusercontent.com/ASOS/dsf2020/main/dsf_asos_train_with_alphanumeric_dummy_ids.parquet\")\n",
    "valid = pd.read_parquet(\"https://raw.githubusercontent.com/ASOS/dsf2020/main/dsf_asos_valid_with_alphanumeric_dummy_ids.parquet\")\n",
    "dummy_users = pd.read_csv(\"https://raw.githubusercontent.com/ASOS/dsf2020/main/dsf_asos_dummy_users_with_alphanumeric_dummy_ids.csv\", header=None).values.flatten().astype(str)\n",
    "products = pd.read_csv(\"https://raw.githubusercontent.com/ASOS/dsf2020/main/dsf_asos_productIds.csv\", header=None).values.flatten().astype(int)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The briefest intro to tf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Tensors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Gradients"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Multiply and add tensors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = tf.constant([[1,2,3]], dtype=tf.float32)\n",
    "Y = tf.constant([[1,2,3, 4], [1,2,3,4], [1,2,3,4]], dtype=tf.float32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z = tf.constant([10, 11, 12, 13], dtype=tf.float32)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This operation is very common in deep learning, so it has been abstracted:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can choose to apply a function to each value in the output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can put different layers together in a sequence:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dl3 = tf.keras.layers.Dense(1, use_bias=False, \\\n",
    "                             weights=[tf.constant([[0], [1], [0], [1]], \\\n",
    "                                                  dtype=tf.float32)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can get more flexibility if you use tf.keras.model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So far we have been setting the weights of the dense layers, but if we don't set the weights than weights get randomly chosen."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dl6 = tf.keras.layers.Dense(4, use_bias=True)\n",
    "dl6(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dl6.get_weights()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Define a Recommender Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The embedding layer gives a list of random numbers for each user and each product."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Scores can be found using the dot product."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can score multiple products at the same time, which is what we need to create a ranking."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And we can score multiple users for multiple products which we will need to do if we are to train quickly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But we need to map product ids to embedding ids."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "product_table = tf.lookup.StaticHashTable(\n",
    "    tf.lookup.KeyValueTensorInitializer(tf.constant(products, dtype=tf.int32), \n",
    "                                        range(len(products))), -1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's put those two things together"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class SimpleRecommender(tf.keras.Model):\n",
    "    def __init__(self, dummy_users, products):\n",
    "        super(SimpleRecommender, self).__init__()\n",
    "        self.products = tf.constant(products, dtype=tf.int32)\n",
    "        self.dummy_users = tf.constant(dummy_users, dtype=tf.string)\n",
    "        self.dummy_user_table = tf.lookup.StaticHashTable(tf.lookup.KeyValueTensorInitializer(self.dummy_users, range(len(dummy_users))), -1)\n",
    "        self.product_table = tf.lookup.StaticHashTable(tf.lookup.KeyValueTensorInitializer(self.products, range(len(products))), -1)\n",
    "        \n",
    "        self.user_embedding = tf.keras.layers.Embedding(len(dummy_users), ?)\n",
    "        self.product_embedding = tf.keras.layers.Embedding(len(products), ?)\n",
    "        \n",
    "    def call(self):\n",
    "        pass\n",
    "    \n",
    "    @tf.function\n",
    "    def call_item_item(self, product):\n",
    "        product_x = self.product_table.lookup(product)\n",
    "        pe = tf.expand_dims(self.product_embedding(product_x), 0)\n",
    "        \n",
    "        all_pe = tf.expand_dims(self.product_embedding.embeddings, 0)#note this only works if the layer has been built!\n",
    "        scores = tf.reshape(self.dot([pe, all_pe]), [-1])\n",
    "        \n",
    "        top_scores, top_indices = tf.math.top_k(scores, k=100)\n",
    "        top_ids = tf.gather(self.products, top_indices)\n",
    "        return top_ids, top_scores"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Creating a dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First create a tf.data.Dataset from the user purchase pairs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dummy_user_tensor = tf.constant(train[[\"dummyUserId\"]].values, dtype=tf.string)\n",
    "product_tensor = tf.constant(train[[\"productId\"]].values, dtype=tf.int32)\n",
    "\n",
    "dataset = tf.data.Dataset.from_tensor_slices((dummy_user_tensor, product_tensor))\n",
    "for x, y in dataset:\n",
    "    print(x)\n",
    "    print(y)\n",
    "    break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For each purchase let's sample a number of products that the user did not purchase. Then the model can score each of the products and we will know we are doing a good job if the product with the highest score is the product that the user actually purchased.\n",
    "\n",
    "We can do this using dataset.map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Mapper():\n",
    "    \n",
    "    def __init__(self, possible_products, num_negative_products):\n",
    "        self.num_possible_products = len(possible_products)\n",
    "        self.possible_products_tensor = tf.constant(possible_products, dtype=tf.int32)\n",
    "        \n",
    "        self.num_negative_products = num_negative_products\n",
    "    \n",
    "    def __call__(self, user, product):\n",
    "        return user, product"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's bring the steps together to define a function which creates a dataset "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_dataset():\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Train a model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to compile a model, set the loss and create an evaluation metric. Then we need to train the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's do a manual check on whether the model is any good."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_product = 11698965"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Recs for item {}: {}\".format(test_product, model.call_item_item(tf.constant(test_product, dtype=tf.int32))))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Save the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_path = \"models/recommender/1\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "inpute_signature = tf.TensorSpec(shape=(), dtype=tf.int32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "signatures = { 'call_item_item': r1.call_item_item.get_concrete_function(inpute_signature)}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "imported_model = tf.saved_model.load('models/recommeder/1')\n",
    "list(imported_model.signatures.keys())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "imported_model.signatures['call_item_item'](tf.constant([14844847]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "os.makedirs(\"dummy/0\")\n",
    "tf.saved_model.save(model, 'dummy/0')    \n",
    "imported = tf.saved_model.load(\"dummy/0\")\n",
    "imported(tf.constant([14844847]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "os.makedirs(\"dummy/1\")\n",
    "tf.saved_model.save(model, 'dummy/1',\n",
    "                    model.call_item_item.get_concrete_function(tf.TensorSpec(shape=(), dtype=tf.int32)))      \n",
    "list(imported_model.signatures.keys())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "imported_model.signatures['serving_default'](tf.constant([14844847]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Zipping the saved model will make it easier to download."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from zipfile import ZipFile\n",
    "import os\n",
    "# create a ZipFile object\n",
    "with ZipFile('recommender.zip', 'w') as zipObj:\n",
    "   # Iterate over all the files in directory\n",
    "    for folderName, subfolders, filenames in os.walk(\"models\"):\n",
    "        for filename in filenames:\n",
    "           #create complete filepath of file in directory\n",
    "           filePath = os.path.join(folderName, filename)\n",
    "           # Add file to zip\n",
    "           zipObj.write(filePath)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}