Practice 14 - Hybrid model with LightFM on ML.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Recommender Systems 2020/21\n",
    "\n",
    "### Practice - Hybrid model with LightFM on MovieLens\n",
    "\n",
    "### Factorization Machines can be used to build different types of models\n",
    "* Pure Collaborative Filtering\n",
    "* Content-based\n",
    "* Hybrid Content-Collaborative\n",
    "* Heterogeneous Feature models\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![FactorizationMachine.png](images/FactorizationMachine.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Movielens10M: Verifying data consistency...\n",
      "Movielens10M: Verifying data consistency... Passed!\n",
      "DataReader: current dataset is: <class 'Data_manager.Dataset.Dataset'>\n",
      "\tNumber of items: 10681\n",
      "\tNumber of users: 69878\n",
      "\tNumber of interactions in URM_all: 10000054\n",
      "\tValue range in URM_all: 0.50-5.00\n",
      "\tInteraction density: 1.34E-02\n",
      "\tInteractions per user:\n",
      "\t\t Min: 2.00E+01\n",
      "\t\t Avg: 1.43E+02\n",
      "\t\t Max: 7.36E+03\n",
      "\tInteractions per item:\n",
      "\t\t Min: 0.00E+00\n",
      "\t\t Avg: 9.36E+02\n",
      "\t\t Max: 3.49E+04\n",
      "\tGini Index: 0.57\n",
      "\n",
      "\tICM name: ICM_all, Value range: 1.00 / 69.00, Num features: 10126, feature occurrences: 128384, density 1.19E-03\n",
      "\tICM name: ICM_genres, Value range: 1.00 / 1.00, Num features: 20, feature occurrences: 21564, density 1.01E-01\n",
      "\tICM name: ICM_tags, Value range: 1.00 / 69.00, Num features: 10106, feature occurrences: 106820, density 9.90E-04\n",
      "\tICM name: ICM_year, Value range: 6.00E+00 / 2.01E+03, Num features: 1, feature occurrences: 10681, density 1.00E+00\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from Data_manager.split_functions.split_train_validation_random_holdout import split_train_in_two_percentage_global_sample\n",
    "from Data_manager.Movielens.Movielens10MReader import Movielens10MReader\n",
    "\n",
    "data_reader = Movielens10MReader()\n",
    "data_loaded = data_reader.load_data()\n",
    "\n",
    "URM_all = data_loaded.get_URM_all()\n",
    "ICM_genres = data_loaded.get_ICM_from_name(\"ICM_genres\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning: 75 (0.11 %) of 69878 users have no sampled items\n",
      "Warning: 235 (0.34 %) of 69878 users have no sampled items\n",
      "EvaluatorHoldout: Ignoring 69643 ( 0.3%) Users that have less than 1 test interactions\n",
      "EvaluatorHoldout: Ignoring 69803 ( 0.1%) Users that have less than 1 test interactions\n"
     ]
    }
   ],
   "source": [
    "from Evaluation.Evaluator import EvaluatorHoldout\n",
    "\n",
    "URM_train_validation, URM_test = split_train_in_two_percentage_global_sample(URM_all, train_percentage = 0.8)\n",
    "URM_train, URM_validation = split_train_in_two_percentage_global_sample(URM_train_validation, train_percentage = 0.8)\n",
    "\n",
    "evaluator_validation = EvaluatorHoldout(URM_validation, cutoff_list=[10])\n",
    "evaluator_test = EvaluatorHoldout(URM_test, cutoff_list=[10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A pure collaborative filtering model\n",
    "\n",
    "In order to train a pure CF model it is sufficient to provide to the FM the user-item interaction data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\ferra\\Anaconda3\\envs\\RecSysFramework38\\lib\\site-packages\\lightfm\\_lightfm_fast.py:9: UserWarning: LightFM was compiled without OpenMP support. Only a single thread will be used.\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "## In order to evaluate put it in a recommender class\n",
    "from Recommenders.BaseRecommender import BaseRecommender\n",
    "from lightfm import LightFM\n",
    "import numpy as np\n",
    "\n",
    "class LightFMCFRecommender(BaseRecommender):\n",
    "    \"\"\"LightFMCFRecommender\"\"\"\n",
    "\n",
    "    RECOMMENDER_NAME = \"LightFMCFRecommender\"\n",
    "\n",
    "    def __init__(self, URM_train):\n",
    "        super(LightFMCFRecommender, self).__init__(URM_train)\n",
    "\n",
    "\n",
    "    def fit(self, epochs = 300, alpha = 1e-6, n_factors = 10, n_threads = 4):\n",
    "        \n",
    "        # Let's fit a WARP model\n",
    "        self.lightFM_model = LightFM(loss='warp',\n",
    "                                     item_alpha=alpha,\n",
    "                                     no_components=n_factors)\n",
    "\n",
    "        self.lightFM_model = self.lightFM_model.fit(URM_train, \n",
    "                                       epochs=epochs,\n",
    "                                       num_threads=n_threads)\n",
    "\n",
    "\n",
    "    def _compute_item_score(self, user_id_array, items_to_compute = None):\n",
    "        \n",
    "        # Create a single (n_items, ) array with the item score, then copy it for every user\n",
    "        items_to_compute = np.arange(self.n_items)\n",
    "        \n",
    "        item_scores = - np.ones((len(user_id_array), self.n_items)) * np.inf\n",
    "\n",
    "        for user_index, user_id in enumerate(user_id_array):\n",
    "            item_scores[user_index] = self.lightFM_model.predict(int(user_id), \n",
    "                                                                 items_to_compute)\n",
    "\n",
    "        return item_scores\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LightFMCFRecommender: URM Detected 63 ( 0.6%) items with no interactions.\n",
      "EvaluatorHoldout: Processed 69643 (100.0%) in 3.32 min. Users per second: 349\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PRECISION</th>\n",
       "      <th>PRECISION_RECALL_MIN_DEN</th>\n",
       "      <th>RECALL</th>\n",
       "      <th>MAP</th>\n",
       "      <th>MAP_MIN_DEN</th>\n",
       "      <th>MRR</th>\n",
       "      <th>NDCG</th>\n",
       "      <th>F1</th>\n",
       "      <th>HIT_RATE</th>\n",
       "      <th>ARHR_ALL_HITS</th>\n",
       "      <th>...</th>\n",
       "      <th>COVERAGE_ITEM_CORRECT</th>\n",
       "      <th>COVERAGE_USER</th>\n",
       "      <th>COVERAGE_USER_CORRECT</th>\n",
       "      <th>DIVERSITY_GINI</th>\n",
       "      <th>SHANNON_ENTROPY</th>\n",
       "      <th>RATIO_DIVERSITY_HERFINDAHL</th>\n",
       "      <th>RATIO_DIVERSITY_GINI</th>\n",
       "      <th>RATIO_SHANNON_ENTROPY</th>\n",
       "      <th>RATIO_AVERAGE_POPULARITY</th>\n",
       "      <th>RATIO_NOVELTY</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cutoff</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0.183077</td>\n",
       "      <td>0.226539</td>\n",
       "      <td>0.144772</td>\n",
       "      <td>0.095729</td>\n",
       "      <td>0.115251</td>\n",
       "      <td>0.390238</td>\n",
       "      <td>0.17913</td>\n",
       "      <td>0.161686</td>\n",
       "      <td>0.737562</td>\n",
       "      <td>0.601279</td>\n",
       "      <td>...</td>\n",
       "      <td>0.078363</td>\n",
       "      <td>0.996637</td>\n",
       "      <td>0.735081</td>\n",
       "      <td>0.012019</td>\n",
       "      <td>7.35778</td>\n",
       "      <td>0.990365</td>\n",
       "      <td>0.061727</td>\n",
       "      <td>0.649685</td>\n",
       "      <td>1.985386</td>\n",
       "      <td>0.087982</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1 rows × 25 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       PRECISION PRECISION_RECALL_MIN_DEN    RECALL       MAP MAP_MIN_DEN  \\\n",
       "cutoff                                                                      \n",
       "10      0.183077                 0.226539  0.144772  0.095729    0.115251   \n",
       "\n",
       "             MRR     NDCG        F1  HIT_RATE ARHR_ALL_HITS  ...  \\\n",
       "cutoff                                                       ...   \n",
       "10      0.390238  0.17913  0.161686  0.737562      0.601279  ...   \n",
       "\n",
       "       COVERAGE_ITEM_CORRECT COVERAGE_USER COVERAGE_USER_CORRECT  \\\n",
       "cutoff                                                             \n",
       "10                  0.078363      0.996637              0.735081   \n",
       "\n",
       "       DIVERSITY_GINI SHANNON_ENTROPY RATIO_DIVERSITY_HERFINDAHL  \\\n",
       "cutoff                                                             \n",
       "10           0.012019         7.35778                   0.990365   \n",
       "\n",
       "       RATIO_DIVERSITY_GINI RATIO_SHANNON_ENTROPY RATIO_AVERAGE_POPULARITY  \\\n",
       "cutoff                                                                       \n",
       "10                 0.061727              0.649685                 1.985386   \n",
       "\n",
       "       RATIO_NOVELTY  \n",
       "cutoff                \n",
       "10          0.087982  \n",
       "\n",
       "[1 rows x 25 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Set the number of threads; you can increase this\n",
    "# if you have more physical cores available.\n",
    "\n",
    "recommender = LightFMCFRecommender(URM_train)\n",
    "recommender.fit(epochs = 10)\n",
    "\n",
    "result_df, _ = evaluator_validation.evaluateRecommender(recommender)\n",
    "result_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A content-based model\n",
    "\n",
    "It is possible to use only item features to try to predict the user-item interactions. LightFM automatically discards ItemIDs when item features are provided. The same happens for UserIDs when the UCM is provided.\n",
    "\n",
    "#### Warning: this may not be the behaviour you expect"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "class LightFMCBFRecommender(BaseRecommender):\n",
    "    \"\"\"LightFMCBFRecommender\"\"\"\n",
    "\n",
    "    RECOMMENDER_NAME = \"LightFMCBFRecommender\"\n",
    "\n",
    "    def __init__(self, URM_train, ICM_train):\n",
    "        super(LightFMCBFRecommender, self).__init__(URM_train)\n",
    "        \n",
    "        self.ICM_train = ICM_train.copy()\n",
    "\n",
    "\n",
    "    def fit(self, epochs = 300, alpha = 1e-6, n_factors = 10, n_threads = 4):\n",
    "        \n",
    "        # Let's fit a WARP model\n",
    "        self.lightFM_model = LightFM(loss='warp',\n",
    "                                     item_alpha=alpha,\n",
    "                                     no_components=n_factors)\n",
    "\n",
    "        self.lightFM_model = self.lightFM_model.fit(URM_train, \n",
    "                                       item_features=self.ICM_train, \n",
    "                                       epochs=epochs, \n",
    "                                       num_threads=n_threads)\n",
    "\n",
    "\n",
    "    def _compute_item_score(self, user_id_array, items_to_compute = None):\n",
    "        \n",
    "        # Create a single (n_items, ) array with the item score, then copy it for every user\n",
    "        items_to_compute = np.arange(self.n_items)\n",
    "        \n",
    "        item_scores = - np.ones((len(user_id_array), self.n_items)) * np.inf\n",
    "\n",
    "        for user_index, user_id in enumerate(user_id_array):\n",
    "            item_scores[user_index] = self.lightFM_model.predict(int(user_id), \n",
    "                                                                 items_to_compute,\n",
    "                                                                 item_features = self.ICM_train)\n",
    "\n",
    "        return item_scores\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LightFMCBFRecommender: URM Detected 63 ( 0.6%) items with no interactions.\n",
      "EvaluatorHoldout: Processed 69643 (100.0%) in 2.95 min. Users per second: 393\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PRECISION</th>\n",
       "      <th>PRECISION_RECALL_MIN_DEN</th>\n",
       "      <th>RECALL</th>\n",
       "      <th>MAP</th>\n",
       "      <th>MAP_MIN_DEN</th>\n",
       "      <th>MRR</th>\n",
       "      <th>NDCG</th>\n",
       "      <th>F1</th>\n",
       "      <th>HIT_RATE</th>\n",
       "      <th>ARHR_ALL_HITS</th>\n",
       "      <th>...</th>\n",
       "      <th>COVERAGE_ITEM_CORRECT</th>\n",
       "      <th>COVERAGE_USER</th>\n",
       "      <th>COVERAGE_USER_CORRECT</th>\n",
       "      <th>DIVERSITY_GINI</th>\n",
       "      <th>SHANNON_ENTROPY</th>\n",
       "      <th>RATIO_DIVERSITY_HERFINDAHL</th>\n",
       "      <th>RATIO_DIVERSITY_GINI</th>\n",
       "      <th>RATIO_SHANNON_ENTROPY</th>\n",
       "      <th>RATIO_AVERAGE_POPULARITY</th>\n",
       "      <th>RATIO_NOVELTY</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cutoff</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0.010149</td>\n",
       "      <td>0.0111</td>\n",
       "      <td>0.004839</td>\n",
       "      <td>0.00283</td>\n",
       "      <td>0.003056</td>\n",
       "      <td>0.024269</td>\n",
       "      <td>0.005407</td>\n",
       "      <td>0.006554</td>\n",
       "      <td>0.089442</td>\n",
       "      <td>0.026144</td>\n",
       "      <td>...</td>\n",
       "      <td>0.032113</td>\n",
       "      <td>0.996637</td>\n",
       "      <td>0.089141</td>\n",
       "      <td>0.011345</td>\n",
       "      <td>7.146886</td>\n",
       "      <td>0.987461</td>\n",
       "      <td>0.058263</td>\n",
       "      <td>0.631064</td>\n",
       "      <td>0.226407</td>\n",
       "      <td>0.136275</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1 rows × 25 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       PRECISION PRECISION_RECALL_MIN_DEN    RECALL      MAP MAP_MIN_DEN  \\\n",
       "cutoff                                                                     \n",
       "10      0.010149                   0.0111  0.004839  0.00283    0.003056   \n",
       "\n",
       "             MRR      NDCG        F1  HIT_RATE ARHR_ALL_HITS  ...  \\\n",
       "cutoff                                                        ...   \n",
       "10      0.024269  0.005407  0.006554  0.089442      0.026144  ...   \n",
       "\n",
       "       COVERAGE_ITEM_CORRECT COVERAGE_USER COVERAGE_USER_CORRECT  \\\n",
       "cutoff                                                             \n",
       "10                  0.032113      0.996637              0.089141   \n",
       "\n",
       "       DIVERSITY_GINI SHANNON_ENTROPY RATIO_DIVERSITY_HERFINDAHL  \\\n",
       "cutoff                                                             \n",
       "10           0.011345        7.146886                   0.987461   \n",
       "\n",
       "       RATIO_DIVERSITY_GINI RATIO_SHANNON_ENTROPY RATIO_AVERAGE_POPULARITY  \\\n",
       "cutoff                                                                       \n",
       "10                 0.058263              0.631064                 0.226407   \n",
       "\n",
       "       RATIO_NOVELTY  \n",
       "cutoff                \n",
       "10          0.136275  \n",
       "\n",
       "[1 rows x 25 columns]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recommender = LightFMCBFRecommender(URM_train, ICM_genres)\n",
    "recommender.fit(epochs = 10)\n",
    "\n",
    "result_df, _ = evaluator_validation.evaluateRecommender(recommender)\n",
    "result_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A hybrid content-collaborative model\n",
    "\n",
    "In order to account for both the collaborative and content-based data, a new ICM must be created in which the itemID is also present. The ItemID becomes a feature, this can be done by concatenating the ICM with a simple diagonal matrix of shape |n_items x n_items|. The same happens for UserIDs in the UCM.\n",
    "\n",
    "#### Challenges\n",
    "* FM are not particularly easy to optimize\n",
    "* If the content-based data has poor quality, the FM will simply only use the collaborative information. Providing a high weight for the content data may help in this regard\n",
    "* Using a large number of embeddings rapidly increases the training time and the number of parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "import scipy.sparse as sps\n",
    "\n",
    "class LightFMItemHybridRecommender(LightFMCBFRecommender):\n",
    "    \"\"\"LightFMItemHybridRecommender\"\"\"\n",
    "\n",
    "    RECOMMENDER_NAME = \"LightFMItemHybridRecommender\"\n",
    "\n",
    "    def __init__(self, URM_train, ICM_train):\n",
    "        super(LightFMItemHybridRecommender, self).__init__(URM_train, ICM_train)\n",
    "\n",
    "        # Need to hstack item_features to ensure each ItemIDs are present in the model\n",
    "        eye = sps.eye(self.n_items, self.n_items).tocsr()\n",
    "        self.ICM_train = sps.hstack((eye, self.ICM_train)).tocsr()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LightFMItemHybridRecommender: URM Detected 63 ( 0.6%) items with no interactions.\n",
      "EvaluatorHoldout: Processed 69643 (100.0%) in 3.29 min. Users per second: 353\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PRECISION</th>\n",
       "      <th>PRECISION_RECALL_MIN_DEN</th>\n",
       "      <th>RECALL</th>\n",
       "      <th>MAP</th>\n",
       "      <th>MAP_MIN_DEN</th>\n",
       "      <th>MRR</th>\n",
       "      <th>NDCG</th>\n",
       "      <th>F1</th>\n",
       "      <th>HIT_RATE</th>\n",
       "      <th>ARHR_ALL_HITS</th>\n",
       "      <th>...</th>\n",
       "      <th>COVERAGE_ITEM_CORRECT</th>\n",
       "      <th>COVERAGE_USER</th>\n",
       "      <th>COVERAGE_USER_CORRECT</th>\n",
       "      <th>DIVERSITY_GINI</th>\n",
       "      <th>SHANNON_ENTROPY</th>\n",
       "      <th>RATIO_DIVERSITY_HERFINDAHL</th>\n",
       "      <th>RATIO_DIVERSITY_GINI</th>\n",
       "      <th>RATIO_SHANNON_ENTROPY</th>\n",
       "      <th>RATIO_AVERAGE_POPULARITY</th>\n",
       "      <th>RATIO_NOVELTY</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cutoff</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0.180479</td>\n",
       "      <td>0.218305</td>\n",
       "      <td>0.134624</td>\n",
       "      <td>0.093949</td>\n",
       "      <td>0.110002</td>\n",
       "      <td>0.382283</td>\n",
       "      <td>0.176081</td>\n",
       "      <td>0.154215</td>\n",
       "      <td>0.728673</td>\n",
       "      <td>0.588949</td>\n",
       "      <td>...</td>\n",
       "      <td>0.090628</td>\n",
       "      <td>0.996637</td>\n",
       "      <td>0.726223</td>\n",
       "      <td>0.014357</td>\n",
       "      <td>7.584035</td>\n",
       "      <td>0.991341</td>\n",
       "      <td>0.073737</td>\n",
       "      <td>0.669664</td>\n",
       "      <td>1.941037</td>\n",
       "      <td>0.088522</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1 rows × 25 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       PRECISION PRECISION_RECALL_MIN_DEN    RECALL       MAP MAP_MIN_DEN  \\\n",
       "cutoff                                                                      \n",
       "10      0.180479                 0.218305  0.134624  0.093949    0.110002   \n",
       "\n",
       "             MRR      NDCG        F1  HIT_RATE ARHR_ALL_HITS  ...  \\\n",
       "cutoff                                                        ...   \n",
       "10      0.382283  0.176081  0.154215  0.728673      0.588949  ...   \n",
       "\n",
       "       COVERAGE_ITEM_CORRECT COVERAGE_USER COVERAGE_USER_CORRECT  \\\n",
       "cutoff                                                             \n",
       "10                  0.090628      0.996637              0.726223   \n",
       "\n",
       "       DIVERSITY_GINI SHANNON_ENTROPY RATIO_DIVERSITY_HERFINDAHL  \\\n",
       "cutoff                                                             \n",
       "10           0.014357        7.584035                   0.991341   \n",
       "\n",
       "       RATIO_DIVERSITY_GINI RATIO_SHANNON_ENTROPY RATIO_AVERAGE_POPULARITY  \\\n",
       "cutoff                                                                       \n",
       "10                 0.073737              0.669664                 1.941037   \n",
       "\n",
       "       RATIO_NOVELTY  \n",
       "cutoff                \n",
       "10          0.088522  \n",
       "\n",
       "[1 rows x 25 columns]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recommender = LightFMItemHybridRecommender(URM_train, ICM_genres)\n",
    "recommender.fit(epochs = 10)\n",
    "\n",
    "result_df, _ = evaluator_validation.evaluateRecommender(recommender)\n",
    "result_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Combining heterogeneous features\n",
    "\n",
    "FactorizationMachines can also account for more varied feature types, for example:\n",
    "* Other items in the user profile\n",
    "* Features related to how active a user is or popular an item is\n",
    "* One-hot (or multiple-hot) encoding of user groups\n",
    "* (normalized) Recommendation quality of other models for that user group (e.g., knowing how good is a TopPopular for them may help the model decide to increase the importance of models with higher popularity bias, or not)\n",
    "* User-item prediction computed by *other models*\n",
    "this way leaving to the FM to learn the weights and interactions.\n",
    "\n",
    "#### Challenges\n",
    "It may be difficult to train a FM if you have a set of heterogeneous features. Ensure they are normalized."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}