Top-K Off-Policy Correction for a REINFORCE Recommender System

Read the article:

Check the notebooks out!

• Inner workings of REINFORCE inside recnn (optional).ipynb
• Basic Reinforce with RecNN.ipynb
• Reinforce Off Policy Correction.ipynb
• TopK Reinforce Off Policy Correction.ipynb

All of the notebooks are located under RecNN/examples/2. REINFORCE TopK Off Policy Correction/

Look at it online with TensorBoard visualization

Algorithms (DDPG, TD3), Tests, Docs, and Environment overhaul

Phew, that's been a journey.

Features:

• Base algorithms are added and tested
• Environments are now completely redone and can be used for your data
• Online tutorial
• Code Climate grades my code B
• CircleCI tests written
• Somewhat reminiscent of documentation. It will be more complete soon.

Coming soon

• PyPi page
• BCQ implementation will be stress tested and tweaked
• TopK Off Policy Correction

RecNN Environment Package created

RecNN Environment Package created

1. At this point, I have removed all the junk code from the notebooks, allowing you to focus on the implementation.
   Code was moved over to following packages: Debugger, Plotter, Optimizers (only Radam copy for now), Models (Actor, Critic, bcqGenerator, bcqPerturbator), DataLoader. Learning functions for the models are coming next release.
2. Copyrighted ML20M dataset was completely removed, I implemented custom DataLoader that natively supports ml20m dataset and others with a similar data structure. As of now, it only supports pandas, however, Dask + Numba support will be added

As of now, this repo is in Alpha stage. Next things I am working on:

1. Seamless reco-gym [github.com/criteo-research/reco-gym] integration
2. LSTM versions of algorithms
3. Advanced BCQ VAE generator implementation with autoregressive normalizing flows