Model ranch

On this page we are collecting all the models we might want to use as "anchor models" for the first Pyro release -- that is, models / examples that must perform well before we are happy to release.

The goal is to initially make a tool that makes it easy to do VI with deep generative models. So most of these models should be examples from papers that use VI of one sort or another. (So models that we expect to only work with Monte Carlo methods aren't great candidates for the anchor set... but if we're not sure we should include in the brain storm list and we'll sort through later.)

Brain storm

Some concepts / components to cover with the examples:

• Recognition models
• Local per data point (mean-field) params
• Reparam Elbo estimator
• Discrete variables and their estimators
• IID and time-series
• Non-elbo objectives
• Poisson process and extensions

All the examples

Put any model / example that might be an anchor candidate here. We'll sort them out below.

Ideally each example is a model with a target data set and a reference result from a paper.

• VAE
  • VLAE, LVAE, etc
  • Variational Inference with Normalizing Flows
  • IW-ELBo eg IWAE
• NADE (or other scaleable density estimation models -- normalizing flows?)
• Sigmoid belief net
• Classifier
  • Bayesian linear/logistic regression;
  • Bayesian neural net (paper with good VI BNN benchmark?)
• vanilla linear and logistic/probit regression
• cute bayesian robust modeling example, e.g. something like the examples in https://arxiv.org/pdf/1510.05078.pdf
• Semi-Supervised VAE
• Some mixture models (which?)
• A model with stochastic recursion (AIR? Grammar?)
• An inverse simulation model?
• LDA? (Could use same dataset as AVITM? Their model looks similar to what we did for DAIPP.)
• DMM (DMM subsumes DKF) / DVBF?
• VRNN / RVAE?
  • Generating Sentences from a Continuous Space
• Something on ImageNet (to verify scaling)
• Bayesian RL survey
  • Policy gradient
• DRAW
• Some active learning example
  • Deep Bayesian Active Learning with Image Data
• Collaborative filtering
  • Probabilistic matrix factorization
    • Probabilistic PCA
  • Collaborative filtering + deep regression
• GP regression and classification on UCI datasets
  • GPs in PyTorch
• log-Gaussian Cox process (Poisson process with GP intensity), maybe applied to basketball data? ex
• A state-of-the-art language model (even if it's not variational)
  • AWD-LSTM
• Hierarchical models
• Weak labeling ala snorkel

Prioritized anchor candidates

Gold

• VAE (and friends? IWAE, normalizing flows)
• Semi-Supervised VAE
• AIR
• DMM and/or DVBF
• Bayesian regression

Silver

• Weak labeling
• Bayesian NN