Mixture of experts layer for Keras

This repository contains a Keras layer implementing a dense mixture of experts model:

Some of the main arguments are as follows:

• units: the output dimensionality
• n_experts: the number of experts ()
• expert_activation: activation function for the expert model ()
• gating_activation: activation function for the gating model ()

Please see MixtureOfExperts.py for additional arguments. The file moe_demo.py contains an example demonstrating how to use this layer. The example there essentially implements the simulations reported in this blog post.