[EXP] Reinitialization

[AlexanderFabisch]

Fitting time-indexed trajectories with GMMs is very difficult without reasonable restart strategies. With this improvement it is possible to fit the following trajectories:

The time variable is not visible but it is proportional to x_1. Each of the 200 lines corresponds to a trajectory of 100 steps. We fit a GMM with 200 x 100 samples. Each sample consists of (t, x_1, x_2). Ellipses correspond to individual Gaussians projected to (x_1, x_2)-axes. Dots are samples from GMMs conditioned on t.

Here is a direct comparison between GMMs learned with intelligent resets (left) and without (right) for 30 Gaussians:

A comparison between GMM with reinitialization (left) and Dirichlet process GMM from sklearn (right):

Conclusion

• Bayesian GMMs work better than restart strategies for this problem.
  • Add an example
  • Document this in readme
• Make a new PR, extract the following from this one:
  • regularization for covariance should be a parameter
  • Example
  • k-means++ refactoring
  • GMM.to_mvn()
  • Small fixes for examples
  • Docstring improvements
  • Better code documentation

[AlexanderFabisch]

This is more effective than reinitialization.