sequential learning with time constant

[jacowp357]

Hi, is it possible to adapt model parameters across time placing higher importance on more recent data points than previous data points? Perhaps a time constant can be introduced to re-scale the model parameters? Or is this already possible in the current version of river? In other words, it assumes non-stationary data within some time window.

[MaxHalford]

That's already exactly what online learning does :). See this thread.

[jacowp357]

So I guess the learning rate will do something like this: https://github.com/infer-dot-net/2-naive-bayes-classifier/blob/e921bf6d130faa5b451fc3d7535321cb1e060096/models/model/Program.cs#L149-L171 ?

[MaxHalford]

Sorry but I have no idea what's going on in that piece of code. The learning rate simply affects the rate at which the weights move. A higher learning rate will better adapt to drift but might not converge as well.

[jacowp357]

In the "Bayesian world", the idea of the learning rate is captured by the amount of allowable noise when transitioning from one time step to another. Can the learning rate also adapt in river?

[MaxHalford]

There are adaptive learning rates yes. You can check out the optim.schedulers submodule :)