Idea: hyperparameter searching

[pfeatherstone]

This library offers so many useful parameters to tweak your architecture.
However, though @lucidrains offers insights from the papers and from experience, what works and what doesn't work ultimately depends on your data and your compute.
It's a bit daunting trying to figure out how to tune your model.
Rather than doing a blind manual search, maybe a hyperparameter searching algorithm would be a good idea. Maybe something like a genetic algorithm or similar.

[pfeatherstone]

Or maybe use something like https://github.com/optuna/optuna

[pfeatherstone]

This probably doesn't need to be added to the library. But maybe an example snippet or jupyter notebook would be cool. I might try it at some point. If i have some success, I'll submit a PR.