Data prefetching?

[Ebanflo42]

Question mark because it might be most reasonable to depend on another crate for this.

At the most abstract level, I imagine us having a Training struct which takes of course a model context and optimizer context but also very importantly an example generator. The generator should have a specified number of threads which it uses to load and preprocess and specified number of samples before they are required by the training loop. It should be known whether the generator is finite or infinite (counting epochs on an infinite training set would of course be silly).

Here is the simplest implementation in python: https://github.com/justheuristic/prefetch_generator. Note that all loading and preprocessing is user-defined, it basically just implements the multiprocessing aspect. Is there already a crate we can use for this?

[Ebanflo42]

We can just use rust iterators