benchmark costs of preparing activations

[jlquinn]

When neural net code runs, typically one matrix is the parameters, which can be reorganized once and reused. The other matrix is typically activations, which must be prepared on every matrix multiply call. It would be interesting to have benchmarks that include the time for preparing the activation values.

[XapaJIaMnu]

Yes, you are correct, we should factor in the Preparation functions. So far i have only benchmarked the best case scenario, which is rather unrealistic, since activations do not come prepared.

On the other hand I realised that OneDNN doesn't expose its prepare routine...