std.Random.shuffle: optimize for cache utilizing @prefetch input queue

[Olvilock]

I've been doing a simulation, where one of the steps was shuffling a big table of particles (~10^7 elements). The std.Random.shuffle method turned out to be the major contributor into overall runtime. The main reason lies in inherent cache unfriendliness of the random-access swaps.

The solution that worked for me: precompute the swap indices 32 steps ahead of time, @prefetch them and put them into a ring buffer. As the array was cold, prefetching achieved over 3x speedup.

This pull request presents a hybrid approach that works well on both hot and cold arrays (threshold optimized for hot memory)

Benchmark for worst case (hot memory) results are attached in screenshots (ReleaseFast, CPU: Intel i5 8300H, memory freq. 2400 MHz).

[Rexicon226]

Nice - I wonder if setting the prefetch locality to 0 would improve it further.

[Olvilock]

@Rexicon226 strangely, it did not (on my machine)

[Rexicon226]

not too surprising - im not aware of x86 having a less-temporal prefetch and i doubt llvm is smart enough to do anything with the information. thanks for checking!

[Olvilock]

Is there/might there be a standard way of querying cache sizes of target machine, if applicable? I hardcoded a constant to switch between implementations in this pull request, a function of (L3) cache size is more suited as a choice of that constant