Tuning reorder123x321_kernel

[bartvstratum]

I started on this kernel...

[isazi]

For this particular kernel, the order of dimensions may matter, because ii and ik are mapped to x and z:

const int idx_out = ii + ij*ni + ik*nj*ni;
const int idx_in = ik + ij*nk + ii*nj*nk;
arr_out[idx_out] = arr_in[idx_in];

In this case best memory bandwidth could be achieved by reading in shared memory, reordering, and then writing to output.

[bartvstratum]

Yes, I agree. So far I only did a simple block size tuning, which already makes the kernel almost a factor 2 faster: 951ebd3

[isazi]

Do you want to try shared memory, or should I do it?

[isazi]

I did actually write a first (dirty) implementation (in my test branch). I will test it tomorrow and let you know.

[bartvstratum]

Yes, please, go ahead :-)

[isazi]

Implemented the shared memory version. Performance is the same as the standard (tuned) version, most probably because the cache behavior is so good. So it does not make sense to use this implementation in practice right now. However, if in the future data size is going to increase, shared memory may be able to outperform the cache.

[benvanwerkhoven]

I was thinking that we could use the CUB library to make a fast version for this kernel: https://nvlabs.github.io/cub/classcub_1_1_block_exchange.html

CUB has all kinds of optimizations built-in to avoid memory bank conflicts for all kinds of devices, data types, and block sizes, and uses warp shuffle instructions where possible. If you can fit the use case that we have here onto such a block-wide primitive in CUB it's probably the best performing option and you can still tune things like items per thread and thread block dimensions.