Investigate the effect of sorting synIDs for coalescing

[denisalevi]

This should include an evaluation (profiling) for difference sorting options and should possibly be possible to set by the user in the brian2cuda preferences.

[denisalevi]

In the synapses kernel we have a thread <-> synapse correspondence, where in heterogeneous delay mode the synapses are sorted by delays and in homogeneous delay mode by synapse IDs. Therefore the writes to postsynaptic neurons will very likely not be coalesced (depending on the synapse creation, e.g. for all to all connections synapse IDs might already be sorted by postsynaptic neuron IDs). Sorting the synapses by postsynaptic neuron IDs (in heterogeneous delay mode only within the same delay) could increase coalescing.

EDIT: probably only useful to coalescing if a pre neuron is connected to e.g. all post neurons. So user preferences make sense here.

[moritzaugustin]

heterog mode:

• bundle queues: simply sort bundles in target mode by target neurons, in synapse mode by synapse ids
• synapse queues: we would have to sort the spikequeues in each timestep and we will put this to wishlist (if useful at all)

homog mode

• sort connectivity matrix w.r.t. post neuron in target mode and w.r.t. synapse id in synapse mode

ensure that performance with standard examples does not degrade

[denisalevi]

Also check again what I wrote in #30

[denisalevi]

Special case to take into account:

If there is only one synaptic pathway direction (either pre->post or post->pre, not both), then all synapses should just be relabeled once they are sorted into the connectivity matrix, such that the synapse IDs are consecutive in the connectivity matrix. This makes the connectivity matrix obsolete an lets all synapse variable automatically be stored in the connectivity matrix structure, allowing coalesced memory accesses. If there are both synaptic pathway directions, variables can only be sorted for one of them (Brette2012 review). In that case, the graph-cut idea might be interesting.

This all should in the best case already be happening during synapse creation (if possible), by changing the looping order (e.g. first over pre then over post neurons to sort by post per synapses group, as is likely happening by default). Should be tackled together with #177 .

[denisalevi]

Any sorting of synaptic variables would require some mechanism for sorting variables (and potentially monitor recordings) back into the format expected on Python side, see my comment in #271.