Future improvements for processing delays

[kjohnsen]

At present, "serial" processing only operates on the entire processor, leading to the unrealistic scenario of the processor being unable to sample until the output is finished. On the contrary, we expect that I/O could take up much of the round-trip time and not cause samples to back up, even if the core computations must be executed serially.

What would be better would be for serial sampling to separate the computational part of the processing, which can't overlap one sample to another, from the I/O, which could.

But what about when some processing blocks can be executed in parallel while others can't? In that case, the most correct, general capability would be to specify the processing as a directed, acyclical graph between components, where the dependencies would automatically determine what could be parallelized and the resulting latency would reflect the longest path through that graph.

That said, on 1/26/22 Chris, Adam, Nathan, and I decided that this complexity is unneeded for the vast majority of use cases, and we'll simply make fixed sampling, parallel processing the default. The other two (fixed+parallel, when idle+serial) would probably only be useful after implementing some of these proposed changes, if we ever decide it's worth the effort.