Benchmarking runtimes

[scottstanie]

Top level issue tracking the improvements/additions needed to track a full-frame workflow runtime.

Current knobs to test are

1. Using CPU vs GPU
2. SLC input stack size
3. Output posting (or equivalently strides)
4. Size of blocks loaded at one time from the input stack
5. For CPU, number of CPUs/number of threads per CPU
6. Algorithm for phase linking (MLE vs EVD)

I've put these in the order of my guess for which will have the biggest effect, but we clearly need to do the tests to see.

Things we need for good testing

• the single-update workflow script (Create the single-update version of the s1_disp workflow #11)
• recording the threads (Setting thread counts for each process #28)
• recording the block size/fixing the max_ram_gb option (Block size setting and fixing the max_ram_gb option of dolphin config #32)
• adding the ability to use EVD instead of MLE (Add use_evd option to force EVD over EMI #138 )
• using vmtouch -e on the SLC stack before starting the workflow: https://github.com/hoytech/vmtouch . This will make sure we don't get very fast runs just because the SLC data as cached, as we can't count on that happening for the production runs.