Only send flushes when Downstairs is idle; send Barrier otherwise

[mkeeter]

This PR removes automatic flushes, per RFD 518. Instead, the new Barrier operation is sent. If the system is idle for a particular amount of time, we send a final Flush to put everything into a known state.

When the Upstairs retires jobs after a barrier operation, the system as a whole becomes ineligible for replay. This state determines whether the new Downstairs reconnects through Offline (which does replay) or Faulted (which does live-repair instead).

Removing automatic flushes is a noticeable performance improvement:

(tested on the London mini-cluster, with Upstairs and 3x Downstairs on different sleds)

region info:
  block size:      4096 bytes
  blocks / extent: 16384
  extent size:     64 MiB
  extent count:    2048
  total blocks:    33554432
  total size:      128 GiB
  encryption:      yes

[mkeeter]

Rebased to stage on top of #1507, because we want to only send Barrier operations if there's enough bytes / jobs buffered (which requires keeping track!)