feat(node): new voting reactor state machine

[cryptoAtwill]

This PR is the start of in total 4 PRs that implements the new topdown flow. The implementation is broken down as follows:

• Implement the vote tally skeleton state machine (This PR). The skeleton state machine only outlines the state transition from Paused -> Active -> Paused. The other two recovery modes are yet to be implemented.
• Implement the vote tally logic: feat(node): migrate vote tally  #1139
• Implement the new syncer: feat(node): refactor parent view syncer #1151
• Integrate certified topdown checkpoint proposal into fendermint execution (prepare, process, execute): feat(node): integrating topdown cert into interpreters #1187

For this PR, the key changes are:

• Using tokio single thread reactor mode
• Use an enum to represent the different states of the state machine
• Introduce operation mode state machine skeleton with Active and Paused operation mode
• Use events emitted from syncer to communicate with the state machine: TopDownSyncEvent

The state machine is as follow:

stateDiagram-v2
  [*] --> Paused : Process start
  Paused --> Active : Synced
  Active --> Recovery : Checkpoints quiet
  Recovery --> Active : Checkpoints observed
  Active --> Paused : Stopping
  Recovery --> Paused : Stopping
	
  state Recovery {
    [*] --> SoftRecovery
    SoftRecovery --> HardRecovery : Still no new topdown checkpoints
    SoftRecovery --> [*] : New checkpoints
    HardRecovery --> [*] : New checkpoints
  }

Loading

[LePremierHomme]

I found the Paused naming to be misleading, because it only relates to the state of the votes publishing (I assume), while other operations are still running. What about Sync/Syncing?

[cryptoAtwill]

Paused could be triggered by many other condition, mostly not just in syncing. Just not implemented yet.

[LePremierHomme]

What I mean is that Paused isn't really idle, so this name is a bit confusing. I didn't refer to what triggers that mode, but to how it's best to describe it (given what it's doing).

[cryptoAtwill]

Paused pauses the handling internal events.
If we want to change the name, I'm open to any better naming but I feel Idle is not much different either. think we can do this from the doc first or at the final PR.

[LePremierHomme]

I didn't suggest a rename to Idle, but rather that Paused suggests a state of idleness, while that's not the case.

Which doc are you referring to?