[rush] (BREAKING CHANGE) Overhaul watch-mode to facilitate orchestration

[dmichon-msft]

Summary

Completely retools the watch engine in Rush to facilitate better interaction with plugins that wish to orchestrate the build process. Makes the Rush execution engine stateful across an entire Rush watch session.

BREAKING CHANGES

• The only hook on PhasedCommandHooks that has not been moved or renamed is beforeLog.
• createOperations is now createOperationsAsync, and the properties on the context parameter have changed.
• All manipulation of the runtime graph and taps into the build process are now in the ExecutionManagerHooks object, accessible via the executionManagerAsync hook. Access executionManager.hooks for beforeExecuteOperationAsync, afterExecuteOperationAsync, beforeExecuteOperationsAsync, afterExecuteOperationsAsync, and onWaitingForChanges.

Details

The new lifecycle of a Rush phased command is that the command first invokes createOperationsAsync to create the session-long operation graph. This set of operations is then used to construct an IOperationExecutionManager that will own the lifecycle of the execution session.

There is a new watch option includeAllProjectsInWatchGraph that, if set to true, will cause Rush to build the graph with all projects in rush.json, regardless of CLI selection parameters. Selected projects will only affect which projects are enabled for execution during the initial run. This also allows for a bare command, e.g. rush start to select no projects. This feature is intended for use with plugins that offer the ability to alter the enabled/disabled states of operations in the graph while the session is ongoing. For an example, see @rushstack/rush-serve-plugin, which facilitates altering these states via Web Socket messages.

All in-repo plugins that interact with the Rush execution graph have also been updated.

This PR in its initial draft includes an AI-coded interactive live status dashboard; this dashboard will be extracted prior to checkin and moved to its own PR because it needs refactoring to have a proper build system with static analysis applied.
While this dashboard is included, you can test out all the functionality by checking out this branch and running:
rush install && rush build -t rush
node ./apps/rush/lib/start-dev.js start --port 4321 (or whatever number)
Then open https://localhost:4321/ in the browser. Currently it does not attempt to automatically launch, but VSCode may offer you a popup to launch it for you when it does automatic port forwarding.

How it was tested

Added unit tests for all functionality of the new IOperationExecutionManager API contract.
Manual validation via the rushstack repo's rush start command for the CLI interaction (enable/disable debug or verbose, alter parallelism, pause/resume, kick a single build, invalidate, close runners).

Impacted documentation

All watch-mode documentation. Plugin documentation for phased commands.