Group trigger

[hjoliver]

Implement group trigger proposal: cylc/cylc-admin#197

(Not approved yet, but it's a clear winner and a relatively small change, and I wanted to code it up while fresh in the brain).

On current master:

• on triggering one or more tasks: run them all immediately regardless of prerequisites.

On this branch:

• on triggering one or more tasks, satisfy any off-group (aka off-flow) prerequisites

Result: we can trigger a sub-graph "naturally" by specifying all of its member tasks:

• tasks with only off-group prerequisites will trigger immediately
• the subsequent flow will respect in-group prerequisites
• no stalling due to unsatisfied off-group prerequisites

This provides an alternative, easier for some cases, to triggering just the initial task(s) of the sub-graph and setting off-flow prerequisites (if any) manually.

---

example

[scheduler]
    allow implicit tasks = True
[scheduling]
    [[graph]]
        R1 = """
            foo => a => b => c => bar
            off => b
        """
[runtime]
    [[root]]
        pre-script = sleep 4
    [[bar]]
        script = "test $CYLC_TASK_SUBMIT_NUMBER != 1"

Run it and wait for bar to fail and stall the workflow.

$ cylc trigger grp //1/a //1/b //1/c --flow=new
  # 1/a runs immediately in flow 2 - its only prerequisite (foo => a) is off-group 
  # 1/b gets its off-group prerequisite satisfied (off => b) and spawns to wait on in-group (a => b)
  # flow 2 traverses a => b => c, then flows on to merge with and run bar to complete the workflow

---

updated post cylc remove extension

By default (no use of --flow) we "remove" target tasks first, to allow re-run without starting a new flow.

---

upcoming inactive task matching developments

We can't trigger a group of inactive tasks by family name or glob yet, but that's a general problem, not specific to this branch.

---

Check List

• I have read CONTRIBUTING.md and added my name as a Code Contributor.
• Contains logically grouped changes (else tidy your branch by rebase).
• Does not contain off-topic changes (use other PRs for other changes).
• Applied any dependency changes to both setup.cfg (and conda-environment.yml if present).
• Tests are included (or explain why tests are not needed).
• Changelog entry included if this is a change that can affect users
• Cylc-Doc pull request opened if required at cylc/cylc-doc/pull/XXXX.
• If this is a bug fix, PR should be raised against the relevant ?.?.x branch.

[hjoliver]

(Updated after major conflicting changes on master)

[hjoliver]

Note on changes to reference logs: group trigger causes triggering by satisfying prerequisites, rather than by direct triggering despite unsatisfied prerequisites ... so triggering now gets logged as (e.g.) 1/b triggered off ['1/a'] instead of 1/b triggered off [ ].

[hjoliver]

(All tests were passing until my latest change to incorporate the new cylc remove stuff ... TBD)