Release + testing integration with -runner & action

[cisaacstern]

0.10.0 (i.e. beam-refactor) is now released, woohoo! 🥳

And one of the first things that happens was that @jbusecke realized it broke the GitHub Action due to a -runner incompatibility.

The fix was easy enough, but the purpose of this issue is discuss how we can catch and fix this type of integration problem before there is an official -recipes release.

One way of handling this would be to consolidate -recipes and -runner into a monorepo, i.e. #514, however @yuvipanda has made some compelling points in favor of keeping them separate, and I'm mostly convinced keeping them separate is best.

Assuming -recipes and -runner do remain separate, would this be a manageable release process:

• New releases of -recipes always start with a release candidate (0.10.0rc, 0.10.0alpha or similarly named).
• Upon release of a -recipes release candidate, a workflow automatically opens a PR on -runner to test it against the -recipes release candidate. Tests will either pass immediately or require some triage, but the PR will eventually be merged. (In cases where no changes are needed to -runner, the content of this PR would just be to add the new -recipes release to the testing matrix there.)
• Once the automated -runner PR is merged, a subsequent automated PR is opened on the GitHub Action, again confirming compatibility.
• Once the GitHub Action testing PR passes, a final automated issue is opened on -recipes notifying the core maintainers here that it's okay to upgrade the release candidate to an official release.

So, visually:

sequenceDiagram
    recipes->>recipes:cuts release candidate
    recipes->>runner:opens testing PR
    runner->>runner:tests pass
    runner->>action:opens testing PR
    action->>action:tests pass
    action->>recipes:opens issue approving release

Loading

What do others think of this? It's a bit complex, yes, but I'm not sure how it could be simpler, while also maintaining all of this code in separate repos, and ensuring releases don't break downstream components of the platform.

[thodson-usgs]

I'm ignorant and hesitant to chime in, but you asked. So, why wouldn't you do it this way:

To release a recipe, I open pr -> action tests recipe with runner -> hit the green button

To release a runner, you open a pr -> action dynamically populates testing matrix will all recipes -> hit the green button