AUT-105: Add GitHub Workflows for building, integration and unit testing.

[oskirby]

This attempts to add a Github workflow to build the docker images then run the unit and integration tests. While github seems to build slower than CircleCI, we make up for it by running the integration tests in parallel with a matrix job.

In order to get the monitor tests working, we had to cherry-pick PR #908 onto this branch to clean up the handoff of the root hash between autograph and the lambda emulator.

This makes no attempt to deploy the image as we will need Dockerhub credentials to do that, and it would be better suited for a followup PR.

For a quick high-level tour of how this works:

• We create a new docker compose file tools/autograph-client/integration-tests.yml which extends the top level compose file, and adds a service for each integration test. This allows each test to define which container it runs on, the dependencies and how to run the test.
• Use Docker entrypoints for the tests to resolve startup timing between containers (IMO, this is a bug in Docker compose, but it's above my paygrade to fix that).
• Write a github action to build the docker containers and upload it as an artifact.
• Parse the integration test YAML to enumerate the test jobs.
• Use a matrix job to download the container and run the tests in parallel.

[jmhodges]

So, I've been thinking about this over the weekend. You've put a lot of work into this and it's good, but I'm kind of not sure about the goal here. My understanding of this AUTOGRAPH_ROOT_HASH variable was to prevent configurations from accidentally being changed when we didn't want them to be changed. This seems to make this check not actually a backstop check anymore. Are we sure we want to do this?

[jmhodges]

(If we wanted to make sure we were future proofed for changes to existing signer configs' chains, I'd recommend adding key agility here, instead, and not doing this config lookup work.)

[oskirby]

It's all a bit of a moot point as I am not sure if the monitor lambda was ever deployed anywhere, but this doesn't actually change anything in how the monitor lambda works, and the lambda-setup-entrypoint.sh script here is only being used in the test containers. In theory, if a lambda was deployed in production, it's value of AUTOGRAPH_ROOT_HASH would be hardcoded in the AWS environment and it would serve as the backstop check as intended.

[oskirby]

That being said, I do admit that I very much over-engineered the solution :)

[oskirby]

The objective that I was working towards specifically in this train of work was to reproduce the monitor tests using a Github workflow despite making zero code changes to the autograph-monitor binary.

[jmhodges]

Okay, so, in the time since this and while you were out, Jon and I have figured out that the monitor lambda does actually run, and is actually erroring. Interestingly, those errors are different in stage and in prod. I'll show you tomorrow when we're going through AWS stuff.

I think we could drop back this change to the non-config stuff, update the normandy root key value in autograph.softhsm.yaml and weave it through the other bits, and still get some success. Do you think so?

Maybe I'm missing a piece, though.

[oskirby]

Okay - I have rebased this work to abandon the config API, and implemented the monitor test for the non-HSM version only. I don't know if the monitor-hsm test really adds any value right now anyways.