(Kept from previous group [paf22])
Summary: This page explains the GitHub build action we use to first:
- create an executable image of our work
- evaluate our Agent with the leaderboard
Tim Dreier, Korbinian Stein
2.12.2022
- GitHub actions
The workflow defined in .github/workflows/build.yml
creates an executable image
which can later be submitted to the CARLA leaderboard and pushes it
to GitHub Packages.
The image can then be pulled with docker pull ghcr.io/ll7/paf22:latest
to get the latest version
or docker pull ghcr.io/ll7/paf22:<version>
to get a specific version.
If action is triggered by a pull request the created image is then used to execute a test run in the leaderboard, using the devtest routes. The results of this simulation are then added as a comment to the pull request.
The Dockerfile (build/docker/build/Dockerfile
)
The Dockerfile uses Dockerfile+ to include the agent Dockerfile to avoid duplicate code. The code folder is then copied into the container instead of mounting it as in our dev setup.
1. Checkout repository (actions/checkout@v3
)
Trivial, just checks out the repo.
2. Set up Docker Buildx (docker/setup-buildx-action@v2
)
Set's up Buildx. This is needed to set up the correct driver to allow caching in step 5.
Detailed description why this is needed can be found here.
3. Log in to the Container registry (docker/login-action@v2
)
Logs in with GITHUB_TOKEN
into the registry (ghcr.io).
Example taken from here
4. Bump version and push tag (mathieudutour/github-tag-action
)
If the current commit is on the main
branch, this action bumps the version and pushes a new tag to the repo.
Creates a new tag with a semantic version number for the release.
The version number is determinated by the name of the commits in the release.
This is possible since conventional commits are enforced by comlipy as described here.
Commit message | Release type | Previous version number | New version number |
---|---|---|---|
fix(#39): build failing due to incorrect configuration | Patch Release | 0.0.1 | 0.0.2 |
feat(#39): Add automatic build process | Minor Release | 0.0.1 | 0.1.0 |
Major releases can be done manually (e.g. git tag v1.0.0
).
If Step 4 was skipped, this step gets the commit hash of the current commit, to be used as a tag for the Docker image.
Build and push the image to the registry. To avoid large downloads of the base image
the GitHub Actions cache
is used to cache the image after build.
If the action is run on a branch other than main
, the image is tagged with the commit hash from Step 5.
Otherwise, the image is tagged with both the tag created in Step 4 and latest
.
The drive
job is executed conditionally on pull_request
, after the build successfully ran through.
1. Checkout repository (actions/checkout@v3
)
Same step as in the build job
Runs the agent with the build/docker-compose.test.yml
that only contains the
bare minimum components for test execution:
- Carla Simulator (running in headless mode)
- roscore
- Agent container, run through the
Carla
leaderboard_evaluator
.
Copies the created simulation_results.json
file out of the agent container into the current container
Stops the remaining containers (Carla, roscore) and removes the volumes with:
$ docker-compose down -v
.
This step is important to clean up the remaining containers to have a clean run everytime. This is also the reason for
the if: always()
, that ensures step execution.
5. Comment result in pull request actions/github-script@v6
This steps uses a JS script to parse the simulation results and add a comment with a results table to the corresponding pull request.
An example comment for this would be:
Metric | Value |
---|---|
Avg. driving score | 0.06006 |
Avg. route completion | 0.22 |
Avg. infraction penalty | 0.273 |
Collisions with pedestrians | 0.0 |
Collisions with vehicles | 62.046 |
Collisions with layout | 62.046 |
Red lights infractions | 0.0 |
Stop sign infractions | 0.0 |
Off-road infractions | 0 |
Route deviations | 0.0 |
Route timeouts | 62.046 |
Agent blocked | 0.0 |
Yield emergency vehicles infractions | 0.0 |
Scenario timeouts | 62.046 |
Min speed infractions | 0.0 |