Skip to content

Commit

Permalink
Fix issues with Docker Hub rate limits in CI (#283)
Browse files Browse the repository at this point in the history
Similar to the Node.js CNB repo, this repo is occasionally hitting
Docker Hub rate limit errors on the ARM jobs in the matrix like so:
https://github.com/heroku/buildpacks-go/actions/runs/9796334219/job/27051359595#step:8:10

These are due to the fact that the `pub-hk-ubuntu-22.04-arm-small`
ARM runner also has a static IP enabled, which means all traffic from
any runners in that group go through a single load balance with the
fixed IP. This not only causes throughput/reliability issues, but means
that we're more likely to hit the Docker Hub rate limit.

As such, we requested the creation of some new runner groups that
separate out the "need a fixed IP for writes" vs "need more compute
or an ARM CPU" use-cases. As an added bonus these groups are also
using Ubuntu 24.04. See:
https://salesforce.quip.com/bu6UA0KImOxJ#temp:C:GZR57becb9df8d94f80b132168fd

This PR switches to those new groups, and also adds pre-pulling of
the run image to prevent it being pulled multiple times when the
tests run (`libcnb-test` uses `--pull-policy if-not-present` with
`pack build`, but the tests run concurrently so on first run multiple
pulls will otherwise be performed). See also:
heroku/buildpacks-python#223

GUS-W-16238120.
  • Loading branch information
edmorley authored Jul 15, 2024
1 parent d376675 commit a94957b
Show file tree
Hide file tree
Showing 3 changed files with 15 additions and 5 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/check_changelog.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ permissions:

jobs:
check-changelog:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
if: (!contains(github.event.pull_request.labels.*.name, 'skip changelog'))
steps:
- name: Checkout
Expand Down
16 changes: 13 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ env:

jobs:
lint:
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
steps:
- name: Checkout
uses: actions/checkout@v4
Expand All @@ -29,7 +29,7 @@ jobs:
run: cargo fmt -- --check

unit-test:
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
steps:
- name: Checkout
uses: actions/checkout@v4
Expand All @@ -42,7 +42,7 @@ jobs:

integration-test:
name: integration-tests ${{ matrix.builder }} / ${{ matrix.arch }}
runs-on: ${{ matrix.arch == 'arm64' && 'pub-hk-ubuntu-22.04-arm-small' || 'ubuntu-latest' }}
runs-on: ${{ matrix.arch == 'arm64' && 'pub-hk-ubuntu-24.04-arm-medium' || 'ubuntu-24.04' }}
strategy:
matrix:
arch: ["amd64"]
Expand All @@ -63,8 +63,18 @@ jobs:
uses: Swatinem/[email protected]
- name: Install Pack CLI
uses: buildpacks/github-actions/[email protected]
# The images are pulled up front to prevent duplicate pulls due to the tests being run concurrently.
- name: Pull builder image
run: docker pull ${{ matrix.builder }}
- name: Pull run image
# Using `docker inspect` rather than `pack builder inspect` since the latter makes
# additional requests to Docker Hub even when the image is available locally.
run: |
RUN_IMAGE=$(
docker inspect --format='{{index .Config.Labels "io.buildpacks.builder.metadata"}}' '${{ matrix.builder }}' \
| jq --exit-status --raw-output '.stack.runImage.image'
)
docker pull "${RUN_IMAGE}"
- name: Run integration tests
env:
INTEGRATION_TEST_BUILDER: ${{ matrix.builder }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/inventory.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ permissions:
jobs:
update-go-inventory:
name: Update Go Inventory
runs-on: pub-hk-ubuntu-22.04-small
runs-on: pub-hk-ubuntu-24.04-ip
steps:
- uses: actions/create-github-app-token@v1
id: generate-token
Expand Down

0 comments on commit a94957b

Please sign in to comment.