Skip to content

Commit

Permalink
Login to docker.io (#5957)
Browse files Browse the repository at this point in the history
This is an attempt to mitigate the rate limit issue we are seeing when
pulling the base image from docker.io
https://docs.docker.com/docker-hub/download-rate-limit/#rate-limit.

Here is the summary of my support ticket to GitHub about the issue:

1. This behavior is from Docker Hub itself and isn't something GitHub
can directly control. You might consider a paid subscription to Docker
Hub which increases the rate limit. You would need to pre-authenticate
to Docker Hub on these runners before the job starts as GitHub builds
the Docker containers before the first explicit step on the job.
2. Alternatively, we could move the images to https://ghcr.io which does
not have such rate limits.
3. GitHub has an agreement with Docker so that the rate limits do not
apply for publicly accessible images downloaded through a GitHub-hosted
runner but this does not apply to self-hosted runners.
 
This is basically the first approach. We could try it out and see if it
helps. The worst thing that could happen is to be rate limited again.

This goes together with
pytorch-labs/pytorch-gha-infra#544
  • Loading branch information
huydhn authored Nov 21, 2024
1 parent 7b1ffef commit f12c0d4
Showing 1 changed file with 8 additions and 0 deletions.
8 changes: 8 additions & 0 deletions .github/actions/calculate-docker-image/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,14 @@ runs:
retry login "${DOCKER_REGISTRY}"
- name: Login to Docker Hub
shell: bash
# It's ok if this steps fails, it would then be an anonymous user like what we used to have
continue-on-error: true
run: |
set -eux
aws secretsmanager get-secret-value --secret-id docker_hub_readonly_token | jq --raw-output '.SecretString' | jq -r .docker_hub_readonly_token | docker login --username pytorchbot --password-stdin
- name: Build docker image
if: ${{ steps.calculate-image.outputs.skip != 'true' && (inputs.always-rebuild || steps.check-image.outputs.rebuild) }}
env:
Expand Down

0 comments on commit f12c0d4

Please sign in to comment.