feat: Support GitHub Container Registry

[jopemachine]

Partially fix #2337.

Overview of the changes

base.py's _scan_tag update for handling MEDIA_TYPE_DOCKER_MANIFEST

In the existing code, only MEDIA_TYPE_DOCKER_MANIFEST_LIST and MEDIA_TYPE_OCI_INDEX types were handled in _scan_tag, so a runtime error occurred when MEDIA_TYPE_DOCKER_MANIFEST type was passed as content_type.

However, in harbor.py, handling based on manifest_media_type is processed by separate functions such as _process_oci_index and _process_docker_v2_multiplatform_image, so handling for MEDIA_TYPE_DOCKER_MANIFEST has been added by extracting these functions with the same signatures.

Adding handling for the MEDIA_TYPE_DOCKER_MANIFEST type (_process_docker_v2_image) seems not to be specific only to ghcr, so it was added to base.py.

Test

Manually tested it using the following methods.

In this PR, the cr.backend.ai/stable/python:3.9-ubuntu20.04 image is used as a placeholder for testing.

Prerequisite

1. Tag and push the cr.backend.ai/stable/python image to your GitHub package for testing.

❯ docker tag cr.backend.ai/stable/python:3.9-ubuntu20.04 ghcr.io/<github_username>/python:3.9-ubuntu20.04
❯ docker push ghcr.io/<github_username>/python:3.9-ubuntu20.04

2. Enter the following command for putting the necessary key-values in etcd.

❯ ./backend.ai mgr etcd put config/docker/registry/ghcr.io "https://ghcr.io"; \
./backend.ai mgr etcd put config/docker/registry/ghcr.io/type "github"; \
./backend.ai mgr etcd put config/docker/registry/ghcr.io/entity_type "users"; \
./backend.ai mgr etcd put config/docker/registry/ghcr.io/username "<github_username>"; \
./backend.ai mgr etcd put config/docker/registry/ghcr.io/password "<github_PAT>"

Note

The entity_type should be either "users" or "orgs".

3. Insert the following value into the container_registry column of the groups table.

{
	"registry": "ghcr.io",
	"project": "<github_username>"
}

4. Add ghcr.io value to allowed_docker_registries column of domains table using below command

❯ ./backend.ai admin domain update default --allowed-docker-registries=ghcr.io

Test scenarios

Verify that the container registry added in this PR is functioning based on several scenarios.

If there are additional scenarios that need testing, please leave a comment.

1. Image Rescan

Image rescanning should work through the following command.

❯ ./backend.ai mgr image rescan ghcr.io

2. Create a session from the pulled image

Run a session using the downloaded image.

❯ ./backend.ai session create ghcr.io/<github_username>/python:3.9-ubuntu20.04

3. Commit the changes and push it to the container registry.

For committing a session, it is necessary to assume that the your "GitHub ID" is included in config/docker/registry/ghcr.io/project in etcd.

Enter the key value using the following command.

❯ ./backend.ai mgr etcd put config/docker/registry/ghcr.io/project "<github_username>"

Commit the changes using the convert-to-image command and push them to the container registry.

❯ ./backend.ai session convert-to-image <session_id> test_imgname

∙ Request to commit Session(name or id: ce592e27-b90a-4859-92b1-360fdd6d8464)
100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:01<00:00,  2.02it/s]
✓ Session export process completed.

---

Checklist: (if applicable)

• Milestone metadata specifying the target backport version
• Mention to the original issue

[graphite-app]

Your org has enabled the Graphite merge queue for merging into main

Add the label “flow:merge-queue” to the PR and Graphite will automatically add it to the merge queue when it’s ready to merge. Or use the label “flow:hotfix” to add to the merge queue as a hot fix.

You must have a Graphite account and log in to Graphite in order to use the merge queue. Sign up using this link.

[jopemachine]

• feat: Support AWS ECR Private Container Registry #2553
• feat: Support AWS ECR Public Container Registry #2549
• feat: Support GitLab Container Registry #2348
• feat: Support GitHub Container Registry #2341 👈
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @jopemachine and the rest of your teammates on Graphite

[jopemachine]

Closed as #1917 stack was merged.