feat: Support GitLab Container Registry

[jopemachine]

Partially fix #2337.

Test

Manually tested it using the following methods.

In this PR, the cr.backend.ai/stable/python:3.9-ubuntu20.04 image is used as a placeholder for testing.

Prerequisite

Note: The example given refers to GitLab Hosted Registry, but it also works in an on-premises environment.

1. Tag and push the cr.backend.ai/stable/python image to your GitLab package for testing.

❯ docker tag cr.backend.ai/stable/python:3.9-ubuntu20.04 registry.gitlab.com/<gitlab_username>/python:3.9-ubuntu20.04
❯ docker push registry.gitlab.com/<gitlab_username>/python:3.9-ubuntu20.04

2. Enter the following command for putting the necessary key-values in etcd.

❯ ./backend.ai mgr etcd put config/docker/registry/registry.gitlab.com "https://registry.gitlab.com"; \
./backend.ai mgr etcd put config/docker/registry/registry.gitlab.com/type "gitlab"; \
./backend.ai mgr etcd put config/docker/registry/registry.gitlab.com/gitlab_url "https://gitlab.com"; \
./backend.ai mgr etcd put config/docker/registry/registry.gitlab.com/project "<gitlab_username>"; \
./backend.ai mgr etcd put config/docker/registry/registry.gitlab.com/gitlab_project "<project_name>"; \
./backend.ai mgr etcd put config/docker/registry/registry.gitlab.com/username "<gitlab_username>"; \
./backend.ai mgr etcd put config/docker/registry/registry.gitlab.com/password "<gitlab_PAT>"

3. Insert the following value into the container_registry column of the groups table.

{
	"registry": "registry.gitlab.com",
	"project": "<gitlab_username>"
}

4. Add registry.gitlab.com value to allowed_docker_registries column of domains table using below command

❯ ./backend.ai admin domain update default --allowed-docker-registries=registry.gitlab.com

Test scenarios

Verify that the container registry added in this PR is functioning based on several scenarios.

If there are additional scenarios that need testing, please leave a comment.

1. Image Rescan

Image rescanning should work through the following command.

❯ ./backend.ai mgr image rescan registry.gitlab.com

2. Create a session from the pulled image

Run a session using the downloaded image.

❯ ./backend.ai session create registry.gitlab.com/<gitlab_username>/python:3.9-ubuntu20.04

3. Commit the changes and push it to the container registry.

For committing a session, it is necessary to assume that the your "GitLab ID" is included in config/docker/registry/registry.gitlab.com in etcd.

Enter the key value using the following command.

❯ ./backend.ai mgr etcd put config/docker/registry/registry.gitlab.com/project "<gitlab_username>"

Commit the changes using the convert-to-image command and push them to the container registry.

❯ ./backend.ai session convert-to-image <session_id> test_imgname

∙ Request to commit Session(name or id: ce592e27-b90a-4859-92b1-360fdd6d8464)
100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:01<00:00,  2.02it/s]
✓ Session export process completed.

---

Checklist: (if applicable)

• Milestone metadata specifying the target backport version
• Mention to the original issue

[graphite-app]

Your org has enabled the Graphite merge queue for merging into main

Add the label “flow:merge-queue” to the PR and Graphite will automatically add it to the merge queue when it’s ready to merge. Or use the label “flow:hotfix” to add to the merge queue as a hot fix.

You must have a Graphite account and log in to Graphite in order to use the merge queue. Sign up using this link.

[jopemachine]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• feat: Support AWS ECR Private Container Registry #2553
• feat: Support AWS ECR Public Container Registry #2549
• feat: Support GitLab Container Registry #2348 👈
• feat: Support GitHub Container Registry #2341
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @jopemachine and the rest of your teammates on Graphite

[kyujin-cho]

What is the meaning of project directive on Gitlab registry etcd config? By the description it looks pretty similar with username config.

[jopemachine]

Closed as #1917 stack was merged.