Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proxy Docker calls to private registry #250

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions e2e/e2e.sh
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,6 @@ if [[ $failed -gt 0 ]]; then
echo "FAILED $failed tests"
exit 1
fi
if [ "$(docker container inspect -f '{{.State.Running}}' docker_registry_proxy)" = "true" ]; then
echo "Docker registry cache hits: $(docker logs docker_registry_proxy | grep '"upstream_cache_status":"HIT"' | wc -l)"
fi
12 changes: 12 additions & 0 deletions e2e/lib/capi.sh
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,18 @@ function capi_cluster_ready {
k8s_wait_ready "machineset" "$machineset"
done

# Use Docker local registry to reduce external traffic
if [ "$(docker container inspect -f '{{.State.Running}}' docker_registry_proxy)" = "true" ]; then
_kubeconfig=$(k8s_get_capi_kubeconfig "$cluster")
pids=""
for node in $(kubectl get nodes -l node-role.kubernetes.io/control-plane!= -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{end}' --kubeconfig "$_kubeconfig"); do
docker exec "$node" sh -c "curl http://$(hostname -i):3128/setup/systemd | sed s/docker\.service/containerd\.service/g \
| sed '/Environment/ s/$/ \"NO_PROXY=127.0.0.0\/8,10.0.0.0\/8,172.16.0.0\/12,192.168.0.0\/16\"/' | bash" &
pids="$pids $!"
done
wait $pids
fi

# Wait for package variants
for pv in cluster configsync kindnet local-path-provisioner multus repo rootsync vlanindex; do
k8s_wait_exists "packagevariants" "${cluster}-$pv"
Expand Down
3 changes: 3 additions & 0 deletions e2e/provision/playbooks/roles/bootstrap/defaults/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,9 @@ k8s:
kind:
enabled: true

registry:
enabled: true

nephio_catalog_repo_uri: https://github.com/nephio-project/catalog.git
nephio_catalog_revision: v2.0.0

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
---
# SPDX-license-identifier: Apache-2.0
##############################################################################
# Copyright (c) 2024 The Nephio Authors.
# All rights reserved. This program and the accompanying materials
# are made available under the terms of the Apache License, Version 2.0
# which accompanies this distribution, and is available at
# http://www.apache.org/licenses/LICENSE-2.0
##############################################################################

- name: Create a local docker registry
become: true
community.docker.docker_container:
name: docker_registry_proxy
image: rpardini/docker-registry-proxy:0.6.4
hostname: docker-registry-proxy
detach: true
restart: true
restart_policy: "always"
networks:
- name: kind
ports:
- 0.0.0.0:3128:3128
volumes:
- /tmp/docker_mirror_cache:/docker_mirror_cache
- /tmp/docker_mirror_certs:/ca
env:
ENABLE_MANIFEST_CACHE: "true"
REGISTRIES: "k8s.gcr.io gcr.io quay.io"

- name: Waits for local docker registry readiness
ansible.builtin.uri:
url: http://127.0.0.1:3128
until: _result.status == 200
retries: 600
delay: 1
register: _result

- name: Disable IPv6 DNS resolvers
become: true
community.docker.docker_container_exec:
container: docker_registry_proxy
argv:
- /bin/bash
- -c
- 'echo "resolver 8.8.8.8 4.2.2.2 ipv6=off;" > /etc/nginx/resolvers.conf && /usr/sbin/nginx -s reload'
4 changes: 4 additions & 0 deletions e2e/provision/playbooks/roles/bootstrap/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@
ansible.builtin.include_tasks: create-mgmt.yml
when: kind.enabled

- name: Create Local Docker Registry
ansible.builtin.include_tasks: create-local_registry.yml
when: registry.enabled

- name: Apply kpt packages
ansible.builtin.include_tasks: apply-pkgs.yml

Expand Down