Buildx doesn't respect insecure registry configuration

[Re4zOon]

Contributing guidelines

• I've read the contributing guidelines and wholeheartedly agree

I've found a bug and checked that ...

• ... the documentation does not mention anything about my problem
• ... there are no open or closed issues that are related to my problem

Description

Hi,

Re-opening 2226, as the issue still exists.
See comments in the other ticket.
The attached logs/outputs are fresh.

Expected behaviour

Use port 80

Actual behaviour

Uses port 443

Buildx version

github.com/docker/buildx v0.17.1 257815a

Docker info

Client: Docker Engine - Community
 Version:    27.3.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.17.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.7
    Path:     /usr/libexec/docker/cli-plugins/docker-compose
WARNING: Plugin "/usr/libexec/docker/cli-plugins/docker-buildx-014.bak" is not valid: plugin candidate "buildx-014.bak" did not match "^[a-z][a-z0-9]*$"

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 29
 Server Version: 27.3.1
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 57f17b0a6295a39009d861b89e3b3b87b005ca27
 runc version: v1.1.14-0-g2c9f560
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.14.0-503.14.1.el9_5.x86_64
 Operating System: Red Hat Enterprise Linux 9.5 (Plow)
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 15.37GiB
 Name: hostname
 ID: 2864ada1-48b8-44ae-8094-dc18a3baed6c
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: username
 Experimental: false
 Insecure Registries:
  our-registry
  127.0.0.0/8
 Registry Mirrors:
  http://our-registry/
 Live Restore Enabled: false
 Default Address Pools:
   Base: 172.31.0.0/16, Size: 24

WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

Builders list

NAME/NODE         DRIVER/ENDPOINT                   STATUS     BUILDKIT   PLATFORMS
default           docker
 \_ default        \_ default                       running    v0.16.0    linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/amd64/v4, linux/386

Configuration

FROM our-registry/base-images/debian:latest

RUN echo "hello"

Build logs

[root@sgujenkins2 ~]# docker build .
[+] Building 0.3s (2/2) FINISHED                                                                         docker:default
 => [internal] load build definition from Dockerfile                                                               0.1s
 => => transferring dockerfile: 112B                                                                               0.0s
 => ERROR [internal] load metadata for our-registry/base-images/debian:latest                         0.0s
------
 > [internal] load metadata for our-registry/base-images/debian:latest:
------
Dockerfile:1
--------------------
   1 | >>> FROM our-registry/base-images/debian:latest
   2 |
   3 |     RUN echo "hello"
--------------------
ERROR: failed to solve: our-registry/base-images/debian:latest: failed to resolve source metadata for our-registry/base-images/debian:latest: failed to do request: Head "https://our-registry/v2/base-images/debian/manifests/latest": dial tcp 10.0.0.1:443: connect: no route to host

Additional info

❯ cat /etc/docker/daemon.json
{
"insecure-registries":["http://our-registry", "our-registry"],
"registry-mirrors": ["http://our-registry"]
}

[Kraust]

Have you tried removing the running docker buildx container docker buildx rm? That is what I believe solved the issue for me.

[Re4zOon]

Have you tried removing the running docker buildx container docker buildx rm? That is what I believe solved the issue for me.

Hi @Kraust,

As you see I only have one builder, which is the default (not even a container).
The weirdest thing is the "no route to host", which is incorrect, as with the legacy builder (on the same URL/IP) I can connect.

Anyways, this happen from all multiple of our systems, not just one (even from every DEV local WSL/Ubuntu).

[tonistiigi]

From the error it seems to be like network route error, not "insecure registry" config error. Insecure registries in dockerd mean both HTTP and HTTPS with insecure registry.

[Re4zOon]

Hi,

There is no 443 port open on the registry.
I never wanted docker to try 443 (https).
However it still does, which is not the same behaviour as legacy builder.
export DOCKER_BUILDKIT=0 "solves" the issue (as in it wont use buildx/this repo).

[uliss3s]

Same issue: #2712

Same temp solution: export DOCKER_BUILDKIT=0