Nameserver not set correctly in resolv.conf

[jhoogstraat]

What happened:

For new containers in the cluster the dns setup does not work correctly.
Instead of the coredns service ip, a more "local network"-looking address is used.
The search is completly missing.
This prevents communication to other pods.

What you expected to happen:

Containers can communicate via dns with containers on other nodes and pods.

How to reproduce it (as minimally and precisely as possible):

Command used: kind create cluster --config=kind-cluster.yaml

with kind-cluster.yaml:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4

name: dev-cloud
networking:
  ipFamily: ipv4
nodes:
  - role: control-plane
    # ingress-controller uses nodeSelector "ingress-ready" to force its pod to this node.
    kubeadmConfigPatches:
      - |
        kind: InitConfiguration
        nodeRegistration:
          kubeletExtraArgs:
            node-labels: "ingress-ready=true"
    extraPortMappings:
      - containerPort: 30001
        hostPort: 80
      - containerPort: 30002
        hostPort: 8080
    extraMounts:
      - hostPath: data/
        containerPath: /data/
  - role: worker
    extraMounts:
      - hostPath: data/
        containerPath: /data/
  - role: worker
    extraMounts:
      - hostPath: data/
        containerPath: /data/

Anything else we need to know?:
kind-logs.zip - I started a debug container and printed the content of resolv.conf.

Environment:

• kind version: (use kind version): kind v0.25.0 go1.23.3 darwin/arm64
• Runtime info: (use docker info, podman info or nerdctl info):

Client:
 Version:    27.3.1
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.17.1-desktop.1
    Path:     /Users/XXXX/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.7-desktop.1
    Path:     /Users/XXXX/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.37
    Path:     /Users/XXXX/.docker/cli-plugins/docker-debug
  desktop: Docker Desktop commands (Alpha) (Docker Inc.)
    Version:  v0.0.15
    Path:     /Users/XXXX/.docker/cli-plugins/docker-desktop
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /Users/XXXX/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.27
    Path:     /Users/XXXX/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.5
    Path:     /Users/XXXX/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.3.0
    Path:     /Users/XXXX/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/XXXX/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.14.0
    Path:     /Users/XXXX/.docker/cli-plugins/docker-scout

Server:
 Containers: 4
  Running: 4
  Paused: 0
  Stopped: 0
 Images: 37
 Server Version: 27.3.1
 Storage Driver: overlayfs
  driver-type: io.containerd.snapshotter.v1
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 472731909fa34bd7bc9c087e4c27943f9835f111
 runc version: v1.1.13-0-g58aa920
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.10.11-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 4
 Total Memory: 9.705GiB
 Name: docker-desktop
 ID: 2e072ff7-0f10-4c9c-94a6-2863dadc00a4
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/XXXX/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  192.168.1.81:5000
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

• OS (e.g. from /etc/os-release): MBP with MacOS 15.1 (24B83) and Docker Desktop
• Kubernetes version: (use kubectl version):

Client Version: v1.30.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.31.2

• Any proxies or other special environment settings?:

[BenTheElder]

It seems unlikely that the config above is the minimum reproducer, which also needs some actual pods we can run and some more concrete ways that this deviates from any other kubernetes cluster.

For new containers in the cluster the dns setup does not work correctly.

containers => pods?

what pods? how are they configured?

this was working for old containers? what old containers? what is different with the old and the new?

Instead of the coredns service ip, a more "local network"-looking address is used.

are they by any chance host network pods?

The search is completly missing.

The in-cluster search settings are kubernetes, not kind.
The host search parameters are another story.

[BenTheElder]

/triage needs-information

[BenTheElder]

@jhoogstraat we need more information to follow up on this one, otherwise I'm going to have to close it.

This sounds like intended standard kubernetes behavior actually but I can't confirm without clarification.