[release-v3.30] Auto pick #10117: Add support for custom DNS in the operator

[caseydavenport]

Cherry pick of #10117 on release-v3.30.

#10117: Add support for custom DNS in the operator

Original PR Body below

Description

When deploying the operator with Windows and the KUBERNETES_SERVICE_HOST set (required by Windows) then the operator fails to resolve the DNS. This allows for passing in a custom DNS with will be appended to the pods DNS configuration allowing for communication to the API server for initial configuration.

The error you get is something like

2025/03/27 22:51:30 [INFO] Go OS/Arch: linux/amd64
W0327 22:51:50.070616       1 reflector.go:535] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:229: failed to list *v1.ConfigMap: Get "https://capz-fl5y4z-9a54ece6.australiaeast.cloudapp.azure.com:6443/api/v1/namespaces/calico-system/configmaps?fieldSelector=metadata.name%3Dactive-operator&limit=500&resourceVersion=0": dial tcp: lookup capz-fl5y4z-

with the operator template values:

    installation:
      cni:
        type: Calico
      calicoNetwork:
        bgp: Disabled
        windowsDataplane: HNS
        mtu: 1350
        ipPools:
        - cidr: <cidr>
          encapsulation: VXLAN{{end}}
      serviceCIDRs: 
        - 10.96.0.0/12 # must match cluster service CIDR (this is the default)
    kubernetesServiceEndpoint:
      host: "{{ .Cluster.spec.controlPlaneEndpoint.host }}"
      port: "{{ .Cluster.spec.controlPlaneEndpoint.port }}"

this allows to add something like this to the operator helm values file:

    # when kubernetesServiceEndpoint need to pass dns to look up the name properly
    dnsConfig: 
      nameservers:
        - 127.0.0.53
      options:
        - name: edns0
        - name: trust-ad

Related issues/PRs

fixes #9536

Enables automated deployments in CAPZ:
kubernetes-sigs/cluster-api-provider-azure#5523

Partly Addresses:
tigera/operator#3113

Todos

• Tests
• Documentation
• Release note

Release Note

Update Operator Helm values to support custom DNS settings

Reminder for the reviewer

Make sure that this PR has the correct labels and milestone set.

Every PR needs one docs-* label.

• docs-pr-required: This change requires a change to the documentation that has not been completed yet.
• docs-completed: This change has all necessary documentation completed.
• docs-not-required: This change has no user-facing impact and requires no docs.

Every PR needs one release-note-* label.

• release-note-required: This PR has user-facing changes. Most PRs should have this label.
• release-note-not-required: This PR has no user-facing changes.

Other optional labels:

• cherry-pick-candidate: This PR should be cherry-picked to an earlier release. For bug fixes only.
• needs-operator-pr: This PR is related to install and requires a corresponding change to the operator.