Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(source): allow to register all pods and their associated PTR record #4782

Merged
merged 1 commit into from
Jan 28, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/annotations/annotations.md
Original file line number Diff line number Diff line change
@@ -64,6 +64,8 @@ Specifies the domain for the resource's DNS records.
Multiple hostnames can be specified through a comma-separated list, e.g.
`svc.mydomain1.com,svc.mydomain2.com`.

For `Pods`, uses the `Pod`'s `Status.PodIP`, unless they are `hostNetwork: true` in which case the NodeExternalIP is used for IPv4 and NodeInternalIP for IPv6.

## external-dns.alpha.kubernetes.io/ingress-hostname-source

Specifies where to get the domain for an `Ingress` resource.
@@ -80,7 +82,7 @@ Specifies the domain for the resource's DNS records that are for use from intern

For `Services` of type `LoadBalancer`, uses the `Service`'s `ClusterIP`.

For `Pods`, uses the `Pod`'s `Status.PodIP`.
For `Pods`, uses the `Pod`'s `Status.PodIP`, unless they are `hostNetwork: true` in which case the NodeExternalIP is used for IPv4 and NodeInternalIP for IPv6.

## external-dns.alpha.kubernetes.io/target

2 changes: 2 additions & 0 deletions docs/flags.md
Original file line number Diff line number Diff line change
@@ -24,11 +24,13 @@
| `--fqdn-template=""` | A templated string that's used to generate DNS names from sources that don't define a hostname themselves, or to add a hostname suffix when paired with the fake source (optional). Accepts comma separated list for multiple global FQDN. |
| `--[no-]combine-fqdn-annotation` | Combine FQDN template and Annotations instead of overwriting |
| `--[no-]ignore-hostname-annotation` | Ignore hostname annotation when generating DNS names, valid only when --fqdn-template is set (default: false) |
| `--[no-]ignore-non-host-network-pods` | Ignore pods not running on host network when using pod source (default: true) |
| `--[no-]ignore-ingress-tls-spec` | Ignore the spec.tls section in Ingress resources (default: false) |
| `--gateway-namespace=GATEWAY-NAMESPACE` | Limit Gateways of Route endpoints to a specific namespace (default: all namespaces) |
| `--gateway-label-filter=GATEWAY-LABEL-FILTER` | Filter Gateways of Route endpoints via label selector (default: all gateways) |
| `--compatibility=` | Process annotation semantics from legacy implementations (optional, options: mate, molecule, kops-dns-controller) |
| `--[no-]ignore-ingress-rules-spec` | Ignore the spec.rules section in Ingress resources (default: false) |
| `--pod-source-domain=""` | Domain to use for pods records (optional) |
| `--[no-]publish-internal-services` | Allow external-dns to publish DNS records for ClusterIP services (optional) |
| `--[no-]publish-host-ip` | Allow external-dns to publish host-ip for headless services (optional) |
| `--[no-]always-publish-not-ready-addresses` | Always publish also not ready addresses for headless services (optional) |
25 changes: 25 additions & 0 deletions docs/sources/pod.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Pod Source

The pod source creates DNS entries based on `Pod` resources.

## Pods not running with host networking

By default, the pod source will not consider the pods that aren't running with host networking enabled. You can override this behavior by using the `--ignore-non-host-network-pods` option.

## Using a default domain for pods

By default, the pod source will look into the pod annotations to find the FQDN associated with a pod. You can also use the option `--pod-source-domain=example.org` to build the FQDN of the pods. The pod named "test-pod" will then be registered as "test-pod.example.org".

## Configuration for registering all pods with their associated PTR record

A use case where combining these options can be pertinent is when you are running on-premise Kubernetes clusters without SNAT enabled for the pod network. You might want to register all the pods in the DNS with their associated PTR record so that the source of some traffic outside of the cluster can be rapidly associated with a workload using the "nslookup" or "dig" command on the pod IP. This can be particularly useful if you are running a large number of Kubernetes clusters.

You will then use the following mix of options:
- `--domain-filter=example.org`
- `--domain-filter=10.0.0.in-addr.arpa`
- `--source=pod`
- `--pod-source-domain=example.org`
- `--no-ignore-non-host-network-pods`
- `--rfc2136-create-ptr`
- `--rfc2136-zone=example.org`
- `--rfc2136-zone=10.0.0.in-addr.arpa`
6 changes: 2 additions & 4 deletions docs/tutorials/kops-dns-controller.md
Original file line number Diff line number Diff line change
@@ -21,11 +21,9 @@ The DNS record mappings try to "do the right thing", but what this means is diff

### Pods

For the external annotation, ExternalDNS will map a HostNetwork=true Pod to the external IPs of the Node.
For the external annotation, ExternalDNS will map a Pod to the external IPs of the Node.

For the internal annotation, ExternalDNS will map a HostNetwork=true Pod to the internal IPs of the Node.

ExternalDNS ignore Pods that are not HostNetwork=true
For the internal annotation, ExternalDNS will map a Pod to the internal IPs of the Node.

Annotations added to Pods will always result in an A record being created.

2 changes: 2 additions & 0 deletions main.go
Original file line number Diff line number Diff line change
@@ -124,11 +124,13 @@ func main() {
FQDNTemplate: cfg.FQDNTemplate,
CombineFQDNAndAnnotation: cfg.CombineFQDNAndAnnotation,
IgnoreHostnameAnnotation: cfg.IgnoreHostnameAnnotation,
IgnoreNonHostNetworkPods: cfg.IgnoreNonHostNetworkPods,
IgnoreIngressTLSSpec: cfg.IgnoreIngressTLSSpec,
IgnoreIngressRulesSpec: cfg.IgnoreIngressRulesSpec,
GatewayNamespace: cfg.GatewayNamespace,
GatewayLabelFilter: cfg.GatewayLabelFilter,
Compatibility: cfg.Compatibility,
PodSourceDomain: cfg.PodSourceDomain,
PublishInternal: cfg.PublishInternal,
PublishHostIP: cfg.PublishHostIP,
AlwaysPublishNotReadyAddresses: cfg.AlwaysPublishNotReadyAddresses,
5 changes: 5 additions & 0 deletions pkg/apis/externaldns/types.go
Original file line number Diff line number Diff line change
@@ -57,11 +57,13 @@ type Config struct {
FQDNTemplate string
CombineFQDNAndAnnotation bool
IgnoreHostnameAnnotation bool
IgnoreNonHostNetworkPods bool
IgnoreIngressTLSSpec bool
IgnoreIngressRulesSpec bool
GatewayNamespace string
GatewayLabelFilter string
Compatibility string
PodSourceDomain string
PublishInternal bool
PublishHostIP bool
AlwaysPublishNotReadyAddresses bool
@@ -286,6 +288,7 @@ var defaultConfig = &Config{
PDNSServerID: "localhost",
PDNSAPIKey: "",
PDNSSkipTLSVerify: false,
PodSourceDomain: "",
TLSCA: "",
TLSClientCert: "",
TLSClientCertKey: "",
@@ -442,11 +445,13 @@ func App(cfg *Config) *kingpin.Application {
app.Flag("fqdn-template", "A templated string that's used to generate DNS names from sources that don't define a hostname themselves, or to add a hostname suffix when paired with the fake source (optional). Accepts comma separated list for multiple global FQDN.").Default(defaultConfig.FQDNTemplate).StringVar(&cfg.FQDNTemplate)
app.Flag("combine-fqdn-annotation", "Combine FQDN template and Annotations instead of overwriting").BoolVar(&cfg.CombineFQDNAndAnnotation)
app.Flag("ignore-hostname-annotation", "Ignore hostname annotation when generating DNS names, valid only when --fqdn-template is set (default: false)").BoolVar(&cfg.IgnoreHostnameAnnotation)
app.Flag("ignore-non-host-network-pods", "Ignore pods not running on host network when using pod source (default: true)").BoolVar(&cfg.IgnoreNonHostNetworkPods)
app.Flag("ignore-ingress-tls-spec", "Ignore the spec.tls section in Ingress resources (default: false)").BoolVar(&cfg.IgnoreIngressTLSSpec)
app.Flag("gateway-namespace", "Limit Gateways of Route endpoints to a specific namespace (default: all namespaces)").StringVar(&cfg.GatewayNamespace)
app.Flag("gateway-label-filter", "Filter Gateways of Route endpoints via label selector (default: all gateways)").StringVar(&cfg.GatewayLabelFilter)
app.Flag("compatibility", "Process annotation semantics from legacy implementations (optional, options: mate, molecule, kops-dns-controller)").Default(defaultConfig.Compatibility).EnumVar(&cfg.Compatibility, "", "mate", "molecule", "kops-dns-controller")
app.Flag("ignore-ingress-rules-spec", "Ignore the spec.rules section in Ingress resources (default: false)").BoolVar(&cfg.IgnoreIngressRulesSpec)
app.Flag("pod-source-domain", "Domain to use for pods records (optional)").Default(defaultConfig.PodSourceDomain).StringVar(&cfg.PodSourceDomain)
app.Flag("publish-internal-services", "Allow external-dns to publish DNS records for ClusterIP services (optional)").BoolVar(&cfg.PublishInternal)
app.Flag("publish-host-ip", "Allow external-dns to publish host-ip for headless services (optional)").BoolVar(&cfg.PublishHostIP)
app.Flag("always-publish-not-ready-addresses", "Always publish also not ready addresses for headless services (optional)").BoolVar(&cfg.AlwaysPublishNotReadyAddresses)
6 changes: 6 additions & 0 deletions pkg/apis/externaldns/types_test.go
Original file line number Diff line number Diff line change
@@ -136,6 +136,7 @@ var (
Sources: []string{"service", "ingress", "connector"},
Namespace: "namespace",
IgnoreHostnameAnnotation: true,
IgnoreNonHostNetworkPods: false,
IgnoreIngressTLSSpec: true,
IgnoreIngressRulesSpec: true,
FQDNTemplate: "{{.Name}}.service.example.com",
@@ -197,6 +198,7 @@ var (
TLSCA: "/path/to/ca.crt",
TLSClientCert: "/path/to/cert.pem",
TLSClientCertKey: "/path/to/key.pem",
PodSourceDomain: "example.org",
Policy: "upsert-only",
Registry: "noop",
TXTOwnerID: "owner-1",
@@ -265,6 +267,7 @@ func TestParseFlags(t *testing.T) {
"--source=connector",
"--namespace=namespace",
"--fqdn-template={{.Name}}.service.example.com",
"--no-ignore-non-host-network-pods",
"--ignore-hostname-annotation",
"--ignore-ingress-tls-spec",
"--ignore-ingress-rules-spec",
@@ -301,6 +304,7 @@ func TestParseFlags(t *testing.T) {
"--tls-ca=/path/to/ca.crt",
"--tls-client-cert=/path/to/cert.pem",
"--tls-client-cert-key=/path/to/key.pem",
"--pod-source-domain=example.org",
"--domain-filter=example.org",
"--domain-filter=company.com",
"--exclude-domains=xapi.example.org",
@@ -385,6 +389,7 @@ func TestParseFlags(t *testing.T) {
"EXTERNAL_DNS_SOURCE": "service\ningress\nconnector",
"EXTERNAL_DNS_NAMESPACE": "namespace",
"EXTERNAL_DNS_FQDN_TEMPLATE": "{{.Name}}.service.example.com",
"EXTERNAL_DNS_IGNORE_NON_HOST_NETWORK_PODS": "0",
"EXTERNAL_DNS_IGNORE_HOSTNAME_ANNOTATION": "1",
"EXTERNAL_DNS_IGNORE_INGRESS_TLS_SPEC": "1",
"EXTERNAL_DNS_IGNORE_INGRESS_RULES_SPEC": "1",
@@ -413,6 +418,7 @@ func TestParseFlags(t *testing.T) {
"EXTERNAL_DNS_INMEMORY_ZONE": "example.org\ncompany.com",
"EXTERNAL_DNS_OVH_ENDPOINT": "ovh-ca",
"EXTERNAL_DNS_OVH_API_RATE_LIMIT": "42",
"EXTERNAL_DNS_POD_SOURCE_DOMAIN": "example.org",
"EXTERNAL_DNS_DOMAIN_FILTER": "example.org\ncompany.com",
"EXTERNAL_DNS_EXCLUDE_DOMAINS": "xapi.example.org\nxapi.company.com",
"EXTERNAL_DNS_REGEX_DOMAIN_FILTER": "(example\\.org|company\\.com)$",
38 changes: 26 additions & 12 deletions source/pod.go
Original file line number Diff line number Diff line change
@@ -31,15 +31,17 @@ import (
)

type podSource struct {
client kubernetes.Interface
namespace string
podInformer coreinformers.PodInformer
nodeInformer coreinformers.NodeInformer
compatibility string
client kubernetes.Interface
namespace string
podInformer coreinformers.PodInformer
nodeInformer coreinformers.NodeInformer
compatibility string
ignoreNonHostNetworkPods bool
podSourceDomain string
}

// NewPodSource creates a new podSource with the given config.
func NewPodSource(ctx context.Context, kubeClient kubernetes.Interface, namespace string, compatibility string) (Source, error) {
func NewPodSource(ctx context.Context, kubeClient kubernetes.Interface, namespace string, compatibility string, ignoreNonHostNetworkPods bool, podSourceDomain string) (Source, error) {
informerFactory := kubeinformers.NewSharedInformerFactoryWithOptions(kubeClient, 0, kubeinformers.WithNamespace(namespace))
podInformer := informerFactory.Core().V1().Pods()
nodeInformer := informerFactory.Core().V1().Nodes()
@@ -65,11 +67,13 @@ func NewPodSource(ctx context.Context, kubeClient kubernetes.Interface, namespac
}

return &podSource{
client: kubeClient,
podInformer: podInformer,
nodeInformer: nodeInformer,
namespace: namespace,
compatibility: compatibility,
client: kubeClient,
podInformer: podInformer,
nodeInformer: nodeInformer,
namespace: namespace,
compatibility: compatibility,
ignoreNonHostNetworkPods: ignoreNonHostNetworkPods,
podSourceDomain: podSourceDomain,
}, nil
}

@@ -84,7 +88,7 @@ func (ps *podSource) Endpoints(ctx context.Context) ([]*endpoint.Endpoint, error

endpointMap := make(map[endpoint.EndpointKey][]string)
for _, pod := range pods {
if !pod.Spec.HostNetwork {
if ps.ignoreNonHostNetworkPods && !pod.Spec.HostNetwork {
log.Debugf("skipping pod %s. hostNetwork=false", pod.Name)
continue
}
@@ -146,6 +150,16 @@ func (ps *podSource) Endpoints(ctx context.Context) ([]*endpoint.Endpoint, error
}
}
}

if ps.podSourceDomain != "" {
domain := pod.ObjectMeta.Name + "." + ps.podSourceDomain
if len(targets) == 0 {
addToEndpointMap(endpointMap, domain, suitableType(pod.Status.PodIP), pod.Status.PodIP)
}
for _, target := range targets {
addToEndpointMap(endpointMap, domain, suitableType(target), target)
}
}
}
endpoints := []*endpoint.Endpoint{}
for key, targets := range endpointMap {
102 changes: 94 additions & 8 deletions source/pod_test.go
Original file line number Diff line number Diff line change
@@ -32,18 +32,22 @@ func TestPodSource(t *testing.T) {
t.Parallel()

for _, tc := range []struct {
title string
targetNamespace string
compatibility string
expected []*endpoint.Endpoint
expectError bool
nodes []*corev1.Node
pods []*corev1.Pod
title string
targetNamespace string
compatibility string
ignoreNonHostNetworkPods bool
PodSourceDomain string
expected []*endpoint.Endpoint
expectError bool
nodes []*corev1.Node
pods []*corev1.Pod
}{
{
"create IPv4 records based on pod's external and internal IPs",
"",
"",
true,
"",
[]*endpoint.Endpoint{
{DNSName: "a.foo.example.org", Targets: endpoint.Targets{"54.10.11.1", "54.10.11.2"}, RecordType: endpoint.RecordTypeA},
{DNSName: "internal.a.foo.example.org", Targets: endpoint.Targets{"10.0.1.1", "10.0.1.2"}, RecordType: endpoint.RecordTypeA},
@@ -114,6 +118,8 @@ func TestPodSource(t *testing.T) {
"create IPv4 records based on pod's external and internal IPs using DNS Controller annotations",
"",
"kops-dns-controller",
true,
"",
[]*endpoint.Endpoint{
{DNSName: "a.foo.example.org", Targets: endpoint.Targets{"54.10.11.1", "54.10.11.2"}, RecordType: endpoint.RecordTypeA},
{DNSName: "internal.a.foo.example.org", Targets: endpoint.Targets{"10.0.1.1", "10.0.1.2"}, RecordType: endpoint.RecordTypeA},
@@ -184,6 +190,8 @@ func TestPodSource(t *testing.T) {
"create IPv6 records based on pod's external and internal IPs",
"",
"",
true,
"",
[]*endpoint.Endpoint{
{DNSName: "a.foo.example.org", Targets: endpoint.Targets{"2001:DB8::1", "2001:DB8::2"}, RecordType: endpoint.RecordTypeAAAA},
{DNSName: "internal.a.foo.example.org", Targets: endpoint.Targets{"2001:DB8::1", "2001:DB8::2"}, RecordType: endpoint.RecordTypeAAAA},
@@ -252,6 +260,8 @@ func TestPodSource(t *testing.T) {
"create IPv6 records based on pod's external and internal IPs using DNS Controller annotations",
"",
"kops-dns-controller",
true,
"",
[]*endpoint.Endpoint{
{DNSName: "a.foo.example.org", Targets: endpoint.Targets{"2001:DB8::1", "2001:DB8::2"}, RecordType: endpoint.RecordTypeAAAA},
{DNSName: "internal.a.foo.example.org", Targets: endpoint.Targets{"2001:DB8::1", "2001:DB8::2"}, RecordType: endpoint.RecordTypeAAAA},
@@ -320,6 +330,8 @@ func TestPodSource(t *testing.T) {
"create records based on pod's target annotation",
"",
"",
true,
"",
[]*endpoint.Endpoint{
{DNSName: "a.foo.example.org", Targets: endpoint.Targets{"208.1.2.1", "208.1.2.2"}, RecordType: endpoint.RecordTypeA},
{DNSName: "internal.a.foo.example.org", Targets: endpoint.Targets{"208.1.2.1", "208.1.2.2"}, RecordType: endpoint.RecordTypeA},
@@ -392,6 +404,8 @@ func TestPodSource(t *testing.T) {
"create multiple records",
"",
"",
true,
"",
[]*endpoint.Endpoint{
{DNSName: "a.foo.example.org", Targets: endpoint.Targets{"54.10.11.1"}, RecordType: endpoint.RecordTypeA},
{DNSName: "a.foo.example.org", Targets: endpoint.Targets{"2001:DB8::1"}, RecordType: endpoint.RecordTypeAAAA},
@@ -462,6 +476,8 @@ func TestPodSource(t *testing.T) {
"pods with hostNetwore=false should be ignored",
"",
"",
true,
"",
[]*endpoint.Endpoint{
{DNSName: "a.foo.example.org", Targets: endpoint.Targets{"54.10.11.1"}, RecordType: endpoint.RecordTypeA},
{DNSName: "internal.a.foo.example.org", Targets: endpoint.Targets{"10.0.1.1"}, RecordType: endpoint.RecordTypeA},
@@ -532,6 +548,8 @@ func TestPodSource(t *testing.T) {
"only watch a given namespace",
"kube-system",
"",
true,
"",
[]*endpoint.Endpoint{
{DNSName: "a.foo.example.org", Targets: endpoint.Targets{"54.10.11.1"}, RecordType: endpoint.RecordTypeA},
{DNSName: "internal.a.foo.example.org", Targets: endpoint.Targets{"10.0.1.1"}, RecordType: endpoint.RecordTypeA},
@@ -602,6 +620,8 @@ func TestPodSource(t *testing.T) {
"split record for internal hostname annotation",
"",
"",
true,
"",
[]*endpoint.Endpoint{
{DNSName: "internal.a.foo.example.org", Targets: endpoint.Targets{"10.0.1.1"}, RecordType: endpoint.RecordTypeA},
{DNSName: "internal.b.foo.example.org", Targets: endpoint.Targets{"10.0.1.1"}, RecordType: endpoint.RecordTypeA},
@@ -638,6 +658,72 @@ func TestPodSource(t *testing.T) {
},
},
},
{
"create IPv4 records for non-host network pods",
"",
"",
false,
"example.org",
[]*endpoint.Endpoint{
{DNSName: "my-pod1.example.org", Targets: endpoint.Targets{"192.168.1.1"}, RecordType: endpoint.RecordTypeA},
{DNSName: "my-pod2.example.org", Targets: endpoint.Targets{"192.168.1.2"}, RecordType: endpoint.RecordTypeA},
},
false,
[]*corev1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "my-node1",
},
Status: corev1.NodeStatus{
Addresses: []corev1.NodeAddress{
{Type: corev1.NodeExternalIP, Address: "54.10.11.1"},
{Type: corev1.NodeInternalIP, Address: "10.0.1.1"},
},
},
},
{
ObjectMeta: metav1.ObjectMeta{
Name: "my-node2",
},
Status: corev1.NodeStatus{
Addresses: []corev1.NodeAddress{
{Type: corev1.NodeExternalIP, Address: "54.10.11.2"},
{Type: corev1.NodeInternalIP, Address: "10.0.1.2"},
},
},
},
},
[]*corev1.Pod{
{
ObjectMeta: metav1.ObjectMeta{
Name: "my-pod1",
Namespace: "kube-system",
Annotations: map[string]string{},
},
Spec: corev1.PodSpec{
HostNetwork: false,
NodeName: "my-node1",
},
Status: corev1.PodStatus{
PodIP: "192.168.1.1",
},
},
{
ObjectMeta: metav1.ObjectMeta{
Name: "my-pod2",
Namespace: "kube-system",
Annotations: map[string]string{},
},
Spec: corev1.PodSpec{
HostNetwork: false,
NodeName: "my-node2",
},
Status: corev1.PodStatus{
PodIP: "192.168.1.2",
},
},
},
},
} {
tc := tc
t.Run(tc.title, func(t *testing.T) {
@@ -662,7 +748,7 @@ func TestPodSource(t *testing.T) {
}
}

client, err := NewPodSource(context.TODO(), kubernetes, tc.targetNamespace, tc.compatibility)
client, err := NewPodSource(context.TODO(), kubernetes, tc.targetNamespace, tc.compatibility, tc.ignoreNonHostNetworkPods, tc.PodSourceDomain)
require.NoError(t, err)

endpoints, err := client.Endpoints(ctx)
4 changes: 3 additions & 1 deletion source/store.go
Original file line number Diff line number Diff line change
@@ -50,11 +50,13 @@ type Config struct {
FQDNTemplate string
CombineFQDNAndAnnotation bool
IgnoreHostnameAnnotation bool
IgnoreNonHostNetworkPods bool
IgnoreIngressTLSSpec bool
IgnoreIngressRulesSpec bool
GatewayNamespace string
GatewayLabelFilter string
Compatibility string
PodSourceDomain string
PublishInternal bool
PublishHostIP bool
AlwaysPublishNotReadyAddresses bool
@@ -230,7 +232,7 @@ func BuildWithConfig(ctx context.Context, source string, p ClientGenerator, cfg
if err != nil {
return nil, err
}
return NewPodSource(ctx, client, cfg.Namespace, cfg.Compatibility)
return NewPodSource(ctx, client, cfg.Namespace, cfg.Compatibility, cfg.IgnoreNonHostNetworkPods, cfg.PodSourceDomain)
case "gateway-httproute":
return NewGatewayHTTPRouteSource(p, cfg)
case "gateway-grpcroute":