Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Topology-aware hinting docs #1920

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
47 changes: 47 additions & 0 deletions linkerd.io/content/2.12/features/topology-aware-routing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---
title: Topology Aware Routing
description: |-
Linkerd's implementation of Kubernetes topology aware routing enables endpoint consumption based on a node's zone label.
---

Kubernetes clusters are increasingly deployed in multi-zone environments with
network traffic often relying on obscure endpoint matching and routing for a
given service. Linkerd's implementation of Kubernetes ([Topology Aware
Routing][topology aware routing]) provides users with a mechanism for
asserting more control over endpoint selection and routing within a cluster
that spans multiple zones. Users can now implement routing constraints and
prefer endpoints in a specific zone in order to limit cross-zone networking
costs or improve performance through lowered cross-zone latency and bandwidth
constraints.

The goal of topology aware routing is to to provide a simpler way for users to
prefer endpoints by basing decisions solely off the node's
`topology.kubernetes.io/zone` label. If a client is in `zone-a`, then it
should prefer endpoints marked for use by clients in `zone-a`. When the
feature is enabled and the label set, Linkerd's destination controller will
attempt to find endpoints whose `routing.ForZones` field matches the client's
zone.

To get started with topology aware routing take a look at the [enabling
topology aware routing](../../tasks/enabling-topology-aware-routing/) task
documentation.

[topology aware routing]:
https://kubernetes.io/docs/concepts/services-networking/topology-aware-routing/

{{< note >}}

Topology aware routing requires that the cluster have the `TopologyAwareHints`
feature gate enabled (which is the default starting with Kubernetes 1.24), and
that Linkerd's `endpointSlice` feature be turned on (this is the default
starting with Linkerd stable-2.12).

{{< /note >}}

{{< note >}}

Starting in Kubernetes 1.31, Kubernetes also has the `trafficDistribution`
feature available as an alternative to topology aware routing.
`trafficDistribution` is not yet supported by Linkerd.

{{< /note >}}
77 changes: 77 additions & 0 deletions linkerd.io/content/2.12/tasks/enabling-topology-aware-routing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
---
title: "Enabling Topology Aware Routing"
description: |-
Enable topology aware routing to allow Linkerd to intelligently choose same-zone endpoints
---

[Topology aware routing](../../features/topology-aware-routing/) allows
Linkerd users to specify endpoint selection boundaries when a request is made
against a service, making it possible to limit cross-zone endpoint selection
and lower the associated costs and latency.

There are three requirements to successfully enabling topology aware routing
with Linkerd:

1. The `TopologyAwareHints` feature gate MUST be enabled on the Kubernetes
cluster. (This feature gate is enabled by default in Kubernetes 1.24 and
later.)

2. Each Kubernetes node needs to be assigned to a zone, using
`"topology.kubernetes.io/zone` label.

3. Relevant Kubernetes services will need to be modified with the
`service.kubernetes.io/topology-aware-hints=auto` annotation.

When Linkerd receives a set of endpoint slices and translates them to an
address set, a copy of the `Hints.forZones` field is made from each endpoint
slice if one is present. This field is only present if it's set by the
endpoint slice controller, and there are several
[safeguards][topology-aware-routing-safeguards] that Kubernetes checks before
setting it. To have the endpoint slice controller set the `forZones` field
users will have to add the `service.kubernetes.io/topology-aware-hints=auto`
annotation to the service.

After the endpoints have been translated into an address set and the endpoint
translator's available endpoints have been updated, filtering is applied to
ensure only the relevant sets of addresses are returned when a request is
made.

For each potential address in the available endpoint set Linkerd returns a set
of addresses whose consumption zone (zones in `forZones` field) matches that
of the node's zone. By doing so Linkerd ensures communication is limited to
endpoints that have been labeled by the endpoint slice controller for the same
node the client is on and limits cross-node and cross-zone communication.

## Constraints

Kubernetes places some [constraints][topology-aware-routing-constraints] on
topology aware routing that you should review before trying to enable topology
aware routing on Linkerd.

The main things to be aware of are:

- Topology aware routing assumes that traffic to a given zone will be roughly
proportional to the capacity of the nodes in the zone. This can be a concern
when using horizontal pod autoscaling, as HPA may start nodes in the "wrong"
zone.

- Services with `internalTrafficPolicy` set to `Local` ignore topology aware
routing, by design.

- If you have workloads running on Nodes labeled with `control-plane` or
`master` role, topology aware routing will not route to endpoints on these
Nodes.

## Configuring Topology Aware Routing

Successful topology aware routing can be confirmed by looking at the Linkerd
proxy logs for the relevant service. The logs should show a stream of messages
similar to the ones below:

```text
time="2021-08-27T14:04:35Z" level=info msg="Establishing watch on endpoint [default/nginx-deploy-svc:80]" addr=":8086" component=endpoints-watcher
time="2021-08-27T14:04:35Z" level=debug msg="Filtering through addresses that should be consumed by zone zone-b" addr=":8086" component=endpoint-translator remote="127.0.0.1:49846" service="nginx-deploy-svc.default.svc.cluster.local:80"
```

[topology-aware-routing-safeguards]: https://kubernetes.io/docs/concepts/services-networking/topology-aware-hints/#safeguards
[topology-aware-routing-constraints]: https://kubernetes.io/docs/concepts/services-networking/topology-aware-hints/#constraints
47 changes: 47 additions & 0 deletions linkerd.io/content/2.13/features/topology-aware-routing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---
title: Topology Aware Routing
description: |-
Linkerd's implementation of Kubernetes topology aware routing enables endpoint consumption based on a node's zone label.
---

Kubernetes clusters are increasingly deployed in multi-zone environments with
network traffic often relying on obscure endpoint matching and routing for a
given service. Linkerd's implementation of Kubernetes ([Topology Aware
Routing][topology aware routing]) provides users with a mechanism for
asserting more control over endpoint selection and routing within a cluster
that spans multiple zones. Users can now implement routing constraints and
prefer endpoints in a specific zone in order to limit cross-zone networking
costs or improve performance through lowered cross-zone latency and bandwidth
constraints.

The goal of topology aware routing is to to provide a simpler way for users to
prefer endpoints by basing decisions solely off the node's
`topology.kubernetes.io/zone` label. If a client is in `zone-a`, then it
should prefer endpoints marked for use by clients in `zone-a`. When the
feature is enabled and the label set, Linkerd's destination controller will
attempt to find endpoints whose `routing.ForZones` field matches the client's
zone.

To get started with topology aware routing take a look at the [enabling
topology aware routing](../../tasks/enabling-topology-aware-routing/) task
documentation.

[topology aware routing]:
https://kubernetes.io/docs/concepts/services-networking/topology-aware-routing/

{{< note >}}

Topology aware routing requires that the cluster have the `TopologyAwareHints`
feature gate enabled (which is the default starting with Kubernetes 1.24), and
that Linkerd's `endpointSlice` feature be turned on (this is the default
starting with Linkerd stable-2.12).

{{< /note >}}

{{< note >}}

Starting in Kubernetes 1.31, Kubernetes also has the `trafficDistribution`
feature available as an alternative to topology aware routing.
`trafficDistribution` is not yet supported by Linkerd.

{{< /note >}}
77 changes: 77 additions & 0 deletions linkerd.io/content/2.13/tasks/enabling-topology-aware-routing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
---
title: "Enabling Topology Aware Routing"
description: |-
Enable topology aware routing to allow Linkerd to intelligently choose same-zone endpoints
---

[Topology aware routing](../../features/topology-aware-routing/) allows
Linkerd users to specify endpoint selection boundaries when a request is made
against a service, making it possible to limit cross-zone endpoint selection
and lower the associated costs and latency.

There are three requirements to successfully enabling topology aware routing
with Linkerd:

1. The `TopologyAwareHints` feature gate MUST be enabled on the Kubernetes
cluster. (This feature gate is enabled by default in Kubernetes 1.24 and
later.)

2. Each Kubernetes node needs to be assigned to a zone, using
`"topology.kubernetes.io/zone` label.

3. Relevant Kubernetes services will need to be modified with the
`service.kubernetes.io/topology-aware-hints=auto` annotation.

When Linkerd receives a set of endpoint slices and translates them to an
address set, a copy of the `Hints.forZones` field is made from each endpoint
slice if one is present. This field is only present if it's set by the
endpoint slice controller, and there are several
[safeguards][topology-aware-routing-safeguards] that Kubernetes checks before
setting it. To have the endpoint slice controller set the `forZones` field
users will have to add the `service.kubernetes.io/topology-aware-hints=auto`
annotation to the service.

After the endpoints have been translated into an address set and the endpoint
translator's available endpoints have been updated, filtering is applied to
ensure only the relevant sets of addresses are returned when a request is
made.

For each potential address in the available endpoint set Linkerd returns a set
of addresses whose consumption zone (zones in `forZones` field) matches that
of the node's zone. By doing so Linkerd ensures communication is limited to
endpoints that have been labeled by the endpoint slice controller for the same
node the client is on and limits cross-node and cross-zone communication.

## Constraints

Kubernetes places some [constraints][topology-aware-routing-constraints] on
topology aware routing that you should review before trying to enable topology
aware routing on Linkerd.

The main things to be aware of are:

- Topology aware routing assumes that traffic to a given zone will be roughly
proportional to the capacity of the nodes in the zone. This can be a concern
when using horizontal pod autoscaling, as HPA may start nodes in the "wrong"
zone.

- Services with `internalTrafficPolicy` set to `Local` ignore topology aware
routing, by design.

- If you have workloads running on Nodes labeled with `control-plane` or
`master` role, topology aware routing will not route to endpoints on these
Nodes.

## Configuring Topology Aware Routing

Successful topology aware routing can be confirmed by looking at the Linkerd
proxy logs for the relevant service. The logs should show a stream of messages
similar to the ones below:

```text
time="2021-08-27T14:04:35Z" level=info msg="Establishing watch on endpoint [default/nginx-deploy-svc:80]" addr=":8086" component=endpoints-watcher
time="2021-08-27T14:04:35Z" level=debug msg="Filtering through addresses that should be consumed by zone zone-b" addr=":8086" component=endpoint-translator remote="127.0.0.1:49846" service="nginx-deploy-svc.default.svc.cluster.local:80"
```

[topology-aware-routing-safeguards]: https://kubernetes.io/docs/concepts/services-networking/topology-aware-hints/#safeguards
[topology-aware-routing-constraints]: https://kubernetes.io/docs/concepts/services-networking/topology-aware-hints/#constraints
47 changes: 47 additions & 0 deletions linkerd.io/content/2.14/features/topology-aware-routing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---
title: Topology Aware Routing
description: |-
Linkerd's implementation of Kubernetes topology aware routing enables endpoint consumption based on a node's zone label.
---

Kubernetes clusters are increasingly deployed in multi-zone environments with
network traffic often relying on obscure endpoint matching and routing for a
given service. Linkerd's implementation of Kubernetes ([Topology Aware
Routing][topology aware routing]) provides users with a mechanism for
asserting more control over endpoint selection and routing within a cluster
that spans multiple zones. Users can now implement routing constraints and
prefer endpoints in a specific zone in order to limit cross-zone networking
costs or improve performance through lowered cross-zone latency and bandwidth
constraints.

The goal of topology aware routing is to to provide a simpler way for users to
prefer endpoints by basing decisions solely off the node's
`topology.kubernetes.io/zone` label. If a client is in `zone-a`, then it
should prefer endpoints marked for use by clients in `zone-a`. When the
feature is enabled and the label set, Linkerd's destination controller will
attempt to find endpoints whose `routing.ForZones` field matches the client's
zone.

To get started with topology aware routing take a look at the [enabling
topology aware routing](../../tasks/enabling-topology-aware-routing/) task
documentation.

[topology aware routing]:
https://kubernetes.io/docs/concepts/services-networking/topology-aware-routing/

{{< note >}}

Topology aware routing requires that the cluster have the `TopologyAwareHints`
feature gate enabled (which is the default starting with Kubernetes 1.24), and
that Linkerd's `endpointSlice` feature be turned on (this is the default
starting with Linkerd stable-2.12).

{{< /note >}}

{{< note >}}

Starting in Kubernetes 1.31, Kubernetes also has the `trafficDistribution`
feature available as an alternative to topology aware routing.
`trafficDistribution` is not yet supported by Linkerd.

{{< /note >}}
77 changes: 77 additions & 0 deletions linkerd.io/content/2.14/tasks/enabling-topology-aware-routing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
---
title: "Enabling Topology Aware Routing"
description: |-
Enable topology aware routing to allow Linkerd to intelligently choose same-zone endpoints
---

[Topology aware routing](../../features/topology-aware-routing/) allows
Linkerd users to specify endpoint selection boundaries when a request is made
against a service, making it possible to limit cross-zone endpoint selection
and lower the associated costs and latency.

There are three requirements to successfully enabling topology aware routing
with Linkerd:

1. The `TopologyAwareHints` feature gate MUST be enabled on the Kubernetes
cluster. (This feature gate is enabled by default in Kubernetes 1.24 and
later.)

2. Each Kubernetes node needs to be assigned to a zone, using
`"topology.kubernetes.io/zone` label.

3. Relevant Kubernetes services will need to be modified with the
`service.kubernetes.io/topology-aware-hints=auto` annotation.

When Linkerd receives a set of endpoint slices and translates them to an
address set, a copy of the `Hints.forZones` field is made from each endpoint
slice if one is present. This field is only present if it's set by the
endpoint slice controller, and there are several
[safeguards][topology-aware-routing-safeguards] that Kubernetes checks before
setting it. To have the endpoint slice controller set the `forZones` field
users will have to add the `service.kubernetes.io/topology-aware-hints=auto`
annotation to the service.

After the endpoints have been translated into an address set and the endpoint
translator's available endpoints have been updated, filtering is applied to
ensure only the relevant sets of addresses are returned when a request is
made.

For each potential address in the available endpoint set Linkerd returns a set
of addresses whose consumption zone (zones in `forZones` field) matches that
of the node's zone. By doing so Linkerd ensures communication is limited to
endpoints that have been labeled by the endpoint slice controller for the same
node the client is on and limits cross-node and cross-zone communication.

## Constraints

Kubernetes places some [constraints][topology-aware-routing-constraints] on
topology aware routing that you should review before trying to enable topology
aware routing on Linkerd.

The main things to be aware of are:

- Topology aware routing assumes that traffic to a given zone will be roughly
proportional to the capacity of the nodes in the zone. This can be a concern
when using horizontal pod autoscaling, as HPA may start nodes in the "wrong"
zone.

- Services with `internalTrafficPolicy` set to `Local` ignore topology aware
routing, by design.

- If you have workloads running on Nodes labeled with `control-plane` or
`master` role, topology aware routing will not route to endpoints on these
Nodes.

## Configuring Topology Aware Routing

Successful topology aware routing can be confirmed by looking at the Linkerd
proxy logs for the relevant service. The logs should show a stream of messages
similar to the ones below:

```text
time="2021-08-27T14:04:35Z" level=info msg="Establishing watch on endpoint [default/nginx-deploy-svc:80]" addr=":8086" component=endpoints-watcher
time="2021-08-27T14:04:35Z" level=debug msg="Filtering through addresses that should be consumed by zone zone-b" addr=":8086" component=endpoint-translator remote="127.0.0.1:49846" service="nginx-deploy-svc.default.svc.cluster.local:80"
```

[topology-aware-routing-safeguards]: https://kubernetes.io/docs/concepts/services-networking/topology-aware-hints/#safeguards
[topology-aware-routing-constraints]: https://kubernetes.io/docs/concepts/services-networking/topology-aware-hints/#constraints
Loading