From 5d58b073e042ce912372d26aa53e63eb6328d691 Mon Sep 17 00:00:00 2001 From: SkalaNetworks <127797154+SkalaNetworks@users.noreply.github.com> Date: Tue, 2 Jul 2024 14:38:58 +0200 Subject: [PATCH 1/2] chore(doc): add information about policies and rephrase --- docs/advance/with-bgp.en.md | 63 ++++++++++++++++++++++++++++++------- 1 file changed, 52 insertions(+), 11 deletions(-) diff --git a/docs/advance/with-bgp.en.md b/docs/advance/with-bgp.en.md index 23f0077f9..762d0ab77 100644 --- a/docs/advance/with-bgp.en.md +++ b/docs/advance/with-bgp.en.md @@ -1,16 +1,18 @@ # BGP Support -Kube-OVN supports broadcasting the IP address of the Pod or Subnet to the outside world via BGP protocol, -so that the outside world can access the Pod directly through the Pod IP. +Kube-OVN supports broadcasting the IP address of Pods/Subnets to the outside world via the BGP protocol, +so that the outside world can access the Pods directly through their Pod IP. To use this feature, you need to install `kube-ovn-speaker` on specific nodes and add the corresponding annotation to the Pod or Subnet that needs to be exposed to the outside world. -## Install kube-ovn-speaker +Kube-OVN also supports broadcasting the IP address of services of type `ClusterIP` via the same annotation. -`kube-ovn-speaker` use [GoBGP](https://osrg.github.io/gobgp/) to publish routing information to the outside world and -set the next-hop route to itself. +## Installing `kube-ovn-speaker` -Since the node where `kube-ovn-speaker` is deployed needs to carry return traffic, specific labeled nodes need to be selected for deployment: +`kube-ovn-speaker` uses [GoBGP](https://osrg.github.io/gobgp/) to publish routing information to the outside world and to +set the `next-hop` route to itself. + +Since the nodes where `kube-ovn-speaker` is deployed need to carry return traffic, specific labeled nodes need to be selected for deployment: ```bash kubectl label nodes speaker-node-1 ovn.kubernetes.io/bgp=true @@ -38,13 +40,13 @@ Modify the corresponding configuration in yaml: - `neighbor-as`: The AS number of the BGP Peer. - `cluster-as`: The AS number of the container network. -Deploy yaml: +Apply the YAML: ```bash kubectl apply -f speaker.yaml ``` -## Publish Pod/Subnet Routes +## Publishing Pod/Subnet Routes To use BGP for external routing, first set `natOutgoing` to `false` for the corresponding Subnet to allow the Pod IP to enter the underlying network directly. @@ -58,15 +60,54 @@ kubectl annotate subnet ovn-default ovn.kubernetes.io/bgp=true Delete annotation to disable the publishing: ```bash -kubectl annotate pod perf-ovn-xzvd4 ovn.kubernetes.io/bgp- +kubectl annotate pod sample ovn.kubernetes.io/bgp- kubectl annotate subnet ovn-default ovn.kubernetes.io/bgp- ``` -## BGP Advance Options +## Publishing Services of type `ClusterIP` + +To announce the ClusterIP of services to the outside world, the `kube-ovn-speaker` option `announce-cluster-ip` needs to be set to `true`. +See the advanced options for more details. + +Set the annotation to enable publishing: + +```bash +kubectl annotate service sample ovn.kubernetes.io/bgp=true +``` + +Delete annotation to disable the publishing: + +```bash +kubectl annotate service sample ovn.kubernetes.io/bgp- +``` + +## Announcement policies + +There are 2 policies used by `kube-ovn-speaker` to announce the routes: +- **Cluster**: this policy makes the Pod IPs/Subnet CIDRs be announced from every speaker, whether there's Pods +that have that specific IP or that are part of the Subnet CIDR on that node. In other words, traffic may enter from +any node hosting a speaker, and then be internally routed in the cluster to the actual Pod. In this configuration +extra hops might be used. This is the default policy to Pods and Subnets. +- **Local**: this policy makes the Pod IPs be announced only from speakers on nodes that are actively hosting +them. In other words, traffic will only enter from the node hosting the Pod marked as needing BGP advertisement, +or from the node hosting a Pod with an IP belonging to a Subnet marked as needing BGP advertisement. +This makes the network path shorter as external traffic arrives directly to the physical host of the Pod. + +NOTE: You'll probably need to run `kube-ovn-speaker` on every node for the **Local** policy to work. +If a Pod you're trying to announce lands on a node with no speaker on it, its IP will simply not be announced. + +The default policy used is **Cluster**. Policies can be overriden for each Pod/Subnet using the `ovn.kubernetes.io/bgp` annotation: +- `ovn.kubernetes.io/bgp=cluster` or the default `ovn.kubernetes.io/bgp=yes` will use policy **Cluster** +- `ovn.kubernetes.io/bgp=local` will use policy **Local** + +NOTE: Announcement of Services of type `ClusterIP` doesn't support any policy other than **Cluster** as routing to the actual pod +is handled by a daemon such as `kube-proxy`. The annotation for Services only supports value `yes` and not `cluster`. + +## BGP Advanced Options `kube-ovn-speaker` supports more BGP parameters for advanced configuration, which can be adjusted by users according to their network environment: -- `announce-cluster-ip`: Whether to publish Service routes to the public, default is `false`. +- `announce-cluster-ip`: Whether to publish routes for Services of type `ClusterIP` to the public, default is `false`. - `auth-password`: The access password for the BGP peer. - `holdtime`: The heartbeat detection time between BGP neighbors. Neighbors with no messages after the change time will be removed, the default is 90 seconds. - `graceful-restart`: Whether to enable BGP Graceful Restart. From b3d9386dd4661751b938d338cbebd1938288fee3 Mon Sep 17 00:00:00 2001 From: oilbeater Date: Wed, 3 Jul 2024 02:33:03 +0000 Subject: [PATCH 2/2] fix lint issues Signed-off-by: oilbeater --- docs/advance/with-bgp.en.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/advance/with-bgp.en.md b/docs/advance/with-bgp.en.md index 762d0ab77..61e2bf2e1 100644 --- a/docs/advance/with-bgp.en.md +++ b/docs/advance/with-bgp.en.md @@ -82,8 +82,9 @@ kubectl annotate service sample ovn.kubernetes.io/bgp- ``` ## Announcement policies - + There are 2 policies used by `kube-ovn-speaker` to announce the routes: + - **Cluster**: this policy makes the Pod IPs/Subnet CIDRs be announced from every speaker, whether there's Pods that have that specific IP or that are part of the Subnet CIDR on that node. In other words, traffic may enter from any node hosting a speaker, and then be internally routed in the cluster to the actual Pod. In this configuration @@ -97,6 +98,7 @@ NOTE: You'll probably need to run `kube-ovn-speaker` on every node for the **Loc If a Pod you're trying to announce lands on a node with no speaker on it, its IP will simply not be announced. The default policy used is **Cluster**. Policies can be overriden for each Pod/Subnet using the `ovn.kubernetes.io/bgp` annotation: + - `ovn.kubernetes.io/bgp=cluster` or the default `ovn.kubernetes.io/bgp=yes` will use policy **Cluster** - `ovn.kubernetes.io/bgp=local` will use policy **Local**