Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LoadBalancer IPAM #1788

Merged
merged 6 commits into from
Jan 22, 2025
Merged

LoadBalancer IPAM #1788

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
d3ac121
Add loadbalancer docs
MichalFupso Nov 25, 2024
40d7f95
Loadbalancer page update
MichalFupso Jan 20, 2025
dd9d8c3
Merge branch 'main' into loadbalancer
MichalFupso Jan 21, 2025
2849015
update wording
MichalFupso Jan 21, 2025
17cb3d5
Merge branch 'loadbalancer' of github.com:MichalFupso/tigera-docs int…
MichalFupso Jan 21, 2025
cfd056b
copy docs to cloud and enterprise
MichalFupso Jan 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 21 additions & 19 deletions calico-cloud/networking/configuring/advertise-service-ips.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -221,28 +221,30 @@ The following steps will configure $[prodname] to advertise Service `status.Load

For help, see [BGP configuration resource](../../reference/resources/bgpconfig.mdx).

Service LoadBalancer address allocation is outside the current scope of $[prodname], but can be implemented with an external controller.
You can build your own, or use a third-party implementation like the MetalLB project.
$[prodname] IPAM can manage address allocation for Service of type LoadBalancer with LoadBalancerController. To enable $[prodname] IPAM to start managing addresses for Services set up an IPPool for LoadBalancer.

To install the MetalLB controller for allocating addresses, perform the following steps.

1. Follow [the MetalLB documentation](https://metallb.universe.tf/installation/#installation-by-manifest) to install the `metallb-system/controller` resources.

However, do not install the `metallb-system/speaker` component. The speaker component also attempts to establish BGP sessions on the node, and will conflict with Calico.
```bash
kubectl apply -f - <<EOF
apiVersion: projectcalico.org/v3
kind: IPPool
metadata:
name: loadBalancerIPPool
spec:
cidr: 192.210.0.0/20
blockSize: 24
natOutgoing: true
disabled: false
assignmentMode: Automatic
allowedUses:
- LoadBalancer
EOF
```

1. Configure MetalLB to provision addresses by creating the following IPAddressPool, replacing `x.x.x.x/16` with the CIDR given to $[prodname] in the steps above. Please note IPAddressPool requires Metallb `0.13+`.
For more information see, [LoadBalancer IP address management](../ipam/service-loadbalancer.mdx)

```bash
kubectl create -f - <<EOF
kind: IPAddressPool
metadata:
name: default
namespace: metallb-system
spec:
addresses:
- x.x.x.x/16
EOF
```
:::note
If you have previously used alternative IPAM solution for Service of type LoadBalancer address management and wish to continue using it, do not create Calico IPPool for LoadBalancer.
:::

### Exclude certain nodes from advertisement

Expand Down
1 change: 1 addition & 0 deletions calico-cloud/networking/index.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ The $[prodname] network plugins provide a range of networking options to fit you
<DocCardLink docId='networking/configuring/workloads-outside-cluster' />
<DocCardLink docId='networking/configuring/pod-mac-address' />
<DocCardLink docId='networking/configuring/node-local-dns-cache' />
<DocCardLink docId='networking/ipam/service-loadbalancer' />
</DocCardLinkLayout>


Expand Down
120 changes: 120 additions & 0 deletions calico-cloud/networking/ipam/service-loadbalancer.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
---
description: LoadBalancer IP address management
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

# LoadBalancer IP address management

## Understanding $[prodname] LoadBalancer IP address management

You might want to utilize Service of type LoadBalancer in your cluster to provide stable long-lasting IP address to access your deployed application.
$[prodname] can help you with providing and managing IP addresses to your Services in your cluster. $[prodname] comes with LoadBalancer IPAM deployed as part of $[prodname] Kube-controllers.

### Before you begin...

Ensure that you have a cluster with $[prodname] installed, and kube-controllers configured and running.

### IP Pool for Service of type LoadBalancer

$[prodname] does not automatically provide default IP Pool for LoadBalancer IP address management. You will need to create an IP Pool with `allowedUses` LoadBalancer for $[prodname] to start assigning Service IPs.

```yaml
apiVersion: projectcalico.org/v3
kind: IPPool
metadata:
name: loadbalancerIPPool
spec:
cidr: 192.210.0.0/20
blockSize: 24
natOutgoing: true
disabled: false
assignmentMode: Automatic
allowedUses:
- LoadBalancer
```

:::note

You can create multiple IP Pools with allowedUses LoadBalancer as long as there are no CIDR conflicts. By setting assignmentMode to Manual you can reserve the IP Pool for manual assignments only. Explore more about [manual assignment](#manual-service-ip-address-assignment)

:::

### Automatic Service IP address assignment

When you create Service of type LoadBalancer $[prodname] kube-controller will automatically detect the new Service and assign an IP address from available IP Pool. If no address is available, the Service will remain in pending state until an address is available.

```yaml
apiVersion: v1
kind: Service
metadata:
name: service-loadbalancer
spec:
selector:
app: nginx
ports:
- port: 80
targetPort: 80
name: default
type: LoadBalancer
```

### Manual Service IP address assignment

There are cases where you would like to be more specific about what IP address $[prodname] assigns to your Service. With annotations, you can specify how IP address should be assigned. The annotations can be added and removed as needed during the lifetime of the Service. When you remove an annotation $[prodname] kube-controller will check if the assigned IP is still valid and potentially assign a new one.

#### Specify IP Pool

##### IPv4 Pool

Annotate the Service with projectcalico.org/ipv4pools to assign IP address from the selected IP Pools. You can specify multiple IP Pools in this annotation, $[prodname] will pick an available IP address to assign.

```
"projectcalico.org/ipv4pools": '["loadBalancerIPv4Pool"]'
```

##### IPv6 Pool

Annotate the Service with projectcalico.org/ipv6pools to assign IP address from the selected IP Pools. You can specify multiple IP Pools in this annotation, $[prodname] will pick an available IP address to assign.

```
"projectcalico.org/ipv6pools": '["loadBalancerIPv6Pool"]'
```

:::note

In dual-stack environment you don't have to specify both annotations. If IP Pool was not specified for IP family, $[prodname] will automatically assign available IP address from available IP Pool

:::

#### Specifying IP address

Annotate the Service with projectcalico.org/loadBalancerIPs to assign specific IP address. The address must be available otherwise $[prodname] will not be able to assign the address. Currently you can only specify one IPv4 and one IPv6 address at the same time.

```
"projectcalico.org/loadBalancerIPs": '["x.x.x.x"]'
```

:::note

If Service contains "projectcalico.org/loadBalancerIPs" annotation and either "projectcalico.org/ipv6pools" or "projectcalico.org/ipv4pools" are present, $[prodname] will favour projectcalico.org/loadBalancerIPs annotation and try to assign that IP address. There is no fall back to an IP Pool if the specified address is not available.

:::

### Manage LoadBalancer kube-controller assignment mode

In certain cases you might not wish for $[prodname] to automatically assign IP addresses to your Service. This can be useful in case you have multiple Service IPAM solutions in your cluster.
LoadBalancer kube-controller is able to operate in two distinct modes ```Automatic``` in which $[prodname] assigns IP address to each Service in your cluster and ```RequestedServicesOnly``` in which $[prodname] only assigns IP addresses to Service with annotations mentioned above.
You can change the mode at any point, but note that switching to ```RequestedServicesOnly``` will unassign any addresses from Services that do not contain the above annotations.

```bash
kubectl patch kubecontrollersconfiguration default --patch '{"spec": {"controllers":{"loadBalancer":{"AssignIPs": "RequestedServicesOnly"}}}}'
```

### Additional resources

Calico LoadBalancer IP address management works in conjunction with other $[prodname] components. $[prodname] kube-controllers provides only the IP address management, to advertise LoadBalancer IPs you will have to update your BGP configuration. You can find out more information at:
* [Advertising Kubernetes service IP addresses](../../networking/configuring/advertise-service-ips.mdx#advertise-service-load-balancer-ip-addresses)
* [IP Pool](../../reference/resources/ippool.mdx)
* [Kube-controllers configuration](../../reference/resources/kubecontrollersconfig.mdx#loadbalancercontroller)
27 changes: 13 additions & 14 deletions calico-cloud/reference/resources/ippool.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,12 +45,13 @@ spec:
| blockSize | The CIDR size of allocation blocks used by this pool. Blocks are allocated on demand to hosts and are used to aggregate routes. The value can only be set when the pool is created. | 20 to 32 (inclusive) for IPv4 and 116 to 128 (inclusive) for IPv6 | int | `26` for IPv4 pools and `122` for IPv6 pools. |
| ipipMode | The mode defining when IPIP will be used. Cannot be set at the same time as `vxlanMode`. | Always, CrossSubnet, Never | string | `Never` |
| vxlanMode | The mode defining when VXLAN will be used. Cannot be set at the same time as `ipipMode`. | Always, CrossSubnet, Never | string | `Never` |
| natOutgoing | When enabled, packets sent from $[prodname] networked containers in this pool to destinations outside of any Calico IP pools will be masqueraded. | true, false | boolean | `false` |
| disabled | When set to true, $[prodname] IPAM will not assign addresses from this pool. | true, false | boolean | `false` |
| natOutgoing | When enabled, packets sent from $[prodname] networked containers in this pool to destinations outside of any Calico IP pools will be masqueraded. | true, false | boolean | `false` |
| disabled | When set to true, $[prodname] IPAM will not assign addresses from this pool. | true, false | boolean | `false` |
| disableBGPExport _(since v3.11.0)_ | Disable exporting routes from this IP Pool’s CIDR over BGP. | true, false | boolean | `false` |
| nodeSelector | Selects the nodes where $[prodname] IPAM should assign pod addresses from this pool. Can be overridden if a pod [explicitly identifies this IP pool by annotation](../component-resources/configuration.mdx#using-kubernetes-annotations). | | [selector](#node-selector) | all() |
| allowedUses _(since v3.11.0)_ | Controls whether the pool will be used for automatic assignments of certain types. See [below](#allowed-uses). | Workload, Tunnel, HostSecondaryInterface | list of strings | `["Workload", "Tunnel"]` |
| awsSubnetID _(since v3.11.0)_ | May be set to the ID of an AWS VPC Subnet that contains the CIDR of this IP pool to activate the AWS-backed pool feature. See [below](#aws-backed-pools). | Valid AWS Subnet ID. | string | |
| nodeSelector | Selects the nodes where $[prodname] IPAM should assign pod addresses from this pool. Can be overridden if a pod [explicitly identifies this IP pool by annotation](../component-resources/configuration.mdx#using-kubernetes-annotations). | | [selector](#node-selector) | all() |
| allowedUses _(since v3.11.0)_ | Controls whether the pool will be used for automatic assignments of certain types. See [below](#allowed-uses). | Workload, Tunnel, HostSecondaryInterface, LoadBalancer | list of strings | `["Workload", "Tunnel"]` |
| awsSubnetID _(since v3.11.0)_ | May be set to the ID of an AWS VPC Subnet that contains the CIDR of this IP pool to activate the AWS-backed pool feature. See [below](#aws-backed-pools). | Valid AWS Subnet ID. | string | |
| assignmentMode | Controls whether the pool will be used for automatic assignments or only if requested manually | Automatic, Manual | strings | `Automatic` |

:::note

Expand All @@ -66,6 +67,13 @@ When automatically assigning IP addresses to workloads, only pools with "Workloa
consulted. Similarly, when assigning IPs for tunnel devices, only "Tunnel" pools are eligible. Finally, when
assigning IP addresses for AWS secondary ENIs, only pools with allowed use "HostSecondaryInterface" are candidates.

Combining options for the `allowedUses` field is limited. You can specify only the following options and option combinations:
* `allowedUses: ["Tunnel","Workload"]` (default)
* `allowedUses: ["Tunnel"]`
* `allowedUses: ["Workload"]`
* `allowedUses: ["LoadBalancer"]`
* `allowedUses: ["HostSecondaryInterface"]`

If the `allowedUses` field is not specified, it defaults to `["Workload", "Tunnel"]` for compatibility with older
versions of Calico. It is not possible to specify a pool with no allowed uses.

Expand Down Expand Up @@ -130,15 +138,6 @@ Reducing the block size from the default (e.g., using `28` for IPv4 to give 16 a
For details on configuring IP pool node selectors, please read the
[Assign IP addresses based on topology guide.](../../networking/ipam/assign-ip-addresses-topology.mdx).

:::tip

To prevent an IP pool from being used automatically by $[prodname] IPAM, while still allowing
it to be used manually for static assignments, set the `IPPool`'s `nodeSelector` to `!all()`. Since the selector
matches no nodes, the IPPool will not be used automatically and, unlike setting `disabled: true`, it can still be
used for manual assignments.

:::

#### Selector reference

<Selectors />
Expand Down
12 changes: 11 additions & 1 deletion calico-cloud/reference/resources/kubecontrollersconfig.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ spec:
syncLabels: Enabled
hostEndpoint:
autoCreate: Disabled
loadbalancer:
assignIPs: AllServices
```

## Kubernetes controllers configuration definition
Expand Down Expand Up @@ -76,7 +78,15 @@ The federated services controller syncs Kubernetes services from remote clusters

| Field | Description | Schema | Default |
| ---------------- | ---------------------------------------------------------------- | --------------------------------- | ------- |
| reconcilerPeriod | Period to perform reconciliation with the $[prodname] datastore | [Duration string][parse-duration] | 5m |
| reconcilerPeriod | Period to perform reconciliation with the $[prodname] datastore | [Duration string][parse-duration] | 5m |

### LoadBalancerController

The load balancer controller manages IPAM for Services of type LoadBalancer.

| Field | Description | Accepted Values | Schema | Default |
| ---------------- | ---------------------------------------------------------------- | ---------------------------------------- | ------ | ----------- |
| assignIPs | Mode in which LoadBalancer controller operates | AllServices, RequestedServicesOnly | String | AllServices |

## Supported operations

Expand Down
Loading
Loading