diff --git a/calico-cloud/networking/configuring/advertise-service-ips.mdx b/calico-cloud/networking/configuring/advertise-service-ips.mdx index 748840379c..dc7b40d9b2 100644 --- a/calico-cloud/networking/configuring/advertise-service-ips.mdx +++ b/calico-cloud/networking/configuring/advertise-service-ips.mdx @@ -221,28 +221,30 @@ The following steps will configure $[prodname] to advertise Service `status.Load For help, see [BGP configuration resource](../../reference/resources/bgpconfig.mdx). -Service LoadBalancer address allocation is outside the current scope of $[prodname], but can be implemented with an external controller. -You can build your own, or use a third-party implementation like the MetalLB project. +$[prodname] IPAM can manage address allocation for Service of type LoadBalancer with LoadBalancerController. To enable $[prodname] IPAM to start managing addresses for Services set up an IPPool for LoadBalancer. -To install the MetalLB controller for allocating addresses, perform the following steps. - -1. Follow [the MetalLB documentation](https://metallb.universe.tf/installation/#installation-by-manifest) to install the `metallb-system/controller` resources. - - However, do not install the `metallb-system/speaker` component. The speaker component also attempts to establish BGP sessions on the node, and will conflict with Calico. +```bash +kubectl apply -f - < + diff --git a/calico-cloud/networking/ipam/service-loadbalancer.mdx b/calico-cloud/networking/ipam/service-loadbalancer.mdx new file mode 100644 index 0000000000..cd19b4f216 --- /dev/null +++ b/calico-cloud/networking/ipam/service-loadbalancer.mdx @@ -0,0 +1,120 @@ +--- +description: LoadBalancer IP address management +--- + +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + +# LoadBalancer IP address management + +## Understanding $[prodname] LoadBalancer IP address management + +You might want to utilize Service of type LoadBalancer in your cluster to provide stable long-lasting IP address to access your deployed application. +$[prodname] can help you with providing and managing IP addresses to your Services in your cluster. $[prodname] comes with LoadBalancer IPAM deployed as part of $[prodname] Kube-controllers. + +### Before you begin... + +Ensure that you have a cluster with $[prodname] installed, and kube-controllers configured and running. + +### IP Pool for Service of type LoadBalancer + +$[prodname] does not automatically provide default IP Pool for LoadBalancer IP address management. You will need to create an IP Pool with `allowedUses` LoadBalancer for $[prodname] to start assigning Service IPs. + +```yaml +apiVersion: projectcalico.org/v3 +kind: IPPool +metadata: + name: loadbalancerIPPool +spec: + cidr: 192.210.0.0/20 + blockSize: 24 + natOutgoing: true + disabled: false + assignmentMode: Automatic + allowedUses: + - LoadBalancer +``` + +:::note + +You can create multiple IP Pools with allowedUses LoadBalancer as long as there are no CIDR conflicts. By setting assignmentMode to Manual you can reserve the IP Pool for manual assignments only. Explore more about [manual assignment](#manual-service-ip-address-assignment) + +::: + +### Automatic Service IP address assignment + +When you create Service of type LoadBalancer $[prodname] kube-controller will automatically detect the new Service and assign an IP address from available IP Pool. If no address is available, the Service will remain in pending state until an address is available. + +```yaml +apiVersion: v1 +kind: Service +metadata: + name: service-loadbalancer +spec: + selector: + app: nginx + ports: + - port: 80 + targetPort: 80 + name: default + type: LoadBalancer +``` + +### Manual Service IP address assignment + +There are cases where you would like to be more specific about what IP address $[prodname] assigns to your Service. With annotations, you can specify how IP address should be assigned. The annotations can be added and removed as needed during the lifetime of the Service. When you remove an annotation $[prodname] kube-controller will check if the assigned IP is still valid and potentially assign a new one. + +#### Specify IP Pool + +##### IPv4 Pool + +Annotate the Service with projectcalico.org/ipv4pools to assign IP address from the selected IP Pools. You can specify multiple IP Pools in this annotation, $[prodname] will pick an available IP address to assign. + +``` +"projectcalico.org/ipv4pools": '["loadBalancerIPv4Pool"]' +``` + +##### IPv6 Pool + +Annotate the Service with projectcalico.org/ipv6pools to assign IP address from the selected IP Pools. You can specify multiple IP Pools in this annotation, $[prodname] will pick an available IP address to assign. + +``` +"projectcalico.org/ipv6pools": '["loadBalancerIPv6Pool"]' +``` + +:::note + +In dual-stack environment you don't have to specify both annotations. If IP Pool was not specified for IP family, $[prodname] will automatically assign available IP address from available IP Pool + +::: + +#### Specifying IP address + +Annotate the Service with projectcalico.org/loadBalancerIPs to assign specific IP address. The address must be available otherwise $[prodname] will not be able to assign the address. Currently you can only specify one IPv4 and one IPv6 address at the same time. + +``` +"projectcalico.org/loadBalancerIPs": '["x.x.x.x"]' +``` + +:::note + +If Service contains "projectcalico.org/loadBalancerIPs" annotation and either "projectcalico.org/ipv6pools" or "projectcalico.org/ipv4pools" are present, $[prodname] will favour projectcalico.org/loadBalancerIPs annotation and try to assign that IP address. There is no fall back to an IP Pool if the specified address is not available. + +::: + +### Manage LoadBalancer kube-controller assignment mode + +In certain cases you might not wish for $[prodname] to automatically assign IP addresses to your Service. This can be useful in case you have multiple Service IPAM solutions in your cluster. +LoadBalancer kube-controller is able to operate in two distinct modes ```Automatic``` in which $[prodname] assigns IP address to each Service in your cluster and ```RequestedServicesOnly``` in which $[prodname] only assigns IP addresses to Service with annotations mentioned above. +You can change the mode at any point, but note that switching to ```RequestedServicesOnly``` will unassign any addresses from Services that do not contain the above annotations. + +```bash +kubectl patch kubecontrollersconfiguration default --patch '{"spec": {"controllers":{"loadBalancer":{"AssignIPs": "RequestedServicesOnly"}}}}' +``` + +### Additional resources + +Calico LoadBalancer IP address management works in conjunction with other $[prodname] components. $[prodname] kube-controllers provides only the IP address management, to advertise LoadBalancer IPs you will have to update your BGP configuration. You can find out more information at: + * [Advertising Kubernetes service IP addresses](../../networking/configuring/advertise-service-ips.mdx#advertise-service-load-balancer-ip-addresses) + * [IP Pool](../../reference/resources/ippool.mdx) + * [Kube-controllers configuration](../../reference/resources/kubecontrollersconfig.mdx#loadbalancercontroller) diff --git a/calico-cloud/reference/resources/ippool.mdx b/calico-cloud/reference/resources/ippool.mdx index 035e33cc1d..245efb3470 100644 --- a/calico-cloud/reference/resources/ippool.mdx +++ b/calico-cloud/reference/resources/ippool.mdx @@ -45,12 +45,13 @@ spec: | blockSize | The CIDR size of allocation blocks used by this pool. Blocks are allocated on demand to hosts and are used to aggregate routes. The value can only be set when the pool is created. | 20 to 32 (inclusive) for IPv4 and 116 to 128 (inclusive) for IPv6 | int | `26` for IPv4 pools and `122` for IPv6 pools. | | ipipMode | The mode defining when IPIP will be used. Cannot be set at the same time as `vxlanMode`. | Always, CrossSubnet, Never | string | `Never` | | vxlanMode | The mode defining when VXLAN will be used. Cannot be set at the same time as `ipipMode`. | Always, CrossSubnet, Never | string | `Never` | -| natOutgoing | When enabled, packets sent from $[prodname] networked containers in this pool to destinations outside of any Calico IP pools will be masqueraded. | true, false | boolean | `false` | -| disabled | When set to true, $[prodname] IPAM will not assign addresses from this pool. | true, false | boolean | `false` | +| natOutgoing | When enabled, packets sent from $[prodname] networked containers in this pool to destinations outside of any Calico IP pools will be masqueraded. | true, false | boolean | `false` | +| disabled | When set to true, $[prodname] IPAM will not assign addresses from this pool. | true, false | boolean | `false` | | disableBGPExport _(since v3.11.0)_ | Disable exporting routes from this IP Pool’s CIDR over BGP. | true, false | boolean | `false` | -| nodeSelector | Selects the nodes where $[prodname] IPAM should assign pod addresses from this pool. Can be overridden if a pod [explicitly identifies this IP pool by annotation](../component-resources/configuration.mdx#using-kubernetes-annotations). | | [selector](#node-selector) | all() | -| allowedUses _(since v3.11.0)_ | Controls whether the pool will be used for automatic assignments of certain types. See [below](#allowed-uses). | Workload, Tunnel, HostSecondaryInterface | list of strings | `["Workload", "Tunnel"]` | -| awsSubnetID _(since v3.11.0)_ | May be set to the ID of an AWS VPC Subnet that contains the CIDR of this IP pool to activate the AWS-backed pool feature. See [below](#aws-backed-pools). | Valid AWS Subnet ID. | string | | +| nodeSelector | Selects the nodes where $[prodname] IPAM should assign pod addresses from this pool. Can be overridden if a pod [explicitly identifies this IP pool by annotation](../component-resources/configuration.mdx#using-kubernetes-annotations). | | [selector](#node-selector) | all() | +| allowedUses _(since v3.11.0)_ | Controls whether the pool will be used for automatic assignments of certain types. See [below](#allowed-uses). | Workload, Tunnel, HostSecondaryInterface, LoadBalancer | list of strings | `["Workload", "Tunnel"]` | +| awsSubnetID _(since v3.11.0)_ | May be set to the ID of an AWS VPC Subnet that contains the CIDR of this IP pool to activate the AWS-backed pool feature. See [below](#aws-backed-pools). | Valid AWS Subnet ID. | string | | +| assignmentMode | Controls whether the pool will be used for automatic assignments or only if requested manually | Automatic, Manual | strings | `Automatic` | :::note @@ -66,6 +67,13 @@ When automatically assigning IP addresses to workloads, only pools with "Workloa consulted. Similarly, when assigning IPs for tunnel devices, only "Tunnel" pools are eligible. Finally, when assigning IP addresses for AWS secondary ENIs, only pools with allowed use "HostSecondaryInterface" are candidates. +Combining options for the `allowedUses` field is limited. You can specify only the following options and option combinations: + * `allowedUses: ["Tunnel","Workload"]` (default) + * `allowedUses: ["Tunnel"]` + * `allowedUses: ["Workload"]` + * `allowedUses: ["LoadBalancer"]` + * `allowedUses: ["HostSecondaryInterface"]` + If the `allowedUses` field is not specified, it defaults to `["Workload", "Tunnel"]` for compatibility with older versions of Calico. It is not possible to specify a pool with no allowed uses. @@ -130,15 +138,6 @@ Reducing the block size from the default (e.g., using `28` for IPv4 to give 16 a For details on configuring IP pool node selectors, please read the [Assign IP addresses based on topology guide.](../../networking/ipam/assign-ip-addresses-topology.mdx). -:::tip - -To prevent an IP pool from being used automatically by $[prodname] IPAM, while still allowing -it to be used manually for static assignments, set the `IPPool`'s `nodeSelector` to `!all()`. Since the selector -matches no nodes, the IPPool will not be used automatically and, unlike setting `disabled: true`, it can still be -used for manual assignments. - -::: - #### Selector reference diff --git a/calico-cloud/reference/resources/kubecontrollersconfig.mdx b/calico-cloud/reference/resources/kubecontrollersconfig.mdx index 4a6cc6451f..47e1a96784 100644 --- a/calico-cloud/reference/resources/kubecontrollersconfig.mdx +++ b/calico-cloud/reference/resources/kubecontrollersconfig.mdx @@ -24,6 +24,8 @@ spec: syncLabels: Enabled hostEndpoint: autoCreate: Disabled + loadbalancer: + assignIPs: AllServices ``` ## Kubernetes controllers configuration definition @@ -76,7 +78,15 @@ The federated services controller syncs Kubernetes services from remote clusters | Field | Description | Schema | Default | | ---------------- | ---------------------------------------------------------------- | --------------------------------- | ------- | -| reconcilerPeriod | Period to perform reconciliation with the $[prodname] datastore | [Duration string][parse-duration] | 5m | +| reconcilerPeriod | Period to perform reconciliation with the $[prodname] datastore | [Duration string][parse-duration] | 5m | + +### LoadBalancerController + +The load balancer controller manages IPAM for Services of type LoadBalancer. + +| Field | Description | Accepted Values | Schema | Default | +| ---------------- | ---------------------------------------------------------------- | ---------------------------------------- | ------ | ----------- | +| assignIPs | Mode in which LoadBalancer controller operates | AllServices, RequestedServicesOnly | String | AllServices | ## Supported operations diff --git a/calico-enterprise/networking/configuring/advertise-service-ips.mdx b/calico-enterprise/networking/configuring/advertise-service-ips.mdx index 618746530e..4eaa61033b 100644 --- a/calico-enterprise/networking/configuring/advertise-service-ips.mdx +++ b/calico-enterprise/networking/configuring/advertise-service-ips.mdx @@ -221,28 +221,30 @@ The following steps will configure $[prodname] to advertise Service `status.Load For help, see [BGP configuration resource](../../reference/resources/bgpconfig.mdx). -Service LoadBalancer address allocation is outside the current scope of $[prodname], but can be implemented with an external controller. -You can build your own, or use a third-party implementation like the MetalLB project. +$[prodname] IPAM can manage address allocation for Service of type LoadBalancer with LoadBalancerController. To enable $[prodname] IPAM to start managing addresses for Services set up an IPPool for LoadBalancer. -To install the MetalLB controller for allocating addresses, perform the following steps. - -1. Follow [the MetalLB documentation](https://metallb.universe.tf/installation/#installation-by-manifest) to install the `metallb-system/controller` resources. - - However, do not install the `metallb-system/speaker` component. The speaker component also attempts to establish BGP sessions on the node, and will conflict with Calico. +```bash +kubectl apply -f - < + diff --git a/calico-enterprise/networking/ipam/service-loadbalancer.mdx b/calico-enterprise/networking/ipam/service-loadbalancer.mdx new file mode 100644 index 0000000000..cd19b4f216 --- /dev/null +++ b/calico-enterprise/networking/ipam/service-loadbalancer.mdx @@ -0,0 +1,120 @@ +--- +description: LoadBalancer IP address management +--- + +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + +# LoadBalancer IP address management + +## Understanding $[prodname] LoadBalancer IP address management + +You might want to utilize Service of type LoadBalancer in your cluster to provide stable long-lasting IP address to access your deployed application. +$[prodname] can help you with providing and managing IP addresses to your Services in your cluster. $[prodname] comes with LoadBalancer IPAM deployed as part of $[prodname] Kube-controllers. + +### Before you begin... + +Ensure that you have a cluster with $[prodname] installed, and kube-controllers configured and running. + +### IP Pool for Service of type LoadBalancer + +$[prodname] does not automatically provide default IP Pool for LoadBalancer IP address management. You will need to create an IP Pool with `allowedUses` LoadBalancer for $[prodname] to start assigning Service IPs. + +```yaml +apiVersion: projectcalico.org/v3 +kind: IPPool +metadata: + name: loadbalancerIPPool +spec: + cidr: 192.210.0.0/20 + blockSize: 24 + natOutgoing: true + disabled: false + assignmentMode: Automatic + allowedUses: + - LoadBalancer +``` + +:::note + +You can create multiple IP Pools with allowedUses LoadBalancer as long as there are no CIDR conflicts. By setting assignmentMode to Manual you can reserve the IP Pool for manual assignments only. Explore more about [manual assignment](#manual-service-ip-address-assignment) + +::: + +### Automatic Service IP address assignment + +When you create Service of type LoadBalancer $[prodname] kube-controller will automatically detect the new Service and assign an IP address from available IP Pool. If no address is available, the Service will remain in pending state until an address is available. + +```yaml +apiVersion: v1 +kind: Service +metadata: + name: service-loadbalancer +spec: + selector: + app: nginx + ports: + - port: 80 + targetPort: 80 + name: default + type: LoadBalancer +``` + +### Manual Service IP address assignment + +There are cases where you would like to be more specific about what IP address $[prodname] assigns to your Service. With annotations, you can specify how IP address should be assigned. The annotations can be added and removed as needed during the lifetime of the Service. When you remove an annotation $[prodname] kube-controller will check if the assigned IP is still valid and potentially assign a new one. + +#### Specify IP Pool + +##### IPv4 Pool + +Annotate the Service with projectcalico.org/ipv4pools to assign IP address from the selected IP Pools. You can specify multiple IP Pools in this annotation, $[prodname] will pick an available IP address to assign. + +``` +"projectcalico.org/ipv4pools": '["loadBalancerIPv4Pool"]' +``` + +##### IPv6 Pool + +Annotate the Service with projectcalico.org/ipv6pools to assign IP address from the selected IP Pools. You can specify multiple IP Pools in this annotation, $[prodname] will pick an available IP address to assign. + +``` +"projectcalico.org/ipv6pools": '["loadBalancerIPv6Pool"]' +``` + +:::note + +In dual-stack environment you don't have to specify both annotations. If IP Pool was not specified for IP family, $[prodname] will automatically assign available IP address from available IP Pool + +::: + +#### Specifying IP address + +Annotate the Service with projectcalico.org/loadBalancerIPs to assign specific IP address. The address must be available otherwise $[prodname] will not be able to assign the address. Currently you can only specify one IPv4 and one IPv6 address at the same time. + +``` +"projectcalico.org/loadBalancerIPs": '["x.x.x.x"]' +``` + +:::note + +If Service contains "projectcalico.org/loadBalancerIPs" annotation and either "projectcalico.org/ipv6pools" or "projectcalico.org/ipv4pools" are present, $[prodname] will favour projectcalico.org/loadBalancerIPs annotation and try to assign that IP address. There is no fall back to an IP Pool if the specified address is not available. + +::: + +### Manage LoadBalancer kube-controller assignment mode + +In certain cases you might not wish for $[prodname] to automatically assign IP addresses to your Service. This can be useful in case you have multiple Service IPAM solutions in your cluster. +LoadBalancer kube-controller is able to operate in two distinct modes ```Automatic``` in which $[prodname] assigns IP address to each Service in your cluster and ```RequestedServicesOnly``` in which $[prodname] only assigns IP addresses to Service with annotations mentioned above. +You can change the mode at any point, but note that switching to ```RequestedServicesOnly``` will unassign any addresses from Services that do not contain the above annotations. + +```bash +kubectl patch kubecontrollersconfiguration default --patch '{"spec": {"controllers":{"loadBalancer":{"AssignIPs": "RequestedServicesOnly"}}}}' +``` + +### Additional resources + +Calico LoadBalancer IP address management works in conjunction with other $[prodname] components. $[prodname] kube-controllers provides only the IP address management, to advertise LoadBalancer IPs you will have to update your BGP configuration. You can find out more information at: + * [Advertising Kubernetes service IP addresses](../../networking/configuring/advertise-service-ips.mdx#advertise-service-load-balancer-ip-addresses) + * [IP Pool](../../reference/resources/ippool.mdx) + * [Kube-controllers configuration](../../reference/resources/kubecontrollersconfig.mdx#loadbalancercontroller) diff --git a/calico-enterprise/reference/clis/calicoctl/ipam/check.mdx b/calico-enterprise/reference/clis/calicoctl/ipam/check.mdx index beda092ade..7579b652eb 100644 --- a/calico-enterprise/reference/clis/calicoctl/ipam/check.mdx +++ b/calico-enterprise/reference/clis/calicoctl/ipam/check.mdx @@ -17,13 +17,14 @@ Usage: calicoctl ipam check [--config=] [--show-all-ips] [--show-problem-ips] [-o ] Options: - -h --help Show this screen. - -o --output= Path to output report file. - --show-all-ips Print all IPs that are checked. - --show-problem-ips Print all IPs that are leaked or not allocated properly. - -c --config= Path to the file containing connection configuration in - YAML or JSON format. - [default: /etc/calico/calicoctl.cfg] + -h --help Show this screen. + -o --output= Path to output report file. + --show-all-ips Print all IPs that are checked. + --show-problem-ips Print all IPs that are leaked or not allocated properly. + -c --config= Path to the file containing connection configuration in + YAML or JSON format. + [default: /etc/calico/calicoctl.cfg] + --kubeconfig= Path to Kubeconfig file Description: The ipam check command checks the integrity of the IPAM datastructures against Kubernetes. diff --git a/calico-enterprise/reference/resources/ippool.mdx b/calico-enterprise/reference/resources/ippool.mdx index a1d04dba05..1b40a3a604 100644 --- a/calico-enterprise/reference/resources/ippool.mdx +++ b/calico-enterprise/reference/resources/ippool.mdx @@ -45,12 +45,13 @@ spec: | blockSize | The CIDR size of allocation blocks used by this pool. Blocks are allocated on demand to hosts and are used to aggregate routes. The value can only be set when the pool is created. | 20 to 32 (inclusive) for IPv4 and 116 to 128 (inclusive) for IPv6 | int | `26` for IPv4 pools and `122` for IPv6 pools. | | ipipMode | The mode defining when IPIP will be used. Cannot be set at the same time as `vxlanMode`. | Always, CrossSubnet, Never | string | `Never` | | vxlanMode | The mode defining when VXLAN will be used. Cannot be set at the same time as `ipipMode`. | Always, CrossSubnet, Never | string | `Never` | -| natOutgoing | When enabled, packets sent from $[prodname] networked containers in this pool to destinations outside of any Calico IP pools will be masqueraded. | true, false | boolean | `false` | -| disabled | When set to true, $[prodname] IPAM will not assign addresses from this pool. | true, false | boolean | `false` | +| natOutgoing | When enabled, packets sent from $[prodname] networked containers in this pool to destinations outside of any Calico IP pools will be masqueraded. | true, false | boolean | `false` | +| disabled | When set to true, $[prodname] IPAM will not assign addresses from this pool. | true, false | boolean | `false` | | disableBGPExport _(since v3.11.0)_ | Disable exporting routes from this IP Pool’s CIDR over BGP. | true, false | boolean | `false` | -| nodeSelector | Selects the nodes where $[prodname] IPAM should assign pod addresses from this pool. Can be overridden if a pod [explicitly identifies this IP pool by annotation](../component-resources/configuration.mdx#using-kubernetes-annotations). | | [selector](#node-selector) | all() | -| allowedUses _(since v3.11.0)_ | Controls whether the pool will be used for automatic assignments of certain types. See [below](#allowed-uses). | Workload, Tunnel, HostSecondaryInterface | list of strings | `["Workload", "Tunnel"]` | -| awsSubnetID _(since v3.11.0)_ | May be set to the ID of an AWS VPC Subnet that contains the CIDR of this IP pool to activate the AWS-backed pool feature. See [below](#aws-backed-pools). | Valid AWS Subnet ID. | string | | +| nodeSelector | Selects the nodes where $[prodname] IPAM should assign pod addresses from this pool. Can be overridden if a pod [explicitly identifies this IP pool by annotation](../component-resources/configuration.mdx#using-kubernetes-annotations). | | [selector](#node-selector) | all() | +| allowedUses _(since v3.11.0)_ | Controls whether the pool will be used for automatic assignments of certain types. See [below](#allowed-uses). | Workload, Tunnel, HostSecondaryInterface, LoadBalancer | list of strings | `["Workload", "Tunnel"]` | +| awsSubnetID _(since v3.11.0)_ | May be set to the ID of an AWS VPC Subnet that contains the CIDR of this IP pool to activate the AWS-backed pool feature. See [below](#aws-backed-pools). | Valid AWS Subnet ID. | string | | +| assignmentMode | Controls whether the pool will be used for automatic assignments or only if requested manually | Automatic, Manual | strings | `Automatic` | :::note @@ -66,6 +67,13 @@ When automatically assigning IP addresses to workloads, only pools with "Workloa consulted. Similarly, when assigning IPs for tunnel devices, only "Tunnel" pools are eligible. Finally, when assigning IP addresses for AWS secondary ENIs, only pools with allowed use "HostSecondaryInterface" are candidates. +Combining options for the `allowedUses` field is limited. You can specify only the following options and option combinations: + * `allowedUses: ["Tunnel","Workload"]` (default) + * `allowedUses: ["Tunnel"]` + * `allowedUses: ["Workload"]` + * `allowedUses: ["LoadBalancer"]` + * `allowedUses: ["HostSecondaryInterface"]` + If the `allowedUses` field is not specified, it defaults to `["Workload", "Tunnel"]` for compatibility with older versions of Calico. It is not possible to specify a pool with no allowed uses. @@ -130,15 +138,6 @@ Reducing the block size from the default (e.g., using `28` for IPv4 to give 16 a For details on configuring IP pool node selectors, please read the [Assign IP addresses based on topology guide.](../../networking/ipam/assign-ip-addresses-topology.mdx). -:::tip - -To prevent an IP pool from being used automatically by $[prodname] IPAM, while still allowing -it to be used manually for static assignments, set the `IPPool`'s `nodeSelector` to `!all()`. Since the selector -matches no nodes, the IPPool will not be used automatically and, unlike setting `disabled: true`, it can still be -used for manual assignments. - -::: - #### Selector reference diff --git a/calico-enterprise/reference/resources/kubecontrollersconfig.mdx b/calico-enterprise/reference/resources/kubecontrollersconfig.mdx index e6ba483ee5..f528c9cdca 100644 --- a/calico-enterprise/reference/resources/kubecontrollersconfig.mdx +++ b/calico-enterprise/reference/resources/kubecontrollersconfig.mdx @@ -24,6 +24,8 @@ spec: syncLabels: Enabled hostEndpoint: autoCreate: Disabled + loadbalancer: + assignIPs: AllServices ``` ## Kubernetes controllers configuration definition @@ -77,6 +79,14 @@ The federated services controller syncs Kubernetes services from remote clusters | ---------------- | ---------------------------------------------------------------- | --------------------------------- | ------- | | reconcilerPeriod | Period to perform reconciliation with the $[prodname] datastore | [Duration string][parse-duration] | 5m | +### LoadBalancerController + +The load balancer controller manages IPAM for Services of type LoadBalancer. + +| Field | Description | Accepted Values | Schema | Default | +| ---------------- | ---------------------------------------------------------------- | ---------------------------------------- | ------ | ----------- | +| assignIPs | Mode in which LoadBalancer controller operates | AllServices, RequestedServicesOnly | String | AllServices | + ## Supported operations | Datastore type | Create | Delete (Global `default`) | Update | Get/List | Notes | diff --git a/calico/networking/configuring/advertise-service-ips.mdx b/calico/networking/configuring/advertise-service-ips.mdx index 58d7693bd6..1eeda8dfad 100644 --- a/calico/networking/configuring/advertise-service-ips.mdx +++ b/calico/networking/configuring/advertise-service-ips.mdx @@ -199,28 +199,30 @@ The following steps will configure $[prodname] to advertise Service `status.Load For help, see [BGP configuration resource](../../reference/resources/bgpconfig.mdx). -Service LoadBalancer address allocation is outside the current scope of $[prodname], but can be implemented with an external controller. -You can build your own, or use a third-party implementation like the MetalLB project. +$[prodname] IPAM can manage address allocation for Service of type LoadBalancer with LoadBalancerController. To enable $[prodname] IPAM to start managing addresses for Services set up an IPPool for LoadBalancer. -To install the MetalLB controller for allocating addresses, perform the following steps. - -1. Follow [the MetalLB documentation](https://metallb.universe.tf/installation/#installation-by-manifest) to install the `metallb-system/controller` resources. - - However, do not install the `metallb-system/speaker` component. The speaker component also attempts to establish BGP sessions on the node, and will conflict with Calico. +```bash +kubectl apply -f - < + diff --git a/calico/networking/ipam/service-loadbalancer.mdx b/calico/networking/ipam/service-loadbalancer.mdx new file mode 100644 index 0000000000..cd19b4f216 --- /dev/null +++ b/calico/networking/ipam/service-loadbalancer.mdx @@ -0,0 +1,120 @@ +--- +description: LoadBalancer IP address management +--- + +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + +# LoadBalancer IP address management + +## Understanding $[prodname] LoadBalancer IP address management + +You might want to utilize Service of type LoadBalancer in your cluster to provide stable long-lasting IP address to access your deployed application. +$[prodname] can help you with providing and managing IP addresses to your Services in your cluster. $[prodname] comes with LoadBalancer IPAM deployed as part of $[prodname] Kube-controllers. + +### Before you begin... + +Ensure that you have a cluster with $[prodname] installed, and kube-controllers configured and running. + +### IP Pool for Service of type LoadBalancer + +$[prodname] does not automatically provide default IP Pool for LoadBalancer IP address management. You will need to create an IP Pool with `allowedUses` LoadBalancer for $[prodname] to start assigning Service IPs. + +```yaml +apiVersion: projectcalico.org/v3 +kind: IPPool +metadata: + name: loadbalancerIPPool +spec: + cidr: 192.210.0.0/20 + blockSize: 24 + natOutgoing: true + disabled: false + assignmentMode: Automatic + allowedUses: + - LoadBalancer +``` + +:::note + +You can create multiple IP Pools with allowedUses LoadBalancer as long as there are no CIDR conflicts. By setting assignmentMode to Manual you can reserve the IP Pool for manual assignments only. Explore more about [manual assignment](#manual-service-ip-address-assignment) + +::: + +### Automatic Service IP address assignment + +When you create Service of type LoadBalancer $[prodname] kube-controller will automatically detect the new Service and assign an IP address from available IP Pool. If no address is available, the Service will remain in pending state until an address is available. + +```yaml +apiVersion: v1 +kind: Service +metadata: + name: service-loadbalancer +spec: + selector: + app: nginx + ports: + - port: 80 + targetPort: 80 + name: default + type: LoadBalancer +``` + +### Manual Service IP address assignment + +There are cases where you would like to be more specific about what IP address $[prodname] assigns to your Service. With annotations, you can specify how IP address should be assigned. The annotations can be added and removed as needed during the lifetime of the Service. When you remove an annotation $[prodname] kube-controller will check if the assigned IP is still valid and potentially assign a new one. + +#### Specify IP Pool + +##### IPv4 Pool + +Annotate the Service with projectcalico.org/ipv4pools to assign IP address from the selected IP Pools. You can specify multiple IP Pools in this annotation, $[prodname] will pick an available IP address to assign. + +``` +"projectcalico.org/ipv4pools": '["loadBalancerIPv4Pool"]' +``` + +##### IPv6 Pool + +Annotate the Service with projectcalico.org/ipv6pools to assign IP address from the selected IP Pools. You can specify multiple IP Pools in this annotation, $[prodname] will pick an available IP address to assign. + +``` +"projectcalico.org/ipv6pools": '["loadBalancerIPv6Pool"]' +``` + +:::note + +In dual-stack environment you don't have to specify both annotations. If IP Pool was not specified for IP family, $[prodname] will automatically assign available IP address from available IP Pool + +::: + +#### Specifying IP address + +Annotate the Service with projectcalico.org/loadBalancerIPs to assign specific IP address. The address must be available otherwise $[prodname] will not be able to assign the address. Currently you can only specify one IPv4 and one IPv6 address at the same time. + +``` +"projectcalico.org/loadBalancerIPs": '["x.x.x.x"]' +``` + +:::note + +If Service contains "projectcalico.org/loadBalancerIPs" annotation and either "projectcalico.org/ipv6pools" or "projectcalico.org/ipv4pools" are present, $[prodname] will favour projectcalico.org/loadBalancerIPs annotation and try to assign that IP address. There is no fall back to an IP Pool if the specified address is not available. + +::: + +### Manage LoadBalancer kube-controller assignment mode + +In certain cases you might not wish for $[prodname] to automatically assign IP addresses to your Service. This can be useful in case you have multiple Service IPAM solutions in your cluster. +LoadBalancer kube-controller is able to operate in two distinct modes ```Automatic``` in which $[prodname] assigns IP address to each Service in your cluster and ```RequestedServicesOnly``` in which $[prodname] only assigns IP addresses to Service with annotations mentioned above. +You can change the mode at any point, but note that switching to ```RequestedServicesOnly``` will unassign any addresses from Services that do not contain the above annotations. + +```bash +kubectl patch kubecontrollersconfiguration default --patch '{"spec": {"controllers":{"loadBalancer":{"AssignIPs": "RequestedServicesOnly"}}}}' +``` + +### Additional resources + +Calico LoadBalancer IP address management works in conjunction with other $[prodname] components. $[prodname] kube-controllers provides only the IP address management, to advertise LoadBalancer IPs you will have to update your BGP configuration. You can find out more information at: + * [Advertising Kubernetes service IP addresses](../../networking/configuring/advertise-service-ips.mdx#advertise-service-load-balancer-ip-addresses) + * [IP Pool](../../reference/resources/ippool.mdx) + * [Kube-controllers configuration](../../reference/resources/kubecontrollersconfig.mdx#loadbalancercontroller) diff --git a/calico/reference/calicoctl/ipam/check.mdx b/calico/reference/calicoctl/ipam/check.mdx index c48cfa5959..6ae6cdae39 100644 --- a/calico/reference/calicoctl/ipam/check.mdx +++ b/calico/reference/calicoctl/ipam/check.mdx @@ -17,13 +17,14 @@ Usage: calicoctl ipam check [--config=] [--show-all-ips] [--show-problem-ips] [-o ] Options: - -h --help Show this screen. - -o --output= Path to output report file. - --show-all-ips Print all IPs that are checked. - --show-problem-ips Print all IPs that are leaked or not allocated properly. - -c --config= Path to the file containing connection configuration in - YAML or JSON format. - [default: /etc/calico/calicoctl.cfg] + -h --help Show this screen. + -o --output= Path to output report file. + --show-all-ips Print all IPs that are checked. + --show-problem-ips Print all IPs that are leaked or not allocated properly. + -c --config= Path to the file containing connection configuration in + YAML or JSON format. + [default: /etc/calico/calicoctl.cfg] + --kubeconfig= Path to Kubeconfig file Description: The ipam check command checks the integrity of the IPAM datastructures against Kubernetes. diff --git a/calico/reference/resources/ippool.mdx b/calico/reference/resources/ippool.mdx index c282390c5a..3a34470080 100644 --- a/calico/reference/resources/ippool.mdx +++ b/calico/reference/resources/ippool.mdx @@ -25,6 +25,7 @@ spec: allowedUses: - Workload - Tunnel + assignmentMode: Automatic ``` ## IP pool definition @@ -43,11 +44,12 @@ spec: | blockSize | The CIDR size of allocation blocks used by this pool. Blocks are allocated on demand to hosts and are used to aggregate routes. The value can only be set when the pool is created. | 20 to 32 (inclusive) for IPv4 and 116 to 128 (inclusive) for IPv6 | int | `26` for IPv4 pools and `122` for IPv6 pools. | | ipipMode | The mode defining when IPIP will be used. Cannot be set at the same time as `vxlanMode`. | Always, CrossSubnet, Never | string | `Never` | | vxlanMode | The mode defining when VXLAN will be used. Cannot be set at the same time as `ipipMode`. | Always, CrossSubnet, Never | string | `Never` | -| natOutgoing | When enabled, packets sent from $[prodname] networked containers in this pool to destinations outside of any Calico IP pools will be masqueraded. | true, false | boolean | `false` | -| disabled | When set to true, $[prodname] IPAM will not assign addresses from this pool. | true, false | boolean | `false` | +| natOutgoing | When enabled, packets sent from $[prodname] networked containers in this pool to destinations outside of any Calico IP pools will be masqueraded. | true, false | boolean | `false` | +| disabled | When set to true, $[prodname] IPAM will not assign addresses from this pool. | true, false | boolean | `false` | | disableBGPExport _(since v3.21.0)_ | Disable exporting routes from this IP Pool’s CIDR over BGP. | true, false | boolean | `false` | -| nodeSelector | Selects the nodes that $[prodname] IPAM should assign addresses from this pool to. | | [selector](#node-selector) | all() | -| allowedUses _(since v3.21.0)_ | Controls whether the pool will be used for automatic assignments of certain types. See [below](#allowed-uses). | Workload, Tunnel | list of strings | `["Workload", "Tunnel"]` | +| nodeSelector | Selects the nodes that $[prodname] IPAM should assign addresses from this pool to. | | [selector](#node-selector) | all() | +| allowedUses _(since v3.21.0)_ | Controls whether the pool will be used for automatic assignments of certain types. See [below](#allowed-uses). | Workload, Tunnel, LoadBalancer | list of strings | `["Workload", "Tunnel"]` | +| assignmentMode | Controls whether the pool will be used for automatic assignments or only if requested manually | Automatic, Manual | strings | `Automatic` | :::note @@ -62,6 +64,12 @@ IP pool by creating blocks of incorrect size. When automatically assigning IP addresses to workloads, only pools with "Workload" in their `allowedUses` field are consulted. Similarly, when assigning IPs for tunnel devices, only "Tunnel" pools are eligible. +Combining options for the `allowedUses` field is limited. You can specify only the following options and option combinations: + * `allowedUses: ["Tunnel","Workload"]` (default) + * `allowedUses: ["Tunnel"]` + * `allowedUses: ["Workload"]` + * `allowedUses: ["LoadBalancer"]` + If the `allowedUses` field is not specified, it defaults to `["Workload", "Tunnel"]` for compatibility with older versions of Calico. It is not possible to specify a pool with no allowed uses. @@ -115,20 +123,15 @@ Increasing the block size from the default (e.g., using `24` for IPv4 to give 25 Reducing the block size from the default (e.g., using `28` for IPv4 to give 16 addresses per block) means more blocks per host and therefore potentially more routes. This can be beneficial if it allows the blocks to be more fairly distributed amongst the hosts. +### Assignment Mode + +Determines if the IP pool should be used by Calico IPAM for automatic IP assignments. With `Automatic` assignmentMode, Calico IPAM automatically assigns IP addresses from this pool. If you set `assignmentMode` to `Manual`, Calico IPAM does not assign IP addresses from this pool unless the user specifies this pool. + ### Node Selector For details on configuring IP pool node selectors, please read the [Assign IP addresses based on topology guide.](../../networking/ipam/assign-ip-addresses-topology.mdx). -:::tip - -To prevent an IP pool from being used automatically by $[prodname] IPAM, while still allowing -it to be used manually for static assignments, set the `IPPool`'s `nodeSelector` to `!all()`. Since the selector -matches no nodes, the IPPool will not be used automatically and, unlike setting `disabled: true`, it can still be -used for manual assignments. - -::: - #### Selector reference diff --git a/calico/reference/resources/kubecontrollersconfig.mdx b/calico/reference/resources/kubecontrollersconfig.mdx index a3ddcdd47f..686a66bde8 100644 --- a/calico/reference/resources/kubecontrollersconfig.mdx +++ b/calico/reference/resources/kubecontrollersconfig.mdx @@ -33,6 +33,8 @@ spec: reconcilerPeriod: 5m namespace: reconcilerPeriod: 5m + loadbalancer: + assignIPs: AllServices ``` ## Kubernetes controllers configuration definition @@ -117,4 +119,12 @@ $[prodname] datastore. | ---------------- | ---------------------------------------------------------------- | --------------------------------- | ------- | | reconcilerPeriod | Period to perform reconciliation with the $[prodname] datastore | [Duration string][parse-duration] | 5m | +### LoadBalancerController + +The load balancer controller manages IPAM for Services of type LoadBalancer. + +| Field | Description | Accepted Values | Schema | Default | +| ---------------- | ---------------------------------------------------------------- | ---------------------------------------- | ------ | ----------- | +| assignIPs | Mode in which LoadBalancer controller operates | AllServices, RequestedServicesOnly | String | AllServices | + [parse-duration]: https://golang.org/pkg/time/#ParseDuration diff --git a/sidebars-calico-cloud.js b/sidebars-calico-cloud.js index 14dd348941..efe213a38d 100644 --- a/sidebars-calico-cloud.js +++ b/sidebars-calico-cloud.js @@ -356,6 +356,7 @@ module.exports = { 'networking/ipam/migrate-pools', 'networking/ipam/change-block-size', 'networking/ipam/legacy-firewalls', + 'networking/ipam/service-loadbalancer' ], }, ], diff --git a/sidebars-calico-enterprise.js b/sidebars-calico-enterprise.js index d6e70773bc..75b3526ed9 100644 --- a/sidebars-calico-enterprise.js +++ b/sidebars-calico-enterprise.js @@ -206,6 +206,7 @@ module.exports = { 'networking/ipam/migrate-pools', 'networking/ipam/change-block-size', 'networking/ipam/legacy-firewalls', + 'networking/ipam/service-loadbalancer' ], }, ], diff --git a/sidebars-calico.js b/sidebars-calico.js index 0bf4d74297..2e73e82561 100644 --- a/sidebars-calico.js +++ b/sidebars-calico.js @@ -295,6 +295,7 @@ module.exports = { 'networking/ipam/migrate-pools', 'networking/ipam/change-block-size', 'networking/ipam/legacy-firewalls', + 'networking/ipam/service-loadbalancer' ], }, {