v2.1.0

Summary

The major theme of this kube-router release is to make kube-router fully conformant with the end-to-end upstream sig-network tests. These are tests that the Kubernetes project provides for testing significant networking features and becoming conformant with the tests means that the networking functionality of kube-router is more consistent with other network providers like kube-proxy, Calico, and Cilium.

Major New Features / Fixes

• kube-router's init container now installs CNI plugins when they are missing. The location for these plugins can be controlled by setting the environment variable HOST_BIN_PATH within the init container, but will default to /opt/cni/bin. This is something that all major Kubernetes networking providers do, but kube-router has not done in the past. For more information on CNI plugins, please see: https://github.com/containernetworking/plugins
• kube-router now implements .spec.healthCheckNodePort which has long been a part of the Kubernetes service specification, but kube-router hasn't implemented it until now. This port is meant to give visibility about whether or not an endpoint for a service exists on a node to workloads outside the Kubernetes cluster. If a service endpoint is on the node, then this port returns a 200 HTTP response, otherwise it returns a 503 HTTP response if no endpoint exists on the node. By choosing a node that contains an endpoint, a client can ensure source IP preservation.
• Hairpinning in kube-router is now fixed. kube-router now has a dedicated controller to ensuring that the veth interface inside the container is in promiscuous mode. This used to be handled by kubelet and then it was handled by docker-shim and then it was removed entirely. As far as I can tell, it was never ported to containerd or cri-o. Without this functionality, return traffic ends up getting black-holed before it is routed outside of the container. For more details see: 0f3714b
• Users can now specify --metrics-addr to choose the IP address that kube-router listens on when providing Prometheus metrics
• Users can now specify custom protocol timeouts for IPVS services exposed by kube-router
  • --service-tcp-timeout - (default: 0s preserves system value, typically 900 seconds)
  • --service-tcpfin-timeout - (default: 0s preserves system value, typically 120 seconds)
  • --service-udp-timeout - (default: 0s preserves system value, typically 300 seconds)
• kube-router now abides by the service label service.kubernetes.io/service-proxy-name. Setting this label to something other than kube-router will result in kube-router ignoring the service
• kube-router now honors spec.internalTrafficPolicy and implements spec.externalTrafficPolicy correctly. For more information see: https://kubernetes.io/docs/reference/networking/virtual-ips/#traffic-policies for more information

Backward Compatibility Breaks

kube-router now serves traffic to endpoints that that have a status of either ready or serving. See: https://kubernetes.io/docs/concepts/services-networking/endpoint-slices/#conditions for more information on the differences between these conditions. Before this release, kube-router only served traffic to ready endpoints, which improperly ignored pods that were in Terminating status. This change makes kube-router compliant with the upstream standards.

The release switches from Alpine base 3.18 to 3.19 which changes the iptables backend from iptables to nftables. This is a pretty big change, since the two backends are not mixable. The kube-router project recommends coupling this upgrade with a rolling restart of the nodes in your cluster so that the iptables rules get flushed alongside the upgrade.

Other Notes

• If you're running kube-router in a container and use hairpinning, make sure that hostPID: true is set on your kube-router pod. This is the only way that kube-router is able to properly set the veth interface into promiscuous mode. This attribute has now been added to all of the official kube-router DeaemonSet YAML files

Major Dependency Updates

This release also updates:

• Alpine base container from 3.18 to 3.19
• Go from v1.20 to v1.21
• Kubernetes from v1.28 to v1.29

Contributions

Thanks to:

• @nextsux - For improving overlay tunnel device binding
• @jpiper - For adding the ability to configure an IP to bind metrics to
• @mrueg - For always keeping our dependencies up to date
• @AlexisDucastel - For testing the CNI plugin installation functionality
• @hakman - For helping us improve the upstream e2e tests and providing kOps support and feedback

Changelog (from v2.0.1 - v2.1.0)

• a7d7ac5 - feat(alpine): update 3.18 -> 3.19 <Aaron U'Ren>
• 5b56e6c - feat(goreleaser): update v1.21.2 -> v1.24.0 <Aaron U'Ren>
• a648800 - feat(moq): update v0.3.2 -> v0.3.4 <Aaron U'Ren>
• 46a1b17 - feat(go): upgrade 1.20.13 -> 1.21.7 + dep update <Aaron U'Ren>
• c356289 - feat(lint): update markdownlint v0.37.0 -> 0.39.0 <Aaron U'Ren>
• 47fe189 - feat(lint): update golangci-lint and fix lint errors <Aaron U'Ren>
• 9a136c1 - feat(NSC): implement NodePort Health Check <Aaron U'Ren>
• 7aec8d0 - doc(NSC): add comment for hairpin controller <Aaron U'Ren>
• 959022f - feat(NSC): add endpoint statuses to internal struct <Aaron U'Ren>
• 16daa08 - feat(NSC): add endpoints that are ready or serving <Aaron U'Ren>
• efddb2e - Bump dependencies <Manuel Rüger>
• 3a1efa0 - dual_stack_issue_report.md: remove issue report template <Aaron U'Ren>
• dda7ce6 - doc(cni): CNI requirements and install process <Aaron U'Ren>
• a57a226 - feat(cni): add binaries to host when missing <Aaron U'Ren>
• 98eea79 - doc(user-guide.md): add info about traffic policies <Aaron U'Ren>
• fcd21b4 - feat: fully support service traffic policies <Aaron U'Ren>
• 8404260 - feat: increase unit test coverage <Aaron U'Ren>
• 24505f0 - fact(service_endpoints_sync.go): standardize error handling <Aaron U'Ren>
• d3cf4d1 - feat(NSC): add / clarify log messages <Aaron U'Ren>
• d757f49 - feat(NSC): honor headless label <Aaron U'Ren>
• 8afdee8 - fact(NSC): differentiate headless services <Aaron U'Ren>
• 30d3769 - fact(NSC): update Errorf syntax <Aaron U'Ren>
• a0fe844 - feat(NSC): honor service-proxy-name label <Aaron U'Ren>
• ced5102 - feat(NSC): add IPVS service timeouts <Aaron U'Ren>
• 47290a7 - build(deps): bump k8s.io/client-go from 0.28.3 to 0.28.4 <dependabot[bot]>
• 27f2283 - build(deps): bump k8s.io/cri-api from 0.28.3 to 0.28.4 <dependabot[bot]>
• a6de8cb - build(deps): bump github.com/aws/aws-sdk-go from 1.48.10 to 1.48.16 <dependabot[bot]>
• eb462ba - feat(linux_networking.go): add more error info <Aaron U'Ren>
• eb26288 - fix(manifests): add hostPID for hairpin <Aaron U'Ren>
• aebaa48 - fix(NSC): handle endpoint slice ready nil <Aaron U'Ren>
• 0f3714b - fix(hairpin): set hairpin_mode for veth iface <Aaron U'Ren>
• 3ea293d - build(deps): bump github.com/aws/aws-sdk-go from 1.47.9 to 1.48.10 <dependabot[bot]>
• 4b863e9 - build(deps): bump k8s.io/api from 0.28.3 to 0.28.4 <dependabot[bot]>
• f575d31 - build(deps): bump github.com/osrg/gobgp/v3 from 3.20.0 to 3.21.0 <dependabot[bot]>
• 724588c - build(deps): bump golang.org/x/net from 0.18.0 to 0.19.0 <dependabot[bot]>
• 448f8b6 - build(deps): bump github.com/aws/aws-sdk-go from 1.45.24 to 1.47.9 <dependabot[bot]>
• 0209ac5 - build(deps): bump google.golang.org/grpc from 1.58.3 to 1.59.0 <dependabot[bot]>
• b00a997 - build(deps): bump github.com/onsi/gomega from 1.29.0 to 1.30.0 <dependabot[bot]>
• 595d3e4 - build(deps): bump golang.org/x/net from 0.17.0 to 0.18.0 <dependabot[bot]>
• fcf0ad9 - prometheus metrics: add option to specify listen address <Jason Piper>
• 961c63b - build(deps): bump github.com/onsi/gomega from 1.27.10 to 1.29.0 <dependabot[bot]>
• b13cedc - build(deps): bump k8s.io/client-go from 0.28.2 to 0.28.3 <dependabot[bot]>
• 49314ab - build(deps): bump k8s.io/cri-api from 0.28.2 to 0.28.3 <dependabot[bot]>
• e0861fc - build(deps): bump github.com/osrg/gobgp/v3 from 3.19.0 to 3.20.0 <dependabot[bot]>
• 44d8108 - build(deps): bump github.com/docker/docker <dependabot[bot]>
• 66890d5 - feat: Disable binding overlay tunnels to specific device <Martin -nexus- Mlynář>
• f41defc - build(deps): bump google.golang.org/grpc from 1.58.2 to 1.58.3 <dependabot[bot]>
• 9d6b647 - doc(user-guide): remove bootkube, add k3s <Aaron U'Ren>
• e8be179 - doc(RELEASE.md): add more information <Aaron U'Ren>
• a00778e - feat(.goreleaser.yml): add prerelease and header to release <Aaron U'Ren>
• eb30de9 - lint: add markdownlint to linting <Aaron U'Ren>
• dad4991 - doc: update and format all documentation <Aaron U'Ren>
• 6442689 - doc(user-guide.md): update formatting <Aaron U'Ren>
• dbd741a - doc(introduction.md): update and format <Aaron U'Ren>
• 4379104 - doc(generic.md): update and format <Aaron U'Ren>
• 1957907 - doc(kubeadm.md): update and format <Aaron U'Ren>

v2.0.1

Summary

Fixes a major bug for single stack systems in the service proxy controller. Additionally adds a warning between release v2.0.0 - v2.1.0 about breaking changes.

Changelog

• b36775a - feat(version): add version message mechanism <Aaron U'Ren>
• 503f9fe - fix(NSC): only run for enabled families <Aaron U'Ren>
• 9b0c575 - build(deps): bump golang.org/x/net from 0.16.0 to 0.17.0 <dependabot[bot]>
• 8ccd687 - fix(Makefile): version and build date for v2 <Aaron U'Ren>

v2.0.0

Summary

kube-router v2.0.0 is a major release that the project has been working on for almost a year. We're very excited to finally enable some long-requested features.

Its worth noting, that almost 50% of the existing code base had to be modified in order to bring this release to fruition. We've done testing via the several release candidates that we've released over the last few months (huge thanks to all of the users that engaged that by submitting fixes or reporting bugs), but I'm sure that there are still a few out there that we haven't caught yet.

The project recommends that users approach deploying this release to their production critical clusters with caution. If you do find a bug, please open a new GitHub Issue with the Report a v2.0.0 Release Issue issue type so that we can resolve it as soon as possible.

Major New Features

• Dual-Stack implemented in all kube-router controllers (e.g. --run-firewall, --run-router, --run-loadbalancer, & --run-service-proxy)
• IPAM management for LoadBalancer service types (previously kube-router would only perform BGP, firewall, and proxy functionality and users would need to supplement kube-router with something like MetalLB to get IPs from a pool for their LoadBalancer services)
• FoU tunneling capability added for pod networking (previously only ipip was supported)

Backward Compatibility Breaks

You'll also notice that this has been tagged as a major release version. This is due to the fact that there are breaking changes in this version of kube-router that are not backwards compatible with previous versions of kube-router.

Tunnel names have changed, so if you run kube-router with an overlay network (ipip tunnels), you'll want to deploy this update carefully. At this point, the project recommends doing a rolling-reboot of nodes after the application (or as part of) of this release to ensure that unused tunnels are properly cleaned up and don't have a negative impact on traffic flows.

As part of this work, kube-router now needs to be able to get, list, and watch EndpointSlice objects. Please make sure to update your ClusterRole RBAC before deploying this release if you use kube-router with the --run-service-proxy functionality enabled or you'll get an error. For examples of this RBAC please see the daemonset examples.

Other Notes

Dual-stack handling is fairly complex and it comes with a few gotchas. Users that want to enable dual-stack features should carefully read the IPv6 / Dual-Stack Support in kube-router documentation to ensure that they fully understand the feature set, requirements, and limitations.

If you want to run the new load balancer controller (e.g. --run-loadbalancer) kube-router will need some additional RBAC and requires a few environment variables set. See load-balancer allocation docs for more information.

For more information on enabling or migrating to FoU tunneling and any caveats on doing so, please see the Tunnels in kube-router documentation.

Now that kube-router has dual-stack capability, it doesn't make sense to have an annotation that can only represent a single pod CIDR any longer. As such, with this release we are announcing the deprecation of the kube-router.io/pod-cidr annotation in favor of the new kube-router.io/pod-cidrs annotation see the IPv6 / Dual-Stack Support in kube-router documentation for more information.

Major Dependency Updates

This release also updates:

• Alpine base container from 3.17 to 3.18
• Go from v1.19 to v1.20
• Kubernetes from v1.25 to v1.27
• GoBGP from v3.11.0 to v3.17.0.

Contributions

A release this big would not be possible without an awesome community. Thanks so much to all of the following people that helped play a part in this release: @brandond, @iggy, @whooo, @BoleynSu, @k-raval, @rwagoner, @vadorovsky, @thomasferrandiz, and @mrueg

Changelog (from v1.6.0 - v2.0.0)

1c89be8 - feat(go): upgrade 1.20.7 -> 1.20.9 <Aaron U'Ren>
3db03cc - feat(dep): upgrade all dependencies <Aaron U'Ren>
d2c3858 - feat(goreleaser): upgrade v1.14.1 -> v1.21.2 <Aaron U'Ren>
1a4896f - feat(lint): upgrade golangci-lint v1.50.1 -> v1.54.2 <Aaron U'Ren>
678b712 - fix(ecmp_vip.go): non-local service advertisement <Aaron U'Ren>
1a891c3 - fix(dsr): add family specific link inside pod <Aaron U'Ren>
514a8af - fix(dsr): add family for fwmark <Aaron U'Ren>
c92f76a - fix(service_endpoints_sync.go): use save command <Aaron U'Ren>
9abe20d - fix(NSC): compare all pod IPs for endpoint check <Aaron U'Ren>
9f23cf5 - fix(linux_networking.go): add better error messages <Aaron U'Ren>
7ce09a6 - fix(linux_networking.go): don't return err on warn <Aaron U'Ren>
9d63cc6 - feat(debug): add some extra debug at level 3 <Aaron U'Ren>
4c6e19f - feat(ipset): consolidate ipset usage across controllers <Aaron U'Ren>
c62e1b7 - feat(linux_networking.go): add more logging info <Aaron U'Ren>
da73dea - feat(NSC): use EndpointSlice instead of Endpoints <Aaron U'Ren>
15cd4eb - feat(nsc): add more insight into sync steps <Aaron U'Ren>
81bc9e2 - fix(nsc): don't modify netmask during flag setup <Aaron U'Ren>
903466b - fix(nsc): fail fast during init <Aaron U'Ren>
25ecb09 - feat(nsc): add dualstack capabilities <Aaron U'Ren>
f397a1f - feat: increase log level for save/restore msgs <Aaron U'Ren>
a6cd5b0 - lint(metrics_controller.go): reduce line length <Aaron U'Ren>
68a7d03 - fix: take family metrics out of defer <Aaron U'Ren>
301e856 - fix(NPC): remove redundant assign <Aaron U'Ren>
b06b4f0 - Move ipset restore outside policy loop <Brad Davidson>
e34ef29 - Add additional save/restore metrics <Brad Davidson>
aa107d6 - Make metrics registerer/gathererer replacable <Brad Davidson>
e6f668c - fix: syntax updates for Go 1.20.X and k8s 1.27 <Aaron U'Ren>
5cf1265 - fix(NRC): prevent adding routes with mixed families <Aaron U'Ren>
bab0d4f - feat(bgp_policies.go): don't override-nexthop for internal peers <Aaron U'Ren>
afdf553 - add loadbalancer address allocator <Erik Larsson>
7699d16 - doc(tunnels.md): add info about changing live clusters <Aaron U'Ren>
ddf857d - doc(tunnel): add information about tunnels <Aaron U'Ren>
944ab91 - fix(FoU): make more robust <Aaron U'Ren>
bac4ae6 - fix(FoU): add docs, sanity checking, and logic reduction <Aaron U'Ren>
2a57d6c - Adding FoU encapsulation over IPIP tunnel : added checks for restart and multi-node cases <Kartik Raval>
6ce37e6 - Support for FoU encapsulation for IPIP tunnel <Kartik Raval>
4861021 - fix(NPC): update IPBlocks to be ipFamily specific <Aaron U'Ren>
f0d7f1e - netpol: Fix ipset only containing one IP when port name is used. <Boleyn Su>
384ed97 - fix(bgp_policy): allow for statement add / remove <Aaron U'Ren>
1d5c9ce - fix(ecmp_vip): update VIPs based on svc change <Aaron U'Ren>
f5ac980 - fix(bgp_policies.go): return -> continue on family set evaluation <Aaron U'Ren>
3387f5c - use JoinHostPort for GRPC listen address <Erik Larsson>
76ffcbd - add generation of router id based on hash of primary IP <Erik Larsson>
57c9b08 - fix(ecmp_vip.go): ClusterIP -> ClusterIPs <Aaron U'Ren>
fe93978 - feat(bgp_policies_test.go): use different IP ranges <Aaron U'Ren>
31c22ff - fix(bgp_policies.go): don't get BGP peers twice <Aaron U'Ren>
06f5f8b - feat(go): update package version to /v2 <Aaron U'Ren>
e51ee3a - fix(NPC): add warning for unsupported family <Aaron U'Ren>
4e1679f - fix(NPC): don't add chains for missing family <Aaron U'Ren>
a2bb2ba - doc(bgp.md): clean up grammar and syntax <Aaron U'Ren>
0ecb51d - fix(NPC/pod): check drop policy on ipv4 & ipv6 <Aaron U'Ren>
367aedf - fix(bgp_policies): add empty DS set checking <Aaron U'Ren>
aeb51ba - fact(bgp_policies): rename clusterIPPrefixSet -> serviceVIPIPPrefixSet <Aaron U'Ren>
6e03836 - fact(bgp_policies): abstract get DS for GoBGP <Aaron U'Ren>
67254ad - fix(ecmp_vip): handle ipv4 & ipv6 protocols <Aaron U'Ren>
5f952e0 - test(bgp_policies_test): add local address <Aaron U'Ren>
ec12fda - fix(node): do nil checking on FindBestIP util funcs <Aaron U'Ren>
5d7f62c - fix(NRC): ensure local addr IP is bindable early <Aaron U'Ren>
67abc4b - fix(bgp_peers): adv. AfiSafi based on capabability <Aaron U'Ren>
c491bcb - fix(bgp_peers): do peer only if IP protos match <Aaron U'Ren>
0023ded - fix(NRC): error when nec. host IP not found <Aaron U'Ren>
4f284be - fix(NRC): add IPv6 logic to bgp-local-addresses <Aaron U'Ren>
51f4696 - feat(ci): run CI on version prep branches and MRs <Aaron U'Ren>
b3e0768 - fix(options): make clusterIP specification similar to other options <Aaron U'Ren>
a31511d - fix(NPC): actually separate chain indices for ipv4 / ipv6 <Aaron U'Ren>
096da81 - fact(NPC): pluralize newIPTablesHandler <Aaron U'Ren>
ddb0e63 - feat(NRC): make NRC dual stack <Aaron U'Ren>
01f2ff2 - fact(NRC): convert BGP set names to const <Aaron U'Ren>
85cecb6 - feat(pod_cidr): handle multiple pod CIDRs <Aaron U'Ren>
c18d811 - fix(kube-router.go): metric message -> not error <Aaron U'Ren>
3db482b - fix(NPC): separate chain indices for ipv4 / ipv6 <Aaron U'Ren>
9d2b3c7 - fix(node.go): make node address errors more helpful <Aaron U'Ren>
d7e2a14 - fix golangci issues <Thomas Ferrandiz>
12561f9 - fix test co...

v2.0.0-rc7

Summary

The big headline of this release is support for dual-stack capabilities in the Network Services Controller (--proxy option). This was the last major outstanding feature milestone that was left to round out the v2.0.0 release. Now the entire kube-router code base has support for dual-stack capabilities.

As part of this work, kube-router now needs to be able to get, list, and watch EndpointSlice objects. Please make sure to update your ClusterRole RBAC before deploying this release if you use kube-router with the --proxy functionality enabled or you'll get an error. For examples of this RBAC please see the daemonset examples from the prep-v2.0 branch.

I want to make this release candidate available for the community to test for the next 1 - 2 weeks, and get a chance to resolve any open issues. After this, if the release candidate is stable, I'll merge the prep-v2.0 branch to master and make an official v2.0.0 release. This release has been a long time in coming and I'm excited to get it out there! A huge thanks to everyone that has helped test it and contributed to it!

Additionally, this release fixed a significant performance regression with Network Policies that was introduced in the v2.0.0 codebase during dual-stack implementation.

This release also updates Go from 1.19 to 1.20 and adds support for Kubernetes v1.27. We also upgraded our Alpine base version from 3.17 to 3.18 and GoBGP from v3.11.0 to v3.17.0.

Contributions

Big thanks goes to @brandond (and the k3s project) for finding and fixing the Network Policy performance issues and also adding additional metrics to the Network Policy Controller for increased observability.

Thanks @iggy for adding RISC-V 64 support for kube-router binaries. We are not yet building official container images for RISC-V 64 because Alpine doesn't yet officially support it, but when they do we'll enable container builds for RISC-V64 as well.

Changelog

65ac87d - fix(ecmp_vip.go): non-local service advertisement <Aaron U'Ren>
d780687 - fix(dsr): add family specific link inside pod <Aaron U'Ren>
907565d - fix(dsr): add family for fwmark <Aaron U'Ren>
9e86535 - fix(service_endpoints_sync.go): use save command <Aaron U'Ren>
4761bbb - fix(NSC): compare all pod IPs for endpoint check <Aaron U'Ren>
6758598 - fix(linux_networking.go): add better error messages <Aaron U'Ren>
b2d2b59 - fix(linux_networking.go): don't return err on warn <Aaron U'Ren>
d2ae639 - feat(debug): add some extra debug at level 3 <Aaron U'Ren>
a3c5694 - feat(ipset): consolidate ipset usage across controllers <Aaron U'Ren>
854440d - feat(linux_networking.go): add more logging info <Aaron U'Ren>
327e894 - feat(NSC): use EndpointSlice instead of Endpoints <Aaron U'Ren>
f8d9812 - feat(nsc): add more insight into sync steps <Aaron U'Ren>
b03f3b7 - fix(nsc): don't modify netmask during flag setup <Aaron U'Ren>
859658a - fix(nsc): fail fast during init <Aaron U'Ren>
d425077 - feat(nsc): add dualstack capabilities <Aaron U'Ren>
56cd602 - feat: increase log level for save/restore msgs <Aaron U'Ren>
2d4ad45 - lint(metrics_controller.go): reduce line length <Aaron U'Ren>
a1db517 - fix: take family metrics out of defer <Aaron U'Ren>
fc34ae5 - fix(NPC): remove redundant assign <Aaron U'Ren>
bbcedc8 - Move ipset restore outside policy loop <Brad Davidson>
92201bb - Add additional save/restore metrics <Brad Davidson>
df7905b - Make metrics registerer/gathererer replacable <Brad Davidson>
d3ecb4e - fix: syntax updates for Go 1.20.X and k8s 1.27 <Aaron U'Ren>
08f05a8 - fix(moq): chown generated moqs <Aaron U'Ren>
ee85441 - fix(moq): remove previous moqs before generation <Aaron U'Ren>
4556aa3 - fix(lint): convert sh -> bash <Aaron U'Ren>
3c16d3c - feat(close_stale.yml): increase operations 30 -> 100 <Aaron U'Ren>
0d12e61 - feat(close_stale.yml): add workflow for closing stale issues & PRs <Aaron U'Ren>
b084c2a - fix: available typo <guoguangwu>
82cd7c9 - build(deps): bump github.com/aws/aws-sdk-go from 1.44.334 to 1.45.1 <dependabot[bot]>
16474cf - build(deps): bump github.com/aws/aws-sdk-go from 1.44.332 to 1.44.334 <dependabot[bot]>
0110d1d - Partially revert riscv64 support <Manuel Rüger>
9805c79 - Makefile: Bump go + alpine as well <Manuel Rüger>
ade98d6 - Bump go 1.20 + k8s 1.27 <Manuel Rüger>
e7a521a - feat(ci.yml): add ppc64le and riscv64 to ci <Aaron U'Ren>
8b76e9c - Add RISC-V 64 support <Iggy Jackson>
70defa4 - build(deps): bump github.com/aws/aws-sdk-go from 1.44.313 to 1.44.332 <dependabot[bot]>
8de9b64 - build(deps): bump golang.org/x/net from 0.12.0 to 0.14.0 <dependabot[bot]>

v2.0.0-rc6

Summary

Adds a new controller for kube-router which allows it to do IP address allocation for load-balancer type services. Previously, one had to use something like MetalLB for this functionality, now kube-router is able to handle this itself. See load-balancer allocation docs for more information.

Contributions

Huge thanks to @whooo for contributing the load-balancer allocation code

Changelog

55efa33 - fix(NRC): prevent adding routes with mixed families <Aaron U'Ren>
7700915 - feat(bgp_policies.go): don't override-nexthop for internal peers <Aaron U'Ren>
c94ff7c - add loadbalancer address allocator <Erik Larsson>

v2.0.0-rc5

Summary

Updates dependencies and incorporates a bug fix from the mainline branch where advertisement annotations were not being evaluated correctly.

Changelog

• 099664a build(deps): bump github.com/aws/aws-sdk-go from 1.44.309 to 1.44.313 <dependabot[bot]>
• 9372d62 build(deps): bump github.com/osrg/gobgp/v3 from 3.16.0 to 3.17.0 <dependabot[bot]>
• c50bdfc build(deps): bump github.com/onsi/gomega from 1.27.7 to 1.27.10 <dependabot[bot]>
• 98479d8 build(deps): bump github.com/aws/aws-sdk-go from 1.44.308 to 1.44.309 <dependabot[bot]>
• 64784c4 build(deps): bump google.golang.org/grpc from 1.56.2 to 1.57.0 <dependabot[bot]>
• b900acb github: Add github-actions dependabot config <Manuel Rüger>
• fb37a64 build(deps): bump golang.org/x/net from 0.10.0 to 0.12.0 <dependabot[bot]>
• a42466a build(deps): bump google.golang.org/grpc from 1.56.1 to 1.56.2 <dependabot[bot]>
• 371a3a8 build(deps): bump github.com/osrg/gobgp/v3 from 3.14.0 to 3.16.0 <dependabot[bot]>
• 09940db build(deps): bump github.com/aws/aws-sdk-go from 1.44.273 to 1.44.308 <dependabot[bot]>
• 5d39fe0 build(deps): bump github.com/prometheus/client_golang <dependabot[bot]>
• 68e0fe5 build(deps): bump k8s.io/cri-api from 0.27.2 to 0.27.4 <dependabot[bot]>
• 1701f9c build(deps): bump github.com/stretchr/testify from 1.8.2 to 1.8.4 <dependabot[bot]>
• d5fcc78 build(deps): bump github.com/docker/docker <dependabot[bot]>
• 7a3a495 build(deps): bump google.golang.org/protobuf from 1.30.0 to 1.31.0 <dependabot[bot]>
• dd3b75a build(deps): bump github.com/docker/docker <dependabot[bot]>
• 5d71897 fix(NRC): withdraw advertised VIPs based on annotation <Aaron U'Ren>

v1.6.0

Summary

It has been a long time since we've done a minor release from the main stable development branch (May of 2022 if you can believe it). This release is less about features and more about bringing a bunch of dependency updates into a stable release.

Contributions

Special thanks to @juchem, @KSauter, & @tamihiro for their contributions on this release!

Changelog

Excludes bugfixes that were already present on the 1.5.X release line

• 5d71897 - fix(NRC): withdraw advertised VIPs based on annotation <Aaron U'Ren>
• 7db274e - feat(ci): specify GitHub actions cache <Aaron U'Ren>
• 10417cc - fact(Makefile): make spacing consistent <Aaron U'Ren>
• f3dc9b3 - feat(ci): use caching for build, test, lint <Aaron U'Ren>
• 168a2b4 - fix(ci): checkout before go setup <Aaron U'Ren>
• 39ff9f6 - build(deps): bump google.golang.org/grpc from 1.55.0 to 1.56.1 <dependabot[bot]>
• c22c6af - fix(ci.yml): remove deprecated goreleaser flag <Aaron U'Ren>
• 0b267b0 - feat(ci.yml): update github action versions <Aaron U'Ren>
• ca18201 - build(deps): bump github.com/aws/aws-sdk-go from 1.44.266 to 1.44.273 <dependabot[bot]>
• 4013c00 - build(deps): bump github.com/containernetworking/plugins <dependabot[bot]>
• 4a5014d - build(deps): bump k8s.io/cri-api from 0.27.1 to 0.27.2 <dependabot[bot]>
• a026c8d - build(deps): bump google.golang.org/grpc from 1.54.0 to 1.55.0 <dependabot[bot]>
• 7559745 - build(deps): bump github.com/prometheus/client_golang <dependabot[bot]>
• 44327b8 - build(deps): bump github.com/aws/aws-sdk-go from 1.44.246 to 1.44.266 <dependabot[bot]>
• 438b1ed - build(deps): bump github.com/onsi/gomega from 1.27.6 to 1.27.7 <dependabot[bot]>
• bb82f55 - build(deps): bump k8s.io/klog/v2 from 2.80.1 to 2.100.1 <dependabot[bot]>
• cc70e7c - build(deps): bump github.com/docker/distribution <dependabot[bot]>
• 60e304f - build(deps): bump github.com/osrg/gobgp/v3 from 3.13.0 to 3.14.0 <dependabot[bot]>
• ef4a806 - go.mod: Bump dependencies <Manuel Rüger>
• 2b4087e - build(deps): bump github.com/docker/docker <dependabot[bot]>
• e1825ed - build(deps): bump golang.org/x/net from 0.5.0 to 0.7.0 <dependabot[bot]>
• ef77a2d - Fix lint <Manuel Rüger>
• 809f2fb - Update dependencies <Manuel Rüger>
• 4c751b0 - Register BGP sent metric <Kevin Sauter>
• 4c7ca8a - Add sent metric to vip bgp announcement. To distinguish between the different sent counters, the new "type" label can be used. <Kevin Sauter>
• 1d1ff05 - fix(NSC): add check for podCidr before use <Aaron U'Ren>
• 240cac2 - doc(ipv6): add additional information <Aaron U'Ren>
• 27f1d92 - doc(ipv6.md): incorporate review feedback <Aaron U'Ren>
• e446467 - feat(dual_stack_bug_report): add new issue template <Aaron U'Ren>
• e1b1a31 - doc(ipv6.md): update for increased dual-stack support <Aaron U'Ren>
• e2f0f18 - build(deps): bump github.com/vishvananda/netns from 0.0.2 to 0.0.3 <dependabot[bot]>
• 787e3b3 - build(deps): bump github.com/containernetworking/plugins <dependabot[bot]>
• 17a3df9 - build(deps): bump github.com/onsi/gomega from 1.24.1 to 1.25.0 <dependabot[bot]>
• 435f640 - build(deps): bump github.com/moby/ipvs from 1.0.2 to 1.1.0 <dependabot[bot]>
• 3007ada - build(deps): bump github.com/docker/docker <dependabot[bot]>
• 5df539d - build(deps): bump golang.org/x/net from 0.4.0 to 0.5.0 <dependabot[bot]>
• c367cc2 - build(deps): bump github.com/osrg/gobgp/v3 from 3.9.0 to 3.10.0 <dependabot[bot]>
• 31dd271 - build(deps): bump github.com/aws/aws-sdk-go from 1.44.157 to 1.44.171 <dependabot[bot]>
• bfdbfbc - build(deps): bump github.com/docker/docker <dependabot[bot]>
• 47a1045 - build(deps): bump github.com/stretchr/testify from 1.8.0 to 1.8.1 <dependabot[bot]>
• 3e2af39 - build(deps): bump github.com/aws/aws-sdk-go from 1.44.150 to 1.44.157 <dependabot[bot]>
• c37d96f - build(deps): bump google.golang.org/grpc from 1.50.1 to 1.51.0 <dependabot[bot]>
• 557794a - build(deps): bump github.com/osrg/gobgp/v3 from 3.7.0 to 3.9.0 <dependabot[bot]>
• faa52b8 - build(deps): bump golang.org/x/net from 0.2.0 to 0.4.0 <dependabot[bot]>
• 8d24611 - build(deps): bump github.com/prometheus/client_golang <dependabot[bot]>
• c110f98 - build(deps): bump github.com/aws/aws-sdk-go from 1.44.146 to 1.44.150 <dependabot[bot]>
• a06a6a8 - build(deps): bump github.com/onsi/gomega from 1.23.0 to 1.24.1 <dependabot[bot]>
• 31154f6 - build(deps): bump github.com/aws/aws-sdk-go from 1.44.138 to 1.44.146 <dependabot[bot]>
• 7137ca3 - README.md: remove kube-router.io link <Aaron U'Ren>
• 9c23ede - build(deps): bump github.com/aws/aws-sdk-go from 1.44.124 to 1.44.138 <dependabot[bot]>
• c41ec7a - build(deps): bump github.com/onsi/gomega from 1.22.1 to 1.23.0 <dependabot[bot]>
• 8873765 - build(deps): bump github.com/docker/docker <dependabot[bot]>
• a7a462b - build(deps): bump github.com/aws/aws-sdk-go from 1.44.116 to 1.44.124 <dependabot[bot]>
• f102bc5 - .github: Fix name of the RC step <Manuel Rüger>
• e5336c8 - .github: Allow tagging RCs without updating :latest <Manuel Rüger>
• a5e6ed0 - .github: Update github actions <Manuel Rüger>
• 0813b76 - feat(Makefile): make local builds behave like CI <Aaron U'Ren>
• efd1001 - fix invalid MTU in CNI config file <Tamihiro Lee>
• 24f8734 - doc(user-guide.md): add info for netfilter tooling <Aaron U'Ren>
• 5e2e225 - build(deps): bump github.com/onsi/gomega from 1.20.2 to 1.22.1 <dependabot[bot]>
• b45c2cc - build(deps): bump github.com/osrg/gobgp/v3 from 3.5.0 to 3.7.0 <dependabot[bot]>
• ba5561e - Update go dependencies <Manuel Rüger>
• 2d82195 - build(deps): bump github.com/aws/aws-sdk-go from 1.44.89 to 1.44.109 <dependabot[bot]>
• cbea4bb - build(deps): bump github.com/docker/docker <dependabot[bot]>
• 31b029a - build(deps): bump k8s.io/klog/v2 from 2.70.1 to 2.80.1 <dependabot[bot]>
• 35c7aa3 - build(deps): bump github.com/onsi/gomega from 1.20.0 to 1.20.2 <dependabot[bot]>
• 71a61f3 - build(deps): bump github.com/aws/aws-sdk-go from 1.44.70 to 1.44.89 <dependabot[bot]>
• df329c2 - build(deps): bump google.golang.org/grpc from 1.48.0 to 1.49.0 <dependabot[bot]>
• 39116b7 - logging ipset/iptables commands <Marcelo Juchem>
• 9eccc04 - build(deps): bump k8s.io/cri-api from 0.24.3 to 0.24.4 <dependabot[bot]>
• 951f801 - build(deps): bump k8s.io/client-go from 0.24.3 to 0.24.4 <dependabot[bot]>
• 7b55e9c - build(deps): bump github.com/osrg/gobgp/v3 from 3.2.0 to 3.5.0 <dependabot[bot]>
• 818fba2 - build(deps): bump github.com/prometheus/client_golang <dependabot[bot]>
• 32f6b1e - build(deps): bump github.com/aws/aws-sdk-go from 1.44.66 to 1.44.70 <dependabot[bot]>
• 48fe83d - build(deps): bump github.com/containernetworking/cni from 1.1.1 to 1.1.2 <dependabot[bot]>
• e1134d4 - build(deps): bump github.com/onsi/gomega from 1.19.0 to 1.20.0 <dependabot[bot]>
• 118270f - build(deps): bump google.golang.org/protobuf from 1.28.0 to 1.28.1 <dependabot[bot]>
• 6aa7edf - build(deps): bump github.com/aws/aws-sdk-go from 1.44.56 to 1.44.66 <dependabot[bot]>
• 836b49e - build(deps): bump k8s.io/cri-api from 0.24.2 to 0.24.3 <dependabot[bot]>
• a5dc081 - build(deps): bump k8s.io/client-go from 0.24.2 to 0.24.3 <dependabot[bot]>
• a111738 - build(deps): bump google.golang.org/grpc from 1.47.0 to 1.48.0 <dependabot[bot]>
• aa57b83 - build(deps): bump k8s.io/klog/v2 from 2.70.0 to 2.70.1 <dependabot[bot]>
• c9c3e8d - build(deps): bump github.com/aws/aws-sdk-go from 1.44.46 to 1.44.56 <dependabot[bot]>
• 0c366c1 - build(deps): bump github.com/stretchr/testify from 1.7.4 to 1.8.0 <dependabot[bot]>
• 2ab1462 - build(deps): bump github.com/aws/aws-sdk-go from 1.44.38 to 1.44.46 <dependabot[bot]>
• f452e77 - build(deps): bump k8s.io/klog/v2 from 2.60.1 to 2.70.0 <dependabot[bot]>
• c93178d - build(deps): bump github.com/stretchr/testify from 1.7.1 to 1.7.4 <dependabot[bot]>
• 5d74933 - build(deps): bump k8s.io/client-go from 0.24.1 to 0.24.2 <dependabot[bot]>
• 378c8e6 - build(deps): bump k8s.io/cri-api from 0.24.1 to 0.24.2 <dependabot[bot]>
• e0b3728 - build(deps): bump github.com/docker/docker <dependabot[bot]>
• 5ac6f32 - build(deps): bump google.golang.org/protobuf from 1.27.1 to 1.28.0 <dependabot[bot]>
• 1f3ece8 - build(deps): bump github.com/aws/aws-sdk-go from 1.44.27 to 1.44.38 <dependabot[bot]>
• 88266bc - fix(gobgp): update binary in container image to v3.2.0 <Aaron U'Ren>
• 2c5bb4c - feat(gobgp): update to v3.2.0 <Aaron U'Ren>
• e370cb0 - gobgp: update to 3.X <Aaron U'Ren>
• 94158b9 - build(deps): bump google.golang.org/grpc from 1.46.2 to 1.47.0 <dependabot[bot]>
• 15e166c - build(deps): bump github.com/aws/aws-sdk-go from 1.44.24 to 1.44.27 <dependabot[bot]>
• a75a1fd - build(deps): bump github.com/containernetworking/cni from 1.1.0 to 1.1.1 <dependabot[bot]>
• 0b4d1e0 - build(deps): bump github.com/moby/ipvs from 1.0.1 to 1.0.2 <dependabot[bot]>
• be84ceb - .github/workflows/codeql-analysis.yml: Update to v2 <Manuel Rüger>

v2.0.0-rc4

Summary

This release updates dependencies (including a switch to using Go 1.20 and Alpine 3.18), addresses some problems with IPv6 ipsets when port names are used, and adds FoU tunneling for environments that cannot support IPIP tunnels directly (cough Azure cough).

For more information on enabling FoU tunneling and any caveats on doing so, please see: https://github.com/cloudnativelabs/kube-router/blob/prep-v2.0/docs/tunnels.md

Contributions

Special thanks to @whooo, @BoleynSu, & @k-raval for their contributions to this release!

Changelog

• cbdcf31 - Use alpine linux 3.18 for docker image <Erik Larsson>
• 22b4718 - doc(tunnels.md): add info about changing live clusters <Aaron U'Ren>
• 1863d54 - doc(tunnel): add information about tunnels <Aaron U'Ren>
• b14d930 - fix(FoU): make more robust <Aaron U'Ren>
• 97a6a8f - fix(FoU): add docs, sanity checking, and logic reduction <Aaron U'Ren>
• 55dc26b - Adding FoU encapsulation over IPIP tunnel : added checks for restart and multi-node cases <Kartik Raval>
• af708b4 - Support for FoU encapsulation for IPIP tunnel <Kartik Raval>
• e3d6bcc - fix(NPC): update IPBlocks to be ipFamily specific <Aaron U'Ren>
• 0bcd9a4 - netpol: Fix ipset only containing one IP when port name is used. <Boleyn Su>

v1.5.4

While this is technically a bug fix release as it is fixing a bug with a long deprecated API, it does have the potential to break clients that are running with DSR configurations on unmaintained versions of cri-o or containerd.

The runtime/v1 release is only compatible with >=cri-o v1.21.0 and >=containerd v1.6.0. If you don't use DSR, or use the docker-shim, this change won't affect you.

On the converse side of things, this release is needed if you are using >=cri-o v1.26.0 or >=containerd v1.7.0 otherwise DSR will break because the runtime/v1alpha2 support was removed in those versions.

If you are between those two releases for your given container runtime, then both v1.5.3 and v1.5.4 will work for you.

Changelog

• f350859 - fix(dsr): CRI runtime/v1alpha2 -> runtime/v1 <Aaron U'Ren>

v2.0.0-rc3

Summary

Addresses more of the bugs that were introduced with the initial IPv6 work. Specifically this addresses issues found with the routing functionality where BGP policies weren't updated under some circumstances and also some cases where VIPs weren't being withdrawn.

Noteably the service controller (--run-service-proxy) functionality is still missing (#1432).

Please test and let us know if you find any issues so that we can fix them up before cutting a 2.0.0 proper release.

Contributions

Special thanks to @rwagoner for testing and reporting bugs.

Changelog

• a861f84 - fix(bgp_policy): allow for statement add / remove <Aaron U'Ren>
• 4558820 - fix(ecmp_vip): update VIPs based on svc change <Aaron U'Ren>
• 4365f61 - fix(bgp_policies.go): return -> continue on family set evaluation <Aaron U'Ren>