v1.26.16

Full Changelog: v1.26.15..v1.26.16

Changes by Kind

Bug or Regression

• Fix: skip tagging route table that is not in the cluster resource group. (#4792, @k8s-infra-cherrypick-robot)

Dependencies

Added

Nothing has changed.

Changed

• golang.org/x/crypto: v0.13.0 → v0.14.0
• golang.org/x/net: v0.10.0 → v0.17.0
• golang.org/x/sys: v0.12.0 → v0.13.0
• golang.org/x/term: v0.12.0 → v0.13.0

Removed

Nothing has changed.

v1.25.20

Full Changelog: v1.25.19..v1.25.20

Changes by Kind

Bug or Regression

• Fix: skip tagging route table that is not in the cluster resource group. (#4791, @k8s-infra-cherrypick-robot)

Dependencies

Added

Nothing has changed.

Changed

• golang.org/x/crypto: v0.13.0 → v0.14.0
• golang.org/x/net: v0.10.0 → v0.17.0
• golang.org/x/sys: v0.12.0 → v0.13.0
• golang.org/x/term: v0.12.0 → v0.13.0

Removed

Nothing has changed.

v1.28.1

Full Changelog: v1.28.0..v1.28.1

Changes by Kind

Feature

• By default, all managed private link service (PLS) are created in the resource group configured by azure config (az.privateLinkServiceResourceGroup or az.ResourceGroup).
  Add ServiceAnnotationPLSResourceGroup = "service.beta.kubernetes.io/azure-pls-resource-group" to control a specific PLS creation resource group. (#4675, @k8s-infra-cherrypick-robot)
• Implement node non-graceful shut down feature. Add "node.kubernetes.io/out-of-service" taint to node when cloud provider determines the node is shutdown. (#4520, @k8s-infra-cherrypick-robot)
• LoadBalancer tcp reset could be disabled with Service annotation "service.beta.kubernetes.io/azure-load-balancer-disable-tcp-reset=true" (#4552, @k8s-infra-cherrypick-robot)
• Support customizing probe config when externalTrafficPolicy is local and health-probe_port annotation is defined for svc port (#4639, @k8s-infra-cherrypick-robot)

Failing Test

• Fix: check endpoint slice update after backend pool update for local service to prevent mismatch (#4659, @k8s-infra-cherrypick-robot)

Bug or Regression

• Fix possible duplicate podCIDR allocation when node podCIDR patch request fails. (#4582, @k8s-infra-cherrypick-robot)
• Fix privateLinkService creation failure in case customer's subnet name or privateLinkService name is too long. (#4538, @k8s-infra-cherrypick-robot)

Dependencies

Added

Nothing has changed.

Changed

• github.com/evanphx/json-patch: v5.6.0+incompatible → v5.7.0+incompatible
• github.com/google/cel-go: v0.16.0 → v0.16.1
• github.com/onsi/ginkgo/v2: v2.11.0 → v2.12.1
• golang.org/x/crypto: v0.12.0 → v0.13.0
• golang.org/x/mod: v0.10.0 → v0.12.0
• golang.org/x/net: v0.13.0 → v0.14.0
• golang.org/x/sys: v0.11.0 → v0.12.0
• golang.org/x/term: v0.11.0 → v0.12.0
• golang.org/x/text: v0.12.0 → v0.13.0
• golang.org/x/tools: v0.9.3 → v0.12.0
• k8s.io/api: v0.28.1 → v0.28.2
• k8s.io/apimachinery: v0.28.1 → v0.28.2
• k8s.io/apiserver: v0.28.1 → v0.28.2
• k8s.io/client-go: v0.28.1 → v0.28.2
• k8s.io/cloud-provider: v0.28.1 → v0.28.2
• k8s.io/component-base: v0.28.1 → v0.28.2
• k8s.io/component-helpers: v0.28.1 → v0.28.2
• k8s.io/controller-manager: v0.28.1 → v0.28.2
• k8s.io/cri-api: v0.28.0 → v0.28.2
• k8s.io/kms: v0.28.1 → v0.28.2
• k8s.io/kubelet: v0.28.0 → v0.28.2

Removed

Nothing has changed.

v1.27.9

Full Changelog: v1.27.8..v1.27.9

Changes by Kind

Feature

• By default, all managed private link service (PLS) are created in the resource group configured by azure config (az.privateLinkServiceResourceGroup or az.ResourceGroup).
  Add ServiceAnnotationPLSResourceGroup = "service.beta.kubernetes.io/azure-pls-resource-group" to control a specific PLS creation resource group. (#4678, @jwtty)
• Implement node non-graceful shut down feature. Add "node.kubernetes.io/out-of-service" taint to node when cloud provider determines the node is shutdown. (#4522, @jwtty)
• LoadBalancer tcp reset could be disabled with Service annotation "service.beta.kubernetes.io/azure-load-balancer-disable-tcp-reset=true" (#4553, @feiskyer)
• Support customizing probe config when externalTrafficPolicy is local and health-probe_port annotation is defined for svc port (#4661, @MartinForReal)

Bug or Regression

• Fix possible duplicate podCIDR allocation when node podCIDR patch request fails. (#4583, @k8s-infra-cherrypick-robot)
• Fix privateLinkService creation failure in case customer's subnet name or privateLinkService name is too long. (#4541, @k8s-infra-cherrypick-robot)

Dependencies

Added

Nothing has changed.

Changed

• github.com/evanphx/json-patch: v5.6.0+incompatible → v5.7.0+incompatible
• github.com/google/cel-go: v0.12.6 → v0.12.7
• github.com/onsi/ginkgo/v2: v2.11.0 → v2.12.1
• golang.org/x/crypto: v0.12.0 → v0.13.0
• golang.org/x/mod: v0.10.0 → v0.12.0
• golang.org/x/net: v0.12.0 → v0.14.0
• golang.org/x/sys: v0.11.0 → v0.12.0
• golang.org/x/term: v0.11.0 → v0.12.0
• golang.org/x/text: v0.12.0 → v0.13.0
• golang.org/x/tools: v0.9.3 → v0.12.0
• k8s.io/api: v0.27.4 → v0.27.6
• k8s.io/apimachinery: v0.27.4 → v0.27.6
• k8s.io/apiserver: v0.27.4 → v0.27.6
• k8s.io/client-go: v0.27.4 → v0.27.6
• k8s.io/cloud-provider: v0.27.1 → v0.27.6
• k8s.io/component-base: v0.27.4 → v0.27.6
• k8s.io/component-helpers: v0.27.1 → v0.27.6
• k8s.io/controller-manager: v0.27.1 → v0.27.6
• k8s.io/kms: v0.27.4 → v0.27.6
• k8s.io/kubelet: v0.27.1 → v0.27.6

Removed

Nothing has changed.

v1.26.15

Full Changelog: v1.26.14..v1.26.15

Changes by Kind

Feature

• By default, all managed private link service (PLS) are created in the resource group configured by azure config (az.privateLinkServiceResourceGroup or az.ResourceGroup).
  Add ServiceAnnotationPLSResourceGroup = "service.beta.kubernetes.io/azure-pls-resource-group" to control a specific PLS creation resource group. (#4677, @jwtty)
• Implement node non-graceful shut down feature. Add "node.kubernetes.io/out-of-service" taint to node when cloud provider determines the node is shutdown. (#4521, @jwtty)
• LoadBalancer tcp reset could be disabled with Service annotation "service.beta.kubernetes.io/azure-load-balancer-disable-tcp-reset=true" (#4554, @feiskyer)
• Support customizing probe config when externalTrafficPolicy is local and health-probe_port annotation is defined for svc port (#4672, @MartinForReal)

Bug or Regression

• Fix possible duplicate podCIDR allocation when node podCIDR patch request fails. (#4584, @k8s-infra-cherrypick-robot)
• Fix privateLinkService creation failure in case customer's subnet name or privateLinkService name is too long. (#4540, @k8s-infra-cherrypick-robot)

Dependencies

Added

Nothing has changed.

Changed

• github.com/evanphx/json-patch: v5.6.0+incompatible → v5.7.0+incompatible
• github.com/google/cel-go: v0.12.6 → v0.12.7
• go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp: v0.35.0 → v0.35.1
• golang.org/x/crypto: v0.12.0 → v0.13.0
• golang.org/x/sys: v0.11.0 → v0.12.0
• golang.org/x/term: v0.11.0 → v0.12.0
• golang.org/x/text: v0.12.0 → v0.13.0
• k8s.io/api: v0.26.8 → v0.26.9
• k8s.io/apimachinery: v0.26.8 → v0.26.9
• k8s.io/apiserver: v0.26.7 → v0.26.9
• k8s.io/client-go: v0.26.8 → v0.26.9
• k8s.io/cloud-provider: v0.26.7 → v0.26.9
• k8s.io/component-base: v0.26.7 → v0.26.9
• k8s.io/component-helpers: v0.26.7 → v0.26.9
• k8s.io/controller-manager: v0.26.7 → v0.26.9
• k8s.io/kms: v0.26.7 → v0.26.9
• k8s.io/kubelet: v0.26.7 → v0.26.9

Removed

Nothing has changed.

v1.25.19

Full Changelog: v1.25.18..v1.25.19

Changes by Kind

Feature

• By default, all managed private link service (PLS) are created in the resource group configured by azure config (az.privateLinkServiceResourceGroup or az.ResourceGroup).
  Add ServiceAnnotationPLSResourceGroup = "service.beta.kubernetes.io/azure-pls-resource-group" to control a specific PLS creation resource group. (#4676, @jwtty)
• LoadBalancer tcp reset could be disabled with Service annotation "service.beta.kubernetes.io/azure-load-balancer-disable-tcp-reset=true" (#4555, @feiskyer)
• Support customizing probe config when externalTrafficPolicy is local and health-probe_port annotation is defined for svc port (#4671, @MartinForReal)

Bug or Regression

• Fix possible duplicate podCIDR allocation when node podCIDR patch request fails. (#4585, @k8s-infra-cherrypick-robot)
• Fix privateLinkService creation failure in case customer's subnet name or privateLinkService name is too long. (#4539, @k8s-infra-cherrypick-robot)

Dependencies

Added

Nothing has changed.

Changed

• github.com/evanphx/json-patch: v5.6.0+incompatible → v5.7.0+incompatible
• golang.org/x/crypto: v0.12.0 → v0.13.0
• golang.org/x/sys: v0.11.0 → v0.12.0
• golang.org/x/term: v0.11.0 → v0.12.0
• golang.org/x/text: v0.12.0 → v0.13.0
• k8s.io/api: v0.25.13 → v0.25.14
• k8s.io/apimachinery: v0.25.13 → v0.25.14
• k8s.io/apiserver: v0.25.13 → v0.25.14
• k8s.io/client-go: v0.25.13 → v0.25.14
• k8s.io/cloud-provider: v0.25.13 → v0.25.14
• k8s.io/component-base: v0.25.13 → v0.25.14
• k8s.io/component-helpers: v0.25.13 → v0.25.14
• k8s.io/controller-manager: v0.25.13 → v0.25.14
• k8s.io/kubelet: v0.25.12 → v0.25.14

Removed

Nothing has changed.

v1.28.0

Full Changelog: v1.27.0..v1.28.0

Urgent Upgrade Notes

(No, really, you MUST read this before you upgrade)

• VmType's default value is changed from 'standard' to 'vmss' since v1.28.

  if your cluster is not using any VMSS instances and 'vmType' is not configured in the cloud provider config file, please add vmType=standard in the config file before upgrading to v1.28. (#4214, @feiskyer)

Changes by Kind

Testing

• Fix ensureBackendPoolDeleted for standalone VM (#4217, @lzhecheng)

Feature

• Apply the custom probe configs when externalTrafficPolicy is local and PLSProxyProtocol is enabled. (#3931, @MartinForReal)
• Chore: upgrade to sdk api version 2022-08-01
  feat: add PerformancePlus option in disk creation (#3855, @andyzhangx)
• Feat: Support migrate from NIC-based to IP-based backend pool by migration API by setting "enableMigrateToIpBasedBackendPoolAPI": true. By using this API, there will be no downtime during the migration. (#3972, @nilo19)
• Feat: add GetLatestAccountKey in account key fetch (#4067, @andyzhangx)
• Feat: add load balancer backend pool batch updater (#4391, @nilo19)
• Feat: support load balancer choosing logic for multi-slb (#4075, @nilo19)
• Feat: support local service in multiple standard load balancer mode (#4450, @nilo19)
• Feat: support node selection for multiple standard load balancers (#4201, @nilo19)
• Feat: support sharing IP address acorss services by public IP name (#4257, @nilo19)
• Feat: support workload identity (#3378, @cvvz)
• Health probe port can be any port assigned by customer. (#4380, @MartinForReal)
• Skip unmanaged Nodes for instancesV2 (#4294, @lzhecheng)
• Support NSG and clean LBs for dualstack
  • Support related UTs for dualstack
  • Refactor (#3898, @lzhecheng)
• [ARG] Add an option to disable API call cache
  New option: disableAPICallCache
  When ARG is enabled, this option should be true. (#4135, @lzhecheng)
• [DualStack] Support FrontendIPConfig and reconcileLB() * DualStack feature code * Refactor related functions and methods * Refactor and add new UTs (#3819, @lzhecheng)
• add PickRandomMatchingAccount in account search
  • add PublicNetworkAccess in disk creation (#3811, @andyzhangx)

Failing Test

• Chore: skip exclude node label e2e test on aks as it is only supported in self-managed clusters (#4076, @nilo19)

Bug or Regression

• Fix IPv6/dual-stack EnsureBackendPoolDeleted() failure. IP config of IPv6 is not primary, it should not be skipped in EnsureBackendPoolDeleted(). Updated e2e code. (#4272, @lzhecheng)
• Fix vmssflex ensureBackendPoolDeletedFromNode
  • Fix loop pointer issue
  • Use lock on nicUpdated var
  • Fix log format (#4074, @lzhecheng)
• Fix: PerformancePlus setting issue (#4193, @andyzhangx)
• Fix: add StorageAccountCache to avoid querying storage account frequently (#4422, @andyzhangx)
• Fix: make sure the pip dns tag will not be removed when systemTags is set (#3956, @nilo19)
• Fix: remove deleted node IP address from IP-based LB backend pools (#4136, @nilo19)
• Fix: replace deprecated labels with new labels in nodeCache update (#4047, @andyzhangx)
• Fix: storage account search default values (#4203, @andyzhangx)
• Fix: the pip without tags should be user-assigned
  fix: refresh the pip cache when necessary
  fix: do not tag user-assigned pip with kubernetes-dns-label-service: <svcName> (#3877, @nilo19)
• Fix: update the lb list after changing lb to prevent etag mismatches
  fix: return the existing lb it if the lb exists without creating a new lb when the service was moved to the lb
  fix: should skip non-existent lb when arranging nodes (#4289, @nilo19)
• Fixed: Remove shared nsg rule immediately when no destinations left (#3787, @MartinForReal)
• Fixes issue 4230 and removes the additional filtering on NotReady nodes by the azure cloud provider code (#4234, @alexanderConstantinescu)
• Increase limit for TCP Idle Timeout to 100 minutes (#4361, @JoelSpeed)
• Support customization of numOfProbe and probeInterval when externaltrafficpolicy is local (#4207, @MartinForReal)
• The deprecated beta topology labels are no longer applied by default, to maintain the legacy behaviour use --deprecated-apply-beta-topology-labels (#3685, @JoelSpeed)
• Virtual node will always exists (#4393, @MartinForReal)
• [IPv6] Fix reconcileFrontendIPConfigs(). Current logic handles lb.FrontendIPConfigurations according to Service's IP family, which is incorrect. For an IPv6 cluster, there're still some IPv4 FIPs due to Azure limitation, which will be removed. For an IPv4 cluster, all resources are of IPv4, which is not affected. (#3914, @lzhecheng)
• [IPv6] backend pool name should be case-insensitive (#3932, @lzhecheng)

Other (Cleanup or Flake)

• Build images from debian bullseye (#4066, @jackfrancis)
• Chore: cleanup unused multi-slb code of the previous design (#3997, @nilo19)
• Chore: set default loadBalancerSKU to Standard (#3768, @nilo19)
• Helm: add logVerbosity to cloud-node-manager (#4111, @lzhecheng)
• Release helm v1.27.7+20230815 (#4446, @nilo19)
• [DualStack] IPv6 PIP uses suffix only when DualStack. For CCM v1.27.1, the IPv6 PIP created has suffix. After CCM is upgraded, such PIP will be recreated. (#3823, @lzhecheng)
• [Log] Print Service name and resource basename in ReconcileService(). Adding such log shows relation between Service name and its related resource name (Frontend IP config, etc.). It helps debugging. ([#3957](https://github.com/kubernetes-sigs/cloud-pro...

v1.27.8

Full Changelog: v1.27.7..v1.27.8

Changes by Kind

Feature

• [DualStack] Support FrontendIPConfig and reconcileLB() * DualStack feature code * Refactor related functions and methods * Refactor and add new UTs
  [IPv6] Fix reconcileFrontendIPConfigs(). Current logic handles lb.FrontendIPConfigurations according to Service's IP family, which is incorrect. For an IPv6 cluster, there're still some IPv4 FIPs due to Azure limitation, which will be removed. For an IPv4 cluster, all resources are of IPv4, which is not affected.
  Support NSG and clean LBs for dualstack (#4397, @lzhecheng)

Bug or Regression

• Virtual node will always exists (#4394, @k8s-infra-cherrypick-robot)

Dependencies

Added

Nothing has changed.

Changed

• golang.org/x/crypto: v0.11.0 → v0.12.0
• golang.org/x/sys: v0.10.0 → v0.11.0
• golang.org/x/term: v0.10.0 → v0.11.0
• golang.org/x/text: v0.11.0 → v0.12.0

Removed

Nothing has changed.

v1.26.14

Full Changelog: v1.26.13..v1.26.14

Dependencies

Added

Nothing has changed.

Changed

• golang.org/x/crypto: v0.11.0 → v0.12.0
• golang.org/x/sys: v0.10.0 → v0.11.0
• golang.org/x/term: v0.10.0 → v0.11.0
• golang.org/x/text: v0.11.0 → v0.12.0

Removed

Nothing has changed.

v1.25.18

Full Changelog: v1.25.17..v1.25.18

Dependencies

Added

Nothing has changed.

Changed

• golang.org/x/crypto: v0.11.0 → v0.12.0
• golang.org/x/sys: v0.10.0 → v0.11.0
• golang.org/x/term: v0.10.0 → v0.11.0
• golang.org/x/text: v0.11.0 → v0.12.0

Removed

Nothing has changed.