Merge pull request #22 from ishuar/feat/add-np-type-and-temp-name-for…

…-rotation

Feat: Add NodePool type and Node temp name for rotation

CHANGELOG.md

@@ -5,6 +5,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 
 <!--
 ## version
+
+### Breaking
+  - Changes which may cause recreation of cluster or resources.
+
 ### Added
   - Added new feature
 
@@ -16,11 +20,31 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 
 ### Others
   - Other changes
+
 -->
 
 > **INFO:** This file is only maintained after `v2.0.0` due to no initial availability , please refer to release notes for versions equal or older than `v2.0.0`.
 
+## v2.3.0
+
+### Added
+
+- Support attribute `temporary_name_for_rotation`  which is used to cycle the default node pool for VM resizing.
+
+- Support attribute `node_pool_type` which is used to define the type of Node Pool which should be created. If `enable_auto_scaling` needs to be true then the `node_pool_type` has to be VirtualMachineScaleSets.
+
+- Preconditons to support early warnings and better usability.
+  - if `workload_identity_enabled` is true  then `oidc_issuer_enabled` has to be true.
+  - if `network_plugin_mode` is overlay then `network_plugin` has to be azure.
+  - if `ebpf_data_plane` is cillium then `network_plugin` has to be azure.
+  - if `ebpf_data_plane` is cillium then either `network_plugin_mode` is overlay or `vnet_subnet_id` not set to null.
+
+### Removed
+
+- Deprecated attribute and variable `docker_bridge_cidr`
+
 ## v2.2.0
+
 ### Added
 
 - A complete example is added for the module at [examples/complete](./examples/complete).

README.md

@@ -172,7 +172,6 @@ No modules.
 | <a name="input_dns_prefix"></a> [dns\_prefix](#input\_dns\_prefix) | (optional) Required when dns\_prefix\_private\_cluster is not specified. DNS prefix specified when creating the managed cluster. | `string` | `null` | no |
 | <a name="input_dns_prefix_private_cluster"></a> [dns\_prefix\_private\_cluster](#input\_dns\_prefix\_private\_cluster) | (optional) Required when dns\_prefix is not specified. Specifies the DNS prefix to use with private clusters. | `string` | `null` | no |
 | <a name="input_dns_service_ip"></a> [dns\_service\_ip](#input\_dns\_service\_ip) | (Optional) IP address within the Kubernetes service address range that will be used by cluster service discovery (kube-dns). | `string` | `null` | no |
-| <a name="input_docker_bridge_cidr"></a> [docker\_bridge\_cidr](#input\_docker\_bridge\_cidr) | (Optional) IP address (in CIDR notation) used as the Docker bridge IP address on nodes. **NOTE**: docker\_bridge\_cidr has been deprecated as the API no longer supports it and will be removed in version 4.0 of the provider. | `string` | `null` | no |
 | <a name="input_ebpf_data_plane"></a> [ebpf\_data\_plane](#input\_ebpf\_data\_plane) | (Optional) Specifies the eBPF data plane used for building the Kubernetes network. Possible value is cilium. Changing this forces a new resource to be created. | `string` | `null` | no |
 | <a name="input_enable_allowed_maintenance_window"></a> [enable\_allowed\_maintenance\_window](#input\_enable\_allowed\_maintenance\_window) | (optional) Whether to enable the [allowed maintenance window](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/kubernetes_cluster#allowed) block or not? | `bool` | `true` | no |
 | <a name="input_enable_api_server_access_profile"></a> [enable\_api\_server\_access\_profile](#input\_enable\_api\_server\_access\_profile) | (Optional) Whether to enable API server access profile or not? | `bool` | `false` | no |
@@ -268,6 +267,7 @@ No modules.
 | <a name="input_network_plugin"></a> [network\_plugin](#input\_network\_plugin) | (Optional) Network plugin to use for networking. | `string` | `"kubenet"` | no |
 | <a name="input_network_plugin_mode"></a> [network\_plugin\_mode](#input\_network\_plugin\_mode) | (Optional) Specifies the network plugin mode used for building the Kubernetes network. Possible value is overlay. Changing this forces a new resource to be created. | `string` | `null` | no |
 | <a name="input_network_policy"></a> [network\_policy](#input\_network\_policy) | (Optional) Sets up network policy to be used with Azure CNI. | `string` | `"calico"` | no |
+| <a name="input_node_pool_type"></a> [node\_pool\_type](#input\_node\_pool\_type) | (Optional) The type of Node Pool which should be created. Possible values are AvailabilitySet and VirtualMachineScaleSets. Defaults to VirtualMachineScaleSets. Changing this forces a new resource to be created | `string` | `"VirtualMachineScaleSets"` | no |
 | <a name="input_node_resource_group"></a> [node\_resource\_group](#input\_node\_resource\_group) | (Optional) The name of the Resource Group where the Kubernetes Nodes should exist. | `string` | `null` | no |
 | <a name="input_not_allowed_maintenance_window_end"></a> [not\_allowed\_maintenance\_window\_end](#input\_not\_allowed\_maintenance\_window\_end) | (optional) Required if `enable_not_allowed_maintenance_window` is set to true The end of a time span, formatted as an RFC3339 string. | `string` | `null` | no |
 | <a name="input_not_allowed_maintenance_window_start"></a> [not\_allowed\_maintenance\_window\_start](#input\_not\_allowed\_maintenance\_window\_start) | (optional) Required if `enable_not_allowed_maintenance_window` is set to true The start of a time span, formatted as an RFC3339 string. | `string` | `null` | no |
@@ -287,6 +287,7 @@ No modules.
 | <a name="input_sku_tier"></a> [sku\_tier](#input\_sku\_tier) | (Optional) The SKU Tier that should be used for this Kubernetes Cluster | `string` | `null` | no |
 | <a name="input_snapshot_controller_enabled"></a> [snapshot\_controller\_enabled](#input\_snapshot\_controller\_enabled) | (Optional) Is the Snapshot Controller enabled? Defaults to true. | `bool` | `true` | no |
 | <a name="input_tags"></a> [tags](#input\_tags) | Tags for the cluster | `map(string)` | `null` | no |
+| <a name="input_temporary_name_for_rotation"></a> [temporary\_name\_for\_rotation](#input\_temporary\_name\_for\_rotation) | (optional) Optional) Specifies the name of the temporary node pool used to cycle the default node pool for VM resizing. | `string` | `"tempnp"` | no |
 | <a name="input_vnet_integration_enabled"></a> [vnet\_integration\_enabled](#input\_vnet\_integration\_enabled) | Should API Server VNet Integration be enabled? For more details please visit [Use API Server VNet Integration.](https://learn.microsoft.com/en-us/azure/aks/api-server-vnet-integration) | `bool` | `false` | no |
 | <a name="input_vnet_subnet_id"></a> [vnet\_subnet\_id](#input\_vnet\_subnet\_id) | (optional) The ID of the Subnet where this Node Pool should exist.At this time the vnet\_subnet\_id must be the same for all node pools in the cluster | `string` | `null` | no |
 | <a name="input_windows_profile_admin_password"></a> [windows\_profile\_admin\_password](#input\_windows\_profile\_admin\_password) | (optional) The Admin Password for Windows VMs.(Required) if windows\_profile\_enabled is true. | `string` | `"Super$ecUreP@$$w04d"` | no |

examples/complete/main.tf

@@ -32,17 +32,18 @@ module "complete" {
   kubelet_identity_object_id                 = azurerm_user_assigned_identity.kubelet.principal_id
 
   ##? Default node pool
+  node_pool_type                        = "VirtualMachineScaleSets"
   default_node_pool_name                = "system"
   default_node_pool_enable_auto_scaling = true
-  default_node_pool_vm_size             = "standard_d2ds_v5"
+  default_node_pool_vm_size             = "standard_ds2_v2"
   default_node_pool_min_count           = 1
   default_node_pool_max_count           = 2
   default_node_pool_max_pods            = 110
-
+  temporary_name_for_rotation           = "tmpcomp"
   ##? additional_node_pools
   additional_node_pools = {
     "nodepool01" = {
-      vm_size             = "standard_d2ds_v5"
+      vm_size             = "standard_ds2_v2"
       enable_auto_scaling = true
       max_count           = 2
       min_count           = 1

main.tf

@@ -10,6 +10,7 @@ data "azurerm_kubernetes_service_versions" "current" {
   location        = var.location
   include_preview = var.include_preview
 }
+
 locals {
   aks_cluster                      = var.existing_aks_cluster ? data.azurerm_kubernetes_cluster.this[0] : azurerm_kubernetes_cluster.this[0]
   enable_api_server_access_profile = var.api_server_authorized_ip_ranges != null || var.api_server_access_profile_subnet_id != null || var.vnet_integration_enabled
@@ -50,7 +51,6 @@ resource "azurerm_kubernetes_cluster" "this" {
     service_cidr        = var.service_cidr
     service_cidrs       = var.service_cidrs
     dns_service_ip      = var.dns_service_ip
-    docker_bridge_cidr  = var.docker_bridge_cidr
     outbound_type       = var.outbound_type
     ebpf_data_plane     = var.ebpf_data_plane
     network_plugin_mode = var.network_plugin_mode
@@ -104,6 +104,8 @@ resource "azurerm_kubernetes_cluster" "this" {
     proximity_placement_group_id = var.default_node_pool_proximity_placement_group_id
     message_of_the_day           = var.default_node_pool_message_of_the_day
     workload_runtime             = var.default_node_pool_workload_runtime
+    temporary_name_for_rotation  = var.temporary_name_for_rotation
+    type                         = var.node_pool_type
     upgrade_settings {
       max_surge = var.default_node_pool_upgrade_max_surge
     }
@@ -199,7 +201,37 @@ resource "azurerm_kubernetes_cluster" "this" {
       }
     }
   }
+  lifecycle {
+    ignore_changes = [
+
+    ] ##TODO: To avoid conflicts between auto upgrade and terraform
 
+    precondition {
+      condition     = var.workload_identity_enabled && var.oidc_issuer_enabled
+      error_message = "`oidc_issuer_enabled` must be set to `true` to enable Azure AD Workload Identity"
+    }
+    precondition {
+      condition     = var.network_plugin_mode != "overlay" || var.network_plugin == "azure"
+      error_message = "When network_plugin_mode is set to `overlay`, the network_plugin field can only be set to azure."
+    }
+    precondition {
+      condition     = var.ebpf_data_plane != "cilium" || var.network_plugin == "azure"
+      error_message = "When ebpf_data_plane is set to cilium, the network_plugin field can only be set to azure."
+    }
+    precondition {
+      condition     = var.ebpf_data_plane != "cilium" || var.network_plugin_mode == "overlay" || var.vnet_subnet_id != null
+      error_message = "When ebpf_data_plane is set to cilium, one of either network_plugin_mode = `overlay` or pod_subnet_id must be specified."
+    }
+    ## when defender is enabled, a log analytics workspace id must be provided
+    precondition {
+      condition     = !var.enable_microsoft_defender || var.log_analytics_workspace_id != null
+      error_message = "Enabling Microsoft Defender requires a valid log analytics workspace id."
+    }
+    precondition {
+      condition     = var.default_node_pool_enable_auto_scaling != true || var.node_pool_type == "VirtualMachineScaleSets"
+      error_message = "When Auto Scaling is enabled, the default node pool type must be VirtualMachineScaleSets."
+    }
+  }
 }
 
 resource "azurerm_kubernetes_cluster_node_pool" "this" {
@@ -248,5 +280,14 @@ resource "azurerm_kubernetes_cluster_node_pool" "this" {
     var.tags,
     try(each.value["node_pool_tags"], null)
   )
+  lifecycle {
+    create_before_destroy = true
+    ignore_changes = [
+      name
+    ]
+    precondition {
+      condition     = var.node_pool_type == "VirtualMachineScaleSets"
+      error_message = "To create multiple node pools, the default node pool type must be VirtualMachineScaleSets."
+    }
+  }
 }
-

release-version.yaml

@@ -1,6 +1,6 @@
 ## Update this file for a new release version.
 
-module_version: "2.2.0"
+module_version: "2.3.0"
 
 ## Example for manual release notes.
 # release_notes: |

variables.tf

@@ -259,12 +259,6 @@ variable "dns_service_ip" {
   default     = null
 }
 
-variable "docker_bridge_cidr" {
-  type        = string
-  description = "(Optional) IP address (in CIDR notation) used as the Docker bridge IP address on nodes. **NOTE**: docker_bridge_cidr has been deprecated as the API no longer supports it and will be removed in version 4.0 of the provider."
-  default     = null
-}
-
 variable "enable_api_server_access_profile" {
   type        = bool
   description = "(Optional) Whether to enable API server access profile or not?"
@@ -552,6 +546,20 @@ variable "default_node_pool_message_of_the_day" {
   description = "(Optional) A base64-encoded string which will be written to /etc/motd after decoding. This allows customization of the message of the day for Linux nodes. It cannot be specified for Windows nodes and must be a static string (i.e. will be printed raw and not executed as a script). Changing this forces a new resource to be created"
   default     = null
 }
+
+variable "temporary_name_for_rotation" {
+  type        = string
+  description = "(optional) Optional) Specifies the name of the temporary node pool used to cycle the default node pool for VM resizing."
+  default     = "tempnp"
+  nullable    = false
+}
+
+variable "node_pool_type" {
+  type        = string
+  description = "(Optional) The type of Node Pool which should be created. Possible values are AvailabilitySet and VirtualMachineScaleSets. Defaults to VirtualMachineScaleSets. Changing this forces a new resource to be created"
+  default     = "VirtualMachineScaleSets"
+}
+
 #######################
 # auto scaler profile #
 #######################