Jfrog: Migrate pgsql to flex server

[Souheil-Yazji]

cc. @Karivenu

We need to migrate the Jfrog pgSQL server from single to flex server before march 28.

To Do

• Create new pgsql flex server resource using Azurerm
• Migrate the data from the old pgsql server to the new one
• Configure Jfrog to correctly use this resource (Since this will be live migration, create an announcement for possible service disruption)
• Verify that Jfrog works as intended (both Artifactory and X-ray don't have problems)
• Document work done
• Update Github Ticket

Resources

You can find the new modules for flexible servers here https://gitlab.k8s.cloud.statcan.ca/managed-databases/flexible-server/terraform-azurerm-flex-postgresql

Once you have one created, you can follow one of the following mechanisms to migrate data.

https://learn.microsoft.com/en-us/azure/postgresql/migrate/migration-service/tutorial-migration-service-single-to-flexible?tabs=portal%2Coffline

https://learn.microsoft.com/en-us/azure/postgresql/migrate/how-to-migrate-using-dump-and-restore?tabs=psql

[EveningStarlight]

https://gitlab.k8s.cloud.statcan.ca/cloudnative/aaw/daaas-infrastructure/aaw-prod-cc-00/-/blob/main/jfrog.tf?ref_type=heads#L95

[EveningStarlight]

Changes between our current single module and the new flexible module

They build off of azurerm_postgresql_server and azurerm_postgresql_flexible_server respectively.

source:
source = "./modules/terraform-azurerm-postgresql" ->
source = "git::https://gitlab.k8s.cloud.statcan.ca/managed-databases/flexible-server/terraform-azurerm-flex-postgresql.git?ref=v0.5.0"

Remapped

administrator_login_password -> administrator_password
keyvault_enable -> kv_pointer_enable

Reformatted

database_names(list) -> databases(map(map(string)))

Administrator Group:
active_directory_administrator_object_id(string) and active_directory_administrator_tenant_id(string) ->
active_directory_administrator(list(object({}))

client_min_messages part of postgresql_configurations(map)

Removed items

Hard coded as part of key_vault.tf
key_type = "RSA-HSM"
key_size = 2048

ssl_enforcement_enabled and ssl_minimal_tls_version_enforced are not inputs for azurerm_postgresql_flexible_server and are removed.
min_tls_version -> hard coded in source module
threat_detection_policy isn't an input for azurerm_postgresql_flexible_server, this originally contained emails

Storage Account

The new terraform module handles the creation of it's own Storage account. So some things are moved there

All settings moved to codeblock in module

  # Storage Account Settings
  storage_account_name = "${var.prefix}-sa-jfrog"
  ip_rules             = var.infrastructure_authorized_ip_ranges
  sa_subnet_ids        = concat([data.azurerm_subnet.aks_system.id], var.infrastructure_pipeline_subnet_ids)
  sa_create_log        = true

Conflicts:
The redundancy is hard coded and cannot be changed.
account_replication_type = "ZRS" -> account_replication_type = "LRS"

Cannot create container named artifactory" in SA. Can set sa_create_log = true` to create a container for diagnostic logging, currently I've enabled this feature.

Key Vault Key

The new terraform module handles the creation of it's own Key Vault Key. So some things are moved there

Conflicts:
Name changed: "${var.prefix}-key-jfrog-storage" -> "${var.name}-key-cmk"
Key Size: key_size = "4096" -> key_size = "2048"

Customer Managed Key

Breaking
azurerm_storage_account_customer_managed_key isn't defined in the module, and the module outputs are not enough to create it.
If needed, we might need to look into fetching ids off names.

resource "azurerm_storage_account_customer_managed_key" "jfrog" {
  storage_account_id = module.jfrog_storage_account.id
  key_vault_id       = azurerm_key_vault.keys.id
  key_name           = azurerm_key_vault_key.jfrog_storage.name
}

Key Vault Access Policy

azurerm_key_vault_access_policy is included in the module.

Conflicts:
Additional permission: "List"