diff --git a/CHANGELOG.md b/CHANGELOG.md index 578bb63..a423446 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,14 @@ All notable changes to the Packer template for creating GraphDB Azure VM images - Added provisioning of GraphDB backup script. - Added keepalive and file max size settings. - Improved logging of the GraphDB install script. +- Added proper prefixes to the variable names in variables.pck.hcl +- Grouped variables by usage in variables.pck.hcl +- Added new variables in variables.pck.hcl + - `tags` - Additional tags to apply to the build resources + - `build_temp_resource_group_name` - Name of the temporary resource group created by Packer during the build + - `build_resource_group_name` - Name of a resource group to be used by Packer during the build + - `gallery_subscription_id` - ID of the subscription where the Shared Image Gallery is located +- Updated [create_image_definition.sh](create_image_definition.sh) to just create the definition and to skip invoking `packer build` ## 1.0.0 diff --git a/README.md b/README.md index 0003275..e30b9d6 100644 --- a/README.md +++ b/README.md @@ -31,20 +31,20 @@ Follow these steps to build an AMI for GraphDB using Packer: 3. **Edit Variables**: The Packer configuration allows you to customize various parameters, such as the GraphDB version, Azure build and - replication regions, subscription, client and tennat IDs. To do so, create a variables file `variables.pkrvars.hcl`, + replication regions, subscription, client and tenant IDs. To do so, create a variables file `variables.pkrvars.hcl`, example file: ```bash - subscription_id = "" - client_id = "" - client_secret = "" - tenant_id = "" - primary_location = "East US" - replication_regions = ["North Europe", "UK South"] - image_definition_name = "10.4.0-x86_64" - gdb_version = "10.4.0" - gallery_resource_group = "Packer-RG" - gallery_name = "GraphDB" - allowed_inbound_ip_addresses = "" + subscription_id = "" + tenant_id = "" + client_id = "" + client_secret = "" + build_location = "East US" + graphdb_version = "10.4.0" + gallery_image_definition = "10.4.0-x86_64" + gallery_resource_group = "Packer-RG" + gallery_name = "GraphDB" + gallery_image_replication_regions = ["North Europe", "UK South"] + build_allowed_inbound_ip_addresses = "" ``` 4. **Build the AMI**: @@ -56,7 +56,7 @@ Follow these steps to build an AMI for GraphDB using Packer: This command will initiate the Packer build process. Packer will launch an VM instance, install GraphDB, and create an image based on the instance. - Please note that the image definition you specify in `image_definition_name` must exist in the SIG. + Please note that the image definition you specify in `gallery_image_definition` must exist in the SIG. To automate this process you can use the `create_image_definition.sh` script, which will read the variables in `variables.pkrvars.hcl` and create the image definition in the gallery. This script utilizes the Azure CLI, so you need to install and configure it beforehand. @@ -65,10 +65,10 @@ Follow these steps to build an AMI for GraphDB using Packer: az_command="az sig image-definition create \ -g $gallery_resource_group \ --gallery-name $gallery_name \ - --gallery-image-definition "$image_definition_name" \ + --gallery-image-definition "$gallery_image_definition" \ --publisher Ontotext \ --offer GraphDB \ - --sku "$gdb_version" \ + --sku "$graphdb_version" \ --os-type Linux \ --hyper-v-generation v2 \ --minimum-cpu-core 4 \ @@ -85,39 +85,46 @@ The following points can be customized in a packer variables file `variables.pkr **Subscription Configuration** -* subscription_id (string): Your Azure subscription ID. -* tenant_id (string): Your Azure Active Directory tenant ID. -* client_id (string): The client ID (Service Principal ID) used for authentication. -* client_secret (string): The client secret (Service Principal Secret) used for authentication. -* primary_location (string): The primary Azure location you want to use. +* `subscription_id` (string): Your Azure subscription ID. +* `tenant_id` (string): Your Azure Active Directory tenant ID. +* `client_id` (string): The client ID (Service Principal ID) used for authentication. +* `client_secret` (string): The client secret (Service Principal Secret) used for authentication. +* `use_azure_cli_auth` (bool): Use the information from an active az login session instead of authenticating with a client secret -**Image Configuration** +**GraphDB Configuration** -* image_definition_name (string): The name of the x86_64 image to use. -* gdb_version (string): The version of GraphDB to install. -* replication_regions (list(string)): A list of Azure regions for replication of the created image. +* `graphdb_version` (string): The version of GraphDB to install and package as VM -**Gallery Configuration** +**Build and OS Configuration** -* gallery_resource_group (string): The resource group where the image gallery is located. -* gallery_name (string): The name of the image gallery. -* image_replica_count (number): The number of replicas of the Image Version to be created per region. (default is 1). +* `build_location` (string): The primary Azure location you want to use for building the image +* `build_temp_resource_group_name` (string): Name of the temporary resource group created by Packer during the build. The default is a random + generated name. +* `build_resource_group_name` (string): Name of a resource group to be used by Packer during the build. Cannot specify this together with + build_location or build_temp_resource_group_name. +* `build_allowed_inbound_ip_addresses` (string): Specify the list of IP addresses and CIDR blocks that should be allowed access to the VM. +* `build_vm_size` (string): The Azure VM size (default is "Standard_B1ls"). +* `build_os_type` (string): The operating system type (default is "Linux"). +* `build_os_disk_size_gb` (number): size of the OS disk in GB (default is 30). Depends on base VM image limitation, + e.g., Ubuntu Server image has `build_os_disk_size_gb = 30`. -**Networking Configuration** +**Base Image Configuration** -* allowed_inbound_ip_addresses (string): Specify the list of IP addresses and CIDR blocks that should be allowed access to the VM. +* `base_image_offer` (string): The offer for the base image (default is "0001-com-ubuntu-server-jammy"). +* `base_image_publisher` (string): The publisher for the base image (default is "canonical"). +* `base_image_sku` (string): The SKU for the base image (default is "22_04-lts-gen2"). -**OS and Image Defaults** +**Gallery Configuration** -* os_type (string): The operating system type (default is "Linux"). -* image_offer (string): The offer for the base image (default is "0001-com-ubuntu-server-jammy"). -* image_publisher (string): The publisher for the base image (default is "canonical"). -* image_sku (string): The SKU for the base image (default is "22_04-lts-gen2"). -* vm_size (string): The Azure VM size (default is "Standard_B1ls"). -* shared_gallery_image_version_exclude_from_latest (bool): If set to true, Virtual Machines deployed from the latest +* `gallery_subscription_id` (string): ID of the subscription where the Shared Image Gallery is located. Will use subscription_id as default if + unspecified +* `gallery_resource_group` (string): The resource group where the image gallery is located. +* `gallery_name` (string): The name of the image gallery. +* `gallery_image_definition` (string): The name of the x86_64 image to use. +* `gallery_image_replication_regions` (list(string)): A list of Azure regions for replication of the created image. +* `gallery_image_replica_count` (number): The number of replicas of the Image Version to be created per region. (default is 1). +* `gallery_image_version_exclude_from_latest` (bool): If set to true, Virtual Machines deployed from the latest version of the Image Definition won't use this Image Version (default is false) -* os_disk_size_gb (number): size of the OS disk in GB (default is 30). Depends on base VM image limitation, - e.g., Ubuntu Server image has `os_disk_size_gb = 30`. **Provisioning Scripts**: You can replace or modify the provisioning scripts located in the `./files/` directory. These scripts and files are copied and executed during the AMI creation process. diff --git a/azure.pkr.hcl b/azure.pkr.hcl index a7befe2..10465f0 100644 --- a/azure.pkr.hcl +++ b/azure.pkr.hcl @@ -1,7 +1,8 @@ locals { - timestamp = regex_replace(timestamp(), "[- TZ:]", "") + timestamp = regex_replace(timestamp(), "[- TZ:]", "") # See https://github.com/hashicorp/packer-plugin-azure/issues/65 - version_timestamp = formatdate("YYYY.MM.DD", timestamp()) + version_timestamp = formatdate("YYYY.MM.DD", timestamp()) + gallery_subscription_id = var.gallery_subscription_id || var.subscription_id } source azure-arm ubuntu-x86-64 { @@ -10,68 +11,75 @@ source azure-arm ubuntu-x86-64 { client_secret = var.client_secret subscription_id = var.subscription_id tenant_id = var.tenant_id - location = var.primary_location + + location = var.build_location + temp_resource_group_name = var.build_temp_resource_group_name + build_resource_group_name = var.build_resource_group_name + + image_offer = var.base_image_offer + image_publisher = var.base_image_publisher + image_sku = var.base_image_sku shared_image_gallery_destination { - subscription = var.subscription_id + subscription = local.gallery_subscription_id resource_group = var.gallery_resource_group gallery_name = var.gallery_name - image_name = var.image_definition_name + image_name = var.gallery_image_definition image_version = local.version_timestamp - replication_regions = var.replication_regions + replication_regions = var.gallery_image_replication_regions } - shared_image_gallery_replica_count = var.image_replica_count - shared_gallery_image_version_exclude_from_latest = var.shared_gallery_image_version_exclude_from_latest + shared_image_gallery_replica_count = var.gallery_image_replica_count + shared_gallery_image_version_exclude_from_latest = var.gallery_image_version_exclude_from_latest - azure_tags = { - GDB_Version = "${var.gdb_version}" + azure_tags = merge({ + GraphDB_Version = "${var.graphdb_version}" CPU_Architecture = "x86-64" Build_Timestamp = "${local.timestamp}" - } - - communicator = "ssh" - ssh_clear_authorized_keys = true - os_type = var.os_type - os_disk_size_gb = var.os_disk_size_gb - image_offer = var.image_offer - image_publisher = var.image_publisher - image_sku = var.image_sku + }, var.tags) - vm_size = var.vm_size - allowed_inbound_ip_addresses = var.allowed_inbound_ip_addresses + communicator = "ssh" + ssh_clear_authorized_keys = true + os_type = var.build_os_type + os_disk_size_gb = var.build_os_disk_size_gb + vm_size = var.build_vm_size + allowed_inbound_ip_addresses = var.build_allowed_inbound_ip_addresses } -#TODO Not yet supported by Packer https://github.com/hashicorp/packer/issues/12188#issuecomment-1380736642 +# TODO Not yet supported by Packer https://github.com/hashicorp/packer/issues/12188#issuecomment-1380736642 #source azure-arm ubuntu-arm64 { # client_id = var.client_id # client_secret = var.client_secret # subscription_id = var.subscription_id # tenant_id = var.tenant_id # -# location = var.primary_location +# location = var.build_location +# temp_resource_group_name = var.build_temp_resource_group_name +# build_resource_group_name = var.build_resource_group_name +# +# image_offer = "0001-com-ubuntu-server-jammy" +# image_publisher = "canonical" +# image_sku = "22_04-lts-arm64" +# # shared_image_gallery_destination { # subscription = var.subscription_id # resource_group = var.gallery_resource_group # gallery_name = var.gallery_name # image_name = var.image_name_arm64 # image_version = local.version_timestamp -# replication_regions = var.replication_regions +# replication_regions = var.gallery_image_replication_regions # } # -# azure_tags = { -# GDB_Version = "${var.gdb_version}" +# azure_tags = merge({ +# GraphDB_Version = "${var.graphdb_version}" # CPU_Architecture = "arm64" # Build_Timestamp = "${local.timestamp}" -# } -# -# communicator = "ssh" -# os_type = "Linux" -# image_offer = "0001-com-ubuntu-server-jammy" -# image_publisher = "canonical" -# image_sku = "22_04-lts-arm64" -# -# vm_size = "Standard_D2ps_v5" +# }, var.tags) # -# allowed_inbound_ip_addresses = [var.allowed_inbound_ip_addresses] +# communicator = "ssh" +# ssh_clear_authorized_keys = true +# os_type = "Linux" +# os_disk_size_gb = var.build_os_disk_size_gb +# vm_size = "Standard_D2ps_v5" +# allowed_inbound_ip_addresses = [var.build_allowed_inbound_ip_addresses] #} diff --git a/build.pkr.hcl b/build.pkr.hcl index 154d876..3495109 100644 --- a/build.pkr.hcl +++ b/build.pkr.hcl @@ -16,7 +16,7 @@ build { provisioner "shell" { environment_vars = [ - "GRAPHDB_VERSION=${var.gdb_version}", + "GRAPHDB_VERSION=${var.graphdb_version}", ] inline = ["sudo -E bash /tmp/install_graphdb.sh"] max_retries = 2 diff --git a/create_image_definition.sh b/create_image_definition.sh old mode 100644 new mode 100755 index fe3c675..e9cce37 --- a/create_image_definition.sh +++ b/create_image_definition.sh @@ -1,11 +1,11 @@ -#!/bin/bash +#!/usr/bin/env bash # This script sets up Azure Shared Image Gallery (SIG) image definition and builds an image using Packer. # It checks for a variables file, extracts necessary parameters, constructs an Azure CLI command for image definition creation, # executes the command, waits for creation completion, # and finally uses Packer to build the image based on the specified variables. -set -euxo pipefail +set -euo pipefail # Required when running the script on Windows alias az=az.cmd || true @@ -14,49 +14,49 @@ alias az=az.cmd || true variables_file="variables.pkrvars.hcl" # Check if the file exists -if [ -f "$variables_file" ]; then - # Extracts required variables from the variables_file - subscription_id=$(grep 'subscription_id' "$variables_file" | cut -d '"' -f 2) - echo "Subscription ID: $subscription_id" - image_definition_name=$(grep 'image_definition_name' "$variables_file" | cut -d '"' -f 2) - echo "Image definition name: $image_definition_name" - gdb_version=$(grep 'gdb_version' "$variables_file" | cut -d '"' -f 2) - echo "GraphDB version: $gdb_version" - gallery_resource_group=$(grep 'gallery_resource_group' "$variables_file" | cut -d '"' -f 2) - echo "Resource group: $gallery_resource_group" - gallery_name=$(grep 'gallery_name' "$variables_file" | cut -d '"' -f 2) - echo "Gallery: $gallery_name" - - # Checks if any of the required variables is empty - if [ -z "$image_definition_name" ] || [ -z "$gdb_version" ] || [ -z "$gallery_resource_group" ] || [ -z "$gallery_name" ] || [ -z "$subscription_id" ]; then - echo "One or more required variables are not defined in $variables_file." - exit 1 - fi - - # Constructs the az sig image-definition create command - az_command="az sig image-definition create \ - --subscription $subscription_id \ - -g $gallery_resource_group \ - --gallery-name $gallery_name \ - --gallery-image-definition "$image_definition_name" \ - --publisher Ontotext \ - --offer GraphDB \ - --sku "$gdb_version" \ - --os-type Linux \ - --hyper-v-generation v2 \ - --minimum-cpu-core 4 \ - --maximum-cpu-core 64 \ - --minimum-memory 4 \ - --maximum-memory 128 " - - echo "Extracted variables and constructed Azure CLI command:" - echo "Creating SIG" - eval "$az_command" - # Waits for the Shared Image Gallery to be created - az sig image-definition wait -i "$image_definition_name" -r "$gallery_name" -g "$gallery_resource_group" --created --subscription $subscription_id - echo "Begin building of the Azure VM image" - packer build -var-file="variables.pkrvars.hcl" . -else +if ! [ -f "$variables_file" ]; then echo "The variables file $variables_file does not exist." exit 1 fi + +# Extracts required variables from the variables_file +subscription_id=$(grep 'subscription_id' "$variables_file" | cut -d '"' -f 2) +echo "Subscription ID: $subscription_id" +gallery_image_definition=$(grep 'gallery_image_definition' "$variables_file" | cut -d '"' -f 2) +echo "Image definition name: $gallery_image_definition" +graphdb_version=$(grep 'graphdb_version' "$variables_file" | cut -d '"' -f 2) +echo "GraphDB version: $graphdb_version" +gallery_resource_group=$(grep 'gallery_resource_group' "$variables_file" | cut -d '"' -f 2) +echo "Resource group: $gallery_resource_group" +gallery_name=$(grep 'gallery_name' "$variables_file" | cut -d '"' -f 2) +echo "Gallery: $gallery_name" + +# Checks if any of the required variables is empty +if [ -z "$gallery_image_definition" ] || [ -z "$graphdb_version" ] || [ -z "$gallery_resource_group" ] || [ -z "$gallery_name" ] || [ -z "$subscription_id" ]; then + echo "One or more required variables are not defined in $variables_file." + exit 1 +fi + +# Constructs the az sig image-definition create command +az_command="az sig image-definition create \ + --subscription $subscription_id \ + -g $gallery_resource_group \ + --gallery-name $gallery_name \ + --gallery-image-definition "$gallery_image_definition" \ + --publisher Ontotext \ + --offer GraphDB \ + --sku "$graphdb_version" \ + --os-type Linux \ + --hyper-v-generation v2 \ + --minimum-cpu-core 4 \ + --maximum-cpu-core 64 \ + --minimum-memory 4 \ + --maximum-memory 128 " + +# TODO: defuk is this : +echo "Extracted variables and constructed Azure CLI command:" +echo "Creating SIG" +eval "$az_command" +# Waits for the Shared Image Gallery to be created +az sig image-definition wait -i "$gallery_image_definition" -r "$gallery_name" -g "$gallery_resource_group" --created --subscription $subscription_id +echo "Begin building of the Azure VM image" diff --git a/plugins.pkr.hcl b/plugins.pkr.hcl index 7f50735..8f16fd8 100644 --- a/plugins.pkr.hcl +++ b/plugins.pkr.hcl @@ -2,7 +2,7 @@ packer { required_plugins { azure = { source = "github.com/hashicorp/azure" - version = ">=2.0.0" + version = "~>2.0.0" } } } diff --git a/variables.pkr.hcl b/variables.pkr.hcl index 19ec100..33e1038 100644 --- a/variables.pkr.hcl +++ b/variables.pkr.hcl @@ -1,13 +1,27 @@ +# +# Common variables +# + +variable tags { + description = "Additional tags to apply to the build resources, including the image version" + type = map(string) + default = {} +} + +############################ +# Authentication variables # +############################ + # The following 4 variables are for access via Service Principal variable subscription_id { - description = "ID of the subscription the VM image would be build in" + description = "ID of the subscription where the VM image would be built" type = string default = null } variable tenant_id { - description = "Microsoft Entra tenant ID" + description = "Microsoft Entra tenant ID for the subscription_id" type = string default = null } @@ -24,93 +38,133 @@ variable client_secret { default = null } +# Or you can use an existing auth session from Azure CLI + variable use_azure_cli_auth { description = "CLI auth will use the information from an active az login session to connect to Azure and set the subscription id and tenant id associated to the signed in account." type = bool default = true } -# Generic variables +# +# GraphDB variables +# -variable primary_location { - description = "Azure datacenter in which your VM will build" +variable graphdb_version { + description = "Version of GraphDB which will be installed and packaged as VM" type = string } -variable image_definition_name { - description = "Image Name" - type = string -} +# +# Build variables +# -variable gdb_version { - description = "Version of GraphDB which will be installed" +variable build_location { + description = "The Azure region where the VM will be built with Packer" type = string + default = null } -variable gallery_resource_group { - description = "Name of the resource group where the Shared Image Gallery is located" +variable build_temp_resource_group_name { + description = "Name of the temporary resource group created by Packer during the build. The default is a random generated name." type = string + default = null } -variable gallery_name { - description = "Name of the Shared Image Gallery" +variable build_resource_group_name { + description = "Name of a resource group to be used by Packer during the build. Cannot specify this together with build_location or build_temp_resource_group_name." type = string + default = null } -variable allowed_inbound_ip_addresses { - description = "Specify the list of IP addresses and CIDR blocks that should be allowed access to the VM" +variable build_allowed_inbound_ip_addresses { + description = "List of IP addresses and CIDR blocks that should be allowed access to the VM during the building" type = list(string) } -variable replication_regions { - description = "A list of regions to replicate the image version in" - type = list(string) +// Note: Architecture specific +variable build_vm_size { + description = "Size of the VM used for building" + type = string + default = "Standard_B1ls" } -variable os_type { - description = "Type of the OS, this configures an SSH authorized key" +variable build_os_type { + description = "Type of the OS, this configures the SSH authorized key" type = string default = "Linux" } -variable image_offer { +variable build_os_disk_size_gb { + description = "Size of the OS disk in GB (gigabytes). Values of zero or less than zero are ignored." + type = number + default = 30 +} + +# +# Base image variables +# + +// Note: Architecture specific +variable base_image_offer { description = "Name of the publisher's offer to use for your base image" type = string default = "0001-com-ubuntu-server-jammy" } -variable image_publisher { +variable base_image_publisher { description = "Name of the publisher to use for your base image (Azure Marketplace Images only)" type = string default = "canonical" } -variable image_sku { +// Note: Architecture specific +variable base_image_sku { description = "SKU of the image offer to use for your base image (Azure Marketplace Images only)." type = string default = "22_04-lts-gen2" } -variable vm_size { - description = "Size of the VM used for building" +# +# Shared Image Gallery (SIG) variables +# + +variable gallery_subscription_id { + description = "ID of the subscription where the Shared Image Gallery is located. Will use subscription_id as default if unspecified." type = string - default = "Standard_B1ls" + default = null +} + +variable gallery_resource_group { + description = "Name of the resource group where the Shared Image Gallery is located" + type = string +} + +variable gallery_name { + description = "Name of the Shared Image Gallery" + type = string +} + +// Note: Architecture specific +variable gallery_image_definition { + description = "Name of the gallery definition for the built images of particular GraphDB version" + type = string +} + +variable gallery_image_replication_regions { + description = "A list of regions to replicate the image version in" + type = list(string) + default = [] } -variable image_replica_count { +variable gallery_image_replica_count { description = "The number of replicas of the Image Version to be created per region. Replica count must be between 1 and 100" type = number default = 1 } -variable shared_gallery_image_version_exclude_from_latest { +variable gallery_image_version_exclude_from_latest { description = "If set to true, Virtual Machines deployed from the latest version of the Image Definition won't use this Image Version" type = bool default = false } - -variable os_disk_size_gb { - description = "Specify the size of the OS disk in GB (gigabytes). Values of zero or less than zero are ignored." - type = number - default = 30 -}