Add support for all DNSimple record types and configurable default re…

…cords, including SRV (#12)

This PR entirely overhauls the module's capability. It now supports:

- All DNS record types supported by DNSimple
- It allows for multiple records to be created as part of a single service definition by using index notation in the **service.meta** keys e.g. `meta.dnsimple_<attribute>[<index>]` or `meta.dnsimple_zone_name[1]`. This is due to a limitation of the meta section only supporting string values.
- Variable expansion in the record name and content attributes so that developers can use any of the values defined in the service spec without hardcoding.

Belongs to dnsimple/dnsimple-marketing#619
Belongs to https://github.com/dnsimple/dnsimple-external-integrations/issues/4

CHANGELOG.md

@@ -1,5 +1,21 @@
 ## main
 
+## 1.0.0 (Unreleased)
+
+BREAKING CHANGES:
+
+* **Zone Name:** The `meta.zone_name` has been renamed to `meta.dnsimple_zone_name` for better namespacing
+* **Record Type:** The `meta.record_type` has been renamed to `meta.dnsimple_record_type` for better namespacing
+* **Record TTL:** The `meta.record_ttl` has been renamed to `meta.dnsimple_record_ttl` for better namespacing
+
+FEATURES:
+
+* **All Record Type Support:** Added support for all record types supported by the DNSimple (dnsimple/terraform-dnsimple-cts#12)
+* **Multiple Record Support:** Added support for the configuration of multiple records per service (dnsimple/terraform-dnsimple-cts#12)
+
+NOTES:
+
+* Developers can now configure multiple records per service by adding an index to the `meta.dnsimple_<attribute>[<index>]` e.g. `meta.dnsimple_record_name` and `meta.dnsimple_record_name[1]`. The `0` index is reserved for the default record. This due to a constraint in the Consul service definition that does not allow for complex types in the `meta` section. In addition the `meta` section allows up to 64 key/value pairs, so the amount of records per service would be limited. As a result we have added the ability to set default values for the `dnsimple_zone_name`, `dnsimple_record_type` and `dnsimple_record_ttl` in the service meta definition.
 * Documented the filtering required when using Consul connect proxies
 
 ## 0.2.0

README.md

@@ -21,7 +21,7 @@ Define the task in your HCL config file like so:
 # Task Block
 task {
   name        = "dnsimple-task"
-  description = "Create/delete/update DNS records"
+  description = "Dynamic DNS record management in DNSimple based on Consul service metadata"
   module      = "dnsimple/cts/dnsimple"
 
   condition "services" {
@@ -34,13 +34,61 @@ task {
 
 For a complete example of what a HCL config file might look like refer to [test/cts-config.hcl](test/cts-config.hcl)
 
-Ensure the services that you have specified have the following parameters added in their `meta` section:
+The services that you have specified can have the following parameters added in their `meta` section:
 
-* `meta.zone_name`:`string` - The zone (domain name) that is managed through DNSimple. e.g. `vegan.pizza`
-* `meta.record_name`:`string` - A valid label to create an A record in the specified zone. e.g. `api` which will result in `api.vegan.pizza`
-* `meta.record_ttl`:`string` - (Optional) Valid TTL value which will be used for the A record. e.g. `600` - 10 minutes
+* `meta.dnsimple_default_zone`:`string` - (Optional) The default zone (domain name) that will be used for all records where the `meta.dnsimple_zone_name` is not specified. e.g. `vegan.pizza`
+* `meta.dnsimple_default_record_type`:`string` - (Optional) The default record type that will be used for all records where the `meta.dnsimple_record_type` is not specified. (Default: `A`)
+* `meta.dnsimple_default_record_ttl`:`string` - (Optional) The default record TTL that will be used for all records where the `meta.dnsimple_record_ttl` is not specified. (Default: `3600`)
 
-For an exmaple please refer to [test/web-service.json](test/web-service.json). And [DNSimple Provider](https://www.terraform.io/docs/providers/dnsimple/index.html).
+* `meta.dnsimple_zone_name`:`string` - (Optional) The zone (domain name) that will be used to create the record. And is only optional if the `meta.dnsimple_default_zone` is specified. e.g. `vegan.pizza`
+* `meta.dnsimple_record_name`:`string` - (Required) A valid label to create an the record in the specified zone. e.g. `api` which will result in `api.vegan.pizza`
+* `meta.dnsimple_record_ttl`:`string` - (Optional) Valid TTL value which will be used for the record. e.g. `600` - 10 minutes (Default: `3600` - 1 hour)
+* `meta.dnsimple_record_type`:`string` - (Optional) Valid record type which will be used for the record. And is only optional if the `meta.dnsimple_default_record_type` is specified. e.g. `A` (Default: `A`)
+* `meta.dnsimple_record_content`:`string` - (Optional) Valid record content which will be used for the record. e.g. `$address`, `127.0.0.1`
+
+To create more records for a single service you can add an index (max index value is 60) to the above attributes for example:
+
+```hcl
+meta = {
+  dnsimple_zone_name = "vegan.pizza"
+  dnsimple_record_name = "api"
+  dnsimple_record_content = "$address"
+  dnsimple_record_ttl = "600"
+
+  dnsimple_zone_name[1] = "meatlover.pizza"
+  dnsimple_record_name[1] = "api-internal"
+  dnsimple_record_content[1] = "$node_address"
+  dnsimple_record_ttl[1] = "300"
+}
+```
+
+**Attribute Expansion**
+
+The **meta.dnsimple_record_name** and **meta.dnsimple_record_content** attributes support attribute expansion. To expand an attribute you can use the following syntax:
+
+```
+$<attribute>
+```
+
+Where the `<attribute>` is one of the following service attributes:
+
+```hcl
+id        = string
+name      = string
+address   = string
+port      = number
+namespace = string
+status    = string
+
+node                  = string
+node_id               = string
+node_address          = string
+node_datacenter       = string
+```
+
+If you would like to see more attributes supported please raise an issue.
+
+For more exmaples please refer to [test/web-service.json](test/web-service.json). And [DNSimple Provider](https://www.terraform.io/docs/providers/dnsimple/index.html).
 
 NOTE: In the example above as part of the filtering we exclude all events that are part of a Consul **connect proxy** service, as those events will result in duplication of DNS records.
 
@@ -51,15 +99,15 @@ Please refer to [CONTRIBUTING.md](CONTRIBUTING.md).
 <!-- BEGIN_TF_DOCS -->
 ## Requirements
 
-| Name                                                                   | Version   |
-| ---------------------------------------------------------------------- | --------- |
-| <a name="requirement_dnsimple"></a> [dnsimple](#requirement\_dnsimple) | >= 0.13.x |
+| Name                                                                   | Version  |
+| ---------------------------------------------------------------------- | -------- |
+| <a name="requirement_dnsimple"></a> [dnsimple](#requirement\_dnsimple) | >= 1.3.0 |
 
 ## Providers
 
-| Name                                                             | Version   |
-| ---------------------------------------------------------------- | --------- |
-| <a name="provider_dnsimple"></a> [dnsimple](#provider\_dnsimple) | >= 0.13.x |
+| Name                                                             | Version  |
+| ---------------------------------------------------------------- | -------- |
+| <a name="provider_dnsimple"></a> [dnsimple](#provider\_dnsimple) | >= 1.3.0 |
 
 ## Modules
 
@@ -82,7 +130,7 @@ No modules.
 
 ## Outputs
 
-| Name                                                                    | Description |
-| ----------------------------------------------------------------------- | ----------- |
-| <a name="output_service_map"></a> [service\_map](#output\_service\_map) | n/a         |
+| Name                                                                                | Description |
+| ----------------------------------------------------------------------------------- | ----------- |
+| <a name="output_service_records"></a> [service\_records](#output\_service\_records) | n/a         |
 <!-- END_TF_DOCS -->

main.tf

@@ -2,7 +2,12 @@ terraform {
   required_providers {
     dnsimple = {
       source  = "dnsimple/dnsimple"
-      version = ">= 0.13"
+      version = ">= 1.0"
+    }
+
+    util = {
+      source  = "poseidon/util"
+      version = "0.2.2"
     }
   }
 }
@@ -14,26 +19,45 @@ provider "dnsimple" {
   user_agent = "DNSimple-Consul-Terraform"
 }
 
-# Add a record to a service specific domain
-resource "dnsimple_zone_record" "records_a" {
+module "service_records" {
+  source = "./modules/service-records"
+
   for_each = local.consul_services
 
-  zone_name = each.value.zone_name
-  name      = each.value.record_name
-  value     = each.value.address
-  type      = "A"
-  ttl       = each.value.record_ttl
+  consul_service = each.value.service
+  defaults       = each.value.defaults
+
+  zone_records = each.value.zone_records
 }
 
 locals {
   consul_services = {
     for id, service in var.services :
     id => {
-      "name"        = service.name,
-      "address"     = service.address,
-      "zone_name"   = service.meta["zone_name"],
-      "record_name" = service.meta["record_name"],
-      "record_ttl"  = lookup(service.meta, "record_ttl", 3600),
+      "name" = service.name,
+      "defaults" = {
+        "zone_name"   = lookup(service.meta, "dnsimple_default_zone", null),
+        "record_type" = lookup(service.meta, "dnsimple_default_record_type", "A"),
+        "record_ttl"  = lookup(service.meta, "dnsimple_default_record_ttl", 3600),
+      },
+      "zone_records" = concat([{
+        "zone_name"      = lookup(service.meta, "dnsimple_zone_name", null),
+        "record_name"    = lookup(service.meta, "dnsimple_record_name", null),
+        "record_content" = lookup(service.meta, "dnsimple_record_content", null),
+        "record_type"    = lookup(service.meta, "dnsimple_record_type", null),
+        "record_ttl"     = lookup(service.meta, "dnsimple_record_ttl", null),
+        }], [
+        for n in range(0, 60) :
+        {
+          "zone_name"      = lookup(service.meta, "dnsimple_zone_name[${n}]", null),
+          "record_name"    = lookup(service.meta, "dnsimple_record_name[${n}]", null),
+          "record_content" = lookup(service.meta, "dnsimple_record_content[${n}]", null),
+          "record_type"    = lookup(service.meta, "dnsimple_record_type[${n}]", null),
+          "record_ttl"     = lookup(service.meta, "dnsimple_record_ttl[${n}]", null),
+        } if lookup(service.meta, "dnsimple_record_content[${n}]", null) != null
+      ])
+
+      "service" = service,
     }
   }
 }

modules/service-records/main.tf

@@ -0,0 +1,57 @@
+terraform {
+  required_providers {
+    dnsimple = {
+      source  = "dnsimple/dnsimple"
+      version = ">= 1.0"
+    }
+
+    util = {
+      source  = "poseidon/util"
+      version = "0.2.2"
+    }
+  }
+}
+
+locals {
+  zone_records_with_defaults = [
+    for record in var.zone_records :
+    {
+      "zone_name"      = record["zone_name"] == null ? var.defaults["zone_name"] : record["zone_name"],
+      "record_name"    = record["record_name"]
+      "record_content" = record["record_content"],
+      "record_type"    = record["record_type"] == null ? var.defaults["record_type"] : record["record_type"],
+      "record_ttl"     = record["record_ttl"] == null ? var.defaults["record_ttl"] : record["record_ttl"],
+    }
+  ]
+
+  zone_records = {
+    for record in local.zone_records_with_defaults :
+    join("", [record["record_name"], record["record_type"], record["record_content"], record["zone_name"]]) => record
+  }
+}
+
+data "util_replace" "record_names" {
+  for_each = local.zone_records
+
+  content      = each.value["record_name"]
+  replacements = { for match in regexall("\\$\\w+_?\\w+", each.value["record_name"]) : match => try(var.consul_service[replace(match, "$", "")], null) }
+}
+
+data "util_replace" "record_contents" {
+  for_each = local.zone_records
+
+  content      = each.value["record_content"]
+  replacements = { for match in regexall("\\$\\w+_?\\w+", each.value["record_content"]) : match => try(var.consul_service[replace(match, "$", "")], null) }
+}
+
+resource "dnsimple_zone_record" "consul_service_records" {
+  for_each = local.zone_records
+
+  zone_name = each.value.zone_name
+
+  name  = data.util_replace.record_names[each.key].replaced
+  value = data.util_replace.record_contents[each.key].replaced
+
+  type = each.value.record_type
+  ttl  = each.value.record_ttl
+}

modules/service-records/output.tf

@@ -0,0 +1,10 @@
+output "defaults" {
+  value = var.defaults
+}
+
+output "service_record_map" {
+  value = [
+    for record in dnsimple_zone_record.consul_service_records :
+    "${record.zone_name}:${record.name}:${record.type}:${record.ttl}:${record.value}"
+  ]
+}

modules/service-records/variables.tf

@@ -0,0 +1,42 @@
+variable "zone_records" {
+  type = list(object({
+    zone_name      = optional(string)
+    record_name    = string
+    record_content = string
+    record_type    = optional(string)
+    record_ttl     = optional(number)
+  }))
+}
+
+variable "consul_service" {
+  type = object({
+    id        = string
+    name      = string
+    address   = string
+    port      = number
+    meta      = map(string)
+    tags      = list(string)
+    namespace = string
+    status    = string
+
+    node                  = string
+    node_id               = string
+    node_address          = string
+    node_datacenter       = string
+    node_tagged_addresses = map(string)
+    node_meta             = map(string)
+  })
+
+}
+
+variable "defaults" {
+  type = object({
+    zone_name   = optional(string)
+    record_type = optional(string)
+    record_ttl  = optional(number)
+  })
+  default = {
+    record_ttl  = 3600
+    record_type = "A"
+  }
+}

output.tf

@@ -1,5 +1,3 @@
-output "service_map" {
-  value = [for id, s in local.consul_services :
-    "${id}:${s.record_name}.${s.zone_name}:A:${s.record_ttl}:${s.address}"
-  ]
+output "service_records" {
+  value = module.service_records
 }

test/cts-config.hcl

@@ -34,7 +34,7 @@ driver "terraform" {
 # Task Block
 task {
   name        = "dnsimple-task"
-  description = "Create/delete/update DNS records"
+  description = "Dynamic DNS record management in DNSimple based on Consul service metadata"
   module      = "/dnsimple-consul"
 
   condition "services" {