Terraform module for creating infrastructure resources needed to a cloud native Drupal on GCP

This module creates the resources needed to deploy a Cloud Native Drupal instance on Google Cloud Platform.

Prerequisites are a GCP project in which a MySQL CloudSQL instance exists, with administrator credentials needed to create the databases and users on the instance itself.

The module uses two sub-modules (gcp-application-bucket-creation-helper and gcp-mysql-db-and-user-creation-helper) that take care of database/user creation and bucket creation. The names and characteristics of the resources created are highly opinionated and configured for a Drupal project. In the event that it is necessary to create resources for a different non Drupal application, it is recommended to use and configure the individual modules.

The module accept a list of objects as input, each object represents a Drupal project and resource configuration.

The required fields for each project object are the project_name, the gitlab_project_id used to name all resources; the database_host field is also mandatory if we want to create the secrets for the database resources.

The variable structure is the following:

  {
    # The name of the project, it will be used to create the bucket name, the database name and the database user name,
    # will usually match the project gitlab path, but in case of long nomenclature or multi-site project it might be
    # different.
    project_name                    = string
    # The ID of the Drupal project in Gitlab, it is useful to identify the project the resources belong to.
    gitlab_project_id               = number
    # It is the name of the release branch and is used for naming all resources (namespaces, buckets, databases, etc.)
    release_branch_name             = optional(string, "main")
    # If not specified, the kubernetes_namespace by default it is built as
    # <project_name>-<gitlab_project_id>-<release_branch_name>.
    kubernetes_namespace            = optional(string, null)
    # Namespace labels added to default_k8s_labels
    kubernetes_namespace_labels     = optional(map(string), {})
    # The Helm release name by default corresponds to the Drupal PKG release that corresponds to
    # drupal-${CI_COMMIT_REF_SLUG}-${CI_PROJECT_ID} and is used for the name of secrets.
    helm_release_name               = optional(string, null)
    # By default the name is <project_name>_<gitlab_project_id>_<release_branch_name>_dp, where dp stands for Drupal.
    database_name                   = optional(string, null)
    # By default the name is <project_name>_<gitlab_project_id>_<release_branch_name>_dp_u, where dp_u stands
    # for Drupal user.
    database_user_name              = optional(string, null)
    # The IP of the CloudSQL instance, it's mandatory to create the secret with credentials to connect to the database.
    database_host                   = optional(string, null)
    # The port of the CloudSQL instance, default to 3306.
    database_port                   = optional(number, 3306)
    # The name of the bucket, by default it is built as <project_name>-<gitlab_project_id>-<release_branch_name>.
    bucket_name                     = optional(string, null)
    # The host of the bucket, by default for Google buckets it is storage.googleapis.com.
    bucket_host                     = optional(string, "storage.googleapis.com")
    # True by default, and is used to prevent name collision for created resources.
    bucket_append_random_suffix     = optional(bool, true)
    # The location of the bucket, by default it is the same as the project region.
    bucket_location                 = optional(string, null)
    # The storage class of the bucket (https://cloud.google.com/storage/docs/storage-classes), by default it is STANDARD.
    bucket_storage_class            = optional(string, "STANDARD")
    # The versioning of the bucket, by default it is enabled.
    bucket_enable_versioning        = optional(bool, true)
    # Here you can choose to enable or disable the disaster recovery bucket, by default it is enabled. You can disable it
    # for example for test or development environments.
    bucket_enable_disaster_recovery = optional(bool, true)
    # Set to true to enable the force destroy of the bucket, by default it is false. If true, the bucket and all its objects
    # will be deleted when the terraform resource is removed.
    bucket_force_destroy            = optional(bool, false)
    # Here you can customize the path of public files inside the drupal bucket. This values are used to create
    # the secrets for the application.
    bucket_legacy_public_files_path = optional(string, "/public")
    # The property `set_all_users_as_viewer` controls if the bucket content will be globally readable by anonymous users
    # (default false).
    bucket_set_all_users_as_viewer  = optional(bool, false)
    # Here you can also pass a map of key/value label pairs to assign to the bucket, i.e. `{ env = "stage", app = "mysite" }`.
    bucket_labels                   = optional(map(string), {})
    # You can also pass a list of tags values written in the user friendly name <TAG_KEY_SHORTNAME>/<TAG_VALUE_SHORTNAME>, 
    # i.e. `["dev/editor","ops/admin"]`) to bind to the buckets using the `tag_list` property. The tags must exist in 
    # the google project, otherwise the module will fail.
    bucket_tag_list                 = optional(list(string), [])
    # Properties bucket_obj_vwr and bucket_obj_adm set a list of specific IAM members as objectViewers and objectAdmin
    bucket_obj_adm                  = optional(list(string), [])
    bucket_obj_vwr                  = optional(list(string), [])
    #  The duration in seconds that soft-deleted objects in the bucket will be retained and cannot be permanently 
    #  deleted. Default value is 604800. 
    bucket_soft_delete_retention_seconds = optional(number, 604800)
  }

The module will create a bucket, a database and a user for each project and as output will return the application credentials for each resource.

terraform output drupal_apps_database_credentials
terraform output drupal_apps_bucket_credentials
terraform output helm_values_for_databases
terraform output helm_values_for_buckets

If you need to import an existing bucket or database/user, you can specify the bucket_name, database_name and database_user_name. You also need to disable the random suffix bucket_append_random_suffix for the bucket name.

Providers

Name	Version
google	>= 4.47.0
kubernetes	>= 2.19
random	3.6.2

Requirements

Name	Version
terraform	>= 1.2
google	>= 4.47.0
kubernetes	>= 2.19
random	3.6.2

Inputs

Name	Description	Type	Default	Required
bucket_disaster_recovery_location	The location in which the disaster recovery bucket will be created. For a list of available regions, see https://cloud.google.com/storage/docs/locations. By default, the disaster recovery bucket will be created in the same location as the primary bucket.	string	""	no
cloudsql_instance_name	The name of the existing Google CloudSQL Instance name. Actually only a MySQL 5.7 or 8 instance is supported.	string	""	no
cloudsql_privileged_user_name	The name of the privileged user of the Cloud SQL instance	string	""	no
cloudsql_privileged_user_password	The password of the privileged user of the Cloud SQL instance	string	""	no
create_buckets	If true, the module will create a bucket for each project.	bool	true	no
create_clousql_dumps_bucket	If true, the module will create a Google Storage bucket that can be used as a destination for CloudSQL dumps. The bucket will also be tagged with the global tags.	bool	false	no
create_databases_and_users	If true, the module will create a user and a database for each project.	bool	true	no
default_k8s_labels	A map of labels to be applied to all the kubernetes resources created by this module. If a resource specify a map of labels, the default labels will merged with those specified in the resource.	map(string)	{ "managed-by": "terraform" }	no
drupal_projects_list	The list of Drupal projects, add a project name and this will create all infrastructure resources needed to run your project (bucket, database, user with relative credentials). Database resources are created in the CloudSQL instance you specified. Please not that you can assign only a database to a single user, the same user cannot be assigned to multiple databases. The default values are thought for a production environment, they will need to be adjusted accordingly for a stage environment.	list(object({ project_name = string gitlab_project_id = number release_branch_name = optional(string, "main") kubernetes_namespace = optional(string, null) kubernetes_namespace_labels = optional(map(string), {}) helm_release_name = optional(string, null) database_name = optional(string, null) database_user_name = optional(string, null) database_host = optional(string, null) database_port = optional(number, 3306) bucket_name = optional(string, null) bucket_host = optional(string, "storage.googleapis.com") bucket_append_random_suffix = optional(bool, true) bucket_location = optional(string, null) bucket_storage_class = optional(string, "STANDARD") bucket_enable_versioning = optional(bool, true) bucket_enable_disaster_recovery = optional(bool, true) bucket_force_destroy = optional(bool, false) bucket_legacy_public_files_path = optional(string, "/public") bucket_set_all_users_as_viewer = optional(bool, false) bucket_labels = optional(map(string), {}) bucket_tag_list = optional(list(string), []) bucket_obj_adm = optional(list(string), []) bucket_obj_vwr = optional(list(string), []) bucket_soft_delete_retention_seconds = optional(number, 0) }))	n/a	yes
global_tags	A list of tags to be applied to all the drupal buckets, in the form <TAG_KEY_SHORTNAME>/<TAG_VALUE_SHORTNAME>. If a resource specify a list of tags, the global tags will be overridden and replaced by those specified in the resource. Please note that actually only the buckets are tagged by this module.	list(string)	[]	no
logging_bucket_name	The name of the logging bucket. If empty, no logging bucket will be added and bucket logs will be disabled.	string	""	no
project_id	The ID of the project in which the resource belongs.	string	n/a	yes
region	The region in which the resources belongs.	string	n/a	yes
use_existing_kubernetes_namespaces	If false, the module will create the various namespaces for Kubernetes resources (secrets). Set to true to prevent at a global level the namespaces creation, useful if the namespaces have been created outside of Terraform, for example, by the Helm release during the deploy of the application or in other ways.	bool	false	no

Outputs

Name	Description
cloudsql_dumps_bucket_name	CloudSQL dumps bucket name.
details_of_used_tag_keys	Details of the tag keys passed to this module.
details_of_used_tag_values	Details of the tag values passed to this module.
drupal_apps_all_bucket_credentials	Bucket credentials for each Drupal project
drupal_apps_all_bucket_secrets	Bucket kubernetes secrets for each Drupal project
drupal_apps_all_data	All data for each Drupal project.
drupal_apps_all_database_credentials	Database credentials for each Drupal project
drupal_apps_all_database_secrets	Database kubernetes secrets for each Drupal project
drupal_apps_all_namespaces	Namespace for each Drupal project
drupal_apps_bucket_credentials	Drupal apps bucket credentials for each Drupal project.
drupal_apps_database_credentials	Drupal apps database credentials for each Drupal project.
drupal_buckets_names_list	The list with the names of the Drupal buckets managed by this module.

Resources

Name	Type
google_storage_bucket.cloudsql_dumps	resource
google_storage_bucket_iam_member.cloudsql_dumps_bucket_writer	resource
google_tags_location_tag_binding.binding	resource
kubernetes_namespace.namespace	resource
kubernetes_secret.bucket_secret_name	resource
kubernetes_secret.database_secret_name	resource
random_id.cloudsql_dumps_bucket_name_suffix	resource
google_sql_database_instance.cloudsql_instance	data source
google_tags_tag_key.tag_keys	data source
google_tags_tag_value.tag_values	data source

Modules

Name	Source	Version
drupal_buckets	github.com/sparkfabrik/terraform-google-gcp-application-bucket-creation-helper	0.10.0
drupal_databases_and_users	github.com/sparkfabrik/terraform-google-gcp-mysql-db-and-user-creation-helper	0.3.2