Skip to content

Latest commit

 

History

History
116 lines (81 loc) · 4.46 KB

README.md

File metadata and controls

116 lines (81 loc) · 4.46 KB
Raw

terraform-google-composer

This module makes it easy to create a Cloud Composer Environment. As the module develops, this README should be updated.

The resources/services/activations/deletions that this module will create/trigger are:

  • Create a GCP Composer Environment

Compatibility

This module is meant for use with Terraform 1.3+ and tested using Terraform 1.3+. If you find incompatibilities using Terraform >=1.3, please open an issue.

Version

Current version is 4.0. Upgrade guides:

Usage

Cloud Composer currently has two versions: V2 has greater flexibility in the Airflow core services (scheduler, webserver, worker) and has a more practical and scalable infrastructure. Therefore, we recommend prioritizing the use of V2 for new environments.

You can find an overview of the product here and the list of major differences. Plans for Cloud Composer V1 end of support is documented here.

Simple usage is as follows:

module "composer" {
  source  = "terraform-google-modules/composer/google"
  version = "~> 5.2"

  project_id        = "<PROJECT ID>"
  region            = "us-central1"
  composer_env_name = "composer-env-test"
  network           = "test-network"
  subnetwork        = "composer-subnet"
  enable_private_endpoint = false
}

Functional examples are included in the examples directory.

Inputs

Name Description Type Default Required
composer_env_name Name of Cloud Composer Environment string n/a yes
enable_private_endpoint Configure public access to the cluster endpoint. bool false no
network Network where Cloud Composer is created. string n/a yes
project_id Project ID where Cloud Composer Environment is created. string n/a yes
region Region where the Cloud Composer Environment is created. string n/a yes
subnetwork Subetwork where Cloud Composer is created. string n/a yes
zone Zone where the Cloud Composer Environment is created. string n/a yes

Outputs

Name Description
airflow_uri URI of the Apache Airflow Web UI hosted within the Cloud Composer Environment.
composer_env_id ID of Cloud Composer Environment.
composer_env_name The name of the Cloud Composer Environment.
gcs_bucket Google Cloud Storage bucket which hosts DAGs for the Cloud Composer Environment.
gke_cluster Google Kubernetes Engine cluster used to run the Cloud Composer Environment.

Requirements

These sections describe requirements for using this module.

Software

The following dependencies must be available:

Service Account

A service account with the following roles must be used to provision the resources of this module:

  • Project Editor: roles/editor
  • Network Admin: roles/compute.networkAdmin
  • Instance Admin: roles/compute.instanceAdmin.v1
  • Service Account User: roles/iam.serviceAccountUser
  • Composer Worker: roles/composer.worker

The Project Factory module and the IAM module may be used in combination to provision a service account with the necessary roles applied.

APIs

A project with the following APIs enabled must be used to host the resources of this module:

  • Cloud Composer API: composer.googleapis.com

The Project Factory module can be used to provision a project with the necessary APIs enabled.

Contributing

Refer to the contribution guidelines for information on contributing to this module.