Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add RDS cluster activity stream #407

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -248,6 +248,7 @@ No modules.
| [aws_iam_role.rds_enhanced_monitoring](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
| [aws_iam_role_policy_attachment.rds_enhanced_monitoring](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_rds_cluster.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/rds_cluster) | resource |
| [aws_rds_cluster_activity_stream.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/rds_cluster_activity_stream) | resource |
| [aws_rds_cluster_endpoint.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/rds_cluster_endpoint) | resource |
| [aws_rds_cluster_instance.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/rds_cluster_instance) | resource |
| [aws_rds_cluster_parameter_group.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/rds_cluster_parameter_group) | resource |
Expand Down Expand Up @@ -286,12 +287,15 @@ No modules.
| <a name="input_copy_tags_to_snapshot"></a> [copy\_tags\_to\_snapshot](#input\_copy\_tags\_to\_snapshot) | Copy all Cluster `tags` to snapshots | `bool` | `null` | no |
| <a name="input_create"></a> [create](#input\_create) | Whether cluster should be created (affects nearly all resources) | `bool` | `true` | no |
| <a name="input_create_cloudwatch_log_group"></a> [create\_cloudwatch\_log\_group](#input\_create\_cloudwatch\_log\_group) | Determines whether a CloudWatch log group is created for each `enabled_cloudwatch_logs_exports` | `bool` | `false` | no |
| <a name="input_create_db_cluster_activity_stream"></a> [create\_db\_cluster\_activity\_stream](#input\_create\_db\_cluster\_activity\_stream) | Determines whether a cluster activity stream is created. | `bool` | `false` | no |
| <a name="input_create_db_cluster_parameter_group"></a> [create\_db\_cluster\_parameter\_group](#input\_create\_db\_cluster\_parameter\_group) | Determines whether a cluster parameter should be created or use existing | `bool` | `false` | no |
| <a name="input_create_db_parameter_group"></a> [create\_db\_parameter\_group](#input\_create\_db\_parameter\_group) | Determines whether a DB parameter should be created or use existing | `bool` | `false` | no |
| <a name="input_create_db_subnet_group"></a> [create\_db\_subnet\_group](#input\_create\_db\_subnet\_group) | Determines whether to create the database subnet group or use existing | `bool` | `false` | no |
| <a name="input_create_monitoring_role"></a> [create\_monitoring\_role](#input\_create\_monitoring\_role) | Determines whether to create the IAM role for RDS enhanced monitoring | `bool` | `true` | no |
| <a name="input_create_security_group"></a> [create\_security\_group](#input\_create\_security\_group) | Determines whether to create security group for RDS cluster | `bool` | `true` | no |
| <a name="input_database_name"></a> [database\_name](#input\_database\_name) | Name for an automatically created database on cluster creation | `string` | `null` | no |
| <a name="input_db_cluster_activity_stream_kms_key_id"></a> [db\_cluster\_activity\_stream\_kms\_key\_id](#input\_db\_cluster\_activity\_stream\_kms\_key\_id) | The AWS KMS key identifier for encrypting messages in the database activity stream | `string` | `null` | no |
| <a name="input_db_cluster_activity_stream_mode"></a> [db\_cluster\_activity\_stream\_mode](#input\_db\_cluster\_activity\_stream\_mode) | Specifies the mode of the database activity stream. Database events such as a change or access generate an activity stream event. One of: sync, async | `string` | `null` | no |
| <a name="input_db_cluster_db_instance_parameter_group_name"></a> [db\_cluster\_db\_instance\_parameter\_group\_name](#input\_db\_cluster\_db\_instance\_parameter\_group\_name) | Instance parameter group to associate with all instances of the DB cluster. The `db_cluster_db_instance_parameter_group_name` is only valid in combination with `allow_major_version_upgrade` | `string` | `null` | no |
| <a name="input_db_cluster_instance_class"></a> [db\_cluster\_instance\_class](#input\_db\_cluster\_instance\_class) | The compute and memory capacity of each DB instance in the Multi-AZ DB cluster, for example db.m6g.xlarge. Not all DB instance classes are available in all AWS Regions, or for all database engines | `string` | `null` | no |
| <a name="input_db_cluster_parameter_group_description"></a> [db\_cluster\_parameter\_group\_description](#input\_db\_cluster\_parameter\_group\_description) | The description of the DB cluster parameter group. Defaults to "Managed by Terraform" | `string` | `null` | no |
Expand All @@ -312,6 +316,7 @@ No modules.
| <a name="input_endpoints"></a> [endpoints](#input\_endpoints) | Map of additional cluster endpoints and their attributes to be created | `any` | `{}` | no |
| <a name="input_engine"></a> [engine](#input\_engine) | The name of the database engine to be used for this DB cluster. Defaults to `aurora`. Valid Values: `aurora`, `aurora-mysql`, `aurora-postgresql` | `string` | `null` | no |
| <a name="input_engine_mode"></a> [engine\_mode](#input\_engine\_mode) | The database engine mode. Valid values: `global`, `multimaster`, `parallelquery`, `provisioned`, `serverless`. Defaults to: `provisioned` | `string` | `"provisioned"` | no |
| <a name="input_engine_native_audit_fields_included"></a> [engine\_native\_audit\_fields\_included](#input\_engine\_native\_audit\_fields\_included) | Specifies whether the database activity stream includes engine-native audit fields. This option only applies to an Oracle DB instance. By default, no engine-native audit fields are included | `bool` | `false` | no |
| <a name="input_engine_version"></a> [engine\_version](#input\_engine\_version) | The database engine version. Updating this argument results in an outage | `string` | `null` | no |
| <a name="input_final_snapshot_identifier"></a> [final\_snapshot\_identifier](#input\_final\_snapshot\_identifier) | The name of your final DB snapshot when this DB cluster is deleted. If omitted, no final snapshot will be made | `string` | `null` | no |
| <a name="input_global_cluster_identifier"></a> [global\_cluster\_identifier](#input\_global\_cluster\_identifier) | The global cluster identifier specified on `aws_rds_global_cluster` | `string` | `null` | no |
Expand Down Expand Up @@ -389,6 +394,7 @@ No modules.
| <a name="output_cluster_reader_endpoint"></a> [cluster\_reader\_endpoint](#output\_cluster\_reader\_endpoint) | A read-only endpoint for the cluster, automatically load-balanced across replicas |
| <a name="output_cluster_resource_id"></a> [cluster\_resource\_id](#output\_cluster\_resource\_id) | The RDS Cluster Resource ID |
| <a name="output_cluster_role_associations"></a> [cluster\_role\_associations](#output\_cluster\_role\_associations) | A map of IAM roles associated with the cluster and their attributes |
| <a name="output_db_cluster_activity_stream_kinesis_stream_name"></a> [db\_cluster\_activity\_stream\_kinesis\_stream\_name](#output\_db\_cluster\_activity\_stream\_kinesis\_stream\_name) | The name of the Amazon Kinesis data stream to be used for the database activity stream |
| <a name="output_db_cluster_cloudwatch_log_groups"></a> [db\_cluster\_cloudwatch\_log\_groups](#output\_db\_cluster\_cloudwatch\_log\_groups) | Map of CloudWatch log groups created and their attributes |
| <a name="output_db_cluster_parameter_group_arn"></a> [db\_cluster\_parameter\_group\_arn](#output\_db\_cluster\_parameter\_group\_arn) | The ARN of the DB cluster parameter group created |
| <a name="output_db_cluster_parameter_group_id"></a> [db\_cluster\_parameter\_group\_id](#output\_db\_cluster\_parameter\_group\_id) | The ID of the DB cluster parameter group created |
Expand Down
3 changes: 3 additions & 0 deletions examples/mysql/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,9 @@ Note that this example may create resources which cost money. Run `terraform des
| Name | Source | Version |
|------|--------|---------|
| <a name="module_aurora"></a> [aurora](#module\_aurora) | ../../ | n/a |
| <a name="module_kms"></a> [kms](#module\_kms) | terraform-aws-modules/kms/aws | ~> 2.0 |
| <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | ~> 5.0 |
| <a name="module_vpc_endpoints"></a> [vpc\_endpoints](#module\_vpc\_endpoints) | terraform-aws-modules/vpc/aws//modules/vpc-endpoints | ~> 5.0 |

## Resources

Expand Down Expand Up @@ -63,6 +65,7 @@ No inputs.
| <a name="output_cluster_reader_endpoint"></a> [cluster\_reader\_endpoint](#output\_cluster\_reader\_endpoint) | A read-only endpoint for the cluster, automatically load-balanced across replicas |
| <a name="output_cluster_resource_id"></a> [cluster\_resource\_id](#output\_cluster\_resource\_id) | The RDS Cluster Resource ID |
| <a name="output_cluster_role_associations"></a> [cluster\_role\_associations](#output\_cluster\_role\_associations) | A map of IAM roles associated with the cluster and their attributes |
| <a name="output_db_cluster_activity_stream_kinesis_stream_name"></a> [db\_cluster\_activity\_stream\_kinesis\_stream\_name](#output\_db\_cluster\_activity\_stream\_kinesis\_stream\_name) | The name of the Amazon Kinesis data stream to be used for the database activity stream |
| <a name="output_db_cluster_cloudwatch_log_groups"></a> [db\_cluster\_cloudwatch\_log\_groups](#output\_db\_cluster\_cloudwatch\_log\_groups) | Map of CloudWatch log groups created and their attributes |
| <a name="output_db_cluster_parameter_group_arn"></a> [db\_cluster\_parameter\_group\_arn](#output\_db\_cluster\_parameter\_group\_arn) | The ARN of the DB cluster parameter group created |
| <a name="output_db_cluster_parameter_group_id"></a> [db\_cluster\_parameter\_group\_id](#output\_db\_cluster\_parameter\_group\_id) | The ID of the DB cluster parameter group created |
Expand Down
55 changes: 55 additions & 0 deletions examples/mysql/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,12 @@ module "aurora" {
vpc_ingress = {
cidr_blocks = module.vpc.private_subnets_cidr_blocks
}
kms_vpc_endpoint = {
type = "egress"
from_port = 443
to_port = 443
source_security_group_id = module.vpc_endpoints.security_group_id
}
}

apply_immediately = true
Expand Down Expand Up @@ -142,6 +148,12 @@ module "aurora" {

enabled_cloudwatch_logs_exports = ["audit", "error", "general", "slowquery"]

create_db_cluster_activity_stream = true
db_cluster_activity_stream_kms_key_id = module.kms.key_id

# https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/DBActivityStreams.Overview.html#DBActivityStreams.Overview.sync-mode
db_cluster_activity_stream_mode = "async"

tags = local.tags
}

Expand All @@ -163,3 +175,46 @@ module "vpc" {

tags = local.tags
}

module "kms" {
source = "terraform-aws-modules/kms/aws"
version = "~> 2.0"

deletion_window_in_days = 7
description = "KMS key for ${local.name} cluster activity stream."
enable_key_rotation = true
is_enabled = true
key_usage = "ENCRYPT_DECRYPT"

aliases = [local.name]

tags = local.tags
}

# https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/DBActivityStreams.Prereqs.html#DBActivityStreams.Prereqs.KMS
module "vpc_endpoints" {
source = "terraform-aws-modules/vpc/aws//modules/vpc-endpoints"
version = "~> 5.0"

vpc_id = module.vpc.vpc_id

create_security_group = true
security_group_name_prefix = "${local.name}-vpc-endpoints-"
security_group_description = "VPC endpoint security group"
security_group_rules = {
ingress_https = {
description = "HTTPS from VPC"
cidr_blocks = [module.vpc.vpc_cidr_block]
}
}

endpoints = {
kms = {
service = "kms"
private_dns_enabled = true
subnet_ids = module.vpc.database_subnets
}
}

tags = local.tags
}
9 changes: 9 additions & 0 deletions examples/mysql/outputs.tf
Original file line number Diff line number Diff line change
Expand Up @@ -157,3 +157,12 @@ output "db_cluster_cloudwatch_log_groups" {
description = "Map of CloudWatch log groups created and their attributes"
value = module.aurora.db_cluster_cloudwatch_log_groups
}

################################################################################
# Cluster Activity Stream
################################################################################

output "db_cluster_activity_stream_kinesis_stream_name" {
description = "The name of the Amazon Kinesis data stream to be used for the database activity stream"
value = module.aurora.db_cluster_activity_stream_kinesis_stream_name
}
2 changes: 2 additions & 0 deletions examples/postgresql/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ Note that this example may create resources which cost money. Run `terraform des
| Name | Source | Version |
|------|--------|---------|
| <a name="module_aurora"></a> [aurora](#module\_aurora) | ../../ | n/a |
| <a name="module_kms"></a> [kms](#module\_kms) | terraform-aws-modules/kms/aws | ~> 2.0 |
| <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | ~> 5.0 |

## Resources
Expand Down Expand Up @@ -63,6 +64,7 @@ No inputs.
| <a name="output_cluster_reader_endpoint"></a> [cluster\_reader\_endpoint](#output\_cluster\_reader\_endpoint) | A read-only endpoint for the cluster, automatically load-balanced across replicas |
| <a name="output_cluster_resource_id"></a> [cluster\_resource\_id](#output\_cluster\_resource\_id) | The RDS Cluster Resource ID |
| <a name="output_cluster_role_associations"></a> [cluster\_role\_associations](#output\_cluster\_role\_associations) | A map of IAM roles associated with the cluster and their attributes |
| <a name="output_db_cluster_activity_stream_kinesis_stream_name"></a> [db\_cluster\_activity\_stream\_kinesis\_stream\_name](#output\_db\_cluster\_activity\_stream\_kinesis\_stream\_name) | The name of the Amazon Kinesis data stream to be used for the database activity stream |
| <a name="output_db_cluster_cloudwatch_log_groups"></a> [db\_cluster\_cloudwatch\_log\_groups](#output\_db\_cluster\_cloudwatch\_log\_groups) | Map of CloudWatch log groups created and their attributes |
| <a name="output_db_cluster_parameter_group_arn"></a> [db\_cluster\_parameter\_group\_arn](#output\_db\_cluster\_parameter\_group\_arn) | The ARN of the DB cluster parameter group created |
| <a name="output_db_cluster_parameter_group_id"></a> [db\_cluster\_parameter\_group\_id](#output\_db\_cluster\_parameter\_group\_id) | The ID of the DB cluster parameter group created |
Expand Down
19 changes: 19 additions & 0 deletions examples/postgresql/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,10 @@ module "aurora" {
enabled_cloudwatch_logs_exports = ["postgresql"]
create_cloudwatch_log_group = true

create_db_cluster_activity_stream = true
db_cluster_activity_stream_kms_key_id = module.kms.key_id
db_cluster_activity_stream_mode = "async"

tags = local.tags
}

Expand All @@ -129,3 +133,18 @@ module "vpc" {

tags = local.tags
}

module "kms" {
source = "terraform-aws-modules/kms/aws"
version = "~> 2.0"

deletion_window_in_days = 7
description = "KMS key for ${local.name} cluster activity stream."
enable_key_rotation = true
is_enabled = true
key_usage = "ENCRYPT_DECRYPT"

aliases = [local.name]

tags = local.tags
}
9 changes: 9 additions & 0 deletions examples/postgresql/outputs.tf
Original file line number Diff line number Diff line change
Expand Up @@ -157,3 +157,12 @@ output "db_cluster_cloudwatch_log_groups" {
description = "Map of CloudWatch log groups created and their attributes"
value = module.aurora.db_cluster_cloudwatch_log_groups
}

################################################################################
# Cluster Activity Stream
################################################################################

output "db_cluster_activity_stream_kinesis_stream_name" {
description = "The name of the Amazon Kinesis data stream to be used for the database activity stream"
value = module.aurora.db_cluster_activity_stream_kinesis_stream_name
}
15 changes: 15 additions & 0 deletions main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -412,3 +412,18 @@ resource "aws_cloudwatch_log_group" "this" {

tags = var.tags
}

################################################################################
# Cluster Activity Stream
################################################################################

resource "aws_rds_cluster_activity_stream" "this" {
count = local.create && var.create_db_cluster_activity_stream ? 1 : 0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a limitation on the activity stream as it's not supported by all the instances types.
works only on few of them per this doc : https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/DBActivityStreams.Overview.html

how is that taken into account ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not - that's up to users


resource_arn = aws_rds_cluster.this[0].arn
mode = var.db_cluster_activity_stream_mode
kms_key_id = var.db_cluster_activity_stream_kms_key_id
engine_native_audit_fields_included = var.engine_native_audit_fields_included

depends_on = [aws_rds_cluster_instance.this]
}
9 changes: 9 additions & 0 deletions outputs.tf
Original file line number Diff line number Diff line change
Expand Up @@ -170,3 +170,12 @@ output "db_cluster_cloudwatch_log_groups" {
description = "Map of CloudWatch log groups created and their attributes"
value = aws_cloudwatch_log_group.this
}

################################################################################
# Cluster Activity Stream
################################################################################

output "db_cluster_activity_stream_kinesis_stream_name" {
description = "The name of the Amazon Kinesis data stream to be used for the database activity stream"
value = try(aws_rds_cluster_activity_stream.this[0].kinesis_stream_name, null)
}
28 changes: 28 additions & 0 deletions variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -683,3 +683,31 @@ variable "cloudwatch_log_group_kms_key_id" {
type = string
default = null
}

################################################################################
# Cluster Activity Stream
################################################################################

variable "create_db_cluster_activity_stream" {
description = "Determines whether a cluster activity stream is created."
type = bool
default = false
}

variable "db_cluster_activity_stream_mode" {
description = "Specifies the mode of the database activity stream. Database events such as a change or access generate an activity stream event. One of: sync, async"
type = string
default = null
}

variable "db_cluster_activity_stream_kms_key_id" {
description = "The AWS KMS key identifier for encrypting messages in the database activity stream"
type = string
default = null
}

variable "engine_native_audit_fields_included" {
description = "Specifies whether the database activity stream includes engine-native audit fields. This option only applies to an Oracle DB instance. By default, no engine-native audit fields are included"
type = bool
default = false
}