generated from clowdhaus/terraform-aws-module-template
-
-
Notifications
You must be signed in to change notification settings - Fork 101
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: Add Serverless replication task support (#56)
- Loading branch information
Showing
13 changed files
with
605 additions
and
6 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,4 @@ | ||
| # Terraform AWS DMS examples | ||
|
|
||
| - [Complete](https://github.com/terraform-aws-modules/terraform-aws-dms/tree/master/examples/complete) | ||
| - [Serverless](https://github.com/terraform-aws-modules/terraform-aws-dms/tree/master/examples/serverless) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
| # Serverless AWS DMS Example | ||
|
|
||
| Configuration in this directory creates: | ||
|
|
||
| - AWS IAM roles [necessary for AWS DMS](https://aws.amazon.com/premiumsupport/knowledge-center/dms-redshift-connectivity-failures/) | ||
| - [AWS DMS subnet group](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_ReplicationInstance.VPC.html) | ||
| - [AWS DMS serverless replication configuration](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Serverless.html) | ||
| - Two [AWS DMS replication endpoints](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Endpoints.Creating.html) - one `source` and one `target` to migrate data from an Aurora PostgreSQL cluster to Aurora MySQL cluster | ||
| - Necessary supporting resources to demonstrate the capabilities of the module (VPC, Aurora clusters, security groups, etc.) | ||
|
|
||
| ## Usage | ||
|
|
||
| To run this example you need to execute: | ||
|
|
||
| ```bash | ||
| $ terraform init | ||
| $ terraform plan | ||
| $ terraform apply | ||
| ``` | ||
|
|
||
| Note that this example may create resources which will incur monetary charges on your AWS bill. Run `terraform destroy` when you no longer need these resources. | ||
|
|
||
| <!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK --> | ||
| ## Requirements | ||
|
|
||
| | Name | Version | | ||
| |------|---------| | ||
| | <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.0 | | ||
| | <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 5.32 | | ||
| | <a name="requirement_random"></a> [random](#requirement\_random) | >= 3.0 | | ||
|
|
||
| ## Providers | ||
|
|
||
| | Name | Version | | ||
| |------|---------| | ||
| | <a name="provider_aws"></a> [aws](#provider\_aws) | >= 5.32 | | ||
|
|
||
| ## Modules | ||
|
|
||
| | Name | Source | Version | | ||
| |------|--------|---------| | ||
| | <a name="module_dms_aurora_postgresql_aurora_mysql"></a> [dms\_aurora\_postgresql\_aurora\_mysql](#module\_dms\_aurora\_postgresql\_aurora\_mysql) | ../.. | n/a | | ||
| | <a name="module_rds_aurora"></a> [rds\_aurora](#module\_rds\_aurora) | terraform-aws-modules/rds-aurora/aws | ~> 8.0 | | ||
| | <a name="module_secrets_manager_mysql"></a> [secrets\_manager\_mysql](#module\_secrets\_manager\_mysql) | terraform-aws-modules/secrets-manager/aws | ~> 1.0 | | ||
| | <a name="module_secrets_manager_postgresql"></a> [secrets\_manager\_postgresql](#module\_secrets\_manager\_postgresql) | terraform-aws-modules/secrets-manager/aws | ~> 1.0 | | ||
| | <a name="module_security_group"></a> [security\_group](#module\_security\_group) | terraform-aws-modules/security-group/aws | ~> 5.0 | | ||
| | <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | ~> 5.0 | | ||
| | <a name="module_vpc_endpoints"></a> [vpc\_endpoints](#module\_vpc\_endpoints) | terraform-aws-modules/vpc/aws//modules/vpc-endpoints | ~> 5.0 | | ||
|
|
||
| ## Resources | ||
|
|
||
| | Name | Type | | ||
| |------|------| | ||
| | [aws_kms_key.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kms_key) | resource | | ||
| | [aws_rds_cluster_parameter_group.postgresql](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/rds_cluster_parameter_group) | resource | | ||
| | [aws_availability_zones.available](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/availability_zones) | data source | | ||
|
|
||
| ## Inputs | ||
|
|
||
| No inputs. | ||
|
|
||
| ## Outputs | ||
|
|
||
| | Name | Description | | ||
| |------|-------------| | ||
| | <a name="output_dms_access_for_endpoint_iam_role_arn"></a> [dms\_access\_for\_endpoint\_iam\_role\_arn](#output\_dms\_access\_for\_endpoint\_iam\_role\_arn) | Amazon Resource Name (ARN) specifying the role | | ||
| | <a name="output_dms_access_for_endpoint_iam_role_id"></a> [dms\_access\_for\_endpoint\_iam\_role\_id](#output\_dms\_access\_for\_endpoint\_iam\_role\_id) | Name of the IAM role | | ||
| | <a name="output_dms_access_for_endpoint_iam_role_unique_id"></a> [dms\_access\_for\_endpoint\_iam\_role\_unique\_id](#output\_dms\_access\_for\_endpoint\_iam\_role\_unique\_id) | Stable and unique string identifying the role | | ||
| | <a name="output_dms_cloudwatch_logs_iam_role_arn"></a> [dms\_cloudwatch\_logs\_iam\_role\_arn](#output\_dms\_cloudwatch\_logs\_iam\_role\_arn) | Amazon Resource Name (ARN) specifying the role | | ||
| | <a name="output_dms_cloudwatch_logs_iam_role_id"></a> [dms\_cloudwatch\_logs\_iam\_role\_id](#output\_dms\_cloudwatch\_logs\_iam\_role\_id) | Name of the IAM role | | ||
| | <a name="output_dms_cloudwatch_logs_iam_role_unique_id"></a> [dms\_cloudwatch\_logs\_iam\_role\_unique\_id](#output\_dms\_cloudwatch\_logs\_iam\_role\_unique\_id) | Stable and unique string identifying the role | | ||
| | <a name="output_dms_vpc_iam_role_arn"></a> [dms\_vpc\_iam\_role\_arn](#output\_dms\_vpc\_iam\_role\_arn) | Amazon Resource Name (ARN) specifying the role | | ||
| | <a name="output_dms_vpc_iam_role_id"></a> [dms\_vpc\_iam\_role\_id](#output\_dms\_vpc\_iam\_role\_id) | Name of the IAM role | | ||
| | <a name="output_dms_vpc_iam_role_unique_id"></a> [dms\_vpc\_iam\_role\_unique\_id](#output\_dms\_vpc\_iam\_role\_unique\_id) | Stable and unique string identifying the role | | ||
| | <a name="output_endpoints"></a> [endpoints](#output\_endpoints) | A map of maps containing the endpoints created and their full output of attributes and values | | ||
| | <a name="output_replication_subnet_group_id"></a> [replication\_subnet\_group\_id](#output\_replication\_subnet\_group\_id) | The ID of the subnet group | | ||
| | <a name="output_s3_endpoints"></a> [s3\_endpoints](#output\_s3\_endpoints) | A map of maps containing the S3 endpoints created and their full output of attributes and values | | ||
| | <a name="output_serverless_replication_tasks"></a> [serverless\_replication\_tasks](#output\_serverless\_replication\_tasks) | A map of maps containing the serverless replication tasks created and their full output of attributes and values | | ||
| <!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK --> | ||
|
|
||
| Apache-2.0 Licensed. See [LICENSE](https://github.com/terraform-aws-modules/terraform-aws-dms/blob/master/LICENSE). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| { | ||
| "rules": [{ | ||
| "rule-type": "selection", | ||
| "rule-id": "1", | ||
| "rule-name": "1", | ||
| "object-locator": { | ||
| "schema-name": "%", | ||
| "table-name": "%" | ||
| }, | ||
| "rule-action": "include" | ||
| }] | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| { | ||
| "TargetMetadata": { | ||
| "SupportLobs": true, | ||
| "FullLobMode": false, | ||
| "LobChunkSize": 64, | ||
| "LimitedSizeLobMode": true, | ||
| "LobMaxSize": 32, | ||
| "BatchApplyEnabled": true | ||
| }, | ||
| "FullLoadSettings": { | ||
| "TargetTablePrepMode": "DO_NOTHING", | ||
| "CreatePkAfterFullLoad": false, | ||
| "StopTaskCachedChangesApplied": false, | ||
| "StopTaskCachedChangesNotApplied": false, | ||
| "MaxFullLoadSubTasks": 8, | ||
| "TransactionConsistencyTimeout": 600, | ||
| "CommitRate": 10000 | ||
| }, | ||
| "Logging": { | ||
| "EnableLogging": true | ||
| }, | ||
| "ControlTablesSettings": { | ||
| "HistoryTimeslotInMinutes": 5, | ||
| "HistoryTableEnabled": false, | ||
| "SuspendedTablesTableEnabled": false, | ||
| "StatusTableEnabled": false | ||
| }, | ||
| "StreamBufferSettings": { | ||
| "StreamBufferCount": 3, | ||
| "StreamBufferSizeInMB": 8 | ||
| }, | ||
| "ChangeProcessingTuning": { | ||
| "BatchApplyPreserveTransaction": true, | ||
| "BatchApplyTimeoutMin": 1, | ||
| "BatchApplyTimeoutMax": 30, | ||
| "BatchApplyMemoryLimit": 500, | ||
| "BatchSplitSize": 0, | ||
| "MinTransactionSize": 1000, | ||
| "CommitTimeout": 1, | ||
| "MemoryLimitTotal": 1024, | ||
| "MemoryKeepTime": 60, | ||
| "StatementCacheSize": 50 | ||
| }, | ||
| "ChangeProcessingDdlHandlingPolicy": { | ||
| "HandleSourceTableDropped": true, | ||
| "HandleSourceTableTruncated": true, | ||
| "HandleSourceTableAltered": true | ||
| }, | ||
| "ErrorBehavior": { | ||
| "DataErrorPolicy": "LOG_ERROR", | ||
| "DataTruncationErrorPolicy": "LOG_ERROR", | ||
| "DataErrorEscalationPolicy": "SUSPEND_TABLE", | ||
| "DataErrorEscalationCount": 50, | ||
| "TableErrorPolicy": "SUSPEND_TABLE", | ||
| "TableErrorEscalationPolicy": "STOP_TASK", | ||
| "TableErrorEscalationCount": 50, | ||
| "RecoverableErrorCount": 0, | ||
| "RecoverableErrorInterval": 5, | ||
| "RecoverableErrorThrottling": true, | ||
| "RecoverableErrorThrottlingMax": 1800, | ||
| "ApplyErrorDeletePolicy": "IGNORE_RECORD", | ||
| "ApplyErrorInsertPolicy": "LOG_ERROR", | ||
| "ApplyErrorUpdatePolicy": "LOG_ERROR", | ||
| "ApplyErrorEscalationPolicy": "LOG_ERROR", | ||
| "ApplyErrorEscalationCount": 0, | ||
| "FullLoadIgnoreConflicts": true | ||
| } | ||
| } |
Oops, something went wrong.