Add docdb, neptune, mariadb, mysql, oracle, redshift (#6)

* Add docdb, neptune, mariadb, mysql, oracle, redshiftmodules * update 1.0.2 release date --------- Co-authored-by: mattJsonar <[email protected]> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
imperva · Jul 12, 2024 · acde11d · acde11d
1 parent 0b19323
commit acde11d
Show file tree

Hide file tree

Showing 160 changed files with 8,408 additions and 15 deletions.
diff --git a/.github/workflows/pull-request.yml b/.github/workflows/pull-request.yml
@@ -8,40 +8,56 @@ jobs:
     outputs: 
       directories: ${{ steps.dirs.outputs.matrix }}
     steps:
+    # Checkout the repository to the GitHub Actions runner
     - name: Checkout
-      uses: actions/checkout@v3
+      uses: actions/checkout@v4
       with:
         ref: ${{ github.event.pull_request.head.ref }}
+
+    # Find all subdirectories within "modules/" and "examples/"
     - name: Get root directories
       id: dirs
       run: echo "matrix=$(ls -d {modules,examples}/* | jq -R -s -c 'split("\n")[:-1]')" >> $GITHUB_OUTPUT
-
-  validateConfigs:
-    name: Validate terraform pull PR
-    needs: collectInputs
+
+  validatePR:
+    name: Validate PR
     runs-on: ubuntu-latest
+    needs: collectInputs
     strategy:
       matrix:
         directory: ${{ fromJson(needs.collectInputs.outputs.directories) }}
     steps:
+    # Checkout the repository to the GitHub Actions runner
     - name: Checkout
-      uses: actions/checkout@v3
+      uses: actions/checkout@v4
       with:
         ref: ${{ github.event.pull_request.head.ref }}
-    - name: Validate configs
-      uses: dflook/terraform-validate@v1
-      with:
-        path: ${{ matrix.directory }}
+
+    # Install the latest version of Terraform
+    - name: Setup Terraform
+      uses: hashicorp/setup-terraform@v2
+
+    # Initialize terraform and validate submodules
+    - name: Terraform validate
+      working-directory: ${{ matrix.directory }}
+      run: |
+        if [[ -d ".terraform" ]]; then echo "removing exisiting .terraform directory"; rm -rf .terraform; fi
+        terraform workspace new ${{ github.run_id }}
+        terraform init
+        terraform validate
 
   updateDocumentation:
     name: Update READMEs
-    needs: validateConfigs
+    needs: validatePR
     runs-on: ubuntu-latest
     steps:
+    # Checkout the repository to the GitHub Actions runner
     - name: Checkout
-      uses: actions/checkout@v3
+      uses: actions/checkout@v4
       with:
         ref: ${{ github.event.pull_request.head.ref }}
+
+    # Update READMEs using terraform-docs
     - name: Render terraform docs inside the README.md and push changes back to PR branch
       uses: terraform-docs/[email protected]
       with:
@@ -59,18 +75,25 @@ jobs:
     permissions:
       contents: write
     steps:
+    # Checkout the repository to the GitHub Actions runner
     - name: Checkout
-      uses: actions/checkout@v3
+      uses: actions/checkout@v4
       with:
         ref: ${{ github.event.pull_request.head.ref }}
+
+    # Format Terraform modules
     - name: Terraform fmt modules
       uses: dflook/terraform-fmt@v1
       with:
         path: modules/
+
+    # Format Terraform examples
     - name: Terraform fmt examples
       uses: dflook/terraform-fmt@v1
       with:
         path: examples/
+
+    # Push changes to PR
     - name: Commit changes to PR
       uses: stefanzweifel/git-auto-commit-action@v5
-
+
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,19 @@
 # Changelog
 
+## 1.0.2 (2024-07-12)
+
+### Features
+- Amazon DocumentDB module
+- Amazon Neptune module
+- Amazon Neptune Slow query module
+- Amazon RDS MariaDB module
+- Amazon RDS MySQL module
+- Amazon RDS MySQL Slow query module
+- Amazon RDS Oracle (standard auditing) module
+- Amazon RDS Oracle (unified auditing) module
+- Amazon Redshift (via ODBC) module
+- Amazon Redshift (via S3 bucket) module
+
 ## 1.0.1 (2024-06-27)
 
 ### Bug Fixes

diff --git a/DSF_VERSION_COMPATABILITY.md b/DSF_VERSION_COMPATABILITY.md
@@ -7,9 +7,45 @@ The following table lists the DSF versions that each module is tested and mainta
    <th>DSF Versions</th>
   </tr>
 
+   <tr>
+      <td>onboard-aws-docdb-cluster</td>
+      <td>4.17+</td>
+   </tr>
+   <tr>
+      <td>onboard-aws-rds-neptune</td>
+      <td>4.17+</td>
+   </tr>
+   <tr>
+      <td>onboard-aws-rds-neptune-slow-query</td>
+      <td>4.17+</td>
+   </tr>
+   <tr>
+      <td>onboard-aws-rds-mariadb</td>
+      <td>4.16+</td>
+   </tr>
+   <tr>
+      <td>onboard-aws-rds-mysql</td>
+      <td>4.16+</td>
+   </tr>
+   <tr>
+      <td>onboard-aws-rds-mysql-slow-query</td>
+      <td>4.16+</td>
+   </tr>
+   <tr>
+      <td>onboard-aws-rds-oracle-standard</td>
+      <td>4.16+</td>
+   </tr>
+   <tr>
+      <td>onboard-aws-rds-oracle-unified</td>
+      <td>4.16+</td>
+   </tr>
    <tr>
       <td>onboard-aws-rds-postgresql</td>
-      <td>4.16</td>
+      <td>4.16+</td>
+   </tr>
+   <tr>
+      <td>onboard-aws-rds-redshift</td>
+      <td>4.17+</td>
    </tr>
 
 </table>
diff --git a/examples/onboard-aws-docdb-cluster/README.md b/examples/onboard-aws-docdb-cluster/README.md
@@ -0,0 +1,42 @@
+# Onboard Amazon DocumentDB example
+This example includes additional prerequisites that will need to be completed to fully utilize the module. More details can be found in the [onboarding documentation](https://docs.imperva.com/bundle/onboarding-databases-to-sonar-reference-guide/page/Amazon-DocumentDB-Onboarding-Steps_48366944.html).
+
+This example creates both 'aws' and 'dsfhub' resources. More information regarding authentication to each can be found in the relevant provider documentation:
+- [aws](https://registry.terraform.io/providers/hashicorp/aws/latest/docs)
+- [dsfhub](https://registry.terraform.io/providers/imperva/dsfhub/latest/docs)
+
+## Prerequisites
+### Account Asset Permissions
+An AWS account asset will need to be onboarded to your DSF hub prior to using this module. The account asset will need to be granted permissions to be able to read from the newly created CloudWatch log group. 
+
+<!-- BEGIN_TF_DOCS -->
+## Requirements
+
+No requirements.
+
+## Providers
+
+No providers.
+
+## Modules
+
+| Name | Source | Version |
+|------|--------|---------|
+| <a name="module_aws-default-account-asset"></a> [aws-default-account-asset](#module\_aws-default-account-asset) | imperva/agentless-onboarding/dsfhub//modules/dsfhub-aws-cloud-account | n/a |
+| <a name="module_aws-docdb-cluster-1"></a> [aws-docdb-cluster-1](#module\_aws-docdb-cluster-1) | ../../modules/onboard-aws-docdb-cluster | n/a |
+
+## Resources
+
+No resources.
+
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|------|---------|:--------:|
+| <a name="input_dsfhub_host"></a> [dsfhub\_host](#input\_dsfhub\_host) | n/a | `any` | n/a | yes |
+| <a name="input_dsfhub_token"></a> [dsfhub\_token](#input\_dsfhub\_token) | n/a | `any` | n/a | yes |
+
+## Outputs
+
+No outputs.
+<!-- END_TF_DOCS -->
diff --git a/examples/onboard-aws-docdb-cluster/main.tf b/examples/onboard-aws-docdb-cluster/main.tf
@@ -0,0 +1,80 @@
+locals {
+  aws_region = "us-east-1"
+
+  admin_email       = "[email protected]"
+  apply_immediately = true
+  gateway_id        = "a1b2c3d4-e5f6-g8h9-wxyz-123456790"
+}
+
+################################################################################
+# Providers
+################################################################################
+terraform {
+  required_providers {
+    dsfhub = {
+      source = "imperva/dsfhub"
+    }
+  }
+}
+
+provider "aws" {
+  region = local.aws_region
+}
+
+variable "dsfhub_host" {}  # TF_VAR_dsfhub_host env variable
+variable "dsfhub_token" {} # TF_VAR_dsfhub_token env variable
+
+provider "dsfhub" {
+  dsfhub_host  = var.dsfhub_host
+  dsfhub_token = var.dsfhub_token
+}
+
+################################################################################
+# Prerequisites
+# 1. AWS cloud account
+################################################################################
+module "aws-default-account-asset" {
+  source = "imperva/agentless-onboarding/dsfhub//modules/dsfhub-aws-cloud-account"
+
+  admin_email        = local.admin_email
+  asset_display_name = "aws-account-asset"
+  asset_id           = "arn:aws:iam::1234567890"
+  auth_mechanism     = "default"
+  gateway_id         = local.gateway_id
+  region             = local.aws_region
+}
+
+################################################################################
+# Amazon DocumentDB Cluster 5.0.0
+################################################################################
+
+module "aws-docdb-cluster-1" {
+  source = "../../modules/onboard-aws-docdb-cluster"
+
+  aws_log_group_admin_email        = local.admin_email
+  aws_log_group_audit_pull_enabled = true
+  aws_log_group_gateway_id         = local.gateway_id
+  aws_log_group_region             = local.aws_region
+
+  aws_docdb_cluster_admin_email     = local.admin_email
+  aws_docdb_cluster_gateway_id      = local.gateway_id
+  aws_docdb_cluster_parent_asset_id = module.aws-default-account-asset.this.asset_id
+  aws_docdb_cluster_region          = local.aws_region
+
+  cluster_apply_immediately               = local.apply_immediately
+  cluster_identifier                      = "example-tf-docdb"
+  cluster_db_subnet_group_name            = "default"
+  cluster_enabled_cloudwatch_logs_exports = ["audit"]
+  cluster_engine_version                  = "5.0.0"
+  cluster_master_password                 = "abcd1234"
+  cluster_master_username                 = "docdbadmin"
+  cluster_skip_final_snapshot             = true
+  cluster_vpc_security_group_ids          = ["sg-0123456789abcdefg"]
+
+  instance_apply_immediately = local.apply_immediately
+  instance_count             = 1
+  instance_instance_class    = "db.t3.medium"
+
+  parameter_group_family = "docdb5.0"
+  parameter_group_name   = "docdb-pg-test4"
+}
diff --git a/examples/onboard-aws-neptune-slow-query/README.md b/examples/onboard-aws-neptune-slow-query/README.md
@@ -0,0 +1,42 @@
+# Onboard Amazon Neptune with Slow Query example
+This example includes additional prerequisites that will need to be completed to fully utilize the module. More details can be found in the [onboarding documentation](https://docs.imperva.com/bundle/onboarding-databases-to-sonar-reference-guide/page/Amazon-Neptune-Onboarding-Steps_48367003.html).
+
+This example creates both 'aws' and 'dsfhub' resources. More information regarding authentication to each can be found in the relevant provider documentation:
+- [aws](https://registry.terraform.io/providers/hashicorp/aws/latest/docs)
+- [dsfhub](https://registry.terraform.io/providers/imperva/dsfhub/latest/docs)
+
+## Prerequisites
+### Account Asset Permissions
+An AWS account asset will need to be onboarded to your DSF hub prior to using this module. The account asset will need to be granted permissions to be able to read from the newly created CloudWatch log group. 
+
+<!-- BEGIN_TF_DOCS -->
+## Requirements
+
+No requirements.
+
+## Providers
+
+No providers.
+
+## Modules
+
+| Name | Source | Version |
+|------|--------|---------|
+| <a name="module_aws-default-account-asset"></a> [aws-default-account-asset](#module\_aws-default-account-asset) | imperva/agentless-onboarding/dsfhub//modules/dsfhub-aws-cloud-account | n/a |
+| <a name="module_aws-neptune-slowquery-1"></a> [aws-neptune-slowquery-1](#module\_aws-neptune-slowquery-1) | ../../modules/onboard-aws-neptune-slow-query | n/a |
+
+## Resources
+
+No resources.
+
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|------|---------|:--------:|
+| <a name="input_dsfhub_host"></a> [dsfhub\_host](#input\_dsfhub\_host) | n/a | `any` | n/a | yes |
+| <a name="input_dsfhub_token"></a> [dsfhub\_token](#input\_dsfhub\_token) | n/a | `any` | n/a | yes |
+
+## Outputs
+
+No outputs.
+<!-- END_TF_DOCS -->