Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add purge commands to CLI tool #373

Merged
merged 27 commits into from
Sep 27, 2023
Merged

Add purge commands to CLI tool #373

merged 27 commits into from
Sep 27, 2023

Conversation

TylerHendrickson
Copy link
Member

@TylerHendrickson TylerHendrickson commented Sep 26, 2023

Description

This PR adds a purge command to the CLI tool, which provides two subcommands:

  • grants-ingest purge prepared-data-bucket: purges objects (from FFIS, Grants.gov, or everything) from a named S3 bucket containing data stored by SplitFFISSpreadsheet and/or SplitGrantsGovXMLDB Lambda functions.
  • grants-ingest purge prepared-data-table: purges items (from FFIS, Grants.gov, or everything) from a named DynamoDB table containing data stored by PersistFFISData and/or PersistGrantsGovXMLDB Lambda functions.

Additionally, the parent purge command provides extended help test containing runbook suggestions and detailed usage instructions, which may be accessed by running grants-ingest purge --help.

Testing

All commands can be tested agains a deployed sandbox or LocalStack environment. Suggested tests are shown below.

# Build the CLI tool (outputs an executable at bin/grants-ingest)
task build-cli

# View detailed help text:
./bin/grants-ingest purge --help 

# View options for purging S3 data
./bin/grants-ingest purge prepared-data-bucket --help

# View S3 purge behavior without actually deleting data:
./bin/grants-ingest purge prepared-data-bucket \
    my-bucket-name \
    --purge-gov \
    --dry-run \
    --log-level=debug

# View options for purging DynamoDB data
./bin/grants-ingest purge prepared-data-table --help

# View DynamoDB purge behavior without actually deleting data:
./bin/grants-ingest purge prepared-data-table \
    my-table-name \
    --purge-gov \
    --dry-run \
    --log-level=debug

# Actually purge DynamoDB data
grants-ingest purge prepared-data-table \
    my-table-name \
    --purge-gov \
    --read-concurrency 3 \
    --write-concurrency 5 \
    --totals-after 10000 \
    --ignore-streams

Note: When testing against a LocalStack environment, provide the --s3-use-path-style whenever running the prepared-data-bucket subcommand in order to remain compatible with LocalStack bucket addressing requirements.

Automated and Unit Tests

  • Added Unit tests

Manual tests for Reviewer

  • Added steps to test feature/functionality manually

Checklist

  • Provided ticket and description
  • Provided testing information
  • Provided adequate test coverage for all new code
  • Added PR reviewers

@TylerHendrickson TylerHendrickson requested review from as1729 and a team September 26, 2023 03:10
@TylerHendrickson TylerHendrickson self-assigned this Sep 26, 2023
@github-actions github-actions bot added the go Pull requests that update Go code label Sep 26, 2023
)

var (
FFISMatcher = &RegexMatcher{regexp.MustCompile(`\d{3}/\d{3,}/ffis.org/v1.json`)}

Check failure

Code scanning / CodeQL

Incomplete regular expression for hostnames High

This regular expression has an unescaped dot before 'org', so it might match more hosts than expected when
the regular expression is used
.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dismissed with: This is matching an S3 path segment which references a hostname, not an actual hostname.


var (
FFISMatcher = &RegexMatcher{regexp.MustCompile(`\d{3}/\d{3,}/ffis.org/v1.json`)}
GrantsGovMatcher = &RegexMatcher{regexp.MustCompile(`\d{3}/\d{3,}/grants.gov/v2.xml`)}

Check failure

Code scanning / CodeQL

Incomplete regular expression for hostnames High

This regular expression has an unescaped dot before 'gov', so it might match more hosts than expected when
the regular expression is used
.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dismissed with: This is matching an S3 path segment which references a hostname, not an actual hostname.

@github-actions
Copy link

github-actions bot commented Sep 26, 2023

Terraform Summary

Step Result
🖌 Terraform Format & Style
⚙️ Terraform Initialization
🤖 Terraform Validation
📖 Terraform Plan

Output

Validation Output
Success! The configuration is valid.


Plan Output
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  ~ update in-place
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # datadog_metric_metadata.custom["grants_ingest.DownloadFFISSpreadsheet.source_size"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.DownloadFFISSpreadsheet.source_size"
-       type            = "gauge" -> null
        # (6 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.DownloadGrantsGovDB.source_size"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.DownloadGrantsGovDB.source_size"
-       type            = "gauge" -> null
        # (6 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.ExtractGrantsGovDBToXML.xml.extracted"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.ExtractGrantsGovDBToXML.xml.extracted"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.ExtractGrantsGovDBToXML.xml.uploaded"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.ExtractGrantsGovDBToXML.xml.uploaded"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.PersistFFISData.opportunity.saved"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.PersistFFISData.opportunity.saved"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.PersistGrantsGovXMLDB.opportunity.failed"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.PersistGrantsGovXMLDB.opportunity.failed"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.PersistGrantsGovXMLDB.opportunity.saved"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.PersistGrantsGovXMLDB.opportunity.saved"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.PublishGrantEvents.event.published"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.PublishGrantEvents.event.published"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.PublishGrantEvents.grant_data.invalid"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.PublishGrantEvents.grant_data.invalid"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.PublishGrantEvents.invocation_batch_size"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.PublishGrantEvents.invocation_batch_size"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.PublishGrantEvents.item_image.build"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.PublishGrantEvents.item_image.build"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.PublishGrantEvents.item_image.malformatted_field"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.PublishGrantEvents.item_image.malformatted_field"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.PublishGrantEvents.record.failed"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.PublishGrantEvents.record.failed"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.SplitFFISSpreadsheet.opportunity.created"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.SplitFFISSpreadsheet.opportunity.created"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.SplitFFISSpreadsheet.spreadsheet.row_count"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.SplitFFISSpreadsheet.spreadsheet.row_count"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.SplitGrantsGovXMLDB.opportunity.created"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.SplitGrantsGovXMLDB.opportunity.created"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.SplitGrantsGovXMLDB.opportunity.failed"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.SplitGrantsGovXMLDB.opportunity.failed"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.SplitGrantsGovXMLDB.opportunity.skipped"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.SplitGrantsGovXMLDB.opportunity.skipped"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.SplitGrantsGovXMLDB.opportunity.updated"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.SplitGrantsGovXMLDB.opportunity.updated"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # module.DownloadFFISSpreadsheet.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-DownloadFFISSpreadsheet"
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-DownloadFFISSpreadsheet:19" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-DownloadFFISSpreadsheet:19/invocations" -> (known after apply)
        tags                           = {}
      ~ version                        = "19" -> (known after apply)
        # (21 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                      = "git.commit.sha:b189f7b142efc7040fab778deaa6b96c4380015e,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:downloadffisspreadsheet" -> "git.commit.sha:85a882a863ed3ed6237b3e373f44268e38f6fa71,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:downloadffisspreadsheet"
              ~ "DD_VERSION"                   = "b189f7b142efc7040fab778deaa6b96c4380015e" -> "85a882a863ed3ed6237b3e373f44268e38f6fa71"
                # (11 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.DownloadFFISSpreadsheet.module.lambda_function.aws_lambda_permission.current_version_triggers["SQSQueueNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "SQSQueueNotification" -> (known after apply)
      ~ qualifier           = "19" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (4 unchanged attributes hidden)
    }

  # module.DownloadGrantsGovDB.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-DownloadGrantsGovDB"
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-DownloadGrantsGovDB:19" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-DownloadGrantsGovDB:19/invocations" -> (known after apply)
        tags                           = {}
      ~ version                        = "19" -> (known after apply)
        # (21 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                        = "git.commit.sha:b189f7b142efc7040fab778deaa6b96c4380015e,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:downloadgrantsgovdb" -> "git.commit.sha:85a882a863ed3ed6237b3e373f44268e38f6fa71,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:downloadgrantsgovdb"
              ~ "DD_VERSION"                     = "b189f7b142efc7040fab778deaa6b96c4380015e" -> "85a882a863ed3ed6237b3e373f44268e38f6fa71"
                # (12 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.DownloadGrantsGovDB.module.lambda_function.aws_lambda_permission.current_version_triggers["Schedule"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "Schedule" -> (known after apply)
      ~ qualifier           = "19" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.EnqueueFFISDownload.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-EnqueueFFISDownload"
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-EnqueueFFISDownload:19" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-EnqueueFFISDownload:19/invocations" -> (known after apply)
        tags                           = {}
      ~ version                        = "19" -> (known after apply)
        # (21 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                      = "git.commit.sha:b189f7b142efc7040fab778deaa6b96c4380015e,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:enqueueffisdownload" -> "git.commit.sha:85a882a863ed3ed6237b3e373f44268e38f6fa71,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:enqueueffisdownload"
              ~ "DD_VERSION"                   = "b189f7b142efc7040fab778deaa6b96c4380015e" -> "85a882a863ed3ed6237b3e373f44268e38f6fa71"
                # (11 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.EnqueueFFISDownload.module.lambda_function.aws_lambda_permission.current_version_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ qualifier           = "19" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.ExtractGrantsGovDBToXML.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-ExtractGrantsGovDBToXML"
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-ExtractGrantsGovDBToXML:19" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-ExtractGrantsGovDBToXML:19/invocations" -> (known after apply)
        tags                           = {}
      ~ version                        = "19" -> (known after apply)
        # (21 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                      = "git.commit.sha:b189f7b142efc7040fab778deaa6b96c4380015e,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:extractgrantsgovdbtoxml" -> "git.commit.sha:85a882a863ed3ed6237b3e373f44268e38f6fa71,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:extractgrantsgovdbtoxml"
              ~ "DD_VERSION"                   = "b189f7b142efc7040fab778deaa6b96c4380015e" -> "85a882a863ed3ed6237b3e373f44268e38f6fa71"
                # (11 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.ExtractGrantsGovDBToXML.module.lambda_function.aws_lambda_permission.current_version_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ qualifier           = "19" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.PersistFFISData.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-PersistFFISData"
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-PersistFFISData:19" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-PersistFFISData:19/invocations" -> (known after apply)
        tags                           = {}
      ~ version                        = "19" -> (known after apply)
        # (21 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                       = "git.commit.sha:b189f7b142efc7040fab778deaa6b96c4380015e,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:persistffisdata" -> "git.commit.sha:85a882a863ed3ed6237b3e373f44268e38f6fa71,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:persistffisdata"
              ~ "DD_VERSION"                    = "b189f7b142efc7040fab778deaa6b96c4380015e" -> "85a882a863ed3ed6237b3e373f44268e38f6fa71"
                # (11 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.PersistFFISData.module.lambda_function.aws_lambda_permission.current_version_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ qualifier           = "19" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.PersistGrantsGovXMLDB.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-PersistGrantsGovXMLDB"
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-PersistGrantsGovXMLDB:19" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-PersistGrantsGovXMLDB:19/invocations" -> (known after apply)
        tags                           = {}
      ~ version                        = "19" -> (known after apply)
        # (21 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                       = "git.commit.sha:b189f7b142efc7040fab778deaa6b96c4380015e,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:persistgrantsgovxmldb" -> "git.commit.sha:85a882a863ed3ed6237b3e373f44268e38f6fa71,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:persistgrantsgovxmldb"
              ~ "DD_VERSION"                    = "b189f7b142efc7040fab778deaa6b96c4380015e" -> "85a882a863ed3ed6237b3e373f44268e38f6fa71"
                # (11 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.PersistGrantsGovXMLDB.module.lambda_function.aws_lambda_permission.current_version_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ qualifier           = "19" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.PublishGrantEvents.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-PublishGrantEvents"
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-PublishGrantEvents:20" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-PublishGrantEvents:20/invocations" -> (known after apply)
        tags                           = {}
      ~ version                        = "20" -> (known after apply)
        # (21 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                      = "git.commit.sha:b189f7b142efc7040fab778deaa6b96c4380015e,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:publishgrantevents" -> "git.commit.sha:85a882a863ed3ed6237b3e373f44268e38f6fa71,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:publishgrantevents"
              ~ "DD_VERSION"                   = "b189f7b142efc7040fab778deaa6b96c4380015e" -> "85a882a863ed3ed6237b3e373f44268e38f6fa71"
                # (11 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.PublishGrantEvents.module.lambda_function.aws_lambda_permission.current_version_triggers["dynamodb"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "dynamodb" -> (known after apply)
      ~ qualifier           = "20" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.ReceiveFFISEmail.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-ReceiveFFISEmail"
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-ReceiveFFISEmail:18" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-ReceiveFFISEmail:18/invocations" -> (known after apply)
        tags                           = {}
      ~ version                        = "18" -> (known after apply)
        # (21 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                        = "git.commit.sha:b189f7b142efc7040fab778deaa6b96c4380015e,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:receiveffisemail" -> "git.commit.sha:85a882a863ed3ed6237b3e373f44268e38f6fa71,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:receiveffisemail"
              ~ "DD_VERSION"                     = "b189f7b142efc7040fab778deaa6b96c4380015e" -> "85a882a863ed3ed6237b3e373f44268e38f6fa71"
                # (12 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.ReceiveFFISEmail.module.lambda_function.aws_lambda_permission.current_version_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ qualifier           = "18" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.SplitFFISSpreadsheet.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-SplitFFISSpreadsheet"
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-SplitFFISSpreadsheet:19" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-SplitFFISSpreadsheet:19/invocations" -> (known after apply)
        tags                           = {}
      ~ version                        = "19" -> (known after apply)
        # (21 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                          = "git.commit.sha:b189f7b142efc7040fab778deaa6b96c4380015e,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:splitffisspreadsheet" -> "git.commit.sha:85a882a863ed3ed6237b3e373f44268e38f6fa71,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:splitffisspreadsheet"
              ~ "DD_VERSION"                       = "b189f7b142efc7040fab778deaa6b96c4380015e" -> "85a882a863ed3ed6237b3e373f44268e38f6fa71"
                # (14 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.SplitFFISSpreadsheet.module.lambda_function.aws_lambda_permission.current_version_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ qualifier           = "19" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.SplitGrantsGovXMLDB.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-SplitGrantsGovXMLDB"
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-SplitGrantsGovXMLDB:19" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-SplitGrantsGovXMLDB:19/invocations" -> (known after apply)
        tags                           = {}
      ~ version                        = "19" -> (known after apply)
        # (21 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                          = "git.commit.sha:b189f7b142efc7040fab778deaa6b96c4380015e,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:splitgrantsgovxmldb" -> "git.commit.sha:85a882a863ed3ed6237b3e373f44268e38f6fa71,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:splitgrantsgovxmldb"
              ~ "DD_VERSION"                       = "b189f7b142efc7040fab778deaa6b96c4380015e" -> "85a882a863ed3ed6237b3e373f44268e38f6fa71"
                # (14 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.SplitGrantsGovXMLDB.module.lambda_function.aws_lambda_permission.current_version_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ qualifier           = "19" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

Plan: 10 to add, 29 to change, 10 to destroy.

Pusher: @TylerHendrickson, Action: pull_request_target, Workflow: Continuous Integration

@TylerHendrickson TylerHendrickson enabled auto-merge (squash) September 26, 2023 03:21
Copy link
Contributor

@as1729 as1729 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

@TylerHendrickson TylerHendrickson merged commit bd8b0c6 into main Sep 27, 2023
17 checks passed
@TylerHendrickson TylerHendrickson deleted the feat/cli-purge-data branch September 27, 2023 18:50
@TylerHendrickson TylerHendrickson added the enhancement New feature or request label Oct 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request go Pull requests that update Go code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants