-
Notifications
You must be signed in to change notification settings - Fork 486
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add module to enable bringing your own bucket file carving destination #15206
Merged
+334
−0
Merged
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
# S3 File Carving backend | ||
|
||
This module creates the necessary IAM role for Fleet to attach when it's running in server mode. | ||
|
||
It also exports the `fleet_extra_environment_variables` to configure Fleet server to use S3 as the backing carve results store. | ||
|
||
Usage typically looks like: | ||
|
||
```terraform | ||
fleet_config = { | ||
extra_environment_variables = merge( | ||
local.extra_environment_variables, | ||
module.carving.fleet_extra_environment_variables | ||
) | ||
} | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
header-from: .header.md |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,53 @@ | ||
# S3 File Carving backend | ||
|
||
This module creates the necessary IAM role for Fleet to attach when it's running in server mode. | ||
|
||
It also exports the `fleet_extra_environment_variables` to configure Fleet server to use S3 as the backing carve results store. | ||
|
||
Usage typically looks like: | ||
|
||
```terraform | ||
fleet_config = { | ||
extra_environment_variables = merge( | ||
local.extra_environment_variables, | ||
module.carving.fleet_extra_environment_variables | ||
) | ||
} | ||
``` | ||
|
||
## Requirements | ||
|
||
No requirements. | ||
|
||
## Providers | ||
|
||
| Name | Version | | ||
|------|---------| | ||
| <a name="provider_aws"></a> [aws](#provider\_aws) | n/a | | ||
|
||
## Modules | ||
|
||
No modules. | ||
|
||
## Resources | ||
|
||
| Name | Type | | ||
|------|------| | ||
| [aws_iam_policy.fleet-assume-role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource | | ||
| [aws_iam_policy_document.fleet-assume-role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source | | ||
|
||
## Inputs | ||
|
||
| Name | Description | Type | Default | Required | | ||
|------|-------------|------|---------|:--------:| | ||
| <a name="input_iam_role_arn"></a> [iam\_role\_arn](#input\_iam\_role\_arn) | IAM Role ARN to assume into for file carving uploads to S3 | `string` | n/a | yes | | ||
| <a name="input_s3_bucket_name"></a> [s3\_bucket\_name](#input\_s3\_bucket\_name) | The S3 bucket for carve results to be written to | `string` | n/a | yes | | ||
| <a name="input_s3_bucket_region"></a> [s3\_bucket\_region](#input\_s3\_bucket\_region) | The S3 bucket region | `string` | n/a | yes | | ||
| <a name="input_s3_carve_prefix"></a> [s3\_carve\_prefix](#input\_s3\_carve\_prefix) | The S3 object prefix to use when storing carve results | `string` | `""` | no | | ||
|
||
## Outputs | ||
|
||
| Name | Description | | ||
|------|-------------| | ||
| <a name="output_fleet_extra_environment_variables"></a> [fleet\_extra\_environment\_variables](#output\_fleet\_extra\_environment\_variables) | n/a | | ||
| <a name="output_fleet_extra_iam_policies"></a> [fleet\_extra\_iam\_policies](#output\_fleet\_extra\_iam\_policies) | n/a | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
data "aws_iam_policy_document" "fleet-assume-role" { | ||
statement { | ||
effect = "Allow" | ||
actions = ["sts:AssumeRole"] | ||
resources = [var.iam_role_arn] | ||
} | ||
} | ||
|
||
resource "aws_iam_policy" "fleet-assume-role" { | ||
policy = data.aws_iam_policy_document.fleet-assume-role.json | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
output "fleet_extra_environment_variables" { | ||
value = { | ||
FLEET_S3_STS_ASSUME_ROLE_ARN = var.iam_role_arn | ||
FLEET_S3_BUCKET = var.s3_bucket_name | ||
FLEET_S3_REGION = var.s3_bucket_region | ||
FLEET_S3_PREFIX = var.s3_carve_prefix | ||
} | ||
} | ||
|
||
output "fleet_extra_iam_policies" { | ||
value = [ | ||
aws_iam_policy.fleet-assume-role.arn | ||
] | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
variable "iam_role_arn" { | ||
type = string | ||
description = "IAM Role ARN to assume into for file carving uploads to S3" | ||
} | ||
|
||
variable "s3_bucket_name" { | ||
type = string | ||
description = "The S3 bucket for carve results to be written to" | ||
} | ||
|
||
variable "s3_bucket_region" { | ||
type = string | ||
description = "The S3 bucket region" | ||
} | ||
|
||
variable "s3_carve_prefix" { | ||
type = string | ||
description = "The S3 object prefix to use when storing carve results" | ||
default = "" | ||
} |
25 changes: 25 additions & 0 deletions
25
terraform/addons/byo-file-carving/target-account/.header.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
# AWS S3 File Carving Infrastructure | ||
|
||
This Terraform configuration sets up the necessary resources for a secure file carving infrastructure in AWS. File carving is a significant capability for security and forensic analysis, enabling organizations to extract and analyze the content of files from their endpoints. | ||
|
||
## Overview of Resources | ||
|
||
The resources configured in this Terraform script include: | ||
|
||
- **AWS Key Management Service (KMS) Key**: A customer-managed KMS key is created to provide server-side encryption for the S3 bucket where carved files will be stored. The policy attached to this key grants full KMS permissions to the AWS account's root user. | ||
|
||
- **Amazon S3 Bucket**: An S3 bucket is provisioned to act as the central repository for storing the results of the file carving process. The bucket is named according to the provided variable `var.bucket_name`. | ||
|
||
- **S3 Bucket Server-Side Encryption Configuration**: This resource configures server-side encryption for the S3 bucket, specifying the custom-created KMS key as the master key for encrypting objects stored in the bucket. | ||
|
||
- **IAM Policy**: An IAM policy is created to enable specific access to the S3 bucket. This policy is defined via a detailed policy document which grants permissions to perform various actions essential for managing the file carving process. Actions include object retrieval (`GetObject*`), object creation (`PutObject*`), listing the bucket (`ListBucket*`), and managing multipart uploads. It also allows for certain KMS actions necessary for encrypting and decrypting the stored data. | ||
|
||
- **IAM Role**: An IAM role (`aws_iam_role`) is provisioned with a trust relationship policy that permits an external entity, specified by `var.fleet_iam_role_arn`, to assume the role. This allows secure access to the S3 bucket and KMS key based on assuming roles across AWS accounts or services. | ||
|
||
- **IAM Role Policy Attachment**: This attachment links the previously created IAM policy to the IAM role, ensuring that the permissions are in effect when the role is assumed by the external entity. | ||
|
||
## Usage | ||
|
||
To use this Terraform configuration, ensure that you have Terraform installed and configured with the necessary AWS credentials. You should define the `bucket_name` and `fleet_iam_role_arn` variables according to your organization's requirements before applying the Terraform plan. | ||
|
||
This infrastructure enables secure storage and access for file carving results, facilitating forensic analysis and the capability to respond to security incidents effectively. |
1 change: 1 addition & 0 deletions
1
terraform/addons/byo-file-carving/target-account/.terraform-docs.yml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
header-from: .header.md |
69 changes: 69 additions & 0 deletions
69
terraform/addons/byo-file-carving/target-account/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,69 @@ | ||
# AWS S3 File Carving Infrastructure | ||
|
||
This Terraform configuration sets up the necessary resources for a secure file carving infrastructure in AWS. File carving is a significant capability for security and forensic analysis, enabling organizations to extract and analyze the content of files from their endpoints. | ||
|
||
## Overview of Resources | ||
|
||
The resources configured in this Terraform script include: | ||
|
||
- **AWS Key Management Service (KMS) Key**: A customer-managed KMS key is created to provide server-side encryption for the S3 bucket where carved files will be stored. The policy attached to this key grants full KMS permissions to the AWS account's root user. | ||
|
||
- **Amazon S3 Bucket**: An S3 bucket is provisioned to act as the central repository for storing the results of the file carving process. The bucket is named according to the provided variable `var.bucket_name`. | ||
|
||
- **S3 Bucket Server-Side Encryption Configuration**: This resource configures server-side encryption for the S3 bucket, specifying the custom-created KMS key as the master key for encrypting objects stored in the bucket. | ||
|
||
- **IAM Policy**: An IAM policy is created to enable specific access to the S3 bucket. This policy is defined via a detailed policy document which grants permissions to perform various actions essential for managing the file carving process. Actions include object retrieval (`GetObject*`), object creation (`PutObject*`), listing the bucket (`ListBucket*`), and managing multipart uploads. It also allows for certain KMS actions necessary for encrypting and decrypting the stored data. | ||
|
||
- **IAM Role**: An IAM role (`aws_iam_role`) is provisioned with a trust relationship policy that permits an external entity, specified by `var.fleet_iam_role_arn`, to assume the role. This allows secure access to the S3 bucket and KMS key based on assuming roles across AWS accounts or services. | ||
|
||
- **IAM Role Policy Attachment**: This attachment links the previously created IAM policy to the IAM role, ensuring that the permissions are in effect when the role is assumed by the external entity. | ||
|
||
## Usage | ||
|
||
To use this Terraform configuration, ensure that you have Terraform installed and configured with the necessary AWS credentials. You should define the `bucket_name` and `fleet_iam_role_arn` variables according to your organization's requirements before applying the Terraform plan. | ||
|
||
This infrastructure enables secure storage and access for file carving results, facilitating forensic analysis and the capability to respond to security incidents effectively. | ||
|
||
## Requirements | ||
|
||
No requirements. | ||
|
||
## Providers | ||
|
||
| Name | Version | | ||
|------|---------| | ||
| <a name="provider_aws"></a> [aws](#provider\_aws) | n/a | | ||
|
||
## Modules | ||
|
||
No modules. | ||
|
||
## Resources | ||
|
||
| Name | Type | | ||
|------|------| | ||
| [aws_iam_policy.s3_access_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource | | ||
| [aws_iam_role.carve_s3_delegation_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource | | ||
| [aws_iam_role_policy_attachment.s3_access_attachment](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource | | ||
| [aws_kms_key.s3_encryption_key](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kms_key) | resource | | ||
| [aws_s3_bucket.carve_results_bucket](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) | resource | | ||
| [aws_s3_bucket_server_side_encryption_configuration.sse](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_server_side_encryption_configuration) | resource | | ||
| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source | | ||
| [aws_iam_policy_document.assume_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source | | ||
| [aws_iam_policy_document.kms_key_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source | | ||
| [aws_iam_policy_document.s3_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source | | ||
|
||
## Inputs | ||
|
||
| Name | Description | Type | Default | Required | | ||
|------|-------------|------|---------|:--------:| | ||
| <a name="input_bucket_name"></a> [bucket\_name](#input\_bucket\_name) | The name of the osquery carve results bucket | `string` | n/a | yes | | ||
| <a name="input_fleet_iam_role_arn"></a> [fleet\_iam\_role\_arn](#input\_fleet\_iam\_role\_arn) | The IAM role ARN of the Fleet service | `string` | n/a | yes | | ||
|
||
## Outputs | ||
|
||
| Name | Description | | ||
|------|-------------| | ||
| <a name="output_iam_role_arn"></a> [iam\_role\_arn](#output\_iam\_role\_arn) | n/a | | ||
| <a name="output_s3_bucket_name"></a> [s3\_bucket\_name](#output\_s3\_bucket\_name) | n/a | | ||
| <a name="output_s3_bucket_region"></a> [s3\_bucket\_region](#output\_s3\_bucket\_region) | n/a | |
11 changes: 11 additions & 0 deletions
11
terraform/addons/byo-file-carving/target-account/outputs.tf
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
output "iam_role_arn" { | ||
value = aws_iam_role.carve_s3_delegation_role.arn | ||
} | ||
|
||
output "s3_bucket_name" { | ||
value = aws_s3_bucket.carve_results_bucket.id | ||
} | ||
|
||
output "s3_bucket_region" { | ||
value = aws_s3_bucket.carve_results_bucket.region | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,104 @@ | ||
data "aws_caller_identity" "current" {} | ||
|
||
# IAM policy document for the KMS key | ||
data "aws_iam_policy_document" "kms_key_policy" { | ||
statement { | ||
sid = "Enable IAM User Permissions" | ||
actions = ["kms:*"] | ||
resources = ["*"] | ||
principals { | ||
type = "AWS" | ||
identifiers = ["arn:aws:iam::${data.aws_caller_identity.current.account_id}:root"] | ||
} | ||
} | ||
} | ||
|
||
# Create a KMS key for encrypting the S3 bucket | ||
resource "aws_kms_key" "s3_encryption_key" { | ||
description = "KMS key for S3 bucket encryption" | ||
is_enabled = true | ||
policy = data.aws_iam_policy_document.kms_key_policy.json | ||
} | ||
|
||
# Create an S3 bucket with server-side encryption using the customer-managed key | ||
resource "aws_s3_bucket" "carve_results_bucket" { | ||
bucket = var.bucket_name | ||
} | ||
|
||
resource "aws_s3_bucket_public_access_block" "carve_results" { | ||
bucket = aws_s3_bucket.carve_results_bucket.id | ||
|
||
block_public_acls = true | ||
block_public_policy = true | ||
ignore_public_acls = true | ||
restrict_public_buckets = true | ||
} | ||
|
||
resource "aws_s3_bucket_server_side_encryption_configuration" "sse" { | ||
bucket = aws_s3_bucket.carve_results_bucket.id | ||
rule { | ||
apply_server_side_encryption_by_default { | ||
sse_algorithm = "aws:kms" | ||
kms_master_key_id = aws_kms_key.s3_encryption_key.key_id | ||
} | ||
} | ||
} | ||
|
||
# Create an IAM policy which allows the necessary S3 actions | ||
resource "aws_iam_policy" "s3_access_policy" { | ||
name = "s3_access_policy" | ||
policy = data.aws_iam_policy_document.s3_policy.json | ||
} | ||
|
||
# IAM policy document | ||
data "aws_iam_policy_document" "s3_policy" { | ||
statement { | ||
actions = [ | ||
"s3:GetObject*", | ||
"s3:PutObject*", | ||
"s3:ListBucket*", | ||
"s3:ListMultipartUploadParts*", | ||
"s3:DeleteObject", | ||
"s3:CreateMultipartUpload", | ||
"s3:AbortMultipartUpload", | ||
"s3:ListMultipartUploadParts", | ||
"s3:GetBucketLocation" | ||
] | ||
|
||
resources = [ | ||
aws_s3_bucket.carve_results_bucket.arn, | ||
"${aws_s3_bucket.carve_results_bucket.arn}/*" | ||
] | ||
} | ||
|
||
statement { | ||
effect = "Allow" | ||
actions = [ | ||
"kms:Decrypt", | ||
"kms:GenerateDataKey", | ||
"kms:Encrypt" | ||
] | ||
resources = [aws_kms_key.s3_encryption_key.arn] | ||
} | ||
} | ||
|
||
data "aws_iam_policy_document" "assume_role" { | ||
statement { | ||
effect = "Allow" | ||
actions = ["sts:AssumeRole"] | ||
principals { | ||
identifiers = [var.fleet_iam_role_arn] | ||
type = "AWS" | ||
} | ||
} | ||
} | ||
|
||
resource "aws_iam_role" "carve_s3_delegation_role" { | ||
assume_role_policy = data.aws_iam_policy_document.assume_role.json | ||
} | ||
|
||
# Attach the policy to the role | ||
resource "aws_iam_role_policy_attachment" "s3_access_attachment" { | ||
role = aws_iam_role.carve_s3_delegation_role.name | ||
policy_arn = aws_iam_policy.s3_access_policy.arn | ||
} |
9 changes: 9 additions & 0 deletions
9
terraform/addons/byo-file-carving/target-account/variables.tf
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
variable "bucket_name" { | ||
type = string | ||
description = "The name of the osquery carve results bucket" | ||
} | ||
|
||
variable "fleet_iam_role_arn" { | ||
type = string | ||
description = "The IAM role ARN of the Fleet service" | ||
} |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should we have a aws_s3_bucket_public_access_block for this bucket just to cover these are set explicitly rather than implicitly?