Add support for shielded instance initial state

[NotTheEvilOne]

This PR provides support to use UEFI secure boot with terraform-provider-google. The requirement originates from enabling UEFI secure boot with custom config and keys. See gardenlinux/gardenlinux#2473

Fixes hashicorp/terraform-provider-google#20303

Release Note Template for Downstream PRs (will be copied)

compute: Added `shieldedInstanceInitialState` structure to `google_compute_image` resource

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[github-actions]

Hello! I am a robot. Tests will require approval from a repository maintainer to run.

@SirGitsalot, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 2 files changed, 497 insertions(+))
google-beta provider: Diff ( 2 files changed, 497 insertions(+))
terraform-google-conversion: Diff ( 1 file changed, 191 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_compute_image (567 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_compute_image" "primary" {
  shielded_instance_initial_state {
    dbs {
      content   = # value needed
      file_type = # value needed
    }
    dbxs {
      content   = # value needed
      file_type = # value needed
    }
    keks {
      content   = # value needed
      file_type = # value needed
    }
    pk {
      content   = # value needed
      file_type = # value needed
    }
  }
}

[modular-magician]

Tests analytics

Total tests: 1064
Passed tests: 989
Skipped tests: 73
Affected tests: 2

Click here to see the affected service packages

• compute

Action taken

Found 2 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

• TestAccComputeInstanceFromTemplate_confidentialInstanceConfigMain
• TestAccComputeInstanceNetworkIntefaceWithSecurityPolicy

Get to know how VCR tests work

[modular-magician]

🔴 Tests failed during RECORDING mode:
TestAccComputeInstanceFromTemplate_confidentialInstanceConfigMain [Error message] [Debug log]
TestAccComputeInstanceNetworkIntefaceWithSecurityPolicy [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

[NotTheEvilOne]

🔴 Tests failed during RECORDING mode: TestAccComputeInstanceFromTemplate_confidentialInstanceConfigMain [Error message] [Debug log] TestAccComputeInstanceNetworkIntefaceWithSecurityPolicy [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

The error message [...] does not have storage.objects.list access to the Google Cloud Storage bucket. Permission 'storage.objects.list' denied on resource (or it may not exist). alternatively to 403 Forbidden is not helpful in finding out what error occurred.

[github-actions]

@BBBmau This PR has been waiting for review for 3 weekdays. Please take a look! Use the label disable-review-reminders to disable these notifications.

[github-actions]

@GoogleCloudPlatform/terraform-team @BBBmau This PR has been waiting for review for 1 week. Please take a look! Use the label disable-review-reminders to disable these notifications.

[github-actions]

@GoogleCloudPlatform/terraform-team @BBBmau This PR has been waiting for review for 2 weeks. Please take a look! Use the label disable-review-reminders to disable these notifications.

[github-actions]

@GoogleCloudPlatform/terraform-team @BBBmau This PR has been waiting for review for 3 weeks. Please take a look! Use the label disable-review-reminders to disable these notifications.

[github-actions]

@GoogleCloudPlatform/terraform-team @BBBmau This PR has been waiting for review for 4 weeks. Please take a look! Use the label disable-review-reminders to disable these notifications.

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 2 files changed, 497 insertions(+))
google-beta provider: Diff ( 2 files changed, 497 insertions(+))
terraform-google-conversion: Diff ( 1 file changed, 191 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_compute_image (581 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_compute_image" "primary" {
  shielded_instance_initial_state {
    dbs {
      content   = # value needed
      file_type = # value needed
    }
    dbxs {
      content   = # value needed
      file_type = # value needed
    }
    keks {
      content   = # value needed
      file_type = # value needed
    }
    pk {
      content   = # value needed
      file_type = # value needed
    }
  }
}

[modular-magician]

Tests analytics

Total tests: 1076
Passed tests: 1002
Skipped tests: 73
Affected tests: 1

Click here to see the affected service packages

• compute

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

• TestAccComputeInstanceFromTemplate_confidentialInstanceConfigMain

Get to know how VCR tests work

[modular-magician]

🔴 Tests failed during RECORDING mode:
TestAccComputeInstanceFromTemplate_confidentialInstanceConfigMain [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

[BBBmau]

although you've added the fields it's still required to include example tests as well as tests that test the update functionality. More can be found here: https://googlecloudplatform.github.io/magic-modules/test/test/#add-an-update-test

[github-actions]

@NotTheEvilOne, this PR is waiting for action from you. If no action is taken, this PR will be closed in 28 days.

Please address any comments or change requests, or re-request review from a core reviewer if no action is required.

This notification can be disabled with the disable-automatic-closure label.

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 566 insertions(+))
google-beta provider: Diff ( 3 files changed, 566 insertions(+))
terraform-google-conversion: Diff ( 1 file changed, 191 insertions(+))

[modular-magician]

Tests analytics

Total tests: 1081
Passed tests: 1005
Skipped tests: 73
Affected tests: 3

Click here to see the affected service packages

• compute

Action taken

Found 3 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

• TestAccComputeImage_shieldedInstanceInitialState
• TestAccComputeInstanceFromTemplate_confidentialInstanceConfigMain
• TestAccComputeRegionPerInstanceConfig_removeInstanceOnDestroy

Get to know how VCR tests work

[github-actions]

@GoogleCloudPlatform/terraform-team @BBBmau This PR has been waiting for review for 4 weeks. Please take a look! Use the label disable-review-reminders to disable these notifications.

[github-actions]

@GoogleCloudPlatform/terraform-team @BBBmau This PR has been waiting for review for 5 weeks. Please take a look! Use the label disable-review-reminders to disable these notifications.

[melinath]

@modular-magician reassign-reviewer

[SirGitsalot]

The problem is that the provider is sending a base64 string using standard encoding and getting back a base64 string in URL safe encoding. That's valid (the proto JSON encoding standard says "JSON value will be the data encoded as a string using standard base64 encoding with paddings. Either standard or URL-safe base64 encoding with/without paddings are accepted.") I'm actually surprised we haven't run into this before.

The fix is to assign a diff_suppress_func to content that does a standard encoding/URL encoding agnostic compare a la:

// Base64DiffSuppress compares two Base64 strings, ignoring differences
// between standard encoding and web safe URL encoding, padding, and 
// embedded line endings.
func Base64DiffSuppress(_, old, new string, _ *schema.ResourceData) bool {
	r := strings.NewReplacer("\r", "", "\n", "", "+", "-", "/", "_", "=", "")
	normalizedOld = r.Replace(old)
	normalizedNew = r.Replace(new)
	return normalizedOld == normalizedNew
}

I wrote this in the GitHub comment editor so it may be a little off. It should go in mmv1/third_party/terraform/tpgresource/common_diff_suppress.go.tmpl (along with an accompanying unit test in mmv1/third_party/terraform/tpgresource/common_diff_suppress_test.go please!)

[NotTheEvilOne]

Interesting. I'll have a look at this implementation detail.

[yeoldegrove]

@SirGitsalot I stepped in for @NotTheEvilOne and added Base64DiffSuppress and an additional test.

[SirGitsalot]

/gcbrun

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 6 files changed, 772 insertions(+), 3 deletions(-))
google-beta provider: Diff ( 6 files changed, 772 insertions(+), 3 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 191 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_compute_image (585 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_compute_image" "primary" {
  raw_disk {
    sha1 = # value needed
  }
}

[modular-magician]

Tests analytics

Total tests: 4614
Passed tests: 4176
Skipped tests: 432
Affected tests: 6

Click here to see the affected service packages

All service packages are affected

Action taken

Found 6 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

• TestAccCloudbuildWorkerPool_basic
• TestAccComputeImage_shieldedInstance_InitialState
• TestAccComputeImage_shieldedInstance_UpdatedState
• TestAccComputeInstanceFromTemplate_confidentialInstanceConfigMain
• TestAccDataSourceGoogleGkeHubFeature_basic
• TestAccEphemeralServiceAccountKey_basic

Get to know how VCR tests work

[modular-magician]

🟢 Tests passed during RECORDING mode:
TestAccComputeImage_shieldedInstance_InitialState [Debug log]
TestAccComputeImage_shieldedInstance_UpdatedState [Debug log]
TestAccDataSourceGoogleGkeHubFeature_basic [Debug log]

🔴 Tests failed when rerunning REPLAYING mode:
TestAccDataSourceGoogleGkeHubFeature_basic [Error message] [Debug log]

Tests failed due to non-determinism or randomness when the VCR replayed the response after the HTTP request was made.

Please fix these to complete your PR. If you believe these test failures to be incorrect or unrelated to your change, or if you have any questions, please raise the concern with your reviewer.

---

🔴 Tests failed during RECORDING mode:
TestAccCloudbuildWorkerPool_basic [Error message] [Debug log]
TestAccComputeInstanceFromTemplate_confidentialInstanceConfigMain [Error message] [Debug log]
TestAccEphemeralServiceAccountKey_basic [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

[github-actions]

@SirGitsalot This PR has been waiting for review for 3 weekdays. Please take a look! Use the label disable-review-reminders to disable these notifications.

[SirGitsalot]

The good news is that the diff suppress did fix the problem, the bad news is that there's a broken test elsewhere (but in more good news it's an easy fix - see the other comment)

[SirGitsalot]

It looks like shieldedInstanceInitialState needs default_from_api: true (documented here).

TestAccComputeInstanceFromTemplate_confidentialInstanceConfigMain creates a new google_compute_image from an existing Ubuntu image, and that Ubuntu image has shieldedInstanceInitialState set, causing a plan mismatch (that is, if you create a new image from an existing image that has shieldedInstanceInitialState but don't specify your own shieldedInstanceInitialState, the API is returning the shieldedInstanceInitialState from the source image; Terraform is expecting there to be no shieldedInstanceInitialState on output since none was specified on input).

[yeoldegrove]

@SirGitsalot added default_from_api: true

[SirGitsalot]

/gcbrun

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 6 files changed, 773 insertions(+), 3 deletions(-))
google-beta provider: Diff ( 6 files changed, 773 insertions(+), 3 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 191 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_compute_image (585 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_compute_image" "primary" {
  raw_disk {
    sha1 = # value needed
  }
}

[modular-magician]

Tests analytics

Total tests: 4645
Passed tests: 4202
Skipped tests: 436
Affected tests: 7

Click here to see the affected service packages

All service packages are affected

Action taken

Found 7 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

• TestAccAccessContextManager
• TestAccCloudbuildWorkerPool_basic
• TestAccDataSourceGoogleGkeHubFeature_basic
• TestAccEphemeralServiceAccountKey_basic
• TestAccSecretManagerSecretVersion_secretVersionBasicWriteOnlyExample
• TestAccSecretManagerSecretVersion_secretVersionWithBase64StringSecretDataWriteOnlyExample
• TestAccSqlUser_password_wo

Get to know how VCR tests work

[modular-magician]

🟢 Tests passed during RECORDING mode:
TestAccAccessContextManager [Debug log]
TestAccDataSourceGoogleGkeHubFeature_basic [Debug log]

🔴 Tests failed when rerunning REPLAYING mode:
TestAccDataSourceGoogleGkeHubFeature_basic [Error message] [Debug log]

Tests failed due to non-determinism or randomness when the VCR replayed the response after the HTTP request was made.

Please fix these to complete your PR. If you believe these test failures to be incorrect or unrelated to your change, or if you have any questions, please raise the concern with your reviewer.

---

🔴 Tests failed during RECORDING mode:
TestAccCloudbuildWorkerPool_basic [Error message] [Debug log]
TestAccEphemeralServiceAccountKey_basic [Error message] [Debug log]
TestAccSecretManagerSecretVersion_secretVersionBasicWriteOnlyExample [Error message] [Debug log]
TestAccSecretManagerSecretVersion_secretVersionWithBase64StringSecretDataWriteOnlyExample [Error message] [Debug log]
TestAccSqlUser_password_wo [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test