Consider documenting the specific terraform commands executed during acceptance tests

[bendbennett]

Does this documentation exist?

• This is new documentation
• This is an enhancement to existing documentation

Where would you expect to find this documentation?

• On terraform.io
• In the GoDoc for this module
• In this repo as a markdown file
• Somewhere else

Details

This documentation could appear in Acceptance Tests and provide details of the specific terraform commands that are executed during a particular "test mode".

For example, if an acceptance test is set-up to use TestStep.Config:

func TestAccExampleResource(t *testing.T) {
	resource.Test(t, resource.TestCase{
		PreCheck:                 func() { testAccPreCheck(t) },
		ProtoV6ProviderFactories: testAccProtoV6ProviderFactories,
		Steps: []resource.TestStep{
			// Create and Read testing
			{
				Config: `resource "example_resource" "test" {
	id = "one"
}`,
				Check: resource.ComposeAggregateTestCheckFunc(
					resource.TestCheckResourceAttr("example_resource.test", "configurable_attribute", "1.797693134862315"),
					resource.TestCheckResourceAttr("example_resource.test", "id", "one"),
				),
			},
		},
	})
}

The following terraform commands are executed during the course of the test:

terraform init 
terraform plan
terraform apply
terraform show (state)
terraform plan
terraform show (plan)
terraform plan
terraform show (plan)
terraform show (state)
terraform destroy

Description

If the terraform command calls are documented and include details as to which flags are supplied (e.g., -refresh-only, -refresh=true|false) this might be useful for provider developers who are writing acceptance tests and aid in the generation of a mental model of which commands are being called and how this relates to direct execution of Terraform commands.

References

• helper/resource: Clarify configuration step error messages #238
• helper/resource: Refresh during plan rather than separately and remove state shimming when possible #223

[bflad]

I'm generally 👍 for this, although I caution about putting too many implementation details into this sort of documentation as:

• It can be difficult to accurately maintain over time
• It can potentially cause consumers to depend on those specific implementation details
• Some of the details are generally irrelevant for most provider developers, like how terraform show is used (many times) to inspect the plan/state -- that's the interface Terraform gives, so its expected/required

My recommendation would be saying things like:

• A configuration-based step without PlanOnly runs plan, apply, a non-refresh plan, and a refresh plan
• A configuration-based step with PlanOnly runs a non-refresh plan and a refresh plan

etc.