Provisioning of Cloud logs instance fails when cos buckets are attached in parameters

[maheshwarishikha]

Provisioning of ICL fails if COS bucket details are provided in parameters, though all required auth policies are in-place.

 Error: [ERROR] Error waiting for create resource instance (crn:v1:bluemix:public:logs:us-south:a/abac0df06b644a9cabc6e44f55b3880e:227aa5ac-e97a-4477-b99d-c33a193cae36::) to be succeeded: [ERROR] The resource instance 'crn:v1:bluemix:public:logs:us-south:a/abac0df06b644a9cabc6e44f55b3880e:227aa5ac-e97a-4477-b99d-c33a193cae36::' creation failed: <nil>
│ 
│   with ibm_resource_instance.cloud_logs,
│   on test.tf line 62, in resource "ibm_resource_instance" "cloud_logs":
│   62: resource "ibm_resource_instance" "cloud_logs" {
│ 
│ ---
│ id: terraform-446cc00a
│ summary: '[ERROR] Error waiting for create resource instance (crn:v1:bluemix:public:logs:us-south:a/abac0df06b644a9cabc6e44f55b3880e:227aa5ac-e97a-4477-b99d-c33a193cae36::)
│   to be succeeded: [ERROR] The resource instance ''crn:v1:bluemix:public:logs:us-south:a/abac0df06b644a9cabc6e44f55b3880e:227aa5ac-e97a-4477-b99d-c33a193cae36::''
│   creation failed: <nil>'
│ severity: error
│ resource: ibm_resource_instance
│ operation: create
│ component:
│   name: github.com/IBM-Cloud/terraform-provider-ibm
│   version: 1.71.0

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform IBM Provider Version

tf 1.9.2
provider 1.70.0 / 1.71.1

Affected Resource(s)

• ibm_resource_instance

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

# Copy-paste your Terraform configurations here - for large Terraform configs,
# please share a link to the ZIP file.
provider "ibm" {
  ibmcloud_api_key = var.ibmcloud_api_key
  region           = "us-south"
}

terraform {
  required_version = ">= 1.3.0"
  required_providers {
    ibm = {
      source  = "ibm-cloud/ibm"
      version = "1.71.0"
    }
    time = {
      source  = "hashicorp/time"
      version = "0.12.1"
    }
  }
}

variable "ibmcloud_api_key" {
  type        = string
  description = "The IBM Cloud API Token"
  sensitive   = true
}


data "ibm_resource_group" "cos_group" {
  name = "Default"
}

resource "ibm_resource_instance" "cos_instance" {
  name              = "debug-cos"
  resource_group_id = data.ibm_resource_group.cos_group.id
  service           = "cloud-object-storage"
  plan              = "standard"
  location          = "global"
  tags              = []
}

resource "ibm_cos_bucket" "cos_bucket" {
  bucket_name          = "debug-icl-bucket"
  resource_instance_id = ibm_resource_instance.cos_instance.id
  storage_class        = "standard"
  endpoint_type = "public"
  region_location = "us-south"
}

resource "ibm_iam_authorization_policy" "cos_policy" {
  source_service_name      = "logs"
  source_resource_group_id = data.ibm_resource_group.cos_group.id
  roles                    = ["Writer"]
  description              = "Allow Cloud logs instances `Writer` access to the COS bucket."
  target_service_name         = "cloud-object-storage"
  target_resource_instance_id = ibm_resource_instance.cos_instance.id
}

resource "time_sleep" "wait_for_cos_authorization_policy" {
  depends_on = [ibm_iam_authorization_policy.cos_policy]
  create_duration = "30s"
}

resource "ibm_resource_instance" "cloud_logs" {
  depends_on        = [time_sleep.wait_for_cos_authorization_policy]
  name              = "debug-cloudlogs"
  resource_group_id = data.ibm_resource_group.cos_group.id
  service           = "logs"
  plan              = "standard"
  tags              = []
  location          = "us-south"
  parameters = {
    "logs_bucket_crn"         = ibm_cos_bucket.cos_bucket.crn
    "logs_bucket_endpoint"    = ibm_cos_bucket.cos_bucket.s3_endpoint_public
  }
}

Debug Output

Panic Output

Expected Behavior

It should create Cloud logs instance successfully.

Actual Behavior

Provisioning fails.

Steps to Reproduce

1. terraform apply

Important Factoids

References

• #0000

[kavya498]

@maheshwarishikha, Looks like the authorisation policy is not set right.
It is not been used..
Can you use guid instead of id for target_instance_id?
target_resource_instance_id = ibm_resource_instance.cos_instance.id ==> target_resource_instance_id = ibm_resource_instance.cos_instance.guid
something like below?

resource "ibm_iam_authorization_policy" "cos_policy" {
  source_service_name      = "logs"
  source_resource_group_id = data.ibm_resource_group.cos_group.id
  roles                    = ["Writer"]
  description              = "Allow Cloud logs instances `Writer` access to the COS bucket."
  target_service_name         = "cloud-object-storage"
  target_resource_instance_id = ibm_resource_instance.cos_instance.guid
}

[maheshwarishikha]

Thanks for the response @kavya498

Actually that was typo when I created a test code, changing it to guid worked for me. But we have auth policy in GoldenEye code scoped to the bucket like this https://github.com/terraform-ibm-modules/terraform-ibm-observability-instances/blob/main/modules/cloud_logs/main.tf#L44
that was working all fine few days back and now giving error. Checking more on this...

[maheshwarishikha]

@kavya498 , If we changed our auth policy like as below....i.e. scoped to the cos bucket

resource "ibm_iam_authorization_policy" "cos_policy" {
  source_service_name      = "logs"
  source_resource_group_id = data.ibm_resource_group.cos_group.id
  roles                    = ["Writer"]
  description              = "Allow Cloud logs instances `Writer` access to the COS bucket."
#   target_service_name         = "cloud-object-storage"
#   target_resource_instance_id = ibm_resource_instance.cos_instance.guid

  resource_attributes {
    name     = "serviceName"
    operator = "stringEquals"
    value    = "cloud-object-storage"
  }

  resource_attributes {
    name     = "accountId"
    operator = "stringEquals"
    value    = "xxx"
  }

  resource_attributes {
    name     = "serviceInstance"
    operator = "stringEquals"
    value    = ibm_resource_instance.cos_instance.guid
  }

  resource_attributes {
    name     = "resourceType"
    operator = "stringEquals"
    value    = "bucket"
  }

  resource_attributes {
    name     = "resource"
    operator = "stringEquals"
    value    = "debug-icl-bucket"
  }
}

ICL provisioning fails. It was working all fine few days back. Can you please help to find out whats missing in auth policy?? Or it is happening because of any change in provider code.

[maheshwarishikha]

Nopes, recreated everything and it worked this time. Checking in GoldenEye Module now...
.

[ocofaigh]

@kavya498 It would be nice if the service could respond with a nicer error message if you know that its an issue related to auth policy. Is that something you plan on improving?

[maheshwarishikha]

Issue identified is with the metrics bucket attachment. Cloud logs team is working on this currently - internal issue to track is https://github.ibm.com/Observability/logs-support/issues/223

Regarding the proper error message, it is being tracked by #5729

Regarding the fix related to metrics bucket attachment, we will receive an update via the support ticket. Hence closing this issue.