Running terraform destroy produces an error although resources are deleted

[eldada]

Using the example in https://github.com/eldada/terraform-playground/tree/main/4.artifactory-config, I applied some repositories, users and groups.
When I run terraform destroy to clean them up, I get the following errors, although the resources are successfully deleted!

│ Error: Unable to Delete Resource
│ 
│ An unexpected error occurred while attempting to delete the resource. Please retry the operation or report this issue to the provider developers.
│ 
│ Error: {
│   "errors" : [ {
│     "status" : 500,
│     "message" : "HTTP response status 500:Failed on executing request. Response: {\n  \"errors\" : [ {\n    \"code\" : \"INTERNAL_SERVER_ERROR\",\n    \"message\" : \"Something went wrong\"\n  } ]\n}"
│   } ]
│ }
╵
╷
│ Error: Unable to Delete Resource
│ 
│ An unexpected error occurred while attempting to delete the resource. Please retry the operation or report this issue to the provider developers.
│ 
│ Error: {
│   "errors" : [ {
│     "status" : 500,
│     "message" : "HTTP response status 500:Failed on executing request. Response: {\n  \"errors\" : [ {\n    \"code\" : \"INTERNAL_SERVER_ERROR\",\n    \"message\" : \"Something went wrong\"\n  } ]\n}"
│   } ]
│ }

I see in access-service.log the following

2024-11-04T04:56:26.902Z [jfac ] [ERROR] [98c116449019d226] [.e.m.DefaultExceptionMapper:30] [27.0.0.1-8040-exec-5] - Request DELETE /access/v1/groups/ops failed: Cannot invoke "org.jfrog.access.server.model.User.getStatus()" because "newUser" is null
java.lang.NullPointerException: Cannot invoke "org.jfrog.access.server.model.User.getStatus()" because "newUser" is null
	at org.jfrog.access.server.service.jfevent.JFEventServiceImpl.isUserLocked(JFEventServiceImpl.java:97)
	at org.jfrog.access.server.service.jfevent.JFEventServiceImpl.triggerEvent(JFEventServiceImpl.java:86)
	at java.base/java.lang.Iterable.forEach(Iterable.java:75)
	at org.jfrog.access.server.service.jfevent.JFEventServiceImpl.onSecurityChanges(JFEventServiceImpl.java:112)

[alexhung]

@eldada Thanks for the report. I'll add this to our plan to investigate.

[alexhung]

@eldada The reason this error is triggered is because the artifactory_user configuration are not using references to the artifactory_group resources. This means the user resource may (and likely) will be deleted before the groups, since TF does not know of their dependent relationship.

When this error occurs, TF doesn't remove the resource from the TF state file. Here's mine after getting the error:

alexh@alexh-mac terraform-provider-artifactory % terraform show
# artifactory_group.devs:
resource "artifactory_group" "devs" {
    admin_privileges = false
    auto_join        = true
    description      = "Developers group"
    external_id      = null
    id               = "devs"
    name             = "devs"
    policy_manager   = false
    realm            = "internal"
    realm_attributes = null
    reports_manager  = false
    watch_manager    = false
}

The content of the state file:

{
  "version": 4,
  "terraform_version": "1.9.5",
  "serial": 78,
  "lineage": "4d46d97b-a59a-4eb7-6185-0d05564a65d2",
  "outputs": {},
  "resources": [
    {
      "mode": "managed",
      "type": "artifactory_group",
      "name": "devs",
      "provider": "provider[\"registry.terraform.io/jfrog/artifactory\"]",
      "instances": [
        {
          "schema_version": 0,
          "attributes": {
            "admin_privileges": false,
            "auto_join": true,
            "description": "Developers group",
            "detach_all_users": null,
            "external_id": "",
            "id": "devs",
            "name": "devs",
            "policy_manager": false,
            "realm": "internal",
            "realm_attributes": "",
            "reports_manager": false,
            "users_names": null,
            "watch_manager": false
          },
          "sensitive_attributes": []
        }
      ]
    }
  ],
  "check_results": null
}

However, when I check my local Artifactory, the group in question no longer exists. So even though the groups API returns the error, the group is deleted eventually?

Running terraform plan shows that TF will attempt to create this group "devs" again since it doesn't exist on Artifactory anymore.

alexh@alexh-mac terraform-provider-artifactory % terraform plan
artifactory_group.devs: Refreshing state... [id=devs]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # artifactory_group.devs will be created
  + resource "artifactory_group" "devs" {
      + admin_privileges = false
      + auto_join        = true
      + description      = "Developers group"
      + id               = (known after apply)
      + name             = "devs"
      + policy_manager   = false
      + realm            = "artifactory"
      + reports_manager  = false
      + watch_manager    = false
        # (2 unchanged attributes hidden)
    }

  # artifactory_group.ops will be created
  + resource "artifactory_group" "ops" {
      + admin_privileges = false
      + auto_join        = true
      + description      = "Operations group"
      + id               = (known after apply)
      + name             = "ops"
      + policy_manager   = false
      + realm            = "artifactory"
      + reports_manager  = false
      + watch_manager    = false
        # (2 unchanged attributes hidden)
    }

  # artifactory_local_generic_repository.generic-local will be created
  + resource "artifactory_local_generic_repository" "generic-local" {
      + blacked_out          = false
      + cdn_redirect         = false
      + id                   = (known after apply)
      + includes_pattern     = "**/*"
      + key                  = "generic-local"
      + package_type         = (known after apply)
      + priority_resolution  = false
      + project_environments = (known after apply)
      + repo_layout_ref      = "simple-default"
      + xray_index           = false
    }

  # artifactory_local_maven_repository.maven-local will be created
  + resource "artifactory_local_maven_repository" "maven-local" {
      + blacked_out                     = false
      + cdn_redirect                    = false
      + checksum_policy_type            = "client-checksums"
      + handle_releases                 = true
      + handle_snapshots                = true
      + id                              = (known after apply)
      + includes_pattern                = "**/*"
      + key                             = "maven-local"
      + max_unique_snapshots            = 10
      + package_type                    = (known after apply)
      + priority_resolution             = false
      + project_environments            = (known after apply)
      + repo_layout_ref                 = "maven-2-default"
      + snapshot_version_behavior       = "unique"
      + suppress_pom_consistency_checks = false
      + xray_index                      = false
    }

  # artifactory_remote_maven_repository.maven-remote will be created
  + resource "artifactory_remote_maven_repository" "maven-remote" {
      + allow_any_host_auth                   = false
      + archive_browsing_enabled              = false
      + assumed_offline_period_secs           = 300
      + blacked_out                           = false
      + block_mismatching_mime_types          = true
      + bypass_head_requests                  = false
      + cdn_redirect                          = false
      + client_tls_certificate                = (known after apply)
      + curated                               = false
      + disable_proxy                         = false
      + disable_url_normalization             = false
      + download_direct                       = false
      + enable_cookie_management              = false
      + fetch_jars_eagerly                    = true
      + fetch_sources_eagerly                 = false
      + handle_releases                       = true
      + handle_snapshots                      = true
      + hard_fail                             = false
      + id                                    = (known after apply)
      + includes_pattern                      = "**/*"
      + key                                   = "maven-remote"
      + list_remote_folder_items              = false
      + max_unique_snapshots                  = 0
      + metadata_retrieval_timeout_secs       = 120
      + missed_cache_period_seconds           = 1800
      + offline                               = false
      + package_type                          = (known after apply)
      + priority_resolution                   = false
      + project_environments                  = (known after apply)
      + reject_invalid_jars                   = true
      + remote_repo_checksum_policy_type      = "generate-if-absent"
      + repo_layout_ref                       = "maven-2-default"
      + retrieval_cache_period_seconds        = 7200
      + share_configuration                   = (known after apply)
      + socket_timeout_millis                 = 15000
      + store_artifacts_locally               = true
      + suppress_pom_consistency_checks       = false
      + synchronize_properties                = false
      + unused_artifacts_cleanup_period_hours = 0
      + url                                   = "https://repo1.maven.org/maven2/"
      + xray_index                            = false

      + content_synchronisation (known after apply)
    }

  # artifactory_user.developer1 will be created
  + resource "artifactory_user" "developer1" {
      + admin                      = false
      + disable_ui_access          = true
      + email                      = "[email protected]"
      + groups                     = [
          + "devs",
        ]
      + id                         = (known after apply)
      + internal_password_disabled = false
      + name                       = "developer1"
      + password                   = (sensitive value)
      + profile_updatable          = true
    }

  # artifactory_user.developer2 will be created
  + resource "artifactory_user" "developer2" {
      + admin                      = false
      + disable_ui_access          = true
      + email                      = "[email protected]"
      + groups                     = [
          + "devs",
          + "ops",
        ]
      + id                         = (known after apply)
      + internal_password_disabled = false
      + name                       = "developer2"
      + password                   = (sensitive value)
      + profile_updatable          = true
    }

  # artifactory_virtual_maven_repository.maven-virtual will be created
  + resource "artifactory_virtual_maven_repository" "maven-virtual" {
      + artifactory_requests_can_retrieve_remote_artifacts = false
      + description                                        = "A test virtual repo"
      + force_maven_authentication                         = true
      + id                                                 = (known after apply)
      + includes_pattern                                   = "**/*"
      + key                                                = "maven-virtual"
      + notes                                              = "Internal description"
      + package_type                                       = (known after apply)
      + pom_repository_references_cleanup_policy           = "discard_active_reference"
      + project_environments                               = (known after apply)
      + repo_layout_ref                                    = "maven-2-default"
      + repositories                                       = [
          + "maven-local",
          + "maven-remote",
        ]
    }

Plan: 8 to add, 0 to change, 0 to destroy.

─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now.

My conclusion so far is that the TF resource is working as designed.

The question is why does Artifactory deletes the group even though it clearly encounters an error?

[eldada]

I think we might need to show this to Artifactory dev and ask for their guidance? Can it be my template is malformed? Can you confirm?

[alexhung]

@eldada Yes, in your template, the user resources are using hardcoded group name. This means TF doesn't know of the dependencies between user and group resources, and therefore will delete the groups and users in non-deterministic ordering.

When I swap them to references, everything works fine.

e.g.

# Create the user developer1 and add them to the devs group
resource "artifactory_user" "developer1" {
    name     = "developer1"
    email    = "[email protected]"
    password = "Password1"
    admin    = false
    groups   = [artifactory_group.devs.name]
}

# Create the user developer2 and add them to the devs and ops group
resource "artifactory_user" "developer2" {
    name     = "developer2"
    email    = "[email protected]"
    password = "Password2"
    admin    = false
    groups   = [artifactory_group.devs.name, artifactory_group.ops.name]
}

For the error in Artifactory, we should definitely pass the issue onto the dev team.

[eldada]

Good. I'll fix my template example. I'll try to take this internally as well. Thx!