Tofu/terraform fails to apply multiple actions in restricted user mode (unix socket)

[gdonval]

Minmal non-working example:

terraform {
  required_providers {
    incus = {
      source = "lxc/incus"
      version = "0.2.0"
    }
  }
}

provider incus {
  # Configuration options
}


resource "incus_instance" "instance-user-1000-1"{
  project = "user-1000"
  name    = "gene-instance1"
  image   = "images:alpine/edge"
  wait_for_network = false
}

resource "incus_instance" "instance-user-1000-2"{
  project = "user-1000"
  name    = "gene-instance2"
  image   = "images:alpine/edge"
  wait_for_network = false
}

---

$ tofu apply
... # plan

Plan: 2 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  OpenTofu will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

incus_instance.instance-user-1000-1: Creating...
incus_instance.instance-user-1000-2: Creating...
incus_instance.instance-user-1000-1: Creation complete after 3s [name=gene-instance1]
╷
│ Error: Failed to retrieve Incus InstanceServer
│ 
│   with incus_instance.instance-user-1000-2,
│   on test.tf line 22, in resource "incus_instance" "instance-user-1000-2":
│   22: resource "incus_instance" "instance-user-1000-2"{
│ 
│ Environment variable INCUS_DIR points to an Incus directory that does not contain a writable unix socket
╵

It looks like tofu reverts to try using the privileged unix socket instead of the user one when more than 1 action is queued. I guess that this is not a problem if using the remote API.

[stgraber]

@maveonair any idea what's going on here?

[maveonair]

Do you run Opentofu and Terraform on the same computer as the restricted project user-1000?

---

I tested it on my Macbook with OpenTofu against a Debian Bookworm server with a restricted project, and it worked fine:

Client Setup:

Server:

$ sudo incus config trust add --projects user-1000 --restricted mbpro

Client:

$ incus remote add incus-dev https://incus-dev:8443

OpenTofu / Terraform Config:

terraform {
  required_providers {
    incus = {
      source  = "lxc/incus"
      version = "0.2.0"
    }
  }
}

provider "incus" {
  accept_remote_certificate    = true
  generate_client_certificates = false

  remote {
    name    = "incus-dev" # my remote server
    default = true
  }
}

resource "incus_instance" "instance-user-1000-1" {
  project          = "user-1000"
  name             = "gene-instance1"
  image            = "images:alpine/edge"
  wait_for_network = false
}

resource "incus_instance" "instance-user-1000-2" {
  project          = "user-1000"
  name             = "gene-instance2"
  image            = "images:alpine/edge"
  wait_for_network = false
}

Running Tofu:

$ tofu apply

OpenTofu used the selected providers to generate the following execution plan. Resource actions are indicated with the following
symbols:
  + create

OpenTofu will perform the following actions:

  # incus_instance.instance-user-1000-1 will be created
  + resource "incus_instance" "instance-user-1000-1" {
      + config           = {}
      + ephemeral        = false
      + image            = "images:alpine/edge"
      + ipv4_address     = (known after apply)
      + ipv6_address     = (known after apply)
      + mac_address      = (known after apply)
      + name             = "gene-instance1"
      + profiles         = [
          + "default",
        ]
      + project          = "user-1000"
      + running          = true
      + status           = (known after apply)
      + target           = (known after apply)
      + type             = "container"
      + wait_for_network = false
    }

  # incus_instance.instance-user-1000-2 will be created
  + resource "incus_instance" "instance-user-1000-2" {
      + config           = {}
      + ephemeral        = false
      + image            = "images:alpine/edge"
      + ipv4_address     = (known after apply)
      + ipv6_address     = (known after apply)
      + mac_address      = (known after apply)
      + name             = "gene-instance2"
      + profiles         = [
          + "default",
        ]
      + project          = "user-1000"
      + running          = true
      + status           = (known after apply)
      + target           = (known after apply)
      + type             = "container"
      + wait_for_network = false
    }

Plan: 2 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  OpenTofu will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

incus_instance.instance-user-1000-1: Creating...
incus_instance.instance-user-1000-2: Creating...
incus_instance.instance-user-1000-1: Creation complete after 7s [name=gene-instance1]
incus_instance.instance-user-1000-2: Creation complete after 7s [name=gene-instance2]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

$ incus ls
+----------------+---------+----------------------+-----------------------------------------------+-----------+-----------+
|      NAME      |  STATE  |         IPV4         |                     IPV6                      |   TYPE    | SNAPSHOTS |
+----------------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| gene-instance1 | RUNNING | 10.28.248.116 (eth0) | fd42:95a8:a939:c62e:216:3eff:fe8e:1223 (eth0) | CONTAINER | 0         |
+----------------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| gene-instance2 | RUNNING | 10.28.248.120 (eth0) | fd42:95a8:a939:c62e:216:3eff:fed4:29b4 (eth0) | CONTAINER | 0         |
+----------------+---------+----------------------+-----------------------------------------------+-----------+-----------+

[gdonval]

Do you run Opentofu and Terraform on the same computer as the restricted project user-1000?

Yes indeed. I'm sorry if I did not make that obvious. I fully expect explicit remotes to work well. :)

It looks like the first time tofu tries to perform an action, it tries the privileged socket and then the user one. Then, on subsequent actions, it fails to fallback to the user one.

[maveonair]

I fixed your issue in the following PR: #203