adds wait_for_shutoff to resource_libvirt_domain

[marshallford]

Open to feedback, there is likely a better way to implement this. I took a first swing at the docs, but I wasn't sure where to start on testing.

Tested with Terraform version 1.7.0.

# kvm host used for testing
ubuntu@test:~$ uname -a
Linux test 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
ubuntu@test:~$ kvm --version
QEMU emulator version 6.2.0 (Debian 1:6.2+dfsg-2ubuntu6.16)
Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers

# /etc/systemd/system/test-wait.service in guest VM to validate delayed shut off.
[Unit]
Description=...

[Service]
Type=oneshot
RemainAfterExit=true
ExecStop=/usr/bin/sleep 90

[Install]
WantedBy=multi-user.target

Fixes: #356, #1060

[dmacvicar]

While I acknowledge the problem, I am not sure we are fixing it in the right place.

• How do other cloud providers behave here? do we have some similar setting to model against?
• Given that we already track "running" as state, wouldn't make more sense to implement graceful shutdown there?
• If we implement it in Destroy, wouldn't make more sense to always do it graceful and just have a timeout for destroy? (those are customizable, so one could have a 0 timeout)

[Silvenga]

How do other cloud providers behave here?

I don't think cloud providers ultimately need to destroy the resource to update it e.g. scaling up typically requires just rebooting the node. This is mostly a TF'isum, where it's just easier to think of the resources as immutable.

To my knowledge, the cloud providers that I'm aware of don't do graceful shutdowns on destroying (Digital Ocean and Azure).

If we implement it in Destroy, wouldn't make more sense to always do it graceful and just have a timeout for destroy? (those are customizable, so one could have a 0 timeout)

IMO, this ultimately should be opt-in - as there are a lot of cases that would prevent libvirt from communicating with the guest for a graceful shutdown. Some OS's don't even respond to graceful shutdowns, one being at least some versions of VyOS (a router OS, "common" in public clouds). I also don't believe the agent is installed by default in at least the slim version of Ubuntu Server.

[scabala]

I really like this comment on #356 - there should be just a timeout variable that controls the logic. If it is greater than 0, do graceful shutdown and force shutdown when it expires. If it is 0, do forceful shutdown as currently it is done.

[dmacvicar]

Yes, it is a good solution. I hope people don't confuse it with the destroy timeout that terraform provides as a setting (which is the timeout of the destroy function itself).

[tiaden]

Here is a link to an implementation of this issue that I worked on some time ago, which may provide an alternative perspective:
GitHub link.

I have successfully used this solution in my projects without encountering any problems. It leverages the "running" attribute to shut down the virtual machine, utilizing the graceful default shutdown method or the QEMU agent (if specified), and, as a last resort, forcibly powers off the machine if the default Terraform method timeout is reached. If you'd like to test it quickly, you can use the associated Terraform provider available here: Terraform Provider.

I hope this helps in some way