Unable to deploy app on recently patched AL2023 servers

[briri]

We were unable to run the Capistrano deploy on the recently patched dev and stg instances after the latest kernel updates. Capistrano (technically rake assets:precompile) is getting stuck when running the yarn install command. The logs stop on the linking dependencies step.

Once the process gets stuck, it runs forever. examination of ps aux | grep node shows them as "uninterruptible" processes, so a server reboot is needed to kill them (not even sudo kill -9 was working).

It turns out that the PSA Martin posted in early December:

Amazon Linux 2023 release 2023.6.20241111.0 with the Linux kernel 6.1.115-126.197.amzn2023 is affected by an issue in the io_uring subsystem. It can be observed when running the npm cli, and results in hung processes in uninterruptible sleep state (“D” in ps output).
To work around the issue, temporarily disable the libuv use of io_uring by setting the corresponding environment variable UV_USE_IO_URING=0. Using an earlier Linux kernel version also works around the issue. A correction will be made available by December 9, 2024.
I experienced npm hanging yesterday, so the kernel bug is real. The workaround worked for me.
If you use npm (or yarn) and have the option, avoid upgrading the AL2023 kernel until Dec 9.

On dev, I was able to set the UV_USE_IO_URING=0 env variable and run yarn install manually from within the /dmp/install/dmptool directory successfully afterward.

We are past the Dec. 9th date AWS promised to release a patch for this, so I think it's worth seeing if patching the dev/stg servers resolves the issue.