system freezes on NovaCustom V54 (SSD polling?)

[mouflon72]

Qubes OS release

4.2.3 running kernel-latest

Brief summary

• certain actions in VMs cause the whole system to partially freeze

Steps to reproduce

a few actions that seemed to trigger this behaviour so far (will update if there are more observations):

• running a fast (~20MB/s) download in a Debian 12 VM over some time
• mounting an external Exfat USB drive on a Debian 12 VM
• attaching and using a USB microphone on a Debian 12 VM
• edit: sometimes just starting the system i enough, where e.g. sys-net freezes and forces me to hard reset

Expected behavior

system runs smoothly, no freezes

Actual behavior

• system partly freezes, only some interaction with windows is still possible, e.g: I can click on an e-mail in a Thunderbird window, but the content won't get displayed
• VMs that seem to cause this can't be shut down / killed any more, only hard system reset helps
• dom0 terminal, e.g. running dmesg remains working

The difference I see in kernel log between functioning / non-functioning state is that I see repeated messages like "nvme nvme0: I/O xxx QID x timeout, completion polled".
I haven't really found any issues here referencing this message except referring to AMD Ryzen systems with hyperthreading enabled.

Note: I am not using NovaCustom OEM SSD but a SanDisk Extreme PRO M.2 NVMe, which I already previously used in another Qubes installation on a Thinkpad (not using kernel-latest though). It was working fine there.

[mouflon72]

I moved my system to a Goodram PX-700 now, which seems to have fixed the SSD timeouts. The SanDisk was PCIe 3, the new one is PCIe 4.0.
Partial freeze wih Exfat drive remains, but that seems to have a different cause. Will open a seperate issue, if that persists.

[apparatius]

The difference I see in kernel log between functioning / non-functioning state is that I see repeated messages like "nvme nvme0: I/O xxx QID x timeout, completion polled".

Try to switch the storage mode from RAID to AHCI/NVME or disable Intel Volume Management Device (VMD) in BIOS.