Desktop Icons do not stick to desktops when swirtching desktops on X11

[Cm4nXD]

This behavior occurs on both Debian 12, and Ubuntu 23.04 with gnome 43, and gnome 44 respectively.

Expected Behavior - Screenshot with Gtk4 Desktop Icons NG (DING):

Actual Behavior - Screenshot with Desktop Icons Neo:

I̶ ̶h̶a̶v̶e̶ ̶n̶o̶t̶ ̶t̶e̶s̶t̶e̶d̶ ̶o̶n̶ ̶W̶a̶y̶l̶a̶n̶d̶,̶ ̶a̶s̶ ̶I̶ ̶c̶a̶n̶n̶o̶t̶ ̶g̶e̶t̶ ̶W̶a̶y̶l̶a̶n̶d̶ ̶t̶o̶ ̶w̶o̶r̶k̶ ̶w̶i̶t̶h̶ ̶m̶y̶ ̶N̶V̶I̶D̶I̶A̶ ̶g̶r̶a̶p̶h̶i̶c̶s̶ ̶c̶a̶r̶d̶,̶ ̶n̶o̶r̶ ̶h̶a̶v̶e̶ ̶i̶ ̶t̶e̶s̶t̶e̶d̶ ̶o̶n̶ ̶a̶n̶y̶t̶h̶i̶n̶g̶ ̶o̶t̶h̶e̶r̶ ̶t̶h̶a̶n̶ ̶D̶e̶b̶i̶a̶n̶ ̶a̶n̶d̶ ̶U̶b̶u̶n̶t̶u̶.̶

[Cm4nXD]

Can confirm on Fedora 36 / 37 / 38 is broken as well. I am fairly confident the only thing consistent is x11 display server. The issue doesn't seem to affect Wayland.