Bring back DAX #205
Bring back DAX #205
Conversation
We will use half for DAX and half for virtio-gpu. Signed-off-by: Asahi Lina <[email protected]>
With DAX disabled, the shmem region was reused for virtio-gpu. Split it in half and use one half for DAX and the other half for virtio-gpu. Signed-off-by: Asahi Lina <[email protected]>
Commit eac9478 ("virtio/fs: move work to a separate thread") removed support for shmem regions, and therefore DAX. Pass through the shmem region to the worker to make DAX work again. Fixes: eac9478 ("virtio/fs: move work to a separate thread") Signed-off-by: Asahi Lina <[email protected]>
This reverts commit 0baaa7f. Signed-off-by: Asahi Lina <[email protected]>
|
Hey @asahilina, what's the best way to test that DAX is working again? |
|
The kernel will try to use it unconditionally if the mount flag is present (that's how I figured out what was missing, enabling DAX just broke everything until I got it right).
I haven't truly tested it end to end though, but I'll find out soon enough if it's working as advertised since a use case I'm interested in relies on coherent /dev/shm memory mappings between the host and the guest. ^^
…On July 8, 2024 2:10:41 AM GMT+09:00, David Gageot ***@***.***> wrote:
Hey @asahilina, what's the best way to test that DAX is working again?
--
Reply to this email directly or view it on GitHub:
#205 (comment)
You are receiving this because you were mentioned.
Message ID: ***@***.***>
|
Thanks a lot! If I understand correctly, your use case is using DAX on the boot "disk" that's over virtiofs. I'm interested in using DAX on virtiofs mounts for file sharing between the host and the guest and couldn't see a difference with this PR. |
|
You need to set the "dax" mount option on your mount, as the last commit (revert) does for the root filesystem. If that works and the flag shows up in /proc/mounts then it should be working as far as I know!
…On July 8, 2024 4:23:23 PM GMT+09:00, David Gageot ***@***.***> wrote:
> The kernel will try to use it unconditionally if the mount flag is present (that's how I figured out what was missing, enabling DAX just broke everything until I got it right). I haven't truly tested it end to end though, but I'll find out soon enough if it's working as advertised since a use case I'm interested in relies on coherent /dev/shm memory mappings between the host and the guest. ^^
> […](#)
Thanks a lot! If I understand correctly, your use case is using DAX on the boot "disk" that's over virtiofs. I'm interested in using DAX on virtiofs mounts for file sharing between the host and the guest and couldn't see a difference with this PR.
But it's more than possible that I understood it wrong or that I'm doing it wrong.
--
Reply to this email directly or view it on GitHub:
#205 (comment)
You are receiving this because you were mentioned.
Message ID: ***@***.***>
|
|
Thank! Yes, that's what I did and in does support the |
| intc.clone(), | ||
| virgl_flags, | ||
| #[cfg(target_os = "macos")] | ||
| _map_sender, | ||
| )?; | ||
| } | ||
| #[cfg(not(feature = "tee"))] | ||
| attach_fs_devices(&mut vmm, &vm_resources.fs, None, intc.clone())?; | ||
| attach_fs_devices(&mut vmm, &vm_resources.fs, dax_shm_region, intc.clone())?; |
There was a problem hiding this comment.
I have the feeling, from what I have tested, that this share memory region can't be used by more than one virtio device.
There was a problem hiding this comment.
Seems to work fine for me? I don't see why it wouldn't work as long as the range used by each device does not overlap (which is why I split it in half here).
There was a problem hiding this comment.
Yes, the half/half splitting between gpu and fs clearly works.
If the user has two virtiofs devices, they end up sharing the exact same portion, though.
That makes all my virtiofs devices fail to setup, except for the first one.
There was a problem hiding this comment.
Ohh, sorry, I misunderstood. Maybe @slp knows what the intent was here? I only brought back the previous behavior from back when DAX was enabled, so I'm guessing this same problem already existed before?
There was a problem hiding this comment.
We need to implement some kind of shm region manager that manages the window and can be used to request regions of a certain size. I'm adding this to my TODO list and I think I'll have bandwidth to implement it next week, in case no one beats me implementing it first.
| @@ -554,13 +554,27 @@ pub fn build_microvm( | |||
| )?; | |||
| } | |||
|
|
|||
| // Use half of the shmem region for virtio-gpu and half for DAX | |||
| let _shm_region_size = arch_memory_info.shm_size >> 1; | |||
There was a problem hiding this comment.
My rust is still very recent: should the underscore be removed, here?
There was a problem hiding this comment.
That silences the unused variable warning, in the case of a build with the tee feature enabled and the gpu feature disabled (which would cause it to become unused).
|
This one got superseded by #212 |
Re-add the shmem region required for DAX again (after it got removed when virtio-gpu was implemented), and enable DAX.