Device: Don't remove an existing target directory when unmounting a disk device if the original dir hasn't been created by LXD
#12700
Conversation
|
Working on the integration tests now. |
gabrielmougard
force-pushed
the
fix/rm-disk-dev-deletes-empty--dir
branch
from
9ee6693 to
20ba221
Compare
|
Tests should be ready |
gabrielmougard
force-pushed
the
fix/rm-disk-dev-deletes-empty--dir
branch
from
20ba221 to
06cfd51
Compare
|
Also, @tomponline, the Or, we can introduce a new device option to say "not to track that kind of changes" so that I can set |
gabrielmougard
force-pushed
the
fix/rm-disk-dev-deletes-empty--dir
branch
2 times, most recently
from
9504543 to
db861a7
Compare
gabrielmougard
force-pushed
the
fix/rm-disk-dev-deletes-empty--dir
branch
from
db861a7 to
eedfede
Compare
gabrielmougard
force-pushed
the
fix/rm-disk-dev-deletes-empty--dir
branch
from
eedfede to
8838e7c
Compare
|
@tomponline I implemented the logic for the VM as well within the lxd-agent. I tested it on my side without an issue. |
|
Thanks @gabrielmougard please can you add a PR description with what has changed so the reviewer can compare the code to the intended design. Please do include any new API endpoints you've added to the lxd-agent. Cheers |
|
Please can you rebase @gabrielmougard |
gabrielmougard
force-pushed
the
fix/rm-disk-dev-deletes-empty--dir
branch
2 times, most recently
from
9140527 to
844ac0f
Compare
There was a problem hiding this comment.
@gabrielmougard when the disk device is hot-plugged its mount inside the guest is triggered by an event in eventsProcess(). Is there a reason you didn't feel that we could unmount the disk in the same fashion and required the API endpoint instead?
gabrielmougard
force-pushed
the
fix/rm-disk-dev-deletes-empty--dir
branch
from
844ac0f to
f4c1453
Compare
gabrielmougard
force-pushed
the
fix/rm-disk-dev-deletes-empty--dir
branch
2 times, most recently
from
8820872 to
78fec08
Compare
|
@gabrielmougard in your tests were you stopping with |
I don't think so... Which tests are you talking about? |
the stop tests, so the time taken doesnt take into account the guest OS shutdown part. |
aaah, the benchmark.. No, I'm not using |
are you able to try that so we dont get the guest OS shutdown time in the results? |
|
absolutely. I will update the benchmark taking that into account |
|
@tomponline It should be updated now |
Signed-off-by: Gabriel Mougard <[email protected]>
Some devices like disk devices with a target path need to be cleanly unmounted. In the case of a VM instance, we prefer to handle the unmount logic within the agent and handle the unmounting of the potential overmounts of the guest. In some cases, we also need to remove a path resource (or not) on the guest side. That is why we also pass some extra metadata contained in the `Volatile` field, to handle the resource removal process in the agent and not on the LXD side (this would result in spawning aan SFTP client, which is a wasteful approach) Signed-off-by: Gabriel Mougard <[email protected]>
…k_dev_name>.last_state.created` for the disk device Signed-off-by: Gabriel Mougard <[email protected]>
… a device Signed-off-by: Gabriel Mougard <[email protected]>
When creating a directory within an instance filesystem, we should be able to track
the directory structure that has been created in order to remove the chain of directories
when removing a disk device associated to an instance.
This function takes an sftp client that, given an absolute path (this path is the desired directory path that a user whishes to create in the instance),
it creates a path encoding like the following:
<existing-path-part>/./<non-existing-path-part>
~~~~
SEP_MARK
Using this encoding, when unmounting a device associated to a created directory within an instance, we can remove the chain of created directories, now that we know which have been created and which were there before.
In the end, we remove at the path `<existing-path-part>/<non-existing-path-part>[0]` (for the VM case)
For the container case, we remove directories using the sftp client and since this one doesn't have a recursive remove API for directories,
we need to directories from the deepest level all the way up:
1) rm `<existing-path-part>/<non-existing-path-part>[:L]`
2) rm `<existing-path-part>/<non-existing-path-part>[:L-1]`
3) rm `<existing-path-part>/<non-existing-path-part>[:L-2]`
...
Signed-off-by: Gabriel Mougard <[email protected]>
gabrielmougard
force-pushed
the
fix/rm-disk-dev-deletes-empty--dir
branch
from
78fec08 to
5910823
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
gabrielmougard
force-pushed
the
fix/rm-disk-dev-deletes-empty--dir
branch
2 times, most recently
from
b14b3b1 to
33239e8
Compare
…ory when unmounting When a disk device is removed while relying on a host directory and mapped to a target within a container, we detect if the target directory has been created by LXD or not in order to not delete the content of a target directory during the unmount. Example: - Let say we mounted a custom host empty directory (`test`) on the existing `/opt` directory of the container, when unmounted (`lxc device remove ...`), the target `/opt` won't be removed, because we marked it as NOT being created by LXD at mount time. - If, however, we created a custom empty target directory at mount time: `lxc config device add u1 test disk source=/home/user/test path=/new_dir`, the directory `new_dir` will be created on the target instance and if we decide to unmount `test`, the target `/new_dir` will be removed because is has been created by LXD Signed-off-by: Gabriel Mougard <[email protected]>
Signed-off-by: Gabriel Mougard <[email protected]>
Signed-off-by: Gabriel Mougard <[email protected]>
Signed-off-by: Gabriel Mougard <[email protected]>
Signed-off-by: Gabriel Mougard <[email protected]>
… been unmounted Signed-off-by: Gabriel Mougard <[email protected]>
…n created by lxd Signed-off-by: Gabriel Mougard <[email protected]>
Signed-off-by: Gabriel Mougard <[email protected]>
Signed-off-by: Gabriel Mougard <[email protected]>
gabrielmougard
force-pushed
the
fix/rm-disk-dev-deletes-empty--dir
branch
from
33239e8 to
d290c7f
Compare
closes #12648
closes #12716
Overall description
When a disk device is removed while relying on a host directory and mapped to a target within a container or a VM,
we detect if the target directory has been created by LXD or not in order to not delete the content of a target
directory during the unmount. Additionaly, with the VM case, we cleanly unmount the target path inside the VM.
Container case
Example:
Let say we mounted a custom host empty directory (
test) on the existing/optdirectory of the container,when unmounted (
lxc device remove ...), the target/optwon't be removed, because we marked it as NOT being createdby LXD at mount time.
If, however, we created a custom empty target directory at mount time:
lxc config device add u1 test disk source=/home/user/test path=/new_dir,the directory
new_dirwill be created on the target instance and if we decide to unmounttest,the target
/new_dirwill be removed because is has been created by LXDParticular case for VMs
In addition to that, this fix also cleanly unmount the target directory inside the VM through an new LXD-agent API call. Before that, here is what would have happened:
mkdir /tmp/empty-dir lxc launch ubuntu:jammy v1 --vm lxc config device add v1 empty-dir disk source=/tmp/empty-dir path=/opt lxc config device remove v1 empty-dir lxc shell v1 -- stat /opt stat: cannot statx '/opt': Transport endpoint is not connectedThis happens because the mounted device and its associated char device were removed using QEMU's QMP without unmounting the target.
Now, we inspect for the mounts and the over-mounts if any on the VM, and unmount them in the right order.
Benchmark
Each time, there are 10 runs on each benchmark case (10 starts, 10 stops)
masterbranch (CONTAINER):Startup time with: Stop time (
--force) with:598.2 ms ± 14.3 ms* 0 disk:883.5 ms ± 37.1 ms595.0 ms ± 8.2 ms* 5 disk:892.6 ms ± 23.3 ms601.0 ms ± 8.5 ms* 10 disks:900.1 ms ± 22.0 ms606.9 ms ± 12.7 ms* 20 disks:886.8 ms ± 23.4 ms634.3 ms ± 10.1 ms* 100 disks:934.1 ms ± 16.4 msOur fix branch (CONTAINER):
Startup time with: Stop time (
--force) with:632.8 ms ± 21.3 ms* 0 disk:870.3 ms ± 25.6 ms624.4 ms ± 15.4 ms* 5 disk:879.9 ms ± 23.6 ms626.6 ms ± 17.2 ms* 10 disks:877.1 ms ± 19.7 ms641.0 ms ± 23.3 ms* 20 disks:932.0 ms ± 148.6 ms660.0 ms ± 18.3 ms* 100 disks:942.7 ms ± 16.9 ms