udisksd isn't smart enough to handle Btrfs multiple devices for automounting, can cause data loss

[cmurf]

This is the original bug I filed against udisks2, but I think it's a dead end now (?) so I'm opening up this issue for storaged, as I can reproduce all of the same problems with storaged's udisksd on Fedora 25, Nautilus, and Btrfs multiple device volumes.

This comment is the behavior and analysis (in-line) as well as the options for how to handle this better.
https://bugs.freedesktop.org/show_bug.cgi?id=87277#c3

The gist is that Btrfs arrays (multiple device volumes) do not have a single device node and this is causing confusion what to mount and how to umount. LVM arrays appear as a single /dev/mapper/LV node, and mdadm arrays appear as a single /dev/md/0 type of node, whereas Btrfs lets you mount any of its member nodes and it'll automatically go find all of the other members. The problem is that udisksd sees each device as individually mountable, and does so, and then problems ensue including file system corruption when the user goes to eject a device, and udisksd obliterates the device node (using sysfs block device delete I guess) causing that whole node to go missing, before it's completely umounted the whole file system.

In the near term it's probably best to just blacklist Btrfs volumes from udiskd automounting (option C), it's that dangerous right now.

Option A: Use sysfs once a device is mounted to discover its other member devices, and then black list them from additional mounting or visibility by upper layers.
Option B: Use Btrfs ioctl to discover member devices before mounting.
Option D: Did not mention in the cited bug, just thought of this, libblkid could be used to discover member devices for a given fs volume UUID, and then have udisks only mount one of them.

[cmurf]

Another possibility is to use mount --uuid instead of device node, anytime the UUID fs type is Btrfs, and filter for additional instances (i.e. only mount a UUID one time no matter how many times it appears).

A known (upstream) problem exists for cloned block devices, where uuid only mounting could lead to corruption if the devices= mount option is not also used; a solution for this is being explored but there's no work done yet on it.
https://btrfs.wiki.kernel.org/index.php/Gotchas

[tsmetana]

Thanks for the report. I will need to test this myself somewhere to see how difficult would it be to add workarounds for btrfs.

[cmurf]

During boot, a udev Btrfs rule uses BTRFS_IOC_DEVICES_READY to check if all devices are present and prevents systemd from trying to mount a Btrfs volume if all of its devices aren't present. But this same udev rule doesn't affect udisksd in the same way. Instead udisksd tries to mount the device as soon as it appears even if it needs other devices to mount properly. The failed mount doesn't hurt anything, it just spits out errors in kernel messages. Presumably the reason why the mount succeeds is because udisksd tries to mount the 2nd device when it appears, now the volume's devices are all ready.

So if you were to go with 'mount --uuid' the new problem is maybe all devices aren't yet ready when that command is issued; and then mount is only issued once, no retries. So that'd mean you'd need to use BTRFS_IOC_DEVICES_READY or maybe rely on udev (?) before issuing the mount command.

[vpodzime]

I'm gonna look at this. A function telling if a btrfs device is complete could probably be useful in libblockdev-btrfs and then easily used in storaged.

[vpodzime]

@cmurf, please take a look at storaged-project/libblockdev#244

[vpodzime]

I'm afraid we are not able to resolve this issue now. See storaged-project/libblockdev#244 for more details.

[lkjell]

I am unable to hide a multi device btrfs unencrypted dm-crypt with the given rule:
ENV{ID_FS_UUID}=="#BTRFS-UUID", ENV{UDISKS_IGNORE}="1"

Is this a related issue?

[vpodzime]

@lkjell do you have such rules for all the devices in that volume?

[lkjell]

@vpodzime what do you mean? Multi device Btrfs share the same UUID.

[vpodzime]

@vpodzime what do you mean? Multi device Btrfs share the same UUID.

Right, sorry for a stupid question. :)

Can you check if the UDISKS_IGNORE property is really set on the devices? You can use something like udevadm info --path=/sys/class/block/dm-0 to get the properties/variables of a given device.

[lkjell]

Here is the info for the two devices.

P: /devices/virtual/block/dm-0
N: dm-0
L: 0
S: mapper/crypt_home2
S: disk/by-id/dm-uuid-CRYPT-LUKS2-d261c53dc24b4cc29ed811eb4a7249b7-crypt_home2
S: disk/by-uuid/fae1b052-b84b-4180-92b1-8bd12d76d65e
S: disk/by-id/dm-name-crypt_home2
E: DEVPATH=/devices/virtual/block/dm-0
E: DEVNAME=/dev/dm-0
E: DEVTYPE=disk
E: MAJOR=254
E: MINOR=0
E: SUBSYSTEM=block
E: USEC_INITIALIZED=6606901
E: DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1
E: DM_UDEV_PRIMARY_SOURCE_FLAG=1
E: DM_UDEV_RULES_VSN=2
E: DM_NAME=crypt_home2
E: DM_UUID=CRYPT-LUKS2-d261c53dc24b4cc29ed811eb4a7249b7-crypt_home2
E: DM_SUSPENDED=0
E: ID_FS_UUID=fae1b052-b84b-4180-92b1-8bd12d76d65e
E: ID_FS_UUID_ENC=fae1b052-b84b-4180-92b1-8bd12d76d65e
E: ID_FS_UUID_SUB=fe705fd0-dc61-46e8-ad19-2de66d2ac0d9
E: ID_FS_UUID_SUB_ENC=fe705fd0-dc61-46e8-ad19-2de66d2ac0d9
E: ID_FS_TYPE=btrfs
E: ID_FS_USAGE=filesystem
E: ID_BTRFS_READY=1
E: DEVLINKS=/dev/mapper/crypt_home2 /dev/disk/by-id/dm-uuid-CRYPT-LUKS2-d261c53dc24b4cc29ed811eb4a7249b7-crypt_home2 /dev/disk/by-uuid/fae1b052-b84b-4180-92b1-8bd12d76d65e /dev/disk/by-id/dm-name-crypt_home2
E: TAGS=:systemd:

P: /devices/virtual/block/dm-1
N: dm-1
L: 0
S: mapper/crypt_home1
S: disk/by-uuid/fae1b052-b84b-4180-92b1-8bd12d76d65e
S: disk/by-id/dm-name-crypt_home1
S: disk/by-id/dm-uuid-CRYPT-LUKS2-dafe1fe85b7d432bb4974f05e81ba5ac-crypt_home1
E: DEVPATH=/devices/virtual/block/dm-1
E: DEVNAME=/dev/dm-1
E: DEVTYPE=disk
E: MAJOR=254
E: MINOR=1
E: SUBSYSTEM=block
E: USEC_INITIALIZED=6619911
E: DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1
E: DM_UDEV_PRIMARY_SOURCE_FLAG=1
E: DM_ACTIVATION=1
E: DM_NAME=crypt_home1
E: DM_UUID=CRYPT-LUKS2-dafe1fe85b7d432bb4974f05e81ba5ac-crypt_home1
E: DM_SUSPENDED=0
E: DM_UDEV_RULES_VSN=2
E: ID_FS_UUID=fae1b052-b84b-4180-92b1-8bd12d76d65e
E: ID_FS_UUID_ENC=fae1b052-b84b-4180-92b1-8bd12d76d65e
E: ID_FS_UUID_SUB=11fbc144-9626-461f-b059-39fa0c3f7ad3
E: ID_FS_UUID_SUB_ENC=11fbc144-9626-461f-b059-39fa0c3f7ad3
E: ID_FS_TYPE=btrfs
E: ID_FS_USAGE=filesystem
E: ID_BTRFS_READY=1
E: DEVLINKS=/dev/mapper/crypt_home1 /dev/disk/by-uuid/fae1b052-b84b-4180-92b1-8bd12d76d65e /dev/disk/by-id/dm-name-crypt_home1 /dev/disk/by-id/dm-uuid-CRYPT-LUKS2-dafe1fe85b7d432bb4974f05e81ba5ac-crypt_home1
E: TAGS=:systemd:

[vpodzime]

No UDISKS_IGNORE there. So the rule from #88 (comment) applied doesn't work (if you have it enabled).

[lkjell]

The rule is written under /etc/udev/rules.d/11-hide-crypt.rules
but apparently it is not applied.

[tbzatek]

This can be partially fixed by #838.

TODO: write test cases for the hotplug + automount scenario and another one for device unplug.