mergerfs to "extend and mask"? (A+B = A++)

[incansvl]

I'm setting up a Samba file server.

Each valid user will have a login and the usual "local user stuff" in dotfiles in /home/<user>. Call this tree "A"

What I would like to do is then merge in from another "bulk file space" file tree "B" (which will either be a local btrfs mirror or a remote mounted share) each users application files. These will be in folders including /home/<user>/Documents, /home/<user>/Pictures etc. (which you'll recognise as the common Windows home dir layout that some unix desktops seem to mirror)

AND THEN-

1. Present the merged result to the user back at the /home/<user> path (which is what I meant by A+B=A++) as the second FS is extending the first
2. Have writes to dotfiles applied to the local /home filesystem, while writes to application files go to the bulk files FS

Questions

1. The mergerfs examples all show source trees being merged to a new target path (A+B=C). I'm guessing that trying to merge a path to itself (A+B=A) might either be refused, or cause an infinite loop (true?). Can this be dealt with by appropriate juggling of mount points, something like-

• mount both the local /home and "bulk files" FSs to intermediate mount points (e.g. /A and /B). If /home is not a separate FS this would require a loopback mount?
• add an fstab entry to merge /A and /B with appropriate options and mount the result on /home

2. I think requirement (2) above should be achievable using a path-preserving merge rule as long as path globbing is flexible enough, so I can merge /home as A with /home/*/[^.]* from B (to include everything except dotfiles, or are dotfiles excluded by default from a glob?)

3. Ideally there would be the facility to set include and exclude rules based on regex matching of pathnames to give fine grain control over which files have priority for reading, and which FS writes go to, but I don't think that's available currently. A valid enhancement request or too specialised?

If there's a better/easier way of achieving the desired result i'd like to know, obviously. Otherwise is this something mergerfs can be adapted to do?

[trapexit]

I really am not exactly sure what you're asking.

Sorry, but adding a regex to the path selection is just too niche. You only have 2 paths. You can write a small script to generate the values or use symlinks as config in some path.

No, you can't mount on top of something itself. It causes a loop given the way mergerfs is designed so as to deal with certain situations.

[trapexit]

Sorry, but I'm still not quite understanding the goal. You mention /home going to the root partition. Why does that matter? You can always just change the mounts and put stuff wherever you want. I'm also not following the "I can still boot" thing. /home should have no impact on boot process.

mergerfs does what is listed on the tin. No more or less. If you want to make /home a mergerfs mount just create your branches and mount /home. Just don't create a loop. I just don't see the advantage of such a setup vs just mounting /home from some other storage device. Is root so big as you don't want to waste the space?

[incansvl]

Losing part or all of /home doesn't impact boot directly, but in my current setup it interferes with the immediately preceding step of running nixos-rebuild (from ~/.config/nixos) to alter things like disk mount points and options. In nixos files like /etc/fstab are generated from the config file rather than being directly editable, so the nixos-rebuild step is a necessary part of any persistent config change preceding a reboot. No doubt other setups for the nixos config files vs. the one I currently have are possible but as i'm very new to nixos i'm working from online examples of "how to version-manage a nixos configuration in git" and trying to stay reasonably close to them where I can.

The data that will be migrated from the existing server is much larger than the small SSD I have as a boot disk, so it is necessary for all o or nearly all of /home to ultimately be housed on the "bulk data" btrfs mirror.

Maybe I could have fed a non-default disk+partition layout into the nixos installer and set this up at install time, but i've had trouble with that before in other linux installers, and (rightly or wrongly) forsaw a problem with telling nixos to use a btrfs mirror for /home before i'd even got the system installed and working so I could edit the configuration file to add btrfs support... Hence why i'm looking at the options for moving and/or extending the home filesystem after the initial install.

If I switch the whole of /home to the mirror, I will need to do the manual cleanup and merge operation I described above before swapping mount points, otherwise I risk losing access to the important stuff in ~/.config and also risk other clashes with dotfiles (SSH and Gnome keyrings for example) carried over from the historical Debian system.

I looked at a couple of options for doing this merge of filesystems. Most, like UnionFS and AuFS are based on having the "bottom" FS be read-only. Mergerfs seemed to be the best fit in terms of being able to merge two (or more) existing filesystems where both are writable, with the read-from and write-to decisions based on pathname rules.

I realise this may not be a common use case, I was tryng to understand whether it could work in principle, and whether there were certain elements (like the "merge /home back onto /home" idea) that were never going to work, and might kill the setup or obliterate my data if I tried them.

[trapexit]

But what does mergerfs have to do with nixos-rebuild or keeping /home on a different storage device? Or when you set it up? I'm just not understanding what a union filesystem gives you here. If you can do the orchestration to set up one filesystem I'm not understanding why that can't be done for other setups. I have a hard time believing that this is some unique problem. Having /home or any other part of the filesystem on different devices is extremely common.

[incansvl]

I feel like we've covered most of this above. If I go ahead with the original idea (minus the "merge a tree to itself" option which I now know isn't viable) then the mergerfs will be providing the content of /home, and it will be the result of merging trees from 2 filesystems/devices. In my current config, /home/<myusername> is central to the rebuild-and-reboot cycle.

As it turns out, that interdependency is proving to be a pain. It came out of following online recipes on configuring nixos with git, but I think it probably needs to change.

I may also end up needing to change my partition layout and mount points, although that would somewhat undermine the original attraction to mergerfs. I will continue to experiment with mergerfs for a bit and see if I can adapt it to fit my use case.

The one oustanding question from the original list at this point is how path globbing works? I didnt't find the detail in the docs. It would be helpful to know whether mergerfs uses "classic" shell style globbing, or the extended POSIX version. I tried looking at the source code but my coding skills are not up to the task, it looks like the glob function is being pulled from a standard library (maybe?), if so it comes down to what that library supports.

[trapexit]

It uses standard, default gnu libc POSIX glob:

mergerfs/src/fs_glob.cpp

Line 37 in dd987db

::glob(pattern_.c_str(),flags,NULL,&gbuf);

No GNU extensions.

https://www.man7.org/linux/man-pages/man3/glob.3.html