Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Even allocation of files among writable branches #89

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

csabahenk
Copy link

Unionfs can be used as poor man's clustered storage. To make this use case feasible, it's desired that file creations are distributed evenly among the writable branches (using a heuristics that a given new file is randomly placed on branches with likelihoods proportional to available space).

To ease debugging while I was working on this feature, I added timestamps to the debug messages. I include that in the PR as a nicety, but it's not essential for the feature delivered.

@lavdnone
Copy link

good feature, seems stable, using it on couple occasions
what I encountered: when in place rename it tries to move file between branches
it should also check and do the move on the same branch

@rpodgorny
Copy link
Owner

thanks for the (independent) feedback! ...i'll try to spare some to review the patch.

first impressions from a quick look:

  1. why the heuristic? why not just strictly decide by the free space? i tend to prefer determinism...
  2. if bsd/macos is still in a "todo" phase, i'd rather have this option ifdef-ed for linux only
  3. malloc+free - can we do with static pre-allocation?

@lavdnone
Copy link

Quick fix for the move problem was

+448 src/fuse_ops.c
        } else {
                j = i;
        }

@lavdnone
Copy link

PS I would not call it "poor man's clustered storage".
It is more of a Single mount JBOD.
With the idea that failure of one drive will not require fixing whole raid0 (for example).
And if you have 100 processes writing something on those drives it will still distribute IO and get you speed off all the drives.
If the drive is gone - it is just files on that drive. With good SMART check reporting one can easy copy real simple files from failing drive to another.

@ajax-lives
Copy link

Is this available with fstab at this time? I'm interested in making use of this :D

@ajax-lives
Copy link

And will this be merged eventually? Looks extremely neat!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants