Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Allow mergerfs.balance to balance against a specified amount of free space per disk #104

Open
undaunt opened this issue Jun 26, 2020 · 1 comment

Comments

@undaunt
Copy link

undaunt commented Jun 26, 2020

2 - Allow mergerfs.balance to balance disks against a target of total free space or total used space, similar to mfs/lfs policies.

Use case:

disk1 8TB, 50% full (4TB)
disk2 12TB, 50% full (6TB)
disk3 12TB, 75% full (9TB)
disk4 12TB, 0% full (0TB)

Currently, running balance would get each disk to be 2% apart in usage, winding up in this situation:

disk1 8TB, 43% full (3.45TB)
disk2 12TB, 43% full (5.18TB)
disk3 12TB, 43% full (5.18TB)
disk4 12TB, 43% full (5.18TB)

If possible, I'd like to be able to balance against a value of free space similar to empfs's behavior. IE: mergerfs.balance knows there are approximately 25TB free across all disks, so it will balance until there is approximately free space / number of disks, or in this case 25/4 = 6.25TB free on each disk, leaving us here:

disk1 8TB, 21.8% full (1.75TB)
disk2 12TB, 48% full (5.75TB)
disk3 12TB, 48% full (5.75TB)
disk4 12TB, 48% full (5.75TB)

This would then allow net new data (in a mfs policy scenario) to write alternating across all four drives. Without this, data would go to disk2, disk3, and disk4 until they each had 4.55 TB of free space, similar to disk1 in the current state.

@celogeek
Copy link

celogeek commented Aug 3, 2020

Hi,

I replace the method "freespace_percentage" by:

def freespace_percentage(srcmounts):
    lfsp = []
    total_fs = 0
    for srcmount in srcmounts:
        vfs = os.statvfs(srcmount)
        avail = vfs.f_bavail * vfs.f_frsize
        total_fs += avail
        lfsp.append((srcmount,avail))

    lfsp = [(x[0], x[1] / total_fs) for x in lfsp]
    return sorted(lfsp, key=lambda x: x[1])

This sum the available freespace, then create a percentage by disk against it.

The effect is that it will balance using a "mfs" rules.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants