Backups complete with errors if snapshot size is bigger than remaining free space on disk (auto-remove).

[virtuallyunknown]

Greetings! Thanks for developing this great piece of software that I've been using for a while now.

I am using the GUI and I have set up automatic backups on a designated disk, which is around 80GB big with the following settings for "auto-remove":

So far my snapshots were relatively small and it all worked well, until today where the data I was backing up grew significantly. As a result of that, my backup completed with errors, and I got bombarded with notifications for every single file that failed :)

My assumption so far for these particular settings was that if the space on the designated backup disk was under 5GB, then start removing old snapshots to make space for the new one. However, now that the new backup would result in a snapshot that is 20GB big, 5GB is not enough to store the new snapshot and as a result of that the backup fails.

So I guess my question is, is there a way to configure Back In Time in a way that would calculate the new snapshot size, and just remove as much of the oldest backups as needed to make space? I thought about experimenting with the "smart remove" options, but I couldn't find any documentation available on how they work.

Cheers!

[buhtz]

Hello virtuallyunknown,

Thank you for your kind words and taking the time to report the issue and providing the details. I appreciate your feedback.

The point is that BIT runs the "(smart) remove" routine after each snapshot, not before. Before doing a snapshot, currently BIT has no way to estimate the size of the new snapshot to take. This behavior might not be that "smart" or elegant but it is intended.

In your situation I would recommend to remove some of your older snapshots by your self. Just right click on each snapshot in the time line and remove it. Let me know if this solve your problem.

As a follow up I think about three new features. Let me know what you think of it:

1. Before each snapshot estimate the size of the files and folders to backup. This can be done in the background by a second thread without interrupting the user actions. But this would be an estimate using the include and exclude folders, ignoring the include-/exclude-pattern and also ignoring the hard-links feature of rsync. It just would a very broad (and unrealistic) estimate. But if the resulting size is bigger than the free space a warning could be givin to the user that the next snapshot might cause some problems with free space.

2. A new config option giving a threshold at which BIT does warn the user about less free space on the backup destination.

3. Give the user the ability to execute the smart remove routine independent from running a snapshot. In your case this would prevent you from manually removing old snapshots.

If you have any more details to share, feel free to reach out.

Not sure when we'll find the time to work on it. Please see the projects background information to get an idea about our workflow and priorities:

• Maintenance status
• The team
• Strategy outline / Roadmap

Best regards,
Christian

[virtuallyunknown]

Hey @buhtz thanks for reaching out so fast, and thanks for the in-depth explanation.

Given the fact that estimating snapshot sizes in advance would only be a broad estimate since it can't take into account exclude patterns, and given the other options you also presented, then for my personal use case I would just stick to what I have for now, so that means just increasing the "if free space is less than X" to 20GB, and then relying on system notifications if BIT runs into issues.

I am underlying the word personal here because you asked for feedback, and this is what works best for my use case, which is by no means a bad solution/resolution. Other than that, I can potentially see other people finding value in smart remove routine (option 3), but it's probably best to reach for community feedback.

Not sure what else to add, but perhaps it would be useful if the settings documentation had a bit more detailed explanation (or any at all), whenever your time allows it.

Thanks again for reaching out!

[buhtz]

Thank you for your feedback. I appreciate it.
Yes, the docu is not worth its name. 😄 We do have some issues about it and beside of them it needs a whole rework and restructuring.

I would keep this issue open until I decided which one of the 3 options I would choose. Thanks in advance.

[noyannus]

Could this work? When a smart remove is due, remove each folder individually from the to-be-removed-backup already while writing the fresh backup -- right after that folder has been backed up. Before starting such a piecemeal remove, mark the remaining old backup in the backup list, e.g. "INCOMPLETE".
That would mitigate the OP's problem, unless they have a folder of immense size...