Retention policy

[LadyMikea]

Looking for convert my scripts to Npbackup, only three things are missing: retention policy, deleting a snapshots and prune option. As I understand, currently they are not directly available? I suppose "post-exec" command may be used for this purpose.

[deajan]

ATM, I use NPbackup with append-only rest-server so I didn't implement the gui, although the delete command already exists in CLI.

I'll have retention/prune/deletion implemented in 2.3 which should come next month.

Post-exec commands don't have access to environment variables for security reasons.

[LadyMikea]

I see, right, there's "--forget" command. Thanks for info.

[deajan]

Retention policies are being developped in v2.3.
As of now, I will allow setting --keep-X or -keep-within-X variables for hours, days, weeks, months and years.
Any special requests ?

[deajan]

Also, I'm developping a multi repository version, in order to have that privileged client that can check/forget/prune.

I'll happily take requests. Of course the gui is ugly for now as in dev :)

[LadyMikea]

Thank you! Multi repository version looks very useful. I have multiple usb backup destinations with multiple sources path: usb-X, usb-X/music, usb-X/pictures, usb-X/etc (each source directory has own sets of rules, has own npbackup.cmd script). I understand, multi repository version allows grouping together all sources under one repository (usb-X) and each source can retain own tags and sets of rules?

[deajan]

The idea of multi repository version is double:

• Have a special privileged client (the only one with delete permissions) that can handle retention strategies for multiple repositories
• Have a standard client with default settings, and repositories which inherit from default settings when no specific overrides are set

Willing to take comments and ideas at this stage ;)
Currently developping the settings inheritance.

[deajan]

Customer request: Have another client that doesn't have restore privileges.
That makes alot of different clients for each privilege level:

• backup only
• backup, restore and check repo health
• backup, restore, check repo health and retention policies

Now the question is, how to bake the privileges into the client, without being overridable just by editing the config file.
An easy way to do so would be to bake the privileges directly into the repo URI.
This way, when a malicious user modifies the config file, he will loose access to the repo.

The tricky part is the encryption algorithm. One that has access to the source code of NPBackup, can use it to decrypt the config file, thus getting the required privileges.
As of today, I resolve this by using private builds with non public symmetric AES encryption keys.
Using asymmetric encryption like RSA won't help here, since the client still needs to decrypt the repo URI & passwords.

Possible solutions:

• load encryption key from alternate file (supposes the client's integrity is safe)
• load encryption key from key server (makes it really harder to get the key, but backups won't work without internet access, which is quite lame for a backup program)
• Limit client's permissions server side (via rest-server or s3): can be sufficent for deletion / retention policy permissions, but not for restoration / health check since this only requires read permissions
• Use a magica unicorn

I'm all out of unicorn magic.
I think the best way is a hybrid solution where the server limits deletion permissions, and the client has repo URI baked-in restore/check permissions, which are less secure, but then again, if you get restore permissions to any data that you already own, unless you dedup alot of different clients into the same repo, which is specifically the case I'd like to address here.

Any better magic unicorns are welcome ;)

[LadyMikea]

Well, I am just a home user using couple of usb drivers, so all those multi user encryption grade stuff is well above my pay-grade ;) Dunno, maybe some simple "no-so-secure" path for home user would be helpful, while maintaining options for advances users like admins and/or multi users environments.

[deajan]

Well, I am just a home user using couple of usb drivers, so all those multi user encryption grade stuff is well above my pay-grade ;) Dunno, maybe some simple "no-so-secure" path for home user would be helpful, while maintaining options for advances users like admins and/or multi users environments.

Sorry, I used this issue as "idea notepad" ;)

May I ask, do you use a remote server ? If so, which kind ?
Also, do you want your backup client to be able to modify the remote repo or not ?

[LadyMikea]

No problem! No, I do not use a remote server for now. Just a few usb drivers acts as a local storage. So, every thing is "on the site". In the future maybe I'll use a proper NAS, so ability to control remote repository will be handy.

[deajan]

Thanks for your answer ;)

[deajan]

Retention policies, snapshot forgetting, pruning etc is now implemented in v3.0rc1.