Add support for client-side scheduling

[richfitz]

This PR adds a scheduler to run on the clients. I've not tried this out yet in anger, but will do so on Friday.

With this PR we can run a long-running process on production/production2 that will ship data to annex2, running daily.

Some things here:

• I've used the timezone argument to yacron to make the cron commands interpretable against the local time even when the container is running in utc (the default).
• we can expose the yacron web service out of the container, which will be nice for getting metrics out from backups, and better than doing something weird ourselves I think
• I've run the generated yaml through the yacron validator before writing it out, which makes up for the fact that I am not using a yaml emitter here but just writing it out by hand like a savage
• the scheduler has the same start/status/stop pair as the server - I took the liberty of a little light refactoring in c876866 which I snuck in just before Prototype #1 was merged
• note also I did a really boring bit of code moving around in Break configure and check into their own modules #3, so the keys.py module is a little less sprawling than before, and had to hotfix some bugs in Fix restore bugs #4 this morning as I got backup running from production and production2

Don't be afraid about the diff - this is 168 lines of code changed within src/, the rest is tests and config

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (b74e039) 100.00% compared to head (868737f) 100.00%.

Additional details and impacted files

@@            Coverage Diff             @@
##              main        #2    +/-   ##
==========================================
  Coverage   100.00%   100.00%            
==========================================
  Files           24        28     +4     
  Lines         1520      1810   +290     
  Branches       177       211    +34     
==========================================
+ Hits          1520      1810   +290     

Files	Coverage Δ
src/privateer2/backup.py	100.00% <100.00%> (ø)
src/privateer2/cli.py	100.00% <100.00%> (ø)
src/privateer2/config.py	100.00% <100.00%> (ø)
src/privateer2/configure.py	100.00% <100.00%> (ø)
src/privateer2/restore.py	100.00% <100.00%> (ø)
src/privateer2/schedule.py	100.00% <100.00%> (ø)
src/privateer2/service.py	100.00% <100.00%> (ø)
src/privateer2/util.py	100.00% <100.00%> (ø)
src/privateer2/yacron.py	100.00% <100.00%> (ø)
tests/test_backup.py	100.00% <ø> (ø)
... and 8 more

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[r-ash]

Looks good

[weshinsley]

Looks good and clear - I had one thought which might save about 5 lines of code! (But might change the order - pehaps that's why you've written it that way...)

[weshinsley]

Is this equiv to:

def unique(x):
    return list(set(x))

[richfitz]

that does not preserve ordering, otherwise yes that's the way to do it

[M-Kusumgar]

LGTM! not familiar with yachron, couldnt find the @daily and @weekly options for schedule tag but im sure you know what youre doing with that!

[richfitz]

https://github.com/josiahcarlson/parse-crontab - the cron parsing comes from this module, and is interpreted the same way as: https://crontab.guru/#@daily

I'll have a quick go firing this up in anger and see if it works...

[richfitz]

Pretty close in the end, but the jobs failed because I was mounting things inconsistently between the backup and the schedule modules - fixed that in 04fbe5a and it works