Prototype

[richfitz]

Basic implementation. I'll probably go back and carve out a slightly smaller surface area here for the first real PR...

Before merging:

• update actions to drop :prototype tag
• update config.py to use main as default tag

[M-Kusumgar]

Looking good!

[M-Kusumgar]

I think run_docker_command sounds too much like it runs an actual command into an existing docker container, bit like exec_run

Suggested change

	run_docker_command("Backup", image, command=command, mounts=mounts)
	run_container_with_command("Backup", image, command=command, mounts=mounts)

[richfitz]

good shout

[M-Kusumgar]

that needs the f right?

Suggested change

	print("Container '{machine.container}' for '{name}' does not exist")
	print(f"Container '{machine.container}' for '{name}' does not exist")

[richfitz]

yeah, that's right. I've fixed that and added a test as this behaviour was never tested properly

[r-ash]

This looks great, couple of questions comments

• Is it worth namespacing the keys in the vault with the config name too? e.g. for the example with local.json with server alice and client bob. Write these to secrets/local/alice and secrets/local/bob. I guess otherwise if 2 configs are using the same local names they could overwrite each other? Or you already working around that somehow
• If we have a service running which is using the volume we are backing up or restoring do we have to take down that service? Or is rsync smart enough that that can work if some other process is potentially modify the files?

[r-ash]

Is this going to replace privateer eventually?

[richfitz]

yeah, I'll do some sort of merge into that package source

[richfitz]

Is it worth namespacing the keys in the vault with the config name too? e.g. for the example with local.json with server alice and client bob. Write these to secrets/local/alice and secrets/local/bob. I guess otherwise if 2 configs are using the same local names they could overwrite each other? Or you already working around that somehow

In the configuration, the vault.prefix controls this. There's no control for collisions though, that's on us. I imagine we'd end up with /secret/vimc/privateer, /secret/hiv/privateer, /secret/wodin/epimodels/privateer, etc.

If we have a service running which is using the volume we are backing up or restoring do we have to take down that service? Or is rsync smart enough that that can work if some other process is potentially modify the files?

This depends on the service unfortunately. Most things will cope acceptably well, but there's always a chance that a partly written file will be copied over and we can't stop that. If the system needs taking down before backup, then that system won't be able to use scheduled backups, and we'd need to think about how to do backup in a consistent way. We'd have similar issues with the privateer1 approach of using tar as that really dislikes having files change as it writes them. For things like orderly it will be fine, because of the order in which files are written - an incomplete write won't result in corruption of anything and the next backup will bring it all up to date