Backup of S3 objects

[karthik-murugan-hpe]

Let's assume we have a S3 like component running in our cluster and we want to backup its data using k8up.

We could write a Pre-Backup Pod which syncs the S3 objects to local file system (within the Pre-Backup Pod's container). I'd like to then use K8up to backup these files as a snapshot. I understand, we could just use a "backupcommand" and stdout all the file contents, but for better performance, I want K8up to process these files just like how it processes a folder (PVC). i.e file by file. Any suggestions?

[Kidswiss]

Hi @karthik-murugan-hpe

I think it should be doable via a pre-backup pod or an annotated pod that mounts the PVC you want to sync to. The PVC backups are always triggered after the backupcommands have finished.

So you could:

• annotate an existing pod with the command, or define a pre-backup pod
• the command then syncs the data to the PVC attached to the pod
• the command could simply print some status information or statistics to stdout, so they get saved in a snapshot as well
• k8up will then process the PVCs as normally and create a snapshot with the contents

However I'm not quite sure if we currently support mounting PVCs in a pre-backup pod. I'd need to check that.
But with an annotated pod that provides the tools for the sync, I don't see why this shouldn't work.

[karthik-murugan-hpe]

Thanks @Kidswiss. I could verify the Backup of the S3 service using a Pre-Backup pod and an annotated PVC to hold the files for k8up to process.

For Restore, I could create a Restore Resource which would copy the snapshot dump back to the PVC.
I'd like to then run a Job which syncs the PVC with my S3 service. But I'd like this Job to wait till the K8up has finished populating the PVC from the snapshot. Any suggestions for this?