Continous restore / Restartable restore

[sylr]

Hi,

First thank you for litestream! I use it to backup sqlite3 databases for read only debug purposes.

I don't know if it is even possible but it would be really nice if we could do multiple restore of the same database without having to remove the local file and download the last snapshot + available WAL history.

Another idea, something like a -continuous flag for the restore command that would trigger a mechanism which would watch for new WAL on the storage and apply them as they appear.

Regards.

[slaveofcode]

Any idea on this?

[yanc0]

The continuous mode would be hard to do.

Watching from an object storage in not trivial. You could use event notification from Cloud providers (S3, GCS, Azure) but it's very specific to each one. Some Cloud won't ever provide notifications system at all.

Another option is to regularly list objects from S3 and keep track of the state of the current and remote database. Sadly, LIST operations are not cheap, it will incur cost even if there is no write on the primary.

At the end, Litestream should force readonly in "continuous mode", I don't know if this is possible to force other connections to be read only.

Maybe you just need a litestream restore -force to skip the error message cannot restore, output path already exists and run this command inside a cronjob ?