Q4 Disaster Recovery Exercise

[kiwixbot]

In order to strengthen our infrastructure, we want to:

• Document our Disaster Recovery Plan
• Simulate a Disaster and recover the service to ensure documentation exists and is up to date

This ticket requests the assignees to:

• Pick a Service from Kiwix/openZIM/Offspot
• Redeploy it completly using its backup, using the documentation or writing it.

See Zimfarm documentation.

Note: this is an automatic reminder intended for the assignee(s).

[rgaudin]

@kelson42 @benoit74 which service should we do?

[kelson42]

@rgaudin The website? The wiki? The object-cache service?

[rgaudin]

Might be a good opportunity to document the website setup since it's not on our infra and AFAIK we don't backup it (maybe there's a backup feature in cyon)

In the other hand, it's entirely different than the rest of our service and maintained by martin. In a crash scenario he'd probably be the one restoring it

[rgaudin]

Can we take 15mn early this week to discuss it all together ?

S3 is not backed up (the policies and all) and that's somewhat important we do

[kelson42]

@rgaudin Agree. Usually such outsourced services are the less resilient exactly because its outsourced and we "trust" the provider. Here is the danger.

[rgaudin]

Opened #128 and #129. We'll do the Kiwix Wiki this week.

[benoit74]

Done.
Procedure is now documented (https://github.com/kiwix/k8s/wiki/Restore-Wiki-from-a-backup) and tested on both wikis.