Monitoring Backups to S3

[murphyj21]

Problem Description:

If AWS S3 experiences issues when writing backups, cronjobs pile up and crash the instance. To remedy this a timeout was added to the cronjob. However, if the cronjob times out backups are not written to S3.

Fix recommendation:

Create a lambda function to poll S3 and generate an alert if backups are not written at the required interval.

[ferricoxide]

This stopped being a problem for a few months, but has recently returned. Probably requires wholly rearchitecting the backups ...and possibly the storage used for ${JENKINS_HOME} (to allow for filesystem snapshotting to create a "static" backup-set to copy from).

[ferricoxide]

Added an enhancement issue (#47) to describe a better backup system. While a monitoring lambda function would be useful, in general, the problem likely is best solved by a less-simplistic backup method.