[ops] Introduce GitpodWsManagerMk2BackupFailureError and GitpodWsManagerMk2BackupFailureCritical

[kylos101]

Description

Notify in Slack and PagerDuty when backup failures occur. I'll create a separate PR for Enterprise.

Related Issue(s)

Related to ENT-839
Relates to https://github.com/gitpod-io/runbooks/pull/433

How to test

Slack notifications will trigger in the future:

If it gets "bad", so will PagerDuty.

Documentation

Slack notifications will trigger in the future:

Preview status

gitpod:summary

Build Options

Build

• /werft with-werft
  Run the build with werft instead of GHA
• leeway-no-cache
• /werft no-test
  Run Leeway with --dont-test

Publish

• /werft publish-to-npm
• /werft publish-to-jb-marketplace

Installer

• analytics=segment
• with-dedicated-emulation
• workspace-feature-flags
  Add desired feature flags to the end of the line above, space separated

Preview Environment / Integration Tests

• /werft with-local-preview
  If enabled this will build install/preview
• /werft with-preview
• /werft with-large-vm
• /werft with-gce-vm
  If enabled this will create the environment on GCE infra
• /werft preemptible
  Saves cost. Untick this only if you're really sure you need a non-preemtible machine.
• with-integration-tests=all
  Valid options are all, workspace, webapp, ide, jetbrains, vscode, ssh. If enabled, with-preview and with-large-vm will be enabled.
• with-monitoring

/hold

[geropl]

/unhold