Manage with Terraform

[rjackson]

Managing our deployment so far with raw Kubernetes CLI and a bunch of manifests has been difficult. Careful consideration has to be placed when deploying new resources, making sure they're deployed in the correct order, making sure any previous instances of jobs are cleared up first, etc. All in all it's a real pain.

With this PR I'll begin converting our Kubernetes manifests into Terraform definitions, and using Terraform to keep track of all of this. This will give us a much nicer interface for managing the deployment, as well as additional protections against resources changing under the hood.

On top of just reducing maintenance overheads, I am also hoping this will make dealing with underlying Kubernetes upgrades easier as well.

Resources to import and properly track

• Cluster itself
  • GKE cluster
  • GKE node pool
• Supporting infrastructure
  • Blackfire
  • Ingress
  • Cert manager ??
  • Filestore
  • CloudSQL database
  • (any other external resources?)
• Kubernetes deployments
  • Cloudsql-proxy daemonset
  • mcrouter daemonset
  • Mediawiki deployment
  • Mediawiki-update deployment
  • Memcached stateful set
  • Run-jobs deployment
  • Update special pages cron job
  • Varnish deployment
• Kubernetes services
  • all-varnish
  • cloudsql-proxy
  • mcrouter
  • mediawiki
  • memcached
  • nfs-server
  • nfs-varnish

Additional tidy ups:

• Extract appropriate variables from kubernetes configs
• Replace hardcoded resource references with usage of resource attributes
• Remote state file