Migrate apps from default-apps-vsphere

[Rotfuks]

Motivation

In order to match the target state of our defined release framework we need to remove the use of the default-apps charts from all providers. Only then can we realise the single entrypoint to apply changes to a cluster for customers and reduce complexity in how and where our apps are managed.

Todo

• Migrate apps from default-apps-vsphere to cluster chart
  • capi-node-labeler v0.5.0
  • cert-exporter v2.9.1
  • cert-manager v3.8.1
  • chart-operator-extensions v1.1.2
  • cilium-servicemonitors v0.1.2
  • etcd-k8s-res-count-exporter v1.10.0
  • k8s-dns-node-cache v2.8.1
  • metrics-server v2.4.2
  • net-exporter v1.21.0
  • node-exporter v1.19.0
  • observability-bundle v1.5.2
  • observability-policies v0.0.1
  • security-bundle v1.8.0
  • vertical-pod-autoscaler v5.2.4
  • vertical-pod-autoscaler-crd v3.1.0
  • teleport-kube-agent v0.9.2
• Enable default apps in cluster-vsphere
  • capi-node-labeler
  • cert-exporter
  • cert-manager
  • chart-operator-extensions
  • cilium-servicemonitors
  • etcd-k8s-res-count-exporter
  • k8s-dns-node-cache
  • metrics-server
  • net-exporter
  • node-exporter
  • observability-bundle
  • observability-policies
  • security-bundle
  • vertical-pod-autoscaler
  • vertical-pod-autoscaler-crd
  • teleport-kube-agent
• Check if remaining default apps from cluster chart should be also enabled in cluster-vsphere (those apps that are not currently in default-apps-vshere)
  • cluster-autoscaler
  • external-dns
  • k8s-audit-metrics
  • prometheus-blackbox-exporter
• Migrate apps from default-apps-vsphere to cluster-vsphere chart (there are no vSphere-specific apps in default-apps-vsphere)
• Move vSPhere-specific default apps default config from default-apps-vsphere to cluster-vsphere (or ensure that the needed default config is already present in cluster chart)
  • cert-exporter (added vSphere-specific config to cluster-vsphere)
  • cert-manager (added vSphere-specific config to cluster-vsphere)
  • etcd-k8s-res-count-exporter (uses default config from cluster chart)
  • metrics-server (uses default config from cluster chart)
  • net-exporter (uses default config from cluster chart)
  • node-exporter (added vSphere-specific config to cluster-vsphere)
  • vpa (CNP config from default-apps-vsphere does not exist anymore in the VPA app)
• Remove provider-independent HelmReleases from cluster-vsphere and enable them from the cluster chart. Also add vSphere-specific app config in cluster-vsphere.
  • cilium
  • coredns
  • network-policies
• Adapt mc-bootstrap to the removal of default apps
• Adapt happa to the removal of default apps
• Adapt e2e tests to the removal of default apps
  • Add support for unified cluster-vsphere app clustertest#283
• Add support for unified cluster-vsphere app kubectl-gs#1410
• Update docs on how to apply CAPV clusters
• Clearify with Team Rocket how to roll out the change on all CAPV Installations
• Support migration of apps from default-apps-vsphere to cluster-vsphere
• Remove the default-apps-vsphere chart

Outcome

• The complexity of where and how apps are managed is reduced since we now only have cluster-vsphere and cluster chart
• There is only one chart to apply for creating and updating clusters reducing complexity