Support creating Management Clusters using a proxy in mc-bootstrap

[cornelius-keller]

mc-bootstrap needs to be adapted to ensure clusters can be configured with proxy variables if required.

• install cluster- chart with proxy configuration
• bootstrap the app plattform with proxy configuration -> https://github.com/giantswarm/chart-operator/blob/master/helm/chart-operator/values.yaml#L54 and https://github.com/giantswarm/chart-operator/blob/a0c60e37dc1bbb468ce27fc8f4a91d21eff9aa83/helm/chart-operator/templates/deployment.yaml#L102
• What do we do with flux?

[bavarianbidi]

Kyverno connectivity policies PRs:

• remove orphaned values as they aren't in use kyverno-policies-connectivity#18
• inject proxy vars on specific namespaces kyverno-policies-connectivity#17

[bavarianbidi]

Implementation is blocked:

As kyverno is currently deployed in namespace giantswarm, it's not possible to create a mutating config which affects pods in namespace giantswarm.

The future goal in general would be moving kyverno out of namespace/giantswarm, but this isn't implemented yet:

• https://github.com/giantswarm/giantswarm/issues/23971
• slack thread #1
• slack thread #2

[bavarianbidi]

Open PRs

• add .svc suffix to alertmanager target prometheus-meta-operator#1010
• add .svc suffix to alertmanager address silence-operator#231
• https://github.com/giantswarm/management-clusters-fleet/pull/164

Other findings/open tasks

• fix organization-operator to point http://companyd:8000 to a proxy-ignored address (or improve organization-operator)
• cert-manager is in namespace/kube-system - current status in this slack discussion
• kyverno addoption from namespace/kyverno --> namespace/giantswarm --> namespace/kyverno again 🤷
• inject proxy-vars into containerd
• kyverno-policy-connectivity managed via app-platform - only if this installation need a proxy - asked in AMA

[bavarianbidi]

2022-10-17:

• CAPVCD: no_proxy must be configured - otherwise webhooks will fail (uptream tracking issue)
• related kubernetes/kubeadm upstream PRs/Issues:
  • issue #2771 to discuss about the proxy discovery
  • issue #2770 to track proper json patches
  • and PR #113092 to implement a fix for MC default CM in each App CR #2770

[bavarianbidi]

2022-10-18:

• CAPVCD isn't able to create instances if httpsProxy and noProxy is set - could be also a bug which is already solved. Waiting for a new CAPVCD version before continue with debugging
• continue on extending cluster-operator
• Prepare a summary about the current state of implementation.
• Update fixed silence-operator and prometheus-meta-operator to latest version

[bavarianbidi]

with the recent changes in mc-bootstrap, cluster-cloud-director and default-apps-cloud-director it's possible to create a MC behind a corporate proxy.

All follow up topics/open questions are documented in #1424 and in the RFC giantswarm/rfc#50