[EPIC] Multi-Provider MCs - CAPV on CAPA

[anvddriesch]

Based on CAPZ & CAPV epic #2840

We want to enable a truly hybrid Installation where the MC is on AWS (CAPA) and Workload clusters are AWS as well as on-prem vSphere (CAPA & CAPV)

Acceptance Criteria: Create a GS Installation on AWS that is able to create 1 CAPA WC and 1 CAPV WC on IONOS

User Story
As a GiantSwarm Customer & Platform Engineer I want to centralize as much of my applications and Administration as possible onto AWS, while still using some compute power available in the Factories with vSphere

Assumptions
Based on CAPZ hybrid documentation:

Our MC is on CAPA. It is capable of provisioning CAPV and CAPA workload clusters.
CAPA MC is able to access the vSphere API.
CAPA MC is able to access all CAPV WCs.
CAPV WCs are able to access endpoints (k8s API and ingresses) of the MC.
There is a single base domain for all clusters and it is managed on AWS via dns-operator-route53. See https://github.com/giantswarm/dns-operator-route53/blob/main/README.md

Tasks

Beta Give feedback

1. challenge assumptions above and refine tasks
2. CAPA MC as CAPV hybrid dev environment #3586
   5 of 5
   team/rocket +1
   
3. Network architecture for CAPA/CAPV hybrid MC #3587
   5 of 5
   team/rocket +1
   
4. e2e tests for capa/capv hybrid env #3619
   team/rocket +1
   
5. Sort out dns for CAPA/CAPV hybrid MC (likely via dns-operator-route53) #3624
   5 of 5
   team/rocket +1
   
6. Modify clusters-app-operator to fix vSphere proxy behaviour #3627
   team/rocket +1
   
7. Document CAPA-CAPV setup #3642
   team/rocket +1
   

[anvddriesch]

Closing this since the hybrid environment is functional, e2e tests are present and we have the documentation. We can open another issue once we have more specific requirements from customers.