Replace Provisioner by Kyma Infrastructure Manager [EPIC]

[tobiscr]

Description

The Provisioner has to be replaced by the Kyma Infrastrcuture Manager. The logic of the Provisioner has to be migrated into the Infrastructure Manager, but also considering already planned new features. This could required a rethinking of the current software architecture to ensure a flexible and extensible but also maintainable software structure of the Infrastructure Manager.

AC:

• Define and document KIM architecture #134

  • Create proposal for Cluster CR we expect from KEB #125
    • Provisioning CRD structure in KIM #176
    • Prepare CR examples for each plan we have
  • PoC that creates gardener cluster when CR is created akgalwas/infrastructure-manager#1
  • Finalise alignments under an open source ADR

• Main implementation part

  • Adopt the project structure to the agreed contract #197
    • (optional?) consider extracting that to a separate module (ask @tobiscr about this one)
  • Implement feature-flag in KIM to generate Gardener clusters from cluster CRs #184
  • KIM should create clusters for all hyperscalers  #206
  • Valid CRs statuses should be set when KIM's operations succeed or fail #193
  • kyma-projectinfrastructure-manager#185
  • Use Feature-Flag to decide whether KIM or Proivisioner is the "leading" system which is allowed to change the Shoot-Spec #250
  • Upgrading Runtime CR should lead to changes on Gardener Cluster #270
  • kyma-projectinfrastructure-manager#209
  • KIM must manage GardenerCluster resource once cluster is provisioned and deleted #282
  • kyma-projectinfrastructure-manager#198
  • Have a strategy on how to react if Gardener changes the Shoot spec #232
  • Provisioner should persist shoots under a feature flag and not manage runtimes #264
  • Implement remaining converter features #230
  • KIM is configuring auditlog for each SKR cluster #286
  • KIM has to disable SSH support for SKR clusters per default #284
  • KIM has to ensure that Assured Workloads are used for SKRs in KSA #299
  • Converter configs should work on different landscapes #320
  • KIM should stop processing any action on Gardener shoot after configured timeout  #324
  • Runtime is Failed even if Gardener Shoot is "in progress" #346
  • Shoot produced by KIM differs from the one created by Provisioner #343
  • Enable KIM on Dev and Stage in preview plan

• Extended Provisioning functionalities (Prio I -required with go-live)

  • Make sure that the new clusters shoot specs generated by KIM are the same as the ones created by Provisioner
  • Set force-deletion flag when creating shoot-cluster [KIM/feature] #103
  • Handle shoot-oidc-service extension when provisioning kyma [KIM/feature] #381
  • [QG] Perform load and stress test to verify operator's behaviour under load #14
  • [QG] Metrics: Identify and implement business critical metrics / KPIs, define an action plan and configure alerting rules #113

• Extended Provisioning functionalities (Prio II - required after go live)

  • Support the configuration of seeds within the same region as the shoot [KIM/feature] #241
  • Migrated shoot differs from the original one #342
  • Multiple worker groups [KIM/feature] #46 (Q4/24 -> Deadline: end of Nov)
  • Configurable networking filter for ingress traffic (geo-blocking) [EPIC] #136 (Q4/24)
  • Enable Registry Cache Extension [KIM/feature] #135 (not ready yet - Q1/25)

• Migration

  • Prepare a plan for migration from current GardenerCluster CR to new Kubeconfig CRs #187
  • Implement migration script to generate Runtime CRs by using Shoot-Spec CRs from Gardener #183
  • KIM and migration should correctly populate OIDC fields #242
  • Agree with KEB team how we migrate from Provisioner to KIm will be handled (e.g. firing REST-call to provisioner and generate in parallel a corresponding cluster CR) #186
  • Migrate from Provisioner to KIM on DEV #207
  • Setup release pipeline with SREs to ship KIM to KCP #335
  • Migrate STAGE
  • Migrate from GardenerCluster CR to KIM #251
  • Migrate PROD

• Stabilisation and Hardening (after MVP)

  • make a semver release of kim and use that version in all landscapes
  • Refactor Intialize state to make it smaller
  • Unable to provision cluster #332
  • Establish testing approach for KIM and KIM related components: setup e2e tests #216
  • [QG] Metrics: Identify and implement business critical metrics / KPIs, define an action plan and configure alerting rules #113
  • Configure IMDS2 for AWS instances per default [KIM/feature] #156
  • KIM should validate relevant parts of the shoot spec when Runtime CR is updated #208
  • Relevant provisioning metrics should be collected #192
  • Runtime CRs should be validated to assure they're correct #229
  • Implement mechanism for copying types related to provider specific config from Gardener  #254
  • Add validation for Runtime CRs created via KEB and that are available as samples  #244
  • Converters hardening #231
  • Move Runtime CR examples to a place that would be a single source of truth for both KIM and KEB and would not need to be obfuscated (and therefore could be applied right away for testing purposes).

• Cleanup

  • Migrate infrastructure related logic from KEB to KIM #127
  • Cover state machine with unit tests #333

Reasons

Replacing the old Kyma Provisioner with the Kyma Infrastructure Manager to follow new KCP architectural paradigm (K8s native application).

Attachments

Depends on
#134

[tobiscr]

Issue for @kyma-project/gopher for enabling KIM in Dry-Run mode on KCP DEV + STAGE: kyma-project/control-plane#3478