Automated testing of management cluster apps

[yulianedyalkova]

Now that we have the tooling and infrastructure around WC apps, it's time adapt them for MC apps.

There are a few open questions before we can tackle that though:

• How do we provide reliable and reusable MCs without increasing costs too much?
• How do we deal with MCs and upgrade tests? Do we need to introduce releases for MCs?
• Do we structure the tests in the same way we do for WCs? (separate validation test suite (meaning black box tests, no MC changes) and per app tests (white box, can modify the MC).

More context and internal team discussion from onsite in Morocco.

Current plan:

• we create a pool of ephemeral MCs that are pre-configured and we can use for app testing
• size of the pool of ephemeral MCs is easily configured
• we always have at least one MC "pre-warmed" and available for app tests to instantly run on
• when an app test suite is triggered, that automatically warms up a new one from the pool to ensure availability for next run (best effort)
• there is a queuing mechanism for when demand is bigger that the pool of available MCs
• after a test run finishes, it destroys the MC it ran on to ensure no test pollution
• idea: prioritize runs triggered by individuals over bot ones (renovate etc)

Tasks

Beta Give feedback

1. Investigate ephemeral MCs for MC app testing #3718
   area/kaas team/tenet topic/testing +3
   
2. Configure an ephemeral pool of MCs #3742
   area/kaas team/tenet topic/testing +3
3. Define MC pool: automation, pre-warm up (At the beginning of the day, I have a configured pool of MCs and one pre-warmed up)
4. Example app test repo: define framework, run against MC from the pool, delete MC (I can run tests against a MC from the pool)
5. Create and integrate tasks for automatic pre-warm of MC on test run (Automate pre-warming up MCs for next test run)
6. Prioritize human-triggered runs over bot ones (I can ensure my run will run before automatic / bot ones)