Automated testing of CAPI workload clusters

[puja108]

User Story

• As a Giant Swarm Engineer I want to run tests on Workload Clusters provisioned by upstream Cluster API (CAPI) controllers so that I can ensure our future CAPI releases are working correctly.

Details, Background

We are currently using automated tests to ensure reliability and functionality of Giant Swarm Workload Cluster releases. We aim to enable the same testing functionality for clusters created through upstream CAPI controllers, so we can continue using our tooling for testing and releasing vanilla CAPI releases.

This is essential for upcoming stories to make CAPI clusters as reliable (and repeatable) as our current production setups as well as align with current defaults that our customers are used to get out of the box.

Tasks

Beta Give feedback

1. Brainstorm ideas for implementing a WC testing pipeline #2055
   team/tinkerers topic/testing +2
2. Evaluate upstream e2e framework #2067
   team/tinkerers topic/testing +2
   
3. Create a framework for basic WC cluster tests #2078
   team/tinkerers topic/testing +2
   
4. Create a Ginkgo framework for WC cluster tests #2080
   team/tinkerers topic/testing +2
   
5. Integrate our Ginkgo framework tests into Tekton #2081
   team/tinkerers topic/testing +2
   
6. https://github.com/giantswarm/giantswarm/issues/26378
7. Enable upgrade testing for workload clusters #2679
   team/tinkerers topic/testing topic/workload-cluster +3
   
8. Periodically run and validate WC tests for all providers #2623
   team/tinkerers topic/testing +2
   
9. Notify on failed and successful PR runs of workload cluster tests #2767
   team/tinkerers topic/testing +2
   

[gawertm]

topic for new Releng Team in Kaas. Will still keep in backlog to track and see what we can already do until then

[gawertm]

@puja108 In regards to automation we agreed on these next steps for Rockets and the current mc-bootstrap epic. #685 (comment)
Having it for all providers with basic smoke tests, is already a huge improvement. Other Follow-ups (e.g. the automatic workload cluster testing) I consider out of scope for Rocket. So I would also unassign Rocket from this Issue here. Do you agree?

[teemow]

A higher level brainstorming of release engineering is here atm: https://github.com/giantswarm/giantswarm/issues/24704

[JosephSalisbury]

@yulianedyalkova do you know what's left here? the todo list in the description looks done done

[yulianedyalkova]

@JosephSalisbury I somehow missed your ping, sorry!

We have the basic functionality in place (incl. framework and tests), there will be more follow-up stories to fix and expand the features but they can be separate from the epic so closing.