O11y-Op testing

[Rotfuks]

Motivation

In order to raise our confidence in the stability of our observability platform and be sure that our ongoing work and releases won't negatively impact our observability platform operations we need to create extensive tests giving us early feedback loops.

Todo

• Investigate some good initial test cases to give us feedback on the stability of a release on mimir on CI. This can be:
  • Validate Helm chart templating
  • Deploy chart on Kind (single instance)
  • Implement those CI test cases
• Investigate how we can setup an e2e test case for the o11y-op
  • We can just check if the configmaps are created and if the content makes sense

Outcome

• We have early, automatic feedback about the impact of a release of the o11y-op before releasing.

[QuantumEnigmaa]

✔️ Helm chart CI tests done.

[QuentinBisson]

@QuantumEnigmaa anything we still have to do here?

[QuantumEnigmaa]

Yeah, as discussed during refinement, we'll have to create a "e2e" testing framework on our own :

• We will update the PR message to empower the user to perform the required tests before asking reviews for its PR.
• A dedicated directory will be created at the repo's root with a file describing the test procedure and possible scripts to run manually after deploying the custom branch version on a MC.
• Concerning olly-op specifically, the testing procedure might consist in creating a WC after having deployed olly-op custom branch's version on a testing MC let it run a bit and check on if the remote-write secret for the WC has been created.

Sorry I only updated Loki issue but that will be the same thing for all MC-only app.

[QuentinBisson]

No worries :)

[Rotfuks]

Here's the ticket of the new @giantswarm/team-tenet for tests of apps only running on MC: #2930
have you taken the considerations in there into account?

[QuantumEnigmaa]

Yes but as even Marcus isn't sure on how or when this will eventually end, I'd prefer that we craft something on our own for now that will still be usable if tenet's framework become available, even if it's a bit more "manual".

[QuantumEnigmaa]

Manual e2e testing procedure and script now available in the app's repo + PR message updated.