Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test: add quality gate for percentage of namespaces exercised by at least one test image #308

Open
spiffcs opened this issue Sep 29, 2023 · 0 comments
Labels
enhancement New feature or request

Comments

@spiffcs
Copy link
Contributor

spiffcs commented Sep 29, 2023

What would you like to be added:
During the quality gate we want to check if results are being tested from > 70% of namespaces.

Example: coverage of namespaces sampled is at 71%, a user adds a new provider that adds a new namespace, coverage would drop < 70% causing the gate to fail since the new namespace is not yet exercised by at least one test image.

Why is this needed:
To catch if there are failures using our test data against current/new providers before running a vunnel release

Additional context:
The tools team is able to point to a Post-Mortem regarding missing debian data that resulted in this issue being filed

@spiffcs spiffcs added the enhancement New feature or request label Sep 29, 2023
@spiffcs spiffcs changed the title Test: quality gate for percentage of namespaces covered Test: add quality gate for percentage of namespaces covered Sep 29, 2023
@spiffcs spiffcs changed the title Test: add quality gate for percentage of namespaces covered Test: add quality gate for percentage of namespaces exercised by at least one test image Sep 29, 2023
@spiffcs spiffcs self-assigned this Sep 29, 2023
@wagoodman wagoodman removed this from OSS Sep 29, 2023
@wagoodman wagoodman added this to OSS Feb 7, 2024
@wagoodman wagoodman moved this to In Progress in OSS Feb 7, 2024
@spiffcs spiffcs moved this from In Progress to Back Burner in OSS May 24, 2024
@spiffcs spiffcs moved this from Stalled to Ready in OSS Aug 1, 2024
@spiffcs spiffcs removed their assignment Oct 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: Ready
Development

No branches or pull requests

1 participant