Fail measurements if any of the jobs returned issues #389
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
olichtne
force-pushed
the
fail-measurements
branch
from
dcfc3c5
to
e0543e5
Compare
This property indicates if the Measurement finished ok or if there were any errors reported from the individual measurement test tool jobs that invalidate the results. The results can still be evaluated but they should be considered as invalid for some use cases e.g. setting new baselines. Signed-off-by: Ondrej Lichtner <[email protected]>
This is equivalent to the measurement_success so this is no longer needed. Signed-off-by: Ondrej Lichtner <[email protected]>
olichtne
force-pushed
the
fail-measurements
branch
from
e0543e5
to
9703145
Compare
The measurement_success property is now exposed as the result value of the MeasurementResult object. Signed-off-by: Ondrej Lichtner <[email protected]>
Not using **kwargs means that it wasn't possible to use named init arguments when creating a new instance of the object. Signed-off-by: Ondrej Lichtner <[email protected]>
olichtne
force-pushed
the
fail-measurements
branch
from
9703145
to
aadca4c
Compare
Axonis
previously approved these changes
Nov 26, 2024
The 'result' name conflicts with the 'result' from the main for loop, this overrides the value of the 'result' name which is used a few lines lower to calculate measurement_success. Signed-off-by: Ondrej Lichtner <[email protected]>
|
Looks good. |
jtluka
previously approved these changes
Nov 26, 2024
For some reason this Aggregated class was missing the individual_results property. Signed-off-by: Ondrej Lichtner <[email protected]>
olichtne
force-pushed
the
fail-measurements
branch
from
b6f7efb
to
012921b
Compare
|
testing done internally in beaker job i believe this is ready to be merged. |
|
Ack to merge. |
jtluka
approved these changes
Dec 2, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This is necessary to detect situation when something went wrong with the measurement and we should disqualify the measurement results from being fully trustworthy, e.g. for setting baselines.
Tests
Needs to be tested for all types of measurements, ideally with failures in the measurement process to see the changed results.
Reviews
@Axonis @enhaut @jtluka
Closes: #