Add CICD metrics #1681
Add CICD metrics #1681
Conversation
model/cicd/metrics.yaml
Outdated
| - id: metric.cicd.errors | ||
| type: metric | ||
| metric_name: cicd.errors | ||
| brief: 'The number of errors in the controller of the CICD system.' |
There was a problem hiding this comment.
The cicd.errors name seems too broad - it seems it would cover all kinds of errors in all CI/CD system parts and then it stops being practical.
The first time I read it, I assumed it to count pipeline run errors, but it's probably something else.
Wonder if we could limit the scope of this metric to something reasonable. E.g. should it be cicd.controller.errors? I could envision we'd want to differentiate per component, e.g. have a cicd.scheduler.errors, etc
There was a problem hiding this comment.
I intended to count the errors from the CICD system (controller, scheduler, agent) in this metric and not errors from the pipeline run.
It is indeed good to be able to distinguish the different system components as well as errors from the pipeline runs.
Adriel also made a good point that it would be good to be able to count pipeline run errors separately from just the cicd.pipeline.run.duration with result=error because a single pipeline run might encounter several different errors (some recoverable and others not).
Most likely this will be a derived metric, eg. by using a span metrics connector on the pipeline run spans or count connector on the controller logs.
I will define the following metrics to cover these:
cicd.pipeline.run.errors for errors encountered as part of the pipeline run execution
cicd.system.errors for errrors encountered in CICD system components with an attribute component (eg. scheduler, agent, controller, …)
model/cicd/metrics.yaml
Outdated
| - id: metric.cicd.errors | ||
| type: metric | ||
| metric_name: cicd.errors | ||
| brief: 'The number of errors in the controller of the CICD system.' |
There was a problem hiding this comment.
I intended to count the errors from the CICD system (controller, scheduler, agent) in this metric and not errors from the pipeline run.
It is indeed good to be able to distinguish the different system components as well as errors from the pipeline runs.
Adriel also made a good point that it would be good to be able to count pipeline run errors separately from just the cicd.pipeline.run.duration with result=error because a single pipeline run might encounter several different errors (some recoverable and others not).
Most likely this will be a derived metric, eg. by using a span metrics connector on the pipeline run spans or count connector on the controller logs.
I will define the following metrics to cover these:
cicd.pipeline.run.errors for errors encountered as part of the pipeline run execution
cicd.system.errors for errrors encountered in CICD system components with an attribute component (eg. scheduler, agent, controller, …)
This ensures that all result values are nouns.
|
Overall LGTM. A small (non-blocking) question: were |
No I did not consider it until now. It's only with the latest changes that the similarity between these two metrics became apparent. I'm open to combine these two metric pairs and distinguishing them with |
| value: timeout | ||
| brief: "A timeout caused the pipeline run to be interrupted." | ||
| stability: experimental | ||
| - id: cancellation |
There was a problem hiding this comment.
cicd-metrics.md needs to be regenerated. Not seeing these there.
Fixes #1600
Changes
This PR adds metrics for CICD systems and related attributes.
Merge requirement checklist
[chore]