Add Metrics for Running PipelinesRuns at Pipeline and Namespace level #8280

pramodbindal · 2024-09-19T06:23:22Z

Changes

Currently metrics shown for Running Pipeline Count is at cluster level.
There is no way we can get that metric at namespace or pipeline level.
We have added the PipelineRun metric at pipeline and namespace level.
Level of PipelineRun Can be configured via ConfigMap. Default is "" . but it can be set to namespace or Pipelinerun.

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

Has Docs if any changes are user facing, including updates to minimum requirements e.g. Kubernetes version bumps
Has Tests included if any functionality added or changed
pre-commit Passed
Follows the commit message standard
Meets the Tekton contributor standards (including functionality, content, code)
Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings). See some examples of good release notes.
Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes

NONE

tekton-robot · 2024-09-19T06:23:31Z

Hi @pramodbindal. Thanks for your PR.

I'm waiting for a tektoncd member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

khrm

/ok-to-test

khrm · 2024-09-19T06:25:32Z

/kind feature

khrm · 2024-09-19T06:25:45Z

/test check-pr-has-kind-label

tekton-robot · 2024-09-19T06:25:48Z

@khrm: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test pull-tekton-pipeline-alpha-integration-tests
/test pull-tekton-pipeline-beta-integration-tests
/test pull-tekton-pipeline-build-tests
/test pull-tekton-pipeline-integration-tests
/test pull-tekton-pipeline-unit-tests

The following commands are available to trigger optional jobs:

/test pull-tekton-pipeline-go-coverage

Use /test all to run all jobs.

In response to this:

/test check-pr-has-kind-label

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

tekton-robot · 2024-09-19T06:33:37Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/metrics.go	76.2%	78.3%	2.1
pkg/pipelinerunmetrics/metrics.go	85.4%	83.7%	-1.7

tekton-robot · 2024-09-19T06:35:11Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/metrics.go	76.2%	78.3%	2.1
pkg/pipelinerunmetrics/metrics.go	85.4%	83.7%	-1.7

tekton-robot · 2024-09-19T07:03:34Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/metrics.go	76.2%	78.3%	2.1
pkg/pipelinerunmetrics/metrics.go	85.4%	83.7%	-1.7

tekton-robot · 2024-09-19T07:05:23Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/metrics.go	76.2%	78.3%	2.1
pkg/pipelinerunmetrics/metrics.go	85.4%	83.7%	-1.7

pramodbindal · 2024-09-19T07:24:31Z

/test pull-tekton-pipeline-build-tests
/test pull-tekton-pipeline-integration-tests

vdemeester

@pramodbindal can you squash your commits ? Otherwise LGTM.

pramodbindal · 2024-09-19T08:51:12Z

@pramodbindal can you squash your commits ? Otherwise LGTM.

Done

tekton-robot · 2024-09-19T08:53:22Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/metrics.go	76.2%	78.3%	2.1
pkg/pipelinerunmetrics/metrics.go	85.4%	83.7%	-1.7

tekton-robot · 2024-09-19T08:56:56Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/metrics.go	76.2%	78.3%	2.1
pkg/pipelinerunmetrics/metrics.go	85.4%	83.7%	-1.7

vdemeester

We might want to add a release note entry
It seems it needs to be rebased ?

tekton-robot · 2024-09-25T08:20:09Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: vdemeester

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [vdemeester]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

pramodbindal · 2024-09-26T07:25:41Z

@pritidesai @abayer
This PR is Approved by @vdemeester .
Please provide /lgtm tag so I can merge this.

tekton-robot · 2024-09-26T11:14:49Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/metrics.go	76.2%	78.3%	2.1
pkg/pipelinerunmetrics/metrics.go	85.4%	83.7%	-1.7

tekton-robot · 2024-09-26T11:17:28Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/metrics.go	76.2%	78.3%	2.1
pkg/pipelinerunmetrics/metrics.go	85.4%	83.7%	-1.7

afrittoli

Thanks for this PR. I left some comments, we definitely need some docs about new metrics and I would like to understand if the change is backwards compatible.

pkg/pipelinerunmetrics/metrics_test.go

config/config-observability.yaml

pkg/apis/config/metrics.go

afrittoli · 2024-09-26T13:10:38Z

pkg/pipelinerunmetrics/metrics.go

+	runningPRTag, err := getRunningPipelineRunTags(cfg)
+	if err != nil {
+		return err
+	}


NIT: It looks a bit strange to have some of the config processing logic directly in the viewRegister and some in a separate getRunningPipelineRunTags function.

Earlier logic is to get tags for PipelineRuns and new logic is to get RunningPipelineRun tags.

Now we have 2 options here.
extract older logic to new function just like we did it for running PRs or keep the new logic in view register only.
I will prefer former as that offers more clean code.

If the function is used in more than one place or if you write unit tests for it I see how it helps, otherwise I think we don't need a separate function

afrittoli · 2024-09-26T13:18:47Z

pkg/pipelinerunmetrics/metrics.go

+	case config.PipelinerunLevelAtPipelinerun:
+		tags = []tag.Key{pipelinerunTag, pipelineTag, namespaceTag}


If I read this right this counts how many PipelineRuns are running for a certain PipelineRun, i.e. 0 or 1?
I'm not sure I understand the value of this metric, it would be completely non-aggregated - is the intention to use this to build a graph of which pipeline is running at a certain point in time?

aggregation is being done in RunningPipelineRuns function.
intention is get count of running PRs in current time only. There is no intention of getting the historical data on the same.

pkg/pipelinerunmetrics/metrics.go

tekton-robot · 2024-09-27T08:12:31Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/metrics.go	76.2%	78.3%	2.1
pkg/pipelinerunmetrics/metrics.go	85.4%	82.7%	-2.7

tekton-robot · 2024-09-27T08:24:12Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/metrics.go	76.2%	78.3%	2.1
pkg/pipelinerunmetrics/metrics.go	85.4%	82.7%	-2.7

tekton-robot · 2024-09-27T08:27:53Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/metrics.go	76.2%	78.3%	2.1
pkg/pipelinerunmetrics/metrics.go	85.4%	82.5%	-2.9

tekton-robot · 2024-09-27T08:29:54Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/metrics.go	76.2%	78.3%	2.1
pkg/pipelinerunmetrics/metrics.go	85.4%	82.5%	-2.9

pramodbindal · 2024-09-27T09:53:16Z

/retest

pramodbindal · 2024-09-27T10:30:34Z

/retest

tekton-robot · 2024-09-27T10:43:38Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/metrics.go	76.2%	78.3%	2.1
pkg/pipelinerunmetrics/metrics.go	85.4%	82.5%	-2.9

tekton-robot · 2024-09-27T10:48:14Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/metrics.go	76.2%	78.3%	2.1
pkg/pipelinerunmetrics/metrics.go	85.4%	82.5%	-2.9

tekton-robot · 2024-09-27T10:52:06Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/metrics.go	76.2%	78.3%	2.1
pkg/pipelinerunmetrics/metrics.go	85.4%	82.5%	-2.9

tekton-robot · 2024-09-27T10:54:42Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/metrics.go	76.2%	78.3%	2.1
pkg/pipelinerunmetrics/metrics.go	85.4%	82.5%	-2.9

tekton-robot · 2024-09-27T11:40:23Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/metrics.go	76.2%	78.3%	2.1
pkg/pipelinerunmetrics/metrics.go	85.4%	82.5%	-2.9

tekton-robot · 2024-09-27T11:41:32Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/metrics.go	76.2%	78.3%	2.1
pkg/pipelinerunmetrics/metrics.go	85.4%	82.5%	-2.9

Currently metrices shown for Running Pipeline Count is at cluster level. There is no way we can get that metric at namespace or pipeline level. We have enhanced the running pipelinerun metric at pipelinerun, pipeline and namespace level (Can be configured via ConfigMap, Default behaviour is same as earlier).

tekton-robot · 2024-09-27T12:52:01Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/metrics.go	76.2%	78.3%	2.1
pkg/pipelinerunmetrics/metrics.go	85.4%	81.8%	-3.6

tekton-robot · 2024-09-27T12:55:46Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/metrics.go	76.2%	78.3%	2.1
pkg/pipelinerunmetrics/metrics.go	85.4%	81.8%	-3.6

tekton-robot · 2024-09-27T12:57:01Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/config/metrics.go	76.2%	78.3%	2.1
pkg/pipelinerunmetrics/metrics.go	85.4%	81.8%	-3.6

tekton-robot added the release-note-none Denotes a PR that doesnt merit a release note. label Sep 19, 2024

tekton-robot requested review from abayer and pritidesai September 19, 2024 06:23

tekton-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Sep 19, 2024

khrm reviewed Sep 19, 2024

View reviewed changes

tekton-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Sep 19, 2024

tekton-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Sep 19, 2024

vdemeester reviewed Sep 19, 2024

View reviewed changes

pramodbindal force-pushed the SRVKP-6184 branch from b802e66 to 6951973 Compare September 19, 2024 08:41

vdemeester approved these changes Sep 25, 2024

View reviewed changes

tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 25, 2024

pramodbindal force-pushed the SRVKP-6184 branch 2 times, most recently from 5c2ef54 to 4a05727 Compare September 26, 2024 11:01

pramodbindal changed the title ~~SRVKP-6184 - Adding running PR metrics~~ Add Metrics for Running PipelinesRuns t Pipeline and Namespace level Sep 26, 2024

pramodbindal changed the title ~~Add Metrics for Running PipelinesRuns t Pipeline and Namespace level~~ Add Metrics for Running PipelinesRuns at Pipeline and Namespace level Sep 26, 2024

tekton-robot added this to the Pipeline v0.64 milestone Sep 26, 2024

afrittoli reviewed Sep 26, 2024

View reviewed changes

tekton-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Sep 27, 2024

afrittoli modified the milestones: Pipeline v0.64, Pipeline v0.65 Sep 27, 2024

pramodbindal force-pushed the SRVKP-6184 branch 2 times, most recently from aa38559 to b0679b0 Compare September 27, 2024 10:39

pramodbindal force-pushed the SRVKP-6184 branch from b0679b0 to 7891813 Compare September 27, 2024 11:28

pramodbindal force-pushed the SRVKP-6184 branch from 7891813 to d3be2f8 Compare September 27, 2024 12:42

pramodbindal force-pushed the SRVKP-6184 branch from d3be2f8 to a73f04d Compare September 27, 2024 12:43

		case config.PipelinerunLevelAtPipelinerun:
		tags = []tag.Key{pipelinerunTag, pipelineTag, namespaceTag}

Add Metrics for Running PipelinesRuns at Pipeline and Namespace level #8280

Are you sure you want to change the base?

Add Metrics for Running PipelinesRuns at Pipeline and Namespace level #8280

Conversation

pramodbindal commented Sep 19, 2024 • edited Loading

Changes

Submitter Checklist

Release Notes

tekton-robot commented Sep 19, 2024

khrm left a comment

Choose a reason for hiding this comment

khrm commented Sep 19, 2024

khrm commented Sep 19, 2024

tekton-robot commented Sep 19, 2024

tekton-robot commented Sep 19, 2024

tekton-robot commented Sep 19, 2024

tekton-robot commented Sep 19, 2024

tekton-robot commented Sep 19, 2024

pramodbindal commented Sep 19, 2024

vdemeester left a comment

Choose a reason for hiding this comment

pramodbindal commented Sep 19, 2024

tekton-robot commented Sep 19, 2024

tekton-robot commented Sep 19, 2024

vdemeester left a comment

Choose a reason for hiding this comment

tekton-robot commented Sep 25, 2024

pramodbindal commented Sep 26, 2024

tekton-robot commented Sep 26, 2024

tekton-robot commented Sep 26, 2024

afrittoli left a comment

Choose a reason for hiding this comment

afrittoli Sep 26, 2024

Choose a reason for hiding this comment

pramodbindal Sep 27, 2024

Choose a reason for hiding this comment

afrittoli Sep 27, 2024

Choose a reason for hiding this comment

afrittoli Sep 26, 2024

Choose a reason for hiding this comment

pramodbindal Sep 27, 2024

Choose a reason for hiding this comment

tekton-robot commented Sep 27, 2024

tekton-robot commented Sep 27, 2024

tekton-robot commented Sep 27, 2024

tekton-robot commented Sep 27, 2024

pramodbindal commented Sep 27, 2024

pramodbindal commented Sep 27, 2024

tekton-robot commented Sep 27, 2024

tekton-robot commented Sep 27, 2024

tekton-robot commented Sep 27, 2024

tekton-robot commented Sep 27, 2024

tekton-robot commented Sep 27, 2024

tekton-robot commented Sep 27, 2024

tekton-robot commented Sep 27, 2024

tekton-robot commented Sep 27, 2024

tekton-robot commented Sep 27, 2024

pramodbindal commented Sep 19, 2024 •

edited

Loading