COS integration improvements

[cbartz]

Applicable spec: n/a

Overview

• Log repo policy check http status code in case of failure.
• Log job conclusion.
• Update the dashboard to visualise job metrics, with the option to specify specific repositories. Job metrics are grouped in a separate row.

Rationale

More metrics are being collected and visualised to improve long-term monitoring and alerting.

Module Changes

• github_client, github_metrics, github_type, metrics_type, metrics, runner_manager, runner_metrics: Retrieve and include job conclusion
• pre-job script has been adapted to include repo policy http code

Checklist

• The charm style guide was applied
• The contributing guide was applied
• The changes are compliant with ISD054 - Managing Charm Complexity
• The documentation is generated using src-docs
• The documentation for charmhub is updated.
• The PR is tagged with appropriate label (urgent, trivial, complex)

[github-actions]

license-eye has totally checked 135 files.

Valid	Invalid	Ignored	Fixed
0	58	77	0

Click to see the invalid file list

• actions.yaml
• charmcraft.yaml
• config.yaml
• generate-src-docs.sh
• lib/charms/grafana_agent/v0/cos_agent.py
• metadata.yaml
• pyproject.toml
• scripts/build-image.sh
• scripts/pre-integration-test.sh
• src/charm.py
• src/charm_state.py
• src/errors.py
• src/event_timer.py
• src/firewall.py
• src/github_client.py
• src/github_metrics.py
• src/github_type.py
• src/lxd.py
• src/lxd_type.py
• src/metrics.py
• src/metrics_type.py
• src/repo_policy_compliance_client.py
• src/runner.py
• src/runner_logs.py
• src/runner_manager.py
• src/runner_manager_type.py
• src/runner_metrics.py
• src/runner_type.py
• src/shared_fs.py
• src/utilities.py
• templates/repo_policy_compliance_service.py
• tests/init.py
• tests/conftest.py
• tests/integration/conftest.py
• tests/integration/helpers.py
• tests/integration/test_charm_fork_repo.py
• tests/integration/test_charm_metrics.py
• tests/integration/test_charm_no_runner.py
• tests/integration/test_charm_one_runner.py
• tests/integration/test_charm_scheduled_events.py
• tests/integration/test_charm_with_proxy.py
• tests/integration/test_self_hosted_runner.py
• tests/status_name.py
• tests/unit/init.py
• tests/unit/conftest.py
• tests/unit/mock.py
• tests/unit/test_charm.py
• tests/unit/test_charm_state.py
• tests/unit/test_github_client.py
• tests/unit/test_github_metrics.py
• tests/unit/test_metrics.py
• tests/unit/test_runner.py
• tests/unit/test_runner_logs.py
• tests/unit/test_runner_manager.py
• tests/unit/test_runner_metrics.py
• tests/unit/test_shared_fs.py
• tests/unit/test_utilities.py
• tox.ini

[github-actions]

license-eye has totally checked 135 files.

Valid	Invalid	Ignored	Fixed
0	58	77	0

Click to see the invalid file list

• actions.yaml
• charmcraft.yaml
• config.yaml
• generate-src-docs.sh
• lib/charms/grafana_agent/v0/cos_agent.py
• metadata.yaml
• pyproject.toml
• scripts/build-image.sh
• scripts/pre-integration-test.sh
• src/charm.py
• src/charm_state.py
• src/errors.py
• src/event_timer.py
• src/firewall.py
• src/github_client.py
• src/github_metrics.py
• src/github_type.py
• src/lxd.py
• src/lxd_type.py
• src/metrics.py
• src/metrics_type.py
• src/repo_policy_compliance_client.py
• src/runner.py
• src/runner_logs.py
• src/runner_manager.py
• src/runner_manager_type.py
• src/runner_metrics.py
• src/runner_type.py
• src/shared_fs.py
• src/utilities.py
• templates/repo_policy_compliance_service.py
• tests/init.py
• tests/conftest.py
• tests/integration/conftest.py
• tests/integration/helpers.py
• tests/integration/test_charm_fork_repo.py
• tests/integration/test_charm_metrics.py
• tests/integration/test_charm_no_runner.py
• tests/integration/test_charm_one_runner.py
• tests/integration/test_charm_scheduled_events.py
• tests/integration/test_charm_with_proxy.py
• tests/integration/test_self_hosted_runner.py
• tests/status_name.py
• tests/unit/init.py
• tests/unit/conftest.py
• tests/unit/mock.py
• tests/unit/test_charm.py
• tests/unit/test_charm_state.py
• tests/unit/test_github_client.py
• tests/unit/test_github_metrics.py
• tests/unit/test_metrics.py
• tests/unit/test_runner.py
• tests/unit/test_runner_logs.py
• tests/unit/test_runner_manager.py
• tests/unit/test_runner_metrics.py
• tests/unit/test_shared_fs.py
• tests/unit/test_utilities.py
• tox.ini

[github-actions]

license-eye has totally checked 135 files.

Valid	Invalid	Ignored	Fixed
0	58	77	0

Click to see the invalid file list

• actions.yaml
• charmcraft.yaml
• config.yaml
• generate-src-docs.sh
• lib/charms/grafana_agent/v0/cos_agent.py
• metadata.yaml
• pyproject.toml
• scripts/build-image.sh
• scripts/pre-integration-test.sh
• src/charm.py
• src/charm_state.py
• src/errors.py
• src/event_timer.py
• src/firewall.py
• src/github_client.py
• src/github_metrics.py
• src/github_type.py
• src/lxd.py
• src/lxd_type.py
• src/metrics.py
• src/metrics_type.py
• src/repo_policy_compliance_client.py
• src/runner.py
• src/runner_logs.py
• src/runner_manager.py
• src/runner_manager_type.py
• src/runner_metrics.py
• src/runner_type.py
• src/shared_fs.py
• src/utilities.py
• templates/repo_policy_compliance_service.py
• tests/init.py
• tests/conftest.py
• tests/integration/conftest.py
• tests/integration/helpers.py
• tests/integration/test_charm_fork_repo.py
• tests/integration/test_charm_metrics.py
• tests/integration/test_charm_no_runner.py
• tests/integration/test_charm_one_runner.py
• tests/integration/test_charm_scheduled_events.py
• tests/integration/test_charm_with_proxy.py
• tests/integration/test_self_hosted_runner.py
• tests/status_name.py
• tests/unit/init.py
• tests/unit/conftest.py
• tests/unit/mock.py
• tests/unit/test_charm.py
• tests/unit/test_charm_state.py
• tests/unit/test_github_client.py
• tests/unit/test_github_metrics.py
• tests/unit/test_metrics.py
• tests/unit/test_runner.py
• tests/unit/test_runner_logs.py
• tests/unit/test_runner_manager.py
• tests/unit/test_runner_metrics.py
• tests/unit/test_shared_fs.py
• tests/unit/test_utilities.py
• tox.ini

[github-actions]

license-eye has totally checked 135 files.

Valid	Invalid	Ignored	Fixed
57	1	77	0

Click to see the invalid file list

• src/metrics_type.py

[jdkandersson]

Is it possible to see a breakdown of # of jobs by repo? So that we can get a high level overview of which repo is using it? I couldn't see it in the screenshot

[jdkandersson]

Looks fine, just the question regarding the repository. I couldn't spot where that info was being collected either?

[github-actions]

Test coverage for 64fcb09

Name                         Stmts   Miss Branch BrPart  Cover   Missing
------------------------------------------------------------------------
src/charm.py                   432     98    107     25    75%   79-80, 86-88, 112-113, 117-119, 155-157, 168->180, 178, 221-229, 241-245, 276, 358-363, 384, 401-402, 411-434, 452-454, 462-465, 472, 475-476, 492, 496-501, 546->549, 562-563, 565-566, 600-607, 631-632, 644-646, 661-662, 687-688, 690-691, 693-694, 768->770, 831-832, 848, 866-868, 872
src/charm_state.py              75      6     10      0    91%   90-97
src/errors.py                   37      0      0      0   100%
src/event_timer.py              42      8      4      0    74%   94-97, 117-120
src/firewall.py                 43     25     10      0    38%   38-42, 64-67, 75-149
src/github_client.py            68      6     32      3    87%   124, 146, 158-165, 183->216
src/github_metrics.py           14      0      0      0   100%
src/github_type.py              50      0      0      0   100%
src/lxd_type.py                 37      0      2      0   100%
src/metrics.py                  73      2     10      1    96%   61->64, 170-171
src/metrics_type.py              6      0      0      0   100%
src/runner.py                  313     64     92     23    74%   44->46, 125, 252-253, 294-303, 327-332, 337, 357, 361-371, 420->423, 426-428, 435, 449, 459, 463, 465, 480, 506-511, 527-540, 551-583, 588, 626, 672-674, 678, 696, 731, 757, 762-774, 788, 793->795
src/runner_logs.py              32      4      6      2    84%   33-36, 56->55, 60-61
src/runner_manager.py          266     40     88      8    84%   122, 167-169, 182-183, 195-197, 203-208, 212-213, 223-224, 243, 321-323, 388, 407-411, 436, 488, 522, 628-639, 649, 653-660
src/runner_manager_type.py      33      0      6      0   100%
src/runner_metrics.py          123      8     20      2    93%   147-148, 160, 191-192, 307-311
src/runner_type.py              45      0     10      0   100%
src/shared_fs.py                80     10     12      0    89%   73-74, 160-161, 166-167, 173-174, 196-197
src/utilities.py                68      6     20      8    84%   74->76, 78->84, 89-91, 121, 135, 173, 226
------------------------------------------------------------------------
TOTAL                         1837    277    429     72    82%

Static code analysis report

Run started:2024-01-03 17:35:11.408066

Test results:
  No issues identified.

Code scanned:
  Total lines of code: 4048
  Total lines skipped (#nosec): 0
  Total potential issues skipped due to specifically being disabled (e.g., #nosec BXXX): 12

Run metrics:
  Total issues (by severity):
  	Undefined: 0
  	Low: 0
  	Medium: 0
  	High: 0
  Total issues (by confidence):
  	Undefined: 0
  	Low: 0
  	Medium: 0
  	High: 0
Files skipped (0):

[cbartz]

Is it possible to see a breakdown of # of jobs by repo? So that we can get a high level overview of which repo is using it? I couldn't see it in the screenshot

Added a visualisation. Unfortunately the repo values are not sorted by count, this should be possible with Loki 2.8.x using the sort operator. See also grafana/grafana#17245 for this problem.

Looks fine, just the question regarding the repository. I couldn't spot where that info was being collected either?

The repos to be filtered can be defined in the mask for the variable "repository". (see first screenshot). Unfortunately, the list of all possible repos cannot be pre-calculated and just selected like the other variables (juju_application,...), because this is a query variable and the labels we use are calculated using a json parser expression (see https://community.grafana.com/t/grafana-variable-loki-label-values-for-json-parser-created-labels/50180). If it's important, we'd have to add the repository as a real label, but that would mean changing the promtail configuration and could be more complex (might need an adaptation of the grafana agent).