[SonarQube] Re-enable collection of metrics related to "New Code"

[IanBrison]

What does this PR do?

Re-enables the collection of metrics that start with new_ (e.g., new_vulnerabilities), which have been skipped since the following PR was merged.

• Fix automatic discovery of metrics from web API and improve example config documentation #8552

Additionally, it includes a change in the method of extracting the value to prevent errors.

Motivation

We want to utilize the metrics provided by SonarQube.

The official Datadog documentation states that these metrics can be used, creating a discrepancy between the feature and the documentation.

• https://docs.datadoghq.com/integrations/sonarqube

About the metrics

The metrics that start with new_ are about the stats of "New Code".
You can check the response data structure of GET api/measures/component in the "Response Example" of the following link.
https://next.sonarqube.com/sonarqube/web_api/api/measures/component

...
    "measures": [
      {
        "metric": "complexity",
        "value": "12"
      },
      {
        "metric": "new_violations",
        "period": {
          "value": "25",
          "bestValue": false
        }
      },
...

(In the PR that I linked above had "periods" in the response data, but that was deprecated and is gone now)
So, with metrics that start with new_, you have to extract the value from not measure['value'] but from measure['period']['value'].

Testing this change with my local datadog-agent, I have confirmed that the new metrics were also sent to Datadog.

Review checklist (to be filled by reviewers)

• Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
• Add the qa/skip-qa label if the PR doesn't need to be tested during QA.
• If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

[IanBrison]

I’ve also confirmed that the test data I added caused the test to fail, and then the later changes fixed it as I intended.

• https://github.com/DataDog/integrations-core/actions/runs/12981189797/job/36199283673

error message

FAILED tests/test_unit.py::test_service_check_ok - Exception: 
Traceback (most recent call last):
  File "/home/runner/work/integrations-core/integrations-core/datadog_checks_base/datadog_checks/base/checks/base.py", line 1301, in run
    self.check(instance)
  File "/home/runner/work/integrations-core/integrations-core/datadog_checks_base/datadog_checks/base/utils/tracing.py", line 47, in wrapper
    return f(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/integrations-core/integrations-core/sonarqube/datadog_checks/sonarqube/check.py", line 33, in check
    self.collect_metrics()
  File "/home/runner/work/integrations-core/integrations-core/sonarqube/datadog_checks/sonarqube/check.py", line 47, in collect_metrics
    self.collect_components(available_metrics)
  File "/home/runner/work/integrations-core/integrations-core/sonarqube/datadog_checks/sonarqube/check.py", line 53, in collect_components
    self.collect_metrics_from_component(available_metrics, component, tag_name, should_collect_metric)
  File "/home/runner/work/integrations-core/integrations-core/sonarqube/datadog_checks/sonarqube/check.py", line 107, in collect_metrics_from_component
    self.gauge(available_metrics[measure['metric']], measure['value'], tags=tags)
               ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
KeyError: 'new_vulnerabilities'  

• https://github.com/DataDog/integrations-core/actions/runs/12981531501/job/36200083686

[IanBrison]

Hi. Sorry for sending a review request with an incomplete PR. I'm new here.

I couldn't figure out how to add the change log. I read the document of ddev changelog and followed the steps, but I couldn't see any changes. Please let me know what I might be missing or guide me on what to do next.

$ pwd
/my/user/directory/integrations-core/sonarqube
$ ls
CHANGELOG.md   README.md      assets         datadog_checks hatch.toml     manifest.json  metadata.csv   pyproject.toml tests
$ ddev release changelog new  
No changelog entries to create

Thank you in advance.

[HadhemiDD]

These metrics had previously caused issues with the metrics collection a few years ago. Reviewing this again to see if the metrics naming is still a blocker.

[IanBrison]

@HadhemiDD
Hi! Thank you for your comment.
Could I ask how this is progressing? I'm not trying to hurry you; I'm just unsure whether I should wait for further updates.
I'm quite interested in these metrics, so if further investigation is needed, please let me know. I'm happy to research more about SonarQube and try out solutions in our environment if necessary!
Thank you very much for your help!