systemd integration currently does not expose all metrics.

[mateoflorido]

Summary

Currently, there is an issue where metrics cannot be scraped from the services running in the node.

Expected Behavior

The node_exporter should expose the following metrics:

ubuntu@juju-067522-4:~$ curl -s localhost:9100/metrics | grep systemd
...
node_systemd_unit_state{name="nginx.service",state="activating",type="forking"} 0
node_systemd_unit_state{name="nginx.service",state="active",type="forking"} 1
node_systemd_unit_state{name="nginx.service",state="deactivating",type="forking"} 0
node_systemd_unit_state{name="nginx.service",state="failed",type="forking"} 0
node_systemd_unit_state{name="nginx.service",state="inactive",type="forking"} 0
...

Current Behavior

However, at the moment, the node_exporter only exposes the following metrics:

ubuntu@juju-067522-4:~$ curl -s localhost:12345/integrations/node_exporter/metrics | grep systemd
node_scrape_collector_duration_seconds{collector="systemd"} 0.076081123
node_scrape_collector_success{collector="systemd"} 0
# HELP node_systemd_version Detected systemd version
# TYPE node_systemd_version gauge
node_systemd_version{version=""} 0

Steps to Reproduce

1. Deploy the Grafana Agent machine charm as a subordinate.
2. Attempt to scrape the systemd metrics inside the charm unit.

[peppepetra]

This is still an issue on grafana-agent 0.35.4.

Having node_systemd_unit_state will allow us to replace all systemd nrpe check and move quicker to native COS, removing the need of cos-proxy