Implement Micrometer Metrics Client

[adamgfraser]

We should implement a Micrometer based metrics client. This can be used to interface with a variety of other metrics systems and will dramatically improve the ability of ZIO based applications to integrate with other metrics solutions.

As with other metrics clients, this would be implemented in terms of calling the unsafeSnapshot and unsafeInstallListener operators on MetricClient in ZIO core and implementing a ZIOApp that would perform any necessary setup and teardown.

[atooni]

This is a new backend, we need to study the documentation at https://micrometer.io/ to understand how we can expose the metrics to micrometer.

[pjfanning]

I've been playing around with https://github.com/pjfanning/zio-metrics-micrometer - maybe some of this work could be applied here

[atooni]

Hi @pjfanning That is a great idea. At the moment we are on the finishing touches to do a ZMX release based on ZIO2 RC4. In parallel we are adding New Relic support - I would be very interested to have a look before we are getting out the final release of ZIO 2 - in case we are missing something in ZIO core that would make reporting to Micrometer possible at all all or easier.

[pjfanning]

Hi @atooni - I had a quick look at the JVMMetrics in zio 2 core and at the zio-zmx code this morning. I'm not sure how good a fit Micrometer will be (at least in the short term).

You're probably very aware of the issues below but allow me to list them just in case you haven't had a good look at Micrometer.

Micrometer is more of an API with many backend implementations (with a MeterRegistry being the main entry point). To support it, we'd probably have to have a jar dependency on micrometer-core and allow users a mechanism to register their preferred MeterRegistry (including a composite registry that allows metrics to be reported to multiple backends simultaneously).

To me, it also feels like Prometheus and Micrometer (and probably many others) supply their own mechanisms to report the common JVM metrics - so there would be a fair degree of overlap (and potentially wasteful CPU cycles gathering these metrics for zio-zmx and Prometheus simpleclient_hotspot, as an example).

Some of the Micrometer backend work on a push mechanism where the MeterRegistry is configured to periodically push to a SasS service. The idea of a snapshot that returned a String like val snapshot: ZIO[PrometheusClient, Nothing, String] would probably not make sense with at least some micrometer backends.

I see no harm in continuing with the existing solution of ignoring Micrometer and writing Prometheus, NewRelic and StatsD support explicitly.

One type of metric that I like in Micrometer, is the way Gauges wrap function calls as opposed to storing state themselves. I can see the benefit of supporting Gauges that store the state themselves too. Would it be possible to have ZIO core support both types of Gauge?

Please feel free to correct me on any mistakes I've made and if there is anything I can help with, give me a shout.