You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
One of the issues of using this exporter outside of the sidecar model is that it might generate false values when using rate(), specially for counters and (conventional) histograms (which are also counters at the end). Due to how the function itself works and how data is incremented inside the exporter.
Lenghty explanation - click to expand
For example, an application could send 5 increments to the metric `myapp.http.requests.total` before the exporter is scraped for the first time, this will result in a scrape like:
myapp_http_requests_total{} 5
If the metric stays with that same value for a while, when using rate() function over it will result in 0 because the value never changed, when in reality it was not existent before so it went from 0 to 5 because the pod just gone up.
More on this problem, in the excellent talk from [Bartek and Arthur](https://www.youtube.com/watch?v=nWf0BfQ5EEA&t=9s).
(Very) Recently, Prometheus started to accept zero-samples to tell it when a counter started to exist and therefore be able to properly calculate rates on series that are created like non-zero values.
Would be really nice to start preparing the exporter for this to work and we have more consistent metrics between Statsd and Prometheus. I would be willing to provide a PR in the near future if the maintainers think there is value in it.
The text was updated successfully, but these errors were encountered:
Summary
One of the issues of using this exporter outside of the sidecar model is that it might generate false values when using
rate()
, specially for counters and (conventional) histograms (which are also counters at the end). Due to how the function itself works and how data is incremented inside the exporter.Lenghty explanation - click to expand
For example, an application could send 5 increments to the metric `myapp.http.requests.total` before the exporter is scraped for the first time, this will result in a scrape like:If the metric stays with that same value for a while, when using rate() function over it will result in 0 because the value never changed, when in reality it was not existent before so it went from 0 to 5 because the pod just gone up.
(Very) Recently, Prometheus started to accept zero-samples to tell it when a counter started to exist and therefore be able to properly calculate rates on series that are created like non-zero values.
Would be really nice to start preparing the exporter for this to work and we have more consistent metrics between Statsd and Prometheus. I would be willing to provide a PR in the near future if the maintainers think there is value in it.
The text was updated successfully, but these errors were encountered: