Menu
Is there a way to make a Prometheus counter in Grafana truly monotonic?
The counters on my server (using the Prometheus Java library) get reset whenever the server is restarted and the counter drops to zero in Grafana too. I cannot find a way in the documentation for Prometheus queries. Neither does the Java library provide a way to make a Counter persistent over restarts.
816
Use the rate() / irate() functions in Prometheus to calculate the rate of increase of a Counter. By convention, the names of Counters are suffixed by _total . To create a counter use either new/1 or declare/1 , the difference is that new/ will raise Prometheus.
Prometheus evolves the counter approach a little more. It chooses to have its counters monotonically increasing until the client has to restart. The client never resets the counters.
Counters are a fundamental way for tracking how often an event occurs within an application or service. They are used to track and measure Prometheus metrics with continually increasing values (i.e. monotonically increasing values) which get exposed as time series.
irate() irate(v range-vector) calculates the per-second instant rate of increase of the time series in the range vector. This is based on the last two data points. Breaks in monotonicity (such as counter resets due to target restarts) are automatically adjusted for.
With counters, you almost never care about the value itself, but only about its rate of increase. Thus, counters are always meant to be used in combination with the rate() or increase() functions. These functions handle counter resets for you (any non-monotonic increase will be treated as a counter reset and neutralized in the rate calculation).
137
The way to approach this is to use the rate function, which handles counter resets.
39
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
