Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why there are both counters and gauges in Prometheus if gauges can act as counters?

When deciding between Counter and Gauge, Prometheus documentation states that

To pick between counter and gauge, there is a simple rule of thumb: if the value can go down, it is a gauge. Counters can only go up (and reset, such as when a process restarts).

They seem to cover overlapping use cases: you could use a Gauge that only ever increases. So why even create the Counter metric type in the first place? Why don't you simply use Gauges for both?

like image 322
Jose Armesto Avatar asked Nov 02 '19 18:11

Jose Armesto


People also ask

What is the difference between gauge and counter?

Gauge metrics represent data that has a specific value at each point in time. Counter metrics represent a count of occurrences in a time interval. Cumulative counter metrics represent a running count of occurrences.

What are counters in Prometheus?

Counters are running or cumulative counts with metric client libraries that keep an ever increasing total sum of the number of events for the lifetime of the application. These events can be periodically measured by having Prometheus scrape the metrics endpoint exposed by a client library.

What is a gauge in Prometheus?

A gauge in Prometheus is represented by a 64-bit floating point number. That means it can store very large or small decimal numbers, either positive or negative.

How do Prometheus gauges work?

The Prometheus gauge is essentially the same simple idea as gauges in other monitoring systems. Gauges can go up and down over time, and scrapes take a snapshot of the current value. There's no potential for messing around with moving averages or resets, as there is with counters.


2 Answers

From a conceptual point of view, gauge and counter have different purposes

  • a gauge typically represent a state, usually with the purpose of detecting saturation.
  • the absolute value of a counter is not really meaningful, the real purpose is rather to compute an evolution (usually a utilization) with functions like irate/rate(), increase() ...

Those evolution operations requires a reliable computation of the increase that you could not achieve with a gauge because you need to detect resets of the value.

Technically, a counter has two important properties:

  1. it always starts at 0
  2. it always increases (i.e. incremented in the code)

If the application restarts between two Prometheus scrapes, the value of the second scrape in likely to be less than the previous scrape and the increase can be recovered (somewhat because you'll always loose the increase between the last scrape and the reset).

A simple algorithm to compute the increase of counter between scrapes from t1 to t2 is:

  • if counter(t2) >= counter(t1) then increase=counter(t2)-counter(t1)
  • if counter(2) < counter(t1)then increase=counter(t2)

As a conclusion, from a technical point of view, you can use a gauge instead of a counter provided you reset it to 0 at startup and only increment it but any violation of contract will lead to wrong values.

As a side note, I also expect a counter implementation to use unsigned integer representation while gauge will rather use a floating point representation. This has some minor impacts on the code such as the ability to overflow to 0 automatically and better support for atomic operations on current cpus.

like image 123
Michael Doubez Avatar answered Oct 02 '22 13:10

Michael Doubez


For counters you care about how fast it is increasing, whereas for gauges you care about the actual value. While there can be gauges that (in theory) only go up, that doesn't make them counters.

like image 29
brian-brazil Avatar answered Oct 02 '22 14:10

brian-brazil