Prometheus how to handle counters on server

Question

I have articles and for each article I want to have read count

# TYPE news_read_counter2 Counter
news_read_counter2{id="2000"} 168

now the counters on the servers are saved in redis\memcached so they can get reset from time to time so after a while the redis machine is restart and the server dont have the last news_read_counter number and if I start from zero again

# TYPE news_read_counter2 Counter
news_read_counter2{id="2000"} 2

now looking at the news_read_counter2{id="2000"} graph I see that the counter is getting dropped to 2 while the docs says:

A counter is a cumulative metric that represents a single numerical value that only ever goes up.

so now to keep track of the news_read_counter I need to save the data into db and I back to the start zone where I need to use mysql to handle my data

here an Image of counter after redis got restart:

Answer 1

You generally don't want to look at the total of a counter the way that you are in your example, because it's not very meaningful once you actually try to use it analytically.

The idea is that you want to know increases over a period of time. For example, do you want to know the total amount of article views for the last 7 days, for this month so far, for the last 30 days, etc.

This answer and this article do an excellent job of explaining all this, but here are some examples. For demonstration purposes I use a counter called walks_started_total.

The problem

Query: `walks_started_total`

Solution 1

Seeing the total for the last week: `increase(walks_started_total[1w])`

Solution 2

Over a 1 minute period: `increase(walks_started_total[1m])`

Answer 2

Counters are allowed to be reset to 0, so there's no need to do anything special here to handle it. See http://www.robustperception.io/how-does-a-prometheus-counter-work/ for more detail.

It's recommended to use a client library which will handle all of this for you.

Also, by convention you should suffix counters with _total so that metric should be news_reads_total.