Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can I group labels in a Prometheus query?

If I have a metric with the following labels:

my_metric{group="group a"}  100 my_metric{group="group b"}  100 my_metric{group="group c"}  100 my_metric{group="misc group a"}  1 my_metric{group="misc group b"}  2 my_metric{group="misc group c"}  1 my_metric{group="misc group d"}  1 

Is it possible to do a query or even a label_replace that combines the 'misc' groups together?

(I realize that the metric cardinality needs to be improved, and I've updated the app to fix it. However it left me with this question for if I wanted to fix the metrics via a query later)

like image 471
checketts Avatar asked Jul 17 '17 23:07

checketts


People also ask

How can we join two metrics in a Prometheus query?

PromQL supports the ability to join two metrics together: You can append a label set from one metric and append it to another at query time. This can be useful in Prometheus rule evaluations, since it lets you generate a new metric for a series by appending labels from another info metric.

What is TOPK Prometheus?

The topk() function in Prometheus and Loki returns the topk per interval. This means that at time T1, the topk series returned is a different set as that on T2. As a result of this, the graph in Grafana may contain more series in total than what you would expect.

Which query language does Prometheus use to filter the data?

Prometheus provides a functional query language called PromQL (Prometheus Query Language) that lets the user select and aggregate time series data in real time.


2 Answers

Yes, you can you use label replace to group all the misc together:

sum by (new_group) (   label_replace(     label_replace(my_metric, "new_group", "$1", "group", ".+"),     "new_group", "misc", "group", "misc group.+"   ) ) 

The inner label_replace copies all values from group into new_group, the outer overwrites those which match "misc group.+" with "misc", and we then sum by the "new_group" label. The reason for using a new label is the series would no longer be unique if we just overwrote the "group" label, and the sum wouldn't work.

like image 75
tom.wilkie Avatar answered Sep 20 '22 17:09

tom.wilkie


It's even easier

sum by (group) (my_metric) 
like image 23
uamanager Avatar answered Sep 20 '22 17:09

uamanager