Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to provide label_values in grafana variables with time range for prometheus data source?

I have used a variable in grafana which looks like this: label_values(some_metric, service)

If the metric is not emitted by the data source at the current time the variable values are not available for the charts. The variable in my case is the release name and all the charts of grafana are dependent on this variable.

After the server I was monitoring crashed, this metric is not emitted. Even if I set a time range to match the time when metric was emitted, it has no impact as the query for the variable is not taking the time range into account.

In Prometheus I can see the values for the metric using the query: some_metric[24h]

In grafana this is invalid: label_values(some_metric[24h], service)

Also as per the documentation its invalid to provide $__range etc for label_values.

If I have to use the query_result instead how do I write the above invalid grafana query in correct way so that I get the same result as label_values? Is there any other way to do this?

The data source is Prometheus.

like image 800
Neeraj Krishna Avatar asked Oct 12 '18 10:10

Neeraj Krishna


1 Answers

I'd suggest query_result(count by (somelabel)(count_over_time(some_metric[$__range]))) and then use regular expressions to extract out the label value you want.

That I'm using count here isn't too important, it's more that I'm using an over_time function and then aggregating.

like image 159
brian-brazil Avatar answered Sep 19 '22 17:09

brian-brazil