Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can I sort the legend by series name in Prometheus/Grafana

I have a Grafana dashboard panel configured to render the results of a Prometheus query. There are a large number of series returned by the query, with the legend displayed to the right. If the user is looking for a specific series, they have to potentially scroll through all of them, and it's easy to miss the one they're looking for. So I'd like to sort the legend by series name, but I can't find any way to do that.

My series name is a concatenation of two labels, so if I could sort the instant vector returned from the PromQL query by label value, I think Grafana would use that order in the legend. But I don't see any way to do that in Prometheus. There is a sort() function, but it sorts by sample value. And I don't see any way to sort the legend in Grafana.

like image 494
MidnightJava Avatar asked Oct 16 '20 19:10

MidnightJava


People also ask

Does Grafana support sorting by value when displaying legend in Prometheus?

According to this Issue, Grafana displays the query results from Prometheus without sorting. According to this Issue, Grafana supports sorting by value when displaying legend.

How to sort the Order of the legend columns alphabetically?

So you can see the order is sorted alphabetically, but I promoted the "total" legend to be first. Something like "sort order override". Then I can put "total" as 1st, and the rest of the series are ordered alphabetically asc or desc. Hope this makes sense! Thanks again! you can specify sort order by clicking on the legend column headers.

How do I create a legend for my data in Grafana?

In the panel display options pane, locate the Legend section. In the Legend values field, select the values you want to appear in the legend. Click Apply to save your changes are navigate back to the dashboard. By default, Grafana specifies the color of your series data, which you can change.

How do I change the Order of the legend order?

I think the ideal place to put this would be Edit Panel -> Axis & Grid -> Legend -> Sort Order However, optionally I would also like to override the sort order. For example, in the image above, I would like to have the order as: So you can see the order is sorted alphabetically, but I promoted the "total" legend to be first.


2 Answers

As far as I know, You can only use the function sort() to sort metrics by value.

  • According to this PR, Prometheus does not intend to provide the function sort_by_label().
  • According to this Issue, Grafana displays the query results from Prometheus without sorting.
  • According to this Issue, Grafana supports sorting by value when displaying legend.

In Grafana 7, Prometheus metrics can be transformed from time series format to table format using the Transform module, so that you can sort the metrics by any label or value.

like image 96
LeoHsiao Avatar answered Sep 29 '22 18:09

LeoHsiao


Prometheus doesn't support sorting results by series names as mentioned in https://stackoverflow.com/a/65353296/274937 . If you still need sorting results by metric name or by arbitrary set of labels, then take a look at VictoriaMetrics. It provides sort_by_label and sort_by_label_desc functions. For example, the following query sorts time series returned by query by metric names:

sort_by_label(query, "__name__")

The __name__ refers to metric name. See these docs for more details.

like image 41
valyala Avatar answered Sep 29 '22 17:09

valyala