PromQL avg_over_time for non-zero values

Question

I am trying to get avg_over_time value from gauge metric, but I would want average only from the non-zero values of the metric (or values higher than zero, to be exact).

Example:

avg_over_time(foo[2d] > 0)

But I alwas get parse error: binary expression must contain only scalar and instant vector types

I tried setting up recording rule

expr: foo > 0

But unfortunately with the same result.

Is this possible in PromQL?

Answer 1

You can use a sub-query with Prometheus version above 2.7:

avg_over_time((foo > 0)[2d:])

Answer 2

While it is possible to use Prometheus subquery such as avg_over_time((foo > 0)[2d:]) as Michael already mentioned, the proposed query may either miss certain data points or double-count certain datapoints depending on the resolution value after the colon in square brackets. For example, avg_over_time((foo > 0)[2d:1h]) would take into account only a single raw data point per each hour when calculating avg_over_time, while avg_over_time((foo>0)[2d:1s]) would count the same raw datapoints multiple times if scrape interval for foo exceeds 1s. By default the resolution value equals to step value passed in the query to /api/v1/query_range, so the end result of the query may be quite different depending on the step query arg passed to /api/v1/query_range.

If you need to calculate the exact share of raw datapoints exceeding the given threshold independently of step query arg, then take a look at share_gt_over_time(foo[2d], threshold) function from MetricsQL. This function may be useful for calculating SLO/SLI over uptime and/or latencies.