Grafana Heatmap display for prometheus histograms

Question

I cannot make grafana to display Latency heatmap, whatever i do it does not function correctly for me.

I have a prometheus histogram defined like this:

serverHandledHistogram: prometheus.NewHistogramVec(
            prometheus.HistogramOpts{
                Namespace: metricsconst.PrometheusNamespace,
                Name:      "grpc_server_handling_seconds",
                Help:      "XXX",
                Buckets: []float64{.005, .01, .025, .05, .1, .25, .5, 1, 2.5, 5, 10, 30, 60},
            },
            []string{metricsconst.AccountLabel, grpcServiceLabel, grpcMethodLabel, grpcStatusCodeLabel, clientLabel},
        ),

Now data in prometheus looks like this for specific time:

so data is correct and note that it is measured in seconds, so according to this data requests in this specific timeframe should have been displayed 5sec-30sec [there is where all hits are]Now when i try to display them with grafana heatmap this is what i get:

(so data is correct and note that it is measured in seconds, so according to this data requests in this specific timeframe should have been displayed 5sec-30sec [there is where all hits are])

Does anyone know how to configure grafana correctly to display prometheus histograms

Answer 1

Ok so i finally managed to do it, the correct way is to set the query like this:

sum(rate(pensieve_grpc_server_handling_seconds_bucket{namespace="$namespace", pod=~"pensieve.*", account=~"$account_id", grpc_method=~"$grpc_method"}[$__rate_interval])) by (le)

and select following options:

and