How to register Micrometer Timer with SLA and tags?

Question

I'm trying to migrate my Prometheus metrics to micrometer but now I'm stuck with one thing here...

At the moment I have a Prometheus histogram configured as follows:

private static final Histogram REQUEST_DURATION = Histogram
        .build("http_request_duration_milliseconds", "Duration in milliseconds for processing a request.")
        .labelNames("http_method", "http_status", "java_class", "java_method")
        .buckets(10, 25, 50, 100, 500, 1000)
        .register();

So for switching to Micrometer I replaced it as follows:

Timer.builder("http.request.duration")
            .description("Duration in seconds for processing a request.")
            .sla(Duration.ofMillis(10), Duration.ofMillis(25), Duration.ofMillis(50), Duration.ofMillis(100), Duration.ofMillis(500), Duration.ofMillis(1000), Duration.ofMillis(5000))
            .register(registry);

Ok. Let's see how I want to use it... At the moment I simply call

REQUEST_DURATION.labels(httpMethod, httpStatus, javaClass, javaMethod).observe(milliseconds);

So I replaced this to

Metrics.timer("http.request.duration",
            "http.method", httpMethod,
            "http.status", httpStatus,
            "java.class", javaClass,
            "java.method", javaMethod)
            .record(Duration.ofNanos(nanoseconds));

But the problem now is, that Micrometer complains that I previously configured the metric without those tags. Of course I did, because I don't know the values at that point. Here the Exception:

java.lang.IllegalArgumentException: Prometheus requires that all meters with the same name have the same set of tag keys. There is already an existing meter containing tag keys []. The meter you are attempting to register has keys [http.method, http.status, java.class, java.method].

Ok. So I thought, then let's specify the buckets with the Metrics.timer call. But that doesn't work because there is no method for passing these values.

So... How can I set the sla buckets and the tags for my metric?

Answer 1

I got the answer on Micrometer slack channel. The Micrometer-way of solving this, is not to register the metric itself but instead to register a filter as follows:

registry.config().meterFilter(new MeterFilter() {
    @Override
    public DistributionStatisticConfig configure(Meter.Id id, DistributionStatisticConfig config) {
        if (id.getName().equals("http.request.duration")) {
            return DistributionStatisticConfig.builder()
                    .sla(Duration.ofMillis(10).toNanos(),
                         Duration.ofMillis(25).toNanos(),
                         Duration.ofMillis(50).toNanos(), 
                         Duration.ofMillis(100).toNanos(),
                         Duration.ofMillis(500).toNanos(),
                         Duration.ofMillis(1000).toNanos(), 
                         Duration.ofMillis(5000).toNanos())
                    .build()
                    .merge(config);
        }
        return config;
    }
});

When pushing a metric value using Metrics.timer(...) as above Micrometer will call this filter and apply all the configuration specified here. This filter is only called on meter initialization, i.e. when Metrics.timer(...) is called the first time with this specific name and tags. So we don't have to worry about performance here.

Answer 2

First, it is best to use dot separators between name parts. This keeps the metric vendor-neutral so if you ever decide to ship to something other than Prometheus, it works! In other words, best to record it as http.request.duration. When the Prometheus naming convention is applied, it will appear as http_request_duration_seconds.

If this is a Spring Boot application, you can declare the SLAs in a property-based meter filter:

management.metrics.distribution.sla.http.request.duration=10ns,25ns,50ns,100ns,500ns,1000ns,5000ns

If you are on Spring Boot 1.x, you'll have to escape the name as documented here.

As an aside, Spring already automatically records request duration with a metric named http_server_requests. It doesn't have Java class and Java method on it, but has method and status, and a few other things that you don't have here. You have an opportunity to override WebMvcTagsProvider to provide additional tags (including Java class and method):

@Bean
WebMvcTagsProvider requestTags() {
   return customProvider; // can be extended from DefaultWebMvcTagsProvider
}

There is a handler parameter to WebMvcTagsProvider#httpRequestTags that you can infer the type and method from.