Spring Boot 2.3.2.RELEASE - Micrometer - KafkaMetrics - Failed to bind meter logs

Question

I am using /actuator/prometheus endpoint for its kafka_consumer_* metrics. Upgrading from Spring Boot 2.3.1.RELEASE to 2.3.2.RELEASE, is showing me a lot of these "extra" logs - whenever there is a failure to bind:

INFO io.micrometer.core.instrument.binder.kafka.KafkaMetrics.lambda$checkAndBindMetrics$1[173] -- Failed to bind meter: kafka.consumer.fetch.manager.[metric1]...However, this could happen and might be restored in the next refresh.
INFO io.micrometer.core.instrument.binder.kafka.KafkaMetrics.lambda$checkAndBindMetrics$1[173] -- Failed to bind meter: kafka.consumer.fetch.manager.[metric2]...However, this could happen and might be restored in the next refresh.
INFO io.micrometer.core.instrument.binder.kafka.KafkaMetrics.lambda$checkAndBindMetrics$1[173] -- Failed to bind meter: kafka.consumer.fetch.manager.[metric3]...However, this could happen and might be restored in the next refresh.
INFO io.micrometer.core.instrument.binder.kafka.KafkaMetrics.lambda$checkAndBindMetrics$1[173] -- Failed to bind meter: kafka.consumer.fetch.manager.[metric4]...However, this could happen and might be restored in the next refresh.

It is an INFO log level with the somewhat reassuring ending words of However, this could happen and might be restored in the next refresh., therefore, not meant to be alarming, but what is its purpose for showing it in this upgrade?

In the meantime, i have suppressed them(the extra logs) with: logging.level.io.micrometer.core.instrument.binder.kafka.KafkaMetrics=WARN

Answer 1

Had the same issue. I "fixed" mine by refining a MeterFilter that appears to have broken when upgrading from 1.3.2 to 1.5.4. See https://github.com/micrometer-metrics/micrometer/issues/2256 for more details. Best I could provide is that if you can reliably reproduce this locally then try break pointing on the if block in https://github.com/micrometer-metrics/micrometer/blob/master/micrometer-core/src/main/java/io/micrometer/core/instrument/binder/kafka/KafkaMetrics.java#L195-L204

In IntelliJ use "find class" of io.micrometer.core.instrument.binder.kafka.KafkaMetrics

e.g.: if (message != null && message.contains("Prometheus requires")) {

And then read what the actual message is. My error turned out to be:

Prometheus requires that all meters with the same name have the same set of tag keys. There is already an existing meter named 'kafka_consumer_fetch_manager_bytes_consumed_total' containing tag keys [client_id, env, instance, kafka_version, spring_id]. The meter you are attempting to register has keys [client_id, env, instance, kafka_version, spring_id, topic].