Topic consumption rate

Question

I want to know the rate at which a topic is being consumed. There are Messages in /sec in the Kafka-manager metric, but there is no Messages out /sec I want. What should I do?

Answer 1

Kafka brokers do not have Messages out /sec metrics because when batches are sent to consumers they are not decompressed hence brokers do not know the exact raw message count.

Kafka brokers have bytes out / sec metrics per topic. That usually gives a good idea of consuemr activity. If you know the average message size, you can deduce a rough estimate of the messages out / sec value. See http://kafka.apache.org/documentation/#monitoring

On the other hand, Kafka Consumers do expose received messages rates: http://kafka.apache.org/documentation/#consumer_fetch_monitoring