Kafka console consumer get partition

Question

I'm using Kafka console consumer to consume messages from the topic with several partitions:

kafka-console-consumer.bat --bootstrap-server localhost:9092 --from-beginning --topic events

But it prints only message body. Is there any way to print record metadata or partition number as well? Cause I want to know where the message сonsumed from.

I've explored console consumer doc http://documentation.kamanja.org/_static/command-ref/kafka-console-consumer.pdf but didn't find any related properties.

UPDATED:

So, as I see the only solution is to override DefaultMessageFormatter.class (we can set it by running kafka console consumer with --formatter property) and add custom logic that prints record metadata in the #writeTo(..) method.

Answer 1

Consider using a more powerful Kafka command line consumer like kafkacat https://github.com/edenhill/kafkacat/blob/master/README.md

For example, the following command will print the topic, partition, offset and message payload/value for each message consumed:

kafkacat -b <broker> -C -t <topic> -f '%t %p @ %o: %s\n'

Answer 2

For kafka 9 there is nothing out-of-the-box that can print you that info.

According to the code, message formatter gets key and value only.

try {
    formatter.writeTo(msg.key, msg.value, System.out)
}

--property print.key=true allows to print message key.

In kafka 10 there is one more useful param print.timestamp

Answer 3

I found I was successfully able to subscribe to a topic and output the key and partition using this --properties flag like so:

kafka-console-consumer --topic <some-topic> --bootstrap-server localhost:9092 --from-beginning  --property print.key=true --property print.partition=true   

It gave me an output like this:

Partition:0 "3" {"id": 2}