I recently set up a test Kafka cluster. I am running a consumer group listening on items and things seem to work. The name of consumer group is default
. What surprises me is that listing consumer groups gives me an empty list:
$ ./bin/kafka-consumer-groups.sh --zookeeper localhost:2181 --list
$
Also, explicitly querying the offsets doesn't yield anything:
$ ./bin/kafka-consumer-groups.sh --zookeeper localhost:2181 --describe --group default
No topic available for consumer group provided
GROUP, TOPIC, PARTITION, CURRENT OFFSET, LOG END OFFSET, LAG, OWNER
Do I need to manually create a consumer group using kafka-consumer-groups.sh —new-consumer
to be able to track its offsets?
The consumer group-id is mandatory, it plays a major role when it comes to scalable message consumption. To start a consumer group-id is mandatory.
Get the list of consumer groups for a topic. Use kafka-consumer-groups.sh to list all consumer groups. Note that the below command will list all the consumer groups for all topics managed by the cluster.
You can get the value of group.id for your kafka cluster by looking into $KAFKA_HOME/config/consumer. properties . There you can see the line #consumer group id . Use this value and your code will work.
Okay, I figured it out. I am using the new consumer/producer API. This means I need to use the —new-consumer
flag. Example:
$ ./bin/kafka-consumer-groups.sh --new-consumer --bootstrap-server localhost:9092 --list
default
$
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With