Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What is the command for getting the list of Consumers of a particular topic in Kafka

Tags:

apache-kafka

I know we can get the list of consumer groups and then pass any one of he consumer group to a particular topic to determine the consumers offset.

Step 1: Use the following command to get list of consumer groups

kafka-consumer-groups --zookeeper <Server-URL> --list

From the list of consumers obtained, pick a consumer group Step 2: Get that particular consumer offset for a particular topic

kafka-consumer-offset-checker --zookeeper <Server-URL> --topic <topic-name> --group <group-name>

Same thing for bootstrap server.

But in a single step i need a command to get list of consumers for a particular topic.

like image 989
Sujan Davangere Sunil Avatar asked Oct 08 '18 19:10

Sujan Davangere Sunil


People also ask

How can I get data from Kafka consumer?

Consumers and consumer groups Since the data record is in some partition, we can say that consumers simply read from partitions. Kafka consumers use a pull model to consume data. This means that a consumer periodically sends a request to a Kafka broker in order to fetch data from it. This is called a FetchRequest.

How do I list all groups in Kafka?

A '-list' command is used to list the number of consumer groups available in the Kafka Cluster. The command is used as: 'kafka-consumer-groups. bat -bootstrap-server localhost:9092 -list'.

How does Kafka keep track of consumers?

Kafka brokers use an internal topic named __consumer_offsets that keeps track of what messages a given consumer group last successfully processed. As we know, each message in a Kafka topic has a partition ID and an offset ID attached to it.


1 Answers

As far as I know there is no direct command to do that:

One workaround for this could be to list all consumer groups

~ $ $KAFKA_DIR/bin/kafka-consumer-groups.sh -bootstrap-server localhost:9092 -list
consumer-group-1
consumer-group-2
consumer-group-3

Then describe the consumer group using the below command which will show the topic.

~ $ $KAFKA_DIR/bin/kafka-consumer-groups.sh --describe --bootstrap-server 0.0.0.0:9092 --group consumer-group-1

TOPIC                        PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID                                     HOST            CLIENT-ID
topic-1                      0          -               0               -               consumer-1-9bf71d08-bcf8-4eb1-adb8-91589fd034b3 /11.2.9.159     consumer-1
like image 65
Santosh Sindham Avatar answered Sep 22 '22 21:09

Santosh Sindham