How to set group name when consuming messages in kafka using command line?

Question

Any idea how to set group name when consuming messages in kafka using command line.

I tried with the following command :

bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic nil_RF2_P2 --from-beginning --config group.id=test1
'config' is not a recognized option

The goal is to find the offset of consumed messages with the following command:

bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --zookeeper localhost:2181 --group test1

Can somebody help in this regards!!

Thanks in advance !!

Answer 1

The simplest solution is:

bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic nil_RF2_P2 --from-beginning --consumer-property group.id=test1

In case you specify the flag --from-beginning just remember that the consumer group should not have consumed any records in the past or else your consumer will start consuming from the earliest not consumed record by the specified group (and not from the actual beginning, as you may incorrectly assume).

Answer 2

Got the answer to change the groupname from command prompt!!

steps:

1. create a new consumer.properties file, say consumer1.properties.
2. change group.id=<Give a new group name> in the consumer1.properties.
3. bin/kafka-console-consumer.sh --new-consumer --bootstrap-server localhost:9092 --topic topicname --from-beginning --consumer.config config/consumer1.properties --delete-consumer-offsets

Answer 3

if you want change group id without lost offset of record you have get offset manually of current Group.id and set to new run consumer that have new id. if don't have any control to get offset in consumer instance you can run this command.

/bin/kafka-consumer-groups.sh --new-consumer --bootstrap-server <ip_address>:<Broker_port>  --group Group_name --describe

and then you can seek data from specific offset. pay attention you should call seek after call poll، Assign command not work. also you can see my sample of code in github

Example here