Create multiple consumers in Kafka in command line

Question

I'm new in Kafka. When I was running the quick start example in command line, I found I can't create multiple consumers in command line.

Condition:

I built a topic named test with 3 partitions, and I also built a producer on this topic.

Then I wanted to create two different consumers sharing a same consumer-group named test1 on this topic.

I ran the command like below twice:

   bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --group test1

The first one worked but when I ran the second time the first one would disconnect and the second one worked.

So how can I create two or more consumers in a same consumer group in command line?

    WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1146)

Answer 1

Besides using --consumer.config option like the secfree's answer, you can also use

--consumer-property group.id=your_group

option to specify a group name without editing the config file.

Answer 2

1. Default, kafka-console-consumer.sh will create a random group.
2. If you want to specify the group name, you can:
   1. Add group.id=group_name to a local file filename
   2. Use --consumer.config filename option of kafka-console-consumer.sh to set group
3. You can check your groups at zookeeper's /consumers/ directory.

Refer: kafka/core/src/main/scala/kafka/tools/ConsoleConsumer.scala