How do I delete a Kafka Consumer Group to reset offsets?

Question

I want to delete a Kakfa consumer group so that when the application creates a consumer and subscribes to a topic it can start at the beginning of the topic data.

This is with a single node development vm using the current latest Confluent Platform 3.1.2 which uses Kafka 0.10.1.1.

I try the normal syntax:

sudo /usr/bin/kafka-consumer-groups --new-consumer --bootstrap-server localhost:9092 --delete --group my_consumer_group

I get the error:

Option [delete] is only valid with [zookeeper]. Note that there's no need to delete group metadata for the new consumer as the group is deleted when the last committed offset for that group expires.

If I try the zookeeper variant:

sudo /usr/bin/kafka-consumer-groups --zookeeper localhost:2181 --delete --group my_consumer_group

I get:

Delete for group my_consumer_group failed because group does not exist.

If I list using the "old" consumer, I do not see my consumer group (or any other consumer groups)

sudo /usr/bin/kafka-consumer-groups --zookeeper localhost:2181 --list

If I list using the "new" consumer, I can see my consumer group but apparently I can't delete it:

sudo /usr/bin/kafka-consumer-groups --new-consumer --bootstrap-server localhost:9092 --list

Answer 1

This can be done with Kafka 1.1.x. From the documentation:

bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --delete --group my-group --group my-other-group