Delete unused kafka consumer group

Question

I'm using Apache Kafka 0.10 with a compacted topic as a distributed cache synch mechanism. When the application starts up it generates an instance specific consumer group id. As instances are added and removed for horizontal scalability, obviously we get a large number of group ids that should never be used again.

I'm sure that this is the perfect use case for KStreams and KTables, but I am trying to do this myself for intellectual reasons as well as that the KStreams and KTables are defined as alpha quality in 0.10.

Is there a Kafka API call that I can use that could delete an existing consumer group, knowing that it should never be used again?

Since Zookeeper is not maintaining consumer offsets in version 0.10, Is there a way delete the consumer group using Kafka?

Answer 1

Since Kafka 0.9, an internal topic is used to store committed offsets. You can configure how long those offsets should be kept via offsets.retention.minutes. (See also offsets.retention.check.interval.ms).

Answer 2

It's possible with CLI in Kafka

./bin/kafka-consumer-groups \
    --bootstrap-server <bootstrap_server(s)> \
    --topic <topic_name> \
    --delete \
    --group <consumer_group_name> 

kafka-consumer-groups should be available in Kafka installation home.