How to remove a Kafka consumer group from a specific topic?

Question

I changed the consumer-group-id of my web service listening to a Kafka topic. Now, the old group id is still registered to the topic, but there is no consumer with that group id. Therefore, it is lagging. How can I remove a specific consumer group from a specific topic?

I tried this:

kafka-consumer-groups --bootstrap-server kafka01.myserver.com:9092 --topic notification-topic --delete --group old-consumer-group --execute

But it returns: "The consumer does not support topic-specific offset deletion from a consumer group."

Should I remove the consumer group totally? I use the same group id listening to other topics, are they going to be affected?

Answer 1

Starting with Kafka 2.4.0 it is possible to delete individual consumer group id offsets from a topic.

The call is very close to what you have already tried, but requires --delete-offsets instead of --delete:

./kafka-consumer-groups.sh \
  --bootstrap-server <bootstrap-server-url> \
  --delete-offsets \
  --group <my-group> \
  --topic <topic-name>

To date this is not reflected in the official documentation (Kafka 2.7.0 and below). However, it is described in a confluence document of improvement proposal KIP-496 that has been implemented and released.

Answer 2

You cannot remove a specific group on its own, nor a topic from a group.

Consumer groups remain available based on the retention you've defined for the offsets topic on the broker. There is no performance impact by having lagging groups that are not being consumed.

Note: Every time you use kafka-console-consumer, a brand new group is created.