Menu
I noticed that the Consumer configuration has two IDs. One is group.id (mandatory) and second one is consumer.id (not Mandatory).
What is the difference between these 2 IDs?
556
group.id specifies the name of the consumer group a Kafka consumer belongs to. When the Kafka consumer is constructed and group.id does not exist yet (i.e. there are no existing consumers that are part of the group), the consumer group will be created automatically. Note.
An optional identifier of a Kafka consumer (in a consumer group) that is passed to a Kafka broker with every request. The sole purpose of this is to be able to track the source of requests beyond just ip and port by allowing a logical application name to be included in Kafka logs and monitoring aggregates.
The group ID is very important to how different consumers "load balance" partitions. For example, if you have a topic with 10 partitions then two consumers with the same groupId will read from 5 partitions each. If you have two consumers with different group ids, both consumers will read from 10 partitions.
The kafka documentation describes the clientId as: Client-id is a logical grouping of clients with a meaningful name chosen by the client application. The tuple (user, client-id) defines a secure logical group of clients that share both user principal and client-id.
Consumers groups is a Kafka abstraction that enables supporting both point-to-point and publish/subscribe messaging. A consumer can join a consumer group (let us say group_1) by setting its group.id to group_1. Consumer groups is also a way of supporting parallel consumption of the data i.e. different consumers of the same consumer group consume data in parallel from different partitions.
In addition to group.id, each consumer also identifies itself to the Kafka broker using consumer.id. This is used by Kafka to identify the currently ACTIVE consumers of a particular consumer group.
Read this documentation for more details.
147
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
