Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Difference between group id, Client id and id in KafkaListener Spring Boot

I am starting to work with Spring Boot 2 and Spring Kafka, I don't quite understand what's the difference between group id, Client id, and id in in KafkaListener interface.
I know group ID is used by Kafka broker to manage multiple Consumer in the same group, but what about the others? what advantage do I get by setting them? where can I see the effect of setting or not setting them?

Based on their java doc :

/**
     * The unique identifier of the container managing for this endpoint.
     * <p>If none is specified an auto-generated one is provided.
     * @return the {@code id} for the container managing for this endpoint.
     * @see org.springframework.kafka.config.KafkaListenerEndpointRegistry#getListenerContainer(String)
     */
String id() default "";

/**
 * Override the {@code group.id} property for the consumer factory with this value
 * for this listener only.
 * @return the group id.
 * @since 1.3
 */
String groupId() default "";

/**
 * When provided, overrides the client id property in the consumer factory
 * configuration. A suffix ('-n') is added for each container instance to ensure
 * uniqueness when concurrency is used.
 * @return the client id prefix.
 * @since 2.1.1
 */
String clientIdPrefix() default "";
like image 225
Am1rr3zA Avatar asked Mar 16 '18 15:03

Am1rr3zA


People also ask

What is id in @KafkaListener?

id. java.lang.String id. The unique identifier of the container for this listener. If none is specified an auto-generated id is used. Note: When provided, this value will override the group id property in the consumer factory configuration, unless idIsGroup() is set to false or groupId() is provided.

What is the use of group ID in Kafka consumer?

The consumer group ID is the unique identifier for the consumer group within the cluster. This is part of the consumer configuration for the application client. Active members shows the number of consumers in the group that are assigned to a topic partition in the Kafka instance.

Is group ID mandatory for Kafka consumer?

The consumer group-id is mandatory, it plays a major role when it comes to scalable message consumption. To start a consumer group-id is mandatory.

How does Group ID work in Kafka?

Consumers can join a group by using the same group.id. The maximum parallelism of a group is that the number of consumers in the group ← no of partitions. Kafka assigns the partitions of a topic to the consumer in a group, so that each partition is consumed by exactly one consumer in the group.


1 Answers

Your groupId understanding is correct.

The id is like a bean name in Spring Framework. However this one is used in the KafkaListenerEndpointRegistry boundaries only. So, if you need a lifecycle control over the particular KafkaListenerContainer created for the mentioned @KafkaListener, you need to inject KafkaListenerEndpointRegistry and use the mentioned getListenerContainer() for the appropriate id.

The clientIdPrefix is reflection of exact client.id property of the Kafka Consumer:

An id string to pass to the server when making requests. The purpose of this is to be able to track the source of requests beyond just ip/port by allowing a logical application name to be included in server-side request logging.

like image 123
Artem Bilan Avatar answered Sep 19 '22 15:09

Artem Bilan