Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Kafka consumer stuck in (Re-)joining group

Whats the default behavior of kafka (version 0.10) consumer if it tries to rejoin the consumer group. I am using a single consumer for a consumer group but it seems like it got struck at rejoining. After each 10 min it print following line in consumer logs.

2016-08-11 13:54:53,803 INFO o.a.k.c.c.i.ConsumerCoordinator [pool-5-thread-1] ****Revoking previously assigned partitions**** [] for group image-consumer-group

2016-08-11 13:54:53,803 INFO o.a.k.c.c.i.AbstractCoordinator [pool-5-thread-1] (Re-)joining group image-consumer-group

2016-08-11 14:04:53,992 INFO o.a.k.c.c.i.AbstractCoordinator [pool-5-thread-1] Marking the coordinator dead for group image-consumer-group

2016-08-11 14:04:54,095 INFO o.a.k.c.c.i.AbstractCoordinator [pool-5-thread-1] Discovered coordinator for group image-consumer-group.

2016-08-11 14:04:54,096 INFO o.a.k.c.c.i.AbstractCoordinator [pool-5-thread-1] (Re-)joining group image-consumer-group

Restart consumer application is not helping.

like image 359
Bharat Bhagat Avatar asked Aug 11 '16 08:08

Bharat Bhagat


1 Answers

If you're gonna to have only one consumer instance in a group, then use the consumer with manual assignment strategy. (Simple Consumer).

Manual topic assignment does not use the consumer's group management functionality so heart beats are not required.

like image 143
Kamal Chandraprakash Avatar answered Nov 16 '22 12:11

Kamal Chandraprakash