Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Kafka 10 kafka-consumer-groups.sh vs. Kafka 8 kafka-run-class.sh of ConsumerOffsetChecker

I need to name my consumers in Kafka 10 the same as I did in Kafka 8, since I have scripts that sniff out and use this info further.

Apparently the default naming of the consumer.id has changed (and it also now shows the client.id separately). I didn't see this mentioned or compared in the docs.

In Kafka 10 terms, what is the equivalent I need to do to have it generate the same '<group><host>...' consumer and/or client id format as in Kafka 8 (instead of the 'consumer-#-<guid>' format seen below)?

Kafka 8:

/usr/local/kafka-0.8/bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --zkconnect localhost:2181 --group optimizer-group --topic optimizer-default-topic

Group           Topic                          Pid Offset          logSize         Lag             Owner
optimizer-group optimizer-default-topic        0   17059718        17059725        7               my-group_myhost.com-1497048808577-1575fa11-0
...

Kafka 10:

/usr/local/kafka/bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group optimizer-group --describe

TOPIC                          PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG        CONSUMER-ID                                       HOST                           CLIENT-ID
eddude-error-topic             0          0               0               0          consumer-2-e839517b-a150-498d-9054-31ec7a655936   /10.xxx.xx.xxx                 consumer-2
...
like image 649
redgiant Avatar asked Jul 31 '17 19:07

redgiant


1 Answers

Default consumer id naming convention in 0.8 is hostname-timestamp-[most significant 8 bits of UUID] although you could set consumer.id to override it.

However, default consumer id in 0.10 changes to [client.id]-[UUID] and it seems there is no way to specify it directly. The only way is to set client.id to indirectly set the consumer id.

like image 182
amethystic Avatar answered Oct 16 '22 15:10

amethystic