Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Does a Kafka Streams app commit offets for a topic used to fill a Global KTable?

I'm observing my Kafka Streams app reporting consumer lag for topics used to fill Global KTables. Is it correct that offsets are not committed for such topics?

This would make sense as the topic is read from the beginning on every startup, so keeping track of the offest in the consumer would be sufficient.

It would however be useful to know for monitoring to exclude such consumer topic pairs.

like image 362
Pieter Hameete Avatar asked Sep 20 '25 00:09

Pieter Hameete


1 Answers

Correct, offsets are not committed for "global topics" -- the main reason is, that all KafkaStreams instances read all partitions and committing multiple offsets is not possible.

You can still access the "global consumer" metrics, that also report their local lag.

like image 56
Matthias J. Sax Avatar answered Sep 23 '25 12:09

Matthias J. Sax