Kafka: How to get last modified time for a topic i.e. last message added to any partition of the topic

Question

Our use case is to delete stale/unused topics from kafka i.e. if a topic (on all partitions) doesn't have any new message in last 7 days then we would consider it as stale/unused and delete it.

Many google results suggested to add timestamp to messages and then parse it. For new topics & messages that soultion would work but our existing topics & messages doesn't have any timestamp in them.

How can I get this working ?.

Answer 1

kafka.api.OffsetRequest.LatestTime() will return the latest message added to a queue. You can use the Simple Consumer API to determine which offset to read from.

For more details take look at the wiki page