Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to see the retention for a particular topic in kafka

I want to see the retention period set for a particular topic. Is there any command? I tried with

bin/kafka-topics.sh --zookeeper hostname:2181 --alter --config retention.ms=172800000 --topic <topic_name> 

for deleting 2 days old data. But I want to see how many days are set in the retention period for all the topics.

like image 295
manoj Avatar asked Dec 14 '16 05:12

manoj


People also ask

What is retention in Kafka topic?

log.retention.hours The most common configuration for how long Kafka will retain messages is by time. The default is specified in the configuration file using the log. retention. hours parameter, and it is set to 168 hours, the equivalent of one week.

What is the default retention period for a Kafka topic?

If the log retention is set to five days, then the published message is available for consumption five days after it is published. After that time, the message will be de discarded to free up space. The performance of Kafka is not affected by the data size of messages, so retaining lots of data is not a problem.

How long do messages stay in Kafka topic?

The Kafka cluster retains all published messages—whether or not they have been consumed—for a configurable period of time. For example if the log retention is set to two days, then for the two days after a message is published it is available for consumption, after which it will be discarded to free up space.


1 Answers

If you have altered a topic and want to view the topic configuration the following command will be helpful

kafka-topics.sh --zookeeper localhost:2181 --describe --topics-with-overrides 

This will describe only the topics along with configurations that have configurations set that differ from the cluster defaults.

If you want to view the configurations for all topic Either you can view these properties log.retention.hours or log.retention.ms in server.properties in kafka config directory.

like image 124
Abhimanyu Avatar answered Sep 21 '22 16:09

Abhimanyu