Menu
I'm looking to have a master topic (with log retention 7 days) and several smaller topics with a filtered corpus with a smaller log retention (2 days). Is this possible?
NOTE: I'm using Kafka v0.10.1.1.
647
log.retention.hours The most common configuration for how long Kafka will retain messages is by time. The default is specified in the configuration file using the log. retention. hours parameter, and it is set to 168 hours, the equivalent of one week.
We can notice here that the default retention time is seven days.
With the kafka-configs command you can inspect any of the topic configs, along with that you can alter them too. So I'm going to alter the retention.ms and set it to 30 minutes (30 minutes * 60 seconds * 1000 milliseconds = 1,800,000 milliseconds).
If you want to view the configurations for all topic Either you can view these properties log. retention. hours or log.retention.ms in server. properties in kafka config directory.
log.retention.ms, whose default value is 7 days, is at the global level for all topics, whereas you could override it using a topic-level config retention.ms when creating the topic as below:
bin/kafka-topics.sh --create --zookeeper localhost:2181 --topic test
--partitions 1 --replication-factor 1 --config retention.ms=172800000
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
