Menu
My disk space is running out, after syncing 2 millions message in one of my Kafka topic. I even deleted the topic but still the space is somehow seems consumed. Any help of link would be appreciated.
849
Furthermore, Kafka uses heap space very carefully and does not require setting heap sizes more than 6 GB. This will result in a file system cache of up to 28-30 GB on a 32 GB machine. You need sufficient memory to buffer active readers and writers.
The Disk Usage metric shows the percentage of disk space being used by Kafka. Kafka will gradually use more and more disk space in order to store messages for each topic.
Often the data will stay there and get deleted once a specified retention period or max size/data limit has been reached.
Use the below command to know how much disk space consumed by each topic
du -sh /tmp/kafka-logs/*
Another way to determine the size of the topic to send metrics, such as graphite. Then the size can be found as follows:
kafka.log.Log.<topic_name>-<partition_number>-Size.value
25
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
