Menu
Is there a way to delete all the data from a topic or delete the topic before every run?
Can I modify the KafkaConfig.scala file to change the logRetentionHours property? Is there a way the messages gets deleted as soon as the consumer reads it?
I am using producers to fetch the data from somewhere and sending the data to a particular topic where a consumer consumes, can I delete all the data from that topic on every run? I want only new data every time in the topic. Is there a way to reinitialize the topic somehow?
910
1. stop zookeeper & Kafka server, 2. then go to 'kafka-logs' folder , there you will see list of kafka topic folders, delete folder with topic name 3. go to 'zookeeper-data' folder , delete data inside that.
To delete all the messages from a Kafka topic, you need to change the retention time of that topic the default retention time of Kafka topic is 168 hours, i.e. 7 days. In this case, you have to change the retention time to 1 second, after which the messages from the topic will be deleted.
Connect to each broker (from step 1), and delete the topic data folder, e.g. rm -rf /tmp/kafka-logs/MyTopic-0 . Repeat for other partitions, and all replicas. Delete the topic metadata: zkCli.sh then rmr /brokers/MyTopic. Start the Apache Kafka daemon for each stopped machine.
As I mentioned here Purge Kafka Queue:
Tested in Kafka 0.8.2, for the quick-start example: First, Add one line to server.properties file under config folder:
delete.topic.enable=true then, you can run this command:
bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic test If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
