Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to delete multiple topics in Apache Kafka

Assuming that I have a number of topics with the same prefix, e.g:

giorgos-topic1 giorgos-topic2 giorgos-topic3 ... 

The command used for deleting a single topic (say giorgos-topic1) is the following:

./bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic giorgos-topic1 

Is it possible to delete multiple topics using a single command and possibly a regular expression/wildcard (e.g. giorgos-*) instead of typing all the topic names that need to be deleted one by one?

like image 561
Giorgos Myrianthous Avatar asked Feb 05 '18 10:02

Giorgos Myrianthous


People also ask

How do I remove all topics from Kafka?

Use kafka-topics.sh --describe to list details for remove-me topic. Note that the broker 100 is the leader for remove-me topic. Stop the broker 100 and start another with broker ID 200 . Use kafka-topics.sh --delete to delete remove-me topic.

Is it possible to delete a Kafka topic?

In the last few versions of Apache's Kafka, deleting a topic is fairly easy. You just need to set one property in the configuration to 'true', and just issue a command to delete a topic. It'll be deleted in no time.

How do I enable topic deletion in Kafka?

Topic deletion in Kafka removes data and the operation is not reversible (i.e. there is no "undelete" operation). Hence, there is a valid use-case to keep the flag "delete. topic. enable" as "false" in server.

Which is the correct syntax for eliminating the topic with respect to Kafka?

Remove the topic folder from ZooKeeper using: rmr /brokers/topics/yourtopic.


Video Answer


2 Answers

Yes you can use regex-like expressions when deleting topics with the kafka-topics.sh tool:

For example, to delete all topics starting with giorgos-:

./bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic 'giorgos-.*' 

Using the Admin APIs, you can also delete several topics at once, see AdminClient.deleteTopics

like image 65
Mickael Maison Avatar answered Sep 30 '22 20:09

Mickael Maison


In cases where regex is not possible we can use a comma seperated list of topic names for the deletion of topics.

kafka-topics.sh --zookeeper 10.0.0.160:2181 --delete --topic giorgos-topic1,giorgos-topic2,giorgos-topic3,... 
like image 27
Aman Saurav Avatar answered Sep 30 '22 20:09

Aman Saurav