Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What is the most efficient way to delete/expire all messages in a Apache Pulsar topic?

I'm trying to figure out what would be the best way to remove all the messages from a Pulsar topic (either logically or physically), so that they are no longer consumable by subscriptions?

I know we can simply do $ pulsar-admin persistent delete persistent://tenant/namespace/topic.

But, this solution has some drawbacks: it removes the topic completely (so we have to recreate it later) then there should be no active client connected to it (i.e: subscriptions or producers).

Alternatively is there a way to programmatically make all messages between two MessageId unavailable to the subscriptions ?

Thanks

like image 615
fhussonnois Avatar asked Sep 01 '25 20:09

fhussonnois


1 Answers

There are a couple of options you can choose from.

  • You can use topics skip to skip N messages for a specific subscription of a given topic. https://pulsar.apache.org/docs/en/admin-api-persistent-topics/#skip-messages

  • You can use topics skip-all to skip all the old messages for a specific subscription for a given topic. https://pulsar.apache.org/docs/en/admin-api-persistent-topics/#skip-all-messages

  • You can use topics clear-backlog to clear the backlog of a specific subscription. It is same as topics skip-all.

  • You can also use topics reset-cursor to move the subscription cursor to a specific message id or a timestamp back.

like image 107
Sijie Guo Avatar answered Sep 10 '25 07:09

Sijie Guo