Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

azure service bus not receiving all messages (only ~65%)

This is the closest previous question I could find: Azure Service Bus Subscription OnMessage not receiving messages.

The same thing happens to me too. When I change the name of the topic it works for a while again. Then that service bus topic is corrupt again. Only 65-71% of messages arriving. Doesn't help to delete the subscritption, nor the topic. The topic name seems to be polluted somehow after a while. It is really really bad, because I have no way of telling when the topic is corrupt, except that the system doesn't work like it should when messages don't arrive. Creating a new topic with new name randomly now and then seems like an utterly bad solution.

I'm testing it through a loop in one process, sending the messages, then a loop in another process, receiving and counting. With a new topic name it works perfectly. And I know I only have one listener to the subscription, and it's a peek lock, requires the message to be completed.

Anyone? How can I solve this?

UPDATE: There's a gotcha to be found here. I've had 1 subscription created and maintained a connection to it; 1 topic created, and the bus recreated 10 times, sending 100 msg every time. No msg lost. I have had 1 subscription created, and a new subscriptionclient created for every time the bus is recreated and 100 msg sent. Losing 50% of msgs. Seems like the topic is aware of the previous subscriptionclient and the msgs are dealt to the two?

NEW QUESTIONS: I'm trying to wrap my head around how to handle this. Can anyone confirm that a restart of the process, leading to creating a new subscription client with the same subscription name, would make the topic deal out the messages between the first and the second subscription client, even though the first is no longer there? Since I'm trying to handle faults by restarting my subscription module, i.e. going through the steps of checking if topic exists, if subscription exists, and then create subscription client, I'm struggling to understand how I could avoid the above described, and avoid messages being dealt out to the non existant subscriber also..

Suggestion for solution, atm: Keep track of old subscriptions, and if I have to restart process, create a new subscription? Leaves a window between process going down and new subscription created, where messages will be pumped out to the "dead" subscription only. These messages will be lost. But at least any messages after that will be received by the new subscription. Man.. this problem must have been dealt with before. I'm not doing it right. Would highly appreciate some guidance here.

SOLUTION: It's all about the right tool for the job. This situation calls for a Queue, not a Pub/Sub. Everything solved. I'm doing the same tests as above, but with queue instead, and of course, since it's decided clientside who receives a message, there is no problem with previous (dead) subscriptionclients taking messages from new ones. Only one queue client will be alive at a time, so there is only that one who can take msgs off the queue.

like image 997
jarchinio Avatar asked Oct 01 '15 08:10

jarchinio


1 Answers

SOLUTION: It's all about the right tool for the job. This situation calls for a Queue, not a Pub/Sub. Everything solved. I'm doing the same tests as above, but with queue instead, and of course, since it's decided clientside who receives a message, there is no problem with previous (dead) subscriptionclients taking messages from new ones. Only one queue client will be alive at a time, so there is only that one who can take messages off the queue.

like image 78
jarchinio Avatar answered Oct 21 '22 19:10

jarchinio