Menu
Good afternoon.
We're using Service Bus Topics as the engine for a pub/sub system. Our logic involves our C# services hooking up to a topic with a subscription. We remove $Default (TrueFilter) and set AutoDeleteOnIdle to 5 minutes.
As other parts of the system need things, they tell our C# service, "I need this." The C# service then adds new rules (usually CorrelationFilter).
As those same parts of the system no longer need things, they tell our C# service, "I don't need this any more." The C# service then removes the corresponding rules.
As such, it is possible for a Topic Subscription to still be connected (complete with a SubscriptionClient object) but to have no rules at all.
Issue
The subscriptions "disappear" and I couldn't put my finger on why. After all, I have an active subscription with a SubscriptionClient instance and a callback function hooked in.
Then, when I go to take an action with my SubscriptionClient object, it throws a MessagingEntityNotFoundException.
It seemed to me as though Service Bus was randomly and arbitrarily losing or deleting my subscriptions.
Service Bus's Definition of "Idle"
My understanding is that the subscription is not "idle" so long as there is an active connection to the subscription (in my case, via a SubscriptionClient instance). Even if there are no messages of interest coming through, it's still not idle and thus still not deleted. If a message comes a day later, the SubscriptionClient instance will receive it.
This has been my experience with other parts of the system that don't dynamically add/remove rules. It works quite well.
But then I started wondering:
In spite of having a connection to the subscription, does Service Bus see my subscription as being idle since it has no rules and thus couldn't possibly receive messages? And would Service Bus then follow the AutoDeleteOnIdle property and delete it?
If the above is true then would adding a FalseFilter as the $Default keep the subscription alive?
Any insight and help would be most appreciated.
Many thanks - Shaun
Update
I made a rudimentary test in a WinForms application and something seems to be fundamentally wrong with Service Bus. Or, at least, our instance of Service Bus.
I have 17 Topics, as follows:
notify-root forwards all messages it receives to the other notify topics. We do this for sharding.
So...
I created 3 subscriptions on each of the 0 through f topics:
I created a SubscriptionClient instance for each of these 48 subscriptions and kept the instances in memory. I also used the OnMessageAsync method to pass in a callback that tells me when a message gets written.
Each subscription's AutoDeleteOnIdle is set to TimeSpan.FromMinutes(5).
I iterated through my topics, subscriptions and rules every 1 minute to see when subscriptions and rules appear and disappear.
Finally, I published a message every 1 minute on notify-root.
Each one of the 16 subscriptions with the TrueFilter received a message every one minute and displayed it on my WinForms app as expected.
At the 5 minute mark, Service Bus automatically deleted all of my subscriptions for topics ending in 0, 1, 2, 4, 5, 6, B, C and F.
Service Bus deleted these subscriptions in spite of the fact that they are not idle and actively received messages less than 1 minute before being deleted.
Some 30 minutes later, topics ending in 3, 7, 8, 9, A and E were still receiving messages with no signs of having their subscriptions deleted.
Also, Service Bus did not delete some of the Subscriptions that either weren't receiving messages due to no rules or having a FalseFilter. The SubscriptionClient instances did have a callback hooked up via OnMessageAsync. It's noteworthy that the ones that did not get deleted were in the same topic as the TrueFilter subscriptions that also did not get deleted.
It seems like it's deleting subscriptions from some topics in spite of having activity but not others.
I repeated the test and it was the exact same topics that had their subscriptions deleted (in spite of having activity).
Once I stopped publishing, Service Bus deleted the remaining subscriptions after 5 minutes (as expected).
I will repeat the test with a different set of 17 topics.
374
A topic subscription resembles a virtual queue that receives copies of the messages that are sent to the topic. Consumers receive messages from a subscription identically to the way they receive messages from a queue.
A queue allows processing of a message by a single consumer. In contrast to queues, topics and subscriptions provide a one-to-many form of communication in a publish and subscribe pattern. It's useful for scaling to large numbers of recipients.
As it turns out, AutoDeleteOnIdle works exactly the way I thought it did.
As long as the Topic Subscription has a connection, it shouldn't be deleted. It doesn't matter whether you have no rules, a FalseFilter or no messages getting published. As long as you have an active connection and you are currently trying to get messages using OnMessage(), OnMessageAsync(), Receive() or ReceiveAsync() then the subscription is not idle and will not be deleted.
However, our Service Bus Topic Subscriptions were still disappearing from some of the Topics. They were still being deleted at the AutoDeleteOnIdle time. I was never able to reproduce the issue on my dev box.
We've had the issue in our Production environment for the last few months and it has only gotten worse. We assumed we were doing something wrong.
As it so happens, Microsoft confirmed to me today that their West US 2 region has this issue. No other regions are affected. MS has not confirmed why it's happening, for how long it has been happening or how long it will take to rectify.
This is the last thing I would have imagined. I assumed I was doing something wrong.
Most troubling is how long this issue existed without being discovered by Microsoft.
Hopefully it will be mended soon.
If someone else encounters Service Bus subscriptions disappearing or being deleted in spite of activity, I'd recommend building a test as I did. If the test doesn't behave as expected, reach out to Microsoft.
170
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
