Best practice for ServiceBus message versioning

Question

I am setting up a system where we will transport messages between several internal services on ServiceBus Topics. The messages will hold serialized objects. The model objects are defined as quite complex trees of classes. This means it is not practical to maintain duplet versions of the model structures in the code.

We expect the model structure to change so I have exposed the model version as a property on the brokered message.

What is the best way to handle the transition when we need to upgrade the model version?

I don't think we will really need to support two parallell model versions. But I am concerned we don't loose messages during the transition. I assume it is a good strategy to upgrade the sending services first and let all subscribers continue to process messages. When all messages of the previous version are processed, then it is time to upgrade the subscribing services.

What is the best mechanism for skipping messages with a new version that the listening service is currently not handling?

1. I know I could go back to the old school and define parallell model versions by using schemas for json or xml, thus making it possible for the listening service to handle parallell versions. But that would be cumbersome, so I really want to avoid that.

2. I noticed the BrokeredMessage has a Defer method. Would that be useful? It looked promising until I realized the messages will be "moved" from the live queue into a separate state where they need to be pulled by referencing them by key. Not practical.

3. Is it possible to postpone the message by modifying delivery time? A couple of minutes would be fine. If the same service is still running by that time it can be postponed once again. (A working code example would be appreciated!)

4. Do I need to create separate subscriptions based on model version? So far we allow different message types to travel on the same topic so that would call for some redesign.

Answer 1

As a rule of thumb: upgrades on a live system are difficult. The easiest option that minimises risks of system downtime is:

• add next message version support to the current code base
• run two message versions concurrently
• ensure all versions are supported and system runs without a problem
• remove previous version

Answer 2

I have been looking at something similar but am yet to implement it so can't provide full guidance, but to answer your question on #3... I have messages which have a flag to re-queue the message to run again, e.g. to get a process to run every 5 minutes.

So during the process I extract the object from the BrokeredMessage:

  var myObject = receivedMessage.GetBody<MyModel>();  

I then complete that message to remove it from the queue and create a new BrokeredMessage based on that object and you can then set the ScheduledEnqueueTimeUtc field to something in the future.

 BrokeredMessage brokeredMsg = new BrokeredMessage(myObject);
 brokeredMsg.ScheduledEnqueueTimeUtc = DateTime.UtcNow.AddMinutes(5);
 Client.Send(brokeredMsg);

So if you only want to process one model version at a time, you could assign a version number to your Model and code something in to your processor to look for a certain model number. If the model is higher, then re-queue it for a future time (Until you have updated your code). If it is lower (a missed message), then perhaps have some exception handling.

Answer 3

Use custom MessageProperty on message, say Version.

Under SB topic - create new subscription that will accept only messages with new version (using Rules), and modify existing subscription(s) to NOT accept new version messages.

Then you can upgrade senders - new messages will be stored only in new 'temporary' subscription.

After that, you upgrade listeners, change rules on subscriptions (remove version rule from 'main' subscription, disable receive on temporary subscription).

And now you have choice:

• using any tool read messages from temporary subscription and write them back to topic - they will arrive to upgraded listeners.
• temporarily start one more listener that will read temporary subscription and process all messages in it
• other ways, specific to your architecture