Best practices for (over)using Azure queues

Question

I'm in the early phases of designing an Azure-based application. One of the things that attracts me to Azure is the scalability, given the variability of the demand I'm likely to expect. As such I'm trying to keep things loosely coupled so I can add instances when I need to.

The recommendations I've seen for architecting an application for Azure include keeping web role logic to a minimum, and having processing done in worker roles, using queues to communicate and some sort of back-end store like SQL Azure or Azure Tables. This seems like a good idea to me as I can scale up either or both parts of the application without any issue. However I'm curious if there are any best practices (or if anyone has any experiences) for when it's best to just have the web role talk directly to the data store vs. sending data by the queue?

I'm thinking of the case where I have a simple insert to do from the web role - while I could set this up as a message, send it on the queue, and have a worker role pick it up and do the insert, it seems like a lot of double-handling. However I also appreciate that it may be the case that this is better in the long run, in case the web role gets overwhelmed or more complex logic ends up being required for the insert.

I realize this might be a case where the answer is "it depends entirely on the situation, check your perf metrics" - but if anyone has any thoughts I'd be very appreciative!

Answer 1

Here's my metaphor, do what you will with it

Imagine you're entering a nightclub, that borders on a dodgy area, but is alright once you're inside.

The management employ some big meaty bouncers on the door to sort out the riff raff. If you're an idiot, you're not getting in. Extend the metaphor as much as you like here.

If you're OK, then they let you in the door, and you join, yes, The Queue to pay at the Box Office to enter the actual club.

Depending if the football's on or something, you might want some more bouncers on the door, but this can be independent of the Box Office staff. Busy night, you might open another window to get the money in quicker, but what you're probably not going to do get the bouncers to handle cash. They've got other things to do with their hands.

So:

• Bouncers - web roles. Handle the incoming traffic, repel invalid requests and add the valid requests to:
• The Queue - the Queue!
• Box Office - worker roles, performing a different role to the webrole

So, there's no reason why your web roles can't do the box office role, but it's probably best not in the long-run

that's my metaphor

Toby