Programmatically spawn new instances of a worker role

Question

Is it possible to programmatically spawn new instances of a worker role? I have a worker role that's dependent on some Service Bus queue(s) where it grabs the messages and puts the data into Table Storage, SQL Database and sends out notifications. I need to be able to add more instances depending on the SB queue sizes as they can gradually receive larger amounts of messages in the future. Is it possible for me to control this from my code? And in that case how?

Answer 1

There are multiple ways to achieve this functionality:

1) Windows Azure supports auto-scaling based on the amount of messages in Service Bus queues. You can simply configure the scaling to work from inside Azure Management Portal. It is currently free while in beta, and price has not yet been announced for when it is live.

2) There is an open-source API called WASABi from Microsoft's Patterns and Practices group. You'll need a place to host it, but it'll communicate with Azure Service Management API on your behalf and provide you with the framework for implementing your own auto-scaling based on Service Bus queues. Using it, you will mostly be instrumenting some configuration XML files

3) If you don't mind using 3rd party products & services, there is a service called AzureWatch (which I'm affiliated with) that will give you a way to do the scaling based on Service Bus queues just like Azure Portal, but it'll have a bunch of other choices, features and services.

4) You can use Service Management API directly to change instance counts, but I really don't recommend using it directly with API's such as WASABi being around.

I recently wrote a blog on auto-scaling choices http://blog.paraleap.com/post/2013/07/29/Windows-Azure-Auto-scaling-Options-Side-by-Side-Comparison