What's the pattern for an Azure application that scales itself without any third party?

Question

I want to make my Azure web role scalable in a self-contained manner.

All scaling solutions I've seen so far include some third party - it can be a special service that monitors the Azure role and scales it or it may be an application for manual scaling. What I want is extra code inside my role that makes decisions on when to scale and initiates scaling.

I forsee the following problems. First, there should be only one role instance at each point of time deciding when to scale and how to scale so that two or more instances don't create a scaling mess. Second, if that specific instance dies for whatever reason another instance must be selected within some reasonable period of time. And finally, all stuff should not introduce too much overhead.

Are there any patterns for implementing such self-contained scaling?

Answer 1

We have a been running autoscaled apps with on Windows Azure with our own provisioning lib for a while now, even before the release of WASABi.

The pattern that has been working best for us so far includes the following aspects:

• Don't trust any low-level metric (aka memory or CPU usage for ex) to define your scaling metric, craft a simple but highly relevant metric that defines your workload (and the number of VMs being needed).
• Keep a strict symmetry between all VMs (no master or slave): every VM is potentially responsible for changing the number of VM being provisioned.
• Each VM has a background thread dealing with the autoscaling. 99.999% of the time, this thread is doing nothing. Every minute, the thread tries to get a blob lease, and then run the provisioning logic. If blob lease cannot be acquired right away, give up, another VM is on it already.
• Instantiating VMs over Azure takes about 5min, and you're charged for 1h immediately. I suggest to be a bit lazy when it comes to deinstanciation and to wait 30min or so, no benefit in de-instanciating after 10min.
• The provisioning API is slow, make sure the calling thread is not blocking anything on the VM during that time.
• Keep the minimal number of VMs to 2 to improve uptime.

Answer 2

You can certainly place auto-scale code in your web role. To prevent multiple instances from executing the scale logic, one technique is to use a "mutex" in the form of a blob lease (you can only have one blob writer).

• In each instance, have code that attempts to obtain a lease on a well-known blob.
• Whoever gets the lease runs the scale code (presumably some type of scheduled check, possibly using the autoscale application block @tjrobinson mentioned).
• Whoever gets an exception sleeps and periodically tries again. If the instance holding the lease dies for some reason, another instance will be able to obtain the soon-to-be-lost lease, and start executing the auto-scale code.