Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Azure Cloud Service role instances - auto-scaling - Changing event not firing

I got a Cloud Service deployment with 4 worker roles, one of which got auto-scaling enabled. As soon as auto-scaling occurs, all instances of all roles are recycling.

Ideally, I'd like to stop the roles from recycling or at least terminate the work of all other roles in a controlled way.

I found out, that you can react to the RoleEnvironment.Changing event and cancel it to request a graceful shutdown (i.e. make OnStop being called). However, by adding tracing output to the Changing event handler, I noticed that the Changing event was obviously not even fired, so the cancellation was not being registered either.

private void RoleEnvironmentChanging(object sender, RoleEnvironmentChangingEventArgs e)
{
    // This tracing output does not show up in the logs table.
    Trace.TraceInformation("RoleEnvironmentChanging event fired.");
    if ((e.Changes.Any(change => change is RoleEnvironmentConfigurationSettingChange)))
    {
        // This one neither.
        Trace.TraceInformation("One of the changes is a RoleEnvironmentConfigurationSettingChange. Cancelling..");

        e.Cancel = true;
    }
    if ((e.Changes.Any(change => change is RoleEnvironmentTopologyChange)))
    {
        // This one neither.
        Trace.TraceInformation("One of the changes is a RoleEnvironmentTopologyChange. Cancelling.");

        e.Cancel = true;
    }
}

public override bool OnStart()
{
    // Hook up to the changing event to prevent roles from unnecessarily restarting.
    RoleEnvironment.Changing += RoleEnvironmentChanging;

    // Set the maximum number of concurrent connections
    ServicePointManager.DefaultConnectionLimit = 12;

    bool result = base.OnStart();

    return result;
}

Also adding an internal endpoint to each role did not bring the change. Here the configuration from the .csdef:

<WorkerRole name="MyRole" vmsize="Medium">
[...ConfigurationSettings...]
<Endpoints>
  <InternalEndpoint name="Endpoint1" protocol="http" />
</Endpoints>
</WorkerRole>

Also changing the protocol to "any" wasn't successful.

How can I stop my role instances from recycling after a scaling operation?

EDIT:
» Included code snippets
» Fixed typos

like image 624
Ben Sch Avatar asked Apr 22 '15 15:04

Ben Sch


People also ask

What is automatic scaling in Azure App service?

Automatic scaling has its own internal health probes that are used to make informed scaling decisions. You can only have Azure App Service web apps in the app service plan where you wish to enable automatic scaling.

How do I find a role instance in azure?

Once the instance starts, open the Server Explorer. Expand the Azure\Cloud Services node and locate the deployment. Expand the deployment until you see the role instances. Right-click on one of the instances.

How do I configure scale settings for a cloud service role?

On the cloud service blade, on the Roles and Instances tile, select the name of the cloud service. IMPORTANT: Make sure to click the cloud service role, not the role instance that is below the role. Select the scale tile. You can configure scale settings for a role with either two modes manual or automatic.

What deployment model should I use for Azure cloud services roles?

New deployments should use the new Azure Resource Manager based deployment model Azure Cloud Services (extended support). Here are some common problems and solutions related to Azure Cloud Services roles that fail to start.


1 Answers

Did you try one of the following?

  • Check whether the event is being fired in the instances of role which is auto-scaling (to make sure its not a problem with the internal endpoint)
  • Do a complete re-deployment (instead of update).
  • Add a short Thread.Sleep() after the Tracing output in the event handler (sometimes the role is being shut down before the trace output can be registered)
  • Do a change in one of the configs via the management portal (and check whether event is being triggered)
  • Check whether the other events (for instance RoleEnvironment.Changed) are being fired
like image 122
Paul Facklam Avatar answered Oct 06 '22 00:10

Paul Facklam