Auto scaling using cloud formation according to Request count

Question

We are using cloud formation for auto scaling according based upon the load balancer RequestCount metric. Currently we scale up an instance if the request is increased to 1500 for 1 minute (each instance can handle 1500 request per minute). The problem is, since the autoscaling group continually checks the RequestCount and adds a new instance if the request count is greater than 1500 for 1 min. But it is not required as I now have 2 instances which can handle 3000 req. per minute. Is there any facility to make matrices custom? i.e. if new instance is added then the scale up policy will change to 3000 req.

Example scenario:

1. Initially there is 1 ELB, 1 tomcat instance attached to ELB(can handle 1500 req. per min).
2. 1 cloud watch with action of scale up ploicy if the req. count on ELB is increased to 1500 for min.
3. Currently request load on ELB is 1500 for 1 min. now req. load is increased to 1700 for min. so it will attach a new tomcat instance on ELB. So i have 2 instance which can handle 3000 req. for min.
4. But now what problem is cloud watch still check the req. count on ELB and if req. load is 1700 for min. it will add one new tomcat instance which is not required.

How can i over come from this problem?

Answer 1

What you want to do is use the average for the load balancer. You can have different types of metrics. Sum, Average, Minimum, Maximum and Sample. If you choose Average it will give you an average for all the instances under the loadbalancer. So it will only trigger a new instance launch when all the servers in your group are at 1500 requests per minute.

Quick description of the type:

• Average - Average for the load balancer
• Sum - The total number of request (example: 3000)
• Maximum - The max number of requests any server has (because it might not be balanced exactly)
• Minimum - The min number of requests any server has (because it might not be balanced exactly)
• Sample - The number of servers used to calculate the average (essentially how many servers are on the load balancer)

You can create you're own custom metrics as well but you'll need to create an application that tells amazon what the values are. Using the cloud watch api you can easily create your own. Take a look here http://docs.amazonwebservices.com/AmazonCloudWatch/latest/DeveloperGuide/Welcome.html?r=1540

Answer 2

As dsummersl said, RequestCount cannot be used straightaway with Average metrics.

I found a couple of workarounds here. Basically, they tell you to define a custom CloudWatch metric so you can instrument your instances to emit the number of requests that they are receiving.

For this you will need to gather the request count from each instance and send it to cloudwatch using put-metric-data, or creating an ebextension so the EBS do it for you (I think this is the simplest way).

Then you can set Auto Scaling to scale up/down based on that custom metric.

Hope it helps.

Answer 3

I've made the scaling based on RequestCount work by applying Scaling Policy with associated Cloudwatch alarms that trigger policy evaulation. Below you'll find cloudformation template that I've used in ElasticBeanstalk app:

   RequestCountScalingAlarmLt2000:
     Type: "AWS::CloudWatch::Alarm"
     Properties:
       ActionsEnabled: true
       AlarmActions:
         - Ref: RequestCountScalingPolicy
       OKActions:
         - Ref: RequestCountScalingPolicy
       AlarmDescription: "Scale when Request Count < 2000"
       AlarmName: {"Fn::Join": ["-", ["Scale when Request Count < 2000", { "Ref":"AWSEBEnvironmentName" }]]}
       ComparisonOperator: LessThanThreshold
       Dimensions:
         - Name: LoadBalancerName
           Value:
             Ref: AWSEBLoadBalancer
       EvaluationPeriods: "1"
       MetricName: RequestCount
       Namespace: AWS/ELB
       Period: "300"
       Statistic: Sum
       Threshold: 2000
   RequestCountScalingAlarmGt2000:
     Type: "AWS::CloudWatch::Alarm"
     Properties:
       ActionsEnabled: true
       AlarmActions:
         - Ref: RequestCountScalingPolicy
       OKActions:
         - Ref: RequestCountScalingPolicy
       AlarmDescription: "Scale when 2000 < Request Count < 20000"
       AlarmName: "Scale when Request 2000 < Count < 20000"
       ComparisonOperator: GreaterThanOrEqualToThreshold
       Dimensions:
         - Name: LoadBalancerName
           Value:
             Ref: AWSEBLoadBalancer
       EvaluationPeriods: "1"
       MetricName: RequestCount
       Namespace: AWS/ELB
       Period: "300"
       Statistic: Sum
       Threshold: 2000
   RequestCountScalingAlarmGt20000:
     Type: "AWS::CloudWatch::Alarm"
     Properties:
       ActionsEnabled: true
       AlarmActions:
         - Ref: RequestCountScalingPolicy
       OKActions:
         - Ref: RequestCountScalingPolicy
       AlarmDescription: "Scale when 20000 < Request Count < 30000"
       AlarmName: "Scale when 20000 < Request Count < 30000"
       ComparisonOperator: GreaterThanOrEqualToThreshold
       Dimensions:
         - Name: LoadBalancerName
           Value:
             Ref: AWSEBLoadBalancer
       EvaluationPeriods: "1"
       MetricName: RequestCount
       Namespace: AWS/ELB
       Period: "300"
       Statistic: Sum
       Threshold: 20000
   RequestCountScalingAlarmGt30000:
     Type: "AWS::CloudWatch::Alarm"
     Properties:
       ActionsEnabled: true
       AlarmActions:
         - Ref: RequestCountScalingPolicy
       OKActions:
         - Ref: RequestCountScalingPolicy
       AlarmDescription: "Scale when 30000 < Request Count < 40000"
       AlarmName: "Scale when 30000 < Request Count < 40000"
       ComparisonOperator: GreaterThanOrEqualToThreshold
       Dimensions:
         - Name: LoadBalancerName
           Value:
             Ref: AWSEBLoadBalancer
       EvaluationPeriods: "1"
       MetricName: RequestCount
       Namespace: AWS/ELB
       Period: "300"
       Statistic: Sum
       Threshold: 30000
   RequestCountScalingAlarmGt40000:
     Type: "AWS::CloudWatch::Alarm"
     Properties:
       ActionsEnabled: true
       AlarmActions::
         - Ref: RequestCountScalingPolicy
       OKActions:
         - Ref: RequestCountScalingPolicy
       AlarmDescription: "Scale when 40000 < Request Count < 50000"
       AlarmName: "Scale when 40000 < Request Count < 50000"
       ComparisonOperator: GreaterThanOrEqualToThreshold
       Dimensions:
         - Name: LoadBalancerName
           Value:
             Ref: AWSEBLoadBalancer
       EvaluationPeriods: "1"
       MetricName: RequestCount
       Namespace: AWS/ELB
       Period: "300"
       Statistic: Sum
       Threshold: 40000
   RequestCountScalingAlarmGt50000:
     Type: "AWS::CloudWatch::Alarm"
     Properties:
       ActionsEnabled: true
       AlarmActions:
         - Ref: RequestCountScalingPolicy
       OKActions:
         - Ref: RequestCountScalingPolicy
       AlarmDescription: "Scale when 50000 < Request Count < 60000"
       AlarmName: "Scale when 50000 < Request Count < 60000"
       ComparisonOperator: GreaterThanOrEqualToThreshold
       Dimensions:
         - Name: LoadBalancerName
           Value:
             Ref: AWSEBLoadBalancer
       EvaluationPeriods: "1"
       MetricName: RequestCount
       Namespace: AWS/ELB
       Period: "300"
       Statistic: Sum
       Threshold: 50000
   RequestCountScalingAlarmGt60000:
     Type: "AWS::CloudWatch::Alarm"
     Properties:
       ActionsEnabled: true
       AlarmActions::
         - Ref: RequestCountScalingPolicy
       OKActions:
         - Ref: RequestCountScalingPolicy
       AlarmDescription: "Scale when 60000 < Request Count < 70000"
       AlarmName: "Scale when 60000 < Request Count < 70000"
       ComparisonOperator: GreaterThanOrEqualToThreshold
       Dimensions:
         - Name: LoadBalancerName
           Value:
             Ref: AWSEBLoadBalancer
       EvaluationPeriods: "1"
       MetricName: RequestCount
       Namespace: AWS/ELB
       Period: "300"
       Statistic: Sum
       Threshold: 60000
   RequestCountScalingAlarmGt70000:
     Type: "AWS::CloudWatch::Alarm"
     Properties:
       ActionsEnabled: true
       AlarmActions:
         - Ref: RequestCountScalingPolicy
       OKActions:
         - Ref: RequestCountScalingPolicy
       AlarmDescription: "Scale when Request Count >= 70000"
       AlarmName: "Scale when Request Count >= 70000"
       ComparisonOperator: GreaterThanOrEqualToThreshold
       Dimensions:
         - Name: LoadBalancerName
           Value:
             Ref: AWSEBLoadBalancer
       EvaluationPeriods: "1"
       MetricName: RequestCount
       Namespace: AWS/ELB
       Period: "300"
       Statistic: Sum
       Threshold: 70000
   RequestCountScalingPolicy:
     Type: "AWS::AutoScaling::ScalingPolicy"
     Properties:
       AutoScalingGroupName:
         Ref: "AWSEBAutoScalingGroup"
       AdjustmentType: "ExactCapacity"
       PolicyType: "StepScaling"
       EstimatedInstanceWarmup: 120
       StepAdjustments:
         -
           MetricIntervalLowerBound: "0"
           MetricIntervalUpperBound: "2000"
           ScalingAdjustment: "1"
         -
           MetricIntervalLowerBound: "2000"
           MetricIntervalUpperBound: "20000"
           ScalingAdjustment: "2"
         -
           MetricIntervalLowerBound: "20000"
           MetricIntervalUpperBound: "30000"
           ScalingAdjustment: "3"
         -
           MetricIntervalLowerBound: "30000"
           MetricIntervalUpperBound: "40000"
           ScalingAdjustment: "4"
         -
           MetricIntervalLowerBound: "40000"
           MetricIntervalUpperBound: "50000"
           ScalingAdjustment: "5"
         -
           MetricIntervalLowerBound: "50000"
           MetricIntervalUpperBound: "60000"
           ScalingAdjustment: "6"
         -
           MetricIntervalLowerBound: "60000"
           MetricIntervalUpperBound: "70000"
           ScalingAdjustment: "7"
         -
           MetricIntervalLowerBound: "70000"
           ScalingAdjustment: "8"