AWS ELB doesn't distribute requests to auto scaling group EC2 instances in some cases

Question

I'm trying to do performance testing for my AWS auto scaling group using jmeter.

Firstly, I did scale-in/out testing. I set the threshold to be 70% cpu utilization for 2 periods, each period is 2 minutes. The ELB works fine, and the requests was distributed to all EC2 instances in auto scaling group, in spite of un-equality, after the system scale-out.

In next, I want to test whether the two instances' load can be twice of one instance's. I fixed the instance number of auto scaling group, I set the min/max/desired instance count to be 2. When I push load from single JMeter, there are always just only one instance works and its cpu utilization reach almost 100 percent, but the cpu utilization of the other instance is still zero.... If I push load from an JMeter cluster which contains several slaves, all instances take load.

Somebody said, maybe the load is not heavy enough, so the ELB considered that just one instance can handle it and didn't dispatch requests to other instance. I don't think so, because I push load from just one slave of this JMeter cluster, however I increase the load, just only one instance handle requests.

I found an blog which said ELB is great in HA but not load balancing. https://www.stackdriver.com/elb-affinity-problems But, I don't think the behavior, only one instance handle requests, is normal.

What the hell in the ELB load balance mechanism? I'm confused.

Answer 1

Elastic Load Balancing mechanism

• DNS server uses DNS round robin to determine which load balancer node in a specific Availablity Zone will receive the request
• The selected load balancer checks for "sticky session" cookie
• The selected oad balancer sends the request to the least loaded instance

And in greater details:

Availability Zones (unlikely your case)

By default, the load balancer node routes traffic to back-end instances within the same Availability Zone. To ensure that your back-end instances are able to handle the request load in each Availability Zone, it is important to have approximately equivalent numbers of instances in each zone. For example, if you have ten instances in Availability Zone us-east-1a and two instances in us-east-1b, the traffic will still be equally distributed between the two Availability Zones. As a result, the two instances in us-east-1b will have to serve the same amount of traffic as the ten instances in us-east-1a.

Sessions (most likely your case)

By default a load balancer routes each request independently to the server instance with the smallest load. By comparison, a sticky session binds a user's session to a specific server instance so that all requests coming from the user during the session will be sent to the same server instance.

AWS Elastic Beanstalk uses load balancer-generated HTTP cookies when sticky sessions are enabled for an application. The load balancer uses a special load balancer–generated cookie to track the application instance for each request. When the load balancer receives a request, it first checks to see if this cookie is present in the request. If so, the request is sent to the application instance specified in the cookie. If there is no cookie, the load balancer chooses an application instance based on the existing load balancing algorithm. A cookie is inserted into the response for binding subsequent requests from the same user to that application instance. The policy configuration defines a cookie expiry, which establishes the duration of validity for each cookie.

Routing Algorithm (less likely your case)

Load balancer node sends the request to healthy instances within the same Availability Zone using the leastconns routing algorithm. The leastconns routing algorithm favors back-end instances with the fewest connections or outstanding requests.

Source: Elastic Load Balancing Terminology And Key Concepts

Hope it helps.