AWS ELB - will it retry request if node fails?

Question

I have an ELB and 3 nodes behind it.

Can someone please explain me what will ELB do in these scenarios:

1. Client Request -> ELB -> Node1 fails in the middle of the request (ELB timeout)
2. Client Request -> ELB -> Node1 timeouts (Server timeout and health check haven't kicked in yet)

Particularly I'm wondering if ELB retries the request to another node?

I made a test and it doesn't seem to, but maybe there's a setting that I've missed.

Thanks,

Answer 1

This may have been a matter of passage of time, but these days ELBs do retry requests that abort:

• Either because of an Idle Timeout (60s by default);
• Or because the instance went unhealthy due to failing health checks, with Connection Draining disabled (default is enabled)

However, this holds only if you haven't sent any response bytes yet. If you have sent incomplete headers, you will get a 408 Request Timeout. If you have sent full headers (but didn't send a body or got halfway through the body) the client will get the incomplete response as-is.

The experiments I've performed were all with a single HTTP request per connection. If you use Keep-Alive connections, the behavior might be different.

Answer 2

The AWS Elastic Load Balancing service uses Health Checks to identify healthy/unhealthy Amazon EC2 instances. If an instance is marked as Unhealthy, then no new traffic will be sent to that server. Once it is identified as Heathy, traffic will once again be sent to the instance.

If a request is sent to an instance and no response is received (either because the app fails or a timeout is triggered), the request will not be resent nor sent to another server. It would be up to the originator (eg a user or an app) to resend the request.