AWS Elastic Load Balancing: Seeing extremely long initial connection time

Question

For a couple of days, we often see an extremely long initial connection time (15s - 1.3 minutes) to our ELBs when making any request via ssl. Oddly, I was only able to observe this in Google Chrome (not Safari nor Firefox nor curl).

It does not occur every single request, but around 50% of requests. It occurs with the first request (OPTIONS-call).

Our setup is the following: Cross-Zone ELB that connects to a node.js backend (currently in 2 AZs in eu-west-1). All instances are healthy and once the request comes through, it is processed normally. Currently, there is basically no load on the system. Cloudwatch for ELB does not report any backend connection errors, neither a SurgeQueue (value 0) nor a spillover count. The ELB metrics show a low latency (< 100 ms). We have Route53 configured to route to the ELB (we don't see any dns trouble, see attached screenshot).

We have different REST-APIs that all have this setup. It occurs to all of the ELBs (each of them is connecting to an indipendent node.js backend). All of these ELBs are set up the same way via our cloudformation template.

The ELBs also do our SSL-termination.

What could lead to such a behavior? Is it possible that the ELBs are not configured properly? And why could it only appear on Google Chrome?

Answer 1

I think it is a possible ELB misconfiguration. I had the same problem when I put private subnets to ELB. Fixed it by changing private subnets to public. See https://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/elb-manage-subnets.html

Answer 2

Just to follow up on @Nikita Ogurtsov's excellent answer; I had the same problem except that it was just one of my subnets that happened to be private and the rest public.

Even if you think your subnets are public, I recommend you double check the route tables to ensure that they all have a Gateway.

You can use a single Route Table that has a Gateway for all your LB subnets if this make sense

VPC/Subnets/(select subnet)/Route Table/Edit