Is significant latency introduced by API Gateway?

Question

I'm trying to figure out where the latency in my calls is coming from, please let me know if any of this information could be presented in a format that is more clear!

Some background: I have two systems--System A and System B. I manually (through Postman) hit an endpoint on System A that invokes an endpoint on System B. System A is hosted on an EC2 instance.

• When System B is hosted on a Lambda function behind API Gateway, the latency for the call is 125 ms.
• When System B is hosted on an EC2 instance, the latency for the call is 8 ms.
• When System B is hosted on an EC2 instance behind API Gateway, the latency for the call is 100 ms.

So, my hypothesis is that API Gateway is the reason for increased latency when it's paired with the Lambda function as well. Can anyone confirm if this is the case, and if so, what is API Gateway doing that increases the latency so much? Is there any way around it? Thank you!

Answer 1

It might not be exactly what the original question asks for, but I'll add a comment about CloudFront.

In my experience, both CloudFront and API Gateway will add at least 100 ms each for every HTTPS request on average - maybe even more.

This is due to the fact that in order to secure your API call, API Gateway enforces SSL in all of its components. This means that if you are using SSL on your backend, that your first API call will have to negotiate 3 SSL handshakes:

1. Client to CloudFront
2. CloudFront to API Gateway
3. API Gateway to your backend

It is not uncommon for these handshakes to take over 100 milliseconds, meaning that a single request to an inactive API could see over 300 milliseconds of additional overhead. Both CloudFront and API Gateway attempt to reuse connections, so over a large number of requests you’d expect to see that the overhead for each call would approach only the cost of the initial SSL handshake. Unfortunately, if you’re testing from a web browser and making a single call against an API not yet in production, you will likely not see this.

In the same discussion, it was eventually clarified what the "large number of requests" should be to actually see that connection reuse:

Additionally, when I meant large, I should have been slightly more precise in scale. 1000 requests from a single source may not see significant reuse, but APIs that are seeing that many per second from multiple sources would definitely expect to see the results I mentioned.

...

Unfortunately, while cannot give you an exact number, you will not see any significant connection reuse until you approach closer to 100 requests per second.

Bear in mind that this is a thread from mid-late 2016, and there should be some improvements already in place. But in my own experience, this overhead is still present and performing a loadtest on a simple API with 2000 rps is still giving me >200 ms extra latency as of 2018.

source: https://forums.aws.amazon.com/thread.jspa?messageID=737224

Answer 2

Heard from Amazon support on this:

With API Gateway it requires going from the client to API Gateway, which means leaving the VPC and going out to the internet, then back to your VPC to go to your other EC2 Instance, then back to API Gateway, which means leaving your VPC again and then back to your first EC2 instance.

So this additional latency is expected. The only way to lower the latency is to add in API Caching which is only going to be useful is if the content you are requesting is going to be static and not updating constantly. You will still see the longer latency when the item is removed from cache and needs to be fetched from the System, but it will lower most calls.

So I guess the latency is normal, which is unfortunate, but hopefully not something we'll have to deal with constantly moving forward.