Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

AWS Lambda TooManyRequestsException: Rate Exceeded

Tags:

When attempting to execute an Amazon Web Services (AWS) Lambda function (many times) we were seeing the error:

AWS Lambda TooManyRequestsException: Rate Exceeded

How do we resolve this?

like image 520
nelsonic Avatar asked Apr 24 '16 17:04

nelsonic


People also ask

What happens if Lambda it runs for more than 15 minute?

When the specified timeout is reached, Amazon Lambda terminates execution of your Lambda function. As a best practice, you should set the timeout value based on your expected execution time to prevent your function from running longer than intended.

Can we increase Lambda timeout more than 15 minutes?

Historically, the maximum execution timeout limit was 5 minutes. At the time of writing this article, AWS Lambda functions cannot run continuously for more than 15 minutes. This is a hard limit in the AWS Lambda service. In practice, there are use cases for which those 15 minutes are not enough.

What happens when Lambda is throttled?

At the highest level, throttling just means that Lambda will intentionally reject one of your requests and so what we see from the user side is that when making a client call, Lambda will throw a throttling exception, which you need to handle. Typically, people handle this by backing off for some time and retrying.


1 Answers

As noted by Michael, this is the error message you will see when you reach the documented default "safety" limit of 100 concurrent invocations:

"AWS Lambda has a default safety throttle of 100 concurrent executions per account per region. If you wish to submit a request to increase the throttle of 100 concurrent executions you can visit our Support Center..."

The solution was to open a support ticket providing the following info:

Limit increase request 1  Service: Lambda  Region: EU (Ireland)  Limit name: concurrent requests (average duration * average TPS)  New limit value: 2000 

And then in the body of the ticket/request try to estimate your usage pattern:

Expected average requests per second: 200  Expected peak requests per second: 2000  Expected function duration: 2 seconds  Function memory size: 1000mb  Invocation Type: Request-response  Event Source: Api Gateway & Lambda<->Lambda 

It can take a while to get a response from AWS Support unless you are paying for premium support, so its best to load test your app during development/staging and request to increase concurrent invocations before you launch your app!

In our case it took 45 hours from initial support request to get the invocation limit increase. aws-lambda-limits-increase-requestaws-lambda-limits-request-sorted

The AWS support people are lovely, it just took what felt like ages (almost two days!) to get the service level increase, which would have been fatal if our launch had been public!

like image 148
nelsonic Avatar answered Nov 15 '22 08:11

nelsonic