Zappa / Async AWS Lambda Function times out in 30s

Question

I have a Python 3.6 - Flask application deployed onto AWS Lambda using Zappa, in which I have an asynchronous task execution function defined using @Task as discussed here

However, I find that the function call still times out at 30 seconds as against the 5 minute timeout that AWS Lambda enforces for non-API calls. I even checked the timeout in my Lambda settings and it is set to 5 minutes.

The way I discovered this is when the lambda's debug output started repeating without a request - something that happens because the lamba is called 2 more times because of either an error or timeout (as per the AWS Lambda documentation).

Can anyone help me with getting this resolved?

[EDIT : The lambda function is also not part of any VPC and is set to be accessible from the internet.]

Here are the logs below. Basically, the countdown is a sleep timer counting to 20 seconds, followed by a @task call to application.reviv_assign_responder, but as we see, there is no outpust past 'NEAREST RESPONDER' and the countdown starts again, indicating that the function has timed out and has been called again by (AWS') design.

Log output in Pastebin : https://pastebin.com/VEbdCALg

Second incident - https://pastebin.com/ScNhbMcn

As we can see in the second log, it clearly states:

[1515842321866] wait_one_and_notify : 30 : 26 [1515842322867] wait_one_and_notify : 30 : 27 [1515842323868] wait_one_and_notify : 30 : 28 [1515842324865] 2018-01-13T11:18:44.865Z 72a8d34a-f853-11e7-ac2f-dd12a3d35bcb Task timed out after 30.03 seconds

Answer 1

You can check the default settings that Zappa applies to all your lambda functions here, and you will see that by default timeout_seconds is set-up to 30 seconds, This will apply over the default Lambda setup in AWS Console, because by default this is 3 seconds (you can check this limit in AWS Lambda FAQ.

For your @Task you must increase/setup your timeout_seconds in your zappa_settings.(json|yaml) file and redeploy this, You can put 5 mins (5*60==300 seconds) but this increase will be for all your functions defined in your virtualenv deployed with zappa.

You can check more details exposed in this issue in Zappa repo.

Answer 2

The timeout_seconds parameter in Zappa is misleading. That is, it does limit the timeout of the Lambda function, but the requests are served through CloudFront, which has a default timeout of 30 seconds. To verify that, try lowering the timeout_seconds to 20 - it will correctly timeout in 20 seconds. However past 30 there is no effect because of CloudFront limitation.

The default timeout is 30 seconds. You can change the value to be from 4 to 60 seconds. If you need a timeout value outside that range, request a change to the limit.

In other words, there is nothing you can do in either Zappa or Lambda to fix this, because the problem lies elsewhere (CloudFront).

I haven't tried it myself, but you might be able to up the limit by creating the cloudfront distribution in front of Lambda, though it seems you are still limited by max. 60s (unless you request more through AWS support, as indicated in the previous link).