AWS API Gateway + Lambda + EC2 returning 503 Service Unavailable error in 5 seconds

Question

I am looking to deploy my machine learning model to production. I am using API Gateway + Lambda to trigger inferencing on an EC2 instance. The inferencing takes a while (~30 secs). The Lambda timeout is set to 2 minutes.

When I use Postman or my browser to call the HTTP API, I am getting a 503 Service Unavailable error in 5 secs. At the backend I am seeing that the request is successful and success response is sent back to the Lambda (after about 30 secs). I have read that max API Gateway timeout is 30 secs. I am wondering why I am getting a timeout in 5 secs then?

I am following instructions given here to deploy my model: https://francescopochetti.com/deploying-a-pretrained-gpt-2-model-on-aws/#Deploying_with_Lambda_EC2_and_DynamoDB

The lambda code is similar to given here: https://github.com/gabrielelanaro/ml-prototypes/blob/master/prototypes/styletransfer/huggingface/hugging_lambda.py.

If it helps, you can invoke the API using this URL: https://eekq9x0azg.execute-api.ap-south-1.amazonaws.com/gpt2?prompt=%20%22Nikhil%22&num_samples=3&length=60&temperature=0.7&top_p=0.9&top_k=40

Lambda logs in Cloudwatch:

2020-03-18T09:56:51.520+05:30 START RequestId: 59129756-d233-4bd5-8059-4087bd9f47e3 Version: $LATEST

2020-03-18T09:56:51.524+05:30

EVENT

2020-03-18T09:56:51.704+05:30 ['cd /home/ubuntu', 'shutdown -h +15', 'sudo -i -u ubuntu bash <<-EOF', 'source ~/.bashrc', 'source env/bin/activate', 'python3 gpt2-tweets.py --prompt=" "Nikhil"" --dynamoid=189377 --num_samples=3 --length=60 --temperature=0.7 --top_p=0.9 --top_k=40']

2020-03-18T09:57:22.051+05:30 The query returned the following items:

2020-03-18T09:57:22.051+05:30 [{'id': Decimal('189377'), 'text': '" Nikhil ji:\n\" I don't know, what do you want to say to the nation? I am in the UK. But I don't know what you are talking about. Sorry, that's not the point. I don t know what you want to say to the nation. I am"'}]

2020-03-18T09:57:22.091+05:30 END RequestId: 59129756-d233-4bd5-8059-4087bd9f47e3

2020-03-18T09:57:22.091+05:30 REPORT RequestId: 59129756-d233-4bd5-8059-4087bd9f47e3 Duration: 30567.70 ms Billed Duration: 30600 ms Memory Size: 128 MB Max Memory Used: 100 MB

Answer 1

Found that the API that a default timeout set to 5 seconds. See the attached image to change it.