s3.getObject().promise() never returns anything

Question

If use this code within a Lambda which complies with everything I read on stackoverflow and on the AWS SDK documentation.

However, it neither returns anything nor throws an error. The code is simply stuck on s3.getObject(params).promise() so the lambda function runs on a timeout, even after more then 30 seconds. The file i try to fetch is actually 25kb.

Any idea why this happens?

var AWS = require('aws-sdk');
var s3 = new AWS.S3({httpOptions: {timeout: 3000}});    

async function getObject(bucket, objectKey) {
        try {
            const params = {
                Bucket: bucket,
                Key: objectKey
            }
            console.log("Trying to fetch " + objectKey + " from bucket " + bucket)
            const data = await s3.getObject(params).promise()
            console.log("Done loading image from S3")
            return data.Body.toString('utf-8')
        } catch (e) {
            console.log("error loading from S3")
            throw new Error(`Could not retrieve file from S3: ${e.message}`)
        }
    }

When testing the function, i receive the following timeout.

START RequestId: 97782eac-019b-4d46-9e1e-3dc36ad87124 Version: $LATEST 2019-03-19T07:51:30.225Z 97782eac-019b-4d46-9e1e-3dc36ad87124 Trying to fetch public-images/low/ZARGES_41137_PROD_TECH_ST_LI.jpg from bucket zarges-pimdata-test 2019-03-19T07:51:54.979Z 97782eac-019b-4d46-9e1e-3dc36ad87124 error loading from S3 2019-03-19T07:51:54.981Z 97782eac-019b-4d46-9e1e-3dc36ad87124 {"errorMessage":"Could not retrieve file from S3: Connection timed out after 3000ms","errorType":"Error","stackTrace":["getObject (/var/task/index.js:430:15)","","process._tickDomainCallback (internal/process/next_tick.js:228:7)"]} END RequestId: 97782eac-019b-4d46-9e1e-3dc36ad87124 REPORT RequestId: 97782eac-019b-4d46-9e1e-3dc36ad87124 Duration: 24876.90 ms Billed Duration: 24900 ms Memory Size: 512 MB Max Memory Used: 120 MB

The image I am fetching is actually public available: https://s3.eu-central-1.amazonaws.com/zarges-pimdata-test/public-images/low/ZARGES_41137_PROD_TECH_ST_LI.jpg

Answer 1

const data = (await (s3.getObject(params).promise())).Body.toString('utf-8')

Answer 2

Default timeout of AWS SDK is 120000 ms. If your lambda's timeout is shorter then that, you will never receive the actual error.

Either extend your AWS timeout

var AWS = require('aws-sdk');
var s3 = new AWS.S3({httpOptions: {timeout: 3000}});

or extend the timout of your lambda.

Answer 3

If your Lambda function is associated with a VPC it loses internet access which is required to access S3. However, instead of following the Lambda warning that says "Associate a NAT" etc, you can create an S3 endpoint in the VPC > Endpoints settings, and your Lambda function will work as expected, with no need to manually set up Internet access for your VPC.

https://aws.amazon.com/blogs/aws/new-vpc-endpoint-for-amazon-s3/