S3.putObject - callback never gets called

Question

I'm trying to upload an image to S3 but when I call s3.putObject(params, callback), my callback never gets called and no errors are logged.

Here's the relevant code:

var params = {
  Key: key,
  Body: imageData,
  ContentLength: imageData.byteCount,
  ContentType: contentType,
};
this.s3.putObject(params, function(err, data) {
  console.log('here');
  if (err) {
    callback(err);
    return;
  }
  callback(null, key);
});

Where params is { Key: 'e2f99bf3a321282cc7dfaef69fe8ca62.jpg', Body: {imageData parsed from request using node-multiparty}, ContentLength: 27802, ContentType: 'image/jpeg', }

I have verified that this.s3 is valid and typeof this.s3.putObject is function as expected.

Answer 1

I got exactly the same behavior with all the IAM rights, and lost a bit of time before I got it to work.

If your lambda function runs inside a VPC you have to create an endpoint for S3 as describe in this article from AWS blog.

If you want to see more in details where it hangs, you can use the following code. Instead of giving a callback a keep the reference on the request and basically observe its events (see the documentation of S3.putObject and AWS.Request).

var obj = s3.putObject(params);
obj.on('validate',             (...args)=>{args.unshift('validate');             console.log(...args);})
   .on('build',                (...args)=>{args.unshift('build');                console.log(...args);})
   .on('sign',                 (...args)=>{args.unshift('sign');                 console.log(...args);})
   .on('send',                 (...args)=>{args.unshift('send');                 console.log(...args);})
   .on('retry',                (...args)=>{args.unshift('retry');                console.log(...args);})
   .on('extractError',         (...args)=>{args.unshift('extractError');         console.log(...args);})
   .on('extractData',          (...args)=>{args.unshift('extractData');          console.log(...args);})
   .on('success',              (...args)=>{args.unshift('success');              console.log(...args);})
   .on('error',                (...args)=>{args.unshift('error');                console.log(...args);})
   .on('complete',             (...args)=>{args.unshift('complete');             console.log(...args);})
   .on('httpHeaders',          (...args)=>{args.unshift('httpHeaders');          console.log(...args);})
   .on('httpData',             (...args)=>{args.unshift('httpData');             console.log(...args);})
   .on('httpUploadProgress',   (...args)=>{args.unshift('httpUploadProgress');   console.log(...args);})
   .on('httpDownloadProgress', (...args)=>{args.unshift('httpDownloadProgress'); console.log(...args);})
   .on('httpError',            (...args)=>{args.unshift('httpError');            console.log(...args);})
   .on('httpDone',             (...args)=>{args.unshift('httpDone');             console.log(...args);})
   .send();

By doing so I got to see the underlying HTTP request was trying to reach the public urls of the bucket, which is not possible from a VPC unless you have the endpoint :).

Here is also another post about accessing AWS ressources from a VPC also from AWS blog.

Answer 2

If this code is being used for a lambda function you need to add context.done() within the putObject's callback function like so:

        s3.putObject(params, function(err, data) {
            if (err) console.log(err, err.stack); // an error occurred
            else     console.log(data);           // successful response
            context.done();
        });

This forces the execution to wait until the callback is filled before it exits. For this to work you will also need to remove any context.succeed or context.done from the main handler if you have one there.