How can I upload a file from an HTTP response body to S3 using putObject() from the AWS SDK (in Node.js)?

Question

I'm trying to save a PDF file into S3 with the AWS SDK. I'm getting the PDF through the body of a POST request (Application/PDF).

When saving the file into the local HD with fs.writeFile, the file looks ok. But when uploading it to S3, the file is corrupted (it's just a single page PDF).

Any help or hint would be greatly appreciated!

var data = body // body from a POST request.
var fileName = "test.pdf";

fs.writeFile(fileName, data, {encoding : "binary"}, function(err, data) {
    console.log('saved'); // File is OK!
});

s3.putObject({ Bucket: "bucketName", Key: fileName, Body: data }, function(err, data) {
    console.log('uploaded') // File uploads incorrectly.
});

EDIT:

It works if I write and then read the file and then upload it.

fs.writeFile(fileName, data, {encoding : "binary"}, function(err, data) {
    fs.readFile(fileName, function(err, fileData) {
        s3.putObject({ Bucket: "bucketName", Key: fileName, Body: fileData }, function(err, data) {
            console.log('uploaded') // File uploads correctly.
        });
    });
});

Answer 1

I think it is because the data is consumed (i.e. a stream).

It would explain why after writting the data you send nothing to S3 and reading again the data you can send a valid PDF.

Try and see if it works by just sending the data directly to S3 without writting it to disk.

Answer 2

Try setting the contentType and/or ContentEncoding on your put to S3.

 ContentType: 'binary', ContentEncoding: 'utf8'

See the code sample here for working example putObject makes object larger on server in Nodejs