how to pipe a network image to an s3 upload link using request.js

Question

I'm trying to get a photo from a url and pipe it into a PUT link that uploads to an amazon s3 bucket. This PUT link is a pre-signed upload url so all that needs to happen is the body of the PUT request needs to contain the photo data.

I've tried the following but it doesn't seem to work - it doesn't seem to pass through any of the data from the get.

var request = require('request');

request.get('https://SomeUrlThatRedirectsAFewTimes.com').pipe(request.put('https://mys3uploadlink.com'));

Answer 1

Using the AWS SDK, you can pass a stream as the Body of the upload. So I'd just save the stream to a variable, and pass that as the body. You can see the documentation for this here.

I've never done this myself, but I'd assume you do something like this:

function upload(cb) {
   var s3 = new AWS.S3(...);
   var stream = request.get(myURL);
   stream.on('error', cb)
         .on('close', cb);
   var params = {Bucket: 'bucket', Key: 'key', Body: stream};
   var options = {partSize: 10 * 1024 * 1024, queueSize: 1};
   s3.upload(params, options, cb);
}