Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

NodeJS How do I Download a file to disk from an aws s3 bucket?

My goal:

Display a dialog box prompting the user to save a file being downloaded from aws.

My problem:

I am currently using awssum-amazon-s3 to create a download stream. However I've only managed to save the file to my server or stream it to the command line... As you can see from my code my last attempt was to try and manually set the content disposition headers which failed. I cannot use res.download() as the headers have already been set?

How can I achieve my goal?

My code for node:

app.post('/dls/:dlKey', function(req, res, next){         // download the file via aws s3 here         var dlKey = req.param('dlKey');          Dl.findOne({key:dlKey}, function(err, dl){             if (err) return next(err);             var files = dl.dlFile;              var options = {                 BucketName    : 'xxxx',                 ObjectName    : files,             };              s3.GetObject(options, { stream : true }, function(err, data) {                 // stream this file to stdout                 fmt.sep();                 data.Headers['Content-Disposition'] = 'attachment';                 console.log(data.Headers);                 data.Stream.pipe(fs.createWriteStream('test.pdf'));                 data.Stream.on('end', function() {                     console.log('File Downloaded!');                 });             });         });          res.end('Successful Download Post!');     }); 

My code for angular:

$scope.dlComplete = function (dl) {         $scope.procDownload = true;         $http({             method: 'POST',             url: '/dls/' + dl.dlKey         }).success(function(data/*, status, headers, config*/) {             console.log(data);             $location.path('/#!/success');         }).error(function(/*data, status, headers, config*/) {             console.log('File download failed!');         });     }; 

The purpose of this code it to let users use a generated key to download a file once.

like image 714
gbachik Avatar asked Mar 03 '14 10:03

gbachik


People also ask

How do I transfer files from S3 bucket to local machine?

You can use cp to copy the files from an s3 bucket to your local system. Use the following command: $ aws s3 cp s3://bucket/folder/file.txt .

How do I download content from S3 bucket?

To download an entire bucket to your local file system, use the AWS CLI sync command, passing it the s3 bucket as a source and a directory on your file system as a destination, e.g. aws s3 sync s3://YOUR_BUCKET . . The sync command recursively copies the contents of the source to the destination.

How do I download from Amazon S3 folder?

How to Download a Folder from AWS S3 # Use the s3 cp command with the --recursive parameter to download an S3 folder to your local file system. The s3 cp command takes the S3 source folder and the destination directory as inputs and downloads the folder.


1 Answers

This is the entire code using streaming on the latest version of aws-sdk

var express = require('express'); var app = express(); var fs = require('fs');  app.get('/', function(req, res, next){     res.send('You did not say the magic word'); });   app.get('/s3Proxy', function(req, res, next){     // download the file via aws s3 here     var fileKey = req.query['fileKey'];      console.log('Trying to download file', fileKey);     var AWS = require('aws-sdk');     AWS.config.update(       {         accessKeyId: "....",         secretAccessKey: "...",         region: 'ap-southeast-1'       }     );     var s3 = new AWS.S3();     var options = {         Bucket    : '/bucket-url',         Key    : fileKey,     };      res.attachment(fileKey);     var fileStream = s3.getObject(options).createReadStream();     fileStream.pipe(res); });  var server = app.listen(3000, function () {     var host = server.address().address;     var port = server.address().port;     console.log('S3 Proxy app listening at http://%s:%s', host, port); }); 
like image 133
Yash Dayal Avatar answered Oct 04 '22 05:10

Yash Dayal