Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

NodeJS download file from AWS S3 Bucket

I'm trying to make an endpoint in NodeJS/Express for downloading content from my AWS S3 Bucket.

It works well, I can download the file in the client side but I can also see the stream preview in the Network tab which is annoying...

QUESTION

I'm wondering if what I'm doing is correct and a good practice. Also would like to know if it's normal to see the output stream in the Network tab.

How should I properly send I file from S3 to my client application using NodeJS/Express?

I'm pretty sure other websites requests don't let you preview the content with a: "Fail to load response data".


This is what I do in my NodeJS application to get the stream file from AWS S3:

download(fileId) {
  const fileObjectStream = app.s3
    .getObject({
      Key: fileId
    })
    .createReadStream();
  this.res.set("Content-Type", "application/octet-stream");
  this.res.set(
    "Content-Disposition",
    'attachment; filename="' + fileId + '"'
  );
  fileObjectStream.pipe(this.res);
}

And in the client side I can see this:

enter image description here

like image 328
Jose A. Ayllón Avatar asked Oct 16 '22 16:10

Jose A. Ayllón


People also ask

How do I download data from S3 bucket?

To download an entire bucket to your local file system, use the AWS CLI sync command, passing it the s3 bucket as a source and a directory on your file system as a destination, e.g. aws s3 sync s3://YOUR_BUCKET . . The sync command recursively copies the contents of the source to the destination.

How do I download from S3 bucket to local using command line?

Your answer You can use cp to copy the files from an s3 bucket to your local system. Use the following command: $ aws s3 cp s3://bucket/folder/file.txt . To know more about AWS S3 and its features in detail check this out!

Can I download a folder from S3 bucket?

Use the cp command to download a folder inside a bucket from S3 to local. recursive option will download all files and folders if you have a recursive folder/file structure.


1 Answers

I think the issue is with the header :

          //this line will set proper header for file and make it downloadable in client's browser

          res.attachment(key); 

          // this will execute download 
          s3.getObject(bucketParams)
          .createReadStream()
          .pipe(res);

So code should be like this (This is what I am doing it in my project handling file as res.attachment or res.json in case of error so client can display error to end user) :

router.route("/downloadFile").get((req, res) => {
      const query = req.query; //param from client
      const key = query.key;//param from client
      const bucketName = query.bucket//param from client

      var bucketParams = {
        Bucket: bucketName,  
        Key: key
      };

      //I assume you are using AWS SDK
      s3 = new AWS.S3({ apiVersion: "2006-03-01" });

      s3.getObject(bucketParams, function(err, data) {
        if (err) {
          // cannot get file, err = AWS error response, 
          // return json to client
          return res.json({
            success: false,
            error: err
          });
        } else {
          res.attachment(key); //sets correct header (fixes your issue ) 
          //if all is fine, bucket and file exist, it will return file to client
          s3.getObject(bucketParams)
            .createReadStream()
            .pipe(res);
        }
      });
    });
like image 82
StefaDesign Avatar answered Oct 19 '22 00:10

StefaDesign