How to download a file into a Buffer in Node.js?

Question

I'm building a Node package that needs to download some files given the url. However, I don't want to save them into the filesystem directly. Instead, I need them to be preprocessed before saving into disk. Thus, I want to save the file content in a Buffer. How can this be acomplished with pipe() ?

Offcourse I could save a temp file and then open the file, process it and save the result but this brings unnecessary reads/writes into disk.

The files have generally small size and are .txt or .zip files so saving them in memory doesn't bring any harm.

One of the objectives is to find and extract only the pretended file when the downloaded file is a .zip . Using Adm-Zip I can do this, but have to pass a Buffer.

Thak You beforehand.

Answer 1

https.get(url, (res) => {
  const data = [];
  res.on('data', (chunk) => {
    data.push(chunk);
  }).on('end', () => {
    let buffer = Buffer.concat(data);
    // Do something with the buffer
  });
}).on('error', (err) => {
  console.log('download error:', err);
});

Answer 2

You can use node-fetch library. Code sample from repository readme:

// if you prefer to cache binary data in full, use buffer()
// note that buffer() is a node-fetch only API

fetch('https://assets-cdn.github.com/images/modules/logos_page/Octocat.png')
    .then(res => res.buffer())
    .then(buffer => { /* do whatever you want with buffer */ })