Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

node.js compressing ZIP to memory

I want to zip some data into a writableStream.

the purpose is to do all in memory and not to create an actual zip file on disk.

For testing only, i'm creating a ZIP file on disk. But when I try to open output.zip i get the following error: "the archive is either in unknown format or damaged". (WinZip at Windows 7 and also a similar error on MAC)

What am I doing wrong?

const   fs = require('fs'),
    archiver = require('archiver'),
    streamBuffers = require('stream-buffers');

let outputStreamBuffer = new streamBuffers.WritableStreamBuffer({
    initialSize: (1000 * 1024),   // start at 1000 kilobytes.
    incrementAmount: (1000 * 1024) // grow by 1000 kilobytes each time buffer overflows.
});

let archive = archiver('zip', {
    zlib: { level: 9 } // Sets the compression level.
});
archive.pipe(outputStreamBuffer);

archive.append("this is a test", { name: "test.txt"});
archive.finalize();

outputStreamBuffer.end();

fs.writeFile('output.zip', outputStreamBuffer.getContents(), function() { console.log('done!'); });
like image 372
Yoni Mayer Avatar asked Jul 21 '17 14:07

Yoni Mayer


People also ask

How much memory does node js take?

By default, Node. js (up to 11. x ) uses a maximum heap size of 700MB and 1400MB on 32-bit and 64-bit platforms, respectively.

What is the buffer class in Nodejs?

The Buffer class in Node. js is designed to handle raw binary data. Each buffer corresponds to some raw memory allocated outside V8. Buffers act somewhat like arrays of integers, but aren't resizable and have a whole bunch of methods specifically for binary data.


1 Answers

You are not waiting for the content to be written to the output stream.

Comment out outputStreamBuffer.end(); from your code and change it to the following...

outputStreamBuffer.on('finish', function () {
    fs.writeFile('output.zip', outputStreamBuffer.getContents(), function() { 
        console.log('done!');
    });
});
like image 173
Mahesh J Avatar answered Nov 07 '22 01:11

Mahesh J