Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Error while using unzip with 12mb file size

Im using the following open source to unzip file and its working as expected on zip with size 2-5 MB but when I put zip on 10 more MB I got error, there is more stable open source which I can use for large zip files? I need it to be under MIT license. this is what I've used https://github.com/EvanOxfeld/node-unzip

var extractor = unzip.Extract({ path: "../"});

extractor.on("close", function() {
    console.log("Success unzip");
});

extractor.on("close", function(err) {
    console.log(err);
});

req.pipe(extractor);
like image 484
07_05_GuyT Avatar asked Nov 05 '15 20:11

07_05_GuyT


Video Answer


1 Answers

according to this issue of the unzip github page - https://github.com/EvanOxfeld/node-unzip/issues/73,

  • the unzip module is not supported anymore.
  • a problem related to streaming from http seem to have been solved in a fork called unzip2 (which is also very old)

try with

npm install unzip2

there are also other zip MIT modules you can try :

  • https://www.npmjs.com/package/adm-zip
  • https://www.npmjs.com/package/julien-f-unzip

you can also find this library - https://www.npmjs.com/package/yauzl which explains why the 'zip' file format is not very well suited to streaming.

https://www.npmjs.com/package/yauzl#no-streaming-unzip-api

Due to the design of the .zip file format, it's impossible to interpret a .zip file from start to finish (such as from a readable stream) without sacrificing correctness. The Central Directory, which is the authority on the contents of the .zip file, is at the end of a .zip file, not the beginning. A streaming API would need to either buffer the entire .zip file to get to the Central Directory before interpreting anything (defeating the purpose of a streaming interface), or rely on the Local File Headers which are interspersed through the .zip file. However, the Local File Headers are explicitly denounced in the spec as being unreliable copies of the Central Directory, so trusting them would be a violation of the spec.

Any library that offers a streaming unzip API must make one of the above two compromises, which makes the library either dishonest or nonconformant (usually the latter). This library insists on correctness and adherence to the spec, and so does not offer a streaming API.

there are 354 results to "unzip" on NPM - https://www.npmjs.com/search?q=unzip

whatever library you choose, make sure to first write a node.js program using this library that extracts the content of you errored zip file directly from the filesystem.

Once this works, you will be able to add the web server additional complexity.

like image 90
Jerome WAGNER Avatar answered Oct 16 '22 16:10

Jerome WAGNER