Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I ungzip (decompress) a NodeJS request's module gzip response body?

How do I unzip a gzipped body in a request's module response?

I have tried several examples around the web but none of them appear to work.

request(url, function(err, response, body) {     if(err) {         handleError(err)     } else {         if(response.headers['content-encoding'] == 'gzip') {                 // How can I unzip the gzipped string body variable?             // For instance, this url:             // http://highsnobiety.com/2012/08/25/norse-projects-fall-2012-lookbook/             // Throws error:             // { [Error: incorrect header check] errno: -3, code: 'Z_DATA_ERROR' }             // Yet, browser displays page fine and debugger shows its gzipped             // And unzipped by browser fine...             if(response.headers['content-encoding'] && response.headers['content-encoding'].toLowerCase().indexOf('gzip') > -1) {                    var body = response.body;                                     zlib.gunzip(response.body, function(error, data) {                     if(!error) {                         response.body = data.toString();                     } else {                         console.log('Error unzipping:');                         console.log(error);                         response.body = body;                     }                 });             }         }     } } 
like image 727
izk Avatar asked Aug 27 '12 20:08

izk


2 Answers

I couldn't get request to work either, so ended up using http instead.

var http = require("http"),     zlib = require("zlib");  function getGzipped(url, callback) {     // buffer to store the streamed decompression     var buffer = [];      http.get(url, function(res) {         // pipe the response into the gunzip to decompress         var gunzip = zlib.createGunzip();                     res.pipe(gunzip);          gunzip.on('data', function(data) {             // decompression chunk ready, add it to the buffer             buffer.push(data.toString())          }).on("end", function() {             // response and decompression complete, join the buffer and return             callback(null, buffer.join(""));           }).on("error", function(e) {             callback(e);         })     }).on('error', function(e) {         callback(e)     }); }  getGzipped(url, function(err, data) {    console.log(data); }); 
like image 144
WearyMonkey Avatar answered Sep 18 '22 17:09

WearyMonkey


try adding encoding: null to the options you pass to request, this will avoid converting the downloaded body to a string and keep it in a binary buffer.

like image 30
Iftah Avatar answered Sep 21 '22 17:09

Iftah