Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

node js .10 fs.createReadStream streams2 end event not firing

I'm using Node.js v0.10.26, and trying to use the new stream2 apis http://blog.nodejs.org/2012/12/20/streams2/, in particular the Readable api, http://nodejs.org/api/stream.html#stream_class_stream_readable

but the 'end' event never gets called and I can't figure out why. wich I think is why my downstream Transform streams component aren't calling Their _flush, finish,close events,

var rqt = fs.createReadStream('some.csv.gz');

//readable
rqt.on('readable',  function () { console.log('readable!');  })
    .on('data',  function (data) { console.log('Data!', data);  })
    .on('error', function (err)  { console.error('Error', err);/* READ if there was an error receiving data. */ })
    .on('end',   function ()     { console.log('All done!');   /* READ fires when no more data will be provided. */ })
    .on('finish',   function ()     { console.log('All Finished!');  /*WRITEABLE */  })
    .on('close',   function ()     { console.log('close!'); /*WRITEABLE not all streams emit this*/   })

Outputs:

readable
readable
Data 
readable
readable
Data
readable

UPDATE the real goal of this is to do a stream like so:

rqt.pipe(zlib.createGunzip()).pipe(csvp).on('finish',function(line){
    console.log("Finished Parsing " + csvp.records + ' ' + line);
});

which doesn tuse the classic mode at all, Functionally which loads a gzipped csv, parses the csv and shows how many were found. The loading, unzip and parsing all work fine.

however none of the following get called: - the _flush of csvp never gets called - the on('finish' event

like image 970
TroyWorks Avatar asked Nov 02 '22 01:11

TroyWorks


1 Answers

You are mixing the flowing mode and non-flowing mode APIs a bit. From the docs:

Note that the end event will not fire unless the data is completely consumed. This can be done by switching into flowing mode, or by calling read() repeatedly until you get to the end.

Simplest fix to your snippet above is just to remove the 'readable' binding then you should be in regular flowing mode and 'data' events will fire until the stream is empty then 'end' will fire.

like image 129
Peter Lyons Avatar answered Nov 09 '22 23:11

Peter Lyons