How to tell if a readStream has no more lines to read?

Question

I am reading a large file (bigger than memory) line by line using the following code:

var fs = require('fs');
var readline = require('readline');
var stream = require ('stream');

function begin(){

    var instream = fs.createReadStream(fileLocation);
    var outstream = new stream;
    outstream.readable = true;
    outstream.writable = true;

    var rl = readline.createInterface({
        input: instream,
        output: outstream,
        terminal: false
    });

    var ctr = 0;
    rl.on('line', function(line) { //line reading function
       //work done here.
    });
}

I have done some processing, and then need to write a file to disk (JSON). But I don't want to write the object before every line has been processed. All the code within the line reading function is synchronous.

How do I know when it is safe to write my file ?

Answer 1

It's a bit tricky here. An object stored in r1 is a result of readline.createInterface call: that means it's an Interface, not a readStream. As Interface is used to interact (with some agent), you, as written in the docs, should listen for close event instead:

Event: close

Emitted when close() is called.

Also emitted when the input stream receives its end event. The Interface instance should be considered "finished" once this is emitted. For example, when the input stream receives ^D, respectively known as EOT.

So it should look like the following:

rl.on('line', function(line) { 
   // process emitted line
}).on('close', function() {
   // all the lines read, go on
});

Have you been working with readStream instance, you should have been listening for end event instead. It's emitted by readStream instance when the corresponding stream (created by fs.createReadStream) emits its own end event. The catch is that Interface object doesn't emit end event - as end of communication in its case means closing the channel completely.