Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Node read stream: when does the streaming happen?

Here is a code example that is not much different from what you can get off the net or documentation:

var fs = require('fs');
var r = fs.createReadStream(process.argv[2], { encoding: 'utf8' });
r.on('data', function (chunk) {
    console.log("chunk: >>>" + chunk + "<<<");
});
r.on('end', function () {
    console.log("This is the end");
});

What puzzles me: when does the streaming that triggers events happen? Apparently not directly on construction of the read stream, because then it would be done before we get to the ons, and the event-listening code would never be executed (which it is, this works perfectly).

What worries me: is there a theoretical chance that an event is missed if the on call comes too late?

like image 517
njlarsson Avatar asked Mar 04 '26 06:03

njlarsson


2 Answers

The answer is no, it's not possible in node 0.10.x and later. When stream is created, it is paused, so neither data nor end events can be emitted. When you add data listener, the stream is automatically resumed.

Also worth mentioning that no IO can occur before current "tick" ends, so if you are attaching data listeners in the same tick, it is always safe, even for earlier node versions. For example:

stream.resume();
stream.on('data', ...); // <- same tick, same javascript invocation = safe

stream.resume();
setImmediate(function () {
  stream.on('data', ...); // <- different tick, different javascript invocation = unsafe
});

It might sound confusing, but adding listener in process.nextTick callback is also safe because it's actually called right after CURRENT tick, before any IO callback (a case of really bad naming).

like image 65
vkurchatkin Avatar answered Mar 05 '26 19:03

vkurchatkin


The easiest way to think of this is that all the code you've provided is blocking therefore no activity can occur on the stream until the current tick ends as vkurchatkin explains. Only when the JavaScript execution for the current tick finishes can the stream IO begin.

var fs = require('fs');
var r = fs.createReadStream(process.argv[2], { encoding: 'utf8' });
// Stream created in paused state, JS code is blocking IO

// We're still in the current tick so no IO could have occured since the above lines
r.on('data', function (chunk) {
  console.log("chunk: >>>" + chunk + "<<<");
});

// We're still in the current tick so no IO could have occured since the above lines
r.on('end', function () {
  console.log("This is the end");
});

// We've left the current tick so the internal IO code will now execute and call the bound events if necessary
like image 27
Evan Shortiss Avatar answered Mar 05 '26 20:03

Evan Shortiss



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!