Node.JS through2 callback

Question

I am new to Node.JS and trying to understand the through2 library.

I wonder how the callback (in the following example code which is copied from the above link) is useful. Please explain using a small piece of code if possible.

fs.createReadStream('ex.txt')
  .pipe(through2(function (chunk, enc, callback) {
    for (var i = 0; i < chunk.length; i++)
      if (chunk[i] == 97)
        chunk[i] = 122 // swap 'a' for 'z' 

    this.push(chunk)

    callback()
   }))
  .pipe(fs.createWriteStream('out.txt'))

Answer 1

I believe it is needed to continue the pipe chaining. If you wouldn't call it, the pipe would break.

This statement is from through2 documentation:

A minimal implementation should call the callback function to indicate that the transformation is done, even if that transformation means discarding the chunk.

Answer 2

If you read the through2 documentation from the link you provided you'd have seen this:

API

through2([ options, ] [ transformFunction ] [, flushFunction ])

Consult the stream.Transform documentation for the exact rules of the transformFunction (i.e. this._transform) and the optional flushFunction (i.e. this._flush).

Then, if you click on the stream.Transform link and read the documentation there, you'd get to this sooner or later: https://nodejs.org/docs/latest/api/stream.html#stream_transform_transform_chunk_encoding_callback

And it says:

transform._transform(chunk, encoding, callback)#

• chunk Buffer | String The chunk to be transformed. Will always be a buffer unless the decodeStrings option was set to false.
• encoding String If the chunk is a string, then this is the encoding type. (Ignore if decodeStrings chunk is a buffer.)
• callback Function Call this function (optionally with an error argument and data) when you are done processing the supplied chunk.

So basically it's a function that you should call to signal to the stream that your're done processing. The reason you can't simply return from the function in order to signal that you're done processing is because you may have some asynchronous task (consult a database, send a packet over the network etc.) which will cause the function to return before the task is done.

Personally I think callback is a bad name for this. A better name would be something like Mocha's done() or a promise resolve(). Fortunately, the name of the argument is not decided by node.js or the through2 library, it's decided by you. So if I were you I'd write it like this:

fs.createReadStream('ex.txt')
.pipe(through2(function (chunk, enc, done) {
    for (var i = 0; i < chunk.length; i++)
      if (chunk[i] == 97)
        chunk[i] = 122; // swap 'a' for 'z' 

    this.push(chunk);

    done();
}))
.pipe(fs.createWriteStream('out.txt'))