nodejs streams vs callbacks

Question

Im reading this article: http://elegantcode.com/2011/04/06/taking-baby-steps-with-node-js-pumping-data-between-streams/ and having some slight troubles understanding streams.

Quote:

"Suppose we want to develop a simple web application
that reads a particular file from disk and send it to the browser.
The following code shows a very simple and naïve implementation
in order to make this happen."

So the code sample is as follows:

var readStream = fileSystem.createReadStream(filePath);
readStream.on('data', function(data) {
    response.write(data);
});

readStream.on('end', function() {
    response.end();        
});

Why would we use that above way when we could simply do:

fs.readFile(filePath, function(err, data){
  response.write(data);
  response.end();
});

When or why would I use streams?

Answer 1

You'd use stream when working with large files. With a callback, all of the file's contents must be loaded into memory at once, while with a stream, only a chunk of the file is in memory at any given time.

Also, the stream interface is arguably more elegant. Instead of explicitly attaching data, drain, and end callbacks, you can instead use pipe:

var readStream = fileSystem.createReadStream(filePath);
readStream.pipe(response);

Answer 2

One big reason is that you can begin doing work on the data before it is all in memory. Think "streaming video", where you can begin watching a clip while it is still loading. In many use cases, a stream will allow you to begin processing data from a file before you have loaded the entire thing.

The other common use case is when you only want to read an object up until you detect some condition in the data. Say you needed to check to see if a large file contained the word "rabbit". If you use a callback pattern, you will need to read the entire file into memory, then go through the file and check to see whether or not the word is inside. With a stream, you might detect the word on line 5 of the file, then you are able to close the stream, without loading the entire thing.

There are obviously many more complex use cases, and there are still plenty of times where a callback still makes more sense for simplicity (such as if you needed to count the total times "rabbit" appeared, in which case you have to load the entire file anyway).