HTTP request stream not firing readable when reading fixed sizes

Question

I am trying to work with the new Streams API in Node.js, but having troubles when specifying a fixed read buffer size.

var http = require('http');
var req = http.get('http://143.226.75.100/waug_mp3_128k', function (res) {
    res.on('readable', function () {
        var receiveBuffer = res.read(1024);
        console.log(receiveBuffer.length);
    });
});

This code will receive a few buffers and then exit. However, if I add this line after the console.log() line:

res.read(0);

... all is well again. My program continues to stream as predicted.

Why is this happening? How can I fix it?

Answer 1

It's explained here.

As far as I understand it, by reading only 1024 bytes with each readable event, Node is left to assume that you're not interested in the rest of the data that's in the stream buffers, and discards it. Issuing the read(0) (in the same event loop iteration) 'resets' this behaviour. I'm not sure why the process exits after reading a couple of 1024-byte buffers though; I can recreate it, but I don't understand it yet :)

If you don't have a specific reason to use the 1024-byte reads, just read the entire buffer for each event:

var receiveBuffer = res.read();

Or instead of using non-flowing mode, use flowing mode by using the data/end events instead:

var http = require('http');
var req = http.get('http://143.226.75.100/waug_mp3_128k', function (res) {
  var chunks = [];
  res.on('data', function(chunk) {
    chunks.push(chunk);
    console.log('chunk:', chunk.length);
  });
  res.on('end', function() {
    var result = Buffer.concat(chunks);
    console.log('final result:', result.length);
  });
});