Whats is back pressure on streams processing

Question

I start learning node and streams seems to be something that you use a lot, in most of the documentation that I had read is mention the "back-pressure problem" when you are processing big sized files but I haven't fount a clear explanation of what exactly this problem is. Also I have read that use pipes can help with this problem, bu how exactly pipes fix the back-pressure problem?

thanks for any explanation in advance.

Answer 1

Backpressure is when you write to a stream faster than the other process can handle/consume Using Pipes, you can control the flow, pause and resume the stream; Here is an example implementing backpressure in nodejs

var http = require('http'),
    fs = require('fs');

var server = http.createServer(function(request, response) {
  var file = fs.createWriteStream('upload.jpg'),
      fileBytes = request.headers['content-length'],
      uploadedBytes = 0;

  request.on('data', function(chunk) {
    uploadedBytes += chunk.length;
    var progress = (uploadedBytes / fileBytes) * 100;
    response.write('progress: ' + parseInt(progress, 10) + '%\n');

    var bufferOK = file.write(chunk);

    if (!bufferOK) {
      request.pause();
    }
  });

  file.on('drain', function() {
    request.resume();
  });

  request.on('end', function() {
    response.end('upload complete\n');
  });

});

server.listen(8080);

Ben foster solution - Souce: https://gist.github.com/benfoster/9543337