node express/request: piping a POST request with body parsing

Question

I'm attempting to pipe a request for handling by a remote server, along the following lines:

var destination = request(url);
req.pipe(destination).pipe(res);

This works just fine for GET requests. But for POST requests I'm struggling. An important point to note, I think, is that for POST requests I'm using a body parser before my POST route handler in order to extract the body from the POST request... it's just a basic text body parser because the body contains plain text:

var postRouteHandler = someFunction;
var bodyParser = require('body-parser');
var textParser = bodyParser.text({
    limit: '50kb'
});
app.use('/postRoute', [textParser, postRouteHandler]);

From this issue and this issue it seems to me that doing any processing on the POST request before you pipe it will cause a problem. Indeed, when I remove the parser, the piping seems to work OK.

The problem is that I need to examine the body first, to do some initial processing and then to determine whether or not to pipe the request on to the remote server at all. So I need to parse the body before piping.

Is there any way around this problem?

Answer 1

The issue is that with streams (like req), once you've read it you can't reset it.

Because body-parser has read the stream already, piping it won't work because that will try to read the stream again (which at that point has been exhausted).

A workaround would be take the text read by body-parser, and create a minimal req "clone" in order for request to be able to pass the request along:

var intoStream = require('into-stream');
var bodyParser = require('body-parser');
var textParser = bodyParser.text({ limit: '50kb' });

var postRouteHandler = function(req, res) {
  let text = req.body;
  if (! shouldPipe(text)) {
    return res.sendStatus(400); // or whatever
  }

  // Here's where the magic happens: create a new stream from `text`,
  // and copy the properties from `req` that `request` needs to pass
  // along the request to the destination.
  let stream     = intoStream(text);
  stream.method  = req.method;
  stream.headers = req.headers;

  // Pipe the newly created stream to request.
  stream.pipe(request(url)).pipe(res);
};
app.use('/postRoute', [textParser, postRouteHandler]);

Answer 2

You should create your own middleware if the parser is causing the problem. In all honesty you may want to parse the body a different way than the body parser.

Given the limited knowledge about what the rest of your project is doing you could create middleware to hook the body parser middleware and just send a clone of request. This however is not very efficient but will work and may help point you in the right direction.

var postRouteHandler = someFunction;
var bodyParser = require('body-parser');
var textParseMiddleware = function (req, res, next) {

  var requestclone = _.clone(req);
  var textParser = bodyParser.text({
    limit: '50kb'
  });

  textParser(requestclone, res, function(){
    console.log('examine the body here', requestclone.body);
  });


}

app.use('/postRoute', [textParserMiddleWare, postRouteHandler]);

I have not tested the above code.