How can I gunzip POST request data in express?

Question

I am trying to build a server that can accept gzipped POST data with express. I think I could just write my own middleware to pipe the request stream to a zlib.createGunzip() stream. The question is, how can I achieve that, afterwards, the express.bodyParser() middleware is still able to parse my gunzipped POST data?

I tried to replace the original request stream methods by the ones of the zlib stream, but that just made the bodyParser return a "Bad Request" Error:

var express = require('express');
var app = express();

function gUnzip(req, res, next) {
  var newReq;
  if (req.headers['content-encoding'] === 'gzip') {
    console.log("received gzipped body");
    newReq = req.pipe(zlib.createGunzip());
    Object.getOwnPropertyNames(newReq).forEach(function (p) {
      req[p] = newReq[p];
    });
  }
  next();
}

app.use(gUnzip);
app.use(express.bodyParser());

app.listen(8080);

Is there a way to make this work without rewriting the bodyParser() middleware within my own middleware?

EDIT: This is the same question: Unzip POST body with node + express. But in the answer he just does in his own middleware what the express.bodyParser() should do, which is what I want to avoid. I am looking for a way to simply unzip the request data from the stream and then pass it to the bodyParser(), which expects a stream itself, as can be seen at http://www.senchalabs.org/connect/json.html.

Answer 1

compressed request bodies are generally not used because you can't negotiate content encodings between the client and server easily (there's another stackoverflow question about that i believe). most servers don't support compressed request bodies, and the only time you really need it is for APIs where the client will send large bodies.

body-parser, specifically raw-body, does not support it because the use-case is so minimal, though i've though about adding it. for now, you'll have to create your body-parser. fortunately, that's easy since you can just fork body-parser and leverage raw-body. the main code you'll add around https://github.com/expressjs/body-parser/blob/master/index.js#L80:

    var zlib = require('zlib')

        var stream
        switch (req.headers['content-encoding'] || 'identity') {
            case 'gzip': 
                stream = req.pipe(zlib.createGunzip())
                break
            case 'deflate': 
                stream = req.pipe(zlib.createInflate())
                break
            case 'identity': 
                break
            default:
                var err = new Error('encoding not supported')
                err.status = 415
                next(err)
                return
        }

        getBody(stream || req, {
            limit: '1mb',
            // only check content-length if body is not encoded
            length: !stream && req.headers['content-length'],
            encoding: 'utf8'
        }, function (err, buf) {

        })