Bytes sent/received for Node.js HTTP request

Question

Once an HTTP request has been served, I would like to log the number of bytes sent/received.

A simple source for this data is req.connection.bytesRead/.bytesWritten. However, this is problematic for HTTP 1.1 keep-alive connections, as the same socket can be used for multiple requests. I need to log per-request, not per-connection.

The solution must lie on the HTTP side of things, but I see no methods documented for getting the data I need.

What is the proper way to calculate bytes read/written for HTTP requests served by Node.js's http.Server?

Answer 1

Unfortunately, I never found a proper way to do this. I've resorted some fairly terrible duck punching, but it works for my particular use case. In case anyone else stumbles along with this problem, you can start with this and refine from there.

Module #1: "Extra Events"

All this module does is make the response object emit a finishBeforeSocketDestroy event. Since I needed this event in a few places in my application, I effectively made a separate module just for this duck punch. app.use() it before Module #2.

module.exports = function (req, res, next) {
    var end = res.end;

    res.end = function () {
        res.end = end;
        res.emit('finishBeforeSocketDestroy');
        res.end.apply(this, arguments);
    }

    next();
}

Module #2: "Stats"

This module creates a req.stats object, containing all sorts of useful goodies for tracking bandwidth usage during usage of the connection, and after it is finished.

var pollTime = 1000;
module.exports = function (req, res, next) {
    var pollInterval;

    function pollStats () {
        if (typeof req.stats._lastMeasuredTime === 'object') {
            var secondsSinceLastMeasurement = ((new Date() - req.stats._lastMeasuredTime) / 1000);
            req.stats.averageRate = {
                read: (req.socket.bytesRead - req.stats.bytesRead) / secondsSinceLastMeasurement,
                write: (req.socket.bytesWritten - req.stats.bytesWritten) / secondsSinceLastMeasurement
            };
        }
        req.stats._lastMeasuredTime = new Date();
        req.stats.bytesRead = req.socket.bytesRead;
        req.stats.bytesWritten = req.socket.bytesWritten;
    }

    req.stats = {
        startTime: new Date(),
        endTime: null,
        averageRate: {read: null, write: null},
        bytesRead: req.socket.bytesRead,
        bytesWritten: req.socket.bytesWritten,
        _lastMeasuredTime: new Date()
    };

    pollInterval = setInterval(pollStats, pollTime);

    res.on('finishBeforeSocketDestroy', function () {
        clearInterval(pollInterval);
        pollStats();
        req.stats.endTime = new Date();
    });

    next();
}

Like I said... messy. I'm only posting it as duck punching may be your only option. Also beware that socket may get re-used for multiple HTTP requests, which could cause you to double-count some bytes if you're not careful.

Answer 2

Just store traffic value after each response and calculate difference in 'finish' or 'end' handler:

// server.onRequest:
  ...
  req._prevBytesWritten = 0;

// response.onFinish/onEnd:
  ...
  responseLen = req.socket.bytesWritten - req._prevBytesWritten;
  req._prevBytesWritten = req.socket.bytesWritten;