Much slower response in node.js HTTP server with smaller files

Question

I am developing a performance test for a simple HTTP server in node.js:

var http = require('http');
var fs = require('fs');

var NO_CACHE = true;
var fileCache;
var sendFile = function(conn, file) {
    conn.writeHead(200, {'Content-Type': 'text/html', 'Content-Length': file.length});
    conn.write(file);
    conn.end();
}
http.createServer(function (req, res) {
    if (NO_CACHE || fileCache == undefined) {
        fs.readFile('index.html', function(err, file) {
        fileCache = file;
        sendFile(res, fileCache);
        });
    } else {
        sendFile(res, fileCache);
    }
}).listen(8080, 'localhost');

(code can also be found here).

The server reads always the file "index.html" and returns it. I realize that the server give the result in 1-3 ms if the file is greater than or equal to 65483 bytes (very close to 2^16 but not exact), and if the file is smaller, it last 38-40 ms to give a response (index.html file with exactly 65483 bytes can be found here). Headers in the response are about 128 bytes:

Content-Type: text/html
Content-Length: 65483
Date: Thu, 14 May 2015 13:58:21 GMT
Connection: keep-alive

The first two headers are set by the server, and the last two are set automatically by any middleware. The file size plus the first two headers are 65533 bytes, pretty close to 65535 (2^16 - 1), the difference could be because of carriage return in those two headers.

This behaviour looks strange to me, as bigger files should last more time being read, and the difference in time is very big.

I used node.js 0.10.38 and 0.12.2 with same results. For the performance tests, I used jMeter 2.13.

For the record, this behaviour does not happen using vert.x with same file and same test plan in jMeter (with vert.x server, bigger files take more time to be read).

If anyone knows what the cause could be (and how to avoid it for small files), I am very curious to know.

Answer 1

As bayou.io pointed, the behaviour is caused by TCP noDelay. I added to the server:

server.on("connection", function (socket) {
    socket.setNoDelay(true);
});

and it is giving the response within 1-3 ms as expected.