Node.js Http.request slows down under load testing. Am I doing something wrong?

Question

Here is my sample code:

var http = require('http');

var options1 = {
          host: 'www.google.com',
          port: 80,
          path: '/',
          method: 'GET'
        };

http.createServer(function (req, res) {

        var start = new Date();
        var myCounter = req.query['myCounter'] || 0;

        var isSent = false;
        http.request(options1, function(response) {
            response.setEncoding('utf8');
            response.on('data', function (chunk) {
                var end = new Date();
                console.log(myCounter + ' BODY: ' + chunk  + " time: " + (end-start) + " Request start time: " + start.getTime());

                if (! isSent) {
                    isSent = true;
                    res.writeHead(200, {'Content-Type': 'application/xml'});
                    res.end(chunk);
                }
            });
        }).end();


}).listen(3013);

console.log('Server running at port 3013');

What i found out is that, if I connect to other server, (google or any other), the response will get slower and slower to few seconds. It doesn't happen if I connect to another node.js server within the same network.

I use JMeter to test. 50 concurrent per sec with 1000 loop.

I have no idea what the problem is...

=========================

Further investigation:

I run the same script on Rackspace, and also on EC2 for testing. And the script will use http.request to connect to: Google, Facebook, and also my another script that simply output data (like hello world) which is hosted by another EC2 instance.

The test tool I just is jMeter on my desktop.

Pre-node.js test: jMeter -> Google Result: fast and consistent. jMeter -> Facebook result: fast and consistent. jMeter -> My Simple Output Script Result: fast and consistent.

Then I make a 50 Concurrent Threads /sec , with 100 loops, testing to my Rackspace nodejs, and then EC2 node.js which has the same king of performance issue jMeter -> node.js -> Google Result: from 50 ms goes to 2000 ms in 200 requsets.
jMeter -> node.js -> Facebook Result: from 200 ms goes to 3000 ms after 200 requsets.
jMeter -> node.js -> My Simple Output Script Result: from 100 ms goes to 1000 ms after 200 requsets.
The first 10-20 requests are fast, then start slowing down.

Then, when I change to 10 Concurrent Threads, things start changing.. The response is very consistent, no slow down.

Something to do with # of concurrent threads that Node.js (http.request) can handle.

------------ More --------------

I did more test today and here it is: I used http.Agent and increase the max socket. However, the interesting this is that, on one testing server (EC2), it improves a lot and no more slow down. HOwever, the other server (rackspace) only improves a bit. It still shows slow down. I even set "Connection: close" in the request header, it only improves 100ms.

if http.request is using connection pooling, how to increase it?

in both server, if I do "ulimit -a", the # of file open is 1024.

------------- ** MORE AND MORE ** -------------------

It seems that even I set maxSockets to higher number, it only works at some limit. There seems to be a internal or OS dependent socket limitation. However to bump it up?

------------- ** AFTER EXTENSIVE TESTING ** ---------------

After reading lots of post, I find out:

---

---

quoted from: https://github.com/joyent/node/issues/877

---

1) If I set the headers with connection = 'keep-alive', the performance is good and can go up to maxSocket = 1024 (which is my linux setting).

var options1 = {
                  host: 'www.google.com',
                  port: 80,
                  path: '/',
                  method: 'GET',
    **headers: {
            'Connection':'keep-alive'
    }**
                };

If I set it to "Connection":"close", the response time would be 100 times slower.

Funny things happened here:

1) in EC2, when I first test with Connection:keep-alive, it will take about 20-30 ms. Then if I change to Connection:Close OR set Agent:false, the response time will slow down to 300 ms. WIHTOUT restarting the server, if I change to Connection:keep-alive again, the response time will slow down even further to 4000ms. Either I have to restart server or wait for a while to get back my 20-30ms lighting speed response.

2) If I run it with agent:false, at first, the response time will slow down to 300ms. But then, it will get faster again and back to "normal".

My guess is connection pooling still in effect even you set agent:false. However, if you keep connection:keep-alive, then it will be fast for sure. just don't switch it.

---

---

---

Update on July 25, 2011

I tried the latest node.js V0.4.9 with the http.js & https.js fix from https://github.com/mikeal/node/tree/http2

the performance is much better and stable.

Answer 1

I solved the problem with

require('http').globalAgent.maxSockets = 100000

or

agent = new http.Agent()
agent.maxSockets = 1000000  # 1 million
http.request({agent:agent})