Socket.io and node.js, can't understand memory usage

Question

I currently use socket.io v1.4.2 and node.js v0.10.29 on my server. I try to track a memory leak in my app, I'm not sure, but I think socket.io is a part of my problem.

So here a code of the server (demo example):

var server = require ('http').createServer ();
var io = require ('socket.io')(server);

io.on ("connection", function (socket) {

    socket.on ('disconnect', function (data) { /* Do nothing */ });
});

Step 1 : Memory : 58Mb

Step 2 : I create A LOT of clients (~10000), Memory : 300 Mb

Step 3 : I close all clients and waiting the GC doing his work

Step 4 : I look at my memory : 100 Mb :'(

Step 5 : Same as step 2 and 3

Step 6 : Memory 160Mb...

And so on and the memory keeps growing.

I presume the GC was lazy so I retry with the follow code :

setInterval (function () {
    global.gc ();
}, 30000);

And I start my app.js with :

node --expose-gc app.js

But I had the same result.

Finally I try

var server = require ('http').createServer ();
var io = require ('socket.io')(server);

clients = {};
io.on ("connection", function (socket) {

    clients[socket.id] = socket;        
    socket.on ('disconnect', function (data) { 
        delete clients[socket.id];
    });

});

And I had the same result. How can I free this memory ?

EDIT

I create snapshot directly on my main source.

I install the new module with the follow command :

 npm install heapdump

I write in my code this :

 heapdump = require ('heapdump');
 setInterval (function () { heapdump.writeSnapshot (); }, 30000);

It took heapdump of the program every 30 seconds, and save it in the current directory. I read the heapdump with the module 'profiles' of Chrome.

So, the issue is probably socket.io, cause I found many strings not released, that I emit with socket. Perhaps I don't write the emit in the right way ? I do that :

     var data1 = [1, 2, 3];
     var data2 = [4, 5, 6];
     var data3 = [7, 8, 9];
     socket.emit ('b', data1, data2, data3);
     data1 = [];
     data2 = [];
     data3 = [];

And in my snapshot say that the program keeps the following string: "b [1, 2, 3] [4, 5, 6] [7, 8, 9]" in my memory, millions of time What I'm suppose to do ?

I also make an another (perhaps stupid ?) test :

    var t1 = new Date ();

    ...

    var t2 = new Date ();
    var data1 = [1, 2, 3];
    var data2 = [4, 5, 6];
    var data3 = [7, 8, 9];
    socket.emit ('b', data1, data2, data3);
    data1 = [];
    data2 = [];
    data3 = [];

    console.log ("LAG: " + t2 - t1);
    t1 = new Date ();

I had this result :

    LAG: 1
    LAG: 1
    ...
    LAG: 13
    LAG: 2
    LAG: 26
    LAG: 3
    ...
    LAG: 100
    LAG: 10
    LAG: 1
    LAG: 1
    LAG: 120
    ...
    keeps growing

EDIT 2 :

This is my entire test code :

/* Make snapshot every 30s in current directory */
heapdump = require ('heapdump');
setInterval (function () { heapdump.writeSnapshot (); }, 30000);

/* Create server */
var server = require ('http').createServer ();
var io = require ('socket.io')(server);
var t1 = new Date ();

clients = {};
io.on ("connection", function (socket) {

    clients[socket.id] = socket;        
    socket.on ('disconnect', function (data) { 
        delete clients[socket.id];
    });
});

setInterval (function () { 

    var t2 = new Date ();
    for (c in clients) {
        var data1 = [1, 2, 3];
        var data2 = [4, 5, 6];
        var data3 = [7, 8, 9];
        clients[c].emit ('b', data1, data2, data3);
        data1 = [];
        data2 = [];
        data3 = [];
    }
    console.log ("LAG: " + t2 - t1);
    t1 = new Date (); 
}, 100);

I don't give the code of client. Because I assume that: if the problem is in the client, so it's a security issue. In fact, it will be an easy way to saturate the RAM of the server. So it's a kind of better DDOS, I juste hope the problem is not in the client.

Answer 1

Edit based on the server code you included

On your server:

c.emit ('b', data1, data2, data3);` 

should be changed to:

clients[c].emit('b', data1, data2, data3);

c.emit() was probably throwing an exception because c is the socket.id string and strings don't have a .emit() method.

Original answer

What you need to determine is whether the growth in memory is actually memory that is allocated within the node.js heap or if it's free memory that has just not been returned to the operating system and is available for future reuse within node.js? Measuring the memory used by the node.js process is useful to see what it has taken from the system and that should not continually go up forever over time, but it doesn't tell you what is really going on inside.

FYI, as long as your node.js app has a few free cycles, you shouldn't ever have to manually call the GC. It will do that itself.

The usual way to measure what is being used within the node.js heap is to take a heap snapshot, run your Steps 1-4, take a heap snapshot, run those steps again, take another heap snapshot, diff the snapshots and see what memory in the node.js heap is actually different between the two states.

That will show you what is actually in use within node.js that has changed.

Here's an article on taking heap snapshots and reading them in the debugger: https://strongloop.com/strongblog/how-to-heap-snapshots/