Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

node.js application suddenly loads CPU at 100% and hangs

I have simple application on node.js running on linux virtual machine. It listens for tcp messages and sending them to the clients using socket.io library. And after some time of low CPU usage it suddenly starts to load CPU higher and higher until application hangs up. The script is simple and I can not understand what's wrong with it.

var net = require('net');
var io = require('socket.io').listen(socketPort);

net.createServer(function (socket) {
    socket.setEncoding("utf8");
    socket.on('data', function (dataStr) {
        console.log("TCP dataStr " + dataStr);
        var data = JSON.parse(dataStr);
        io.sockets.in(data.room).emit('publish', data);
    });
}).listen(tcpPort);

io.sockets.on('connection', function (socket) {

    socket.on('subscribe', function (room) {
        console.log('subscribe room ' + room);
        if (Array.isArray(room)) {
            var i;
            for (i = 0; i < room.length; i++) {
                console.log('subscribe join room ' + room[i]);
                socket.join(room[i]);
            }
        } else if (typeof room === 'string') {
            console.log('subscribe join room ' + room);
            socket.join(room);
        }
    });

    socket.on('unsubscribe', function (room) {
        console.log('unsubscribe room ' + room);

        if (Array.isArray(room)) {
            var i;
            for (i = 0; i < room.length; i++) {
                console.log('unsubscribe leave room ' + room[i]);
                socket.leave(room[i]);
            }
        } else if (typeof room === 'string') {
            console.log('unsubscribe leave room ' + room);
            socket.leave(room);
        }

    });

});

Also with cluster module I tried to run multiple workers that communicate with clients. And every worker after some time hangs own CPU core at 100% with time difference in about a minute.

UPD: Client code (run in browser):

    socketObj = new function() {
        var that = this;
        that.socket;

        that.init = function(nodeServerUrl, rooms, onPublishFunc) {
            that.socket = io.connect(nodeServerUrl);
            that.socket.emit('subscribe', rooms);

            that.socket.on('publish', function(data) {
                        onPublishFunc(data);
            });
        };
        that.subscribe = function(room) {
            that.socket.emit('subscribe', room);
        };
        that.unsubscribe = function(room) {
            that.socket.emit('unsubscribe', room);
        };
    }

    ...

    try {
        socketObj.init('application url', ["room1", "room2"], nodeJsCallback);
    } catch(err) {
    }

    ...

    nodeJsCallback = function(jsonData) {
        //Only updates data on UI, no subscribing, unsubscribing, emitting etc.
        ...
    }

UPD2: I tried to reproduce the problem with synthetic tests on production machine and on my local Windows machine. I have done some stress testing:

  1. Multiple client socket connections
  2. Multiple static data downloads (socket.io script for browser)
  3. Increased frequence of tcp updates.

After few hours of testing I failed to reproduce. But when it is running on production with real users, it is hanging up earlier or later.

I'm starting to think this is either environment or specific message problem. Probably next things I'll try are:

  1. Update Node.js to current version
  2. Try to log all data transfer and replay it later hoping hanging will reproduce
like image 265
ooops Avatar asked May 22 '13 21:05

ooops


People also ask

Why is Node taking up so much CPU?

Huge payloads from Node. js services also can be a problem, because Node. js stringifies objects to JSON first and then sends them to the client. All of these operations can cause high CPU, make sure that payload size is not huge, use pagination, and don't prepopulate unnecessary data.

Why is NodeJS bad for CPU intensive applications?

However, there is a downside to Node. js being single-threaded. The single-threaded implementation makes Node a bad choice for CPU-intensive programs. When a time-consuming task is running in the program it blocks the event loop from moving forward for a longer period.

Is NodeJS CPU intensive?

If my workload is mostly done in Database does nodejs waiting for database to complete does that count as cpu intensive task for nodejs? No. As long as the database is running in another process, that will not count as a CPU intensive task for node. js and will not block the event loop in node.

How much RAM does NodeJS need?

256 MB is sufficient amount of RAM to run Node. js (e.g. on Linux VPS instance), assuming no other memory-hog software is run.


1 Answers

Changed Nodejs from version v0.10.4(Stable) to v0.11.2(Unstable). All woking good so far, consuming 1-2% CPU. Now we are testing on v0.10.8(Stable).

UPD On v0.10.8 application is stable too.

Even though the problem dissapeared on v0.10.4(Stable), it is still very strange and discouraging.

like image 61
ooops Avatar answered Oct 03 '22 20:10

ooops