Why does my code disconnects a Socket.io connection in Node?

Question

First of sorry I'am not native english and thanks for interest! I don't know why when I perform some semi-heavy operations on arrays my browser break connection with Socket.io. Browser receives disconnect event, then it reconnects but shouldn't disconnect in the first place.

The code contains a couple nested loops, I know it's not the best practice but I did't figure out any other way of doing the task. I started testing on large arrays of users (about 10k) and the problem started occurring, so I guess it's a performance issue.

I was trying to find memory leeks but with no success, its almost like for the time of performing operations app freezes and breaks Socket.io.

Thank you for trying to help.

var pushUsersToCorrectLeagues = function(appConfig, callback, usersArray, newSeason, db) {
    //determinate which league is the last one
    var leaguesOrderArray = [];
    _.each(newSeason.leagues, function(league) {
        leaguesOrderArray.push(league.key);
        return false;
    });

    var lastLeagueKey = _.last(leaguesOrderArray.sort(function(a, b) {
        return a - b;
    }));

    _.each(usersArray, function(user) {
        var keyUserData = {};
        keyUserData.userLogin = user.userLogin;
        keyUserData._id = user._id;
        keyUserData.avatarUrl = user.avatarUrl;
        keyUserData.isActive = true;
        keyUserData.scores = {};

        //now put users in correct leagues
        var userPushed = false;
        var endedInLeague = null;


        //if found a user in league
        _.each(newSeason.leagues, function(league) {
            _.find(league.users, function(alredyInLeagueUser, i) {
                if (alredyInLeagueUser.userLogin == user.userLogin) {
                    newSeason.leagues[league.key].users[i] = keyUserData;
                    userPushed = true;
                    endedInLeague = league.key;
                    return true;
                }
            });

            return false;
        });

        //user wasnt found in any league, has to be pushed to the last one then
        if (!userPushed) {
            newSeason.leagues[lastLeagueKey].users.push(keyUserData);
            endedInLeague = lastLeagueKey;
        }

        if (endedInLeague) {
            process.nextTick(function() {
                var callback = function(updatedUser) {
                    //console.log('updatedUser', updatedUser);
                    return false;
                };
                updateUserCurrentLeague(callback, user._id, endedInLeague, db);
            });
        }

        return false;
    });

    callback(usersArray, newSeason);
    return false;
};

Answer 1

It sounds like your operation is taking longer than the socket heartbeat interval so one end of the connection thinks the connection has been dropped.

You have the following choices:

1. You can lengthen the heartbeat interval on the socket (really just a bandaid, not the best fix).
2. You can offload your time consuming node operation to a child process and let it run, collect the results and send the results back in JSON. This allows your main node process to stay responsive the whole time.
3. You can break your time consuming operation up into chunks where each chunk takes no longer than 500ms to run and the server will then be able to process other events while the time consuming operation is going on. This requires reworking how you process things because you can't use .each() loops. Instead, you have to store some state somewhere, use counters, do a certain amount of work, update the state, set a timer to do some other work after other events have been processed, do another chunk of work, repeat until all work done.

FYI, here's an example of how a very large array can be processed in chunks: Best way to iterate over an array without blocking the UI. This was coded for the browser, but the concept is the same in node.