NodeJS: invoking callback function inside a for loop

Question

Basically, I am trying to call a function, where a loop is running inside of which many callback function are present (callback hell).. as below:

for(var i=0;i<data.id.length;i++)
{
    DAO.getUserById(data.id[i],function(err,resp1)
    {
       /* some other work based on resp1 */
       DAO.getOtherData(resp1.username,resp1.userId,function(err,resp2)
       {
           /* similary some other work*/
       });
    });
}  

I have same pattern at several location in my app, and some time I faced issue with the callback, that for loop get over, but callback do not give response, that seems like DAO method has been called up but still waiting for response. Is there any optimized way, to overcome this issue ?

It would be nice to know if there is some javascript coding pattern available to overcome this issue.(other than any 3rd party library) Thanks

Answer 1

Have you heard about Deferreds or Promises? I think that's what you're looking for. In it's basic form it's basically a object which has two handler. One for failure and one for success.

But there are other helper functions like then or when which let you chain the functions in a more readable way. Have look at q or the jQuery implementation. For a very good introduction read the book Async JavaScript.

Edit:/

I did a little working example as js fiddle for you.

    var data = { id : [] };

    for(var i = 0; i < 10; i++) {
        data.id.push(i);
    }

    // DAO definition
    var DAO = {
        getUserById : function(id) {
            var deferred = $.Deferred();

            setTimeout(function() { 
                var isError = Math.floor(Math.random()*11) > 5;

                if(isError) {
                    deferred.reject("WOW - Much Handler - So error");
                } else {
                    deferred.resolve({
                        username :  'Max',
                        userId : id
                    }); 
                }
            }, 50);

            return deferred.promise();
        },

        getOtherData : function(username, userId) {
            var deferred = $.Deferred();

            setTimeout(function() {
                deferred.resolve((username + ' id: ' + userId));
            }, 20);

            return deferred.promise();
        }
    };



    function printResult(res) {
        $('#result').html($('#result').html() + '<br />' + res);
    };

    // DAO usage

    for(var i=0;i<data.id.length;i++)
    {
        DAO.getUserById(data.id[i])
        .done(function(res) {
            DAO.getOtherData(res.username, res.userId).done(function(result) {
                printResult(result);
            });
        })
        .fail(function(res) {
            printResult(res);
        });
    }

The great advantage of that is twofold:

1. You get seperation of error handler code and result handler code for free
2. It prevents you from nesting hell. (callbacks, in callbacks, in callbacks ...) Due to the deferreds you're able to factore the actual logic out.
3. Synchronizing of callbacks gets very easy because you only need to use when.

I used jQuerys Deferreds because jsFiddle has jquery in a dropdown box. You could use any implementation you want.

When you got the concept implementing it yourself shouldn't be too hard.

Answer 2

You can use the async.waterfall function that is exactly meant to deal with your issue. All functions are called in series, with the result of a function sent as a parameter to the next function. Here is a sample usage, from the documentation:

async.waterfall([
    function(callback){
        callback(null, 'one', 'two');
    },
    function(arg1, arg2, callback){
        callback(null, 'three');
    },
    function(arg1, callback){
        // arg1 now equals 'three'
        callback(null, 'done');
    }
], function (err, result) {
   // result now equals 'done'    
});