async queue never triggers drain randomly

Question

I'm continuing to play around with nodejs. So below is a brief summary of my launch.js app

var counter = 0;
var queue = async.queue(function(objectWithVariousProperties, callback) {
    methodDoingVariousAsynchronousStuff(objectWithVariousProperties.link, function(err,result) 
        counter++;
        callback();
    });
}, 20);

queue.drain = function() {
    log.debug("Finished Queue", {
        "Objects processed" : counter
    });
};
//TO various iterations over several array and construct some object and then push them in the queue via 
queue.push(media_object)

My issue is that 80% of the time the process works. the application launch via the shell command nodejs launch.js , process the queue, when queue is finished it displays the 'Finished Queue' then back to shell.

However, sometimes the drain event is never triggered but the application finished and back to the shell without any exception. I inspected during several hours my code to find if there was a missing callback, but it does not seems so.

Any idea which can explain this strange random behaviour ? Or any tools which would help to find the leaking code ?

As I said, sometimes, the queue process more than 10000 objects without any problem and sometimes when I launch it, it just does not trigger the drain event but the process stop (again without any exception or error or anything, just a normal stop)

I spent a couple of evenings trying to figure out why, so if you guys had any ideas it would be gladly welcomed.

Answer 1

I cannot give you an appropriate answer, but help you to debug your code instead. When node.js is not exiting, there's something left in the event loop.

This tool may help you to find it: https://github.com/mafintosh/why-is-node-running

I'm pretty sure that your problem may belong to something in methodDoingVariousAsynchronousStuff() - the async.queue code looks almost fine.

Maybe something like an uncalled callback, unhandled error or an open network connection.