Multiple Concurrent Operations in Node.js

Question

I'm wondering what the best way to handle multiple concurrent operation in node.js. For example, let's say a given HTTP request needs to make 3 database queries before it can render the response, but each of the requests are independent. My current naive implementation works like this (in pseudo code)

doRequest1(query, function (response1) {
   doRequest2(someQuery, function (response2) {
      doRequest3(someQuery, function (response3) {
          renderHTML(response1,response2,response3);
      });
   });
});

Is there a better way to do this? It seems wasteful to wait for the response to each of the queries to start the next one.

Answer 1

You should use a flow control library, like async

https://github.com/caolan/async

Or step

https://github.com/creationix/step

Edit: added an example with async

async.parallel([
    function(callback){
        doRequest1(query, callback);
    },
    function(callback){
        doRequest2(query2, callback);
    },
    function(callback){
        doRequest3(query3, callback);
    },
],
function(err, results){
    if (err) {
        renderError(err);
    } else {
        renderHTML(results[0], results[1], results[2]);
    }
});

If you want to make each function run as series (wait for the previous function to complete), you may use async.series instead of async.parallel. If queries depend on the result of previous queries, you may use async.waterfall. If there is a complex dependency graph between queries, you may use async.auto.