For loop in callback. Blocking?

Question

//defining schemas, models, db connections, etc..

http.createServer(function (req, res) {
  // some irrelevant stuff..

  Model.find({name : regex}).exec(function (err, results) {  

    var localArray = []; 

    for (var i = 0, len = results.length; i < len; ++i) {
      localArray.push(results[i].name);
      localArray.push(results[i].id);
    };  // is this for loop blocking?

  // some more irrelevant stuff..
  });
}).listen(8080);

My callback has a for loop which can be long (results.length = 100 sometimes).

Have I written blocking code? If yes, how can I make it non-blocking?

Answer 1

To make the long story short, the answer is yes, it is blocking. Any request you receive while this loop is running will be queued. Use child process to unlock your parent code. It requires you machine to be multicore (not to spawn a process but to be efficient).

The long story:

JavaScript (and Node consequently) is single-threaded. What it means in context of this question - while you have your loop running no other function can be called. The only way to unblock it is to use a child process. You have two options, each with it's own strength and faults.

1. Run a cluster, so that while one node process is "blocked" you have another process to serve incoming requests.
2. Spawn a child process to run something heavy for you and then report result back.

The first option is really simple and requires you to wrap your server in cluster http://nodejs.org/api/cluster.html However chances are that you "block" entire cluster with this loop running within each node process.

The second option is more complicated to implement and requires much more resources. But it is amazing when you have to do something very heavy and/or potentially memory-leaking. What this allows you to do is to create a separate Node process (which will require some RAM and take some milliseconds to start so you cannot spawn millions of these guys), pass some arguments to it and wait for result to come. Your parent process will be able to serve upcoming requests. But don't forget to kill child process once it's done. http://nodejs.org/api/child_process.html

Whatever you choose, note that you still running on the same machine and next bottleneck you encounter is your machine resources. Anyway run some benchmarks. Iteration over a hundred items is not so heavy to spawn a child, but if you iterate over thousands or more?...

p.s. Inverted loops run much faster:

for (var i = results.length; i > 0; i--) {
  //Do something
};