How to sync JavaScript callbacks?

Question

I've been developing in JavaScript for quite some time but net yet a cowboy developer, as one of the many things that always haunts me is synching JavaScript's callbacks.

I will describe a generic scenario when this concern will be raised: I have a bunch of operations to perform multiple times by a for loop, and each of the operations has a callback. After the for loop, I need to perform another operation but this operation can only execute successfully if all the callbacks from the for loop are done.

Code Example:

 for ... in ... {
   myFunc1(callback); // callbacks are executed asynchly
 }

 myFunc2(); // can only execute properly if all the myFunc1 callbacks are done

Suggested Solution:

Initiate a counter at the beginning of the loop holding the length of the loop, and each callback decrements that counter. When the counter hits 0, execute myFunc2. This is essentially to let the callbacks know if it's the last callback in sequence and if it is, call myFunc2 when it's done.

Problems:

1. A counter is needed for every such sequence in your code, and having meaningless counters everywhere is not a good practice.
2. If you recall how thread conflicts in classical synchronization problem, when multiple threads are all calling var-- on the same var, undesirable outcomes would occur. Does the same happen in JavaScript?

Ultimate Question:

Is there a better solution?

Answer 1

The second problem is not really a problem as long as every one of those is in a separate function and the variable is declared correctly (with var); local variables in functions do not interfere with each other.

The first problem is a bit more of a problem. Other people have gotten annoyed, too, and ended up making libraries to wrap that sort of pattern for you. I like async. With it, your code might look like this:

async.each(someArray, myFunc1, myFunc2);

It offers a lot of other asynchronous building blocks, too. I'd recommend taking a look at it if you're doing lots of asynchronous stuff.

Answer 2

The good news is that JavaScript is single threaded; this means that solutions will generally work well with "shared" variables, i.e. no mutex locks are required.

If you want to serialize asynch tasks, followed by a completion callback you could use this helper function:

function serializeTasks(arr, fn, done)
{
    var current = 0;

    fn(function iterate() {
        if (++current < arr.length) {
            fn(iterate, arr[current]);
        } else {
            done();
        }
    }, arr[current]);
}

The first argument is the array of values that needs to be passed in each pass, the second argument is a loop callback (explained below) and the last argument is the completion callback function.

This is the loop callback function:

function loopFn(nextTask, value) {
    myFunc1(value, nextTask);
}

The first argument that's passed is a function that will execute the next task, it's meant to be passed to your asynch function. The second argument is the current entry of your array of values.

Let's assume the asynch task looks like this:

function myFunc1(value, callback)
{
  console.log(value);
  callback();
}

It prints the value and afterwards it invokes the callback; simple.

Then, to set the whole thing in motion:

serializeTasks([1,2, 3], loopFn, function() {
    console.log('done');
});

Demo

To parallelize them, you need a different function:

function parallelizeTasks(arr, fn, done)
{
    var total = arr.length,
    doneTask = function() {
      if (--total === 0) {
        done();
      }
    };

    arr.forEach(function(value) {
      fn(doneTask, value);
    });
}

And your loop function will be this (only parameter name changes):

function loopFn(doneTask, value) {
    myFunc1(value, doneTask);
}

Demo