How to avoid the callback "waterfall"?

Question

One of the reasons that I tend to dread writing Javascript for anything other than relatively trivial bits of functionality is that I've never found a decent approach for avoiding the callback waterfall when one thing really depends on another. Is there any such approach?

I'm working on a Titanium app right now and bumping into this real world scenario:

I have a set of facilities for which I need to the calculate distance from the user's current location. This entails getting the user's current position (which only needs to happen once) and, while looping through the facility locations, getting the position for each one and calculating the distance. The APIs that retrieve locations (long/lat) are asynchronous so the "easy" approach looks like this (pseudo-code follows):

foreach facility {
  API.getCurrentLocation( function( location ) { // async, takes a callback fxn arg
    var here = location.coordinates;

    API.getFacilityLocation( function( e ) { // async, takes a callback fxn arg
      var there    = e. coordinates;
      var distance = API.calculateFrom( here, there );
    });
  });
}

Because this is all in a loop, though, I'm calculating my current position each time -- more work than I really need to do. I haven't yet managed to refactor this in such a way that I'm only getting the current location once and still having that location available for the distance calculation.

Given the explosion of languages that support lambdas and closures, I keep thinking that someone must have found an approach to keep these waterfalls manageable, but I have yet to find a good explanation of how to organize such a solution.

Any suggestions or hints?

Any insight would be tremendously appreciated.

Answer 1

The basic strategy: don't use anonymous functions for callbacks, and move the loop into the callback that runs when the current location is returned.

Example:

function recieveCurrentLocation(location) { // async, takes a callback fxn arg
    var here = location.coordinates;

    // Have to define this callback in the inner scope so it can close
    // over the 'here' value
    function recieveFacilityLocation(e) {
        var there    = e. coordinates;
        var distance = API.calculateFrom( here, there );
    }

    foreach facility {
        API.getFacilityLocation(recieveFacilityLocation);
    }
}

API.getCurrentLocation(recieveCurrentLocation);

Answer 2

You must start to think more event-oriented. Define function for each callback level and provide it as argument when needed, and don't think of it as callback waterfall. Note that you have the same in each non-batch process: you wait for user actions, there's a big event loop that runs action, which waits for other user action, which runs another event processing action etc.

Simply do what can be done at the moment and for anything asynchronous register handler. User actions and async responses from computer systems are not so different :)

Answer 3

There are two separate problems here. The first is nesting callbacks (in an "watterfall" manner) and the second is calling an async function without knowing what continutation you want to pas it.

To avoid nesting hell the basic idea is to use names functions instead. So

f1(arg1, function(){
    arg2 = g(arg1);
    f2(function(){
        ...use arg2
    });
});

Can become

var arg2;
f1(arg1, afterf1);

function afterf1(){
    arg2 = g(arg1);
    f2(afterf2);
}

function afterf2(){
    ...use arg2;
}

Note that the only other main refactoring is that we need to move all the variables the inner functions closed over to the outer scope, since the inner functions won't be inner functions anymore (do try to keep shared variables at a minimum - there are many tricks to refactor them into more mantainable code if you feel you are starting to get too many of them).

Now, the other problem is having a callback you don't know when you you use the value of.

In a synchronous case you can do

var x = f();

and whoever wants x can just just access it anytime afterwards.

But in the async case you are limited to doing

f(function(x){
   ...use x here
});

And the only code that will ever be able to see x will be controlled by this callback.

The trick then is having a way to add extra "real" callbacks afterwards and have the callback you passed to the original function just pass on the result to the interested parties, instead of using it directly.

var got_result = false;
var result = null;
var waiting_for_result = [];

function register_callback(f){
    if(got_result){
        f(result);
    }else{
        waiting_for_result.push(f);
    }
}

var real_callback = function(x){
    got_result = true;
    result = x;
    for(var i=0; i< waiting_for_result.length; i++){
        waiting_for_result[i](result);
    }
}

//

API.getCurrentLocation(real_callback);
foreach facility {
    register_callback(function(location){
        ...stuff
    })

Of course, since doing this is a repetitive PITA, there are many Promise libraries that do precisely this. They mostly also have neat methods that allow you do the non-nesting "named callbacks" pattern with anonymous functions as well.

For example, in Dojo this might look like

var location_promise = API.GetLocationPromise();
foreach facility {
    location_promise.then(function(location){
        ...use location
    });
}