Async function nested within async.js waterfall

Question

Disclaimer: non-engineer, very new to JS

Hey all - I'm trying to leverage the async.js module to chain a group of functions together. My desired output is to iterate over mapData (array of objects), prior to passing it to the final function (right now - just console.log( result ).

async.waterfall([
    function( callback ) {
        getCoords ( function( data ) {
            mapData = data;
        });
        callback(null, mapData);
    },
    function( mapData, callback ) {
        //getEmail ( mapData );
        callback( null, mapData );
    } 
    ], function( err, result ) {
    console.log( result );
});

However, getCoords contains another async function (found here). What I'm seeing is that the first callback ( null, mapData) happens before it returns, leading to a null result.

How do I structure this so that the getCoords returns mapData prior to proceeding to the next block? I am probably missing something super obvious, thanks!

Answer 1

Callback fun... You need to understand how program flow works when using callbacks. This can bee seen with a very simple example.

Example:

function doWork( callback ) {
  console.log( 2 );
  setTimeout(callback, 1000);
}

function master () {
  console.log(1);

  doWork(function () {
    console.log(3);
  });

  console.log(4);
}
master();

The expected results would be console logs in the propper order 1, 2, 3, 4. But when running the example you see something strange as the logs are out of order 1, 2, 4, 3. This is because logging of 3 happens after doWork finishes, while logging of 4 happens after starting doWork, not waiting for it to finish.

async:

You can do a lot with the async library, but the most important thing to remember is that the callback function is always to receive the error as the first argument followed by the arguments you want to pass to the next function in the list.

The gist you linked to is not setup to return that way. You can either fix it or deal with it in your code. First lets just use the function as is:

Use existing getCoords:

async.waterfall([
  function( callback ) {
    // getCoords only returns one argument
    getCoords ( function( mapData ) {
      // First argument is null because there
      // is no error. When calling the waterfall
      // callback it *must* happen inside the getCoords
      // callback. If not thing will not work as you
      // have seen.
      callback( null, mapData);
    });
  },
  function( mapData, callback ) {
    // Do work with the results of the 1st step
    // in the waterfall.

    // Finish the water fall
    callback( null, mapData );
  } 
], function( err, result ) {
  if ( err ) {
    console.log( err );
    return;
  }
  console.log( result );
});

Now there are two issues with getCoords. First is that it does not return the correct arguments to it's callback, second it doesn't always use its callback. This second issue is huge as it will cause your program to hang and break.

I've commented in the function the 2 fixes that were made.

Fixed getCoords:

function getCoords ( callback ) {   
  var query = new Parse.Query( 'userGeoCoordinates' );
  query.exists( 'location' )
  query.find( {
    success: function ( result ) {
      for ( var i = 0; i < result.length; i++ ) {
        var object = result[ i ];
        var user = {};

        user.userId = object.get( 'userId' );
        user.coords = [];

        if ( !user_dedupe( user.userId ) ) {
                all_users.push( user );
        }
      }

      for ( var i = 0; i < all_users.length; i++ ) {
        for ( var j = 0; j < result.length; j++ ) {
          var object = result [ j ];
          if( object.get( 'userId' ) == all_users[ i ].userId ) {
            all_users[i].coords.push(
              [ object.get( 'location' )._longitude , object.get( 'location' )._latitude ]
            );
          }
        }

      }
      // This is the original callback, let fix it
      // so that it uses the normal return arguments
      // callback( all_users );

      // Return null for no error, then the resutls
      callback( null, all_users );
    },
    error: function( error ) {
      // Here is the second, bigger, issue. If the
      // getCoords had an error, nothing the callback
      // isn't called. Lets fix this
      //  console.log( 'error' );

      // Return the error, an no results.
      callback( error );
    }
  });
}

With the getCoords function fixed you can simplify your waterfall:

1st Simplified Waterfall:

async.waterfall([
  function( callback ) {
    // getCoords returns the expected results
    // so just pass in our callback
    getCoords ( callback );
  },
  function( mapData, callback ) {
    // Do work with the results of the 1st step
    // in the waterfall.

    // Finish the water fall
    callback( null, mapData );
  } 
], function( err, result ) {
  if ( err ) {
    console.log( err );
    return;
  }
  console.log( result );
});

But async has a nice feature. If your waterfall step is simply calling a function that returns the expect results you can simplify it further by using async.apply.

2st Simplified Waterfall:

async.waterfall([
  async.apply( getCoords ),
  function( mapData, callback ) {
    // Do work with the results of the 1st step
    // in the waterfall.

    // Finish the water fall
    callback( null, mapData );
  } 
], function( err, result ) {
  if ( err ) {
    console.log( err );
    return;
  }
  console.log( result );
});