Lodash: _.forEach with function

Question

I am trying to use the lodash forEach method with a nested function that calls a mongo database.

var jobs = [];
_.forEach(ids, function(id) {
    JobRequest.findByJobId(id, function(err, result) {
        if(err) callback(err);
        jobs.push(result);
    });
});

callback(null, jobs);

I am having problems because the forEach and callbacks will run through before the inner function is ever called. How can I resolve this?

I want the callback to be called after the for each and inner function have completed.

Answer 1

One more approach is to wrap everything into promises, in this case job results will be pushed into array in correct order:

var promises = ids.map(function(id) {
    return new Promise(function(resolve, reject) {
        JobRequest.findByJobId(id, function (err, result) {
            if (err) reject(err);
            resolve(result);
        });
    });
});

Promise.all(promises).then(function(jobs) {
    callback(null, jobs);
}, callback);

// or shorter: Promise.all(promises).then(callback.bind(null, null), callback);

Note, that you also need to handle potential situation when JobRequest.findByJobId request fails, with promises it's very easy: just pass callback as error callback to Promise.all.

Answer 2

JobRequest.findByJobId is an asynchronous operation. You cannot block asynchronous operations in JavaScript, so you'll need to manually synchronize by counting. Example (error handling omitted for the sake of brevity):

var results = [];
var pendingJobCount = ids.length;

_.forEach(ids, function(id) {
    JobRequest.findByJobId(id, function(err, result) {
        results.push(result);
        if (--pendingJobCount === 0) callback(null, results);
    });
});

There are, of course, wrapper constructs for doing stuff like this, but I prefer to explain how it actually works. Check out dfsq's answer for more details on one of those wrappers, called promises.

Also note that asynchronous operations may complete out of order. The order in the results array will not necessarily match the order of the ids array. If you need that information connected, you'll need to track it yourself, for example by collecting the results in a map instead of an array:

var results = {};
var pendingJobCount = ids.length;

_.forEach(ids, function(id) {
    JobRequest.findByJobId(id, function(err, result) {
        results[id] = result;
        if (--pendingJobCount === 0) callback(null, results);
    });
});

This example assumes that there are no duplicates in your ids array. Results for duplicate keys would be overridden.

Error handling would work similarly, by inserting additional information into your result. Another example:

results.push({id: id, error: null, value: result});