Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

NodeJS and asynchronous hell

I just came to this awful situation where I have an array of strings each representing a possibly existing file (e.g. var files = ['file1', 'file2', 'file3']. I need to loop through these file names and try to see if it exists in the current directory, and if it does, stop looping and forget the rest of the remaining files. So basically I want to find the first existing file of those, and fallback to a hard-coded message if nothing was found.

This is what I currently have:

var found = false;
files.forEach(function(file) {
  if (found) return false;

  fs.readFileSync(path + file, function(err, data) {
    if (err) return;

    found = true;
    continueWithStuff();
  });
});

if (found === false) {
  // Handle this scenario.
}

This is bad. It's blocking (readFileSync) thus it's slow.

I can't just supply callback methods for fs.readFile, it's not that simple because I need to take the first found item... and the callbacks may be called at any random order. I think one way would be to have a callback that increases a counter and keeps a list of found/not found information and when it reaches the files.length count, then it checks through the found/not found info and decides what to do next.

This is painful. I do see the performance greatness in evented IO, but this is unacceptable. What choices do I have?

like image 849
Tower Avatar asked Jun 05 '11 15:06

Tower


1 Answers

Don't use sync stuff in a normal server environment -- things are single threaded and this will completely lock things up while it waits for the results of this io bound loop. CLI utility = probably fine, server = only okay on startup.

A common library for asynchronous flow control is https://github.com/caolan/async

async.filter(['file1','file2','file3'], path.exists, function(results){
    // results now equals an array of the existing files
});

And if you want to say, avoid the extra calls to path.exists, then you could pretty easily write a function 'first' that did the operations until some test succeeded. Similar to https://github.com/caolan/async#until - but you're interested in the output.

like image 147
Josh Avatar answered Sep 23 '22 15:09

Josh