Safe way to run multiple async functions in parallel?

Question

I'm writing some code to scan a directory and it occurred to me this may not be the best idea:

files.forEach(async fileName => {
  stat = await lstat(fileName);
});

as I'm going to fire off an lstat for every single file in a directory at the same time. Does anyone know of a "clean" way to do this? I'm thinking a lib that maintains a queue and drains it.

I know some "old" async libraries do this, but I don't know of anything that do it with native async/await calls

Answer 1

Generally, no code does run in parallel so a few hundred open promises shouldn't be a problem.

If you want to run one after another instead a simple for loop will do it :

async function iterate(){
 for(var i=0;i<files.length;i++){
  stat = await lstat(files[i]);
 }
}

To run multiple at once, but not all may do so:

async function iterate(){
 var atonce=10;
 for(var i=0;i<files.length;i+=atonce){
  stats = await Promise.all(files.slice(i,i+atonce).map(file=>lstat(file));
 }
}

Another way would be a few Promise queues:

var current=0;
async function retrieve(){
 if(current>=files.length) return;
 current++;
 await lstat(files[current-1]);
 retrieve();
}
retrieve();//two in parallel
retrieve();

If you want to run all in parallel, may use Promise.all to catch the results ( depends on the usecase):

Promise.all(files.map(async function(file){
 return await lstat(file);
}).then(results=>...);