Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Node.js - Wait for multiple async calls to finish before continuing in code

So basically i have a for loop with an async function in it. Problem is that the program just continues after the loop and i want it to wait until all async functions which were called in the loop are finished before the code continues.

In my code "bar" is a json array with other json arrays in it.

function write(bla) { // gets called one after another

  for(var url in bla) {
    asyncFunctionCall(url); // Executed about 50 times, it has to run parallel
  }
  // Wait for all called functions to finish before next stuff happens and
  // write gets called again.

}

for(var foo in bar) {
  // Here i parse the json array "foo" which is in the json array "bar"
  write(foo[bla]); // bla is an array of multiple urls.
}

The async function call looks something like this:

var request = require('request');

request(url, function (error, response, body) {
  if(typeof response !== 'undefined') {
    if((response.statusCode >= 400 && response.statusCode <= 451)
    || (response.statusCode >= 500 && response.statusCode <= 511))
      return true;
    return false;
  }
  return false;
});
like image 696
Rxchard Avatar asked Jun 19 '18 09:06

Rxchard


People also ask

How do you handle multiple asynchronous calls in Node JS?

In order to run multiple async/await calls in parallel, all we need to do is add the calls to an array, and then pass that array as an argument to Promise. all() . Promise. all() will wait for all the provided async calls to be resolved before it carries on(see Conclusion for caveat).

How do I make code wait for async?

async and await Inside an async function, you can use the await keyword before a call to a function that returns a promise. This makes the code wait at that point until the promise is settled, at which point the fulfilled value of the promise is treated as a return value, or the rejected value is thrown.

How do you wait for async to finish JavaScript?

The await operator is used to wait for a Promise. It can be used inside an Async block only. The keyword Await makes JavaScript wait until the promise returns a result. It has to be noted that it only makes the async function block wait and not the whole program execution.

How do you handle multiple promises in Node JS?

Approach 1: In this approach, we will use Promise. all() method which takes all promises in a single array as its input. As a result, this method executes all the promises in itself and returns a new single promise in which the values of all the other promises are combined together.


1 Answers

The simplest way here is to use promises, directly or via async/await syntax. In this case, probably directly.

First, you have to make asyncFunctionCall return a promise. It looks like you always return a boolean, so in this case we'll always resolve the promise:

function asyncFunctionCall(url) {
  return new Promise(resolve => {
    request(url, function (error, response, body) {
      if(typeof response !== 'undefined') {
        if((response.statusCode >= 400 && response.statusCode <= 451)
        || (response.statusCode >= 500 && response.statusCode <= 511)) {
          resolve(true);
          return;
        }
      }
      resolve(false);
    });
  });
}

Then, build up an array of your promises, and use Promise.all to wait for all of them to complete. These calls run in parallel:

function write(bla) { // gets called one after another
  const promises = [];
  for(var url in bla) {
    promises.push(asyncFunctionCall(url)); // Executed about 50 times.
  }
  return Promise.all(promises);
}

Then you can build a chain of all of the promises from write so they run in series:

let p = Promise.resolve();
for (const foo in bar) { // <== Notice `const`
  // See "Edit" below
  p = p.then(() => {
    // Here i parse the json object "foo" in the json array "bar"
    // bla is an array of multiple urls.
    return write(foo[bla]));
  });
}

Note that it's important that you use const or let, not var, for foo in that loop, because the then callback closes over it; see this question's answers for why const and let make that work.

Each call to write will only be made when the previous one's work is done.

Then wait for the whole process to complete:

p.then(() => {
  // All done
});

You haven't shown anything using the booleans from write's requests, but they're available (as an array) as the resolution value of the promise from write.


The second part of the process, where we're calling write, can also be written in an async function which may make the logical flow clearer:

async function doTheWrites() {
  for (const foo in bar) {
    // Here i parse the json object "foo" in the json array "bar"
    // bla is an array of multiple urls.
    await write(foo[bla]);
  }
}

Then the overall process:

doTheWrites().then(() => {
  // All done
});

...or if this is also in an async function:

await doTheWrites();
like image 128
T.J. Crowder Avatar answered Oct 04 '22 02:10

T.J. Crowder