Make sure a forEach with async calls is executed before another one?

Question

I have a function with multiple forEach loops:

async insertKpbDocument(jsonFile) {
    jsonFile.doc.annotations.forEach((annotation) => {
      annotation.entities.forEach(async (entity) => {
        await this.addVertex(entity);
      });
      annotation.relations.forEach(async (relation) => {
        await this.addRelation(relation);
      });
    });
    return jsonFile;
  }

I need to make sure that the async code in the forEach loop calling the this.addVertex function is really done before executing the next one.

But when I log variables, It seems that the this.addRelation function is called before the first loop is really over.

So I tried adding await terms before every loops like so :

await jsonFile.doc.annotations.forEach(async (annotation) => {
      await annotation.entities.forEach(async (entity) => {
        await this.addVertex(entity);
      });
      await annotation.relations.forEach(async (relation) => {
        await this.addRelation(relation);
      });
    });

But same behavior.

Maybe it is the log function that have a latency? Any ideas?

Answer 1

As we've discussed, await does not pause a .forEach() loop and does not make the 2nd item of the iteration wait for the first item to be processed. So, if you're really trying to do asynchronous sequencing of items, you can't really accomplish it with a .forEach() loop.

For this type of problem, async/await works really well with a plain for loop because they do pause the execution of the actual for statement to give you sequencing of asynchronous operations which it appears is what you want. Plus, it even works with nested for loops because they are all in the same function scope:

To show you how much simpler this can be using for/of and await, it could be done like this:

async insertKpbDocument(jsonFile) {
    for (let annotation of jsonFile.doc.annotations) {
        for (let entity of annotation.entities) {
            await this.addVertex(entity);
        }
        for (let relation of annotation.relations) {
            await this.addRelation(relation);
        }
    }
    return jsonFile;
}

You get to write synchronous-like code that is actually sequencing asynchronous operations.

---

If you are really avoiding any for loop, and your real requirement is only that all calls to addVertex() come before any calls to addRelation(), then you can do this where you use .map() instead of .forEach() and you collect an array of promises that you then use Promise.all() to wait on the whole array of promises:

insertKpbDocument(jsonFile) {
    return Promise.all(jsonFile.doc.annotations.map(async annotation => {
        await Promise.all(annotation.entities.map(entity => this.addVertex(entity)));
        await Promise.all(annotation.relations.map(relation => this.addRelation(relation)));
    })).then(() => jsonFile);
}

To fully understand how this works, this runs all addVertex() calls in parallel for one annotation, waits for them all to finish, then runs all the addRelation() calls in parallel for one annotation, then waits for them all to finish. It runs all the annotations themselves in parallel. So, this isn't very much actual sequencing except within an annotation, but you accepted an answer that has this same sequencing and said it works so I show a little simpler version of this for completeness.

---

If you really need to sequence each individual addVertex() call so you don't call the next one until the previous one is done and you're still not going to use a for loop, then you can use the .reduce() promise pattern put into a helper function to manually sequence asynchronous access to an array:

// helper function to sequence asynchronous iteration of an array
// fn returns a promise and is passed an array item as an argument
function sequence(array, fn) {
    return array.reduce((p, item) => {
        return p.then(() => {
            return fn(item);
        });
    }, Promise.resolve());
}


insertKpbDocument(jsonFile) {
    return sequence(jsonFile.doc.annotations, async (annotation) => {
        await sequence(annotation.entities, entity => this.addVertex(entity));
        await sequence(annotation.relations, relation => this.addRelation(relation));
    }).then(() => jsonFile);
}

This will completely sequence everything. It will do this type of order:

addVertex(annotation1)
addRelation(relation1);
addVertex(annotation2)
addRelation(relation2);
....
addVertex(annotationN);
addRelation(relationN);

where it waits for each operation to finish before going onto the next one.