ES7 async/await conceptual issue

Question

I am migrating an existing program to use async/await (via Babel's bluebirdCoroutines) in order to learn this style. I have been looking at this tutorial.

I am a bit troubled by the following behavior. This snippet works as expected:

let parts = [];
let urlsP = urls.map((url, index) => { 
    return dlPart(url, index, tempDir); 
});
for (let urlP of urlsP) { // Parallel (yay!)
    parts.push(await urlP);
}
for (let part of parts) { // Sequential
    await appendFile(leFile, part);
}

Re-written as follows, it still works but the fist operation is not parallel any more (it takes much longer to finish)!

let index = 0;
let parts = [];
for (let url of urls) { // NOT Parallel any more!!!
    index++;
    parts.push(await dlPart(url, index, tempDir));
}
for (let part of parts) {
    await appendFile(leFile, part);
}

This is the implementation of dlPart()

function dlPart(url, num, dir) {
    var cmd = 'wget --quiet "' + url + '" -O ' + dir + "/" + num;
    return exec(cmd).then(() => {
        return dir + '/' + num;
    });
}

What am I missing?

Answer 1

The reason that it is no longer running in parallel is because of the timing of when you are creating your promises in the two examples. This is described above comment more clearly.

In your first example you start all the Promises which begin to execute their functions. Then in this loop:

for (let urlP of urlsP) { // Parallel (yay!)
    parts.push(await urlP);
}

You wait for the first promise to be done, and then the second promise to be done, etc. But the whole time you are waiting for the first promise to be done ALL the other promises are still executing. Hence they run in "parallel".

In your second example, you both START and AWAIT the promises inside the loop instead of starting them all before the loop. So in this code:

for (let url of urls) { // NOT Parallel any more!!!
    index++;
    parts.push(await dlPart(url, index, tempDir));
}

The parts.push line does the following in order:

1. Runs dlPart() which returns a promise and starts downloading the part
2. Waits for the promise to resolve
3. Pushes the resolved value into parts.

All the other promises have not been started and are not running "in parallel" they only get started when it's their turn in the loop. This means they get called one at a time, and only start the next one executing once the previous one is done and that is why they run iteratively.

Note: .map is not asynchronous if it were then your first example wouldn't work with large lists because the for of loop would start before all the promises were added to your urlsP array.

Answer 2

The .map function is asynchronous so the rest of your code doesn't have to wait on it, it will finish when ready. Then you replaced it with a for loop that holds everything back while it completes.