JavaScript array .reduce with async/await

Question

Seem to be having some issues incorporating async/await with .reduce(), like so:

const data = await bodies.reduce(async(accum, current, index) => {
  const methodName = methods[index]
  const method = this[methodName]
  if (methodName == 'foo') {
    current.cover = await this.store(current.cover, id)
    console.log(current)
    return {
      ...accum,
      ...current
    }
  }
  return {
    ...accum,
    ...method(current.data)
  }
}, {})
console.log(data)

The data object is logged before the this.store completes...

I know you can utilise Promise.all with async loops, but does that apply to .reduce()?

Answer 1

The problem is that your accumulator values are promises - they're return values of async functions. To get sequential evaluation (and all but the last iteration to be awaited at all), you need to use

const data = await array.reduce(async (accumP, current, index) => {
  const accum = await accumP;
  …
}, Promise.resolve(…));

That said, for async/await I would in general recommend to use plain loops instead of array iteration methods, they're more performant and often simpler.

Answer 2

I like Bergi's answer, I think it's the right way to go.

I'd also like to mention a library of mine, called Awaity.js

Which lets you effortlessly use functions like reduce, map & filter with async / await:

import reduce from 'awaity/reduce';

const posts = await reduce([1,2,3], async (posts, id) => {

  const res = await fetch('/api/posts/' + id);
  const post = await res.json();

  return {
    ...posts,
    [id]: post
  };
}, {})

posts // { 1: { ... }, 2: { ... }, 3: { ... } }

Answer 3

[Not addressing OPs exact prob; focused on others who land here.]

Reduce is commonly used when you need the result of the previous steps before you can process the next. In that case, you can string promises together a la:

promise = elts.reduce(
    async (promise, elt) => {
        return promise.then(async last => {
            return await f(last, elt)
        })
    }, Promise.resolve(0)) // or "" or [] or ...

Here's an example with uses fs.promise.mkdir() (sure, much simpler to use mkdirSync, but in my case, it's across a network):

const Path = require('path')
const Fs = require('fs')

async function mkdirs (path) {
    return path.split(/\//).filter(d => !!d).reduce(
        async (promise, dir) => {
            return promise.then(async parent => {
                const ret = Path.join(parent, dir);
                try {
                    await Fs.promises.lstat(ret)
                } catch (e) {
                    console.log(`mkdir(${ret})`)
                    await Fs.promises.mkdir(ret)
                }
                return ret
            })
        }, Promise.resolve(""))
}

mkdirs('dir1/dir2/dir3')

Below is another example which add 100 + 200 ... 500 and waits around a bit:

async function slowCounter () {
    const ret = await ([100, 200, 300, 400, 500]).reduce(
        async (promise, wait, idx) => {
            return promise.then(async last => {
                const ret = last + wait
                console.log(`${idx}: waiting ${wait}ms to return ${ret}`)
                await new Promise((res, rej) => setTimeout(res, wait))
                return ret
            })
        }, Promise.resolve(0))
    console.log(ret)
}

slowCounter ()

Answer 4

Sometimes the best thing to do is simply put both code versions side by side, sync and async:

Sync version:

const arr = [1, 2, 3, 4, 5];

const syncRev = arr.reduce((acc, i) => [i, ...acc], []); // [5, 4, 3, 2, 1] 

Async one:

(async () => { 
   const asyncRev = await arr.reduce(async (promisedAcc, i) => {
      const id = await asyncIdentity(i); // could be id = i, just stubbing async op.
      const acc = await promisedAcc;
      return [id, ...acc];
   }, Promise.resolve([]));   // [5, 4, 3, 2, 1] 
})();

//async stuff
async function asyncIdentity(id) {
   return Promise.resolve(id);
}

const arr = [1, 2, 3, 4, 5];
(async () => {
    const asyncRev = await arr.reduce(async (promisedAcc, i) => {
        const id = await asyncIdentity(i);
        const acc = await promisedAcc;
        return [id, ...acc];
    }, Promise.resolve([]));

    console.log('asyncRev :>> ', asyncRev);
})();

const syncRev = arr.reduce((acc, i) => [i, ...acc], []);

console.log('syncRev :>> ', syncRev);

async function asyncIdentity(id) {
    return Promise.resolve(id);
}

Answer 5

The current accepted answer advises to use Promise.all() instead of an async reduce. However this does not have the same behavior as an async reduce and is only relevant for the case where you want an exception to stop all iterations immediately, which is not always the case.

Additionally in the comments of that answer it's suggested that you should always await the accumulator as the first statement in the reducer, because otherwise you might risk unhandled promise rejections. The poster also says that this was what the OP is asking for, which is not the case. Instead he just wants to know when everything is done. In order to know that you indeed need to do await acc, but this could be at any point in the reducer.

const reducer = async(acc, key) => {
  const response = await api(item);

  return {
    ...await acc, // <-- this would work just as well for OP
    [key]: response,
  }
}
const result = await ['a', 'b', 'c', 'd'].reduce(reducer, {});
console.log(result); // <-- Will be the final result

---

How to safely use async reduce

That being said, using a reducer this way does mean that you need to guarantee it does not throw, else you will get "unhandled promise rejections". It's perfectly possible to ensure this by using a try-catch, with the catch block returning the accumulator (optionally with a record for the failed API call).

const reducer = async (acc, key) => {
    try {
        data = await doSlowTask(key);
        return {...await acc, [key]: data};
    } catch (error) {
        return {...await acc, [key]: {error}};
    };
}
const result = await ['a', 'b', 'c','d'].reduce(reducer, {});

---

Difference with Promise.allSettled You can get close to the behavior of an async reduce (with error catching) by using Promise.allSettled. However this is clunky to use: you need to add another synchronous reduce after it if you want to reduce to an object.

The theoretical time complexity is also higher for Promise.allSettled + regular reduce, though there are probably very few use cases where this will make a difference. async reduce can start accumulating from the moment the first item is done, whereas a reduce after Promise.allSettled is blocked until all promises are fulfilled. This could make a difference when looping over a very large amount of elements.

const responseTime = 200; //ms
function sleep(ms) {
    return new Promise(resolve => setTimeout(resolve, ms));
}

const api = async (key) => {
    console.log(`Calling API for ${ key }`);
    // Boz is a slow endpoint.
    await sleep(key === 'boz' ? 800 : responseTime);
    console.log(`Got response for ${ key }`);

    if (key === 'bar') throw new Error(`It doesn't work for ${ key }`);

    return {
        [key]: `API says ${ key }`,
    };
};

const keys = ['foo', 'bar', 'baz', 'buz', 'boz'];

const reducer = async (acc, key) => {
    let data;
    try {
        const response = await api(key);
        data = {
            apiData: response
        };
    } catch (e) {
        data = {
            error: e.message
        };
    }

    // OP doesn't care how this works, he only wants to know when the whole thing is ready.
    const previous = await acc;
    console.log(`Got previous for ${ key }`);

    return {
        ...previous,
        [key]: {
            ...data
        },
    };
};
(async () => {
    const start = performance.now();
    const result = await keys.reduce(reducer, {});
    console.log(`After ${ performance.now() - start }ms`, result); // <-- OP wants to execute things when it's ready.
})();

Check the order of execution with Promise.allSettled:

const responseTime = 200; //ms
function sleep(ms) {
    return new Promise(resolve => setTimeout(resolve, ms));
}

const api = async (key) => {
    console.log(`Calling API for ${ key }`);
    // Boz is a slow endpoint.
    await sleep(key === 'boz' ? 800 : responseTime);
    console.log(`Got response for ${ key }`);

    if (key === 'bar') throw new Error(`It doesn't work for ${ key }`);

    return {
        key,
        data: `API says ${ key }`,
    };
};

const keys = ['foo', 'bar', 'baz', 'buz', 'boz'];

(async () => {
    const start = performance.now();
    const apiResponses = await Promise.allSettled(keys.map(api));
    const result = apiResponses.reduce((acc, {status, reason, value}) => {
        const {key, data} = value || {};
        console.log(`Got previous for ${ key }`);
        return {
            ...acc,
            [key]: status === 'fulfilled' ? {apiData: data} : {error: reason.message},
        };
    }, {});
    console.log(`After ${ performance.now() - start }ms`, result); // <-- OP wants to execute things when it's ready.
})();

Answer 6

You can wrap your entire map/reduce iterator blocks into their own Promise.resolve and await on that to complete. The issue, though, is that the accumulator doesn't contain the resulting data/object you'd expect on each iteration. Due to the internal async/await/Promise chain, the accumulator will be actual Promises themselves that likely have yet to resolve themselves despite using an await keyword before your call to the store (which might lead you to believe that the iteration won't actually return until that call completes and the accumulator is updated.

While this is not the most elegant solution, one option you have is to move your data object variable out of scope and assign it as a let so that proper binding and mutation can occur. Then update this data object from inside your iterator as the async/await/Promise calls resolve.

/* allow the result object to be initialized outside of scope 
   rather than trying to spread results into your accumulator on iterations, 
   else your results will not be maintained as expected within the 
   internal async/await/Promise chain.
*/    
let data = {}; 

await Promise.resolve(bodies.reduce(async(accum, current, index) => {
  const methodName = methods[index]
  const method = this[methodName];
  if (methodName == 'foo') {
    // note: this extra Promise.resolve may not be entirely necessary
    const cover = await Promise.resolve(this.store(current.cover, id));
    current.cover = cover;
    console.log(current);
    data = {
      ...data,
      ...current,
    };
    return data;
  }
  data = {
    ...data,
    ...method(current.data)
  };
  return data;
}, {});
console.log(data);

Answer 7

For typescript previous value and initial value need to be same.

const data = await array.reduce(async (accumP: Promise<Tout>, curr<Tin>) => {
    const accum: Tout = await accumP;
    
    doSomeStuff...

    return accum;

}, Promise<Tout>.resolve({} as Tout);