Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Promise.all() with limit?

Is there a way/pattern to implement let res = Promise.all([...p], limit)?

  • The promise-array holds functions which construct and return the promise
  • res should be resolved after all p resolves
  • Only limit=3 Promises should run in parallel
  • The n+1th Promise should start immediately after n finishes. So that there are always limit resolvers running in parallel.

Especially the last point creates my headaches.

My current solution is to split the promises-array into chunks of limit size and chain them. The disadvantages here is that the second bunch don't start until all Promises from bunch 1 has been resolved.

like image 934
Psi Avatar asked Nov 02 '16 08:11

Psi


People also ask

Is there a limit to Promise all?

Assuming we have the processing power and that our promises can run in parallel, there is a hard limit of just over 2 million promises.

What does Promise all () do?

Promise.all() The Promise.all() method takes an iterable of promises as an input, and returns a single Promise that resolves to an array of the results of the input promises. This returned promise will fulfill when all of the input's promises have fulfilled, or if the input iterable contains no promises.

Does Promise all run concurrently?

JavaScript's Promise. all() method takes in a parameter of iterable promises runs them concurrently then returns a single Promise that resolves to an array of results of the input promises.

Does Promise all improve performance?

Notice that it's not await that resolves them. Promise. all does not improve performance. It's the "not waiting for the first promise before starting the second task" that improves performance (if done correctly).


1 Answers

I came up with the idea of creating n = limit chains which run in parallel and append as long as there are promises:

let promises = [];
for(let i=0; i<11; i++) promises[i] = () => {
  console.log('Construct:',i);
  return new Promise(resolve => {
    setTimeout(function() {
      console.log('Resolve:',i);
      resolve(i);
    }, Math.round(Math.random() * (2000 - 500) + 2000));
  });
}


function parallelLimit(promiseFactories, limit) {
  let result = [];
  let cnt = 0;

  function chain(promiseFactories) {
    if(!promiseFactories.length) return;
    let i = cnt++; // preserve order in result
    return promiseFactories.shift()().then((res) => {
      result[i] = res; // save result
      return chain(promiseFactories); // append next promise
    });
  }

  let arrChains = [];
  while(limit-- > 0 && promiseFactories.length > 0) {
    // create `limit` chains which run in parallel
    arrChains.push(chain(promiseFactories));
  }

  // return when all arrChains are finished
  return Promise.all(arrChains).then(() => result);
}


parallelLimit(promises, 4).then(console.log);

Excited to read your comments and suggestions :)

like image 101
Psi Avatar answered Sep 28 '22 06:09

Psi