Delaying a Promise.all sequence

Question

I'm fetching data from an external API and I get the following error on some of the endpoints:

reason: getaddrinfo EAI_AGAIN [the url]:443

I can only assume that the owner of the API has a rate limiter of some sort and since all the fetching is done at once some URLs get blocked.

So I'm trying to put a delay between each fetch, but with no luck. Here is my try:

class someClass {
  ...
  dealyedFetch(url) {
    return new Promise ((resolve, reject) => {
      setTimeout(() => {
        fetch(url, (err, res) => { 
          if (err) return reject(err)
          return resolve(res)
        })
      }, 5000)    
    })
  }

  fetchUrls() {
    return Promise.all(
      this.urlList.map(url => {
        return this.dealyedFetch(url)
        .then(/* macht krieg */)
        .catch(err => ({err, url}))
      })
    )
  }    
}

the above doesn't work and it seems to fire all the fetching at once. Please illuminate me.

Answer 1

You're delaying them all at once, so they all wait 5000ms, then fire together.

If you want to string them out, you need to make their delays progressively longer. You could do that by using the index in the map to multiple a delay of, say, 500ms:

delayedFetch(url, delay) {                   // ***
  return new Promise ((resolve, reject) => {
    setTimeout(() => {
      fetch(url, (err, res) => { 
        if (err) return reject(err)
        return resolve(res)
      })
    }, delay)                                // ***
  })
}
fetchUrls() {
  return Promise.all(
    this.urlList.map((url, i) => {           // ***
      return this.delayedFetch(url, i * 500) // ***
      .then(/* macht krieg */)
      .catch(err => ({err, url}))
    })
  )
}    

The first request won't be delayed at all, the second will be 1 * 500 = 500ms, the third will be 2 * 500 = 1000ms, etc.

---

Side note: In the above I fixed a typo in the name delayedFetch (it was dealyedFetch in the question, a and l were reversed).

Side note 2: The above addresses what you actually asked, but you might consider doing them one at a time, in which case this question's answers show you how to do that. Or perhaps a concurrency limit on the outstanding ones, in which case this question's answers would help. Just a fixed delay may not be your best solution.