How to loop request-promise API request call?

Question

I am learning Node.JS and I am introduced to request-promise package. I use it for API call but I faced a problem where I cannot apply loop to it.

This is the example showing a simple API call:

var read_match_id = {
    uri: 'https://api.steampowered.com/IDOTA2Match_570/GetMatchHistory/V001',
    qs: {
        match_id: "123",
        key: 'XXXXXXXX'
    },
    json: true
};

rp(read_match_id)
.then(function (htmlString) {
    // Process html...
})
.catch(function (err) {
    // Crawling failed...
});

How can i have loop like this:

 var match_details[];
 for (i = 0; i < 5; i++) {
     var read_match_details = {
              uri: 'https://api.steampowered.com/IDOTA2Match_570/GetMatchDetails/V001',
            qs: {
                  key: 'XXXXXXXXX',
                  match_id: match_id[i]
                },
            json: true // Automatically parses the JSON string in the response 
    };
    rp(read_match_details)
       .then (function(read_match){
            match_details.push(read_match)//push every result to the array
        }).catch(function(err) {
            console.log('error');
        });
    }

And how can I know when all the async request are done?

Answer 1

request-promise uses Bluebird for Promise.

The simple solution is Promise.all(ps), where ps is array of promises.

var ps = [];
for (var i = 0; i < 5; i++) {
    var read_match_details = {
        uri: 'https://api.steampowered.com/IDOTA2Match_570/GetMatchDetails/V001',
        qs: {
            key: 'XXXXXXXXX',
            match_id: match_id[i]
        },
        json: true // Automatically parses the JSON string in the response 
    };
    ps.push(rp(read_match_details));
}

Promise.all(ps)
    .then((results) => {
        console.log(results); // Result of all resolve as an array
    }).catch(err => console.log(err));  // First rejected promise

The only disadvantage of this is, this will go to catch block immediately after any of the promise is rejected. 4/5 resolved, won't matter, 1 rejected will throw it all to catch.

Alternative approach is to use Bluebird's inspection (refer this). We'll map all promises to their reflection, we can do an if/else analysis for each promise, and it will work even if any of the promise is rejected.

// After loop
ps = ps.map((promise) => promise.reflect()); 

Promise.all(ps)
    .each(pInspection => {
        if (pInspection.isFulfilled()) {
            match_details.push(pInspection.value())
        } else {
            console.log(pInspection.reason());
        }
    })
    .then(() => callback(match_details)); // Or however you want to proceed

Hope this will solve your problem.