.each and callbacks

Question

I am using request and cheerio node modules to create get some data out of a website. I'd like to get a list of item and once this list is complete, call an asynchronous function:

request('http://myurl', function(req,res,data){
    var $ = cheerio.load(data);
    var List = [];

    $('.myItems').each(function(i, element){
        console.log( typeof $(this).text() )
        List.push($(this).text());
    });

   for (var i=0; i <  List.length; i++){
      // make an asynchronous call to a API
   }
});

My question is how do I wait for the list to be completed, ie, how can I know than the .each function has looped through all the items?

Can I do this with async?

Thanks

Answer 1

The .each function of cheerio is synchronous (see source). So as long as you do nothing asynchronous in the callback (which is the case in the question), you have nothing to do : at the following line the loop will be complete.

---

If you do call asynchronous functions in the loop, the cleanest and simplest solution is to use a Promise library.

In a sequential way (exemple with Bluebird) :

var p = Promise.resolve();
$('.myItems').each(function(i, element){
    p = p.then(function(){ 
         // do something with $(this).text() and return a promise
    });
});
p.then(function(){
   // all asynchronous tasks are finished
});

If sequential requests aren't needed (here an example with Q) :

Q.allSettled($('.myItems').map(function(){
   // return a promise
})).then(function(){
   // all asynchronous tasks are finished
});

Answer 2

Well, for anyone else having this problem, just know that you don't really need to use the each method, you can transform them in an array with the toArray method and iterate over them in a for loop.

That way, there is no need to use an external lib like Bluebird.

request('http://myurl', function(req,res,data){
    var $ = cheerio.load(data);
    var List = [];

    const items = $('.myItems').toArray();
    for(let i = 0; i < items.length; i++){
        const el = items[i];
        console.log( typeof $(el).text() )
        List.push($(el).text());
        // make an asynchronous call to a API
    }
});

Answer 3

You could use the async module to easily handle those kinds of async tasks.

Specifically async.each or async.eachLimit if you need concurrency > 1 or async.eachSeries if you want to go through the items in the array, one at a time.