Promise map keep order

Question

I'm using bluebird Promise map to process an array of file lines, in which some lines need some transformation. Transformation is done calling a web service.

I wrote a function that returns a promise which resolves with the transformed array.

function tokenizeChunk(data){

  return new Promise(async (resolve, reject) => {

    let processed = [];

    await Promise.map(data, async (line) => {

      try{

        const lineCode = line.substring(0,4);
        if (lineCode != "0500") processed.push(line);
        else{
          // find string, tokenize, replace
          const stringFound = line.substring(55,71);
          var re = new RegExp(stringFound,"g");

          processed.push(line.replace(re, await Tokenize(stringFound)));


        }

      }catch(err){
        console.error(err);
        process.exit();
      }

    }, {concurrency: 50}).then(() => {
      resolve(processed.join("\r\n"));
    });

  });

}

However, processed is not in the same order as data, and I need to keep the same order (as this is a file processing which needs to output the processed file with the same order than the input file).

This is the Tokenize function (which calls the webservice):

function Tokenize(value){

    return new Promise(function(resolve, reject){
        var requestPath = `http://localhost:8080/transform/${value}`;
        request.get(requestPath, function(err, response, body){
            if (!err && response.statusCode == 200){
                resolve(body);
            }else{
                reject(err);
            }
        });
    });

}

How can I keep the array order and return the same array but transformed? Considering that the webservice is able to handle over 1000 TPS.

Answer 1

Promise.map resolved value is an array where each element is the returned/resolved value of each callback, in order.

So instead of pushing to an array, just return the pushed value, and Promise.map will handle the order for you.

async function tokenizeChunk(data) {

    const result = await Promise.map(data, async(line) => {

        const lineCode = line.substring(0, 4);
        if (lineCode != "0500")
            return line;

        // find string, tokenize, replace
        const stringFound = line.substring(55, 71);
        var re = new RegExp(stringFound, "g");

        return line.replace(re, await Tokenize(stringFound));


    }, { concurrency: 50 });

    return result.join("\r\n")
}

You can remove the new Promise() wrapper, making the function async, making the code clearer.

Answer 2

Promise.map does not guarantee execution order - use Promise.each or Promise.reduce if you need that and are ok with sequential processing.

API: http://bluebirdjs.com/docs/api/promise.each.html

More details: Promise.map concurrency execution order