Save JavaScript Promise for future use

Question

I'm working on Node.js to build the server side RESTApi. Node works fine when I tested by myself. But when it is really out there it could still facing the overflow problems. When there are lots of requests, say there are more than 5 child_process (spawn) working at the same time, each process is taking longer time, basically slow down everything.

My idea is to check if the current process is below a certain limit (like, limit to 3 processes at a time), if it exceed the limit, I save the requests into an array, and whenever the current processes is below the limit, I use .shift() to pop the oldest one in the array and process it.

However when it comes to Promises, it becomes difficult as I don't know if we can store a Promise into an array or not, or whether I should simply let the process pause for seconds which I don't think it is a good idea.

What is the normal way if you want to hold a promise and return the promise to client in future?

Sorry if I didn't state this clear. Here is a summary of my doubts: 1. Can we save a promise for future use? 2. Do we save them in arrays? 3. Shall I use other methods to hold the promise, like, using sleep() or simply a while loop to wait until this process is proceed?

Thank you!

Answer 1

say there are more than 5 child_process (spawn) working at the same time, each process is taking longer time, basically slow down everything.

In a real world deployment - you would not handle CPU intensive tasks with child tasks this way - you would use a sane concurrent data structure (like a queue on mqtt or a database) and distribute the work to workers you deploy who would then send it back to the server.

The reason is that the server can always go down - and you want to guard against partial work.

My idea is to check if the current process is below a certain limit (like, limit to 3 processes at a time), if it exceed the limit, I save the requests into an array, and whenever the current processes is below the limit, I use .shift() to pop the oldest one in the array and process it.

Here is code that does that, I beg you to read the first point and not use that code in production but instead limit that at your deployment (for example if you use AWS to limit the scale to 3 instances):

// lifted from my bluebird-api project
function throttle(fn, concurrency = 20, Promise) {
    // create an array of workers as resolved promises
    const workers = Array(concurrency).fill(Promise.resolve());
    const work = []; // pending work
    return function (...args) {
        // when there is work, pull the next worker
        const worker = workers.pop();
        if (worker === undefined) { // if there is no pending worker
            let resolve, reject;
            // store the promise for the result
            const promise = new Promise((res, rej) => {
                resolve = res; 
                reject = rej;
            });
            // and add it to the queue
            work.unshift({ args, resolve, reject, promise });
            return promise;
        }
        // after the worker is ready, call the function
        worker = worker.then(() => (fn(...args), null));
        worker.then(function pump() {
            if (work.length) { // after you're ready
                const {resolve, reject, args} = work.pop();
                // continue draining the queue
                worker = worker.then(() => fn(...args)).then(resolve, reject).then(pump);
            } else { // or declare ready
                workers.push(worker);
            }
            return null;
        });
        return worker;
    }
} 

The code is lifted from bluebird-api which is still WIP.

What is the normal way if you want to hold a promise and return the promise to client in future?

Yes, it is entirely a supported case - and it doesn't leak memory and is safe (in modern promise implementation). Although again - this is an XY issue - you should not distribute work that way on a Node server.

When you implement the correct solution (queueing and offloading to different services) you can create a queue of promises where you return a promise and resolve it later when the queue is ready.