How to poll another server periodically from a node.js server?

Question

I have a node.js server A with mongodb for database. There is another remote server B (doesn't need to be node based) which exposes a HTTP/GET API '/status' and returns either 'FREE' or 'BUSY' as the response.

When a user hits a particular API endpoint in server A(say POST /test), I wish to start polling server B's status API every minute, until server B returns 'FREE' as the response. The user doesn't need to wait till the server B returns a 'FREE' response (polling B is a background job in server A). Once the server A gets a 'FREE' response from B, it shall send out an email to the user.

How can this be achieved in server A, keeping in mind that the number of concurrent users can go large ?

Answer 1

I suggest you use Agenda. https://www.npmjs.com/package/agenda With agenda you can create recurring schedules under which you can schedule anything pretty flexible.

I suggest you use request module to make HTTP get/post requests. https://www.npmjs.com/package/request

Answer 2

Going from the example in node.js docs I'd go with something like the code here. I tested and it works. BTW, I'm assuming here that the api response is something like {"status":"BUSY"} & {"status":"FREE"}

const http = require('http');

const poll = {
    pollB: function() {
        http.get('http://serverB/status', (res) => {
            const { statusCode } = res;

            let error;
            if (statusCode !== 200) {
                error = new Error(`Request Failed.\n` +
                    `Status Code: ${statusCode}`);
            }

            if (error) {
                console.error(error.message);
                res.resume();
            } else {
                res.setEncoding('utf8');
                let rawData = '';
                res.on('data', (chunk) => { rawData += chunk; });
                res.on('end', () => {
                    try {
                        const parsedData = JSON.parse(rawData);

                        // The important logic comes here
                        if (parsedData.status === 'BUSY') {
                            setTimeout(poll.pollB, 10000); // request again in 10 secs
                        } else {
                            // Call the background process you need to
                        }
                    } catch (e) {
                        console.error(e.message);
                    }
                });
            }
        }).on('error', (e) => {
            console.error(`Got error: ${e.message}`);
        });
    }
}

poll.pollB();

You probably want to play with this script and get rid of unnecessary code for you, but that's homework ;)

Update:

For coping with a lot of concurrency in node.js I'd recommend to implement a cluster or use a framework. Here are some links to start researching about the subject:

How to fully utilise server capacity for Node.js Web Apps

How to Create a Node.js Cluster for Speeding Up Your Apps

Node.js v7.10.0 Documentation :: cluster

ActionHero.js :: Fantastic node.js framework for implementing an API, background tasks, cluster using http, sockets, websockets