Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

NodeJs scheduling jobs on multiple nodes

I have two nodeJs servers running behind a Load Balancer. I have some scheduled jobs that i want to run only once on any of the two instances in a distributed manner.

Which module should i use ? Will node-quartz(https://www.npmjs.com/package/node-quartz) be useful for this ?

like image 419
Vikas Tiwari Avatar asked Jan 18 '16 14:01

Vikas Tiwari


1 Answers

Adding redis and using node-redlock seemed like overkill for the little caching job I needed to schedule for once a day on a single server with three Node.js processes behind a load balancer.

I discovered http://kvz.io/blog/2012/12/31/lock-your-cronjobs/ - and that led me to the concept behind Tim Kay's solo.

The concept goes like this - instead of locking on an object (only works in a single process) or using a distributed lock (needed for multiple servers), "lock" by listening on a port. All the processes on the server share the same ports. And if the process fails, it will (of course) release the port.

Note that hard-failing (no catch anywhere surrounding) or releasing the lock in catch are both OK, but neglecting to release the lock when catching exceptions around the critical section will mean that the scheduled job never executes until the locking process gets recycled for some other reason.

I'll update when I've tried to implement this.

Edit

Here's my working example of locking on a port:

multiProc.js

var net = require('net');
var server = net.createServer();

server.on('error', function () { console.log('I am process number two!'); });

server.listen({ port: 3000 },
    function () { console.log('I am process number one!');
                  setTimeout(function () {server.close()}, 3000); });

If I run this twice within 3 seconds, here's the output from the first and second instances

first

I am process number one!

second

I am process number two!

If, on the other hand, more than 3 seconds pass between executing the two instances, both claim to be process number one.

like image 184
sq33G Avatar answered Sep 21 '22 17:09

sq33G