Node.js workers/background processes

Question

How can I create and use background jobs in node.js?

I've come across two libs (node-resque and node-worker) but would like to know if there's something more used.

Answer 1

I did some research on this and I would do it like this.

Setup

beanstalkd

1. Install beanstalkd. Another message queue, BUT this one supports DELAYED PUTS. If you compile from source it is going to be a little harder because it depends on libevent(like memcached). But then again, I don't think you have to compile it from source, because there are a lot of binary packages available. For example on Ubuntu you can install beanstalkd by issuing the command:

   sudo apt-get install beanstalkd

node-beanstalk-client

2. Install a beanstalkd client library. The best one I found was node-beanstalk-client. Because on the beanstalkd client library list this library isn't/wasn't mentioned(Then again I can add entries to the list, so I will add this one). The reasons I prefer this library over the others are:

   1. Npm package: I liked to use a npm package to install client library. The others did not have any.
   2. Active development: I prefer libraries which have later/more commits.

So to install it, after you have installed npm(the write way) you would just issue the following command:

npm install beanstalk_client 

Code

consumer.js

var client = require('beanstalk_client').Client;  client.connect('127.0.0.1:11300', function(err, conn) {     var reserve = function() {         conn.reserve(function(err, job_id, job_json) {             console.log('got job: ' + job_id);             console.log('got job data: ' + job_json);             console.log('module name is ' + JSON.parse(job_json).data.name);             conn.destroy(job_id, function(err) {                 console.log('destroyed job');                 reserve();             });         });     }      reserve(); }); 

First start the consumer:

node consumer.js  

Next start producer.js. Five seconds(delay I specified) after you execute producer.js, consumer.js will process the message.

producer.js

var client = require('beanstalk_client').Client; client.connect('127.0.0.1:11300', function(err, conn) {     var job_data = {"data": {"name": "node-beanstalk-client"}};     var priority = 0;     var delay = 5;     var timeToRun = 1;     conn.put(priority, delay, timeToRun, JSON.stringify(job_data), function(err, job_id) {         console.log('put job: ' + job_id);         process.exit();     }); }); 

to start just issue:

node producer.js 

Answer 2

I'm using Kue.

Kue is a feature rich priority job queue for node.js backed by redis. A key feature of Kue is its clean user-interface for viewing and managing queued, active, failed, and completed jobs.