Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Creating a Celery worker using node.js

Using node-celery, we can enable node to push Celery jobs to the task queue. How can we allow node to be a Celery worker and consume the queue?

like image 440
Nyxynyx Avatar asked Jan 19 '14 00:01

Nyxynyx


People also ask

What is celery worker node?

A Node is just a Worker in a Cluster. In short Node = Worker. A Cluster is a number of Workers running in parallel (using celery multi as per the document I introduced with). A Cluster is just a convenient way of starting and stopping and managing multiple workers on the same machine.

What is the difference between celery and RabbitMQ?

From my understanding, Celery is a distributed task queue, which means the only thing that it should do is dispatching tasks/jobs to others servers and get the result back. RabbitMQ is a message queue, and nothing more. However, a worker could just listen to the MQ and execute the task when a message is received.

What is broker and worker in celery?

Celery communicates via messages, usually using a broker to mediate between clients and workers. To initiate a task the client adds a message to the queue, the broker then delivers that message to a worker.

What is broker and backend in celery?

Celery has the ability to communicate and store with many different backends (Result Stores) and brokers (Message Transports).


1 Answers

For Celery if the end point is amqp. Checkout Celery.js Github any node process started as amqp consumer would work fine. For every other self.conf.backend_type types you can have varied consumer. Following example is merely for amqp.

One such example. The message below may be the Celery task object.

var amqp = require('amqp'); var connection = amqp.createConnection({ host: "localhost", port: 5672 }); connection.on('ready', function () {   connection.queue("my_celery_queue", function(queue){     queue.bind('#');      queue.subscribe(function (message) {       //eat your Celery work here     })   }) }) 
like image 165
bhantol Avatar answered Oct 11 '22 05:10

bhantol