Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Consumer Connection error with django and celery+rabbitmq?

I'm trying to set up celeryd with django and rabbit-mq. So far, I've done the following:

  • Installed celery from pip
  • Installed rabbitmq via the debs available from their repository
  • Added a user and vhost to rabbitmq via rabbitmqctl, as well as permissions for that user
  • Started the rabbitmq-server
  • Installed django-celery via pip
  • Set up django-celery, including its tables
  • Configured the various things in settings.py (BROKER_HOST, BROKER_PORT, BROKER_USER, BROKER_PASSWORD, BROKER_VHOST, as well as importing djecelery, calling the setup function and adding it to the INSTALLED APPS). I've double checked and all of these values are correct (at least, user, password and vhost are correct).

So now, when I run python manage.py celeryd -l info I get connection errors (see below). Anyone have any idea why?

$ python manage.py celeryd -l info
/usr/local/lib/python2.7/dist-packages/djcelery/loaders.py:108: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments!
  warnings.warn("Using settings.DEBUG leads to a memory leak, never "
[2012-05-15 18:38:04,486: WARNING/MainProcess]  

 -------------- celery@ubuntu v2.5.3
---- **** -----
--- * ***  * -- [Configuration]
-- * - **** ---   . broker:      amqp://celeryuser@localhost:5672/celeryhost
- ** ----------   . loader:      djcelery.loaders.DjangoLoader
- ** ----------   . logfile:     [stderr]@INFO
- ** ----------   . concurrency: 1
- ** ----------   . events:      OFF
- *** --- * ---   . beat:        OFF
-- ******* ----
--- ***** ----- [Queues]
 --------------   . celery:      exchange:celery (direct) binding:celery


[Tasks]


[2012-05-15 18:38:04,562: INFO/PoolWorker-1] child process calling self.run()
[2012-05-15 18:38:04,565: WARNING/MainProcess] celery@ubuntu has started.
[2012-05-15 18:38:07,572: ERROR/MainProcess] Consumer: Connection Error: [Errno 104] Connection reset by peer. Trying again in 2 seconds...
^C[2012-05-15 18:38:08,434: WARNING/MainProcess] celeryd: Hitting Ctrl+C again will terminate all running tasks!
[2012-05-15 18:38:08,435: WARNING/MainProcess] celeryd: Warm shutdown (MainProcess)
[2012-05-15 18:38:09,372: INFO/PoolWorker-1] process shutting down
[2012-05-15 18:38:09,373: INFO/PoolWorker-1] process exiting with exitcode 0
[2012-05-15 18:38:09,376: INFO/MainProcess] process shutting down
like image 255
Paul Wicks Avatar asked May 16 '12 01:05

Paul Wicks


People also ask

Does Celery use RabbitMQ?

Celery is an open-source task queue software written in Python. It's incredibly lightweight, supports multiple brokers (RabbitMQ, Redis, and Amazon SQS), and also integrates with many web frameworks, e.g. Django, etc.

What is the difference between Celery and RabbitMQ?

Celery: Distributed task queue. Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well; RabbitMQ: A messaging broker - an intermediary for messaging.

What is AMQP in Celery?

celery-amqp-backend is a rewrite of the Celery's original amqp:// result backend, which was removed from Celery with version 5.0. Celery encourages you to use the newer rpc:// result backend, as it does not create a new result queue for each task and thus is faster in many circumstances.


2 Answers

Your problem is in the BROKER_URL.

With an additional VHOST, the right config would be:

BROKER_URL='amqp://celeryuser@localhost:5672//'
BROKER_VHOST='/celeryhost'
like image 127
Amit Avatar answered Oct 05 '22 08:10

Amit


For me, the following URL ending worked: ...@localhost:5672/celeryvhost

like image 43
user1804633 Avatar answered Oct 05 '22 07:10

user1804633