Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Celery not connecting to Redis Broker (Django)

This is my first time using Celery and redis so there's probably something obvious that I'm not inferring from the documentation and searching through others' questions on here. Whenever I try to run a worker my connection keeps timing out with:

ResponseError: unknown command 'WATCH'
[2013-06-12 18:25:23,059: ERROR/MainProcess] consumer: Connection to broker lost. Trying    to re-establish the connection...

here's my requirements.txt

South==0.7.6
amqp==1.0.11
anyjson==0.3.3
billiard==2.7.3.28
boto==2.9.4
celery==3.0.19
celery-with-redis==3.0
dj-database-url==0.2.1
django-admin-bootstrapped==0.3.2
django-celery==3.0.17
django-jsonfield==0.9.4
django-stripe-payments==2.0b20
mimeparse==0.1.3
oauthlib==0.4.0
paramiko==1.10.1
psycopg2==2.5
pycrypto==2.6
python-dateutil==2.1
python-openid==2.2.5
pytz==2013b
redis==2.7.5
requests==1.2.0
requests-oauthlib==0.3.1
six==1.3.0
stripe==1.7.9
wsgiref==0.1.2  

settings.py

import djcelery
djcelery.setup_loader()

INSTALLED_APPS = (
...
'djcelery',
...
)

CACHES = {
    "default": {
        "BACKEND": "redis_cache.cache.RedisCache",
        "LOCATION": "127.0.0.1:6379:1",
        "OPTIONS": {
        "CLIENT_CLASS": "redis_cache.client.DefaultClient",
        }
    }
}

BROKER_URL = 'redis://localhost:6379/0'

when I start my redis server and run

./manage.py celeryd -B

My connect just keeps timing out with:

Traceback (most recent call last):
File "/venv/lib/python2.7/site-packages/celery/worker/consumer.py", line 395, in start
self.consume_messages()
File "/venv/lib/python2.7/site-packages/celery/worker/consumer.py", line 407, in    consume_messages
with self.hub as hub:
File "/venv/lib/python2.7/site-packages/celery/worker/hub.py", line 198, in __enter__
self.init()
File "/venv/lib/python2.7/site-packages/celery/worker/hub.py", line 146, in init
callback(self)
File "/venv/lib/python2.7/site-packages/celery/worker/consumer.py", line 401, in on_poll_init
self.connection.transport.on_poll_init(hub.poller)
File "/venv/lib/python2.7/site-packages/kombu/transport/redis.py", line 749, in on_poll_init
self.cycle.on_poll_init(poller)
File "/venv/lib/python2.7/site-packages/kombu/transport/redis.py", line 266, in on_poll_init
num=channel.unacked_restore_limit,
File "/venv/lib/python2.7/site-packages/kombu/transport/redis.py", line 159, in restore_visible
self.restore_by_tag(tag, client)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 35, in __exit__
self.gen.throw(type, value, traceback)
File "/venv/lib/python2.7/site-packages/kombu/transport/redis.py", line 94, in Mutex
pipe.watch(name)
File "/venv/lib/python2.7/site-packages/redis/client.py", line 1941, in watch
return self.execute_command('WATCH', *names)
File "/venv/lib/python2.7/site-packages/redis/client.py", line 1760, in execute_command
return self.immediate_execute_command(*args, **kwargs)
File "/venv/lib/python2.7/site-packages/redis/client.py", line 1779, in immediate_execute_command
return self.parse_response(conn, command_name, **options)
File "/venv/lib/python2.7/site-packages/redis/client.py", line 1883, in parse_response
self, connection, command_name, **options)
File "/venv/lib/python2.7/site-packages/redis/client.py", line 388, in parse_response
response = connection.read_response()
File "/venv/lib/python2.7/site-packages/redis/connection.py", line 309, in read_response
raise response
ResponseError: unknown command 'WATCH'
[2013-06-12 18:25:23,059: ERROR/MainProcess] consumer: Connection to broker lost. Trying to re-establish the connection...

redis:

[1197] 12 Jun 18:50:09 * Server started, Redis version 1.3.14
[1197] 12 Jun 18:50:09 * DB loaded from disk: 0 seconds
[1197] 12 Jun 18:50:09 * The server is now ready to accept connections on port 6379
[1197] 12 Jun 18:50:09 - Accepted 127.0.0.1:53061
[1197] 12 Jun 18:50:09 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[1197] 12 Jun 18:50:09 - 1 clients connected (0 slaves), 1076976 bytes in use
[1197] 12 Jun 18:50:09 - Accepted 127.0.0.1:53062
[1197] 12 Jun 18:50:09 - Accepted 127.0.0.1:53063
[1197] 12 Jun 18:50:09 - Client closed connection
[1197] 12 Jun 18:50:09 - Accepted 127.0.0.1:53064
[1197] 12 Jun 18:50:09 - Client closed connection
[1197] 12 Jun 18:50:09 - Accepted 127.0.0.1:53065
[1197] 12 Jun 18:50:09 - Client closed connection
[1197] 12 Jun 18:50:09 - Accepted 127.0.0.1:53066
[1197] 12 Jun 18:50:09 - Client closed connection
etc etc.

Any guidance for where I should be looking or what possible culprits are? thanks

like image 639
Chris B. Avatar asked Jun 12 '13 22:06

Chris B.


People also ask

How Redis works with Celery?

Redis is the datastore and message broker between Celery and Django. In other words, Django and Celery use Redis to communicate with each other (instead of a SQL database). Redis can also be used as a cache as well. An alternative for Django & Celery is RabbitMQ (not covered here).

Can't connect to Redis?

Firewall restriction is another common reason that can trigger the “could not connect to Redis connection refused”. By default Redis server listen to the TCP port 6379. If another application is using the port or if the firewall restrictions blocks the port, it can trigger the connection refused error.


1 Answers

Your Redis server is too old (1.3.14) to be used with Celery. From this error you can see Celery is trying to use the WATCH command which was introduced in Redis 2.2.

like image 69
Mark Lavin Avatar answered Oct 12 '22 23:10

Mark Lavin