Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to configure docker to use redis with celery

docker-compose.yml

version: '3.1'
services:

  redis:
    image: redis:latest
    container_name: rd01
    ports:
     - '6379:6379'

  webapp:
    image: webapp
    container_name: wa01
    ports: 
      - "8000:8000"
    links:
      - redis
    depends_on:
      - redis


  celery:
    build: .
    container_name: cl01
    command: celery -A server worker -l info
    depends_on:
      - redis

I also don't feel I understand links and depends_on, I tried different combinations.

Celery cannot connect to redis. I get the following error -

[2018-08-01 13:59:42,249: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 111] Connection refused.

I believe I have set broker URLs correctly in settings.py in my django application (webapp image)

CELERY_BROKER_URL = 'redis://redis:6379/0' 
CELERY_RESULT_BACKEND = 'redis://redis:6379/0'

What is the right way to dockerize a django project with celery and redis? TIA.

EDITS

celery.py

from __future__ import absolute_import, unicode_literals
import os
from celery import Celery
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'server.settings')

app = Celery('server')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()


@app.task(bind=True)
def debug_task(self):
    print('Request: {0!r}'.format(self.request))

This is my django project to it's simplest form to reproduce the error.

like image 304
sP_ Avatar asked Aug 01 '18 18:08

sP_


People also ask

How celery works with Redis?

Celery is a task queue with focus on real-time processing, while also supporting task scheduling. Redis is a message broker. This means it handles the queue of "messages" between Django and Celery. Django is a web framework made for perfectionists with deadlines.


Video Answer


1 Answers

You have to add the redis url while initialize the Celery classas,

from __future__ import absolute_import, unicode_literals
import os
from celery import Celery

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'server.settings')

app = Celery('server', broker='redis://redis:6379/0') # Change is here <<<<
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()


@app.task(bind=True)
def debug_task(self):
    print('Request: {0!r}'.format(self.request))

UPDATE

[After a long discussion] Change your docker-compose.yml as

version: '3.1'
services:

  redis:
    image: redis:latest
    container_name: rd01

  webapp:
    build: .
    container_name: wa01
    ports:
      - "8000:8000"
    links:
      - redis
    depends_on:
      - redis


  celery:
    build: .
    volumes:
      - .:/src
    container_name: cl01
    command: celery -A server worker -l info
    links:
      - redis

and Dockerfile as

FROM python:3.6
RUN mkdir /webapp
WORKDIR /webapp
COPY . /webapp
RUN pip install -r requirements.txt
EXPOSE 8000
CMD ["/start.sh"]
like image 63
JPG Avatar answered Oct 23 '22 08:10

JPG