Celery Ignoring Config Values

Question

Celery seems to be ignoring my config values for some reason. I have set the following values in my apps config.py

BROKER_URL = 'redis://localhost:6379/0'
CELERY_RESULT_BACKEND = 'redis://localhost:6379/0'
CELERY_RESULT_ENGINE_OPTIONS = {"pool_recycle": 7200, 'echo': True}
BROKER_TRANSPORT_OPTIONS = {'visibility_timeout': 3600}

however celery continues to try to connect to the following broker url amqp://guest:**@localhost:5672//

Here is where I try to configure celery

def configure_extensions(app):
  # flask-sqlalchemy
  db.init_app(app)

  # marshmallow
  ma.init_app(app)

  # bcrypt
  bcrypt.init_app(app)

  #celery
  celery.config_from_object(app.config)

And here is my extensions.py

# Flask-SQLAlchemy extension instance
from flask_sqlalchemy import SQLAlchemy
# flask_marshmallow extension instance
from flask_marshmallow import Marshmallow
# Bcrypt
from flask_bcrypt import Bcrypt
# flask_restful
from flask_restful import Api
#celery
from celery import Celery

celery = Celery()

db = SQLAlchemy()

ma = Marshmallow()

bcrypt = Bcrypt()

api = Api()

I have printed out app.config before calling celery.config_from_object(app.config) and it does include the celery values I listed above. I've looked at similar posts on stack overflow and have yet to find one that answers my question.

I am using python 3.6 and celery 4.1

Does anyone know why it is ignoring the config values? I have checked celery docs and I think I am using the right values for the config

Here is a link to a minimal example repository Github Repo

Answer 1

Celery has a problem with Flask and the application factory pattern. Miguel Grinberg wrote a great blog post about it:

https://blog.miguelgrinberg.com/post/celery-and-the-flask-application-factory-pattern

Essentially, you need to pass in the broker URL at the point of creation rather than defer it and update the configuration later.

I modified your app/extensions.py like so:

from celery import Celery                                                       

from . import celeryconfig                                                      
celery = Celery(__name__, broker=celeryconfig.broker_url,                       
                backend=celeryconfig.result_backend) 

And now, running the celery worker yields the following:

bash# celery worker -A app.extensions

transport:   redis://localhost:6379/0
results:     redis://localhost:6379/0