How do I run celery status/flower without the -A option?

Question

Consider this bash session:

$ export DJANGO_SETTINGS_MODULE=web.settings
$ celery status -b redis://redis.businessoptics.dev:6379/1 -t 10
Error: No nodes replied within time constraint.
$ celery status -b redis://redis.businessoptics.dev:6379/1 -t 10 -A scaffold.tasks.celery_app
[email protected]: OK

Why do I need the -A option? As far as I can tell celery should be able to detect the necessary metadata on redis.

Similarly if I run celery flower -b <redis url> it shows that it successfully connects to redis but doesn't show any real workers/tasks/queues and shows several messages like 'stats' inspect method failed. Again, adding -A causes it to work.

I want to run flower in a minimal standalone Docker container that doesn't contain any of my code or its dependencies. Several repos such as this one offer this kind of thing. So how can I do that? The linked repo offers many options but no way to specify the -A option, which suggests it is not necessary.

I'm a beginner to celery so I may be missing something stupid. What am I supposed to do?

The scaffold.tasks.celery_app module simply looks like this:

from celery import Celery
from django.conf import settings

app = Celery()
app.config_from_object(settings)

And these are the Django settings that involve celery:

{'BROKER_HEARTBEAT': 0,
 'BROKER_TRANSPORT_OPTIONS': {'fanout_patterns': True,
                              'fanout_prefix': True,
                              'visibility_timeout': 172800},
 'BROKER_URL': 'redis://redis.businessoptics.dev:6379/1',
 'CELERYBEAT_SCHEDULE': {'journey-heartbeat': {'args': (),
                                               'schedule': <crontab: * * * * * (m/h/d/dM/MY)>,
                                               'task': 'kms.data.journey.tasks.heartbeat'}},
 'CELERYD_CONCURRENCY': 1,
 'CELERYD_HIJACK_ROOT_LOGGER': False,
 'CELERYD_LOG_COLOR': False,
 'CELERYD_MAX_TASKS_PER_CHILD': 1,
 'CELERYD_PREFETCH_MULTIPLIER': 1,
 'CELERY_ACCEPT_CONTENT': ['pickle'],
 'CELERY_ACKS_LATE': True,
 'CELERY_DEFAULT_EXCHANGE': 'default',
 'CELERY_DEFAULT_EXCHANGE_TYPE': 'direct',
 'CELERY_DEFAULT_QUEUE': 'default',
 'CELERY_DEFAULT_ROUTING_KEY': 'default',
 'CELERY_IGNORE_RESULT': False,
 'CELERY_IMPORTS': ['kms.knowledge.query.tasks2',
                    # names of several more modules...
                   ],
 'CELERY_QUEUES': [<unbound Queue tablestore -> <unbound Exchange default(direct)> -> kms.data.table_store.tasks.#>,
                    # several more similar-looking Queues...
                   <unbound Queue default -> <unbound Exchange default(direct)> -> default>],
 'CELERY_REDIRECT_STDOUTS': False,
 'CELERY_RESULT_BACKEND': 'database',
 'CELERY_RESULT_DBURI': 'mysql://businessoptics:[email protected]:3306/product',
 'CELERY_RESULT_DB_SHORT_LIVED_SESSIONS': True,
 'CELERY_ROUTES': ['scaffold.tasks.routers.TaskNameRouter'],
 'CELERY_SEND_EVENTS': True,
 'CELERY_SEND_TASK_ERROR_EMAILS': False,
 'CELERY_SEND_TASK_SENT_EVENT': True,
 'CELERY_STORE_ERRORS_EVEN_IF_IGNORED': True,
 'CELERY_TASKNAME_ROUTES': [('tablestore', 'kms.data.table_store.tasks.#'),
                            # bunch of routes...
                            ],
 'CELERY_TASK_RESULT_EXPIRES': None,
 'CELERY_TIMEZONE': 'UTC',
 'CELERY_TRACK_STARTED': True,
 'CELERY_WORKER_DIRECT': True
}

Here are the relevant versions:

celery==3.1.19
Django==1.8
django-celery==3.1.0
redis==2.10.3

Answer 1

The -A option is the one that passes the celery instance with related configuration including the package containing your tasks.

To use all the features flowers needs to be configured like a worker, this means knowing the package where your celery tasks are and knowing them.

Add to your docker container the needed python lib shouldn't be that hard, for example you could add to this file the configuration line CELERY_IMPORTS in the following way:

CELERY_IMPORTS  = os.getenv('CELERY_IMPORTS  ', 'default.package') 

UPDATE

As @asksol, celery creator, pointed out in the comments here's a more detailed explanation of why you need the -A option:

Flower is also a message consumer and so will help recover unacked messages. Since you have a custom visibility defined, starting flower unconfigured means it will use the default visibility timeout and thus will redeliver unacked messages faster than your workers. Always use -A so that worker, flower and client configuration is in sync