Celery AttributeError: async error

Question

I have RabbitMQ and Celery running locally on my Mac (OS/X 10.13.4), the following code works locally when I run add.delay(x,y):

#!/usr/bin/env python
from celery import Celery
from celery.utils.log import get_task_logger

logger = get_task_logger(__name__)

app = Celery('tasks', \
        broker='pyamqp://appuser:xx@c2/appvhost', \
        backend='db+mysql://appuser:xx@c2/pigpen')

@app.task(bind=True)
def dump_context(self, x, y):
    print('Executing task id {0.id}, args: {0.args!r} kwargs {0.kwargs!r}'.format(self.request))

@app.task
def add(x, y):
    logger.info('Adding {0} + {1}'.format(x, y))
    return x + y

However when I try to run the Celery worker on an ODROID-C2 running Kali 2018.2 (w. current updates, I get the following error when running celery -A tasks worker --loglevel=info:

Traceback (most recent call last):
  File "/usr/local/bin/celery", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/celery/__main__.py", line 14, in main
    _main()
  File "/usr/local/lib/python2.7/dist-packages/celery/bin/celery.py", line 326, in main
    cmd.execute_from_commandline(argv)
  File "/usr/local/lib/python2.7/dist-packages/celery/bin/celery.py", line 488, in execute_from_commandline
    super(CeleryCommand, self).execute_from_commandline(argv)))
  File "/usr/local/lib/python2.7/dist-packages/celery/bin/base.py", line 281, in execute_from_commandline
    return self.handle_argv(self.prog_name, argv[1:])
  File "/usr/local/lib/python2.7/dist-packages/celery/bin/celery.py", line 480, in handle_argv
    return self.execute(command, argv)
  File "/usr/local/lib/python2.7/dist-packages/celery/bin/celery.py", line 412, in execute
    ).run_from_argv(self.prog_name, argv[1:], command=argv[0])
  File "/usr/local/lib/python2.7/dist-packages/celery/bin/worker.py", line 221, in run_from_argv
    return self(*args, **options)
  File "/usr/local/lib/python2.7/dist-packages/celery/bin/base.py", line 244, in __call__
    ret = self.run(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/celery/bin/worker.py", line 255, in run
    **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/celery/worker/worker.py", line 99, in __init__
    self.setup_instance(**self.prepare_args(**kwargs))
  File "/usr/local/lib/python2.7/dist-packages/celery/worker/worker.py", line 122, in setup_instance
    self.should_use_eventloop() if use_eventloop is None
  File "/usr/local/lib/python2.7/dist-packages/celery/worker/worker.py", line 241, in should_use_eventloop
    self._conninfo.transport.implements.async and
  File "/home/autossh/.local/lib/python2.7/site-packages/kombu/transport/base.py", line 125, in __getattr__
    raise AttributeError(key)
AttributeError: async

From the Kali ODROID I am able to connect to the RabbitMQ instance on the host named c2 using a Python Pika script and mysql from that device works to the c2 machine as well. I have found similar errors, none of those solutions have worked for me.

Celery version installed on the ODROID-C2 via pip is:

celery --version
4.1.0 (latentcall)

Answer 1

We sorted by just updating to celery==4.1.1

it seems the latest release for the 4.1.X sorted out the module name change on kombu

Answer 2

Make sure you are using Kombu 4.1.0. The latest version of Kombu renames async to asynchronous.

Answer 3

Celery does not pin its requirements for kombu and billiard to specific versions. They require the following:

billiard>=3.5.0.2,<3.6.0
kombu>=4.0.2,<5.0

https://github.com/celery/celery/blob/v4.1.0/requirements/default.txt

kombu 4.2.0 was released with a breaking change and previous versions of celery automatically install it.

Since Celery doesn't pin specific versions, you should pin to the following if you will continue to use celery 4.1.0:

kombu==4.1.0
billiard==3.5.0.2