Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Celery task always PENDING

I try to run Celery example on Windows with redis backend. The code looks like:

from celery import Celery

app = Celery('risktools.distributed.celery_tasks',
             backend='redis://localhost',
             broker='redis://localhost')

@app.task(ignore_result=False)
def add(x, y):
    return x + y

@app.task(ignore_result=False)
def add_2(x, y):
    return x + y

I start the tasks using iPython console:

>>> result_1 = add.delay(1, 2)    
>>> result_1.state
'PENDING'
>>> result_2 = add_2.delay(2, 3)    
>>> result_2.state
'PENDING'

It seems that both tasks were not executed, but Celery worker output shows that they succeeded:

[2014-12-08 15:00:09,262: INFO/MainProcess] Received task: risktools.distributed.celery_tasks.add[01dedca1-2db2-48df-a4d6-2f06fe285e45]
[2014-12-08 15:00:09,267: INFO/MainProcess] Task celery_tasks.add[01dedca1-2db2-48df-a4d6-2f06fe28
5e45] succeeded in 0.0019998550415s: 3
[2014-12-08 15:00:24,219: INFO/MainProcess] Received task: risktools.distributed.celery_tasks.add[cb5505ce-cf93-4f5e-aebb-9b2d98a11320]
[2014-12-08 15:00:24,230: INFO/MainProcess] Task celery_tasks.add[cb5505ce-cf93-4f5e-aebb-9b2d98a1
1320] succeeded in 0.010999917984s: 5

I've tried to troubleshoot this issue according to Celery documentation, but none of the advices were useful. What am I doing wrong and how can I receive results from a Celery task?

UPD: I've added a task without ignore_result parameter, but nothing has changed

@app.task
def add_3(x, y):
    return x + y

>>>r = add_3.delay(2, 2)
>>>r.state
'PENDING'
like image 782
Ivan Gromov Avatar asked Dec 08 '14 12:12

Ivan Gromov


4 Answers

According to Celery 'Getting Started' not able to retrieve results; always pending and https://github.com/celery/celery/issues/2146 it is a Windows issue. Celery --pool=solo option solves the issue.

like image 50
Ivan Gromov Avatar answered Oct 09 '22 22:10

Ivan Gromov


Instead of Celery --pool=solo option, try -P threads on Windows.

like image 44
Jinshuai Ma Avatar answered Oct 09 '22 21:10

Jinshuai Ma


Remove the ignore_result=False from the celery docs

Task.ignore_result

Don’t store task state. Note that this means you can’t 
use AsyncResult to check if the task is ready, or get its return value.
like image 1
user2097159 Avatar answered Oct 09 '22 21:10

user2097159


Setting CELERY_TASK_TRACK_STARTED = True (or track_started=True on individual tasks) can also help - this will enable the STARTED status.

like image 3
Aaron Neugebauer Avatar answered Oct 09 '22 22:10

Aaron Neugebauer