What is the point of Celery chain if the whole chain breaks if one of the tasks fail?!!
I have this Celery chain:
res = chain(workme.s ( y=1111 ), workme2.s( 2222 ), workme3.s( 3333 ),)()
And I made workme2 fails with retries like this:
@celery.task(default_retry_delay=5, max_retries = 10, queue="sure")
def workme2(x,y):
# try:
try:
print str(y)
sleep(2)
print str(x)
## adding any condition that makes the task fail
if x!=None:
raise Exception('Aproblem from your workme task')
print 'This is my username: ' + str(x['user']) + \
' And Password: ' + str(x['pas'])
return "22xx"
except Exception, exc:
workme2.retry(args=[x,y], exc=exc,)
That is the point.
Forming a chain means your subtasks have some kind of serial dependency: Each one only makes sense if the previous one has been performed. Without this, you would simply use queueing or use a group rather than a chain.
So if one subtask fails (and still fails after attempting all its retries), the chain fails.
I readily admit that the documentation (as of Celery 3.1.18) is far from explicit in this respect, but the name suggests this semantics: "A chain is only as strong as its weakest link."
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With