How to debug Celery/Django tasks running locally in Eclipse

Question

I need to debug Celery task from the Eclipse debugger. I'm using Eclipse, PyDev and Django.

First, I open my project in Eclipse and put a breakpoint at the beginning of the task function.

Then, I'm starting the Celery workers from Eclipse by Right Clicking on manage.py from the PyDev Package Explorer and choosing "Debug As->Python Run" and specifying "celeryd -l info" as the argument. This starts MainThread, Mediator and three more threads visible from the Eclipse debugger.

After that I return back to the PyDev view and start the main application by Right Click on the project and choosing Run As/PyDev:Django

My issues is that once the task is submitted by the mytask.delay() it doesn't stop on the breakpoint. I put some traces withing the tasks code so I can see that it was executed in one of the worker threads.

So, how to make the Eclipse debugger to stop on the breakpoint placed withing the task when it executed in the Celery workers thread?

Answer 1

You should consider the option to run the celery task in the same thread as the main process (normally it runs on a separate process), this will make the debug much easier.

You can tell celery to run the task in sync by adding this setting to your settings.py module:

CELERY_TASK_ALWAYS_EAGER  = True # use this if you are on older versions of celery # CELERY_ALWAYS_EAGER = True  

Note: this is only meant to be in use for debugging or development stages!

Answer 2

You can do it using Celery's rdb:

from celery.contrib import rdb rdb.set_trace() 

Then, in a different terminal type telnet localhost 6900, and you will get the debug prompt.