Can I use Python requests with celery?

Question

I have the following defined in a celery module named tasks.py with the requests library imported:

@celery.task
def geturl(url):
    res = requests.get(url)
    return res.content

Whenever I call the task (either from tasks.py or REPL) with:

res = geturl.delay('http://www.google.com')
print res.get()

Here are the log entries on the celery server:

[2012-12-19 18:49:58,400: INFO/MainProcess] Starting new HTTP connection (1): www.google.com [2012-12-19 18:49:58,594: INFO/MainProcess] Starting new HTTP connection (1): www.google.ca [2012-12-19 18:49:58,801: INFO/MainProcess] Task tasks.geturl[48182400-7d67-466b-ba5c-867747cb3974] succeeded in 0.402245998383s: None

But when I run this in a synchronous fashion by calling geturl('http://www.google.com'), I get a response. I have read through the docs and cannot seem to clue into why this is not working. The primary use of this is to poll API's could someone please explain why this does not work?

Thanks!

Answer 1

res.content is just an str instance, there's no reason you wouldn't be able to return it. Your issue likely lies elsewhere, probably in your celery configuration.

---

Many configuration parameters play a role in how the results will be stored; have a look at them.
The first one you'll want to check may be CELERY_IGNORE_RESULT.

Your should also check your result broker configuration.