Running asynchronous python code in a Django web application

Question

Is it OK to run certain pieces of code asynchronously in a Django web app. If so how?

For example:

I have a search algorithm that returns hundreds or thousands of results. I want to enter into the database that these items were the result of the search, so I can see what users are searching most. I don't want the client to have to wait an extra hundred or thousand more database inserts. Is there a way I can do this asynchronously? Is there any danger in doing so? Is there a better way to achieve this?

Answer 1

As far as Django is concerned yes.

The bigger concern is your web server and if it plays nice with threading. For instance, the sync workers of gunicorn are single threads, but there are other engines, such as greenlet. I'm not sure how well they play with threads.

Combining threading and multiprocessing can be an issue if you're forking from threads:

Status of mixing multiprocessing and threading in Python

http://bugs.python.org/issue6721

That being said, I know of popular performance analytics utilities that have been using threads to report on metrics, so seems to be an accepted practice.

In sum, seems safest to use the threading.Thread object from the standard library, so long as whatever you do in it doesn't fork (python's multiprocessing library)

https://docs.python.org/2/library/threading.html