Multithreading for Python Django

Question

Some functions should run asynchronously on the web server. Sending emails or data post-processing are typical use cases.

What is the best (or most pythonic) way write a decorator function to run a function asynchronously?

My setup is a common one: Python, Django, Gunicorn or Waitress, AWS EC2 standard Linux

For example, here's a start:

from threading import Thread  def postpone(function):     def decorator(*args, **kwargs):         t = Thread(target = function, args=args, kwargs=kwargs)         t.daemon = True         t.start()     return decorator 

desired usage:

@postpone def foo():     pass #do stuff 

Answer 1

I've continued using this implementation at scale and in production with no issues.

Decorator definition:

def start_new_thread(function):     def decorator(*args, **kwargs):         t = Thread(target = function, args=args, kwargs=kwargs)         t.daemon = True         t.start()     return decorator 

Example usage:

@start_new_thread def foo():   #do stuff 

Over time, the stack has updated and transitioned without fail.

Originally Python 2.4.7, Django 1.4, Gunicorn 0.17.2, now Python 3.6, Django 2.1, Waitress 1.1.

If you are using any database transactions, Django will create a new connection and this needs to be manually closed:

from django.db import connection  @postpone def foo():   #do stuff   connection.close()