Python Threading inside a class

Question

I recently started with python's threading module. After some trial and error I managed to get basic threading working using the following sample code given in most tutorials.

class SomeThread(threading.Thread):     def __init__(self, count):         threading.Thread.__init__(self)      def run(self):         print "Do something" 

My problem is: I have a Class that has class variables and a function that I want to be run in a separate thread. However the function uses class variables and also writes to class variables. Like so:

class MyClass:     somevar = 'someval'      def func_to_be_threaded(self):         # Uses other class functions         # Do something with class variables 

So how would I essentially 'put the thread class in MyClass'. So that if MyClass().func_to_threaded() is called it would run in a thread.

Answer 1

If I understand correctly you want to run a function in a separate thread? There are several ways to do that. But basically you wrap your function like this:

class MyClass:     somevar = 'someval'      def _func_to_be_threaded(self):         # main body      def func_to_be_threaded(self):         threading.Thread(target=self._func_to_be_threaded).start() 

It can be shortened with a decorator:

def threaded(fn):     def wrapper(*args, **kwargs):         threading.Thread(target=fn, args=args, kwargs=kwargs).start()     return wrapper  class MyClass:     somevar = 'someval'      @threaded     def func_to_be_threaded(self):         # main body 

---

Edit Updated version with a handle:

def threaded(fn):     def wrapper(*args, **kwargs):         thread = threading.Thread(target=fn, args=args, kwargs=kwargs)         thread.start()         return thread     return wrapper  class MyClass:     somevar = 'someval'      @threaded     def func_to_be_threaded(self):         print 'xyz' 

This can be used as follows:

>>> my_obj = MyClass() >>> handle = my_obj.func_to_be_threaded() >>> handle.join() 

---

Now it is possible to extend it even more if you wish to return a value from the function. Consider this:

from threading import Thread from concurrent.futures import Future  def call_with_future(fn, future, args, kwargs):     try:         result = fn(*args, **kwargs)         future.set_result(result)     except Exception as exc:         future.set_exception(exc)  def threaded(fn):     def wrapper(*args, **kwargs):         future = Future()         Thread(target=call_with_future, args=(fn, future, args, kwargs)).start()         return future     return wrapper   class MyClass:     @threaded     def get_my_value(self):         return 1  >>> my_obj = MyClass() >>> fut = my_obj.get_my_value()  # this will run in a separate thread >>> fut.result()  # will block until result is computed 1 

If you don't have concurrent.futures.Future class (because for example you are using Python2.7 or older) then you can use this simplified implementation:

from threading import Event  class Future(object):     def __init__(self):         self._ev = Event()      def set_result(self, result):         self._result = result         self._ev.set()      def set_exception(self, exc):         self._exc = exc         self._ev.set()      def result(self):         self._ev.wait()         if hasattr(self, '_exc'):             raise self._exc         return self._result 

I advice reading through concurrent.futures module since it has a lot of neat tools. For example Thread class should be replaced with a ThreadPoolExecutor instance to limit concurrency (e.g. you don't want to spam 10k threads). Also with ThreadPoolExecutor the code is even simplier (and less error prone):

from concurrent.futures import ThreadPoolExecutor  tp = ThreadPoolExecutor(10)  # max 10 threads  def threaded(fn):     def wrapper(*args, **kwargs):         return tp.submit(fn, *args, **kwargs)  # returns Future object     return wrapper 

Just remember you have to tp.shutdown() after you're done with all parallel work.