Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

A clean API for making a function call threaded in python

Tags:

I would like to call a function in a thread. Calling it with the conventional API looks like:

from threading import Thread
import numpy as np
a = np.random.rand(int(1e8),1)

Thread(target=np.savez_compressed, args=('/tmp/values.a', dict(a=a))).start()

I was wondering if there is a pythonic was of making this threaded call with a cleaner API, without defining a function which is specific for np.savez_compressed.

E.g. something in the style of (pseudo-code):

@make_threaded
np.savez_compressed('/tmp/values.a', dict(a=a))

Unfortunately decorators can only be applied to function definitions, so the pseudo-code above is not legal.

EDIT: I am not looking specifically for a decorator API. Rather, a cleaner way to make a function call threaded

like image 807
Yuval Atzmon Avatar asked Mar 01 '20 09:03

Yuval Atzmon


People also ask

Can you thread a function in Python?

Threading in Python is simple. It allows you to manage concurrent threads doing work at the same time. The library is called “threading“, you create “Thread” objects, and they run target functions for you. You can start potentially hundreds of threads that will operate in parallel.

How do you call a thread in Python?

start() − The start() method starts a thread by calling the run method. join([time]) − The join() waits for threads to terminate. isAlive() − The isAlive() method checks whether a thread is still executing. getName() − The getName() method returns the name of a thread.

Is Python good for multithreading?

Python doesn't support multi-threading because Python on the Cpython interpreter does not support true multi-core execution via multithreading. However, Python does have a threading library. The GIL does not prevent threading.


1 Answers

The concurrent.futures module provides a more high-level API for using threads or processes for individual operations.

from concurrent.futures import ThreadPoolExecutor

executor = ThreadPoolExecutor()
executor.submit(np.savez_compressed, '/tmp/values.a', dict(a=a))

If you don't want the entire Executor API, you can define your own helper to run a function in a thread.

def threaded(call, *args, **kwargs):
    """Execute ``call(*args, **kwargs)`` in a thread"""
    thread = threading.Thread(target=call, args=args, kwargs=kwargs)
    thread.start()
    return thread

threaded(np.savez_compressed, '/tmp/values.a', dict(a=a))
like image 59
MisterMiyagi Avatar answered Sep 19 '22 11:09

MisterMiyagi