Parallel programming with coroutines in Python

Question

Coroutines are a great paradigm to ease concurrent programming. And most of the time, concurrent tasks are easily parallelizable. In Go language, it is easy to use goroutines to perform parallel tasks. Is there a way to do the same thing in Python, e.g. to use coroutines to create processes and to synchronize them?

Answer 1

Yes, Python has support for coroutines in libraries and via generators: see the Greenlet library, for example. Also, there is a derivative named Stackless Python that has built-in support for several concurrent programming features, such as microthreads and channels.

Note that in default CPython, the Global Interpreter Lock will only allow one thread to run at once, which may be a problem.

Answer 2

If you want to use standard python interpreter, greenlet library is the way to go.

As for GIL, it shouldn't be a problem for coroutines. You could consider greenlet coroutine model as multiple light-weight user-space 'threads' running within one kernel thread. So from GIL and OS point of view, it's still single threaded.