Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do threads work in Python, and what are common Python-threading specific pitfalls?

I've been trying to wrap my head around how threads work in Python, and it's hard to find good information on how they operate. I may just be missing a link or something, but it seems like the official documentation isn't very thorough on the subject, and I haven't been able to find a good write-up.

From what I can tell, only one thread can be running at once, and the active thread switches every 10 instructions or so?

Where is there a good explanation, or can you provide one? It would also be very nice to be aware of common problems that you run into while using threads with Python.

like image 306
jdd Avatar asked Aug 27 '08 23:08

jdd


People also ask

How does threads work in Python?

Python threads are used in cases where the execution of a task involves some waiting. One example would be interaction with a service hosted on another computer, such as a webserver. Threading allows python to execute other code while waiting; this is easily simulated with the sleep function.

What are the limitations of threading in Python?

In fact, a Python process cannot run threads in parallel but it can run them concurrently through context switching during I/O bound operations. This limitation is actually enforced by GIL. The Python Global Interpreter Lock (GIL) prevents threads within the same process to be executed at the same time.

How many types of threads are there in Python?

There are two distinct types of thread. These are: User-level threads: These are the ones we can actively play with within our code etc. Kernel-level threads: These are very low-level threads that act on behalf of the operating system.

How are Python threads executed?

The threading module provided with Python includes a simple-to-implement locking mechanism that allows you to synchronize threads. A new lock is created by calling the Lock() method, which returns the new lock. The acquire(blocking) method of the new lock object is used to force threads to run synchronously.


1 Answers

Yes, because of the Global Interpreter Lock (GIL) there can only run one thread at a time. Here are some links with some insights about this:

  • http://www.artima.com/weblogs/viewpost.jsp?thread=214235
  • http://smoothspan.wordpress.com/2007/09/14/guido-is-right-to-leave-the-gil-in-python-not-for-multicore-but-for-utility-computing/

From the last link an interesting quote:

Let me explain what all that means. Threads run inside the same virtual machine, and hence run on the same physical machine. Processes can run on the same physical machine or in another physical machine. If you architect your application around threads, you’ve done nothing to access multiple machines. So, you can scale to as many cores are on the single machine (which will be quite a few over time), but to really reach web scales, you’ll need to solve the multiple machine problem anyway.

If you want to use multi core, pyprocessing defines an process based API to do real parallelization. The PEP also includes some interesting benchmarks.

like image 153
Peter Hoffmann Avatar answered Sep 23 '22 08:09

Peter Hoffmann