Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python threading and GIL

I was reading about the GIL and it never really specified if this includes the main thread or not (i assume so). Reason I ask is because I have a program with threads setup that modify a dictionary. The main thread adds/deletes based on player input while a thread loops the data updating and changing data.

However in some cases a thread may iterate over the dictionary keys where one could delete them. If there is a so called GIL and they are run sequentially, why am I getting dict changed errors? If only one is suppose to run at a time, then technically this should not happen.

Can anyone shed some light on such a thing? Thank you.

like image 823
Charles Avatar asked Jan 20 '11 15:01

Charles


People also ask

How does threading work in Python with the GIL?

The GIL allows only one OS thread to execute Python bytecode at any given time, and the consequence of this is that it's not possible to speed up CPU-intensive Python code by distributing the work among multiple threads. This is, however, not the only negative effect of the GIL.

Does Python still have GIL?

The GIL's low performance overhead really shines for single-threaded operations, including I/O-multiplexed programs where libraries like asyncio are used, and this is still a predominant use of Python.

How does Python multiprocessing work with GIL?

In CPython, the Global Interpreter Lock (GIL) is a mutex that allows only one thread at a time to have the control of the Python interpreter. In other words, the lock ensures that only one thread is running at any given time. Therefore, it is impossible to take advantage of multiple processors with threads.

Is Python good for threading?

No, it is not a good idea. Multithreading is not possible in Python due to something called the Global Interpreter Lock.


2 Answers

They are running at the same time, they just don't execute at the same time. The iterations might be interleaved. Quote Python:

The mechanism used by the CPython interpreter to assure that only one thread executes Python bytecode at a time.

So two for loops might run at the same time, there will just be no (for example) two del dict[index]'s at the same time.

like image 70
orlp Avatar answered Sep 29 '22 14:09

orlp


The GIL locks at a Python byte-code level, and applies to all threads, even the main thread. If you have one thread modifying a dictionary, and another iterating keys, they will interfere with each other.

"Only one runs at a time" is true, but you have to understand the unit of granularity. In the case of CPython's GIL, the granularity is a bytecode instruction, so execution can switch between threads at any bytecode.

like image 21
Ned Batchelder Avatar answered Sep 29 '22 12:09

Ned Batchelder