Python: GIL context - switching

Question

So, I generally have a pretty good understanding of how the Global Interpreter Lock (GIL) in Python works. Essentially, while the interpreter is running, one thread holds the GIL for N ticks (where N can be set using sys.setcheckinterval), at which point the GIL is released and another thread can acquire the GIL. The also happens if one thread begins an I/O operation.

What I'm a bit confused about is how this all works with C extension modules.

If you have a C extension module that acquires the GIL, and then executes some python code using PyEval_EvalCode, can the interpreter release the GIL and give it to some other thread? Or will the C thread that acquired the GIL hold the GIL permanently until PyEval_EvalCode returns and the GIL is explicitly released in C?

PyGILState gstate = PyGILState_Ensure();

....

/* Can calling PyEval_EvalCode release the GIL and let another thread acquire it?? */
PyObject* obj = PyEval_EvalCode(code, global_dict, local_dict); 

PyGILState_Release(gstate);

Answer 1

Yes, the interpreter can always release the GIL; it will give it to some other thread after it has interpreted enough instructions, or automatically if it does some I/O. Note that since recent Python 3.x, the criteria is no longer based on the number of executed instructions, but on whether enough time has elapsed.

To get a different effect, you'd need a way to acquire the GIL in "atomic" mode, by asking the GIL not to be released until you release it explicitly. This is impossible so far (but see https://bitbucket.org/arigo/cpython-withatomic for an experimental version).