Menu
The Python Global Interpreter Lock or GIL, in simple words, is a mutex (or a lock) that allows only one thread to hold the control of the Python interpreter. This means that only one thread can be in a state of execution at any point in time.
Python Global Interpreter Lock (GIL) is a type of process lock which is used by python whenever it deals with processes. Generally, Python only uses only one thread to execute the set of written statements. This means that in python only one thread will be executed at a time.
A global interpreter lock (GIL) is a mechanism used in computer-language interpreters to synchronize the execution of threads so that only one native thread (per process) can execute at a time. An interpreter that uses GIL always allows exactly one thread to execute at a time, even if run on a multi-core processor.
In CPython, the global interpreter lock, or GIL, is a mutex that protects access to Python objects, preventing multiple threads from executing Python bytecodes at once. The GIL prevents race conditions and ensures thread safety.
Yes, calls to C extensions (C routines called from Python) are still subject to the GIL.
However, you can manually release the GIL inside your C extension, so long as you are careful to re-assert it before returning control to the Python VM.
For information, take a look at the Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS macros: http://docs.python.org/c-api/init.html#thread-state-and-the-global-interpreter-lock
C/C++ extensions to Python are not bound by the GIL. However, you really need to know what you're doing. From http://docs.python.org/c-api/init.html:
The global interpreter lock is used to protect the pointer to the current thread state. When releasing the lock and saving the thread state, the current thread state pointer must be retrieved before the lock is released (since another thread could immediately acquire the lock and store its own thread state in the global variable). Conversely, when acquiring the lock and restoring the thread state, the lock must be acquired before storing the thread state pointer.
Why am I going on with so much detail about this? Because when threads are created from C, they don’t have the global interpreter lock, nor is there a thread state data structure for them. Such threads must bootstrap themselves into existence, by first creating a thread state data structure, then acquiring the lock, and finally storing their thread state pointer, before they can start using the Python/C API. When they are done, they should reset the thread state pointer, release the lock, and finally free their thread state data structure.
Check out Cython, it has similar syntax to Python but with a few constructs like "cdef", fast numpy access functions, and a "with nogil" statement (which does what it says).
If you’re writing your extension in C++, you can use RAII to easily and legibly write code manipulating the GIL. I use this pair of RAII structlets:
namespace py {
namespace gil {
struct release {
PyThreadState* state;
bool active;
release()
:state(PyEval_SaveThread()), active(true)
{}
~release() { if (active) { restore(); } }
void restore() {
PyEval_RestoreThread(state);
active = false;
}
};
struct ensure {
PyGILState_STATE* state;
bool active;
ensure()
:state(PyGILState_Ensure()), active(true)
{}
~ensure() { if (active) { restore(); } }
void restore() {
PyGILState_Release(state);
active = false;
}
};
}
}
… allowing the GIL to be toggled for a given block (in a semantic manner that may seem dimly familiar for any context-manager Pythonista fans):
PyObject* YourPythonExtensionFunction(PyObject* self, PyObject* args) {
Py_SomeCAPICall(…); /// generally, if it starts with Py* it needs the GIL
Py_SomeOtherCall(…); /// ... there are exceptions, see the docs
{
py::gil::release nogil;
std::cout << "Faster and less block-y I/O" << std::endl
<< "can run inside this block -" << std::endl
<< "unimpeded by the GIL";
}
Py_EvenMoreAPICallsForWhichTheGILMustBeInPlace(…);
}
… Indeed, personally also I find the ease of extending Python, and the level of control one has over the internal structures and state, a killer feature.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With