Profiling the GIL

Question

Is there a way to profile a python process' usage of the GIL? Basically, I want to find out what percentage of the time the GIL is held. The process is single-threaded.

My motivation is that I have some code written in Cython, which uses nogil. Ideally, I would like to run it in a multi-threaded process, but in order to know if that can potentially be a good idea, I need to know if the GIL is free a significant amount of the time.

---

I found this related question, from 8 years ago. The sole answer there is "No". Hopefully, things have changed since then.

Answer 1

Completely by accident, I found a tool which does just this: gil_load.

It was actually published after I posted the question.

Well done, @chrisjbillington.

>>> import sys, math
>>> import gil_load
>>> gil_load.init()
>>> gil_load.start(output = sys.stdout)
>>> for x in range(1, 1000000000):
...     y = math.log(x**math.pi)
[2017-03-15 08:52:26]  GIL load: 0.98 (0.98, 0.98, 0.98)
[2017-03-15 08:52:32]  GIL load: 0.99 (0.99, 0.99, 0.99)
[2017-03-15 08:52:37]  GIL load: 0.99 (0.99, 0.99, 0.99)
[2017-03-15 08:52:43]  GIL load: 0.99 (0.99, 0.99, 0.99)
[2017-03-15 08:52:48]  GIL load: 1.00 (1.00, 1.00, 1.00)
[2017-03-15 08:52:52]  GIL load: 1.00 (1.00, 1.00, 1.00)
<...>

>>> import sys, math
>>> import gil_load
>>> gil_load.init()
>>> gil_load.start(output = sys.stdout)
>>> for x in range(1, 1000000000):
...     with open('/dev/null', 'a') as f:
...         print(math.log(x**math.pi), file=f)

[2017-03-15 08:53:59]  GIL load: 0.76 (0.76, 0.76, 0.76)
[2017-03-15 08:54:03]  GIL load: 0.77 (0.77, 0.77, 0.77)
[2017-03-15 08:54:09]  GIL load: 0.78 (0.78, 0.78, 0.78)
[2017-03-15 08:54:13]  GIL load: 0.80 (0.80, 0.80, 0.80)
[2017-03-15 08:54:19]  GIL load: 0.81 (0.81, 0.81, 0.81)
[2017-03-15 08:54:23]  GIL load: 0.81 (0.81, 0.81, 0.81)
[2017-03-15 08:54:28]  GIL load: 0.81 (0.81, 0.81, 0.81)
[2017-03-15 08:54:33]  GIL load: 0.80 (0.80, 0.80, 0.80)
<...>