How often does Python switch threads?

Question

In cPython (the default Python implementation), how often does thread switching happen? This is not a question about using multiple cores; I know about the GIL, and realize that only one thread will be run at at time.

I have seen a few cases where I have two log messages in sequence, and the first log message will be emitted, but the second is not emitted for several seconds. This probably happens because Python decided to switch threads in between the two log statements. This leads me to believe that Python switches between threads once every couple of seconds, which is much slower than I would have expected. Is this correct?

Answer 1

By default, Python 2 will switch threads every 100 instructions. This can be adjusted with sys.setcheckinterval which is documented here: https://docs.python.org/2/library/sys.html#sys.setcheckinterval

I found additional information on pages 10, 11, and 12 of this presentation: http://www.dabeaz.com/python/UnderstandingGIL.pdf

(These answers come from the comments of the question, but since nobody has written them down in an answer I will do it myself. It has been 6+ months since the comments were made.)

UPDATE:

Since Python 3.2, sys.setcheckinterval was deprecated. CPython has another approach to improve the GIL. Instead of release the GIL after 100 virtual instruction, they changed to switch by time interval instead.

By default, the GIL will be released after 5 milliseconds (5000 microseconds) so that other thread can have a change to acquire the GIL. And you can change this default value by using sys.setswitchinterval

Answer 2

Since Python 3.2 the interpreter’s thread switch time interval can be adjusted using sys.setswitchinterval: https://docs.python.org/3.6/library/sys.html#sys.setswitchinterval

In earlier versions the interpreter checks for periodic things such as thread switches and signal handlers by default every 100 virtual instructions. This can be modified by sys.setcheckinterval: https://docs.python.org/2/library/sys.html#sys.setcheckinterval