Forcing a thread to block all other threads from executing

Question

UPDATE:

This answer states that what I'm trying to do is impossible as of April 2013. This, however, seems to contradict what Alex Martelli says in Python Cookbook (p. 624, 3rd ed.):

Upon return, PyGILState_Ensure() always guarantees that the calling thread has exclusive access to the Python interpreter. This is true even if the calling C code is running a different thread that is unknown to the interpreter.

The docs also seem to suggest GIL can be acquired, which would give me hope (except I don't think I can call PyGILState_Ensure() from pure python code, and if I create a C extension to call it, I'm not sure how to embed my memory_daemon() in that).

Perhaps I'm misreading either the answer or Python Cookbook and the docs.

ORIGINAL QUESTION:

I want a given thread (from threading module) to prevent any other thread from running while a certain segment of its code is executing. What's the easiest way to achieve it?

Obviously, it would be great to minimize code changes in the other threads, to avoid using C and direct OS calls, and to make it cross-platform for windows and linux. But realistically, I'll be happy to just have any solution whatsoever for my actual environment (see below).

Environment:

• CPython
• python 3.4 (but can upgrade to 3.5 if it helps)
• Ubuntu 14.04

Use case:

For debugging purposes, I calculate memory used by all the objects (as reported by gc.get_objects()), and print some summary report to sys.stderr. I do this in a separate thread, because I want this summary delivered asynchronously from other threads; I put time.sleep(10) at the end of the while True loop that does the actual memory usage calculation. However, the memory reporting thread takes a while to complete each report, and I don't want all the other threads to move ahead before the memory calculation is finished (otherwise, the memory snapshot will be really hard to interpret).

Example (to clarify the question):

import threading as th
import time

def report_memory_consumption():
  # go through `gc.get_objects()`, check their size and print a summary
  # takes ~5 min to run

def memory_daemon():
  while True:
    # all other threads should not do anything until this call is complete
    report_memory_consumption()
    # sleep for 10 sec, then update memory summary
    # this sleep is the only time when other threads should be executed
    time.sleep(10)


def f1():
  # do something, including calling many other functions
  # takes ~3 min to run

def f2():
  # do something, including calling many other functions
  # takes ~3 min to run


def main():
  t_mem = th.Thread(target = memory_daemon)
  t1 = th.Thread(target = f1)
  t2 = th.Thread(target = f2)
  t_mem.start()
  t1.start()
  t2.start()

# requirement: no other thread is running while t_mem is not sleeping

Answer 1

You should use threading locks to execute code synchronously between threads. The answer given is somewhat correct but I would use reentrant locals to check again to see if you indeed have the lock.

Do not use variables as described in another answer to check for lock possession. The variables can get corrupted between multiple threads. Reentrant locks were meant to solve this problem.

Also what's incorrect in that code is that lock is released assuming the code between doesn't throw exception. so always do in with context or try-catch-finally.

Here is an excellent article explaining synchronization in Python and threading docs.

Edit: Answering OP's update on embedding Python in C

You misunderstood what he said in the cookbook. PyGILState_Ensure returns the GIL if a GIL is available in the current python interpreter but not C threads which is unknown to the python interpreter.

You can't force to get GIL from other threads in the current interpreter. Imagine if you were able to, then basically you will cannibalize all other threads.