How to interrupt Python I/O operations when threading?

Question

For example,

with open("foo") as f:
  f.read()

(But it could be a file write, a DNS lookup, any number of other I/O operations.)

If I interrupt this program while reading (SIGINT), the I/O operation is halted and KeyboardInterrupt is thrown, and the finalizers run.

However, if this happens on a thread other than the main thread, the I/O operation is not interrupted.

So...how do I interrupt an I/O operation on another thread (similar to how it's interrupted on the main thread)?

Answer 1

Keyboard-interrupt events are always captured on the main thread, they do not directly impact other threads (in the sense that they won't be interrupted due to a Ctrl+C). src1 src2 (in a comment)

Here you have a sample example of a long IO bound operation, which gives us time to kill it before it finishes. KeyboardInterrupt works as you would expect.

import random
import threading


def long_io(file_name):
    with open(file_name, "w") as f:
        i = 0
        while i < 999999999999999999999999999:
            f.write(str(random.randint(0, 99999999999999999999999999)))
            i += 1


t = threading.Thread(target=long_io, args=("foo",), daemon=True)
t.start()

# keep the main thread alive, listening to possible KeyboardInterupts
while t.is_alive():
    t.join(1)  # try to join for 1 second, this gives a small window between joins in which the KeyboardInterrupt can rise

Notice that:

• The thread is flagged as daemon; this way, on KeyboardInterrupt, the main thread will not wait until the IO is finished, but kill it. You could use non-daemonic threads (recommended) as explained here, but for this example killing them straight away suffices.

A thread can be flagged as a “daemon thread”. The significance of this flag is that the entire Python program exits when only daemon threads are left. The initial value is inherited from the creating thread. The flag can be set through the daemon property or the daemon constructor argument. Daemon threads are abruptly stopped at shutdown. Their resources (such as open files, database transactions, etc.) may not be released properly. If you want your threads to stop gracefully, make them non-daemonic and use a suitable signalling mechanism such as an Event. src

make the child thread daemonic, which means that its parent (the main thread here) will kill it when it exits (only non-daemon threads are not killed but joined when their parent exits) src

• We keep the main thread alive, so that it does not immediately finish but waits until the child thread finishes. Otherwise, the main thread (the one in charge of detecting Keyboard Interrupts) would be gone (killing the child thread if it is daemonic, or waiting for a join if it is not daemonic). We could have used a simple t.join() for that, but we did not. Why? Because the KeyboardInterrupt would also be impacted and it would only be raised after the join is completed.

the execution in the main thread remains blocked at the line thread.join(). src