Python threads with os.system() calls. Main thread doesn't exit on ctrl+c

Question

Please don't consider it a duplicate before reading, There are a lot of questions about multithreading and keyboard interrupt, but i didn't find any considering os.system and it looks like it's important.

I have a python script which makes some external calls in worker threads. I want it to exit if I press ctrl+c But it look like the main thread ignores it.

Something like this:

from threading import Thread
import sys
import os

def run(i):
    while True:
        os.system("sleep 10")
        print i

def main():
    threads=[]
    try:
        for i in range(0, 3):
            threads.append(Thread(target=run, args=(i,)))
            threads[i].daemon=True
            threads[i].start()
        for i in range(0, 3):
            while True:
                threads[i].join(10)
                if not threads[i].isAlive():
                    break

    except(KeyboardInterrupt, SystemExit):
        sys.exit("Interrupted by ctrl+c\n")


if __name__ == '__main__': 
    main() 

Surprisingly, it works fine if I change os.system("sleep 10") to time.sleep(10).

Answer 1

I'm not sure what operating system and shell you are using. I describe Mac OS X and Linux with zsh (bash/sh should act similar).

When you hit Ctrl+C, all programs running in the foreground in your current terminal receive the signal SIGINT. In your case it's your main python process and all processes spawned by os.system.

Processes spawned by os.system then terminate their execution. Usually when python script receives SIGINT, it raises KeyboardInterrupt exception, but your main process ignores SIGINT, because of os.system(). Python os.system() calls the Standard C function system(), that makes calling process ignore SIGINT (man Linux / man Mac OS X).

So neither of your python threads receives SIGINT, it's only children processes who get it.

When you remove os.system() call, your python process stops ignoring SIGINT, and you get KeyboardInterrupt.

You can replace os.system("sleep 10") with subprocess.call(["sleep", "10"]). subprocess.call() doesn't make your process ignore SIGINT.

Answer 2

I've had this same problem more times than I could count back when i was first learning python multithreading.

Adding the sleep call within the loop makes your main thread block, which will allow it to still hear and honor exceptions. What you want to do is utilize the Event class to set an event in your child threads that will serve as an exit flag to break execution upon. You can set this flag in your KeyboardInterrupt exception, just put the except clause for that in your main thread.

I'm not entirely certain what is going on with the different behaviors between the python specific sleep and the os called one, but the remedy I am offering should work for what your desired end result is. Just offering a guess, the os called one probably blocks the interpreter itself in a different way?

Keep in mind that generally in most situations where threads are required the main thread is going to keep executing something, in which case the "sleeping" in your simple example would be implied.

http://docs.python.org/2/library/threading.html#event-objects