Handling keyboard interrupt when using subproccess

Question

I have python script called monitiq_install.py which calls other scripts (or modules) using the subprocess python module. However, if the user sends a keyboard interrupt (CTRL + C) it exits, but with an exception. I want it to exit, but nicely.

My Code:

import os
import sys
from os import listdir
from os.path import isfile, join
from subprocess import Popen, PIPE
import json

# Run a module and capture output and exit code
def runModule(module):
    try:
        # Run Module
        process = Popen(os.path.dirname(os.path.realpath(__file__)) + "/modules/" + module, shell=True, stdout=PIPE, bufsize=1)
        for line in iter(process.stdout.readline, b''):
            print line,

        process.communicate()
        exit_code = process.wait();

        return exit_code;
    except KeyboardInterrupt:
        print "Got keyboard interupt!";
        sys.exit(0);

The error I'm getting is below:

python monitiq_install.py -a
Invalid module filename: create_db_user_v0_0_0.pyc
Not Running Module: '3parssh_install' as it is already installed
######################################
Running Module: 'create_db_user' Version: '0.0.3'
Choose username for Monitiq DB User [MONITIQ]
^CTraceback (most recent call last):
  File "/opt/monitiq-universal/install/modules/create_db_user-v0_0_3.py", line 132, in <module>
    inputVal = raw_input("");
Traceback (most recent call last):
  File "monitiq_install.py", line 40, in <module>
KeyboardInterrupt
    module_install.runModules();
  File "/opt/monitiq-universal/install/module_install.py", line 86, in runModules
    exit_code = runModule(module);
  File "/opt/monitiq-universal/install/module_install.py", line 19, in runModule
    for line in iter(process.stdout.readline, b''):
KeyboardInterrupt

A solution or some pointers would be helpful :)

--EDIT With try catch

Running Module: 'create_db_user' Version: '0.0.0'
Choose username for Monitiq DB User [MONITIQ]
^CGot keyboard interupt!
Traceback (most recent call last):
  File "monitiq_install.py", line 36, in <module>
    module_install.runModules();
  File "/opt/monitiq-universal/install/module_install.py", line 90, in runModules
    exit_code = runModule(module);
  File "/opt/monitiq-universal/install/module_install.py", line 29, in runModule
    sys.exit(0);
NameError: global name 'sys' is not defined
Traceback (most recent call last):
  File "/opt/monitiq-universal/install/modules/create_db_user-v0_0_0.py", line 132, in <module>
    inputVal = raw_input("");
KeyboardInterrupt

Answer 1

If you press Ctrl + C in a terminal then SIGINT is sent to all processes within the process group. See child process receives parent's SIGINT.

That is why you see the traceback from the child process despite try/except KeyboardInterrupt in the parent.

You could suppress the stderr output from the child process: stderr=DEVNULL. Or start it in a new process group: start_new_session=True:

import sys
from subprocess import call

try:
    call([sys.executable, 'child.py'], start_new_session=True)
except KeyboardInterrupt:
    print('Ctrl C')
else:
    print('no exception')

If you remove start_new_session=True in the above example then KeyboardInterrupt may be raised in the child too and you might get the traceback.

If subprocess.DEVNULL is not available; you could use DEVNULL = open(os.devnull, 'r+b', 0). If start_new_session parameter is not available; you could use preexec_fn=os.setsid on POSIX.

Answer 2

This code spawns a child process and hands signals like SIGINT, ... to them just like shells (bash, zsh, ...) do it.

This means KeyboardInterrupt is no longer seen by the Python process, but the child receives this and is killed correctly.

It works by running the process in a new foreground process group set by Python.

import os
import signal
import subprocess
import sys
import termios

def run_as_fg_process(*args, **kwargs):
    """
    the "correct" way of spawning a new subprocess:
    signals like C-c must only go
    to the child process, and not to this python.

    the args are the same as subprocess.Popen

    returns Popen().wait() value

    Some side-info about "how ctrl-c works":
    https://unix.stackexchange.com/a/149756/1321

    fun fact: this function took a whole night
              to be figured out.
    """

    old_pgrp = os.tcgetpgrp(sys.stdin.fileno())
    old_attr = termios.tcgetattr(sys.stdin.fileno())

    user_preexec_fn = kwargs.pop("preexec_fn", None)

    def new_pgid():
        if user_preexec_fn:
            user_preexec_fn()

        # set a new process group id
        os.setpgid(os.getpid(), os.getpid())

        # generally, the child process should stop itself
        # before exec so the parent can set its new pgid.
        # (setting pgid has to be done before the child execs).
        # however, Python 'guarantee' that `preexec_fn`
        # is run before `Popen` returns.
        # this is because `Popen` waits for the closure of
        # the error relay pipe '`errpipe_write`',
        # which happens at child's exec.
        # this is also the reason the child can't stop itself
        # in Python's `Popen`, since the `Popen` call would never
        # terminate then.
        # `os.kill(os.getpid(), signal.SIGSTOP)`

    try:
        # fork the child
        child = subprocess.Popen(*args, preexec_fn=new_pgid,
                                 **kwargs)

        # we can't set the process group id from the parent since the child
        # will already have exec'd. and we can't SIGSTOP it before exec,
        # see above.
        # `os.setpgid(child.pid, child.pid)`

        # set the child's process group as new foreground
        os.tcsetpgrp(sys.stdin.fileno(), child.pid)
        # revive the child,
        # because it may have been stopped due to SIGTTOU or
        # SIGTTIN when it tried using stdout/stdin
        # after setpgid was called, and before we made it
        # forward process by tcsetpgrp.
        os.kill(child.pid, signal.SIGCONT)

        # wait for the child to terminate
        ret = child.wait()

    finally:
        # we have to mask SIGTTOU because tcsetpgrp
        # raises SIGTTOU to all current background
        # process group members (i.e. us) when switching tty's pgrp
        # it we didn't do that, we'd get SIGSTOP'd
        hdlr = signal.signal(signal.SIGTTOU, signal.SIG_IGN)
        # make us tty's foreground again
        os.tcsetpgrp(sys.stdin.fileno(), old_pgrp)
        # now restore the handler
        signal.signal(signal.SIGTTOU, hdlr)
        # restore terminal attributes
        termios.tcsetattr(sys.stdin.fileno(), termios.TCSADRAIN, old_attr)

    return ret


# example:
run_as_fg_process(['openage', 'edit', '-f', 'random_map.rms'])