Can't terminate a sudo process created with python, in Ubuntu 15.10

Question

I just updated to Ubuntu 15.10 and suddenly in Python 2.7 I am not able to terminate a process I created when being root. For example, this doesn't terminate tcpdump:

import subprocess, shlex, time
tcpdump_command = "sudo tcpdump -w example.pcap -i eth0 -n icmp"
tcpdump_process = subprocess.Popen(
                                shlex.split(tcpdump_command),
                                stdout=subprocess.PIPE,
                                stderr=subprocess.PIPE)
time.sleep(1)
tcpdump_process.terminate()
tcpdump_out, tcpdump_err = tcpdump_process.communicate()

What happened? It works on previous versions.

Answer 1

TL;DR: sudo does not forward signals sent by a process in the command's process group since 28 May 2014 commit released in sudo 1.8.11 -- the python process (sudo's parent) and the tcpdump process (grandchild) are in the same process group by default and therefore sudo does not forward SIGTERM signal sent by .terminate() to the tcpdump process.

---

It shows the same behaviour when running that code while being the root user and while being a regular user + sudo

Running as a regular user raises OSError: [Errno 1] Operation not permitted exception on .terminate() (as expected).

Running as root reproduces the issue: sudo and tcpdump processes are not killed on .terminate() and the code is stuck on .communicate() on Ubuntu 15.10.

The same code kills both processes on Ubuntu 12.04.

tcpdump_process name is misleading because the variable refers to the sudo process (the child process), not tcpdump (grandchild):

python
└─ sudo tcpdump -w example.pcap -i eth0 -n icmp
   └─ tcpdump -w example.pcap -i eth0 -n icmp          

As @Mr.E pointed out in the comments, you don't need sudo here: you're root already (though you shouldn't be -- you can sniff the network without root). If you drop sudo; .terminate() works.

In general, .terminate() does not kill the whole process tree recursively and therefore it is expected that a grandchild process survives. Though sudo is a special case, from sudo(8) man page:

When the command is run as a child of the sudo process, sudo will relay signals it receives to the command.^{emphasis is mine}

i.e., sudo should relay SIGTERM to tcpdump and tcpdump should stop capturing packets on SIGTERM, from tcpdump(8) man page:

Tcpdump will, ..., continue capturing packets until it is interrupted by a SIGINT signal (generated, for example, by typing your interrupt character, typically control-C) or a SIGTERM signal (typically generated with the kill(1) command);

i.e., the expected behavior is: tcpdump_process.terminate() sends SIGTERM to sudo which relays the signal to tcpdump which should stop capturing and both processes exit and .communicate() returns tcpdump's stderr output to the python script.

Note: in principle the command may be run without creating a child process, from the same sudo(8) man page:

As a special case, if the policy plugin does not define a close function and no pty is required, sudo will execute the command directly instead of calling fork(2) first

and therefore .terminate() may send SIGTERM to the tcpdump process directly -- though it is not the explanation: sudo tcpdump creates two processes on both Ubuntu 12.04 and 15.10 in my tests.

If I run sudo tcpdump -w example.pcap -i eth0 -n icmp in the shell then kill -SIGTERM terminates both processes. It does not look like Python issue (Python 2.7.3 (used on Ubuntu 12.04) behaves the same on Ubuntu 15.10. Python 3 also fails here).

It is related to process groups (job control): passing preexec_fn=os.setpgrp to subprocess.Popen() so that sudo will be in a new process group (job) where it is the leader as in the shell makes tcpdump_process.terminate() work in this case.

What happened? It works on previous versions.

The explanation is in the sudo's source code:

Do not forward signals sent by a process in the command's process group, do not forward it as we don't want the child to indirectly kill itself. For example, this can happen with some versions of reboot that call kill(-1, SIGTERM) to kill all other processes.^{emphasis is mine}

preexec_fn=os.setpgrp changes sudo's process group. sudo's descendants such as tcpdump process inherit the group. python and tcpdump are no longer in the same process group and therefore the signal sent by .terminate() is relayed by sudo to tcpdump and it exits.

Ubuntu 15.04 uses Sudo version 1.8.9p5 where the code from the question works as is.

Ubuntu 15.10 uses Sudo version 1.8.12 that contains the commit.

sudo(8) man page in wily (15.10) still talks only about the child process itself -- no mention of the process group:

As a special case, sudo will not relay signals that were sent by the command it is running.

It should be instead:

As a special case, sudo will not relay signals that were sent by a process in the process group of the command it is running.

You could open a documentation issue on Ubuntu's bug tracker and/or on the upstream bug tracker.