Why do I get subprocess resource warnings despite the process being dead?

Question

I've been trying to figure out how to spin up different subprocess instances and then killing them and then creating new ones. The parent python process never does, it just kills the subprocesses. I followed a lot of links on SO but I keep getting the following message once the parent python process ends:

F/Users/Lucifer/miniconda3/envs/rltp/lib/python3.6/subprocess.py:761: ResourceWarning: subprocess 40909 is still running ResourceWarning, source=self)

it seems interesting because I did ps but I get nothing:

  PID TTY           TIME CMD
 7070 ttys001    0:00.06 /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp Lucifer
 7072 ttys001    0:00.61 -bash
17723 ttys002    0:00.06 /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp Lucifer
17725 ttys002    0:00.06 -bash
38586 ttys002    0:00.16 sertop --no_init

I simply want to start a process:

self.serapi = subprocess.Popen(['sertop','--no_init'],
    stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE,
    preexec_fn=os.setsid,shell=True
    ,)

and kill it:

    os.killpg(os.getpgid(self.serapi.pid), signal.SIGTERM)

the above code is essentially copied from the top answer:

How to terminate a python subprocess launched with shell=True

but I am unsure why I get this message. Am I killing the child process successfully? I plan to start and kill many of them.

---

Note I don't know or need shell=True. I just copied that cuz thats how the answer/question I posted has it. I'd prefer to not have that parameter.

---

according to the answer I tried:

def kill(self):
    self.serapi.wait()
    #self.serapi.kill()
    self.serapi.terminate()
    #os.killpg(os.getpgid(self.serapi.pid), signal.SIGTERM)
    #self.serapi.wait()

and different permutations of the above but nothing really seemed to work. Any advice?

Answer 1

The ResourceWarning: subprocess N is still running warning comes from the __del__ method of the subprocess.Popen class.

If you look at the source for that method, you'll see this comment:

        # Not reading subprocess exit status creates a zombie process which
        # is only destroyed at the parent python process exit
        _warn("subprocess %s is still running" % self.pid,
              ResourceWarning, source=self)

The solution is to ensure you call wait() on the child process.

See the NOTES section of the man page for wait(2) for more background information.

In Python, the easiest way to handle this situation is to keep track of all the Popen objects you have created, and ensure that something calls wait() on them, either directly or indirectly.

Alternatively, you can install a SIGCHLD handler that ignores SIGCHLD events; then your child processes will disappear immediately, but you will now be unable to call wait() on them. See also How can I prevent zombie child processes? and How can I handle SIGCHLD?