We are having some problems with the dreaded "too many open files" on our Ubuntu Linux machine rrunning a python Twisted application. In many places in our program, we are using subprocess Popen, something like this:
Popen('ifconfig ' + iface, shell=True, stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
output = process.stdout.read()
while in other places we use subprocess communicate:
process = subprocess.Popen(['/usr/bin/env', 'python', self._get_script_path(script_name)],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
close_fds=True)
out, err = process.communicate(data)
What exactly do I need to do in both cases in order to close any open file descriptors? Python documentation is not clear on this. From what I gather (which could be wrong) both communicate() and wait() will indeed clean up any open fds on their own. But what about Popen? Do I need to close stdin, stdout, and stderr explicitly after calling Popen if I don't call communicate or wait?
According to this source for the subprocess module (link) if you call communicate
you should not need to close the stdout
and stderr
pipes.
Otherwise I would try:
process.stdout.close()
process.stderr.close()
after you are done using the process
object.
For instance, when you call .read()
directly:
output = process.stdout.read()
process.stdout.close()
Look in the above module source for how communicate()
is defined and you'll see that it closes each pipe after it reads from it, so that is what you should also do.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With