I need a execute a command that produces a lot of output and takes a lot of time to execute (> 30 minutes). I was thinking of using subprocess.Popen to do it. I need to capture the output of the command, so I pass PIPE to stdout and stderr.
A deadlock problem when using Popen.wait() is well documented on a lot of forums, so Popen.communicate() is the proposed way of avoiding the deadlock. The problem with that solution is that communicate() blocks until the command is completed. I need to print everything that arrives on stdout while the command is executed. If there is no output after 20 minutes, the script execution will be killed.
Here are some constraints that I need to respect:
Is there a way to do it?
communicate() writes input (there is no input in this case so it just closes subprocess' stdin to indicate to the subprocess that there is no more input), reads all output, and waits for the subprocess to exit.
Popen is more general than subprocess. call . Popen doesn't block, allowing you to interact with the process while it's running, or continue with other things in your Python program. The call to Popen returns a Popen object.
check_call will raise an exception if the command it's running exits with anything other than 0 as its status.
The subprocess module provides a function named call. This function allows you to call another program, wait for the command to complete and then return the return code.
stdin
to a NUL
deviceimport os from subprocess import PIPE, STDOUT, Popen lines = [] p = Popen(cmd, bufsize=1, stdin=open(os.devnull), stdout=PIPE, stderr=STDOUT) for line in iter(p.stdout.readline, ''): print line, # print to stdout immediately lines.append(line) # capture for later p.stdout.close() p.wait()
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With