Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Explain example pipeline from Python subprocess module

Section 17.1.4.2: Replacing shell pipeline of the python subprocess module says to replace

output=`dmesg | grep hda`

with

p1 = Popen(["dmesg"], stdout=PIPE)
p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
p1.stdout.close()  # Allow p1 to receive a SIGPIPE if p2 exits.
output = p2.communicate()[0]

The comment to the third line explains why the close function is being called, but not why it makes sense. It doesn't, to me. Will not closing p1.stdout before the communicate method is called prevent any output from being sent through the pipe? (Obviously it won't, I've tried to run the code and it runs fine). Why is it necessary to call close to make p1 receive SIGPIPE? What kind of close is it that doesn't close? What, exactly, is it closing?

Please consider this an academic question, I'm not trying to accomplish anything except understanding these things better.

like image 759
Lauritz V. Thaulow Avatar asked May 18 '11 14:05

Lauritz V. Thaulow


People also ask

What is a pipe in Python subprocess?

We will explain the process with the help of examples, so let us go further to see the examples but, first, let us see what a pipe is to a subprocess in python. The PIPE in python is used to send or receive data from a program that is running as a subprocess in python.

What is the subprocess module in Python?

The subprocess module enables you to start new applications from your Python program. How cool is that? You can start a process in Python using the Popen function call. The program below starts the unix program ‘cat’ and the second parameter is the argument. This is equivalent to ‘cat test.py’.

How to run an external command in subprocess in Python?

Call () function in Subprocess Python. This function can be used to run an external command without disturbing it, wait till the execution is completed, and then return the output. Its syntax is. subprocess.call(args, *, stdin=None, stdout=None, stderr=None, shell=False, timeout=None)

How to use subprocess in Python to check Internet connectivity?

Use subprocess.call or subprocess.run to run the command described by args. Wait for command to complete, then return the returncode attribute. Use subprocess.Popen with wait () to wait for the command to complete Here we use subprocess.call to check internet connectivity and then print " Something "


1 Answers

You are closing p1.stdout in the parent process, thus leaving dmesg as the only process with that file descriptor open. If you didn't do this, even when dmesg closed its stdout, you would still have it open, and a SIGPIPE would not be generated. (The OS basically keeps a reference count, and generates SIGPIPE when it hits zero. If you don't close the file, you prevent it from ever reaching zero.)

like image 55
andrewdski Avatar answered Sep 20 '22 07:09

andrewdski