I have a very simple Python 3 script:
f1 = open('a.txt', 'r') print(f1.readlines()) f2 = open('b.txt', 'r') print(f2.readlines()) f3 = open('c.txt', 'r') print(f3.readlines()) f4 = open('d.txt', 'r') print(f4.readlines()) f1.close() f2.close() f3.close() f4.close()
But it always says:
IOError: [Errno 32] Broken pipe
I saw on the internet all the complicated ways to fix this, but I copied this code directly, so I think that there is something wrong with the code and not Python's SIGPIPE.
I am redirecting the output, so if the above script was named "open.py", then my command to run would be:
open.py | othercommand
Approach 2: We can handle this type of error by using the functionality of try/catch block which is already approved by the python manual and is advised to follow such procedure to handle the errors. Example: Python3.
Broken pipe may refer to: a character ¦, also known as a broken bar. a condition in programming (also known in POSIX as EPIPE error code and SIGPIPE signal), when a process requests an output to pipe or socket, which was closed by peer.
The problem is due to SIGPIPE handling. You can solve this problem using the following code:
from signal import signal, SIGPIPE, SIG_DFL signal(SIGPIPE,SIG_DFL)
Update: As pointed out in the comments, python docs already have a good answer.
See here for background on this solution. Better answer here.
To bring information from the many helpful answers together, with some additional information:
Standard Unix signal SIGPIPE
is sent to a process writing to a pipe when there's no process reading from the pipe (anymore).
head
by design stop reading prematurely from a pipe, once they've received enough data.head
[1]; e.g.: python -c 'for x in range(10000): print(x)' | head -n 1
By default - i.e., if the writing process does not explicitly trap SIGPIPE
- the writing process is simply terminated, and its exit code is set to 141
, which is calculated as 128
(to signal termination by signal in general) + 13
(SIGPIPE
's specific signal number).
However, by design Python itself traps SIGPIPE
and translates it into a Python BrokenPipeError
(Python 3) / IOError
(Python 2) instance with errno
value errno.EPIPE
.
If a Python script does not catch the exception, Python outputs error message BrokenPipeError: [Errno 32] Broken pipe
(Python 3, possibly twice, with Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>
sandwiched in between) / IOError: [Errno 32] Broken pipe
(Python 2) and terminates the script with exit code 1
[2] - this is the symptom Johannes (the OP) saw.
Windows considerations (SIGPIPE
is a Unix-only signal)
If your script needs to run directly on Windows too, you may have to conditionally bypass code that references SIGPIPE
, as shown in this answer.
If your script runs in a Unix subsystem on Windows, the SIGPIPE
signal may surface differently than on Unix - see this answer.
There are two ways to solve this problem:
Generally, it is not advisable to silence this exception, as it may signal a severe error condition, depending on your script's purpose, such as the receiving end of a network socket unexpectedly closing.
head
utility, for instance, you can abort quietly as follows, using signal.signal()
to install the platform's default signal handler (which behaves as described above), as also shown in akhan's answer (works in both Python 3 and 2):# ONLY SUITABLE FOR COMMAND-LINE UTILITIES # Install the default signal handler. from signal import signal, SIGPIPE, SIG_DFL signal(SIGPIPE, SIG_DFL) # Start printing many lines. # If this gets interrupted with SIGPIPE, # the script aborts quietly, and the process exit code is set to # 141 (128 + SIGPIPE) for x in range(10000): print(x)
import sys, os, errno try: # Start printing many lines. for x in range(10000): print(x) # IMPORTANT: Flush stdout here, to ensure that the # SIGPIPE-triggered exception can be caught. sys.stdout.flush() except IOError as e: # Note: Python 3 has the more specific BrokenPipeError, # but this way the code works in Python 2 too. if e.errno != errno.EPIPE: raise e # Unrelated error, re-throw. # Python flushes standard streams on exit; redirect remaining output # to devnull to avoid another BrokenPipeError at shutdown devnull = os.open(os.devnull, os.O_WRONLY) os.dup2(devnull, sys.stdout.fileno()) # ... perform other handling. # Note: You can't write to stdout here. # (print() and sys.stdout.write won't work) # However, sys.stderr.write() can be used. sys.stderr.write("SIGPIPE received, terminating.\n") # Finally, exit with an exit code of choice. sys.exit(141)
[1] Note that in bash
you will by default only see head
's exit code - which is 0
- reflected in $?
afterwards. Use echo ${PIPESTATUS[0]}
to see Python's exit code.
[2] Curiously, on macOS 10.15.7 (Catalina), with Python 3.9.2 (but not 2.x), I see exit code 120
, but the docs say 1
, and that's what I also see on Linux.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With