subprocess.check_output with grep command fails when grep finds no matches

Question

I'm trying to search a text file and retrieve lines containing a specific set of words. This is the code I'm using:

tyrs = subprocess.check_output('grep "^A" %s | grep TYR' % pocket_location, shell = True).split('\n')

This works fine when the file contains at least one line that grep identifies. But when grep doesn't identify any lines, grep returns exit status 1 and I get the following error:

Traceback (most recent call last):
  File "../../Python_scripts/cbs_wrapper2.py", line 324, in <module>
    tyrs = subprocess.check_output('grep "^ATOM" %s | grep TYR' % pocket_location, shell = True).split('\n')
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 544, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command 'grep "^ATOM" cbsPrediction_files/1u9c_clean/1u9c_clean_fpocket_out/pockets/pocket0_atm.pdb | grep TYR' returned non-zero exit status 1

How can I avoid this issue? I just want subprocess.check_output to return an empty string if grep doesn't find anything.

Thanks

Answer 1

I just want subprocess.check_output to return an empty string if grep doesn't find anything.

Well, too bad. grep considers no matches to be failure, and the whole point of the check in check_output is to check for failure, so you're explicitly asking to do things this way. Here are the relevant docs:

If the return code was non-zero it raises a CalledProcessError. The CalledProcessError object will have the return code in the returncode attribute and any output in the output attribute.

And for grep:

The following exit values shall be returned:
  0 One or more lines were selected.
  1 No lines were selected.
  >1 An error occurred.

So, if you want to treat "no lines" as success, but actual errors as errors, you have to handle that 1 value differently than other non-zero values. And check_output has no idea that you want to do that.

So, either you have to handle the CalledProcessError, or you have to do your own checking. In other words, either this:

try:
    tyrs = subprocess.check_output('grep "^A" %s | grep TYR' % pocket_location, shell = True).split('\n')
except subprocess.CalledProcessError as e:
    if e.returncode > 1:
        raise
    tyrs = []

… or this:

p = subprocess.Popen('grep "^A" %s | grep TYR' % pocket_location, shell=True,
                     stdout=subprocess.PIPE)
output, _ = p.communicate()
if p.returncode == 1: # no matches found 
    tyrs = []
elif p.returncode == 0: # matches found
    tyrs = output.split('\n')
else:
    # error, do something with it

Answer 2

tyrs = subprocess.check_output('grep "^A" %s | grep TYR || true' % pocket_location, shell = True).split('\n')