Python - How to call bash commands with pipe?

Question

I can run this normally on the command line in Linux:

$ tar c my_dir | md5sum

But when I try to call it with Python I get an error:

>>> subprocess.Popen(['tar','-c','my_dir','|','md5sum'],shell=True)
<subprocess.Popen object at 0x26c0550>
>>> tar: You must specify one of the `-Acdtrux' or `--test-label'  options
Try `tar --help' or `tar --usage' for more information.

Answer 1

You have to use subprocess.PIPE, also, to split the command, you should use shlex.split() to prevent strange behaviours in some cases:

from subprocess import Popen, PIPE
from shlex import split
p1 = Popen(split("tar -c mydir"), stdout=PIPE)
p2 = Popen(split("md5sum"), stdin=p1.stdout)

But to make an archive and generate its checksum, you should use Python built-in modules tarfile and hashlib instead of calling shell commands.

Answer 2

Ok, I'm not sure why but this seems to work:

subprocess.call("tar c my_dir | md5sum",shell=True)

Anyone know why the original code doesn't work?

Answer 3

What you actually want is to run a shell subprocess with the shell command as a parameter:

>>> subprocess.Popen(['sh', '-c', 'echo hi | md5sum'], stdout=subprocess.PIPE).communicate()
('764efa883dda1e11db47671c4a3bbd9e  -\n', None)