The difference between bash and python pipes

Question

I have the following three python scripts:

parent1.py

import subprocess, os, sys

relpath = os.path.dirname(sys.argv[0])
path = os.path.abspath(relpath)
child = subprocess.Popen([os.path.join(path, 'child.lisp')], stdout = subprocess.PIPE)
sys.stdin = child.stdout
inp = sys.stdin.read()
print(inp.decode())

parent2.py:

import sys
inp = sys.stdin
print(inp)

child.py:

print("This text was created in child.py")

If i call parent1.py with:

python3 parent1.py

it gives me like expected the following output:

This text was created with child.py

if i call parent2.py with:

python3 child.py | python3 parent2.py

i get the same output. But in the first example i get the output of child.py as bytes and in the second i get it directly as a string. Why is this? Is it just a difference between python and bash pipes or is there something i could do otherwise to avoid this?

Answer 1

When python opens stdin and stdout, it detects what encoding to use and uses text I/O to give you unicode strings.

But subprocess does not (and can not) detect the encoding of the subprocess you start, so it'll return bytes. You can use a io.TextIOWrapper() instance to wrap the child.stdout pipe to provide unicode data:

sys.stdin = io.TextIOWrapper(child.stdout, encoding='utf8')