subprocess and Type Str doesnt support the buffer API

Question

I have

cmd = subprocess.Popen('dir',shell=True,stdout=subprocess.PIPE)
for line in cmd.stdout:
  columns = line.split(' ')
  print (columns[3])

have error in line 3 Type Str doesnt support the buffer API.

What am i doing wrong i am on Python 3.3

Answer 1

You are reading binary data, not str, so you need to decode the output first. If you set the universal_newlines argument to True, then stdout is automatically decoded using the result of the locale.getpreferredencoding() method (same as for opening text files):

cmd = subprocess.Popen(
    'dir', shell=True, stdout=subprocess.PIPE, universal_newlines=True)
for line in cmd.stdout:
    columns = line.decode().split()
    if columns:
        print(columns[-1])

If you use Python 3.6 or newer, you can use an explicit encoding argument for to the Popen() call to specify a different codec to use, like, for example, UTF-8:

cmd = subprocess.Popen(
    'dir', shell=True, stdout=subprocess.PIPE, encoding='utf8')
for line in cmd.stdout:
    columns = line.split()
    if columns:
        print(columns[-1])

If you need to use a different codec in Python 3.5 or earlier, don't use universal_newlines, just decode text from bytes explicitly.

You were trying to split a bytes value using a str argument:

>>> b'one two'.split(' ')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Type str doesn't support the buffer API

By decoding you avoid that problem, and your print() call will not have to prepend the output with b'..' either.

However, you probably just want to use the os module instead to get filesystem information:

import os

for filename in os.listdir('.'):
    print(filename)

Answer 2

A simpler solution of the first part of Martijn Pieters's answer is to pass the universal_newlines=True argument to the Popen call.

I would even simplify this to:

output = subprocess.check_output('dir', universal_newlines=True)
columns = output.split()
print(columns)

NOTE: If file or directory names contain spaces, use os.listdir('.') as suggested in Martijn Pieters's answer or something like the following:

output = subprocess.check_output('dir', universal_newlines=True)
columns = []
for e in output.split():
    if len(columns) > 0 and columns[-1].endswith('\\'):
        columns[-1] = columns[-1][:-1] + " " + e
    else:
        columns.append(e)
print(columns)