OSError: raw write() returned invalid length when using print() in python

Question

I'm using python tensorflow to train a model to recognise images in python. But I'm getting the below error when trying to execute train.py from github

Traceback (most recent call last):
File "train.py", line 1023, in <module>
tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
File "C:\Users\sande\Anaconda3\envs\tensorflow\lib\site-
packages\tensorflow\python\platform\app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "train.py", line 766, in main
bottleneck_tensor)
File "train.py", line 393, in cache_bottlenecks
jpeg_data_tensor, bottleneck_tensor)
File "train.py", line 341, in get_or_create_bottleneck
bottleneck_tensor)
File "train.py", line 290, in create_bottleneck_file
print('Creating bottleneck at ' + bottleneck_path)
OSError: raw write() returned invalid length 112 (should have been between 0 
and 56)

Below is the code for create_bottleneck_file()

def create_bottleneck_file(bottleneck_path, image_lists, label_name, index,
                       image_dir, category, sess, jpeg_data_tensor,
                       bottleneck_tensor):
"""Create a single bottleneck file."""
print('Creating bottleneck at ' + bottleneck_path)
image_path = get_image_path(image_lists, label_name, index,
                          image_dir, category)
if not gfile.Exists(image_path):
    tf.logging.fatal('File does not exist %s', image_path)
image_data = gfile.FastGFile(image_path, 'rb').read()
try:
    bottleneck_values = run_bottleneck_on_image(
        sess, image_data, jpeg_data_tensor, bottleneck_tensor)
except:
    raise RuntimeError('Error during processing file %s' % image_path)

bottleneck_string = ','.join(str(x) for x in bottleneck_values)
with open(bottleneck_path, 'w') as bottleneck_file:
    bottleneck_file.write(bottleneck_string)

I tried reducing the file names so that bottleneck_path will be a small value but that did not work. I tried to search online for this error but did not find anything useful. Please let me know if you have a fix to this issue

Answer 1

If you're unable to migrate to 3.6 or from Windows like me, install the win_unicode_console package, import it and add this line at the beggining of your script to enable it:

win_unicode_console.enable()

This issue appears to be generally unique to pre-3.6 Python as the code responsible for handling text output was rewritten for this latest version. This also means that we will most likely not see a fix coming for this issue.

Source: https://bugs.python.org/issue32245

Answer 2

I think this is a bug on the stdout/stderr streams introduced by the November's creators update, it happens in both powershell.exe and cmd.exe

It seems to only happen on Windows 10 Version 1709 (OS Build 16299.64). My guess is that it is unicode realted (output size is twice the length expected)

A (very) quick and dirty fix is to only output ASCII on your console :

mystring.encode("utf-8").decode("ascii")

https://github.com/Microsoft/vscode/issues/39149#issuecomment-347260954

Answer 3

Adding more to @AMSAntiago answer. You could run the win_unicode_console.enable(). But instead of using it on every file, you could run it on every Python invocation (docs). That works for me.