Change encoding of stdin / stdout at runtime in Python 3

Question

In Python 3, stdin and stdout are TextIOWrappers that have an encoding and hence spit out normal strings (not bytes).

I can change the encoding that is being used with an environment variable PYTHONIOENCODING. Is there also a way to change this in my script itself?

Answer 1

Actually TextIOWrapper does return bytes. It takes a Unicode string and returns a byte string in a particular encoding. To change sys.stdout to use a particular encoding in a script, here's an example:

Python 3.2.3 (default, Apr 11 2012, 07:15:24) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> print('\u5000')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\dev\python32\lib\encodings\cp437.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_map)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u5000' in position 0: character maps to <undefined>>>> import io
>>> import io
>>> import sys
>>> sys.stdout = io.TextIOWrapper(sys.stdout.buffer,encoding='utf8')
>>> print('\u5000')
倀

(my terminal isn't UTF-8)

sys.stdout.buffer accesses the raw byte stream. You can also use the following to write to stdout in a particular encoding:

sys.stdout.buffer.write('\u5000'.encode('utf8'))

Answer 2

Since Python 3.7 TextIOWrapper has a reconfigure() method that can change stream settings, including the encoding:

sys.stdout.reconfigure(encoding='utf-8')

One caveat: You can only change the encoding of sys.stdin if you haven't started reading from it.