Default encoding of Python 3's open() under Windows. sys.getdefaultencoding() setting not respected

Question

I've been pulling my hair out over this issue for several hours now.

I have a message file I want to generate using django's makemessages command, which works just fine in a Linux environment which actually handles locale settings in a sane way. However, when I try the same thing under Windows, every time python tries to open a file, it assumes it is encoded in cp932 (SHIFT-JIS), which causes all sorts of havoc.

Manually adding encoding='utf-8' to every open call works, but that's hardly a good way of fixing the problem. Is there any way to force open to use a specific default encoding?

• sys.getdefaultencoding() returns 'utf-8', for some arcane reason this setting is not respected
• PYTHONIOENCODING and PYTHONENCODING are both set to 'utf-8'
• My code page is set to cp65001

This is my python version string:

Python 3.3.2 (v3.3.2:d047928ae3f6, May 16 2013, 00:06:53) [MSC v.1600 64 bit (AMD64)] on win32

EDIT: I've noticed that locale.getpreferredencoding() returns cp932, so I guess finding a Windows locale with utf-8 as its default would do the trick. Does such a thing even exist?

Answer 1

Try this

import locale
locale.setlocale(locale.LC_ALL, 'en_US.utf-8')