Python: UnicodeEncodeError: 'latin-1' codec can't encode characters in position

Question

I am getting this error:

File "run.py", line 37, in <module>
 print str1
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 24-29: ordinal not in range(256)

When trying to simply print some Japanese text. Actually it seems the string looks like this:

\u5149\u66dc\u65e5\u3067\u30e9\u30c6 \u30d4\u30af\u30b7\u30fc\u4e71\u7372\u884c\u304d\u307e\u3059 \u5e0c\u671b\u8005\u52df\u96c6\u4e2d\u3067\u3059\uff3e\uff3e

Which comes in from a JSON file. How can I print this?

Code:

url = "http://www.blah.com/json"
try:
  result = simplejson.load(urllib2.urlopen(url))
except IOError:
  print "Cannot open URL"
  data = "error"

for msg in result["msg"]:
  str1 = msg["character"] + " : " + msg["message"]
  print str1

repr(str1) is

u'Anys : \u5149\u66dc\u65e5\u3067\u30e9\u30c6 \u30d4\u30af\u30b7\u30fc\u4e71\u7372\u884c\u304d\u307e\u3059 \u5e0c\u671b\u8005\u52df\u96c6\u4e2d\u3067\u3059\uff3e\uff3e'

print(sys.stdout.encoding) is

ISO-8859-1

Answer 1

The error that you see is because you terminal use latin-1 as encoding, as a side note you can check the encoding of your terminal (assuming that it's your stdout) by doing in your shell:

$ python -c "import sys; print sys.stdout.encoding"

And now for printing in UTF-8 you should encode your string to utf-8 manually like this:

s = u"\u5149\u66dc\u65e5\u3067\u30e9\u30c6 \u30d4\u30af\u30b7\u30fc\u4e71\u7372\u884c\u304d\u307e\u3059 \u5e0c\u671b\u8005\u52df\u96c6\u4e2d\u3067\u3059\uff3e\uff3e"
print s.encode('utf-8')
#Output: 光曜日でラテ ピクシー乱獲行きます 希望者募集中です＾＾