Python UTF-8 comparison

Question

a = {"a":"çö"}
b = "çö"
a['a']
>>> '\xc3\xa7\xc3\xb6'

b.decode('utf-8') == a['a']
>>> False

What is going in there?

edit= I'm sorry, it was my mistake. It is still False. I'm using Python 2.6 on Ubuntu 10.04.

Answer 1

Possible solutions

Either write like this:

a = {"a": u"çö"}
b = "çö"
b.decode('utf-8') == a['a']

Or like this (you may also skip the .decode('utf-8') on both sides):

a = {"a": "çö"}
b = "çö"
b.decode('utf-8') == a['a'].decode('utf-8')

Or like this (my recommendation):

a = {"a": u"çö"}
b = u"çö"
b == a['a']

Explanation

Updated based on Tim's comment. In your original code, b.decode('utf-8') == u'çö' and a['a'] == 'çö', so you're actually making the following comparison:

u'çö' == 'çö'

One of the objects is of type unicode, the other is of type str, so in order to execute the comparison, the str is converted to unicode and then the two unicode objects are compared. It works fine in the case of purely ASCII strings, for example: u'a' == 'a', since unicode('a') == u'a'.

However, it fails in case of u'çö' == 'çö', since unicode('çö') returns the following error: UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 0: ordinal not in range(128), and therefore the whole comparison returns False and issues the following warning: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal.