Unicode elementwise string comparison in numpy

Question

I had a question about equality comparison with numpy and arrays of strings. Say I define the following array:

x = np.array(['yes', 'no', 'maybe'])

Then I can test for equality with other strings and it does element wise comparison with the single string (following, I think, the broadcasting rules here: http://docs.scipy.org/doc/numpy-1.10.1/user/basics.broadcasting.html ?):

'yes' == x
#op : array([ True, False, False], dtype=bool)

x == 'yes'
#op : array([ True, False, False], dtype=bool)

However, if I compare with unicode strings I get different behaviour with element wise comparison only happening if I compare the array to the string and only a single comparison being made if I compare the string to the array.

x == u'yes'
#op : array([ True, False, False], dtype=bool)

u'yes' == x
#op : False

I can't find details of this behaviour in the numpy docs and was hoping someone could explain or point me to details of why comparison with unicode strings behaves differently?

Answer 1

The relevant piece of information is this part of the Python's coercion rules:

For objects xand y, first x.__op__(y) is tried. If this is not implemented or returns NotImplemented, y.__rop__(x) is tried.

Using your numpy array x, when the left-hand side is a str ('yes' == x):

• 'yes'.__eq__(x) returns NotImplemented and
• therefore resolves to x.__eq__('yes') – resulting in numpy's element-wise comparison.

However, when the left-hand side is a unicode (u'yes' == x):

• u'yes'.__eq__(x) simply returns False.

The reason for the different __eq__ behaviours is that str.__eq__() simply returns NotImplemented if its argument is not a str type, whereas unicode.__eq__() first tries to convert its argument to a unicode, and only returns NotImplemented if that conversion fails. In this case, the numpy array is convertible to a unicode: u'yes' == x is essentially u'yes' == unicode(x).