Python - __eq__ method not being called

Question

I have a set of objects, and am interested in getting a specific object from the set. After some research, I decided to use the solution provided here: http://code.activestate.com/recipes/499299/

The problem is that it doesn't appear to be working.

I have two classes defined as such:

class Foo(object):
    def __init__(self, a, b, c):
        self.a = a
        self.b = b
        self.c = c
    def __key(self):
        return (self.a, self.b, self.c)
    def __eq__(self, other):
        return self.__key() == other.__key()
    def __hash__(self):
        return hash(self.__key())

class Bar(Foo):
    def __init__(self, a, b, c, d, e):
        self.a = a
        self.b = b
        self.c = c
        self.d = d
        self.e = e

Note: equality of these two classes should only be defined on the attributes a, b, c.

The wrapper _CaptureEq in http://code.activestate.com/recipes/499299/ also defines its own __eq__ method. The problem is that this method never gets called (I think). Consider,

bar_1 = Bar(1,2,3,4,5)
bar_2 = Bar(1,2,3,10,11)
summary = set((bar_1,))
assert(bar_1 == bar_2)
bar_equiv = get_equivalent(summary, bar_2)

bar_equiv.d should equal 4 and likewise bar_equiv .e should equal 5, but they are not. Like I mentioned, it looks like the __CaptureEq __eq__ method does not get called when the statement bar_2 in summary is executed.

Is there some reason why the __CaptureEq __eq__ method is not being called? Hopefully this is not too obscure of a question.

Answer 1

Brandon's answer is informative, but incorrect. There are actually two problems, one with the recipe relying on _CaptureEq being written as an old-style class (so it won't work properly if you try it on Python 3 with a hash-based container), and one with your own Foo.__eq__ definition claiming definitively that the two objects are not equal when it should be saying "I don't know, ask the other object if we're equal".

The recipe problem is trivial to fix: just define __hash__ on the comparison wrapper class:

class _CaptureEq:
    'Object wrapper that remembers "other" for successful equality tests.'
    def __init__(self, obj):
        self.obj = obj
        self.match = obj
    # If running on Python 3, this will be a new-style class, and
    # new-style classes must delegate hash explicitly in order to populate
    # the underlying special method slot correctly.
    # On Python 2, it will be an old-style class, so the explicit delegation
    # isn't needed (__getattr__ will cover it), but it also won't do any harm.
    def __hash__(self):
        return hash(self.obj)
    def __eq__(self, other):
        result = (self.obj == other)
        if result:
            self.match = other
        return result
    def __getattr__(self, name):  # support anything else needed by __contains__
        return getattr(self.obj, name)

The problem with your own __eq__ definition is also easy to fix: return NotImplemented when appropriate so you aren't claiming to provide a definitive answer for comparisons with unknown objects:

class Foo(object):
    def __init__(self, a, b, c):
        self.a = a
        self.b = b
        self.c = c
    def __key(self):
        return (self.a, self.b, self.c)
    def __eq__(self, other):
        if not isinstance(other, Foo):
            # Don't recognise "other", so let *it* decide if we're equal
            return NotImplemented
        return self.__key() == other.__key()
    def __hash__(self):
        return hash(self.__key())

With those two fixes, you will find that Raymond's get_equivalent recipe works exactly as it should:

>>> from capture_eq import *
>>> bar_1 = Bar(1,2,3,4,5)
>>> bar_2 = Bar(1,2,3,10,11)
>>> summary = set((bar_1,))
>>> assert(bar_1 == bar_2)
>>> bar_equiv = get_equivalent(summary, bar_2)
>>> bar_equiv.d
4
>>> bar_equiv.e
5

Update: Clarified that the explicit __hash__ override is only needed in order to correctly handle the Python 3 case.

Answer 2

The problem is that the set compares two objects the “wrong way around” for this pattern to intercept the call to __eq__(). The recipe from 2006 evidently was written against containers that, when asked if x was present, went through the candidate y values already present in the container doing:

x == y

comparisons, in which case an __eq__() on x could do special actions during the search. But the set object is doing the comparison the other way around:

y == x

for each y in the set. Therefore this pattern might simply not be usable in this form when your data type is a set. You can confirm this by instrumenting Foo.__eq__() like this:

def __eq__(self, other):
    print '__eq__: I am', self.d, self.e, 'and he is', other.d, other.e
    return self.__key() == other.__key()

You will then see a message like:

__eq__: I am 4 5 and he is 10 11

confirming that the equality comparison is posing the equality question to the object already in the set — which is, alas, not the object wrapped with Hettinger's _CaptureEq object.

Update:

And I forgot to suggest a way forward: have you thought about using a dictionary? Since you have an idea here of a key that is a subset of the data inside the object, you might find that splitting out the idea of the key from the idea of the object itself might alleviate the need to attempt this kind of convoluted object interception. Just write a new function that, given an object and your dictionary, computes the key and looks in the dictionary and returns the object already in the dictionary if the key is present else inserts the new object at the key.

Update 2: well, look at that — Nick's answer uses a NotImplemented in one direction to force the set to do the comparison in the other direction. Give the guy a few +1's!