Where does a python descriptor's state go?

Question

Background

I'm trying to figure out Python's descriptors by reading Lutz's Learning Python's section on the topic in which he says: "Like properties, descriptors are designed to handle specific attributes... Unlike properties, descriptors have their own state..."

Throughout the chapter he shows examples in which the managed attribute is actually stashed on the containing/wrapping object, as in:

def __set__(self, instance, value):
    instance._name = value.lower()

I understand these examples and they seem to be common in write ups on the topic. That said, their benefit over properties isn't obvious to me and they seem to fall short of the internal state promised in the above quote.

At the end of the chapter he shows an example that is closer to what I pictured after reading "have their own state", as in:

def __set__(self, instance, value):
    self.name = value.lower()

The example runs but does not do what I'd expect it to do. As the example is a bit long I've put it on Pastebin and added a last line that shows the unexpected behavior (Bob's name is now Sue). Here's a shorter demo snippet:

class Wrapper(object):
    class ExampleDescriptor(object):
        def __get__(self, instance, owner):
            print "get %s" % self.state
            return self.state

        def __set__(self, instance, value):
            print "set %s" % value
            self.state = value
    ex = ExampleDescriptor()


w1 = Wrapper()
w1.ex = 1
print w1.ex
w2 = Wrapper()
print w2.ex
w2.ex = 2
print  w1.ex
print w1.ex is w2.ex

The output of which is:

set 1 
get 1
1
get 1
1
set 2
get 2
2
get 2
get 2
True

None of this execution comes as a surprise after looking at the code carefully. The validation logic in the descriptor is making a de facto singleton out of this attribute on the wrapper class; however, it's hard to imagine this shared state was Lutz's intention, or the intention in this widely linked tutorial on the topic.

Question

Is it possible to make a descriptor that has internal state that is unique to the wrapping object without stashing that state on the wrapping object instances (as in the first snippet)? Is it possible to modify the CardHolder class from the linked example such that Bob does not end up as Sue?

Answer 1

"Like properties, descriptors are designed to handle specific attributes... Unlike properties, descriptors have their own state..."

I am not sure what point Lutz is trying to make as properties are, in fact, descriptors themselves.

But, even though descriptors do have their own state, it's not widely useful as, as you have discovered, you only get one descriptor object per class attribute instead of one per instance. This is why the instance is passed in, so that instance-unique values can be saved/accessed.

To prove the point that it is one descriptor object per attribute, you can try this slightly modified code from one of your links:

class RevealAccess(object):
    """A data descriptor that sets and returns values
       normally and prints a message logging their access.
    """
    def __init__(self, initval=None, name='var'):
        self.val = initval
        self.name = name
    def __get__(self, obj, objtype):
        print 'Retrieving', self.name
        return self.val
    def __set__(self, obj, val):
        print 'Updating' , self.name
        self.val = val

class MyClass(object):
    x = RevealAccess(10, 'var "x"')
    y = RevealAccess(5, 'var "y"')

m = MyClass()
m.x
m.x = 20
m.x
m.y

What you should see:

Retrieving var "x"
Updating var "x"
Retrieving var "x"
Retrieving var "y"

To answer your question: Yes. But it's a pain.

class Stored(object):
    """A data descriptor that stores instance values in itself.
    """
    instances = dict()
    def __init__(self, val):
        self.instances[self, None] = val
    def __get__(self, obj, objtype):
        return self.instances[self, obj]
    def __set__(self, obj, val):
        self.instances[self, obj] = val

class MyClass(object):
    x = Stored(3)
    y = Stored(9)

print(MyClass.x)
print(MyClass.y)
m = MyClass()
m.x = 42
print(m.x)
m.y = 19
print(m.y)
print(m.x)