Dynamically adding methods with or without metaclass

Question

Update - 2012/12/13

Just to clarify - I'm not so much interested in ways how to add methods to classes - as you can see below in my question and in people's answers, there is more than one way to do that (tongue in cheek and hat tip to my Perl self).

The thing I am interested in is learning what's the fundamental difference of adding methods to classes using different approaches and the big question really is why do I need to use metaclasses for. For example A Primer on Python Metaclass Programming states that:

Perhaps the most common use of metaclasses [...]: adding, deleting, renaming, or substituting methods for those defined in the produced class.

And since there's more ways to do that, I'm puzzled and looking for explanation.

Thanks!

Original - 2012/12/12

I need to dynamically add methods to a class (and newly generated classes based on that class). I came up with two approaches, one involving metaclass, the other one doing without one. I cannot see any difference in what the two approaches give me other than the fact the latter doesn't involve the "black magic" metaclass ;)

Approach #1 with metaclass:

class Meta(type):
        def __init__(cls, *args, **kwargs):
                setattr(cls, "foo", lambda self: "foo@%s(class %s)" % (self,
                        cls.__name__))

class Y(object):
        __metaclass__ = Meta

y = Y()
y.foo() # Gives 'foo@<__main__.Y object at 0x10e4afd10>(class Y)'

Approach #2 without metaclass:

class Z(object):
        def __init__(self):
                setattr(self.__class__, "foo",
                        lambda self: "foo@%s(class %s)" %
                        (self, self.__class__.__name__))

z = Z()
z.foo() # Gives 'foo@<__main__.Z object at 0x10c865dd0>(class Z)'

As far as I can tell, both approaches give me the same results and "expressivity". Even when I try to create a new classes using type("NewClassY", (Y, ), {}) or type("NewClassZ", (Z, ), {}) I get the same expected results which don't differ between the two approaches.

So, I'm wondering if there really is any "underlying" difference in the approaches, or if there's anything which will "bite" me later if I used either #1 or #2 or if it's just a syntactic sugar?

PS: Yes, I did read the other threads here talking about metaclasses in Python and pythonic data model.

Answer 1

The obvious reason to use metaclasses is because they really provide metadata about the class as soon as the class is known, unrelated to the presence of objects or not. Trivial, right? Well, let us show some commands I executed on your original Z and Y classes to see what this means:

In [287]: hasattr(Y,'foo')
Out[287]: True

In [288]: hasattr(Z,'foo')
Out[288]: False

In [289]: Y.__dict__
Out[289]:
<dictproxy {..., 'foo': <function __main__.<lambda>>}>

In [290]: Z.__dict__
Out[290]:
<dictproxy {...}>

In [291]: z= Z()

In [292]: hasattr(Z,'foo')
Out[292]: True

In [293]: Z.__dict__
Out[293]:
<dictproxy {..., 'foo': <function __main__.<lambda>>}>

In [294]: y = Y()

In [295]: hasattr(Y,'foo')
Out[295]: True

In [296]: Y.__dict__
Out[296]:
<dictproxy {..., 'foo': <function __main__.<lambda>>}>

As you can see the second version actually alters the class Z significantly after it's been declared, something you often like to avoid. It is certainly not an uncommon operation for python to do operations on 'types' (class objects) and you likely want them to be as consistent as possible, certainly for this case (where the method is not really dynamic at runtime, just at declaration time).

One application that jumps to mind is documenation. If you'd add a docstring to foo using the metaclass, documentation might pick it up, via the __init__ method this is very unlikely.

It could also lead to hard-to-spot bugs. consider a piece of code that uses metainformation of a class. It could be that in 99.99% of the cases this is executed after an instance of Z is already created, but that 0.01% can lead to weird behaviour, if not crashes.

It could also get tricky in hierarchy chains, where you'd have to take good care where to call the parent's constructor. A class like this for example could give issues:

class Zd(Z):
    def __init__(self):
        self.foo()
        Z.__init__(self)
a = Zd()
...
AttributeError: 'Zd' object has no attribute 'foo'

While this works fine:

class Yd(Y):
    def __init__(self):
        self.foo()
        Y.__init__(self)
a = Yd()

It might seem stupid to call a method without the relevant __init__ being used but it can happen when you're not careful, and certainly in more complex hierarchies where the MRO is not immediately obvious. And it's hard to spot this error because most often Zd() will succeed as soon as Z.__init__ was called somewhere before.

Answer 2

If the problem is adding methods dynamically, python does handle it in a quite straightforward manner. It goes as follows:

#!/usr/bin/python
class Alpha():

    def __init__(self):
        self.a = 10
        self.b = 100

alpha = Alpha()
print alpha.a
print alpha.b

def foo(self):
    print self.a * self.b * self.c

Alpha.c = 1000
Alpha.d = foo

beta = Alpha()
beta.d()

Output:

$ python script.py 
10
100
1000000

Regards!

P.S.: I see no resemblance of black magic here (:

Edit:

Considering martineau's comment, I'm adding data attributes, not method function attributes. I really can see no difference between doing Alpha.d = foo and Alpha.d = lambda self: foo(self), except that I'd be using a lambda function as a wrapper to the function foo added.

The addition of the method is the same, and python itself name both additions the same:

#!/usr/bin/python
class Alpha():

    def __init__(self):
        self.a = 10
        self.b = 100

alpha = Alpha()
print alpha.a
print alpha.b

def foo(self):
    print self.a * self.b * self.c

Alpha.c = 1000

Alpha.d = lambda self: foo(self)
Alpha.e = foo

print Alpha.d
print Alpha.e

a = Alpha()
a.d()
a.e()

Output:

10
100
<unbound method Alpha.<lambda>>
<unbound method Alpha.foo>
1000000
1000000

As shown, python itself name both resultant additions as methods -- the only difference is that one is a reference to function foo, and the other is a reference to a lambda function that uses the function foo in its definition body.

If I said something wrong, please, correct me.

Regards!