Python and Smalltalk - Metaprogramming capabilities comparison

Question

I have of late been learning Python, and am amazed by its superb runtime metaprogramming capabilities. Previously I came across the term 'runtime metaprogramming' was when I was reading about Smalltalk, which as far as I know boasts of best runtime metaprogramming capabilities. How well does Python stack up against Smalltalk w.r.t. metaprogramming? What are the notable differences between the approaches taken by two languages?

Answer 1

Python actually holds up fairly well here. Smalltalk usually doesn't make explicit distinction between program and metaprogramm, but Python is more explicit - eg, the special syntax for decorators or the __foo__() naming convention for metaprogramming hooks. This is a good thing.

On the other hand, it's a bit of an apples-to-oranges comparison. Smalltalk is a smaller, tighter language than Python, and so there's just less material to manipulate with metaprograms. For example, consider __getattr__(). This is a hook that lets Python objects provide a custom implementation of attribute access. Smalltalk doesn't have anything like this. But! Smalltalk enforces tighter encapsulation of an object's internal state, and there's no equivalent of the object.attribute syntax that's used in Python. So reading an object's state requires going through a method... which is exactly what __getattr__() provides. So for a lot of cases where you'd use __getattr__() in Python, you'd just write a normal method in Smalltalk - no metaprogramming needed.

And it's like that all over the place: Python's __getitem__() and friends make it possible to write classes that mimic lists or dictionaries. Smalltalk doesn't need that because Array and Dictionary are just regular Smalltalk classes and there's no special syntax for using them. Python __eq__() and so on enable operator overloading. Smalltalk doesn't have operators, so you can implement + without doing anything special. Python's contextlib provides some nifty tools for implementing your own context managers. Smalltalk doesn't have a with construct, but it does have really lightweight syntax for lambdas, which lets you do the same sort of thing in a straightforward way.

Smalltalk's metaprogramming facilities tend to be pretty low-level. You can, for example, create your own CompiledMethod instances, and stick them into a class's method dictionary. You can also write your own compiler and specify that all the methods of a particular class be compiled with it. That enables all sorts of things - I've seen projects that experiment with alternate syntaxes, instrument bytecode for profiling, trap reads and writes to instance variables for transparent persistence, and so on.

Smalltalk's metaprogramming facilities are powerful, but they're not as neatly organized as Python's, and don't get used as often.

Answer 2

Posted as an answer at questioner's request.

One of the big ideas of Smalltalk is orthogonality. Frankly Python suffers in this respect. Not everything works on everything. Examples:

• inspect.getargspec() does not work on built-in functions or the results of calls to functools.partial (in the C interpreter anyway).
• eval only works for expression strings, and exec only works for statement strings.
• Lambda expressions cannot be pickled.
• myclass = type('x', (object,), {'__init__': partial(foo, value)}) produces a class that can't be instantiated, whereas passing an equivalent lambda expression instead of a partial works fine. (Though this may just be a bug not a feature.)

Maybe PyPy doesn't have these problems, I'm not sure. But I do love Python very much and find it very convenient to use metaclasses, currying and the occasional descriptor in real applications.