Functions, Callable Objects, and how both are created in Python

Question

I'm wondering about the more intricate differences between functions and callable objects. For example, if you do:

def foo1():
   return 2 + 4

class foo2:
   def __call__():
      return 2 + 4

import sys 
sys.getsizeof(foo1)  # returns 136
sys.getsizeof(foo2)  # returns 1016

There's clearly a big difference between functions and callable objects. However, I can't find a lot of documentation on what is going on behind the scenes. I know functions are first-class objects, but I also know that classes have a lot more going on than your regular functions. class foo2 is created with a metaclass, type().

My questions then, are these:

1. When you create a function def foo1():, how does this differ from the process of defining a class with a metaclass? Is there a version of type() but for functions, a metafunction?
2. Say someone wanted to write their own metafunction (the real reason behind this), would it be better to just use decorators, or maybe a metaclass that makes callable classes? What advantages would either offer (a metaclass that makes callable classes seems clunky)?
3. Is the only purpose behind having a callable object to have a function that can store info in it as well?

Answer 1

Functions are also callable objects:

>>> foo1.__call__
<method-wrapper '__call__' of function object at 0x105bafd90>
>>> callable(foo1)
True

But a class needs to keep track of more information; it doesn't matter here that you gave it a __call__ method. Any class is bigger than a function:

>>> import sys
>>> def foo1():
...    return 2 + 4
...
>>> class foo3:
...    pass
...
>>> sys.getsizeof(foo1)
136
>>> sys.getsizeof(foo3)
1056

A function object is a distinct object type:

>>> type(foo1)
<class 'function'>

and is reasonably compact because the meat is not actually in the function object but in other objects referenced by the function object:

>>> sys.getsizeof(foo1.__code__)
144
>>> sys.getsizeof(foo1.__dict__)
240

And that's it really; different types of objects have different sizes because they track different things or use composition to store stuff in other objects.

You can use the type(foo1) return value (or types.FunctionType, which is the same object) to produce new function objects if you so desire:

>>> import types
>>> types.FunctionType(foo1.__code__, globals(), 'somename')
<function foo1 at 0x105fbc510>

which is basically what the interpreter does whenever a def function(..): ... statement is being executed.

Use __call__ to make custom classes callable when that makes sense to your API. The enum.Enum() class is callable, for example, specifically because using the call syntax gives you a syntax distinct from subscription, which was used for other purposes. And a xmlrpc.client.ServerProxy() object produces method objects that are instances of _Method, because they proxy a remote call, not a local function.

Answer 2

Is there a version of type() but for functions, a metafunction?

Sort of. Functions have a type, and that type can be used to construct new functions from, at minimum, a code object and a globals dictionary. (The code object can be constructed using compile() or grabbed from an existing function, or, if you're a masochist, built from a bytecode string and other information using the code type's constructor.) The function type is not a meta-anything because functions are instances, not classes. You can get this type using type(lambda:0) (putting any function in the parentheses), and do help(type(lambda:0)) to see its arguments. The function type is an instance of type.

Say someone wanted to write their own metafunction

You can't subclass the function type, sorry.

class FunkyFunc(type(lambda: 0)): pass

TypeError: Error when calling the metaclass bases
    type 'function' is not an acceptable base type

Is the only purpose behind having a callable object to have a function that can store info in it as well?

There are many uses for it. Your example is one (although function instances can have attributes, a class provides better documentation of the attributes); the flip side, making a plain old data object callable, is another (I gave a kind of funky example of that in this answer). You can use classes to write decorators if the instances are callable. You can use __call__ on a metaclass to customize instance construction. And you can use it to write functions with different behavior, sort of as if you could in fact subclass the function type. (If you want C-style "static" variables in a function, you might write it as a class with a __call__ method and use attributes to store the static data.)

A fun thing about function objects is that, because they're callable, they themselves have a __call__ method. But a method is callable, and so that __call__ method also has a __call__ method, and that __call__ method (being callable) also has a __call__ method, and so on ad infinitum. :-)