How can I document methods inherited from a metaclass?

Question

Consider the following metaclass/class definitions:

class Meta(type):
    """A python metaclass."""
    def greet_user(cls):
        """Print a friendly greeting identifying the class's name."""
        print(f"Hello, I'm the class '{cls.__name__}'!")

    
class UsesMeta(metaclass=Meta):
    """A class that uses `Meta` as its metaclass."""

As we know, defining a method in a metaclass means that it is inherited by the class, and can be used by the class. This means that the following code in the interactive console works fine:

>>> UsesMeta.greet_user()
Hello, I'm the class 'UsesMeta'!

However, one major downside of this approach is that any documentation that we might have included in the definition of the method is lost. If we type help(UsesMeta) into the interactive console, we see that there is no reference to the method greet_user, let alone the docstring that we put in the method definition:

Help on class UsesMeta in module __main__:
class UsesMeta(builtins.object)
 |  A class that uses `Meta` as its metaclass.
 |  
 |  Data descriptors defined here:
 |  
 |  __dict__
 |      dictionary for instance variables (if defined)
 |  
 |  __weakref__
 |      list of weak references to the object (if defined)

Now of course, the __doc__ attribute for a class is writable, so one solution would be to rewrite the metaclass/class definitions like so:

from pydoc import render_doc
from functools import cache

def get_documentation(func_or_cls):
    """Get the output printed by the `help` function as a string"""
    return '\n'.join(render_doc(func_or_cls).splitlines()[2:])


class Meta(type):
    """A python metaclass."""

    @classmethod
    @cache
    def _docs(metacls) -> str:
        """Get the documentation for all public methods and properties defined in the metaclass."""

        divider = '\n\n----------------------------------------------\n\n'
        metacls_name = metacls.__name__
        metacls_dict = metacls.__dict__

        methods_header = (
            f'Classmethods inherited from metaclass `{metacls_name}`'
            f'\n\n'
        )

        method_docstrings = '\n\n'.join(
            get_documentation(method)
            for method_name, method in metacls_dict.items()
            if not (method_name.startswith('_') or isinstance(method, property))
        )

        properties_header = (
            f'Classmethod properties inherited from metaclass `{metacls_name}`'
            f'\n\n'
        )

        properties_docstrings = '\n\n'.join(
            f'{property_name}\n{get_documentation(prop)}'
            for property_name, prop in metacls_dict.items()
            if isinstance(prop, property) and not property_name.startswith('_')
        )

        return ''.join((
            divider,
            methods_header,
            method_docstrings,
            divider,
            properties_header,
            properties_docstrings,
            divider
        ))


    def __new__(metacls, cls_name, cls_bases, cls_dict):
        """Make a new class, but tweak `.__doc__` so it includes information about the metaclass's methods."""

        new = super().__new__(metacls, cls_name, cls_bases, cls_dict)
        metacls_docs = metacls._docs()

        if new.__doc__ is None:
            new.__doc__ = metacls_docs
        else:
            new.__doc__ += metacls_docs

        return new

    def greet_user(cls):
        """Print a friendly greeting identifying the class's name."""
        print(f"Hello, I'm the class '{cls.__name__}'!")

    
class UsesMeta(metaclass=Meta):
    """A class that uses `Meta` as its metaclass."""

This "solves" the problem; if we now type help(UsesMeta) into the interactive console, the methods inherited from Meta are now fully documented:

Help on class UsesMeta in module __main__:
class UsesMeta(builtins.object)
 |  A class that uses `Meta` as its metaclass.
 |  
 |  ----------------------------------------------
 |  
 |  Classmethods inherited from metaclass `Meta`
 |  
 |  greet_user(cls)
 |      Print a friendly greeting identifying the class's name.
 |  
 |  ----------------------------------------------
 |  
 |  Classmethod properties inherited from metaclass `Meta`
 |  
 |  
 |  
 |  ----------------------------------------------
 |  
 |  Data descriptors defined here:
 |  
 |  __dict__
 |      dictionary for instance variables (if defined)
 |  
 |  __weakref__
 |      list of weak references to the object (if defined)

That's an awful lot of code to achieve this goal, however. Is there a better way?

How does the standard library do it?

I'm also curious about the way certain classes in the standard library manage this. If we have an Enum definition like so:

from enum import Enum

class FooEnum(Enum):
    BAR = 1

Then, typing help(FooEnum) into the interactive console includes this snippet:

 |  ----------------------------------------------------------------------
 |  Readonly properties inherited from enum.EnumMeta:
 |  
 |  __members__
 |      Returns a mapping of member name->value.
 |      
 |      This mapping lists all enum members, including aliases. Note that this
 |      is a read-only view of the internal mapping.

How exactly does the enum module achieve this?

The reason why I'm using metaclasses here, rather than just defining classmethods in the body of a class definition

Some methods that you might write in a metaclass, such as __iter__, __getitem__ or __len__, can't be written as classmethods, but can lead to extremely expressive code if you define them in a metaclass. The enum module is an excellent example of this.

Answer 1

The help() function relies on dir(), which currently does not always give consistent results. This is why your method gets lost in the generated interactive documentation. There's a open python issue on this topic which explains the problem in more detail: see bugs 40098 (esp. the first bullet-point).

In the meantime, a work-around is to define a custom __dir__ in the meta-class:

class Meta(type):
    """A python metaclass."""
    def greet_user(cls):
        """Print a friendly greeting identifying the class's name."""
        print(f"Hello, I'm the class '{cls.__name__}'!")

    def __dir__(cls):
        return super().__dir__() + [k for k in type(cls).__dict__ if not k.startswith('_')]

class UsesMeta(metaclass=Meta):
    """A class that uses `Meta` as its metaclass."""

which produces:

Help on class UsesMeta in module __main__:

class UsesMeta(builtins.object)
 |  A class that uses `Meta` as its metaclass.
 |
 |  Methods inherited from Meta:
 |
 |  greet_user() from __main__.Meta
 |      Print a friendly greeting identifying the class's name.

This is essentially what enum does - although its implementation is obviously a little more sophisticated than mine! (The module is written in python, so for more details, just search for "__dir__" in the source code).

Answer 2

I haven't looked at the rest of the stdlib, but EnumMeta accomplishes this by overriding the __dir__ method (i.e. specifying it in the EnumMeta class):

class EnumMeta(type):
    .
    .
    .
    def __dir__(self):
        return (
                ['__class__', '__doc__', '__members__', '__module__']
                + self._member_names_
                )