Python C extension: method signatures for documentation?

Question

I am writing C extensions, and I'd like to make the signature of my methods visible for introspection.

static PyObject* foo(PyObject *self, PyObject *args) {

    /* blabla [...] */

}

PyDoc_STRVAR(
    foo_doc,
    "Great example function\n"
    "Arguments: (timeout, flags=None)\n"
    "Doc blahblah doc doc doc.");

static PyMethodDef methods[] = {
    {"foo", foo, METH_VARARGS, foo_doc},
    {NULL},
};

PyMODINIT_FUNC init_myexample(void) {
    (void) Py_InitModule3("_myexample", methods, "a simple example module");
}

Now if (after building it...) I load the module and look at its help:

>>> import _myexample
>>> help(_myexample)

I will get:

Help on module _myexample:

NAME
    _myexample - a simple example module

FILE
    /path/to/module/_myexample.so

FUNCTIONS
    foo(...)
        Great example function
        Arguments: (timeout, flags=None)
        Doc blahblah doc doc doc.

I would like to be even more specific and be able to replace foo(...) by foo(timeout, flags=None)

Can I do this? How?

Answer 1

It has been 7 years but you can include the signature for C-extension function and classes.

Python itself uses the Argument Clinic to dynamically generate signatures. Then some mechanics create a __text_signature__ and this can be introspected (for example with help). @MartijnPieters explained this process quite well in this answer.

You may actually get the argument clinic from python and do it in a dynamic fashion but I prefer the manual way: Adding the signature to the docstring:

In your case:

PyDoc_STRVAR(
    foo_doc,
    "foo(timeout, flags=None, /)\n"
    "--\n"
    "\n"
    "Great example function\n"
    "Arguments: (timeout, flags=None)\n"
    "Doc blahblah doc doc doc.");

I made heavy use of this in my package: iteration_utilities/src. So to demonstrate that it works I use one of the C-extension functions exposed by this package:

>>> from iteration_utilities import minmax
>>> help(minmax)
Help on built-in function minmax in module iteration_utilities._cfuncs:

minmax(iterable, /, key, default)
    Computes the minimum and maximum values in one-pass using only
    ``1.5*len(iterable)`` comparisons. Recipe based on the snippet
    of Raymond Hettinger ([0]_) but significantly modified.

    Parameters
    ----------
    iterable : iterable
        The `iterable` for which to calculate the minimum and maximum.
[...]

The docstring for this function is defined this file.

It is important to realize that this isn't possible for python < 3.4 and you need to follow some rules:

• You need to include --\n\n after the signature definition line.

• The signature must be in the first line of the docstring.

• The signature must be valid, i.e. foo(a, b=1, c) fails because it's not possible to define positional arguments after arguments with default.

• You can only provide one signature. So it doesn't work if you use something like:

  foo(a)
  foo(x, a, b)
  --
  
  Narrative documentation
  

Answer 2

My usual approach to finding out about things like this is: "use the source".

Basically, I would presume that the standard modules of python would use such a feature when available. Looking at the source (for example here) should help, but in fact even the standard modules add the prototype after the automatic output. Like this:

torsten@pulsar:~$ python2.6
>>> import fcntl
>>> help(fcntl.flock)
flock(...)
    flock(fd, operation)

    Perform the lock operation op on file descriptor fd.  See the Unix [...]

So as upstream is not using such a feature, I would assume it is not there. :-)

Okay, I just checked current python3k sources and this is still the case. That signature is generated in pydoc.py in the python sources here: pydoc.py. Relevant excerpt starting in line 1260:

        if inspect.isfunction(object):
            args, varargs, varkw, defaults = inspect.getargspec(object)
            ...
        else:
            argspec = '(...)'

inspect.isfunction checks if the object the documentation is requested for is a Python function. But C implemented functions are considered builtins, therefore you will always get name(...) as the output.