Python C Extensions - Why must callable C functions take arguments and return PyObject *

Question

I'm just starting to play with Python C extensions and am curious as to why a C function, which is callable from Python must take 2 PyObject* arguments and return a PyObject*. I wrote the following "Hello World" extension:

#include <Python.h>

static PyObject *
hello_world(PyObject *self, PyObject *noargs)
{
   printf("Hello World\n");
   return Py_BuildValue("");
}


// Module functions table.

static PyMethodDef
module_functions[] = {
    { "hello_world", hello_world, METH_NOARGS, "hello world method" },
    { NULL }
};


// This function is called to initialize the module.
PyMODINIT_FUNC
inittesty2(void)
{
    Py_InitModule("testy2", module_functions);
}

Why can't I (especially with METH_NOARGS) use the following hello_world method:

static void
hello_world()
{
   printf("Hello World\n");
}

?

Answer 1

There are several things to say about the various PyObject pointers.

1. The one required as return type is used for the exception handling mechanism. Specifically, if your function returns a null pointer, the Python interpreter will throw an exception. (You should only do that after calling one of the PyErr_.. functions to set a specific exception.)

   This also means that whenever you do not want to throw an exception, you must return a pointer to some real PyObject. If there is nothing in particular your function is supposed to return, simply return Py_None (best use the Py_RETURN_NONE macro to get the reference count right), or "true" (using Py_RETURN_TRUE).

2. The first argument, PyObject *self points to the object the function is called from, or to the module instance it belongs to. Note that every function you define is either a class method, or a module method. There are no totally independent functions.

3. The second argument, PyObject *args points to the function argument (which may be a tuple or list of multiple arguments). You are right in pointing out that a function that does not take any arguments should not need this — and, as far as I can tell, you are right. You do not have to define it; you can simply define a function as

   static PyObject *PyMyClass_MyFunc(PyObject *self) {
     /* ..do something.. */
     Py_RETURN_TRUE;
   }
   

   You will still have to cast this to PyCFunction when you put it into the PyMethodDef for the data type you define, but I believe that cast is safe as long as you use the METH_NOARGS flag. But note the comments below for possible risks.

4. Finally, a function may in fact have a third argument like this:

   static PyObject *PyMyClass_Func(PyObject *self, PyObject *args, PyObject *kwds)
   {
     /*...*/
   }
   

   The third argument is used for named, optional arguments. In this case, too, you must cast the function pointer to PyCFunction, but that, too, is safe if you set the right flag (METH_KEYWORDS) for your function in the method table.

Answer 2

The first argument for module level functions is the module object. When defining classes in C (the same PyMethodDef structure is used for methods there), the first argument is the instance (similar to self in Python).

When using METH_NOARGS, Python will pass NULL as the second argument. They could cast it to a function with one argument but I guess they didn't think it's needed.

The return value is easy to explain. Every Python function has a return value. If you don't explicitly use return in Python, the function will return None.

Of course in C you have to be explicit about the return value so if you don't use it, you have to return None yourself. Python provides a macro for this:

Py_RETURN_NONE;

Alternatively you could access the global None instance yourself:

Py_INCREF(Py_None);
return Py_None;

but the macro is easier to use.

You could think that returning NULL should be equivalent to None but NULL is used to indicate that the function raised an exception.