Class property using Python C-API

Question

What is the best way to create class properties (as here and here) using the Python C-API? Static properties would also work in my case.

Follow up:

I tried to implement yak's suggestion. I defined a class P with get and set functions in its tp_descr_get and tp_descr_set slots. Then I added an instance of P to the dictionary of the type object for the class X under the key p. Then

x1 = X()
x2 = X()
x1.p = 10
print x1.p
print x2.p
print X.p
x2.p = 11
print x1.p
print x2.p
print X.p

works (first 10 is printed three times, then 11 is printed three times), but

X.p = 12

fails with the error message

TypeError: can't set attributes of built-in/extension type 'X'

How do I fix that?

Follow up 2:

If I allocate the type object with PyMem_Malloc and set the Py_TPFLAGS_HEAPTYPE flag, then everything works ; I can do X.p = 12 with the expected result.

Things also work if I keep the type object in a static variable and set the Py_TPFLAGS_HEAPTYPE flag, but that is obviously not a good idea. (But why does it matter whether the type object is in static or dynamic memory? I never let its reference count drop to 0 anyway.)

The restriction that you can only set attributes on dynamic types seems very strange. What is the rationale behind this?

Follow up 3:

No, it does not work. If I make the type X dynamic, then X.p = 12 does not set the property X.p to twelve; it actually binds the object 12 to the name X.p. In other words, afterwards, X.p is not an integer-valued property but an integer.

Follow up 4:

Here is the C++ code for the extension:

#include <python.h>
#include <exception>

class ErrorAlreadySet : public std::exception {};

// P type ------------------------------------------------------------------

struct P : PyObject
{
    PyObject* value;
};

PyObject* P_get(P* self, PyObject* /*obj*/, PyObject* /*type*/)
{
    Py_XINCREF(self->value);
    return self->value;
}

int P_set(P* self, PyObject* /*obj*/, PyObject* value)
{
    Py_XDECREF(self->value);
    self->value = value;
    Py_XINCREF(self->value);
    return 0;
}

struct P_Type : PyTypeObject
{
    P_Type()
    {
        memset(this, 0, sizeof(*this));
        ob_refcnt = 1;
        tp_name = "P";
        tp_basicsize = sizeof(P);
        tp_descr_get = (descrgetfunc)P_get;
        tp_descr_set = (descrsetfunc)P_set;
        tp_flags = Py_TPFLAGS_DEFAULT;

        if(PyType_Ready(this)) throw ErrorAlreadySet();
    }
};

PyTypeObject* P_type()
{
    static P_Type typeObj;
    return &typeObj;
}


// P singleton instance ----------------------------------------------------

P* createP()
{
    P* p_ = PyObject_New(P, P_type());
    p_->value = Py_None;
    Py_INCREF(p_->value);
    return p_;
}

P* p()
{
    static P* p_ = createP();
    Py_INCREF(p_);
    return p_;
}

PyObject* p_value()
{
    PyObject* p_ = p();
    PyObject* value = p()->value;
    Py_DECREF(p_);
    Py_INCREF(value);
    return value;
}


// X type ------------------------------------------------------------------

struct X : PyObject {};

void X_dealloc(PyObject* self)
{
    self->ob_type->tp_free(self);
}

struct X_Type : PyTypeObject
{
    X_Type()
    {
        memset(this, 0, sizeof(*this));
        ob_refcnt = 1;
        tp_name = "M.X";
        tp_basicsize = sizeof(X);
        tp_dealloc = (destructor)X_dealloc;
        tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HEAPTYPE;

        tp_dict = PyDict_New();
        PyObject* key = PyString_FromString("p");
        PyObject* value = p();
        PyDict_SetItem(tp_dict, key, value);
        Py_DECREF(key);
        Py_DECREF(value);

        if(PyType_Ready(this)) throw ErrorAlreadySet();
    }

    void* operator new(size_t n) { return PyMem_Malloc(n); }
    void operator delete(void* p) { PyMem_Free(p); }
};

PyTypeObject* X_type()
{
    static PyTypeObject* typeObj = new X_Type;
    return typeObj;
}

// module M ----------------------------------------------------------------

PyMethodDef methods[] = 
{
    {"p_value", (PyCFunction)p_value, METH_NOARGS, 0},
    {0, 0, 0, 0}
};

PyMODINIT_FUNC
initM(void)
{
    try {
        PyObject* m = Py_InitModule3("M", methods, 0);
        if(!m) return;
        PyModule_AddObject(m, "X", (PyObject*)X_type());
    }
    catch(const ErrorAlreadySet&) {}
}

This code defines a module M with a class X with a class property p as described before. I have also added a function p_value() that lets you directly inspect the object that implements the property.

Here is the script I have used to test the extension:

from M import X, p_value

x1 = X()
x2 = X()

x1.p = 1
print x1.p
print x2.p
print X.p
print p_value()
print

x2.p = 2
print x1.p
print x2.p
print X.p
print p_value()
print

X.p = 3
print x1.p
print x2.p
print X.p
print p_value()     # prints 2
print

x1.p = 4       # AttributeError: 'M.X' object attribute 'p' is read-only

Answer 1

Similar to these Python solutions, you will have to create a classproperty type in C and implement its tp_descr_get function (which corresponds to __get__ in Python).

Then, if you want to use that in a C type, you would have to create an instance of your classproperty type and insert it into dictionary of your type (tp_dict slot of your type).

Follow up:

It would seem that it's impossible to set an attribute of a C type. The tp_setattro function of the metaclass (PyType_Type) raises a "can't set attributes of built-in/extension type" exception for all non-heap types (types with no Py_TPFLAGS_HEAPTYPE flag). This flag is set for dynamic types. You could make your type dynamic but it might be more work then it's worth.

This means that the solution I gave initially allows you to create a property (as in: computed attribute) on a C type object with the limitation that it's read only. For setting you could use a class/static-method (METH_CLASS/METH_STATIC flag on a method in tp_methods).

Answer 2

I'll try to convey the essence of what I've discovered about using class static properties.

My (edited) code is as follows:

// Prepare your type object, which you will no doubt complete 
// by calling PyType_Ready, as here.
if (PyType_Ready(typeObj) < 0)
{
  return;
}

Py_INCREF(typeObj);
PyModule_AddObject(module, typeName, (PyObject*) typeObj);

// Now we add static members directly to the type's tp_dict, but 
// only *after* we've registered the type (above)
PyObject* dict = typeObj->tp_dict;

// You'll have your own wrapper / C values in the line below. This is just
// a pseudo-version of what I am (successfully) using.
PyObject* tmp = MyCreateWrapper(myStaticValueInC);

// Py_INCREF(tmp); // You may need this, depending on how line above works.

PyDict_SetItemString(dict, staticPropertyName, tmp);

Py_DECREF(tmp);

I believe all the essentials are here in terms of what order to construct your code in order to implement a class property.