I originally asked this question on the Python capi-sig list: How to pass arguments to tp_new and tp_init from subtypes?
I'm reading the Python PEP-253 on subtyping and there are plenty of good recommendations on how to structure the types, call tp_new
and tp_init
slots, etc.
But, it lacks an important note on passing arguments from sub to super type. It seems the PEP-253 is unfinished as per the note:
(XXX There should be a paragraph or two about argument passing here.)
So, I'm trying to extrapolate some strategies well known from the Python classes subtyping, especially techniques that each level strips-off arguments, etc.
I'm looking for techniques to achieve similar effect to this, but using plain Python C API (3.x):
class Shape:
def __init__(self, shapename, **kwds):
self.shapename = shapename
super().__init__(**kwds)
class ColoredShape(Shape):
def __init__(self, color, **kwds):
self.color = color
super().__init__(**kwds)
What would be the equivalent in Python C API?
How to deal with similar situation but with arguments specific to derived class expected in different order?
It is arguments given at the end of the args tuple (or kwds
dict, I assume principle would be same).
Here is some (pseudo-)code that illustrates the situation:
class Base:
def __init__(self, x, y, z):
self.x = x
self.y = y
self.z = z
class Derived(Base):
def __init__(self, x, y, a):
self.a = a
super().__init__(x, y, None):
Note, if the a
was expected first:
Derived.__init__(self, a, x, y)
it would be similar situation to the Shape
and ColoredShape
above.
It would also be easier to deal with, I assume.
Could anyone help to figure out the missing XXX comment mentioned above and correct technique for passing arguments from subtype up to super type(s) on construction?
UPDATE 2012-07-17:
Inspired by ecatmur's answer below I looked through Python 3 sources and I found defdict_init
constructor of collections.defaultdict type object interesting. The type is derived from PyDictObject
and its constructor takes additional argument of default_factory
. The constructor signature in Python class is this:
class collections.defaultdict([default_factory[, ...]])
Now, here is how the default_factory
is stripped from original args
tuple, so the rest of arguments is forwarded to the tp_init
of base type, it is PyDictObject
:
int result;
PyObject *newargs;
Py_ssize_t n = PyTuple_GET_SIZE(args);
...
newargs = PySequence_GetSlice(args, 1, n);
...
result = PyDict_Type.tp_init(self, newargs, kwds);
Note, this snipped presence only the relevant part of the defdict_init
function.
The problem is that PyArgs_ParseTupleAndKeywords
doesn't provide a way to extract extra *args
and **kwargs
from the input args and keywords; indeed, any extra arguments result in a TypeError
; "Function takes %s %d positional arguments (%d given)", or "'%U' is an invalid keyword argument for this function".
This means that you're going to have to parse args and keywords yourself; you're guaranteed that args is a tuple and keywords is a dict, so you can use the standard methods (PyTuple_GET_ITEM
and PyDict_GetItemString
) to extract the arguments you're interested in, and identify and construct a tuple and dict to pass on from the remainder. You obviously can't modify args, because tuples are immutable; and while popping items from keywords should be OK it does seem a little risky (example crash).
A more ambitious but definitely feasible route would be to copy vgetargskeywords
from getargs.c
(http://hg.python.org/cpython/file/tip/Python/getargs.c) and extend it to take optional out-parameters for remainder *args
and **kwargs
. This should be fairly straightforward as you just need to modify the parts where it detects and throws TypeError
on extra arguments (extra args; extra keywords). Good luck if you choose this route.
OK, to answer your first question: first of all you have a C Structure representing your class object, but from a C side. So in a header file called Shapes.h
typedef struct {
PyObject_HEAD
char *shapename;
} Shape;
typedef struct {
PyObject_HEAD
char *shapename;
char *color;
} ColouredShape;
The things to note here are the following:
Next we need a type for our classes. The type basically defines the object from a Python point of view, i.e., its methods, members and any special methods its has. It's also where we tell Python that ColouredShape is a subclass of Shape. Its looks like the following:
PyTypeObject ShapeType = {
PyObject_HEAD_INIT(NULL)
0, /*ob_size*/
"mymod.Shape", /*tp_name*/
sizeof(ShapeType), /*tp_basicsize*/
0, /*tp_itemsize*/
(destructor) SimpleParameter_dealloc, /*tp_dealloc*/
0, /*tp_print*/
0, /*tp_getattr*/
0, /*tp_setattr*/
0, /*tp_compare*/
0, /*tp_repr*/
0, /*tp_as_number*/
0, /*tp_as_sequence*/
0, /*tp_as_mapping*/
0, /*tp_hash */
0, /*tp_call*/
Shape__str__, /*tp_str*/
0, /*tp_getattro*/
0, /*tp_setattro*/
0, /*tp_as_buffer*/
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /*tp_flags*/
"A class representing a Shape", /* tp_doc */
0, /* tp_traverse */
0, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
0, /* tp_iter */
0, /* tp_iternext */
Shape_methods, /* tp_methods */
0, /* tp_members */
0, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
0, /* tp_descr_get */
0, /* tp_descr_set */
0, /* tp_dictoffset */
(initproc) Shape_init, /* tp_init */
0, /* tp_alloc */
Shape_new, /* tp_new */
};
PyTypeObject ColouredShapeType = {
PyObject_HEAD_INIT(NULL)
0, /*ob_size*/
"mymod.ColouredShape", /*tp_name*/
sizeof(ColouredShape), /*tp_basicsize*/
0, /*tp_itemsize*/
(destructor) ColouredShape_dealloc, /*tp_dealloc*/
0, /*tp_print*/
0, /*tp_getattr*/
0, /*tp_setattr*/
0, /*tp_compare*/
0, /*tp_repr*/
0, /*tp_as_number*/
0, /*tp_as_sequence*/
0, /*tp_as_mapping*/
0, /*tp_hash */
0, /*tp_call*/
0, /*tp_str*/
0, /*tp_getattro*/
0, /*tp_setattro*/
0, /*tp_as_buffer*/
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /*tp_flags*/
"A class representing a coloured shape", /* tp_doc */
0, /* tp_traverse */
0, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
0, /* tp_iter */
0, /* tp_iternext */
ColouredShape_methods, /* tp_methods */
0, /* tp_members */
0, /* tp_getset */
&ShapeType, /* tp_base */
0, /* tp_dict */
0, /* tp_descr_get */
0, /* tp_descr_set */
0, /* tp_dictoffset */
(initproc) ColouredShape_init, /* tp_init */
0, /* tp_alloc */
ColouredShape_new, /* tp_new */
};
One important point to note is that mymod must be the name of your Python C extension as imported from Python. In answer to your second question, your init function would look as follows:
int Shape_init(Shape *self, PyObject *args, PyObject *kwds){
char *colour = null;
static char *kwdlist[] = {"colour", NULL};
if (!PyArg_ParseTupleAndKeywords(args, kwds, "s", kwdlist,
&colour)){
return -1;
}
//Initialise your object here
}
There is no reason why ColouredShape_init cannot call Shape_init. However my understanding of PyArgs_ParseTupleAndKeywords is:
Which is where your difficulty will be if you tried to go that way.
If you have any further questions on it let me know. But I suggest you look at the PyArgs_ParseTupleAndKeywords to get a better understanding of it
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With