Returning a complex object containing PyObject from c++ function Cython

Question

I am trying to wrap some C++ classes and functions to Python using Cython. So far I have wrapped 2 classes, and now I want to wrap a function.

The function's signature is

std::map<std::string, std::vector<PyObject*>> analyze(PyObject* img, LandmarkDetector::CLNF& clnf_model, LandmarkDetector::FaceModelParameters& params);

I have successfully wrapped the CLNF and FaceModelParameters classes, and I am having trouble wrapping this analyze function.

The function deals with PyObject*s because it deals with opencv, and I'd like to be able to pass them easily between the languages. I am using these functions in order to perform the casting between cv::Point to python objects and between python Mat to cv::Mat.

This is my pyx file:

from libcpp.vector cimport vector
from libcpp.map cimport map
from libcpp.string cimport string
from cpython.ref cimport PyObject
from cython.operator cimport dereference as deref

cdef extern from "LandmarkDetectorModel.h" namespace "LandmarkDetector":
    cdef cppclass CLNF:
        CLNF(string) except +

cdef extern from "LandmarkDetectorParameters.h" namespace "LandmarkDetector":
    cdef cppclass FaceModelParameters:
        FaceModelParameters(vector[string] &) except +

cdef class PyCLNF:
    cdef CLNF *thisptr
    def __cinit__(self, arg):
        self.thisptr = new CLNF(<string> arg)

cdef class PyLandmarkDetectorParameters:
    cdef FaceModelParameters *thisptr
    def __cinit__(self, args):
        self.thisptr = new FaceModelParameters(args)

cdef extern from "FaceLandmarkVid.h":
    map[string, vector[object]] analyze(object, CLNF&, FaceModelParameters&)

cdef PyAnalyze(object img, PyCLNF clnf, PyLandmarkDetectorParameters facemodel):
    return analyze(img, deref(clnf.thisptr), deref(facemodel.thisptr))

But upon trying to compile it I get the error message

landmarks.pyx:26:23: Python object type 'Python object' cannot be used as a template argument

(which refers to the line map[string, vector[object]] analyze [...])

Answer 1

Unfortunately you can't use Cython's automatic std::map->dict and std::vector->list conversions here. I think the fundamental issue with writing such a conversion is that Cython doesn't know what C++ is doing with reference counting, so would struggle to get that right reliably.

Instead you have to template the vector as being a PyObject* and write your own conversion functions. There's a slight complication that templating with PyObject* seems to confuse Cython but that can be worked round with a typedef:

# unchanged [...]
from cpython.ref cimport PyObject, Py_DECREF
from cython.operator cimport dereference as deref, preincrement
# unchanged [...]

ctypedef PyObject* PyObjectPtr # I run into a bug templaing vector otherwise

cdef extern from "FaceLandmarkVid.h":
    map[string, vector[PyObjectPtr]] analyze(object, CLNF&, FaceModelParameters&)

# an extra function to convert the vector to a list
cdef convertVector(vector[PyObjectPtr]& v):
    cdef vector[PyObjectPtr].iterator i = v.begin()
    cdef vector[PyObjectPtr].iterator end = v.end()

    cdef list out = []

    while i != end:
        out.append(<object>deref(i))
        # reduce reference count to account for destruction of the
        # vector at the end of the conversion process
        Py_DECREF(<object>deref(i))
        preincrement(i)

    return out

cdef PyAnalyze(object img, PyCLNF clnf, PyLandmarkDetectorParameters facemodel):
    cdef map[string, vector[PyObjectPtr]] res = analyze(img, deref(clnf.thisptr), deref(facemodel.thisptr))
    cdef map[string, vector[PyObjectPtr]].iterator i = res.begin()
    cdef map[string, vector[PyObjectPtr]].iterator end = res.end()

    cdef dict out = {}    

    while i!=end:

        out[deref(i).first] = convertVector(deref(i).second)
        preincrement(i)
    return out

Essentially, we iterate over the map, and iterate over the vectors within it, casting the PyObject* to <object>. I've made an assumption about reference counting here - I'm assuming that your C++ code never decreases the reference count of the PyObjects in the vectors (and so you need to decrease it yourself to account for the destruction of the vector). A second assumption is that none of the PyObject* are NULL.

(This is tested to compile in Cython but I have no way of testing that it compiles in C++ or runs correctly).

---

Edit: I realized that I'd made a slight mistake in reference counting which should now be corrected.