Cython: How to wrap a C++ function that returns a C++ object?

Question

I'm working on a Python project where I'd like to interface with a C++ package that has already been written. Since I'll be using Cython in other portions of this project, I'd prefer to wrap using Cython.

In brief, I need to wrap a function, FooBar, that returns an object of custom class type Bar.

Here's the Bar.h:

#include <cstddef> // For size_t
#include <vector>

/* data returned by function FooBar()*/
class Bar {
public:
    size_t X;
    std::vector<size_t> Y;      
    std::vector<double> Z;  
    std::vector<double> M;  
    std::vector<size_t> N;  

};


Bar FooBar(const std::vector<double> & O, size_t P, size_t Q);

And PyBar.pyx:

from libcpp.vector cimport vector

cdef extern from "Bar.h":
    cdef cppclass Bar:
        size_t X
        vector[size_t] Y    
        vector[double] Z    
        vector[double] M 
        vector[size_t] N    
    cdef Bar FooBar(const vector[double] & O, size_t P, size_t Q)

cdef class PyBar:
    cdef Bar *thisptr      # hold a C++ instance which we're wrapping

    def __cinit__(self, O, P, Q):
        C_Bar = FooBar(O, P, Q)
        self.thisptr = &C_Bar

    def __dealloc__(self):
        del self.thisptr

Actual Question: Is this even the right approach to what I want to do? For reference, if I just tried to wrap the class by itself I have had no problem: I can import the module, create objects using PyBar(), and underlying C methods implemented on the class would work. The issue is trying to wrap a function that returns objects of the C++ class. In the wild, I'll never actually want to create PyBar representation of any Bar object that wasn't created by FooBar, so this is the approach I decided upon after much head scratching.

Answer 1

With respect to the first part of the problem, I think the more elegant change would be to have FooBar defined as:

Bar* FooBar(const std::vector<double> & O, size_t P, size_t Q);

and have it return a "new" allocated pointer. I think in your original Cython code __cinit__ you'll create a stack allocated Bar, take a pointer of that, and then that will expire resulting in eventual disaster.

An alternative solution that might work would be be to keep FooBar returning Bar, change PyBar so it starts

cdef class PyBar:
   cdef Bar this_obj

   def __cinit__(self, O, P, Q):
      self.this_obj = FooBar(O,P,Q)

i.e. keeps an object rather than a pointer. No __dealloc__ should be necessary.

I don't know about the undefined symbol error...

Answer 2

After playing with this for awhile, the only semi-elegant (emphasis on the semi) solutions I found do involve modifying the existing C++ code. The approach I half-implemented in my question has many problems and should probably be ignored.

Maybe someone with more experience writing C++ code can come up with something better, but for the sake of posterity:

I personally found it easier to modify FooBar() so that it is a member function of Bar: instead of returning a Bar object, it now modifies the instance it is called from. Then, when wrapping Bar in Cython, I do not expose FooBar() as a class method, but I do call the method in the constructor for the Python (and thus, the corresponding C++) object. This works for me because, as I stated, I really only ever intend to deal with Bar objects that have been initialized with some set of values by FooBar().

In the end, I chose this approach over using a copy constructor (which would allow me to initialize a new Python/corresponding C++ Bar object from an existing Bar object created by FooBar), because it seemed more readable to me. The advantage of the copy constructor approach would be that one would only have to modify the Bar class definition in C (adding a copy constructor), which might be preferable if you're truly uncomfortable about changing the implementation of FooBar(). In my case, since Bar objects can sometimes contain very large vectors, a copy constructor also seemed like a bad idea for performance reasons.

Here's my final code:

Bar.h:

#include <cstddef> // For size_t
#include <vector>

class Bar {
public:
    size_t X;
    std::vector<size_t> Y;
    std::vector<double> Z;
    std::vector<double> M;
    std::vector<size_t> N;

    void FooBar(const std::vector<double> & O, size_t P, size_t Q);

    ClusterResult(){}


};

PyBar.pyx:

from libcpp.vector cimport vector

cdef extern from "Bar.h":
    cdef cppclass Bar:
        size_t X
        vector[size_t] Y    
        vector[double] Z    
        vector[double] M  
        vector[size_t] N
        Bar()    
        void FooBar(const vector[double] & O, size_t P, size_t Q)


cdef class PyBar:
    cdef Bar *thisptr      # hold a C++ instance which we're wrapping

    def __cinit__(self, O, P, Q):
        self.thisptr = new Bar()
        self.thisptr.FooBar(O, P, Q)

    def __dealloc__(self):
        del self.thisptr


#Below, I implement the public attributes as get/setable properties.
#could have written get/set functions, but this seems more Pythonic.

    property X:
        def __get__(self): return self.thisptr.X
        def __set__(self, X): self.thisptr.X = X

    property Y:
        def __get__(self): return self.thisptr.Y
        def __set__(self, Y): self.thisptr.Y = Y

    property Z:
        def __get__(self): return self.thisptr.Z
        def __set__(self, Z): self.thisptr.centers = Z

    property M:
        def __get__(self): return self.thisptr.M
        def __set__(self, size): self.thisptr.M = M

    property N:
        def __get__(self): return self.thisptr.N
        def __set__(self, size): self.thisptr.N = N

Refactoring FooBar() Implementation:

Then, I rewrote the implementation of FooBar() in Bar.cpp, changing the return type to void and substituting the Bar result object previously returned by the function for this. For example (and being explicit in my use of this for the sake of clarity):

Bar FooBar(const std::vector<double> & O, size_t P, size_t Q)
{

    Bar result = new Bar();
    result.X = P + 1;
    result.Z = std::sort(O.begin()+1, O.end());
    const size_t newParam = Q + 2;

    someOtherFunction(newParam, result);

    ...

}

would become something like this:

void Bar::FooBar(const std::vector<double> & O, size_t P, size_t Q)
{

    this->X = P + 1;
    this->Z = std::sort(O.begin()+1, O.end());
    const size_t newParam = Q + 2;

    someOtherFunction(newParam, *this);

...

}