Cython: templates in python class wrappers

Question

Question

Is there a way to create a Python wrapper for Cython-wrapped C++ class with templates? (i.e. do exactly what is show here but with templates: http://docs.cython.org/src/userguide/wrapping_CPlusPlus.html#create-cython-wrapper-class).

I know about the fused types workaround (https://groups.google.com/forum/#!topic/cython-users/qQpMo3hGQqI) but that doesn't allow you to instatiate classes like vector<vector<int>>: the fused types have, quite unsurprisingly, no notion of recursion.

Rephrasing

What I would like to achieve is for a wrapped class like:

cdef extern from "header.h":
    cdef cppclass Foo[T]:
        Foo(T param)
        # ...

create a simple Python wrapper:

cdef class PyFoo[T]:  # I know the '[T]' can't be here, it's a wish
    cdef Foo[T] *thisptr
    def __cinit__(self, param):
        self.thisptr = new Foo[T](param)
    # ...

I'm quite certain that Cython doesn't support that per se, but maybe someone can think of a workaround. I'm not looking for idiomatic or nice examples, I'm just wondering if that's possible in any way.

Answer 1

As you say, Cython doesn't really support this.

I think by far the simplest approach is just to manually generate a bunch of Cython files using string substitution. Start with a "foowrapper.pxi.src" file (name as you wish...):

cdef class PyFoo_{T}:
  cdef Foo[{T}] *thisptr
  def __cinit__(self, param):
    self.thisptr = new Foo[{T}](param)
    # etc

Next, run it through a simple program (may as well be Python) to load the file, do the string substitution, and save the file again under a new name. The key line is just:

output = code.format(T=T) # where T is a string with a C++ class name 
              # e.g. "int" or "std::vector<double>"

(There's obviously a bit of code relating to loading and saving that I've skipped out of laziness)

Then, in your Cython file you just "include" the generated files for each class. The "include" command in Cython is a literal textual include (like the C preprocessor) and expects a .pxi file:

cdef extern from "header.h":
    cdef cppclass Foo[T]:
        Foo(T param)
        # ...

include "foowrapper_int.pxi"
include "foowrapper_vectordouble.pxi
# etc

You have to pick the classes to generate at compile time, but this is unavoidable (templates are a compile-time feature), so you're never going to be able to generate them dynamically from a Python scripting environment since the corresponding C++ class won't be generated.

Other options

A couple of other options are worth brief consideration.

1. You could inherit Foo<T> from a base class (say FooBase) which doesn't depend on the template parameter. You'd then wrap FooBase in Cython (generating constructor-like functions for the cases you care about). This is only really viable if the functions you want to call don't have arguments that depend on the template type. Obviously this also involves changing the C++ code.

   Consider as an example a std::vector-like class. Many of its members don't depend on the template type so could exist in a common base (maybe as pure virtual functions?). The Cython cdef extern might look like:

   cdef extern from "somewhere.h":
       cdef cppclass VectorBase:
           int size()
           void pop_back()
       cdef cppclass Vector[T](VectorBase):
           void push_back(T)
   

   You could then define a base Python class to wrap this

   cdef class PyVectorBase:
       cdef VectorBase* vb
       def size(self):
           return self.vb.size()
       def pop_back(self):
           self.vb.pop_back()
   

   and specific derived Python classes for the functions that do depend on the type.

   cdef class PyVectorDouble(PyVectorBase):
       def __cinit__(self):
           self.vb = new Vector[double]()
       def push_back(self, value):
           cdef Vector[double]* vd = <Vector[double]*>(self.vb)  # cast is OK because we constructed it...
           vd.push_back(value)
   

   depending on how many "template-dependent" parameters there are this could save significant duplication.

2. Look at a different way of wrapping. Boost Python will certainly support this natively (but comes with its own disadvantages). I imagine SIP/SWIG would also cope (but I don't know). You can could fairly cleanly mix-and-match these with Cython if necessary (by importing the generated module containing your template classes).