I have a little project that works beautifully with SWIG. In particular, some of my functions return std::vector
s, which get translated to tuples in Python. Now, I do a lot of numerics, so I just have SWIG convert these to numpy arrays after they're returned from the c++ code. To do this, I use something like the following in SWIG.
%feature("pythonappend") My::Cool::Namespace::Data() const %{ if isinstance(val, tuple) : val = numpy.array(val) %}
(Actually, there are several functions named Data, some of which return floats, which is why I check that val
is actually a tuple.) This works just beautifully.
But, I'd also like to use the -builtin
flag that's now available. Calls to these Data functions are rare and mostly interactive, so their slowness is not a problem, but there are other slow loops that speed up significantly with the builtin option.
The problem is that when I use that flag, the pythonappend feature is silently ignored. Now, Data just returns a tuple again. Is there any way I could still return numpy arrays? I tried using typemaps, but it turned into a giant mess.
Borealid has answered the question very nicely. Just for completeness, I include a couple related but subtly different typemaps that I need because I return by const reference and I use vectors of vectors (don't start!). These are different enough that I wouldn't want anyone else stumbling around trying to figure out the minor differences.
%typemap(out) std::vector<int>& { npy_intp result_size = $1->size(); npy_intp dims[1] = { result_size }; PyArrayObject* npy_arr = (PyArrayObject*)PyArray_SimpleNew(1, dims, NPY_INT); int* dat = (int*) PyArray_DATA(npy_arr); for (size_t i = 0; i < result_size; ++i) { dat[i] = (*$1)[i]; } $result = PyArray_Return(npy_arr); } %typemap(out) std::vector<std::vector<int> >& { npy_intp result_size = $1->size(); npy_intp result_size2 = (result_size>0 ? (*$1)[0].size() : 0); npy_intp dims[2] = { result_size, result_size2 }; PyArrayObject* npy_arr = (PyArrayObject*)PyArray_SimpleNew(2, dims, NPY_INT); int* dat = (int*) PyArray_DATA(npy_arr); for (size_t i = 0; i < result_size; ++i) { for (size_t j = 0; j < result_size2; ++j) { dat[i*result_size2+j] = (*$1)[i][j]; } } $result = PyArray_Return(npy_arr); }
Though not quite what I was looking for, similar problems may also be solved using @MONK's approach (explained here).
A SwigPyObject object stores an instance of a C++ object: When -builtin is used, the pure python layer is stripped off. Each wrapped class is turned into a new python built-in type which inherits from SwigPyObject , and SwigPyObject instances are returned directly from the wrapped methods.
In a nutshell, SWIG is a compiler that takes C/C++ declarations and creates a wrapper needed to access those declarations from other languages like Python, Tcl, Ruby etc. It normally required no changes in existing code and create an interface within a minute. Building interpreted interface for existing C programs.
I agree with you that using typemap
gets a little messy, but it is the right way to accomplish this task. You are also right that the SWIG documentation does not directly say that %pythonappend
is incompatible with -builtin
, but it is strongly implied: %pythonappend
adds to the Python proxy class, and the Python proxy class does not exist at all in conjunction with the -builtin
flag.
Before, what you were doing was having SWIG convert the C++ std::vector
objects into Python tuples, and then passing those tuples back down to numpy
- where they were converted again.
What you really want to do is convert them once, at the C level.
Here's some code which will turn all std::vector<int>
objects into NumPy integer arrays:
%{ #include "numpy/arrayobject.h" %} %init %{ import_array(); %} %typemap(out) std::vector<int> { npy_intp result_size = $1.size(); npy_intp dims[1] = { result_size }; PyArrayObject* npy_arr = (PyArrayObject*)PyArray_SimpleNew(1, dims, NPY_INT); int* dat = (int*) PyArray_DATA(npy_arr); for (size_t i = 0; i < result_size; ++i) { dat[i] = $1[i]; } $result = PyArray_Return(npy_arr); }
This uses the C-level numpy functions to construct and return an array. In order, it:
arrayobject.h
file is included in the C++ output fileimport_array
to be called when the Python module is loaded (otherwise, all NumPy methods will segfault)std::vector<int>
into NumPy arrays with a typemap
This code should be placed before you %import
the headers which contain the functions returning std::vector<int>
. Other than that restriction, it's entirely self-contained, so it shouldn't add too much subjective "mess" to your codebase.
If you need other vector types, you can just change the NPY_INT
and all the int*
and int
bits, otherwise duplicating the function above.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With