SWIG wrapped vector of vectors (C++ to python) - how to recognise the inner vector as a proxy object?

Question

I'm facing a similar issue to Wrap std::vector of std::vectors, C++ SWIG Python - but it's not just simple C++ parsing. I have the following in my C++ code

namespace ns {
    typedef unsigned long long uint64_t;
    typedef std::vector<uint64_t> Vector;
    typedef std::vector<Vector> VectorOfVectors;

    class MyClass {
        /// ...

        /// Returns a reference to the internal vector allocated in C++ land
        const VectorOfVectors &GetVectors() const;
    };
}

And in the SWIG wrapper

%module myswig    
// ...
%template(Uint64V) std::vector<ns::uint64_t>;
%template(VUint64V) std::vector<std::vector<ns::uint64_t> >;

So the wrapping works fine, including the class, and I can retrieve the class's vector of vectors OK:

import myswig
m = myswig.MyClass()
v = m.GetVectors()
print v

Which gives me:

<myswig.VUint64V; proxy of <Swig Object of type 'std::vector< std::vector< ns::uint64_t,std::allocator< ns::uint64_t > > > *' at 0x994a050> >

But if I access an element in the vector, I don't get a proxy which is a myswig.Uint64V - and this is my problem.

x = v[0]
print x

What I'd hope to get is:

<myswig.Uint64V; proxy of <Swig Object of type 'std::vector< ns::uint64_t, std::allocator< ns::uint64_t > > *' at 0x994a080> >

Instead, I'm getting:

(<Swig Object of type 'ns::uint64_t *' at 0x994a080>, <Swig Object of type 'ns::uint64_t *' at 0x994a098>) 

That is, the index into the vector of vectors is giving me a 2-entry tuple, and not a proxy to the vector class that I need (so that accessing the internal vector is as easy as accessing other vectors).

I'm also getting the warning:

swig/python detected a memory leak of type 'ns::uint64_t *', no destructor found.

because of course there isn't a destructor defined for this type.

Any ideas?

Answer 1

I worked on this with a colleague of mine, and we have managed to come up with some solutions.

First of all, in the SWIG .i file, it's important to define this preprocessor variable:

%{
#   define SWIG_PYTHON_EXTRA_NATIVE_CONTAINERS 
%}

And then, to ensure that the references returned from methods such as front(), back(), operator[], etc are actually mapped to the correct proxy type for the internal vector, the following typemaps help:

// In pop()
%typemap(out) std::vector<std::vector<ns::uint64_t> >::value_type { 
$result = SWIG_NewPointerObj(SWIG_as_voidptr(&$1), $descriptor(std::vector<ns::uint64_t>), 0 |  0 ); 
} 

// In front(), back(), __getitem__()
%typemap(out) std::vector<std::vector<ns::uint64_t> >::value_type & { 
    $result = SWIG_NewPointerObj(SWIG_as_voidptr($1), $descriptor(std::vector<ns::uint64_t>), 0 |  0 ); 
} 

We also discovered that if you want the ns::uint64_t to be treated as a python long variable (equivalent to a C unsigned long long) then a few further typemaps were required to ensure the vector methods using values and references would instead just use 64-bit integer values.

// In __getitem__()
%typemap(out) ns::uint64_t {
    $result = PyLong_FromUnsignedLongLong($1);
}
// Not used (but probably useful to have, just in case)
%typemap(in) ns::uint64_t {
    $1 = PyLong_AsUnsignedLongLong($input);
}
// In pop()
%typemap(out) std::vector<ns::uint64_t>::value_type {
    $result = PyLong_FromUnsignedLongLong($1);
}
// In __getitem__(), front(), back()
%typemap(out) std::vector<ns::uint64_t>::value_type & {
    $result = PyLong_FromUnsignedLongLong(*$1);
}
// In __setitem__(), append(), new Uint64Vector, push_back(), assign(), resize(), insert()
// This allows a python long literal number to be used as a parameter to the above methods. 
// Note the use of a local variable declared at the SWIG wrapper function scope,
// by placing the variable declaration in parentheses () prior to the open brace {
%typemap(in) std::vector<ns::uint64_t>::value_type & (std::vector<ns::uint64_t>::value_type temp) {
    temp = PyLong_AsUnsignedLongLong($input);
    $1 = &temp;
}

I hope this solution helps people in future!