Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to convert numpy arrays to vector<int>& (reference) with SWIG

My goal:

Create 3 numpy arrays in python (2 of them will be initialized with specific values), then send all three of them through swig into a c++ function as vector references (this is in order to avoid copying over the data and loosing efficiency). Once in the c++ function, add 2 of the arrays and put their sum in the 3rd array.

vec_ref.h

#include <vector>
#include <iostream>

void add_vec_ref(std::vector<int>& dst, std::vector<int>& src1, std::vector<int>& src2);

vec_ref.cpp

#include "vec_ref.h"
#include <cstring> // need for size_t
#include <cassert>

void add_vec_ref(std::vector<int>& dst, std::vector<int>& src1, std::vector<int>& src2) {
    std::cout << "inside add_vec_ref" << std::endl;
    assert(src1.size() == src2.size());
    dst.resize(src1.size());

    for (size_t i = 0; i < src1.size(); i++) {
        dst[i] = src1[i] + src2[i];
    }
}

vec_ref.i

%module vec_ref
%{
    #define SWIG_FILE_WITH_INIT
    #include "vec_ref.h"
%}

%include "numpy.i"
%init %{
import_array();
%}

%include "std_vector.i"
%template(vecInt) std::vector<int>;
// %template(vecIntRef) std::vector<int> &; 

// %apply (std::vector<int> * INPLACE_ARRAY1, int DIM1) {(std::vector<int> * dst, int a),(std::vector<int> * src1, int b),(std::vector<int> * src2, int c)};
// %apply (std::vector<int> * INPLACE_ARRAY1) {(std::vector<int> * dst),(std::vector<int> * src1),(std::vector<int> * src2)};
// %apply (std::vector<int> & INPLACE_ARRAY1) {(std::vector<int> & dst),(std::vector<int> & src1),(std::vector<int> & src2)};
// %apply (std::vector<int> & INPLACE_ARRAY1, int DIM1) {(std::vector<int> & dst, int a),(std::vector<int> & src1, int b),(std::vector<int> & src2, int c)};

%include "vec_ref.h"

Makefile

all:
    rm -f *.so *.o *_wrap.* *.pyc *.gch vec_ref.py
    swig -c++ -python vec_ref.i
    g++ -O0 -g3 -fpic -c vec_ref_wrap.cxx vec_ref.h vec_ref.cpp -I/home/lmckeereid/tools/anaconda3/pkgs/python-3.7.3-h0371630_0/include/python3.7m/
    g++ -O0 -g3 -shared vec_ref_wrap.o vec_ref.o -o _vec_ref.so

tester.py

import vec_ref as vec
import numpy as np

a = np.array([1,2,3], dtype=np.intc)
b = np.array([4,5,6], dtype=np.intc)
c = np.zeros(len(a), dtype=np.intc)

print('---Before---\na:', a)
print('b:', b)
print('c:', c)

vec.add_vec_ref(c,a,b)

print('---After---\na:', a)
print('b:', b)
print('c:', c)

Output:

---Before---
a: [1 2 3]
b: [4 5 6]
c: [0 0 0]
Traceback (most recent call last):
  File "tester.py", line 12, in <module>
    vec.add_vec_ref(c,a,b)
TypeError: in method 'add_vec_ref', argument 1 of type 'std::vector< int,std::allocator< int > > &'

I've tried all the commented out %apply and %template directives found in vec_ref.i, but they didn't work.

Are there some typemaps that I should be including that I'm not?

like image 838
Otherness Avatar asked Oct 21 '19 22:10

Otherness


People also ask

Is a NumPy array a vector?

The NumPy ndarray class is used to represent both matrices and vectors. A vector is an array with a single dimension (there's no difference between row and column vectors), while a matrix refers to an array with two dimensions. For 3-D or higher dimensional arrays, the term tensor is also commonly used.

How do I change the Dtype of a NumPy array?

In order to change the dtype of the given array object, we will use numpy. astype() function. The function takes an argument which is the target data type. The function supports all the generic types and built-in types of data.


1 Answers

I agree with @pschill: it’s not possible to get an std::vector without copying data.

One alternative is to use the std::span class template (introduced in C++20), or a similar span class template defined in a library.

Creating a std::span<int> would provide a view of existing data in a numpy array, and provide many convenient member functions (such as operator[], iterators, front(), back(), etc.) in C++.

Creating a span would never copy data from the numpy array.

like image 137
NicholasM Avatar answered Oct 12 '22 22:10

NicholasM