Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to handle C++ return type std::vector<int> in Python ctypes?

Tags:

c++

python

ctypes

I cannot find how ctypes will bridge the gap between std::vector and Python; no where on the internet is the combination mentioned. Is this bad practice, does it not exist or am I missing something?

C++ : xxx.cpp

#include <fstream>
#include <string>
using namespace std;
extern "C" std::vector<int> foo(const char* FILE_NAME)
{
    string line;
    std::vector<int> result;
    ifstream myfile(FILE_NAME);
    while (getline(myfile, line)) {
      result.push_back(1);
    }

    return(result);
}

Python: xxx.py

import ctypes
xxx = ctypes.CDLL("./libxxx.so")
xxx.foo.argtypes = ??????
xxx.foo.restype = ??????
like image 325
PascalVKooten Avatar asked Jun 02 '13 17:06

PascalVKooten


People also ask

What is Ctypes module in Python?

ctypes is a foreign function library for Python. It provides C compatible data types, and allows calling functions in DLLs or shared libraries. It can be used to wrap these libraries in pure Python.


1 Answers

Whether or not this approach actually provides faster execution time, I'll explain a bit about how you could go about doing it. Basically, create a pointer to a C++ vector which can interface with Python through C functions. You can then wrap the C++ code in a Python class, hiding the implementation details of ctypes.

I included what I thought would be helpful magic methods to include in the Python class. You can choose to remove them or add more to suit your needs. The destructor is important to keep though.

C++

// vector_python.cpp
#include <vector>
#include <iostream>
#include <fstream>
#include <string>

using namespace std;

extern "C" void foo(vector<int>* v, const char* FILE_NAME){
    string line;
    ifstream myfile(FILE_NAME);
    while (getline(myfile, line)) v->push_back(1);
}

extern "C" {
    vector<int>* new_vector(){
        return new vector<int>;
    }
    void delete_vector(vector<int>* v){
        cout << "destructor called in C++ for " << v << endl;
        delete v;
    }
    int vector_size(vector<int>* v){
        return v->size();
    }
    int vector_get(vector<int>* v, int i){
        return v->at(i);
    }
    void vector_push_back(vector<int>* v, int i){
        v->push_back(i);
    }
}

Compile it as a shared library. On Mac OS X this might look like,

g++ -c -fPIC vector_python.cpp -o vector_python.o
g++ -shared -Wl,-install_name,vector_python_lib.so -o vector_python_lib.so vector_python.o

Python

from ctypes import *

class Vector(object):
    lib = cdll.LoadLibrary('vector_python_lib.so') # class level loading lib
    lib.new_vector.restype = c_void_p
    lib.new_vector.argtypes = []
    lib.delete_vector.restype = None
    lib.delete_vector.argtypes = [c_void_p]
    lib.vector_size.restype = c_int
    lib.vector_size.argtypes = [c_void_p]
    lib.vector_get.restype = c_int
    lib.vector_get.argtypes = [c_void_p, c_int]
    lib.vector_push_back.restype = None
    lib.vector_push_back.argtypes = [c_void_p, c_int]
    lib.foo.restype = None
    lib.foo.argtypes = [c_void_p]

    def __init__(self):
        self.vector = Vector.lib.new_vector()  # pointer to new vector

    def __del__(self):  # when reference count hits 0 in Python,
        Vector.lib.delete_vector(self.vector)  # call C++ vector destructor

    def __len__(self):
        return Vector.lib.vector_size(self.vector)

    def __getitem__(self, i):  # access elements in vector at index
        if 0 <= i < len(self):
            return Vector.lib.vector_get(self.vector, c_int(i))
        raise IndexError('Vector index out of range')

    def __repr__(self):
        return '[{}]'.format(', '.join(str(self[i]) for i in range(len(self))))

    def push(self, i):  # push calls vector's push_back
        Vector.lib.vector_push_back(self.vector, c_int(i))

    def foo(self, filename):  # foo in Python calls foo in C++
        Vector.lib.foo(self.vector, c_char_p(filename))

You can then test it out in the interpreter (file.txt just consists of three lines of jibberish).

>>> from vector import Vector
>>> a = Vector()
>>> a.push(22)
>>> a.push(88)
>>> a
[22, 88]
>>> a[1]
88
>>> a[2]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "vector.py", line 30, in __getitem__
    raise IndexError('Vector index out of range')
IndexError: Vector index out of range
>>> a.foo('file.txt')
>>> a
[22, 88, 1, 1, 1]
>>> b = Vector()
>>> ^D
destructor called in C++ for 0x1003884d0
destructor called in C++ for 0x10039df10
like image 74
Jared Avatar answered Nov 13 '22 13:11

Jared