CPPYY/CTYPES passing array of strings as char* args[]

Question

I only recently started using cppyy and ctypes, so this may be a bit of a silly question. I have the following C++ function:

float method(const char* args[]) {
    ...
}

and from Python I want to pass args as a list of strings, i.e.:

args = *magic*
x = cppyy.gbl.method(args)

I have previously found this, so I used

def setParameters(strParamList):
    numParams    = len(strParamList)
    strArrayType = ct.c_char_p * numParams
    strArray     = strArrayType()
    for i, param in enumerate(strParamList):
        strArray[i] = param
    lib.SetParams(numParams, strArray)

and from Python:

args = setParameters([b'hello', b'world'])

c_types.c_char_p expects a bytes array. However, when calling x = cppyy.gbl.method(args) I get

TypeError: could not convert argument 1 (could not convert argument to buffer or nullptr)

I'm not entirely sure why this would be wrong since the args is a <__main__.c_char_p_Array_2> object, which I believe should be converted to a const char* args[].

Answer 1

For the sake of having a concrete example, I'll use this as the .cpp file:

#include <cstdlib>

extern "C"
float method(const char* args[]) {
    float sum = 0.0f;
    const char **p = args;
    while(*p) {
        sum += std::atof(*p++);
    }
    return sum;
}

And I'll assume it was compiled with g++ method.cpp -fPIC -shared -o method.so. Given those assumptions, here's an example of how you could use it from Python:

#!/usr/bin/env python3

from ctypes import *

lib = CDLL("./method.so")
lib.method.restype = c_float
lib.method.argtypes = (POINTER(c_char_p),)

def method(args):
    return lib.method((c_char_p * (len(args) + 1))(*args))

print(method([b'1.23', b'45.6']))

We make a C array to hold the Python arguments. len(args) + 1 makes sure there's room for the null pointer sentinel.

Answer 2

ctypes does not have a public API that is usable from C/C++ for extension writers, so the handling of ctypes by cppyy is by necessity somewhat clunky. What's going wrong, is that the generated ctypes array of const char* is of type const char*[2] not const char*[] and since cppyy does a direct type match for ctypes types, that fails.

As-is, some code somewhere needs to do a conversion of the Python strings to low-level C ones, and hold on to that memory for the duration of the call. Me, personally, I'd use a little C++ wrapper, rather than having to think things through on the Python side. The point being that an std::vector<std::string> can deal with the necessary conversions (so no bytes type needed, for example, but of course allowed if you want to) and it can hold the temporary memory.

So, if you're given some 3rd party interface like this (putting it inline for cppyy only for the sake of the example):

import cppyy

cppyy.cppdef("""
    float method(const char* args[], int len) {
        for (int i = 0; i < len; ++i)
            std::cerr << args[i] << " ";
        std::cerr << std::endl;
        return 42.f;
    }
""")

Then I'd generate a wrapper:

# write a C++ wrapper to hide C code
cppyy.cppdef("""
    namespace MyCppAPI {
       float method(const std::vector<std::string>& args) {
           std::vector<const char*> v;
           v.reserve(args.size());
           for (auto& s : args) v.push_back(s.c_str());
           return ::method(v.data(), v.size());
       }
    }
""")

Then replace the original C API with the C++ version:

# replace C version with C++ one for all Python users
cppyy.gbl.method = cppyy.gbl.MyCppAPI.method

and things will be as expected for any other person downstream:

# now use it as expected
cppyy.gbl.method(["aap", "noot", "mies"])

All that said, obviously there is no reason why cppyy couldn't do this bit of wrapping automatically. I created this issue: https://bitbucket.org/wlav/cppyy/issues/235/automatically-convert-python-tuple-of