Decode C const char* in Python with ctypes

Question

I am using ctypes (imported as c) in Python 3 to execute a C++ shared library. The library is loaded into python using:

smpLib = c.cdll.LoadLibrary(os.getcwd()+os.sep+'libsmpDyn.so')

One of the functions has the extern 'C' declaration const char* runSmpModel(...). The python function prototype is coded and run as:

proto_SMP = c.CFUNCTYPE(c.c_char_p,...)
runSmpModel = proto_SMP(('runSmpModel',smpLib))
res = runSmpModel(...)

This all works beautifully, but I'm unable to decode the res variable and obtain the string passed out by the C runSmpModel function. The value of res is displayed (I'm using ipython3) as b'\xd0'. The best solution I've found online - res.decode('utf-8') gives me the error:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd0 in position 0: unexpected end of data

The const char* return value from the runSmpModel function comes from

std::string scenID = SMPLib::SMPModel::runModel(...);
return scenID.c_str();

inside runModel, it is ultimately defined as shown here, where scenName is an input string:

auto utcBuffId = newChars(500);
sprintf(utcBuffId, "%s_%u", scenName.c_str(), microSeconds); // catenate scenario name & time
uint64_t scenIdhash = (std::hash < std::string>() (utcBuffId)); // hash it

auto hshCode = newChars(100);
sprintf(hshCode, "%032llX", scenIdhash);
scenId = hshCode;

The value of this specific res should be 0000000000000000BBB00C6CA8B8872E. How can I decode this string?

After a lot of further testing, I've identified the problem as the length of the string passed from the C function. No problems if the string is up to 15 characters in length, but if it's 16 or longer - no dice. For a minimum-working example, the C-code is:

extern "C" {
  const char* testMeSO()
  {
    string scenarioID = "abcdefghijklmnop";
    return scenarioID.c_str();
  }
}

and python code is (same definition of smpLib as shown above):

proto_TST = c.CFUNCTYPE(c.c_char_p)
testMeSO = proto_TST(('testMeSO',smpLib))
res = testMeSO()
print("Scenario ID: %s"%res.decode('utf-8'))

This gives the decode error, unless any character is removed from the scenarioID variable in the C function. So it seems the question is "how can Python read a C char* longer than 15 characters, using ctypes.

Answer 1

After several days of debugging and testing, I've finally gotten this working, using the second solution posted by @Petesh on this SO post. I don't understand why ctypes is apparently limiting the char * value passed from C to 15 characters (+termination = 256 bits?).

Essentially, the solution is to pass into the C function an extra char * buff buffer that has already been created using ctypes.create_string_buffer(32*16), as well as an unsigned int buffsize of value 32*16. Then, in the C function execute scenarioID.copy(buff,buffsize). The python prototype function is modified in an obvious way.