ctypes how to pass string from python to c++ function, and how to return string from c++ function to python

Question

I want to call a c++ function from python, this c++ function takes char* as parameter, and return string. Below is my code.

wrapper.cpp

#include <Python.h>
#include <string>
#include <iostream>

using namespace std;

extern "C"
string return_string(char* name){
    cout<<strlen(name)<<endl;
    cout<<name<<endl;
    string s = "hello ";
    s += name;
    return s;
}

compile wrapper.cpp to example.so

g++ -fPIC wrapper.cpp -o example.so -shared -I/usr/include/python2.7/

wrapper.py

import os
from ctypes import *

lib = cdll.LoadLibrary('./example.so')
lib.return_string.restype = c_char_p
lib.return_string.argtypes = [c_char_p]
name = create_string_buffer("Tom")
s = lib.return_string(name);
print s
print name

here is my output

18
��H�L�l���A���
1
<ctypes.c_char_Array_4 object at 0x7f5f480be710>

How to make it works?

Answer 1

This has nothing to do with ctypes; your C++ code is invalid in itself. You can't define an extern "C" function that returns a string.

In a quick test with a C++ program that uses the same library, it also prints garbage.

I also wrote a C program that defines something called string with the same layout as std::string just so I could compile it and see what happens; it also prints garbage, and then it segfaults in ~string.

So, it's not surprising that the Python program also prints garbage.

With a minor change, everything works:

extern "C"
const char *return_string(char* name){
    cout<<strlen(name)<<endl;
    cout<<name<<endl;
    static string s = "hello ";
    s += name;
    return s.c_str();
}

I get this output:

3
Tom
hello Tom
<ctypes.c_char_Array_4 object at 0x11011c7a0>

(Or, from the C++ version, the same thing but with "Tom" in the last line.)

Of course for obvious reasons this isn't a very good solution, but it shows that returning string is the problem.

Both g++-4.5 and clang-apple-4.0 warned me about exactly this problem when I tried to compile your C++ code (although g++-apple-4.2 didn't, unless I added an extra -W flag). When the compiler gives you a warning, that's often the answer to "why does my code do the wrong thing even though it compiles".

A few other things wrong with your code:

• You don't make any use of anything from Python in your .cpp file. And in general, the whole point of using ctypes is so your C or C++ code doesn't have to know anything about Python; it's your Python code that knows about it. So, don't include or link.
• It's generally a bad idea to take a non-const char* if you aren't planning to modify it. My C++ driver had to call it with const_cast<char*>(name.c_str()) instead of just name.c_str(). Also, this can prevent the compiler from noticing other things that you're doing.

Here's the C++ driver I mentioned above:

#include <iostream>
#include <string>
using namespace std;

extern "C" string return_string(char* name);

int main(int argc, char *argv[]) {
  string name("Tom");
  string s(return_string(const_cast<char *>(name.c_str())));
  cout << s << "\n";
  cout << name << "\n";
  return 0;
}

Also, if I play around with different optimization settings or reorganize the code a bit, in my C++ driver, sometimes your code actually works, sometimes it prints garbage, and sometimes it segfaults. My first guess would be that it depends where the ~string call gets inlined—but really, the details don't matter; the code shouldn't work, and it doesn't, so who cares why?