create_string_buffer throwing error TypeError: str/bytes expected instead of str instance

Question

I am trying this simple ctypes example and getting the error mentioned

>>> from ctypes import create_string_buffer
>>> str = create_string_buffer("hello")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python32\lib\ctypes\__init__.py", line 59, in create_string_buffer
buf.value = init
TypeError: str/bytes expected instead of str instance

Does anyone know what am I doing wrong?

On the same note, I am trying to pass a pointer to a string to a C function from my python code so that I can perform some string operation there and return another string. Can someone give me some sample code on how to do that?

extern "C" __declspec(dllexport) char * echo(char* c)
{
      // do stuff
    return c;
}

Answer 1

You're using Python 3, and strings (str-type) in Python 3 are Unicode objects. create_string_buffer creates C char arrays, so you are required to pass a bytes object. If you have a str object encode with an appropriate encoding to create a bytes object. Note there is a create_unicode_buffer that creates C wchar arrays as well and takes Python 3 str objects.

Python 3.2.1 (default, Jul 10 2011, 21:51:15) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes
>>> ctypes.create_unicode_buffer('abcd')
<ctypes.c_wchar_Array_5 object at 0x00C2D300>
>>> ctypes.create_string_buffer(b'abcd')
<ctypes.c_char_Array_5 object at 0x00C2D1C0>
>>> ctypes.create_string_buffer('abcd'.encode('utf-8'))
<ctypes.c_char_Array_5 object at 0x00C2D300>

In regards to the 2nd part of your question, do you want to edit the existing string or return an entirely new string allocated on the heap? If the former, use create_string_buffer to pass a mutable string to the function, then modifying it in place will work. Here's a simple example calling _strupr() from the Windows MSVCRT library:

Python 3.2.1 (default, Jul 10 2011, 21:51:15) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes
>>> c = ctypes.CDLL('msvcr90.dll')
>>> strupr = c._strupr
>>> a=ctypes.create_string_buffer(b'abc')
>>> strupr.restype=ctypes.c_char_p
>>> strupr(a)
b'ABC'

Note setting the restype (result type) of the function to a c_char_p tells ctypes to convert the return value to a bytes object. Also note that _strupr converts a string in place, so a mutable string buffer must be passed. Only pass Python byte strings to C functions that take const char*, since modifying Python strings directly via ctypes is Bad^TM.

Answer 2

With regards to getting it working, if you pass it a bytes object, it works:

>>> import ctypes
>>> ctypes.create_string_buffer(b'hello')
<ctypes.c_char_Array_6 object at 0x25258c0>

Looking at the code for create_string_buffer:

def create_string_buffer(init, size=None):
    """create_string_buffer(aBytes) -> character array
    create_string_buffer(anInteger) -> character array
    create_string_buffer(aString, anInteger) -> character array
    """
    if isinstance(init, (str, bytes)):
        if size is None:
            size = len(init)+1
        buftype = c_char * size
        buf = buftype()
        buf.value = init
        return buf
    elif isinstance(init, int):
        buftype = c_char * init
        buf = buftype()
        return buf
    raise TypeError(init)

Doing it directly,

>>> (ctypes.c_char * 10)().value = b'123456789'

This works fine.

>>> (ctypes.c_char * 10)().value = '123456789'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: str/bytes expected instead of str instance

This shows that same behaviour. Looks to me as though you've found a bug.

Time to visit http://bugs.python.org. There are a few bugs related to c_char and create_string_buffer which are in the same field, but none reporting that giving it a str now fails (but there are definite examples showing it used to work in Py3K).