Convert ctype byte array to bytes

Question

I have a ctypes array that I defined like this:

buff= (c_ubyte*buff_size)()

After filling the buffer up with data, I need to have this data in bytes format. Right now I'm doing it as follows:

buff= [n for n in buff]
buff = ''.join(map(chr, buff))

The problem with this is that it converts it to a 4-byte (or whatever number of bytes) int before converting it back into a single byte string, which wastes a lot of CPU.

How can I convert the ctypes buffer into bytes directly? I'm not trying to save myself a copy because I do have to do a copy anyway since I cannot keep the original buffer. Does python have a cast feature for such things?

Thanks.

Answer 1

If you did want a copy, you could use bytearray:

>>> buff = (c_ubyte * 4)(*[97,98,99,100])
>>> bs = bytearray(buff)
>>> bs
bytearray(b'abcd')
>>> str(bs)
'abcd'

Edit:

For Python versions prior to 2.6 that lack bytearray, you can use one of the following instead:

• cast(buff, c_char_p).value
• buffer(buff)[:]

If you want to share the same buffer, you can create a c_char array:

>>> buff2 = (c_char * len(buff)).from_buffer(buff)
>>> buff2.value # string copy
'abcd'
>>> buff2[:] = 'efgh'
>>> buff[:]  # modified original
[101, 102, 103, 104]

Edit:

The from_buffer class method was added in 2.6. In prior versions you can use cast:

• buff2 = cast(buff, POINTER(c_char * len(buff)))[0]

Is there a reason you aren't using a c_char array to begin with? I understand if you need to work with it as both a numeric array and as a string.

Addendum:

The 2nd method is more 'cast' like since it doesn't copy the buffer. With the first approach it gets copied twice, once to make the bytearray and again to make the str (bytes is an alias for str in 2.x). But a bytearray has string methods and may be all you need; it's basically a mutable version of 3.x bytes.

c_char is the C char type. Multiplied to an array, it's a mutable buffer of bytes like your current c_ubyte array. However, it may be more convenient than c_ubyte since it has the value and raw descriptors that return Python byte strings. It also indexes and iterates as single character byte strings instead of integers.

What you're not supposed to do is create a c_char_p -- a pointer to character data -- from a Python string if the function will modify the it. Python strings objects are immutable; you can get weird bugs if you modify their buffer. I recently answered a question on that topic.