ValueError in numpy.fromstring

Question

I want to convert these string values to numpy array of int16 datatype

import numpy as np
raw=b''
w="\x01\x02 \x01\x02"
w1="\x01\x03 \x04"
p=w.replace(" ", "")
w1=w1.replace(" ","")
raw +=p
raw +=w1
results = np.fromstring(raw, dtype=np.uint16)
print results

I am getting the error as:

>File "prj1.py", line 11, in <module>
> results = np.fromstring(raw, dtype=np.uint16)
>ValueError: string size must be a multiple of element size

How can I convert these strings to numpy arrray with data type as int16?

Answer 1

As the error message states, if fromstring is fed binary input data, the data length must be a multiple of the element size. This is also stated in the documentation. In your case, the element size is 2, because a uint16 is composed of two bytes. However in your second string, w1, you only provide 1 byte. One way to solve this problem is to add a leading zero to the smaller number:

import numpy as np
raw=b''
w="\x01\x02 \x01\x02"
w1="\x01\x03 \x04"
elements=w.split(' ')+w1.split(' ')
raw=b''.join(['\x00'+e if len(e)==1 else e for e in elements ])
results = np.fromstring(raw, dtype=np.uint16)
print results

This outputs:

[ 513  513  769 1024]

For me this result was surprising. Apparently the bytes are read from left to right (smallest to biggest). I don't know if this is platform specific (I'm on osx) or always like this in numpy. Anyway, if your desired byte order is from right to left, you can reverse the order like so:

raw2=b''.join([e+'\x00' if len(e)==1 else e[1]+e[0] for e in elements])
results2 = np.fromstring(raw2, dtype=np.uint16)
print results2

which results in:

[258 258 259   4]