Numpy np.fromstring() not working as hoped

Question

I have saved an array in a text file as a string hoping I would be able to convert it back into an array when read from file:

str_arr = "[0.01 0.01 0.01 0.01 0.01 0.01]"
num_arr = np.fromstring(str_arr,dtype = np.float64 ,count = 6,sep = ' ')

the result of whic is num_array:

array([-1.00000000e+000,  6.94819184e-310,  6.94819184e-310,
        6.94818751e-310,  6.94818751e-310,  6.94818751e-310])

I was expecting an array of 0.01s

Answer 1

It seems that np.fromstring does not know how to interpret the brackets. You could solve that by stripping them before calling the function:

a = "[0.01 0.01 0.01 0.01 0.01 0.01]"
num_arr = np.fromstring(a.strip('[]'), count = 6, sep = ' ')

array([0.01, 0.01, 0.01, 0.01, 0.01, 0.01])

Also note that, dtype defaults to float, so there's no need to specify it in this case.

Answer 2

Most likely, you saved your array in the file using str. That's wrong.
Although in this case, the error is not visible, for larger arrays, it will become clear that the value saved this way will yield an incorrect buffer. Check [SO]: ValueError: sequence too large; cannot be greater than 32 (@CristiFati's answer) for more details.

Although there are trivial ways to get past the current situation (extra processing on the existing string), they would only be workarounds (gainarii), while the proper way (or, at least one of them) to solve would be to correctly serialize the array (using [SciPy.Docs]: numpy.ndarray.tostring) when saving it to file.

>>> arr = np.array([0.01, 0.01, 0.01, 0.01, 0.01, 0.01])
>>> arr
array([0.01, 0.01, 0.01, 0.01, 0.01, 0.01])
>>>
>>> str_arr0 = str(arr)
>>> str_arr0
'[0.01 0.01 0.01 0.01 0.01 0.01]'
>>> str_arr1 = arr.tostring()
>>> str_arr1
b'{\x14\xaeG\xe1z\x84?{\x14\xaeG\xe1z\x84?{\x14\xaeG\xe1z\x84?{\x14\xaeG\xe1z\x84?{\x14\xaeG\xe1z\x84?{\x14\xaeG\xe1z\x84?'
>>>
>>> arr_final = np.fromstring(str_arr1, dtype=np.float64)
>>> arr_final
array([0.01, 0.01, 0.01, 0.01, 0.01, 0.01])
>>>
>>> arr_final_wrong = np.fromstring(str_arr0[1:-1], dtype=np.float64, count=6, sep= " ")
>>> arr_final_wrong
array([0.01, 0.01, 0.01, 0.01, 0.01, 0.01])
>>>
>>> arr = np.array([0.01, 0.01, 0.01, 0.01, 0.01, 0.01] * 10)
>>> # This time, str(arr) will produce an invalid result
...
>>> str(arr)
'[0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01\n 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01\n 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01\n 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01\n 0.01 0.01 0.01 0.01]'