Save & Retrieve Numpy Array From String

Question

I would like to convert a multi-dimensional Numpy array into a string and, later, convert that string back into an equivalent Numpy array.

I do not want to save the Numpy array to a file (e.g. via the savetxt and loadtxt interface).

Is this possible?

Answer 1

You could use np.tostring and np.fromstring:

In [138]: x = np.arange(12).reshape(3,4)

In [139]: x.tostring()
Out[139]: '\x00\x00\x00\x00\x01\x00\x00\x00\x02\x00\x00\x00\x03\x00\x00\x00\x04\x00\x00\x00\x05\x00\x00\x00\x06\x00\x00\x00\x07\x00\x00\x00\x08\x00\x00\x00\t\x00\x00\x00\n\x00\x00\x00\x0b\x00\x00\x00'

In [140]: np.fromstring(x.tostring(), dtype=x.dtype).reshape(x.shape)
Out[140]: 
array([[ 0,  1,  2,  3],
       [ 4,  5,  6,  7],
       [ 8,  9, 10, 11]])

Note that the string returned by tostring does not save the dtype nor the shape of the original array. You have to re-supply those yourself.

---

Another option is to use np.save or np.savez or np.savez_compressed to write to a io.BytesIO object (instead of a file):

import numpy as np
import io

x = np.arange(12).reshape(3,4)
output = io.BytesIO()
np.savez(output, x=x)

The string is given by

content = output.getvalue()

And given the string, you can load it back into an array using np.load:

data = np.load(io.BytesIO(content))
x = data['x']

This method stores the dtype and shape as well.

For large arrays, np.savez_compressed will give you the smallest string.

---

Similarly, you could use np.savetxt and np.loadtxt:

import numpy as np
import io

x = np.arange(12).reshape(3,4)
output = io.BytesIO()
np.savetxt(output, x)
content = output.getvalue()
# '0.000000000000000000e+00 1.000000000000000000e+00 2.000000000000000000e+00 3.000000000000000000e+00\n4.000000000000000000e+00 5.000000000000000000e+00 6.000000000000000000e+00 7.000000000000000000e+00\n8.000000000000000000e+00 9.000000000000000000e+00 1.000000000000000000e+01 1.100000000000000000e+01\n'

x = np.loadtxt(io.BytesIO(content))
print(x)

---

Summary:

• tostring gives you the underlying data as a string, with no dtype or shape
• save is like tostring except it also saves dtype and shape (.npy format)
• savez saves the array in npz format (uncompressed)
• savez_compressed saves the array in compressed npz format
• savetxt formats the array in a humanly readable format

Answer 2

If you want to save the dtype as well you can also use the pickle module from python.

import pickle
import numpy as np

a = np.ones(4)
string = pickle.dumps(a)
pickle.loads(string)