Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Efficient way to convert numpy record array to a list of dictionary

Tags:

python

numpy

How do I convert the numpy record array below:

recs = [('Bill', 31, 260.0), ('Fred', 15, 145.0)]
r = rec.fromrecords(recs, names='name, age, weight', formats='S30, i2, f4')

to a list of dictionary like:

[{'name': 'Bill', 'age': 31, 'weight': 260.0}, 
'name': 'Fred', 'age': 15, 'weight': 145.0}]
like image 269
Vishal Avatar asked Feb 04 '10 22:02

Vishal


People also ask

Can you convert NumPy array to list in Python?

We can convert the Numpy array to the list by tolist() method, we can have a list of data element which is converted from an array using this method.

Can I store a NumPy array in a dictionary?

np. save is designed to save numpy arrays. data is a dictionary. So it wrapped it in a object array, and used pickle to save that object.

Is NumPy array more efficient than list?

Because the Numpy array is densely packed in memory due to its homogeneous type, it also frees the memory faster. So overall a task executed in Numpy is around 5 to 100 times faster than the standard python list, which is a significant leap in terms of speed.


3 Answers

I am not sure there is built-in function for that or not, but following could do the work.

>>> [dict(zip(r.dtype.names,x)) for x  in r]
[{'age': 31, 'name': 'Bill', 'weight': 260.0}, 
{'age': 15, 'name': 'Fred', 'weight': 145.0}]
like image 154
YOU Avatar answered Oct 05 '22 13:10

YOU


This depends on the final structure required. This example shows a numpy recarray composed of several 1D subsets. To have a python dictionary instead, a possible conversion is:

import numpy as np

a = np.rec.array([np.array([1,3,6]), np.array([3.4,4.2,-1.2])], names=['t', 'x'])

b = {name:a[name] for name in a.dtype.names}
like image 22
cvr Avatar answered Oct 05 '22 14:10

cvr


Here's a solution that works in the following cases not covered by the other answers:

  • 0-dimensional arrays (scalars). e.g. np.array((1, 2), dtype=[('a', 'float32'), ('b', 'float32')])
  • elements of type np.void (result of indexing a record array)
  • multi-dimensional arrays of structs
  • structs containing structs, (e.g. structured_to_dict(np.zeros((), dtype=[('a', [('b', 'float32', (2,))])])))
  • Combinations of any of the above.
def structured_to_dict(arr: np.ndarray):
    import numpy as np

    if np.ndim(arr) == 0:
        if arr.dtype.names == None:
            return arr.item()
        # accessing by int does *not* work when arr is a zero-dimensional array!
        return {k: structured_to_dict(arr[k]) for k in arr.dtype.names}
    return [structured_to_dict(v) for v in arr]
like image 23
phiresky Avatar answered Oct 05 '22 15:10

phiresky