Numpy dtype for list with mixed data types

Question

I have a list, my_list, with mixed data types that I want to convert into a numpy array. However, I get the error TypeError: expected a readable buffer object. See code below. I've tried to base my code on the NumPy documentation.

my_list = [['User_0', '2012-2', 1, 6, 0, 1.0], ['User_0', '2012-2', 5, 6, 0, 1.0], ['User_0', '2012-3', 0, 0, 4, 1.0]]
my_np_array = np.array(my_list, dtype='S30, S8, i4, i4, f32')   

Answer 1

Why don't use dtype=object?

In [1]: my_list = [['User_0', '2012-2', 1, 6, 0, 1.0], ['User_0', '2012-2', 5,
6, 0, 1.0], ['User_0', '2012-3', 0, 0, 4, 1.0]]
In [2]: my_np_array = np.array(my_list, dtype=object)
In [3]: my_np_array
Out[3]:
array([['User_0', '2012-2', 1, 6, 0, 1.0],
       ['User_0', '2012-2', 5, 6, 0, 1.0],
       ['User_0', '2012-3', 0, 0, 4, 1.0]], dtype=object)

Note It's about memory usage, when you specify the dtype of each column, memory allocated to your ndarray will be less than when you use dtype=object which contain all possible type in python so the memory allocated for each column will be maximal.

Answer 2

Your nested items should be tuple also you omitted one i4 in your types :

>>> my_np_array = np.array(map(tuple,my_list), dtype='|S30, |S8, i4, i4, i4, f32')  
>>> my_np_array
array([('User_0', '2012-2', 1, 6, 0, 1.0),
       ('User_0', '2012-2', 5, 6, 0, 1.0),
       ('User_0', '2012-3', 0, 0, 4, 1.0)], 
      dtype=[('f0', 'S30'), ('f1', 'S8'), ('f2', '<i4'), ('f3', '<i4'), ('f4', '<i4'), ('f5', '<f4')])

As far as is know since numpy use tuples to preserve its types when you used multiple type for array items you need to convert your sub arrays to tuple like dtype elements.