how to get a 2d numpy array from a pandas dataframe? - wrong shape

Question

I want to get a 2d-numpy array from a column of a pandas dataframe df having a numpy vector in each row. But if I do

df.values.shape

I get: (3,) instead of getting: (3,5)

(assuming that each numpy vector in the dataframe has 5 dimensions, and that the dataframe has 3 rows)

what is the correct method?

Answer 1

Ideally, avoid getting into this situation by finding a different way to define the DataFrame in the first place. However, if your DataFrame looks like this:

s = pd.Series([np.random.randint(20, size=(5,)) for i in range(3)])
df = pd.DataFrame(s, columns=['foo'])
#                   foo
# 0   [4, 14, 9, 16, 5]
# 1  [16, 16, 5, 4, 19]
# 2  [7, 10, 15, 13, 2]

then you could convert it to a DataFrame of shape (3,5) by calling pd.DataFrame on a list of arrays:

pd.DataFrame(df['foo'].tolist())
#     0   1   2   3   4
# 0   4  14   9  16   5
# 1  16  16   5   4  19
# 2   7  10  15  13   2

pd.DataFrame(df['foo'].tolist()).values.shape
# (3, 5)