How do I turn a dataframe into a series of lists?

Question

I have had to do this several times and I'm always frustrated. I have a dataframe:

df = pd.DataFrame([[1, 2, 3, 4], [5, 6, 7, 8]], ['a', 'b'], ['A', 'B', 'C', 'D'])  print df     A  B  C  D a  1  2  3  4 b  5  6  7  8 

I want to turn df into:

pd.Series([[1, 2, 3, 4], [5, 6, 7, 8]], ['a', 'b'])  a    [1, 2, 3, 4] b    [5, 6, 7, 8] dtype: object 

I've tried

df.apply(list, axis=1) 

Which just gets me back the same df

What is a convenient/effective way to do this?

Answer 1

You can first convert DataFrame to numpy array by values, then convert to list and last create new Series with index from df if need faster solution:

print (pd.Series(df.values.tolist(), index=df.index)) a    [1, 2, 3, 4] b    [5, 6, 7, 8] dtype: object 

Timings with small DataFrame:

In [76]: %timeit (pd.Series(df.values.tolist(), index=df.index)) 1000 loops, best of 3: 295 µs per loop  In [77]: %timeit pd.Series(df.T.to_dict('list')) 1000 loops, best of 3: 685 µs per loop  In [78]: %timeit df.T.apply(tuple).apply(list) 1000 loops, best of 3: 958 µs per loop 

and with large:

from string import ascii_letters letters = list(ascii_letters) df = pd.DataFrame(np.random.choice(range(10), (52 ** 2, 52)),                   pd.MultiIndex.from_product([letters, letters]),                   letters)  In [71]: %timeit (pd.Series(df.values.tolist(), index=df.index)) 100 loops, best of 3: 2.06 ms per loop  In [72]: %timeit pd.Series(df.T.to_dict('list')) 1 loop, best of 3: 203 ms per loop  In [73]: %timeit df.T.apply(tuple).apply(list) 1 loop, best of 3: 506 ms per loop