Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why is pandas.DataFrame.apply printing out junk?

Consider this simple dataframe:

   a  b
0  1  2
1  2  3

I perform a .apply as such:

In [4]: df.apply(lambda x: [x.values])
Out[4]: 
a    [[140279910807944, 140279910807920]]
b    [[140279910807944, 140279910807920]]
dtype: object

In [5]: df.apply(lambda x: [x.values])
Out[5]: 
a    [[37, 37]]
b    [[37, 37]]
dtype: object

In [6]: df.apply(lambda x: [x.values])
Out[6]: 
a    [[11, 11]]
b    [[11, 11]]
dtype: object

Why is pandas printing out junk each time?

I've verified this happens in v0.20.

Edit: Looking for an answer, not a workaround.

like image 898
cs95 Avatar asked Sep 10 '17 04:09

cs95


Video Answer


2 Answers

It looks like bug, so was opened Issue 17487.

For me working add tolist:

print (df.apply(lambda x: [x.values.tolist()]))
a    [[1, 2]]
b    [[2, 3]]
dtype: object

print (df.apply(lambda x: [list(x.values)]))
a    [[1, 2]]
b    [[2, 3]]
dtype: object
like image 189
jezrael Avatar answered Nov 23 '22 23:11

jezrael


I don't have an answer... just a work around

f = lambda x: x.values.reshape(1, -1).tolist()

df.apply(f)

a    [[1, 2]]
b    [[2, 3]]
dtype: object

I tracked it down to pd.lib.reduce

pd.lib.reduce(df.values, lambda x: [list(x)])

array([list([[1, 2]]), list([[2, 3]]), list([['a', 'b']])], dtype=object)

Versus

pd.lib.reduce(df.values, lambda x: [x])

array([list([array([None, None], dtype=object)]),
       list([array([None, None], dtype=object)]),
       list([array([None, None], dtype=object)])], dtype=object)
like image 43
piRSquared Avatar answered Nov 24 '22 01:11

piRSquared