Reshape long to wide using columns names

Question

Hi I am having trouble reshaping my df.

I have:

Netflix     TV      DVD 
   0.1      0.2     0.3
   0.12     0.5     0.15
   0.4      0.6     0.8
            0.5     0.41
            0.41
            0.2 

And I want to convert my df to look like:

Netflix  [0.1, 0.12, 0.4]
TV       [0.2, 0.5, 0.6, 0.5, 0.41, 0.2] 
DVD      [0.3, 0.15, 0.8, 0.41]

Not sure how stack() or pivot() would work on a df of this kind. Any help appreciated.

Answer 1

stack

Stacking drops null values while reshaping the array

df.stack().groupby(level=1).agg(list)

DVD                 [0.3, 0.15, 0.8, 0.41]
Netflix                   [0.1, 0.12, 0.4]
TV         [0.2, 0.5, 0.6, 0.5, 0.41, 0.2]
dtype: object

Answer 2

Remove missing values by Series.dropna and convert to Series in dictionary comprehension:

s = pd.Series({x: df[x].dropna().tolist() for x in df.columns})
print (s)
Netflix                   [0.1, 0.12, 0.4]
TV         [0.2, 0.5, 0.6, 0.5, 0.41, 0.2]
DVD                 [0.3, 0.15, 0.8, 0.41]
dtype: object

...or in DataFrame.apply:

s = df.apply(lambda x: x.dropna().tolist())
print (s)

Netflix                   [0.1, 0.12, 0.4]
TV         [0.2, 0.5, 0.6, 0.5, 0.41, 0.2]
DVD                 [0.3, 0.15, 0.8, 0.41]
dtype: object

Last if need 2 columns DataFrame:

df1 = s.rename_axis('a').reset_index(name='b')
print (df1)
         a                                b
0  Netflix                 [0.1, 0.12, 0.4]
1       TV  [0.2, 0.5, 0.6, 0.5, 0.41, 0.2]
2      DVD           [0.3, 0.15, 0.8, 0.41]

Answer 3

I think this is what you are looking for:

> df.T.apply(lambda x: x.dropna().tolist(), axis=1)

Netflix    [0.1, 0.12, 0.4, 0.5, 0.41, 0.2]
TV                    [0.2, 0.5, 0.6, 0.41]
DVD                        [0.3, 0.15, 0.8]
dtype: object

Answer 4

Using groupby with columns

df.groupby(level=0,axis=1).apply(lambda x : x.dropna().iloc[:,0].tolist())
Out[20]: 
DVD                 [0.3, 0.15, 0.8, 0.41]
Netflix                   [0.1, 0.12, 0.4]
TV         [0.2, 0.5, 0.6, 0.5, 0.41, 0.2]
dtype: object