Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Pandas Merge row data with multiple values to Python list for a column

I have a data-frame that looks like

DATA

*id*,             *name*,                      *URL*,                 *Type*  
    2,             birth_france_by_region,    http://abc. com,       T1 
    2,             birth_france_by_region,    http://pt. python,     T2 
    3,             long_lat,                  http://abc. com,       T3 
    3,             long_lat,                  http://pqur. com,      T1 
    4,             random_time_series,        http://sadsdc. com,    T2 
    4,             random_time_series,        http://sadcadf. com,   T3
    5,             birth_names,               http://google. com,    T1 
    5,             birth_names,               http://helloworld. com,T2 
    5,             birth_names,               http://hu. com,        T3

I want a this dataframe to merge the rows where id are equal and have a list of Type corresponding list of URL so final output should be like

*id*, *name*,             *URL*,                               *Type*  
2,birth_france_by_region,  [http://abc .com,http://pt.python], [T1,T2] 
3,long_lat,           [http://abc .com,http://pqur. com],       [T3,T1] 
4,random_time_series, [http://sadsdc. com,http://sadcadf .com,],[T2,T3] 
5,birth_names,        [http://google .com,http://helloworld. com,
                                       http://hu. com] ,   [T1,T2,T3]
like image 394
Kalpish Singhal Avatar asked Sep 18 '17 09:09

Kalpish Singhal


People also ask

How do you concatenate multiple column values into a single column in Pandas DataFrame?

You can use DataFrame. apply() for concatenate multiple column values into a single column, with slightly less typing and more scalable when you want to join multiple columns .

How do I merge multiple Dataframes in Pandas based on a column?

Pandas merge() function is used to merge multiple Dataframes. We can use either pandas. merge() or DataFrame. merge() to merge multiple Dataframes.


1 Answers

I think you need groupby and aggregate tuple and then convert to list:

df = df.groupby(['id','name']).agg(tuple).applymap(list).reset_index()

print (df)
   id                    name  \
0   2  birth_france_by_region   
1   3                long_lat   
2   4      random_time_series   
3   5             birth_names   

                                                 URL          Type  
0                 [http://abc.cm, http://pt.python]      [T1, T2]  
1                  [http://abc.cm, http://pqur.com]      [T3, T1]  
2            [http://sadsdc.com, http://sadcadf.com]      [T2, T3]  
3  [http://google.;com, http://helloworld.com, ht...  [T1, T2, T3] 

Because in version 0.20.3 raise error:

df = df.groupby(['id','name']).agg(lambda x: x.tolist())

ValueError: Function does not reduce

like image 139
jezrael Avatar answered Sep 28 '22 08:09

jezrael