Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to merge/combine columns in pandas?

I have a (example-) dataframe with 4 columns:

data = {'A': ['a', 'b', 'c', 'd', 'e', 'f'],
    'B': [42, 52, np.nan, np.nan, np.nan, np.nan],  
    'C': [np.nan, np.nan, 31, 2, np.nan, np.nan],
    'D': [np.nan, np.nan, np.nan, np.nan, 62, 70]}
df = pd.DataFrame(data, columns = ['A', 'B', 'C', 'D'])

    A   B       C       D
0   a   42.0    NaN     NaN
1   b   52.0    NaN     NaN
2   c   NaN     31.0    NaN
3   d   NaN     2.0     NaN
4   e   NaN     NaN     62.0
5   f   NaN     NaN     70.0

I would now like to merge/combine columns B, C, and D to a new column E like in this example:

data2 = {'A': ['a', 'b', 'c', 'd', 'e', 'f'],
    'E': [42, 52, 31, 2, 62, 70]}
df2 = pd.DataFrame(data2, columns = ['A', 'E'])

    A   E
0   a   42
1   b   52
2   c   31
3   d   2
4   e   62
5   f   70

I found a quite similar question here but this adds the merged colums B, C, and D at the end of column A:

0      a
1      b
2      c
3      d
4      e
5      f
6     42
7     52
8     31
9      2
10    62
11    70
dtype: object

Thanks for help.

like image 332
mati Avatar asked Oct 04 '17 11:10

mati


People also ask

How do I merge two columns in pandas?

By use + operator simply you can combine/merge two or multiple text/string columns in pandas DataFrame. Note that when you apply + operator on numeric columns it actually does addition instead of concatenation.

How do I merge two panda data frames?

The concat() function can be used to concatenate two Dataframes by adding the rows of one to the other. The merge() function is equivalent to the SQL JOIN clause. 'left', 'right' and 'inner' joins are all possible.

How do I merge two DataFrames in pandas based on common column?

To merge two Pandas DataFrame with common column, use the merge() function and set the ON parameter as the column name.

How do I merge column names in pandas?

Different column names are specified for merges in Pandas using the “left_on” and “right_on” parameters, instead of using only the “on” parameter. Merging dataframes with different names for the joining variable is achieved using the left_on and right_on arguments to the pandas merge function.


1 Answers

Option 1
Using assign and drop

In [644]: cols = ['B', 'C', 'D']

In [645]: df.assign(E=df[cols].sum(1)).drop(cols, 1)
Out[645]:
   A     E
0  a  42.0
1  b  52.0
2  c  31.0
3  d   2.0
4  e  62.0
5  f  70.0

Option 2
Using assignment and drop

In [648]: df['E'] = df[cols].sum(1)

In [649]: df = df.drop(cols, 1)

In [650]: df
Out[650]:
   A     E
0  a  42.0
1  b  52.0
2  c  31.0
3  d   2.0
4  e  62.0
5  f  70.0

Option 3 Lately, I like the 3rd option.
Using groupby

In [660]: df.groupby(np.where(df.columns == 'A', 'A', 'E'), axis=1).first() #or sum max min
Out[660]:
   A     E
0  a  42.0
1  b  52.0
2  c  31.0
3  d   2.0
4  e  62.0
5  f  70.0

In [661]: df.columns == 'A'
Out[661]: array([ True, False, False, False], dtype=bool)

In [662]: np.where(df.columns == 'A', 'A', 'E')
Out[662]:
array(['A', 'E', 'E', 'E'],
      dtype='|S1')
like image 122
Zero Avatar answered Nov 15 '22 20:11

Zero