Is there a way to sum multiple pandas DataFrames using syntax similar to pd.concat([df1, df2, df3, df4])
. I understand from documentation that I can do df1.sum(df2, fill_value=0)
, but I have a long list of DataFrames I need to sum and was wondering if I could do it without writing a loop.
Somewhat related question/answer: Pandas sum multiple dataframes (Stack Overflow)
Example of what the result should look like:
idx1 = pd.MultiIndex.from_tuples([('a', 'A'), ('a', 'B'), ('b', 'A'), ('b', 'D')])
idx2 = pd.MultiIndex.from_tuples([('a', 'A'), ('a', 'C'), ('b', 'A'), ('b', 'C')])
idx3 = pd.MultiIndex.from_tuples([('a', 'A'), ('a', 'D'), ('b', 'A'), ('b', 'C')])
np.random.seed([3,1415])
df1 = pd.DataFrame(np.random.randn(4, 1), idx1, ['val'])
df2 = pd.DataFrame(np.random.randn(4, 1), idx2, ['val'])
df3 = pd.DataFrame(np.random.randn(4, 1), idx3, ['val'])
df1
df2
df3
The result should look like:
Pandas DataFrame sum() MethodThe sum() method adds all values in each column and returns the sum for each column. By specifying the column axis ( axis='columns' ), the sum() method searches column-wise and returns the sum of each row.
To sum given or list of columns then create a list with all columns you wanted and slice the DataFrame with the selected list of columns and use the sum() function. Use df['Sum']=df[col_list]. sum(axis=1) to get the total sum.
To sum all the rows of a DataFrame, use the sum() function and set the axis value as 1. The value axis 1 will add the row values.
Use reduce
with add
with parameter fill_value=0
:
np.random.seed(12)
a = pd.DataFrame(np.random.randint(3, size=(5,3)), columns=list('abc'))
b = pd.DataFrame(np.random.randint(3, size=(5,2)), columns=list('ab'))
c = pd.DataFrame(np.random.randint(3, size=(5,2)), columns=list('ac'))
print(a)
a b c
0 2 1 1
1 2 0 0
2 2 1 0
3 1 1 1
4 2 2 2
print(b)
a b
0 0 1
1 0 0
2 1 2
3 1 2
4 0 1
print(c)
a c
0 2 0
1 2 2
2 2 0
3 0 2
4 1 1
from functools import reduce
dfs = [a,b, c]
d = reduce(lambda x, y: x.add(y, fill_value=0), dfs)
print (d)
a b c
0 4 2.0 1.0
1 4 0.0 2.0
2 5 3.0 0.0
3 2 3.0 3.0
4 3 3.0 3.0
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With