I have a list of pandas Series objects. I want to add up all these series, such that rows that have the same index will have their values added together, and rows that have unique indices will just be added. Other than accessing the first Series object and iterating through and using the Series add function, is there a neater way to do this? I know that for concatenation of dataframes, one could pass a list of dataframe objects, but I haven't found something similar for adding series in this way.
Background: I have very large datasets where each data set is broken up into dataframe chunks. for each dataframe chunk I did a values_count on each column, resulting in the series and got a list of lists essentially. using list comprehension, I can get one column at a time for all the chunks, but I need to add these value_counts together.
for vcl in allvc:
#vcl is a list where each element is a list of value counts
lengthvcl = len(vcl)
for x in range(lengthvcl):
lst2 = [item(x) for item in vcl]
#what to do next...
reduce + pd.Series.addOne Pandas-based solution is possible via functools.reduce and pd.Series.add:
from functools import reduce, partial
s1 = pd.Series([1, 2, 3])
s2 = pd.Series([4, 5, 6, 7])
s3 = pd.Series([8, 9, 10, 11, 12])
adder = partial(pd.Series.add, fill_value=0)
res = reduce(adder, [s1, s2, s3])
print(res)
0 13.0
1 16.0
2 19.0
3 18.0
4 12.0
dtype: float64
pd.concat + pd.DataFrame.sumThe reduce solution is generic since adder can be replaced with an arbitrary function. For summation, you can also create a dataframe first by concatenating your series:
res = pd.concat([s1, s2, s3], axis=1).sum(1)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With