Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Adding a list of Pandas series

I have a list of pandas Series objects. I want to add up all these series, such that rows that have the same index will have their values added together, and rows that have unique indices will just be added. Other than accessing the first Series object and iterating through and using the Series add function, is there a neater way to do this? I know that for concatenation of dataframes, one could pass a list of dataframe objects, but I haven't found something similar for adding series in this way.

Background: I have very large datasets where each data set is broken up into dataframe chunks. for each dataframe chunk I did a values_count on each column, resulting in the series and got a list of lists essentially. using list comprehension, I can get one column at a time for all the chunks, but I need to add these value_counts together.

for vcl in allvc:
    #vcl is a list where each element is a list of value counts
    lengthvcl = len(vcl)
    for x in range(lengthvcl):
        lst2 = [item(x) for item in vcl]
        #what to do next...
like image 908
MLDS Avatar asked Feb 02 '26 10:02

MLDS


1 Answers

reduce + pd.Series.add

One Pandas-based solution is possible via functools.reduce and pd.Series.add:

from functools import reduce, partial

s1 = pd.Series([1, 2, 3])
s2 = pd.Series([4, 5, 6, 7])
s3 = pd.Series([8, 9, 10, 11, 12])

adder = partial(pd.Series.add, fill_value=0)
res = reduce(adder, [s1, s2, s3])

print(res)

0    13.0
1    16.0
2    19.0
3    18.0
4    12.0
dtype: float64

pd.concat + pd.DataFrame.sum

The reduce solution is generic since adder can be replaced with an arbitrary function. For summation, you can also create a dataframe first by concatenating your series:

res = pd.concat([s1, s2, s3], axis=1).sum(1)
like image 97
jpp Avatar answered Feb 05 '26 02:02

jpp



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!