I have a dataset made of True and False.
Sample Table:
A B C
0 False True False
1 False False False
2 True True False
3 True True True
4 False True False
5 True True True
6 True False False
7 True False True
8 False True True
9 True False False
I want to count the number of consecutive True values for every column, and if there's more than one consecutive True series, I want to get the max of it.
For the table above, I would get:
length = [3, 4, 2]
I found similar threads but none resolved my problem.
Since I do and will have many more columns(products), I need to do this regardless of the column name, for the whole table and get an array as the result.
And if possible, I'd like to learn the index of the first true of the longest sequence aka where this longest true series starts, so the result would be for this one:
index = [5, 2, 7]
Select the Dataframe column using the column name and subscript operator i.e. df['C']. It returns the column 'C' as a Series object of only bool values. After that, call the sum() function on this boolean Series object, and it will return the count of only True values in the Series/column.
In Pandas, You can get the count of each row of DataFrame using DataFrame. count() method. In order to get the row count you should use axis='columns' as an argument to the count() method. Note that the count() method ignores all None & nan values from the count.
The str. count() function is used to count occurrences of pattern in each string of the Series/Index. This function is used to count the number of times a particular regex pattern is repeated in each of the string elements of the Series.
Use Sum Function to Count Specific Values in a Column in a Dataframe. We can use the sum() function on a specified column to count values equal to a set condition, in this case we use == to get just rows equal to our specific data point.
Solution should be simplify, if always at least one True
per column:
b = df.cumsum()
c = b.sub(b.mask(df).ffill().fillna(0)).astype(int)
print (c)
A B C
0 0 1 0
1 0 0 0
2 1 1 0
3 2 2 1
4 0 3 0
5 1 4 1
6 2 0 0
7 3 0 1
8 0 1 2
9 1 0 0
#get maximal value of all columns
length = c.max().tolist()
print (length)
[3, 4, 2]
#get indexes by maximal value, subtract length and add 1
index = c.idxmax().sub(length).add(1).tolist()
print (index)
[5, 2, 7]
Detail:
print (pd.concat([b,
b.mask(df),
b.mask(df).ffill(),
b.mask(df).ffill().fillna(0),
b.sub(b.mask(df).ffill().fillna(0)).astype(int)
], axis=1,
keys=('cumsum', 'mask', 'ffill', 'fillna','sub')))
cumsum mask ffill fillna sub
A B C A B C A B C A B C A B C
0 0 1 0 0.0 NaN 0.0 0.0 NaN 0.0 0.0 0.0 0.0 0 1 0
1 0 1 0 0.0 1.0 0.0 0.0 1.0 0.0 0.0 1.0 0.0 0 0 0
2 1 2 0 NaN NaN 0.0 0.0 1.0 0.0 0.0 1.0 0.0 1 1 0
3 2 3 1 NaN NaN NaN 0.0 1.0 0.0 0.0 1.0 0.0 2 2 1
4 2 4 1 2.0 NaN 1.0 2.0 1.0 1.0 2.0 1.0 1.0 0 3 0
5 3 5 2 NaN NaN NaN 2.0 1.0 1.0 2.0 1.0 1.0 1 4 1
6 4 5 2 NaN 5.0 2.0 2.0 5.0 2.0 2.0 5.0 2.0 2 0 0
7 5 5 3 NaN 5.0 NaN 2.0 5.0 2.0 2.0 5.0 2.0 3 0 1
8 5 6 4 5.0 NaN NaN 5.0 5.0 2.0 5.0 5.0 2.0 0 1 2
9 6 6 4 NaN 6.0 4.0 5.0 6.0 4.0 5.0 6.0 4.0 1 0 0
EDIT:
General solution working with only False
columns - add numpy.where
with boolean mask created by DataFrame.any
:
print (df)
A B C
0 False True False
1 False False False
2 True True False
3 True True False
4 False True False
5 True True False
6 True False False
7 True False False
8 False True False
9 True False False
b = df.cumsum()
c = b.sub(b.mask(df).ffill().fillna(0)).astype(int)
mask = df.any()
length = np.where(mask, c.max(), -1).tolist()
print (length)
[3, 4, -1]
index = np.where(mask, c.idxmax().sub(c.max()).add(1), 0).tolist()
print (index)
[5, 2, 0]
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With