Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Pandas DataFrame use previous row value for complicated 'if' conditions to determine current value

I want to know if there is any faster way to do the following loop? Maybe use apply or rolling apply function to realize this Basically, I need to access previous row's value to determine current cell value.

df.ix[0] = (np.abs(df.ix[0]) >= So) * np.sign(df.ix[0])
for i in range(1, len(df)):
    for col in list(df.columns.values):
        if ((df[col].ix[i] > 1.25) & (df[col].ix[i-1] == 0)) | :
            df[col].ix[i] = 1
        elif ((df[col].ix[i] < -1.25) & (df[col].ix[i-1] == 0)):
            df[col].ix[i] = -1
        elif ((df[col].ix[i] <= -0.75) & (df[col].ix[i-1] < 0)) | ((df[col].ix[i] >= 0.5) & (df[col].ix[i-1] > 0)):
            df[col].ix[i] = df[col].ix[i-1]
        else:
            df[col].ix[i] = 0

As you can see, in the function, I am updating the dataframe, I need to access the most updated previous row, so using shift will not work.

For example: Input:

A      B     C
1.3  -1.5   0.7
1.1  -1.4   0.6
1.0  -1.3   0.5
0.4   1.4   0.4

Output:

 A      B     C
1     -1      0
1     -1      0
1     -1      0
0      1      0
like image 296
user5025141 Avatar asked Apr 28 '16 19:04

user5025141


People also ask

How do you replace values based on conditions?

You can replace values of all or selected columns based on the condition of pandas DataFrame by using DataFrame. loc[ ] property. The loc[] is used to access a group of rows and columns by label(s) or a boolean array. It can access and can also manipulate the values of pandas DataFrame.

How do you determine whether a pandas column contains a particular value?

You can check if a column contains/exists a particular value (string/int), list of multiple values in pandas DataFrame by using pd. series() , in operator, pandas. series. isin() , str.

How do you use between conditions in pandas?

Boolean Series in PandasThe between() function is used to get boolean Series equivalent to left <= series <= right. This function returns a boolean vector containing True wherever the corresponding Series element is between the boundary values left and right. NA values are treated as False.


1 Answers

you can use .shift() function for accessing previous or next values:

previous value for col column:

df['col'].shift()

next value for col column:

df['col'].shift(-1)

Example:

In [38]: df
Out[38]:
   a  b  c
0  1  0  5
1  9  9  2
2  2  2  8
3  6  3  0
4  6  1  7

In [39]: df['prev_a'] = df['a'].shift()

In [40]: df
Out[40]:
   a  b  c  prev_a
0  1  0  5     NaN
1  9  9  2     1.0
2  2  2  8     9.0
3  6  3  0     2.0
4  6  1  7     6.0

In [43]: df['next_a'] = df['a'].shift(-1)

In [44]: df
Out[44]:
   a  b  c  prev_a  next_a
0  1  0  5     NaN     9.0
1  9  9  2     1.0     2.0
2  2  2  8     9.0     6.0
3  6  3  0     2.0     6.0
4  6  1  7     6.0     NaN
like image 115
MaxU - stop WAR against UA Avatar answered Sep 18 '22 15:09

MaxU - stop WAR against UA