Determining when a column value changes in pandas dataframe

Question

I am looking to write a quick script that will run through a csv file with two columns and provide me the rows in which the values in column B switch from one value to another:

eg:

dataframe:

# |  A  |  B   --+-----+----- 1 |  2  |  3 2 |  3  |  3 3 |  4  |  4 4 |  5  |  4 5 |  5  |  4 

would tell me that the change happened between row 2 and row 3. I know how to get these values using for loops but I was hoping there was a more pythonic way of approaching this problem.

Answer 1

You can create a new column for the difference

> df['C'] = df['B'].diff() > print df    #  A  B   C 0  1  2  3 NaN 1  2  3  3   0 2  3  4  4   1 3  4  5  4   0 4  5  5  4   0  > df_filtered = df[df['C'] != 0] > print df_filtered    #  A  B  C 2  3  4  4  1 

This will your required rows

Answer 2

You can do the following which also works for non numerical values:

>>> import pandas as pd >>> df = pd.DataFrame({"Status": ["A","A","B","B","C","C","C"]}) >>> df["isStatusChanged"] = df["Status"].shift(1, fill_value=df["Status"].head(1)) != df["Status"] >>> df   Status  isStatusChanged 0      A            False 1      A            False 2      B             True 3      B            False 4      C             True 5      C            False 6      C            False >>>   

Note the fill_value could be different depending on your application.