Pandas: Find previous row of matching value

Question

I'm trying to create a column with values from one column, but based on matching another column with the previous value.

Here is my current code:

d = {'a':[1,2,3,1,2,3,2,1], 'b':[10,20,30,40,50,60,70,80]}

df = pd.DataFrame(d)

df['c'] = df['b'][df['a'] == df['a'].prev()]

And my desired output:

   a   b    c
0  1  10  NaN
1  2  20  NaN
2  3  30  NaN
3  1  40   10
4  2  50   20
5  3  60   30
6  2  70   50
7  1  80   40

...which I'm not getting because .prev() is not a real thing. Any thoughts?

Answer 1

We can group by a column, which by default sorts values and then "attach" shifted b column:

In [110]: df['c'] = df.groupby('a')['b'].transform(lambda x: x.shift())

In [111]: df
Out[111]:
   a   b     c
0  1  10   NaN
1  2  20   NaN
2  3  30   NaN
3  1  40  10.0
4  2  50  20.0
5  3  60  30.0
6  2  70  50.0
7  1  80  40.0

Or much better option - using GroupBy.shift() (thank you @Mitch)

In [114]: df['c'] = df.groupby('a')['b'].shift()

In [115]: df
Out[115]:
   a   b     c
0  1  10   NaN
1  2  20   NaN
2  3  30   NaN
3  1  40  10.0
4  2  50  20.0
5  3  60  30.0
6  2  70  50.0
7  1  80  40.0