Using a shift() function within an apply function to compare rows in a Pandas Dataframe

Question

I would like to use shift() to pull in data from the previous index, provided values in one of the columns, Letter, is the same.

import pandas as pd
df = pd.DataFrame(data=[['A', 'one'],
                        ['A', 'two'],
                        ['B', 'three'],
                        ['B', 'four'],
                        ['C', 'five']],
                  columns=['Letter', 'value'])

df['Previous Value'] = df.apply(lambda x : x['value'] if x['Letter'].shift(1) == x['Letter'] else "", axis=1)
print df

I am getting the error:

AttributeError: ("'str' object has no attribute 'shift'", u'occurred at index 0')

Desired Output:

  Letter  value Previous Value
0      A    one               
1      A    two            one
2      B  three               
3      B   four          three
4      C   five               

Answer 1

Use where on your condition where the current row matches previous row using shift:

In [11]:
df = pd.DataFrame(data=[['A', 'one'],
                        ['A', 'two'],
                        ['B', 'three'],
                        ['B', 'four'],
                        ['C', 'five']],
                  columns=['Letter', 'value'])
​
df['Previous Value'] = df['value'].shift().where(df['Letter'].shift() == df['Letter'], '')
df
​
Out[11]:
  Letter  value Previous Value
0      A    one               
1      A    two            one
2      B  three               
3      B   four          three
4      C   five               

Answer 2

You are trying to apply .shift() to a value of a given column of a given row instead of a Series. I would do this, using groupby:

In [6]: df['Previous letter'] = df.groupby('Letter').value.shift()

In [7]: df
Out[7]:
  Letter  value Previous letter
0      A    one             NaN
1      A    two             one
2      B  three             NaN
3      B   four           three
4      C   five             NaN