Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Applying Savitzky-Golay filter on a pandas dataframe

Tags:

python

pandas

I have the following time series data set:

import pandas as pd
from datetime import datetime
import numpy as np
from scipy.signal import savgol_filter

date_rng = pd.date_range(start='2020-07-01', end='2020-07-20', freq='d')
df = pd.DataFrame(date_rng, columns=['date'])
df['data'] = np.random.randint(0,100,size=(len(date_rng)))

I would like to calculate a Savitzky-Golay filter and a pandas dataframe, which I defined in the following function:

def savgol(x, wl=3, p=2):
    return savgol_filter(x, window_length=wl, polyorder=p)

df['sav_gol'] = df['data'].apply(savgol)

When executing the script I recieve the following error message:

ValueError: If mode is 'interp', window_length must be less than or equal to the size of x.

EDIT:

Here is my adjusted dataset with groups. I would apply the savgol function to this dataset:

df = pd.DataFrame({
    'date':date_rng,
    'value':np.random.randint(0,100,size=(len(date_rng))),
    'group':'a'
})
df2 = pd.DataFrame({
    'date':date_rng,
    'value':np.random.randint(0,100,size=(len(date_rng))),
    'group':'b'
})

df = df.append(df2, ignore_index=True)

This would be my attempt:

df['sav_gol'] = df.groupby('group')['value'].apply(savgol)

Any help would be much appreciated!

like image 413
cwohlfart Avatar asked Dec 31 '25 08:12

cwohlfart


1 Answers

This actually worked:

df['savgol'] = df.groupby('group')['value'].transform(lambda x: savgol_filter(x, 5,2))
like image 159
cwohlfart Avatar answered Jan 02 '26 20:01

cwohlfart



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!