For my evaluation, I wanted to run a rolling 1000 window OLS regression estimation
of the dataset found in this URL:
https://drive.google.com/open?id=0B2Iv8dfU4fTUa3dPYW5tejA0bzg
using the following Python
script.
# /usr/bin/python -tt
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
from statsmodels.formula.api import ols
df = pd.read_csv('estimated.csv', names=('x','y'))
model = pd.stats.ols.MovingOLS(y=df.Y, x=df[['y']],
window_type='rolling', window=1000, intercept=True)
df['Y_hat'] = model.y_predict
However, when I run my Python script, I am getting this error: AttributeError: module 'pandas.stats' has no attribute 'ols'
. Could this error be from the version that I am using? The pandas
installed on my Linux node has a version of 0.20.2
pd.stats.ols.MovingOLS
was removed in Pandas version 0.20.0
http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0200-prior-deprecations
https://github.com/pandas-dev/pandas/pull/11898
I can't find an 'off the shelf' solution for what should be such an obvious use case as rolling regressions.
The following should do the trick without investing too much time in a more elegant solution. It uses numpy to calculate the predicted value of the regression based on the regression parameters and the X values in the rolling window.
window = 1000
a = np.array([np.nan] * len(df))
b = [np.nan] * len(df) # If betas required.
y_ = df.y.values
x_ = df[['x']].assign(constant=1).values
for n in range(window, len(df)):
y = y_[(n - window):n]
X = x_[(n - window):n]
# betas = Inverse(X'.X).X'.y
betas = np.linalg.inv(X.T.dot(X)).dot(X.T).dot(y)
y_hat = betas.dot(x_[n, :])
a[n] = y_hat
b[n] = betas.tolist() # If betas required.
The code above is equivalent to the following and about 35% faster:
model = pd.stats.ols.MovingOLS(y=df.y, x=df.x, window_type='rolling', window=1000, intercept=True)
y_pandas = model.y_predict
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With