Using predict() on statsmodels.formula data with different column names using Python and Pandas

Question

I've got some regressions results from running statsmodels.formula.api.ols. Here's a toy example:

import pandas as pd
import numpy as np
import statsmodels.formula.api as smf

example_df = pd.DataFrame(np.random.randn(10, 3))
example_df.columns = ["a", "b", "c"]
fit = smf.ols('a ~ b', example_df).fit()

I'd like to apply the model to column c, but a naive attempt to do so doesn't work:

fit.predict(example_df["c"])

Here's the exception I get:

PatsyError: Error evaluating factor: NameError: name 'b' is not defined
    a ~ b
        ^

I can do something gross and create a new, temporary DataFrame in which I rename the column of interest:

example_df2 = pd.DataFrame(example_df["c"])
example_df2.columns = ["b"]
fit.predict(example_df2)

Is there a cleaner way to do this? (short of switching to statsmodels.api instead of statsmodels.formula.api)

Answer 1

You can use a dictionary:

>>> fit.predict({"b": example_df["c"]})
array([ 0.84770672, -0.35968269,  1.19592387, -0.77487812, -0.98805215,
        0.90584753, -0.15258093,  1.53721494, -0.26973941,  1.23996892])

or create a numpy array for the prediction, although that is much more complicated if there are categorical explanatory variables:

>>> fit.predict(sm.add_constant(example_df["c"].values), transform=False)
array([ 0.84770672, -0.35968269,  1.19592387, -0.77487812, -0.98805215,
        0.90584753, -0.15258093,  1.53721494, -0.26973941,  1.23996892])

Answer 2

If you replace your fit definition with this line:

fit = smf.ols('example_df.a ~ example_df.b', example_df).fit()

It should work.

fit.predict(example_df["c"])

array([-0.52664491, -0.53174346, -0.52172484, -0.52819856, -0.5253607 ,
       -0.52391618, -0.52800043, -0.53350634, -0.52362988, -0.52520823])