Pipeline with and without Target-Based Encoding

Question

I'm confused about the best way to assemble a pipeline if I'm doing both a simple encoder, and a target-encoder. I've found this example here, which illustrates the problem is related to having to pass the target variable along w/ the variable to be encoded.

from examples.source_data.loaders import get_mushroom_data
from sklearn.compose import ColumnTransformer
from category_encoders import TargetEncoder

# get data from the mushroom dataset
X, y, _ = get_mushroom_data()

# encode the specified columns
ct = ColumnTransformer(
    [
        ('Target encoding', TargetEncoder(), ['bruises', 'odor'])
    ], remainder='passthrough'
)
encoded = ct.fit_transform(X=X, y=y)

However, instead of directly doing a fit_transform, I'd like to add it as a part of my pipeline so that I can do it within a cross-fold validation scheme.

So, the code that doesn't work is:

pipeline_ordinal = Pipeline(steps=[('imputer', SimpleImputer(strategy='constant', fill_value='missing'))
    ,('ord encoding', ce.ordinal.OrdinalEncoder())])

pipeline_loo = Pipeline(steps=[('imputer', SimpleImputer(strategy='constant', fill_value='missing'))
    ,('loo encoding', ce.LeaveOneOutEncoder())])

preprocessor = ColumnTransformer(
    transformers=[('simple', pipeline_ordinal, ['x1','x2','x3']),
                  ('targetbased', pipeline_loo, ['x4','x5','y'])
                 ])

rf = RandomForestRegressor()

pipe = Pipeline(steps=[('preprocessor', preprocessor),('regression', rf)])

gs = GridSearchCV(pipe, param_grid=params, cv = cv)

gs.fit(X, y)

Any ideas on a better way to patch this all together?

Edit:

The problem lies in passing X into gs.fit(). As is, the code above says: ValueError: A given column is not a column of the dataframe

If I try to get clever and send 'y' along in X, then it tells me ValueError: cannot reindex from a duplicate axis

Answer 1

The target variable y gets passed along and treated specially in gs.fit(X, y). You don't need to (and shouldn't) specify it as a column in the ColumnTransformer.

(Both pipeline_ordinal and pipeline_loo will have access to y, though the former won't actually use it.)