Scikit-learn multi-output classifier using: GridSearchCV, Pipeline, OneVsRestClassifier, SGDClassifier

Question

I am attempting to build a multi-output model with GridSearchCV and Pipeline. The Pipeline is giving me trouble because standard classifier examples don't have the OneVsRestClassifier() wrapping the classifier. I'm using scikit-learn 0.18 and python 3.5

## Pipeline: Train and Predict
## SGD: support vector machine (SVM) with gradient descent
from sklearn.multiclass import OneVsRestClassifier
from sklearn.pipeline import Pipeline
from sklearn.linear_model import SGDClassifier

clf = Pipeline([
               ('vect', CountVectorizer(ngram_range=(1,3), max_df=0.50 ) ),
               ('tfidf', TfidfTransformer() ),
               ('clf', SGDClassifier(loss='modified_huber', penalty='elasticnet',
                                          alpha=1e-4, n_iter=5, random_state=42,
                                          shuffle=True, n_jobs=-1) ),
                ])

ovr_clf = OneVsRestClassifier(clf ) 

from sklearn.model_selection import GridSearchCV
parameters = {'vect__ngram_range': [(1,1), (1,3)],
              'tfidf__norm': ('l1', 'l2', None),
              'estimator__loss': ('modified_huber', 'hinge',),
             }

gs_clf = GridSearchCV(estimator=pipeline, param_grid=parameters, 
                      scoring='f1_weighted', n_jobs=-1, verbose=1)
gs_clf = gs_clf.fit(X_train, y_train)

But this yields the error: ....

ValueError: Invalid parameter estimator for estimator Pipeline(steps=[('vect', CountVectorizer(analyzer='word', binary=False, decode_error='strict', dtype=, encoding='utf-8', input='content', lowercase=True, max_df=0.5, max_features=None, min_df=1, ngram_range=(1, 3), preprocessor=None, stop_words=None, strip...er_t=0.5, random_state=42, shuffle=True, verbose=0, warm_start=False), n_jobs=-1))]). Check the list of available parameters with estimator.get_params().keys().

So what is the correct way to pass parameters to clf through the OneVsRestClassifier using param_grid and Pipeline? Do I need to separate the vectorizer and tdidf from the classifier in the Pipeline?

Answer 1

Pass OneVsRestClassifier() as a step of pipeline itself and SGDClassifier as estimator of OneVsRestClassifier. You can go like this.

pipeline = Pipeline([
               ('vect', CountVectorizer(ngram_range=(1,3), max_df=0.50 ) ),
               ('tfidf', TfidfTransformer() ),
               ('clf', OneVsRestClassifier(SGDClassifier(loss='modified_huber', penalty='elasticnet',
                                          alpha=1e-4, n_iter=5, random_state=42,
                                          shuffle=True, n_jobs=-1) )),
                ])

Rest of the code can remain same. OneVsRestClassifier acts as a wrapper on other estimators.