scikit-learn pipeline

Question

Each sample in my (iid) dataset looks like this:
x = [a_1,a_2...a_N,b_1,b_2...b_M]

I also have the label of each sample (This is supervised learning)

The a features are very sparse (namely bag-of-words representation), while the b features are dense (integers,there are ~45 of those)

I am using scikit-learn, and I want to use GridSearchCV with pipeline.

The question: is it possible to use one CountVectorizer on features type a and another CountVectorizer on features type b?

What I want can be thought of as:

pipeline = Pipeline([
    ('vect1', CountVectorizer()), #will work only on features [0,(N-1)]
    ('vect2', CountVectorizer()), #will work only on features [N,(N+M-1)]
    ('clf', SGDClassifier()), #will use all features to classify
])

parameters = {
    'vect1__max_df': (0.5, 0.75, 1.0),       # type a features only
    'vect1__ngram_range': ((1, 1), (1, 2)),  # type a features only
    'vect2__max_df': (0.5, 0.75, 1.0),       # type b features only
    'vect2__ngram_range': ((1, 1), (1, 2)),  # type b features only
    'clf__alpha': (0.00001, 0.000001),
    'clf__penalty': ('l2', 'elasticnet'),
    'clf__n_iter': (10, 50, 80),
}

grid_search = GridSearchCV(pipeline, parameters, n_jobs=-1, verbose=1)
grid_search.fit(X, y)

Is that possible?

A nice idea was presented by @Andreas Mueller. However, I want to keep the original non-chosen features as well... therefore, I cannot tell the column index for each phase at the pipeline upfront (before the pipeline begins).

For example, if I set CountVectorizer(max_df=0.75), it may reduce some terms, and the original column index will change.

Thanks

Answer 1

Unfortunately, this is currently not as nice as it could be. You need to use FeatureUnion to concatenate to kinds of features, and the transformer in each needs to select the features and transform them. One way to do that is to make a pipeline of a transformer that selects the columns (you need to write that yourself) and the CountVectorizer. There is an example that does something similar here. That example actually separates the features as different values in a dictionary, but you don't need to do that. Also have a look at the related issue for selecting columns which contains code for the transformer that you need.

It would looks something like this with the current code:

make_pipeline(
    make_union(
        make_pipeline(FeatureSelector(some_columns), CountVectorizer()),
        make_pipeline(FeatureSelector(other_columns), CountVectorizer())),
    SGDClassifier())