OnevsrestClassifier and random forest

Question

I am trying to reproduce the example here but using RandomForestClassifer.

I can't see how to transform this part of the code

# Learn to predict each class against the other
classifier = OneVsRestClassifier(svm.SVC(kernel='linear', probability=True,
                                 random_state=random_state))
y_score = classifier.fit(X_train, y_train).decision_function(X_test)

I tried

# Learn to predict each class against the other
classifier = OneVsRestClassifier(RandomForestClassifier())
y_score = classifier.fit(X_train, y_train).decision_function(X_test)

but I get

AttributeError: Base estimator doesn't have a decision_function attribute.

Is there a workaround?

Answer 1

Well you should know what is decision_function used for. Its only used with a SVM classifier reason being it gives out the distance of your data points from the hyperplane that separates the data, whereas when you do it using a RandomForestClassifier it makes no sense. You can use other methods that are supported by RFC. You can use predict_proba if you want to get the probabilities of your classified data points.

Here is the reference for the supported functions

Just to mention RFC do supports oob_decision_function, which is the out of bag estimate on your training set.

So just replace your line like -

y_score = classifier.fit(X_train, y_train).predict_proba(X_test)

or

y_score = classifier.fit(X_train, y_train).predict(X_test)