I am using the following code to get the optimised parameters for randomforest
using gridsearchcv
.
x_train, x_test, y_train, y_test = train_test_split(X, y, random_state=0)
rfc = RandomForestClassifier(random_state=42, class_weight = 'balanced')
param_grid = {
'n_estimators': [200, 500],
'max_features': ['auto', 'sqrt', 'log2'],
'max_depth' : [4,5,6,7,8],
'criterion' :['gini', 'entropy']
}
k_fold = StratifiedKFold(n_splits=10, shuffle=True, random_state=0)
CV_rfc = GridSearchCV(estimator=rfc, param_grid=param_grid, cv= 10, scoring = 'roc_auc')
CV_rfc.fit(x_train, y_train)
print(CV_rfc.best_params_)
print(CV_rfc.best_score_)
Now, I want to apply the tuned parameters to X_test
. For that I did the following,
pred = CV_rfc.decision_function(x_test)
print(roc_auc_score(y_test, pred))
However, decision_function
does not seem to support randomforest
as I got the following error.
AttributeError: 'RandomForestClassifier' object has no attribute 'decision_function'.
Is there any other way of doing this?
I am happy to provide more details if needed.
The documentation of oob_decision_function_ in RandomForestClassifier is: Decision function computed with out-of-bag estimate on the training set. If n_estimators is small it might be possible that a data point was never left out during the bootstrap.
If your intention is to get a model scoring function so that the scoring can be used for auc_roc_score
, then you can go for predict_proba()
y_pred_proba = CV_rfc.predict_proba(x_test)
print(roc_auc_score(y_test, y_pred_proba[:,1]))
you can use either predict() method or to get the optimized random forest model using best_estimator_
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With