GridSearchCV - FitFailedWarning: Estimator fit failed

Question

I am running this:

# Hyperparameter tuning - Random Forest #

# Hyperparameters' grid
parameters = {'n_estimators': list(range(100, 250, 25)), 'criterion': ['gini', 'entropy'], 
              'max_depth': list(range(2, 11, 2)), 'max_features': [0.1, 0.2, 0.3, 0.4, 0.5], 
              'class_weight': [{0: 1, 1: i} for i in np.arange(1, 4, 0.2).tolist()], 'min_samples_split': list(range(2, 7))}


# Instantiate random forest
from sklearn.ensemble import RandomForestClassifier
classifier = RandomForestClassifier(random_state=0)


# Execute grid search and retrieve the best classifier
from sklearn.model_selection import GridSearchCV
classifiers_grid = GridSearchCV(estimator=classifier, param_grid=parameters, scoring='balanced_accuracy',
                                   cv=5, refit=True, n_jobs=-1)
classifiers_grid.fit(X, y)

and I am receiving this warning:

.../anaconda/lib/python3.7/site-packages/sklearn/model_selection/_validation.py:536: 
FitFailedWarning: Estimator fit failed. The score on this train-test partition for these parameters will be set to nan. Details: 
TypeError: '<' not supported between instances of 'str' and 'int'

Why is this and how can I fix it?

Answer 1

I had similar issue of FitFailedWarning with different details, after many runs I found, the parameter value passing has the error, try

parameters = {'n_estimators': [100,125,150,175,200,225,250], 
              'criterion': ['gini', 'entropy'], 
              'max_depth': [2,4,6,8,10], 
              'max_features': [0.1, 0.2, 0.3, 0.4, 0.5], 
              'class_weight': [0.2,0.4,0.6,0.8,1.0],               
              'min_samples_split': [2,3,4,5,6,7]}

This will pass for sure, for me it happened in XGBClassifier, somehow the values datatype mixing up

One more is if the value exceeds the range, for example in XGBClassifier 'subsample' paramerters max value is 1.0, if it is set as 1.1, FitFailedWarning will occur

Answer 2

For me this was giving same error but after removing none from max_dept it is fitting properly.

param_grid={'n_estimators':[100,200,300,400,500],
            'criterion':['gini', 'entropy'],
            'max_depth':['None',5,10,20,30,40,50,60,70],
            'min_samples_split':[5,10,20,25,30,40,50],
            'max_features':[ 'sqrt', 'log2'],
            'max_leaf_nodes':[5,10,20,25,30,40,50],
            'min_samples_leaf':[1,100,200,300,400,500]
            }

code which is running properly:

param_grid={'n_estimators':[100,200,300,400,500],
            'criterion':['gini', 'entropy'],
            'max_depth':[5,10,20,30,40,50,60,70],
            'min_samples_split':[5,10,20,25,30,40,50],
            'max_features':[ 'sqrt', 'log2'],
            'max_leaf_nodes':[5,10,20,25,30,40,50],
            'min_samples_leaf':[1,100,200,300,400,500]
            }