How does sample_weight compare to class_weight in scikit-learn?

Question

I would like to use sklearn.ensemble.GradientBoostingClassifier on an imbalanced classification problem. I intend to optimize for Area Under the Receiver Operating Characteristic Curve (ROC AUC). For this I would like to reweight my classes to make the small class more important to the classifier.

This would normally be done (in RandomForestClassifier for example) by setting class_weight = “balanced” but there is no such parameter in the GradientBoostingClassifier.

The documentation says:

The “balanced” mode uses the values of y to automatically adjust weights inversely proportional to class frequencies in the input data as n_samples / (n_classes * np.bincount(y))

If y_train is my dataframe of my target with elements in {0,1}, then the documentation implies that this should reproduce the same as class_weight = “balanced”

sample_weight = y_train.shape[0]/(2*np.bincount(y_train))
clf = ensemble.GradientBoostingClassifier(**params)
clf.fit(X_train, y_train,sample_weight = sample_weight[y_train.values])

Is this correct or am I missing something?

Answer 1

I would suggest you use the class_weight.compute_sample_weight utility in scikit-learn. For example:

from sklearn.utils.class_weight import compute_sample_weight
y = [1,1,1,1,0,0,1]
compute_sample_weight(class_weight='balanced', y=y)

Output:

array([ 0.7 ,  0.7 ,  0.7 ,  0.7 ,  1.75,  1.75,  0.7 ])

You can use this as input to the sample_weight keyword.