XGBoost for multilabel classification?

Question

Is it possible to use XGBoost for multi-label classification? Now I use OneVsRestClassifier over GradientBoostingClassifier from sklearn. It works, but use only one core from my CPU. In my data I have ~45 features and the task is to predict about 20 columns with binary (boolean) data. Metric is mean average precision (map@7). If you have a short example of code to share, that would be great.

Answer 1

One possible approach, instead of using OneVsRestClassifier which is for multi-class tasks, is to use MultiOutputClassifier from the sklearn.multioutput module.

Below is a small reproducible sample code with the number of input features and target outputs requested by the OP

import xgboost as xgb from sklearn.datasets import make_multilabel_classification from sklearn.model_selection import train_test_split from sklearn.multioutput import MultiOutputClassifier from sklearn.metrics import accuracy_score  # create sample dataset X, y = make_multilabel_classification(n_samples=3000, n_features=45, n_classes=20, n_labels=1,                                       allow_unlabeled=False, random_state=42)  # split dataset into training and test set X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=123)  # create XGBoost instance with default hyper-parameters xgb_estimator = xgb.XGBClassifier(objective='binary:logistic')  # create MultiOutputClassifier instance with XGBoost model inside multilabel_model = MultiOutputClassifier(xgb_estimator)  # fit the model multilabel_model.fit(X_train, y_train)  # evaluate on test data print('Accuracy on test data: {:.1f}%'.format(accuracy_score(y_test, multilabel_model.predict(X_test))*100))