Menu
I am currently testing Support Vector Regression (SVR) for a regression problem with two outputs. This means that Y_train_data has two values for each sample. Since SVR can only produce a single output, I use the MultiOutputRegressor from scikit.
from sklearn.svm import SVR
from sklearn.multioutput import MultiOutputRegressor
svr_reg = MultiOutputRegressor(SVR(kernel=_kernel, C=_C, gamma=_gamma, degree=_degree, coef0=_coef0))
svr_reg.fit(X_train_data, Y_train_data)
Now I have noticed that even after hyperparameter optimization, SVR delivers significantly worse results than single decision trees.
Is this a known problem when using SVR with multiple outputs?
And would it better to create two seperate SVR models with different hyperparameters?
597
Multi-output regression involves predicting two or more numerical variables. Unlike normal regression where a single value is predicted for each sample, multi-output regression requires specialized machine learning algorithms that support outputting multiple variables for each prediction.
New in version 1.6. Starting from version 1.6, XGBoost has experimental support for multi-output regression and multi-label classification with Python package. Multi-label classification usually refers to targets that have multiple non-exclusive class labels.
A random forest regressor is used, which supports multi-output regression natively, so the results can be compared. The random forest regressor will only ever predict values within the range of observations or closer to zero for each of the targets.
January 21, 2022. Multi-output classification is a type of machine learning that predicts multiple outputs simultaneously. In multi-output classification, the model will give two or more outputs after making any prediction. In other types of classifications, the model usually predicts only a single output.
As indicated in the comments by Vivek Kumar your SVR might perform worse, because it does not calculate outputs simultaneously to take correlations between outputs into account. Hence I would propose to use an adapted SVR regression that is able to handle multiple targets such as Multiple-output Support Vector Regression. You can find an implementation here. You can then compare the results with your output from MultiOutputRegressor to check for improvements.
96
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
