How to use XGBoost algorithm for regression in R?

Question

I was trying the XGBoost technique for the prediction. As my dependent variable is continuous, I was doing the regression using XGBoost, but most of the references available in various portal are for classification. Though i know by using

objective = "reg:linear"

we can do the regression but still I need some clarity for other parameters as well. It would be a great help if somebody can provide me an R snippet of it.

Answer 1

xgboost(data = X, 
        booster = "gbtree", 
        objective = "binary:logistic", 
        max.depth = 5, 
        eta = 0.5, 
        nthread = 2, 
        nround = 2, 
        min_child_weight = 1, 
        subsample = 0.5, 
        colsample_bytree = 1, 
        num_parallel_tree = 1)

These are all the parameters you can play around with while using tree boosters. For linear booster you can use the following parameters to play with...

xgboost(data = X, 
        booster = "gblinear", 
        objective = "binary:logistic", 
        max.depth = 5, 
        nround = 2, 
        lambda = 0, 
        lambda_bias = 0, 
        alpha = 0)

You can refer to the description of xg.train() in the xgboost CRAN document for detailed meaning of these parameters.

Answer 2

The best description of the parameters that I have found is at

https://github.com/dmlc/xgboost/blob/master/doc/parameter.md

There are many examples of using XGBoost in R available in the Kaggle scripts repository. For example:

https://www.kaggle.com/michaelpawlus/springleaf-marketing-response/xgboost-example-0-76178/code