Menu
I have a question about XGBoost.
Do you know how to know the number of tree created in XGBoost? Unlike RandomForest, which model maker decides how many trees are made, XGBoost basically continues to create the trees till the loss function reaches certain figure. Therefore I want to know this.
Thank you.
745
In practice, you'll typically see Gradient Boost being used with a maximum number of leaves of between 8 and 32.
Xgboost is a gradient boosting method, as such it adds trees in every iteration to improve the prediction accuracy.
In boosting, the trees are built sequentially such that each subsequent tree aims to reduce the errors of the previous tree. Each tree learns from its predecessors and updates the residual errors. Hence, the tree that grows next in the sequence will learn from an updated version of the residuals.
It's a bit crooked, but what I'm currently doing is dump-ing the model (XGBoost produces a list where each element is a string representation of a single tree), and then counting how many elements are in the list:
# clf is a XGBoost model fitted using the sklearn API
dump_list = clf.get_booster().get_dump()
num_trees = len(dump_list)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
