Menu
How to get access of individual trees of a xgboost model in python/R ?
Below I'm getting from Random Forest trees from sklearn.
estimator = RandomForestRegressor(
oob_score=True,
n_estimators=10,
max_features='auto'
)
estimator.fit(tarning_data,traning_target)
tree1 = estimator.estimators_[0]
leftChild = tree1.tree_.children_left
rightChild = tree1.tree_.children_right
782
The number of trees (or rounds) in an XGBoost model is specified to the XGBClassifier or XGBRegressor class in the n_estimators argument. The default in the XGBoost library is 100.
To make predictions we use the scikit-learn function model. predict(). By default, the predictions made by XGBoost are probabilities. Because this is a binary classification problem, each prediction is the probability of the input pattern belonging to the first class.
Do you want to inspect the trees?
In Python, you can dump the trees as a list of strings:
m = xgb.XGBClassifier(max_depth=2, n_estimators=3).fit(X, y)
m.get_booster().get_dump()
>
['0:[sincelastrun<23.2917] yes=1,no=2,missing=2\n\t1:[sincelastrun<18.0417] yes=3,no=4,missing=4\n\t\t3:leaf=-0.0965415\n\t\t4:leaf=-0.0679503\n\t2:[sincelastrun<695.025] yes=5,no=6,missing=6\n\t\t5:leaf=-0.0992546\n\t\t6:leaf=-0.0984374\n',
'0:[sincelastrun<23.2917] yes=1,no=2,missing=2\n\t1:[sincelastrun<16.8917] yes=3,no=4,missing=4\n\t\t3:leaf=-0.0928132\n\t\t4:leaf=-0.0676056\n\t2:[sincelastrun<695.025] yes=5,no=6,missing=6\n\t\t5:leaf=-0.0945284\n\t\t6:leaf=-0.0937463\n',
'0:[sincelastrun<23.2917] yes=1,no=2,missing=2\n\t1:[sincelastrun<18.175] yes=3,no=4,missing=4\n\t\t3:leaf=-0.0878571\n\t\t4:leaf=-0.0610089\n\t2:[sincelastrun<695.025] yes=5,no=6,missing=6\n\t\t5:leaf=-0.0904395\n\t\t6:leaf=-0.0896808\n']
Or dump them to a file (with nice formatting):
m.get_booster().dump_model("out.txt")
>
booster[0]:
0:[sincelastrun<23.2917] yes=1,no=2,missing=2
1:[sincelastrun<18.0417] yes=3,no=4,missing=4
3:leaf=-0.0965415
4:leaf=-0.0679503
2:[sincelastrun<695.025] yes=5,no=6,missing=6
5:leaf=-0.0992546
6:leaf=-0.0984374
booster[1]:
0:[sincelastrun<23.2917] yes=1,no=2,missing=2
1:[sincelastrun<16.8917] yes=3,no=4,missing=4
3:leaf=-0.0928132
4:leaf=-0.0676056
2:[sincelastrun<695.025] yes=5,no=6,missing=6
5:leaf=-0.0945284
6:leaf=-0.0937463
booster[2]:
0:[sincelastrun<23.2917] yes=1,no=2,missing=2
1:[sincelastrun<18.175] yes=3,no=4,missing=4
3:leaf=-0.0878571
4:leaf=-0.0610089
2:[sincelastrun<695.025] yes=5,no=6,missing=6
5:leaf=-0.0904395
6:leaf=-0.0896808
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
