Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why does Monte Carlo Tree Search reset Tree

I had a small but potentially stupid question about Monte Carlo Tree Search. I understand most of it but have been looking at some implementations and noticed that after the MCTS is run for a given state and a best move returned, the tree is thrown away. So for the next move, we have to run MCTS from scratch on this new state to get the next best position.

I was just wondering why we don't retain some of the information from the old tree. It seems like there is valuable information about the states in the old tree, especially given that the best move is one where the MCTS has explored most. Is there any particular reason we can't use this old information in some useful way?

like image 887
gowrath Avatar asked Nov 20 '17 10:11

gowrath


1 Answers

Some implementations do indeed retain the information.

For example, the AlphaGo Zero paper says:

The search tree is reused at subsequent time-steps: the child node corresponding to the played action becomes the new root node; the subtree below this child is retained along with all its statistics, while the remainder of the tree is discarded

like image 200
Peter de Rivaz Avatar answered Oct 01 '22 13:10

Peter de Rivaz