A Survey of Monte Carlo Tree Search Methods

  title={A Survey of Monte Carlo Tree Search Methods},
  author={Cameron Browne and Edward Jack Powley and Daniel Whitehouse and Simon M. M. Lucas and Peter I. Cowling and Philipp Rohlfshagen and Stephen Tavener and Diego Perez Liebana and Spyridon Samothrakis and Simon Colton},
  journal={IEEE Transactions on Computational Intelligence and AI in Games},
  • C. BrowneE. Powley S. Colton
  • Published 3 February 2012
  • Computer Science
  • IEEE Transactions on Computational Intelligence and AI in Games
Monte Carlo tree search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. [] Key Method We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarize the results from the key game and nongame domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work.

Monte Carlo Tree Search in The Octagon Theory

Comparisons between various policies and enhancements with the best known greedy approach and standard Monte Carlo Search reveal that the usage of Move Groups, Decisive Moves, Upper Confidence Bounds for Trees (UCT) and Limited Simulation Lengths turn a losing MCTS agent into the best performing one in a domain with estimated game-tree complexity of 10^293, even when the provided computational budget is kept low.

Computer Go and Monte Carlo Tree Search: Opening Book and Parallel Solutions

A method to guide a Monte Carlo Tree Search in the initial moves of the game of Go, which matches the current state of a Go board against clusters of board configurations that are derived from a large number of games played by experts.

Research Summary

  • H. Baier
  • Computer Science
    Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment
  • 2021
This research summary describes the first steps taken in addressing the problem of understanding the nature, the underlying principles, of MCTS, as well as the plans for future work.

Monte Carlo Tree Search with Robust Exploration

A new Monte-Carlo tree search method that focuses on identifying the best move and outperformed UCT and similar methods, except for trees having uniform width and depth.

Monte Carlo Tree Search: A Review of Recent Modifications and Applications

In more complex games (e.g. those with a high branching factor or real-time ones), an efficient MCTS application often requires its problem-dependent modification or integration with other techniques and domain-specific modifications and hybrid approaches are the main focus of this survey.

Monte-Carlo Tree Search in Board Games

This chapter gives an overview of popular and effective enhancements for board game playing mcts agents, and mentions techniques to parallelize mCTs in a straightforward but effective way.

Monte-Carlo Tree Search using Batch Value of Perfect Information

This paper defines batch value of perfect information in game trees as a generalization of value of computation as proposed by Russell and Wefald, and uses it for selecting nodes to sample in MCTS.

Enhancements in Monte Carlo tree search algorithms for biased game trees

The results showed that the bias in suboptimal moves degraded the performance of all algorithms and that the enhancement alleviated the effect caused by this property.

Monte Carlo Tree Search in Go

This chapter reviews the development of current strong Go programs and the most important extensions to MCTS that are used in this game.

Incentive Learning in Monte Carlo Tree Search

This paper proposes an MCTS improvement, called incentive learning, which learns the default policy online, based on ideas from combinatorial game theory, and hence is particularly useful when the underlying game is a sum of games.



Evaluation Function Based Monte-Carlo LOA

This paper investigates how to use a positional evaluation function in a Monte-Carlo simulation-based LOA program (MC-LOA), and finds that the Mixed strategy is the best among them.

Time Management for Monte-Carlo Tree Search Applied to the Game of Go

Results indicate that clever time management can have a very significant effect on playing strength in the case of Monte-Carlo tree search.

Score Bounded Monte-Carlo Tree Search

The proposed algorithm improves significantly a MCTS solver to take into account bounds on the possible scores of a node in order to select the nodes to explore in games that can end in draw positions.

Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search

A new framework to combine tree search with Monte-Carlo evaluation, that does not separate between a min-max phase and a Monte- carlo phase is presented, that provides finegrained control of the tree growth, at the level of individual simulations, and allows efficient selectivity.

Progressive Strategies for Monte-Carlo Tree Search

Two progressive strategies for MCTS are introduced, called progressive bias and progressive unpruning, which enable the use of relatively time-expensive heuristic knowledge without speed reduction.

A Shogi Program Based on Monte-Carlo Tree Search

An implementation of MCTS in shogi which combines techniques used in computer Go with a number of shogi-specific enhancements and it is observed that this program could solve certain opening and endgame positions that are considered hard to solve with the current methods.

Heuristics in Monte Carlo Go

Experimental results show that the use of four heuristics used to bias the selection of moves during Monte Carlo sampling significantly improves the ability of the program to defeat GNU Go, a widely-used Go program based on traditional, knowledge-intensive techniques.

Monte-Carlo tree search in Ms. Pac-Man

A performance comparison between the proposed system and existing programs showed significant improvement in the performance of proposed system over existing programs was observed in terms of its ability to survive, implying the effectiveness of proposed method.

αβ-based play-outs in Monte-Carlo Tree Search

The newest version of the simulation-based LOA program, MC-LOAαβ, uses a selective 2-ply αβ-search at each step in its play-outs for choosing a move, and outperforms previous versions by a large margin.

Fast Approximate Max-n Monte Carlo Tree Search for Ms Pac-Man

This work approached the problem by performing Monte Carlo tree searches on a five player maxn tree representation of the game of Ms Pac-Man with limited tree search depth, outperforming previous non-MCTS opponent approaches to the game by up to two orders of magnitude.