A Survey of Monte Carlo Tree Search Methods

@article{Browne2012ASO,
  title={A Survey of Monte Carlo Tree Search Methods},
  author={Cameron Browne and Edward Jack Powley and Daniel Whitehouse and Simon M. M. Lucas and Peter I. Cowling and Philipp Rohlfshagen and Stephen Tavener and Diego Perez Liebana and Spyridon Samothrakis and Simon Colton},
  journal={IEEE Transactions on Computational Intelligence and AI in Games},
  year={2012},
  volume={4},
  pages={1-43}
}
  • C. Browne, E. Powley, S. Colton
  • Published 3 February 2012
  • Computer Science
  • IEEE Transactions on Computational Intelligence and AI in Games
Monte Carlo tree search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. [] Key Method We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarize the results from the key game and nongame domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work.

Monte Carlo Tree Search in The Octagon Theory

TLDR
Comparisons between various policies and enhancements with the best known greedy approach and standard Monte Carlo Search reveal that the usage of Move Groups, Decisive Moves, Upper Confidence Bounds for Trees (UCT) and Limited Simulation Lengths turn a losing MCTS agent into the best performing one in a domain with estimated game-tree complexity of 10^293, even when the provided computational budget is kept low.

Computer Go and Monte Carlo Tree Search: Opening Book and Parallel Solutions

TLDR
A method to guide a Monte Carlo Tree Search in the initial moves of the game of Go, which matches the current state of a Go board against clusters of board configurations that are derived from a large number of games played by experts.

Monte Carlo Tree Search with Robust Exploration

TLDR
A new Monte-Carlo tree search method that focuses on identifying the best move and outperformed UCT and similar methods, except for trees having uniform width and depth.

Monte-Carlo Tree Search in Board Games

TLDR
This chapter gives an overview of popular and effective enhancements for board game playing mcts agents, and mentions techniques to parallelize mCTs in a straightforward but effective way.

Monte-Carlo Tree Search using Batch Value of Perfect Information

TLDR
This paper defines batch value of perfect information in game trees as a generalization of value of computation as proposed by Russell and Wefald, and uses it for selecting nodes to sample in MCTS.

Enhancements in Monte Carlo tree search algorithms for biased game trees

TLDR
The results showed that the bias in suboptimal moves degraded the performance of all algorithms and that the enhancement alleviated the effect caused by this property.

Monte Carlo Tree Search in Go

TLDR
This chapter reviews the development of current strong Go programs and the most important extensions to MCTS that are used in this game.

Incentive Learning in Monte Carlo Tree Search

TLDR
This paper proposes an MCTS improvement, called incentive learning, which learns the default policy online, based on ideas from combinatorial game theory, and hence is particularly useful when the underlying game is a sum of games.

Using evaluation functions in Monte-Carlo Tree Search

Exploratory analysis of the Monte Carlo tree search for solving the linear ordering problem

TLDR
This paper investigates the capability of the Monte-Carlo Tree Search algorithm to optimize permutation-based problems, making use of problem-specific knowledge, and introduces a Multi-Objective Optimization approach to deal with the exploration-exploitation dilemma during the tree search.
...

References

SHOWING 1-10 OF 277 REFERENCES

Evaluation Function Based Monte-Carlo LOA

TLDR
This paper investigates how to use a positional evaluation function in a Monte-Carlo simulation-based LOA program (MC-LOA), and finds that the Mixed strategy is the best among them.

Time Management for Monte-Carlo Tree Search Applied to the Game of Go

TLDR
Results indicate that clever time management can have a very significant effect on playing strength in the case of Monte-Carlo tree search.

Score Bounded Monte-Carlo Tree Search

TLDR
The proposed algorithm improves significantly a MCTS solver to take into account bounds on the possible scores of a node in order to select the nodes to explore in games that can end in draw positions.

Improved Monte-Carlo Search

TLDR
A new algorithm is introduced, UCT, which extends a bandit algorithm for Monte-Carlo search, and it is proven that the probability that the algorithm selects the correct move converges to 1.

Progressive Strategies for Monte-Carlo Tree Search

TLDR
Two progressive strategies for MCTS are introduced, called progressive bias and progressive unpruning, which enable the use of relatively time-expensive heuristic knowledge without speed reduction.

A Shogi Program Based on Monte-Carlo Tree Search

TLDR
An implementation of MCTS in shogi which combines techniques used in computer Go with a number of shogi-specific enhancements and it is observed that this program could solve certain opening and endgame positions that are considered hard to solve with the current methods.

Heuristics in Monte Carlo Go

TLDR
Experimental results show that the use of four heuristics used to bias the selection of moves during Monte Carlo sampling significantly improves the ability of the program to defeat GNU Go, a widely-used Go program based on traditional, knowledge-intensive techniques.

Monte-Carlo tree search in Ms. Pac-Man

TLDR
A performance comparison between the proposed system and existing programs showed significant improvement in the performance of proposed system over existing programs was observed in terms of its ability to survive, implying the effectiveness of proposed method.

αβ-based play-outs in Monte-Carlo Tree Search

TLDR
The newest version of the simulation-based LOA program, MC-LOAαβ, uses a selective 2-ply αβ-search at each step in its play-outs for choosing a move, and outperforms previous versions by a large margin.

Fast Approximate Max-n Monte Carlo Tree Search for Ms Pac-Man

TLDR
This work approached the problem by performing Monte Carlo tree searches on a five player maxn tree representation of the game of Ms Pac-Man with limited tree search depth, outperforming previous non-MCTS opponent approaches to the game by up to two orders of magnitude.
...