• Corpus ID: 232147848

Monte Carlo Tree Search: A Review of Recent Modifications and Applications

  title={Monte Carlo Tree Search: A Review of Recent Modifications and Applications},
  author={Maciej Swiechowski and Konrad Godlewski and Bartosz Sawicki and Jacek Ma'ndziuk},
Monte Carlo Tree Search (MCTS) is a powerful approach to designing game-playing bots or solving sequential decision problems. The method relies on intelligent tree search that balances exploration and exploitation. MCTS performs random sampling in the form of simulations and stores statistics of actions to make more educated choices in each subsequent iteration. The method has become a state-of-the-art technique for combinatorial games. However, in more complex games (e.g. those with a high… 
An Efficient Dynamic Sampling Policy For Monte Carlo Tree Search
A dynamic sampling tree policy is proposed that efficiently allocates limited computational budget to maximize the probability of correct selection of the best action at the root node of the tree.
Deep Learning and Artificial General Intelligence: Still a Long Way to Go
This article approaches this statement critically showing major reasons of why deep neural networks, as of the current state, are not ready to be the technique of choice for reaching AGI.
Developing a Successful Bomberman Agent
This paper presents various enhancements leading to improve the agents’ strength that concern search, opponent prediction, game state evaluation, and game engine encoding and reaches the top one position among the 2,300 AI agents submitted on the CodinGame arena.
A Fast Evolutionary adaptation for MCTS in Pommerman
This paper proposes a novel Evolutionary Monte Carlo Tree Search (FEMCTS) agent which borrows ideas from Evolutionary Algorthims and Monte CarloTree Search to play the game of Pommerman and performs almost as well as MCTS for most game seeds.
A Practical Solution to Handling Randomness and Imperfect Information in Monte Carlo Tree Search
Practical guidelines for developing strong AI agents based on the Monte Carlo Tree Search algorithm in a game with imperfect information and/or randomness are provided and a few novel ideas such as complete elimination of the so-called nature moves are introduced.
An In-depth Summary of Recent Artificial Intelligence Applications in Drug Design
This survey includes the theoretical development of the previously mentioned AI models and detailed summaries of 42 recent applications of AI in drug design, and provides a holistic discussion of the abundant applications so that the tasks, potential solutions, and challenges in AI-based drug design become evident.
Learning compositional programs with arguments and sampling
A state of the art model, AlphaNPI, is extended upon by learning to generate functions that can accept arguments, and an Approximate version of Monte Carlo Tree Search (A-MCTS) is investigated to speed up convergence.
Research on Monte Carlo Tree Search Method of Adaptive Resource Scheduling for Multi-agent Game
The design of the adaptive resource scheduling Monte Carlo tree search algorithm is proved to be effective in improving the performance of the game by comparing the search algorithms before and after the optimization.
Searching for More Efficient Dynamic Programs
This work describes a set of program transformations, a simple metric for assessing the efficiency of a transformed program, and a heuristic search procedure to improve this metric, and shows that in practice, automated search—like the mental search performed by human programmers—can find substantial improvements to the initial program.
Split Moves for Monte-Carlo Tree Search
A generalization of MCTS that works with arbitrarily split moves that can be greatly beneficial for singleas well as multi-action games is proposed and can be used for any kind of problem that is solvable via a simulation-based approach.


Move Prediction Using Deep Convolutional Neural Networks in Hex
It is shown that deep convolutional neural networks can be used to produce reliable move evaluation in the game of Hex, and the reigning Monte-Carlo-tree-search-based world champion player MoHex 2.0 can be enhanced.
Ilhan and Etaner-Uyar (2017)
  • Silver et al
  • 2016
A Problem Case for UCT
  • C. Browne
  • Business
    IEEE Transactions on Computational Intelligence and AI in Games
  • 2013
This paper examines a simple 5 × 5 Hex position that not only completely defeats flat Monte Carlo search, but also initially defeats plain upper confidence bounds for trees (UCT) search until an
Enhancing Monte Carlo Tree Search for Playing Hearthstone
Empirical evaluations of the proposed framework for Hearthstone are presented and experimental results suggest that the approach is well suited for developing a better AI agent.
Re-determinizing MCTS in Hanabi
  • J. Goodman
  • Computer Science
    2019 IEEE Conference on Games (CoG)
  • 2019
We introduce Re-determinizing IS-MCTS, a novel extension of Information Set Monte Carlo Tree Search (IS-MCTS) that prevents a leakage of hidden information into opponent models that can occur in
Stackelberg Equilibrium Approximation in General-Sum Extensive-Form Games with Double-Oracle Sampling Method
A comparison with two state-of-the-art exact methods for this genre of games shows that in vast majority of test cases the simulation-based approach leads to optimal Leader's strategies, while excelling both exact methods in terms of time scalability and much lower memory usage.
A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
This paper generalizes the AlphaZero approach into a single AlphaZero algorithm that can achieve superhuman performance in many challenging games, and convincingly defeated a world champion program in the games of chess and shogi (Japanese chess), as well as Go.
Decentralized Cooperative Planning for Automated Vehicles with Hierarchical Monte Carlo Tree Search
This work presents a Monte Carlo Tree Search (MCTS) based approach for decentralized cooperative planning using macro-actions for automated vehicles in heterogeneous environments, and evaluates the state-action-values of each agent in a cooperative and decentralized manner, explicitly modeling the interdependence of actions between traffic participants.
Multilabeled Value Networks for Computer Go
A new approach to a novel value network architecture for the game Go, called a multilabeled (ML) value network, where different values are trained simultaneously for different settings of komi, a compensation given to balance the initiative of playing first.