Understanding the Success of Perfect Information Monte Carlo Sampling in Game Tree Search

@inproceedings{Long2010UnderstandingTS,
  title={Understanding the Success of Perfect Information Monte Carlo Sampling in Game Tree Search},
  author={Jeffrey Richard Long and Nathan R Sturtevant and Michael Buro and Timothy Furtak},
  booktitle={AAAI Conference on Artificial Intelligence},
  year={2010},
  url={https://api.semanticscholar.org/CorpusID:583108}
}
Synthetic game trees are used to identify game properties that result in strong or weak performance for PIMC search as compared to an optimal player, and it is shown how these properties can be detected in real games and demonstrate that they do indeed appear to be good predictors of the strength of PimC search.
View via Publisher
ojs.aaai.org
94 Citations
Highly Influential Citations
9
Background Citations
43
Methods Citations
29
Results Citations
2

Figures and Tables from this paper

94 Citations

Monte Carlo Tree Search in Imperfect-Information Games

This thesis studies MCTS in two-player zero-sum extensive-form games with imperfect information and focuses on game-theoretic properties of the produced strategies and proposes explicit modelling of player’s beliefs about the probability of being in a specific game state during a match.

Comparison of Monte Carlo Tree Search Methods in the Imperfect Information Card Game Cribbage

The implementation of Cribbage for two players and several MCTS and non-MCTSbased AI players is described and it is found that Single-Observer Information Set M CTS performs well in this domain.

Monte Carlo Tree Search for games with hidden information and uncertainty

The ISMCTS algorithm is shown to outperform the existing approach of Perfect Information Monte Carlo (PIMC) search and can be used to solve two known issues with PIMC search, namely strategy fusion and non-locality.

Search in Imperfect Information Games Using Online Monte Carlo Counterfactual Regret Minimization

This paper presents Online Outcome Sampling (OOS), the first imperfect information search algorithm that is guaranteed to converge to an equilibrium strategy in two-player zero-sum games and shows that unlike with Information Set Monte Carlo Tree Search (ISMCTS), the exploitability of the strategies produced by OOS decreases as the amount of search time increases.

Determinization in Monte-Carlo Tree Search for the card game

This paper combines determinizati on techniques with MCTS for the popular Chinese card game Dou Di Zhu and shows that the abili ty to see opponents’ hidden cards in Dou Di Zhu is a signific ant advantage, which suggests that inference techniques could lead to much stronger play.

Recursive Monte Carlo search for imperfect information games

RecPIMC - a recursive IIMC search variant based on perfect information evaluation - performs considerably better than PIMC search in a large class of synthetic imperfect information games and the popular card game of Skat, for which PIMc search is the state-of-the-art cardplay algorithm.

Iterative Tree Search in General Game Playing with Incomplete Information

This paper adapts the classic idea of fictitious play to introduce an Iterative Tree Search algorithm for incomplete-information GGP and demonstrates both theoretically and experimentally that this algorithm provides an improvement over existing solutions on several classes of games that have been discussed in the literature.

Determinization and information set Monte Carlo Tree Search for the card game Dou Di Zhu

A novel variant of MCTS that operates directly on trees of information sets is introduced and it is shown that this algorithm performs well in precisely those situations where determinization using random deals performs poorly.

Enhancements in Monte Carlo tree search algorithms for biased game trees

The results showed that the bias in suboptimal moves degraded the performance of all algorithms and that the enhancement alleviated the effect caused by this property.

Monte Carlo Continual Resolving for Online Strategy Computation in Imperfect Information Games

A domain-independent formulation of CR applicable to any two-player zero-sum extensive-form games (EFGs) and an empirical comparison of MCCR with incremental tree building to Online Outcome Sampling and Information-set MCTS on several domains is presented.
...

8 References

GIB: Imperfect Information in a Computationally Challenging Game

GIB, the program being described, involves five separate technical advances: partition search, the practical application of Monte Carlo techniques to realistic problems, a focus on achievable sets to solve problems inherent in the Monte Carlo approach, an extension of alpha-beta pruning from total orders to arbitrary distributive lattices, and the use of squeaky wheel optimization to find approximately optimal solutions to cardplay problems.

Search in Games with Incomplete Information: A Case Study Using Bridge Card Play

Regret Minimization in Games with Incomplete Information

It is shown how minimizing counterfactual regret minimizes overall regret, and therefore in self-play can be used to compute a Nash equilibrium, and is demonstrated in the domain of poker, showing it can solve abstractions of limit Texas Hold'em with as many as 1012 states, two orders of magnitude larger than previous methods.

Approximating game-theoretic optimal strategies for full-scale poker

The computation of the first complete approximations of game-theoretic optimal strategies for full-scale poker is addressed, and linear programming solutions to the abstracted game are used to create substantially improved poker-playing programs.

An Analysis of UCT in Multi-Player Games

This paper provides an analysis of the UCT algorithm in multi-player games, showing that UCT, when run in a multi- player game, is computing a mixed-strategy equilibrium, as opposed to maxn, which computes a pure-str strategy equilibrium.

Improving State Evaluation, Inference, and Search in Trick-Based Card Games

This paper presents the world's first computer skat player that plays at the level of human experts, achieved by improving state evaluations using game data produced by human players and by using these state evaluations to perform inference on the unobserved hands of opposing players.

GIB: Imperfect Information in a Computationally Challenging Game