• Corpus ID: 3003190

Solving Imperfect Information Games Using Decomposition

@inproceedings{Burch2014SolvingII,
  title={Solving Imperfect Information Games Using Decomposition},
  author={Neil Burch and Michael Bradley Johanson and Michael Bowling},
  booktitle={AAAI},
  year={2014}
}
Decomposition, i.e., independently analyzing possible subgames, has proven to be an essential principle for effective decision-making in perfect information games. However, in imperfect information games, decomposition has proven to be problematic. To date, all proposed techniques for decomposition in imperfect information games have abandoned theoretical guarantees. This work presents the first technique for decomposing an imperfect information game into subgames that can be solved… 

Figures and Topics from this paper

Time and Space: Why Imperfect Information Games are Hard
TLDR
The thesis introduces an analysis of counterfactual regret minimisation (CFR), an algorithm for solving extensive-form games, and presents tighter regret bounds that describe the rate of progress, as well as presenting a series of theoretical tools for using decomposition, and creating algorithms which operate on small portions of a game at a time.
Endgame Solving in Large Imperfect-Information Games
TLDR
Theoretically, it is shown that endgame solving can produce highly exploitable strategies in some games; however, it can guarantee a low exploitability in certain games where the opponent is given sufficient exploitative power within the endgame.
Scalable sub-game solving for imperfect-information games
TLDR
A framework for imperfect-information games is designed, which can not only deal with two- player imperfect- information games but also can efficiently solve three-player imperfect- Information games and proposes real-time hand abstraction (RTHA), which can reduce the error caused by the abstraction.
Safe and Nested Subgame Solving for Imperfect-Information Games
TLDR
This work introduces subgame-solving techniques that outperform prior methods both in theory and practice and shows that subgame solving can be repeated as the game progresses down the game tree, leading to far lower exploitability.
Monte Carlo Continual Resolving for Online Strategy Computation in Imperfect Information Games
TLDR
A domain-independent formulation of CR applicable to any two-player zero-sum extensive-form games (EFGs) and an empirical comparison of MCCR with incremental tree building to Online Outcome Sampling and Information-set MCTS on several domains is presented.
Subgame solving without common knowledge
TLDR
This work provides a method where safety is achieved by limiting the infosets at which subgame solving is performed, and proves that affine equilibria cannot be exploited by any Nash strategy of the opponent, so an opponent who wishes to exploit must open herself to counter-exploitation.
Sound Search in Imperfect Information Games
TLDR
The definition of soundness and the consistency hierarchy finally provide appropriate tools to analyze online algorithms in imperfect information games.
Search in Imperfect Information Games Using Online Monte Carlo Counterfactual Regret Minimization
TLDR
This paper presents Online Outcome Sampling (OOS), the first imperfect information search algorithm that is guaranteed to converge to an equilibrium strategy in two-player zero-sum games and shows that unlike with Information Set Monte Carlo Tree Search (ISMCTS), the exploitability of the strategies produced by OOS decreases as the amount of search time increases.
Online Monte Carlo Counterfactual Regret Minimization for Search in Imperfect Information Games
TLDR
It is shown that OOS can overcome the problem of non-locality encountered by previous search algorithms and perform well against its worst-case opponents and that preexisting Information Set Monte Carlo tree search (ISMCTS) can get more exploitable over time.
Depth-Limited Solving for Imperfect-Information Games
TLDR
A principled way to conduct depth-limited solving in imperfect-information games by allowing the opponent to choose among a number of strategies for the remainder of the game at the depth limit, which defeats two prior top agents using only a 4-core CPU and 16 GB of memory.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 18 REFERENCES
CFR-D: Solving Imperfect Information Games Using Decomposition
TLDR
CFR-D is introduced, a new variant of the counterfactual regret minimising family of algorithms which can handle the trunk and each subproblem independently and has memory requirements which are sub-linear in the number of decision points, a desirable property more commonly associated with perfect information algorithms.
Improving Performance in Imperfect-Information Games with Large State and Action Spaces by Solving Endgames
TLDR
This work introduces techniques that enable one to conduct endgame solving in a scalable way even when the number of states and actions in the game is large, and discusses each of these topics in detail.
Finding Optimal Abstract Strategies in Extensive-Form Games
TLDR
This work presents for the first time an algorithm which efficiently finds optimal abstract strategies -- strategies with minimal exploitability in the unabstracted game -- and uses this technique to find the least exploitable strategy ever reported for two-player limit Texas hold'em.
Regret Minimization in Games with Incomplete Information
TLDR
It is shown how minimizing counterfactual regret minimizes overall regret, and therefore in self-play can be used to compute a Nash equilibrium, and is demonstrated in the domain of poker, showing it can solve abstractions of limit Texas Hold'em with as many as 1012 states, two orders of magnitude larger than previous methods.
Efficient Nash equilibrium approximation through Monte Carlo counterfactual regret minimization
TLDR
This work presents new sampling techniques that consider sets of chance outcomes during each traversal to produce slower, more accurate iterations of Counterfactual Regret Minimization, and demonstrates that this new CFR update converges more quickly than chance-sampled CFR in the large domains of poker and Bluff.
Monte Carlo Sampling for Regret Minimization in Extensive Games
TLDR
A general family of domain-independent CFR sample-based algorithms called Monte Carlo counterfactual regret minimization (MCCFR) is described, of which the original and poker-specific versions are special cases.
Abstraction pathologies in extensive games
TLDR
This paper shows that the standard approach to finding strong strategies for large extensive games rests on shaky ground, and shows that pathologies arise when abstracting both chance nodes as well as a player's actions.
Efficient Computation of Equilibria for Extensive Two-Person Games
Abstract The Nash equilibria of a two-person, non-zero-sum game are the solutions of a certain linear complementarity problem (LCP). In order to use this for solving a game in extensive form, the
Evaluating state-space abstractions in extensive-form games
TLDR
This paper uses a recent metric for abstraction quality and examines imperfect recall abstractions, in which agents "forget" previously observed information to focus the abstraction effort on more recent and relevant state information, and presents experimental results in the domain of Texas hold'em poker.
Strategy Grafting in Extensive Games
TLDR
This paper presents a new method for computing strategies in large games that allows us to compute more expressive strategies without increasing the size of abstract games that the authors are required to solve.
...
1
2
...