• Corpus ID: 2884219

Computational Rationalization: The Inverse Equilibrium Problem

@inproceedings{Waugh2011ComputationalRT,
  title={Computational Rationalization: The Inverse Equilibrium Problem},
  author={K. Waugh and Brian D. Ziebart and J. Andrew Bagnell},
  booktitle={ICML},
  year={2011}
}
Modeling the purposeful behavior of imperfect agents from a small number of observations is a challenging task. When restricted to the single-agent decision-theoretic setting, inverse optimal control techniques assume that observed behavior is an approximately optimal solution to an unknown decision problem. These techniques learn a utility function that explains the example behavior and can then be used to accurately predict or imitate future behavior in similar observed or unobserved… 

Figures and Tables from this paper

Evaluating Strategic Structures in Multi-Agent Inverse Reinforcement Learning
TLDR
It is observed that rewards learned naively tend to lack insightful structure, which causes them to produce undesirable behavior when optimized in games with different players from those encountered during training.
Inverse Game Theory
One of the central questions in game theory deals with predicting the behavior of an agent. Here, we study the inverse of this problem: given the agents’ equilibrium behavior, what are possible
Inverse Game Theory: Learning Utilities in Succinct Games
TLDR
The results show that rationalizing an equilibrium is computationally easier than computing it; from a practical perspective a practitioner can use the algorithms to validate behavioral models.
Data-Driven Behaviour Estimation in Parametric Games
A central question in multi-agent strategic games deals with learning the underlying utilities driving the agents’ behaviour. Motivated by the increasing availability of large data-sets, we develop
An inverse correlated equilibrium framework for utility learning in multiplayer, noncooperative settings
TLDR
Through the learned utility functions from the fitness game, this work hopes to gain insight into the relative importance each user places on safeguarding their privacy vs. achieving the other desirable objectives in the game.
On the Existence of Low-Rank Explanations for Mixed Strategy Behavior
TLDR
This paper identifies three structural conditions and shows that if the data set of observed behavior satisfies any of these conditions, then it can be explained by payoff matrices for which Nash equilibria are efficiently computable.
Cooperative Inverse Reinforcement Learning
TLDR
It is shown that computing optimal joint policies in CIRL games can be reduced to solving a POMDP, it is proved that optimality in isolation is suboptimal in C IRL, and an approximate CirL algorithm is derived.
Large Scale Learning of Agent Rationality in Two-Player Zero-Sum Games
TLDR
This paper draws upon well-known ideas in decision theory to obtain a concise and interpretable agent behavior model, and derive solvers and gradients for end-to-end learning and proposes an efficient first-order primal-dual method which exploits the structure of extensive-form games.
Maximum-Entropy Multi-Agent Dynamic Games: Forward and Inverse Solutions
TLDR
A new notion of stochastic Nash equilibrium for boundedly rational agents, which is called the Entropic Cost Equilibrium (ECE), is defined and it is shown that ECE is a natural extension to multiple agents of Maximum Entropy optimality for single agents.
Multiagent Inverse Reinforcement Learning for Two-Person Zero-Sum Games
TLDR
A theoretical foundation for competitive two-agent zero-sum MIRL problems is established and a Bayesian solution approach is proposed in which the generative model is based on an assumption that the two agents follow a minimax bipolicy.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 87 REFERENCES
On adaptation, maximization, and reinforcement learning among cognitive strategies.
TLDR
This model of reinforcement learning among cognitive strategies (RELACS) captures the 3 deviations, the learning curves, and the effect of information on uncertainty avoidance and outperforms other models in fitting the data and in predicting behavior in other experiments.
Maximum Entropy Inverse Reinforcement Learning
TLDR
A probabilistic approach based on the principle of maximum entropy that provides a well-defined, globally normalized distribution over decision sequences, while providing the same performance guarantees as existing methods is developed.
Algorithmic Game Theory: Learning, Regret Minimization, and Equilibria
TLDR
This chapter presents algorithms for repeated play of a matrix game with the guarantee that against any opponent, they will perform nearly as well as the best fixed action in hindsight, and presents a general reduction showing how to convert any algorithm for minimizing external regret to one that minimizes this stronger form of regret as well.
Apprenticeship learning via inverse reinforcement learning
TLDR
This work thinks of the expert as trying to maximize a reward function that is expressible as a linear combination of known features, and gives an algorithm for learning the task demonstrated by the expert, based on using "inverse reinforcement learning" to try to recover the unknown reward function.
Quantal Response Equilibria for Normal Form Games
We investigate the use of standard statistical models for quantal choice in a game theoretic setting. Players choose strategies based on relative expected utility and assume other players do so as
Calibrated Learning and Correlated Equilibrium
TLDR
For each correlated equilibrium there is some calibrated learning rule that the players can use which results in their playing this correlated equilibrium in the limit, and the statistical concept of a calibration is strongly related to the game theoretic concept of correlated equilibrium.
Correlated Equilibrium and the Estimation of Discrete Games of Complete Information
In order to understand strategic interactions among firms, we often need to estimate the parameters of static discrete games with complete information. This class of games is difficult to estimate
Maximum margin planning
TLDR
This work learns mappings from features to cost so an optimal policy in an MDP with these cost mimics the expert's behavior, and demonstrates a simple, provably efficient approach to structured maximum margin learning, based on the subgradient method, that leverages existing fast algorithms for inference.
Help or Hinder: Bayesian Models of Social Goal Inference
TLDR
A model for how people can infer social goals from actions, based on inverse planning in multiagent Markov decision problems (MDPs), is proposed and behavioral evidence is presented in support of this model over a simpler, perceptual cue-based alternative.
Game theory, maximum entropy, minimum discrepancy and robust Bayesian decision theory
We describe and develop a close relationship between two problems that have customarily been regarded as distinct: that of maximizing entropy, and that of minimizing worst-case expected loss. Using a
...
1
2
3
4
5
...