Inverse Reinforcement Learning for Strategy Identification

  title={Inverse Reinforcement Learning for Strategy Identification},
  author={Mark Rucker and Stephen C. Adams and Roy Hayes and Peter A. Beling},
  journal={2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC)},
In adversarial environments, one side could gain an advantage by identifying the opponent’s strategy. For example, in combat games, if an opponent’s strategy is identified as overly aggressive, one could lay a trap that exploits the opponent’s aggressive nature. However, an opponent’s strategy is not always apparent and may need to be estimated from observations of their actions. This paper proposes to use inverse reinforcement learning (IRL) to identify strategies in adversarial environments… 

Figures and Tables from this paper


Imitation Learning via Kernel Mean Embedding
This work shows that the kernelization of a classical algorithm naturally reduces the imitation learning to a distribution learning problem, where the imitation policy tries to match the state-action visitation distribution of the expert.
Maximum Entropy Inverse Reinforcement Learning
A probabilistic approach based on the principle of maximum entropy that provides a well-defined, globally normalized distribution over decision sequences, while providing the same performance guarantees as existing methods is developed.
Inverse reinforcement learning with Gaussian process
  • Qifeng Qiao, P. Beling
  • Mathematics, Computer Science
    Proceedings of the 2011 American Control Conference
  • 2011
It is argued that finite-space IRL can be posed as a convex quadratic program under a Bayesian inference framework with the objective of maximum a posteriori estimation.
Apprenticeship learning via inverse reinforcement learning
This work thinks of the expert as trying to maximize a reward function that is expressible as a linear combination of known features, and gives an algorithm for learning the task demonstrated by the expert, based on using "inverse reinforcement learning" to try to recover the unknown reward function.
Inverse Reinforcement Learning for Strategy Extraction
This work model the game of table tennis as a Markov decision problem, where the reward function models the goal of the task as well as all strategic information, and shows that the resulting reward functions are able to distinguish the expert among players with different skill levels aswell as different playing styles.
Multiagent Inverse Reinforcement Learning for Two-Person Zero-Sum Games
A theoretical foundation for competitive two-agent zero-sum MIRL problems is established and a Bayesian solution approach is proposed in which the generative model is based on an assumption that the two agents follow a minimax bipolicy.
Agent-based model construction using inverse reinforcement learning
The experimental results show that the proposed method can extract rules and construct an agent-based model with rich but concise behavioral rules for agents while still maintaining aggregate-level properties.
Recognition of Agents Based on Observation of Their Sequential Behavior
Empirical comparisons of the method with several existing IRL algorithms and with direct methods that use feature statistics observed in state-action space suggest it may be superior for agent recognition problems, particularly when the state space is large but the length of the observed decision trajectory is small.
Identification of animal behavioral strategies by inverse reinforcement learning
An inverse reinforcement-learning (IRL) framework to identify an animal’s behavioral strategy from behavioral time-series data is developed and applied to C. elegans thermotactic behavior; after cultivation at a constant temperature with or without food, fed worms prefer, while starved worms avoid the cultivation temperature on a thermal gradient.
Reinforcement Learning: An Introduction
This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.