Evolutionary game theory and multi-agent reinforcement learning

@article{Tuyls2005EvolutionaryGT,
  title={Evolutionary game theory and multi-agent reinforcement learning},
  author={Karl Tuyls and Ann Now{\'e}},
  journal={The Knowledge Engineering Review},
  year={2005},
  volume={20},
  pages={63 - 90}
}
  • K. Tuyls, A. Nowé
  • Published 1 March 2005
  • Computer Science
  • The Knowledge Engineering Review
In this paper we survey the basics of reinforcement learning and (evolutionary) game theory, applied to the field of multi-agent systems. [] Key Method Next we summarize the most important aspects of evolutionary game theory. Finally, we discuss the state-of-the-art of multi-agent reinforcement learning and the mathematical connection with evolutionary game theory.
An Overview of Multi-Agent Reinforcement Learning from Game Theoretical Perspective
TLDR
This work provides a self-contained assessment of the current state-of-the-art MARL techniques from a game theoretical perspective and expects this work to serve as a stepping stone for both new researchers who are about to enter this fast-growing domain and existing domain experts who want to obtain a panoramic view and identify new directions based on recent advances.
Multi-agent learning dynamics
TLDR
This dissertation describes how methods from the field of evolutionary game theory can be used to gain qualitative and theoretical insight into the dynamics of these learning multi-agent systems, as their results are often hard to predict in advance.
The Evolutionary Dynamics of Independent Learning Agents in Population Games
TLDR
A novel unified framework for characterising population dynamics via a single partial differential equation (Theorem 1) is provided and extensive experimental results validating that Theorem 1 holds for a variety of learning methods and population games are presented.
Reinforcement learning for multi-agent systems
TLDR
A survey of multiagent reinforcement learning is given, starting with a review of the different viewpoints on the learning goal, which is a central issue in the field.
Dynamical systems as a level of cognitive analysis of multi-agent learning
  • W. Barfuss
  • Computer Science
    Neural Comput. Appl.
  • 2022
TLDR
This article proposes to embed the dynamical systems description of multi-agent learning into different abstraction levels of cognitive analysis, and proposes an on-line sample-batch temporal-difference algorithm which is characterized by the combination of applying a memory-batch and separated state-action value estimation.
Experience Weighted Learning in Multiagent Systems
TLDR
A learning algorithm for heterogeneous agents with bounded rationality that integrates reinforcement learning as well as fictitious play to evaluate the historical information and adopt mechanisms in evolutionary game to adapt to uncertainty, referred to as experience weighted learning (EWL) is proposed.
A Gradient-Based Reinforcement Learning Algorithm for Multiple Cooperative Agents
TLDR
The Q-value function is applied to estimate the gradient and derive the probability of maximal reward based on estimated gradient ascent (PMR-EGA) algorithm, which can be naturally extended to optimize cooperative stochastic games.
Game Theory and Multi-agent Reinforcement Learning
TLDR
A basic learning framework based on the economic research into game theory is described, and a representative selection of algorithms for the different areas of multi-agent reinforcement learning research is described.
Multi-Agent Reinforcement Learning: A Survey
TLDR
An integrated survey of the field of multi-agent learning is presented, in which the issue of the multi- agent learning goal is discussed and a representative selection of algorithms is reviewed.
...
...

References

SHOWING 1-10 OF 112 REFERENCES
Towards a relation between learning agents and evolutionary dynamics
TLDR
This paper examines whether evolutionary game theory, and more specifically the replicator dynamics, is an adequate theoretical model for the study of the dynamics of reinforcement learning agents in a multi-agent system.
Multiagent Reinforcement Learning in Stochastic Games
TLDR
This work designs a multiagent reinforcement learning method which allows agents to learn Nash equilibrium strategies and shows in both theory and experiments that this algorithm converges.
Extended Replicator Dynamics as a Key to Reinforcement Learning in Multi-agent Systems
TLDR
This paper introduces an extension of the Replicator Dynamics from Evolutionary Game Theory and develops a Reinforcement Learning algorithm that attains a stable Nash equilibrium for all types of games.
Markov Games as a Framework for Multi-Agent Reinforcement Learning
A selection-mutation model for q-learning in multi-agent systems
TLDR
This work shows how the Replicator Dynamics (RD) can be used as a model for Q-learning in games and reveals an interesting connection between the exploitation-exploration scheme from RL and the selection-mutation mechanisms from evolutionary game theory.
On a Dynamical Analysis of Reinforcement Learning in Games: Emergence of Occam's Razor
TLDR
This paper discusses three learning models whose dynamics are related to the Replicator Dynamics and shows how a classical Reinforcement Learning(RL) technique, i.e. Q-learning relates to the RD, allows to better understand the learning process and it allows to determine how complex a RL model should be.
Evolutionary Game Theory
This text introduces current evolutionary game theory--where ideas from evolutionary biology and rationalistic economics meet--emphasizing the links between static and dynamic approaches and
Evolutionary Games and Equilibrium Selection
Evolutionary game theory is one of the most active and rapidly growing areas of research in economics. Unlike traditional game theory models, which assume that all players are fully rational and have
Multiagent reinforcement learning in Markov games : asymmetric and symmetric approaches
TLDR
Interactions between intelligent and rational agents are modeled with Markov games and the emphasis is on the adaptation and learning in multiagent systems.
Game theory and economics
Introduction to Game Theory and Outline of the Book Optimal Decentralised Decisions Non-Cooperative Games with Complete and Perfect Information Non-Cooperative Games with Imperfect or Incomplete
...
...