A Comprehensive Survey of Multiagent Reinforcement Learning

  title={A Comprehensive Survey of Multiagent Reinforcement Learning},
  author={Lucian Buşoniu and Robert Babu{\vs}ka and Bart De Schutter},
  journal={IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews)},
Multiagent systems are rapidly finding applications in a variety of domains, including robotics, distributed control, telecommunications, and economics. The complexity of many tasks arising in these domains makes them difficult to solve with preprogrammed agent behaviors. The agents must, instead, discover a solution on their own, using learning. A significant part of the research on multiagent learning concerns reinforcement learning techniques. This paper provides a comprehensive survey of… 

Figures and Tables from this paper

Multi-agent Reinforcement Learning: An Overview

This chapter reviews a representative selection of multi-agent reinforcement learning algorithms for fully cooperative, fully competitive, and more general (neither cooperative nor competitive) tasks.

Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms

This chapter reviews the theoretical results of MARL algorithms mainly within two representative frameworks, Markov/stochastic games and extensive-form games, in accordance with the types of tasks they address, i.e., fully cooperative, fully competitive, and a mix of the two.

Evolutionary Dynamics of Multi-Agent Learning: A Survey

This article surveys the dynamical models that have been derived for various multi-agent reinforcement learning algorithms, making it possible to study and compare them qualitatively, and provides a roadmap on the progress that has been achieved in analysing the evolutionary dynamics of multi- agent learning.

Communicating Intention in Decentralized Multi-Agent Multi-Objective Reinforcement Learning Systems

The method of enabling the agents to shape and communicate their intention using multi-objective reinforcement learning managed to demonstrate a faster learning process than the centralized MARL system in the complex scenarios, even with limited observability of the environment.

A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems

A taxonomy of solutions for the general knowledge reuse problem is defined, providing a comprehensive discussion of recent progress on knowledge reuse in Multiagent Systems (MAS) and of techniques for knowledge reuse across agents (that may be actuating in a shared environment or not).

Learning Cooperative Behaviours in Multiagent Reinforcement Learning

In this study, cooperative behaviours among agents were learned using the proposed context dependent multiagent SARSA algorithms (CDM-SARSA), which reduced the size of the state space considerably and confirmed that the proposed CDM- SARSA could learn cooperative behaviours successfully.

A survey on multi-agent reinforcement learning: Coordination problems

  • Young-Cheol ChoiH. Ahn
  • Computer Science
    Proceedings of 2010 IEEE/ASME International Conference on Mechatronic and Embedded Systems and Applications
  • 2010
A survey on coordination problems in cooperative multiagent reinforcement learning is provided, and a new approach to solve coordination problems is proposed.

Decentralized Multi-Agent Reinforcement Learning with Networked Agents: Recent Advances

This paper reviews some recent advances in decentralized MARL with networked agents, which finds broad applications in the control and operation of robots, unmanned vehicles, mobile sensor networks, and smart grid.

Deep Reinforcement Learning for Multiagent Systems: A Review of Challenges, Solutions, and Applications

A survey of different approaches to problems related to multiagent deep RL (MADRL) is presented, including nonstationarity, partial observability, continuous state and action spaces, multiagent training schemes, and multiagent transfer learning.

Reinforcement Learning in Decentralized MultiGoal Multi-Agent Settings Reinforcement learning in dezentralisierten

A novel scenario is created, where a test of social skill is singled out to measure how well an algorithm does in that regard, and it is shown that the newly created scenario distinguishes between social and non-social behavior through examination of the expected social behavior.



Cooperative Multi-Agent Learning: The State of the Art

This survey attempts to draw from multi-agent learning work in a spectrum of areas, including RL, evolutionary computation, game theory, complex systems, agent modeling, and robotics, and finds that this broad view leads to a division of the work into two categories.

Planning, Learning and Coordination in Multiagent Decision Processes

The extent to which methods from single-agent planning and learning can be applied in multiagent settings is investigated and the decomposition of sequential decision processes so that coordination can be learned locally, at the level of individual states.

Hierarchical reinforcement learning in communication-mediated multiagent coordination

The idea here is that “physical” acting can be preceded by communication to allow for a prediction of actions to come and to make two contributions to the solution of communication-mediated multiagent coordination problems.

Multiagent learning in the presence of agents with limitations

The thesis introduces a general model for the effect of limitations on agent behavior, and introduces GraWoLF, a general-purpose, scalable, multiagent learning algorithm that combines policy gradient learning techniques with the WoLF variable learning rate.

Multiagent reinforcement learning using function approximation

Two new multiagent based domain independent coordination mechanisms for reinforcement learning; multiple agents do not require explicit communication among themselves to learn coordinated behavior.

Multiagent learning using a variable learning rate

Learning Coordination Strategies for Cooperative Multiagent Systems

A new approach for learning multiagent coordination strategies that addresses these issues is presented and the effectiveness of the technique is demonstrated using a synthetic domain and the predator and prey pursuit problem.

Hierarchical multi-agent reinforcement learning

The multi-agent HRL framework is extended to include communication decisions and a cooperative multi- agent HRL algorithm called COM-Cooperative HRL is proposed, which allows agents to learn coordination faster by sharing information at the level of cooperative subtasks.

An Evolutionary Dynamical Analysis of Multi-Agent Learning in Iterated Games

It is shown how evolutionary dynamics from Evolutionary Game Theory can help the developer of a MAS in good choices of parameter settings of the used RL algorithms and how the improved results for MAS RL in COIN, and a developed extension, are predicted by the ED.