A Comprehensive Survey of Multiagent Reinforcement Learning

@article{Buoniu2008ACS,
  title={A Comprehensive Survey of Multiagent Reinforcement Learning},
  author={Lucian Buşoniu and Robert Babu{\vs}ka and Bart De Schutter},
  journal={IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews)},
  year={2008},
  volume={38},
  pages={156-172}
}
Multiagent systems are rapidly finding applications in a variety of domains, including robotics, distributed control, telecommunications, and economics. The complexity of many tasks arising in these domains makes them difficult to solve with preprogrammed agent behaviors. The agents must, instead, discover a solution on their own, using learning. A significant part of the research on multiagent learning concerns reinforcement learning techniques. This paper provides a comprehensive survey of… 

Figures and Tables from this paper

Multi-agent Reinforcement Learning: An Overview

TLDR
This chapter reviews a representative selection of multi-agent reinforcement learning algorithms for fully cooperative, fully competitive, and more general (neither cooperative nor competitive) tasks.

Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms

TLDR
This chapter reviews the theoretical results of MARL algorithms mainly within two representative frameworks, Markov/stochastic games and extensive-form games, in accordance with the types of tasks they address, i.e., fully cooperative, fully competitive, and a mix of the two.

Evolutionary Dynamics of Multi-Agent Learning: A Survey

TLDR
This article surveys the dynamical models that have been derived for various multi-agent reinforcement learning algorithms, making it possible to study and compare them qualitatively, and provides a roadmap on the progress that has been achieved in analysing the evolutionary dynamics of multi- agent learning.

Communicating Intention in Decentralized Multi-Agent Multi-Objective Reinforcement Learning Systems

TLDR
The method of enabling the agents to shape and communicate their intention using multi-objective reinforcement learning managed to demonstrate a faster learning process than the centralized MARL system in the complex scenarios, even with limited observability of the environment.

A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems

TLDR
A taxonomy of solutions for the general knowledge reuse problem is defined, providing a comprehensive discussion of recent progress on knowledge reuse in Multiagent Systems (MAS) and of techniques for knowledge reuse across agents (that may be actuating in a shared environment or not).

Learning Cooperative Behaviours in Multiagent Reinforcement Learning

TLDR
In this study, cooperative behaviours among agents were learned using the proposed context dependent multiagent SARSA algorithms (CDM-SARSA), which reduced the size of the state space considerably and confirmed that the proposed CDM- SARSA could learn cooperative behaviours successfully.

A survey on multi-agent reinforcement learning: Coordination problems

  • Young-Cheol ChoiH. Ahn
  • Computer Science
    Proceedings of 2010 IEEE/ASME International Conference on Mechatronic and Embedded Systems and Applications
  • 2010
TLDR
A survey on coordination problems in cooperative multiagent reinforcement learning is provided, and a new approach to solve coordination problems is proposed.

Decentralized Multi-Agent Reinforcement Learning with Networked Agents: Recent Advances

TLDR
This paper reviews some recent advances in decentralized MARL with networked agents, which finds broad applications in the control and operation of robots, unmanned vehicles, mobile sensor networks, and smart grid.

Deep Reinforcement Learning for Multiagent Systems: A Review of Challenges, Solutions, and Applications

TLDR
A survey of different approaches to problems related to multiagent deep RL (MADRL) is presented, including nonstationarity, partial observability, continuous state and action spaces, multiagent training schemes, and multiagent transfer learning.

Reinforcement Learning in Decentralized MultiGoal Multi-Agent Settings Reinforcement learning in dezentralisierten

TLDR
A novel scenario is created, where a test of social skill is singled out to measure how well an algorithm does in that regard, and it is shown that the newly created scenario distinguishes between social and non-social behavior through examination of the expected social behavior.
...

References

SHOWING 1-10 OF 147 REFERENCES

Cooperative Multi-Agent Learning: The State of the Art

TLDR
This survey attempts to draw from multi-agent learning work in a spectrum of areas, including RL, evolutionary computation, game theory, complex systems, agent modeling, and robotics, and finds that this broad view leads to a division of the work into two categories.

Planning, Learning and Coordination in Multiagent Decision Processes

TLDR
The extent to which methods from single-agent planning and learning can be applied in multiagent settings is investigated and the decomposition of sequential decision processes so that coordination can be learned locally, at the level of individual states.

Hierarchical reinforcement learning in communication-mediated multiagent coordination

TLDR
The idea here is that “physical” acting can be preceded by communication to allow for a prediction of actions to come and to make two contributions to the solution of communication-mediated multiagent coordination problems.

Multiagent learning in the presence of agents with limitations

TLDR
The thesis introduces a general model for the effect of limitations on agent behavior, and introduces GraWoLF, a general-purpose, scalable, multiagent learning algorithm that combines policy gradient learning techniques with the WoLF variable learning rate.

Multiagent reinforcement learning using function approximation

TLDR
Two new multiagent based domain independent coordination mechanisms for reinforcement learning; multiple agents do not require explicit communication among themselves to learn coordinated behavior.

Multiagent learning using a variable learning rate

Learning Coordination Strategies for Cooperative Multiagent Systems

TLDR
A new approach for learning multiagent coordination strategies that addresses these issues is presented and the effectiveness of the technique is demonstrated using a synthetic domain and the predator and prey pursuit problem.

Hierarchical multi-agent reinforcement learning

TLDR
The multi-agent HRL framework is extended to include communication decisions and a cooperative multi- agent HRL algorithm called COM-Cooperative HRL is proposed, which allows agents to learn coordination faster by sharing information at the level of cooperative subtasks.

An Evolutionary Dynamical Analysis of Multi-Agent Learning in Iterated Games

TLDR
It is shown how evolutionary dynamics from Evolutionary Game Theory can help the developer of a MAS in good choices of parameter settings of the used RL algorithms and how the improved results for MAS RL in COIN, and a developed extension, are predicted by the ED.
...