• Corpus ID: 6925519

Learning Multiagent Communication with Backpropagation

@inproceedings{Sukhbaatar2016LearningMC,
  title={Learning Multiagent Communication with Backpropagation},
  author={Sainbayar Sukhbaatar and Arthur D. Szlam and Rob Fergus},
  booktitle={NIPS},
  year={2016}
}
Many tasks in AI require the collaboration of multiple agents. Typically, the communication protocol between agents is manually specified and not altered during training. In this paper we explore a simple neural model, called CommNet, that uses continuous communication for fully cooperative tasks. The model consists of multiple agents and the communication between them is learned alongside their policy. We apply this model to a diverse set of tasks, demonstrating the ability of the agents to… 
Decentralization of Multiagent Policies by Learning What to Communicate
TLDR
This work presents an agent architecture and training methodology using neural networks to learn task-oriented communication semantics based on the example of a communication-unaware expert policy and illustrates the system’s ability to handle dynamically changing numbers of agents.
Learning to Ground Multi-Agent Communication with Autoencoders
TLDR
It is found that a standard representation learning algorithm – autoencoding – is sufficient for arriving at a grounded common language, which facilitates decentralized multi-agent communication and coordination.
Biases for Emergent Communication in Multi-agent Reinforcement Learning
TLDR
This work introduces inductive biases for positive signalling and positive listening, which ease the learning problem in emergent communication and applies these methods to a more extended environment, showing that agents with these inductive bias achieve better performance.
Networked Multi-Agent Reinforcement Learning with Emergent Communication
TLDR
This work forms and studies a MARL problem where cooperative agents are connected to each other via a fixed underlying network, and proposes a method for training these agents using emergent communication, the only work that performs an in depth analysis of emergent Communication in a networked MARL setting while being applicable to a broad class of problems.
Multi Agent Deep Learning with Cooperative Communication
TLDR
This article describes Asynchronous Advantage Actor-Critic with Communication (A3C2), an end-to-end differentiable approach where agents learn policies and communication protocols simultaneously, which outperforms other state-of-the-art proposals in multiple environments.
Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks
TLDR
This paper presents Individualized Controlled Continuous Communication Model (IC3Net) which has better training efficiency than simple continuous communication model, and can be applied to semi-cooperative and competitive settings along with the cooperative settings.
When Does Communication Learning Need Hierarchical Multi-Agent Deep Reinforcement Learning
TLDR
A hierarchical deep reinforcement learning model for multi-agent systems that separates the communication and coordination task from the action picking through a hierarchical policy is contributed.
Learning to Communicate with Deep Multi-Agent Reinforcement Learning
TLDR
By embracing deep neural networks, this work is able to demonstrate end-to-end learning of protocols in complex environments inspired by communication riddles and multi-agent computer vision problems with partial observability.
Multi-agent Neural Reinforcement-Learning System with Communication
TLDR
An neural-based actor-critic algorithm is proposed, which learns communication protocols between agents and implicitly shares information during the learning phase, which can reliably learn complex strategies and protocols for partially observable multi-agent environments.
Policy Distillation and Value Matching in Multiagent Reinforcement Learning
TLDR
A multiagent actor-critic algorithm for combining knowledge from homogeneous agents through distillation and value-matching that outperforms policy distillation alone and allows further learning in discrete and continuous action spaces is introduced.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 46 REFERENCES
Learning Communication for Multi-agent Systems
TLDR
Simulations show that introducing noise to the decision process of the hard-coded predators allow them to significantly perform all previously published work on similar preys, which indicates that the system was able to learn superior communicating strategies not readily available to the human designer.
Learning to Communicate to Solve Riddles with Deep Distributed Recurrent Q-Networks
TLDR
Empirical results on two multi-agent learning problems based on well-known riddles are presented, demonstrating that DDRQN can successfully solve such tasks and discover elegant communication protocols to do so, the first time deep reinforcement learning has succeeded in learning communication protocols.
QueryPOMDP: POMDP-Based Communication in Multiagent Systems
TLDR
The experimental results show that the approach successfully exploits sparse interactions: the approach can effectively identify the situations in which it is beneficial to communicate, as well as trade off the cost of communication with overall task performance.
Coordination of Communication in Robot Teams by Reinforcement Learning
TLDR
This paper presents the application of Reinforcement Learning to the self-emergence of a common lexicon in robot teams and shows that RL allows the convergence to lexicon consensus in a population of autonomous agents.
A Comprehensive Survey of Multiagent Reinforcement Learning
TLDR
The benefits and challenges of MARL are described along with some of the problem domains where the MARL techniques have been applied, and an outlook for the field is provided.
Multiagent cooperation and competition with deep reinforcement learning
TLDR
The present work shows that Deep Q-Networks can become a useful tool for studying decentralized learning of multiagent systems coping with high-dimensional environments and describes the progression from competitive to collaborative behavior when the incentive to cooperate is increased.
Coordinating multi-agent reinforcement learning with limited communication
TLDR
This paper develops a learning approach that generalizes previous coordinated MARL approaches that use DCOP algorithms and enables MARL to be conducted over a spectrum from independent learning (without communication) to fully coordinated learning depending on agents' communication bandwidth.
Multiagent Planning with Factored MDPs
TLDR
This work presents a principled and efficient planning algorithm for cooperative multiagent dynamic systems that avoids the exponential blowup in the state and action space and is an efficient alternative to more complicated algorithms even in the single agent case.
Learning of communication codes in multi-agent reinforcement learning problem
TLDR
This paper considers pursuit problem for multi-agent reinforcement learning with communication between the agents, and it has been shown that, in learning precision, the case of 1 bit or more bits communication outperformed the cases of no communication.
Efficient Distributed Reinforcement Learning through Agreement
TLDR
This paper addresses the twin problems of limited local experience and locally observed but not necessarily telling reward signals encountered in distributed robotic systems and combines direct search in policy space with an agreement algorithm to efficiently exchange local rewards and experience among agents.
...
1
2
3
4
5
...