• Corpus ID: 232045971

Learning Emergent Discrete Message Communication for Cooperative Reinforcement Learning

  title={Learning Emergent Discrete Message Communication for Cooperative Reinforcement Learning},
  author={Sheng Li and Yutai Zhou and R. Allen and Mykel J. Kochenderfer},
Communication is a important factor that enables agents work cooperatively in multi-agent reinforcement learning (MARL). Most previous work uses continuous message communication whose high representational capacity comes at the expense of interpretability. Allowing agents to learn their own discrete message communication protocol emerged from a variety of domains can increase the interpretability for human designers and other agents.This paper proposes a method to generate discrete messages… 


Learning Multiagent Communication with Backpropagation
A simple neural model is explored, called CommNet, that uses continuous communication for fully cooperative tasks and the ability of the agents to learn to communicate amongst themselves is demonstrated, yielding improved performance over non-communicative agents and baselines.
Learning Attentional Communication for Multi-Agent Cooperation
This paper proposes an attentional communication model that learns when communication is needed and how to integrate shared information for cooperative decision making and shows the strength of the model in a variety of cooperative scenarios, where agents are able to develop more coordinated and sophisticated strategies than existing methods.
TarMAC: Targeted Multi-Agent Communication
This work proposes a targeted communication architecture for multi-agent reinforcement learning, where agents learn both what messages to send and whom to address them to while performing cooperative tasks in partially-observable environments, and augment this with a multi-round communication approach.
Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks
This paper presents Individualized Controlled Continuous Communication Model (IC3Net) which has better training efficiency than simple continuous communication model, and can be applied to semi-cooperative and competitive settings along with the cooperative settings.
Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning
Empirical results demonstrate that influence leads to enhanced coordination and communication in challenging social dilemma environments, dramatically increasing the learning curves of the deep RL agents, and leading to more meaningful learned communication protocols.
On the Pitfalls of Measuring Emergent Communication
By training deep reinforcement learning agents to play simple matrix games augmented with a communication channel, this paper finds a scenario where agents appear to communicate, and yet the messages do not impact the environment or other agent in any way.
Cooperative Multi-agent Control Using Deep Reinforcement Learning
It is shown that policy gradient methods tend to outperform both temporal-difference and actor-critic methods and that curriculum learning is vital to scaling reinforcement learning algorithms in complex multi-agent domains.
Attentional Policies for Cross-Context Multi-Agent Reinforcement Learning
This work follows the spirit of recent work on the power of relational inductive biases in deep networks by learning multi-agent relationships at the policy level via an attentional architecture, and shows superior results to a full-knowledge, fully-centralized reference solution and significantly outperforming it when scaling to large numbers of agents.
Actor-Attention-Critic for Multi-Agent Reinforcement Learning
This work presents an actor-critic algorithm that trains decentralized policies in multi-agent settings, using centrally computed critics that share an attention mechanism which selects relevant information for each agent at every timestep, which enables more effective and scalable learning in complex multi- agent environments, when compared to recent approaches.
Emergent Communication in a Multi-Modal, Multi-Step Referential Game
A novel multi-modal, multi-step referential game, where the sender and receiver have access to distinct modalities of an object, and their information exchange is bidirectional and of arbitrary duration is proposed.