Translating Neuralese

  title={Translating Neuralese},
  author={Jacob Andreas and Anca D. Dragan and Dan Klein},
Several approaches have recently been proposed for learning decentralized deep multiagent policies that coordinate via a differentiable communication channel. While these policies are effective for many tasks, interpretation of their induced communication strategies has remained a challenge. Here we propose to interpret agents’ messages by translating them. Unlike in typical machine translation problems, we have no parallel data to learn from. Instead we develop a translation model based on the… 

Figures and Tables from this paper

Countering Language Drift via Visual Grounding

It is shown that a combination of syntactic (language model likelihood) and semantic (visual grounding) constraints gives the best communication performance, allowing pre-trained agents to retain English syntax while learning to accurately convey the intended meaning.

Emergence of Compositional Language with Deep Generational Transmission

It is shown that this implicit cultural transmission encourages the resulting languages to exhibit better compositional generalization and suggest how elements of cultural dynamics can be further integrated into populations of deep agents.

Emergent Translation in Multi-Agent Communication

This work proposes a communication game where two agents, native speakers of their own respective languages, jointly learn to solve a visual referential task and finds that the ability to understand and translate a foreign language emerges as a means to achieve shared goals.

Emergence of Grounded Compositional Language in Multi-Agent Populations

This paper proposes a multi-agent learning environment and learning methods that bring about emergence of a basic compositional language that is represented as streams of abstract discrete symbols uttered by agents over time, but nonetheless has a coherent structure that possesses a defined vocabulary and syntax.

Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning

This work poses a cooperative ‘image guessing’ game between two agents who communicate in natural language dialog so that Q-BOT can select an unseen image from a lineup of images and shows the emergence of grounded language and communication among ‘visual’ dialog agents with no human supervision.

Emergent Compositionality in Signaling Games

Experimental evidence is provided suggesting that incremental pragmatic reasoning may lead to compositional referring behavior in both computational agents and in humans.

Inferring Rewards from Language in Context

On a new interactive flight–booking task with natural language, the model more accurately infers rewards and predicts optimal actions in unseen environments, in comparison to past work that first maps language to actions and then maps actions to rewards.

Analogs of Linguistic Structure in Deep Representations

This work investigates the compositional structure of message vectors computed by a deep network trained on a communication game and suggests that neural representations are capable of spontaneously developing a “syntax" with functional analogues to qualitative properties of natural language.

Explainable Neural Computation via Stack Neural Module Networks

A novel neural modular approach that performs compositional reasoning by automatically inducing a desired sub-task decomposition without relying on strong supervision is presented, which is more interpretable to human evaluators compared to other state-of-the-art models.

Learning to Interactively Learn and Assist

This paper introduces a multi-agent training framework that enables an agent to learn from another agent who knows the current task, and produces an agent that is capable of learning interactively from a human user, without a set of explicit demonstrations or a reward function.



Learning Multiagent Communication with Backpropagation

A simple neural model is explored, called CommNet, that uses continuous communication for fully cooperative tasks and the ability of the agents to learn to communicate amongst themselves is demonstrated, yielding improved performance over non-communicative agents and baselines.

Towards Multi-Agent Communication-Based Language Learning

An interactive multimodal framework for language learning where learners engage in cooperative referential games starting from a tabula rasa setup, and thus develop their own language from the need to communicate in order to succeed at the game.

Learning to Communicate with Deep Multi-Agent Reinforcement Learning

By embracing deep neural networks, this work is able to demonstrate end-to-end learning of protocols in complex environments inspired by communication riddles and multi-agent computer vision problems with partial observability.

Multi-Agent Cooperation and the Emergence of (Natural) Language

It is shown that two networks with simple configurations are able to learn to coordinate in the referential game and how to make changes to the game environment to cause the "word meanings" induced in the game to better reflect intuitive semantic properties of the images.

Reasoning about Pragmatics with Neural Listeners and Speakers

A model for pragmatically describing scenes, in which contrastive behavior results from a combination of inference-driven pragmatics and learned semantics, that succeeds 81% of the time in human evaluations on a referring expression game.

Deep Recurrent Q-Learning for Partially Observable MDPs

The effects of adding recurrency to a Deep Q-Network is investigated by replacing the first post-convolutional fully-connected layer with a recurrent LSTM, which successfully integrates information through time and replicates DQN's performance on standard Atari games and partially observed equivalents featuring flickering game screens.

A Paradigm for Situated and Goal-Driven Language Learning

A general situated language learning paradigm is proposed which is designed to bring about robust language agents able to cooperate productively with humans.

Implicatures and Nested Beliefs in Approximate Decentralized-POMDPs

This work shows that agents in the multi-agent DecentralizedPOMDP reach implicature-rich interpretations simply as a by-product of the way they reason about each other to maximize joint utility.

Sequence to Sequence Learning with Neural Networks

This paper presents a general end-to-end approach to sequence learning that makes minimal assumptions on the sequence structure, and finds that reversing the order of the words in all source sentences improved the LSTM's performance markedly, because doing so introduced many short term dependencies between the source and the target sentence which made the optimization problem easier.

Reasoning about joint beliefs for execution-time communication decisions

This paper presents an approach that generates "centralized" policies for multi-agent POMDPs at plan-time by assuming the presence of free communication, and at run-time, handles the problem of limited communication resources by reasoning about the use of communication as needed for effective execution.