Compositionality and Capacity in Emergent Languages

@inproceedings{Gupta2020CompositionalityAC,
  title={Compositionality and Capacity in Emergent Languages},
  author={Abhinav Gupta and Cinjon Resnick and Jakob N. Foerster and Andrew M. Dai and Kyunghyun Cho},
  booktitle={REPL4NLP},
  year={2020}
}
Recent works have discussed the extent to which emergent languages can exhibit properties of natural languages particularly learning compositionality. In this paper, we investigate the learning biases that affect the efficacy and compositionality in multi-agent communication in addition to the communicative bandwidth. Our foremost contribution is to explore how the capacity of a neural network impacts its ability to learn a compositional language. We additionally introduce a set of evaluation… 

Figures from this paper

Emergent Communication for Understanding Human Language Evolution: What's Missing?
TLDR
This work describes three important phenomena with respect to the emergence and benefits of compositionality: ease-of-learning, generalization, and group size effects (i.e., larger groups create more systematic languages).
Few-shot Language Coordination by Modeling Theory of Mind
TLDR
This work studies the effect of the speaker explicitly modeling the listeners’ mental states in theory-of-mind (ToM), and examines the hypothesis that the instructions generated with ToM modeling yield better communication performance in both a referential game and a language navigation task.
E MERGENT C OMMUNICATION FOR U NDERSTANDING H UMAN L ANGUAGE E VOLUTION : W HAT ’ S M ISSING ?
TLDR
This work describes three important phenomena with respect to the emergence and benefits of compositionality: ease-of-learning, generalization, and group size effects (i.e., larger groups create more systematic languages).
Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents
TLDR
It is shown that the emergent communication between embodied AI agents can be grounded to the agent observations and the spatial structure of the 3D environment.

References

SHOWING 1-10 OF 21 REFERENCES
Exploring Structural Inductive Biases in Emergent Communication
TLDR
This work explores various structural priors in multi-agent communication and proposes a novel graph referential game that empirically shows that graph neural networks induce a better compositional language prior and a stronger generalization to out-of-domain data.
Analyzing structural priors in multi-agent communication
TLDR
This work explores various structural priors in multi-agent communication and proposes a novel graph referential game that empirically shows that graph neural networks induce a better compositional language prior and a stronger generalization to out-of-domain data.
Emergence of Grounded Compositional Language in Multi-Agent Populations
TLDR
This paper proposes a multi-agent learning environment and learning methods that bring about emergence of a basic compositional language that is represented as streams of abstract discrete symbols uttered by agents over time, but nonetheless has a coherent structure that possesses a defined vocabulary and syntax.
Linguistic generalization and compositionality in modern artificial neural networks
  • Marco Baroni
  • Computer Science
    Philosophical Transactions of the Royal Society B
  • 2019
TLDR
It is argued that the intriguing behaviour of these devices should be of interest to linguists and cognitive scientists, as it offers a new perspective on possible computational strategies to deal with linguistic productivity beyond rule-based compositionality.
Natural Language Does Not Emerge ‘Naturally’ in Multi-Agent Dialog
TLDR
This paper presents a sequence of ‘negative’ results culminating in a ‘positive’ one – showing that while most agent-invented languages are effective, they are decidedly not interpretable or compositional.
Emergent Linguistic Phenomena in Multi-Agent Communication Games
TLDR
It is concluded that intricate properties of language evolution need not depend on complex evolved linguistic capabilities, but can emerge from simple social exchanges between perceptually-enabled agents playing communication games.
Multi-Agent Cooperation and the Emergence of (Natural) Language
TLDR
It is shown that two networks with simple configurations are able to learn to coordinate in the referential game and how to make changes to the game environment to cause the "word meanings" induced in the game to better reflect intuitive semantic properties of the images.
Emergent Translation in Multi-Agent Communication
TLDR
This work proposes a communication game where two agents, native speakers of their own respective languages, jointly learn to solve a visual referential task and finds that the ability to understand and translate a foreign language emerges as a means to achieve shared goals.
Iconicity and the Emergence of Combinatorial Structure in Language
TLDR
In this paper, the role of iconicity and human cognitive learning biases in the emergence of combinatorial structure in artificial whistled languages are studied to gain insight into when and how words may lose their iconic origins when they become part of an organized linguistic system.
...
1
2
3
...