• Publications
  • Influence
Neural Module Networks
Visual question answering is fundamentally compositional in nature-a question like where is the dog? shares substructure with questions like what color is the dog? and where is the cat? This paperExpand
  • 489
  • 57
  • PDF
Learning to Reason: End-to-End Module Networks for Visual Question Answering
Natural language questions are inherently compositional, and many are most easily answered by reasoning about their decomposition into modular sub-problems. For example, to answer “is there an equalExpand
  • 300
  • 46
  • PDF
Speaker-Follower Models for Vision-and-Language Navigation
Navigation guided by natural language instructions presents a challenging reasoning problem for instruction followers. Natural language instructions typically identify only a few high-level decisionsExpand
  • 101
  • 38
  • PDF
Learning to Compose Neural Networks for Question Answering
We describe a question answering model that applies to both images and structured knowledge bases. The model uses natural language strings to automatically assemble neural networks from a collectionExpand
  • 390
  • 37
  • PDF
Modeling Relationships in Referential Expressions with Compositional Modular Networks
People often refer to entities in an image in terms of their relationships with other entities. For example, the black cat sitting under the table refers to both a black cat entity and itsExpand
  • 171
  • 32
  • PDF
Modular Multitask Reinforcement Learning with Policy Sketches
We describe a framework for multitask deep reinforcement learning guided by policy sketches. Sketches annotate each task with a sequence of named subtasks, providing high-level structuralExpand
  • 179
  • 27
  • PDF
A Minimal Span-Based Neural Constituency Parser
In this work, we present a minimal neural model for constituency parsing based on independent scoring of labels and spans. We show that this model is not only compatible with classical dynamicExpand
  • 93
  • 27
  • PDF
Semantics-Based Machine Translation with Hyperedge Replacement Grammars
We present an approach to semantics-based statistical machine translation that uses synchronous hyperedge replacement grammars to translate into and from graph-shaped intermediate meaningExpand
  • 119
  • 16
  • PDF
Deep Compositional Question Answering with Neural Module Networks
Visual question answering is fundamentally compositional in nature---a question like "where is the dog?" shares substructure with questions like "what color is the dog?" and "where is the cat?" ThisExpand
  • 122
  • 12
  • PDF
Parsing Graphs with Hyperedge Replacement Grammars
Hyperedge replacement grammar (HRG) is a formalism for generating and transforming graphs that has potential applications in natural language understanding and generation. A recognition algorithm dueExpand
  • 83
  • 12
  • PDF