• Corpus ID: 3728944

Compositional Attention Networks for Machine Reasoning

  title={Compositional Attention Networks for Machine Reasoning},
  author={Drew A. Hudson and Christopher D. Manning},
We present the MAC network, a novel fully differentiable neural network architecture, designed to facilitate explicit and expressive reasoning. [] Key Method By stringing the cells together and imposing structural constraints that regulate their interaction, MAC effectively learns to perform iterative reasoning processes that are directly inferred from the data in an end-to-end approach. We demonstrate the model's strength, robustness and interpretability on the challenging CLEVR dataset for visual reasoning…
Learning by Abstraction: The Neural State Machine
The Neural State Machine is introduced, seeking to bridge the gap between the neural and symbolic views of AI and integrate their complementary strengths for the task of visual reasoning, by transforming both the visual and linguistic modalities into semantic concept-based representations, thereby achieving enhanced transparency and modularity.
Learning Dynamics of Attention: Human Prior for Interpretable Machine Reasoning
This work proposes Dynamics of Attention for Focus Transition as a human prior for machine reasoning, a novel method that regularizes attention-based reasoning by modelling it as a continuous dynamical system using neural ordinary differential equations, and proposes a new metric, Total Length of Transition (TLT), which represents the effective reasoning step size.
Dynamic Inference with Neural Interpreters
This work presents Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules, which they call functions, which can flexibly compose computation along width and depth, and lends itself well to capacity extension after training.
Explainable Neural Computation via Stack Neural Module Networks
A novel neural modular approach that performs compositional reasoning by automatically inducing a desired sub-task decomposition without relying on strong supervision is presented, which is more interpretable to human evaluators compared to other state-of-the-art models.
Attention on Abstract Visual Reasoning
This work proposes a hybrid network architecture, grounded on self-attention and relational reasoning, and calls this new model Attention Relation Network (ARNe), which combines features from the recently introduced Transformer and the Wild RelATION Network (WReN).
Linguistically Driven Graph Capsule Network for Visual Question Reasoning
This work proposes a hierarchical compositional reasoning model called the "Linguistically driven Graph Capsule Network", where the compositional process is guided by the linguistic parse tree, inspired by the property of a capsule network that can carve a tree structure inside a regular convolutional neural network (CNN).
Attention over Learned Object Embeddings Enables Complex Visual Reasoning
The success of this combination suggests that there may be no need to trade off flexibility for performance on problems involving spatio-temporal or causal-style reasoning, and with the right soft biases and learning objectives in a neural network may be able to attain the best of both worlds.
Visual Reasoning by Progressive Module Networks
By evaluating the reasoning process using human judges, this work shows that the proposed model effectively combines previous skill-sets, does not suffer from forgetting, and is fully differentiable.
A dataset and architecture for visual reasoning with a working memory
This work developed an artificial, configurable visual question and answer dataset (COG) to parallel experiments in humans and animals and proposes a deep learning architecture that performs competitively on other diagnostic VQA datasets (i.e. CLEVR) as well as easy settings of the COG dataset.
Think before you act: A simple baseline for compositional generalization
This work proposes an attention-inspired modification of the baseline model from Ruis et al. 2020, together with an auxiliary loss, that takes into account the sequential nature of steps (i) and (ii), and finds that two compositional tasks are trivially solved with this approach.


A simple neural network module for relational reasoning
This work shows how a deep learning architecture equipped with an RN module can implicitly discover and learn to reason about entities and their relations.
Towards Deep Symbolic Reinforcement Learning
It is shown that the resulting system -- though just a prototype -- learns effectively, and, by acquiring a set of symbolic rules that are easily comprehensible to humans, dramatically outperforms a conventional, fully neural DRL system on a stochastic variant of the game.
Inferring and Executing Programs for Visual Reasoning
A model for visual reasoning that consists of a program generator that constructs an explicit representation of the reasoning process to be performed, and an execution engine that executes the resulting program to produce an answer is proposed.
Learning to Reason: End-to-End Module Networks for Visual Question Answering
End-to-End Module Networks are proposed, which learn to reason by directly predicting instance-specific network layouts without the aid of a parser, and achieve an error reduction of nearly 50% relative to state-of-theart attentional approaches.
Building machines that learn and think like people
It is argued that truly human-like learning and thinking machines should build causal models of the world that support explanation and understanding, rather than merely solving pattern recognition problems, and harness compositionality and learning-to-learn to rapidly acquire and generalize knowledge to new tasks and situations.
Neural Module Networks
A procedure for constructing and learning neural module networks, which compose collections of jointly-trained neural "modules" into deep networks for question answering, and uses these structures to dynamically instantiate modular networks (with reusable components for recognizing dogs, classifying colors, etc.).
Hybrid computing using a neural network with dynamic external memory
A machine learning model called a differentiable neural computer (DNC), which consists of a neural network that can read from and write to an external memory matrix, analogous to the random-access memory in a conventional computer.
Dynamic Memory Networks for Visual and Textual Question Answering
The new DMN+ model improves the state of the art on both the Visual Question Answering dataset and the \babi-10k text question-answering dataset without supporting fact supervision.
CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
This work presents a diagnostic dataset that tests a range of visual reasoning abilities and uses this dataset to analyze a variety of modern visual reasoning systems, providing novel insights into their abilities and limitations.
From machine learning to machine reasoning
  • L. Bottou
  • Computer Science
    Machine Learning
  • 2013
Instead of trying to bridge the gap between machine learning systems and sophisticated “all-purpose” inference mechanisms, the set of manipulations applicable to training systems can be algebraically enriched, and reasoning capabilities from the ground up are built.