• Corpus ID: 3728944

Compositional Attention Networks for Machine Reasoning

  title={Compositional Attention Networks for Machine Reasoning},
  author={Drew A. Hudson and Christopher D. Manning},
We present the MAC network, a novel fully differentiable neural network architecture, designed to facilitate explicit and expressive reasoning. [] Key Method By stringing the cells together and imposing structural constraints that regulate their interaction, MAC effectively learns to perform iterative reasoning processes that are directly inferred from the data in an end-to-end approach. We demonstrate the model's strength, robustness and interpretability on the challenging CLEVR dataset for visual reasoning…

Learning by Abstraction: The Neural State Machine

The Neural State Machine is introduced, seeking to bridge the gap between the neural and symbolic views of AI and integrate their complementary strengths for the task of visual reasoning, by transforming both the visual and linguistic modalities into semantic concept-based representations, thereby achieving enhanced transparency and modularity.

Learning Dynamics of Attention: Human Prior for Interpretable Machine Reasoning

This work proposes Dynamics of Attention for Focus Transition as a human prior for machine reasoning, a novel method that regularizes attention-based reasoning by modelling it as a continuous dynamical system using neural ordinary differential equations, and proposes a new metric, Total Length of Transition (TLT), which represents the effective reasoning step size.

Dynamic Inference with Neural Interpreters

This work presents Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules, which they call functions, which can flexibly compose computation along width and depth, and lends itself well to capacity extension after training.

Explainable Neural Computation via Stack Neural Module Networks

A novel neural modular approach that performs compositional reasoning by automatically inducing a desired sub-task decomposition without relying on strong supervision is presented, which is more interpretable to human evaluators compared to other state-of-the-art models.

Linguistically Driven Graph Capsule Network for Visual Question Reasoning

This work proposes a hierarchical compositional reasoning model called the "Linguistically driven Graph Capsule Network", where the compositional process is guided by the linguistic parse tree, inspired by the property of a capsule network that can carve a tree structure inside a regular convolutional neural network (CNN).

Attention over Learned Object Embeddings Enables Complex Visual Reasoning

The success of this combination suggests that there may be no need to trade off flexibility for performance on problems involving spatio-temporal or causal-style reasoning, and with the right soft biases and learning objectives in a neural network may be able to attain the best of both worlds.

A dataset and architecture for visual reasoning with a working memory

This work developed an artificial, configurable visual question and answer dataset (COG) to parallel experiments in humans and animals and proposes a deep learning architecture that performs competitively on other diagnostic VQA datasets (i.e. CLEVR) as well as easy settings of the COG dataset.

Think before you act: A simple baseline for compositional generalization

This work proposes an attention-inspired modification of the baseline model from Ruis et al. 2020, together with an auxiliary loss, that takes into account the sequential nature of steps (i) and (ii), and finds that two compositional tasks are trivially solved with this approach.

Relational inductive biases, deep learning, and graph networks

It is argued that combinatorial generalization must be a top priority for AI to achieve human-like abilities, and that structured representations and computations are key to realizing this objective.

Learning compositionally through attentive guidance

Attentive Guidance, a mechanism to direct a sequence to sequence model equipped with attention to find more compositional solutions, is introduced, and it is shown that vanilla sequence tosequence models with attention overfit the training distribution, while the guided versions come up with Compositional solutions that fit the training and testing distributions almost equally well.



A simple neural network module for relational reasoning

This work shows how a deep learning architecture equipped with an RN module can implicitly discover and learn to reason about entities and their relations.

Towards Deep Symbolic Reinforcement Learning

It is shown that the resulting system -- though just a prototype -- learns effectively, and, by acquiring a set of symbolic rules that are easily comprehensible to humans, dramatically outperforms a conventional, fully neural DRL system on a stochastic variant of the game.

Inferring and Executing Programs for Visual Reasoning

A model for visual reasoning that consists of a program generator that constructs an explicit representation of the reasoning process to be performed, and an execution engine that executes the resulting program to produce an answer is proposed.

Learning to Reason: End-to-End Module Networks for Visual Question Answering

End-to-End Module Networks are proposed, which learn to reason by directly predicting instance-specific network layouts without the aid of a parser, and achieve an error reduction of nearly 50% relative to state-of-theart attentional approaches.

Neural Module Networks

A procedure for constructing and learning neural module networks, which compose collections of jointly-trained neural "modules" into deep networks for question answering, and uses these structures to dynamically instantiate modular networks (with reusable components for recognizing dogs, classifying colors, etc.).

Hybrid computing using a neural network with dynamic external memory

A machine learning model called a differentiable neural computer (DNC), which consists of a neural network that can read from and write to an external memory matrix, analogous to the random-access memory in a conventional computer.

Dynamic Memory Networks for Visual and Textual Question Answering

The new DMN+ model improves the state of the art on both the Visual Question Answering dataset and the \babi-10k text question-answering dataset without supporting fact supervision.

CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning

This work presents a diagnostic dataset that tests a range of visual reasoning abilities and uses this dataset to analyze a variety of modern visual reasoning systems, providing novel insights into their abilities and limitations.

From machine learning to machine reasoning

  • L. Bottou
  • Computer Science
    Machine Learning
  • 2013
Instead of trying to bridge the gap between machine learning systems and sophisticated “all-purpose” inference mechanisms, the set of manipulations applicable to training systems can be algebraically enriched, and reasoning capabilities from the ground up are built.

Learning to Compose Neural Networks for Question Answering

A question answering model that applies to both images and structured knowledge bases that uses natural language strings to automatically assemble neural networks from a collection of composable modules that achieves state-of-the-art results on benchmark datasets.