Be Consistent! Improving Procedural Text Comprehension using Label Consistency

@article{Du2019BeCI,
  title={Be Consistent! Improving Procedural Text Comprehension using Label Consistency},
  author={X. Du and Bhavana Dalvi and Niket Tandon and Antoine Bosselut and Wen-tau Yih and Peter Clark and Claire Cardie},
  journal={ArXiv},
  year={2019},
  volume={abs/1906.08942}
}
Our goal is procedural text comprehension, namely tracking how the properties of entities (e.g., their location) change with time given a procedural text (e.g., a paragraph about photosynthesis, a recipe. [] Key Method We present a new learning framework that leverages label consistency during training, allowing consistency bias to be built into the model. Evaluation on a standard benchmark dataset for procedural text, ProPara (Dalvi et al., 2018), shows that our approach significantly improves prediction…
[PDF] Semantic Reader
22 Citations
Highly Influential Citations
1
Background Citations
16
Methods Citations
2

Figures and Tables from this paper

22 Citations

Knowledge-Aware Procedural Text Understanding with Multi-Stage Training

A novel KnOwledge-Aware proceduraL text understAnding (KoaLa) model is proposed, which effectively leverages multiple forms of external knowledge in this task of procedural text understanding and achieves state-of-the-art performance in comparison to various baselines.

Procedural Reasoning Networks for Understanding Multimodal Procedures

This paper introduces a new entity-aware neural comprehension model augmented with external relational memory units and finds that this model learns effective dynamic representations of entities even though it does not use any supervision at the level of entity states.

Reasoning over Entity-Action-Location Graph for Procedural Text Understanding

This paper proposes a novel approach (REAL) to procedural text understanding, where a general framework is built to systematically model the entity-entity, entity-action, and entity-location relations using a graph neural network and develops algorithms for graph construction, representation learning, and state and location tracking.

Understanding Procedural Text Using Interactive Entity Networks

This paper proposes a novel Interactive Entity Network (IEN), which is a recurrent network with memory equipped cells for state tracking that outperforms state-of-the-art models by precisely capturing the interactions of multiple entities and explicitly leverage the relationship between entity interactions and subsequent state changes.

Logic-Guided Data Augmentation and Regularization for Consistent Question Answering

This paper addresses the problem of improving the accuracy and consistency of responses to comparison questions by integrating logic rules and neural models by leveraging logical and linguistic knowledge to augment labeled training data and then uses a consistency-based regularizer to train the model.

Structured Tuning for Semantic Role Labeling

This paper presents a structured tuning framework to improve models using softened constraints only at training time, which leverages the expressiveness of neural networks and provides supervision with structured loss components.

Predicting State Changes in Procedural Text using Analogical Question Answering

A novel approach that uses analogical question answering to predict what state changes affect entities in a paragraph describing a process, starting from the hypothesis that human level QA requires multiple layers of rich, relational representations.

Measuring and Improving Consistency in Pretrained Language Models

The creation of PARAREL, a high-quality resource of cloze-style query English paraphrases, and analysis of the representational spaces of PLMs suggest that they have a poor structure and are currently not suitable for representing knowledge in a robust way.

Joint Semantic Analysis with Document-Level Cross-Task Coherence Rewards

A neural network architecture for joint coreference resolution and semantic role labeling for English, and train graph neural networks to model the 'coherence' of the combined shallow semantic graph, calling for a more holistic approach for semantics in NLP.

Procedural Text Understanding via Scene-Wise Evolution

A new scene-wise paradigm for procedural text understanding is proposed, which jointly tracks states of all entities in a scene-by-scene manner and introduces a series of dynamically evolving scene graphs to jointly formulate the evolution of entities, states and their associations throughout the narrative.

References

SHOWING 1-10 OF 27 REFERENCES

Tracking State Changes in Procedural Text: a Challenge Dataset and Models for Process Paragraph Comprehension

A new dataset and models for comprehending paragraphs about processes, an important genre of text describing a dynamic world, are presented and two new neural models that exploit alternative mechanisms for state prediction are introduced, in particular using LSTM input encoding and span prediction.

Reasoning about Actions and State Changes by Injecting Commonsense Knowledge

This paper shows how the predicted effects of actions in the context of a paragraph can be improved in two ways: by incorporating global, commonsense constraints (e.g., a non-existent entity cannot be destroyed), and by biasing reading with preferences from large-scale corpora.

Building Dynamic Knowledge Graphs from Text using Machine Reading Comprehension

A neural machine-reading model that constructs dynamic knowledge graphs recurrently for each step of the described procedure, and uses them to track the evolving states of participant entities to present some evidence that the model’s knowledge graphs help it to impose commonsense constraints on its predictions.

A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task

A thorough examination of this new reading comprehension task by creating over a million training examples by pairing CNN and Daily Mail news articles with their summarized bullet points, and showing that a neural network can be trained to give good performance on this task.

Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks

This work argues for the usefulness of a set of proxy tasks that evaluate reading comprehension via question answering, and classify these tasks into skill sets so that researchers can identify (and then rectify) the failings of their systems.

Mise en Place: Unsupervised Interpretation of Instructional Recipes

An unsupervised hard EM approach to automatically mapping instructional recipes to action graphs, which define what actions should be performed on which objects and in what order, which incorporates aspects of procedural semantics and world knowledge.

Modeling Biological Processes for Reading Comprehension

This paper focuses on a new reading comprehension task that requires complex reasoning over a single document, and demonstrates that answering questions via predicted structures substantially improves accuracy over baselines that use shallower representations.

Query-Reduction Networks for Question Answering

Query-Reduction Network (QRN), a variant of Recurrent Neural Network (RNN) that effectively handles both short-term and long-term sequential dependencies to reason over multiple facts, is proposed.

Globally Coherent Text Generation with Neural Checklist Models

The neural checklist model is presented, a recurrent neural network that models global coherence by storing and updating an agenda of text strings which should be mentioned somewhere in the output, and demonstrates high coherence with greatly improved semantic coverage of the agenda.

AllenNLP: A Deep Semantic Natural Language Processing Platform

AllenNLP is described, a library for applying deep learning methods to NLP research that addresses issues with easy-to-use command-line tools, declarative configuration-driven experiments, and modular NLP abstractions.