Tracking State Changes in Procedural Text: a Challenge Dataset and Models for Process Paragraph Comprehension

@inproceedings{Dalvi2018TrackingSC,
  title={Tracking State Changes in Procedural Text: a Challenge Dataset and Models for Process Paragraph Comprehension},
  author={Bhavana Dalvi and Lifu Huang and Niket Tandon and Wen-tau Yih and Peter Clark},
  booktitle={NAACL},
  year={2018}
}
We present a new dataset and models for comprehending paragraphs about processes (e.g., photosynthesis), an important genre of text describing a dynamic world. [...] Key Method We find that previous models that have worked well on synthetic data achieve only mediocre performance on ProPara, and introduce two new neural models that exploit alternative mechanisms for state prediction, in particular using LSTM input encoding and span prediction. The new models improve accuracy by up to 19%. The dataset and models…Expand
View on ACL
arxiv.org
64 Citations
Highly Influential Citations
12
Background Citations
44
Methods Citations
20
Results Citations
4

64 Citations

Citation Type

Citation Type

Reasoning about Actions and State Changes by Injecting Commonsense Knowledge
TLDR
This paper shows how the predicted effects of actions in the context of a paragraph can be improved in two ways: by incorporating global, commonsense constraints (e.g., a non-existent entity cannot be destroyed), and by biasing reading with preferences from large-scale corpora. Expand
Understanding Procedural Text Using Interactive Entity Networks
TLDR
This paper proposes a novel Interactive Entity Network (IEN), which is a recurrent network with memory equipped cells for state tracking that outperforms state-of-the-art models by precisely capturing the interactions of multiple entities and explicitly leverage the relationship between entity interactions and subsequent state changes. Expand
Everything Happens for a Reason: Discovering the Purpose of Actions in Procedural Text
TLDR
This work presents a new model (XPAD) that biases effect predictions towards those that explain more of the actions in the paragraph and are more plausible with respect to background knowledge, and extends an existing benchmark dataset for procedural text comprehension, ProPara, by adding the new task of explaining actions by predicting their dependencies. Expand
Be Consistent! Improving Procedural Text Comprehension using Label Consistency
TLDR
This work presents a new learning framework that leverages label consistency during training, allowing consistency bias to be built into the model and significantly improves prediction performance over prior state-of-the-art systems. Expand
Effective Use of Transformer Networks for Entity Tracking
TLDR
This paper tests standard lightweight approaches for prediction with pre-trained transformers, and finds that these approaches underperforms even simple baselines, and shows that much stronger results can be attained by restructuring the input to guide the model to focus on a particular entity. Expand
What Happened? Leveraging VerbNet to Predict the Effects of Actions in Procedural Text
TLDR
This work leverages VerbNet to build a rulebase of the preconditions and effects of actions, and uses it along with commonsense knowledge of persistence to answer questions about change in paragraphs describing processes. Expand
Building Dynamic Knowledge Graphs from Text using Machine Reading Comprehension
TLDR
A neural machine-reading model that constructs dynamic knowledge graphs recurrently for each step of the described procedure, and uses them to track the evolving states of participant entities to present some evidence that the model’s knowledge graphs help it to impose commonsense constraints on its predictions. Expand
Procedural Reading Comprehension with Attribute-Aware Context Flow
TLDR
An algorithm for procedural reading comprehension is introduced by translating the text into a general formalism that represents processes as a sequence of transitions over entity attributes (e.g., location, temperature). Expand
Predicting State Changes in Procedural Text using Analogical Question Answering
Many of the changes in the world that happen over time are characterized by processes. Creating programs that comprehend procedural text (e.g. the stages of photosynthesis) is a crucial task in… Expand
Time-Stamped Language Model: Teaching Language Models to Understand The Flow of Events
TLDR
A Time-Stamped Language Model (TSLM) is proposed to encode event information in LMs architecture by introducing the timestamp encoding to enable pre-trained transformer-based language models to be used on other QA benchmarks by adapting those to the procedural text understanding. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 27 REFERENCES
What Happened? Leveraging VerbNet to Predict the Effects of Actions in Procedural Text
TLDR
This work leverages VerbNet to build a rulebase of the preconditions and effects of actions, and uses it along with commonsense knowledge of persistence to answer questions about change in paragraphs describing processes. Expand
Tracking the World State with Recurrent Entity Networks
TLDR
The EntNet sets a new state-of-the-art on the bAbI tasks, and is the first method to solve all the tasks in the 10k training examples setting, and can generalize past its training horizon. Expand
Bidirectional Attention Flow for Machine Comprehension
TLDR
The BIDAF network is introduced, a multi-stage hierarchical process that represents the context at different levels of granularity and uses bi-directional attention flow mechanism to obtain a query-aware context representation without early summarization. Expand
Simple and Effective Multi-Paragraph Reading Comprehension
We consider the problem of adapting neural paragraph-level question answering models to the case where entire documents are given as input. Our proposed solution trains models to produce well… Expand
Constructing Datasets for Multi-hop Reading Comprehension Across Documents
TLDR
A novel task to encourage the development of models for text understanding across multiple documents and to investigate the limits of existing methods, in which a model learns to seek and combine evidence — effectively performing multihop, alias multi-step, inference. Expand
Modeling Biological Processes for Reading Comprehension
TLDR
This paper focuses on a new reading comprehension task that requires complex reasoning over a single document, and demonstrates that answering questions via predicted structures substantially improves accuracy over baselines that use shallower representations. Expand
Query-Reduction Networks for Question Answering
TLDR
Query-Reduction Network (QRN), a variant of Recurrent Neural Network (RNN) that effectively handles both short-term and long-term sequential dependencies to reason over multiple facts, is proposed. Expand
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
TLDR
It is shown that, in comparison to other recently introduced large-scale datasets, TriviaQA has relatively complex, compositional questions, has considerable syntactic and lexical variability between questions and corresponding answer-evidence sentences, and requires more cross sentence reasoning to find answers. Expand
A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task
TLDR
A thorough examination of this new reading comprehension task by creating over a million training examples by pairing CNN and Daily Mail news articles with their summarized bullet points, and showing that a neural network can be trained to give good performance on this task. Expand
SQuAD: 100,000+ Questions for Machine Comprehension of Text
TLDR
A strong logistic regression model is built, which achieves an F1 score of 51.0%, a significant improvement over a simple baseline (20%). Expand
...
1
2
3
...