QuaRTz: An Open-Domain Dataset of Qualitative Relationship Questions

@article{Tafjord2019QuaRTzAO,
  title={QuaRTz: An Open-Domain Dataset of Qualitative Relationship Questions},
  author={Oyvind Tafjord and Matt Gardner and Kevin Lin and P. Clark},
  journal={ArXiv},
  year={2019},
  volume={abs/1909.03553}
}
We introduce the first open-domain dataset, called QuaRTz, for reasoning about textual qualitative relationships. QuaRTz contains general qualitative statements, e.g., “A sunscreen with a higher SPF protects the skin longer.”, twinned with 3864 crowdsourced situated questions, e.g., “Billy is wearing sunscreen with a lower SPF than Lucy. Who will be best protected from the sun?”, plus annotations of the properties being compared. Unlike previous datasets, the general knowledge is textual and… Expand
View on ACL
arxiv.org
17 Citations
Highly Influential Citations
3
Background Citations
11
Methods Citations
6

17 Citations

Citation Type

Citation Type

Prediction or Comparison: Toward Interpretable Qualitative Reasoning
TLDR
This work categorizes qualitative reasoning tasks into two types: prediction and comparison, and adopts neural network modules trained in an end-to-end manner to simulate the two reasoning processes. Expand
QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension
TLDR
The largest survey of the field to date of question answering and reading comprehension, providing an overview of the various formats and domains of the current resources, and highlighting the current lacunae for future work. Expand
Competency Problems: On Finding and Removing Artifacts in Language Data
TLDR
This work argues that for complex language understanding tasks, all simple feature correlations are spurious, and formalizes this notion into a class of problems which are called competency problems, and gives a simple statistical test for dataset artifacts that is used to show more subtle biases. Expand
CURIE: An Iterative Querying Approach for Reasoning About Situations
TLDR
It is shown that st graphs generated by CURIE improve a situational reasoning end task (WIQA-QA) by 3 points on accuracy by simply augmenting their input with the authors' generated situational graphs, especially for a hard subset that requires background knowledge and multi-hop reasoning. Expand
Natural Language QA Approaches using Reasoning with External Knowledge
TLDR
A survey of the recent work on the traditional fields of knowledge representation and reasoning and the field of NL understanding and NLQA is presented to help establish a bridge between multiple fields of AI. Expand
Probabilistic Graph Reasoning for Natural Proof Generation
TLDR
This paper proposes PROBR, a novel approach for joint answer prediction and proof generation via an induced graphical model that defines a joint probabilistic distribution over all possible proof graphs and answers via an inducing graphical model. Expand
CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP
TLDR
This paper introduces CROSSFIT, a task setup for studying cross-task few-shot learning ability, which standardizes seen/unseen task splits, data access during different learning stages, and the evaluation protocols, and presents NLP Few-shot Gym, a repository of 160 few- Shots tasks, covering diverse task categories and applications, and converted to a unified text-to-text format. Expand
PRover: Proof Generation for Interpretable Reasoning over Rules
TLDR
This work proposes PROVER, an interpretable transformer-based model that jointly answers binary questions over rule-bases and generates the corresponding proofs, and learns to predict nodes and edges corresponding to proof graphs in an efficient constrained training paradigm. Expand
Deeply Embedded Knowledge Representation & Reasoning For Natural Language Question Answering: A Practitioner’s Perspective
TLDR
Deeply Embedded Knowledge Representation & Reasoning (DeepEKR) is proposed where the parser is replaced by a neural network, the symbolic representation is softened, a deterministic mapping exists between the parser neural network and the interpretable logical form, and the symbolic solver is replace by an equivalent neural network so the model can be trained end-to-end. Expand
Transformers as Soft Reasoners over Language
TLDR
This work trains transformers to reason (or emulate reasoning) over natural language sentences using synthetically generated data, thus bypassing a formal representation and suggesting a new role for transformers, namely as limited "soft theorem provers" operating over explicit theories in language. Expand
...
1
2
...

References

SHOWING 1-10 OF 15 REFERENCES
QuaRel: A Dataset and Models for Answering Questions about Qualitative Relationships
TLDR
This work makes inroads into answering complex, qualitative questions that require reasoning, and scaling to new relationships at low cost, with two novel models for this task built as extensions of type-constrained semantic parsing. Expand
Semantic Parsing on Freebase from Question-Answer Pairs
TLDR
This paper trains a semantic parser that scales up to Freebase and outperforms their state-of-the-art parser on the dataset of Cai and Yates (2013), despite not having annotated logical forms. Expand
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
TLDR
It is shown that HotpotQA is challenging for the latest QA systems, and the supporting facts enable models to improve performance and make explainable predictions. Expand
Neural Semantic Parsing with Type Constraints for Semi-Structured Tables
TLDR
A new semantic parsing model for answering compositional questions on semi-structured Wikipedia tables with a state-of-the-art accuracy and type constraints and entity linking are valuable components to incorporate in neural semantic parsers. Expand
Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
TLDR
A new question set, text corpus, and baselines assembled to encourage AI research in advanced question answering constitute the AI2 Reasoning Challenge (ARC), which requires far more powerful knowledge and reasoning than previous challenges such as SQuAD or SNLI. Expand
Scaling up Linguistic Processing of Qualitative Processes
Many natural language systems either focus on specific domains or sacrifice deep representations for broad coverage. We propose that a combination of a domain independent grammar and semantics along… Expand
RACE: Large-scale ReAding Comprehension Dataset From Examinations
TLDR
The proportion of questions that requires reasoning is much larger in RACE than that in other benchmark datasets for reading comprehension, and there is a significant gap between the performance of the state-of-the-art models and the ceiling human performance. Expand
Simple and Effective Multi-Paragraph Reading Comprehension
We consider the problem of adapting neural paragraph-level question answering models to the case where entire documents are given as input. Our proposed solution trains models to produce well… Expand
AllenNLP: A Deep Semantic Natural Language Processing Platform
TLDR
AllenNLP is designed to support researchers who want to build novel language understanding models quickly and easily and provides a flexible data API that handles intelligent batching and padding, and a modular and extensible experiment framework that makes doing good science easy. Expand
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
TLDR
A new language representation model, BERT, designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks. Expand
...
1
2
...