Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoning

  title={Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoning},
  author={Pradeep Dasigi and Nelson F. Liu and Ana Marasovi{\'c} and Noah A. Smith and Matt Gardner},
  • Pradeep Dasigi, Nelson F. Liu, +2 authors Matt Gardner
  • Published in EMNLP/IJCNLP 2019
  • Computer Science
  • Machine comprehension of texts longer than a single sentence often requires coreference resolution. [...] Key Method We deal with this issue by using a strong baseline model as an adversary in the crowdsourcing loop, which helps crowdworkers avoid writing questions with exploitable surface cues. We show that state-of-the-art reading comprehension models perform significantly worse than humans on this benchmark—the best model performance is 70.5 F1, while the estimated human performance is 93.4 F1.Expand Abstract
    36 Citations
    On Making Reading Comprehension More Comprehensive
    • 11
    • Highly Influenced
    • PDF
    IIRC: A Dataset of Incomplete Information Reading Comprehension Questions
    Coreference Resolution as Query-based Span Prediction
    • 12
    • PDF
    TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questions
    • 5
    • PDF
    CorefQA: Coreference Resolution as Query-based Span Prediction
    • 11
    • Highly Influenced
    • PDF
    A Simple and Effective Model for Answering Multi-span Questions
    • 4
    • PDF
    Comprehensive Multi-Dataset Evaluation of Reading Comprehension
    • 2
    • PDF
    Coreferential Reasoning Learning for Language Representation
    • 9
    • PDF


    DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
    • 141
    • PDF
    MCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text
    • 457
    • PDF
    TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
    • 486
    • PDF
    SQuAD: 100, 000+ Questions for Machine Comprehension of Text
    • 2,303
    • Highly Influential
    • PDF
    PreCo: A Large-scale Dataset in Preschool Vocabulary for Coreference Resolution
    • 15
    • PDF
    Model-based annotation of coreference
    • 1
    • PDF
    MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
    • 554
    • PDF