Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering

  title={Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering},
  author={Todor Mihaylov and Peter Clark and Tushar Khot and A. Sabharwal},
  • Todor Mihaylov, Peter Clark, +1 author A. Sabharwal
  • Published in EMNLP 2018
  • Computer Science
  • We present a new kind of question answering dataset, OpenBookQA, modeled after open book exams for assessing human understanding of a subject. [...] Key Result Our oracle experiments designed to circumvent the knowledge retrieval bottleneck demonstrate the value of both the open book and additional facts. We leave it as a challenge to solve the retrieval problem in this multi-hop setting and to close the large gap to human performance.Expand Abstract
    126 Citations

    Figures, Tables, and Topics from this paper

    Careful Selection of Knowledge to Solve Open Book Question Answering
    • 21
    • PDF
    Answering Science Exam Questions Using Query Reformulation with Background Knowledge
    • 8
    Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering
    • 1
    • PDF
    Answering Science Exam Questions Using Query Rewriting with Background Knowledge
    • 5
    • PDF
    Common Sense-Based Reasoning Using External Knowledge for Question Answering
    • PDF
    CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
    • 172
    • PDF
    Improving Question Answering with External Knowledge
    • 25
    • Highly Influenced
    • PDF
    Transfer Learning on Natural YES/NO Questions


    Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
    • 168
    • PDF
    A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task
    • 431
    • PDF
    TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
    • 544
    • PDF
    KG^2: Learning to Reason Science Exam Questions with Contextual Knowledge Graph Embeddings
    • 18
    • Highly Influential
    • PDF
    SQuAD: 100, 000+ Questions for Machine Comprehension of Text
    • 2,582
    • PDF
    Reading Wikipedia to Answer Open-Domain Questions
    • 796
    • Highly Influential
    • PDF
    MCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text
    • 482
    • PDF
    NewsQA: A Machine Comprehension Dataset
    • 367
    • PDF
    Question Answering via Integer Programming over Semi-Structured Knowledge
    • 74
    • PDF