Are You Smarter Than a Sixth Grader? Textbook Question Answering for Multimodal Machine Comprehension

@article{Kembhavi2017AreYS,
  title={Are You Smarter Than a Sixth Grader? Textbook Question Answering for Multimodal Machine Comprehension},
  author={Aniruddha Kembhavi and Minjoon Seo and D. Schwenk and Jonghyun Choi and Ali Farhadi and Hannaneh Hajishirzi},
  journal={2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2017},
  pages={5376-5384}
}
  • Aniruddha Kembhavi, Minjoon Seo, +3 authors Hannaneh Hajishirzi
  • Published 2017
  • Computer Science
  • 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • We introduce the task of Multi-Modal Machine Comprehension (M3C), which aims at answering multimodal questions given a context of text, diagrams and images. [...] Key Method We extend state-of-the-art methods for textual machine comprehension and visual question answering to the TQA dataset. Our experiments show that these models do not perform well on TQA. The presented dataset opens new challenges for research in question answering and reasoning across multiple modalities.Expand Abstract
    92 Citations

    Figures, Tables, and Topics from this paper

    MoQA - A Multi-modal Question Answering Architecture
    • PDF
    Textbook Question Answering Under Instructor Guidance with Memory Networks
    • 7
    • Highly Influenced
    • PDF
    XTQA: Span-Level Explanations of the Textbook Question Answering
    • PDF
    ISAAQ - Mastering Textbook Questions with Pre-trained Transformers and Bottom-Up and Top-Down Attention
    • Highly Influenced
    • PDF
    Answering Questions about Data Visualizations using Efficient Bimodal Fusion
    • 12
    • PDF
    Diverse Visuo-Lingustic Question Answering (DVLQA) Challenge
    • Highly Influenced

    References

    SHOWING 1-10 OF 33 REFERENCES
    MCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text
    • 479
    • PDF
    Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
    • 779
    • PDF
    SQuAD: 100, 000+ Questions for Machine Comprehension of Text
    • 2,491
    • PDF
    Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question
    • 350
    • PDF
    VQA: Visual Question Answering
    • 2,008
    • Highly Influential
    • PDF
    Dynamic Memory Networks for Visual and Textual Question Answering
    • 559
    • Highly Influential
    • PDF
    Bidirectional Attention Flow for Machine Comprehension
    • 1,252
    • PDF
    MovieQA: Understanding Stories in Movies through Question-Answering
    • 339
    • PDF
    A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task
    • 428
    • PDF
    Ask Your Neurons: A Neural-Based Approach to Answering Questions about Images
    • 462
    • PDF