Are You Smarter Than a Sixth Grader? Textbook Question Answering for Multimodal Machine Comprehension

  title={Are You Smarter Than a Sixth Grader? Textbook Question Answering for Multimodal Machine Comprehension},
  author={Aniruddha Kembhavi and Min Joon Seo and Dustin Schwenk and Jonghyun Choi and Ali Farhadi and Hannaneh Hajishirzi},
  journal={2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
We introduce the task of Multi-Modal Machine Comprehension (M3C), which aims at answering multimodal questions given a context of text, diagrams and images. We present the Textbook Question Answering (TQA) dataset that includes 1,076 lessons and 26,260 multi-modal questions, taken from middle school science curricula. Our analysis shows that a significant portion of questions require complex parsing of the text and the diagrams and reasoning, indicating that our dataset is more complex compared… CONTINUE READING
Highly Cited
This paper has 28 citations. REVIEW CITATIONS


Publications citing this paper.
Showing 1-10 of 20 extracted citations

Essay-Anchor Attentive Multi-Modal Bilinear Pooling for Textbook Question Answering

2018 IEEE International Conference on Multimedia and Expo (ICME) • 2018
View 5 Excerpts
Highly Influenced


Publications referenced by this paper.
Showing 1-10 of 33 references

Long Short-Term Memory

Neural Computation • 1997
View 7 Excerpts
Highly Influenced

and P

P. Rajpurkar, J. Zhang, K. Lopyrev
Liang. Squad: 100,000+ questions for machine comprehension of text. In EMNLP • 2016
View 8 Excerpts
Highly Influenced

VQA: Visual Question Answering

International Journal of Computer Vision • 2015
View 6 Excerpts
Highly Influenced

Memory Networks

View 3 Excerpts
Highly Influenced


H. Xu, K. Saenko
attend and answer: Exploring question-guided spatial attention for visual question answering. In ECCV • 2016
View 2 Excerpts

Similar Papers

Loading similar papers…