Are You Smarter Than a Sixth Grader? Textbook Question Answering for Multimodal Machine Comprehension

@article{Kembhavi2017AreYS,
  title={Are You Smarter Than a Sixth Grader? Textbook Question Answering for Multimodal Machine Comprehension},
  author={Aniruddha Kembhavi and Minjoon Seo and Dustin Schwenk and Jonghyun Choi and Ali Farhadi and Hannaneh Hajishirzi},
  journal={2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2017},
  pages={5376-5384}
}
We introduce the task of Multi-Modal Machine Comprehension (M3C), which aims at answering multimodal questions given a context of text, diagrams and images. [...] Key Method We extend state-of-the-art methods for textual machine comprehension and visual question answering to the TQA dataset. Our experiments show that these models do not perform well on TQA. The presented dataset opens new challenges for research in question answering and reasoning across multiple modalities.Expand Abstract

Citations

Publications citing this paper.
SHOWING 1-10 OF 52 CITATIONS

Look, Read and Enrich - Learning from Scientific Figures and their Captions

VIEW 6 EXCERPTS
CITES METHODS & BACKGROUND
HIGHLY INFLUENCED

Essay-Anchor Attentive Multi-Modal Bilinear Pooling for Textbook Question Answering

VIEW 5 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

Textbook Question Answering Under Instructor Guidance with Memory Networks

VIEW 8 EXCERPTS
CITES METHODS & BACKGROUND
HIGHLY INFLUENCED

FILTER CITATIONS BY YEAR

2017
2020

CITATION STATISTICS

  • 6 Highly Influenced Citations

  • Averaged 17 Citations per year from 2017 through 2019

  • 100% Increase in citations per year in 2019 over 2018

References

Publications referenced by this paper.
SHOWING 1-10 OF 33 REFERENCES

Long Short-Term Memory

VIEW 7 EXCERPTS
HIGHLY INFLUENTIAL

and P

  • P. Rajpurkar, J. Zhang, K. Lopyrev
  • Liang. Squad: 100,000+ questions for machine comprehension of text. In EMNLP
  • 2016
VIEW 8 EXCERPTS
HIGHLY INFLUENTIAL

VQA: Visual Question Answering

VIEW 6 EXCERPTS
HIGHLY INFLUENTIAL

Memory Networks

VIEW 3 EXCERPTS
HIGHLY INFLUENTIAL

Hierarchical Memory Networks

VIEW 2 EXCERPTS

A Diagram is Worth a Dozen Images

VIEW 2 EXCERPTS