Solving Geometry Problems: Combining Text and Diagram Interpretation

@inproceedings{Seo2015SolvingGP,
  title={Solving Geometry Problems: Combining Text and Diagram Interpretation},
  author={Minjoon Seo and Hannaneh Hajishirzi and Ali Farhadi and Oren Etzioni and Clint Malcolm},
  booktitle={EMNLP},
  year={2015}
}
This paper introduces GEOS, the first automated system to solve unaltered SAT geometry questions by combining text understanding and diagram interpretation. [...] Key Method GEOS then feeds the description to a geometric solver that attempts to determine the correct answer. In our experiments, GEOS achieves a 49% score on official SAT questions, and a score of 61% on practice questions. 1 Finally, we show that by integrating textual and visual information, GEOS boosts the accuracy of dependency and semantic…Expand
Understanding Plane Geometry Problems by Integrating Relations Extracted from Text and Diagram
TLDR
The proposed method can mine geometric relations in high accuracy and it can understand some problems that cannot be understood by using text only or by using diagram only. Expand
Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning
TLDR
This work constructs a new largescale benchmark, Geometry3K, consisting of 3,002 geometry problems with dense annotation in formal language, and proposes a novel geometry solving approach with formal language and symbolic reasoning, called Interpretable Geometry Problem Solver (InterGPS). Expand
AUTOMATIC UNDERSTANDING AND FORMALIZATION OF NATURAL LANGUAGE GEOMETRY PROBLEMS USING SYNTAX-SEMANTICS MODELS
Automatic understanding of natural language problems is a long-standing challenge research problem in automatic solving. This paper models the understanding of geometry questions as a problem ofExpand
Discourse in Multimedia: A Case Study in Extracting Geometry Knowledge from Textbooks
TLDR
It is concluded that the discourse and text layout features in multimedia text provide information that is complementary to lexical semantic information and can be used to improve an existing solver for geometry problems, making it more accurate as well as more explainable. Expand
Discourse in Multimedia: A Case Study in Extracting Geometry Knowledge from Textbooks
To ensure readability, text is often written and presented with due formatting. These text formatting devices help the writer to effectively convey the narrative. At the same time, these help theExpand
A Diagram is Worth a Dozen Images
TLDR
An LSTM-based method for syntactic parsing of diagrams and a DPG-based attention model for diagram question answering are devised and a new dataset of diagrams with exhaustive annotations of constituents and relationships is compiled. Expand
Parsing to Programs: A Framework for Situated QA
TLDR
Parsing to Programs, a framework that combines ideas from parsing and probabilistic programming for situated question answering, is introduced and a system that solves pre-university level Newtonian physics questions is built. Expand
Ontology-Controlled Geometric Solver
TLDR
An ontology-controlled geometric solver that functions as a part of an integrated problem-solving system with a natural language interface and is implemented in a prototype version of the system for the “school geometry” subject area. Expand
A Framework for Solving Explicit Arithmetic Word Problems and Proving Plane Geometry Theorems
This paper presents a framework for solving math problems stated in a natural language (NL) and applies the framework to develop algorithms for solving explicit arithmetic word problems and provingExpand
Understanding explicit arithmetic word problems and explicit plane geometry problems using syntax-semantics models
TLDR
The approach proposed in this paper models understanding math problems as a problem of relation extraction, instead of as the problem of understanding the semantics of natural language, and proposes a syntax-semantics (S2) model method to extract math relations. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 56 REFERENCES
Diagram Understanding in Geometry Questions
TLDR
This paper presents a method for diagram understanding that identifies visual elements in a diagram while maximizing agreement between textual and visual data, and shows that the method's objective function is submodular. Expand
Understanding text with an accompanying diagram
TLDR
A program which understands elementary physics problems, parsing the text and picture components of the problem together, and produces an abstract model of the information contained within. Expand
A Discriminative Graph-Based Parser for the Abstract Meaning Representation
TLDR
The first approach to parse sentences into meaning representation, a semantic formalism for which a grow- ing set of annotated examples is available, is introduced, providing a strong baseline for improvement. Expand
Synthesis of Geometry Proof Problems
TLDR
The experimental results indicate that the semi-automated methodology for generating geometric proof problems of the kind found in a highschool curriculum can effectively generate proof problems in elementary geometry. Expand
Every Picture Tells a Story: Generating Sentences from Images
TLDR
A system that can compute a score linking an image to a sentence, which can be used to attach a descriptive sentence to a given image, or to obtain images that illustrate a given sentence. Expand
Understanding Machines from Text and Diagrams.
Publisher Summary This chapter provides an overview of information involved in understanding a machine and the role that diagrams play in communicating this information. The chapter proposes a modelExpand
Large-scale Semantic Parsing without Question-Answer Pairs
TLDR
This paper introduces a novel semantic parsing approach to query Freebase in natural language without requiring manual annotations or question-answer pairs and converts sentences to semantic graphs using CCG and subsequently grounds them to Freebase guided by denotations as a form of weak supervision. Expand
Baby Talk : Understanding and Generating Image Descriptions
We posit that visually descriptive language offers computer vision researchers both information about the world, and information about how people describe the world. The potential benefit from thisExpand
Computational models for integrating linguistic and visual information: A survey
  • R. Srihari
  • Computer Science
  • Artificial Intelligence Review
  • 2004
TLDR
An important contribution of this paper is to categorize existing research based on inputs and objectives, namely how to associate visual events with words and vice versa. Expand
Semantic Parsing on Freebase from Question-Answer Pairs
TLDR
This paper trains a semantic parser that scales up to Freebase and outperforms their state-of-the-art parser on the dataset of Cai and Yates (2013), despite not having annotated logical forms. Expand
...
1
2
3
4
5
...