Discourse in Multimedia: A Case Study in Extracting Geometry Knowledge from Textbooks

  title={Discourse in Multimedia: A Case Study in Extracting Geometry Knowledge from Textbooks},
  author={Mrinmaya Sachan and Kumar Avinava Dubey and Eduard H. Hovy and Tom Michael Mitchell and Dan Roth and Eric P. Xing},
  journal={Computational Linguistics},
To ensure readability, text is often written and presented with due formatting. These text formatting devices help the writer to effectively convey the narrative. At the same time, these help the readers pick up the structure of the discourse and comprehend the conveyed information. There have been a number of linguistic theories on discourse structure of text. However, these theories only consider unformatted text. Multimedia text contains rich formatting features that can be leveraged for… Expand
2 Citations
AI2D-RST: A multimodal corpus of 1000 primary school science diagrams
A multi-layer annotation schema that provides a rich description of diagram elements into perceptual units, the connections set up by diagrammatic elements such as arrows and lines, and the discourse relations between diagram elements, which are described using Rhetorical Structure Theory (RST). Expand
Emerging materials intelligence ecosystems propelled by machine learning
The age of cognitive computing and artificial intelligence (AI) is just dawning. Inspired by its successes and promises, several AI ecosystems are blossoming, many of them within the domain ofExpand


Integrating Text Formatting and Text Generation
This paper presents a model for representing the architecture of documents for natural language generation from a specialized sublanguage (in the sense of Z. Harris) of natural language, and proposes a brief survey of a system of formatted text generation, based on this model. Expand
Automatic Generation of Formatted Text
This work describes how work on the automated planning of multisentence text and on the display of information in a multimedia system led to the insight that text formatting devices such as footnotes, italicized regions, enumerations, etc., can be planned automatically by a text structure planning process. Expand
Discourse indicators for content selection in summarization
The results establish the usefulness of discourse features and find that lexical overlap provides a simple and cheap alternative to discourse for computing text structure with comparable performance for the task of content selection. Expand
Towards Constructive Text, Diagram, and Layout Generation for Information Presentation
It is demonstrated that layout offers a rich resource for achieving presentational coherence, alongside more traditional resources such as text-formatting and the text-internal marking of discourse connections, and an integrated approach to layout, text, and diagram generation is introduced. Expand
Pattern Matching and Discourse Processing in Information Extraction from Japanese Text
A Japanese information extraction system that merges information using a pattern matcher and discourse processor that approaches human performance is reported on. Expand
Discourse segmentation in aid of document summarization
  • B. Boguraev, Mary S. Neff
  • Computer Science
  • Proceedings of the 33rd Annual Hawaii International Conference on System Sciences
  • 2000
Evaluated against the corpus used in the development of the baseline summarizer, summaries derived either by means of segmentation analysis alone, or by a mix of strategies for combining salience calculation and topic shift detection, are shown to be of comparable, and under certain conditions even better quality. Expand
Rhetorical relations for information retrieval
A language model modification is presented that considers rhetorical relations when estimating the relevance of a document to a query and shows that certain rhetorical relations can benefit retrieval effectiveness notably. Expand
Recalling and Summarizing Complex Discourse
In this paper we investigate the properties of complex semantic information processing involved in the comprehension, (re-)production and summarizing of longer narrative discourse. In the theoreticalExpand
Structure and Rules in Automated Multimedia Presentation Planning
A prototype planning system that performs the information-to-media allocation is described, arguing that since media allocation rules depend on the characteristics of the information to be presented, they can only be applied once the overall discourse structure has been essentially planned out and the individual portions of information have become apparent. Expand
Discourse processing for context question answering based on linguistic knowledge
Three models driven by Centering Theory for discourse processing are examined: a reference model that resolves pronoun references for each question, a forward model that makes use of the forward looking centers from previous questions, and a transition model that takes into account the transition state between adjacent questions. Expand