Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content

  title={Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content},
  author={Alan Lundgard and Arvind Satyanarayan},
Natural language descriptions sometimes accompany visualizations to better communicate and contextualize their insights, and to improve their accessibility for readers with disabilities. However, it is difficult to evaluate the usefulness of these descriptions, and how effectively they improve access to meaningful information, because we have little understanding of the semantic content they convey, and how different readers receive this content. In response, we introduce a conceptual model for… 

Figures and Tables from this paper


Augmenting Visualizations with Interactive Data Facts to Facilitate Interpretation and Communication
Voder is presented, a system that lets users interact with automatically-generated data facts to explore both alternative visualizations to convey a data fact as well as a set of embellishments to highlight a fact within a visualization.
Eviza: A Natural Language Interface for Visual Analysis
Eviza provides a natural language interface for an interactive query dialog with an existing visualization rather than starting from a blank sheet and asking closed-ended questions that return a single text answer or static visualization.
Collecting and Characterizing Natural Language Utterances for Specifying Data Visualizations
An online study shows participants a series of visualizations and asks them to provide utterances they would pose to generate the displayed charts, then curated a dataset of 893 utterances and characterized the utterances according to their phrasing.
Communicative Visualizations as a Learning Problem
  • Eytan Adar, Elsie Lee
  • Computer Science, Medicine
    IEEE Transactions on Visualization and Computer Graphics
  • 2021
This study identified the broad range of objectives in communicative visualizations and the prevalence of certain objective types, and illustrates how the learning-based approach provides a framework for understanding a wide array of communicative goals.
Visualization Rhetoric: Framing Effects in Narrative Visualization
It is described how designers and researchers can benefit from the potentially positive aspects of visualization rhetoric in designing engaging, layered narrative visualizations and how the framework can shed light on how a visualization design prioritizes specific interpretations.
The Curious Case of Combining Text and Visualization
Eye tracking is used to compare how people extract information from text and visualization and provides evidence that visualization makes it easier to identify critical information, but that once identified as critical, information is more easily extracted from the text.
Understanding Visualization: A Formal Approach Using Category Theory and Semiotics
This paper generalizes previous work on the formal characterization of visualization by, inter alia, Ziemkiewicz and Kosara and allows us to formally distinguish properties of the visualization process that previous work does not.
Automatic Annotation Synchronizing with Textual Description for Visualization
This paper proposes a technique for automatically annotating visualizations according to the textual description with a Mask R-CNN model, and demonstrates the utility and usability of the method through a user study with use cases.
Accessible Visualization: Design Space, Opportunities, and Challenges
This work mapped a design space for accessible visualization that includes seven dimensions: user group, literacy task, chart type, interaction, information granularity, sensory modality, assistive technology and reflected on the dimensions.
Answering Questions about Charts and Generating Visual Explanations
An automatic chart question answering pipeline that generates visual explanations describing how the answer was obtained is developed that significantly outperform in transparency and are comparable in usefulness and trust to human-generated explanations.