A Design Engineering Approach for Quantitatively Exploring Context-Aware Sentence Retrieval for Nonspeaking Individuals with Motor Disabilities

  title={A Design Engineering Approach for Quantitatively Exploring Context-Aware Sentence Retrieval for Nonspeaking Individuals with Motor Disabilities},
  author={Per Ola Kristensson and James Lilley and Rolf Black and Annalu Waller},
  journal={Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems},
  • P. Kristensson, J. Lilley, A. Waller
  • Published 21 April 2020
  • Computer Science
  • Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
Nonspeaking individuals with motor disabilities typically have very low communication rates. This paper proposes a design engineering approach for quantitatively exploring context-aware sentence retrieval as a promising complementary input interface, working in tandem with a word-prediction keyboard. We motivate the need for complementary design engineering methodology in the design of augmentative and alternative communication and explain how such methods can be used to gain additional design… 
KWickChat: A Multi-Turn Dialogue System for AAC Using Context-Aware Sentence Generation by Bag-of-Keywords
KWickChat is a sentence-based text entry system that automatically generates suitable sentences for the nonspeaking partner based on keyword entry that is underpinned by a GPT-2 language model and leverages context information, including dialogue history and persona tags, to improve the quality of the generated responses.
Automated Generation of Storytelling Vocabulary from Photographs for use in AAC
A novel method for generating context-related vocabulary from photographs of personally relevant events aimed at supporting people with language impairments in retelling their past experiences is contributed.
Design and Analysis of Intelligent Text Entry Systems with Function Structure Models and Envelope Analysis
A complementary approach is introduced by adapting function structure models from engineering design to extend their use by extracting controllable and uncontrollable parameters from function structure model and visualizing their impact using envelope analysis.
A model-based design tool for 3D GUI layout design that accommodates user attributes
This dissertation creates a novel model-based design toolkit that facilitates the design, creation, and exploration of inclusively immersive 3D UI layouts and is the first to combine models related to physical ergonomics, cognition, and visual perception into a single product.
Using Computer Simulations to Investigate the Potential Performance of ‘A to B’ Routing Systems for People with Mobility Impairments
To explore what is required to produce an effective routing tool, Monte-Carlo simulations are conducted, illustrating the need to focus on barrier minimization, instead of barrier avoidance, due to the limitations of what can be achieved by any accessibility documentation tool.


What we talk about: designing a context-aware communication tool for people with aphasia
The design and development of TalkAbout is described, a context-aware, adaptive AAC system that provides users with a word list that is adapted to their current location and conversation partner and guidelines for developing and evaluating context- Aware technology for people with aphasia are presented.
Text input for motor-impaired people
This paper provides an overview of 150 publications regarding text input for motor-impaired people and describes current state of the art. It focuses on common techniques of text entry including
Evaluating the use of TalksBac, a predictive communication device for nonfluent adults with aphasia.
This study has shown that TalksBac has the potential to augment the communication abilities of nonfluent adults with aphasia, who have not been able to develop their own compensatory strategies.
Humsher: a predictive keyboard operated by humming
Humsher -- a novel text entry method operated by the non-verbal vocal input, specifically the sound of humming, utilizes an adaptive language model for text prediction and uses dynamic layout and static layout.
Computer-aided conversation: A prototype system for nonspeaking people with physical disabilities
ABSTRACT This article describes the development, use, and initial evaluation of a prototype computer system to enable nonspeaking persons with severe disabilities to engage in conversation on broad
The Imagination of Crowds: Conversational AAC Language Modeling using Crowdsourcing and Large Data Sources
This paper demonstrates how to crowd-source the creation of a large set of fictional AAC messages and shows that these messages model conversational AAC better than the currently used datasets based on telephone conversations or newswire text.
Using semantic lexicons for full text message retrieval in a communication aid
Evaluation results show that information about word frequency can solve problems that occurred in the semantic query expansion because of taxonomies that have too many intermediate steps between closely related words.
Effective Phrase Prediction
A FussyTree structure is introduced and the concept of a significant phrase is introduced to address the challenges of autocompletion at the level of a multi-word "phrase".
Utterance-based systems: organization and design of AAC interfaces
This research involves the design and development of a coherent and intuitive AAC interface for an utterance-based system built upon theoretical evidence and observation of commercial-grade AAC interface software.
Whole Utterance Approaches in AAC
This account focuses on the theoretical origins of utterance-based devices (UBDs) and the range of approaches that these have produced and the issue of a causal relationship between conversational rate and positive attributions.