Root Cause Analysis of Miscommunication Hotspots in Spoken Dialogue Systems

@inproceedings{Georgiladakis2016RootCA,
  title={Root Cause Analysis of Miscommunication Hotspots in Spoken Dialogue Systems},
  author={Spiros Georgiladakis and Georgia Athanasopoulou and Raveesh Meena and Jos{\'e} Lopes and Arodami Chorianopoulou and Elisavet Palogiannidi and Elias Iosif and Gabriel Skantze and Alexandros Potamianos},
  booktitle={INTERSPEECH},
  year={2016}
}
A major challenge in Spoken Dialogue Systems (SDS) is the detection of problematic communication (hotspots), as well as the classification of these hotspots into different types (root cause analysi ... 
View via Publisher
speech.kth.se
4 Citations
Background Citations
1

Figures and Tables from this paper

4 Citations

In-the-wild chatbot corpus: from opinion analysis to interaction problem detection
TLDR
A system built on chatbot data corresponding to conversations between customers and a virtual assistant provided by a French energy supplier company is presented, aimed at detecting in this data the expressions of user's opinions that are linked to interaction problems.
Predicting Causes of Reformulation in Intelligent Assistants
TLDR
This work focuses on reformulation, which is a useful signal of user dissatisfaction, and proposes a method to predict the reformulation causes, and demonstrates that features designed to detect the error of a specific component improve the performance of reformulation cause detection.
Finding a Meta-dialogue Strategy for Multi-intent Spoken Dialogue Systems
TLDR
Which strategies users prefer regarding metrics such as flexibility, joy, and consistency are revealed, including explaining topic transitions and topic introductions via speech receive a high user rating and is cognitively less demanding then visual cues.
What Went Wrong and Why? Diagnosing Situated Interaction Failures in the Wild
TLDR
This work presents a methodology for assessing performance, identifying problems, and diagnosing the root causes and influences of different types of failures on the overall performance of a situated interaction system functioning in the wild, applied to a dataset of interactions collected with a robot deployed in a public space inside an office building.

References

SHOWING 1-10 OF 26 REFERENCES
Data-driven Methods for Spoken Dialogue Systems : Applications in Language Understanding, Turn-taking, Error Detection, and Knowledge Acquisition
TLDR
A spoken dialogue system that enables humans to interact with computers using spoken natural language and the challenges of dealing with the ubiquity of such systems are studied.
Automatic Detection of Miscommunication in Spoken Dialogue Systems
TLDR
A range of generic features that are both automatically extractable and manually annotated were used to train two models for online detection and one for offline analysis for detecting instances of miscommunication in dialogue system interactions.
Detecting Problematic Turns in Human-Machine Interactions: Rule-induction Versus Memory-based Learning Approaches
TLDR
By applying both rule-induction and memory-based learning techniques to data obtained with a Dutch train time-table information system, the current paper demonstrates that the aforementioned features indeed lead to a method for problem detection that performs significantly above baseline.
Let's go public! taking a spoken dialog system to the real world
TLDR
The changes necessary to make the Let’s Go Public spoken dialog system usable for the general public are described and analysis of the calls and strategies used to ensure high performance is presented.
Learning to Predict Problematic Situations in a Spoken Dialogue System: Experiments with How May I Help You?
TLDR
Results on learning to automatically identify and predict problematic human-computer dialogues in a corpus of 4774 dialogues collected with the How May I Help You spoken dialogue system are reported.
Error Detection in Spoken Human-Machine Interaction
TLDR
It appears that people systematically use negative/marked cues when there are communication problems, and it is claimed that if spoken dialogue systems were able to detect such cues and change their strategy accordingly, the interaction between user and system would improve.
Prosodic and other cues to speech recognition failures
Automatic Detection of Poor Speech Recognition at the Dialogue Level
TLDR
This paper adopts a machine learning approach to learn rules from a dialogue corpus for identifying situations where the speech recognizer is performing poorly and shows a significant improvement over the baseline.
User states, user strategies, and system performance: how to match the one with the other
TLDR
This paper wants to deal with additional information that can be useful for deciding whether an automatic dialogue system performs well or not and correlate it with system performance.
A principled approach for rejection threshold optimization in spoken dialog systems
TLDR
A data-driven method is proposed for determining the relative costs of errors in spoken dialog systems, and then using these costs to optimize state-specific rejection thresholds to avoid misunderstandings and false rejections.
...
...