An Integrated Dialog Simulation Technique for Evaluating Spoken Dialog Systems

Abstract

This paper proposes a novel integrated dialog simulation technique for evaluating spoken dialog systems. Many techniques for simulating users and errors have been proposed for use in improving and evaluating spoken dialog systems, but most of them are not easily applied to various dialog systems or domains because some are limited to specific domains or others require heuristic rules. In this paper, we propose a highly-portable technique for simulating user intention, utterance and Automatic Speech Recognition (ASR) channels. This technique can be used to rapidly build a dialog simulation system for evaluating spoken dialog systems. We propose a novel user intention modeling and generating method that uses a linear-chain conditional random field, a data-driven domain specific user utterance simulation method, and a novel ASR channel simulation method with adjustable error recognition rates. Experiments using these techniques were carried out to evaluate the performance and behavior of previously developed dialog systems designed for navigation dialogs, and it turned out that our approach is easy to set up and shows the similar tendencies of real users.

Extracted Key Phrases

11 Figures and Tables

Cite this paper

@inproceedings{Jung2008AnID, title={An Integrated Dialog Simulation Technique for Evaluating Spoken Dialog Systems}, author={Sangkeun Jung and Cheongjae Lee and Kyungduk Kim and Gary Geunbae Lee}, year={2008} }