Roland Roller

Learn More
In this paper, we introduce the SpeechEval system, a platform for the automatic evaluation of spoken dialog systems on the basis of learned user strategies. The increasing number of spoken dialog systems calls for efficient approaches for their development and testing. The goal of SpeechEval is the minimization of hand-crafted resources to maximize the(More)
We describe ongoing work to integrate statistical user models in the usability engineering process of spoken dialogue systems. The idea is to generate user dialogue actions up to the spoken utterances in response to system utterances and directly feed them to the system under test. The underlying user models are derived semi-automatically from dialogue(More)
Distant supervision is a useful technique for creating relation classifiers in the absence of labelled data. The approaches are often evaluated using a held-out portion of the distantly labelled data, thereby avoiding the need for lablelled data entirely. However, held-out evaluation means that systems are tested against noisy data, making it difficult to(More)
In this paper, we describe our contribution to the Spoken Dialog Challenge. We set up a user simulation using the large Let's Go corpus as resource to build our models. Automatic calls were made to all four dialog systems in the SDC, bus information systems that cover the schedule of Pittsburgh, PA. We discuss in detail the architecture and required setup(More)
The increasing number of spoken dialog systems calls for efficient approaches for their development and testing. Our goal is the minimization of hand-crafted resources to maximize the portability of this evaluation environment across spoken dialog systems and domains. In this paper we discuss the user simulation technique which allows us to learn general(More)
Distant supervision is a widely applied approach to automatic training of relation extraction systems and has the advantage that it can generate large amounts of labelled data with minimal effort. However , this data may contain errors and consequently systems trained using distant supervision tend not to perform as well as those based on manually labelled(More)
This paper presents a system for the nor-malization of concept mentions in clinical narratives. We evaluate and compare it against a popular, open-source solution that is frequently used for natural language processing of clinical text. The evaluation is based on a manually annotated dataset of 72 discharge summaries taken from the i2b2-corpus. Besides the(More)