Given a recording of a lecture, one cannot easily locate a topic of interest, or skim for important points. However, by presenting the user with a summary of a discourse, listening to speech can be made more efficient. One approach to the problem of summarizing and skimming speech has been termed "emphasis detection." This study evaluates an emphasis detection approach by comparing the speech segments selected by the algorithm with a hierarchical segmentation of a discourse sample (based on [Grosz & Sidner 1986]). The results show that a high percentage of segments selected by the algorithm correspond to discourse boundaries, in particular, segment beginnings in the discourse structure. Further analysis is needed to identify cues that distinguish the hierarchical structure. The ultimate goal is to determine whether it is feasible to "outline" speech recordings using intonational and limited text-based analyses. tions. A limitation of this work is that the structure of the speech is not identifiedmwhile salient segments are determined, the relationships among them are not. This study evaluates Arons’ emphasis detection approach by comparing the speech segments selected by the algorithm with a hierarchical segmentation of the discourse (based on [Grosz & Sidner 1986]). By incorporating knowledge about discourse structure, speech summarization work can be expanded in two significant ways. First, techniques are needed for determining the structure and relationships among speech segments identified as salient. Secondly, better methods can be developed for determining the validity of the results. Currently, evaluation is difficult since there is a lack of a clear definition of "emphasis" or what constitutes a good audio summary. Discourse structure provides a foundation upon which emphasis detection and structure recognition algorithms can be evaluated.