The DET curve in assessment of detection task performance

@article{Martin1997TheDC,
  title={The DET curve in assessment of detection task performance},
  author={Alvin F. Martin and George R. Doddington and Teresa M. Kamm and Mark Ordowski and Mark A. Przybocki},
  journal={5th European Conference on Speech Communication and Technology (Eurospeech 1997)},
  year={1997}
}
Abstract : We introduce the DET Curve as a means of representing performance on detection tasks that involve a tradeoff of error types. We discuss why we prefer it to the traditional ROC Curve and offer several examples of its use in speaker recognition and language recognition. We explain why it is likely to produce approximately linear curves. We also note special points that may be included on these curves, how they are used with multiple targets, and possible further applications. 

Figures from this paper

Discriminative learning for optimizing detection performance in spoken language recognition
TLDR
Novel approaches for optimizing the detection performance in spoken language recognition by directly relate model parameters to two performance metrics of interest, the detection cost function and the area under the detection-error-tradeoff curve, respectively are proposed.
On Linear DETs
TLDR
This paper derives a set of conditions on the underlying probability distributions to produce linear DET plots in a generalized setting and shows that the linear DETs on a normal deviate scale are not exclusively produced by normal distributions, however, that normal distributions play an unique role in the threshold behavior as one moves along the DET line.
Application-independent evaluation of speaker detection
Optimizing the Performance of Spoken Language Recognition With Discriminative Training
TLDR
A performance metrics optimization (PMO) approach to optimizing the detection performance of Gaussian mixture model classifiers and shows that the PMO approach effectively improves the performance over the maximum-likelihood training approach.
Speaker detection without models
  • D. GillickS. StaffordB. Peskin
  • Computer Science
    Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005.
  • 2005
TLDR
An algorithm for speaker detection that scores a test segment by comparing it directly to similar instances of that speech in the training data, which achieves error rates close to 1% on the NIST 2001 extended data task and performs extremely well in combination with a standard Gaussian mixture model system.
An Introduction to Application-Independent Evaluation of Speaker Recognition Systems
TLDR
This chapter introduces an evaluation measure, , that can properly evaluate the discrimination abilities of the log-likelihood-ratio scores, as well as the quality of the calibration of a speaker detector.
The NIST Speaker Recognition Evaluations: 1996-2001
We discuss the history and purposes of the NIST evaluations of speaker recognition performance. We cover the sites that have participated, the performance measures used, and the formats used to
DETAC: a discriminative criterion for speaker verification
TLDR
This paper argues that the new criterion extends several conventional methods such as the maximum posterior training by logistic regression and the linear discriminative analysis projection, by a second aspect “reshaping” the Bayes error area in favor of a relevant operating range.
COMPARISON OF EVALUATION METRICS FOR SENTENCE BOUNDARY DETECTION
TLDR
It is shown that it is helpful to use curves as well as a single performance metric, and that different curves show different advantages in visualization, and the data skewness also has an impact on the metrics.
...
...

References

SHOWING 1-10 OF 10 REFERENCES
The Relative Operating Characteristic in Psychology
TLDR
The ROC is an analytical technique that quite effectively isolates the effects of the observer's response bias, or decision criterion, in the study of discrimination behavior and enhances the understanding of the perceptual and cognitive phenomena that depend directly on these fundamental processes.
Detection theory and psychophysics
"October 30, 1956." "This report is based on a thesis submitted to the Department of Economics and Social Science, M.I.T., in partial fulfillment of the requirements for the degree of Doctor of
Signal detection theory and psychophysics
TLDR
This book discusses statistical decision theory and sensory processes in signal detection theory and psychophysics and describes how these processes affect decision-making.
Signal detection theory and ROC analysis
ed., “Signal Detection and Recognition by Human Observers
  • 1964
NIST web site -http://www.nist.gov/speech
  • NIST web site -http://www.nist.gov/speech
Signal Detection and Recognition by Human Observers
  • Signal Detection and Recognition by Human Observers
  • 1964
Language Recognition Workshop Notebook
  • Language Recognition Workshop Notebook
  • 1996
Speaker Recognition Workshop Notebook
  • Speaker Recognition Workshop Notebook
  • 1996