# Measuring the accuracy of diagnostic systems.

@article{Swets1988MeasuringTA, title={Measuring the accuracy of diagnostic systems.}, author={John A. Swets}, journal={Science}, year={1988}, volume={240 4857}, pages={ 1285-93 } }

Diagnostic systems of several kinds are used to distinguish between two classes of events, essentially "signals" and "noise". For them, analysis in terms of the "relative operating characteristic" of signal detection theory provides a precise and valid measure of diagnostic accuracy. It is the only measure available that is uninfluenced by decision biases and prior probabilities, and it places the performances of diverse systems on a common, easily interpreted scale. Representative values of…

## 8,535 Citations

### ROC Analysis in the Evaluation of Intelligent Medical Systems

- Computer Science
- 2004

A new method for generating the probability density functions and confidence bounds for ROC points which is robust and accurate for any sample size is presented, particularly suited for evaluating the performance of systems where sample sizes are small.

### Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine.

- MedicineClinical chemistry
- 1993

Receiver-operating characteristic (ROC) plots provide a pure index of accuracy by demonstrating the limits of a test's ability to discriminate between alternative states of health over the complete spectrum of operating conditions.

### Classification of diagnostic tests.

- Computer ScienceInternational journal of bio-medical computing
- 1994

### Assessing Test Accuracy and its Clinical Consequences: A Primer for Receiver Operating Characteristic Curve Analysis

- MedicineAnnals of clinical biochemistry
- 1993

These are the basic features necessary for the acceptable performance of ROC curve analysis, which are not dealt with in any detail in current textbooks addressed to the needs of clinical chemists.

### Evaluation of intelligent medical systems

- Computer Science
- 2002

This work establishes an important new method for generating pdf's, and provides an accurate and robust method of producing confidence intervals for ROC curves for the small sample sizes typical of intelligent medical systems.

### Comparing and Optimizing Diagnostic Tests

- MedicineMedical decision making : an international journal of the Society for Medical Decision Making
- 1992

Though it is not appropriate for all clinical situations, the ROC analysis method can be used for most diagnostic tests whenever information is desired for its own sake or when reducing uncertainty is the goal of testing.

### Advances in statistical methodology for the evaluation of diagnostic and laboratory tests.

- MathematicsStatistics in medicine
- 1994

Two new simultaneous confidence regions associated with the ROC curve are presented, one based on Kolmogorov-Smirnov confidence bands for distribution functions and the other based on bootstrapping.

### Receiver operating characteristic analysis for intelligent medical systems-a new approach for finding confidence intervals

- Computer ScienceIEEE Trans. Biomed. Eng.
- 2000

A novel, robust and accurate method is proposed, derived from first principles, which calculates the probability density function for each point on a ROC curve for any given sample size, and provides an accurate and robust method of producing confidence intervals for ROC curves for the small sample sizes typical of intelligent medical systems.

### Receiver operating characteristic studies and measurement errors.

- MathematicsBiometrics
- 1997

Monte Carlo simulation indicates that, for several families of distributions, the bias-corrected estimators have smaller bias and comparable MSE to the usual estimator.

### Relation of Observer Agreement to Accuracy According to a Two-receiver Signal Detection Model of Diagnosis

- PsychologyMedical decision making : an international journal of the Society for Medical Decision Making
- 1989

A model of test interpretation is defined which accounts for disagreement when observers use identical diagnostic strategies, and determines an optimal diagnostic strategy with advantages over maximum likelihood and Bayes diagnostic strategies.

## References

SHOWING 1-10 OF 38 REFERENCES

### Indices of discrimination or diagnostic accuracy: their ROCs and implied models.

- PsychologyPsychological bulletin
- 1986

It is suggested that candidate indices be evaluated by calculating their relative operating characteristics (ROCs) so that the form of an index's ROC identifies the model of the discrimination process that is implied by the index, and that theoretical form can be compared with the forms of empirical ROCs.

### Problems of spectrum and bias in evaluating the efficacy of diagnostic tests.

- MedicineThe New England journal of medicine
- 1978

To determine why many diagnostic tests have proved to be valueless after optimistic introduction into medical practice, we reviewed a series of investigations and identified two major problems that…

### ROC analysis applied to the evaluation of medical imaging techniques.

- MedicineInvestigative radiology
- 1979

This paper presents a brief description of the ROC, and shows how it provides a measure of diagnostic accuracy that is free of judgmental bias.

### ROC Methodology in Radiologic Imaging

- MedicineInvestigative radiology
- 1986

This article develops ROC concepts in an intuitive way by identifying the fundamental issues that motivate ROC analysis and practical techniques for ROC data collection and data analysis.

### On the scientific evaluation of diagnostic procedures.

- MedicineRadiology
- 1949

Diagnosis is the cornerstone of modern clinical medicine. It requires accurate observation and rational deduction. In its most complete form it is the process of identifying a disease by…

### An Evaluation of Polygraphers' Judgments: A Review From a Decision Theoretic Perspective

- Sociology
- 1982

Hebrew University, Jerusalem, IsraelA critical discussion of the theory, practice, and evaluation process of the ControlQuestions Technique (CQT) is undertaken. Sources of possible contaminationin…

### Form of empirical ROCs in discrimination and diagnostic tasks: implications for theory and measurement of performance.

- PsychologyPsychological bulletin
- 1986

Measuring both accuracy and the decision criterion appropriately enables one to delineate which of these dependent variables is affected by which independent variables in psychological studies, and to assess the efficacy as well as the accuracy of diagnostic systems.

### The Effect of Verification on the Assessment of Imaging Techniques

- MedicineInvestigative radiology
- 1983

To assess relative accuracies of three different radiographic techniques, chest radiographs were taken of 66 patients with each of the three techniques, and the films were evaluated by six radiologists, finding that any one of the techniques could be shown to be better than the other, depending on how the truth was defined.

### Construction of Receiver Operating Characteristic Curves when Disease Verification Is Subject to Selection Bias

- MathematicsMedical decision making : an international journal of the Society for Medical Decision Making
- 1984

A method to provide an unbiased estimate of the ROC curve under these sample conditions is proposed, which requires information on the probability distribution of test results in the population from which the verified sample is drawn.

### Assessment of diagnostic technologies.

- MedicineScience
- 1979

A general protocol for rigorous evaluation of diagnostic systems in medicine was applied successfully in a comparative study of two radiologic techniques, and computed tomography was found to be substantially more accurate than radionuclide scanning.