Intraclass correlations: uses in assessing rater reliability.

@article{Shrout1979IntraclassCU,
  title={Intraclass correlations: uses in assessing rater reliability.},
  author={Patrick E. Shrout and Joseph L. Fleiss},
  journal={Psychological bulletin},
  year={1979},
  volume={86 2},
  pages={
          420-8
        }
}
Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed. 
View on PubMed
aliquote.org
20,963 Citations
Highly Influential Citations
1,891
Background Citations
2,165
Methods Citations
7,453
Results Citations
106

Tables from this paper

20,963 Citations

Computing Intraclass Correlations (ICC) as Estimates of Interrater Reliability in SPSS

Intraclass correlation (ICC) is one of the most commonly misused indicators of interrater reliability, but a simple step-by-step process will get it right. In this article, I provide a brief review…

Intraclass Correlation Coefficients

Intraclass correlation coefficients (ICCs) have many uses in psychology and related disciplines. They are often used to assess reliability, to deduce heritability, and to quantify similarity in…

Inter-Rater Reliability: Intraclass Correlation Coefficients

A computer program to calculate six different forms of intraclass correlation coefficients is described. The computational formulas used, input preparation and output description are given. The…

Inter-Rater Reliability: Intraclass Correlation Coefficients

A computer program to calculate six different forms of intraclass correlation coefficients is described. The computational formulas used, input preparation and output description are given. The…

On the Usefulness of Interrater Reliability Coefficients

For four data sets of different measurement levels, we computed 20 coefficients that estimate interrater reliability. The results show that the coefficients provide very different numerical values…

On the Usefulness of Interrater Reliability Coefficients

For four data sets of different measurement levels, we computed 20 coefficients that estimate interrater reliability. The results show that the coefficients provide very different numerical values…

Testing the Equality of Two Related Intraclass Reliability Coefficients

An approximate statistical test of the equality of two intraclass reliability coefficients based on the same sample of people is derived. Such a test is needed when a researcher wishes to compare the…

Rater Reliability: A Maximum Likelihood Confirmatory Factor-Analytic Approach.

A procedure for evaluating a variety of rater reliability models is presented, and the results contrasted to those found with an intraclass correlation approach.

A Review of Intraclass Correlation.

Against an historical backdrop, this paper summarizes four uses of intraclass correlation of importance to contemporary researchers in the behavioral sciences. First, it shows how the intraclass…

Choosing the best index for the average score intraclass correlation coefficient

  • G. Shieh
  • Mathematics
    Behavior research methods
  • 2016
Analytical derivations and numerical examinations are presented to assess the bias and mean square error of the alternative estimators and suggest that more advantageous indices can be recommended over ICC(2) for their theoretical implication and computational ease.
...

References

SHOWING 1-10 OF 24 REFERENCES

The Intraclass Correlation Coefficient as a Measure of Reliability

A procedure for estimating the reliability of sets of ratings in terms of the intraclass correlation coefficient is discussed. The procedure is based upon the analysis of variance and the estimation…

Approximate interval estimation for a certain intraclass correlation coefficient

When the raters participating in a reliability study are a random sample from a larger population of raters, inferences about the intraclass correlation coefficient must be based on the three mean…

The theory of the estimation of test reliability

The theoretically best estimate of the reliability coefficient is stated in terms of a precise definition of the equivalence of two forms of a test. Various approximations to this theoretical formula…

Estimation of the reliability of ratings

A procedure for estimating the reliability of sets of ratings, test scores, or other measures is described and illustrated. This procedure, based upon analysis of variance, may be applied both in the…

On Various Intraclass Correlation Reliability Coefficients

Bartko (1966, 1974) has presented some analysis of variance (ANOVA) intraclass correlation reliability coefficients that avoid some serious deficiencies not uncommonly found in reliability measures.…

Reliability formulas for independent decision data when reliability data are matched

Reliability formulas are developed (for both single observations and for composites ofk observations) for the case where reliability data are matched but decision data are independent.

The approximate sampling distribution of Kuder-Richardson reliability coefficient twenty

An approximation to the sampling distribution of Kuder-Richardson reliability formula 20 is derived, using its algebraic equivalent obtained through an items-by-subjects analysis of variance. The…

Coefficient alpha and the internal structure of tests

AbstractA general formula (α) of which a special case is the Kuder-Richardson coefficient of equivalence is shown to be the mean of all split-half coefficients resulting from different splittings of…

Unbiased Estimation of Certain Correlation Coefficients

1. Summary and introduction. This paper deals with the unbiased estimation of the correlation of two variates having a bivariate normal distribution (Sec. 2), and of the intraclass correlation, i.e.,…

Statistical Theories of Mental Test Scores.

This is a reprint of the orginal book released in 1968. Our primary goal in this book is to sharpen the skill, sophistication, and in- tuition of the reader in the interpretation of mental test data,…