An Examination of Interrater Reliability for Scoring the Rorschach Comprehensive System in Eight Data Sets

@article{Meyer2002AnEO,
  title={An Examination of Interrater Reliability for Scoring the Rorschach Comprehensive System in Eight Data Sets},
  author={Gregory J. Meyer and Mark Hilsenroth and Dirk Baxter and John E. Exner and James Christopher Fowler and Craig Piers and J.D. Resnick},
  journal={Journal of Personality Assessment},
  year={2002},
  volume={78},
  pages={219 - 274}
}
In this article, we describe interrater reliability for the Comprehensive System (CS; Exner, 1993) in 8 relatively large samples, including (a) students, (b) experienced researchers, (c) clinicians, (d) clinicians and then researchers, (e) a composite clinical sample (i.e., a to d), and 3 samples in which randomly generated erroneous scores were substituted for (f) 10%, (g) 20%, or (h) 30% of the original responses. Across samples, 133 to 143 statistically stable CS scores had excellent… 
View on Taylor & Francis
utoledo.edu
150 Citations
Highly Influential Citations
19
Background Citations
60
Methods Citations
31
Results Citations
11

150 Citations

An Inter-Rater Reliability Study for the Rorschach Performance Assessment System
TLDR
Overall, the findings indicate good to excellent inter-rater reliability for the great majority of codes and are consistent with previous findings of strong inter- raters reliability for alternative Rorschach systems and scores.
The Interclinician Reliability of Rorschach Interpretation in Four Data Sets
TLDR
Compared to meta-analyses of interrater reliability in psychology and medicine, the findings indicate these clinicians could reliably interpret Rorschach CS data.
Rorschach Performance Assessment System (R-PAS) Interrater Reliability in a Brazilian Adolescent Sample and Comparisons With Three Other Studies
TLDR
Examination of interrater reliability for scoring the Rorschach Performance Assessment System (R-PAS) in a sample of 89 adolescents using exact agreement intraclass correlations coefficient (ICCs) showed that the ICCs for most variables had low variability across studies, suggesting clear coding guidelines.
An Interrater Reliability Study of Rorschach Performance Assessment System (R–PAS) Raw and Complexity-Adjusted Scores
TLDR
Examination of R–PAS IRR of Page 1 and Page 2 raw and complexity-adjusted scores with 112 Italian Rorschach protocols found most of the variables were characterized by good to excellent IRR.
A Rorschach Stability Study in a Nonpatient Adult Sample
TLDR
Results provide further support for the reliability of several measures, including personality, cognitive or self/relational variables yielded higher test�retest correlations than emotional and coping variables.
No agreement between the Rorschach and self-assessed personality traits derived from the Comprehensive System.
TLDR
The mean correlation suggests that SAPCS statements overlapping with Rorschach Comprehensive System interpretive statements do not measure the same constructs as the RCS variables to which theinterpretive statements are referring.
Empirical support for interrater reliability of Rorschach Comprehensive System coding.
TLDR
Results confirm the strong empirical evidence of the interrater reliability of this measure when scored by a well-trained and diverse group of researchers and clinicians and suggest that large samples are needed for stable reliability estimates of low base-rate variables and that estimates of the reliability of low Base rate variables are subject to error.
An Investigation of Interrater Reliability for the Rorschach Performance Assessment System (R–PAS) in a Nonpatient U.S. Sample
TLDR
Results suggest R–PAS is an effective method with high interrater reliability supporting its empirical basis, and concordance rates achieved good to excellent agreement.
Rorschach Comprehensive System Data for a Sample of 108 Normative Subjects From The Netherlands
TLDR
It is argued, on the basis of findings from psychiatric epidemiology, that the exclusion of subjects with a history of mental disorder or mental health treatment from Rorschach normative data bases yields a dataset that is nonrepresentative of the general population.
Field Reliability of Comprehensive System Scoring in an Adolescent Inpatient Sample
TLDR
The reliability of both response-level codes and protocol-level scores among 84 adolescent psychiatric inpatients in a clinical setting is evaluated, indicating reliable scoring is possible even in field settings.
...
...

References

SHOWING 1-10 OF 96 REFERENCES
Interobserver Agreement, Intraobserver Reliability, and the Rorschach Comprehensive System
TLDR
Reliability was analyzed at multiple levels of Comprehensive System data, including response-level individual codes and coding decisions and ratios, percentages, and derivations from the Structural Summary.
Simple Procedures to Estimate Chance Agreement and Kappa for the Interrater Reliability of Response Segments Using the Rorschach Comprehensive System
When determining interrater reliability for scoring the Rorschach Comprehensive System (Exner, 1993), researchers often report coding agreement for response segments (i.e., Location, Developmental…
Error in Research and Assessment Data With an Erratum for Meyer (1993)
After reviewing literature detailing the ubiquity of error in research and assessment data, I describe mistakes found with MMPI-2 and Rorschach scores in an earlier publication (Meyer, 1993). The…
Assessing Reliability : Critical Corrections for a Critical Examination of the Rorschach Comprehensive System
Wood, Nezworski, and Stejskal (1996a, 1996b) argued that the Rorschach Comprehensive System (CS) lacked many essential pieces of reliability data and that the available evidence indicated that…
Rorschach interrater agreement estimates: An empirical evaluation
A standardized estimation of Rorschach interrater agreement is needed. Percentage agreement, although widely used, is found to be unsuitable. Forty-one protocols from adults in both a normal and a…
The Rorschach Schizophrenia Index (SCZI): an examination of reliability, validity, and diagnostic efficiency.
TLDR
The results of this study indicate that the SCZI is internally consistent and can be reliably scored and was used effectively in differentiating PD patients from patients with an Axis II disorder and from the participants in the nonclinical sample.
Standardizing procedures for calculating rorschach interrater reliability: conceptual and empirical foundations.
TLDR
It is shown that the magnitude of differences between methods of quantifying interrater agreement and the associated reductions of error are significant, and a standard method for quantifyingInterrater Agreement in Rorschach research is proposed.
The Comprehensive System for the Rorschach: A Critical Examination
The Comprehensive System (Exner, 1993) is widely accepted as a reliable and valid approach to Rorschach interpretation However, the present article calls attention to significant problems with the…
Current nonpatient data for the Rorschach, WAIS-R, and MMPI-2
A literature review yields a surprisingly small number of nonpatient studies focusing on three widely used assessment tools: the WAIS-R, Rorschach, and MMPI-2. In this study, 123 nonpatient adults…
The Structured Clinical Interview for DSM-III-R (SCID). II. Multisite test-retest reliability.
A test-retest reliability study of the Structured Clinical Interview for DSM-III-R was conducted on 592 subjects in four patient and two nonpatient sites in this country as well as one patient site…
...
...