Understanding the latent structure of job performance ratings.

@article{Scullen2000UnderstandingTL,
  title={Understanding the latent structure of job performance ratings.},
  author={Steven E. Scullen and Michael K. Mount and Maynard Goff},
  journal={The Journal of applied psychology},
  year={2000},
  volume={85 6},
  pages={
          956-70
        }
}
This study quantified the effects of 5 factors postulated to influence performance ratings: the ratee's general level of performance, the ratee's performance on a specific dimension, the rater's idiosyncratic rating tendencies, the rater's organizational perspective, and random measurement error. Two large data sets, consisting of managers (n = 2,350 and n = 2,142) who received developmental ratings on 3 performance dimensions from 7 raters (2 bosses, 2 peers, 2 subordinates, and self) were… 

Figures and Tables from this paper

A field study of the effects of rating purpose on the quality of multisource ratings.

Using a field sample of peers and subordinates, the current study employed generalizability theory to estimate sources of systematic variability associated with both developmental and administrative

The Use of Random Coefficient Modeling for Understanding and Predicting Job Performance Ratings

Earlier research using confirmatory factor analysis (CFA) suggests that most variance in job performance ratings is not attributable to ratee main effects. In this article, the authors point out

Manifestation of Idiosyncratic Rater Effect in Employee Performance Appraisal

Performance appraisal is the bedrock of talent management and has received much attention from scholars and researchers alike in their pursuit to develop accurate, objective, and robust Performance

RATER SOURCE EFFECTS ARE ALIVE AND WELL AFTER ALL

Recent research has questioned the importance of rater perspective effects on multisource performance ratings (MSPRs). Although making a valuable contribution, we hypothesize that this research has

Is there a general factor in ratings of job performance? A meta-analytic framework for disentangling substantive and error influences.

TLDR
Construct-level correlations among rated dimensions of job performance were substantially inflated by halo for both supervisory (33%) and peer (63%) intrarater correlations, which have important implications for the measurement ofJob performance and for theories of jobperformance.

The Performance Appraisal Milieu: A Multilevel Analysis of Context Effects in Performance Ratings

AbstractPurpose The purpose of this study was to take an inductive approach in examining the extent to which organizational contexts represent significant sources of variance in supervisor

EVIDENCE FOR THE VALIDITY OF 360 DIMENSIONS IN THE PRESENCE OF RATER-SOURCE FACTORS

Empirical research on the structure of 360-degree feedback ratings indicates that the source of the ratings (e.g., superiors, peers, subordinates) explains more variance than do the performance

Variance Components of Job Performance Ratings

Previous literature suggests that performance ratings are saturated with rater-related idiosyncratic variance. Given that modern psychometric theories relegate this source of variance to measurement

Explaining the Weak Relationship Between Job Performance and Ratings of Job Performance

  • K. Murphy
  • Psychology, Business
    Industrial and Organizational Psychology
  • 2008
Ratings of job performance are widely viewed as poor measures of job performance. Three models of the performance–performance rating relationship offer very different explanations and solutions for

Preventing Rater Biases in 360-Degree Feedback by Forcing Choice

We examined the effects of response biases on 360-degree feedback using a large sample (N = 4,675) of organizational appraisal data. Sixteen competencies were assessed by peers, bosses, and
...

References

SHOWING 1-10 OF 69 REFERENCES

TRAIT, RATER AND LEVEL EFFECTS IN 360‐DEGREE PERFORMANCE RATINGS

Method and trait effects in multitrait-multirater (MTMR) data were examined in a sample of 2,350 managers who participated in a developmental feedback program. Managers rated their own performance

RATER‐RATEE RACE EFFECTS IN DEVELOPMENTAL PERFORMANCE RATINGS OF MANAGERS

The effects of rater and ratee race on performance ratings of managers were examined. Ratings were obtained from peers, subordinates and bosses as part of a multirater, developmental feedback program

The rating of individuals in organizations: An alternate approach

Models of Job Performance Ratings: An Examination of Ratee Race, Ratee Gender, and Rater Level Effects

This research extends the work of Hunter (1983) and Borman, White, Pulakos, and Oppler (1991) on models of supervisory performance ratings. A rating model that included measures of cognitive ability,

Differential Dimensions Employed in Rating Subordinates, Peers, and Superiors

Abstract In this research, we examined whether people utilize different dimensions in their ratings of subordinates, peers, and superiors. Middle managers (N = 98) from various Israeli organizations

Test of a Latent Structure of Performance Ratings Derived from Wherry’s (1952) Theory of Rating

A model of the latent structure of performance ratings based on Wherry's (19.52) theory of rating was developed which sought to explain relationships between multiple ratings, each of multiple

Comparative analysis of the reliability of job performance ratings

This study used meta-analytic methods to compare the interrater and intrarater reliabilities of ratings of 10 dimensions of job performance used in the literature; ratings of overall job performance

A new look at within-source interrater reliability of 360-degree feedback ratings.

This study investigated within-source interrater reliability of supervisor, peer, and subordinate feedback ratings made for managerial development. Raters provided 360-degree feedback ratings on a

INTERPERSONAL AFFECT AND PERFORMANCE APPRAISAL: A FIELD STUDY

Interpersonal affect has been found, in prior laboratory research, to be related to ratings of job performance. Such findings have been taken to mean that affect creates bias in ratings. The present

Cognitive categorization processes in performance evaluation: Confirmatory tests of two models

We tested a data-driven ‘traditional’ model and a schema-driven ‘categorization’ model of rater cognitive processes. Raters viewed one of two videotaped lectures on economics. Next, they completed
...