Understanding the latent structure of job performance ratings.

@article{Scullen2000UnderstandingTL,
  title={Understanding the latent structure of job performance ratings.},
  author={S. Scullen and M. Mount and M. Goff},
  journal={The Journal of applied psychology},
  year={2000},
  volume={85 6},
  pages={
          956-70
        }
}
This study quantified the effects of 5 factors postulated to influence performance ratings: the ratee's general level of performance, the ratee's performance on a specific dimension, the rater's idiosyncratic rating tendencies, the rater's organizational perspective, and random measurement error. Two large data sets, consisting of managers (n = 2,350 and n = 2,142) who received developmental ratings on 3 performance dimensions from 7 raters (2 bosses, 2 peers, 2 subordinates, and self) were… Expand

Figures, Tables, and Topics from this paper

A field study of the effects of rating purpose on the quality of multisource ratings.
Using a field sample of peers and subordinates, the current study employed generalizability theory to estimate sources of systematic variability associated with both developmental and administrativeExpand
The Use of Random Coefficient Modeling for Understanding and Predicting Job Performance Ratings
Earlier research using confirmatory factor analysis (CFA) suggests that most variance in job performance ratings is not attributable to ratee main effects. In this article, the authors point outExpand
Manifestation of Idiosyncratic Rater Effect in Employee Performance Appraisal
Performance appraisal is the bedrock of talent management and has received much attention from scholars and researchers alike in their pursuit to develop accurate, objective, and robust PerformanceExpand
RATER SOURCE EFFECTS ARE ALIVE AND WELL AFTER ALL
Recent research has questioned the importance of rater perspective effects on multisource performance ratings (MSPRs). Although making a valuable contribution, we hypothesize that this research hasExpand
Self-other agreement in job performance ratings: a meta-analytic test of a process model.
TLDR
A 3-stage model of the rating process is suggested and the empirical evidence for the relevance of each of these 3 stages is reviewed to an understanding of agreement in ratings. Expand
Is there a general factor in ratings of job performance? A meta-analytic framework for disentangling substantive and error influences.
TLDR
Construct-level correlations among rated dimensions of job performance were substantially inflated by halo for both supervisory (33%) and peer (63%) intrarater correlations, which have important implications for the measurement ofJob performance and for theories of jobperformance. Expand
The Performance Appraisal Milieu: A Multilevel Analysis of Context Effects in Performance Ratings
AbstractPurpose The purpose of this study was to take an inductive approach in examining the extent to which organizational contexts represent significant sources of variance in supervisorExpand
EVIDENCE FOR THE VALIDITY OF 360 DIMENSIONS IN THE PRESENCE OF RATER-SOURCE FACTORS
Empirical research on the structure of 360-degree feedback ratings indicates that the source of the ratings (e.g., superiors, peers, subordinates) explains more variance than do the performanceExpand
Variance Components of Job Performance Ratings
Previous literature suggests that performance ratings are saturated with rater-related idiosyncratic variance. Given that modern psychometric theories relegate this source of variance to measurementExpand
Explaining the Weak Relationship Between Job Performance and Ratings of Job Performance
  • K. Murphy
  • Psychology
  • Industrial and Organizational Psychology
  • 2008
Ratings of job performance are widely viewed as poor measures of job performance. Three models of the performance–performance rating relationship offer very different explanations and solutions forExpand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 80 REFERENCES
TRAIT, RATER AND LEVEL EFFECTS IN 360‐DEGREE PERFORMANCE RATINGS
Method and trait effects in multitrait-multirater (MTMR) data were examined in a sample of 2,350 managers who participated in a developmental feedback program. Managers rated their own performanceExpand
RATER‐RATEE RACE EFFECTS IN DEVELOPMENTAL PERFORMANCE RATINGS OF MANAGERS
The effects of rater and ratee race on performance ratings of managers were examined. Ratings were obtained from peers, subordinates and bosses as part of a multirater, developmental feedback programExpand
The rating of individuals in organizations: An alternate approach
Raters at different organizational levels probably observe significantly different facets of a ratee's job performance in most organizations. If so, their ratings ought to reflect these differences.Expand
Models of Job Performance Ratings: An Examination of Ratee Race, Ratee Gender, and Rater Level Effects
This research extends the work of Hunter (1983) and Borman, White, Pulakos, and Oppler (1991) on models of supervisory performance ratings. A rating model that included measures of cognitive ability,Expand
Differential Dimensions Employed in Rating Subordinates, Peers, and Superiors
Abstract In this research, we examined whether people utilize different dimensions in their ratings of subordinates, peers, and superiors. Middle managers (N = 98) from various Israeli organizationsExpand
Test of a Latent Structure of Performance Ratings Derived from Wherry’s (1952) Theory of Rating
A model of the latent structure of performance ratings based on Wherry's (19.52) theory of rating was developed which sought to explain relationships between multiple ratings, each of multipleExpand
Comparative analysis of the reliability of job performance ratings
This study used meta-analytic methods to compare the interrater and intrarater reliabilities of ratings of 10 dimensions of job performance used in the literature; ratings of overall job performanceExpand
A new look at within-source interrater reliability of 360-degree feedback ratings.
This study investigated within-source interrater reliability of supervisor, peer, and subordinate feedback ratings made for managerial development. Raters provided 360-degree feedback ratings on aExpand
INTERPERSONAL AFFECT AND PERFORMANCE APPRAISAL: A FIELD STUDY
Interpersonal affect has been found, in prior laboratory research, to be related to ratings of job performance. Such findings have been taken to mean that affect creates bias in ratings. The presentExpand
Cognitive categorization processes in performance evaluation: Confirmatory tests of two models
We tested a data-driven ‘traditional’ model and a schema-driven ‘categorization’ model of rater cognitive processes. Raters viewed one of two videotaped lectures on economics. Next, they completedExpand
...
1
2
3
4
5
...