Can training improve the quality of inferences made by raters in competency modeling? A quasi-experiment.
@article{Lievens2007CanTI, title={Can training improve the quality of inferences made by raters in competency modeling? A quasi-experiment.}, author={Filip Lievens and Juan I. Sanchez}, journal={The Journal of applied psychology}, year={2007}, volume={92 3}, pages={ 812-9 } }
A quasi-experiment was conducted to investigate the effects of frame-of-reference training on the quality of competency modeling ratings made by consultants. Human resources consultants from a large consulting firm were randomly assigned to either a training or a control condition. The discriminant validity, interrater reliability, and accuracy of the competency ratings were significantly higher in the training group than in the control group. Further, the discriminant validity and interrater…
59 Citations
Rater Training: Understanding Effects of Training Content, Practice Ratings, and Feedback
- Psychology
- 2017
Despite the popularity of frame‐of‐reference training (FORT), it is not clear how different structural elements of FORT work in concert to improve rating accuracy. Furthermore, past rater…
The Impact of Training on the Accuracy of Teacher-Completed Direct Behavior Ratings (DBRs)
- Psychology, Education
- 2010
The purpose of this study was to investigate the effects of three levels of training (direct, indirect, and none) on teachers’ ability to accurately rate video of student behavior. Direct and…
Lack of consensus among competency ratings of the same occupation: noise or substance?
- Psychology, MedicineThe Journal of applied psychology
- 2010
Drawing from job crafting, creativity, and role theories to uncover situational factors related to differences among competency ratings of the same occupation showed that 25% of the variance associated with differences in competency Rating was related to the complexity, the context, and primarily the nature of the occupation's work activities.
Sources of Variance Underlying Competency Ratings 2
- Psychology
Although rating differences among incumbents of the same occupation have traditionally been viewed as error variance in the work analysis domain, such differences might often capture substantive…
A Cognitively-Based Competency Model for Small Unit Counter-IED Performance
- Computer Science
- 2011
A cognitively-based competency model developed to understand the types of tasks performed at the small unit level for counter-IED operations for the purpose of developing assessment items is described.
Does effectiveness in performance appraisal improve with rater training?
- BusinessPloS one
- 2019
The main results show that training based on performance dimensions and the creation of a common framework, in addition to the training that includes the four programmes, increases the level of knowledge of performance and its dimensions.
A Critical Review of the Science and Practice of Competency Modeling
- Business
- 2013
As an alternative to traditional job analysis, the practice of competency modeling may be appealing to scholars and practitioners of human resource development (HRD) to serve as the foundation for…
EFFECTS OF DESCRIPTOR SPECIFICITY AND OBSERVABILITY ON INCUMBENT WORK ANALYSIS RATINGS
- Business
- 2009
Judgments regarding the requirements of jobs and the requirements of individuals performing those jobs comprise the critical groundwork on which human resource practices are built. Yet, such…
Rater training revisited: An updated meta-analytic review of frame-of-reference training
- Psychology
- 2012
The present study updates Woehr and Huffcutt's (1994) rater training meta-analysis and demonstrates that frame-of-reference (FOR) training is an effective method of improving rating accuracy. The…
Disappointing Interventions and Weak Criteria: Carving Out a Solution Is Still Possible
- BusinessIndustrial and Organizational Psychology
- 2016
In their focal article, Adler and his colleagues (2016) elaborate on the pros and cons of abolishing the performance appraisal process in organizations. Sherman-Garr (2014) contends that this trend…
References
SHOWING 1-10 OF 38 REFERENCES
The impact of raters' cognition on judgment accuracy: An extension to the job analysis domain
- Psychology
- 1994
Eighty-six incumbents of three different jobs produced job-analytic ratings using either a decomposed (task-based) or a holistic (job-based) rating strategy. Approximately half of them received rater…
Rater training for performance appraisal: A quantitative review
- Psychology
- 1994
A substantial amount of research in the performance appraisal literature has focused on rater training as a means of improving performance ratings. Unfortunately the value of this research is…
Strategies in Rater Training
- Psychology
- 1981
There is no evidence to support the belief that training raters to change rating distributions will increase accuracy or validity. Such training may merely promote a temporary and situation-specific…
THE PRACTICE OF COMPETENCY MODELING
- Education
- 2000
The purpose of this article is to define and explain a trend that has caused a great deal of confusion among HR researchers, practitioners, and consumers of HR-related services: competency modeling.…
A new look at within-source interrater reliability of 360-degree feedback ratings.
- Psychology, Business
- 1998
This study investigated within-source interrater reliability of supervisor, peer, and subordinate feedback ratings made for managerial development. Raters provided 360-degree feedback ratings on a…
Effects of training and information on the accuracy and reliability of job evaluations.
- Psychology
- 1988
Subjects evaluated 23 benchmark jobs on 10 dimensions after either receiving training or no training in job evaluation and after being given only a title, only a job description, or both a title and…
Understanding frame-of-reference training: the impact of training on the recall of performance information
- Psychology
- 1994
Research on the cognitive mechanisms underlying frame-of-reference (FOR) rater training is extended by examining the impact of FOR training on the recall of performance information. It was…
Self-presentation processes in job analysis: a field experiment investigating inflation in abilities, tasks, and competencies.
- PsychologyThe Journal of applied psychology
- 2004
Results indicated that ability statements were more subject to inflation than were task statements across all rating scales, and frequency and importance ratings of global competency statements were generally higher than decomposed ability and task scales, but required-at-entry judgments demonstrated the opposite relationship.
Further investigation of common knowledge effects on job analysis ratings.
- Psychology
- 1987
Previous research (Smith & Hakel, 1979) raised the possibility that the Position Analysis Questionnaire (PAQ) only captures common knowledge, or stereotypes, about jobs, Cornelius, DeNisi, and…
Effects of frame-of-reference training and information configuration on memory organization and rating accuracy.
- Psychology
- 1995
It was hypothesized that encoding conditions would substitute for, or neutralize, the effects of frame-of-reference (FOR) training on rating accuracy by encouraging or impeding the person…