Systematic Sources of Variance in Supervisory Job Performance Ratings: A Multilevel Analysis of Between-Rater and Between-Context Variance

contextual variance;rater variance;supervisory performance ratings;multilevel modeling;task performance;rating context;citizenship performance

Ellington, James Kemp ; Lori F. Thompson, Committee Member,Adam W. Meade, Committee Member,Donald W. Drewes, Committee Member,Mark A. Wilson, Committee Chair,Ellington, James Kemp ; Lori F. Thompson ; Committee Member ; Adam W. Meade ; Committee Member ; Donald W. Drewes ; Committee Member ; Mark A. Wilson ; Committee Chair

University:North Carolina State University

关键词: contextual variance;    rater variance;    supervisory performance ratings;    multilevel modeling;    task performance;    rating context;    citizenship performance;   

Others  :  https://repository.lib.ncsu.edu/bitstream/handle/1840.16/3134/etd.pdf?sequence=1&isAllowed=y

美国|英语

来源: null

【 摘 要 】

The appraisal of job performance is critical for both the practice of human resource management and organizational research.Furthermore, the most frequently used method for measuring performance is a supervisory rating.Given the prevalence of this method, it is crucial to understand the factors which influence rater behaviors.Recent research has indicated that a large portion of the variance in ratings is idiosyncratic to the rater (Scullen, Mount, & Goff, 2000).However, the nature of this idiosyncratic variance remains unclear.Previous models of appraisal have focused on either the cognitive processes involved, or more recently, the appraisal context.Although this recent focus on contextual issues has shown promise, the extent to which raters are influenced by the context in which they work is unknown.Therefore, the purpose of this research was to contribute to our understanding of supervisory ratings by incorporating a multilevel analytic approach in order to partition the variance between raters from the variance between contexts.This approach allowed for the investigation of several rater and context-level characteristics, in attempt to explain the variance associated with these two sources.More specifically, a conceptual model was proposed in order to examine rater-level variables including rater tendencies for leniency and halo, along with rater opportunities to observe performance.Contextual factors proposed for study included norms for leniency and halo, opportunities to observe performance at the context-level, and the nature of work/activity itself within various contexts.Moreover, this research incorporated a multidimensional performance criterion, in order to provide a more thorough investigation of the relationships of interest here.Results suggested that the rating context accounted for significant variance in both task and citizenship performance ratings.Furthermore, the rater tendency for leniency explained significant between-rater variation in both criteria.The rater tendency for halo was also significant, however this finding did not recur when analyzing a replication sample of data.At the context-level, the norm for leniency consistently predicted variance in citizenship performance, but was only a significant predictor of task performance in one sample.Finally, although these relationships were not consistent across samples, the nature of work/activity and the contextual norm for halo explained significant between-context variation in citizenship performance ratings.The interpretation and implications of these results are discussed, along with limitations of this research and suggestions for future research.