Skip to main content
main-content
Top

Hint

Swipe to navigate through the articles of this issue

Published in: Journal of Business and Psychology 4/2021

13-06-2020 | Original Paper

The Effects of Team Context on Peer Ratings of Task and Citizenship Performance

Authors: Joseph A. Schmidt, Thomas A. O’Neill, Patrick D. Dunlop

Published in: Journal of Business and Psychology | Issue 4/2021

Login to get access
share
SHARE

Abstract

Recent trends indicate that organizations will continue their strategic pursuit of teamwork for the foreseeable future, which will create a need for accurate assessments of individuals’ performance in teams. Although individual behaviors can be perceived and assessed by fellow team members (i.e., peers), the extent to which the team shapes perceivers’ judgments versus the target’s behavior is unclear. We conducted two studies to understand how and why team context influences peer ratings of individual performance. In study 1, we conducted cross-classified modeling on a sample of 7160 performance observations of 568 targets made by 567 perceivers, who were each members of four separate teams. Results indicated that team membership accounted for a substantially higher proportion of perceiver, relative to target, variance. In study 2, we conducted social relations modeling with a sample of 679 performance observations collected from 217 individuals nested in 46 teams to test the effects of psychological safety on perceiver, target, and team variance components. Perceptions of psychological safety accounted for proportionally larger perceiver, relative to target, variance in OCB, and task performance ratings. Altogether, team context appears to affect perceivers’ judgments of behavior more than the target’s behavior itself, implying that peer ratings sourced from different teams may not be comparable. We consider the implications for the collection and interpretation of peer performance ratings in teams and the potential implications for social cognitive theory, such that certain aspects of the team context, including psychological safety, may act as a cognitive heuristic by molding perceiver judgments of targets.
Footnotes
1
A portion of the data in study 1 appears in a published paper (O’Neill et al. 2019). That paper is intended for educators and is focused on how to apply peer ratings in the classroom. As such, it only reports the means, standard deviations, inter-rater reliability, and correlations for each item and the aggregated scale score of the performance rating measure.
 
2
This differs from the usual application of dummy variables, where the model would contain one fewer (k − 1) dummy variables than there are teams. In those types of models, a dummy variable slope is interpreted as the difference in the y-variable between the team represented by the dummy and the reference category, or the team that was excluded from the model.
 
Literature
go back to reference Aguinis, H. (2013). Performance management (3rd ed.). Essex: Pearson. Aguinis, H. (2013). Performance management (3rd ed.). Essex: Pearson.
go back to reference Allen, N. J., & O’Neill, T. A. (2015). The trajectory of emergence of shared group-level constructs. Small Group Research, 46, 352–390. Allen, N. J., & O’Neill, T. A. (2015). The trajectory of emergence of shared group-level constructs. Small Group Research, 46, 352–390.
go back to reference Bamberger, P. A. (2007). Competitive appraising: A social dilemma perspective on the conditions in which multi-round peer evaluation may result in counter-productive team dynamics. Human Resource Management Review, 17, 1–18. Bamberger, P. A. (2007). Competitive appraising: A social dilemma perspective on the conditions in which multi-round peer evaluation may result in counter-productive team dynamics. Human Resource Management Review, 17, 1–18.
go back to reference Behfar, K. J., Friedman, R., & Oh, S. H. (2016). Impact of team (dis) satisfaction and psychological safety on performance evaluation biases. Small Group Research, 47, 77–107. Behfar, K. J., Friedman, R., & Oh, S. H. (2016). Impact of team (dis) satisfaction and psychological safety on performance evaluation biases. Small Group Research, 47, 77–107.
go back to reference Bonito, J. A., & Kenny, D. A. (2010). The measurement of reliability of social relations components from round-robin designs. Personal Relationships, 17, 235–251. Bonito, J. A., & Kenny, D. A. (2010). The measurement of reliability of social relations components from round-robin designs. Personal Relationships, 17, 235–251.
go back to reference Christensen, P. N., & Kashy, D. A. (2012). Using the social relations model to understand interpersonal perception and behavior. In H. Cooper (Ed.), APA handbook of research methods in psychology (Vol. 3, pp. 425–437). Washington, DC: American Psychological Association. Christensen, P. N., & Kashy, D. A. (2012). Using the social relations model to understand interpersonal perception and behavior. In H. Cooper (Ed.), APA handbook of research methods in psychology (Vol. 3, pp. 425–437). Washington, DC: American Psychological Association.
go back to reference Dierdorff, E. C., & Morgeson, F. P. (2009). Effects of descriptor specificity and observability on incumbent work analysis ratings. Personnel Psychology, 62, 601–628. Dierdorff, E. C., & Morgeson, F. P. (2009). Effects of descriptor specificity and observability on incumbent work analysis ratings. Personnel Psychology, 62, 601–628.
go back to reference Dierdorff, E. C., & Surface, E. A. (2007). Placing peer ratings in context: Systematic influences beyond ratee performance. Personnel Psychology, 60, 93–126. Dierdorff, E. C., & Surface, E. A. (2007). Placing peer ratings in context: Systematic influences beyond ratee performance. Personnel Psychology, 60, 93–126.
go back to reference Dominick, P. G., Reilly, R. R., & McGourty, J. W. (1997). The effects of peer feedback on team member behavior. Group & Organization Management, 22, 508–520. Dominick, P. G., Reilly, R. R., & McGourty, J. W. (1997). The effects of peer feedback on team member behavior. Group & Organization Management, 22, 508–520.
go back to reference Edmondson, A. (1999). Psychological safety and learning behavior in work teams. Administrative Science Quarterly, 44, 350–383. Edmondson, A. (1999). Psychological safety and learning behavior in work teams. Administrative Science Quarterly, 44, 350–383.
go back to reference Ellington, J. K., & Wilson, M. A. (2017). The performance appraisal milieu: A multilevel analysis of context effects in performance ratings. Journal of Business & Psychology, 32, 87–100. Ellington, J. K., & Wilson, M. A. (2017). The performance appraisal milieu: A multilevel analysis of context effects in performance ratings. Journal of Business & Psychology, 32, 87–100.
go back to reference Erez, A., Schilpzand, P., Leavitt, K., Woolum, A. H., & Judge, T. A. (2015). Inherently relational: Interactions between peers’ and individuals’ personalities impact reward giving and appraisal of individual performance. Academy of Management Journal, 58, 1761–1784. Erez, A., Schilpzand, P., Leavitt, K., Woolum, A. H., & Judge, T. A. (2015). Inherently relational: Interactions between peers’ and individuals’ personalities impact reward giving and appraisal of individual performance. Academy of Management Journal, 58, 1761–1784.
go back to reference Favero, J. L., & Ilgen, D. R. (1989). The effects of ratee prototypicality on rater observation and accuracy. Journal of Applied Social Psychology, 19, 932–946. Favero, J. L., & Ilgen, D. R. (1989). The effects of ratee prototypicality on rater observation and accuracy. Journal of Applied Social Psychology, 19, 932–946.
go back to reference Frazier, M. L., Fainshmidt, S., Klinger, R. L., Pezeshkan, A., & Vracheva, V. (2017). Psychological safety: A meta-analytic review and extension. Personnel Psychology, 70, 113–165. Frazier, M. L., Fainshmidt, S., Klinger, R. L., Pezeshkan, A., & Vracheva, V. (2017). Psychological safety: A meta-analytic review and extension. Personnel Psychology, 70, 113–165.
go back to reference Furr, R. M., & Bacharach, V. R. (2014). Psychometrics: An introduction (2nd ed.). Thousand Oaks, CA: Sage. Furr, R. M., & Bacharach, V. R. (2014). Psychometrics: An introduction (2nd ed.). Thousand Oaks, CA: Sage.
go back to reference Geldhof, G. J., Preacher, K. J., & Zyphur, M. J. (2014). Reliability estimation in a multilevel confirmatory factor analysis framework. Psychological Methods, 19, 72–91. PubMed Geldhof, G. J., Preacher, K. J., & Zyphur, M. J. (2014). Reliability estimation in a multilevel confirmatory factor analysis framework. Psychological Methods, 19, 72–91. PubMed
go back to reference Goffin, R. D., Jelley, R. B., Powell, D. M., & Johnston, N. G. (2009). Taking advantage of social comparisons in performance appraisal: The relative percentile method. Human Resource Management, 48, 251–268. Goffin, R. D., Jelley, R. B., Powell, D. M., & Johnston, N. G. (2009). Taking advantage of social comparisons in performance appraisal: The relative percentile method. Human Resource Management, 48, 251–268.
go back to reference Greguras, G. J., Robie, C., & Born, M. P. (2001). Applying the social relations model to self and peer evaluations. Journal of Management Development, 20, 508–525. Greguras, G. J., Robie, C., & Born, M. P. (2001). Applying the social relations model to self and peer evaluations. Journal of Management Development, 20, 508–525.
go back to reference Greguras, G. J., Robie, C., Schleicher, D. J., & Maynard, G. (2003). A field study of the effects of rating purpose on the quality of multisource ratings. Personnel Psychology, 56, 1–21. Greguras, G. J., Robie, C., Schleicher, D. J., & Maynard, G. (2003). A field study of the effects of rating purpose on the quality of multisource ratings. Personnel Psychology, 56, 1–21.
go back to reference Griffen, M. A., Neal, A., & Parker, S. K. (2007). A new model of work role performance: Positive behavior in uncertain and interdependent contexts. Academy of Management Journal, 50, 327–347. Griffen, M. A., Neal, A., & Parker, S. K. (2007). A new model of work role performance: Positive behavior in uncertain and interdependent contexts. Academy of Management Journal, 50, 327–347.
go back to reference Halbesleben, J. R. B., Bowler, W. M., Bolino, M. C., & Turnley, W. H. (2010). Organizational concern, prosocial values, or impression management? How supervisors attribute motives to organizational citizenship behavior. Journal of Applied Social Psychology, 40, 1450–1489. Halbesleben, J. R. B., Bowler, W. M., Bolino, M. C., & Turnley, W. H. (2010). Organizational concern, prosocial values, or impression management? How supervisors attribute motives to organizational citizenship behavior. Journal of Applied Social Psychology, 40, 1450–1489.
go back to reference Hilbert, M. (2012). Toward a synthesis of cognitive biases: How noisy information processing can bias human decision making. Psychological Bulletin, 138, 211–237. PubMed Hilbert, M. (2012). Toward a synthesis of cognitive biases: How noisy information processing can bias human decision making. Psychological Bulletin, 138, 211–237. PubMed
go back to reference Hoffman, B., Lance, C. E., Bynum, B., & Gentry, W. A. (2010). Rater source effects are alive and well after all. Personnel Psychology, 63, 119–151. Hoffman, B., Lance, C. E., Bynum, B., & Gentry, W. A. (2010). Rater source effects are alive and well after all. Personnel Psychology, 63, 119–151.
go back to reference Ilgen, D. R., Barnes-Farrell, J. L., & McKellin, D. B. (1993). Performance appraisal process research in the 1980s: What has it contributed to appraisals in use? Organizational Behavior and Human Decision Processes, 54, 321–368. Ilgen, D. R., Barnes-Farrell, J. L., & McKellin, D. B. (1993). Performance appraisal process research in the 1980s: What has it contributed to appraisals in use? Organizational Behavior and Human Decision Processes, 54, 321–368.
go back to reference Jawahar, I. M., & Williams, C. R. (1997). Where all the children are above average: The performance appraisal purpose effect. Personnel Psychology, 50, 905–925. Jawahar, I. M., & Williams, C. R. (1997). Where all the children are above average: The performance appraisal purpose effect. Personnel Psychology, 50, 905–925.
go back to reference Keeler, K. R., Kong, W., Dalal, R. S., & Cortina, J. M. (2019). Situational strength interactions: Are variance patterns consistent with the theory? Journal of Applied Psychology, 104, 1487–1513. Keeler, K. R., Kong, W., Dalal, R. S., & Cortina, J. M. (2019). Situational strength interactions: Are variance patterns consistent with the theory? Journal of Applied Psychology, 104, 1487–1513.
go back to reference Kenny, D. A., & Albright, L. (1987). Accuracy in interpersonal perception: A social relations analysis. Psychological Bulletin, 102, 390–402. PubMed Kenny, D. A., & Albright, L. (1987). Accuracy in interpersonal perception: A social relations analysis. Psychological Bulletin, 102, 390–402. PubMed
go back to reference Kenny, D. A., Kashy, D. A., & Cook, W. L. (2006). Dyadic data analysis. New York, NY: Guilford. Kenny, D. A., Kashy, D. A., & Cook, W. L. (2006). Dyadic data analysis. New York, NY: Guilford.
go back to reference Kuncel, N. R., Klieger, D. M., Connelly, B. S., & Ones, D. S. (2013). Mechanical versus clinical data combination in selection and admissions decisions: A meta-analysis. Journal of Applied Psychology, 98, 1060–1072. Kuncel, N. R., Klieger, D. M., Connelly, B. S., & Ones, D. S. (2013). Mechanical versus clinical data combination in selection and admissions decisions: A meta-analysis. Journal of Applied Psychology, 98, 1060–1072.
go back to reference Lee, K., & Allen, N. J. (2002). Organizational citizenship behavior and workplace deviance: The role of affect and cognitions. Journal of Applied Psychology, 87, 131–142. Lee, K., & Allen, N. J. (2002). Organizational citizenship behavior and workplace deviance: The role of affect and cognitions. Journal of Applied Psychology, 87, 131–142.
go back to reference LePine, J. A., Piccolo, R. F., Jackson, C. L., Mathieu, J. E., & Saul, J. R. (2008). A meta-analysis of teamwork process: Towards a better understanding of the dimensional structure and relationships with team effectiveness criteria. Personnel Psychology, 61, 273–307. LePine, J. A., Piccolo, R. F., Jackson, C. L., Mathieu, J. E., & Saul, J. R. (2008). A meta-analysis of teamwork process: Towards a better understanding of the dimensional structure and relationships with team effectiveness criteria. Personnel Psychology, 61, 273–307.
go back to reference Levy, P. E., & Williams, J. R. (2004). The social context of performance appraisal: A review and framework for the future. Journal of Management, 30, 881–905. Levy, P. E., & Williams, J. R. (2004). The social context of performance appraisal: A review and framework for the future. Journal of Management, 30, 881–905.
go back to reference Loignon, A. C., Woehr, D. J., Thomas, J. S., Loughry, M. L., Ohland, M. W., & Ferguson, D. M. (2017). Facilitating peer evaluation in team contexts: The impact of frame-of-reference rater training. Academy of Management Learning & Education, 16, 562–578. Loignon, A. C., Woehr, D. J., Thomas, J. S., Loughry, M. L., Ohland, M. W., & Ferguson, D. M. (2017). Facilitating peer evaluation in team contexts: The impact of frame-of-reference rater training. Academy of Management Learning & Education, 16, 562–578.
go back to reference Loughry, M. L., Ohland, M. W., & Moore, D. D. (2007). Development of a theory-based assessment of team member effectiveness. Educational and Psychological Measurement, 67, 505–524. Loughry, M. L., Ohland, M. W., & Moore, D. D. (2007). Development of a theory-based assessment of team member effectiveness. Educational and Psychological Measurement, 67, 505–524.
go back to reference Luan, S., & Reb, J. (2017). Fast-and-frugal trees as noncompensatory models of performance-based personnel decisions. Organizational Behavior and Human Decision Processes, 141, 29–42. Luan, S., & Reb, J. (2017). Fast-and-frugal trees as noncompensatory models of performance-based personnel decisions. Organizational Behavior and Human Decision Processes, 141, 29–42.
go back to reference Meyer, R. D., Dalal, R. S., & Hermida, R. (2010). A review and synthesis of situational strength in the organizational sciences. Journal of Management, 36, 121–140. Meyer, R. D., Dalal, R. S., & Hermida, R. (2010). A review and synthesis of situational strength in the organizational sciences. Journal of Management, 36, 121–140.
go back to reference Mischel, W. (1973). Toward a cognitive social learning reconceptualization of personality. Psychological Review, 80, 252–283. PubMed Mischel, W. (1973). Toward a cognitive social learning reconceptualization of personality. Psychological Review, 80, 252–283. PubMed
go back to reference Murphy, K. R., & Cleveland, J. N. (1995). Understanding performance appraisal: Social, organizational, and goal-based perspectives. Thousand Oaks, CA: Sage. Murphy, K. R., & Cleveland, J. N. (1995). Understanding performance appraisal: Social, organizational, and goal-based perspectives. Thousand Oaks, CA: Sage.
go back to reference Murphy, K. R., Cleveland, J. N., Kinney, T. B., Skattebo, A. L., Newman, D. A., & Sin, H. P. (2003). Unit climate, rater goals and performance ratings in an instructional setting. Irish Journal of Management, 24, 48–65. Murphy, K. R., Cleveland, J. N., Kinney, T. B., Skattebo, A. L., Newman, D. A., & Sin, H. P. (2003). Unit climate, rater goals and performance ratings in an instructional setting. Irish Journal of Management, 24, 48–65.
go back to reference O’Neill, T. A., Deacon, A., Gibbard, K., Larson, N., Hoffart, G., Smith, J., & Donia, M. (2018). Team dynamics feedback for post-secondary student learning teams. Assessment and Evaluation in Higher Education, 43, 571–585. O’Neill, T. A., Deacon, A., Gibbard, K., Larson, N., Hoffart, G., Smith, J., & Donia, M. (2018). Team dynamics feedback for post-secondary student learning teams. Assessment and Evaluation in Higher Education, 43, 571–585.
go back to reference O’Neill, T. A., Goffin, R. D., & Gellatly, I. R. (2012). The use of random coefficient modeling for understanding and predicting job performance ratings. Organizational Research Methods, 15, 436–462. O’Neill, T. A., Goffin, R. D., & Gellatly, I. R. (2012). The use of random coefficient modeling for understanding and predicting job performance ratings. Organizational Research Methods, 15, 436–462.
go back to reference O’Neill, T. A., Larson, N., Smith, J., Deng, C., Donia, M., Rosehart, W., & Brennan, R. (2019). Introducing a scalable peer feedback system for learning teams. Assessment and Evaluation in Higher Education, 44, 848–862. O’Neill, T. A., Larson, N., Smith, J., Deng, C., Donia, M., Rosehart, W., & Brennan, R. (2019). Introducing a scalable peer feedback system for learning teams. Assessment and Evaluation in Higher Education, 44, 848–862.
go back to reference O’Neill, T. A., McLarnon, M. J. W., & Carswell, J. J. (2015). Variance components of job performance ratings. Human Performance, 28, 66–91. O’Neill, T. A., McLarnon, M. J. W., & Carswell, J. J. (2015). Variance components of job performance ratings. Human Performance, 28, 66–91.
go back to reference Oh, S. H. D., Chen, Y., & Sun, F. (2015). When is a good citizen valued more? Organizational citizenship behavior and performance evaluation. Social Behavior and Personality: An International Journal, 43, 1009–1020. Oh, S. H. D., Chen, Y., & Sun, F. (2015). When is a good citizen valued more? Organizational citizenship behavior and performance evaluation. Social Behavior and Personality: An International Journal, 43, 1009–1020.
go back to reference Ohland, M. W., Loughry, M. L., Woehr, D. J., Bullard, L. G., Felder, R. M., Finelli, C. J., Layton, R. A., Pomeranz, H. R., & Schmucker, D. G. (2012). The comprehensive assessment of team member effectiveness: Development of a behaviorally anchored rating scale for self-and peer evaluation. Academy of Management Learning & Education, 11, 609–630. Ohland, M. W., Loughry, M. L., Woehr, D. J., Bullard, L. G., Felder, R. M., Finelli, C. J., Layton, R. A., Pomeranz, H. R., & Schmucker, D. G. (2012). The comprehensive assessment of team member effectiveness: Development of a behaviorally anchored rating scale for self-and peer evaluation. Academy of Management Learning & Education, 11, 609–630.
go back to reference Putka, D. J., & Hoffman, B. J. (2013). Clarifying the contribution of assessee-, dimension-, exercise-, and assessor-related effects to reliable and unreliable variance in assessment center ratings. Journal of Applied Psychology, 98, 114–133. Putka, D. J., & Hoffman, B. J. (2013). Clarifying the contribution of assessee-, dimension-, exercise-, and assessor-related effects to reliable and unreliable variance in assessment center ratings. Journal of Applied Psychology, 98, 114–133.
go back to reference Savitsky, K., Van Boven, L., Epley, N., & Wight, W. M. (2005). The unpacking effect in allocations of responsibility for group tasks. Journal of Experimental Social Psychology, 41, 447–457. Savitsky, K., Van Boven, L., Epley, N., & Wight, W. M. (2005). The unpacking effect in allocations of responsibility for group tasks. Journal of Experimental Social Psychology, 41, 447–457.
go back to reference Scullen, S. E., Mount, M. K., & Goff, M. (2000). Understanding the latent structure of job performance ratings. Journal of Applied Psychology, 85, 956–970. Scullen, S. E., Mount, M. K., & Goff, M. (2000). Understanding the latent structure of job performance ratings. Journal of Applied Psychology, 85, 956–970.
go back to reference Snijders, T. A. B., & Kenny, D. A. (1999). The social relations model for family data: A multilevel approach. Personal Relationships, 6, 471–486. Snijders, T. A. B., & Kenny, D. A. (1999). The social relations model for family data: A multilevel approach. Personal Relationships, 6, 471–486.
go back to reference Srull, T. K., & Wyer, R. S. (1989). Person memory and judgment. Psychological Review, 96, 58–83. PubMed Srull, T. K., & Wyer, R. S. (1989). Person memory and judgment. Psychological Review, 96, 58–83. PubMed
go back to reference Waldman, D. A., Yammarino, F. J., & Avolio, B. J. (1990). A multiple level investigation of personnel ratings. Personnel Psychology, 43, 811–835. Waldman, D. A., Yammarino, F. J., & Avolio, B. J. (1990). A multiple level investigation of personnel ratings. Personnel Psychology, 43, 811–835.
go back to reference Wherry Sr., R. J., & Bartlett, C. J. (1982). The control of bias in ratings: A theory of rating. Personnel Psychology, 35(3), 521-551. Wherry Sr., R. J., & Bartlett, C. J. (1982). The control of bias in ratings: A theory of rating. Personnel Psychology, 35(3), 521-551.
Metadata
Title
The Effects of Team Context on Peer Ratings of Task and Citizenship Performance
Authors
Joseph A. Schmidt
Thomas A. O’Neill
Patrick D. Dunlop
Publication date
13-06-2020
Publisher
Springer US
Published in
Journal of Business and Psychology / Issue 4/2021
Print ISSN: 0889-3268
Electronic ISSN: 1573-353X
DOI
https://doi.org/10.1007/s10869-020-09701-8

Other articles of this Issue 4/2021

Journal of Business and Psychology 4/2021 Go to the issue

Premium Partner