nach oben

Journal of Business and Psychology

Erschienen in:

01.12.2010

BARS and Those Mysterious, Missing Middle Anchors

verfasst von: Neil M. A. Hauenstein, Reagan D. Brown, Andrea L. Sinclair

Erschienen in: Journal of Business and Psychology | Ausgabe 4/2010

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Purpose

A contributing reason for the common problem of missing middle anchors on behaviorally anchored rating scales (BARS) is the standard deviation (SD) criterion used in scaling phase. An alternative BARS scaling process is proposed based on the a _wg(1) index of interrater agreement.

Design/Methodology/Approach

Algebraic principles are used to explicate that the SD criterion is analogous to using a r _wg(1) interrater agreement statistic with the assumption of a uniform null distribution, and this reliance on r _wg(1) decreases the likelihood of anchoring behaviors in the mid-range of the rating scale. Archival data from a law enforcement agency were used to compare the success and failure of anchoring a BARS using a SD criterion versus an a _wg(1) criterion.

Findings

The a _wg(1) criterion was successful at anchoring the full range of the rating scale, but only if the cut-off for anchoring behaviors required a “weak” level of interrater agreement.

Implications

The most surprising finding was that the traditional 1.5 SD criterion on a 9-point rating scale is not a particularly stringent agreement requirement for anchoring behaviors. Although we demonstrated the advantages of using a _wg(1) to anchor BARS, an equally important conclusion is that incumbents need to be better trained prior to scaling behaviors.

Originality/Value

We provide a theoretically defensible approach for anchoring BARS that ameliorates the missing middle anchor problem. Further, the utility of a _wg(1) in the BARS context is yet another example of the limitation of r _wg(1) when assuming a uniform null distribution.

Vorheriger Artikel The Impact of Organizational Culture on Attraction and Recruitment of Job Applicants

Nächster Artikel Perceptions of Human Capital Measures: From Corporate Executives and Investors

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

It is interesting that neither the 1.25 SD criterion with a 7-point scale nor the 1.0 SD criterion with a 5-point scale are comparable to the more stringent 1.5 SD criterion using a 9-point scale. The 1.0 SD with a 5-point scale is the most lenient combination for anchoring behaviors, and the 1.25 SD criterion with a 7-point scale is between the other two combinations in terms of the probability of anchoring behaviors. To achieve comparability with the 1.5 SD criterion using a 9-point scale, a SD criterion of 1.125 is needed for the 7-point scale, and a SD criterion of .75 is needed for the 5-point scale.

Barnes-Farrell, J. L., & Lynch, A. M. (2003). Performance appraisal and feedback programs. In J. E. Edwards, J. C. Scott, & N. S. Raju (Eds.), The Human Resources Program-Evaluation Handbook (pp. 155–176). Thousand Oaks, CA: Sage Publications.

Bernardin, H. J., & Smith, P. C. (1981). A clarification of some issues regarding the development and use of behaviorally anchored ratings scales (BARS). Journal of Applied Psychology, 66, 458–463.CrossRef

Bownas, D. A., & Bernardin, H. J. (1988). Critical incident technique. In S. Gael (Ed.), The job analysis handbook for business, industry, and government (pp. 1120–1137). New York, NY: Wiley.

Brown, R. D., & Hauenstein, N. M. A. (2005). Interrater agreement reconsidered: An alternative to the r_wg indices. Organizational Research Methods, 8, 1–20.CrossRef

Burke, M. J., & Dunlop, W. P. (2002). Estimating interrater agreement with the average deviation index: A user’s guide. Organizational Research Methods, 5, 159–172.CrossRef

Burke, M. J., Finkelstein, L. M., & Dusig, M. S. (1999). On average deviation indices for estimating interrater agreement. Organizational Research Methods, 2, 49–68.CrossRef

Cardy, R. L. (1998). Performance appraisal in a quality context: A new look at an old problem. In J. Smither (Ed.), Performance appraisal: State of the art in practice (pp. 132–162). Josey-Bass: San Francisco.

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37–46.CrossRef

Conte, J. M., Landy, F. J., & Mathieu, J. E. (1995). Time urgency: Conceptual and construct development. Journal of Applied Psychology, 80, 178–185.CrossRef

DeNisi, A. S. (1996). Cognitive approach to performance appraisal: A program of research. New York: Routledge.

Engelbrecht, A. S., & Fischer, H. (1995). The managerial performance implications of a developmental assessment center process. Human Relations, 48, 387–404.CrossRef

Finn, R. H. (1970). A note on estimating the reliability of categorical data. Educational and Psychological Measurement, 30, 71–76.CrossRef

George, J. M. (1990). Personality, affect and behavior in groups. Journal of Applied Psychology, 75, 107–116.CrossRef

Grote, D. (1996). The complete guide to performance appraisal. New York: American Management Association.

Harari, O., & Zedeck, S. (1973). Development of behaviorally anchored scales for the evaluation of faculty teaching. Journal of Applied Psychology, 58, 261–265.CrossRef

Hauenstein, N. M. A., & Foti, R. J. (1989). From laboratory to practice: Neglected issues in implementing frame-of-reference rater training. Personnel Psychology, 42, 359–378.CrossRef

James, L. R., Demaree, R. G., & Wolf, G. (1984). Estimating within-group interrater reliability with and without response bias. Journal of Applied Psychology, 69, 85–98.CrossRef

Judge, T. A., & Bono, J. E. (2000). Five factor model of personality and transformational leadership. Journal of Applied Psychology, 85, 751–765.CrossRefPubMed

Kozlowski, S. W., & Hattrup, K. (1992). A disagreement about within-group agreement: Disentangling issues of consistency versus consensus. Journal of Applied Psychology, 77, 161–167.CrossRef

Landy, F. J., & Guion, R. M. (1970). Development of scales for the measurement of work motivation. Organizational Behavior and Human Decision Processes, 5, 93–103.

Latham, G. P., Saari, L. M., Pursell, E. D., & Campion, M. A. (1980). The situational interview. Journal of Applied Psychology, 65, 422–427.CrossRef

London, M., Mone, E. M., & Scott, J. C. (2004). Performance management and assessment: Methods for improved rater accuracy and employee goal setting. Human Resource Management, 43, 319–336.CrossRef

Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.

Malos, S. B. (1998). Current legal issues in performance appraisal. In J. Smither (Ed.), Performance appraisal: State of the art in practice (pp. 49–94). San Francisco: Josey-Bass.

Martin, D. D., & Bartol, K. M. (1998). Performance appraisal: Maintaining system effectiveness. Public Personnel Management, 27, 223–230.

Maurer, S. D. (2002). A practitioner-based analysis of interviewer job expertise and scale format as contextual factors in situational interviews. Personnel Psychology, 55, 307–328.CrossRef

Murphy, K. R., & Cleveland, J. N. (1995). Understanding performance appraisal: Social, organizational, and goal-based perspectives. Thousand Oaks, CA: Sage Publications.

Murphy, K. R., & De Shon, R. (2000a). Interrater correlations do not estimate the reliability of job performance ratings. Personnel Psychology, 53, 873–900.CrossRef

Murphy, K. R., & De Shon, R. (2000b). Progress in psychometrics: Can industrial and organizational psychology catch up? Personnel Psychology, 53, 913–924.CrossRef

Ramus, C. A., & Steger, U. (2000). The roles of supervisory support behaviors and environmental policy in employee “ecoinitiatives” at leading-edge European companies. Academy of Management Journal, 43, 605–626.CrossRef

Schleicher, D. J., Day, D. V., Mayes, B. T., & Riggio, R. E. (2002). A new frame for frame-of-reference training: Enhancing the construct validity of assessment centers. Journal of Applied Psychology, 87, 735–746.CrossRefPubMed

Schmidt, F. L., Viswesvaran, C., & Ones, D. S. (2000). Reliability is not validity and validity is not reliability. Personnel Psychology, 53, 901–912.CrossRef

Smith, P. C., & Kendall, L. M. (1963). Retranslation of expectations: An approach to the construction of unambiguous anchors for rating scales. Journal of Applied Psychology, 47, 149–255.CrossRef

Tziner, A., Joanis, C., & Murphy, K. (2000). A comparison of three methods of performance appraisal with regard to goal properties, goal perception, and ratee satisfaction. Group & Organization Management, 25, 175–190.CrossRef

Woods, R. H., Sciarini, M., & Breiter, D. (1998). Performance appraisals in hotels. Cornell Hotel and Restaurant Administration Quarterly, 39, 25–29.

Titel: BARS and Those Mysterious, Missing Middle Anchors
verfasst von: Neil M. A. Hauenstein
Reagan D. Brown
Andrea L. Sinclair
Publikationsdatum: 01.12.2010
Verlag: Springer US
Erschienen in: Journal of Business and Psychology / Ausgabe 4/2010
Print ISSN: 0889-3268
Elektronische ISSN: 1573-353X
DOI: https://doi.org/10.1007/s10869-010-9180-7

Springer Professional

Abstract

Purpose

Design/Methodology/Approach

Findings

Implications

Originality/Value

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Weitere Artikel der Ausgabe 4/2010

Emotional Exhaustion Among Employees Without Social or Client Contact: The Key Role of Nonstandard Work Schedules

The Impact of Organizational Culture on Attraction and Recruitment of Job Applicants

The Mediating Role of Psychological Empowerment on the Relationships between P–O Fit, Job Satisfaction, and In-role Performance

Perceptions of Human Capital Measures: From Corporate Executives and Investors

The Joint Influence of Supervisor and Subordinate Emotional Intelligence on Leader–Member Exchange

Work Status Congruence’s Relation to Employee Attitudes and Behaviors: The Moderating Role of Procedural Justice

Premium Partner