Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Journal of Business and Psychology
Published in: Journal of Business and Psychology 4/2010

01-12-2010

BARS and Those Mysterious, Missing Middle Anchors

Authors: Neil M. A. Hauenstein, Reagan D. Brown, Andrea L. Sinclair

Published in: Journal of Business and Psychology | Issue 4/2010

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Purpose

A contributing reason for the common problem of missing middle anchors on behaviorally anchored rating scales (BARS) is the standard deviation (SD) criterion used in scaling phase. An alternative BARS scaling process is proposed based on the a wg(1) index of interrater agreement.

Design/Methodology/Approach

Algebraic principles are used to explicate that the SD criterion is analogous to using a r wg(1) interrater agreement statistic with the assumption of a uniform null distribution, and this reliance on r wg(1) decreases the likelihood of anchoring behaviors in the mid-range of the rating scale. Archival data from a law enforcement agency were used to compare the success and failure of anchoring a BARS using a SD criterion versus an a wg(1) criterion.

Findings

The a wg(1) criterion was successful at anchoring the full range of the rating scale, but only if the cut-off for anchoring behaviors required a “weak” level of interrater agreement.

Implications

The most surprising finding was that the traditional 1.5 SD criterion on a 9-point rating scale is not a particularly stringent agreement requirement for anchoring behaviors. Although we demonstrated the advantages of using a wg(1) to anchor BARS, an equally important conclusion is that incumbents need to be better trained prior to scaling behaviors.

Originality/Value

We provide a theoretically defensible approach for anchoring BARS that ameliorates the missing middle anchor problem. Further, the utility of a wg(1) in the BARS context is yet another example of the limitation of r wg(1) when assuming a uniform null distribution.
previous article The Impact of Organizational Culture on Attraction and Recruitment of Job Applicants
next article Perceptions of Human Capital Measures: From Corporate Executives and Investors

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
Footnotes
1
It is interesting that neither the 1.25 SD criterion with a 7-point scale nor the 1.0 SD criterion with a 5-point scale are comparable to the more stringent 1.5 SD criterion using a 9-point scale. The 1.0 SD with a 5-point scale is the most lenient combination for anchoring behaviors, and the 1.25 SD criterion with a 7-point scale is between the other two combinations in terms of the probability of anchoring behaviors. To achieve comparability with the 1.5 SD criterion using a 9-point scale, a SD criterion of 1.125 is needed for the 7-point scale, and a SD criterion of .75 is needed for the 5-point scale.
 
Literature
Barnes-Farrell, J. L., & Lynch, A. M. (2003). Performance appraisal and feedback programs. In J. E. Edwards, J. C. Scott, & N. S. Raju (Eds.), The Human Resources Program-Evaluation Handbook (pp. 155–176). Thousand Oaks, CA: Sage Publications.
Bernardin, H. J., & Smith, P. C. (1981). A clarification of some issues regarding the development and use of behaviorally anchored ratings scales (BARS). Journal of Applied Psychology, 66, 458–463.CrossRef
Bownas, D. A., & Bernardin, H. J. (1988). Critical incident technique. In S. Gael (Ed.), The job analysis handbook for business, industry, and government (pp. 1120–1137). New York, NY: Wiley.
Brown, R. D., & Hauenstein, N. M. A. (2005). Interrater agreement reconsidered: An alternative to the rwg indices. Organizational Research Methods, 8, 1–20.CrossRef
Burke, M. J., & Dunlop, W. P. (2002). Estimating interrater agreement with the average deviation index: A user’s guide. Organizational Research Methods, 5, 159–172.CrossRef
Burke, M. J., Finkelstein, L. M., & Dusig, M. S. (1999). On average deviation indices for estimating interrater agreement. Organizational Research Methods, 2, 49–68.CrossRef
Cardy, R. L. (1998). Performance appraisal in a quality context: A new look at an old problem. In J. Smither (Ed.), Performance appraisal: State of the art in practice (pp. 132–162). Josey-Bass: San Francisco.
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37–46.CrossRef
Conte, J. M., Landy, F. J., & Mathieu, J. E. (1995). Time urgency: Conceptual and construct development. Journal of Applied Psychology, 80, 178–185.CrossRef
DeNisi, A. S. (1996). Cognitive approach to performance appraisal: A program of research. New York: Routledge.
Engelbrecht, A. S., & Fischer, H. (1995). The managerial performance implications of a developmental assessment center process. Human Relations, 48, 387–404.CrossRef
Finn, R. H. (1970). A note on estimating the reliability of categorical data. Educational and Psychological Measurement, 30, 71–76.CrossRef
George, J. M. (1990). Personality, affect and behavior in groups. Journal of Applied Psychology, 75, 107–116.CrossRef
Grote, D. (1996). The complete guide to performance appraisal. New York: American Management Association.
Harari, O., & Zedeck, S. (1973). Development of behaviorally anchored scales for the evaluation of faculty teaching. Journal of Applied Psychology, 58, 261–265.CrossRef
Hauenstein, N. M. A., & Foti, R. J. (1989). From laboratory to practice: Neglected issues in implementing frame-of-reference rater training. Personnel Psychology, 42, 359–378.CrossRef
James, L. R., Demaree, R. G., & Wolf, G. (1984). Estimating within-group interrater reliability with and without response bias. Journal of Applied Psychology, 69, 85–98.CrossRef
Judge, T. A., & Bono, J. E. (2000). Five factor model of personality and transformational leadership. Journal of Applied Psychology, 85, 751–765.CrossRefPubMed
Kozlowski, S. W., & Hattrup, K. (1992). A disagreement about within-group agreement: Disentangling issues of consistency versus consensus. Journal of Applied Psychology, 77, 161–167.CrossRef
Landy, F. J., & Guion, R. M. (1970). Development of scales for the measurement of work motivation. Organizational Behavior and Human Decision Processes, 5, 93–103.
Latham, G. P., Saari, L. M., Pursell, E. D., & Campion, M. A. (1980). The situational interview. Journal of Applied Psychology, 65, 422–427.CrossRef
London, M., Mone, E. M., & Scott, J. C. (2004). Performance management and assessment: Methods for improved rater accuracy and employee goal setting. Human Resource Management, 43, 319–336.CrossRef
Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
Malos, S. B. (1998). Current legal issues in performance appraisal. In J. Smither (Ed.), Performance appraisal: State of the art in practice (pp. 49–94). San Francisco: Josey-Bass.
Martin, D. D., & Bartol, K. M. (1998). Performance appraisal: Maintaining system effectiveness. Public Personnel Management, 27, 223–230.
Maurer, S. D. (2002). A practitioner-based analysis of interviewer job expertise and scale format as contextual factors in situational interviews. Personnel Psychology, 55, 307–328.CrossRef
Murphy, K. R., & Cleveland, J. N. (1995). Understanding performance appraisal: Social, organizational, and goal-based perspectives. Thousand Oaks, CA: Sage Publications.
Murphy, K. R., & De Shon, R. (2000a). Interrater correlations do not estimate the reliability of job performance ratings. Personnel Psychology, 53, 873–900.CrossRef
Murphy, K. R., & De Shon, R. (2000b). Progress in psychometrics: Can industrial and organizational psychology catch up? Personnel Psychology, 53, 913–924.CrossRef
Ramus, C. A., & Steger, U. (2000). The roles of supervisory support behaviors and environmental policy in employee “ecoinitiatives” at leading-edge European companies. Academy of Management Journal, 43, 605–626.CrossRef
Schleicher, D. J., Day, D. V., Mayes, B. T., & Riggio, R. E. (2002). A new frame for frame-of-reference training: Enhancing the construct validity of assessment centers. Journal of Applied Psychology, 87, 735–746.CrossRefPubMed
Schmidt, F. L., Viswesvaran, C., & Ones, D. S. (2000). Reliability is not validity and validity is not reliability. Personnel Psychology, 53, 901–912.CrossRef
Smith, P. C., & Kendall, L. M. (1963). Retranslation of expectations: An approach to the construction of unambiguous anchors for rating scales. Journal of Applied Psychology, 47, 149–255.CrossRef
Tziner, A., Joanis, C., & Murphy, K. (2000). A comparison of three methods of performance appraisal with regard to goal properties, goal perception, and ratee satisfaction. Group & Organization Management, 25, 175–190.CrossRef
Woods, R. H., Sciarini, M., & Breiter, D. (1998). Performance appraisals in hotels. Cornell Hotel and Restaurant Administration Quarterly, 39, 25–29.
Metadata
Title
BARS and Those Mysterious, Missing Middle Anchors
Authors
Neil M. A. Hauenstein
Reagan D. Brown
Andrea L. Sinclair
Publication date
01-12-2010
Publisher
Springer US
Published in
Journal of Business and Psychology / Issue 4/2010
Print ISSN: 0889-3268
Electronic ISSN: 1573-353X
DOI
https://doi.org/10.1007/s10869-010-9180-7

Other articles of this Issue 4/2010

Journal of Business and Psychology 4/2010 Go to the issue

OriginalPaper

Polynomial Regression with Response Surface Analysis: A Powerful Approach for Examining Moderation and Overcoming Limitations of Difference Scores

OriginalPaper

The Joint Influence of Supervisor and Subordinate Emotional Intelligence on Leader–Member Exchange

OriginalPaper

Work Status Congruence’s Relation to Employee Attitudes and Behaviors: The Moderating Role of Procedural Justice

OriginalPaper

The Mediating Role of Psychological Empowerment on the Relationships between P–O Fit, Job Satisfaction, and In-role Performance

OriginalPaper

A General Structure of Job Performance: Evidence from Two Studies

OriginalPaper

The Impact of Organizational Culture on Attraction and Recruitment of Job Applicants

Premium Partner

    Image Credits