Financial versus non-financial information: The impact of information organization and presentation in a Balanced Scorecard

https://doi.org/10.1016/j.aos.2010.05.003Get rights and content

Abstract

This paper investigates how the organization and presentation of performance measures affect how evaluators weight financial and non-financial measures when evaluating performance. We conduct two experiments, in which participants act as senior executives charged with evaluating two business-unit managers. Performance differences between business units are contained in either a financial or one of the three non-financial categories. Specifically, the first experiment studies how organizing measures in a Balanced Scorecard (BSC) format affects performance evaluations. Our results show that when the performance differences are contained in the financial category, evaluators that use a BSC-format place more weight on financial category measures than evaluators using an unformatted scorecard. Conversely, when performance differences are contained in the non-financial categories, whether measures are organized into a BSC-format or into an unformatted scorecard has no impact on the evaluation. The second experiment shows that when performance markers are added to the scorecards (i.e., +, −, and = signs for above-target, below-target, and on-target performance), evaluators that use a BSC-format weight measures in any category containing a performance difference more heavily than evaluators using an unformatted scorecard. Our findings suggest that firms should carefully consider how to present and organize measures to get the intended effect on performance evaluations.

Introduction

Kaplan and Norton (1992) originally introduced the Balanced Scorecard (BSC) to overcome problems that result from a sole focus on financial measures. A BSC enables financial performance measures (grouped into a single financial category) and non-financial performance measures (grouped into non-financial categories including customer, internal business process, and learning and growth) to be displayed in combination. In practice, the format of performance scorecards varies significantly across firms (Lohman, Fortuin, & Wouters, 2004). Some firms organize their measures into BSC performance categories while others simply provide a general list of measures. How results are presented in a scorecard also varies. Many firms show only target levels and actual results, while other firms supplement this information with performance markers (i.e., +, −, =) or qualitative signs (e.g., red, yellow, and green indicators) to more explicitly indicate the status of the actual results in relation to the target levels (e.g., Malina et al., 2007, Malina and Selto, 2001, Merchant and Van der Stede, 2007). Despite the prevalence of these different formats in practice, little work has been done on how variations in scorecard formats affect performance evaluations.

In this study, we examine how variations in, first, the organization (i.e., BSC versus unformatted scorecard) and, second, the presentation of measures (i.e., the use of markers) affect how evaluators weight financial and non-financial measures in performance evaluations. Prior studies have primarily focused on the finding that, when firms use both common measures (i.e., measures common across multiple units) and unique measures (i.e., measures unique to particular units) for their business units, evaluators ignore the unique measures (Lipe & Salterio, 2000). Solutions to this problem have also been explored (Libby et al., 2004, Roberts et al., 2004). Many firms, however, use similar scorecards that contain only measures common to all business units (e.g., Malina & Selto, 2001). In such cases, presentation formats and features may well affect how evaluators weight financial and non-financial information in performance evaluations. To investigate these issues, we present two experiments that extend the basic setup of Lipe and Salterio (2002).

Lipe and Salterio (2002) study how information organization (i.e., how organizing measures into a BSC as opposed to an unformatted list) affects the performance evaluation of two business-unit managers. They consider, however, only the case wherein performance differences between the two business units (i.e., consistent above-target performance for one business unit and consistent below-target performance for the other) are located on the non-financial category of customer measures. They show that evaluators using a BSC weigh these measures less heavily than evaluators viewing the same measures in an unformatted scorecard.

Our first experiment extends Lipe and Salterio’s work by examining whether the effect of how the measures are organized depends on which type of category—that is, financial or non-financial—contains the performance differences between business units. We predict that information organization will have a greater effect on evaluations when performance differences appear in the financial category. We base this prediction on performance-measurement as well as psychology literature, which suggest both that people are heavily led by financial outcomes and that how people use a BSC to process information may lead these users to place more weight on financial performance measures than users of an unformatted scorecard. We use a 2 × 4 design, manipulating how information is organized (i.e., in a BSC or an unformatted scorecard) when performance differences between two business units are located in either the financial category or one of three non-financial categories. We qualify the results of Lipe and Salterio (2002) by showing that a BSC only “increases” the weight evaluators attach to performance differences when these differences are located in the financial category. We find that when performance differences are located in one of the three non-financial categories, information organization has no effect. We thus also observe no decrease in how measures are weighted for the customer category, which is the only case comparable to that of Lipe and Salterio (2002). We attribute this latter finding to some differences in design choices, which we will explain in ‘Methods and results’.

Increasing the weight evaluators place on financials may not always be the effect firms hope to achieve by using a BSC instead of an unformatted list of measures.1 Therefore, our second experiment examines whether the use of markers (i.e., +, −, and = signs for above-target, below-target, or on-target performance) offers a counterbalancing effect. The design of Experiment 2 is similar to that of Experiment 1 except that we add performance markers to the scorecards’ results. We hypothesize, and find, that, when supplemented with markers, performance differences on measures of any category, be it financial or non-financial, are always weighted more heavily in a BSC than in an unformatted scorecard.

Our research contributes to the literature in several ways. First, prior results on the use of financial and non-financial measures are still inconclusive (Luft and Shields, 2001, Schiff and Hoffman, 1996). Although the BSC has gained prominence in accounting research as a way of integrating financial and non-financial performance measures (Hoque & James, 2000), we show a consequence of organizing the measures into the BSC categories that may well be uncalled-for if firms adopt a BSC to stimulate the use of non-financials. Our finding in Experiment 1 that a BSC only increases the weight evaluators assign to the financial category, leaving non-financial categories unaffected, adds a new issue to the BSC literature, which to date has focused on the problem of common versus unique measures.

Second, we show how different presentation formats can produce different processing strategies (Payne, 1982, Schkade and Kleinmuntz, 1994). In Experiment 1, we show that grouping and labeling measures (i.e., in a BSC), as opposed to leaving measures unlabeled and in no particular order (i.e., in an unformatted scorecard), helps evaluators identify financials more easily and may activate their beliefs in the relative importance of financials. As a result, a BSC-format increases an evaluator’s basic tendency to weight financial measures more heavily than non-financial measures. Experiment 2 shows that performance markers in a BSC can also direct an evaluator’s attention to other non-financial categories that contain important performance differences. In this case, BSC users compared with users of an unformatted scorecard, give more weight to any category (financial and non-financial alike) that shows consistently good or bad performance.

These findings have important practical implications for the many firms that use the BSC as a tool to evaluate and reward managers (Kaplan and Norton, 1996, Liedka et al., 2008). If evaluators assimilated all measures without bias, then the format of a scorecard would not matter. However, because format, in fact, appears to have a strong impact on how evaluators assimilate measures, firms should carefully consider how they display these measures. Given that managers’ behavior is driven by weights placed on the performance measures (e.g., Ittner et al., 2003, Smith, 2002), formatting can thus have far-reaching consequences for the firm.

Section snippets

Hypothesis development

Assessing and combining the scores of various performance measures into an overall evaluation is a complex task (Lipe and Salterio, 2000, Lipe and Salterio, 2002). Due to information processing limitations (Baddeley, 1994, Hastie, 1991, Shanteau, 1988), evaluators often have cognitive difficulties making evaluation judgments. While Kaplan and Norton (1996) have proposed the BSC as a tool that enables managers to utilize important non-financial as well as financial measures, prior work has not

Selection of the performance measures for both experiments

For both experiments, we use case materials adapted from prior studies on the BSC (e.g., Banker et al., 2004, Lipe and Salterio, 2000, Lipe and Salterio, 2002). Participants assume the role of senior executive of the retail firm, “VQS Inc.,” which specializes in clothing. Participants review the performance of two VQS business units, “Streetware” and “Family Fashion.” Streetware specializes in youth fashion, and Family Fashion in clothing for young families. Managers and strategies for these

Discussion

Our paper studies how variations in the format of scorecards and the presentation of measures therein affect how evaluators weight financial versus non-financial information in performance evaluations. Experiment 1 shows that when performance differences are located in the financial category, BSC users place more weight on financial measures than do users of an unformatted scorecard. In contrast, when performance differences are located in one of the non-financial categories, the type of

Acknowledgements

We want to thank Mike Shields (editor) and the two anonymous referees for their helpful suggestions. We further want to thank Maggie Abernethy, Jan Bouwens, Penelope Cray, Chris Ittner, Ken Merchant, Mina Pizzini, Steve Salterio, Ed Vosselman, William Waller, and seminar participants at Tilburg University, the University of Leuven, the ARN and ERIM seminars in Rotterdam, the MAS mid-year Conference in Tampa, the EIASM conference for new directions in management accounting in Brussels, and the

References (57)

  • J. Shanteau

    How much information does an expert use? Is it relevant

    Acta Psychologica

    (1992)
  • G.A. Sundstrom

    Information search and decision making: The effects of information displays

    Acta Psychologica

    (1987)
  • P.M.G. Van Veen-Dirks

    Complementary choices and management control: Field research in a flexible production environment

    Management Accounting Research

    (2006)
  • P.M.G. Van Veen-Dirks

    Different uses of performance measures: The evaluation versus reward of production managers

    Accounting, Organizations and Society

    (2010)
  • E.D. Almer et al.

    A research tool to increase attention to experimental materials: Manipulating presentation format

    Journal of Business and Psychology

    (2003)
  • R.N. Anthony et al.

    Management control systems

    (2001)
  • A. Baddeley

    The magical number seven: Still magic after all these years

    Psychological Review

    (1994)
  • R.D. Banker et al.

    The Balanced Scorecard: Judgmental effects of performance measures linked to strategy

    The Accounting Review

    (2004)
  • J. Bauer et al.

    Developing a performance measurement audit template—A benchmarking study

    Measuring Business Excellence

    (2004)
  • S.E. Bonner et al.

    Determinants of auditor expertise

    Journal of Accounting Research

    (1990)
  • F.A. Buckless et al.

    Contrast Coding: A refinement of ANOVA in behavioral analysis

    The Accounting Review

    (1990)
  • S.K. Card et al.

    Readings in information visualization: Using vision to think

    (1999)
  • W.N. Dilla et al.

    Relative weighting of common and unique Balanced Scorecard measures by knowledgeable decision makers

    Behavioral Research in Accounting

    (2005)
  • G.A. Feltham et al.

    Performance measure congruity and diversity in multi-task principal/agent relations

    The Accounting Review

    (1994)
  • S. Few

    Information dashboard design: The effective visual communication of data

    (2006)
  • S.T. Fiske et al.

    Social cognition: From brains to culture

    (2008)
  • R. Hastie

    A review from a high place: The field of judgment and decision making as revealed in current textbooks

    Psychological Science

    (1991)
  • S.A. Hawkins et al.

    Hindsight: Biased judgments of past events after the outcomes are known

    Psychological Bulletin

    (1990)
  • Cited by (153)

    View all citing articles on Scopus
    View full text