Financial versus non-financial information: The impact of information organization and presentation in a Balanced Scorecard
Introduction
Kaplan and Norton (1992) originally introduced the Balanced Scorecard (BSC) to overcome problems that result from a sole focus on financial measures. A BSC enables financial performance measures (grouped into a single financial category) and non-financial performance measures (grouped into non-financial categories including customer, internal business process, and learning and growth) to be displayed in combination. In practice, the format of performance scorecards varies significantly across firms (Lohman, Fortuin, & Wouters, 2004). Some firms organize their measures into BSC performance categories while others simply provide a general list of measures. How results are presented in a scorecard also varies. Many firms show only target levels and actual results, while other firms supplement this information with performance markers (i.e., +, −, =) or qualitative signs (e.g., red, yellow, and green indicators) to more explicitly indicate the status of the actual results in relation to the target levels (e.g., Malina et al., 2007, Malina and Selto, 2001, Merchant and Van der Stede, 2007). Despite the prevalence of these different formats in practice, little work has been done on how variations in scorecard formats affect performance evaluations.
In this study, we examine how variations in, first, the organization (i.e., BSC versus unformatted scorecard) and, second, the presentation of measures (i.e., the use of markers) affect how evaluators weight financial and non-financial measures in performance evaluations. Prior studies have primarily focused on the finding that, when firms use both common measures (i.e., measures common across multiple units) and unique measures (i.e., measures unique to particular units) for their business units, evaluators ignore the unique measures (Lipe & Salterio, 2000). Solutions to this problem have also been explored (Libby et al., 2004, Roberts et al., 2004). Many firms, however, use similar scorecards that contain only measures common to all business units (e.g., Malina & Selto, 2001). In such cases, presentation formats and features may well affect how evaluators weight financial and non-financial information in performance evaluations. To investigate these issues, we present two experiments that extend the basic setup of Lipe and Salterio (2002).
Lipe and Salterio (2002) study how information organization (i.e., how organizing measures into a BSC as opposed to an unformatted list) affects the performance evaluation of two business-unit managers. They consider, however, only the case wherein performance differences between the two business units (i.e., consistent above-target performance for one business unit and consistent below-target performance for the other) are located on the non-financial category of customer measures. They show that evaluators using a BSC weigh these measures less heavily than evaluators viewing the same measures in an unformatted scorecard.
Our first experiment extends Lipe and Salterio’s work by examining whether the effect of how the measures are organized depends on which type of category—that is, financial or non-financial—contains the performance differences between business units. We predict that information organization will have a greater effect on evaluations when performance differences appear in the financial category. We base this prediction on performance-measurement as well as psychology literature, which suggest both that people are heavily led by financial outcomes and that how people use a BSC to process information may lead these users to place more weight on financial performance measures than users of an unformatted scorecard. We use a 2 × 4 design, manipulating how information is organized (i.e., in a BSC or an unformatted scorecard) when performance differences between two business units are located in either the financial category or one of three non-financial categories. We qualify the results of Lipe and Salterio (2002) by showing that a BSC only “increases” the weight evaluators attach to performance differences when these differences are located in the financial category. We find that when performance differences are located in one of the three non-financial categories, information organization has no effect. We thus also observe no decrease in how measures are weighted for the customer category, which is the only case comparable to that of Lipe and Salterio (2002). We attribute this latter finding to some differences in design choices, which we will explain in ‘Methods and results’.
Increasing the weight evaluators place on financials may not always be the effect firms hope to achieve by using a BSC instead of an unformatted list of measures.1 Therefore, our second experiment examines whether the use of markers (i.e., +, −, and = signs for above-target, below-target, or on-target performance) offers a counterbalancing effect. The design of Experiment 2 is similar to that of Experiment 1 except that we add performance markers to the scorecards’ results. We hypothesize, and find, that, when supplemented with markers, performance differences on measures of any category, be it financial or non-financial, are always weighted more heavily in a BSC than in an unformatted scorecard.
Our research contributes to the literature in several ways. First, prior results on the use of financial and non-financial measures are still inconclusive (Luft and Shields, 2001, Schiff and Hoffman, 1996). Although the BSC has gained prominence in accounting research as a way of integrating financial and non-financial performance measures (Hoque & James, 2000), we show a consequence of organizing the measures into the BSC categories that may well be uncalled-for if firms adopt a BSC to stimulate the use of non-financials. Our finding in Experiment 1 that a BSC only increases the weight evaluators assign to the financial category, leaving non-financial categories unaffected, adds a new issue to the BSC literature, which to date has focused on the problem of common versus unique measures.
Second, we show how different presentation formats can produce different processing strategies (Payne, 1982, Schkade and Kleinmuntz, 1994). In Experiment 1, we show that grouping and labeling measures (i.e., in a BSC), as opposed to leaving measures unlabeled and in no particular order (i.e., in an unformatted scorecard), helps evaluators identify financials more easily and may activate their beliefs in the relative importance of financials. As a result, a BSC-format increases an evaluator’s basic tendency to weight financial measures more heavily than non-financial measures. Experiment 2 shows that performance markers in a BSC can also direct an evaluator’s attention to other non-financial categories that contain important performance differences. In this case, BSC users compared with users of an unformatted scorecard, give more weight to any category (financial and non-financial alike) that shows consistently good or bad performance.
These findings have important practical implications for the many firms that use the BSC as a tool to evaluate and reward managers (Kaplan and Norton, 1996, Liedka et al., 2008). If evaluators assimilated all measures without bias, then the format of a scorecard would not matter. However, because format, in fact, appears to have a strong impact on how evaluators assimilate measures, firms should carefully consider how they display these measures. Given that managers’ behavior is driven by weights placed on the performance measures (e.g., Ittner et al., 2003, Smith, 2002), formatting can thus have far-reaching consequences for the firm.
Section snippets
Hypothesis development
Assessing and combining the scores of various performance measures into an overall evaluation is a complex task (Lipe and Salterio, 2000, Lipe and Salterio, 2002). Due to information processing limitations (Baddeley, 1994, Hastie, 1991, Shanteau, 1988), evaluators often have cognitive difficulties making evaluation judgments. While Kaplan and Norton (1996) have proposed the BSC as a tool that enables managers to utilize important non-financial as well as financial measures, prior work has not
Selection of the performance measures for both experiments
For both experiments, we use case materials adapted from prior studies on the BSC (e.g., Banker et al., 2004, Lipe and Salterio, 2000, Lipe and Salterio, 2002). Participants assume the role of senior executive of the retail firm, “VQS Inc.,” which specializes in clothing. Participants review the performance of two VQS business units, “Streetware” and “Family Fashion.” Streetware specializes in youth fashion, and Family Fashion in clothing for young families. Managers and strategies for these
Discussion
Our paper studies how variations in the format of scorecards and the presentation of measures therein affect how evaluators weight financial versus non-financial information in performance evaluations. Experiment 1 shows that when performance differences are located in the financial category, BSC users place more weight on financial measures than do users of an unformatted scorecard. In contrast, when performance differences are located in one of the non-financial categories, the type of
Acknowledgements
We want to thank Mike Shields (editor) and the two anonymous referees for their helpful suggestions. We further want to thank Maggie Abernethy, Jan Bouwens, Penelope Cray, Chris Ittner, Ken Merchant, Mina Pizzini, Steve Salterio, Ed Vosselman, William Waller, and seminar participants at Tilburg University, the University of Leuven, the ARN and ERIM seminars in Rotterdam, the MAS mid-year Conference in Tampa, the EIASM conference for new directions in management accounting in Brussels, and the
References (57)
- et al.
The usefulness of a performance measurement system in the daily life of an organisation: A note on a case study
British Accounting Review
(2003) The interplay between cost accounting knowledge and presentation formats in cost-based decision making
Accounting, Organizations and Society
(2008)- et al.
Components and relative weights in utilization of performance measurement systems like the Balanced Scorecard
British Accounting Review
(2003) - et al.
The effects of alternative supplementary display formats on Balanced Scorecard judgments
International Journal of Accounting Information Systems
(2005) Feedback representation and prediction strategies
Organizational Behavior and Human Decision Processes
(1994)- et al.
Outcome effect, controllability, and performance evaluation of managers: Some field evidence from multi-outlet business
Accounting, Organization and Society
(2000) - et al.
A note on the judgmental effects of the Balanced Scorecard’s information organization
Accounting, Organizations and Society
(2002) - et al.
Designing a performance measurement system: A case study
European Journal of Operational Research
(2004) - et al.
Information displays and choice processes: Differential effects of organization, form, and sequence
Organizational Behavior and Human Decision Processes
(1994) Psychological characteristics and strategies of expert decision makers
Acta Psychologica
(1988)
How much information does an expert use? Is it relevant
Acta Psychologica
Information search and decision making: The effects of information displays
Acta Psychologica
Complementary choices and management control: Field research in a flexible production environment
Management Accounting Research
Different uses of performance measures: The evaluation versus reward of production managers
Accounting, Organizations and Society
A research tool to increase attention to experimental materials: Manipulating presentation format
Journal of Business and Psychology
Management control systems
The magical number seven: Still magic after all these years
Psychological Review
The Balanced Scorecard: Judgmental effects of performance measures linked to strategy
The Accounting Review
Developing a performance measurement audit template—A benchmarking study
Measuring Business Excellence
Determinants of auditor expertise
Journal of Accounting Research
Contrast Coding: A refinement of ANOVA in behavioral analysis
The Accounting Review
Readings in information visualization: Using vision to think
Relative weighting of common and unique Balanced Scorecard measures by knowledgeable decision makers
Behavioral Research in Accounting
Performance measure congruity and diversity in multi-task principal/agent relations
The Accounting Review
Information dashboard design: The effective visual communication of data
Social cognition: From brains to culture
A review from a high place: The field of judgment and decision making as revealed in current textbooks
Psychological Science
Hindsight: Biased judgments of past events after the outcomes are known
Psychological Bulletin
Cited by (153)
Computerized accounting information systems: An application of task technology fit model for microfinance
2024, International Journal of Information Management Data InsightsExperiencing with visuals in accounting education: The case of integrated reports
2023, International Journal of Management EducationA comprehensive bibliometric study of the balanced scorecard
2023, Evaluation and Program PlanningThe lure of the visual: Multimodality, simplification, and performance measurement visualizations in a megaproject
2022, Accounting, Organizations and Society