A fuzzy AHP and BSC approach for evaluating performance of IT department in the manufacturing industry in Taiwan

https://doi.org/10.1016/j.eswa.2006.08.022Get rights and content

Abstract

In this ever-changing world, information technology (IT) is a must for the survival of a company, and the functions of IT department is becoming increasingly important. The assessment of IT department is critical to understand how the department contributes to organizational and strategic goals. Because IT department performs many tasks that cannot simply be measured by monetary units, evaluation methods that solely rely on financial measures are not adequate. The objective of this study is to construct an approach based on the fuzzy analytic hierarchy process (FAHP) and balanced scorecard (BSC) for evaluating an IT department in the manufacturing industry in Taiwan. The BSC concept is applied to define the hierarchy with four major perspectives (i.e. financial, customer, internal business process, and learning and growth), and performance indicators are selected for each perspective. A fuzzy AHP (FAHP) approach is then proposed in order to tolerate vagueness and ambiguity of information. A FAHP information system is finally constructed to facilitate the solving process. The results provide guidance to IT departments in the manufacturing industry in Taiwan regarding strategies for improving department performance. The constructed information system is suggested to be a good tool for solving other multiple-criteria decision-making problems.

Introduction

Information technology (IT) involves computers, software and services, but good IT must synthesize these elements to achieve the goal of an organization. As a demand to collect, process, store, and disseminate information grows, the functions of IT department is becoming increasingly important. Although businesses invest huge amount of intellectual and financial capital in a range of communication and information technologies and services, the results of some surveys revealed that some companies have started to freeze IT budgets because there are insufficient evidence of a return from the investments and IT applications seem to be simply a black hole (Martinsons, Davison, & Tse, 1999). The reason behind is that it is difficult for managers to demonstrate tangible returns on the resources expended to plan, develop, implement and operate computer-based information system (IS). Some frequently asked questions by the organizations are whether the investment in IT/IS is really worthwhile, whether the implemented IT application is a success, and whether the IT department functions productively. The measurement of the value of IT and the evaluation of IS performance, thus, become of great importance to managers.

Many methods and techniques have been suggested over the years to evaluate the investments in IT/IS or the performance of IT departments. However, well-known financial measures such as return on investment (ROI), internal rate of return (IRR), net present value (NPV) and payback period have been demonstrated to be inadequate (Abran & Buglione, 2003). In the assessment of IT/IS investments or departments, it is critical to understand how IT/IS contribute to organizational and strategic goals, and evaluation methods that rely on financial measures alone are not suitable for IT applications. The balanced scorecard (BSC), a performance measurement framework that provides an integrated look at the business performance of a company by a set of both financial and non-financial measures, seems to be a good solution. However, conventional BSC does not consolidate these performance measures, and an incorporation of BSC and analytic hierarchy process (AHP) is an improvement. Since fuzziness and vagueness are common characteristics in many decision-making problems, a fuzzy AHP (FAHP) and BSC method should be able to tolerate vagueness or ambiguity, and therefore, is proposed in this research.

The remainder of this paper is organized as follows. Section 2 briefly introduces the BSC and AHP. Section 3 goes over the fuzzy set theory. Section 4 reviews the incorporation of BSC with other methodologies and the application of the BSC in IT/IS field. Section 5 is the proposed model, in which a FAHP and BSC method is proposed, a FAHP information system is constructed, and the performance evaluation for IT department is carried out. Some conclusion remarks are made in the last section.

Section snippets

The balanced scorecard (BSC) and the analytic hierarchy process (AHP)

Focusing exclusively on traditional financial accounting measures, such as return on investment and payback period, has implications, and has been criticized as the root cause for many problems in industries (Hafeez, Zhang, & Malak, 2002). As managers stress on short-term financial performance metrics, they have a tendency to trade off actions, such as new product development, process improvements, human resource development, information technology and customer and market development that can

Fuzzy set theory

Zadeh in 1965 introduced fuzzy set theory to solve problems involving the absence of sharply defined criteria (Zadeh, 1965). If uncertainty (fuzziness) of human decision-making is not taken into account, the results can be misleading. A commonality among terms of expression, such as “very likely”, “probably so”, “not very clear”, “rather dangerous” that are often heard in daily life, is that they all contain some degree of uncertainty (Tsaur et al., 1997, Tsaur et al., 2002). Fuzzy theory thus

The incorporation of BSC with other methodologies and the application of BSC in IT/IS field

Some recent researches related to the combination of the BSC and other methodologies are reviewed here. Banker et al. (2004) do a BSC analysis of performance metrics in the US telecommunications industry. Four performance metrics are used to fit the template of four perspectives of the BSC, i.e., return on assets (ROA), number of access lines per employee, percentage of digital access lines and percentage of business access lines for the financial, internal process, innovation and learning, and

Proposed model

In this research, we first base on the four perspectives of the BSC to prepare a list of performance evaluation indicators, and then have an interview with the experts in IT departments of manufacturing companies in Taiwan to modify the list. A questionnaire is designed using the conventional AHP questionnaire format, and the four perspectives of the BSC and the selected performance indicators are included. The questionnaire is distributed to senior managers of the IT departments in the

Conclusions

This paper proposes an approach based on the FAHP and BSC for evaluating the performance of IT department in the manufacturing industry in Taiwan. The analytic hierarchy is structured by the four major perspectives of the BSC including financial, customer, internal business process, and learning and growth, followed by performance indicators. Because human decision-making process usually contains fuzziness and vagueness, the FAHP is adopted to solve the problem. A well-organized FAHP

References (50)

  • E.S. Lee et al.

    Comparison of fuzzy numbers based on the probability measure of fuzzy events

    Computers and Mathematics with Applications

    (1988)
  • G.S. Liang et al.

    Personnel selection using fuzzy MCDM algorithm

    European Journal of Operational Research

    (1994)
  • M. Martinsons et al.

    The balanced scorecard: a foundation for the strategic management of information systems

    Decision Support Systems

    (1999)
  • K. Milis et al.

    The use of the balanced scorecard for the evaluation of information and communication technology projects

    International Journal of Project Management

    (2004)
  • E.W.T. Ngai et al.

    Evaluation of knowledge management tools using AHP

    Expert Systems with Applications

    (2005)
  • V. Ravi et al.

    Analyzing alternatives in reverse logistics for end-of-life computers: ANP and balanced scorecard approach

    Computers and Industrial Engineering

    (2005)
  • M.H. Sohn et al.

    Corporate strategies, environmental forces, and performance measures: a weighting decision support system using the k-nearest neighbor technique

    Expert Systems with Applications

    (2003)
  • S.H. Tsaur et al.

    The evaluation of airline service quality by fuzzy MCDM

    Tourism Management

    (2002)
  • C.S. Yu

    A GP-AHP method for solving group decision-making fuzzy AHP problems

    Computers and Operations Research

    (2002)
  • L.A. Zadeh

    Fuzzy sets

    Information and Control

    (1965)
  • J.M. Adamo

    Fuzzy decision trees

    Fuzzy Sets and Systems

    (1980)
  • Bozbura, F.T., Beskese, A. & Kahraman, C. (2006). Prioritization of human capital measurement indicators using fuzzy...
  • C.H. Cheng

    Evaluating naval tactical missile systems by fuzzy AHP based on the grade value of membership function

    European Journal of Operational Research

    (1996)
  • Z. Chiang

    A dynamic decision approach for long-term vendor selection based on AHP and BSC

  • S.C. Chi et al.

    Examination of the influence of fuzzy analytic hierarchy process in the development of an intelligent location selection support system of convenience store

    IFSA World Congress and 20th NAFIPS International Conference

    (2001)
  • Cited by (494)

    View all citing articles on Scopus
    View full text