A multi-attribute approach to ranking departments based on performance: a balanced scorecard pilot study

Enterprise performance issue is one of the main concerns in the business area. Therefore, companies make performance analyses to follow their status. In this direction, they develop a performance system including key performance indicators (KPI) for departments. Following critical factor results helps the decision makers to understand strengths and weaknesses. Basu and Wright [1], Letza [8], and Kaplan and Norton [9] discussed that the companies should determine critical KPIs reflecting the departments’ success to measure the accurate company performance. Furthermore, many systems developed to meet the performance evaluation system (PES) requirement such as Strategic Measurement Analysis and Reporting Technique (SMART), the Balanced Scorecard Systems, and Performance Measurement System. These systems use classical evaluation that measure the KPIs and determine the success scores. Next, the total score of the all KPIs accepted as department final success. However, they are not capable of showing overall performance scores, and they ignore department’s differences and do not take KPI’s prioritization issue into consideration [26].

The output of performance evaluation systems (PES) is used as the input of the multi attribute decision-making (MADM) models. To obtain overall performance scores for all departments and a company, various MADM steps implemented. In our study, we have two main requirements, determining of KPI weights and department rankings. KPIs should have exact weight scores. Moreover, department rankings show the success orders and dominancy. In this direction, we searched the literature to satisfy our requirements. As a result, we decided two respective MADM methods to use. Firstly, we use analytical hierarchical process (AHP) because AHP method has a great success to determine rank values according to pairwise comparisons. In this point, performance KPIs considered as attributes in the AHP model to obtain accurate values. On the other hand, we suggest grouping similar KPIs and turning them into a strong critical factor as an attribute of the AHP model. Since fewer attributes give more certain results for AHP method. Secondly, Elimination and Choice Expressing the Reality (ELECTRE) approach applied to rank departments since ELECTRE method based on pair comparison and shows dominancy among departments. At the end, we obtain overall scores that present a reliable and systematic evaluation result. As a result, even if departments have different job missions, overall performance scores give ranks and high-performing or low-performing departments are easily selected. In this paper, we present a case study within a company that uses balance scorecard performance evaluation system to show the implementation of these methods. We show model correctness with the implementing sensitivity analyses and comparisons among other MCDM methods.

With this paper, our major focus is to change performance management viewpoint of the pilot company. We hope other companies will benefit from our proposal and use MADM techniques while they consider performance issues. In the next stages, literature review for MADM methods and related PES presented. After that, AHP and ELECTRE methods introduced to explain the methodology. To illustrate the working mechanism of the developed model, a case study given. Model outputs discussed at the conclusion.

We covered over 30 research published during the previous 15 years in this chapter. We investigate many facets of the problem and synthesize our findings to establish a solution technique. Sehhat et al. [21] propose ways to enhance PES. In the past, financial measurements and divisions were regarded the most essential variables in determining a company's performance, however, these no longer accurately reflect an organization's performance today. According to the writers, all types of KPIs and departments have a critical influence on a business's success. According to Yurdakul and Ic [26], the existing PES are insufficient to demonstrate a department's or company's overall performance. They assert that using MADM techniques can help assure the evaluation's correctness. The majority of research on department ranking or PES are mainly undertaken in the financial sector. Onder and Hepsen [11] conduct a comparative analysis of two MADM techniques, AHP and TOPSIS. The authors classify 17 banks into three categories (state, private, and international) and utilize the AHP technique to calculate the realized weight of qualities for the TOPSIS strategy. Similarly, Tunay and Akhisar [23] employ the TOPSIS technique to order banks according to the weights assigned to AHP’s criterion and sub-criteria in their work. Percin and Aldalou [13], on the other hand, examine two MADM stages: weighting qualities and rating options. Çağıl [5] uses the ELECTRE technique to conduct a performance study in the finance sector.

Multiple MADM approaches are frequently used to address issues. Rouyendegh et al. [15, 16] demonstrate how AHP and fuzzy TOPSIS approaches may be integrated. Rouyendegh [14] employs fuzzy MADM algorithms to identify high-performing university departments. Although the majority of research employ the AHP and TOPSIS methodologies to tackle ranking issues, Roy [18] developed the ELECTRE method for sorting departments (Benayoun and Bilsberry [2]. Omurbek et al. [10] demonstrate that the AHP-TOPSIS and AHP-ELECTRE approaches are equivalent. Ozdemir [12] solves the supplier selection problem using fuzzy AHP and ELECTRE algorithms. Attribute weights are determined at the first stage and then ranks calculated for the alternatives using ELECTRE method at the second step.

Bouri et al. [4] applies both ELECTRE and PROMETHEE methods for the portfolio selection. Additionally, Rouyendegh and Erkan [17] employ the fuzzy ELECTRE methodology as an MCDM method for selecting academic personnel. Scarelli and Narula [20] appoint referees to football matches using the ELECTRE technique.

Pérez-Domínguez et al. [6] conducted a study on lean manufacturing performance. They noted that businesses employ lean manufacturing but must evaluate its effectiveness. They addressed this issue through the use of MADM algorithms, the Technique for Order Preference by Similarity to the Ideal Solution (TOPSIS), and hesitant fuzzy linguistic term sets (HFLTS).

Şenel et al. [22] conducted more research on PES. They advocate for a comprehensive performance management methodology. Additionally, they employed AHP to score critical performance metrics. Following that, they rate the departments based on their data envelopment analysis. Additionally, they advocate for the clustering of ranking departments.

Table 1 contains a summary of the literature review. Within the table, the materials and studies are summarized per research area. PES and MADM research can be viewed independently or in combination.

Overall, after literature review, we found out that PES should be strengthened using MADM approaches. In this point, AHP is one of the most appropriate method for KPI weight determination. Its primary features are ease of computation, pairwise calculation, and the ability to solicit decision maker input. Additionally, TOPSIS, ELECTRE, and PROMETHEE methodologies, as well as other fuzzy approaches, may be suitable for comparison to identify weak and strong departments. We chose ELECTRE because it provides more straightforward results for each pair of departments. Other MADM techniques can also be used to conduct comparisons in the study.

A case study is used to demonstrate how these strategies are used inside a company that utilizes the balance scorecard performance evaluation system. As a pilot, five upper hierarchical departments selected. There are between 13 and 16 key performance indexes (KPI) in total at the scorecards, but similar metrics are combined and under eight KPIs which are attributes of the model. High number of metrics decrease the comparison strength and KPI importance. As a result, the case study applied to rank 5 department with respect to 8 metrics. We do not have permission to share performance metrics information because of the case company requirements. However, we can say that they are related to Finance, Delivery Performance, Non-Conformity Issues, Customer Relations, and Employee Relations generally. All data taken from the case company performance evaluation specialist, and processed on Microsoft EXCEL.

We propose two step methodologies for performance evaluation in this study: AHP and ELECTRE. We present the methodology for companies that makes classical performance evaluation. Instead of only evaluation of metric results, we propose metric rankings and pair comparison among departments in terms of overall success. It also gives the systematic approach that ensures fair evaluation. In the model AHP determine the weight value of performance metrics and then ELECTRE ensures the ranking departments in terms of dominancy relationship.

We indicate a shift in managerial viewpoint in order for decision makers to perceive relative strengths and weaknesses. On the other hand, because these MADM procedures are extremely beneficial for decision makers, we expect that this study will result in an increase in the usage of MADM by companies, particularly case companies.

Sensitivity analyses demonstrate that the model provides showing department’s performance and also their weakness and strengths. Even if they focus on one low-performing KPI, they can replace the department ranks. Additionally, establishing weight values results in a more rational evaluation, as each KPI does not contribute equally to the company's performance.

We used the Classic, TOPSIS, and PROMETHEE techniques in place of the ELECTRE method in the second stage and saw that the ranking results were consistent with the recommended model. It demonstrates the model's accuracy. However, we test these models with a small number of departments in order to demonstrate the model's usefulness.

As a consequence of the study, it can be concluded that the proposed model is suitable for companies with comparable job-focused divisions or departments, such as banks or service organizations. However, when departments have distinct job descriptions, performance review is necessary to ensure that inputs and outputs are comparable. As a result, data envelopment analysis may be utilized to achieve a more precise and equitable solution. On the other hand, the model is only applicable to a limited number of departments due to case limitations. As a result, it may be prudent to examine a large variety of departments. For the future studies, developed model will be applied for bigger cases. A decision support system will be built to aid in the usage of the model by the decision maker.