Elsevier

Knowledge-Based Systems

Volume 168, 15 March 2019, Pages 70-79
Knowledge-Based Systems

An improvement to swing techniques for elicitation in MCDM methods

https://doi.org/10.1016/j.knosys.2019.01.001Get rights and content

Abstract

Several approaches that utilise various questioning procedures to elicit criteria weights exist, ranging from direct rating and point allocation to more elaborate methods. However, decision makers often find it difficult to understand how these methods work and how they should be comprehended. This article discusses the SWING family of elicitation techniques and suggests a refined method: the P-SWING method. Based on this, we provide an integrated framework for elicitation, modelling and evaluation of multi-criteria decision problems.

Introduction

Although promising from a decision-theoretical perspective, formal and semi-formal decision methods such as multi-criteria decision methods (MCDM) remain rather uncommon in real-life decision modelling and analyses. This seems to owe at least to some extent to perceived difficulties in understanding the decision models available. In particular, there exist several methods and approaches designed to elicit criteria weights that utilise various questioning procedures, ranging from direct rating and point allocation to more elaborate methods. Numerous methods use trade-offs in a structured manner, with significant effects for actual decision-making. However, decision makers continue to find it difficult to understand their own preferences and how these correspond to the elicitation methods used for this purpose. Furthermore, most decision information is imprecise, rendering many prevalent decision tools inappropriate in the sense that they cannot inherently represent uncertainties. Some decision methods allow for the modelling of imprecision, in particular ordinal rankings and interval approaches (both for criteria weights and values), with the aim of avoiding unrealistic, overprecise or even meaningless statements, and instead only demanding information that the decision maker is able to express with confidence. Many MCDM researchers have thus argued that unreasonable exactness is counterproductive and that other means are necessary. Preference rankings appear to constitute one of the most commonly used means in this regard.1

There are consequently a multitude of approaches to express preference intensities, such as the MACBETH method [4], ranking using the delta-ROC (Rank Order Centroid) approach [5], or more simplified methods such as Simos’ method and varieties [6]. The Smart Swaps methods also exist [7], while [8] combine various techniques in the GMAA system. Elicitations are based on attribute trade-offs or by directly assign weight intervals. These relaxations of precise judgments are understood to model decision problems more realistically (see e.g. [9], [10]). However, solutions to such problems are sometimes hard to find and the results can be difficult to interpret. Numerous suggestions have also been made over the years, based on (for example) sets of probability measures, upper and lower probabilities, interval probabilities and utilities [11], fuzzy measures [12], [13], [14] and evidence and possibility theory (cf., e.g. [15], [16], [17]). There are also approaches based on second-order techniques [18], [19]. Other approaches modify some classical decision rules, such as the central value rule based on the midpoint of the range of possible performances (cf. [5], [20], [21], [22]). Salo and Hämäläinen [23] have suggested a set of approaches for handling imprecise information in these contexts, such as the PRIME method for preference ratios, while the SMART method has also been implemented in software (see e.g. [7]). Nevertheless, these approaches exhibit various difficulties, including combining both interval and qualitative estimates with weighted decision rules but without introducing very rough evaluation measures such as Γ-maximin or (Levi’s) E-admissibility (cf., e.g. [24]). Greco et al. [25] suggest UTAGMS for a purpose similar to this paper (which uses an ordinal regression technique), generating a representation extracted from pairwise comparisons even when ordering is incomplete. Figueira et al. [26] generalise this by taking cardinalities into account in order to obtain a class of total preference functions compatible with user assessments, restricting the polytope in various respects. For our purposes, this is less suitable because it is unclear how it can be extended when other types of information (such as interval constraints) also exist, resulting in computational issues as explained in, for instance, Danielson and Ekenberg [18]. Furthermore, in many cases the structural constraints can be represented by second-order information [27], which provides further information that should be handled. Hence, our representation is in such respects more appropriate to the purpose of this paper, as explained below. In any case, the formalism suggested is by no means the only possibility, and should instead be considered an example (as well as being the foundation for the computer tool used below).

One of the most important problems in many MCDM methods is the handling of trade-off effects between the value scales of different criteria. Trade-off methods are quite useful, but given the number of judgements required of the decision maker they can also be very demanding and sometimes intractable. For example, Fischer [28] highlights that trade-off methods tend to give greater weight to the most important attribute. One prominent family of methods addressing this and other problems is SWING weighting [29]. As an example, the popular SMART family of MCDM methods was extended with SWING trade-offs, yielding the SMARTS method [30].

This article suggests a refined method – the P-SWING method – in an attempt to overcome some of the typical problems associated with elicitation. The method consists of an amended swing-type technique at its core. However, whereas a traditional SWING session only contains from-worst-to-best swings, the suggested method adheres to the core ideas while allowing for intermediate comparisons as well. This will aid the convergence of the weights for the criteria. Furthermore, there is no use of zero alternatives or similar synthetic constructs, and instead many more available real data points are utilised. Based on this, we provide an integrated framework for elicitation, modelling and evaluation of multi-criteria decision problems.

The following section describes an experiment to compare different MCDM methods in which some problems with SWING techniques were detected as side effects, and subsequently explored alongside remedies via focus groups. In Section 3, we formalise these remedies into an extended method for criteria weight elicitation with improved precision, called P-SWING (Partial SWING). Section 4 describes how P-SWING is integrated into a framework for elicitation, modelling and evaluation of multi-criteria decision problems. Sections 5 Example of P-SWING evaluation process and use, 6 Comparison then describe in detail how the framework is used in practice, in order to demonstrate its advantages. Finally, Section 7 concludes the paper.

Section snippets

MCDM methods

In order to investigate how some popular classes of MCDM methods are perceived and used in real-life decision making situations, we conducted a study involving 100 people making one large real-life decision each [31]. A requirement was that such a decision was important, not obvious to the decision maker, and required substantial information collection in advance. The decisions included selecting a country or area in which to live, choosing a university programme and buying an apartment. The

P-SWING

Modelling realistic decision problems often results in numerically imprecise and vague sentences, such as “the value of alternative A1 under criterion C1 is greater than 40%” or comparative sentences such as “the value of alternative A1 under criterion C1 is preferred to the value of alternative A2 under criterion C1.” Such sentences are easily translated into a numerical format. In the interval case, the translation is of the format vij [a1, b1], i.e. the two linear inequalities vija1 and b1

Evaluation

The evaluation process is uncomplicated to perform. Assume a standard MCDM method that seeks to evaluate each alternative, yielding a most representative point (MR-point5

Example of P-SWING evaluation process and use

Consider a procurement process in which a large organisation is looking for a new office space, as its existing space has become less adequate. The decision situation is to select a space from four real estate developers, A, B, C, and D, in order to realise this project. The criteria emphasised in the selection process are functionality (basically the degree of adequacy of the new premises), localisation (geographical and infrastructural), opportunities for interaction with the surrounding

Comparison

The proposed method can be compared and validated in two steps. In the first step, the proposed P-SWING method is compared to the same decision analytical method without P-SWING, and in the second step, the latter is compared with other well-known methods such as SMART and AHP. The P-SWING method was conceptually validated in focus group discussions, where the inadequacy of standard SWING and non-SWING methods were discussed. Both the conceptual functionality and the actual process implied by

Conclusions

The elicitation methods that are today available in MCDM are often too cognitively demanding for normal real-life decision makers, and there is a clear need for weighting methods that do not require formal decision analysis knowledge. The SMART method and SWING weighting (in their varieties) are highly beneficial for actual decision-making, in spite of the fact that they are occasionally difficult to understand. Following experiments with 139 participants, we advise against the use of pure

Acknowledgements

This research was funded by the Swedish Research Council FORMAS, project number 2011-3313-20412-31, as well as by strategic funds from the Swedish government within ICT — The Next Generation.

References (42)

  • FigueiraJ.R. et al.

    Building a set of additive value functions representing a reference preorder and intensities of preference: GRIP method

    European J. Oper. Res.

    (2009)
  • EkenbergL. et al.

    Value differences using second order distributions

    Internat. J. Approx. Reason.

    (2005)
  • FischerG.W.

    Range sensitivity of attribute weights in multiattribute value models

    Organ. Behav. Hum. Decis. Process.

    (1995)
  • EdwardsW. et al.

    SMARTS and SMARTER: improved simple methods for multiattribute utility measurement

    Organ. Behav. Hum. Decis. Process.

    (1994)
  • DanielsonM. et al.

    Distribution of belief in decision trees

    Internat. J. Approx. Reason.

    (2007)
  • ChenS.M. et al.

    Multicriteria linguistic decision making based on hesitant fuzzy linguistic term sets and the aggregation of fuzzy sets

    Inform. Sci.

    (2014)
  • RiabackeM. et al.

    State-of-the-art in prescriptive weight elicitation

    Adv. Decis. Sci.

    (2012)
  • DanielsonM. et al.

    Weighting under ambiguous preferences and imprecise differences in a cardinal rank ordering process

    Int. J. Comput. Intell. Syst.

    (2014)
  • SarabandoP. et al.

    Multi-attribute choice with ordinal information: a comparison of different decision rules

    IEEE Trans. Syst. Man Cybern. Part A

    (2009)
  • MustajokiJ. et al.

    A preference programming approach to make the even swaps method even easier

    Decis. Anal.

    (2005)
  • LarssonA. et al.

    Cardinal and rank ordering of criteria – addressing prescription within weight elicitation

    Int. J. Inf. Technol. Decis. Mak.

    (2015)
  • Cited by (36)

    • A modified CRITIC with a reference point based on fuzzy logic and hamming distance

      2022, Knowledge-Based Systems
      Citation Excerpt :

      In the subjective approach, the expert provides the criteria weight based on a managerial hunch and tacit knowledge [10]. Some examples of subjective weighting methods are pairwise-comparison-based methods [11], P-SWING [12], SWARA [13], just to name a few. The Entropy based method [14,15] is one of the most used objective methods for criteria ranking along with CRITIC method [1].

    • Evaluating community question-answering websites using interval-valued intuitionistic fuzzy DANP and TODIM methods

      2021, Applied Soft Computing
      Citation Excerpt :

      Table 2 summarizes recent papers investigating the application of the MCDM model under IVIF environments. Considering the bias of single users caused by differences in backgrounds, a group of users can provide a more objective evaluation [29]. To consider the ratings given by multiple decision makers from various perspectives, a novel MCDM approach based on the DANP-TODIM in the IVIF environment is proposed.

    View all citing articles on Scopus
    View full text