An Update for Taxonomy Designers

Taxonomies are fundamental in research and practice, which is why taxonomy design has been investigated in various disciplines such as natural sciences (Sokal and Sneath 1963), social sciences (Bailey 1994), organizational science (McKelvey 1975), and strategic management (Hambrick 1984). As a well-accepted method in terms of citations, Bailey (1994) provided a sound overview of and guidance on how to develop taxonomies in social science. Bailey differentiates between the conceptual (i.e., deducing taxonomic structure from a theoretical foundation), the empirical (i.e., grouping inductively via statistical methods), and the operational (i.e., mapping both conceptual and empirical levels) approach. In computer science, Bayona-Oré et al. (2014) proposed a method with 24 activities along five phases (i.e., planning, identification and extraction of information, design and construction, testing and validation, and deployment) to develop taxonomies in software engineering. Thereby, they highlighted demands for clearly formulating a user problem that should be addressed by means of a taxonomy as well as for testing and using a taxonomy to obtain insights concerning its practical usefulness. Usman et al. (2017) refined Bayona-Oré et al. (2014) method and proposed 13 activities ranging from the selection of a classification procedure type to the identification of top-level dimensions and validation through benchmarking. Next to these general activities, other researchers present more specific guidelines for testing taxonomies with conceptual and analytical models (Doty and Glick 1994) or for typical taxonomy building issues (Hambrick 1984). Additional guidance focuses on fields such as requirements-driven taxonomies (Notheisen et al. 2019) or business reporting taxonomies (Ojala et al. 2018). A detailed overview of related guidance on taxonomy design can be found in Appendix 1 (available online via http://​link.​springer.​com).

While other disciplines extensively reflected on taxonomy design, taxonomies in IS research have often been built rather intuitively, for instance by applying methods such as clustering (Posey et al. 2017), qualitative content analysis (Goo et al. 2000), or deductive reasoning (Al-Debei and Avison 2010). Nickerson et al. (2013) provided the first – and so far the only – well-conceived taxonomy development method for the IS discipline. This is important as IS research is concerned with emerging and rapidly evolving technologies, and thus needs a foundation for understanding and analyzing socio-technical phenomena. Nickerson et al. (2013) method comprises seven steps, combining both an inductive and a deductive approach to build taxonomies iteratively. It starts with the determination of a meta-characteristic that is derived from the purpose and target users of the taxonomy. Next, the objective (i.e., taxonomy is valid) and subjective (i.e., taxonomy is useful) ending conditions help to determine when the iterative method can be terminated. Afterward, researchers have to decide on the approach to be adopted for the first/next iteration. If data about real-world objects are available, researchers are advised to follow an inductive approach (i.e., empirical-to-conceptual). If a significant understanding of the phenomenon in focus already exists, a deductive approach (i.e., conceptual-to-empirical) should be employed. The taxonomy development process continues with the next iteration until all ending conditions are met (find the original method in Appendix 1).

Despite being the de-facto standard in the IS discipline, some researchers have adapted Nickerson et al. (2013) method. For instance, Mwilu et al. (2015) argued that the ending conditions are “often difficult to apply in practice” (p. 3) and highlighted the need for advancing steps such as the specification of the object sample. In another example, Land et al. (2013) have extended the original method by introducing a library of case studies (i.e., real-life objects) and a testing phase to validate a taxonomy version using (new) cases and stakeholder feedback. In this vein, Sarkintudu et al. (2018) have provided an extension that seeks to refine a taxonomy based on insights gained during the taxonomy’s usage. In some instances, researchers have explicitly raised the need for specific further guidance regarding taxonomy evaluation (e.g., Kazan et al. 2018; Krieger and Drews 2018), which is apparent in statements such as “the main development cycle [for taxonomies] does not include the evaluation step, leaving it open for researchers” (Chasin et al. 2018, p. 299).

Even though first attempts have started to advance and complement existing methodological guidance, Nickerson et al.’s (2013) method is by far the most referenced method in IS that has been served as a blueprint for numerous of taxonomy projects (see status quo in Sect. 4). Moreover, this method is grounded in the literature on taxonomy design in IS and adjacent fields (see seminal work such as Bailey 1994; Doty and Glick 1994) and serves as a starting point for refined and adapted methodological guidance (e.g., Land et al. 2013; Sarkintudu et al. 2018). Against this backdrop, we have decided to build this study primarily upon the seminal work of Nickerson et al. (2013), while considering additional guidance also beyond IS research to advance and extend the taxonomy design process.

Taxonomies help to identify and structure characteristics and dimensions towards describing, understanding, and analyzing phenomena. This applies to both, DSR and non-DSR (such as behavioral science that seeks to develop and test theories, Doty and Glick 1994). This study builds upon previous work by Nickerson and colleagues positioning taxonomies and their design in the DSR paradigm (Nickerson et al. 2013). Given that DSR has the potential to contribute “both practical relevance (via its emphasis on useful artefacts) and scientific rigor (via the formulation of design theories)” (Baskerville et al. 2018, p. 358) this study relies on DSR for the advancement and extension of methodological guidance for taxonomy design.

DSR contributions come in forms as diverse as design artefacts, design theories, and design processes (Baskerville et al. 2018). We see taxonomies as artefacts and consider them – in contrast to real-world phenomena – as artificial objects that solve practical problems (Simon 1996; Hevner et al. 2004) (here identifying and structuring dimensions and characteristics of phenomena). In accordance with the DSR paradigm, this study seeks to provide useful artefacts and not, as common in non-DSR research, to achieve truth or truthlikeness (Hevner and Chatterjee 2010; Goldkuhl 2004) depending on the underlying epistemological and ontological assumptions (Baskerville et al. 2015; Niehaves 2007; Frank 2006).

Generally, four basic artefact types are differentiated in DSR (March and Smith 1995), namely constructs (i.e., concepts and vocabulary of a domain), models (i.e., constructs and their relationship to represent a phenomenon), methods (i.e., processes to perform tasks and achieve goals), and instantiations (i.e., situated implementations that operationalize constructs, models, and methods). Following previous research examining DSR artefacts (e.g., Offermann et al. 2010; Sangupamba et al. 2014), we consider taxonomies as structure-giving artefacts in the form of models for four reasons: First, models “can be viewed simply as a description that is a representation of how things are” (March and Smith 1995, p. 256), which also applies to taxonomies that capture what things are out there (e.g., Iivari 2007). Second, models provide “certain concepts [of a domain] and relationships among them” (March and Smith 1995, p. 256) for understanding phenomena. Taxonomies help researchers to organize knowledge by representing relevant dimensions and corresponding characteristics (i.e., constructs) and thereby reflect relationships among dimensions and characteristics. These relationships are visualized, for instance, through hierarchical structures (e.g., Prat et al. 2015) or multi-layer structures encompassing abstract layers, dimensions, and characteristics (e.g., Janssen et al. 2020). Third, models share three essential properties (borrowed from general model theory as presented by Stachowiak 1973) that are also addressed by taxonomies: Representation – taxonomies represent existing or future, and natural or artificial objects; reduction – taxonomies capture not all attributes of phenomena and instead focus on those attributes that are relevant to a taxonomy’s purpose and target user group; and pragmatism – taxonomies can fulfil various purposes such as describing or analyzing phenomena. Fourth, Nickerson et al. (2013) and other IS taxonomy designers have already positioned their research outcomes as models (e.g., Oberländer et al. 2018; Yang and Varshney 2017).

Positioning taxonomies as models allows taxonomy designers to draw from the rich body of DSR knowledge. As a consequence, the DSR knowledge provides foundations and methodological guidance that help researchers to motivate an artefact’s objective (Peffers et al. 2007) and thereby anchor artefacts in the problem and solution space (vom Brocke et al. 2020). DSR generally differentiates two main activities for building and evaluating artefacts, commonly referred to as the build-evaluate pattern (e.g., Hevner et al. 2004; March and Smith 1995; Sonnenberg and vom Brocke 2012). A need for evaluation arises not only from DSR in general (e.g., Prat et al. 2015; Venable et al. 2016) but also from the application of taxonomies in particular, as taxonomies need to be useful for certain goals such understanding phenomena or making decisions (e.g., De Langhe and Fernbach 2019; Morana et al. 2020). Transferring the build-evaluate pattern to taxonomy design, we observed that further guidance with regards to the evaluation of taxonomies would be helpful, which has been emphasized by Nickerson et al. (2013) who stated that “the resulting taxonomy needs to be evaluated for its usefulness” (p. 346). Rigorously built and evaluated artefacts allow researchers to generalize knowledge (Baskerville and Pries Heje 2019). This is an important aim of DSR projects − including those that intend to design taxonomies – to add design knowledge to the DSR knowledge base (Hevner et al. 2004; Niehaves 2007). Depending on the scope and size, the design of a taxonomy can be a stand-alone DSR project or part of a larger DSR project. Moreover, anchoring taxonomy design under the DSR paradigm allows us to promote the iterative nature of artefact design as well as to organize the design process (ETDP) and the design recommendations (TDR) in a structured way, for example, following the steps of the DSR methodology proposed by Peffers et al. (2007) (see Sect. 5).

To assess the status quo of taxonomy design in IS, we followed the recommendations of Templier and Paré (2018) in a two-phased approach. In the first phase, we aimed at gaining insights into the operationalization of taxonomy design, presentation and evaluation. Given this aim, we searched and screened relevant literature (see Table 1). In the second phase, we analyzed our sample of identified taxonomy articles and extracted relevant data (using coding attributes such as taxonomy design method, presentation form, evaluation method and criteria). On this empirical basis, we synthesized findings and identified ‘good practices’ as transparent and comprehensible operationalizations of taxonomy design steps. We present a summary of our coding results in Table 2 and methodological details on the status quo analysis in Appendix 2.

For deriving the ETDP and the TDR, we particularly drew on examples of transparent and comprehensible taxonomy design, adherence to good practice, and methodological gaps collected during the coding of the sample. Following inductive reasoning (e.g., Hempel 1966), we critically discussed the collected codes (i.e., attributes such as good practices and challenges) to select the most potent attributes in terms of guiding future taxonomy designers. Based on this selection, we advanced and extended the steps of the method proposed by Nickerson et al. (2013) to create the ETDP, formulated TDR, and assigned these recommendations to the steps of the ETDP. The ETDP and the TDR were iteratively (re-)formulated by the author team. Specifically, with regards to further guidance on the evaluation of taxonomies (particularly evaluation goals and ex post evaluation), we built on our sample of taxonomy articles that evaluate taxonomies (56 unique articles) following analytical and abductive reasoning principles (e.g., Gregory and Muntermann 2011; Van de Ven and Johnson 2006). Further, we anchor our guidance in well-established DSR knowledge on artefact evaluation, among others, following the structuring questions of ‘why’, ‘how’, and ‘what’ to evaluate (e.g., March and Smith 1995; Prat et al. 2015; Venable et al. 2016) (see Table 6).

For the evaluation of the ETDP and the TDR, we conducted semi-structured expert interviews (Myers and Newman 2007) that can be used “to confirm what is already known whilst at the same time providing the opportunity for learning” (Recker 2013, p. 91). Our evaluation's goal was to verify the understandability and expected usefulness of the ETDP and the TDR. We first tested our interview procedure and conducted four pre-tests with IS researchers who have already published at least one taxonomy in the conference proceedings of ICIS or ECIS. Once the interview procedure was robust, we started to recruit experts (i.e., authors who have published taxonomy articles in journals listed in the AIS Senior Scholars’ Basket). The participants were provided with the ETDP and the TDR beforehand. Each interview consisted of four parts, i.e., (1) motivation and problem awareness related to taxonomy design, discussions on (2) the ETDP and (3) the TDR, and (4) additional feedback (e.g., assessment of understandability and usefulness). During the interview, one co-author took the role of the interviewer. At least one other co-author was responsible for asking clarifying questions and documenting the interviewee’s responses and questions in the form of a protocol. We analyzed the feedback and revised both the ETDP and the TDR (see Appendix 4 for details on the taxonomy experts and their feedback).

Moreover, to validate our result’s coverage and ensure alignment with the existing body of knowledge, we cross-checked the ETDP and TDR with additional taxonomy design guidance in and beyond IS (see Appendix 1 for a detailed overview of related guidance).

Steps 1 to 3 of the ETDP anchor the taxonomy in the DSR problem and solution space (vom Brocke et al. 2020). Thereby, they support explicating the specific problem that is to be solved using the taxonomy to be designed in a given context. This involves specifying the observed phenomenon (Step 1), the taxonomy’s target user group(s) (Step 2), and the intended purpose(s) (Step 3). Justifying the value of the proposed solution (e.g., Hevner et al. 2004; vom Brocke et al. 2020; vom Brocke and Maedche 2019) helps not only to understand why a taxonomy is important for its intended user group(s), but also allows to emphasize the importance of the phenomenon under consideration. Many researchers face the problem that the constructs underlying a phenomenon are either unknown or known but unstructured. This in turn leads to two typical purposes of taxonomies:

There is an (re-)entry point before Step 2, because later demonstration or evaluation of the taxonomy may reveal that it is only be suitable for parts of the originally defined target group(s) (i.e., specification) or even beyond the originally defined target group(s) suitable (i.e., generalization).

TDR 1 to 3 help clarifying how a taxonomy can be a solution to the problem of an unknown/unstructured phenomenon and how such a taxonomy relates to potentially already existing taxonomies.

Steps 4 and 5 of the ETDP support reflecting upon the taxonomy’s objectives. So far, researchers have followed Nickerson et al. (2013) and determined both the meta-characteristic and the ending conditions. Beyond that, we suggest determining evaluation goal(s) before initiating the taxonomy design to align the problem and the solution space of a taxonomy project at an early stage (e.g., McKay et al. 2012; vom Brocke et al. 2020; vom Brocke and Maedche 2019). This requires not only deciding which angle a taxonomy takes on the phenomenon under consideration (determining meta-characteristic, Step 4), but also when taxonomy building is completed (determining ending conditions) and how to evaluate taxonomies (determining evaluation goal(s), Step 5). This is because the definition of a future solution includes setting the goals not only of the artefact’s building but also of its evaluation (Sonnenberg and vom Brocke 2012). We derived potential evaluation goals from our sample of taxonomy articles following analytical and abductive reasoning principles (e.g., Gregory and Muntermann 2011; Van de Ven and Johnson 2006) (see Sect. 3.2). The evaluation goals comprise, for example, better describing, identifying, classifying, analyzing, and clustering of objects that represent a certain phenomenon compared to doing so without a taxonomy or other classification schemes (see Table 4).

In line with defining the problem space in DSR, the meta-characteristic defines what is relevant for the specific taxonomy design and what is not. Consequently, all the following characteristics and dimensions of the taxonomy must relate to this meta-characteristic. A problem-space chosen in this way must always be considered in its environment (Simon 1996). A taxonomy’s environment is shaped by the target user group(s), purpose(s) and evaluation goal(s) which in turn should be aligned to each other right from the beginning of the taxonomy design process. Therefore, we stress the importance of defining the meta-characteristic, bearing in mind that it can be further refined after the first iteration(s), as it is impossible to robustly determine it at this stage – even more so in the case of emerging phenomena. Further, there is no ‘right’ meta-characteristic since phenomena can be viewed from different angles. Rather it is important to explicate the specific angle chosen. To allow refining the meta-characteristic during taxonomy design, the ETDP provides a (re-)entry point before Step 4.

TDR 4 to 6 help determine and refine the meta-characteristic that defines the objective of the taxonomy design as the specific angle from which the phenomenon is perceived (see Fig. 2).

Steps 6 to 10 of the ETDP support building the taxonomy. We build on the corresponding steps as proposed by Nickerson et al. (2013) and carefully refine a selected part. In this way, we emphasize the idea of combining inductive and deductive approaches for designing taxonomies (Steps 6 to 9). As the creation/revision of taxonomies is often not made explicit, we constitute a new, separate step in which we offer researchers pre-defined taxonomy operations (Step 10). By defining a taxonomy operation as a concrete change made to a taxonomy following an iteration, we thus allow for a taxonomy being altered without the need to re-examine existing or examine new objects (e.g., through renaming or re-structuring characteristics and/or dimensions after the evaluation). Taxonomy operations can change either one or more characteristics and/or one or more (sub-) dimensions. We combine previous work on taxonomy operations (Mwilu et al. 2015) and ‘CRUD functions’ from computer science (Martin 1983) to conceptualize a set of taxonomy operations which includes adding, updating, and deleting characteristics and dimensions (see Table 5). These operations support creating, revising, and documenting taxonomies in a more consistent and transparent way. Since taxonomy design is iterative without knowing a priori the kind and number of iterations as well as whether objects need to be investigated or not, the ETDP provides (re-)entry points before Step 6 and 10.

TDR 7 to 16 help taxonomy designers to decide on the number of iterations required for taxonomy building, which approach to adopt, and when (i.e., conceptual-to-empirical or empirical-to-conceptual) as well as how to implement the individual iterations for iteratively creating and revising the taxonomy.

Steps 11 and 12 of the ETDP support demonstrating that a taxonomy is formally valid and satisfies the definition of a taxonomy independently of its purpose(s) and target user group(s). The objective ending conditions proposed by Nickerson et al. (2013) help to demonstrate whether a taxonomy meets the essential criteria for a taxonomy. This involves objectively verifying whether a taxonomy is mutually exclusive and collectively exhaustive. TDR 17 to 19 help demonstrating the validity of the taxonomy.

Steps 13 and 14 of the ETDP support initiating the evaluation of a taxonomy while considering the taxonomy’s purpose(s) and targeted user group(s). Here, researchers have to evaluate whether the taxonomy is applicable and useful for providing structure to the phenomenon under consideration. The subjective ending conditions proposed by Nickerson et al. (2013) collectively constitute the necessary condition of an ex ante evaluation during the taxonomy building iterations (Nickerson et al. 2013; Szopinski et al. 2019b). This comes down to subjectively verifying that a taxonomy is concise, robust, comprehensive, extendible, and explanatory. Our status quo analysis shows that subjective ending conditions are often reviewed by reaching a consensus among the co-authors of an article as to whether the subjective ending conditions have been met (i.e., whether the taxonomy is perceived applicable). Apart from this being a necessary condition, the rigorous evaluation of a taxonomy requires that it also meets sufficient conditions after the building process has been terminated (i.e., ex post evaluation) (Nickerson et al. 2013; Prat et al. 2015; Sonnenberg and vom Brocke 2012).

Steps 15 and 16 of the ETDP support assessing the sufficient conditions of taxonomy evaluation upon completion of Nickerson et al.’s (2013) taxonomy development method (i.e., that ends/restarts after Step 14) that depends on the taxonomy’s evaluation goal(s), purpose(s) and targeted user group(s). This implies adequately configuring an evaluation (Step 15) and subsequently performing it (Step 16). To observe and measure how well the taxonomy supports target user group(s) in achieving the intended purpose(s) (Step 17), researchers need to configure and perform an ex post evaluation in light of the evaluation goal(s). In DSR, evaluation is critical and should challenge the usefulness of the artefact (Hevner et al. 2004; Prat et al. 2015; Venable et al. 2016), here the usefulness of taxonomies. In addition, the evaluation in DSR depends on the type of artefact which is reflected in the evaluation methods (Cleven et al. 2009; Hevner et al. 2004; Prat et al. 2015) and evaluation criteria (March and Smith 1995; Prat et al. 2015; Sonnenberg and vom Brocke 2012) for a given artefact type. As we consider taxonomies as models, from the DSR perspective, the evaluation of a taxonomy is the evaluation of a model. Furthermore, taxonomies are usually reported together with definitions or descriptions of the taxonomies’ dimensions and characteristics (i.e., constructs of the model). Consequently, the evaluation of taxonomies also involves the evaluation of the constructs. The configuration and performance of taxonomy evaluation require taxonomy-specific answers to the ‘why’, ‘how’, and ‘what’ of DSR evaluation (Prat et al. 2015; Venable et al. 2016). In Table 6 we provide an integrated overview of the steps from demonstration to evaluation along these three guiding questions. For more operational guidance, we describe in Appendix 3 commonly used, taxonomy-related evaluation methods and evaluation criteria. In this way, the ETDP provides an evaluation trajectory starting from ex ante evaluation by checking the necessary (Is it a taxonomy?) and sufficient condition (Is it an applicable taxonomy?) to an ex post evaluation (Is it a useful taxonomy?).

In summary, taxonomy evaluation involves configuring a triad of evaluation methods, evaluation criteria, and target values/thresholds in light of the taxonomy’s purpose(s). It further includes performing the evaluation with partners relevant to or recruited from the target user group(s) (e.g., Sun and Kantor 2006). This triad allows investigating whether the evaluation goal(s) have been met and whether the actual use of the taxonomy enables the structuring of a phenomenon of interest (i.e., the taxonomy is useful). Bearing the iterative nature of taxonomy design in mind, it is important to note that we have complemented the additional guidance for the ex post evaluation of taxonomies with another exit point (after Step 17) for (re-entering) and reiterating into the taxonomy building before Step 2, 4, 6 or 10. For example, when the ex post evaluation shows that the taxonomy is not useful and does not meet the evaluation goal(s).

TDR 20 to 23 help configuring and performing an evaluation that fits the taxonomy’s target user group(s), purpose(s) and evaluation goal(s).

Step 18 of the ETDP supports the reporting of both the process of designing a taxonomy and the resulting design product (i.e., the taxonomy). Furthermore, reporting a taxonomy involves providing visualizations that fit the purpose(s) and target user group(s) (Szopinski et al. 2020) as well as descriptions for each characteristic and dimension. Beside this taxonomy-specific communication, it is also important to consider communication that is specific to the phenomenon under consideration (e.g., Hevner et al. 2004). By doing so, researchers can ensure that their taxonomies serve as a structure-giving artefact for the relevant target user group(s) and towards understanding certain phenomena in domains of interest.

TDR 24 to 26 help communicating the taxonomy and making it accessible to the target user group(s), for example, for practitioners who wish to use the taxonomy for decision-making or for researchers who seek to build their research upon the taxonomy.