A systematic literature review of studies on business process modeling quality

Abstract

Context

Business process modeling is an essential part of understanding and redesigning the activities that a typical enterprise uses to achieve its business goals. The quality of a business process model has a significant impact on the development of any enterprise and IT support for that process.

Objective

Since the insights on what constitutes modeling quality are constantly evolving, it is unclear whether research on business process modeling quality already covers all major aspects of modeling quality. Therefore, the objective of this research is to determine the state of the art on business process modeling quality: What aspects of process modeling quality have been addressed until now and which gaps remain to be covered?

Method

We performed a systematic literature review of peer reviewed articles as published between 2000 and August 2013 on business process modeling quality. To analyze the contributions of the papers we use the Formal Concept Analysis technique.

Results

We found 72 studies addressing quality aspects of business process models. These studies were classified into different dimensions: addressed model quality type, research goal, research method, and type of research result. Our findings suggest that there is no generally accepted framework of model quality types. Most research focuses on empirical and pragmatic quality aspects, specifically with respect to improving the understandability or readability of models. Among the various research methods, experimentation is the most popular one. The results from published research most often take the form of intangible knowledge.

Conclusion

We believe there is a lack of an encompassing and generally accepted definition of business process modeling quality. This evidences the need for the development of a broader quality framework capable of dealing with the different aspects of business process modeling quality. Different dimensions of business process quality and of the process of modeling still require further research.

Introduction

Business process modeling is arguably one of the important domains of interest of information systems research over the past three decades. From an Enterprise Modeling perspective, business process modeling is valued as a complement to domain modeling. It allows capturing the organizational dimension in terms of actors, activities, and workflows. Business process models (or process models for short) are required as a basis for knowledge transfer, quality purposes, regulations, communication between internal and external collaborative partners, and documentation in general [1]. Business process models also play an important role in the requirements engineering process of software systems development. As-is models help to understand the work that needs to be supported by information systems. Designing high quality to-be business process models is a prerequisite for leveraging the benefits of process improvement and it is crucial for the design of information systems [2]. Good process model design can help to avoid errors right from the start. This is vital, since the cost of errors increases exponentially over the development lifecycle [3]. All of these factors explain why quality assurance of business process models has been recognized as an important factor for modeling success at an enterprise level over the past years. Therefore, business process modeling is an essential part of understanding and redesigning the activities a typical enterprise uses to achieve its business goals. Moreover, the quality of business process models will impact on the quality of (the design of) information systems and on envisaged business process improvements.

Business process models are not always of high quality. Various studies have shown that many business process models contain errors, such as syntactical mistakes (for an overview see [4]). There is clearly a need to offer guidelines to practitioners on how models of high quality are to be created [5], [6].

Several research works on business process modeling quality have contributed to the evolution of the knowledge in this area. At the same time, research on conceptual modeling (CM) quality has evolved. An example of this evolution is the successive creation of quality frameworks [5], [7], [8], [9], [10] with an increasing number of quality dimensions. Since process models can also be seen as conceptual models these two research areas are very close. Nonetheless, only a small amount of research works have contributed to the interrelation between business process modeling quality and CM quality frameworks (see for example [11], [12]). Besides, early research on business process modeling quality is not always based on recent advances in the understanding of CM dimensions. Because of this, guidelines for good process models are unorganized and dispersed in different papers (e.g. [5], [6], [13], [14], [15], [16], [17], [18]), and there is no view on how complete this set of guidelines is.

In order to advance the field on modeling quality it is useful to determine its current state of the art by identifying, evaluating and interpreting relevant research to date that is related to business process modeling quality.

A search for literature reviews yielded a number of reviews that have been performed in the business process modeling area. As far as we are aware of, no systematic review on the topic of business process modeling quality has been performed yet. O’Neill and Sohal [19] performed a review on business process reengineering. Biazzo [20] highlighted and compared alternative techniques and approaches for business process analysis. Aguilar Saven [21] reviewed and described the main process modeling techniques. Genero et al. presented a literature review on the quality of UML models [22]. Aldin and de Cesare [23] made a literature review on business process models reusability. Sanchez-Gonzalez et al. [24] analyzed the current state of the art and trends with regard to business process model metrics. They performed a literature review on this subject covering the period from 1998 until 2008. Another summary of related work on metrics can be found in Mendling’s book: Metrics for Process Models [25]. None of the above studies provide a clear overview of the state of the art on business process modeling quality.

Given the absence of a literature overview on business process modeling quality, the goal of the research presented here is to perform a systematic literature review (SLR) of papers dealing with business process modeling quality based on the original SLR guidelines as proposed by Kitchenham in [26]. In particular, the goal of this SLR is to provide an inventory of “what has been done” in previous years in the context of quality guidelines for business process modeling. We thus focus on (i) papers that aim to evaluate or improve the business process models quality as a product by proposing practical modeling artifacts (i.e. quality metrics, pragmatic guidelines) and (ii) papers that contribute to the improvement of business process modeling as a process (i.e. enhanced methods for the process of business process modeling). In an attempt to bring about a closer alignment between business process modeling quality and CM quality, we will use a CM quality framework for the interpretation of quality concepts addressed by business process modeling quality papers (i.e. types of quality issues addressed by researchers).

An SLR involves three main activities: planning the review, conducting the review, and reporting the review [26]. Each activity has several stages associated to it. Planning the review includes the specification of the research questions and the development of the review protocol. Conducting the review includes study selection, data extraction, and data synthesis. Finally, the stages associated with reporting the review are mainly concerned with the presentation and interpretation of the results. As Da Silva [27] pointed out, some of the above mentioned activities in an SLR require decisions about possibly conflicting situations. Each disagreement in this SLR was resolved by seeking a consensus between all four researchers. Particularly, for the study selection and the reliability of inclusion decisions, one author of the paper made a list of included/excluded papers from the initial set of papers that resulted from the systematic search process (see below). Subsequently, the list of included and excluded papers was discussed with the three other authors.

The rest of the paper is structured as follows. Section 2 presents the research questions that will drive the entire systematic review methodology. Section 3 discusses the strategy used to search for primary studies and the study selection criteria. Section 4 presents the extracted data of our SLR. Section 5 presents an analysis of the findings and directions for future research. Finally, Section 6 presents conclusions.