Selecting a process variant modeling approach: guidelines and application

This study aims to comparatively evaluate process variant modeling approaches to support practitioners in selecting suitable approaches. The action research method involves researchers and practitioners working on a problem together, developing solutions, and achieving reflective learning [15]. In this way, researchers may obtain first-hand understanding of the phenomenon, while providing value to the participant organization [16, 17]. It is extensively used in information systems research [18, 19], and found to be a promising method specifically for the BPM field [20]. Action research is suggested for cases where a demonstration or evaluation of an artifact is necessary [20]. For these characteristics, we chose to apply action research in this study. Action research fits particularly well to our research aim, because it enables us to acquire deep knowledge for advancing scientific knowledge while solving practical problems [21, 22]. To conduct the action research method for comparatively applying and evaluating process variant modeling approaches, we identified a company in need of process variant modeling as our case. In applying intensive qualitative research methods that examine a phenomenon, such as case study and action research, it is common to perform the study on a single case with carefully-identified units of analysis (in this study, variant modeling approaches) [23‐25]. This enables the researchers to get a deeper understanding of the subject and produce richer results. We followed the approach of participatory action research (PAR), which involves practitioners participating in research as both subjects and co-researchers [26]. We followed the cyclical process of action research [17, 23]. As a result of the first stage in the cycle, diagnosis of the problem, user needs are specified (Sect. 2.2) and requirements for selecting the process variant approaches are determined (Sect. 3.1). The rest of Sect. 3 together with Sects. 4 and 5 report on the stages of action planning and action taking. Lastly, the application of evaluation and specifying learning stages are described in Sect. 6. We performed two iterations of the action research cycle, as explained in Sect. 2.3. We applied PAR in an experimental format, where the researchers and the company collaborated in all phases to take action and evaluate the consequences [23]. In the next section, we introduce the company selected as the case.

We chose 4S for this action research as we identified that the needs of the company suit well with the aim of our research. 4S is a consultancy company that provides process analysis, improvement, and automation services to its customers using the HP PPM product [11]. HP PPM provides a flexible process definition and execution environment with a special emphasis on project and demand management process areas. Other process areas 4S works on are change, risk, and issue management. 4S has customers from various countries and industries, which are active in different domains. 4S maintains a process model repository for each customer that contain process models in a hierarchical structure. Due to the fact that 4S provides consultancy services in certain process areas, it has repositories containing multiple variants of the same process. Usually, 4S analysts need to rely on their own expertise to examine and maintain these repositories. Moreover, for new customers, they cannot systematically exploit process knowledge obtained from similar companies that the analysts worked with previously. We identified four user needs of 4S which together motivate the use of process variant modeling as a suitable solution:

In the following section, we explain our research plan for performing the action research in the selected company.

A team of six employees was established in 4S to perform the study. One of the authors of this paper, who is affiliated with 4S, worked as a leader of the team in the company that identified the requirements of 4S, implemented the selected approaches, and evaluated the results. The rest of the authors also supported the team throughout the study. Thus, the team was composed of one of the researchers who led the study in 4S, five other 4S analysts, and the rest of the researchers that provided knowledge on variant modeling approaches and how to apply them. In this way, we established an action research setting where we aimed “to solve current practical problems while expanding scientific knowledge” [17].

The team decided to use Software Project Management (SPM) processes of five customers for this study. Four of the companies were from Turkey, and the other was a Turkish branch of an international company. For all of them, their SPM processes were defined based on PMIs PMBOK guide [33]. Although 4S used the PMBOK guide as the baseline, the best practices provided just the essential steps of a project management process. The SPM process was selected due to its business value for 4S, since it was the most frequent process that 4S works with its customers and it had the highest variation. 4S used the HP PPM tool to define their processes as workflow definitions. The analysts were experienced in using BPMN process models to analyze and communicate processes. The team used available process models and workflow definitions in the company to analyze and discuss the process variants through the application of the two selected approaches. A summary of the selected companies and process metrics for the initially available SPM processes can be found in Table 1. We planned to execute the following steps to fulfill the aim of this research in conformance with the action research cycle [17]:The team performed the action research cycle in two iterations. In the first iteration, which was reported in [14], all the steps were performed for the top-level SPM process. In the second iteration, which is reported in this paper, user needs were revised, the steps 1 and 4 were repeated for a subprocess, and steps 5 and 6 were performed to integrate the new findings with the previous ones. The detailed analysis in the second iteration resulted in refinements in some of the outputs of the first iteration, which are visible in Table 3 and Figs. 3, 5, 6 and 7.

In the next section, we describe the selection procedure for process variant modeling approaches as the third step of our action research plan.

Next to the aforementioned user needs, also the company and the problem domain pose requirements on the process variant modeling approaches. Within this section, we build upon the work of La Rosa et al. [5]. The team identified these requirements based on their process variant modeling needs and organizational characteristics. In the following requirement list, we map each requirement on the evaluation criteria postulated by La Rosa et al. indicated in parenthesis at the end of the requirement text, if applicable.

To better position the requirements, we have subdivided them into three sets. In the first set, we have the requirements specific for the problem domain, i.e., another company within the same domain would have the same requirements. The second set consists of the requirements specific for 4S, i.e., these are determined by the focus of 4S onto particular solution directions. The third and final set consists of requirements specific for the 4S employees, i.e., these requirements are based on the experience of the employees of 4S.

To identify the potential process variant modeling approaches that meet the requirements as identified in Sect. 3.1, a list of approaches were identified from the most recent surveys and systematic literature reviews [5, 9, 10]. Next to this, two recent theses were examined on the topic of variability to ensure that the list is up-to-date [36, 37].

In total, there are 240 publications listed in [5, 9, 10, 36, 37]. After removal of publications in non English, and publications that could not be obtained, we have a total of 238 publications. Some publications were listed multiple times. After removal of the duplicates, we had a total of 160 publications.

For each of these 160 publications, we have read the title, abstract, and keywords. Based on this, we determined whether this work might be relevant for our setting, e.g., some work was related to run-time flexibility, similarity in process models, or correctness of process models. This resulted in 69 publications. Each of these publications has been read to determine the relevance; which resulted in 43 publications being relevant.

We decided to first differentiate the publications based on the used modeling formalism (Req. 6) since this is easiest to verify for a given publications. In total 20 publications contained among others BPMN, e.g., ADOM [27] has been applied to i.a. UML Activity Diagrams, and BPMN. Using, among others, the work of La Rosa [5], our second differentiator was the validation in real life (Req. 4). If [5] indicated that there has been a validation, then we have followed this. If [5] did not indicate there has been a validation, then we have searched for papers possibly containing the validation since a validation paper could have been published after the work of [5]. This resulted in 10 publications related to four approaches, e.g., Provop [13] was listed with 5 different publications. The four approaches are as follows:Note that, among others, the ideas for C-EPCs can theoretically be transferred to other formalisms, e.g., C-BPMN (see [37] for a discussion). From 4S, there was a requirement (Req. 4) that the approach has a real-life evaluation. Furthermore, applying an evaluated approach on a different formalism does not necessarily mean that the evaluation will also hold for the other formalism. As a result, 4S did not want to consider approaches that can theoretically be applied upon other process modeling formalisms; even though, from an academic perspective, unevaluated approaches might be more interesting. We present the list of evaluated approaches and our comparison criteria in Table 2.

Within Table 2, we use \(+\) if an approach supports a requirement, − if it is not supported, and ± if there is only partial support, e.g., the concepts required for a requirement are listed, but it is not described how to use these concepts.

The evaluation in Table 2 follows the insights as presented by La Rosa et al. [5]. On the requirements not covered by La Rosa et al., we present the reasoning behind the scores. For ADOM [27], we have a “\(+\)” for domain independence since the approach does not make any assumptions on the application domain. As the team chose approaches that use BPMN as the process modeling language (Req. 6), hierarchy is natively supported through the decomposition of activities to subprocesses. However, the hierarchy support provided by BPMN may not be sufficient for handling variabilities, and additional mechanisms may be needed to manage variability within a hierarchical structure [40]. ADOM gets a “±” for hierarchy support since a subprocess is defined as a dependent element as part of the method, but no specific mechanism is provided to manage such elements. DDM is developed for design-time analysis of process variations for the control-flow perspective [12]. The method provides decision support in building variant models via the concept of business driver, though specific support is not defined for configuring the variants. Thus, DDM gets a “±” for decision support. The approach is applied in real-life applications from different domains and the results are validated with domain experts, giving it a “\(+\)” for validation. DDM is developed by considering the hierarchical structure of process models and incorporates a decomposition mechanism to the management of variants, so it is scored with a “\(+\)” for hierarchy support. For PESOA, we have a “±” for domain independence since the main goal of PESOA is the customization of process-aware software systems [38] and not the process models themselves. PESOA provides partial decision support by means of the definition of constraints over feature values. However, guidance is not provided for selecting an appropriate feature set to configure variants. PESOA defines techniques to encapsulate varying subprocesses and configuring them using feature models, giving it a “\(+\)” for hierarchy support. Provop has a “\(+\)” for domain independence since it has been applied in automotive and healthcare industries [41]. It provides decision support not for modeling the variants but configuring them by designing options that include reusable operations (“±” for decision support). Provop has a “±” for hierarchy support since it mentions the use of subgraphs to represent one level of decomposition as part of the options.

The team selected two approaches to be able to comparatively apply different process variant modeling approaches that meet their requirements but offer distinctive variant modeling mechanisms. The selected approaches were DDM and Provop. PESOA was excluded due to limitations in managing variations in the control-flow perspective and limitations in decision support. ADOM was eliminated due to a lack of decision support. We evaluated that the selected approaches, DDM and Provop, were good representatives of diverse variability management mechanisms due to the following approach differences:In the following two sections (Sects. 4, 5), we describe the application of the DDM and Provop approaches in a stepwise manner. Then, we compare the two applications and present the findings in Sect. 6.

The team started by developing a main SPM process that acts as a process map applicable to all variants. The process can be seen in Fig. 2. All activities of the best practice model in Fig. 1 were used. Two more activities, Plan Project and Create Asset, were added, since they were found to be common and frequent activities in process variants. The established process model captures the milestones of the process and provides a basis for further decomposition to subprocesses.

A key tenet of DDM is the consideration of variation drivers, which include business and syntactic drivers, to understand the emergence of variations and using them to flexibly develop the models [42]. Business drivers are the drivers causing variations in an organization’s processes due to its business environment. To understand its business environment and identify relevant business drivers, an organization may consider the product and services produced (what), physical and virtual markets it operates in (where), internal and external customers (who), and time intervals such as seasons that affect operation (when) [12]. An example business driver in the “what” category is different products customized for customers, and an example in the “when” category is high season when the number of process executions increase. Other reasons of variation identified based on the subjective judgment of domain experts, such as variations in processes due to the different ways employees perform, are identified as syntactic drivers.

In our case, the team focused on how the activities in SPM process in Fig. 2 are performed and possible causes of variation. The team observed that the main cause of variation is the variety of 4S customers, or companies, which is identified as the main business driver. Another driver was identified as the location of the services. This driver was used to differentiate the processes of Company 4, leading to different process variants for national and international services. No syntactic drivers were identified by the team for SPM process. Accordingly, when we refer to “drivers” in the rest of the paper, we consider only the business drivers identified.

In this step, variation drivers are analyzed to specify their priority as well as their effect on defining variants [43]. The business driver with the highest priority was found to be the companies in our case. This driver was found as the distinguishing driver for process variants of five companies. Additionally, the team identified the driver for the location of services. This driver was used to define two variations for Company 4 processes: international (variant 4-1) versus national (variant 4-2) services. Although two variants were identified for Company 5 at the beginning, they were decided to be merged as a distinguishing driver was not found.

In this step of applying DDM, a variation matrix is developed that shows the variants of each subprocess of the main process by using the identified variation drivers, and discussing how the processes are performed [12]. The team populated a variation matrix for each subprocess of SPM process as suggested by the approach. The variation matrix can be seen in Table 3. To generate this matrix, the team first identified the set of activities performed for the subprocess of each activity, and categorized those sets into a table format (Table 4). This table helped the team to reveal possible variations of each subprocess. For example, six different variants were identified for the “Plan Project” subprocess in this way, while fewer variants were identified for the other subprocesses. The categories used (i.e. basic, fast, simple, moderate, detailed, and complex) served as labels that reflect the characteristics of these subprocesses based on their cost and speed. For example, the variants in the “basic” category reflect the leanest way of performing the related subprocess, the “fast” variants represent a more costly but still time-efficient way, the “simple” variants describe less-detailed and fast options with respect to the rest. From “moderate” to “detailed” and “complex” categories, the time and cost of executing the variants increase. It should be noted that these categories are used only within the context of 4S and may not be relevant in other cases.

DDM suggests a subjective identification and categorization of variants by domain experts. However, since no systematic approach is suggested by DDM, it is highly possible that variant definitions are updated in later stages as more details are revealed about subprocesses [12]. The team also needed to change some of the variant definitions after the first iteration of the research cycle [14], resulting in updates in Table 3 and Fig. 3. The approach followed by the team at the second iteration via Table 4, variant identification by defining activities and assigning them to categories, provided a basis for team members to systematically observe possible variants, discuss their similarities based on the activities, and come to an agreement on the variants.

In the variation matrix in Table 3, the subprocess variants applicable for each business driver are depicted. For example “Simple Initiation” variant of Initiate Project subprocess is used by Company 3 and Company 4-2. This variant can then be related to a variant category in Table 4, for this example “simple” as no 3. As can be seen in Table 4, this subprocess includes two activities: Initiate Project and Assign PM. “Complex Initiation” variant used by Company 1 (having the category “complex” in no 6 of Table 4) includes a wider extent of activities, namely Initiate quality control and Approve scope.

In this step, the team assessed the similarity of variants by analyzing each subprocess of the variation matrix in Table 3. The team discussed the similarity of the activities in the subprocesses for each driver. To evaluate the similarity, the experts focused on how those activities are performed. For this, they used the activities defined in Table 4 for each variant and investigated the information on data used and produced while performing activities, and the workflow steps. As a result, the team decided which variants are similar and marked them gray in Table 4. For example, while “Moderate Initiation” variant of Initiate Project subprocess contained an activity named Approve Initiation, “Detailed Initiation” variant did not contain this activity but another one called “Announce Initiation”. The team indicated that these two activities are actually highly similar, thus merging the “Moderate Initiation” and “Detailed Initiation” variants. Additionally, the “Moderate Planning” variant was marked to be similar with the “Simple Planning” variant for the Plan Project activity, and the “Basic Closure” variant was marked to be similar with the “Fast Closure” variant for the Close Project activity. For Analyze and Design, Implement, and Test activities, basic and detailed variants were assessed to be similar. While DDM suggests subjective assessment of similarity of variants, the team observed that the use of Table 4, which was not defined as part of DDM, proved to be helpful to visualize variants and evaluate their similarities.

In the previous steps, the team obtained the variation matrix and the similarity assessments of the variants. Using these inputs, the team mapped the variants in the variation map as seen in Fig. 3. By using the similarity assessments and the decision framework of DDM, the team merged some variants in the variation matrix [12]. As a result, four variants for the Initiate Project subprocess, five variants of the Plan Project subprocess, and four variants of the Close Project subprocess were identified. No variants were identified for the subprocesses of Analyze and Design, Implement, Test, and Create Asset activities at this level of decomposition. This does not mean that all variants are performed in the same way for these activities. Although the variants are similar on this level of decomposition, there may be variations in deeper levels of the hierarchy.

The generated variation map acts as a reference model to observe both the process map and help the experts to arrive at possible variations by means of the flow defined by gateways. This model does not include knowledge of a specific variant. Thus, if one wants to configure a process variant, she needs to understand that specific variant and go through the variation map to select relevant activities. This selection was highlighted for the variant of Company 4-1 as shown with darker colored activities in Fig. 3. The process of Company 4-1 starts with the “Moderate Initiation” variant of the Initiate Project subprocess (this is shown as “Detailed Initiation” in Table 3, but these variants were assessed to be similar in Table 4). As specified by the variation matrix (Table 3), “Moderate Initiation” is either followed by the “Simple Planning” (for Company 4-1), or “Complex Planning” (for Company 2) variant. This is represented by an exclusive gateway after the activity symbol representing the “Moderate Initiation” variant in the variation map. The team manually verified that all variants could be generated as syntactically correct and sound [44].

The variation map helped the team to find out patterns about the process variants. For example, the team observed that “Simple Initiation” may be followed by either “Basic Planning” and “Fast Planning”. As 4S develops variant models for a higher number of customers in the future, such patterns may support the analysts to discover more relations between variants and perform deeper analysis on the processes.

DDM by definition builds on the construction of a hierarchical structure to define the family of process variants [12]. In this way, the approach provides a mechanism to manage process variants in a process repository by applying the same steps for the subprocesses identified in the variation map in Fig. 3. It is reported that DDM was applied within a hierarchical structure, but an example of a subprocess variant model was not presented [12]. To apply the approach in a hierarchical structure in conformance with the needs of 4S, the team decided to focus on the variations in the subprocesses of the Plan Project activity. This activity was selected because from a variation management perspective, it was the most complex one due to the high number of variants and number of subprocesses with respect to the others. Hence, this subprocess was evaluated to provide the most business value to 4S. Here, to exemplify the application and the problem encountered, we focus on the two variants among five variants of the Plan Project subprocess, namely “Detailed Planning”, and “Complex Planning” variants.

DDM explains a subprocess variant modeling mechanism for activities that does not have variants [12]. Examples of such activities in our case are the Analyze and Design, Implement, and Test activities in the variation map (Fig. 3). There are no variants of the subprocesses of these activities, according to the analysis in Table 4. However, DDM does not explicitly specify a solution for a case where an activity has variation in its subprocesses. In our case, namely, Initiate Project, Plan Project, and Close Project activities are examples of this case because they have more than one variant subprocess.

The two variants that we use as examples, “Detailed Planning” and “Complex Planning”, have some common activities, for example Prepare WBS, that may also have variations. These variations may be the same or different for “Detailed Planning” and “Complex Planning” subprocesses. The point not specifically addressed by DDM is how to analyze variant subprocesses of the same activity in SPM process (i.e. Plan Project) that share common activities, while also considering variations in those common activities (i.e. Prepare WBS). To handle this case based on our selected subprocess, the team identified three alternative solutions:For 4S, the team decided to apply the third solution. Thus, the team developed a single variation map and a variation matrix for the variants of the WBS preparation subprocess. The business drivers were identified as company, and the variants as “Company 2” and “Company 5” as imposed by the SPM variation map (Fig. 3). Moreover, the team identified one more driver applicable to this decomposition level: project type which can be “new project”, or “configuration project (conf. project)”. This driver is relevant for both company variants. The emergence of a new driver made the team realize that different drivers may be in place on different decomposition levels of a process repository. The team identified the variations of the WBS preparation subprocess in the variation matrix in Table 5. It was noted that, for some variants, there is no applicable subprocess such as for the second and fourth variants of the PAE polling activity. Based on the variation matrix, the team developed the variation map in Fig. 4 for this subprocess.

In the next section, we describe how the team applied the Provop approach to SPM process, and to one immediate subprocess in the process hierarchy.

Provop offers different policies to identify the base process from which the process variants are configured. The policies show alternative approaches to design the base process. After the base process is created, it is then enriched with adjustment points. Considering the variations of a process, the policies that can be followed to identify the base process are as follows. One can either (1) use the standard reference process used within the particular industry, (2) use the most frequent process variant, (3) design a version that has minimal average distance to all variants, (4) create a superset or (5) intersection of all process variants. The first two policies are different from the others in the sense that a base process designed according to one of these policies are valid processes and represent a specific use case, while in other cases the base process may not be semantically valid [41]. The selection of the policy depends on the context and purpose of variant management, and it is not mandatory to stick to one policy in the application of Provop [45]. In our case, the team applied a combination of these policies. First, the standard PMBOK reference model is taken as the starting point (policy 1). Next, the team extended this model by consideration of policy 2, that is the variant of Company 1 which is the most frequent process worked on in 4S. The team utilized policy 3 to identify process elements so that it will require the least number of operations in total to reach process variants, while the team also included activities at the intersection of all variants as suggested by policy 4. As a result, the initial version of the base process evolved from the best practice model in Fig. 1 to the version prepared at the end of the first iteration of the research cycle [14]. The team found this base process to be simple but still to provide an overview of the key activities observed in most of the variants. However, due to the issues encountered in designing the options as described in Sect. 5.3 below, the team updated the initial base process to the final one in Fig. 5 in the second iteration. Based on the different base process, the application of Provop was updated as seen in Figs. 6 and 7.

The next step of Provop is to determine the explicit positions of the adjustment points that specify where the options can be applied on the base process. In this step, the team analyzed the base process and identified the adjustment points necessary to be able to generate all process variants. Initially, the team placed an adjustment point at the start and end of all activities and single-entry-single-exit (SESE) fragments ([46]) on the main flow. Later, the team removed the unnecessary adjustment points based on the option design as described in Sect. 5.3 below. The final model with adjustment points can be seen in Fig. 6.

In this step, options are designed so that the process variants can be derived from the base process. For this purpose, Provop offers four different operations: “insert”, “delete”, “move”, and “modify”. The first three operations are on changing the control flow of the process, while the “modify” operation is for changing the properties of process elements. With the “insert” and “delete” operations, it is possible to add or remove process fragments between the adjustment points [41]. The team initially developed the base process in [14]. However, further investigation of the design revealed some issues, which made the team decide to update the base process to the one in Fig. 5 and design the options exemplified in Fig. 7. The issues encountered and our solutions are described below.As mentioned above, the team found that the design of the options is a challenging task, but it can also support the analysts to discover detailed control-flow properties of the variants. Granularity of options refers to the number of operations combined to define the options. It is an important factor in option design to enhance reusability and maintainability of options while keeping the number of options minimal [13]. As Provop suggests, two policies can be followed to group the operations into options: (1) to design options consisting of one operation and reusing them for multiple variants; (2) to create one option for each process variant that contains all operations for that variant. In practice, it is necessary to find a balance between these two policies by keeping the number of options low while increasing reusability. In 4S, the team first identified the minimum number of operations necessary to create the variants and the context rules for them. The context rules identify, for each option, the variant(s) that option is to be applied. Then, based on the context rules, the team started merging the operations. The team focused on increasing the reusability of options among the process variants, thus preferring low granularity as favored by policy one. The team preferred this approach because in this way, they were able to use the option list to observe the similarities in different process variants. However, it was possible to combine operations in various other ways to define the options. An alternative design for our options could be as follows. The last “delete” operation of Option 3 and the “delete” operation of Option 11 are the same. The team could take out these operations from these options and create another option applicable for Company 1, 3, and 4-2. Although this would create an option reused by three variants, it would increase the number of options in total and decrease the number of reused operations in Option 3. Provop provides hardly any support to discover and choose between such alternative option designs.

For variant configuration, Provop suggests the use of three substeps. First, relevant options need to be selected to configure a process variant. This can be done by asking users to manually choose specific variants, which is hard if there are a plethora of options and specialized knowledge is required. To overcome the problem, Provop suggests the definition of context rules by identifying, for each option, the context in which the options are applicable. In our case, the six SPM process variations of five 4S customers identified at the beginning of the study (Table 1) was used to define the context. For each option, the team identified the set of variants that are to be configured via this option. This can be seen in Fig. 7 as context rules. Another point to be considered while applying the options is the possible constraints with the options. For example, there may be implication relations between options, an option implying the usage of another one [13]. It was observed that the modelers need to pay special attention for constraints especially for options effective on the same adjustment point pairs.

In conformance with the constraints, the team applied the relevant set of options (exemplified in Fig. 7) to the base process on Fig. 6 to achieve the variant processes of Company 1 and 2 as can be seen in Fig. 8.

After the definition of the base process with adjustment points for the SPM process, the team needed to analyze the variants for subprocesses of the activities in the base process. The team was not able to find any guidelines for applying Provop in a hierarchical process structure in the literature. The team decided to review the base process for SPM process, and apply the steps of the approach in the same way for the subprocesses of this process. The team developed a base process WBS preparation subprocess, which we call low-level base process, that can be seen in Fig. 9. Example options from the option list for this base process are shown in Fig. 10. The team encountered the following issues during the application of Provop for the subprocesses and handled them as described below.

The application of the two approaches in Sects. 4 and 5 on the selected processes not only provides a high business value for 4S due to the processes’ organizational importance but also academic value as these processes have a higher number of activities and gateways than current examples in the literature [12, 13, 41, 42]. In the following section, we present our findings from applying the two approaches and provide a set of guidelines for process variant modeling approach selection.

The team evaluated the business value to be achieved by switching from the current way of process modeling to a specific process model variant management approach. For this purpose, the team calculated the expected Return on Investment (ROI) for DDM and Provop, as presented in Table 7. We describe the details of this table below.

Variant creation   4S works on average with nine customers per year, two of them being added as a new customer, and seven being existing customers. For these existing customers, only 30% of the effort is spent on defining new processes. 4S analysts works on around eight different process areas per customer for which variant modeling is performed. Around 90 h of effort are spent by 4S analysts for creating a new variant. As a result, \((2 + 7 \cdot 0.3) \cdot 8 \cdot 90 = 2952\) h are spend in total for variant creation.

For both DDM and Provop, a reduction is expected in the variant creation time. For DDM, a reduction of 49% is expected. For Provop, a reduction of 64% is expected.

Maintenance   As mentioned, 4S works with nine customers per year. Two of these are new and do not require any maintenance. For the seven existing customers, 30% of the effort is spent on variant creation. The remaining 70% are on maintenance. As mentioned, 4S works on eight different process areas. 4S analysts spend on average 54 h of effort per process area. As a result, \((7 \cdot 0.7) \cdot 8 \cdot 54 = 2116.8\) h are spent in total on maintenance per year.

Similar to the variant creation, also in the maintenance of variants a reduction in effort is expected by the team. For DDM, this reduction is expected to be 12%. For Provop, a reduction of 27% is expected.

In the past five years, 4S has worked with 17 customers. For each customer, a variant has been created for each of the aforementioned eight process areas. For DDM, it is expected that, on average, the transformation of one variant takes around 32, 4 h. For Provop, this is 25, 2 h. Currently 20 analysts are employed by 4S. 4S expects that a total of 24 and 32 training hours are needed for DDM and Provop respectively. In total, the transformation for DDM is \((17 \cdot 8 \cdot 32,4) + (20 \cdot 24) = 4886,4\) h and for Provop, this is \((17 \cdot 8 \cdot 25,2) + (20 \cdot 32) = 4067,2\) h.

In this section, we evaluate the application of the two approaches with respect to the user needs identified in Sect. 2.2 and the requirements specified in Sect. 3.1. We drop three of the requirements, namely Req. 1 domain independence, Req. 2 conceptual process type, and Req. 4 validation, since the definition of the approaches already supports these requirements and there were no further arguments by the team. We keep the four remaining requirements, control-flow (Req. 3), hierarchy (Req. 5), process modeling language (Req. 6), and decision support (Req. 7) that deserve further discussion about the extent the approaches satisfy them. Table 8 provides a summary of the evaluation. In this table, a benefit and/or advantage provided by an approach is indicated by a “+” sign, whereas a challenge is expressed by a “−” sign before the item. We further elaborate on each user need/requirement below.

User Need 1Establishing a knowledge-base Both Provop and DDM were observed to provide a knowledge-base by combining the process knowledge from multiple customers in an integrated model. This was found to be an improvement with respect to the current way of working where 4S analysts developed and maintained process models of each customer separately. The team found it relatively easier to read Provop’s base process as a reference. The base process provided essential information about the process without the need to use the option list. Moreover, the option list proved to be an effective tool to discover detailed similarities between the variants by means of operations grouped for multiple variants. Other interesting characteristics of control-flow variations were also realized by means of the option list. For example, although three variants included the create asset activity, each of them used this activity in the control-flow in a different way. Thus, three different operations had to be defined for inserting the same activity in different ways. The team found it non-intuitive to interpret the variation map of DDM, so it was hard to use it as a knowledge-base. However, main variation map included high-level activities, and provided a more summarized view that was detailed through lower levels.

In conformance with this evaluation, the team estimated a drop in the variant creation activities for both approaches, but more for Provop as seen in Table 7. Such an effort saving is foreseen since the analysts can use the created models as a knowledge-base while developing new process models.

User Need 2Maintenance Since both approaches provided an integrated environment that facilitates the impact analysis for changes, the team evaluated that maintenance effort to be lowered with respect to the current status. Maintaining DDM variation maps was evaluated to be harder with respect to Provop’s base process as there may be multiple subprocesses defined for variants of the same activity. A generic change may require the analysts to check each subprocess and apply the change in a consistent way. The team tried to alleviate this issue by applying their solution to integrate subprocess variant models that are referenced from multiple processes. Provop outputs were found easier to maintain than the outputs of DDM as changes can be applied on a single model. Provop was found to facilitate the maintenance of specific changes in addition to generic ones due to this fact. However, special attention needs to be given to maintain consistency with the option list. Accordingly, the team estimated the maintenance effort to be lowered for both approaches and more for Provop (Table 7).

User Need 3Reuse By providing an integrated knowledge-base from which similar variants can be looked up and a new variant can be configured, the team evaluated that both approaches can facilitate systematic reuse of process knowledge in comparison with the current approach. The team found DDM’s variation map more flexible than the Provop’s base process for defining a new process, as they can see all alternatives together with the constraints on the map. Business drivers helped the team to identify potentially similar variants in the existing models. However, Provop was found to be more practical for configuration-based reuse as described in the decision support item below. The support for variant modeling of hierarchical processes was found to be another aspect that affected reuse, which is discussed below. The expected improvements on the reuse were reflected in the effort estimation for variant creation in Table 7.

User Need 4Variability in a hierarchical structure and Requirement 5Hierarchy This user need emerged with the implementation of a process variant management approach, since BPMN natively supports the organization of process models in a hierarchical structure. The team evaluated that the application of both DDM and Provop requires additional consideration and effort for managing hierarchical process models. Although DDM by definition supports variant modeling in hierarchies, in practice the team encountered a problem to develop variant models for subprocesses that are used by multiple process variants. Among the solutions identified, the team applied the one which increased the reusability on low-level process variants. For Provop, the team could apply the native subprocess definition mechanism of BPMN. In this case, the team realized that they need to represent activities that are invisible in the base process because they are inserted by options. To establish a consistent hierarchical structure in the repository, the team placed such “standalone” activities in the base process. Moreover, another issue that arose with the application of Provop in a hierarchical structure is that, special attention must be paid to the consistency of adjustment points and events between different process levels.

Requirement 6Process modeling language 4S preferred to use a variability management approach based on BPMN since it was the notation used in the company. This requirement was met by both approaches. However, for applying Provop, it was necessary to add specific symbols representing adjustment points on top of standard BPMN models. Moreover, the team had to tailor the use of BPMN for specific purposes (such as standalone activity symbols for invisible activities). In summary, Provop required some changes over standard BPMN notation that the analysts need to learn and deal with. On the other hand, standard BPMN constructs were sufficient to use DDM.

Based on the metrics in Table 6, Provop seems to produce process models with less complexity in terms of BPMN constructs. This may enhance the understandability of the approach in comparison with DDM [48]. However, there is an extra artifact, the list of options, required to read and customize the Provop base process. The use of an extra artifact can cause cognitive difficulty due to the split attention effect [34]. Nevertheless, the team indicated that it was easier for them to read the Provop’s base process with respect to DDM’s variation map, and “picturize” how the adjustments may be conducted even without seeing the option list.

Requirement 3Control-flow The team found DDM relatively more flexible in defining control-flow variations, because the variants were developed as distinct subprocess models. The team experienced some limitations in modeling control-flow variability in Provop, such as inserting optional and consecutive activities, deleting process fragments, and dealing with possible configuration conflicts. The team had to extend and limit the application of the approach to resolve those limitations. An important limitation introduced by the team’s decision was avoiding the use of the fragment insertion feature of Provop. While this decision may have increased the complexity of the base process, it alleviated the maintenance effort related to process fragments. In the end, as Provop required the team to handle control-flow variations in detail, it also enabled them to understand detailed differences between variants from this perspective. The transformation costs in Table 7 were estimated based on the characteristics discussed under this and the previous item (Req. 3 and 6). The overall transformation costs of Provop is estimated to be lower, whereas the training costs are foreseen to be higher.

Requirement 7Decision support We evaluate the decision support provided by the approaches in two dimensions: the development of variant models, and configuring the models for process variants. For the first dimension, the team appreciated the support of DDM to identify business and syntactic drivers. In this way, they could better understand their business context. This benefit became even more prominent with the emergence of new drivers at low levels. Provop provides policies to define the base process. However, the team had to update the initial base process prepared according to such policies. Moreover, the team found the option design a challenging task which can also end up in different design alternatives. The approach provided hardly any guidance on how to design them.

For the configuration dimension, the team found it easier to use the Provop base process for the configuration of process variants. Similar change operations grouped in the option list decreased the complexity to generate a variant, and made it easier to configure a variant without much knowledge of the customer. The variation map of DDM does not provide any information on variants, one needs to have specialized knowledge for this task. The team suggested to add the driver names causing variations in related edges of the variation map to alleviate this problem. Overall, this requirement is strongly related with the user needs of establishing a knowledge-base and reuse, as decision support features of the approaches facilitated the conduct of the activities related to these needs. Both approaches were evaluated to outperform the current modeling approach.

Based on the evaluations, the team defined the guidelines in Table 9 for variant modeling approach selection. For each guideline, we refer to its source, the evaluated user needs and requirements that lead to its definition.

The highlight of DDM was its capability to embed all variant information in a single type of artifact, process model, while following standard BPMN notation and rules. The variation drivers based on business context provided a clear mechanism to identify and reuse variants. Based on DDM’s notable properties, guidelines 1, 5 and 8 were identified. DDM may struggle if low-level variations are to be analyzed in detail and if variation starts at the top-level of the process hierarchy. Provop comes forward with its support to discover control-flow variations in fine detail, and accordingly provide a robust configuration and maintenance mechanism. Consequently, guidelines 2, 3, 4, and 6 were identified. These features come with the difficulties in consistently defining and maintaining artifacts other than the process models. Although such challenges may be even more prominent in a hierarchical process structure, Provop may be applied similar to standard BPMN way in those cases. This brings us to the identification of the guideline 7.

To identify these guidelines, in conformance with the participatory action research method, we worked on the problem of selecting a suitable process variant modeling approach in a specific case for two approaches. This work allowed us to obtain first-hand and deep knowledge on the problem. Eventually, the meticulous identification of user needs and requirements and in-depth analysis of the approach applications may show the way for other organizations to implement process variant modeling approaches in their own context. Although these user needs and requirements are not directly generalizable to other organizations, they point out the concerns to focus on for approach selection. Similarly, the guidelines identified here for the two approaches potentially represent key points in variant modeling, which organizations can use to assess their situation and evaluate other approaches as well.