Improving the operational effectiveness and efficiency of processes is a fundamental task of business process management (BPM). There exist many proposals of process improvement patterns (PIPs) as practices that aim at supporting this goal. Selecting and implementing relevant PIPs are therefore an important prerequisite for establishing process-aware information systems in enterprises. Nevertheless, there is still a gap regarding the validation of PIPs with respect to their actual business value for a specific application scenario before implementation investments are incurred. Based on empirical research as well as experiences from BPM projects, this paper proposes a method to tackle this challenge. Our approach toward the assessment of process improvement patterns considers real-world constraints such as the role of senior stakeholders or the cost of adapting available IT systems. In addition, it outlines process improvement potentials that arise from the information technology infrastructure available to organizations, particularly regarding the combination of enterprise resource planning with business process intelligence. Our approach is illustrated along a real-world business process from human resource management. The latter covers a transactional volume of about 29,000 process instances over a period of 1 year. Overall, our approach enables both practitioners and researchers to reasonably assess PIPs before taking any process implementation decision.
Hinweise
Communicated by Dr. Selmin Nurcan.
1 Introduction
Research on business process management (BPM) and process-aware information systems (PAISs) has resulted in many contributions that discuss options to improve the quality, performance, and economic viability of business processes [1]. Examples range from individual “best practices” [2] to comprehensive business process quality frameworks [3, 4]. In this context, we refer to process improvement patterns (PIPs) as generic concepts for enhancing particular aspects of business processes. As an example, consider decision processes that require to appraise various decision criteria. The respective appraisal tasks can be arranged to reach a decision with as little effort and as quickly as possible. This can be achieved by executing tasks with a high probability of providing sufficient information for a decision and with comparably low execution effort earlier in the process. This principle is known as “knockout” [5]. It constitutes a first example of a process improvement pattern.
Example 1
(Knockout principle) Consider a process for handling invoices received from suppliers. To determine whether the invoice should be paid, we want to check whether it is in line with purchase order data. In addition, we need to ensure that there is a sign-off from the responsible manager. The former check can be fully automated in the context of ERP systems and therefore be executed with little effort. Thus, it makes sense to execute this check first and possibly “knock out” the invoice before incurring the much greater effort of (manual) sign-off.
Anzeige
1.1 Research challenges
To ensure practical relevance, the actual business value of PIPs needs to be demonstrated to practitioners, thus enabling reasonable implementation decisions. In the context of this issue, there exist many propositions for empirically establishing the effectiveness of PIPs. These include anecdotal evidence [6], case studies [7], and surveys [8]. Commonly, these approaches are based on ex-post (i.e., hindsight) appraisal of qualitative evidence given by process managers or other stakeholders to obtain general insights applicable to comparable cases.
However, there still exists a gap regarding the a priori (i.e., in advance) assessment of PIPs considering a particular application scenario, which may range from an organization’s strategy and goals to its existing business process and information systems landscape. In particular, this gap should be bridged for the following reasons:
Similar to design patterns in software engineering [9], PIPs constitute abstract concepts that may or may not be useful in a particular context. Experience from other scenarios, which may widely differ from the one at hand, is thus not sufficient to take reasonable decisions on the implementation of organizational changes or process-aware information systems (PAISs).
Ex-post evidence is usually obtained from persons involved in the respective implementation projects. In turn, this leads to a source of bias. Moreover, a priori assessment allows addressing a far wider spectrum of PIPs. In particular, it is not necessary to complete implementation projects before a PIP can be assessed.
Combining PAISs with process intelligence tools [10‐12] opens up new opportunities to quantitatively and qualitatively gauge real-world business processes. This should be leveraged for scenario-specific PIP assessment.
Effective PAIS development requires to consider process improvement potentials before any implementation effort is incurred. Accordingly, PAIS development should start with a requirements definition which, in turn, is based on adequate process design considering relevant PIPs.
To enable a priori PIP assessment, this paper tackles the following challenges:
Challenge 1 Describe an approach toward a priori PIP assessment reflecting and summarizing common practice in the field.
Challenge 2 Evaluate the approach by applying it to a substantial real-world case.
Anzeige
Challenge 3 Reconcile the approach to scientific standards by applying guidelines for empirical research in information systems (IS).
1.2 Contribution
This article constitutes an amended version of the work we presented in [13]. Reflecting the considerations made above, [13] provides an approach toward a priori assessment of PIPs. In particular, the contribution of [13] is based on the following aspects:
The proposal we presented in [13] provides a standard approach to evaluate the impact of PIPs on organizational objectives for specific application scenarios. Thus, it supports well-founded decisions on the implementation of corresponding adaptations of business processes. Moreover, it contributes to bridge the gap between generic PIPs proposed by the BPM research community and real-world application scenarios. This way, it enhances the practical relevance of proposed PIPs.
It considers scientific rigor by applying an appropriate research framework.
It reflects practical requirements, which could be demonstrated through an experience report covering a substantial real-world business process.
In particular, when discussing the validity of our research design, execution, and results with practitioners, we made a number of observations that are generally applicable to process improvement projects based on PIPs. Since these may be helpful for practical application as well as future research, they are included as project recommendations in this paper.
This article complements our previous results with the following additional contributions:
It extends the presentation of the sample case used for our experience report with empirical results based on process mining [11]. Thus, it illustrates the application of this technology to a practical case.
It provides a discussion of open challenges regarding process improvement tools.
It provides a more profound discussion of the applied process improvement methodology, thus rendering the approach more accessible for application to other scenarios.
It discusses the complete set of process improvement measures that resulted from the application of PIPs to the sample case.
It describes the results obtained when revisiting our process improvement measures 14 months after having completed the initial process improvement project.
It provides a substantially extended discussion of related work.
It comprises a more detailed reflection of our results against Challenges 1–3, as well as an assessment of the general applicability of our approach. This includes a discussion of relevant limitations and strategies to address these.
The remainder of this article is structured as follows: Sect. 2 describes the sample process we use to illustrate our approach. Section 3 presents our approach toward PIP assessment. In the sense of an experience report, Sect. 4 describes the results obtained when applying the approach to the sample process from Sect. 2. Section 5 discusses the state of the art in PIP assessment as well as other related work. Finally, Sects. 6 and 7 evaluate our results referring to the challenges discussed above and conclude the paper.
2 Sample case: applications management process
The business process we use to illustrate the concepts presented in this paper stems from the field of human resource management. It addresses the handling of incoming job applications to fill open positions in a professional services firm. Figure 1 describes the business objective of this process according to a notation we developed in [14]. The objective of the process is to achieve one of two states for each job application: Either the application is refused, or a job offer is sent to the applicant. A job offer shall be sent if the following conditions are met: (1) The application documents have been accepted in terms of quality (e.g., with regard to the CV), (2) an interview has taken place with a positive feedback, (3) basic conditions have been agreed on between both parties, and (4) senior management approval has been obtained. If one of these requirements is not met, a letter of refusal has to be sent.
Fig. 1
Sample business objective: handling incoming applications
×
Based on our discussions with stakeholders and the results of process mining, we can model the business process implementing this business objective. For this purpose, we use BPMN (cf. Fig. 2, [15]). For the sake of brevity, we slightly simplify the model and omit a detailed description of its elements. As an example of the relation between the business objective and the process model, consider the conditions the business objective poses toward sending a job offer. The process model transforms these conditions into respective checking activities (e.g., Technical quality into Check documents) and XOR decision gateways. Note that there is not necessarily a one-on-one relation between conditions and checking activities. Further, there may be multiple process implementation alternatives for a given business objective (e.g., multiple conditions may be checked within one activity).
Figure 3 breaks down the total number of applications handled in a time period of one fiscal year into the number of applications for each possible termination state of the process. Note that the termination states from Fig. 3 correspond to potential paths through the process model from Fig. 2. We will refer to this overview when discussing our research execution in Sect. 4. We obtained a corresponding data sample of 27,205 process instances from the log database tables of the PAIS supporting the business process (in this case, an SAP ERP system). Each process instance covers one application. Thus, 1,972 out of the 29,177 applications of Fig. 3 are not included in the data sample. These comprise, for example, applications handled in the business units without involvement in the human resource (HR) function. These applications are not traceable in the PAIS.
Fig. 3
Termination states of the application process: one fiscal year sample
×
Figure 4 shows a process map generated with the Disco process mining tool [16] when applying it to the data sample.1 For the sake of readability, this process map has been filtered to solely comprise enactment traces that occur frequently and events that are relevant for our analyses.
Fig. 4
Filtered process map: one fiscal year data sample
×
The process map is an example of the results that can be generated with process mining tools. In the following, we will use process mining and other techniques to analyze our log data sample with respect to process improvement potentials. The process map should be considered as an amendment, but not as a replacement of “traditional” process models such as the one presented in Fig. 2:
The process map is based on events logged in the PAIS. Not all events directly reflect a corresponding activity in the process model, and identifiers of events might differ from the ones of corresponding activities. There may be activities not reflected in a logged event or events not triggered by an activity from the process model.
The process map shows the actual frequency of events in the data sample. Thus, it reflects as-is process execution, which may differ from to-be process design as recorded in the process model.
The process map needs to be interpreted with the support of experienced stakeholders. In our sample case, for example, application refusal events are used to purge the database of received applications to comply with privacy regulations. Further, not all hirings are handled through the corresponding end events. Issues like these need to be understood when interpreting the process map. However, this understanding is useful for process improvement as well.
3 Methodology
Like other IS artifacts, PIPs constitute goal-bound artificial constructs in the sense of the design science paradigm [20] to be evaluated in terms of “value or utility” [21]. In our context, this results in a particular challenge. While PIPs are abstract concepts applicable to a broad range of scenarios, their business value must be determined considering the specific use case to enable a decision whether the PIP should be implemented. To this end, we use an extended conceptual framework as summarized in Fig. 5.
Fig. 5
Extended conceptual framework
×
Beyond the concepts of PIPs and business processes or application scenarios, we introduce organizational objectives, process improvement objectives, and process improvement measures:
1.
Organizational objectivesreflect strategic goals an organization wants to achieve with respect to an application scenario. Examples of organizational objectives that apply to many scenarios include the effectiveness of process output, cost savings, or compliance with regulations [1, 22]. Note that these examples can be used as a starting point to identify organizational objectives relevant to a particular application scenario. In principle, such objectives are generic, but how they are prioritized against each other is specific to an organization’s strategy.
2.
Process improvement objectives (PIOs)comprise characteristics that enhance a process considering organizational objectives. PIOs can be viewed as a refinement of organizational objectives considering the particular challenges associated with a concrete application scenario. In a step-by-step approach, PIOs can be refined into a tree structure, as will be exemplified when discussing our application scenario in Sect. 4. The resulting top-down model is a useful mental technique to ensure a comprehensive perspective on process improvement. Note that similar considerations are used in goal-oriented requirements engineering (cf. Sect. 5.3) and value-based management [23]. This procedure can be aborted as soon as the resulting PIOs are sufficiently granular to allow for the application of PIPs. PIOs thus constitute the “bridge” between abstract organizational objectives and concrete PIPs. The relevance of PIOs to organizational objectives may be evident, or it may require additional validation. As an example of immediately evident PIOs, consider the elimination of obviously redundant tasks to reduce costs. As an example of PIOs that require validation, consider short cycle times. It is not necessarily a strategic goal to enact processes as fast as possible. However, this may be a PIO if a link between cycle times and a particular organizational objective (e.g., reducing costs) can be demonstrated. PIOs thus provide an additional layer of abstraction as a “shortcut” between improvement measures and organizational objectives. For the above example, potential improvement measures might be validated by demonstrating a positive impact on cycle times instead of overall cost. PIOs can also be viewed as a tool to identify PIPs relevant for the application scenario: Available PIPs are considered with regard to whether they can contribute to a PIO. For example, the parallel execution of formerly sequential tasks constitutes a PIP that may contribute to shorter cycle times as an exemplary PIO. Note that the concept of PIOs corresponds to the identification of stakeholders’ goals, which has been proposed as a requirement for empirical IS research in [24].
3.
Process improvement measures (PIMs)are bundles of actions considered for joint implementation.2 They reflect the application of process improvement patterns (PIPs) to a specific process in order to realize PIOs. Several PIPs may be bundled into one PIM for joint implementation, depending on the given application scenario. As an example of a PIM, consider the implementation of a new workflow tool, which may incorporate multiple abstract PIPs. A PIM thus applies one or more PIPs to a specific business process to address one or more particular PIOs. Assessing PIPs for a particular application scenario thus amounts to the assessment of the business value of corresponding PIMs considering relevant PIOs.
Note that, considering the arrows, Fig. 5 may also be read as a top-down method for process improvement. Section 4 further describes its application: General organizational objectives are refined to PIOs specific to the considered business process or application scenario. Then, PIPs relevant to the concrete scenario are selected from a generic set of generally available PIPs and bundled into concise PIMs. Specifically to the application scenario, PIMs are described in sufficient detail to enable to discuss and decide on their implementation.
Business processes and PIMs, as our unit of study, are implemented by means of PAISs. To maintain scientific rigor, their assessment should take into account requirements known from the empirical evaluation of propositions in software engineering or IS research. In [24], the authors subsume requirements in terms of scientific methodology for evaluation approaches in IS research. Figure 6 provides an overview on the basic concepts described there. In the following, we align our approach to [24]. We describe how each component is reflected in our proposition. Note that the (general) statements made should be further refined for each application scenario. From a practical perspective, this will ensure a common understanding by all project participants. Thus, respective considerations are included in the following paragraphs as well.
Fig. 6
Problem statement and research design: required components
×
3.1 Problem statement
The first four components we address constitute the problem statement according to [24].
Research question (“What do we want to know?”) Should PIMs be implemented to better meet organizational objectives? Note that this research question refers to PIMs instead of PIPs in order to reflect our goal of scenario-specific assessment.
For our sample case, the research question can be refined to the question whether PIMs should be implemented to reduce cost per hire (cf. Sect. 4.1).
Unit of study (“About what?”) The business process to be improved and the proposed PIMs comprising PIPs constitute our unit of study. Effectively selecting PIPs and bundling them into scenario-specific PIMs require the participation of knowledgeable, but also creative project members. For example, the participants of workshops to discuss PIMs should be carefully selected. In this regard, researchers may contribute a valuable “outside-in view” based on, for example, experience from other scenarios.
Regarding our sample case, the application management process and the proposed PIMs as the unit of study are described in detail in Sects. 2 and 4.3, respectively.
Relevant concepts (“What do we know in advance?”) Related work to be considered generally includes conceptual work on PIPs, case studies on comparable processes, and benchmarks available for the application scenario. In this regard, it is helpful to ensure proper research of available literature as well as a thorough use of available organizational knowledge (e.g., through selection of appropriate interview partners).
For our exemplary application scenario, we use a framework of process redesign practices [2], our own research into PIPs, a cost per hire benchmark, and available research on “knockout” processes [5].
Research goal (“Why do we want to know?”) Implementing PIMs will result in cost and risks incurred (e.g., process disruptions). To avoid unnecessary cost and risks, implementation decisions should be based on appropriate investigation of whether implementing PIMs will enable to better meet organizational objectives. Implementation decisions should consider not only benefits in day-to-day process operations, but also required investments and future operating cost or total cost of ownership.
3.2 Research design
The five components described in the following constitute the research design of our approach in terms of data collection, measurement, and data analysis.
Unit of data collection To understand the application scenario, we require an as-is process description to reflect process design, a process instances sample to reflect process execution, and PIOs to reflect refined organizational objectives. Depending on the application scenario and practical considerations, the process instances sample can be given as a PAIS data extract, as a set of interviews with involved people, as a set of cases directly observed, or as a combination thereof. To assess PIPs, we require descriptions of available PIPs and scenario-specific propositions of PIMs. Note that data collection should cover both process design and actual process execution. This way, PIOs can be identified prospectively (based on the process design) and retrospectively (based on the process execution). Immediate observations are preferable to indirectly related process information. Depending on the application scenario and practical considerations, the process instances sample can be given as a PAIS data extract, as a set of interviews with involved people, as a set of cases directly observed, or as a combination thereof.
Regarding our sample application scenario, we could refer to a business objective model and a flowchart of the process, a statistic on the results of process execution, and a substantial PAIS execution data extract (cf. Sect. 2). To assess PIPs, we use PIP descriptions available in the literature and from our own research, and PIMs as described below (cf. Sect. 4.3).
Environment of data collection Our proposition primarily aims at improving existing business processes. Hence, data are collected in the field to reflect the actual situation as best as possible. The environment of data collection thus generally comprises process stakeholders (i.e., contact partners involved in process execution, recipients of process output, or suppliers of process input) as well as relevant documentation and PAISs. The environment of data collection should be as broad as practically reasonable in order to facilitate identifying all PIOs that are relevant to organizational objectives, and to enable appropriate assessment of PIPs and PIMs.
Regarding our sample case, the environment of data collection comprised the head of recruiting, a business unit HR partner, business unit team managers, the PAIS administrator, and recruiting team members as process stakeholders. In terms of documentation, we used regular recruiting management reports and PAIS status codes. The PAIS used to support the business process was available as well. As a limitation to our sample environment of data collection, applicants as a group of process stakeholders were not represented in the environment of data collection due to practical requirements. Because of privacy regulations, applicants’ contact data may only be used to process the application, but not for other purposes.
Measurement instruments Our approach is based on elaborating PIOs and PIMs in a step-by-step approach. Draft PIOs and PIMs are thus used to document input from the environment of data collection, and constitute measurement instruments comparable to semi-structured questionnaires. These are amended with the proceedings documentation from interviews and workshops (see measurement procedures below). In addition, depending on the process instances sample, process execution tracing capabilities in PAISs or statistical process control (SPC) procedures also need to be considered. Note that measurement instruments should consider usability criteria with regard to stakeholders involved in measurement procedures. For example, this requires using terms customary to the organization when phrasing PIOs and PIMs.
Regarding our sample application scenario, PIOs and PIMs used as measurement instruments are described in Sects. 4.2 and 4.3, respectively. In addition, we used workshop proceedings, confirmation letters on results reconciliation (via email), and procedures to extract execution data from the PAIS used to manage incoming applications.
Measurement procedures Depending on the application scenario and practical considerations, relevant measurement procedures comprise stakeholder interviews, stakeholder workshops, and questionnaire procedures. Process mining can be used if the sample of process instances is based on a PAIS data extract. Measurement procedures should take into account customary practices of the organization, e.g., by using standard templates for meeting proceedings.
On-site measurement procedures (i.e., observing the process in its operations environment) can help to identify additional PIOs to be addressed for process improvement by giving a clearer picture of day-to-day process issues.
Regarding our sample case, we used telephone and face-to-face interviews with follow-up reconciliation of proceedings, a recruitment center site visit, and process mining with Disco.
Data analysis procedures In general, relevant data analysis procedures include qualitative analysis of workshop and interview results and quantitative analyses of process instance samples depending on the measurement instruments applied. Note that data analysis procedures need to be flexibly adapted to the step-by-step refinement of PIOs and PIMs and to the form of quantitative data available on the process instances sample. In practice, this may lead to a mix of tools actually applied. In this context, for example, statistical analysis tools can significantly reduce quantitative analysis effort and therefore enable enhancing the search scope for relevant PIOs.
Regarding our sample case, a qualitative analysis was conducted together with stakeholders as described in Sect. 4.3. In turn, the quantitative analysis comprised filtering of sub-process views in a process mining tool (Disco), re-extraction of filtered samples and import into a spreadsheet application, conversion of the event log into a “case log” (i.e., an array of events for each process instance), computation of cycle time attributes for each case, and statistical analysis with Minitab.
4 Sample case: process improvement patterns assessment
We now apply our extended conceptual framework comprising organizational objectives, PIOs, PIPs, and PIMs (cf. Fig. 5) as well as our research design to our sample application scenario. Further, we summarize our observations regarding the use of tools and systems for empirically analyzing our sample process, which may be useful for further developments in this regard.
4.1 Organizational objectives
As discussed, obtaining clarity about the content and business value of organizational objectives constitutes a fundamental prerequisite to ensure the relevance of PIP assessment. In our sample application field (i.e., recruiting), organizations strive to fill vacant positions quickly, cost-effectively, and with suitable candidates. To achieve these goals, personnel marketing is responsible to generate a sufficient number of suitable applications, while the purpose of our sample process (i.e., managing job applications) is to convert applications into actual hires.
Thus, organizational objectives for the sample application scenario include reducing the time needed until open positions are filled, reducing cost per hire, and improving the quality of hired applicants. Out of this set of objectives, reducing cost per hire is well suited for illustrating our approach. In particular, the issue of cost is transferable to many other scenarios. More precisely, the following considerations apply for our sample process:
Reducing cost per hire as organizational objective The cost per hire key performance indicator captures the total cost of both personnel marketing and applications management. While recruiting cost spent per application is proprietary data, based on experiences from projects with clients we assume an amount of about 400 Euros. In our sample scenario, generating and managing about 29,000 applications per year would thus result in 11.6m Euros total cost, with cost per hire at around 4,000 Euros. Since hiring cost for talent in professional services will be higher than in, for example, manufacturing, this value corresponds well to the average of 4,285 USD reported as cost per hire for larger organizations by a benchmarking organization [25]. Further, it seems rather conservative considering that professional recruiting consultants commonly charge half a year’s salary for successful hires, depending on industry. This calculation demonstrates the high relevance of reducing cost per hire through an improved application handling process.
Note that while reducing cost per hire has been chosen to illustrate our approach, the other objectives remain highly relevant. In particular, they need to be kept in mind when designing PIMs to avoid improving the process toward one objective at the expense of others. As an example, improving recruitment cost should not result in eliminating face-to-face interviews with candidates since this would probably reduce cost at the expense of the quality of applicants hired.
4.2 Process improvement objectives (PIOs)
PIOs pertain to characteristics of the business process that affect the organizational objectives we want to improve on. They serve as a “shortcut” to facilitate discussing the business value of PIMs without reverting to high-level organizational objectives. In our sample case, we refine the organizational objective to reduce cost per hire in a step-by-step approach by asking the question what drives cost per hire or, subsequently, the resulting lower-level PIOs.
Figure 7 presents relevant aspects of the resulting tree structure: Initially, total cost per hire is driven by the cost associated with each process instance (i.e., with each individual application), and, since it is possible that multiple process instances are needed to fill one position, with the overall number of process instances required. Both aspects may be optimized to reduce cost per hire, but are still rather abstract and will not allow applying PIPs without further refinement.
Fig. 7
Deriving process improvement objectives
×
On the one hand, cost per process instance is determined by the cost of production factors (e.g., the cost of employees’ working time or the cost of IT systems used) and the amount of effort spent on each process instance. Both drivers will occur in any tree of PIOs dealing with cost reduction. Factor costs, however, are generally unsuitable as a PIO since they are not governed by process designers and managers. Therefore, they cannot be subjected to process improvement efforts. Rather, they should be considered as a factor given externally which may affect assessment results. As an example, consider the impact of location decisions on labor costs.
Example 2
(The impact of factor costs on PIP evaluation) Particularly in large organizations, it is a common practice to bundle administrative business processes into “shared services organizations” [26]. In this context, labor costs constitute an important factor when deciding on the location of the shared services organization. In turn, this decision impacts considerations on the economic viability of process improvement measures. For example, when considering capital investments to automate manual activities, like matching incoming payments on bank statements against invoices issued to customers, lower labor costs will increase the payback time of the investment, thus rendering its implementation less attractive.
On the other hand, cost per process instance is determined by the quantity of production factors associated with each process instance. In our sample process, factors besides manual labor can be neglected, as will be the case for most administrative business processes. Accordingly, reducing effort per process instance pertains to reducing manual processing effort. This PIO lies at the core of many PIPs commonly used in practice, such as task elimination, task automation, or knockout [2], and has thus reached a sufficient degree of refinement.
Besides reducing cost per process instance, cost per hire might be improved by reducing the number of process instances required over time. This option corresponds to the elimination of in-efficacious process instances that do not terminate in a desirable state according to the underlying business objective. It closely resembles methods applied in common quality management approaches that aim at reducing “causes of poor quality” [27]. In particular, every in-efficacious process instance can be viewed as a quality issue in the business process. Note that the overall effect of quality management on cost and firm performance has been well recognized and empirically demonstrated [28]. This option is particularly interesting since associated measures can often be implemented with limited investments. Hence, further considerations on our sample case will focus on this PIO.
In our sample case, cost per hire is driven by the general efficiency of the application management process, but also by the frequency of process instances terminating in one of the states marked as “critical” in Fig. 3. The following considerations apply in this regard:
Not approving a job offer after a successful interview may be caused by defective steering of capacities (i.e., job vacancies), defective communication of terms to be offered, or defective review of application documents.
Job offers declined by applicants mostly means that the applicant does not approve of conditions offered, did not have a good impression during the application process, or has decided to take another job offer.
Since terminating the process in these states means that significant effort has been incurred while still failing to hire a promising candidate, organizational objectives are clearly violated: On average, only one out of six applications will successfully pass interviews. However, considering critical cases with defective termination events (cf. Fig. 3), only one out of ten applicants can be hired. In other words, if the process enactment defects lined out could be fully eliminated, only about 18,000 applications would have to be acquired and managed to cover demand. This would reduce total hiring cost by about 4.6m Euros.
Based on the considerations made above, we focus on PIOs to reduce effort per process instance or to reduce the probability of the defective process outcomes described. Table 1 summarizes the resulting topics.
Table 1
Sample case: process improvement objectives
PIOs
Rationale
Reduce manual processing effort
Emerging potentials in terms of reducing process enactment effort per instance should be addressed
Reduce failed approvals
Final approval of job offers by senior management fails if there are issues regarding vacancy management, reconciliation of terms, or checking of documents. The probability of these “late defects” should be addressed
Reduce cycle times
The probability of applicants’ obtaining and taking alternative job offers increases with time. Therefore, cycle times between applications being received and job offers made should be as short as possible
Note that for the first PIO (i.e., Reduce manual processing effort) there is an evident link to our organizational objective of reducing cost per hire. However, the second and third PIOs (i.e., Reduce failed approvals and Reduce cycle times) are based on hypotheses on what can be done to reduce process enactment defects affecting the organizational objective. Accordingly, they require qualitative or quantitative evidence to corroborate their relevance for reducing defects and thus improving cost per hire.
For the second PIO (i.e., Reduce failed approvals), we obtained qualitative evidence by interviewing responsible managers, which confirmed the topics described in Table 1. Since the reasons for failed approvals are not captured in the PAIS used to manage the application process, quantitative evidence is not available.
For the third PIO (i.e., Reduce cycle times), the causal link to its underlying defect of applications withdrawn by candidates is not as obvious. Further quantitative analysis is thus required. Figure 8 summarizes the duration between interviews and job offers for the subsets of applicants accepting and declining their offer in a boxplot (this part of total cycle time will later be the subject of process improvement, cf. Sect. 4.3). In the figure, the differences between subsets regarding quartiles, median, and mean values appear as relatively small. However, a correlation between cycle times and the probability of a candidate to accept or decline a job offer can be statistically demonstrated
Fig. 8
Boxplot Duration interview to job offer in weeks vs. acceptance of job offers
×
Correlation between job offers declined and cycle times We want to determine whether there is a significant influence of cycle time between application receipt and job offer in weeks on the probability of an applicant accepting or declining a job offer. Accordingly, we use a binary logistic regression test to evaluate the influence of a metric independent variable on a binary dependent variable. For this test, we use a sample of 2,721 job offers representing about 70 % of the annual volume (cf. Fig. 3) and consisting of instances fully covered in the PAIS (not all interviews and feedbacks are documented in the PAIS). The sample contains 261 cases where the job offer was eventually declined by the applicant. This is the latest point in the process where withdrawal by the applicant is possible, and a significant amount of effort will have been spent on each respective case. The two samples are independent from each other and have a size of more than 100 cases each. Thus, the binary logistic regression can be applied.
Figure 9 shows an excerpt from the output of the statistical software package we used (Minitab). The p-value of less than 0.05 indicates sufficient evidence to assume a significant correlation between the occurrence of withdrawal and cycle time. Regarding the question whether this correlation can be interpreted as a causal link of cycle times impacting the probability of withdrawal, the following considerations apply:
Cycle time is measured between receipt of the application and the ultimate feedback to the candidate, whereas the withdrawal sample refers to candidates that declined a job offer thereafter. An impact of the occurrence of a withdrawal on cycle time can therefore be ruled out.
There is a plausible explanation for longer cycle times causing withdrawals: It is possible that candidates find another job while waiting for feedback after an interview. This explanation is substantiated by data on withdrawal reasons collected for a sample of 51 withdrawals between October 2013 and January 2014 for one business unit. The sample covers only cases where a reason was given for the withdrawal. In 33 out of 51 cases, the reason cited was a job offer by a third party, and we may assume that longer cycle times provide more opportunities for candidates to find alternative employment.
We discussed potential additional independent variables with a positive effect on both cycle times and the probability of withdrawal with HR management and obtained no evidence on such variables. HR managers even mentioned that particularly sought-after candidates, who can be expected to quickly obtain alternative job offers, are handled with higher priority by business units. This effect might even “hide” part of the correlation between cycle times and probability of withdrawal. However, quantitative evidence on this issue is not available.
According to the “odds ratio” column, a 1-week delay can thus be expected to increase the probability of an applicant declining a job offer by 16 %.
Fig. 9
Minitab output excerpt: binary regression test
×
The significant correlation between cycle times and the probability of withdrawal did not become obvious when just considering median and mean values, but only when executing the binary logic regression test. This observation stresses the necessity to use both sufficient sample sizes and appropriate statistical methods when dealing with empirical data on business process enactment.
4.3 Process improvement measures (PIMs)
PIPs relevant to our application scenario have been selected by considering a “longlist” of known PIPs in terms of potential contributions to the PIOs described above. In our case, the “longlist” comprised PIPs from a framework by Reijers and Limam Mansar on process redesign practices [2] (these are marked with an asterisk “*”) as well as from our ongoing research on improving business process quality [4, 29]. However, organizations are not limited with regard to the sources of PIPs that can be used. PIOs are thus used as a mental technique to guide the identification of patterns that are useful for the organization. Relevant PIPs are then bundled into PIMs specific to the application scenario.
Table 2 summarizes PIOs and corresponding PIMs as bundles of PIPs.
Table 2
Defining process improvement measures for process improvement objectives
PIM 2 (Utilization and capacity management): Empowerment\(\mathrm{a}\), knockout\(^\mathrm{a,b}\)
PIM 3 (Standardized terms and conditions): Triage\(^\mathrm{a}\), buffering\(^\mathrm{a}\)
Reduce cycle times
PIM 4 (Managing interview feedback cycle times for successful applicants): Control addition\(^\mathrm{a}\), routing automation, escalation procedure
PIM 5 (Improving application routing): Case manager\(^\mathrm{a}\), knockout\(^\mathrm{a}\), mitigation of repetitive loops
\(^\mathrm{a}\)Process redesign practices comprised in [2]
\(^\mathrm{b}\)According to [2], knockout decisions should be ordered “in a decreasing order of effort and in an increasing order of termination probability”. Since this would contradict the goal to knock out respective instances with as little effort as possible, we assume that knockout decisions should be arranged in reversed order
In actual design and implementation projects, it is common to document and track individual PIMs through measure cards. In the following, we describe the PIMs from Table 2 in more detail using this metaphor. For each PIM, we give a short content description—with PIPs involved marked as italic—and additional details on the following issues:
Implementation describes steps required to realize the measure.
Business value appraises the expected implications considering the impact on PIOs as well as implementation effort.
Stakeholder verification describes the results of validating the PIM through interviews with respective stakeholders.
Note that the PIMs presented here are, by definition, specific to our application scenario. However, their structure as well as the underlying methodology is generally applicable. In terms of content, they exemplify issues commonly found in business process improvement projects, such as the evaluation of IT implementation effort. Moreover, beyond the scope presented here, actual measure cards comprise additional information relevant to project management. This includes project planning, project organization, key milestones with “traffic light” status, risks, next steps, and decision requirements. Reporting on measure cards usually takes place in steering committee meetings of senior management.
×
PIM Card 2 addresses the reduction in failed approvals as one of the critical cases identified in Fig. 3.
×
The abbreviated measure cards presented above exemplify how PIM implementation benefits can be projected and matched against expected implementation effort. However, beyond this quantitative reasoning, qualitative (or “political”) topics may play a role in taking implementation decisions as well, as will be exemplified with PIM Cards 3 and 4.
Fig. 10
Changes to the sample process when implementing PIM 3
×
×
PIM Card 4 addresses the Reduce cycle times PIO by dealing with one of the underlying drivers for unnecessarily long cycle times.
×
The final PIM we identified exemplifies an issue that occurs regularly in a top-down approach as employed in our case: Since it is possible that similar PIPs can be used to address various PIOs, overlaps in PIMs content may emerge. This topic needs to be considered in implementation planning and when consolidating recommended PIMs into a “management summary” view (e.g., in terms of overall implementation cost and cost savings potentials). In general, it is preferable to consolidate corresponding PIMs into one overall PIM addressing multiple PIOs. However, even in this case, the top-down approach facilitates obtaining a clear overall picture of potentials to be realized by PIP implementation.
×
Figure 11 compares PIMs 1–4 in terms of one-off implementation cost and recurring savings potential per year (i.e., gross cost reduction minus additional operating effort, PIM 5 could not be quantified in this respect). Note that the presented PIMs exhibit a fairly positive business case, with ratios between implementation cost and total annual savings below 1 year, and that the most positive business cases can be realized by implementing organizational measures without expensive IT implementation. They constitute good examples of a phenomenon often encountered in practice: in many cases, it is interesting to first identify and resolve existing process defects within the framework of available technology before additional process automation is implemented at huge cost. Once these “quick win” potentials have been leveraged, further process automation should be considered.
Fig. 11
Comparison of PIMs
×
4.4 Implementation results
The process improvement project facilitating our sample case has been concluded in early 2013 with the go-live of the newly implemented PAIS. This section briefly revisits the PIMs discussed above with regard to the results actually achieved. Statements are based on follow-up interviews conducted with stakeholders in March 2014, i.e., about 1 year after go-live. Our interview partners included the head of recruiting operations, the application management process manager, the HR partner of a business unit, and two business unit team managers involved in recruiting (e.g., as interviewers of applicants). To structure the interviews, we used the available PIM cards and collected feedback on their implementation and corresponding results.
PIM Card 1 (application management automation) Discussing this measure with stakeholders during our analysis phase resulted in postponement of the implementation decision because of the required restructuring of job ads. By now, it has been decided to implement the PIM with slight changes as discussed in the following. This decision has been taken because the demonstration of business value documented in PIM Card 1 has led to increased management attention regarding the avoidable manual effort spent on routing applications. The organization is currently undergoing an effort to significantly reduce variability in job ads. In the future, each job ad will have exactly one contact partner from a business unit assigned who will automatically receive screened applications. If the contact partner wants to pass the application to her colleagues for an interview, she will choose the appropriate person from a contact partner data base. This way, manual routing of applications can be largely eliminated.
PIM Card 2 (utilization and capacity management) The issue of utilization and capacity management has been referred to an entirely new “workforce management system” currently under development. This system will interface with the application management PAIS to avoid the issue of routing applications to teams with limited utilization. Note that this functionality will build on the implementation of PIM 1 as discussed above by controlling proposed contact partners from the contact partners database.
PIM Card 3 (standardized terms and conditions) Terms and conditions offered to candidates have been reconciled with an HR consultancy. This assessment has led to the result that current terms and conditions are in line with applicable benchmark values. On that basis, a new policy has been issued that requires deviating terms to be reconciled with business unit management. According to our interview partners, this policy has reduced corresponding cases to a minimum, which has led to a significant reduction in “late refusals” as proposed in the PIM.
PIM Card 4 (managing interview feedback cycle times for successful applicants) According to application management reporting, interview feedback cycle times could be reduced to an average value of 1.4 weeks based on implementing this PIM. Concurrently, the share of “late withdrawals” could be reduced to 7.7 % for the timespan of October 2013 to March 2014, in comparison with 9.6 % in the original data sample we analyzed. However, one needs to keep in mind that this reduction might be caused by varying reasons, such as changes in the labor market environment. Nevertheless, our interview partners confirmed their impression that reducing cycle times significantly contributed to this development.
PIM Card 5 (improving application routing) As proposed, this measure is being implemented in conjunction with PIM Card 1, Application Management Automation, namely in relation to managing job ads master data. Extensive loops and cycle times during application are now controlled by maintaining the responsibility of initial contact partners for timely feedback, even if the application is passed on to colleagues within the business unit. This way, contact partners have an incentive to avoid redundant loops. Outstanding feedback is then escalated to senior management.
4.5 Deployment of tools for empirical process analysis in practice
This section amends the experience report on our sample case with practical challenges observed when using available technology to drive the empirical analysis of process data. We believe that our sample scenario is fairly typical in this regard and that the issues described may be useful for the further development of corresponding tools and systems. In our empirical analysis, we used three types of technology: first, a process-aware ERP system, second, a process mining tool, and third, a tool for statistical analysis.
When using the ERP system of our sample process as a source of information for process improvement, we found it rather challenging to extract and interpret empirical data on process enactment. The major issues in this respect lay in the complexity of the underlying data model, which was partly subject to customization, its available documentation, and the availability of corresponding analysis and extraction reports. From our perspective, the usability of ERP systems in this respect might be improved by providing corresponding standard reports aiding systems administrators. In particular, this issue pertains to combining all relevant tables for particular application scenarios and to the matching of events to underlying process instances. In our case, the latter issue was exacerbated by the use of differing primary keys in related database tables. We are not in a position to judge whether the resulting complexity of the data model is really required. However, we believe that investigating its actual necessity might be worthwhile.
With respect to available process mining tools, we found that there are still certain functions that might be integrated in such systems to improve their effectiveness. This relates not only to process improvement projects, but also to other settings such as compliance management [22] or benchmarking [30]. However, we wish to stress that these issues pertain to commercially available tools in general. Disco, the tool used in our project, was chosen as it represents the state of the art of commercially available tools, in particular regarding ease of use and speed of deployment. Beyond the issues discussed here, [31] comprises a more detailed summary of process mining success factors based on multiple case studies. The following topics should be considered as functions not yet fully implemented:
Pattern analysis allows comparing multiple process enactment variants [32] including their actual frequency. For example, with regard to repetitive loops (cf. PIM card 5), this functionality would be very useful to identify and prioritize process variants that should be restricted or eliminated.
Compliance rules modeling allows describing relevant regulations for business processes in a way sufficiently formalized to automatically check whether these regulations have been adhered to in a process enactment data sample [22, 33]. As an example, consider the requirement of obtaining approval before job offers are issued.
Approximation of manual effort facilitates amending event-based process maps with the underlying manual processing effort. This would tremendously enhance the utility of corresponding analyses and could be achieved by enriching event types with assumptions on the corresponding activities. By matching a material sample log against total capacity used for processing (the so-called baselining in practice), the required degree of validity for the assumptions made could be achieved.
Automated regression analysis allows finding correlations between characteristics of data samples (e.g., between the number of contact partners involved and cycle times). If characteristics are derived from PIOs, respective drivers for process improvement can be identified automatically.
Sample delineation addresses the issue of restricting a data sample to exclude process instances without particular characteristics, such as the presence of start and end events. Since this topic is important to ensure the validity of analyses, tool developers might want to consider guiding users through the sample delineation procedure by way of an appropriate user interface.
Out of the topics listed above, compliance rules modeling and sample delineation can also be addressed through pattern analysis, which constitutes the basic functionality to enable process enactment optimization. Like in our sample case, process improvement projects utilizing empirical process enactment data will employ spreadsheet applications if pattern analysis is not readily available in a process mining tool.
In addition, we used Minitab as an exemplary statistical tool to support process improvement, e.g., with regard to analyzing the correlation between cycle times and candidates’ probability to withdraw their applications. This class of tools can be considered as advanced today and will generally provide accurate implementations of the relevant statistical tests.
5 Related work
Besides approaches directly addressing the topic, the assessment of PIPs relates to a broad array of fields. These range from general process improvement and quality to considerations on empirical research on information systems and are shortly described below.
5.1 Validation of process improvement patterns
Approaches aiming at empirical validation of PIPs can be traced back to quality management and business process reengineering approaches which have evolved since the 1950s and the early 1990s, respectively.
In terms of quality management, Six Sigma [27] is particularly interesting because it aims at eliminating errors in manufacturing processes through a set of quantitatively oriented tools used to identify and control “sources of poor quality.” While the scope of BPM usually lies in administrative processes instead, there are important analogies since Six Sigma is based on step-by-step optimization of production processes through experimental changes to parameters. This means that measures are subject to a priori assessment before they are implemented.
Business process reengineering (BPR), as exemplified in [6, 34], aims at optimizing processes “in the large” instead of implementing incremental PIPs. Transferring process enactment to an external supplier or customer constitutes a good example of this paradigm. While the potentials of this disruptive approach may seem tempting, more recent empirical evaluation has shown that the risk of projects failing is substantial [35]. Thus, incremental implementation of individual PIPs remains a valid approach.
In contemporary BPM [8], proposes a framework to select and implement redesign practices. As opposed to our research, this approach does not aim at assessing individual PIPs for a specific applications scenario, but at efficiently appraising a broad framework of practices in order to identify the most relevant propositions. We used earlier results from the same authors as a source of PIPs to be assessed in more detail [2].
The same authors also developed an approach to appraise BPR practices [36] based on the analytic hierarchy process (AHP) [37]. This proposition aims at ranking PIPs (or “best practices”) according to their importance for the organization. This enables limiting further considerations to a prioritized set of PIPs. In contrast, the goal of our approach is to assess individual PIPs for a given application scenario based on a total set of PIPs that should, in the end, be as large as possible. However, we believe that the approach of step-by-step refinement of organizational objectives and PIOs might be used as input to the AHP in terms of scenario-specific impact criteria.
In addition, [38] proposes PIPs to tackle the findings of an earlier case study on workflow implementation regarding issues with team collaboration. While it is not the objective of this paper to document a general methodology, it can nevertheless be viewed as an approach to prospectively appraise the implementation of PIPs for a particular application scenario.
5.2 Identification of process improvement patterns
In the following, we shortly summarize the relevant state of the art with regard to identifying PIPs that may be subject to assessment.
Besides leveraging PIPs that emerged from the BPR wave of the late 1990s and early 2000s, there also exist more recent attempts to provide a basis for process improvement by appraising perspectives on business process quality based on the software quality [3, 39] or managerial analysis and control [4]. These approaches result in sets of quality attributes or characteristics for business processes. Since measures that aim at fulfilling quality characteristics constitute process improvement measures, quality characteristics can be viewed as PIPs as well. In this context, comprehensively validating the set of quality characteristics provided by an approach through empirical analysis remains challenging, because it will be virtually impossible to find practical cases where the entire set of quality characteristics “adds value.” In this regard, the present approach instead enables organizations to validate quality characteristics to be improved specifically for a particular application scenario.
Benchmarking constitutes an approach widely employed in practice today [30]. Organizations seek to identify “best practices” to improve their business processes by comparing implementation options and results with “peers.” With regard to specific industries or application fields, the resulting “best practices” have also been compiled into specialized frameworks for process management and improvement such as the IT Infrastructure Library (ITIL) for information management [40].
In general, contemporary quality management methods used in manufacturing and logistics (e.g., Poka yoke to eliminate potential sources of errors [41]) can provide interesting pointers for improving business processes. A respective summary of the state of the art in “total quality management” (TQM) can be found in [42]. As an example for a TQM approach, the European Foundation for Quality Management (EFQM) proposition for achieving “organisational excellence” can be based on a business process perspective [43]. However, the underlying evaluation dimensions, which can be used to identify process improvement potentials, are rather abstract and require general and scenario-specific interpretation.
Note that research on PIPs addresses the quality of process models and process implementations in the sense of business content. In contrast [44‐49], exemplify propositions addressing process model quality in terms of structure, comprehensibility etc., i.e., the proper representation of actual business content by model elements.
5.3 Additional aspects
In IS research, there have been diverse propositions to ensure common standards of scientific rigor in empirical research such as field experiments or case studies [50, 51]. Hevner et al. [52] summarized empirical “design evaluation methods” for information systems research. As a basis of this paper, we chose the requirements summary by Wieringa et al. [24] due to its concise, checklist-based character, which makes it readily applicable to research as well as to discussion with practitioners.
Gregor [53] provided a taxonomy on various types of theory in information systems research. In this context, PIPs would fall in the category of “design and action” theories since they give prescriptions on how to construct artifacts (in this case, business processes). This perspective is interesting for the purposes of this paper, since it highlights the limitations of treating a PIP as an object of validation, and hence as a theory, on its own. Rather, whether a PIP is valid as a prescription to implement or change a business process clearly depends on the relevant application scenario and organizational context. In line with our propositions, a corresponding scenario-aware assessment method is required. This method then constitutes a “design and action” theory.
The top-down approach of deriving PIOs and PIMs from organizational objectives is similar to techniques for eliciting requirements in goal-oriented requirements engineering such as KAOS [54] or i* [55]. Propositions in this respect are based on working with stakeholders to identify goals to be met by a system [56]. Goals are refined step by step until a level is reached that is suitable for technical implementation. We stipulated that a structure of PIOs based on organizational objectives is useful to avoid redundant discussions of the business value of lower-level PIOs. Similarly, the state of the art in requirements engineering recognizes the practical benefits of a “goal refinement tree” linking strategic objectives to detailed technical requirements [57]. In this respect, the concept of organizational objectives compares well to the notion of “soft goals” [58]. The step-by-step refinement of PIOs corresponds to the basic AND/OR decomposition of goals which has been extended to common notations for goal documentation such as KAOS [54]. These parallels are based on the shared underlying notion of breaking down a larger problem, such as overall cost improvement, into more manageable chunks. This principle can also be found in contemporary approaches toward project management, e.g., in software implementation [59].
6 Discussion
When motivating this paper, we identified three challenges to be addressed in order to enable a priori PIP assessment. This section discusses how we addressed these challenges. Further, it discusses relevant limitations of our approach and presents recommendations that may be useful for similar projects. For quick reference, Fig. 12 recapitulates our proposition as a simplified approach overview: We first seek to gain a profound understanding of the considered application scenario including the corresponding organizational objectives. These are then refined into PIOs that are sufficiently granular to allow identifying relevant PIPs in the next step. Finally, relevant PIPs are bundled into PIMs and are appraised to enable implementation decisions.
Fig. 12
Approach overview
×
6.1 Revisiting research challenges
In the previous sections, we described an approach toward a priori PIP assessment. With respect to Challenge 1 (cf. Sect. 1.1), we believe that this approach is better suited to reflect common practice in the field than the available state of the art in IS research (cf. Sect. 5.1). While there have been propositions toward ex-post empirical validation of PIPs in the past, to our knowledge, the approach presented in this paper constitutes the first proposition toward a priori assessment of PIPs in the area of BPM. In particular, the two perspectives on PIP appraisal differ in their treatment of the available set of PIPs:
The ex-post perspective seeks to narrow down the set of PIPs to a selection of aspects with demonstrable empirical relevance in a wide variety of application scenarios, thus following common scientific practice.
The a priori perspective seeks to accommodate a comprehensive set of PIPs, but limits assessment to one particular application scenario. It thus “constructively validates” only a limited set of PIPs at a time.
Without doubt, the first perspective reflects common scientific practice, since it enables generic statements on PIPs that are independent of a particular context. Nevertheless, we found that the second perspective tends to be more in line with the expectations of practitioners. In our opinion, this reflects a central characteristic of PIPs and the corresponding implications for their practical adoption: As becomes clear when considering PIOs for various application scenarios, characteristics that drive organizational objectives may differ substantially for varying sample processes. Hence, a validation of PIPs based on other application scenarios is of limited value for implementation decisions. In this context, the practitioners we interviewed observed that a preselection of PIPs will be effective only in the case of frameworks addressing a particular field of application. As examples, consider industry-specific “best practices” such as ITIL for the field of information management [40], or guidelines for dermato-oncology in medicine [60].
Assessing PIPs for each individual project requires substantial effort by qualified personnel to understand the application scenario, identify and refine organizational objectives and PIOs, select appropriate PIPs, and finally bundle them into implementable PIMs. Whether this effort can be justified before initiating the assessment depends on the creation of business value that may be reasonably expected. Organizations should consider three topics before starting a PIPs assessment project:
Is the business process substantially relevant to the organization, e.g., with regard to the output produced or the cost volume incurred?
May the organization assume that there are improvement potentials in the process, for example, when considering existing problem reports or benchmarks [30]?
Are there particular circumstances that require analyzing the process anyway such as, in our case, intentions to replace the underlying PAIS?
In our sample case, we could assume an overall annual process cost of about 11.6m Euros (cf. Sect. 4.1). Thus, it became clear that even minor cost potentials identified would suffice to cover assessment effort.
Based on these observations, we believe that our approach toward PIP assessment is better aligned with common practice in the field and thus better suited to address Challenge 1 (cf. Sect. 1.1) than the previous state of the art.
Regarding Challenge 2, evaluating our approach through a substantial experience report, Sects. 2 and 4 presented the real-world case we used to this end and the results of applying our propositions. The sample case dealt with a business process found in most organizations and comprised 27,205 cases managed through a standard ERP system. It exposed typical issues when dealing with empirical analysis of real-world process enactment data, such as the complexity of extracting and interpreting data, as well as relevant process characteristics that do not become apparent by analyzing transactional data. We thus stipulate that our experience report represents common practical problems fairly well and has been suitable to evaluate our proposed approach.
To address Challenge 3, the reconciliation of our propositions to applicable scientific standards, we used a framework by Wieringa et al. [24] to guide the description of our approach. This enables us to trace all relevant components of an approach that fulfills scientific criteria, and simplifies the appraisal whether the corresponding requirements can be considered as fulfilled. We found that the more rigorous documentation of problem statement and research design demanded by scientific rigor required some additional effort in comparison with what is usually found in practice. However, this task proved still worthwhile, since it simplified final discussion of proposed PIMs with stakeholders. As an example, consider the impact of cycle times on the probability of candidates to decline job offers, which could only be demonstrated through rigorous statistical testing.
Overall, we consider our research challenges as met based on the considerations made above. However, there are still some relevant limitations we discuss in the following.
6.2 Relevant limitations
We demonstrated the application of our approach on the basis of a substantial real-world business process with 27,205 process instances. Nevertheless, the first issue that needs to be discussed with respect to limitations of our approach pertains to its evaluation on the basis of only one application scenario and thus a limited set of relevant PIPs. As a limitation to the environment of data collection (cf. Sect. 3.2), applicants could not be interviewed because of privacy regulations. It will thus be useful to apply the approach to additional scenarios to extend the set of PIPs actually applied. Of course, this will require access to additional real-world process improvement projects with substantial sets of empirical data on business processes. To draw meaningful conclusions, these processes should be comparable to what is commonly found in other organizations. Accordingly, additional experience reports (with the potential to extend the underlying approach) shall be an issue for future work taking up such opportunities.
A second topic relates to the availability of a comprehensive set of PIPs to be applied to PIOs. In principle, this does not affect the validity of our approach. However, it impacts its practical effectiveness, since it will determine the actual business value of PIMs identified. In this respect, much work has been undertaken by Reijers and Limam Mansar [2, 8], but the matter of conceptually deriving a comprehensive set of PIPs, which might be amended with more concise additions for specific application fields, still remains an open issue. We intend to contribute to this topic in the course of our ongoing work on business process quality [4] by identifying applicable quality attributes for business processes.
On a more abstract level, the third issue pertains to methodological limitations with respect to empirically validating PIPs. In this context, PIPs can be viewed as a prediction or theory dealing with the impact of certain process characteristics on process performance. However, comparable to design patterns in software engineering [9], PIPs do not constitute a self-contained concept for the following two reasons. As discussed above, currently no approach is available to demonstrate that a set of PIPs is comprehensive. In addition, the degree of utility of any given PIP is highly specific to the application scenario considered. Thus, it is virtually impossible to validate an entire set of PIPs by means of empirical information systems research such as field experiments, participative research, or case studies [61]. As discussed in Sect. 5, researchers have addressed this issue by conducting meta-studies on a broad range of PIPs [8]. This, however, means that individual PIPs are validated based on widely varying research designs. The approach presented in this paper also cannot resolve this issue. Still, it constitutes a generally applicable and reusable approach to assess PIPs for given application scenarios, which can contribute to harmonize PIP appraisal designs.
A fourth issue that needs to be discussed concerns inherent limitations with regard to demonstrating the general validity of the assessment approach proposed. The approach results in recommendations on which PIPs to implement. However, the question is how we can ensure that these recommendations are well founded. This challenge is exacerbated by two topics:
On a more detailed level, the business value of PIPs is appraised considering the business process and the scenario addressed. That is, the general assessment approach is refined specifically for each application scenario. Thus, it is not possible to fully replicate the same assessment approach in other settings, which limits the possibility of empirical validation. In other words, the validity of predictions on the business value of a particular PIP in a particular setting cannot provide assurance on the validity of predictions on other PIPs in other settings.
Revisiting PIMs after implementation will only allow identifying “false positives,” i.e., PIMs that did not deliver the business value expected. “False negatives,” i.e., PIPs not chosen for implementation which would have delivered a positive business value, will always remain undetected.
Nevertheless, it is still good organizational practice to track the results of PIM implementation. This provides an incentive to involved stakeholders to apply due diligence during PIPs assessment. However, since only “false positives” can be tracked, one should be aware that this might lead to overly risk-averse assessment practices. Setting top-down process improvement targets (e.g., via quantitative benchmarking) can be a way to respond to this challenge.
A final limitation takes up the issue of “false negatives” described above. It pertains to the degree of control we have with regard to the procedure of selecting PIPs and proposing PIMs for an application scenario. We have to be aware that this procedure depends on the knowledge, experience, and creativity of project participants. In other words, if no project participant can think of a way in which a PIP could be used to address a PIO, the PIP will not be considered in PIM propositions. However, this does not mean that the PIP cannot provide value in the application scenario. We stipulate that the step-by-step refinement of PIOs is a useful technique to address this issue since it helps to focus efforts on relevant aspects. However, it cannot provide formal assurance on this issue.
6.3 Recommendations for implementing our method
When working with practitioners to identify and assess PIPs applicable to our sample scenario, we encountered several general issues and recommendations that should be considered when applying PIPs in process improvement projects. We discussed these observations with our interview partners in the course of the respective steps in our approach. On that basis, we phrased a number of project recommendations that we present in the following. These recommendations were reconciled with management level interview partners and may be viewed as guidance for researchers and practitioners when setting up and executing comparable projects. Readers familiar with these topics may wish to skip this section.
Our first recommendation pertains to the overall structure of the proposed approach and to the “research design” component as required in [24].
Project Recommendation 1 (top-down process improvement methodology)Top-down process improvement refers to methods based on an initial definition of and agreement on the goals to be pursued, which are then further elaborated and amended with corresponding measures in a step-by-step approach. As a general principle, earlier decisions are refined to a more detailed level in later project phases. Top-down approaches address challenges resulting from two topics: First, process improvement projects typically require effective collaboration between multiple parties in an organization. However, these may tend to “sub-optimize” by focusing on individual interests instead of overall organizational objectives. As an example, consider the recruiting department and the various business units in our sample scenario. To “sub-optimize” its own effort in application handling, the recruiting department might pass applications not to the best, but to the most accessible contact partner in a business unit, thus impeding the goals of the organization as a whole. To realize the full potential of process improvement, parties need to be aligned toward clearly defined common goals and decisions as early as possible. Second, projects without a top-down decision structure may be obstructed by re-discussing goals and decisions multiple times. Besides the additional effort caused, this may lead to inconsistencies in the project. As an example, consider multiple measures addressing cycle times. Without a general understanding that cycle times are an objective of process improvement, this discussion will be led for each corresponding measure individually. With a top-down approach, earlier goals and decisions can serve as a gauge to appraise later decisions and measures.
The top-down principle is reflected in our approach. First, we require an early senior management agreement on the general “call for action” (see the concept of organizational objectives), which is then refined into process-related objectives (see the PIOs concept), and finally into individual improvement measures. This way, the discussion of individual measures focuses on how things are to be achieved instead of what to achieve in general. To implement this recommendation, agreed project results should be strictly documented, e.g., in a decision log.
The second recommendation is applicable to the “unit” and “environment of data collection” components as described in [24].
Project Recommendation 2 (identification of potential PIOs, PIMs, and PIPs based on process design and enactment) Potentially applicable PIOs, PIMs, and PIPs should be identified not only by considering the process model, but also by analyzing empirical data on actual process enactment. This is crucial to focus on topics of actual value potential. For example, consider the selection of critical cases in Fig. 3, which is reflected in the PIOs for our sample case.
The third and fourth recommendations address data gathering and analysis procedures required to appraise PIOs and PIMs for a particular application scenario. In terms of [24], they qualify “measurement” and “data analysis procedures”.
Project Recommendation 3 (appropriate qualitative or quantitative demonstration of business value) For each PIO to be addressed by PIMs, the underlying business value must be empirically demonstrated based on proper qualitative or quantitative analyses with respect to organizational objectives. Likewise, the business value of PIMs must be made transparent through appropriate analyses.
The specific analytic approach for individual PIOs and PIMs must consider the actual application scenario, balancing expected insights against analysis efforts. For example, the omission of tasks that obviously do not contribute to the business objective of the process can be justified by a short qualitative description. In contrast, the introduction of additional control tasks to diminish defects later on in the process will require careful quantitative weighing of pros and cons.
Project Recommendation 4 (identifying relevant stakeholders as interview partners) To ensure the validity of measurement procedures, proper selection of interview partners is particularly relevant for PIOs and PIMs that should be validated qualitatively. For BPM scenarios, it is important to interview experienced senior personnel overlooking the end-to-end business process and to represent both the “supplier” and the “customer” perspective to avoid lopsided optimization. For our sample process, we interviewed the head of recruiting operations and the administrator of the application management process from the “supplier” side, and the HR partner of a business unit as well as team managers from the business unit from the “customer” side.
The fifth and sixth recommendations concern the final assessment of PIMs. Hence, they refer to “data analysis procedures” as well [24].
Project Recommendation 5 (considering implementation effort in business value appraisal) When discussing the business value of particular PIMs for a business process, the respective implementation effort must be taken into account. This includes measures required, cost, time, and change management issues (e.g., training personnel to enact new activities). A PIM will only provide business value if implementation effort is justified by realized process improvement potentials. For example, an organization may demand that the required investment must not exceed three times the projected annual cost savings when appraising operational cost optimization measures.
Project recommendation 6 (leveraging “quick win” potentials) In many practical scenarios, it is possible to identify “quick win” PIMs that can be implemented with limited effort and should thus be given higher priority than others, in particular in comparison with full-scale PAIS implementation measures which are usually very costly. Examples include the elimination of process defects caused by process participants’ behavior, interface issues between departments, or issues of data quality. Note that these topics are often identified through empirical analyses (e.g., using process mining).
7 Summary and outlook
This paper described an approach for a priori, scenario-specific assessment of process improvement patterns based on organizational objectives, process improvement objectives, and process improvement measures. In our approach, we leveraged available work on generic requirements toward empirical research in IS engineering [24]. We thus demonstrated how these principles can be applied to practical cases while ensuring the general appeal of our approach.
We reported on the application of the approach to a real-world business process, including validation of the respective results with practitioners. The approach led to the identification of five potential process improvement measures that bundle and refine individual process improvement patterns for the given application scenario. Matching the expected gains against implementation and operating efforts, the organization was enabled to take well-informed implementation decisions. Revisiting the proposed process improvement measures more than 1 year after initial data collection confirmed that these decisions could be used to guide further development of the business process in practice.
In future research, we will integrate our PIP assessment approach into a broader proposal to manage the quality of business processes [4]. Moreover, we will apply it to further real-world application scenarios to gain additional experience. This will enable us to further validate and elaborate the approach. The assessment of PIPs as well as the validation of PIOs and PIMs will always require to refine our general approach according to the application scenario. However, considering additional sample scenarios might enable us to define standard types of assessment and validation procedures, for example, based on corresponding types of PIOs that occur in multiple application scenarios. In turn, this may further facilitate the practical adoption of our approach.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.
The field of process intelligence deals with analyzing the actual enactment of business processes [17]. In this context, process mining refers to using processing events logged with a timestamp to generate process maps, i.e., graphic representations of actual process enactment traces, and additional process information [11]. Note that Disco was selected as a representative of a number of tools available to practitioners in commercial settings today. Alternatives like ProM [18] or Celonis Discovery [19] might be used as well.
Note that in the given context, the term “measure” is not to be understood as a means of measuring something (e.g., a performance indicator) or as a unit of quantity, but as a coordinated set of activities.
