A Portfolio Management Method for Process Mining-Enabled Business Process Improvement Projects

Process mining (PM) is a specialized form of data-driven process analysis that organizations use to understand and improve their business processes (Martin et al. 2021; vom Brocke et al. 2021). Applying PM techniques such as process discovery, conformance checking, or enhancement using event logs as a central data source generates insights into process behavior, performance, and compliance (van der Aalst 2016). Turning these insights into action supports evidence-based business process improvement (BPI) and strategic decision-making (Martin et al. 2021).

PM, originating in scientific research, has rapidly evolved into a highly sought-after technology in the business world. Despite its emergence just a decade ago, Gartner (2023) reports the presence of 35 specialized PM companies, and the ecosystem continues to grow rapidly. Founded in 2011, Celonis, the undisputed PM market leader, is already valued at $13 billion (Metinko 2022). This growth is also reflected in the global PM market, which has grown at a CAGR of approximately 70% in recent years and is projected to surpass $15 trillion (Insights 2022). Major tech industry players such as SAP, Microsoft, and IBM have also recognized the huge market potential as evident in their recent acquisitions of PM vendors (Signavio 2021; Graham 2022; Lunden 2022).

Despite the promising future of the PM market, recent studies have shown that organizations face several barriers to the adoption of PM (Martin et al. 2021). Although the technological maturity of PM solutions is considered high, having benefited from substantial investments in algorithm development over the past decades, organizations still lack mature guidance on how to integrate PM to support their business process management (BPM) (vom Brocke et al. 2021; Grisold et al. 2020). When implementing PM, identifying and selecting valuable business processes and use cases to apply PM is a key challenge that remains largely unresolved and continues to plague those in charge (Thiede et al. 2018; Grisold et al. 2020). Further, organizations still face challenges in translating value cases from data analysis into actionable insights and subsequently business value, thus achieving a positive business case (Martin et al. 2021; Reinkemeyer et al. 2022). Project portfolio management (PPM) is an established research domain with global standards in the project management literature. Common PPM methods aim to maximize portfolio value by linking corporate strategy with portfolios and prioritizing projects sharing strategic objectives, following critical steps (Archer and Ghasemzadeh 1999). Despite a wealth of guidance, many sources provide detailed guidance only for specific steps, such as portfolio assessment tools (Henriksen and Traynor 1999) and project prioritization models (Hartmann 2002), leaving a gap in comprehensive end-to-end PPM support.

In contrast, holistic frameworks provide a comprehensive overview of the various steps but lack detailed guidance on their execution. Archer and Ghasemzadeh (1999) emphasize screening, individual project analysis, optimal portfolio selection, and portfolio adjustment, with minimal attention to upstream and downstream phases such as strategy development and project execution. Furthermore, traditional plan-driven management methods mainly describe sequential steps for identifying, prioritizing, allocating, balancing, and reviewing projects within a portfolio (Stettina and Hörz 2015). While these existing PPM models adhere to a linear sequence of steps from project definition to delivery, PM, as a data-driven and inherently iterative technology, allows for the iterative delivery of intermediate project deliverables (Nerur and Balijepally 2007), a dimension not anticipated by traditional models.

In response to these limitations, custom methods for PM have emerged, but they often focus on specific industries (Rebuge and Ferreira 2012; Bozkaya et al. 2009), specific steps (Rott and Böhm 2022), or individual project management and execution (van Eck et al. 2015; Aguirre et al. 2017). However, there is a lack of a holistic method for managing and scaling PM project portfolios that guides each step beyond piloting and provides structural support towards generating of business value from PM (Reinkemeyer et al. 2022). In response, this paper seeks to answer the following research question: How can organizations manage PM project portfolios?

Adopting the design science research (DSR) paradigm and situational method engineering (SME) (Gregor and Hevner 2013; Peffers et al. 2007), we approach the research question in a search process within the solution space. For the solution space, we define design objectives (DOs) and evaluation criteria ex-ante that must be met to satisfy the research question (Peffers et al. 2007). As per Gregor and Hevner (2013), DSR projects should always aim for useful artifacts. In this paper, we propose a method for managing portfolios of so-called PM value cases, which we define as PM-enabled BPI projects, towards an evolutionary roadmap mapper¹. Drawing on justificatory knowledge from PPM, BPM, and PM, mapper helps organizations in managing PM project portfolios to generate value with data-driven BPI. Comprising five activities, mapper guides users by suggesting techniques, roles, and tools for each activity. A prototypical instantiation complements the method, designed and evaluated with an expert panel of academics and practitioners to ensure applicability and real-world fidelity. Additionally, a case study substantiates the usefulness of the artifact in a naturalistic setting.

The remainder of this paper is organized as follows: Sect. 2 provides background on relevant justificatory knowledge. Section 3 outlines the applied research method and evaluation strategy. Section 4 presents the derived DOs, the design specification, and a prototypical instantiation. Section 5 describes the evaluation results. Section 6 discusses the findings and Sect. 7 concludes the work with limitations and areas for further research.

We adopt the DSR paradigm to address our research question of how organizations can manage PM project portfolios (Hevner et al. 2004). For the artifact design, we follow the DSR reference process, including six phases as outlined in Fig. 1 (Peffers et al. 2007).

(1) Problem identification and (2) definition of DOs. We conduct a literature analysis to identify and refine the research problem in Sect. 1, ensuring alignment with a relevant business problem (Hevner et al. 2004). Organizing the DSR process in a goal-oriented manner, we derive DOs for a solution in Sect. 4.1 from the problem statement and relevant literature in BPM, PM and PPM. The ex-ante evaluation further underpins the importance and novelty of our research and the defined DOs.

(3) Design and development. We organize the DSR project, particularly the design and development phase, as a search process within the solution space defined by the DOs (Hevner et al. 2004; Peffers et al. 2007). The research project results in an artifact in the form of a method with a prototypical instantiation as a byproduct. For designing our method, we combine DSR with SME, which assists in developing methods suitable for specific situations (Ralyté et al. 2003). SME has been successfully used in the past to develop methods suitable to particular contexts in the BPM domain (Denner et al. 2018; vom Brocke et al. 2021). Our method was designed along several mandatory method components that aim to support its application. Therefore, we specify that our method has to feature specific attributes and method elements as per Denner et al. (2018), who designed a method to improve business processes by exploiting digitalization potentials, which can be seen as a similar context to our goal-setting. Table 2 summarizes the respective method attributes and elements.

The SME process involves two tasks: setting a method engineering goal and constructing the method (Ralyté et al. 2003). While the research question and DOs guide the goal-setting phase, the method construction phase can follow three SME strategies: The assembly-based strategy (reusing existing method components), the extension-based strategy (applying extension patterns to extend existing methods), and the paradigm-based strategy (constructing a new method by abstracting from a given model or instantiating a meta-model). Given prior concepts addressing related research problems, we opt for the assembly-based strategy involving three steps: (a) specification of method requirements, (b) method chunks selection and (c) method chunks assembly (Henderson-Sellers and Ralyté 2010).

The step of specifying method requirements entails describing situations in which the method can be applied. In the SME context, situations are characterized by a combination of project and context type (Bucher et al. 2007). To define the context type for our proposed method, we draw from the BPM context framework by vom Brocke et al. (2016). This framework covers various BPM context factors with characteristics along four dimensions: goal, process, organization, and environment. The project type defines the state of an organization that we assume to be present when initiating the method (Bucher et al. 2007). Both components, the context and the project type, will be defined in the first part of the design specifications. The method chunks selection phase produces the base for the subsequent method construction (Ralyté et al. 2003). We employed two informative sources to create a knowledge base for selecting relevant method fragments (Gregor and Hevner 2013). First, we thoroughly reviewed relevant literature on the research goal. Through a review of existing methods and framework, five primary activities emerged: strategize, identify, select, implement, and monitor. We mapped existing concepts and selected justificatory knowledge from the literature to these activities to guide our method’s design. The collected knowledge includes general insights, tasks, and techniques, with a concise summary presented in Online Appendix A. Second, we collaborated closely with an expert panel of researchers and practitioners in the PM field throughout the entire DSR to gather additional insights and perspectives. A summary of the collected input from the expert panel for method design is presented in Online Appendix B. The dual approach, combining insights from both industry leaders and the literature, was chosen to develop a holistic approach to PPM. Recognizing the value of the approaches developed by industry experts, we aimed to combine their first-hand experience with our literature-derived insights.

The expert panel, consisting of four researchers and eight practitioners from vendors, adopters (i.e., organizations actively utilizing PM technology in their day-to-day business), and consultancies, offers an overview of the panelists’ background, experience, and involvement during the DSR process (Table 3). The panel’s composition is designed to ensure a comprehensive exploration of various facets and viewpoints, representing a spectrum of expertise. Given the relative youth of PM as a technology, organizations with more than five years of PM experience are limited. However, our methodology is designed to address the needs of more mature organizations in this context, so we have strategically included a larger number of consultants and vendors in the panel. These professionals interact with a wide range of PM clients, allowing them to bring different perspectives and insights to everyday industry needs and challenges. The participation of the researchers is based on their direct involvement in practical PM projects and their extensive methodological knowledge of method development. To collect potential method fragments, we initially interviewed experts from the panel with prior experience with the addressed research problem. The method fragment collection (MFC) interviews followed a semi-structured format, allowing the interviewees to share their experiences with managing PM project portfolios without any prior input or framing (Myers and Newman 2007). During the discussions, the interviewees provided their thoughts on the mandatory method components as per Table 2 (Denner et al. 2018). The insights from these interviews and the literature review formed a robust foundation for the research output positioning (Gregor and Hevner 2013).

The subsequent method chunks assembly phase is structured as an iterative design process that relies heavily on the teams’ cognitive skills in creativity and reasoning to design a novel solution (Gregor and Hevner 2013). Based on the generated knowledge base, the author team conducted several workshops to develop and refine the method and its instantiation. Throughout the design and development, we iteratively engaged with experts from our panel to conduct multiple rounds of evaluation to ensure a goal-oriented approach to the development process (Venable et al. 2016). In Sect. 4, we present the final design specification of the method and the corresponding prototypical instantiation.

(4) Demonstration and (5) Evaluation. Adhering to the principles of DSR and SME, we incorporated multiple evaluation activities into the development process (Peffers et al. 2007; Hevner et al. 2004; Ralyté et al. 2003). To structure the demonstration and evaluation of the method, we adopt the framework for evaluation in design science (FEDS) by Venable et al. (2016), which guides evaluating artifacts designed within a DSR project. Overall, DSR-based projects should aim to design useful artifacts (Gregor and Hevner 2013). The FEDS framework structures design phases and evaluation episodes along the functional purpose (summative vs. formative) and the evaluation context (artificial vs. naturalistic). Furthermore, we extend FEDS with selected components and evaluation criteria from the DSR evaluation framework by Sonnenberg and vom Brocke (2012). The designed evaluation aims to reduce uncertainty and risks during the design process and to underpin the effectiveness of the developed artifact (Venable et al. 2016). The intended artifact is a user-oriented method that meets the outlined research need and engineering goal validated by real users in their natural context. Therefore, we follow the Human Risk & Effectiveness strategy as per Venable et al. (2016). We split the evaluation strategy into three phases: (1) ex-ante evaluation, (2) intermediate evaluation, and (3) ex-post evaluation (Venable et al. 2016).

The ex-ante evaluation is performed immediately after the initial scoping in line with formative evaluations to estimate the expected impact of the intended research outcome (Venable et al. 2016). Operationally, the evaluation phase aims for a justified problem statement and design objectives. It is adapted from EVAL1 by Sonnenberg and vom Brocke (2012) and assesses the importance of our research and the suitability of the defined DOs as evaluation criteria. Thus, we surveyed our expert panel (Table 3) to obtain their opinion on the importance of (1) the addressed research gap and question and (2) the derived DOs for an artifact that is expected to address the stated research problem. The questions used a 5-point Likert scale from 1 (‘not important at all’) to 5 (‘extremely important’). In addition, we sought qualitative input on additions or modifications to DOs.

The intermediate evaluation is an iterative procedure that supplements the artifact design with several evaluation episodes. While these episodes primarily aim to improve the expected applicability of the artifact, they also provide initial insights into the artifact’s behavior in a naturalistic context. Operationally, the evaluation phase contains an interview-based validation using our expert panel that combines guidelines from EVAL2 and EVAL3. According to Sonnenberg and vom Brocke (2012, p. 395), EVAL2 aims to validate the design specification of the artifact. EVAL3 aims to validate an artifact instance to ensure applicability and “proof that the artifact instance is consistent with [the underlying] specification [as] validated in the preceding evaluation activity EVAL2.” Because of the close relationship between the artifact and its corresponding instantiation in the form of a prototype, we chose to include experimental simulations using the prototype in the expert interviews to underpin the applicability and real-world fidelity of the artifact as evaluation criteria. We invited our expert panel to participate in an interview study to reiterate the design-evaluate cycle and requested feedback on the artifact (Sonnenberg and vom Brocke 2012). We concluded each interview by asking the interviewee’s opinion on the method’s maturity and potential contribution to research and practice. If critical flaws were identified, the interviewee would have been invited to an additional interview after the next design iteration. However, no interviewee chose this option, which led to ten³ interviews (each between 45 and 75 min).

The ex-post evaluation is the final phase of the evaluation process, aiming to provide a summative assessment of the usefulness of the artifact in different contexts (Venable et al. 2016). Operationally, the evaluation phase is adapted from EVAL4 by Sonnenberg and vom Brocke (2012) and focuses on demonstrating the practical benefits of the artifact in a real-world context with real users. We conducted a case study and applied the method at Infineon Technologies AG, one of the world’s leading semiconductor manufacturers which operates in over 20 countries worldwide and employs over 56,000 people with several years of experience with PM. That case study helped to assess the usefulness as the main evaluation criterion of the method’s instantiation in the real-world context of Infineon’s operations and to gain valuable insights into how the method can be instantiated in a real-world environment and support different roles in managing and prioritizing use cases for PM. By usefulness, we aim to assess the efficiency (i.e., does it save time and effort), the effectiveness (i.e. does it enhance the quality and accuracy of decision-making), and the value (i.e. does it contribute to the creation of tangible value for the organization) that our method and prototype offer to various stakeholders involved in managing process-mining-enabled BPI projects. Infineon has been productively using PM for a few years. The company has implemented a central PM unit offering PM application development services to all departments in Infineon. However, a hybrid approach is used, which means that proficient departments can also develop their own PM applications to fully leverage the department’s expertise and be close to the customer. Infineon’s finance department opted to develop its own PM applications and recently initiated its first projects. Based on first pilot projects, they made first experiences using and integrating PM to improve business processes and support operations. On a value case level, the first project has already been completed, and the use of PM has significantly increased Infineon’s discount capturing rate. Following this success story, finance now aspires to expand PM on a large scale throughout the entire department. This status represents an ideal opportunity to offer guidance for managing PM value cases in preparation for the upcoming expansion. Therefore, we consider Infineon an ideal evaluation partner and appreciate their willingness to participate. The results of all evaluation phases are presented in Sect. 5.

For the design specification, we define our method requirements and DOs (Sect. 4.1). In addition, we introduce the general method to manage PM project portfolios towards an evolutionary roadmap (mapper; Sect. 4.2), which serves as the core artifact of our research and provides a framework for real-world instantiations. mapper addresses our research objective by supporting organizations in managing PM project portfolios to generate value by improving business processes in a data-driven way. Section 4.3 describes the first exemplary instantiation of our artifact as a software prototype.

Overall, DSR projects should aim to design useful artifacts (Gregor and Hevner 2013). To achieve this goal, we structured the evaluation of mapper into three distinct phases. The ex-ante evaluation (Sect. 5.1) concerns the justification of our research problem and the DOs by employing a survey within our expert panel. Section 5.2 presents an interview-backed validation of mapper’s design specification to underpin its applicability and real-world fidelity. Finally, we present the results of a real-world case study as the ex-post evaluation that validates the usefulness of mapper (Sect. 5.3).

After demonstrating and evaluating the design of our core artifact, we build on the components of an IS design theory by Jones and Gregor (2007), as summarized in Table 8, to synthesize our work (Gregor and Hevner 2013). We further highlight our contributions, limitations, and future research opportunities.

Jones and Gregor (2007) propose purpose and scope as the first dimension, that is, what the artifact aims to deliver and its associated requirements. mapper, as highlighted in Sect. 1, is designed to help organizations manage PM project portfolios for value generation through data-driven BPI. We define the context and project type to identify situations where mapper can be applied. Derived from the BPM context framework from vom Brocke et al. (2016), mapper is designed for medium or large organizations willing to invest resources to improve their repetitive core and support processes with some variability. The project type caters to organizations with prior PM experience seeking further scalability. Since our case study partner fulfills these requirements and the evaluation confirmed its usefulness, we conclude that the artifact addresses the initial research question and related requirements.

mapper’s design is based on justificatory knowledge from literature (Online Appendix A) and descriptive knowledge obtained from the expert panel interviews (Online Appendix B). As described in Sect. 2, we derive method fragments from PPM, BPM, and PM to form the basis for subsequent design. This helped identify the method components and revealed that no existing approach offers a holistic view to address the identified research problem. Existing work overlooks the data-driven nature of PM, which our expert panel repeatedly highlighted as critical for assessing impact and gaining management support for further projects. We also find that iterative value case management most accurately represents current practice.

We define principles of form and function in shape of method attributes and DOs. mapper is designed to meet the mandatory attributes outlined in Sect. 3. Comprising five activities, it enables informed decision-making for developing a valuable portfolio of PM projects, in line with the goal orientation. mapper and its activities are structured to offer guidance for systematic execution. Detailed specifications for each activity, including techniques, tools, roles, and outputs, enhance mapper’s repeatability across different contexts. Regarding principles orientation, mapper accounts for the three DOs 1 to 3, drawn from literature and validated with the expert panel, substantiating the novelty and contribution of our work. When compared to competing artifacts, it becomes evident that prior works (e.g., Rott and Böhm (2022)) support the comparison of PM projects, satisfying DO 3 but lack holistic guidance (DO 1) on preceding and subsequent activities. Others, offering holistic guidance on PPM (e.g., Archer and Ghasemzadeh (1999); Stettina and Hörz (2015)), fall short when considering the projects’ data-driven nature. Thus, mapper is a novel approach that sufficiently satisfies all DOs.

For the constructs of mapper, we draw on justificatory knowledge to define the overarching activities, namely, strategize, identify, select, implement, and monitor. In addition, the method elements according to Denner et al. (2018) (see Table 2) provide a useful structure for each activity. The defined activities have dependencies through input–output relationships, depending on the contextual data. By iteratively collecting data along the value case states, we support the successive confirmation of each project’s value. This results in a funnel-like structure accompanied by a gradual sorting process.

Regarding the artifact mutability, mapper allows a continuous adaptation to the current situation. The initial input collection of the organization’s strategy and value case ideas already underpins the possibility of adapting mapper to the organization’s preferences. Furthermore, the iterative nature of the method execution also allows for the adaptation of relevant parameters to environmental changes.

We also demonstrate an expository instantiation to portray the theoretical artifact and practically apply it for testing purposes. The software prototype, developed in close alignment with the methods’ design specifications, supports organizations in the end-to-end execution of mapper.

Finally, we formulated an evaluation strategy with multiple evaluation criteria serving as testable propositions. Since DSR projects should aim for useful artifacts (Gregor and Hevner 2013), the overarching evaluation criterion is mapper’s usefulness in a naturalistic setting. Along the evaluation path, we supplement this criterion with secondary evaluation criteria according to Sonnenberg and vom Brocke (2012). The results of an ex-ante survey underpin the importance of our work. Concurrent with the design, the expert panel iteratively confirmed the real-world fidelity and completeness of the artifact during interviews. A case study demonstrates the applicability and usefulness of the artifact in a naturalistic context. Since we can conclude that our artifact meets the defined DOs and evaluation criteria, we infer that our DSR project has produced an artifact that sufficiently addresses the research question.

During method development and evaluation, we identified some limitations. First, our expert panel, while diverse in PM backgrounds and knowledge, includes just two very experienced adopters which limits insights into different application domains. Second, given the extensive time frame required to conduct all method activities, a detailed assessment of all individual activities within the case study framework was challenging. While appropriate for a meaningful real-life evaluation, an extended case study could reveal new issues and limitations. Furthermore, defining PM value cases as BPI projects and leveraging the Devil’s quadrangle to assess the value potential may neglect other important dimensions, warranting a more profound scientific foundation of the value concept. Also, the corresponding criteria and items for the three value case assessment dimensions, i.e., value potential, risk minimization, and strategic importance, require further validation and potential extension.

Our work also facilitates further PM research. Integrating interfaces with BPM techniques such as robotic process automation (RPA) could improve usability and add benefits for adopters. An in-depth scientific evaluation of the PM value concept, followed by refining and validating value assessment criteria, would advance research and guide practice in assessing and prioritizing value cases. Further, the implementation and adoption of PM requires the introduction of governance structures for a clear understanding of responsibilities, roles, and communication channels. Lastly, exploring additional practical applications, such as a direct link to an existing PM platform, aligns with experts’ suggestions and our vision for an online PPM of PM value cases, with this work constituting a first cornerstone.

The increasing adoption of PM in practice has created a demand for prescriptive knowledge on managing PM project portfolios. As outlined in the background section, pre-existing concepts that we evaluated neither cover the full scope of mapper nor do they utilize new potentials from data. Our proposed method for managing PM project portfolios (mapper) stands out from existing PPM knowledge by considering the projects’ data-driven nature. It guides the management of PM value cases throughout their lifecycle, from instantiation to closure and monitoring. A prototypical instantiation complements the developed method. Finally, we demonstrate an extensive evaluation that supports our design, which contributes to the current body of knowledge and opens up avenues for future research in the field.