AI-artifacts in engineering change management – a systematic literature review

Although ECM research is driven by the five overarching goals of less, more effective, more efficient, earlier, and better (Fricke et al. 2000), publications along the core EC process focus on earlier, more effective, and more efficient EC handling (Hamraz et al. 2013). The goal of increased efficiency is commonly interpreted as the reduction of EC-related costs (Wänström et al. 2006) and the best use of resources (Iakymenko et al. 2020b). Effectiveness is improved by decision-making support, while earliness is improved by predicting problems before they occur. It is generally recommended to involve downstream process partners as soon as an EC is requested (Ouertani and Grebici 2008). However, decision-making with partial knowledge, lengthy processes, and a multi-departmental environment without visible ownership cause ECs to become a bottleneck process instead of an enabler for product maturity (Potdar and Jonnalagedda 2018). Thus, Radisic-Aberger et al. (2022) suggest research on new AI methods based on four identified business needs which have been simplified to:

The first business need targets the main challenges of complexity control and capacity increase, which emerge due to the EC process itself. Past literature mainly resolved the issue with qualitative solutions. For instance, Liu et al. (2004) conclude that the complexity of the EC process requires a flexible workflow system such as Petri-nets, while Bhuiyan et al. (2006) have elaborated that introducing multiple ECs simultaneously as a ‘batch’ is superior to individual introduction. This batch idea was further discussed by Wynn et al. (2010) and Ahmad et al. (2010) who provide methods to identify optimal frequencies and batch sizes for EC scheduling.

The second business need has been addressed in several contributions by simulations or rule-based methods. Wänström et al. (2006), for instance, suggested a decision tree model for application with various logistic EC variables. Similarly, Li and Moon (2011) provide a model to address company-dependent decision-making suggestions by integrating various data sources. Ouertani (2009) as well as Reddi and Moon (2011) show that predicting the impact of changes is effective for conflict resolution and effectiveness in EC.

Business need 3 is more related to later phases of the EC process, with several publications concerning effectivity date optimization, e.g., (Barzizza et al. 2001; Shiau 2011) and optimal test schedules (Kukulies et al. 2016), but also optimization of change propagation paths (Haibing et al. 2021). Although differing in their goal, the overall procedure is similar: An optimization function is solved for a set of variables and based on the nature and different strategies based on the EC trigger suggested. Lastly, due to the EC process being lengthy and problems occurring unexpectedly (Diprima 1982), the entire process should be monitored dynamically resulting in business need 4.

AI is a vast research area, composed of six main disciplines along the total touring test: natural language processing (NLP), knowledge representation, automated reasoning, machine learning (ML), computer vision, and robotics (Russell and Norvig 2010). Furthermore, the algorithms developed in AI research are often applied in other domains, which leads to confusion on what is and what is not AI (Bzdok et al. 2018). For our review, we only consider methods as AI, if they fit into the definition given by Russell and Norvig (2010): ‘AI is the study of agents, that receive percepts from the environment and perform actions based on them’. What characterizes AI algorithms from simple input–output algorithms, is the ability to improve performance by receiving feedback on the actions they took towards a specific goal. We give a brief introduction of the algorithms in the discussion section for those algorithms, which are used in ECM. For a discussion of other AI methods, we refer to classical AI literature such as Nilsson (2010) and Russell and Norvig (2010).

The application of algorithms is neither limited to their AI discipline nor the domain of AI. For instance, computer vision often utilizes the ML algorithm convolutional neural networks, NLP approaches employ recurrent neural networks and some algorithms find application in other domains such as data mining (Ertel 2016; Russell and Norvig 2010) or dependency modeling (Kumar and Ratneshwer 2016). Thus, AI algorithms are often not classified by their discipline, but rather by the task, they were initially developed for. For this, Russell and Norvig (2010) classify tasks into five types: Problem-solving (PS); knowledge, reasoning, and planning (KRP); uncertain knowledge and reasoning (UKR); ML; and communication, perceiving, and acting (CPA).

With the abundant amount of data recorded in the EC process, it becomes apparent that AI-supported handling of EC might help control complexity and mitigate the pains in ECM. However, although several literature reviews in the domain of EC exist, none focuses on AI applications. Whereas Jarratt et al. (2011) focused more on standardizing terminology, Hamraz et al. (2013) sorted literature into a holistic framework (Figs. 1, 2), which we are later adapting in Sect. 3.4 for classifying AI applications in the ECM domain. This framework by Hamraz et al. (2013) is especially effective at identifying research gaps in the literature, as it builds on the generalized EC process (Jarratt et al. 2005). Within this framework, group ‘A’ consists of literature focusing on the pre-change phase, dedicated to minimizing the number of changes. Group ‘B’ considers literature explaining and developing methods and tools to improve the in-change tasks and processes. Analogous to group ‘A’, group ‘C’ embodies research exploring the post-change stage of changes. The residual group ‘D’ covers publications on general topics and surveys in the field of ECM and related research areas. Additionally, Hamraz et al. (2013) identified groups ‘B, ‘C, and sub-group ‘D2’ as the core areas of ECM research, with the others crossing domains with product design and project management.

Most literature reviews that followed afterward are based on one or both major publications described above. Of the seven literature reviews identified after (Hamraz et al. 2013), none discusses AI application in ECM extensively. While change propagation understanding and impact prediction are discussed in (Helms et al. 2014; Ullah et al. 2015, 2016) as well as (Masmoudi et al. 2018), (Tale-Yazdi et al. 2018) is a review on data usage in ECM. Literature reviews focusing on EC implementation discuss the applicability of ECM research in engineer-to-order industries (Iakymenko et al. 2018) or classify EC research towards production, product design, and logistics (Balakrishnan and Suresh 2022). Among the reviews identified, (Colombo et al. 2017) stands out as a meta-study toward standardizing the language used in ECM. Finally, the review on change propagation analysis, its use-cases, and associated data representation by Brahma and Wynn (2022) touched on the application of AI-methods, although they did not discuss AI-methods for other EC tasks.

In conclusion, although several literature reviews have been conducted in the past within ECM, none has widely discussed the application of AI. Hence, to close this research gap and provide a knowledge base for future AI-based research, we present the results of a literature review on AI applications in ECM.

As our research domain, we focus on EC in the context of goods-producing manufacturing industries. This excludes EC in the context of civil engineering, service industries, and software engineering, as these have essential differences in the production system and separate research streams. Furthermore, we exclude EC for electric circuit layout design. This stems from the extensive literature available in this area, as VLSI layout problem solving via AI has been researched since the 1980s (Russell and Norvig 2010), and has dedicated reviews (e.g., (Singh et al. 2016)).

The scope of our research was further classified according to (Cooper 1988) as an exhaustive review with selective citation, focusing on practices or applications in the ECM domain represented by descriptions of a sample. The overarching goal is a descriptive review of AI literature in the domain of ECM with the identification of central issues, coming from a neutral point of view, with a special focus on EC. It is organized conceptually in a framework (Figs. 3, 4, 5, 6, 7, 8), grouping publications by methodological applications, and targeted toward specialized scholars.

A wide conceptualization of the topic was done to get a broad understanding and an overview of current trends in the scientific community. This step was done by consulting standard literature in the field of ECM, such as books for basic definitions and other literature reviews to identify current ideas. Gaining a broad understanding and looking for synonyms of the topic, a concept map as suggested by Rowley and Slack (2004) was drawn (Fig. 2).

With the key term of search string 1 (S1) being the target domain of ‘EC’, F1 represents the four business needs introduced before and their respective synonyms. Likewise, F2 embodies the target solution principle of AI. Although Rowley and Slack (2004) suggest applying different combinations of the search strings to discover literature, not all are sensible. Terms F1 and F2 combined, as well as on their own, would result in unspecific general literature outside the target domain, thus, these were omitted. Initially, it was intended to apply only the full combination of the three strings. However, during keyword search of the major databases ‘Scopus’, ‘IEEE’ and ‘WebofScience’, the results were comparatively low (< 100 results for S1 + F1 + F2 and < 250 for S1 + F2), which is why we settled on searching with only string S1 to make sure, that any publication within ECM applying AI-research would not evade the search due to inconsistent wording.

Hence, according to the methodology of Rowley and Slack (2004), the search string was defined as S1: (“Engineering Change” OR “Design Change” OR “Product Change” OR “Engineering Design Management” OR “Technical Change”). The additional strings of F1 and F2 were then used as filtering criteria during the manual screening of titles, abstracts, and the full text.

To identify relevant databases, research by Gusenbauer and Haddaway (2020) was consulted. From their list of databases, we chose those that focus on ‘Engineering’, ‘Computer Science’, or ‘Multidisciplinary’, and have a Boolean search logic and a repeatable search. The result of the initial search is presented in Table 1. To evaluate the relevance of these contributions, quality criteria (QC) for inclusion and exclusion had to be defined. These are, according to the goal and scope of the review (sec. 3.1), and the definition of AI given, as follows:

Due to language restrictions, only German and English literature is included.

Adhering to the PRISMA (Moher et al. 2010) flow chart (shown in the appendix, Fig. 9), we filtered relevant contributions for deeper investigation. Following the screening of titles and abstracts towards relevancy, 400 full-text entries were kept. After reading these 400 full-texts and applying the QCs, 28 publications on AI application in ECM remained and were considered for further discussion. To ensure we did not miss any publications, we further conducted a forward and backward citation search, based on the discovered 28 publications, as well as previous literature reviews. As a result, twelve publications were added to the review.

Following the literature search, a three-criteria organizing system as suggested by Webster and Watson (2002) was developed. The criteria (2) and (3) are used for quantitative discussion and criterion (1) for synthesis of future research needs:

With the main goal given as establishing a knowledge base of AI-based applications for implementation of ECs, the presented four business needs were defined as the primary criterion for identifying research gaps (Table 2, Sect. 6). As a second clustering criterion, we applied the holistic framework by Hamraz et al. (2013) (cf. Section 2.3) to give an overview of where along the EC process AI methods are primarily researched and applied. Lastly, we differentiate the publications by primary AI task. To be considered as AI, we take the definition as presented in sec 2.2, and classify them into: PS, KPR, UKR, ML, CPA (Fig. 3).

Overall, AI-supported EC methods have been constantly researched, with most contributions being published after 2012 (Fig. 4). As seen in Fig. 3, the focus of general ECM research on the first half of the EC process is also present in AI-based literature, with most publications being published in the areas of solution development and impact analysis.

Of the identified contributions, 49% offer methods for decision support (business need 2), with the rest being split towards optimization (business need 3) and automation (business need 1) (see Fig. 4). Literature focusing on this second business need employs various methods, though it UKR methods are dominant, whereas PS algorithms are mostly applied for optimization algorithms. Interestingly, no publication discusses AI approaches to supervise the process (business need 4). This might be due to two reasons: Either the EC process is too complex for supervision by AI applications or the current state-of-the-art is satisfactory enough and there is no need for AI methods.

Of all AI research, ML is the most extensively researched subfield (Nilsson 2010). Three main learning approaches exist: supervised, unsupervised, and reinforcement learning (Russell and Norvig 2010). Supervised learning algorithms are often applied for classification or prediction, by learning from input data with known output data (Ertel 2016). Correspondingly, unsupervised learning algorithms find patterns without prior knowledge of the outcome, most often used for clustering (Russell and Norvig 2010). The third, reinforcement learning is mainly used for optimization tasks, based on rewards and punishment (Ertel 2016). Additionally, ML algorithms are differentiated as shallow and deep learning algorithms based on the complexity of their deployment function (Russell and Norvig 2010). Shallow learners require extensive manual data preprocessing (Ertel 2016), an application obstacle circumvented by introducing deep learning algorithms at the price of additional computational cost (Russell and Norvig 2010). As the performance of the algorithms varies depending on the application (Pasupa and Sunhem 2016), a lot of ML-based research in non-AI domains focuses on identifying the best algorithm and data preparation method for a given problem (Bzdok et al. 2018). As with all algorithms, the performance of ML models varies with the preset hyperparameters. Thus, before final testing, hyperparameter optimization runs should be conducted beforehand (Feurer and Hutter 2019). For evaluation of which algorithm and hyperparameter set is best, usually a metric such as the F1-score or AUC-ROC (area under the receiver operating curve) is considered. In general, ML research follows the classic procedure depicted in Fig. 6a, b, supported by methodologies like CRISP-DM (Wirth and Hipp 2000).

From an ECM perspective, 23% of the publications primarily applied supervised or unsupervised learning methods for three business needs (Fig. 5). With a wide range of applications, ML provides support for multiple EC tasks. Out of the nine ML publications identified, three applied ML for change propagation, three for change clustering, and two for understanding ECs and the EC process (Table 2). The remaining publication utilized ML for EC effort prediction. No contribution applied reinforcement learning. Since we classified the publications by the primary nature of the method applied, it shall be noted here that two contributions, (Mehta et al. 2013) and (Chen et al. 2017), used ML as a baseline to compare the performance of their algorithms for impact prediction. These are discussed in the respective sections of their primary contribution.

The research field of CPA methods lies at hand: Retrieving information from data in the form of natural texts, images in large databases and learning machines to act upon these (Russell and Norvig 2010). Within the field of ECM, the only primary CPA application is NLP (Table 2), with image processing being only remarked as a possibility by Zheng et al. (2019). NLP is the ability of a machine to interpret human language input and thereby retrieve knowledge from large data bases (Russell and Norvig 2010). For training NLP models an extensive amount of data is required as the possible combination of letters to words are theoretically infinite. Furthermore, as human language is messy, data cleaning methods such as lemmatization and stop word removal are applied during preprocessing steps (Russell and Norvig 2010) (Fig. 8). Once trained, NLP models can, among other applications, be applied for text classification, query response, and knowledge retrieval (Russell and Norvig 2010).

In ECM, four recent publications exist which have primarily applied NLP to advance ECM. The goal of the first three publications (Arnarsson et al. 2019, 2021; Riesener et al. 2020) is to retrieve knowledge from previous ECs, such that mistakes are not repeated. Contrary to their focus on retrieval from past ECs, Koh (2022) applied NLP to evaluate social media comments, and derive future ECs from that. In their method, they follow the approach in Fig. 8.

While the first publication by Arnarsson et al. (2019) investigates whether a document embedding model outperforms an elastic search model, the other two apply a Latent Dirichlet Allocation algorithm to cluster ECs and provide support in knowledge management. Although all conclude that NLP is beneficial from an ECM perspective in retrieving information, they comment that the tailoring of the solution needs to be driven by the applying organization. Furthermore, as the comparison in (Arnarsson et al. 2019) shows, NLP outperforms elastic search only for longer search queries. These observations are attributed to highly individualized processes and language within companies. Additionally, due to applying the model to the EC database of a single company, the results cannot be generalized (Arnarsson et al. 2019, 2021).

As introduced, Koh (2022) applied NLP for extensive mining of information from social media. In their research method, they do not vary so much from the ML approach by Zheng et al. (2019), although instead of applying the RF, they apply the classical CPA by Clarkson et al. (2004). However, as they focus on the application of NLP, they describe the application of NLP in more depth and offer pseudo code. With their approach, they were able to analyze 3665 YouTube comments, pinpoint the components they affect, and derive ECs therefrom. Furthermore, they have shown that their method is not only applicable to the diesel engine they investigated, but also for ‘Car’, ‘Vacuum Cleaner’, and ‘Washing Machine’, suggesting it is applicable to all products.

Furthermore, we do note here that NLP methods are also important for data preparation for other AI-methods, as seen by the ML approach by Pan and Stark (2020) and Morkos et al. (2014).

Problem solving via algorithms is a common subfield of operations research or computer science with countless exact solutions for some complex problems (Russell and Norvig 2010; Taha 2017). However, AI research focuses on attempting to build machines that can find these solutions autonomously (Russell and Norvig 2010). For that, AI research devised informed search algorithms known as metaheuristics (MH) (Russell and Norvig 2010). Their main drawback is the inability of knowing whether the solution found is only a good solution or the best possible solution (Ertel 2016). They are quick to adapt and find reasonable solutions even for complex combinatorial problems (Taha 2017). MH can also be combined with other AI-approaches. For instance, it is common to use GAs for hyperparameter optimization or as layers in neural networks (Ertel 2016). In principle, all MH application follows the same procedure (Fig. 9): Develop a mathematical model of the problem, run the algorithm for n number of iterations to find new possible solutions, evaluate the solutions, and accept the best-discovered solution as the solution to the problem.

The application of MH in ECM is shared across all business needs, although its frequent with optimization tasks (Fig. 4.). Two contributions for automation (Fathianathan et al. 2007; Sharp et al. 2021) investigated how component design can be parameterized and subsequently design changes generated. Furthermore, MH were applied thrice for change propagation prediction (Li et al. 2019; Li and Zhao 2014; Lian et al. 2017), once for EC shop floor scheduling (Wang 2012), once for impact prediction (Mehta et al. 2013), and once for supply chain optimization (Wang and Che 2009). The most common algorithm applied was the genetic algorithm (GA), an easy-to-implement and flexible MH (Sharp et al. 2021) (Table 2).

The two most notable contributions in this AI subfield in ECM utilize evolutionary-inspired algorithms in feasibility studies for automatic design generation. As a study object, Sharp et al. (2021) demonstrated how GA can be applied for automatic design optimization after an EC is requested. Similarly, Fathianathan et al. (2007) applied a related evolutionary search algorithm to automate fixture changes after ECs. After defining the needs on how to approach the technical optimization problem, both present a simulated case study. In the descriptive depth of their study, Sharp et al. (2021) laid out a detailed automatic EC request workflow, depicted in Fig. 10. Comparatively, Fathianathan et al. (2007) focused more on arguing for and explaining the concept of evolutionary search algorithms. Both together have shown that MH are a possibility to automate parametric design, be it a discrete (Fathianathan et al. 2007) or continuous (Sharp et al. 2021) optimization problem. Sharp et al. (2021) further compared the performance of the MH vs. a human engineer, and although the calculation consumed more time, based on the input parameters the computer algorithm identified a better solution with less monetary resources invested, showing the potential of using AI to automate EC design activities. Fathianathan et al. (2007), however, has failed to show a comparison.

Li and Zhao (2014) as well as Lian et al. (2017) employed MHs to identify change propagation paths for improved scheduling of solution development. Both approaches are not standalone applications of MH, but rather a combination of graph networks and MHs. With their research, they offer methods to reduce change propagation and optimize the duration of the design phase of an EC. A key challenge faced by Li and Zhao (2014) when applying the GA was the inability to a priori determine which design tasks are affected by an EC. As a result, the model had to include all possible paths, irrelevant whether it will be followed, resulting in redundancy. In their follow-up research, Li et al. (2019), they improved on this method by introducing more constraints, providing an optimal EC propagation path. The publication by Lian et al. (2017) considers using the improved Cuckoo search algorithm, instead of the GA. As with other publications, these three papers present their research based on an exemplary sample but do not compare different algorithms or to other publications, thus they can also only be considered as feasibility studies.

Other MH research for capacity optimization using a GA was done by Wang (2012), who combined it with a Particle Swarm Optimizer (PSO) to minimize customization costs by decreasing the job floor make span time of ECs by optimizing the manufacturing schedule. They showed that a hybrid solution provides better results than using only GA or only PSO, as it enables the algorithms to escape local optima. To further solidify their outcome, they compared the approach to an already established method, which performed worse than any MH application. Overall, their research offers a method to improve the EC realization phase at the shop floor, an often overlooked area in ECM (Hamraz et al. 2013).

Impact prediction by application of MHs was shown by Mehta et al. (2013), who developed an own algorithm based on ant colony optimization, a common MH. Their extensive research shows, how important attributes of ECs can be discovered by MHs and utilized for knowledge retrieval. In their research, they compared their MH approach vs. ML models such as Decision Tree and Naïve Bayes classifiers. They have demonstrated that their approach is significantly better, although they do remark that their test set is comparatively small. This in turn leads to reduced interpretability and generalization of their research outcome. At this point we note that their approach was further tested upon by Chen et al. (2017), which is discussed in the KRP section of our review, in accordance with its primary AI type.

Another MH application was suggested by Wang and Che (2009) for preventive supplier selection concerning the complexities of EC. They applied four variations of the PSO to define the optimal product modification strategy by incorporating supplier data into the model. Not only did they mathematically optimize which EC to implement, but also provide a strategy on how to allocate call-off quantities among four suppliers. The extensive data used incorporated hard data as well as fuzzy data.

A common issue discovered in most PS approaches is the neglection of comparing the developed method with existing approaches and algorithms and testing with only a few test cases. Except for the contributions by Wang (2012) and Mehta et al. (2013), none has tested their approach against existing methods. Statistical significance was only proven in one contribution (Mehta et al. 2013), although even here the authors mention that the test set size was too small. For those studies declared as exploratory studies this is acceptable to a certain degree, however, it does reduce the scientific impact of their outcome. Furthermore, no approach reported implementation in a running environment.

The AI field of KRP centers on the ability of intelligent machines to store knowledge about their environment (Russell and Norvig 2010). This subfield deals with the logical representation of knowledge and lies the foundation of many advanced AI methods (Ertel 2016).

In ECM, logic can be applied for knowledge retrieval as shown by Mehta et al. (2012b), or for scheduling optimization by temporal logic as seen in (Chinn and Madey 2000).

In an initial publication, Mehta et al. (2012b) applied KRP to predict the impact of new ECs by searching for similar ECs among historical data. In doing so, they provide decision support to avoid change effects before they are implemented into the product. This was done by the introduction of an own algorithm tested on several expert knowledge-based simulated data sets and compared to an available statistical approach and two classical approaches (metric space and set-theoretic). They show in their research, that EC similarity is indeed a good indicator for impact prediction, and in doing so, they set the foundations for their more advanced approaches (discussed in sec. 4.4 and 4.6 according to the AI-method). Like their other publications, they compare to other algorithms and include statistical significance tests. However, by basing on a simulated test case, their publication shows only theoretical results. Furthermore, the test set (n = 23) is relatively small (17 train cases, 6 test cases).

Although situated in the domain of ECM, the temporal AI representation researched by Chinn and Madey (2000) was a demonstrator for the AI-method itself in the wider area of workflow management. Nevertheless, their research shows the possibility to improve the workflow of EC during the design phase by scheduling ECs and associated tasks based on urgency, feeding to business need3. However, their research was an exploratory study and mainly contributed to possible new research paths to incorporate temporal relations in scheduling problems. As such, no implementation in practice is reported.

The concept of knowledge representation is shown in sec. 4.5 mainly consisted of Boolean logic, where the outcome of an agent’s actions are deterministic (Russell and Norvig 2010). However, this is not necessarily always the case (Ertel 2016). In some cases, the outcome of a process cannot be definitely precalculated, but only a probability given of a certain outcome (Nilsson 2010). Designing agents that can deal with this uncertainty is the AI-subfield of UKR. From our literature search, we have identified four approaches to deal with uncertainty in the ECM process: Probabilistic knowledge representation, Fuzzy logic, Bayesian networks, and distributed artificial networks.