Creation and validation of systems for product and process configuration based on data analysis

According to Haug et al. [11], the process of creating configuration systems consists first of all of transferring knowledge from domain experts to knowledge engineers (elicitation), who then transfer it via an analysis model (translation) to a design model (formalization) before finally implementing it in a configuration program. Numerous works show challenges in transferring expert knowledge, referred to as externalization of tacit knowledge (see [12] for a comprehensive review). This term corresponds to Nonaka and Takeuchi’s SECI model according to which tacit knowledge can be transformed into explicit knowledge through externalization [13]. Besides fundamental criticism of the empirical validation of the SECI model by Gourlay [14], Tsoukas [15] shows that the SECI model is based on a wrong interpretation of tacit knowledge according to Polanyi [16] and that tacit knowledge should actually be understood as knowledge that cannot be transformed into explicit knowledge. According to this use of the term, which also corresponds to Haug’s [12] understanding, tacit knowledge can only be represented by a simplifying model which mimics decisions based on tacit knowledge but is not a one-to-one depiction of the tacit knowledge. Furthermore, Haug [12] shows that tacit knowledge in this sense plays a minor role for the creation of configuration systems. Thus, the overarching challenges in transferring expert knowledge as described in the literature are to transfer knowledge that is externalizable, but whose externalization is difficult. Aldanondo et al. [17], based on 10 use cases, attribute this to communication between the domain expert and the knowledge engineer. The coincidence of the two roles is desirable, but hardly realizable in reality, since the qualification effort is high in each case. A method to transfer domain knowledge directly into a configuration system without the domain expert needing knowledge of formal modeling therefore seems promising. A secondary challenge is to represent tacit knowledge in a model even if this is of subordinate importance. Decisions made by experts may be based on tacit knowledge even if this knowledge cannot directly be transferred. Therefore, Haug [12] mentions the approach of observing decisions of domain experts to create a model that mimics those decisions.

Configuration systems can be divided by their function into high-level and low-level configuration systems (HLCS and LLCS, respectively) [18]. In high-level configuration systems, the characteristics to be defined by the customer and their predefined values are stored [18]. Furthermore, there are constraints regarding the combination of characteristic values. I.e., certain characteristic values cannot be combined with other characteristic values or must always be combined with certain other characteristic values [18]. E.g., the customer cannot order a convertible car with a sunroof. High-level configuration systems allow to check the permissibility of the characteristic values selected by the customer or to suggest only permissible combinations of values to the customer. They, thus, form the basis for interaction with the customer. The product variant is defined by specifying the characteristic values in the high-level configuration. Low-level configuration systems, on the other hand, enable the creation of production-related documents, such as bills of materials and routings in particular, depending on the selected variant. Low-level configuration systems store super BOMs and super routings on the one hand and dependencies on the other hand [8]. Super BOMs are BOMs that contain all components that can occur in the configurable product across all variants. Similarly, super routings are routings that contain all operations that can occur in the configurable product across all variants. Dependencies are Boolean expressions that establish a relationship between the characteristics of a variant and the presence of a particular element in the BOM or routing. The focus of the presented approach is on the creation and validation of low-level configuration systems i.e., specifically on the creation of super BOMs, super routings, and dependencies. An approach for the creation and validation specifically of low-level configuration systems does not exist in the literature, which is why all approaches for the creation, validation or verification of models that allow the derivation of BOMs or routings are included in the following overview.

The creation of models for the derivation of BOMs is done in the literature via super BOMs. Numerous approaches exist that address an efficient representation of super BOMs e.g., with plan skeletons [19] or object oriented [20]. Kashkoush and ElMaraghy [21] show an approach to generate a master assembly sequence that is comparable to a super BOM. It is created by calculating, through mathematical optimization, the master assembly sequence that has the smallest distance to all variant-specific assembly sequences entered. Here, however, alternatives regarding the structure cannot be depicted. Furthermore, an approach by Moussa and ElMaraghy [22] exists for the creation of master assembly sequences. However, when transferring this approach to the creation of super BOMs, the assemblies contained in the BOMs would not be preserved, which has the disadvantage for configurable products that assembly-specific configurable routings can no longer be assigned to the assemblies of the super BOM.

In the literature, the creation of models for generating routings can be rule-based (i.e., the user enters the model manually) or data-based (i.e., the model is created based on data using data analysis). On the one hand, rule-based creation is done for part manufacturing applications based on manufacturing feature models [23‐25]. Here, a technology database is built that assigns a manufacturing technology and a processing station to individual manufacturing features. Inference is performed by creating a feature model for a new part—e.g., via feature recognition [26]—and deriving a routing via the technology database. On the other hand, rule-based creation for assembly applications is based on CAD assembly models [27‐30]. Here, the assembly sequence and assembly technology for an assembly are determined based on the interaction of its individual CAD models. All approaches assume manual creation of the underlying set of rules and do not address the difficulties described above when transferring expert knowledge into models. They further do not consider that previously created BOMs and routings may contain relevant information that could be used to deduct rules. Data-based approaches generate models for the creation of routings based on routings already created in the past. For this purpose, there are approaches that also assume feature models and make data-based assignments of manufacturing technologies and processing stations to features [31‐33]. In addition, approaches exist that assume that product variants are described by parameters [34, 35]. Even if this is not the focus of these approaches, they can in principle be applied to configurable products whose variants are also described by one-dimensional characteristics. However, low-level configuration systems cannot be directly derived from the models of these approaches, as they are non-interpretable machine learning models in the sense of Rudin [36]. In addition, there are approaches for learning assembly precedence graphs [37‐39], which have some analogy to a super routing. However, no dependencies are learned, which is why no variant-specific inference is possible. A different approach is taken by Tseng et al. [40] and Denkena et al. [41], who propose routings and BOMs for variants not yet produced using case-based reasoning based on variants already produced. However, the creation of routings and BOMs is done by the user. Lastly, approaches from decision mining [42‐45], in which process models with variant-specific branches are learned, are particularly relevant for the presented work. This is analogous to a super routing with associated dependencies. All in all the existing data-based approaches can only consider knowledge that is already documented in existing bill of materials, routings or case descriptions. Knowledge that is already explicitly described in the form of rules, cannot be taken into account, nor can expert knowledge that has not yet been externalized. Furthermore the results of those approaches aren’t LLCS as they are common in industry.

When checking models, a distinction must be made between verification and validation. Verification ensures that a model is correct in itself, i.e., that it does not contain any logical contradictions [46]. Validation, however, ensures that the model correctly represents reality [46]. Several approaches to verification of high-level configuration systems and of product configuration systems exist in the literature [47‐49]. In this context, methods for checking the satisfiability of Boolean expressions are usually used (so-called SAT problem) to ensure that contradictory expressions of BOMs, such as double assignment of a BOM item, are not satisfiable. However, whether the resulting BOMs are correct with respect to reality cannot be checked automatically so far. To the best of the authors’ knowledge, there are no approaches for checking process configuration systems and, thus, in particular, no approaches for their validation.

Table 1 summarizes on the one hand the necessary aspects to be covered by the presented approach and on the other hand to what extent existing approaches already take them into account. First, it should be possible to create inference models with the existing approach (1). Both, the data used for this purpose and the resulting model shall be validated (2, 3). For the sake of completeness, previous work is also classified with respect to verification of models (4), although this is not considered in the presented approach. The resulting LLCS should be suitable to predict the elements and the structure of the bill of materials and the routings of a variant (5–8) and, thus, enable a complete creation of production related documents. For the creation of LLCS different types of knowledge shall be considered which have been differentiated in Sect. 2.1. Firstly there is knowledge which is already externalized or is readily available for externalization in the form of rules (9). Secondly there are existing documents (10) i.e., BOMs and routings which contain relevant information and may have been created by experts using their tacit knowledge. Thirdly there is expert knowledge (11) which may be difficult to transfer into a model or which is tacit and can therefore not directly be transferred but only indirectly as described in Sect. 2.1. Last, the resulting LLCS should be interpretable (12) i.e., consist of super BOM and super routings as well as rules as is common in industry and not be replaced by e.g., a black box machine learning model. This shall ensure the acceptance of the approach in practice.

Altogether it can be stated that in the literature there are no approaches yet.

In the presented approach, these aspects are to be implemented and, thus, these deficits are to be remedied. The verification of the resulting models may also require further research, especially with respect to process configuration, that is not addressed in the paper at hand.

Each selected variant is accompanied by an effort for the expert team for creating or checking the associated bill of materials and the associated routings. Therefore, the number of necessary variants for achieving a sufficiently precise LLCS should be kept as low as possible. For the creation of the LLCS, the BOMs and routings take the function of labels in supervised learning. Thus, the task of Module 1 can be generalized to the request of using as few labels as possible for the creation of a machine learning model with high prediction quality. Thus, active learning strategies can be used to systematically select those variants that promise the highest possible information gain. Those variants can then be handed over to the expert team for creating the variant specific BOM and routings. There are several active learning strategies in the literature, but not all of them are suitable for the present case (see Table 2).

The LLCS is an inference model based on Boolean expressions. Therefore, it can only predict if e.g., a certain item is part of the BOM or not but is not able to give probabilities for predictions. Thus, strategies based on uncertainty sampling that require information on probabilities for different predictions are excluded. Likewise, strategies based on variance reduction and expected error reduction, which require analytical optimization procedures and thus analytical models, are excluded because an LLCS is not an analytical model. All other methods are applicable. Density weighted strategies can be used when constraints are present in the HLCS and thus an inhomogeneous distribution of variants exists over the space spanned by the product characteristics. Otherwise, the space is homogeneous and the density is the same everywhere. As an example demonstration of the approach, a configuration of pumps from the company Liquiflo is used here and in the following, as in the work of Kashkoush and ElMaraghy [50]. For these pumps, the customer can choose between type 620 (\({x}_{1}=0\)) and 621 (\({x}_{1}=1\)), between a single seal (\({x}_{2}= 0\)), a double seal (\({x}_{2} = 1\)), a packing seal (\({x}_{2} = 2\)) and a sealless magnetic coupling (\({x}_{2} = 3\)). Other features include the presence of a replacement cartridge (\({x}_{3}\)), base plate mounting (\({x}_{4}\)), S-adapter (\({x}_{5}\)), power frame (\({x}_{6}\)) and accessories & spare parts (\({x}_{7}\)). Full details can be found on the manufacturer's website (http://​www.​liquiflo.​com/​v2/​centro/​centry/​index.​html) and in the work of Kashkoush and ElMaraghy [50]. From the HLCS \(\left({x}_{1}= 1\right)\wedge \left(\left({x}_{2}=1\right)\vee \left({x}_{2}=2\right)\right)\leftrightarrow False\) is required i.e. type 621 cannot be combined with a double seal or a packing seal. If, for example, 4 variants are selected according to the greedy sampling, starting with type 621 single seal, the data pool contains the variants (\({x}_{1}=1\), \({x}_{2}=0\), …), (\({x}_{1}=0\), \({x}_{2}=1\), …), (\({x}_{1}=0\), \({x}_{2}=2\), …), (\({x}_{1}=1\), \({x}_{2}=3\),…), hereafter referred to as (1,0), (0,1) etc. according to the values of their first two characteristics, with the associated BOMs and routings.

When learning super BOMs, the aim is to create a super BOM that contains every variant-specific BOM available in the data pool as one specification. In addition, the variables are to be defined by whose assignment a variant-specific BOM can be created from the super BOM. The method presented in the following is intended to eliminate the deficits for the creation of super BOMs described in the state of the art. Figure 2 shows the BOMs of the example case. Figure 3 illustrates the procedure by means of the variant-specific BOMs (0, 1) and (0, 2) as introduced in chapter 3.1. The steps described in the following are shown as horizontal arrows in the figure.

After the super BOM has been created, variables \({y}_{{c}_{j}}\) are introduced that can be used to derive variant-specific BOMs from the super BOM. As can be seen, the variant-specific BOMs resulting from the super BOM can differ not only in the elements they contain, but also in their structure, e.g., the O-ring can appear as a sub-component of sealing or of mech. seal, depending on the variant. Therefore, identical components at different positions must be coded with different variables \({y}_{{c}_{j}}\). Assigning values to all \({y}_{{c}_{j}}\) thus results in a complete variant-specific BOM.

When learning super routings, the aim is to create a super routing that contains every variant-specific routing in the data pool as one specification. In addition, the variables by whose assignment a variant-specific routing can be created from the super routing must be defined. In current ERP systems, super routings are represented as a sequence of operations. For the final assembly of the pump described above, this would result in the super routing shown in Fig. 4. For the purpose of demonstration, we assume that there are alternative sequences for the operation Mount Outer Magnet, depending on the variant. Such structural alternatives are represented in current ERP systems by alternative operations. I.e., the structure results from the elements at the same time. For instance if a certain variant requires the operations Assemble Housing, Mounting Outer Magnet (1) and Mount Impeller Subassembly this also determines that they have to be executed in this order. This representation has the disadvantage that explicit knowledge about the existence of elements cannot be entered without entering knowledge about the structure, which may not be explicitly available. For instance if it is explicitly known that Mounting Outer Magnet is required for a certain variant this cannot be represented as a rule if it is not clear as well if the operation has to be executed at position 1 or 2. Furthermore, this representation has the disadvantage that in case of faulty systems the execution of both alternative operations is not generally excluded. For the presented approach, therefore, two alternative forms of representation (A and B, respectively) are presented, as shown in Fig. 5. Representation form A in YAWL representation has variables \({y}_{{o}_{k}}\), which indicate the contained elements and variables \({y}_{{s}_{l}}\), which indicate the structure independently and thus solves the two problems mentioned above. For example there could be a rule in the LLCS that determines that \({y}_{{s}_{1}}\) is equal to 1 for a certain variant which would lead to the sequence represented by the lower path. Representation form B in precedence graph representation specifies precedence relationships between operations that exist or do not exist depending on the variant, which is encoded by variables \({y}_{{s}_{l}}\). I.e., elements and structure are also specified independently here. For instance if \({y}_{{s}_{2}}\) is equal to 1 for a certain variant, there is a precedence relationship between the operations Mount Sealing and Mount Outer Magnet for this variant meaning that the operation Mount Outer Magnet may not be started before the operation Mount Sealing is finished. In addition to alternative structures resulting from differences between variants, there are also alternative structures in the disposition for individual variants themselves. All alternatives that are not violating the precedence relationship can be chosen depending on the current situation in production. In other words, disposition alternatives can be represented directly by representation form B. On the other hand, representation form B, in contrast to A, does not fundamentally exclude contradictory variant-specific routings. Therefore, depending on the case, one or the other is to be preferred. To learn super routings of representation form A, process mining approaches can be used. The commonly used alpha algorithm [52] is not applicable, since it may generate cyclic graphs, which cannot be automatically evaluated in an LLCS. In contrast, heuristic mining approaches are suitable, which use metaheuristics to vary the process model to find a model that has the best possible fit to the input paths. For learning super routings of representation form B, the method of Klindworth et al. [37] can be used, which, starting from a maximum constrained precedence graph, drops precedence relations one by one while reading in sequences of operations until finally only the necessary ones remain. The calculation of all \({y}_{{o}_{k}}\) and \({y}_{{s}_{l}}\) results in complete, variant-specific routings in each case. Routings for assemblies can go beyond joining operations and may contain operations such as cleaning, adjustment and inspection. These operations depend on the configuration of the variant, but not necessarily on its bill of materials. Finally, it should be noted that super routings can exist not only for the final product, but also for each assembly of the super BOM, if there are variant-specific differences for this as well.

After the super BOM and the super routing for the selected variants are known from modules 2 and 3, the dependencies of the target variables \({{y}_{c}}_{j}\), \({{y}_{o}}_{k}\) and \({{y}_{s}}_{l}\) on the product characteristics \({x}_{i}\) are to be established in module 4 using machine learning. Since the data is in the form of BOMs and routings, it must first be converted into a tabular form suitable for machine learning. Since all product characteristics \({x}_{i}\) are defined by the HLCS and all target variables \({{y}_{c}}_{j}\), \({{y}_{o}}_{k}\) and \({{y}_{s}}_{l}\) are known from modules 2 and 3, a table can be created using the product characteristics as feature columns and the target variables as target columns. Non-binary product features must be transformed into a binary representation by one-hot coding. Table 3 shows an extract of the transformed data for the example case. For instance row 1 represents the variant for which product characteristic 1 is chosen to be 0 (type 620), product characteristic 2 is chosen to be 1 (double seal) etc. The variant specific BOM was defined to contain component \({c}_{1}\) (Housing), component \({c}_{2}\) (Sealing 1) etc. The routing was defined to contain operation \({o}_{1}\) (Assemble Housing) etc. and was defined to follow the sequence \({y}_{{s}_{1}}=0\) (the upper path of representation form A in Fig. 5).

Classification methods from the field of machine learning can be used to create the dependencies by using existing BOMs and routings as well as BOMs and routings created by experts within the overall procedure (see Fig. 1) for training after they have been converted into a table representation as described above. However, in order to allow users to manually edit the LLCS as is common today, only methods that generate a model from Boolean expressions or models that can be transformed into such expressions can be used. This excludes non-interpretable models such as Deep Neural Networks. Table 4 provides an overview of the methods under consideration. In order to take into account rules explicitly entered by users, these must be incorporated into the model. For Quine-McCluskey classification [53], this is possible by including the rules as minterms within the method. In Associative Classification, the rules can be added to the learned model and the resulting model simplified using the Quine-McCluskey method. For decision trees, conversion to Boolean expressions is done first, followed by addition of rules, and finally simplification of the model. The greedy Quine-McCluskey classification introduced by Safaei and Beigy [53] is an adaption of the Quine-McCluskey classification which does not search the whole solution space in order to improve processing speed. It can therefore be assumed that the quality of its results is at most as good as that of the Quine-McCluskey classification. However, it can be shown that it nevertheless achieves better results for binary classification problems than a Decision Tree Classification. In this we see circumstantial evidence that a Quine-McCluskey classification achieves superior results, which is why we use it here and in the following. However, the approach described here is not limited to the use of this method. If we apply the Quine-McCluskey classification to the example case, we obtain the expression \(\neg {x}_{1}\to {y}_{{c}_{20}}\) for \({y}_{{c}_{20}}\), for example. Using the two previously unselected variants as test data results in an accuracy of 100% for this expression, i.e., this relationship is valid on all test data. Across all target variables, however, the accuracy is only 76.3%. This rather low value is due to the fact that in this case only few training data are available, since all but 4 data points do not contain any additional information. Therefore, a review for general cases will take place in chapter.

When applying the method, any number of data points i.e. variants with related BOM and routings can be generated in practice via module 1. The choice of the number of generated data points leads to a tradeoff between the prediction quality of the model and the effort of label generation. The prediction quality of the model can only be evaluated on the data for which labels are available. It follows that part of the labeled data must not be used for training, but must remain available for testing. Which data is used for training and which for testing has an influence on the result. Therefore, cross-validation is used. When a certain prediction quality is reached in the cross validation, the iterative procedure terminates. Optimal prediction quality can be determined as optimization of the total costs resulting from label creation and errors.

The input data for the creation of the LLCS results from the input of BOMs and routings by experts. Errors can occur in the process, which is why the data is to be checked for anomaly using unsupervised learning. To do this, the data from Table 3 must first be suitably transformed. Anomalies can be detected by two different views on the data.

In the first case, a method of Muller and Markert [55] can be used. Step by step, all variants are considered that have been classified into a certain class with respect to a certain target variable. These variants are now clustered and checked for outliers, i.e., variants that are unusual for the selected class. E.g., in case of Liquiflo pumps all variants for which \({y}_{{c}_{6}}\) is equal to 1 (i.e., the space plate is contained in the BOM) have product characteristic 1 chosen to be 0 (i.e., type is 620). By clustering all variants for which \({y}_{{c}_{6}}\) is equal to 1 a variant with product characteristic chosen as 1 (i.e., type 621) would be recognized as an anomaly. There may be a technical reason for such an anomaly or it may be the result of a mistake. In the second case, the values of the target variables themselves are considered as data points and are also checked for anomalies by clustering, i.e. for cases in which the target variables have unusual combinations of values. For instance components \({c}_{17}\) (gasket) and \({c}_{18}\) (containment can) always go along with each other. If there would be a BOM that would include \({c}_{17}\) but not \({c}_{18}\) this would be seen as an anomaly which can be detected by clustering as described above. Found anomalies are reported to the user and require a manual check.

The LLCS is learned from data on the one hand and is based on rules entered by experts on the other. Errors can occur when rules are entered, which is why the LLCS itself also needs to be checked. Since the LLCS corresponds to a model of Boolean expressions, the methods for verification known from the state of the art can be applied to it. In addition, the model can be validated by checking for anomalies using unsupervised learning. For this purpose, the Boolean expressions of the model are first transformed into two tables. Table 6 shows the literals on which the minterms occurring in the LLCS depend. Next, Table 5 shows the minterms on which the target variables occurring in the LLCS depend. For demonstration purposes, an error was introduced into the system learned in module 4, representing an incorrect manual user input, the expression \({x}_{1}\wedge {x}_{4}\to {y}_{{c}_{1}}.\) This expression consists of only one minterm which has never occurred before and is therefore added as last row of Table 6. This row shows a 1 in column \({x}_{1}\) and a 1 in column \({x}_{4}\) because this minterm depends on \({x}_{1}\) and \({x}_{4}\). Furthermore, the minterm is added as a new column to Table 5 which shows a 1 for \({y}_{{c}_{1}}\) because \({y}_{{c}_{1}}\) depends on this minterm. When clustering the entries of the two tables we see an anomaly in Table 5, as it results in the only entry out of 38 that depends on the \({x}_{1}\wedge {x}_{4}\) minterm. All other minterms have influence on several target variables. In general, both datasets can be analyzed for anomalies using unsupervised learning. Thus, also more complex anomalies can be detected.