Single and ensemble classifiers for defect prediction in sheet metal forming under variability

Sheet metal forming includes simple processes, such as bending, stretch forming and spinning, and more complex processes, like roll forming and deep drawing [16]. Each type of process has its specifications and parameters, including the tools geometry. Process design becomes even more complex when it is required to combine several processes and/or steps to produce the component. The main driver for the development of numerical tools, enabling the virtual tryout of sheet metal forming components, is the industry, in particular the automotive industry, due to the enormous amount of components involved in car production. The outer panels are usually the largest components, and their production involves the most complex operations, including deep drawing. As shown in Fig. 1, in deep drawing the metallic sheet is plastically deformed into the desired shape by the action of forming tools, which typically consist of a punch, a die and a blank holder. The blank (i.e. non-deformed metal sheet) is placed over the die, and it is forced to flow into the die cavity by the movement of the punch; the flow of the sheet is typically controlled with a blank holder, i.e. a tool that imposes a constant force on the flange region of the sheet. Thus, even for a simple forming process and assuming that the mechanical behaviour of the metallic sheet is known, there are many design variables to be considered, which are related to the blank and tools geometry and with the process control, such as the blank holder force. As previously mentioned, the FEM-based virtual tryout of sheet metal forming components enables a feasible process design to be achieved through the repetitive adjustment of process parameters based on the personal experience of the designer. However, to fully explore the finite element analysis, it has been combined with optimization algorithms in order to determine the process parameters automatically (e.g. [29, 37, 42]). This approach is more or less computationally intensive, depending on the number of design variables and the type of optimization algorithm selected. The number of trial experiments (i.e. numerical simulations) can be reduced resorting to a surrogate model (also called meta-model), used to guide the search for optimized parameter combinations. Different meta-modeling methods have been applied to the optimization of manufacturing process parameters (e.g. [18]), including artificial neural networks (ANNs) (e.g. [26, 34]). In the particular case of forming processes, researchers are also trying to explore the large amount of data generated (both experimental and numerical) while designing new products, to guide the process design from its early stage with the help of ANN meta-models to predict product feasibility (e.g. [40, 45]). In the context of early design stages, neural network classifiers have been applied in automating the sheet forming selection process, as an alternative to rule-based programmes [16]. Moreover, the robustness of the process design is questionable when neglecting the sources of scatter inherent to the process. The process design can be optimum for a specific combination of parameters but can easily lead to defective components due to slight variations introduced by scatter. In this context, a robust process window should be evaluated, in order to minimize the production of defective components (e.g. [46]). Therefore, much of the recent work has focused on statistical descriptions of variability within FEM, for assessing the sensitivity of defect predictions to the scatter of the parameters under analysis [19, 35, 47]. In FEM, the material properties are commonly described using physics-based constitutive models. ML-based models have been pursued as an alternative to this type of models (e.g. [21, 23]), since the neural network does not require any prior assumption on the mathematical formulation between the input and output variables. The prime value added by ML is the ability to unveil the intrinsic response of a material in case of convoluted experimental data [24]. Nevertheless, some authors point out that physics-based constitutive models continue to provide useful insights to interpret the phenomena taking place, pursuing a different approach that uses machine learning to construct automatic corrections to existing models, based on data [14].

This subsection provides an overview of the literature on ML applications in sheet metal forming. Table 1 shows a comparative outline of ML applications in the prediction of forming defects [9, 13, 15, 20, 22, 25, 28, 30, 32, 39, 41]), which is the focus of the current work. Additional applications include: (i) material parameters' identification (e.g. [1‐3, 6]); (ii) bend angles' prediction in laser forming processes (e.g. [7, 10]); (iii) die roll height prediction in fine blanking (e.g. [43, 48]); (iv) optimization of incremental sheet metal forming processes (e.g. [12, 17, 44]).

The summary presented in Table 1 highlights that ML applications have been focused on regression. In this regard, back-propagation-based artificial neural network (BP-ANN) is the primary option for the development of prediction models, some of them coupled with FEM analysis [9, 22, 28, 30, 32, 41]; ANN models trained with genetic algorithms (GA-ANN) were also developed [25, 39]. Most ML strategies in Table 1 are used to predict and account for springback in steel and aluminium parts obtained by sheet bending. This may be connected with the fact that springback (related to the elastic recovery of the material after tool release) is one of the main sources of geometrical and dimensional inaccuracy in sheet metal formed components, but also because of the simple geometries used. Nevertheless, models were also built to predict wrinkling and necking defects [9, 32]. In general, the features for training the ML models are material parameters (namely elastic and/or plastic properties) and the initial sheet thickness. This can be related to the fact that the standards for commercial metal sheets specify only a minimum allowable value or a relatively large range of values for the mechanical properties. Nevertheless, there are models that also consider process parameters. Globally, the literature review reveals that ML techniques to predict defects in sheet metal forming take into account different set-ups. Although promising results were reported, techniques to predict more than one type of defect for different types of materials and forming processes have rarely been considered. To the best of the authors’ knowledge, there are currently no studies available in the literature regarding ML classification focused on defect prediction in sheet metal forming processes under variability, which is the main subject of the current work.

To accomplish the task for evaluating the best predictor of single sheet metal forming defects, seven ML classifiers were selected:Seven ML models were created for each of the two types of defects considered in each of the two forming processes under analysis, for three different materials. The models were built using Python v3.6.2 and related libraries, such as SciPy Ecosystem and SciKit-learn, using default values for the parameters of each classifier [5, 33]. The following sections provide a theoretical background concerning each of the studied classifiers.

Ensemble methods combine single classifiers, called base learners in this context, and are able to be more stable and predict better than single classifiers. The rationale is to reduce the bias and variance of the model to improve predictions. The goal is to build a model less noisy, more stable and less prone to overfitting. Since various base learners are used, each one can lead to a different prediction, where diversity among the base learners is a key aspect to ensemble performance. In this work, the following ensemble methods were used:

Taking into account that majority voting favours the use of odd numbers of base learners, both methods were tested using combinations of 3 and 5 base learners. All possible combinations of the single classifiers described in Sect. 3.1 were tested. These classifiers were also tested as meta-learner. All this leads to a total of 56 combinations, for majority voting, and 392 combinations, for stacking.

The sampling data were generated using numerical simulation results, obtained with DD3IMP in-house FEM code [27, 31]. The numerical models for the U-Channel and the Square Cup processes are shown in Fig. 3. In both cases the total punch displacement is 30 mm. The U-Channel corresponds to a bending process, and thus, the sheet is prone to significant springback. This type of defect is negligible in the Square Cup. In both cases, the occurrence of excessive thinning is an indicator of necking, which can also be controlled using the maximum equivalent plastic strain (EPS). Thus, two types of defects were considered: (i) springback and maximum thinning, for the U-Channel, and (ii) maximum EPS and maximum thinning, for the Square Cup. The tool geometry and the initial in-plane shape of the sheet are assumed fixed. Each process was simulated considering three steels commonly used in the automotive industry that cover a wide range of mechanical properties and applications: DC06 (mild steel), HSLA340 (high-strength low-alloy steel) and DP600 (dual-phase steel). The constitutive model considers: (i) elastic behaviour, Young’s modulus, E, and Poisson ratio, \(\nu\); (ii) plastic behaviour, yield stress, \(Y_0\), strength and hardening coefficients, C and n, and anisotropy coefficients \(r_{0}\), \(r_{45}\) and \(r_{90}\). The initial sheet thickness \(t_0\) is also considered. The variability in the input features related to the material parameters is typified by a normal distribution, with mean (\(\mu\)) and standard deviation (\(\sigma\)) values shown in Table 2.

Two input features related to process parameters were also considered: the friction coefficient and the blank holder force (BHF). The mean value of the friction coefficient is 0.144 for all materials, with \(\sigma\)/\(\mu =20\%\) [19]. For the BHF, two mean values were considered, which correspond to a lower and an upper level of the process window. For the U-Channel, the mean values used were 4.9 and 19.6 kN, while for the Square Cup, they were 2.45 and 9.8 kN. For each BHF value, the variability is \(\sigma\)/\(\mu\) = 5%. Thus, the variability of a total of 11 features was considered in analysis of both forming processes, for the three materials. In this context, random numerical simulations were performed within the range of variation of the input features (see Table 2). The numerical simulations using the mean values of the input features presented in Table 2 lead to a non-defective solution, which is considered as a reference solution. A defect occurs when the output value obtained from the random simulations is greater than that of the reference solution, whose values are presented in Table 3.

The data sets were split in training (70%) and testing data (30%). Data scaling was performed in all the data sets. A maximum of 2000 experiments (i.e. numerical simulation results) was considered for each material and forming process. The data were randomly shuffled in order to repeat the process 30 times (runs).

The F-score measure is used to evaluate the performance of both single and ensemble classifiers. This performance metric combines both precision and recall metrics and provides a break-even between them. The F-score is calculated as the harmonic mean between precision and recall, as follows:where precision takes into account the proportion of correctly classified instances (true positives (TPs)), among all the positive instances classified (true positives (TP) and false positives (FP)),and recall evaluates the percentage of correctly identified instances of a class (TP) among all the instances of a given class (true positives (TP) and false negatives (FN)):

Figures 4 and 5 show the evolution of the F-score values with the sampling data size, respectively, for the U-Channel and Square Cup forming predictive models, using 200, 500, 1000, 1500 and 2000 samples. In general, for both the U-Channel and Square Cup, the values of F-score increase with the increase in the sampling data size; exceptions include the cases “HSLA340-springback” and “DP600-springback“ with the LR classifier (see Fig. 4c, e), where the F-score is nearly constant. Accordingly, the highest values of F-score are generally obtained for 2000 samples, with few exceptions. Adding more training data would reduce variance but increase bias. Therefore, the performance analysis will focus on the results with 2000 samples, which is considered the critical sampling size for this problem [38]. Figure 6 shows the values of F-score, for the critical sampling size, obtained by the U-Channel predictive models in the cases of springback (Fig. 6b) and maximum thinning (Fig. 6c). The mean and standard deviation values of the F-score were obtained from 30 runs of each single classifier. The mean values of F-score range from 79.85% (springback prediction with DP600, using KNN—see Fig. 6b) to 93.63% (springback prediction with HSLA340, using MLP—see Fig. 6b), with relatively low standard deviation values. MLP is the highest performing classifier for predicting springback, with mean F-score values of 91.39% (DC06), 93.63% (HSLA340) and 92.76% (DP600), and LR is the highest performing classifier for predicting maximum thinning, with mean F-score values of 93.28% (DC06), 92.17% (HSLA340) and 91.38% (DP600). On the other hand, KNN is the lowest performing classifier, with mean values of F-score close to 80% for all materials and both defects (see Fig. 6b, c). The dissimilarity between the performances of the classifiers is more noticeable for the springback (Fig. 6b) than for the maximum thinning (Fig. 6c), in which all the classifiers except KNN are competitive.

Figure 7 shows the values of F-score, for the critical sampling size, obtained by the Square Cup predictive models of maximum EPS (Fig. 7b) and maximum thinning (Fig. 7c). The mean values of F-score range from 74.65% (maximum thinning prediction with DP600, using DT—see Fig. 7c) to 90.50% (maximum EPS prediction with HSLA340, using MLP—see Fig. 7b), with relatively low standard deviation values. The MLP is the highest performing classifier for predicting both the maximum EPS and maximum thinning in all materials, with mean values of F-score ranging from 84.37% (maximum thinning, HSLA340) to 90.50% (maximum EPS, HSLA340); also, the SVM classifiers show a relatively good performance. The NB, KNN and DT classifiers are the lowest performing classifiers.

The results show that, for a given type of defect, most classifiers show similar performances among the materials. For a given material, the difference in the performance of the classifiers between the two types of defects is more noticeable in the U-Channel than in the Square Cup. It is further noticed that MLP and KNN are, respectively, the highest and the lowest performing classifiers. This indicates that learning is an important step for finding the nonlinear decision boundaries. In fact, KNN is a “lazy learner” and it is harder to discriminate between the sought classes; thus, as a predictor, it becomes less useful. Finally, a Friedman test was conducted on the respective F-score values of each classifier, to check whether the performances of the single classifiers are significantly different; this nonparametric statistical test allows for performance comparison when dealing with several classifiers over multiple data sets [8, 11]. In this test, the null hypothesis states that all single classifiers performed equally. The rejection of this hypothesis means that differences between the performances of single classifiers are statistically significant. The obtained Friedman statistic, equal to 59.82 (i.e. corresponding to a p value equal to 4.89\(\times 10^{-11}\)), is greater than its critical values at significance levels of 5% (12.59) and 1% (16.81), which lead us to reject the null hypothesis.

Tables 4 and 5 present the best combinations of the classifiers obtained for majority voting and stacking ensembles, respectively. The mean and standard deviation values of the F-score were obtained from 30 runs of each ensemble. When comparing both tables, the use of stacking ensembles generally leads to an increase in the performance relatively to majority voting; the increase in performance corresponds to more than 1.5% in the case of the Square Cup (see cases “HSLA340—maximum EPS” and “DC06—maximum thinning” in Table 5). The opposite occurs only for the maximum thinning prediction in the U-Channel (see cases “U-Channel—Maximum Thinning” in Tables 4 and 5), where a maximum performance reduction in 0.4% is obtained for DP600. In the case of stacking ensembles, the area under the curve (AUC) metric was determined (see Table 5), which depicts the trade-off between the true-positive rate and the false-positive rate. The relatively high values of AUC (generally above 90% average performance) and their low variance (ca. 1%) confirm the high robustness of the stacking ensembles.

The smallest standard deviation values were obtained in both springback and maximum thinning U-Channel ensembles, respectively, stacking for DP600 (see Table 5) and majority voting for DC06 (see Table 4). The low values of standard deviations on all the 30 runs in all the experiments reveal a great deal of stability in the procedure as well as relatively good significance of the results.

The performance comparison between single and ensemble classifiers shows that the latter provide generally better defects predictors. In particular, in stacking ensembles it is expected that the meta-learner is prone to less errors by reducing error variance and thus generalizing well in the test set. Ensemble methods combine several machine learning techniques into one predictive model in order to decrease variance and bias, or improve predictions. The fact that the single classifiers already provide relatively high performances (ca. 85%, on average) and thus are stable learners; the best combinations of the classifiers obtained for the majority voting and stacking ensembles (ca. 90% average performance) do not show to outperform significantly the single classifiers. On the other hand, the overall increase in the ensembles’ performance is coupled with a lower variance, which promotes robustness and stability of the procedure and allows a better bias–variance trade-off.