nach oben

Journal of Intelligent Manufacturing

Erschienen in:

Open Access 17.05.2021

Review of image segmentation techniques for layup defect detection in the Automated Fiber Placement process

A comprehensive study to improve AFP inspection

verfasst von: Sebastian Meister, Mahdieu A. M. Wermes, Jan Stüve, Roger M. Groves

Erschienen in: Journal of Intelligent Manufacturing | Ausgabe 8/2021

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Patentsuche

Aus

Abstract

The aerospace industry has established the Automated Fiber Placement process as a common technique for manufacturing fibre reinforced components. In this process multiple composite tows are placed simultaneously onto a tool. Currently in such processes manual testing requires often up to 50% of the manufacturing duration. Moreover, the accuracy of quality assurance varies significantly with the inspector in charge. Thus, inspection automation provides an effective way to increase efficiency. However, to achieve a proper inspection performance, the segmentation of layup defects need to be examined. In order to improve such defect detection systems, this paper performs a comprehensive ranking of segmentation techniques. Thus, 29 statistical, spectral and structural algorithms from related work were evaluated based on nine substantial criteria as assessed from literature and process requirements. For reasons of determinism and easy technology transferability without the need of much training data, the development of new Machine Learning algorithms is not part of this paper. Afterwards, seven of the most auspicious algorithms were studied experimentally. Therefore, laser line scan sensor depth maps from fibre placement defects were utilised. Furthermore noisy images were generated and applied for testing algorithm robustness. The test data contained five defect categories with 50 samples per class. It was concluded that Adaptive Thresholding and Cell Wise Standard Deviation Thresholding work best yielding detection accuracies mostly $> 97$%. Noteworthy is that influenced input data can affect the detection results. Feasible algorithms with sensible parameter settings were able to perform reliable defect segmentation for layed material.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Introduction

Nowadays, lightweight structures are widely used in aerospace manufacturing. Examples of an increasing demand for these lightweight components are the Airbus A350 XWB or Boeing 787 wingcover and fuselage production (Marsh 2010; McIlhagger et al. 2020). Carbon Fiber Reinforced Plastic (CFRP) offers superior stiffness and strength properties compared to metallic materials. For this reason lightweight structures are often made from CFRP. The production of these mostly complex lightweight structures is usually quite expensive. To make manufacturing economical, fast and efficient production techniques are essential. On this account, the relatively novel Automated Fiber Placement (AFP) technique is increasingly used in industry. This is the reason for its choice for further investigation of the inspection process, in this paper. To meet aerospace’s high safety requirements a visual inspection step follows the fibre layup process.

Today, this manual inspection takes typically between 32% (Rudberg et al. 2014) to 50% (Eitzinger 2019) of the overall production time. Due to the manual inspection process, the inspection quality is often insufficient with regard to the specifications. This aspect provides great potential for improvements in terms of quality and speed.

The Laser Line Scan Sensor (LLSS) is frequently used in research and development for the inline inspection of AFP processes. For this reason, we are focussing on grey scale depth images from such a sensor. A LLSS is based on the principle of triangulation to obtain geometrical data. Therefore a laser beam is projected onto a surface and then reflected back to a camera sensor. This sensor has a slightly different position than the emitting laser.

One critical aspect in the automated inline inspection is the robust and reliable segmentation of defects within a sensor image. Previously, Hanbay et al. (2016), Mahajan et al. (2009) and Kumar (2008) summarised some research from the field of fabric defect segmentation in the textile industry. Additionally, Bulnes et al. (2014) mentioned predominantly periodic defects as critical for the web industry. Because textiles are somehow very similar to CFRP, research from this field can be very helpful for the algorithm selection. Later on Sacco et al. (2018) and Zambal et al. (2019) published Neural Network based defect detection methods for laser scanner depth images of AFP defects. Moreover, Tabernik et al. (2019) have investigated a deep learning system for crack segmentation in components. In particular, they emphasised the necessity of a small initial setup data set for establishing a system in an industrial application. These deep learning methods work quite well (Du et al. 2020), but face the challenge of determinism and traceability of Neural Network decisions (Lee et al. 2021). The categories of segmentation algorithms without the use of advanced more abstract models can be visually verified in terms of their correct operation (Joshi et al. 2018). Simple image metrics might be sufficient as an indicator for evaluating the validity of a calculation (Meng et al. 2020). Such an evaluation and traceability of the machine decision is much more difficult for Neural Networks or other advanced models (Lee et al. 2021), which is the reason for categorically excluding them in this study. Meister et al. (2020) announced our first results on defect detection algorithms for fiber placement processes. In this the focus was on the general overview of feasible algorithms.

Based on this previous work, a detailed investigation of suitable algorithms with regard to different geometrical groups of layup defects and a varying quality of the input data is necessary.

Within this postulation we like to focus on defect detection for different geometrical characteristics of defects. This image segmentation step is basically the first task in the inspection of composite parts. This serves to reduce the amount of data and prepare the image data for subsequent analysis procedures. Prior to the determination of the defect types or geometrical measures, as many as possible of these manufacturing deviations have to be detected on the produced component. Afterwards, in a following stage, a detailed analysis of the conspicuous image regions is carried out using techniques of varying complexity. Such a sequential inspection approach is similarly used from of Kuo et al. (2016) for the inspection of light emitting diodes. The detailed defect analysis and interpretation will be considered separately in further research and is therefore not part of this paper. In addition, the presented procedure can serve as a partially automated input generator for the synthesis of further defect images in AFP inspection, as described in our publication from Meister et al. (2021) or in similar fields as mentioned from Jain et al. (2020). For this reason the following research questions are selected for this publication:

Which algorithms are feasible to perform rapid defect detection for previous defined layup defect types under consideration of various geometrical clusters?
What is the actual performance of feasible image segmentation methods under consideration of a varying data quality?

Regarding these research questions, this paper uses findings from previous fabric inspection and fiber layup defect detection research and reviewed these algorithms in the field of AFP defect segmentation. For this, various geometrical clusters of defects are examined. Thus, our research provides a novel perspective on segmentation algorithms and their application to fibre placement inspection. The findings of this postulation should be beneficial for manufacturers of inline inspection systems for AFP processes. Thus, these results can support their development and configuration of defect detection algorithms for various materials. Furthermore, the findings will support the certification process of such systems, especially in the aerospace industry. This article initially presents the related research on sensor and algorithm development. Subsequently all algorithms from the literature are summarised and evaluated according to a defined procedure. The seven most promising algorithms are then implemented and studied based on exemplary fibre layup defects.

This section provides an overview of the state of research and industrial developments of the manufacturing process, data acquisition techniques as well as algorithms for image smoothing and defect segmentation.

Manufacturing process

Following, the corresponding manufacturing process is explained. To begin with, various Fiber Placement techniques are currently available on the market. Very common methods are the Automated Fiber Placement (AFP), Dry Fiber Placement (DFP), Automated Tape Laying (ATL) and Direct Roving Placement (DRP). (Maass 2012; Lengsfeld et al. 2014; Grohmann et al. 2016) These processes apply CFRP material layer by layer onto a tool. (Campbell 2004) explained this procedure, which is also schematically shown in Fig. 1.

AFP is a technique which is needed to manufacture complex composite structures. Nowadays, this technique gets more and more established in industrial aerospace manufacturing. This means, that the process is quite novel, but also used in industry. With the aim to realise a good transferability of the research results, we selected this method for further investigations. During the AFP process multiple narrow preimpregnated material stripes (tows) are placed along a previously programmed path (course) (Oromiehie et al. 2019). Each tow is supplied from an individual spool. The supplied material consists of the prepreg material itself and a release film. This release film is removed during the placement process and stored on another spool. Within the AFP process, composite material e.g. carbon prepreg material is fed into an effector. This effector guides the material to the tool’s surface. Then, the material is heated at the moment of tow placement, to achieve better tack properties (Lengsfeld et al. 2014). Following to the material deposition, a compaction roller presses the material onto the mold. Thus, each component is made from many CFRP prepreg layers (Campbell 2004). This previously explained AFP process can be used to manufacture various part geometries. For this reason, Rudberg (2019) expects that the AFP method will be used more and more frequently in future applications.

Various defects possibly occur during fibre layup. These defects are often directly related to the lay up process itself (Oromiehie et al. 2019). Harik et al. (2018) have investigated the link between the AFP defects and the process planning, layup strategies and machining. Furthermore, Potter (2009) has analysed factors for the variability in the AFP production. Referring to Potter (2009) and Harik et al. (2018), all defects which can occur during fibre layup result in geometrical changes and deviations from an accurate lay up surface. Hence, common AFP defect types are wrinkles, twists, gaps, overlaps and foreign materials (foil) (Harik et al. 2018; Heinecke and Willberg 2019; Oromiehie et al. 2019). A sketch and a corresponding exemplary LLSS scan image is presented in the Fig. 2 for each defect type. The associated geometric measures and characteristics of these defects are summarised in the Table 1.

Table 1

The table presents an overview of the geometrical measures of the considered defects from the Fig. 2

	Wrinke	Twist	Gap	Overlap	For. Mat.
Typical ratio: (l/w)	0.5 to 2	5 to 10	$\le $ course length	$\le $ course length	unk.
Thickness deviation (+/-)	$\ge $ 3x CPT (+)	$\ge $ 2x CPT (+)	$\le $ 1x CPT (-)	$\le $ 1x CPT (+)	unk.
Source	Harik et al. (2018); Oromiehie et al. (2019)	Harik et al. (2018); Heinecke and Willberg (2019); Oromiehie et al. (2019)	Nardi et al. (2018); Harik et al. (2018); Heinecke and Willberg (2019); Oromiehie et al. (2019)	Nardi et al. (2018); Harik et al. (2018); Heinecke and Willberg (2019); Oromiehie et al. (2019)	Harik et al. (2018); Heinecke and Willberg (2019)

The range of the length-to-width (l/w) ratio is given here, due to the large variance of the geometry within each defect category. CPT in this application case is about 0.125 mm. For the thickness measure + indicates an increase in thickness and - means a decrease in thickness

The wrinkle and twist defects have varying but distinct topologies. Both types of these defects protrude from the layup surface. This results in large height differences and form clear defect edges. In longitudinal direction wrinkles form one clear edge. Twists in contrast show a very small growth in altitude over their length. Gaps and overlaps are very similar to each other regarding their geometrical characteristics. These two defects are very flat and hardly show any topology changes. Gaps reveal two slight edges at the beginning and end of this defect, in transverse to the fibre direction. However, overlaps form three small edges in this direction, since these defect types are mostly a combination of a gap and a tow overlap. In contrast, gaps and overlaps have almost no edges apparent along the tows. Due to their similarities, the differentiation between these two types is often quite difficult. The thin shape of these defects gives the opportunity to analyse algorithms for this scenario. These defect types are also often used as example defects by other researchers (Harik et al. 2018; Heinecke and Willberg 2019; Oromiehie et al. 2019). Thus (Nardi et al. 2018) emphasises especially the disturbing influences of gaps and overlaps in the AFP prepreg layup for fibre metal laminates. Furthermore, foils are very common foreign materials in manufacturing processes. They show a very different reflection behaviour compared to the deposited fibre material (Miesen et al. 2015; Potter 2009). Following, feasible techniques for the recording of these defects are explained.

Sensors for data acquisition

Nowadays, the inline inspection for AFP processes is a highly discussed topic in research and industry. For this reason, various sensors are investigated to record the required inspection data. Sun et al. (2020) provides a comprehensive overview of the currently available systems and their performances. Accordingly, the Fraunhofer Institute for Integrated Circuits (IIS) investigated polarisation camera based systems (Atkinson et al. 2018; Schöberl et al. 2016). Furthermore, the National Aeronautics and Space Administr. (NASA) Gregory and Juarez (2018) as well as the Institute of Production Engineering and Machine Tools - University Hanover (IFW) focused on thermographic imaging inspection (Denkena et al. 2016; Schmidt et al. 2019a). Both types of sensors capture only 2D images.

Thus, InFactory Solutions (Weimer et al. 2016), Profactor (Gardiner 2018) and Danobat Composites (Black 2018) have developed LLSS based systems for inline Quality Assurance (QA) of AFP processes. These companies use a single LLSS for monitoring of the entire course. In contrast, Electroimpact (Cemenska et al. 2015) applied multiple LLSS systems to observe each individual tow.

A big advantage of these LLSS systems is the capability to provide topographical information of a surface. This may be the reason of the success of this measurement principle for the AFP inspection (Weimer et al. 2016). Schmitt et al. (2008), Schmitt et al. (2007) began studies on LLSS based methods for the contour scanning of fabrics and preforms, in 2007. They also investigated edge detection methods for the determination of preform misplacement errors. They observed sub-pixel accuracy for the contour measurement and showed that a LLSS is a suitable system for the fabric and preform inspection. Subsequently in 2012, Faidi et al. (2012) investigated laser triangulation systems for CFRP manufacturing. They aimed to find and measure wires with a diameter between 0.5 mm and 1 mm, which were incorporated into CFRP laminates. Miesen et al. (2015) suggested a method to sense defects with a point measurement laser displacement system. Additionally, they discussed influencing factors for deviations in their research and analysed the accuracy of such a system. Furthermore, they presented various defect types and their corresponding geometrical dimensions. Tonnaer et al. (2017) also showed a technique for LLSS based monitoring of AFP processes. They investigated the measurement precision of Otsu’s algorithm for the edge detection on LLSS depth images (Otsu 1979). Further methods for data usage and processing are described below.

Image processing techniques

This section describes related work for defect detection and image smoothing, in order to process the sensor data. Hanbay et al. (2016), Mahajan et al. (2009) and Kumar (2008) have presented plenty of algorithms for fabric defect segmentation in the period between 2008 and 2016. The corresponding methods mostly excluded the use of Neural Networks. In the main, they all clustered the methods in structural, statistical and spectral approaches as well as other more advanced techniques and discussed their individual strengths and weaknesses. Hanbay et al. have mentioned, that in particular the statistical and the spectral analysis methods lead to good fabric defect detection results. They have also pointed out, that spectral techniques work better for regular texture patterns like fabrics. Mahajan et al. concurred with this view. Kumar goes so far as to suggest, that Gabor filters perform well individually, but can lead to even better results in combination with other methods. Meister et al. (2020) summarised various defect segmentation algorithms from other computer vision fields and gave a rough overview of their advantages and limitations.

Further crucial tasks in the field of image processing are the smoothing of sensor data and the adjustment of pixel values. For this purpose, the Contrast Limited Adaptive Histogram Equalization (CLAHE) algorithm is a suitable method for image contrast equalisation, as Meister et al. (2020, 2021) have already explained. This technique was developed in the mid 80’s and Pizer et al. (1986) used it first in the field of medical imaging. This CLAHE method considers regionally limited small image areas called ’tiles’ and performs a histogram equalisation for each of these regions. To counteract the effect of image noise over amplification, the contrast is limited to a defined score. Pixel values above this relative threshold are shifted to a corresponding neighbouring bin.

As mentioned above, plenty of algorithms and sensor systems from various fields are already available. Especially algorithms from textile inspection seem feasible to adapt to our use case. Within this paper we like to apply a suitable choice of algorithms on LLSS depth data with varying data quality. In addition, we gather types of defects into appropriate clusters and perform a detailed analysis on the defect segmentation for them. Subsequently, the methodology for the algorithm evaluation is discussed.

Methodology

The following section describes the experimental setup, the algorithm selection approach and their actual implementation as well as the procedure for results evaluation. Within this paper we use the abbreviations from Table 2 for the algorithms. Moreover, Table 3 give the notations corresponding to the results evaluation.

Table 2

List of abbreviations regarding the notations of the algorithms investigated

Abbreviation	Definition
AT	Adaptive Thresholding
CT	Cell Wise Standard Deviation Thresholding
FI	Full Image
GT	Gradient Thresholding
GB	Grid Based
GF	Gabor Filter Segmentation
IP	Image Projection
MS	Morphological Segmentation
OT	OTSU Thresholding

Table 3

Notations corresponding to the results evaluation

Symbol	Definition
d	Detection accuracy
p	Position accuracy
N	False negatives rate
$P_{n}$	False positives rate (number)
$P_{a}$	False positives rate (area)
$A_{e}$	Excess area
$A_{o}$	Overlap area
$A_{sd}$	Superfluous recognised defect area per image
$A_{ed}$	Excess defect area per image
$A_{od}$	Overlap defect area per image
$A_{gt}$	Ground truth defect area per image
$n_{d}$	Number correct detected defects
$n_{sd}$	Number superfluous detected defects per image
$K = n_{gt}$	Number ground truth defects
C	Number detected defects with associated ground truth defects
$E_{d}$	Euclidean distance: Estimated defect location $\leftrightarrow $ Ground truth location

Experimental setup

For the performance tests of the desired segmentation algorithms a manageable and heterogeneous set of defect types needs to be selected. Due to the varying defect characteristics from Sect. 2.1, we choose wrinkles, twists, gaps, overlaps and foil as foreign materials for the experiments. Figure 2 presents these defects and the Table 1 gives the corresponding geometrical measures. It should be noted, that other defect types were not examined in this paper.

With the aim to review image analysis techniques for the AFP application case, representative data needs to be recorded. This collection of fiber layup images must be generated reproducibly and should be representative regarding the actual fiber placement process. For this reason, a setting which is not influenced by disturbing influences from the manufacturing process like heating radiation, undesirable contamination or geometric tilting of the lay up machine was used. A suitable test setup is presented in the Fig. 3. This arrangement consisted of an articulated robot from KUKA, the Automation Technology GmbH (AuTech) - C5 Sensor (Automation Technology GmbH 2019) and a CFRP prepreg material sample which contained the defects from the Fig. 2. The linear moving robotic arm records the data along the entire sample, with a velocity of 200 mm/s.

The AuTech C5 Sensor captured 4096 (w) x 625 (h) px, 16-bit grey scale depth images from the 250 x 150 mm fibre layup sample. The width of the image indicates the maximum resolution of the sensor in width direction. The height resolution is affected by the exposure time per pixel row and the time between frames. On this account, the image resolution decreases with increasing exposure time, assuming the same sample size and scanning speed. A laser voltage of 5V and the FIR-PEAK laser line detection mode (Automation Technology GmbH 2014) were used to calculate precise topological data. Within this device the FIR-PEAK algorithm is implemented as a derivative filter, which detects the zero crossing point of the first derivative of the laser intensity image. The data was transmitted via an ethernet connection using the GenICam protocol (EMVA 2009).

The image processing was carried out on a computer with Intel Xeon Gold 5122 @ 3.60 GHz CPU, 48 GB RAM, a NVIDIA Quadro P6000 GPU and OpenCV 3.4.1 (Bradski 2000) with Python. The image analysis processes were preferably executed on the CPU. The GPU was only used for the visual display. In addition the following section explains the selection of the algorithms to be investigated and the procedure for evaluation.

Evaluation of detection methods

Subsequently, the evaluation procedure for segmentation algorithms is presented. With regard to the defect characteristics described above, edge-based features can be a valuable image feature for most of the chosen defects. In addition, the regular structures of gaps and overlaps may be well represented by frequency-based attributes.

Table 4

Overview of the theoretical review of algorithms for defect segmentation. A total of 29 algorithms from the literature are gathered and analysed based on nine characteristics found in the related research

		Criteria
Algorithms		Number of Invariances	Calculationspeed	Implementationeffort	Detection accuracy	Falsepositiverate	Falsenegativerate	Localisingaccuracy	Robustness	Adaptability (to input)	Weightedsum interval:weights+/− 0.5	Reference
	Expected weights ($w_{e}$) [1,5]	2	5	1	5	3	5	3	4	5
	Edge detection	3	5	4	3	3	4	3	3	3	[3.44,3.51]	Lee et al. (2014); Kaur and Sharma (2016); Dhanachandra and Chanu (2017); Sarkar and Padmavathi (2018)
	Region operations	0	4	4	2	2	2	2	2	2	[2.20,2.30]	Neubert and Meinel (2003); Lee et al. (2014); Masood et al. (2015); Kaur and Sharma (2016); Mustafa and Hassan (2016); Dhanachandra and Chanu (2017)
	Laplaceian of Gaussian	3	4	4	2	1	1	3	2	2	[2.23,2.33]	Lee et al. (2014)
	Projection (horz./vert.)	1	5	4	4	3	3	4	3	3	[3.41,3.52]	Kumar and Pang (2002)
	Local Binary Patterns	1	4	4	4	3	4	4	4	2	[3.38,3.49]	Tajeripour et al. (2007); Liu et al. (2017); Xie (2008)
	Principle Comp. Analysis	0	5	4	3	2	2	3	2	1	[2.43,2.58]	Liu et al. (2017); Shanbhag and Deshmukh (2013); Lee et al. (2019)
	Linear discr. analysis	0	5	2	3	2	2	3	2	1	[2.36,2.49]	ALGLIB-Project (2019)
	STD per cell thresholding	2	4	4	3	3	4	4	3	2	[3.18,3.25]	Lee et al. (2014); Hanbay et al. (2016)
	Histogram approach	2	4	4	3	3	3	1	3	2	[2.75,2.82]	Lee et al. (2014); Hanbay et al. (2016); Kaur and Sharma (2016)
	OTSU	2	5	4	4	3	4	4	3	4	[3.77,3.85]	Otsu (1979); Ng (2004); Norouzi et al. (2014); Lee et al. (2014)
	Adaptive Thresholding	3	5	4	5	5	5	4	4	4	[4.45,4.51]	Kumar (2008); Mahajan et al. (2009); Norouzi et al. (2014); Lee et al. (2014); Masood et al. (2015)
	Fractal dimension	1	4	3	4	2	1	1	3	3	[2.55,2.68]	Kumar (2008); Mahajan et al. (2009)
	First-/ Sec. order statistic	2	4	4	3	3	3	4	3	2	[3.03,3.12]	Mahajan et al. (2009)
	Texture energy by Laws	0	3	4	3	3	3	2	3	3	[2.72,2.82]	Mahajan et al. (2009); Meister et al. (2018)
	Cross correlation analysis	0	3	2	4	2	2	2	3	1	[2.26,2.36]	Kumar (2008)
	Auto correlation	0	2	3	4	4	3	1	3	1	[2.37,2.49]	Hanbay et al. (2016); Xie (2008)
	Morphological operations	3	4	3	4	4	4	4	3	4	[3.76,3.80]	Kumar (2008); Lee et al. (2014); Hanbay et al. (2016); Kaur and Sharma (2016); Mustafa and Hassan (2016)
	Co-occurrence matrix	1	3	3	4	3	3	3	2	1	[2.55,2.65]	Kumar (2008); Mahajan et al. (2009); Hanbay et al. (2016); Xie (2008)
	Eigen filters	0	4	3	3	3	3	1	1	1	[2.18,2.28]	Kumar (2008); Cattin (2016)
	Rank-order histogram	2	5	4	3	3	3	3	3	4	[3.39,3.46]	Kumar (2008)
	Local linear transforms	0	0	0	0	0	0	0	0	0	[0,0]	Kumar (2008)
	(Discrete) Fourier Trafo.	3	4	3	4	3	4	1	2	2	[2.94,3.04]	Kumar (2008); Mahajan et al. (2009); Hanbay et al. (2016)
	Optical Fourier Trafo.	−	−	−	−	−	−	−	−	−	−	Kumar (2008)
	Windowed Fourier Trafo.	3	4	2	4	3	3	3	3	2	[3.08,3.15]	Kumar (2008); Mahajan et al. (2009); Hanbay et al. (2016)
	Gabor transform	2	4	3	5	4	4	4	4	4	[3.96,4.03]	Kumar (2008); Mahajan et al. (2009); Hanbay et al. (2016); Rodriguez et al. (2020)
	Optimized FIR Filters	1	3	2	4	2	2	4	4	1	[2.60,2.75]	Kumar (2008)
	Wigner Distributions	1	1	3	4	2	2	4	3	4	[2.67,2.77]	Kumar (2008)
	Wavelet Transform	1	3	2	4	4	4	3	4	2	[3.17,3.28]	Kumar (2008); Mahajan et al. (2009); Hanbay et al. (2016)
	Hotelling T2 statistic	1	3	1	4	4	4	4	3	2	[3.10,3.22]	Lin (2007)

Assessment results are given in bold. Italic row gives the weights for the weighting of the assessment criteria for evaluation. The selected algorithms for further investigations in this paper are given in Bold & Italic The table lists all algorithms found during the literature search. The importance measure expected value $w_{e}$ for each category varies between unimportant (1) and very important (5). The rating $v_{a}$ indicates the conformity of each algorithm with the considered analysis categories. It ranges from 0 (no match) to 5 (absolutely correct). “−” indicates: Insufficient information available. The algorithms further investigated in this paper are highlighted in bold italic

Some defect segmentation algorithms are already known from similar applications like textile inspection, which have been discussed in Sect. 2.3. These algorithms are theoretically evaluated here, on the basis of the literature. This summary from the Table 4 contains all structural, statistical and spectral algorithms found during the literature review in the fields medical imaging, autonomous driving and textile inspection. The individual evaluation criteria are listed above each column. The corresponding references are given in the right column. With the aim to review these algorithms, the entire inspection process chain needs to be taking into account. As a consequence, the defect segmentation must be calculated quickly and find most defects, but a small positioning error during the image segmentation is tolerable. For these reasons, the nine performance characteristics number of invariances, calculation speed, implementation effort, detection accuracy, false positive rate, false negative rate, localising accuracy, robustness and adaptability to input were chosen and weighted according to their importance. Therefore, the weighting of the individual criteria is specified subjectively, based on the process and system requirements from Sect. 2.1. Subsequently, the algorithms were then evaluated against these criteria. The accumulated rating $v_{a}$ is calculated as presented in the Eq. 1. For each considered criterion, $w_{i}$ describes the absolute weighting within the interval [1,5]. Further, $c_{i}$ expresses the algorithms rating for the individual criteria.

$$\begin{aligned} v_{a} = \sum _{i=1}^{9} \frac{w_{i} c_{i}}{w_{all}} \text {with} w_{all} = \sum _{i=1}^{9} w_{i}, w_{i} \in {\mathbb {R}} \end{aligned}$$

(1)

Due to the subjective weighting and evaluation of the algorithms based on the literature, it is essential to consider the robustness of the assessment results. Thus, the premature rejection of potentially suitable algorithms shall be avoided. Therefore the individual weights $w_{i}$ were randomly varied within the defined value range $\mathbf {[w_{e}-0.5}$ $\mathbf {\le w_{i}}$ $\mathbf {\le w_{e}+0.5]}$. Based on this, we performed 20 individual Monte Carlo observations with the expected value $w_{e}$. The corresponding results are presented in the Table 4. In summary, a weighted performance value was calculated. The ten algorithms with the best rating, which implies a mean performance value $\ge 3.21$, were considered for the following investigations. The defect segmentation with Gradient Thresholding (GT), Image Projection(IP), Cell Wise Standard Deviation Thresholding (CT), OTSU Thresholding (OT), Adaptive Thresholding (AT), Morphological Segmentation (MS) and Gabor Filter Segmentation (GF) algorithms have been identified for continuing study in this paper. These methods are highlighted in the Table 4 and were applied for the validation. This selection and the exclusion of certain algorithms is briefly justified in the following.

Due to the traceability and an easy technology transfer, it is important to use simply adaptable algorithms which do not require comparative data. This causes the Local Binary Patterns segmentation to be discarded. The Rank- order histogram method sorts the bins of grey values due to the number of contained pixels. In this manner, the distribution of pixel values can be displayed, but it is not possible to distinguish between areas with and without defects. Apart from this, it is a global criterion that does not allow the localisation of prominent areas. On this account, the Rank- order histogram can not be used for a proper defect detection. Thus, the algorithms Local Binary Patterns and Rank- order histogram were rejected despite a high scoring rate.

In addition, we would like to emphasise that this paper aims at comparing various appropriate methods for defect detection in the AFP process. Accordingly, algorithms with different operating principles and varying degrees of complexity are considered. Thus, for different methods, this can lead to varying efforts in identifying suitable algorithm parameters. However, for the performance comparison in this paper only those configurations are considered which yield the highest detection accuracies for each algorithm. The operating principle of the chosen algorithms is described below.

Implementation of selected algorithms

This subsection briefly explains the implementation of the previously selected algorithms and their operating principle. The CT method determines the ’mean’ and the ’standard deviation’ of the pixel values of each individual cell in a large, image-spanning grid. Cells with an ascertained standard deviation above a given threshold value are marked as a conspicuous region. These are then used to estimate and localise the defects.

For signal evaluation using the wavelet transform, a base function is applied to interpret the input signal. To analyse an image, a matrix with the corresponding base function is shifted over the image and thus individual areas are examined. A popular and easy to use base function is the Gaussian function, which is used for the so called Gabor filtering. On this account, we decided to apply the OpenCV implementation of the GF (OpenCV 2018a), which is also frequently used in the textile inspection.

For the GT a Sobel filter in combination with masking operation was carried out. Therefore, the convolution $G_{i} = S_{i} *A$ is performed on the image matrix A. Thus, two separate convolution masks $S_{i}$ regarding the images width and height directions are applied. The results are aggregated to Eq. 2.

$$\begin{aligned} G = \sqrt{G_{x}^{2} + G_{y}^{2}} \end{aligned}$$

(2)

Finally, the edges were approximated by applying a binary thresholding operation to the matrix G (OpenCV 2019a).

The IP method first computes a dimensional reduction vector $\mathbf {r_{k}} = (r_{k,a},...,r_{k,p})$. Therefore, every a-th entry of $\mathbf {r_{k}}$ is calculated for each row i $\mathbf {r_{r}} (i)$ by Eq. 3. Analogous for every column j $\mathbf {r_{c}}(j)$ is determined using Eq. 4 (OpenCV 2019b).

$$\begin{aligned} r_{r,a} (i, j=const.) = \sum _{i=0}^{m} (I (i,a)) \end{aligned}$$

(3)

$$\begin{aligned} r_{c,a} (i=const., j) = \sum _{j=0}^{n} (I (a,j)) \end{aligned}$$

(4)

This IP algorithm treats each image row or column as an individual vector and accumulates the corresponding values. Subsequently, a matrix of ones with size equal to the original image $J_{n,m}$ is multiplied by the projection vectors $\mathbf {r_{k}}$. This can be expressed as Eq. 5

$$\begin{aligned} P_{r} = J_{n,m} r_{r} \text { and } P_{c} = J_{n,m} r_{c} \end{aligned}$$

(5)

Afterwards, the total projection matrix P is computed by $P = P_{r} + P_{c}$. With the aim to estimate the defect regions a binary thresholding was applied to this matrix P.

The OT searches for the threshold t which minimises the variance between two classes. This is defined as the weighted variance of these two classes (OpenCV 2018b). This again can be written as Eq. 6. For this $w_{i}$ are the weights and $\sigma _{i}^{2}$ are the variances of these classes Otsu (1979).

$$\begin{aligned} \sigma _{w}^{2}(t) = w_{0}(t) \sigma _{0}^{2}(t) + w_{1}(t) \sigma _{1}^{2}(t) \end{aligned}$$

(6)

AT converts an input image I(i,j) into a binary image $I_{b}$. This is expressed in Eq. 7 using T(i, j) as a pixel-based individual threshold value (OpenCV 2018b).

$$\begin{aligned} I_{b}(i,j) = {\left\{ \begin{array}{ll} 255 &{} I(i,j) \ge T(i,j) \\ 0 &{} other \end{array}\right. } \end{aligned}$$

(7)

Lastly, the MS algorithm was implemented here as a morphological closing method. Hence, a dilation operation combined with a subsequent erosion procedure was calculated. This can be expressed as $(I \oplus K) \ominus K$ with the input image I and the morphological kernel K (OpenCV 2018c).

Image pre-processing and parameter adjustment

Subsequently, the image pre-processing procedure and the parameter optimisation approach is explained. Initially, the image borders were cropped to compensate for artefacts from the image acquisition. Due to the large number of zero value pixels within the raw image, a pre-processing is necessary to reduce their influences. For this purpose, first the images were dilated to increase the number of information-containing image pixels. Afterwards a contrast equalisation was carried out in order to utilise the entire available value range. Gaussian filtering was then applied to smooth the edges from the dilation step. Finally, the image was resized to a constant size of $1000 \times 1000$ px. This raw image pre-processing is shown in the flow chart in the Fig. 4. For the initial cropping step, the image borders were cropped by 60 px at the top and the bottom as well as 40 px at the left and right edge to eliminate artefacts from the data recording. Regarding the image dilation the kernel size $k_{size} = 3 \times 3$ was chosen from preliminary tests. In order to find the configuration with the best performance, the number of dilation steps were here varied from 4 - 12. Then, the contrast equalisation was achieved using the CLAHE algorithm with varying tile sizes, ranging from 5 - 25 px squared. Further, its clipping limit was varied within the range of 2 and 26, to find the most effective configuration. Afterwards an image padding with the same parameters as in the previous cropping step were applied, to compensate for the resulting error in the defect position. For the subsequent Gaussian filtering the kernel size was set to $k_{size} = 5 \times 5$ with $\sigma = 0.5$. These CLAHE configuration ranges from above were based on the settings from Ma et al. (2017) and Muniyappan et al. (2013).

With respect to our optimisation strategy, the settings of the pre-processing are expected to have a great influence on the defect detection. For this purpose, every feasible combination of a pre-processing setting and a defect detection algorithm configuration was tested crosswise by the computer. The value ranges for the algorithms tests were determined on basis of the corresponding references for each algorithm from Table 4. The most suitable parameter combination for pre-pressing and defect segmentation for this application case were experimentally determined for each algorithm, as stated above. The related settings are presented in the Table 5 for the pre-processing and in the Table 6 for the defect segmentation algorithms. Each parameter was modified with a given increment. The configurations with the best performance for a given combination of certain settings for the pre-processing and segmentation algorithms were used for further investigations in this paper. The respective groups of defects were taken into account for this parameter optimisation.

Table 5

The parameters used for pre-processing are summarised

Parameter	Interval	Step size
Dilation steps	[4,12]	1
CLAHE - Clip Limit	[2,26]	6
CLAHE - Tile size	[5,25]	4
Thresh Img. Dilat. x - Kernel	[3,9]	1
Thresh Img. Dilat. y - Kernel	[3,9]	1
Thresh Img. Dilat. Steps x	[6,12]	2
Thresh Img. Dilat. Steps y	[6,12]	2

These are incrementally varied with a given step size to determine the parameter configuration that produces the best detection results for each segmentation algorithm

Table 6

The table shows the adjustable parameters with the permissible value ranges and the step sizes for the setting of each segmentation algorithm

Algorithm	Parameter	Interval	Step size
GT	Kernel size sobel filter	[3,7]	2
	Sobel threshold x	[120,150]	10
	Sobel threshold y	[120,150]	10
CT	Cell size x	[10,40]	10
	Cell size y	[10,40]	10
	Cell overlap	[0,0.5]	0.5
	Threshold STD	[4,10]	1
IP	Numb. rot. x	[0,3]	1
IP	Threshold	[90,140]	10
OT	None	None	None
AT	Thresh. Offset	[-40,-10]	4
GF	Kernel size x	[7,31]	4
	$\sigma $	[1,10]	1
	$\lambda $	[1,10]	1
	Threshold Gabor filter	[5,10]	1

Regarding the manufacturing process the lowest possible number of undetected manufacturing defects is desirable. For this reason, the reduction of undiscovered defect areas was chosen as a criterion for the optimisation. This in turn means the decrease of false negative values. As already mentioned in Sect. 1, the defect segmentation examined here is only the first stage of the overall defect analysis, within the entire inspection system. Therefore an excess area value up to 100% can be tolerated. This implies, that the segmentation area can be up to double the area defined as ’ground truth’.

This iterative parameter optimisation presented here, was conducted for each segmentation algorithm investigated. The optimisation criterion for this step was the maximisation of the defect detection accuracy, ideally up to its optimum of 100%. Following to both optimisation steps, the results of each algorithm under review were evaluated and then compared.

The aim here was to approximate the parameter setting with the best overall detection performance, with respect to the previously mentioned optimisation constraints. The findings discussed in this publication are always based on the most suitable configuration for the particular case under consideration.

All the investigated algorithms were applied to the entire scan image directly, which is subsequently termed the Full Image (FI) variant. Additionally, a Grid Based (GB) variant was implemented. Here, the pre-processed measurement image with $1000 \times 1000$ px was split into nine equally sized partial images of $333 \times 333$ px each. An examined algorithm was executed for each of these partial images. In this way slight differences in the height of the defect sample, respectively resulting variations in image brightness, were to be reduced. Below, the evaluation of the individual performance characteristics is outlined.

Evaluation of results

For the analysis of the detection performance, eight attributes were examined. These are based on the criteria available in the literature and presented in the Table 4. Therefore 50 defect samples per class were used as the maximum amount of test data currently available. For the automated evaluation of the algorithms segmentation behaviour, the previously acquired image data was manually labeled. For this task, the Python tool ’LabelImg’ (Tzutalin 2015) was used to provided ’ground truth’ data for the comparison of different algorithms.

At first, the processing duration was evaluated to reject algorithms with an enormous computing time. Afterwards, the defect accuracy d and the false negatives rate N as their counterpart were analysed to determine the detection performance of the algorithms. The detection accuracy d is the ratio of the number of the computerised determined defects $n_{d}$ divided by the number of labelled ground truth defects $n_{gt}$. These values referred to all defects for an observation. This leads to Eq. 8.

$$\begin{aligned} d = \frac{n_{d}}{n_{gt}} \text { and } N = 1-d \end{aligned}$$

(8)

Initially, the detection results are presented considering all types of defects. Afterwards, separate investigations are carried out considering two geometrical distinguishable defect groups. Group 1: Wrinkle, twist; Group 2: Gap, overlap. Foreign materials are difficult to assign to a distinct group. Therefore, they are not considered separately, here.

For the following detailed analysis, the false positive values were measured and divided into the counts of false positive detections $P_{n\_i}$ and the pixel area of false positive estimations $P_{a\_i}$. This separation aims to distinguish between the size and the number of these measurements. For correctly detected defects, the excess area $A_{e\_i}$ and the overlap area $A_{o\_i}$ were calculated, in order to identify unnecessarily selected pixels and dispensable marked areas.

For the individual observations, the following calculations $X_{i}$ were performed for each examined defect sample image I. For the results presented in Sect. 4, the mean value and the standard deviation were calculated from all individual observations $X_{i}$. $n_{d\_i}$ gives the number of computerised determined defects, for the examination of each sample image I. Moreover, $K = n_{gt\_i}$ describes the corresponding number of ground truth defects. $n_{sd\_i}$ is defined as the number of superfluously predicted defects with the corresponding superfluously recognised defect area $A_{sd\_i}$. The variable $A_{gt\_i}$ is the manually labelled and accumulated defect area for the evaluated defect sample. C represents the number of detected defects which can be linked to a corresponding ground truth region. The individual detection accuracy $d_{i}$ and false negatives rate $N_{i}$ are given from Eq. 9.

$$\begin{aligned} d_{i} = \frac{n_{d\_i}}{K} \text { and } N_{i} = 1-d_{i} \end{aligned}$$

(9)

The position accuracy per defect was calculated as described in Eq. 10. Therefore, $E_{d\_i} \in {\mathbb {R}}_{0}^{+}$ represents the Euclidean distance between the computerised estimated and the corresponding ground truth defect position.

$$\begin{aligned} p_{i} = 1 - \frac{\vert E_{d\_i}- M_{i} \vert }{E_{d\_i} + M_{i}} \text { with } M_{i} = \frac{\sum _{d=1}^{C} E_{d\_i}}{C} \end{aligned}$$

(10)

The number based $P_{n\_i}$ and area based $P_{a\_i}$ false positive rates are given in the Eq. 11.

$$\begin{aligned} P_{n\_i} = \frac{n_{sd\_i}}{K} \text { and } P_{a\_i} = \frac{A_{sd\_i}}{A_{gt\_i}} \end{aligned}$$

(11)

The calculation of the excess area $A_{e\_i}$ and the overlap area $A_{o\_i}$ are presented in the Eq. 12

$$\begin{aligned} A_{e\_i} = \frac{A_{ed\_i}}{A_{gt\_i}} \text { and } A_{o\_i} = \frac{A_{od\_i}}{A_{gt\_i}} \end{aligned}$$

(12)

Furthermore, the two most powerful algorithms were analysed more closely. The aim here was the more detailed examination of the segmentation behaviour of each algorithm. For this purpose, initially, the findings considering all types of defects are presented. Afterwards, the results are again split up into the two groups of defects mentioned above.

Moreover, with the aim to investigate the robustness of the algorithms for other input data, noise was applied to the input images. This noise was Gaussian distributed for an individual sample image I at position (x,y), with each pixel value I(x,y) as its mean $I(x,y) = \mu (x,y)$ and a standard deviation of $\sigma = 0.5$. Afterwards, the above investigations were repeated. The algorithms were executed without the previously required adjustments of the pre-processing and algorithm settings. This was also intended to check, if the related parameters have a high influence on the performance of the defect detection, as claimed before. Furthermore, this investigation served to examine the reproducibility of the results using different input data but the given configurations. All the related results are presented in the following section.

Results

This section analyses the performed experiments and their results. The individually modified pre-processing settings from Table 7 and for the segmentation algorithms from Table 8, were used for all subsequent attempts. These settings were identified using our individual optimisation procedure. This means that for each algorithm considered, all settings for the pre-processing and the algorithm itself are individually adjusted. All parameter combinations are tested crosswise and the setting with the highest detection accuracy are then applied. The corresponding parameter adjustments are listed in the Table 5 for the pre-processing settings and in the Table 6 for the detection algorithm parameters.

At first, the execution times of the seven considered algorithms are investigated. Therefore, the Fig. 5 displays the calculation times for each selected algorithm. The Table 9 summarises the corresponding values including their standard deviations. The MS, AT and OT algorithms were swiftest to perform the segmentation. They require less than 31 ms for the FI implementation and below 10 ms for the GB version on each entire input image. The GF algorithms perform worst with an average computing time of more than 300 ms. The calculation time in this case is independent of the content of the input image.

In order to assess the calculation times, we should remember that the scan of the considered fibre material sample would be performed in 0.25 s at a maximum AFP process speed of 1 $\frac{m}{s}$. The goal must be the completion of the defect segmentation of a measurement section before the scan of the next section is completed. Thus all algorithms with $< 250$ ms execution time are basically applicable. However, a faster defect detection helps to react as quickly as possible to arising defects. Thus, a very short calculation time is desired.

Afterwards the detection performances of these seven techniques are examined. The Fig. 6a displays the results for the average detection accuracies d and the corresponding false negative rates N, considering all the available data from all defect types. Figure 6b shows the results for the gap and overlap defect detection. Figure 6c presents only the findings for the segmentation of wrinkles and twists.

In order to assess the results, we should bear in mind that the defect detection is implemented at the beginning of the image processing chain. Thus, it is important to segment as many defects as possible and pass them to the following image processing. Unnecessarily segmented areas are less problematic due to the subsequent classification step. Moreover, for the requirements in aerospace industry it is essential to have as few defect misses as possible. Hence detection accuracies of $\ge 95$% are highly desirable.

Starting with, we have a look at the detection performance considering all defect types. Here, we can recognise, that the AT and CT algorithms in the FI and the GB implementations perform best for this experimental assembly. The 5th most accurate is the FI GT algorithm. All the other techniques have detection rates of less than 50%, which is not suitable for this application.

Table 7

The ascertained, most capable parameter configuration for the image pre-processing in relation to the best segmentation result is presented

	Parameters
Algorithms	Variant	Dilation steps	CLAHE - Clip Limit	CLAHE - Tile size	Thresh Img. Dilat. x - Kernel	Thresh Img. Dilat. y - Kernel	Thresh Img. Dilat. Steps x	Thresh Img. Dilat. Steps y
GT	all	4	20;2	(8,8)	(9,1)	(1,3)	6;10	8
	w/t	8	20	”	”	”	12	”
	g/o	4	20;2	”	”	”	6;10	”
CT	all	4	26;14	”	”	”	6	”
	w/t	8	20	”	”	”	12	”
	g/o	4	26;14	”	”	”	6	”
IP	all	12;6	26;2	”	”	”	6	”
	w/t	8	20	”	”	”	12	”
	g/o	12;6	26;2	”	”	”	6	”
OT	all	6;12	2;14	”	”	”	6	”
	w/t	8	20	”	”	”	12	”
	g/o	6;12	2;14	”	”	”	6	”
AT	all	8;12	20	”	”	”	12;6	”
	w/t	8	20	”	”	”	12	”
	g/o	8;12	20	”	”	”	12;6	”
GF	all	12	26	”	”	”	12	”
	w/t	8	20	”	”	”	12	”
	g/o	12	26	”	”	”	12	”
MS	all	10	14	”	”	”	6	”
	w/t	8	20	”	”	”	12	”
	g/o	10	14	”	”	”	6	”

Single values: $<\text{ FI } = \text{ GB }>$ $\vert $ Different values $<\text{ FI }> ; <\text{ GB }>$ $\vert $ “ same value as cell above. (w/t) := (Wrinkle/Twist), (g/o) := (Gap/Overlap)

Table 8

The settings used for the experiments are listed. The parameter types are equal for FI and GB.

A	Parameter	Config. (all)	Config. (w/t)	Config. (g/o)
GT	Kernel size sobel filter	5	5	5
	Sobel threshold x	130	130	130
	Sobel threshold y	140	140	140
CT	Cell size x	10	10	10;20
	Cell size y	10	10	20;30
	Cell overlap	0.5	0.5	0.5
	Threshold STD	7	15	12;11
IP	Numb. rot. x	1	3	1;3
IP	Threshold	130	140	130;140
OT	None	None	None	None
AT	Thresh. Offset	-20	$-16$;$-32$	$-12$;$-20$
GF	Kernel size x	31	31	31
	$\sigma $	10	10	10
	$\lambda $	10	10	10
	Threshold Gabor filter	10	10	10
MS	None	None	None	None

Single values: $<\text{ FI } = \text{ GB }>$ $\vert $ Different values $<\text{ FI }> ; <\text{ GB }>$ $\vert $ (w/t) := (Wrinkle/Twist), (g/o) := (Gap/Overlap), A : = Algorithm

For an independent segmentation of wrinkles and twists with a pre-processing optimised for this purpose we see, that the detection rates for the GT, CT and AT methods are almost 100%. On the other hand, the GF algorithm founds no defects at all. The remaining algorithms all have detection rates between about 45% and 85%. This trend is very similar to the analysis of all defects. These results slightly changed for the individual detection of gaps and overlaps. In this case, the findings for both CT implementations as well as the calculated detection results for the GB AT are $> 95$%. Except the FI OT, GF and IP implementations, all the other algorithms generate detection results $> 80$%.

These results reveal the previously predicted major influence of image pre-processing and algorithm parametrisation. Furthermore, the categories of defects defined in this paper seem to be meaningful, since the detection performance for the individual groups increases significantly. For a realistic application of the defect segmentation, with initially unknown defect types, a parallel performed image pre-processing and defect segmentation with several different settings could be beneficial. The individual results could then be linked in a suitable manner.

Due to the good performance of the AT and CT techniques they are investigated in detail, afterwards. Especially the detection results for the analysis of all considered defects are much better than for the competing algorithms. Their detailed findings are presented in the Fig. 7. Related, the Fig. 7a shows the corresponding segmentation performance considering all defects. Sub-Figure 7c presents the detection results only for gaps and overlaps. Similarly, Fig. 7d displays the findings for the detection of wrinkles and twists. In order to investigate the robustness of these algorithms, Fig. 7b visualises the detection results for noisy input images using the same algorithm settings that are used for the outcome in Fig. 7a. The findings for all defects analysis belonging to Fig. 7a are additionally listed in Table 10.

To begin with, we examine the results presented in Fig. 7a for all types of defects. In this case, the GB AT algorithm yields the highest overall score of 100% detection accuracy. The GB CT method then follows with 98.2% ($\sigma = 4.1$%) detection rate. A quite comparable effect can be seen for the accuracies of the defect position. Therefore, the GB AT and both CT algorithms have positional accuracies of about 78%. Then, the FI AT algorithm follows, having a positioning accuracy of around 68%. Strangely enough, the number based false positive rates are much smaller for the two FI implementations than for the GB ones. It is also noteworthy, that the average ground truth areas and the automatically segmented regions overlap by only about 15% in both methods. However, the protruding areas for detected defects are rather large. Particularly, the detection results for the evaluation of all considered defects are significantly superior to those of competing methods. The detection accuracy is $> 95$% with a false negative rate $< 5$%. Hereto, the FI AT method for overlaps and gaps constitutes an exception with around 83% detection accuracy. The defect position accuracy for wrinkles and twists is about 65%. For gaps and overlaps this measurement value is somewhat greater with around 85%. Among these, the FI AT forms an exception with about 75% positioning accuracy. A contradictory behaviour can be recognised for the area based false positive assessment. Gaps and overlaps generate false positive values for less than 20% of cases. For wrinkles and twists this value is larger than 35%, apart from the GB CT algorithm with about 19%. The number of false positive detections is always between about 20% and 42%. This means, that the size and presumably the shape of a defect area changes due to the defect geometry, but the numbers of false positive detections are very similar. The values for the excess areas are between 45% and 80% for all defects and algorithms. The overlapping of the ground truth regions and the detected areas is between 5% and 20%, whereby gaps and overlaps show somewhat slightly better overlapping results in this context. These findings show, that the examined algorithms represent the area of a defect poorly. In order to use them in an inspection application the region of interest must be artificially enlarged. However, due to the sufficient positional accuracy of the selected algorithms, this extension of the viewing area is reasonably achievable. The shape of the expanded area could be connected to the algorithm configuration, since this is of varying sensitivity for different defect types.

Table 9

The calculation times and standard deviations for all examined algorithms are listed, each for the FI and GB implementation

Calculation time [ms]
Algorithm	FI	GB
GT	67.9 ($\sigma = 3.9$)	22.9 ($\sigma = 1.0$)
CT	143.4 ($\sigma = 8.4$)	69.8 ($\sigma = 0.6$)
IP	51.3 ($\sigma = 3.9$)	11.8 ($\sigma = 1.0$)
OT	30.7 ($\sigma = 3.2$)	8.0 ($\sigma = 1.6$)
AT	30.0 ($\sigma = 4.0$)	7.6 ($\sigma = 0.1$)
GF	340.5 ($\sigma = 49.3$)	320.6 ($\sigma = 8.6$)
MS	21.0 ($\sigma = 1.9$)	3.0 ($\sigma = 0.6$)

The values belong to Fig. 5 and are given in milliseconds [ms]

The findings for the input images with noise are slightly different. They are displayed in Fig. 7b. These outcomes are generated from the same data set as well as the same pre-processing and algorithm configuration as for the results in Fig. 7a, but with additionally applied artificial noise. As already introduced in Sect. 3.5, this modification of the input data serves to test the robustness of the algorithms and respectively the settings for adjusted images, with constant configurations. The requirement for a data-dependent parameter adjustment has already been claimed above and is also reviewed in this way. This noise leads to the reduction of the detection accuracies for the GB AT algorithm by about 10% and for both CT methods by around 40%. The number of false positive detections stays equal for the AT algorithms. Surprisingly, the number of false positive detections for the CT algorithms significantly decreases by about 10%–20% compared to the non noise data. An improved setting, especially for noisy images, may cause much better results. However, similar statistical noise as exemplary applied for this experiment can be caused by disturbing scattering or beam propagation behaviour of the laser on the material. This obviously can lead to local noise deviations in the image, depending on the geometry of the component. This must be taken into account for a real application. In this case, a local image pre-processing technique might lead to even better detection results.

In order to compare the theoretical ranking of the algorithms from the Table 4, with the experimental test results, these findings are comprehensively summarised in the Table 11. Therefore, the same expectation weights are used as for the theoretical appraisal.

Table 10

Shown are the results of the defect detection for the AT and CT methods, taking into account all the available defect data

	FI AT	GB AT	FI CT	GB CT
Detection accuracy (d)	81.0 (28.5)	100.0 (0.0)	97.6 (6.1)	98.2 (4.1)
Position accuracy (p)	67.9 (28.8)	77.9 (24.5)	76.8 (24.3)	77.9 (24.5)
False neg. rate (N)	18.7 (28.4)	0.0 (0.0)	2.4 (6.1)	1.8 (4.1)
False pos. rate (num) ($P_{n}$)	22.3 (20.3)	44.2 (33.3)	29.2 (23.5)	35.7 (30.4)
False pos. rate (area) ($P_{a}$)	3.1 (0.9)	8.7 (4.6)	14.8 (6.2)	11.8 (4.8)
Excess area ($A_{e}$)	64.1 (16.2)	69.0 (24.2)	61.4 (22.4)	68.3 (16.8)
Overlap area ($A_{o}$)	11.6 (10.6)	9.0 (10.1)	9.9 (9.1)	15.7 (11.6)

The values belong to Fig. 7a and are given as $<\text{ value }> (<\text{ standard } \text{ deviation }>)$ in %

Table 11

Evaluation survey of the individual segmentation algorithms based on the experimental results and using criteria from the literature

	Criteria
Algorithms		Number invariances	Calculation speed	Implementation effort	Detection accuracy	False positive rate	False negative rate	Localising accuracy	Robustness	Adaptability (input)	Weighted sum
	Weights [1-5]	2	5	1	5	3	5	3	4	5
	GT	3	4	4	4	4	4	4	3	4	3.82
	IP	1	4	4	4	4	4	4	3	4	3.03
	CT	2	3	4	5	3	5	5	3	4	3.91
	OT	2	5	4	2	4	2	3	3	3	3.06
	AT	3	5	4	5	3	5	5	5	5	4.67
	MS	3	5	3	2	4	2	3	3	3	3.09
	GF	2	2	3	0	0	0	0	0	0	0.52

Assessment results are given in bold Italic row gives the weights for the weighting of the assessment criteria for evaluation

What is noticeable here is the miscalculation of the GF algorithm. Based on the literature from other use cases, the GF algorithm was judged to be very promising for the defect segmentation. However, the results in our experiments show the opposite. All the other algorithms behave approximately as expected. The findings presented above are discussed in the following section.

Discussion

In the following, the experimental results are discussed and reviewed in the context of related studies. Additionally, the research questions are answered and an outlook on further research is proposed.

Compared to the defect detection findings based on Neural Networks from e.g. Schmidt et al. (2019a, 2019b), the classical image segmentation techniques investigated in this paper also provide quite good results. The chosen segmentation algorithms worked quite well except for the GF algorithms, which yield very poor detection results. On this matter, Hanbay et al. (2016) pointed out in their publication, that the GF is suitable only for fairly uniformly structured image data and otherwise works rather poorly. The LLSS topology data of the prepreg material in this paper contains unevenly distributed artifacts. Considering this aspect, the performance of the GF algorithms are comprehensible. The OT methods also provide weaker performance in comparison to Tonnaer et al. (2017), which is probably due to the brightness gradient in the images. This effect causes the algorithms to segment a very large region of the image. An improved pre-processing, which is more tolerant for changes in brightness, may reduce this issue. It should also be noted, that the CLAHE image adjustment algorithm used in this paper is very sensitive to changes in the “clip limit” parameter. This is a previously defined value which spreads areas of the histogram above this clip limit across the entire base of the histogram. From the results it can be seen that the pre-processing is very important for noise reduction. This stage needs to be individually adapted for each setup and manufacturing process. Furthermore, the selection of algorithms mainly based on findings from fabric inspection is considered valuable for our use case.

Regarding the research questions it should be noted, that a sufficiently good defect detection can be performed with the AT and the CT algorithms for CFRP prepreg material. The GT algorithms also provide satisfactory defect detection results, but only for geometrically larger defects or as GB variant.

For most of the defect types considered, the AT and CT algorithms have a detection accuracy of $> 95$%. For prominent defects, these mean detection accuracy is even up to 100%. Moreover, the defect position is usually determined with sufficient accuracy. In concrete terms this means, a position deviation of less than 30% from the marked ground truth defect position. Unfortunately, the actual defect segmentation in terms of segmented region is much worse, for both overlap and excess area. Below, valuable future research opportunities are outlined.

Future work and prospect

In future work, the effects of different CFRP materials and their interaction with the LLSS settings must be investigated. In particular, varying optical material characteristics can have a great influence on the performance of certain algorithms. Thus, these optical material parameters have to be examined in detail and linked to the current results. It is also worth noting, that the pre-processing can be improved for better detection performance, though this was not considered part of this work.

In addition, it is conceivable to supplement the defect segmentation procedure presented in this paper with a machine learning approach. For instance, a filter-based technique could be retained for better comprehensibility of the algorithms behaviour, but the parameter setting might be carried out through a machine learning approach. Furthermore, the pre-processing parameters could be adjusted through machine learning techniques for optimised performance. Beyond that, a combined one stage segmentation and classification of the defects via a deep learning algorithm is also possible. However, for the traceability of such advance machine learning or even deep learning algorithms and its application in the aviation sector, additional methods for the explainability of the deep learning decision must be applied (Lee et al. 2021). Besides, the necessary amount and quality of training data need to be available, as explained earlier. The key findings of this paper are briefly repeated in the subsequent section.

Conclusion

Declarations

Conflict of interest:

The authors declare that they have no conflict of interest.

Code availability:

The corresponding Python code is made available via 4TU portal https://www.doi.org/10.4121/c.5180657 in case the paper is accepted.

Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Vorheriger Artikel Mitigating congestion in wireless sensor networks through clustering and queue assistance: a survey

Nächster Artikel A note to: A multiple-rule based constructive randomized search algorithm for solving assembly line worker assignment and balancing problem

ALGLIB-Project: Linear discriminant analysis. ALGLIB - numerical analysis library. http://www.citationmachine.net/bibtex/cite-a-website. Accessed: 2019-06-1

Atkinson, G. A., Thornton, T. J., Peynado, D. I., & Ernst, J. D. (2018). High-precision polarization measurements and analysis for machine vision applications. In 2018 7th European Workshop on Visual Information Processing (EUVIP). IEEE https://doi.org/10.1109/euvip.2018.8611762

Automation Technology GmbH: C5 series - user manual for high speed 3d sensors. techreport 1.2, Automation Technology GmbH, Hermann-Bössow-Straße 6-8, 23843 Bad Oldesloe, Germany (2019). https://www.automationtechnology.de/cms/wp-content/uploads/2019/03/C5-Series_specifications_web.pdf. Rev 1.2

Automation Technology GmbH: The FIR filter. techreport 1.0, Automation Technology GmbH, Hermann-Bössow-Straße 6-8, 23843 Bad Oldesloe, Germany (2014). https://www.stemmer-imaging.com/media/uploads/cameras/12/122195-Automation_Technology_AppNote_FIR_Filter.pdf. Rev. 1.0

Black, S. (2018). Improving composites processing with automated inspection, part II. compositesworld https://www.compositesworld.com/articles/improving-composites-processing-with-automated-inspection-part-ii. https://www.compositesworld.com/articles/improving-composites-processing-with-automated-inspection

Bradski, G. (2000). The OpenCV Library. Dr. Dobb’s Journal of Software Tools https://opencv.org/

Bulnes, F. G., Usamentiaga, R., Garcia, D. F., & Molleda, J. (2014). An efficient method for defect detection during the manufacturing of web materials. Journal of Intelligent Manufacturing, 27(2), 431–445. https://doi.org/10.1007/s10845-014-0876-9.CrossRef

Campbell, F. (2004). Manufacturing processes for advanced composites. Amsterdam: Elsevier Science & Technology.

Cattin, P. (2016). Texture segmentation - introduction to signal and image processing. MIAC, University of Basel https://miac.unibas.ch/SIP/08-Texture.html#(1). Accessed: 2019-06-17

Cemenska, J., Rudberg, T., & Henscheid, M. (2015). Automated in-process inspection system for AFP machines. SAE International Journal of Aerospace, 8(2), 303–309. https://doi.org/10.4271/2015-01-2608.CrossRef

Denkena, B., Schmidt, C., Völtzer, K., & Hocke, T. (2016). Thermographic online monitoring system for automated fiber placement processes. Composites Part B: Engineering, 97, 239–243. https://doi.org/10.1016/j.compositesb.2016.04.076.CrossRef

Dhanachandra, N., & Chanu, Y. J. (2017). A survey on image segmentation methods using clustering techniques. European Journal of Engineering Research and Science, 2(1), 15 (10.24018/ejers.2017.2.1.237).

Du, W., Shen, H., Fu, J., Zhang, G., Shi, X., & He, Q. (2020). Automated detection of defects with low semantic information in x-ray images based on deep learning. Journal of Intelligent Manufacturing, 32(1), 141–156. https://doi.org/10.1007/s10845-020-01566-1.CrossRef

Eitzinger, C. (2019). Inline inspection helps accelerate production by up to 50 %. Lightweight Design worldwide

European Machine Vision Association: Emva genicam standard. techreport 2.0, European Machine Vision Association (EMVA) (2009). https://www.emva.org/. Release 2.0

Faidi, W., Nafis, C., Sinha, S., Yerramalli, C., Waas, A., Advani, S., Gangloff, J., & Simacek, P. (2012). Wind turbine manufacturing process monitoring. Tech. rep., General Electric Global Research Center https://doi.org/10.2172/1039054

Gardiner, G. (2018). Zero-defect manufacturing of composite parts. CompositesWorld https://www.compositesworld.com/blog/post/zero-defect-manufacturing-of-composite-parts.

Gregory, E. D., & Juarez, P. D. (2018). In-situ thermography of automated fiber placement parts. AIP Conference Proceedings, doi, 10(1063/1), 5031551.

Grohmann, Y., Stoffers, N., Kühn, A., & Mahrholz, T. (2016). Development of the direct roving placement technology (DRP). In ECCM17 - 17th European Conference on Composite Materials. https://elib.dlr.de/107706/

Hanbay, K., Talu, M. F., & Özgüven, Ö. F. (2016). Fabric defect detection systems and methods–a systematic literature review. Optik, 127(24), 11960–11973. https://doi.org/10.1016/j.ijleo.2016.09.110.CrossRef

Harik, R., Saidy, C., J. Williams, S., Gurdal, Z., & Grimsley, B. (2018). Automated fiber placement defect identity cards: cause, anticipation, existence, significance, and progression. In SAMPE 18

Heinecke, F., & Willberg, C. (2019). Manufacturing-induced imperfections in composite parts manufactured via automated fiber placement. Journal of Composites Science, 3(2), 56. https://doi.org/10.3390/jcs3020056.CrossRef

Jain, S., Seth, G., Paruthi, A., Soni, U., & Kumar, G. (2020). Synthetic data augmentation for surface defect detection and classification using deep learning. Journal of Intelligent Manufacturing,. https://doi.org/10.1007/s10845-020-01710-x.CrossRef

Joshi, K. D., Chauhan, V., & Surgenor, B. (2018). A flexible machine vision system for small part inspection based on a hybrid SVM/ANN approach. Journal of Intelligent Manufacturing, 31(1), 103–125. https://doi.org/10.1007/s10845-018-1438-3.CrossRef

Kaur, H., & Sharma, D. R. (2016). A survey on techniques for brain tumor segmentation from MRI. IOSR Journal of Electronics and Communication Engineering, 11(05), 01–05. https://doi.org/10.9790/2834-1105010105.CrossRef

Kumar, A. (2008). Computer-vision-based fabric defect detection: A survey. IEEE Transactions on Industrial Electronics, 55(1), 348–363. https://doi.org/10.1109/TIE.1930.896476.CrossRef

Kumar, A., & Pang, G. (2002). Defect detection in textured materials using Gabor filters. IEEE Transactions on Industry Applications, 38(2), 425–440. https://doi.org/10.1109/28.993164.CrossRef

Kuo, C. F. J., Fang, T. Y., Lee, C. L., & Wu, H. C. (2016). Automated optical inspection system for surface mount device light emitting diodes. Journal of Intelligent Manufacturing, 30(2), 641–655. https://doi.org/10.1007/s10845-016-1270-6.CrossRef

Lee, L. K., Liew, S. C., & Thong, W. J. (2014). A review of image segmentation methodologies in medical image. In Lecture Notes in Electrical Engineering, (pp. 1069–1080). Springer International Publishing https://doi.org/10.1007/978-3-319-07674-4-99

Lee, M., Jeon, J., & Lee, H. (2021). Explainable AI for domain experts: a post hoc analysis of deep learning for defect classification of TFT-LCD panels. Journal of Intelligent Manufacturing,. https://doi.org/10.1007/s10845-021-01758-3.CrossRef

Lee, W. J., Mendis, G. P., Triebe, M. J., & Sutherland, J. W. (2019). Monitoring of a machining process using kernel principal component analysis and kernel density estimation. Journal of Intelligent Manufacturing, 31(5), 1175–1189. https://doi.org/10.1007/s10845-019-01504-w.CrossRef

Lengsfeld, H., Fabris, F. W., Krämer, J., Lacalle, J., & Altstädt, V. (2014). Faserverbundwerkstoffe. Hanser Fachbuchverlag.

Lin, H. D. (2007). Computer-aided visual inspection of surface defects in ceramic capacitor chips. Journal of Materials Processing Technology, 189(1–3), 19–25. https://doi.org/10.1016/j.jmatprotec.2006.12.051.CrossRef

Liu, Z., Yan, L., Li, C., Dong, Y., & Gao, G. (2017). Fabric defect detection based on sparse representation of main local binary pattern. International Journal of Clothing Science and Technology, 29(3), 282–293. https://doi.org/10.1108/IJCST-04-2016-0040.CrossRef

Ma, J., Fan, X., Yang, S. X., Zhang, X., & Zhu, X. (2017). Contrast limited adaptive histogram equalization based fusion for underwater image enhancement. International Journal of Pattern Recognition and Artificial Intelligence, 2017, 2017030086. https://doi.org/10.20944/preprints201703.0086.v1

Maass, D. (2012). Automated dry fiber placement for aerospace composites. In Composites Manufacturing 2012. Danobat

Mahajan, P. M., Kolhe, S., & Patil, P. M. (2009). A review of automatic fabric defect detection techniques. Advances in Computational Research, 1, 18–29.

Marsh, G. (2010). Airbus A350 XWB update. Reinforced Plastics, 54(6), 20–24. https://doi.org/10.1016/s0034-3617(10)70212-5.CrossRef

Masood, S., Sharif, M., Masood, A., Yasmin, M., & Raza, M. (2015). A survey on medical image segmentation. Current Medical Imaging Reviews, 11(1), 3–14. https://doi.org/10.2174/157340561101150423103441.CrossRef

McIlhagger, A., Archer, E., & McIlhagger, R. (2020). Manufacturing processes for composite materials and components for aerospace applications. In Polymer Composites in the Aerospace Industry, (pp. 59–81). Elsevier https://doi.org/10.1016/b978-0-08-102679-3.00003-4

Meister, S., Kaestner, S., & Krombholz, C. (2018). Enhancements of an inline QA system for fiber layup processes. In ISCM 2018https://elib.dlr.de/122063/

Meister, S., Möller, N., Stüve, J., & Groves, R. M. (2021). Synthetic image data augmentation for fibre layup inspection processes: Techniques to enhance the data set. Journal of Intelligent Manufacturing,. https://doi.org/10.1007/s10845-021-01738-7.CrossRef

Meister, S., Wermes, M. A. M., Stueve, J., & Groves, R. M. (2020). Algorithm assessment for layup defect segmentation from laser line scan sensor based image data. In D. Zonta & H. Huang (Eds.), Sensors and Smart Structures Technologies for Civil, Mechanical, and Aerospace Systems 2020. Washington: SPIE. https://doi.org/10.1117/12.2558434.

Meng, S., Pan, R., Gao, W., Zhou, J., Wang, J., & He, W. (2020). A multi-task and multi-scale convolutional neural network for automatic recognition of woven fabric pattern. Journal of Intelligent Manufacturing, 32(4), 1147–1161. https://doi.org/10.1007/s10845-020-01607-9.CrossRef

Miesen, N., Sinke, J., Groves, R. M., & Benedictus, R. (2015). Simulation and detection of flaws in pre-cured CFRP using laser displacement sensing. The International Journal of Advanced Manufacturing Technology, 82(1–4), 341–349. https://doi.org/10.1007/s00170-015-7305-x.CrossRef

Muniyappan, S., Allirani, A., & Saraswathi, S. (2013). A novel approach for image enhancement by using contrast limited adaptive histogram equalization method. In 2013 Fourth International Conference on Computing, Communications and Networking Technologies (ICCCNT). IEEE https://doi.org/10.1109/icccnt.2013.6726470

Mustafa, I. D., & Hassan, M. A. (2016). A comparison between different segmentation techniques used in medical imaging. American Journal of Biomedical Engineering, 6(2), 59–69. https://doi.org/10.5923/j.ajbe.20160602.03.CrossRef

Nardi, D., Abouhamzeh, M., Leonard, R., & Sinke, J. (2018). Detection and evaluation of pre-preg gaps and overlaps in glare laminates. Applied Composite Materials, 25(6), 1491–1507. https://doi.org/10.1007/s10443-018-9679-z.CrossRef

Neubert, M., & Meinel, G. (2003). Evaluation of segmentation programs for high resolution remote sensing applications

Ng, H. F. (2004). Automatic thresholding for defect detection. In Third International Conference on Image and Graphics (ICIG’04). IEEE https://doi.org/10.1109/icig.2004.43

Norouzi, A., Rahim, M. S. M., Altameem, A., Saba, T., Rad, A. E., Rehman, A., et al. (2014). Medical image segmentation methods, algorithms, and applications. IETE Technical Review, 31(3), 199–213. https://doi.org/10.1080/02564602.2014.906861.CrossRef

OpenCV: Image processing - Image Filtering. techreport, OpenCV (2018). https://docs.opencv.org/3.4.1/d4/d86/group__imgproc__filter.html

OpenCV: Image Processing (imgproc module) - Sobel Derivatives. techreport, OpenCV (2019). https://docs.opencv.org/master/d2/d2c/tutorial_sobel_derivatives.html

OpenCV: Image Thresholding. techreport, OpenCV (2018). https://docs.opencv.org/3.4.1/d7/d4d/tutorial_py_thresholding.html

OpenCV: Morphological Transformations. techreport, OpenCV (2018). https://docs.opencv.org/3.4.1/d9/d61/tutorial_py_morphological_ops.html

OpenCV: OpenCV API Reference - Operations on Arrays - Reduce. techreport, OpenCV (2019). https://docs.opencv.org/2.4/modules/core/doc/operations_on_arrays.html#reduce

Oromiehie, E., Prusty, B. G., Compston, P., & Rajan, G. (2019). Automated fibre placement based composite structures: Review on the defects, impacts and inspections techniques. Composite Structures, 224, 110987. https://doi.org/10.1016/j.compstruct.2019.110987.CrossRef

Otsu, N. (1979). A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics, 9(1), 62–66. https://doi.org/10.1109/tsmc.1979.4310076.CrossRef

Pizer, S. M., Austin, J. D., Perry, J. R., Safrit, H. D., & Zimmerman, J. B. (1986). Adaptive histogram equalization for automatic contrast enhancement of medical images. In S. J. D. III & R. H. Schneider (Eds.), Application of Optical Instrumentation in Medicine XIV and Picture Archiving and Communication Systems (Vol. 0626, pp. 242–250). Washington: SPIE. https://doi.org/10.1117/12.975399.

Potter, K. (2009). Understanding the origins of defects and variability in composites manufacture. In ICCM International Conferences on Composite Materialshttp://iccm-central.org/Proceedings/ICCM17proceedings/Themes/Plenaries/P1.5%20Potter.pdf

Rodriguez, A., Gonzalez, C., Fernandez, A., Rodriguez, F., Delgado, T., & Bellman, M. (2020). Automatic solar cell diagnosis and treatment. Journal of Intelligent Manufacturing, 32(4), 1163–1172. https://doi.org/10.1007/s10845-020-01642-6.CrossRef

Rudberg, T. (2019). Webinar: Building AFP system to yield extreme availability. Video: CompositesWorld.

Rudberg, T., Nielson, J., Henscheid, M., & Cemenska, J. (2014). Improving AFP cell performance. SAE International Journal of Aerospace, 7(2), 317–321. https://doi.org/10.4271/2014-01-2272.CrossRef

Sacco, C., Radwan, A. B., Harik, R., & Tooren, M. V. (2018). Automated fiber placement defects: Automated inspection and characterization. In SAMPE 18 - Long Beach, (p. 13). McNAIR Center for Aerospace Innovation and Research, Department of Mechanical Engineering, College of Engineering and Computing, University of South Carolina 1000 Catawba St., Columbia, SC, 29201, USA https://www.nasampe.org/store/ViewProduct.aspx?ID=11833782

Sarkar, A., & Padmavathi, S. (2018). Image pyramid for automatic segmentation of fabric defects. In Computational Vision and Bio Inspired Computing, (pp. 569–578). Springer International Publishing, New York https://doi.org/10.1007/978-3-319-71767-8_49

Schmidt, C., Hocke, T., & Denkena, B. (2019a). Deep learning-based classification of production defects in automated-fiber-placement processes. Production Engineering, 13(3–4), 501–509. https://doi.org/10.1007/s11740-019-00893-4.

Schmidt, C., Hocke, T., & Denkena, B. (2019). Artificial intelligence for non-destructive testing of CFRP prepreg materials. Production Engineering,. https://doi.org/10.1007/s11740-019-00913-3.

Schmitt, R., Niggemann, C., & Mersmann, C. (2008). Contour scanning of textile preforms using a light-section sensor for the automated manufacturing of fibre-reinforced plastics. In F. Berghmans, A. G. Mignani, A. Cutolo, P. P. Meyrueis, & T. P. Pearsall (Eds.), Optical Sensors 2008 (Vol. 7003, pp. 436–447). Washington: SPIE. https://doi.org/10.1117/12.779005.

Schmitt, R., Orth, A., & Niggemann, C. (2007). A method for edge detection of textile preforms using a light-section sensor for the automated manufacturing of fibre-reinforced plastics. In W. Osten, C. Gorecki, & E. L. Novak (Eds.), Optical Measurement Systems for Industrial Inspection V. Washington: SPIE. https://doi.org/10.1117/12.726177.

Schöberl, M., Kasnakli, K., & Nowak, A. (2016). Measuring strand orientation in carbon fiber reinforced plastics (CFRP) with polarization. In 19th World Conference on Non-Destructive Testing 2016

Shanbhag, P. M., Deshmukh, M. P. (2013). Fabric defect detection using principal component analysis. In International Journal of Engineering Research & Technology, (vol. 2). IJERT

Sun, S., Han, Z., Fu, H., Jin, H., Dhupia, J. S., & Wang, Y. (2020). Defect characteristics and online detection techniques during manufacturing of FRPs using automated fiber placement: A review. Polymers, 12(6), 1337. https://doi.org/10.3390/polym12061337.CrossRef

Tabernik, D., Šela, S., Skvarč, J., & Skočaj, D. (2019). Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing, 31(3), 759–776. https://doi.org/10.1007/s10845-019-01476-x.CrossRef

Tajeripour, F., Kabir, E., & Sheikhi, A. (2007). Fabric defect detection using modified local binary patterns. EURASIP Journal on Advances in Signal Processing,. https://doi.org/10.1155/2008/783898.CrossRef

Tonnaer, R., Shroff, S., & Groves, R. M. (2017). Online preventive non-destructive evaluation for automated fibre placement. In 3rd International Symposium on Composite Manufacturing, (pp. 114–123)

Tzutalin: LabelImg. https://github.com/tzutalin/labelImg (2015).

Weimer, C., Friedberger, A., Helwig, A., Heckner, S., Buchmann, C., & Engel, F. (2016). Increasing the productivity of CFRP production processes by robustness and reliability enhancement. In CAMX 2016 - The Composites and Advanced Materials Expo and Conference. Airbus Group Innovations, 81663 Munich, Germany; AirbusInfactory Solutions GmbH, 81663 Munich, Germany

Xie, X. (2008). A review of recent advances in surface defect detection using texture analysis techniques. ELCVIA Electronic Letters on Computer Vision and Image Analysis, 7(3), 1. https://doi.org/10.5565/rev/elcvia.268.CrossRef

Zambal, S., Heindl, C., Eitzinger, C., & Scharinger, J. (2019). End-to-end defect detection in automated fiber placement based on artificially generated data. In C. Cudel, S. Bazeille, & N. Verrier (Eds.), Fourteenth International Conference on Quality Control by Artificial Vision. Washington: SPIE. https://doi.org/10.1117/12.2521739.

Titel: Review of image segmentation techniques for layup defect detection in the Automated Fiber Placement process
A comprehensive study to improve AFP inspection
verfasst von: Sebastian Meister
Mahdieu A. M. Wermes
Jan Stüve
Roger M. Groves
Publikationsdatum: 17.05.2021
Verlag: Springer US
Erschienen in: Journal of Intelligent Manufacturing / Ausgabe 8/2021
Print ISSN: 0956-5515
Elektronische ISSN: 1572-8145
DOI: https://doi.org/10.1007/s10845-021-01774-3

Premium Partner

Marktübersichten

Die im Laufe eines Jahres in der „adhäsion“ veröffentlichten Marktübersichten helfen Anwendern verschiedenster Branchen, sich einen gezielten Überblick über Lieferantenangebote zu verschaffen.

Zur Marktübersicht

Calculation time [ms]
Algorithm	FI	GB
GT	67.9 (\(\sigma = 3.9\))	22.9 (\(\sigma = 1.0\))
CT	143.4 (\(\sigma = 8.4\))	69.8 (\(\sigma = 0.6\))
IP	51.3 (\(\sigma = 3.9\))	11.8 (\(\sigma = 1.0\))
OT	30.7 (\(\sigma = 3.2\))	8.0 (\(\sigma = 1.6\))
AT	30.0 (\(\sigma = 4.0\))	7.6 (\(\sigma = 0.1\))
GF	340.5 (\(\sigma = 49.3\))	320.6 (\(\sigma = 8.6\))
MS	21.0 (\(\sigma = 1.9\))	3.0 (\(\sigma = 0.6\))

Springer Professional

Abstract

Publisher's Note

Introduction

Related research

Manufacturing process

Sensors for data acquisition

Image processing techniques

Methodology

Experimental setup

Evaluation of detection methods

Implementation of selected algorithms

Image pre-processing and parameter adjustment

Evaluation of results

Results

Discussion

Future work and prospect

Conclusion

Declarations

Conflict of interest:

Code availability:

Publisher's Note

Weitere Artikel der Ausgabe 8/2021

DyS-IENN: a novel multiclass imbalanced learning method for early warning of tardiness in rocket final assembly process

Correction to: Quality 4.0: a review of big data challenges in manufacturing

Real-time edge framework (RTEF): task scheduling and realisation

Prediction and optimization of performance measures in electrical discharge machining using rapid prototyping tool electrodes

Multitask learning for health condition identification and remaining useful life prediction: deep convolutional neural network approach

Machine learning to determine the main factors affecting creep rates in laser powder bed fusion

Premium Partner

Marktübersichten