Abstract
As the traditional methods of insulator fault detection rely on the low-level feature extraction of images and classifier design, it is difficult to achieve fault detection of insulator for images with complex background. To address this issue, a fault detection method using second-order full convolutional network (SOFCN) is proposed in this paper. Firstly, the first-order FCN is used to learn the image features to segment insulator areas from images with complex background. Secondly, the mathematical morphology reconstruction operation is used to improve the segmentation result to get the accurate localization of the insulator areas. Finally, the FCN network is used again to detect the insulator fault and obtain the fault region. Experiments show that the proposed SOFCN is not only able to obtain accurate insulator region, but also able to effectively suppress the interference of noninsulator region. Compared to the conventional methods, the proposed SOFCN obtains higher recognition accuracy without feature extraction and the selection of a classifier. Moreover, the computational complexity of the proposed method is low. Furthermore, compared to the classical CNN and FCN segmentation methods, the proposed SOFCN can effectively suppress complex background interference to improve the accuracy of insulator fault detection.
1. Introduction
As the core component of the power line, insulator is exposed to the outside for a long time and eroded by the external force in natural, which is prone to breakage, dropping, and aging. Once insulator fails to work, which will cause the interruption of the entire transmission line or widespread power failure. To ensure the safe and reliable operation of the entire transmission line, it is necessary to timely and effectively check the transmission line insulators and find troubleshooting [1–6]. With the development of smart grid technology, unmanned aerial vehicle (UAV) inspection technology is relatively mature in the application of power transmission line inspection [7, 8]. Compared with the traditional manual inspection, the UAV inspection has a clear advantage because it can go deep into the high-voltage working area to achieve automatic inspection. Besides, automatic inspection greatly reduces the workload, shortens the inspection cycle, and reduces the false detection rate as well. Therefore, insulator recognition based on aerial images becomes an important evidence used for judging the operation state of the transmission line.
A lot of relevant researches on the fault detection of insulators for aerial images have been reported in recent years [9–15]. Jiang et al. [16] proposed the method of insulator fault recognition based on multifeature fusion. The method improves the accuracy of fault recognition. But it generates poor results for insulator images with complex background since the selected threshold value is fixed. For the problem, Xu et al. [17] proposed a new algorithm for insulator image segmentation by using the idea of local mean value. This method improves the segmentation effect of insulator images with multiple peaks and valleys, but the segmentation effect is still largely affected by the segmentation threshold. Therefore, Shan et al. [18] achieved insulator defect detection by combining the morphological features and the AdaBoost classifier. Compared with unsupervised learning algorithms, this approach had strong capabilities of self-adaption and self-learning, for it accumulates multiple weak classifiers into strong classifiers and learns insulator features from training samples. Liu et al. [19] proposed an insulator detection method using deep convolutional neural networks. The method can effectively detect insulators, and thus it has a strong robustness, but it has a low computational efficiency and it is unsuitable for images with complex background. Tao et al. [20] proposed a new deep CNN cascade structure for insulator location and defect detection. The method achieves the requirement of high robustness and accuracy for insulator defect detection. Cheng et al. [21] employed faster R-CNN framework to learn and identify aerial insulators. The method has a high recognition accuracy and efficiency, and it is able to accurately identify various types of insulator. However, this method is invalid in the case of occlusion issue between insulators. Although the methods mentioned above achieve the fault detection of insulators, they have two shortcomings. On the one hand, the traditional method requires extracting features of insulators by designing feature descriptors. On the other hand, the traditional approaches obtain a low accuracy of fault recognition for complex images because the proposed model is only effective for simple images.
Aiming at the two problems, an insulator self-explosion fault detection method using second-order full convolutional network (SOFCN) model is proposed for insulator images with complex background. Firstly, the insulator region is marked to generate label images used for training network. The image features are learned by using the first-order FCN, and then the insulator region is segmented. Secondly, the mathematical morphology reconstruction operation is used to optimize the segmentation result, so that the accurate location of insulator area can be obtained. Finally, based on the regional insulator image, the second FCN network is used to detect the fault insulators. This method can effectively suppress the interference of noninsulator regions and thus improve the recognition accuracy of fault insulators.
2. Convolutional Neural Network
Traditional insulator fault recognition methods require manual feature descriptors and classifier design, which leads to a result that only shallow image features are extracted. Moreover, feature extraction results are poor for insulator images with complex background. Therefore, the conventional approaches easily result in missed detection, false detection, and low recognition rate. With the extensive application of deep learning in visual recognition, speech recognition, natural language processing, and image classification and segmentation, it adapts to the characteristics of big data and automatically extracts features in layers, which provides a new way for insulator fault detection based on aerial photography images. Moreover, insulator fault detection using deep learning has become a new research hotspot [22–29].
At present, insulator fault detection methods using deep learning mainly depend on the general convolutional neural network (CNN). Compared with the traditional approaches, CNN can automatically learn hierarchical image features, where the shallow convolutional layer has a small perception field and it is therefore used for learning local regional features. The deeper convolutional layer has a large perception field, which is used for learning more abstract high-level semantic features. The deep convolutional network takes into account the global and local information of the image. At the same time, the abstract features learned by deeper convolution layers are insensitive to the size, position, and direction of the object, which is thus suitable for insulator fault detection. However, the method only classifies the image pixels and considers the image blocks around the pixels as the input of the CNN for training and prediction. Thus, the method cannot obtain the segmentation result directly. Due to the redundancy in the adjacent pixel blocks, the amount of image input data is too large and the processing speed is slow. Simultaneously, as the size of pixel neighborhood is difficult to be determined and the spatial location information of images cannot be taken into account efficiently, the accurate contour of objects cannot be well obtained [30].
FCN is a fully convolutional network that realizes the leap from image classification to segmentation; it is thus able to improve CNN for image semantic segmentation. Compared with the traditional CNN, FCN is an end-to-end network that does not limit the input image size, which can perform dense prediction without the full connection layer. It eventually generates segmentation maps of any size and improves the processing speed and segmentation accuracy. However, the loss of detailed information is a problem caused by the expansion of the perceptual field of view in the FCN network. When an insulator image with complex background is directly used for fault detection, it is difficult to accurately locate the insulator area, which results in false detection [31–34].
3. Method
3.1. Area Insulator Segmentation
The FCN network is an end-to-end and pixel-to-pixel training that allows the network to make pixel-level predictions, and it obtains higher segmentation accuracy without any preprocessing. The FCN model replaces the fully connected layer in the convolutional neural network with the convolutional layer without limiting the size of the input image. For different-size insulator images, feature extraction is carried out by automatic delamination. Compared with the traditional method of artificial feature extraction, the effective features of insulator images can be extracted, and the recognition accuracy of insulator images can be improved.
Figure 1 shows the typical VGG-16 FCN network structure, where convolution and pooling operations are repeatedly performed for five times. After the operation of each group, the original image size is the half of the image at previous layer. After the fifth conv5 and pool5 operations, the image size is reduced to 1/32 of the original image size, followed by two convolution operations. The obtained feature map is a high-dimensional vector named heatmap, and it represents architecture between the whole original image. If the heatmap is directly upsampled, then the obtained prediction result is coarse and the local detail information is missing. To improve the final segmentation result, the 1/32-size heatmap is first upsampled two times, and the enlarged image is fused with the featuremap at the layer of pool4. Repeating the previous operation and the enlarged image is further fused with the featuremap at the layer of pool3. Finally, the 8 times’ upsampling is performed to obtain the segmented image. The FCN network structure comprehensively considers the local and global information of the insulator image, and it is thus able to improve the final segmentation accuracy. The detailed operation of convolution and pooling, and the FCN network training process are presented as follows.
FCN convolution process iswhere is the number of layers in which the convolution layer is located, is the input feature map combination, is the convolution kernel matrix, is the bias, and is the activation function; we use the linear correction function ReLU in this paper.
FCN pooling process is where is the sampling function and the maximum pooling function is used in the paper.
Forward propagation process of FCN network is as follows.
The process is similar to that of BP neural network, and it computes the actual output of training samples after layer-by-layer transmission.where is the number of layers of the neural network, is the weighted input of the layer of neurons, is input data for each layer of the corresponding images, and are the weight and bias of each layer of neurons in the full convolutional neural network, and is a linear correction function ReLU.
FCN network backpropagation process is as follows.
The process is the reverse transfer process of gradient or error. The detailed formula is given as follows:where is the objective function, is standard output, and is predicted value. By minimizing an objective function using the stochastic gradient descent method, the optimal values of and can be obtained.
3.2. Morphological Reconstruction Filter
Morphological reconstruction can filter noises and useless texture details while preserving the contour details of important objects. For the insulator image preliminarily segmented by FCN, it often includes some false detection areas, so the morphological reconstruction filter is used to accurately locate the insulator region. Let and be an image defined on the same discrete domain D and a mask image, respectively. The mask image is reconstructed by the identified image until it is stabilized. We have the specific reconstruction process.
Erosion reconstruction operation process is as follows.
Let , denotes erosion reconstruction;where is the number of iterations and satisfies .
Dilation reconstruction operation process is as follows.
Let and denote erosion reconstruction;where satisfies .
In Figure 2, we can see that the segmented result obtained by FCN includes lots of false insulator areas because the tower and other background are similar to insulators, which will bring interference to the later fault detection. After reconstructing and filtering, the false areas are removed and the real insulator areas are maintained.
3.3. Fault Detection Based on Regional Insulator
To detect fault insulators, the obtained image including insulator areas as shown in Figure 2 is multiplied with the original image, which is able to remove the complex background such as the pole and tower line in the original image. The result is simpler than the original image and it is considered as training images that are the input of the second-order FCN network. Finally, the trained model is used to detect fault insulator. The proposed framework is shown in Figure 3.
3.4. Algorithm Description
The single-layer FCN network leads to the loss of the target location and other detailed information due to the expansion of the perception field of the pooling layer and the polymerization context. For the insulator image of complex background, it is difficult to accurately locate the regional insulator when the fault detection is directly performed, which causes false detection of partial background as the foreground, and the improvement of accurate fault identification rate is limited. Therefore, an insulator fault detection method using the SOFCN model is proposed. As shown in Figure 4, the SOFCN model includes a two-layer FCN network. The first layer FCN is used for the positioning of the insulator region, and the region insulator image is first segmented in a complex background. The second-stage FCN network can identify the fault insulators. For the initial segmentation of the insulator region image, the morphological reconstruction is used to filter out the false detection of the background as the foreground, after accurately locating the region insulator, and multiply it with the original image as the input set of the second layer FCN network for fault detection. The steps of the algorithm are presented as follows.
Input: Aerial insulator image
Step 1. Initialize the FCN network, where in the convolution kernel size is 33, the learning rate is , and the number of iterations is 100,000 times, and the input insulator image is normalized to the size of 400600.
Step 2. Input the training images and the corresponding label images into the first-order FCN network and then train the network.
Step 3. Perform morphological reconstruction filtering on the initially segmented insulator image to obtain detected insulator images.
Step 4. Multiply the reconstructed image by the original image to obtain a region insulator image that removes the background of the original image.
Step 5. The region insulator images and label images are inputted into the second-order FCN network model, and then train the network.
Step 6. Output the detection result of fault insulators.
Output: Detection result of fault insulators.
4. Experimental Results and Analysis
To demonstrate the effectiveness of the algorithm, the paper compares the regional insulators and fault recognition effects separated by the proposed SOFCN with the traditional morphological erosion combined branch segmentation algorithm, CNN segmentation algorithm, and FCN segmentation algorithm.
4.1. Dataset
The testing images in this paper are derived from the UAV 330kv line inspection insulator image. They are composed of 300 insulator images. The experimental environment is IW4206-2Q deep learning workstation, Ubuntu 16.04 64-bit operating system, 64 GB memory, NVIDIA GeForce GTX10802 graphics card, and CPU E5-1602V4.
4.2. Comparison of Regional Insulator Segmentation Results
The detection results of insulators on images with simple background are shown in Figure 5. The figure shows that both the comparative approaches and the proposed SOFCN can accurately detect the region of insulator. However, the traditional method requires artificial morphological segmentation and the selection of the connected segmentation threshold; it thus has a low practicality. Compared with the traditional method, CNN extracts feature maps by automatic layering to locate the insulator region. However, because CNN only classifies image pixels, the image blocks around the pixels are applied as input of the network to train and predict, and the results of the segmentation are indirectly obtained. There is redundancy in adjacent pixel blocks, too large amount of image input data, and slow processing speed. Moreover, it is difficult to decide the neighborhood window of pixels, and the spatial position information in the image cannot be considered, and it is not well recognized that the specific contour of the object is difficult to accurately segment. Compared with the CNN segmentation method, the FCN network fuses the low-level features by forward iteration to improve the segmentation accuracy of the region insulator. The proposed SOFCN is further reconstructed and filtered based on the FCN segmentation, which can also realize fine segmentation of insulator regions in simple background.
(a)
(b)
(c)
(d)
(e)
The results of insulator segmentation in complex background regions are shown in Figure 6.
(a)
(b)
(c)
(d)
(e)
In Figure 6, the insulator images have complex background; it is difficult to determine the segmentation threshold by using traditional methods. For complex insulator images with multiple segmentation thresholds, they are greatly affected by similar backgrounds, such as pole and tower and lines, and the segmentation effect is poor. Compared with the traditional method, CNN can realize the coarse positioning of the insulator region, but the segmentation effect is rougher due to the problem of determining the neighborhood size of the pixel. Compared with the CNN method, the FCN segmentation method directly predicted the pixels without limiting the size of the input image and improved the processing speed. At the same time, the insulator region segmentation results were relatively fine which combined the high-dimensional features and low-order features. However, due to the expansion of the perceptual field of view in the FCN network, the polymerization context leads to the loss of detailed information such as the target position. For insulator images with complex backgrounds, it is difficult to accurately locate the regional insulators, which causes partial background false detection to be an insulator region. In the SOFCN, the morphological filtering operation is added to the FCN network when the insulator region is segmented, and the background which is misdetected as the insulator region is filtered out. Then the precise segmentation of the insulator region is realized.
4.3. Fault Recognition Effect Comparison
Because the traditional morphological corrosion combined with the branch method can only realize the segmentation of partial insulator images, it is difficult to recognize the complex background insulator fault. Therefore, the paper compares and analyzes the insulator fault recognition effect of this algorithm with CNN segmentation algorithm and FCN segmentation algorithm.
The ceramic insulator fault detection effect is shown in Figure 7. To facilitate the intuitive observation of the fault identification effect, we superimposed the identified faults on the original images for comparative analysis.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
It can be seen from Figure 7 that pixel neighborhood size is difficult to be determined, spatial location information cannot be taken into account, and the fault segmentation is rough when using CNN segmentation method for insulator fault detection. Especially when the distance between two faults is close, it is easy to be mistakenly detected as a fault. In comparison, the FCN segmentation method integrates the high-dimensional feature and low-level feature information to improve the segmentation accuracy. However, due to the enlargement of the perceptual horizon in the pooling layer and the polymerization of contexts, details such as target location are lost, part of the background is mistakenly detected as the fault part, and the fault recognition accuracy is limited. In contrast, the reconstruction filtering method is used to filter the background that is mistakenly detected as the foreground by the initial FCN segmentation. The image of the regional insulator is obtained. Based on the regional insulator image, fault recognition is conducted, effectively removing the interference from the fault recognition caused by the complex background.
To accurately evaluate the insulator segmentation effect, the paper introduces the pixel prediction accuracy of the segmentation image as the evaluation index of the reconstruction result. The specific formula is as follows:where is the class of image pixels, is the set of i-th pixels that predict the correct prediction image, is the set of i-th pixels of the standard image, is the set of j-th pixels of the standard image, and is the pixel prediction accuracy.
The corresponding prediction accuracy is shown in Table 1.
To analyze the accuracy of insulator fault recognition, we introduce the pixel prediction accuracy rate SA as comparative analysis.
It can be seen from Table 1 that the CNN segmentation result is rough and the fault recognition accuracy is low. Compared with CNN, the FCN segmentation has a finer fault recognition accuracy, but there is partial misidentification. Compared with FCN, the SOFCN reduces the false recognition rate and improves the detection accuracy.
The glass insulator inspection effect is shown in Figure 8.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
In Figure 8, influenced by factors such as light, the pixel points at the glass insulator are very close to the background, which leads to the fault detection of the insulator directly when CNN and FCN are used, the fault location is missed or mistakenly detected as the background, and the recognition effect is poor. In contrast, the proposed SOFCN is based on the regional insulator to detect the fault, which avoids the interference caused by the similar background, recognizes the fault of glass insulator, and improves the accuracy of the glass insulator fault detection. The detailed fault recognition accuracy rate is shown in Table 2.
It can be seen that the detection accuracy of CNN and FCN is low according to Table 2. There are many phenomena of missed detection and false detection of insulator faults, and the recognition effect is poor. Compared with CNN and FCN segmentation algorithm, the accuracy of insulator fault recognition in the proposed SOFCN is obviously improved, which is an effective insulator fault detection method.
5. Conclusion
Aiming at the fault identification of aerial insulators in images with complex background, this paper proposed a method using SOFCN model for insulator fault detection. Firstly, the first-order FCN is used to learn the image features, and thus insulator regions are extracted from complex background. Then the mathematical morphology reconstruction operation is used to optimize the segmentation result to obtain the accurate positioning of the insulator regions. Finally, the SOFCN network is used to detect the insulator fault. The method can effectively eliminate the interference of complex background of noninsulated region. Compared with conventional CNN and FCN, the segmentation accuracy is obviously improved, and the insulator fault in images with complex background can be accurately identified.
Data Availability
The insulator images data used to support the findings of this study have not been made available because it was obtained from the State Grid Corporation of China and has confidentiality.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
The authors acknowledge the financial support of the National Natural Science Foundation of China (nos. 61871259, 61461025, 61871260, and 61811530325).