To Detect Defects Which Are Three-Dimensional Changes by Using Their bird’s Eye View Images and Convolutional Neural Networks

The current understanding is that automating inspection is one of the most important issues in the manufacturing industry, and many studies are being conducted for automated inspection. Recently, there has been a lot of research on detecting defective products from images using neural networks. Previous research has focused on detecting only two-dimensional defects (flaws, chips, etc.) from images and not three-dimensional defects (warpage, overlap, etc.) using images. Additionally, the equipment required for 3D inspection tends to be expensive, large-scale, and costly, so the initial cost is high. Many manufacturing sites have not been able to introduce it. In this study, we perform the 3D inspection using industrial cameras introduced in many manufacturing sites. Specifically, we will detect 3D defects such as war pages of transport trays used in actual manufacturing sites from 2D images. First, since the captured image contains noise due to many shooting environments, appropriate image processing is applied to remove the noise. The two types of images used in this process are those taken at the actual manufacturing site and those taken in a laboratory with a good shooting environment. We took the tray images in the laboratory at different angles between 10 and 90 in 10° increments. The processed images are then inputted to transition trained CNNs (Convolutional Neural Networks) for deep learning to perform binary classification of abnormal and normal. We also use Grad-Cam to visualize the learning to understand which part of the image the network focused on for classification. As a result, it shows that the network was unsuccessful in performing binary classification on any birds-eye view images. In comparison, the Grad-Cam visualization results show that the network obtained candidate features for tray warping, a 3D change, from the 30 to 40° images for the images taken at the manufacturing site and the images taken in the laboratory.