Bildverarbeitung in der Automation

Anomaly detection (AD) methods that are based on deep learning (DL) have considerably improved the state of the art in AD performance on natural images recently. Combined with the public release of large-scale datasets that target AD for automated visual inspection (AVI), this has triggered the development of numerous, novel AD methods specific to AVI. However, with the rapid emergence of novel methods, the need to systematically categorize them arises. In this review, we perform such a categorization, and identify the underlying assumptions as well as working principles of DL-based AD methods that are geared towards AVI. We perform this for 2D AVI setups, and find that the majority of successful AD methods currently combines features generated by pre-training DL models on large-scale, natural image datasets with classical AD methods in hybrid AD schemes. Moreover, we give the main advantages and drawbacks of the two identified model categories in the context of AVI’s inherent requirements. Last, we outline open research questions, such as the need for an improved detection performance of semantic anomalies, and propose potential ways to address them.

Die Sicherstellung qualitativ hochwertiger Bilder in der automatisierten optischen Inspektion (AOI) auf der Basis von Bildverarbeitungsmethoden ist eine notwendige Voraussetzung, um sinnvolle Entscheidungen zu treffen. Verschiedene Einflussfaktoren können sich jedoch negativ auf diese Aufgabe auswirken und zu ungeeigneten Bildern führen. Der Systemaufbau, die Charakteristika des zu untersuchenden Objekts, wie beispielsweise Unregelmäßigkeiten oder Muster, sowie die Durchführung der Untersuchung sind nur einige Beispiele von vielen. In Anwendungsfällen, in denen eine hohe Bildqualität aufgrund der genannten Einflussfaktoren nicht sichergestellt werden kann, ist es erforderlich, die erfassten Bilddaten automatisch hinsichtlich ihrer Eignung für eine robuste automatisierte optische Inspektion zu bewerten. Zu diesem Zweck müssen geeignete Bewertungsmetriken verglichen und entsprechend den spezifischen Anforderungen des Anwendungsfalls ausgewählt werden. In diesem Beitrag wird eine in den laufenden Betrieb eines AOI-Systems integrierte Bildqualitätsbewertung vorgestellt. Zu diesem Zweck wird im Prozess zwischen der Bilderfassung und der Bildauswertung ein zusätzliches Modul integriert, das in der Lage ist, nicht verwertbare Bilder zu detektieren. Die Anwendung wird anhand eines Demonstrators gezeigt, der Erkenntnisse für die konkrete Umsetzung von AOI-Aufgaben liefern soll.

Im Fabrikkontext spielt die Detektion von Personen eine wichtige Rolle bei Maßnahmen zur Erhöhung der Sicherheit von Arbeitern oder zur Optimierung des Fabriklayouts. Kamerasensoren bilden die Grundlage für robuste bildbasierte Personendetektionsverfahren, werden aber aufgrund von Datenschutzaspekten häufig kritisch gesehen. Diese Bedenken können durch die Lokalisation von IoT-Devices, die von Personen getragen werden, adressiert werden, jedoch muss eine Person stets mit einem entsprechendem IoT-Device ausgerüstet sein. In diesem Beitrag wird ein alternativer Ansatz zur Adressierung der Problematik vorgeschlagen, der auf etablierten Bildverarbeitungsverfahren beruht, jedoch inhärent DSGVO-konform ist. Hierzu wird distanzmessende 3D-LiDAR-Sensorik genutzt, um 3D-Punktwolken der Umgebung aufzunehmen. Diese ermöglichen eine Detektion von Personen (Klassifikation und Lokalisation), jedoch keine Identifizierung der Personen. Hierfür wird ein Verfahren vorgestellt, das einzelne Objekte in einer Punktwolke zunächst in ein Tiefenbild umwandelt, um auf diesem anschließend robuste Bildverarbeitungsverfahren basierend auf Deep Learning einzusetzen. Die Evaluation des Verfahrens zeigt eine Genauigkeit (Accuracy) von 98%, um zwischen Personen und anderen Objekten zu unterscheiden, und ist somit für darauf aufbauende Anwendungen gut geeignet.

Object recognition is well known to have a high importance in various fields. Example applications are anomaly detection and object sorting. Common methods for object recognition in images divide into neural and non-neural approaches: Neural-based concepts, e.g. using deep learning techniques, require a lot of training data and involve a resource intensive learning process. Additionally, when working with a small number of images, the development effort increases. Common non-neural feature detection approaches, such as SIFT, SURF or AKAZE, do not require these steps for preparation. They are computationally less expensive and often more efficient than the neural-based concepts. On the downside, these algorithms usually require grey-scale images as an input. Thus, information about the color of the reference image cannot be considered as a determinant for recognition. Our objective is to achieve an object recognition approach by eliminating the “color blindness” of key point extraction methods by using a combination of SIFT, color histograms and contour detection algorithms. This approach is evaluated in context of object recognition on a conveyor belt. In this scenario, objects can only be recorded while passing the camera’s field of vision. The approach is divided into three stages: In the first step, Otsu’s method is applied among other computer vision algorithms to perform automatic edge detection for object localization. Within the subsequent second stage, SIFT extracts key points out of the previously identified region of interest. In the last step, color histograms of the specified region are created to distinguish between objects that feature a high similarity in the extracted key points. Only one image is sufficient to serve as a template. We are able to show that developing and applying a concept with a combination of SIFT, histograms and edge detection algorithms successfully compensates the color blindness of the SIFT algorithm. Promising results in the conducted proof of concept are achieved without the need for implementing complex and time consuming methods.

Research of image recognition allows for improvements in animal welfare compliant and increase in data yield in animal experiments. One application for improvements are the so-called rotational tests with rats in Parkinson research. Here, the Rat Rotation Detection System (RRDS) frees the rat from the usually used breast belt while achieving similar results as the previous system, with a difference of 12.4 %. RRDS basically consists of an off-the-shelf camera combined with a YoloV4″​=Neural″​=Network, which detects the coordinates of the head, the tail, and the torso of the rat. With these coordinates, RRDS calculates two vectors, which are further used to calculate the rotation of the rat. The RRDS is a step towards improved animal welfare and more accurate results in animal experimentations.

Deep learning-based defect detection is rapidly gaining importance for automating visual quality control tasks in industrial applications. However, due to usually low rejection rates in manufacturing processes, industrial defect detection datasets are inherent to three severe data challenges: data sparsity, data imbalance, and data shift. Because the acquisition of defect data is highly cost″​=intensive, and Deep Learning (DL) algorithms require a sufficiently large amount of data, we are investigating how to solve these challenges using data oversampling and data augmentation (DA) techniques. Given the problem of binary defect detection, we present a novel experimental procedure for analyzing the impact of different DA-techniques. Accordingly, pre-selected DA-techniques are used to generate experiments across multiple datasets and DL models. For each defect detection use-case, we configure a set of random DA-pipelines to generate datasets of different characteristics. To investigate the impact of DA-techniques on defect detection performance, we then train convolutional neural networks with two different but fixed architectures and hyperparameter sets. To quantify and evaluate the generalizability, we compute the distances between dataset derivatives to determine the degree of domain shift. The results show that we can precisely analyze the influences of individual DA-methods, thus laying the foundation for establishing a mapping between dataset properties and DA-induced performance enhancement aiming for enhancing DL development. We show that there is no one-fits all solution, but that within the categories of geometrical and color augmentations, certain DA-methods outperform others.

Manufacturing companies face the challenge of combining increasing productivity and quality standards with customer″​=oriented mass production. To achieve the required quality standards, quality controls are carried out after selected production steps. These are often visual inspections by trained personnel based on checklists. To automate visual inspection industrial, cameras and powerful machine vision algorithms are needed. Large amounts of visual training data are usually required in order to train these algorithms. However, collecting training data is time″​=consuming, especially in customer″​=oriented mass production. Synthetic training data generated by CAD tools and rendering software can alleviate the lack of available training data. Within the paper at hand, a novel approach is presented examining the use of synthetic training data in machine vision applications. The results show that synthetically generated training data used to train machine vision quality gates is fundamentally suitable. This offers great potential to relieve process and productions developers in the development of quality gates in the future.