Computer Vision & Laser Vibrometry, Vol. 6

Recent years have seen a significant increase in the use of optical techniques for structural health monitoring and structural dynamics as an alternative to traditional contact-based methods. Techniques such as three-dimensional digital image correlation (3D-DIC) and three-dimensional point tracking (3DPT) have been widely used to extract full line-of-sight displacements and deformations of structures by analyzing images acquired from synchronized stereo cameras. To provide the necessary contrast for the correlation algorithm to work, the application of recognizable high-contrast markers in the form of a stochastic speckle pattern or optical targets on the surface of the structure of interest is necessary. This requirement can quickly become an obstacle to the analysis of large-scale civil and mechanical engineering structures where access, and therefore the ability to apply speckled patterns or optical targets, is limited. This research aims at overcoming this limitation by identifying and tracking naturally occurring features on the surface of the structure of interest (e.g., oil spills, rust stains, printed letters, bolts). This is achieved by using a newly proposed Augmented Centroid-Based Detector (A-CBD) method to extract features from an object of interest and track their displacement in two synchronized images over time. In this paper, the performance of the A-CBD method is compared with traditional 3DPT measurements performed using optical targets by extracting the time and frequency domain response of a cantilever beam. The results of the laboratory tests performed show that the precision of the proposed A-CBD is comparable to traditional 3DPT, yielding Time Response Assurance Criterion (TRAC) values consistently above 98% for both in-plane and out-of-plane displacements.

A laser Doppler vibrometer can measure the surface velocity of a point on a structure. A continuously scanning laser Doppler vibrometer (CSLDV) was developed to significantly improve efficiency and spatial resolution of vibration measurement of the structure. As a non-contact system, it can avoid the mass-loading problem in vibration measurement using accelerometers. The CSLDV was made by adding two orthogonal scan mirrors in front of a single-point laser Doppler vibrometer. Two scan mirrors can be referred to as X and Y mirrors based on their rotation axes, respectively. During CSLDV measurement, two scan mirrors can be controlled to continuously rotate about their rotation axes, and the laser spot of the CSLDV can continuously move along a pre-designed scan trajectory on the structure, which is a major difference compared to a conventional scanning laser Doppler vibrometer (SLDV) system that has a point-by-point scanning capability. This tutorial first overviews principles in vibration measurement using a CSDLV, such as signal processing methods for structures under various excitations such as sinusoidal, impact, and random excitations, and scan trajectory design methods for structures with various shapes. Recent developments on (1) a novel general-purpose three-dimensional (3D) CSLDV system for measuring 3D full-field vibration of a structure with arbitrarily curved surfaces and (2) a novel zero-contact image-based tracking CSLDV system for measuring vibration of a rotating structure are presented. The general-purpose 3D CSLDV system can measure vibrations of difficult to access areas of structures with the assistance of reflective mirrors and obtain their 3D panoramic modal parameters through a novel vibration stitching method. The image-based tracking CSLDV system can track and scan a rotating structure such as a rotating wind turbine blade through a novel edge detection method and estimate its modal parameters through an improved lifting method and an improved demodulation method. Applications of continuous scanning laser vibrometry to structural damage detection will be discussed.

Experimental modal analysis (EMA) is a technique that helps to identify the natural frequencies, modal damping, and mode shapes of a structure. The traditional approach to modal analysis involves using pointwise sensors like accelerometers, which only offer pointwise measurements. However, this method may not be sufficient for validating and updating Finite Element (FE) models using experimental data. On the other hand, numerical simulation of a structure’s dynamic behavior through FE Analysis (FEA) can provide full-field results, but the lack of points/DOFs creates a challenge for validating and updating the FE models.

Digital Image Correlation (DIC) is a powerful and non-intrusive optical method that can provide full-field measurements of the mode shapes. DIC data can guide the FE model validation and update procedure, which enables a more accurate representation of the structure’s dynamic behavior. This enhanced predictive capability of the FE model allows for better decision-making regarding the structure’s performance and durability.

This paper highlights the application of DIC in modal analysis and presents a comparison and data merging with traditional sensors to validate the obtained results. Additionally, the paper demonstrates the effectiveness of DIC in accurately characterizing the dynamic behavior of structures and improving the predictive capability of FE models through FE model validation and update.

A novel robust edge detection method is developed for an image-based tracking continuously scanning laser Doppler vibrometer (CSLDV) system to track a rotating structure without attaching any mark or encoder to it and scan its whole surface. The robust edge detection method can extract edges of the rotating structure from its complex background by processing grayscale images of the rotating structure. Once edges of the rotating structure are clearly shown in the processed image, their positions can be easily determined. To track a rotating structure like a rotating blade of a wind turbine with multiple blades, a distance condition is used to determine the position of an edge of the structure that needs to be tracked and estimate its real-time rotation speed. Once the position of the edge of the rotating structure is determined, the image-based tracking CSLDV system can scan the whole surface of the rotating structure via a two-dimensional scan scheme. An improved demodulation method is used to process measured data of response of the rotating structure under random excitation and estimate its modal parameters including damped natural frequencies and full-field undamped mode shapes. The robust edge detection method is investigated by tracking and scanning a rotating fan blade under random excitation with a complex background using the image-based tracking CSLDV system, and estimating modal parameters of the rotating fan blade with different constant speeds.

Environmental conditions such as wind and ground traffic will introduce motions in camera systems, which contribute as noise and thus affect measurement accuracy. The conventional camera motion correction methods need to track static reference objects by one or multiple cameras, reducing applicability and increasing costs. This study proposes a novel 6-degree-of-freedom (DOF) camera motion correction method based on an inertial measurement unit (IMU) sensor. The Kalman filter is adopted as a data fusion method to estimate the camera orientation and translation. Six pinhole camera models are built to evaluate and correct 6-DOF camera motions. The system hardware configuration is detailly introduced. The motion correction efficiency and robustness have been tested under different object distances and focal lengths. The motion correction ratio has been statistically analysed and achieved approximately 80%. The object distance has little effect on the correction ratio.

In this work, we demonstrate the unique advantages of using event-based imagers for optical in-process monitoring of metallic manufacturing processes that feature a melt pool (e.g., welding, additive manufacturing). In-process monitoring is an important problem that must be solved to allow the use of additive manufacturing for mission-critical metallic components. Unfortunately, there are a number of challenges that make in-process monitoring difficult for the high-temperature, high light intensity environment presented by metallic additive manufacturing. The first problem is that conventional imagers become saturated and typically do not have the dynamic range needed to observe melt pool processes. High dynamic range imagers exist, but they typically only have a framerate on the order of tens of Hertz which is not suitable for the fast dynamics occurring in a melt pool on the order of hundreds of Hertz. High speed laser illumination can be used with high-speed cameras, but these approaches are expensive and result in extremely large amounts of data (order terabytes) not suitable for in-process monitoring or forming digital twins of additively manufactured components. To solve these issues, we propose the use of event-based imagers. Event-based sensing is an alternative measurement paradigm where data is only transmitted/recorded when changes in light intensity exceeds a set threshold. The result is that event-based sensors consume less power and less memory/bandwidth, and they operate across a wide range of timescales and dynamic ranges. In this work we will demonstrate the high dynamic range properties of event-based imagers and their ability to observe melt pools produced by electric arcs and lasers. We demonstrate the efficient nature of the sampling associated with event-driven imagers in the context of observing melt pool dynamics and the resulting memory savings. We demonstrate the ability to detect and track contaminates in the melt pool. Finally we demonstrate unsupervised learning of principle components of melt pools in order to facilitate additional data compression for down-stream processing and control.

In tribology and material science, tribometers are used to investigate friction, wear, and lubrication under realistic conditions. Different tribometer configurations are available to reproduce the respective contact situation from application in the best possible way. In paper production, calender rolls improve the paper surface, and in order to keep the quality of the product constant, these components should hardly show any surface alteration or wear. To reproduce the tribological effects during paper production in the respective twin-disc tribometer, two discs are rolling against each other under realistic load and slip conditions to investigate the wear resistance of different roll covers.

During tribometer testing, just as in real production, dynamic machine effects can be detected, which partially influence the contact processes. Therefore, the movement of the test rig and the test specimen is recorded using different non-contact high-end measuring systems, to determine the influence of the vibrations on the contact zone. The stationary vibrations are precisely recorded using a 3D scanning vibrometer. Thermal effects are captured by a high-speed thermal camera. In order to visualize and analyze transient effects, the machine motion is also recorded using a high-speed camera. After tribometer testing, the worn roller surfaces are analyzed using 3D topography imaging. The investigations show that mechanical vibrations of the tribometer directly influence the wear behavior and therefore the knowledge of the system dynamics is important for the analysis of the underlying tribological effects. The combination of the different measurement methods allows us the development of an overall understanding of the complex processes in the tribometer.

Recently the Holistic Engine Rotor Measurement System (HERMES) has been installed in the Dynamics Lab at Imperial College London. It allows us to study the complex vibration behaviour of a spinning gas turbine in a well-controlled laboratory environment. As part of the test facility, a 3D-SLDV system has been purchased to provide highly dense vibration information of the engine casing. A detailed study of the modal response of the casing has recently been conducted, highlighting some very interesting modal behaviour, especially across the many flange joints of the engine. This paper will discuss some of the challenges in setting up and aligning the 3D-SLDV system on a relatively small but highly curved surface, introduce some of the obtained response data and discuss the implications of using full field 3D measurements to monitor engine casings.

Extracting 3D mode-shapes from vibrating structures in the field presents a number of challenges; accelerometers attached to a structure provide full 3D motion at high temporal resolution but cannot match the spatial resolution of conventional 2D imagers (both logistically by cost and practically by weight). Furthermore, installation costs associated with contact-sensors can be very high. Conventional 2D imagers have higher spatial resolution, but for wide area monitoring, perspective projection lenses must be used. The use of perspective projection results in spatial ambiguities which complicates extracting 3D geometry information from the scene. Time-of-flight imagers return 2D surfaces in 3D space but have lower frame-rates than conventional high-speed cameras. All three devices traditionally poll scenes at a uniform rate regardless of whether the scene is static, both consuming power unnecessarily and generating redundant data, warranting either post-processing or extended storage space. We propose a fusion between silicon retina event-based neuromorphic imagers and light field imager architectures to capture an ‘event-field.’ Event-based imagers differ from conventional frame-based cameras in that they do not capture frames and instead only report information for pixels at which the intensity value change exceeds a set threshold. As a result, event-based imagers tend to effectively sample the scene in an efficient manner that naturally adapts to the spatiotemporal scales observed in the field of view. Because the rate of events generated by the silicon retina varies proportional to the spatiotemporal frequency of dynamics within the observed scene, it consumes less power and generates less data than a conventional 2D imager operating at an equivalent spatial and temporal resolution. By using an array of silicon retina imagers (or micro-lens array fitted to a single silicon retina), it is possible to obtain an approximation of the depth in post-processing. Additional benefits of the silicon-retina light field imager include a larger dynamic range than conventional cameras, the ability to apply digital coded exposure, shift focus, and building stacked-focused images in post-processing. We demonstrate the utility of event-based light field imagers for structural dynamics applications, particularly those that feature constraints on power and memory while simultaneously needing to capture structural vibrations.

Spider webs serve not only as a tool for prey capture, but also as an instrument for vibration monitoring and prey localization. The enhancements in this monitoring process, leading to greater success in prey captures, are believed to provide spiders an evolutionary advantage. It is suggested that the web serves as an extension of the spider’s central nervous system, acting as a tool for signal processing. A key characteristic of spider web design, the eccentric location of the central hub, is investigated in this research. This chapter emphasizes experimental work using a high-speed camera to capture the web’s dynamic responses, potentially increasing the understanding of the web’s extended cognition function. With the high-speed camera, full-field displacements are identified and modal parameters are estimated. The resulting full-field modal shapes demonstrate how the hub’s eccentricity affects distinct spatial modal shapes, further reinforcing the importance of eccentricity in web’s design. With a change of the prey’s position, the natural frequencies of the web change. It was shown that the maximum change in the natural frequency increases with the increase of eccentricity. The insights gained from experimentally analyzing spider webs will aid in understanding the impact of design geometry on other network-like structures.

A novel general-purpose three-dimensional (3D) continuously scanning laser Doppler vibrometer (CSLDV) system was recently developed by the authors to measure 3D vibration of a structure with a curved surface. As a non-contact system, it can avoid the mass-loading problem in 3D vibration measurement using triaxial accelerometers. In previous studies, the 3D CSLDV system was used to measure 3D full-field vibration of a turbine blade with a curved surface and identify its operating deflection shapes (ODSs) and mode shapes. It had the same level of accuracy as that with a commercial 3D scanning laser Doppler vibrometer system but had much higher efficiency than the latter. However, the 3D CSLDV system can be limited by its field of view, which is a common problem for optical-based measurement devices. Moving the 3D CSLDV system to different positions to measure different parts of a test structure is not practical during 3D CSLDV measurement, since the system has to be re-calibrated once it has been moved, which can be time-consuming and introduce measurement errors. This work proposes a novel mirror-assisted testing methodology for 3D CSLDV measurement that aims to measure vibration of difficult-to-access areas of a structure without moving the 3D CSLDV system during the test and stitch ODSs of its different parts together to obtain its panoramic 3D ODSs. The proposed methodology includes a novel scan trajectory design method that uses virtual areas of the structure behind the mirror to conduct continuous and synchronous scanning of three laser spots, and a novel velocity transformation method that uses virtual positions of three laser heads behind the mirror to stitch ODSs of different parts of the structure together. To demonstrate the proposed methodology, 3D CSLDV measurement is conducted on an aluminum hollow cylinder specimen, which has difficult-to-access areas such as its side and back surfaces, with the assistance of the mirror to obtain its panoramic 3D ODSs corresponding to its first two modes. Comparison between identified ODSs of the hollow cylinder specimen from the experiment and mode shapes from its finite element model is made and modal assurance criterion values are larger than 0.98.

Event-based imagers are particularly interesting for capturing stand-off measurements of rapidly evolving physical phenomena and rapid transient events. Engineering applications such as structural dynamics/mechanics, rotating machinery monitoring, manufacturing in-process monitoring, transportation monitoring, smart cities, and robotics. Many engineering applications of imagers require exploiting the sub-pixel measurement properties of imagers for tracking motion and deformation at high spatial resolution. For scenes with dynamic elements, the phenomenon of motion blur becomes apparent when objects move over multiple pixels during the exposure time of a conventional camera. For event-based imager sensors that report on changes in illumination in a scene, motion blur is not present in the event data in a conventional sense. However, it is often necessary to visualize event data in a conventional frame-based video format for display on a conventional monitoring. Alternatively, there is interest in feeding event data into a conventional frame-based computer vision pipeline for downstream analysis. It is also desirable to perform data fusion between images captured by a conventional camera and events in a physically-motivated manner. In these cases, a technique to introduce motion blur phenomena into the frames in a physically principled manner is desirable. This work introduces the use of a digital coded exposure combined with irradiance interpolation at a pixel to generate frames that exhibit physically-motivated motion blur phenomena.

Photogrammetry is a noncontact technique that has the unique ability to provide spatially dense full-field measurements by tracking visible features presented on surfaces of a structure. Nevertheless, the process of surface preparation entails challenges but serves as a crucial prerequisite for effective feature tracking. Moreover, surface preparation may lead to unintended consequences, such as mass loading and permanent marks on the surface of the structure. In this chapter, a fully noncontact baseline-free damage detection method for plate-like structures is developed by tracking laser-projected features that provide a spatially dense measurement grid without any mass loading and permanent marks. A contactless excitation method is implemented by using an acoustic speaker to excite a plate-like structure at a frequency close to one of its natural frequencies. The displacements of the plate are measured by processing the images of the projected laser captured by a high-speed camera from which the operating deflection shape (ODS) of the plate is estimated. It has been well established that any local damage on a structure which can be characterized by a loss of stiffness results in local anomalies in the ODS of the structure, and this local defect can be manifested by observing the second derivative of the ODS, which is referred to as its curvature. Formulations for higher-order derivatives with different orders of accuracy including the second derivative are provided using central finite difference schemes. The effectiveness and robustness of the proposed method for identifying these local anomalies is thoroughly investigated and experimentally validated.

Recent advancements in machine learning and neural network algorithms have introduced new approaches for reconstructing full-field data from sparse sets of measurements data. However, these approaches have shown limited accuracy when applied to optical data due to the complex physical modeling requirements. For example, traditional autoencoders (AEs), a subset of neural network algorithms, lack the ability to capture complex phenomena and their underlying physics in the latent space. To overcome these limitations, this study proposes a novel framework called time-inferred sparse autoencoder (TIS-AE). The TIS-AE aims to learn the underlying physics of a system of interest by using a physics-based regularizer (i.e., the governing equation of the targeted system) to enhance the accuracy of data reconstruction. To validate the proposed approach, data were generated using a finite element model collected during the cooling process of a metallic plate from two different states A and B. The data from system A were then used to train the TIS-AE model, along with the physics-based regularizer. Then, the trained TIS-AE model was used to reconstruct the full-field data from system B using sparse measurements. The robustness of the proposed TIS-AE model was evaluated by using metrics such as average reconstruction error, average peak signal to noise ratio, and Kullback–Leibler divergence. If further developed, the TIS-AE holds potential for applications in structural health monitoring and nondestructive evaluation.

Digital Image Correlation (DIC) is a promising noncontact method for measuring the full-field dynamics of vibrating structures. This method needs one or two cameras to measure 2D or 3D dynamics. The frames of the recorded videos are then postprocessed to correlate the structure points at different times and obtain displacement information. A plethora of cameras is available—in terms of frame rate, resolution, versatility, and overall quality—the choice of which depends on the specific application and spending constraints. The idea behind this work is to evaluate the feasibility of making DIC-ready videos using cameras built into a device that is part of everyday life: the smartphone. The proposed approach is an easy-to-handle procedure, suitable for teaching purposes or an early qualitative investigation. This chapter discusses a simple experiment students or beginners can perform on simple vibrating structures. First, a simple 2D dynamics was investigated. The vibration of a clamped beam with moving constraints was recorded with a single smartphone. The beam had an L-type cross section to emphasize in-plane motion. The frames were postprocessed with open-source DIC software and the first modal shape was extracted. The goal of this assignment was to introduce students to the basics of DIC. Second, the 3D dynamics of a beam was studied with a single smartphone. The image of the beam was reflected by two mirrors, and the reflected images were recorded by a single smartphone. A tool was developed to split the recorded frame into two independent images. The images were postprocessed to obtain the full 3D dynamics of the beam. Third, the 3D dynamics of a beam was studied with two smartphones.

This work presents a novel method to estimate angular positions and angular velocities of rotating structures using edge detection with background subtraction. While non-contact methods have been proposed to measure the angular position and velocity of a rotating structure, there is an initial stage of physical interaction with the structure where a distinguishing feature is fixed on the structure’s surface that facilitates object detection and tracking via image processing techniques. The algorithm described in this work eliminates this step of physical interaction and maintains zero physical contact before, during, and after measurements are performed. The methodology is established using an experimental setup comprised of a horizontally fixed ceiling fan that mimics a three-bladed wind turbine. Laboratory experiments are conducted by using the proposed algorithm to measure the angular velocity of the fan at its high, medium, and low speed settings. A rotary encoder is used to measure the fan’s angular velocities and the data is compared to that of the proposed algorithm for validation. The lab-based methodology is extended for use outside a controlled environment, and simulations are performed on videos of large-scale wind turbines to demonstrate the algorithm’s performance and limitations when used in various environmental settings.

Dynamic ultrasonic waves can be utilized to assess concrete for defects. This study introduces an optimized approach using computer vision algorithms on ultrasound images to assist in localizing and quantifying substantial defects and components. To detect concrete details, the ultrasonic signals are assembled to create 3D images for the concrete material and then converted into 2D so-called B-scans. A computer vision algorithm was trained on a dataset of labeled images that show different levels of delamination as 2D ultrasound images. The algorithm can then analyze new ultrasound images and classify them according to the degree of delamination present. The computer vision techniques that can be used for this purpose include convolutional neural networks (CNN) by applying YOLO V5, commonly used for image classification tasks. In addition to detecting delamination, computer vision techniques can identify other abnormalities in concrete structures, such as cracks, voids, and inclusions. These techniques can help engineers and maintenance professionals to identify and address problems in concrete structures before they lead to more severe issues. The paper will present these methodologies and applications with a case study.