Computer Vision & Laser Vibrometry, Volume 6

Optical motion magnification (OMM) is a non-contact monitoring technique that is gaining popularity in the vibration analysis and structural health monitoring (SHM) communities. Using ordinary video of a targeted structure, OMM algorithms can extract imperceptible motion and identify issues that may lead to structural failure. This technique can produce not only qualitative magnified videos revealing exaggerated displacements but also quantitative assessments of displacements from which frequency content can be obtained. The objective of this work is to investigate the performance of OMM as a vibration monitoring technique that has application for analysis of large-scale structures. Therefore, a comparative study between two commercially available OMM systems and one open-source algorithm is performed to determine the accuracy of those techniques compared to traditional contact-based measurements. Extensive laboratory tests performed on a structure oscillating at a known amplitude and frequency showed that OMM can measure displacements as low as three microns with 95% correlation in the time domain when compared to traditional contact-based sensing. An example of the potential of OMM to be a cost-effective, non-invasive, and quick condition monitoring technique for large structures (e.g., wind turbines and bridges) is discussed. By using OMM, the issues inherent in standard testing/monitoring procedures can be eliminated, allowing the time and cost of SHM to be significantly reduced.

MIT Lincoln Laboratory and NASA Marshall Space Flight Center have been collaborating on using video camera measurements and motion magnification for modal testing of large aerospace components for several years. This presentation will discuss results from the Space Launch System Integrated Modal Test (IMT) and Dynamic Rollout/Rollback Test (DRRT) in support of the Artemis I mission.

During the IMT, the data collection focused on operational mode shapes. In addition, the cameras were repositioned mid-test to better understand the physics of a low-frequency torsion mode. The non-contact nature of video data capture allowed for the quick redeployment of the cameras while not causing any delay in test schedule, whereas traditional instrumentation would have required a pause in testing to attach the sensors to the test article. The motion magnification analysis was able to find the low-frequency operational mode shapes and help the test team better understand the physics of the torsion mode.

Building upon the success of the IMT motion magnification work, a camera system was used during the DRRT to find operational mode shapes, if the physics of the low-frequency torsion mode remained with different boundary conditions, and relative deflection of the vehicle and ML tower during the roll. In this chapter, we will present operational mode shape results, discuss the physics of the torsion mode, and review experimental setup idiosyncrasies to help the community in designing video camera measurement systems.

Phase-based optical flow (PBOF) is a growing discipline in vibration extraction methods. Based on camera data, PBOF uses image gradients and image-pair differences to extract motion from video. Commonly, this technique is combined with the more ubiquitous digital image correlation, due in part to the restriction of PBOF to small, sub-pixel vibrations. In this chapter, work was done to delve deeper into the nominal upper bound and determine its relation to the chosen image-pair processing style. In particular, the upper bound was evaluated with respect to static (current frame-to-reference), dynamic (current frame-to-previous frame), and hybrid dynamic-static processing styles. Results include advantages of standard static and dynamic processing styles, leading to the presentation of the so-called region of goodness associated with the hybrid processing style. Discussion follows for how the hybrid processing style can be used for large-motion handling in simple structures, as well as modifications required for those that are heavily textured. The hybrid processing style is shown to extend the upper bound to the number of pixels between parallel edges within the structure, increasing the efficacy of PBOF to larger stationary vibrations.

The exploitation of experiment-based optical full-field technologies in a broad frequency band is here proposed for the sound radiation numerical simulation of a vibrating plate, instead of linear structural finite-element or analytical models—overly simplified on the boundary conditions, frictions, mistunings, and non-linearities—commonly used in limited investigations about a single effective eigenmode of the structure. Spatially detailed operative deflection shapes coming from real testing, indeed, can be a viable dataset for the best achievable representation in the spatial and frequency domains of the real behaviour of manufactured and mounted components around their working dynamic load levels. The Rayleigh’s integral formulation is here adopted for the numerical approximation of the sound radiation field from a lightweight rectangular plate retaining a complex and tightly populated structural dynamics, with effective constraints and damping characteristics. The Rayleigh’s approach is here reformulated to take advantage of the full-field receptances, to obtain maps of acoustic pressure frequency response functions, or of acoustic pressure, once the structural excitation signature is defined. Examples are given in the space and frequency domains, with special attention on the contribution of the experiment-based full-field receptance maps to the accuracy of the radiated acoustic fields.

Optical measurement techniques such as digital image correlation (DIC) and laser Doppler vibrometry (LDV) are advantageous to measure structural vibrations due to their non-contact nature. While these techniques are immune to electromagnetic interference, they can suffer from optical distortions due to index-of-refraction gradients associated with boundary layers, shock, shock and expansion waves, and combustion. When the optical distortion is due to unsteady processes, such as those associated with turbulence or other time-dependent flow phenomena, these effects cannot be easily accounted for and may result in erroneous measurements of the vibrations. An exploratory test campaign was performed to characterize these errors in a flow-structure interaction experiment conducted in a Mach 5 low-enthalpy blowdown wind tunnel. The flow-induced vibration of a compliant brass panel (0.25 mm thick) was measured using 3D-DIC and LDV, by imaging the panel through the flow. Additional optical distortions were generated by a 27.5\({ }^{\circ }\) compression ramp that was installed on the floor of the tunnel and generated a shock-induced, turbulent separated flow. Results showed that the LDV data were not affected within the uncertainty of the measurement by flow-induced optical distortions. LDV and DIC results generally agreed well for frequencies between 250 Hz and 2000 Hz, which contained all dominant vibration modes. The shock unsteadiness was expected to generate dynamic distortions for all locations that were measured through the shock. No evidence of this was found, which indicates that such distortions were below the noise floor of the 3D-DIC setup used for this experiment. In one test, the local discrepancy between DIC and LDV differed more when imaged through the shock than for a case where both systems measured the vibration upstream of the shock.

Three-dimensional digital image correlation (3D-DIC) has shown to be a powerful tool to extract full-field displacement and deformations of structures using a series of synchronized stereo images. Before performing 3D-DIC measurements, the stereovision system must be calibrated to determine the relative position of the two cameras. Traditionally, this has been done by taking pictures of a calibration target whose dimensions are well known. Because the calibration target must be as large as the inspected object to properly calibrate the entire measurement volume (i.e., large-area calibration), for large fields of view, the calibration procedure becomes a challenge. This research aims at reducing the complexity of large-area calibration by measuring the extrinsic parameters of the stereovision system using a suite of sensors. In particular, three inertial measurement units and a laser distance meter are used to measure the cameras’ relative position and orientation in space. In this chapter, the performance of the proposed sensor-based extrinsic calibration is compared with the traditional image-based calibration method. Laboratory tests show that the extrinsic parameters computed with the sensor-based method can be used for performing a 3D-DIC analysis that yields an error below 5% when the results are compared with the displacement retrieved using a traditional approach. The results of this study demonstrate that the proposed sensor-based calibration approach is a valid and a faster alternative to traditional image-based calibration methods. The sensor-based calibration has the potential to expand the use of 3D-DIC by making it more suitable for large-scale applications.

We evaluate the applicability of simple optical measurements of humans’ abdomens to monitor pulse propagation as a possible means to monitor the development of abdominal aortic aneurysms (AAAs) in patients. The setup consists of a single high-speed camera and a mirror that enables a second viewing angle captured by the same image sensor. This setup thus allows for 3D measurements of the motions on the patient’s body surface. Subsequent signal processing is a crucial step to extract meaningful data from measurements containing many disturbances such as sensor noise or motion induced by breathing. We demonstrate that the relatively simple test setup is capable of recording clean data of the abdomen’s vibrations. Also, we show that breathing can only be removed from the signals to a certain extent with a simple high-pass filter and make suggestions on how to evaluate the flow speed based on the signals’ delay downstream on the skin.

We propose a method to measure the vibrations amplitude of normal modes of a 3D mechanical structure with only one camera. Our method requires only a video sequence taken by a single camera and a collection of admissible 3D deflection shapes. The deflection shapes are projected in the image frame of the camera. This is done thanks to a linearization of the perspective projection model. Comparing the motion of targets seen by the camera and the motion of 2D deflection shapes at the same location gives the time evolution of modes amplitude. Finally, the motion of the model is reconstructed in 2D in the image frame and in 3D in the world frame. By following this procedure, vibration amplitude can be magnified applying a scale factor on a modal basis covering a large frequency bandwidth.

A lightweight internal damage segmentation network (IDSNet) (Ali and Cha, Autom Constr 141:104412, 2022) segments internal damages in concrete using active thermography. This study further investigates the performance of IDSNet for segmentation of subsurface damage. The IDSNet consists of an intensive module and a superficial module. The intensive module focuses on contextual features and learning complex correlations, and the superficial module learns spatial features in the input. The lightweight IDSNet has only 0.085 million parameters and processes a thermal image of 640 × 480 × 3 with 74 frames per second. The deep learning network requires extensive data for training; therefore, attention generative adversarial network known as AGAN (Ali and Cha, Autom Constr 141:104412, 2022), was used to generate artificial data for training IDSNet. The IDSNet was trained with and without AGAN data. The IDSNet achieved a 0.767 average intersection over union (aIoU) without using AGAN data and 0.891 aIoU with using AGAN data.

Despite the success of traditional structural health monitoring (SHM) techniques that rely on discrete sensors, several challenges still exist: (1) structures of interest need to be accessed for instrumentation, which is not always feasible due to complex site conditions; (2) the reliability of the results is highly dependent on the locations and the limited number of sensors mounted on the structure. To overcome these challenges, there is a critical need for non-contact monitoring techniques (e.g., laser scanners). While there has been substantial research conducted on the use of terrestrial laser scanners (TLS) (i.e., ground-based LiDAR) in monitoring the static deformation of civil structures, there have been only a few studies on the use of TLS in monitoring the dynamic vibrations of structures, which is critical information for SHM. Therefore, the main objective of this study is to develop a novel end-to-end framework to monitor the dynamic vibrations of structures using a TLS operating in the helical mode, where the TLS is operating at a fixed horizontal angle. To accomplish this goal, an extensive experimental study was conducted to investigate the effect of structure-based parameters on the robustness of TLS-based dynamic monitoring. The key parameter investigated in this chapter is the natural frequency of the specimen. A reconfigurable steel tower and weight plates were used to vary the dynamic structural parameters in this experiment. Accelerometers and infrared-based sensors were used in testing for the validation of TLS measurements. A novel spatio-temporal framework was developed to extract the dynamic vibrations of the structure of interest from the helical point clouds. The framework utilizes the density-based spatial clustering of applications with noise (DBSCAN) and change detection algorithms. The results show that the TLS can detect sub-millimeter structural vibrations. Furthermore, the dynamic response and characteristics extracted by the TLS framework closely match those of the accelerometers and infrared-based sensors. Hence, the results indicate the great potential of using TLS in monitoring the dynamic response of structures remotely.

Video-based structural dynamics techniques have shown great promise for applications such as monitoring the structural health of critical infrastructure such as locks and dams. Full-field approaches that make use of direct methods such as optical flow have the added attractive quality that they have demonstrated an ability to detect small damage on account of the high spatial density of pixels associated with imager measurements. For the case of inspecting critical infrastructure such as locks and dams, deformation measurements have also been shown to have utility. Imagers can be used to measure deformation; however, for the case of a complex 3D scene, every pixel can potentially have different sensitivity to deformation on account of the perspective transformation associated with pinhole camera photography. Telecentric lenses can be used to avoid perspective projection effects; however, telecentric lenses are large, expensive, and can only observe an area equal in size to their aperture greatly reducing their suitability for infrastructure inspection applications. In this work, we adapt techniques for orthorectification to attempt to address the issue of calibrating individual pixel deformations measurements across a scene. Orthorectification typically requires a height map information in the direction normal to the plane used to generate the orthophoto. The emergence of sensors such as time-of-flight imagers with high spatial resolution has made collecting these measurements more accessible for terrestrial infrastructure inspection applications. We demonstrate the ability to fuse subpixel motion measurements captured using a perspective camera, with 3D geometry data such as that which can be captured using a time-of-flight imager. This work focuses on the case of structures exhibiting planar geometry. We then show how these techniques impact the ability to perform video-based structural dynamics analysis.

Digital twins are virtual representations of real-world structures that can be used for modeling and simulation. Because of digital twins’ ability to simulate complex structural behaviors, they also have potential for structural health monitoring (SHM) applications. Video-based SHM techniques are advantageous due to the lower installation/maintenance costs, analysis in high-spatial resolution, and its non-contact monitoring features. Both digital twins and video-based techniques hold particular interest in the fields of non-destructive evaluation, damage identification, and modal analysis. An effective use of these techniques for SHM applications still poses several challenges. Neural radiance fields (NeRFs) are an emerging and promising type of neural network that can render photorealistic novel views of a complex scene using a sparse data set of 2D images. Originally, NeRF was designed to capture static scenes, but recent work has extended its capability to capture dynamic scenes which has implications for medium and long-term SHM. However, to date, most NeRFs use frame-based images and videos as input data. Frame-based video monitoring approaches result in redundant information derived from the fact that, for structural dynamics monitoring, only a small number of active pixels record the actual dynamical changes in the structure, resulting in intensive computational loads for data processing and storage. A promising alternative is event-based imaging, which only records pixel-wise changes on the illumination of a scene. Event-based imaging creates a sparse set of data, while accurately capturing the dynamics. The work proposes a method to extract the dynamics of a structure using a generated digital twin. Using Unreal Engine 5, digital twins of rigid and non-rigid structures were generated. The digital twin model was then used along with an event-based camera simulator to generate event-based data. A frequency analysis framework was then developed to extract the modal information on the structure. Validation was performed on a structure of known dynamics using event-based cameras.

This feasibility study evaluates the use of a laser Doppler vibrometer to measure strain. Specifically, it uses multipath laser Doppler vibrometry (MDV) to measure high-frequency strain required for high-cycle fatigue (HCF) and very high cycle fatigue (VHCF) testing on various engineered materials. As an indicator of higher durability, there is an emerging need to measure at higher number of cycles while keeping the measurement time to a minimum. One way to meet this diametrically opposed requirement is to perform the testing at higher frequencies. Conventional sensors have a limitation in terms of highest frequency and maximum temperature of operation which are addressed with the proposed method. In this research, we provide a detailed account of the motivation and the procedure to go from vibrational velocity to dynamic strain using MDV. Comparison is shown between strain measured with MDV, 3-dimensional scanning LDV and a strain gage. Results show a good agreement between these techniques.

The speed of light in air is dependent on the air’s instantaneous density. Since air density is modulated by sound, sound in the air can be observed and measured using optical methods. One such optical method is laser Doppler vibrometry (LDV). In this method, the time derivative of a laser beam’s optical path is measured. The optical path length is the product of the physical path length traversed by a laser beam and the refractive index of the medium. Most commonly, such instruments measure the time rate-of-change of the physical path (i.e., the mechanical velocity of a surface), and index changes of the medium are small in comparison. In contrast, by placing a rigid reflector beneath a sound beam in air, it is possible to measure the time rate-of-change of refractive index and to therefore measure dynamic changes in air density, or sound. Because modern LDVs enable rapid and convenient scanning, entire sound fields and sound beams can be observed. In prior demonstrations by other teams, this method has been used to visualize sound fields in the audible frequency range and ultrasound range underwater. In this work, we present the first measurements of high-intensity airborne ultrasound beams in the frequency range spanning 60–500 kHz. We present a sound field generated by a piezoelectric MEMS transducer at 228 kHz. We also observe interesting phenomena predicted from nonlinear acoustics including accumulated distortion, wave steepening, and weak shock formation as a sound beam propagates. LDV measurements are compared against more conventional measurement methods using pressure transducers.

This study develops a novel general-purpose 3D continuously scanning laser Doppler vibrometer (CSLDV) system to measure 3D full-field vibration of a turbine blade with a curved surface under random excitation and proposes an operational modal analysis (OMA) method to identify its modal parameters. The 3D CSLDV system developed in this study contains three CSLDVs, an external controller, and a profile scanner. A 3D zig-zag scan trajectory is designed on the blade surface based on profile scanning, and scan angles of mirrors in CSLDVs are adjusted based on relations among their laser beams to ensure that three laser spots can continuously and synchronously move along the same scan trajectory. The OMA method referred to as the extended demodulation method is used to process the measured response of the blade under random excitation to obtain its damped natural frequencies and 3D full-field undamped mode shapes. Comparison between the first six modal parameters from the proposed 3D CSLDV system and those from a commercial 3D scanning laser Doppler vibrometer (SLDV) system is made in this study. Errors between the first six damped natural frequencies from 3D CSLDV measurement and those from 3D SLDV measurement are less than 1.5%, and modal assurance criterion values between the first six undamped mode shapes from 3D CSLDV measurement and corresponding damped mode shapes from 3D SLDV measurement are larger than 95%. It took the 3D SLDV system about 900 seconds to scan 85 measurement points in the experiment, and the 3D CSLDV system 115.5 seconds to scan 132,000 points, which means that the 3D CSLDV system can measure much more points in much less time than the 3D SLDV system in 3D full-field vibration measurement.

A novel edge detection method is developed for an image-based tracking continuously scanning laser Doppler vibrometer (CSLDV) system to track a rotating structure without attaching any mark or encoder to it. The edge detection method can determine real-time positions of points on edges of the rotating structure by processing images captured by a camera in the tracking CSLDV system. Once a point on an edge of a rotating structure is determined, the position of the rotating structure is determined. The tracking CSLDV system can generate a scan path on the rotating structure and control its laser spot to sweep along the scan path. A newly developed improved demodulation method is used to process measured data of response of the rotating structure under random excitation and estimate its modal parameters including damped natural frequencies and undamped mode shapes. Damped natural frequencies of the rotating structure are estimated from fast Fourier transforms of measured data. Undamped mode shapes are estimated by multiplying measured data by sinusoidal signals whose frequencies are estimated damped natural frequencies and applying low pass filters to measured data multiplied by the sinusoidal signals. Experimental investigation of the edge detection method is conducted by using the tracking CSLDV system to track and scan a rotating fan blade. Modal parameters of the rotating fan blade under random excitation with different constant speeds and its instantaneous undamped mode shapes with a non-constant speed are estimated.

This study presents the results of the numerical study that aims to identify the missing fastener along the rail using train-induced guided waves. This study is part of an ongoing project that intends to develop a non-contact damage detection system based on the measurements obtained from a laser Doppler vibrometer placed on a moving platform. To achieve the goal, (i) modal analysis was conducted to determine the frequency range of interest in which the dynamic response of the rail is sensitive to the missing fastener (i.e., the frequency range in which the waves localize at the rail foot), (ii) a three-dimensional finite element method (FEM) simulation was conducted to imitate the movement of the wheel excitation and the measurements. The goal of the FEM model is to examine the effect of the missing fasteners on the dynamic response of the rail in the frequency range of interest. The guided waves recorded from a rail through an accelerometer during the passage of an operating train were used as the excitation signal in the FEM simulation. A damage function was introduced to examine the change in the wave energy caused by the missing fastener. Consequently, the location of the missing fastener was identified.

Feature-tracking is widely used in the vibration community for its noninvasive way of extracting subtle motion. High-frequency optical features, such as edges, are prime candidates for motion estimation; however, their rapid motion can pose problems for computer vision techniques. Most vibration seen in video is unperceivable to the naked eye, which can make sub-pixel displacement extraction more complex. Phase-based motion magnification (PMM) is a computer vision technique that can amplify motion seen in video. A boost in the signal-to-noise ratio of a particular frequency band can permit further evaluation of higher order dynamics. A need for larger magnification is necessary to visualize and quantify resonant frequencies. This amplification can produce ringing effects which degrade image quality and key features such as edges. In this work, the use of dynamic mode decomposition (DMD) permits an unsupervised approach of extracting structural dynamic parameters from video. Out-of-plane resonant frequencies are estimated from a 2.7 (m) wind turbine blade. The findings are compared to current state-of-the-art methodologies such as traditional wired sensing. The determination of resonant frequencies ultimately fosters further understanding of large structure dynamics that are present in optical data.

Dynamic spatiotemporal (ST) characteristics of a structure, such as natural frequency, frequency spectra, and operational deflection shapes, are used to assess and monitor a system’s condition. Recent developments in machine learning and computer vision approaches have created new paradigms to utilize ST characteristics and extract complex dynamics. Commonly used machine learning models, such as traditional autoencoders, lack the ability to learn complex phenomena in the latent space and extract important structural dynamic parameters. Hence, this study proposes a novel model based on a time-inferred autoencoder (TIA) to learn the ST characteristics of a structure of interest. To validate the proposed approach, experiments were conducted by collecting high-speed videos of multi-degree of freedom systems, which were used to train the TIA model and understand the underlying dynamics of the targeted system. Following the training stage, the TIA model was able to reconstruct the full line-of-sight optical data that can be used for identifying resonant frequencies. The robustness of the TIA approach and its capability to adapt to changes in the dynamics of the inspected structure was evaluated as a function of different levels of structural damage. If further developed, TIA can be used as a structural health monitoring tool to learn the dynamics of a system from videos without having to utilize contact-based sensors.

By using high-speed cameras, non-contact and high-resolution measurements of vibrations are possible for beginners and experts with little configuration effort. This makes surface measurements feasible, where each image pixel can be used as a vibration sensor. By using optical flow algorithms, measurements are possible without pretreatment of the target. This also applies to color homogeneous structures. This technique is very well suited for vibration analysis and online condition monitoring of buildings, such as bridges and wind turbines. Within the scope of this chapter, this technique is used for the first time to investigate the model of a wind turbine in terms of an order-based operating deflection shapes analysis. In addition to the deformation data, the speed signal will also be generated from the video to perform an order analysis, so that the operating deflection shape analysis can be visualized depending on the order or speed.