Finds documents with both search terms in any word order, permitting "n" words as a maximum distance between them. Best choose between 15 and 30 (e.g. NEAR(recruit, professionals, 20)).
Finds documents with the search term in word versions or composites. The asterisk * marks whether you wish them BEFORE, BEHIND, or BEFORE and BEHIND the search term (e.g. lightweight*, *lightweight, *lightweight*).
The article introduces MorphFlow, a groundbreaking method for estimating motion in in-situ tests of concrete. Traditional Digital Volume Correlation (DVC) methods struggle with concrete's brittle nature and large data volumes. MorphFlow overcomes these challenges by adopting Optical Flow algorithms and incorporating morphological wavelets for efficient processing. The method is validated on real-world datasets, showcasing its superior performance in identifying cracks and handling compressed data. This innovative approach promises to revolutionize the evaluation of concrete in critical infrastructure.
AI Generated
This summary of the content was generated with the help of AI.
Abstract
Background
In situ Computed Tomography is a valuable tool to investigate failure mechanics of materials in 3D. For brittle materials with sudden fracture like concrete however, state-of-the-art methods such as Digital Volume Correlation fail to produce displacement fields that display the discontinuous behavior of load induced cracking correctly.
Objective
The main objective is to develop an algorithm that calculates displacement fields for large-scale in situ experiments on concrete.
Methods
The algorithm presented is based on a 3D Optical Flow method solved by a primal-dual procedure and equipped with a coarse-to-fine scheme based on morphological wavelets. The algorithm is publicly available. Our evaluation focuses on the beneficial use of morphological wavelets over classical ones, and on the ability to produce reliable results with limited data. Applying the primal-dual scheme to in situ tests and using morphological wavelets are novel contributions.
Results
The results show that our algorithm cannot only cope with large volume images, but also produces discontinuous displacement fields that yield high strain in fractured regions. It does not only perform better than state-of-the-art methods, but also achieves sufficient results on reduced data. The morphological wavelets play a key role in this finding - they even allow to deduce cracks of widths less than a voxel.
Conclusion
Displacement calculation for in situ tests of brittle materials requires voxel-accurate displacement fields that allow for discontinuities. The presented algorithm fulfills these requirements and therefore is a powerful tool for future understanding of failure mechanics in concrete.
Claudia Redenbach and Katja Schladitz contributed equally to this work.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Introduction
Concrete is for sure among the most important materials in construction. Especially its use in critical infrastructure like bridges requires a thorough assessment of the material’s behavior. A very common way to do so is by in situ materials tests. Such tests couple any kind of materials test with an imaging modality. Using computed tomography (CT) as modality is of special interest as it allows to get a complete, three-dimensional digitized version of the whole concrete sample. This allows to assess the inside of the specimen and – when combined with a materials test – also its response to load. In situ materials tests follow a specific procedure: A sample is scanned without any load in a CT device. Then, ideally without removing the sample from the CT, a limited amount of load is applied. The load is kept constant and, after a suitable time period of relaxation, the sample is scanned again. This is repeated until the desired material failure happens. This way, a time series of volume images is generated.
When applied to concrete, such in situ tests have already proven to increase the understanding of fracture behavior. The authors of Tsitova et al. [31] for example used in situ tests to validate numerical predictions of cracking areas. However, concrete in situ tests are yet particularly difficult to evaluate. In our work, we focus on tackling two specific difficulties. First, due to its brittle nature, concrete exhibits discontinuous behavior. Its response to load is mainly fracture or cracking – a pattern that many state-of-the-art motion estimation algorithms in materials science fail to display. Second, there is a rising demand in efficient processing of large scale data. Tomographs like the tomography portal Gulliver1 produce volume images of a size of several hundreds of gigabytes or even some terrabytes. Efficient processing is crucial in these applications. Unfortunately, crack segmentation in concrete is a very costly task, even when Deep Learning methods are applied [1]. One therefore is in need for a preprocessing technique, that indicated areas of potential cracks and guides the costly methods to positions where cracks can potentially be found. Therefore, a special focus shall be put on working with compressed and therefore much smaller data.
Advertisement
State-of-the-art algorithms for computing the motion between each element of the time series in materials science go under the name Digital Volume Correlation (DVC), which was introduced first by Bay et al. in 1999 [2]. We recently showed that adopting algorithms initially dedicated to 2D pixel-accurate motion estimation can outperform modern and classical DVC methods. More precisely, we generalized a variant of Optical Flow (OF) to 3D to compute voxel-accurate displacements and proved it to be better capable to deal with sudden changes [21]. Due to this capability of capturing discontinuous behavior such as cracking or fracture, OF algorithms are also especially suitable to estimate displacement in in situ tests of concrete. OF methods naturally incorporate a multiscale approach, that means the problem of motion estimation is solved on a coarse scale and the solution is then used as initial point for a solution on a finer scale. We make use of this feature and exchange the classical Gaussian coarse-to-fine scheme by an image pyramid based on morphological wavelets [11]. Wavelets have been used successfully in powerful image compression algorithms. The use of a wavelet-based image pyramid therefore means that we apply the actual motion estimation algorithms to compressed data. This way, it becomes more feasible to tackle the large image sizes that we expect from large-scale tomographs.
However, the algorithm that we used in our previous work [21] contains a highly involved solution scheme based on successive overrelaxation. The incorporation of morphological wavelets is not straightforward, we therefore decided to choose a variant of OF that is easier to implement, namely the TV-\(L_1\) algorithm by Zach et al. [39].
Besides the utilization of compressed data, there is a further reason to exchange the common Gaussian pyramids in OF by morphological wavelets. Traditionally, Gaussian pyramids have been used as they satisfy the scale-space axioms. Moreover, they are also very useful when considering images of scences, see for example [17]. Gaussian smoothing and subsampling reduce small scale details. Combined with motion estimation, this follows the assumption that motion of large details in the image dominates also the motion of small details. In classical digital images, it is not too harmful to blur and compute motion based on large objects first. In an in situ test on concrete, however, the dominant feature that indicates the motion is the crack. And this feature will be blurred by Gaussian smoothing.
Morphological wavelets overcome this problem. The wavelet that we use is a rather uncommon representative of the so-called second generation wavelets. Wavelets in their proper sense are actually not really suitable for replacing Gaussian pyramids. Classical wavelets are supposed to preserve as much detail as possible from one scale to the next. In OF, however, it is required that part of the detail is lost in between scales to prevent algorithms from being trapped in local minima. We show that morphological wavelets exhibit this behavior much better than conventional wavelets of first and second generation.
Advertisement
Our algorithm is available open-source at https://github.com/TessaNogatz/morph-flow-matlab. We show the power of our algorithm by applying it to image data obtained in two in situ tests of concrete. The first one is a classical in situ test of refractory concrete. The second one is a more involved three point bending test on reinforced concrete. Reinforcement is provided by a glass fiber rebar. Sliceviews of the volume images of both samples are shown in Fig. 1.
The remainder of this manuscript is organized as follows. In Section “Related Work” we give a short review on in situ tests on concrete and wavelet based motion estimation. In Section “Optical Flow for Volume Image” we present the plain motion estimation algorithm, which is a 3D extension of a classical algorithm from digital image processing, without any comments on the multiresolution decomposition. This is presented in detail in Section “The Morphological Coarse-to-fine Scheme”, where morphological wavelets are defined and their use in the implementation of OF is outlined. Section “Datasets and Methodology” briefly introduces data sets and methodology used for evaluation. The results of the evaluation are presented in Section “Results and Discussion”. Section “Conclusion” contains concluding remarks.
Fig. 1
Sliceviews of used data sets. (a) and (b) show two slices of the in situ test on refractory concrete. Experiment and imaging: TUBAF Freiberg. The volume images have a size of \(580\times 580\times 638\) voxels at a voxel edge length of 50 \(\mu \)m. (c) shows the sliceview of the reinforced concrete before loading. Although d) already depicts a loading stage, the cracks are difficult to spot by human eyesight. The image has a size of \(550 \times 835 \times 1000\) voxels with a voxel edge length of \(125\mu \)m. CT scanning: Fraunhofer EZRT, Fürth. Experiment: Civil Engineering, RPTU Kaiserslautern-Landau, Kaiserslautern
In situ investigations of concrete require a delicate planning of the testing procedure. Concrete is a brittle material meaning that initiation of damage such as cracks happens abruptly after a short elastic phase. Due to this behaviour, it is very difficult to fix the loading stages beforehand. Therefore, loading is often increased stepwise, and an external pressure management is applied to decide whether the concrete failed or not. Unfortunately, microcracking does not influence the mechanical properties notably. Therefore, when failure is detected by external monitoring, the concrete usually already exhibits severe, large cracks.
Nevertheless, various in situ investigations of concrete can be found. They are performed either in laboratory CT devices or at synchrotron beamlines. Many of them apply some kind of motion estimation, mainly DVC, to assess the loading induced motion and eventually the crack. DVC does not compute displacement per voxel but per subvolume. Local approaches fix independent subvolumes and calculate an average displacement for each subvolume independently from the surrounding ones, global methods use finite element grids as underlying subvolumes and calculate nodal displacements.
Analysis by local DVC
Contributions that specifically use local DVC methods for concrete data are for example [6, 18, 19, 30, 37]. The main problem that arises when using local DVC based methods is that they often fail to display the crack in the displacement field. This is especially visible in the manuscripts [18] and [19], where the results rather show smooth fields instead of jumps, as one would expect if a crack forms. Even further, Chateau et al. [6] use the poor performance of DVC at cracks to segment displacement based on errors in residuals.
Local DVC was also applied to concrete for estimating motion that was not induced by load but by environmental influences. Chavez et al. [7] investigated cement carbonation while Jiang et al. [14] studied corrosion induced cracking.
Analysis by global DVC
Global methods are usually not available open-source. Therefore, one of the few works combining in situ tests on concrete and global DVC is quite a recent one by Tsitova et al. [31]. It shows many of the struggles that come with using global DVC for brittle materials such as concrete. As global DVC produces rather smooth displacement fields, it tends to smear out the strain extrema that one would expect at cracks. The authors of Tsitova et al. [31] had to apply quite heavy post-processing to deduce information in cracked areas nevertheless. They extracted the cracks by erroneous residuals and incorporated this information in another iteration of motion estimation.
Analysis without motion estimation
There are also several contributions that do not fully exploit the underlying data, as they only visualize the data and do not apply any motion estimation algorithm. Among them are [13, 15, 38].
Wavelets in Multiresolution Schemes
General images
In motion estimation and specifically OF, wavelets have so far mainly been used to decompose the displacement field rather than the image itself. Instead of calculating a displacement vector for every voxel, one formulates the field in terms of a wavelet basis. Conceptwise, the approach is comparable to using an FE grid as in DVC or even classical FE simulations. From a mathematical point of view, using wavelet bases allows for much more freedom in terms of the underlying nature of the displacement fields. That is why these approaches perform reasonably well compared to classical OF approaches [9, 35].
When it comes to the multiresolution decomposition, the use of wavelets is quite rare, as wavelets are not supposed to smooth from one scale to the next. Nevertheless, one can find some very advanced applications mainly in the area of ultrasound images. Saputro et al. [25] propose to use a wavelet based multiresolution approach to support OF computation for myocardial motion. Gastounioti et al. [10] aim for arterial wall displacements. Both applications deal with 2D images. To the best of our knowledge, no 3D applications have been reported so far.
CT images
Wavelets have sucessfully been applied to concrete in several applications. Similar to classical image processing, compression [27] and denoising [3] approaches by wavelets can be found. Wavelets have also been used for segmenting [22] and classifying CT data [12]. Further, one can find them also in artifact reduction [20].
Optical Flow for Volume Image
As mentioned earlier, we use the TV\(L_1\)-variant of Zachet al. [39] to compute the displacement fields. It is based on the primal-dual approach by Chambolle [5] for solving total variation regularized problems.
The general assumption of Optical Flow (OF) is that grey values in images are preserved under displacement. That means that for volume images \(I_0, I_1 :\Omega \subset \mathbb {R}^3\rightarrow \mathbb {R}\) we have
with a suitable displacement \(\textbf{u}:\Omega \subset \mathbb {R}^3\rightarrow \mathbb {R}^3\). This equation is ill-posed, therefore regularization by total variation is proposed. By penalizing first order gradients, one can remove spurious oscillations while preserving sharp edges [23]. One solves the following minimization problem
Due to the use of \(L_1\)-norms, the problem is neither linear nor convex nor differentiable. Several steps are required to transform the problem into a feasible formulation. The first step is to perform a first order Taylor approximation to remove the nonlinearity introduced by \(\textbf{u}\) being evaluated inside the image. That is,
To deal with the non-differentiability of the problem, Zach et al. propose to split the above equation into two parts: One dealing with motion estimation based on the image data, and another one with reconstructing a solution that satisfies the regularizer best. Thus, we introduce an auxiliary variable \(\textbf{v}\) and solve
for some given initial \(\textbf{v}_0\). Equation (5) is then solved by alternating between the solution concerning the \(\textbf{u}\) and the \(\textbf{v}\) parts
We refer to the original work of Chambolle [5] and its implementation in 2D [39] for a detailed derivation of this scheme.
It remains to solve equation (8). As the problem is is differentiable and convex, the problem can be solved by a case distinction based on first-order optimality, i.e. considering the three cases \(\rho (\textbf{v})<0\), \(\rho (\textbf{v})>0\) and \(\rho (\textbf{v})=0\), we can derive that
Note that the use of first order Taylor approximations to formulate equation (4) is one of the central reasons why a coarse-to-fine scheme needs to be employed in almost all OF algorithms. Taylor expansions are good approximations only in close vicinity of the point of interest. That means, \(\textbf{u}_0\) needs to be close to \(\textbf{u}\), and close in image processing means closer than one voxel. For motion estimation on the original scale, this sounds not very reasonable. However, if the image is downscaled rigorously, the requirement of one voxel displacement magnitude can be easily met. If the coarse displacement is then used as starting point on the next finer scale, the assumption of small displacements is met again. However, coarse-to-fine schemes often build on Gaussian filtering and downsampling. This usually smooths out cracks, and therefore erases one of the main objective when applying motion estimation to many mechanical in situ tests of concrete.
The Morphological Coarse-to-fine Scheme
As the name indicates, morhological wavelets use morphological image operations to subsample an image. They are based on the lifting scheme by Wim Sweldens [28], who proposed a pure space-domain and inplace calculation of wavelet transforms.
Lifted Wavelets
The lifting scheme applies three consecutive steps to generate an approximation of a signal on a next coarser level. Let f be a discrete, one-dimensional signal at points k, \(k\in \mathbb {Z}\), and let \(\lambda _{j,k}\) be the approximation of the signal in the j-th coarser space at point k. Initially, we set \(\lambda _{0,k} = f(k)\). The steps now read
1.
The signal is split into 2 subbands, called \(\text {even}_{1,k}\) and \(\text {odd}_{1, k}\), by an invertible transformation \(\Sigma \). We write
Calculate the wavelet coefficient by the difference of \(\text {odd}_{1,k}\) and a prediction of \(\text {odd}_{1,k}\) based on \(\text {even}_{1,k}\). We have
The choice of \(\Sigma , P\) and U is not unique and can be adjusted to particular use cases. One can also cast classical wavelets such as the Haar-wavelet into the lifting scheme. In this case, \(\text {even}_{1,k}\) and \(\text {odd}_{1, k}\) resemble even and odd sampling positions, i.e.
3D Extension of separating a signal into even and odd resulting in a classical checkerboard pattern. Even and odd is now resembled by black and white. Neighboring relations among each label are now given by quincunxes, depicted in yellow in Figure b
The prediction operator of \(\text {odd}_{1,k}\) based on \(\text {even}_{1,k}\) is now the average of the two even neighbors of an odd position. That is,
In the last section we considered 1D signals and split them into even and odd based on their discrete index. The extension to 2D is straightforward by using a checkerboard pattern and splitting into black and white pixels or voxels in 3D, see Fig. 2(a).
In a wavelet context, this is also called the quincunx scheme. From now on, we denote by image grid or Cartesian grid a grid whose nodes are the pixel or voxel center points.
The morphological wavelet transform based on the lifting scheme was introduced by Heijmans et al. [11]. They denote by S the set of all original points, by Q the set of black points, and by \(R=S\backslash Q\) the set of white points. Further, they define an adjacency relation to identify neighbors \(s'\) of s, i.e. \(s\sim s'\) if \(||s-s'||_1 = 1\). \(\sim \) is a symmetric relation on \(S\times S\). Note that only either s or \(s'\) can be a member of Q if \(s\sim s'\), but not both at the same time. In a first step, one therefore has \(\lambda _{1,r} = \lambda _{0,r}\) and \(\gamma _{1, q} = \lambda _{0,q}\) for \(r\in R\) and \(q\in Q\). We now define specific prediction and update operators by
In practice, when one considers classical cartesian pixel or voxel grids, these neighboring relations are just minima and maxima among the 4- and 6-connected neighbors, respectively.
This scheme is known as Max-Lifting, but the maximum can also be exchanged by a minimum. The use of Max- or Min-Lifting can be directly translated into the morphological operators dilation or erosion, respectively, which explains the name morphological wavelets.
A major advantage of lifting schemes as above is the fact that calculations can be performed inplace due to separation into even/odd or black/white. Consider first the 2D case, which is just the front slice of (Fig. 2(b)). After the first lifting, the image only consists of the yellow quincunx points, so a 45 degree tilted version (or with diagonal instead of horizontal and vertical neighboring relations). If we now want to perform a prediction and an update step on the tilted version, we can use the exact same neighboring relations (horizontal and vertical neighbors) to apply the wavelet transform. This is even mathematically justified: The dilation matrix that transforms a quincunx sublattice into a Cartesian lattice in two dimensions is a similarity matrix. Unfortunately, in higher dimensions we fail to find such a transformation. A scheme which
allows for a 2-channel design (split into even and odd, black and white),
generates a Cartesian lattice again after d iterations,
and is a similarity transform,
does not exist in any dimension larger than \(d=2\) [32]. However, the third point is quite a crucial one: It guarantees that we can “reuse” neighboring relations we stated once. Lacking this similarity transform means that we need to borrow steps from another wavelet transform.
Why does the scheme that worked so well in 2D fail in 3D? A rough intuition can be given by the Voronoi cells of the quincunx points.
Voronoi cells can be used to state an equivalent notion of neighborhood between points: Two points are neighbors if their Voronoi cells share exactly one face (one edge in 2D). Let us now consider the stepwise outcome of a wavelet transform on a quincunx grid. The first step, so splitting into black and white balls works fine in 2D and in 3D. We remain with two distinct sets from which we can predict and update. Performing prediction and update then yields an approximation on the positions of the black points. In 2D, we now just rotate the black points by 45 degree, and once more we can use the previous neighboring relation, split the remaining points into two distinct sets, perform prediction and update steps on those and arrive eventually on a Cartesian grid with \(\frac{1}{2}\) of image edge length in each coordinate direction, so an image whose total number of pixels is \(\frac{1}{4}\) of the original.
In 3D, the first step works similarly. But now considering the remaining points, we immediately unveil the issue in 3D. The Voronoi cells of the 3D quincunx grid have the shape of a rhombic dodecahedron, a twelve-faced polygon with rhombic faces, which means that every point has twelve neighbors. A similarity transform however requires the same amount of neighbors as in the Cartesian lattice, which are six. Even worse, the amount of twelve neighbors prevents us from dividing the remaining points into two distinct subsets, from which we can predict and update. We therefore proceed as proposed by Sandic-Stankovic et al. [24] and borrow the idea of 5/3-LeGall filters [16], which will not be introduced in detail here.
The resulting Voronoi cell is now a cuboid, with two equal sides and the remaining one twice as long. The cell is not aligned with Cartesian coordinates. Clearly we loose isotropy at this stage. It may therefore be worth checking if the cuboidal cell should be aligned problem dependent to achieve better performance in whatever the wavelets are used for. This study will however not be part of this paper.
Efficient Implementation of Morphological Wavelets
As indicated in the previous section, applying a full cycle of the morphological wavelet transform requires three substeps, each on a different grid. Consequently, to implement the morphological wavelet transform, we need three different update and predict operators. Starting with a 3D image I at integer valued coordinates (i, j, k), and using the checkerboard pattern to split the voxels into two sets, we denote the predict and update operators in the cubic voxel setting \(P^{\text {cub}}\) and \(U^{\text {cub}}\), in the dodecahedral voxel setting \(P^{\text {dod}}\) and \(U^{\text {dod}}\), and in the cuboidal voxel setting \(P^{\text {cbd}}\) and \(U^{\text {cbd}}\), respectively. The wavelet scheme then reads
$$\begin{aligned} {\begin{matrix} (P^{\text {cub}}\lambda ^{\text {cub}}_0)(i,j,k) = \max \big (& \lambda ^{\text {cub}}_0(i-1,j, k),\lambda ^{\text {cub}}_0(i+1,j, k), \\ & \lambda ^{\text {cub}}_0(i,j-1, k),\\ & \lambda ^{\text {cub}}_0(i,j+1, k),\lambda ^{\text {cub}}_0(i,j, k-1),\\ & \lambda ^{\text {cub}}_0(i,j, k+1)\big ) \\ & \text {if } i \text { mod }2 = 1 \text { and } j \text { mod }2 = k \text { mod }2 \\ & \text {or } i \text { mod }2 = 0 \text { and } j \text { mod }2 \ne k \text { mod }2, \end{matrix}} \end{aligned}$$
(19)
$$\begin{aligned} {\begin{matrix} \gamma ^{\text {cub}}_1(i,j,k) = & \lambda ^{\text {cub}}_0(i,j,k) - (P^{\text {cub}}\lambda ^{\text {cub}}_0)(i,j,k)\\ & \text {if } i \text { mod }2 = 1 \text { and } j \text { mod }2 = k \text { mod }2 \\ & \text {or } i \text { mod }2 = 0 \text { and } j \text { mod }2 \ne k \text { mod }2, \end{matrix}} \end{aligned}$$
(20)
$$\begin{aligned} {\begin{matrix} (U^{\text {cub}}\gamma ^{\text {cub}}_1)(i,j,k) = \max \big (0, & \gamma ^{\text {cub}}_1(i-1,j, k),\gamma ^{\text {cub}}_1(i+1,j, k), \\ & \gamma ^{\text {cub}}_1(i,j-1, k),\\ & \gamma ^{\text {cub}}_1(i,j+1, k),\gamma ^{\text {cub}}_1(i,j, k-1), \\ & \gamma ^{\text {cub}}_1(i,j, k+1)\big ) \\ & \text {if } i \text { mod }2 = 0 \text { and } j \text { mod }2 = k \text { mod }2 \\ & \text {or } i \text { mod }2 = 1 \text { and } j \text { mod }2 \ne k \text { mod }2, \end{matrix}} \end{aligned}$$
(21)
$$\begin{aligned} {\begin{matrix} \lambda ^{\text {dod}}_1(i,j,k) = & \lambda ^{\text {cub}}_0(i,j,k) + (U^{\text {cub}}\gamma ^{\text {cub}}_1)(i,j,k)\\ & \text {if } i \text { mod }2 = 0 \text { and } j \text { mod }2 = k \text { mod }2 \\ & \text {or } i \text { mod }2 = 1 \text { and } j \text { mod }2 \ne k \text { mod }2, \end{matrix}} \end{aligned}$$
(22)
$$\begin{aligned} {\begin{matrix} (P^{\text {dod}}\lambda ^{\text {dod}}_1)(i,j,k) = \max \big (& \lambda ^{\text {dod}}_1(i-1,j-1, k),\lambda ^{\text {dod}}_1(i+1,j-1, k), \\ & \lambda ^{\text {dod}}_1(i+1,j+1, k),\lambda ^{\text {dod}}_1(i-1,j+1, k),\\ & \lambda ^{\text {dod}}_1(i,j-1, k-1),\lambda ^{\text {dod}}_1(i,j+1, k-1)\big ), \\ & \lambda ^{\text {dod}}_1(i,j+1, k-1),\lambda ^{\text {dod}}_1(i,j+1, k+1)\big ) \\ & \text {if } i \text { mod }2 \!=\! 1 \text { and } j \text { mod }2 \!=\! 1,\, k \text { mod }2 \!=\! 0, \\ & \text {or } i \text { mod }2 \!=\! 0 \text { and } j \text { mod }2 \!=\! 1,\, k \text { mod }2 \!=\! 1, \end{matrix}} \end{aligned}$$
(23)
$$\begin{aligned} {\begin{matrix} \gamma ^{\text {dod}}_1(i,j,k) = & \lambda ^{\text {dod}}_1(i,j,k) - (P^{\text {dod}}\lambda ^{\text {dod}}_1)(i,j,k)\\ & \text {if } i \text { mod }2 = 1 \text { and } j \text { mod }2 = 1,\, k \text { mod }2 = 0, \\ & \text {or } i \text { mod }2 = 0 \text { and } j \text { mod }2 = 1,\, k \text { mod }2 \!=\! 1, \end{matrix}} \end{aligned}$$
(24)
$$\begin{aligned} {\begin{matrix} (U^{\text {dod}}\gamma ^{\text {dod}}_1)(i,j,k) = \max \big (& 0,\gamma ^{\text {dod}}_1(i-1,j-1, k),\gamma ^{\text {dod}}_1(i+1,j+1, k), \\ & \gamma ^{\text {dod}}_1(i+1,j-1, k),\gamma ^{\text {dod}}_1(i-1,j+1, k),\\ & \gamma ^{\text {dod}}_1(i,j-1, k-1),\gamma ^{\text {dod}}_1(i-1,j-1, k+1)\big ), \\ & \gamma ^{\text {dod}}_1(i,j+1, k-1),\gamma ^{\text {dod}}_1(i,j+1, k+1)\big ) \\ & \text {if } i \text { mod }2 = 0 \text { and } j \text { mod }2 = 0 \text { and } k \text { mod }2 = 0, \\ & \text {or } i \text { mod }2 = 1 \text { and } j \text { mod }2 = 0 \text { and } k \text { mod }2 \!=\! 1, \end{matrix}} \end{aligned}$$
(25)
$$\begin{aligned} {\begin{matrix} \lambda ^{\text {cbd}}_1(i,j,k) = & \lambda ^{\text {dod}}_1(i,j,k) + (U^{\text {dod}}\gamma ^{\text {dod}}_1)(i,j,k)\\ & \text {if } i \text { mod }2 = 0 \text { and } j \text { mod }2 = 0 \text { and } k \text { mod }2 = 0, \\ & \text {or } i \text { mod }2 = 1 \text { and } j \text { mod }2 = 0 \text { and } k \text { mod }2 = 1, \end{matrix}} \end{aligned}$$
(26)
$$\begin{aligned} {\begin{matrix} (P^{\text {cbd}}\lambda ^{\text {cbd}}_1)(i,j,k) = \max \big (& \lambda ^{\text {cbd}}_1(i-1,j, k-1),\lambda ^{\text {cbd}}_1(i+1,j, k-1), \\ & \lambda ^{\text {cbd}}_1(i-1,j, k+1),\lambda ^{\text {cbd}}_1(i+1,j, k+1),\\ & \text {if } i \text { mod }2 = 1 \text { and } j \text { mod }2 = 0 \text { and } k \text { mod }2 = 1, \end{matrix}} \end{aligned}$$
(27)
$$\begin{aligned} {\begin{matrix} \gamma ^{\text {cbd}}_1(i,j,k) = & \lambda ^{\text {cbd}}_1(i,j,k) - (P^{\text {cbd}}\lambda ^{\text {cbd}}_1)(i,j,k)\\ & \text {if } i \text { mod }2 = 1 \text { and } j \text { mod }2 = 0 \text { and } k \text { mod }2 = 1, \end{matrix}} \end{aligned}$$
(28)
$$\begin{aligned} {\begin{matrix} (U^{\text {cbd}}\gamma ^{\text {cbd}}_1)(i,j,k) = \max \big (& 0,\gamma ^{\text {cbd}}_1(i-1,j, k-1),\gamma ^{\text {cbd}}_1(i+1,j, k-1), \\ & \gamma ^{\text {cbd}}_1(i-1,j, k+1),\gamma ^{\text {cbd}}_1(i+1,j, k+1),\\ & \text {if } i \text { mod }2 = 0 \text { and } j \text { mod }2 = 0 \text { and } k \text { mod }2 = 0, \end{matrix}} \end{aligned}$$
(29)
$$\begin{aligned} {\begin{matrix} \lambda ^{\text {cub}}_1(i,j,k) = & \lambda ^{\text {cbd}}_1(i,j,k) + (U^{\text {cbd}}\gamma ^{\text {cbd}}_1)(i,j,k)\\ & \text {if } i \text { mod }2 = 0 \text { and } j \text { mod }2 = 0 \text { and } k \text { mod }2 = 0. \end{matrix}} \end{aligned}$$
(30)
For ease of reading, we equipped \(\lambda \) and \(\gamma \) with upper indices indicating in which voxel settings we are. This is in principle not necessary, as the modulo restrictions already indicate which voxels are used. Also note that the scheme presented above allows to calculate the wavelet transform completely in place.
However, assembling the positions iteratively and checking if the positions suffice the modulo restrictions everytime, are costly operations. To perform the transform more efficiently, we follow the efficient implementation approach of de Zeeuw et al. [8] and extend it to 3D. A Matlab-implementation is published open-source2.
Fig. 3
Voxel labels for efficient implementation. To each voxel, a label is assigned. A voxel is depticted in dashed lines top left, solid lines represent neighboring relations between voxels. The presented pattern is repeated periodically in each coordinate direction over the whole image
Neighboring relations for morphological wavelets. Exemplary depicted for
in hv-lifting (Figure a),
in d1-lifting (Figure b) and
in d2-lifting (Figure c). Actual neighboring relations are depicted in color, initial neighboring relations in dashed lines are kept for better orientation
To do so, we split the voxels into eight subsets. These are labelled by the symbols
,
,
,
,
,
,
,
in Fig. 3. These subsets can be directly identified with their modulo restrictions, i.e.
(31)
Instead of checking for the modulo restrictions, maxima (or minima) can now efficiently be taken over coinciding positions in neighboring subsets, i.e. the prediction operator in the cubic case equation (19) becomes
(32)
and accordingly for
,
,
\(\big )\) to define the full operator \(P^{\text {cub}}\). This allows for very efficient implementations, as not only possibly large datasets are divided into smaller ones, but also because there is no single index check anymore. Because the neighboring relations in this first step are only horizontally and vertically (and in plane, which we discard in naming conventions), see (Fig. 4(a)), we call this step Horizontal/Vertical Lifting or hv-Lifting.
Similarly, prediction steps are perfomed for \(\big (\)
,
,
\(\big )\). The result is then used for an update step, again with straightforward neighboring relations, for \(\big (\)
,
,
,
\(\big )\)
We will call the transition from the dodecahedral grid to the cuboidal grid Diagonal Lifting 1 or d1-Lifting. The points are now partitioned into \(\big (\)
,
\(\big )\) and \(\big (\)
,
\(\big )\). Again, we only depict the neighboring relation for prediction of
. It can be found in (Fig. 4(b)).
Application of the wavelet transform to the cuboidal grid is denoted by Diagonal Lifting 2 or d2-Lifting. As only two sets are left, the split is clear, the neighboring relations are however depicted in (Fig. 4(c)).
Derivatives on the Lifted Grids
In the previous subsection we learned that only in every third step we end up with a Cartesian grid in the wavelet transform. The other two steps produce grids that consist of cells in shapes of dodecahedra and tilted cuboids, respectively. However, if we want to perform motion estimation via OF on all scales, we need to estimate derivatives also on these special grids. We use the divergence theorem to do so – a classical approach from finite element and volume methods. In what follows, we strongly follow the notation of Syrakos et al. [29] for unstructured grids, which clearly have our structured ones as special case. A cell of a grid will be denoted by P, with \(\Omega _P\) its volume and \(S_P\) its bounding surface. Then, for continuously differentiable \(\Phi \) we have
$$\begin{aligned} \int _{\Omega _p} \nabla \Phi \,\text {d}\textbf{x}= \int _{S_P}\Phi n\,\mathrm d s. \end{aligned}$$
Our bounding surface \(S_P\) consists of F planar faces \(S_f\), and each face has a constant unit normal, therefore
Now apply the midpoint rule: It states that the mean value of a quantity over a cell P (or face f) is equal to its value at the centroid \(\textbf{P}\) of the cell (or \(c_f\) of the face) plus a second order correction. We get
where h is the characteristic grid spacing. Together with the assumptions that multiplying by \(\Omega _P\) causes an error of order \(O(h^3)\) and by \(S_f\) of order \(O(h^2)\), this yields
In the context of finite volume methods, one would now need to bother about evaluation of \(\Phi \) at cell and face centers. But as we calculate derivatives on a voxel grid, \(\Phi \) is just constant over the cell and the face.
To actually calculate the derivatives, we recycle the neighborhood relations from the last section as they immediately yield the values of \(\Phi \) of the neighboring faces. What remains to be calculated are the face normals. However, once more observing Fig. 3, the normal vectors are just directional vectors between the center points multiplied by a normalization factor. Exemplarily considering the gradient calculation in the cuboidal setting, so neighborhood relations as in Fig. 4(c), we see that neighbors to
are horizontally given by
itself, and diagonally by
. Therefore, the gradient is calculated via
(35)
Having now derived a method to compute gradients on arbitrary grids, we can apply the TV OF variant of Section “Optical Flow for Volume Image” to the quincunx grids. We just replace all differences by their corresponding components in the gradient, computed at every grid position as in equation (34).
MorphFlow
We will end this section with an exact description of the individual steps of our algorithm. Due to the three-fold downsampling with morphological wavelets, the number of levels in the coarse-to-fine scheme has to be a multiple of 3. Our convention is that start and endpoints of the morphological coarse-to-fine scheme are always classical cubic voxel settings. Each complete iteration step therefore starts with solving total variation regularized OF on the lowest scale. Next, the detail signals are retrieved, such that image information is available on a cuboidal voxel setting. To upscale the displacement fields from the coarser scale suitably, we act as if our fields stemmed from lossy compression with wavelets. That usually means that small detail coefficients in the wavelet approximation are set to 0. The displacement fields from the previous scale are therefore retrieved with 0 detail signal. Next, the procedure is repeated, but this time with image information on a dodecahedral voxel setting. If the final stage in the coarse-to-fine scheme is reached, an additional computation on the cubic voxel setting is performed. A pseudo-code description can be found in Algorithm 1, and an implemention in Matlab is published open-source3. To emphasize that one outer iteration requires three different types of gradient calculations due to the different underlying grids, these code blocks are listed separately.
To demonstrate the performance of our algorithm, we use two experiments on concrete. The first one is a classical in situ test on refractory concrete. Due to the requirement of high refractoriness, the solid component consists of \(70\%\) tabular alumina T60 aggregates, \(20\%\) reactive alumina CTC50 and CT3000SG as fines, and only \(5\%\) cement (alumina cement Secar 71). 6 mass percent of water are added. The sample has a diameter of 24.5mm and a height of 25.2mm. It has been extracted by core drilling from a larger sample of 50mm diameter and 50mm height. The voxel edge length of the CT images is \(44\,\mu \)m.
Second, we use a three point bending test on concrete reinforced by a glass fiber rebar. The samples were produced at the chair of Concrete Structures and Structural Design at the department of Civil Engineering at RPTU Kaiserslautern-Landau. The concrete mixture used consists of CEM II 32.5 cement mixed with water in a water-cement ratio of 0.7. Aggregates between 0mm and 8mm diameter were chosen. The in situ procedure has been performed at Fraunhofer EZRT in Fürth. The overall specimen size is \(80\times 80\times 600\)mm, however a field-of-view scan was performed, such that the final image only covers \(80\times 80\times 400\)mm. The final voxel edge length of the volume images is \(128\,\mu \)m. Note that in this work we show only excerpts of the sample to validate our method. A thorough mechanical evaluation based on our method will be postponed to future work.
The loading stages of this experiment were carefully chosen, which brings us in a rather unique position for in situ tests on brittle materials: We in fact can observe microcracking and early stage failure very close to the resolution limits. That means that we supposedly see cracks that have a width not more than one voxel or even lower. It is therefore an ideal test dataset to demonstrate the capacity of our algorithm to identify early stage failure.
When applying the motion estimation algorithms, we focus on two aspects: We show that for full scale computations our algorithm performs comparable to or even better than state-of-the-art methods. Furthermore, due to the use of wavelets, we can also use low scales to obtain a comparably cheap first analysis to – for example – decide with low computational effort, if a concrete sample in an in situ test is already fractured.
Therefore, our algorithm as presented in Section “MorphFlow” will be applied first to compute displacement fields at full resolution. To show that our approach indeed performs better than state-of-the-art methods from materials science, we also apply Augmented Lagrangian Digital Volume Correlation as comparison.
Then we focus on the performance of morphological wavelets at low resolution scales. We show that neither 3DOF with Gaussians nor 3DOF with Haar- or Daubechies-wavelets yield comparable results at similar resolution scales. Yet this performance is quite difficult to quantify. Our “measures” of choice will be residuals, root-mean-square error (RMSE) and multilevel structural similarity (SSIM). Recall that motion estimation algorithms are supposed to reconstruct displacement vector fields \(\textbf{u}\) such that \(I_0(\textbf{x}) = I_1(\textbf{x}+\textbf{u})\). It is therefore very obvious to assess the performance of an algorithm based on this exact relation. Hence, one can consider the absolute value of the initial residual
$$\begin{aligned} r = |I_0(\textbf{x})-I_1(\textbf{x}+\textbf{u})|, \end{aligned}$$
where \(I_1(\textbf{x}+\textbf{u})\) was computed by trilinear interpolation at deformed coordinates. Note that we will often slightly carelessly call these quantities only initial residual and residual.
Though the RMSE is widely used in image processing, it is well-known that it lacks robustness towards outliers, see for example the discussion of Chai et al. [4]. Yet, the main reason for Wang et al. to propose SSIM [33] and later multi level SSIM [34] was that “classical” error measures like RMSE do not coincide well with human perception of similarity.
Table 1
Parameters for MorphFlow used in the application examples
Parameter
\(L_{\text {start}}\)
\(\tau \)
\(\lambda \)
\(\theta \)
\(i_{\text {warps}}\)
\(i_{\text {max}}\)
Refractory
12
0.125
25
0.2
20
30
Reinforced
12
0.125
25
0.2
15
30
Fig. 5
Sliceviews of the residuals of refractory concrete before and after warping at full resolution for MorphFlow and ALDVC
To define SSIM, consider two discrete, non-negative signals \(\textbf{p} = \{p_i | i = 1, 2, \ldots , N\}\) and \(\textbf{q} = \{q_i | i = 1, 2, \ldots , N\}\) of length N (for example image patches with consecutively numbered voxels in \(I_0\) and \(I_1\), respectively). The ingredients for SSIM are means \(\mu _{\textbf{p}}, \mu _{\textbf{q}}\), variances \(\sigma _{\textbf{p}}^2\) and \(\sigma _{\textbf{q}}^2\), and covariance \(\sigma _{\textbf{p}\textbf{q}}\). As mentioned, SSIM aims for mimicking the human perception, and it turns out that \(\mu _{\textbf{p}}\) is an estimate of luminance, \(\sigma _\textbf{p}^2\) an estimate of contrast, and \(\sigma _{\textbf{p}\textbf{q}}\) gives a tendency on how \(\textbf{p}\) and \(\textbf{q}\) vary together, so how structurally similar they are. Thus, the multiplicative components for SSIM are
\(C_1, C_2\) and \(C_3\) are small constants to prevent numerical instabilities, and the exponents can be seen as weights, but are often just chosen such that \(\alpha =\beta =\gamma = 1\),which are also our values of choice. Clearly, the SSIM ranges between 0 and 1, and equals 1, if both patches are the same. The SSIM index for a whole image is now the mean over all patches. It becomes multiscale, if the SSIM is computed on different scales, i.e. on M downscaled and filtered versions of the signals \(\textbf{p}\) and \( \textbf{q}\)
To visualize the consequences of downscaling when considering cracks we use so called scanline plots which show grey value profiles along a horizontal or vertical line of one slice of the image. When applied at positions with local minima, it unveils very visibly how morphological wavelets preserve local minima best.
For both samples, we use strain as indicator for cracks. In our displacement fields, we expect discontinuities along the cracks. This means that strains in this area will be large. We use the following (mechanically slightly inaccurate) discrete approximation to local strain. Each component of the strain tensor \(\varepsilon _{ij}\) is computed via
Thus, crack closure is a local maximum in the \((3,3)-\) component of the strain. As we have voxel-wise displacement fields, we can approximate this component easily by finite differences.
Results and Discussion
Comparison to ALDVC
We want to start this section by confirming the findings of our previous work, namely that OF based motion estimations perform much better than state-of-the-art DVC based methods. Therefore, we compare our algorithm to Augmented Lagrangian DVC (ALDVC) [36], which performed best among other DVC methods in our previous study [21]. This comparison will be based on a full coarse-to-fine iteration of MorphFlow, and a full iteration of ALDVC following the original authors’ guidelines to verify, that our method can compete with a state-of-the-art method in materials science.
Fig. 7
Sliceviews of the residuals of refractory concrete before and after warping at reduced resolution. Haar, Daubechies and morphological wavelet approximations were applied twice to have 1/4 of the original voxel edge length. Note that the range of the color bar for Haar and Daubechies differs from MorphFlow due to the min- and max-preserving property of morphological wavelets. Ranges are set according to the 90-percent quantile of all values in the initial residuals to guarantee similar impressions without weight on outliers
The parameters used for MorphFlow for the two datasets are given in Table 1. ALDVC was computed with a subvolume size of 40 voxels and a subset size of 10 voxels in each coordinate direction. Figure 5 shows sliceviews of the full residuals. Visually, MorphFlow seems to perform slightly better, as the residual is in fact minimally darker on average. This is also verified by the RMSEs in Table 2. We can also observe the typical behavior of DVC-based methods, when examining Fig. 6. Though the crack is also visible up to some extent in the solution provided by ALDVC, the full-field displacement calculation produces much clearer and distinct fields, not only regarding the crack, but also regarding sample edges and boundary artifacts. They also appear to be a noisy, downsampled version of our displacement fields.
Fig. 8
Sliceviews of the displacement field components u and v of refractory concrete at reduced resolution. Haar, Daubechies and morphological wavelet approximations were applied twice to reduce the number of voxels in each direction by a factor 4. Note that due to the different value range images are equipped with different color bars
Sliceviews of deformed image downscaled with Daubechies wavelet. Figure a shows the loaded image with 1/4 resolution. Figure b shows the same image, warped by the computed displacement fields. Note the blow “up” of the grains
Scanline representation of sliceviews of refractory concrete. Top row: Selected slices. The red line indicates the position for the grey value profile. We color coded the original gray value 3D images for better visibility of the image content. Bottom row: Corresponding grey value profile, calculated along the red line
As already mentioned, our algorithm has very specific use cases in mind: It shall perform well when applied to low resolution approximations of concrete. In this evaluation we therefore compare our algorithm to several multiresolution schemes: We compute OF based on a coarse-to-fine scheme using i) the classical Gaussian pyramid, ii) Haar wavelets and iii) Daubechies Wavelets. In cases ii) and iii), wavelet approximations that yield similar sizes as the morphological wavelet decomposition are used. Table 2 shows the RMSE for different multiresolution schemes, but at comparable stages. All values are computed for volume images that have roughly 1/4 resolution. Comparing the values immediately reveals another problem when comparing performances against each other: It lies in the nature of the wavelets that the further the approximation proceeds the larger the coefficients become. That is due to the assumption that large coefficients contribute significantly to the actual information content of the signal. Small coefficients on the other side are associated with noise.
Fig. 11
Sliceviews of reinforced concrete. This sliceview allows to spot the glass fiber rebar in the upper third of the image. In both loading stages, the crack forms around the rebar. Figure d shows the displacement in w-direction (horizontal in this view) between unloaded and first loading stage
Sliceviews of strains calculated from displacement between unloaded and first loading step. depicted at the same slice as in Fig. 11. The strain values show good agreement with literature (see for example [26]). Note that strain minima are mainly found at clamping and boundaries and can therefore be neglected
Sliceviews of strains calculated from displacement between unloaded and first loading step depicted at the same slice as in Fig. 1(c) and (d).The original gray value image depicts a very early loading stage and therefore contains very thin cracks, which are difficult to spot. Nevertheless, the strain clearly indicates a crack on all downscaling steps
In contrast, morphological wavelets and Gaussian pyramids preserve the original image range. The RMSEs with respect to Haar and Daubechies wavelet approximations are therefore much higher. It is therefore more interesting to consider how much the RMSE actually decayed. Therefore, Table 2 also includes the initial RMSE for each dataset. Based on this, we can observe another fact: If we only look at the RMSE decay, our method performs better than Gaussian pyramids, but apparently not better than Haar or Daubechies wavelets. This is no surprise: As already mentioned, the RMSE is very susceptible to noise. That means, that the RMSE of a a pair of noisy volume images will be higher than the RSME of its denoised variants. Classical wavelets like Haar and Daubechies aim to suppress noise, as these are the highly oscillatory signal parts which are omitted first in signal decomposition. Morphological wavelets on the other side use operators that are likewise sensitive to noise, and even worse, may amplify it. This is also visually supported when looking at the residuals in Fig. 7, where Fig. 7(a), (b), (d) and (e) look much smoother than Fig. 7(c) and (f).
It makes therefore sense to consider an error measure that is less susceptible to noise, the already mentioned ML-SSIM. Here in fact our method does perform better – the volume images are more similar with MorphFlow. Interestingly, the ML-SSIM regarding Gaussian smoothing even decays: When applied to concrete, classical coarse-to-fine methods destroy so much of the signal, that a motion estimation algorithm fails to recover any detail.
Residuals together with displacement fields now unveil another problem with classical wavelet approximations. Haar and Daubechies wavelets approximate the original image so well on the lower scales, that almost all details are still preserved. That unfortunately traps the algorithm in a local minimum, which can be seen by the wavy pattern in Fig. 8. These give the correct idea, but produce unlikely valleys and hills in the displacement. Even further: If we compare the residuals from classical wavelet approaches with the one stemming from MorphFlow, we observe a bubble-like error pattern in the first two, whereas the latter just exhibits a rather regular noise pattern. If we now look exemplary at the deformed image based on the Daubechies wavelet in Fig. 9, we see that in fact the wavy pattern of the displacement field “blows up” the grains in the mortar phase. Clearly, this cannot represent a correct displacement field.
Morphological wavelets now unite the best of both worlds. They discard unwanted or unnecessary details such as grain boundaries, and dissolve them into noise. At the same time, they preserve the load induced detail of interest, namely the crack. Therefore, the displacement field appears to be very plausible, and nicely presents a jump, or a large variation along the crack. This will yield a maximum in strain – exactly what is expected mechanically.
Let us once more carefully observe Fig. 7. Interestingly, the crack is not displayed as strongly in the residual as one would expect. Recalling, that several authors use bad residuals to indicate where the crack occurs [18, 19], this is surprising at first sight. A different illustration can be found in Fig. 10. Here, we used the previously mentioned scanline representation, where the plot displays the grey value profile along the red line in the volume images. Clearly, Gaussian approximations smooth out all local minima and maxima. But surprisingly, this also holds true up to certain extent for classical wavelets. The local minimum in form of a crack around \(x=60\) is damped quite significantly. This is also a reason, why motion estimation with classical wavelets does not produce plausible displacement fields.
Performance on Compressed Data
The second sample now shows how well our method predicts cracks already at low stages. Figure 11 shows a different sliceview with an additional loading stage. Again, the crack is difficult to spot in (Fig. 11(b)), but clearly visible in (Fig. 11(c)). The displacement field in w-direction in (Fig. 11(d)) also indicates a relatively small, smooth displacement. However, if we now calculate strain as described in equation (42), we see a clear and distinct local maximum along the crack surface in Figs. 12 and 13. Even further, up until the third downscaling step we still see a distinct local minimum at the crack location. This reduction of complexity is one of the core contributions of our algorithm: To receive a decent first indication and approximation of the crack we require only \(0.2\%\) of the original number of voxels, as the third downscaling step reduces the number of voxels in each coordinate direction by a factor 8.
Conclusion
We presented a novel approach to multiresolution motion estimation based on optical flow and morphological wavelets. We showed that our algorithm is beneficial when applied to concrete datasets. Multiresolution schemes have been applied successfully in motion estimation numerous times. Most applications however involve real world camera images. In this area, Gaussian image pyramids are considered state-of-the-art. In applications such as in situ investigations of concrete they unfortunately often fail. Other “classical” multiresolution schemes like wavelets on the other side turned out to preserve too much detail to allow for accurate and plausible displacement fields. We showed that morphological wavelets unify the best of both approaches: They preserve the detail of interest – the crack – while removing unnecessary detail, such as aggregates.
We showed that with this approach very accurate displacement fields can be achieved. Even further, due to their signal approximating nature, using morphological wavelets can be considered as calculations on compressed data. Therefore, we also investigated how well low resolution approximations perform in comparison. Morphological wavelets turned out to be superior once more. They allow for crack indicators on a fraction of the original number of voxels and even performed well for cracks with a width less than a voxel.
Acknowledgements
Funding by the Research Initiative of the Federal State of Rhineland-Palatinate through the priority area Mathematics Applied to Real-World Problems (MathApp) is gratefully acknowledged. This work was supported by the German Federal Ministry of Education and Research (BMBF) [grant number 05M2020 (DAnoBi)]. We thank Jana Hubálková from TU Bergakademie Freiberg for providing the refractorary concrete samples and image data. We thank Szymon Grzesiak from RPTU Kaiserslautern-Landau for providing the samples for the experiments on the reinforced concrete and Michael Salamon from Fraunhofer EZRT for scanning.
Declarations
Conflicts of Interest
The authors declare that they have no conflict of interest.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Barisin T, Jung C, Müsebeck F et al (2022) Methods for segmenting cracks in 3d images of concrete: a comparison based on semi-synthetic images. Pattern Recogn 129:108747CrossRefMATH
2.
Bay BK, Smith TS, Fyhrie DP et al (1999) Digital volume correlation: three-dimensional strain mapping using X-ray tomography. Exp Mech 39(3):217–226CrossRef
3.
Borsdorf A, Raupach R, Flohr T et al (2008) Wavelet based noise reduction in ct-images using correlation analysis. IEEE Trans Med Imaging 27(12):1685–1703CrossRefMATH
4.
Chai T, Draxler RR (2014) Root mean square error (rmse) or mean absolute error (mae)?-arguments against avoiding rmse in the literature. Geoscientific Model Development 7(3):1247–1250CrossRef
5.
Chambolle A (2004) An algorithm for total variation minimization and applications. Journal of Mathematical Imaging and Vision 20:89–97MathSciNetCrossRefMATH
6.
Chateau C, Nguyen TT, Bornert M, et al (2018) DVC-based image subtraction to detect microcracking in lightweight concrete. Strain 54
7.
Chavez Panduro EA, Cordonnier B, Gawel K et al (2020) Real time 3d observations of portland cement carbonation at co2 storage conditions. Environmental Science & Technology 54(13):8323–8332CrossRef
8.
De Zeeuw P (2002) A toolbox for the lifting scheme on quincunx grids (lisq)
9.
Dérian P, Héas P, Herzet C et al (2013) Wavelets and optical flow motion estimation. Numerical Mathematics: Theory, Methods and Applications 6(1):116–137MathSciNetMATH
10.
Gastounioti A, Tsiaparas NN, Golemati S et al (2011) Affine optical flow combined with multiscale image analysis for motion estimation of the arterial wall from b-mode ultrasound. In: 2011 Annual international conference of the IEEE engineering in medicine and biology society, IEEE, pp 559–562
11.
Heijmans HJ, Goutsias J (2000) Nonlinear multiresolution signal decomposition schemes. ii. morphological wavelets. IEEE Transactions on Image Processing 9(11):1897–1913
12.
Hosntalab M, Aghaeizadeh Zoroofi R, Abbaspour Tehrani-Fard A et al (2010) Classification and numbering of teeth in multi-slice ct images using wavelet-fourier descriptor. Int J Comput Assist Radiol Surg 5:237–249CrossRefMATH
13.
Huang Y, Yan D, Yang Z et al (2016) 2d and 3d homogenization and fracture analysis of concrete based on in-situ x-ray computed tomography images and monte carlo simulations. Eng Fract Mech 163:37–54CrossRefMATH
14.
Jiang H, Ji H, Jin N et al (2020) Simulation and experimental verification of the non-uniform corrosion cracking process of reinforced mortar specimen. Constr Build Mater 265:120522CrossRefMATH
15.
Landis E, Bolander JE (2009) Explicit representation of physical processes in concrete fracture. J Phys D Appl Phys 42:214002CrossRefMATH
16.
Le Gall D, Tabatabai A (1988) Sub-band coding of digital images using symmetric short kernel filters and arithmetic coding techniques. In: ICASSP-88., International conference on acoustics, speech, and signal processing, IEEE, pp 761–764
17.
Lindeberg T (1994) Scale-space theory: a basic tool for analyzing structures at different scales. J Appl Stat 21(1–2):225–270CrossRefMATH
18.
Lorenzoni R, Curosu I, Léonard F et al (2020) Combined mechanical and 3d-microstructural analysis of strain-hardening cement-based composites (shcc) by in-situ x-ray microtomography. Cem Concr Res 136:106139CrossRefMATH
19.
Mao L, Yuan Z, Yang M et al (2019) 3d strain evolution in concrete using in situ x-ray computed tomography testing and digital volumetric speckle photography. Measurement 133:456–467CrossRefMATH
20.
Mehranian A, Ay MR, Rahmim A et al (2013) X-ray ct metal artifact reduction using wavelet domain l_ {0} sparse regularization. IEEE Trans Med Imaging 32(9):1707–1722CrossRef
21.
Nogatz T, Redenbach C, Schladitz K (2022) 3D optical flow for large CT data of materials microstructures. Strain 58(3):e12412
22.
Padma A, Sukanesh R (2014) Segmentation and classification of brain ct images using combined wavelet statistical texture features. Arab J Sci Eng 39:767–776CrossRefMATH
23.
Rudin LI, Osher S, Fatemi E (1992) Nonlinear total variation based noise removal algorithms. Physica D 60(1–4):259–268MathSciNetCrossRefMATH
24.
Sandic-Stankovic D (2011) Morphological wavelets for 3d volume image decorrelation. In: 2011 17th International Conference on Digital Signal Processing (DSP), IEEE, pp 1–6
25.
Saputro AH, Mustafa MM, Hussain A et al (2010) Myocardial motion analysis using optical flow and wavelet decomposition. In: 2010 6th International Colloquium on Signal Processing & its Applications, IEEE, pp 1–5
26.
Shen Q, Chen W, Liu C et al (2019) The tensile strength and damage characteristic of two types of concrete and their interface. Materials 13(1):16CrossRefMATH
27.
Stock AM, Herl G, Sauer T et al (2020) Edge-preserving compression of ct scans using wavelets. Insight-Non-Destructive Testing and Condition Monitoring 62(6):345–351CrossRefMATH
28.
Sweldens W (1998) The lifting scheme: a construction of second generation wavelets. SIAM J Math Anal 29(2):511–546MathSciNetCrossRefMATH
29.
Syrakos A, Varchanis S, Dimakopoulos Y et al (2017) A critical analysis of some popular methods for the discretisation of the gradient operator in finite volume methods. Phys Fluids 29(12):127103
30.
Trtik P, Stähli P, Landis E et al (2007) Microtensile testing and 3D imaging of hydrated Portland cement. Proceedings 6th international conference on fracture mechanics of concrete and concrete structures (FraMCoS-VI). Taylor & Francis, London, pp 1277–1282
31.
Tsitova A, Bernachy-Barbe F, Bary B et al (2022) Damage quantification via digital volume correlation with heterogeneous mechanical regularization: application to an in situ meso-flexural test on mortar. Experimental Mechanics pp 1–17
32.
Van De Ville D, Blu T, Unser M (2005) On the multidimensional extension of the quincunx subsampling matrix. IEEE Signal Process Lett 12(2):112–115CrossRefMATH
33.
Wang Z, Bovik AC, Lu L (2002) Why is image quality assessment so difficult? In: 2002 IEEE International conference on acoustics, speech, and signal processing, IEEE, pp IV–3313
34.
Wang Z, Simoncelli EP, Bovik AC (2003) Multiscale structural similarity for image quality assessment. In: The thrity-seventh Asilomar conference on signals, systems & computers, 2003, Ieee, pp 1398–1402
35.
Wu YT, Kanade T, Cohn J et al (1998) Optical flow estimation using wavelet motion model. In: Sixth international conference on computer vision (IEEE Cat. No. 98CH36271), IEEE, pp 992–998
36.
Yang J, Hazlett L, Landauer A et al (2020) Augmented lagrangian digital volume correlation (aldvc). Exp Mech 60:1205–1223CrossRef
37.
Yang Z, Ren W, Mostafavi M et al (2013) Characterisation of 3d fracture evolution in concrete using in-situ x-ray computed tomography testing and digital volume correlation. In: 8th International conference on fracture mechanics of concrete and concrete structures, FraMCoS 2013, International Center for Numerical Methods in Engineering (CIMNE), pp 236–242
38.
Yang Z, Qsymah A, Peng Y et al (2020) 4d characterisation of damage and fracture mechanisms of ultra high performance fibre reinforced concrete by in-situ micro x-ray computed tomography tests. Cement Concr Compos 106:103473
39.
Zach C, Pock T, Bischof H (2007) A duality based approach for realtime TV-\(L_1\) optical flow. In: Joint pattern recognition symposium, Springer, pp 214–223
in hv-lifting (Figure a),
in d1-lifting (Figure b) and
in d2-lifting (Figure c). Actual neighboring relations are depicted in color, initial neighboring relations in dashed lines are kept for better orientation
,
,
,
,
,
,
,
in Fig. 3. These subsets can be directly identified with their modulo restrictions, i.e.
Instead of checking for the modulo restrictions, maxima (or minima) can now efficiently be taken over coinciding positions in neighboring subsets, i.e. the prediction operator in the cubic case equation (19) becomes
and accordingly for
,
,
\(\big )\) to define the full operator \(P^{\text {cub}}\). This allows for very efficient implementations, as not only possibly large datasets are divided into smaller ones, but also because there is no single index check anymore. Because the neighboring relations in this first step are only horizontally and vertically (and in plane, which we discard in naming conventions), see (Fig. 4(a)), we call this step Horizontal/Vertical Lifting or hv-Lifting.
,
,
\(\big )\). The result is then used for an update step, again with straightforward neighboring relations, for \(\big (\)
,
,
,
\(\big )\)
,
\(\big )\) and \(\big (\)
,
\(\big )\). Again, we only depict the neighboring relation for prediction of
. It can be found in (Fig. 4(b)).
are horizontally given by
itself, and diagonally by
. Therefore, the gradient is calculated via
Having now derived a method to compute gradients on arbitrary grids, we can apply the TV OF variant of Section “Optical Flow for Volume Image” to the quincunx grids. We just replace all differences by their corresponding components in the gradient, computed at every grid position as in equation (34).
