Measuring the 3D shape of high temperature objects using blue sinusoidal structured light

Planck's law shows the relationship between radiation energy and radiation wavelength of black-bodies at different temperatures [1]. As shown in figure 1 and described as:

$$\begin{eqnarray}&&E(\lambda ,T)=\frac{{{C}_{1}}}{{{\lambda}^{5}}{{\text{e}}^{\frac{{{C}_{2}}}{\lambda T}}}-1}.\end{eqnarray} \tag{ 1 }$$

where E is monochromatic radiation intensity (W cm⁻² μm⁻¹), ${{C}_{1}}=3.74\times {{10}^{4}}$ W μm cm⁻², ${{C}_{2}}=1.44\times {{10}^{4}}$ μm, λ is radiation wavelength.

Zoom In Zoom Out Reset image size

Many objects (such as heat sources) can be considered to approximate a black body. Figure 1 shows that the radiation energy of the forgings is mainly distributed in red and infrared waves when the temperature is less than 1200 °C. It suggests that PMP might be applied to measure a shiny surface based on color structured light [16–18]. In this paper, 3D measurement of high temperature objects below 1200 °C was investigated by green or blue structured light. Since the spectrum of green structured light is in close proximity to red visible radiation, the degree of crosstalk between R and G channels is greater than that between R and B channels. In order to avoid the crosstalk among channels as much as possible, the blue structured light is proposed for the fringe projection.

The measurement system is shown in figure 2. It consists of a digital light processing (DLP) projector, a 3-CCD camera and a computer. Three sets of blue sinusoidal fringe pattern images with different pitch are generated by software and projected onto the object surface by the DLP projector. The deformed pattern images are captured by the 3-CCD camera which is positioned at a certain angle to the DLP projector. The images are then separated into their RGB components by software. B grayscale images are used to carry out the wrapped phase and the unwrapping phase algorithms. Finally, global coordinates of the measured object are established based on the calibration relationship between the 2D image coordinate system and the 3D global coordinate system.

Zoom In Zoom Out Reset image size

In theory, the shape measurement can be attained by substituting the monochrome camera and the band-pass filter for the 3-CCD cameras described above. The projector still projects the blue structured light onto the surface of the high temperature object. Only the blue structured light passes the band-pass filter and arrives at the CCD plane in the camera. Taking band-pass width and band-pass ripple of the filter into consideration, these factors may influence the sinusoidal properties of the deformed pattern [19]. Consequently, the former is adopted as the measurement method of a high temperature object.

In figure 3, o_c-u_cv_cw_c, o_p-u_pv_pw_p and O-XYZ represent the camera, the DLP projector and global coordinate system, respectively. Suppose B is a point on the measured object and its coordinate is (X, Y, Z) in the global coordinate system. The pixel coordinate of point B is (u_c, v_c) at the sensor plane (c_c) of the camera and the pixel coordinate of point B is (u_p, v_p) at the sensor plane (c_p) of the projector. According to the geometry relationship of perspective projection and the relationship between the pixel coordinate system of the camera and the global coordinate system [20, 21], the following equations are established:

$$\begin{eqnarray}&&{{s}_{\text{c}}}{{\left[{{u}_{\text{c}}},{{v}_{\text{c}}},1\right]}^{\text{T}}}={{A}_{\text{c}}}{{M}_{\text{c}}}{{\left[X,Y,Z,1\right]}^{\text{T}}},\end{eqnarray} \tag{ 2 }$$

where s_c is a scaling factor and is an unknown, A_c and M_c are the intrinsic and extrinsic parameter matrixes of the camera. A_c is a 3  ×  3 matrix and M_c is a 3  ×  4 matrix.

Zoom In Zoom Out Reset image size

The parameter matrixes of the camera are obtained by using the Matlab camera calibration toolbox with checkerboard. The camera is calibrated as follows: (1) Moving the checkerboard, the camera captures eight checkerboard images at different positions from the camera. (2) Using the eight checkerboard images, the parameter matrixes and distortion coefficient of the camera are obtained by the Matlab calibration toolbox.

Similarly, the relationship between the projector coordinate system and the global coordinate system is established:

$$\begin{eqnarray}&&{{s}_{\text{p}}}{{\left[{{u}_{\text{p}}},{{v}_{\text{p}}},1\right]}^{\text{T}}}={{A}_{\text{p}}}{{M}_{\text{p}}}{{\left[X,Y,Z,1\right]}^{\text{T}}},\end{eqnarray} \tag{ 3 }$$

where s_p is a scaling factor and is an unknown, A_p and M_p are the intrinsic and extrinsic parameter matrixes of the projector. A_p is a 3  ×  3 matrix and M_p is a 3  ×  4 matrix.

Viewed as an inverse camera, the calibration of the projector requires a red/blue checkerboard because the reflectivity of the red is close to the blue about white light. The calibration procedure includes the following steps [22–24]:

• (1)  
  An image of red/blue checkerboard with red light illumination is captured by the camera, as shown in figure 4(a). The coordinates of each grid corner are extracted, as shown in figure 4(b).
• (2)  
  The projector projects a series of horizontal and vertical phase-shifting images onto the checkerboard and the camera captures the deformed pattern images, as shown in figures 4(c) and (d). Using the four-step phase-shifting algorithm and the 3-frequency heterodyne algorithm, the horizontal and vertical wrapped phase and unwrapping phase maps are calculated as shown in figures 4(e)–(h). The unwrapping phase values of each grid corner are obtained by using a linear interpolation algorithm including the horizontal and vertical unwrapping phase.
• (3)  
  According to the horizontal and vertical unwrapping phase values of each grid corner, the position of the grid corner is found in the projected image. Thus the Digital Micro-mirror Device (DMD) image of the projector is obtained as shown in figure 4(i).
• (4)  
  The position of the checkerboard changed and steps 1–3 are repeated, which provides eight DMD images calibrating the projector.
• (5)  
  The eight DMD images are then used to calculate the intrinsic and extrinsic parameters using the Matlab camera calibration toolbox.

Zoom In Zoom Out Reset image size

u_p is given by:

$$\begin{eqnarray}&&{{u}_{\text{p}}}=\frac{\Phi\left({{u}_{\text{c}}},{{v}_{\text{c}}}\right)}{N\times 2\pi}\times W,\end{eqnarray} \tag{ 4 }$$

where Φ(u_c, v_c) is the unwrapping phase of point B, W is the resolution of the DLP projector in the horizontal direction and N is the resolution of the camera in the horizontal direction. s_c, s_p, v_p, X, Y and Z are six unknowns in equations (2) and (3). Since there are six unknowns and six linearly independent equations in equations (2) and (3), the global coordinate (X, Y, Z) of point B can be obtained to solve these six equations. Hence, the key is to solve the unwrapping phase Φ(u_c, v_c) for the global coordinate of point B.

Two orders of radial and tangential distortions are considered when the intrinsic and extrinsic parameters are calculated using the Matlab calibration toolbox. The calibration accuracies of the camera and projector are 0.03 pixels and 0.22 pixels after the unwrapping phase is compensated by the Look-up table algorithm. The measurement accuracy is greater than 2  ×  10⁻⁴ within the measurement volume size [25]. The algorithms between the Matlab calibration toolbox and Robert Sitnik are consistent with only an accurate calibration model required. However, Sitnik's calibration method takes all unknown parameters into consideration including all existing objective aberrations. The measurement uncertainty within 10⁻⁴ can be found with respect to the measurement volume size [26]. This new method can be applied for calibration. The global coordinate is defined by polynomials of the unwrapping phase. Without precise placement of the calibration model, these polynomials are calculated by phase distribution on a 2D-calibration model positioned manually in several unknown positions.

Resolving the unwrapping phase is critical to implementing the shape measurement rapidly for the measured object. Therefore, the wrapped phase calculated by B grayscale images should be unwrapped using the 3-frequency heterodyne algorithm which is mentioned above.

The 3-frequency heterodyne algorithm means that three sets of fringe images are required to project onto the measured object because the unwrapping phase is calculated by three wrapped maps. The unwrapping phase at each pixel is calculated by the values of the pixel at three wrapped phase maps and independent of other pixels. Therefore, the phase is not affected by the discontinuities on the object surface. Simultaneously, the independence of other pixels eliminates the phase error in the spatial propagation.

Suppose the fringe pitch of three sets of sinusoidal fringe patterns is p₁, p₂ and p₃, respectively. Each set of the fringe patterns includes four phase-shifting images. The phase shift between neighboring images is 90°. A four-step phase-shifting algorithm is used to compute the three wrapped phase maps. ${{\varphi}_{j}}$ is given by:

$$\begin{eqnarray}&&{{\varphi}_{j}}(x,y)=\arctan \frac{{{I}_{j}}_{4}(x,y)-{{I}_{j}}_{2}(x,y)}{{{I}_{j}}_{1}(x,y)-{{I}_{j3}}(x,y)},\,\,\,\,\,j=1,2,3,\end{eqnarray} \tag{ 5 }$$

where I_ji(x, y) is the intensity of each pixel in the deformed patterns, φ_j(x, y) is wrapped phase. Two phase maps(φ₁ and φ₂), whose pitch is p₁ and p₂, respectively, subtract from each other to set up another phase map φ₁₂ whose pitch is p₁₂ according to the 3-frequency heterodyne principle. p₁₂ is given by:

$$\begin{eqnarray}&&{{p}_{12}}=\frac{{{p}_{1}}\centerdot {{p}_{2}}}{{{p}_{2}}-{{p}_{1}}}\,.\end{eqnarray} \tag{ 6 }$$

The phase maps, whose pitch is p₁₂ and p₃ respectively, subtract from each other to obtain the unwrapping phase, whose pitch is p. p is defined as:

$$\begin{eqnarray}&&p=\frac{{{p}_{12}}\centerdot {{p}_{3}}}{{{p}_{3}}-{{p}_{12}}}\,.\end{eqnarray} \tag{ 7 }$$

Select p₁  =  20, p₂  =  30 and p₃  =  64. According to equations (6) and (7), the values of p₁₂ and p can be determined as 60 and 960, respectively, as shown in figure 7.

Zoom In Zoom Out Reset image size

Suppose Q is a point on the deformed pattern images. Its pixel coordinates are (x, y) and the three wrapped phase values are φ₁(x, y), φ₂(x, y) and φ₃(x, y), corresponding to three wrapped phase maps. The orders of the point Q are N₁  +  ▵n₁, N₂  +  ▵n₂ and N₃  +  ▵n₃ on three wrapped phase maps φ₁, φ₂ and φ₃, respectively. m₁₂  +  ▵m₁₂ is the synthetic fringe order of the point on the synthetic phase map constructed by φ₁ and φ₂. N₁, N₂, N₃ and m₁₂ are the integer parts of the order and the ▵n₁, ▵n₂, ▵n₃ and ▵m₁₂ are the fractional parts of the order. The relationship between the order and wrapped phase is expressed as equations (8) and (9):

$$\begin{eqnarray}&&{{p}_{1}}\left({{N}_{1}}+\Delta {{n}_{1}}\right)={{p}_{2}}\left({{N}_{2}}+\Delta {{n}_{2}}\right)={{p}_{3}}\left({{N}_{3}}+\Delta {{n}_{3}}\right)\\ &&={{p}_{12}}\left({{m}_{12}}+\Delta {{m}_{12}}\right)\,\,,\end{eqnarray} \tag{ 8 }$$

$$\begin{eqnarray}&&\Delta {{n}_{j}}=\frac{{{\varphi}_{j}}+\pi}{2\pi}\,,\,\,\,\,\,j=1,2,3.\end{eqnarray} \tag{ 9 }$$

Solving equations (8) and (9) for ▵m₁₂ and m₁₂, we obtain:

$$\begin{eqnarray}\Delta {{m}_{12}}=\left\{\begin{array}{*{35}{l}} \begin{array}{c} \frac{{{\varphi}_{1}}-{{\varphi}_{2}}}{2\pi} & {{\varphi}_{1}}-{{\varphi}_{2}}\geqslant 0 \end{array} \\ \begin{array}{c} \frac{{{\varphi}_{1}}-{{\varphi}_{2}}}{2\pi}+1 & {{\varphi}_{1}}-{{\varphi}_{2}}<0 \end{array} \end{array}\right. \end{eqnarray} \tag{ 10 }$$

$$\begin{eqnarray}{{m}_{12}}=\left\{\begin{array}{*{35}{l}} \begin{array}{c} \frac{{{p}_{3}}\left(1+\Delta {{m}_{12}}-\left({{\varphi}_{3}}+\pi \right)/2\pi \right)}{{{p}_{3}}-{{p}_{12}}}-\Delta {{m}_{12}}, & \Delta {{m}_{12}}-\left({{\varphi}_{3}}+\pi \right)/2\pi \leqslant 0 \end{array} \\ \begin{array}{c} \frac{{{p}_{3}}\left(\Delta {{m}_{12}}-\left({{\varphi}_{3}}+\pi \right)/2\pi \right)}{{{p}_{3}}-{{p}_{12}}}-\Delta {{m}_{12}}, & \Delta {{m}_{12}}-\left({{\varphi}_{3}}+\pi \right)/2\pi \geqslant 0 \end{array} \end{array}.\right. \end{eqnarray} \tag{ 11 }$$

Computing m₁₂ whose pitch is p₁₂ is the key to the 3-frequency heterodyne method because the errors of m₁₂ lead to the incorrect unwrapping phase. Since m₁₂ is an integer, the error of m₁₂ must be limited to the range between  −0.5 to  +0.5 when m₁₂ is rounded. It means that the errors of φ₁, φ₂ and φ₃ must be within the scope shown in equation (12). Outside the scope, the value of m₁₂ is incorrect. From equation (11), it can be obtained:

$$\begin{eqnarray}|\text{d}{{m}_{12}}|&=&\left|\frac{{{p}_{12}}}{2\pi \left(\,{{p}_{3}}-{{p}_{12}}\right)}\text{d}{{\varphi}_{1}}-\frac{{{p}_{12}}}{2\pi \left(\,{{p}_{3}}-{{p}_{12}}\right)}\text{d}{{\varphi}_{2}}\right.\\ &&-\left.\frac{{{p}_{3}}}{2\pi \left(\,{{p}_{3}}-{{p}_{12}}\right)}\text{d}{{\varphi}_{3}}\right|<0.5\,.\end{eqnarray} \tag{ 12 }$$

where dm₁₂, dφ₁, dφ₂ and dφ₃ are the errors of m₁₂, φ₁, φ₂ and φ₃ respectively, and p₁₂ is the pitch of the synthetic fringe constructed by φ₁ and φ₂.

Although the deformed pattern image with fine sinusoidal property is obtained, as mentioned in section 3, there are still errors in grayscale value (intensity) about the deformed pattern image induced by ambient noise, quantization of digital equipment, micro lens, etc. Hence, B color images contain high frequency components in addition to the fundamental frequency component, shown in figures 8(d)–(f). Ideally, the deformed pattern should only contain fundamental frequency components in the frequency domain [27]. Moreover, the high frequency component in the B components will reduce the accuracy of the wrapped phase and lead to incorrect values of the unwrapping phase.

Zoom In Zoom Out Reset image size

There are three color channels in the DLP projection and camera system. The crosstalk between the neighboring channels should be considered because it will lower measurement accuracy. This will be studied in future work. Here, we focus on resolving and correcting the unwrapping phase.

In this paper, a filtering method is proposed to improve the accuracy of the unwrapping phase. The deformed pattern images are filtered by a Butterworth lowpass filter to remove the high frequency from the fundamental frequency component. The deformed pattern images are filtered and their Fourier transform (FT) images are shown in figures 8(g)–(l). Figures 9(a) and (b) are cross-section plots of the 70th row in figures 8(c) and (i). It can be seen that better sinusoidal property is present in the filtered deformed pattern images than the initial ones captured by CCD camera from figures 8 and 9.

Zoom In Zoom Out Reset image size

${{\varphi}_{j}}\left(\,j=1,2,3\right)$ and m₁₂ are calculated by the deformed pattern images and the $\varphi _{j}^{\prime}\left(\,j=1,2,3\right)$ and $m_{12}^{\prime}$ are calculated by the filtered deformed pattern images. The unwrapping phase map Φ_j is calculated by the wrapped phase maps ${{\varphi}_{j}}\left(\,j=1,2,3\right)$. The equation ${{\Phi}_{j}}=2\pi \centerdot k+{{\varphi}_{j}}\left(k=0,1,2\ldots \right)$ is adopted to calculate the unwrapping phase ${{\Phi}_{j}}\left(\,j=1,2\right)$ corresponding to the wrapped phase ${{\varphi}_{j}}\left(\,j=1,2\right)$. Φ_j is expressed by equation (13):

$$\begin{eqnarray}\scriptsize{{{\Phi}_{j}}(x,y)=\left\{\begin{array}{c} 2\pi \left(\text{round}\left(\frac{{{p}_{3-j}}}{{{p}_{2}}-{{p}_{1}}}\left({{m}_{12}}+1+\frac{{{\varphi}_{1}}-{{\varphi}_{2}}}{2\pi}\right)\right)\right)+{{\varphi}_{j}} & {{\varphi}_{1}}-{{\varphi}_{2}}<0 \\ 2\pi \left(\text{round}\left(\frac{{{p}_{3-j}}}{{{p}_{2}}-{{p}_{1}}}\left({{m}_{12}}+\frac{{{\varphi}_{1}}-{{\varphi}_{2}}}{2\pi}\right)\right)\right)+{{\varphi}_{j}}\,\,\,\, & {{\varphi}_{1}}-{{\varphi}_{2}}\geqslant 0 \end{array}\,\,\,j=1,2.\right. }\end{eqnarray} \tag{ 13 }$$

m₁₂ is shown in figure 10(a). The incorrect values are obtained when they are rounded according to equation (11). Moreover, the unwrapping phase Φ₁ is shown in figure 10(b) by solving equation (13). Because there is a one-to-one correspondence between the incorrect value of m₁₂(x, y) and the unwrapping phase Φ₁(x, y), there are certain incorrect values in figure 10(b). Figures 10(c) and (d) are cross-section plots of the 90th row in figures 10(a) and (b) which indicate the values of m₁₂(90, 87) and Φ₁(90, 87) are incorrect. The correction to the unwrapping phase $\Phi_{1}^{\prime}$ is expressed as:

$$\begin{eqnarray}\scriptsize{\Phi_{j}^{\prime}(x,y)=\left\{\begin{array}{c} 2\pi \left(\text{round}\left(\frac{{{p}_{3-j}}}{{{p}_{2}}-{{p}_{1}}}\left(m_{12}^{\prime}+1+\frac{\varphi _{1}^{\prime}-\varphi _{2}^{\prime}}{2\pi}\right)\right)\right)+{{\varphi}_{j}} & \varphi _{1}^{\prime}-\varphi _{2}^{\prime}<0 \\ 2\pi \left(\text{round}\left(\frac{{{p}_{3-j}}}{{{p}_{2}}-{{p}_{1}}}\left(m_{12}^{\prime}+\frac{\varphi _{1}^{\prime}-\varphi _{2}^{\prime}}{2\pi}\right)\right)\right)+{{\varphi}_{j}}\,\,\,\, & \varphi _{1}^{\prime}-\varphi _{2}^{\prime}\geqslant 0 \end{array}\,\,\,j=1,2.\right. }\end{eqnarray} \tag{ 14 }$$

Zoom In Zoom Out Reset image size

The unwrapping phase $\Phi_{j}^{\prime}$ can be corrected by adding or subtracting the integer times of $2\pi$ using the filtering method from equation (14). Therefore, $\Phi_{j}^{\prime}(x,y)$ at discontinuities on the forging surface is not influenced by the filtering method proposed in this paper. $m_{12}^{\prime}$ and $\Phi_{1}^{\prime}$ in figures 10(e) and (f) indicate all incorrect points are corrected. Figures 10(g) and (h) are the cross-section plots of the 90th row in figures 10(e) and (f) which indicate the correction values of m₁₂(90, 87) and Φ₁(90, 87).