Robotic weld groove scanning for large tubular T-joints using a line laser sensor

Weld groove parametrization is a procedure of finding a minimal set of parameters sufficient to fully describe the geometry of the groove. In some literature, this procedure is also referred to as feature extraction. In this work, we will parameterize grooves by the coordinates of the corners points, as well as the coordinates of the end points.

The parametrization algorithm is based on RANSAC (random sample consensus) for sequential searching of line segments. The intersection points between the segments are used as the groove parameters. It is additionally implemented several correction and data noise detection steps in the algorithm. The details of the parametrization procedure are given in [25]. It is, however, important to note that the other parametrization procedures available in the literature can also be used [11, 27, 28], as long as they provide the same type of parameters as an output.

For the groove i in a T-joint, the parametrization gives a set of 5 points given in the coordinates of Frame s (the sensor frame) and relative to the origin of Frame swhere \(\mathbf {p}^s_{s,gj}\) is a vector from the origin of Frame s to the corner point \(\mathbf {p}_{gj}\). The points in Eq. (1) can be expressed in the coordinates of Frame b by the transformationwhere \(\mathbf {R}^b_s \in \mathrm {SO(3)}\) is a rotation matrix [26]. The matrix (1) can also be represented with the homogeneous points as

Then the points in Eq. (3) can be expressed in the coordinates of Frame b and can be given relative to the origin of Frame b by the homogeneous transformationwhere \(\mathbf {T}^b_s \in \mathrm {SE(3)}\) is a homogeneous transformation matrix [26]. The parametrization points are shown graphically in Fig. 3.

The local coordinate frame for the groove i is denoted Frame \(g_i\), see Sect. 2. The origin of Frame \(g_i\) is obtained using the scanned data points and is defined as an intersection between the first and the last line segment in the groove model. In Fig. 3, it is an intersection between the lines along the vectors \(\mathbf {v}_1\) and \(\mathbf {v}_4\). The axes of Frame \(g_i\) are defined as follows. The \(x_{g_i}\)-axis is defined by the normalized product \(\mathbf {v}_{4}^\times \mathbf {v}_{1}\), the \(y_{g_i}\)-axis is defined as negative \(\mathbf {v}_1\) of the groove i, and \(z_{g_i}\)-axis is found as the cross-product of \(x_{g_i}\) and \(y_{g_i}\).

Consider that the vectors \(\mathbf {v}_i^b\) in Fig. 3 are given in the coordinates of Frame b and the Frame \(g_i\) axes are also given as column vectors in the coordinates of Frame b, then the homogeneous transformation from Frame b to Frame \(g_i\) is defined as

In this subsection, we present the mathematical derivations of the proposed stepwise scanning procedure. Such a scanning approach can be beneficial when a weld groove has to be scanned discretely in the sections of interest. The advantage of the procedure is that it does not require frequent scanning, which might be relevant when the groove parametrization algorithm is computationally expensive.

In this subsection, the theoretical background of the proposed procedure is derived for the counterclockwise scanning around the brace element (in the top view). Considerations for the clockwise scanning are given in Appendix 2.

As stated previously, the strategy for robotic weld groove scanning is to perform discrete scans along the groove with a defined distance \(d_{st}\) between each scan. A step motion from the scanned groove section \(i-1\) to the scanned groove section i is referred to as step i. Every step i consists of two control substeps: substep i1 referred to as the prediction substep, and substep i2 referred to as the correction substep. A new control set-point for Frame \(s'\) is defined in every substep. After the substep execution, the frame is denoted \(s'_{ij}\), where ij is a step stamp. It means that the frame is defined at substep j of step i. In this subsection, we will define the set-point for Frame \(s'_{ij}\) as the homogeneous transformation matrix \(\mathbf {T}^b_{s'_{ij}}\). The numbering convention for the weld groove scan frames is similar. The frame of the scan performed after execution of the prediction substep is denoted Frame \(g_{i1}\), and the frame of the scan performed after execution of the correction substep is denoted Frame \(g_{i2}\). The graphical representation of Frame \(s'\) motion is given in Fig. 4.

Consider the configuration of the system when step \(i-1\) has been finished, scan \(i-1,2\) has been done, and Frame \(g_{i-1,2}\) has been formulated. In this configuration, Frame \(s'_{i-1,2}\) is in close vicinity with Frame \(g_{i-1,2}\), that is \(\mathbf {T}^b_{s'_{i-1,2}} \approx \mathbf {T}^b_{g_{i-1,2}}\).

The set-point for the prediction substep of step i is defined aswhich alternatively can be formulated aswhere

The advantage of formulation Eq. (7) instead of formulation Eq. (6) is that vector \(\mathbf {p}^b_{s'_{i-1,2},s'_{i1}}\) appears in the coordinates of Frame b, which are the coordinates the vector is initially obtained in.

In this work, several strategies for the prediction of \(\mathbf {T}^{s'_{i-1,2}}_{s'_{i1}}\) are discussed. Since the geometry of the weld groove is unknown, we propose to base this prediction on the results from the two previous scans, that is, the scans \(i-1\) and \(i-2\). Then, the rotation matrix in \((\mathbf {T}^{s'_{i-1,2}}_{s'_{i1}})_\mathrm {II}\) in Eq. (8) can be predicted aswhich means that the prediction of the rotation from the initial \(i-1\) to the next i scan orientation is assumed to be equal to the rotation between the two previous scans (\(i-2\) and \(i-1\)). An alternative assumption could simply be \(\mathbf {R}^{s'_{i-1,2}}_{s'_{i1}}=\mathbf {I}\), which disregards rotation in the prediction step. We will get back to the discussion on the choice of the rotation prediction in Sect. 4.

The position vector in \((\mathbf {T}^{s'_{i-1,2}}_{s'_{i1}})_\mathrm {I}\) in Eq. (8) can be predicted aswhere Eq. (9) is used and

Here, \(\mathbf {\bar{p}}\) is the normalized form of a vector \(\mathbf {p}\) and \(d_{st}\) is the defined step length. The vector (11) defines the initial prediction of the Frame \(s'\) location for the next scan. Although the assumption \(\mathbf {p}^{b}_{s'_{i-1,2},s'_{i1}}=\mathbf {v}^{b}_{st}\) in Eq. (8) can be a reasonable prediction, especially for the small \(d_{st}\) values, it will be demonstrated in Sect. 4 that Eq. (10) gives better results.

In the correction step, the position and orientation of Frame \(s'\) are corrected to match the predefined scanning distance and angle. This leads to the new set-pointwhere \(\mathbf {p}^{b}_{b,s'_{i2}} = \mathbf {p}^{b}_{b,g_{i1}}\) and the Frame \(s'_{i2}\) axes are defined by Frame \(g_{i-1,2}\) and Frame \(g_{i1}\) as followswhere \(\mathbf {\bar{y}}^b_t\) is the normalized form of the vector \(\mathbf {y}^b_t\). When the robot executes motion to Eq. (12), a new weld groove scan i is recorded, and Frame \(g_{i2}\) is defined. Again, it is noted that after execution of the correction step, Frames \(s'_{i2}\) and \(g_{i2}\) have approximately the same location and orientation.

In Sect. 3.2, we defined a set-point for Frame \(s'\) as the homogeneous transformation matrix \(\mathbf {T}^b_{s'}\). Frame \(s'\) was introduced as the second sensor-fixed frame in Sect. 2, but little was said on how it was defined. In this subsection, we discuss the definition of Frame \(s'\), or, to be more precise, the definition of \(\mathbf {T}^s_{s'}\). Note that we skip the step stamp ij in this subsection for simplicity of notations.

In the stepwise control sequence presented in the previous subsection, the control goal was to move Frame \(s'\) to the desired location. The desired location was aligned with the predicted or corrected (depending on a substep) location of Frame g. In addition, transformation \(\mathbf {T}^s_{s'}\) should be selected such that execution of a set-point for Frame \(s'\) leads to reasonable scanning location and orientation of the laser, that is, reasonable location and orientation of Frame s. In a T-joint, the weld groove geometry is continuously changing, leading to varying reasonable location and orientation of Frame s. It means that the homogeneous transformation \(\mathbf {T}^s_{s'}\) has to be redefined in every step of the procedure.

Assume that the homogeneous transformation \(\mathbf {T}^s_{s'}\) is given aswhere \(\mathbf {p}^{s}_{s,s'}\) is the distance between the origins of Frames s and \(s'\), which, for smaller grooves, can be selected as \(\mathbf {p}^{s}_{s,s'} = [0 \ 0 \ z_{s,s'}]^{\mathrm T}\). The rotation matrix in Eq. (14) is split into two parts for the convenience of derivations: the initial large rotation part and the local small rotation part. The initial rotation matrix \(\mathbf {R}^s_{s'}\) is given as the sequence \(\mathbf {R}_z(\phi )\mathbf {R}_x(\psi )\) [26] with \(\phi = 90\) deg and \(\psi = 180\) deg. The graphical representation of the initial rotation from Frame s to Frame \(s'\) is given in Fig. 2. The local rotation matrix is given aswhere \(\alpha\) is referred to as a pull angle and \(\beta\) is referred to as a work angle. The pull angle is an angle from the plane where the plane normal is parallel to the local weld groove direction vector. Optimally, the pull angle should be \(\alpha = 0\) deg, which would result in the minimal (and the most precise) area of the scanned groove section. However, in practice, having a small pull angle helps to avoid problems with noise caused by the laser reflections. The work angle is introduced to ensure the correct positioning of the laser relative to the scanned groove section. An important aspect is that none of the scanned surfaces should be oriented at acute angles towards the laser rays, where the laser ray is a ray from the origin of Frame s to any scanned point. This gives low scanning resolution and potentially noisier data. A reasonable position of the laser is when the central laser ray is approximately in the middle of the groove opening angle. The groove opening angle is defined as an angle between \(-\mathbf {v}_2\) and \(\mathbf {v}_4\), see Fig. 3. A graphical demonstration of an incorrect and correct choice of the \(\beta\) angle is shown in Fig. 5. In the a) case, \(\beta = 0\) deg, which leads to an acute scanning angle towards the leg pipe surface. In the b) case, \(\beta\) is selected such that the laser (and Frame s) is approximately located in the middle of the groove opening angle. The work angle is defined as followswhere \(\gamma _i\) is given in Fig. 6, while the groove opening angle is \(\gamma _1 + \gamma _2\).

The complete algorithm for the proposed robotic weld groove scanning procedure is presented in Algorithm 1.

The experimental setup is shown in Fig. 7. It consisted of the robotic manipulator Kuka KR5 Arc, the laser scanner Micro-epsilon scanCONTROL 2610-100 attached to the robot end-effector, and the T-joint test object. The test object was a tubular T-joint with a 90 deg angle between a brace and a leg. The two parts were welded together with a one-pass root weld. The leg segment was produced of the tubular profile 600\(\times\)20 mm, and the brace segment was produced of the tubular profile 457.2\(\times\)25.4 mm.

The laser sensor was attached to the robot end-effector with a bracket. The calibration procedure for the determination of \(\mathbf {T}^f_s\) transformation is derived and presented in Appendix 1.

The proposed procedure was implemented on an external control computer. The communication between the external computer, the robot, and the laser sensor was set up over a local network. The control computer generated the \(\mathbf {T}^b_{s'}\) set-points, which were converted to the \(\mathbf {T}^b_f\) set-points for the robot flange. The rotation matrix \(\mathbf {R}^b_{f}\) was converted to ZYX Euler angles before the set-point \(\mathbf {T}^b_f\) was sent to the robot controller. The laser sensor generated 2D point clouds, where points were given in the coordinates of Frame s (the laser sensor local frame). The robot pose was read as ZYX Euler angles and position of the robot flange (relative to the robot base), which were sent to the external computer and converted to the \(\mathbf {T}^b_f\) transformation.

The scanning was done in the counterclockwise direction (in the plan view) over a groove segment of approximately 180 deg. Scanning of the entire groove was not possible due to the limitations of the robot’s operational space. The experimental program consisted of 36 tests, where different step lengths and parameters for the prediction substep were used. The following four cases of the prediction substep parameters were considered:

For each case of the prediction substep parameters, the following step lengths were considered: 10, 20, 30, 40, 50, 60, 70, 80, and 100 mm with corresponding 56, 28, 19, 14, 11, 9, 8, 7, and 5 steps in a test.

It is noted that scanning in the reverse (i.e., clockwise) direction was also tested outside the main experimental program. This was done for verification of the considerations given in Appendix 2.

The scope of the robotic weld groove scanning operation is to automatically collect the groove geometrical data, which could be used for robotic welding path generation. Each executed scanning case generated discrete raw data points obtained directly from the laser scanner. An example of raw data points from the experiment with the 20-mm step length is shown in Fig. 8.

The corresponding parametrization of the raw data is given in Fig. 9. The red dots indicate the root corners, and the black dots indicate all other points. The parametrization was done using the procedure presented in [25] with the groove parameter set described in Sect. 3.1.

As mentioned previously, the geometry of the groove is variable due to the relatively complex interface between the tubular sections. This is demonstrated by two examples of groove section scans given in Fig. 10, where the 1st and the 14th sections from Figs. 8 and 9 are plotted in the coordinates of Frame s.

In general, collecting the results presented above is the main practical goal of the robotic scanning procedure. From the perspective of the algorithm performance, it is important to discuss the precision and efficiency of the proposed procedure. As suggested in Sect. 2, discrete scanning of weld grooves in a large tubular T-joint is a good strategy for groove data collection. The quality of the groove parametrization itself is out of the scope in this work and is discussed in [25]. However, an important aspect of this work is that the parametrization results are of sufficient quality to be used with the proposed scanning algorithm. While the discrete scanning results obtained using the proposed algorithm are of good quality for all four parameter cases, the two-step structure of the procedure led to a slight zig-zag motion of the robot end-effector. An example of the robot end-effector path recorded from the experiment with the 50-mm step length is shown in Fig. 11.

This 3D path plot can, for the sake of clarity, be transformed into the cylindrical coordinates \(\theta , R, z\) and be shown with two plots, see Fig. 12. The origin of the cylindrical coordinates is moved to the center of the brace axis. From Fig. 12, it is seen that the paths in cases 1 and 3 deviate more in the \(\theta R, R\) plot, while the paths in cases 2 and 4 deviate more in the \(\theta R, z\) plot.

Therefore, in the following part of this section, we present an analysis of the optimality of the parameter choices for the different step lengths. Optimality is defined in terms of the step length precision, the total length of the robot end-effector path, and the total cumulative rotation of the robot end-effector.

The actual step length was collected during all the experiments as the distance between the origins of Frames \(g_{i2}\). The mean step length values, as well as the standard deviations, are shown as a bar diagram in Fig. 13. The precision of the mean step length is important for planning the number of scanning steps for a weld groove of known length, while the standard deviation from the mean value is an important factor if step length consistency is a requirement for a weld path planner. The mean step length errors relative to the nominal values are shown in Fig. 14.

The results show that for the 10-mm steps, the step lengths were of good precision for all four cases with the relative errors of 0.3–0.6 %. For the larger steps, the relative errors for parameter case 2 remained below 0.6 %, while for parameter case 3 — below 0.4 %. For parameter cases 1 and 4, however, the results looked differently. For parameter case 1, the relative mean errors increased up to 2.9 %, while for parameter case 4, the relative mean errors increased up to 13.5 %. The standard deviation was the lowest in all experiments for case 1, while the highest standard deviation was observed for case 4. The parameter cases 2 and 3 had insignificant standard deviation up to the 50-mm step length, and then it raised more rapidly for larger steps.

These results suggested that the step lengths for cases 2 and 3 are close to their nominal values and are preferable if the step length precision is important for the application case. If the step length consistency is the most important factor, parameter case 1 is the preferred choice. Parameter case 4 performed poorly both in terms of the mean step length and standard deviation.

The efficiency of the proposed scanning algorithm is evaluated in terms of the total end-effector travel distance and the cumulative rotation of the end-effector. Given constant end-effector translation and rotation speeds, this metric is equivalent to the scanning time. The efficiencies are compared for all four prediction parameter cases and all nine different scanning step lengths.

The results for the end-effector travel distances are shown as a bar diagram in Fig. 15. The lighter and darker colors indicate the cumulative travel distances of the prediction and correction substeps, respectively.

The obtained results suggested that the total travel distance for parameter cases 1 and 3 is shorter than the total travel distance for parameter cases 2 and 4. The difference was largest for the shortest 10-mm scanning step length, with the travel distances of 1945 mm for case 3 and 3344 mm for case 4. For the largest 100-mm step, the difference was practically negligible, with the travel distances of 1619 mm for case 3 and 1718 mm for case 2. For parameter cases 1 and 3, the cumulative travel distance during the prediction substeps was shorter than the travel distance during the correction substeps. The relative relation was opposite for parameter cases 2 and 4. This was due to the fact that in cases 1 and 3, no rotation prediction was used (i.e., \(\mathbf {R}^{s'_{i-1,2}}_{s'_{i1}} = \mathbf {I}\)), and all the correcting motion was made during the correction substep. For cases 2 and 4, the orientation of the following scan was predicted by (9), which means that a part of the correcting motion was already introduced in the prediction substep. Since the overall travel distance was consistently larger for cases 2 and 4, it can be concluded that the correction motion during the prediction substep was overestimating the orientation of the following scan. This led to the additional motion in the prediction substep and the additional correction motion in the correction substep. The parameter cases 1 and 3 performed nearly equally good with case 1 being marginally better for 20- and 30-mm steps, while case 3 was marginally better for all other step lengths.

The results for the cumulative rotation angle of the end-effector had a similar pattern as the results for the travel distance, see Fig. 16.

The parameter cases 1 and 3 performed better for all the tested step lengths. Case 1 performed marginally better for all the step lengths except for the 10-mm step. Comparing cases 1 and 3 with 2 and 4, the difference was the largest for the 10-mm step length, with the cumulative rotation angle of 182 deg for case 3 and 402 deg for case 4. The difference was the smallest for the 100-mm step, with 133 deg for case 1 and 195 deg for case 2. The justification of why cases 2 and 4 performed worse is the same as in the travel distance analysis. The orientation of the next scanned section was overestimated, which led to an additional rotation of the robot end-effector during both the prediction and correction substeps.

It can be concluded that the prediction parameter cases 1 and 3 provided overall better performance than the parameter cases 2 and 4. Case 1 is recommended to use when the scanning step length consistency (low standard deviation) is important for a robotic welding path generator. Case 3 is recommended to use when the mean step length should be closer to the nominal step length value. However, for groove scanning with very large steps, it is recommended to use the prediction parameter cases 2 or 4. The reason is that for large steps, cases 1 and 3 resulted in rather acute scanning angles, which sometimes led to that the correction step scan was not obtainable. In the conducted experiments, this was observed for the 100-mm step length.

In an industrial setup, scanning of grooves in T-joints is still mostly a manual operation, where a robot operator manipulates the robot to the desired scanning positions. This is a time-consuming process, while the scanning step and laser angles towards the groove are of lower accuracy due to manual adjustment. The authors, for the sake of comparison, have performed several manual scanning operations. The next scanning step was possible to plan by measuring with a simple ruler, and it was, however, more challenging to measure and adjust the two angles towards the groove due to the lack of a good measurement reference. The average time used for one scan was about 1 min. Considering a case with the scanning step of 40 mm, this would result in the total scanning time of 14 min (i.e., 840 s). Alternatively, using the proposed robotic scanning procedure (with the parameter set 3) results in the total scanning time of 72 s, which is more than 10 times faster compared to manual scanning.

As mentioned previously, the precision of parametrization of each separate groove scan is out of the scope of this work and is discussed in [25]. It is, however, important to verify that the relative location of all groove scans of the T-joint has acceptable precision. This verifies that the obtained groove corners actually represent the groove geometry of the T-joint. The verification was done using the additional 3D data acquired using a high-precision Zivid Two ZVD2 structured light 3D camera. The point cloud of the T-joint used in the comparison is shown in Fig. 17.

The raw point clouds acquired by the robotic scanning and Zivid camera had different reference coordinate systems, and, therefore, were fitted using the features of pipes and grooves. This allowed for obtaining the transformation between two coordinate systems. Then, the least-square error between the groove corners and the Zivid point cloud was found. It is noted that the comparison was done for one prediction parameter case and for a segment of around 140 deg due to limitations of the field of view in the Zivid camera. The average groove corner point error for all the tested step lengths in one parameter case was found to be 0.49 mm, which also includes the error from the groove parametrization itself, see [25]. The magnitude of the error is considered to be acceptable for welding applications.