Skip to main content
Top
Published in: The International Journal of Advanced Manufacturing Technology 3-4/2022

31-08-2022 | ORIGINAL ARTICLE

Multiseam tracking with a portable robotic welding system in unstructured environments

Authors: Shuangfei Yu, Yisheng Guan, Zhi Yang, Chutian Liu, Jiacheng Hu, Jie Hong, Haifei Zhu, Tao Zhang

Published in: The International Journal of Advanced Manufacturing Technology | Issue 3-4/2022

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Most welding manufacturing in heavy industries, such as shipbuilding and construction, is conducted in unstructured workspaces, indicating that the production environment is irregular, changeable, and unmodeled. In this case, changeable workpiece position/shape and environmental background/illumination should be carefully considered. Owing to these complicated characteristics, welding currently relies on manual operation, resulting in high cost, low efficiency, and inconsistent quality. This study proposes a portable robotic welding system and a novel seam-tracking method. Compared with existing methods, it can track more general and complex spatial weld seams. First, the tracking pose of the robot was modeled using a proposed dual-sequence tracking strategy. On this basis, the working parameters can be adjusted to avoid robot-workpiece collision around the workpiece corners during tracking processes. By associating the forward direction of a welding torch with the viewpoint direction of a camera, the problem that the weld seam feature points are prone to lost during tracking processes can be solved by conventional methods. Point cloud registration is adopted to locate the multi-segment weld seams in the workpiece globally because the system deployment location is not fixed. Various experiments on single or multiple weld seams under different environmental conditions show that even if the robot was deployed in different positions, it could reach the initial points of the seams smoothly and accurately track along them.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Nikolov M (2014) Trends in development of weld overlaying during the 21 century. Acta Technologica Agriculturae 17(2):35–38CrossRef Nikolov M (2014) Trends in development of weld overlaying during the 21 century. Acta Technologica Agriculturae 17(2):35–38CrossRef
2.
go back to reference Xu Y, Yu H, Zhong J, Lin T, Chen S (2012) Real-time seam tracking control technology during welding robot GTAW process based on passive vision sensor. J Mater Process Technol 212(8):1654–1662CrossRef Xu Y, Yu H, Zhong J, Lin T, Chen S (2012) Real-time seam tracking control technology during welding robot GTAW process based on passive vision sensor. J Mater Process Technol 212(8):1654–1662CrossRef
3.
go back to reference Zou Y, Chen J, Wei X (2020) Research on a real-time pose estimation method for a seam tracking system. Opt Lasers Eng 127:105947CrossRef Zou Y, Chen J, Wei X (2020) Research on a real-time pose estimation method for a seam tracking system. Opt Lasers Eng 127:105947CrossRef
4.
go back to reference De Graaf M, Aarts R, Jonker B, Meijer J (2010) Real-time seam tracking for robotic laser welding using trajectory-based control. Control Eng Pract 18(8):944–953CrossRef De Graaf M, Aarts R, Jonker B, Meijer J (2010) Real-time seam tracking for robotic laser welding using trajectory-based control. Control Eng Pract 18(8):944–953CrossRef
5.
go back to reference Xu Y, Lv N, Fang G, Du S, Zhao W, Ye Z, Chen S (2017) Welding seam tracking in robotic gas metal arc welding. J Mater Process Technol 248:18–30CrossRef Xu Y, Lv N, Fang G, Du S, Zhao W, Ye Z, Chen S (2017) Welding seam tracking in robotic gas metal arc welding. J Mater Process Technol 248:18–30CrossRef
6.
go back to reference Moon HS, Ko SH, Kim JC (2009) Automatic seam tracking in pipeline welding with narrow groove. Int J Adv Manuf Technol 41(3–4):234–241CrossRef Moon HS, Ko SH, Kim JC (2009) Automatic seam tracking in pipeline welding with narrow groove. Int J Adv Manuf Technol 41(3–4):234–241CrossRef
7.
go back to reference Regaard B, Kaierle S, Poprawe R (2009) Seam-tracking for high precision laser welding applications–methods, restrictions and enhanced concepts. J Laser Appl 21(4):183–195 Regaard B, Kaierle S, Poprawe R (2009) Seam-tracking for high precision laser welding applications–methods, restrictions and enhanced concepts. J Laser Appl 21(4):183–195
8.
go back to reference Li G, Hong Y, Gao J, Hong B, Li X (2020) Welding seam trajectory recognition for automated skip welding guidance of a spatially intermittent welding seam based on laser vision sensor. Sensors 20(13):3657CrossRef Li G, Hong Y, Gao J, Hong B, Li X (2020) Welding seam trajectory recognition for automated skip welding guidance of a spatially intermittent welding seam based on laser vision sensor. Sensors 20(13):3657CrossRef
9.
go back to reference Xu Y, Wang Z (2021) Visual sensing technologies in robotic welding: recent research developments and future interests. Sens Actuators A Phys p 112551 Xu Y, Wang Z (2021) Visual sensing technologies in robotic welding: recent research developments and future interests. Sens Actuators A Phys p 112551
10.
go back to reference Rout A, Deepak B, Biswal B (2019) Advances in weld seam tracking techniques for robotic welding: a review. Robot Comput Integr Manuf 56:12–37CrossRef Rout A, Deepak B, Biswal B (2019) Advances in weld seam tracking techniques for robotic welding: a review. Robot Comput Integr Manuf 56:12–37CrossRef
11.
go back to reference Lee GY, Oh MS, Kim SB (2003) Development of a high speed rotating arc sensor system for tracking complicate curved fillet welding lines. Int J Precis Eng Manuf 4(6):20–28 Lee GY, Oh MS, Kim SB (2003) Development of a high speed rotating arc sensor system for tracking complicate curved fillet welding lines. Int J Precis Eng Manuf 4(6):20–28
12.
go back to reference Liu W, Li L, Hong Y, Yue J (2017) Linear mathematical model for seam tracking with an arc sensor in P-GMAW processes. Sensors 17(3):591CrossRef Liu W, Li L, Hong Y, Yue J (2017) Linear mathematical model for seam tracking with an arc sensor in P-GMAW processes. Sensors 17(3):591CrossRef
13.
go back to reference Mao Zw, Pan Jl, Zhang H (2010) Mobile welding robot system based on rotating arc sensor applied for large fillet welding seam tracking. In: 2010 Sixth International Conference on Natural Computation, IEEE 1:394–397 Mao Zw, Pan Jl, Zhang H (2010) Mobile welding robot system based on rotating arc sensor applied for large fillet welding seam tracking. In: 2010 Sixth International Conference on Natural Computation, IEEE 1:394–397
14.
go back to reference Ushio M, Mao W (1994) Sensors for arc welding: advantages and limitations. Trans JWRI 23(2):135–141 Ushio M, Mao W (1994) Sensors for arc welding: advantages and limitations. Trans JWRI 23(2):135–141
15.
go back to reference Xue K, Wang Z, Shen J, Hu S, Zhen Y, Liu J, Wu D, Yang H (2021) Robotic seam tracking system based on vision sensing and human-machine interaction for multi-pass mag welding. J Manuf Process 63:48–59CrossRef Xue K, Wang Z, Shen J, Hu S, Zhen Y, Liu J, Wu D, Yang H (2021) Robotic seam tracking system based on vision sensing and human-machine interaction for multi-pass mag welding. J Manuf Process 63:48–59CrossRef
16.
go back to reference Dinham M, Fang G (2013) Autonomous weld seam identification and localisation using eye-in-hand stereo vision for robotic arc welding. Robot Comput Integr Manuf 29(5):288–301CrossRef Dinham M, Fang G (2013) Autonomous weld seam identification and localisation using eye-in-hand stereo vision for robotic arc welding. Robot Comput Integr Manuf 29(5):288–301CrossRef
17.
go back to reference Shen H, Lin T, Chen S, Li L (2010) Real-time seam tracking technology of welding robot with visual sensing. J Intell Robot Syst 59(3):283–298MATHCrossRef Shen H, Lin T, Chen S, Li L (2010) Real-time seam tracking technology of welding robot with visual sensing. J Intell Robot Syst 59(3):283–298MATHCrossRef
18.
go back to reference Nilsen M, Sikström F, Christiansson AK, Ancona A (2017) Vision and spectroscopic sensing for joint tracing in narrow gap laser butt welding. Opt Laser Technol 96:107–116CrossRef Nilsen M, Sikström F, Christiansson AK, Ancona A (2017) Vision and spectroscopic sensing for joint tracing in narrow gap laser butt welding. Opt Laser Technol 96:107–116CrossRef
19.
go back to reference Zhang Z, Wen G, Chen S (2019) Weld image deep learning-based on-line defects detection using convolutional neural networks for Al alloy in robotic arc welding. J Manuf Process 45:208–216CrossRef Zhang Z, Wen G, Chen S (2019) Weld image deep learning-based on-line defects detection using convolutional neural networks for Al alloy in robotic arc welding. J Manuf Process 45:208–216CrossRef
20.
go back to reference Bračun D, Sluga A (2015) Stereo vision based measuring system for online welding path inspection. J Mater Process Technol 223:328–336CrossRef Bračun D, Sluga A (2015) Stereo vision based measuring system for online welding path inspection. J Mater Process Technol 223:328–336CrossRef
21.
go back to reference Li X, Li X, Ge SS, Khyam MO, Luo C (2017) Automatic welding seam tracking and identification. IEEE Trans Ind Electron 64(9):7261–7271CrossRef Li X, Li X, Ge SS, Khyam MO, Luo C (2017) Automatic welding seam tracking and identification. IEEE Trans Ind Electron 64(9):7261–7271CrossRef
22.
go back to reference Yang L, Liu Y, Peng J, Liang Z (2020) A novel system for off-line 3D seam extraction and path planning based on point cloud segmentation for arc welding robot. Robot Comput Integr Manuf 64:101929 Yang L, Liu Y, Peng J, Liang Z (2020) A novel system for off-line 3D seam extraction and path planning based on point cloud segmentation for arc welding robot. Robot Comput Integr Manuf 64:101929
23.
go back to reference Fang Z, Xu D, Tan M (2010) A vision-based self-tuning fuzzy controller for fillet weld seam tracking. IEEE/ASME Trans Mechatron 16(3):540–550CrossRef Fang Z, Xu D, Tan M (2010) A vision-based self-tuning fuzzy controller for fillet weld seam tracking. IEEE/ASME Trans Mechatron 16(3):540–550CrossRef
24.
go back to reference Zou Y, Chen T, Chen X, Li J (2022) Robotic seam tracking system combining convolution filter and deep reinforcement learning. Mech Syst Signal Process 165:108372CrossRef Zou Y, Chen T, Chen X, Li J (2022) Robotic seam tracking system combining convolution filter and deep reinforcement learning. Mech Syst Signal Process 165:108372CrossRef
25.
go back to reference Xu P, Tang X, Yao S (2008) Application of circular laser vision sensor (CLVS) on welded seam tracking. J Mater Process Technol 205(1–3):404–410 Xu P, Tang X, Yao S (2008) Application of circular laser vision sensor (CLVS) on welded seam tracking. J Mater Process Technol 205(1–3):404–410
26.
go back to reference Fan J, Deng S, Jing F, Zhou C, Yang L, Long T, Tan M (2019) An initial point alignment and seam-tracking system for narrow weld. IEEE Trans Ind Inf 16(2):877–886CrossRef Fan J, Deng S, Jing F, Zhou C, Yang L, Long T, Tan M (2019) An initial point alignment and seam-tracking system for narrow weld. IEEE Trans Ind Inf 16(2):877–886CrossRef
27.
go back to reference Liu C, Wang H, Huang Y, Rong Y, Meng J, Li G, Zhang G (2022) Welding seam recognition and tracking for a novel mobile welding robot based on multi-layer sensing strategy. Meas Sci Technol 33(5):055109CrossRef Liu C, Wang H, Huang Y, Rong Y, Meng J, Li G, Zhang G (2022) Welding seam recognition and tracking for a novel mobile welding robot based on multi-layer sensing strategy. Meas Sci Technol 33(5):055109CrossRef
28.
go back to reference Yan Z, Xu D, Li Y (2008) A visual servoing system for the torch alignment to initial welding position. In: International Conference on Intelligent Robotics and Applications, Springer, pp 697–706 Yan Z, Xu D, Li Y (2008) A visual servoing system for the torch alignment to initial welding position. In: International Conference on Intelligent Robotics and Applications, Springer, pp 697–706
29.
go back to reference Sc Wei, Wang J, Lin T, Sb Chen (2012) Application of image morphology in detecting and extracting the initial welding position. Journal of Shanghai Jiaotong University (Science) 17(3):323–326CrossRef Sc Wei, Wang J, Lin T, Sb Chen (2012) Application of image morphology in detecting and extracting the initial welding position. Journal of Shanghai Jiaotong University (Science) 17(3):323–326CrossRef
30.
go back to reference Chen XZ, Chen SB (2010) The autonomous detection and guiding of start welding position for arc welding robot. Ind Robot 37(1):70–78CrossRef Chen XZ, Chen SB (2010) The autonomous detection and guiding of start welding position for arc welding robot. Ind Robot 37(1):70–78CrossRef
31.
go back to reference Ma Y, Fan J, Deng S, Luo Y, Ma X, Jing F, Tan M (2021) Efficient and accurate start point guiding and seam tracking method for curve weld based on structure light. IEEE Trans Instrum Meas 70:1–10 Ma Y, Fan J, Deng S, Luo Y, Ma X, Jing F, Tan M (2021) Efficient and accurate start point guiding and seam tracking method for curve weld based on structure light. IEEE Trans Instrum Meas 70:1–10
32.
go back to reference Wang N, Shi X, Zhang X (2017) Recognition of initial welding position based on structured-light for arc welding robot. In: International Conference on Intelligent Robotics and Applications, Springer, pp 564–575 Wang N, Shi X, Zhang X (2017) Recognition of initial welding position based on structured-light for arc welding robot. In: International Conference on Intelligent Robotics and Applications, Springer, pp 564–575
33.
go back to reference Zhang L, Xu Y, Du S, Zhao W, Hou Z, Chen S (2018) Point cloud based three-dimensional reconstruction and identification of initial welding position. Transactions on Intelligent Welding Manufacturing. Springer, Singapore, pp 61–77CrossRef Zhang L, Xu Y, Du S, Zhao W, Hou Z, Chen S (2018) Point cloud based three-dimensional reconstruction and identification of initial welding position. Transactions on Intelligent Welding Manufacturing. Springer, Singapore, pp 61–77CrossRef
34.
go back to reference Kim J, Lee J, Chung M, Shin YG (2021) Multiple weld seam extraction from RGB-depth images for automatic robotic welding via point cloud registration. Multimed Tools Appl 80(6):9703–9719 Kim J, Lee J, Chung M, Shin YG (2021) Multiple weld seam extraction from RGB-depth images for automatic robotic welding via point cloud registration. Multimed Tools Appl 80(6):9703–9719
37.
go back to reference Feng X, Gao L, Tian W, Wei R, Wang Z, Chen Y (2020) Application of wall climbing welding robot in automatic welding of island spherical tank. J Coast Res 107(SI):1–4 Feng X, Gao L, Tian W, Wei R, Wang Z, Chen Y (2020) Application of wall climbing welding robot in automatic welding of island spherical tank. J Coast Res 107(SI):1–4
38.
go back to reference Yanfeng G, Hua Z, Yanhui Y (2011) Back-stepping and neural network control of a mobile robot for curved weld seam tracking. Procedia Engineering 15:38–44CrossRef Yanfeng G, Hua Z, Yanhui Y (2011) Back-stepping and neural network control of a mobile robot for curved weld seam tracking. Procedia Engineering 15:38–44CrossRef
39.
go back to reference Wang Z, Zhang K, Chen Y, Luo Z, Zheng J (2017) A real-time weld line detection for derusting wall-climbing robot using dual cameras. J Manuf Process 27:76–86CrossRef Wang Z, Zhang K, Chen Y, Luo Z, Zheng J (2017) A real-time weld line detection for derusting wall-climbing robot using dual cameras. J Manuf Process 27:76–86CrossRef
40.
go back to reference Zhang L, Sun J, Yin G, Zhao J, Han Q (2015) A cross structured light sensor and stripe segmentation method for visual tracking of a wall climbing robot. Sensors 15(6):13725–13751CrossRef Zhang L, Sun J, Yin G, Zhao J, Han Q (2015) A cross structured light sensor and stripe segmentation method for visual tracking of a wall climbing robot. Sensors 15(6):13725–13751CrossRef
41.
go back to reference Kermorgant O (2018) A magnetic climbing robot to perform autonomous welding in the shipbuilding industry. Robot Comput Integr Manuf 53:178–186CrossRef Kermorgant O (2018) A magnetic climbing robot to perform autonomous welding in the shipbuilding industry. Robot Comput Integr Manuf 53:178–186CrossRef
42.
go back to reference Zhang L, Ke W, Ye Q, Jiao J (2014) A novel laser vision sensor for weld line detection on wall-climbing robot. Opt Laser Technol 60:69–79CrossRef Zhang L, Ke W, Ye Q, Jiao J (2014) A novel laser vision sensor for weld line detection on wall-climbing robot. Opt Laser Technol 60:69–79CrossRef
43.
go back to reference Corke P (2017) Robotics, vision and control: fundamental algorithms in MATLAB® second, completely revised, vol 118 Corke P (2017) Robotics, vision and control: fundamental algorithms in MATLAB® second, completely revised, vol 118
44.
go back to reference Yu S, Hong J, Zhang T, Yang Z, Guan Y (2020) A self-correction based algorithm for single-shot camera calibration. In: International Conference on Intelligent Robotics and Applications, Springer, pp 442–455 Yu S, Hong J, Zhang T, Yang Z, Guan Y (2020) A self-correction based algorithm for single-shot camera calibration. In: International Conference on Intelligent Robotics and Applications, Springer, pp 442–455
45.
go back to reference Idrobo-Pizo GA, Motta JMS, Sampaio RC (2019) A calibration method for a laser triangulation scanner mounted on a robot arm for surface mapping. Sensors 19(8):1783CrossRef Idrobo-Pizo GA, Motta JMS, Sampaio RC (2019) A calibration method for a laser triangulation scanner mounted on a robot arm for surface mapping. Sensors 19(8):1783CrossRef
46.
go back to reference Yanbiao Zou, Xiangzhi Chen (2018) Hand-eye calibration of arc welding robot and laser vision sensor through semidefinite programming. Robot Comput Integr Manuf 45(5):597–610 Yanbiao Zou, Xiangzhi Chen (2018) Hand-eye calibration of arc welding robot and laser vision sensor through semidefinite programming. Robot Comput Integr Manuf 45(5):597–610
47.
go back to reference Xiao R, Xu Y, Hou Z, Chen C, Chen S (2021) An automatic calibration algorithm for laser vision sensor in robotic autonomous welding system. J Intell Manuf pp 1–14 Xiao R, Xu Y, Hou Z, Chen C, Chen S (2021) An automatic calibration algorithm for laser vision sensor in robotic autonomous welding system. J Intell Manuf pp 1–14
48.
go back to reference HICKS J (1997) Chapter 5 - weld preparations. In: HICKS J (ed) Welded joint design (second edition), second edition edn, Woodhead Publishing, Cambridge England, pp 29–38 HICKS J (1997) Chapter 5 - weld preparations. In: HICKS J (ed) Welded joint design (second edition), second edition edn, Woodhead Publishing, Cambridge England, pp 29–38
49.
go back to reference Li J, Li H, Wei H, Gao Y (2016) Effect of torch position and angle on welding quality and welding process stability in pulse on pulse MIG welding-brazing of aluminum alloy to stainless steel. Int J Adv Manuf Technol 84(1–4):705–716 Li J, Li H, Wei H, Gao Y (2016) Effect of torch position and angle on welding quality and welding process stability in pulse on pulse MIG welding-brazing of aluminum alloy to stainless steel. Int J Adv Manuf Technol 84(1–4):705–716
50.
go back to reference Teeravarunyou S, Poopatb B (2009) Computer based welding training system. International Journal of Industrial Engineering 16(2):116–125 Teeravarunyou S, Poopatb B (2009) Computer based welding training system. International Journal of Industrial Engineering 16(2):116–125
51.
go back to reference Shultz EF, Cole EG, Smith CB, Zinn MR, Ferrier NJ, Pfefferkorn FE (2010) Effect of compliance and travel angle on friction stir welding with gaps. J Manuf Sci Eng 132(4) Shultz EF, Cole EG, Smith CB, Zinn MR, Ferrier NJ, Pfefferkorn FE (2010) Effect of compliance and travel angle on friction stir welding with gaps. J Manuf Sci Eng 132(4)
52.
go back to reference Besl PJ, Mckay HD (1992) A method for registration of 3-D shapes. IEEE Trans Pattern Anal Mach Intell 14(2):239–256 Besl PJ, Mckay HD (1992) A method for registration of 3-D shapes. IEEE Trans Pattern Anal Mach Intell 14(2):239–256
53.
go back to reference Rusu RB, Cousins S (2011) 3D is here: Point Cloud Library (PCL). In: IEEE International Conference on Robotics and Automation (ICRA), IEEE, Shanghai, China Rusu RB, Cousins S (2011) 3D is here: Point Cloud Library (PCL). In: IEEE International Conference on Robotics and Automation (ICRA), IEEE, Shanghai, China
54.
go back to reference Fang Z, Xu D, Tan M (2013) Vision-based initial weld point positioning using the geometric relationship between two seams. Int J Adv Manuf Technol 66(9–12):1535–1543CrossRef Fang Z, Xu D, Tan M (2013) Vision-based initial weld point positioning using the geometric relationship between two seams. Int J Adv Manuf Technol 66(9–12):1535–1543CrossRef
Metadata
Title
Multiseam tracking with a portable robotic welding system in unstructured environments
Authors
Shuangfei Yu
Yisheng Guan
Zhi Yang
Chutian Liu
Jiacheng Hu
Jie Hong
Haifei Zhu
Tao Zhang
Publication date
31-08-2022
Publisher
Springer London
Published in
The International Journal of Advanced Manufacturing Technology / Issue 3-4/2022
Print ISSN: 0268-3768
Electronic ISSN: 1433-3015
DOI
https://doi.org/10.1007/s00170-022-10019-3

Other articles of this Issue 3-4/2022

The International Journal of Advanced Manufacturing Technology 3-4/2022 Go to the issue

Premium Partners