Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
The International Journal of Advanced Manufacturing Technology
Published in: The International Journal of Advanced Manufacturing Technology 12/2024

05-03-2024 | ORIGINAL ARTICLE

An automatic feature point extraction method based on laser vision for robotic multi-layer multi-pass weld seam tracking

Authors: Fengjing Xu, Lei He, Zhen Hou, Runquan Xiao, Tianyi Zuo, Jiacheng Li, Yanling Xu, Huajun Zhang

Published in: The International Journal of Advanced Manufacturing Technology | Issue 12/2024

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Laser vision-based seam tracking has been an important research hotspot in modern welding manufacturing. However, severe noise interference during welding and the complex contour curves of filling welds hinder the development of high-precision seam tracking in multi-layer multi-pass (MLMP) welding. To solve this problem, a point distribution model (PDM) has been implemented to express the laser stripe pattern of MLMP welds. Then, an end-to-end feature point extraction algorithm is proposed. The “coarse-to-fine” positioning strategy achieves global correlation and local constraints. The low-resolution heatmap regression and coordinate offset regression balance the efficiency and precision, where the backbone is improved with attention mechanisms. Furthermore, the soft coordinate loss and the Gaussian mixture model were combined to improve the generalization performance. Based on the model, an automatic ROI extraction method and output points filtering are implemented to complete the whole tracking process. In experiments, the proposed method achieved good tracking performance even under strong noise, with the mean absolute error (MAE) being controlled within 0.3 mm. The feature point extraction method shows advantages in both precision and stability, laying a foundation for advanced robotic MLMP welding production.
previous article Wear prediction model of hot rolling backup roll based on FEM + ML algorithm
next article Fabrication of high-performance lens arrays for micro-concentrator photovoltaics using ultraviolet imprinting

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now
Literature
1.
Xu FJ, Xiao RQ, Hou Z, Xu YL, Zhang HJ, Chen, SB (2021) Multi-layer multi-pass welding of medium thickness plate: technologies, advances and future prospects. transactions on intelligent welding manufacturing: Volume III No. 4 2019, 3–33.
2.
Xu FJ, Xu YL, Zhang HJ, Chen SB (2022) Application of sensing technology in intelligent robotic arc welding: a review. J Manuf Process 79:854–880CrossRef
3.
Wang NF, Zhong KF, Shi XD, Zhang XM (2020) A robust weld seam recognition method under heavy noise based on structured-light vision. Robotics and Computer-Integrated Manufacturing 61:101821CrossRef
4.
Zou YB, Chen XZ, Gong GJ, Li JC (2018) A seam tracking system based on a laser vision sensor. Measurement 127:489–500CrossRef
5.
Fan JF, Deng S, Ma YK, Zhou C, Jing FS, Tan M (2020) Seam feature point acquisition based on efficient convolution operator and particle filter in GMAW. IEEE Trans Industr Inf 17(2):1220–1230CrossRef
6.
Zou YB, Wei XZ, Chen JX (2020) Conditional generative adversarial network-based training image inpainting for laser vision seam tracking. Opt Lasers Eng 134:106140CrossRef
7.
Wu KY, Wang TQ, He JJ, Liu Y, Jia ZW (2020) Autonomous seam recognition and feature extraction for multi-pass welding based on laser stripe edge guidance network. The International Journal of Advanced Manufacturing Technology 111(9–10):2719–2731CrossRef
8.
He YS, Xu YL, Chen YX, Chen HB, Chen SB (2016) Weld seam profile detection and feature point extraction for multi-pass route planning based on visual attention model. Robotics and Computer-Integrated Manufacturing 37:251–261CrossRef
9.
Chen SF, Liu J, Chen B, Suo XY (2022) Universal fillet weld joint recognition and positioning for robot welding using structured light. Robotics and Computer-Integrated Manufacturing 74:102279CrossRef
10.
Xiao RQ, Xu YL, Hou Z, Chen C, Chen SB (2021) A feature extraction algorithm based on improved Snake model for multi-pass seam tracking in robotic arc welding. J Manuf Process 72:48–60CrossRef
11.
He YS, Ma GH, Chen SB (2021) Autonomous decision-making of welding position during multipass GMAW with T-joints: a Bayesian network approach. IEEE Trans Industr Electron 69(4):3909–3917CrossRef
12.
Zhao Z, Luo J, Wang YY, Bai LF, Han J (2021) Additive seam tracking technology based on laser vision. The International Journal of Advanced Manufacturing Technology 116(1–2):197–211CrossRef
13.
Yang GW, Wang YZ, Zhou N (2021) Detection of weld groove edge based on multilayer convolution neural network. Measurement 186:110129CrossRef
14.
Yang HL, Lyu JY, Cheng PJ, Tang XY (2021) Lddmm-face: large deformation diffeomorphic metric learning for flexible and consistent face alignment. arXiv preprint arXiv:​2108.​00690.
15.
Li JF, Bian SY, Zeng AL, Wang C, Pang B, Liu, WT, Lu CW (2021) Human pose regression with residual log-likelihood estimation. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 11025–11034).
16.
Saragih JM, Lucey S, Cohn JF (2011) Deformable model fitting by regularized landmark mean-shift. Int J Comput Vision 91:200–215MathSciNetCrossRef
17.
Xiao RQ, Xu YL, Hou Z, Chen C, Chen SB (2022) An automatic calibration algorithm for laser vision sensor in robotic autonomous welding system. Journal of Intelligent Manufacturing, 1–14.
18.
Milborrow S, Nicolls F (2008) Locating facial features with an extended active shape model. In Computer Vision–ECCV 2008: 10th European Conference on Computer Vision, Marseille, France, October 12–18, 2008, Proceedings, Part IV 10 (pp. 504–513). Springer Berlin Heidelberg.
19.
Geng ZG, Sun K, Xiao B, Zhang ZX, Wang JD (2021) Bottom-up human pose estimation via disentangled keypoint regression. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 14676–14686).
20.
Fard AP, Mahoor MH (2022) Facial landmark points detection using knowledge distillation-based neural networks. Comput Vis Image Underst 215:103316CrossRef
21.
Xu ZX, Li BH, Yuan Y, Geng M (2021, May) Anchorface: an anchor-based facial landmark detector across large poses. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 35, No. 4, pp. 3092–3100).
22.
Wang JD, Sun K, Cheng TH, Jiang BR, Deng CR, Zhao Y, Liu D, Mu YD, Tan MK, Wang XG, Liu WY, Xiao B (2020) Deep high-resolution representation learning for visual recognition. IEEE Trans Pattern Anal Mach Intell 43(10):3349–3364CrossRef
23.
Lan X, Hu QH, Chen Q, Xue J, Cheng J (2021) Hih: towards more accurate face alignment via heatmap in heatmap. arXiv preprint arXiv:​2104.​03100
24.
Xiong YL, Zhou ZJ, Dou YH, Su ZZ (2020) Gaussian vector: an efficient solution for facial landmark detection. In Proceedings of the Asian Conference on Computer Vision
25.
Jin HB, Liao SC, Shao L (2021) Pixel-in-pixel net: towards efficient facial landmark detection in the wild. Int J Comput Vision 129:3174–3194CrossRef
26.
He KM, Zhang XY, Ren SQ, Sun J (2016) Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770–778)
27.
Jin X, Xie YP, Wei XS, Zhao BR, Chen ZM, Tan XY (2022) Delving deep into spatial pooling for squeeze-and-excitation networks. Pattern Recogn 121:108159CrossRef
28.
Feng ZH., Kittler J, Awais M, Huber P, Wu XJ (2018) Wing loss for robust facial landmark localisation with convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2235–2245)
29.
Fard AP, Abdollahi H, Mahoor M (2021) ASMNet: a lightweight deep neural network for face alignment and pose estimation. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition (pp. 1521–1530)
30.
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:​1409.​1556
31.
Liu Z, Lin YT, Cao Y, Hu H, Wei YX, Zhang Z, Lin S, Guo BN (2021) Swin transformer: hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 10012–10022)
32.
Ren SQ, Cao XD, Wei YC, Sun J (2014) Face alignment at 3000 fps via regressing local binary features. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1685–1692)
Metadata
Title
An automatic feature point extraction method based on laser vision for robotic multi-layer multi-pass weld seam tracking
Authors
Fengjing Xu
Lei He
Zhen Hou
Runquan Xiao
Tianyi Zuo
Jiacheng Li
Yanling Xu
Huajun Zhang
Publication date
05-03-2024
Publisher
Springer London
Published in
The International Journal of Advanced Manufacturing Technology / Issue 12/2024
Print ISSN: 0268-3768
Electronic ISSN: 1433-3015
DOI
https://doi.org/10.1007/s00170-024-13245-z

Other articles of this Issue 12/2024

The International Journal of Advanced Manufacturing Technology 12/2024 Go to the issue

ORIGINAL ARTICLE

Efficient-Unet: Intelligent identification of abrasive grain on the entire surface of monolayer brazing wheel based on encoder–decoder network

ORIGINAL ARTICLE

Cutting chatter recognition based on spectrum characteristics and extreme gradient boosting

ORIGINAL ARTICLE

Modeling and experimental validation of the contact characteristics for laying during automated fiber placement

ORIGINAL ARTICLE

Advancing surface mount technology quality: a computer-assisted approach for enhanced X-ray inspection of solder joints

ORIGINAL ARTICLE

Wear prediction model of hot rolling backup roll based on FEM + ML algorithm

ORIGINAL ARTICLE

Tool-pin profile effects on thermal and material flow in friction stir butt welding of AA2219-T87 plates: computational fluid dynamics model development and study

Premium Partners

    Image Credits