Skip to main content
Erschienen in: The International Journal of Advanced Manufacturing Technology 1-4/2019

30.05.2019 | ORIGINAL ARTICLE

A pose estimation system based on deep neural network and ICP registration for robotic spray painting application

verfasst von: Zhe Wang, Junfeng Fan, Fengshui Jing, Zhaoyang Liu, Min Tan

Erschienen in: The International Journal of Advanced Manufacturing Technology | Ausgabe 1-4/2019

Einloggen

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Nowadays, off-line robot trajectory generation methods based on pre-scanned target model are highly desirable for robotic spray painting application. For actual implementation of the generated trajectory, the relative pose between the actual target and the model needs to be calibrated in the first place. However, obtaining this relative pose remains a challenge, especially from a safe distance in industrial setting. In this paper, a pose estimation system that is able to meet the robotic spray painting requirements is proposed to estimate the pose accurately. The system captures the image of the target using RGB-D vision sensor. The image is then segmented using a modified U-SegNet segmentation network and the resulting segmentation is registered with the pre-scanned model candidates using iterative closest point (ICP) registration to obtain the estimated pose. To strengthen the robustness, a deep convolutional neural network is proposed to determine the rough orientation of the target and guide the selection of model candidates accordingly thus preventing misalignment during registration. The experimental results are compared with relevant researches and validate the accuracy and effectiveness of the proposed system.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Zhang BB, Wu J, Wang LP, Yu ZY, Fu P (2018) A method to realize accurate dynamic feedforward control of a spray-painting robot for airplane wings. Ieee-Asme T Mech 23(3):1182–1192 Zhang BB, Wu J, Wang LP, Yu ZY, Fu P (2018) A method to realize accurate dynamic feedforward control of a spray-painting robot for airplane wings. Ieee-Asme T Mech 23(3):1182–1192
2.
Zurück zum Zitat Ren SN, Xie Y, Yang XD, Xu J, Wang GL, Chen K (2017) A method for optimizing the base position of mobile painting manipulators. IEEE T Autom Sci Eng 14(1):370–375CrossRef Ren SN, Xie Y, Yang XD, Xu J, Wang GL, Chen K (2017) A method for optimizing the base position of mobile painting manipulators. IEEE T Autom Sci Eng 14(1):370–375CrossRef
3.
Zurück zum Zitat Trigatti G, Boscariol P, Scalera L, Pillan D, Gasparetto A (2018) A new path-constrained trajectory planning strategy for spray painting robots - rev.1. Int J Adv Manuf Tech 98(9–12):2287–2296CrossRef Trigatti G, Boscariol P, Scalera L, Pillan D, Gasparetto A (2018) A new path-constrained trajectory planning strategy for spray painting robots - rev.1. Int J Adv Manuf Tech 98(9–12):2287–2296CrossRef
4.
Zurück zum Zitat Chen HP, Xi N (2008) Automated tool trajectory planning of industrial robots for painting composite surfaces. Int J Adv Manuf Tech 35(7–8):680–696CrossRef Chen HP, Xi N (2008) Automated tool trajectory planning of industrial robots for painting composite surfaces. Int J Adv Manuf Tech 35(7–8):680–696CrossRef
5.
Zurück zum Zitat Andulkar MV, Chiddarwar SS, Marathe AS (2015) Novel integrated offline trajectory generation approach for robot assisted spray painting operation. J Manuf Syst 37:201–216CrossRef Andulkar MV, Chiddarwar SS, Marathe AS (2015) Novel integrated offline trajectory generation approach for robot assisted spray painting operation. J Manuf Syst 37:201–216CrossRef
6.
Zurück zum Zitat Wang G, Cheng J, Li R, Chen K (2015) A new point cloud slicing based path planning algorithm for robotic spray painting. In: IEEE international conference on robotics and biomimetics, pp 1717–1722 Wang G, Cheng J, Li R, Chen K (2015) A new point cloud slicing based path planning algorithm for robotic spray painting. In: IEEE international conference on robotics and biomimetics, pp 1717–1722
7.
Zurück zum Zitat Chen H, Fuhlbrigge T, Li X (2008) Automated industrial robot path planning for spray painting process: a review. In: IEEE international conference on automation science and engineering, pp 522–527 Chen H, Fuhlbrigge T, Li X (2008) Automated industrial robot path planning for spray painting process: a review. In: IEEE international conference on automation science and engineering, pp 522–527
8.
Zurück zum Zitat Kharidege A, Ting D, Yajun Z (2017) A practical approach for automated polishing system of free-form surface path generation based on industrial arm robot. Int J Adv Manuf Tech 93(9–12):3921–3934CrossRef Kharidege A, Ting D, Yajun Z (2017) A practical approach for automated polishing system of free-form surface path generation based on industrial arm robot. Int J Adv Manuf Tech 93(9–12):3921–3934CrossRef
9.
Zurück zum Zitat Chen R, Wang GL, Zhao JG, Xu J, Chen K (2018) Fringe pattern based plane-to-plane visual servoing for robotic spray path planning. IEEE-Asme T Mech 23(3):1083–1091CrossRef Chen R, Wang GL, Zhao JG, Xu J, Chen K (2018) Fringe pattern based plane-to-plane visual servoing for robotic spray path planning. IEEE-Asme T Mech 23(3):1083–1091CrossRef
10.
Zurück zum Zitat Xu Z, He W, Yuan K (2011) A real-time position and posture measurement device for painting robot. In: International conference on electric information and control engineering, pp 1942–194 Xu Z, He W, Yuan K (2011) A real-time position and posture measurement device for painting robot. In: International conference on electric information and control engineering, pp 1942–194
11.
Zurück zum Zitat Lin CY, Abebe ZA, Chang SH (2015) Advanced spraying task strategy for bicycle-frame based on geometrical data of workpiece. In: International conference on advanced robotics, pp 277– 282 Lin CY, Abebe ZA, Chang SH (2015) Advanced spraying task strategy for bicycle-frame based on geometrical data of workpiece. In: International conference on advanced robotics, pp 277– 282
12.
Zurück zum Zitat Lin W, Anwar A, Li Z, Tong M, Qiu J, Gao H (2019) Recognition and pose estimation of auto parts for an autonomous spray painting robot. IEEE T Ind Inform 15(3):1709–1719CrossRef Lin W, Anwar A, Li Z, Tong M, Qiu J, Gao H (2019) Recognition and pose estimation of auto parts for an autonomous spray painting robot. IEEE T Ind Inform 15(3):1709–1719CrossRef
13.
Zurück zum Zitat Besl PJ, Mckay ND (1992) A method for registration of 3-D shapes. IEEE T Pattern Anal 14(2):239–256CrossRef Besl PJ, Mckay ND (1992) A method for registration of 3-D shapes. IEEE T Pattern Anal 14(2):239–256CrossRef
14.
Zurück zum Zitat Hodan T, Zabulis X, Lourakis M, Obdrzalek S, Matas J (2015) Detection and fine 3D pose estimation of texture-less objects in RGB-D images. In: IEEE/RSJ international conference on intelligent robots and systems, pp 4421–4428 Hodan T, Zabulis X, Lourakis M, Obdrzalek S, Matas J (2015) Detection and fine 3D pose estimation of texture-less objects in RGB-D images. In: IEEE/RSJ international conference on intelligent robots and systems, pp 4421–4428
15.
Zurück zum Zitat Schwarz M, Schulz H, Behnke S (2015) RGB-D object recognition and pose estimation based on pre-trained convolutional neural network features. In: 2015 IEEE international conference on robotics and automation, pp 1329–1335 Schwarz M, Schulz H, Behnke S (2015) RGB-D object recognition and pose estimation based on pre-trained convolutional neural network features. In: 2015 IEEE international conference on robotics and automation, pp 1329–1335
17.
Zurück zum Zitat Collet A, Martinez M, Srinivasa S (2011) The MOPED framework: object recognition and pose estimation for manipulation. I J Robot Res 30(10):1284–1306CrossRef Collet A, Martinez M, Srinivasa S (2011) The MOPED framework: object recognition and pose estimation for manipulation. I J Robot Res 30(10):1284–1306CrossRef
18.
Zurück zum Zitat Zeng A, Yu K, Song S, Suo D, Walker E, Rodriguez A, Xiao J (2017) Multi-view self-supervised deep learning for 6D pose estimation in the Amazon picking challenge. In: 2017 IEEE international conference on robotics and automation, pp 1386–1383 Zeng A, Yu K, Song S, Suo D, Walker E, Rodriguez A, Xiao J (2017) Multi-view self-supervised deep learning for 6D pose estimation in the Amazon picking challenge. In: 2017 IEEE international conference on robotics and automation, pp 1386–1383
19.
Zurück zum Zitat Shelhamer E, Long J, Darrell T (2017) Fully convolutional networks for semantic segmentation. IEEE T Pattern Anal 39(4):640–651CrossRef Shelhamer E, Long J, Darrell T (2017) Fully convolutional networks for semantic segmentation. IEEE T Pattern Anal 39(4):640–651CrossRef
20.
Zurück zum Zitat Wong JM, Kee V, Le T, Wagner S, Mariottini GL, Schneider A, Hamilton L, Chipalkatty R, Hebert M, Johnson DMS (2017) SegICP: integrated deep semantic segmentation and pose estimation. In: 2017 IEEE/RSJ international conference on intelligent robots and systems, pp 5784–5789 Wong JM, Kee V, Le T, Wagner S, Mariottini GL, Schneider A, Hamilton L, Chipalkatty R, Hebert M, Johnson DMS (2017) SegICP: integrated deep semantic segmentation and pose estimation. In: 2017 IEEE/RSJ international conference on intelligent robots and systems, pp 5784–5789
21.
Zurück zum Zitat Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE T Pattern Anal 39(12):2481–2495CrossRef Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE T Pattern Anal 39(12):2481–2495CrossRef
22.
Zurück zum Zitat Lin CM, Tsai CY, Lai YC, Li SA, Wong CC (2018) Visual object recognition and pose estimation based on a deep semantic segmentation network. IEEE Sens J 18(22):9370–9381CrossRef Lin CM, Tsai CY, Lai YC, Li SA, Wong CC (2018) Visual object recognition and pose estimation based on a deep semantic segmentation network. IEEE Sens J 18(22):9370–9381CrossRef
23.
Zurück zum Zitat Yang G, Wang S, Yang J, Shen B (2018) Active pose estimation of daily objects. In: 2018 IEEE international conference on mechatronics and automation, pp 837–842 Yang G, Wang S, Yang J, Shen B (2018) Active pose estimation of daily objects. In: 2018 IEEE international conference on mechatronics and automation, pp 837–842
24.
Zurück zum Zitat Kumar P, Nagar P, Arora C, Gupta A (2018) U-Segnet: fully convolutional neural network based automated brain tissue segmentation tool. In: 2018 25th IEEE international conference on image processing, pp 3503–3507 Kumar P, Nagar P, Arora C, Gupta A (2018) U-Segnet: fully convolutional neural network based automated brain tissue segmentation tool. In: 2018 25th IEEE international conference on image processing, pp 3503–3507
26.
Zurück zum Zitat Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention – MICCAI, pp 234–241 Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention – MICCAI, pp 234–241
27.
Zurück zum Zitat He K, Gkioxari G, Dollár P, Girshick R (2017) Mask R-CNN. In: IEEE international conference on computer vision, pp 2980–2988 He K, Gkioxari G, Dollár P, Girshick R (2017) Mask R-CNN. In: IEEE international conference on computer vision, pp 2980–2988
28.
Zurück zum Zitat Deng J, Dong W, Socher R, Li LJ, Li K, Li FF (2009) ImageNet: a large-scale hierarchical image database. In: IEEE conference on computer vision and pattern recognition, pp 248–255 Deng J, Dong W, Socher R, Li LJ, Li K, Li FF (2009) ImageNet: a large-scale hierarchical image database. In: IEEE conference on computer vision and pattern recognition, pp 248–255
29.
Zurück zum Zitat Everingham M, Van Gool L, Williams CKI, Winn J, Zisserman A (2010) The pascal visual object classes (VOC) challenge. Int J Comput Vis 88(2):303–338CrossRef Everingham M, Van Gool L, Williams CKI, Winn J, Zisserman A (2010) The pascal visual object classes (VOC) challenge. Int J Comput Vis 88(2):303–338CrossRef
30.
Zurück zum Zitat Russell BC, Torralba A, Murphy KP, Freeman WT (2008) LabelMe: a database and web-based tool for image annotation. Int J Comput Vis 77(1–3):157–173CrossRef Russell BC, Torralba A, Murphy KP, Freeman WT (2008) LabelMe: a database and web-based tool for image annotation. Int J Comput Vis 77(1–3):157–173CrossRef
31.
Zurück zum Zitat Sutskever I, Martens J, Dahl G, Hinton G (2013) On the importance of initialization and momentum in deep learning. In: International conference on machine learning, pp 1139–1147 Sutskever I, Martens J, Dahl G, Hinton G (2013) On the importance of initialization and momentum in deep learning. In: International conference on machine learning, pp 1139–1147
32.
Zurück zum Zitat Rusu RB, Cousins S (2011) 3D is here: point cloud library (PCL). In: 2011 IEEE international conference on robotics and automation, pp 1–4 Rusu RB, Cousins S (2011) 3D is here: point cloud library (PCL). In: 2011 IEEE international conference on robotics and automation, pp 1–4
Metadaten
Titel
A pose estimation system based on deep neural network and ICP registration for robotic spray painting application
verfasst von
Zhe Wang
Junfeng Fan
Fengshui Jing
Zhaoyang Liu
Min Tan
Publikationsdatum
30.05.2019
Verlag
Springer London
Erschienen in
The International Journal of Advanced Manufacturing Technology / Ausgabe 1-4/2019
Print ISSN: 0268-3768
Elektronische ISSN: 1433-3015
DOI
https://doi.org/10.1007/s00170-019-03901-0

Weitere Artikel der Ausgabe 1-4/2019

The International Journal of Advanced Manufacturing Technology 1-4/2019 Zur Ausgabe

    Marktübersichten

    Die im Laufe eines Jahres in der „adhäsion“ veröffentlichten Marktübersichten helfen Anwendern verschiedenster Branchen, sich einen gezielten Überblick über Lieferantenangebote zu verschaffen.