nach oben

Erschienen in:

2020 | OriginalPaper | Buchkapitel

6. Learning from Visual-Based Teleoperation Demonstration

verfasst von : Bin Fang, Fuchun Sun, Huaping Liu, Chunfang Liu, Di Guo

Erschienen in: Wearable Technology for Robotic Manipulation and Learning

Verlag: Springer Singapore

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

This chapter proposes the deep neural networks to enhance visual teleoperation considering human–robot posture consistence. Firstly, the teacher–student network (TeachNet) is introduced, which is a novel neural network architecture for intuitive and markerless vision-based teleoperation of dexterous robotic hands. It is combined with a consistency loss function, which handles the differences in appearance and anatomy between human and robotic hands. Then the multi-stage structure of visual teleoperation network is designed for robotic arm. Finally, imitation experiments are carried out on the robots to demonstrate that the proposed visual-based posture-consistent teleoperation is effective and reliable.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Learning from Wearable-Based Teleoperation Demonstration

Nächstes Kapitel Learning from Wearable-Based Indirect Demonstration

Aracil R, Balaguer C, Buss M, Ferre M, Melchiorri C (2007) Book advances in telerobotics. Springer, Berlin. https://doi.org/10.1007/978-3-540-71364-7

Du G, Zhang P, Mai J, Li Z (2012) Markerless kinect-based hand tracking for robot teleoperation. Int J Adv Robot Syst 9:1. https://doi.org/10.5772/50093 CrossRef

Oikonomidis I, Kyriazis N, Argyros A (2011) Efficient model-based 3D tracking of hand articulations using kinect. In: British machine vision conference, vol 1. https://doi.org/10.5244/C.25.101

Pedro LM, Caurin GA, Belini VL, Pechoneri RD, Gonzaga A, Neto I, Nazareno F, Stucheli M (2011) Hand gesture recognition for robot hand teleoperation. In: Proceedings of the 21st international congress of mechanical engineering

Dogan E, Eren G, Wolf C, Lombardi E, Baskurt A (2017) Multi-view pose estimation with mixtures-of-parts and adaptive viewpoint selection. IET Comput Vis 12. https://doi.org/10.1049/iet-cvi.2017.0146 .

Wang H, Dai L, Cai Y, Sun X, Chen L (2018) Salient object detection based on multi-scale contrast. Neural Netw 101. https://doi.org/10.1016/j.neunet.2018.02.005

Rawat W, Wang Z (2017) Deep convolutional neural networks for image classification: a comprehensive review. Neural Comput 29:1–98. https://doi.org/10.1162/NECO_a_00990 MathSciNetCrossRef

Chen C, Jafari R, Kehtarnavaz N (2015) Action recognition from depth sequences using depth motion maps-based local binary patterns. In: Proceedings – 2015 IEEE winter conference on applications of computer vision, WACV 2015, pp 1092–1099. https://doi.org/10.1109/WACV.2015.150

Wang F, Jiang M, Qian C, Yang S, Li C, Zhang H, Wang X, Tang X (2017) Residual attention network for image classification. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 6450–6458. https://doi.org/10.1109/CVPR.2017.683

10.

Tome D, Russell C, Agapito L (2017) Lifting from the deep: convolutional 3D pose estimation from a single image. In: IEEE conference on computer vision and pattern recognition (CVPR). https://doi.org/10.1109/CVPR.2017.603

11.

Chen T-Y, Ting P-W, Wu M-Y, Fu L-C (2018) Learning a deep network with spherical part model for 3D hand pose estimation. Pattern Recogn 80. https://doi.org/10.1016/j.patcog.2018.02.029

12.

Chen C (2015) UTD-MHAD: a multimodal dataset for human action recognition utilizing a depth camera and a wearable inertial sensor. In: IEEE international conference on image processing (ICIP). https://doi.org/10.1109/ICIP.2015.7350781

13.

Ionescu C, Papava D, Olaru V, Sminchisescu C (2013) Human3.6M: large scale datasets and predictive methods for 3D human sensing in natural environments. IEEE Trans Pattern Anal Mach Intell 36. https://doi.org/10.1109/TPAMI.2013.248

14.

Cerulo I, Ficuciello F, Lippiello V, Siciliano B (2017) Teleoperation of the SCHUNK S5FH under-actuated anthropomorphic hand using human hand motion tracking. Robot Auton Syst. https://doi.org/10.1016/j.robot.2016.12.004

15.

Fang B, Sun F, Liu H, Guo D, Chen W, Yao G (2017) Robotic teleoperation systems using a wearable multimodal fusion device. Int J Adv Robot Syst 14:1–11. https://doi.org/10.1177/1729881417717057 CrossRef

16.

17.

Cho S, Jin H, Lee JM, Yao B (2010) Teleoperation of a mobile robot using a force-reflection joystick with sensing mechanism of rotating magnetic field. IEEE/ASME Trans Mechatron 15:17–26. https://doi.org/10.1109/TMECH.2009.2013848 CrossRef

18.

Stanton C, Bogdanovych A, Ratanasena E (2012) Teleoperation of a humanoid robot using full-body motion capture, example movements, and machine learning. In: Australasian conference on robotics and automation, ACRA

19.

Kofman J, Verma S, Wu X (2007) Robot-manipulator teleoperation by markerless vision-based hand-arm tracking. Int J Optomechatron 1:331–357. https://doi.org/10.1080/15599610701580467 CrossRef

20.

Romero J (2011) From human to robot grasping. Ph.D. dissertation, KTH Royal Institute of Technology

21.

Michel D, Qammaz A, Argyros A (2017) Markerless 3D human pose estimation and tracking based on RGBD cameras: an experimental evaluation. In: International conference on PErvasive technologies related to assistive environments. https://doi.org/10.1145/3056540.3056543

22.

Yuan S, Garcia-Hernando G, Stenger B, Moon G, Chang J, Lee K, Molchanov P, Kautz J, Honari S, Ge L, Yuan J, Chen X, Wang G, Yang F, Akiyama K, Wu Y, Wan Q, Madadi M, Escalera S, Kim T-K (2018) Depth-based 3D hand pose estimation: from current achievements to future goals. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 2636–2645. https://doi.org/10.1109/CVPR.2018.00279

23.

Moon G, Chang J, Lee K (2017) V2V-PoseNet: voxel-to-voxel prediction network for accurate 3D hand and human pose estimation from a single depth map. In: IEEE conference on computer vision and pattern recognition (CVPR)

24.

Oberweger M, Lepetit V (2017) DeepPrior++: Improving fast and accurate 3D hand pose estimation. In: Workshop of IEEE international conference on computer vision (ICCV), pp 585–594. https://doi.org/10.1109/ICCVW.2017.75

25.

Guo H, Wang G, Chen X, Zhang C, Qiao F, Yang H (2017) Region ensemble network: improving convolutional network for hand pose estimation. In: IEEE international conference on image processing (ICIP), pp 4512–4516. https://doi.org/10.1109/ICIP.2017.8297136

26.

Shon AP, Grochow K, Rao R (2005) Robotic imitation from human motion capture using Gaussian processes. In: IEEE/RAS international conference on humanoid robots, pp 129–134. https://doi.org/10.1109/ICHR.2005.1573557

27.

Sermanet P, Lynch C, Chebotar Y, Hsu J, Jang E, Schaal S, Levine S, Brain G (2018) Time-contrastive networks: self-supervised learning from video. In: IEEE international conference on robotics and automation (ICRA). https://doi.org/10.1109/ICRA.2018.8462891

28.

Andrychowicz OM, Baker B, Chociej M, Jozefowicz R, McGrew B, Pachocki J, Petron A, Plappert M, Powell G, Ray A, Schneider J, Sidor S, Tobin J, Welinder P, Weng L, Zaremba W (2019) Learning dexterous in-hand manipulation. Int J Robot Res. 027836491988744. https://doi.org/10.1177/0278364919887447

29.

Villegas R, Yang J, Ceylan D, Lee H (2018) Neural kinematic networks for unsupervised motion retargetting. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 8639–8648. https://doi.org/10.1109/CVPR.2018.00901

30.

Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014). Generative adversarial nets. In: Advances in neural information processing systems (NIPS)

31.

Yuan S, Ye Q, Stenger B, Jain S, Kim T-K (2017) BigHand2.2M benchmark: hand pose dataset and state of the art analysis. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 2605–2613. https://doi.org/10.1109/CVPR.2017.279

32.

Shadow Robot Company. Shadow dexterous hand e1 series. Available: http://www.shadowrobot.com/wpcontent/uploads/shadow. Dexterous hand technical specification

Titel: Learning from Visual-Based Teleoperation Demonstration
verfasst von: Bin Fang
Fuchun Sun
Huaping Liu
Chunfang Liu
Di Guo
Verlag: Springer Singapore
Buch: Wearable Technology for Robotic Manipulation and Learning
Print ISBN: 978-981-15-5123-9

Electronic ISBN: 978-981-15-5124-6

Copyright-Jahr: 2020
DOI: https://doi.org/10.1007/978-981-15-5124-6_6

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner