Skip to main content
Erschienen in: Autonomous Robots 5/2018

07.11.2017

Skill-based human–robot cooperation in tele-operated path tracking

verfasst von: Nima Enayati, Giancarlo Ferrigno, Elena De Momi

Erschienen in: Autonomous Robots | Ausgabe 5/2018

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This work proposes a shared-control tele-operation framework that adapts its cooperative properties to the estimated skill level of the operator. It is hypothesized that different aspects of an operator’s performance in executing a tele-operated path tracking task can be assessed through conventional machine learning methods using motion-based and task-related features. To identify performance measures that capture motor skills linked to the studied task, an experiment is conducted where users new to tele-operation, practice towards motor skill proficiency in 7 training sessions. A set of classifiers are then learned from the acquired data and selected features, which can generate a skill profile that comprises estimations of user’s various competences. Skill profiles are exploited to modify the behavior of the assistive robotic system accordingly with the objective of enhancing user experience by preventing unnecessary restriction for skilled users. A second experiment is implemented in which novice and expert users execute the path tracking on different pathways while being assisted by the robot according to their estimated skill profiles. Results validate the skill estimation method and hint at feasibility of shared-control customization in tele-operated path tracking.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Ahmidi, N., et al. (2017). A dataset and benchmarks for segmentation and recognition of gestures in robotic surgery. IEEE Transactions on Biomedical Engineering, 9294(c), 1–16. Ahmidi, N., et al. (2017). A dataset and benchmarks for segmentation and recognition of gestures in robotic surgery. IEEE Transactions on Biomedical Engineering, 9294(c), 1–16.
Zurück zum Zitat Anderson, S. J., Karumanchi, S. B., & Iagnemma, K. (2012). Constraint-based planning and control for safe, semi-autonomous operation of vehicles. In 2012 IEEE intelligent vehicles symposium (pp. 383–88). IEEE. Anderson, S. J., Karumanchi, S. B., & Iagnemma, K. (2012). Constraint-based planning and control for safe, semi-autonomous operation of vehicles. In 2012 IEEE intelligent vehicles symposium (pp. 383–88). IEEE.
Zurück zum Zitat Bauzano, E., Estebanez, B., & Muñoz, V. F. (2014). Collaborative human-robot system for HALS suture procedures. IEEE Systems Journal, 10, 1–10. Bauzano, E., Estebanez, B., & Muñoz, V. F. (2014). Collaborative human-robot system for HALS suture procedures. IEEE Systems Journal, 10, 1–10.
Zurück zum Zitat Beretta, E., et al. (2015). Enhanced torque-based impedance control to assist brain targeting during open-skull neurosurgery: A feasibility study. International Journal of Medical Robotics and Computer Assisted Surgery, 12(3), 326–41. https://doi.org/10.1002/rcs.1690.CrossRef Beretta, E., et al. (2015). Enhanced torque-based impedance control to assist brain targeting during open-skull neurosurgery: A feasibility study. International Journal of Medical Robotics and Computer Assisted Surgery, 12(3), 326–41. https://​doi.​org/​10.​1002/​rcs.​1690.CrossRef
Zurück zum Zitat Berthet-Rayne, P., Power, M., King, H., & Yang Fellow, G.-Z. (2016). Hubot?: A three state human–robot collaborative framework for bimanual surgical tasks based on learned models. In International conference on robotics and automation (pp. 715–722). Berthet-Rayne, P., Power, M., King, H., & Yang Fellow, G.-Z. (2016). Hubot?: A three state human–robot collaborative framework for bimanual surgical tasks based on learned models. In International conference on robotics and automation (pp. 715–722).
Zurück zum Zitat Bowyer, S. A., Davies, B. L., & Baena, F. R. Y. (2014). Active constraints/virtual fixtures: A survey. IEEE Transactions on Robotics, 30(1), 138–57.CrossRef Bowyer, S. A., Davies, B. L., & Baena, F. R. Y. (2014). Active constraints/virtual fixtures: A survey. IEEE Transactions on Robotics, 30(1), 138–57.CrossRef
Zurück zum Zitat Bowyer, S. A., & Baena, F. R. Y. (2015). Dissipative control for physical human–robot interaction. IEEE Transactions on Robotics, 31(6), 1281–1293.CrossRef Bowyer, S. A., & Baena, F. R. Y. (2015). Dissipative control for physical human–robot interaction. IEEE Transactions on Robotics, 31(6), 1281–1293.CrossRef
Zurück zum Zitat Bruyninckx, H. (2001). Open robot control software: The OROCOS project. In Proceedings 2001 ICRA. IEEE international conference on robotics and automation (Cat. No. 01CH37164) (pp. 2523–2528). IEEE. Bruyninckx, H. (2001). Open robot control software: The OROCOS project. In Proceedings 2001 ICRA. IEEE international conference on robotics and automation (Cat. No. 01CH37164) (pp. 2523–2528). IEEE.
Zurück zum Zitat Chmarra, M. K., et al. (2010). Objective classification of residents based on their psychomotor laparoscopic skills. Surgical Endoscopy, 24(5), 1031–1039.CrossRef Chmarra, M. K., et al. (2010). Objective classification of residents based on their psychomotor laparoscopic skills. Surgical Endoscopy, 24(5), 1031–1039.CrossRef
Zurück zum Zitat Cotin, S., et al. (2002). Metrics for laparoscopic skills trainers: The weakest link!. Medical Image Computing and Computer-Assisted Intervention—MICCAI, 2488, 35–43.MATH Cotin, S., et al. (2002). Metrics for laparoscopic skills trainers: The weakest link!. Medical Image Computing and Computer-Assisted Intervention—MICCAI, 2488, 35–43.MATH
Zurück zum Zitat Dragan, A. D., & Srinivasa, S. S. (2013). A policy-blending formalism for shared control. The International Journal of Robotics Research, 32(7), 790–805.CrossRef Dragan, A. D., & Srinivasa, S. S. (2013). A policy-blending formalism for shared control. The International Journal of Robotics Research, 32(7), 790–805.CrossRef
Zurück zum Zitat Enayati, N., Costa, E. C. A., Ferrigno, G., & De Momi, E. (2016a). A dynamic non-energy-storing guidance constraint with motion redirection for robot-assisted surgery. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Daejeon, Korea: IEEE. Enayati, N., Costa, E. C. A., Ferrigno, G., & De Momi, E. (2016a). A dynamic non-energy-storing guidance constraint with motion redirection for robot-assisted surgery. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Daejeon, Korea: IEEE.
Zurück zum Zitat Enayati, N., De Momi, E., & Ferrigno, G. (2016b). Haptics in robot-assisted surgery: Challenges and benefits. IEEE Reviews in Biomedical Engineering, 9, 49–65. Enayati, N., De Momi, E., & Ferrigno, G. (2016b). Haptics in robot-assisted surgery: Challenges and benefits. IEEE Reviews in Biomedical Engineering, 9, 49–65.
Zurück zum Zitat Enayati, N., Ferrigno, G., & De Momi, E. (2017). Performance metrics for guidance active constraints in surgical robotics. The International Journal of Medical Robotics and Computer Assisted Surgery. https://doi.org/10.1002/rcs.1873. Enayati, N., Ferrigno, G., & De Momi, E. (2017). Performance metrics for guidance active constraints in surgical robotics. The International Journal of Medical Robotics and Computer Assisted Surgery. https://​doi.​org/​10.​1002/​rcs.​1873.
Zurück zum Zitat Enayati, N., & Najafi, F. (2011). Design and manufacturing of a tele-operative rescue robot with a novel track arrangement. Industrial Robot: An International Journal, 38(5), 476–85.CrossRef Enayati, N., & Najafi, F. (2011). Design and manufacturing of a tele-operative rescue robot with a novel track arrangement. Industrial Robot: An International Journal, 38(5), 476–85.CrossRef
Zurück zum Zitat Fard, M. J., Ameri, S., Chinnam, R. B., & Ellis, R. D. (2017). Soft boundary approach for unsupervised gesture segmentation in robotic-assisted surgery. IEEE Robotics and Automation Letters, 2(1), 171–178.CrossRef Fard, M. J., Ameri, S., Chinnam, R. B., & Ellis, R. D. (2017). Soft boundary approach for unsupervised gesture segmentation in robotic-assisted surgery. IEEE Robotics and Automation Letters, 2(1), 171–178.CrossRef
Zurück zum Zitat Finan, M. A., Clark, M. E., & Rocconi, R. P. (2010). A novel method for training residents in robotic hysterectomy. Journal of Robotic Surgery, 4(1), 33–39.CrossRef Finan, M. A., Clark, M. E., & Rocconi, R. P. (2010). A novel method for training residents in robotic hysterectomy. Journal of Robotic Surgery, 4(1), 33–39.CrossRef
Zurück zum Zitat Franchi, A., et al. (2012). Shared control: Balancing autonomy and human assistance with a group of quadrotor UAVs. IEEE Robotics & Automation Magazine, 19(3), 57–68.CrossRef Franchi, A., et al. (2012). Shared control: Balancing autonomy and human assistance with a group of quadrotor UAVs. IEEE Robotics & Automation Magazine, 19(3), 57–68.CrossRef
Zurück zum Zitat Gao, M., Oberlander, J., Schamm, T., & Zollner, J. M. (2014). Contextual task-aware shared autonomy for assistive mobile robot teleoperation. In 2014 IEEE/RSJ international conference on intelligent robots and systems (pp. 3311–3318). IEEE. Gao, M., Oberlander, J., Schamm, T., & Zollner, J. M. (2014). Contextual task-aware shared autonomy for assistive mobile robot teleoperation. In 2014 IEEE/RSJ international conference on intelligent robots and systems (pp. 3311–3318). IEEE.
Zurück zum Zitat Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F. J., & Marín-Jiménez, M. J. (2014). Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognition, 47(6), 2280–2292.CrossRef Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F. J., & Marín-Jiménez, M. J. (2014). Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognition, 47(6), 2280–2292.CrossRef
Zurück zum Zitat Ghez, C., Krakauer, J. W., & Ghilardi, M.-F. (1999). Independent learning of internal models for kinematic and dynamic control of reaching. Nature Neuroscience, 2(11), 1026–1031.CrossRef Ghez, C., Krakauer, J. W., & Ghilardi, M.-F. (1999). Independent learning of internal models for kinematic and dynamic control of reaching. Nature Neuroscience, 2(11), 1026–1031.CrossRef
Zurück zum Zitat Gomes, P. (2011). Surgical robotics: Reviewing the past, analysing the present, imagining the future. Robotics and Computer-Integrated Manufacturing, 27(2), 261–266.CrossRef Gomes, P. (2011). Surgical robotics: Reviewing the past, analysing the present, imagining the future. Robotics and Computer-Integrated Manufacturing, 27(2), 261–266.CrossRef
Zurück zum Zitat Hajshirmohammadi, I., & Payandeh, S. (2007). Fuzzy set theory for performance evaluation in a surgical simulator. Presence: Teleoperators and Virtual Environments, 16(6), 603–622.CrossRef Hajshirmohammadi, I., & Payandeh, S. (2007). Fuzzy set theory for performance evaluation in a surgical simulator. Presence: Teleoperators and Virtual Environments, 16(6), 603–622.CrossRef
Zurück zum Zitat Van Hove, P. D., et al. (2010). Objective assessment of technical surgical skills. British Journal of Surgery, 97(7), 972–987.CrossRef Van Hove, P. D., et al. (2010). Objective assessment of technical surgical skills. British Journal of Surgery, 97(7), 972–987.CrossRef
Zurück zum Zitat Kranzfelder, M., et al. (2013). Toward increased autonomy in the surgical OR: needs, requests, and expectations. Surgical Endoscopy and Other Interventional Techniques, 27(5), 1681–1688.CrossRef Kranzfelder, M., et al. (2013). Toward increased autonomy in the surgical OR: needs, requests, and expectations. Surgical Endoscopy and Other Interventional Techniques, 27(5), 1681–1688.CrossRef
Zurück zum Zitat Lawrence, D. A. (1993). Stability and transparency in bilateral teleoperation. IEEE Transactions on Robotics and Automation, 9(5), 624–637.CrossRef Lawrence, D. A. (1993). Stability and transparency in bilateral teleoperation. IEEE Transactions on Robotics and Automation, 9(5), 624–637.CrossRef
Zurück zum Zitat Lin, H. C., Shafran, I., Yuh, D., & Hager, G. D. (2006). Towards automatic skill evaluation: Detection and segmentation of robot-assisted surgical motions. Computer Aided Surgery, 11(5), 220–230.CrossRef Lin, H. C., Shafran, I., Yuh, D., & Hager, G. D. (2006). Towards automatic skill evaluation: Detection and segmentation of robot-assisted surgical motions. Computer Aided Surgery, 11(5), 220–230.CrossRef
Zurück zum Zitat Liu, Y.-T., Mayer-Kress, G., & Newell, K. M. (2006). Qualitative and quantitative change in the dynamics of motor learning. Journal of Experimental Psychology: Human Perception and Performance, 32(2), 380–393. Liu, Y.-T., Mayer-Kress, G., & Newell, K. M. (2006). Qualitative and quantitative change in the dynamics of motor learning. Journal of Experimental Psychology: Human Perception and Performance, 32(2), 380–393.
Zurück zum Zitat Marchal-Crespo, L., & Reinkensmeyer, D. J. (2008). Haptic guidance can enhance motor learning of a steering task. Journal of Motor Behavior, 40(6), 545–556.CrossRef Marchal-Crespo, L., & Reinkensmeyer, D. J. (2008). Haptic guidance can enhance motor learning of a steering task. Journal of Motor Behavior, 40(6), 545–556.CrossRef
Zurück zum Zitat Megali, G., Sinigaglia, S., Tonet, O., & Dario, P. (2006). Modelling and evaluation of surgical performance using hidden markov models. IEEE Transactions on Biomedical Engineering, 53(10), 1911–1919.CrossRef Megali, G., Sinigaglia, S., Tonet, O., & Dario, P. (2006). Modelling and evaluation of surgical performance using hidden markov models. IEEE Transactions on Biomedical Engineering, 53(10), 1911–1919.CrossRef
Zurück zum Zitat Milliken, L., & Hollinger, G. A. (2016). Modeling user expertise for choosing levels of shared autonomy. In Planning for human–robot interaction shared autonomy and collaborative robotics workshop, robotics: Science and systems conference. Milliken, L., & Hollinger, G. A. (2016). Modeling user expertise for choosing levels of shared autonomy. In Planning for human–robot interaction shared autonomy and collaborative robotics workshop, robotics: Science and systems conference.
Zurück zum Zitat Nichols, K. A. et al. (2015). Models of human-centered automation in a debridement task. In IEEE international conference on intelligent robots and systems 2015–December (pp. 5784–5789). Nichols, K. A. et al. (2015). Models of human-centered automation in a debridement task. In IEEE international conference on intelligent robots and systems 2015–December (pp. 5784–5789).
Zurück zum Zitat Padoy, N., Hager, G.D. (2011) Human-machine collaborative surgery using learned models. In Proceedings of the IEEE international conference on robotics and automation (pp. 5285–5292). Padoy, N., Hager, G.D. (2011) Human-machine collaborative surgery using learned models. In Proceedings of the IEEE international conference on robotics and automation (pp. 5285–5292).
Zurück zum Zitat Pedregosa, F., et al. (2011). Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12, 2825–2830.MathSciNetMATH Pedregosa, F., et al. (2011). Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12, 2825–2830.MathSciNetMATH
Zurück zum Zitat Penin, L. F., Matsumoto, K., & Wakabayashi, S. (2000). Force reflection for time-delayed teleoperation of space robots. In Proceedings 2000 ICRA. Millennium conference. IEEE International conference on robotics and automation. Symposia proceedings (Cat. No. 00CH37065) (pp. 3120–3125). IEEE. Penin, L. F., Matsumoto, K., & Wakabayashi, S. (2000). Force reflection for time-delayed teleoperation of space robots. In Proceedings 2000 ICRA. Millennium conference. IEEE International conference on robotics and automation. Symposia proceedings (Cat. No. 00CH37065) (pp. 3120–3125). IEEE.
Zurück zum Zitat Rafii-Tari, H., et al. (2013). Learning-based modeling of endovascular navigation for collaborative robotic catheterization. In International conference on medical image computing and computer-assisted intervention (pp. 369–377). Berlin, Heidelberg: Springer. Rafii-Tari, H., et al. (2013). Learning-based modeling of endovascular navigation for collaborative robotic catheterization. In International conference on medical image computing and computer-assisted intervention (pp. 369–377). Berlin, Heidelberg: Springer.
Zurück zum Zitat Reiley, C. E., & Hager, G. D. (2009). Task versus subtask surgical skill evaluation of robotic minimally invasive surgery. In International conference on medical image computing and computer-assisted intervention (pp. 435–442). Berlin, Heidelberg: Springer. Reiley, C. E., & Hager, G. D. (2009). Task versus subtask surgical skill evaluation of robotic minimally invasive surgery. In International conference on medical image computing and computer-assisted intervention (pp. 435–442). Berlin, Heidelberg: Springer.
Zurück zum Zitat Richards, C., et al. (2000). Skills evaluation in minimally invasive surgery using force/torque signatures. Surgical Endoscopy, 14(9), 791–798.CrossRef Richards, C., et al. (2000). Skills evaluation in minimally invasive surgery using force/torque signatures. Surgical Endoscopy, 14(9), 791–798.CrossRef
Zurück zum Zitat Rosen, J., et al. (2006). Generalized approach for modeling minimally invasive surgery as a stochastic process using a discrete Markov model. IEEE Transactions on Biomedical Engineering, 53(3), 399–413.CrossRef Rosen, J., et al. (2006). Generalized approach for modeling minimally invasive surgery as a stochastic process using a discrete Markov model. IEEE Transactions on Biomedical Engineering, 53(3), 399–413.CrossRef
Zurück zum Zitat Sainburg, R. L., & Wang, J. (2002). Interlimb transfer of visuomotor rotations: Independence of direction and final position information. Experimental Brain Research, 145(4), 437–447.CrossRef Sainburg, R. L., & Wang, J. (2002). Interlimb transfer of visuomotor rotations: Independence of direction and final position information. Experimental Brain Research, 145(4), 437–447.CrossRef
Zurück zum Zitat Schmidt, R. A. (1991). Frequent augmented feedback can degrade learning: Evidence and interpretations. In: Tutorials in motor neuroscience (pp. 59–75). Dordrecht, Netherlands: Springer. Schmidt, R. A. (1991). Frequent augmented feedback can degrade learning: Evidence and interpretations. In: Tutorials in motor neuroscience (pp. 59–75). Dordrecht, Netherlands: Springer.
Zurück zum Zitat Schreuder, H. W. R., et al. (2012). Training and learning robotic surgery, time for a more structured approach: A systematic review. BJOG: An International Journal of Obstetrics and Gynaecology, 119(2), 137–149.CrossRef Schreuder, H. W. R., et al. (2012). Training and learning robotic surgery, time for a more structured approach: A systematic review. BJOG: An International Journal of Obstetrics and Gynaecology, 119(2), 137–149.CrossRef
Zurück zum Zitat Shadmehr, R., Smith, M. A., & Krakauer, J. W. (2010). Error correction, sensory prediction, and adaptation in motor control. Annual Review of Neuroscience, 33(1), 89–108.CrossRef Shadmehr, R., Smith, M. A., & Krakauer, J. W. (2010). Error correction, sensory prediction, and adaptation in motor control. Annual Review of Neuroscience, 33(1), 89–108.CrossRef
Zurück zum Zitat Shmuelof, L., Krakauer, J. W., & Mazzoni, P. (2012). How is a motor skill learned? Change and invariance at the levels of task success and trajectory control. Journal of Neurophysiology, 108(2), 578–594.CrossRef Shmuelof, L., Krakauer, J. W., & Mazzoni, P. (2012). How is a motor skill learned? Change and invariance at the levels of task success and trajectory control. Journal of Neurophysiology, 108(2), 578–594.CrossRef
Zurück zum Zitat Sigrist, R., Rauter, G., Riener, R., & Wolf, P. (2013). Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review. Psychonomic Bulletin & Review, 20(1), 21–53.CrossRef Sigrist, R., Rauter, G., Riener, R., & Wolf, P. (2013). Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review. Psychonomic Bulletin & Review, 20(1), 21–53.CrossRef
Zurück zum Zitat Takayama, L., Marder-Eppstein, E., Harris, H., & Beer, J. M. (2011). Assisted driving of a mobile remote presence system: System design and controlled user evaluation. In 2011 IEEE international conference on robotics and automation (pp. 1883–1889). IEEE. Takayama, L., Marder-Eppstein, E., Harris, H., & Beer, J. M. (2011). Assisted driving of a mobile remote presence system: System design and controlled user evaluation. In 2011 IEEE international conference on robotics and automation (pp. 1883–1889). IEEE.
Zurück zum Zitat Trejos, A. L., et al. (2013). The application of force sensing to skills assessment in minimally invasive surgery. In 2013 IEEE international conference on robotics and automation (pp. 4370–4375). IEEE. Trejos, A. L., et al. (2013). The application of force sensing to skills assessment in minimally invasive surgery. In 2013 IEEE international conference on robotics and automation (pp. 4370–4375). IEEE.
Metadaten
Titel
Skill-based human–robot cooperation in tele-operated path tracking
verfasst von
Nima Enayati
Giancarlo Ferrigno
Elena De Momi
Publikationsdatum
07.11.2017
Verlag
Springer US
Erschienen in
Autonomous Robots / Ausgabe 5/2018
Print ISSN: 0929-5593
Elektronische ISSN: 1573-7527
DOI
https://doi.org/10.1007/s10514-017-9675-4

Weitere Artikel der Ausgabe 5/2018

Autonomous Robots 5/2018 Zur Ausgabe

Neuer Inhalt