Skip to main content
Top
Published in: Journal of Intelligent Manufacturing 7/2023

05-08-2022

Combining human guidance and structured task execution during physical human–robot collaboration

Authors: Jonathan Cacace, Riccardo Caccavale, Alberto Finzi, Riccardo Grieco

Published in: Journal of Intelligent Manufacturing | Issue 7/2023

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In this work, we consider a scenario in which a human operator physically interacts with a collaborative robot (CoBot) to perform shared and structured tasks. We assume that collaborative operations are formulated as hierarchical task networks to be interactively executed exploiting the human physical guidance. In this scenario, the human interventions are continuously interpreted by the robotic system in order to infer whether the human guidance is aligned or not with respect to the planned activities. The interpreted human interventions are also exploited by the robotic system to on-line adapt its cooperative behavior during the execution of the shared plan. Depending on the estimated operator intentions, the robotic system can adjust tasks or motions, while regulating the robot compliance with respect to the co-worker physical guidance. We describe the overall framework illustrating the architecture and its components. The proposed approach is demonstrated in a testing scenario consisting of a human operator that interacts with the Kuka LBR iiwa manipulator in order to perform a collaborative task. The collected results show the effectiveness of the proposed approach.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Footnotes
1
The Wilcoxon signed-rank test (Wilcoxon, 1945) with Pratt modification (Pratt, 1959) (for zero differences in Liker values) was deployed since the collected questionnaire results are paired and not normally distributed.
 
Literature
go back to reference Broquère, X., Finzi, A., Mainprice, J., Rossi, S., Sidobre, D., & Staffa, M. (2014). An attentional approach to human-robot interactive manipulation. International Journal of Social Robotics, 6(4), 533–553.CrossRef Broquère, X., Finzi, A., Mainprice, J., Rossi, S., Sidobre, D., & Staffa, M. (2014). An attentional approach to human-robot interactive manipulation. International Journal of Social Robotics, 6(4), 533–553.CrossRef
go back to reference Cacace, J., Caccavale, R., Finzi, A., & Lippiello, V. (2018). Interactive plan execution during human–robot cooperative manipulation. IFAC-PapersOnLine, 51(22), 500–505.CrossRef Cacace, J., Caccavale, R., Finzi, A., & Lippiello, V. (2018). Interactive plan execution during human–robot cooperative manipulation. IFAC-PapersOnLine, 51(22), 500–505.CrossRef
go back to reference Cacace, J., Caccavale, R., Finzi, A., & Lippiello, V. (2019). Variable admittance control based on virtual fixtures for human–robot co-manipulation. In Proceedings of the IEEE international conference on systems, man and cybernetics (SMC 2019), pp. 1569–1574 Cacace, J., Caccavale, R., Finzi, A., & Lippiello, V. (2019). Variable admittance control based on virtual fixtures for human–robot co-manipulation. In Proceedings of the IEEE international conference on systems, man and cybernetics (SMC 2019), pp. 1569–1574
go back to reference Cacace, J., Finzi, A., & Lippiello, V. (2019). Enhancing shared control via contact force classification in human–robot cooperative task execution. In F. Ficuciello, F. Ruggiero, & A. Finzi (Eds.), Human friendly robotics (pp. 167–179). Springer International Publishing. Cacace, J., Finzi, A., & Lippiello, V. (2019). Enhancing shared control via contact force classification in human–robot cooperative task execution. In F. Ficuciello, F. Ruggiero, & A. Finzi (Eds.), Human friendly robotics (pp. 167–179). Springer International Publishing.
go back to reference Caccavale, R., & Finzi, A. (2015). Plan execution and attentional regulations for flexible human–robot interaction. In Proceedings of the IEEE international conference on systems, man, and cybernetics (SMC 2015), IEEE, pp. 2453–2458 Caccavale, R., & Finzi, A. (2015). Plan execution and attentional regulations for flexible human–robot interaction. In Proceedings of the IEEE international conference on systems, man, and cybernetics (SMC 2015), IEEE, pp. 2453–2458
go back to reference Caccavale, R., & Finzi, A. (2017). Flexible task execution and attentional regulations in human–robot interaction. IEEE Transactions on Cognitive and Developmental Systems, 9(1), 68–79.CrossRef Caccavale, R., & Finzi, A. (2017). Flexible task execution and attentional regulations in human–robot interaction. IEEE Transactions on Cognitive and Developmental Systems, 9(1), 68–79.CrossRef
go back to reference Caccavale, R., & Finzi, A. (2019). Learning attentional regulations for structured tasks execution in robotic cognitive control. Autonomous Robots, 43(8), 2229–2243.CrossRef Caccavale, R., & Finzi, A. (2019). Learning attentional regulations for structured tasks execution in robotic cognitive control. Autonomous Robots, 43(8), 2229–2243.CrossRef
go back to reference Caccavale, R., & Finzi, A. (2022). A robotic cognitive control framework for collaborative task execution and learning. Topics in Cognitive Science, 14(2), 327–343.CrossRef Caccavale, R., & Finzi, A. (2022). A robotic cognitive control framework for collaborative task execution and learning. Topics in Cognitive Science, 14(2), 327–343.CrossRef
go back to reference Caccavale, R., Cacace, J., Fiore, M., Alami, R., & Finzi, A. (2016). Attentional supervision of human-robot collaborative plans. In Proceedings of the IEEE international symposium on robot and human interactive communication (RO-MAN 2016), pp. 867–873 Caccavale, R., Cacace, J., Fiore, M., Alami, R., & Finzi, A. (2016). Attentional supervision of human-robot collaborative plans. In Proceedings of the IEEE international symposium on robot and human interactive communication (RO-MAN 2016), pp. 867–873
go back to reference Caccavale, R., Saveriano, M., Finzi, A., & Lee, D. (2019). Kinesthetic teaching and attentional supervision of structured tasks in human–robot interaction. Autonomous Robots, 43(6), 1291–1307.CrossRef Caccavale, R., Saveriano, M., Finzi, A., & Lee, D. (2019). Kinesthetic teaching and attentional supervision of structured tasks in human–robot interaction. Autonomous Robots, 43(6), 1291–1307.CrossRef
go back to reference Carbone, A., Finzi, A., Orlandini, A., & Pirri, F. (2008). Model-based control architecture for attentive robots in rescue scenarios. Autonomous Robots, 24(1), 87–120.CrossRef Carbone, A., Finzi, A., Orlandini, A., & Pirri, F. (2008). Model-based control architecture for attentive robots in rescue scenarios. Autonomous Robots, 24(1), 87–120.CrossRef
go back to reference Chen, T. L., & Kemp, C. C. (2010). Lead me by the hand: Evaluation of a direct physical interface for nursing assistant robots. In Proceedings of the 5th ACM/IEEE international conference on human–robot interaction (HRI 2010), pp. 367–374 Chen, T. L., & Kemp, C. C. (2010). Lead me by the hand: Evaluation of a direct physical interface for nursing assistant robots. In Proceedings of the 5th ACM/IEEE international conference on human–robot interaction (HRI 2010), pp. 367–374
go back to reference Clodic, A., Cao, H., Alili, S., Montreuil, V., Alami, R., & Chatila, R. (2008). SHARY: A supervision system adapted to human-robot interaction. ISER, Springer, Springer Tracts in Advanced Robotics, 54, 229–238.CrossRef Clodic, A., Cao, H., Alili, S., Montreuil, V., Alami, R., & Chatila, R. (2008). SHARY: A supervision system adapted to human-robot interaction. ISER, Springer, Springer Tracts in Advanced Robotics, 54, 229–238.CrossRef
go back to reference Colgate, E., & Hogan, N. (1989). An analysis of contact instability in terms of passive physical equivalents. In Proceedings of the international conference on robotics and automation (ICRA 1989), Vol. 1, pp. 404–409 Colgate, E., & Hogan, N. (1989). An analysis of contact instability in terms of passive physical equivalents. In Proceedings of the international conference on robotics and automation (ICRA 1989), Vol. 1, pp. 404–409
go back to reference Colgate, J. E., & Hogan, N. (1988). Robust control of dynamically interacting systems. International Journal of Control, 48(1), 65–88.CrossRef Colgate, J. E., & Hogan, N. (1988). Robust control of dynamically interacting systems. International Journal of Control, 48(1), 65–88.CrossRef
go back to reference Cooper, R., & Shallice, T. (2000). Contention scheduling and the control of routine activies. Cognitive Neuropsychology, 17(4), 297–338.CrossRef Cooper, R., & Shallice, T. (2000). Contention scheduling and the control of routine activies. Cognitive Neuropsychology, 17(4), 297–338.CrossRef
go back to reference Cooper, R. P., & Shallice, T. (2006). Hierarchical schemas and goals in the control of sequential behavior. Psychological Review, 113(4), 887–916.CrossRef Cooper, R. P., & Shallice, T. (2006). Hierarchical schemas and goals in the control of sequential behavior. Psychological Review, 113(4), 887–916.CrossRef
go back to reference Corrales, J. A., Garcia Gomez, G. J., Torres, F., & Perdereau, V. (2012). Cooperative tasks between humans and robots in industrial environments. International Journal of Advanced Robotic Systems, 9, 1–10.CrossRef Corrales, J. A., Garcia Gomez, G. J., Torres, F., & Perdereau, V. (2012). Cooperative tasks between humans and robots in industrial environments. International Journal of Advanced Robotic Systems, 9, 1–10.CrossRef
go back to reference De Santis, A., Siciliano, B., Luca, A., & Bicchi, A. (2007). An atlas of physical human–robot interaction. Mechanism and Machine Theory, 43(3), 253–270.CrossRef De Santis, A., Siciliano, B., Luca, A., & Bicchi, A. (2007). An atlas of physical human–robot interaction. Mechanism and Machine Theory, 43(3), 253–270.CrossRef
go back to reference Grafakos, S., Dimeas, F., & Aspragathos, N. (2016). Variable admittance control in phri using emg-based arm muscles co-activation. In Proceedings of the IEEE international conference on systems, man, and cybernetics (SMC 2016), pp. 1900–1905 Grafakos, S., Dimeas, F., & Aspragathos, N. (2016). Variable admittance control in phri using emg-based arm muscles co-activation. In Proceedings of the IEEE international conference on systems, man, and cybernetics (SMC 2016), pp. 1900–1905
go back to reference Hart, S., & Staveland, L. (1988). Development of nasa-tlx (task load index): Results of empirical and theoretical research. Human Mental Workload Advances in Psychology, 52, 139–183.CrossRef Hart, S., & Staveland, L. (1988). Development of nasa-tlx (task load index): Results of empirical and theoretical research. Human Mental Workload Advances in Psychology, 52, 139–183.CrossRef
go back to reference Hoffman, G., & Breazeal, C. (2004). Collaboration in human-robot teams. In Proceedings of the AIAA 1st intelligent systems technical conference, pp. 1–18 Hoffman, G., & Breazeal, C. (2004). Collaboration in human-robot teams. In Proceedings of the AIAA 1st intelligent systems technical conference, pp. 1–18
go back to reference Hoffman, G., & Breazeal, C. (2007). Effects of anticipatory action on human-robot teamwork: Efficiency, fluency, and perception of team. In Proceedings of the 2nd ACM/IEEE international conference on human–robot interaction (HRI 2007), pp. 1–8 Hoffman, G., & Breazeal, C. (2007). Effects of anticipatory action on human-robot teamwork: Efficiency, fluency, and perception of team. In Proceedings of the 2nd ACM/IEEE international conference on human–robot interaction (HRI 2007), pp. 1–8
go back to reference Johannsmeier, L., & Haddadin, S. (2017). A hierarchical human–robot interaction-planning framework for task allocation in collaborative industrial assembly processes. IEEE Robotics and Automation Letters, 2(1), 41–48.CrossRef Johannsmeier, L., & Haddadin, S. (2017). A hierarchical human–robot interaction-planning framework for task allocation in collaborative industrial assembly processes. IEEE Robotics and Automation Letters, 2(1), 41–48.CrossRef
go back to reference Joseph, L., & Cacace, J. (2018). Mastering ROS for robotics programming: Design, build, and simulate complex robots using the robot operating system, 2nd edn. Packt Publishing Joseph, L., & Cacace, J. (2018). Mastering ROS for robotics programming: Design, build, and simulate complex robots using the robot operating system, 2nd edn. Packt Publishing
go back to reference Karpas, E., Levine, S. J., Yu, P., & Williams, BC. (2015). Robust execution of plans for human-robot teams. In: Proceedings of the twenty-fifth international conference on international conference on automated planning and scheduling (ICAPS 2015), AAAI Press, pp. 342–346 Karpas, E., Levine, S. J., Yu, P., & Williams, BC. (2015). Robust execution of plans for human-robot teams. In: Proceedings of the twenty-fifth international conference on international conference on automated planning and scheduling (ICAPS 2015), AAAI Press, pp. 342–346
go back to reference Lallement, R., de Silva, L., & Alami, R. (2014). HATP: An HTN planner for robotics. In: Proceedings of the 2nd ICAPS workshop on planning and robotics (PlanRob 2014), pp. 20–27 Lallement, R., de Silva, L., & Alami, R. (2014). HATP: An HTN planner for robotics. In: Proceedings of the 2nd ICAPS workshop on planning and robotics (PlanRob 2014), pp. 20–27
go back to reference Maurtua, I., Ibarguren, A., Kildal, J., Susperregi, L., & Sierra, B. (2017). Human-robot collaboration in industrial applications: Safety, interaction and trust. International Journal of Advanced Robotic Systems, 14, 1–10. Maurtua, I., Ibarguren, A., Kildal, J., Susperregi, L., & Sierra, B. (2017). Human-robot collaboration in industrial applications: Safety, interaction and trust. International Journal of Advanced Robotic Systems, 14, 1–10.
go back to reference Nicolis, D., Zanchettin, AM., & Rocco, P. (2018). Human intention estimation based on neural networks for enhanced collaboration with robots. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 1326–1333 Nicolis, D., Zanchettin, AM., & Rocco, P. (2018). Human intention estimation based on neural networks for enhanced collaboration with robots. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 1326–1333
go back to reference Norman, D. A., & Shallice, T. (1986). Attention to action. In: Consciousness and self-regulation, pp 1–18. Springer Norman, D. A., & Shallice, T. (1986). Attention to action. In: Consciousness and self-regulation, pp 1–18. Springer
go back to reference Park, J. S., Park, C., & Manocha, D. (2019). I-planner: Intention-aware motion planning using learning-based human motion prediction. The International Journal of Robotics Research, 38(1), 23–39.CrossRef Park, J. S., Park, C., & Manocha, D. (2019). I-planner: Intention-aware motion planning using learning-based human motion prediction. The International Journal of Robotics Research, 38(1), 23–39.CrossRef
go back to reference Peternel, L., Tsagarakis, N., Caldwell, D., & Ajoudani, A. (2016). Adaptation of robot physical behaviour to human fatigue in human–robot co-manipulation. In: Proceedings of the IEEE-RAS 16th international conference on humanoid robots (humanoids 2016), pp. 489–494 Peternel, L., Tsagarakis, N., Caldwell, D., & Ajoudani, A. (2016). Adaptation of robot physical behaviour to human fatigue in human–robot co-manipulation. In: Proceedings of the IEEE-RAS 16th international conference on humanoid robots (humanoids 2016), pp. 489–494
go back to reference Pratt, J. (1959). Remarks on zeros and ties in the Wilcoxon signed rank procedures. Journal of the American Statistical Association, 54(287), 655–667.CrossRef Pratt, J. (1959). Remarks on zeros and ties in the Wilcoxon signed rank procedures. Journal of the American Statistical Association, 54(287), 655–667.CrossRef
go back to reference Raiola, G., Lamy, X., & Stulp, F. (2015). Co-manipulation with multiple probabilistic virtual guides. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (IROS 2015), pp. 7–13 Raiola, G., Lamy, X., & Stulp, F. (2015). Co-manipulation with multiple probabilistic virtual guides. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (IROS 2015), pp. 7–13
go back to reference Romero, D., Bernus, P., Noran, O., Stahre, J., & Fast-Berglund, Å. (2016). The operator 4.0: human cyber-physical systems & adaptive automation towards human-automation symbiosis work systems. In: Proceedings of the IFIP international conference on advances in production management systems, Springer, pp. 677–686 Romero, D., Bernus, P., Noran, O., Stahre, J., & Fast-Berglund, Å. (2016). The operator 4.0: human cyber-physical systems & adaptive automation towards human-automation symbiosis work systems. In: Proceedings of the IFIP international conference on advances in production management systems, Springer, pp. 677–686
go back to reference Shah, J., Wiken, J., Williams, B., & Breazeal, C. (2011). Improved human-robot team performance using chaski, a human-inspired plan execution system. In: Proceedings of the 6th ACM/IEEE international conference on human–robot interaction (HRI 2011), pp. 29–36 Shah, J., Wiken, J., Williams, B., & Breazeal, C. (2011). Improved human-robot team performance using chaski, a human-inspired plan execution system. In: Proceedings of the 6th ACM/IEEE international conference on human–robot interaction (HRI 2011), pp. 29–36
go back to reference Siciliano, B., Sciavicco, L., Villani, L., & Oriolo, G. (2008). Robotics: Modelling, planning and control (1st ed.). Springer Publishing Company, Incorporated. Siciliano, B., Sciavicco, L., Villani, L., & Oriolo, G. (2008). Robotics: Modelling, planning and control (1st ed.). Springer Publishing Company, Incorporated.
go back to reference Sisbot, E. A., Marin-Urias, L. F., Alami, R., & Simeon, T. (2007). A human aware mobile robot motion planner. IEEE Transactions on Robotics, 23(5), 874–883.CrossRef Sisbot, E. A., Marin-Urias, L. F., Alami, R., & Simeon, T. (2007). A human aware mobile robot motion planner. IEEE Transactions on Robotics, 23(5), 874–883.CrossRef
go back to reference Steinfeld, A., Fong, T., Kaber, D., Lewis, M., Scholtz, J., Schultz, A., & Goodrich, M. (2006). Common metrics for human-robot interaction. In: Proceedings of the 1st ACM SIGCHI/SIGART conference on human–robot interaction (HRI 2006). ACM, pp 33–40 Steinfeld, A., Fong, T., Kaber, D., Lewis, M., Scholtz, J., Schultz, A., & Goodrich, M. (2006). Common metrics for human-robot interaction. In: Proceedings of the 1st ACM SIGCHI/SIGART conference on human–robot interaction (HRI 2006). ACM, pp 33–40
go back to reference Vernon, D., & Vincze, M. (2016). Industrial priorities for cognitive robotics. In EUCognition, CEUR-WS.org, CEUR workshop proceedings, Vol. 1855, pp. 6–9 Vernon, D., & Vincze, M. (2016). Industrial priorities for cognitive robotics. In EUCognition, CEUR-WS.org, CEUR workshop proceedings, Vol. 1855, pp. 6–9
go back to reference Wilcoxon, F. (1945). Individual comparisons by ranking methods. Biometrics Bulletin, 1(6), 80–83.CrossRef Wilcoxon, F. (1945). Individual comparisons by ranking methods. Biometrics Bulletin, 1(6), 80–83.CrossRef
go back to reference Young, J. E., Sung, J., Voida, A., Sharlin, E., Igarashi, T., Christensen, H. I., & Grinter, R. E. (2011). Evaluating human–robot interaction. International Journal of Social Robotics, 3(1), 53–67.CrossRef Young, J. E., Sung, J., Voida, A., Sharlin, E., Igarashi, T., Christensen, H. I., & Grinter, R. E. (2011). Evaluating human–robot interaction. International Journal of Social Robotics, 3(1), 53–67.CrossRef
Metadata
Title
Combining human guidance and structured task execution during physical human–robot collaboration
Authors
Jonathan Cacace
Riccardo Caccavale
Alberto Finzi
Riccardo Grieco
Publication date
05-08-2022
Publisher
Springer US
Published in
Journal of Intelligent Manufacturing / Issue 7/2023
Print ISSN: 0956-5515
Electronic ISSN: 1572-8145
DOI
https://doi.org/10.1007/s10845-022-01989-y

Other articles of this Issue 7/2023

Journal of Intelligent Manufacturing 7/2023 Go to the issue

Premium Partners