Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Designing a User-Centered Approach to Improve Acceptance of Innovations on the Shop Floor Using Rogers’ ‘Diffusion of Innovations’ Read chapter Read first chapter

2019 | OriginalPaper | Chapter

Evaluation Metrics Regarding Human Well-Being and System Performance in Human-Robot Interaction – A Literature Review

Authors : Jochen Nelles, Sonja Th. Kwee-Meier, Alexander Mertens

Published in: Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018)

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

This literature review provides an overview on evaluation metrics regarding human well-being and system performance in human-robot interaction. In this context a systematic literature search in the Web of Science and IEEE Xplore databases was carried out. Thus, 30 relevant contributions out of 3854 studies were analyzed by multistage filtering. To gain an overview on and compare the different approaches and results, the studies are summarized in tables according to the following criteria: author, year, title, task, study design, measurement methods, population, and results. The evaluation metrics presented in this contribution in principle can be divided into questionnaire-based surveys and psychophysiological measurement methods. In addition, the studies are classified into evaluation metrics for measuring well-being and performance.
The research was carried out with regard to an industrial engineering and ergonomics context, but is independent of specific application areas.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Ergonomic Analysis on the Assembly Line of Home Appliance Company
next chapter Towards an Engineering Process to Design Usable Tangible Human-Machine Interfaces
Literature
1.
2.
ISO 9241-110 (2006) Ergonomics of human-system interaction – Part 110: Dialogue principles
3.
ISO 9241-210 (2010) Ergonomics of human-system interaction – Part 210: Human-centered design for interactive systems
4.
Nelles J, Brandl C, Mertens A (2018, in German) Regelkreismodell für die menschzentrierte Gestaltung und Evaluierung einer Mensch-Roboter-Interaktion am Beispiel eines Mensch-Roboter-Arbeitsplatzes. In: Arbeit(s)Wissenschaf(f)t – Grundlage für Management und Kompetenzentwicklung. 64. Kongress der Gesellschaft für Arbeitswissenschaft
5.
Schlick CM, Bruder R, Luczak H (2010, in German) Arbeitswissenschaft. Springer, Heidelberg
6.
7.
Bortot D, Born M, Bengler K (2013) Directly or on detours? How should industrial robots approximate humans? In: Kuzuoka H (ed) 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI), 3–6 March 2013, Tokyo, Japan (including workshop papers). IEEE, Piscataway, pp 89–90
8.
Charalambous G, Fletcher S, Webb P (2016) The development of a scale to evaluate trust in industrial human-robot collaboration. Int J Soc Robot 8(2):193–209
9.
Daniel B, Thomessen T, Korondi P (2013) Simplified human-robot interaction: modeling and evaluation. Model. Identif. Control Norw. Res. Bull. 34(4):199–211
10.
Dehais F, Sisbot EA, Alami R, Causse M (2011) Physiological and subjective evaluation of a human-robot object hand-over task. Appl Ergon 42(6):785–791
11.
Hancock PA, Billings DR, Schaefer KE, Chen JYC, de Visser EJ, Parasuraman R (2011) A meta-analysis of factors affecting trust in human-robot interaction. Hum Factors 53(5):517–527
12.
Kessler TT, Larios C, Walker T, Yerdon V, Hancock PA (2017) A comparison of trust measures in human–robot interaction scenarios. In: Savage-Knepshield P, Chen J (eds) Advances in human factors in robots and un-manned systems, Proceedings of the AHFE 2016 international conference on human factors in robots and unmanned systems, 27–30 July 2016, Walt Disney World®, Florida, USA. (Advances in Intelligent Systems and Computing, vol 499, pp 353–364. Springer, Cham
13.
Liu D, Kinugawa J, Kosuge K (2016) A projection-based making-human-feel-safe system for human-robot cooperation. In: Proceedings of 2016 IEEE international conference on mechatronics and automation, 7–10 August 2016, Harbin, China, pp 1101–1106. IEEE, Piscataway
14.
Maurtua I, Pedrocchi N, Orlandini A, Fernandez JG, Vogel C, Geenen A, Althoefer K, Shafti A (2016) FourByThree: imagine humans and robots working hand in hand. In: 2016 IEEE 21st international conference on emerging technologies and factory automation (ETFA), 6–9 September 2016, Berlin, Germany, pp 1–8. IEEE, Piscataway
15.
Murata A (2000) Ergonomics and cognitive engineering for robot-human cooperation. In: Proceedings of the 9th IEEE international workshop on robot and human interactive communication, IEEE RO-MAN 2000, 27–29 September 2000, pp 206–211. Osaka Institute of Technology, Osaka, IEEE Service Center, Piscataway
16.
Murphy RR, Schreckenghost D (2013) Survey of metrics for human-robot interaction. In: Kuzuoka H (ed) 2013 8th ACM/IEEE international conference on human-robot interaction (HRI) (including workshop papers), 3–6 March 2013, Tokyo, Japan, pp 197–198. IEEE, Piscataway
17.
Nomura T, Kanda T, Suzuki T, Kato K (2004) Psychology in human-robot communication: an attempt through investigation of negative attitudes and anxiety toward robots. In: Proceedings of the 13th IEEE international workshop on robot and human interactive communication, RO-MAN 2004, 20–22 September 2004, Kurashiki, Okayama, Japan, pp 35–40. IEEE Operations Center, Piscataway
18.
Novak D, Mihelj M, Munih M (2011) Psychophysiological responses to different levels of cognitive and physical workload in haptic interaction. Robotica 29(03):367–374
19.
Ogorodnikova O (2008) Methodology of safety for a human robot interaction designing stage. In: Conference on human system interactions, HSI 2008, 25–27 May 2008, Krakow, Poland, pp 452–457. IEEE, Piscataway
20.
Pini F, Ansaloni M, Leali F (2016) Evaluation of operator relief for an effective design of HRC workcells. In: 2016 IEEE 21st international conference on emerging technologies and factory automation (ETFA), 6–9 September 2016, Berlin, Germany, pp 1–6. IEEE, Piscataway
21.
Profanter S, Perzylo A, Somani N, Rickert M, Knoll A (2015) Analysis and semantic modeling of modality preferences in industrial human-robot interaction. In: Burgard W (ed) 2015 IEEE/RSJ international conference on intelligent robots and systems (IROS), 28 September 2015–2 October 2015, Hamburg, Germany, pp 1812–1818. IEEE, Piscataway
22.
Rahman SMM, Sadrfaridpour B, Wang Y (2016) Trust-based optimal subtask allocation and model predictive control for human-robot collaborative assembly in manufacturing. In: Proceedings of the ASME 8th annual dynamic systems and control conference – 2015, presented at ASME 2015 8th annual dynamic systems and control conference, 28–30 October 2015, Columbus, OH, USA. The American Society of Mechanical Engineers, New York
23.
Sadrfaridpour B, Saeidi H, Wang Y (2016) An integrated framework for human-robot collaborative assembly in hybrid manufacturing cells. In: 2016 IEEE international conference on automation science and engineering (CASE), pp 462–467. IEEE
24.
Sanders TL, Wixon T, Schafer KE, Chen JYC, Hancock PA (2014) The influence of modality and transparency on trust in human-robot interaction. In: 2014 IEEE international inter-disciplinary conference on cognitive methods in situation awareness and decision support (CogSIMA), 3–6 March 2014, San Antonio, TX, USA, pp 156–159. IEEE, Piscataway
25.
Sim DYY, Loo CK (2015) Extensive assessment and evaluation methodologies on assistive social robots for modelling human–robot interaction – a review. Inf Sci 301:305–344
26.
Tan JTC, Duan F, Zhang Y, Arai T (2008) Task decomposition of cell production assembly operation for man-machine collaboration by HTA. In: IEEE international conference on automation and logistics, ICAL 2008, 1–3 September 2008, Qingdao, China, pp 1066–1071. IEEE, Piscataway
27.
Tan J, Duan F, Zhang Y, Arai T (2009) Extending task analysis in HTA to model man-machine collaboration in cell production. In: IEEE international conference on robotics and biomimetics, ROBIO 2008, 22–25 February 2009, Bangkok, Thailand (rescheduled from December 2008, original conference date: 14–17 December 2008), pp 542–547. IEEE, Piscataway
28.
Tan JTC, Duan F, Zhang Y, Watanabe K, Kato R, Arai T (2009) Human-robot collaboration in cellular manufacturing: design and development. In: 2009 IEEE/RSJ international conference on intelligent robots and systems, IROS 2009, St. Louis, Missouri, USA, 10–15 (i.e. 11–15) October 2009, pp 29–34. IEEE, Piscataway
29.
Tan JTC, Zhang Y, Duan F, Watanabe K, Kato R, Arai T (2009) Human factors studies in information support development for human-robot collaborative cellular manufacturing system. In: The 18th IEEE international symposium on robot and human interactive communication, RO-MAN 2009, 27 September 2009–2 October 2009, Toyama, Japan, pp 334–339. IEEE, Piscataway
30.
Tan JTC, Arai T (2010) Analytic evaluation of human-robot system for collaboration in cellular manufacturing system. In: IEEE/ASME international conference on advanced intelligent mechatronics (AIM), 6–9 July 2010, Montreal, QC, Canada, pp 515–520. IEEE, Piscataway
31.
Tan JTC, Arai T (2010) Information support development with human-centered approach for human-robot collaboration in cellular manufacturing. In: IEEE RO-MAN, 19th IEEE international symposium on robot and human interactive communication, 13–15 September 2010, Viareggio, Italy, pp 767–772. IEEE, Piscataway
32.
Weiss A, Bernhaupt R, Lankes M (eds) (2009) The USUS evaluation framework for human-robot interaction AISB2009: proceedings of the symposium on new frontiers in human-robot interaction
33.
Weiss A, Wurhofer D, Bernhaupt R, Altmaninger M, Tscheligi M (2010) A methodological adaptation for heuristic evaluation of HRI. In: IEEE RO-MAN, 19th IEEE international symposium on robot and human interactive communication, 13–15 September 2010, Viareggio, Italy, pp 1–6. IEEE, Piscataway
34.
Weiss A, Huber A, Minichberger J, Ikeda M (2016) First application of robot teaching in an existing industry 4.0 environment. Does it really work? Societies 6(3):20
35.
Weistroffer V, Paljic A, Fuchs P, Hugues O, Chodacki J-P, Ligot P, Morais A (2014) Assessing the acceptability of human-robot co-presence on assembly lines: a comparison between actual situations and their virtual reality counterparts. In: RO-MAN, The 23rd IEEE international symposium on robot and human interactive communication, 25–29 August 2014, Edinburgh, Scotland, UK, pp 377–384. IEEE, Piscataway
36.
Yagoda RE, Gillan DJ (2012) You want me to trust a ROBOT? The development of a human–robot interaction trust scale. Int J Soc Robot 4(3):235–248
37.
Brooke J (2013) SUS: a retrospective. J Usability Stud 8(2):29–40
38.
Hart SG, Staveland LE (1988) Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. Adv Psychol 52:139–183
39.
Jian J-Y, Bisantz AM, Drury CG (2000) Foundations for an empirically deter-mined scale of trust in automated systems. Int J Cognit Ergon 4(1):53–71
40.
McBridge SE (2010) The effect of workload and age on compliance with and reliance on an automated system. Georgia Institute of Technology, Ph.D. thesis
41.
Schaefer KE (2013) The perception and measurement of human-robot trust. Ph.D. thesis, University of Central Florida, Florida
42.
Spielberger C, Gorusch R (1970) State-trait anxiety inventory, manual for the state-trait anxiety inventory. Consulting Psychologist Press
43.
Kwee-Meier S, Nelles J, Mertens A, Schlick C (2017, in German) Integrierte Informationsvisualisierungen bei der Mensch-Roboter-Kooperation zur Unterstützung von Einsatzkräften unter erhöhter psychischer Belastung. In: Soziotechnische Gestaltung des digitalen Wandels – kreativ, innovative, sinnhaft. 63. Kongress der Gesellschaft für Arbeitswissenschaft
Metadata
Title
Evaluation Metrics Regarding Human Well-Being and System Performance in Human-Robot Interaction – A Literature Review
Authors
Jochen Nelles
Sonja Th. Kwee-Meier
Alexander Mertens
Copyright Year
2019
Book
Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018)

Print ISBN: 978-3-319-96067-8

Electronic ISBN: 978-3-319-96068-5

Copyright Year: 2019

DOI
https://doi.org/10.1007/978-3-319-96068-5_14

Premium Partner

    Image Credits