Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Bücher
Zeitschriften
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
ATZ-Fachkongress

Chassis.tech plus 2024


4 Kongresse in einer Veranstaltung

Treffen Sie in München die Fachleute von Chassis, Lenkung, Bremsen sowie Reifen/Räder.
Erweiterte Suche
Anmelden
nach oben
Journal on Multimodal User Interfaces
Erschienen in: Journal on Multimodal User Interfaces 2/2017

03.02.2017 | Original Paper

The impact of human–robot multimodal communication on mental workload, usability preference, and expectations of robot behavior

verfasst von: Julian Abich IV, Daniel J. Barber

Erschienen in: Journal on Multimodal User Interfaces | Ausgabe 2/2017

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Multimodal communication between humans and autonomous robots is essential to enhance effectiveness of human–robot team performance in complex, novel environments, such as in military intelligence, surveillance, and reconnaissance operations in urban settings. It is imperative that a systematic approach be taken to evaluate the factors that each modality contributes to the user’s ability to perform successfully and safely. This paper addresses the effects of unidirectional speech and gesture methods of communication on perceived workload, usability preferences, and expectations of robot behavior while commanding a robot teammate to perform a spatial-navigation task. Each type of communication was performed alone or simultaneously. Results reveal that although the speech-alone condition elicited the lowest level of perceived workload, the usability preference and expectations of robot behavior after interacting through each communication condition was the same. Further, workload ratings between the gesture and speech-gesture conditions were similar indicating systems that employ gesture communication could also support speech communication with little to no additional subjectively perceived cognitive burden on the user. Findings also reveal that workload alone should not be used as a sole determining factor of communication preference during system and task evaluation and design. Additionally, perceived workload did not seem to negatively impact the level of expectations regarding the robot’s behavior. Recommendations for future human–robot communication evaluation are provided.
Vorheriger Artikel Providing plasticity and redistribution for 3D user interfaces using the D3PART model
Nächster Artikel Extraction of texture and geometrical features from informative facial regions for sign language recognition

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Literatur
1.
Phillips E, Ososky S, Grove J, Jenstch F (2011) From tools to teammates: toward the development of appropriate mental models for intelligent robots. In: Proceedings from the human factors and ergonomics society 55th annual meeting, pp 1491–1495
2.
Redden E, Elliott l, Barnes M (2013) Robots: the new team members. In: Coovert M, Thompson L (eds) The psychology of workplace technology. Society of industrial organizational psychology frontiers series, Routledge Press
3.
Barber D, Lackey S, Reinerman-Jones L, Hudson I (2013) Visual and tactile interfaces for bi-directional human robot communication. SPIE defense, security, and sensing. International society for optics and photonics, pp 87410U–87410U
4.
5.
Gleeson B, Maclean K, Haddadi A, Croft E, Alcazar J (2013) Gestures for industry: intuitive human–robot communication from human observation. In: Proceedings from the 8th ACM/IEEE intenational conference on human–robot interaction. IEEE Press, Piscataway, pp 349–356
6.
Harriott C, Zhang T, Adams J (2011) Evaluating the applicability of current models of workload to peer-based human–robot teams. In: Proceedings from the 6th international conference on human-robot interaction, New York, ACM, pp 45–52
7.
Medicherla H, Sekmen A (2007) Human–robot interaction via voice-controllable intelligent user interface. Robotica 25(5):521–527CrossRef
8.
Neisser U (1976) Cognition and reality. W.H. Freeman, San Francisco
9.
Lackey SJ, Barber DJ, Reinerman-Jones L, Badler N, Hudson I (2011) Defining next-generation multi-modal communication in human–robot interaction. Human factors and ergonomics society conference. HFES, Las Vegas
10.
Oviatt S (2012) Multimodal interfaces. In: Jacko J (ed) Handbook of human–computer interaction, 3rd edn. Lawrence Erlbaum, Mahwah
11.
Barber D, Reinerman-Jones L, Matthews G (2014) Toward a tactile language for human–robot interaction: two studies of tacton learning and performance. Hum Factors. doi:10.​1177/​0018720814548063​
12.
Jaimes A, Sebe N (2007) Multimodal human–computer interaction: a survey. Comput Vis Image Underst 108(1):116–134CrossRef
13.
Alonso-Martin F, Castro-González Á, Gorostiza J, Salichs M (2013) Multidomain voice activity detection during human–robot interaction. In: Herrmann MPG, Bremmer P, Spiers A, Leonards U (eds) Social robotics. Springer International Publishing, Bristol, pp 64–73CrossRef
14.
Kamm C, Walker M, Rabiner L (1997) The role of speech processing in human–computer intelligent communication. Speech Commun 23:263–278CrossRef
15.
Breazeal C, Aryananda L (2002) Recognition of affective communicative intent in robot-directed speech. Auton Robots 12:83–104CrossRefMATH
16.
17.
Tikhanoff V, Cangelosi A, Metta G (2011) Integration of speech and action in humanoid robots: iCub simulation experiments. IEEE Trans Auton Mental Dev 3(1):17–29CrossRef
18.
Erickson D, DeWees M, Lewis J, Matson E (2012) Communication for task completion with heterogeneous robots. In Kim J-H, Matson E, Myung H, Xu P (eds) Robot intelligence technology and applications 2012, vol 208, pp 873–882
19.
Pourmehr S, Monajjemi V, Vaughn R, Mori G. (2013) “You two! Take off!”: creating, modifying and commanding groups of robots using face engagement and indirect speech in voice commands. In: Proceedings from IEEE/RSJ international conference on intelligent robots and systems 2013, pp 137–142. IEEE
20.
Stiefelhagen R, Ekenel HK, Fügen C, Gieselmann P, Holzapfel H, Kraft F, Nickel K, Voit M, Waibel A (2007) Enabling multimodal human–robot interaction for the Karlsruhe humanoid robot. IEEE Trans Robot 23(5):840–851CrossRef
21.
Pettitt R, Redden E, Carsten C (2009) Scalablity of robotic controllers: speech-based robotic controller evaluation (ARL-TR-4858). US Army Research Laboratory, Aberdeen Proving Ground
22.
Pettitt R, Carstens C, Elliot L (2014) Speech-based robotic control for dismounted soldiers: evaluation of visual display options. Army Research Laboratory, Department of Defense, USA
23.
Kennedy W, Bugajska M, Marge M, Adams W, Fransen B, Perzanowski D, Schultz AC, Trafton G (2007) Spatial representation and reasoning for human–robot collaboration. In: Proceedings of the 22nd conference on artificial intelligence. AAAI Press, Vancouver, pp 1554–1559
24.
Harris J, Barber D (2014) Speech and gesture interfaces for squad level human robot teaming. In: Proceedings from SPIE 9084: Unmanned Systems Technology XVI, 90840B. Baltimore
25.
Redden E, Carstens C, Pettitt R (2010) Intuitive speech-based robotic control. Army Research Lab, Department of Defense, USA
26.
Jung S-W, Sung K-W, Park M-Y, Kang E-U, Hwang W-J, Won J-D et al (2013) A study on robust control of mobile robot by voice command. In: Proceedings of the 13th international conference on control, automation and systems (ICCAS), pp 657–659. Gwangju. doi:10.​1109/​ICCAS.​2013.​6703950
27.
Pigeon S, Swail C, Geoffrois E, Bruckner G, van Leeuwen D, Teixeira C et al (2005) Use of speech and language technology in military environments. Res Technol Organ
28.
Ekman P (2004) Emotional and conversational nonverbal signals. In: Larrazabal M, Miranda L (eds) Language, knowledge, and representation. Kluwer Academic Publishers, Amsterdam, pp 39–50CrossRef
29.
Mehrabian A (1972) Nonverbal communication. Aldine Transaction, New Brunswick
30.
Ekman P, Friesen W (1968) Nonverbal behavior in psychotherapy research. In: Shlien J (ed) Research in psychotherapy conference. American Psychological Association, Washington, pp 179–216CrossRef
31.
Efron D (1941) Gesture and environment. King’s Crown, New York
32.
Stajonov A (2009) Gesture-based human–computer interaction [Doctoral dissertation]. Jacobs University, Bremen
33.
Burger B, Ferrané I, Lerasle F, Infantes G (2012) Two-handed gesture recognition and fusion with speech to command a robot. Auton Robot 32(1):129–147CrossRef
34.
Barber D, Abich J IV, Phillips E, Talone A, Jentsch F, Hill S (2015) Field assessment of multimodal communication for dismounted human–robot teams. In: Proceedings of the 59th human factors and ergonomics society 2014, Los Angeles
35.
Boonpinon N, Sudsng A (2008) Formation control for multi-robot teams using a data glove. In: Proceedings from the IEEE conference on robotics, automation, and mechatronics Chengdu, IEEE pp 525–531
36.
Hill S, Barber D, Evans A (2015) Achieving the vision of effective Soldier–robot teaming: recent work in multimodal communication. In: HRI ’15 Extended Abstracts. Tenth annual ACM/IEEE international conference on human–robot interaction, ACM, Portland, pp 177–178
37.
Rogalla O, Ehrenmann M, Zöllner R, Becher R, Dillmann R (2002) Using gesture and speech control for commanding a robot assistant. In: Proceedings of the 11th IEEE international workshop on robot and human interactive communication, pp 454–459. IEEE
38.
Cheng L, Sun Q, Su H, Cong Y (2012). Design and implementation of human–robot interactive demonstrationn system based on Kinect. In: IEEE 24th Chinese: control and decision conference, pp 971–975. IEEE
39.
Tran N, Phan H, Dinh V, Ellen J, Berg B, Lum J et al (2009) Wireless data glove for gesture-based robotic control. In Jacko J (ed) Human–computer interaction. Novel interaction methods and techniques: 13th international conference, HCI International 2009 Springer, Berlin, vol 5611, pp 271–280
40.
Wachs J, Kölsch M, Stern H, Edan Y (2011) Vision-based hand-gesture applications. Commun ACM 54(2):60–71CrossRef
41.
Dumas B, Lalanne D, Oviatt S (2009) Multimodal interfaces: a survey of principles, models, and frameworks. In: Lalanne D, Kohlas J (eds) Human machine interaction: lecture notes in computer science. Springer, Berlin Heidelberg, Berlin, pp 3–26CrossRef
42.
Sarter N (2006) Multiple-resource theory as a basis for multimodal interface design: success stories, qualifications, and research needs. In: Kramer A, Wiegmann D, Kirlik A (eds) Attention: from theory to practice. Oxford University Press, New York, pp 187–195CrossRef
43.
Wickens C (2002) Multiple resources and performance prediction. Theor Issues Ergon Sci 3(2):159–177CrossRef
44.
Vitense H, Jacko J, Emery V (2003) Multimodal feedback: an assessment of performance and mental workload. Ergonomics 46(1–3):68–87CrossRef
45.
Perzanowski D, Adams W, Schultz A, Marsh E (2000) Towards seamless integration in a multi-modal interface. Naval Research Lab, Department of Defense, USA
46.
Perzanowski D, Schultz AC, Adams W, Marsh E (2000) Using a natural language and gesture interface for unmanned vehicles. In: AeroSense 2000. International society for optics and photonics, Bellingham, Washington, USA, pp 341–347
47.
Hart S, Staveland L (1988) Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. In: Hancock P, Meshkati N (eds) Human mental workload. North Holland Press, Amsterdam
48.
Matthews G, Reinerman-Jones L, Barber D, Abich J IV (2014). The psychometrics of mental workload: multiple measures are sensitive, but divergent. Hum Factors
49.
Abich J IV, Matthews G, Reinerman-Jones L, Barber D (2015) Predicting performance and workload from baseline and concurrent task measures. In: Proceedings of the human factors and ergonomics society annual meeting, vol 59, Sage CA: Los Angeles, CA: SAGE Publications, pp 1676–1680
50.
Pauzié A, Manzano J (2007) Evaluation of driver mental workload facing new in-vehicle information and communication technology. In: Proceedings of the 20th enhanced safety of vehicles conference, Lyon
51.
Pitt I, Edwards A (1997) An improved auditory interface for the exploration of lists. In: Proceedings of the fifth ACM international conference on multimedia, Seattle, pp. 51–61
52.
White T, Kehring K, Glumm M (2009) Effects of unimodal and multimodal cues about threat locations on target acquisition and workload. Mil Psychol 21:497–512CrossRef
53.
Bolt R (1980) Put-that-there: voice and gesture at the graphics interface. Comput Graph 14(3):262–270CrossRef
54.
Leventhal L, Barnes J (2008) Usability engineering: process, products, and examples. Pearson Prentice Hall, Upper Saddle River
55.
Lingard L, Reznick R, Espin S, Regehr G, DeVito I (2002) Team communications in the operating room: talk patterns, sites of tension, and implications for novices. Acad Med 77(3):232–237CrossRef
56.
Lohse M (2010) Investigating the influence of situations and expectations on user behavior: empirical analyses in human–robot interaction [Doctoral dissertation]. Univeristy of Bielefeld, Bielefeld
57.
Robins B, Dautenhahn K, Boekhorst Rt, Billard A (2004) Effects of repeated exposure to a humanoid robot on children with autism. In: Proceedings from the universal access and assistive technology conference, pp. 225–236
58.
Komatsu T, Kurosawa R, Yamada S (2012) How does the difference bewteen user’s expectations and perceptions about a robotic agent affect their behavior? Int J Soc Robot 4(2):109–116CrossRef
59.
Olson J, Roese N, Zanna M (1996) Expectancies. In: Higgins E, Kruglanski A (eds) Social psychology: handbook of basic principles. Guilford Press, New York, pp 211–238
60.
Norman D (1999) Affordances, conventions and design. Interactions 6(3):38–43CrossRef
61.
Heckhausen H (1977) Achievement motivation and its constructs: a cognitive model. Motiv Emot 1(4):283–329CrossRef
62.
Roese NJ, Sherman JW (2007) Expectancy. In: Kruglanski AW, Higgins ET (eds) Social psychology. Handbook of basic principles. Guilford Press, New York, pp 91–115
63.
Kelley H (1950) The warm–cold variable in first impressions of persons. J Personal 18:431–439CrossRef
64.
Hancock P (2009) Mind, machine, and morality. Ashgate, Chichester
65.
Lin P, Bekey G, Abney K (2008) Autonomous military robotics: risk, ethics, and design. Department of Navy, Office of Naval Research, USA
66.
67.
Fechner G (1876) Vorschule der aesthetik. Breitkopf & Härtel, Leipzig
68.
Titchener E (1910) Textbook of psychology. Macmillan, New York
69.
Robins B, Dautenhahn K, te Boekhorst R, Billard A (2004) Effects of repeated exposure to a humanoid robot on children with autism. In: Keates S, Clarkson J, Langdon P, Robinson P (eds) Designing a more inclusive world. Springer, London, pp 225–236CrossRef
70.
Takayama L, Pantofaru C (2009) Influences on proxemic behaviors in human–robot interaction. In: Proceedings of the international conference on intelligent robots and systems (IROS 2009), St. Louis, MO, pp 5495–5502
71.
Bartneck C, Suzuki T, Kanda T, Nomura T (2007) The influence of people’s culture and prior experiences with Aibo on their attitude towards robots. AI Soc 21(1):217–230
72.
73.
Ekstrom R, French J, Harman H (1979) Cognitive factors: their identification and replication. Multivar Behav Res Monogr 79(2)
74.
Brooke J (1996) SUS—a quick and dirty usability scale. Usability Eval Ind 189(194):4–7
75.
Lohse M (2011) Bridging the gap between users’ expectations and system evaluations. In: 20th IEEE international symposium on robot and human interactive communication, Altanta, pp 485–490
76.
Abich J IV, Barber D, Reinerman-Jones L (2015) Experimental environments for dismounted human–robot multimodal communications. In: Shumaker R, Lackey S (eds) Virtual, augmented and mixed reality: 7th international conference, VAMR 2015, Held as Part of HCI International 2015, vol 9179. Springer International Publishing, Los Angeles, CA, USA, pp 165–173
77.
Frick-Horbury D (2002) The effects of hand gestures on verbal recall as a function of high- and low-verbal skill levels. Gen Psychol 129(2):137–147
78.
Frick-Horbury D (2002) The use of hand gestures as self-generated cues for recall of verbally associated targets. Am J Psychol 115(1):1–20CrossRef
79.
Krauss R, Chen Y, Chawla P (1996) Nonverbal behavior and nonverbal communication: what do conversational hand gestures tell us? In: Zanna M (ed) Advances in experimental social psychology. Academic Press, Tampa
80.
Krauss R, Hadar U (2001) The role of speech-related arm/hand gestures in word retrieval. In: Campbell R, Messing L (eds) Gestures, speech, and sign. Oxford University Press, Oxford, pp 93–116
81.
Hall E (1990) The hidden dimension. Anchor Books, New York
82.
Graham J, Argyle M (1975) A cross-cultural study of the communication of exta-verbal meaning by gestures. Int J Psychol 10(1):57–67CrossRef
83.
Beatty MJ, Friedland MH (1990) Public speaking state anxiety as a function of selected situational and predispositional variables. Commun Educ 39(2):142–147CrossRef
84.
Kagitcibasi C (2005) Autonomy and relatedness in cultural context. J Cross Cult Psychol 36(4):403–422CrossRef
Metadaten
Titel
The impact of human–robot multimodal communication on mental workload, usability preference, and expectations of robot behavior
verfasst von
Julian Abich IV
Daniel J. Barber
Publikationsdatum
03.02.2017
Verlag
Springer International Publishing
Erschienen in
Journal on Multimodal User Interfaces / Ausgabe 2/2017
Print ISSN: 1783-7677
Elektronische ISSN: 1783-8738
DOI
https://doi.org/10.1007/s12193-016-0237-4

Weitere Artikel der Ausgabe 2/2017

Journal on Multimodal User Interfaces 2/2017 Zur Ausgabe

Original Paper

Diegetic user interfaces for virtual environments with HMDs: a user experience study with oculus rift

Original Paper

Multimodal HCI: exploratory studies on effects of first impression and single modality ratings in retrospective evaluation

Original Paper

Design and application of 2D illusory vibrotactile feedback for hand-held tablets

Survey

An insight into assistive technology for the visually impaired and blind people: state-of-the-art and future trends

Original Paper

Extraction of texture and geometrical features from informative facial regions for sign language recognition

OriginalPaper

Cluster-based approach to discriminate the user’s state whether a user is embarrassed or thinking to an answer to a prompt

Premium Partner

    Bildnachweise