Kunal Gupta
ABSTRACT
Human trust is a psycho-physiological state that is difficult to measure, yet is becoming increasingly important for the design of human-computer interactions. This paper explores if human trust can be measured using physiological measures when interacting with a computer interface, and how it correlates with cognitive load. In this work, we present a pilot study in Virtual Reality (VR) that uses a multi-sensory approach of Electroencephalography (EEG), galvanic skin response (GSR), and Heart Rate Variability (HRV) to measure trust with a virtual agent and explore the correlation between trust and cognitive load. The goal of this study is twofold; 1) to determine the relationship between biosignals, or physiological signals with trust and cognitive load, and 2) to introduce a pilot study in VR based on cognitive load level to evaluate trust. Even though we could not report any significant main effect or interaction of cognitive load and trust from the physiological signal, we found that in low cognitive load tasks, EEG alpha band power reflects trustworthiness on the agent. Moreover, cognitive load of the user decreases when the agent is accurate regardless of task’s cognitive load. This could be possible because of small sample size, tasks not stressful enough to induce high cognitive load due to lab study and comfortable environment or timestamp synchronisation error due to fusing data from various physiological sensors with different sample rate.
Supplemental Material
- Mohamed Abouelenien, Mihai Burzo, Rada Mihalcea, Kristen Rusinek, and David Van Alstine. 2017. Detecting Human Thermal Discomfort via Physiological Signals. In Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments(PETRA ’17). ACM, New York, NY, USA, 146–149. https://doi.org/10.1145/3056540.3064957Google Scholar
Digital Library
- Kumar Akash, Wan-Lin Hu, Neera Jain, and Tahira Reid. 2018. A classification model for sensing human trust in machines using eeg and gsr. ACM Transactions on Interactive Intelligent Systems (TiiS) 8, 4(2018), 27.Google Scholar
- Kumar Akash, Wan-Lin Hu, Tahira Reid, and Neera Jain. 2017. Dynamic modeling of trust in human-machine interactions. In 2017 American Control Conference (ACC). IEEE, 1542–1548.Google ScholarCross Ref
- Pavlo Antonenko, Fred Paas, Roland Grabner, and Tamara Van Gog. 2010. Using electroencephalography to measure cognitive load. Educational Psychology Review 22, 4 (2010), 425–438.Google ScholarCross Ref
- Olivier Augereau, Benjamin Tag, and Koichi Kise. 2018. Mental State Analysis on Eyewear. In Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers(UbiComp ’18). ACM, New York, NY, USA, 968–973. https://doi.org/10.1145/3267305.3274119Google ScholarDigital Library
- Bernard Barber. 1983. The logic and limits of trust. Vol. 96. Rutgers University Press New Brunswick, NJ.Google Scholar
- Mathias Benedek and Christian Kaernbach. 2010. A continuous measure of phasic electrodermal activity. Journal of neuroscience methods 190, 1 (2010), 80–91.Google ScholarCross Ref
- Alanah Davis, John D Murphy, Dawn Owens, Deepak Khazanchi, and Ilze Zigurs. 2009. Avatars, people, and virtual worlds: Foundations for research in metaverses. Journal of the Association for Information Systems 10, 2 (2009), 90.Google ScholarCross Ref
- Arindam Dey, Alex Chatburn, and Mark Billinghurst. 2019. Exploration of an EEG-Based Cognitively Adaptive Training System in Virtual Reality. (2019).Google Scholar
- Suh-Yeon Dong, Bo-Kyeong Kim, Kyeongho Lee, and Soo-Young Lee. 2015. A Preliminary Study on Human Trust Measurements by EEG for Human-Machine Interactions. In Proceedings of the 3rd International Conference on Human-Agent Interaction(HAI ’15). ACM, New York, NY, USA, 265–268. https://doi.org/10.1145/2814940.2814993Google ScholarDigital Library
- Jérémy Frey, Maxime Daniel, Julien Castet, Martin Hachet, and Fabien Lotte. 2016. Framework for electroencephalography-based evaluation of user experience. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 2283–2294.Google ScholarDigital Library
- Lynda Gerry, Barrett Ens, Adam Drogemuller, Bruce Thomas, and Mark Billinghurst. 2018. Levity: A Virtual Reality System That Responds to Cognitive Load. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems(CHI EA ’18). ACM, New York, NY, USA, Article LBW610, 6 pages. https://doi.org/10.1145/3170427.3188479Google ScholarDigital Library
- Google Glass [n.d.]. Google Glass - Glass Partners. https://www.google.com/glass/partners/Google Scholar
- David Grimes, Desney S Tan, Scott E Hudson, Pradeep Shenoy, and Rajesh PN Rao. 2008. Feasibility and pragmatics of classifying working memory load with an electroencephalograph. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 835–844.Google ScholarDigital Library
- Jacek Gwizdka. 2010. Using Stroop task to assess cognitive load. In Proceedings of the 28th Annual European Conference on Cognitive Ergonomics. ACM, 219–222.Google ScholarDigital Library
- Joanna Hale and F De C Antonia. 2016. Testing the relationship between mimicry, trust and rapport in virtual reality conversations. Scientific reports 6(2016), 35295.Google Scholar
- Peter A Hancock, Deborah R Billings, Kristin E Schaefer, Jessie YC Chen, Ewart J De Visser, and Raja Parasuraman. 2011. A meta-analysis of factors affecting trust in human-robot interaction. Human factors 53, 5 (2011), 517–527.Google Scholar
- Sandra G Hart. 1986. NASA Task load Index (TLX). Volume 1.0; Paper and pencil package. (1986).Google Scholar
- Blanca Hernandez-Ortega. 2011. The role of post-use trust in the acceptance of a technology: Drivers and consequences. Technovation 31, 10-11 (2011), 523–538.Google ScholarCross Ref
- Leanne M Hirshfield, Erin Treacy Solovey, Audrey Girouard, James Kebinger, Robert JK Jacob, Angelo Sassaroli, and Sergio Fantini. 2009. Brain measurement for usability testing and adaptive interfaces: an example of uncovering syntactic workload with functional near infrared spectroscopy. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2185–2194.Google ScholarDigital Library
- Kevin Anthony Hoff and Masooda Bashir. 2015. Trust in automation: Integrating empirical evidence on factors that influence trust. Human factors 57, 3 (2015), 407–434.Google ScholarCross Ref
- Wan-Lin Hu, Kumar Akash, Neera Jain, and Tahira Reid. 2016. Real-time sensing of trust in human-machine interactions. IFAC-PapersOnLine 49, 32 (2016), 48–53.Google ScholarCross Ref
- Sirkka L Jarvenpaa, Kathleen Knoll, and Dorothy E Leidner. 1998. Is anybody out there? Antecedents of trust in global virtual teams. Journal of management information systems 14, 4 (1998), 29–64.Google ScholarDigital Library
- Ole Jensen and Claudia D Tesche. 2002. Frontal theta activity in humans increases with memory load in a working memory task. European journal of Neuroscience 15, 8 (2002), 1395–1399.Google ScholarCross Ref
- Jiun-Yin Jian, Ann M Bisantz, and Colin G Drury. 2000. Foundations for an empirically determined scale of trust in automated systems. International Journal of Cognitive Ergonomics 4, 1 (2000), 53–71.Google ScholarCross Ref
- Ahmad Khawaji, Jianlong Zhou, Fang Chen, and Nadine Marcus. 2015. Using galvanic skin response (GSR) to measure trust and cognitive load in the text-chat environment. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 1989–1994.Google ScholarDigital Library
- Wayne K Kirchner. 1958. Age differences in short-term retention of rapidly changing information.Journal of experimental psychology 55, 4 (1958), 352.Google Scholar
- Juyoung Lee, Hui-Shyong Yeo, Murtaza Dhuliawala, Jedidiah Akano, Junichi Shimizu, Thad Starner, Aaron Quigley, Woontack Woo, and Kai Kunze. 2017. Itchy Nose: Discreet Gesture Interaction Using EOG Sensors in Smart Eyewear. In Proceedings of the 2017 ACM International Symposium on Wearable Computers(ISWC ’17). ACM, New York, NY, USA, 94–97. https://doi.org/10.1145/3123021.3123060Google ScholarDigital Library
- Rachel McDonnell and Martin Breidt. 2010. Face reality: investigating the uncanny valley for virtual faces. In ACM SIGGRAPH ASIA 2010 Sketches. ACM, 41.Google ScholarDigital Library
- Daniel J. McDuff, Javier Hernandez, Sarah Gontarek, and Rosalind W. Picard. 2016. COGCAM: Contact-free Measurement of Cognitive Stress During Computer Tasks with a Digital Camera. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems(CHI ’16). ACM, New York, NY, USA, 4000–4004. https://doi.org/10.1145/2858036.2858247Google ScholarDigital Library
- Bonnie M Muir. 1987. Trust between humans and machines, and the design of decision aids. International journal of man-machine studies 27, 5-6 (1987), 527–539.Google ScholarDigital Library
- OpenBCI [n.d.]. OpenBCI - Open Source Brain Computer Interfaces. https://shop.openbci.com/collections/frontpage/products/openbci-eeg-electrocap-kitGoogle Scholar
- Oxford Medical Simulation [n.d.]. Oxford Medical Simulation. https://oxfordmedicalsimulation.com/Google Scholar
- Yun Suen Pai. 2016. Physiological Signal-Driven Virtual Reality in Social Spaces. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology(UIST ’16 Adjunct). ACM, New York, NY, USA, 25–28. https://doi.org/10.1145/2984751.2984787Google ScholarDigital Library
- J. Posada, C. Toro, I. Barandiaran, D. Oyarzun, D. Stricker, R. de Amicis, E. B. Pinto, P. Eisert, J. Döllner, and I. Vallarino. 2015. Visual Computing as a Key Enabling Technology for Industrie 4.0 and Industrial Internet. IEEE Computer Graphics and Applications 35, 2 (2015), 26–40.Google ScholarDigital Library
- Lingyun Qiu and Izak Benbasat. 2005. Online consumer trust and live help interfaces: The effects of text-to-speech voice and three-dimensional avatars. International journal of human-computer interaction 19, 1(2005), 75–94.Google Scholar
- John K Rempel, John G Holmes, and Mark P Zanna. 1985. Trust in close relationships.Journal of personality and social psychology 49, 1(1985), 95.Google Scholar
- David Rozado and Andreas Dunser. 2015. Combining EEG with pupillometry to improve cognitive workload detection. Computer 48, 10 (2015), 18–25.Google ScholarDigital Library
- Davide Salanitri, Glyn Lawson, and Brian Waterfield. 2016. The Relationship Between Presence and Trust in Virtual Reality. In Proceedings of the European Conference on Cognitive Ergonomics(ECCE ’16). ACM, New York, NY, USA, Article 16, 4 pages. https://doi.org/10.1145/2970930.2970947Google ScholarDigital Library
- Katarzyna Samson and Patrycjusz Kostyszyn. 2015. Effects of cognitive load on trusting behavior–an experiment using the trust game. PloS one 10, 5 (2015), e0127680.Google ScholarCross Ref
- Shimmer [n.d.]. Shimmer GSR+ Sensor. https://www.shimmersensing.com/products/shimmer3-wireless-gsr-sensorGoogle Scholar
- Sharon Stansfield, Daniel Shawver, Annette Sobel, Monica Prasad, and Lydia Tapia. 2000. Design and Implementation of a Virtual Reality System and Its Application to Training Medical First Responders. Presence: Teleoperators and Virtual Environments 9, 6(2000), 524–556.Google ScholarDigital Library
- D Susan and John G Holmes. 1991. The dynamics of interpersonal trust: Resolving uncertainty in the face of risk. Cooperation and Prosocial Behavior; Cambridge University Press: New York, NY, USA (1991), 190.Google Scholar
- Wioleta Szwoch. 2015. Emotion Recognition Using Physiological Signals. In Proceedings of the Mulitimedia, Interaction, Design and Innnovation(MIDI ’15). ACM, New York, NY, USA, Article 15, 8 pages. https://doi.org/10.1145/2814464.2814479Google ScholarDigital Library
- Mika P Tarvainen, Juha-Pekka Niskanen, Jukka A Lipponen, Perttu O Ranta-Aho, and Pasi A Karjalainen. 2014. Kubios HRV–heart rate variability analysis software. Computer methods and programs in biomedicine 113, 1(2014), 210–220.Google Scholar
- Unity [n.d.]. Unity. https://unity3d.com/unity/whats-new/2019.2.4Google Scholar
- Vive [n.d.]. Vive. https://www.vive.com/us/product/vive-virtual-reality-system/Google Scholar
- Willem I. M. Willaert, Rajesh Aggarwal, Isabelle Van Herzeele, Nicholas J. Cheshire, and Frank E. Vermassen. 2012. Recent Advancements in Medical Simulation: Patient-Specific Virtual Reality Simulation. World Journal of Surgery 36, 7 (2012), 1703–1712.Google ScholarCross Ref
- Lian Zhang, Joshua Wade, Dayi Bian, Jing Fan, Amy Swanson, Amy Weitlauf, Zachary Warren, and Nilanjan Sarkar. 2017. Cognitive load measurement in a virtual reality-based driving system for autism intervention. IEEE transactions on affective computing 8, 2 (2017), 176–189.Google ScholarDigital Library
Recommendations
Using Galvanic Skin Response (GSR) to Measure Trust and Cognitive Load in the Text-Chat Environment
CHI EA '15: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing SystemsExchanging text messages via software on smart phones and computers has recently become one of the most popular ways for people to communicate and accomplish their tasks. However, there are negative aspects to using this kind of software, for example, ...
Trust and Cognitive Load in semi-automated UAV operation
OzCHI '19: Proceedings of the 31st Australian Conference on Human-Computer-InteractionTrust in automation is an essential precursor to system adoption and use. Given the emerging wave of autonomous systems available for public consumption and the resources devoted to this trend, it's important to understand trust, and how to measure it. ...
Trust and cognitive load in the text-chat environment: the role of mouse movement
OzCHI '14: Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: the Future of DesignThis paper examines how different levels of cognitive load can affect trust in the text-chat environment. It also examines how the mouse movements of participants can indicate the level of cognitive load when they chat with each other. We designed two ...
Comments