Skip to main content
SpringerLink
Log in

Using physiological signals to detect natural interactive behavior

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Many researchers in the Human Robot Interaction (HRI) and Embodied Conversational Agents (ECA) domains try to build robots and agents that exhibit human-like behavior in real-world close encounter situations. One major requirement for comparing such robots and agents is to have an objective quantitative metric for measuring naturalness in various kinds of interactions. Some researchers have already suggested techniques for measuring stress level, awareness etc using physiological signals like Galvanic Skin Response (GSR) and Blood Volume Pulse (BVP). One problem of available techniques is that they are only tested with extreme situations and cannot according to the analysis provided in this paper distinguish the response of human subjects in natural interaction situations. One other problem of the available techniques is that most of them require calibration and some times ad-hoc adjustment for every subject. This paper explores the usefulness of various kinds of physiological signals and statistics in distinguishing natural and unnatural partner behavior in a close encounter situation. The paper also explores the usefulness of these statistics in various time slots of the interaction. Based on this analysis a regressor was designed to measure naturalness in close encounter situations and was evaluated using human-human and human-robot interactions and shown to achieve statistically significant distinction between natural and unnatural situations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Mohammad Y, Xu Y, Matsumura K, Nishida T (2008) The h 3 r explanation corpus:human-human and base human-robot interaction dataset. In: The fourth international conference on intelligent sensors, sensor networks and information processing (ISSNIP2008)

  2. Qazi Z, Wang Z, Haq I (2006) Human likeness of humanoid robots exploring the uncanny valley. In: International conference on emerging technologies, ICET’06, pp 650–656

  3. Shi Y, Choi EHC, Ruiz N, Chen F, Taib R (2007) Galvanic skin respons (gsr) as an index of cognitive load. In: CHI 2007, pp 2651–2656

  4. Mandryk RL, Inkpen KM (2004) Physiological indicators for the evaluation of co-located collaborative play. In: CSCW’04

  5. Lin T, Hu W, Omata M, Imamiya A (2005) Do physiological data relate to traditional usability indexes? In: OZCHI 2005

  6. Lang PJ (1995) The emotion probe: studies of motivation and attention. Am Psychologiest 50(5):372–285

    Article  Google Scholar 

  7. Mower E, Feil-Seifer DJ, Mataric MJ, Narayanan S (2007) Investigating implicit cues for user state estimation in human-robot interaction using physiological measurements. In: 16th international conference on robot & human interactive communication, pp 1125–1130

  8. Bradley MPL (2000) Measuring emotions: behavior, feeling and physiology. In: Cognitive neuroscience of emotion. Oxford University Press, New York

    Google Scholar 

  9. Argyle M (2001) Bodily Communication, new edn. Routledge, London

    Google Scholar 

  10. Mohammad Y, Nishida T (2007) TalkBack: Feedback from a miniature robot. In: AI 2007: Advances in artificial intelligence, 20th Australian joint conference on artificial intelligence, Gold Coast, Australia, December 2007, pp 357–366

  11. Papillo JF, Shapiro D (1990) The cardiovascular system. In: Principles of psychophysiology: physical, social, and inferential elements. Cambridge University Press, Cambridge

    Google Scholar 

  12. Rowe DW, Sibert J, Irwin D (1998) Heart rate variability: indicator of user states an aid to human-computer interaction. In: Conference on human factors in computing systems (CHI98)

  13. Kidd CD, Breazeal C (2005) Human-robot interaction experiments: lessons learned. In: Robot companions: hard problems and open challenges in robot human interaction symposioum of social intelligence and interaction in animals robots and agents, pp 141–142

  14. Safavian SR, Landgrebe D (1991) A survey of decision tree classifier methodology. IEEE Trans Syst Man Cybern 21(3):660–674

    Article  MathSciNet  Google Scholar 

  15. Steinwart I, Christmann A (2008) Support vector machines. Springer, Berlin

    MATH  Google Scholar 

  16. Rosner B (1975) On the detection of many outliers. Technometrics 17(2):221–227

    Article  MATH  MathSciNet  Google Scholar 

  17. Mohammad Y, Nishdia T (2008) Human adaptation to a miniature robot: Precursors of mutual adaptation. In: IEEE 17th international symposium on robot and human interactive communication (IEEE Ro-Man), pp 124–129

  18. Liao W, Zhang W, Zhu Z, Ji O (2005) A decision theoretic model for stress recognition and user assistance. In: AAAI 2005, pp 529–534

  19. Stern RM, Ray WJ, Quigley KS (2001) Psychological recording, 2nd edn. Oxford University Press, Oxford

    Google Scholar 

  20. Mohammad Y, Nishida T (2009) Robust singular spectrum transform. In: IEA/AIE, pp 123–132

  21. Mohammad Y, Nishida T (2009) Measuring naturalness during close encounters using physiological signal processing. In: IEA/AIE, pp 281–290

Download references

Author information

Authors and Affiliations

  1. Electrical Engineering Department, Assiut University, Assiut, Egypt

    Yasser Mohammad

  2. Graduate School of Informatics, Kyoto University, Kyoto, Japan

    Toyoaki Nishida

Authors
  1. Yasser Mohammad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Toyoaki Nishida
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yasser Mohammad.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Mohammad, Y., Nishida, T. Using physiological signals to detect natural interactive behavior. Appl Intell 33, 79–92 (2010). https://doi.org/10.1007/s10489-010-0241-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-010-0241-4

Keywords

Search

Navigation