Top

Cognitive Computation

Published in:

01-04-2016

Mobile Conversational Agents for Context-Aware Care Applications

Authors: David Griol, Zoraida Callejas

Published in: Cognitive Computation | Issue 2/2016

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Smart mobile devices have fostered new interaction scenarios that demand sophisticated interfaces. The main developers of operating systems for such devices provide APIs for developers to implement their own applications, including different solutions for graphical interfaces, sensor control, and voice interaction. Despite the usefulness of such resources, there are no strategies defined for coupling the multimodal interface with the possibilities that the devices offer to identify and adapt to the user needs, which is particularly important in domains such as Ambient Assisted Living. In this paper, we propose a framework that allows developing context-aware multimodal conversational agents that dynamically incorporate user-specific requirements and preferences as well as characteristics about the interaction environment, in order to improve and personalize the service that is provided. Our proposal integrates the facilities of the Android API in a modular architecture that emphasizes interaction management and context-awareness to build user-adapted, robust and maintainable applications. As a proof of concept, we have used the proposed framework to develop an Android app for older adults suffering from Alzheimer's. The app helps them to preserve their cognitive abilities and enhance their relationship with their environment.

previous article An Interactive Fuzzy Inference System for Teletherapy of Older People

next article Software Architecture for Smart Emotion Recognition and Regulation of the Ageing Adult

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

https://www.ispeech.org/developers/android.

http://www.w3.org/TR/soap.

http://docs.oasis-open.org/ws-caf/ws-context/v1.0/wsctx.

http://www.alzheimersblog.org.

Ábalos N, Espejo G, López-Cózar R, Callejas Z, Griol D. A multimodal dialogue system for an ambient intelligent application in home environments. Lect Notes Artif Intell. 2010;6231:484–91.

Ahmad F, Hogg-Johnson S, Stewart D, Skinner H, Glazier R, Levinson W. Computer-assisted screening for intimate partner violence and control: a randomized trial. Ann Intern Med. 2009;151(2):93–102.CrossRefPubMed

Allen J, Ferguson G, Blaylock N, Byron D, Chambers N, Dzikovska M, Galescu L, Swift M. Chester: towards a personal medication advisor. J Biomed Inf. 2006;39(5):500–13.CrossRef

Almeida DD, Baptista CDS, Silva ED, Campelo C, Figueiredo HD, Lacerda Y. A context-aware system based on service-oriented architecture. 2006. In: Proceedings of AINA’06, pp. 205–210.

Andre E, Bevacqua E, Heylen D, Niewiadomski R, Pelachaud C, Peters C, Poggi I, Rehm M. Non-verbal persuasion and communication in an affective agent. In: Emotion oriented systems. The humaine handbook. Cognitive technologies. Berlin: Springer; 2011. pp. 585–608.

Ayesh A, Blewitt W. Models for computational emotions from psychological theories using type I fuzzy logic. Cogn Comput. 2015;7(3):285–308.CrossRef

Bahar-Fuchs A, Clare L, Woods B. Cognitive training and cognitive rehabilitation for persons with mild to moderate dementia of the Alzheimer’s or vascular type: a review. Alzheimer’s Res Thera. 2013;5(35):1–14.

Becker R, Cáceres R, Hanson K, Isaacman S, Loh J, Martonosi M, Rowland J, Urbanek S, Varshavsky A, Volinsky C. Human mobility characterization from cellular network data. Commun ACM. 2013;56(1):74–82.CrossRef

Bee N, Wagner J, André E, Charles F, Pizzi D, Cavazza M. Multimodal interaction with a virtual character in interactive storytelling. 2010. In: Proceedings of AAMAS’10, pp 1535–1536.

10.

Benus S. Social aspects of entrainment in spoken interaction. Cogn Comput. 2014;6(4):802–13.CrossRef

11.

Bevacqua E, Mancini M, Pelachaud C. A listening agent exhibiting variable behaviour. Lect Notes Comput Sci. 2008;5208:262–9.CrossRef

12.

Bickmore T, Giorgino T. Health dialog systems for patients and consumers. J Biomed Inf. 2006;39(5):556–71.CrossRef

13.

Bickmore T, Mitchell S, Jack B, Paasche-Orlow M, Pfeifer L, O’Donnell J. Response to a relational agent by hospital patients with depressive symptoms. Interact Comput. 2010a;22:289–98.CrossRefPubMedPubMedCentral

14.

Bickmore T, Puskar K, Schlenk E, Pfeifer L, Sereika S. Maintaining reality: Relational agents for antipsychotic medication adherence. Interact Comput. 2010b;22:276–88.CrossRef

15.

Black LA, McTear MF, Black ND, Harper R, Lemon M. Appraisal of a conversational artefact and its utility in remote patient monitoring. 2005. In: Proceedings of CBMS’05, pp. 506–508.

16.

Blázquez G, Berlanga A, Molina J. Contexto: a fusion architecture to obtain mobile context. 2011. In: Proceedings of FUSION’11, pp. 1–8.

17.

Bonino D, Corno F. What would you ask to your home if it were intelligent? Exploring user expectations about next-generation homes. J Ambient Intell Smart Environ. 2011;3(2):111–6.

18.

Bunt H, Alexandersson J, Carletta J, Choe J, Fang A, Hasida K, Lee K, Petukhova V, Popescu-Belis A, Romary L, Soria C, Traum D. Towards an ISO standard for dialogue act annotation. 2010. In: Proceedings of LREC’10, pp. 2548–2555.

19.

Campillo-Sánchez P, Gómez-Sanz J. Agent based simulation for creating ambient assisted living solutions. 2014. In: Proceedings of PAAMS’14, pp. 319–322.

20.

Cassell J. More than just another pretty face: embodied conversational interface agents. Commun ACM. 2000;43(4):70–8.CrossRef

21.

Castells M, Fernández-Ardévol M, Linchuan-Qiu J, Sey A. Mobile communication and society: a global perspective. Cambridge: MIT Press; 2009.

22.

Cavazza M, de la Cámara RS, Turunen M. How was your day? A companion ECA. 2010. In: Proceedings of AAMAS’10, pp. 1629–1630.

23.

Choi J, Twamley E. Cognitive rehabilitation therapies for Alzheimer’s disease: a review of methods to improve treatment engagement and self-efficacy. Neuropsychol Rev. 2013;23(1):48–62.CrossRefPubMedPubMedCentral

24.

Corchado J, Bajo J, Abraham A. GerAmi: improving healthcare delivery in geriatric residences. Intell Syst. 2008;23(2):19–25.CrossRef

25.

Delichatsios H, Friedman R, Glanz K, Tennstedt S, Smigelski C, Pinto B. Randomized trial of a talking computer to improve adults eating habits. Am J Health Promot. 2000;15:215–24.CrossRef

26.

Dutoit T. An introduction to text-to-speech synthesis. Berlin: Kluwer Academic Publishers; 1996.

27.

Evanini K, Hunter P, Liscombe J, Suendermann D, Dayanidhi K, Pieraccini R. Caller experience: a method for evaluating dialog systems and its automatic prediction. 2008. In: Proceedings of SLT’08, pp. 129–132.

28.

Farzanfar R, Frishkopf S, Migneault J, Friedman R. Telephone-linked care for physical activity: a qualitative evaluation of the use patterns of an information technology program for patients. J Biomed Inf. 2005;38:220–8.CrossRef

29.

Fernández J, Pavón J. Talking agents: a distributed architecture for interactive artistic installations. Integr Comput Aided Eng. 2010;17(3):243–59.

30.

Ghanem K, Hutton H, Zenilman J, Zimba R, Erbelding E. Audio computer assisted self interview and face to face interview modes in assessing response bias among STD clinic patients. Sex Transm Infect. 2005;81(5):421–5.CrossRefPubMedPubMedCentral

31.

Giorgino T, Azzini I, Rognoni C, Quaglini S, Stefanelli M, Gretter R, Falavigna D. Automated spoken dialogue system for hypertensive patient home management. J Med Inf. 2004;74:159–67.CrossRef

32.

Glanz K, Shigaki D, Farzanfar R, Pinto B, Kaplan B, Friedman R. Participant reactions to a computerized telephone system for nutrition and exercise counseling. Patient Educ Couns. 2003;49:157–63.CrossRefPubMed

33.

Gnjatovic M. Therapist-centered design of a Robot’s dialogue behavior. Cogn Comput. 2014;6(4):775–88.CrossRef

34.

González-Vélez H, Mier M, Juliá-Sapé M, Arvanitis T, García-Gómez J, Robles M, Lewis P, Dasmahapatra S, Dupplaw D, Peet A, Arús C, Celda B, Van-Huffel S, Lluch-Ariet M. HealthAgents: distributed multi-agent brain tumor diagnosis and prognosis. J Appl Intell. 2009;30(3):191–202.CrossRef

35.

Griol D, Molina J. A framework to develop adaptive multimodal dialog systems for android-based mobile devices. 2014. In: Proceedings of HAIS’14, pp. 25–36.

36.

Griol D, Hurtado L, Segarra E, Sanchis E. A statistical approach to spoken dialog systems design and evaluation. Speech Commun. 2008;50(8–9):666–82.CrossRef

37.

Griol D, Callejas Z, López-Cózar R, Riccardi G. A domain-independent statistical methodology for dialog management in spoken dialog systems. Comput Speech Lang. 2014;28(3):743–68.CrossRef

38.

Han B, Jia W, Shen J, Yuen M. Context-awareness in mobile web services. 2008. In: Proceedings of ISPA’04, pp. 519–528.

39.

Harrington T, Harrington M. Gerontechnology: why and how?. Maastricht: Shaker Publishing; 2000.

40.

Hassenzahl M, Burmester M, Koller F. Mensch & Computer 2003. Interaktion in Bewegung, Vieweg+Teubner Verlag, chap AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität [A questionnaire for measuring perceived hedonic and pragmatic quality]; 2003. pp. 187–196.

41.

Heinroth T, Minker W. Introducing spoken dialogue systems into intelligent environments. Berlin: Kluwer Academic Publishers, Springer-Verlag; 2012.

42.

Hofmann H, Hermanutz M, Tobisch V, Ehrlich U, Berton A, Minker W. Evaluation of in-car SDS notification concepts for incoming proactive events. 2014. In: Proceedings of IWSDS’14, pp. 102–112.

43.

Hone K, Graham R . Subjective assessment of speech-system interface usability. 1993. In: Proc. Eurospeech’01.

44.

Hubal R, Day R. Informed consent procedures: an experimental test using a virtual character in a dialog systems training application. J Biomed Inf. 2006;39:532–40.CrossRef

45.

IDC. Worldwide quarterly mobile phone tracker. Tech. Rep. 2015; https://www.idc.com

46.

de Ipiña KL, Alonso J, Solé-Casals J, Barroso N, Henriquez P, Faundez-Zanuy M, Travieso C, Ecay-Torres M, Martínez-Lage P, Eguiraun H. On automatic diagnosis of Alzheimer’s disease based on spontaneous speech analysis and emotional temperature. Cogn Comput. 2015;7(1):44–55.CrossRef

47.

Jokinen K. Natural interaction in spoken dialogue systems. 2003. In: Proceedings on Workshop ontologies and multilinguality in user interfaces, pp. 730–734.

48.

Kartakis S. A design-and-play approach to accessible user interface development in ambient intelligence environments. J Comput Ind. 2010;61(4):318–28.CrossRef

49.

Kaufmann T, Pfister B. Syntactic language modeling with formal grammars. Speech Commun. 2012;54(6):715–31.CrossRef

50.

Larson J. VoiceXML introduction to developing speech applications. New Jersey: Prentice Hall; 2002.

51.

Leite I, Pereira A, Castellano G, Mascarenhas S, Martinho C, Paiva A. Modelling empathy in social robotic companions. Adv User Model. 2012;7138:135–47.CrossRef

52.

Lemon O. Learning what to say and how to say it: joint optimisation of spoken dialogue management and natural language generation. Comput Speech Lang. 2011;25(2):210–21.CrossRef

53.

Li S, Wrede B. Why and how to model multi-modal interaction for a mobile robot companion. 2007. In: Proceedings on AAAI Spring Symposium 2007 on Interaction Challenges for Intelligent Assistants, pp. 72–79.

54.

López V, Eisman E, Castro J, Zurita J. A case based reasoning model for multilingual language generation in dialogues. Expert Syst Appl. 2011;39(8):7330–7.CrossRef

55.

López-Cózar R, Araki M. Spoken, multilingual and multimodal dialogue systems. Hoboken: Wiley; 2005.

56.

Maglogiannis I, Zafiropoulos E, Anagnostopoulos I. An intelligent system for automated breast cancer diagnosis and prognosis using SVM based classifiers. J Appl Intell. 2009;30(1):24–36.CrossRef

57.

McTear M, Callejas Z. Voice application development for android. Birmingham: Packt Publishing; 2013.

58.

Metze F, Wechsung I, Schaffer S, Seebode J, Moller S. Reliable evaluation of multimodal dialogue systems. In: Human-computer interaction. Novel interaction methods and techniques. Berlin: Springer; 2009. pp. 75–83.

59.

Miesenberger K, Klaus J, Zagler W, Karshmer A (Eds). Computers helping people with special needs. 2010. In: Proceedings on ICCHP 2010, Lecture Notes on Computer Science 4061, Springer.

60.

Migneault JP, Farzanfar R, Wright J, Friedman R. How to write health dialog for a talking computer. J Biomed Inf. 2006;39(5):276–88.CrossRef

61.

Minker W. Design considerations for knowledge source representations of a stochastically-based natural language understanding component. Speech Commun. 1999;28(2):141–54.CrossRef

62.

Mooney K, Beck S, Dudley W, Farzanfar R, Friedman R. A computer-based telecommunication system to improve symptom care for women with breast cancer. Ann Behav Med Annu Meet Suppl. 2004;27:152–61.

63.

Ohkawa Y, Suzuki M, Ogasawara H, Ito A, Makino S. A speaker adaptation method for non-native speech using learners’ native utterances for computer-assisted language learning systems. Speech Commun. 2009;51(10):875–82.CrossRef

64.

O’Shaughnessy D. Automatic speech recognition: history, methods and challenges. Pattern Recognit. 2008;41(10):2965–79.CrossRef

65.

Oulasvirta A, Rattenbury T, Ma L, Raita E. Habits make smartphone use more pervasive. Pers Ubiquitous Comput. 2012;16(1):105–14.CrossRef

66.

Paek T, Pieraccini R. Automating spoken dialogue management design using machine learning: an industry perspective. Speech Commun. 2008;50(8–9):716–29.CrossRef

67.

Payr S. Closing and closure in human-companion interactions: analyzing video data from a field study. 2010. In: Proceedings on IEEE RO-MAN’10, pp. 476–481.

68.

Pérez-Marín D, Pascual-Nieto I. Conversational agents and natural language interaction: techniques and effective practices. Hershey: IGI Global; 2011.CrossRef

69.

Pfeifer L, Bickmore T. Designing embodied conversational agents to conduct longitudinal health interviews. 2010. In: Proceedings on IVA’10, pp. 4698–4703.

70.

Piau A, Campo E, Rumeau P, Vellas B, Nourhashemi F. Aging society and gerontechnology: a solution for an independent living? J Nutr Health Aging. 2014;18(1):97–112.CrossRefPubMed

71.

Pieraccini R. The voice in the machine: building computers that understand speech. Cambridge: MIT Press; 2012.

72.

Pinto B, Friedman R, Marcus B, Kelley H, Tennstedt S, Gillman M. Effects of a computer-based, telephone-counseling system on physical activity. Am J Prev Med. 2002;23:113–20.CrossRefPubMed

73.

Prezerakos G, Tselikas N, Cortese G. Model-driven composition of context-aware web services using contextUML and aspects. 2007. In: Proceedings of ICWS’07, pp. 320–329.

74.

Ramelson H, Friedman R, Ockene J. An automated telephone-based smoking cessation education and counseling system. Patient Educ Couns. 1999;36:131–43.CrossRefPubMed

75.

Rehrl T, Geiger J, Golcar M, Gentsch S, Knobloch J, Rigoll G, Scheibl K, Schneider W, Ihsen S, Wallhoff F. The robot ALIAS as a database for health monitoring for elderly people. 2013. In: Proceedings of AAL’13, pp. 414–423.

76.

Riccardi G. Subjective assessment of speech-system interface usability. 1993. In: Proceedings in Workshop on Roadmapping the Future of Multimodal Interaction Research including Business Opportunities and Challenges (RFMIR’14), pp. 53–56.

77.

Rojas-Barahona L. Health care dialogue systems: practical and theoretical approaches to dialogue management. 2009. Ph.D. thesis, Universita degli Studi di Pavia.

78.

Rouillard J. Web services and speech-based applications around VoiceXML. J Netw. 2007;2(1):27–35.

79.

Saz O, Yin SC, Lleida E, Rose R, Vaquero C, Rodríguez WR. Tools and technologies for computer-aided speech and language therapy. Speech Commun. 2009;51(10):948–67.CrossRef

80.

Schatzmann J, Weilhammer K, Stuttle M, Young S. A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies. Knowl Eng Rev. 2006;21(2):97–126.CrossRef

81.

Searle J. Speech acts. An essay on the philosophy of language. Cambridge: Cambridge University Press; 1969.CrossRef

82.

Seneff S, Adler M, Glass J, Sherry B, Hazen T, Wang C, Wu T. Exploiting context information in spoken dialogue interaction with mobile devices. 2007. In: Proceedings of IMUx’07, pp. 1–11.

83.

Sixsmith A, Meuller S, Lull F, Klein M, Bierhoff I, Delaney S, Savage R. SOPRANO—an ambient assisted living system for supporting older people at home. 2009. In: Proceedings of ICOST’09, pp. 233–236.

84.

Stolcke A, Coccaro N, Bates R, Taylor P, Ess-Dykema CV, Ries K, Shriberg E, Jurafsky D, Martin R, Meteer M. Dialogue act modeling for automatic tagging and recognition of conversational speech. Comput Linguist. 2000;26(3):339–73.CrossRef

85.

Syrdal D, Dautenhahn K, Koay K, Ho W. Views from within a narrative: evaluating long-term human-robot interaction in a naturalistic environment using open-ended scenarios. Cogn Comput. 2014;6(4):741–59.CrossRef

86.

Torres F, Hurtado L, García F, Sanchis E, Segarra E. Error handling in a stochastic dialog system through confidence measures. Speech Commun. 2005;45(3):211–29.CrossRef

87.

Traum D. Speech acts for dialogue agents. In: Foundations of rational agency. 1999. Berlin: Kluwer. pp 169–201.

88.

Traum D, Larsson S. Current and new directions in discourse and dialogue. In: The information state approach to dialogue management. 2003. Berlin: Kluwer, pp. 325–353.

89.

Tsilfidis A, Mporas I, Mourjopoulos J, Fakotakis N. Automatic speech recognition performance in different room acoustic environments with and without dereverberation preprocessing. Comput Speech Lang. 2013;27(1):380–95.CrossRef

90.

Villarrubia G, de Paz J, Corchado J, Bajo J. EKG intelligent mobile system for home users. 2014. In: Proceedings of IBERAMIA’14, pp. 767–778.

91.

Wahlster W, Reithinger N, Blocher A. Smartkom: towards multimodal dialogues with anthropomorphic interface agents. 2001. In: Proceedings of Status Conference: Lead Projects Human-Computer Interaction, pp. 22–34.

92.

Wahlster W, editor. SmartKom: foundations of multimodal dialogue systems. Berlin: Springer; 2006.

93.

Wang Y, Acero A. Rapid development of spoken language understanding grammars. Speech Commun. 2006;48(3–4):390–416.CrossRef

94.

Williams J, Young S. Partially observable markov decision processes for spoken dialog systems. Comput Speech Lang. 2007;21(2):393–422.CrossRef

95.

Williams J, Plassman B, Burke J, Holsinger T, Benjamin S. Preventing Alzheimer’s disease and cognitive decline. 2010. Evidence Report and Technology Assessment. Agency for Healthcare Research and Quality.

96.

Wolters M, Georgila K, Moore J, Logie R, MacPherson S. Reducing working memory load in spoken dialogue systems. Interact Comput. 2009;21(4):276–87.CrossRef

97.

Wu WL, Lu RZ, Duan JY, Liu H, Gao F, Chen YQ. Spoken language understanding using weakly supervised learning. Comput Speech Lang. 2010;24(2):358–82.CrossRef

98.

Young S. The statistical approach to the design of spoken dialogue systems. 2002. In: Tech Rep, Cambridge University Engineering Department.

99.

Young S. Cognitive user interfaces. IEEE Signal Process Magaz. 2011;27(3):128–40.CrossRef

Title: Mobile Conversational Agents for Context-Aware Care Applications
Authors: David Griol
Zoraida Callejas
Publication date: 01-04-2016
Publisher: Springer US
Published in: Cognitive Computation / Issue 2/2016
Print ISSN: 1866-9956
Electronic ISSN: 1866-9964
DOI: https://doi.org/10.1007/s12559-015-9352-x

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Other articles of this Issue 2/2016

A Novel Hybrid Text Summarization System for Punjabi Text

Autonomous Agents and Ethical Decision-Making

Assistive Technologies for Older Adults in Urban Areas: A Literature Review

A Novel Switching Delayed PSO Algorithm for Estimating Unknown Parameters of Lateral Flow Immunoassay

Combining Expression Data and Knowledge Ontology for Gene Clustering and Network Reconstruction

Integrated e-Healthcare System for Elderly Support

Premium Partner