Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Bücher
Zeitschriften
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
SLX-Digitalkonferenz: Zukunftswerkstatt 2024

Mehr Umsatz mit Top-Kontakten

Am 7. Mai erfahren Sie, wie Vertriebsteams Leadgenerierung, Leadnurturing und Leadstrategien effizient steuern können.
Erweiterte Suche
Anmelden
nach oben
Journal on Multimodal User Interfaces
Erschienen in: Journal on Multimodal User Interfaces 1-2/2012

01.07.2012 | Original Paper

Generating context-sensitive ECA responses to user barge-in interruptions

verfasst von: Nigel Crook, Debora Field, Cameron Smith, Sue Harding, Stephen Pulman, Marc Cavazza, Daniel Charlton, Roger Moore, Johan Boye

Erschienen in: Journal on Multimodal User Interfaces | Ausgabe 1-2/2012

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

We present an Embodied Conversational Agent (ECA) that incorporates a context-sensitive mechanism for handling user barge-in. The affective ECA engages the user in social conversation, and is fully implemented. We will use actual examples of system behaviour to illustrate. The ECA is designed to recognise and be empathetic to the emotional state of the user. It is able to detect, react quickly to, and then follow up with considered responses to different kinds of user interruptions. The design of the rules which enable the ECA to respond intelligently to different types of interruptions was informed by manually analysed real data from human–human dialogue. The rules represent recoveries from interruptions as two-part structures: an address followed by a resumption. The system is robust enough to manage long, multi-utterance turns by both user and system, which creates good opportunities for the user to interrupt while the ECA is speaking.
Vorheriger Artikel Visual SceneMaker—a tool for authoring interactive virtual characters
Nächster Artikel A listener model: introducing personality traits

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Anhänge
Nur mit Berechtigung zugänglich
Fußnoten
1
The graphical user interface for the HWYD? prototype including the animated agent were created by Telefonica ID, Madrid.
 
2
Telefonica I+D in Madrid have designed the skin, lip-synching, and gestures for the HWYD? avatar (that we call Matilda) [30, 31], which is driven by a Haptek engine. Loquendo in Turin have innovated emotional speech for the HWYD? TTS module [32].
 
3
All inter-component messages sent and received during run-time are recorded in a log file.
 
4
The user’s turns are reproduced exactly as was output by the ASR, and therefore include some ungrammatical structures.
 
5
Recall that we equate an interruption with overlapping/simultaneous speech (Sect. 2).
 
7
The domain of the corpus we used was different from that of the HWYD? system, whose domain was social dialogue focusing on one’s day at the office. There was no opportunity at this late stage for us to collect a corpus in that domain, and no corpus available in this domain that was also full of interruptions.
 
8
20070112, 20070119, 20070126.
 
9
Owing to the preliminary nature of the work, no inter-annotator agreement is yet in force.
 
10
Ignoring turn content (ign-t-content) was the second most frequent way of recovering from an interruption that we observed in the corpus, the most frequent being to supply information that was requested by the interruption (at 30.65%), which we also modelled (see Sect. 5.2.3).
 
11
The system did not recognise Turn (b) as a WH question because it did not begin with a WH word. This is one of many undesirable shortcomings that are currently being addressed.
 
12
Additionally, we take note that many interruptions are considered by much of the interruptions literature to be hostile, in that the interruptor snatches the conversational floor before it is his turn. By implication it is reasonable to posit that if a user interrupts the system, the system may have said something that has not been well received by the user, and this adds weight to the appropriateness of an apology in an interruption recovery.
 
Literatur
1.
Cavazza M, Santos de la Cámara R, Turunen M (The COMPANIONS Consortium) (2010) How was your day? A companion ECA. In: Proceedings of the 9th international conference on autonomous agents and multiagent systems (AAMAS2010), Toronto, Canada, May 10–14, 2010, pp 1629–1630
2.
Young S (2010) Still talking to machines (cognitively speaking), 2010. In: Proc Interspeech, Chiba, Japan, 26–30 September, 2010
3.
Lemon O, Georgila K, Henderson J, Stuttle M (2006) An ISU dialogue system exhibiting reinforcement learning of dialogue policies: generic slot-filling in the TALK in-car system. In: Proceedings of the eleventh conference of the European chapter of the association for computational linguistics: posters & demonstrations, EACL ’06, Morristown, NJ, USA. Association for Computational Linguistics, Stroudsburg, pp 119–122 CrossRef
4.
Allen J, Chambers N, Ferguson G, Galescu L, Jung H, Swift M, Taysom W (2007) Plow: a collaborative task learning agent. In: Proceedings of the 22nd national conference on artificial intelligence, vol 2. AAAI Press, Menlo Park, pp 1514–1519
5.
West C, Zimmerman D (1983) Small insults: A study of interruptions in cross-sex conversations between unacquainted persons. In: Thorne B, Kramarae C, Henley N (eds) Language, gender and society. Newbury House, Cambridge, pp 102–117
6.
Lakoff RT (1995) Cries and whispers: the shattering of the silence. In: Hall K, Bucholtz M (eds) Gender articulated: language and the socially constructed self. Routledge, New York, pp 25–50
7.
Sacks H, Schegloff EA, Jefferson G (1974) A simplest systematics for the organization of turn-taking for conversation. Language 50(4):696–735 CrossRef
8.
Coates J (1993) Women, men, and language: a sociolinguistic account of gender differences in language, 2nd edn. Longman, London/New York
9.
Bevacqua E, Pammi S, Hyniewska SJ, Schröder M, Pelachaud C (2010) Multimodal backchannels for embodied conversational agents. In: Allbeck JM, Badler NI, Bickmore TW, Pelachaud C, Safonova A (eds) IVA. Lecture notes in computer science, vol 6356. Springer, Berlin, pp 194–200
10.
Morency L-P, de Kok I, Gratch J (2008) Predicting listener backchannels: a probabilistic multimodal approach. In: Prendinger H, Lester JC, Ishizuka M (eds) IVA. Lecture notes in computer science, vol 5208. Springer, Berlin, pp 176–190
11.
Zimmerman D, West C (1975) Sex roles, interruptions and silences in conversation. In: Thorne B, Henly N (eds) Language and sex: difference and dominance. Newbury House, Cambridge, pp 10–129
12.
Murray SO (1985) Toward a model of members’ methods for recognizing interruptions. Lang Soc 14(1):31–40 CrossRef
13.
Raux A, Eskenazi M (2007) A multi-layer architecture for semi-synchronous event-driven dialogue management. In: ASRU, Kyoto, Japan, pp 514–519
14.
Barnett J, Singh M (1996) Designing a portable spoken dialogue system. In: Maier E, Mast M, LuperFoy S (eds) ECAI workshop on dialogue processing in spoken language systems. Lecture notes in computer science, vol 1236. Springer, Berlin, pp 156–170 CrossRef
15.
Rose RC, Kim HK (2003) A hybrid barge-in procedure for more reliable turn-taking in human-machine dialog systems. In: Proceedings of the automatic speech recognition and understanding workshop
16.
Balentine B, Morgan DP (1999) How to build speech recognition applications—a style guide for telephony dialogs. Enterprise Integration Group, San Ramon
17.
Setlur AR, Sukkar RA (1998) Recognition-based word counting for reliable barge-in and early endpoint detection in continuous speech recognition. In: Proceeding of the international conference on spoken language processing, pp 2135–2138
18.
Matsuyama K, Komatani K, Ogata T, Okuno HG (2009) Enabling a user to specify an item at any time during system enumeration—item identification for barge-in-able conversational dialogue systems. In: Proceedings of the 10th annual conference of the international speech communication association (INTERSPEECH 2009), Brighton UK, 6–10 September 2009, pp 252–255
19.
Komatani K, Rudnicky AI (2009) Predicting barge-in utterance errors by using implicitly-supervised ASR accuracy and barge-in rate per user. In: Proceedings of the ACL-IJCNLP conference short papers, Suntec, Singapore, August 2009. Association for Computational Linguistics, Stroudsburg, pp 89–92 CrossRef
20.
Brooks RA (1985) A robust layered control system for a mobile robot. Technical report, Massachusetts Institute of Technology, Cambridge, MA, USA
21.
Brooks RA (1995) Intelligence without representation. In: Computation & intelligence: collected readings. American Association for Artificial Intelligence, Menlo Park, pp 343–362
22.
Moore RK (2007) Presence: A human-inspired architecture for speech-based human-machine interaction. IEEE Trans Comput 56(9):1176–1188 MathSciNetCrossRef
23.
Reidsma D, de Kok T, Neiberg D, Pammi S, van Straalen B, Truong K, van Welbergen H (2011) Continuous interaction with a virtual human. J Multimodal User Interfaces 4:97–118 CrossRef
24.
Santos de la Cámara R, Turunen M, Hakulinen J, Field D (2010) How was your day? an architecture for multimodal ECA systems, 2010. In: Proc 11th annual meeting of the special interest group on discourse and dialogue (SIGDIAL), 24–25 September, 2010. University of Tokyo, Tokyo, pp 47–50
25.
Vogt T, André E, Bee N (2008) Emovoice—a framework for online recognition of emotions from voice. In: Proceedings of the 4th IEEE tutorial and research workshop on perception and interactive technologies for speech-based systems: perception in multimodal dialogue systems, PIT ’08. Springer, Berlin, pp 188–199
26.
Moilanen K, Pulman S (2007) Sentiment composition. In: Proceedings of the recent advances in natural language processing international conference (RANLP-2007), Borovets, Bulgaria, 27–29 September 2007, pp 378–382
27.
Bremond C (1973) Logique du Récit. Editions du Seuil, Paris
28.
Cavazza M, Smith C, Charlton D, Crook N, Boye J, Pulman S, Moilanen K, Pizzi D, Santos de la Cámara R, Turunen M (2010) Persuasive dialogue based on a narrative theory: an ECA implementation. In: Proceedings of the fifth international conference on persuasive technology (Persuasive 2010), Copenhagen, Denmark, 7–10 June 2010
29.
Smith C, Crook N, Boye J, Charlton D, Dobnik S, Pizzi D, Cavazza M, Pulman S, Santos de la Cámara R, Turunen M (2010) Interaction strategies for an affective conversational agent. In: Proc of the 10th int. conf. on intelligent virtual agents (IVA 2010), Philadelphia, PA, September 2010
30.
Hernández A, López B, Pardo D, Santos R, Hernández L, Relaño Gil J, Rodríguez M (2008) Modular definition of multimodal ECA communication acts to improve dialogue robustness and depth of intention. In: Proc 1st functional markup language workshop, 7th international joint conference on autonomous agents and multiagent systems (AAMAS 2008), Estoril, Portugal, 12–16 May 2008
31.
López B, Hernández A, Pardo D, Santos R, Rodríguez M (2008) ECA gesture strategies for robust SLDS. In: Proc artificial intelligence and simulation behaviour convention (AISB 2008) symposium on multimodal output generation, Aberdeen, UK, 1–4 April, 2008
32.
Danieli M, Zovato E (2010) The affective dimension of speech acts and voice expressiveness. In: Pettorino M, Giannini A, Chiari I, Dovetto Fr (eds) Spoken communication. Cambridge Scholars Publishing, Newcastle upon Tyne, pp 191–204
33.
Stoness S, Tetreault J, Allen J (2004) Incremental parsing with reference interaction. In: ACL workshop on incremental parsing, pp 18–25
34.
Aist G, Allen J, Campana E, Gallo C, Stoness S, Swift M, Tanenhaus M (2007) Incremental understanding in human-computer dialogue and experimental evidence for advantages over nonincremental methods. In: Proceedings of the 11th workshop on the semantics and pragmatics of dialogue, Trento, Italy, 30 May–1 June 2007, pp 149–154
35.
Brick T, Scheutz M (2007) Incremental natural language processing for HRI. In: Proceedings of the ACM/IEEE international conference on Human-robot interaction, Arlington, Virginia, USA, pp 263–270 CrossRef
36.
Skantze G, Schlangen D (2009) Incremental dialogue processing in a micro-domain. In: Proceedings of the 12th conference of the European chapter of the ACL (EACL 2009), Athens, Greece, April 2009, pp 745–753,
37.
Schlangen D, Skantze G (2009) A general, abstract model of incremental dialogue processing. In: Proc of the 12th conference of the European chapter of the ACL (EACL 2009), Athens, Greece, April 2009, pp 710–718
38.
Starkey D (1972) Some signals and rules for taking speaking turns in conversations. J Pers Soc Psychol 23:283–292 CrossRef
39.
Wiemann JM, Knapp ML (1975) Turn-taking in conversations. J Commun 25:75–92
40.
Schegloff EA (2000) Overlapping talk and the organization of turn-taking for conversation. Lang Soc 29(1):1–63 CrossRef
41.
Kennedy CW, Camden CT (1983) A new look at interruptions. West J Commun 47:45–58
42.
Roger D (1989) 4: Experimental studies of dyadic turn-taking behaviour. In: Roger D, Bull P (eds) Conversation: an interdisciplinary perspective. Multilingual Matters, Clevedon
43.
Hutchby I (1992) Confrontation talk: Aspects of interruption in argument sequences on talk radio. Interdiscip J Study Discourse 12:343–372 CrossRef
44.
Walker M, Whittaker S (1990) Mixed initiative in dialogue: An investigation into discourse segmentation. In: Proc. 28th annual meeting of the ACL, pp 70–79
45.
Heins R, Franzke M, Durian M, Bayya A (1997) Turn-taking as a design principle for barge-in in spoken language systems. Int J Speech Technol 2:155–164. doi:10.​1007/​BF02208827 CrossRef
46.
Oth RKEN, Kieling A, Kuhn T, Mast M, Niemann H, Ott K, Batliner A (1994) Prosody takes over: towards a prosodically guided dialog system. Speech Commun 15(15):155–167
47.
Austin JL (1962) How to do things with words, 2nd edn. Oxford University Press, New York
48.
Bunt HC (2000) Dynamic interpretation and dialogue theory. In: Taylor MM, Neel F, Bouwhuis DG (eds) The structure of multimodal dialogue, vol 2. North-Holland, Amsterdam, pp 139–166
49.
Traum DR (2000) 20 questions on dialogue act taxonomies. J Semant 17:7–30 CrossRef
50.
Jurafsky D, Shriberg E, Biasca D (1997) Switchboard SWBD-DAMSL shallow-discourse-function annotation coders manual, draft 13. University of Colorado, Boulder. Institute of Cognitive Science Technical Report 97-02
51.
Thomason RH (1990) Accommodation, meaning, and implicature: interdisciplinary foundations for pragmatics. In: Intentions and communication, pp 325–363
52.
Lewis D (1979) Scorekeeping in a language game. J Philos Log 8:339–359. Reprinted in Lewis, D (1983) Philosophical papers, vol. I. Oxford University Press, New York/Oxford, pp 233–249 CrossRef
53.
Stalnaker R (1972) Pragmatics. In: Davidson D, Harman G (eds) Semantics of natural language. Synthese library, vol 40. Reidel, Dordrecht, pp 380–397 CrossRef
Metadaten
Titel
Generating context-sensitive ECA responses to user barge-in interruptions
verfasst von
Nigel Crook
Debora Field
Cameron Smith
Sue Harding
Stephen Pulman
Marc Cavazza
Daniel Charlton
Roger Moore
Johan Boye
Publikationsdatum
01.07.2012
Verlag
Springer-Verlag
Erschienen in
Journal on Multimodal User Interfaces / Ausgabe 1-2/2012
Print ISSN: 1783-7677
Elektronische ISSN: 1783-8738
DOI
https://doi.org/10.1007/s12193-012-0090-z

Weitere Artikel der Ausgabe 1-2/2012

Journal on Multimodal User Interfaces 1-2/2012 Zur Ausgabe

Original Paper

Sound/tracks: artistic real-time sonification of train journeys

Original Paper

Studying and enhancing talking condition recognition in stressful and emotional talking environments based on HMMs, CHMM2s and SPHMMs

EditorialNotes

Preface

Original Paper

Natural interaction with culturally adaptive virtual characters

Original Paper

A listener model: introducing personality traits

Original Paper

Measuring the Quality of Service and Quality of Experience of multimodal human–machine interaction