Ewa Luger
Abigail Sellen
ABSTRACT
The past four years have seen the rise of conversational agents (CAs) in everyday life. Apple, Microsoft, Amazon, Google and Facebook have all embedded proprietary CAs within their software and, increasingly, conversation is becoming a key mode of human-computer interaction. Whilst we have long been familiar with the notion of computers that speak, the investigative concern within HCI has been upon multimodality rather than dialogue alone, and there is no sense of how such interfaces are used in everyday life. This paper reports the findings of interviews with 14 users of CAs in an effort to understand the current interactional factors affecting everyday use. We find user expectations dramatically out of step with the operation of the systems, particularly in terms of known machine intelligence, system capability and goals. Using Norman's 'gulfs of execution and evaluation' [30] we consider the implications of these findings for the design of future systems.
- ALICE http://www.alicebot.org/bios/richardwallace.html Artificial Intelligence FoundationGoogle Scholar
- Jennifer Attride-Stirling. 2001. Thematic networks: an analytic tool for qualitative research. Qualitative Research, 1(3), 385-405Google ScholarCross Ref
- Margaret A Boden. 2007. Conversationalists and Confidants. In Proceedings of Artificial Companions in Society: Perspectives on the Present and Future. Oxford Internet Institute. Retrieved from http://www.academia.edu/2669520/Artificial_Companions_in_Society_Perspectives_on_the_Present_and_FutureGoogle Scholar
- Richard, A Bolt. 1980. "Put-That-There": Voice and Gesture at the Graphics Interface, In the proceedings of SIGGRAGH '80 Proceedings, 14, 3 (July 1980), 262270. doi: 10.1145/800250.807503 Google ScholarDigital Library
- Timothy Bickmore and Rosalind W Picard. 2005. Establishing and maintaining long-term human-computer relationships. ACM Transactions on Computer-Human Interaction. 12, 2: 293--327. doi: 10.1145/1067860.1067867 Google ScholarDigital Library
- Susan Brennan. 1990. Conversation as direct manipulation: An iconoclastic view. In The Art of Human-Computer Interface Design. B.K. Laurel (Ed.), Reading, MA: Addison-Wesley.Google Scholar
- Alan Bryman. 2004. Social Research Methods (2nd Ed). NY: Oxford University PressGoogle Scholar
- Justine Cassell. 2001. Embodied Conversational Agent: Representation and Intelligence in User Interfaces. AI Magazine. 22, 4: 67--83. doi: http://dx.doi.org/10.1609/aimag.v22i4.1593 Google ScholarDigital Library
- Justine Cassell. Tim Bickmore, Lee Campbell, Hannes Vihjalmsson & Hao Yan. 2000. Human Conversation as a System Framework: Designing Embodied Conversational Agents. In Embodied Conversational Agents. Justine Cassell. (ed) MIT Press: 29--63 Google ScholarDigital Library
- Martin Davis. 2000. Engines of Logic: Mathematicians and the Origin of the Computer. W W Norton & Company Google ScholarDigital Library
- Anind Dey and Jennifer Mankoff. 2005. Designing mediation for context-aware applications. ACM Transactions on Computer-Human Interaction 12, 1: 53--80. doi 10.1145/1057237.1057241 Google ScholarDigital Library
- Abbe Don, Susan Brennen., Brenda Laurel and Ben Shneiderman. 1992 Anthropomorphism: from Eliza to Terminator 2. Panel In Proc. CHI '92, Bauersfeld, P., Bennett, J., and Lynch, G. (eds.). ACM, 67--70 Google ScholarDigital Library
- Jilian D'Onfro. 2015. Microsoft Created a Chatbot in China that has Millions of Loyal Followers who talk to it like in the Movie 'Her'. Business Insider UK. http://uk.businessinsider.com/microsoft-chatbotxiaoice-2015-8?r=US&IR=TGoogle Scholar
- Mauro Dragone, Thomas Holz, Brian R. Duffy, Greogory M.P. O'Hare. 2005. Social Situated Agents in Virtual, Real and Mixed Reality Environments. In Intelligent Virtual Agents. Themis Panayiotopoulos, Jonathan Gratch, Ruth Aylett, Daniel Ballin, Patrick Olivier & Thomas Rist (Eds.) Proceedings of 5th International Working Conference, IVA 2005 Kos, Greece, September 12--14, 2005. Springer: 166--177. doi: 10.1007/11550617_15 Google ScholarDigital Library
- James R. Glass, 1999. Challenges for Spoken Dialogue Systems. In the proceedings of IEEE Workshop on Automatic Speech Recognition and Understanding. (ASRU), CO, USAGoogle Scholar
- Erving Goffman. 1967. Interaction Ritual: Essays on Face-to-Face Behaviour. PathenonGoogle Scholar
- Aurthur Graesser, Haiying Li & Carol Forsyth. 2014. Learning by Communicating in Natural Language with Conversational Agents. Current Directions in Psychological Science. 23, 5: 374--380 doi: 10.1177/0963721414540680Google ScholarCross Ref
- Greg Guest, Arwen Bunce & Laura Johnson. 2006. How many interviews are enough? An experiment with data saturation and variability. Field Methods, 18, 1: 59--82. doi: 10.1177/1525822X05279903Google ScholarCross Ref
- Joakim Gustafson, Johan Boye, Morgan Fredriksson, Lasse Johanneson, Jürgen Königsmann. 2005. Providing Computer Game Characters with Conversational Abilities. In Intelligent Virtual Agents. Themis Panayiotopoulos, Jonathan Gratch, Ruth Aylett, Daniel Ballin, Patrick Olivier & Thomas Rist (Eds.) Proceedings of 5th International Working Conference, IVA 2005 Kos, Greece, September 12--14, 2005. Springer: 37--51 doi: 10.1007/11550617_4 Google ScholarDigital Library
- Jürgen Habermas. 1998. On the Pragmatics of Communication. Cambridge: Polity Press.Google Scholar
- Richard H. R. Harper. 2010. Texture: Human Expression in the Age of Communications Overload. MIT Press Google ScholarCross Ref
- Kerstin Heuwinkel. 2012. Framing the Invisible - The Social Background of Trust. In Your Virtual Butler: The Making-of. Robert Trappl (ed). Springer, 16--26. doi: 10.1007/978--3--642--37346--6_3Google Scholar
- Ido A. Iurgel & Manuel Ziegler. 2005. Ask & Answer: An Educational Game Where It Pays to Endear Your Capricious Virtual Companion. In Intelligent Virtual Agents. Themis Panayiotopoulos, Jonathan Gratch, Ruth Aylett, Daniel Ballin, Patrick Olivier & Thomas Rist (Eds.) Proceedings of 5th International Working Conference, IVA 2005 Kos, Greece, September 12--14, 2005. Springer: 15--24 doi: 10.1007/11550617_2 Google ScholarDigital Library
- Stephan Kopp, Lars Gesellensetter, Nicole, C. Krämer & Ipke Wachsmuth. 2005. A Conversational Agent as Museum Guide: Design and Evaluation of a RealWorld Application. Intelligent Virtual Agents. 3661: 329--343 doi: 10.1007/11550617_28 Google ScholarDigital Library
- Joseph Carl Robnett Licklider. Man-Computer Symbiosis. IRE Transactions on Human Factors in Electronics, HFE-1,(March, 1960), 4--11Google Scholar
- Paul P. Maglio and Christopher S. Campbell. 2003. Attentive agents. Communications of the ACM 46, 3: 47--51 doi: 10.1145/636772.636797 Google ScholarDigital Library
- Cade Metz. 2015. Get a Peek at Using Facebook's New Assistant, 'M'. Wired. Retrieved from http://www.wired.com/2015/09/get-peek-someoneusing-facebooks-new-assistant-m/ (accessed 07.09.15)Google Scholar
- Roger Moore. 2012. Spoken Language Processing: Where do we go from Here? In Your Virtual Butler: The Making-of. Robert Trappl (ed). Springer, 119--133 doi: 10.1007/978--3--642--37346--6_10 Google ScholarDigital Library
- Andreea I Niculescu, Kheng Hui Yeo, Luis F D'Haro, Seokhwan Kim, Ridong Jiang & Rafael E Banchs. 2014. Design and evaluation of a conversational agent for the touristic domain. In Proceedings of APSIPA'14: 1-10. doi:10.1109/APSIPA.2014.7041744Google ScholarCross Ref
- Don Norman. 2013. The Design of Everyday Things. Basic Books.Google ScholarDigital Library
- Kenton O'Hara, Richard Harper, Helena Mentis. Abigail Sellen and Alex Taylor. 2013. On the naturalness of touchless: putting the "interaction" back into NUI. ACM Transactions on Computer-Human Interaction. 20, 1 doi: 10.1145/2442106.2442111 Google ScholarDigital Library
- Sabine Payr. 2012. Virtual Butlers and Real People: Styles and Practices in Long Term Use of a Companion. In Your Virtual Butler: The Making-of. Robert Trappl (ed). Springer, 134--178 doi: 10.1007/978--3--642--37346--6_11 Google ScholarDigital Library
- Jenny Preece, Yvonne Rogers & Helen Sharp. 2015. Interaction Design: Beyond Huma- Computer Interaction. Wiley & Sons LtdGoogle Scholar
- Stephen Pulman. Johan Boye, Marc Cavazza, Cameron Smith & Raúl Santos de la Cámara. 2010. How was your Day? In Proceedings of the 2010 Workshop on Companionable Dialogue Systems. Assoc. for Computational Linguistics, Stroudsburg, PA, USA: 37- Google ScholarDigital Library
- Byron Reeves & Clifford Nass. 1996. The Media Equation: How People Treat Computers, Television, and New Media Like Real People and Places. NY: Cambridge University Press. Google ScholarDigital Library
- Deborah Richards. 2012. Agent-based museum and tour guides: applying the state of the art. In Proceedings of The 8th Australasian Conference on Interactive Entertainment: Playing the System (IE '12). ACM (2012) doi: 10.1145/2336727.2336742 Google ScholarDigital Library
- Lina Maria Rojas-Barahona and Christophe Cerisara. 2014. Bayesian Inverse Reinforcement Learning for Modelling Conversational Agents in a Virtual Environment. Computational Linguistics and Intelligent Text Processing. 8403: 503--514 doi: 10.1007/978--3--642--54906--9_41Google ScholarDigital Library
- Nicole Shechtman, and Leonard M Horowitz. 2003 Media Inequality in Conversation: How People Behave Differently when Interacting with Computers and People. In Proceedings of the. SIGCHI Conference on Human Factors in Computing Systems (CHI '03). ACM, (2003), 281--288 doi: 10.1145/642659.642661 Google ScholarDigital Library
- Nathan Shedroff & Christopher Noessel. 2012. Make it so: Interaction Design Lessons from Science Fiction. Brooklyn, NY: RosenfeldGoogle Scholar
- Stergios Tegos, Stavros Demetriadis, Thrasyvoulos Tsiatsos. 2012. Using a Conversational Agent for Promoting Collaborative Language Learning. In Proceedings of Fourth International Conference on Intelligent Networking and Collaborative Systems: 162--165 doi: 10.1109/iNCoS.2012.105 Google ScholarDigital Library
- Janienke Sturm, Else den Os, Lou Boves. 1999. Issues in Spoken Dialogue Systems: Experiences with the Dutch ARISE System. In the proceedings of ESCA Workshop on Interactive Dialogue in Multi-Modal Systems (UDS-99), Kolster Irsee, Germany.Google Scholar
- Robert Trappl. 2012. From Jeeves Jeannie to Siri, and Then? In Your Virtual Butler: The Making-of. Robert Trappl (ed). Springer, 1--8 doi: 10.1007/978--3--64237346--6_1 Google ScholarDigital Library
- Giorgio Vassallo, Giovanni Pilato, Agnese Augello & Salvatore Gaglio. 2010. Phrase Coherence in Conceptual Spaces for Conversational Agents. In Semantic Computing. Sheu, Yu, Ramamoorthy, Joshi & Zadeh.(eds). IEEE: 357--371 doi: 10.1002/9780470588222.ch18Google Scholar
- Astrid M. Von der Pütten, Nicloe C. Krämer, Jonathan Gratch & Sin-Hwa Kang. 2010. "It doesn't matter what you are!" Explaining Social Effects of Agents and Avatars. In Computers in Human Behaviour, 26: 16411650 doi: 10.1016/j.chb.2010.06.012 Google ScholarDigital Library
- Joseph Weizenbaum. 1976. Computer Power and Human Reason: From Judgement to Calculation. W. H. Freeman Google ScholarDigital Library
- Joseph Weizenbaum. 1965. Eliza: A Computer Program for the Study of Natural Language Communication between Man and Machine. In Communications of the ACM. 9,1:36--45 doi: 10.1145/365153.365168 Google ScholarDigital Library
- Yorick Wilks. 2010. Is a companion a distinctive kind of relationship with a machine?. In Proceedings of the 2010 Workshop on Companionable Dialogue Systems (CDS '10). Association for Computational Linguistics, Stroudsburg, PA, USA, 13--18 Google ScholarDigital Library
- Yorick Wilks (ed.). 2010. Close Engagements with Artificial Companions. Key social, psychological, ethical and design issues. Amsterdam: John Benjamins.Google Scholar
Index Terms
- "Like Having a Really Bad PA": The Gulf between User Expectation and Experience of Conversational Agents
Recommendations
Computers are social actors
CHI '94: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"Alexa is my new BFF": Social Roles, User Satisfaction, and Personification of the Amazon Echo
CHI EA '17: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing SystemsAmazon's Echo and its conversational agent Alexa open exciting opportunities for understanding how people perceive and interact with virtual agents. Drawing from user reviews of the Echo posted to Amazon.com, this case study explores the degree to which ...
Comments