Michelle X. Zhou
Skip Abstract Section
Abstract
We present artificial intelligent (AI) agents that act as interviewers to engage with a user in a text-based conversation and automatically infer the user's personality traits. We investigate how the personality of an AI interviewer and the inferred personality of a user influences the user's trust in the AI interviewer from two perspectives: the user's willingness to confide in and listen to an AI interviewer. We have developed two AI interviewers with distinct personalities and deployed them in a series of real-world events. We present findings from four such deployments involving 1,280 users, including 606 actual job applicants. Notably, users are more willing to confide in and listen to an AI interviewer with a serious, assertive personality in a high-stakes job interview. Moreover, users’ personality traits, inferred from their chat text, along with interview context, influence their perception of and their willingness to confide in and listen to an AI interviewer. Finally, we discuss the design implications of our work on building hyper-personalized, intelligent agents.
- S. Adali and J. Golbeck. 2012. Predicting personality with social behavior. In ASONAM’12. 302--309. Google Scholar
Digital Library
- N. Al Moubayed, Y. Vazquez-Alvarez, A. McKay, and A. Vinciarelli. 2014. Face-based automatic personality perception. In Proceedings of ACM Multimedia 2014. 1153--1156. Google ScholarDigital Library
- A. Aly and A. Tapus. 2013. A model for synthesizing a combined verbal and non-verbal behavior based on personality traits in human-robot interaction. In ACM/IEEE Proceedings of HRI’13. 325--332. Google ScholarDigital Library
- S. Argamon, S. Dhawle, M. Koppel, and J. Pennbaker. 2005. Lexical predictors of personality type. In Proceedings of Interface and the Classification Society of North America.Google Scholar
- D. Arellano, J. Varona, F. Perales, N. Bee, K. Janowski, and E. Andre. 2011. Influence of head orientation in perception of personality traits in virtual agents. In Proceedings of the International Conference on Autonomous Agents and Multi-Agent Systems. 1093--1094. Google ScholarDigital Library
- R. Baron. 1993. Interviewers’ moods and evaluations of job applicants: The role of applicant qualifications. J. Appl. Social Psychol. 23, 4 (1993), 253--271.Google ScholarCross Ref
- A. Baylor and S. Ebbers. 2003. The Pedagogical Agent Split-persona Effect: When Two Agents Are Better Than One. EDMEDIA, Honolulu, Hawaii.Google Scholar
- L. Batrinca, N. Mana, B. Lepri, F. Pianesi, and N. Sebe. 2011. Please, tell me about yourself: Automatic personality assessment using short self-presentations. In Proceedings of the International Conference on Multimodal Interfaces. 255--262. Google ScholarDigital Library
- T. Bickmore, A. Gruber, and R. Picard. 2005. Establishing the computer-patient working alliance in automated health behavior change interventions. Patient Educ. Couns. 59, 1 (2005), 21--30.Google ScholarCross Ref
- T. Bickmore and J. Cassell. 1999. Small talk and conversational storytelling in embodied interface agents. In Proceedings of the AAAI Fall Symposium on Narrative Intelligence.Google Scholar
- F. Bouchet and J. Sansonnet. 2012. Intelligent agents with personality: From adjectives to behavioral schemes. In Cognitively Informed Interfaces: System Design and Development, E. M. Alkhalifa and K. Gaid (Eds.). IGI Global, 177--200.Google Scholar
- P. Brown and S. Levinson. 1987. Politeness: Some Universals in Language Usage. Cambridge University Press.Google ScholarCross Ref
- A. Cafaro et al. 2012. First impressions: Users’ judgments of virtual agents’ personality and interpersonal attitude in first encounters. In Proceedings of the International Conference on Intelligent Virtual Agents. Springer, Berlin. Google ScholarDigital Library
- R. Cialdini. 2006. Influence: The Psychology of Persuasion. Harper Business.Google Scholar
- Jacob Cohen. 1988. Statistical Power Analysis for the Behavioral Sciences. Routledge.Google Scholar
- L. Cronbach. 1951. Coefficient alpha and the internal structure of tests. Psychometrika 16, 3 (1951), 297--334.Google ScholarCross Ref
- P. T. Costa Jr. and R. R. McCrae. 1992. Revised NEO Personality Inventory (Neo-PI-R) and NEO Five-Factor Inventory (NEO-FFI): Professional manual. Psychological Assessment Resources, Odessa, FL.Google Scholar
- N. Draper and H. Smith. 1981. Applied Regression Analysis, 2nd ed. John Wiley 8 Sons, Inc., New York.Google Scholar
- R. J. de Ayala. 2009. The Theory and Practice of Item Response Theory. Guilford Press.Google Scholar
- C. DeYoung, L. Quilty, and J. Peterson. 2007. Between facets and domains: 10 aspects of the Big Five. J. Personality Social Psychol. 93, 5 (2007), 880--896.Google ScholarCross Ref
- A. Dempster, N. Laird, and D. Rubin. 1977. Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. 39, 1 (1977), 1--38.Google Scholar
- D. DeVault, R. Benn Artstein, G. Dey, E. R. Fast, A. Gainer, K. Georgila, D. Gratch, A. Hartholt, A. Lhommet, G. Lucas, S. Marsella, F. Morbini, A. Nazarian, S. Scherer, S. Stratou, A. Suri, D. Traum, W. Wood, Y. Xu, A. Rizzo, and L. P. Morency. 2014. SimSensei kiosk: A virtual human interviewer for healthcare decision support. In AAMAS'14. Google ScholarDigital Library
- J. M. Digman. 1990. Personality structure: Emergence of the five-factor model. Annu. Rev. Psychol. 41, 1 (1990), 417--440.Google ScholarCross Ref
- D. Christopher Dryer. 1999. Getting personal with computers: How to design personalities for agents. J. Appl. Artif. Intell. 13, 3 (1999), 273--295.Google ScholarCross Ref
- S. Escalera, V. Athitsos, and I. Guyon. 2016. Challenges in multimodal gesture recognition. J. Mach. Learn. Res. 17 (2016), 1--54. Google ScholarDigital Library
- B. Endrass, E. André, M. Rehm, and Y. Nakano. 2013. Investigating culture-related aspects of behavior for virtual characters. J. Auton Agents Multi-Agent Syst. 27, 2 (2013), 277--304.Google ScholarCross Ref
- J. Fan, D. Gao, A. Carroll, S. Lopez, J. Siva, T. Tian, and H. Meng. 2012. Testing the efficacy of a new procedure for reducing faking on personality tests within selection contexts. J. Appl. Psychol. 97 (2012), 866--880.Google ScholarCross Ref
- B. J. Fogg and C. Nass. 1997. Silicon sycophants: The effects of computers that flatter. Int. J. Hum. Comput. Stud. 46, 5 (1997), 551--561. Google ScholarDigital Library
- M. Freitag and P. Bauer. 2016. Personality traits and the propensity to trust friends and strangers. Social Sci. J. 53, 4 (2016), 467--476.Google Scholar
- P. Gebhard, T. Baur, I. Damian, G. Mehlmann, J. Wagner, and E. André. 2014. Exploring interaction strategies for virtual characters to induce stress in simulated job interviews. In AAMAS'14. 661--668. Google ScholarDigital Library
- A. Gill, S. Nowson, and J. Oberlander. 2009. What are they blogging about? Personality, topic and motivation in blogs. In ICWSM'09. 18--25.Google Scholar
- M. Gludice, T. Booth, and P. Irwing. 2012. The distance between Mars and Venus: Measuring global sex differences in personality. Pub. Lib. Sci. 7, 1 (2012).Google Scholar
- L. Goldberg. 1981. Language and individual differences: The search for universals in personality lexicons. In Review of Personality and Social Psychology, vol. 2. 141--165.Google Scholar
- B. Grosz and C. Sidner. 1986. Attention, intention, and the structure of discourse. Comput. Ling. 12, 3 (1986), 175--204. Google ScholarDigital Library
- L. Gou, M. X. Zhou, and H. Yang. 2014. KnowMe and ShareMe: Understanding Automatically Discovered Personality Traits from Social Media and User Sharing Preferences (CHI’14). 955--964. Google ScholarDigital Library
- L. Havens. 1986. Making Contact: Uses of Language in Psychotherapy. Harvard University Press.Google Scholar
- L. Helgoe. 2013. Introvert Power: Why Your Inner Life is Your Hidden Strength, 2nd ed. Sourcebooks.Google Scholar
- W. Hofstee, B. Raad, and L. Goldberg. 1992. Integration of the Big Five and circumplex approaches to trait structure. J. of Personality Social Psychol. 63, 1 (1992), 146--163.Google ScholarCross Ref
- J. Holland. 1985. Making Vocational Choices: A Theory of Vocational Personalities and Work Environments. Prentice-Hall, Engelwood Cliffs, NJ.Google Scholar
- J. Hopcroft. 1971. An Nlogn Algorithm for Minimizing States in a Finite Automaton, Theory of Machines and Computations. Academic Press, 189--196.Google Scholar
- J. Hopcroft and J. Ullman. 2006. Introduction to Automata Theory, Languages, and Computation, 3rd ed. Pearson. Google ScholarDigital Library
- D. Hoover. 2003. Another perspective on vocabulary richness. Comput. Humanities 37 (2003), 151--178.Google ScholarCross Ref
- R. Hu, P. Pu. 2011. Enhancing collaborative filtering systems with personality information. In RecSys’11. Google ScholarDigital Library
- A. V. Ivanov, G. Riccardi, A. J. Sporka, and J. Franc. 2011. Recognition of personality traits from human spoken conversations. In Proceedings of InterSpeech. 1549--1552.Google Scholar
- A. Jackle, P. Lynn, J. Sinibaldi, and S. Tippping. 2013. The effect of interviewer experience, attitudes, personality, and skills on respondent cooperation with face-to-face surveys. Surv. Res. Methods 7, 1 (2013), 1--15.Google Scholar
- H. Jones, N. Sabouret, I. Damian, T. Baur, E. André, K. Porayska-Pomsta, and P. Rizzo. 2014. Interpreting social cues to generate credible affective reactions of virtual job interviewers. CoRR abs/1402.5039.Google Scholar
- M. Kern, J. Eichstaedt, A. Schwartz, L. Dziurzynski, L. Ungar, D. Stillwell, M. Kosinski, S. Ramones, and M. Seligman. 2013. The online social self: An open vocabulary approach to personality. J. Assess. 21, 2 (2013), 158--169.Google ScholarCross Ref
- T. Koda and P. Maes. 1996. Agents with faces: The effects of personification of agents. In Proc. HCI’ 96. 98--103.Google Scholar
- M. Kosinski, D. Stillwell, and T. Graepel. 2013. Private traits and attributes are predictable from digital records of human behavior. Proc. Natl. Acad. Sci. USA 110, 15 (2013), 5802--5805.Google ScholarCross Ref
- M. Kubat and J. Milicka. 2013. Vocabulary richness measure in genres. J. Quant. Ling. 20, 4 (2013), 339--349.Google ScholarCross Ref
- V. Kulkarni, M. Kern, D. Stillwell, M. Kosinski, S. Matz, L. Ungar, S. Skiena, and A. Schwartz. 2017. Latent human traits in the language of social media: An open-vocabulary approach. In CORR, http://arxiv.org/abs/1705.0803.Google Scholar
- N. Kuncel, D. Klieger, B. Connelly, and D. Ones. 2013. Mechanical versus clinical data combination in selection and admissions decisions: A meta-analysis. J. Appl. Psychol. 98, 6 (2013), 1060--1072.Google ScholarCross Ref
- J. E. Laird. 2012. The Soar Cognitive Architecture. MIT Press. Google ScholarDigital Library
- K. Lee, W. Peng, S. Jin, and C. Yan. 2006. Can robots manifest personality? An empirical test of personality recognition, social responses, and social presence in human-robot interaction. J. Comm. 56, 4 (2006), 754--772.Google ScholarCross Ref
- K. Lee, J. Mahmud, J. Chen, and M. X. Zhou, and J. Nichols. 2014. Who will retweet this? In Automatically Identifying and Engaging Strangers on Twitter to Spread Information (IUI’14). 247--256. Google ScholarDigital Library
- B. Lepri, R. Subramanian, K. Kalimeri, J. Staiano, F. Pianes, and N. Sebe. 2012. Connecting meeting behavior with extraversion: A systematic study. IEEE Trans. Affective Comput. 3, 4 (2012), 443--455. Google ScholarDigital Library
- S. Li and M. Roloff. 2006. Strategic emotion in negotiation: Cognition, emotion, and culture. In Communication to Presence: Cognition, Emotions and Culture towards the Ultimate Communicative Experience. 166--185.Google Scholar
- L. Liu, D. Preotiuc-Pietro, Z. Riahi Samani, M. Moghaddam, and L. Ungar. 2016. Analyzing personality through social media profile picture choice. In ICWSM'16.Google Scholar
- G. Lucas, J. Gratch, A. King, and L. Morency. 2014. It's only a computer: Virtual humans increase willingness to disclose. Comp. Hum. Behav. 37 (2014), 94--100. Google ScholarDigital Library
- E. Luger and A. Sellen. 2016. “Like having a really bad PA”: The gulf between user expectation and experience of conversational agents. In ACM SIGCHI'16. Google ScholarDigital Library
- L. Luo, F. Wang, M. X. Zhou, Y. Pan, and H. Chen. 2014. Who have got answers? Growing the pool of answerers in a smart enterprise social QA system. In IUI’14. 7--16. Google ScholarDigital Library
- J. Mahmud, M. X. Zhou, N. Megiddo, J. Nichols, and C. Drews. 2013. Recommending targeted strangers from whom to solicit information on social media. In IUI’13. 37--48. Google ScholarDigital Library
- F. Mairesse, M. A. Walker, M. R. Mehl, and R. K. Moore. 2007. Using linguistic cues for the automatic recognition of personality in conversation and text. JAIR 30 (2007), 457--500. Google ScholarDigital Library
- R. C. Mayer, J. H. Davis, and F. D. Schoorman. 1995. An integrative model of organizational trust. Acad. Manage. Rev. 20, 3 (1995), 709--734.Google ScholarCross Ref
- M. McRorie, I. Sneddon, G. McKeown, E. Bevacqua, E. de Sevin, and C. Pelachaud. 2011. Evaluation of four designed virtual agent personalities. IEEE Trans. Affective Comput. 3, 3 (2011), 311--322. Google ScholarDigital Library
- G. Mehlmann, K. Janowski, and E. André. 2016. Modeling grounding for interactive social companions. KI 30, 1 (2016), 45--52.Google Scholar
- M. Mohri. 1997. Finite-state transducers in language and speech processing. Comput. Ling. 23, 2 (1997), 269--311. Google ScholarDigital Library
- C. Nass and C. Yen. 2012. The Man Who Lied to His Laptop. Current.Google Scholar
- C. Nass, J. Steuer, and E. Tauber. 1994. Computers are social actors. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 72--78. Google ScholarDigital Library
- K. Nowak, J. Fox, and Y. Ranjit. 2015. Inferences about avatars: Sexism, appropriateness, anthropomorphism, and the objectification of female virtual representations. J. Comput. Mediated Commun. 20, 5 (2015), 554--569. Google ScholarDigital Library
- J. Nunamaker et al. 2011. Embodied conversational agent-based kiosk for automated interviewing. J. Manage. Inf. Syst. 28, 1 (2011), 17--48. Google ScholarDigital Library
- B. Okun. 2007. Effective Helping: Interviewing and Counseling Techniques, 7th ed. Cengage Learning.Google Scholar
- D. L. Paulhus. 1991. Measurement and control of response bias. In Measures of Personality and Social Psychological Attitudes, J. P. Robinson and P. R. Shaver (Eds.). Vol. 1. Academic Press, 17--59.Google Scholar
- J. Pennebaker and L. King. 1999. Linguistic styles: Language use as an individual difference. J. Personality Social Psychol. 77, 6 (1999), 1296--1312.Google ScholarCross Ref
- S. Piana, A. Staglianò, F. Odone, and A. Camurri. 2016. Adaptive body gesture representation for automatic emotion recognition. ACM TIIS 6, 1 (2016), 1--31. Google ScholarDigital Library
- D. Preotiuc-Pietro, J. Carpenter, S. Giorgi, L. Ungar. 2016. Studying the dark triad of personality through Twitter behavior. In CIKM’16. 761--770. Google ScholarDigital Library
- A. Powers, S. Kiesler, and J. Goetz. 2003. Matching robot appearance and behavior to tasks to improve human-robot cooperation. In Proceedings of the 12th IEEE International Workshop on Robot and Human Interactive Communication, vol. IXX, 55--60.Google Scholar
- B. Reeves and C. Nass. 2002. The Media Equation. CSLI Publications.Google Scholar
- T. Rath. 2007. Strengths Finder 2.0. Gallup Press.Google Scholar
- P. Robinette, W. Li, R. Allen, A. M. Howard, and A. Wagner. 2016. Overtrust of robots in emergency evacuation scenarios. In HRI’16. 101--108. Google ScholarDigital Library
- J. Sansonnet and F. Bouchet. 2013. Managing personality influences in dialogical agents. In Proceedings of ICAART’13, Vol. 1. 89--98.Google Scholar
- M. Schmitz, A. Kruger, and S. Schmidt. 2007. Modeling personality in voice of talking products through prosodic parameters. In Proceedings of the International Conference on the Internet of Things. 313--316. Google ScholarDigital Library
- J. Searle. 1969. Speech Acts: An Essay in the Philosophy of Language. Cambridge University Press.Google Scholar
- I. Serban, A. Sordoni, Y. Bengio, A. Courville, and J. Pineau. 2016. Building end-to-end dialogue systems using generative hierarchical neural network models. In AAAI’16. 3776--3783. Google ScholarDigital Library
- L. Shao, S. Gouws, D. Britz, A. Goldie, B. Strope, and R. Kurzweil. 2017. Generating long and diverse responses with neural conversational models. In ICLR’17.Google Scholar
- L. Sproull, M. Subramani, S. Kiesler, J. Walker, and K. Waters. 1996. When the interface is a face. J. Hum. Comput. Interact. 11, 2 (1996), 97--124. Google ScholarDigital Library
- A. Tapus, C. Tapus, and M. Mataric. 2008. User-robot personality matching and assistive robot behavior adaptation for post-stroke rehabilitation therapy. Intell. Serv. Rob. 1, 2 (2008), 169--196.Google ScholarCross Ref
- Y. Tausczik and J. Pennebaker. 2010. The psychological meaning of words: LIWC and computerized text analysis methods. J. Lang. Social Psychol. 29, 1 (2010), 24--54.Google ScholarCross Ref
- P. Thiel and B. Masters. 2014. Zero to One: Notes on Startups, or How to Build the Future. Currency.Google Scholar
- N. Tintarev, M. Dennis, and J. Masthoff. 2013. Adapting recommendation diversity to openness to experience: A study of human behavior. In UMAP'13. 190--202.Google Scholar
- D. Traum. 2017. Computational approaches to dialogue. In The Routledge Handbook of Language and Dialogue, E. Weigand (Ed.), New York.Google Scholar
- J. Trouvain, S. Schmidt, M. Schroder, M. Schmitz, and W. J. Barry. 2006. Modeling personality features by changing prosody in synthetic speech. In Proceedings of Speech Prosody.Google Scholar
- M. Turk. 2014. Multimodal interaction: A review. Pattern Recognit. Lett. 36 (2014), 189--195. Google ScholarDigital Library
- L. Uziel. 2013. Impression management (“lie”) scales are associated with interpersonally oriented self-control, not other deception. J. Personality 82, 3 (2013), 200--212.Google ScholarCross Ref
- A. Vinciarelli and G. Mohammadi. 2014. A survey of personality computing. IEEE Trans. Affective Comput. 5, 3 (2014), 273--291.Google ScholarCross Ref
- O. Vinyals and Q. Le. 2015. A neural conversational model. In Proceedings of ICML, Deep Learning Workshop.Google Scholar
- R. Wallace, H. Tomabechi, and D. Aimless. 2007. Chatterbots Go Native: Considerations for an eco-system fostering the development of artificial life forms in a human world. Retrieved from http://www.pandorabots.com/pandora/pics/ chatterbotsgonative.doc.Google Scholar
- J. Warshaw, T. Matthews, S. Whittaker, C. Kau, M. Bengualid, and B. Smith. 2015. Can an algorithm know the real you? Understanding people's reactions to hyper-personal analytics systems. In CHI’15. 797--806. Google ScholarDigital Library
- Y. Weisberg, C. DeYoung, and J. Hirsh. 2011. Gender differences in personality across the ten aspects of Big Five. J. Front Psychol. 2, Article 178 (2011), 11 pages.Google Scholar
- J. Weizenbaum. 1966. ELIZA—A computer program for the study of natural language communication between man and machine. Commun. ACM 9 (1966), 36--45. Google ScholarDigital Library
- A. Wexelblat. 1998. Don't make that face: A report on anthropomorphizing and interface. In AAAI’98. 173--179.Google Scholar
- W. Xiong, L. Wu, F. Alleva, J. Droppo, X. Huang, and A. Stolcke. 2017. The Microsoft 2017 Conversational Speech Recognition System, Tech Report, Retrieved from https://www.microsoft.com/en-us/research/publication/microsoft-2017-conversational-speech-recognition-system/.Google Scholar
- A. Xu, Z. Liu, Y. Guo, V. Sinha, R. Akkiraju. 2017. A new chatbot for customer service on social media. In Proc. ACM CHI’17. 3506--3510. Google ScholarDigital Library
- T. Yarkoni. 2010. Personality in 100,000 words: A large-scale analysis of personality and word usage among bloggers. J. Res. Personality 44, 3 (2010), 363--373.Google ScholarCross Ref
- N. Yee, J. Bailenson, and K. Rickertsen. 2007. A meta-analysis of the impact of the inclusion and realism of human-like faces on user experiences in interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1--10. Google ScholarDigital Library
- R. Yu, Y. Liu, and M. Yang. 2011. Does interviewer personality matter for survey outcomes? In Proceedings of the 64th Annual Conference on World Association for Public Opinion Research. 89--118.Google Scholar
- R. B. Zajonc. 1980. Feeling and thinking: Preferences need no inferences. Am. Psychologist 35, 2 (1980), 151--175.Google ScholarCross Ref
- M. Ziegler, C. MacCann, and R. Roberts, (Ed). 2010. New Perspectives on Faking in Personality Assessment. Oxford University Press.Google Scholar
Index Terms
- Trusting Virtual Agents: The Effect of Personality
Recommendations
Confiding in and Listening to Virtual Agents: The Effect of Personality
IUI '17: Proceedings of the 22nd International Conference on Intelligent User InterfacesWe present an intelligent virtual interviewer that engages with a user in a text-based conversation and automatically infers the user's psychological traits, such as personality. We investigate how the personality of a virtual interviewer influences a ...
Getting virtually personal: chatbots who actively listen to you and infer your personality
IUI '19 Companion: Companion Proceedings of the 24th International Conference on Intelligent User InterfacesWe build conversational agents to serve as AI interviewers who engage a user in a one-on-one, text-based conversation. Our live demos showcase two special skills of an AI interviewer: (a) ability to actively listen to a user during an interview-...
How do Conversational Agents Transform Qualitative Interviews? Exploration and Support of Researchers’ Needs in Interviews at Scale
IUI '22 Companion: Companion Proceedings of the 27th International Conference on Intelligent User InterfacesIn recent years, conversational agents (CAs) have been receiving more attention as tools for collecting data through qualitative interviews. The problem is we know little about how CAs affect both the interviewees and interviewers. This PhD project is ...
Comments