Top

Published in:

2016 | OriginalPaper | Chapter

6. Computing Technologies for Social Signals

Author : Alessandro Vinciarelli

Published in: Emotions and Personality in Personalized Services

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Social signal processing is the domain aimed at modelling, analysis and synthesis of nonverbal communication in human–human and human–machine interactions. The core idea of the field is that common nonverbal behavioural cues—facial expressions, vocalizations, gestures, postures, etc—are the physical, machine-detectable evidence of social phenomena such as empathy, conflict, interest, attitudes, dominance, etc. Therefore, machines that can automatically detect, interpret and synthesize social signals will be capable to make sense of the social landscape they are part of while, possibly, participating in it as full social actors.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Acquisition of Personality

next chapter Sentiment Analysis in Social Streams

http://www.cogitocorp.com.

http://www.engagem8.com.

http://www.behaviouralinsights.co.uk.

http://groupspaces.com/ROCKIT.

http://www.eu-robotics.net.

http://www.sony-aibo.co.uk.

Al Moubayed, N., Vazquez-Alvarez, Y., McKay, A., Vinciarelli, A.: Face-based automatic personality perception. In: Proceedings of the ACM International Conference on Multimedia, pp. 1153–1156 (2014)

André, E.: Exploiting unconscious user signals in multimodal human-computer interaction. ACM Trans. Multimedia Comput. Commun. Appl. 9(1s), 48 (2013)CrossRef

André, E.: Challenges for social embodiment. In: Proceedings of the Workshop on Roadmapping the Future of Multimodal Interaction Research Including Business Opportunities and Challenges, pp. 35–37 (2014)

Argamon, S., Dhawle, S., Koppel, M., Pennbaker, J.: Lexical predictors of personality type. In: Proceedings of Interface and the Classification Society of North America (2005)

Avril, M., Leclere, C., Viaux-Savelon, S., Michelet, S., Achard, C., Missonnier, S., Keren, M., Cohen, D., Chetouani, M.: Social signal processing for studying parent-infant interaction. Frontiers Psychol. 5, 1437 (2014)CrossRef

Bakeman, R., Gottman, J.: Observing Interaction: An Introduction to Sequential Analysis. Cambridge University Press (1997)

Benford, S., Greenhalgh, C., Giannachi, G., Walker, B., Marshall, J., Rodden, T.: Uncomfortable interactions. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2005–2014. ACM (2012)

Bonin, F., Gilmartin, E., Vogel, C., Campbell, N.: Topics for the future: genre differentiation, annotation, and linguistic content integration in interaction analysis. In: Proceedings of the Workshop on Roadmapping the Future of Multimodal Interaction Research Including Business Opportunities and Challenges, pp. 5–8 (2014)

Bousmalis, K., Méhu, M., Pantic, M.: Spotting agreement and disagreement: a survey of nonverbal audiovisual cues and tools. In: Proceedings of Social Signal Processing Workshop, pp. 1–9 (2009)

10.

Brunet, P., Cowie, R.: Towards a conceptual framework of research on social signal processing. J. Multimodal User Interfaces 6(3–4), 101–115 (2012)CrossRef

11.

Brunet, P.M., Cowie, R., Heylen, D., Nijholt, A., Schroeder, M.: Conceptual frameworks for multimodal social signal processing. J. Multimodal User Interfaces 6(3–4), 1–5 (2012)

12.

Brunswik, E.: Perception and the Representative Design of Psychological Experiments. University of California Press (1956)

13.

Cristani, M., Ferrario, R.: Statistical pattern recognition meets formal ontologies: towards a semantic visual understanding. In: Proceedings of the Workshop on Roadmapping the Future of Multimodal Interaction Research Including Business Opportunities and Challenges, pp. 23–25 (2014)

14.

Cristani, M., Paggetti, G., Vinciarelli, A., Bazzani, L., Menegaz, G., Murino, V.: Towards computational proxemics: inferring social relations from interpersonal distances. In: Proceedings of IEEE International Conference on Social Computing, pp. 290–297 (2011)

15.

Cristani, M., Raghavendra, R., Del Bue, A., Murino, V.: Human behavior analysis in video surveillance: a social signal processing perspective. Neurocomputing 100, 86–97 (2013)CrossRef

16.

Cristani, M., Vinciarelli, A., Segalin, C., Perina, A.: Unveiling the multimedia unconscious: implicit cognitive processes and multimedia content analysis. In: Proceedings of the ACM International Conference on Multimedia, pp. 213–222. ACM (2013)

17.

Curhan, J.R., Pentland, A.: Thin slices of negotiation: predicting outcomes from conversational dynamics within the first 5 minutes. J. Appl. Psychol. 92(3), 802 (2007)CrossRef

18.

Dautenhahn, K., Werry, I.: Towards interactive robots in autism therapy: background, motivation and challenges. Pragmat. Cogn. 12(1), 1–35 (2004)CrossRef

19.

Delaherche, E., Chetouani, M., Mahdhaoui, A., Saint-Georges, C., Viaux, S., Cohen, D.: Interpersonal synchrony: a survey of evaluation methods across disciplines. IEEE Trans. Affect. Comput. 3(3), 349–365 (2012)CrossRef

20.

Fong, T., Nourbakhsh, I., Dautenhahn, K.: A survey of socially interactive robots. Rob. Auton. Syst. 42(3), 143–166 (2003)CrossRefMATH

21.

Garg, N.P., Favre, S., Salamin, H., Hakkani Tür, D., Vinciarelli, A.: Role recognition for meeting participants: an approach based on lexical information and social network analysis. In: Proceedings of the ACM International Conference on Multimedia, pp. 693–696 (2008)

22.

Gosling, S.D., Ko, S.J., Mannarelli, T., Morris, M.E.: A room with a cue: personality judgments based on offices and bedrooms. J. Pers. Soc. Psychol. 82(3), 379 (2002)CrossRef

23.

Hammal, Z., Cohn, J.F.: Intra- and interpersonal functions of head motion in emotion communication. In: Proceedings of the Workshop on Roadmapping the Future of Multimodal Interaction Research Including Business Opportunities and Challenges, pp. 19–22 (2014)

24.

Hung, H., Kroese, B.: Detecting F-formations as dominant sets. In: Proceedings of the International Conference on Multimodal Interfaces, pp. 231–238 (2011)

25.

Jayagopi, D.B., Hung, H., Yeo, C., Gatica-Perez, D.: Modeling dominance in group conversations using nonverbal activity cues. IEEE Trans. Audio Speech Lang. Process. 17(3), 501–513 (2009)CrossRef

26.

Koutsombogera, M., Papageorgiou, H.: Multimodal analytics and its data ecosystem. In: Proceedings of the Workshop on Roadmapping the Future of Multimodal Interaction Research Including Business Opportunities and Challenges, pp. 1–4 (2014)

27.

Lu, L., Liu, D., Zhang, H.-J.: Automatic mood detection and tracking of music audio signals. IEEE Trans. Audio Speech Lang. Process. 14(1), 5–18 (2006)MathSciNetCrossRef

28.

Martin, P., Bateson, P.: Measuring Behaviour. Cambridge University Press (2007)

29.

McRorie, Margaret, Sneddon, Ian, McKeown, Gary, Bevacqua, Elisabetta, de Sevin, Etienne, Pelachaud, Catherine: Evaluation of four designed virtual agent personalities. IEEE Trans. Affect. Comput. 3(3), 311–322 (2011)CrossRef

30.

Mehu, M., Scherer, K.: A psycho-ethological approach to social signal processing. Cogn. Process. 13(2), 397–414 (2012)CrossRef

31.

Nass, C., Brave, S.: Wired for Speech: How Voice Activates and Advances the Human-Computer relationship. The MIT Press (2005)

32.

Nass, C., Steuer, J., Tauber, E.R.: Computers are social actors. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 72–78 (1994)

33.

Nie, J., Cui, P., Yan, Y., Huang, L., Li, Z., Wei, Z.: How your portrait impresses people?: inferring personality impressions from portrait contents. In: Proceedings of the ACM International Conference on Multimedia, pp. 905–908 (2014)

34.

Pantic, M., Vinciarelli, A.: Implicit human-centered tagging [social sciences]. IEEE Sign. Process. Mag. 26(6), 173–180 (2009)CrossRef

35.

Pentland, A.: Social signal processing. IEEE Sign. Proces. Mag. 24(4), 108–111 (2007)CrossRef

36.

Pentland, A.: Honest Signals. MIT Press (2010)

37.

Pianesi, F., Zancanaro, M., Not, E., Leonardi, C., Falcon, V., Lepri, B.: Multimodal support to group dynamics. Pers. Ubiquit. Comput. 12(3), 181–195 (2008)CrossRef

38.

Picard, R.W., Vyzas, E., Healey, J.: Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans. Pattern Anal. Mach. Intell. 23(10), 1175–1191 (2001)CrossRef

39.

Poggi, I., D’Errico, F.: Social signals: a framework in terms of goals and beliefs. Cogn. Process. 13(2), 427–445 (2012)CrossRef

40.

Poppe, R.: A survey on vision-based human action recognition. Image Vis. Comput. 28(6), 976–990 (2010)CrossRef

41.

Potamianos, A.: Cognitive multimodal processing: from signal to behavior. In: Proceedings of the Workshop on Roadmapping the Future of Multimodal Interaction Research Including Business Opportunities and Challenges, pp. 27–34 (2014)

42.

Quercia, D., Kosinski, M., Stillwell, D., Crowcroft, J.: Our twitter profiles, our selves: predicting personality with twitter. In: Proceedings of the IEEE International Conference on Social Computing, pp. 180–185 (2011)

43.

Quercia, D., Lambiotte, R., Stillwell, D., Kosinski, M., Crowcroft, J.: The personality of popular facebook users. In: Proceedings of the ACM Conference on Computer Supported Cooperative Work, pp. 955–964 (2012)

44.

Raento, M., Oulasvirta, A., Eagle, N.: Smartphones an emerging tool for social scientists. Sociol. Methods Res. 37(3), 426–454 (2009)MathSciNetCrossRef

45.

Ramanathan, S., Katti, H., Sebe, N., Kankanhalli, M., Chua, T.-S.: An eye fixation database for saliency detection in images. In: Proceedings of European Conference on Computer Vision, pp. 30–43 (2010)

46.

Reeves, B., Nass, C.: The media equation: how people treat computers, television, and new media like real people and places. Cambridge University Press (1996)

47.

Renals, S., Carletta, J., Edwards, K., Bourlard, H., Garner, P., Popescu-Belis, A., Klakow, D., Girenko, A., Petukova, V., Wacker, P., Joscelyne, A., Kompis, C., Aliwell, S., Stevens, W., Sabbah, Y.: ROCKIT: roadmap for conversational interaction technologies. In: Proceedings of the Workshop on Roadmapping the Future of Multimodal Interaction Research Including Business Opportunities and Challenges, pp. 39–42 (2014)

48.

Riccardi, G.: Towards healthcare personal agents. In: Proceedings of the Workshop on Roadmapping the Future of Multimodal Interaction Research Including Business Opportunities and Challenges, pp. 53–56 (2014)

49.

Sabourin, J.L., Lester, J.C.: Affect and engagement in game-based learning environments. IEEE Trans. Affect. Comput. 5(1), 45–56 (2014)CrossRef

50.

Salah, A.A.: Natural multimodal interaction with a social robot: what are the premises? In: Proceedings of the Workshop on Roadmapping the Future of Multimodal Interaction Research Including Business Opportunities and Challenges, pp. 43–45 (2014)

51.

Salamin, H., Polychroniou, A., Vinciarelli, A.: Automatic recognition of personality and conflict handling style in mobile phone conversations. In: Proceedings of International Workshop on Image Analysis for Multimedia Interactive Services, pp. 1–4 (2013)

52.

Sanchez-Cortes, D., Aran, O., Schmid-Mast, M., Gatica-Perez, D.: A nonverbal behavior approach to identify emergent leaders in small groups. IEEE Trans. Multimedia 14(3), 816–832 (2012)CrossRef

53.

Schegloff, E.A.: Analyzing single episodes of interaction: an exercise in conversation analysis. Soc. Psychol. Q. 101–114 (1987)

54.

Schroeder, M.: Expressive speech synthesis: past, present, and possible futures. In: Affective information processing, pp. 111–126. Springer (2009)

55.

Schuller, B., Batliner, A.: Computational Paralinguistics: Emotion, Affect and Personality in Speech and Language Processing. Wiley (2013)

56.

Schuller, B., Steidl, S., Batliner, A., Noeth, E., Vinciarelli, A., Burkhardt, F., van Son, R., Weninger, F., Eyben, F., Bocklet, T., Mohammadi, G., Weiss, B.: A survey on perceived speaker traits: personality, likability, pathology, and the first challenge. Comput. Speech Lang. 29(1), 100–131 (2015)CrossRef

57.

Staiano, J., Pianesi, F., Lepri, B., Sebe, N., Aharony, N., Pentland, A.: Friends don’t lie—inferring personality traits from social network structure. In: Proceedings of the ACM International Conference on Ubiquitous Computing, pp. 321–330 (2012)

58.

Tapus, A., Mataric, M.: Socially assistive robots: the link between personality, empathy, physiological signals, and task performance. In: Proceedings of AAAI Spring Symposium (2008)

59.

Tapus, A., Ţăpuş, C., Mataric, M.J.: User–robot personality matching and assistive robot behavior adaptation for post-stroke rehabilitation therapy. Intell. Serv. Robot. 1(2), 169–183 (2008)CrossRef

60.

Tausczik, Y.R., Pennebaker, J.W.: The psychological meaning of words: LIWC and computerized text analysis methods. J. Lang. Soc. Psychol. 29(1), 24–54 (2010)CrossRef

61.

Tkalčič, M., Burnik, U., Košir, A.: Using affective parameters in a content-based recommender system for images. User Model. User Adap. Inter. 20(4), 279–311 (2010)CrossRef

62.

Uleman, J.S., Newman, L.S., Moskowitz, G.B.: People as flexible interpreters: evidence and issues from spontaneous trait inference. Adv. Exp. Soc. Psychol. 28, 211–279 (1996)CrossRef

63.

Valstar, M.: Automatic behaviour understanding in medicine. In: Proceedings of the Workshop on Roadmapping the Future of Multimodal Interaction Research Including Business Opportunities and Challenges, pp. 57–60 (2014)

64.

Vinciarelli, A., Mohammadi, G.: A survey of personality computing. IEEE Trans. Affect. Comput. 5(3), 273–291 (2014)CrossRef

65.

Vinciarelli, A., Pantic, M., Bourlard, H., Pentland, A.: Social signal processing: state-of-the-art and future perspectives of an emerging domain. In: Proceedings of the ACM International Conference on Multimedia, pp. 1061–1070 (2008)

66.

Vinciarelli, A., Pantic, M., Bourlard, H.: Social signal processing: survey of an emerging domain. Image Vis. Comput. J. 27(12), 1743–1759 (2009)CrossRef

67.

Vinciarelli, A., Kim, S., Valente, F., Salamin, H.: Collecting data for socially intelligent surveillance and monitoring approaches: the case of conflict in competitive conversations. In: Proceedings of the International Symposium on Communications, Control and Signal Processing, pp. 1–4 (2012)

68.

Vinciarelli, A., Pantic, M., Heylen, D., Pelachaud, C., Poggi, I., D’Errico, F., Schroeder, M.: Bridging the gap between social animal and unsocial machine: a survey of social signal processing. IEEE Trans. Affect. Comput. 3(1), 69–87 (2012)CrossRef

69.

Vinciarelli, A., Chatziioannou, P., Esposito, A.: When the words are not everything: the use of laughter, fillers, back-channel, silence and overlapping speech in phone calls. Frontiers Hum. Media Interact. (2015)

70.

Vodlan, T., Tkalčič, M., Košir, A.: The impact of hesitation, a social signal, on a user’s quality of experience in multimedia content retrieval. Multimedia Tools Appl. 1–26 (2014)

71.

Weisman, O., Delaherche, E., Rondeau, M., Chetouani, M., Cohen, D., Feldman, R.: Oxytocin shapes parental motion during father-infant interaction. Biol. Lett. 9(6), 20130828 (2013)CrossRef

72.

Yaakub, C.Y., Sulaiman, N., Kim, C.W.: A study on personality identification using game based theory. In: Proceedings of the International Conference on Computer Technology and Development, pp. 732–734 (2010)

73.

Yanulevskaya, V., Uijlings, J., Bruni, E., Sartori, A., Zamboni, E., Bacci, F., Melcher, D., Sebe, N.: In the eye of the beholder: employing statistical analysis and eye tracking for analyzing abstract paintings. In: Proceedings of ACM International Conference on Multimedia, pp. 349–358 (2012)

74.

Yee, N., Ducheneaut, N., Nelson, L., Likarish, P.: Introverted elves and conscientious gnomes: the expression of personality in world of warcraft. In: Proceedings of the Annual Conference on Human Factors in Computing Systems, pp. 753–762, Vancouver (2011)

75.

Zhao, W., Chellappa, R., Phillips, P.J., Rosenfeld, A.: Face recognition: a literature survey. ACM Comput. Surv. 35(4), 399–458 (2003)CrossRef

Title: Computing Technologies for Social Signals
Author: Alessandro Vinciarelli
Publisher: Springer International Publishing
Book: Emotions and Personality in Personalized Services
Print ISBN: 978-3-319-31411-2

Electronic ISBN: 978-3-319-31413-6

Copyright Year: 2016
DOI: https://doi.org/10.1007/978-3-319-31413-6_6

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"