Johannes Wagner
ABSTRACT
Automatic detection and interpretation of social signals carried by voice, gestures, mimics, etc. will play a key-role for next-generation interfaces as it paves the way towards a more intuitive and natural human-computer interaction. The paper at hand introduces Social Signal Interpretation (SSI), a framework for real-time recognition of social signals. SSI supports a large range of sensor devices, filter and feature algorithms, as well as, machine learning and pattern recognition tools. It encourages developers to add new components using SSI's C++ API, but also addresses front end users by offering an XML interface to build pipelines with a text editor. SSI is freely available under GPL at http://openssi.net.
- A. Camurri, P. Coletta, G. Varni, and S. Ghisio. Developing multimodal interactive systems with eyesweb xmi. In Proc. NIME, pages 305--308, New York, USA, 2007. ACM. Google Scholar
Digital Library
- G. Caridakis, J. Wagner, A. Raouzaiou, Z. Curto, E. André, and K. Karpouzis. A multimodal corpus for gesture expressivity analysis. In Proc. LREC, 2010.Google Scholar
- F. Eyben, M. Wöllmer, and B. Schuller. Opensmile: the munich versatile and fast open-source audio feature extractor. In Proc. MM, pages 1459--1462, New York, USA, 2010. ACM. Google ScholarDigital Library
- S. W. Gilroy, M. Cavazza, R. Chaignon, S.-M. M\"akel\"a, M. Niranen, E. André, T. Vogt, J. Urbain, H. Seichter, M. Billinghurst, and M. Benayoun. An affective model of user experience for interactive art. In Proc. ACE, pages 107--110, New York, USA, 2008. ACM. Google ScholarDigital Library
- F. Kistler, B. Endrass, I. Damian, C. Dang, and E. André. Natural interaction with culturally adaptive virtual characters. JMUI, pages 1--9.Google Scholar
- R. Niewiadomski, J. Hofmann, J. Urbain, T. Platt, J. Wagner, B. PIOT, H. Cakmak, S. Pammi, T. Baur, S. Dupont, M. Geist, F. Lingenfelser, G. McKeown, O. Pietquin, and W. Ruch. Laugh-aware virtual agent and its impact on user amusement . In Proc. AAMAS, Saint Paul, USA, May 2013. Google ScholarDigital Library
- M. Pantic, A. Nijholt, A. Pentland, and T. S. Huang. Human-centred intelligent human-computer interaction (hci$^2$): how far are we from attaining it? IJAACS, 1(2):168--187, August 2008. Google ScholarDigital Library
- S. Scherer, G. Stratou, M. Mahmoud, J. Boberg, J. Gratch, A. Rizzo, and L.-P. Morency. Automatic behavior descriptors for psychological disorder analysis. In Proc. FG, 2013.Google ScholarCross Ref
- M. Serrano, L. Nigay, J.-Y. L. Lawson, A. Ramsay, R. Murray-Smith, and S. Denef. The openinterface framework: a tool for multimodal interaction. In Proc. CHI, pages 3501--3506, New York, USA, 2008. ACM. Google ScholarDigital Library
- A. Spagnolli and L. Gamberini, editors. Validating presence by relying on recollection: Human experience and performance in the mixed reality system XIM, Padova, Italy, 16/10/2008 2008. CLEUP Cooperativa Libraria Universitaria Padova.Google Scholar
- J. Urbain, R. Niewiadomski, E. Bevacqua, T. Dutoit, A. Moinet, C. Pelachaud, B. Picart, J. Tilmanne, and J. Wagner. Avlaughtercycle. JMUI, 4:47--58, 2010.Google Scholar
- T. Vogt, E. André, and N. Bee. Emovoice - a framework for online recognitionof emotions from voice. In Proc. PIT, Kloster Irsee, Germany, June 2008. Springer. Google ScholarDigital Library
- J. Wagner, F. Lingenfelser, and E. André. The social signal interpretation framework (ssi) for real time signal processing and recognition. In Proc. of INTERSPEECH, 2011.Google ScholarCross Ref
- J. Wagner, F. Lingenfelser, E. André, and J. Kim. Exploring fusion methods for multimodal emotion recognition with missing data. IEEE TAC, 99, 2011. Google ScholarDigital Library
- J. Wagner, F. Lingenfelser, E. André, D. Mazzei, A. Tognetti, A. Lanatà, D. D. Rossi, A. Betella, R. Zucca, P. Omedas, and P. F. Verschure. A sensing architecture for empathetic data systems. In Proc. AH, page 96\textendash99, Stuttgart, Germany, 2013. ACM. Google ScholarDigital Library
Index Terms
- The social signal interpretation (SSI) framework: multimodal signal processing and recognition in real-time
Recommendations
ASSP4MI2016: 2nd international workshop on advancements in social signal processing for multimodal interaction (workshop summary)
ICMI '16: Proceedings of the 18th ACM International Conference on Multimodal InteractionThis paper gives a summary of the 2nd International Workshop on Advancements in Social Signal Processing for Multimodal Interaction (ASSP4MI). Following our successful 1st International Workshop on Advancements in Social Signal Processing for ...
Social signal processing for dummies
ICMI '16: Proceedings of the 18th ACM International Conference on Multimodal InteractionWe introduce SSJ Creator, a modern Android GUI enabling users to design and execute social signal processing pipelines using nothing but their smartphones and without writing a single line of code. It is based on a modular Java-based social signal ...
Challenges for Social Embodiment
RFMIR '14: Proceedings of the 2014 Workshop on Roadmapping the Future of Multimodal Interaction Research including Business Opportunities and ChallengesCurrent research in the area of social signal processing focuses on offline analysis of previously recorded human social cues. Approaches to exploit social signal processing techniques in naturalistic environments where agents socially interact with ...
Comments