Top

Published in:

2022 | OriginalPaper | Chapter

Emotion Recognition - A Tool to Improve Meeting Experience for Visually Impaired

Authors : Mathieu Lutfallah, Benno Käch, Christian Hirt, Andreas Kunz

Published in: Computers Helping People with Special Needs

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Facial expressions play an important role in human communication since they enrich spoken information and help convey additional sentiments e.g. mood. Among others, they non-verbally express a partner’s agreement or disagreement to spoken information. Further, together with the audio signal, humans can even detect nuances of changes in a person’s mood. However, facial expressions remain inaccessible to the blind and visually impaired, and also the voice signal alone might not carry enough mood information.

Emotion recognition research mainly focused on detecting one of seven emotion classes. Such emotions are too detailed, and having an overall impression of primary emotional states such as positive, negative, or neutral is more beneficial for the visually impaired person in a lively discussion within a team. Thus, this paper introduces an emotion recognition system that allows a real-time detection of the emotions “agree”, “neutral”, and “disagree”, which are seen as the most important ones during a lively discussion. The proposed system relies on a combination of neural networks that allow extracting emotional states while leveraging the temporal information from videos.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Non-verbal Communication and Joint Attention Between People with and Without Visual Impairments: Deriving Guidelines for Inclusive Conversations in Virtual Realities

next chapter Pointing, Pairing and Grouping Gesture Recognition in Virtual Reality

https://cs.anu.edu.au/few/AFEW.html.

http://dlib.net/.

Chen, J., Chen, Z., Chi, Z., Fu, H.: Emotion recognition in the wild with feature fusion and multiple kernel learning. In: Proceedings of the 16th International Conference on Multimodal Interaction, pp. 508–513. ICMI 2014, Association for Computing Machinery, New York, NY, USA (2014). https://doi.org/10.1145/2663204.2666277

Dhall, A., Goecke, R., Joshi, J., Wagner, M., Gedeon, T.: Emotion recognition in the wild challenge (EmotiW) challenge and workshop summary. In: Proceedings of the 15th ACM on International conference on multimodal interaction, pp. 371–372. ICMI 2013, Association for Computing Machinery, New York, NY, USA (2013). https://doi.org/10.1145/2522848.2531749

Dzedzickis, A., Kaklauskas, A., Bucinskas, V.: Human emotion recognition: review of sensors and methods. Sensors 20(3) (2020). https://doi.org/10.3390/s20030592

Ekman, P.: An argument for basic emotions. Cogn. Emot. 6(3–4), 169–200 (1992)CrossRef

El-Gayyar, M., ElYamany, H.F., Gaber, T., Hassanien, A.E.: Social network framework for deaf and blind people based on cloud computing. In: 2013 Federated Conference on Computer Science and Information Systems, pp. 1313–1319. IEEE (2013)

He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CoRR abs/1512.03385 (2015). http://arxiv.org/abs/1512.03385

Kaklauskas, A., et al.: Affective tutoring system for built environment management. Comput. Educ. 82, 202–216 (2015). https://doi.org/10.1016/j.compedu.2014.11.016, https://www.sciencedirect.com/science/article/pii/S0360131514002693

Knyazev, B., Shvetsov, R., Efremova, N., Kuharenko, A.: Convolutional neural networks pretrained on large face recognition datasets for emotion classification from video. CoRR abs/1711.04598 (2017). http://arxiv.org/abs/1711.04598

Kunz, A., et al.: Accessibility of brainstorming sessions for blind people. In: Miesenberger, K., Fels, D., Archambault, D., Peňáz, P., Zagler, W. (eds.) ICCHP 2014. LNCS, vol. 8547, pp. 237–244. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-08596-8_38CrossRef

10.

Li, S., et al.: Bi-modality fusion for emotion recognition in the wild. In: 2019 International Conference on Multimodal Interaction, pp. 589–594. ICMI 2019, Association for Computing Machinery, New York, NY, USA (2019). https://doi.org/10.1145/3340555.3355719

11.

Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, pp. 94–101 (2010). https://doi.org/10.1109/CVPRW.2010.5543262

12.

Marinoiu, E., Zanfir, M., Olaru, V., Sminchisescu, C.: 3D human sensing, action and emotion recognition in robot assisted therapy of children with autism. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2158–2167 (2018)

13.

Mittal, T., Guhan, P., Bhattacharya, U., Chandra, R., Bera, A., Manocha, D.: Emoticon: context-aware multimodal emotion recognition using frege’s principle. CoRR abs/2003.06692 (2020). https://arxiv.org/abs/2003.06692

14.

Peng, A.Y., Koh, Y.S., Riddle, P.J., Pfahringer, B.: Using supervised pretraining to improve generalization of neural networks on binary classification problems. In: ECML/PKDD (2018)

15.

Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015, Conference Track Proceedings (2015). http://arxiv.org/abs/1409.1556

16.

Xie, S., Girshick, R., Dollar, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017

17.

Yildirim, S., et al.: An acoustic study of emotions expressed in speech. In: Eighth International Conference on Spoken Language Processing (2004)

Title: Emotion Recognition - A Tool to Improve Meeting Experience for Visually Impaired
Authors: Mathieu Lutfallah
Benno Käch
Christian Hirt
Andreas Kunz
Publisher: Springer International Publishing
Book: Computers Helping People with Special Needs
Print ISBN: 978-3-031-08647-2

Electronic ISBN: 978-3-031-08648-9

Copyright Year: 2022
DOI: https://doi.org/10.1007/978-3-031-08648-9_35

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"