Skip to main content

2023 | OriginalPaper | Buchkapitel

Emotion Recognition in Human Face Through Video Surveillance—A Survey of State-of-the-Art Approaches

verfasst von : Krishna Kant, D. B. Shah

Erschienen in: Information and Communication Technology for Competitive Strategies (ICTCS 2021)

Verlag: Springer Nature Singapore

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Emotion is a state of thoughts and feelings related to events and happenings. Emotion plays a significant role in all aspects of life. If we capture the emotion of an individual, then many issues can be resolved. Emotion recognition is becoming an interesting field of research nowadays due to huge amount of information available from various communication platforms. Emotion detection will play vital role as we are moving towards digital era from all the aspects of life. The capacity to understand the emotion by computer is necessary in many applications, especially emotion detected from video. At the present, emotional factors are significant because we get efficient aspect of customer behaviour. Video surveillance plays important role in recent times for face detection and feature extraction. Video surveillance will help us to understand the emotion that is being carried out by an individual. Facial expression is the natural aspect which helps to integrate the quality of emotion. Human face helps to get the emotion a person expressing in due course of action and activities. Many clues we can get from the human face which will help us to formulate and solve the issues and threat in the domain of emotion detection. This scholarly work is a comparative study of emotion recognition and classification of emotion from video sequences which exhibits the current trends and challenges in emotion recognition. Intensity of emotion and duration of an emotion are the two significant attributes for effective identification of emotion of an individual though video surveillance. This study of emotion recognition has given insight and future direction in the domain of emotion recognition technology.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Cohn JF (2010) Advances in behavioral science using automated facial image analysis and synthesis [social sciences]. IEEE Signal Process Mag 27(6):128–133 Cohn JF (2010) Advances in behavioral science using automated facial image analysis and synthesis [social sciences]. IEEE Signal Process Mag 27(6):128–133
2.
Zurück zum Zitat Smith GM (2003) Film structure and the emotion system. Cambridge University Press Smith GM (2003) Film structure and the emotion system. Cambridge University Press
3.
Zurück zum Zitat Dorai C, Venkatesh S (2001) Computational media aesthetics: finding meaning beautiful. IEEE Multimed 8:10–12CrossRef Dorai C, Venkatesh S (2001) Computational media aesthetics: finding meaning beautiful. IEEE Multimed 8:10–12CrossRef
4.
Zurück zum Zitat Chu W-S, De la Torre F, Cohn JF (2016) Selective transfer machine for personalized facial expression analysis. IEEE Trans Pattern Anal Mach Intell 39(3):529–545CrossRef Chu W-S, De la Torre F, Cohn JF (2016) Selective transfer machine for personalized facial expression analysis. IEEE Trans Pattern Anal Mach Intell 39(3):529–545CrossRef
5.
Zurück zum Zitat Eleftheriadis S, Rudovic O, Pantic M (2014) Discriminative shared Gaussian processes for multiview and view-invariant facial expression recognition. IEEE Trans Image Process 24(1):189–204MathSciNetCrossRef Eleftheriadis S, Rudovic O, Pantic M (2014) Discriminative shared Gaussian processes for multiview and view-invariant facial expression recognition. IEEE Trans Image Process 24(1):189–204MathSciNetCrossRef
6.
Zurück zum Zitat Hanjalic A (2005) Adaptive extraction of highlights from a sport video based on excitement modeling. IEEE Trans Multimed 7(6):1114–1122CrossRef Hanjalic A (2005) Adaptive extraction of highlights from a sport video based on excitement modeling. IEEE Trans Multimed 7(6):1114–1122CrossRef
7.
Zurück zum Zitat Hanjalic A (2006) Extracting moods from pictures and sounds: towards truly personalized tv. IEEE Signal Process Mag 23(2):90–100CrossRef Hanjalic A (2006) Extracting moods from pictures and sounds: towards truly personalized tv. IEEE Signal Process Mag 23(2):90–100CrossRef
8.
Zurück zum Zitat Mo S, Niu J, Su Y, Das SK (2018) A novel feature set for video emotion recognition. Neurocomputing 291:11–20CrossRef Mo S, Niu J, Su Y, Das SK (2018) A novel feature set for video emotion recognition. Neurocomputing 291:11–20CrossRef
10.
Zurück zum Zitat Scherer KR (1985) Handbook of methods in nonverbal behavior research. Cambridge University Press Scherer KR (1985) Handbook of methods in nonverbal behavior research. Cambridge University Press
11.
Zurück zum Zitat Sikkandar H, Thiyagarajan R (2021) Deep learning based facial expression recognition using improved cat swarm optimization. J Ambient Intell Humaniz Comput 12(2):3037–3053CrossRef Sikkandar H, Thiyagarajan R (2021) Deep learning based facial expression recognition using improved cat swarm optimization. J Ambient Intell Humaniz Comput 12(2):3037–3053CrossRef
12.
Zurück zum Zitat Niu B, Gao Z, Guo B (2021) Facial expression recognition with LBP and ORB features. Comput Intell Neurosci Niu B, Gao Z, Guo B (2021) Facial expression recognition with LBP and ORB features. Comput Intell Neurosci
13.
Zurück zum Zitat Tian W (2021) Personalized emotion recognition and emotion prediction system based on cloud computing. Math Probl Eng Tian W (2021) Personalized emotion recognition and emotion prediction system based on cloud computing. Math Probl Eng
14.
Zurück zum Zitat Arora M, Kumar M (2021) AutoFER: PCA and PSO based automatic facial emotion recognition. Multimed Tools Appl 80(2):3039–3049CrossRef Arora M, Kumar M (2021) AutoFER: PCA and PSO based automatic facial emotion recognition. Multimed Tools Appl 80(2):3039–3049CrossRef
15.
Zurück zum Zitat Gunawan TS, Ashraf A, Riza BS, Haryanto EV, Rosnelly R, Kartiwi M, Janin Z (2020) Development of video-based emotion recognition using deep learning with Google Colab. TELKOMNIKA 18(5):2463–2471CrossRef Gunawan TS, Ashraf A, Riza BS, Haryanto EV, Rosnelly R, Kartiwi M, Janin Z (2020) Development of video-based emotion recognition using deep learning with Google Colab. TELKOMNIKA 18(5):2463–2471CrossRef
16.
Zurück zum Zitat Liu X, Wang M (2020) Context-aware attention network for human emotion recognition in video. Adv Multimed Liu X, Wang M (2020) Context-aware attention network for human emotion recognition in video. Adv Multimed
17.
Zurück zum Zitat Gupta A, Arunachalam S, Balakrishnan R (2020) Deep self-attention network for facial emotion recognition. Procedia Comput Sci 171:1527–1534CrossRef Gupta A, Arunachalam S, Balakrishnan R (2020) Deep self-attention network for facial emotion recognition. Procedia Comput Sci 171:1527–1534CrossRef
18.
Zurück zum Zitat Ivanova E, Borzunov G (2020) Optimization of machine learning algorithm of emotion recognition in terms of human facial expressions. Procedia Comput Sci 169:244–248CrossRef Ivanova E, Borzunov G (2020) Optimization of machine learning algorithm of emotion recognition in terms of human facial expressions. Procedia Comput Sci 169:244–248CrossRef
19.
Zurück zum Zitat Zhang J, Wen X, Whang M (2020) Recognition of emotion according to the physical elements of the video. Sensors 20(3):649CrossRef Zhang J, Wen X, Whang M (2020) Recognition of emotion according to the physical elements of the video. Sensors 20(3):649CrossRef
20.
Zurück zum Zitat El Dahshan KA, Elsayed EK, Aboshoha A, Ebeid EA (2020) Recognition of facial emotions relying on deep belief networks and quantum particle swarm optimization. Int J Intell Eng Syst 13:90–101 El Dahshan KA, Elsayed EK, Aboshoha A, Ebeid EA (2020) Recognition of facial emotions relying on deep belief networks and quantum particle swarm optimization. Int J Intell Eng Syst 13:90–101
21.
Zurück zum Zitat Afdhal R, Ejbali R, Zaied M (2020) Primary emotions and recognition of their intensities. Comput J Afdhal R, Ejbali R, Zaied M (2020) Primary emotions and recognition of their intensities. Comput J
22.
Zurück zum Zitat Mellouk W, Handouzi W (2020) Facial emotion recognition using deep learning: review and insights. Procedia Comput Sci 175:689–694CrossRef Mellouk W, Handouzi W (2020) Facial emotion recognition using deep learning: review and insights. Procedia Comput Sci 175:689–694CrossRef
23.
Zurück zum Zitat Hossain MS, Muhammad G (2019) Emotion recognition using secure edge and cloud computing. Inf Sci 504:589–601MathSciNetCrossRef Hossain MS, Muhammad G (2019) Emotion recognition using secure edge and cloud computing. Inf Sci 504:589–601MathSciNetCrossRef
24.
Zurück zum Zitat Vinola C et al (2019) Smile intensity recognition in real time videos: fuzzy system approach. Multimed Tools Appl 78(11):15033–15052CrossRef Vinola C et al (2019) Smile intensity recognition in real time videos: fuzzy system approach. Multimed Tools Appl 78(11):15033–15052CrossRef
25.
Zurück zum Zitat Bahreini K, Van der Vegt W, Westera W (2019) A fuzzy logic approach to reliable real-time recognition of facial emotions. Multimed Tools Appl 78(14):18943–18966CrossRef Bahreini K, Van der Vegt W, Westera W (2019) A fuzzy logic approach to reliable real-time recognition of facial emotions. Multimed Tools Appl 78(14):18943–18966CrossRef
26.
Zurück zum Zitat Adouani A, Henia WMB, Lachiri Z (2019) Comparison of Haar-like, HOG and LBP approaches for face detection in video sequences. In: 2019 16th international multi-conference on systems, signals & devices (SSD). IEEE, pp 266–271 Adouani A, Henia WMB, Lachiri Z (2019) Comparison of Haar-like, HOG and LBP approaches for face detection in video sequences. In: 2019 16th international multi-conference on systems, signals & devices (SSD). IEEE, pp 266–271
27.
Zurück zum Zitat Avots E, Sapiński T, Bachmann M, Kamińska D (2019) Audiovisual emotion recognition in wild. Mach Vis Appl 30(5):975–985CrossRef Avots E, Sapiński T, Bachmann M, Kamińska D (2019) Audiovisual emotion recognition in wild. Mach Vis Appl 30(5):975–985CrossRef
28.
Zurück zum Zitat Niu G, Chen Q (2018) Learning an video frame-based face detection system for security fields. J Vis Commun Image Represent 55:457–463CrossRef Niu G, Chen Q (2018) Learning an video frame-based face detection system for security fields. J Vis Commun Image Represent 55:457–463CrossRef
29.
Zurück zum Zitat Wang Q, Yang H, Yu Y (2018) Facial expression video analysis for depression detection in Chinese patients. J Vis Commun Image Represent 57:228–233CrossRef Wang Q, Yang H, Yu Y (2018) Facial expression video analysis for depression detection in Chinese patients. J Vis Commun Image Represent 57:228–233CrossRef
30.
Zurück zum Zitat Baskar A, Kumar TG (2018) Facial expression classification using machine learning approach: a review. Data Eng Intell Comput 337–345 Baskar A, Kumar TG (2018) Facial expression classification using machine learning approach: a review. Data Eng Intell Comput 337–345
31.
Zurück zum Zitat Giannakakis G, Pediaditis M, Manousos D, Kazantzaki E, Chiarugi F, Simos PG, Marias K, Tsiknakis M (2017) Stress and anxiety detection using facial cues from videos. Biomed Signal Process Control 31:89–101CrossRef Giannakakis G, Pediaditis M, Manousos D, Kazantzaki E, Chiarugi F, Simos PG, Marias K, Tsiknakis M (2017) Stress and anxiety detection using facial cues from videos. Biomed Signal Process Control 31:89–101CrossRef
32.
Zurück zum Zitat Rani PI, Muneeswaran K (2017) Recognize the facial emotion in video sequences using eye and mouth temporal gabor features. Multimed Tools Appl 76(7):10017–10040CrossRef Rani PI, Muneeswaran K (2017) Recognize the facial emotion in video sequences using eye and mouth temporal gabor features. Multimed Tools Appl 76(7):10017–10040CrossRef
33.
Zurück zum Zitat Abdulsalam WH, Alhamdani RS, Abdullah MN (2019) Facial emotion recognition from videos using deep convolutional neural networks. Int J Mach Learn Comput 9(1):14–19CrossRef Abdulsalam WH, Alhamdani RS, Abdullah MN (2019) Facial emotion recognition from videos using deep convolutional neural networks. Int J Mach Learn Comput 9(1):14–19CrossRef
34.
Zurück zum Zitat Selvam IRP, Prasath TH (2019) Dynamic eyes and mouth reinforced LBP histogram descriptors based on emotion classification in video sequences. In: Smart techniques for a smarter planet. Springer, pp 185–209 Selvam IRP, Prasath TH (2019) Dynamic eyes and mouth reinforced LBP histogram descriptors based on emotion classification in video sequences. In: Smart techniques for a smarter planet. Springer, pp 185–209
35.
Zurück zum Zitat Salah AA, Kaya H, Gürpınar F (2019) Video-based emotion recognition in the wild. In: Multimodal behavior analysis in the wild. Elsevier, pp 369–386 Salah AA, Kaya H, Gürpınar F (2019) Video-based emotion recognition in the wild. In: Multimodal behavior analysis in the wild. Elsevier, pp 369–386
Metadaten
Titel
Emotion Recognition in Human Face Through Video Surveillance—A Survey of State-of-the-Art Approaches
verfasst von
Krishna Kant
D. B. Shah
Copyright-Jahr
2023
Verlag
Springer Nature Singapore
DOI
https://doi.org/10.1007/978-981-19-0095-2_6