Skip to main content
Top

2020 | OriginalPaper | Chapter

Affective Video Content Analysis Based on Two Compact Audio-Visual Features

Authors : Xiaona Guo, Wei Zhong, Long Ye, Li Fang, Qin Zhang

Published in: Digital TV and Wireless Multimedia Communication

Publisher: Springer Singapore

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In this paper, we propose a new framework for affective video content analysis by using two compact audio-visual features. In the proposed framework, the eGeMAPS is first calculated as global audio feature and then the key frames of optical flow images are fed to VGG19 network for implementing the transfer learning and visual feature extraction. Finally for model learning, the logistic regression is employed for affective video content classification. In the experiments, we perform the evaluations of audio and visual features on the dataset of Affective Impact of Movies Task 2015 (AIMT15), and compare our results with those of competition teams participated in AIMT15. The comparison results show that the proposed framework can achieve the comparable classification result with the first place of AIMT15 with a total feature dimension of 344, which is only about one thousandth of feature dimensions used in the first place of AIMT15.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Kang, H.-B.: Affective content detection using HMMs. In: Proceedings of the 11th ACM International Conference on Multimedia, pp. 259–262 (2003) Kang, H.-B.: Affective content detection using HMMs. In: Proceedings of the 11th ACM International Conference on Multimedia, pp. 259–262 (2003)
2.
go back to reference Wang, H.L., Cheong, L.-F.: Affective understanding in film. IEEE Trans. Circuits Syst. Video Technol. 16(6), 689–704 (2006)CrossRef Wang, H.L., Cheong, L.-F.: Affective understanding in film. IEEE Trans. Circuits Syst. Video Technol. 16(6), 689–704 (2006)CrossRef
3.
go back to reference Zhang, S., Huang, Q., Jiang, S., Gao, W., Tian, Q.: Affective visualization and retrieval for music video. IEEE Trans. Multimed. 12(6), 510–522 (2010)CrossRef Zhang, S., Huang, Q., Jiang, S., Gao, W., Tian, Q.: Affective visualization and retrieval for music video. IEEE Trans. Multimed. 12(6), 510–522 (2010)CrossRef
5.
go back to reference Baecchi, C., Uricchio, T., Bertini, M., Bimbo, A.D.: Deep sentiment features of context and faces for affective video analysis. In: Proceedings of the ACM International Conference on Multimedia Retrieval, pp. 72–77 (2017) Baecchi, C., Uricchio, T., Bertini, M., Bimbo, A.D.: Deep sentiment features of context and faces for affective video analysis. In: Proceedings of the ACM International Conference on Multimedia Retrieval, pp. 72–77 (2017)
6.
go back to reference Yi, Y., Wang, H.-L., Li, Q.Y.: CNN features for emotional impact of movies task. In: MediaEval (2018) Yi, Y., Wang, H.-L., Li, Q.Y.: CNN features for emotional impact of movies task. In: MediaEval (2018)
7.
go back to reference Baveye, Y., Chamaret, C., Dellandréa, E., Chen, L.M.: Affective video content analysis: a multidisciplinary insight. IEEE Trans. Affect. Comput. 9(4), 396–409 (2018)CrossRef Baveye, Y., Chamaret, C., Dellandréa, E., Chen, L.M.: Affective video content analysis: a multidisciplinary insight. IEEE Trans. Affect. Comput. 9(4), 396–409 (2018)CrossRef
8.
go back to reference Sjöberg, M., Baveye, Y., Wang, H.L., Quang, V.L., Ionescu, B., et al.: The MediaEval 2015 affective impact of movies task. In: MediaEval (2015) Sjöberg, M., Baveye, Y., Wang, H.L., Quang, V.L., Ionescu, B., et al.: The MediaEval 2015 affective impact of movies task. In: MediaEval (2015)
9.
go back to reference Dellandréa, E., Chen, L. M., Baveye, Y., Sjöberg, M. V. and Chamaret, C.: The MediaEval 2016 emotional impact of movies task. In: MediaEval (2016) Dellandréa, E., Chen, L. M., Baveye, Y., Sjöberg, M. V. and Chamaret, C.: The MediaEval 2016 emotional impact of movies task. In: MediaEval (2016)
10.
go back to reference Dellandréa, E., Huigsloot, M., Chen, L.M., Baveye, Y., Sjöberg, M.: The MediaEval 2017 emotional impact of movies task. In: MediaEval (2017) Dellandréa, E., Huigsloot, M., Chen, L.M., Baveye, Y., Sjöberg, M.: The MediaEval 2017 emotional impact of movies task. In: MediaEval (2017)
11.
go back to reference Dellandréa, E., Huigsloot, M., Chen, L.M., Baveye, Y., Xiao, Z.Z., Sjöberg, M.: The MediaEval 2018 emotional impact of movies task. In: MediaEval (2018) Dellandréa, E., Huigsloot, M., Chen, L.M., Baveye, Y., Xiao, Z.Z., Sjöberg, M.: The MediaEval 2018 emotional impact of movies task. In: MediaEval (2018)
12.
go back to reference Yi, Y., Wang, H.L., Zhang, B.W., Yu, J.: MIC-TJU in MediaEval 2015 affective impact of movies task. In: MediaEval (2015) Yi, Y., Wang, H.L., Zhang, B.W., Yu, J.: MIC-TJU in MediaEval 2015 affective impact of movies task. In: MediaEval (2015)
13.
go back to reference Lam, V., Le, S.P., Le, D.D., Satoh, S., Duong, D.A.: NII-UIT at MediaEval 2015 affective impact of movies task. In: MediaEval (2015) Lam, V., Le, S.P., Le, D.D., Satoh, S., Duong, D.A.: NII-UIT at MediaEval 2015 affective impact of movies task. In: MediaEval (2015)
14.
go back to reference Trigeorgis, G., Coutinho, E., Ringeval, F., Marchi, E., Zafeiriou, S., Schuller, B.W.: The ICL-TUM-PASSAU approach for the MediaEval 2015 affective impact of movies task. In: MediaEval (2015) Trigeorgis, G., Coutinho, E., Ringeval, F., Marchi, E., Zafeiriou, S., Schuller, B.W.: The ICL-TUM-PASSAU approach for the MediaEval 2015 affective impact of movies task. In: MediaEval (2015)
15.
go back to reference Dai, Q., et al.: Fudan-Huawei at MediaEval 2015: detecting violent scenes and affective impact in movies with deep learning. In: MediaEval (2015) Dai, Q., et al.: Fudan-Huawei at MediaEval 2015: detecting violent scenes and affective impact in movies with deep learning. In: MediaEval (2015)
16.
go back to reference Vlastelica, P.M., Hayrapetyan, S., Tapaswi, M., Stiefelhagen, R.: KIT at MediaEval 2015-evaluating visual cues for affective impact of movies task. In: MediaEval (2015) Vlastelica, P.M., Hayrapetyan, S., Tapaswi, M., Stiefelhagen, R.: KIT at MediaEval 2015-evaluating visual cues for affective impact of movies task. In: MediaEval (2015)
17.
go back to reference Seddati, O., Kulah, E., Pironkov, G., Dupont, S., Mahmoudi, S., Dutoit, T.: UMons at MediaEval 2015 affective impact of movies task including violent scenes detection. In: MediaEval (2015) Seddati, O., Kulah, E., Pironkov, G., Dupont, S., Mahmoudi, S., Dutoit, T.: UMons at MediaEval 2015 affective impact of movies task including violent scenes detection. In: MediaEval (2015)
18.
go back to reference Mironica, I., Ionescu, B., Sjöberg, M., Schedl, M., Skowron, M.: RFA at MediaEval 2015 affective impact of movies task: a multimodal approach. In: MediaEval (2015) Mironica, I., Ionescu, B., Sjöberg, M., Schedl, M., Skowron, M.: RFA at MediaEval 2015 affective impact of movies task: a multimodal approach. In: MediaEval (2015)
19.
go back to reference Chakraborty, R., Kumar Maurya, A., Pandharipande, M., Hassan, E., Ghosh, H., Kopparapu, S.K.: TCS-ILAB-MediaEval 2015: affective impact of movies and violent scene detection. In: MediaEval (2015) Chakraborty, R., Kumar Maurya, A., Pandharipande, M., Hassan, E., Ghosh, H., Kopparapu, S.K.: TCS-ILAB-MediaEval 2015: affective impact of movies and violent scene detection. In: MediaEval (2015)
20.
go back to reference Eyben, F., Scherer, K.R., Schuller, B.W., Sundberg, J., Andre, E., et al.: The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for voice research and affective computing. IEEE Trans. Affect. Comput. 7(2), 190–202 (2016)CrossRef Eyben, F., Scherer, K.R., Schuller, B.W., Sundberg, J., Andre, E., et al.: The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for voice research and affective computing. IEEE Trans. Affect. Comput. 7(2), 190–202 (2016)CrossRef
21.
go back to reference Aytar, Y., Vondrick, C., Torralba, A.: SoundNet: learning sound representations from unlabeled video. In: Advances in Neural Information Processing Systems, pp. 892–900 (2016) Aytar, Y., Vondrick, C., Torralba, A.: SoundNet: learning sound representations from unlabeled video. In: Advances in Neural Information Processing Systems, pp. 892–900 (2016)
Metadata
Title
Affective Video Content Analysis Based on Two Compact Audio-Visual Features
Authors
Xiaona Guo
Wei Zhong
Long Ye
Li Fang
Qin Zhang
Copyright Year
2020
Publisher
Springer Singapore
DOI
https://doi.org/10.1007/978-981-15-3341-9_29