Skip to main content

2017 | OriginalPaper | Buchkapitel

Emotional Video Scene Retrieval Using Multilayer Convolutional Network

verfasst von : Hiroki Nomiya, Shota Sakaue, Mitsuaki Maeda, Teruhisa Hochin

Erschienen in: Applied Computing and Information Technology

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In order to retrieve impressive scene from a video database, a scene retrieval method based on facial expression recognition (FER) is proposed. The proposed method will be useful to retrieve interesting scenes from lifelog videos. When an impressive event occurs, a certain facial expression will be observed in a person in the video. It is, therefore, important for the impressive scene retrieval to precisely recognize the facial expression of the person. In this paper, we try to construct accurate FER models by introducing a learning framework on the basis of multilayer convolutional network using a number of facial features defined as the positional relations between some facial feature points. The effectiveness of the proposed method is evaluated through an experiment to retrieve emotional scenes from a lifelog video database.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
1
One point which is a common end point of two line segments (i.e., \(p_{i}\) shown in Fig. 2) can be selected from 59 facial feature points. Then, two points can be selected from remaining 58 facial feature points. The number of possible facial features is thus \(59\times {}_{58}C_{2}=97527\).
 
2
For example, the 1st facial feature corresponds to \(f_{1,2,3}\) and the 2nd one is \(f_{1,2,4}\). The 97527th facial feature is \(f_{57,58,59}\).
 
3
The (i, 1) entry of \(\bar{P_{2}}\) is \(P_{2}(i, 1, 1, 1)\), and the (i, 2) entry of it is \(P_{2}(i, 1, 1, 2)\), and so on. The \((i, \frac{M}{T^{2}}\times \frac{M}{T^{2}}\times L_{2})\) entry of \(\bar{P_{2}}\) is \(P_{2}(i, \frac{M}{T^{2}}, \frac{M}{T^{2}}, L_{2})\).
 
Literatur
1.
Zurück zum Zitat T. Datchakorn, T. Yamasaki, and K. Aizawa, “Practical Experience Recording and Indexing of Life Log Video,” Proc. of the 2nd ACM Workshop on Continuous Archival and Retrieval of Personal Experiences, pp. 61–66, 2005. T. Datchakorn, T. Yamasaki, and K. Aizawa, “Practical Experience Recording and Indexing of Life Log Video,” Proc. of the 2nd ACM Workshop on Continuous Archival and Retrieval of Personal Experiences, pp. 61–66, 2005.
2.
Zurück zum Zitat D. Datcu and L. Rothkrantz, “Facial Expression Recognition in Still Pictures and Videos Using Active Appearance Models: A Comparison Approach,” Proc. of the 2007 International Conference on Computer Systems and Technologies, pp. 1–6, 2007. D. Datcu and L. Rothkrantz, “Facial Expression Recognition in Still Pictures and Videos Using Active Appearance Models: A Comparison Approach,” Proc. of the 2007 International Conference on Computer Systems and Technologies, pp. 1–6, 2007.
3.
Zurück zum Zitat G. Fanelli, A. Yao, P.-L. Noel, J. Gall, and L. V. Gool, “Hough Forest-based Facial Expression Recognition from Video Sequences,” Proc. of the 11th European Conference on Trends and Topics in Computer Vision, pp. 195–206, 2010. G. Fanelli, A. Yao, P.-L. Noel, J. Gall, and L. V. Gool, “Hough Forest-based Facial Expression Recognition from Video Sequences,” Proc. of the 11th European Conference on Trends and Topics in Computer Vision, pp. 195–206, 2010.
4.
Zurück zum Zitat H. Nomiya and T. Hochin, “Emotional Scene Retrieval from Lifelog Videos Using Evolutionary Feature Creation,” Studies in Computational Intelligence, Vol. 612, pp. 61–75, 2015. H. Nomiya and T. Hochin, “Emotional Scene Retrieval from Lifelog Videos Using Evolutionary Feature Creation,” Studies in Computational Intelligence, Vol. 612, pp. 61–75, 2015.
5.
Zurück zum Zitat Y. LeCun and Y. Bengio, “Convolutional Networks for Images, Speech, and Time Series,” The Handbook of Brain Theory and Neural Networks, MIT Press, pp. 255–258, 1998. Y. LeCun and Y. Bengio, “Convolutional Networks for Images, Speech, and Time Series,” The Handbook of Brain Theory and Neural Networks, MIT Press, pp. 255–258, 1998.
6.
Zurück zum Zitat Y. LeCun, Y. Bengio, and G. Hinton, “Deep Learning,” Nature Vol. 521, pp. 436–444, 2015. Y. LeCun, Y. Bengio, and G. Hinton, “Deep Learning,” Nature Vol. 521, pp. 436–444, 2015.
7.
Zurück zum Zitat T. Kanade, J. F. Cohn, and Y. Tian, “Comprehensive Database for Facial Expression Analysis,” Proc. of the 4th IEEE International Conference on Automatic Face and Gesture Recognition, pp. 46–53, 2000. T. Kanade, J. F. Cohn, and Y. Tian, “Comprehensive Database for Facial Expression Analysis,” Proc. of the 4th IEEE International Conference on Automatic Face and Gesture Recognition, pp. 46–53, 2000.
9.
Zurück zum Zitat H. Nomiya, S. Sakaue, and T. Hochin, “Recognition and Intensity Estimation of Facial Expression Using Ensemble Classifiers,” Proc. of 15th International Conference on Computer and Information Science, pp. 825–830, 2016. H. Nomiya, S. Sakaue, and T. Hochin, “Recognition and Intensity Estimation of Facial Expression Using Ensemble Classifiers,” Proc. of 15th International Conference on Computer and Information Science, pp. 825–830, 2016.
10.
Zurück zum Zitat N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A Simple Way to Prevent Neural Networks from Overfitting,” Journal of Machine Learning Research, Vol. 15, No. 1, pp. 1929–1958, 2014. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A Simple Way to Prevent Neural Networks from Overfitting,” Journal of Machine Learning Research, Vol. 15, No. 1, pp. 1929–1958, 2014.
11.
Zurück zum Zitat D. Kingma, J. Ba, “Adam: A Method for Stochastic Optimization,” Proc. of the 3rd International Conference for Learning Representations, arXiv:1412.6980, 2015. D. Kingma, J. Ba, “Adam: A Method for Stochastic Optimization,” Proc. of the 3rd International Conference for Learning Representations, arXiv:​1412.​6980, 2015.
Metadaten
Titel
Emotional Video Scene Retrieval Using Multilayer Convolutional Network
verfasst von
Hiroki Nomiya
Shota Sakaue
Mitsuaki Maeda
Teruhisa Hochin
Copyright-Jahr
2017
DOI
https://doi.org/10.1007/978-3-319-51472-7_8