Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Visual Content Indexing and Retrieval with Psycho-Visual Models Read chapter Read first chapter

2017 | OriginalPaper | Chapter

Predicting Interestingness of Visual Content

Authors : Claire-Hélène Demarty, Mats Sjöberg, Mihai Gabriel Constantin, Ngoc Q. K. Duong, Bogdan Ionescu, Thanh-Toan Do, Hanli Wang

Published in: Visual Content Indexing and Retrieval with Psycho-Visual Models

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

The ability of multimedia data to attract and keep people’s interest for longer periods of time is gaining more and more importance in the fields of information retrieval and recommendation, especially in the context of the ever growing market value of social media and advertising. In this chapter we introduce a benchmarking framework (dataset and evaluation tools) designed specifically for assessing the performance of media interestingness prediction techniques. We release a dataset which consists of excerpts from 78 movie trailers of Hollywood-like movies. These data are annotated by human assessors according to their degree of interestingness. A real-world use scenario is targeted, namely interestingness is defined in the context of selecting visual content for illustrating a Video on Demand (VOD) website. We provide an in-depth analysis of the human aspects of this task, i.e., the correlation between perceptual characteristics of the content and the actual data, as well as of the machine aspects by overviewing the participating systems of the 2016 MediaEval Predicting Media Interestingness campaign. After discussing the state-of-art achievements, valuable insights, existing current capabilities as well as future challenges are presented.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Visual Saliency for the Visualization of Digital Paintings
Footnotes
Literature
1.
Almeida, J.: UNIFESP at MediaEval 2016 Predicting Media Interestingness Task. In: Proceedings of the MediaEval Workshop, Hilversum (2016)
2.
Almeida, J., Leite, N.J., Torres, R.S.: Comparison of video sequences with histograms of motion patterns. In: IEEE ICIP International Conference on Image Processing, pp. 3673–3676 (2011)
3.
Baveye, Y., Dellandréa, E., Chamaret, C., Chen, L.: Liris-accede: a video database for affective content analysis. IEEE Trans. Affect. Comput. 6(1), 43–55 (2015)CrossRef
4.
Berg, A.C., Berg, T.L., Daume, H., Dodge, J., Goyal, A., Han, X., Mensch, A., Mitchell, M., Sood, A., Stratos, K., et al.: Understanding and predicting importance in images. In: IEEE CVPR International Conference on Computer Vision and Pattern Recognition, pp. 3562–3569. IEEE, Providence (2012)
5.
Berlyne, D.E.: Conflict, Arousal and Curiosity. Mc-Graw-Hill, New York (1960)
6.
Boiman, O., Irani, M.: Detecting irregularities in images and in video. Int. J. Comput. Vis. 74(1), 17–31 (2007)CrossRef
7.
Bradley, R.A., Terry, M.E.: Rank analysis of incomplete block designs: the method of paired comparisons. Biometrika 39(3-4), 324–345 (1952)MathSciNetCrossRefMATH
8.
Breunig, M.M., Kriegel, H.P., Ng, R.T., Sander, J.: Lof: identifying density-based local outliers. In: ACM Sigmod Record, vol. 29, pp. 93–104. ACM, New York (2000)
9.
Bulling, A., Roggen, D.: Recognition of visual memory recall processes using eye movement analysis. In: Proceedings of the 13th international conference on Ubiquitous Computing, pp. 455–464. ACM, New York (2011)
10.
Chamaret, C., Demarty, C.H., Demoulin, V., Marquant, G.: Experiencing the interestingness concept within and between pictures. In: Proceeding of SPIE, Human Vision and Electronic Imaging (2016)
11.
Chen, A., Darst, P.W., Pangrazi, R.P.: An examination of situational interest and its sources. Br. J. Educ. Psychol. 71(3), 383–400 (2001)CrossRef
12.
Chen, S., Dian, Y., Jin, Q.: RUC at MediaEval 2016 Predicting Media Interestingness Task. In: Proceedings of the MediaEval Workshop, Hilversum (2016)
13.
14.
Constantin, M.G., Boteanu, B., Ionescu, B.: LAPI at MediaEval 2016 Predicting Media Interestingness Task. In: Proceedings of the MediaEval Workshop, Hilversum (2016)
15.
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE CVPR International Conference on Computer Vision and Pattern Recognition (2005)CrossRef
16.
Danelljan, M., Hager, G., Khan, F.S., Felsberg, M.: Accurate scale estimation for robust visual tracking. In: British Machine Vision Conference (2014)CrossRef
17.
Datta, R., Joshi, D., Li, J., Wang, J.Z.: Studying aesthetics in photographic images using a computational approach. In: IEEE ECCV European Conference on Computer Vision, pp. 288–301. Springer, Berlin (2006)
18.
Demarty, C.H., Sjöberg, M., Ionescu, B., Do, T.T., Wang, H., Duong, N.Q.K., Lefebvre, F.: Mediaeval 2016 Predicting Media Interestingness Task. In: Proceedings of the MediaEval Workshop, Hilversum (2016)
19.
Dhar, S., Ordonez, V., Berg, T.L.: High level describable attributes for predicting aesthetics and interestingness. In: IEEE International Conference on Computer Vision and Pattern Recognition (2011)CrossRef
20.
Elazary, L., Itti, L.: Interesting objects are visually salient. J. Vis. 8(3), 3–3 (2008)CrossRef
21.
Erdogan, G., Erdem, A., Erdem, E.: HUCVL at MediaEval 2016: predicting interesting key frames with deep models. In: Proceedings of the MediaEval Workshop, Hilversum (2016)
22.
23.
Gygli, M., Grabner, H., Riemenschneider, H., Nater, F., van Gool, L.: The interestingness of images. In: ICCV International Conference on Computer Vision (2013)CrossRef
24.
25.
Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: Advances in Neural Information Processing Systems, pp. 545–552 (2006)
26.
27.
Hsieh, L.C., Hsu, W.H., Wang, H.C.: Investigating and predicting social and visual image interestingness on social media by crowdsourcing. In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4309–4313. IEEE, Providence (2014)
28.
Hua, X.S., Yang, L., Wang, J., Wang, J., Ye, M., Wang, K., Rui, Y., Li, J.: Clickage: towards bridging semantic and intent gaps via mining click logs of search engines. In: ACM International Conference on Multimedia (2013)CrossRef
29.
Isola, P., Parikh, D., Torralba, A., Oliva, A.: Understanding the intrinsic memorability of images. In: Advances in Neural Information Processing Systems, pp. 2429–2437 (2011)
30.
Isola, P., Xiao, J., Torralba, A., Oliva, A.: What makes an image memorable? In: IEEE CVPR International Conference on Computer Vision and Pattern Recognition, pp. 145–152. IEEE, Providence (2011)
31.
Jiang, Y.G., Wang, Y., Feng, R., Xue, X., Zheng, Y., Yan, H.: Understanding and predicting interestingness of videos. In: AAAI Conference on Artificial Intelligence (2013)
32.
Jiang, Y.G., Dai, Q., Mei, T., Rui, Y., Chang, S.F.: Super fast event recognition in internet videos. IEEE Trans. Multimedia 177(8), 1–13 (2015)CrossRef
33.
Joachims, T.: Optimizing search engines using clickthrough data. In: ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 133–142. ACM, New York (2002)
34.
Ke, Y., Hoiem, D., Sukthankar, R.: Computer vision for music identification. In: IEEE CVPR International Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 597–604. IEEE, Providence (2005)
35.
Ke, Y., Tang, X., Jing, F.: The design of high-level features for photo quality assessment. In: IEEE CVPR International Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 419–426. IEEE, Providence (2006)
36.
Khosla, A., Raju, A.S., Torralba, A., Oliva, A.: Understanding and predicting image memorability at a large scale. In: International Conference on Computer Vision (ICCV) (2015)
37.
Krippendorff, K.: Content Analysis: An Introduction to Its Methodology, 3rd edn. Sage, Thousand Oaks (2013)
38.
Lam, V., Do, T., Phan, S., Le, D.D., Satoh, S., Duong, D.: NII-UIT at MediaEval 2016 Predicting Media Interestingness Task. In: Proceedings of the MediaEval Workshop, Hilversum (2016)
39.
Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: IEEE CVPR International Conference on Computer Vision and Pattern Recognition, pp. 2169–2178 (2006)
40.
Li, J., Barkowsky, M., Le Callet, P.: Boosting paired comparison methodology in measuring visual discomfort of 3dtv: performances of three different designs. In: Proceedings of SPIE Electronic Imaging, Stereoscopic Displays and Applications, vol. 8648 (2013)
41.
Li, L.J., Su, H., Fei-Fei, L., Xing, E.P.: Object bank: a high-level image representation for scene classification & semantic feature sparsification. In: Advances in Neural Information Processing Systems, pp. 1378–1386 (2010)
42.
Liem, C.: TUD-MMC at MediaEval 2016 Predicting Media Interestingness Task. In: Proceedings of the MediaEval Workshop, Hilversum (2016)
43.
Liu, F., Niu, Y., Gleicher, M.: Using web photos for measuring video frame interestingness. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 2058–2063 (2009)
44.
Liu, Y., Gu, Z., Cheung, Y.M.: Supervised manifold learning for media interestingness prediction. In: Proceedings of the MediaEval Workshop, Hilversum (2016)
45.
Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004)CrossRef
46.
47.
McCrae, R.R.: Aesthetic chills as a universal marker of openness to experience. Motiv. Emot. 31(1), 5–11 (2007)CrossRef
48.
Murray, N., Marchesotti, L., Perronnin, F.: Ava: a large-scale database for aesthetic visual analysis. In: IEEE CVPR International Conference on Computer Vision and Pattern Recognition, pp. 2408–2415. IEEE, Providence (2012)
49.
Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)CrossRefMATH
50.
Oliva, A., Torralba, A.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis. 42, 145–175 (2001)CrossRefMATH
51.
52.
Parekh, J., Parekh, S.: The MLPBOON Predicting Media Interestingness System for MediaEval 2016. In: Proceedings of the MediaEval Workshop, Hilversum (2016)
53.
Rayatdoost, S., Soleymani, M.: Ranking images and videos on visual interestingness by visual sentiment features. In: Proceedings of the MediaEval Workshop, Hilversum (2016)
54.
Schaul, T., Pape, L., Glasmachers, T., Graziano, V., Schmidhuber, J.: Coherence progress: a measure of interestingness based on fixed compressors. In: International Conference on Artificial General Intelligence, pp. 21–30. Springer, Berlin (2011)
55.
Shechtman, E., Irani, M.: Matching local self-similarities across images and videos. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE, Providence (2007)
56.
Shen, Y., Demarty, C.H., Duong, N.Q.K.: Technicolor@MediaEval 2016 Predicting Media Interestingness Task. In: Proceedings of the MediaEval Workshop, Hilversum (2016)
57.
Shen, Y., Demarty, C.H., Duong, N.Q.K.: Deep learning for multimodal-based video interestingness prediction. In: IEEE International Conference on Multimedia and Expo, ICME’17 (2017)
58.
Silvia, P.J.: What is interesting? Exploring the appraisal structure of interest. Emotion 5(1), 89 (2005)
59.
Silvia, P.J., Henson, R.A., Templin, J.L.: Are the sources of interest the same for everyone? using multilevel mixture models to explore individual differences in appraisal structures. Cognit. Emot. 23(7), 1389–1406 (2009)CrossRef
60.
Sjöberg, M., Baveye, Y., Wang, H., Quang, V.L., Ionescu, B., Dellandréa, E., Schedl, M., Demarty, C.H., Chen, L.: The mediaeval 2015 affective impact of movies task. In: Proceedings of the MediaEval Workshop, CEUR Workshop Proceedings (2015)
61.
62.
Spain, M., Perona, P.: Measuring and predicting object importance. Int. J. Comput. Vis. 91(1), 59–76 (2011)CrossRef
63.
Stein, B.E., Stanford, T.R.: Multisensory integration: current issues from the perspective of the single neuron. Nat. Rev. Neurosci. 9(4), 255–266 (2008)CrossRef
64.
Torresani, L., Szummer, M., Fitzgibbon, A.: Efficient object category recognition using classemes. In: IEEE ECCV European Conference on Computer Vision, pp. 776–789. Springer, Berlin (2010)
65.
Turner, S.A. Jr, Silvia, P.J.: Must interesting things be pleasant? A test of competing appraisal structures. Emotion 6(4), 670 (2006)
66.
Valdez, P., Mehrabian, A.: Effects of color on emotions. J. Exp. Psychol. Gen. 123(4), 394 (1994)CrossRef
67.
Vasudevan, A.B., Gygli, M., Volokitin, A., Gool, L.V.: Eth-cvl @ MediaEval 2016: Textual-visual embeddings and video2gif for video interestingness. In: Proceedings of the MediaEval Workshop, Hilversum (2016)
68.
Xiao, J., Hays, J., Ehinger, K., Oliva, A., Torralba, A.: Sun database: large-scale scene recognition from abbey to zoo. In: IEEE CVPR International Conference on Computer Vision and Pattern Recognition, pp. 3485–3492 (2010)
69.
Xu, B., Fu, Y., Jiang, Y.G.: BigVid at MediaEval 2016: predicting interestingness in images and videos. In: Proceedings of the MediaEval Workshop, Hilversum (2016)
70.
Yang, Y.H., Chen, H.H.: Ranking-based emotion recognition for music organization and retrieval. IEEE Trans. Audio Speech Lang. Process. 19(4), 762–774 (2011)CrossRef
71.
Yannakakis, G.N., Hallam, J.: Ranking vs. preference: a comparative study of self-reporting. In: International Conference on Affective Computing and Intelligent Interaction, pp. 437–446. Springer, Berlin (2011)
Metadata
Title
Predicting Interestingness of Visual Content
Authors
Claire-Hélène Demarty
Mats Sjöberg
Mihai Gabriel Constantin
Ngoc Q. K. Duong
Bogdan Ionescu
Thanh-Toan Do
Hanli Wang
Copyright Year
2017
Book
Visual Content Indexing and Retrieval with Psycho-Visual Models

Print ISBN: 978-3-319-57686-2

Electronic ISBN: 978-3-319-57687-9

Copyright Year: 2017

DOI
https://doi.org/10.1007/978-3-319-57687-9_10

Premium Partner

    Image Credits