Skip to main content
Erschienen in: Multimedia Systems 2/2015

01.03.2015 | Special Issue Paper

Image aesthetics enhancement using composition-based saliency detection

verfasst von: Handong Zhao, Jingjing Chen, Yahong Han, Xiaochun Cao

Erschienen in: Multimedia Systems | Ausgabe 2/2015

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Visual saliency detection and segmentation are widely used in many applications in image processing and computer vision. However, existing saliency detection methods have not fully taken the spatial information of salient regions into account. Inspired by the basic photographic composition rules, we present a novel saliency detection method, which utilizes the knowledge of photographic composition as priors to improve the saliency detection results. Moreover, an online parameter selection method is proposed when utilizing GrabCut to achieve the saliency segmentation result. Besides, to test the applicability of our method, we present a novel post-processing framework for the photographs to be more artistic. The salient region and depth map are firstly computed. The salient region keeps its sharpness, while other parts in the photograph get blurred based on the depth map. To our best knowledge, this is a novel image-based attempt to enhance aesthetics by post-processing a photograph via realistic blurring. We test our method on the 1,000 benchmark test images and dataset MSRA. Extensive experimental results show the applicability and effectiveness of our method.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Achanta, R., Estrada, F.J., Wils, P., Süsstrunk, S.: Salient region detection and segmentation. In: International conference on computer vision, system, pp. 66–75 (2008) Achanta, R., Estrada, F.J., Wils, P., Süsstrunk, S.: Salient region detection and segmentation. In: International conference on computer vision, system, pp. 66–75 (2008)
2.
Zurück zum Zitat Achanta, R., Hemami, S.S., Estrada, F.J., Süsstrunk, S.: Frequency-tuned salient region detection. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1597–1604 (2009) Achanta, R., Hemami, S.S., Estrada, F.J., Süsstrunk, S.: Frequency-tuned salient region detection. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1597–1604 (2009)
3.
Zurück zum Zitat Bae, S., Durand, F.: Defocus magnification. Comput. Graph. Forum 26, 571–579 (2007)CrossRef Bae, S., Durand, F.: Defocus magnification. Comput. Graph. Forum 26, 571–579 (2007)CrossRef
4.
Zurück zum Zitat Chen, J., Zhao, H., Han, Y., Cao, X.: Visual saliency detection based on photographic composition. In: International conference on internet multimedia computing and service, pp. 13–16 (2013) Chen, J., Zhao, H., Han, Y., Cao, X.: Visual saliency detection based on photographic composition. In: International conference on internet multimedia computing and service, pp. 13–16 (2013)
5.
Zurück zum Zitat Cheng, M.M., Mitra, N.J., Huang, X., Hu, S.M.: Salient shape: group saliency in image collections. Vis. Comput. 30(4), 443–453 (2014) Cheng, M.M., Mitra, N.J., Huang, X., Hu, S.M.: Salient shape: group saliency in image collections. Vis. Comput. 30(4), 443–453 (2014)
6.
Zurück zum Zitat Cheng, M.M., Zhang, G.X., Mitra, N.J., Huang, X., Hu, S.M.: Global contrast based salient region detection. In: CVPR, pp. 409–416 (2011) Cheng, M.M., Zhang, G.X., Mitra, N.J., Huang, X., Hu, S.M.: Global contrast based salient region detection. In: CVPR, pp. 409–416 (2011)
7.
Zurück zum Zitat Daly, S.: The visible differences predictor: an algorithm for the assessment of image fidelity. In: SPIE/IS&T 1992 Symposium on Electronic Imaging: Science and Technology, pp 2–15 (1992) Daly, S.: The visible differences predictor: an algorithm for the assessment of image fidelity. In: SPIE/IS&T 1992 Symposium on Electronic Imaging: Science and Technology, pp 2–15 (1992)
8.
Zurück zum Zitat Das, S., Ahuja, N.: Performance analysis of stereo, vergence, and focus as depth cues for active vision. IEEE. Trans. Pattern Anal. Mach. Intell.17(12), 1213–1219 (1995)CrossRef Das, S., Ahuja, N.: Performance analysis of stereo, vergence, and focus as depth cues for active vision. IEEE. Trans. Pattern Anal. Mach. Intell.17(12), 1213–1219 (1995)CrossRef
9.
Zurück zum Zitat Datta, R., Joshi, D., Li, J., Wang, J.Z.: Studying aesthetics in photographic images using a computational approach. In: ECCV, pp. 288–301 (2006) Datta, R., Joshi, D., Li, J., Wang, J.Z.: Studying aesthetics in photographic images using a computational approach. In: ECCV, pp. 288–301 (2006)
10.
Zurück zum Zitat Datta, R., Li, J., Wang, J.Z.: Learning the consensus on visual quality for next generation image management. In: ACM multimedia, pp. 533–536 (2007) Datta, R., Li, J., Wang, J.Z.: Learning the consensus on visual quality for next generation image management. In: ACM multimedia, pp. 533–536 (2007)
11.
Zurück zum Zitat Datta, R., Li, J., Wang, J.Z.: Algorithmic inferencing of aesthetics and emotion in natural images: An exposition. In: ICIP, special session on image aesthetics: mood and emotion, pp. 105–108 (2008) Datta, R., Li, J., Wang, J.Z.: Algorithmic inferencing of aesthetics and emotion in natural images: An exposition. In: ICIP, special session on image aesthetics: mood and emotion, pp. 105–108 (2008)
12.
Zurück zum Zitat Davies, E.R.: Machine vision: theory, algorithms and practicalities. In: pp. 42–44. Academic Press, London (1990) Davies, E.R.: Machine vision: theory, algorithms and practicalities. In: pp. 42–44. Academic Press, London (1990)
13.
Zurück zum Zitat Eltoukhy, H.A., Kavusi, S.: Computationally efficient algorithm for multifocus image reconstruction. In: Sensors and camera systems for scientific, industrial, and digital photography applications, pp. 332–341 (2003) Eltoukhy, H.A., Kavusi, S.: Computationally efficient algorithm for multifocus image reconstruction. In: Sensors and camera systems for scientific, industrial, and digital photography applications, pp. 332–341 (2003)
14.
Zurück zum Zitat Forsyth, D.A., Ponce, J.: Computer vision: a modern approach. Prentice Hall Professional Technical Reference (2002) Forsyth, D.A., Ponce, J.: Computer vision: a modern approach. Prentice Hall Professional Technical Reference (2002)
15.
Zurück zum Zitat Goferman, S., Manor, L.Z., Tal, A.: Context-aware saliency detection. In: CVPR, pp. 2376–2383 (2010) Goferman, S., Manor, L.Z., Tal, A.: Context-aware saliency detection. In: CVPR, pp. 2376–2383 (2010)
16.
Zurück zum Zitat Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: Advances in Neural Information Processing Systems (NIPS), pp. 545–552 (2006) Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: Advances in Neural Information Processing Systems (NIPS), pp. 545–552 (2006)
17.
Zurück zum Zitat Hong, R., Wang, M., Xu, M., Yan, S., Chua, T.S.: Dynamic captioning: Video accessibility enhancement for hearing impairment. In: ACM multimedia, pp. 421–430 (2010) Hong, R., Wang, M., Xu, M., Yan, S., Chua, T.S.: Dynamic captioning: Video accessibility enhancement for hearing impairment. In: ACM multimedia, pp. 421–430 (2010)
18.
Zurück zum Zitat Hong, R., Wang, M., Yuan, X.T., Xu, M., Jiang, J., Yan, S., Chua, T.S.: Video accessibility enhancement for hearing impaired users. ACM. Trans. Multimed. Comput.7S, 24–42 (2011) Hong, R., Wang, M., Yuan, X.T., Xu, M., Jiang, J., Yan, S., Chua, T.S.: Video accessibility enhancement for hearing impaired users. ACM. Trans. Multimed. Comput.7S, 24–42 (2011)
19.
Zurück zum Zitat Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8 (2007) Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8 (2007)
20.
Zurück zum Zitat Huhle, B., Schairer, T., Jenke, P., Straßer, W.: Realistic depth blur for images with range data. Dynamic 3D, imaging pp. 84–95 (2009) Huhle, B., Schairer, T., Jenke, P., Straßer, W.: Realistic depth blur for images with range data. Dynamic 3D, imaging pp. 84–95 (2009)
21.
Zurück zum Zitat Krages, B.: Photography: the art of composition, 1st edn. Allworth Press, New York (2005) Krages, B.: Photography: the art of composition, 1st edn. Allworth Press, New York (2005)
22.
Zurück zum Zitat Liu, L., Chen, R., Wolf, L., Cohen-Or, D.: Optimizing photo composition. Comput. Graph. Forum 29, 469–478 (2010)CrossRef Liu, L., Chen, R., Wolf, L., Cohen-Or, D.: Optimizing photo composition. Comput. Graph. Forum 29, 469–478 (2010)CrossRef
23.
Zurück zum Zitat Liu, T., Sun, J., Zheng, N.N., Tang, X., Shum, H.Y.: Learning to detect a salient object. In: CVPR, pp. 1–8 (2007) Liu, T., Sun, J., Zheng, N.N., Tang, X., Shum, H.Y.: Learning to detect a salient object. In: CVPR, pp. 1–8 (2007)
24.
Zurück zum Zitat Ma, Y.F., Zhang, H.: Contrast-based image attention analysis by using fuzzy growing. In: ACM multimedia, pp. 374–381 (2003) Ma, Y.F., Zhang, H.: Contrast-based image attention analysis by using fuzzy growing. In: ACM multimedia, pp. 374–381 (2003)
25.
Zurück zum Zitat Mahmoud, T.A., Marshall, S.: Threshold decomposition driven adaptive morphological filter for image sharpening. In: VISAPP, pp. 40–45 (2007) Mahmoud, T.A., Marshall, S.: Threshold decomposition driven adaptive morphological filter for image sharpening. In: VISAPP, pp. 40–45 (2007)
26.
Zurück zum Zitat Maki, A., Watanabe, M., Geotensity, C.W.: Combining motion and lighting for 3D surface reconstruction. Int. J. Comput. Vis.48(2), 75–90 (2002)CrossRefMATH Maki, A., Watanabe, M., Geotensity, C.W.: Combining motion and lighting for 3D surface reconstruction. Int. J. Comput. Vis.48(2), 75–90 (2002)CrossRefMATH
27.
Zurück zum Zitat Malik, J., Rosenholtz, R.: Computing local surface orientation and shape from texture for curved surfaces. Int. J. Comput. Vis. 23(2), 149–168 (1997)CrossRef Malik, J., Rosenholtz, R.: Computing local surface orientation and shape from texture for curved surfaces. Int. J. Comput. Vis. 23(2), 149–168 (1997)CrossRef
28.
Zurück zum Zitat McGuire, M., Matusik, W., Pfister, H., Hughes, J.F., Durand, F.: Defocus video matting. ACM Trans. Graph. 24(3), 567–576 (2005) McGuire, M., Matusik, W., Pfister, H., Hughes, J.F., Durand, F.: Defocus video matting. ACM Trans. Graph. 24(3), 567–576 (2005)
29.
Zurück zum Zitat Moutoussis, K., Zeki, S.: A direct demonstration of perceptual asynchrony in vision. In: Proceedings of the Royal Society of London. Series B: Biological Sciences, pp. 393–399 (1997) Moutoussis, K., Zeki, S.: A direct demonstration of perceptual asynchrony in vision. In: Proceedings of the Royal Society of London. Series B: Biological Sciences, pp. 393–399 (1997)
30.
Zurück zum Zitat Nagai, T., Ikehara, M., Kurematsu, A.: Hmm-based surface reconstruction from single images. Syst. Comput. Jpn. 38(11), 80–89 (2007)CrossRef Nagai, T., Ikehara, M., Kurematsu, A.: Hmm-based surface reconstruction from single images. Syst. Comput. Jpn. 38(11), 80–89 (2007)CrossRef
31.
Zurück zum Zitat Peng, B., Veksler, O.: Parameter selection for graph cut based image segmentation. In: BMVC, pp. 332–341 (2008) Peng, B., Veksler, O.: Parameter selection for graph cut based image segmentation. In: BMVC, pp. 332–341 (2008)
32.
Zurück zum Zitat Peters, G.: Aesthetic primitives of images for visualization. In: IEEE international conference on information visualization, pp. 316–325 (2007) Peters, G.: Aesthetic primitives of images for visualization. In: IEEE international conference on information visualization, pp. 316–325 (2007)
33.
Zurück zum Zitat Rother, C., Kolmogorov, V., Blake, A.: Grabcut: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph 23, 309–314 (2004)CrossRef Rother, C., Kolmogorov, V., Blake, A.: Grabcut: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph 23, 309–314 (2004)CrossRef
34.
Zurück zum Zitat Saxena, A., Chung, S.H., Ng, A.Y.: Learning depth from single monocular images. In: Advances in Neural Information Processing Systems (NIPS) (2005) Saxena, A., Chung, S.H., Ng, A.Y.: Learning depth from single monocular images. In: Advances in Neural Information Processing Systems (NIPS) (2005)
35.
Zurück zum Zitat Saxena, A., Chung, S.H., Ng, A.Y.: 3-d depth reconstruction from a single still image. Int. J. Comput. Vis. 76, 53–69 (2008)CrossRef Saxena, A., Chung, S.H., Ng, A.Y.: 3-d depth reconstruction from a single still image. Int. J. Comput. Vis. 76, 53–69 (2008)CrossRef
36.
Zurück zum Zitat Saxena, A., Sun, M., Ng, A.Y.: Make3d: learning 3d scene structure from a single still image. IEEE Trans. Pattern Anal. Mach. Intell. 31(5), 824–840 (2009)CrossRef Saxena, A., Sun, M., Ng, A.Y.: Make3d: learning 3d scene structure from a single still image. IEEE Trans. Pattern Anal. Mach. Intell. 31(5), 824–840 (2009)CrossRef
37.
Zurück zum Zitat Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47, 1–35 (2002)CrossRef Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47, 1–35 (2002)CrossRef
38.
Zurück zum Zitat Schavemaker, J.G.M., Reinders, M.J.T., Gerbrands, J.J., Backer, E.: Image sharpening by morphological filtering. Pattern Recogn. 33(6), 997–1012 (2000) Schavemaker, J.G.M., Reinders, M.J.T., Gerbrands, J.J., Backer, E.: Image sharpening by morphological filtering. Pattern Recogn. 33(6), 997–1012 (2000)
39.
Zurück zum Zitat Subbarao, M., Wei, T.C., Surya, G.: Focused image recovery from two defocused images recorded with different camera settings. IEEE Trans. Image Process. 4(12), 1613–1628 (1995) Subbarao, M., Wei, T.C., Surya, G.: Focused image recovery from two defocused images recorded with different camera settings. IEEE Trans. Image Process. 4(12), 1613–1628 (1995)
40.
Zurück zum Zitat Tatler, B.W.: The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions. J. Vis. pp. 1–17 (2007) Tatler, B.W.: The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions. J. Vis. pp. 1–17 (2007)
41.
Zurück zum Zitat Valenti, R., Jaimes, A., Sebe, N.: Sonify your face: Facial expressions for sound generation. In: ACM multimedia, pp. 1363–1372 (2010) Valenti, R., Jaimes, A., Sebe, N.: Sonify your face: Facial expressions for sound generation. In: ACM multimedia, pp. 1363–1372 (2010)
42.
Zurück zum Zitat Valenti, R., Sebe, N., Gevers, T.: Facial expression recognition: a fully integrated approach. In: International conference on image analysis and processing workshops, pp. 125–130 (2007) Valenti, R., Sebe, N., Gevers, T.: Facial expression recognition: a fully integrated approach. In: International conference on image analysis and processing workshops, pp. 125–130 (2007)
43.
Zurück zum Zitat Wang, M., Hong, R., Yuan, X.T., Yan, S., Chua, T.S.: Movie2comics: towards a lively video content presentation. Trans. Multimed.14, 858–870 (2012)CrossRef Wang, M., Hong, R., Yuan, X.T., Yan, S., Chua, T.S.: Movie2comics: towards a lively video content presentation. Trans. Multimed.14, 858–870 (2012)CrossRef
44.
Zurück zum Zitat Watson, A.B.: Toward a perceptual video quality metric. In: SPIE, pp. 139–147 (1998) Watson, A.B.: Toward a perceptual video quality metric. In: SPIE, pp. 139–147 (1998)
45.
Zurück zum Zitat Zhai, Y., Shah, M.: Visual attention detection in video sequences using spatiotemporal cues. In: ACM multimedia, pp. 815–824 (2006) Zhai, Y., Shah, M.: Visual attention detection in video sequences using spatiotemporal cues. In: ACM multimedia, pp. 815–824 (2006)
46.
Zurück zum Zitat Zhang, M., Zhang, L., Sun, Y., Feng, L., Ma, W.Y.: Auto cropping for digital photographs. In: ICME, pp. 438–441 (2005) Zhang, M., Zhang, L., Sun, Y., Feng, L., Ma, W.Y.: Auto cropping for digital photographs. In: ICME, pp. 438–441 (2005)
Metadaten
Titel
Image aesthetics enhancement using composition-based saliency detection
verfasst von
Handong Zhao
Jingjing Chen
Yahong Han
Xiaochun Cao
Publikationsdatum
01.03.2015
Verlag
Springer Berlin Heidelberg
Erschienen in
Multimedia Systems / Ausgabe 2/2015
Print ISSN: 0942-4962
Elektronische ISSN: 1432-1882
DOI
https://doi.org/10.1007/s00530-014-0373-1

Weitere Artikel der Ausgabe 2/2015

Multimedia Systems 2/2015 Zur Ausgabe

Neuer Inhalt