Skip to main content

2019 | OriginalPaper | Buchkapitel

Local Saliency Estimation and Global Homogeneity Refinement for Video Saliency Detection

verfasst von : Rahma Kalboussi, Mehrez Abdellaoui, Ali Douik

Erschienen in: Intelligent Interactive Multimedia Systems and Services

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Saliency detection aims to segment the object of interest from the rest of the scene. While there has been a big number of saliency detection methods in still images, video saliency is in its early stages. In this paper, we propose a two stages video saliency detection method using local saliency estimation and global homogeneity refinement. Starting from a patch, the problem of saliency detection is modeled as a growing region which starts from a patch with high saliency information to the background. Local saliency is measured by combining spatial priors presented by local surrounding contrast with temporal information issued from the motion estimation feature. Temporal and spatial information are fused and then used to label each patch as foreground and background patches and produce the final saliency maps. Finally, Global homogeneity refinement is used to refine the saliency results by evaluating the foreground and background probabilities ratio propagated from the patches. Experiments have proved that the proposed method outperforms state-of-the-art methods over two benchmark datasets.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Avidan, S., Shamir, A.: Seam carving for content-aware image resizing. ACM Trans. Graph. (TOG) 26, 10 (2007)CrossRef Avidan, S., Shamir, A.: Seam carving for content-aware image resizing. ACM Trans. Graph. (TOG) 26, 10 (2007)CrossRef
2.
Zurück zum Zitat Bi, S., Li, G., Yu, Y.: Person re-identification using multiple experts with random subspaces. J. Image Graph. 2 (2014) Bi, S., Li, G., Yu, Y.: Person re-identification using multiple experts with random subspaces. J. Image Graph. 2 (2014)
3.
Zurück zum Zitat Borji, A., Cheng, M.-M., Jiang, H., Li, J.: Salient object detection: a benchmark. IEEE Trans. Image Process. 24, 5706–5722 (2015)MathSciNetCrossRef Borji, A., Cheng, M.-M., Jiang, H., Li, J.: Salient object detection: a benchmark. IEEE Trans. Image Process. 24, 5706–5722 (2015)MathSciNetCrossRef
4.
Zurück zum Zitat Fang, Y., Lin, W., Chen, Z., Tsai, C.-M., Lin, C.-W.: A video saliency detection model in compressed domain. IEEE Trans. Circ. Syst. Video Technol. 24, 27–38 (2014)CrossRef Fang, Y., Lin, W., Chen, Z., Tsai, C.-M., Lin, C.-W.: A video saliency detection model in compressed domain. IEEE Trans. Circ. Syst. Video Technol. 24, 27–38 (2014)CrossRef
5.
Zurück zum Zitat Fu, H., Cao, X., Tu, Z.: Cluster-based co-saliency detection. IEEE Trans. Image Process. 22, 3766–3778 (2013)MathSciNetCrossRef Fu, H., Cao, X., Tu, Z.: Cluster-based co-saliency detection. IEEE Trans. Image Process. 22, 3766–3778 (2013)MathSciNetCrossRef
6.
Zurück zum Zitat Fukuchi, K., Miyazato, K., Kimura, A., Takagi, S., Yamato, J.: Saliency-based video segmentation with graph cuts and sequentially updated priors. In: 2009 IEEE International Conference on Multimedia and Expo, pp. 638–641. IEEE (2009) Fukuchi, K., Miyazato, K., Kimura, A., Takagi, S., Yamato, J.: Saliency-based video segmentation with graph cuts and sequentially updated priors. In: 2009 IEEE International Conference on Multimedia and Expo, pp. 638–641. IEEE (2009)
7.
Zurück zum Zitat Gao, D., Mahadevan, V., Vasconcelos, N.: The discriminant center-surround hypothesis for bottom-up saliency. In: Advances in Neural Information Processing Systems, pp. 497–504 (2008) Gao, D., Mahadevan, V., Vasconcelos, N.: The discriminant center-surround hypothesis for bottom-up saliency. In: Advances in Neural Information Processing Systems, pp. 497–504 (2008)
8.
Zurück zum Zitat Gao, D., Vasconcelos, N.: Bottom-up saliency is a discriminant process. In: IEEE 11th International Conference on Computer Vision, ICCV 2007, pp. 1–6. IEEE (2007) Gao, D., Vasconcelos, N.: Bottom-up saliency is a discriminant process. In: IEEE 11th International Conference on Computer Vision, ICCV 2007, pp. 1–6. IEEE (2007)
9.
Zurück zum Zitat Goferman, S., Zelnik-Manor, L., Tal, A.: Context-aware saliency detection. IEEE Trans. Pattern Anal. Mach. Intell. 34, 1915–1926 (2012)CrossRef Goferman, S., Zelnik-Manor, L., Tal, A.: Context-aware saliency detection. IEEE Trans. Pattern Anal. Mach. Intell. 34, 1915–1926 (2012)CrossRef
10.
Zurück zum Zitat Harel, J., Koch, C., Perona, P., et al.: Graph-based visual saliency. In: NIPS, vol. 1, p. 5 (2006) Harel, J., Koch, C., Perona, P., et al.: Graph-based visual saliency. In: NIPS, vol. 1, p. 5 (2006)
11.
Zurück zum Zitat Itti, L., Baldi, P.: A principled approach to detecting surprising events in video. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 631–637. IEEE (2005 ) Itti, L., Baldi, P.: A principled approach to detecting surprising events in video. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 631–637. IEEE (2005 )
12.
Zurück zum Zitat Kim, H., Kim, Y., Sim, J.-Y., Kim, C.-S.: Spatiotemporal saliency detection for video sequences based on random walk with restart. IEEE Transa. Image Process. 24, 2552–2564 (2015)MathSciNetCrossRef Kim, H., Kim, Y., Sim, J.-Y., Kim, C.-S.: Spatiotemporal saliency detection for video sequences based on random walk with restart. IEEE Transa. Image Process. 24, 2552–2564 (2015)MathSciNetCrossRef
13.
Zurück zum Zitat Lee, S.-H., Kim, J.-H., Choi, K.P., Sim, J.-Y., Kim, C.-S.: Video saliency detection based on spatiotemporal feature learning. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 1120–1124. IEEE (2014) Lee, S.-H., Kim, J.-H., Choi, K.P., Sim, J.-Y., Kim, C.-S.: Video saliency detection based on spatiotemporal feature learning. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 1120–1124. IEEE (2014)
14.
Zurück zum Zitat Li, F., Kim, T., Humayun, A., Tsai, D., Rehg, J.M.: Video segmentation by tracking many figure-ground segments. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2192–2199 (2013) Li, F., Kim, T., Humayun, A., Tsai, D., Rehg, J.M.: Video segmentation by tracking many figure-ground segments. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2192–2199 (2013)
15.
Zurück zum Zitat Lucas, B.D., Kanade, T., et al.: An iterative image registration technique with an application to stereo vision. In: IJCAI, vol. 81, pp. 674–679 (1981) Lucas, B.D., Kanade, T., et al.: An iterative image registration technique with an application to stereo vision. In: IJCAI, vol. 81, pp. 674–679 (1981)
16.
Zurück zum Zitat Mahadevan, V., Vasconcelos, N.: Spatiotemporal saliency in dynamic scenes. IEEE Trans. Pattern Anal. Mach. Intell. 32, 171–177 (2010)CrossRef Mahadevan, V., Vasconcelos, N.: Spatiotemporal saliency in dynamic scenes. IEEE Trans. Pattern Anal. Mach. Intell. 32, 171–177 (2010)CrossRef
17.
Zurück zum Zitat Mancas, M., Riche, N., Leroy, J., Gosselin, B.: Abnormal motion selection in crowds using bottom-up saliency. In: 2011 18th IEEE International Conference on Image Processing (ICIP), pp. 229–232. IEEE (2011) Mancas, M., Riche, N., Leroy, J., Gosselin, B.: Abnormal motion selection in crowds using bottom-up saliency. In: 2011 18th IEEE International Conference on Image Processing (ICIP), pp. 229–232. IEEE (2011)
18.
Zurück zum Zitat Mauthner, T., Possegger, H., Waltner, G., Bischof, H.: Encoding based saliency detection for videos and images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2494–2502 (2015) Mauthner, T., Possegger, H., Waltner, G., Bischof, H.: Encoding based saliency detection for videos and images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2494–2502 (2015)
19.
Zurück zum Zitat Navalpakkam, V., Itti, L.: An integrated model of top-down and bottom-up attention for optimizing detection speed. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), vol. 2, pp. 2049–2056. IEEE (2006) Navalpakkam, V., Itti, L.: An integrated model of top-down and bottom-up attention for optimizing detection speed. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), vol. 2, pp. 2049–2056. IEEE (2006)
20.
Zurück zum Zitat Rahtu, E., Kannala, J., Salo, M., Heikkilä, J.: Segmenting salient objects from images and videos. In: Computer Vision-ECCV 2010, pp. 366–379 (2010)CrossRef Rahtu, E., Kannala, J., Salo, M., Heikkilä, J.: Segmenting salient objects from images and videos. In: Computer Vision-ECCV 2010, pp. 366–379 (2010)CrossRef
21.
Zurück zum Zitat Seo, H.J., Milanfar, P.: Static and space-time visual saliency detection by self-resemblance. J. Vis. 9, 15–15 (2009)CrossRef Seo, H.J., Milanfar, P.: Static and space-time visual saliency detection by self-resemblance. J. Vis. 9, 15–15 (2009)CrossRef
22.
Zurück zum Zitat Singh, A., Chu, C.-H.H., Pratt, M.: Learning to predict video saliency using temporal superpixels. In: 4th International Conference on Pattern Recognition Applications and Methods, pp. 201–209 (2015) Singh, A., Chu, C.-H.H., Pratt, M.: Learning to predict video saliency using temporal superpixels. In: 4th International Conference on Pattern Recognition Applications and Methods, pp. 201–209 (2015)
23.
Zurück zum Zitat Treisman, A.M., Gelade, G.: A feature-integration theory of attention. Cogn. psychol. 12, 97–136 (1980)CrossRef Treisman, A.M., Gelade, G.: A feature-integration theory of attention. Cogn. psychol. 12, 97–136 (1980)CrossRef
24.
Zurück zum Zitat Wang, W., Shen, J., Porikli, F.: Saliency-aware geodesic video object segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3395–3402 (2015) Wang, W., Shen, J., Porikli, F.: Saliency-aware geodesic video object segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3395–3402 (2015)
25.
Zurück zum Zitat Wu, R., Yu, Y., Wang, W.: Scale: supervised and cascaded Laplacian eigenmaps for visual object recognition based on nearest neighbors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 867–874 (2013) Wu, R., Yu, Y., Wang, W.: Scale: supervised and cascaded Laplacian eigenmaps for visual object recognition based on nearest neighbors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 867–874 (2013)
26.
Zurück zum Zitat Yeh, H.-H., Liu, K.-H., Chen, C.-S.: Salient object detection via local saliency estimation and global homogeneity refinement. Pattern Recogn. 47, 1740–1750 (2014)CrossRef Yeh, H.-H., Liu, K.-H., Chen, C.-S.: Salient object detection via local saliency estimation and global homogeneity refinement. Pattern Recogn. 47, 1740–1750 (2014)CrossRef
27.
Zurück zum Zitat Zhong, S., Liu, Y., Ren, F., Zhang, J., Ren, T.: Video saliency detection via dynamic consistent spatio-temporal attention modelling. In: AAAI, pp. 1063–1069 (2013) Zhong, S., Liu, Y., Ren, F., Zhang, J., Ren, T.: Video saliency detection via dynamic consistent spatio-temporal attention modelling. In: AAAI, pp. 1063–1069 (2013)
Metadaten
Titel
Local Saliency Estimation and Global Homogeneity Refinement for Video Saliency Detection
verfasst von
Rahma Kalboussi
Mehrez Abdellaoui
Ali Douik
Copyright-Jahr
2019
DOI
https://doi.org/10.1007/978-3-319-92231-7_17