Skip to main content

2016 | OriginalPaper | Buchkapitel

Deep Learning Features Inspired Saliency Detection of 3D Images

verfasst von : Qiudan Zhang, Xu Wang, Jianmin Jiang, Lin Ma

Erschienen in: Advances in Multimedia Information Processing - PCM 2016

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Saliency detection of 3D images is important for many 3D applications, such as bit allocation in 3D video coding, spatial pooling in stereoscopic image quality assessment and feature extraction in 3D object retrieval. However, traditional saliency detection approaches only target for the 2D images. Meanwhile, the traditional hand-crafted low-level feature extraction process may be not suitable for the 3D images. In this paper, we propose a deep learning feature based 3D visual saliency detection model. The pre-trained CNN model is employed to extract the feature vectors for both color and depth images after multi-level image segmentation. Then, we train a neutral network based classifier to generate the color and depth saliency maps from the feature vectors. Final, the linear fusion method is adopted to obtain the final saliency map for 3D image. Experimental results demonstrate that our proposed model can achieve appealing performance improvement over two public benchmark datasets.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Guo, C., Zhang, L.: A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression. IEEE Trans. Image Process. 19(1), 185–198 (2010)MathSciNetCrossRef Guo, C., Zhang, L.: A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression. IEEE Trans. Image Process. 19(1), 185–198 (2010)MathSciNetCrossRef
2.
Zurück zum Zitat Ma, L., Lin, W., Deng, C., Ngan, K.N.: Image retargeting quality assessment: a study of subjective scores and objective metrics. IEEE J. Sel. Top. Sign. Process. 6(6), 626–639 (2012)CrossRef Ma, L., Lin, W., Deng, C., Ngan, K.N.: Image retargeting quality assessment: a study of subjective scores and objective metrics. IEEE J. Sel. Top. Sign. Process. 6(6), 626–639 (2012)CrossRef
3.
Zurück zum Zitat Ma, L., Li, S., Zhang, F., Ngan, K.N.: Reduced-reference image quality assessment using reorganized DCT-based image representation. IEEE Trans. Multimedia 13(4), 824–829 (2011)CrossRef Ma, L., Li, S., Zhang, F., Ngan, K.N.: Reduced-reference image quality assessment using reorganized DCT-based image representation. IEEE Trans. Multimedia 13(4), 824–829 (2011)CrossRef
4.
Zurück zum Zitat Fang, Y.M., Lin, W.S., Chen, Z.Z., Tsai, C.M., Lin, C.W.: A video saliency detection model in compressed domain. IEEE Trans. Circuits Syst. Video Technol. 24(1), 27–38 (2014)CrossRef Fang, Y.M., Lin, W.S., Chen, Z.Z., Tsai, C.M., Lin, C.W.: A video saliency detection model in compressed domain. IEEE Trans. Circuits Syst. Video Technol. 24(1), 27–38 (2014)CrossRef
5.
Zurück zum Zitat Fang, Y.M., Chen, Z.Z., Lin, W.S., Lin, C.W.: Saliency detection in the compressed domain for adaptive image retargeting. IEEE Trans. Image Process. 21(9), 3888–3901 (2012)MathSciNetCrossRef Fang, Y.M., Chen, Z.Z., Lin, W.S., Lin, C.W.: Saliency detection in the compressed domain for adaptive image retargeting. IEEE Trans. Image Process. 21(9), 3888–3901 (2012)MathSciNetCrossRef
6.
Zurück zum Zitat Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)CrossRef Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)CrossRef
7.
Zurück zum Zitat Song, X., Zhang, J., Han, Y., Jiang, J.: Semi-supervised feature selection via hierarchical regression for web image classification. Multimedia Syst. 22(1), 41–49 (2016)CrossRef Song, X., Zhang, J., Han, Y., Jiang, J.: Semi-supervised feature selection via hierarchical regression for web image classification. Multimedia Syst. 22(1), 41–49 (2016)CrossRef
8.
Zurück zum Zitat Zhang, J., Han, Y., Jiang, J.: Tensor rank selection for multimedia analysis. J. Vis. Commun. Image Represent. 30, 376–392 (2015)CrossRef Zhang, J., Han, Y., Jiang, J.: Tensor rank selection for multimedia analysis. J. Vis. Commun. Image Represent. 30, 376–392 (2015)CrossRef
9.
Zurück zum Zitat Fang, Y., Wang, J., Narwaria, M., Callet, P.L., Lin, W.: Saliency detection for stereoscopic images. IEEE Trans. Image Process. 23(6), 2625–2636 (2014)MathSciNetCrossRef Fang, Y., Wang, J., Narwaria, M., Callet, P.L., Lin, W.: Saliency detection for stereoscopic images. IEEE Trans. Image Process. 23(6), 2625–2636 (2014)MathSciNetCrossRef
11.
Zurück zum Zitat Li, G., Yu, Y.: Visual saliency based on multiscale deep features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5455–5463 (2015) Li, G., Yu, Y.: Visual saliency based on multiscale deep features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5455–5463 (2015)
12.
Zurück zum Zitat Bruce, N., Tsotsos, J.: Saliency based on information maximization. In: Advances in Neural Information Processing Systems, pp. 155–162 (2005) Bruce, N., Tsotsos, J.: Saliency based on information maximization. In: Advances in Neural Information Processing Systems, pp. 155–162 (2005)
13.
Zurück zum Zitat Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007) Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)
14.
Zurück zum Zitat Goferman, S., Zelnik-Manor, L., Tal, A.: Context-aware saliency detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(10), 1915–1926 (2012)CrossRef Goferman, S., Zelnik-Manor, L., Tal, A.: Context-aware saliency detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(10), 1915–1926 (2012)CrossRef
15.
Zurück zum Zitat Yang, J., Yang, M.H.: Top-down visual saliency via joint CRF and dictionary learning. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2296–2303 (2012) Yang, J., Yang, M.H.: Top-down visual saliency via joint CRF and dictionary learning. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2296–2303 (2012)
16.
Zurück zum Zitat Kanan, C., Tong, M.H., Zhang, L., Cottrell, G.W.: Sun: Top-down saliency using natural statistics. Visual Cogn. 17(6–7), 979–1003 (2009)CrossRef Kanan, C., Tong, M.H., Zhang, L., Cottrell, G.W.: Sun: Top-down saliency using natural statistics. Visual Cogn. 17(6–7), 979–1003 (2009)CrossRef
17.
Zurück zum Zitat Itti, L., Koch, C.: A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Res. 40(10), 1489–1506 (2000)CrossRef Itti, L., Koch, C.: A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Res. 40(10), 1489–1506 (2000)CrossRef
18.
Zurück zum Zitat Cheng, M., Mitra, N.J., Huang, X., Torr, P.H., Hu, S.: Global contrast based salient region detection. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 569–582 (2015)CrossRef Cheng, M., Mitra, N.J., Huang, X., Torr, P.H., Hu, S.: Global contrast based salient region detection. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 569–582 (2015)CrossRef
19.
Zurück zum Zitat Zhao, R., Ouyang, W., Li, H., Wang, X.: Saliency detection by multi-context deep learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1265–1274 (2015) Zhao, R., Ouyang, W., Li, H., Wang, X.: Saliency detection by multi-context deep learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1265–1274 (2015)
20.
Zurück zum Zitat Wang, J., DaSilva, M.P., Callet, P.L., Ricordel, V.: Computational model of stereoscopic 3D visual saliency. IEEE Trans. Image Process. 22(6), 2151–2165 (2013)MathSciNetCrossRef Wang, J., DaSilva, M.P., Callet, P.L., Ricordel, V.: Computational model of stereoscopic 3D visual saliency. IEEE Trans. Image Process. 22(6), 2151–2165 (2013)MathSciNetCrossRef
21.
Zurück zum Zitat Kim, H., Lee, S., Bovik, A.C.: Saliency prediction on stereoscopic videos. IEEE Trans. Image Process. 23(4), 1476–1490 (2014)MathSciNetCrossRef Kim, H., Lee, S., Bovik, A.C.: Saliency prediction on stereoscopic videos. IEEE Trans. Image Process. 23(4), 1476–1490 (2014)MathSciNetCrossRef
22.
23.
Zurück zum Zitat Jiang, H., Wang, J., Yuan, Z., Wu, Y., Zheng, N., Li, S.: Salient object detection: a discriminative regional feature integration approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2083–2090 (2013) Jiang, H., Wang, J., Yuan, Z., Wu, Y., Zheng, N., Li, S.: Salient object detection: a discriminative regional feature integration approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2083–2090 (2013)
24.
Zurück zum Zitat Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Li, F.: ImageNet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009) Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Li, F.: ImageNet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)
25.
Zurück zum Zitat İmamoğlu, N., Lin, W., Fang, Y.: A saliency detection model using low-level features based on wavelet transform. IEEE Trans. Multimedia 15(1), 96–105 (2013)CrossRef İmamoğlu, N., Lin, W., Fang, Y.: A saliency detection model using low-level features based on wavelet transform. IEEE Trans. Multimedia 15(1), 96–105 (2013)CrossRef
26.
Zurück zum Zitat Lang, C., Nguyen, T.V., Katti, H., Yadati, K., Kankanhalli, M., Yan, S.: Depth matters: influence of depth cues on visual saliency. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part II. LNCS, vol. 7573, pp. 101–115. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33709-3_8 CrossRef Lang, C., Nguyen, T.V., Katti, H., Yadati, K., Kankanhalli, M., Yan, S.: Depth matters: influence of depth cues on visual saliency. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part II. LNCS, vol. 7573, pp. 101–115. Springer, Heidelberg (2012). doi:10.​1007/​978-3-642-33709-3_​8 CrossRef
27.
Zurück zum Zitat Ma, C.Y., Hang, H.M.: Learning-based saliency model with depth information. J. Vision 15(6), 19 (2015)CrossRef Ma, C.Y., Hang, H.M.: Learning-based saliency model with depth information. J. Vision 15(6), 19 (2015)CrossRef
28.
Zurück zum Zitat Judd, T., Durand, F., Torralba, A.: A benchmark of computational models of saliency to predict human fixations. Massachusetts Inst. Technol., MA, USA, Computer Science and Artificial Intelligence Lab (CSAIL), Technical rep. MIT-CSAIL-TR-2012–001 (2012) Judd, T., Durand, F., Torralba, A.: A benchmark of computational models of saliency to predict human fixations. Massachusetts Inst. Technol., MA, USA, Computer Science and Artificial Intelligence Lab (CSAIL), Technical rep. MIT-CSAIL-TR-2012–001 (2012)
Metadaten
Titel
Deep Learning Features Inspired Saliency Detection of 3D Images
verfasst von
Qiudan Zhang
Xu Wang
Jianmin Jiang
Lin Ma
Copyright-Jahr
2016
DOI
https://doi.org/10.1007/978-3-319-48896-7_57

Neuer Inhalt