Skip to main content

2015 | OriginalPaper | Buchkapitel

Depth Map Upsampling Using Segmentation and Edge Information

verfasst von : Shuai Zheng, Ping An, Yifan Zuo, Xuemei Zou, Jianxin Wang

Erschienen in: Image and Graphics

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

A depth upsampling method based on Markov Random Field is proposed, considering the depth and color information. First, the initial interpolated depth map is inaccurate and oversmooth, we use a rectangle window centered on every pixel to search the maximum and minimum depth value of the depth map to find out the edge pixels. Then, we use the depth information to guide the segmentation of the color image and build different data terms and smoothness terms for the edge and non-edge pixels. The result depth map is piecewise smooth and the edge is sharp. In the meanwhile, the result is good where the color information is consistent while the depth is not or where the depth information is consistent while the color is not. Experiments show that the proposed method performs better than other upsampling methods in terms of mean absolute difference (MAD).

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Remondino, F., Stoppa, D.: TOF Range-Imaging Cameras, vol. 68121. Springer, Heidelberg (2013)CrossRef Remondino, F., Stoppa, D.: TOF Range-Imaging Cameras, vol. 68121. Springer, Heidelberg (2013)CrossRef
4.
Zurück zum Zitat Kopf, J., Cohen, M.F., Lischinski, D., Uyttendaele, M.: Joint bilateral upsampling. J. ACM Trans. Graph. 26(3), 96:1–96:5 (2007) Kopf, J., Cohen, M.F., Lischinski, D., Uyttendaele, M.: Joint bilateral upsampling. J. ACM Trans. Graph. 26(3), 96:1–96:5 (2007)
5.
Zurück zum Zitat Chan, D., Buisman, H., Theobalt, C., Thrun, S.: A noise-aware filter for real-time depth upsampling. In: ECCV Workshop on Multicamera and Multimodal Sensor Fusion Algorithms and Applications, pp. 1–12 (2008) Chan, D., Buisman, H., Theobalt, C., Thrun, S.: A noise-aware filter for real-time depth upsampling. In: ECCV Workshop on Multicamera and Multimodal Sensor Fusion Algorithms and Applications, pp. 1–12 (2008)
6.
Zurück zum Zitat Xiong, Q., Yang, R., Davis, J., Nister, D.: Spatial-depths super resolution for range images. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2007, pp. 1–8 (2007) Xiong, Q., Yang, R., Davis, J., Nister, D.: Spatial-depths super resolution for range images. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2007, pp. 1–8 (2007)
7.
Zurück zum Zitat He, K., Sun, J., Tang, X.: Guided image filtering. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 1–14. Springer, Heidelberg (2010)CrossRef He, K., Sun, J., Tang, X.: Guided image filtering. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 1–14. Springer, Heidelberg (2010)CrossRef
8.
Zurück zum Zitat Lu, J., Shi, K., Min, D., Lin, L., Do, M.N.: Cross-based local multipoint filtering. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 430–437 (2012) Lu, J., Shi, K., Min, D., Lin, L., Do, M.N.: Cross-based local multipoint filtering. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 430–437 (2012)
9.
Zurück zum Zitat Ferstl, D., Reinbacher, C., Ranftl, R., Rüther, M., Bischof, H.: Image guided depth upsampling using anisotropic total generalized variation. In: 2013 IEEE International Conference on Computer Vision (ICCV), pp. 993–1000 (2013) Ferstl, D., Reinbacher, C., Ranftl, R., Rüther, M., Bischof, H.: Image guided depth upsampling using anisotropic total generalized variation. In: 2013 IEEE International Conference on Computer Vision (ICCV), pp. 993–1000 (2013)
10.
Zurück zum Zitat Min, D., Lu, J., Do, M.N.: Depth video enhancement based on weighted mode filtering. IEEE Trans. Image Process. 21(3), 1176–1190 (2012)MathSciNetCrossRef Min, D., Lu, J., Do, M.N.: Depth video enhancement based on weighted mode filtering. IEEE Trans. Image Process. 21(3), 1176–1190 (2012)MathSciNetCrossRef
11.
Zurück zum Zitat Diebel, J., Thrun, S.: An application of markov random fields to range sensing. In: Proceedings of Advances in Neural Information Processing Systems, vol. 18, pp. 291–298 (2006) Diebel, J., Thrun, S.: An application of markov random fields to range sensing. In: Proceedings of Advances in Neural Information Processing Systems, vol. 18, pp. 291–298 (2006)
12.
Zurück zum Zitat Lu, J., Min, D., Pahwa, R.S., Do, M.N.: A revisit to MRF-based depth map super-resolution and enhancement. In: 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 985–988 (2011) Lu, J., Min, D., Pahwa, R.S., Do, M.N.: A revisit to MRF-based depth map super-resolution and enhancement. In: 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 985–988 (2011)
13.
Zurück zum Zitat Park, J., Kim, H., Tai, Y.W., Brown, M.S., Kweon, I.S.: High quality depth map upsampling for 3D-TOF cameras. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 1623–1630 (2011) Park, J., Kim, H., Tai, Y.W., Brown, M.S., Kweon, I.S.: High quality depth map upsampling for 3D-TOF cameras. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 1623–1630 (2011)
14.
Zurück zum Zitat Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 26(2), 147–159 (2004)CrossRef Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 26(2), 147–159 (2004)CrossRef
15.
Zurück zum Zitat Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)CrossRef Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)CrossRef
16.
Zurück zum Zitat Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26(9), 1124–1137 (2004)CrossRef Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26(9), 1124–1137 (2004)CrossRef
17.
Zurück zum Zitat Ihler, A.T., Lii, J., Willsky, A.S.: Loopy belief propagation: convergence and effects of message errors. J. Mach. Learn. Res. 6, 905–936 (2005)MathSciNetMATH Ihler, A.T., Lii, J., Willsky, A.S.: Loopy belief propagation: convergence and effects of message errors. J. Mach. Learn. Res. 6, 905–936 (2005)MathSciNetMATH
18.
Zurück zum Zitat Yin, Z.Z., Collins, R.: Belief propagation in a 3D spatio-temporal MRF for moving object. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007) Yin, Z.Z., Collins, R.: Belief propagation in a 3D spatio-temporal MRF for moving object. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)
19.
Zurück zum Zitat Choi, O., Jung, S.-W.: A consensus-driven approach for structure and texture aware depth map upsampling. IEEE Trans. Image Process. 23(8), 3321–3335 (2014)MathSciNetCrossRef Choi, O., Jung, S.-W.: A consensus-driven approach for structure and texture aware depth map upsampling. IEEE Trans. Image Process. 23(8), 3321–3335 (2014)MathSciNetCrossRef
20.
Zurück zum Zitat Schwarz, S., Sjostrom, M., Olsson, R.: A weighted optimization approach to time-of-flight sensor fusion. IEEE Trans. Image Process. 23(1), 214–225 (2014)MathSciNetCrossRef Schwarz, S., Sjostrom, M., Olsson, R.: A weighted optimization approach to time-of-flight sensor fusion. IEEE Trans. Image Process. 23(1), 214–225 (2014)MathSciNetCrossRef
21.
Zurück zum Zitat Zhu, J., Wang, L., Gao, J.: Spatial-temporal fusion for high accuracy depth maps using dynamic MRFs. IEEE Trans. Pattern Anal. Mach. Intell. 32(5), 899–909 (2010)CrossRef Zhu, J., Wang, L., Gao, J.: Spatial-temporal fusion for high accuracy depth maps using dynamic MRFs. IEEE Trans. Pattern Anal. Mach. Intell. 32(5), 899–909 (2010)CrossRef
22.
Zurück zum Zitat Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. Int. J. Comput. Vis. 58(2), 167–181 (2004)CrossRef Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. Int. J. Comput. Vis. 58(2), 167–181 (2004)CrossRef
23.
Zurück zum Zitat Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)CrossRef Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)CrossRef
Metadaten
Titel
Depth Map Upsampling Using Segmentation and Edge Information
verfasst von
Shuai Zheng
Ping An
Yifan Zuo
Xuemei Zou
Jianxin Wang
Copyright-Jahr
2015
DOI
https://doi.org/10.1007/978-3-319-21963-9_10