Top

Machine Vision and Applications

Published in:

07-12-2017 | Original Paper

Robust spatial–temporal Bayesian view synthesis for video stitching with occlusion handling

Authors: Jianmei Su, Hong Cheng, Lu Yang, Ao Luo

Published in: Machine Vision and Applications | Issue 2/2018

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Occlusion is visible in only one frame and cannot be seen in the other frame which is a vital challenge in video stitching. Occlusion always brings ghost artifacts in the blended area. Meanwhile, the traditional image stitching approaches ignore temporal consistency and cannot avoid flicking problem. To solve these challenges, we propose a unified framework in which the stitching quality and stabilization both perform well. Specifically, we explicitly detect the potential occlusion regions to indicate blending information. Then, based on the occlusion maps, we choose a proper strip in the overlapped region as the blending area. With spatial–temporal Bayesian view synthesis, spatial ghost-like artifacts can be significantly eliminated and the output videos can be kept stable. The experimental results show the out performance of the proposed approach compared to state-of-the-art approaches.

previous article Spatio-temporal elastic cuboid trajectories for efficient fight recognition using Hough forests

next article Exercise classification and event segmentation in Hammersmith Infant Neurological Examination videos

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Available only for authorised users

Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)CrossRef

Brown, M., Lowe, D.G.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vision 74(1), 59–73 (2007)CrossRef

El-Saban, M., Izz, M., Kaheel, A., Refaat, M.: Improved optimal seam selection blending for fast video stitching of videos captured from freely moving devices. In: ICIP, IEEE (2011)

Fitzgibbon, A., Wexler, Y., Zisserman, A.: Image-based rendering using image-based priors. In: ICCV (2003)

Fitzgibbon, A., Wexler, Y., Zisserman, A.: Image-based rendering using image-based priors. Int. J. Comput. Vision 63(2), 141–151 (2005)CrossRef

Gao, J.H., Li, Y., Chin, T.J., Brown, M.S.: Seam-driven image stitching. In: Eurographics Short Papers. IEEE (2013)

Hsiao, E., Hebert, M.: Occlusion reasoning for object detectionunder arbitrary viewpoint. IEEE Trans. Pattern Anal. Mach. Intell. 36(9), 1803–1815 (2014)CrossRef

Hu, J., Zhang, D.Q., Yu, H., Chen, C.W.: High resolution free-view interpolation of planar structure. In: ICME (2014)

Huang, K.C., Chien, P.Y., Chien, C.A., Chang, H.C., Guo, J.I.: A 360-degree panoramic video system design. In: International Symposium on VLSI Design. IEEE (2014)

10.

Jang, W.S., Ho, Y.S.: Discontinuity preserving disparity estimation with occlusion handling. J. Vis. Commun. Image Represent. 25(7), 1595–1603 (2014)CrossRef

11.

Jiang, W., Gu, J.: Video stitching with spatial-temporal content-preserving warping. In: CVPR (2015)

12.

Kang, S.B., Szeliski, R., Uyttendaele, M.: Seamless stitching using multi-perspective plane sweep. Technical Report. Microsoft Research, Redmond (2004)

13.

Kauff, P., Atzpadin, N., Fehn, C., Müller, M., Schreer, O., Smolic, A., Tanger, R.: Depth map creation and image-based rendering for advanced 3DTV services providing interoperability and scalability. Sig. Process. Image Commun. 22(2), 217–234 (2007)CrossRef

14.

Leordeanu, M., Zanfir, A., Sminchisescu, C.: Locally affine sparse-to-dense matching for motion and occlusion estimation. In: ICCV. IEEE (2013)

15.

Lin, C.C., Pankanti, S.U., Ramamurthy, K.N., Aravkin, A.Y.: Adaptive as-natural-as-possible image stitching. In: CVPR. IEEE (2015)

16.

McCloskey, S.: Masking light fields to remove partial occlusion. In: ICPR. IEEE (2014)

17.

McGuire, M., Stone, H.S.: Techniques for multiresolution image registration in the presence of occlusions. IEEE Trans. Geosci. Remote Sens. 38(3), 1476–1479 (2000)CrossRef

18.

Oh, J.D., Kuo, C.C.J.: Robust stereo matching with improved graph and surface models and occlusion handling. J. Vis. Commun. Image Represent. 21(5), 404–415 (2010)CrossRef

19.

Okumura, K., Raut, S., Gu, Q., Aoyama, T., Takaki, T., Ishii, I.: Real-time feature-based video mosaicing at 500 fps. In: International Conference on Intelligent Robots and Systems. IEEE (2013)

20.

Owens, A., Barnes, C., Flint, A., Singh, H., Freeman, W.: Camouflaging an object from many viewpoints. In: CVPR. IEEE (2014)

21.

Pujades, S., Devernay, F., Goldluecke, B.: Bayesian view synthesis and image-based rendering principles. In: CVPR (2014)

22.

Srinivasan, A., Balamurugan, V.: Occlusion detection and image restoration in 3d face image. In: TENCON. IEEE (2014)

23.

Su, J., Luo, A., Yang, L., Cheng, H.: Video stitching with spatial-temporal content-preserving warping. In: ICME Workshop (2016)

24.

Szeliski, R.: Computer Vision: Algorithms and Applications. Springer, Berlin (2010)MATH

25.

Thomas, B., Jitendra, M.: Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Trans. Pattern Anal. Mach. Intell. 33(3), 500–513 (2011)CrossRef

26.

Uyttendaele, M., Eden, A., Skeliski, R.: Eliminating ghosting and exposure artifacts in image mosaics. In: CVPR. IEEE (2001)

27.

Vladimir, K., Ramin, Z.: Computing visual correspondence with occlusions using graph cuts. In: ICCV. IEEE (2001)

28.

Xiao, X., Daneshpanah, M., Javidi, B.: Occlusion removal using depth mapping in three-dimensional integral imaging. J. Disp. Technol. 8(8), 483–490 (2012)CrossRef

29.

Xiong, H., Wang, Z., He, R., Feng, D.D.: Video object segmentation with occlusion map. In: DICTA. IEEE (2012)

30.

Yang, L., Cheng, H., Su, J.A.: Pixel-to-model background modeling in crowded scenes. In: ICME (2014)

31.

Yang, L., Yendo, T., Tehrani, M.P.: Artifact reduction using reliability reasoning for image generation of FTV. J. Vis. Commun. Image Represent. 21, 542–560 (2010)CrossRef

32.

Yang, L., Yendo, T., Tehranil, M.P., Fujii, T.: Probabilistic reliability based view synthesis for FTV. In: ICIP (2010)

33.

Yoo, J.C., Ahn, C.W.: Image matching using peak signal-to-noise ratio-based occlusion detection. Image Process. 6(5), 483–495 (2012)MathSciNetCrossRef

34.

Yuan, Z., Kebin, J., Peng-Yu, L.: Video stitch algorithm based on dynamic foreground extraction. In: CISP. IEEE (2009)

35.

Zagrouba, E., Barhoumi, W., Amri, S.: An efficient image-mosaicing method based on multifeature matching. Mach. Vis. Appl. 20(3), 139–162 (2009)CrossRef

36.

Zaragoza, J., Chin, T.J., Brown, M.S., Suter, D.: Asprojective-as-possible image stitching with moving DLT. In: CVPR. IEEE (2013)

37.

Zeng, L., Zhang, S., Zhang, J., Zhang, Y.: Dynamic image mosaic via sift and dynamic programming. Mach. Vis. Appl. 25(5), 1271–1282 (2014)CrossRef

38.

Zhang, F., Liu, F.: Parallax-tolerant image stitching. In: CVPR. IEEE (2014)

39.

Zhang, F., Liu, F.: Casual stereoscopic panorama stitching. In: CVPR. IEEE (2015)

40.

Zhang, H.M., Zeng, W., Chen, X.: Foreground based borderline adjusting for real time multi-camera video stitching. In: ICIG. IEEE (2009)

41.

Zhang, T., Jia, K., Xu, C., Ma, Y., Ahuja, N.: Partial occlusion handling for visual tracking via robust part matching. In: CVPR. IEEE (2014)

42.

Zhao, Y.J., Lu, Z.Q., Liu, Y.K.: Video image stitching based on moving object detection and motion prediction compensation. In: CISP. IEEE (2010)

Title: Robust spatial–temporal Bayesian view synthesis for video stitching with occlusion handling
Authors: Jianmei Su
Hong Cheng
Lu Yang
Ao Luo
Publication date: 07-12-2017
Publisher: Springer Berlin Heidelberg
Published in: Machine Vision and Applications / Issue 2/2018
Print ISSN: 0932-8092
Electronic ISSN: 1432-1769
DOI: https://doi.org/10.1007/s00138-017-0888-5

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Other articles of this Issue 2/2018

Almost constant-time 3D nearest-neighbor lookup using implicit octrees

Facial expression analysis and expression-invariant face recognition by manifold-based synthesis

Geometric primitive refinement for structured light cameras

Spatio-temporal elastic cuboid trajectories for efficient fight recognition using Hough forests

Tracking using Numerous Anchor Points

Beyond Eleven Color Names for Image Understanding

Premium Partner