Skip to main content

Advertisement

SpringerLink
Log in

Perception-based seam cutting for image stitching

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Image stitching is still challenging in consumer-level photography due to imperfect image captures. Recent works show that seam-cutting approaches can effectively relieve the artifacts generated by local misalignment. Normally, the seam-cutting approach is described in terms of energy minimization. However, few of existing methods consider the human perception in their energy functions, which sometimes causes that there exists another seam that is perceptually better than the one with the minimum energy. In this paper, we propose a novel perception-based seam-cutting approach that considers the nonlinearity and the nonuniformity of human perception into the energy minimization. Our method uses a sigmoid metric to characterize the perception of color discrimination and a saliency weight to simulate that the human eye inclines to pay more attention to the salient objects. In addition, our approach can be easily integrated into other stitching pipelines. Representative experiments demonstrate substantial improvements over the conventional seam-cutting approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. http://research.microsoft.com/en-us/um/redmond/groups/ivm/ice.

  2. http://web.cecs.pdx.edu/~fliu/project/stitch/.

  3. http://www.linkaimo.com/publications/ImageStitching/ImageStitching.html.

References

  1. Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., Cohen, M.: Interactive digital photomontage. ACM Trans. Graph. 23(3), 294–302 (2004)

    Article  Google Scholar 

  2. Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26(9), 1124–1137 (2004)

    Article  MATH  Google Scholar 

  3. Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)

    Article  Google Scholar 

  4. Brown, M., Lowe, D.G.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vis. 74(1), 59–73 (2007)

    Article  Google Scholar 

  5. Brown, M., Lowe, D.G., et al.: Recognising panoramas. In: ICCV, vol. 3, p. 1218 (2003)

  6. Burt, P.J., Adelson, E.H.: A multiresolution spline with application to image mosaics. ACM Trans. Graph. 2(4), 217–236 (1983)

    Article  Google Scholar 

  7. Chang, C.H., Sato, Y., Chuang, Y.Y.: Shape-preserving half-projective warps for image stitching. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 3254–3261 (2014)

  8. Chen, Y.S., Chuang, Y.Y.: Natural image stitching with the global similarity prior. In: Proceedings of 14th European Conference on Computer Vision, pp. 186–201 (2016)

  9. Davis, J.: Mosaics of scenes with moving objects. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 354–360 (1998)

  10. Duplaquet, M.L.: Building large image mosaics with invisible seam lines. In: Aerospace/Defense Sensing and Controls, pp. 369–377. International Society for Optics and Photonics Proc. SPIE 3387, Visual Information Processing VII (1998)

  11. Eden, A., Uyttendaele, M., Szeliski, R.: Seamless image stitching of scenes with large motions and exposure differences. In: Proceedings of IEEE Conference on Computer Vision Pattern Recognition, vol. 2, pp. 2498–2505 (2006)

  12. Efros, A.A., Freeman, W.T.: Image quilting for texture synthesis and transfer. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH’01, pp. 341–346. ACM (2001)

  13. Fang, X., Zhu, J., Luo, B.: Image mosaic with relaxed motion. SIViP 6(4), 647–667 (2012)

    Article  Google Scholar 

  14. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  15. Gao, J., Kim, S.J., Brown, M.S.: Constructing image panoramas using dual-homography warping. In: Proceedings of IEEE Conference on Computer Vision Pattern Recognition, pp. 49–56 (2011)

  16. Gao, J., Li, Y., Chin, T.J., Brown, M.S.: Seam-driven image stitching. In: Eurographics, pp. 45–48 (2013)

  17. Gracias, N., Mahoor, M., Negahdaripour, S., Gleason, A.: Fast image blending using watersheds and graph cuts. Image Vis. Comput. 27(5), 597–607 (2009)

    Article  Google Scholar 

  18. Jia, J., Tang, C.K.: Image stitching using structure deformation. IEEE Trans. Pattern Anal. Mach. Intell. 30(4), 617–631 (2008)

    Article  Google Scholar 

  19. Kolmogorov, V., Zabin, R.: What energy functions can be minimized via graph cuts? IEEE Trans. Pattern Anal. Mach. Intell. 26(2), 147–159 (2004)

    Article  Google Scholar 

  20. Kwatra, V., Schödl, A., Essa, I., Turk, G., Bobick, A.: Graphcut textures: image and video synthesis using graph cuts. ACM Trans. Graph. 22(3), 277–286 (2003)

    Article  Google Scholar 

  21. Levin, A., Zomet, A., Peleg, S., Weiss, Y.: Seamless image stitching in the gradient domain. In: Proceedings of the 8th European Conference on Computer Vision, pp. 377–389 (2004)

  22. Lin, C.C., Pankanti, S.U., Ramamurthy, K.N., Aravkin, A.Y.: Adaptive as-natural-as-possible image stitching. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1155–1163 (2015)

  23. Lin, K., Jiang, N., Cheong, L.F., Do, M., Lu, J.: Seagull: seam-guided local alignment for parallax-tolerant image stitching. In: Proceedings of 14th European Conference on Computer Vision, pp. 370–385 (2016)

  24. Liu, F., Gleicher, M., Jin, H., Agarwala, A.: Content-preserving warps for 3d video stabilization. ACM Trans. Graph. (TOG) 28(3), 44 (2009)

    Google Scholar 

  25. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  26. Mills, A., Dudek, G.: Image stitching with dynamic elements. Image Vis. Comput. 27(10), 1593–1602 (2009)

    Article  Google Scholar 

  27. Otsu, N.: A threshold selection method from gray-level histograms. Automatica 11(285–296), 23–27 (1975)

    Google Scholar 

  28. Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. ACM Trans. Graph. 22(3), 313–318 (2003)

    Article  Google Scholar 

  29. Rzhanov, Y.: Photo-mosaicing of images of pipe inner surface. Signal Image Video Process 7(5), 865–871 (2013)

    Article  Google Scholar 

  30. Szeliski, R.: Image alignment and stitching: A tutorial. Technical Report MSR-TR-2004-92, Microsoft Research (2004)

  31. Szeliski, R., Shum, H.Y.: Creating full view panoramic image mosaics and environment maps. In: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH’97, pp. 251–258. ACM Press (1997)

  32. Xu, Z.: Consistent image alignment for video mosaicing. SIViP 7(1), 129–135 (2013)

    Article  Google Scholar 

  33. Yang, L., Tan, Z., Huang, Z., Cheung, G.: A content-aware metric for stitched panoramic image quality assessment. In: The IEEE International Conference on Computer Vision (ICCV) (2017)

  34. Zaragoza, J., Chin, T.J., Tran, Q.H., Brown, M.S., Suter, D.: As-projective-as-possible image stitching with moving dlt. IEEE Trans. Pattern Anal. Mach. Intell. 7(36), 1285–1298 (2014)

    Google Scholar 

  35. Zeng, L., Zhang, W., Zhang, S., Wang, D.: Video image mosaic implement based on planar-mirror-based catadioptric system. SIViP 8(6), 1007–1014 (2014)

    Article  Google Scholar 

  36. Zhang, F., Liu, F.: Parallax-tolerant image stitching. In: Proceedings of IEEE Conference on Computer Vision Pattern Recognition, pp. 3262–3269 (2014)

  37. Zhang, G., He, Y., Chen, W., Jia, J., Bao, H.: Multi-viewpoint panorama construction with wide-baseline images. IEEE Trans. Image Process. 25(7), 3099–3111 (2016)

    Article  MathSciNet  Google Scholar 

  38. Zhang, J., Sclaroff, S., Lin, Z., Shen, X., Price, B., Mech, R.: Minimum barrier salient object detection at 80 fps. In: Proceedings of IEEE International Conference on Computer Vision, pp. 1404–1412 (2015)

Download references

Acknowledgements

This work is supported by Natural Science Foundation of China (No. 11626250) and No. 11601378.

Author information

Authors and Affiliations

  1. Center for Applied Mathematics, Tianjin University, Tianjin, 300072, China

    Nan Li

  2. Center for Combinatorics, Nankai University, Tianjin, 300071, China

    Tianli Liao

  3. Department of Software, Nankai University, Tianjin, 300071, China

    Chao Wang

Authors
  1. Nan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tianli Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tianli Liao.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 36670 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, N., Liao, T. & Wang, C. Perception-based seam cutting for image stitching. SIViP 12, 967–974 (2018). https://doi.org/10.1007/s11760-018-1241-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-018-1241-9

Keywords

Search

Navigation