Skip to main content
SpringerLink
Log in

Real-time feature-aware video abstraction

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

This paper presents a novel feature-aware rendering system that automatically abstracts videos and images with the goal of improving the effectiveness of imagery for visual communication tasks. We integrate the bilateral grid to simplify regions of low contrast, which is faster than the separable approximation to the bilateral filter, and use a feature flow-guided anisotropic edge detection filter to enhance regions of high contrast. The edges detected in this paper are smoother, more coherent and stylistic than those of the isotropic difference-of-Gaussian filter. The presented algorithms are highly parallel, allowing a real-time performance on modern GPUs. The implementation of our approach is straightforward. Several experimental examples are given at the end of the paper to demonstrate the effectiveness of our approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aurich, V., Weule, J.: Non-linear gaussian filters performing edge preserving diffusion. In: Proceedings of the DAGM Symposium, pp. 538–545. Springer, London (1995)

    Google Scholar 

  2. Barash, D.: A fundamental relationship between bilateral filtering, adaptive smoothing and the nonlinear diffusion equation. IEEE Trans. Pattern Anal. Mach. Intell. 24(6), 844–847 (2002)

    Article  Google Scholar 

  3. Blythe, D.: The Direct3D 10 system. In: Proceedings of the ACM SIGGRAPH ’06, pp. 724–734. ACM, New York (2006)

    Chapter  Google Scholar 

  4. Bousseau, A., Neyret, F., Thollot, J., Salesin, D.: Video watercolorization using bidirectional texture advection. In: Proceedings of the ACM SIGGRAPH ’07, article 104. ACM, New York (2007)

  5. Cabral, B., Leedom, L.C.: Imaging vector fields using line integral convolution. In: Proceedings of the ACM SIGGRAPH ’93, pp. 263–270. ACM, New York (1993)

    Google Scholar 

  6. Canny, J.F.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 679–698 (1986)

    Google Scholar 

  7. Chen, J., Parism, S., Durand, F.: Real-time edge-aware image processing with the bilateral grid. In: Proceedings of the ACM SIGGRAPH ’07, article 103. ACM, New York (2007)

  8. Collomosse, J.P., Rowntree, D., Hall, P.M.: Stroke surfaces: temporally coherent artistic animations from video. IEEE Trans. Vis. Comput. Graph. 11(5), 540–549 (2005)

    Article  Google Scholar 

  9. Decarlo, D., Santella, A.: Stylization and abstraction of photographs. In: Proceedings of the ACM SIGGRAPH ’02, pp. 769–776. ACM, New York (2002)

    Google Scholar 

  10. Durand, F., Dorsey, J.: Fast bilateral filtering for the display of high-dynamic-range images. In: Proceedings of the ACM SIGGRAPH ’02, pp. 257–266. ACM, New York (2002)

    Google Scholar 

  11. Fischer, J., Bartz, D.: Stylized augmented reality for improved immersion. In: Proceedings of the IEEE Conference on Virtual Reality, pp. 195–202. IEEE Computer Society, Washington (2005)

    Chapter  Google Scholar 

  12. Gooch, B., Reinhard, E., Gooch, A.: Human facial illustrations: creation and psychophysical evaluation. ACM Trans. Graph. 23(1), 27–44 (2004)

    Article  Google Scholar 

  13. Kang, H., Lee, S., Chui, C.K.: Coherent line drawing. In: Proceedings of the ACM International Symposium on Non-Photorealistic Animation and Rendering (NPAR07), pp. 43–50. ACM, New York (2007)

    Google Scholar 

  14. Marr, D., Hildreth, E.C.: Theory of edge detection. Proc. R. Soc. Lond. Ser. B, Biol. Sci. 207(1167), 187–217 (1980)

    Article  Google Scholar 

  15. Microsoft Corporation: DirectX 10 HLSL Compiler. DirectX Software Development Kit, Apr (2007)

  16. Orzan, A., Bousseau, A., Barla, P., Thollot, J.: Structure-preserving manipulation of photographs. In: Proceedings of the ACM International Symposium on Non-Photorealistic Animation and Rendering, pp. 103–110. ACM, New York (2007)

    Google Scholar 

  17. Paris, S., Durand, F.: A fast approximation of the bilateral filter using a signal processing approach. In: Proceedings of the European Conference on Computer Vision, pp. 568–580. Springer, Berlin (2006)

    Google Scholar 

  18. Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12(7), 629–639 (1991)

    Article  Google Scholar 

  19. Pham, T.Q., van Vliet, L.J.: Separable bilateral filtering for fast video preprocessing. In: Proceedings of the IEEE Internationl Conference on Multimedia and Expo, pp. 454–457. IEEE Computer Society, Washington (2005)

    Chapter  Google Scholar 

  20. Smith, S.M., Brady, J.M.: SUSAN – a new approach to low level image processing. Int. J. Comput. Vis. 23(1), 45–78 (1997)

    Article  Google Scholar 

  21. Son, M., Kang, H., Lee, Y., Lee, S.: Abstract line drawings from 2D images. In: Proceedings of the IEEE Pacific Conference on Computer Graphics and Applications (PG’07), pp. 333–342. IEEE Computer Society, Washington (2007)

    Chapter  Google Scholar 

  22. Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Proceedings of the International Conference on Computer Vision (ICCV98), pp. 839–846. IEEE Computer Society, Washington (1998)

    Google Scholar 

  23. Wang, J., Xu, Y. Shum, H.-Y., Cohen M.F.: Video tooning. In: Proceedings of the ACM SIGGRAPH ’04, pp. 574–583. ACM, New York (2004)

    Chapter  Google Scholar 

  24. Weiss, B.: Fast median and bilateral filtering. In: Proceedings of the ACM SIGGRAPH ’06, pp. 519–526. ACM, New York (2006)

    Chapter  Google Scholar 

  25. Winnemöller, H., Olsen, S.C., Gooch, B.: Real-time video abstraction. In: Proceedings of the ACM SIGGRAPH ’06, pp. 1221–1226. ACM, New York (2006)

    Chapter  Google Scholar 

  26. Wyszecki, G., Stiles, W.S.: Color Science: Concepts and Methods, Quantitative Data and Formulae. Wiley, New York, NY (1982)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. State Key Lab of CAD & CG, Zhejiang University, Hangzhou, 310027, P.R. China

    Hanli Zhao, Xiaogang Jin & Jieqing Feng

  2. Indiana University-Purdue University, Indianapolis, USA

    Jianbing Shen

  3. University of Yamanashi, Yamanashi, Japan

    Xiaoyang Mao

Authors
  1. Hanli Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaogang Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianbing Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaoyang Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jieqing Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaogang Jin.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhao, H., Jin, X., Shen, J. et al. Real-time feature-aware video abstraction. Visual Comput 24, 727–734 (2008). https://doi.org/10.1007/s00371-008-0254-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-008-0254-8

Keywords

Search

Navigation