ABSTRACT
We present an automatic, real-time video and image abstraction framework that abstracts imagery by modifying the contrast of visually important features, namely luminance and color opponency. We reduce contrast in low-contrast regions using an approximation to anisotropic diffusion, and artificially increase contrast in higher contrast regions with difference-of-Gaussian edges. The abstraction step is extensible and allows for artistic or data-driven control. Abstracted images can optionally be stylized using soft color quantization to create cartoon-like effects with good temporal coherence. Our framework design is highly parallel, allowing for a GPU-based, real-time implementation. We evaluate the effectiveness of our abstraction framework with a user-study and find that participants are faster at naming abstracted faces of known persons compared to photographs. Participants are also better at remembering abstracted images of arbitrary scenes in a memory task.
Supplemental Material
- Arad, N., and Gotsman, C. 1999. Enhancement by image-dependent warping. IEEE Trans. on Image Processing 8, 9, 1063--1074. Google ScholarDigital Library
- Barash, D., and Comaniciu, D. 2004. A common framework for non-linear diffusion, adaptive smoothing, bilateral filtering and mean shift. Image and Video Computing 22, 1, 73--81.Google ScholarCross Ref
- Boomgaard, R. V. D., and de Weijer, J. V. 2002. On the equivalence of local-mode finding, robust estimation and mean-shift analysis as used in early vision tasks. 16th Internat. Conf. on Pattern Recog. 3, 927--390. Google ScholarDigital Library
- Canny, J. F. 1986. A computational approach to edge detection. IEEE Trans. on Pattern Analysis and Machine Intelligence 8, 769--798. Google ScholarDigital Library
- Collomosse, J. P., Rowntree, D., and Hall, P. M. 2005. Stroke surfaces: Temporally coherent artistic animations from video. IEEE Trans. on Visualization and Computer Graphics 11, 5, 540--549. Google ScholarDigital Library
- DeCarlo, D., and Santella, A. 2002. Stylization and abstraction of photographs. ACM Trans. Graph. 21, 3, 769--776. Google ScholarDigital Library
- Elder, J. H. 1999. Are edges incomplete? Internat. Journal of Computer Vision 34, 2-3, 97--122. Google ScholarDigital Library
- Fischer, J., Bartz, D., and Strasser, W. 2005. Stylized Augmented Reality for Improved Immersion. In Proc. of IEEE VR, 195--202. Google ScholarDigital Library
- Gooch, B., Reinhard, E., and Gooch, A. 2004. Human facial illustrations: Creation and psychophysical evaluation. ACM Trans. Graph. 23, 1, 27--44. Google ScholarDigital Library
- Hertzmann, A. 2001. Paint by relaxation. In CGI '01:Computer Graphics Internat. 2001, 47--54. Google ScholarDigital Library
- Itti, L., and Koch, C. 2001. Computational modeling of visual attention. Nature Reviews Neuroscience 2, 3, 194--203.Google ScholarCross Ref
- Loviscach, J. 1999. Scharfzeichner: Klare bilddetails durch verformung. Computer Technik 22, 236ff.Google Scholar
- Marr, D., and Hildreth, E. C. 1980. Theory of edge detection. Proc. Royal Soc. London, Bio. Sci. 207, 187--217.Google ScholarCross Ref
- Palmer, S. E. 1999. Vision Science: Photons to Phenomenology. The MIT Press.Google Scholar
- Perona, P., and Malik, J. 1991. Scale-space and edge detection using anisotropic diffusion. IEEE Trans. on Pattern Analysis and Machine Intelligence 12, 7. Google ScholarDigital Library
- Pham, T. Q., and Vliet, L. J. V. 2005. Separable bilateral filtering for fast video preprocessing. In IEEE Internat. Conf. on Multimedia & Expo, CD1-4.Google Scholar
- Privitera, C. M., and Stark, L. W. 2000. Algorithms for defining visual regions-of-interest: Comparison with eye fixations. IEEE Trans. on Pattern Analysis and Machine Intelligence 22, 9, 970--982. Google ScholarDigital Library
- Raskar, R., Tan, K.-H., Feris, R., Yu, J., and Turk, M. 2004. Non-photorealistic camera: depth edge detection and stylized rendering using multi-flash imaging. ACM Trans. Graph. 23, 3, 679--688. Google ScholarDigital Library
- Saito, T., and Takahashi, T. 1990. Comprehensible rendering of 3-D shapes. In Proc. of ACM SIGGRAPH 90, 197--206. Google ScholarDigital Library
- Santella, A., and DeCarlo, D. 2004. Visual interest and NPR: an evaluation and manifesto. In Proc. of NPAR '04, 71--78. Google ScholarDigital Library
- Stevenage, S. V. 1995. Can caricatures really produce distinctiveness effects? British Journal of Psychology 86, 127--146.Google ScholarCross Ref
- Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In Proceedings of ICCV '98, 839. Google ScholarDigital Library
- Wang, J., Xu, Y., Shum, H.-Y., and Cohen, M. F. 2004. Video tooning. ACM Trans. Graph. 23, 3, 574--583. Google ScholarDigital Library
- Winkenbach, G., and Salesin, D. H. 1994. Computer-generated pen-and-ink illustration. In Proc. of ACM SIGGRAPH 94, 91--100. Google ScholarDigital Library
- Wyszecki, G., and Styles, W. 1982. Color Science: Concepts and Methods, Quantitative Data and Formulae. Wiley, New York, NY.Google Scholar
Index Terms
- Real-time video abstraction
Recommendations
Real-time video abstraction
We present an automatic, real-time video and image abstraction framework that abstracts imagery by modifying the contrast of visually important features, namely luminance and color opponency. We reduce contrast in low-contrast regions using an ...
Real-time saliency-aware video abstraction
Existing real-time automatic video abstraction systems rely on local contrast only for identifying perceptually important information and abstract imagery by reducing contrast in low-contrast regions while artificially increasing contrast in higher ...
Abstraction of photographs: a comics style approach
ECC'09: Proceedings of the 3rd international conference on European computing conferenceWe present an automatic image abstraction framework that abstracts photographs through a comics style approach. Our model extracts image structures and creates a depth map. Then it automatically adapts line drawing and colorization to stylize the image. ...
Comments