Tai-Pang Wu
Abstract
This article addresses the problem of natural shadow matting, the removal or extraction of natural shadows from a single image. Because textures are maintained in the shadowless image after the extraction process, our approach produces some of the best results to date among shadow removal techniques. Using the image formation equation typical of computer vision, we advocate a new model for shadow formation where shadow effect is understood as light attenuation instead of a mixture of two colors governed by the conventional matting equation. This leads to a new shadow equation with fewer unknowns to solve, where a three-channel shadow matte and a shadowless image are considered in our optimization. Our problem is formulated as one of energy minimization guided by user-supplied hints in the form of a quadmap which can be specified easily by the user. This formulation allows for robust shadow matte extraction while maintaining texture in the shadowed region by considering color transfer, texture gradient, and shadow smoothness. We demonstrate the usefulness of our approach in shadow removal, image matting, and compositing.
- Baba, M., Mukunoki, M., and Asada, N. 2004. Shadow removal from a real image based on shadow density. In Proceedings of the SIGGRAPH 2004 Sketches and Applications. Google Scholar
Digital Library
- Barrow, H. and Tenenbaum, J. 1978. Recovering intrinsic scene characteristics from images. In Symposium of Center for Visual Science (CVS'78). 3--26.Google Scholar
- Bertalmio, M., Sapiro, G., Ballester, C., and Caselles, V. 2000. Image inpainting. SIGGRAPH 2000, 417--424. Google ScholarDigital Library
- Chuang, Y.-Y., Curless, B., Salesin, D. H., and Szeliski, R. 2001. A bayesian approach to digital matting. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR'01). Vol. 2, 264--271.Google Scholar
- Chuang, Y.-Y., Goldman, D. B., Curless, B., Salesin, D. H., and Szeliski, R. 2003. Shadow matting and compositing. ACM Trans. Graph. 22, 3, 494--500. Google ScholarDigital Library
- Drori, I., Cohen-Or, D., and Yeshurun, H. 2003. Fragment-based image completion. ACM Trans Graph. 22, 303--312. Google ScholarDigital Library
- Duda, R., Hart, P., and Stork, D. 2001. Pattern Classification. John Wiley. Google ScholarDigital Library
- Efros, A. and Leung, T. 1999. Texture synthesis by non-parametric sampling. In Proceedings of the International Conference on Computer Vision (ICCV'99). 1033--1038. Google ScholarDigital Library
- Finlayson, G., Drew, M., and Lu, C. 2004. Intrinsic images by entropy minimization. In European Conference on Computer Vision (ECCV'04). Vol 3, 582--595.Google Scholar
- Finlayson, G. D., Hordley, S. D., and Drew, M. S. 2002. Removing shadows from images. Proceedings of European Conference on Computer Vision (ECCV'02), Vol. 4, 823--836. Google ScholarDigital Library
- Jia, J. and Tang, C.-K. 2004. Inference of segmented color and texture description by tensor voting. Trans. Patt. Analy. and Mach. Intell. 26, 6 (June), 771--786. Google ScholarDigital Library
- Land, E. and McCann, J. 1971. Lightness and the retinex theory. J. Opt. Soc. Am, 1--11.Google ScholarCross Ref
- Li, Y., Sun, J., Tang, C.-K., and Shum, H.-Y. 2004. Lazy snapping. ACM Trans. Graph. 23, 3, 303--308. Google ScholarDigital Library
- Mitchell, T. 1997. Machine Learning. McGraw Hill. Google ScholarDigital Library
- Mortensen, E. N. and Barrett, W. A. 1995. Intelligent scissors for image composition. In SIGGRAPH '1995. 191--198. Google ScholarDigital Library
- Oh, B. M., Chen, M., Dorsey, J., and Durand, F. 2001. Image-based modeling and photo editing. In SIGGRAPH 2001. 433--442. Google ScholarDigital Library
- Porter, T. and Duff, T. 1984. Compositing digital images. Proceedings of SIGGRAPH 2004, 253--259. Google ScholarDigital Library
- Reinhard, E., Ashikhmin, M., Gooch, B., and Shirley, P. 2001. Color transfer between images. IEEE Comput. Graph. Applic. 21, 34--41. Google ScholarDigital Library
- Rother, C., Kolmogorov, V., and Blake, A. 2004. “grabcut”: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23, 3, 309--314. Google ScholarDigital Library
- Sun, J., Jia, J., Tang, C.-K., and Shum, H.-Y. 2004. Poisson matting. ACM Trans. Graph. 23, 3, 351--321. Google ScholarDigital Library
- Tappen, M., Freeman, W., and Adelson, E. 2002. Recovering intrinsic images from a single image. In MIT Advances in Modeling (AIM).Google Scholar
- Weiss, Y. 2001. Deriving intrinsic images from image sequences. In International Conference on Computer Vision (ICCV), Vol. 2, 68--75.Google ScholarCross Ref
- Welsh, T., Ashikhmin, M., and Mueller, K. 2002. Transferring color to greyscale images. ACM Trans. Graph. 21, 277--280. Google ScholarDigital Library
- Wright, S. 2001. Digital Compositing for Film and Video. Focal Press. Google ScholarDigital Library
- Wu, T.-P. and Tang, C.-K. 2005. A Bayesian approach for shadow extraction from a single image. In Proceedings of the International Conference on Computer Vision (ICCV'05). IEEE, 480--487. Google ScholarDigital Library
Index Terms
- Natural shadow matting
Recommendations
Learning to Remove Soft Shadows
Manipulated images lose believability if the user's edits fail to account for shadows. We propose a method that makes removal and editing of soft shadows easy. Soft shadows are ubiquitous, but remain notoriously difficult to extract and manipulate. We ...
Shadow Remover: Image Shadow Removal Based on Illumination Recovering Optimization
In this paper, we present a novel shadow removal system for single natural images as well as color aerial images using an illumination recovering optimization method. We first adaptively decompose the input image into overlapped patches according to the ...
Shadow matting and compositing
SIGGRAPH '03: ACM SIGGRAPH 2003 PapersIn this paper, we describe a method for extracting shadows from one natural scene and inserting them into another. We develop physically-based shadow matting and compositing equations and use these to pull a shadow matte from a source scene in which the ...
Comments