Yung-Yu Chuang
Richard Szeliski
Abstract
This paper describes a new framework for video matting, the process of pulling a high-quality alpha matte and foreground from a video sequence. The framework builds upon techniques in natural image matting, optical flow computation, and background estimation. User interaction is comprised of garbage matte specification if background estimation is needed, and hand-drawn keyframe segmentations into "foreground," "background" and "unknown". The segmentations, called trimaps, are interpolated across the video volume using forward and backward optical flow. Competing flow estimates are combined based on information about where flow is likely to be accurate. A Bayesian matting technique uses the flowed trimaps to yield high-quality mattes of moving foreground elements with complex boundaries filmed by a moving camera. A novel technique for smoke matte extraction is also demonstrated.
- BARRON, J. L., FLEET, D. J., AND BEAUCHEMIN, S. S. 1994. Performance of optical flow techniques. International Journal of Computer Vision 12, 1, 43-77. Google Scholar
- BERMAN, A., DADOURIAN, A., AND VLAHOS, P., 2000. Method for removing from an image the background surrounding a selected object. U.S. Patent 6,134,346.Google Scholar
- BLACK, M. J., AND ANANDAN, P. 1996. The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. Computer Vision and Image Understanding 63, 1, 75-104. Google Scholar
- BLAKE, A., AND ISARD, M. 1998. Active Contours. Springer Verlag, London.Google Scholar
- CHUANG, Y.-Y., CURLESS, B., SALESIN, D., AND SZELISKI, R. 2001. A Bayesian approach to digital matting. In Proceedings of Computer Vision and Pattern Recognition (CVPR 2001), vol. II, 264 - 271. Google Scholar
- GLEICHER, M. 1995. Image snapping. In Proceedings of ACM SIGGRAPH 95, 183-190. Google Scholar
- HILLMAN, P., HANNAH, J., AND RENSHAW, D. 2001. Alpha channel estimation in high resolution images and image sequences. In Proceedings of Computer Vision and Pattern Recognition (CVPR 2001), vol. I, 1063-1068.Google Scholar
- KELLY, D. 2000. Digital Composition. The Coriolis Group.Google Scholar
- LEE, M.-C., ET AL. 1997. A layered video object coding system using sprite and affine motion model. lEEE Transactions on Circuits and Systems for Video Technology 7, 1, 130-145. Google Scholar
- MITSUNAGA, T., YOKOYAMA, T., AND TOTSUKA, T. 1995. Autokey: Human assisted key extraction. In Proceedings of ACM SIGGRAPH 95, 265-272. Google Scholar
- MORTENSEN, E. N., AND BARRETT, W. A. 1995. Intelligent scissors for image composition. In Proceedings of ACM SIGGRAPH 95, 191-198. Google Scholar
- PORTER, T., AND DUFF, T. 1984. Compositing digital images. In Computer Graphics (Proceedings of ACM SIGGRAPH 84), vol. 18, 253-259. Google Scholar
- RUZON, M. A., AND TOMASI, C. 2000. Alpha estimation in natural images. In Proceedings of Computer Vision and Pattern Recognition (CVPR 2000), 18-25.Google Scholar
- SMITH, A. R., AND BLINN, J. F. 1996. Blue screen matting. In Proceedings of ACM SIGGRAPH 96, 259-268. Google Scholar
- SUN, S., HAYNOR, D., AND KIM, Y. 2000. Motion estimation based on optical flow with adaptive gradients. In Proceedings of International Conference on Image Processing (ICIP 2000), vol. I, 852-855.Google Scholar
- SZELISKI, R., AND SHUM, H.-Y. 1997. Creating full view panoramic mosaics and environment maps. In Proceedings of ACM SIGGRAPH 97, 251-258. Google Scholar
- WANG, J. Y. A., AND ADELSON, E. H. 1994. Representing moving images with layers. IEEE Transactions on Image Processing 3, 5, 625-638.Google Scholar
Index Terms
- Video matting of complex scenes
Recommendations
Poisson matting
SIGGRAPH '04: ACM SIGGRAPH 2004 PapersIn this paper, we formulate the problem of natural image matting as one of solving Poisson equations with the matte gradient field. Our approach, which we call Poisson matting, has the following advantages. First, the matte is directly reconstructed ...
Poisson matting
In this paper, we formulate the problem of natural image matting as one of solving Poisson equations with the matte gradient field. Our approach, which we call Poisson matting, has the following advantages. First, the matte is directly reconstructed ...
Video matting of complex scenes
SIGGRAPH '02: Proceedings of the 29th annual conference on Computer graphics and interactive techniquesThis paper describes a new framework for video matting, the process of pulling a high-quality alpha matte and foreground from a video sequence. The framework builds upon techniques in natural image matting, optical flow computation, and background ...
Comments