Yin Li
Abstract
In this paper, we present a system for cutting a moving object out from a video clip. The cutout object sequence can be pasted onto another video or a background image. To achieve this, we first apply a new 3D graph cut based segmentation approach on the spatial-temporal video volume. Our algorithm partitions watershed presegmentation regions into foreground and background while preserving temporal coherence. Then, the initial segmentation result is refined locally. Given two frames in the video sequence, we specify two respective windows of interest which are then tracked using a bi-directional feature tracking algorithm. For each frame in between these two given frames, the segmentation in each tracked window is refined using a 2D graph cut that utilizes a local color model. Moreover, we provide brush tools for the user to control the object boundary precisely wherever needed. Based on the accurate binary segmentation result, we apply coherent matting to extract the alpha mattes and foreground colors of the object.
Supplemental Material
- Agarwala, A., Hertzmann, A., Seitz, S., and Salesin, D. H. 2004. Keyframe-based tracking for rotoscoping and animation. In Proceedings of ACM SIGGRAPH 2004, 584--591. Google Scholar
Digital Library
- Apostoloff, N. E., and Fitzgibbon, A. W. 2004. Bayesian video matting using learnt image priors. In Proceedings of CVPR 2004, I: 407--414.Google Scholar
- Bellman, R. E. 1957. Dynamic Programming. Princeton University Press. Princeton. NJ. Google ScholarDigital Library
- Black, M. J., and Ananda, P. 1996. The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. In Computer Vision and Image Understanding, vol. 63, 75--104. Google ScholarDigital Library
- Blake, A., and Isard. M. 1998. Active contours. In Springer Verlag, London.Google Scholar
- Blake, A., Rother, C., Brown, M., P. Perez, and P. Torr. 2004. Interactive image segmentation using an adaptive gmmrf model. In Proceedings of ECCV. I: 428--441.Google Scholar
- Boykov, Y., and Jolly, M. P. 2001. Interactive graph cuts for optimal boundary & region segmentation of objects in n-d images. In Proceedings of ICCV 2001. I: 105--112.Google Scholar
- Chuang, Y.-Y. Curless, B., Salesin, D. H., and Szeliski. R. 2001. A bayesian approach to digital matting. In Proceedings of CVPR 2001, II: 264--271.Google Scholar
- Chuang, Y.-Y., Agarwala, A., Curless, B., Salesin, D. H., and Szeliski. R. 2002. Video matting of complex scenes. In Proceedings of ACM SIGGRAPH 2002, 243--248. Google ScholarDigital Library
- Drori, I., Leyvand, T., Cohen-Or, D., and Yeshurun, H. 2004. Interactive object segmentation in video by fitting splines to graph cuts. In ACM SIGGRAPH 2004 Posters Session. Google ScholarDigital Library
- Kass, M., Witkin, A., and Terzopoulos, D. 1987. Snakes: Active contour models. International Journal on Computer Vision 1, 4, 321--331.Google ScholarCross Ref
- Kwatra, V., Schödl, A., Essa, I., Turk, G., and Bobick, A. 2003. Graphcut textures: Image and video synthesis using graph cuts. In Proceedings of ACM SIGGRAPH 2003, 277--286. Google ScholarDigital Library
- Li, Y., Sun, J., Tang, C. K., and Shum, H. Y. 2004. Lazy snapping. In Proceedings of ACM SIGGRAPH 2004, 303--308. Google ScholarDigital Library
- Mitsunaga, T., Yokoyama, T., and Totsuka, T. 1995. Autokey: Human assisted key extraction. In Proceedings of ACM SIGGRAPH'95, 265--272. Google ScholarDigital Library
- Mortensen, E. N., and Barrett, W. A. 1999. Toboggan-based intelligent scissors with a four parameter edge model. In Proceedings of CVPR 1999, II: 452--458.Google Scholar
- Rother, C., Blake, A., and Kolmogorov, V. 2004. Grabcut - interactive fore-ground extraction using iterated graph cuts. In Proceedings of ACM SIGGRAPH 2004, 309--314. Google ScholarDigital Library
- Shum, H., Sun, J., Yamazaki, S., Li, Y., and Tang, C. 2004. Pop-up light field: An interactive image-based modeling and rendering system. ACM Transaction of Graphics 23, 2, 143--162. Google ScholarDigital Library
- Smith, A. R., and Blinn, J. F. 1996. Blue screen matting. In Proceedings of ACM SIGGRAPH 1996, 259--268. Google ScholarDigital Library
- Vincent, L., and Soille, P. 1991. Watersheds in digital spaces: an efficient algorithm based on immersion simulations. IEEE Tran. on PAMI 13, 6, 583--598. Google ScholarDigital Library
Index Terms
- Video object cut and paste
Recommendations
Video object cut and paste
SIGGRAPH '05: ACM SIGGRAPH 2005 PapersIn this paper, we present a system for cutting a moving object out from a video clip. The cutout object sequence can be pasted onto another video or a background image. To achieve this, we first apply a new 3D graph cut based segmentation approach on ...
Bi-layer video segmentation with foreground and background infrared illumination
MM '08: Proceedings of the 16th ACM international conference on MultimediaIn this paper, we investigate two ways of employing infrared video with color video for automatic foreground-background video segmentation: foreground infrared (IR) illumination and background IR illumination. Foreground IR illumination gives an initial ...
Random Walk and Graph Cut for Co-Segmentation of Lung Tumor on PET-CT Images
Accurate lung tumor delineation plays an important role in radiotherapy treatment planning. Since the lung tumor has poor boundary in positron emission tomography (PET) images and low contrast in computed tomography (CT) images, segmentation of tumor in ...
Comments