Aseem Agarwala
Abstract
We describe a new approach to rotoscoping --- the process of tracking contours in a video sequence --- that combines computer vision with user interaction. In order to track contours in video, the user specifies curves in two or more frames; these curves are used as keyframes by a computer-vision-based tracking algorithm. The user may interactively refine the curves and then restart the tracking algorithm. Combining computer vision with user interaction allows our system to track any sequence with significantly less effort than interpolation-based systems --- and with better reliability than "pure" computer vision systems. Our tracking algorithm is cast as a spacetime optimization problem that solves for time-varying curve shapes based on an input video sequence and user-specified constraints. We demonstrate our system with several rotoscoped examples. Additionally, we show how these rotoscoped contours can be used to help create cartoon animation by attaching user-drawn strokes to the tracked contours.
Supplemental Material
Available for Download
- AGARWALA, A. 2002. SnakeToonz: A Semi-Automatic Approach to Creating Cel Animation from Video. In NPAR 2002: Second International Symposium on Non Photorealistic Rendering, 139--146. Google Scholar
Digital Library
- BEIER, T., AND NEELY, S. 1992. Feature-based image metamorphosis. In Computer Graphics (Proceedings of SIGGRAPH 92), vol. 26, 35--42. Google ScholarDigital Library
- BERGEN, J. R., ANANDAN, P., HANNA, K. J., AND HINGORANI, R. 1992. Hierarchical model-based motion estimation. In European Conference on Computer Vision, 237--252. Google ScholarDigital Library
- BLACK, M. J., AND ANANDAN, P. 1996. The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. Computer Vision and Image Understanding 63, 1, 75--104. Google ScholarDigital Library
- BLAKE, A., AND ISARD, M. 1998. Active Contours, Springer-Verlag.Google Scholar
- BURTNYK, N., AND WEIN, M. 1976. Interactive skeleton techniques for enhancing motion dynamics in key frame animation. CACM 19 (Oct.), 564--569. Google ScholarDigital Library
- CHUANG, Y.-Y., AGARWALA, A., CURLESS, B., SALESIN, D. H., AND SZELISKI, R. 2002. Video matting of complex scenes. ACM Transactions on Graphics 21, 3, 243--248. Google ScholarDigital Library
- COHEN, M. F. 1992. Interactive spacetime control for animation. In Computer Graphics (Proceedings of SIGGRAPH 92), vol. 26, 293--302. Google ScholarDigital Library
- FLEISCHER, M., 1917. Method of Producing Moving Picture Cartoons. US Patent no. 1,242,674.Google Scholar
- GEIGER, D., GUPTA, A., COSTA, L., AND VLONTZOS, J. 1995. Dynamic programming for detecting, tracking and matching deformable contours. IEEE Transactions On Pattern Analysis and Machine Intelligence 17, 3, 294--302. Google ScholarDigital Library
- GOLDMAN, D., 2003. Computer graphics supervisor, Industrial Light & Magic, personal communication.Google Scholar
- HALL, J., GREENHILL, D., AND JONES, G. 1997. Segmenting film sequences using active surfaces. In International Conference on Image Processing (ICIP), 751--754. Google ScholarDigital Library
- HOCH, M., AND LITWINOWICZ, P. C. 1996. A semi-automatic system for edge tracking with snakes. The Visual Computer 12, 2, 75--83.Google ScholarCross Ref
- HSU, S. C., AND LEE, I. H. H. 1994. Drawing and animation using skeletal strokes. In Proceedings of SIGGRAPH 94, 109--118. Google ScholarDigital Library
- IRANI, M. 2002. Multi-Frame Correspondence Estimation Using Subspace Constraints. International Journal of Computer Vision 48, 3, 173--194. Google ScholarDigital Library
- KALNINS, R. D., MARKOSIAN, L., MEIER, B. J., KOWALSKI, M. A., LEE, J. C., DAVIDSON, P. L., WEBB, M., HUGHES, J. F., AND FINKELSTEIN, A. 2002. WYSIWYG NPR: Drawing Strokes Directly on 3D Models. ACM Transactions on Graphics 21, 3, 755--762. Google ScholarDigital Library
- KASS, M., WITKIN, A., AND TERZOPOULOS, D. 1987. Snakes: Active contour models. International Journal of Computer Vision 1, 4, 321--331.Google ScholarCross Ref
- KORT, A. 2002. Computer aided inbetweening. In NPAR 2002: Second International Symposium on Non Photorealistic Rendering, 125--132. Google ScholarDigital Library
- LINKLATER, R., 2001. Waking Life. 20th Century Fox.Google Scholar
- LITWINOWICZ, P., AND WILLIAMS, L. 1994. Animating images with drawings. In Proceedings of SIGGRAPH 94, 409--412. Google ScholarDigital Library
- LITWINOWICZ, P. 1997. Processing images and video for an impressionist effect. In Proceedings of SIGGRAPH 97, 407--414. Google ScholarDigital Library
- LUCAS, B. D., AND KANADE, T. 1981. An iterative image registration technique with an application to stereo vision. In Proceedings of the 7th International Joint Conference on Artificial Intelligence (IJCAI'81), 674--679.Google Scholar
- LUO, H., AND ELEFTHERIADIS, A. 1999. Spatial temporal active contour interpolation for semi-automatic video object generation. In International Conference on Image Processing (ICIP), 944--948.Google ScholarCross Ref
- MEIER, B. J. 1996. Painterly rendering for animation. In Proceedings of SIGGRAPH 96, 477--484. Google ScholarDigital Library
- MITSUNAGA, T., YOKOYAMA, T., AND TOTSUKA, T. 1995. Autokey: Human assisted key extraction. In Proceedings of SIGGRAPH 95, 265--272. Google ScholarDigital Library
- MORTENSEN, E. N., AND BARRETT, W. A. 1995. Intelligent scissors for image composition. In Proceedings of SIGGRAPH 95, 191--198. Google ScholarDigital Library
- MORTENSEN, E. N., 1999. Vision-assisted image editing. Computer Graphics 33, 4 (Nov.), 55--57. Google ScholarDigital Library
- NOCEDAL, J., AND WRIGHT, S. J. 1999. Numerical Optimization. Springer.Google Scholar
- SEDERBERG, T. W., AND GREENWOOD, E. 1992. A physically based approach to 2d shape blending. In Computer Graphics (Proceedings of SIGGRAPH 92), vol. 26, 25--34. Google ScholarDigital Library
- SEDERBERG, T. W., GAO, P., WANG, G., AND MU, H. 1993. 2d shape blending: An intrinsic solution to the vertex path problem. In Proceedings of SIGGRAPH 93, 15--18. Google ScholarDigital Library
- SMITH, A. R., AND BLINN, J. F. 1996. Blue screen matting. In Proceedings of SIGGRAPH 96, 259--268. Google ScholarDigital Library
- STEIHAUG, T. 1983. The conjugate gradient method and trust regions in large scale optimization. SIAM Journal on Numerical Analysis 20, 3, 626--637.Google ScholarCross Ref
- STEWART, S., 2003. Confessions of a roto artist: Three rules for better mattes. http://www.pinnaclesys.com/SupportFiles/Rotoscoping.pdf.Google Scholar
- SZELISKI, R. 1990. Fast surface interpolation using hierarchical basis functions. IEEE Transactions On Pattern Analysis and Machine Intelligence 12, 6, 513--528. Google ScholarDigital Library
- TORRESANI, L., AND BREGLER, C. 2002. Space-time tracking. In European Conference on Computer Vision, 801--802. Google ScholarDigital Library
- WITKIN, A., AND KASS, M. 1988. Spacetime constraints. In Computer Graphics (Proceedings of SIGGRAPH 88), vol. 22, 159--168. Google ScholarDigital Library
Index Terms
- Keyframe-based tracking for rotoscoping and animation
Recommendations
Keyframe-based tracking for rotoscoping and animation
SIGGRAPH '04: ACM SIGGRAPH 2004 PapersWe describe a new approach to rotoscoping --- the process of tracking contours in a video sequence --- that combines computer vision with user interaction. In order to track contours in video, the user specifies curves in two or more frames; these ...
Stylized keyframe animation of fluid simulations
NPAR '14: Proceedings of the Workshop on Non-Photorealistic Animation and RenderingWe present a method that combines hand-drawn artwork with fluid simulations to produce animated fluids in the visual style of the artwork. Given a fluid simulation and a set of keyframes rendered by the artist in any medium, our system produces a set of ...
Interactive rotoscoping: extracting and tracking object sketch
ICIP'09: Proceedings of the 16th IEEE international conference on Image processingIn this paper, we discuss an integrated system for video rotoscoping extracting and tracking object sketch (structural shape) across video sequence. This system consists of two key components: object sketch computing and graph-based object tracking. ...
Comments