Dan B Goldman
Brian Curless
Abstract
We present a method for visualizing short video clips in a single static image, using the visual language of storyboards. These schematic storyboards are composed from multiple input frames and annotated using outlines, arrows, and text describing the motion in the scene. The principal advantage of this storyboard representation over standard representations of video -- generally either a static thumbnail image or a playback of the video clip in its entirety -- is that it requires only a moment to observe and comprehend but at the same time retains much of the detail of the source video. Our system renders a schematic storyboard layout based on a small amount of user interaction. We also demonstrate an interaction technique to scrub through time using the natural spatial dimensions of the storyboard. Potential applications include video editing, surveillance summarization, assembly instructions, composition of graphic novels, and illustration of camera technique for film studies.
Supplemental Material
- Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., and Cohen, M. 2004. Interactive digital photomontage. ACM Transactions on Graphics (Proc. SIGGRAPH) 23, 4, 294--301. Google Scholar
Digital Library
- Assa, J., Caspi, Y., and Cohen-Or, D. 2005. Action synopsis: Pose selection and illustration. ACM Transactions on Graphics (Proc. SIGGRAPH) 24, 3, 667--676. Google ScholarDigital Library
- Cheong, L., and Huo, H. 2001. Shot change detection using scene-based constraint. Multimedia Tools and Applications 14, 2 (June), 175--186. Google ScholarDigital Library
- Cutting, J. E. 2002. Representing motion in a static image: constraints and parallels in art, science, and popular culture. Perception 31, 1165--1193.Google ScholarCross Ref
- Freeman, W. T., and Zhang, H. 2003. Shape-time photography. In Proc. Computer Vision and Pattern Recognition, 151--157.Google Scholar
- Hart, J. 1999. The art of the storyboard: storyboarding for film, TV and animation. Focal Press.Google Scholar
- Heng, W., and Ngan, K. 2001. An object-based shot boundary detection using edge tracing and tracking. Journal of Visual Communication and Image Representation 12, 3 (September), 217--239.Google ScholarDigital Library
- Horn, B., Hilden, H., and Negahdaripour, S. 1988. Closed-form solution of absolute orientation using orthonormal matrices. Journal of the Optical Society of America A 5, 7, 1127--1135.Google ScholarCross Ref
- Irani, M., and Anandan, P. 1998. Video indexing based on mosaic representations. IEEE Transactions on Pattern Analysis and Machine Intelligence 86, 5 (May), 905--921.Google Scholar
- Jia, J., and Tang, C.-K. 2005. Eliminating structure and intensity misalignment in image stitching. In Proc. ICCV. Google ScholarDigital Library
- Jojic, N., Basu, S., Petrovic, N., Frey, B., and Huang, T., 2003. Joint design of data analysis algorithms and user interface for video applications. presented at NIPS 2003 workshop on Machine Learning in User Interface, extended abstract at: http://research.microsoft.com/workshops/MLUI03/jojic.html.Google Scholar
- Katz, S. D. 1991. Film directing shot by shot: visualizing from concept to screen. Michael Wiese Productions.Google Scholar
- Kawagishi, Y., Hatsuyama, K., and Kondo, K. 2003. Cartoon blur: nonphotorealistic motion blur. In Proc. Comp. Graph. Intl., 276--281.Google Scholar
- Kim, B., and Essa, I. 2005. Video-based nonphotorealistic and expressive illustration of motion. In Proc. Comp. Graph. Intl., 32--35. Google ScholarDigital Library
- Kumar, M. P., Torr, P. H. S., and Zisserman, A. 2005. Learning layered motion segmentations of video. In Proc. ICCV, 33--40. Google ScholarDigital Library
- Lee, M., Yang, Y., and Lee, S. 2001. Automatic video parsing using shot boundary detection and camera operation analysis. Pattern Recognition 34, 3 (March), 711--719.Google ScholarCross Ref
- Li, Y., Zhang, T., and Tretter, D. 2001. An overview of video abstraction techniques. Tech. Rep. HPL-2001-191, HP Laboratories.Google Scholar
- Massey, M., and Bender, W. 1996. Salient stills: process and practice. IBM Systems Journal 35, 3, 4, 557--574. Google ScholarDigital Library
- Masuch, M., Schlechtweg, S., and Schultz, R. 1999. Speedlines: depicting motion in motionless pictures. In ACM SIGGRAPH 99 Conference abstracts and applications, 277. Google ScholarDigital Library
- McCloud, S. 1993. Understanding Comics: The Invisible Art. Harper-Collins.Google Scholar
- Nicolas, H., Manaury, A., Benois-Pineau, J., Dupuy, W., and Barba, D. 2004. Grouping video shots into scenes based on 1d mosaic descriptors. In Proc. Intl. Conf. on Image Proc., I: 637--640.Google Scholar
- Nienhaus, M., and Döllner, J. 2003. Dynamic glyphs -- depicting dynamics in images of 3D scenes. In Third International Symposium on Smart Graphics, 102--111. Google ScholarDigital Library
- Rother, C., Kolmogorov, V., and Blake, A. 2004. "GrabCut" -- interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics (Proc. SIGGRAPH) 23, 3, 309--314. Google ScholarDigital Library
- Rubin, M. 2005. Droidmaker: George Lucas and the digital revolution. Triad Publishing. 327, 338.Google Scholar
- Sangster, C., 2005. Personal Communication.Google Scholar
- Taniguchi, Y., Akutsu, A., and Tonomura, Y. 1997. PanoramaExcerpts: extracting and packing panoramas for video browsing. In Proc. ACM Intl. Conf. on Multimedia, 427--436. Google ScholarDigital Library
- Teodosio, L., and Bender, W. 1993. Salient video stills: Content and context preserved. In Proc. ACM Intl. Conf. on Multimedia, 39--46. Google ScholarDigital Library
- Umeyama, S. 1991. Least-squares estimation of transformation parameters between two point patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 13, 4, 376--380. Google ScholarDigital Library
- Ward, J. 1979. Perception and Pictorial Representation, vol. 1. Praeger, New York, ch. 13, "A piece of the action: Moving figures in still pictures", 246--271.Google Scholar
- Wexler, Y., and Simakov, D. 2005. Space-time scene manifolds. In Proc. ICCV, 858--863. Google ScholarDigital Library
- Wood, D. N., Finkelstein, A., Hughes, J. F., Thayer, C. E., and Salesin, D. H. 1997. Multiperspective panoramas for cel animation. In Proc. SIGGRAPH, 243--250. Google ScholarDigital Library
- Zelnik-Manor, L., Peters, G., and Perona, P. 2005. Squaring the circle in panoramas. In Proc. ICCV, 1292--1299. Google ScholarDigital Library
Index Terms
- Schematic storyboarding for video visualization and editing
Recommendations
Schematic storyboarding for video visualization and editing
SIGGRAPH '06: ACM SIGGRAPH 2006 PapersWe present a method for visualizing short video clips in a single static image, using the visual language of storyboards. These schematic storyboards are composed from multiple input frames and annotated using outlines, arrows, and text describing the ...
Multi-clip video editing from a single viewpoint
CVMP '14: Proceedings of the 11th European Conference on Visual Media ProductionWe propose a framework for automatically generating multiple clips suitable for video editing by simulating pan-tilt-zoom camera movements within the frame of a single static camera. Assuming important actors and objects can be localized using computer ...
Interactive 3D video editing
We present a generic and versatile framework for interactive editing of 3D video footage. Our framework combines the advantages of conventional 2D video editing with the power of more advanced, depth-enhanced 3D video streams. Our editor takes 3D video ...
Comments