Dan B. Goldman
Chris Gonterman
ABSTRACT
We explore the use of tracked 2D object motion to enable novel approaches to interacting with video. These include moving annotations, video navigation by direct manipulation of objects, and creating an image composite from multiple video frames. Features in the video are automatically tracked and grouped in an off-line preprocess that enables later interactive manipulation. Examples of annotations include speech and thought balloons, video graffiti, path arrows, video hyperlinks, and schematic storyboards. We also demonstrate a direct-manipulation interface for random frame access using spatial constraints, and a drag-and-drop interface for assembling still images from videos. Taken together, our tools can be employed in a variety of applications including film and video editing, visual tagging, and authoring rich media such as hyperlinked video.
Supplemental Material
- A. Agarwala, M. Dontcheva, M. Agrawala, S. Drucker, A. Colburn, B. Curless, D. Salesin, and M. Cohen. Interactive digital photomontage. ACM Trans. Graph. (Proc. SIGGRAPH), 23(4):294--301, 2004. Google Scholar
Digital Library
- A. Agarwala, A. Hertzmann, D. H. Salesin, and S. M. Seitz. Keyframe-based tracking for rotoscoping and animation. ACM Trans. Graph. (Proc. SIGGRAPH), 23(3):584--591, 2004. Google ScholarDigital Library
- ASTAR Learning Systems. http://www.astarls.com, 2006. {Online; accessed 5-January-2008}.Google Scholar
- C. Bregler, M. Covell, and M. Slaney. Video rewrite: Driving visual speech with audio. In Proc. SIGGRAPH 97, pages 353--360, 1997. Google ScholarDigital Library
- Y.-Y. Chuang, A. Agarwala, B. Curless, D. H. Salesin, and R. Szeliski. Video matting of complex scenes. ACM Trans. Graph., 21(3):243--248, 2002. Google ScholarDigital Library
- J. Dakss, S. Agamanolis, E. Chalom, and V. M. Bove Jr. Hyperlinked video. In Proc. SPIE, volume 3528, pages 2--10, 1999.Google Scholar
- P. Dragicevic, G. Ramos, J. Bibliowicz, D. Nowrouzezahrai, R. Balakrishnan, and K. Singh. Video browsing by direct manipulation. In CHI, pages 237--246, 2008. Google ScholarDigital Library
- D. B Goldman, B. Curless, S. M. Seitz, and D. Salesin. Schematic storyboarding for video visualization and editing. ACM Trans. Graph. (Proc. SIGGRAPH), 25(3):862--871, 2006. Google ScholarDigital Library
- J. Goldman. Kind of a Blur. http://phobos.apple.com/WebObjects/MZStore.woa/wa/viewMovie?id=197994758, 2005. {Short film available online; accessed 5-January-2008}.Google Scholar
- R. I. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision, page 109. Cambridge University Press, ISBN: 0521540518, second edition, 2004. Google ScholarDigital Library
- T. Karrer, M. Weiss, E. Lee, and J. Borchers. DRAGON: A direct manipulation interface for frame-accurate in-scene video navigation. In CHI, pages 247--250, 2008. Google ScholarDigital Library
- D. Kimber, T. Dunnigan, A. Girgensohn, F. Shipman, T. Turner, and T. Yang. Trailblazing: Video playback control by direct object manipulation. In ICME, pages 1015--1018, 2007.Google ScholarCross Ref
- D. Kurlander, T. Skelly, and D. Salesin. Comic chat. In SIGGRAPH '96, pages 225--236, 1996. Google ScholarDigital Library
- Y. Li, J. Sun, and H.-Y. Shum. Video object cut and paste. ACM Trans. Graph. (Proc. SIGGRAPH), 24(3):595--600, 2005. Google ScholarDigital Library
- C. Morningstar and R. F. Farmer. The lessons of Lucasfilm's Habitat. In M. Benedikt, editor, Cyberspace: First Steps, pages 273--301. MIT Press, Cambridge, MA, 1991. Google ScholarDigital Library
- Y. Pritch, A. Rav-Acha, A. Gutman, and S. Peleg. Webcam synopsis: Peeking around the world. In Proc. ICCV, pages 1--8, 2007.Google ScholarCross Ref
- Y. Pritch, A. Rav-Acha, and S. Peleg. Non-chronological video synopsis and indexing. IEEE Trans. PAMI, 2008. (to appear). Google ScholarDigital Library
- G. Ramos and R. Balakrishnan. Fluid interaction techniques for the control and annotation of digital video. In Proc. UIST '03, pages 105--114, 2003. Google ScholarDigital Library
- A. Rav-Acha, Y. Pritch, D. Lischinski, and S. Peleg. Dynamosaicing: Video mosaics with non-chronological time. In Proc. CVPR, pages 58--65, 2005. Google ScholarDigital Library
- A. Rav-Acha, Y. Pritch, and S. Peleg. Making a long video short: Dynamic video synopsis. In Proc. CVPR, pages 435--441, 2006. Google ScholarDigital Library
- E. Rosten, G. Reitmayr, and T. Drummond. Real-time video annotations for augmented reality. In Proc. Intl. Symp. on Visual Computing, 2005. Google ScholarDigital Library
- P. Sand. Long-Range Video Motion Estimation using Point Trajectories. PhD thesis, MIT, 2006. Google ScholarDigital Library
- P. Sand and S. Teller. Particle video: Long-range motion estimation using point trajectories. In Proc. CVPR '06, pages 2195--2202, 2006. Google ScholarDigital Library
- A. Schödl and I. A. Essa. Controlled animation of video sprites. In Proc. ACM/Eurographics Symp. on Comp. Animation, pages 121--127, 2002. Google ScholarDigital Library
- A. Schödl, R. Szeliski, D. H. Salesin, and I. Essa. Video textures. In SIGGRAPH '00, pages 489--498, 2000. Google ScholarDigital Library
- J. Sivic, F. Schaffalitzky, and A. Zisserman. Object level grouping for video shots. Intl. J. of Comp. Vis., 67(2):189--210, 2006. Google ScholarDigital Library
- J. M. Smith, D. Stotts, and S.-U. Kum. An orthogonal taxonomy for hyperlink anchor generation in video streams using OvalTine. In Proc. ACM Conf. on Hypertext and Hypermedia, pages 11--18, 2000. Google ScholarDigital Library
- Sportvision. Changing The Game. http://www.sportvision.com, 2006. {Online; accessed 5-January-2008}.Google Scholar
- J. Wang, P. Bhat, R. A. Colburn, M. Agrawala, and M. F. Cohen. Interactive video cutout. ACM Trans. Graph. (Proc. SIGGRAPH), 24(3):585--594, 2005. Google ScholarDigital Library
- Wikipedia. Telestrator. http://en.wikipedia.org/w/index.php?title=Telestrator&oldid=180785495, 2006. {Online; accessed 5-January-2008}.Google Scholar
- A. Yilmaz, O. Javed, and M. Shah. Object tracking: A survey. ACM Computing Surveys, 38(4):13, December 2006. Google ScholarDigital Library
- L. Zhang, N. Snavely, B. Curless, and S. Seitz. Spacetime faces: high resolution capture for modeling and animation. ACM Trans. Graph. (Proc. SIGGRAPH), 23(3):548--558, 2004. Google ScholarDigital Library
Index Terms
- Video object annotation, navigation, and composition
Recommendations
Direct manipulation video navigation in 3D
CHI '13: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsDirect Manipulation Video Navigation (DMVN) systems allow a user to navigate a video by dragging an object along its motion trajectory. These systems have been shown effective for space-centric video browsing. Their performance, however, is often ...
Direct manipulation video navigation on touch screens
MobileHCI '14: Proceedings of the 16th international conference on Human-computer interaction with mobile devices & servicesDirect Manipulation Video Navigation (DMVN) systems allow a user to directly drag an object of interest along its motion trajectory and have been shown effective for space-centric video browsing tasks. This paper designs touch-based interface techniques ...
Schematic storyboarding for video visualization and editing
SIGGRAPH '06: ACM SIGGRAPH 2006 PapersWe present a method for visualizing short video clips in a single static image, using the visual language of storyboards. These schematic storyboards are composed from multiple input frames and annotated using outlines, arrows, and text describing the ...
Comments