Feng Liu
ABSTRACT
When a video is displayed on a smaller display than originally intended, some of the information in the video is necessarily lost. In this paper, we introduce Video Retargeting that adapts video to better suit the target display, minimizing the important information lost. We define a framework that measures the preservation of the source material, and methods for estimating the important information in the video. Video retargeting crops each frame and scales it to fit the target display. An optimization process minimizes information loss by balancing the loss of detail due to scaling with the loss of content and composition due to cropping. The cropping window can be moved during a shot to introduce virtual pans and cuts, subject to constraints that ensure cinematic plausibility. We demonstrate results of adapting a variety of source videos to small display sizes.
- D. Arijon. Grammar of the Film Language. Silman-James Press; Reprint edition, 1991.Google Scholar
- W. Bares and J. Lester. Intelligent multi-shot visualization interfaces for dynamic 3d worlds. In Proceedings of the 1999 International Conference on Intelligent User Interfaces, pages 119--126, 1999. Google ScholarDigital Library
- M. Bianchi. Automatic video production of lectures using an intelligent and aware environment. In MUM '04: Proceedings of the 3rd international conference on Mobile and ubiquitous multimedia, pages 117--123, New York, NY, USA, 2004. ACM Press. Google ScholarDigital Library
- J. Cantine, B. Lewis, and S. Howard. Shot by Shot; A Practical Guide to Filmmaking. Pittsburgh Filmmakers, 1995.Google Scholar
- L.-Q. Chen, X. Xie, X. Fan, W.-Y. Ma, H.-J. Zhang, and H.-Q. Zhou. A visual attention model for adapting images on small displays. ACM Multimedia Systems Journal, 9(4):353--364, November 2003.Google ScholarDigital Library
- F. C. Crow. Summed-area tables for texture mapping. In Proc. of SIGGRAPH 84, pages 207--212, 1984. Google ScholarDigital Library
- X. Fan, X. Xie, H.-Q. Zhou, and W.-Y. Ma. Looking into video frames on small displays. In Proc. of ACM Multimedia, pages 247--250, 2003. Google ScholarDigital Library
- A. Girgensohn, J. Boreczky, P. Chiu, J. Doherty, J. Foote, G. Golovchinsky, S. Uchihashi, and L. Wilcox. A semi-automatic approach to home video editing. In Proceedings of UIST, pages 81--89, 2000. Google ScholarDigital Library
- L. He, M. Cohen, and D. Salesin. The virtual cinematographer: A paradigm for automatic real-time camera control and directing. Proceedings of SIGGRAPH 96, pages 217--224, August 1996. Google ScholarDigital Library
- R. Heck, M. Wallick, and M. Gleicher. Virtual videography. ACM Transactions on Multimedia Computing, Communications and Applications, 3(1), 2007. to appear. Google ScholarDigital Library
- Y. Hu, D. Rajan, and L.-T. Chia. Robust subspace analysis for detecting visual attention regions in images. In Proc. of ACM Multimedia, pages 716--724, 2005. Google ScholarDigital Library
- L. Itti and C. Koch. Computational modeling of visual attention. Nature Reviews Neuroscience, 2(3):194--203, March 2001.Google ScholarCross Ref
- L. Itti, C. Koch, and E. Niebur. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell., 2 (11):1254--1259, November 1998. Google ScholarDigital Library
- S. Katz. Film Directing: Shot by Shot: Visualizing from Concept to Screen. Michael Wiese Productions, 1991.Google Scholar
- F. Liu and M. Gleicher. Automatic image retargeting with fisheye-view warping. In Proc. of the 18th annual ACM symposium on User interface software and technology, pages 153--162, 2005. Google ScholarDigital Library
- B. Lucas and T. Kanade. An iterative image registration technique with an application to stereo vision. In Proc. of International Joint Conference on Artificial Intelligence, pages 674--679, 1981.Google ScholarDigital Library
- Y.-F. Ma and H.-J. Zhang. A model of motion attention for video skimming. In Proc. IEEE ICIP, pages 129--132, 2002.Google Scholar
- Y.-F. Ma and H.-J. Zhang. Contrast-based image attention analysis by using fuzzy growing. In Proc. of ACM Multimedia, pages 374--381, 2003. Google ScholarDigital Library
- H. Nothdurft. Salience from feature contrast: additivity across dimensions. Vision Research, 40(11-12):1183--1201, 2000.Google ScholarCross Ref
- H. Nothdurft. Salience from feature contrast: variations with texture density. Vision Research, 40(23):3181--3200, 2000.Google ScholarCross Ref
- W. R. and H. T. Fast camera motion analysis in mpeg domain. In Proc. of IEEE ICIP, pages 691--694, 1999.Google Scholar
- M. Rasheed, Z. and Shah. Scene detection in hollywood movies and tv shows. In Proc. of IEEE CVPR, pages 343--348, 2003.Google ScholarCross Ref
- R. Rosenholtz. A simple saliency model predicts a number of motion popout phenomena. Vision Research, 39(19):3157--3163, 1999.Google ScholarCross Ref
- Y. Rui, L. He, A. Gupta, and Q. Liu. Building an intelligent camera management system. In Proceedings of ACM Multimedia Conference, 2001. Google ScholarDigital Library
- A. Santella, M. Agrawala, D. DeCarlo, D. Salesin, and M. Cohen. Gaze-based interaction for semi-automatic photo cropping. In Proc. of CHI, 2006. Google ScholarDigital Library
- V. Setlur, S. Takagi, R. Raskar, M. Gleicher, and B. Gooch. Automatic image retargeting. In Proc. of International Conference on Mobile and Ubiquitous Multimedia, 2005. Google ScholarDigital Library
- B. Suh, H. Ling, B. B. Bederson, and D. W. Jacobs. Automatic thumbnail cropping and its effectiveness. In Proc. of the 16th annual ACM symposium on User interface software and technology, pages 95--104, 2003. Google ScholarDigital Library
- C. Tomasi and R. Manduchi. Bilateral filtering for gray and color images. In Proc. of IEEE ICCV 98, pages 839--846, 1998. Google ScholarDigital Library
- P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features. In Proc. of IEEE CVPR, pages 511--518, 2001.Google ScholarCross Ref
- J. Wang, M. Reinders, R. Lagendijk, J. Lindenberg, and M. Kankanhalli. Video content presentation on tiny devices. In Proc. of IEEE Conference on Multimedia and Expo, pages 1711--1714, 2004.Google Scholar
Index Terms
- Video retargeting: automating pan and scan
Recommendations
A Survey on Content-Aware Image and Video Retargeting
This survey introduces the current state of the art in image and video retargeting and describes important ideas and technologies that have influenced the recent work. Retargeting is the process of adapting an image or video from one screen resolution ...
Improved seam carving for video retargeting
Video, like images, should support content aware resizing. We present video retargeting using an improved seam carving operator. Instead of removing 1D seams from 2D images we remove 2D seam manifolds from 3D space-time volumes. To achieve this we ...
Gaze-Driven Video Re-Editing
Given the current profusion of devices for viewing media, video content created at one aspect ratio is often viewed on displays with different aspect ratios. Many previous solutions address this problem by retargeting or resizing the video, but a more ...
Comments