Ariel Shamir
Olga Sorkine
ABSTRACT
The increasing variety of commonly used display devices, especially mobile devices, requires adapting visual media to different resolutions and aspect ratios - a process called "retargeting." The media retargeting problem is further accentuated by the explosion of image and video content on the web. This course presents a comparative overview of the latest research in visual-media retargeting. It focuses on content-aware approaches, which, contrary to traditional scaling and cropping, adapt to the salient information within the image or video and rescale the content while preserving visually important information.
Topics include:
•Algorithmic details and practical considerations of the retargeting pipeline, including its two main parts (saliency estimation and resizing operators).
•Recent trends in retargeting operators, namely discrete graph-based approaches, also known as seam carving.
•Continuous methods that operate by image and video warping.
•Temporally coherent video retargeting and multi-operator frameworks.
The course illuminates the theoretical foundations and practical issues involved in media retargeting, and provides attendees a comprehensive understanding of the state of the art. It includes many live demos of the various resizing techniques.
- Avidan, S., and Shamir, A. 2007. Seam carving for content-aware image resizing. ACM Trans. Graph. 26, 3, 10. Google Scholar
Digital Library
- Boykov, Y., and Jolly, M.-P. 2001. Interactive graph cuts for optimal boundary&region segmentation of objects in n-d images. In International Conference on Computer Vision, (ICCV), vol. I, 105--112.Google Scholar
- Dalal, N., and Triggs, B. 2005. Histograms of oriented gradients for human detection. In International Conference on Computer Vision&Pattern Recognition, vol. 2, 886--893. Google ScholarDigital Library
- DeCarlo, D., and Santella, A. 2002. Stylization and abstraction of photographs. In Proceedings of SIGGRAPH, 769--776. Google ScholarDigital Library
- Fan, X., Xie, X., Zhou, H.-Q., and Ma, W.-Y. 2003. Looking into video frames on small displays. In Multimedia '03, ACM, New York, NY, USA, 247--250. Google ScholarDigital Library
- Gal, R., Sorkine, O., and Cohen-Or, D. 2006. Feature-aware texturing. In Eurographics Symposium on Rendering, 297--303. Google ScholarDigital Library
- Harris, C., and Stephens, M. 1988. A combined corner and edge detector. In Proceedings of the 4th Alvey Vision Conference, 147--151.Google Scholar
- Itti, L., Koch, C., and Niebur, E. 1998. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20, 11, 1254--1259. Google ScholarDigital Library
- Krähenbühl, P., Lang, M., Hornung, A., and Gross, M. 2009. A system for retargeting of streaming video. ACM Trans. Graph. 28, 5. Google ScholarDigital Library
- Kwatra, V., Schödl, A., Essa, I., Turk, G., and Bobick, A. 2003. Graphcut textures: image and video synthesis using graph cuts. ACM Trans. Graph. 22, 3, 277--286. Google ScholarDigital Library
- Lombaert, H., Sun, Y., Grady, L., and Xu, C. 2005. A multilevel banded graph cuts method for fast image segmentation. In Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV), vol. 1, 259--265. Google ScholarDigital Library
- Roditty, L., and Zwick, U. 2004. On dynamic shortest paths problems. In In Proceedings of 12th Annual European Symposium on Algorithms (ESA), 580--591.Google Scholar
- Rubinstein, M., Shamir, A., and Avidan, S. 2008. Improved seam carving for video retargeting. ACM Trans. Graph. 27, 3. Google ScholarDigital Library
- Rubinstein, M., Shamir, A., and Avidan, S. 2009. Multioperator media retargeting. ACM Trans. Graph. 28, 3. Google ScholarDigital Library
- Santella, A., Agrawala, M., DeCarlo, D., Salesin, D., and Cohen, M. 2006. Gaze-based interaction for semiautomatic photo cropping. In Proceedings of CHI, 771--780. Google ScholarDigital Library
- Schödl, A., Szeliski, R., Salesin, D. H., and Essa, I. 2000. Video textures. In SIGGRAPH '00: Proceedings of the 27th annual conference on Computer graphics and interactive techniques, ACM Press/Addison-Wesley Publishing Co., 489--498. Google ScholarDigital Library
- Shamir, A., and Avidan, S. 2009. Seam carving for media retargeting. Communications of the ACM 52, 1, 77--85. Google ScholarDigital Library
- Viola, P., and Jones, M. J. 2004. Robust real-time face detection. Int. J. Comput. Vision 57, 2, 137--154. Google ScholarDigital Library
- Wang, J., Xu, Y., Shum, H.-Y., and Cohen, M. F. 2004. Video tooning. ACM Trans. Graph. 23, 3, 574--583. Google ScholarDigital Library
- Wang, J., Bhat, P., Colburn, R. A., Agrawala, M., and Cohen, M. F. 2005. Interactive video cutout. ACM Trans. Graph. 24, 3, 585--594. Google ScholarDigital Library
- Wang, Y.-S., Tai, C.-L., Sorkine, O., and Lee, T.-Y. 2008. Optimized scale-and-stretch for image resizing. ACM Trans. Graph. 27, 5. Google ScholarDigital Library
- Wang, Y.-S., Fu, H., Sorkine, O., Lee, T.-Y., and Seidel, H.-P. 2009. Motion-aware temporal coherence for video resizing. ACM Trans. Graph. (Proceedings of ACM SIGGRAPH ASIA) 28, 5. Google ScholarDigital Library
- Wolf, L., Guttmann, M., and Cohen-Or, D. 2007. Non-homogeneous content-driven video-retargeting. In IEEE International Conference on Computer Vision (ICCV).Google Scholar
- Zabih, R., Miller, J., and Mai, K. 1995. A feature-based algorithm for detecting and classifying scene breaks. In ACM Multimedia, 189--200. Google ScholarDigital Library
- Zhang, Y.-F., Hu, S.-M., and Martin, R. R. 2008. Shrinkability maps for content-aware video resizing. Computer Graphics Forum 27, 7, 1797--1804.Google ScholarCross Ref
Index Terms
- Visual media retargeting
Recommendations
A comparative study of image retargeting
SIGGRAPH ASIA '10: ACM SIGGRAPH Asia 2010 papersThe numerous works on media retargeting call for a methodological approach for evaluating retargeting results. We present the first comprehensive perceptual study and analysis of image retargeting. First, we create a benchmark of images and conduct a ...
Issues and Challenges in Content-Based Media Retargeting
CSNT '14: Proceedings of the 2014 Fourth International Conference on Communication Systems and Network TechnologiesSmartphones are rapidly increasing in this technological world, in that visual media is the most important thing in mobile arena and due to visual graphical world everyone wants high graphical visual media. Now a day demands of videos and images are ...
Compressibility-Aware Media Retargeting With Structure Preserving
A number of algorithms have been proposed for intelligent image/video retargeting with image content retained as much as possible. However, they usually suffer from some artifacts in the results, such as ridge or structure twist. In this paper, we ...
Comments