Lamberto Ballan
Marco Bertini
Alberto Del Bimbo
Marco Meoni
ABSTRACT
Nowadays, almost any web site that provides means for sharing user-generated multimedia content, like Flickr, Facebook, YouTube and Vimeo, has tagging functionalities to let users annotate the material that they want to share. The tags are then used to retrieve the uploaded content, and to ease browsing and exploration of these collections, e.g. using tag clouds. However, while tagging a single image is straightforward, and sites like Flickr and Facebook allow also to tag easily portions of the uploaded photos, tagging a video sequence is more cumbersome, so that users just tend to tag the overall content of a video. Moreover, the tagging process is completely manual, and often users tend to spend as few time as possible to annotate the material, resulting in a sparse annotation of the visual content. A semi-automatic process, that helps the users to tag a video sequence would improve the quality of annotations and thus the overall user experience. While research on image tagging has received a considerable attention in the latest years, there are still very few works that address the problem of automatically assigning tags to videos, locating them temporally within the video sequence. In this paper we present a system for video tag suggestion and temporal localization based on collective knowledge and visual similarity of frames. The algorithm suggests new tags that can be associated to a given keyframe exploiting the tags associated to videos and images uploaded to social sites like YouTube and Flickr and visual features.
- S. Choudhury, J. Breslin, and A. Passant. Enrichment and ranking of the YouTube tag space and integration with the linked data cloud. In Proc. of International Semantic Web Conference (ISWC), 2009. Google Scholar
Digital Library
- M. Guillaumin, T. Mensink, J. Verbeek, and C. Schmid. Tagprop: Discriminative metric learning in nearest neighbor models for image auto-annotation. In Proc. of ICCV, 2009.Google ScholarCross Ref
- L. S. Kennedy, S.-F. Chang, and I. V. Kozintsev. To search or to label? Predicting the performance of search-based automatic image classifiers. In Proc. of ACM MIR, 2006. Google ScholarDigital Library
- L. S. Kennedy, M. Slaney, and K. Weinberger. Reliable tags using image similarity. In Proc. of ACM MM Workshop on Web-Scale Multimedia Corpus, Beijing, China, 2009. Google ScholarDigital Library
- X. Li, C. Snoek, and M. Worring. Learning tag relevance by neighbor voting for social image retrieval. In Proc. of ACM MIR, 2008. Google ScholarDigital Library
- X. Li, C. Snoek, and M. Worring. Unsupervised multi-feature tag relevance learning for social image retrieval. In Proc. of ACM CIVR, 2010. Google ScholarDigital Library
- X. Li, C. G. M. Snoek, and M. Worring. Learning social tag relevance by neighbor voting. IEEE Transactions on Multimedia, 11(7):1310--1322, 2009. Google ScholarDigital Library
- D. Liu, X.-S. Hua, L. Yang, M. Wang, and H.-J. Zhang. Tag ranking. In Proc. of International World Wide Web Conference (WWW), 2009. Google ScholarDigital Library
- Y. Liu and N. Yu. Dual linkage refinement for YouTube video topic discovery. In Proc. of IEEE ICME, 2010.Google ScholarCross Ref
- S. G. Sevil, O. Kucuktunc, P. Duygulu, and F. Can. Automatic tag expansion using visual similarity for photo sharing websites. Multimedia Tools and Applications, 49(1):81--99, 2009. Google ScholarDigital Library
- S. Siersdorfer, J. San Pedro, and M. Sanderson. Automatic video tagging using content redundancy. In Proc. of ACM SIGIR, pages 395--402, New York, NY, USA, 2009. Google ScholarDigital Library
- B. Sigurbjörnsson and R. van Zwol. Flickr tag recommendation based on collective knowledge. In Proc. of International World Wide Web Conference (WWW), 2008. Google ScholarDigital Library
- H.-K. Tan, C.-W. Ngo, R. Hong, and T.-S. Chua. Scalable detection of partial near-duplicate videos by visual-temporal consistency. In Proc. of ACM Multimedia, pages 145--154, 2009. Google ScholarDigital Library
- L. von Ahn and L. Dabbish. Labeling images with a computer game. In Proc. of ACM Conference on Human Factors in Computing Systems, 2004. Google ScholarDigital Library
- C. Wang, F. Jing, L. Zhang, and H.-J. Zhang. Scalable search-based image annotation of personal images. In Proc. of ACM MIR, pages 269--278, New York, NY, USA, 2006. Google ScholarDigital Library
- L. Wu, L. Yang, N. Yu, and X.-S. Hua. Learning to tag. In Proc. of International World Wide Web Conference (WWW), 2009. Google ScholarDigital Library
- X. Wu, A. Hauptmann, and C.-W. Ngo. Practical elimination of near-duplicates from web video search. In Proc. of ACM Multimedia, pages 218--227, 2007. Google ScholarDigital Library
- X. Wu, C.-W. Ngo, A. G. Hauptmann, and H.-K. Tan. Real-time near-duplicate elimination for web video search with content and context. IEEE Transactions on Multimedia, 11(2):196--207, 2009. Google ScholarDigital Library
- X. Wu, W.-L. Zhao, and C.-W. Ngo. Towards Google challenge: Combining contextual and social information for web video categorization. In Proc. of ACM Multimedia, 2009. Google ScholarDigital Library
- W. Zhao, X. Wu, and C. Ngo. On the annotation of web videos by efficient near-duplicate search. IEEE Transactions on Multimedia, to appear in 2010. Google ScholarDigital Library
Index Terms
- Tag suggestion and localization in user-generated videos based on social knowledge
Recommendations
Enriching and localizing semantic tags in internet videos
MM '11: Proceedings of the 19th ACM international conference on MultimediaTagging of multimedia content is becoming more and more widespread as web 2.0 sites, like Flickr and Facebook for images, YouTube and Vimeo for videos, have popularized tagging functionalities among their users. These user-generated tags are used to ...
Tag suggestion using visual content and social tag
ICUIMC '11: Proceedings of the 5th International Conference on Ubiquitous Information Management and CommunicationWith the popularity of social media sharing sites such as Flickr or YouTube, tagging has become a more important task to describe the content of the multimedia object. Recently, automatic tagging or tag recommendation has studied to automatically provide ...
Estimating translation probabilities for social tag suggestion
We present a new perspective to tag suggestion and treat it as a translation process.We propose two methods to estimate the translation probabilities.Our methods can solve the problem of vocabulary gap.Our methods are effective and robust compared with ...
Comments