Ramesh Jain
ABSTRACT
We revisit one of the most fundamental problems in multimedia that is receiving enormous attention from researchers without making much progress in solving it: the problem of bridging the semantic gap. Research in this area has focused on developing increasingly rigorous techniques using the content. Researchers consider that Content is King and ignore everything else. In this paper, first we will discuss how this infatuation with content continues to be the biggest hurdle in the success of, ironically, content based approaches for multimedia search. Lately, many commercial systems have ignored content in favor of context and demonstrated better success. Given that the mobile phones are the major platform for the next generation of computing, context becomes easily available and more relevant. We show that it is not Content Versus Context; rather it is Content and Context that is required to bridge the semantic gap. In this paper, first we will discuss reasons for our approach and then present approaches that appropriately combine context with content to help bridge the semantic gap and solve important problems in multimedia computing.
- Google Research Blog: http://googleresearch.blogspot.com/2010/04/lessons-learned-developing-practical.html.Google Scholar
- Science Daily Article: http://bit.ly/QHiJh.Google Scholar
- T. Ahonen, A. Hadid, et al. Face description with local binary patterns: Application to face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 2037--2041, 2006. Google ScholarDigital Library
- H. Andersen and V. Burton. The emperor's new clothes. Sandpiper, 2004.Google Scholar
- H. Barrow and J. Tenenbaum. MSYS: A system for reasoning about scenes. Technical Note, AI Center SRI, 121, 1976.Google Scholar
- G. Berkeley and C. Krauth. A treatise concerning the principles of human knowledge. JB Lippincott & co., 1878.Google Scholar
- G. Bradski, A. Kaehler, and V. Pisarevsky. Learning-based computer vision with Intel's open source computer vision library. Intel Technology Journal, 9(2):119--130, 2005.Google Scholar
- C. Desai, D. Ramanan, and C. Fowlkes. Discriminative models for multi-class object layout. In Proc. ICCV, 2009.Google ScholarCross Ref
- S. Divvala, D. Hoiem, J. Hays, A. Efros, and M. Hebert. An empirical study of context in object detection. 2009.Google Scholar
- R. Duda, P. Hart, and D. Stork. Pattern classification. Citeseer, 1971.Google Scholar
- M. Everingham, L. Van Gool, C. Williams, J. Winn, and A. Zisserman. The PASCAL Visual Object Classes Challenge 2009 (VOC2009) Results. 2010.Google Scholar
- J. Exner. The Rorschach: A comprehensive system. John Wiley & Sons.Google Scholar
- E. Freuder. Recognition of Real Objects. Vision Flash, No. 33, 1972.Google Scholar
- C. Galleguillos and S. Belongie. Context based object categorization: A critical survey. 2009.Google Scholar
- R. Gregory. The intelligent eye. 1970.Google Scholar
- J. Hays and A. Efros. IM2GPS: estimating geographic information from a single image. In IEEE Conference on Computer Vision and Pattern Recognition, 2008. CVPR 2008, pages 1--8, 2008.Google ScholarCross Ref
- D. Hoiem, A. Efros, and M. Hebert. Putting objects in perspective. International Journal of Computer Vision, 80(1):3--15, 2008. Google ScholarDigital Library
- Japan Electronics and Information Technology Industries Assoc. 2.2 Digital Still Camera Image File Format Standard (Exif) Version 2.2 .Google Scholar
- Y. Joung, M. El Zarki, and R. Jain. A user model for personalization services. In Fourth International Conference on Digital Information Management, 2009. ICDIM 2009, pages 1--6, 2009.Google ScholarCross Ref
- J. Kittler and J. Illingworth. Relaxation labelling algorithms - a review. Image and Vision Computing, 3(4):206--216, 1985. Google ScholarDigital Library
- U. Neisser. Cognitive psychology. Appleton-Century-Crofts New York, 1967.Google Scholar
- K. Popper and S. Popper. Objective knowledge: An evolutionary approach. Clarendon Press, 1979.Google Scholar
- V. Ramachandran and S. Anstis. The perception of apparent motion. Scientific American, 254(6):102--109, 1986.Google ScholarCross Ref
- I. Rock. The logic of perception. 1983.Google Scholar
- S. Santini and R. Jain. Beyond query by example. In Proceedings of the sixth ACM international conference on Multimedia, page 350. ACM, 1998. Google ScholarDigital Library
- C. Shirky. Cognitive Surplus: Creativity and Generosity in a Connected Age. Penguin Press, 2010.Google Scholar
- P. Sinha and R. Jain. Classification and annotation of digital photos using optical context data. In Proceedings of the 2008 international conference on Content-based image and video retrieval, pages 309--318. ACM, 2008. Google ScholarDigital Library
- P. Sinha and R. Jain. Semantics In Digital Photos: A Contenxtual Analysis. In Proceedings of the 2008 IEEE International Conference on Semantic Computing, pages 58--65. IEEE Computer Society, 2008. Google ScholarDigital Library
- A. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain. Content-based image retrieval at the end of the early years. IEEE Transactions on pattern analysis and machine intelligence, 22(12):1349--1380, 2000. Google ScholarDigital Library
- A. Torralba. Contextual priming for object detection. International Journal of Computer Vision, 53(2):169--191, 2003. Google ScholarDigital Library
- D. Waltz and P. Winston. The psychology of computer vision, 1975.Google Scholar
Index Terms
- Content without context is meaningless
Recommendations
Context-based ranking in folksonomies
HT '09: Proceedings of the 20th ACM conference on Hypertext and hypermediaWith the advent of Web 2.0 tagging became a popular feature. People tag diverse kinds of content, e.g. products at Amazon, music at Last.fm, images at Flickr, etc. Clicking on a tag enables the users to explore related content. In this paper we ...
Searching for Important But Neglected Content from Community-Type-Content
SITIS '08: Proceedings of the 2008 IEEE International Conference on Signal Image Technology and Internet Based SystemsIn online community-type content such as SNSs and blogs, users occasionally do not understand the theme of the content from multiple viewpoints, and much of the information is often lost. Because the discussion in a community is concentrated, the ...
GAIUS: a new mobile content creation and diffusion ecosystem for emerging regions
ICTD '19: Proceedings of the Tenth International Conference on Information and Communication Technologies and DevelopmentDespite increasing mobile Internet penetration in emerging regions, the growth of web page complexity combined with inadequate content delivery mechanisms and lack of relevant local content make the web experience poor for users in these regions. In ...
Comments