Top

International Journal of Multimedia Information Retrieval

Published in:

01-07-2012 | Regular Paper

Semantics-based selection of everyday concepts in visual lifelogging

Authors: Peng Wang, Alan F. Smeaton

Published in: International Journal of Multimedia Information Retrieval | Issue 2/2012

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Concept-based indexing, based on identifying various semantic concepts appearing in multimedia, is an attractive option for multimedia retrieval and much research tries to bridge the semantic gap between the media’s low-level features and high-level semantics. Research into concept-based multimedia retrieval has generally focussed on detecting concepts from high-quality media such as broadcast TV or movies, but it is not well addressed in other domains like lifelogging where the original data is captured with poorer quality. We argue that in noisy domains such as lifelogging, the management of data needs to include semantic reasoning in order to deduce a set of concepts to represent lifelog content for applications like searching, browsing or summarization. Using semantic concepts to manage lifelog data relies on the fusion of automatically detected concepts to provide a better understanding of the lifelog data. In this paper, we investigate the selection of semantic concepts for lifelogging which includes reasoning on semantic networks using a density-based approach. In a series of experiments we compare different semantic reasoning approaches and the experimental evaluations we report on lifelog data show the efficacy of our approach.

previous article Interactive search in image retrieval: a survey

next article An efficient framework for location-based scene matching in image databases

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

http://lastlaugh.inf.cs.cmu.edu/lscom/. Accessed 26 Nov 2011

http://www.statistics.gov.uk/statbase/ssdataset.asp?vlnk=7038&more=y. Accessed 6 Sep 2010

http://www.statistics.gov.uk/statbase/ssdataset.asp?vlnk=9497&pos=&colrank=1&rank=272. Accessed 6 Sep 2010

Banerjee S, Pedersen T (2002) An adapted Lesk algorithm for word sense disambiguation using WordNet. In: CICLing ’02: Proceedings of the third international conference on computational linguistics and intelligent text processing. Springer, London, pp 136–145

Blum M, Pentland AS, Tröster G (2006) InSense: interest-based life logging. IEEE Multimed 13(4):40–48CrossRef

Bukhin M, DelGaudio M (2006) WayMarkr: acquiring perspective through continuous documentation. In: MUM ’06: Proceedings of the 5th international conference on mobile and ubiquitous multimedia, ACM, New York, USA, NY, p 9

Bush V (1945) As we may think. The Atlantic Monthly

Byrne D, Doherty AR, Snoek CGM, Jones GJF, Smeaton AF (2010) Everyday concept detection in visual lifelogs: validation, relationships and trends. Multimed Tools Appl 49(1):119–144CrossRef

Chang SF, Hsu W, Jiang W, Kennedy L, Xu D, Yanagawa A, Zavesky E (2006) Evaluating the impact of 374 visual-based LSCOM concept detectors on automatic search. In: Proceedings of the 4th TRECVid workshop, USA, Gaithersburg

10.

Chilvers R, Corr S, Hayley S (2010) Investigation into the occupational lives of healthy older people through their use of time. Aust Occup Ther J 57(1):24–33CrossRef

11.

Christel MG, Hauptmann AG (2005) The use and utility of high-level semantic features in video retrieval. In: CIVR ’05: Proceedings of the international conference on video retrieval, Ireland, Dublin, pp 134–144

12.

Doherty AR, Caprani N, O’Conaire C, Kalnikaite V, Gurrin C, O’Connor NE, Smeaton AF (2011) Passively recognising human activities through lifelogging. Comput Hum Behav 27(5):1948–1958CrossRef

13.

Haubold A, Natsev A, Naphade M (2006) Semantic multimedia retrieval using lexical query expansion and model-based reranking. IEEE international conference on multimedia and expo, pp 1761– 1764

14.

Hauff C, Aly R, Hiemstra D (2007) The effectiveness of concept based search for video retrieval. In: Workshop information retrieval (FGIR 2007), Halle-Wittenberg, Germany, pp 205–212

15.

Hirst G, St-Onge D (1998) Lexical chains as representation of context for the detection and correction malapropisms. In: Fellbaum C (ed) WordNet: an electronic lexical database. MIT Press, Cambridge, MA

16.

Hodges S, Williams L, Berry E, Izadi S, Srinivasan J, Butler A, Smyth G, Kapur N, Wood K (2006) Sense Cam: a retrospective memory aid. In: Proceedings of 8th international conference on Ubicomp, Orange County, CA, USA, pp 177–193

17.

Hori T, Aizawa K (2003) Context-based video retrieval system for the life-log applications. In: Proceedings of the 5th ACM SIGMM international workshop on multimedia information retrieval, MIR ’03, ACM, New York, USA, NY, pp 31–38

18.

Huurnink B, Hofmann K, de Rijke M (2008) Assessing concept selection for video retrieval. In: MIR ’08: Proceeding of the 1st ACM international conference on multimedia information retrieval, ACM, New York, NY, USA, pp 459–466

19.

Kahneman D, Krueger AB, Schkade DA, Schwarz N, Stone AA (2004) A survey method for characterizing daily life experience: the day reconstruction method. Science 306(5702):1776– 1780CrossRef

20.

Lakoff G (1990) Women, fire, and dangerous things. University of Chicago Press, Chicago

21.

Leacock C, Chodorow M (1998) Combining local context and WordNet similarity for word sense identification. In: Fellbaum C (ed) WordNet: an electronic lexical, database. MIT Press, Cambridge, MA, pp 265–283

22.

Li X, Wang D, Li J, Zhang B (2007) Video search in concept subspace: a text-like paradigm. In: Proceedings of the 6th ACM international conference on image and video retrieval, CIVR ’07, ACM, New York, NY, USA, pp 603–610

23.

Liu H, Singh P (2004) Commonsense reasoning in and over natural language. In: Proceedings of the 8th international conference on knowledge-based intelligent information and engineering systems, Springer, Wellington, New Zealand

24.

Mann S (1997) Wearable computing: a first step toward personal imaging. Computer 30(2):25–32CrossRef

25.

Mann S, Fung J, Aimone C, Sehgal A, Chen D (2005) Designing EyeTap digital eyeglasses for continuous lifelong capture and sharing of personal experiences. In: Proceedings of CHI 2005 conference on computer human interaction. ACM Press, Portland, Oregon, USA

26.

Miller GA (1995) WordNet: a lexical database for English. Commun ACM 38(11):39–41CrossRef

27.

Naphade M, Smith JR, Tesic J, Chang SF, Hsu W, Kennedy L, Hauptmann A, Curtis J (2006) Large-scale concept ontology for multimedia. IEEE Multimed 13(3):86–91CrossRef

28.

Natsev AP, Haubold A, Tesic J, Xie L, Yan R (2007) Semantic concept-based query expansion and re-ranking for multimedia retrieval. In: MULTIMEDIA ’07: Proceedings of the 15th international conference on multimedia, ACM, New York, NY, USA, pp 991–1000

29.

Over P, Ianeva T, Kraaij W, Smeaton AF (2005) TRECVid 2005—an overview. In: Proceedings of TRECVid

30.

Rada R, Mili H, Bicknell E, Blettner M (1989) Development and application of a metric on semantic nets. IEEE Trans Syst Man Cybern 19(1):17–30CrossRef

31.

Reddy S, Parker A, Hyman J, Burke J, Estrin D, Hansen M (2007) Image browsing, processing, and clustering for participatory sensing: lessons from a dietsense prototype. In: EmNets’07: Proceedings of the 4th workshop on embedded networked sensors, ACM Press, Cork, Ireland, pp 13–17

32.

Resnik P (1995) Using information content to evaluate semantic similarity in a taxonomy. In: Proceedings of the 14th international joint conference on artificial intelligence, Morgan Kaufmann, San Francisco, CA, USA, pp 448–453

33.

Resnik P (1999) Semantic similarity in a taxonomy: an information-based measure and its application to problems of ambiguity in natural language. J Artif Intell Res 11:95–130MATH

34.

Richardson R, Smeaton AF (1995) Using WordNet in a knowledge-based approach to information retrieval. Tech Rep CA-0395, Dublin City University

35.

Sellen A, Fogg A, Aitken M, Hodges S, Rother C, Wood K (2007) Do life-logging technologies support memory for the past? An experimental study using Sense Cam. In: Proceedings of CHI 2007, ACM Press, New York, NY, USA, pp 81–90

36.

Sellen AJ, Whittaker S (2010) Beyond total capture: a constructive critique of lifelogging. Commun ACM 53(5):70–77CrossRef

37.

Smeaton AF, Over P, Kraaij W (2009) High-level feature detection from video in TRECVid: a 5-year retrospective of achievements. In: Divakaran A (ed) Multimedia content analysis, theory and applications. Springer, Berlin, pp 151–174

38.

Smeaton AF, Quigley I (1996) Experiments on using semantic distances between words in image caption retrieval. In: Proceedings of the 19th annual international ACM SIGIR conference on research and development in information retrieval, SIGIR ’96, ACM, New York, NY, USA, pp 174–180

39.

Snoek CGM, van Gemert JC, Gevers T, Huurnink B, Koelma DC, van Liempt M, de Rooij O, van de Sande KEA, Seinstra FJ, Smeulders AWM, Thean AHC, Veenman CJ, Worring M (2006) The MediaMill TRECVid 2006 semantic video search engine. In: Proceedings of the 4th TRECVid workshop, Gaithersburg, USA

40.

Snoek CGM, Huurnink B, Hollink L, Rijke MD, Schreiber G, Worring M (2007) Adding semantics to detectors for video retrieval. IEEE Trans Multimed 9(5):975–986CrossRef

41.

Snoek CGM, Worring M, van Gemert JC, Geusebroek JM, Smeulders AWM (2006) The challenge problem for automated detection of 101 semantic concepts in multimedia. In: Proceedings of the 14th annual ACM international conference on multimedia, MULTIMEDIA ’06, ACM, New York, NY, USA, pp 421–430

42.

Wei XY, Ngo CW (2007) Ontology-enriched semantic space for video search. In: MULTIMEDIA ’07: Proceedings of the 15th international conference on multimedia, ACM, New York, NY, USA, pp 981–990

43.

Wu Z, Palmer M (1994) Verb semantics and lexical selection. In: Proceedings of the 32nd annual meeting on Association for Computational Linguistics, Stroudsburg, PA, USA, pp 133–138

Title: Semantics-based selection of everyday concepts in visual lifelogging
Authors: Peng Wang
Alan F. Smeaton
Publication date: 01-07-2012
Publisher: Springer-Verlag
Published in: International Journal of Multimedia Information Retrieval / Issue 2/2012
Print ISSN: 2192-6611
Electronic ISSN: 2192-662X
DOI: https://doi.org/10.1007/s13735-012-0010-8

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Other articles of this Issue 2/2012

Exploiting contextual information for image re-ranking and rank aggregation

An efficient framework for location-based scene matching in image databases

Interactive search in image retrieval: a survey

Á trous gradient structure descriptor for content based image retrieval

Premium Partner