Skip to main content
SpringerLink
Log in

Recognizing context for annotating a live life recording

  • Original Article
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

In the near future, it will be possible to continuously record and store the entire audio–visual lifetime of a person together with all digital information that the person perceives or creates. While the storage of this data will be possible soon, retrieval and indexing into such large data sets are unsolved challenges. Since today’s retrieval cues seem insufficient we argue that additional cues, obtained from body-worn sensors, make associative retrieval by humans possible. We present three approaches to create such cues, each along with an experimental evaluation: the user’s physical activity from acceleration sensors, his social environment from audio sensors, and his interruptibility from multiple sensors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. Assuming a lifespan of 100 years, 24 h recording per day, and 10 MB per min recording results in approximately 500 TB.

References

  1. Lyman P, Varian HR, Swearingen K, Charles P, Good N, Jordan L, Pal J (2003) How much information 2003? Technical report, UC Berkeley. http://www.sims.berkeley.edu/research/projects/how-much-info-2003/.

  2. Mann S (2004) Continuous lifelong capture of personal experience with EyeTap. In: Proceedings of 1st ACM workshop on continuous archival and retrieval of personal experiences

  3. Aizawa K, Tancharoen D, Kawasaki S, Yamasaki T (2004) Efficient retrieval of life log based on context and content. In: Proceedings of 1st ACM workshop on continuous archival and retrieval of personal experiences, pp 22–31, 2004

  4. Schmidt A, Beigl M, Gellersen H-W (1999) There is more to context than location. Comput Graph J 23:893–902

    Article  Google Scholar 

  5. Patel SN, Abowd G (2004) The contextcam: automated point of capture video annotation. In: Proceedings of Ubicomp, Nottinham, United Kingdom. Lecture notes in computer science, vol 3205, pp 301–318. Springer Berlin Heidelberg New York, October 2004.

  6. Holleis P, Kranz M, Gall M, Schmidt A (2005) Adding context information to digital photos. In: Proceedings of the 5th international workshop on smart appliances and wearable computing (IWSAWC), Columbus, Ohio, United States, June 2005

  7. Kern N, Schiele B (2003) Context-aware notification for wearable computing. In: Proceedings of ISWC, White Plains, NY, USA, pp 223–230, October 2003

  8. Schiele B, Oliver N, Jebara T, Pentland A (1999) An interactive computer vision system, DyPERS: dynamic personal enhanced remembrance system. In: Proceedings of ICVS, pp 51–65, January 1999

  9. Kern N, Schiele B, Junker H, Lukowicz P, Tröster G (2002) Wearable sensing to annotate meetings recordings. In: Proceedings of ISWC, Seattle, WA, USA, 186–193, October 2002

  10. Farringdon J, Moore A, Tilbury N, Church J, Biemond P (1999) Wearable sensor badge & sensor jacket for context awareness. In: Proceedings of ISWC, San Francisco, pp 107–113, 1999

  11. Randell C, Muller H (2000) Context awareness by analysing accelerometer data. In: Proceedings of ISWC, Atlanta, GA, USA, pp 175–176, October 2000

  12. Mäntyjärvi J, Himberg J, Seppanen T (2001) Recognizing human motion with multiple acceleration sensors. In: Proceedings of systems, man and cybernetics, pp 747–752, 2001

  13. Kern N, Schiele B, Schmidt A (2003) Multi-sensor activity context detection for wearable computing. In: Proceedings of EUSAI, LNCS, vol 2875, Eindhoven, The Netherlandsm, pp 220–232, November 2003

  14. Bao L, Intille S (2004) Activity recognition from user-annotated acceleration data. In: Proceedings of pervasive, Vienna, Austria. Lecture notes in computer science, vol 3001, pp 1–17. Springer, Berlin Heidelberg New York, April 2004

  15. van Laerhoven K, Aido K, Lowette S (2001) Real-time analysis of data from many sensors with neural networks. In: Proceedings of ISWC, Zurich, Switzerland, pp 115–123, 2001

  16. van Laerhoven K, Schmidt A, Gellersen H-W (2002) Multi-sensor context-aware clothing. In: Proceedings of ISWC, Seattle, USA, pp 49–57, October 2002

  17. van Laerhoven K, Gellersen H-W (2004) Spine vs. porcupine: a study in distributed wearable activity recognition. In: Proceedings of ISWC, Arlington, USA, pp 142–149, November 2004

  18. Degen Th, Jaeckel H, Rufer M, Wyss S (2003) Speedy: a fall detector in a wrist watch. In: Proceedings of ISWC, White Plains, NY, USA, pp 184–188, October 2003

  19. Akay M, Sekine M, Tamura T, Higashi Y, Fujimoto T (2003) Unconstrained monitoring of body motion during walking. IEEE Eng Med Biol Magazine 22(3):104–109

    Article  Google Scholar 

  20. Schmidt A (2002) Ubiquitous computing—computing in context. Ph.D. thesis, University of Lancaster

  21. Beigl M, Zimmer T, Krohn A, Decker C, Robinson P (2003) Smart-its - communication and sensing technology for UbiComp environments. Technical report, TeCo, University Karlsruhe, Germany

  22. Peltonen V, Tuomi J, Klapuri A, Huopaniemi J, Sorsa T (2002) Computational auditory scene recognition. In: Proceedings of ICASSP, pp 1941–1944, 2002

  23. Clarkson B, Sawhney N, Pentland A (1998) Auditory context awareness via wearable computing. In: Workshop on perceptual user interfaces (PUI), November 1998

  24. Büchler M (2002) Algorithms for sound classification in hearing instruments. Ph.D. thesis, ETH Zurich. Diss. No. 14498, 2002

  25. Korpipää P, Koskinen M, Peltola J, Mäkelä S-M, Seppänen T (2003) Bayesian approach to sensor-based context awareness. Pers Ubiquitous Comput 7(2):113–124

    Article  Google Scholar 

  26. Lukowicz P, Ward J, Junker H, Stäger M, Tröster G, Atrash A, Starner T (2004) Recognizing workshop activity using body-worn microphones and accelerometers. In: Proceedings of pervasive, Vienna, Austria. Lecture notes in computer science, vol 3001, pp 18–32. Springer, Berlin Heidelberg New York, April 2004

  27. Kern N, Antifakos S, Schiele B, Schwaninger A (2004) A model of human interruptability: experimental evaluation and automatic estimation from wearable sensors. In: Proceedings of ISWC, Washington, DC, USA, pp 158–165, November 2004

  28. Cutrell M, Czerwinski M, Horvitz E (2001) Notification, disruption, and memory: effects of messaging interruptions on memory and performance. In: Proceedings of Interact, pp 263–269, 2001

  29. Hudson JM, Christensen J, Kellogg WA, Erickson T (2002) ‘I’d be overwhelmed, but it’s just one more thing to do’: availability and interruption in research management. In: Proceedings of ACM CHI, ACM Press, pp 97–104, 2002

  30. Obermayer RW, Nugent WA (2000) Human-computer interaction (HCI) for alert warning and attention allocation systems for the multi–modal watchstation (MMWS). In: Proceedings of SPIE, vol 4126, 2000

  31. Fogarty J, Hudson SE, Lai J (2004) Examining the robustness of sensor-based statistical models of human interruptability. In: Proceedings of ACM CHI, pp 207–214, 2004

  32. Scott McCrickard D, Richard Catrambone CM, Chewar Stasko JT (2003) Establishing tradeoffs that leverage attention for utility: empirically evaluating information display in notification systems. Int J Hum Comput Stud 8(5):547–582

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Darmstadt University of Technology, Darmstadt, Germany

    Nicky Kern & Bernt Schiele

  2. Embedded Interaction Research Group, Universität München, München, Germany

    Albrecht Schmidt

Authors
  1. Nicky Kern
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bernt Schiele
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Albrecht Schmidt
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nicky Kern.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kern, N., Schiele, B. & Schmidt, A. Recognizing context for annotating a live life recording. Pers Ubiquit Comput 11, 251–263 (2007). https://doi.org/10.1007/s00779-006-0086-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-006-0086-3

Keywords

Search

Navigation