Mobile Lifelogger – Recording, Indexing, and Understanding a Mobile User’s Life

Lifelog system involves capturing personal experiences in the form of digital multimedia during an entire lifespan. Recent advancements in mobile sensor technologies have helped to develop these systems using commercial smart phones. These systems have the potential to act as a secondary memory and also aid people who struggle with episodic memory impairment (EMI). Despite their huge potential, there are major challenges that need to be addressed to make them useful. One of them is how to index the inherently large lifelog data so that the person can efficiently retrieve the log segments that interest him / her most. In this paper, we present an ongoing research of using mobile phones to record and index lifelogs using activity language. By converting sensory data such as accelerometer and GPS readings into activity language, we are able to apply statistical natural language processing techniques to index, recognize, segment, cluster, retrieve, and infer high-level semantic meanings of the collected lifelogs. Based on this indexing approach, our lifelog system supports easy retrieval of log segments representing past similar activities and automatic lifelog segmentation for efficient browsing and activity summarization.