Skip to main content
Top

2018 | Book

Personal Multimedia Preservation

Remembering or Forgetting Images and Video

insite
SEARCH

About this book

This unique text/reference advocates a novel forgetful approach to dealing with personal multimedia content in the long run, which is inspired by the effectiveness of human forgetting as a mechanism for helping us to stay focused on important things. The text presents theoretical foundations, technologies, applications, and case study results that help the reader to understand the problems and challenges associated with personal digital preservation, and the solutions that can be developed in response to these challenges.

Topics and features: Highlights the value of an intelligent and selective approach to personal multimedia preservation, involving managed forgetting and contextualized remembering; considers how a conceptual understanding of human memory function can be used to inspire the design of digital managed preservation and forgetting; discusses methods that endow computers with capabilities to understand digital content, in order to support intelligent preservation decisions; examines the assessment of the importance of information items, introducing the concepts of memory buoyancy and preservation value; reviews methods for preserving the context associated with a digital item, and for assessing how this context evolves over time; proposes a reference model for the Preserve-or-Forget (PoF) approach which is integrative, value-driven, brain-inspired, forgetful, and evolution-aware; describes the integration of preservation functionalities in a Personal Information Management (PIM) application; presents a user study on a photo selection task, using the results to design methods for automatically selecting important photos from personal collections.

This interdisciplinary volume provides significant insights from computer science, engineering and psychology that will be of great interest to researchers involved in multimedia and software engineering, multimedia preservation, cultural informatics, digital heritage, and the digital humanities.

Table of Contents

Frontmatter

Interdisciplinary Foundations

Frontmatter
Chapter 1. Multimedia Preservation: Why Bother?
Abstract
Multimedia content and especially personal multimedia content is created in abundance today. Short- to mid-term storage of this content is typically no problem due to decreased storage prices and the availability of storage services. However, for the long-term perspective, i.e., preservation, adequate technologies and best practices for keeping the content accessible and meaningful are still missing. Instead, the breakdown of devices and changes in technologies lead to some form of random survival and random forgetting for digital content. In this chapter, we motivate a more intelligent and selective approach to personal multimedia preservation. This approach introduces and combines three key building blocks: (a) “managed forgetting” for focusing on important and useful content inspired by human forgetting and remembering; (b) “contextualized remembering” for dealing with evolution and keeping content meaningful over time; and (c) “synergetic preservation” for bridging the gap between active information use and long-term information management.
Claudia Niederée, Vasileios Mezaris, Heiko Maus, Robert H. Logie
Chapter 2. Preserving and Forgetting in the Human Brain
Abstract
Humans have evolved to be very efficient at managed preservation of what is necessary to preserve. Humans are also extremely efficient at forgetting trivial or irrelevant details when they are no longer needed. Indeed, managed preservation and forgetting could be viewed as a set of human ‘superpowers’ achieved through use of a lifetime of accumulated knowledge, highly effective contextualisation, aggregation, organisation, summarisation and reconstruction of key features of experiences. But humans are poor at preservation of large amounts of detail. Typically, memories are partially reconstructed during the retrieval process, and this reconstruction process can sometimes lead to false memories. Many of these strengths and limitations of human memory are well understood by human memory researchers, although important questions and uncertainties remain. In complete contrast, digital systems excel in preserving large amounts of detail, and are getting better at contextualisation. But they remain rather poor at systematic forgetting of irrelevant detail. Often, digital forgetting occurs by accident through disk crashes, incompatible upgrades of software and hardware, lost or stolen storage devices. Even if the data are still present and safely stored, insufficient indexing and poor information retrieval may result in those data effectively being forgotten. This chapter will provide a detailed overview of the state of the science of human memory, based on empirical studies and conceptual modelling. It will discuss how the human superpowers of managed preservation and forgetting are achieved, and show how a conceptual understanding of human memory function could be used to inspire the design of digital managed preservation and forgetting. It will argue that human-inspired digital forgetting is key for achieving a truly synergetic relationship between human and digital memory, and explore how such a synergetic relationship can address aspects of the paradox that massive investment in technology has not necessarily led to the expected increase in productivity (IT/productivity paradox). Next, we will describe an in-depth study of personal, digital photograph collections that were contributed by volunteer research participants. This study explored human management of photographic collections and contrasted it with managed preservation and forgetting of the same photo collection by an example digital system that incorporates automated conceptualisation and forgetting. The chapter will conclude with a summary of how understanding human cognitive function can help to inspire more useful digital storage systems that offer reliable and usable tools to complement and support human memory rather than attempt to replace it.
Robert H. Logie, Maria Wolters, Elaine Niven
Chapter 3. Multimedia Processing Essentials
Abstract
As multimedia applications have become part of our life, preservation and long-term access to the multimedia elements that are continuously produced is a major consideration, both for many organizations that generate or collect and need to maintain digital content, and for individuals. In this chapter, we focus primarily on the following multimedia analysis and organization techniques that provide the basis for multimedia preservation: (a) photo/video annotation, which refers to the problem of assigning one or more semantic concepts to photos or video fragments, (b) photo/video quality assessment, which refers to the automatic prediction of a photo’s or video’s aesthetic value, (c) near-duplicate detection, which aims to identify groups of very similar items in large media collections, and (d) event-based photo clustering and summarization, which concern the selection of the most characteristic photos of a photo collection so as to create a storyline that conveys the gist of this collection.
Konstantinos Apostolidis, Foteini Markatopoulou, Christos Tzelepis, Vasileios Mezaris, Ioannis Patras

Multimedia Preservation Theory

Frontmatter
Chapter 4. Preservation Value and Managed Forgetting
Abstract
In the cognitive processes of humans, forgetting is a very effective way for focusing on the important things, while unstressing things, which are (currently) less important. The translation of forgetting into the digital world is, thus, a promising approach for better dealing with the increasing problem of information overload. Information overload is not only caused by the mere volume of information, it is also triggered by the fact that all information is seemingly on the same level of importance. In the ideal case, a perfect dynamic assessment of importance could restrict the information space strictly to the information currently needed, thus dramatically reducing information overload. The role of a digital memory including digital forgetting is to support, not to replace or to hinder human memory. Therefore, a useful approach for “managed forgetting”—a controlled form of digital forgetting—in a digital memory has to be carefully designed, such that it complements human memory. A core ingredient of managed forgetting is the assessment of the importance of information items. Furthermore, forgetting actions are required that go beyond the binary decision between keep and delete. For the short-term perspective, managed forgetting replaces the binary decision on importance by a gradually changing value: Information sinks away from the user with a decreasing value, which we call “Memory Buoyancy”. The transition from short-term value to long-term importance brings a variety of new challenges. When we look into the “Preservation Value” of an information item, we have to estimate the future importance of a resource. This challenging task is further complicated by the facts that (a) preservation looks into very long time frames (e.g., decades rather than months) and (b) the importance of information items may change over time. The Preservation Value provides the basis for making preservation decisions, e.g., how much to invest for ensuring that a media item such as a photo will survive the next years or decades. We, furthermore, investigate into methods for information value assessment in support of managed forgetting. For this purpose, we analyze existing methods for information value assessment and discuss their usefulness in the context of computing the Preservation Value. We also outline methods for Preservation Value computation for different exemplary settings. This will also point to the more in-depth discussion of computing Preservation Value in the semantic desktop and photo preservation as it is discussed in later chapters of the book. Finally, we discuss managed forgetting beyond assessing the importance of information items. We study a portfolio of forgetting methods, i.e., methods that can be used to implement managed forgetting on top of the values for information importance. This includes methods such as information hiding, forgetful search, summarization and aggregation as well as deletion.
Claudia Niederée, Nattiya Kanhabua, Tuan Tran, Kaweh Djafari Naini
Chapter 5. Keeping Information in Context
Abstract
Without context, words have no meaning, and the same is true for documents, in that often a wider context is required to fully interpret the information they contain. For example, a family photo is practically useless if you do not know who the people portrayed in it are, and likewise, a document that refers to the president of the US is of little use without knowing who held the job at the time the document was written. This becomes even more important when considering the long-term preservation of documents, as not only is human memory fallible, but over long periods the people accessing the documents will change (e.g. photos passed down through generations), as will their understanding and knowledge of the world. While preserving the context associated with a document is an important first step in ensuring information remains useful over long periods of time, we also need to consider how information evolves. Over any significant time period, the meaning of information changes. This evolution can range from changes in the meaning of individual words to more general terms or concepts, such as who holds a specific position in an organization. In this chapter, we look in detail at all of these challenges and describe the development of a conceptual framework in which context information can be collected, preserved, evolved and used to access and interpret documents. A number of techniques are presented showing real examples of context in action that fit within the framework, and applying to both text documents and image collections.
Mark A. Greenwood, Nam Khanh Tran, Konstantinos Apostolidis, Vasileios Mezaris
Chapter 6. Bridging Information Management and Preservation: A Reference Model
Abstract
The idea of the Preserve-or-Forget (PoF) approach introduced in this book is to follow a forgetful, focused approach to digital preservation, which is inspired by human forgetting and remembering. Its goal is to ease the adoption of preservation technology especially in the personal and organizational context and to ensure that important content is kept safe, useful, and understandable in the long run. For this purpose, it stresses the smooth interaction between information management and preservation management. Leveraging the PoF approach, in this chapter we introduce a reference model, which will be referred to in the following as PoF Reference Model. The model pays special attention to the functionality which bridges between Information Management System (Active System) and Digital Preservation System (DPS), such as the selection of content for preservation and the transfer of content between the systems. The model aims to encapsulate the core ideas of the PoF approach, which considers Active System and DPS as a joint ecosystem into a re-usable model, and is inspired by the core principles of this approach: synergetic preservation, managed forgetting, and contextualized remembering. The design of the PoF Reference Model was driven by the identification of five required characteristics for such a reference model: it has to be integrative, value-driven, brain-inspired, forgetful, and evolution-aware. The PoF Reference Model consists of a functional part (Functional Model) and of an associated Information Model. The Functional Model is made up of three layers: Core Layer, Remember and Forget Layer, and Evolution Layer. For each layer, we discuss the main functional entities and the representative workflows, also relating them to existing standards and practices in digital preservation. The functionality required to mediate between the Active System and the DPS has been encapsulated into the PoF Middleware, which has been designed and implemented as part of the ForgetIT project. The Information Model describes the preservation entities and their relationships, also discussing the interoperability with existing digital preservation standards.
Francesco Gallo, Claudia Niederée, Walter Allasia

Multimedia Preservation in Practice

Frontmatter
Chapter 7. Remembering and Forgetting for Personal Preservation
Abstract
Chapter 1 detailed the need for preservation especially for our personal life. Even so, it is not part of most users’ regular practice. Preservation challenges users with a manual effort and a somewhat stern discipline to continuously perform the required steps. Being aware of these challenges, the chapter will present the novel approach realized in ForgetIT to embed support for personal preservation in users’ daily activities of Personal Information Management (PIM), reducing the effort and users’ cognitive burden for preservation. In this approach, the previously introduced concepts of remembering and forgetting play a major role for a successful realization and for providing benefits to users. It is a method rooted in artificial intelligence research that allows us to derive and represent a user’s mental model and making data machine understandable. This “semantification” of the user’s resources paves the way for more effective functionalities for automated preservation, forgetting, and remembering embedded in the user’s daily activities and applications. The chapter will detail a pilot based on this ForgetIT approach realizing remembering and forgetting for preservation. We further investigate how activities from PIM provide context to these. Regarding remembering, we present applications allowing a semantic photo organization and providing a self-generating diary to remember past events. Considering forgetting, we investigate how forgetting functionalities can be embedded in applications and present how different variants for forgetting are used in the pilot. Finally, a comprehensive approach for personal preservation is presented utilizing the PoF Framework introduced in Chap. 6 and overcoming the obstacles for personal preservation. The chapter concludes with a discussion of experience of using the pilot in daily work of our research group.
Heiko Maus, Christian Jilek, Sven Schwarz
Chapter 8. Personal Photo Management and Preservation
Abstract
Thanks to the spread of digital photography and available devices, taking photographs has become effortless and tolerated nearly everywhere. This makes people easily ending up with hundreds or thousands of photos, for example, when returning from a holiday trip or taking part in ceremonies, concerts, and other events. Furthermore, photos are also taken of more mundane motives, such as food and aspects of everyday life, further increasing the number of photos to be dealt with. The decreased prices of storage devices make dumping the whole set of photos common and affordable. However, this practice frequently makes the stored collections a kind of dark archives, which are rarely accessed and enjoyed again in the future. The big size of the collections makes revisiting them time demanding. This suggests to identify, with the support of automated methods, the sets of most important photos within the whole collections and to invest some preservation effort for keeping them accessible over time. Evaluating the importance of photos to their owners is a complex process, which is often driven by personal attachment, memories behind the content, and personal tastes that are difficult to capture automatically. Therefore, to better understand the selection process for photo preservation and future revisiting, the first part of this chapter presents a user study on a photo selection task where participants selected subsets of most important pictures from their own collections. In the second part of this chapter, we present methods to automatically select important photos from personal collections, in light of the insights emerged from the user study. We model a notion of photo importance driven by user expectations, which represents what photos users perceive as important and would have selected. We present an expectation-oriented method for photo selection, where information at both photo and collection levels is considered to predict the importance of photos.
Andrea Ceroni
Backmatter
Metadata
Title
Personal Multimedia Preservation
Editors
Dr. Vasileios Mezaris
Dr. Claudia Niederée
Prof. Robert H. Logie
Copyright Year
2018
Electronic ISBN
978-3-319-73465-1
Print ISBN
978-3-319-73464-4
DOI
https://doi.org/10.1007/978-3-319-73465-1