Bridging the Semantic Gap in Content Management Systems

With the explosion of digital media and online services, a key challenge in the area of media management is automation of content annotation, indexing, and organization for efficient media access, search, retrieval, and browsing. A major failing of current media annotation systems is the semantic gap — the incompatibility between the low-level features that can be currently computed automatically to describe media content and the high-level meaning associated with the content by users in media search and retrieval. This inevitably leads to the problem of content management systems returning media clips that are similar to one another in terms of low-level descriptions, but are completely different in terms of semantics sought by the users in their search. This chapter introduces Computational Media Aesthetics as an approach to bridging the semantic gap, outlines its foundations in media production principles, presents a computational framework to deriving high-level semantic constructs from media, and describes the structure of this collection.