Toward Unification of Keywords and Low-Level Contents

The performance of a image retrieval system is inherently constrained by the use of the low-level features and cannot give satisfactory retrieval results in many cases, especially when the high-level concepts in the user’s mind are not easily expressible in terms of the low-level features. Therefore, for real world applications, whenever possible, textual annotations shall be added or extracted and/or processed to improve the retrieval performance. In this part we explore the unification of keywords and feature contents for image retrieval. We propose a seamless joint querying and relevance feedback scheme based on both keywords and low-level feature contents incorporating keyword similarities. We propose a WARF (word association via relevance feedback) formula as a pseudoclassification algorithm for the learning of the term similarity matrix during user interaction. This learned similarity matrix, specific to the dataset as well as the users, can be applied for keyword semantic grouping, thesaurus construction, and soft query expansion during intelligent image retrieval with user-in-the-loop.