Relevance feedback for content-based image retrieval: what can three mouse clicks achieve?

We introduce a novel relevance feedback method for content-based image retrieval and demonstrate its effectiveness using a subset of the Corel Gallery photograph collection and five low-level colour descriptors. Relevance information is translated into updated, analytically computed descriptor weights and a new query representation, and thus the system combines movement in both query and weight space. To assess the effectiveness of relevance feedback, we first determine the weight set that is optimal on average for a range of possible queries. The resulting multiple-descriptor retrieval model yields significant performance gains over all the single-descriptor models and provides the benchmark against which we measure the additional improvement through relevance feed-back. We model a number of scenarios of user-system interaction that differ with respect to the precise type and the extent of relevance feedback. In all scenarios, relevance feedback leads to a significant improvement of retrieval performance suggesting that feedback-induced performance gain is a robust phenomenon. Based on a comparison of the different scenarios, we identify optimal interaction models that yield high performance gains at a low operational cost for the user. To support the proposed relevant feedback technique we developed a novel presentation paradigm that allows relevance to be treated as a continuous variable.