Adaptive diversification for tag-based social image retrieval

With the great popularity of social photos sharing websites, a tremendous volume of digital images is hosted together with their associated tags. Thus, extensive research efforts have been dedicated to tag-based social image search which enables users to formulate their queries using tags. However, tag queries are often ambiguous and typically short. Diversifying search results is a common solution in the absence of further knowledge about the user’s intention. Such approach aims to retrieve relevant images covering as much of the diverse meanings the query may have. However, not all queries are uniformly ambiguous and hence different diversification strategies might be suggested. In such a context, two new processes are jointly investigated at query pre-processing and post-processing levels. On the one hand, we propose a multi-view concept-based query expansion process, using a predefined list of semantic concepts, which aims to weight concepts from different views or contexts, aggregate the obtained weights and select the most representative ones using a dynamic threshold. On the other hand, we propose a new ranking process called “adaptive diverse relevance ranking” which automatically predicts an effective trade-off between relevance scores and diversity scores according to the query ambiguity level. Thorough experiments using 12 ambiguous queries over the NUS-WIDE dataset show the effectiveness of our approach versus classical uniform diversification approaches.