Probabilistic Feature Relevance Learning for Content-Based Image Retrieval

doi:10.1006/cviu.1999.0770

Computer Vision and Image Understanding

Volume 75, Issues 1–2, July 1999, Pages 150-164

https://doi.org/10.1006/cviu.1999.0770 Get rights and content

Abstract

Most of the current image retrieval systems use “one-shot” queries to a database to retrieve similar images. Typically a K-nearest neighbor kind of algorithm is used, where weights measuring feature importance along each input dimension remain fixed (or manually tweaked by the user), in the computation of a given similarity metric. However, the similarity does not vary with equal strength or in the same proportion in all directions in the feature space emanating from the query image. The manual adjustment of these weights is time consuming and exhausting. Moreover, it requires a very sophisticated user. In this paper, we present a novel probabilistic method that enables image retrieval procedures to automatically capture feature relevance based on user's feedback and that is highly adaptive to query locations. Experimental results are presented that demonstrate the efficacy of our technique using both simulated and real-world data.

References (18)

T.P. Minka et al.
Interactive learning with a “society of models”
Pattern Recognit.
(1997)
D.W. Aha et al.
Instance-based learning algorithms
Mach. Learning
(1991)
C. Atkeson et al.
Locally weighted learning
AI Rev.
(1997)
R.H. Creecy et al.
Trading mips and memory for knowledge engineering
Comm. ACM
(1992)
R.O. Duda et al.
Pattern Classification and Scene Analysis
(1973)
M. Flickner
Query by image and video content: The QBIC system
IEEE Comput.
(September 1995)
J.H. Friedman
Flexible Metric Nearest Neighbor Classification
(1994)
T. Hastie et al.
Discriminant adaptive nearest neighbor classification
IEEE Trans. Pattern Anal. Mach. Intell.
(1996)

There are more references available in the full text version of this article.

Cited by (121)

Information fusion in content based image retrieval: A comprehensive overview
2017, Information Fusion
An ever increasing part of communication between persons involve the use of pictures, due to the cheap availability of powerful cameras on smartphones, and the cheap availability of storage space. The rising popularity of social networking applications such as Facebook, Twitter, Instagram, and of instant messaging applications, such as WhatsApp, WeChat, is the clear evidence of this phenomenon, due to the opportunity of sharing in real-time a pictorial representation of the context each individual is living in. The media rapidly exploited this phenomenon, using the same channel, either to publish their reports, or to gather additional information on an event through the community of users. While the real-time use of images is managed through metadata associated with the image (i.e., the timestamp, the geolocation, tags, etc.), their retrieval from an archive might be far from trivial, as an image bears a rich semantic content that goes beyond the description provided by its metadata. It turns out that after more than 20 years of research on Content-Based Image Retrieval (CBIR), the giant increase in the number and variety of images available in digital format is challenging the research community. It is quite easy to see that any approach aiming at facing such challenges must rely on different image representations that need to be conveniently fused in order to adapt to the subjectivity of image semantics. This paper offers a journey through the main information fusion ingredients that a recipe for the design of a CBIR system should include to meet the demanding needs of users.
A software system for automated identification and retrieval of moth images based on wing attributes
2016, Pattern Recognition
Citation Excerpt :
These systems are attractive but still present a number of problems. For example, a powerful function of CBIR is the ability to integrate user interaction where retrieval precision is adjusted according to the user provided relevance feedback (RF) information [33–36]. However, none of the existing systems has adopted the RF scheme into the retrieval framework.
Manually collecting, identifying, archiving and retrieving specimen images is an expensive and time-consuming work for entomologists. There is a clear need to introduce fast systems integrated with modern image processing and analysis algorithms to accelerate the process. In this paper, we describe the development of an automated moth species identification and retrieval system (SPIR) using computer vision and pattern recognition techniques. The core of the system is a probabilistic model that infers Semantically Related Visual (SRV) attributes from low-level visual features of moth images in the training set, where moth wings are segmented into information-rich patches from which the local features are extracted, and the SRV attributes are provided by human experts as ground-truth. For the large amount of unlabeled test images in the database or added into the database later on, an automated identification process is evoked to translate the detected salient regions of low-level visual features on the moth wings into meaningful semantic SRV attributes. We further propose a novel network analysis based approach to explore and utilize the co-occurrence patterns of SRV attributes as contextual cues to improve individual attribute detection accuracy. Working with a small set of labeled training images, the approach constructs a network with nodes representing the SRV attributes and weighted edges denoting the co-occurrence correlation. A fast modularity maximization algorithm is proposed to detect the co-occurrence patterns as communities in the network. A random walk process working on the discovered co-occurrence patterns is applied to refine the individual attribute detection results. The effectiveness of the proposed approach is evaluated in automated moth identification and attribute-based image retrieval. In addition, a novel image descriptor called SRV attribute signature is introduced to record the visual and semantic properties of an image and is used to compare image similarity. Experiments are performed on an existing entomology database to illustrate the capabilities of our proposed system. We observed that the system performance is improved by the SRV attribute representation and their co-occurrence patterns.
A new relevance feedback technique for iconic image retrieval based on spatial relationships
2009, Journal of Systems and Software
Due to the popularity of Internet and the growing demand of image access, the volume of image databases is exploding. Hence, we need a more efficient and effective image searching technology. Relevance feedback technique has been popularly used with content-based image retrieval (CBIR) to improve the precision performance, however, it has never been used with the retrieval systems based on spatial relationships. Hence, we propose a new relevance feedback framework to deal with spatial relationships represented by a specific data structure, called the 2D B_e-string. The notions of relevance estimation and query reformulation are embodied in our method to exploit the relevance knowledge. The irrelevance information is collected in an irrelevant set to rule out undesired pictures and to expedite the convergence speed of relevance feedback. Our system not only handles picture-based relevance feedback, but also deals with region-based feedback mechanism, such that the efficacy and effectiveness of our retrieval system are both satisfactory.
A feature relevance estimation method for content-based image retrieval
2011, International Journal of Information Technology and Decision Making
Efficient approximate nearest neighbor search for multiple weighted l<inf>p≤</inf><inf>2</inf> distance functions
2020, arXiv
CBIR Using features derived by deep learning
2020, arXiv

View all citing articles on Scopus

^f1: {jp,bhanu,shan}@vislab.ucr.edu

¹: Corresponding author.

View full text

Regular ArticleProbabilistic Feature Relevance Learning for Content-Based Image Retrieval

Abstract

Pattern Recognit.

Instance-based learning algorithms

Mach. Learning

Locally weighted learning

AI Rev.

Trading mips and memory for knowledge engineering

Comm. ACM

Pattern Classification and Scene Analysis

Query by image and video content: The QBIC system

IEEE Comput.

Flexible Metric Nearest Neighbor Classification

Discriminant adaptive nearest neighbor classification

IEEE Trans. Pattern Anal. Mach. Intell.

Regular Article
Probabilistic Feature Relevance Learning for Content-Based Image Retrieval