Elsevier

Pattern Recognition

Volume 36, Issue 11, November 2003, Pages 2615-2633
Pattern Recognition

SASI: a generic texture descriptor for image retrieval

https://doi.org/10.1016/S0031-3203(03)00171-7Get rights and content

Abstract

In this paper, a generic texture descriptor, namely, Statistical Analysis of Structural Information (SASI) is introduced as a representation of texture. SASI is based on statistics of clique autocorrelation coefficients, calculated over structuring windows. SASI defines a set of clique windows to extract and measure various structural properties of texture by using a spatial multi-resolution method. Experimental results, performed on various image databases, indicate that SASI is more successful then the Gabor Filter descriptors in capturing small granularities and discontinuities such as sharp corners and abrupt changes. Due to the flexibility in designing the clique windows, SASI reaches higher average retrieval rates compared to Gabor Filter descriptors. However, the price of this performance is increased computational complexity.

Introduction

In recent years, textural information has been widely used as a visual primitive in many image processing applications [1], [2], [3], [4]. The potential areas include industrial and biomedical surface inspection, ground classification and segmentation of satellite or aerial imagery, document analysis, scene analysis, texture synthesis for computer graphics and animation, biometric person authentication, content-based image retrieval and model-based image coding [5], [6], [7].

Although the above application areas necessitate the utilization of texture analysis, only a limited number of successful interpretations of texture exist so far. The success of a texture descriptor heavily depends on the data type and the application area. A major problem in representing texture is that the textures in the real world are often quite complex due to changes in orientation, scale or other visual appearance such as brightness and contrast [8]. Additionally, it is difficult to include extremely large number of attributes of texture under a single mathematical representation.

Texture, generally, refers to repetition of basic texture elements called texels [9]. Mathematically speaking, texture can be defined as stochastic, possibly periodic, two-dimensional image field. In practice, texture descriptors represent distinctive characteristics of a texture, which are specific to the problem domain. Unfortunately, none of the existing descriptors has been shown to give satisfactory results over a wide range of textures.

Textures can be represented by statistical, spectral and/or structural descriptors [9]. Well-known statistical descriptors are co-occurrence matrix, histogram features and random fields [10], [11], [9]. Gabor, Fourier and wavelet filters are the examples of spectral descriptors [12], [4]. Structural descriptors make use of texture primitives, where syntactic rules are employed for generating the texture [9]. Statistical descriptors exploit the local correlation of image pixels, whereas spectral descriptors capture global information about the energy on different scales. While statistical descriptors successfully analyze textures with weak edges or random nature, spectral and structural descriptors are best suited for periodic or almost periodic textures. In a given problem domain various types of textures may be mixed.

Currently, Gabor Filters are the most popular descriptors, used for texture similarity problems. Among many others [12], [13] the successful results are reported by Manjunath and Ma [14], [15] where the image is first Gabor filtered, then, the second order statistics of the filter responses is used as a texture descriptor.

In [14], [15] Manjunath and Ma compare Gabor Filter features with other texture features, namely, pyramid-structured wavelet transform (PWT) features, tree-structured wavelet transform (TWT) features and multiresolution simultaneous autoregressive model (MR-SAR) features, on the image retrieval problem by using Brodatz Album. They found that Gabor features slightly improves the overall performance by achieving an average retrieval rate close to 74% whereas MR-SAR features remains at 73%.

Selection of the parameters for Gabor Filter descriptor depends on the characteristics of the textures in the image database. Since the Gabor functions are not orthogonal, there is a trade-off between redundancy and completeness in the design of the Gabor Filter Banks. Otherwise, the implementation of a complete Gabor expansion would entail a generally impractical number of filters. Also, in a digital world, it is not always possible to cope with all sizes of analog Gabor Filters, which may cause problems, especially, with the textures that consist of small texels or sharp corners. Another limitation of the Gabor descriptor is the restriction of the filtering area, which must fit in a rectangle, unless some pre-processing is done.

In this study, we explore a generic texture descriptor, which overcomes the above-mentioned difficulties and works well on a wide range of textures. The SASI descriptor, proposed in this paper, is based on second-order statistics of clique autocorrelation coefficients, which are the autocorrelation coefficients over a set of moving windows. The clique windows of various size and shape, which are defined by a neighborhood system, are used as a tool for describing the characteristics of textures in different granularity. The order of the neighborhood system controls the structure of the clique windows. Because of the flexibility in the definition of clique windows, SASI can cope with a broad class of textures, which may consist of discontinuities or small primitives.

SASI is tested on Brodatz Album [16], CUReT [17], PhoTex [18] and VisTex [19] image databases. The experiments are also performed on a combined database obtained by joining all of the images in these databases. It is observed that SASI improves the retrieval rates compared to Gabor Filters. However, the price of this improvement is the increased computational complexity.

The paper is organized as follows. In Section 2, we introduce SASI descriptor behind a series of definition. Experimental results are given in Section 3. Section 4 concludes with discussions on the strengths and weaknesses of SASI compared to Gabor Filter descriptors.

Section snippets

Definitions

SASI is based on the concepts of clique [20] and autocorrelation coefficient. In the following, SASI descriptor is introduced along with the background definitions.

Definition 1 Neighboring set of a pixel

For a regular lattice ℓ, the neighboring set of a pixel ij with coordinate (i,j) is defined by the following recurrence relation:∀kl∈ℓ,ij≠kl,ηijdijd−1argminkl∉ηijd−1D(ij,kl)andηij1=argminD(ij,kl),where ℓ={ij|i,j∈N,1⩽i⩽Widthand1⩽j⩽Height}, D(ij,kl) denotes the distance function between pixel ij and kl, d is the order of

Experiments

Two sets of experiments are done to show the power of SASI. First, SASI descriptor is analyzed in detail and compared to Gabor Filter descriptor. Latter, SASI and Gabor Filter descriptor are tested on the image retrieval problem by using four different image databases, namely Brodatz Album, CUReT [17], PhoTex [18] and VisTex [19]. The experiments are also performed on a database generated by joining all the images of these four databases.

Brodatz Album contains 112 pictures with size 512×512 and

Conclusions

In this paper, a new texture descriptor, namely SASI, is introduced and compared to Gabor Filters. SASI descriptor consists of second order statistics of autocorrelation coefficient at different lags over a set of clique windows. The concept of clique chain is employed for constructing these structural windows. Clique windows are defined by using a set of neighborhood systems. Changing the order of the neighborhood system, various regular or irregular clique windows are generated. The size of

About the AuthorABDURRAHMAN CARKACIOGLU received the BSc degree in Computer Science and Engineering from Hacettepe University in 1993, MSc degree in Computer Engineering from Middle East Technical University (METU) in 1997. He is currently a PhD candidate in computer engineering department at METU. He also works as a system analyst in Capital Markets Board of Turkey. His research interests include texture retrieval, texture representation, machine learning and analysis of stock market

References (27)

  • A.K. Jain

    Fundamentals of Digital Image Processing

    (1989)
  • T.R. Reed, J.M.H. du Buf, A review of recent texture segmentation and feature extraction techniques, CVGIP: Image...
  • G. Cross et al.

    Markov random field texture models

    IEEE Trans. Pattern Anal. Mach. Intell.

    (1983)
  • Cited by (39)

    • Nutrients deficiency diagnosis of rice crop by weighted average ensemble learning

      2023, Smart Agricultural Technology
      Citation Excerpt :

      Transfer learning is a popular method for developing machine learning models that does not require a large quantity of data [29]. The key advantages of transfer learning are that it reduces training time, improves neural network performance, and only requires a small amount of data [30–33]. After the emergence of several convolutional neural network (CNN) models, such as ResNet50, VGGNet, DensNet121, Xception, and Inception, to solve visual classification tasks in trained natural images created on natural image databases such as ImageNet, transfer learning has recently been applied to classifying images [34].

    • Towards better exploiting convolutional neural networks for remote sensing scene classification

      2017, Pattern Recognition
      Citation Excerpt :

      Thus, we tested a diverse set of 7 descriptors based on color, texture, and gradient properties, in order to extract visual features from each image and evaluate the potential of deep features when compared them. The global low-level descriptors considered are: Auto-Correlogram Color (ACC) [53], Border/Interior Pixel Classification (BIC) [54], Local Color Histogram (LCH) [55], Local Activity Spectrum (LAS) [56], Statistical Analysis of Structural Information (SASI) [57], Histogram of Oriented Gradients (HOG) [58], and GIST [59]. The implementations of ACC, BIC, LCH, SASI, and LAS descriptors follow the specifications of [50].

    • Visual words dictionaries and fusion techniques for searching people through textual and visual attributes

      2014, Pattern Recognition Letters
      Citation Excerpt :

      Finally, given a multiple-attribute query, we use some fusion methods to combine the outputs of the classifiers generating a reduced list of people complying with the describable attributes given by the user. With the visual dictionaries introduction, we achieve significant improvements on the results in comparison to the results obtained in the state of the art (Kumar et al., 2011), our own prior work (Fabian et al., 2012), and also a top-performer texture descriptor in the literature (Çarkacıogˇlu and Yarman-Vural, 2003; Penatti et al., 2012). Our work here can be regarded as a form of Content-based Image Retrieval (CBIR), in which the content is limited to face images and the queries are describable attributes or keywords related to the face.

    View all citing articles on Scopus

    About the AuthorABDURRAHMAN CARKACIOGLU received the BSc degree in Computer Science and Engineering from Hacettepe University in 1993, MSc degree in Computer Engineering from Middle East Technical University (METU) in 1997. He is currently a PhD candidate in computer engineering department at METU. He also works as a system analyst in Capital Markets Board of Turkey. His research interests include texture retrieval, texture representation, machine learning and analysis of stock market movements.

    About the AuthorFATOS T. YARMAN-VURAL received the BSc degree with honors in electrical engineering from the Technical University of ISTANBUL in 1973, the MSc degree in electrical engineering from Bogazici University in 1975 and the PhD degree in electrical engineering and computer science from Princeton University in 1981.

    From 1981 to 1983, she was research scientist in Marmara Research Institute in Turkey. From 1983 to 1985, she was a visiting professor at Drexel University, USA. From 1985 to 1992, she was a technical and deputy manager at Yapitel Inc. Since 1992, she has been a professor in Computer Engineering Department of Middle East Technical University (METU). Her research area covers computer vision, image processing, pattern recognition and artificial intelligence. She has been involved in teaching, consulting and organizing conferences in these areas.

    Dr. Yarman-Vural was the chair woman in the Computer Engineering Department between 1996-2000. Currently, she is assistant to the President at METU. She is a senior member of IEEE and member of Turkish Informatics Foundation, Turkish Information Association and chamber of Electrical Engineers in Turkey.

    View full text